This undated file photo shows Liang Yinxia helping a senior citizen practicing calligraphy at Nanning Wuxiang Nursing Service Center in Nanning, South China's Guangxi Zhuang autonomous region. [Photo/Xinhua]

Introducing an artificial intelligence-powered companion in an elderly person's home changes the dynamics of testing. While the system may answer questions promptly and engage in conversations, the real issues are: Does it help the person feel less isolated? Does it support independent living? And what happens when the initial novelty wears off? These are questions that a laboratory test, however rigorous, alone cannot answer.

As AI moves into homes, schools, hospitals and workplaces, such questions are becoming hard to ignore. The technology may perform exactly as intended, yet produce effects its developers never anticipated. For instance, a tutoring system might alter how students approach learning, an AI assistant can change the way colleagues divide their work and a companion agent may affect how an elderly person interacts with family members. The consequences often emerge gradually through everyday use.

This presents a familiar problem in technology governance. Known as the Collingridge dilemma, it highlights the challenge of predicting a technology's social consequences before widespread adoption.

Once those consequences become visible, changing course can be expensive, disruptive or politically difficult. Generative AI adds another layer of uncertainty because it responds to users and adapts to their behavior, making its social effects difficult to predict from technical specifications alone.

Waiting for problems to become obvious is therefore a poor strategy. Yet holding back innovation until every possible risk is understood is neither realistic nor desirable.

A more practical approach is to test emerging technologies in real-world settings before their use becomes too widespread to change easily. This is the essential purpose of AI social experimentation.

This principle has roots in both research and public policy.

The Hawthorne studies in the United States during the 1920s examined how working conditions influenced workers' productivity and behavior.

In the 1930s, Liang Shuming's rural reconstruction experiments in China's Shandong province explored how education, local governance and economic cooperation might improve rural life.

Decades later, field experiments became an important tool for evaluating interventions in education, healthcare and poverty alleviation. Abhijit Banerjee, Esther Duflo and Michael Kremer received the 2019 Nobel Prize in economics for their experimental approach to alleviating global poverty.

AI social experiments extend this practical tradition to a technology whose effects are still unfolding. Researchers introduce a system into a carefully selected setting, establish a baseline, track outcomes and compare results over time. The findings can help developers refine their products and give policymakers a stronger basis for decisions.

The process involves more than observing whether people use a product or express satisfaction. Researchers must define what they aim to measure, consider alternative explanations and follow participants long enough to distinguish lasting changes from temporary enthusiasm. Informed consent and privacy protections are essential, particularly when experiments involve elderly people or other potentially vulnerable groups.

China has incorporated this approach into its AI governance agenda. Research initiatives have evolved into pilot programs, supported by national intelligent-society governance experimental bases and AI innovation pilot zones.

Government guidance issued in 2022 called for further AI social experiments and the development of standards for intelligent-society governance. The State Council's 2025 AI+ initiative again emphasized these experiments as vital for integrating AI into economic and social life.

Eldercare offers a particularly useful test. In 2025, researchers affiliated with Tsinghua University and industry partners launched a three-year project to study how AI agents could serve people in their daily lives. The project plans to involve around 300 households, assessing the technology's effects on older people's quality of life, emotional well-being, family interactions and social participation.

Initial findings have highlighted practical considerations: putting an intelligent device into a home is just the beginning. Some elderly users need help with setting up equipment and learning how to operate it. Community workers and volunteers can facilitate adoption, while regular visits help researchers understand how the technology fits into daily routines.

Surveys have also pointed to loneliness and limited digital access as concerns among some elderly people in rural areas.

These observations help identify where support is needed, but do not yet establish that AI companionship improves well-being. That requires continued observation and appropriate comparisons. An elderly person's willingness to talk to a machine, for example, does not necessarily mean the person is feeling less lonely. Nor does frequent use prove that a device has improved the quality of daily life.

This is precisely why social experimentation matters. It can reveal a technology's actual impact in real-world settings, including effects that neither developers nor users anticipated.

It can also show which groups benefit, which require additional support and where an application falls short.

Evidence from these trials should inform standards to extend benefits beyond individual projects. In 2023, China established a national standards working group on the social application and evaluation of intelligent technologies. Since then, five national standards have been issued, covering AI social experiments and the assessment and compliance management of generative AI's social impact. These standards can provide common methods for evaluating applications and help translate lessons from pilot programs into consistent practice.

At the World Artificial Intelligence Conference and the High-Level Meeting on Global AI Governance in July 2026, President Xi Jinping raised questions about human-machine relations, security, ethics and the equitable sharing of AI's benefits. Addressing these questions requires more than technical progress; it demands a deeper understanding of how the technology transforms people's lives.

Standards and experiments serve different purposes. Experiments reveal what happens when AI enters the real world; standards help ensure that the lessons are applied more consistently.

Neither can eliminate uncertainty, but together they can make it easier to identify problems while there is still time to address them.

When an AI agent enters an elderly person's home, its success should not be judged by a product demonstration alone.

Researchers need evidence to determine whether it genuinely enhances the person's life. The same principle should guide AI adoption elsewhere. Before a technology becomes part of everyday life, society must understand its actual impact.

The author is a professor at the School of Public Policy and Management, Tsinghua University.

The views don't necessarily reflect those of China Daily.