Artificial intelligence companies and lawmakers are focusing on preventing immediate harm from chatbots, such as suicide risks or mental health crises. New features include break reminders, usage limits, and crisis detection. A dozen states now require chatbot operators to disclose their tools are not human and to offer crisis referrals.
A chatbot might seem safe in each conversation but, over months, could displace human relationships. Such cumulative harm is harder to identify than a single crisis event.
Current AI Safety Measures Focus on Single Sessions
Major AI companies have built safeguards to work within individual chat sessions. Break reminders and usage limits operate per message or conversation. Crisis detection also functioned this way until OpenAI announced it would now track user distress across multiple sessions.
This shift shows companies can track user context over time. However, tracking distress still targets acute problems. A person gradually pulling away from friends and family may not show signs of distress in any single chat. Researchers lack a way to measure this kind of displacement.
AI chatbots are increasingly used for companionship. Early studies suggest heavy users may have less social interaction and feel more lonely. This evidence is preliminary, but it points to a need for more data.
Historical Precedents Show the Danger of Ignoring Cumulative Harm
Other industries have used product-level safety features to defend against claims of long-term harm. The tobacco industry used warning labels to argue that individuals assumed the risk of smoking, which protected companies from liability for decades. Similarly, Meta has pointed to parental controls in discussions about social media’s impact on teen mental health.
Accountability for cumulative harm often arrives only after public systems bear the cost. For tobacco, states could prove industry costs through Medicaid expenses. No such measurement infrastructure exists for AI’s potential effect on social connection.
AI companies hold usage data closely. Public agencies, like health and education departments, know the cost of social isolation. There is little connection between these two datasets. Frontier labs analyze aggregate data in privacy-preserving ways, but this has not yet been applied to cumulative harm.
Three groups could change this. Researchers in academia and civil society should develop measurement approaches to distinguish AI chatbot use from other contributors to social isolation. Public health, education, and other agencies can measure and share what social isolation already costs them, in areas such as school counselor time, primary care utilization, or demand on social prescription programs. Finally, state legislatures can encourage greater access for independent third parties to conduct evaluations related to social isolation.
Establishing a Baseline for AI’s Social Impact
Researchers must develop tools to measure social isolation. They need ways to distinguish chatbot use from other causes. They can share data on counselor time or care utilization. This would show the scale of existing problems.
The Need for Measurement Before Harm Becomes Undeniable
By Qian Yang, Ned Cooper, and Ibrahim Emara. Yang is an assistant professor in the Bowers College of Computing and Information Science at Cornell University, where she directs the DesignAI Group. Cooper is a postdoctoral researcher in the DesignAI Lab at Cornell. Emara is a graduate student at the Cornell Brooks School of Public Policy.
