A pattern repeats across every front of digital regulation: technology runs, the law chases. In few places is that distance as delicate as with AI “companion chatbots” — systems designed to sustain an emotional relationship with the user over time, to converse like a friend, a confidant, or a partner. Because here the gap between what the technology can do and what the law governs is not measured in market share or fines, but in its effect on the most vulnerable people who use it: often, minors.
It helps to define the object, because this is not just any chatbot. A companion chatbot, in the definition the new laws adopt, is an AI system that provides adaptive, human-like responses and can meet a user’s social or emotional needs — such that a reasonable person might believe they are interacting with a human. It is not an assistant that answers a question and ends; it is a system designed to make you come back, to simulate closeness, to occupy an emotional space. Platforms such as Character.AI or Replika let users build personas they talk to as if they were friends or partners, and their use has spread widely, including among teenagers.
The case that triggered the regulation
The following must be handled with care, as the public and judicial matter it is. Regulation of these systems in the United States accelerated after the death of Sewell Setzer III, a 14-year-old in Florida, in 2024, and the lawsuit his mother brought against Character.AI. According to the complaint, the teenager had formed an intense relationship with a chatbot on the platform, and the family alleges the system used an “addictive” design and failed to act when the minor expressed distress. The case was not unique: other families have sued, and in early 2026 Character.AI and Google reportedly settled several wrongful-death suits brought by families of teens, including Setzer’s. State attorneys general and lawmakers have documented further cases.
What matters for this coverage is not the detail of each tragedy, but the regulatory pattern they set off. As with cybersecurity or age verification, the law arrived after the harm, not before. And it arrived with force.
The US regulatory wave
In a matter of months, the United States went from having almost no specific rules to a proliferation of them. New York was the first state to require safeguards: since November 5, 2025, operators of “AI companions” available to its residents must implement protocols to detect and address expressions of self-harm or suicidal ideation, and to clearly disclose that the user is not talking to a human.
California followed with what is perhaps the most complete statute: SB 243, signed by Governor Newsom in October 2025 and effective January 1, 2026, after passing with broad bipartisan support. Its requirements are concrete and set an emerging standard: operators must notify users — especially minors — that responses are artificially generated and not human; for minors, remind them at least every three hours of continuous use to take a break; maintain protocols to prevent content about suicide and self-harm, redirecting users to crisis services; take reasonable steps to prevent the chatbot from producing sexual content to a minor; and, beginning in 2027, report to state public-health authorities how often crisis alerts are triggered. Crucially, California’s law authorizes a private right of action — letting harmed individuals sue — whereas New York’s allows only the Attorney General to seek penalties. Nationwide, dozens of bills have been filed across more than half the states, several aimed specifically at preventing an AI from presenting itself as a mental-health professional.
The EU AI Act and the global gap
In Europe, the approach differs but converges in substance. The AI Act does not regulate “companion chatbots” as a named category, but reaches them on two fronts. The transparency obligations of Article 50 require users to be told they are interacting with an AI. And a chatbot that crosses into the assessment or diagnosis of mental health can fall into the “high-risk” category, with far stricter requirements. The distinction, jurists note, is key: a tool that guides breathing exercises has lighter obligations than one that applies depression-screening questionnaires or claims to detect risk signals. The closer it comes to a clinical function, the stricter the regulation. In Spain, the designated authority already has sanctioning powers over prohibited practices, and official data show that more than a third of Spanish internet users regularly use some AI chatbot.
Much of the rest of the world, meanwhile, shows the gap this coverage keeps finding: incipient regulatory frameworks that leave a wide space without specific rules. While the US and Europe build safeguards, in many countries responsibility falls, in practice, on the very companies deploying these products — precisely the ones with a commercial incentive to maximize time-on-app.
Two readings, with comparable weight
The regulatory debate admits two legitimate positions, worth presenting without tilting the scale.
Those who push for regulation — affected families, lawmakers, child-protection groups — argue this is a real and documented risk: systems designed to generate emotional dependence, used by minors at moments of vulnerability, without mandatory safeguards until very recently. For them, requiring crisis protocols, disclosures that one is talking to a machine, and limits on content directed at minors is not an excessive burden but a minimal protection. The California bill’s sponsor framed it as common-sense protection for children and other vulnerable users against the addictive properties chatbots can have.
The technology industry and some jurists, for their part, warn about the risks of hasty regulation. Industry groups have called the definition of “companion chatbot” too broad, cautioning it could end up covering general-purpose AI systems never designed as emotional companions. They add a proportionality argument: compliance costs — certifications, protocols, documentation — may be manageable for large firms but prohibitive for small ones, consolidating the market in a few hands. And they note that many companies are already acting voluntarily: OpenAI rolled out parental controls and detection systems; Character.AI announced it would stop allowing minors free-form chat; and the major firms in the sector — including Anthropic, alongside Google, OpenAI, Meta, and others — convened at an academic workshop to debate common safety guidelines for companion chatbots.
It is not for this outlet to decree which approach is right. What can be stated is that both describe the same dilemma: how to protect vulnerable users without stifling a technology that also has legitimate uses, and without regulation, by defining its object poorly, reaching what it did not intend.
The companion-chatbot gap
What this case adds to the coverage is a particularly human variant of the gap between technology and regulation. Elsewhere that distance showed up as fines that never land or data left exposed; here it becomes something more intimate: systems able to simulate affection and sustain an emotional relationship reached the hands — and the private conversations — of millions of minors before any rule existed about how they should behave when a user expressed distress. The technology of simulated intimacy ran far ahead of any reflection on its limits.
The distinguishing feature is that regulation, for once, is arriving relatively fast — in months, not years — and fairly consistently across jurisdictions: artificiality disclosures, crisis protocols, reinforced protection for minors. It is as if society, after several fronts where the law came too late, had learned to react sooner. But it is still a response to tragedies already suffered, not an anticipation.
The verifiable fact is that AI companion chatbots reached mass use, including among minors, before any regulation; that after tragic cases reached the courts, California, New York, and dozens of states have legislated concrete safeguards; that Europe reaches them through the AI Act; and that much of the world still lacks specific rules. Whether these safeguards will be enough, or whether the technology will once again outrun the law, will depend on something not yet resolved: on whether the rules are genuinely enforced, on whether they define their object well without choking off legitimate innovation, and on whether the regions without a framework today decide whether to wait for their own tragedy or learn from someone else’s. As in every story of this kind, what is decisive is not the rule that gets announced, but whether it arrives in time to protect those it must protect.
This article addresses the regulation of artificial intelligence systems and mentions cases related to mental health in an informational and public-policy context. If you or someone you know is going through a difficult time, consider seeking support from a professional or a helpline in your country.