Guest post by Dr. Cody Rei-Anderson, Lecturer in Law and Technology at Edinburgh Napier University
The recent scandal over the chatbot application Grok and its use to generate nonconsensual intimate imagery and child sexual abuse material (“deepfakes”), previously covered in this blog in relation to the EU DSA, has led the UK government to announce that it will, according to the Financial Times, “seek powers to ‘move fast’ to close a legal loophole so that AI chatbots — such as Grok, Google’s Gemini and OpenAI’s ChatGPT — would be brought clearly within the scope of the Online Safety Act” [1].
The verb “to grok” means to deeply understand something, and originates with Robert Heinlein’s 1961 science fiction novel Stranger in a Strange Land. And it certainly takes more than a superficial understanding of the UK’s “landmark” online safety legislation to make sense of why OFCOM, the regulatory body in charge of enforcing the Act, would decide in early February against investigating xAI, the company providing the standalone Grok application, while continuing their investigation into the social media platform X.
The key point, as OFCOM’s announcement emphasises, is that not all chatbots are regulated under the Online Safety Act (OSA). The OSA places duties about content which is illegal or harmful to children upon search and user-to-user services. The definitions of these types of services would not include the core functionality of LLM chatbots. Yet to call this a “loophole” risks understating the problem: in fact, the OSA is fundamentally structured around requiring platforms to limit the dissemination of content which is illegal or harmful, especially to children. Insofar as the issue with AI chatbots is that users can generate illegal content like deepfakes, or that users can be harmed directly by interactions with chatbots, the OSA may be of little help even if it is expanded to cover chatbot providers as regulated services. To make the OSA effective at regulating chatbot providers would require a deeper rethink.
Dust jacket of the first edition of Stranger in a Strange Land by Robert A. Heinlein (Source: Wikimedia Commons)
Chatbot providers and the OSA
When we talk about a “chatbot”, what we mean is an interface through which users interact with a conversational computer program through text or other media input. Today, the backend of such applications is generally a large language model (LLM). The chatbot is, to date, the main consumer application of LLMs and likely the most common way in which people encounter “artificial intelligence”. Within the implementation of the chatbot, service providers have various ways to tweak and control the access which users have to the LLM through hidden settings and the “system prompt” (default instructions to the model) as well as usage limits and automatic moderation of inputs and responses.
The OSA was passed on 26 October 2023, nearly a year after the release of ChatGPT. Yet the Act reflects primarily pre-AI concerns, putting user-to-user services and search services under duties relating to illegal content and the protection of children. Web services which allow pornographic content were also made subject to certain requirements. These categories do not easily apply to the core functionality of chatbots.
As OFCOM recently clarified, chatbot providers will not fall to be regulated under the OSA where they:
- only allow people to interact with the chatbot and not other users;
- are not able to search across multiple websites; and
- cannot generate pornographic content [3].
Undoubtedly, many existing chatbots do fall within the OSA’s regulatory scope. Chatbots may be offered as part of a social media network or other user-to-user service. Grok is one example; chatbots provided by Meta through their Messenger app are another. Many chatbots are able to search the web: mainstream chatbot services like ChatGPT, Claude, and Perplexity have all had this functionality for some time. Yet as I will elaborate on below, for both user-to-user and search-enabled chatbot services, OSA duties as they are currently constructed will not extend to material which is generated by the LLM.
The OSA’s pornographic content duties certainly apply to Grok, which explicitly allows generation of pornographic content subject to its terms of service. However, the duties placed on services which allow pornographic content (and which are not user-to-user or search services) are essentially limited to a duty to implement “highly effective age verification”. xAI continues to face scrutiny from OFCOM over whether its age verification meets this standard, but it is clear enough from the government’s response that these duties are not seen as adequate to regulate the risks posed by chatbots.
Chatbot content and the OSA
Other chatbot providers appear to have largely avoided the specific issues that arose with Grok by employing “guardrails” which constrain what they will generate. But they have hardly been free from other concerns. And indeed, the Grok controversy was not the first time that OFCOM turned its attention to chatbot providers. In an open letter published in November 2024, following tragic reports of suicides related to chatbots, OFCOM emphasised that chatbot providers with certain types of functionality would fall within their regulatory purview.
The tone of the 2024 letter is notably more expansive than the 2026 announcement addressing the OSA’s limits; it states that “[w]here a site or app allows users to upload or create their own Generative AI chatbots – ‘user chatbots’ – which are also made available to other users, it is also a user-to-user service” and therefore regulated by the OSA.[3] This is undoubtedly an allusion to sites like Character.ai, which allow users to produce custom chatbots through prompts and images which then generate responses through the site’s LLM.
The 2024 letter goes further to suggest that “[a]ny text, images or videos created by these ‘user chatbots’ is ‘user-generated content’ and is regulated by the Act”. However, this interpretation is questionable: “user-generated content” for the purposes of the illegal content duties and many other sections of the OSA must be content “that may be encountered by another user, or other users, of the service by means of the service” [4]. Content which stays within the context of a user-to-bot chat would likely not qualify for regulation at all.
Similarly, for search services the “search content” covered by the illegal content duties means content which may be encountered in search results [5]. At a minimum this means that the regulated content in the context of a search-enabled chatbot is limited to that which it finds through search. However, it is not even clear that should the chatbot be doing the searching on behalf of the user that the results would qualify: “search results” mean content presented “in response to a search request made by the user” [6].
However, even should both chatbot providers and their content fall into the scope of the OSA, the Act’s illegal content duties are not phrased in a way which would make them easy to apply to chatbot providers. The illegal content and children’s risk assessment duties for user-to-user services require that they take “into account (in particular) algorithms used by the service and how easily, quickly and widely content may be disseminated by means of the service” [7]. The equivalent duties for search services use similar language, also requiring that the service take into account “risks presented by algorithms used by the service” [8].
Whether “algorithms” in this context could include the LLMs used on the backend of chatbot providers is unclear; “algorithm” appears 22 times in the Act but receives no definition. It might support a broader reading, but the language referring to content being “disseminated” seems to suggest that “algorithm” here should be read in the colloquial sense referring to the methods by which social platforms determine what content to show users.
Can the OSA provide an effective framework for regulating chatbots?
Even were chatbot content to be covered by OSA, what would effective provider compliance with the illegal content duties look like? Answering this question is made more difficult by the fact that OFCOM’s enforcement of the OSA to date has largely focused on low-hanging fruit: pornography sites with insufficiently stringent age verification, suicide forums, and scofflaw operations such as 4chan [9].
With the Online Safety Act still only a few years old, it is not entirely clear what effective or sufficient compliance to the Act looks like in practice even for clearly in-scope social media networks like Facebook or X. Even before the recent controversy over Grok, X has been observed to take a notably light-touch approach to moderation since its change of ownership. (It would be surprising if this were not a factor in the Grok fiasco.) Yet until January 2026, X had avoided investigation (at least in the UK).
Chatbot applications in particular pose unique issues for OSA enforcement. Their responses are unpredictable and may not be amenable to the same methods of risk assessment as social media content. The effects of chatbots may be harmful even where they do not generate content which is illegal or harmful in itself.
At the same time, expanding the OSA to cover chatbot providers carries a risk of overenforcement which could hamper open source development or compromise user privacy. Could a platform which provides access to downloads of model files like Hugging Face, for example, effectively risk assess models which it hosts? This recalls the content moderation challenges which OSA presents to platforms, but at a much greater scale should each model need to be interrogated individually for possible illegality. Furthermore, for traditional chatbot providers, an interpretation of illegal content duties requiring monitoring of individual user conversations could compromise user privacy [10].
Conclusion
Working one’s way through the text of the OSA, one is left with the distinct sense that the mischief for which the Act was designed differs in important respects from that presented by LLM chatbots. While the failures of Grok’s content moderation certainly merit further regulatory scrutiny, other routes aside from the OSA already exist, such as through privacy law. Around the same time as the February OFCOM announcement, it was reported that the Information Commissioner’s Office would investigate X and xAI for possible violations of UK GDPR over the deepfakes. OFCOM’s announcement also mentions “working closely” with the ICO. Given that this alternative exists, one hopes that the government is not overhasty in closing the chatbot “loophole” in the OSA by simply extending the Act to new types of functionality and content for which its existing categories and duties are not well-suited.
Footnotes
[1] Mari Novik, “UK to tighten online safety laws to include AI chatbots” Financial Times (15 February 2026) (online).
[2] “Ofcom update: Investigation into X, and scope of the Online Safety Act” (OFCOM, 3 February 2026) (online).
[3] “Open letter to UK online service providers regarding Generative AI and chatbots” (OFCOM, 8 November 2024) (online).
[4] Online Safety Act 2023, s 55(3)(b) (OSA).
[5] OSA s 57(2).
[6] OSA s 57(3).
[7] OSA ss 9(5), 11(6).
[8] OSA ss 26(5)(a), 28(5)(a).
[9] An investigation of the image board 4chan is ongoing, and as of the time of writing it has been served with a provisional notice of contravention: “Investigation into 4chan and its compliance with duties to protect its users from illegal content” (OFCOM, 12 February 2026) (online). 4chan for its part rejects OFCOM’s jurisdiction over US-based internet services that it is their operations within the United Kingdom which are at issue.
[10] The extent to which chatbot providers already monitor user interactions is unclear, and is likely to vary between services. Notably, ChatGPT provider OpenAI has recently been summoned to appear before Canada’s AI minister following reports that the company declined to notify Canadian authorities after suspending the account of an individual who later perpetrated a school shooting. The account was suspended for “furtherance of violent activities” and was flagged by OpenAI’s automated content monitoring: Leyland Cecco, “Canada seeks answers from OpenAI for failing to alert police after suspending school shooter’s account” The Guardian(23 February 2026) (online).