The online network X has provoked major anger after individuals utilized its AI chatbot Grok to change images of women by removing their attire.
Numerous cases have been seen where the technology has been used to remove clothing from women to make them appear in swimwear non-consensually, as well as to insert them in inappropriate contexts.
Company Response and Legal Stance
XAI, the organization behind Grok, remained silent to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to prohibit nudification tools. Under a proposed criminal offence, anyone who provided such tools would "face a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' prompts when they mention it in a post.
It is often used to give reactions or further context to other posters' comments, but people on X are also capable of alter an uploaded photo through its AI image editing function.
History of Controversy and Policy
The tool has been faulted for allowing users to generate pictures and clips with explicit content and sexually suggestive material, and it was earlier accused of creating graphic material of celebrities.
"X has been allowing the production and sharing of these images for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "suitable measures" to "lessen the risk" of users coming across illegal content on their platforms and to remove it quickly when they become aware of it.
