The social media network X has sparked widespread backlash after people used its AI chatbot Grok to change photos of females by stripping their garments.
Several examples have emerged where the tool has been used to virtually strip female subjects to render them in swimwear without consent, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, offered no comment to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification technology. Under a proposed criminal offence, anyone who provided such technology would "face a prison sentence and heavy penalties".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their networks, but did not confirm whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they mention it in a message.
It is commonly used to provide commentary or additional background to other users' comments, but people on X are also can edit an uploaded image through its AI image editing function.
History of Controversy and Policy
The chatbot has been criticised for allowing users to create images and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in making sexually explicit content of public figures.
"X has been allowing the generation and distribution of these images for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the risk" of users viewing illegal material on their platforms and to remove it quickly when they become aware of it.
