The social media network X has caused major backlash after people used its AI chatbot Grok to manipulate pictures of females by taking off their clothing.
Multiple examples have been seen where the system has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization that created Grok, remained silent to questions, aside from an standard message stating "legacy media lies".
A government official stated that new laws is being brought forward to outlaw nudification software. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their services, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they tag it in a post.
It is commonly used to offer commentary or further background to other posters' remarks, but people on X are also capable of modify an posted picture through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to generate images and footage with nudity and sexually suggestive content, and it was in the past accused of generating graphic content of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of persons in a pornographic way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said networks such as X were mandated to take "suitable actions" to "reduce the risk" of users viewing illegal content on their services and to delete it swiftly when they become aware of it.
