The social media platform X has caused major anger after individuals used its AI chatbot Grok to change pictures of females by stripping their garments.
Numerous instances have been seen where the system has been employed to digitally undress female subjects to show them in bikinis non-consensually, as well as to put them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the company that created Grok, did not respond to questions, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to outlaw nudification software. Under a new law, anyone who distributed such tools would "receive a jail term and substantial fines".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people seeing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a message.
It is commonly used to give reactions or additional background to other posters' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to create pictures and clips with explicit content and sexualised content, and it was earlier accused of generating graphic material of celebrities.
"The platform has been enabling the production and spread of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "portraying likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable actions" to "reduce the risk" of users viewing unlawful material on their platforms and to take down it swiftly when they become aware of it.
