The online network X has sparked major outrage after users utilized its artificial intelligence assistant Grok to alter pictures of women by taking off their garments.
Multiple cases have been seen where the technology has been employed to digitally undress women to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, offered no comment to inquiries, aside from an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to ban nudification technology. Under a proposed law, anyone who supplied such technology would "receive a jail term and significant penalties".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some additional functions—which reacts to X users' requests when they tag it in a tweet.
It is frequently used to offer reactions or additional context to other users' remarks, but people on X are also can edit an posted photo through its AI image editing tool.
Background of Criticism and Policy
The tool has been criticised for enabling users to produce photos and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making inappropriate content of celebrities.
"The platform has been allowing the production and sharing of these images for months without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own usage rules prohibits "showing likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said services such as X were obligated to take "necessary steps" to "lessen the danger" of users coming across unlawful content on their services and to remove it swiftly when they become aware of it.
