The social media network X has sparked widespread anger after people utilized its AI chatbot Grok to manipulate images of women by stripping their attire.
Several cases have emerged where the technology has been employed to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, remained silent to questions, except for an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a proposed law, anyone who distributed such technology would "face a jail term and heavy penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing illegal content on their services, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they reference it in a post.
It is commonly used to give reactions or further context to other users' comments, but people on X are also able to modify an uploaded picture through its AI image editing function.
History of Controversy and Policy
The tool has been faulted for permitting users to produce photos and videos with nudity and sexually suggestive content, and it was earlier accused of generating sexually explicit material of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules bans "showing likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said services such as X were required to take "appropriate steps" to "lessen the risk" of users viewing unlawful material on their services and to delete it quickly when they learn of it.
