The online platform X has sparked widespread anger after individuals used its AI chatbot Grok to alter photos of women by taking off their attire.
Multiple instances have been seen where the tool has been used to digitally undress female subjects to render them in swimwear without permission, as well as to place them in sexually explicit contexts.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to ban image stripping technology. Under a upcoming law, anyone who supplied such tools would "receive a jail term and substantial penalties".
"The platform could prevent these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their services, but did not confirm whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they tag it in a tweet.
It is often used to give commentary or further context to other posters' comments, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Rules
The tool has been faulted for allowing users to generate photos and videos with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in generating inappropriate material of celebrities.
"The platform has been enabling the generation and spread of these pictures for a long time without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of individuals in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were required to take "necessary measures" to "minimize the risk" of users encountering unlawful content on their platforms and to remove it quickly when they are notified of it.
