The online network X has caused widespread anger after individuals used its AI chatbot Grok to change images of females by taking off their garments.
Several examples have emerged where the system has been used to virtually strip female subjects to render them in swimwear non-consensually, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the organization behind Grok, offered no comment to requests for comment, except for an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification software. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people seeing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' requests when they reference it in a post.
It is commonly used to give commentary or additional context to other users' remarks, but people on X are also able to alter an posted image through its AI image editing feature.
Background of Controversy and Policy
The system has been condemned for permitting users to generate pictures and footage with explicit content and sexually suggestive material, and it was earlier accused of generating sexually explicit content of public figures.
"The platform has been enabling the generation and spread of these images for months without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service forbids "showing likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the risk" of users viewing illegal content on their platforms and to delete it promptly when they are notified of it.
