The social media network X has provoked widespread anger after individuals used its AI chatbot Grok to alter photos of women by removing their attire.
Numerous instances have emerged where the system has been used to digitally undress women to show them in bikinis without consent, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the organization that created Grok, offered no comment to questions, other than an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to ban image stripping technology. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and significant fines".
"The platform is capable of halting these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some premium functions—which answers X users' queries when they reference it in a tweet.
It is frequently used to offer commentary or more background to other users' remarks, but people on X are also can edit an uploaded image through its AI image editing feature.
Background of Criticism and Rules
The system has been criticised for permitting users to generate photos and clips with nudity and sexualised content, and it was earlier accused of generating sexually explicit material of public figures.
"X has been enabling the generation and distribution of these pictures for months without taking any steps and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable actions" to "reduce the danger" of users encountering illegal material on their services and to take down it promptly when they learn of it.
