The social media network X has provoked major backlash after people utilized its artificial intelligence assistant Grok to manipulate images of women by removing their garments.
Numerous examples have been seen where the system has been employed to digitally undress female subjects to show them in bikinis without consent, as well as to place them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the company responsible for Grok, did not respond to questions, aside from an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to ban image stripping technology. Under a proposed criminal offence, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people accessing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which responds to X users' requests when they tag it in a tweet.
It is frequently used to provide commentary or further background to other posters' remarks, but people on X are also can modify an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The chatbot has been condemned for enabling users to create photos and videos with nudity and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable actions" to "minimize the risk" of users viewing illegal content on their platforms and to remove it swiftly when they become aware of it.
