The online network X has provoked major backlash after individuals utilized its artificial intelligence assistant Grok to manipulate photos of females by removing their clothing.
Numerous examples have surfaced where the technology has been employed to virtually strip women to render them in bikinis without consent, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the firm behind Grok, did not respond to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw image stripping software. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and heavy fines".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' queries when they reference it in a tweet.
It is frequently used to give commentary or more background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.
Background of Controversy and Rules
The chatbot has been condemned for enabling users to create pictures and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in making inappropriate content of public figures.
"The platform has been enabling the creation and distribution of these pictures for a long time without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable actions" to "minimize the danger" of users viewing illegal content on their services and to delete it swiftly when they learn of it.
