The social media network X has provoked major backlash after individuals utilized its artificial intelligence assistant Grok to change images of females by removing their clothing.
Multiple instances have surfaced where the technology has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit contexts.
Developer Reaction and Legal Stance
XAI, the company responsible for Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to ban nudification technology. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' queries when they mention it in a message.
It is frequently used to offer commentary or more background to other posters' comments, but people on X are also can modify an uploaded picture through its AI image editing function.
History of Controversy and Policy
The system has been faulted for allowing users to produce pictures and clips with nudity and sexually suggestive material, and it was in the past accused of generating sexually explicit material of celebrities.
"X has been enabling the creation and distribution of these images for months without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service prohibits "depicting likenesses of people in a graphic way".
In a comment, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said services such as X were required to take "necessary actions" to "minimize the danger" of users coming across unlawful content on their platforms and to remove it swiftly when they learn of it.
