Elon Musk's social media network X has sparked significant backlash after users used its AI chatbot Grok to change pictures of females by removing their garments.
Multiple instances have been seen where the technology has been employed to digitally undress women to render them in bikinis without permission, as well as to place them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, other than an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their networks, but did not confirm whether it was currently investigating X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they reference it in a message.
It is commonly used to give commentary or further background to other users' remarks, but people on X are also capable of edit an uploaded image through its AI image editing feature.
Background of Criticism and Rules
The tool has been criticised for enabling users to produce images and footage with explicit content and sexualised material, and it was previously accused of generating sexually explicit content of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any steps and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of people in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were required to take "suitable steps" to "reduce the risk" of users encountering illegal content on their platforms and to delete it swiftly when they are notified of it.
