The online platform X has caused major outrage after individuals utilized its AI chatbot Grok to alter images of women by removing their garments.
Numerous instances have surfaced where the tool has been used to remove clothing from women to render them in bikinis without permission, as well as to place them in sexualized scenarios.
Company Response and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to outlaw nudification technology. Under a proposed law, anyone who distributed such tools would "receive a prison sentence and heavy fines".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' prompts when they reference it in a message.
It is often used to give commentary or further context to other posters' remarks, but people on X are also can edit an uploaded picture through its AI image editing feature.
History of Controversy and Rules
The system has been criticised for enabling users to create pictures and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making inappropriate material of public figures.
"The platform has been allowing the generation and spread of these images for months without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "necessary measures" to "minimize the danger" of users coming across illegal content on their platforms and to take down it quickly when they become aware of it.
