🔗 Share this article

The social media network X has provoked significant outrage after individuals employed its AI chatbot Grok to alter images of women by stripping their clothing.

Multiple instances have surfaced where the tool has been used to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in sexualized scenarios.

Company Response and Legal Position

XAI, the company that created Grok, did not respond to questions, aside from an automatically-generated message stating "traditional news outlets lies".

A Home Office representative stated that new laws is being introduced to ban nudification software. Under a upcoming law, anyone who distributed such tools would "receive a jail term and significant penalties".

"The platform is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."

Oversight Body Stance and Platform Capabilities

The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing illegal content on their services, but did not specify whether it was presently looking into X or Grok in relation to AI-generated images.

Grok is a no-cost chatbot—with some premium features—which answers X users' queries when they tag it in a message.

It is often used to provide reactions or more context to other posters' comments, but people on X are also capable of modify an posted image through its AI image editing tool.

Background of Controversy and Rules

The tool has been condemned for allowing users to generate pictures and videos with nudity and sexually suggestive content, and it was previously accused of creating sexually explicit material of celebrities.

"The platform has been enabling the creation and distribution of these images for a long time without taking any steps and we have yet to see any action by regulators," the expert added.

XAI's own terms of service forbids "depicting likenesses of people in a graphic fashion".

In a statement, Ofcom said it was against the law to "make or distribute non-consensual intimate images or illegal content" and clarified this included sexual deepfakes created with AI.

It said networks such as X were mandated to take "suitable actions" to "minimize the danger" of users viewing unlawful material on their services and to take down it swiftly when they are notified of it.