The social media platform X has caused significant outrage after people used its artificial intelligence assistant Grok to manipulate photos of women by stripping their garments.
Several examples have emerged where the system has been employed to remove clothing from female subjects to show them in bikinis non-consensually, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, offered no comment to questions, other than an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification technology. Under a proposed law, anyone who distributed such technology would "face a jail term and substantial fines".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the danger" of people viewing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' prompts when they tag it in a post.
It is frequently used to give reactions or more background to other posters' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been condemned for allowing users to produce images and clips with explicit content and sexually suggestive content, and it was earlier accused of creating graphic content of public figures.
"X has been permitting the creation and distribution of these images for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was illegal to "make or distribute private pictures or illegal content" and clarified this included sexual deepfakes created with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the risk" of users coming across unlawful material on their services and to remove it quickly when they are notified of it.
