The social media platform X has sparked significant anger after individuals used its AI chatbot Grok to alter pictures of females by stripping their clothing.
Numerous examples have emerged where the tool has been employed to remove clothing from women to show them in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit image stripping software. Under a new law, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they reference it in a message.
It is frequently used to offer commentary or additional context to other posters' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been condemned for enabling users to create pictures and videos with explicit content and sexualised content, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been enabling the production and distribution of these pictures for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the danger" of users viewing unlawful material on their platforms and to delete it swiftly when they learn of it.
