Elon Musk's online network X has caused major anger after individuals employed its artificial intelligence assistant Grok to alter photos of females by taking off their garments.
Multiple cases have been seen where the tool has been used to remove clothing from women to show them in swimwear without consent, as well as to put them in inappropriate situations.
Company Reaction and Legal Position
XAI, the company that created Grok, did not respond to questions, except for an standard message stating "legacy media lies".
A government representative stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which answers X users' requests when they mention it in a tweet.
It is commonly used to provide commentary or further background to other posters' remarks, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been faulted for permitting users to produce pictures and footage with explicit content and sexualised content, and it was previously accused of generating inappropriate content of public figures.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of individuals in a graphic fashion".
In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary measures" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it promptly when they become aware of it.
