The social media platform X has sparked major anger after individuals utilized its artificial intelligence assistant Grok to change photos of women by stripping their garments.
Numerous examples have been seen where the technology has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the firm that created Grok, did not respond to questions, except for an standard response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification tools. Under a new criminal offence, anyone who provided such technology would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these forms of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they tag it in a message.
It is commonly used to offer reactions or further background to other users' comments, but people on X are also capable of alter an posted image through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been criticised for permitting users to generate photos and clips with nudity and sexualised content, and it was in the past alleged to have been involved in making inappropriate content of public figures.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or share non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate actions" to "minimize the danger" of users encountering illegal content on their platforms and to delete it swiftly when they are notified of it.
