California Attorney General Rob Bonta has launched an investigation into Elon Musk's xAI and issued a cease-and-desist letter ordering the company to stop creating non-consensual sexually explicit deepfake images via its Grok AI chatbot.
The investigation found that Grok users have been transforming ordinary photos of women and children into explicit deepfakes using the platform's "spicy mode" feature. Analysis of 20,000 images generated between Christmas and New Years revealed over half depicted people in minimal clothing, with some appearing to show minors.
The probe leverages California's new AB 621 law, which targets both creators and distributors of deepfake pornography, allowing victims to sue for damages up to $250,000. International regulators in Malaysia, Indonesia, UK, France, Australia, and the EU have also opened investigations, with Malaysia and Indonesia already suspending Grok.
Governor Gavin Newsom endorsed the investigation, calling xAI's platform "a breeding ground for predators." The AG gave xAI until January 20 to confirm it has halted the problematic content generation.
