California is investigating Grok over AI-generated CSAM and nonconsensual deepfakes

California authorities have launched an investigation into xAI following weeks of reports that the chatbot was generating sexualized images of children. "xAI appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California Attorney General Rob Bonta's office said in a statement. 

The statement cited a report that "more than half of the 20,000 images generated by xAI between Christmas and New Years depicted people in minimal clothing," including some that appeared to be children. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material,” Bonta said. “Today, my office formally announces an investigation into xAI to determine whether and how xAI violated the law.

The investigation was announced as California Governor Gavin Newsom also called on Bonta to investigate xAI. "xAI’s decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile," Newsom wrote.

California authorities aren't the first to investigate the company following widespread reports of AI-generated child sexual abuse material (CSAM) and non-consensual intimate images of women. UK regulator Ofcom has also opened an official inquiry, and European Union officials have said they are also looking into  the issue. Malaysia and Indonesia have moved to block Grok. 

Last week, xAI began imposing rate limits on Grok's image generation abilities, but has so far declined to pull the plug entirely. When asked to comment on the California investigation, xAI responded with an automated email that said "Legacy Media Lies." 

Earlier on Wednesday, Elon Musk said he was "not aware of any naked underage images generated by Grok." Notably, that statement does not directly refute Bonta's allegation that Grok is being used "to alter images of children to depict them in minimal clothing and sexual situations." Musk said that "the operating principle for Grok is to obey the laws" and that the company works to address cases of "adversarial hacking of Grok prompts."

This article originally appeared on Engadget at https://www.engadget.com/ai/california-is-investigating-grok-over-ai-generated-csam-and-nonconsensual-deepfakes-202029635.html?src=rss