Crackdown on AI-generated nudity: The EU’s fight for greater online safety

AFP, Agerpres, ANSA, BTA, dpa, EFE, FENA

Faced with strong reactions from the wider public against the use of artificial intelligence that violates people’s privacy and dignity, the European Union is bringing out the heavy artillery.

Brussels is considering classifying the creation of sexually explicit montages as a prohibited practice under the Artificial Intelligence Act, after the scandal involving sexualized images created by Grok, the chatbot integrated into the X platform of Elon Musk.

Reactions against Grok

Musk’s company, xAI – after being criticized by the international community for a long time – introduced new restrictions in mid-January on sexually suggestive images generated by artificial intelligence with Grok. This move followed criticism that Grok had allowed users to digitally replace women’s clothing with bikinis and, in some cases, to create sexualized images of minors.

The first images of people stripped naked without their consent (“artificial undressing”) began to circulate in the days after the option was launched, but spread particularly around New Year. According to CNN, between 5 and 6 January alone, Grok was used to generate at least 6,700 sexual images. They often featured women or minors.

“Grok now offers a ‘spicy option’ that displays explicit sexual content, and some results are generated with images that resemble children. This is not spicy. This is illegal. This is horrible,” said the EU spokesperson for digital affairs Thoma Renier at the time, speaking to reporters.

The European Commission, which acts as the bloc’s digital watchdog, said it would review the new measures taken by X. Officials warned that if the steps prove insufficient, the EU will consider full enforcement of its Digital Services Act.

The Vice-President of the EC, Henna Virkkunen, said that the EC is considering explicitly banning such sexually explicit images generated by artificial intelligence under the Artificial Intelligence Act, classifying them as unacceptable risks.

The ban on harmful practices in the field of artificial intelligence could be relevant for addressing the problem of child pornography and non-consensual sexual montages, said Virkkunen, who is also the European Commissioner for technological sovereignty, security and democracy, during the plenary session of the European Parliament in Strasbourg last week. She also said that the Digital Services Act reduces the risk of spreading sexual content online without consent.

She also recalled that the EC had sent a request to X to provide information regarding Grok, as part of its investigation of the platform under the Digital Services Act.

The platform was asked to preserve all internal documents and related data until the end of the year. “We are now examining the extent to which X may in any case be violating the Digital Services Act and we will not hesitate to take further measures if the evidence points to that,” she said.

The Commission had previously increased pressure on X, which was fined 120 million euros at the beginning of December for violating transparency rules. The EU insists it will enforce its rules despite having caused displeasure in the US administration.

“The Digital Services Act is very clear in Europe. All platforms must comply with the rules, because what is being generated here is unacceptable, and compliance with EU laws is not an option. It is an obligation,” said Renier as the scandal escalated at the beginning of January.

The week before last, a group of about 50 MEPs appealed to the EC to ban from the EU market all applications that use artificial intelligence to create nude images.

Life without X is impossible

Despite the criticism directed at X, almost all senior EU officials continue to post there instead of on alternative European platforms, according to research by dpa.

The President of the European Commission Ursula von der Leyen and other senior officials still do not have official profiles on Mastodon, an alternative platform based in Germany. Virkkunen opened an official profile on Mastodon in January. Some senior EU politicians are also active on Bluesky, another platform based in the United States that is becoming increasingly popular.

The Commission justifies the continued use of X by its reach: Mastodon has around 750,000 monthly users, compared with 100 million on X, according to the companies.

The long legal road to better online safety

The road to protecting minors in the EU is long, as issues of privacy and protection clash with business interests. Several regulations overlap:

Chat control

In 2022, the Commission proposed a regulation that would require platforms to detect and report images and videos of abuse (child sexual abuse material), as well as attempts by predators to contact minors.

Supported by several child protection groups, the plan called “Chat control” sparked fierce privacy debates in the 27-country bloc and led to accusations that it introduced mass surveillance.

The final law is expected to be agreed at the beginning of 2026, in order to bridge the gap between the European Parliament’s privacy-focused approach and the Council of the EU’s desire to broadly introduce powers for voluntary scanning.

Although they extended temporary measures for voluntary scanning until April 2026 to avoid a legal vacuum, MEPs called for a faster search for a permanent solution.

Digital Services Act

The European Union uses the Digital Services Act to sanction online platforms by imposing huge fines, demanding urgent operational changes and – in extreme cases – temporarily suspending their services. It can impose penalties if platforms do not fulfill their duties under the Digital Services Act, do not comply with temporary measures or violate obligations.

It is an EU regulation for a safer internet, which requires platforms to tackle illegal content, protect users and increase transparency.

Artificial Intelligence Act

The Artificial Intelligence Act was adopted in 2024 and is the world’s first and only comprehensive legal framework in the context of artificial intelligence. It establishes a risk-based system for regulating artificial intelligence technologies within the EU, to ensure that they are safe, trustworthy and respect fundamental rights, while also being innovative.

It bans certain unacceptable AI practices, such as social scoring, and sets rules for areas where there is a high risk if artificial intelligence is used – such as critical infrastructure or employment. It also sets limits on manipulative uses of artificial intelligence, such as montages targeting children, among other things.

Social media bans

France, which is considering banning social media for children under 15, has been testing since the summer an age-verification app developed by the EC. This tool is one of several methods for verifying the age of internet users, which is a problem for both tech giants and authorities.

Individual efforts

Spanish Minister for Youth and Children Sira Rego asked the public prosecutor at the beginning of January to investigate whether Grok is committing criminal offences related to the dissemination of child sexual abuse material.

At the moment, Spain is working on its own law to protect minors in digital environments. The law strengthens the framework for protecting personal integrity and privacy from new forms of violation related to the use of technologies such as artificial intelligence, confirming that the best interests of the child must always be more important than any digital business model.

Bulgaria has stepped up efforts to combat online child sexual abuse through cooperation in enforcing international law, national prevention campaigns and discussions on rules aligned with EU legislation. In 2025, Bulgarian authorities took part in a major international operation that shut down Kidflix, one of the world’s largest platforms for child sexual exploitation, which had been used by almost 2 million users between 2022 and 2025.

In its criminal code, Romania has established legal mechanisms to combat child sexual abuse material, and the authorities intend to expand and modernize these rules.

Since 2025, an important law on the protection of children online (called the Internet Age of Majority Act) has been in parliamentary procedure, and Romania

is gradually implementing EU rules for preventing and combating online sexual abuse. The Internet Age of Majority Act introduces mandatory age verification and parental consent for minors (under 16) to access online services such as social networks and gaming and live-streaming platforms.

Bosnia and Herzegovina, an EU candidate country, still does not have a specific law regulating this area. In BiH, criminal liability for the production, distribution and possession of such material is based on criminal laws that cover child sexual abuse, but do not contain explicit provisions for content generated or simulated by artificial intelligence.

The EU has introduced additional tools and measures to protect its citizens – both young and old – from harmful online practices, but weak points remain problematic enforcement, algorithmic promotion of harmful content, inconsistent application across countries and disagreements over balancing safety with privacy.

The story is published weekly. The content is based on news from the agencies participating in ENR.