Elon Musk’s AI model Grok will no longer be able to edit photos of real people to show them in revealing clothing, after widespread concern over sexualised AI deepfakes in countries including the UK and US.

“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis.

“This restriction applies to all users, including paid subscribers,” reads an announcement on X, which operates the Grok AI tool.

The change was announced hours after California’s top prosecutor said the state was probing the spread of sexualised AI deepfakes, including of children, generated by the AI model.

The update expands measures that stop all users, including paid subscribers, editing images of real people in revealing outfits.

X, formerly known as Twitter, also reiterated in a statement on Wednesday that only paid users will be able to edit images using Grok on its platform.

This will add an extra layer of protection by helping to ensure that those who try and abuse Grok to violate the law or X’s policies are held accountable, it said.

Users who try to generate images of real people in bikinis, underwear and similar clothing using Grok will be stopped from doing so according to the laws of their jurisdiction, X’s statement said.

In a statement on Wednesday, California Attorney General Rob Bonta said: “This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet.”

Malaysia and Indonesia have blocked access to the chatbot over the images and UK Prime Minister Sir Keir Starmer warned X could lose the “right to self regulate” amid outrage over the AI images.

Britain’s media regulator, Ofcom, said on Monday that it would investigate whether X had failed to comply with UK law over the sexual images.

California’s top prosecutor has launched an investigation into the spread of sexualised AI deepfakes generated by Elon Musk’s AI model Grok.

Attorney General Rob Bonta said in a statement announcing the probe: “The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking.”

xAI, which develops Grok, has previously said “anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content”.

California’s inquiry comes as British Prime Minister Sir Keir Starmer warns of possible action against X.

In Wednesday’s statement, Bonta said: “This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet.”

The Democratic prosecutor urged xAI to take immediate action.

California Governor Gavin Newsom, a Democrat, posted to X on Wednesday that xAI’s decision to “create and host a breeding ground for predators… is vile”.

The BBC has contacted xAI for comment.

On Wednesday, Musk posted to X that he is “not aware of any naked underage images generated by Grok. Literally zero.”

“Obviously, Grok does not spontaneously generate images,” Musk wrote. “It does so only according to user requests.”

The tech billionaire, a Republican mega-donor, has also said that critics of X were politically motivated and using the Grok controversy as an “excuse for censorship”.

In November, Wired magazine reported that tools from other AI companies like OpenAI and Google have also been used to digitally undress people.

Last week, three US Democratic senators asked Apple and Google to remove X and Grok from their app stores.

Within hours of the request, X restricted its image generation tool so that it would only be available to paying subscribers.

X and Grok remain available on Apple’s App Store and Google Play.

It comes amid a debate over whether US tech companies are shielded from responsibility for what users post on AI platforms.

Section 230 of the Communications Decency Act of 1996 provides legal immunity to online platforms for user-generated content.

But Prof James Grimmelmann of Cornell University argues this law “only protects sites from liability for third-party content from users, not content the sites themselves produce”.

Grimmelmann said xAI was trying to deflect blame for the imagery on to users, but expressed doubt this argument would hold up in court.

“This isn’t a case where users are making the images themselves and then sharing them on X,” he said.

In this case “xAI itself is making the images. That’s outside of what Section 230 applies to”, he added.

Senator Ron Wyden of Oregon has argued that Section 230, which he co-authored, does not apply to AI-generated images. He said companies should be held fully responsible for such content.

“I’m glad to see states like California step up to investigate Elon Musk’s horrific child sexual abuse material generator,” Wyden told the BBC on Wednesday.

Wyden is one of the three Democratic senators who asked Apple and Google to remove X and Grok from their app stores.

The announcement of the probe in California comes as the UK is preparing legislation that would make it illegal to create non-consensual intimate images.

The UK watchdog Ofcom has also launched an investigation into Grok.

If it determines the platform has broken the law, it can issue fines of up to 10% of its worldwide revenue or £18m, whichever is greater.

On Monday, Sir Keir Starmer told Labour MPs that Musk’s social media platform X could lose the “right to self regulate” adding that “if X cannot control Grok, we will.”