eSafety Says Proposed Duty of Care Could Enable Tests of AI Child Abuse Risks

A handheld smartphone displays the icons of some of the main artificial intelligence based apps in Paris, France, on July 15, 2026. Martin Lelievre/AFP via Getty Images

Australia’s eSafety Commissioner Julie Inman Grant has told a parliamentary inquiry that her office is not allowed by law to test whether artificial intelligence (AI) tools will produce child abuse or terror material, but a proposed new law would change that.

Appearing before the Joint Select Committee on Artificial Intelligence, Inman Grant was pressed by Nationals Senator Ross Cadell on how the regulator could check an AI service’s safeguards without asking it to create illegal material.

“How do you know it can do child porn if you don’t ask it to generate child porn?” he asked.

Inman Grant said eSafety was “not currently allowed under the law” to type requests into AI tools to see whether they would produce such material.

When Cadell suggested the government’s planned digital duty of care would let eSafety generate illegal content, she replied, “Yes, but for a very specific purpose.”

Heidi Snell, eSafety’s general manager of regulatory operations, said a properly built service should refuse such a request.

“But the only way that you can confirm whether a service has the appropriate guardrails correctly is to put in a prompt asking it to generate what would be illegal content,” she said.

If a service did produce the material, Snell said, eSafety would have grounds to act against the company.

Fake Accounts Power in Draft Bill

The draft law, released for public comment in September, would let the Commissioner use fake identities, which it calls “sock puppet” accounts, to test online services.

It also allows the commissioner to use those accounts to create material with AI, but only where making or having that material is not a crime.

Inman Grant said eSafety currently had to tell companies such as gaming platform Roblox which accounts it was using for testing, meaning the company could watch what eSafety did, “so you can’t really accurately test.”

Australia already has enforceable online safety standards requiring technology companies to take steps to prevent child sexual abuse material and pro-terror content from being stored, distributed or generated through their services.

Introduced in December 2024, the standards cover major technology providers and certain messaging, storage and online services.

The proposed digital duty of care would go further by giving the eSafety Commissioner new ways to test whether companies’ safeguards work in practice.

Grok Under Investigation

eSafety’s written submission (pdf) to the inquiry said it is investigating xAI’s Grok chatbot following reports in January that it was being used to create child sexual abuse material.

The submission also said AI model-sharing site Hugging Face changed its rules after eSafety raised concerns, and now requires users to take steps to stop their models being used to make child abuse or terror material.

eSafety said that, as far as it knew, Australia’s online safety law was the only one in the world covering services that share AI models.

When the change was announced in November last year, Inman Grant said Australians had downloaded models from such sites and used them to create abuse material, “including depictions of real children and survivors of sexual abuse.”