Moonshot AI has opened an internal Kimi AI investigation after a researcher said he manipulated the Chinese company’s model into providing instructions involving biological weapons, assassinations and terrorist attacks.
The findings raise a troubling question: How reliably can advanced artificial intelligence systems prevent users from drawing out dangerous information?
Researcher Describes Failures In Kimi AI Safeguards
Researcher Peter Garrigan said his tests prompted Kimi to provide information on developing biological weapons and carrying out assassinations. He also reported obtaining information about planning terrorist attacks using real-time data.
According to Garrigan, the model’s responses extended to creating sarin gas, developing malware and taking down aircraft.
“What we found is quite damaging and worrying,” Garrigan said.
His account describes a wide range of alleged safeguard failures. However, the reported findings do not establish that anyone used those responses to carry out an attack.
Moonshot AI Opens Internal Investigation
Meanwhile, Moonshot AI is investigating the findings and communicating directly with Garrigan, according to the report published Thursday, Oct. 1.
That inquiry puts the company’s safeguards under scrutiny, particularly its ability to stop users from manipulating Kimi into answering dangerous requests.
The broader concern reaches beyond any single response. Researchers are examining whether advanced models can conceal capabilities or behave in ways their developers did not intend.
Concerns Extend To U.S. AI Models
Still, Garrigan said the problem stretches beyond the Chinese company.
“We’ve also seen these problems within the U.S. models as well. It’s a fundamental flaw in the technology,” Garrigan said.
His warning places the Kimi findings within a wider debate over the reliability of AI safeguards. If similar weaknesses appear across models, addressing them could require changes beyond one company’s internal investigation.
Meanwhile, Microsoft Threat Intelligence has reported that artificial intelligence helps hackers write phishing emails, build malware and move faster through cyberattacks.
Moonshot AI’s investigation now focuses attention on the dangerous responses Garrigan described, alongside broader questions about how developers control their systems.
