China's J-36 designers warn AI hallucinations risk defence miscalculations
Synopsis
Key Takeaways
Engineers at China's AVIC Chengdu Aircraft Research and Design Institute — the developer behind the People's Liberation Army's frontline J-20 and next-generation J-36 stealth fighters — have publicly warned that AI hallucinations pose a serious threat to military intelligence operations, potentially triggering strategic miscalculations in high-stakes defence scenarios.
The Warning from the Frontlines of Chinese Aerospace
In a paper published on June 20, AVIC Chengdu engineer Zhang Xianzhe cautioned that large language models deployed for defence intelligence work are prone to fabricating critical data. According to the paper, Zhang wrote that 'AI models can invent aircraft specifications — length, payload, weapons, speed, combat radius — and get radar scan ranges and frequency bands wrong.' The research was published in Information Studies: Theory & Application, a journal operated by Norinco, China's state-owned arms manufacturer.
Why It Matters: Hallucinations in a Zero-Error Domain
'The so-called hallucination effect, where models produce plausible but false outputs, could have severe consequences in the high-stakes, low-error world of defence intelligence, possibly even causing strategic miscalculations,' Zhang said in the paper. The concern is not theoretical: large language models are increasingly being evaluated for roles in battlefield intelligence, threat assessment, and weapons performance analysis — tasks that once demanded extensive human analyst hours.
According to the paper, every fighter jet programme begins with a granular understanding of the adversary and the operational environment. Designers must resolve critical questions — what is the opponent's radar coverage, which frequencies do they use — before a single component is engineered. Faulty AI-generated answers to those questions could corrupt the entire design calculus.
The Competitive Backdrop: AI Races Into Military Systems
The caution from AVIC Chengdu arrives as militaries worldwide accelerate the integration of AI into intelligence and decision-support systems. The United States Department of Defense has been expanding its own AI-enabled programmes, while Iran and other regional powers have signalled interest in autonomous defence technologies. The PLA's own modernisation roadmap has placed AI at the centre of next-generation warfare doctrine, making the internal dissent from AVIC Chengdu's engineers all the more striking.
What's Next: Guardrails Before Deployment
The publication of Zhang Xianzhe's findings in a Norinco-affiliated journal suggests the concern has reached institutional levels within China's defence-industrial complex. Analysts will be watching whether the paper prompts formal verification standards or human-in-the-loop mandates for AI tools used in military intelligence pipelines. As the J-36 programme advances toward operational readiness, the guardrails placed around AI-assisted design and threat analysis will be a critical variable to monitor.