Spirit AI knocked off RoboArena top spot over benchmark hacking
Synopsis
Hangzhou start-up Spirit AI briefly topped the prestigious RoboArena global robotics benchmark above Nvidia in June 2026, only to be removed days later after benchmark creators found evidence of manipulation — exposing a growing integrity crisis in physical AI evaluation.
Key Takeaways
Spirit AI , founded in 2024 and based in Hangzhou, Zhejiang province , temporarily held the top spot on RoboArena with its Spirit v1.6 model in June 2026 , surpassing Nvidia .
The RoboArena team retroactively removed Spirit AI and X Square Robot (formerly ranked fourth) from the leaderboard after discovering evidence of benchmark manipulation.
Pranav Atreya , lead author and PhD student at UC Berkeley , confirmed the removals publicly on X but declined to name specific companies.
RoboArena is co-developed by Nvidia , Stanford University , and the University of California, Berkeley , and evaluates generalist robot policies on real-world task execution.
The incident highlights integrity risks in physical AI benchmarking as US-China competition in embodied intelligence intensifies.
Spirit AI, a Hangzhou, Zhejiang province-based physical AI start-up founded in 2024, was removed from the RoboArena global robotics benchmark in June 2026 after its Spirit v1.6 model was found to have engaged in benchmark manipulation — just days after it briefly overtook Nvidia to claim the top position on the leaderboard. The incident has cast a spotlight on the integrity of AI evaluation systems and the fierce US-China rivalry in physical AI development.
A brief moment at the top
Spirit v1.6, launched at the beginning of June 2026, climbed to the number-one spot on RoboArena — a benchmark co-developed by Nvidia and academic institutions including Stanford University and the University of California, Berkeley. The company publicly celebrated the achievement, calling RoboArena the 'Olympics' of embodied intelligence in North America.
The benchmark evaluates how effectively generalist robot policies — the core software governing movement and task execution — translate digital instructions into real-world physical actions. Topping it would have represented a significant milestone for any robotics firm, let alone a two-year-old start-up.
Benchmark hacking and the fallout
Pranav Atreya, a lead author of the RoboArena project and a PhD student at UC Berkeley, announced in a post on X in June 2026 that the team had 'retroactively removed evaluations from organisations who [it] found to be engaging in benchmark manipulation.' He did not name specific companies in the post.
Following a methodology overhaul, Spirit AI's model was removed from the official rankings. X Square Robot, another Chinese start-up whose model had ranked fourth on the leaderboard, was also dropped. The simultaneous removal of multiple entrants suggests the problem extended beyond a single actor.
Why it matters
Benchmarks like RoboArena serve as the primary public yardstick for physical AI capability, influencing investor decisions, partnership talks, and government procurement. When those rankings are gamed, the downstream effects on capital allocation and technology policy can be significant.
The controversy arrives as competition between US and Chinese firms in embodied AI — robots capable of operating autonomously in real-world environments — has intensified sharply. Other major players active in this space include Google DeepMind, Physical Intelligence, Pi, Allen Institute for AI, and Nvidia itself with its Cosmos platform.
The competitive backdrop
Spirit AI's rise and fall reflects a broader pattern: as physical AI benchmarks gain prestige, the incentive to manipulate them grows proportionally. The RoboArena team's decision to retroactively audit and revise its methodology is a direct response to that pressure.
Other models currently competing on the leaderboard include Cosmos 3, DreamZero, MolmoAct2-droid from the Allen Institute for AI, and entries from ECUST Robot Lab, indicating the field remains highly contested across both US and Chinese institutions.
What's next
The RoboArena team's methodology revision may prompt other physical AI benchmarks to introduce stricter anti-manipulation safeguards. For Spirit AI and X Square Robot, the reputational damage could complicate future fundraising and partnership efforts at a critical stage of the physical AI investment cycle. Investors and enterprise buyers will be watching whether either company can demonstrate verified, third-party-audited performance results going forward.
Point of View
And the incentive to game them scales with their prestige. What mainstream coverage tends to underplay is that RoboArena's own methodology was already under revision when the manipulation was detected — meaning the benchmark's authority was already contested before the scandal broke. This mirrors earlier controversies in large language model leaderboards, where selective test-set exposure produced misleading rankings. As physical AI moves closer to commercial deployment in logistics, manufacturing, and defence supply chains, the absence of credible, tamper-resistant evaluation frameworks is not an academic concern — it is a market-integrity and national-security issue.
NationPress
4 Aug 2026
Frequently Asked Questions
What is RoboArena and why does it matter?
RoboArena is a global benchmark for physical AI, co-developed by Nvidia, Stanford University, and UC Berkeley, that evaluates how well generalist robot software translates digital instructions into real-world physical actions. It is considered one of the most authoritative public rankings in the embodied AI field, influencing investor and enterprise decisions.
What did Spirit AI do wrong on RoboArena?
Spirit AI's Spirit v1.6 model was removed from RoboArena in June 2026 after the benchmark's creators found evidence of 'benchmark hacking' — a term for manipulating evaluation conditions to achieve artificially high scores. The RoboArena team retroactively audited submissions and stripped the model from the official leaderboard.
Which other companies were removed from RoboArena?
X Square Robot, a Chinese start-up whose model had ranked fourth on RoboArena, was also removed alongside Spirit AI. The simultaneous removals suggest the manipulation issue was not isolated to a single organisation.
How did Nvidia perform on RoboArena after Spirit AI was removed?
Spirit AI had briefly overtaken Nvidia at the top of the RoboArena leaderboard before its removal. Following the methodology overhaul and the stripping of manipulated entries, Nvidia's position on the benchmark was restored to prominence, though the exact post-revision standings were not specified.
What does this mean for the physical AI industry?
The incident exposes a growing integrity risk in physical AI benchmarking as competition between US and Chinese firms intensifies. It is likely to push benchmark operators across the sector to adopt stricter anti-manipulation protocols, and could complicate fundraising for Spirit AI and X Square Robot at a critical point in the physical AI investment cycle.
