Gemini 4 Carbon hallucination rate: what Artificial Analysis found and how to choose
Artificial Analysis reports Gemini 4 Carbon's hallucination rate at 11% versus 4% for Argon. See what Google claims and how to test before the Nov 30 price.
Artificial Analysis reports that Gemini 4 Carbon scores 59 on its Intelligence Index v5, 9 points above Gemini 4 Argon, and that its AA-Omniscience hallucination rate rises from 4% to 11%. Google released on November 10 at an introductory $2 and $10 per million input and output tokens, rising to $4 and $20 after November 30.
TL;DR: Artificial Analysis reports that Gemini 4 Carbon answers more questions correctly than Gemini 4 Argon (58% accuracy, up 10 points) but is wrong more often when it does not know (hallucination rate 4% to 11%). Google separately reports safety and honesty gains on its own tests, which measure different things. Test Carbon on your own questions at the effort level you plan to run before paying the post-November 30 price.
Key Takeaways
- Index lead: Artificial Analysis scores Carbon 59 on Intelligence Index v5.
- Hallucination rate: Artificial Analysis reports 11% for Carbon versus 4% for Argon on AA-Omniscience v1.1.
- Effort level matters: the 11% sits under Carbon with high reasoning, while Artificial Analysis's evaluation page lists Carbon (Medium) at 4%.
- Google's claims are separate: the 92% versus 70% honesty result comes from Google's own 375-task benchmark.
- Price changes November 30: the introductory $2 and $10 becomes $4 and $20 per million tokens.
What did Artificial Analysis find about Carbon's hallucination rate?
Artificial Analysis found that Carbon's hallucination rate rose from 4% to 11% while its accuracy rose 10 points to 58%. The same article puts Carbon's AA-Omniscience v1.1 score at 54, 8 points above Argon's 46 and 6 points above Claude Opus 5.5 (max) at 48.
The article calls 11% the second lowest of any model, behind only Argon, against 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max). It does not say why the rate rose, and this post does not guess.
Rows marked Google are the vendor's own reports.
| Measure | Gemini 4 Carbon | Gemini 4 Argon | Source |
|---|---|---|---|
| Intelligence Index v5 | 59 | 9 points below Carbon | Artificial Analysis |
| AA-Omniscience v1.1 score | 54 | 46 | Artificial Analysis |
| Accuracy (% correct) | 58% | 10 points below Carbon | Artificial Analysis |
| Hallucination rate | 11% | 4% | Artificial Analysis |
| Red-team breach rate per attempt (Google) | 0.8% | 3.1% | |
| Honesty benchmark, 375 tasks (Google) | 92% passed | 70% passed |
How does AA-Omniscience define a hallucination, and why does effort level matter?
The AA-Omniscience v1.1 hallucination rate is the proportion of incorrect answers out of all non-correct responses. Artificial Analysis's evaluation page writes it as incorrect divided by the sum of incorrect, partial and not-attempted answers. The page says it measures how often a model answers incorrectly when it should have refused or admitted it did not know.
Effort level changes the picture. The 11% figure appears under the heading "Key results for Gemini 4 Carbon with high reasoning" in the article. The evaluation page's hallucination summary names Gemini 4 Carbon (Medium) as the lowest at 4%, followed by Gemini 4 Argon (High) at 4% and Gemini 4 Hydrogen (Xhigh) at 8%. The text of that summary does not give a Carbon (High) figure, and the article does not label the effort level behind Argon's 4%.
I could not confirm from those pages how Carbon's rate varies across other effort levels. If you plan to run Carbon at medium effort, do not assume the 11% applies to you.

What does Google say about Carbon's reliability?
Google reports that Carbon passed 92% of tasks on a 375-task agentic honesty benchmark, against 70% for Argon. In the launch post, Google adds that Carbon declined to fill in missing experimental data 98% of the time.
These are vendor-reported numbers from different benchmarks than AA-Omniscience, so the 92% honesty pass rate and the 11% hallucination rate are not directly comparable. One is a task pass rate on Google's agentic test. The other is a share of non-correct answers on an independent test.
What does Carbon cost, and when does the price change?
Carbon costs $2 per million input tokens and $10 per million output tokens until November 30, then $4 and $20. Google's post states the introductory price is the same as Argon's and that the higher price applies after that date.
Artificial Analysis's charts use standard pricing of $4 and $20, the same as Argon's. At that price it reports Carbon costing $7.76 per Intelligence Index task, against $7.68 for Argon, $10.04 for Claude Opus 5.5 (max) and $6.25 for GPT-6 Astra (max).

When should you pay the post-November 30 price for Carbon?
Pay the post-November 30 price once your task rewards correct answers and you have measured its wrong-answer behavior on your own data. The sources support less: Artificial Analysis shows higher accuracy and a higher hallucination rate than Argon, and Google shows gains on its own tests.
- Pick the effort level first. The 11% belongs to high reasoning, and the 4% for Carbon belongs to medium.
- Weigh the cost of a confident wrong answer. If a wrong answer reaches a customer or a decision, the hallucination rate matters more to you than the index score.
- Run your own evaluation. Include questions where the right answer is "I don't know," and compare Carbon with Argon at the price you will pay.
Frequently Asked Questions
What is Gemini 4 Carbon's hallucination rate?
Artificial Analysis reports 11% on AA-Omniscience v1.1 for Carbon with high reasoning, up from 4% for Argon. Its evaluation page lists Carbon (Medium) at 4%.
Did Gemini 4 Carbon become more accurate than Argon?
Artificial Analysis reports that Carbon's accuracy (% correct) rose 10 points over Argon to 58%. It also reports a 9-point higher Intelligence Index v5 score, 59 for Carbon.
Is Google's 92% honesty result the same as a lower hallucination rate?
No. Google's 92% is the share of tasks passed on its own 375-task agentic honesty benchmark, and Google reports 70% for Argon. The AA-Omniscience hallucination rate is a separate, independent measure on a different test, so the two cannot be compared directly.
When does Carbon's price go up?
Google says the introductory $2 and $10 per million input and output tokens runs until November 30, after which $4 and $20 applies.
Conclusion
The sources report both gains and a higher hallucination rate, from different tests. Pick your effort level, run your own questions through Carbon and Argon, and decide at the price that starts after November 30.
Learn from me

Agent Engineering Bootcamp: Developers Edition, my Maven cohort. Advanced agentic RAG, multi-agent orchestration, memory, evals, and guardrails. Take agents from prototype to production. Join the next cohort →
Hire us
Traversaal.ai. We're a team of forward deployed engineers solving the toughest AI problems for Fortune 100 companies: document intelligence, agentic data platforms, and real-time web intelligence, deployed in production. Work with our team to deploy your next agentic ecosystem. Talk to Traversaal.ai →
Join us
Want to solve these problems with us? We're always looking for forward deployed engineers who want to ship production AI. jobs@traversaal.ai