Gemini 4 Carbon Tops Benchmarks on Multimodal Reasoning and Cybersecurity

Google DeepMind has released Gemini 4 Carbon, a frontier AI model that takes the top spot on the Artificial Analysis Intelligence Index v5.0 with a score of 59, up from predecessor Argon's 50, according to The Decoder. The model arrives just two weeks after Argon's launch and is initially restricted to a vetted group of cybersecurity defense companies rather than the general public, undergoing the U.S. government's voluntary pre-release testing process before broader rollout.

What Happened

According to The Decoder, Carbon was built for what Google calls 'much harder, long-horizon engineering problems,' with internal teams reportedly running the model for multiple days straight on single tasks, including root-cause analysis and bug fixes across large codebases. Google reported improvements in scientific reasoning, multimodal understanding, and full-stack web development based on user feedback. Higher-quality outputs and artifacts in web development specifically drove Carbon's 89 Elo point gain on WebDev Arena, where it reached fourth place at 1,769.

On the multimodal and reasoning benchmarks, MarkTechPost reports Carbon's widest performance gap came on GraphWalks, a long-context benchmark spanning 256K to 1M tokens, where it scored 96.2% F1 compared to Argon's 84.2% and GPT-6 Astra's 71.8%. The model also posted 80.0% on RiemannBench, a scientific reasoning test, and reached 77.4% on Agents' Last Exam v1.1, a computer-use evaluation. The Decoder noted that on AA-Briefcase, an agentic knowledge-work benchmark, Carbon's answer accuracy climbed from 48% to 58% compared to Argon. On Terminal-Bench 4.0, Carbon scored 65.2%, ahead of Claude Opus 5.5 at 59.6%. On SWE-sweep, Carbon reached 9.6%, narrowly ahead of GPT-6 Astra at 9.5%.

The hallucination trade-off emerged as a notable constraint on Carbon's improvements. The Decoder reported that Carbon's hallucination rate on AA-Briefcase rose from 4% to 11%, still the second-lowest in the ranking according to Artificial Analysis, suggesting that enhanced reasoning capability came paired with increased tendency to generate unsupported claims. This tension between capability and reliability represents a design choice Google made in optimizing for long-horizon task completion rather than conservative output generation.

Carbon's cybersecurity results drew the most attention in Google's launch materials. The model scored 88.2% on SEC-bench Pro, a vulnerability-discovery benchmark tested without safety guardrails, and 54.8% on ExploitGym, which measures the ability to turn a discovered flaw into a working exploit. On CWE-Bench, measuring the ability to write security patches, Carbon reached 71.0%, ahead of both Argon at 68.0% and GPT-6 Astra at 68.0%. The Verge noted Google described the gains as a 'significant leap' in cyberdefense, with the company citing internal results showing Carbon produced secure code more than twice as often as Argon and that its Chrome team found it more than ten times better than an unnamed competitor's model at writing security patches. These internal figures have not been independently verified by external security researchers or benchmarking platforms.

MarkTechPost clarified that Google published a 17-benchmark comparison in its launch post, with Carbon leading 15 of those tests. The comparison spans rivals including Claude Opus 5.5, GPT-6 Astra, and predecessor Argon, with most rival numbers self-reported by each provider according to Google's evaluation methodology. Independent verification from Vals AI placed Carbon at 70.36% accuracy in their v2.2 Index, between Claude Sonnet 5.5 at 70.85% and Claude Opus 5.5 at 70.18%, running faster and cheaper than both Anthropic models at just under 48 minutes per task at $10.51 per test. On Arena.ai's Agent Arena across more than 21,900 real sessions, Carbon placed second with a net improvement of 16.2%, behind Claude Opus 5.5 (Max) at 21.1% and ahead of GPT-6 Astra at 13.4%, while maintaining the lowest median task cost at $1.80, 72% below Claude Opus 5.5.

Access remains tightly controlled through staged rollout. The Decoder reported that Carbon is going through the U.S. government's voluntary pre-release testing process because of its advanced security capabilities, with selected cyber defenders getting access this week before expansion through Google's Fairwind Program to governments, infrastructure operators, and core tech platforms. Developers, businesses, and consumers will receive access later through Google's product interfaces, though no public date has been provided. Pricing holds at the same introductory rate as Argon, $2 per million input tokens and $10 per million output tokens, before rising to $4 and $20 after the promotional period ends (scheduled through end of 2026). Cached input tokens receive a 95% discount off the regular input price, with per-task costs in the Artificial Analysis Intelligence Index calculated at $7.76, approximately 23% less than Claude Opus 5.5 (max) at $10.04.

Why It Matters

The GraphWalks and RiemannBench results point to something beyond raw coding speed: Carbon is narrowing the gap on tasks that require holding and reasoning across very large context windows, a capability that matters for scientific research, legal discovery, and enterprise document analysis as much as for cybersecurity. On Vals AI's Legal Research benchmark, Carbon recorded roughly 45% more cited sources, more frequent tool calls, and longer responses than Argon, though tasks averaged more than 50 minutes each. The two-week gap between Argon and Carbon also signals that Google is now iterating on frontier models at a pace closer to software patch cycles than traditional research releases, a trend worth tracking alongside how compute allocation scales with release frequency. The rising hallucination rate alongside improved accuracy is a tension worth watching, since it suggests more capable reasoning doesn't automatically mean more reliable output—a critical constraint for autonomous systems operating on production infrastructure. Restricting early access to vetted cyber defenders rather than the open API reflects a dual-use calculus: a frontier model that finds and exploits vulnerabilities faster than rivals is as useful to attackers as to defenders. Google appears to be managing this tradeoff through staged rollout rather than withholding the capability altogether, consistent with how frontier AI companies are balancing capability and safety in deployment strategies as models become more autonomous.

Background

Carbon's emphasis on autonomous, multi-day task execution echoes the broader industry shift toward agentic systems, where AI models move from conversational tools to self-directing agents capable of managing entire software workflows without constant human oversight. Claude 4 established this pattern, demonstrating that frontier models could take responsibility for complete features from conception through deployment, maintaining coherent project awareness across extended timeframes. That same arc—models completing longer, more independent tasks while facing tradeoffs between capability and oversight—now extends into cybersecurity, where the stakes of autonomous action are measured in exploits rather than lines of code.

What to Watch

Independent verification will matter most in the coming weeks. Several of Google's strongest claims, including the Chrome team's security-patch comparison and the internal secure-code generation figures, come from self-run tests that MarkTechPost noted haven't been independently replicated by external security firms. Watch for third-party evaluators like Artificial Analysis and Vals AI to publish follow-up numbers, and for how quickly Carbon moves beyond the Fairwind Program to developers and consumers, since Google has provided no public date for that expansion. The rising hallucination rate on agentic knowledge tasks is also worth tracking as Carbon's user base grows beyond cybersecurity specialists into general enterprise use—a 7-percentage-point increase from Argon suggests the model's training optimized for task persistence rather than output reliability, a tradeoff that may limit adoption in domains where hallucinations carry high cost.