Ex-Anthropic Engineer Launches AI Agent Safety Auditing Startup AIUC

A former early-stage engineer at Anthropic and the former chief operating officer of METR have launched a startup designed to address one of the most concrete unsolved problems in deploying autonomous AI systems: ensuring that agents behave as intended once handed elevated permissions in enterprise environments. According to TechCrunch, Rune Kvist and Rajiv Dattani founded Artificial Intelligence Underwriting Company (AIUC), which has built a third-party audit and certification layer for AI agents modeled on cybersecurity standards.

The startup announced a $40 million Series A led by Ribbit Capital, with participation from First Harmonic. This followed a $15 million seed round from Nat Friedman through his fund NFDG, Emergence, Terrain, and Anthropic co-founder Ben Mann, bringing total funding to $55 million. The backing from prominent venture investors and an Anthropic co-founder reflects confidence in the founders' approach to a problem that enterprises increasingly face as agentic AI systems move from research environments into production workflows.

The Core Problem

The challenge of controlling AI agents is rooted in how modern agentic systems operate. According to Kvist, "AI is getting smarter at an increasingly rapid rate. The surprising thing about AI is that it becomes harder to adopt and harder to control as AI gets smarter, not easier." As agentic AI systems move from research demonstrations into production developer workflows—automating code review, data pipelines, and multi-step software tasks—the risk that an agent pursues an objective in an unexpected way transforms from a theoretical concern into an engineering problem with real consequences.

Kvist emphasized the commercial constraint driving the problem: "Banks, hospitals, governments and militaries no longer decline to deploy AI because a model isn't smart enough. They decline because they've made commitments to their own customers about what a system will and won't do, and nobody can currently guarantee that." This gap between capability and controllability is what AIUC aims to address.

The AIUC-1 Standard and Testing Framework

AIUC has developed a standard called AIUC-1 and a testing service to validate agents against it, using the widely adopted cybersecurity standard SOC 2 as its conceptual model. Rather than building the standard in isolation, AIUC assembled a consortium of approximately 250 security and risk leaders—the actual buyers of agents—and conducted monthly meetings asking what they would look for when evaluating agents, what questions they would ask, and what issues they would want to see addressed.

The company runs agents through a suite of approximately 5,000 tests to see how they behave in scenarios involving jailbreaks, hallucinations, and data leaks. The results produce a roughly 100-page report detailing where an agent performs safely and reliably—and where it does not. This granular reporting allows enterprise buyers to understand not whether an agent is universally safe, but rather where it can be trusted and where guardrails or restrictions must remain in place.

Notably, AIUC uses AI agents to run the tests and AI to analyze the data, though humans verify the final audit. This approach reflects a pragmatic acknowledgment that testing agentic systems at scale requires computational efficiency, while still maintaining human oversight of conclusions.

Early Adoption and Market Positioning

AIUC has already attracted notable customers, including Cursor, Lovable, Harvey, and ElevenLabs. These companies span different segments of the AI development ecosystem—from code editors and low-code platforms to specialized AI applications—indicating that agent safety concerns cut across multiple use cases.

Dattani explained the value proposition to potential buyers: "Here's where it passes and where you can trust it. And here's where there's concerns. You should be aware of those references before you make the decision to buy." This framing positions AIUC's service not as a binary pass-fail certification but as an informational layer that allows enterprises to make informed procurement and deployment decisions aligned with their own risk tolerance.

Relationship to Prior Work and Regulatory Momentum

AIUC's approach builds on the founders' prior experience in AI evaluation. Dattani was COO of METR from 2024 to 2025 and remains a board member. METR conducts similar testing for frontier labs, though its work has focused mostly on performance—whether agents can reliably complete tasks—rather than safety. METR was one of the independent research organizations OpenAI used to investigate its Hugging Face incident, giving it credibility with frontier labs and enterprise buyers alike.

The market opportunity for AIUC is reinforced by emerging regulatory signals. Anthropic CEO Dario Amodei has called for the AI industry to pace frontier development and floated the idea of requiring frontier labs to use embedded third-party evaluators to observe and verify safety, naming METR as one possibility. While AIUC operates through auditing rather than embedded evaluation, the principle is aligned: independent verification of agent behavior before deployment at scale.

Technical and Competitive Context

Understanding why agent control is non-trivial requires understanding how modern agentic systems are constructed. The Transformer architecture, introduced in 2017, processes context across long sequences through parallel self-attention mechanisms, which is precisely what allows agents to plan multi-step tasks dynamically. This same architectural feature—the ability to generate action sequences based on continuous reasoning over context—makes agents' intermediate reasoning difficult to monitor or interrupt through simple rule-based controls. Runtime intervention mechanisms to constrain such systems require deep understanding of the model's decision-making process, not just its outputs.

For AIUC to gain traction against competitors or alternative approaches, the key technical question is whether its framework can reliably identify failure modes under realistic agentic workloads. The breadth of the testing suite (5,000 tests) and the diversity of the consortium that shaped AIUC-1 (250 security and risk leaders) suggest an attempt to capture the range of concerns that matter to actual enterprise deployments. However, enterprise adoption will ultimately depend on demonstrable performance and on whether the AIUC-1 standard itself becomes recognized as meaningful by procurement teams.

Strategic Implications

AIUC's launch reflects a broader market recognition that as agentic AI systems gain elevated permissions in enterprise environments, there will be substantial commercial value in providing verifiable guarantees about their behavior. The startup is not competing on the capability of agents themselves but on the reliability and transparency of how those capabilities are constrained and monitored. This is analogous to how cybersecurity became a distinct industry sector once it became clear that security assurance, not just system functionality, had market value.

The founders' backgrounds—Kvist's early experience at Anthropic and Dattani's operational leadership at METR—position them as credible translators between the safety research community and enterprise buyers. The Series A funding from established venture firms alongside direct participation from Anthropic co-founder Ben Mann signals that this approach has resonated with investors who understand both the technical feasibility and the commercial timing of agent safety infrastructure. As enterprises scale agentic AI deployments over the next 12-24 months, third-party verification mechanisms like AIUC's are likely to become compliance prerequisites before agents are granted access to critical systems or sensitive data.