Medical AI Tools Deployed Without Rigorous Clinical Proof, FT Investigation Finds

Medical AI tools are reaching patients faster than the evidence supporting them, according to the Financial Times, whose September 18 investigation found a persistent gap between clinical claims and rigorous proof across dozens of deployed systems. The Financial Times investigation documents what healthcare researchers have long described as a structural weakness in medical AI: tools are approved and deployed in clinical settings without the randomised controlled trial evidence typically required for drugs or medical devices.

The Evidence Gap

The Financial Times found that many AI systems reach hospitals through regulatory pathways that do not require sponsors to demonstrate patient outcome improvements, only that the software performs consistently. The result is a growing installed base of AI tools whose real-world clinical benefit remains unestablished. Tools making diagnostic suggestions or flagging drug interactions are already embedded in hospital workflows in the United States, the United Kingdom, and across the EU, yet the evidentiary bar that cleared them for use would not pass muster in a phase III drug trial.

The US FDA has cleared hundreds of AI-enabled medical devices, many under the 510(k) pathway that requires demonstration of substantial equivalence to a predicate device rather than independent clinical proof. The EU's Medical Device Regulation introduces stricter post-market surveillance obligations, but neither framework currently mandates age-stratified performance benchmarks before deployment. This regulatory structure creates a fundamental misalignment: AI systems can be deployed at scale without the type of controlled evidence that would be mandatory for a new pharmaceutical product.

The integration of AI into clinical medicine has accelerated sharply since multimodal models capable of processing both medical images and text became commercially available. Following the release of systems with advanced vision and language capabilities in September 2023, dozens of vendors have built hospital-facing products using similar architectures. This speed of deployment has outpaced the development of validation standards appropriate to the scale and complexity of the tools being introduced.

Specific Vulnerabilities in Older Populations

Compounding systemic concerns about clinical evidence, a practising geriatrician writing for STAT News has highlighted that older adults face specific vulnerabilities when AI systems are embedded in their care. The commentary warns that AI models trained predominantly on younger, healthier populations may perform poorly when applied to patients with multiple chronic conditions, polypharmacy regimens, or atypical disease presentations common in older age groups.

The geriatrician's critique sharpens the broader validation problem considerably. If an AI model's training data skews toward patients aged 30 to 60, its outputs for an 82-year-old with heart failure, type 2 diabetes, and early cognitive decline are extrapolations, not predictions grounded in validated performance. Older patients present clinical complexity that differs fundamentally from the training populations on which most widely deployed AI systems are based. Their diseases often present atypically, their medication interactions are more numerous, and their physiological responses to interventions can diverge significantly from younger cohorts.

The geriatrician argues that age-specific validation data should be a prerequisite for any AI tool deployed in geriatric settings, a standard that few current products meet. Regulators have begun to acknowledge algorithmic bias in medical AI, but population-specific validation requirements remain inconsistent across jurisdictions. This inconsistency means that a system cleared for use in one market may be deployed in another without evidence that its performance holds across the age spectrum of actual patients.

Systemic Risk and Regulatory Implications

The combination of rapid deployment, weak evidentiary standards, and documented vulnerabilities in understudied populations creates a systemic risk that extends beyond individual patient harm. When hundreds of hospitals adopt AI tools whose real-world clinical benefit has not been rigorously measured, the cumulative effect is a shift in standard medical practice based on incomplete evidence. This pattern mirrors historical episodes in medicine where widespread adoption preceded adequate validation—but with AI systems, the scale and speed of deployment can be orders of magnitude faster than traditional medical device adoption.

The most consequential near-term development will be whether the FDA or the European Medicines Agency moves to formalise clinical evidence requirements for AI tools that directly influence treatment decisions, as opposed to those used only for administrative or workflow purposes. The geriatrician commentary suggests a coalition of specialty physicians may begin pushing professional bodies to issue guidance ahead of formal regulation. Such guidance could establish minimum evidentiary thresholds that exceed current regulatory requirements and demand validation across age groups and disease complexity profiles.

The disconnect between deployment speed and validation standards is not inevitable. Regulatory pathways exist to fast-track evidence generation while still requiring meaningful clinical proof before widespread adoption. The current system, however, incentivises vendors to clear regulatory hurdles with minimal evidence and deploy at scale immediately thereafter. Without intervention, this dynamic will likely persist, with AI systems reaching vulnerable populations—particularly older patients with complex medical histories—before adequacy of evidence has been established through rigorous, population-specific clinical research.

Healthcare systems relying on these tools face an implicit dilemma: adopt AI systems that may improve efficiency but whose clinical benefit is unproven, or forgo potential gains in workflow and diagnostic speed. That choice should not be presented as binary. Stronger validation requirements, particularly for populations at highest risk from poor AI performance, would align medical AI deployment standards with the evidentiary rigor already established for other clinical interventions. Until such alignment occurs, the gap between deployment timelines and validation standards will remain a structural vulnerability in how medical AI is integrated into patient care.