Artificial intelligence tools designed to predict cancer biology from microscope images may not be as reliable as previously thought, according to new research from the University of Warwick. The study, published in Nature Biomedical Engineering, warns that these systems often depend on hidden shortcuts rather than genuine biological signals, raising concerns about their suitability for real-world patient care.
AI systems have been heralded for their potential to expedite cancer diagnoses and reduce testing costs. However, the research team led by Dr. Fayyaz Minhas, Associate Professor at the University of Warwick, found that many AI models might be using correlations between biomarkers and obvious tissue features instead of isolating biomarker-specific signals. “It’s a bit like judging a restaurant’s quality by the queue of people waiting to get in,” Dr. Minhas explained. “Many AI pathology models are doing the same thing, relying on correlations rather than true biological understanding.”
Understanding the Research Findings
The research involved analyzing over 8,000 patient samples across four major cancer types: breast, colorectal, lung, and endometrial. The team compared the performance of leading machine learning approaches and found that while these models often achieved high accuracy, this success frequently resulted from statistical “shortcuts.”
For instance, a model might predict mutations in the BRAF gene by learning that these mutations often occur alongside another clinical feature, such as microsatellite instability (MSI). This reliance on correlated features rather than the causal BRAF signal itself means that accurate predictions are only possible when these biomarkers co-occur, making the models unreliable when they do not.
“Predicting a BRAF mutation by looking at correlated features like MSI is often like predicting rain by looking at umbrellas—it works, but it doesn’t mean you understand meteorology,” said Kim Branson, SVP Global Head of AI and Machine Learning at GSK.
Implications for AI in Medical Pathology
When AI model performance was assessed within stratified patient subgroups, such as only high-grade breast cancers or only MSI-positive tumors, the accuracy fell significantly. This revealed the models’ dependence on shortcut signals that vanish once confounding factors are controlled. For certain prediction tasks, the performance advantage of deep learning over human-derived clinical information was modest, with AI systems achieving accuracy scores just over 80% compared to around 75% using tumor grade alone.
Professor Nasir Rajpoot, Director of the Tissue Image Analytics Centre at the University of Warwick, emphasized the need for rigorous evaluation. “This study highlights a critical point about the rollout of AI in medicine: the value of AI-based clinically important predictions must be judged through rigorous, bias-aware evaluation,” he stated.
The Road Ahead for AI Pathology Tools
Despite these concerns, machine learning methods remain valuable for research, drug development candidate screening, and clinical triaging. However, the researchers argue that future AI tools must move beyond correlation-based learning and adopt approaches that explicitly model biological relationships and causal structures. They call for stronger evaluation standards, including subgroup testing and comparison against simple clinical baselines, before deploying these tools in routine care.
Dr. Minhas concluded, “This research is not a condemnation of AI in pathology. It is a wake-up call. Current models may perform well in controlled settings but rely on statistical shortcuts rather than genuine biological understanding. Until more robust evaluation standards are in place, these tools should not be seen as replacements for molecular testing.”
The findings underscore the importance of understanding the limitations of AI tools in medical settings and using them with appropriate caution. As the field of AI in pathology continues to evolve, the focus must remain on developing models that truly understand the biological complexities of cancer, ensuring that AI enhances rather than replaces the expertise of human pathologists.
