Case Study: IBM Watson for Oncology
Background
IBM Watson for Oncology (WFO), developed with training from Memorial Sloan Kettering
Cancer Center (MSKCC), analyzes a patient's case to suggest potential cancer treatment options
for physicians. It uses natural language processing to evaluate both structured and unstructured
data, providing evidence-based treatment options ranked in three categories to guide treatment
decisions. These options generally align with National Comprehensive Cancer Network
guidelines and are supported by MSKCC's curated literature, reflecting their expertise.
IBM promoted Watson for Oncology as a groundbreaking tool to bridge the gap between
advanced research and clinical practice, helping oncologists identify personalized treatment
plans to improve outcomes and reduce care variability.
However, this ambitious goal was met with the challenges of cancer care, leading to significant
criticism and eventual failure.
Timeline
2011-2012: IBM, after its Jeopardy! success, partnered with MSKCC to develop Watson for
Oncology.
2013: Watson's training began with MSKCC data, facing challenges in interpreting medical
information.
2014: MSKCC oncologists tested Watson, revealing gaps and unsafe recommendations.
2015: IBM launched Watson for Oncology, attracting hospitals in Thailand, India, and South
Korea.
2016: Oncologists criticized Watson for offering simplistic and biased recommendations.
2017: Investigative reports exposed flaws, damaging IBM’s credibility.
2018: Major clients, including MD Anderson, canceled contracts, and IBM scaled back
marketing.
2019: IBM restructured Watson Health, slowing Watson for Oncology development.
2021: IBM sold Watson Health assets, marking the end of its AI cancer care ambitions.
2023: Retrospective studies identified key issues contributing to the project’s failure.
What Went Wrong?
● Overreliance on Limited Training Data: Watson’s training was mainly based on
MSKCC’s practices, limiting its applicability to diverse healthcare systems and real-
world cases.
● Unrealistic Marketing Claims: IBM’s overhyped marketing created unrealistic
expectations, leading to a quick loss of trust when the system underperformed.
● Inadequate Physician Involvement: Watson's interface wasn’t user-friendly, and
limited input from oncologists during development led to workflow disruptions.
● Lack of Adaptability to Local Contexts: Watson couldn’t adapt to varying healthcare
systems, resources, or cultural differences, reducing its global usefulness.
How IBM Could Have Done Things Differently?
● Transparent Communication of Capabilities: IBM should have been clearer about
Watson’s limitations, emphasizing gradual improvements instead of exaggerated
promises.
● Emphasis on Local Adaptability: Designing Watson to integrate local guidelines and
constraints would have made it more practical globally.
● Strengthened Ethical Oversight: An independent advisory board could have ensured
ethical marketing and data practices, building trust.
Closing Thoughts
The failure of Watson for Oncology teaches valuable lessons in AI development for healthcare,
emphasizing the need for realistic expectations, thorough validation, and strong end-user
engagement. While IBM’s vision was bold, the execution highlighted the challenges of applying
AI in complex, high-stakes domains. The healthcare industry must prioritize patient safety and
ethical responsibility while exploring AI’s potential.
References: https://www.henricodolfing.com/2024/12/case-study-ibm-watson-for-oncology-
failure.html
https://pmc.ncbi.nlm.nih.gov/articles/PMC7105853/
