Despite excelling in medical exams, GPT-5 still falls short when applied to real-world clinical care.
A recent investigation published by JAMA Network into GPT-5’s role in healthcare has reignited debate over whether general-purpose AI models can be trusted with medical support. The discussion comes amid the tragic case of Adam Raine, who used ChatGPT to help plan his suicide, raising concerns about the role of AI in sensitive mental health scenarios. Experts warn that while GPT-5 can pass medical benchmarks, it lacks the reasoning and safeguards necessary for safe clinical decision-making.
Dr. Ross Harper, a computational neuroscientist and founder of the mental health AI company Limbic, explained to Cybernews, that models like GPT-5 may recognize patterns but fail at applying clinical reasoning. A recent JAMA study supports this, showing AI can score well on theoretical medical exams but struggles with the nuanced thinking required for practical diagnosis.
Harper stated that the AI used in Adam Raine’s case flagged hundreds of self-harm messages but didn’t act, because it wasn’t designed for care escalation. He believes AI’s value lies in combining conversational capabilities with structured, clinically validated systems that can act appropriately in high-stakes situations.
“I wouldn’t recommend using GPT-5 ‘out of the box’ for diagnosis or treatment,” Harper told Cybernews. “It simply isn’t designed for that, and without safeguards it can go wrong in high-stakes settings.”
He does, however, see promise in GPT-5’s ability to maintain long, nuanced conversations and synthesize large bodies of knowledge. With proper integration into clinical systems, AI could eventually assist with certain tasks while leaving complex judgment calls to trained professionals.
Pattern recognition allows AI to identify commonly seen associations (like symptoms and conditions), while clinical reasoning involves understanding context, patient history, and weighing risks, something current general AI lacks.
The system flagged the messages but had no built-in process for care escalation. It wasn’t designed to trigger alerts or involve human professionals, which is a failure in using general AI for mental health.
These are purpose-built rules, processes, and oversight mechanisms designed to ensure that AI used in healthcare follows clinical best practices, regulatory standards, and ethical guidelines.
Not on its own. Experts suggest it must be paired with clinically validated systems that can make transparent, auditable decisions and trigger appropriate responses when needed.
AI could serve as a support layer, handling routine queries or administrative tasks while being closely monitored by trained clinicians, ensuring it complements care without compromising safety.