A small randomized pilot published in JMIR Mental Health has found that a purpose-built, solution-focused generative AI therapy chatbot kept far fewer users engaged than a group simply using off-the-shelf ChatGPT, a counterintuitive result that complicates the narrative that specialized mental-health AI is automatically safer or more effective than general chatbots.
Inside the Trial
Researchers randomized 185 English-speaking adults into three arms: a custom AI therapy chatbot built around solution-focused techniques, plain ChatGPT with no specialized prompting, and an assessment-only control group that received no chatbot access. The study’s authors describe it explicitly as an early-stage pilot rather than a definitive efficacy trial, and caution against over-interpreting the results.
The Completion Gap
Only 39% of participants assigned to the purpose-built AI therapy chatbot completed all sessions, compared with 62% of those using plain ChatGPT. That 23-point gap is the headline finding, and it runs opposite to what many mental-health AI developers would predict, since the specialized tool was designed with therapeutic frameworks in mind while ChatGPT was not.
How This Fits Other 2026 Findings
The result sits alongside other recent data on AI and mental health. A RAND survey reported earlier this year that nearly 1 in 5 adolescents and young adults now turn to AI chatbots for advice when they feel sad, angry, nervous or stressed, up from about 13% in early 2025. Separately, a trial of the chatbot Therabot — originally published in NEJM AI — found that its 210 participants showed greater symptom reduction than waitlist controls, though that study required human oversight to screen out unsafe content and only followed users for four weeks after the intervention ended.
Possible Explanations for the Drop-Off
The JMIR authors have not published a definitive explanation, but the pattern raises several possibilities worth testing: a structured, clinically-styled chatbot may feel more like an obligation or a clinical appointment, discouraging casual return visits, whereas ChatGPT’s open-ended, low-pressure format may better match how people actually want to vent or seek advice informally. It is also possible that participants found the specialized bot’s therapeutic framing repetitive or impersonal compared with a general-purpose assistant that can follow tangents.
Two Views From the Field
Digital-health researchers who favor specialized clinical chatbots argue that completion rates alone don’t capture safety and efficacy — a purpose-built tool can include guardrails against harmful advice that a general chatbot lacks, even if users drop off sooner. Critics on the other side counter that a therapy tool nobody finishes using cannot help anyone, and argue the industry has been too quick to assume that adding a clinical veneer to a chatbot automatically improves outcomes, when user engagement data increasingly suggests otherwise.
What’s Next
With the pilot’s authors stopping short of efficacy claims, the next step will be larger, longer trials that track not just completion rates but actual clinical outcomes and safety incidents across both specialized and general-purpose chatbots. Regulators and insurers weighing whether to reimburse AI mental-health tools will likely want that data before treating completion-rate findings like this one as more than a flag for further study.
Photo: Solen Feyissa / PEXELS via Pexels
