Contents
Meta-analyses consistently indicate that mental health chatbots alleviate symptoms of both anxiety and depression. Although an effect size hovering around 0.3 remains modest within this academic field, it is nonetheless tangible. The core issue surfaces upon reviewing individual clinical trials one by one. Specifically, when a specialized therapy application was pitted against standard ChatGPT, the dedicated app failed to surpass the generic bot across any evaluated metric. Meanwhile, a six-week study tracking 540 participants revealed that the artificial intelligence variant boosted interaction frequency by 2.4 times and engagement duration by 3.8 times, yielding symptom reduction comparable to traditional static workbooks. Furthermore, a meta-analysis featuring a three-month post-intervention check demonstrated that the initial benefits had completely faded away. Five key metrics are thoroughly examined throughout this article.
The 30-second summary: the seven figures that matter
Each row names the study and the year. Always look at what the chatbot is being compared against.
| Finding | Result | Study and year |
|---|---|---|
| Effect on depression across 38 randomised trials | g = 0.31 (95% CI: 0.17-0.46), with 7,401 participants | Sohn et al., NPJ Digital Medicine (2026) |
| Effect on anxiety across 34 of those trials | g = 0.28 (95% CI: 0.05-0.51) | Sohn et al., NPJ Digital Medicine (2026) |
| Effect at three-month follow-up | No substantial effects on depression or anxiety | Zhong et al., J. Affect. Disord. (2024) |
| Therapy app against generic ChatGPT | No differences on any measure | Kuta et al., JMIR Mental Health (2025) |
| Use of an AI app against workbooks | 2.4 times the frequency and 3.8 times the duration | McFadyen et al., Comm. Medicine (2026) |
| Symptoms in that same comparison | Comparable: no added improvement in anxiety or depression | McFadyen et al., Comm. Medicine (2026) |
| Comparator in the most-cited trial | A waiting list with no access to any app | Heinz et al., NEJM AI (2025) |
What you will find here
• How much these systems reduce symptoms, according to the synthesis of 39 trials.
• The trial where the therapy chatbot drew level with plain ChatGPT.
- Why multiplying usage fourfold did not improve symptoms.
• What happened when measured three months after the end.
The size of the effect
The most recent and comprehensive synthesis was published by Jun-Seok Sohn and colleagues in NPJ Digital Medicine in 2026, searching five databases from 2017 with prior PROSPERO registration. Out of 39 eligible randomised trials, depression was evaluated in 38 studies involving 7,401 participants, whereas anxiety was assessed in 34 trials comprising 7,621 individuals.
The pooled results: show a statistically significant decrease in depressive symptoms, yielding g = 0.31 (95% CI: 0.17-0.46), alongside a drop in anxiety symptoms, producing g = 0.28 (95% CI: 0.05-0.51). Standard measurement scales indicate these impacts remain modest.
There exists a helpful nuance within the subgroup analysis: the impact on depression proved greater among clinical and subclinical cohorts relative to non-clinical groups (p = 0.001). In other words, these interventions perform more effectively for individuals experiencing greater distress, which represents the preferred trajectory.
What that figure does not distinguish
An effect of 0.3 against a group receiving nothing is not the same as an effect of 0.3 against a reasonable alternative. Many of these trials compare the chatbot with a waiting list, that is, with doing nothing, and in mental health that systematically inflates the result through expectation, attention and the simple passage of time.
| Study | What it compared against | What it found | What it cannot support |
|---|---|---|---|
| Heinz et al., NEJM AI (2025) | Waiting list, with 210 adults over 4 weeks | The reference trial of an expert-fine-tuned chatbot | Without an active comparator, it cannot isolate the effect of receiving something |
| Kuta et al., JMIR Mental Health (2025) | Generic ChatGPT and a no-intervention group, 185 adults | Both reduce depression; between them, no difference | Small sample and a three-week intervention |
| McFadyen et al., Comm. Medicine (2026) | Digital cognitive behavioural therapy workbooks, 540 adults | Far more use, the same symptoms | Self-guided and unblinded |
| Sohn et al., NPJ Digital Medicine (2026) | Synthesis of 39 trials with any control | g = 0.31 on depression and 0.28 on anxiety | It pools very different comparators |
| Zhong et al., J. Affect. Disord. (2024) | 18 trials with 3,477 participants | Improvements at eight weeks and nothing at three months | Heterogeneous designs and therapeutic approaches |
The trial that compared against plain ChatGPT
This is the research that ought to be shared most widely yet travels the least. In 2025, Barbora Kuta along with her colleagues published a trial in JMIR Mental Health featuring 185 adults split into three groups: a generative therapy app based on solution-focused therapy, general-purpose ChatGPT, and a no-intervention control group. Three weeks, with nine scheduled sessions per active cohort.
Compared to the control cohort, both active arms showed reductions in depressive symptoms according to the PHQ-9 inventory: d = −0.47 (p = 0.01) within the AI therapy cohort alongside d = −0.44 (p = 0.02) inside the ChatGPT cohort. Regarding anxiety levels and overall wellbeing, neither arm achieved statistical significance.
And the conclusive outcome: the therapy app group failed to surpass the ChatGPT group across every single metric. Discrepancies between them proved negligible and statistically insignificant regarding depression, anxiety, and overall wellbeing.
A brief three-week test ought not to be given excessive weight. Even so, it prompts an unsettling dilemma for the entire industry: if an instrument built expressly for therapeutic work proves indistinguishable from a standard chatbot, the active ingredient might not stem from the clinical framework at all.
Far more use, the same symptoms
The trial led by Jessica McFadyen and her team, featured in Communications Medicine in 2026, stands as the most extensive among the separate investigations: 540 adults across the United States dealing with heightened levels of anxiety or depression, whose study design was preregistered, were divided evenly between a cognitive behavioural therapy application driven by generative AI and digital workbooks based on the identical therapy, undertaken independently across a six-week duration.
| What was measured | AI app | Digital workbooks |
|---|---|---|
| Usage frequency | 2.4 times higher | Baseline |
| Usage duration | 3.8 times higher | Baseline |
| Anxiety symptoms (GAD-7) | Comparable | Comparable |
| Depression symptoms (PHQ-9) | Comparable | Comparable |
| Adverse events | No differences | No differences |
The authors’ conclusion is honest and applies across the field: in self-directed use, tailored generative-AI-enabled therapy safely enhances user engagement above and beyond static materials, without showing an overall enhancement in anxiety or depression symptom reduction.
Put another way: AI solves the problem of people abandoning these tools, which is real and important, and not the problem of making them work better. An exploratory analysis in the same work suggests that those who used the clinical personalisation features did see greater anxiety reduction, but that is a comparison between people who chose to use them, not a random assignment.
What remains after three months
Wei Zhong, Jiang-Hua Luo and Hong Zhang published a meta-analysis in the Journal of Affective Disorders in 2024 covering 18 randomised trials with 3,477 participants, searching five databases.
Their immediate results are consistent with the rest: improvements in depression (g = −0.26; 95% CI: −0.34 to −0.17) and anxiety (g = −0.19; 95% CI: −0.29 to −0.09), with maximum benefit after eight weeks of treatment.
And the figure that almost never makes the headlines: at three-month follow-up, no substantial effects were detected for either condition. The benefit exists while use continues and does not appear to leave anything behind.
That constraint links directly to a nuance within the domain’s foundational study, conducted by Michael Heinz and co-authors and published in NEJM AI in 2025, which evaluated an expert-adapted generative chatbot involving 210 adults with clinically significant symptoms of major depression, generalised anxiety or high risk for an eating disorder. The control cohort relied on a waiting list with no access to any app throughout the four-week investigation. Such a setup invariably maximizes apparent outcomes, and this very methodology drives a significant share of the broader public discourse surrounding the topic.
Practical ways to handle this
Ordered by the strength of the evidence behind it:
• Ask what it was compared against before believing a figure. A waiting list is not an active comparator, and many of these trials use one.
• Do not expect the effect to last. The three-month follow-up meta-analysis uncovered no meaningful impact regarding anxiety or depression.
• If the goal is retention, AI does deliver. It multiplied usage frequency by 2.4 and duration by 3.8 against static materials.
- Do not assume a specialised tool is better. In the only trial that compared them, it did not outperform generic ChatGPT on any measure.
• Treat it as a supplement in mild or moderate cases. The pooled effect, g = 0.31, is real but small, and larger in clinical samples than in the general population.
Frequently asked questions
Do therapy chatbots work?
They exert a genuine, modest impact. A meta-analysis incorporating 39 randomized trials released in NPJ Digital Medicine throughout 2026 calculates a decrease regarding depressive symptoms of g = 0.31 (95% CI: 0.17-0.46) across 7,401 subjects, alongside anxiety symptom drops amounting to g = 0.28 (95% CI: 0.05-0.51), demonstrating more pronounced outcomes among clinical and subclinical cohorts relative to the broader public.
Does the improvement hold over time?
The available evidence says no. The meta-analysis of 18 trials with 3,477 participants found improvements in depression (g = −0.26) and anxiety (g = −0.19), peaking at eight weeks, but detected no substantial effects at three-month follow-up.
Does a therapy app surpass using ChatGPT directly?
In the only trial to have compared them, no. With 185 adults split between a generative therapy app, general-purpose ChatGPT and a control group, both active groups reduced depression against the control (d = −0.47 and d = −0.44), and the specialised app did not outperform ChatGPT on any measure. It is a small, three-week study.
What is the actual purpose of AI within these applications?
Above all, for getting people to use them. In the trial with 540 participants, the generative-AI version multiplied usage frequency by 2.4 and duration by 3.8 compared with digital workbooks of the same therapy, with comparable results on anxiety and depression and no differences in adverse events. Dropout is a real problem for digital therapies, and that is where it does help.
Why is the comparison group of such vital importance?
Because it completely alters the interpretation of the outcome. The discipline’s most-frequently referenced study, released in NEJM AI during 2025 involving 210 adults experiencing clinically meaningful symptoms, contrasted the chatbot against a control group on a waiting list without access to any software application. Compared to receiving zero treatment, virtually any intervention enhances the quantified symptoms.
What we do not know
• How they compare with human therapy. None of these trials compares the chatbot with equivalent in-person psychotherapy.
- What happens in a crisis. Trials usually exclude people at acute risk, who are precisely the greatest concern.
- Whether the impact stems from the material or the interaction itself. The appeal seen with a standard chatbot implies that therapeutic structuring might not be the defining element.
• Virtually nothing concerning teenagers and older individuals. The 2026 assessment explicitly highlights that investigations within those two demographic cohorts are entirely lacking.
References
1. Sohn, J.-S., Ha, B.-G. et al. Systematic review and meta analysis of chatbots in the management of depressive and anxiety symptoms. NPJ Digital Medicine, 2026. DOI: 10.1038/s41746-026-02566-w
2. Kuta, B., Novak, L. et al. Effectiveness of a Fully Automated Mobile Therapeutic Versus a General Chatbot in Reducing Depression and Anxiety and Improving Well-Being. JMIR Mental Health, 2025. DOI: 10.2196/82642
3. McFadyen, J., Habicht, J. et al. Increasing engagement with cognitive-behavioral therapy (CBT) using generative AI: a randomized controlled trial. Communications Medicine, 2026. DOI: 10.1038/s43856-025-01321-8
4. Zhong, W., Luo, J.-H. and Zhang, H. The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments. Journal of Affective Disorders, 2024. DOI: 10.1016/j.jad.2024.04.057
5. Heinz, M. V., Mackin, D. M. et al. Randomized Trial of a Generative AI Chatbot for Mental Health Treatment. NEJM AI, 2025. DOI: 10.1056/aioa2400802
This content is for information only and does not constitute medical or psychological advice, nor does it replace care from a mental health professional. If you are in a situation of risk, contact your local emergency services or helpline.