How to Navigate Quality Issues With Mental Health Apps

Psychiatrists and mental health clinicians can be limited in the amount of information they gather in a single visit. For instance, much of the session may be dedicated to retrospective recollection of progress since the last visit.

Now, there are new techniques to make this process more effective. Smartphone apps, digital therapeutics, and mobile device monitoring—known as passive data—hold promise for making psychiatric visits more time-effective.

But how can patients choose—and best adopt—these technologies?

Navigating the minefield of mental health apps

Critics of mental health apps point to a lack of evidence supporting their real-world effectiveness. A wide array of disconnected strategies—including technology literacy, socioeconomic determinants of health and the Digital Divide—further underscores the difficulty in distributing the right apps to the right people.

A large survey of 811 individuals with mental health needs found they chose apps through a combination of social media marketing, search engines, or word-of-mouth, as opposed to professional recommendations from medical providers.¹

Most app users will tend to rely on online app reviews, which can often be misleading. And, of the 25 popular apps for anxiety and worry, which were ranked by user ratings, none of them included content consistent with evidence-based treatments.² And currently, there are over 20,000 mental health apps available in app distribution platforms.³

With a low barrier to entry and minimal regulation, quality is not guaranteed. With very few of these apps having an evidence-based approach, issues with privacy of user data, lack of security and unregulated content, the problems become increasingly concerning.

A recent study that reviewed 73 of the top mental health apps from 2 major app stores reported that 64% of these apps claimed to be effective. When further reviewed, it was found that only 2 apps (2.7%) actually had evidence-based claims of effectiveness.⁴ Users and physicians should be careful when looking at ratings in app stores, as these are entirely user-driven, with no input from clinicians, researchers and/or technologists.

Finally, mental health apps—like any app on the Google Play Store or Apple’s App Store—can have variable combinations of features that may lead to inconsistent patient engagement. Some apps may not have critical features, such as tailored feedback, help in emergencies, and help in high-risk situations. In terms of engagement, 23% of users abandon mobile apps after the first use, with even greater variability seen in mental health users. In a systematic review of 93 mental health apps, user retention was reported to be low, with a median 15-day retention rate of 3.9% and a 30-day median retention rate of 3.3%.⁵

With these issues in mind, we recommend bringing together users and clinicians in the development process for apps. We also need to understand reasons behind low user engagement, low retention rates, and counteract them with engaging, compelling experiences.

How can we get the right apps into the hands of patients?

Keeping the end-user and clinicians in mind, the American Psychiatric Association (APA) has created the App Advisor framework and model to guide evaluation of these smartphone apps.⁶ The framework helps people evaluate whether a smartphone app is useful toward their clinical care, how to accurately judge its ratings, and whether they may be potentially dangerous towards a user.

This framework includes an 8-question “evaluation model screener,” along with 5 levels within the comprehensive app evaluation model. These questions include, but are not limited to:

• Access and Background: What information is known about the app, including who created it, how much it costs, platforms it works on, and if it has been updated within the last 180 days?

• Privacy and Security: Does the app provide a clear, accessible privacy policy; describe use of protected health information (PHI); provide ways to opt out of data collection and delete data; and secure data?

• Clinical Foundation: Does the app do what it claims to do? What evidence is there for its effectiveness and efficacy? Does it provide benefit?

• Usability: Is it customizable? Is it easy to use? And, how does it engage with you?

• Data Integration Toward Therapeutic Goal: Can the data be shared with an electronic health record, other data tools, and other individuals such as the provider? Does it improve therapeutic alliance?

However, with users often navigating these marketplaces without expert help, finding the right fit for their needs is still problematic. There are many different kinds of mental health apps, some targeting a specific disorder and some offering general therapies. Without clinical involvement, the retention rates may also be lower.

Given that the app markets have algorithms based on user perception and popularity—with little to no vetting from clinical experts—there need to be better ways by which consumers obtain their apps. APA’s evaluation guidance closes this gap.

Additionally, for apps to reach patients, the gap between developers and patients must be narrowed, as currently very little expert opinion goes into the app development process.⁷ The involvement of clinicians in the development process would improve the quality of these apps greatly, and help in their integration in clinical workflows, thus driving usability.

Ultimately, it is the clinicians who decide on therapeutic agents for patients, whether they may be digital or nondigital, and their prescription of these apps will be the biggest driver in adoption.

References

1. Schueller SM, Neary M, O’Loughlin K, Adkins EC. Discovery of and interest in health apps among those with mental health needs: survey and focus group study. J Med Internet Res. 2018;20(6):e10141. doi:10.2196/10141
2. Kertz SJ, MacLaren Kelly J, Stevens KT, Schrock M, Danitz SB. A review of free iPhone applications designed to target anxiety and worry. J Technol Behav Sci. 2017;2(2):61-70. doi:10.1007/s41347-016-0006-y
3. Pappas S. Providing care in innovative ways. American Psychological Association. January 1, 2020. Accessed April 21, 2022.
4. Larsen ME, Huckvale K, Nicholas J, et al. Using science to sell apps: evaluation of mental health app store quality claims. Npj Digit Med. 2019;2(1):18. doi:10.1038/s41746-019-0093-1
5. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9):e14567-e14567. doi:10.2196/14567
6. App Advisor. American Psychiatric Association. Accessed April 21, 2022.
7. Ondersma SJ, Walters ST. Clinician’s guide to evaluating and developing eHealth interventions for mental health. Psychiatr Res Clin Pract. 2020;2(1):26-33. doi:10.1176/appi.prcp.2020.20190036