As generative artificial intelligence (genAI) becomes increasingly integrated into healthcare, regulators are facing new challenges in ensuring safety, transparency, and ethical use. During a recent U.S. Food and Drug Administration (FDA) Digital Health Advisory Committee (DHAC) meeting, representatives from the American Psychological Association (APA) and the American Psychiatric Association (APA) offered detailed recommendations on how the agency should regulate genAI-enabled mental health devices.
Balancing Innovation and Oversight
The FDA convened the 6 November meeting to discuss the regulatory landscape for AI-based mental health tools, including hypothetical chatbots designed to diagnose and treat major depressive disorder. Experts expressed concern about the rapid adoption of such tools without adequate validation, noting the risks of misinformation, data misuse, and patient harm.
Dr. Vaile Wright, Senior Director of Health Care Innovation at the American Psychological Association, highlighted a pressing issue: the U.S. mental health crisis is colliding with a workforce shortage, leaving millions without access to evidence-based care. While telehealth has helped bridge some gaps, Wright cautioned that direct-to-consumer mental health apps are not replacements for psychotherapy.
Prescription digital therapeutics, she explained, hold greater promise. These FDA-regulated software tools deliver validated, evidence-based interventions and require prescriptions from licensed providers. However, the growing gray area between unregulated mental health apps and approved therapeutics presents new risks for consumers and providers alike.
The Growing Use of AI in Mental Health
Recent surveys indicate that nearly half of U.S. adults with mental health conditions (48.7%) have used large language models such as ChatGPT for psychological support. Users frequently turn to these tools for reassurance, anxiety management, and advice. While the accessibility is appealing, experts warn that the lack of regulatory oversight could lead to misinformation or harmful self-treatment.
Wright urged the FDA to modernize its regulatory approach to reflect “the realities of AI and mental healthcare.” She recommended:
- Using existing frameworks to govern AI applications where possible.
- Strengthening pre-market evidence requirements and post-market monitoring.
- Establishing a public repository of FDA-cleared digital mental health products to enhance transparency.
- Requiring AI developers to adhere to FDA’s transparency guidelines as a baseline for all products, from entertainment apps to prescription therapeutics.
Accountability and Ethical Development
Brooke Trainum, Senior Director of Practice Policy at the American Psychiatric Association, echoed these concerns, emphasizing the ethical risks associated with AI-driven mental health interventions. She noted that physicians are currently bearing the majority of safety and liability risks—responsibilities that should also rest with developers.
“Developers must be accountable for the safety, accuracy, and ethical use of these models—not just the providers,” Trainum said.
She also called for the FDA to establish:
- Standardized labeling for all AI tools, detailing model identifiers, training data, validation studies, privacy protocols, and known limitations.
- Patient-centered model design, developed with input from mental health experts and diverse populations.
- Clinical oversight requirements for high-risk tools and restricted public access where appropriate.
Trainum underscored that there is currently no high-quality evidence demonstrating the efficacy of genAI mental health tools, and urged more rigorous research with standardized benchmarks before wide-scale use.
A Path Forward for Safe and Ethical AI
Both organizations agreed that while generative AI could play a transformative role in mental healthcare, human oversight must remain at the center of any AI-driven model. The FDA, they said, should take a proactive stance to ensure safety, enforce transparency, and promote responsible innovation.
At EMMA International
At EMMA International, we understand the regulatory complexities of emerging AI technologies in healthcare. Our experts help organizations navigate FDA and global digital health regulations, design compliant AI/ML validation strategies, and integrate risk-based quality systems to support innovation responsibly.
From software-as-a-medical-device (SaMD) oversight to ethical AI deployment, EMMA International ensures that technology advancements align with both regulatory integrity and patient safety.
For more information on how EMMA International can assist, visit www.emmainternational.com or contact us at (248) 987-4497 or info@emmainternational.com.
Reference:
Craven, J. (2025, November 11). Medical groups make recommendations for FDA regulation of genAI mental health devices. Regulatory Focus, Regulatory Affairs Professionals Society.
U.S. Food and Drug Administration. (2025). Digital Health Advisory Committee Meeting Summary: Generative AI in Mental Health Applications.
