Key insights
- FDA pathway established – UpDoc’s approval demonstrates a viable regulatory pathway for SaMD products with patient-focused LLMs, although future approvals will depend on the intended features, safety and effectiveness of each product.
- Human oversight remains crucial – Clinical AI will likely continue to act as a supporting tool for physicians, rather than making independent diagnostic or treatment decisions based on human-in-the-loop and state law requirements for medical practice.
- Comprehensive compliance required — Organizations developing or deploying clinical AI must address a variety of legal concerns, including HIPAA and privacy and security, data governance, use case limitations, human-in-the-loop requirements and related concerns in medical practice, monitoring the safety and effectiveness of such tools, and complying with the rapidly evolving AI regulatory landscape.
On June 25, 2026, UpDoc Inc., a clinical AI company, announced that it has received FDA approval for what the company calls the first Software as a Medical Device (SaMD) that uses patient-centered Large Language Models (LLMs). UpDoc describes its platform as an enterprise-grade agentic clinical AI platform designed to enable physicians to deploy AI agents integrated into a provider’s electronic health record and other workflows to perform specific care-related tasks.
UpDoc received a 510K letter of approval on December 23, 2025 for its type 2 diabetes medication management software, which implements a treatment plan determined by the healthcare provider. The summary accompanying the letter describes a conversational data capture module that allows patients to enter data via voice or chat interfaces with the UpDoc agent and receive new treatment plan instructions.
While this development presents a potential path for SaMD products to incorporate LLMs, the release and approval of future clinical AI products will depend on the intended functions, safety and effectiveness of these products. Clinical AI will likely continue to serve as a support tool for physicians and not be approved for independent diagnostic or treatment decisions due to human-in-the-loop and state law requirements for medical practice.
Organizations developing clinical AI platforms should carefully consider the specific FDA approval/approval pathway applicable to the platform based on its intended use, functionality, safety, and effectiveness, including 510(k), De Novo, and PMA pathways. Developers should also monitor FDA’s evolving expectations for AI-enabled devices, including post-market surveillance, transparent labeling and risk communication, cybersecurity, data integrity and other controls, and pre-established change control plans.
Clinical AI tools can also raise a number of complex legal and operational issues for healthcare providers, including how organizations should implement and manage the use of such tools, validate AI-powered interventions, integrate such tools into healthcare workflows, and address related liability, licensing and ethical concerns. Healthcare providers should carefully define the boundaries between autonomous AI actions, such as implementing physician-defined protocols, and required medical interventions. They should also determine how human oversight will be implemented and documented, assess whether and how payers and regulators will allow the use of AI agents, and review malpractice coverage and provider contracts to address liability concerns.
LLMs with access to patient data also raise concerns about HIPAA, privacy, cybersecurity, consent and data management. Accordingly, healthcare providers should carefully evaluate potential providers and partners to ensure compliance with applicable laws and data security and, if necessary, to impose data use restrictions and sharing restrictions.
McGuireWoods’ AI practice team includes attorneys from the FDA & Life Sciences group and the Healthcare Compliance, Regulatory & Policy group who understand the unique healthcare regulatory and compliance issues associated with the development and deployment of AI products in the healthcare industry. Our multidisciplinary teams are uniquely positioned to support life sciences and healthcare companies in the development and delivery of AI products, including consideration of FDA regulatory strategy, healthcare compliance, privacy and data security, reimbursement, enforcement risk, policy developments, and the rapidly evolving federal and state AI regulatory landscape. If you need assistance developing or deploying healthcare AI products, contact one of the authors.
https://www.mcguirewoods.com/client-resources/alerts/2026/7/a-pathway-for-clinical-ai-developers-opens-fda-clears-first-software-as-a-medical-device-with-patient-facing-llm/
