The public comment period on the FDA’s discussion paper governing generative AI-enabled medical devices closed October 19, capping a two-month stretch in which hospitals, AI developers, and patient groups weighed in on how the agency should regulate software that can reason, converse, and make autonomous clinical recommendations rather than simply flag an abnormal scan. The paper, issued by the FDA’s Digital Health Center of Excellence on August 18, is the agency’s first attempt to sketch out oversight for foundation models and agentic AI systems operating in clinical settings.
Why Generative AI Breaks the Old Framework
The FDA’s existing device framework was built around AI tools with narrow, fixed functions, software that flags a suspicious mole or scores a lung nodule’s cancer risk, where the range of possible outputs can be validated in advance. Generative AI models, particularly those built on large language models, can produce open-ended text responses, adapt behavior based on conversation history, and in agentic configurations take sequential actions without a clinician approving each step, none of which fits cleanly into premarket review processes designed for fixed-output software.
The Two-Axis Risk Framework
Central to the discussion paper is a proposed risk framework that would calibrate regulatory scrutiny along two dimensions simultaneously: the clinical significance of the device’s output, meaning how much harm a wrong answer could cause, and its level of autonomy, meaning how much a human clinician reviews before anything happens to a patient. A low-autonomy tool that drafts a summary for a doctor to check would face lighter requirements than a highly autonomous agent that could order tests or adjust medication dosing on its own, even if both used similar underlying models.
A ‘Competency Assessment’ Modeled on Medical Training
The paper also floats a premarket evaluation concept its authors compare to physician competency testing: a combination of non-clinical benchmarking, essentially standardized exams for the AI model, followed by clinical confirmation studies that test performance in real care settings before deployment. The analogy is meant to address a core criticism of current AI device review, that passing a narrow validation dataset doesn’t guarantee safe performance across the messier range of patients and scenarios a model will encounter once deployed broadly.
Industry and Hospital Reaction
Health systems have pushed back on parts of the FDA’s early thinking, arguing that overly rigid premarket requirements could slow deployment of tools clinicians say are already useful in draft form, such as AI that drafts clinical notes or summarizes patient histories, even though those same institutions generally support stronger oversight for higher-autonomy tools making direct treatment decisions. AI developers, meanwhile, have pressed for clearer definitions of where documentation and chatbot-style tools end and regulated “device” functions begin, since overly broad definitions could pull low-risk productivity software into the same review pipeline as autonomous diagnostic systems.
This Isn’t the Only AI Rule in Motion
The generative AI discussion paper sits alongside a separate FDA effort, finalized guidance on evaluating the credibility of AI models used in drug and biologic development submissions, reflecting a broader agency push across multiple offices to build AI-specific review pathways rather than retrofit decades-old device and drug rules. Agencies in the EU and UK are working through similar questions in parallel, and FDA officials have signaled interest in aligning US approach with international counterparts to avoid forcing AI developers to build separate compliance pathways per market.
What’s Next
Because this is a discussion paper rather than binding guidance, the FDA is not obligated to act on any specific timeline, but the agency typically uses this format to gather industry input before drafting formal guidance, a process that has historically taken six months to over a year in comparable cases. Hospitals and AI vendors now await the agency’s synthesis of comments, with the two-axis risk model and competency-assessment concept the most likely elements to carry into whatever formal framework follows.
Photo: DarkoStojanovic / PIXABAY via Pixabay
