Finance

FDA's Bold Move into AI: A Promising Tool or Just a Buggy Mess?

2025-06-05

Author: Yan

In a surprising turn under the Trump administration, the Food and Drug Administration (FDA) has rushed to embrace cutting-edge artificial intelligence with the introduction of an agency-wide tool named Elsa. However, early reports from within the agency suggest that this initiative might not be the triumph that officials envisioned.

On Monday, the FDA unveiled Elsa, a large language model (LLM) designed to assist employees across various roles, including scientific reviewers and investigators. The agency touted Elsa's capabilities to "accelerate clinical protocol reviews, shorten scientific evaluations, and pinpoint high-priority inspection targets." According to the announcement, this generative AI could efficiently summarize adverse events and even generate coding for database development.

A Promising Start or a Rush to Failure?

Despite the optimistic rollout by FDA Chief AI Officer Jeremy Walsh, who called this the "dawn of the AI era at the FDA," the reality may be hitting harder than anticipated. Walsh emphasized that Elsa represents a transformative force for FDA employees. However, concerns are surfacing about the tool's quality and reliability.

A recent report from NBC News revealed that during tests conducted by FDA staff, Elsa produced summaries that were either completely inaccurate or partially incorrect when queried about FDA-approved products. Insiders expressed their frustration, labeling the tool "rushed" and claiming that its potential had been greatly exaggerated by agency officials and those at the Department of Government Efficiency (DOGE), which was previously led by the controversial Elon Musk.

Internal Concerns: Is AI Really Ready?

Several FDA employees voiced skepticism about the tool's current capabilities, suggesting it should be limited to administrative tasks rather than key scientific evaluations. An insider noted, "There’s a belief among leadership that AI can replace staff and expedite review times, but that’s just not the case." Additionally, the absence of established guidelines for the tool has raised alarms; one employee expressed concern that in the rush to deploy Elsa, critical oversight and policy considerations were overlooked.

Elsa is powered by Anthropic's Claude LLM and developed with the help of consulting firm Deloitte, which has received substantial funding—over $28 million in contracts since 2020—to support this AI project. Designed to operate within a secure GovCloud environment, Elsa is intended to maintain confidentiality by restricting access to internal documents.

Redistributing Resources: From Multiple Pilots to One Tool

Previously, various departments within the FDA were piloting their own AI tools. However, due to recent cost-cutting measures, the FDA decided to standardize these efforts by adopting the Center for Drug Evaluation and Research's existing AI pilot, CDER-GPT, rebranding it as Elsa. Yet further feedback from the Center for Devices and Radiological Health revealed that their version, CDRH-GPT, is plagued with issues, including connectivity problems and bugs.

As the FDA forges ahead with its AI ambitions, the pressing question remains: Will Elsa prove to be a revolutionary asset or just another example of hasty innovation in the face of complex regulatory challenges?