Current and former members of the FDA told CNN about issues with the Elsa generative AI tool unveiled by the federal agency last month. Three employees said that in practice, Elsa has hallucinated nonexistent studies or misrepresented real research. “Anything that you don’t have time to double-check is unreliable,” one source told the publication. “It hallucinates confidently.” Which isn’t exactly ideal for a tool that’s supposed to be speeding up the clinical review process and aiding with making efficient, informed decisions to benefit patients.
Leadership at the FDA appeared unfazed by the potential problems posed by Elsa. “I have not heard those specific concerns,” FDA Commissioner Marty Makary told CNN. He also emphasized that using Elsa and participating in the training to use it are currently voluntary at the agency.
A spokesperson for the Department of Health and Human Services told Engadget that “the information provided by FDA to CNN was mischaracterized and taken out of context.” The spokesperson also claimed that CNN led its story with “disgruntled former employees and sources who have never even used the current version of Elsa.” The agency claims to have guardrails and guidance for how its employees can use the tool, but its statement doesn’t address that Elsa, like any AI platform, can and will deliver incorrect or incomplete information at times. We have not yet received a response to our request for additional details.
ADVERTISEMENT
Advertisement
The CNN investigation highlighting these flaws with the FDA’s artificial intelligence arrived on the same day as the White House introduced an “AI Action Plan.” The program presented AI development as a technological arms race that the US should win at all costs, and it laid out plans to remove “red tape and onerous regulation” in the sector. It also demanded that AI be free of “ideological bias,” or in other words, only following the biases of the current administration by removing mentions of climate change, misinformation, and diversity, equity and inclusion efforts. Considering each of those three topics has a documented impact on public health, the ability of tools like Elsa to provide genuine benefits to both the FDA and to US patients looks increasingly doubtful.
Update, July 24, 2025, 6:35PM ET: Added a statement from the Department of Health and Human Services.
