Google’s AI Overviews, a feature designed to provide quick answers to health-related queries, cites YouTube more frequently than any medical website, according to a new study. This revelation raises concerns about the reliability of information accessed by the two billion users who interact with Google’s AI summaries each month.
The company has consistently maintained that its AI-generated summaries are “reliable,” drawing from reputable sources like the Centers for Disease Control and Prevention and the Mayo Clinic. However, research conducted by SE Ranking, a search engine optimization platform, challenges this claim. Their study analyzed over 50,000 health-related queries, revealing that YouTube was the most cited source, overshadowing traditional medical sites.
Study Findings and Implications
The study captured Google search responses from Berlin, showing that YouTube accounted for 4.43% of all AI Overview citations. This figure surpasses citations from any hospital network, government health portal, or academic institution. Researchers emphasized the significance of this finding, noting that YouTube is a general-purpose video platform where content can be uploaded by anyone, ranging from board-certified physicians to wellness influencers without medical training.
Google responded to the findings, asserting that AI Overviews are designed to highlight high-quality content from credible sources, regardless of format. The company also pointed out that many authoritative health professionals and institutions create content on YouTube. However, the study’s regional focus on German-language queries limits its generalizability, according to Google.
Concerns Over Health Misinformation
The research follows a Guardian investigation that exposed risks associated with false health information in Google AI Overviews. In one alarming instance, Google provided incorrect data about liver function tests, potentially misleading individuals with serious liver conditions into believing they were healthy. This incident prompted Google to remove AI Overviews for some medical searches.
SE Ranking’s study involved an extensive analysis of 50,807 healthcare-related prompts and keywords to determine the sources AI Overviews relied on. The choice of Germany as the study location was strategic, given the country’s strict healthcare regulations governed by both national and EU directives.
“AI Overviews surfaced on more than 82% of health searches, with YouTube being the most frequently cited domain, accounting for 20,621 out of 465,823 citations,” the researchers reported.
Expert Opinions and Broader Context
Hannah van Kolfschooten, an AI, health, and law researcher at the University of Basel, commented on the study’s implications. “This study provides empirical evidence that the risks posed by AI Overviews for health are structural, not anecdotal. It becomes difficult for Google to argue that misleading or harmful health outputs are rare cases,” she said.
According to van Kolfschooten, the findings suggest that the design of AI Overviews prioritizes visibility and popularity over medical reliability. This reliance on YouTube, rather than public health authorities or medical institutions, could undermine the credibility of health information provided by Google’s AI.
Google’s Defense and Future Considerations
Google has defended its AI Overviews, highlighting that the most cited domains in the study are reputable websites. The company noted that 96% of the 25 most cited YouTube videos were from medical channels. Despite this, researchers cautioned that these videos represent less than 1% of all YouTube links cited by AI Overviews.
“Most of them (24 out of 25) come from medical-related channels like hospitals, clinics, and health organizations,” the researchers wrote. “However, these 25 videos are just a tiny slice (less than 1% of all YouTube links AI Overviews actually cite). With the rest of the videos, the situation could be very different.”
The findings underscore the need for ongoing scrutiny of AI-driven tools and their impact on public health information. As Google continues to refine its AI Overviews, the balance between accessibility and accuracy remains a critical consideration. The study’s revelations may prompt further investigations into how AI technologies curate and present health data globally.
