Large language models (LLMs) have generated significant buzz in the digital marketing world, promising to change how users discover and interact with online content. Google's recent launch of AI Mode search promises a future where answers are our fingertips but content providers may be left in the dark.
We ask whether these AI-driven referrals are truly reshaping website performance, or is the impact overstated? Recent research takes a closer look, revealing that the influence of LLMs on traffic and conversions may be less dramatic than some headlines suggest.
Examining the Data: LLMs vs. Organic Search
A six-month analysis by Amsive, covering 54 diverse websites, set out to compare the effectiveness of LLM referrals to traditional organic search traffic. The findings challenge the narrative that AI chatbots are poised to overtake search engines as the primary source of high-quality leads.
- Conversion rates are nearly identical: Organic search users converted at a 4.6% rate, while LLM referrals saw a marginally higher rate of 4.87%. Statistical analysis indicates this difference is insignificant.
- LLM traffic volume is minimal: Less than 1% of total sessions came from LLMs, compared to about 32% from organic search. This gap persists even on high-traffic sites.
- Results vary across websites: The performance of LLM referrals fluctuates by site, with some seeing better results from AI sources and others from organic search. This suggests that how LLMs present content (and site relevance) matters more than any inherent traffic quality advantage.
What Marketers and SEOs Should Know
Despite claims from search engines that LLM-driven traffic may deliver more qualified clicks, the data tells a different story. This is now the second major study highlighting that while LLMs can refer engaged users, the lack of scale limits their overall impact. Organic search remains the dominant force, delivering both a higher share of sessions and consistent conversions.
Further supporting this, research from SALT.agency’s Dan Taylor indicates that organic search typically outperforms LLM referrals in user engagement. The only exceptions are niche industries like health or careers, where AI-driven traffic occasionally shows a slight edge.
Behind the Numbers: Methodology
- Robust data set: Researchers analyzed six months of Google Analytics 4 data across 54 sites, focusing on validated conversions such as form submissions and purchases.
- Statistical significance: Rigorous testing ensured that observed differences in conversion rates were genuine and not random fluctuations.
Key Takeaways for Businesses
LLMs are not set to replace organic search as a major source of conversions in the near future. While it is wise to monitor new channels like AI chatbots, the overwhelming majority of qualified traffic and sales continue to originate from traditional search. Marketers should prioritize optimizing for organic visibility and stay informed about AI advancements—but there’s no need to radically shift strategy based on current LLM performance.
For now, focus your efforts on proven channels, and view LLMs as an emerging supplement rather than a replacement.
Source: Search Engine Land
LLMs vs. Organic Search: What the Latest Data Means for Your Marketing Strategy