What political science teaches us about AI SEO
Imagine you ask somebody at a CrossFit gym what they eat after a workout. They recommend boiled ground beef. Now imagine, you ask ChatGPT for the best diapers. According to The Information, you will be recommended Coterie. As SEO evolves to cater to AI search, companies are working to understand how to get these bots to recommend their particular products. The above article mentions several methods companies now employ to better understand AI search. These methods closely resemble those that political scientists have long-used to understand belief formation.
Media diets impact both people and chatbots. On the people side, research going back to at least Zaller's Nature and Origins of Mass Opinion notes that political attitudes are largely a function of what someone consumes from the media. Supporting this idea, researchers recently ran an experiment where they paid Fox News viewers to watch CNN for a month. After a month of watching CNN, these users had more moderate political beliefs compared to a control group. Just as the source matters for political -- or nutritional -- beliefs, companies are applying similar thinking to AI systems. The Information article above notes that companies closely monitor what sources LLMs cite in their answers. For example, startups like Profound use several variants of a search and count what sources are cited. Similarly, the article notes that marketers are posting more content on Reddit, which they believe is a hotbed for raw content that will eventually be featured in AI search results.
But people don't uncritically parrot all of the news they come across. People, and LLMs, still need to be persuaded. Both LLM creators and new companies are working to understand what "persuades" these bots to respond as they do. This can also benefit from political science research. A recent paper studying the impact of a personal attack in a campaign advertisement noted that you can't simply A/B test one version of an ad that contained an attack and another that does not. For instance, imagine a policy that induced anger among many respondents. Because this policy makes people angry, it is more amenable to an attack ad. Comparing the attack ad to another version of the same ad minus the attack won't accurately measure the causal impact of the attack on a viewer's attitude because both the attack and the response to the ad is confounded by the anger-inducing policy. This type of political science research shows us that A/B testing with AI search has subtle differences from A/B testing typically used to evaluate marketing campaigns.
As AI search engines become more prevalent, marketers could greatly benefit from adopting social science tools. Content may need to be hosted on platforms that LLMs are likely to scrape and A/B testing will need to account for confounders in the text. Political scientists have spent decades studying and measuring opinion formation. In the process they have built a large toolbox that can help navigate AI search.