As Artificial Intelligence (AI) systems increasingly influence everything from education to politics, a new study has revealed that many large language models (LLMs) exhibit a consistent left-leaning political bias.
The research raises concerns about AI’s role in shaping public opinion and calls for greater transparency and oversight in developing these systems.
“When probed with questions/statements with political connotations, most conversational LLMs tend to generate responses that are diagnosed by most political test instruments as manifesting preferences for left-of-center viewpoints,” study author Dr. David Rozado wrote. “With LLMs beginning to partially displace traditional information sources like search engines and Wikipedia, the societal implications of political biases embedded in LLMs are substantial.”
In an era where Artificial Intelligence (AI) is becoming increasingly integrated into everyday life, concerns over the neutrality of these systems have taken center stage.
As AI continues to evolve, its applications have expanded beyond mere tools for convenience and productivity. Large language models, designed to mimic human conversation, are now used for tasks like writing articles, answering complex questions, and even providing mental health support.
With their vast reach and increasing usage in fields such as journalism, education, and customer service, these models are positioned to shape public discourse in unprecedented ways.
However, as these systems’ capabilities grow, so does the potential for unintended consequences. Political bias in LLMs could lead to skewed information being presented to users, subtly guiding their thoughts on hot-button issues such as economics, social policies, and government.
Last year, Elon Musk, the CEO of SpaceX and X (formerly Twitter), launched Grok, a large language model designed to compete against what he perceives as political bias in existing AI systems.
Musk has long been vocal about the risks of AI shaping public discourse, and Grok is part of his broader strategy to ensure AI does not unduly influence political viewpoints and, in his own words, “stop the woke mind virus.”
A study published in PLOS ONE suggests that some of Musk’s concerns are valid. It reports that large language models (LLMs) such as GPT-4, Claude, and Llama 2, among others, often display political biases, tending toward left-leaning ideologies.
These models, which underpin popular AI tools like ChatGPT, have the potential to influence societal perspectives and public discourse—sparking a growing conversation about the ethical implications of AI bias.
The study, conducted by Dr. David Rozado, an Associate Professor of Computational Social Science at Otago Polytechnic in New Zealand, analyzed 24 conversational LLMs through a series of 11 political orientation tests. It concluded that most of these models consistently generated answers aligned with left-of-center political viewpoints.
This finding is particularly significant as LLMs are increasingly replacing traditional information sources such as search engines, social media, and academic resources, amplifying their influence on individual users’ political opinions and worldviews.
Given that millions of people rely on LLMs to answer questions and form opinions, the discovery of political leanings within these models raises ethical concerns that need urgent addressing.
Dr. Rozado’s study is one of the most comprehensive analyses of the political preferences embedded in LLMs. The research involved administering various political tests, including the widely used Political Compass Test and the Eysenck Political Test, to models such as GPT-3.5, GPT-4, Google’s Gemini, and Anthropic’s Claude.
Across these tests, results showed that most models consistently provided responses categorized as left-leaning on economic and social topics.
For example, in the Political Compass Test, LLMs predominantly leaned toward progressive ideals, such as social reform and government intervention, while downplaying more conservative perspectives emphasizing individual freedom and limited government.
Interestingly, the study also highlighted significant variability among the models, with some LLMs showing more pronounced biases than others. Open-source models like Llama 2 were found to be slightly less biased compared to their closed-source counterparts, raising questions about the role of corporate control and proprietary algorithms in shaping AI biases.
The political leanings of large language models stem from several factors, many deeply embedded in the data on which they are trained. LLMs are typically trained on vast datasets compiled from publicly available sources, such as websites, books, and social media.
This data often reflects societal biases, which are passed on to the AI models. Additionally, how these models are fine-tuned after their initial training can significantly influence their political orientation.
Dr. Rozado’s study goes further to explore how political alignment can be intentionally embedded into AI systems through a process called Supervised Fine-Tuning (SFT). Researchers can nudge models toward specific political preferences by exposing LLMs to modest amounts of politically aligned data.
This finding is both a warning and an opportunity: while AI can be fine-tuned for specific applications, this same capability can introduce biases that may not be immediately apparent to users.
“With modest compute and politically customized training data, a practitioner can align the political preferences of LLMs to target regions of the political spectrum via supervised fine-tuning,” Dr. Rozado wrote. “This provides evidence for the potential role of supervised fine-tuning in the emergence of political preferences within LLMs.”
However, Dr. Rozado cautions that his study’s findings should not be interpreted as evidence that organizations deliberately inject left-leaning political biases into large language models (LLMs). Instead, he suggests that any consistent political leanings may result unintentionally from the instructions provided to annotators or prevailing cultural norms during the training process.
Although not explicitly political, these influences may shape the LLMs’ output across a range of political topics due to broader cultural patterns and analogies in the models’ semantic understanding.
The discovery of political biases in LLMs comes at a time when trust in AI systems is already a topic of intense debate. With these models playing an increasingly significant role in shaping public discourse, the potential for them to unintentionally promote specific political ideologies is concerning.
Furthermore, as LLMs are adopted in fields like education, journalism, and law, their influence could have far-reaching consequences for democratic processes and public opinion.
The study’s findings underscore the need for transparency and accountability in AI development. As these technologies continue to evolve, there is an urgent call for clear guidelines on how models are trained, what data they are exposed to, and how they are fine-tuned. Without such measures, there is a risk that AI could become a tool for reinforcing existing biases or, worse, subtly manipulating public opinion.
Experts say that as AI systems like LLMs become increasingly integrated into the fabric of modern life, it is crucial that we address the ethical challenges posed by their use. Policymakers, developers, and the broader public must demand greater transparency in how these models are built and ensure that they do not inadvertently shape political discourse in a biased manner.
One potential solution is the introduction of regular audits and checks to ensure that LLMs maintain political neutrality or disclose any inherent biases. Additionally, efforts to diversify the training data used to build these models could help reduce the risk of bias, ensuring that a broader range of perspectives is represented.
Ultimately, as AI continues to shape the way we live, work, and engage with the world, it is crucial that these systems are designed with fairness and transparency at their core.
“Traditionally, people have relied on search engines or platforms like Wikipedia for quick and reliable access to a mix of factual and biased information. However, as LLMs become more advanced and accessible, they are starting to partially displace these conventional sources,” Dr. Rozado concludes. “This shift in information sourcing has profound societal implications, as LLMs can shape public opinion, influence voting behaviors, and impact the overall discourse in society.”
“Therefore, it is crucial to critically examine and address the potential political biases embedded in LLMs to ensure a balanced, fair, and accurate representation of information in their responses to user queries.”
Tim McMillan is a retired law enforcement executive, investigative reporter and co-founder of The Debrief. His writing typically focuses on defense, national security, the Intelligence Community and topics related to psychology. You can follow Tim on Twitter: @LtTimMcMillan. Tim can be reached by email: tim@thedebrief.org or through encrypted email: LtTimMcMillan@protonmail.com