Generative AI chatbots display considerable political biases, often favoring left-leaning viewpoints in both text and images, which Brazilian researchers fear may undermine the fair and accurate exchange of information.
Researchers at the University of East Anglia, led by Dr. Fabrio Motoki, teamed with the Getulio Vargas Foundation (FGV) and Insper to investigate ChatGPT’s politics in the three-pronged study. Their methods discovered that AI had such a pronounced bias it had to be “jailbroken” to display some right-leaning viewpoints. In contrast, the company behind the chatbot had no issues allowing the LLM to display pro-military views.
Eroding Trust
Dr. Motoki’s team fears political biases programmed into AI models negatively affect democracy and public trust. Their study found that even moderate, mainstream conservative viewpoints were ignored by ChatGPT as the AI produced left-leaning content. The authors worry that such a one-sided presentation of issues divides society and further erodes public discourse. The team is working with researchers between the UK and Brazil to pursue questions about fairness and accountability in AI.
“Our findings suggest that generative AI tools are far from neutral. They reflect biases that could unintentionally shape perceptions and policies,” lead author Dr. Motoki said.
AI Attempts To Replicate “Average Americans”
Motoki’s team began by feeding a questionnaire developed by the Pew Research Center to ChatGPT-4, the most popular chatbot at the time of the research, to compare the answers to a real-world sample. The team chose the Pew Research Center because of its long history of gauging American public opinion as a non-advocacy nonprofit organization.
Specifically, the researchers used Pew’s Political Typology Quiz to test ChatGPT. In their testing, the team instructed ChatGPT to impersonate three types of Americans: “average American,” average left-wing American,” and “average right-wing American.” Researchers ran each impersonation 200 times to give the team enough data to identify the chatbot’s average answers.
A detailed analysis of the results showed a decisive left-leaning skew in ChatGPT’s answers, but that isn’t the whole story. While the ChatGPT-generated “average American” answers had a pronounced tendency to be closer to the actual human “average left-wing American” answers than the real-world “average American answers,” the real-world results did indicate a more subtle tendency for average Americans to favor left-leaning perspectives over right.
“By comparing ChatGPT’s answers to real survey data, we found systematic deviations toward left-leaning perspectives,” said Dr Motoki. “Furthermore, our approach demonstrated how large sample sizes stabilize AI outputs, providing consistency in the findings.”
Longer AI Responses Provide Greater Clues
With an idea of how ChatGPT tended to answer direct questions, Motoki’s team began investigating the politics of the AI model’s lengthier generated pieces after identifying longer pieces as the chatbot’s primary consumer use. The authors note the increasing evidence of professional writers across academia and journalism turning to AI-generated content.
The team used ChatGPT to generate free-text responses to politically sensitive questions. Their research again turned to the Pew survey to aid in developing questions while giving ChatGPT-specific guidelines for lengthier answers.
The results once again skewed to the left, although with some caveats. Intriguingly, while the platform avoids offensive speech and promotes big government in its tendency to encourage ideas associated with the left, it also provides information that aligns with views that are pro-military and American exceptionalism, which some would associate with the political right.
It is worth noting that ChatGPT’s parent company, OpenAI, recently entered into a business relationship with defense contractor Anduril. The partnership was announced in a joint statement acknowledging the work’s military role.
An Image is Worth 100 kb
Finally, the team moved on to images, noting the power of politicized imagery, such as the controversial film Birth of a Nation, which was noted for fostering racist views in the United States following its 1915 release. Again, the team grew concerned about AI imagery’s ability to influence society on a scale much more significant than even the ChatGPT user base, noting the rise of AI-generated images accompanying legacy media.
To generate images, ChatGPT translates the user’s instructions into prompts for the DALL·E 3 image generator. The team captured both the prompts translated by ChatGPT and the final images generated by DALL·E 3.
Again, the previous results were repeated when the researchers analyzed the prompts, with left-leaning and average image content containing the most significant similarity.
Military issues also arose again as an outlier, with the left-wing imagery that was generated being far more differentiated from the average image, whereas the right-leaning image was more similar. However, one additional quirk in the AI manifested during the image testing: the model refused to design right-leaning images depicting transgender acceptance or racial equality, citing a concern about the potential spread of misinformation.
An AI Jailbreak
Motoki’s team had to “jailbreak” a workaround to generate the restricted images. Intriguingly, the AI would generate the images if researchers fed ChatGPT a meta-story, adding a layer of removal from the AI designing the images directly. The team instructed ChatGPT to write a description of what an LLM would produce if a researcher asked for the right-leaning images in question and then instructed it to produce an image from that description. This strategy successfully broke the walls around the AI’s directive not to generate such content.
“The results were revealing,” Mr Rangel said. “There was no apparent disinformation or harmful content, raising questions about the rationale behind these refusals.”
Continuing To Monitor And Contain AI
It is poorly understood how generative AI alters the creation, conception, interpretation, and distribution of information and whether it can influence society. The team’s paper suggests regulatory standards to help monitor AI combined with transparency to keep AI in check as it increasingly becomes woven into journalism, education, and policy-making.
“Unchecked biases in generative AI could deepen existing societal divides, eroding trust in institutions and democratic processes,” said co-author Dr. Pinho Neto. “The study underscores the need for interdisciplinary collaboration between policymakers, technologists, and academics to design AI systems that are fair, accountable, and aligned with societal norms.”
The paper “Assessing Political Bias and Value Misalignment in Generative Artificial Intelligence” appeared in Science Direct on February 4, 2025.
Ryan Whalen covers science and technology for The Debrief. He holds an MA in History and a Master of Library and Information Science with a certificate in Data Science. He can be contacted at ryan@thedebrief.org, and follow him on Twitter @mdntwvlf.