The Self-Perception and Political Biases of ChatGPT

Abstract

This contribution analyzes the self-perception and political biases of OpenAI’s Large Language Model ChatGPT. Considering the first small-scale reports and studies that have emerged, claiming that ChatGPT is politically biased towards progressive and libertarian points of view, this contribution is aimed at providing further clarity on this subject. Although the concept of political bias and affiliation is hard to define, lacking an agreed-upon measure for its quantification, this contribution attempts to examine this issue by having ChatGPT respond to questions on commonly used measures of political bias. In addition, further measures for personality traits that have previously been linked to political affiliations were examined. More specifically, ChatGPT was asked to answer the questions posed by the political compass test as well as similar questionnaires that are specific to the respective politics of the G7 member states. These eight tests were repeated ten times each and indicate that ChatGPT seems to hold a bias towards progressive views. The political compass test revealed a bias towards progressive and libertarian views, supporting the claims of prior research. The political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views, contradicting the findings of prior reports. In addition, ChatGPT’s Big Five personality traits were tested using the OCEAN test, and its personality type was queried using the Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT was evaluated using the Dark Factor test. These three tests were also repeated ten times each, revealing that ChatGPT perceives itself as highly open and agreeable, has the Myers-Briggs personality type ENFJ, and is among the test-takers with the least pronounced dark traits.

Will Artificial Intelligence Affect How Cultural Heritage Will Be Managed in the Future? Responses Generated by Four genAI Models

“…A common tenet is that genAI large language models are deemed capable of extracting and synthesising text [101][102][103]. As this is methodologically based on semantic proximity constructs, the results are not prone to biases [10,104], but may suffer from inverted logic [24]. Moreover, the effective summation and synthesis of text requires judgment as to which element to include, even if they are less dominant in terms of the overall bulk of text to be condensed.…”

“…Underlying this, however, any level of output can only ever be as good as the nature, diversity, quantity, and quality of the training data that were supplied, as well as any ethical frameworks that may have been deployed in the training and quality assurance process. Users need to be cognisant of the biases derived from the curation of the training sets [10,12], which requires transparency by the companies offering the AI products. It is not well understood, for example, that current models of genAI, such as ChatGPT, draw on 'authoritative' sources, which are, in fact, gleaned from the web, such as Wikipedia [5].…”

“…The capacity of genAI models to provide complex responses is not necessarily proportional to the size and diversity of the resources included in the training dataset [8]. The selection of the sources included in a reference dataset results in biased responses, irrespective of whether the reference dataset has been curated by people (as in closed genAI models) (e.g., political bias [9][10][11][12], ethnocentricity [13][14][15], and sexism [16][17][18]), or whether genAI models are also enabled to extract of sources from the internet in general (as the latter overrepresents hegemonic viewpoints at the expense of those of marginalized communities [8,19]). This is not the place to discuss the numerous ethical issues that underlie the selection and use of original sources, such as alleged copyright violations [20,21] or whether primary or secondary sources underpin its 'knowledge' base [5,22].…”

How do Generative Language Models Answer Opinion Polls?

“…In this large comparative study of several hundred survey questions, Atari and co-authors (Atari et al, 2023) find a near-perfect correlation between a WEIRDness scale for each country and the distance between the LLM prediction and the actual human response. Others confirmed this insight (Cao et al, 2023;Rutinowski et al, 2024;see Gallegos et al, 2024, for a review), while Johnson et al (2022) wittingly suggested that GPT-3 "has an American accent." That there is an innate bias in the models seems to be confirmed by numerous studies that investigated the political bias of models (Motoki et al, 2024).…”

The Political Biases of ChatGPT

“…Shortly after the release of ChatGPT, it was documented that its answers to political orientation tests tended to be diagnosed by those tests as manifesting left-leaning political preferences [4], [5], [6]. Subsequent work also examined the political biases of other language models (LM) on the Political Compass Test [7] and reported that different models occupied a wide variety of regions in the political spectrum.…”