Using GPT for Market Research

Brand, James; Israeli, Ayelet; Ngwe, Donald

doi:10.2139/ssrn.4395751

Cited by 46 publications

(11 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, from the perspective of cost and efficiency, many social science researches began to employ LLMs to replace humans as research subjects (Aher, Arriaga, and Kalai 2022;Argyle et al 2023;Bybee 2023;Park et al 2022). For example, in order to explore fairness and framing effects in sociology, LLMs were introduced into the classic game experiments (Horton 2023), which demonstrated the potential of LLMs to deal with social issues; in the research of consumer behavior (Brand, Israeli, and Ngwe 2023), the behavior of LLMs was consistent with economic theory in many respects (i.e. downward-sloping demand curves, diminishing marginal utility of income, and state dependence); in fi-nance research (Chen et al 2023), LLMs' decisions in budgetary allocation scenarios received higher rationality scores compared to humans; and in psychology experiments (Dillion et al 2023), the behavior of LLMs was highly consistent with the mainstream values of society.…”

Section: Llms In Social Sciencementioning

confidence: 99%

Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis

Fan,

Chen,

Jin

et al. 2024

AAAI

View full text Add to dashboard Cite

Game theory, as an analytical tool, is frequently utilized to analyze human behavior in social science research. With the high alignment between the behavior of Large Language Models (LLMs) and humans, a promising research direction is to employ LLMs as substitutes for humans in game experiments, enabling social science research. However, despite numerous empirical researches on the combination of LLMs and game theory, the capability boundaries of LLMs in game theory remain unclear. In this research, we endeavor to systematically analyze LLMs in the context of game theory. Specifically, rationality, as the fundamental principle of game theory, serves as the metric for evaluating players' behavior --- building a clear desire, refining belief about uncertainty, and taking optimal actions. Accordingly, we select three classical games (dictator game, Rock-Paper-Scissors, and ring-network game) to analyze to what extent LLMs can achieve rationality in these three aspects. The experimental results indicate that even the current state-of-the-art LLM (GPT-4) exhibits substantial disparities compared to humans in game theory. For instance, LLMs struggle to build desires based on uncommon preferences, fail to refine belief from many simple patterns, and may overlook or modify refined belief when taking actions. Therefore, we consider that introducing LLMs into game experiments in the field of social science should be approached with greater caution.

show abstract

Section: Llms In Social Sciencementioning

confidence: 99%

Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis

Fan,

Chen,

Jin

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…They document that ChatGPT is able to replicate results from experiments with human subjects, and that results vary according to different demographic characteristics of the personas. Brand et al (2023) document that ChatGPT can replicate patterns of consumer behavior, yielding estimates of willingness-to-pay similar to humans. Finally, Park et al (2023) document that ChatGPT can simulate human behavior, taking actions that vary with the agents' experiences and environment.…”

Section: Can Current Llms Impersonate People?mentioning

confidence: 99%

More Human than Human: Measuring ChatGPT Political Bias

Motoki

Neto²,

Rodrigues³

2023

SSRN Journal

View full text Add to dashboard Cite

“…Psychologists and marketing researchers have started reflecting on how LLMs might impact consumer and marketing research (e.g., Peres et al, 2023). Studies in this domains emphasize LLMs' potential to improve marketing communications (e.g., content marketing campaigns and content design), deliver superior customer experience through hyperpersonalization, and enhance classic marketing research functions (Brand et al, 2023;Ooi et al, 2023;Paul et al, 2023). Researchers have also started using LLMs to substitute human participants in academic empirical research (Argyle et al, 2023;Demszky et al, 2023;Dillion et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Sarstedt,

Adler,

Rau

et al. 2024

Psychology and Marketing

View full text Add to dashboard Cite

Should consumer researchers employ silicon samples and artificially generated data based on large language models, such as GPT, to mimic human respondents' behavior? In this paper, we review recent research that has compared result patterns from silicon and human samples, finding that results vary considerably across different domains. Based on these results, we present specific recommendations for silicon sample use in consumer and marketing research. We argue that silicon samples hold particular promise in upstream parts of the research process such as qualitative pretesting and pilot studies, where researchers collect external information to safeguard follow‐up design choices. We also provide a critical assessment and recommendations for using silicon samples in main studies. Finally, we discuss ethical issues of silicon sample use and present future research avenues.

show abstract

Using GPT for Market Research

Cited by 46 publications

References 17 publications

Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis

Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis

More Human than Human: Measuring ChatGPT Political Bias

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Contact Info

Product

Resources

About