The ethical ambiguity of AI data enrichment: Measuring gaps in research ethics norms and practices

Abstract

The technical progression of artificial intelligence (AI) research has been built on breakthroughs in fields such as computer science, statistics, and mathematics. However, in the past decade AI researchers have increasingly looked to the social sciences, turning to human interactions to solve the challenges of model development. Paying crowdsourcing workers to generate or curate data, or 'data enrichment', has become indispensable for many areas of AI research, from natural language processing to reinforcement learning from human feedback (RLHF). Other fields that routinely interact with crowdsourcing workers, such as Psychology, have developed common governance requirements and norms to ensure research is undertaken ethically. This study explores how, and to what extent, comparable research ethics requirements and norms have developed for AI research and data enrichment. We focus on the approach taken by two leading conferences: ICLR and NeurIPS, and journal publisher Springer. In a longitudinal study of accepted papers, and via a comparison with Psychology and CHI papers, this work finds that leading AI venues have begun to establish protocols for human data collection, but these are are inconsistently followed by authors. Whilst Psychology papers engaging with crowdsourcing workers frequently disclose ethics reviews, payment data, demographic data and other information, similar disclosures are far less common in leading AI venues despite similar guidance. The work concludes with hypotheses to explain these gaps in research ethics practices and considerations for its implications.CCS Concepts: • Social and professional topics → Codes of ethics; • General and reference → General conference proceedings;• Computing methodologies → Artificial intelligence.

Do large language models have a legal duty to tell the truth?

“…RLHF workers can provide subjective perceptions of the truthfulness or accuracy of statements and indicate a preference for factual responses. 4 The ever-popular solution of 'more data' can make LLMs sound more convincing without necessarily increasing their reliability ( [5], pp. 9-11, [48,49]).…”

“…2-6], LLMs would need to transform from incidental to deterministic truth tellers. 4 It is worth noting, however, that RLHF and other data enrichment work are often highly opaque and secretive processes. Users will rarely have full awareness of how precisely models have been aligned with ground truth through human feedback.…”

“…Users will rarely have full awareness of how precisely models have been aligned with ground truth through human feedback. See ( [4], p. 262, [47], p. 16, [5], pp. 6-8).…”

“…Others warn of the danger of outsourcing the social, reflective and iterative processes of learning and research ( [58], pp. [4][5][6]. Learning can be outsourced to LLM-generated analyses and summaries of topics capable of making one seem knowledgeable without any underlying training or expertise.…”