The Offense-Defense Balance of Scientific Knowledge

Shevlane, Toby; Dafoe, Allan

doi:10.1145/3375627.3375815

Cited by 32 publications

(26 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sophisticated or institutional actors with the capacity to embark on large-scale disinformation, cyberwarfare, or targeted phishing also are likely to have the capacity to create a similar model if none were released. Although potentially significant, these harms should not therefore weight heavily on a release calculus [Solaiman et al 2019;Shevlane and Dafoe 2020]. The harms to be weighed against the benefits are those from less well-resourced actors who would not be able to create their own foundation model but may be motivated to generate spam or abuse, fake reviews, or cheat on tests.…”

Section: Release and Auditingmentioning

confidence: 99%

On the Opportunities and Risks of Foundation Models

Bommasani¹,

Hudson²,

Adeli³

et al. 2021

Preprint

653

652

View full text Add to dashboard Cite

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

show abstract

Section: Release and Auditingmentioning

confidence: 99%

On the Opportunities and Risks of Foundation Models

Bommasani¹,

Hudson²,

Adeli³

et al. 2021

Preprint

653

652

View full text Add to dashboard Cite

show abstract

“…Researchers should weigh national security and societal harm in decisions about how widely to publicize findings, and they should build mitigations prior to release. 214 The "do no harm" principle should weigh heavily in the decision of how much of the model, code, and tutorial to release publicly. 215…”

Section: Build and Apply Ethical Principles For The Publication Of Ai Research That Can Fuel Disinformation Campaignsmentioning

confidence: 99%

AI and the Future of Disinformation Campaigns: Part 2: A Threat Model

Sedova¹,

McNeill²,

Johnson³

et al. 2021

View full text Add to dashboard Cite

Artificial intelligence offers enormous promise to advance progress and powerful capabilities to disrupt it. This policy brief is the second installment of a series that examines how advances in AI could be exploited to enhance operations that automate disinformation campaigns. Building on the RICHDATA framework, this report describes how AI can supercharge current techniques to increase the speed, scale, and personalization of disinformation campaigns.

show abstract

“…Many if not most of the AI capabilities described above are -or derive from -dual-use capabilities that are innocuous or beneficial in other applications. Moreover, the culture of AI is characterized by a high degree of openness, and even in cases where the source code is not already openly shared, many new AI algorithms can be independently reproduced by other researchers in a matter of months, making for a low barrier to proliferation (Brundage et al, 2018: 17;Shevlane and Dafoe, 2020). On the supply-side, AI tools, especially pre-trained versions, are as accessible as any software; on the demand-side, many of these tools offer extensions on, or improvements over, the precise sort of criminal capabilities or technologies which (cyber)criminals have long sought to acquire, whether in terms of pursuing 'zero-day exploits', or through tools such as Blackshades.…”

Section: Aic: Estimating the Threatmentioning

confidence: 99%

Artificial intelligence and crime: A primer for criminologists

Hayward¹,

Maas

2020

Crime, Media, Culture

View full text Add to dashboard Cite

This article introduces the concept of Artificial Intelligence (AI) to a criminological audience. After a general review of the phenomenon (including brief explanations of important cognate fields such as ‘machine learning’, ‘deep learning’, and ‘reinforcement learning’), the paper then turns to the potential application of AI by criminals, including what we term here ‘crimes with AI’, ‘crimes against AI’, and ‘crimes by AI’. In these sections, our aim is to highlight AI’s potential as a criminogenic phenomenon, both in terms of scaling up existing crimes and facilitating new digital transgressions. In the third part of the article, we turn our attention to the main ways the AI paradigm is transforming policing, surveillance, and criminal justice practices via diffuse monitoring modalities based on prediction and prevention. Throughout the paper, we deploy an array of programmatic examples which, collectively, we hope will serve as a useful AI primer for criminologists interested in the ‘tech-crime nexus’.

show abstract

The Offense-Defense Balance of Scientific Knowledge

Cited by 32 publications

References 7 publications

On the Opportunities and Risks of Foundation Models

On the Opportunities and Risks of Foundation Models

AI and the Future of Disinformation Campaigns: Part 2: A Threat Model

Artificial intelligence and crime: A primer for criminologists

Contact Info

Product

Resources

About