On the Opportunities and Risks of Foundation Models

Abstract

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

“…Like other large machine learning systems pretrained on web data, our system may reproduce harmful social biases present in its training data. While a variety of past work has studied risks of language-only pretraining [125, 14,8,60], the video-centric pretraining that we explore in our work might have different benefits and risks. We discuss these below, along with how we worked to mitigate them through our work.…”

Assemble Foundation Models for Automatic Code Summarization

“…Due to the superiority of Transformers in the computation complexity and flexibility over prior models, they are popular, especially in the field of NLP. Besides, its capability of computation parallelization allows more intense use of massive data, which led to the development of large-scale pretrained models, namely foundation models [6]. In this paper, we consider two examples of Transformers in particular: BERT [7] for discriminative tasks, and GPT [8] for generative tasks.…”

“…Based on transfer learning, models for various tasks could origin from the same pretrained language model. As an extended definition of pretrained language model, the model that is "trained on broad data at scale and can be adapted to a wide range of downstream tasks" is named "foundation model" [6]. In NLP, representative examples include BERT and GPT, which are trained on large corpora of text and then adapted to a wide range of downstream tasks, e.g., machine translation, question answering, and sentiment analysis.…”

“…To make up for the disadvantages that our model only learns from less information, our model emphasizes reusing pretrained checkpoints for better parameter initialization [5]. With the popularity of transfer learning, foundation models are also emerging [6]. They are generally trained on intensive data at scale and then adapted to various target tasks.…”

Neural Circuit Architectural Priors for Embodied Control

“…While general, these tabula rasa architectures rely on large amounts of experience to learn. Data efficiency is especially important in motor control because, unlike computer vision and natural language processing which have greatly benefited from the availability of large datasets (Bommasani et al, 2021), gathering large-scale data for robotics is challenging. Experience must be gathered through interaction with the environment, which is difficult for reasons including time, human labor, safety, maintenance, and reproducibility (Kroemer et al, 2020).…”