This paper surveys the emerging science of how to design a \COllective INtelligence" (COIN). A COIN is a large multi-agent system where: i) There is little to no centralized, personalized communication and/or control. ii) There is a provided world utility function that rates the possible histories of the full system.In particular, we a r e i n terested in COINs in which e a c h a g e n t runs a reinforcement learning (RL) algorithm. The conventional approach to designing large distributed systems to optimize a world utility does not use agents running RL algorithms. Rather, that approach begins with explicit modeling of the dynamics of the overall system, followed by detailed hand-tuning of the interactions between the components to ensure that they \cooperate" as far as the world utility i s concerned. This approach is labor-intensive, often results in highly nonrobust systems, and usually results in design techniques that have limited applicability. In contrast, we w i s h t o s o l v e the COIN design problem implicitly, via the \adaptive" character of the RL algorithms of each of the agents. This approach i n troduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we d o n o t have phenomena like t h e tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research speci cally concentrating on the COIN design problem has already resulted in successes in arti cial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed science of COIN design may provide much insight into other already established scienti c elds, such as economics, game theory, and population biology.