In this paper we discuss the use of the social concept 'conviviality' for computer science in general, and for the development of ambient technologies in particular. First, we give a survey of the use of the concept 'conviviality' in the social sciences. Conviviality is usually considered a positive concept related to sociability. However, further analysis reveals a negative side related to lack of diversity, privacy and ethical issues. Second, we argue that conviviality requirements for ambient intelligence are challenging, because ambient technologies give rise to a new virtual and social reality, and conviviality issues play a central role in applications that are concerned with the interaction of material, virtual and social realities. Conviviality highlights an important challenge that we illustrate with examples that emphasize ethical issues, such as privacy threats, surveillance of users and identity theft. Intelligent interfaces, for example, allow instant interactions and thereby create strong needs for coordination and regulation mechanisms that have to be addressed to ensure the safeguard of individuals against abuses, such as privacy intrusions and identity manipulations. Third, we propose a conviviality ontology by operationalizing the fuzzy concept of 'conviviality,' such that it can be used in computer science in the same way as other social concepts such as 'service,' 'contract' or 'trust' are used in this area. Conviviality is defined using dependence networks, and tools for conviviality are based on, what we call, conviviality masks. Fourth, we illustrate how convivial ambient intelligence applications can be designed using our operationalized concept of conviviality. We illustrate our arguments and contributions with a running example on the use of ambient technologies in digital cities, as a prototypical example where material reality such as ambient technologies interacts with virtual and social realities.