Argumentation in Artificial Intelligence (AI) builds on formal approaches to reasoning argumentatively. Common to many such approaches is to use argumentation frameworks (AFs) as reasoning engines, with AFs being composed of arguments and attacks between arguments, which are instantiated from knowledge bases in a principle-based manner. While representing what can be argued for in an AF provides a conceptually clean way, this process can face challenges arising from generating a large number of arguments, which can act as a barrier to explainability. Inspired by successful approaches to model checking where the state explosion is mitigated by applying existential abstraction, we study an adaption of existential abstraction in form of clustering arguments in an AF to address an associated "argument explosion". In this paper, we provide a foundational investigation of this form of existential abstraction by defining semantics of the resulting clustered AFs, which balance two inherent aspects of existential abstractions: abstracting from concrete AFs and not permitting too much spuriousness (i.e., conclusions that hold on the abstraction but not on the original AF). Moreover, we show properties of clustered AFs, including complexity results, discuss use of clusterings for explaining results of reasoning tasks, and employ the recently introduced methodology of abstraction in answer set programming (ASP) for obtaining and reasoning over clustered AFs.