The history of species immigration can dictate how species interact in local communities, thereby causing historical contingency in community assembly. Since immigration history is rarely known, these historical influences, or priority effects, pose a major challenge in predicting community assembly. Here, we provide a graph-based, non-parametric, theoretical framework for understanding the predictability of community assembly as affected by priority effects. To develop this framework, we first show that the diversity of possible priority effects increases super-exponentially with the number of species. We then point out that, despite this diversity, the consequences of priority effects for multispecies communities can be classified into four basic types, each of which reduces community predictability: alternative stable states, alternative transient paths, compositional cycles and the lack of escapes from compositional cycles to stable states. Using a neural network, we show that this classification of priority effects enables accurate explanation of community predictability, particularly when each species immigrates repeatedly. We also demonstrate the empirical utility of our theoretical framework by applying it to two experimentally derived assembly graphs of algal and ciliate communities. Based on these analyses, we discuss how the framework proposed here can help guide experimental investigation of the predictability of historydependent community assembly.