Knowledge tracing (KT) which aims at predicting learner's knowledge mastery plays an important role in the computer-aided educational system. Given learners' exercise records, a knowledge tracing model can trace their hidden knowledge state dynamically. In recent years, many deep learning models have been applied to tackle the KT task, which has shown promising results. However, they still have limitations. Most existing methods simplify the exercising records as knowledge sequence, which fails to explore rich information existed in exercise texts. Besides, the latent hierarchical graph nature of exercises and knowledge remain unexplored. us, in this paper, we propose a hierarchical graph knowledge tracing model framework (HGKT) which can leverage the advantages of hierarchical exercise graph and sequence model to enhance the ability of knowledge tracing. Besides, we introduce the concept of problem schema to be er represent a group of similar exercises and propose a hierarchical graph neural network to learn representations of problem schemas. Moreover, in the sequence model, we employ two a ention mechanisms to highlight important historical states of students. In the testing stage, we present a K&S diagnosis matrix that could trace the transition of mastery of knowledge and problem schema, which can more easily be applied to di erent applications. Finally, we conduct extensive experiments to evaluate the model on a large scale real-world dataset.e results prove the e ectiveness of our model and the diversity of its application scenarios.