Self-assembly peptide-based hydrogels are well known and popular in biomedical applications due to the fact that they are readily controllable and have biocompatibility properties. A dipeptide is the shortest self-assembling motif of peptides. Due to its small size and simple synthesis method, dipeptide can provide a simple and easy-to-use method to study the mechanism of peptides’ self-assembly. This review describes the design and structures of self-assembly linear dipeptide hydrogels. The strategies for preparing the new generation of linear dipeptide hydrogels can be divided into three categories based on the modification site of dipeptide: 1) COOH-terminal and N-terminal modified dipeptide, 2) C-terminal modified dipeptide, and 3) uncapped dipeptide. With a deeper understanding of the relationship between the structures and properties of dipeptides, we believe that dipeptide hydrogels have great potential application in preparing minimal biocompatible materials.