The presence of fractures increases the difficulty of flow mechanisms analysis, and it remains unclear how fractures affect multiphase flow displacement in the layered rock matrix. Herein, a pore-scale imbibition model considering the layered matrix-fracture system is established using the phase-field method, where oil is displaced by a range of fluids with various properties. Two typical flow modes are carefully analyzed, depending on the locations of the fracture and the interfaces between different layers of the matrix: fracture is parallel to the interface (mode I), and it penetrates through the interface (mode II), which are dominated by the co-current imbibition and countercurrent imbibition mechanisms, respectively. Interestingly, the surface tension is found to be negatively correlated with the ultimate oil recovery rate for mode I and plays an opposite effect on that of mode II. For flow mode I, the conditions of lower injection rate, higher viscosity ratio, higher grain diameter ratio, and injection of the invading fluid from the larger pore throat size (positive direction flow) can improve oil recovery. For flow mode II, the fracture bifurcation angle has little effect on the positive direction flow, while it can significantly regulate the phase distribution in the negative direction flow. Based on scaling analysis of relating pore-filling events to displacement modes and the equilibrium relationship between capillary and viscous forces, two theoretical models are derived to predict the imbibition patterns, and the variation of the flow regime under various parameters in the typical layered matrix-fracture models is systematically concluded.