Formation damage is a common phenomenon and is impaired to the reservoir by reducing the productivity. Formation damage is usually caused by solids plugging, clay swelling, saturation changes, etc., and fracturing fluids with a series of chemical additives are pumped into the well for production enhancement. It is difficult to optimize the fracture fluids and well shut-in time due to lack of fundamental understandings. Currently, little research has been done to investigate the mechanisms of formation damage at the pore scale. In this study, a combination of digital rock technology and core sample laboratory soaking experiments is used to evaluate the formation damages for different fracture fluids in tight sandstone reservoir. Three core samples from a full-diameter core are soaked in three different fracture fluids (surfactant, polymer, and gel) for eight different durations (from 2 h to 15d) to simulate well shut-in process. The samples in various soak times are scanned by X-ray micro-computer tomography (Micro-CT) to obtain the 3D images of the true geometry. The images are then compared to optimize the fracture fluids and quantify the damage degree after various well shut-in times. Then, displacement processes are simulated using lattice Boltzmann method (LBM) to evaluate the residual oil saturations and optimize the well shut-in time. The study suggests that the well shut-in process can cause irreversible damage to tight sandstone reservoir even for optimized fracture fluid. In the initial shut-in stages, clays swelling dominates pore structure alteration and reduces the porosity. Calcite will dissolute after which lead to slight porosity increase. In the flowback process after well shut-in, the simulated residual oil saturation will decrease initially and then increase after, which is complied with the porosity variation. The digital rock technology combined with the soaking experiments will provide alternative method for the evaluation of formation damage and the optimization of well shut-in time in tight sandstone reservoir, which can guide the selection of the fracture fluids and onsite fracturing operation.