Background: The aim of this study was to investigate the possible molecular mechanisms underlying cerebral small vessel disease caused by a missense mutation in the high-temperature serine peptidase A1 gene, HtrA1 (NM_002775.4, Exon4, c.905G>A, p.Arg302Gln). Stable strain models were constructed using wild-type and mutant HtrA1 overexpression lentiviral vectors to infect mouse brain microvascular endothelial cells (bEnd.3 cells). Methods: HtrA1 mRNA and protein expression were analyzed by Western blot and quantitative real-time polymerase chain reaction. Western blot technique was also used to evaluate the expression of transforming growth factor (TGF)-β/Smads-related signaling pathway proteins and the oxidative stress pathway protein nicotinamide adenine dinucleotide phosphate oxidase 4 (NOX4). The level of reactive oxygen species (ROS) was evaluated using dichloro-dihydro-fluorescein diacetate (DCFH-DA) fluorescent probes. Results: HtrA1 mRNA and protein expression levels were found to be decreased in mutant cells, whereas the levels of ROS, the TGF-β/Smads proteins, and the caspase3 and cleaved-caspase3 apoptotic proteins were increased. Conclusions: Lentivirus-mediated missense mutation in HtrA1 leads to activation of the TGF-β/Smads pathway and to increased apoptosis of mouse brain microvascular endothelial cells via the oxidative stress pathway. Further in vivo studies are required to explore the connections between different signaling pathways in animals, and to identify potential molecular targets for clinical therapy.