We initially developed an efficient solver to study photodetectors composed of multiple semiconductor layers with varying thicknesses and doping concentrations. Subsequently, we employed it as the forward solver for three different numerical optimization methods aimed at designing Si-Ge photodetectors with larger bandwidth, higher quantum efficiency, and lower phase noise. Our work offers new insights into the design of high-performance photodetectors—a challenging task due to computation time, design constraints, and the complexity of estimating sensitivity to design parameters.