Organic photodetectors (OPDs) exhibit superior spectral responses but slower photoresponse times compared to inorganic counterparts. Herein, we study the light-intensity-dependent OPD photoresponse time with two small-molecule donors (planar MPTA or twisted NP-SA) co-evaporated with C60 acceptors. MPTA:C60 exhibits the fastest response time at high-light intensities (>0.5 mW/cm2), attributed to its planar structure favoring strong intermolecular interactions. However, this blend exhibits the slowest response at low-light intensities, which is correlated with biphasic photocurrent transients indicative of the presence of a low density of deep trap states. Optical, structural, and energetical analyses indicate that MPTA molecular packing is strongly disrupted by C60, resulting in a larger (370 meV) HOMO level shift. This results in greater energetic inhomogeneity including possible MPTA-C60 adduct formation, leading to deep trap states which limit the low-light photoresponse time. This work provides important insights into the small molecule design rules critical for low charge-trapping and high-speed OPD applications.