Silicon-based photodetectors are attractive as low-cost and environmentally friendly optical sensors. Also, their compatibility with complementary metal-oxide-semiconductor (CMOS) technology is advantageous for the development of silicon photonics systems. However, extending optical responsivity of silicon-based photodetectors to the mid-infrared (mid-IR) wavelength range remains challenging. In developing mid-IR infrared Schottky detectors, nanoscale metals are critical. Nonetheless, one key factor is the Fermi-level pinning effect at the metal/silicon interface and the presence of metal-induced gap states (MIGS). Here, we demonstrate the utilization of the passivated surface layer on semiconductor materials as an insulating material in metal-insulator-semiconductor (MIS) contacts to mitigate the Fermi-level pinning effect. The removal of Fermi-level pinning effectively reduces the Schottky barrier height by 12.5% to 16%. The demonstrated devices exhibit a high responsivity of up to 234 μA/W at a wavelength of 2 μm, 48.2 μA/W at 3 μm, and 1.75 μA/W at 6 μm. The corresponding detectivities at 2 and 3 μm are 1.17 × 108 cm Hz1/2 W−1 and 2.41 × 107 cm Hz1/2 W−1, respectively. The expanded sensing wavelength range contributes to the application development of future silicon photonics integration platforms.