The predicted “Law 19” benchmark for HgCdTe photodiode performance established in 2019 is a milestone in the development of infrared detectors and make the dream of Elliott and colleagues, who in 1999 wrote that there is no fundamental obstacle to obtaining room temperature operation of photon detectors at room temperature with background-limited performance even in reduced fields of view [C.T. Elliott et al., Appl. Phys. Lett. 74(9), 2881 (1999)]. This circumstance will make it possible to achieve in the near future the room-temperature infrared (IR) arrays operation with high pixel density (small pixels) fully compatible with the background and diffraction-limited performance resulting from the system optics. The advent of smaller pixels also results in superior spatial and temperature resolutions of imaging systems. In megapixel imaging systems, the pixel dimension plays a crucial role in determining critical system attributes such as system size, weight, and power consumption (SWaP). In the paper, the physical limitations of pixel size related to the aperture of the optics, which in turn is wavelength dependent, are described. Since the critical parameter of small pixels is quantum efficiency, more attention has been paid to enhancing the coupling of radiation to the detector. Then, the evaluation for assessing the figure-of-merit of different material systems (especially short wavelength IR colloidal quantum dots, both medium and long wavelength IR novel III-V material systems) relative to bulk HgCdTe alloys is considered. Of the various thermal detectors, particular attention has been focused on bolometer arrays due to their largest share of the global commercial market. Also key challenges in realizing ultimate pixel dimensions in focal plane arrays (FPAs) design are presented for different material systems including dark current, pixel hybridization, pixel delineation, and unit cell readout capacity.