This article provides an extensive survey of nonlinear parametric upconversion infrared (IR) imaging, starting from its origin to date. Upconversion imaging is a successful innovative technique for IR imaging in terms of sensitivity, speed, and noise performance. In this approach, the IR image is frequency upconverted to form a visible/near-IR image through parametric three-wave mixing followed by detection using a silicon-based detector or camera. In 1968 (50 years back), J. E. Midwinter first demonstrated upconversion imaging from shortwave-IR (1.6 μm) to visible (484 nm) wavelength using a bulk lithium niobate crystal. This technique quickly gained interest, and several other groups demonstrated upconversion imaging further into the mid-and far-IR with significantly improved quantum efficiency. Although a few excellent reviews on upconversion imaging were published in the early 1970's, the rapid progress in recent years merits an updated comprehensive review. The topic includes linear imaging, nonlinear optics, and laser science, and has shown diverse applications. The scope of this article is to provide an in-depth knowledge of upconversion imaging theory. An overview of different phase matching conditions for the parametric process and the sensitivity of the upconversion detection system are discussed. Furthermore, different design considerations and optimization schemes are outlined for application-specific upconversion imaging. The article comprises a historical perspective of the technique, its most recent technological advances, specific outstanding issues, and some cutting-edge applications of upconversion in IR imaging.