Infrared (IR) photodetection is important for light communications, military, agriculture, and related fields. Organic transistors are investigated as photodetectors. However, due to their large band gap, most organic transistors can only respond to ultraviolet and visible light. Here high performance IR phototransistors with ternary semiconductors of organic donor/acceptor complex and semiconducting single‐walled carbon nanotubes (SWCNTs), without deep cooling requirements are developed. Due to both the ultralow intermolecular electronic transition energy of the complex and charge transport properties of SWCNTs, the phototransistor realizes broadband photodetection with photoresponse up to 2600 nm. Moreover, it exhibits outstanding performance under 2000 nm light with photoresponsivity of 2.75 × 106 A W−1, detectivity of 3.12 × 1014 Jones, external quantum efficiency over 108%, and high Iphoto/Idark ratio of 6.8 × 105. The device exhibits decent photoresponse to IR light even under ultra‐weak light intensity of 100 nW cm−2. The response of the phototransistor to blackbody irradiation is demonstrated, which is rarely reported for organic phototransistors. Interestingly, under visible light, the device can also be employed as synaptic devices, and important basic functions are realized. This strategy provides a new guide for developing high performance IR optoelectronics based on organic transistors.