Various by-products generated from thermal power plants and chemical industries have considerably economic and environmental impacts in South Korea. This study focuses on evaluating thermal conductivity of controlled low strength material (CLSM) made entirely from by-products (e.g., coal ash, gypsum, red mud). In the experimental program, pond ash is used as a full replacement of natural sand whereas fly ash activated by a little lime, red mud, and gypsum is the main binder in the production of CLSM. Various laboratory tests including flowability, bleeding, initial setting time, and unconfined compressive strength were performed on the prepared CLSM mixtures to determine its general characteristics. Thermal conductivity is then measured subjected to saturated curing condition (SC) and room temperature curing condition (RTC). As a result, all general characteristics meet the specification of CLSM reported in ACI 229R by controlling the ratio of gypsum to red mud. In particular, the good flowability of higher than 20 cm is observed as the G/Rm ratio of smaller than 1.33. The bleeding values, ranging from 0.30% to 2.70%, fall into the bleeding requirement of CLSM of less than 5%. Moreover, the initial setting time and strength results are also in the acceptable specification of general CLSM in ACI 229R. Eventually, the thermal conductivity of the proposed CLSM was in the range of 0.84–0.87 (W/mK) and these values were considerably affected by the saturation states and curing conditions rather than binder proportion.