The rapid growth of the Internet of Things (IoT) has contributed to significant challenges in dealing with congestion within IoT communications due to high packet error rates, latency, and interference in networks. With an emphasis on the Constrained Application Protocol (CoAP), the present study aims to propose the design and development of a novel congestion control mechanism, namely, Fast Response Congestion Control—Random Early Detection, abbreviated as FaCoCo-RED, along with performance analysis and comparison of congestion management efficacy between FaCoCo-RED and Default CoAP Congestion Control (Default CoAP CC) under a Cooja simulator on the Contiki OS platform. The findings from both experiment and performance analysis, which were based on statistical testing, showed that, under medium-scale to large-scale node networks across all traffic scenarios in this study, FaCoCo-RED significantly outperformed Default CoAP CC. The improvement can be seen in such metrics as average throughput, packet loss, response time, settling time, and retransmission timeout values (RTOs). The experimental findings also showed that FaCoCo-RED can perform effectively within the IoT networks, thus potentially enhancing the reliability and scalability of CoAP for large-scale and more complex IoT applications in the future.