The continuing effect of COVID-19 pulmonary infection has highlighted the importance of machine-aided diagnosis for its initial symptoms such as fever, dry cough, fatigue, and dyspnea. This paper attempts to address the respiratory-related symptoms, using a low power scalable software and hardware framework. We propose CoughNet, a flexible low power CNN-LSTM processor that can take audio recordings as input to detect cough sounds in audio recordings. We analyze the three different publicly available datasets and use those as part of our evaluation to detect cough sound in audio recordings. We perform windowing and hyperparameter optimization on the software side with regard to fitting the network architecture to the hardware system. A scalable hardware prototype is designed to handle different numbers of processing engines and flexible bitwidth using Verilog HDL on Xilinx Kintex-7 160t FPGA. The proposed implementation of hardware has a low power consumption of o 290 mW and energy consumption of 2 mJ which is about 99 × less compared to the state-of-the-art implementation.