The novel coronavirus (COVID-19) was first reported in the Wuhan City of China in 2019 and became a pandemic. The outbreak has caused shocking effects to the people across the globe. It is important to screen a majority of the population in every country and for the respective governments to take appropriate action. There is a need for a rapid screening system to triage and recommend the patients for appropriate treatment. Chest X-ray imaging is one of the potential modalities, which has ample advantages such as wide availability even in the villages, portability, fast data sharing option from the point of capturing to the point of investigation, etc. The aim of the proposed work is to develop a deep learning algorithm for screening COVID-19 cases by leveraging the widely available X-ray imaging. We have built a deep learning Convolutional Neural Network model utilizing a combination of the public domain (open-source COVID-19) and private data (pneumonia and normal cases). The dataset was used before and after the segmentation of the lung region for training and testing. The outcome of the classification after lung segmentation resulted in significant superiority. The average accuracy achieved by the proposed system was 96%. The heat maps incorporated in the system were helpful for our radiologists to cross-verify whether the appropriate features are identified. This system (COVID-Detect) can be used in remote places in the countries affected by COVID-19 for mass screening of suspected cases and suggesting appropriate actions, such as recommending confirmatory tests.