<p><span lang="EN-US">Currently, the whole world is fighting a very dangerous and infectious disease caused by the novel coronavirus, called COVID-19. The COVID-19 is rapidly spreading around the world due to its high infection rate. Therefore, early discovery of COVID-19 is crucial to better treat the infected person as well as to slow down the spread of this virus. However, the current solution for detecting COVID-19 cases including the PCR test, CT images, epidemiologically history, and clinical symptoms suffer from high false positive. To overcome this problem, we have developed a novel transfer deep learning approach for detecting COVID-19 based on x-ray images. Our approach helps medical staff in determining if a patient is normal, has COVID-19, or other pneumonia. Our approach relies on pre-trained models including Inception-V3, Xception, and MobileNet to perform two tasks: i) binary classification to determine if a person infected with COVID-19 or not and ii) a multi-task classification problem to distinguish normal, COVID-19, and pneumonia cases. Our experimental results on a large dataset show that the F1-score is 100% in the first task and 97.66 in the second task.</span></p>