As the world faces the COVID-19 pandemic, Artificial Intelligence, in particular, Deep Learning (DL) have been called up for help. Several recent research papers have shown the usefulness of these techniques for COVID-19 screening in Chest X-Rays (CXRs). To make this technology accessible and easy to use for the healthcare workers a natural path is to embed it into a mobile app. In these cases, however, the DL models must be prepared to receive as inputs pictures taken with the smartphones.Trying to raise awareness about the limitations of these models in a real-world setup, in this work, a dataset of CXR pictures taken of computer monitors with smartphones is built and DL models are evaluated on it. The results show that the current models are not able to correctly classify this kind of input. In the tested setup, augmenting the dataset with such pictures has shown to mitigate the problem, but it was not enough to raise accuracy to acceptable levels. As an alternative, this work shows that it is possible to build a model that discards pictures of monitors such that the COVID-19 screening module does not have to cope with them.