Pulmonary aspergillosis is a severe infectious disease caused by some members of the Aspergillus genus, that affects immunocompetent as well as immunocompromised patients. Among the different disease forms, Invasive Aspergillosis is the one causing the highest mortality, mainly, although not exclusively, affecting neutropenic patients. This genus is very well known by humans, since different sectors like pharmaceutical or food industry have taken advantage of the biological activity of some molecules synthetized by the fungus, known as secondary metabolites, including statins, antibiotics, fermentative compounds or colorants among others. However, during infection, in response to a hostile host environment, the fungal secondary metabolism is activated, producing different virulence factors to increase its survival chances. Some of these factors also contribute to fungal dissemination and invasion of adjacent and distant organs. Among the different secondary metabolites produced by Aspergillus spp. Gliotoxin (GT) is the best known and better characterized virulence factor. It is able to generate reactive oxygen species (ROS) due to the disulfide bridge present in its structure. It also presents immunosuppressive activity related with its ability to kill mammalian cells and/or inactivate critical immune signaling pathways like NFkB. In this comprehensive review, we will briefly give an overview of the lung immune response against Aspergillus as a preface to analyse the effect of different secondary metabolites on the host immune response, with a special attention to GT. We will discuss the results reported in the literature on the context of the animal models employed to analyse the role of GT as virulence factor, which is expected to greatly depend on the immune status of the host: why should you hide when nobody is seeking for you? Finally, GT immunosuppressive activity will be related with different human diseases predisposing to invasive aspergillosis in order to have a global view on the potential of GT to be used as a target to treat IA.