Context. Increasing the statistics of spectroscopically confirmed evolved massive stars in the Local Group enables the investigation of the mass loss phenomena that occur in these stars in the late stages of their evolution. Aims. We aim to complete the census of luminous mid-IR sources in star-forming dwarf irregular (dIrr) galaxies of the Local Group. To achieve this we employed mid-IR photometric selection criteria to identify evolved massive stars, such as red supergiants (RSGs) and luminous blue variables (LBVs), by using the fact that these types of stars have infrared excess due to dust. Methods. The method is based on 3.6 μm and 4.5 μm photometry from archival Spitzer Space Telescope images of nearby galaxies. We applied our criteria to four dIrr galaxies: Pegasus, Phoenix, Sextans A, and WLM, selecting 79 point sources that we observed with the VLT/FORS2 spectrograph in multi-object spectroscopy mode. Results. We identified 13 RSGs, of which 6 are new discoveries, as well as two new emission line stars, and one candidate yellow supergiant. Among the other observed objects we identified carbon stars, foreground giants, and background objects, such as a quasar and an early-type galaxy that contaminate our survey. We use the results of our spectroscopic survey to revise the mid-IR and optical selection criteria for identifying RSGs from photometric measurements. The optical selection criteria are more efficient in separating extragalactic RSGs from foreground giants than mid-IR selection criteria, but the mid-IR selection criteria are useful for identifying dusty stars in the Local Group. This work serves as a basis for further investigation of the newly discovered dusty massive stars and their host galaxies.