Context. There are a number of very high energy sources in the Galaxy that remain unidentified. Multi-wavelength and variability studies, and catalogue searches, are powerful tools to identify the physical counterpart, given the uncertainty in the source location and extension. Aims. This work carries out a thorough multi-wavelength study of the unidentified, very high energy source HESS J1858+020 and its environs. Methods. We have performed Giant Metrewave Radio Telescope observations at 610 MHz and 1.4 GHz to obtain a deep, low-frequency radio image of the region surrounding HESS J1858+020. We analysed archival radio, infrared, and X-ray data as well. This observational information, combined with molecular data, catalogue sources, and a nearby Fermi gamma-ray detection of unidentified origin, are combined to explore possible counterparts to the very high energy source. Results. We provide with a deep radio image of a supernova remnant that might be related to the GeV and TeV emission in the region.We confirm the presence of an H ii region next to the supernova remnant and coincident with molecular emission. A potential region of star formation is also identified. We identify several radio and X-ray sources in the surroundings. Some of these sources are known planetary nebulae, whereas others may be non-thermal extended emitters and embedded young stellar objects. Three old, background Galactic pulsars also neighbour HESS J1858+020 along the line of sight. Conclusions. The region surrounding HESS J1858+020 is rich in molecular structures and non-thermal objects that may potentially be linked to this unidentified very high energy source. In particular, a supernova remnant interacting with nearby molecular clouds may be a good candidate, but a star forming region, or a non-thermal radio source of yet unclear nature, may also be behind the gamma-ray source. The neighbouring pulsars, despite being old and distant, cannot be discarded as candidates. Further observational studies are needed, however, to narrow the search for a counterpart to the HESS source.