Terrestrial ecosystems have been largely regarded as plant-dominated land surfaces, with the earliest records appearing in the early Phanerozoic (<550 Ma). Yet the presence of biological components in pre-Phanerozoic rocks, in habitats as different as soils, peats, ponds, lakes, streams, and dune fields, implies a much earlier type of terrestrial ecosystems. Microbes were abundant by~3,500 Ma ago and surely adapted to live in subaerial conditions in coastal and inland environments, as they do today. This implies enormous capacities for rapid adaptations to changing conditions, which is supported by a suggestive fossil record. Yet, evidence of "terrestrial" microbes is rare and indirect in comparison with fossils from shallow or deeper marine environments, and its record has been largely overlooked. Consequently, the notion that microbial communities may have formed the earliest land ecosystems has not been widely accepted nor integrated into our general knowledge. Currently, an ample record of shallow marine and lacustrine biota in~3,500 Ma-old deposits, together with evidence of microbial colonization of coastal environments~3,450 Ma ago and indirect geochemical evidence that suggests biological activity in >3,400 Ma-old paleosols endorses the idea that life on land perhaps occurred in parallel with aquatic life back in the Paleoarchean. The rapid adaptations seen in modern terrestrial microbes, their outstanding tolerance to extreme and fluctuating conditions, their early and rapid diversification, and their old fossil record collectively suggest that they constituted the earliest terrestrial ecosystems, at least since the Neoarchean, further succeeding on land and forming a biomass-rich cover with mature soils where plant-dominated ecosystems later evolved. Understanding how life diversified and adapted to non-aquatic conditions from the actualistic and paleontological perspective is critical to understanding the impact of life on the Earth's systems over thousands of millions of years.