Metallicity is a key parameter that controls many aspects in the formation and evolution of stars and galaxies. In this review we focus on the metal deficient galaxies, in particular the most metal-poor ones, because they play a crucial rôle in the cosmic scenery. We first set the stage by discussing the difficult problem of defining a global metallicity and how this quantity can be measured for a given galaxy. The mechanisms that control the metallicity in a galaxy are reviewed in detail and involve many aspects of modern astrophysics: galaxy formation and evolution, massive star formation, stellar winds, chemical yields, outflows and inflows etc. Because metallicity roughly scales as the galactic mass, it is among the dwarfs that the most metalpoor galaxies are found. The core of our paper reviews the considerable progress made in our understanding of the properties and the physical processes that are at work in these objects. The question on how they are related and may evolve from one class of objects to another is discussed. While discussing metal-poor galaxies in general, we present a more detailed discussion of a few very metal-poor blue compact dwarf galaxies like IZw18. Although most of what is known relates to our local universe, we show that it pertains to our quest for primeval galaxies and is connected to the question of the origin of structure in the universe. We discuss what QSO absorption lines and known distant galaxies tell us already? We illustrate the importance of star-forming metal-poor galaxies for the determination of the primordial helium abundance, their use as distance indicator and discuss the possibility to detect nearly metal-free galaxies at high redshift from Lyα emission.