It is suggested that programming of the immune system starts before birth and is shaped by environmental influences acting during critical windows of susceptibility for human development. Prenatal and perinatal exposure to physiological, biological, physical, or chemical factors can trigger permanent, irreversible changes to the developing immune system, which may be reflected in cord blood of neonates. The aim of this narrative review is to summarize the evidence on the role of the prenatal and perinatal environment, including season of birth, mode of delivery, exposure to common allergens, a farming environment, pet ownership, and exposure to tobacco smoking and pollutants, in shaping the immune cell populations and cytokines at birth in humans. We also discuss how reported disruptions in the immune system at birth might contribute to the development of asthma and related allergic manifestations later in life.