All hematopoietic and immune cells are continuously generated by hematopoietic stem cells (HSCs) and hematopoietic progenitor cells (HPCs) through highly organized process of stepwise lineage commitment. In the steady state, HSCs are mostly quiescent, while HPCs are actively proliferating and contributing to daily hematopoiesis. In response to hematopoietic challenges, e.g., life-threatening blood loss, infection, and inflammation, HSCs can be activated to proliferate and engage in blood formation. The HSC activation induced by hematopoietic demand is mediated by direct or indirect sensing mechanisms involving pattern recognition receptors or cytokine/chemokine receptors. In contrast to the hematopoietic challenges with obvious clinical symptoms, how the aging process, which involves low-grade chronic inflammation, impacts hematopoiesis remains undefined. Herein, we summarize recent findings pertaining to functional alternations of hematopoiesis, HSCs, and the bone marrow (BM) microenvironment during the processes of aging and inflammation and highlight some common cellular and molecular changes during the processes that influence hematopoiesis and its cells of origin, HSCs and HPCs, as well as the BM microenvironment. We also discuss how age-dependent alterations of the immune system lead to subclinical inflammatory states and how inflammatory signaling might be involved in hematopoietic aging. Our aim is to present evidence supporting the concept of “Inflamm-Aging,” or inflammation-associated aging of hematopoiesis.