Toll-like receptor (TLR) 4 belongs to the TLR family of receptors inducing pro-inflammatory responses to invading pathogens. TLR4 is activated by lipopolysaccharide (LPS, endotoxin) of Gram-negative bacteria and sequentially triggers two signaling cascades: the first one involving TIRAP and MyD88 adaptor proteins is induced in the plasma membrane, whereas the second engaging adaptor proteins TRAM and TRIF begins in early endosomes after endocytosis of the receptor. The LPS-induced internalization of TLR4 and hence also the activation of the TRIF-dependent pathway is governed by a GPI-anchored protein, CD14. The endocytosis of TLR4 terminates the MyD88-dependent signaling, while the following endosome maturation and lysosomal degradation of TLR4 determine the duration and magnitude of the TRIF-dependent one. Alternatively, TLR4 may return to the plasma membrane, which process is still poorly understood. Therefore, the course of the LPS-induced pro-inflammatory responses depends strictly on the rates of TLR4 endocytosis and trafficking through the endo-lysosomal compartment. Notably, prolonged activation of TLR4 is linked with several hereditary human diseases, neurodegeneration and also with autoimmune diseases and cancer. Recent studies have provided ample data on the role of diverse proteins regulating the functions of early, late, and recycling endosomes in the TLR4-induced inflammation caused by LPS or phagocytosis of E. coli. In this review, we focus on the mechanisms of the internalization and intracellular trafficking of TLR4 and CD14, and also of LPS, in immune cells and discuss how dysregulation of the endo-lysosomal compartment contributes to the development of diverse human diseases.