TLRs are distributed in their characteristic cellular or subcellular compartments to efficiently recognize specific ligands and to initiate intracellular signaling. Whereas TLRs recognizing pathogen-associated lipids or proteins are localized to the cell surface, nucleic acid–sensing TLRs are expressed in endosomes and lysosomes. Several endoplasmic reticulum (ER)–resident proteins are known to regulate the trafficking of TLRs to the specific cellular compartments, thus playing important roles in the initiation of innate immune responses. In this study, we show that an ER-resident protein, Nogo-B (or RTN4-B), is necessary for immune responses triggered by nucleic acid–sensing TLRs, and that a newly identified Nogo-B–binding protein (glucosyltransferases, Rab-like GTPase activators and myotubularins [GRAM] domain containing 4 [GRAMD4]) negatively regulates the responses. Production of inflammatory cytokines in vitro by macrophages stimulated with CpG-B oligonucleotides or polyinosinic:polycytidylic acid was attenuated in the absence of Nogo-B, which was also confirmed in serum samples from Nogo-deficient mice injected with polyinosinic:polycytidylic acid. Although a deficiency of Nogo-B did not change the incorporation or delivery of CpG to endosomes, the localization of TLR9 to endolysosomes was found to be impaired. We identified GRAMD4 as a downmodulator for TLR9 response with a Nogo-B binding ability in ER, because our knockdown and overexpression experiments indicated that GRAMD4 suppresses the TLR9 response and knockdown of Gramd4 strongly enhanced the response in the absence of Nogo-B. Our findings indicate a critical role of Nogo-B and GRAMD4 in trafficking of TLR9.