Inflammatory orbital lesions include a broad list of diagnoses, many of them with overlapping clinical and radiographic features. They often present a diagnostic conundrum, even to the most experienced orbital specialist, thus placing considerable weight on surgical biopsy and histopathological analysis. However, histopathological diagnosis is also inherently challenging due to the rarity of these lesions and the overlaps in histologic appearance among distinct disease entities. We herein present the case of an adolescent male with a subacutely progressive orbital mass that generated a significant diagnostic dilemma. Early orbital biopsy was consistent with a benign fibro-inflammatory lesion, but corticosteroid therapy was ineffective in halting disease progression. After an initial substantial surgical debulking, histopathological analysis revealed several key features consistent with IgG4-related disease (IgG4-RD), a systemic fibro-inflammatory process typically accompanied by multifocal tumor-like lesions. Surprisingly, within months, there was clear evidence of clinical and radiographic disease progression despite second-line rituximab treatment, prompting a second surgical debulking. This final specimen displayed distinctive features of Rosai-Dorfman disease (RDD), a systemic inflammatory disease characterized by uncontrolled histiocytic proliferation. Interestingly, certain features of this re-excision specimen were still reminiscent of IgG4-RD, which not only reflects the difficulty in differentiating RDD from IgG4-RD in select cases, but also illustrates that these diagnoses may exist along a spectrum that likely reflects a common underlying pathogenetic mechanism. This case emphasizes the importance of surgical biopsy or resection and histopathological analysis in diagnosing—and, ultimately, treating—rare, systemic inflammatory diseases involving the orbit, and, furthermore, highlights the shared histopathological features between RDD and IgG4-RD.