Since the early 1990s, when researchers began to explore rare-earth-doped mid-infrared glass fibers, fiber laser systems have emerged as promising high-brightness light sources with wavelengths beyond 2.5 μm for applications in spectroscopy and sensing, optical communications and ranging, and processing of complex materials and bio-tissues, to name a few. Despite a substantial research effort over the years, mid-infrared fiber lasers and amplifiers have yet to reach the maturity required for widespread and/or industrial use. The well-known advantages of fiber lasers over their bulk counterparts, namely superior stability and beam quality, compactness, cost-efficiency, flexibility, and maintenance-free operation, can only be fully harnessed in the mid-infrared wavelength range with the development of non-existent yet essential fiber-based components made of advanced fluoride or chalcogenide-glass materials. This Perspective reports on the recent significant achievements that have been made in the design and fabrication of in-fiber and fiber-pigtailed components for fully integrated mid-infrared fiber laser systems. Building upon a comprehensive overview of the mechanical, thermodynamic, and optical properties of fluoride and chalcogenide glass fibers, as well as their interaction with light, we aim to highlight current challenges and opportunities and provide an informed forecast of future advancements in mid-infrared all-fiber laser research.