When certain polymers are heat-treated beyond their degradation temperature in the absence of oxygen, they pass through a semi-solid phase, followed by the loss of heteroatoms and the formation of a solid carbon material composed of a three-dimensional graphenic network, known as glassy (or glass-like) carbon. The thermochemical decomposition of polymers, or generally of any organic material, is defined as pyrolysis. Glassy carbon is used in various large-scale industrial applications and has proven its versatility in miniaturized devices. In this article, micro and nano-scale glassy carbon devices manufactured by (i) pyrolysis of specialized pre-patterned polymers and (ii) direct machining or etching of glassy carbon, with their respective applications, are reviewed. The prospects of the use of glassy carbon in the next-generation devices based on the material’s history and development, distinct features compared to other elemental carbon forms, and some large-scale processes that paved the way to the state-of-the-art, are evaluated. Selected support techniques such as the methods used for surface modification, and major characterization tools are briefly discussed. Barring historical aspects, this review mainly covers the advances in glassy carbon device research from the last five years (2013–2018). The goal is to provide a common platform to carbon material scientists, micro/nanomanufacturing experts, and microsystem engineers to stimulate glassy carbon device research.