We proposed a novel optical technique to monitor miniature spindle runout in a simple manner. Miniature spindles are commonly used in many machining applications, for example: micro-milling and micro-grinding. However, the capacitive sensors (CS) or eddy current sensors typically used for spindle runout measurements cannot be used for miniature spindle systems. This is due to the nonlinearity of the charge between a curved surface and a flat plate (sensing area) and a curved surface (measuring target area) and the effective sensing area being larger than the measuring target area. The proposed sensor utilizes curved-edge diffraction (CED), which uses the effect of the cylindrical surface curvature on the diffraction phenomenon in the transition regions adjacent to shadow, transmission, and reflection boundaries. The laser beam is incident to the spindle shaft edges along the Y and Z axes, four photodetectors then collect the total fields produced by the interference created by the waves due to CED around the spindle shaft edges. Two CS were used as a baseline comparison with the proposed sensor's performance. A spindle with a shaft diameter of ϕ 5.0 mm (same as CS effective sensing area) was selected to compare the results of the curved-edge sensor (CES) with the results of the CS. The spindle runout was measured and the following results were found: CES-CS calibration nonlinearity (Z 0.35% and Y 0.40%) and resolution (Z 20.1 nm and Y 26.0 nm for CS and Z 20.3 nm and Y 15.9 nm for CES). The fundamental sensing limit of CES was estimated to be: Z 0.52 nm Hz / and Y 0.41 nm Hz / for a working range of approximately 100 µm, respectively.