Recovering the wavelengths from disordered speckle patterns has become an exciting prospect as a wavelength measurement method due to its high resolution and simple design. In previous studies, panel cameras were often utilized as the speckle images receiver. However, high cost (especially at near infrared range), large size and low speed limit its application in optical communications, metrology and optical sensing. In this work, the speckle patterns were highly compressed into four intensities by using a quadrant detector (QD). And wavelengths still can be recovered through only four pixels instead of millions of pixels of a camera. A new CNN based demodulation algorithm, shallow residual network (SRN), was proposed to recognize the wavelengths from the highly compressed speckle images. Finally, a wavelength precision of 4 fm (~ 0.5 MHz) with an updating speed of ~ 1 kHz was achieved in the demonstrations. In addition, the SRN shows a broad measurement range and good noise robustness. Compared with camera based system, the QD detection scheme associated with the CNN algorithm provides a compact, high speed, and low cost method to examine the speckle patterns, which opens new routes in many other fields.