Optical convolution computing is gaining traction owing to its inherent parallelism, multi‐dimensional processing, and energy efficiency. To handle input dimensions of N, conventional implementations necessitate N2 optical elements, such as Mach–Zehnder interferometers or micro‐ring resonators, to process multiply‐accumulate (MAC) operations, limiting scalability and resulting in elevated power consumption. Here, a direct convolution computing method based on wavelength routing, utilizing the unique sliding property of an arrayed waveguide grating router (AWGR) to perform the sliding window operation of the convolution in the wavelength–space domains is proposed. With two input vectors directly loaded onto two modulator arrays, the convolution result is instantaneously produced at a photodetector array. The entire convolution computation is executed within a single clock cycle without the need for preprocessing or decomposition into elementary MAC operations. The number of active elements is minimal, only needed for input/output. The proposed optical convolution unit has striking advantages of high scalability, high speed, and processing simplicity compared to those based on optical matrix‐vector multipliers. In the first experimental demonstration, a remarkable classification accuracy of up to 98.2% in handwritten digit recognition tasks using a LeNet‐5 neural network is achieved.