Displays that employ RGBW primaries have demonstrated greater power efficiency than similar displays with only RGB primaries. Unfortunately, RGBW systems with algorithms that use white to augment luminance generally have color layer noise affected the visual effect and need to be solved. Color layer noise visually shows the phenomenon of uneven color transition, showing a banded change. According to the cause of the color layer noise, we propose an algorithm to reduce the brightness difference between adjacent pixels in the location of color layer noise. Firstly, the gradient feature is used to mark the details in RGB space in order to distinguish the color layer from the texture details. Then the color layer is detected in RGBW space and the brightness of the color layer is smoothed to reduce the brightness difference of the adjacent pixels. The experiment on 6.5 inch RGBW TFT‐LCD screen show that the algorithm can effectively remove the color layer, and the details are lossless.