Insufficient studies of reflectance and fluffiness effects on optical density caused large derivation and limited the application of fibrogram methods in colored cotton fiber length tests for decades. A coupled optical method was proposed here to solve this issue, named reflectivity and fluffiness coupled optical algorithm of the random-beard image method. In this algorithm, a defined theoretical reflectance was declared with an iteration method for optical linear density on the optical analysis of the derived Kubelka–Munk law, which satisfied its assumption of excluding reflected light at the interface with air. Meanwhile, an apparent polynomial relationship was revealed between the actual density and optical densities of fiber assemblies at different fluffiness. Its accuracy on weighted mean fiber length was demonstrated, referred to as advanced fiber information system, using 10 kinds of dyed cotton slivers and 12 kinds of naturally colored lint fibers whose mean and maximum absolute errors are 0.59 mm and 1.6 mm, respectively. Furthermore, their excellent agreement on fibrograms and the weighted mean length was confirmed by Bland–Altman analysis, correlation analysis and the T-test. Finally, an exceptionally high linear correlation (r2 = 0.93) was achieved on their weighted mean length and had no statistical difference with insignificant deviation under a significance level of a = 0.01. This new algorithm, featured fast and low cost, could provide accurate mean fiber length of colored or dark cotton, exhibiting valuable guidance in the potential application of the fiber trade and the quality control of colored cotton fibers or primaries in blending assemblies and yarn.