Detection of yarns in fabric images is a basic task in real-time monitoring in fabric production processes since it relates to yarn density and fabric structure estimation. In this paper, a new detection method is proposed that can automatically and efficiently estimate the locations as well as the numbers of both weft and warp yarn in fabric images. The method has three sequential phases. First, the modulus of discrete partial derivatives at each pixel is projected onto the weft and warp directions to generate the accumulated histograms. Second, for each histogram, a monotone hypothesis of a nonparametric statistical approach is applied to segment the histogram. Third, according to the segmentation result, the locations of each weft and warp yarn are adaptively determined, while the fabric structure is also obtained. Numerical results demonstrate that, compared with classical yarn detection methods, which are based on image smoothing, the proposed method can estimate yarn locations and fabric structures with more accuracy, but also reduce the influence of yarn hairiness.