The rising rates of costs in labor and growth of computerization in fabric industries made defect detection in fabric a promising domain. For a huge time, manual discovery is extensively utilized in textile industries by trained staff that results in high cost. Meanwhile, the strict quality assessment is done by modern textile industries, which made automatic fabric defect detection a reliable choice. Since defect detection is an important and challenging aspect of modern industrial manufacturing, it is necessary to determine the quality and acceptability of garments and to reduce the cost and time waste caused by defects. Different methods are in practice for effective detection of fabric defects, however, they limit due to many reasons. Thus we proposed a new method named Competitive Cat Swarm Optimizer (CCSO) based Deep neuro-fuzzy network (DNFN) for effective fabric detection. Here, the pre-processing is performed with a median filter for eliminating noise contained in the image. Furthermore, features are extracted that involves Tetrolet transform-based features, statistical features, like energy, entropy, homogeneity, contrast, correlation, and texture feature, like Local gradient pattern. The data augmentation is carried out based on obtained features to make it apposite for processing. The defect detection is carried out using RideNN and DNFN. Here, the training of DNFN is done using the proposed CCSO, which is devised by combining Cat Swarm Optimization and Competitive Swarm Optimizer. The correlation is performed on the outputs of RideNN and DNFN for generating the final output of fabric defect detection. The performance of the proposed CCSO-based DNFN is compared with the different existing methods and the proposed CCSO-based DNFN outperforms with the highest specificity of 0.920, the accuracy of 0.919, and sensitivity of 0.916.