To establish diagnostic criteria using comparison of cell cluster shapes, between benign and malignant tumors, breast tumors demonstrating weak cellular atypia in low grade invasive ductal carcinoma (IDC) were compared. Fine-needle aspiration (FNA) specimens of breast tumors were obtained from 37 patients. Among these, 16 were histologically diagnosed as IDC low-grade and the other 21 as benign fibroadenoma (FA). For evaluation, we examined 740 clusters from these 37 FNA specimens. Nine image morphometric parameters were studied, including the cluster area, circumference, maximal length, maximal breadth, ratio of length to breadth, cluster roundness, cluster size, and the edge and distribution image fractal dimensions for cluster analysis. We evaluated the irregularity in cell cluster shape using fractal dimension analysis, and determined the correlation to cluster size. The irregularity in the IDC cluster shape was higher than that in the FA cluster shape. However, six cases (28.5%) of 21 FA clusters showed high fractal dimensions similar to those for IDC. The clusters were classified by cluster analysis into three types: IDC clusters, FA with irregular cluster shape, and FA with no irregular clusters. The average cell cluster area of the FA with irregular shape was found to be about three times larger than that of IDC clusters. When the differential diagnosis between IDC and FA is difficult, it is important to focus on irregularities in the shape and on overall size of the cell clusters. For accurate diagnosis, the cell cluster shape is as important as the individual cellular atypia.