Ciguatera poisoning (CP) results from the consumption of coral reef fish or marine invertebrates contaminated with potent marine polyether compounds, namely ciguatoxins. In French Polynesia, 220 fish specimens belonging to parrotfish (Chlorurus microrhinos, Scarus forsteni, and Scarus ghobban), surgeonfish (Naso lituratus), and groupers (Epinephelus polyphekadion) were collected from two sites with contrasted risk of CP, i.e., Kaukura Atoll versus Mangareva Island. Fish age and growth were assessed from otoliths’ yearly increments and their ciguatoxic status (negative, suspect, or positive) was evaluated by neuroblastoma cell-based assay. Using permutational multivariate analyses of variance, no significant differences in size and weight were found between negative and suspect specimens while positive specimens showed significantly greater size and weight particularly for E. polyphekadion and S. ghobban. However, eating small or low-weight specimens remains risky due to the high variability in size and weight of positive fish. Overall, no relationship could be evidenced between fish ciguatoxicity and age and growth characteristics. In conclusion, size, weight, age, and growth are not reliable determinants of fish ciguatoxicity which appears to be rather species and/or site-specific, although larger fish pose an increased risk of poisoning. Such findings have important implications in current CP risk management programs.