Novel citrate/FeCO3 nanocomposites (CF-NCs) were synthesized for effective arsenic (III and V) sorption with constant addition of Fe2+ into HCO3− solution in the presence of citrate. This paper is the first report on the formation of CF-NCs, and in this study we investigate the mechanisms of arsenic uptake by the sorbent under anoxic conditions through various solid- and liquid-phase spectroscopic methods, including X-ray absorption spectroscopy. In CF-NCs, citrate was found to be incorporated into the structure of siderite (up to 17.94%) through (Fe2+citrate)− complexes. The crystal morphology of rhombohedral siderite was changed into hierarchically nanostructured spherical aggregates composed of several sheet-like crystals, which improved the surface reactivity in the presence of sufficient citrate. Compared to pure siderite (15.2%), enhanced removal of As(III) in the range of 19.3% to 88.2% was observed, depending on the amount of incorporated citrate. The maximum sorption capacities of CF-NCs for As(III) and As(V) were 188.97 and 290.22 mg/g, respectively, which are much higher than those of previously reported siderite-based adsorbents. It was found that arsenic (III and V) sorption on CF-NCs occurred via bidentate corner-sharing surface complexation, predominantly without changes in the arsenic oxidation states. These results suggest that arsenic (III and V) can be attenuated by siderite in anoxic environments, and this attenuation can be even more effective when siderite is modified by incorporation of organic compounds such as citrate.