Gene encoding the large subunit of As(III) oxidase (AioA), an important component of the microbial As(III) oxidation system, is a widely used biomarker to characterize As(III)-oxidizing communities in the environment. However, many studies were restricted to a few sequences generated by clone libraries and Sanger sequencing, which may have underestimated the diversity of As(III)-oxidizers in natural environments. In this study, we designed a primer pair, 1109F (5′-ATC TGG GGB AAY RAC AAY TA−3′) and 1548R (5′-TTC ATB GAS GTS AGR TTC AT−3′), targeting gene sequence encoding for the conserved molybdopterin center of the AioA protein, yielding amplicons approximately 450 bp in size that are feasible for highly parallel amplicon sequencing. By utilizing in silico analyses and the experimental construction of clone libraries using Sanger sequencing, the specificity and resolution of 1109F/1548R are approximated with two other previously published and commonly used primers, i.e., M1-2F/M3-2R and deg1F/deg1R. With the use of the 1109F/1548R primer pair, the taxonomic composition of the aioA genes was similar both according to the Sanger and next-generation sequencing (NGS) platforms. Furthermore, high-throughput amplicon sequencing using the primer pair, 1109F/1548R, successfully identified the well-known As(III)-oxidizers in paddy soils and sediments, and they also revealed the differences in the community structure and composition of As(III)-oxidizers in above two biotopes. The random forest analysis showed that the dissolved As(III) had the highest relative influence on the Chao1 index of the aioA genes. These observations demonstrate that the newly designed PCR primers enhanced the ability to detect the diversity of aioA-encoding microorganisms in environments using highly parallel short amplicon sequencing.