Deep sea is a vast, dark, and difficult-to-access terrain and is now looked upon as a unique niche harboring diverse microorganism. We used a metataxonomic approach to decipher the microbial diversity present in the water column (surface to near bottom), water overlaying the sediments, and the deep-sea sediments (up to 35 cm) from the Indian Contract Region (ICR) in the Central Indian Ocean Basin (CIOB). Samples were collected from #IRZ (Impact Reference Zone), #PRZ (Potential Reference Zone), and #BC20 (Control site, outside potential mining area) with an average water depth of 5,200 m. 16S rRNA (V3–V4 region) amplicon sequencing on the MiSeq platform resulted in 942,851 ASVs across 65 water and sediment samples. Higher prokaryotic diversity was observed below 200 m in the water column to the seafloor. Proteobacteria was the most dominant bacterial phylum among all the water samples while Firmicutes, Actinobacteria and, Bacteroidota dominated the sediments. Sediment (below 10 cm) was co-dominated by Firmicutes. Thermoplasmata was the dominant archaeal group in the water column while Crenarchaeota was in the sediments. BC20 was less diverse than IRZ and PRZ. Deep Sea microorganisms could play a vital role in the mineralization processes, nutrient cycling, and also different biogeochemical cycles.