In order to improve surface polishing quality and efficiency for hard and brittle components, a novel nozzle with specifically designed shroud was proposed for an abrasive jet polishing process. The removal mechanism of the abrasive jet under such a nozzle was investigated by simulating the jet flow in the interaction area of the nozzle shroud and workpiece. The simulation results show that the speed of the abrasive jet increases greatly by the shroud and the direction of the jet is aligned near parallel to the workpiece surface to minimize impact damage to workpiece surface. The constrained abrasive jet polishing (CAJP) experiments were conducted on the quartz glass component, a typical hard and brittle material, showing that the material removal mainly relied on the shearing and scratching of the workpiece surface rather than the mechanical shock impacts, which is consistent with the simulation findings.