During the sintering process in the shaft kiln of refractory factories, a considerable quantity of materials is separated and dumped as Waste Magnesite (WM). The other superior grades, Lightly-Calcined Magnesite (LCM) and Dead-Burned Magnesite (DBM), are separated at different temperatures from the shaft kiln. The WM materials still have 8% of magnesium with some sand and dust particles in huge ranges. These materials are primarily used in the applications of medicine and fertilizers and animal feed processing and additives. This work investigates the potential of WM particles as reinforcement materials in the Al 6061 alloy matrix. In order to compare the characteristics of WM-based composite, LCM and DBM powders were also considered as reinforcement particles. The WM, LCM, and DBM particles were mixed with a 15% weight ratio to Al 6061 alloy, and composites were fabricated using the Stir Casting method. The surface morphology investigations through Scanning Electron Microscopy (SEM) revealed that these particles were well distributed and dispersed within the alloy matrix and with good interfacial adhesion. It is noted that Al 6061/15% wt. LCM composite possesses a better tensile strength than Al 6061/15% wt. WM and Al 6061/15% wt.DBM composites. The impact value produced by the WM-based composite is better than the LCM and DBM composites. The Wear Rate and Coefficient of Friction (COF) were examined through a Pin-on-Disc apparatus. Al 6061/15% wt. WM composite tested a low wear rate (9.74 × 10−6 mm3 m)−1. The results show that Al 6061/15% wt. LCM composite achieved the least COF value of 0.681 at an applied load of 0.5 Kg and a sliding distance of 2826 m. The results prove that the wastage magnesite is good enough and the most robust reinforcement material for Aluminum 6061 alloy-based composite.