Herein, we have attempted to recycle the waste gypsum to be used as filler material in polymer composite. In this regard, aqueous dispersion of gypsum was ultrasonicated for varying time duration (10, 30, and 60 min), dried and used as fillers to develop corresponding EPDM composites. As per the mechanical performance (i.e., tensile strength and elongation at break), the gypsum obtained after ultrasonication for 60 min (i.e., U-60 gypsum) was found best performing filler material compared to the other ultrasonicated gypsum samples, commercial carbon black, and another surface treated gypsum sample. The decreased particle size and enhanced surface area of U-60 gypsum are supposed to be responsible for improving the mechanical property of the composite. Corresponding particle size, surface area, and morphology of all the gypsum samples were thoroughly examined to conclude the reason behind such observation.