Plastic waste management is a critical concern in municipal solid waste management systems worldwide. Despite the efforts of waste management personnel to segregate waste manually, the existing challenges persist. In municipal waste facilities, individuals responsible for waste segregation face numerous obstacles. Consequently, a significant amount of plastic waste ends up in landfills, exacerbating the plastic waste problem. To overcome these challenges, this research focuses on developing an automated system capable of categorizing plastic waste based on its visual characteristics. The trained model exhibits high precision in identifying various types of plastic waste, including PET, HDPE, PVC, LDPE, PP, and PS. Specifically, the model achieves an Average Precision of 0.917 and an Average Recall of 0.801. Moreover, the model maintains a good balance between precision and recall. In real-time operation, an overhead camera locates the positions of both the waste items and the gripper. By calculating the positional difference between the waste and the gripper, the system achieves a higher level of segregation accuracy, resembling human-like hand-eye coordination. The proposed system offers a solution to the challenges faced in MSW facilities, where the timely segregation of waste is crucial. By automating the plastic waste categorization process, the system can significantly improve waste management practices, leading to a more sustainable approach to plastic waste disposal and recycling.