The ongoing Industry 4.0 is characterized by the connectivity between components in the manufacturing system. For modern machines, the Internet of Things is a built-in function. In contrast, there are legacy machines in deployment functioning without digital communication. The need to connect them became popular to improve overall production efficiency. As building a new smart factory as a greenfield investment is a capital-intensive choice, retrofitting the existing infrastructure with IoT capability is more reasonable than replacing them. However, this so-called brownfield development, or retrofitting, requires specific prerequisites, e.g., digitization status assessment, technical and connectivity development, management requirement, and operational need, representing a significant disadvantage: lack of scalability. In the meantime, Industry 5.0 is under human-centric priority, which poses new challenges to the retrofitted system. Aware of the challenge, this paper provides a systematic overview of brownfield development regarding technical difficulties, supporting technologies, and possible applications for the legacy system. The research scope focuses on available Industry 4.0 advancements but considers preparing for the forthcoming Industry 5.0. The proposed retrofitting project approach can be a guideline for manufacturers to transform their factories into intelligent spaces with minimal cost and effort but still gain the most applicable solution for management needs. The future direction for other research in brownfield development for Industry 5.0 is also discussed.