Semi‐rigid steel frame stands out due to its remarkable deformation capability under seismic loading, while the introduction of partitioned walls would disturb the expected seismic behavior of the infilled steel frames. Furthermore, the presence of door openings might exacerbate these negative effects. To mitigate the detrimental multiple‐diagonal strut effect of traditional infill walls around the door opening, an innovative low‐damage wall panels system: prefabricated infill wall panels with a door opening and sliding joints are presented for semi‐rigid steel frames. For testifying the effectiveness of the proposed structure, a quasi‐static test was carried out on three half‐scaled steel frames: a bare frame, a frame infilled with conventional wall panels with a door opening, and a frame infilled with the novel wall panel system with a door opening. Experimental results reveal that the developed innovative wall panels with a door opening could avoid unfavorable crack developments and opening deformation around the door area. It is mainly attributed to the releasement of the adverse multiple‐diagonal strut mechanism of traditional wall panels with door openings. Thus, the presented infilled frame showcases seismic performance similar to the bare steel frame, while having a higher energy dissipation capacity. Moreover, the developed analytical prediction method based on the superposition principle could accurately estimate the lateral load‐carrying capacity of the structures. Due to negligible deformation of the both bottom infill wall panel and the door opening, the innovative infill wall only contributes about 10% lateral load‐carrying capability to the infilled frame.