Building mass dampers (BMDs) are structures designed to mitigate the responses of super tall buildings to earthquakes. These BMDs contain a soft layer at relatively high points in such buildings, and the component above the soft layer acts as a tuned mass damper (TMD). Given that BMDs utilize a part of the buildings as the tuned mass, a large mass ratio can be yielded even in the case of super tall buildings. Therefore, BMDs are expected to demonstrate very good seismic performance against large earthquakes. However, for BMDs to be applicable to these buildings, the initial stiffness of the soft layer must be increased from the optimum level, such that excessive displacement of the layer owing to strong winds can be prevented. This stiffness is larger than the optimum stiffness applicable in the case of small and medium earthquakes. Therefore, a problem arises in that the seismic performance of these BMDs is reduced against small and medium earthquakes that occur more frequently than the large ones. From this standpoint, we installed an active mass damper (AMD) on top of a non-optimal BMD, in which the soft layer's stiffness and damping were increased above the optimal levels. We also designed this AMD such that the seismic performance of the non-optimal BMD was close to that of the optimal BMD, and the habitability during small and medium earthquakes was improved, while explicitly considering the constraints of the AMD through model-referenced predictive control. Ultimately, to demonstrate the effectiveness of the proposed method, we conducted a numerical analysis using input waves with various frequency components.