Ancient civilizations have imprinted their legacy on Zanzibar Stone Town through the construction of revered stone masonry buildings, which are experiencing rapid deterioration due to severe ambient environmental impacts. In response to these challenges, this study presents a comprehensive field exploration through the ambient vibration test (AVT) and numerical prediction of historical stone masonry buildings in Zanzibar Stone Town to analyze the dynamic characteristics. The ambient vibration test (AVT) reveals structural dynamic properties in terms of peak resonance frequencies (fr(avg)), mode shape, and damping ratio, in conjunction with the development of correlation with the geometric parameters of the building. The results reveal fr(avg) ranges between 2.8 and 5.3 Hz for investigated structures, non-uniform deformed mode shapes, and damping ratio ranges between 1.35% and 4.45% at various orientation axes of the understudy buildings. However, the relationship between natural frequencies indicates a higher association with the geometrical parameters of the building, yielding a correlation coefficient (R2) between 0.85 and 0.99. Moreover, the numerical prediction via eigenvalue analysis (EVA) yields a considerable association with the investigated data, quantified by root mean square error (RMSE), mean absolute error (MAE) ranged between 0.29 and 0.3, with Nash–Sutcliffe efficiency (NSE) and R2 between 0.81 and 0.99, respectively. Furthermore, conservation work guidelines were also developed to assist the structural engineer and conservationist in adopting targeted conservation strategies for the efficient preservation of the historical integrity in Stone Town.