During natural disaster, social media can provide real-time or low delayed disaster perception information to help government managers carry out disaster response efforts efficiently,therefore, it is of great significance to mining social media information accurately. Distinguished from previous studies, this study proposed a multi-modal data classification model for mining social media information. This model conducted LDA(Late Dirichlet Allocation) to identify subject information from multi-modal data and then analyzed by Bert (bidirectional encoder representation from transformers) and Vgg-16 (visual geometry group 16), text data and image data were classified separately, resulting in real mining of topic information during disasters. This paper used Weibo data during the 2021 Henan heavy storm as the research object, comparing with previous data experiment results, this study proposed a model that can make natural disaster related topic classification of social media data more accurately, resulting in a real-time understanding of different themed natural disasters, it was helpful in making disastrous decisions.