The power load prediction can ensure the power supply and dispatch, which will be useful for market participants to plan and make strategic decisions to enhance reliability, save operation and maintenance costs. Short-term load data series have obvious approximate periodicity, while long-term load data series show variability and dynamic features. In addition, time series data of various modalities, such as market reports and production management data, could play a role in load prediction. One kind of multi-modal CNN-BiLSTM architecture is proposed to predict short-term and long-term load data, which have an improved shared parameter convolutional network to learn feature representation and an improved attention-based BiLSTM mechanism, which could model the dynamic features of multimodal on time series data. Experimental results on multimodal dataset show that, compared with other baseline systems, this model has some advantages in the prediction accuracy.