To enhance the accuracy of medical document classification, we propose an advanced deep fusion model for sorting medicine document. Specifically, we enhance text representation using the bidirectional encoder representation from transformers (BERT). BERT is a bidirectional model that considers context information in input sequences. This capability is particularly valuable for medical document, as medical information often requires understanding in a global context, such as diagnoses, medical history, and treatment plans. Furthermore, BERT can learn the semantics of words and phrases, comprehending the different meanings of the same word in distinct contexts, which is crucial for representing medical document. For example, In the context of cardiology, stroke often refers to a cerebrovascular accident, which is a condition where blood flow to the brain is disrupted, leading to neurological impairment. This type of stroke is related to the brain and is a significant concern in the field of cardiology due to its impact on the circulatory system. In dermatology, stroke might be used to refer to a type of skin condition, such as stroking the skin. However, this context is less common and not related to the cerebrovascular meaning. Subsequently, we employ both Convolutional Neural Network (ConvNet) and Bidirectional Long Short Term Memory (Bi-LSTM) to extract local features and global long-term dependencies, respectively. Their outputs are then fused to extract useful document features at multiple levels, effectively capturing the documental structure. The proposed deep fusion model leverages the complementary strengths of these components, enhancing the model's generalization ability and mitigating the risk of over-fitting. Ultimately, by comparing our approach with state-of-the-art methods in medical document classification, we demonstrate the effectiveness of the proposed methodology.
