Chest X-ray images are widely used in clinical practice such as diagnosis and treatment. The automatic radiology report generation system can effectively reduce the rate of misdiagnosis and missed diagnosis. Previous studies were focused on the long text generation problem of image paragraph, ignoring the characteristics of the image and the auxiliary role of patient background information for diagnosis. In this paper, we propose a new hierarchical model with multi-attention considering the background information. The multi-attention mechanism can focus on the image's channel and spatial information simultaneously, and map it to the sentence topic. The patient's background information will be encoded by the neural network first, then it will be aggregated into a vector representation by a multi-layer perception and added to the pretrained vanilla word embedding, which finally forms a new word embedding after fusion. Our experimental results demonstrated that the model outperforms all baselines, achieving the state-of-the-art performance in terms of accuracy.

Making inference on clinical texts is a task which has not been fully studied. With the newly released, expert annotated MedNLI dataset, this task is being boosted. Compared with open domain data, clinical texts present unique linguistic phenomena, e.g., a large number of medical terms and abbreviations, different written forms for the same medical concept, which make inference much harder. Incorporating domain-specific knowledge is a way to eliminate this problem, in this paper, we assemble a new incorporating medical concept definitions module on the classic enhanced sequential inference model (ESIM), which first extracts the most relevant medical concept for each word, if it exists, then encodes the definition of this medical concept with a bidirectional long short-term network (BiLSTM) to obtain domain-specific definition representations, and attends these definition representations over vanilla word embeddings. The empirical evaluations are conducted to demonstrate that our model improves the prediction performance and achieves a high level of accuracy on the MedNLI dataset. Specifically, the knowledge enhanced word representations contribute significantly to entailment class. INDEX TERMS Attention mechanism, clinical text, medical domain knowledge, natural language inference, word representation.

The chest X-ray is a simple and economical medical aid for auxiliary diagnosis and therefore has become a routine item for residents' physical examinations. Based on 40 167 images of chest radiographs and corresponding reports, we explore the abnormality classification problem of chest X-rays by taking advantage of deep learning techniques. First of all, since the radiology reports are generally templatized by the aberrant physical regions, we propose an annotation method according to the abnormal part in the images. Second, building on a small number of reports that are manually annotated by professional radiologists, we employ the long short-term memory (LSTM) model to automatically annotate the remaining unlabeled data. The result shows that the precision value reaches 0.88 in accurately annotating images, the recall value reaches 0.85, and the F1-score reaches 0.86. Finally, we classify the abnormality in the chest X-rays by training convolutional neural networks, and the results show that the average AUC value reaches 0.835.INDEX TERMS Annotation, deep neural network, DenseNet, long short term memory.

No abstract

Computer-aided diagnosis (CAD) is an important work which can improve the working efficiency of physicians. With the availability of large-scale data sets, several methods have been proposed to classify pathology on chest X-ray images. However, most methods report performance based on a frontal chest radiograph, ignoring the effect of the lateral chest radiography on the diagnosis. This paper puts forward a kind of model, Dual-Ray Net, of a deep convolutional neural network which can deal with the front and lateral chest radiography at the same time by referring the method of using lateral chest radiography to assist diagnose during the diagnosis used by radiologists. Firstly, we evaluated the performance of parameter migration to small data after pre-training for large datasets. The data sets for pre-training are chest X-ray 14 and ImageNet respectively. The results showed that pre-training with chest X-ray 14 performed better than with the generic dataset ImageNet. Secondly, We evaluated the performance of the Frontal and lateral chest radiographs in different modes of input model for the diagnosis of assisted chest disease. Finally, by comparing different feature fusion methods of addition and concatenation, we found that the fusion effect of concatenation is better, which average AUC reached 0.778. The comparison results show that whether it is a public or a non-public dataset, our Dual-Ray Net (concatenation) architecture shows improved performance in recognizing findings in CXR images when compared to applying separate baseline frontal and lateral classes.