Regular Expression Based Medical Text Classification Using Constructive Heuristic Approach

Abstract

Medical text classification assigns medical related text into different categories such as topics or disease types. Machine learning based techniques have been widely used to perform such tasks despite the obvious drawback in such ''black box'' approach, leaving no easy way to fine-tune the resultant model for better performance. We propose a novel constructive heuristic approach to generate a set of regular expressions that can be used as effective text classifiers. The main innovation of our approach is that we develop a novel regular expression based text classifier with both satisfactory classification performance and excellent interpretability. We evaluate our framework on real-world medical data provided by our collaborator, one of the largest online healthcare providers in the market, and observe the high performance and consistency of this approach. Experimental results show that the machine-generated regular expressions can be effectively used in conjunction with machine learning techniques to perform medical text classification tasks. The proposed methodology improves the performance of baseline methods (Naive Bayes and Support Vector Machines) by 9% in precision and 4.5% in recall. We also evaluate the performance of modified regular expressions by human experts and demonstrate the potential of practical applications using the proposed method. INDEX TERMS Regular expressions, text classification, constructive heuristic method.

A regular expression generator based on CSS selectors for efficient extractionfrom HTML pages

“…Traditional regular expression generators [10][11][12][13][14][15][16][17] focus on trying all variations to obtain the most suitable pattern and ignore time efficiency. Moreover, these generators are suitable for different tasks.…”

“…Flores et al [15] develop an algorithm for automatically generating regular expressions from biomedical texts using a coarse-to-fine text aligning method. Cui et al [16] design an efficient novel regular expression based text classifier. This classifier is tested on real-world medical data.…”

Data-Driven Regular Expressions Evolution for Medical Text Classification Using Genetic Programming

“…Given imminent proliferation of medical data on the Internet and the increasing needs for online medical service of current society, applying AI algorithm in medical service has become one of the most active research topics in the last few decades. Text classification, as a problem in general being studied for many years, has been introduced to medical domain to improve service performance [1]- [5]. Classifying relevant medical data into clinical informative categories such as symptoms or disease could potentially vastly reduce human labor cost in medical services.…”

“…In recent years, there have been many deep learning-based algorithms applied to medical text classifications [16], [17] and their performance has reached a very high level statistically. But still, the "black box" nature of deep learning methods are not preferable for direct use in practice, due to their poor interpretability and lack of performance guarantees [1].…”

“…Hence, the resultant regular expressions often lack required standards and the quality of each individual one highly depends on the particular expert's experience and skills. To overcome this drawback, there have been a few research focus on automatic generation of regular expressions from big data [1], [2], [19], [20]. [1] proposed an iterative constructive heuristic to generate regular expressions which can serve as drafts for manual improvement.…”