Features of Preliminary Processing of Semi-Structured Medical Data in Russian for Use in Ensembles of Data Mining Models

Abstract

The paper considers the problem of integration, processing and mining of poorly structured data of medical information systems in order to make managerial decisions in healthcare. The problems of medical data are described, such as the lack of a sufficient structure, a large number of abbreviations characteristic of specific nosologies, the complexity of the automatic semantic interpretation of some fields. The authors demonstrated an approach to the search and disclosure of abbreviation in texts, based on a combination of machine and human processing. The method proposed by the authors, based on a hybrid approach combining the strengths of machine and human processing, made it possible to increase the number of abbreviations found by automatic methods by 21 %, and also opened up to 55 % of cases in the automated mode (with a probability of correctness above 70 %) and significantly reduce the time spent by specialists in processing the remaining reductions. Further research will be aimed at solving the following problems associated with the processing and specificity of medical data, such as a large number of spelling errors, specific grammatical constructions. Using a hybrid approach to preprocessing poorly structured data will increase the efficiency of management decisions in the field of healthcare by reducing the time spent by experts on their creation and support. The hybrid approach to the preprocessing of text data in Russian can be applied in other subject areas. However, it may be necessary to adjust the technique to the specifics of the processed data.