Benchmarking for Keyword Extraction Methodologies in Maintenance Work Orders

Abstract

Maintenance has largely remained a human-knowledge centered activity, with the primary records of activity being textbased maintenance work orders (MWOs). However, the bulk of maintenance research does not currently attempt to quantify human knowledge, though this knowledge can be rich with useful contextual and system-level information. The underlying quality of data in MWOs often suffers from misspellings, domain-specific (or even workforce specific) jargon, and abbreviations, that prevent its immediate use in computer analyses. Therefore, approaches to making this data computable must translate unstructured text into a formal schema or system; i.e., perform a mapping from informal technical language to some computable format. Keyword spotting (or, extraction) has proven a valuable tool in reducing manual efforts while structuring data, by providing a systematic methodology to create computable knowledge. This technique searches for known vocabulary in a corpus and maps them to designed higher level concepts, shifting the primary effort away from structuring the MWOs themselves, toward creating a dictionary of domain specific terms and the knowledge that they represent. The presented work compares rules-based keyword extraction to data-driven tagging assistance, through quantitative and qualitative discussion of the key advantages and disadvantages. This will enable maintenance practitioners to select an appropriate approach to information encoding that provides needed functionality at minimal cost and effort.

Adapting natural language processing for technical text

“…A number of different named entity recognition classes are being used for MWO annotation: item-activity-state 10 and item-problem/symptom-solution/action. 23,24 The familiar assumptions of NLP often mislead in the analysis of maintenance text. For example, although the overall number of maintenance records can be similar to the number of documents in an NLP corpora, the MWO text tends to be much smaller (Table 2).…”

“…Although for many use cases, a rules-based approach can handle the presence of zero, one, or multiple labels on a single MWO, this can challenge supervised learning approaches 23,30 and performance depends on the handling of class imbalance. Seale 44 handled the challenge of 1200 different component classes by injecting additional information relevant to the physical systems into the model training systems through "privileged information" which is a form of fortuitous data.…”

“…Such dictionaries are reusable, developed/tuned as a data preprocessing step across the data, and often use common NLP tools to assist in their creation. 10,23,40 Sexton et al 40 developed an importance based vocabulary tagging system using term frequency-inverse document frequency weighting. 22 Gao et al 10 used spellcheckers (pyspeller) and string distance (fuzzywuzzy) to support dictionary creation process for domain specific uses.…”

Mining text from natural scene and video images: A survey

“…Keyword spotting, as one of the document image analysis techniques, includes a systematic methodology and framework to facilitate this transformation and help to create computable knowledge. This technique searches for a known vocabulary in a document image dataset and maps them to higher‐level concepts created by indexing the document images and creating a dictionary of domain‐specific terms and the knowledge they represent (Sexton et al, 2018). As an important category of approaches for knowledge discovery and mining, and with the increase in generating textual information in the forms of the scene and video images, word spotting from the scene, and video images has grown in recent years.…”

Nestor: A Tool for Natural Language Annotation of Short Texts

“…The objective of Nestor is to help analysts make their natural language data, which is often unstructured, filled with technical content, jargon, mispellings, and abbreviations, computable to improve analysis. An example of natural language data that could be input to Nestor and the subsequent output data and the corresponding output is shown in Table 1.The annotated datasets generated by Nestor (as either a CSV or .h5 file) can be used for different analysis techniques, such as failure prediction, problem hot spot identification, and maintenance technician expertise assessment, as shown in [2][3][4][5][6][7][8][9][10]. Currently, the majority of use cases involve maintenance in the engineering domain (manufacturing, mining, heating ventilation and air conditioning (HVAC)), however, any natural language CSV file with UTF-8 encoding can be input to Nestor.…”