BackgroundOver a tenth of preventable adverse events in health care are caused by failures in information flow. These failures are tangible in clinical handover; regardless of good verbal handover, from two-thirds to all of this information is lost after 3-5 shifts if notes are taken by hand, or not at all. Speech recognition and information extraction provide a way to fill out a handover form for clinical proofing and sign-off.ObjectiveThe objective of the study was to provide a recorded spoken handover, annotated verbatim transcriptions, and evaluations to support research in spoken and written natural language processing for filling out a clinical handover form. This dataset is based on synthetic patient profiles, thereby avoiding ethical and legal restrictions, while maintaining efficacy for research in speech-to-text conversion and information extraction, based on realistic clinical scenarios. We also introduce a Web app to demonstrate the system design and workflow.MethodsWe experiment with Dragon Medical 11.0 for speech recognition and CRF++ for information extraction. To compute features for information extraction, we also apply CoreNLP, MetaMap, and Ontoserver. Our evaluation uses cross-validation techniques to measure processing correctness.ResultsThe data provided were a simulation of nursing handover, as recorded using a mobile device, built from simulated patient records and handover scripts, spoken by an Australian registered nurse. Speech recognition recognized 5276 of 7277 words in our 100 test documents correctly. We considered 50 mutually exclusive categories in information extraction and achieved the F1 (ie, the harmonic mean of Precision and Recall) of 0.86 in the category for irrelevant text and the macro-averaged F1 of 0.70 over the remaining 35 nonempty categories of the form in our 101 test documents.ConclusionsThe significance of this study hinges on opening our data, together with the related performance benchmarks and some processing software, to the research and development community for studying clinical documentation and language-processing. The data are used in the CLEFeHealth 2015 evaluation laboratory for a shared task on speech recognition.

Background Over the last decade, telemedicine services have been introduced in the public health care systems of several industrialized countries. In Catalonia, the use of eConsulta, an asynchronous teleconsultation service between primary care professionals and citizens in the public health care system, has already reached 1 million cases. Before the COVID-19 pandemic, the use of eConsulta was growing at a monthly rate of 7%, and the growth has been exponential from March 15, 2020 to the present day. Despite its widespread usage, there is little qualitative evidence describing how this tool is used. Objective The aim of this study was to annotate a random sample of teleconsultations from eConsulta, and to evaluate the level of agreement between health care professionals with respect to the annotation. Methods Twenty general practitioners retrospectively annotated a random sample of 5382 cases managed by eConsulta according to three aspects: the type of interaction according to 6 author-proposed categories, whether the practitioners believed a face-to-face visit was avoided, and whether they believed the patient would have requested a face-to-face visit had eConsulta not been available. A total of 1217 cases were classified three times by three different professionals to assess the degree of consensus among them. Results The general practitioners considered that 79.60% (4284/5382) of the teleconsultations resulted in avoiding a face-to-face visit, and considered that 64.96% (3496/5382) of the time, the patient would have made a face-to-face visit in the absence of a service like eConsulta. The most frequent uses were for management of test results (26.77%, 1433/5354), management of repeat prescriptions (24.30%, 1301/5354), and medical enquiries (14.23%, 762/5354). The degree of agreement among professionals as to the annotations was mixed, with the highest consensus demonstrated for the question “Has the online consultation avoided a face-to-face visit?” (3/3 professionals agreed 67.95% of the time, 827/1217), and the lowest consensus for the type of use of the teleconsultation (3/3 professionals agreed 57.60% of the time, 701/1217). Conclusions This study shows the ability of eConsulta to reduce the number of face-to-face visits for 55% (79% × 65%) to 79% of cases. In comparison to previous research, these results are slightly more pessimistic, although the rates are still high and in line with administrative data proxies, showing that 84% of patients using teleconsultations do not make an in-person appointment in the following 3 months. With respect to the type of consultation performed, our results are similar to the existing literature, thus providing robust support for eConsulta’s usage. The mixed degree of consensus among professionals implies that results derived from artificial intelligence tools such as message classification algorithms should be interpreted in light of these shortcomings.

In named entity recognition, we often don't have a large in-domain training corpus or a knowledge base with adequate coverage to train a model directly. In this paper, we propose a method where, given training data in a related domain with similar (but not identical) named entity (NE) types and a small amount of in-domain training data, we use transfer learning to learn a domain-specific NE model. That is, the novelty in the task setup is that we assume not just domain mismatch, but also label mismatch.