Generation of surgical reports using keyword-augmented next sequence prediction

Abstract

The documentation of a surgical procedure remains a time-consuming task that surgeons must incorporate into their daily routine. However, since a surgical report should be produced immediately after the operation with all impressions of the procedure in mind, a means of automation assistance should be provided. We, therefore, propose a method that generates surgical reports based on keywords stated during the procedure. Our report generation is based on a sequence-tosequence model that is trained on sentence pairs of two consecutive sentences in a surgical report. The known sentence is augmented with a keyword based on the following surgical action to be documented and is then passed into a language model to generate the next sentence. In this way, the complexity of predicting a vast number of possible surgical report phrasings is reduced to a next sentence prediction task. For the language model, an encoder-decoder structure was used with bidirectional 2-layer Long-Short Term Memory (LSTM) units for both components and an attention layer between input and output sentences. The training data consisted of 50 ear-,nose- and throat surgery (ENT) reports with 1500 sentences. The model training was performed in a k-fold cross-validation study with k = 10 and cross-entropy loss as the objective function. The generated reports were investigated using NIST, ROUGE, and METEOR metrics. Additionally, three medical experts identified the report content regarding plausibility and text errors. The trained models reached an accuracy of 0.82 for the next sentence predictions. The generated reports show consistent sentence structures and keyword correspondence for about 70 % of provided keyword sequences. The NIST, ROUGE, and METEOR metrics reached 0.65, 0.71, and 0.64, respectively. The model underperformed for not yet known keyword sequences and shows signs of overfitting when keyword sequences deviate from the baseline of the training set. Our approach for the keyword-augmented generation of surgical reports shows the potential of reducing the text generation complexity by providing a sequence of anchor words. However, the automated generation of surgical reports remains a difficult task due to individual report phrasings and the high variance in keyword sequences.

Surgical reporting for laparoscopic cholecystectomy based on phase annotation by a convolutional neural network (CNN) and the phenomenon of phase flickering: a proof of concept

“…[ 1 ] A sage combination with other very promising approaches as real-time text recognition and keyword-augmented sequence prediction could even open up more complex and unpredictable procedures for automated reporting. [ 24 ]…”

Editorial: Artificial intelligence and machine learning in pediatric surgery

“…Another notable example highlighted in this Research Topic involves machine learning algorithms distinguishing between ventral and dorsal roots during selective dorsal rhizotomy using electro-neurophysiological characteristics Jiang et al Furthermore, robotic-assisted surgery is not untouched by AI's transformative impact, offering surgeons greater precision, facilitating minimally invasive surgeries, and contributing to reduced incisions, pain, and faster recovery times for pediatric patients ( 20 – 22 ). Natural language processing might allow for automatic surgery reporting, streamlining documentation by extracting key information from the surgical procedure and generating detailed reports ( 23 ).…”