Data Collection Methods for Building a Free Response Training Simulation

Sharma, Vaibhav; Shpringer, Beni; Yang, Sung Min; Bolger, Martin; Adewole, Sodiq; Brown, Donald E.; Gharavi, Erfaneh

doi:10.1109/sieds.2019.8735621

Cited by 3 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an earlier work [27] we described the foundation for our data collection effort. Based on the dialogue designed by the Chinese culture experts [32], players' in the simulation are evaluated at fourteen ( 14) different points during the interaction.…”

Section: A Data Collection Annotation and Scoring Methodsmentioning

confidence: 99%

“…The collected data is then used to train a natural language understanding model to recognize users' input to the system. Other works have approached data collection using crowd-sourcing through amazon M-turk [27]. Sharma V. et al [27], took a step further by applying a data augmentation technique to improve the sample size and class distribution of an originally crowd-sourced data.…”

Section: Data Collectionmentioning

confidence: 99%

See 1 more Smart Citation

Dialogue-Based Simulation For Cultural Awareness Training

Adewole,

Gharavi,

Shpringer

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Existing simulations designed for cultural and interpersonal skill training rely on pre-defined responses with a menu option selection interface. Using a multiple-choice interface and restricting trainees' responses may limit the trainees' ability to apply the lessons in real life situations. These systems, also rely on a simplistic evaluation model, where trainees' selected options are marked as either correct or incorrect. The model cannot capture sufficient information that could drive an adaptive feedback mechanism to improve trainees' cultural awareness. This paper describes the design of a dialogue-based simulation for cultural awareness training. The simulation, built around a disaster management scenario involving a joint coalition between the US and the Chinese armies. Trainees were able to engage in realistic dialogue with the Chinese agent. Their responses, at different points, get evaluated by different multi-label classification models. Based on training on our dataset, the models score the trainees' responses for their awareness of the Chinese culture. Trainees also get feedback that informs the cultural appropriateness of their responses. The result of this work showed the following; i) A feature-based evaluation model improves the design, modeling and computation of dialogue-based training simulation systems; ii) Output from current automatic speech recognition (ASR) systems gave comparable end results compared with the output from manual transcription; iii) A multi-label classification model trained as a cultural expert gave results which were comparable with scores assigned by human annotators.

show abstract

Section: A Data Collection Annotation and Scoring Methodsmentioning

confidence: 99%

Section: Data Collectionmentioning

confidence: 99%

Dialogue-Based Simulation For Cultural Awareness Training

Adewole,

Gharavi,

Shpringer

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Detection of boundaries or transition points (TP) on sequence data [31] has been considered in solving many sequence segmentation problems across various applications such as medical condition monitoring [32], climate change detection [33], audio activity segmentation and boundary recognition for silence in speech [34], speaker segmentation, scene change detection, and human activity analysis [35]. Other areas where detection and localization of distributional changes in sequence data arises include online sequential time series analysis [36], [37].…”

Section: Boundary Detectionmentioning

confidence: 99%

“…These probabilistic models require a good knowledge of the transition structure between the segments and also require careful pre-training to yield a competitive performance. This may not be practicable for online applications where data are acquired online [31], [40]. Parametric approaches model the distribution before and after the change based on maximum likelihood framework [41] while non-parametric methods [42] have been mostly limited to uni-variate data.…”

Section: Boundary Detectionmentioning

confidence: 99%

Unsupervised Shot Boundary Detection for Temporal Segmentation of Long Capsule Endoscopy Videos

Adewole¹,

Fernandes²,

Jablonski³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Physicians use Capsule Endoscopy (CE) as a noninvasive and non-surgical procedure to examine the entire gastrointestinal (GI) tract for diseases and abnormalities. A single CE examination could last between 8 to 11 hours generating up to 80,000 frames which is compiled as a video. Physicians have to review and analyze the entire video to identify abnormalities or diseases before making diagnosis. This review task can be very tedious, time consuming and prone to error. While only as little as a single frame may capture useful content that is relevant to the physicians' final diagnosis, frames covering the small bowel region alone could be as much as 50,000. To minimize physicians' review time and effort, this paper proposes a novel unsupervised and computationally efficient temporal segmentation method to automatically partition long CE videos into a homogeneous and identifiable video segments. However, the search for temporal boundaries in a long video using high dimensional framefeature matrix is computationally prohibitive and impracticable for real clinical application. Therefore, leveraging both spatial and temporal information in the video, we first extracted high level frame features using a pretrained CNN model and then projected the high-dimensional frame-feature matrix to lower 1-dimensional embedding. Using this 1-dimensional sequence embedding, we applied the Pruned Exact Linear Time (PELT) algorithm to searched for temporal boundaries that indicates the transition points from normal to abnormal frames and viceversa. The key novelty of this work is in three (3) folds -first, the automated detection of temporal boundaries in long CE video has not been previously considered. Secondly, the reduction in the computational cost of the temporal boundary detection search by using a lower dimensional frame feature embedding; and lastly, the entire temporal segmentation of the CE videos requiring no supervision from medical expert is a new concept. The output of our model can be easily integrated into any CE video summarization model where physicians only need to review a selected sample frame from each video segment. We experimented with multiple real patients' CE videos and our result showed PCA was superior in capturing the transition between pair of normal and abnormal frames in the video. We also bench-marked with expert provided label, and our system achieved an AUC of 66% on multiple test videos.

show abstract

“…Meanwhile, many natural interactions between objects can be represented as a graph with the relationship between the objects captured in the edges between the nodes of the graph. Graph Neural Networks (GNN) models are robust and generic enough to also accommodate spatial and sequence data [70], [71] by specifying the nature of the edge and node relationships.…”

Section: B Graph Convolutional Neural Network (Gcnn)mentioning

confidence: 99%

Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos

Adewole,

Fernandes,

Jablonski

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Temporal activity localization in long videos is an important problem. The cost of obtaining frame level label for long Wireless Capsule Endoscopy (WCE) videos is prohibitive. In this paper, we propose an end-to-end temporal abnormality localization for long WCE videos using only weak video level labels. Physicians use Capsule Endoscopy (CE) as a non-surgical and non-invasive method to examine the entire digestive tract in order to diagnose diseases or abnormalities. While CE has revolutionized traditional endoscopy procedures, a single CE examination could last up to 8 hours generating as much as 100,000 frames. Physicians must review the entire video, frameby-frame, in order to identify the frames capturing relevant lesion or abnormality. This, sometimes could be as few as just a single frame. Given this very high level of redundancy, analysing long CE videos can be very tedious, time consuming and also error prone. This paper presents a novel multi-step method for an end-to-end localization of target frames capturing abnormalities of interest in the long video using only weak video labels. First we developed an automatic temporal segmentation using change point detection technique to temporally segment the video into uniform, homogeneous and identifiable segments. Then we employed Graph Convolutional Neural Network (GCNN) to learn a representation of each video segment. Using weak video segment labels, we trained our GCNN model to recognize each video segment as abnormal if it contains at least a single abnormal frame. Finally, leveraging the parameters of the trained GCNN model, we replaced the final layer of the network with a temporal pool layer to localize the relevant abnormal frames within each abnormal video segment. We experimented with multiple real patients' endoscopy videos and achieved an accuracy of 89.9% on the graph classification task and a specificity of 97.5% on the abnormal frames localization task.

show abstract

Data Collection Methods for Building a Free Response Training Simulation

Cited by 3 publications

References 12 publications

Dialogue-Based Simulation For Cultural Awareness Training

Dialogue-Based Simulation For Cultural Awareness Training

Unsupervised Shot Boundary Detection for Temporal Segmentation of Long Capsule Endoscopy Videos

Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos

Contact Info

Product

Resources

About