A Real-Time Region Tracking Algorithm Tailored to Endoscopic Video with Open-Source Implementation

Epperlein, Jonathan P.; Zhuk, Sergiy

doi:10.48550/arxiv.2203.08858

Cited by 2 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is conceptually related to classical object tracking, which is a mature topic in computer vision, but it turns out that due to the often smooth textures and the lack of "objectness" of ROIs (see e.g. ROI 3 in Figure 1), most off-the-shelf object tracking algorithms perform poorly [24]. We developed a simple region tracking algorithm based on optical flow with outstanding performance on endoscopic video at framerates of 30 FPS (on a MacBook Pro at 480x360 resolution).…”

Section: A Computer Visionmentioning

confidence: 99%

See 1 more Smart Citation

Extracting, Visualizing, and Learning from Dynamic Data: Perfusion in Surgical Video for Tissue Characterization

Epperlein

Hardy

Aonghusa

et al. 2022

2022 IEEE International Conference on Digital Health (ICDH)

View full text Add to dashboard Cite

Intraoperative assessment of tissue can be guided through fluorescence imaging which involves systemic dosing with a fluorophore and subsequent examination of the tissue region of interest with a near-infrared camera. This typically involves administering indocyanine green (ICG) hours or even days before surgery and intraoperative visualization at the time predicted for steady-state signal-to-background status. Here, we describe our efforts to capture and utilize the information contained in the first few minutes after ICG administration from the perspective of both signal processing and surgical practice. We prove a method for characterization of cancerous versus benign rectal lesions now undergoing further development and validation via multicenter clinical phase studies.

show abstract

Section: A Computer Visionmentioning

confidence: 99%

“…We developed a simple region tracking algorithm based on optical flow with outstanding performance on endoscopic video at framerates of 30 FPS (on a MacBook Pro at 480x360 resolution). We have released our Python implementation as open source [24][25] and used it ourselves in our previous publications e.g. [23]…”

Section: A Computer Visionmentioning

confidence: 99%

Extracting, Visualizing, and Learning from Dynamic Data: Perfusion in Surgical Video for Tissue Characterization

Epperlein

Hardy

Aonghusa

et al. 2022

2022 IEEE International Conference on Digital Health (ICDH)

View full text Add to dashboard Cite

show abstract

Clinical application of machine learning and computer vision to indocyanine green quantification for dynamic intraoperative tissue characterisation: how to do it

et al. 2023

Self Cite

View full text Add to dashboard Cite

Introduction Indocyanine green (ICG) quantification and assessment by machine learning (ML) could discriminate tissue types through perfusion characterisation, including delineation of malignancy. Here, we detail the important challenges overcome before effective clinical validation of such capability in a prospective patient series of quantitative fluorescence angiograms regarding primary and secondary colorectal neoplasia. Methods ICG perfusion videos from 50 patients (37 with benign (13) and malignant (24) rectal tumours and 13 with colorectal liver metastases) of between 2- and 15-min duration following intravenously administered ICG were formally studied (clinicaltrials.gov: NCT04220242). Video quality with respect to interpretative ML reliability was studied observing practical, technical and technological aspects of fluorescence signal acquisition. Investigated parameters included ICG dosing and administration, distance–intensity fluorescent signal variation, tissue and camera movement (including real-time camera tracking) as well as sampling issues with user-selected digital tissue biopsy. Attenuating strategies for the identified problems were developed, applied and evaluated. ML methods to classify extracted data, including datasets with interrupted time-series lengths with inference simulated data were also evaluated. Results Definable, remediable challenges arose across both rectal and liver cohorts. Varying ICG dose by tissue type was identified as an important feature of real-time fluorescence quantification. Multi-region sampling within a lesion mitigated representation issues whilst distance–intensity relationships, as well as movement-instability issues, were demonstrated and ameliorated with post-processing techniques including normalisation and smoothing of extracted time–fluorescence curves. ML methods (automated feature extraction and classification) enabled ML algorithms glean excellent pathological categorisation results (AUC-ROC > 0.9, 37 rectal lesions) with imputation proving a robust method of compensation for interrupted time-series data with duration discrepancies. Conclusion Purposeful clinical and data-processing protocols enable powerful pathological characterisation with existing clinical systems. Video analysis as shown can inform iterative and definitive clinical validation studies on how to close the translation gap between research applications and real-world, real-time clinical utility.

show abstract

A Real-Time Region Tracking Algorithm Tailored to Endoscopic Video with Open-Source Implementation

Cited by 2 publications

References 19 publications

Extracting, Visualizing, and Learning from Dynamic Data: Perfusion in Surgical Video for Tissue Characterization

Extracting, Visualizing, and Learning from Dynamic Data: Perfusion in Surgical Video for Tissue Characterization

Clinical application of machine learning and computer vision to indocyanine green quantification for dynamic intraoperative tissue characterisation: how to do it

Contact Info

Product

Resources

About