ISINet: An Instance-Based Approach for Surgical Instrument Segmentation

González, Cristina; Bravo-Sánchez, Laura; Pont-Tuset, Jordi

doi:10.1007/978-3-030-59716-0_57

Cited by 43 publications

(40 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the strengths and limitations of our teacherstudent enabled simulation-to-real unsupervised domain adaptation approach, the framework admits multiple straightforward extensions to bridge the remaining domain gap: * Implementing techniques to suppress false detection for empty frames, instruments near the image border and specular reflections, for instance by utilizing temporal information [13] of video frames. * Improving physical properties of simulation to capture instrument-tissue interaction, considering the variations in predictions across instrument boundaries.…”

Section: Discussionmentioning

confidence: 99%

“…The green color in the images represents the network predictions while the yellow color represents under-segmentation as false detection on specular reflection, under-segmentation for small instruments, tool-tissue interaction and partially occluded instruments. These factors can in part be addressed by utilizing the temporal information of video frames [13].…”

Section: Analysis On Endovismentioning

confidence: 99%

See 1 more Smart Citation

Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation

2021

View full text Add to dashboard Cite

Purpose Segmentation of surgical instruments in endoscopic video streams is essential for automated surgical scene understanding and process modeling. However, relying on fully supervised deep learning for this task is challenging because manual annotation occupies valuable time of the clinical experts. Methods We introduce a teacher–student learning approach that learns jointly from annotated simulation data and unlabeled real data to tackle the challenges in simulation-to-real unsupervised domain adaptation for endoscopic image segmentation. Results Empirical results on three datasets highlight the effectiveness of the proposed framework over current approaches for the endoscopic instrument segmentation task. Additionally, we provide analysis of major factors affecting the performance on all datasets to highlight the strengths and failure modes of our approach. Conclusions We show that our proposed approach can successfully exploit the unlabeled real endoscopic video frames and improve generalization performance over pure simulation-based training and the previous state-of-the-art. This takes us one step closer to effective segmentation of surgical instrument in the annotation scarce setting.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Analysis On Endovismentioning

confidence: 99%

Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation

2021

View full text Add to dashboard Cite

show abstract

“…This representation can be seen as a two dimensional image of the molecule. This type of data arrays are typically encountered in computer vision algorithms, which is attractive as we have extensive expertise with such approaches [45–47]. As delineated below, we indeed applied our most recent developments and insights in classification algorithms to classify them accurately.…”

Section: Methodsmentioning

confidence: 99%

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

Puentes¹,

Valderrama²,

González³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules' target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 95.8% in the Receiver Operating Characteristic curve (ROC-AUC) and 58.9% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained [3] potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

show abstract

“…This representation can be seen as a two dimensional image of the molecule. This type of data arrays are typically encountered in computer vision algorithms, which is attractive as we have extensive expertise with such approaches [44][45][46]. As delineated below, we indeed applied our most recent developments and insights in classification algorithms to classify them accurately.…”

Section: Plos Onementioning

confidence: 99%

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

The discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules’ target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 97.7% in the Receiver Operating Characteristic curve (ROC-AUC) and 65.5% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained three (3) potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

show abstract

ISINet: An Instance-Based Approach for Surgical Instrument Segmentation

Cited by 43 publications

References 24 publications

Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation

Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

Contact Info

Product

Resources

About