Deep Nets: What have They Ever Done for Vision?

Yuille, Alan; Liu, Chenxi

doi:10.1007/s11263-020-01405-z

Cited by 63 publications

(33 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Machines are not people, and there remain many reasons to doubt that today's leading AI systems resemble our minds-in terms of how they learn (94), what they can do (26), and even what they are in the first place (36). Even for vision, many core functions remain beyond the reach of the most advanced Deep Learning systems (38,95). But if ever such systems do achieve humanlike perception and cognition, how will we know?…”

Section: What Species-fair Comparisons Showmentioning

confidence: 99%

Performance vs. competence in human–machine comparisons

Firestone

2020

Proc. Natl. Acad. Sci. U.S.A.

125

View full text Add to dashboard Cite

Does the human mind resemble the machines that can behave like it? Biologically inspired machine-learning systems approach “human-level” accuracy in an astounding variety of domains, and even predict human brain activity—raising the exciting possibility that such systems represent the world like we do. However, even seemingly intelligent machines fail in strange and “unhumanlike” ways, threatening their status as models of our minds. How can we know when human–machine behavioral differences reflect deep disparities in their underlying capacities, vs. when such failures are only superficial or peripheral? This article draws on a foundational insight from cognitive science—the distinction between performance and competence—to encourage “species-fair” comparisons between humans and machines. The performance/competence distinction urges us to consider whether the failure of a system to behave as ideally hypothesized, or the failure of one creature to behave like another, arises not because the system lacks the relevant knowledge or internal capacities (“competence”), but instead because of superficial constraints on demonstrating that knowledge (“performance”). I argue that this distinction has been neglected by research comparing human and machine behavior, and that it should be essential to any such comparison. Focusing on the domain of image classification, I identify three factors contributing to the species-fairness of human–machine comparisons, extracted from recent work that equates such constraints. Species-fair comparisons level the playing field between natural and artificial intelligence, so that we can separate more superficial differences from those that may be deep and enduring.

show abstract

Section: What Species-fair Comparisons Showmentioning

confidence: 99%

Performance vs. competence in human–machine comparisons

Firestone

2020

Proc. Natl. Acad. Sci. U.S.A.

125

View full text Add to dashboard Cite

show abstract

“…While the lower DSCs with smaller cysts are consistent with DSCs achieved with other segmentation approaches [ 1 ], [ 2 ], the degraded contrast and gCNR with decreased cyst size might be linked to the context–detail tradeoff inherent to deep learning. Prior work [ 59 ] demonstrated that CNNs rely on sufficient context to make successful predictions. Linearly interpolating the data to a reduced grid size of 256 × 128 pixels provides each neuron in the CNN with greater context as each neuron sees more of the neighborhood of a particular pixel to make a prediction.…”

Section: Discussionmentioning

confidence: 99%

Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data

Nair

Washington

Tran

et al. 2020

IEEE Trans. Ultrason., Ferroelect., Freq. Contr.

View full text Add to dashboard Cite

Single plane wave transmissions are promising for automated imaging tasks requiring high ultrasound frame rates over an extended field of view. However, a single plane wave insonification typically produces suboptimal image quality. To address this limitation, we are exploring the use of deep neural networks (DNNs) as an alternative to delay-and-sum beamforming. The objectives of this work are to obtain information directly from raw channel data and to simultaneously generate both a segmentation map for automated ultrasound tasks and a corresponding ultrasound B-mode image for interpretable supervision of the automation. We focus on visualizing and segmenting anechoic targets surrounded by tissue and ignoring or de-emphasizing less important surrounding structures. DNNs trained with Field II simulations were tested with simulated, experimental phantom, and in vivo datasets that were not included during training. With unfocused input channel data (i.e., prior to the application of receive time delays), simulated, experimental phantom, and in vivo test datasets achieved mean ± standard deviation Dice similarity coefficients of 0.92 ± 0.13, 0.92±0.03, and 0.77±0.07, respectively, and generalized contrastto-noise ratios (gCNR) of 0.95±0.08, 0.93±0.08, and 0.75±0.14, respectively. With subaperture beamformed channel data and a modification to the input layer of the DNN architecture to accept these data, the fidelity of image reconstruction increased (e.g., mean gCNR of multiple acquisitions of two in vivo breast cysts ranged 0.89-0.96), but DNN display frame rates were reduced from 395 Hz to 287 Hz. Overall, the DNNs successfully translated feature representations learned from simulated data to phantom and in vivo data, which is promising for this novel approach to simultaneous ultrasound image formation and segmentation.

show abstract

“…A naive way to achieve generalization is to collect data from all views, for all possible conditions, but this is impractical due to combinatorial explosion (Yuille et al 2018. Instead, we augment the existing real data synthetically to increase the diversity in terms of viewpoints, appearance, and motions.…”

Section: Introductionmentioning

confidence: 99%

Synthetic Humans for Action Recognition from Unseen Viewpoints

et al. 2021

View full text Add to dashboard Cite

Although synthetic training data has been shown to be beneficial for tasks such as human pose estimation, its use for RGB human action recognition is relatively unexplored. Our goal in this work is to answer the question whether synthetic humans can improve the performance of human action recognition, with a particular focus on generalization to unseen viewpoints. We make use of the recent advances in monocular 3D human body reconstruction from real action sequences to automatically render synthetic training videos for the action labels. We make the following contributions: (1) we investigate the extent of variations and augmentations that are beneficial to improving performance at new viewpoints. We consider changes in body shape and clothing for individuals, as well as more action relevant augmentations such as non-uniform frame sampling, and interpolating between the motion of individuals performing the same action; (2) We introduce a new data generation methodology, SURREACT, that allows training of spatio-temporal CNNs for action classification; (3) We substantially improve the state-of-the-art action recognition performance on the NTU RGB+D and UESTC standard human action multi-view benchmarks; Finally, (4) we extend the augmentation approach to in-the-wild videos from a subset of the Kinetics dataset to investigate the case when only one-shot training data is available, and demonstrate improvements in this case as well.

show abstract

Deep Nets: What have They Ever Done for Vision?

Cited by 63 publications

References 95 publications

Performance vs. competence in human–machine comparisons

Performance vs. competence in human–machine comparisons

Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data

Synthetic Humans for Action Recognition from Unseen Viewpoints

Contact Info

Product

Resources

About