Sparse Coding on Stereo Video for Object Detection

Lundquist, Sheng Y.; Mitchell, Melanie; Kenyon, Garrett T.

doi:10.48550/arxiv.1705.07144

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in line with psychophysical experiments with humans, which fits a population coding model that minimizes overall disparity energy in the two half-images (Mallot et al, 1996). Lundquist et al (2017Lundquist et al ( , 2016 used stereo sparse coding, followed by a classifier, for depth inference, as well as for object detection. Their model outperformed others in the case of limited labeled training data.…”

Section: Sparse Coding and Stereo Vision 120supporting

confidence: 74%

Exploitation of Image Statistics with Sparse Coding in the Case of Stereo Vision

Ecke,

Papp,

Mallot

2021

Preprint

View full text Add to dashboard Cite

The sparse coding algorithm has served as a model for early processing in mammalian vision. It has been assumed that the brain uses sparse coding to exploit statistical properties of the sensory stream. We hypothesize that sparse coding discovers patterns from the data set, which can be used to estimate a set of stimulus parameters by simple readout. In this study, we chose a model of stereo vision to test our hypothesis. We used the Locally Competitive Algorithm (LCA), followed by a naïve Bayes classifier, to infer stereo disparity. From the results we report three observations. First, disparity inference was successful with this naturalistic processing pipeline. Second, an expanded, highly redundant representation is required to robustly identify the input patterns. Third, the inference error can be predicted from the number of active coefficients in the LCA representation. We conclude that sparse coding can generate a suitable general representation for subsequent inference tasks.

show abstract

Section: Sparse Coding and Stereo Vision 120supporting

confidence: 74%

Exploitation of Image Statistics with Sparse Coding in the Case of Stereo Vision

Ecke,

Papp,

Mallot

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Previous research has shown that sparse coding can produce robust, semantically meaningful visual features across a variety of tasks, from learning face classifiers (Kim, Hannan, and Kenyon 2018) to aligning binocular video (Lundquist, Mitchell, and Kenyon 2017). It is even robust to adversarial attacks (Schwartz, Alparslan, and Kim 2020).…”

Section: Model Descriptionmentioning

confidence: 99%

MobilePTX: Sparse Coding for Pneumothorax Detection Given Limited Training Examples

Hannan

Nesbit

Wen

et al. 2023

AAAI

View full text Add to dashboard Cite

Point-of-Care Ultrasound (POCUS) refers to clinician-performed and interpreted ultrasonography at the patient's bedside. Interpreting these images requires a high level of expertise, which may not be available during emergencies. In this paper, we support POCUS by developing classifiers that can aid medical professionals by diagnosing whether or not a patient has pneumothorax. We decomposed the task into multiple steps, using YOLOv4 to extract relevant regions of the video and a 3D sparse coding model to represent video features. Given the difficulty in acquiring positive training videos, we trained a small-data classifier with a maximum of 15 positive and 32 negative examples. To counteract this limitation, we leveraged subject matter expert (SME) knowledge to limit the hypothesis space, thus reducing the cost of data collection. We present results using two lung ultrasound datasets and demonstrate that our model is capable of achieving performance on par with SMEs in pneumothorax identification. We then developed an iOS application that runs our full system in less than 4 seconds on an iPad Pro, and less than 8 seconds on an iPhone 13 Pro, labeling key regions in the lung sonogram to provide interpretable diagnoses.

show abstract

“…As a prime example, Le et al [20] was able to build distinctive high-level features using sparsity in unsupervised large scale networks. Lundquist et al [22] was able to use sparse coding to successfully infer depth selective features and detect objects in video. Our computational network is related to these works; we formulate our model as an unsupervised hierarchical sparse coding problem.…”

Section: Introductionmentioning

confidence: 99%

Deep Sparse Coding for Invariant Multimodal Halle Berry Neurons

Kim

Hannan

Kenyon

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Deep feed-forward convolutional neural networks (CNNs) have become ubiquitous in virtually all machine learning and computer vision challenges; however, advancements in CNNs have arguably reached an engineering saturation point where incremental novelty results in minor performance gains. Although there is evidence that object classification has reached human levels on narrowly defined tasks, for general applications, the biological visual system is far superior to that of any computer. Research reveals there are numerous missing components in feed-forward deep neural networks that are critical in mammalian vision. The brain does not work solely in a feed-forward fashion, but rather all of the neurons are in competition with each other; neurons are integrating information in a bottom up and top down fashion and incorporating expectation and feedback in the modeling process. Furthermore, our visual cortex is working in tandem with our parietal lobe, integrating sensory information from various modalities.In our work, we sought to improve upon the standard feed-forward deep learning model by augmenting them with biologically inspired concepts of sparsity, top-down feedback, and lateral inhibition. We define our model as a sparse coding problem using hierarchical layers. We solve the sparse coding problem with an additional top-down feedback error driving the dynamics of the neural network. While building and observing the behavior of our model, we were fascinated that multimodal, invariant neurons naturally emerged that mimicked, "Halle Berry neurons" found in the human brain. These neurons trained in our sparse model learned to respond to high level concepts from multiple modalities, which is not the case with a standard feedforward autoencoder. Furthermore, our sparse representation of multimodal signals demonstrates qualitative and quantitative superiority to the standard feed-forward joint embedding in common vision and machine learning tasks.

show abstract

Sparse Coding on Stereo Video for Object Detection

Cited by 3 publications

References 0 publications

Exploitation of Image Statistics with Sparse Coding in the Case of Stereo Vision

Exploitation of Image Statistics with Sparse Coding in the Case of Stereo Vision

MobilePTX: Sparse Coding for Pneumothorax Detection Given Limited Training Examples

Deep Sparse Coding for Invariant Multimodal Halle Berry Neurons

Contact Info

Product

Resources

About