No abstract

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of dailylife activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards, with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception.

No abstract

Cryo-electron tomography (Cryo-ET) has made possible the observation of cellular organelles and macromolecular complexes at nanometer resolution in native conformations. Without disrupting the cell, Cryo-ET directly visualizes both known and unknown structures in situ and reveals their spatial and organizational relationships. Consequently, structural pattern mining (a.k.a. visual proteomics) needs to be performed to detect, identify and recover different sub-cellular components and their spatial organization in a systematic fashion for further biomedical analysis and interpretation. This chapter presents three major Cryo-ET structural pattern mining approaches to give an overview of traditional methods and recent advances in Cryo-ET data analysis. Template-based, supervised deep learning-based and template-free approaches are introduced in detail. Examples of recent biological and medical applications and future perspectives are provided.

We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN). We first notice CNN's ability in capturing the high-frequency components of images. These high-frequency components are almost imperceptible to a human. Thus the observation can serve as one of the explanations of the existence of adversarial examples, and can also help verify CNN's trade-off between robustness and accuracy. Our observation also immediately leads to methods that can improve the adversarial robustness of trained CNNs. Finally, we also utilize this observation to design a (semi) black-box adversarial attack method.