Object Recognition with Continual Open Set Domain Adaptation for Home Robot

Kishida, Ikki; Chen, Hong; Baba, M.; Jin, Jiren; Amma, Ayako; Nakayama, Hideki

doi:10.1109/wacv48630.2021.00156

Cited by 5 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use two examples to discuss the potential applications of EdgeFM. The first application is robot semantic sensing [44] which is widely used for home services [33] and industrial applications [62]. The main challenge for the current home-service robot system is the high diversity in personal items and dynamic environmental factors across households.…”

Section: Application Scenariosmentioning

confidence: 99%

“…For example, users sometimes buy or throw items. Current recognition models on home-service robots [33,44] require labeled data for model retraining to adapt to the new classes of objects, which is not realistic and user-unfriendly. The second application is human activity recognition (HAR), which is an essential algorithm for many smart embedded systems for health, such as disease monitoring and fitness tracking.…”

Section: Application Scenariosmentioning

confidence: 99%

“…Their performance degrades severely when the class of input is not seen during the training. Although various approaches such as transfer learning [2,42] and meta-learning [15,19] have been proposed to calibrate models on the edge, they still require non-trivial efforts for manual data annotation and on-device training, which are not practical in real-world deployments [33,44,49]. The recent emergence of foundation models (FMs) such as GPT [5] and CLIP [52] have shown impressive general knowledge that can support diverse downstream tasks like image captioning, question answering, and information extraction.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge

Yang,

He,

Ling

et al. 2023

Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems

View full text Add to dashboard Cite

Deep Learning (DL) models have been widely deployed on IoT devices with the help of advancements in DL algorithms and chips. However, the limited resources of edge devices make these ondevice DL models hard to be generalizable to diverse environments and tasks. Although the recently emerged foundation models (FMs) show impressive generalization power, how to effectively leverage the rich knowledge of FMs on resource-limited edge devices is still not explored. In this paper, we propose EdgeFM, a novel edge-cloud cooperative system with open-set recognition capability. EdgeFM selectively uploads unlabeled data to query the FM on the cloud and customizes the specific knowledge and architectures for edge models. Meanwhile, EdgeFM conducts dynamic model switching at run-time taking into account both data uncertainty and dynamic network variations, which ensures the accuracy always close to the original FM. We implement EdgeFM using two FMs on two edge platforms. We evaluate EdgeFM on three public datasets and two self-collected datasets. Results show that EdgeFM can reduce the end-to-end latency up to 3.2x and achieve 34.3% accuracy increase compared with the baseline. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile devices; • Computing methodologies → Artificial intelligence; • Computer systems organization → Embedded and cyberphysical systems.

show abstract

Section: Application Scenariosmentioning

confidence: 99%

Section: Application Scenariosmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge

Yang,

He,

Ling

et al. 2023

Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems

View full text Add to dashboard Cite

show abstract

Kurcuma: a kitchen utensil recognition collection for unsupervised domain adaptation

Rosello

Valero-Mas

Gallego

et al. 2023

Pattern Anal Applic

View full text Add to dashboard Cite

The use of deep learning makes it possible to achieve extraordinary results in all kinds of tasks related to computer vision. However, this performance is strongly related to the availability of training data and its relationship with the distribution in the eventual application scenario. This question is of vital importance in areas such as robotics, where the targeted environment data are barely available in advance. In this context, domain adaptation (DA) techniques are especially important to building models that deal with new data for which the corresponding label is not available. To promote further research in DA techniques applied to robotics, this work presents Kurcuma (Kitchen Utensil Recognition Collection for Unsupervised doMain Adaptation), an assortment of seven datasets for the classification of kitchen utensils—a task of relevance in home-assistance robotics and a suitable showcase for DA. Along with the data, we provide a broad description of the main characteristics of the dataset, as well as a baseline using the well-known domain-adversarial training of neural networks approach. The results show the challenge posed by DA on these types of tasks, pointing to the need for new approaches in future work.

show abstract