GHio-Ca: An Android Application for Automatic Image Classification

Polonio, Davide; Tavella, Federico; Zanella, Marco; Bujari, Armir

doi:10.1007/978-3-319-76111-4_25

“…Often, crowd sourced (Polonio et al, 2018) or automatically constructed datasets such as ours, have a performance ceiling, possibly due to incorrectly assigned ground truth labels or low quality of input data (Chen et al, 2016;Schlegel et al, 2020). To investigate the former, we measure the agreement on videos that all models misclassify using Fleiss' κ.…”

Section: Resultsmentioning

confidence: 99%

WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language

Tavella¹,

Schlegel²,

Romeo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Signed Language Processing (SLP) concerns the automated processing of signed languages, the main means of communication of Deaf and hearing impaired individuals. SLP features many different tasks, ranging from sign recognition to translation and production of signed speech, but it has been overlooked by the NLP community thus far. In this paper, we bring attention to the task of modelling the phonology of sign languages. We leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. We then conduct an extensive empirical study to investigate whether data-driven end-to-end and feature-based approaches can be optimised to automatically recognise these properties. We find that, despite the inherent challenges of the task, graph-based neural networks that operate over skeleton features extracted from raw videos are able to succeed at the task to a varying degree. Most importantly, we show that this performance pertains even on signs unobserved during training.

show abstract

“…Often, crowd sourced (Polonio et al, 2018) or automatically constructed datasets such as ours, have a performance ceiling, possibly due to incorrectly assigned ground truth labels or low quality of input data (Chen et al, 2016;Schlegel et al, 2020). To investigate the former, we measure the agreement on videos that all models misclassify using Fleiss' κ.…”

Section: Resultsmentioning

confidence: 99%

WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language

Tavella¹,

Schlegel²,

Romeo³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

7

0

View full text Add to dashboard Cite

Signed Language Processing (SLP) concerns the automated processing of signed languages, the main means of communication of Deaf and hearing impaired individuals. SLP features many different tasks, ranging from sign recognition to translation and production of signed speech, but it has been overlooked by the NLP community thus far. In this paper, we bring attention to the task of modelling the phonology of sign languages. We leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. We then conduct an extensive empirical study to investigate whether data-driven end-to-end and feature-based approaches can be optimised to automatically recognise these properties. We find that, despite the inherent challenges of the task, graph-based neural networks that operate over skeleton features extracted from raw videos are able to succeed at the task to a varying degree. Most importantly, we show that this performance pertains even on signs unobserved during training.

show abstract

“…Cloud services are a popular method on mobile devices to offload energy and time-consuming tasks to much more powerful servers. For instance, Polonio et al [15] proposed an automatic photo tagging application for sharing pictures on social media, which relies on external services to retrieve the tags.…”

Section: Introductionmentioning

confidence: 99%

Enabling Privacy with Transfer Learning for Image Classification DNNs on Mobile Devices

Seiderer

¹

,

Dietz

²

,

Aslan

³

et al. 2018

Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good

View full text Add to dashboard Cite

More people could benefit of Machine Learning (ML) as an increasingly important technology and service, if state-of-the-art ML techniques with training capability were accessible on personal devices. To this end, we report details on how to deploy Tensor-Flow on off-the-shelf mobile and embedded devices and retrain current deep neural networks for image recognition on-device. Our motivation is to both grant privacy and allow users to efficiently personalize image classifiers for their own needs and purposes, and thus contribute towards turning ML into a "social good", which benefits the largest number of people in the greatest possible way. CCS CONCEPTS • Human-centered computing → Mobile devices; • Computing methodologies → Transfer learning;

show abstract

GHio-Ca: An Android Application for Automatic Image Classification

Cited by 6 publications

References 13 publications

WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language

WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language

WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language

Enabling Privacy with Transfer Learning for Image Classification DNNs on Mobile Devices

Contact Info

Product

Resources

About