Demystifying the Vetting Process of Voice-controlled Skills on Markets

Wang, Dawei; Chen, Kai; Wang, Wei

doi:10.1145/3478101

Cited by 10 publications

(3 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Work looking at voice assistant platforms suggests that there are also problems with the way that skills are certified and moderated. Current work on the certification process suggests that initial checks miss much of the skill conversation tree, and that skills can be crafted to minimise testing coverage by human and automated checks [50]. Similar work examining the efficacy of skill certification showed that policy-violating skills were approved for public use in over 60% cases across the Alexa and Google Assistant skill stores [7].…”

Section: Alexamentioning

confidence: 99%

Legal Obligation and Ethical Best Practice: Towards Meaningful Verbal Consent for Voice Assistants

Seymour,

Cote,

Such

2023

Preprint

View full text Add to dashboard Cite

To improve user experience, Alexa now allows users to consent to data sharing via voice rather than directing them to the companion smartphone app. While verbal consent mechanisms for voice assistants (VAs) can increase usability, they can also undermine principles core to informed consent. We conducted a Delphi study with experts from academia, industry, and the public sector on requirements for verbal consent in VAs. Candidate requirements were drawn from the literature, regulations, and research ethics guidelines that participants rated based on their relevance to the consent process, actionability by platforms, and usability by end-users, discussing their reasoning as the study progressed. We highlight key areas of (dis)agreement between experts, deriving recommendations for regulators, skill developers, and VA platforms towards crafting meaningful verbal consent mechanisms. Key themes include approaching permissions according to the user's ability to opt-out, minimising consent decisions, and ensuring platforms follow established consent principles. CCS CONCEPTS• Security and privacy → Social aspects of security and privacy; Usability in security and privacy; • Human-centered computing → Natural language interfaces; • Applied computing → Law.

show abstract

Section: Alexamentioning

confidence: 99%

Legal Obligation and Ethical Best Practice: Towards Meaningful Verbal Consent for Voice Assistants

Seymour,

Cote,

Such

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, it is apparent that testing of third-party applications' back-end code running on developers' servers is limited to dynamic black-box testing, as providers do not have direct access to this code. Thus, the security of these vetting processes are inadequate [2,10,17,21,43]. Aside from issues around the effectiveness of security vetting processes for third-party voice applications, developers can also modify the back-end code of the voice applications to implement malicious functionality after such applications have been vetted and published [10,27,39].…”

Section: Threat Modelmentioning

confidence: 99%

Misinformation in Third-party Voice Applications

Bispham

Zard

Ferrer-Aran

et al. 2023

Proceedings of the 5th International Conference on Conversational User Interfaces

View full text Add to dashboard Cite

This paper investigates the potential for spreading misinformation via third-party voice applications in voice assistant ecosystems such as Amazon Alexa and Google Assistant. Our work fills a gap in prior work on privacy issues associated with third-party voice applications, looking at security issues related to outputs from such applications rather than compromises to privacy from user inputs. We define misinformation in the context of third-party voice applications and implement an infrastructure for testing third-party voice applications using automated natural language interaction. Using our infrastructure, we identify -for the first time -several instances of misinformation in third-party voice applications currently available on the Google Assistant and Amazon Alexa platforms. We then discuss the implications of our work for developing measures to pre-empt the threat of misinformation and other types of harmful content in third-party voice assistants becoming more significant in the future. CCS CONCEPTS• Security and privacy → Human and societal aspects of security and privacy; • Human-centered computing → Human computer interaction (HCI).

show abstract

“…For example, skills should not have advertisements or promote alcohol. Although these policies are checked during the skill certification process (which rejects a skill if it violates any of the pre-defined policies), prior work demonstrated the ease of policyviolating skills being certified [32,55]. Several recent works [39,41,42,53,59] developed tools to measure the policy compliance of skills on the Amazon Alexa platform through a dynamic analysis approach (i.e., by exploring the outcomes of skills).…”

Section: Introductionmentioning

confidence: 99%

SkillScanner: Detecting Policy-Violating Voice Applications Through Static Analysis at the Development Phase

Liao,

Cheng,

Cai

et al. 2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

The Amazon Alexa marketplace is the largest Voice Personal Assistant (VPA) platform with over 100,000 voice applications (i.e., skills) published to the skills store. In an effort to maintain the quality and trustworthiness of voice-apps, Amazon Alexa has implemented a set of policy requirements to be adhered to by third-party skill developers. However, recent works reveal the prevalence of policyviolating skills in the current skills store. To understand the causes of policy violations in skills, we first conduct a user study with 34 third-party skill developers focusing on whether they are aware of the various policy requirements defined by the Amazon Alexa platform. Our user study results show that there is a notable gap between VPA's policy requirements and skill developers' practices. As a result, it is inevitable that policy-violating skills will be published.To prevent the inflow of new policy-breaking skills to the skills store from the source, it is critical to identify potential policy violations at the development phase. In this work, we design and develop SkillScanner, an efficient static code analysis tool to facilitate third-party developers to detect policy violations early in the skill development lifecycle. To evaluate the performance of SkillScanner, we conducted an empirical study on 2,451 open source skills collected from GitHub. SkillScanner effectively identified 1,328 different policy violations from 786 skills. Our results suggest that 32% of these policy violations are introduced through code duplication (i.e., code copy and paste). In particular, we found that 42 skill code examples from potential Alexa's official accounts (e.g., "alexa" and "alexa-samples" on GitHub) contain policy violations, which lead to 81 policy violations in other skills due to the copy-pasted code snippets from these Alexa's code examples. CCS CONCEPTS• Software and its engineering → Automated static analysis; • Security and privacy → Privacy protections.

show abstract

Demystifying the Vetting Process of Voice-controlled Skills on Markets

Cited by 10 publications

References 40 publications

Legal Obligation and Ethical Best Practice: Towards Meaningful Verbal Consent for Voice Assistants

Legal Obligation and Ethical Best Practice: Towards Meaningful Verbal Consent for Voice Assistants

Misinformation in Third-party Voice Applications

SkillScanner: Detecting Policy-Violating Voice Applications Through Static Analysis at the Development Phase

Contact Info

Product

Resources

About