SkillVet: Automated Traceability Analysis of Amazon Alexa Skills

Abstract

Skills, are essential components in Smart Personal Assistants (SPA). The number of skills has grown rapidly, dominated by a changing environment that has no clear business model. Skills can access personal information and this may pose a risk to users. However, there is little information about how this ecosystem works, let alone the tools that can facilitate its study. In this paper, we present the largest systematic measurement of the Amazon Alexa skill ecosystem to date. We study developers' practices in this ecosystem, including how they collect and justify the need for sensitive information, by designing a methodology to identify over-privileged skills with broken privacy policies. We collect 199,295 Alexa skills and uncover that around 43% of the skills (and 50% of the developers) that request these permissions follow bad privacy practices, including (partially) broken data permissions traceability. In order to perform this kind of analysis at scale, we present SkillVet that leverages machine learning and natural language processing techniques, and generates high-accuracy prediction sets. We report several concerning practices, including how developers can bypass Alexa's permission system through account linking and conversational skills, and offer recommendations on how to improve transparency, privacy and security. Resulting from the responsible disclosure we did, 13% of the reported issues no longer pose a threat at submission time.

Measuring Alexa Skill Privacy Practices across Three Years

“…The number of skills has multiplied in recent years. For instance, the Amazon Alexa skill ecosystem has grown from just 135 skills in early 2016 [27] to over 100k skills by late 2020 [12]. This rapid surge in numbers can be attributed to the continuous proliferation of SPA worldwide: about 4.2 billion SPAs are being used around the world [37].…”

Your Echos are Heard: Tracking, Profiling, and Ad Targeting in the Amazon Smart Speaker Ecosystem

“…Thus, to augment Alexa, Amazon allows third party services to build and publish applications called skills on the Alexa marketplace. As of 2020, the Alexa marketplace hosts more than 200K third party skills [60].…”

“…In these cases, neither users nor Amazon have any visibility or control on the processing, sharing, and selling of users' interpreted voice input. Third party skills often do not publish their privacy policies, nor adhere to them even when they do [60].…”

“…To avoid contamination across personas, we configure each Amazon Echo through a fresh browser profile and assign unique IP address to each device. We then use a Selenium [41] based web crawler to programmatically visit the Alexa skill marketplace, and iteratively install and enable the top-50 skills (based on the number of reviews) for each category-we use the dataset released in [60] to extract top skills. If prompted, we enable all of the requested permissions by a skill.…”

“…To evaluate the effectiveness of this approach, we focus on Amazon's smart speaker platform, as it is the largest platform (46 million devices in the US [43] and 200K thirdparty applications [60]). To address the first challenge, we set up a custom Raspberry RPi (RPi) router [42] to capture the endpoints contacted by Amazon Echo and as well as emulating an Amazon Echo by instrumenting Alexa Voice Service (AVS) SDK [15] and running it on a RPi (we call it AVS Echo) to capture collected data.…”