Autodirective audio capturing through a synchronized smartphone array

Abstract

High-quality, speaker-location-aware audio capturing has traditionally been realized using dedicated microphone arrays. But high cost and lack of portability prevents such systems from being widely adopted. Today's smartphones are relatively more convenient for audio recording, but the audio quality is much lower in noisy environment and speaker location cannot be readily obtained. In this paper, we design and implement Dia, which leverages smartphone cooperation to overcome the above limitations. Dia supports spontaneous setup, by allowing a group of users to rapidly assemble an array of smartphones to emulate a dedicated microphone array. It employs a novel framework to accurately synchronize the audio I/O clocks of the smartphones. The synchronized smartphone array further enables autodirective audio capturing, i.e., tracking the speaker's location, and beamforming the audio capturing towards the speaker to improve audio quality. We implement Dia on a testbed consisting of 8 Android phones. Our experiments demonstrate that Dia can synchronize the microphones of different smartphones with sample-level accuracy. It achieves high localization accuracy, and similar beamforming performance compared with a microphone array with perfect synchronization.

Context-free Attacks Using Keyboard Acoustic Emanations

“…Due to different oscillator frequencies, the build-in clocks may trigger to sample more or less than 44.1k per second. The synchronization issue imposed by the deviation in microphone sampling rate is a complex research problem studied by various researchers [30,28,15]. Figure 8 shows some observations on the sampling offset distributions.…”

“…Unfortunately, Apple does not provide the APIs to proactively access more than one microphone in iPhone 5 or 5S. Besides Samsung and iPhone, other smartphones like Nexus 5 are also tested to show a linear relationship between the elapsed time and the number of offset samples [30].…”

Indoor Self-Localization and Orientation Estimation of Smartphones Using Acoustic Signals

“…Notice that because the nodes are equipped with two microphones, we take the average of the ToAs to obtain the ToA at the center of the array. The specific methods to obtain internode synchronization fall outside the scope of this paper, although there are multiple solutions in the literature, for example, Sur et al [26].…”

Synchronized audio capture with an array of mobile devices

“…Recent work by Sur, Wei, Zhang [2] and Smeding, Bosma, Castañeda [3] showed that it is definitely possible to implement beamforming solutions in a mobile system. Sur, Wei, and Zhang took a clever approach for handling the synchronizing problem.…”

“…Using local timestamps from each of the device's network adapter and creating Linear Regression models around these timestamps, they found an Algebraic solution for synchronizing the devices together. One problem they encountered, however, was that the network adapter's timestamp was not always available in some devices [2]. They also mentioned that you could use an external audio beacon to periodically act as a reference for alignment, but they felt that the beacon would interfere with the recording.…”