Passive acoustic sensing for tracking knocks atop large interactive displays

Abstract

We describe a system that locates the position of knocks and taps atop a large sheet of glass. Our INTRODUCTIONGlass is now a very common construction material, often used as clear walls for room dividers or large windows bordering urban buildings. The techniques described in this paper aim at enabling these surfaces to become interactive. For example, information displayed on a projection or monitor on the inside of the glass can be determined by knocking appropriately on the outside. A straightforward application of this niche is an interactive storefront, where passers-by can navigate through information on the store's merchandise or explore special offers by knocking appropriately, even when the store is closed.

Engaging consumer through the storefront: Evidences from integrating interactive technologies

“…Both male and female respondents agreed on the same factors influencing their entry decisions, and had a similar reaction towards the introduction of interactive elements directly on the storefront. Summarizing, past studies have argued that innovative technologies are becoming prevalent in the actual retail settings (Kourouthanassis et al, 2007;Papagiannidis et al, 2013;Rese et al, 2014;Demirkan and Spohrer, 2014;Pantano and Viassone, 2015), while proposing only few qualitative testing of storefronts enriched with specific technologies such as virtual mannequins or glasses equipped with haptic sensors (Paradiso et al, 2002;Paradiso and Leo, 2005;Reitberger et al, 2009;Meschtscherjakov et al, 2009). Our research figured out the possible consequences of an interactive technology introduction already in the storefront, independently of the specific system.…”

“…It acts as an effective stimulus able to positively influence consumers' mood and subsequent purchase decision. To achieve this goal, some studies developed new storefront windows equipped with a new technology able to display information inside the window glass according to consumers' knock on the surface (even while the store is closed), by figuring out the large pedestrian usage and appreciation in terms of number of people approaching the window and number of people starting interacting with the system (Paradiso et al, 2002;Paradiso and Leo, 2005). Similarly, few studies compared the interactive and the traditional mannequins on the storefront, by investigating the influence on the interactive ones on time spent in front of the shop window ).…”

SurfaceLink

“…Paradiso and Checka did some of the early work in this area [26]. They proposed a system that tracked a user's knock on a glass wall by measuring the time delay of arrival (TDOA) of vibrations at 4 different contact piezoelectric pickups.…”

Tracking contact and free gesture across large interactive surfaces

“…Figure 1 shows a couple of other sensors that can be used by the system. Since the particular piezoelectric pickups and attached amplifiers that we mounted in the window's corners don't respond well to the very low frequencies present in a fist-bash, the superior low-frequency sensitivity of an attached electrodynamic pickup can be exploited to easily discriminate bash events [11]. Because of the poor impedance match to sounds not produced in the glass itself, the adhered pickups are quite insensitive to extraneous audio.…”

“…Rather than try to constrain the type of knock required (essentially impossible in a public installation), we first classify the impact, then process the data with a heuristically-guided cross-correlation and edge-detection procedure [10]. The resultant system has been seen to locate knocks across 2 meter windows with resolutions of σ = 2.5 cm in 5 mm glass and up to σ = 4 cm in 1 cm glass [11]. Although this hardly warrants as a precision pointing device, this resolution is adequate for the relatively coarse selection needed by the applications to which it was aimed (see below).…”