GestureFlow: Streaming gestures to an audience

Abstract

Multi-touch mobile devices (e.g. iPhone and iPad) and motion-sensing game controllers (e.g. Kinect for Xbox 360) share one common feature: users interact with computing devices in non-conventional gesture-intensive ways, be they multi-touch gestures on the iPad or body motion gestures with the Kinect. As a new way to interact with computing devices, gestures have been proven to be intuitive and natural, with very minimal learning curve. They can be used in applications beyond games, such as those that allow the creation of artistic and musical content in a collaborative fashion. In order for multiple users to collaborate or compete in real time, however, such gestures need to be streamed in multiple broadcast sessions with an "all-to-all" broadcast nature, with each session corresponding to one of users as a source of a gesture stream. These streams of gestures typically incur low yet bursty bit rates, but have unique requirements with respect to delay and loss. In this paper, we present the design of GestureFlow, a gesture broadcast protocol designed specifically for concurrent gesture streams in multiple broadcast sessions. We motivate the effectiveness and practicality of using inter-session network coding, and address challenges introduced by linear dependence, discovered in our extensive experiments involving a new gesture-intensive iPad application that we developed from scratch.

Cooperative video streaming on smartphones

“…A toolkit to make network coding practical for system devices from servers to smartphones is introduced in [32]. A gesture broadcast protocol is designed for concurrent gesture streams in multiple broadcast sessions in [33] over smartphones using inter-session network coding. Our work is different in (i) the application scenario, i.e., cooperative video streaming on smartphones using network coding, (ii) the fact that we combine application layer network coding with broadcast in the local connections, and (iii) the fact that we combine NUM and implementation.…”

MicroCast: Cooperative Video Streaming Using Cellular and Local Connections

“…[21] extends [20] for picture transmission. In [50], a gesture broadcast protocol is designed for concurrent gesture streams in multiple broadcast sessions over smartphones using inter-session network coding.…”

On detecting pollution attacks in inter-session network coding

“…For instance, in COPE [1], every encoded packet is decoded at the next hop. There are also other coding schemes where encoded packet are decoded by either the first hop or the second hop, e.g., see [2] and [3]. Furthermore, in inter-session coding, source packets of a source S i may not traverse the whole network but only some parts of the network: for instance, in a directed acyclic graph, packets sent from S i should not travel to nodes that have no path to R i .…”

“…Per-packet per-node computation overhead of the signature scheme [27], the hybrid scheme in [20] (c = 1, δ = 0.1, = 0.01), our hash-based scheme, and InterMac CPK /SpaceMac PM when used with RIPPLE [18]. multiple" method for calculating exponentiation over a finite field F q , each exponentiation over F q takes approximately 3 2 |q| multiplications on average [20]. The total average overhead is 3 2 |q|(n + sg 2 + s) field multiplications.…”