The Bayes Point Machine for computer-user frustration detection via pressuremouse

Abstract

We mount eight pressure sensors on a computer mouse and collect mouse pressure signals from subjects who fill out web forms containing usability bugs. This approach is based on a hypothesis that subjects tend to apply excess pressure to the mouse after encountering frustrating events. We then train a Bayes Point Machine in an attempt to classify two regions of each user's behavior: mouse pressure where the form-filling process is proceeding smoothly, and mouse pressure following a usability bug. Different from current popular classifiers such as the Support Vector Machine, the Bayes Point Machine is a new classification technique rooted in the Bayesian theory. Trained with a new efficient Bayesian approximation algorithm, Expectation Propagation, the Bayes Point Machine achieves a person-dependent classification accuracy rate of 88%, which outperforms the Support Vector Machine in our experiments. The resulting system can be used for many applications in human-computer interaction including adaptive interface design.

Toward a decision-theoretic framework for affect recognition and user assistance

“…The first group uses traditional classification methods in pattern recognition. The approaches include rule-based systems (Pantic et al, 2002), discriminate analysis (Ark et al, 1999), fuzzy rules (Hudlicka and McNeese, 2002;Massaro, 2000;Elliott et al, 1999), case-based and instancebased learning (Scherer, 1993;Petrushin, 2000), linear and nonlinear regression (Moriyama et al, 1997), neural networks (Petrushin, 1999), Bayesian learning (Qi and Picard, 2002;Qi et al, 2001;Kapoor et al, 2004) and other learning techniques (Heishman et al, 2004). Most of these research efforts focus on the low-level mapping between certain sensory data and the underlying affect.…”

A Cross-Domain Approach to Designing an Unobtrusive System to Assess Human State and Predict Upcoming Performance Deficits

“…If the environment is highly physically demanding, and we are attempting to measure a cognitive state such as cognitive workload, we typically pair a sensor to assess brain activation (e.g., our functional near-infrared spectroscopy device shown in Figure 1 that measures cardiac and brain oxygenation and is designed to be mounted into a standardissue helmet, a baseball cap, a headband, or an Astroanut's cowl) with a peripheral sensor (e.g., our armband device, which fits into a standard mp3 player arm band used for running, and includes sensors for galvanic skin response, motion (accelerometry), blood oxygenation, cardiac information, and environmental temperature). However, if the individual is performing tasks through a computer, we can use additional sensors (e.g., cardiac sensor mounted on a webcam with other standoff methods, such as keyboard analytics (Mota and Picard, 2003), computer mouse tracking (Qi, Reynolds, andPicard, 2001), postural changes (D'Mello, Picard, andGraesser, 2007;Frank, 2007), or voice analytics (Mota and Picard, 2003)). One drawback to these methods is that they are only useful when the operator is interacting directly with a computer, sitting in a chair, or communicating verbally with others.…”

Integrating User Affective State Assessment in Enhancing HCI: Review and Proposition

“…Given the evidence, Bayesian theorem calculates the posterior probability of a hypothesis using the prior probability of hypothesis and the conditional probability of the evidence on the hypothesis. In [42,43], authors described a Bayesian classifier to predict the frustration level of users using the features of mean and variance of the sensory pressure on the mouse. The data distributions are modeled by a mixture of Gaussians.…”