A Study on Automatic Classification of Users’ Desktop Interactions

Abstract

Knowledge workers frequently change activities, either by choice or through interruptions. With an increasing number of activities and activity switches, it is becoming more and more difficult for knowledge workers to keep track of their desktop activities. This article presents our efforts to achieve activity awareness through automatic classification of user's everyday desktop activities. For getting a deeper understanding, we investigate performance of various classifiers with respect to discriminative power of time-, interaction-, and content-based feature sets for different work scenarios and users. Specifically, by viewing an activity as a sequence of desktop interactions we present (1) a methodology for translating a user's desktop interactions into activities, (2) evaluation of the discriminative power of different activity features and feature types, and (3) analysis of supervised classification models for classifying desktop activity under two different scenarios, i.e., an activity-centric scenario and a user-centric scenario. The experiments are carried out on a real-world dataset, and the results show satisfactory accuracy using relatively few and simple types of features.

Computer detection of spatial visualization in a location-based task

“…Interface events and interface components were highly ranked as classification attributes. A recent report on task detection appeared in Mirza, Chen, Hussain, Majid, & Chen (2015), where the authors attempted to discriminate between desktop activities during multitasking. While there is superficial likeness between papers in this area and our research -the user modeling features include graphical user interface events -the goals in the field are completely dissimilar from our direction of investigation, and therefore the utility of the literature is limited to possibly intriguing data-mining techniques.…”

Detecting Developers’ Task Switches and Types

“…The results of a field study with 11 researchers and an average of 10 hours of data per participant shows that the prediction is very individual and that a general classifier does not work well. Mirza et al [33] focused on classifying users' desktop interactions into six higher level activity types (not task types): writing, reading, communicating, web browsing, system browsing and miscellaneous. They used temporal, interaction-based (application window events), and semantic features calculated over a 5 minute time window.…”

“…Capturing participants' confidence served as an indicator of the quality and accuracy of their self-reports. The user interface we used to collect the ground truth for task switches and types resembles the one by Mirza et al [15], [25], [33].…”

“…Type All Import. Type UI User Input Features 45.8% 52.0% 81.0% Keystroke differences (4): difference in the number of navigate/backspace/normal/total keystrokes pressed per second between the previous and current application/task segment [22], [32], [33], [46] 16.4% 19.1% 29.9%…”

“…Time spent per application category (13): the percentage of the total duration of the task segment that was spent in each of the 13 application categories [14], [32], [33] NA 34.3% NA Switching Frequency Features 16.6% 13.7% 19.0% Difference in the window switches frequency (1): difference of the number of switches between windows of the same or a different application per second between the current and the previous application/task segment [14], [27], [32] 7.2% 13.7% 19.0%…”