Low overhead high yield integrated surveys

Abstract

Traditionally, a system of survey activities is often conducted as a separate process, apart from the target system, which includes explicit invitations to a predefined or randomly selected group of users, collection of data from submitted forms and compilation into various analytical reports for decision making. Such a process is expensive and time consuming. Also, since traditional surveys are often conducted after the system usage, the users' experiences with the system become somewhat detached in responding to survey invitations at a later time, causing the concentrations and interests to decrease during this period. So it is likely that they may overlook some vital areas in survey questionnaires where it is most expected for them to respond with pragmatic thoughts within the context of the system environment; thus, straying away from the actual purpose of the survey. To eliminate these difficulties, the integration capabilities of any computer-based system can be carefully utilized for accumulating system usage data, workflow path traversal, calculating various feature ranks, as well as lightweight feedbacks at users' own will while they are using the system. In addition, displaying significant real-time system usage statistics, as well as historical data at relevant areas in the system workflow enables the users to be more aware of the dynamicity of the system components, which implicitly encourages them to drop few responses to very simple questions while traversing through the system. In this paper, we propose a framework for an integrated survey mechanism for computer-based systems that is able to serve most of the purposes of conducting manual surveys for the assessment of the system itself, as well as its users.

Dynamics of System Evolution

“…For a system to be evolvable, it needs to be more flexible in interaction with not only the end users, but also various self-contained meta-data collecting agents or data-loggers (Lehman, 1986). These might include automated survey agents, probing points, or task request history (Mubin & Luo, 2010a). Table 1 lists some of the desirable characteristics of an evolvable system.…”

“…Figure 1 shows an example of such placement of a wrapper system. A set of automated surveyagents and probing stations (Mubin & Luo, 2010a) are responsible for capturing various usage factors from the system itself (such as, access rates, feature ranks, utilization factors, path traversals), from its operational environment (such as, system users, work habits, workstations, workspaces, user feedbacks, new system requirements, user satisfaction index), and from the external environment (with influential factors). …”

“…One or more causes will initiate a trigger, which in turn may cause another trigger from that point. Based on the collected data from survey-agents, probing stations and other means (Mubin & Luo, 2010a) the knowledgebase is responsible for identifying a cause in {C T0 ..C Tn }, where C T0 being no cause (nothing significant happened). The influential cause(s) of the transition, as well as the factor(s) in the operational environment, are archived into the knowledgebase for contextual analyses at a later point.…”

“…Hence, a general rule for mapping workflow processes, decision points and control paths can be defined by the system management team with the main objective that covers all areas with a set of nodes and edges of a graph model and apply various metrics related to graph models (such as McCabe's Cyclomatic index). Then at the junction points in the workflow, we may place data-loggers (Lehman, 1986) or, more specifically, probing stations or survey agents (Mubin & Luo, 2010a). …”