Evaluation of latent semantic analysis-based measures of team communications content

Abstract

Team process is thought to mediate team member inputs and team performance. Among the team behaviors identified as process variables, team communications have been widely studied. We view team communications as a team behavior and also as team information processing, or team cognition. Within the context of a Predator Uninhabited Air Vehicle (UAV) synthetic task, we have developed several methods of communications content assessment based on Latent Semantic Analysis (LSA). These methods include: Communications Density (CD) which is the average task relevance of a team's communications, Lag Coherence (LC) which measures taskrelevant topic shifting over UAV missions, and Automatic Tagging (AT) which categorizes team communications. Each method is described in detail. CD and LC are related to UAV team performance. AT-human is comparable to human-human agreement on content coding. The results are promising for the assessment of teams based on LSA applied to communication content.

Measuring Patterns in Team Interaction Sequences Using a Discrete Recurrence Approach

“…If interaction patterns develop as people continue working together as a team (e.g., Achille, Schultz, & Schmidt-Nielson, 1995;Gorman, Foltz, Kiekel, Martin, & Cooke, 2003;Katz, 1982), then changes in team membership may be reflected in changes in the quantity and informational content of those patterns. Because team interaction patterns recur in time, often in discontinuous bursts and lulls of interactivity (Huberman & Glance, 1998), nonlinear dynamics may provide appropriate tools for quantifying pattern changes.…”

Beyond frequency counts: Novel conceptual recurrence analysis metrics to index semantic coordination in team communications

“…However, CRA differs from these approaches in that it evaluates the semantic similarity of utterances, or the alignment of the meaning of the content of the utterances, rather than syntactical, morphological, or lexical alignment (however, see Dale et al, 2018, for an extension of their categorical recurrence approach in combination with word2vec to analyze semantic coordination). CRA is similar in this regard to other semantic similarity measures like LSA (Foltz, 1996;Gorman et al, 2003;Gorman et al, 2016).…”

“…However, these approaches also have limitations. For instance, although LSA is a powerful tool for evaluating team communications (e.g., Foltz & Martin, 2009;Gorman, Foltz, Kiekel, Martin, & Cooke, 2003;Gorman, Martin, Dunbar, Stevens, & Galloway, 2016), it can require sizeable semantic spaces built from a large corpus of relevant documents prior to analyzing the data of primary interest. These semantic spaces can be nontrivial to build (Gefen, Endicott, Fresneda, Miller, & Larsen, 2017;Quesada, 2007).…”

Interaction-Based Measures of Cognitive Systems

“…We applied latent semantic analysis to analyze the content of team discourse (Gorman, Foltz, Kiekel, Martin, & Cooke, 2003). Latent semantic analysis is a statistical technique for extracting "latent" structure of word usage across contexts (Berry, 1992;Deerwester, Dumais, Furnas, Landauer, & Harshman, 1990;Landauer & Dumais, 1997;Landauer, Foltz, & Laham, 1998).…”