No abstract

In 2010, IT-security experts from northern European governments and organizations gathered to conduct the first of a series of NATO-led cyber-defense exercises in a pilot attempt of training cyber defense. To gain knowledge on how to assess team effectiveness in cyber-defense exercises, this case study investigates the role of behavioral assessment techniques as a complement to task-based performance measurement. The collected data resulted in a massive data set including system logs, observer reports, and surveys. Six different methods were compared for feasibility in assessing the teams' performance, including automated availability check, exploratory sequential data analysis, and network intrusion detection system attack analysis. In addition, observer reports and surveys were used to collect aspects relating to team structures and processes, aiming to discover whether these aspects can explain differences in effectiveness. The cross-disciplinary approach and multiple metrics create possibilities to study not only the performance-related outcome of the exercise, but also why this result is obtained. The main conclusions found are (1) a combination of technical performance measurements and behavioral assessment techniques are needed to assess team effectiveness, and (2) cyber situation awareness is required not only for the defending teams, but also for the observers and the game control.

Human factors research popularly employs perception-based techniques to investigate team performance and its dependency to cognitive processes. Such studies frequently rely upon either observer-based or self-assessment techniques to collect data. In this study, we examined behavioral observer ratings and self-assessment ratings for measuring team performance in virtual teams, with team performance regarded as a combination of task outcome and team cognition. Juxtaposing self-assessments and observer ratings from a quasi-experiment comparing team performance rating techniques reveals that they indeed produce overall similar results, with both singling out teamwork effectiveness ratings as the strongest contributor to overall team performance. However, the comparisons show remarkably low correlation on individual questionnaire items. The most striking difference is that the team members' self-assessments of workload are lower than the corresponding observer ratings. In particular, the selfassessments do not correlate at all with overall team performance, whereas the observers' workload ratings are more consistent with contemporary research that suggests a strong correlation between workload and team performance, suggesting that observer-based techniques are more reliable than self-assessments for assessing workload. For other ratings, the results show that the two techniques are fairly equal, suggesting that the choice between methods to employ can be deferred to other considerations such as obtrusiveness, accessibility, and resource availability.

No abstract

It is hardly an exaggeration to claim that one of the most turbulent political areas in recent years has been asylum policy, which has disclosed a rapidly increasing inflow of asylum seekers, and, in many countries, has been followed by fierce media discussion and political controversies. In Sweden, this development has been heated as the Swedish self‐image is one of providing generous policies, which is also reflected in terms of strong refugee policy. The article uses this example to explore assumptions about public responsiveness in previous policy feedback literature and to examine the link between citizens' attitudes towards immigration and changes in asylum policy output, measured as asylums granted, over time in the period 1990–2015. It focuses especially on the link through which citizens become aware of policy output, operationalized as media visualization, and find that including media reveals a suppressed relationship between policy output and public attitudes. The relationship is negative and thus confirms the assumptions of the thermostatic models. Second, the article shows that feedback is mediated by political orientation: People defining themselves politically as right‐oriented respond with negative feedback when the number of granted asylums increases, while left‐oriented people do not change their attitudes. Based on these findings it is concluded, first, that analyses of democratic responsiveness need to incorporate a clear measure of the link by which exogenous factors become visible. Second, the importance needs to be stressed of considering important cleavages in the population in order to display responsiveness processes fairly.