Best practices for US Department of Defense model validation: lessons learned from irregular warfare models

Modern warfighting is not only conducted on the physical battlefield but also engages actors in the information and human landscapes. Across these landscapes many types of actors characterize a comprehensive environment. A commander and staff follow the operations planning process to translate a desired end-state to tactical effects and activities. Understanding the interactions between actors and factors that shape conflict and understanding how they can be influenced is highly challenging. Modeling and simulation methods that aim to describe and understand the behavior of complex systems can serve to structure information, derive problem insights, and identify effective interventions. These methods are not, however, common as part of operations planning and intelligence processes. This paper documents findings based on reported experiences from practice and based on concept development and experimentation workshops conducted with Dutch Armed Forces stakeholders. We identify eight challenges to the adoption of modeling and simulation in operations planning. We then present a decision support concept that incorporates several elements to counter the challenges identified in the form of a prototype approach called Comprehensive Operations Support with Modeling and Simulation (COSMOS). We describe the approach along several process steps, illustrated by examples from the concept development and experimentation case studies. We identify several avenues for future research assisting adoption of modeling and simulation in the operations planning process.

“…Detailed process steps and validation and verification techniques are also available in the modeling methodology literature. 49,50…”

Section: A Prototype Decision Support Environment: the Cosmos Conceptmentioning

confidence: 99%

Section: A Prototype Decision Support Environment: the Cosmos Conceptmentioning

confidence: 99%

Concept development for comprehensive operations support with modeling and simulation

Veldhuis

Reus

Keijser

2018

“…Recently completed research on DoD validation of IW models emphasized the need for DoD modelers to ensure their conceptual models had the appropriate referents, 2 that is, social science theories that had widespread acceptance in their communities of practice. Several DoD IW modeling efforts had been found either to have no social science underpinnings or to take form of proprietary software wherein the underlying conceptual models were not available for scrutiny.…”

Section: Introductionmentioning

confidence: 99%

A demonstration of ABM validation techniques by applying docking to the Epstein civil violence model

Appleget

Burks

Jaye

2013

Self Cite

The increased focus of the United States Department of Defense (DoD) on irregular warfare and counterinsurgency has served to identify the lack of credible models and simulations to represent the relevant civilian populations -the centers of gravity of such operations. While agent-based models (ABMs) have enjoyed widespread use in the social science community, many senior DoD officials are skeptical that agent-based models can provide useful tools to underpin DoD analysis, training, and acquisition needs mainly because of validation concerns. This paper uses docking and other forms of alignment that enable the linking of the Epstein civil violence agent-based model results to other models. These examples of model-to-model analysis could serve to assist and encourage DoD ABM human domain model validation efforts. BackgroundIn 2006, as the campaigns in Iraq and Afghanistan wore on, the first codification of the doctrine that was being developed and applied was published in FM 3-24/MCWP 3-33.5, Counterinsurgency. This document made clear that the focus of a counterinsurgency campaign is on influencing local populations. Senior US Department of Defense (DoD) officials, who sought to better understand how their armed forces should be organized, trained and equipped for such operations, looked to their modeling and simulation community for help. That community, well-equipped to examine kinetic operations, had little to contribute in this domain. Modeling human and social behavior quickly became the focus of the DoD modeling and simulation (M&S) community, and agent-based models (ABMs) began to receive increased interest and scrutiny. ABMs have enjoyed popularity within several areas of the social sciences, and their use, coupled with the situations in Iraq and Afghanistan, has attracted the attention of DoD.This increased DoD focus on irregular warfare (IW) and counterinsurgency has served to identify the lack of credible models and simulations to represent civilian populations in conflict environments. In addition, the lack of social science expertise to inform DoD M&S efforts and the lack of data to represent social science phenomenon have also been identified as critical gaps affecting DoD's ability to model IW-like scenarios. While agentbased models have enjoyed widespread use in the social science community, many senior DoD officials are skeptical that agent-based models can provide useful tools to underpin DoD analysis, training, and acquisition needs.A key requirement for any M&S used by DoD is that it be validated. 1 DoD's definition of validation is ''The process of determining the degree to which a model or simulation and its associated data are an accurate representation

Identifying appropriate scenario termination rules for squad-level simulations of warfighter lethality

Biggs,

Hamilton,

Markwald

2023

Small arms combat simulations are emerging as a method to convert raw human performance metrics into quantifiable lethality outcomes. Specifically, speed and accuracy measures can be directly converted into a percentage chance of winning a gunfight using small arms combat modeling. Although the technique provides an opportunity to clearly communicate implications to a military audience, there are numerous assumptions to this modeling technique that require further exploration. One critical assumption involves the termination rule, or criterion to identify when the combat simulation should end. The termination rule must be clearly identified because it will impact the casualties inflicted and suffered by each side during simulation. Here, we discuss the implications of three broad categories of termination rules: (1) total victory, (2) retreat criteria, and (3) resource limitations. Each approach has different scenarios that might be more or less advantageous to use as a termination rule. Taken together, this information should help improve lethality modeling procedures using different simulation techniques to convert human performance metrics into quantifiable chances of winning a combat engagement.