CE-SAM: a conversational interface for ISR mission support

Pizzocaro, Diego; Parizas, Christos; Preece, Alun; Braines, Dave; Mott, David; Bakdash, Jonathan Z.

doi:10.1117/12.2015686

Cited by 5 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We refer to our previous work 2 and latest paper 3 for more details about CE-SAM. The output of CE-SAM is the mission-resource matching graph in Figure 1, which is utilized by the Resource Allocation Solver component.…”

Section: Figure 2 System Architecture and Work Flowsmentioning

confidence: 99%

“…In our system, we apply a new interactive interface, CE-SAM (Controlled English Sensor Assignment to Missions), developed from the work of A. Preece et al 2 and refined in D. Pizzocaro et al 3 , which guides users to 1) build a scalable knowledge base of various ISR resource functionalities and corresponding ISR requirements which such resources can support, 2) refine their ISR missions without requiring a technical background in formal query languages or ontology building, and 3) match their missions with proper bundles of resources represented in the knowledge base.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A system architecture for decision-making support on ISR missions with stochastic needs and profit

Pizzocaro

Porta

et al. 2013

SPIE Proceedings

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a system architecture for decision-making support on ISR (i.e., Intelligence, Surveillance, Reconnaissance) missions via optimizing resource allocation. We model a mission as a graph of tasks, each of which often requires exclusive access to some resources. Our system guides users through refinement of their needs through an interactive interface. To maximize the chances of executing new missions, the system searches for pre-existent information collected on the field that best fit the needs. If this search fails, a set of new requests representing users' requirements is considered to maximize the overall benefit constrained by limited resources. If an ISR request cannot be satisfied, feedback is generated to help the commander further refine or adjust their information requests in order to still provide support to the mission. In our work, we model both demands for resources and the importance of the information retrieved realistically in that they are not fully known at the time a mission is submitted and may change overtime during execution. The amount of resources consumed by a mission may not be deterministic; e.g., a mission may last slightly longer or shorter than expected, or more of a resource may be required to complete a task. Furthermore, the benefits received from the mission, which we call profits, may also be non-deterministic; e.g., successfully localizing a vehicle might be more important than expected for accomplishing the entire operation. Therefore, when satisfying ISR requirements we take into account both constraints on the underlying resources and uncertainty of demands and profits.

show abstract

Section: Figure 2 System Architecture and Work Flowsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A system architecture for decision-making support on ISR missions with stochastic needs and profit

Pizzocaro

Porta

et al. 2013

SPIE Proceedings

Self Cite

View full text Add to dashboard Cite

show abstract

“…In some cases, terms used by the user may not be interpretable in the modelthe word "suspicious" here has been ignored. In such cases, a more elaborate conversational interaction could be used to extend the model [21] where appropriate.…”

Section: Use Cases Revisitedmentioning

confidence: 99%

Conversational sensing

Preece¹,

Gwilliams²,

Parizas³

et al. 2014

SPIE Proceedings

Self Cite

View full text Add to dashboard Cite

Recent developments in sensing technologies, mobile devices and context-aware user interfaces have made it possible to represent information fusion and situational awareness as a conversational process among actors -human and machine agents -at or near the tactical edges of a network. Motivated by use cases in the domain of security, policing and emergency response, this paper presents an approach to information collection, fusion and sense-making based on the use of natural language (NL) and controlled natural language (CNL) to support richer forms of human-machine interaction. The approach uses a conversational protocol to facilitate a flow of collaborative messages from NL to CNL and back again in support of interactions such as: turning eyewitness reports from human observers into actionable information (from both trained and untrained sources); fusing information from humans and physical sensors (with associated quality metadata); and assisting human analysts to make the best use of available sensing assets in an area of interest (governed by management and security policies). CNL is used as a common formal knowledge representation for both machine and human agents to support reasoning, semantic information fusion and generation of rationale for inferences, in ways that remain transparent to human users. Examples are provided of various alternative styles for user feedback, including NL, CNL and graphical feedback. A pilot experiment with human subjects shows that a prototype conversational agent is able to gather usable CNL information from untrained human subjects.

show abstract

“…SAM uses a controlled natural language as a common human and machinereadable representation of knowledge, thus it is likely to have greater transparency to humans than black box AI. In addition, an interactive conversational interface for SAM is under development, allowing users to change and update ISR allocation tasks [31]. We plan to conduct behavioral research to assess and iteratively improve SAM for human cognitive performance (see [32]).…”

Section: B Adapative Decision Aidsmentioning

confidence: 99%

Human Factors in Intelligence, Surveillance, and Reconnaissance: Gaps for Soldiers and Technology Recommendations

Bakdash

Pizzocaro

Precee

2013

MILCOM 2013 - 2013 IEEE Military Communications Conference

Self Cite

View full text Add to dashboard Cite

Abstract-We investigate the gaps for Soldiers in information collection and resource management for Intelligence, Surveillance, and Reconnaissance (ISR). ISR comprises the intelligence functions supporting military operations; we concentrate on ISR for physical sensors (air and ground platforms). To identify gaps, we use approaches from Human Factors (interactions between humans and technical systems to optimize human and system performance) at the level of Soldier functions/activities in ISR. Key gaps (e.g., the loud auditory signatures of some air assets, unofficial ISR requests, and unintended battlefield effects) are identified. These gaps illustrate that ISR is not purely a technical problem. Instead, interactions between technical systems, humans, and the environment result in unpredictability and adaptability in using technical systems. To mitigate these gaps, we provide technology recommendations.

show abstract

CE-SAM: a conversational interface for ISR mission support

Cited by 5 publications

References 13 publications

A system architecture for decision-making support on ISR missions with stochastic needs and profit

A system architecture for decision-making support on ISR missions with stochastic needs and profit

Conversational sensing

Human Factors in Intelligence, Surveillance, and Reconnaissance: Gaps for Soldiers and Technology Recommendations

Contact Info

Product

Resources

About