Human-machine collaboration in intelligence analysis: An expert evaluation

Toniolo, Alice; Cerutti, Federico; Norman, Timothy J.; Oren, Nir; Allen, J. A.; Srivastava, Mani; Sullivan, Paul

doi:10.1016/j.iswa.2022.200151

Cited by 10 publications

(5 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The coefficients were tested for significance using the Pearson criterion. The procedure of expert analysis is not considered in this article in detail, which is due to its wide coverage in literature [3,19].…”

Section: Methodology For Assessing the Performance Of Ttm Statesmentioning

confidence: 99%

Ensuring the efficiency of the system of technical maintenance and repair of transport and technological mashines

ГОЛОТЮК

Shymko

Martyniuk

et al. 2023

The Archives of Automotive Engineering – Archiwum Motoryzacji

View full text Add to dashboard Cite

Current situation with the fleet of transport and technological machinery in Ukraine requires an individual approach to evaluating the effectiveness of its maintenance and repair system. The article is devoted to the issue of selecting the most effective option for the use of transport and technological machinery taking into account the specific conditions of its operation characterized by certain risks and uncertainties, and considering the real volumes and age structure of the fleet. Solving this problem requires substantiation of management models of maintenance and repair processes. It is necessary to provide scientifically based methods of managing the system of technical maintenance (TM) and repair of transport and technological machinery (RTTM) using specific methods of an individual approach to the technical and economic evaluation of the effectiveness of maintenance and RTTM processes which are adapted to the modern conditions of its operation. The article presents the results of the research, which was carried out using the basics of system analysis, the theory of decision-making under the conditions of uncertainty, and the basics of multi-criteria analysis. In the course of research, an analytical management model of the maintenance and repair system operation was formed that reveals the sequence of implementing the management methodology, which makes it possible to assess its state in successive discrete states, to promptly take into account the effect of external influencing factors and make corrections, which, in turn, allow increasing the validity of strategic decisions aimed at increasing the effectiveness. A numerical calculation was performed, which allowed us to conclude that the value of maintenance intervals has a significant impact on the indicator of effectiveness. Adjusting this value makes it possible to optimize maintenance and repair processes.

show abstract

Section: Methodology For Assessing the Performance Of Ttm Statesmentioning

confidence: 99%

Ensuring the efficiency of the system of technical maintenance and repair of transport and technological mashines

ГОЛОТЮК

Shymko

Martyniuk

et al. 2023

The Archives of Automotive Engineering – Archiwum Motoryzacji

View full text Add to dashboard Cite

show abstract

“…Automatic tools assist intelligence analysts in data collection [5][6][7], data inspection and visualization [8], evidence credibility assessment [9,10], scenariobased question answering [11], hypotheses generation [12,13], and quality evaluation [14]. However, most current tools still require expensive crowdsourcing [15,16] or human-machine collaboration [17][18][19] in report generation.…”

Section: Related Workmentioning

confidence: 99%

“…Traditionally, a situation report requires foraging for different claims and hypotheses from the source documents (i.e., news articles) that help explain a situation [19], which is expensive to obtain through manual crowd-sourcing. However, recent work [47,48] has shown that directed queries, such as strategic questions in our case, can be used to automatically extract claims from news articles relevant to a particular topic.…”

Section: Question-driven Claim Extraction With Validationmentioning

confidence: 99%

SmartBook: AI-Assisted Situation Report Generation

Reddy¹,

Fung²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

Emerging events, such as the COVID pandemic and the Ukraine Crisis, require a time-sensitive comprehensive understanding of the situation to allow for appropriate decision-making and effective action response. Automated generation of situation reports can significantly reduce the time, effort, and cost for domain experts when preparing their official human-curated reports. However, AI research toward this goal has been very limited, and no successful trials have yet been conducted to automate such report generation. Pre-existing natural language processing methods, large language model based text generation, and information retrieval techniques are insufficient to identify, locate, and summarize important information, and lack detailed, structured, and strategic awareness. We propose SmartBook, a novel task formulation targeting situation report generation, which consumes large volumes of news data to produce a structured situation report with multiple hypotheses (claims) summarized and grounded with rich links to factual evidence. We realize SmartBook for the Ukraine-Russia crisis by automatically generating intelligence analysis reports to assist expert analysts. The machine-generated reports are structured in the form of timelines, with each timeline organized by major events (or chapters), corresponding strategic questions (or sections) and their grounded summaries (or section content). Our proposed framework automatically detects realtime event-related strategic questions, which are more directed than manually-crafted analyst questions, which tend to be too complex, hard

show abstract

“…AI and humans should be able to interact with each other effortlessly. A human-in-the-loop setup that takes advantage of both AI and human capabilities could indeed be the ideal solution (Toniolo et al, 2023). The integration of AI with human ingenuity is a promising area of future research.…”

Section: Human In the Loopmentioning

confidence: 99%

“…Humans prefer to oversee decision-making; hence it is unlikely that all AI systems will run without any human input. Current literature in the area of intelligent systems has started to focus on human empowerment, including a stronger focus on augmentation instead of pure automation, which would be beneficial for humanity (Holmes et al, 2021;Maciej Serda et al, 2022;Toniolo et al, 2023). Overall, the goal is to expand the discussion in the hope to trigger new conversations and ultimately convincing more researchers to think about how to incorporate ML models within business processes.…”

Section: Introductionmentioning

confidence: 99%

Automated Machine Learning: Ai-Driven Decision Making in Business Analytics

Schmitt

2022

SSRN Journal

View full text Add to dashboard Cite

Human-machine collaboration in intelligence analysis: An expert evaluation

Cited by 10 publications

References 54 publications

Ensuring the efficiency of the system of technical maintenance and repair of transport and technological mashines

Ensuring the efficiency of the system of technical maintenance and repair of transport and technological mashines

SmartBook: AI-Assisted Situation Report Generation

Automated Machine Learning: Ai-Driven Decision Making in Business Analytics

Contact Info

Product

Resources

About