Crowd-Powered Hybrid Classification Services: Calibration is all you need

Sayin, Burcu; Krivosheev, Evgeny; Ramírez, Jorge; Casati, Fabio; Taran, Ekaterina A.; Malanina, Veronika A.; Yang, Jie

doi:10.1109/icws53863.2021.00019

Cited by 3 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By saying value we mean the value HI systems add to the real-world system deployed, and it consists of several components, such as trust, complementarity, cost-sensitive learning, etc. This has been addressed in our recent papers [5,6,7,8] that propose a novel way of measuring the value of HI systems, and we show that the existing accuracy-based metrics are not compatible with HI systems. We further discuss open research questions to invite the HI community to pay enough attention to how and why HI models are used in practice, and to the aspects and metrics that are relevant to enterprises when they adopt and deploy a model.…”

Section: Introductionmentioning

confidence: 91%

See 1 more Smart Citation

Value-Based Hybrid Intelligence

Sayin

Yang

Passerini

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 91%

“…HI is widely adopted in enterprise scenarios as selective classifiers [4,5], where machine learning (ML) models can abstain from making a prediction and resort to human judgment or a default path (as in Figure 1). We argue that current metrics to evaluate such systems (e.g.…”

Section: Introductionmentioning

confidence: 99%

Value-Based Hybrid Intelligence

Sayin

Yang

Passerini

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…A whole range of different approaches has been developed in the healthcare information service, including mathematical models [12] and data-driven models [13]. However, these approaches are necessary but insufficient to solve the service resource allocation problem due to the personal differences, process optimization in the long-term service, and the individual changing engagement behavior.…”

Section: Related Workmentioning

confidence: 99%

Service resource allocation problem in the IoT driven personalized healthcare information platform

Ji¹,

Lee²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

With real-time monitoring of the personalized healthcare condition, the IoT wearables collect the health data and transfer it to the healthcare information platform. The platform processes the data into healthcare recommendations and then delivers them to the users. The IoT structures in the personalized healthcare information service allows the users to engage in the loop in servitization more convenient in the COVID-19 pandemic. However, the uncertainty of the engagement behavior among the individual may result in inefficient of the service resource allocation. This paper seeks an efficient way to allocate the service resource by controlling the service capacity and pushing the service to the active users automatically. In this study, we propose a deep reinforcement learning method to solve the service resource allocation problem based on the proximal policy optimization (PPO) algorithm. Experimental results using the real world (open source) sport dataset reveal that our proposed proximal policy optimization adapts well to the users' changing behavior and with improved performance over fixed service resource policies.

show abstract

Calibrated bagging deep learning for image semantic segmentation: A case study on COVID-19 chest X-ray image

Nwosu

Qian

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causes coronavirus disease 2019 (COVID-19). Imaging tests such as chest X-ray (CXR) and computed tomography (CT) can provide useful information to clinical staff for facilitating a diagnosis of COVID-19 in a more efficient and comprehensive manner. As a breakthrough of artificial intelligence (AI), deep learning has been applied to perform COVID-19 infection region segmentation and disease classification by analyzing CXR and CT data. However, prediction uncertainty of deep learning models for these tasks, which is very important to safety-critical applications like medical image processing, has not been comprehensively investigated. In this work, we propose a novel ensemble deep learning model through integrating bagging deep learning and model calibration to not only enhance segmentation performance, but also reduce prediction uncertainty. The proposed method has been validated on a large dataset that is associated with CXR image segmentation. Experimental results demonstrate that the proposed method can improve the segmentation performance, as well as decrease prediction uncertainty.

show abstract

Crowd-Powered Hybrid Classification Services: Calibration is all you need

Cited by 3 publications

References 16 publications

Value-Based Hybrid Intelligence

Value-Based Hybrid Intelligence

Service resource allocation problem in the IoT driven personalized healthcare information platform

Calibrated bagging deep learning for image semantic segmentation: A case study on COVID-19 chest X-ray image

Contact Info

Product

Resources

About