Algorithmic bias: review, synthesis, and future research directions

Kordzadeh, Nima; Ghasemaghaei, Maryam

doi:10.1080/0960085x.2021.1927212

Cited by 257 publications

(133 citation statements)

References 81 publications

Supporting

Mentioning

124

Contrasting

Unclassified

Order By: Relevance

“…We, thereby, built on previously established fairness rules [7,22], which provided a thorough base, but extended and adapted those to an AI-based service context. The majority of our AI fairness rules are allocated to procedural fairness, which provides an indication for the importance of the dimension [10,45]. Active demand for the rules of interpersonal AI fairness was the least proactively mentioned by our interviewees.…”

Section: Discussionmentioning

confidence: 97%

“…We believe that all rules are relevant to a holistic AI fairness perception of e-commerce services, but the rules might not be limited to e-commerce services. We recognize that certain rules, such as security, are also relevant in the context of other technology-based services [10]. Furthermore, even though the independency and bias suppression rules are of importance and even though the interviewees would appreciate these rules, they are also aware that a provider of an AI-based e-commerce service most likely pursues economic interests.…”

Section: Discussionmentioning

confidence: 98%

“…Discrimination by AI can also occur in the e-commerce context, by providing products at different prices based on willingness to pay information derived from customer data [27]. Driven by the increasing presence of AI-based services and the indicated unfairness potential of AI, an increase in research on fair AI in information systems, computer science, and management emerged [9,10]. While a predominant focus of research is on the field of human resources (HR) [9], there is a lack of focus not only on ecommerce services [4,28] but also on the perception of AI fairness [9,10].…”

Section: Fair Aimentioning

confidence: 99%

“…This refers mainly to the procedural and distributive fairness dimensions. There is a notable focus on these two dimensions, and a lack of considering the interactional perspective (interpersonal, informational) [9,10]. Furthermore, Robert et al [9] emphasize the neglect of interactional fairness as a potential risk.…”

Section: Fair Aimentioning

confidence: 99%

“…Research on fair AI has experienced a notable increase over the last years [9,10], fueled not only by the diffusion of AI across industries and corporate tasks but also by popular examples of AI-induced unfairness, such as the decommissioned recruiting tool by Amazon [11]. Research on fair AI primarily focused on a technical perspective, whereas research on the customer-centric perspective including the customers' perception of fairness remains scarce [9,10]. The latter is often limited to only two (procedural, distributive) among the four fairness dimensions [9,12,13,14].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

When Do Customers Perceive Artificial Intelligence as Fair? An Assessment of AI-based B2C E-Commerce Services

Weith¹,

Matt²

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Artificial intelligence (AI) enables new opportunities for business-to-consumer (B2C) ecommerce services, but it can also lead to customer dissatisfaction if customers perceive the implemented service not to be fair. While we have a broad understanding of the concept of fair AI, a concrete assessment of fair AI from a customer-centric perspective is lacking. Based on systemic service fairness, we conducted 20 in-depth semi-structured customer interviews in the context of B2C e-commerce services. We identified 19 AI fairness rules along four interrelated fairness dimensions: procedural, distributive, interpersonal, and informational. By providing a comprehensive set of AI fairness rules, our research contributes to the information systems (IS) literature on fair AI, service design, and humancomputer interaction. Practitioners can leverage these rules for the development and configuration of AI-based B2C e-commerce services.

show abstract

Section: Discussionmentioning

confidence: 97%

Section: Discussionmentioning

confidence: 98%

Section: Fair Aimentioning

confidence: 99%

Section: Fair Aimentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

When Do Customers Perceive Artificial Intelligence as Fair? An Assessment of AI-based B2C E-Commerce Services

Weith¹,

Matt²

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

Availability of Evidence for Predictive Machine Learning Algorithms in Primary Care

Rakers,

van Buchem,

Kucenko

et al. 2024

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceThe aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow.ObjectivesTo systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle.Evidence ReviewPubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores.FindingsThe systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%).Conclusions and RelevanceThe findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms’ quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

show abstract

Individual-Level Risk Prediction of Return to Use During Opioid Use Disorder Treatment

Luo,

Feaster,

Liu

et al. 2024

JAMA Psychiatry

View full text Add to dashboard Cite

ImportanceNo existing model allows clinicians to predict whether patients might return to opioid use in the early stages of treatment for opioid use disorder.ObjectiveTo develop an individual-level prediction tool for risk of return to use in opioid use disorder.Design, Setting, and ParticipantsThis decision analytical model used predictive modeling with individual-level data harmonized in June 1, 2019, to October 1, 2022, from 3 multicenter, pragmatic, randomized clinical trials of at least 12 weeks’ duration within the National Institute on Drug Abuse Clinical Trials Network (CTN) performed between 2006 and 2016. The clinical trials covered a variety of treatment settings, including federally licensed treatment sites, physician practices, and inpatient treatment facilities. All 3 trials enrolled adult participants older than 18 years, with broad pragmatic inclusion and few exclusion criteria except for major medical and unstable psychiatric comorbidities.InterventionAll participants received 1 of 3 medications for opioid use disorder: methadone, buprenorphine, or extended-release naltrexone.Main Outcomes and MeasuresPredictive models were developed for return to use, which was defined as 4 consecutive weeks of urine drug screen (UDS) results either missing or positive for nonprescribed opioids by week 12 of treatment.ResultsThe overall sample included 2199 trial participants (mean [SD] age, 35.3 [10.7] years; 728 women [33.1%] and 1471 men [66.9%]). The final model based on 4 predictors at treatment entry (heroin use days, morphine- and cocaine-positive UDS results, and heroin injection in the past 30 days) yielded an area under the receiver operating characteristic curve (AUROC) of 0.67 (95% CI, 0.62-0.71). Adding UDS in the first 3 treatment weeks improved model performance (AUROC, 0.82; 95% CI, 0.78-0.85). A simplified score (CTN-0094 OUD Return-to-Use Risk Score) provided good clinical risk stratification wherein patients with weekly opioid-negative UDS results in the 3 weeks after treatment initiation had a 13% risk of return to use compared with 85% for those with 3 weeks of opioid-positive or missing UDS results (AUROC, 0.80; 95% CI, 0.76-0.84).Conclusions and RelevanceThe prediction model described in this study may be a universal risk measure for return to opioid use by treatment week 3. Interventions to prevent return to regular use should focus on this critical early treatment period.

show abstract

Algorithmic bias: review, synthesis, and future research directions

Cited by 257 publications

References 81 publications

When Do Customers Perceive Artificial Intelligence as Fair? An Assessment of AI-based B2C E-Commerce Services

When Do Customers Perceive Artificial Intelligence as Fair? An Assessment of AI-based B2C E-Commerce Services

Availability of Evidence for Predictive Machine Learning Algorithms in Primary Care

Individual-Level Risk Prediction of Return to Use During Opioid Use Disorder Treatment

Contact Info

Product

Resources

About