Considerations in the reliability and fairness audits of predictive models for advance care planning

Lu, Jonathan; Sattler, Amelia; Wang, Samantha; Khaki, Ali Raza; Callahan, Alison; Fleming, Scott L.; Fong, Rebecca; Ehlert, Benjamin; Li, R.; Shieh, Lisa; Ramchandran, Kavitha; Gensheimer, Michael F.; Chobot, Sarah; Pfohl, Stephen R.; Li, Siyun; Shum, Kenny; Parikh, Nitin; Desai, P.; Seevaratnam, Briththa; Hanson, Melanie; Smith, Margaret; Xu, Yizhe; Gokhale, Arjun; Lin, Steven; Pfeffer, Michael; Teuteberg, Winifred; Shah, Nigam H.

doi:10.3389/fdgth.2022.943768

Cited by 13 publications

(6 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One aging‐focused example where algorithmic decision‐making has the potential to perpetuate healthcare inequities involves the use of mortality prediction models to identify individuals for palliative care consults 52 . Although these models may help target patients most likely to benefit from palliative care services, sensitivity analyses and follow‐up studies should be conducted to ensure that they do not perpetuate inequities in healthcare access 53 …”

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

“…55 Once models are deployed, efforts should be made to perform fairness audits and assess for changes in performance over time. 53 It is essential that researchers highlight potential uses and misuses of the model. For example, a social frailty index was developed using social factors (e.g., social isolation, neighborhood) to estimate mortality risk among older adults.…”

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

“…Several fairness metrics have been proposed, such as equality of odds and demographic parity, although methods that impose fairness constraints may result in decreased model performance (Supplementary Text S1). 55 Once models are deployed, efforts should be made to perform fairness audits and assess for changes in performance over time 53 …”

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

See 2 more Smart Citations

Around the EQUATOR with Clin‐STAR: Prediction modeling opportunities and challenges in aging research

Deardorff,

Diaz‐Ramirez,

Boscardin

et al. 2023

J American Geriatrics Society

View full text Add to dashboard Cite

The 2015 Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement was published to improve reporting transparency for prediction modeling studies. The objective of this review is to highlight methodologic challenges that aging‐focused researchers will encounter when designing and reporting studies involving prediction models for older adults and provide guidance for addressing these challenges. In following the 22‐item TRIPOD checklist, researchers must consider the representativeness of cohorts used (e.g., whether older adults with frailty, cognitive impairment, and social isolation were included), strategies for incorporating common geriatric predictors (e.g., age, comorbidities, functional status, and frailty), methods for handling missing data and competing risk of death, and assessment of model performance heterogeneity across important subgroups (e.g., age, sex, race, and ethnicity). We provide guidance to help aging‐focused researchers develop, validate, and report models that can inform and improve patient care, which we label “TRIPOD‐65.”

show abstract

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

Section: Tripod Discussion: Interpretation and Implicationsmentioning

confidence: 99%

See 1 more Smart Citation

Around the EQUATOR with Clin‐STAR: Prediction modeling opportunities and challenges in aging research

Deardorff,

Diaz‐Ramirez,

Boscardin

et al. 2023

J American Geriatrics Society

View full text Add to dashboard Cite

show abstract

“…76 To combat this, fairness audits were used to reflect on AI/ML performance in prompts for end-of-life care planning, and found application performance differences by race/ethnicity, for example, in Hispanic/Latino males whose race was recorded as "other." 77 This particular audit required 115 person-hours and did not add clinically meaningful information due to poor demographic data quality and lack of data access. AI/ML was also largely unsuccessful at incorporating social determinants of health indicators into prospective risk adjustment for private insurance payments in the U.S improving the predictive ratio by only 3%, 78 though this performance may worsen over time as so-called latent biases emerge with subsequent use of an AI/ML tool.…”

Section: Decrease Administrative Burdenmentioning

confidence: 99%

What Complexity Science Predicts About the Potential of Artificial Intelligence/Machine Learning to Improve Primary Care

Young,

Martin,

Sturmberg

et al. 2024

J Am Board Fam Med

View full text Add to dashboard Cite

Primary care physicians are likely both excited and apprehensive at the prospects for artificial intelligence (AI) and machine learning (ML). Complexity science may provide insight into which AI/ML applications will most likely affect primary care in the future. AI/ML has successfully diagnosed some diseases from digital images, helped with administrative tasks such as writing notes in the electronic record by converting voice to text, and organized information from multiple sources within a health care system. AI/ML has less successfully recommended treatments for patients with complicated single diseases such as cancer; or improved diagnosing, patient shared decision making, and treating patients with multiple comorbidities and social determinant challenges. AI/ML has magnified disparities in health equity, and almost nothing is known of the effect of AI/ML on primary care physician-patient relationships. An intervention in Victoria, Australia showed promise where an AI/ML tool was used only as an adjunct to complex medical decision making. Putting these findings in a complex adaptive system framework, AI/ML tools will likely work when its tasks are limited in scope, have clean data that are mostly linear and deterministic, and fit well into existing workflows. AI/ML has rarely improved comprehensive care, especially in primary care settings, where data have a significant number of errors and inconsistencies. Primary care should be intimately involved in AI/ML development, and its tools carefully tested before implementation; and unlike electronic health records, not just assumed that AI/ ML tools will improve primary care work life, quality, safety, and person-centered clinical decision making.

show abstract

“…For example, EHR-based AI models have demonstrated decreased performance and calibration across various geographic locations and over time. 31 AI models also showed poor performance in historically underrepresented groups, 31 potentially reinforcing existing inequities.…”

Section: Generalizability and Equity Concernsmentioning

confidence: 99%

Artificial Intelligence Decision Support Tools for End-of-Life Care Planning Conversations

Xie,

Butcher

2023

cjht

View full text Add to dashboard Cite

Why Is This an Issue? End-of-life care provides support for patients and their families during the last stage of life. End-of-life conversations aim to help people better understand their disease prognosis and expected survival, enabling them to make informed decisions regarding end-of-life care. Palliative care focuses on relieving symptoms and improving quality of life for patients with serious or life-threatening diseases. Approximately 89% of patients with life-limiting diseases, such as cancer, can benefit from palliative care. However, not all patients receive it in a timely manner. Due in part to prognostic uncertainty and optimism bias, end-of-life planning conversations and palliative care decisions do not occur early enough to have maximum benefit. Interventions that aim to prompt or help identify those patients who can benefit from palliative and/or end-of-life planning could improve the quality of care. What Is the Technology? An artificial intelligence (AI)–based “nudge” is a decision-making support tool that uses prompts and alerts to aid clinicians in deciding whether and when to discuss end-of-life planning with patients. The nudge sends alerts and/or reminders to clinicians to prompt end-of-life conversations with patients who are at high risk of short-term mortality. These patients are identified by machine learning mortality prediction algorithms incorporated in the electronic health record (EHR) system. Two AI-based nudges designed for patients with cancer were identified. Both tools were developed and internally validated in the US. What Is the Potential Impact? AI-based nudges have the potential to increase the number of end-of-life planning conversations between clinicians and patients as well as the number of referrals to end-of life services. Implementing the nudges into clinical workflows could also help clinicians more easily identify patients with palliative care needs. What Else Do We Need to Know? No AI-based nudges have been approved for use in Canada at the time of this writing nor have there been validation studies using Canadian data. As with many AI algorithms, there is uncertainty about the validity and generalizability of the mortality predictive algorithms used in the nudges. The acceptance of AI-based nudges by clinicians is unclear due to varying clinician attitudes and experiences with nudges and because we did not identify any studies that reported the experience of AI-based nudges from the patient perspective.

show abstract

Considerations in the reliability and fairness audits of predictive models for advance care planning

Cited by 13 publications

References 44 publications

Around the EQUATOR with Clin‐STAR: Prediction modeling opportunities and challenges in aging research

Around the EQUATOR with Clin‐STAR: Prediction modeling opportunities and challenges in aging research

What Complexity Science Predicts About the Potential of Artificial Intelligence/Machine Learning to Improve Primary Care

Artificial Intelligence Decision Support Tools for End-of-Life Care Planning Conversations

Contact Info

Product

Resources

About