Understanding Fairness of Gender Classification Algorithms Across Gender-Race Groups

Krishnan, Anoop; Almadan, Ali; Rattani, Ajita

doi:10.1109/icmla51294.2020.00167

Cited by 43 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By building models on data from a variety of different sites with increased awareness of the specific populations included, we may begin to mitigate the potential biases in our results. Ultimately, improvements in DL algorithms remain relatively nascent, and there has been an increased focus on classification performance across gender and race, providing us with impetus to ensure that DL algorithms can be successfully used to mitigate health care disparities based on demographics ( 35 – 37 ).…”

Section: Data: What Are We Using and For What Purpose?mentioning

confidence: 99%

12 Plagues of AI in Healthcare: A Practical Guide to Current Issues With Using Machine Learning in a Medical Context

Doyen¹,

Dadario²

2022

Front. Digit. Health

View full text Add to dashboard Cite

The healthcare field has long been promised a number of exciting and powerful applications of Artificial Intelligence (AI) to improve the quality and delivery of health care services. AI techniques, such as machine learning (ML), have proven the ability to model enormous amounts of complex data and biological phenomena in ways only imaginable with human abilities alone. As such, medical professionals, data scientists, and Big Tech companies alike have all invested substantial time, effort, and funding into these technologies with hopes that AI systems will provide rigorous and systematic interpretations of large amounts of data that can be leveraged to augment clinical judgments in real time. However, despite not being newly introduced, AI-based medical devices have more than often been limited in their true clinical impact that was originally promised or that which is likely capable, such as during the current COVID-19 pandemic. There are several common pitfalls for these technologies that if not prospectively managed or adjusted in real-time, will continue to hinder their performance in high stakes environments outside of the lab in which they were created. To address these concerns, we outline and discuss many of the problems that future developers will likely face that contribute to these failures. Specifically, we examine the field under four lenses: approach, data, method and operation. If we continue to prospectively address and manage these concerns with reliable solutions and appropriate system processes in place, then we as a field may further optimize the clinical applicability and adoption of medical based AI technology moving forward.

show abstract

Section: Data: What Are We Using and For What Purpose?mentioning

confidence: 99%

12 Plagues of AI in Healthcare: A Practical Guide to Current Issues With Using Machine Learning in a Medical Context

Doyen¹,

Dadario²

2022

Front. Digit. Health

View full text Add to dashboard Cite

show abstract

“…For example, Labeled Faces in the Wild is 83.5% white, and the IJB-A dataset, which was specifically created to emphasize geographic diversity, draws only 21.4% of its examples from faces with darker skin tones (Buolamwini & Gebru, 2018). Researchers (Krishnan et al, 2020) compared models trained on the UTKFace dataset with models trained on the FairFace dataset (which is designed to be balanced with respect to race), and found that across 3 model architectures, model results were substantially less biased after training FairFace (Krishnan et al, 2020). These findings underscore the need for better public, large scale datasets labeled with demographic data in order to enable further empirical study of algorithmic bias.…”

Section: Huge Datasets and Algorithmic Fairnessmentioning

confidence: 99%

On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models

Emam,

Kondrich,

Harrison

et al. 2021

Preprint

View full text Add to dashboard Cite

High-quality labeled datasets play a crucial role in fueling the development of machine learning (ML), and in particular the development of deep learning (DL). However, since the emergence of the ImageNet dataset and the AlexNet model in 2012, the size of new open-source labeled vision datasets has remained roughly constant. Consequently, only a minority of publications in the computer vision community tackle supervised learning on datasets that are orders of magnitude larger than Imagenet. In this paper, we survey computer vision research domains that study the effects of such large datasets on model performance across different vision tasks. We summarize the community's current understanding of those effects, and highlight some open questions related to training with massive datasets. In particular, we tackle: (a) The largest datasets currently used in computer vision research and the interesting takeaways from training on such datasets; (b) The effectiveness of pretraining on large datasets; (c) Recent advancements and hurdles facing synthetic datasets; (d) An overview of double descent and sample nonmonotonicity phenomena; and finally, (e) A brief discussion of lifelong/continual learning and how it fares compared to learning from huge labeled datasets in an offline setting. Overall, our findings are that research on optimization for deep learning focuses on perfecting the training routine and thus making DL models less data hungry, while research on synthetic datasets aims to offset the cost of data labeling. However, for the time being, acquiring non-synthetic labeled data remains indispensable to boost performance.

show abstract

“…Further complicating the problem, even an ideal training dataset devoid of any biases does not guarantee that the trained ML model will be bias-free [348]: model underspecification [137], model misspecification [349], and biological differences that render the prediction task easier or harder in different groups [350] may still introduce arbitrary biases. Similarly, Hooker [348] discusses the impact of seemingly innocent technical measures and design choices on the fairness of the resulting model.…”

Section: Obstacles To Achieving Fairnessmentioning

confidence: 99%

Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Technical Challenges and Solutions

Petersen,

Potdevin,

Mohammadi

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning applications must be developed responsibly. A large number of high-level declarations of ethical principles have been put forth for this purpose, but there is a severe lack of technical guidelines explicating the practical consequences for medical machine learning. Similarly, there is currently considerable uncertainty regarding the exact regulatory requirements placed upon medical machine learning systems. In this paper, we survey the technical challenges involved in creating medical machine learning systems responsibly and in conformity with existing regulations, as well as possible solutions to address these challenges. We begin by providing a brief overview of existing regulations affecting medical machine learning, showing that properties such as safety, robustness, reliability, privacy, security, transparency, explainability, and nondiscrimination are all demanded already by existing law and regulations -albeit, in many cases, to an uncertain degree. Next, we discuss the key technical obstacles to achieving these desirable properties, and important techniques to overcome those barriers in the medical context. Since most of the technical challenges are very young and new problems frequently emerge, the scientific discourse is rapidly evolving and has not yet converged on clear best-practice solutions. Nevertheless, we aim to illuminate the underlying technical challenges, possible ways for addressing them, and their respective merits and drawbacks. In particular, we notice that distribution shift, spurious correlations, model underspecification, and data scarcity represent severe challenges in the medical context (and others) that are very difficult to solve with classical black-box deep neural networks. Important measures that may help to address these challenges include the use of large and representative datasets and federated learning as a means to that end, the careful exploitation of domain knowledge wherever feasible, the use of inherently transparent models, comprehensive model testing and verification, as well as stakeholder inclusion.

show abstract

Understanding Fairness of Gender Classification Algorithms Across Gender-Race Groups

Cited by 43 publications

References 13 publications

12 Plagues of AI in Healthcare: A Practical Guide to Current Issues With Using Machine Learning in a Medical Context

12 Plagues of AI in Healthcare: A Practical Guide to Current Issues With Using Machine Learning in a Medical Context

On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models

Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Technical Challenges and Solutions

Contact Info

Product

Resources

About