Medical data mining: insights from winning two competitions

Rosset, Saharon; Perlich, Claudia; Świrszcz, Grzergorz; Melville, Prem; Liu, Yan

doi:10.1007/s10618-009-0158-x

Cited by 44 publications

(27 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to the usual Kaggle contest in which the withheld data in the test set are already recorded, the NCAA contest involved predicting the future. Thus the usual concerns with leakage in data prediction contests (Rosset et al 2010;Kaufman et al 2012) did not exist for the NCAA tournament prediction Kaggle competition.…”

Section: Mark E Glickman* and Jeff Sonasmentioning

confidence: 99%

Introduction to the NCAA men’s basketball prediction methods issue

Glickman

Sonas²

2015

Journal of Quantitative Analysis in Sports

View full text Add to dashboard Cite

Section: Mark E Glickman* and Jeff Sonasmentioning

confidence: 99%

Introduction to the NCAA men’s basketball prediction methods issue

Glickman

Sonas²

2015

Journal of Quantitative Analysis in Sports

View full text Add to dashboard Cite

“…Two medical data mining contests held the following year and which also exhibited leakage are discussed in Perlich et al [2008] and Rosset et al [2010]. KDD Cup 2008 dealt with cancer detection from mammography data.…”

Section: Figmentioning

confidence: 99%

Leakage in data mining

Kaufman

Rosset

Perlich³

et al. 2012

ACM Trans. Knowl. Discov. Data

Self Cite

323

View full text Add to dashboard Cite

Deemed "one of the top ten data mining mistakes", leakage is the introduction of information about the data mining target that should not be legitimately available to mine from. In addition to our own industry experience with real-life projects, controversies around several major public data mining competitions held recently such as the INFORMS 2010 Data Mining Challenge and the IJCNN 2011 Social Network Challenge are evidence that this issue is as relevant today as it has ever been. While acknowledging the importance and prevalence of leakage in both synthetic competitions and real-life data mining projects, existing literature has largely left this idea unexplored. What little has been said turns out not to be broad enough to cover more complex cases of leakage, such as those where the classical independently and identically distributed (i.i.d.) assumption is violated, that have been recently documented. In our new approach, these cases and others are explained by explicitly defining modeling goals and analyzing the broader framework of the data mining problem. The resulting definition enables us to derive general methodology for dealing with the issue. We show that it is possible to avoid leakage with a simple specific approach to data management followed by what we call a learn-predict separation, and present several ways of detecting leakage when the modeler has no control over how the data have been collected. We also offer an alternative point of view on leakage that is based on causal graph modeling concepts.

show abstract

“…Data mining is a process and methodology for applying tools and techniques that explore and analyze a large number of data to discover meaningful patterns and rules. It is applicable to a wide variety of fields, from business to medicine and engineering (Rosset et al, 2010;Cheng et al, 2010). Hsu (2009) applied a data mining framework by using anthropometric data to develop industrial standards for adult females.…”

Section: Introductionmentioning

confidence: 99%

Upgrading Fitness in the Production of Garments

Salehi

Shahrabi²

2012

Research Journal of Textile and Apparel

View full text Add to dashboard Cite

If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service information about how to choose which publication to write for and submission guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information. About Emerald www.emeraldinsight.comEmerald is a global publisher linking research and practice to the benefit of society. The company manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as well as providing an extensive range of online products and additional customer resources and services.Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation. ABSTRACTThis article uses a series of data mining techniques to analyze body types and introduce a new sizing chart in order to produce garments for males. A principle component analysis and hierarchical and non-hierarchical clustering approaches are used to form a new sizing chart. All variables are grouped into two main components with a principle component analysis. Agglomerative hierarchical clustering is used to determine the number of clusters, and then a k-means algorithm is applied to segment the heterogonous population to actually form the clusters. The resultant innovations in designing garments have improved both non-price and price factors, the fittings of garments on all bodies have effectively improved and fabric waste has decreased, so the main goals which include improvement in quality with more comfort and a lower price have been met.

show abstract

Medical data mining: insights from winning two competitions

Cited by 44 publications

References 18 publications

Introduction to the NCAA men’s basketball prediction methods issue

Introduction to the NCAA men’s basketball prediction methods issue

Leakage in data mining

Upgrading Fitness in the Production of Garments

Contact Info

Product

Resources

About