QuaPy: A Python-Based Framework for Quantification

Abstract

QuaPy is an open-source framework for performing quantification (a.k.a. supervised prevalence estimation), written in Python. Quantification is the task of training quantifiers via supervised learning, where a quantifier is a predictor that estimates the relative frequencies (a.k.a. prevalence values) of the classes of interest in a sample of unlabelled data. While quantification can be trivially performed by applying a standard classifier to each unlabelled data item and counting how many data items have been assigned to each class, it has been shown that this "classify and count" method is outperformed by methods specifically designed for quantification. QuaPy provides implementations of a number of baseline methods and advanced quantification methods, of routines for quantificationoriented model selection, of several broadly accepted evaluation measures, and of robust evaluation protocols routinely used in the field. QuaPy also makes available datasets commonly used for testing quantifiers, and offers visualization tools for facilitating the analysis and interpretation of the results. The software is opensource and publicly available under a BSD-3 licence via GitHub 1 , and can be installed via pip 2 . CCS CONCEPTS• Computing methodologies → Learning paradigms; • Information systems → Data mining.

A Concise Overview of LeQua@CLEF 2022: Learning to Quantify

“…SLD is the strongest baseline; this is true in all four subtasks, in which SLD, while never being the best performer, is always in the top ranks. This confirms the fact (already recorded in previous work -see e.g., [34][35][36]) that SLD is a very strong performer when the APP is used for generating the dataset, i.e., when the test data contain many samples characterized by substantial distribution shift.…”

“…CC and PCC obtain very low quantification accuracy; this is the case in all four subtasks, where these two methods are always near the bottom of the ranking. This confirms the fact (already recorded in previous worksee e.g., [34][35][36]) that they are not good performers when the APP is used for generating the dataset, i.e., they are not good performers when there is substantial distribution shift. Interestingly enough, CC always outperforms PCC, which was somehow unexpected.…”

“…In order to set a sufficiently high bar for the participants to overcome, we made them aware of the availability of QuaPy [34], a library of quantification methods that contains, among others, implementations of a number of methods that have performed well in recent comparative evaluations. 3 QuaPy is a publicly available, open-source, Python-based framework that we have recently developed, and that implements not only learning methods, but also evaluation measures, parameter optimisation routines, and evaluation protocols, for LQ.…”

Report on the 1st International Workshop on Learning to Quantify (LQ 2021)

“…In the first such talk, Alejandro Moreo presented (joint work with Andrea Esuli and Fabrizio Sebastiani) QuaPy (https://github.com/HLT-ISTI/QuaPy), an open-source Python-based software library for LQ. QuaPy (which was also the object of a presentation in the main CIKM 2021 conference -see [7]) provides implementations of both baseline and advanced LQ methods, of routines for LQoriented model selection, of several broadly accepted evaluation measures, and of robust evaluation protocols routinely used in the field. QuaPy also makes available datasets commonly used for testing quantifiers, and offers visualization tools for facilitating the analysis and interpretation of the results.…”

LeQua@CLEF2022: Learning to Quantify

“…We have recently developed (and made publicly available) QuaPy, an opensource, Python-based framework that implements several learning methods, evaluation measures, parameter optimisation routines, and evaluation protocols, for LQ [16]. 5 Among other things, QuaPy contains implementations of the baseline methods and evaluation measures officially adopted in LeQua 2022.…”