Information reconstruction in educational institutions data from the European tertiary education registry

Bruni, Renato; Daraio, Cinzia; Aureli, Davide

doi:10.1016/j.dib.2020.106611

Cited by 4 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, we prefer to pursue this compromise between coverage and quality of the imputation. The results of our methodology on ETER database are available in [33].…”

Section: Results On the Real Missing Values Of Eter Databasementioning

confidence: 99%

Imputation techniques for the reconstruction of missing interconnected data from higher Educational Institutions

Bruni

Daraio

Aureli

2021

Knowledge-Based Systems

Self Cite

View full text Add to dashboard Cite

Educational Institutions data constitute the basis for several important analyses on the educational systems; however they often contain not negligible shares of missing values, for several reasons. We consider in this work the relevant case of the European Tertiary Education Register (ETER), describing the Educational Institutions of Europe. The presence of missing values prevents the full exploitation of this database, since several types of analyses that could be performed are currently impracticable. The imputation of artificial data, reconstructed with the aim of being statistically equivalent to the (unknown) missing data, would allow to overcome these problems. A main complication in the imputation of this type of data is given by the correlations that exist among all the variables. We propose several imputation techniques designed to deal with the different types of missing values appearing in these interconnected data. We use these techniques to impute the database. Moreover, we evaluate the accuracy of the proposed approach by artificially introducing missing data, by imputing them, and by comparing imputed and original values. Results show that the information reconstruction does not introduce statistically significant changes in the data and that the imputed values are close enough to the original values.

show abstract

“…Thus, we prefer to pursue this compromise between coverage and quality of the imputation. The results of our methodology on ETER database are available in [33].…”

Section: Results On the Real Missing Values Of Eter Databasementioning

confidence: 99%

Imputation techniques for the reconstruction of missing interconnected data from higher Educational Institutions

Bruni

Daraio

Aureli

2021

Knowledge-Based Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The full mathematical details of the methodology are explained in [3] . The original ETER dataset, possibly integrated with bibliometric information, and the imputed dataset, are available in several variants from [5] .…”

Section: *Methods Detailsmentioning

confidence: 99%

Optimization methods for the imputation of missing values in Educational Institutions Data

2021

View full text Add to dashboard Cite

The imputation of missing values in the detail data of Educational Institutions is a difficult task. These data contain multivariate time series, which cannot be satisfactory imputed by many existing imputation techniques. Moreover, almost all the data of an Institution are interconnected: the number of graduates is not independent from the number of students, the expenditure is not independent from the staff, etc. In other words, each imputed value has an impact on the whole set of data of the institution. Therefore, imputation techniques for this specific case should be designed very carefully. We describe here the methods and the codes of the imputation methodology developed to impute the various patterns of missing values which appear in similar interconnected data. In particular, a first part of the proposed methodology, called ``trend smoothing imputation'', is designed to impute missing values in time series by respecting the trend and the other features of an Institution. The second part of the proposed methodology, called ``donor imputation'', is designed to impute larger chunks of missing data by using values taken form similar Institutions in order to respect again their size and trend. Trend smoothing imputation can handle missing subsequences in time series, and is given by a weighted combination of: (a) weighed average of the other available values of the sequence, and (b) linear regression. Donor imputation can handle full sequence missing in time series. It imputes the Recipient Institution using the values taken from a similar institution, called Donor, selected using optimization criteria. The values imputed by our techniques should respect the trend, the size and the ratios of each Institution.

show abstract

“…A flag of possible recalculation error is raised for period t=3 since Δv4 (2,3) > 0 and Δv4 (3,4) < 0, while simultaneously |Δv4 (2,3) | = 87 > 84.24 and |Δv4 (3,4) | = 155 > 84.24. All the described operations are available in the Microsoft Excel file contained in [22]. This file can be used to operate the described checks with any data, by simply pasting them in the sheet "Main Table ".…”

Section: Recalculation Problemmentioning

confidence: 99%

Flexible Techniques to Detect Typical Hidden Errors in Large Longitudinal Datasets

Bruni,

Daraio,

Di Leo

2024

Preprint

View full text Add to dashboard Cite

The increasing availability of longitudinal data (repeated numerical observations of same units at different times) requires the development of flexible techniques to automatically detect common errors in such data. Besides obvious and easily identifiable cases, such as missing or out-of-range data, large longitudinal dataset often present problems not easily traceable by the techniques used for generic datasets. In particular, elusive and baffling problems are i) inversion of one or more values from one unit to another; ii) anomalous jumps in the series of values, iii) errors in the timing of the values due to a recalculation operated by the data providers to compensate previous errors. This work proposes a statistical-mathematical approach based on a system of indicators that is able to capture the complexity of the described problems by working at the formal level, regardless of the specific meaning of the data. The proposed approach identifies suspect erroneous data and is applicable in a variety of contexts. We implement this approach in a relevant database of European Higher Education institutions (ETER) by analyzing Total academic staff, that is one of the most important variables, used in empirical analysis as proxy of size and also considered by policy makers at European level.

show abstract

Information reconstruction in educational institutions data from the European tertiary education registry

Cited by 4 publications

References 2 publications

Imputation techniques for the reconstruction of missing interconnected data from higher Educational Institutions

Imputation techniques for the reconstruction of missing interconnected data from higher Educational Institutions

Optimization methods for the imputation of missing values in Educational Institutions Data

Flexible Techniques to Detect Typical Hidden Errors in Large Longitudinal Datasets

Contact Info

Product

Resources

About