Inference Using Non-Random Samples? Stop Right There!

Hirschauer, Norbert; Grüner, Sven; Mußhoff, Oliver; Becker, Claudia; Jantsch, Antje

doi:10.1111/1740-9713.01568

Cited by 15 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, it theoretically confirms there is a difference between the average of both distributions (workers and students). Nevertheless, this study is not reporting the p-values obtained, considering the sample is not a random sample, so the “inference becomes tricky or outright impossible” (Hirschauer et al, 2021 : 1), and researchers have preferred to maintain the importance of the size of the sample, rather than artificially randomising it.…”

Section: Methodsmentioning

confidence: 99%

MOOCs as a massive learning resource for a Higher Education Community. The Universitat Politècnica de València experience using the EdX remote access program

2022

View full text Add to dashboard Cite

During the 2020 COVID-19 lockdown, edX launched an initiative, called the Remote Access Program, to give access to free certificates for its Massive Open Online Courses (MOOCs) to the communities of its partners. This paper describes the experience of a Higher Education (HE) institution in Europe participating in this initiative as a strategic action to spread MOOCs, not just as a resource to open the university educational offer but also to improve the professional development possibilities of its community. This case study explores general data about the participation of over 7,700 people (23,4% of the Universitat Politècnica de Valencia’s community) in these courses and data from participants (1,515) about their initiative perceptions. The data obtained evaluates how a university community integrates MOOCs into their day-to-day life under certain conditions and their perception of the quality and usefulness of the courses. Data shows most of the community did not have experience or knowledge about MOOCs (73%), they used it (they obtained 5,202 certificates, a 33% completion rate), they value the course quality (4.1/5) and are happy with the initiative (4.7/5), and they think MOOCs will be useful for their career (3.67/5) and will take more MOOCs in the future (98%, with 71% thinking it is worth paying for the certificate). These results remark the importance of thinking about MOOCs in HE Institutions not just as a punctual resource but as a strategic investment affecting the university teaching offer, the professional development of its community, and their social compromise.

show abstract

Section: Methodsmentioning

confidence: 99%

MOOCs as a massive learning resource for a Higher Education Community. The Universitat Politècnica de València experience using the EdX remote access program

2022

View full text Add to dashboard Cite

show abstract

“…The structure and content of the survey were validated using a content and face validation [44] process that guarantees the appropriateness and relevance of the items as they appear to the persons answering the survey [45].…”

Section: Materials and Data Instrumentsmentioning

confidence: 99%

What Does the Data Say about Effective University Online Internships? The Universitat Politècnica de València Experience Using MOOC during COVID-19 Lockdown

2022

View full text Add to dashboard Cite

Universitat Politècnica de València’s students can take in-company internships during their bachelor’s degrees, and, with the COVID-19 lockdown, 224 students had their internships cancelled. EdX launched a free certificate initiative for its partners, and UPV gave the possibility of using MOOCs to cover for the credit needed to graduate. We have tried to answer the question, “Is it possible to use MOOCs to replace an in-company internship in an emergency?” using Learning Analytics; 179 students chose this possibility. More than 90% of the students got their academic credit, and their satisfaction with the initiative was 4.6/5. They scored MOOCs’ quality with 4/5 and the contribution of MOOCs to their careers with 3.6/5; 95% will take a MOOC, and 69.3% think it is worth paying for the certificate. The answers to the question evaluating if MOOCs had given them the same knowledge as a company internship are positive but much less conclusive, with an average of 2.87/5. We conclude that MOOCs achieved the pursued goal during the emergency. With more time for planning and extra resources for remote support, they can be a good solution in environments where online is the only choice, and they can even be used as a tool to reinforce some of the knowledge needed to be successful in a traditional internship.

show abstract

“…The introduction to this paper lists three suggestions to avoid the p-hacking problem made by Wasserstein et al (2019). More recently, Hirschauer et al (2021) state that formal inference might often be "tricky or outright impossible" when it is not clear whether the observed data represent a random sample. This paper provides software tools to implement some of those suggestions while de-emphasizing p-values.…”

Section: Pseudo-regression Coefficients and Final Remarksmentioning

confidence: 99%

“…We distinguish between (a) the practical numerical significance of a regressor in explaining the variation in the dependent variable, and (b) its statistical significance measured by the t-test based on the sampling distribution of the regression coefficient over the population of all-possible data samples. Hirschauer et al (2021) emphasize sample selection problems with traditional inference. The t-test p-values (used by p-hackers) rely on the unverified assumption that errors are normally distributed.…”

Section: Introduction and Avoiding P-hacking With Enhanced Regression Toolsmentioning

confidence: 99%

Kernel Regression Coefficients for Practical Significance

Vinod

2022

JRFM

View full text Add to dashboard Cite

Quantitative researchers often use Student’s t-test (and its p-values) to claim that a particular regressor is important (statistically significantly) for explaining the variation in a response variable. A study is subject to the p-hacking problem when its author relies too much on formal statistical significance while ignoring the size of what is at stake. We suggest reporting estimates using nonlinear kernel regressions and the standardization of all variables to avoid p-hacking. We are filling an essential gap in the literature because p-hacking-related papers do not even mention kernel regressions or standardization. Although our methods have general applicability in all sciences, our illustrations refer to risk management for a cross-section of firms and financial management in macroeconomic time series. We estimate nonlinear, nonparametric kernel regressions for both examples to illustrate the computation of scale-free generalized partial correlation coefficients (GPCCs). We suggest supplementing the usual p-values by “practical significance” revealed by scale-free GPCCs. We show that GPCCs also yield new pseudo regression coefficients to measure each regressor’s relative (nonlinear) contribution in a kernel regression.

show abstract

Inference Using Non-Random Samples? Stop Right There!

Cited by 15 publications

References 9 publications

MOOCs as a massive learning resource for a Higher Education Community. The Universitat Politècnica de València experience using the EdX remote access program

MOOCs as a massive learning resource for a Higher Education Community. The Universitat Politècnica de València experience using the EdX remote access program

What Does the Data Say about Effective University Online Internships? The Universitat Politècnica de València Experience Using MOOC during COVID-19 Lockdown

Kernel Regression Coefficients for Practical Significance

Contact Info

Product

Resources

About