On the philosophical foundations of psychological measurement

Maul, Andrew; Irribarra, David Torres; Wilson, Mark

doi:10.1016/j.measurement.2015.11.001

Cited by 70 publications

(35 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While it seems appropriate to require that other assessments be accurate, it might well be that there are benefits from engaging in SSA even if it is unrealistic or inaccurate as advocated by some on the basis that any engagement in the process regardless of its accuracy is worthwhile (Andrade 2010;McMillan and Hearn 2008). However, our position takes a realist stance (Maul et al 2015) that requires as validation evidence that the SSA approached as accurately as possible the actual characteristics of his or her performance or product. Holding positive self-efficacy beliefs and generally positive illusions about one's performance contributes to task motivation and performance goals, in contrast to having negative, pessimistic illusions about one's performance, which does seem to have a negative impact on motivation (Butler 2011).…”

Section: Implications For Researchmentioning

confidence: 99%

The Future of Student Self-Assessment: a Review of Known Unknowns and Potential Directions

2015

View full text Add to dashboard Cite

This paper reviews current known issues in student self-assessment (SSA) and identifies five topics that need further research: (1) SSA typologies, (2) accuracy, (3) role of expertise, (4) SSA and teacher/curricular expectations, and (5) effects of SSA for different students. Five SSA typologies were identified showing that there are different conceptions on the SSA components but the field still uses SSA quite uniformly. A significant amount of research has been devoted to SSA accuracy, and there is a great deal we know about it. Factors that influence accuracy and implications for teaching are examined, with consideration that students' expertise on the task at hand might be an important prerequisite for accurate self-assessment. Additionally, the idea that SSA should also consider the students' expectations about their learning is reflected upon. Finally, we explored how SSA works for different types of students and the challenges of helping lower performers. This paper sheds light on SSA research needs to address the known unknowns in this field. Student self-assessment (SSA) most generally involves a wide variety of mechanisms and techniques through which students describe (i.e., assess) and possibly assign merit or worth to (i.e., evaluate) the qualities of their own learning processes and products. This involves retrospective monitoring of previous performance (Baars et al. 2014) and reporting, hopefully truthfully, the quality of work completed. The purpose of this manuscript is to review what it is known and unknown about student self-assessment (SSA) after decades of research and, based on such evidence, highlight plausible lines of research that could make better known what we currently know is unknown. SSA has been extensively and vigorously recommended as an appropriate approach to student involvement in formative assessment, wherever the Bassessment for learning^reform agenda has been advocated (Berry 2011; Black and Wiliam 1998). Hence, it is important for education that we have a clearer understanding of SSA.While the field of empirical studies into SSA is increasing, it seemed timely to review what has been established in SSA since so many studies seem to only be replicating, in new contexts, what we already know about SSA. Hence, the purpose of this paper is to expand upon these known and unknown issues and consider plausible directions for future research. We consider that the known unknowns, which will be established in this review, are so substantial that our recommendations are unlikely to be a roadmap with clear signposts and indicators of progress. Rather, we consider that this paper functions more as a series of lighthouses highlighting well-trodden rocky shores and pointing out more profitable directions in which SSA research should head.SSA is an important skill for at least four reasons. First, students who are trained in SSA have shown an increase in their learning and academic performance Panadero et al. 2012;Ramdass and Zimmerman 2008). Additionally, accuracy (i.e., reliable and val...

show abstract

Section: Implications For Researchmentioning

confidence: 99%

The Future of Student Self-Assessment: a Review of Known Unknowns and Potential Directions

2015

View full text Add to dashboard Cite

show abstract

“…The development of the use of psychological tests in industrial and organizational fields is one of them influenced by the desire of company leaders to get the best candidates with a good process (Russell & Brannan, 2016). A good process starts by selecting the right measuring instrument and has validity and reliability criteria (Maul, Torres, & Wilson, 2016). These issues were then discussed in the work context until the issuance of a set of principles of validation and standardized use of the test by the Society for Industrial and Organizational Psychology in 1987 (Anastasi & Urbina, 2007).…”

Section: Introductionmentioning

confidence: 99%

The Individual Work Performance Scale: A Psychometric Study and Its Application for Employee Performance

Ramdani¹,

Marliani²,

Rahman³

2019

HSSR

View full text Add to dashboard Cite

Purpose: This research aims to get an individual work performance scale of the modified version which is more acceptable and has a good psychometric property. Methodology: This study was conducted using a modified measuring instrument approach. Researchers use existing theories and then make items according to the context in which this research will be conducted. The advantage of this modification approach is getting a more comprehensive understanding of the subject we are researching. Main Findings: Of the 303 employees that were tested in the research, the modification scale of the individual work performance has a good psychometric property with the criteria of reliability more than 0.8 and the fit model that has a good item with RMSEA score (0.062). Other than that, the modification scale of individual work performance shows a good convergent validity with presenteeism and correlate with job burnout. Applications of this study: The Individual Work Performance Scale has a comprehensive methodology and good psychometric properties. This instrument is appropriate to be the general instrument for seeing the employee situation. In addition, this instrument can also be used as a set of tests that are suitable for use by those who will conduct an assessment of employee performance. Novelty: This scale was made as a form of development from previous research which did not explore and develop forms of assessment that were specific to an employee's performance. There is not much-related research that focuses on many dimensions that are actually important to assess in employee performance.

show abstract

“…Third, serious epistemological criticisms of the use of psychometric scaling models to measure cognitive constructs abound. For an overview of literature discussing psychometrics as an entirely or partly pathological science, see H. Johnson (1936), Michell (1997), Michell (2008), Trendler (2013), Trendler (2009), Humphry (2013), Mari et al (2017), Maul, Irribarra, and Wilson (2016), Briggs (2013), Vautier et al (2012) and E. Lacot, Afzali, and Vautier (2016), among others. In a nutshell, these criticisms relate to two characteristics of standardized tests.…”

Section: Present State-of-the-art Methods Follow the Standards For Edmentioning

confidence: 99%

Measuring Unobserved Variables in Educational Inequality Research: Mathematics and Language 'Abilities' in Early Childhood

Sánchez¹

2019

Preprint

View full text Add to dashboard Cite

Evidence of social inequalities in cognitive abilities in early childhood has been documented in many societies; however, three characteristics of the data used to measure cognitive constructs make it difficult to quantify inequalities across groups. First, a causal understanding of validity is not compatible with the standard validation framework, which forces researchers to think critically what it means to measure unobserved constructs. Second, test scores only provide ordinal information about individuals, they are not interval scales and require the use of suitable corresponding methods for their study. Third, measurement invariance, which causes measurement error, may make comparison of test scores across groups invalid. The paper explores these three data problems applied to standardized tests---one mathematics and two language assessments---taken by a cohort of German children. The paper proposes a comparative validation framework for researchers based on nonparametric psychometric models and the representational theory of measurement. This framework can help researchers to determine if data fit the assumptions of a measurement model, to check for various forms of measurement error, and to overcome potential issues. A comparison of competing statistical modeling alternatives reveals substantial differences: By conceptualizing ability as ordinal instead of interval and excluding items that do not fit the assumptions of measurement models, I find a reduction in effect sizes for typical covariates studied in social stratification research.

show abstract

On the philosophical foundations of psychological measurement

Cited by 70 publications

References 43 publications

The Future of Student Self-Assessment: a Review of Known Unknowns and Potential Directions

The Future of Student Self-Assessment: a Review of Known Unknowns and Potential Directions

The Individual Work Performance Scale: A Psychometric Study and Its Application for Employee Performance

Measuring Unobserved Variables in Educational Inequality Research: Mathematics and Language 'Abilities' in Early Childhood

Contact Info

Product

Resources

About