Measurement of Epistemological Beliefs

Ordóñez, Xavier G.; Ponsoda, Vicente; Abad, Francisco; Romero, Sonia J.

doi:10.1177/0013164408323226

Cited by 16 publications

(15 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The alternative to asking experts for truth-apt statements is to ask them for statements indicating their approval or disapproval (acceptance or rejection) of an item, also referred to as consent statements. 28 The basis of such statements is not objectivity in the sense usually meant when referring to interpersonal verifiability. Rather, these statements are based on subjective mental states, subjective preferences, subjective appraisals or something similar.…”

Section: Asking Experts For Subjective Reactionsmentioning

confidence: 99%

See 1 more Smart Citation

Ensuring content validity of psychological and educational tests – the role of experts

Beck

2020

FLR

View full text Add to dashboard Cite

Many test developers try to ensure the content validity of their tests by having external experts review the items, e.g. in terms of relevance, difficulty, or clarity. Although this approach is widely accepted, a closer look reveals several pitfalls need to be avoided if experts' advice is to be truly helpful. The purpose of this paper is to exemplarily describe and critically analyse widespread practices of involving experts to ensure the content validity of tests. First, I offer a classification of tasks that experts are given by test developers, as reported on in the respective literature. Second, taking an exploratory approach by means of a qualitative meta-analysis, I review a sample of reports on test development (N = 72) to identify the common current procedures for selecting and consulting experts. Results indicate that often the choice of experts seems to be somewhat arbitrary, the questions posed to experts lack precision, and the methods used to evaluate experts' feedback are questionable. Third, given these findings I explore in more depth what prerequisites are necessary for their contributions to be useful in ensuring the content validity of tests. Main results are (i) that test developers, contrary to some practice, should not ask for information that they can reliably ascertain themselves ("truth-apt statements"). (ii) Average values from the answers of the experts which are often calculated rarely provide reliable information about the quality of test items or a test. (iii) Making judgements about some aspects of the quality of test items (e.g. comprehensibility, plausibility of distractors) could lead experts to unreliable speculations. (iv)When several experts respond by giving incompatible or even contradictory answers, there is almost always no criterion enabling a decision between them. In conclusion, explicit guidelines on this matter need to be elaborated and standardised (above all, by the AERA, APA, and NCME "Standards").

show abstract

Section: Asking Experts For Subjective Reactionsmentioning

confidence: 99%

“…See for example the Test of Economic Literacy (Third Edition) byWalstad and Rebeck (2001) including several items on further economic development subject to the conditions of the Theory of Competitive Markets 28. Languages comprise several types of statements (e.g., interrogative, imperative, exclamative).…”

mentioning

confidence: 99%

Ensuring content validity of psychological and educational tests – the role of experts

Beck

2020

FLR

View full text Add to dashboard Cite

show abstract

“…In particular, those studies revealed that MSA does not function well in conditions in which the traits correlate (Mroch & Bolt, 2006; van Abswoude et al, 2004), or in which the items load on more than one trait (van Abswoude et al, 2004). Although those test conditions are the rule rather than the exception in empirical practice (van Abswoude, Vermunt, & Hemker, 2007), after publication of these articles, numerous researchers have still used MSA as a dimensionality assessment tool (e.g., Bech, Fava, Trivedi, Wisniewski, & Rush, 2011; Chen, Tseng, Hu, & Koh, 2010; Doyle, Conroy, McGee, & Delaney, 2010; Emons et al, 2010; Koster et al, 2009; Meijer et al, 2011; Ordoñez, Ponsoda, Abad, & Romero, 2009; Roorda, Houwink, Smits, Molenaar, & Geurts, 2011; Sousa et al, 2010). Furthermore, MSA is still recommended to assess unidimensionality (Sijtsma et al, 2011).…”

mentioning

confidence: 99%

Exploratory Mokken Scale Analysis as a Dimensionality Assessment Tool

Smits

Timmerman

Meijer

2012

Applied Psychological Measurement

View full text Add to dashboard Cite

The assessment of the number of dimensions and the dimensionality structure of questionnaire data is important in scale evaluation. In this study, the authors evaluate two dimensionality assessment procedures in the context of Mokken scale analysis (MSA), using a so-called fixed lowerbound. The comparative simulation study, covering various theoretically and empirically relevant conditions, indicates that the MSA procedures may result in scales that are inconsistent with the dimensionality of the data set at hand. That is, a single Mokken scale can be multidimensional, and two Mokken scales can pertain to a single dimension. In an illustrative evaluation, MSA using a range of lowerbounds, rather than a fixed lowerbound, was shown to have some benefits, but not to solve all limitations. The results of this study imply that MSA is perfectly suitable to create Mokken scales. However, MSA appears of limited value as a dimensionality assessment method.

show abstract

“…There have been also growing interest in examining the factor structure of the EBI (Teo & Chai, 2011), in comparing and combining the EBI with other epistemic beliefs instruments (Duell & Schommer-Aikins, 2001;DeBacker, Crowson, Beesley, Thoma, & Hestevold, 2008;Ordoñez, Ponsoda, Abad, & Romero, 2009;Laster, 2010), and in studying its psychometric properties (Teo, 2013;Welch & Ray, 2012). For a comprehensive comparison of previous psychometric analyses of the EBI, please refer to Welch and Ray (2012).…”

Section: Epistemic Beliefs Inventory (Ebi)mentioning

confidence: 99%

Validation of the Chinese Version of the Epistemic Beliefs Inventory Using Confirmatory Factor Analysis

Wang¹,

Zhang²,

Zhang³

et al. 2013

IES

View full text Add to dashboard Cite

The Epistemic Beliefs Inventory (EBI), as a theory-based and empirically validated instrument, was originally developed and widely used in the North American context. Through a strict translation procedure the authors translated the EBI into Chinese, and then administered it to 451 students in 7 universities in mainland China. The construct validity of scores on the Chinese version of the EBI (EBI-C) was examined with confirmatory factor analysis (CFA). The results confirmed the original 5 factors: Simple Knowledge (SK), Certain Knowledge (CK), Innate Ability (IA), Omniscient Authority (OA), and Quick Learning (QL). Yet the overall model fit was unsatisfactory, ? 2 (449) = 1167.32, GFI= .85, and RMSEA = .06. After two model respecifications with interpretable solutions, the overall model fit was substantially improved with ?2 (265) = 510.81, GFI= .92, RMSEA = .04. The EBI-C can be used in China and other Chinese-speaking regions and communities. Cross-cultural research in the area of epistemic beliefs becomes more feasible because of the presence of the parallel instrument

show abstract

Measurement of Epistemological Beliefs

Cited by 16 publications

References 40 publications

Ensuring content validity of psychological and educational tests – the role of experts

Ensuring content validity of psychological and educational tests – the role of experts

Exploratory Mokken Scale Analysis as a Dimensionality Assessment Tool

Validation of the Chinese Version of the Epistemic Beliefs Inventory Using Confirmatory Factor Analysis

Contact Info

Product

Resources

About