Internal Invalidity in Studies Employing Self‐report Instruments: A Suggested Remedy

Howard, George S.; Schmeck, Ronald R.; Bray, James H.

doi:10.1111/j.1745-3984.1979.tb00094.x

Cited by 123 publications

(108 citation statements)

References 4 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…The findings of the present study favor the retrospective pre/post self-report measure of change in providing a measure of self-reported change that better reflects the objective index of change on a construct of knowledge rating. This finding is in line with previous psychometric research (e.g., Hoogstraten, 1982;Howard & Dailey, 1979;Howard, Schmeck, & Bray, 1979;Spranger & Hoogstraten, 1989), and is most likely a result of the self-report posttest and the retrospective selfreport pretest being filled out with respect to the same internal standard, the same metric. This, therefore, mitigates the treatment-induced response shift bias, minimizes errors of measurement, and provides an unconfounded and unbiased estimate of the treatment effect .…”

Section: Treatment Effects In the Retrospective Pre/post Designsupporting

confidence: 90%

“…The conventional self-report pretest and the recalled self-report pretest were only separated by four months, which may have in part mitigated the effect of memory distortion. Previous research (e.g., Finney, 1981;Howard, Dailey, & Gulanick, 1979;Howard, Schmeck, & Bray, 1979;Maisto et al, 1982), nonetheless, suggests that a pretesting effect can be mitigated and moderate-to-high recall accuracy is possible when cognitive constructs are measured (such as knowledge ratings) and when retrospective questions are specific and anchors on response scales are explicit (these conditions are consistent with those used in this study).…”

Section: Testing For Threats To Validitysupporting

confidence: 84%

“…Previous research (e.g., Collins et al, 1985;Finney, 1981;Howard, Schmeck, & Bray, 1979;Maisto et al, 1982) suggests that the magnitude of a response shift bias seems to be smaller when cognitive constructs are measured (such as knowledge ratings) and when questions and anchors on response scales are explicit.…”

Section: Pretesting Effectsmentioning

confidence: 95%

“…When self-report evaluations must be used to measure change, the traditional pretestposttest treatment design can be modified to include a retrospective pretest at the time of the posttest (e.g., Howard & Dailey, 1979;Howard, Millham, Slaten, & O'Donnell, 1981;Howard, Ralph, Gulanick, Maxwell, Nance, & Gerber, 1979;Howard, Schmeck, & Bray, 1979). After filling out the posttest, subjects then report their memory or perception of what their score would have been prior to the treatment (this is referred to as a retrospective self-report pretest).…”

Section: Introductionmentioning

confidence: 99%

“…Nonetheless, in self-report pretestposttest treatment designs, previous psychometric research has demonstrated empirical support for the retrospective pretestposttest difference scores over the traditional pretest-posttest change scores in providing an index of change more in agreement with objective measures of change on both cognitive and behavioral dimensions (e.g., Hoogstraten, 1982;Howard & Dailey, 1979;Howard, Millham, Slaten, & O'Donnell, 1981;Howard, Ralph, Gulanick, Maxwell, Nance, & Gerber, 1979;Howard, Schmeck, & Bray, 1979;Spranger & Hoogstraten, 1989).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

An Empirical Evaluation Of The Retrospective Pretest: Are There Advantages To Looking Back?

Nakonezny

Rodgers

2005

J. Mod. App. Stat. Meth.

View full text Add to dashboard Cite

This article builds on research regarding response shift effects and retrospective self-report ratings. Results suggest moderate evidence of a response shift bias in the conventional pretest-posttest treatment design in the treatment group. The use of explicitly worded anchors on response scales, as well as the measurement of knowledge ratings (a cognitive construct) in an evaluation methodology setting, helped to mitigate the magnitude of a response shift bias. The retrospective pretest-posttest design provides a measure of change that is more in accord with the objective measure of change than is the conventional pretest-posttest treatment design with the objective measure of change, for the setting and experimental conditions used in the present study.

show abstract

Section: Treatment Effects In the Retrospective Pre/post Designsupporting

confidence: 90%

Section: Testing For Threats To Validitysupporting

confidence: 84%

Section: Pretesting Effectsmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An Empirical Evaluation Of The Retrospective Pretest: Are There Advantages To Looking Back?

Nakonezny

Rodgers

2005

J. Mod. App. Stat. Meth.

View full text Add to dashboard Cite

show abstract

Integrating standardized patient safety and quality competencies in pharmacy curriculum: Evaluation of the impact on student development and professional identity

Chang,

Firebaugh,

Jefferson

2024

J Am Coll Clin Pharm

View full text Add to dashboard Cite

IntroductionImproving patient safety and quality of care is a national healthcare priority. While changes in healthcare systems help to address some concerns in this area, the knowledge, skills, and attitudes of healthcare professionals play a role as well. Health professions education programs have a responsibility to ensure comprehensive patient safety and quality training.ObjectivesThis study aimed to evaluate the impact of integrating standardized safety and quality competencies from an evidence‐based framework in a patient safety and quality course within a Doctor of Pharmacy program.MethodsA gap analysis was conducted using guidelines from the Canadian Patient Safety Institute (CPSI) to explore curricular coverage of the included concepts and competencies. The course was designed to address key competencies not addressed elsewhere in the curriculum. An adapted self‐assessment survey tool was administered at the end of the course for students to retrospectively assess changes in perceived skills and attitudes. Mean retrospective pre‐ and post‐scores were compared using the paired sample t‐tests and Cohen d as a measure of effect size. Responses to an open‐ended question regarding how students could impact patient safety were qualitatively analyzed for emergent themes. Student performance on the course final exam was analyzed using descriptive statistics to assess knowledge.ResultsSixty‐three students (80.8%) completed the survey and were included in the analysis. A statistically significant change was observed in 18 of the 24 self‐assessment items, with effect sizes in the modest to moderate range. Key themes emerged regarding student professional identity development including error prevention, error management, reporting culture, learning culture, just culture, and teamwork. The mean individual student score on the final examination was 87.67% ± 4.87%.ConclusionFindings suggest the new course using the CPSI framework had a positive impact on student knowledge, perceived skills, and attitudes in the area of patient safety.

show abstract

Pre‐then‐post testing: A tool to improve the accuracy of management training program evaluation

Mezoff

1981

Nonprofit Management Leadership

View full text Add to dashboard Cite

Internal Invalidity in Studies Employing Self‐report Instruments: A Suggested Remedy

Cited by 123 publications

References 4 publications

An Empirical Evaluation Of The Retrospective Pretest: Are There Advantages To Looking Back?

An Empirical Evaluation Of The Retrospective Pretest: Are There Advantages To Looking Back?

Integrating standardized patient safety and quality competencies in pharmacy curriculum: Evaluation of the impact on student development and professional identity

Pre‐then‐post testing: A tool to improve the accuracy of management training program evaluation

Contact Info

Product

Resources

About