Reliability Testing of the National Database of Nursing Quality Indicators Pressure Ulcer Indicator

Hart, Sara; Bergquist, Sandra; Gajewski, Byron; Dunton, Nancy

doi:10.1097/00001786-200607000-00011

Cited by 67 publications

(39 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When these assumptions are violated, Cohen's j cannot be used to indicate the consistency of the ratings between two raters. For instance, Hart et al (2006) reported Cohen's j between hospital nurses and PU experts. Ratings made by experts were considered correct classifications and the reliability between nurses and experts was considered rater-to-standards reliability.…”

Section: Resultsmentioning

confidence: 99%

Meta-analysis of Cohen’s kappa

Sun

2011

Health Serv Outcomes Res Method

164

View full text Add to dashboard Cite

Cohen's j is the most important and most widely accepted measure of interrater reliability when the outcome of interest is measured on a nominal scale. The estimates of Cohen's j usually vary from one study to another due to differences in study settings, test properties, rater characteristics and subject characteristics. This study proposes a formal statistical framework for meta-analysis of Cohen's j to describe the typical interrater reliability estimate across multiple studies, to quantify between-study variation and to evaluate the contribution of moderators to heterogeneity. To demonstrate the application of the proposed statistical framework, a meta-analysis of Cohen's j is conducted for pressure ulcer classification systems. Implications and directions for future research are discussed.Keywords Cohen's j Á Inter-rater reliability Á Meta-analysis Á GeneralizabilityIn classical test theory proposed by Spearman (1904), an observed score X is expressed as the true score T plus a random error of measurement e, i.e., X = T ? e. Reliability is defined as the squared correlation between observed scores and true scores (Lord and Novick 1968). It indicates the extent to which scores produced by a particular measurement procedure are consistent and reproducible (Thorndike 2005). Reliability is an unobserved property of scores obtained from a sample on a particular test, not an inherent property of the test (Thompson 2002;Thompson and Vacha-Hasse 2000;Vacha-Hasse 1998;Vacha-Hasse et al. 2002). Therefore, it is never appropriate to claim a test is reliable or unreliable in a research article. Instead, researchers should state the scores are reliable or unreliable. Reliability estimates usually vary from one study to another due to differences in study characteristics including study settings, test properties, and subject characteristics. A test that yields reliable scores for one group of subjects in this setting may fail to yield reliable scores for a different group of subjects in another setting. Hence, understanding the generalizability of score reliability and the factors affecting score reliability becomes an important methodological issue.

show abstract

Section: Resultsmentioning

confidence: 99%

Meta-analysis of Cohen’s kappa

Sun

2011

Health Serv Outcomes Res Method

164

View full text Add to dashboard Cite

show abstract

“…Hart et al,16 in a Web-based examination designed to test the reliability of the pressure ulcer measure of the National Database of Nursing Quality Indicators (NDNQI) and to assess 256 nurses’ knowledge of pressure ulcers, found that registered nurses certified in wound, continence, and/or ostomy care scored significantly higher than did other nurses in staging of ulcers.…”

Section: Introductionmentioning

confidence: 99%

Competence and Certification of Registered Nurses and Safety of Patients in Intensive Care Units

Kendall-Gallagher

Blegen

2009

American Journal of Critical Care

166

View full text Add to dashboard Cite

Background Adverse events that place patients at risk for harm are common in intensive care units. Clinicians' level of knowledge and judgment appear to play a role in the prevention, mitigation, and creation of adverse advents. Research suggests a possible association between nurses' specialty certification and clinical expertise. The relationship between specialty certification and clinical competence of registered nurses and safety of patients is a relatively new area of inquiry in nursing. Objective To explore the relationship between the proportion of certified staff nurses in a unit and risk of harm to patients. Methods Hierarchical linear modeling was used in a secondary data analysis of 48 intensive care units from a random sample of 29 hospitals to examine the relationships between unit certification rates, organizational nursing characteristics (magnet status, staffing, education, and experience), and rates of medication administration errors, falls, skin breakdown, and 3 types of nosocomial infections. Medicare case mix index was used to adjust for patient risk. Results Unit proportion of certified staff registered nurses was inversely related to rate of falls, and total hours of nursing care was positively related to medication administration errors. The mean number of years of experience of registered nurses in the unit was inversely related to frequency of urinary tract infections; however, the small sample size requires that caution be exercised when interpreting results. Conclusions Specialty certification and competence of registered nurses are related to patients' safety. Further research on this relationship is needed. (American Journal of Critical Care. 2009;18:106-116) C E 1.0 Hour Notice to CE enrollees:A closed-book, multiple-choice examination following this article tests your under standing of the following objectives: 1. Describe the underlying premises of the certification process.2. Understand the relationship between nursing specialty certification and competence.3. Recognize the findings of the study as they relate to the relationship of certification of intensive care unit nurses and patient outcomes.To read this article and take the CE test online, visit www.ajcconline.org and click "CE Articles in This Issue. " No CE test fee for AACN members. E R BEvidence-Based Review on pp 115-116. by AACN on May 10, 2018 http://ajcc.aacnjournals.org/ Downloaded from Understanding the role that a clinician's knowledge and skill play in the prevention of adverse events is essential for developing effective strategies for reducing the risk of harm to patients. Specialty certification is one method of validating clinicians' knowledge in a specific area of practice.5 In medicine, board certification "is designed to provide an overall assessment of physician competence . . . meant to indicate that a physician has the knowledge, experience, and skills for providing quality health care within a given specialty." 6(p1365) In nursing, the link between specialty certification and competence has...

show abstract

“…Here we included data from all observation periods, irrespective of the estimate . Sun performed subgroup analyses of studies according to the degree of training of the raters involved, and also drew a distinction between inter-rater reliability studies where both raters can be considered to be equivalent and a study [14] comparing ratings from hospital nurses with those from an expert which would more appropriately have been analysed using sensitivity, specificity and related techniques. The QuIS observations were carried out by raters who had all received the training developed by McLean et al: though there was variation in experience of QuIS a further source of inter-rater unreliability relating to the different lines of view from each rater’s position was also considered to be important.…”

Section: Discussionmentioning

confidence: 99%

Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions

Mesa‐Eguiagaray

Böhning

McLean

et al. 2016

BMC Med Res Methodol

View full text Add to dashboard Cite

Background: Recent studies of the quality of in-hospital care have used the Quality of Interaction Schedule (QuIS) to rate interactions observed between staff and inpatients in a variety of ward conditions. The QuIS was developed and evaluated in nursing and residential care. We set out to develop methodology for summarising information from inter-rater reliability studies of the QuIS in the acute hospital setting. Methods: Staff-inpatient interactions were rated by trained staff observing care delivered during two-hour observation periods. Anticipating the possibility of the quality of care varying depending on ward conditions, we selected wards and times of day to reflect the variety of daytime care delivered to patients. We estimated inter-rater reliability using weighted kappa, κ w , combined over observation periods to produce an overall, summary estimate,κ w . Weighting schemes putting different emphasis on the severity of misclassification between QuIS categories were compared, as were different methods of combining observation period specific estimates. Results: Estimatedκ w did not vary greatly depending on the weighting scheme employed, but we found simple averaging of estimates across observation periods to produce a higher value of inter-rater reliability due to overweighting observation periods with fewest interactions. Conclusions: We recommend that researchers evaluating the inter-rater reliability of the QuIS by observing staffinpatient interactions during observation periods representing the variety of ward conditions in which care takes place, should summarise inter-rater reliability by κ w , weighted according to our scheme A4. Observation period specific estimates should be combined into an overall, single summary statisticκ w random , using a random effects approach, withκ w random , to be interpreted as the mean of the distribution of κ w across the variety of ward conditions. We draw attention to issues in the analysis and interpretation of inter-rater reliability studies incorporating distinct phases of data collection that may generalise more widely.

show abstract

Reliability Testing of the National Database of Nursing Quality Indicators Pressure Ulcer Indicator

Cited by 67 publications

References 10 publications

Meta-analysis of Cohen’s kappa

Meta-analysis of Cohen’s kappa

Competence and Certification of Registered Nurses and Safety of Patients in Intensive Care Units

Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions

Contact Info

Product

Resources

About