The Art and Science of Chart Review

Allison, Jeroan J.; Wall, Terry C.; Spettell, Claire M.; Calhoun, Jaimee; Fargason, Crayton A.; Kobylinski, Richard W.; Farmer, Robert; Kiefe, Catarina I.

doi:10.1016/s1070-3241(00)26009-4

Cited by 128 publications

(142 citation statements)

References 22 publications

Supporting

Mentioning

140

Contrasting

Unclassified

Order By: Relevance

“…A 5% reaudit matched the budgetary limits for phase I and was supported by the literature 4,22,23 but with no justifi cation. The IRR checkpoints were unknown to chart abstractors and selected at random intervals.…”

Section: Irr Analysis and Continuous Quality Monitoringmentioning

confidence: 81%

“…4,16 Various sources of bias may affect the initial chart data, including the communication of ailments to a medical professional, followed by the entry of that information into a medical record. The quality of data is therefore infl uenced by whether the required information is available in a form that may be abstracted 4 and is accurate. Recent primary care reforms including incentive payments for prevention activities (eg, smoking cessation fees) and use of a team approach (emphasizing the need to communicate information to others) will also infl uence the data.…”

Section: Discussionmentioning

confidence: 99%

“…1 Establishing rigorous methods for assessing the reliability (consistency) and validity (accuracy) of data is important. 2,3 Although there is evidencebased guidance on performing chart abstractions, 2,[4][5][6][7][8][9] there is minimal and inconsistent advice for methods to ensure interrater reliability (IRR), such as selection of the sample size, frequency of reliability checks, and minimum thresholds for the κ statistic and percent agreement. 4,[9][10][11] There are currently no standard recommendations for the proportion of abstracted data that should be randomly checked for reliability, 12 and sample size calculations can yield dramatically different numbers.…”

mentioning

confidence: 99%

“…2,3 Although there is evidencebased guidance on performing chart abstractions, 2,[4][5][6][7][8][9] there is minimal and inconsistent advice for methods to ensure interrater reliability (IRR), such as selection of the sample size, frequency of reliability checks, and minimum thresholds for the κ statistic and percent agreement. 4,[9][10][11] There are currently no standard recommendations for the proportion of abstracted data that should be randomly checked for reliability, 12 and sample size calculations can yield dramatically different numbers. 5,13,14 Two methods commonly used in the literature, the goodness-of-fi t approach 13 and the 95% confi dence interval precision method, 5,15 rely on estimates that are diffi cult to determine without knowledge from a previous study.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Liddy¹,

Wiens²,

Hogg³

2011

The Annals of Family Medicine

View full text Add to dashboard Cite

PURPOSEWe assessed interrater reliability (IRR) of chart abstractors within a randomized trial of cardiovascular care in primary care. We report our fi ndings, and outline issues and provide recommendations related to determining sample size, frequency of verifi cation, and minimum thresholds for 2 measures of IRR: the κ statistic and percent agreement. METHODSWe designed a data quality monitoring procedure having 4 parts: use of standardized protocols and forms, extensive training, continuous monitoring of IRR, and a quality improvement feedback mechanism. Four abstractors checked a 5% sample of charts at 3 time points for a predefi ned set of indicators of the quality of care. We set our quality threshold for IRR at a κ of 0.75, a percent agreement of 95%, or both.RESULTS Abstractors reabstracted a sample of charts in 16 of 27 primary care practices, checking a total of 132 charts with 38 indicators per chart. The overall κ across all items was 0.91 (95% confi dence interval, 0.90-0.92) and the overall percent agreement was 94.3%, signifying excellent agreement between abstractors. We gave feedback to the abstractors to highlight items that had a κ of less than 0.70 or a percent agreement less than 95%. No practice had to have its charts abstracted again because of poor quality.CONCLUSIONS A 5% sampling of charts for quality control using IRR analysis yielded κ and agreement levels that met or exceeded our quality thresholds. Using 3 time points during the chart audit phase allows for early quality control as well as ongoing quality monitoring. Our results can be used as a guide and benchmark for other medical chart review studies in primary care.

show abstract

Section: Irr Analysis and Continuous Quality Monitoringmentioning

confidence: 81%

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Liddy¹,

Wiens²,

Hogg³

2011

The Annals of Family Medicine

View full text Add to dashboard Cite

show abstract

“…The abstraction procedure was adapted from methods used in medical chart review, 30 with a quality improvement-style data dictionary derived from communication guidelines. [23][24][25][26] Methods were approved by the institutional review boards at Yale University and the Medical College of Wisconsin.…”

Section: Designmentioning

confidence: 99%

Pediatric Residents’ Use of Jargon During Counseling About Newborn Genetic Screening Results

et al. 2008

View full text Add to dashboard Cite

OBJECTIVE. The goal was to investigate pediatric residents’ usage of jargon during discussions about positive newborn screening test results. METHODS. An explicit-criteria abstraction procedure was used to identify jargon usage and explanations in transcripts of encounters between residents and standardized parents of a fictitious infant found to carry cystic fibrosis or sickle cell hemoglobinopathy. Residents were recruited from a series of educational workshops on how to inform parents about positive newborn screening test results. The time lag from jargon words to explanations was measured by using “statements,” each of which contained 1 subject and 1 predicate. RESULTS. Duplicate abstraction revealed reliability κ of 0.92. The average number of unique jargon words per transcript was 20; the total jargon count was 72.3 words. There was an average of 7.5 jargon explanations per transcript, but the explained/total jargon ratio was only 0.17. When jargon was explained, the average time lag from the first usage to the explanation was 8.2 statements. CONCLUSION. The large number of jargon words and the small number of explanations suggest that physicians’ counseling about newborn screening may be too complex for some parents.

show abstract

Assessment of Parental Understanding by Pediatric Residents During Counseling After Newborn Genetic Screening

Farrell

Kuruvilla

2008

Arch Pediatr Adolesc Med

View full text Add to dashboard Cite

To investigate pediatric residents' efforts to assess understanding in discussions about positive newborn screening test results. Newborn screening saves lives, but confusion about false-positive and carrier results often leads to psychosocial problems. Design: Explicit-criteria abstraction of transcripts of encounters with standardized parents of a fictitious infant found to carry cystic fibrosis or sickle cell hemoglobinopathy. Setting: Simulated doctor-patient encounter. Participants: Pediatric residents participating in an educational workshop on how to inform parents about positive newborn screening test results. Main Outcome Measures: Abstraction used an explicitcriteria data dictionary with definitions for 5 different ways to assess understanding. A "partial" designation was used for leading syntax or no pause for response. Results: Interabstractor reliability over 59 transcripts (2 per resident) was =0.93. Only 26 of 59 transcripts (44.1%) met definite criteria for at least 1 assessment of understanding. Most assessments were the less effective close-ended (37.3% of transcripts) and "OK?" question types (32.2% of transcripts). Only 3 transcripts met definite criteria for an open-ended assessment and no transcripts included a request for a teach-back, the type thought to be most effective. Four transcripts (6.8%) included an advance request for questions. With partial-criteria assessments included, an additional 31 transcripts (52%) were identified. Conclusions: The small number of assessments of understanding and the high fraction of less effective assessments do not bode well for parental understanding, especially for parents with limited health literacy. Training programs should address assessments of understanding, but quality improvement activities using these types of assessment methods may also be needed.

show abstract

The Art and Science of Chart Review

Cited by 128 publications

References 22 publications

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Pediatric Residents’ Use of Jargon During Counseling About Newborn Genetic Screening Results

Assessment of Parental Understanding by Pediatric Residents During Counseling After Newborn Genetic Screening

Contact Info

Product

Resources

About