Evaluation of Jackknife and Bootstrap for Defining Confidence Intervals for Pairwise Agreement Measures

Severiano, Ana; Carriço, João André; Robinson, D. Ashley; Ramírez, Mario; Pinto, Francisco

doi:10.1371/journal.pone.0019539

Cited by 55 publications

(42 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Simpson's index of diversity (SID) was used to measure each typing method's ability to discriminate between genetically unique MRSA strains in the study sample (27,28). Probabilities of finding differences between SID values as or more extreme than those observed for rep-PCR, PFGE-80, and PFGE-95 were calculated using a jackknife pseudovalues resampling method (29). Global (adirectional) congruences between rep-PCR, PFGE-80, and PFGE-95 were assessed using the adjusted Rand coefficient (30).…”

Section: Bacterial Isolatesmentioning

confidence: 99%

“…Typing isolates in the opposite sequence usually results in a lower coefficient, hence the directional nature of this measure. Confidence intervals for the adjusted Rand and Wallace coefficients were estimated using a jackknife pseudovalues resampling method (29). Facility-level comparisons between rep-PCR and PFGE-95 were performed using MRSA isolates contemporaneously recovered from subjects in the two study nursing homes.…”

Section: Bacterial Isolatesmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Pulsed-Gel Electrophoresis and a Commercial Repetitive-Element PCR Method for Assessment of Methicillin-Resistant Staphylococcus aureus Clustering in Different Health Care Facilities

et al. 2014

View full text Add to dashboard Cite

b Pulsed-field gel electrophoresis (PFGE) is a common method used to type methicillin-resistant Staphylococcus aureus (MRSA) in nosocomial investigations and epidemiological studies but is time-consuming and methodologically challenging. We compared typing results obtained using a commercial repetitive-element PCR (rep-PCR) system with PFGE in a sample of 86 unique MRSA isolates recovered from subjects in an academic referral hospital and two nursing homes in the same geographic region. Both methods reliably assigned isolates to the same Centers for Disease Control and Prevention (CDC) pulsotype. PFGE was significantly more discriminatory (Simpson's index of diversity, 0.92 at the 95% strain similarity threshold) than the commercial rep-PCR system (Simpson's index of diversity, 0.58). The global (adjusted Rand coefficient, 0.10) and directional congruence (adjusted Wallace coefficient repPCR¡PFGE ‫؍‬ 0.06; adjusted Wallace coefficient PFGE¡repPCR ‫؍‬ 0.52) between the two methods was low. MRSA strains recovered from study nursing homes that were clonal when typed by the commercial rep-PCR method were frequently noted to be genetically distinct when typed using PFGE. These data suggest that the commercial rep-PCR has less utility than PFGE in small-scale epidemiological assessments of MRSA in health care settings.

show abstract

Section: Bacterial Isolatesmentioning

confidence: 99%

Section: Bacterial Isolatesmentioning

confidence: 99%

Comparison of Pulsed-Gel Electrophoresis and a Commercial Repetitive-Element PCR Method for Assessment of Methicillin-Resistant Staphylococcus aureus Clustering in Different Health Care Facilities

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Ridom Epicompare is a stand-alone software that allows the calculation of Simpson's index of diversity [50] and 95% confidence intervals [51] for a typing method, and the concordance indexes of Rand [52], adjusted Rand [53] and Wallace [54] for the assessment of congruence between typing methods [55]. The website www.comparingpartitions.info extends the features of Epicompare, by implementing confidence intervals for Wallace [56] and adjusted Rand [57] indices, as well as an adjusted Wallace coefficient and respective 95% confidence intervals [58]. These discriminatory and concordance indexes are now being used for evaluating the adequacy of a method for epidemiological typing.…”

Section: Tablementioning

confidence: 99%

Bioinformatics in bacterial molecular epidemiology and public health: databases, tools and the next-generation sequencing revolution

et al. 2013

View full text Add to dashboard Cite

Advances in typing methodologies have been the driving force in the field of molecular epidemiology of pathogens. The development of molecular methodologies, and more recently of DNA sequencing methods to complement and improve phenotypic identification methods, was accompanied by the generation of large amounts of data and the need to develop ways of storing and analysing them. Simultaneously, advances in computing allowed the development of specialised algorithms for image analysis, data sharing and integration, and for mining the ever larger amounts of accumulated data. In this review, we will discuss how bioinformatics accompanied the changes in bacterial molecular epidemiology. We will discuss the benefits for public health of specialised online typing databases and algorithms allowing for real-time data analysis and visualisation. The impact of the new and disruptive next-generation sequencing methodologies will be evaluated, and we will look ahead into these novel challenges.

show abstract

“…Both assumptions were already used in the derivation of the Wallace CI (11), and their validity was assessed by simulation of the sampling process as described previously (13) (Fig. A1).…”

Section: Derivation Of Aw the Derivation Of Aw Is Analogous To That mentioning

confidence: 99%

“…To assess whether the estimated W value is significantly different from the W i value, one can use the proposed Wallace 95% confidence interval (CI) (11,13). If the value of W i is within the CI of W, the null hypothesis of independence between classifications cannot be rejected with the respective confidence level (11).…”

mentioning

confidence: 99%

Adjusted Wallace Coefficient as a Measure of Congruence between Typing Methods

et al. 2011

Self Cite

View full text Add to dashboard Cite

We propose a new coefficient, the adjusted Wallace coefficient (AW), and corresponding confidence intervals (CI) as quantitative measures of congruence between typing methods. The performance of the derived CI was evaluated using simulated data. Published microbial typing data were used to demonstrate the advantages of AW over the Wallace coefficient.Several molecular epidemiology studies of clinically relevant microorganisms provide a characterization of isolates based on different typing methods (3, 5, 7). The informed choice of which typing method is more appropriate in a given clinical or microbiological research setting lies in the ability of the method to identify isolates of interest, the execution time, the cost-effectiveness, and the ease of interpretation of the results (16). Nevertheless, to support the decision, a quantitative comparison of the results of the typing methods should also be performed (3).Carriço et al. (3) proposed the use of the adjusted Rand coefficient (AR) and the Wallace coefficient (W) as measures to assess the congruence of typing methods. These have been applied in several studies comparing or proposing new typing methods (2,3,7,9,12,17). AR provides a measure of the overall agreement between two typing methods and corrects the previously used coefficient of typing concordance (18) for chance agreement, avoiding the overestimation of concordance between typing methods (8). W provides information about the directional agreement between typing methods. W A3B is the probability that, for a given data set, two individuals are classified together using method B if they have been classified together using method A. In spite of its simple interpretation, one can obtain high values of W due to chance alone. For instance, if method A creates a high number of partitions (such as pulsed-field gel electrophoresis [PFGE] subtypes) and method B creates only two (such as the presence or absence of a given gene), W A3B will be high but may not be different from the value expected by chance alone.The expected Wallace coefficient under independence (W i ) was previously proposed to evaluate whether the results of two typing methods could agree by chance alone (11). To assess whether the estimated W value is significantly different from the W i value, one can use the proposed Wallace 95% confidence interval (CI) (11,13). If the value of W i is within the CI of W, the null hypothesis of independence between classifications cannot be rejected with the respective confidence level (11). One way to directly take into account W i would be to calculate an adjusted version of W.Albatineh et al. (1) had previously discussed the correction for chance agreement for several similarity indices, including W. Although this correction was never applied in the context of microbial typing studies, others have previously acknowledged the importance and usefulness of such a correction (15).Derivation of AW. The adjusted Wallace coefficient (AW) is derived by following an approach similar to that used for AR (8):The ...

show abstract

Evaluation of Jackknife and Bootstrap for Defining Confidence Intervals for Pairwise Agreement Measures

Cited by 55 publications

References 26 publications

Comparison of Pulsed-Gel Electrophoresis and a Commercial Repetitive-Element PCR Method for Assessment of Methicillin-Resistant Staphylococcus aureus Clustering in Different Health Care Facilities

Comparison of Pulsed-Gel Electrophoresis and a Commercial Repetitive-Element PCR Method for Assessment of Methicillin-Resistant Staphylococcus aureus Clustering in Different Health Care Facilities

Bioinformatics in bacterial molecular epidemiology and public health: databases, tools and the next-generation sequencing revolution

Adjusted Wallace Coefficient as a Measure of Congruence between Typing Methods

Contact Info

Product

Resources

About