The number needed to read—a new measure of journal value

Toth, Ben; Gray, James; Brice, Anne

doi:10.1111/j.1471-1842.2005.00568.x

Cited by 18 publications

(13 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… a Recall = A/(A+C); Precision = A/(A+B); F-measure = 2*precision*recall/(precision + recall); Number needed to read = 1/precision; Fallout = B/(B+D) [15,16]…”

Section: Methodsmentioning

confidence: 99%

Development and Validation of Filters for the Retrieval of Studies of Clinical Examination From Medline

N¹,

Badgett²,

Pí³

et al. 2011

J Med Internet Res

View full text Add to dashboard Cite

BackgroundEfficiently finding clinical examination studies—studies that quantify the value of symptoms and signs in the diagnosis of disease—is becoming increasingly difficult. Filters developed to retrieve studies of diagnosis from Medline lack specificity because they also retrieve large numbers of studies on the diagnostic value of imaging and laboratory tests.ObjectiveThe objective was to develop filters for retrieving clinical examination studies from Medline.MethodsWe developed filters in a training dataset and validated them in a testing database. We created the training database by hand searching 161 journals (n = 52,636 studies). We evaluated the recall and precision of 65 candidate single-term filters in identifying studies that reported the sensitivity and specificity of symptoms or signs in the training database. To identify best combinations of these search terms, we used recursive partitioning. The best-performing filters in the training database as well as 13 previously developed filters were evaluated in a testing database (n = 431,120 studies). We also examined the impact of examining reference lists of included articles on recall.ResultsIn the training database, the single-term filters with the highest recall (95%) and the highest precision (8.4%) were diagnosis[subheading] and “medical history taking”[MeSH], respectively. The multiple-term filter developed using recursive partitioning (the RP filter) had a recall of 100% and a precision of 89% in the training database. In the testing database, the Haynes-2004-Sensitive filter (recall 98%, precision 0.13%) and the RP filter (recall 89%, precision 0.52%) showed the best performance. The recall of these two filters increased to 99% and 94% respectively with review of the reference lists of the included articles.ConclusionsRecursive partitioning appears to be a useful method of developing search filters. The empirical search filters proposed here can assist in the retrieval of clinical examination studies from Medline; however, because of the low precision of the search strategies, retrieving relevant studies remains challenging. Improving precision may require systematic changes in the tagging of articles by the National Library of Medicine.

show abstract

“… a Recall = A/(A+C); Precision = A/(A+B); F-measure = 2*precision*recall/(precision + recall); Number needed to read = 1/precision; Fallout = B/(B+D) [15,16]…”

Section: Methodsmentioning

confidence: 99%

Development and Validation of Filters for the Retrieval of Studies of Clinical Examination From Medline

N¹,

Badgett²,

Pí³

et al. 2011

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…The NNR was calculated as the inverse of the precision. (Toth, Gray, & Brice , 2005) Calculations were made with R statistical software package, version 2.11.1. (R Development Core Team , 2012) Outcomes were tested with the Lilliefors normality test for a normal distribution which would permit reporting means.…”

Section: Reference Standardmentioning

confidence: 99%

An experimental search strategy retrieves more precise results than PubMed and Google for questions about medical interventions

Badgett

Dylla

Megison³

et al. 2014

Preprint

View full text Add to dashboard Cite

Objective: To compare the precision of a search strategy designed specifically to retrieve randomized controlled trials (RCTs) and systematic reviews of RCTs with search strategies designed for broader purposes. Methods:We designed an experimental search strategy that automatically revised searches up to five times by using increasingly restrictive queries as long at least 50 citations were retrieved. We compared the ability of the experimental and alternative strategies to retrieve studies relevant to 312 test questions.The primary outcome, search precision, was defined for each strategy as the proportion of relevant, high quality citations among the first 50 citations retrieved. Results:The experimental strategy had the highest median precision (5.5%; interquartile range [IQR]: 0% -12%) followed by the narrow strategy of the PubMed Clinical Queries (4.0%; IQR: 0% -10%). The experimental strategy found the most high quality citations (median 2; IQR: 0 -6) and was the strategy most likely to find at least one high quality citation (73% of searches; 95% confidence interval 68% -78%). All comparisons were statistically significant. Conclusions:The experimental strategy performed the best in all outcomes although all strategies had low precision.PeerJ PrePrints | http://dx.doi.org/10.7287/peerj.preprints.604v1 | CC-BY 4.0

show abstract

“…How the 1/precision formula was determined for NNR is not clear in Bachmann et al or (2004) define NRR as "number of articles that are needed to be read to obtain one that is clinically relevant and has high-quality methods" (p. 13). It has also been proposed as an alternative to journal impact factor as an indicator of the quality of a particular journal (Toth, Muir, & Brice, 2005). The use of the existing methodology from the systematic review provides the authors with a model to compare their results to, but readers may wish to extend their reading to the original publication to get the most out of this more recent study.…”

Section: Commentarymentioning

confidence: 99%

PubMed is Slightly More Sensitive but More Time Intensive to Search than Ovid MEDLINE

Merkley¹

2011

EBLIP

View full text Add to dashboard Cite

Objective -To compare the results of searching the MEDLINE database through Ovid and the free online version of PubMed administered by the National Library of Medicine for randomized controlled trials on the subject of the drug methotrexate (MTX) for patients suffering from rheumatoid arthritis. Design -Comparative analysis of search results.Setting -Searches conducted by researchers affiliated with Mahidol University in Bangkok, Thailand, and the University of Toronto and the University Health Network in Toronto, Ontario.

show abstract

The number needed to read—a new measure of journal value

Cited by 18 publications

References 2 publications

Development and Validation of Filters for the Retrieval of Studies of Clinical Examination From Medline

Development and Validation of Filters for the Retrieval of Studies of Clinical Examination From Medline

An experimental search strategy retrieves more precise results than PubMed and Google for questions about medical interventions

PubMed is Slightly More Sensitive but More Time Intensive to Search than Ovid MEDLINE

Contact Info

Product

Resources

About