Comparative indexing: Terms supplied by biomedical authors and by document titles

Abstract

The original aim of this study was to obtain objective data bearing on the much argued question of whether author indexing is “good.” Author indexing of 285 documents reporting biomedical research was scored by comparing the author‐supplied terms (author set) for each paper with a criterion set of terms that was established by asking a group of 12 potential users to describe the same document. Terms in the document title (title set) were scored similarly. The average author set contained almost half of all the terms employed by more than one member of the user group and scored 73% of the maximal possible score, as compared with 44% for the average title set. When judged by the method and criterion employed here, author indexing is substantially better than indexing derived from document titles. The findings suggest that indicia supplied by an author should serve scientists in biomedical disciplines other than his own about as well as they serve his disciplinary colleagues. The general method developed for measuring indexing quality may represent a practical yardstick of wide applicability.

Tagging of Biomedical Articles on CiteULike: A Comparison of User, Author and Professional Indexing

“…A few studies have examined author keywords (Schultz, Schultz and Orr 1965;Kipp 2005;Gil-Leiva and Alonso-Arroyo 2007;Kipp 2007b;Strader 2009;Kipp 2011) while a few additional studies have examined title keywords (Bloomfield 1966;Voorbij 1989;Frost 1989;Ansari 2005, Jeong 2009. Schultz, Schultz and Orr (1965) examined author keywords and title keywords for 285 biomedical articles submitted for publication and found that author terms were more likely than title terms to match the controlled vocabulary terms. Kipp (2005) examined tags, author keywords and descriptors and found that while many tags matched descriptors or author keywords exactly there were also substantial differences.…”

Indexing consistency and quality

“…The data used in this investigation came from two sources. One source of data was a study performed by Schultz, Schultz and Orr (11) in which 285 biomedical documents were indexed by the author, by twelve biomedical scientists who were engaged in research in the area, and by eight professional indexers. For the purposes of this study, data from 29 of these documents was used.…”

A Method for Comparing Large Scale Inter-indexer Consistency Using IR Modeling

“…Inter-indexer consistency studies all point to high levels of inconsistency. Foundational studies on this topic go back to the 1960s, exemplified by the research undertaken by Zunde and Dexter (1969), whose analysis of earlier data (Schultz, Schultz, & Orr, 1965) pointed toward "power laws" and fuzzy sets as explanatory devices. Cooper's (1969) contemporary study questioned of the importance of consistency.…”