Snap judgement of publication quality

Abstract

A comprehensive qualitative assessment of a researcher's contribution in a specific narrow discipline takes time and expertise. Given the shortage of both in typical situations, a researcher's productivity is often judged quantitatively by the number of publications, their acceptance ratios, and citation counts that are highly discipline-dependent. We believe such a cursory evaluation is unavoidable and suggest a more intuitive discipline-agnostic approach for perfunctory assessment. We propose a metric called peers' reputation (PR) which ties the selectivity of a publication venue with the reputations of authors' institutions. Briefly, PR conveys the selectivity of a conference with a tuple, say <1/3 , 20>, indicating that 1/3 of the papers at that conference are from the top 20 universities. We compute PR for networking research publication venues, and argue that PR is a better indicator of selectivity than acceptance ratio, and many conferences have similar or better PR than journals. While these insights are not necessarily new to researchers in the networking community, PR metric helps inform a dean or a provost that getting a paper accepted at MobiCom involves competing with researchers from the top 20 US universities.