It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. For investigating this topic, we propose a new inwardness indicator able to gauge the degree of scientific self-referentiality of a country. Inwardness is defined as the proportion of citations coming from the country over the total number of citations gathered by the country. A comparative analysis of the trends for the G10 countries in the years 2000-2016 reveals a net increase of the Italian inwardness. Italy became, both globally and for a large majority of the research fields, the country with the highest inwardness and the lowest rate of international collaborations. The change in the Italian trend occurs in the years following the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A most likely explanation of the peculiar Italian trend is a generalized strategic use of citations in the Italian scientific community, both in the form of strategic author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts.

The exploratory analysis developed in this paper relies on the hypothesis that each editor possesses some power in the definition of the editorial policy of her journal. Consequently if the same scholar sits on the board of editors of two journals, those journals could have some common elements in their editorial policies. The proximity of the editorial policies of two scientific journals can be assessed by the number of common editors sitting on their boards. A database of all editors of ECONLIT journals is used. The structure of the network generated by interlocking editorship is explored by applying the instruments of network analysis. Evidence has been found of a compact network containing different components. This is interpreted as the result of a plurality of perspectives about the appropriate methods for the investigation of problems and the construction of theories within the domain of economics

During the Italian research assessment exercise, the national agency ANVUR performed an experiment to assess agreement between grades attributed to journal articles by informed peer review (IR) and by bibliometrics. A sample of articles was evaluated by using both methods and agreement was analyzed by weighted Cohen’s kappas. ANVUR presented results as indicating an overall “good” or “more than adequate” agreement. This paper re-examines the experiment results according to the available statistical guidelines for interpreting kappa values, by showing that the degree of agreement (always in the range 0.09–0.42) has to be interpreted, for all research fields, as unacceptable, poor or, in a few cases, as, at most, fair. The only notable exception, confirmed also by a statistical meta-analysis, was a moderate agreement for economics and statistics (Area 13) and its sub-fields. We show that the experiment protocol adopted in Area 13 was substantially modified with respect to all the other research fields, to the point that results for economics and statistics have to be considered as fatally flawed. The evidence of a poor agreement supports the conclusion that IR and bibliometrics do not produce similar results, and that the adoption of both methods in the Italian research assessment possibly introduced systematic and unknown biases in its final results. The conclusion reached by ANVUR must be reversed: the available evidence does not justify at all the joint use of IR and bibliometrics within the same research assessment exercise

This paper explores, by using suitable quantitative techniques, to what extent the intellectual proximity among scholarly journals is also a proximity in terms of social communities gathered around the journals. Three fields are considered: statistics, economics and information and library sciences. Co-citation networks (CC) represent the intellectual proximity among journals.The academic communities around the journals are represented by considering the networks of journals generated by authors writing in more than one journal (interlocking authorship: IA), and the networks generated by scholars sitting in the editorial board of more than one journal (interlocking editorship: IE). For comparing the whole structure of the networks, the dissimilarity matrices are considered. The CC, IE and IA networks appear to be correlated for the three fields. The strongest correlations is between CC and IA for the three fields. Lower and similar correlations are obtained for CC and IE, and for IE and IA. The CC, IE and IA networks are then partitioned in communities. Information and library sciences is the field where communities are more easily detectable, while the most difficult field is economics. The degrees of association among the detected communities show that they are not independent. For all the fields, the strongest association is between CC and IA networks; the minimum level of association is between IE and CC. Overall, these results indicate that the intellectual proximity is also a proximity among authors and among editors of the journals. Thus, the three maps of editorial power, intellectual proximity and authors communities tell similar stories.

An original cross-sectional dataset referring to a medium-sized Italian university is implemented in order to analyze the determinants of scientific research production at individual level. The dataset includes 942 permanent researchers of various scientific sectors for a three-year time-span (2008)(2009)(2010). Three different indicators -based on the number of publications and/or citations -are considered as response variables. The corresponding distributions are highly skewed and display an excess of zero-valued observations. In this setting, the goodness-of-fit of several Poisson mixture regression models are explored by assuming an extensive set of explanatory variables. As to the personal observable characteristics of the researchers, the results emphasize the age effect and the gender productivity gap -as previously documented by existing studies. Analogously, the analysis confirm that productivity is strongly affected by the publication and citation practices adopted in different scientific disciplines. The empirical evidence on the connection between teaching and research activities suggests that no univocal substitution or complementarity thesis can be claimed: a major teaching load does not affect the odds to be a non-active researcher and does not significantly reduce the number of publications for active researchers. In addition, new evidence emerges on the effect of researchers administrative tasks -which seem to be negatively related with researcher's productivity -and on the composition of departments. Researchers' productivity is apparently enhanced by operating in department filled with more administrative and technical staff, and it is not significantly affected by the composition of the department in terms of senior/junior researchers.