Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland

Kulczycki, Emanuel; Rozkosz, Ewa A.

doi:10.1007/s11192-017-2261-x

Cited by 42 publications

(32 citation statements)

References 61 publications

(83 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This can be found only after the teachers evaluated their teaching and learning activity. Kulczycki and Rozkosz (2017) point out that the evaluation helps the teacher to assess whether the learning has been successful and to clarify the teachers' expectation. The findings can be seen in the following excerpts:…”

Section: Resultsmentioning

confidence: 99%

Teachers� Creativity in Designing Learning Activities: Sustaining Students� Motivation

Khikmah

2019

ERJEE

View full text Add to dashboard Cite

Sustaining students� motivation during teaching and learning process, unquestionably, is considered as the most challenging task. Therefore, the teachers have to be mindful and creative in designing an activity as it is the core of the lesson. However, in practice, some teachers are sometimes found to aimlessly use an activity as they run out of idea. Thus, this study is conducted to serve as a reference for the teachers in designing a specific activity that helps the students stay motivated. A descriptive qualitative was employed as the research design to gain deeper analysis on types of activity that encourage the students to learn English. The data were collected through interview which involved 5 professional English teachers as participants. The result revealed that the design of the activity should be appealing and correspond to the students� interest, age, level of proficiency, school policy, materials and media availability. Thus, it is suggested that the teachers should be brave to try out new activity and attend some teachers� training, such as CELTA or DELTA, to improve their teaching skills.

show abstract

Section: Resultsmentioning

confidence: 99%

Teachers� Creativity in Designing Learning Activities: Sustaining Students� Motivation

Khikmah

2019

ERJEE

View full text Add to dashboard Cite

show abstract

“…For a journal article, a scientific unit could obtain from 1 to 50 points according to the rules of the Polish Journal Ranking (Kulczycki & Rozkosz, 2017). For a monograph, a scientific unit could obtain 25 points (and 5 points for a chapter or edited volume).…”

Section: Research Papermentioning

confidence: 99%

“…The result of research evaluation was one of the key factors, which influenced the final funding, with the most important parameter of evaluation being publications. Journal articles were assessed according to bibliometric based ranking (Kulczycki & Rozkosz, 2017), whereas scholarly book publications were assessed according to several formal criteria, such as book length (Kulczycki, 2018).…”

Section: Introductionmentioning

confidence: 99%

Redesigning the Model of Book Evaluation in the Polish Performance-based Research Funding System

Liu

He³

2018

Journal of Data and Information Science

Self Cite

View full text Add to dashboard Cite

Purpose This study aims to present the key systemic changes in the Polish book evaluation model to focus on the publisher list, as inspired by the Norwegian Model. Design/methodology/approach In this study we reconstruct the framework of the 2010 and 2018 models of book evaluation in Poland within the performance-based research funding system. Findings For almost 20 years the book evaluation system in Poland has been based on the verification of various technical criteria (e.g. length of the book). The new 2018 model is based on the principle of prestige inheritance (a book is worth as much as its publisher is) and is inspired by the publisher list used in the Norwegian Model. In this paper, we argue that this solution may be a more balanced policy instrument than the previous 2010 model in which neither the quality of the publisher nor the quality of the book played any role in the evaluation. Research limitations We work from the framework of the 2018 model of book evaluation specified in the law on higher education and science from 20 July 2018, as implementation acts are not available yet. Practical implications This study may provide a valuable point of reference on how structural reforms in the research evaluation model were implemented on a country level. The results of this study may be interesting to policy makers, stakeholders and researchers focused on science policy. Originality/value This is the very first study that presents the new framework of the Polish research evaluation model and policy instruments for scholarly book evaluation. We describe what motivated policy makers to change the book evaluation model, and what arguments were explicitly raised to argue for the new solution.

show abstract

“…An example of such issues is the procedure used by the Polish office of institutional research and assessment, whose performance-based research funding system is the so-called Comprehensive Evaluation of Scientific Units (CESU). In this case, the evaluation ranking is elaborated by a mixed procedure taking into account the bibliometric indices and the experts' assessment (see [1,2]). The discussion of which is the best way to define a rank-based evaluation system is one of the most recurrent topics in information science and its applications to research management.…”

Section: Introductionmentioning

confidence: 99%

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p-Variations

2017

View full text Add to dashboard Cite

Journal rankings of specific research fields are often used for evaluation purposes, both of authors and institutions. These rankings can be defined by means of several methods, as expert assessment, scholarly-based agreements, or by the ordering induced by a numeric index associated to the prestige of the journals. In order to be efficient and accepted by the research community, it must preserve the ordering over time, at least up to a point. Otherwise, the procedure for defining the ranking must be revised to assure that it reflects the presumably stable characteristic "prestige" that it claims to be quantifying. A mathematical model based on fractional p-variations of the values of the order number of each journal in a time series of journal rankings is explained, and its main properties are shown. As an example, we study the evolution of two given ordered lists of journals through an eleven-year series. These journal ranks are defined by using the 2-year Impact Factor of Thomson-Reuters (nowadays Clarivate Analytics) lists for MATHEMATICS and PHYSICS, APPLIED from 2002 to 2013. As an application of our model, we define an index that precludes the use of journal ranks for evaluation purposes when some minimal requirements on the associated fractional p-variations are not satisfied. The final conclusion is that the list of mathematics does not satisfy the requirements on the p-variations, while the list of applied physics does.

show abstract

Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland

Cited by 42 publications

References 61 publications

Teachers� Creativity in Designing Learning Activities: Sustaining Students� Motivation

Teachers� Creativity in Designing Learning Activities: Sustaining Students� Motivation

Redesigning the Model of Book Evaluation in the Polish Performance-based Research Funding System

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p-Variations

Contact Info

Product

Resources

About