Trends and Challenges in Measuring Performance of Reviewer Paper Assignment

Patil, Aboli H.; Mahalle, Parikshit N.

doi:10.1016/j.procs.2020.04.077

Cited by 8 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The used evaluation measures are also quite distinct and numerous. This finding is consistent with the one made by Patil and Mahalle [33] in their review of performance measures used for the evaluation of RAP solutions. In addition to several traditional information retrieval evaluation measures, the selected studies used a myriad of custom defined evaluation measures, many of which were used in one study only.…”

Section: Summary Of Evidencesupporting

confidence: 92%

Reviewer Paper Assignment Problem – A Brief Review

Patil¹,

Mahalle²

2022

Handbook on ICT in Developing Countries

View full text Add to dashboard Cite

The quality of peer review, and consequently the published research, depends to a large extent on the ability to recruit adequate reviewers for submitted papers. However, finding such reviewers is an increasingly difficult task. To alleviate this challenge, numerous solutions have been suggested for automated association of papers with "well matching" reviewers -the task often referred to as reviewer assignment problem (RAP). Yet, to our knowledge, a recent systematic synthesis of the RAP-related literature is missing. To fill this gap and support further RAP-related research, we present a scoping review on computational approaches to RAP. Following the latest methodological guidance for scoping reviews, we have collected and synthesised the literature on RAP published over the last five years (Jan 2016 -Sept 2021). Specifically, the review is focused on the following aspects of RAP research: i) the overall approach to RAP; ii) the criteria used for reviewer selection; iii) the modelling of candidate reviewers and submissions; iv) the computational methods for matching reviewers and submissions; and v) the evaluation methods used for assessing the performance of the proposed solutions. The paper summarises and discusses the findings for each of the aforementioned aspects of RAP research and suggests future research directions.

show abstract

Section: Summary Of Evidencesupporting

confidence: 92%

Reviewer Paper Assignment Problem – A Brief Review

Patil¹,

Mahalle²

2022

Handbook on ICT in Developing Countries

View full text Add to dashboard Cite

show abstract

“…In [12], The Reviewer Assignment Problem (RAP) is presented. This is one of the most important and apparent processes that involves the process of assigning a reviewer to a paper.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

“…In general, the assignment problem is formulated as given in Section 4. The variants of the assignment problems presented in sources [7][8][9][10][11][12][13][14] are obtained by modifying the constraints and objective function of Table 1.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

Development of an accelerating hungarian method for assignment problems

Munapo

2020

EEJET

View full text Add to dashboard Cite

The Hungarian method is a well-known method for solving the assignment problem. This method was developed and published in 1955. It was named the Hungarian method because two theorems from two Hungarian mathematicians were used. In 1957, it was noticed that this algorithm is strongly polynomial and has a complexity of order O(n 4) This is the reason why the Hungarian method is also known as the Kuhn-Munkres algorithm. Later on, in 1971 the complexity of the method was improved to order O(n 3) A smallest uncovered element is selected to create a single zero at every iteration. This is a weakness and is alleviated by selecting more than one smallest uncovered element thus creating more than one zero at every iteration to come up with what we now call the Accelerating Hungarian (AH) method. From the numerical illustration of the Hungarian method given in this paper, we require 6 iterations to reach optimality. It can also be shown that selecting a single smallest uncovered element (e s) makes the Hungarian method inefficient when creating zeros. From the same numerical illustration of the proposed algorithm (AH) also given in this paper, it can be noted that only one iteration is required to reach optimality and that a total of six zeros are created in one iteration. Assignment model and the Hungarian method have application in addressing the Weapon Target Assignment (WTA) problem. This is the problem of assigning weapons to targets while considering the maximum probability of kill. The assignment problem is also used in the scheduling problem of physicians and medical staff in the outpatient department of large hospitals with multi-branches. The mathematical modelling of this assignment problem results in complex problems. A hybrid meta-heuristic algorithm SCA-VNS combining a Sine Cosine Algorithm (SCA) and Variable Neighbourhood Search (VNS) based on the Iterated Hungarian algorithm is normally used

show abstract

“…The publication of a scientific article often needs to undergo multiple steps, among which peer review plays an essential role. [1] Appropriate reviewers can offer valuable suggestions to authors for manuscript revision and assist editors in decision-making, thereby helping to improve the quality of the paper and facilitate the peer review process. [2] Despite editors being able to rely on their reviewer database and authors' recommendations to choose reviewers, finding an adequate number of qualified reviewers who are experienced in the research area of the submitted manuscripts is still tough, given the rapid growth of submissions.…”

Section: Introductionmentioning

confidence: 99%

A multilayer network diffusion-based model for reviewer recommendation

Huang 黄,

Xu 徐,

Cai 蔡

et al. 2024

Chinese Phys. B

View full text Add to dashboard Cite

With the rapid growth of manuscript submissions, finding eligible reviewers for every submission has become a heavy task. Recommender systems are powerful tools developed in computer science and information science to deal with this problem. However, most existing approaches resort to text mining techniques to match manuscripts with potential reviewers, which require high-quality textual information to perform well. In this paper, we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network, with no requirement for textual information. The network incorporates the relationship of scholar-paper pairs, the collaboration among scholars, and the bibliographic coupling among papers. Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing, with improvements of over 7.62% in Recall, 5.66% in Hit Rate, and 47.53% in Ranking Score. Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem, which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

show abstract

Trends and Challenges in Measuring Performance of Reviewer Paper Assignment

Cited by 8 publications

References 18 publications

Reviewer Paper Assignment Problem – A Brief Review

Reviewer Paper Assignment Problem – A Brief Review

Development of an accelerating hungarian method for assignment problems

A multilayer network diffusion-based model for reviewer recommendation

Contact Info

Product

Resources

About