Evaluating recommendation and search in the labor market

Abstract

This study evaluates the most popular recommender system algorithms for use on both sides of the labor market: job recommendation and job seeker recommendation. Recent research shows the drawbacks of focusing solely on predictive power when evaluating recommender systems, which become especially prominent in job-and job seeker recommendation, where aspects such as reciprocity and item spread are two other vital performance metrics for the quality of recommendations. Besides evaluating using these extra metrics, we compare recommendation with search using free text search engines. We measure what is gained, and what is lost when consuming items (jobs and job seekers) retrieved using search versus items presented via a recommender system. Based on insights in date recommendation literature, we propose changes to rating matrix construction aimed at mitigating the drawbacks of recommendation in the labor market. Our results, obtained from extensive experimentation on three datasets gathered from the Flemish public employment services, show that popular recommender algorithms perform significantly worse than user search in terms of reciprocity. Furthermore, we show that by swapping the rating matrices between two sides of a reciprocal recommender context, we can outperform user search in terms of reciprocity with limited trade off in predictive power. The insights from this research can help actors in the labor market to better understand the positioning of recommendation versus search, and to provide better job recommendations and job seeker recommendations.

Reciprocal Recommender Systems: Analysis of State-of-Art Literature, Challenges and Opportunities towards Social Recommendation

“…Meanwhile, in [135] it is revealed that implicit feedback is a more powerful indicator of users' broad interests. The same authors suggest in [136] that classical RS tend to predict better from the job seeker viewpoint, but more attention should be paid to reciprocity from the recruiter side.…”

“…Reusens et al [135,136] Identifies best indicators of job seekers' preferences. Analyzes the impact of reciprocity versus non-reciprocity.…”

Job Recommender Systems: A Review

“…Reusens et al [103] compare several CF approaches with keyword search in terms of recall and reciprocity, both for job recommendation and job seeker recommendation. With respect to job recommendations, although traditional CF approaches performed acceptably in terms of recall, they were inferior to keyword search in terms of reciprocity.…”

“…Although some contributions do take this reciprocal nature into account, and usually with success (e.g., [17,103,70]), most contributions consider the one-dimensional job seeker perspective, neglecting possible harm to the employer and/or job portal.…”

A New Similarity Computing Model of Collaborative Filtering

“…(3) The user's common scoring project is a very important factor. Let (1,2,2,1) and (5,5,4,4) be the rating vectors of two users. Calculating only the common scoring project will result in inaccurate similarity,for example, the Jaccard similarity calculation does not take into account the user's specific rating value, so that the similarity of the two is as high as 1, If you don't consider this factor at all, you will lose some important information, such as MSD.…”