The need to integrate refugees into the labour markets across the European Union has put public service organisations at the forefront of support provision, but little is known about their role and challenges. This study investigates how these organisations in Austria, Finland, Germany and Sweden organise labour market integration support measures for refugees and the challenges they face in doing so. Using a relational framework, we draw on the understanding that labour market integration occurs at the interplay between the institutional, organisational and individual level. Thematic analysis based on 23 interviews with representatives from public service organisations and their partners revealed challenges across different levels related to specificities of refugees, mental health issues, training and motivation of staff and public‐private cooperation. We show the interrelations of these levels and implications on public service organisations. We propose a holistic approach for labour market integration support, with the organisations at the centre.