The field of epitranscriptomics is undergoing a technology-driven revolution. During past decades, RNA modifications like N6-methyladenosine (m6A), pseudouridine (ψ), and 5-methylcytosine (m5C) became acknowledged for playing critical roles in gene expression regulation, RNA stability, and translation efficiency. Among modification-aware sequencing approaches, direct RNA sequencing by Oxford Nanopore Technologies (ONT) enabled the detection of modifications in native RNA, by capturing and storing properties of noncanonical RNA nucleosides in raw data. Consequently, the field's cutting edge has a heavy component in computer science, opening new avenues of cooperation across the community, as exchanging data is as impactful as exchanging samples. Therefore, we seize the occasion to bring scientists together within the RMaP challenge to advance solutions for RNA modification detection and discuss current ideas, problems and approaches. Here, we show several computational methods to detect the most researched mRNA modifications (m6A, ψ, and m5C). Results demonstrate that a low prediction error and a high prediction accuracy can be achieved on these modifications across different approaches and algorithms. The RMaP challenge marks a substantial step towards improving algorithms' comparability, reliability, and consistency in RNA modification prediction. It points out the deficits in this young field that need to be addressed in further challenges.