Experimental studies of biotic interactions in real field conditions are essential to understand the structure and functioning of ecological networks. The use of artificial caterpillars to mimic actual prey availability is generally seen as a standard approach to compare the activity and diversity of predators along environmental gradients. Yet, even with standardized material and procedures, biases may still affect data from multiple observers with different expertise. We used pictures of artificial caterpillars with or without various predation marks, in an online survey that was targeted for the participants of the project, to evaluate the reliability of predation marks identification made by non-scientists and by scientists with and without previous experience in predation mark identification. Pictures of artificial caterpillars displayed typical marks left by birds, mammals and arthropods, as well as non-predation marks (false positive). 357 respondents scanned 7140 pictures of these pictures. Self-declared scientists were more confident and accurate in their observations than non-scientists, but the differences in correct identifications among scientists and non-scientists were low. Self-declared scientists with previous experience were also more accurate than scientists without previous experience, while there were no differences in self-confidence among scientists with and without previous experience. Accuracy in predation mark identification did not differ among types of predators, but respondents were more keen to identify marks left by birds or mammals than arthropods. Our results have practical implications for the design of multi-observer projects relying on artificial caterpillars as a proxy to assess predation intensity, in particular in the context of citizen science.