Collaboration is an important skill for children to learn. In this paper we present a small-scale study exploring how technology can be used to elicit collaboration between children. We developed a prototype of a tablet-based robot ('surfacebot') that tried to perform a specific task, while children acted as tutors by giving feedback on the surfacebot's actions. The surfacebot used the feedback to improve its actions by means of reinforcement learning. A pilot study with an early prototype showed that children were engaged and provided consistent feedback to the surfacebot, but showed little collaboration. Instead they made individual decisions and took turns in providing feedback. Based on these observations we made several changes to the prototype that were meant to stimulate collaboration between the children. In our main study with the revised prototype, we measured collaboration using an annotation scheme based on a collaborative problem solving framework. The results suggest that the revisions of the prototype indeed led to more extensive collaborative behavior, with 4 of the 9 participating pairs of children establishing a division of roles that necessitated perspective taking and mutual exchange of information.