The concept of 'responsible innovation' (RI) or 'responsible research and innovation' (RRI) is rapidly gaining currency. However, a persistent critique is that without more concrete elaboration, the interpretive flexibility of the concept is so broad as to effectively render it meaningless. The articulation of quality criteria and indicators therefore seems crucial for RRI to be understood and operationalized by researchers, research funders, innovators and other relevant stakeholders. In this paper, we specifically draw on our knowledge and experience from the transdisciplinary research community, combined with recent multistakeholder deliberative work on the concrete case of nanoremediation, to make an offering on the challenge of articulating quality criteria and approaches to evaluate RRI. In doing so, we present an iteratively arrived at set of quality criteria, designate significant elements of each of these, and then develop an evaluative rubric of performance indicators across them. While the criteria and rubric we present were initiated through the specific context of our work on advancing RI in the research, development and use of nanoparticles for environmental remediation, we believe that they can serve as a useful example for how evaluative criteria and approaches can be developed and offer a helpful frame for sponsoring and structuring the ongoing conversations on quality criteria and indicators that are necessary if RRI is to reach its full potential.