Application of robotics and automation in pasture-grazed agriculture is in an emergent phase. Technology developers face significant challenges due to aspects such as the complex and dynamic nature of biological systems, relative cost of technology versus farm labor costs, and specific market characteristics in agriculture. Overlaying this are socio-ethical issues around technology development, and aspects of responsible research and innovation. There are numerous examples of technology being developed but not adopted in pasture-grazed farming, despite the potential benefits to farmers and/or society, highlighting a disconnect in the innovation system. In this perspective paper, we propose a “responsibility by design” approach to robotics and automation innovation, using development of batch robotic milking in pasture-grazed dairy farming as a case study. The framework we develop is used to highlight the wider considerations that technology developers and policy makers need to consider when envisaging future innovation trajectories for robotics in smart farming. These considerations include the impact on work design, worker well-being and safety, changes to farming systems, and the influences of market and regulatory constraints.