This paper describes Project Spectra, a collective of open source developers that aims to build free and open source voice training technology for transgender people. We demonstrate how a design prioritizing the agency of trans users was made possible through sustained community collaboration. Using an autoethnographic approach, we discuss our community-based design process, which was documented with memos, online meetings and text conversations, sketches, and other data sources. We illustrate how we articulated our values as a group: deciding our programming framework (including a Statement of Principles), elaborating our "Experience Goals" (the feelings we wanted our design to elicit), and determining the features we wanted to implement in our app. We conclude with a reflection on the benefits and challenges of conducting community-based design research through an open-source organizational model.