A teacher’s assessment literacy refers to her or his demonstrated understanding of the principles behind selecting and designing tasks, judging student work, and interpreting and using assessment data to support student learning. This study examines the development of the task design aspect of assessment literacy in 12 Chinese language teachers as they participated in a two-year authentic assessment professional development program. By analysing the quality of assessment tasks designed by the teachers over time, we found that, although teachers quickly grasped many aspects of task design, they found it difficult to incorporate certain knowledge manipulation criteria into their assessments. The study provides insights into the contextual and discipline-embedded challenges that face language teachers with regard to assessment.