Asynchronous online discussion (AOD) is used in many tertiary education courses, and assessing it has been shown to enhance critical thinking outcomes. There has, however, been debate on what should be assessed and how the assessment should be implemented. The most common form of assessment involves grading the individual discussion contributions, but it has been suggested that employing a culminating task based on the AOD may be effective. This preliminary study compared the effect on student critical thinking of two approaches to AOD assessment: using a post-AOD assessment, and assessing the discussion contributions themselves. The results, though tentative, showed that while both assessment approaches resulted in significant improvements in student critical thinking, there was no difference in the impact on critical thinking skills between using the post-AOD assessment and assessing the discussion contributions. This result suggests that the form of assessment used in an AOD may be less important than the fact that assessment is included. Interviews with students also provided some insight into ways in which they perceived the discussion environment had contributed to their critical thinking skills. The findings of this study pave the way for further research in this important area.