Critical evaluation skills when using online information are considered important in many research and education frameworks; critical thinking and information literacy are cited as key twenty-first century skills for students. Higher education may play a special role in promoting students' skills in critically evaluating (online) sources. Today, higher education students are more likely to use the Internet instead of offline sources such as textbooks when studying for exams. However, far from being a value-neutral, curated learning environment, the Internet poses various challenges, including a large amount of incomplete, contradictory, erroneous, and biased information. With low barriers to online publication, the responsibility to access, select, process, and use suitable relevant and trustworthy information rests with the (self-directed) learner. Despite the central importance of critically evaluating online information, its assessment in higher education is still an emerging field. In this paper, we present a newly developed theoretical-conceptual framework for Critical Online Reasoning (COR), situated in relation to prior approaches (“information problem-solving,” “multiple-source comprehension,” “web credibility,” “informal argumentation,” “critical thinking”), along with an evidence-centered assessment framework and its preliminary validation. In 2016, the Stanford History Education Group developed and validated the assessment of Civic Online Reasoning for the United States. At the college level, this assessment holistically measures students' web searches and evaluation of online information using open Internet searches and real websites. Our initial adaptation and validation indicated a need to further develop the construct and assessment framework for evaluating higher education students in Germany across disciplines over their course of studies. Based on our literature review and prior analyses, we classified COR abilities into three uniquely combined facets: (i) online information acquisition, (ii) critical information evaluation, and (iii) reasoning based on evidence, argumentation, and synthesis. We modeled COR ability from a behavior, content, process, and development perspective, specifying scoring rubrics in an evidence-centered design. Preliminary validation results from expert interviews and content analysis indicated that the assessment covers typical online media and challenges for higher education students in Germany and contains cues to tap modeled COR abilities. We close with a discussion of ongoing research and potentials for future development.

The Internet has become one of the main sources of information for university students' learning. Since anyone can disseminate content online, however, the Internet is full of irrelevant, biased, or even false information. Thus, students' ability to use online information in a critical-reflective manner is of crucial importance. In our study, we used a framework for the assessment of students' critical online reasoning (COR) to measure university students' ability to critically use information from online sources and to reason on contentious issues based on online information. In addition to analyzing students' COR by evaluating their open-ended short answers, we also investigated the students' web search behavior and the quality of the websites they visited and used during this assessment. We analyzed both the number and type of websites as well as the quality of the information these websites provide. Finally, we investigated to what extent students' web search behavior as well as the quality of the used website contents are related to higher task performance. To investigate this question, we used five computer-based performance tasks and asked 160 students from two German universities to perform a time-restricted open web search to respond to the open-ended questions presented in the tasks. The written responses were evaluated by two independent human raters. To analyze the students' browsing history, we developed a coding manual and conducted a quantitative content analysis for a subsample of 50 students. The number of visited webpages per participant per task ranged from 1 to 9. Concerning the type of website, the participants relied especially on established news sites and Wikipedia. For instance, we found that the number of visited websites and the critical discussion of sources provided on the websites correlated positively with students' scores. The identified relationships between students' web search behavior, their performance in the CORA tasks, and the qualitative website characteristics are presented and critically discussed in terms of limitations of this study and implications for further research.

Abstract. University students' knowledge and understanding of economics have mostly been investigated cross-sectionally; however, longitudinal analyses are needed to determine which factors influence knowledge development and to draw valid conclusions based on test results. In the WiWiKom II project, a quasi-experimental longitudinal study was conducted assessing bachelor students of business and economics over the course of their studies in Germany (N = 39 universities). In this project, running from 2016 – 2019, the test-takers complete the WiWiKom II-test of economic knowledge (adapted TUCE4G and TEL4G items) and a general cognitive ability test (BEFKI 11) at four measurement points, each one year apart. In this paper, we describe the validation of the test instrument based on the data from the first measurement (winter semester 2016/17). We compare students' results on the economic knowledge test and the general cognitive abilities test between first-year students of economic sciences (N = 3,710) and social sciences (N = 1,347) to determine the discriminant validity of the economic knowledge test. The findings from the confirmatory factor analyses presented here show that the items on general cognitive ability and economic knowledge are empirically separable. As expected, there were no differences in the factorial structure between the comparison groups (economic vs. social science students) at the beginning of their studies.

No abstract

No abstract