The aim of this study is to determine the level of achievement of students participating in Programme for International Student Assessment (PISA) 2003 and PISA 2012 tests in Turkey according to questions in the mathematical literacy test. This study is a descriptive survey. Within the scope of the study, the mathematical literacy test items were classified as multiple-choice, complex multiple-choice and constructed response items according to the different question types. The ratio of correct and partially correct and incorrect response given to each question type has been determined. Findings show that the achievements of students differ according to different types of questions. While the question type with the highest success average in the PISA 2003 test was multiple-choice, students got the highest scores from complex multiple-choice questions in the PISA 2012 test. The questionnaire with the lowest success average was found to be complex multiple-choice questions in the PISA 2003 test while students got the lowest scores from constructed response items in the PISA 2012 test. According to the constructivist education approach effectuated in 2005-2006 academic year, it is expected to observe a rise in constructed response question type; however, findings of the study reveal that the success of constructed response questions is decreased according to the application years.