Evaluating EAP Textbooks for Indonesian University Students Using Impressionistic and In-depth Assessment

Abstract

This study was qualitative content analysis that evaluated the quality of English for University Teaching (EUT), English for Specific Purposes (ESP), and English for Academic Purposes (EAP) textbooks through impressionistic and in-depth analysis. The participants were 9 EAP teachers and 15 EAP students from three universities in Surakarta, Indonesia. Instruments to collect data were checklist and interview guide. The checklist consisted of impressionistic analysis to see (1) cover, (2) content scope, (3) learning activities, (4) book organization, and (5) layout; and the in-depth analysis, namely (1) aim and approach, (2) organization design, language and content, (3) language skills, and (4) practical consideration. Data were collected using checklist and Focus Group Discussion. Data were analyzed through summative content analysis with seven stages: transforming data into narrative text, determining unit of analysis, developing categories, coding, coding the entire text, checking consistency of coding, and drawing conclusion. Results show EUT and ESP have low quality based on the impressionistic evaluation, and fair quality of their in-depth evaluation. Both books are register and grammar oriented. EAP textbook is good in general, and very good in depth. EAP presents its contents in terms of genre reading, academic speaking, academic reading, academic writing and grammar.

Assessing the quality of reference books on the English writing examination in Hong Kong: How can an evaluation checklist help?

“…McGrath (2002) identifies three basic methods of textbook evaluation in the literature, namely, the impressionistic method, the checklist method, and the in-depth method. Similar to the preliminary screening, an assessment based on impressions can be useful in that evaluators could consider some representative features of the materials and assess the quality of the textbooks (e.g., Solikhah, 2020). Given that the impressionistic method may be considered rather subjective, the checklist method is more desirable in that a set of criteria could be explicitly stated for more transparent and convenient evaluation.…”

Investigating English for Aviation: An in-Depth Analysis of Aviation English Textbooks

“…Numerous research studies on textbook analysis and evaluation have been published by utilizing a range of evaluation techniques (Widodo 2007;Ghorbani 2011;Zolfagharian and Khalilpour 2015;Seniwegiasari et al 2018). Prior research includes pre-use evaluation via checklists (Ghorbani 2011;Zolfagharian and Khalilpour 2015;Seniwegiasari et al 2018); in-depth analysis (Widodo 2007); and a combination of impressionistic and in-depth analysis (Fatima, Kazim Shah and Sultan 2015;Solikhah 2020), or the integration of all available methods (Lee 2003), while others have conducted in-use evaluation (Atiqah et al 2014). There are also research studies on postuse evaluations that employed questionnaire to determine teachers' perceptions of the textbook (Litz 2005;Karsudianto 2019); or checklists and interviews to determine teachers' and students' perceptions of the textbook (Sahragard and Rahimi 2018;Solikhah 2020).…”

“…Prior research includes pre-use evaluation via checklists (Ghorbani 2011;Zolfagharian and Khalilpour 2015;Seniwegiasari et al 2018); in-depth analysis (Widodo 2007); and a combination of impressionistic and in-depth analysis (Fatima, Kazim Shah and Sultan 2015;Solikhah 2020), or the integration of all available methods (Lee 2003), while others have conducted in-use evaluation (Atiqah et al 2014). There are also research studies on postuse evaluations that employed questionnaire to determine teachers' perceptions of the textbook (Litz 2005;Karsudianto 2019); or checklists and interviews to determine teachers' and students' perceptions of the textbook (Sahragard and Rahimi 2018;Solikhah 2020). Nearly all of those research studies undertaken have focused on general English language teaching textbooks used as references in primary school, secondary school, senior high school, and college academic writing course.…”

Content Analysis of Entrepreneurship Education in Primary and Secondary School Textbooks

“…METHODS This study used a descriptive approach to analyse the content (Nonaka et al, 2012). The content analysis includes examining entrepreneurship education concept, facts, ideas, figure, learning activities and images used in the Bhutanese PP, primary and secondary school textbooks (Zhang & Barbara, 2016;Solikhak, 2020).…”