Testing aptitude

Abstract

Meara (2005) developed the LLAMA tests as a free, language-neutral, user-friendly suite of aptitude tests incorporating four separate elements: vocabulary learning (LLAMA_B), phonetic (implicit) memory (LLAMA_D), sound-symbol correspondence (LLAMA_E) and grammatical inferencing (LLAMA_F) based on the standardised MLAT tests (Carroll & Sapon 1959). Recently, they have become increasingly popular in L2 acquisition research (Grañena & Long 2013b). However, Meara has expressed concern about the wide use of these tests without validity testing (cf. Grañena 2013a). To this end, we investigated several areas relating to the LLAMA tests, i.e. (1) the role of gender in LLAMA test performance; (2) language neutrality; (3) the role of age; (4) the role of formal education qualifications; (5) the effect of playing logic puzzles on LLAMA scores and (6) the effect of changing the test timings to scores. 229 participants from a range of language backgrounds, aged 10–75 with various education levels, typologically distinct L1s, and varying levels of multilingualism were tested. A subset of participants was also tested with varying timings for the tests. The results showed that the LLAMA tests are gender and language neutral. The younger learners (10–11s) performed significantly worse than the adults in the sound/symbol correspondence task (LLAMA_E). Formal education qualifications show a significant advantage in 3 of the LLAMA subcomponents (B, E, F) but not the implicit measure (LLAMA_D). Playing logic puzzles did not improve LLAMA test scores. The timings appear to be optimal apart from LLAMA_F, which could be shortened. We suggest that the LLAMA aptitude tests are not significantly affected by these factors although researchers using these tests should be aware of the possible impact of education level on some components of the tests.

Examining the LLAMA aptitude tests

“…If the language script of the L1 can influence the acquisition of the L2, then the question arises if the L1 script of the learner influences their aptitude scores. In Rogers et al (2016), we looked at this question but had a small sample size (n = 14) and grouped Arabic and Chinese native speakers together as a non-Roman script group. This was less than satisfactory due to the differences between Chinese as morphosyllabic (Tolchinsky, Levin, Aram & McBride-Chang, 2012, p. 1598 or logographic (Crystal, 1987, p. 200) script and Arabic, which as a consonant alphabetic script shares a common Semitic ancestor with Roman scripts (Sampson, 1985, p. 77).…”

“…For the fourth research question, we have combined these results with Rogers et al (2016) as the data were collected under similar conditions, with similar background questionnaires giving a total of 404 participants. This allows us to carry out a more powerful statistical analysis to consider the effects of various individual background variables on LLAMA test performance.…”

“…For this final research question, we combined the results from this study with Rogers et al (2016) to examine how much of the variance in LLAMA test scores can be accounted for by six individual background variables. These variables are the three we have examined so far -L1 script (language neutrality), L2 status and age plus three other variables -highest formal education qualification, gender and logic puzzles.…”

“…They contain four sub-components: vocabulary acquisition, sound recognition, sound-symbol correspondence and grammatical inferencing. Granena (2013) and Rogers et al (2016) provided initial results regarding factors which might influence LLAMA test scores. This paper develops this previous work by examining some of issues raised with a larger cohort and focuses on the following research questions.…”

“…We combined data from Rogers et al (2016) and this study giving 404 participants in total. Using a multiple regression analysis, we found that prior L2 instruction predicted more of the variance (6%) than any other factor.…”