![Page 1: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/1.jpg)
Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes
Jack BurstonHonorary Research FellowCyprus University of Technology
CALICO 2014
![Page 2: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/2.jpg)
MALL Learning Outcomes: 1994-2012
• Few statistically reliable measures of learning outcomes
• Over 40% of MALL publications unrelated to language learning applications
• Pervasive methodological shortcomings
![Page 3: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/3.jpg)
MALL Research Problems
• Sample size and treatment duration
Figure 1: MALL Project Duration
![Page 4: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/4.jpg)
MALL Research Problems
• Sample size and treatment duration
Figure 2: MALL Student Numbers
![Page 5: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/5.jpg)
MALL Research Problems
Minimal requirements for statistical analysis:
• 4 weeks duration (20 class days)
• 10 experimental subjects
= 109 MALL studies
![Page 6: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/6.jpg)
MALL Research Problems
• Lack of quantifiable data:
74 studies contain no quantitative learning outcome data
= 35 MALL studies with objective learning outcomes from implementations that lasted at least a month and involved ten or more experimental subjects
![Page 7: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/7.jpg)
MALL Research Problems
• Research design shortcomings– Student usage of mobile devices not tracked (3) – Inadequate description of the control group (3)– Uncontrolled variables (4)– Confounding variables (3) – Inadequate statistical analysis (3)
= 19 valid research studies
![Page 8: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/8.jpg)
VALID MALL Studies: General Characteristics
Figure 3: General Characteristics of MALL Studies
![Page 9: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/9.jpg)
VALID MALL Studies: Duration of Study
Figure 4: Duration of Study
![Page 10: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/10.jpg)
VALID MALL Studies: Group Size
Figure 5: Group Size
![Page 11: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/11.jpg)
VALID MALL Studies: Year of Study
Figure 6: Year of Study
![Page 12: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/12.jpg)
VALID MALL Studies: Research Findings
Vocabulary Acquisition: 11/19L2 English8/11 SMS/MMS on basic mobile phones3/11 PDAs and smartphones
![Page 13: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/13.jpg)
VALID MALL Studies: Research Findings
5/11 = Positive MALL advantage 3 SMS (Iran) 2 MMS (Turkey, Taiwan)
3/11 SMS Mixed results 2 Delayed post-tests contradict
initial results (China, Iran) 1 No significant difference (Iran)
![Page 14: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/14.jpg)
VALID MALL Studies: Research Findings
3/11 Non SMS/MMS projects2 PDA-based projects positive
results (Taiwan)1 Smartphone negative
results (Japan)
![Page 15: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/15.jpg)
VALID MALL Studies: Research Findings
Reading Competency: 4/19 All PDA-based All positive MALL results
2 L1 Spanish (Chile) 2 L2 English (Taiwan)
![Page 16: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/16.jpg)
VALID MALL Studies: Research Findings
Listening/Speaking Skills: 3/19 Mobile devices used as audio recording tools All positive MALL results
L2 English (Taiwan)L2 Indonesian (Australia)L2 English (Cyprus)
![Page 17: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/17.jpg)
VALID MALL Studies: Research Findings
Writing Skills: 1/19 Unspecified mobile device L2 English (Taiwan) Positive results
![Page 18: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/18.jpg)
VALID MALL Studies: Conclusion
• Only 35 MALL studies meet minimal conditions of duration and sample size
• Only 19 MALL studies provide statistically reliable outcome data
• 15 Unequivocally positive results (80%): reading, listening, speaking and writing
• 4 No significant difference: vocabulary
![Page 19: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/19.jpg)
MALL Studies: Fundamental Problems
• Inadequate research design• Technocentricity
![Page 20: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/20.jpg)
Thank you for your attention!
Any questions?
MALL Learning Outcomes: 1994-2012
![Page 21: Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022081504/56814652550346895db3673c/html5/thumbnails/21.jpg)
Burston, J. (2014). 20 years of MALL project implementation: A meta-analysis of learning outcomes. ReCALL Journal, 27(1), xxx-xxx. (available online)
Burston, J. (2014). The reality of MALL project implementations: Still on the fringes. CALICO Journal, 31(1), 43-65.
Burston, J. (2013). Mobile-Assisted Language Learning: A selected annotated bibliography of implementation studies 1994-2012. Language Learning & Technology, Special volume on Mobile-Assisted Language Learning, 17(3), 157–224.
MALL Learning Outcomes: Bibliography