alternative reading of iadb study

7/31/2019 Alternative Reading of IADB Study

1/4

An Alternative Reading of the IADB Study

I was surprised by the beginning of t he Economists article Error Message (based on the IADB study) thatsays the Peruvian Una Laptop por Nio project did not accomplish anything in particular . The IADB studyclearly stated that the project substantially increased use of computers both at school and at home ,positive effects were found in general cognitive skills and improved competence in operating laptops in

tasks related to core applications (like a word processor) and searching for information on the compu ter.

The IADB study also says that there was no significant impact on school enrollment or attendance, nor intest scores in Math and Language. Enrollment in target schools was already nearly 100% -- therefore nofurther increase could be expected. Also, the program focus was not on improving test scores in Math orLanguage and there were no applications directly aligned with those subjects.

As the person responsible for Una Laptop por Nio for almost five years I hope to shed light on what seemsto be, probably unintentional, misleading information.

Background

The IADB study, which I personally supported and helped to deploy, did not include important backgroundinformation regarding the Peruvian basic education environment:

A January 2007 census evaluation of 180,000 Peruvian basic education teachers showed 92% of them lacked basic Math reasoning skills and 62% of them did not read at 6 th grade level, 27% wereat levels of zero or below.

After 200 hours of remedial education run by local universities faculties of Education, 13% of theteachers were still at the same zero or below level.

The program designed by the government to improve the average quality of teachers at publicschools will require at least 10 years to complete.

The above described situation left us with the difficult choice of waiting 10 years to do something or begin inparallel. A 2007 study by McKinsey found that the worlds most improved school systems had in commontheir concern for teacher quality and getting the best people into teaching a finding that really reduced ouroptions. We also knew that ICT skills had been identified as key ingredients for success by manyorganizations (see, for example, http://www.p21.org/storage/documents/P21_Report.pdf ). We believedthat putting 21 st century tools into the hands of children would be a good way to begin working on theproblem while a better teacher workforce was being developed.

It was never the programs primary goal to improve Math and Language test scores. What we expected wasthat the childrens lives would be improved by giving them more options for their future -- something thestudy has proven because the children are more proficient in the use of 21 st century tools (Table 5 of thestudy shows a dramatic increase in both access and use of computers by the treatment group) and havebetter cognitive skills (Table 7). It remains to be seen if with appropriate adjustments in strategy thoseimproved cognitive skills will translate into better test scores. I dont agree with the Economist s biasedreading of the study.
http://www.economist.com/node/21552202/printhttp://www.economist.com/node/21552202/printhttp://www.economist.com/node/21552202/printhttp://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=36706954http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=36706954http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=36706954http://www.mckinseyonsociety.com/downloads/reports/Education/Worlds_School_Systems_Final.pdfhttp://www.mckinseyonsociety.com/downloads/reports/Education/Worlds_School_Systems_Final.pdfhttp://www.mckinseyonsociety.com/downloads/reports/Education/Worlds_School_Systems_Final.pdfhttp://www.p21.org/storage/documents/P21_Report.pdfhttp://www.p21.org/storage/documents/P21_Report.pdfhttp://www.p21.org/storage/documents/P21_Report.pdfhttp://www.p21.org/storage/documents/P21_Report.pdfhttp://www.mckinseyonsociety.com/downloads/reports/Education/Worlds_School_Systems_Final.pdfhttp://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=36706954http://www.economist.com/node/21552202/print


2/4

In spite of the fact the program did not aim directly to improve test scores we agreed with the IADBsproposal to study impact on test scores because if positive impacts were found we would have gained 10years. The fact that there was no impact on test scores should not be a surprise nor a reason to dismiss theprogram. It means that we need to adjust the strategy and reinforce teacher training as well as trying tocapitalize on the improved cognitive skills of children in order to pursue more ambitious objectives.

The IADB Study

I would like to comment on specific areas of the study and present alternative interpretations to most of what has been circulating recently. I dont think any of the material circulating is unbiased and t his is also mypersonal opinion.

The program did not affect the quality of instruction in class (p.3)

I cannot agree with the statement that activities done might have little effect on educational outcomes

(word processing, calculator, games, music...). Being able to work with a word processor and a calculatorshould be seen as an educational outcome in itself. The paragraph implies that Math and Language scores

are the only educational outcome to be expected.

The study goes on to mention that on the positive side, the results indicate some benefits on cognitiveskills . What needs to be done, and we tried to prepare the project to be able to do so, is to build on thoseimproved cognitive skills by integrating additional material as teachers quality improves. For example, we

were able to integrate external stakeholders into our effort: the National University of Engineering has anapplication development laboratory that is working on the design of educational games specifically aimed toimprove Math scores; a local firm (http://www.soft-one.org/ ) developed a reading comprehensionapplication that has proven effects and runs on the XOs, offering a possibility to improve childrens readingcomprehension scores from sixth to eleventh grade.

limited information about how to integrate the computers provided into regular pedagogical practices (p.6)

This is true but the problem arises when we identify regular practices with correct or desirable practices. Itis worth noting some teachers still consider hitting students with a stick a regular practice. In a schoolsystem with a teacher profile like the one described it is not reasonable to expect regular practices aresomething we want to reinforce. Choosing to work only with teachers with good practices is not an option asit leaves many teachers out of the picture it was not a decision to ignore them. Many good teachers havemade the extra effort, based on the 40 hours training, and achieved very good results, but the good teachersare a small minority. The main issue here is the lengthy and difficult process of teacher assimilation of technology. I remember a meeting of Peruvian education officers with Clotilde Fonseca in 1998 when shewas president of Omar Dengo Foundation in Costa Rica. Fonseca said that after 10 years of the Programa de

Informatica Educativa, Costa Rican teachers were able to talk about Piaget correctly. Not that they were ableto apply it properly to their classroom practice, but at least they were moving in the right direction.

lack of Internet access and the fact that the laptops did not run Windows (p.7)

We knew from the beginning that universal Internet access would be lacking. This led us to design what wecalled the portable Internet, a 2GB USB drive with educ ational pre-selected pages allowing children andteachers to experience the feeling of navigation and to include a reduced 30,000 entries version of Wikipedia in Spanish. That was what we called asynchronous connectivity meaning that the pages would
http://www.soft-one.org/http://www.soft-one.org/http://www.soft-one.org/http://www.soft-one.org/


3/4

be periodically updated based on teachers and student requests. Regarding Windows, with support fromOLPC and Microsoft we ran a test project with Windows based XOs and had to deal with the highrequirements in terms of Memory Storage and virus infection problems. The project was generallysuccessful, however, Microsoft decided to abandon it because they were not developing their platform forthe XO. We were convinced that the ICT skills would be easily transferable to other platforms in the futuresince upgrading to new versions of Windows can be as complex as switching from Sugar to Windows. Oneunintended advantage of asynchronous connectivity was the protection against access to inappropriatecontent that we couldnt expect from teachers who knew less than their students about ICT.

The central outcomes of the study are achievement and cognitive tests (p.10)

The authors cannot be more specific about what they found and the whole document focuses on the factthat there was no measurable impact in achievement as measured by Math and Language test scores AND that there were positive effects in cognitive skills as measured by Ravens Progressive Matrices test. Why so -called independent analysts decide to write about one and ignore the other may be a matter of study initself.

Motivation toward attendance and homework (p.11) The study says that its findings contradict what has been suggested regarding motivation (p.2). Since I wasthe one who suggested this, I must say that in a study we made, we measured intrinsic motivation towardsschool work, not towards attendance or homework. Therefore, the study does not prove that the suggestionwas wrong. The variables measured that showed improvement were: interest pleasure in school work,relation between effort and results obtained, perceived competence for school work, creative stress,perceived selection of what to do, and improved personal relations. The only variable that did not improvewas perceived importance of school work, which was very high from the beginning.

Computer Access, Use and Skills (pp. 13 -15)

The study describes very promising results in this section. For example, only 13% of laptops malfunctioned

at some point and half of them were successfully repaired. This is a demonstration of the XOs ruggednessand the success of our training program, which devoted 8 hours to technical maintenance by teachers. Alsopromising was that the theft ratio was only 0.3%. Its worth noting that we decided to give the computersbut not the property because in those places where parents were mistakenly told the computers were theirsto own, many of them immediately decided to try to sell them for cash.

One negative finding of the study is that the students used the computers mostly during school time. Giventhat there were no specific Math or Language activities we must assume that they were taking time awayfrom regular school work and using it for OLPC activities. The fact that the test results show no impact maybe interpreted as the teachers work in class has no impact on test scores -- something I am afraid is not farfrom the truth.

self - perceived school competence evidence of small negative effects (p.16)

I dont agree with the implication of these negative effects as a decrease in self-esteem. What may beinferred is that children realize that they are not well prepared for school, a finding that is good if itmotivates them to work harder to achieve what they want. What I found is that children in non-OLPCschools have extremely high perceived competence for school work, in spite of their dismal results,something the presence of the XO seemed to be correcting for good.


4/4

Academic Achievement and Cognitive Skills (pp. 16 -17)

I find this section very instructive. The study states that there is no pedagogical model linking software withparticular curriculum objectives. That is true because we were convinced that, in order for curriculum-related software to work, we needed well-educated, competent and well-trained teachers. Since we couldonly provide the training component there was no way to ensure this approach would work. The fact mostteachers did not find the training enough is a proof training only was not enough. Re-educating them waswell out of our reach.

It is rewarding to learn that the approach chosen did not replicate reported negative effects of homecomputer usage on grades. Another positive impact neglected by most readers.

The main positive effect described here is the access and use of computers translated into improvements of general cognitive skills. In my opinion, this means that the foundation is in place on which to build and thatthe task is to continue working to achieve the desired results. It is to be seen if the new governmentadministration will be willing to keep building or will decide to forget about it and begin again. It is promisingthe person in charge (Sandro Marcone) was the first director of Proyecto Huascarn, a predecesor of the

General Directorate of Educational Technologies.

One surprising positive result is the 4.6 to 6 months advantage in cognitive skills progression for thetreatment group, which means a 30 to 40% improvement in just 15 months. Why the analysts chose toignore this remains to be understood.

Discussion (pp. 19 -20)

I dont share the conclusion governments should consider alternative uses of public funds. It is known thatimproving teachers quality is a long-term effort (Korea took 40 years to improve its education system).Something needs to be done apart from waiting. Whether building on top of improved cognitive skills willresult in an improvement in test scores is something that might be worth pursuing. It seems that the current

administration has a strong focus on teacher training, which will surely have a positive effect.

The mention about improved IQ in countries as emphasized by some researchers is in line with a study byNina Hansen that suggests improved IQ test results in Ethiopian children who participated in an OLPCprogram.

In conclusion, the IADB study is valuable and will hopefully serve as a guide to strategists in getting the mostout of the investment made. In no way does it support t he Economistss Error message article or the manyinterested parties who are trying to use the study to push their own agendas. I will have to finish this as I didat the beginning of the project by quoting Miguel de Cervantes Don Quijote when he said "Let the dogsbark Sancho, it's a sign that we keep advancing".
http://nina.hansen.socialpsychology.org/http://nina.hansen.socialpsychology.org/http://www.economist.com/node/21552202/printhttp://www.economist.com/node/21552202/printhttp://www.economist.com/node/21552202/printhttp://www.economist.com/node/21552202/printhttp://nina.hansen.socialpsychology.org/

alternative reading of iadb study

Documents