Amy Mahan Research Fellowship Program to Assess the Impact of Public Access to ICT

Laying the Foundation

Interim Technical Report Submitted by Universitat Pompeu Fabra to the Inernational Development Research Centre

Prepared by Francisco J. Proenza in collaboration with Abiodun Jagun, Hernán Galperín, Erwin Alampay and Lorena Camats.

March 2010

Table of Contents

Acknowledgements................................................................................................. iiExecutive Summary............................................................................................... ivIntroduction........................................................................................1Research Problem, Capacity Development and Program Objectives........1

Context...............................................................................................................1The Research Problem........................................................................................2Research Capacity Development........................................................................2Program Objectives.............................................................................................2

The Challenge: Attracting Excellent Emerging Scholars with Suitable Topics..................................................................................................3

Suitability of Topic...............................................................................................3Eligibility.............................................................................................................3

Meeting the Challenge: a Service Oriented Approach.............................5Documentation...................................................................................................5Dissemination.....................................................................................................7

Website Monitoring..........................................................................................7Networks..........................................................................................................9Africa and the Middle East..............................................................................10Asia-Pacific.....................................................................................................11Latin America and the Caribbean...................................................................11

Queries.............................................................................................................12General Queries.............................................................................................12Topic Queries.................................................................................................12Getting the topic right....................................................................................14

The Selection Process.......................................................................................15Applications....................................................................................................15Selection........................................................................................................15

Project Outputs: The Fellowship Awards..............................................17Program Management and Administration..........................................20

Management and Implementation Team...........................................................20Administrative Support.....................................................................................20Problems Encountered......................................................................................21Implementation calendar..................................................................................22

Capacity Building and Impact.............................................................22Recommendations..............................................................................23

Cost of preparing detailed documentation should be taken into account.........23Develop formal mechanisms for sharing practical lessons of experience.........23Reaching research communities accustomed to working in languages other than English require special effort and resources.............................................24Web Monitoring is a valuable tool for identifying gaps in outreach...................24A Service Oriented Approach is Essential..........................................................24

Bibliography......................................................................................26Annex A: Relative Weighs Assigned to Selection Criteria and Implications for

Quality of Fellowship Awards................................................................28Annex B. Results of Applicant Survey...................................................................31Annex C. Results of Survey of Non-Applicants who Submited a Topic Query........35Annex D: Charts Preparedto Inform Regional Coordinators and Program Partners

Regarding visits to Amy Mahan Website...............................................38Annex E. Topic Queries: Unsuitable Topics Submitted..........................................41

i

Acknowledgements

The authors gratefully acknowledge the help received from the following persons.

SIRCA colleagues Ang Peng Hwa, Arul Indrasen Chib, Joanna Tan Keng Ling, Naowarat Narula, Sri Ranjini Mei Hua, and IDRC’s program officer, Chaitali Sinha, made detailed comments to the draft Submission Guidelines. All their suggestions were adopted. SIRCA made their own materials available, such as the Agreement used to manage research grants, which served as a basis to prepare our own documents. Overall, SIRCA staff has remained a frequent and reliable source of information and counsel.

Jorge Tamayo of Universidad Nacional de San Agustín de Arequipa carefully read draft documentation, detecting errors and suggesting changes in grant administration proposals that made them more flexible and attractive to research teams.

Sylvia Cadena, Program Officer of the Innovation Fund of the Asia-Pacific Network Information Centre (APNIC), reviewed an early draft of the application form. Her recommendations led to a complete overhaul of this form and the streamlining of submission procedures.

Raul Pertierra’s emphasis on the value of assisting prospective applicants during the dissemination phase, planted the seed for the Topic Query.

Chat Garcia Ramilo, head of APC’s Women’s Network, provided advice on suitable ways to incorporate gender analysis in ICT research, and, on the basis of her experience, provided practical advice regarding IDRC procedures.

External members of the Selection Panel, Roxana Barrantes of the Instituto de Estudios Peruanos, and the three members of the Global Impact Study working as a team - Chris Coward and Araba Sey of the University of Washington and François Bar of the University of Southern California - joined with the IDRC Program Officer, three UPF staff and the three regional partners to carry out the time consuming but critical task of reviewing 29 shortlisted applicants and selecting Fellowship awards.

U. of Washington’s Araba Sey and Michelle Fellows helped us maintain website information on the Global Impact Study’s in-depth study probes up to date, in an effort to prevent the duplication of research efforts.

Many friends and colleagues engaged in ICT and research networks – in APC, DIRSI and Telecentre.org among others, helped disseminate the call for proposal. Professor Stewart Marshall’s dissemination of the call among subscribers of the International Journal of Education and Development was timely and highly effective.

William Melody authorized the publication in our website of his informative and caring obituary of Amy Mahan. Estela Acosta y Lara translated this document into Spanish.

IDRC’s Raymon Hyma, Claude Briand and Frank Tulus have been very much part of the implementation team. Raymond organized the translation of program documents into French, conducted and prepared summaries of the applicant and non-applicant evaluations, and created valuable dissemination materials. Claude helped us plan and understand IDRC financial and administrative requirements. Frank was instrumental getting the program under way, participated in the write up of program documentation and in the Selection Panel. He has accompanied us throughout, with guidance and support on all major program decisions.

ii

Implementation has relied in this first phase on many UPF colleagues. Recognition is due to Rector Josep Joan Moreso, Research Vice Rector, Louise Mcnally,, Department of Political and Social Sciences Director, Clara Riba, and to our colleagues and friends David Sancho, Jacint Jordana, Miquel Oliver, Josep Jofre, Ana Sagardoy, Marc Boldu, Oscar Cortés, Jordi Viñas, Maria Soteras, Fina Lorente, and Bianca Cabañeros.

We give special thanks to Bruce Girard for granting us the privilege to name the program after his lifetime companion. It is our aspiration that our work, our research and our conduct will duly honor Amy Mahan’s memory and make Bruce and their daughter Danielle proud.

iii

Executive Summary

The Amy Mahan Research Fellowship Program to Assess the Impact of Public Access to Information and Communication Technology (ICT) is the research capacity building component of a larger IDRC managed project titled “Investigating the Impact of Public Access of ICT.” A second component of the IDRC initiative is known as the Global Impact Study to Assess the Impact of Public Access to ICT.

The overall objective of the Amy Mahan Research Fellowship Program is to deepen and strengthen the capacity of emerging scholars in developing countries to carry out rigorous research in the area of public access to ICT, while simultaneously increasing the availability of high-quality research in the subject area coming from the developing regions of the world. The Program is an eighteen-months project sponsored by IDRC administered by Universitat Pompeu Fabra (UPF), Barcelona, in partnership with renowned scholars from leading research institutions based in Africa and the Middle East, the Asia-Pacific region, and Latin America and the Caribbean.

This report covers the period September 2009 through February 2010, i.e. the first six months of the Amy Mahan Research Fellowship Program. No research activities took place in this period. Program activities focused instead on laying the foundation for achieving Program objectives.

Successful research requires excellence in scholarship. Excellent emergent scholars will on their own identify proposals that address the central topic of concern – the impact of public access to ICT - that is of interest to them and will therefore engage their effort with commitment and enthusiasm. Having a good group of emergent researchers and sensible study proposals in turn makes it easier to recruit mature top scholars to support and carry out a successful research program and help strengthen research capacities of the emergent scholars and their support institutions.

The challenge facing the Program during this period was to identify and encourage a suitable group of excellent emerging developing-country scholars to compete for up to twelve program Fellowships. The way that the program went about to meet this challenge is the subject of this report.

To meet the Program’s objectives strict requirements were adopted, both in respect of suitability of topic and of the research team, especially the Principal Investigator. The program is specifically geared to understanding the impact of venues such as, for example, telecenters, cybercafés and public libraries providing access to computers and the Internet. To qualify, Principal Investigators had to :

- Be an emerging scholar, presently pursuing a graduate research degree, or have received this degree at most 7 years previously;

- Be a permanent resident of a developing country in Asia-Pacific region, Latin America and the Caribbean or Africa and the Middle East;

- Have a of a commitment to remain at a developing country institution for the 12 month duration of the funding period;

- Give evidence of aa long-term commitment to a research career

- Have a formal affiliation with a developing country research institution that would be designated to receive and administer the funds.

iv

Principal Investigators acting alone were eligible, but award selection criteria favored research teams.

The pool of high quality applicants might have been larger if applications from mature well-established researchers or researchers working in developed countries had been accepted; but this would have limited the program’s capacity development impact. Nevertheless, the strict requirements adopted also made attracting high quality applicants to compete for a Fellowship a challenging task.

To meet the call for proposals challenge a service-oriented strategy was adopted that involved:

preparation and publication in the Web of detailed program documents;

an aggressive dissemination campaign supported by regional partners in each of the 3 regions and careful web monitoring, to identify and encourage applications from suitably qualified candidates; and

personal interaction with prospective applicants to give interested parties a chance to ask specific questions and expect a response with short turnaround, and the establishment of a special facility, the Topic Query, to help them define a suitable research topic.

A total of 66 applications were received: 24 from Africa and the Middle East, 22 from Asia-Pacific and 20 from Latin America and the Caribbean.

Selection proceeded in four stages, with only minor departures from the plan described in the documents that were made available online, in English, Spanish and French.

A first sifting through the applications was done by the Program Manager to make sure that eligibility requirements were met. In a second stage the Program Manager and the corresponding Regional partner prepared a shortlist with the top proposals from each region: 9 from Africa, 10 from each of the other two regions.

The following first-order selection criteria (Annex A) were used to score the applications and arrive at these regional shortlists.

1. Presentation and clarity of the proposal2. Potential impact of research findings3. Quality of research design and methodology4. Scholarship record of Principal investigator.5. Adequacy of Budget, Work Plan and Plan for the Application of Funds6. Gender sensitivity of research proposal7. Impact on developing country research capacity.

In the third stage a Selection Panel applied the same 7 criteria to appraise the 29 shortlisted applications. The fourth stage was the formal announcement of the awards, on 18 February 2010.

Award winners come from 12 countries and represent Asia (4), Francophone (1) and Anglophone Africa (2), the Middle East (1), Latin America (3) and the Caribbean (1). Gender was not a selection criteria, but gender balance was nearly achieved with 5 women and 7 men selected as Fellows.

Significant public access venues in the two largest countries in the World, China and India, will be studied. Investigations will cover rural and urban areas, government sponsored telecenters, cybercafés. and even one library program (Chile’s Biblioredes). Research frameworks to be used are varied. Evidence-

v

gathering methods include quantitative, qualitative and mixed methods. Most research partners will be academic institutions – 7 Universities and 3 research centers - but the program will also be working with 2 NGOs.

The following recommendations are offered.

R1. Future projects wishing to have a similar level of detailed service-oriented documentation should consider the additional 6-month effort that is required, and the cost of this effort should be taken into account.

R2. As was done in the case of this program, IDRC project managers should be put in contact with staff working on similar projects and interaction with these colleagues should be encouraged. R3. IDRC technical research documentation available online is extensive and valuable. It would also be useful if a series of documents were made available addressing the more practical aspects of managing projects.

R4. Reaching out to large research communities who often work in their own language may require more dedicated efforts than were possible through this program.

R5. Journal Editors are usually reticent to distribute calls for proposals. Given the significant impact that such distributions can make, IDRC should consider using its considerable powers of persuasion to convince editors of the importance and benefits to its own members of distributing these calls.

R6. The statistics available using tools like Google Analytics are critical for effective targeting of research communities. It is recommended for use by grant programs that have as an objective to reach out to a broad developing country audience.

R7. The title given to a competitive grant program should make it easy for prospective applicants to identify critical eligibility elements.

R8. The topic query was indispensable for this particular competitive grant. It may not be necessary for other programs where the topic of research is broader or less open to question. What is indispensable is the adoption of a service approach to competitive grants, to economize in effort of program staff as well as participants in the contest and to attract high quality applicants.

vi

Introduction

This report covers the period September 2009 through February 2010, i.e. the first six months of the Amy Mahan Research Fellowship Program to Assess the Impact of Public Access to ICT. No research activities took place in this period. Consequently, no research findings may be reported.

Program activities focused on laying the foundation for achieving Program objectives. The challenge facing the Program during this period was to identify and encourage a suitable group of excellent emerging developing-country scholars to compete for a Fellowship. The way that the program went about to meet this challenge is the subject of the report.

Research Problem, Capacity Development and Program Objectives

Context

The provision of public access to ICTs gained prominence shortly after the appearance of the first commercial Web browser, Netscape, in 1994, as modern ICTs – the Internet and the World Wide Web in particular - became more affordable and versatile. Johan Ernberg (1997, 1998), the conceptual father of telecenters as instruments of policy, was a driving force behind ITU’s pilot initiatives establishing Multipurpose Community Centers. These Centers would provide a broad range of ICT enabled services traditionally lacking in remote rural communities. Service aggregation would enhance impact and generate the revenue needed to assure sustainability. IDRC was another important early player sponsoring pilot initiatives, the formation of telecenter networks and action-oriented research (Gomez 1999).

Evidence of the impact of public access to ICT is readily available: Cybercafés are self-sustaining. Unless they are about to shut down, someone, usually a small entrepreneur, is earning from these centers a return on investment that assures their sustainability, and customers are deriving sufficient benefits to pay for these centers’ services.

This rather narrow view of impact focuses on benefits received by cybercafé users who for the most part live in urban areas and tend to be young, relatively well-off educated individuals familiar with ICTs (Proenza, Bastidas-Buch, Montero 2001; Sey and Fellows 2009). Cybercafés have also served as a model for many Governments that have provided some initial assistance in the expectation that rural public access venues would eventually become self-sustaining. In practice, most rural public access venues have faced and failed to overcome four formidable challenges that are common in rural environments: i. low density of population; ii. high connectivity costs; iii. limited computer literacy; and iv. high costs of equipment and service maintenance (Proenza 2008). The evidence of rural telecenter failure is substantial (e.g. Toyama et al 2005), even as it is overshadowed by abundant anecdotal evidence of impact and calls for new initiatives.

In general, rigorous research has been lacking in the field of ICT for development, ICT4D, (Heeks 2006). The difficulties associated with the attribution of impact add a layer of complexity to the dearth of rigor that has characterized ICT4D research.

1

The Research Problem

According to the Global Impact Study’s literature review, research on public access to ICTs has tended to focus on process outputs as opposed to long-term development outcomes (Sey and Fellows 2009). Reliable rigorous independent impact evaluations are scarce, in striking contrast with the enthusiasm that these initiatives continue to generate.

Also missing is a comprehensive systematic body of knowledge regarding what works and what does not work. There is an urgent need for independent rigorous evaluations that help us understand what are the key determinants of success and failure in public access to ICT initiatives, where and under what circumstances should governments and donors invest in providing access to ICTs, and what policy instruments they should use or avoid.

To some extent, the difficulties experienced by past studies in assessing and attributing impact to public access venues, stem from the limited time that these facilities have been around. By now, ten years after their emergence this limitation is becoming outdated. Furthermore, there is little in the impact assessment literature to suggest there might be idiosyncrasies in the ICT4D domain that would invalidate the adaptation of impact assessment frameworks from other disciplines to ICT4D research (Souter 2007).

Research Capacity Development

Addressing the need for more and better ICT4D impact assessment studies implies a complimentary need to ensure that sustainable production of knowledge and research in this domain will continue to advance in the future. In the context of public access to ICT and impact assessment, a sufficient research capacity needs to be developed and maintained. Sustaining research capacity is a valuable enterprise for a country and its people.

Building and strengthening the supply side of the research capacity community (those who provide/ facilitate the scientific and methodological learning opportunities); is just as important as, on the demand side, enabling those who have acquired capacity to continue to use it in the widest possible way (Bernard 2005a,b). Given the level of investments and proliferation of venues providing public access to ICT in the developing world, it is important to nurture and support the development and strengthening of the local research capacity for assessing the impact public access to ICT.

Program Objectives

The overall objective of the Program is to deepen and strengthen the capacity of emerging scholars in developing countries to carry out rigorous research in the area of public access to ICT, while simultaneously increasing the availability of high-quality research in the subject area coming from the developing regions of the world.

The specific objectives of the Program are:

1.to manage a competitive research grant program that sponsors scientifically rigorous research that results in practical recommendations to help donors, governments and other stakeholders (NGOs, firms) enhance the sustainability and impact of Public Access to ICTs.

2.to encourage emerging scholars from developing countries to address priority research questions identified in the field of public access to ICTs.

2

3.to provide guidance and mentorship to improve the quality of research on public access to ICT conducted by emerging scholars in the developing countries

4.to increase the availability of high-quality research in the area of public access to ICT coming from developing country scholars.

5.to sponsor the development of and help strengthen international networks of researchers investigating issues related to public access to ICT.

The Challenge: Attracting Excellent Emerging Scholars with Suitable Topics

To meet the Program’s objectives strict requirements were adopted, both in respect of suitability of topic and of the research team, especially the Principal Investigator.1

Suitability of Topic

A topic of research is considered suitable for Fellowship support if it is likely to contribute to our understanding of the impact of public access to ICTs; and is unlikely to duplicate efforts of the Global impact study’s in-depth study probes.

The program is specifically geared to understanding the impact of venues such as, for example, telecenters, cybercafés and public libraries providing access to computers and the Internet.

Only the impact of facilities providing shared public access to the Internet (and of programs that give support to such facilities) qualifies as a suitable subject of study under this program. Consideration may be given to ways of providing access to ICTs other than through computers - e.g. mobile phones, WiFi, home computer use - but only to the extent that these other technologies are compared to shared computer access to the Internet.

Eligibility

To qualify, Principal Investigators had to satisfy the following prerequisites:

- Be an emerging scholar, i.e. be presently pursuing a graduate research degree, or be a professional that received his or her doctorate degree at most 7 years prior to the submission;

- Be a permanent resident of a developing country in Asia-Pacific region, Latin America and the Caribbean or Africa and the Middle East;

- Give evidence (long term contract, tenured position, etc) of a commitment to remain at the same institution for the duration of the funding period;

- Give evidence of current involvement in research activities, a long-term commitment to a research career, be currently involved in research training or have completed research training.

1 Eligibility requirements are presented here in summary form. Details may be found in “Benefits, Eligibility and Procedures” (www.upf.edu/amymahan/benefits)

3

- Have a formal affiliation with an institution or with a consortium or institutions based in Asia or in Latin America and the Caribbean or in Africa and the Middle East. These may be academic, public sector, private sector or civil society institutions. The Director of Research of the corresponding institution must sign the application in its behalf; endorsing the application and confirming the institution’s support for the Principal investigator and the proposed research project.

Principal Investigators acting alone were eligible, but award selection criteria favored research teams.

Mature scholars or researchers from developed countries could be part of a research team, but not as Principal Investigators and provided that the total research budget used to fund their participation was 5% or less.

The pool of high quality applicants might have been bigger if applications from mature well-established researchers or researchers working in developed countries had been accepted; but this would have limited the program’s capacity development impact.

These strict eligibility requirements made attracting high quality applicants to compete for a Fellowship a challenging undertaking. The difficulties varied from one region to another. Our regional coordinator, Abiodun Jagun, aptly described the situation in Africa as follows:

“In Africa, the target group of the fellowship, what is referred to as "emerging scholars" are not as numerous or perhaps may not be as distinct as in other regions. ...people come to study at graduate level (Masters and PhD) and then go into the private sector afterwards. It has been difficult to retain academic talent within universities. Furthermore, people that are currently studying for their PhDs work predominantly on established projects (using the project's data set for their own study) rather than on their own independent research (I am told this is for funding reasons).

This means that the range of research amongst the people I have come into contact with has been quite narrow (e.g. technical/solution driven studies on access) and it takes some time in showing them how their topic can be realigned to the topic area of the fellowship.

Another challenge is that once potential candidates have been identified...it is difficult to get them to "bite"! They tend to compare the grant to what they would earn within the same period working on a funded research project; most of them know that getting support from an institution means that a percentage of the grant money will go to the institution - and I have had people tell me that 22,000 Euros is not enough!”

Meeting the Challenge: a Service Oriented Approach

To meet the call for proposals challenge a service-oriented strategy was adopted that involved:

preparation and publication in the Web of documents describing program benefits and requirements in detail;

an aggressive dissemination campaign targeted at research and ICT networks, supported by regional partners in each of the 3 regions and by careful web monitoring, to identify and encourage applications from qualified candidates; and

4

personal interaction with prospective applicants to give interested parties a chance to ask specific questions and expect a response with short turnaround, and the establishment of a special facility, the Topic Query, to help them define a suitable research topic.

Documentation

“The common line is ‘go to our website and look at our guidelines.’ But 90 percent of the guidelines are so ambiguous that we can’t really make sense out of them.”

(Bearman 2008, page 11).

The materials prepared and published in the Web are listed in Table 1. Translation of documents into Spanish was done by UPF. Translation into French was gracefully assumed by IDRC. Three documents were not translated mainly for lack of time. These were considered important but not indispensable.

Table 1 shows the number of pages of the documents prepared, but this should be seen as a rough indicator since for clarity’s sake there is considerable overlap between documents. The Frequently Asked Questions, for example, are mostly drawn from other documents.

How useful were these materials?

To find out whether the documents prepared were useful, UPF proposed that an evaluation of applicants’ perceptions to be carried out by an entity or professional not directly involved in Program execution.

IDRC administered the survey asking applicants to respond anonymously to questions about the usefulness of program documentation. The survey was conducted from December 31, the applications deadline, to 28 February 2009, before Fellowship awards were announced, to prevent award selection results from influencing applicants’ interest in participating in the survey or coloring responses.

The survey (see details Annex B) was completed by 53 (80%) of the 66 applicants. Most respondents (93%) had as their mother tongue a language other than English. Applicants came from a wide range of cultures, and the majority marked “other” (37.8%) as their mother tongue. The single most frequently listed mother tongue was Spanish (31%), followed by Hindi (8.9%) and Chinese (6.7%). English is however the dominant language used (72%) to communicate and publish research work.

5

Table 1. Documentation prepared and published in the Amy Mahan Fellowship Program Website

Title of Document or Web Page English version Availability in Spanish or FrenchURL Pages

Home page www.upf.edu/amymahan/ 1 Spanish, FrenchAmy Mahan – In Memoriam www.upf.edu/amymahan/in-memoriam/ 1 Spanish, FrenchAmy Mahan – Who was Amy www.upf.edu/amymahan/amy-obituary/ 2 Spanish, FrenchBenefits, Eligibility and Procedures www.upf.edu/amymahan/benefits 9 Spanish, FrenchSubmission Guidelines www.upf.edu/amymahan/guidelines 12 Spanish, FrenchAward Criteria and Selection Process www.upf.edu/amymahan/criteria 13 -Sample Research Topics and Overview of Global Impact Study Probes

www.upf.edu/amymahan/topics 21 -

Draft Ethical Standards that will Guide the Conduct of the Research www.upf.edu/amymahan/ethicalstandards 4 -

Frequently Asked Questions www.upf.edu/amymahan/faqs 24 Spanish, FrenchTopic Query www.upf.edu/amymahan/topic-query N/A Spanish, French

Form www.upf.edu/amymahan/topic-query/topic-query.rtf N/A Spanish, French

Instructions www.upf.edu/amymahan/topic-query/topic-query_instructions.html 5 Spanish, French

Application Form www.upf.edu/amymahan/application/ N/A Spanish, French

Form www.upf.edu/amymahan/application/application.rtf N/A Spanish, French

Instructions www.upf.edu/amymahan/application/application_instructions.html 17 Spanish, French

Budget Template www.upf.edu/amymahan/application/budgettemplate.xls 1 Spanish, French

6

Forty one percent of respondents regarded the documents available at the Amy Mahan website to be “Excellent” in terms of clarity and usefulness. Another 49% considered them “Very good”, and the remaining 5% found them “Satisfactory”. No respondent judged the documentation “Poor” or “Very poor”.

Dissemination

The program’s dissemination strategy had two interrelated elements. First, careful monitoring of website visitors to identify gaps reaching specific countries and groups (e.g. female researchers) and helped direct dissemination efforts to filling these gaps. Second, the Program Manager, regional coordinators and UPF and IDRC colleagues disseminated the call for proposals through personal contacts and distribution lists targeting researchers and staff working in ICT for development programs.

Website Monitoring

We sought to reach emergent scholars far and wide, from all fields (not just ICTD), male and female and from small and large countries alike. We also knew that dissemination efforts were bound to be more effective in countries like Peru, Chile, China, Colombia, India, Uganda, Indonesia, Sri Lanka and Brazil, that have had a significant number of public access venues for a while. These experiences deserve to be studied, and in these countries scholarly interest in the subject matter is alive and there usually is already a body of knowledge on which to build.

Data on unique visitors disaggregated by country became available late in October using Google Analytics. This made it possible to identify specific countries where public access is significant but where our call for proposal did not seem to be reaching scholars, and better target our dissemination efforts. Weekly status reports were issued to keep regional partners and UPF and IDRC colleagues up to date on progress or lack thereof (see sample Charts in Annex D).

Special outreach efforts such as searching for contacts in academic websites and writing emails to scholarly communities and to personal contacts targeted countries where the data was telling us there were serious gaps. For example around 5 November we noticed only a few website visitors were from Chile. We mounted a campaign to inform Chilean scholars of our call for proposals and these efforts immediately paid off. By the following week (ending 13 November) the number of unique Chilean visitors to the site had risen fivefold from 15 to 87 (Table 3). Similar efforts helped us increase fourfold the number of unique visitors from Sri Lanka (10-44) and from China (11 to 46).

By the application deadline on 31 December, the Amy Mahan website had been visited from a total of 87 developing countries (Table 2 and Chart D3 in Annex D). Forty two percent of were from Spanish speaking Latin America, 23% from Asia-Pacific countries, and 16% from Anglophone Africa. Unique visitors from four countries - Peru, India, Chile and the Philippines - account for 35% of the total.

In the end, some outreach gaps remained. We had few visitors from Francophone Africa (90), the Middle East (52), or Brazil (65) and other Portuguese speaking countries (9).

7

Table 2. Regional Distribution of Unique Visitors to Amy Mahan Website 20 October – 31 December 2009

Region Unique Visitors Countries# % # %

Africa and the Middle EastKenya 74 4%

5% 6%Uganda 52 3%Nigeria 48 2%Ghana 46 2%South Africa 33 2%Other Anglophone Africa 70 3% 17 20%Francophone Africa 90 4% 14 16%Luso Africa 9 0.4% 2 2%Middle East and Arabic Africa 50 2% 8 9%

Total Africa - Middle East 613 30% 21 24%

Asia-PacificIndia 200 10%

8% 9%

Philippines 127 6%China 46 2%Sri Lanka 44 2%Malaysia 40 2%Pakistan 38 2%Bangladesh 31 2%Indonesia 26 1%Other Asia 47 2% 9 10%Pacific 14 1% 4 5%Total Asia - Pacific 472 23% 41 47%

Latin America and the CaribbeanPeru 235 12%

6 7%

Chile 142 7%Mexico 97 5%Colombia 94 5%Argentina 88 4%Brazil 59 3%Other South America 109 5% 5 6%Caribbean - Anglophone 34 2% 6 7%Central America & Spanish Sp. Caribbean 72 4% 8 9%Total Latin America and Caribbean 930 46% 25 29%

TOTAL 3 REGIONS 2015 100% 87 100%

8

Table 3. Evolution of Unique Visitors - Top 20 Countries with Highest Number20 Oct to 5 Nov 20 Oct to 13 Nov 20 Oct to 31 Dec

1 India 88 1 India 116 1 Peru 2352 Philippines 35 2 Chile 87 2 India 2003 Argentina 33 3 Peru 85 3 Chile 1424 Mexico 33 4 Philippines 66 4 Philippines 1275 Peru 30 5 Argentina 50 5 Mexico 976 Kenya 29 6 Mexico 45 6 Colombia 947 Colombia 28 7 Colombia 44 7 Argentina 888 Nigeria 26 8 Kenya 40 8 Kenya 749 South Africa 21 9 Nigeria 33 9 Brazil 59

10 Brazil 21 10 Brazil 29 10 Uganda 5211 Malaysia 20 11 Ghana 26 11 Nigeria 4612 Pakistan 19 12 Uganda 26 12 China 4613 Ghana 17 13 Malaysia 23 13 Ghana 4614 Chile 15 14 South Africa 23 14 Sri Lanka 4415 Uganda 15 15 Pakistan 20 15 Ecuador 4116 Ecuador 12 16 China 17 16 Malaysia 4017 China 11 17 Venezuela 16 17 Pakistan 3518 Senegal 10 18 Sri Lanka 15 18 South Africa 3319 Sri Lanka 10 19 Ecuador 14 19 Bangladesh 3120 Venezuela 10 20 Indonesia 13 20 Uruguay 28

Networks

Fifty one percent of respondents to the applicants’ survey first heard about the program either by word or mouth or by its electronic equivalent, email. Other ways applicants found out about the program were distribution lists (22.6%) and website postings (17%).

Networks of researchers and of groups interested in ICT played a key role helping spread the call for proposal. Visits to the program’s website received its greatest lift on October 22 when Professor Stewart Marshall, editor of the International Journal of Education and Development using ICT, was contacted by our regional coordinator for Asia-Pacific and Dr. Marshall kindly sent an email to journal subscribers with our program announcement (Chart 1).

9

An article in telecentre.org newsletter on 21 November announcing the program was helpful, mainly to remind the telecenter community of the call for proposals and the application deadlines. It did not lead to a significant rise in unique visitor count (Chart 1), probably because most members of this community were by then aware of the program and had already visited the site.2

Overall, networks played a major role in getting high quality proposals. It is no coincidence that 3 award winning applicants are part of the APC network; and that another one is a former grantee of IDRC’s Strengthening ICTD Research Capacity in Asia (SIRCA) project.

Africa and the Middle East

Disseminating information about the fellowship in Africa and the Middle East presented a number of challenges. The region covers a broad and disparate range of countries spread across three significant continents or sub-continents: the Middle East, North Africa, and Sub-Saharan Africa (although even these sub-regions are not homogenous). Not only are there differences in culture and language, there are also few avenues (or “spaces”) where researchers in this region aggregate and/or interact. In addition, resource limitations resulted in a dissemination strategy that placed emphasis on targeting networks, institutions and some key individuals in these regions. The Call for Applications (and several reminders) was sent to mailing and discussion lists, contacts in relevant institutions (mainly universities), and individuals that are active in research in the area of the Call and/or in the region.

The regional coordinator, Dr. Abiodun Jagun, attended two meetings that had participants that would find the fellowship of relevance. These were IDRC’s Acacia Research and Learning Forum held in October 2009 in Dakar, Senegal and UbuntuNet Alliance’s UbuntuNet-Connect 2009 (held in association with the Research and Education Network of Uganda), which took place in November 2009 in Kampala, Uganda. Information leaflets were produced and distributed at these meetings and face-to-face conversations were had with experienced researchers that could recommend the fellowship to emerging scholars, and also with potential applicants.

Some challenges encountered in administering the Call in this region were described earlier in this document (page 4). With some prominent exceptions, research capacities in Sub-Saharan African institutions can be classified as weak, and researchers are poorly paid. In terms of the context of the research fellowship, the opportunities for assessing the “impact” of public access venues are limited. This can in part be attributed to high connectivity costs, which influence the spread and viability of public access venues. Cybercafés abound in some urban areas in Africa (e.g. in Uganda and South Africa), but the experience of government sponsored rural telecenter initiatives is limited. There are some programs in the planning stage or of recent vintage, but these are not many and their implementation is uncertain.

Public access venues are more common in the Middle East, mainly cybercafés, but as one of our award-winning applicants, GhalebRabab’ah, noted in his team’s proposal, research interest in public access venues has been limited.

Our ability to reach the region was further constrained by the fact that program documentation became available in French only after mid October and that limited resources did not permit translation into Arabic. In the end, whilst the region had the most number of applications submitted; it also had the widest dispersion/variance in terms of quality of applications and relevance of topics. Of 2 Once a person visits the site for the first time, subsequent visits are not registered in the unique visitor count.

10

the selected grantees, four are from region and from the following countries: Cameroon, Jordan, Rwanda, and Uganda.

Asia-Pacific

Prof. Erwin Alampay, the regional program coordinator, is based in Manila. Initial distribution of the call was sent to regional organizations highly invested in ICTD. Among them were SIRCA and NUS in Singapore, LirneAsia (based in Sri Lanka), UNDP, UNESCAP, IT for Change (India), ONI-Asia, etc. and mailing lists from the IDRC sponsored Living the Information Society conference in 2007.

Many of countries in the region do not use English, Spanish, or French, but countries with a strong English educational foundation were at an advantage; i.e. India, Philippines, Pakistan, Malaysia. There were also many young scholars who were interested in participating but were ineligible because of their current status as researchers or students in developed countries, e.g. in Singapore, Australia, HK, Japan, New Zealand.

Notwithstanding these constraints, the region produced the highest number of quality proposals and some countries had multiple candidates (i.e. India and Malaysia). In the end, four proposals from four different countries were awarded fellowships: Thailand, Malaysia, India and China.

Latin America and the Caribbean

The challenge in Latin America and the Caribbean region was to ensure wide diffusion of the Call for Proposals across a region with significant geographic dispersion and language differences.

Working in close collaboration with Prof. Hernán Galperín, regional program coordinator, a two-pronged dissemination strategy was followed. First, to ensure wide distribution, the call was sent to email lists administered by regional organizations with vast experience in the field of ICT4D, among them APC, DIRSI, and telecentros.org. Second, key individuals were identified and contacted as they constitute important nodes in ICT4D research networks; in particular senior researchers mentoring emerging scholars in the field.

The naming of the program after Amy Mahan, a long time resident of Latin America, helped bring attention to the program among many of the senior scholars who knew and worked with Amy in the past.

Key documents such as the Call for Papers and FAQ were translated into Portuguese to attract scholars from Brazil.

During the consultation phase, the Program Manager and the Regional Coordinator responded to several queries (both through email and phone) about topics and eligibility by potential applicants. These efforts resulted in 16 strong applications from emerging scholars in the region. Five of these were initially chosen by the Selection Panel, and had the following regional dispersion: Argentina (1), Perú (1), Chile (2) and Jamaica (1). Ultimately, one of the proposals from Chile was discarded due to issues concerning the PI's eligibility and the capacity development impact of the proposal, resulting in four successful applicants, three from Latin America and one from the Caribbean.

11

Queries

“Just as foundations don’t want to receive proposals that don’t fit their mission, nonprofits don’t want to spend time preparing proposals that aren’t going anywhere.”

“All we want is a five minute conversation – a chance to vet ideas and get direction”

(Bearman 2008, page 11)

General Queries

The following email address AmyMahanFellowship@upf.edu was made available to any researcher with questions about the program. Innumerable questions were answered this way with fairly swift (1-3 days) turnaround. When the material was already available in program documentation, the interested party was usually directed to the specific document page or section and URL.

Topic Queries

Mindful that the program’s research topic was narrowly defined, a formal consultation procedure, the Topic Query (TQ), was set up to help prospective Principal Investigators choose a suitable topic. The deadline for submitting a TQ was 30 November 2009, one month before the applications’ deadline on 31 December. Topic Query participants were asked to formally declare their intent to apply for a Fellowship.

Principal Investigators who submitted a Topic Query received assistance in the form of a review of their choice of topic by the Program Implementation Team. Within five working days – and frequently within two or three days - after submitting a TQ, participants received a confidential email from the Program Manager or the corresponding regional coordinator indicating:

i. Indicating whether the chosen topic was suitable or not for program funding, and if not why this was the case.

ii. Suggesting adjustments in the chosen topic to strengthen the research issues addressed by the study.

iii. Informing them whether other known candidate applicants were addressing a similar topic.

TQ responses focused on the suitability of research topic. In some instances the eligibility of a proponent of a topic was doubtful and he or she was alerted. Technical issues were not addressed because these were considered to be the applicant’s responsibility. Only two applicants were found addressing similar topics and they were duly informed.

By 30 November, twenty-three candidates had presented satisfactory Topic Queries with a "suitable research topic". Some of these had become satisfactory only after first submitting an unsatisfactory topic subsequently corrected in a second TQ after receiving guidance from the program management team. Nineteen different countries were represented in the 23 satisfactory TQs with no country contributing more than two applicants.

An additional 17 topic queries received addressed unsuitable topics. Some of these were presenting very good proposals in a topic not supported by the

12

program and some subsequently adjusted their topic and ended up presenting good high quality applications.

The effectiveness of the Topic Query was subjected to scrutiny in the applicants’ survey (Annex B, Question 4). All 30 respondents who submitted topic queries found the responses they received useful in the sense of helping their team improve their proposal and their application. No respondent chose either of the two other options given: “a waste of time” or “confusing”.

Seven applicants who submitted TQs won Fellowships.3

Out of 66 applicants only 23 (36%) submitted Topic Queries. Why so few?

Question 5 of the applicants’ survey asked respondents who did not submit a TQ, why they did not do so. Most (58%) indicated they had learned of the procedure late and did not have time to submit a TQ before the deadline. A few (4 or 17%) felt there was little they could gain from doing so, and a few others (3 or 12%) did not really understand what the TQ procedure was about or how it could help them.

Most aspiring PIs who submitted Topic Queries (56%) went on to apply for a Fellowship (Table 4). A large majority (83%) of those who submitted suitable topic’s in their TQs applied for a Fellowship. A similarly large majority (78%) of participants who presented an unsuitable topic in their TQ did not apply.

Table 4. Topic Queries Submitted

StatusOK TQ TQ Not OK All TQs

# % # % # %

Submitted TQ and applied 19 83 4 24 23 58

Submitted TQ but did not apply 4 17 13 76 17 43

Number of TQs: 23 100 17 100 40 100

Many reasons could account for the correlation between a satisfactory TQ and applying for a Fellowship and its converse, not applying after submitting an unsatisfactory TQ.

To investigate non-applicant motivations, a survey was administered by IDRC among the 17 TQ participants who did not apply. The results, presented in Annex C, should be regarded with caution since there were only 7 respondents.

Two respondents found website documentation excellent, three judged it “very good” and one deemed it “satisfactory”. As in the case of applicants, no responded marked any of the two other choices, “poor” or “very poor”.

For 3 of the 7 survey respondents, the reason for not applying had little to do with the program’s answer to their TQs. One respondent found out through the topic query that he or she was ineligible. Two respondents, however, indicated that they found the response to their TQ confusing and that they did not know how to proceed.

3 The 7 award-winning applicants who previously submitted Topic Queries are: Jean Damascène (RW), Sylvie Siyam (CM), Nor Aziah Alias (MY), Wei Shang (CN) and Simon Yalams (JA), Balwant Singh Mehta (IN) and Ghaleb Rabab’ah (JO).

13

Getting the topic right

All applicants who took advantage of the Topic Query presented applications addressing suitable topics. One of the award winners, won after having submitted an unsatisfactory topic query and subsequently adjusting the research topic in his application according to the advice he received. Another award winner had first presented two unsatisfactory TQs but was able to reorient his topic in his application, again thanks to the response to his second TQ. If the topic query facility had not been in place, the program would have missed the chance to work with and support the work of these two excellent scholars.

The amount of effort misspent by researchers presenting proposals with unsuitable topic either in their TQ or their application was substantial. Out of 42 Topic Queries received 19 (45%) addressed unsuitable topics4; and out of 66 applications, 17 were discarded because they addressed unsuitable topics. None of these 17 applicants submitted a TQ, so in effect 40% of applicants who did not submit a TQ ended up proposing an unsuitable topic.

Could we have made it easier for applicants? Could we have reduced the submission of topics that fell outside the scope of the fellowship? In principle having a longer period for submitting topic queries would have helped. Unfortunately the application deadline (31 December) fell close to a holiday period and most PIs, as expected, submitted their TQ on or close to the deadline on 30 November. To set the TQ deadline at a later date closer to the application deadline would have generated an unsustainable work burden that could have not been adequately handled during this period.

Annex E lists the topics found to be unsuitable during the Topic Query process. A couple of topics proposed addressed the impact of mobiles, which were clearly specified as being out of line in program documentation. By and large, however, the most common error made in applications and TQs with unsuitable topics was to assume that the program had a broader scope than it did. Most of those who submitted unsuitable topics were addressing the impact of ICT in a broader sense, rather than the assessment of the impact of venues that give public access to computers and the Internet.

The program title may have misled prospective applicants.

Government programs often have the objective and expectation that a broad range of ICTs (fax and photocopy machines, photocopy phones, medical diagnostic equipment, etc.) are installed and used in the public access venues they sponsor. Cybercafé owners do not make the same mistake, aware that the dominant ICTs that users demand and use are the computer and the Internet. This is why whenever the need to describe what occurs in practice arises, the focus of the Global Impact Study becomes narrower than its title might suggest. For example, the first sentence of Sey’s 2009 review of research on public access to ICT states: “Information and communication technologies (particularly computers and the Internet) are widely acknowledged as important resources for socioeconomic advancement in both developed and developing countries.”

This program was named the “Amy Mahan Research Fellowship Program to Assess the Impact of Public Access to ICT”, in deference to the larger component of the IDRC initiative, which started two years earlier and is titled the “Global Impact Study of Public Access to Information & Communication Technologies”.

4 These include 2 unsuitable Topic Queries that were followed by corrected TQs. They do not match numbers in Table 1 because the latter does not count unsuitable TQs that were corrected in a subsequent satisfactory TQ.

14

Since managing a competitive grant was a specific distinct objective of this program, using the same wording as the larger initiative, with its reference to ICT in broad terms, was probably an error. A more suitable title would have sought to make it easier for applicants to realize right away what topics were eligible. To be specific, the following would have been a more effective program title:

“Amy Mahan Fellowship Program to Assess the Impact of Venues Providing Public Access to Computers and the Internet.”

The Selection Process

The program put in place “a selection and review process that was professionally credible, technically competent, contextually relevant and transparent - through the use of recognised international experts” (Bernard 2006, page 5).

Applications

A total of 66 applications were received: 24 from AfME, 22 from AP and 20 from LAC. Unfortunately 16 addressed unsuitable topics and 2 were so poor in quality that they were immediately discarded. Accordingly, the total number of viable applications was 48.

The purchase of software to sort and process the applications was considered, but eventually manual handling was used. Manual procedures worked well and the number of applications received, 66 in total, was manageable.

Selection

The selection process proceeded in four stages, with only minor departures from the plan described in detail in Award Criteria and Selection Process (www.upf.edu/amymahan/criteria) and in summary form in Benefits, Eligibility and Procedures (www.upf.edu/amymahan/benefits) and in the Frequently Asked Questions (www.upf.edu/amymayan/faqs). These three documents were made available online, the first one in English and the other two in English, Spanish and French.

A first sifting through the applications was done by the Program Manager to make sure that eligibility requirements were met. No applications were discarded at this stage, but cases with possible problems and missing information were identified.

In a second stage the Program Manager and the corresponding Regional partner reviewed every application received from each region and prepared a shortlist with the top proposals from each region: 9 from Africa, 10 from each of the other two regions. The following first-order selection criteria (Annex A) were used to score the applications and arrive at these regional shortlists.

1. Presentation and clarity of the proposal2. Potential impact of research findings3. Quality of research design and methodology4. Scholarship record of Principal investigator.5. Adequacy of Budget, Work Plan and Plan for the Application of Funds6. Gender sensitivity of research proposal7. Impact on developing country research capacity.

15

In the third stage a Selection Panel applied the same 7 criteria to appraise the 29 shortlisted applications. The members of the Selection Panel were:

Frank Tulus Senior Program Officer, IDRCGlobal Impact Study associates working and as a group (1 vote):

Christopher Coward Principal Research Scientist and Director, Technology Research Group, U. of Washington

Araba Sey Research Associate, Technology Research Group, University of Washington

François Bar Annenberg School of Communications, University of Southern California

Roxana Barrantes Economist, Researcher, Instituto de EstudiosPeruanos

Abiodun Jagun Regional Coordinator for Africa and the Middle East, research associate at South Africa’s LINK Centre at the University of Witwatersrand

Erwin Alampay Regional Coordinator for Asia-Pacific region, Professor, at University of the Philippines, Manila

Hernán Galperín Regional Coordinator for Latin America and the Caribbean, Professor at Universidad de San Andrés, Argentina

David Sancho Assistant Dean, Public Administration and Management Studies, UPF

Miquel Oliver Vice-Chancellor for Quality and Institutional Strategy, UPF

Francisco J. Proenza Program Manager and Visiting Professor, UPF

Taking into account the international and inter-institutional character of the Selection Panel and the location of the 9 panel members in 7 countries and 5 different continents, their work of choosing the finalists was carried out using digital tools. Dropbox (www.dropbox.com) was used to transmit the applications submitted by the 29 finalists. From 5 January through 8 February Panel Members made a systematic detailed appraisal of the 29 short list applicants, applying the 7 criteria established by Program documents (Award Criteria and Selection Process) and further refined in instructions distributed just before the selection process to all Panel members in a document entitled “Relative Weighs Assigned to Selection Criteria and Implications for Quality of Fellowship Awards” (Annex A).

The target was to grant 4 Grants per region, provided that all awards met a basic standard of quality. This target was achieved, after some adjustments in the Selection Panel’s ordering of proposals were made during the third stage in order to achieve diversity in the origin of proposals and research topic.

In the fourth and final stage the results of the selection process and awards were announced and formal Fellowship offers were made to the Researchers and Support Institutions selected. Four highly ranked alternates were designated, in case negotiations with the 12 finalists were not successful.

16

Project Outputs: The Fellowship Awards

Successful research requires excellence in scholarship. Excellent emergent scholars will on their own identify proposals that address the central topic of concern – the impact of public access to ICT - that is of interest to them and will therefore engage their effort with commitment and enthusiasm. Having a good group of emergent researchers and sensible study proposals in turn makes it easier to recruit mature top scholars to support and carry out a successful research program and help strengthen research capacities of the emergent scholars and their support institutions.

Award winners come from 12 countries (Table 3), and represent Asia (4), Francophone (1) and Anglophone Africa (2), the Middle East (1), Latin America (3) and the Caribbean (1). Gender was not a selection criteria, but gender balance was nearly achieved with 5 women and 7 men selected as Fellows.

Significant public access venues in the two largest countries in the World, China and India, will be studied (Table 4). Investigations will cover rural and urban areas, government sponsored telecenters, cybercafés, and even one library program (Chile’s Biblioredes). Research frameworks to be used are varied. Evidence-gathering methods include quantitative, qualitative and mixed methods. Most research partners will be academic institutions – 7 Universities and 3 research centers - but the program will also be working with 2 NGOs.

Diversity was achieved without undue sacrifice in quality. Award winners were chosen after reviewing a total of 66 applications following a 3-stage screening process. The first two stages were based solely on eligibility, quality of proposals and their potential for strengthening local research capacity.

At the third stage, three countries were represented by two applications each in first order finalists. These were substituted by three other highly ranked applicants. The point spread between discarded first order finalists and the alternates that substituted them to further diversity are small; mostly accountable by reasonable differences of opinion between Panel Members.

The Selection Panel granted exceptions to emergent researcher requirements to three award winners whose last research degree was obtained more than 7 ago. In two instances, the differences were considered minor, as their last degree was obtained in 2001. The third exception was granted to a female Francophone Africa researcher who presented an excellent proposal and whose DEA degree was granted in 1985.

17

18

19

Program Management and Administration

Management and Implementation Team

Universitat Pompeu Fabra (UPF) is the Lead Academic Institution responsible for administering the Program. The program depends directly on the Political and Social Sciences Department.

Visiting Professor Francisco J. Proenza has been designated by UPF to be Program Manager and lead specialist in Public Access to ICTs. Jacint Jordana and Miquel Oliver are co-project managers of the program. These senior UPF officials, are engaged in every major technical, financial, managerial and administrative decision. To help Dr. Proenza expedite UPF procedures and, at the same time, help UPF monitor day to day expenditures and technical decisions, Dr. David Sancho a professor with a long experience working for UPF administration, has been added to the program leadership team, as a third project co-manager. This modification in the program management structure was communicated to IDRC's Grant Manager at a meeting held in Barcelona on November 29, 2009 (Please refer to corresponding aide memoire).

UPF has also designated a Program Officer, Lorena Camats, to support the program regarding administrative matters.

UPF professors will also provide specialized scholarly support as needed.

Three regional partners work together with UPF to implement the Program. The three regional partners and their base of operations are:

Latin America and the Caribbean: Hernán Galperín, professor at Universidad de San Andrés in Argentina;

Asia-Pacific: Erwin Alampay, professor at the University of the Philippines in Manila;

Africa and the Middle East: Abioudun Jagun, research associate at South Africa’s LINK Centre at the University of Witwatersrand.

Together with the Program Manager and the Program Officer, these 3 regional partners make up the Program Implementation Team.

Administrative Support

The University’s Research and Economic Affairs Area gives administrative support to the implementation team.

(See www.upf.edu/universitat/en/estructura/administrativa/vgsi.htmland: www.upf.edu/universitat/en/estructura/administrativa/adir.htmland an overall UPF organization chart here:www.upf.edu/universitat/en/estructura/administrativa/)

20

Within the Research and Economic Affair’s area, the following units interact frequently with the Program Manager and the Program Officer.

The Innovation and Research Parks Unit is the main entity backstopping the program. It is the staff of this unit that comes to our aid or gives us advice whenever we face a problem, need assistance with a procedure or form (contracts, formal announcements, payments), or need to coordinate with another group within the University. (www.upf.edu/universitat/es/contacte/unitats/uipr.html)

UPFs Economic Management Section of the Ciutadella Campus in collaboration with the Budget and Financing Service, especially its Accounting Section help the team manage program funds, approve receipts and complete payment procedures. They give advice regarding the rules that need to be followed to comply with University financial administration policy.(www.upf.edu/universitat/en/estructura/administrativa/secgeciutad.htmlwww.upf.edu/universitat/en/estructura/administrativa/sppff.htmlwww.upf.edu/universitat/en/estructura/administrativa/secc-comp.html)

Web page design and implementation and content management training is provided by UPFs Institutional Information and Promotion Unit, a dependency of the Rectorate’s Office. www.upf.edu/universitat/es/contacte/unitats/usice.html

The Property Management and Contracting Service gives assistance processing some of the program’s larger contracts.http://www.upf.edu/universitat/en/contacte/unitats/sgpat.html

Problems Encountered

The following problems were encountered during implementation.

1. The Spanish Government requires that all government institutions hiring staff either retain 24% for tax purposes, or, in the case of foreigners from countries with which Spain has a tax treaty, that they present a certificate from their country’s tax authority certifying that the person pays income tax in their home country. This requirement has caused delays in issuing payments to contractors.

2. UPF manages several education and research programs for the European Union and has procedures to work internationally. This program however steps out of the mold, and this has led to some delays in the processing of payments. Program Management is working closely with administrative units to address resolve the problem.

3. The selection procedures went smoothly but took about one month longer than expected, beyond the tight scheduled envisaged in the program document.

4. Work demands on program staff are presently at a peak. It is a period between the selection of Fellowship awards and the Research and Planning workshop. The Program Officer has been hired on a full time staff basis between March and June to be able to attend this increase in administrative work.

5. The scheduling of the preparation of the Interim technical and financial reports presented an added burden on staff during a critical peak period in work demands. It would have been preferable to set the submission date

21

for these reports to a date two or three months later, after the workshop (e.g. May).

Implementation calendar

The program will probably need to be extended beyond the period specified in the grant agreement between UPF and IDRC, by one (April 2011) or two (May 2011) months (Table 7).

Some savings will be achieved because IDRC is making available one of its staff members, Ramata Thioune, to serve as International Advisor to support the work of two of the twelve research teams. Accordingly, additional funds may not be needed to cover the expected delay in program completion.

Table 7. Implementation Calendar

Activity Initial Plan Realized or ProjectedCall for Proposals is launched. 15 September 2009 10 September 2009

Consultations: General and Topic Queries

October – November 30, 2009 October – November 30

Submission of Applications Due 31 December 2009 31 December 2009

Selection of Awards January 2010 9 February 20010Announcement of Fellowship

Awards 2 February 2010 18 February 2010Fellowship Agreements

signed with Principal Investigators and Support

Institutions3-15 February 2010 Ongoing

Training and Research Planning Workshop

(4 days)24-27 March 2010 Effective (retroactively if

needed) 15 March 2010Field Research and mentoring

activities15 Feb. 2010 –

14 February 201115 March 2010 -14 March 2011

Research reports submitted and Program publication

prepared and disseminated1 January –

31 March 2011 1 February – 30 April 2011

Final Technical and Financial Reports Submitted to IDRC 31 March 2011 30 April 2011

Capacity Building and Impact

Capacity building activities were negligible during the reporting period. To some extent, the topic query may be considered a “capacity building exercise” in the sense that guidance was given to participants on how to improve the conceptualization of their research proposal and sharpen their research questions. Overall, however, program’s activities during the reporting period were centered on laying the foundation for the next 12 months, when the research studies and capacity building activities will take place.

The program’s inputs will be effective to the extent that they help participants achieve their own objectives. The program did not set capacity building priorities, but instead, asked applicants to identify what those priorities

22

were for themselves and for the organizations they work for. The responses given were used in part to score their applications (see Annex A).

Recommendations

Cost of preparing detailed documentation should be taken into account

The call for proposal was launched 10 September 2009. The program was initially scheduled to start much earlier, as early as March 2009. UPF did not wait for the program’s approval but instead continued to work for the entire 6-month delay on the preparation of these materials. If the program had been approved earlier as planned, the materials available for publication at launching time would have much poorer in quality.

IDRC planners may wish to take note. It is doubtful that other partners would be in a position to use delays in a project’s approval to work on program documentation using own resources.

R1. Future projects wishing to have a similar level of detailed service-oriented documentation should consider the additional 6-month effort that is required, and the cost of this effort should be taken into account.

Develop formal mechanisms for sharing practical lessons of experience

Much was learned from the practical experience of other IDRC competitive grant projects.

SIRCA staff and IDRC’s program officer in charge of that program, provided detailed comments to the draft Submission Guidelines. Practically all their suggestions were incorporated in the final version of the program’s documentation. SIRCA also answered concrete questions of a practical nature that we asked early in the program and gave us access to important materials, such as the Agreement used for the management of the research grant, which we used as a basis to prepare our own.

R2. As was done for this program, IDRC project managers should be put in contact with staff working on similar projects and interaction with these colleagues should be encouraged.

R3. The technical documentation that IDRC makes available online is extensive and valuable. It would also be useful if a series of documents were also made available addressing the more practical aspects of project management.

One way to do this would be to prepare a series of templates. The other one, perhaps more practical given that projects vary widely, would be to have a repository of documents previously used successfully by past IDRC projects. Some of the documents that would be useful to make available online are:

Agreement for the Management of Grant FundsCalls for ProposalsIDRC Budget GuidelinesDraft Ethical StandardsSelection CriteriaApplication Form and instructionsTopic-Query Form and instructions

23

Reaching research communities accustomed to working in languages other than English require special effort and resources

The program had no difficulty reaching Spanish-speaking research communities in Latin America or Caribbean, Asian and African research communities accustomed to publish and conduct research in English.

Special efforts made by regional coordinators to reach out to researchers from Francophone Africa, the Middle East and Portuguese speaking countries ultimately paid off. The program received four applications from Francophone Africa, one from the Middle East and one from Brazil. In the end, awards were granted to well deserving applicants, one each from Francophone Africa and from the Middle East. These two award winners are fluent in English.

R4. Reaching out to large research communities who often work in their own language may require more dedicated efforts than were possible through this program.

R5. Journal Editors are often reticent to distribute calls for proposals. Given the significant impact that such distributions can make (Chart 1), IDRC should consider using its considerable powers of persuasion to convince journal editors of the importance to its own members of distributing these calls.

Web Monitoring is a valuable tool for identifying gaps in outreach

R6. The statistics available using tools like Google Analytics are critical for effective targeting of research communities. It is recommended for use by grant programs that include as an objective to reach out to a broad developing country audience.

A Service Oriented Approach is Essential

At the present stage in the implementation of this program a change in title does not seem necessary and could even be counterproductive. Changes in title often have unintended consequences in branding and identity. What we wish to emphasize and recommend here, based on this program’s experience, is captured in our seventh recommendation.

R7. The title given to a competitive grant program should make it easy for prospective applicants to identify critical eligibility elements.

This is in keeping with a service-oriented approach that takes into consideration the requirements of program administrators and donors, as well as those of participants in the competitive grant process. In our case it was the topic, but other programs may need to consider other fundamental features of their own program.

The Topic Query procedure was time consuming for the program management team but proved its value. It improved the number and quality of suitable applications; it enable us to guide prospective applicants and help them strengthen their applications and seems to have also helped participants save time preventing them from preparing full proposals addressing an unsuitable topic. The TQ procedure also helped determine whether the likely pool and quality of prospective applicants was adequate one full month before the deadline. If by 30 November we had not received a sufficient number of quality topic queries, we would have been able to take corrective action, e.g. extending deadlines, relaxing eligibility requirements, and increasing dissemination efforts.

24

R8. The topic query was indispensable for this particular competitive grant. It may not be necessary for other programs where the topic of research is broader or less open to question. What is indispensable is the adoption of a service approach to competitive grants, to economize in effort of program staff as well as participants in the contest, and to attract high quality applicants.

25

Bibliography

Bearman, Jessica, Drowning in Paperwork, Distracted from Purpose: Challenges and Opportunities in Grant Application and Reporting, A report from Project Streamline, Grants Managers Network, April 2008.(www.projectstreamline.org/documents/PDF_Report_final.pdf)

Bernard, Anne K., Adult Learning and Capacity Development in IDRC: A Concept Paper, February 2005a, (www.idrc.ca/uploads/user-S/11635257651Adult_learning_and_capacity_building_in_IDRC_A_concept_paper.doc)

Bernard, Anne K, Mapping Capacity Development. Report prepared for IDRC Evaluation Unit, February 2005b.

(http://www.idrc.ca/uploads/user-S/11635261961Mapping_Capacity_Development.doc)

Bernard, Anne, K., Lessons from IDRC Evaluations on Competitive Grants: A review of 5 evaluations, Ottawa, April 2006.(www.idrc.ca/en/ev-137870-201-1-DO_TOPIC.html)

Ernberg, Johan, “Universal Access through Multipurpose Community Telecentres – a Business Case?” Global Knowledge Conference, Toronto: May 1997

Ernberg, Johan, “Universal Access for Rural Development: From Action to Strategies”, Paper presented at the First International Conference on Rural Telecommunications, Washington, DC, November 30-December 2, 1998.(www.itu.int/ITU-D/univ_access/seminar/buda/papers/final/f_ernberg.pdf)

Gómez, Ricado. Telecentre Evaluation: A Global Perspective. Report of an International Meeting on Telecentre Evaluation, IDRC: Ottawa, 1999(www.idrc.ca/uploads/user-S/10244248430Farhills.pdf)

Heeks, Richard. Theorizing ICT4D Research, InformationTechnologies and International Development, Vol. 3, No 3 (Spring 2006)

Proenza, Francisco J., Bastidas-Buch, Roberto, and Montero, Guillermo, Telecenters for Socioeconomic and Rural Development in Latin America and the Caribbean, FAO-ITU-IADB, Washington, DC, May 2001.(www.iadb.org/sds/itdev/telecenters/)

Proenza, Francisco “Towards High Impact Sustainable Telecenters in Sri Lanka”, Consultancy Report for the World Bank, 2008.

Sey, Araba and Fellows, Michelle, “Literature Review on the Impact of Public Access to Information and Communication Technologies”, CIS Working Paper No. 6, University of Washington Center for Information & Society, Seattle, April 2009.(http://globalimpactstudy.org/2009/05/new-literature-review/)

Souter, David, James, Tina, and Wild, Kate, Impact Assessment: Component 1 Report, Building Communication Opportunities (BCO), June 2007.(www.bcoalliance.org/system/files/BCOImpact+Assessment_C1Report.pdf)

Toyama, Kentaro, KarishmaKiri, DeepakMenon,Joyojeet Pal,SuneetSethi, and JanakiSrinivasan, “PC Kiosk Trends in Rural India”, Seminar on Policy

26

Options and Models for Bridging Digital Divides, Tampere, Finland, March 13-14, 2005.(http://research.microsoft.com/research/tem/kiosks/PC%20Kiosk%20Trends%20in%20Rural%20India.doc)

27

Annex A: Relative Weighs Assigned to Selection Criteria and Implications for Quality of Fellowship Awards

The following criteria will be used to evaluate the Proposals.

1. Presentation and clarity of the proposal2. Potential impact of research findings3. Quality of research design and methodology4. Scholarship record of Principal investigator.5. Adequacy of Budget, Work Plan and Plan for the Application of Funds6. Gender sensitivity of research proposal7. Impact on developing country research capacity.

Criteria 1-6: Interdependent quality attributes

Our inclination is to see these 7 criteria as “independent”, but they are not. In particular, the grading assigned by each Selection Panel Member with respect to criteria 1-3 and 5 is likely to be correlated. This is because these criteria try to measure “quality attributes” that are under the control of a good Principal Investigator who has the capacity to cover these attributes well when he or she prepares the proposal.

To some extent the same is true for criterion 6; a good applicant will make sure that it is gender sensitive. In practice, however, a good applicant could present a proposal that is important, clearly stated, has a high potential impact and is well planned, but that is nevertheless completely insensitive regarding gender issues. Criterion 6 was explicitly included to make sure this did not happen or that if it did, gender insensitive applicants were penalized. Frankly, its inclusion also forces us, especially male panel members, to look closely at the gender issues when grading each application.

The correlation in the way that applicants address and Panel Members grade criteria 1-3, 5 and, to some extent also 6, which are really “quality” criteria means in practical terms that a good proposal is bound to get high marks, by each panel member with respect to each of these 4 (or 5) criteria; and that a low quality proposal is likely to get low marks in respect of all of these 4 - or 5 - criteria by each and every panel member.

Having separate criteria for various aspects of quality is useful to separate the best from the good. Most importantly, it allows us to score4 (or 5) “quality attributes” and, consequently, give quality attributes, considered as a whole, greater weigh as compared to the implicit unitary weigh assigned to each of the two other “independently determined” criteria.

Criterion 4: A quality attribute that is subject to measurement errors

In an ideal world where we could measure with precision the scholarship record of principal investigator, criterion 4 would also be highly correlated with the other “quality criteria”.5 A good applicant will tend to come from a good school and have a good scholarship record and would therefore be most likely to present better proposals. However, assessing this criterion presents measurement problems: What is a good school? Is having attended a good school a guarantee that the applicant is a good scholar? Aren’t there good student applicants who despite their limited scholarship record hold considerable promise?

Criterion 7: An Attempt to Measure a Central Objective of the Program5 In fact, the Program document “Award Criteria and Selection Process” refers to the first 6 criteria as the “quality criteria”.

28

Criterion 7 is perhaps the one that is most “independent” from quality than any of the others. You could have an excellent proposal submitted by an excellent emerging scholar who proposes to work individually with limited impact on local research capacity. We already received a topic query like this.

It is because the impact on developing country research capacity is central to our Project that it appears as a separate selection criterion. Yes, we want to support high quality research, but we want to do it in a way that helps build the capacity of local researchers, to carry their own ICT4D research after the project has ended. While the program will generate opportunity for building or strengthening the capacity of individual researcher, any proposal that indicated the research could potentially lead to further capacity building (e.g. at an institutional level), should be considered to be of high value.

Fellows will learn to carry out high quality ICT4D research through on the job training – e.g. mentoring – but if we are going to be successful we need to work with willing learners. This is why the way applicants themselves identify and tell us they wish to profit from the opportunities that our program gives them is so important. We are thus incorporating into our selection criteria Anne Bernard’s notion that

“the capacity goals of a project need not simply be relevant, they must be seen to be relevant – to the values, priorities and needs (for knowledge, skills, actions) of al those expected to engage with, support and acquire them.”6

This is why we will reward team applications and give higher marks with respect to this criterion to applications that are most likely to have an impact on local research capacity as understood by the applicants themselves.

Research capacity building is not a narrowly defined concept, and it can take various forms of actions. For the purpose of our program, we are looking for proposals that generate the greatest range of opportunities for research capacity building to take place among the research fellows/teams. Broadly speaking, these opportunities could apply to one or more of the following research capacity dimensions7:

a) Ability to conduct thorough and rigorous research;b) Ability to manage research activities;c) Ability to conceive, generate, and sustain research with respect to a

particular theme or national/regional priorities;d) Ability to apply or use research outcomes in policy or practice; ande) Ability to mobilize research-related policy and program at a systems level

Remember also that we will not award Fellowships to applications that do not meet a basic quality standard. This has a specific meaning in the context of grades assigned by the Selection Panel.8We also include rules to ensure that our

6 Bernard, Anne K., “Adult Learning and Capacity Development at IDRC: A Concept Paper”, February 2005. (www.idrc.ca/en/ev-105773-201-1-DO_TOPIC.html) 7 Bernard, Anne and Greg Armstrong, “Framework for Evaluating Capacity Development in IDRC.” February 2005. (http://www.idrc.ca/evaluation_capacity/ev-105774-201-1-DO_TOPIC.html) 8? To satisfy the basic quality standard an application must fulfill two conditions:

i. It must receive a score of 63 or more from the Selection Panel. ii. Not more than three criteria receive a grade lower than 9.

29

search for diversity (e.g. in regional representation or topic) leads to significantly lower quality proposals.

Summary

In sum, we do not want “local research capacity impact” at the expense of quality, but we insist on a minimum impact on local research capacity. We give greater weigh to quality by separately grading 4 (or 5) attributes of quality. We also include a separate criterion assessing the PI’s scholarship record, for what it may tell us as a predictor of excellence in scholarship under our program.

The first condition may be met, for example, by an application with a score of 1 by every one of the 9 panellists on every criterion.

1 (score) ´ 9 (Panel Members) ´ 7 (criteria) = 63

Such an application would: 1. Have a satisfactory presentation; 2. Advance our understanding of a priority research question (perhaps in a limited sphere); 3. Have an acceptable research design; 4. Have a promising scholar with a good academic record as Principal investigator (even if with limited prior research experience); 5. Present a sound Budget and Implementation and Disbursement Plan (even if insufficiently developed); 6. The proposal shows some gender sensitivity but lacks a well-defined strategy to address gender issues; 7. Application comes from a research team significantly engaging at least two researchers.

The second condition will ensure that award recipients present proposals that are balanced in quality; i.e. that they meet basic quality levels (a score of 9 or more) in at least 4 of the 7 selection criteria.

30

Annex B. Results of Applicant Surveyby Raymond Hyma, IDRC, February 2010

1. Where did you first hear about the Amy Mahan Research Fellowship Program?

Source: # %Website 9 17.0Distribution List 12 22.6e-mail 17 32.1Promotional postcard

0 0

Word of mouth 10 18.9Other 5 9.4No. of respondents 53 100.0

2. From where do you most often get information about research funding opportunities? (check all that apply).

Source Respondents who use given source# %

Websites 35 66.0Social media/ Social networks (e.g. LinkedIn, Twitter, Facebook) 12 22.6email 28 52.8Distribution lists 21 39.6Printed publications 6 11.3Word of Mouth 14 26.4Other 0 0.0Total number of responses 116Total number of respondents: 53

3. How do you feel that the information available on the Amy Mahan Research Fellowship Program was in terms of being clear and useful? (i.e. general program description, selection criteria, frequently asked questions etc.)

# %Excellent 22 41.5Very good 26 49.1Satisfactory 5 9.4Poor 0 0.0Very poor 0 0.0Total number of respondents: 53 100.0

31

4. For this program, we included a Topic Query so that applicants could ask questions and submit draft proposals to receive comments from the committee before submitting their final application. If you DID NOT send a Topic Query, please ignore this and go directly to Question 5. How did you find the Topic Query process?

# %Useful, it helped our teamimprove our study proposal and our application 30 100.0

A waste of time. There was nothing that we were told that we could not have found out on our own.

0 0.0

Confusing. The response we received to our Topic Query was not clear. We did not know how to proceed.

0 0.0

Total number of respondents: 30 100.0

5. Answer this question only if you DID NOT send a Topic Query. Otherwise, please ignore and go directly to question 6.

Why did you decide to submit your application without submiting a Topic Query? (answer all that apply)

We felt there was little to be gained from submitting a Topic Query 4 16.7

A lot of work was required 1 4.2

We did not really understand what the Topic Query was or how it could help us. 3 12.5

We learned of the Topic Query late and could not submit it in time (before 30 November) 14 58.3

Other (Please specify in the comment box below) 2 8.3

Total number of respondents: 24 100.0

Note: Since the total number of questionnaire respondents was 53 and the sum of responses to question 4 and 5 is 54, it appears that one respondent erred by responding to both questions.

32

6. What is your mother tongue?

# %English 3 6.7

Spanish 14 31.1

French 2 4.4

Portuguese 1 2.2

Arabic 1 2.2

Chinese 3 6.7

Hindi 4 8.9

Other 17 37.8

Total number of respondents: 45 100.0

Note: of the 53 questionnaire respondents, 8 did not answer this question.

7. What is the language that you use regularly to communicate and publish your research work?

# %English 38 71.7

Spanish 9 17.0

French 3 5.7

Portuguese 1 1.9

Arabic 0 0.0

Chinese 2 3.8

Hindi 0 0.0

Other 0 0.0

Total number of respondents: 53 100.0

33

8. Please indicate, in order of priority, which of the following reasons motivated you to apply to the Amy Mahan Research Fellowship Program?

Please list i order of priority.

This chart represents the results in terms of how reasons were rated on a scale of 1 (high priority) to 4 (low). It therefore compares the reasons in a personal priority sequence. “Interest on the research topic” received the most amount of 1st placements but tied with “Opportunity to improve my research skills” if included as 1st and 2ndplacements combined. “Funding Amount” was the least important reason with “Opportunity to get my research published” following as less important. These results can beinterpreted through various comparisons of the ranking sequence.

number of responses 1 2 3 4Respondent

s

Funding amount 6 9 11 27 53

Interest on the research topic 24 14 9 6 53

Opportunity to improve my research skills 22 16 9 6 53

Opportunity to get my research published 3 12 25 12 52

responses as % of respondents 1 2 3 4 %

Funding amount 11.3 17.0 20.8 50.9 100

Interest on the research topic 45.3 26.4 17.0 11.3 100

Opportunity to improve my research skills 41.5 30.2 17.0 11.3 100

Opportunity to get my research published 5.8 23.1 48.1 23.1 100

7 respondents provided additional reasons for applying to the fellowship:

1. to get to know other researchers on this area and to be part of this project, which i consider a great initiative

2. 1 - high priority and so on. 3. Opportunity to get involved in a research leading to contribution at local,

provincial and national level 4. To be a part of Global Impact Study and to get affiliation 5. Opportunity for research with other countries 6. Opportunity to contribute to the development of my immediate community via

the findings of the research 7. Opportunity to include Mongolia's case.

34

Annex C. Results of Survey of Non-Applicants who Submited a Topic Query

by Raymond Hyma, IDRC, February 2010

1. Where did you first hear about the Amy Mahan Research Fellowship Program?

Source: # %WebsiteDistribution Liste-mail 5 71.4Promotional postcardWord of mouth 2 28.6OtherNo. of respondents 7 100.0

2. From where do you most often get information about research funding opportunities? (check all that apply).

Source Respondents who use given source# %

Websites 4 57.1Social media/ Social networks (e.g. LinkedIn, Twitter, Facebook)

email 5 71.4Distribution lists 4 57.1Printed publications 2 28.6Word of Mouth 3 42.8Other 0Total number of responses 18Total number of respondents: 7

3. How do you feel that the information available on the Amy Mahan Research Fellowship Program was in terms of being clear and useful? (i.e. general program description, selection criteria, frequently asked questions etc.)

# %Excellent 2 28.6Very good 3 42.9Satisfactory 2 28.6Poor 0 0Very poor 0 0Total number of respondents: 7 100.0

35

4. There are many valid reasons why a potential applicant might first send a Topic Query and then decide not to submit an application to the fellowship program. The objective of the next question is to determine the extent to which the process provided valuable information to interested researchers and how this process can be improved. After submitting a Topic Query, why did you decide not to submit an application?

The reason we did not apply has nothing to do with the Topic Query exercise

3 3

Our team was discouraged by the negative response to the topic we proposed and did not have the time to develop another proposal

The response we received to our Topic Query was confusing and we did not know how to proceed.

2 2

The Topic Query helped us to realise that we did not meet the program's eligibility requirements.

1 1

Other (Please specify in the comment box below)

Total number of respondents: 6 100.0

An “other (please specify)” textbox was included. Three additional comments were gathered from it:

1. Due to our own seasonal work load, we understood that we can not meet the Amy Mahan deadline and therefore we decided not to continue.

2. The Topic Query response was well replied, however, we realised the time and research angle required would not be enough to meet the limited research duration of the Fund.

3. I had to leave Cambodia in mid-2010, so I realized I would not be able to finish the project if I received the grant.

36

5.Please let us know any way that we could have improved the Amy Mahan Research Fellowship Program to have encouraged you to follow through with an application.

This was an open-ended textbox question. 6 of the 7 respondents provided us with comments:

1. Our reason for not submitting has noting to do with the response to our Topic Query from the Amy Mahan. In fact, on the response to our Topic Query, we got clear and enough clarification on how to proceed with the proposal.

2. by clearly stating the different steps involved in the process of submitting the application

3. A little more flexibility on the research topic regarding impact evaluation. Evaluation is quite a broad topic, and impact evaluation is only one of the domains of evaluation that should be considered in ICTD assessment. Therefore, inorder to assess Impact, other domains of evaluation would have to be done, such as a baseline study, process assessment, scalability assessment, needs etc.Irealised the programme would limit the amount of evaluation needed to eventually conduct an impact assessment.

Furthermore, the commitment from each team member was quite demanding, given we are academics. If we chose to live on the funding alone to conduct the research, it would have not been sufficient as an income. Perhaps, if the program allowed for the delegation of research to various postgraduates or honours students, where the team leader would simply act as the supervisor to guide the research exercise, so that he/she may have enough time to commit to other academic responsibilities.

4. Everything offered was already excellent. I hope future programs can ask for general submission as well as allocate a certain proportion of the funding for specifically poor countries, like Cambodia and Laos.

5. Send information earlier.

6. The fellowship seems to be designed strictly for academics who have extensive technical research experience. the Program should also seek to target rural development workers who do not have such experienced people to complete an application using the language and terminology required and meet other requirements of the program.

37

Annex D: Charts Preparedto Inform Regional Coordinators and Program Partners Regarding visits to Amy Mahan Website

38

39

40

Annex E. Topic Queries: Unsuitable Topics Submitted

Public Access to ICT and digital alphabetisation: A case study of rural telecenters in Cameroon

Impact de la musiquedans la luttecontre le VIH/SIDA en RDC      

Impact of Information and Communication Technologies on Jordanian Youth

Impact Assessment of Connecting Conflict-Affected Children and Youth Initiatives in the Middle East

Public Access to ICTs for Rural Under-served Communities within the Context of a Social Business Model

The Role of Information Communication and Technology in Addressing the High Rate of Infants and Maternal Mortality in Sierra Leone

Etude comparée des impacts des usages sociaux des TIC à partir de l'internet et du teléphone portable

Aligning Agricultural Market Information Services with Farmers’ Information-dependent Process: a case study on mobility aspects of the National Agricultural Advisory Services in Uganda

The Networking of Trusted Information in Disaster Prone Areas of Bangladesh

Impact of Internet Access on Politico-Economic Participation

Impact of mobile access to women empowerment in rural India

Socio-economic Impact of Mobile Phone Based Services

Impact of Rural E-Governance upon People Living Below the Poverty Line: A Study of Two States in India

Learning Video Editing through Teacher Directed Instruction versus Computed-Based Video Tutorials

Las Tecnologías de la Información en la educación a distancia. Su aplicación a lasactividadesenseñanzavinculada a los estudiossuperiores de relacionesinternacionalespara la Paz y la SeguridadInternacional

El software librecomoagente de innovacióntecnológica: un estudio de caso en Colombia y Cuba

Hacia la identificacióndeindicadores del uso y acceso a lastecnologíasmóvilesenambienteseducativosdelascomunidadesruralesdel estado de San Luis Potosí, México.

41