Contents

1 Research Methodology 1

1.1 What are the types of research? . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Research based on methods: . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.2 What is educational research? . . . . . . . . . . . . . . . . . . . . . 2

1.2 Salient features of Design based research (DBR): . . . . . . . . . . . . . . . 3

1.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2 Characteristics of design based research . . . . . . . . . . . . . . . . 3

1.3 Rationale for selection of DBR for my research problem: . . . . . . . . . . . 4

1.3.1 Stages of design based research as described by researchers: . . . . 5

1.4 Adaptation of my research problem to the generic design research model: . 8

1

1 Research Methodology

1.1 What are the types of research?

Research can be defined in various ways. It could be getting new information (which one

doesn’t posses), or conducting systematic investigation to solve a problem, or proposing

newer ways of solving an existing problem based on the observations etc. The list is

long. One of the definition by M. Shuttleworth [1] is, In the broadest sense of the word,

the definition of research includes any gathering of data, information and facts for the

advancement of knowledge. Based on this definition, research can be classified in broadly

in two categories. One category is where research is based on a purpose and the other one

is based on the method in which it is carried out.

Research based on purposes: Within the research which is based on the purposes, there

can be various sub categories. It can be about acquiring new knowledge, and extending it

further, testing the validity of that knowledge, solving a problem and many other purposes.

All these purposes are valid, as the research outcomes of these adds new knowledge about

the unknown or lesser known phenomena.

Another important type of research deals about the processes and procedures used in

solving a problem. It can range from solving a unsolved problem using a certain process, or

solving it using a different process, and prove that the latter was efficient than the earlier

one. Documentation of the methodology used for problem solving helps the knowledge

base to strengthen, so that other researchers can use it for future use.

1.1.1 Research based on methods:

• Inductive: Observation-Pattern-Hypothesis-Theory

Inductive reasoning starts from specific observations and goes to formalizing a theory.

It is a way of reasoning in which the researcher starts with specific observations.

He/She then observes and studies the patterns in these and then goes ahead to

formulate temporary hypothesis based on the patterns observed. This culminates in

developing some general conclusions or theories. This approach is more open ended

and exploratory.

• Deductive: Theory-Hypothesis-Observation-Confirmation

Deductive approach starts with a theory. This could be a general theory around

the topic of the research. Based on the theory the researcher constructs hypotheses,

which are slightly more specific than the theory. These hypotheses are then tested

by conducting the required experiments and the observations are recorded. After

analysis of the observations, the researcher confirms or rejects the theory which was

considered at the beginning. The deductive approach is narrow in nature as it works

2

from more general to specific conclusions.

1.1.2 What is educational research?

In terms of the educational environment, the situations are ever changing. In this environ-

ment variables like educational curriculum, modes of teaching, methods which the teachers

follow, teaching aids like laboratory equipment, books or classroom amenities etc. can be

changed. These changes may or may not have impact on the learning of the students,

but it is important for researchers to capture the changes. This data of the changes ob-

served during the interventions have to be captured correctly so that the researchers can

consolidate and recommend a plan of action for effective application of the intervention.

For educational research, it is insufficient to look only at the problem and the interven-

tion (used to solve the problem). It is also important to observe the interactions between

the intervention and other stakeholders. The interactions are important as they provide

the insight of the possible options to customize the intervention in case of a specific con-

text. It is useful to record these interactions in order to facilitate decision making based

on these.

Traditional research is used for different domains, but there are some limitations which

make it difficult to accommodate the requirements of educational research. Let us discuss

some of the prominent ones based on some examples:

• Traditional research methods have either theory-driven or empirical data based

methodologies. These fall short of understanding the complexities of educational

research like why and how did a particular educational innovation worked in actual

real life scenario [DBRC, 2003].

• Traditional research approaches (like surveys and experiments) have more impor-

tance to description. It is observed that less importance is given to the solutions

for the design and development problems in education. Theoretical claims are not

sufficient to implement newer reforms in educational domain. They demand system-

atic and contextual research of development and implementation processes [Van den

Akker, 1999].

• One of the important functions of research is its ability to be adapted. A process

published after doing research on the same has to be useful for another researcher

for implementing it in a new context. Traditional research mainly does have results

about intervention ’K’ working successfully in a given scenario ’A’ with ’B’ as a

sample. It is difficult to adapt the findings of this research to ’X’, ’Y’, or ’Z’ scenarios

with samples ”P’, ’Q’, or ’R’ [ ].

3

One of the solution to these lacunae in the traditional research methods is Design

Based Research (DBR). Let us now see some characteristics of DBR:

1.2 Salient features of Design based research (DBR):

1.2.1 Definitions

Design based research is defined by Tjreed Plomp, [1], (page 9) a systematic study of de-

signing, developing and evaluating educational interventions (such as programs, teaching-

learning strategies and materials, products and systems) as solutions for complex problems

in educational practice, which also aims at advancing our knowledge about the character-

istics of these interventions and the processes of designing and developing them.

1.2.2 Characteristics of design based research

This research is useful in creating interventions for real world settings. The interventions

are not one time creations, but created using iterative method. DBR has iterations of the

products, processes or other interventions through cycles of analysis, design, development,

evaluation and revision.

This type of research not only involves the users, but also involves the practitioners at

various stages of the research. The focus of design based research is process oriented. Un-

derstanding and improvement of the interventions is an important aspect of this research

method. The final product of the research, is evaluated (partly) based on its’ utility. This

is tested by the users in real contexts. Along with this, design based research is (at least

partly) based on theories. The systematic evaluation of the intervention contributes to

the theory of Van den Akker8.

1.3 Rationale for selection of DBR for my research problem:

Before the reasons for selecting DBR as the preferred method for my research, here is a

summary of the research problems:

1. Research problems:

• Investigate and validate the role of various domains in the LO creation process

in order to increase the usability of the LOs.

• Create a taxonomy of (well researched) principles from these domains, which

can contribute in the LO creation process.

• Formulate a method to operationalize these principles in to the LO creation

process.

4

• Validate the usability of the various interventions and products in the LO cre-

ation process (ex: Instruction design documents and final LOs)

• Recommend guidelines for LO creation stakeholders (ex: ID writers, Anima-

tors) to incorporate the operationalisation of these principles

In order to address these research problems, DBR is a preferrable choice because of

the following reasons:

2. Reasons for selecting DBR for my research problem:

• Complex nature of the context:

The context for my research is about process of creating eLearning content

on a large scale. This process involves multiple stake holders who contribute

at various stages. The communication / information flow is crucial in this

process. This makes it complex, as there are multiple interactions between the

stake holders at various stages. In order to benefit from those interactions a

combination of methods is required.

• Iterations involved in creation of the interventions and its validation This re-

search is not based on ’create once and use’ format. It is expected to provide

justification to the answer about why a certain intervention (created using ’X’

theory) was found effective/not effective. This requires a mixed method which

can combine empirical data and the underlining of the theories used in creating

the intervention. This process would then be replicated to validate and justify

other principle/s.

• Possibilities of adapting this research for creating LOs in future As mentioned

earlier, this research is about content creation process. This process has to be

such that it should be useful for future LO creators. In order to make it usable,

it is important to use a method of ’apply-test-record’

In the subsequent subsections, variuos models proposed by researchers are summarized.

Further, a particular model is chosen for this research, as it can be adapted to the require-

ments of the research problems mentioned earlier.

1.3.1 Stages of design based research as described by researchers:

Design based research (DBR) methods not only help in designing the intervention but also

in observing and capturing the interactions of that intervention with other variables. These

observations are helpful in augmenting the established theories to make them suitable in

educational research [2]. Given here are some models presented by researchers:

5

1. Reeves (2006) Reeves presented a ’generic model of design based research’ and has

outlined the following stages:

Figure 1: Generic model of Design based research presented by Reeves, 2006

• Analysis stage: Analysis of practical problems by researchers and practition-

ers is done in collaboration. The important point here is the collaboration of

the practitioner in the process. This is uncommon as compared to the tradi-

tional research, where in the practitioner is mostly participating in the last lap

of the research which is user testing.

• Development of solutions: Existing principles and theories are referred to

create a solution for the problem reported in the analysis phase. The ongoing

innovations in the technical field is also considered while designing the solution.

• Iterative cycles of testing Cycles of testing are carried out for refinement of

solutions in practice. Data is collected in suitable format (for the research ques-

tions of the study) and the results are analyzed. The intervention is recreated

along with the iterations, and tested again.

• Reflection of the feedback from users The refined intervention is tested

by the users and the feedback is collected. The analysis of this feedback is used

to produce ’design principles’ and enhance solution implementation.

2. Plomp (200?)

Broadly, following are the stages of DBR as proposed by Plomp [reference for the

paper by Plomp XXX]:

• Preliminary research: In this phase, need and context analysis is done in

the beginning. Review of the literature regarding the domain is done next.

These two sub-stages are useful in formulation of a conceptual framework for

the study.

6

• Prototyping phase: This is an iterative design phase, where iterations are

created, mainly with an objective of refining the intervention. Iterations go

through a cycle of formative evaluation, to add in the improvement of the

intervention.

• Assessment phase: In this phase, evaluation is done to arrive at a conclusion,

whether the intervention or the solution meets the pre-determined specifications

as per the phase 1, (preliminary research). The results of this phase may be in

the form of recommendations or guidelines for the improvement of the inter-

vention or solution. This is called as Semi-Summative evaluation phase.

3. Wademan 2005

Figure 2: Generic model of Design based research presented by Wademan, 2006

Wademan [2005] agrees to the phases mentioned above, and adds more details to the

process by capturing the features and the characteristics of a generic design research model.

The stakeholders at various stages and their roles are defined. The interaction between

the stakeholders too is defined. The process and the stages are explained below. Summary

of this can be seen in the Figure XXX.

• Stage 1: Problem Identification:

Stake holders: Researchers, Practitioners, Collaborative and other sources.

7

Process: In this phase the context is studied to identify the problem. The stake-

holders interact with each other and and arrive at a common conclusion. This is

possible, as ALL the stakeholders are part of the phase.

• Stage 2: Identification of Tentative Products and Design Principles:

Stakeholders: Experts and practitioners in the specified domain

Process: After the problem is identified, preliminary investigation of problem, con-

text, and approaches is done in consultation with the experts and the practitioners.

This is supported by conducting focussed literature review, and analyzing practical

context. Promising examples, which represent the problem are analyzed. This pro-

cess helps in identifying tentative list of products and design principles to be applied

in the study.

• Stage 3: Applying tentative products and theories:

Stakeholders: Researchers, practitioners and end users

Process: Tentative products are created using the tentative list of design principles.

These are introduced to the users and the results are captured.

• Stage 4: Prototyping and Assessment of Preliminary Products and The-

ories:

Stakeholders: Researchers, practitioners and end users

Process: Redesign and refinement of the problem, solutions (created in stage 3)

and method is done based on the feedback received in stage 3. Formative evaluation

is used along with the reflection of the feedback, to do the redesign and refinement.

This is an iterative stage where, the refinement is done for achieving successive

approximation of theory, and refinement of design theory.

• Stage 5: Problem Resolution and Advancing Theory

Stakeholders: Researchers and end users

Process: Practical products are created, after the iterations conducted in stage 4.

These products have certain aspects which contribute to the existing theory.

1.4 Adaptation of my research problem to the generic design research

model:

Out of the 3 models summarized abobe, the model chosen for this research is the one by

Wademan [2005]. Stages of Wademan’s model coincide with the stages of my research.

Other models by Reeve’s and Plomp, have some stages missing in their models, like pro-

totyping stage (in Reeves) or iterations stage (in Plomp). The stake holders mentioned

in Wademan’s model coincide with the stakeholders in my research. Although the model

8

cannot be adopted as it is, it can be adapted to suit the requirements. These modifica-

tions are explained below. The Figure [XXX] shows all the stages of Wademan’s model

of DBR. Figure [XXX] shows the adaptation of that model to my research. The stages

are the same but the details have been changed according to the context. Summary of

stagewise modification is discussed here. The details of the actual implementation are

given in the [solution chapters]:

Figure 3: Adaptation of Wademan’s model for my research

• Stage 1: Problem identification

Stakeholders: Researcher, ID writers, Animation developers and the SMEs.

The motivation of this research is to create a template to facilitate communication

beween the SMEs and the animators. A template is the product chosen for this

information trasnfer. In the first stage the problems in this communication are

identified by discussing it with the relevant stakeholders [Figure XXX]

In the first stage, the researcher creates a basic template (T1) by referring to well

known principles in eLearning domain. This template is used by Instruction Design-

ers (IDs) to create an Instruction Design Document (IDD). The IDD is then referred

by the Animators as a guideline for creating the animation for an LO.

Feedback on T1 is collected from the stakeholders (who use this intervention-T1)

are collected. Analysis of this feedback is the starting point for the further research.

The feedback is collected in two formats. One is a survey (based on a likert scale)

which can give us the usability score of T1, and the other is a questionaire. where the

animator explains the rationale behind the choice he/she has made. The combination

of the techniques in data collection is done to improve the internal validity.

9

Figure 4: Stage 1 Problem identification (In my research)

This data is categorized according to the research problems which are set in the

beginning. This alignment helps in the second stage where solutions are developed

to rectify the problems reported in this stage.

• Stage 2: Identification of tentative products and design principles

Stakeholder: Researcher

The results and analysis of stage 1 are consolidated by the researcher. A focussed

literature review is carried out to look up for principles in various domains, which

can be useful to address the reported problems.

Figure 5: Stage 2 Development of solutions (In my research)

On the basis of the literature review, domains are which have a significant overlap

on the various aspects of eLearning content process are shortlisted. Ex: Animation,

Graphic design, Multimedia, Interaction design and Instructional design. The con-

10

text of LO creation is used as an important factor in deciding this list. Promising

examples are sighted to further strengthen the decision of shortlisting.

• Stage 3: Applying tentative products and theories

Stakeholders: Researcher

Figure 6: Stage 3 Applying tentative products and theories (In my research)

The wellknown principles from the visual communication domains are filtered on the

basis their usefulness in the process of content creation especially for eLearning. The

principles which have the rationale, are then operationalized for embedding them in

the next version of the Template: T2.

Template T2 created by the researcher to be used by the ID writers to create IDDs.

The visual design principles are operationalized explicitly in this version.

• Stage 4: Prototyping and assesment of prelimenary products and theo-

ries:

Stakeholders:

Researcher, ID writers and Animators

In this stage, the product (ID Template) is used by the ID writers (primary users),

and the feedback is collected from them, as well as from the animators (secondary

users). This feedback is analyzed and further refinement in operartionalization of

the visual communication principles is done. This is reflected in the next version of

the template: T3.

• Stage 5: Problem resolution and advancement of theories

Stakeholders:

Researcher, ID writers and Animators

After multiple iterations of the intervention-ID template (T1, T2, and T3) feedback

11

Figure 7: Stage 4 Prototyping and assesment of prelimenary products and theories (In my

research)

is collected from stakeholders. LOs are the product of the intervention, and there-

fore the feedback of the users of the product is collected at the end. These users are

students and teachers who would be using the LOs for understanding the compli-

cated concepts. Feedback from actual users is collected, in the formats like surveys,

interviews and also a assessment test regarding the understanding of the subject.

Figure 8: Stage 5 Problem resolution and advancement of theories (In my research)

The theories which have been operationalized in the template are redefined and a

guidelines document is prepared.

In the chapter on actual implementation of the solution, details of the model presented in

this chapter is discussed in detail.

12