design of esperiment

Design of experimentMariacarla Memeo

Robotics, Brain and Cognitive Science(RBCS)Multisensory mapping group

Istituto Italiano di Tecnologia (IIT), Italy

● Experiment●Basic elements of experiments●How to choose subjects who will participate in experiment●Experiment noises●External validity●Internal validity

● Research questions● Research Hypotheses● Organization of data

Outline

● In a general sense, an experiment is any process or study which results in the collection of data, the outcome of which is unknown.

What is an experiment?

Independent variables Dependent variablesPhenomenon to study



EXPERIMENTAL UNIT

SAMPLE

● Experimental studies are in general restricted to describe situations where:

●Experimental units have been randomly selected from the population●Experimenter has control of the independent variable●Treatments have been randomly assigned to the experimental units



EXPERIMENTAL UNIT

● Simple random sampling: Any procedure that assures that each element in the population has an equal chance of being selected is referred to as simple random sampling.

● Snowball sampling: uses a small pool of initial informants to nominate, through their social networks, other participants who meet the eligibility criteria and could potentially contribute to a specific study. The term "snowball sampling" reflects an analogy to a snowball increasing in size as it rolls downhill.

● Stratified sampling: is the process of dividing members of the population into homogeneous subgroups before sampling.

● Survay sampling: describes the process of selecting a sample of elements from a target population to conduct a survey. The term "survey" may refer to many different types or techniques of observation.

How to randomize sample?

Selecting units and assigning treatments randomly to experimental units is fundamental to avoid● selection noise for example, choosing the most healthy patients for

the treatment that you are trying to prove is best;● systematic noise for example, doing all the tests on treatment A in

January then all the tests on treatment B in March;● accidental noise for example, using the first rats that the animal

handler takes out of the cage for one treatment and the last rats for the other

● cheating by the experimenter

Possible experimental noises

Ind var Dep var

● External validity (generalizability): A study is externally valid if its conclusions represent the truth for the population to which the results will be applied…

… but always observe a subset of the population of interest

External validity

● We want to be able to draw valid inferences from the sample to the population

● To that end, the sample needs to be representative of the population

SAMPLE

External validity

Internal validity


● Inferences are said to possess internal validity if a causal relation between two variables is properly demonstrated.

● To demonstrate a causal relationship, it is fundamental tat the experiment is able to control the treatment applied to the experimental units.

● Casual observation of world around us

● Systematic observation

● your research reports

● Theory

● Need to solve practical problem

● Reading the literature

Source of research ideas

● Is it interesting to you?

● Is it do-able? Can it be done over the next few months?

● What is already known about this area?

● What are the unanswered questions in this area?

● Why is your study important?

Choosing a research topic

● A broad open-ended question your research tries to answer related to a particular phenomenon.

● Research questions are “How” “Why” “What” questions

● (Avoid “yes” “no” questions)

● Ask a question that can be answered● empirical question rather than metaphysicial one

Research Question


● Your entire research project revolves around the research question(s).

● Your study should be designed to answer the research question(s) ● (internal validity)

Research Question

How servo motor lever can deliver sensation of

stable grasping?

● A prediction statement of the outcome of a study

● “Educated guess”

● May describe relationship between variables

● A specific statement that can be tested to which you can answer

with “yes” or “no” based on the data you collect

(Don’t use “why” “how” “what” questions)

● Your research tries to find out whether your hypothesis is supported

or rejected

Research Hypotheses

Independent variables Phenomenon to study

● Research QuestionHow servo motor lever can deliver sensation of stable grasping?

● Research HypothesesAngular motion of the lever is perceivable and noticeable with 18 degree angle of difference.

JND = The minimum difference we can detect between two is the just noticeable difference of j.n.d.

kR

R


● Research Hypotheses

Try it yourself…


● Research HypothesesGrasping sensation should be efficient delivered, meaningful according to following principles.

Perceptual principles to organize stimuli

● Gestalt psychology – people derive meaning from the totality of a

set of stimuli rather than from any one individual stimuli;

● Principle of closure – consumers tend to perceive an incomplete

pictures as complete, filling in the blanks based on previous

experience;

● Principle of similarity – consumers tend to group togheter objects

that share similar physical characteristics;

● Figure ground principle – where one part of the stimulus will

dominate while others recede into the background.

● Information on the experimental units (subjects)– Age, sex, weight, handedness, level of education, …● Information about the treatments (independent variables)– Experimental condition, stimulus used, …● Dependent variables– Responses of the subjects to questions– Trajectories, time series (Vicon, force sensor, …)– Physiological signals– Videos

What are the data ?

Dependent variablesPhenomenon to study

In a typical behavioral or cognitive science study (subjects’ behavior and stimuli interpretation), a program is used to present stimuli to the subject and other devices would be used to record its responses.

● Keep configuration files of all software / devices together with the data;

● This is particularly important if stimuli are presented in a different order for different subjects;

● Possibly keep the executable of the program

Configuration files are data

Many things can go wrong during the experiment: it is good practice to have scripts ready to look at your data during the experiments to make sure there is no technical problems.● Data preprocessing: Extraction of indicies/variable of interest for the

analysis;● Exploratory data analysis: graphical, visual exploration of relationships

between variables;● Statistical analysis;● Reports writing: Ideally, the report should document the various stage of

data analysis.

Data Analysis

design of esperiment

Science

internal validity inferences

stratified sampling

survay sampling

simple random sampling

term snowball sampling

study whichresults

treatment b

aspecific study