CSIRO Insert presentation title, do not remove CSIRO from start of footer

Experimental Design

• Why design?

• removal of technical variance

• Optimizing your design based on the experimental goal is an important part of a successful microarray experiment

• This requires pre-planning, and cannot be stressed enough.

•  A question you want to ask is what are the most important samples, or comparisons you want to make, and how many experimental factors will be involved? 

• For single-channel array experiments, it is obvious that more replicates should be done for samples of greater importance. 

• For dual-channel array experiments, the many possible choices for designs make for a more complex problem.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

How many replicates do I need?

• As many as possible!

• For an exploratory analysis 3 replicates are usually sufficient, unless the data are particularly noisy (e.g. samples from very small numbers of cells) or the expected effect is particularly small (e.g. changes occur only in very few, specialized cells in the sample).

• Using less than 3 replicates is not a good idea. Most important is the use of real replicates. Do equal numbers of replicates for each condition/comparison to keep the later analysis simple.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Replication! Replication! Replication!

Should I do technical replicates?• No

• Unless you are planning a technical instead of a biological study.

• Repeated hybrization of the same biological sample is a waste of resources.

• Microarrays are reliable and it has been shown repeatedly that this kind of replication doesn't provide any biologically useful information.

• Arrays are becoming ever cheaper, buy more arrays and do some serious biological replication!

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Experimental Design

The common reference design may be more appropriate for designs involving many factors of equal importance, such as comparing the expression profiles of a large number of tissue types, or for experiments that will likely be part of a larger meta-analysis in the future. 

Circular designs may be used when multiple comparisons are needed, such as in a complex time course study.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Common Reference Design

• Perform in duplicate (triplicate would be preferred)

• treatments are only measured twice each

• Reference sample, which is a common control, is measured say 6 times

• Recommended for comparing a large number of tissue types, treatments etc.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Circular design and time courses

• A time course could be constructed using a circular design with multiple comparisons, or a common reference design.

• A common reference provides optimal statistical design for detecting differential expression

• The circular portion allows comparisons between samples of interest

• Need for replication.

• Recommended for comparing a small number of sample types

CSIRO Insert presentation title, do not remove CSIRO from start of footer

What about dye swaps??

• The rationale for the flipped dye design is that it allows for the estimation and removal of gene specific dye effects.  These dye effects have been shown to be reproducible across independent arrays by the use of Control vs. Control arrays.  Any deviation from a ratio of 1 in these arrays is due to either dye effect or residual error. 

• flipped dye arrays in an experiment is one requirement for having a statistically optimal design, which balances all the factors in the experiment- arrays, dyes, and treatments.  (Here, treatments denote any other factor of interest, whether it is a toxin, mouse strain, tissue type, or age group.) 

• Balanced designs do not confound any two experimental factors, meaning that the effect of each factor can be estimated and normalized out of the data. 

CSIRO Insert presentation title, do not remove CSIRO from start of footer

What about dye swaps??

• Balanced designs do not confound any two experimental factors, meaning that the effect of each factor can be estimated and normalized out of the data. 

• When factors are confounded with each other, their effects are indistinguishable.  For example, an experiment done in triplicate without any flipped dye arrays will have treatment completely confounded with dye effects, because each sample is only labelled with one dye each, and there is no way to separate their effects. 

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Replication! Replication! Replication!

What is a real replicate?• A real - or biological - replicate is an independent sample that

is varying all the variables that a colleague in another lab couldn't control.

• In an imaginary "ideal" experiment, each replicate would be performed in a different lab - so it may be advisable to approximate that situation as much as possible. That does not mean that you have to vary all the variables, if you are certain that some of them won't have any effect, e.g. the brand of standard chemicals, the phases of the moon, etc.

• Don’t underestimate the sensitivity of microarrays, variables like batch of cells or time of day can very well have an observable effect. Most of all, be careful to prepare a perfectly matched control for every sample, even if you are going to use single-colour arrays.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

More on Replicates and Pooling

• Biological replicates are arrays that use RNA samples from different individual organisms, pools of organisms or flasks of cells, but yet compare the same treatments or control/treatment combination.

• Technical replicates are arrays that use the same RNA samples and also the same treatments.  Thus the only differences in measurements are due to technical differences in array processing.

• It is highly recommended that more biological replicates are done than technical replicates, especially when dealing with individual organisms rather than a cell line.

• Pooling all samples together is NOT recommended.

• If the degree of variation between biological replicates is thought to be very high, or not enough RNA can be collected from just one individual, then pooling samples may be beneficial.  It is still important to have more than one independent pool of samples in order to estimate biological variance.  Thus, the sample pools may be thought of as individuals when designing your experiment. 

CSIRO Insert presentation title, do not remove CSIRO from start of footer

Pooling samples

• Should I pool samples?

• It is tempting to pool samples to save hybridization costs.

• Unless you do single-cell sampling, every sample is already a pool, as it contains mRNA from many cells.

• Especially if the number of cells obtained from each individual is very small, pooling is the best way of reducing the noise while keeping the number of hybridisations reasonably small.

• Unless you expect to find interesting inter-individual variations, e.g. in a medical study, there is little to argue against pooling.

• However, it is important to pool the biological material (tissue, cells), not the purified RNA or labelled cDNA! In this way, problems are far easier to spot.

• Don't ever include any sample that looks suspicious.

CSIRO Insert presentation title, do not remove CSIRO from start of footer

What else should I do?

• Keep things simple with pair-wise comparisons, and simple time-courses.

• Try to define your expectations (hypotheses) in advance in as much detail as possible.

• Inventing ad hoc explanations later when you get your results is not best practice.

• As Ernst Wit writes in his Ethics of Chance: A statistical "method that makes use of a retrospective study of the data cannot [ever] reach the same significance level as a prior formulation of the hypothesis."

• This is a general issue for the performance and interpretation of scientific experiments and is particularly relevant for microarray studies with their large data sets and surprise observations.