the treatment of experimental errors a short review...29/10/15 errors talk 1 the treatment of...

29/10/15 Errors talk 1

The Treatment of Experimental Errors

A short review

Dr Mark Richards for Third Year Lab

October 2015

Yr 3-4 Office Hours Monday & Wednesdays 12-1pm

Why are errors important?

Two readings of a current before and after an ‘event’ 38.2mA and 38.5mA Is this change significant? - It depends on the associated errors

‘A sensible discussion of errors is the difference between anecdote and science…’

(38.2±0.01)mA and (38.5 ±0.01)mA - significant (38.2±0.2)mA and (38.5 ±0.2)mA – not significant


Introduction

•  It should really be uncertainties rather than errors! •  Experimental errors arise in all measurements •  In third year lab you are expected to handle errors properly:

•  To Identify errors •  To estimate errors •  To manipulate errors •  To interpret errors

•  We will discuss a set of rules for dealing with them •  Remember that errors are never known exactly! •  Make sure that you know what you are calculating and why

What you should already know!

Ø 1st Year Measurement and Errors (Mean, SD, Error Propagation, Line-fitting etc)

Ø 2nd Year Statistics of Measurement Coarse (Probability distributions functions, e.g. binomial, Poisson etc)

Ø 1st and 2nd Year Laboratory work (Application of error analysis to real experimental data)


In 3rd Year Labs you are expected to:

• Understand addition of errors and error propagation • Make sensible calculations from a set of measurements • Work-out error in the mean and standard deviation

• Identify different sources of error (e.g. random vs systematic)

• Make axis choice to give a straight line graph (e.g. log, power)

• Understand dependence upon the number of measurements

• Apply meaningful uncertainty to experimental measurement


Books

•  Barford - Experimental Measurements, Precision, Error and Truth

•  Squires - Practical Physics •  Kirkup - Experimental Methods

These books have useful material on experimental technique as well as on treatment of errors (especially Barford)


Systematic Errors Systematic errors affect all measurements equally •  Systematic errors don’t average out with lots of measurements •  Sometimes there are techniques that can be used to measure the

systematic error – e.g. zero offset in a meter

Sources of Systematic error include: •  Calibration of equipment •  Constant improper use of equipment •  Instrument response with scale (e.g. dynamic range) •  External factors such as change in T or ambient light levels between

readings


Random Errors Random errors affect each measurement differently •  Random errors average out if we take lots of measurements •  Random errors yield results distributed around some mean value

Sources of random errors include: •  Noise (e.g. unwanted electrical signals) •  Lack of sensitivity (instrument may not be able to respond to very

small changes) •  Statistical processes (e.g. radioactive decay)

Random errors displace measurements in an arbitrary direction whereas systematic errors displace measurements in a single direction

Random + Systematic Errors

•  A small systematic error implies high accuracy •  A small random error implies high precision •  Random error and systematic error (if known) are

combined by adding the squares of the separate errors. •  The total error associated with a value provides

information concerning the probability that the value falls within certain limits.

•  Of course, there are mistakes (such as reading x100 scale

as a x10 scale)


Random + systematic errors continued… Example: A length is measured with a reading (random error) given by (90±3) cm using a rule of calibration accuracy 3%.

Absolute error = 0.04 x 90 = 3.6cm

(Total fractional error)2 = (Random error)2 + (Systematic error)2

Total fractional error ≈ 0.04

Value =(90±4)cm €

=

2390"

# $

%

& ' +

2(0.03) = 0.002011


The Rules (1)

•  A measurement result without an uncertainty value or without units is meaningless

•  Quote the result of every measurement in the form a±b with a and b given to the same number of decimal places. •  (i.e. the result is likely to lie between the values a–b and a+b)

•  Measure and/or eliminate systematic errors where possible •  Reduce random errors by taking several measurements

where they are the principal source of error


The Rules (2)

•  The error (b) may be estimated using general principles or it may be calculated from the statistics

•  Use common sense when working with errors •  Errors are usually only known roughly, so only quote to 1

or 2 significant figures


The Rules (3)

•  The mean, , and the sample standard deviation, s, of a set of n numbers are given by:

•  Make sure you understand how to get and s using your calculator! (s may be called σn-1)

12

2 2 2

1 1 1

1

1 1 1( )1 1 ( 1)

n

i

n n n

i i i

x xn

s x x x xn n n n

=

⎛ ⎞= − = − ⎜ ⎟− − − ⎝ ⎠

∑

∑ ∑ ∑

x

x


The Rules (4) •  Most measurements

approximate to a normal distribution (the bell-like curve)

•  The normal distribution has about 2/3 of measurements within ±σ of the mean value, where σ is the standard deviation of the distribution (i.e. for infinite n)

•  About 95% of the measurements lie within ±2σ of the mean

•  The variance is equal to σ2

σ


The Rules (5) •  The best estimates of the following quantities are given by:

•  Note that the standard error of the mean, σm = s/√n, is the final error you should quote from a measurement


An aside – estimation of s

•  For a quick and rough estimate of s and/or σm, use the range method

•  The range r is the difference between the largest and the smallest measurements Then s ≈ r/√n and σm ≈ r/n

•  Note that, e.g. for n=3, s is only known to ~ 50% anyway! •  Also, you can neglect smallest error if less than say 1/3

main one.


An aside – weighted means

•  A weighted mean is used when you want the mean of a set of measurements that each come with error bars

•  If we have n measurements xi each with standard error si the weighted mean is given by:

•  The weight wi takes account of how precise that value is •  The final error σm is given by 1/ σm

2 = Σ(1/si2)

x

12

1

1 where

n

i i

ini

i

x wx w

sw= =∑

∑


The Rules (6) •  Where a result depends on two (or more) quantities, we have

to use combinations of errors •  The three main cases are (where σx is the error in x, etc):

Note that equation 3 is only true for uncorrelated x and y terms

Example of error manipulation-1

2A rπ= Where r = (5±0.5)m A=78.5m2

Hence final result is A=(79±16)m2

€

Aσ =dAdr

× rσ = 2πr rσ = 15.7m2

P=2L+2W where L=(6±0.2)m and W=(5±0.2)m P=22m

Example of error manipulation-2

€

2

Pσ( ) =20.2( ) +

20.2( ) = 0.08

⇒ Pσ( ) = 0.28m⇒ P = (22 ± 0.3)m


Straight Line Fits

•  Get to know a good straight line fit program •  Make sure that it will calculate errors on coefficients •  Always inspect the plot and check for mistakes and rogue

points •  Always use your common sense when interpreting the

results


Curve Expert

•  Curve expert is available to you on several computers on Level 4

•  It is good for simple plotting and fitting •  To get errors on the fitted coefficients you have to use the

covariance matrix which it calculates


Excel

•  Excel is easy to use but you have to be careful •  Make sure the data points are plotted •  don’t let it plot a wiggly line joining the points •  You can plot error bars – click on a data point and use Format

data series, Error bars •  Make sure that your final plot looks professional!

•  Remove shading •  Remove unnecessary text boxes •  Use sensible axis labels, grid lines and symbols

•  Instructions/advice on use of Excel on the lab web page

www.imperial.ac.uk/physicsuglabs


Excel (continued)

•  To plot a fitted line on the graph, use Chart; Add Trendline; Display Equation

•  To get errors on coefficients, use Tools; Data Analysis; Regression. This gives all you need and much more besides!

•  You can choose to fit different types of curves •  For more advanced fits, use Tools; Solver which allows

you to specify your own equations •  Note: Newer versions of Excel will have different

commands for these features


Poor use of Excel gives this:

0

5

10

15

20

25

30

0 2 4 6 8

Series1


Or even this:

0

5

10

15

20

25

30

0 2 4 6 8

Series1


But with care you can get this:

0

5

10

15

20

25

30

0 1 2 3 4 5 6 7 8

Time (s)

Leng

th (m

m)


Origin

•  The best program to use for data analysis and plotting is Origin if you have access to it

•  Origin is now available on Level 3 •  It is also available on the data stations in the Level 4

undergraduate labs


The χ2 test

This is for testing the validity of a hypothesis, e.g.: •  whether a set of experimental data is well described by a

particular mathematical model (like a straight line); •  whether a particular parameter has any effect on a

measured quantity; etc. •  chi-square is a measure of significance - a test that shows

whether the results are really meaningful or just due to chance.

Details can be found in most books on statistics


Student's t-test This is for comparing two numbers or groups of numbers (with their associated errors) in order to see if there is a significant difference between them. The difference is expressed in terms of the combined uncertainty (t) and the probability of obtaining this value by chance can be looked up in a table. For more details, see books on statistics


Summary (Errors) •  Identify all of your sources of error (uncertainty) •  Concentrate on the most important source of uncertainty in

the final result •  Ensure that you have minimized errors •  Remember that errors are usually only known roughly •  Don’t use too many significant figures •  Quote result to same significance as error •  Make sensible estimates – not too high, not too low •  It is just as wrong to indicate an error which is too large as

one which is too small

29/10/15 32

Report Writing in 3rd Year Lab General report writing

•  Key skill •  Crucial in every career path e.g. Academia – papers in refereed journals are a permanent record of scientific achievement Industry R&D – record of development work for colleagues & company Business/Finance – Informs on health of business and where investment/resources should be directed Journalism – communicates that vital story


Report Writing in 3rd Year Lab

3rd year lab reports

• Method used to assess your performance in Y3 lab

• Work in pairs in lab – but reports are produced individually

• 100% of your marks are from your report

• 3 standard experiments => 3 reports


(3) Deadlines • If 3 standard experiments: [1], [2], [3] • Report for [1] due within exactly 4 weeks of signing on • Report for [2] due within exactly 4 weeks of signing on • Report for [3] due after Vacations: - If Y3 lab in Term 1, due start of Term 2 (19th Jan 2016) - If Y3 lab in Term 2, due start of Term 3 (3rd May 2016) • Hand in time: 5pm on due date - As per college directive, there is a zero tolerance on late submission of reports. BOTH electronic and hard copy must be submitted on time. - Markers have been asked to return reports within 1 week. They must give you feedback. - If you don’t have your last report back at the end of the next experiment, go and visit the demonstrator to get some feedback before starting to write.


(4) Report Length [ /thirdyearlab/overview ] Reports should be concise as possible without sacrificing technical content. Report length for standard experiments is 2000-3000 words (4-6 sides A4 printed or 6-9 sides A4 hand-written). 3000 word limit is an absolute maximum. This does not include figures, tables, appendices, and references. 10% penalty if > 3000 words - & demonstrator may refuse to mark an excessively long report.


(5) The report is YOUR work •  Lab work - with a partner, Lab reports - done individually

•  Share results, data plots, numbers with lab partner Also discuss background physics & ideas & conclusions

•  BUT your report is your own work and is written by you alone •  Additionally, all quotes, figures &/or data from books, scientific

papers or the web must be fully and clearly referenced


(5 cont..) Plagiarism [http://www3.imperial.ac.uk/library/subjectsandsupport/plagiarism/undergrads] Also see Departmental website for guidance •  When you submit your report, you will supply an electronic copy to

Graham Axtell. This will be put through software to compare to other reports & on the web.

•  Penalties for plagiarism severe If discovered, all cases will be penalised

•  Cutting and pasting of large amounts of text from web and/or from lab/ project partner has previously resulted in penalties varying from loss of all marks from course unit to loss of complete degree

•  Plagiarism is totally unethical, belittles you and your work, & is theft of someone else's creative / intellectual property. It is never justified.


© Imperial College London Page 8

(6) What we want to see in your report

•! Aim: to produce a high quality report in the style of a research publication in a refereed journal [eg. Physical Review Letters]

•! Written in English; readable & clearly written

•! Logical structure [Title, Author, Abstract, Introduction,…]

•! Shows understanding of background physics of experiment; references books, scientific papers

•! Clearly describes experimental apparatus & method

•! Clearly presents results and data; any errors analysed

•! Results fully discussed relative to background physics

•! Conclusions drawn about what you’ve discovered


Guidance on report writing • Also can download electronic journal publications from library - have a look at a Physical Review Letters paper or equivalent Hints & Tips - Good data Before you start, get a good set of results! Garbage in = garbage out You may be a brilliant writer, but unless you’ve got good results to write about, you’ll loose marks. So get a good set of data, & preferably quite a lot of it


Hints & Tips - Data analysis Before you start writing, do this first with your lab partner: • Analyze the data with the equations, model it, extract the numbers, see whether it fits theory, do the errors, etc • Then do the Plots and do the Tables • Then you both go off & write about it!



Hints & Tips - Numerical values

Make them (& their errors) clear!

Correct:

kB = 1.45 (±0.02) X 10-23 J K-1

In increasing “incorrectness”:

kB = 1.4532 (±0.0212) X 10-23 J K-1

kB = 1.4532 X 10-23 ± 0.0212 X 10-23 J K-1

kB = 1.4532 X 10-23 ± 2.12 X 10-25 J K-1

kB = 145.32 X 10-25 J K-1

kB = 145.32 X 10-25 J K-1



Hints & Tips - English

•! Spend time writing good, clear, concise English.

•! Make it readable.

•! Avoid repetition - you have not got the space.

•! Use a spell-checker.

•! Be careful with formatting physical quantities - make distinct from text & make sure same format as equations

–! “the open circuit voltage Voc increased” - poor

–! “the open circuit voltage Voc increased” - good

•! Do your own equations (equation editor)

•! Make it look good - work on formatting.



Hints & Tips - Structure

•! Title + Author

•! Abstract

•! Introduction

•! Background/Theory

•! Experimental

•! Results, Errors & Discussion

•! Conclusions

•! References/Bibliography

•! Appendices



Hints & Tips - Background/Theory

•! Do not just use Lab script as only source - loose lot of marks if do

•! One or two or so websites - can also get good figures (but must of course reference)

•! Text Books - look at perhaps three or so - many are very good for background - can also scan figure in if good one - again reference

•! Scientific papers - look at full papers (eg. PRLs) & review articles (eg. Nature) & popular reviews (eg. New Scientist, Physics World) - library has brilliant electronic journal section - again reference

•! Library eDatabases - lots of search engines (eg. Web of Science) - could find recent papers & reviews on subject (eg. search “wind” & “turbulence”) - very useful for BSc & MSci projects as well



Hints & Tips - Figures

•! Use different symbols & colours in graphs

•! Make labels & numbers clear (font 12 or 14)

•! Plot error bars if necessary

•! Draw apparatus with graphics package

Voltage

source

Oscilloscope

R

+ -

+

+

+

+

+

+

+

+

+

+

+

Laser

pulse

(t=0)

10

-6

10-5

0.0001

0.001

0.01

0.1

0 100 200 300 400 500 600 700

µ (

cm

2/V

s)

F1/2

(V1/2

/cm1/2

)

F8T2

PFB

PFMO



Hints & Tips - References/Bibliography

•! Do not just have Lab script & perhaps a couple of websites - very poor

•! For a v good report I’d be looking for something like 1 lab script, 2 websites, 3 books, 5 papers (a good BSc project would have 20-25 references)

•! Style - look at papers, books:

•! [1] J. H. Burroughes, D. D. C. Bradley, A. R. Brown, R. N. Marks, K. Mackay, R. H. Friend,

P. L. Burns and A. B. Holmes, Nature, 347, 539, 1990

•! [2] http://cn.fpdisplay.com/forum/images/upfile/ (accessed 10 May 2010)

•! [3] S. Wencheng, D. Poplavskyy, F. So, H. Clearfield, D, Welsh and W. Wu, SID Symposium Digest of Technical Papers, 36(1), 1871, 2005.

•! [4] J. -S. Kim, R. H. Friend, I. Grizzi, and J. H. Burroughes, Appl. Phys. Lett. 87, 023506 2005.

•! [5] S. A. Choulis, V. Choong, M. K. Mathai, and F. So, Appl. Phys. Lett. 87, 113503 2005.

•! [6] M. A. Lampert and P. Mark, Current Injection in Solids (Academic Press, New York, 1970)

More Extensive Feedback Provided


Sumary


(9) Effort???

•! A very good report can get you very high marks

•! There is no reason why all of you can’t achieve this

•! You only have to do 3 of them!!!!!

•! I strongly recommend putting the effort in

- it will be worth it!

Thank You


the treatment of experimental errors a short review...29/10/15 errors talk 1 the treatment of...

Documents