Signal and NoiseSignal and Noise

Yongsik LeeYongsik Lee

Signal vs. noiseSignal vs. noise

►NoiseNoise Extraneous and unwanted signalsExtraneous and unwanted signals Superimposed on the analyte signalSuperimposed on the analyte signal From radio engineering, presence of unwanted siFrom radio engineering, presence of unwanted si

gnals was noise or static soundgnals was noise or static sound

Signal vs. NoiseSignal vs. Noise

5A Signal-to-Noise ratio5A Signal-to-Noise ratio

►S/NS/N Noise is independent of signal intensityNoise is independent of signal intensity Absolute noise level is usually constantAbsolute noise level is usually constant Big signal has lower S/NBig signal has lower S/N Better parameter than absolute noise levelBetter parameter than absolute noise level

►DefinitionDefinition S/N = (mean)/(standard deviation) = S/N = (mean)/(standard deviation) =

<x>/s<x>/s <x>/s = 1/RSD<x>/s = 1/RSD

Estimation S/NEstimation S/N

► S/N (dB)S/N (dB) S/N (dB) = 20 log(S/N)S/N (dB) = 20 log(S/N)

► Confidence levelConfidence level 주어진 확률로 모집단의 평균이 들어있을 한계 범위를 주어진 확률로 모집단의 평균이 들어있을 한계 범위를

구한 것구한 것 신뢰한계 신뢰한계 (CL)(CL) 표본표준편차의 크기에 의존표본표준편차의 크기에 의존 ±5±5σσ 99% means (max-min) = 5 99% means (max-min) = 5σσ

► Good S/NGood S/N S/N = 2-3 S/N = 2-3 이면 눈으로 관찰 불가능이면 눈으로 관찰 불가능 Figure 5-2Figure 5-2

5B Sources of Noise5B Sources of Noise

► Two types of noises in chemical analysisTwo types of noises in chemical analysis Chemical noiseChemical noise

►분석하려는 시스템의 화학적 성질에 영향을 주는 변수로부터분석하려는 시스템의 화학적 성질에 영향을 주는 변수로부터►온도온도 , , 압력의 작은 변동압력의 작은 변동►상대습도상대습도 , , 진동진동 , , 주변 빛의 세기 변화주변 빛의 세기 변화 , , 실험실내 증기실험실내 증기►분석물질에 따라 처리분석물질에 따라 처리

Instrumental noiseInstrumental noise►각 기기 부분에서 나오는 다양한 잡음으로 구성각 기기 부분에서 나오는 다양한 잡음으로 구성►Thermal noiseThermal noise►Shot noiseShot noise►Flicker noiseFlicker noise►Environmental noiseEnvironmental noise

Thermal NoiseThermal Noise

► Johnson NoiseJohnson Noise White noise (independent of frequency itself)White noise (independent of frequency itself) Thermal motion of electronsThermal motion of electrons Will occur even in the absence of a currentWill occur even in the absence of a current T = 0 K, no thermal noiseT = 0 K, no thermal noise

► Rms noise voltageRms noise voltage <<νν>>rmsrms = (4kTR = (4kTR ΔΔf)f)1/2 1/2 (Volts)(Volts) Noise in a frequency bandwidth of Noise in a frequency bandwidth of ΔΔf (Hz)f (Hz) K = Boltzmann constantK = Boltzmann constant R = resistance of the resistive element (R = resistance of the resistive element (ΩΩ))

White NoiseWhite Noise

►DefinitionDefinition 공기 중에는 수많은 종류의 잡음이 있으며공기 중에는 수많은 종류의 잡음이 있으며 , , 이 이

잡음들은 모든 주파수에서 발생하기 때문에 이러한 잡음들은 모든 주파수에서 발생하기 때문에 이러한 잡음을 백색잡음잡음을 백색잡음 (white noise)(white noise) 이라고 한다이라고 한다 . .

주파수가 높은 전자파의 일종인 가시광선이 주파수에 주파수가 높은 전자파의 일종인 가시광선이 주파수에 따라 색상이 다르지만 여러 색광이 겹치면 하얀색이 따라 색상이 다르지만 여러 색광이 겹치면 하얀색이 되듯이되듯이 , , 여러 주파수의 잡음이 모인 것이라서 여러 주파수의 잡음이 모인 것이라서 붙여진 의미붙여진 의미

이와는 반대로 특정 주파수에 집중된 잡음을 이와는 반대로 특정 주파수에 집중된 잡음을 유색잡음유색잡음 (Color noise)(Color noise) 이라고 부른다이라고 부른다 . .

►Rise time and Rise time and ΔΔff ΔΔf = 1/(3tf = 1/(3triserise)) Rise time = time for 10% to 90%Rise time = time for 10% to 90% ΔΔf decrease means rise time increase f decrease means rise time increase

►Slow responseSlow response►Low thermal noiseLow thermal noise

►Temperature and thermal noiseTemperature and thermal noise Low temperature Low temperature

Shot noiseShot noise

►When? Where?When? Where? charged particles across junctionscharged particles across junctions P-n junctionP-n junction evacuated space between the electrodesevacuated space between the electrodes

►Why?Why? Quantized random effectQuantized random effect Statistical fluctuationStatistical fluctuation iirmsrms = (2Ie = (2Ie ΔΔf)f)1/21/2

random effectrandom effect White noiseWhite noise

Flicker noiseFlicker noise

►깜빡이 잡음깜빡이 잡음 1/f noise1/f noise Frequency dependent noiseFrequency dependent noise Long term drift in dc amplifiersLong term drift in dc amplifiers Significant at f < 100 HzSignificant at f < 100 Hz

Environmental NoiseEnvironmental Noise

►환경 잡음환경 잡음 기기 주위에서 생기는 다양한 형태의 잡음기기 주위에서 생기는 다양한 형태의 잡음 전자기 복사선을 수신하는 안테나 역할전자기 복사선을 수신하는 안테나 역할

►Two quiet regionTwo quiet region Good quiet regionGood quiet region

►1-500 kHz1-500 kHz

Fair quiet regionFair quiet region►3-60 Hz3-60 Hz

Environmental Noise SourcesEnvironmental Noise Sources

5C Signal-to-Noise Enhancement5C Signal-to-Noise Enhancement

►Two general methodsTwo general methods Hardware = instrument designHardware = instrument design

►Grounding and shieldingGrounding and shielding►FilterFilter►ChoppingChopping►Lock-in, synchronous detectionLock-in, synchronous detection►modulationmodulation

Software = algorithmSoftware = algorithm►signal averagingsignal averaging►ADC ed formADC ed form

Grounding and ShieldingGrounding and Shielding

► To avoid noise arising electromagnetic radiationTo avoid noise arising electromagnetic radiation GroundingGrounding Shielding – especially for high resistance transducerShielding – especially for high resistance transducer minimizing the lengths of conductorsminimizing the lengths of conductors

►anntenaanntena

► Art not science!Art not science!► Good referenceGood reference

H.V.Malmstad et. al.H.V.Malmstad et. al. Electronics and instrumentation for scientistsElectronics and instrumentation for scientists Appendix AAppendix A

Difference AmplifiersDifference Amplifiers

►Common-mode NoiseCommon-mode Noise Generated in the transducer circuitGenerated in the transducer circuit Appears in an amplified form in the readoutAppears in an amplified form in the readout In phase noiseIn phase noise

►Attenuation using difference ampAttenuation using difference amp Inverting and non-inverting inputsInverting and non-inverting inputs Noise with same phase disappearsNoise with same phase disappears If not enough, use Instrumentation amplifierIf not enough, use Instrumentation amplifier

Analog FilteringAnalog Filtering

► Remove noise that differs in frequencyRemove noise that differs in frequency► Low-pass RC filterLow-pass RC filter

Circuit – Figure 2-11bCircuit – Figure 2-11b Reduce environmental noiseReduce environmental noise Reduce thermal or shot noiseReduce thermal or shot noise For a slowly varying dc signalFor a slowly varying dc signal Figure 5-5Figure 5-5

►High-pass filterHigh-pass filter Circuit - Figure 2-11aCircuit - Figure 2-11a Reduce drift and other flicker noiseReduce drift and other flicker noise For an ac signalFor an ac signal

Narrow band filtersNarrow band filters

►Attenuate noise outside an band of freqAttenuate noise outside an band of freq Noise ∝ (Noise ∝ (ΔΔf)f)1/21/2

Significant noise reduction (NR)Significant noise reduction (NR)►By restricting the input signal to narrow bandwidthBy restricting the input signal to narrow bandwidth

ModulationModulation

►변조변조 // 복조복조 Mo/demMo/dem Low frequency or dc signals are converted to a hiLow frequency or dc signals are converted to a hi

gher frequencygher frequency 1/f noise is less troublesome at higher freq.1/f noise is less troublesome at higher freq.

►After amplificationAfter amplification Filtering with a high-pass filter to remove 1/f noiFiltering with a high-pass filter to remove 1/f noi

sese Then amplify dc signalThen amplify dc signal

Modulation and amplificationModulation and amplification

Chopper AmplifierChopper Amplifier

► Conversion to a Conversion to a square-wave formsquare-wave form Electrical chopperElectrical chopper Mechanical chopperMechanical chopper

► Example - Atomic Example - Atomic AbsorptionAbsorption Use of mechanical Use of mechanical

chopperchopper Use high-pass filterUse high-pass filter amplificationamplification

Lock-in AmplifierLock-in Amplifier

► Recovery of signalsRecovery of signals Even when S/N = 1 or Even when S/N = 1 or

lessless

► SchemeScheme Reference signalReference signal

►Same freq as the signalSame freq as the signal►Fixed phase differenceFixed phase difference

Remove high Remove high frequency noisefrequency noise►Using low-pass filterUsing low-pass filter

Lock-in Amp in LIF detectionLock-in Amp in LIF detection

Lock-in?Lock-in?

Software MethodSoftware Method

►Ensemble averagingEnsemble averaging 종합적 평균법종합적 평균법

►Boxcar averagingBoxcar averaging 소집단 평균법소집단 평균법

►Digital filteringDigital filtering 디지털 필터법디지털 필터법

►Correlation methodsCorrelation methods 상관 관계법상관 관계법

Ensemble averagingEnsemble averaging

► Data in Array formatData in Array format Successive sets of data stored in memorySuccessive sets of data stored in memory Summed point by point (co-addition)Summed point by point (co-addition) Summed data averagedSummed data averaged Figure 5-9Figure 5-9

► S/N increaseS/N increase Mean Sx = (sum of the individual measurements) / (n, nMean Sx = (sum of the individual measurements) / (n, n

umber of measurements)umber of measurements) Mean-square noise = Mean-square noise = ΣΣ(Sx – Si)(Sx – Si)22/n/n Rms noise = (mean-square noise)Rms noise = (mean-square noise)1/21/2

Variance and standard deviation Variance and standard deviation 이라 불리기도 한다이라 불리기도 한다 ..

S/N for ensemble averagingS/N for ensemble averaging

► S/N S/N Mean value divided by itMean value divided by it

s standard deviation s standard deviation = Sx / rms noise= Sx / rms noise Rms = Root mean squarRms = Root mean squar

ee► S/N ∝(n)S/N ∝(n)1/21/2

Nyquist sampling theoremNyquist sampling theorem► Measuring frequency (f)Measuring frequency (f)

f ≥ 1/(2f ≥ 1/(2ΔΔt)t) Where Where ΔΔt = time interval betwet = time interval betwe

en the signal samplesen the signal samples At least twice as great as the hAt least twice as great as the h

ighest frequency component of ighest frequency component of the waveformthe waveform

► ExampleExample Signal f = 150 HzSignal f = 150 Hz Sampling rate = at least 300 sSampling rate = at least 300 s

amples/secamples/sec► Customary practiceCustomary practice

Sampling at 10 times of NyquiSampling at 10 times of Nyquist frequencyst frequency

Using synchronizing pulse is imUsing synchronizing pulse is important to synchronize samplinportant to synchronize samplingg

Nyquist-Shannon sampling theoremNyquist-Shannon sampling theorem

► BackgroundBackground a fundamental tenet in the field of information theory, in particular telecoma fundamental tenet in the field of information theory, in particular telecom

munications. munications. First formulated by Harry Nyquist in 1928First formulated by Harry Nyquist in 1928 was only formally proved by Claude E. Shannon in 1949was only formally proved by Claude E. Shannon in 1949

► The theoremThe theorem when converting from an analog signal to digital (or otherwise sampling a siwhen converting from an analog signal to digital (or otherwise sampling a si

gnal at discrete intervals),gnal at discrete intervals), the sampling frequency must be the sampling frequency must be greater than twicegreater than twice the highest frequency of the highest frequency of

the input signal the input signal in order to be able to reconstruct the original perfectly from the sampled verin order to be able to reconstruct the original perfectly from the sampled ver

sion. sion. ► aliasedaliased

If the sampling frequency is less than this limit, then frequencies in the origIf the sampling frequency is less than this limit, then frequencies in the original signal that are above half the sampling rate will be "aliased" and will ainal signal that are above half the sampling rate will be "aliased" and will appear in the resulting signal as lower frequencies.ppear in the resulting signal as lower frequencies.

Therefore, an analog low-pass filter is typically applied before sampling to eTherefore, an analog low-pass filter is typically applied before sampling to ensure that no components with frequencies greater than half the sample frensure that no components with frequencies greater than half the sample frequency remain. This is called an "anti-aliasing filter". quency remain. This is called an "anti-aliasing filter".

Effect of signal averagingEffect of signal averaging

► Example of NMR spectraExample of NMR spectra Figure 5-10Figure 5-10 Random fluctuations in Random fluctuations in

the noise tend to cancel the noise tend to cancel Signal accumulatesSignal accumulates Thus S/N increasesThus S/N increases

Boxcar averagingBoxcar averaging

► Boxcar averagingBoxcar averaging Digital procedures for smoothing Digital procedures for smoothing

irregularitiesirregularities Enhancing the S/NEnhancing the S/N The average of small number of The average of small number of

adjacent points is a better adjacent points is a better measure of the signal than any measure of the signal than any of the individual pointsof the individual points

Signal details are lostSignal details are lost► ExampleExample

Figure 5-11Figure 5-11 Usually 2-50 points are averagedUsually 2-50 points are averaged Done by computer in real timeDone by computer in real time

Real BoxcarReal Boxcar

Gated integration of a transient Gated integration of a transient signalsignal

Boxcar IntegratorBoxcar Integrator

►Function of boxcar integratorFunction of boxcar integrator Boxcar averaging in analog domainBoxcar averaging in analog domain Sample a repetitive waveformSample a repetitive waveform At a programmable time intervalAt a programmable time interval

►s/n of boxcar integrators/n of boxcar integrator Time constant of the integratorTime constant of the integrator Scan speed of the sampling windowScan speed of the sampling window Aperture timeAperture time

►Time window over which the sampling occursTime window over which the sampling occurs

Digital FilteringDigital Filtering

►TypesTypes (Digital) Ensemble averaging(Digital) Ensemble averaging Fourier transformationFourier transformation Least square polynomial smoothingLeast square polynomial smoothing correlationcorrelation

Fourier Synthesis of Fourier Synthesis of WaveformWaveform

► TheoryTheory formulated by the French mathematician formulated by the French mathematician

Jean Baptiste Joseph Baron Fourier (176Jean Baptiste Joseph Baron Fourier (1768-1830)8-1830)

any periodic function, no matter how triviany periodic function, no matter how trivial or complex, can be expressed in termal or complex, can be expressed in terms of converging series of combinations os of converging series of combinations of sines and/or cosines, known as Fourier f sines and/or cosines, known as Fourier series. series.

Therefore, any periodic signal is a sum oTherefore, any periodic signal is a sum of discrete sinusoidal components.f discrete sinusoidal components.

Square WaveformSquare Waveform

► Although the calculation of Although the calculation of a0, a1, b1, a2, b2, … is a a0, a1, b1, a2, b2, … is a mathematically straightforwmathematically straightforward process, it may becomard process, it may become rather tedious depending e rather tedious depending on the complexity and the on the complexity and the discontinuities of f(x)discontinuities of f(x)

► The Fourier theorem is partThe Fourier theorem is particularly useful in scientific iicularly useful in scientific instrumentationnstrumentation

► http://www.chem.uoa.gr/ahttp://www.chem.uoa.gr/applets/AppletFourier/Appl_pplets/AppletFourier/Appl_Fourier2.htmlFourier2.html

Triangular and SawtoothTriangular and Sawtooth

Fourier transformationFourier transformation

► Sinusoidal frequency componeSinusoidal frequency componentsnts Fourier synthesis of waveformFourier synthesis of waveform Non-sinusoidal periodic signals arNon-sinusoidal periodic signals ar

e made up of many discrete sinue made up of many discrete sinusoidal frequency components. soidal frequency components.

► Fourier Transform (FT)Fourier Transform (FT) The process of obtaining the specThe process of obtaining the spec

trum of frequencies H(f) comprisitrum of frequencies H(f) comprising a time-dependent signal h(t)ng a time-dependent signal h(t)

Fourier analysisFourier analysis

Normal and inverse FTNormal and inverse FT

► H(f) can be derived from h(t) by H(f) can be derived from h(t) by employing the Fourier Integralemploying the Fourier Integral

► All FT algorithms manipulate anAll FT algorithms manipulate and convert data in both directiond convert data in both directions, i.e. H(f) can be calculated fros, i.e. H(f) can be calculated from h(t) and vice versam h(t) and vice versa

Fast Fourier TransformationFast Fourier Transformation

► These conversions (for discretely sampled data) are normallThese conversions (for discretely sampled data) are normally done on a digital computer and involve a great number of y done on a digital computer and involve a great number of complex multiplications (Ncomplex multiplications (N22, for N data points)., for N data points).

► Special fast algorithms have been developed for acceleratinSpecial fast algorithms have been developed for accelerating the overall calculation,g the overall calculation,

► Cooley-Tukey algorithmCooley-Tukey algorithm known as Fast Fourier Transform (FFT)known as Fast Fourier Transform (FFT) With FFT the number of complex multiplications is reduced to Nlog2With FFT the number of complex multiplications is reduced to Nlog2

N. N. The difference between Nlog2N and NThe difference between Nlog2N and N22 is immense is immense with N=106 , it is the difference between 0.1 s and 1.4 hours of CPwith N=106 , it is the difference between 0.1 s and 1.4 hours of CP

U time for a 300 MHz processor.U time for a 300 MHz processor.

Signal Smoothing using FTSignal Smoothing using FT► Noisy signal in time domain : h(t)Noisy signal in time domain : h(t)► FT generates frequency spectrum HFT generates frequency spectrum H

(f)(f) Selected parts of frequency specSelected parts of frequency spec

trum can be attenuated or compltrum can be attenuated or completely removedetely removed

► These manipulations result into a mThese manipulations result into a modified or "filtered" spectrum HΜ(f). odified or "filtered" spectrum HΜ(f).

► applying FTapplying FT-1-1 to HΜ(f) to HΜ(f) the modified signal or "filtered" sthe modified signal or "filtered" s

ignal hΜ(t) can be obtained. ignal hΜ(t) can be obtained.

Fourier TransformationFourier Transformation

Time domain –(FT)-> frequency domain

Fourier TransformationFourier Transformation

►ProceduresProcedures Time domain to frequency domainTime domain to frequency domain Low pass filterLow pass filter

►Remove high frequency (noise) regionRemove high frequency (noise) region

Inverse FT (Inverse FT (FTFT-1-1 ))►UsageUsage

FT-IR, FT-NMRFT-IR, FT-NMR Fast computerFast computer

Least-squares polynomialLeast-squares polynomial

► m-point unweighted smoothm-point unweighted smooth Moving average algorithmMoving average algorithm Simpler software technique for sSimpler software technique for s

moothing signals consisting of eqmoothing signals consisting of equidistant pointsuidistant points

Smoothing width = m Smoothing width = m 개의 점을 개의 점을 평균하여 평균하여 (m+1)/2 (m+1)/2 위치에 표시위치에 표시

An array of raw (noisy) data [y1, yAn array of raw (noisy) data [y1, y2, …, ym] can be converted to a 2, …, ym] can be converted to a new array of smoothed data.new array of smoothed data.

The S/N may be further enhanceThe S/N may be further enhanced by increasing the filter width or d by increasing the filter width or by smoothing the data multiple tiby smoothing the data multiple times. mes.

Advantages of moving Advantages of moving AverageAverage

►Reduction of noiseReduction of noise act as a low pass filteract as a low pass filter Modest S/N enhancementModest S/N enhancement

►Smoothing variables may be Smoothing variables may be determined after data collectiondetermined after data collection Type of smooth, smooth width, number of Type of smooth, smooth width, number of

times that the data are smoothed times that the data are smoothed

►Requires minimal computer timeRequires minimal computer time

Disadvantages of moving Disadvantages of moving averageaverage

►Obviously after each filter pass the first n Obviously after each filter pass the first n and the last n points are lost.and the last n points are lost.

►Moving average algorithm is particularly Moving average algorithm is particularly damaging when the filter passes through damaging when the filter passes through peaks that are narrow compared to the filter peaks that are narrow compared to the filter width.width.

► information is lost and/or distortedinformation is lost and/or distorted because too much statistical weight is given to because too much statistical weight is given to

points that are well removed from the central points that are well removed from the central point. point.

Savitzky-GolaySavitzky-Golay algorithmalgorithm► A least squares fit of a small set of consecutive data points A least squares fit of a small set of consecutive data points

to a polynomial andto a polynomial and► calculated central point of the fitted polynomial curve as the calculated central point of the fitted polynomial curve as the

new smoothed data point.new smoothed data point.► A. Savitzky and M. J. E. Golay, Anal. Chem., 1964, 36, 162A. Savitzky and M. J. E. Golay, Anal. Chem., 1964, 36, 162

77► Convolution integersConvolution integers

a set of integers (A-n, A-(n-1) …, An-1, An) could be derived and usea set of integers (A-n, A-(n-1) …, An-1, An) could be derived and used as weighting coefficients to carry out the smoothing operation. d as weighting coefficients to carry out the smoothing operation.

exactly equivalent to fitting the data to a polynomial, as just describexactly equivalent to fitting the data to a polynomial, as just described and it is computationally more effective and much fastered and it is computationally more effective and much faster

Calculation of DerivativesCalculation of Derivatives

► calculating the derivatives of noisy signalscalculating the derivatives of noisy signals Sets of convolution integers can be used to obtain directly its 1st, 2Sets of convolution integers can be used to obtain directly its 1st, 2

nd, …, m-th order derivativend, …, m-th order derivative Figure 5-14Figure 5-14 Savitzky-Golay algorithm is very useful for calculating the derivatives Savitzky-Golay algorithm is very useful for calculating the derivatives

of noisy signals consisting of discrete and equidistant points.of noisy signals consisting of discrete and equidistant points.► The smoothing effect of the Savitzky-Golay algorithmThe smoothing effect of the Savitzky-Golay algorithm

not so aggressive as in the case of the moving averagenot so aggressive as in the case of the moving average the loss and/or distortion of vital information is comparatively limitethe loss and/or distortion of vital information is comparatively limite

d. d. However, it should be stressed that both algorithms are "lossy", i.e. However, it should be stressed that both algorithms are "lossy", i.e.

part of the original information is lost or distorded. part of the original information is lost or distorded.

Convolution integersConvolution integers

  Filter width (2n+1)

i 11 9 7 5

-5 -36      

-4 9 -21    

-3 44 14 -2  

-2 69 39 3 -3

-1 84 54 6 12

0 89 59 7 17

1 84 54 6 12

2 69 3 3 -3

3 44 14 -2  

4 9 -21    

5 -36      

Correlation methodsCorrelation methods

►Used forUsed for Processing data from analytical Processing data from analytical

instrumentsinstruments Complex mathematical data Complex mathematical data

manipulationsmanipulations Only by computerOnly by computer

HomeworkHomework

►Exercise 5-7, 5-8, 5-9Exercise 5-7, 5-8, 5-9

ReferencesReferences

►http://en.wikipedia.org/wiki/Nyquist-http://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theoremShannon_sampling_theorem

►http://www.chem.uoa.gr/Applets/http://www.chem.uoa.gr/Applets/Applet_Index2.htmApplet_Index2.htm