ni introduction
TRANSCRIPT
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 1/76
BC-0504Natureza da Informação
David Correa Martins [email protected]
Introduction and some bits about theHistory of Information Theory
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 2/76
• Prof. David Correa Martins Junior (CMCC)• Email: [email protected]• Email subject: [N! "ubject
• Delta #uildin$ ("%o #ernardo)& room '
• *esearc+ interests• Pattern reco$nition a,,lied to #ioinformatics and "-stems
#iolo$-.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 3/76
Course Goas
/oal: 0o ,resent t+e foundations of t+e nformation Nature
o
.e.& t+e main conce,ts about information re!resentation and"uantification.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 4/76
#rere"uisites
No formal ,rere1uisites 2i$+ sc+ool ,rere1uisites "et t+eor- Combinatorial anal-sis Probabilit- and statistics 3o$arit+m4s ,ro,erties 0e5t inter,retation
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 5/76
$ssessments % Grade
$ssessments
Midterm E5am: 50& & November rd
6inal E5am: 50&& December 7t+
*eta8e E5am: December 9t+ 'e!ace the smaest of the t(o e)am *rades +!en to anyone (ho (ants to try to im!ro,e the fina*rade
ub.ect/ $ content
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 6/76
$ssessment % 1ina *rade
$ssessment 7& avera$e ; < =&> avera$e < 7& ; # ?&> avera$e < =&> ; C &> avera$e < ?&> ; D <vera$e &> ; 6
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 7/76
Tidia
+tt,:AAtidiaBae.ufabc.edu.br
3o$ in usin$ -our institutionalcredentials. 0+e ,rofessor illre$ister t+e students
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 8/76
Course materia•Course materia/ 2i be made a,aiabe at
Tidia 3htt!/tidia-aeufabcedubr6 on thesection 7'e!osit8rio 3'esources69
• Cass sides• :)ercise ists and ans(ers
•I;#+'T$NT/ study the materia <B:1+':< the cass
• Brin* your o(n "uestions to so,e them andfaciitate earnin*
• =+> are the main res!onsibe for your earnin*• The content is e)tensi,e/ study throu*hout thecourse 3not .ust before the e)ams6
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 9/76
:stimated chrono*ram
Content
22, 28/09 Course introduction, Data, Information, Knowledge,History of Information Theory, Semiótica
29/09, 06/10 Enumeration systems, base conversion, it, ooleanalgebra
13, 20/10 Codes, source coding, digital!analogic conversion
26/10, 27/10 Information Theory03/11 Midterm exam
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 10/76
Crono*rama estimado
Conteúdo09 e 10/11 Efficient coding and data com"ression
17 e 23/11 Error detection and correction
24/11 e01/12 #eural coding and genetic code
07/12 E$ercises and review class08/12 Final exam
15/12 Retake exam
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 11/76
*ecommended readin$:Decodin$ t+e niverse. C+arles "eife ('>>?) Pen$uin #oo8s.(Este livro, embora de caráter divulgativo, é o que captura melhor o foco multidisciplinar dadisciplina. Apresenta um roteiro que pode ser preenchido abordando cada tópico com uma
profundidade maior . Eiste somente um eemplar na biblioteca. Eiste vers!o em pdf"
"istemas Di$itais: fundamentos e a,licaFes. 0+omas 3.6lo-d. (Essencial para eplicar os sistemas de numera#!o, código de $amming, a
convers!o A%& e &%A e o teorema de amostragem de 'hannon)*quist. A álgebra booleanatambém está bem eplicada. Eistem vários eemplares na biblioteca"
"istemas de Comunica%o <nalG$icos e Di$itais. "imon2a-8in ('>>H). (A +eoria da nforma#!o, compress!o de dados e detec#!o%corre#!o
de erros"
<n ntroduction to nformation t+eor-. "-mbols& si$nals and
Noise. Jo+n *. Pierce. Dover. (Embora um pouco ultrapassado, apresentaos desafios encontrados pelos pioneiros da teoria de informa#!o. -ostra as solu#esencontradas para estes desafios"
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 12/76
Informationa $)is
0+e advances of science and tec+nolo$- aremulti,l-in$ our ca,acit- to collect& ,rocess& ,roduceand use information leadin$ it to ne levels neverreac+ed before. t brin$s:
ne o,,ortunities ne social 1uestions more science and tec+nolo$- advances& fosterin$ a
virtuous c-cle
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 13/76
Informationa $)is
1undamentas and !rocesses/ nformation Nature: +at is nformation and +o cane re,resent or measure itI
nformation Processin$: mani,ulation and informationtreatment& under bot+ +uman and com,uter as,ects(,rocessin$)
nformation communication: transmission anddistribution of t+e information and its im,act
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 14/76
Naturea danforma%o
0ransforma% oda
nforma%o
Comunica%oda
nforma%o
(conceitual)0eGrica
(su,orte)0ecnolo$ia
(utilia%o)2umana
<bstrata Concreta "ocial
➢K #it➢Entro,ia➢<nalo$. L Di$ital➢Ca,ac. "+annon
➢0. nforma%o
➢0. Com,uta%o➢Kr$. Com,utadores➢Com,ress%o Dados➢Cri,to$rafia➢Com,le5idade
➢0. ComunicaFes➢Ca,acidade canal➢Canal $aussiano➢nforma%o$entica➢Codifica%o
➢"mbolos e"inais➢*udo➢Proc. EstocOsticos
➢Krdem e Desordem➢Caos
➢"entidosA,erce,%o➢Co$ni%o e <%o➢nteli$ncia➢Conscincia
➢MemGria
➢<,rendiado➢Crebro➢Con+ecimento➢*a%oAEmo%o➢*edes "ociais
➢3in$ua$em2umana➢nternet➢"oc. nforma%o➢Econ. nforma%o➢*e$ula%oAQtica
➢Proc. "inais➢0ransformadas➢Pro$rama%o➢Minera%o Dados➢0radu%o
➢"ist.ComunicaFes➢*edes e 0rOfe$o➢EletrRnica➢6otRnica➢Novas 0ecnolo$ias
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 15/76
$mostras
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 16/76
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 17/76
S+at t+is course is reall- aboutI
% 0+e course Naturea da nforma%os+os +o t+e nformation is ,resent inour lives.
% Not just at t+e tec+nolo$ic level(telecommunications& com,uters&internet)
% #ut also at t+e biolo$ic (DN<& brain) and+uman (lan$ua$e& semiotics) levels
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 18/76
&ata, nformation and /no0ledge
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 19/76
Data
% Data: basic element that quantifies orqualifies something
& Do not carry any intrinsic meaning byitself & Initial perception about the subject
& Identified by symbolic characteristics
& Ex.: screw 6 weights 6! grams –
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 20/76
Data
% " set of facts about the world#% $hey are usually quantifiable#
% %an be easily captured and stored incomputational de&ices#% Do not carry meaning and neither
can be used in judgements#% 'o actions can(t be made based on just data.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 21/76
Data
% $ypes & "lphanumeric & )roduct code* price* amount* etc.
& Images & )hotographs
& "udio
& +ideo & etc...
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 22/76
Data
asic facts
E'g'( sho""ing at the su"ermar)et
*+ -,./eer 012.3
-+ .,//Cheese.404-
2+ 5,//6uice.2504
-+ 3,//utter 2.*4.
0+ -,./7il) 0*2.*
3+ 5,./Chocolate-35-2
Qantit!"ndi#idal $ri%e&e'%ri(tionCode
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 23/76
Information% Information: interpreted and
contextuali,ed data & -equires data interpretation & Is the result of processed data* it(s useful for
decisionma/ing.% It answers questions li/e 0who1* 0what1* 0where1 and
0when1.
& Ex.: screw 23 is the hea&iest of the group
&ata 1 processing 2 information
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 24/76
Information
% " conjugated data set that hasrele&ance and purpose#
% %an be transformed by humans and
be submitted to judgment & 'emiotics 4quality of information5 & Information analysis to produce
/nowledge% %onstitute base for action 4decision
ma/ing5
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 25/76
Information
Transformed data with aditional information
+ 24,//33Total+ 1,//*+ -,./eer 012.3
+ .,//-+ .,//Cheese.404-
+ -3,//2+ 5,//6uice.2504
+ 3,//-+ 3,//utter 2.*4.
+ -3,//0+ -,./7il) 0*2.*
+ 4,//3+ 5,./Chocolate-35-2
)otal $ri%e Qantit!"ndi#idal $ri%e&e'%ri(tionCode
&air* products sold
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 26/76
Information
Can hel" to increase the "rofit
+ 24,//33Total+ 1,//*+ -,./Cerve8a012.3
+ .,//-+ .,//9uei8o.404-
+ -3,//2+ 5,//Suco.2504
+ 3,//-+ 3,//7anteiga2.*4.
+ -3,//0+ -,./:eite0*2.*
+ 4,//3+ 5,./Chocolate-35-2
$re*o )otalQantidade$re*o "ndi#idal&e'%ri*+oCdi-o
3everage products sold
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 27/76
nowledge% nowledge: ability to create a
model and suggest actions ordecisions to ta/e & %omprehension* analysis and synthesis
start at the /nowledge le&el & -equired le&el to ta/e smart decisions
% -esponde quest7es do tipo 0como1
& Ex.: 8hea&ier( definition* comparison rules*procedures 4models5
nformation 1 processing 2 4no0ledge(processing 2 eperience, training, etc."
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 28/76
nowledge
% " structured and organi,ed set ofinformation#
% -equires intelligent human judgement
& 'emiotics: assignment of meaning toinformation#
% 9ffers rules* comparisons*deductions and implications
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 29/76
nowledge
% Ex. supermar/et: & $he items and ; are frequently bought
together & <il/ and chocolate & 'o put them near each other
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 30/76
=ierarchy
+his hierarch* represents the sequence bet0een the elements, but
also the volume of each one
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 31/76
Data x Information x
nowledge
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 32/76
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 33/76
Data x Information x
nowledge
•&ata are ra0 numbers or labels.• A grid segmentation could produce information (e.g. more or less
populous segments" about these ra0 data.•5ules that describe 0hen a segment is red or green, for eample, are
part of 4no0ledge
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 34/76
6hat is information7
• Latin etimology
• Different perspecti&es
8 >iology 8 ?inguistics 8 )hysics 8 %omputer 'cience
What is the relationship between information andcommunication? 8 Communication: information exchange between the
actors
from in "into", 1 formare 9to form, shape:
(into form, delineate"
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 35/76
• Information is inversely proportional to
the probability of occurrence of a fact 8 =igher probability facts are less informati&e 8 Examples: 8 $he first time we listen a disc* it brings new musical
/nowledge. >ut after listen to it many times* we can
predict the next accords* so this disc brings no moreinformation to us.
8 @e all can predict the missing 091 at the word 0?A+E1*because it(s a common word. 'o it is unnecessary towrite that character at that position.
8 Boal: to quantify the information. $he meaningor 0quality1 is irrele&ant a priori.
8 $he information quality or meaning is ofinterest to semiotics
Information $heory
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 36/76
$istor* of nformation +heor*
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 37/76
;laude E. 'hannon (=>?@@" is the 9father: of nformation +heor*. $is
boo4 9+he -athematical +heor* of ;ommunication: 0as published in ==
e seu livro publicado em ==
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 38/76
• Em =B, 'hannon published the paper 9 A Mathematical
Theory of Communication:, republished as boo4 net
*ear
• 3efore him, isolated 0or4s 0ent step b* step to0ards ageneral theor* of communication
• )o0ada*s the +heor* of ;ommunication (or nformation
+heor*" is a huge research area 0ith man* boo4s ands*mposiums about the subCect
Shannon
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 39/76
• nformation +heor* is a broad theor* that involves a lot of
math.
• Bit is the fundamental measure to quantif* information
• At DA3; there is a specific course about nformation
+heor*
Information Theory
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 40/76
• nformation theor* allo0s us toF
• 'a* ho0 man* informative bits could be sent per second through
a certain communication channel
• -easure the rate in 0hich a source can produce information
• 'a* ho0 to represent efficiently, or encode, messages to be
transmitted through some channel .
• 'a* ho0 0e can avoid transmission errors
Information Theory
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 41/76
$he pilars of Information $heory
C $hermodynamic studies
C
%ryptography and computers built atthe @orld @ar II
C $ransmission technology* starting with<orse code and telephony
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 42/76
Entropy and thermodynamics
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 43/76
>olt,mann* the father ofthermodynamics and information theory
• Entropy equation inscribed at>olt,mann(s tomb
• $he bases of Information $heory are pro&ided by
thermodynamic studies >ut >olt,mann didn(t /now
that his studies would be thebases of Information $heory
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 44/76
Industrial -e&olution
• F6G. @attin&ented the
steammachine• -esearch to
impro&e the
machineefficiency
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 45/76
%arnot(s =eat engine
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 46/76
%arnot(s =eat engine
%oldsource
wor/=eat
Engine
=ot source
G
G?
=ot source
%oldsource
=eat'ource
G
G?
wor
Refrigeration
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 47/76
-udolf %laussius
• It is impossible tostop the tendencyto thermodynamicequilibrium of theuni&erse 4H6!5
• In other words* 0the
entropy alwaystends to increase1
mposs e to stop t e
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 48/76
mposs e to stop t etendency to thermodynamic
equilibrium %old source
=ot source
@or/=eat
Engine
G
G?
=eatsource
G
G?
@or/=eat
Engine
G
G?
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 49/76
II $hermodynamics law
• Entropy measures thethermodynamics equilibrium
• <ore entropy means more
equilibrium• $he second thermodynamics law
says that 0the entropy amount of any
system tends to increase with timeuntil its maximum &alue1
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 50/76
'/ log4@5
• Entropy equationinscribed at >olt,mannJstomb
• "toms of a gas tend to
disperse itself uniformly.• $he uni&erse entropy
always increases
• ?ater in the course wewill see the 0bridge1between the entropies of>olt,mann and 'hannon
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 51/76
%ryptography and
%omputers at @orld @ar II
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 52/76
0"K is short of water1
• $hese wordschanged thecourse of the
war at )acific.• $he
cryptographic
japanese code4LM235 wasdecoded by
americans
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 53/76
0?ets attac/ "KN1
%ommander -ochefort"dmiral ;amamoto
Hets attac4 AI A777 6hat is A777
6ould A be -id0a* sland7
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 54/76
0<idway Island is out of waterN1
%ommander -ochefort "dmiral ;amamoto
-id0a* sland is out of 0aterI A is out of 0aterI
A$AIII got *ouIII
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 55/76
End of @ar
• "mericans waitedfor the japanesearri&al
• Kour Lapaneseaircraft carrierswere destroyed.
• $he end of @orld@ar II began
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 56/76
European 0Oboats1
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 57/76
Enigma• $he german "rthur
'cherbius in&ented0Enigma1* a machine
to encrypt messages• x!P differentstates possible
• >rute force would
require: each atomon uni&ersecomputing a trillionof /eys per second*since the beginningof uni&erse
$uring and colleagues bro/e
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 58/76
$uring and colleagues bro/ethe Enigma code
• "ll characters wereswapped at each time
• $he corresponding
symbol to character0K1 was ne&er 0K1 itself
• Osual sentences 4e.g.
the weather is goodtoday5 allowed thedestruction of Oboatsand the end of war.
$h h i l $ i
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 59/76
$he theoretical $uring<achine
• $he machine reads*writes and deletesbits in an endlessstring
• $he $uring <achinehas 0uni&ersal
computability1• I.e. any current
computer is a $uring
<achine
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 60/76
$he allied cryptography helpedto end the @orld @ar II andstarted the Information Era
$uring
contributed tothe end of @orld@ar II
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 61/76
%ryptography
• " simple cryptographic method is toreplace each letter 4character5 by theletter placed M positions forward in
the alphabet• %aesar cipher
• $ry to decode the following message:
Q xgp&q jqlg gu&cxc ow/&q hqtg9 &ento hoje esta&a muito forte
n 2
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 62/76
-edundancy
• Bs TvBnBs dB cBBncBB e dB tBcnBlB$BB BstTBmBltB,lBcBndo Bs nBssBs cB,BcBdBdBs dB
cBlBtBr& trBtBr& $BrBr B BtBlBBrBnfBrmBBes.
9 b i / d f d d
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 63/76
9ur brain ta/es ad&antage of redundancy todecode texts as the following:
• De aorcdo com uma peqsiusa de umauinr&esriddae ignlsea* nRo ipomtra emqaul odrem as ?teras de uma plra&aa
etRso* a Sncia csioa iprotmatne T que apiremria e Stmlia ?teras etejasm no lgaurcrteo. 9 rseto pdoe ser uma bUguana ttaol*que &coV anida pdoe ler sem pobrlmea.
Itso T poqrue nWs nRo lmeos cdaa ?teraisladoa* mas a plra&aa cmoo um tdoo.'ohw de bloa.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 64/76
9r li/e this one...
• 3$ )QOM! $$9 3-+ P)MP3 )P-P<!3$-P- %9<9 M933P %P>XP%9M'BO KPY- %93P3 <)-339MP"M$3N-)P- M339N M9 %9<X9 3$P+P
<! %9<)?%PD9* <P3 M'$P ?M=P 'OP<M$ +P D%K-PMD9 9 %ZDB9QOP3 PO$9<P$%P<M$* '< )-%'P-)M3P- <O$9* %-$9[ )9D K%P-
>< 9-BO?=939 D339N 'OP %P)P%DPD<-%N )P-P>\M3N
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 65/76
$echnologies forInformation $ransmission
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 66/76
!rigins of the modern
Information Theory+here are mathematical analogies bet0een the
nformation entrop* and the entropies of
+hermod*namic and 'tatistical -echanic, but
the modern nformation +heor* are rooted at the
origins of electronic communication
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 67/76
Telegraph origins
• BJBF 'amuel 3. -orse 0or4ed 0ith Alfred Kail on acode 4no0n toda* as the -orse codeF• Alphabetic characters are represented b* spaces, dots anddashes
•Electronic transmission 0as achieved representing spaces b*current absence, dots b* short currents, and dashes b* longcurrents
• ;ombinations of these s*mbols 0ere associated 0ith characters
• E (the most frequent character in English" 0as associated to a
single dot.
<orse %ode
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 68/76
<orse %ode
The "orse code and
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 69/76
The "orse code andInformation Theory
• mportant questionF• ;ould another mapping (dots, dashes, spaces" impl* to faster
telegraphic transmissions of English tets7
• Ans0erF• Dsing the modern nformation +heor* 0e found that the
transmission rate gain 0ould be about LM at most.
• +his suggests that -orse intuitivel* attac4ed on of the mainproblems addressed b* nformation +heor*.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 70/76
• Himitations summar*• Himits related to the signal transmission speed
• nterferences (noises"
• $ard to distinguish among man* possible current values
• Himited current intensit* to avoid destruction of the cableinsulation
• A more precise mathematical anal*sis 0as required
Telegraph limitations
ib i h
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 71/76
Contributions to theInformation Theory
• -an* people contributed mathematicall* to the informationtheor* at NN centur*F• Horde /elvin (6illiam +homson"
• Aleander Oraham 3ell (telephone inventor in BPL"
• $enri Qoincaré• Rliver $eaviside
• -ichael Qupin
• O. A. ;ampbell (A++;"
• 3ut the biggest contribution 0as obtained b* Sosephourier
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 72/76
#ourier$s contribution
• ourier based his 0or4s on the sine function
•'ho0ed that an* function (including electric signals"
can be decomposed in a sum of sines 0ith differentamplitudes, phases and frequencies.
% l ff$ d Wi $
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 73/76
%olmogoroff$s and Wiener$scontributions
n the =@s 5ussians and Americans solved
independentl* the problem of estimating the
correct signal from an un4no0n nois* signal.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 74/76
#inally Shannon
• 'o 0hen 'hannon published his 0or4s in =P,
much 0as alread* done
• n a certain 0a*, he summariTed and producedne0 4no0ledge about all these problems
previousl* studied b* other researchers
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 75/76
Shannon$s contribution
• 3ut 0e could sa* that his great contribution 0as to ans0erthe follo0ing questionsF• $o0 could 0e encode (0ith electric signals" a message from a
source to transmit it as fast as possible through a channel that
introduces noises 0ith a certain pattern7• How fast could 0e transmit certain message through specific
channels without errors7
• All 0e have seen until no0 is part of 0hat 0e call
nformation +heor*. n this course 0e 0ill see introductor*topics about this theor*.
8/17/2019 NI Introduction
http://slidepdf.com/reader/full/ni-introduction 76/76
To do before ne&t class
• 'tud* the slides and ans0er the eercise listcorresponding to the first 0ee4 ('emiotics"
•'ection U5epositórioV (5esources" at +idia
• )o need to deliver the list for grading, but it is
fundamental to do the eercises and stud*
before classes
● and consequentl* for the eams