# info introduction to informatics ?· uptown funk you up, uptown funk you up come on, dance jump on...

Post on 25-Jul-2018

212 views

Embed Size (px)

TRANSCRIPT

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

introduction to informaticslecture 5

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

Readings until now

This week Young, T. (2017). Cease and desist. Nature, 541(7637), 430

430. Lazebnik, Y [2002]. "Can a biologist fix a radio?--Or, what I

learned while studying apoptosis". Cancer Cell, 2(3):179-182. Optional

Presentations G.L. Ciampaglia, P. Shiralkar, L.M. Rocha, J. Bollen, F. Menczer,

A. Flammini [2015]. Computational fact checking from knowledge networks. PLoS One. 10(6): e0128193. Russo, Giampaolo

Piantadosi, S. T.,et al (2011). Word lengths are optimized for efficient communication. PNAS, 108(9), 35263529. Malic, Vincent

Lecture Notes The Nature of Information Formalizing and Modeling the World

Available http://canvas.iu.edu and listed at

http://informatics.indiana.edu/rocha/academics/i501 Also check out

Links and notes at http://sciber.blogspot.com/

introduction to informatics

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

cybernetics

Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology Engineering-inspired Supremacy of mechanism

All can be axiomatized and computed Mculloch&Pitts work was major influence

A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5:115-133 (1943).

A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present)

post-war science

Macy Conferences: 1946-53

Warren S. McCulloch

Claude ShannonMargaret Mead

Heinz Von Foerster

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

design principles of computationBabbage/Lovelace, Turings tape, and roles of information

distinction between numbers that mean things and numbers that do things.

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

At every discrete time instance the machine is in a single state

Program is a state transition table

state Read symbolNext state

Write symbol

Tape move

0 4 1 - left

1 - 0 1 right

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

Information as decrease in uncertainty . uncertainty-based information

C. E. Shannon, A mathematical theory of communication. Bell System Technical Journal, 27:379-423 and 623-656

Hartley, R.V.L., "Transmission of Information", Bell System Technical Journal, July 1928, p.535.

AAH 2log)( AAH 2log)(

Number of ChoicesMeasured in bits

in

iiS xpxpAH 2

1log)(

in

iiS xpxpAH 2

1log)(

Probability of alternativeMeasured in bits

information is surpriseinformation is surprise

including more structure reduces

surprise

including more structure reduces

surprise

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

uncertainty

Mutual Information Amount of information about one variable that can be gained

(uncertainty reduced) by observing another variable Information Gain (Kullback-Leibler Divergence)

Difference between two probability distributions p and q, average number of bits per data point needed in order to represent q

(model approximation) as it deviates from p (true or theoretical distribution)

Other measures

n

i

m

j ji

jiji ypxp

yxpyxpYXI

1 12

,log,);(

n

i

m

j ji

jiji ypxp

yxpyxpYXI

1 12

,log,);(

),()()();( YXHYHXHYXI ),()()();( YXHYHXHYXI

n

i i

ii xq

xpxpXqXpIG1

2log))(),((

n

i i

ii xq

xpxpXqXpIG1

2log))(),((

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

Do, do, do, do, do, do, do, do, doBefore we leave

Lemme tell yall a lil somethingUptown Funk you up, Uptown Funk you up

Come on, danceJump on it

If you sexy, than flaunt itIf you freaky, than own it

Dont brag about it, come show meCome on, dance

Jump on itIf you sexy, than flaunt it

Well, its Saturday night and we in the spot

Dont believe me, just watchUptown Funk you up, Uptown Funk you up (say whaa?)

Uptown Funk you up, Uptown Funk you up

information of sequential messagesrate of removing uncertainty of each symbol

syntactic surprise But what about

function and meaning (semantics)?

syntactic surprise But what about

function and meaning (semantics)?

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

Shannons mouse

trial and error algorithm information as reduction of uncertainty in the

presence of alternatives (combinatorics) lifelike behavior

trial and error to learn path from many alternatives adapts to new situations

how is learning achieved? Correct choices, information gained from reduced

uncertainty, must be stored in memory memory of information as a design principle

of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays

Brain as (switching) machine

controlling information to achieve life-like behavior

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

formalizing knowledge Lord Kelvin's dictum

When you can measure what you are speaking of and express it in numbers you know that on which you are discoursing. But if you cannot measure it and express it in numbers. your knowledge is of a very meagre and unsatisfactory kind. 1824-1907

Absolute scale of temperature, underwater telegraph cables, thermodynamics

Physics The first science to construct precise, rigorous

formal theories of the world. relating the operation of rules upon symbols to the

law-like behavior of the World. Aristotle (384-322 BC) was first to relate symbols

more explicitly to the external world and to successively clarify the nature of the symbol-world relation.

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

understanding Nature with symbols Aristotle (384-322 BC)

First to relate symbols more explicitly to the external world and to successively clarify the nature of the symbol-world relation. Student of Plato, educated Alexander the Great first to consider specific observable factors which

determine motion. In Physics

he recognized (mathematical) rules which could describe the relation between an object's weight, the medium's density and the consequent rate of motion (fall): (1) for freely falling or freely rising bodies, speed is

proportional to the density of the medium. (2) in forced motion, speed is proportional to the force

applied and inversely proportional to the mass of the body moved

first time that observable quantities had been expressed in symbolic (numerical) form allowing the results of observations to be used in calculations The nature of causation http://classics.mit.edu/Aristotle/physics.html

Raphaels Plato and Aristotle

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

abstracting relations Galileo (1564-1642)

Progressive dissociation of the symbols from objects The interrelationships among signs themselves

studied quite apart from the relations among the objects they represent Previously, symbols were still generally regarded as

inherent properties of the referent objects themselves Aristotles Physics postulated certain primary

qualities/elements such as Fire. Galileo regards "primary" properties as only those that can be mathematically quantified, such as size, shape and motion.

Newton (1643-1727) Extends process of abstraction

Distinguishes between symbols Arising from observation

represent initial conditions Arising from symbol relations

representing laws which govern the subsequent motion.

rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017

Heinrich Hertz (1857-1894) Some facts about Hertz

First to broadcast and receive radio waves Established that light is a form of

electromagnetic radiation. His name is associated with the SI unit for

frequenc

Recommended