info introduction to informatics ?· uptown funk you up, uptown funk you up come on, dance jump on...

Download Info introduction to informatics ?· Uptown Funk you up, Uptown Funk you up Come on, dance Jump on it…

Post on 25-Jul-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    Informatics luis rocha 2017

    I501introduction

    to informatics

    introduction to informaticslecture 5

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    Readings until now

    This week Young, T. (2017). Cease and desist. Nature, 541(7637), 430

    430. Lazebnik, Y [2002]. "Can a biologist fix a radio?--Or, what I

    learned while studying apoptosis". Cancer Cell, 2(3):179-182. Optional

    Presentations G.L. Ciampaglia, P. Shiralkar, L.M. Rocha, J. Bollen, F. Menczer,

    A. Flammini [2015]. Computational fact checking from knowledge networks. PLoS One. 10(6): e0128193. Russo, Giampaolo

    Piantadosi, S. T.,et al (2011). Word lengths are optimized for efficient communication. PNAS, 108(9), 35263529. Malic, Vincent

    Lecture Notes The Nature of Information Formalizing and Modeling the World

    Available http://canvas.iu.edu and listed at

    http://informatics.indiana.edu/rocha/academics/i501 Also check out

    Links and notes at http://sciber.blogspot.com/

    introduction to informatics

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    cybernetics

    Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology Engineering-inspired Supremacy of mechanism

    All can be axiomatized and computed Mculloch&Pitts work was major influence

    A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5:115-133 (1943).

    A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present)

    post-war science

    Macy Conferences: 1946-53

    Warren S. McCulloch

    Claude ShannonMargaret Mead

    Heinz Von Foerster

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    Informatics luis rocha 2017

    I501introduction

    to informatics

    design principles of computationBabbage/Lovelace, Turings tape, and roles of information

    distinction between numbers that mean things and numbers that do things.

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    At every discrete time instance the machine is in a single state

    Program is a state transition table

    state Read symbolNext state

    Write symbol

    Tape move

    0 4 1 - left

    1 - 0 1 right

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    Informatics luis rocha 2017

    I501introduction

    to informatics

    Information as decrease in uncertainty . uncertainty-based information

    C. E. Shannon, A mathematical theory of communication. Bell System Technical Journal, 27:379-423 and 623-656

    Hartley, R.V.L., "Transmission of Information", Bell System Technical Journal, July 1928, p.535.

    AAH 2log)( AAH 2log)(

    Number of ChoicesMeasured in bits

    in

    iiS xpxpAH 2

    1log)(

    in

    iiS xpxpAH 2

    1log)(

    Probability of alternativeMeasured in bits

    information is surpriseinformation is surprise

    including more structure reduces

    surprise

    including more structure reduces

    surprise

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    uncertainty

    Mutual Information Amount of information about one variable that can be gained

    (uncertainty reduced) by observing another variable Information Gain (Kullback-Leibler Divergence)

    Difference between two probability distributions p and q, average number of bits per data point needed in order to represent q

    (model approximation) as it deviates from p (true or theoretical distribution)

    Other measures

    n

    i

    m

    j ji

    jiji ypxp

    yxpyxpYXI

    1 12

    ,log,);(

    n

    i

    m

    j ji

    jiji ypxp

    yxpyxpYXI

    1 12

    ,log,);(

    ),()()();( YXHYHXHYXI ),()()();( YXHYHXHYXI

    n

    i i

    ii xq

    xpxpXqXpIG1

    2log))(),((

    n

    i i

    ii xq

    xpxpXqXpIG1

    2log))(),((

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    Informatics luis rocha 2017

    I501introduction

    to informatics

    Do, do, do, do, do, do, do, do, doBefore we leave

    Lemme tell yall a lil somethingUptown Funk you up, Uptown Funk you up

    Come on, danceJump on it

    If you sexy, than flaunt itIf you freaky, than own it

    Dont brag about it, come show meCome on, dance

    Jump on itIf you sexy, than flaunt it

    Well, its Saturday night and we in the spot

    Dont believe me, just watchUptown Funk you up, Uptown Funk you up (say whaa?)

    Uptown Funk you up, Uptown Funk you up

    information of sequential messagesrate of removing uncertainty of each symbol

    syntactic surprise But what about

    function and meaning (semantics)?

    syntactic surprise But what about

    function and meaning (semantics)?

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    Shannons mouse

    trial and error algorithm information as reduction of uncertainty in the

    presence of alternatives (combinatorics) lifelike behavior

    trial and error to learn path from many alternatives adapts to new situations

    how is learning achieved? Correct choices, information gained from reduced

    uncertainty, must be stored in memory memory of information as a design principle

    of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays

    Brain as (switching) machine

    controlling information to achieve life-like behavior

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    formalizing knowledge Lord Kelvin's dictum

    When you can measure what you are speaking of and express it in numbers you know that on which you are discoursing. But if you cannot measure it and express it in numbers. your knowledge is of a very meagre and unsatisfactory kind. 1824-1907

    Absolute scale of temperature, underwater telegraph cables, thermodynamics

    Physics The first science to construct precise, rigorous

    formal theories of the world. relating the operation of rules upon symbols to the

    law-like behavior of the World. Aristotle (384-322 BC) was first to relate symbols

    more explicitly to the external world and to successively clarify the nature of the symbol-world relation.

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    understanding Nature with symbols Aristotle (384-322 BC)

    First to relate symbols more explicitly to the external world and to successively clarify the nature of the symbol-world relation. Student of Plato, educated Alexander the Great first to consider specific observable factors which

    determine motion. In Physics

    he recognized (mathematical) rules which could describe the relation between an object's weight, the medium's density and the consequent rate of motion (fall): (1) for freely falling or freely rising bodies, speed is

    proportional to the density of the medium. (2) in forced motion, speed is proportional to the force

    applied and inversely proportional to the mass of the body moved

    first time that observable quantities had been expressed in symbolic (numerical) form allowing the results of observations to be used in calculations The nature of causation http://classics.mit.edu/Aristotle/physics.html

    Raphaels Plato and Aristotle

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    abstracting relations Galileo (1564-1642)

    Progressive dissociation of the symbols from objects The interrelationships among signs themselves

    studied quite apart from the relations among the objects they represent Previously, symbols were still generally regarded as

    inherent properties of the referent objects themselves Aristotles Physics postulated certain primary

    qualities/elements such as Fire. Galileo regards "primary" properties as only those that can be mathematically quantified, such as size, shape and motion.

    Newton (1643-1727) Extends process of abstraction

    Distinguishes between symbols Arising from observation

    represent initial conditions Arising from symbol relations

    representing laws which govern the subsequent motion.

  • rocha@indiana.eduinformatics.indiana.edu/rocha/academics/i501/

    INDIANAUNIVERSITY

    I501introduction

    to informatics

    Informatics luis rocha 2017

    Heinrich Hertz (1857-1894) Some facts about Hertz

    First to broadcast and receive radio waves Established that light is a form of

    electromagnetic radiation. His name is associated with the SI unit for

    frequenc

Recommended

View more >