introduction to computational intelligence · pdf fileintroduction to computational...

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016

INTRODUCTION TOCOMPUTATIONAL INTELLIGENCE

Lin ShangDept. of Computer Science and Technology

Nanjing University

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016

Introduction to Computational Intelligence

n From AI to CIn What is intelligence ?n What is AI ?n What is CI ?

n Different views of Computational Intelligence n History of Computational Intelligence n Computational Intelligence Paradigmsn Conferences and Journals

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016

From AI to CI

n What is intelligence?


n Characteristics of Intelligence

n Perception

n Action

n Reasoning

n Problem-solving

n LearningandAdaptation

n Sociality

n Creativity

From AI to CI


From AI to CIn What is AI ?

n Philosophy Logic, methods of reasoning, mind as physical system, foundations of learning, language, rationality

n Mathematics Formal representation and proof algorithms, computation,(un)decidability (in)tractability, probability

n Economics utility, decision theory

n Neuroscience physical substrate for mental activity

n Psychology phenomena of perception and motor control, experimental techniques

n Computer building fast computers Science

n Control theory design systems that maximize an objective function over time

n Linguistics knowledge representation, grammar

AI prehistory

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 201616/2/29 AI:Introduction 6

From AI to CIAbridged history of AI

n 1943 McCulloch & Pitts: Boolean circuit model of brainn 1950 Turing's "Computing Machinery and Intelligence"n 1956 Dartmouth meeting: "Artificial Intelligence" adoptedn 1952-69 Great expectations n 1950s Early AI programs, including Samuel's checkers program, Newell & Simon's

Logic Theorist, Gelernter's Geometry Enginen 1965 Robinson's complete algorithm for logical reasoningn 1966-73 AI discovers computational complexity, Neural network research almost

disappearsn 1969-79 Early development of knowledge-based systemsn 1980- AI becomes an industry n 1986- Neural networks return to popularityn 1990- Novelle AI and intelligent agents n 1995- Evolutionary Computation, Swarm Intelligence, and New Generation

Computers


1943:McCulloch & Pitts: Boolean circuit model of brain

MP


1950: Turing test

n When does a system behave intelligently?

n Turing (1950) Computing Machinery and Intelligence

n Operational test of intelligence: imitation game

n Requires the collaboration of major components of AI: knowledge, reasoning, language understanding, learning, …


2011:Cleverbot


1956:Dartmouth meeting: "Artificial Intelligence" adopted


Ideas for AIn Learning

n “child machine”

n Symbolic AI

n Connectionism

n Nouvelle AI

n Evolutionary Computation

n “artificial life”

n Computational Swarm Intelligence


Symbolic AIn Physical Symbol System Hypothesis of Newell and Simon

n the processing of structures of symbols by a digital computer is

sufficient to produce artificial intelligence

n the processing of structures of symbols by the human brain is

the basis of human intelligence

n it remains an open question whether the Physical Symbol System

Hypothesis is true or false

n Top-down strategy


n Problem-sloving à Expert System à Knowledge Engineering- Search, Representation, Reasoning

n Problems- Frame problem (CYC, Go…..)- Substituting large amounts of computation for understanding


Connectionismn The mechanisms of brains are very different in

detail from those in computersn how brains work? à Bottom-up strategy

Natural Neural Network


n A brief historyM-P neuron (McCulloch & Pitts) à Perceptron

(Rosenblatt) à Hopfield Model, B-P Learning Method (Rumelhart & McClelland) à…Deep Learning

n ApplicationsRecognition, Vision, Business, Medical, …….

Connectionism


Evolutionary Computation

n Biological evolutionTo produce an enormous variety of living organisms closely suited to different sets of needs in different environments.

n Simulated evolutionBy modeling those processes of biological evolution on computers, it turns out that we can sometimes get the computers to evolve solutions to problems.


n Genetic AlgorithmUse strings of symbols to encode solutions to problems,like strings of molecules in DNA. Transforming andrecombining portions of strings enables an evolutionarycomputation to search for good solutions, partly analogousto biological evolution.

n Genetic ProgrammingExtends these ideas to automatic programming by usingstructures which are better suited to the problem thanstrings are.


n Evolutionary StrategyUse natural problem-dependent representations, and primarily mutationand selection as search operators. Mutation is normally performed byadding a normally distributed random value to each vector component.The step size or mutation strength is often governed by self-adaptation. The selection in evolution strategies is deterministic andonly based on the fitness rankings, not on the actual fitness values.

n Evolutionary ProgrammingHarder to distinguish from evolutionary strategies. Its main variation

operator is mutation; members of the population are viewed as partof a specific species rather than members of the same speciestherefore each parent generates an offspring


Artificial Life (Alife)n Artificial Life is the study of man-made systems that

exhibit behaviors characteristic of natural living systems. It complements the traditional biological sciences concerned with the analysis of living organisms by attempting to synthesize life-like behaviors within computers and other artificial media. By extending the empirical foundation upon which biology is based beyond the carbon-chain life that has evolved on Earth, Artificial Life can contribute to theoretical biology by locating life-as-we-know-it within the larger picture of life-as-it-could-be."

Chris Langton (in Proc. of first Alife conference)

Ref:http://www.cogs.susx.ac.uk/users/inmanh/easy/alife09/lectures.html


Artificial Life and Evolutionary

Origin of Life

Today

Life, and might have beenas it is…

FromVirgilGriffith,GoogleTechTalk- 2007


Example: Forming body plans with evolution

n Node specifies part type, joint, and range of movement

n Edges specify the joints between parts

n Population?

n Graphs of nodes and edges

n Selection?

n Ability to perform some task (walking, jumping, etc.)

n Mutation?

n Node types change/new nodes grafted on


Computational Swarm Intelligence n Intelligence is often considered a property of individuals.n Are we social because we are intelligent or are we

intelligent because we are social?- Intelligence can emerge from social interaction.

n Emergent behaviour – when a group behaves in ways that were not ”programmed” into its members.

n Swarm intelligence- simulated social interaction- emergent collective intelligence of groups of simple agents


Computational Tools

n Multi-Agent Systems- a system composed of multiple interacting intelligent agents. - application including computer games, networks, transportation, logistics, and etc.

n Ant Colony Optimization- 1991 (Dorigo)- mostly for combinatorial optimization

n Particle Swarm Optimization- 1995 (Kennedy & Eberhart)- more general optimization technique


Fuzzy Systems

n Zadeh 1965 Fuzzy Sets

n Fuzzy Logic

n Fuzzy Systems

n Successfully applied to control systems, home appliances,…


Computational Intelligence

•CI has been based roughly on Darwinian theory and biological analogies.(….)

•Artificial Intelligence -----Computational Intelligence

•The following chapters go into this in more detail(Neural Network, Evolutionary Computing, Fuzzy Systems)


History of Computational Intelligence

n First use of the term (in its current context) by James Bezdek in 1992

n First IEEE World Congress on Computational Intelligence in Orlando in 1994

n First CI text in 1996n Second IEEE World Congress on CI in Anchorage in 1998n Subsequent World Congresses in Hawaii (2002),

Vancouver (2006)…next in Hong Kong (2008)…Beijing (2014)


Historical View of Computational Intelligence

n “Computational Intelligence” was used in the title of a journal in Canada starting in 1980s, but not meaning what we now mean by the term

n First paper using term published by Bezdek in 1992 in Int. Jour. Approximate Reasoning.


Bezdek’s 1994 Definition of CI A system is computationally intelligent when it: deals only with numerical (low-level) data, has a pattern recognition component, does not use knowledge in the AI sense; and additionally, when it (begins to) exhibit (i) computational adaptivity; (ii) computational fault tolerance; (iii) speed approaching human-like turnaround, and (iv) error rates that approximate human performance.


Bezdek’s 1994 Definition of CI

Bezdek’s 1994 Definition of CI


Historyn 1975 Genetic Algorithm by Holland

n 1977 Tabu Search by Glover

n 1983 Simulated Annealing by Kirkpatrick

n 1992 Ant Colony Optimization by Dorigo

n 1995 Particle Swar Optimizationby Kennedy and Eberhart

n 1999 Predatory Search by Linhares

n 2001 Bacterial Foraging Algorithm by Passino

www.themegallery.com



n Fuzzy Computing (FC)

n Neural Computing(NC)

n Evolutionary Computing(EC)

n Probabilistic Computing(PC)

n Granular Computing(GrC)

n Rough Computing(RC)

n …

Soft Computing


Wecansay?

n “CI is a subset of AI”

n “CI is not a subset of AI, there is an overlap between AI

and CI”

n In general, CI ≠AI

n Motivation: “CI”à?


Conferences and Publicationsn IEEE Computational Intelligence Society

http://ieee-cis.orgn Publications

n 6 transactionsIEEE Transactions on Neural NetworksIEEE Transactions on Fuzzy SystemsIEEE Transactions on Evolutionary ComputationIEEE Transactions on Autonomous Mental DevelopmentIEEE Transactions on Computational Intelligence and AI in GamesIEEE/ACM Transactions on Computational Biology and BioinformaticsIEEE Transactions on Information Forensics and Securityn 1 magazinesIEEE Computation Intelligence Magazine


IEEE Computational Intelligence Society

Prof. Gary G.YenPresident


Publications（IEEE Transactions on Neural Networks）

n science and technology of neural networks

n disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

n Emphasis is on artificial neural networks.



Publications（IEEE Transactions on Fuzzy Systems）

n the theory, design and applications of fuzzy systems

n ranging form hardware to software.

n Emphasis will be given to engineering applications.


n the theory, design and applications of evolutionary computation

n emphasis given to engineering systems and scientific applications

n evolutionary optimization, machine learning, intelligent systems design, image processing and machine vision, pattern recognition, evolutionary neurocomputing, evolutionary fuzzy systems, applications in biomedicine and biochemistry, robotics and control, mathematical modelling, civil, chemecal, aeronautical, and industrial engineering applications

Publications（IEEE Transactions on Fuzzy Systems）


n co-sponsored by the IEEE Computational Intelligence Society and the IEEE Consumer Electronics Society

n technically co-sponsored by the IEEE Computer Society, the IEEE Engineering in Medicine and Biology Society, and the IEEE Robotics and Automation Society

Publications（IEEE Transactions on Autonomous

Mental Development


n co-sponsored by IEEE CI Society the IEEE Computer Society, the IEEE Consumer Electronics Society and the IEEE Sensors Council


Publications（IEEE Transactions on Computational

Intelligence and AI in Games)


Publications（IEEE/ACM Transactions on

Computational Biology and Bioinformatics)

n the algorithmic, mathematical, statistical, and computational methods that are central to bioinformatics and computational biology

n the development and testing of optimization of biological databases

n important biological results that are obtained from the use of these method, programs, and databases



n aims to provide a unified focus for archival research on the fundamental contributions and the mathematics behind information forensics, information security, surveillance

n system applications that incorporate these features


Publications（IEEE Transactions on on Information

Forensics and Security)


n aims to introduce the readers to all areas of computational intelligence design and applications

n specific emphasis on applications oriented developments, successful industrial implementations, design tools, technology reviews, computational intelligence education, and applied research


Publications（IEEE Computation Intelligence Magazine)


WCCI(World Conference on CI)



CEC (IEEE Congress on Evolutionay Compuation)


n CEC 2013, Cancun, Mexico, 20-23 June

n CEC 2011, New Orleans USA, 5-8 June

n CEC 2009, Trondheim Norway, 18-21 May

n CEC 2007, Singapore, 23-26, July

n CEC 2005, Edinburgh U.K.,2-5 Sept.


People: Lotfi A. Zadeh



People: Lawrence J. Fogel



People: John Henry Holland



People: Marco Dorigo



People: Yao Xin(姚新)



People: Eberhart Russell



Conclusion

n From AI to CI

n NN, EC and FS

n History

n Conferences and journals

introduction to computational intelligence · pdf fileintroduction to computational...

Documents