computational intelligence: introduction

37
COMPUTATIONAL INTELLIGENCE: INTRODUCTION Ranga Rodrigo January 27, 2014 1

Upload: neo

Post on 24-Feb-2016

75 views

Category:

Documents


2 download

DESCRIPTION

Computational Intelligence: Introduction. Ranga Rodrigo January 27, 2014. Introduction to CI. Natural Systems. We have built algorithmic models, by studying natural systems. Artificial Intelligence (AI). - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Computational Intelligence: Introduction

1

COMPUTATIONAL INTELLIGENCE: INTRODUCTION

Ranga RodrigoJanuary 27, 2014

Page 2: Computational Intelligence: Introduction

2

INTRODUCTION TO CI

Page 3: Computational Intelligence: Introduction

3

NATURAL SYSTEMS• We have built algorithmic models, by studying

natural systems.

Natural System Corresponding Algorithmic Model

Biological neural networks Artificial neural networksEvolution Evolutionary computationSwarm behavior of social organisms

Swarm intelligence

Natural immune systems Artificial immune systemsHuman thinking processes Fuzzy systems

Page 4: Computational Intelligence: Introduction

4

ARTIFICIAL INTELLIGENCE (AI)• Definition from IEEE Neural Networks Council of

1996: the study of how to make computers do things at which people are doing better.

• Computational Intelligence (CI) is a sub-branch of AI.

Page 5: Computational Intelligence: Introduction

5

RELATIONSHIP WITH AI

AIPattern Recognition or

Machine Learning

CI

Evolutionary ComputationArtificial Neural Networks

Swarm IntelligenceArtificial Immune Systems

Fuzzy systems

PerceptionKnowledge Representation

Natural Language Processing Automated

Reasoning

Probabilistic Models

Page 6: Computational Intelligence: Introduction

6

RELATIONSHIP WITH AI

AI

• Make the machine think as we humans do

CI

• Make the machine think as we humans do using biologically inspired methods

Page 7: Computational Intelligence: Introduction

7

ARTIFICIAL NEURAL NETWORKS

Page 8: Computational Intelligence: Introduction

8

WHAT ARE ANNS?• Computational models inspired by the central

nervous system of humans, capable of pattern recognition.

• The basic building blocks of biological neural systems are nerve cells, referred to as neurons.

• An artificial neuron (AN) is a model of a biological neuron (BN).

Page 9: Computational Intelligence: Introduction

9

BIOLOGICAL NEURONS• Signals propagate from the dendrites, through the

cell body to the axon; from where the signals are propagated to all connected dendrites.

http://hplusmagazine.com/2012/10/17/four-statements-about-the-future/

Page 10: Computational Intelligence: Introduction

10

ARTIFICIAL NEURONS• Each AN receives signals from the environment, or

other ANs, gathers these signals, and when fired, transmits a signal to all connected ANs.

.

.

.

weights

output signal

input signals

f(net)activation function

An artificial neuron

Page 11: Computational Intelligence: Introduction

11

ARTIFICIAL NEURAL NETWORKS• An artificial neural network (NN) is a layered

network of ANs.

output layer

output signal

An artificial neural network

hidden layer

input layer

input signals

Page 12: Computational Intelligence: Introduction

12

NEURAL NETWORK TYPES• Single-layer NNs, such as the Hopfield network• Multilayer feedforward NNs, including, e.g., standard

backpropagation functional link and product unit networks

• Temporal NNs, such as the Elman and Jordan simple recurrent networks as well as time-delay neural networks

• Self-organizing NNs, such as the Kohonen self-organizing feature maps and the learning vector quantizer;

• Combined supervised and unsupervised NNs, e.g., some radial basis function networks.

Page 13: Computational Intelligence: Introduction

13

APPLICATIONS OF NN• Diagnosis of diseases• Speech recognition• Data mining• Composing music• Image processing• Forecasting• Robot control

• Classification• Pattern recognition• Planning• Game strategies• Credit approval• Compression

Page 14: Computational Intelligence: Introduction

14

EVOLUTIONARY COMPUTATION

Page 15: Computational Intelligence: Introduction

15

WHAT IS EC?• Evolutionary computation

(EC) has as its objective to mimic processes from natural evolution, where the main concept is survival of the fittest: the weak must die.

• In natural evolution, survival is achieved through reproduction.

http://www.stchem.bham.ac.uk/~roy/Research/ga.html

Page 16: Computational Intelligence: Introduction

16

EC: SOME TERMS• Evolutionary algorithms use a population of

individuals. An individual is referred to as a chromosome.

• A chromosome defines the characteristics of individuals in the population.

• Each characteristic is referred to as a gene. The value of a gene is referred to as an allele.

• For each generation, individuals compete to reproduce offspring.

• Those individuals with the best survival capabilities have the best chance to reproduce.

Page 17: Computational Intelligence: Introduction

17

REPRODUCTION• Offspring are generated by combining parts of the

parents, a process referred to as crossover. • Each individual in the population can also undergo

mutation which alters some of the allele of the chromosome.

• The survival strength of an individual is measured using a fitness function which reflects the objectives and constraints of the problem to be solved.

• After each generation, individuals may undergo culling, or individuals may survive to the next generation (referred to as elitism).

Page 19: Computational Intelligence: Introduction

19

CLASSES OF EVOLUTIONARY ALGORITHMS• Genetic algorithms which model genetic evolution.• Genetic programming which is based on genetic

algorithms, but individuals are programs (represented as trees).

• Evolutionary programming which is derived from the simulation of adaptive behavior in evolution (phenotypic evolution).

• Evolution strategies which are geared toward modeling the strategy parameters that control variation in evolution, i.e. the evolution of evolution.

Page 20: Computational Intelligence: Introduction

20

CLASSES OF EVOLUTIONARY ALGORITHMS• Differential evolution, which is similar to genetic

algorithms, differing in the reproduction mechanism used.

• Cultural evolution which models the evolution of culture of a population and how the culture influences the genetic and phenotypic evolution of individuals.

• Coevolution where initially “dumb” individuals evolve through cooperation, or in competition with one another, acquiring the necessary characteristics to survive.

Page 21: Computational Intelligence: Introduction

21

APPLICATIONS OF EC• Data mining• Combinatorial optimization• Fault diagnosis• Classification• Clustering,• Scheduling, and • Time series approximation

Page 22: Computational Intelligence: Introduction

22

SWARM INTELLIGENCE (SI)

Page 23: Computational Intelligence: Introduction

23http://en.wikipedia.org/wiki/File:Auklet_flock_Shumagins_1986.jpg

Page 24: Computational Intelligence: Introduction

24

WHAT IS SI?• Swarm intelligence (SI) originated from the study of

colonies, or swarms of social organisms.• Useful for optimization and clustering.

Page 25: Computational Intelligence: Introduction

25

PARTICLE SWARM OPTIMIZATION (PSO)• PSO is a stochastic optimization approach, modeled

on the social behavior of bird flocks. • Each particle in the swarm represents a candidate

solution to the optimization problem.• Each particle is “flown” through the

multidimensional search space, adjusting its position in search space according to its own experience and that of neighboring particles.

• Applications of PSO: function approximation, clustering, optimization of mechanical structures, and solving systems of equations.

Page 26: Computational Intelligence: Introduction

26

ARTIFICIAL IMMUNE SYSTEMS (AIS)

Page 27: Computational Intelligence: Introduction

27

NATURAL IMMUNE SYSTEM• The natural immune system (NIS) has an amazing

pattern matching ability, used to distinguish between foreign cells (non-self, or antigen) and the cells belonging to the body (self).

• As the NIS encounters antigen, the adaptive nature of the NIS is exhibited, with the NIS memorizing the structure of these antigen for faster future response the antigen.

Page 28: Computational Intelligence: Introduction

28

FOUR MODELS OF NIS• The classical view: the immune system distinguishes

between self and non-self, using lymphocytes produced in the lymphoid organs. These lymphocytes “learn” to bind to antigen.

• Clonal selection theory: an active B-Cell produces antibodies through a cloning process. The produced clones are also mutated.

• Danger theory: the immune system has the ability to distinguish between dangerous and non-dangerous antigen.

• Network theory: it is assumed that B-Cells form a network. When a B-Cell responds to an antigen, that B-Cell becomes activated and stimulates all other B-Cells to which it is connected in the network.

Page 29: Computational Intelligence: Introduction

29

WHAT ARE AISS?• An artificial immune system (AIS) models some of

the aspects of a NIS.• It is mainly applied to solve pattern recognition

problems, to perform classification tasks, and to cluster data.

Page 30: Computational Intelligence: Introduction

30

APPLICATIONS OF AISS• Anomaly detection, such as fraud detection• Computer virus detection

Page 31: Computational Intelligence: Introduction

31

FUZZY SYSTEMS

Page 32: Computational Intelligence: Introduction

32

WHAT ARE FUZZY SYSTEMS• Traditional set theory requires elements to be

either part of a set or not.• Human reasoning is, however, almost always not

this exact, as our reasoning includes a measure of uncertainty.

• Fuzzy sets and fuzzy logic allow what is referred to as approximate reasoning.

• With fuzzy sets, an element belongs to a set to a certain degree of certainty.

• The uncertainty in fuzzy systems is referred to as non-statistical uncertainty.

Page 33: Computational Intelligence: Introduction

33

APPLICATIONS OF FUZZY SYSTEMS• Control systems• Gear transmission and braking systems in vehicles• Controlling lifts, home appliances• Controlling traffic signals

Page 34: Computational Intelligence: Introduction

34

HISTORY• George Boole in 1854: development of the foundations of

propositional logic.• Alan Turing 1950s: the first definition of artificial intelligence

was established.• Fraser, Bremermann and Reed 1950s: starting EC with genetic

algorithms. However, it is John Holland who is generally viewed as the father of EC, most specifically of genetic algorithms.

• John MacCarthy 1956: at the Dartmouth conference coined the term artificial intelligence.

• 1956 to 1969 much research was done in modeling biological neurons. Most notable was the work on perceptrons by Rosenblatt, and the adaline by Widrow and Hoff.

Page 35: Computational Intelligence: Introduction

35

HISTORY• Lofti Zadeh 1968: Fuzzy sets• Minsky and Papert in 1969: major setback to

artificial neural network research. With their book, called Perceptrons.

• Mamdani, Sugeno, Takagi and Bezdek: Fuzzy systems research.

• Hopfield, Hinton, and Rumelhart and McLelland mid 1980s: landmark publications.

Page 36: Computational Intelligence: Introduction

36

QUESTION• Based on the definition of CI given in this chapter,

show that each of the paradigms (NN, EC, SI, AIS, and FS) does satisfy the definition.

Page 37: Computational Intelligence: Introduction

37

SUMMARY

Natural System Corresponding Algorithmic Model

Biological neural networks Artificial neural networksEvolution Evolutionary computationSwarm behavior of social organisms

Swarm intelligence

Natural immune systems Artificial immune systemsHuman thinking processes Fuzzy systems