percept ron learning

7/29/2019 Percept Ron Learning

http://slidepdf.com/reader/full/percept-ron-learning 1/45

Langston,

Cognitive Psychology



Perceptron Learning

How does a perceptron acquire its

knowledge?

The question really is: How does a

perceptron learn the appropriateweights?



Perceptron Learning

Remember our features:

For output:

Good_Fruit = 1Not_Good_Fruit = 0

Taste Sweet = 1, Not_Sweet = 0

Seeds Edible = 1, Not_Edible = 0

Skin Edible = 1, Not_Edible = 0



Perceptron Learning

Let’s start with no knowledge:

If ∑ > 0.4

then fire

0.0

0.0

0.0

Input

Output

Taste

Seeds

Skin



Perceptron Learning

The weights are empty:

If ∑ > 0.4

then fire

0.0

0.0

0.0

Input

Output

Taste

Seeds

Skin



Perceptron Learning

To train the perceptron, we will show it

each example and have it categorize

each one.

Since it’s starting with no knowledge, itis going to make mistakes. When it

makes a mistake, we are going to adjust

the weights to make that mistake lesslikely in the future.



Perceptron Learning

When we adjust the weights, we’re

going to take relatively small steps to be

sure we don’t over-correct and create

new problems.



Perceptron Learning

I’m going to learn the category “good

fruit” defined as anything that is sweet.

Good fruit = 1

Not good fruit = 0



Trained Perceptron

Show it a banana:

1

0

.00

If ∑ > 0.4

then fire

0.0

0.0

0.0

Input

1

1

0

Output

1Taste

Seeds

Skin



Trained Perceptron

Show it a banana:

1

0

.00

If ∑ > 0.4

then fire

0.0

0.0

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

In this case we have:

(1 X 0) = 0

+ (1 X 0) = 0

+ (0 X 0) = 0 It adds up to 0.0.

Since that is less than the threshold(0.40), we responded “no.”

Is that correct? No.



Perceptron Learning

Since we got it wrong, we know we needto change the weights. We’ll do thatusing the delta rule (delta for change).

∆w = learning rate X (overall teacher -overall output) X node output



Perceptron Learning

The three parts of that are: Learning rate: We set that ourselves. I want it to

be large enough that learning happens in areasonable amount of time, but small enough

that I don’t go too fast. I’m picking 0.25. (overall teacher - overall output): The teacher

knows the correct answer (e.g., that a bananashould be a good fruit). In this case, the teacher says 1, the output is 0, so (1 - 0) = 1.

node output: That’s what came out of the nodewhose weight we’re adjusting. For the first node,1.



Perceptron Learning

To pull it together:

Learning rate: 0.25.

(overall teacher - overall output): 1.

node output: 1. ∆w = 0.25 X 1 X 1 = 0.25.

Since it’s a ∆w, it’s telling us how muchto change the first weight. In this case,we’re adding 0.25 to it.



Perceptron Learning

Let’s think about the delta rule:

(overall teacher - overall output): If we get the categorization right, (overall

teacher - overall output) will be zero (theright answer minus itself).

In other words, if we get it right, we won’tchange any of the weights. As far as we

know we have a good solution, why wouldwe change it?



Perceptron Learning


(overall teacher - overall output): If we get the categorization wrong, (overall

teacher - overall output) will either be -1 or +1.○ If we said “yes” when the answer was “no,” we’re too

high on the weights and we will get a (teacher - output)of -1 which will result in reducing the weights.

○ If we said “no” when the answer was “yes,” we’re toolow on the weights and this will cause them to be

increased.



Perceptron Learning


Node output: If the node whose weight we’re adjusting sent in

a 0, then it didn’t participate in making thedecision. In that case, it shouldn’t be adjusted.Multiplying by zero will make that happen.

If the node whose weight we’re adjusting sent ina 1, then it did participate and we should change

the weight (up or down as needed).



Perceptron Learning

How do we change the weights for banana?

Feature: Learning

rate:

(overall

teacher -overalloutput):

Node

output:

∆w

taste 0.25 1 1 +0.25

seeds 0.25 1 1 +0.25

skin 0.25 1 0 0



Perceptron Learning

Adjusting weight 1:

1

0

If ∑ > 0.4

then fire

0.0

0.0

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 1 = 0.25

1

Teacher



Perceptron Learning

Corrected weight 1:

1

0

If ∑ > 0.4

then fire

0.25

0.0

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

Adjusting weight 2:

1

0

If ∑ > 0.4

then fire

0.25

0.0

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 1 = 0.25

1

Teacher



Perceptron Learning

Corrected weight 2:

1

0

If ∑ > 0.4

then fire

0.25

0.25

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

Adjusting weight 3:

1

0

If ∑ > 0.4

then fire

0.25

0.25

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 0 = 0.00

1

Teacher



Perceptron Learning

Corrected weight 3:

1

0

If ∑ > 0.4

then fire

0.25

0.25

0.0

Input

1

1

0

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

To continue training, we show it the nextexample, adjust the weights…

We will keep cycling through the examples

until we go all the way through one timewithout making any changes to theweights. At that point, the concept islearned.



Perceptron Learning

Show it a pear:

0

1

0.25

If ∑ > 0.4

then fire

0.25

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

How do we change the weights for pear?

Feature: Learning

rate:

(overall


Node

output:

∆w

taste 0.25 1 1 +0.25

seeds 0.25 1 0 0

skin 0.25 1 1 +0.25



Perceptron Learning

Adjusting weight 1:

0

1

If ∑ > 0.4

then fire

0.25

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 1 = 0.25

1

Teacher



Perceptron Learning

Corrected weight 1:

0

1

If ∑ > 0.4

then fire

0.50

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

Adjusting weight 2:

0

1

If ∑ > 0.4

then fire

0.50

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 0 = 0.00

1

Teacher



Perceptron Learning

Corrected weight 2:

0

1

If ∑ > 0.4

then fire

0.50

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

Adjusting weight 3:

0

1

If ∑ > 0.4

then fire

0.50

0.25

0.0

Input

1

0

1

0

Output

1Taste

Seeds

Skin

.25 X (1 – 0) X 1 = 0.25

1

Teacher



Perceptron Learning

Corrected weight 3:

1

0

If ∑ > 0.4

then fire

0.50

0.25

025

Input

1

1

0

0

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

Here it is with the final weights:

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

Output

Taste

Seeds

Skin



Perceptron Learning

Show it a lemon:

0

0

0

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

0

0

0

0

Output

0Taste

Seeds

Skin

0

Teacher



Perceptron Learning

How do we change the weights for lemon?

Feature: Learning

rate:

(overall


Node

output:

∆w

taste 0.25 0 0 0

seeds 0.25 0 0 0

skin 0.25 0 0 0



Perceptron Learning

Here it is with the adjusted weights:

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

Output

Taste

Seeds

Skin



Perceptron Learning

Show it a strawberry:

1

1

1

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

1

1

1

1

Output

1Taste

Seeds

Skin

1

Teacher



Perceptron Learning

How do we change the weights for strawberry?

Feature: Learning

rate:

(overall


Node

output:

∆w

taste 0.25 0 1 0

seeds 0.25 0 1 0

skin 0.25 0 1 0



Perceptron Learning

Here it is with the adjusted weights:

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

Output

Taste

Seeds

Skin



Perceptron Learning

Show it a green apple:

0

1

0.25

If ∑ > 0.4

then fire

0.50

0.25

0.25

Input

0

0

1

0

Output

0Taste

Seeds

Skin

0

Teacher



Perceptron Learning

If you keep going, you will see that this

perceptron can correctly classify the

examples that we have.

percept ron learning

Documents