cdt314 faber formal languages, automata and models of computation lecture 1 - intro

1

CDT314

FABER

Formal Languages, Automata and Models of Computation

Lecture 1 - Intro

Mälardalen University

2010

2

Content

Adminstrivia

Mathematical Preliminaries

Countable Sets (Uppräkneliga mängder)

Uncountable sets (Överuppräkneliga mängder)

3

Examiner

Gordana Dodig-Crnkovic

LecturerStefan Bygde

4

http://www.idt.mdh.se/kurser/cd5560/11_10

visit home page regularly!

Course Home Page

5

How Much Work?

20 hours a week for this type of course (norm)

4 hours lectures

2 hours exercises

14 hours own work a week!

6

Why Theory of Computation?

1. A real computer can be modelled by a mathematical object: a theoretical computer.

2. A formal language is a set of strings, and can represent a computational problem.

3. A formal language can be described in many different ways that ultimately prove to be identical.

4. Simulation: the relative power of computing models can be based on the ease with which one model can simulate another.

7

5. Robustness of a computational model.

6. The Church-Turing thesis: anything that can be computed can be computed by a Turing machine.

7. Nondeterminism: languages can be described by the existence or nonexistence of computational paths.

8. Unsolvability: for some computational problems there is no corresponding algorithm that will unerringly solve them.

8

Practical Applications

1. Efficient compilation of computer languages

2. String searching

3. Identifying the limits; Recognizing difficult problems

4. Applications to other areas:– circuit verification– economics and game theory (finite automata as

strategy models in decision-making); – theoretical biology (L-systems as models of

organism growth) – computer graphics (L-systems) – linguistics (modeling by grammars)

9

History

• Euclid's attempt to axiomatize geometry

(Archimedes realized, during his own efforts to define the area of a planar figure, that Euclid's attempt had failed and that additional postulates were needed. )

• Leibniz's dream of a symbolic logic

• de Morgan, Boole, Frege, Russell, Whitehead:

Mathematics as branch of symbolic logic!

10

1900 Hilberts program

1880 -1936 first programming languages

1931 Gödels incompleteness theorem

1936 Turing machine (showed to be equivalent with recursive functions). Commonly accepted: TM as ultimate computer

1950 automata

1956 language/automata hierarchy

11

Every mathematical truth expressed in a formal language is consisting of

• a fixed alphabet of admissible symbols, and

• explicit rules of syntax for combining those symbols into meaningful words and sentences

12

Turing used a Universal Turing machine (UTM) to prove an even more powerful incompleteness theorem because it destroyed not one but two of Hilbert's dreams:

1. Finding a finite list of axioms from which all

mathematical truths can be deduced

2. Solving the entscheidungsproblem, ("decision

problem“) by producing a "fully automatic procedure"

for deciding whether a given proposition (sentence) is

true or false.

Examination

• Three midterms• Regular Languages• Context Free Languages• Restriction Free Languages

• OR:• Final Exam in three parts• If you have finished the midterm of one

time of language you don’t have to do corresponding part of the final exam

13

Form and Resources

• Lectures• Exercises• Labs (optional)• Midterms• Webpage (many links, news etc)• Literature:

• Swe: Lennart Salling. Formella språk, automater och beräkningar

• Eng: Linz, An Introduction to Formal Languages and Automata

14

15

Mathematical Preliminaries

16

• Sets

• Functions

• Relations

• Proof Techniques

• Languages, Alphabets and Strings

• Strings & String Operations

• Languages & Language Operations

17

}3,2,1{AA set is a collection of elements

SETS

},,,{ airplanebicyclebustrainB

We write

A1

Bship

18

Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

finite set

infinite set

19

A = { 1, 2, 3, 4, 5 }

Universal Set: All possible elements

U = { 1 , … , 10 }

1 2 3

4 5

A

U

6

7

8

910

20

Set Operations

A = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union

A U B = { 1, 2, 3, 4, 5 }

• Intersection

A B = { 2, 3 }

• Difference

A - B = { 1 }

B - A = { 4, 5 }

U

A B

A-B

21

• Complement

Universal set = {1, …, 7}

A = { 1, 2, 3 } A = { 4, 5, 6, 7}

12

3

4

5

6

7

AA

A = A

22

{ even integers } = { odd integers }

02

4

6

1

3

5

7

even

odd

Integers

23

DeMorgan’s Laws

A U B = A BU

A B = A U B

U

24

Empty, Null Set:

= { }

S U = S

S =

S - = S

- S =

U = Universal Set

25

Subset

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

U

A

B

26

Disjoint Sets

A = { 1, 2, 3 } B = { 5, 6}

A B = U

A B

27

Set Cardinality

For finite sets

A = { 2, 5, 7 }

|A| = 3

28

Powersets

A powerset is a set of sets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

29

Cartesian Product

A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

A X B X … X Z

30

PROOF TECHNIQUES

• Proof by construction

• Proof by induction

• Proof by contradiction

31

Proof by Construction

We define a graph to be k-regular

if every node in the graph has degree k.

Theorem. For each even number n > 2 there exists

3-regular graph with n nodes.

1

2

4

3

0

5

1 2

0

3n = 4 n = 6

32

Construct a graph G = (V, E) with n > 2 nodes.

V= { 0, 1, …, n-1 }

E = { {i, i+1} for 0 i n-2} {{n-1,0}} (*)

{{i, i+n/2 for 0 i n/2 –1} (**)

The nodes of this graph can be written consecutively around the circle.

(*) edges between adjacent pairs of nodes

(**) edges between nodes on opposite sides

Proof by Construction

END OF PROOF

33

Inductive Proof

We have statements P1, P2, P3, …

If we know

• for some k that P1, P2, …, Pk are true

• for any n k that

P1, P2, …, Pn imply Pn+1

Then

Every Pi is true

34

Proof by Induction

• Inductive basis

Find P1, P2, …, Pk which are true

• Inductive hypothesis

Let’s assume P1, P2, …, Pn are true,

for any n k

• Inductive step

Show that Pn+1 is true

35

Example

Theorem A binary tree of height n

has at most 2n leaves.

Proof

let L(i) be the number of leaves at level i

L(0) = 1

L(3) = 8

36

We want to show: L(i) 2i

• Inductive basis

L(0) = 1 (the root node)

• Inductive hypothesis

Let’s assume L(i) 2i for all i = 0, 1, …, n

• Induction step

we need to show that L(n + 1) 2n+1

37

Induction Step

hypothesis: L(n) 2n

Level

n

n+1

38

hypothesis: L(n) 2n

Level

n

n+1

L(n+1) 2 * L(n) 2 * 2n = 2n+1

Induction Step

END OF PROOF

39

Proof of induction: The cardinality of the powerset

Claim: A set of n elements has 2n subsets

Check:

• The empty set {} has only one subset: {}.

• The set {a} (a set with exactly one element) has two subsets: {} och {a}

40

Påstående: En mängd med n element har 2n delmängder

Check (cont.)

• The set of two elements {a, b} has four subsets: {}, {a}, {b} and {a,b}

• The set {a, b, c} has eight subsets: {}, {a}, {b}, {c} och {a,b}, {a,c}, {b,c}, {a,b,c}

The claim holds for these basic cases.

41

Base Case

The simplest case is a set with no elements (there is just one such set), which has 20 = 1 subsets.

42

Induction step

Suppose that the claim holds for all sets with k elements, i.e., suppose that every set with k elements has 2k subsets.

Show that the claim in this case holds also for all sets with k+1 elements, i.e., show that every set with k+1 elements has 2k+1 subsets.

43

Consider an arbitrary set with k+1 elements. The subsets of this set can be divided into two groups:

Subsets which does not contain the k+1:th element: Such a subset is a subset to the set of the k original elements, and the cardinality of subsets to a set of k elements is (according to the inductive hypothesis) 2k.

44

Subsets containing the k+1:th element: Such a subset can be constructed by taking any set not containing the k+1th element and add the k+1th element. Since there are 2k subsets without the k+1th element (by hypothesis), it is also possible to create 2k

subsets including this element.

In total there are 2k + 2k = 2. 2k= 2k+1 subsets to the considered set.

END OF PROOF

(Exemple taken from the book: Diskret matematik och diskreta modeller, K Eriksson, H. Gavel)

45

Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false

• then we arrive at a conclusion that contradicts our assumptions

• therefore, statement P must be true

46

Example

Theorem is not rational

Proof

Assume by contradiction that it is rational

= n/m

n and m have no common factors

We will show that this is impossible

2

2

47

Therefore, n2 is evenn is even

n = 2 k

2 m2 = 4k2 m2 = 2k2m is even

m = 2 p

Thus, m and n have common factor 2

Contradiction!

= n/m 2 m2 = n2 2

END OF PROOF

48

Countable Sets

49

Infinite sets are either

Countable or Uncountable

50

Countable set

There is a one to one correspondence

between elements of the set

and natural numbers

51

We started with the natural numbers, then• add infinitely many negative whole numbers to get the integers, • then add infinitely many rational fractions to get the rationals, • then added infinitely many irrational fractions to get the reals.

Each infinite addition seem to increase cardinality: |N| < |Z| < |Q| < |R|

But is this true? NO!

52

Example

Integers: ,2,2,1,1,0

The set of integers is countable

Correspondence:

Natural numbers: ,4,3,2,1,0

oddnnevennnnf 2/)1(;2/)( {

53

ExampleThe set of rational numbers

is countable

Positive

Rational numbers:,

87

,43

,21

54

Naive Idea

Rational numbers: ,31

,21

,11

Natural numbers:

Correspondence:

,3,2,1

Doesn’t work!

we will never count

numbers with nominator 2:,

32

,22

,12

55

Better Approach

11

21

31

41

12

22

32

13

23

14

...

...

...

...

Rows: constant numerator (täljare)

Columns: constant denominator

56

11

21

31

41

12

22

32

13

23

14

...

...

...

...

57

We proved:

the set of rational numbers is countable

by describing an enumeration procedure

58

Definition

An enumeration procedure for is an

algorithm that generates

all strings of one by one

Let be a set of strings S

S

S

59

A set is countable if there is an

enumeration procedure for it

Observation

60

Example

The set of all finite strings

is countable

},,{ cba

We will describe the enumeration procedure

Proof

61

Naive procedure:

Produce the strings in lexicographic order:

aaaaaa

......Doesn’t work!

Strings starting with will never be produced b

aaaa

62

Better procedure

1. Produce all strings of length 1

2. Produce all strings of length 2

3. Produce all strings of length 3

4. Produce all strings of length 4

..........

Proper Order

63

Produce strings in

Proper Order

aaabacbabbbccacbcc

aaaaabaac......

length 2

length 3

length 1abc

64

Theorem

The set of all finite strings is countable

Proof

Find an enumeration procedure

for the set of finite strings

Any finite string can be encoded

with a binary string of 0’s and 1’s

65

Produce strings in Proper Order

length 2

length 3

length 10

1

00

01

10

11

000

001

….

0

1

2

3

4

5

6

7

….

String = program Natural number

66

PROGRAM = STRING (syntactic way)

PROGRAM = FUNCTION (semantic way)

PROGRAMstring string

PROGRAMnatural number

n

natural number

n

67

Uncountable Sets

68

A set is uncountable if it is not countable

Definition

69

Theorem

The set of all infinite strings is uncountable

We assume we have

an enumeration procedure

for the set of infinite strings

Proof (by contradiction)

70

Infinite string Encoding

0w

1w

2w

...

...

...

...

00b

10b

20b

01b

11b

21b

02b

12b

22b

=

=

=

Cantor’s diagonal argument

... ... ... ...

71

Cantor’s diagonal argument

We can construct a new string that is missing in our enumeration!

w

The set of all infinite strings is uncountable!

Conclusion

72

There are some integer functions that

that cannot be described by finite strings (programs/algorithms).

Conclusion

An infinite string can be seen as FUNCTION (n:th output is n:th bit in the string)

73

Theorem

Let be an infinite countable set

The powerset of is uncountable S2 S

S

Example of uncountable infinite sets

74

Proof

Since is countable, we can write S

},,,{ 321 sssS

75

Elements of the powerset have the form:

},{ 31 ss

},,,{ 10975 ssss

……

76

We encode each element of the power set

with a binary string of 0’s and 1’s

1s 2s 3s 4s

1 0 0 0}{ 1s

Powerset

element

Encoding

0 1 1 0},{ 32 ss

1 0 1 1},,{ 431 sss

...

...

...

...

77

Let’s assume (by contradiction)

that the powerset is countable.

we can enumerate

the elements of the powerset

Then:

78

1 0 0 0 0

1 1 0 0 0

1 1 0 1 0

1 1 0 0 1

Powerset

elementEncoding

1p

2p

3p

4p

...

...

...

...

...

79

Take the powerset element

whose bits are the complements

in the diagonal

80

1 0 0 0 0

1 1 0 0 0

1 1 0 1 0

1 1 0 0 1

New element: 0011

(binary complement of diagonal)

...

...

...

...

1p

2p

3p

4p

81

The new element must be some

of the powerset ip

However, that’s impossible:

the i-th bit of must be

the complement of itself

from definition of

Contradiction!

ip

ip

82

Since we have a contradiction:

The powerset of is uncountable S2 S

END OF PROOF

83

Example Alphabet : },{ ba

The set of all finite strings:

},,,,,,,,,{},{ * aabaaabbbaabaababaS

infinite and countable

uncountable infinite

}},,,}{,{},{},{{2 aababaabaaS 1L 2L 3L 4L

The powerset of contains all languages:S

An Application: Languages

84

Finite strings (algorithms): countable

Languages (power set of strings): uncountable

There are infinitely many more languages

than finite strings.

85

There are some languages

that cannot be described by finite strings (algorithms).

Conclusion

86

Cardinality - KardinaltalA Cardinal number is a measure of the size of a set. The cardinality

for a finite set is simply the number of elements in the set. Two sets has equal size if it is possible to pair the elements one by

one in the two sets. That is, if there exists a bijection between the two sets.

This concept can be generalised to infinite sets. For instance, the set of positive integers and the set of integers have equal size.

On the contrary, it is not possible to pair the real numbers with the integers in this fashion. The set of real numbers is larger than the set of integers.

Cardinal numbers can be defined as in the following way: two sets have the same cardinal number if and only if they have the same size (i.e., can be paired together one and one). As an example, the cardinal number for the integers is 0 (alef-0, alef is the first letter in the hebrew alphabet).

These infinite cardinal numbers are called transfinite cardinal numbers.

87

Georg Cantor developed, in the end of the 19th century, the logical foundation of mathematics: set theory.

Cantor introduced the notion transfinite cardinal numbers.

The simplest, ”smallest", infinity, he named 0.

This is the countable infinite sets’ (for instance, the integers) cardinal number.

The cardinality of the set of points on a line, or the points in a plane or field, Cantor named 1.

Are there ”larger” infinities?

More about Infinty

88

Yes! Cantor showed that the set of functions on a line was ”even more infinite” than the points on the line, and

he called that cardinality 2.

Cantor discovered that it was possible to use artithmetic on cardinal numbers just as with finite numbers. But the arithmetic rules became quite simplistic:

0 + 1= 0 0 + 0 = 0 0 · 0 = 0.

89

However, with powers something happened:

0 0 (0 to the power of 0) = 1.

In general:

2 n (2 to the power of n) = n+1

This implies that there are infinitly many infinities, in strictly increasing order.

90

But is it really certain that there is no cardinality in between the countable infinity and the points on the line? Cantor tried to prove the so-called continuum hypothesis.

Cantor: two different infinities 0 and 1 http://www.ii.com/math/ch/#cardinals

Continuum Hypothesis: 0 < 1 = 2 0

See also:http://www.nyteknik.se/pub/ipsart.asp?art_id=26484