1 csse 350 automata, formal languages, and computability

1

CSSE 350Automata, Formal Languages, and

Computability

2

Mathematical Preliminaries and Background

• Sets

• Relations

• Functions

• Logic

• Graphs

• Trees

• Proof Techniques

• Languages and Strings

• Automaton

3

Sets

4

}3,2,1{A

A set is a collection of elements, having a property that

characterizes those elements.

SETS

},,,{ airplanebicyclebustrainB

We write Belongs To and Does not Belong To as:

A1

Bship

5

Set Representations• One way is to enumerate the elements completely.

All the elements belonging to the set are explicitly given. – For example, A = {1,2,3,4,5}

• Alternate way is to give the properties that characterize the elements of the set. – For example, B = {x | x is a positive integer less than

or equal to 5} – Or B = {x : x is a positive integer less than or equal to 5}

• Ellipses may be used in the set specification if the meaning is clear. For example, C = { a, b, …, z}, which stands for all the lower-case letters of the English alphabet

6

Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

finite set

infinite set

7

A = { 1, 2, 3, 4, 5 }

Universal Set: all possible elements U = { 1 , … , 10 }

1 2 3

4 5

A

U

6

7

8

910

8

Set Operations

A = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union

A U B = { 1, 2, 3, 4, 5 }

• Intersection

A B = { 2, 3 }

• Difference

A - B = { 1 }

B - A = { 4, 5 }

U

A B2

31

4

5

2

3

1

Venn diagrams

9

A

• Complement

Universal set = {1, …, 7}

A = { 1, 2, 3 } A = { 4, 5, 6, 7}

12

3

4

5

6

7

A

A = A

10

02

4

6

1

3

5

7

even

{ even integers } = { odd integers }

odd

Integers

11

DeMorgan’s Laws

A U B = A B

U

A B = A U BU

12

Empty, Null Set:= { }

S U = S

S =

S - = S

- S =

U= Universal Set

13

Subset

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

UA

B

14

Disjoint Sets

A = { 1, 2, 3 } B = { 5, 6}

A B =

UA B

15

Set Cardinality

• For finite sets

A = { 2, 5, 7 }

|A| = 3

(set size)

16

Powersets

A powerset is a set of sets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

17

Cartesian ProductA = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

A X B X … X Z

18

Set Identities A, B, C represent arbitrary sets and ø is the empty set

and U is the Universal Set.

The Commutative laws:              A  B = B  A

             A  B = B  A The Associative laws:

             A  (B C) = (A B) C              A  (B C) = (A B) C

The Distributive laws:              A  (B C) = (A B) (A C)

             A  (B C) = (A B) (A C)

19

The Idempotent laws:              A  A = A              A A = A

The Absorptive laws:              A  (A B) = A              A  (A B) = A

The De Morgan laws:              (A  B)' = A' B'              (A  B)' = A' B'

Other laws involving Complements:              ( A' )' = A              A A' = ø              A  A' = U

20

Other laws involving the empty set              A  ø = A              A  ø = ø

Other laws involving the Universal Set:              A  U = U              A  U = A    

21

RELATIONS

22

Relations

Let A and B be sets. A binary relation from A into B is any subset of the Cartesian product A B.

For example: let A = {2, 3, 5, 6}, and B = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, and a relation R from A into B by (a, b)  R if and only if 2a = b

So, R = {(2, 4), (3, 6), (5, 10), (6, 12)}.

23

A typical element in R is an ordered pair (x, y). In some cases R can be described by actually listing the pairs which are in R, as in the previous example. This may not be convenient if R is relatively large.

Other notations are used depending on the past practice. Consider the following relation on real numbers. R = {(x, y) | y is the square of x} and S = {(x, y) | x <= y}.

R could be more naturally expressed as R(x) = x2 or R(x) =y where y = x2.

24

RELATIONS R = {(x1, y1), (x2, y2), (x3, y3), …}

xi R yi

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

25

Relation on a Set

A relation from a set A into itself is called a relation on A.

For example, let A = {1, 2, 3, 4, 5, 6}, and a relation R from A to A itself by (a, b)  R if and only if a is a divisor of b

So, R = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 2), (2, 4), (2, 6), (3, 3), (3, 6), (4, 4), (5, 5), (6, 6)}.

Let A be a set of people and let P = {(a, b) | a  A b  A a is a child of b}. Then P is a relation on A which we might call a parent-child relation.

26

Composition

Let R be a relation from a set A into set B, and S be a relation from set B into set C. The composition of R and S, written as RS, is the set of pairs of the form (a, c)  A C, where (a, c)  RS if and only if there exists b  B such that (a, b)  R and (b, c)  S.

For example PP, where P is the parent-child relation given, is the composition of P with itself and it is a relation which we know as grandparent-grandchild relation.

27

Properties of Relations Assume R is a relation on set A; in other words, R  A

A. Let us write a R b to denote (a, b)  R. 1. Reflexive: R is reflexive if for every a  A, a R a.2. Symmetric: R is symmetric if for every a and b in A, if aRb, then bRa. 3. Transitive: R is transitive if for every a, b and c in A, if aRb and bRc, then aRc. 4. Equivalence: R is an equivalence relation on A if R is reflexive, symmetric and transitive.

For example, the equality relation (=) on a set of numbers such as {1, 2, 3} is an equivalence relation.

28

Equivalence Relations

• Reflexive: x R x

• Symmetric: x R y y R x

• Transitive: x R y and y R z x R z

Example: R = ‘=‘

• x = x

• x = y y = x

• x = y and y = z x = z

29

FUNCTIONS

30

FunctionsA function is something that associates each element of

a set with an element of another set (which may or may not be the same as the first set).

Or a function is a rule that assigns to elements of one set a unique element of another set.

For example, a social security number uniquely identifies a person, in this case, to each member of the set of social security number, some member of the set of person is assigned.

31

FUNCTIONSdomain

12

3

a

bc

range

f : A -> B

A B

If A = domain

then f is a total function

otherwise f is a partial function

f(1) = a4

5

32

Growth Rates of FunctionsLet f(n) and g(n) to be two functions whose domain is a

subset of the positive integersIf there exists a positive constant c such that for all

sufficiently large nf(n) ≤ cg(n)

f has the order at most gf(n) = O(g(n))

cg(n)

f(n)

33

If there exists a positive constant c such that for all sufficiently large n

f(n) cg(n)f has the order at least g

f(n) = (g(n))

cg(n)

f(n)

34

If there exist positive constant c1 and c2, such that for all sufficiently large n

c1g(n) ≤ f(n) ≤ c2g(n)

f and g have the same order of magnitudef(n) = (g(n))

c1g(n)

f(n)

c2g(n)

35

ExamplesExample 1: 2n is Θ(n)

f(n) = 2n, g(n) = n For all integers n ≥ 0, 2n ≤ 2n, which is f(n) ≤ 2g(n);

so constant c ≥ 2 suffices for the definition of O.For all integers n ≥ 0, 2n ≥ n, which is f(n) ≥ 1g(n); so any 0 < c ≤ 2 suffices for the definition of Ω.Since 2n is both O(n) and Ω(n), it is Θ(n).

36

Example 2: n is O(n2) but not Ω(n2)f(n) = n, g(n) = n2

For all integers n ≥ 0, n ≤ n2; which is f(n) ≤ 1g(n); so constant c ≥ 1 suffices for the definition of O.However, n cannot be Ω(n2) since no matter how small c>0 is chosen, for all n > 1/c, n = c(1/c)n < cnn = cn2.

37

Logic

Propositional Logic & Predicate Logic

38

Propositional Logic

Proposition: a declarative statement having a specific truth-value, true or false. For example:8 is an even number. True4 is a negative number. False

39

Two or more propositions can be combined together to make compound propositions with the help of logical connectives. For example: above two propositions can be used to make a compound proposition using any of the logical connectives. 8 is an even number  AND  4 is a negative number.

False8 is an even number  OR 4 is a negative number.

True

For the first compound proposition to be true both the propositions have to be true as the connective is AND. As OR is the connective for the second one, if either of the propositions is true, the truth value of the compound proposition is true.

40

logical Connectives Conjunction: The compound proposition truth-value is

true if and only if all the constituent propositions hold true. It is represented as " ^ ". Truth table for two individual propositions p and q with conjunction is given below.

p q p q

T T T

T F F

F T F

F F F

41

Disjunction: This is logical "or" read as either true value of the individual propositions. Truth table is given below.

p q p V q

T T T

T F T

F T T

F F F

42

Negation: This is the logical "negation" and it is expressed by as  ¬p for "not p". Truth table is given below.

p ¬ pT F

F T

43

Conditional: This is used to define as "a proposition holds true if another proposition is true" i.e. p q is read as "if p, then q". Truth table is given below.

p q p q

T T T

T F F

F T T

F F T

44

Biconditional: A proposition (p  q) ^ (q  p) can be abbreviated using  biconditional conjunction as p  q and is read as "if p then q, and if q then p".

Tautology: A compound proposition, which is true in every case. E.g., p V ¬ p

Contradiction: This is the opposite of tautology, which is false in every case. E.g., p ¬ p

45

Identities ¬ (P Q) (¬ P ¬Q) ----- DeMorgan's Law ¬ (P Q) (¬ P ¬ Q) ----- DeMorgan's Law (P Q) (¬ P Q) ----- implication [(P Q) R] [P (Q R)] ----- exportation (P Q) (¬ Q ¬P) ----- contrapositive

46

Implications

[(P Q) ¬Q] P ----- modus tollens [(P Q) (R S)] [(P R) (Q S)] [(P Q) (Q R)] (P R)

47

Predicate LogicThe propositional logic is not powerful enough to represent all

types of assertions. For example, the statement: x is a non-zero integer, where x is a variable, is not a proposition because you can not tell whether it is true or false unless you know the value of x. Thus the propositional logic can not deal with such sentences.

The predicate logic is one of the extensions of propositional logic and it is fundamental to most other types of logic. Central to the predicate logic are the concepts of predicate and quantifier.

A predicate is a feature of language which you can use to make a statement about something, e.g. to attribute a property to that thing. If you say "Peter is tall", then you have applied to Peter the predicate "is tall". We also might say that you have predicated tallness of Peter or attributed tallness to Peter.

48

A predicate is a template involving a verb that describes a property of objects, or a relationship among objects represented by the variables. For example, the sentences:The flower is red.The sweater is red. The frame is red.

All these sentences come from the template "is red" by placing an appropriate noun/noun phrase in front of it. The phrase "is red" is a predicate and it describes the property of being red. Predicates are often given a name.

For example any of "is_red", "Red" or "R" can be used to represent the predicate "is red" among others. We can adopt Red as the name for the predicate “is_red". Sentences that assert an object is red can be represented as "Red(x)", where x represents an arbitrary object. Red(x) reads as "x is red".

49

A predicate with variables, called atomic formula, can be made a proposition by applying one of the following two operations to each of its variables: assign a value to the variable quantify the variable using a quantifier

For example, x > 1 becomes 3 > 1 if 3 is assigned to x, and it becomes a true statement, and it becomes a proposition.

In general, a quantification is performed on formulas of predicate logic (called wff), such as x > 1 or P(x), by using quantifiers on variables.

50

There are two types of quantifiers:universal quantifierexistential quantifier

The universal quantifier turns, for example, the statement x > 1 to "for every object x in the universe, x > 1", which is expressed as " x x > 1". This new statement is true or false in the universe of discourse. Hence it is a proposition once the universe is specified.

Similarly the existential quantifier turns, for example, the statement x > 1 to "for some object x in the universe, x > 1", which is expressed as " x x > 1." Again, it is true or false in the universe of discourse, and hence it is a proposition once the universe is specified.

51

The universe of discourse, also called universe, is the set of objects of interest. The propositions in the predicate logic are statements on objects of a universe.

The universe is thus the domain of the (individual) variables. It can be:the set of real numbersthe set of integersthe set of all cars on a parking lotthe set of all students in a classroom etc.

The universe is often left implicit in practice. But it should be obvious from the context

52

Predicate logic is more powerful than propositional logic. It allows one to reason about properties and relationships of individual objects. In predicate logic, one can use some additional inference rules, as well as those for propositional logic such as the equivalences, implications and inference rules.

Predicate logic permits reasoning about the propositional connectives and also about quantification ("all" or "some"). A classic, if elementary, example of what can be done with the predicate logic is the inference from the premises: All men are mortal.Socrates is a man.

to the conclusion Socrates is mortal

53

Important Inference Rules of Predicate Logic

First there is the following rule concerning the negation of quantified statement which is very useful:

¬x P(x) x ¬P(x)

Next there is the following set of rules on quantifiers

and connectives: x [P(x) Q(x)] [x P(x) x Q(x)] [x P(x) x Q(x)] x [P(x) Q(x)]

x [P(x) Q(x)] [x P(x) x Q(x)]

x [P(x) Q(x)] [x P(x) x Q(x)]

54

GRAPHS

55

GRAPHSA directed graph

• Nodes (Vertices)

V = { a, b, c, d, e }

• Edges

E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }

node

edge

a

b

c

d

e

56

Labeled Graph

a

b

c

d

e

1 3

56

26

2

57

Walk

a

b

c

d

e

Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)

58

Path

a

b

c

d

e

Path is a walk where no edge is repeated

Simple path: no node is repeated

59

Cycle

a

b

c

d

e

12

3

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

base

60

Euler Tour

a

b

c

d

e1

23

45

6

7

8 base

A cycle that contains each edge once

61

Hamiltonian Cycle

a

b

c

d

e1

23

4

5 base

A simple cycle that contains all nodes

62

Finding All Simple Paths

a

b

c

d

e

origin

63

(c, a)

(c, e)

Step 1

a

b

c

d

e

origin

64

(c, a)

(c, a), (a, b)

(c, e)

(c, e), (e, b)

(c, e), (e, d)

Step 2

a

b

c

d

e

origin

65

Step 3

a

b

c

d

e

origin(c, a)

(c, a), (a, b)

(c, a), (a, b), (b, e)

(c, e)

(c, e), (e, b)

(c, e), (e, d)

66

Step 4

a

b

c

d

e

origin

(c, a)

(c, a), (a, b)

(c, a), (a, b), (b, e)

(c, a), (a, b), (b, e), (e,d)

(c, e)

(c, e), (e, b)

(c, e), (e, d)

67

Trees

68

Treesroot

leaf

parent

child

Trees have no cycles

69

root

leaf

Level 0

Level 1

Level 2

Level 3

Height 3

70

Binary Trees

71

PROOF TECHNIQUES

72

PROOF TECHNIQUES

• Proof by induction

• Proof by contradiction

73

Induction

We have statements P1, P2, P3, …

If we know

• for some b that P1, P2, …, Pb are true

• for any k >= b that

P1, P2, …, Pk imply Pk+1

Then

Every Pi is true

74

Proof by Induction• Inductive basis

Find P1, P2, …, Pb which are true

• Inductive hypothesis

Let’s assume P1, P2, …, Pk are true,

for any k >= b

• Inductive step

Show that Pk+1 is true

75

Example

Theorem: A binary tree of height n

has at most 2n leaves.

Proof by induction:

let L(i) be the maximum number of

leaves of any subtree at height i

76

We want to show: L(i) <= 2i

• Inductive basis

L(0) = 1 (the root node)

• Inductive hypothesis

Let’s assume L(i) <= 2i for all i = 0, 1, …, k

• Induction step

we need to show that L(k + 1) <= 2k+1

77

Induction Step

From Inductive hypothesis: L(k) <= 2k

height

k

k+1

78

L(k) <= 2k

L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1

Induction Step

height

k

k+1

(we add at most two nodes for every leaf of level k)

79

Remark

Recursion is another thing

Example of recursive function:

f(n) = f(n-1) + f(n-2)

f(0) = 1, f(1) = 1

80

Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false

• then we arrive at an incorrect conclusion

• therefore, statement P must be true

81

Example

Theorem: is not rational

Proof:

Assume by contradiction that it is rational

= n/m

n and m have no common factors

We will show that this is impossible

2

2

82

= n/m 2 m2 = n2

Therefore, n2 is evenn is even

n = 2 k

2 m2 = 4k2 m2 = 2k2m is even

m = 2 p

Thus, m and n have common factor 2

Contradiction!

2

83

Languages and Strings

84

A language is a set of strings

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet: zcba ,,,,

85

Alphabets and StringsWe will use small alphabets:

Strings

abbaw

bbbaaav

abu

ba,

baaabbbaaba

baba

abba

ab

a

86

String Operations

m

n

bbbv

aaaw

21

21

bbbaaa

abba

mn bbbaaawv 2121

Concatenation

abbabbbaaa

87

12aaaw nR

naaaw 21 ababaaabbb

Reverse

bbbaaababa

88

Concatenation and Reverse of Strings

Theorem: If w and x are strings, then (w x)R = xR wR.

Example:

(nametag)R = (tag)R (name)R = gateman

89

Concatenation and Reverse of Strings Proof: By induction on |x|:

|x| = 0: Then x = , and (wx)R = (w )R = (w)R = wR = R wR = xR wR.

n 0 (((|x| = n) ((w x)R = xR wR)) ((|x| = n + 1) ((w x)R = xR wR))):

Consider any string x, where |x| = n + 1. Then x = u a for some character a and |u| = n. So:

(w x)R = (w (u a))R rewrite x as ua= ((w u) a)R associativity of concatenation= a (w u)R definition of reversal= a (uR wR) induction hypothesis= (a uR) wR associativity of concatenation= (ua)R wR definition of reversal= xR wR rewrite ua as x

90

String Length

Length:

Examples:

naaaw 21

nw

1

2

4

a

aa

abba

91

Length of Concatenation

Example:

vuuv

853

8

5,

3,

vuuv

aababaabuv

vabaabv

uaabu

92

Empty StringA string with no letters:

Observations:

abbaabbaabba

www

0

93

SubstringSubstring of string:

a subsequence of consecutive characters

String Substring

bbab

b

abba

ab

abbab

abbab

abbab

abbab

94

Prefix and Suffix

Prefixes Suffixesabbab

abbab

abba

abb

ab

a

b

ab

bab

bbab

abbab uvw

prefix

suffix

95

Another Operation (Replication)

Example:

Definition:

n

n wwww

abbaabbaabba 2

0w

0abba

96

The * Operation : the set of all possible strings from alphabet

*

,,,,,,,,,*

,

aabaaabbbaabaaba

ba

97

The + Operation : the set of all possible strings from alphabet except

,,,,,,,,,*

,

aabaaabbbaabaaba

ba

*

,,,,,,,, aabaaabbbaabaaba

98

LanguagesA language is any subset of

Example:

Languages:

*

,,,,,,,,*

,

aaabbbaabaaba

ba

},,,,,{

,,

aaaaaaabaababaabba

aabaaa

99

Note that:

}{}{

0}{

1}{

0

Sets

Set size

Set size

String length

100

Another Example

An infinite language }0:{ nbaL nn

aaaaabbbbb

aabb

ab

L Labb

101

Operations on LanguagesThe usual set operations

Complement:

aaaaaabbbaaaaaba

ababbbaaaaaba

aaaabbabaabbbaaaaaba

,,,,

}{,,,

},,,{,,,

LL *

,,,,,,, aaabbabaabbaa

102

Reverse

Definition:

Examples:

}:{ LwwL RR

ababbaabababaaabab R ,,,,

}0:{

}0:{

nabL

nbaL

nnR

nn

103

Concatenation

Definition:

Example:

2121 ,: LyLxxyLL

baaabababaaabbaaaab

aabbaaba

,,,,,

,,,

104

Another OperationDefinition:

Special case:

n

n LLLL

bbbbbababbaaabbabaaabaaa

babababa

,,,,,,,

,,,, 3

0

0

,, aaabbaa

L

105

More Examples

}0:{ nbaL nn

}0,:{2 mnbabaL mmnn

2Laabbaaabbb

106

Star-Closure (Kleene *)

Definition:

Example:

210* LLLL

,,,,

,,,,

,,

,

*,

abbbbabbaaabbaaa

bbbbbbaabbaa

bbabba

107

Positive Closure

Definition:

*

21

L

LLL

,,,,

,,,,

,,

,

abbbbabbaaabbaaa

bbbbbbaabbaa

bba

bba

108

Grammar

109

GrammarMathematically describe language

English: informal, inadequateSet notations: limited

GrammarEnglish grammar describes whether a sentence is well-formed

or not<sentence> <noun_phrase><predicate>

Then, what is noun_phrase, what is predicate?<noun_phrase> <article><noun><predicate> <verb>

Basic idea of formal grammars: Start from the top level conceptReduce it to the irreducible building blocks

110

Grammar DefinitionA grammar G is defined as a quadruple

G = (V, T, S, P)V is a finite set of objects called variablesT is a finite set of objects called terminal

symbolsS V is a special symbol called the start

variableP is a finite set of productionsv and T are non-empty and disjoint

111

Production RulesHeart of a grammarDescribe how the grammar transforms one string into anotherDefine a language associated with grammarProduction rules are of the form

x y, where x (V T)+, y (V T)* For example, given w = uxv, x y,

then z = uyv, z is a new stringwritten as: w z, means that w derives z or

z is derived from w w1 w2 … wn is the same as w1 wn *

112

DefinitionLet G = (V, T, S, P) be a grammar. Then the set

L(G) = { w T* : S w}is the language generated by G.

If w L(G),

S w1 w2 … wn w

is a derivation of the sentence w.

Strings S, w1, w2, …, wn are called sentential forms of the derivation. All these strings may contain variables and terminals.

*

113

Automaton

114

Automaton

Input

“Accept” or“Reject”

OutputS1 S2 S3

S4 S5

Storage

ControlUnit

115

AutomataAn abstract model of a digital computer• Input mechanism

– Can read but not change an input file • Input file

– Contains a string over a given alphabet– Divided into cells, each cell holds one symbol

– The reading is from left to right, one symbol at a time– Can sense the end of the input string

• Storage device– Consist of an unlimited number of cells– Each cell holds a symbol– Can be read and changed

• Control unit– Internal states– Transition function (or next-state): gives the next state of the control

unit • Current state• Current input symbol• Current information in the temporary storage

116

To Illustrate “State”

State: status, A state stores information about the past, i.e. it reflects the input changes from the system start to

the present moment Example: a robot

117

• Configuration– A particular state of the control unit, input file, and temporary

storage• Move

– The transition of the automaton from one configuration to the next

• Deterministic automata– Each move is uniquely determined by the current

configuration– If the internal state, the input, and the contents of the

temporary storage are known, the future behavior of the automata can be predicted

• Nondeterministic automata– At some point, the automata may have several moves– A set of possible actions can be predicted

• Accepter– An automata whose output response is either “yes” or “no”