MELJUN CORTES -- AUTOMATA THEORY LECTURE - 19

7/31/2019 MELJUN CORTES -- AUTOMATA THEORY LECTURE - 19

http://slidepdf.com/reader/full/meljun-cortes-automata-theory-lecture-19 1/35

CSC 3130: Automata theory and formal languages

Andrej Bogdanov

http://www.cse.cuhk.edu.hk/~andrejb/csc3130

Undecidable problems for CFGsand descriptive complexity



Decidable vs. undecidable

“TM M accepts w ”

“TM M accepts some input”

“TM M and M’ acceptsame inputs”

“TM M accepts all inputs”

undecidable

“TM M halts on w ” “PDA M accepts w ”

“DFA M accepts w ” decidable

“PDA P accepts all

inputs” “CFG G is ambiguous”

other kinds of problems?

?

“DFA M accepts allinputs”



›0q0l0o0t0u0s0

›0o0q6o0t0u0s0›0o0k6q3t0u0s

›0o0k6r0q0u0s0›0o0k6r0a0q1s

›0o0k6r0qaa0☐

Computation is local

lotus

ootus

oktus

okrus

okras

okra

M q 6

q 0

q 3

q 0

q 1

q acc

The changes between rows occur in a 2x3 window

computation

tableau



Computation histories as strings

• If M halts on w , We can represent thecomputation tableau by a string t over alphabetG∪ Q∪{#, ›}›0q0l0o0t0u0s0



›0o0k6r0qaa0☐

›q0lotus#›oq6otus#...#›okrqaa☐#

M accepts w q a occurs in string t

M rejects w q a does not occur in t



Undecidable problems for PDAs

• Theorem

• Proof: We will show that

ALL PDA = {〈P 〉: P is a PDA that accepts allinputs}

The language ALL PDA is undecidable.

If ALL PDA can be decided, so can A TM.



Undecidable problems for PDAs

〈 M 〉, w

reject

if M accepts w

accept

if M rej/loops w

A reject if not

accept if P accepts all inputs 〈P 〉

A 〈P 〉

P accepts all inputs if M rejects or loops on w

P does not accept some input if M accepts w



Undecidability via computation histories

P accepts all inputs if M rejects or loops on w P does not accept some input if M accepts w

P candidate computation

history of M on w

reject accepting

histories

›q0lotus#›oq6otus#...#›okrqaa☐ reject

acceptevery other string

M accepts w P rejects t

M rej/loops on w no accepting historiesP accepts everything



Undecidability via computation histories

• Task: Design a PDA P such that

P candidate computationhistory t of M on w

reject acceptinghistories

›0q0l0o0t0u0s0


›0o0k6r0q0u0s0

›0o0k6r0a0q1s›0o0k6r0qaa0☐

Expect t of the form w 1#w 2#...#w k#

If w 1 ≠›q0w , accept t .

If t does not contain qa, accept t .

If two consecutive blocks w i #w i+1 do not correspond to a proper transition of M , accept t .



Implementing P

If w 1 ≠›q0w , accept t .

If two consecutive blocks w i #w i+1 do notrepresent a valid transition of M , accept t .

On input t :

Nondeterministically make one of the following choices

If t does not contain qa, accept t .

Look for the beginning of the i th block of t

Look in the first block w 1 of t

Look for the appearance of qa

›0o0k6q3t0u0s #

›0o0k6r0q0u0s0

w i #w i+1 represents a valid transition if all 3x2windows correspond to possible transitions of M

valid transition



Valid and invalid windows

… 6c3a0t0 …

… 0c6a0p0 …0

… 6t3t0u0 … … 0t6t0u0 …0

valid window

invalid window

… 6t3t0u0 … … 0t6t0q3 …0

valid window

… 6t3q3u0 …

… 0t6a0q7 …0

valid if d(q 3, u ) = (q 7, a, R)

… 6q3t0u0 … … 0k6t0q0 …0

invalid window

… 6c3a0t0 … … 0b6a0t0 …0

valid window



Implementing P

• To check this it is better to write t in

boustrophedon

w i #w i+1 represent a valid transition of M

›0q0l0o0t0u0s0 ›0o0q6o0t0u0s0›0o0k6q3t0u0s


›0o0k6r0qaa0☐

›q0lotus#›oq6otus#...#›okrqaa☐#

›q0lotus#sutoq6o›#...#›okrqaa☐#

Alternate rows are written in reverse



Implementing P

›0o0k6q3t0u0s #

›0o0k6r0q0u0s0

proper transition

…#›okq3tus#suq0rko›#…

w i w i+1

Nondeterministically look for beginning of 3x2 window

w i #w i+1 represent a valid transition of M

#

Remember first row of window in state

Use stack to detect beginning of second row

Remember second row of window in state

If window is not valid, accept, otherwise reject.



The Post Correspondence Problem

• Input: A set of tiles like this

• Given an infinite supply of such tiles, can youmatch top and bottom?

bab

cc

c

ab

a

ab

baa

a

bab

cc

c

ab

a

ab

baa

a

c

ab

a

baba

a

baba

bab

e

bab

e



Undecidability of PCP

• Theorem


PCP = {D : D is a collection of tiles thatcontains a top-bottom match}

The language PCP is undecidable.

If PCP can be decided, so can A TM.




• Idea: Matches represent accepting histories

〈 M 〉, w T (collection of tiles)

If M accepts w , then T can be matched

If M rej/loops on w , then T cannot be matched

›q0lotus#›oq6otus#›okq3t...#›qa☐☐☐☐

›q0

lotus#›oq6

otus#›okq3

r...#›qa☐☐☐☐

e

›q0lotus# ›q0l

›oq6 t

t u

u s

s #

# ›

›

oq60

okq3 o

o …



Some technicalities

• We will assume that – Before accepting, TM M erases its tape

– One of the PCP tiles is marked as a starting tile

• These assumptions can be made without loss of generality (we will see why later)

bab

cc

c

ab

a

ab

baa

a

a

baba

s




• To decide A TM, we construct these tiles for PCP

〈 M 〉, w T (collection of tiles)

If M accepts w , then T can be matched

If M rej/loops on w , then T cannot be matched

e

›q0w#

sa1qia3

b1b2b3 for each valid

window of this form

a

a for all a in

G∪{#, ›}

#›qa

e

☐#

#

☐

e“final” tiles






• If M rejects on input w , then qr appears on bottomat some point, but it cannot be matched on top

• If M loops on w , then matching keeps going

forever

e

›q0w#

sa1qia3

b1b2b3

a

a

#›qa

e

☐#

#

☐

e



A technicality

• We assumed that one tile marked as starting tile

• We can remove assumption by changing tiles abit

bab

cc

c

ab

a

baba

s

b*a*b*

*c*c

c*

*a*b

*a*

*b*a*b*a

*

“starting tile” begins with *

“ending tile”matches last *

“middle tiles”



Ambiguity of CFGs

AMB = {G: G is an ambiguousCFG}

• Theorem


The language AMB is undecidable.

If AMB can be decided, so can PCP .



Ambiguity of CFGs

• Proof:

Step 1: Number the tiles

T G

If T can be matched, then G is ambiguous

If T cannot be matched, then G is unambiguous

(collection of tiles)

bab

cc

c

ab

a

ab

1 2 3

(CFG)



Ambiguity of CFGs

T G(collection of tiles)

bab

cc

c

ab

a

ab

1 2 3

Productions: T → bab T1

Terminals:

B → ccS1

T → c T2

B → abB2

S → a T3

B → abB3

a,b,c,1,2,3

(CFG)

Variables: S, T, B

T → e

B → e

S → T | B



Ambiguity of CFGs

• Each sequence of tiles gives two derivations

• If the tiles match, these two derive the samestring

bab

cc

c

ab

1 2c

ab

2

S → T → bab T1 → babc T21→ babcc221

S → B → ccB1 → ccabB21→ ccabab221





Descriptive complexity





Randomness

• If we write E for even, O for odd, what we saw is

• It seems the wheel is crooked. If it wasn’t wewould expect something more like

• But both sequences have same probability! Whydoes one appear less random than the other?

OEOEOEOEOEOEOEOEOEOE

OOOEEOEOOEOEOOOEEEOE



Turing Machines with output

• The goal of a Turing Machine with output is towrite something on the output tape and go intostate q halt

M

output tape

… 0 1 0

work tape

… 0 1 0



Descriptive complexity

• The descriptive complexity K( x ) of x is theshortest description of any Turing Machine thatoutputs x

• We will assume x is long Andrey Kolmogorov(1903-1987)



Example of descriptive complexity

• Turing machine implementation:

x = “OE...OE” = (OE)n Repeat for n steps: At odd step print O At even step print E

Write n in binary on work tape

While work tape not equal to 0,Subtract 1 from number on work tape

If number is odd, write OIf number is even, write E

( n = 1,000,000,000)

≈ log 2n states

≈ 3 states

≈ 15 states

≈ 2 states

≈ log 2n + 20 K( x )



Bounds on descriptive complexity

• Theorem 1

• Proof: Let x = x 1...x n and consider the followingTM:

For every x of length n , K( x ) is at most O( n )

Write x 1 to output tape and move rightWrite x 2 to output tape and move right

Write x n to output tape and halt.

...

n + O(1)



Descriptive complexity and randomness

• Theorem 2For 99% of strings of length n , K( x ) ≥ n 10.

0 O(log n ) n 10

“simple”

strings111...1, OEOE...OE,

3.14159265, 1212321234321

“random-looking”

strings

n + O(1)

“randomness-deficient”

strings



Evaluating randomness

• How do we know if the casino is crooked?

• Idea: Compute K(sequence).

If much less than n , indicates sequence is not

random

17 6 5 16 5 2

11 8 31 18 11 13

5 2 29 8 1 12

12

14

12

8

31

4



Computing descriptive complexity

• Proof: Suppose it is, fix n and consider this TM M:

Let x = output of M , then

So (when n is large) we get K( x ) > K( x ),

i ibl !

It is not possible to compute K( x ).

Output the first x of length n (in lexicographic order)such that K( x ) ≥ n 10

K( x ) ≥ n 10 K( x ) ≤ |〈 M 〉| = log 2n + O(1) but

Documents

MELJUN CORTES -- AUTOMATA THEORY LECTURE - 19