Chapter 11Models of Computation

Chapter 11Models of Computation

國立雲林科技大學 電子工程系張傳育 (Chuan-Yu Chang ) 博士Office: ES 709

TEL: 05-5342601 ext. 4337

E-mail: chuanyu@yuntech.edu.tw

國立雲林科技大學 電子工程系張傳育 (Chuan-Yu Chang ) 博士Office: ES 709

TEL: 05-5342601 ext. 4337

E-mail: chuanyu@yuntech.edu.tw

2

What is a Model?What is a Model?

Models are very important way of studying many physical and social phenomena.

The characteristics of modelsCaptures the essence of the real thing

The important properties of the real thing

Probably differs in scale from the real thing

Suppresses details of the real thing

Lacks the full functionality of the real thing.

3

ExamplesExamples

The model might be a physical model or a pencil-and-paper mathematical model.Physical models

Model carsDollsVelcro-covered balls stuck together in a certain way to represent the molecular structure.

Mathematical modelsd = r x t (gives the distance d that a moving vehicle travels as the product of rate r and time t)Weather system

4

What is a Model? (cont.)What is a Model? (cont.)

The modelsMight be a physical model or a pencil-and paper mathematical model.Can enhance our understanding of the phenomena being modeled.Give us a safe and controlled environment to play with “what ifs”Can provide environments for learning and practicing interactions with various phenomena.Can be used as design tools.

But, the information gained is only as good as the assumptions made in building the model.

5

A Model of a Computing AgentA Model of a Computing Agent

We want to construct a model for the “computing agent” that will capture its fundamental properties and enable us to explore the capabilities and limitations of computation in the most general sense.

Constructing a model is to abstract the important properties of the phenomenon being modeled while suppressing lower-level details.

6

Properties of A Computing AgentProperties of A Computing Agent

We shall require that any computing agentCan accept inputCan store information in and retrieve it from memoryCan take actions according to algorithm instructions, and that the choice of what action to take may depend on the present state of the computing agent as well as on the input item presently being processedCan produce output

7

The Turing Machine: Historical ProspectiveThe Turing Machine: Historical Prospective

Computing: something done by electronic computerInterest in the theoretical nature of computation far predated the advent of modern computers.In 1900s, mathematicians are interested in formalizing the nature of proof, with two goals in mind:

Guarantee the correctness of a proofAllow for mechanical theorem proving

In 1931, an Austrian logician named Kurt GidelIn any reasonable system, there will be true statements about arithmetic that cannot be proved using that system (formal system).This led to interest in finding a way to recognize which statements are indeed unprovable in a formal system.Leads to the study of the nature of computation procedure itself.

We will look at the model proposed by Alan Turing.

8

The Turing MachineThe Turing Machine

A Turing machine includes a (conceptual) tape that extends infinitely in both directions.The tape is divided into cells, each of which contains one symbol.The symbols must come from a finite set of symbols called the alphabet.The alphabet always contains a special symbol b (for “blank”), usually both of the symbols 0 and 1, and others.At any point in time, only a finite number of cells contain nonblank symbols.

9

The Turing Machine (cont’d)The Turing Machine (cont’d)

Example:

The tape will be used to hold input to the Turing machine.The Turing machine will write its output on the tape.The tape will also serve as memory.

. b b 0 1 1 b b . .. .

10

The Turing Machine (cont’d)The Turing Machine (cont’d)

The rest of the Turing machine consists of a unit that reads one cell of the tape at a time and writes a symbol in that cell.

There is a finite number k of “states” of the machine, labeled 1,2,…,k, and at any moment, the unit is in one of these states.

A Turing machine configuration:

. b b 0 1 1 b b . .. .

1 (current state of the machine)

11

The Turing Machine (cont’d)The Turing Machine (cont’d)

The Turing machine is designed to carry out only one type of primitive operation.Each time such an operation is done, three actions take place:

Write a symbol in the cellGo into a new stateMove one cell left or right

The details of the actions depend on the current state of the machine and on the contents of the tape cell currently being read (the input).

12

Primitive Operation (Instruction)Primitive Operation (Instruction)

Each instruction tells what to do for a specific current state and current input symbol.

If you are in state iand

you are reading symbol jthen

write symbol k onto the tapego into state smove in direction d

13

Shorthand Notation for Turing Machine InstructionsShorthand Notation for Turing Machine Instructions

Five components:Current state

Current symbol

Next symbol

Next state

Direction of move

(current state, current symbol, next symbol, next state, direction of move)

14

ExampleExample

If you are in state 1and

you are reading symbol 0then

write symbol 1 onto the tapego into state 2move right

15

Example (cont.)Example (cont.)

(1,0,1,2,R)

. b b 0 1 1 b b . .. .

1 (current state of the machine)

. b b 1 1 1 b b . .. .

2 (current state of the machine)

16

More on Turing Machine InstructionsMore on Turing Machine Instructions

What if we have (1,0,1,2,R) and (1,0,0,3,L)Avoid ambiguity by requiring that a set of instructions for a Turing machine can never contain two different instructions of the form (i,j,-,-,-) and (i,j,-,-,-) If there is no instruction that applies to the current state-current symbol for the machine, then the machine halts.Conventions about initial configuration:

The start-up will always be state 1The machine will be reading the leftmost nonblank cell on the tape.

17

1. (1,0,1,2,R)

2. (1,1,1,2,R)

3. (2,0,1,2,R)

4. (2,1,0,2,R)

5. (2,b,b,3,L)

ExampleExample

. b b 0 1 1 b b . .. .

1 (1,0,-,-,-) ，選擇指令 1 來執行

Original

18

Example (cont’d)Example (cont’d)

. b b 1 1 1 b b . .. .

2

. b b 1 0 1 b b . .. .

2

. b b 1 0 0 b b . .. .

2

. b b 1 0 0 b b . .. .

3

(2,1,-,-,-) ，選擇指令 4 來執行

(2,1,-,-,-) ，選擇指令 4 來執行

(2,b,-,-,-) ，選擇指令 5 來執行

(3,0,-,-,-) ，無對應指令可執行，所以 Turing machine 完成計算。

19

Turing Machine as a Computing AgentTuring Machine as a Computing Agent

The Turing machine can:Accept input

Read symbols on its tape.

Store information in and retrieve it from memoryWrite symbols on its tape and, by moving around over the tape, can go back and read those symbols at a later time.

Take actions according to algorithm instructions, and the choice of action to take may depend on the present state of the computing agent and on the input item presently being processed.

The present state and symbol determine the appropriate instruction

If the Turing machine halts, what is written on the tape at that time can be considered output.

20

SummarySummary

A Turing machine is a general computing machine.

It has no limit on the amount of memory.

21

A Model of An AlgorithmA Model of An Algorithm

An algorithm is a collection of instructions intended for a computing agent to follow.Requirements:

Be a well-ordered collectionThe Turing machine must know which operation to carry out first and which to do next at any step.

Consists of unambiguous and effectively computable operationsEach individual Turing machine instruction describes an operation that is unambiguous, and any Truing machine is able to carry out the operation described.

Halts in a finite amount of timeWhen executing a collection of instructions, it must reach a configuration where no appropriate instruction exist.

Produce a resultWhatever is written on the tape when the machine halts is the result.

22

Turing Machine ExamplesTuring Machine Examples

A state diagram is a visual representation of a Turing machine algorithm where circles represent states, and arrows represent transitions from one state to another.Along each transition arrow, we show three things:

The input symbol that causes the transitionThe corresponding outputThe direction of move

23

Example 1: A Bit InverterExample 1: A Bit Inverter

Bit InverterMove along its tape inverting all of the bits.

The complete Turing machine program for the bit inverter is

(1,0,1,1,R) Change the symbol 0 to 1.

(1,1,0,1,R) Change the symbol 1 to 0.

State 1

1/0/R 0/1/R

The input symbol that caused the transition

The corresponding output symbol to be printed

The direction of move

24

Example 1:A Bit Inverter (cont.)Example 1:A Bit Inverter (cont.)

Example: beginning with the string 1101 on the tape

. b 1 1 0 1 b . . .. .

1 (1,1,-,-,-) ，選擇指令 2 來執行. b 0 1 0 1 b . . .. .

1 (1,1,-,-,-) ，選擇指令 2 來執行

. b 0 0 0 1 b . . .. .

1 (1,0,-,-,-) ，選擇指令 1 來執行

. b 0 0 1 1 b . . .. .

1 (1,1,-,-,-) ，選擇指令 2 來執行. b 0 0 1 0 b . . .. .

1 (1,b,-,-,-) ，停止執行

25

Example 2: A Parity Bit MachineExample 2: A Parity Bit Machine

Odd parity bit: will be set such that the number of 1s in the whole string of bits, including the parity bit, is odd.

State diagram for parity bit machine

State 1 State 2

State 3

0/0/R1/1/R 0/0/R

b/0/R

1/1/R

b/1/R

表示 even parity state

表示 odd parity state

表示 final state

26

Example 2: A Parity Bit Machine (cont)Example 2: A Parity Bit Machine (cont)

The Turing machine program of parity bit machine(1,1,1,2,R): even parity state reading 1, change state(1,0,0,1,R):even parity state reading 0, don’t change state(2,1,1,1,R): odd parity state reading 1, change state(2,0,0,2,R):odd parity state reading 0, don’t change state(1,b,1,3,R): end of string in even parity state, write 1 and go to state 3(2,b,0,3,R): end of string in even parity state, write 1 and go to state 3

27

Example 2: A Parity Bit Machine (cont)Example 2: A Parity Bit Machine (cont)

The initial string is 101

. b b 1 0 1 b b . .. .

1 (1,1,-,-,-) ，選擇指令 1 來執行. b b 1 0 1 b b . .. .

2 (2,0,-,-,-) ，選擇指令 4 來執行

. b b 1 0 1 b b . .. .

2 (2,1,-,-,-) ，選擇指令 3 來執行

. b b 1 0 1 b b . .. .

1 (1,b,-,-,-) ，選擇指令 5 來執行

. b b 1 0 1 1 b . .. .

3 (3,b,-,-,-) ，停止執行

28

Unary IncrementingUnary Incrementing

Unary representation of numbers:In unary representation, any number n will be encoded by a sequence of n+1 1s.

Number Turing machine tape representation0 11 112 1113 1111. …..

29

Unary Incrementing (cont.)Unary Incrementing (cont.)

Incrementing (algorithm 1):We need only stay in state 1 and travel over the string of 1s to the right-hand end.

When we encounter the first blank cell, we write a 1 in it and go to state 2

(1,1,1,1,R): pass to the right over 1s

(1,b,1,2,R): Add a single one at the right hand side of the string

State 1 State 2

1/1/R

b/1/R

30

Unary Incrementing (cont.): Algorithm 1Unary Incrementing (cont.): Algorithm 1

. b b 1 1 1 b b . .. .

1

. b b 1 1 1 b b . .. .

1

. b b 1 1 1 b b . .. .

1

. b b 1 1 1 b b . .. .

1

. b b 1 1 1 1 b . .. .

2

31

Incrementing: Algorithm 2Incrementing: Algorithm 2

Moving to the left-hand end of the string and adding a 1.

(1,1,1,1,L): pass to the left over 1s(1,b,1,2,L): Add a single 1 at the left-hand end of the string

Time complexity: 2 steps vs. n+2 stepsSpace complexity: both algorithm use n+2 cells

State 1 State 2

1/1/L

b/1/L

32

Unary Incrementing (cont.): Algorithm 2Unary Incrementing (cont.): Algorithm 2

. b b 1 1 1 b b . .. .

1

. b b 1 1 1 b b . .. .

1

. b 1 1 1 1 b b . .. .

2

33

Unary Addition MachineUnary Addition Machine

The two input numbers on the tape are separated by a single blank.

When the Tuning machine halts, the tape should contain the unary representation of the sum of the two numbers.

Exampleoriginal tape: …bb111b1111bb…final representation:..bbbb111111bb…

…bbb11b1111bb… (Erase the leftmost 1)

…bbbb1b1111bb… (Erase a second 1 from the left end)

…bbbbbb1111bb… (move to right and fill in the blank)

…bbbbb11111bb…

34

Unary Addition Machine (cont.)Unary Addition Machine (cont.)

(1,1,b,2,R): erase the leftmost 1 and move right

(2,1,b,3,R): erase the second 1 and move right

(3,1,1,3,R):pass over any 1s until a blank is found

(3,b,1,4,R):write a 1 over the blank and halt.

State 1 State 2 State 3 State 4

1/b/R 1/b/R b/1/R

1/1/R

35

The Church-Turing ThesisThe Church-Turing Thesis

是否有演算法可做，但 Turing machine 無法實現的工作？是否有演算法可做的符號操作工作，但 Turing machine無法實現的工作？

Church-Turing Thesis

If there is an algorithm to do a symbol manipulation task, then there is a Turing machine to do that task.

If we find a symbol manipulation task that no Turing machine can perform, then there is no algorithm for this task.

Turing machines define the limits of Computability.

YES

No

36

The Church-Turing Thesis (cont.)The Church-Turing Thesis (cont.)

There are two really two parts to writing a Turing machine for a symbol manipulation task.

Encoding symbolic information as strings of 0s and 1s so that it can appear on Turing machine tapes.

Write the Turing machine instructions that will produce the encoded form of the correct output.

37

The Church-Turing Thesis (cont.)The Church-Turing Thesis (cont.)

Emulating an Algorithm by a Turing Machine

Bit string on tape when Turing machine starts

Bit string on tape when Turing machine halts

TuringMachine

Symbolic input Symbolic output

Algorithm

Encoding Decoding

38

The Church-Turing Thesis (cont.)The Church-Turing Thesis (cont.)

To perform this emulation, we mustEncode the symbolic input into a bit string on a Turing machine tape.

Write the Turing machine program that solves the problem.

Decode the resulting bit string into symbolic output

39

The Church-Turing Thesis (cont.)The Church-Turing Thesis (cont.)

ThesisIt is “ statement advanced for consideration and maintained by argument”

TheoremTheorems are ideas that can be proved in a formal, mathematical way.

The Church-Turing thesis can never be proved, because the definition of an algorithm is still descriptive, not mathematical.If we find a symbol manipulation task that no Turing machine can perform, then there is no algorithm for this task.

40

Unsolvable ProblemsUnsolvable Problems

Example:(1,0,1,2,R)(1,1,0,2,R)(2,0,0,2,R)(2,b,b,2,L)…b11bbb…

2…b1bbb…

1(It will never halt)

41

The Halting ProblemThe Halting Problem

在較複雜的情況下，如果我們知道 Turing machine 程式和磁帶的初始內容，我們很難決定此 Turing machine 最後是否會停止。

我們可以藉由設定 time-out 機制來判斷 Turing machine 是否會停。但如果在時間內未停，是否就意謂他不會停？

42

The Halting ProblemThe Halting Problem

Formal statement: Decide, given any collection of Turing machine instructions together with any initial tape contents, whether that Turing machine will ever halt if started on that tape.Can we find one Turing machine that will solve every instance of this problem?

That will give us the answer “Yes halts” or “No, never halts”

The halting problem is unsolvable, no Turing machine exists to solve this program.

we will prove it by contradiction.以矛盾證法證明：假設此機器確實存在，找出並非所有輸入均滿足此機器。

43

ProofProof

Assume that P is a Turing machine that solves the halting problem. On the initial for P we will have to put a description of a collection T of Turing machine instructions, as well as the initial tape content t on which those instructions run.The initial tape for P looks like

…bbT*btbbb… (T* symbolize the binary form of T)

44

Proof (cont.)Proof (cont.)

P will give us an answer Yes, halts (1)

No, never halts (0)

When begun on a tape containing T* and t,P halts with 1 on its tape exactly when T eventually halts when begun on t

P halts with 0 on its tape exactly when T never halts when begun on t

Refer to Figure 10.10

45

Proof (cont.)Proof (cont.)

T*bt

P

Input

outputHalts with 1 on tape exactly when T eventually halts on t

Halts with 0 on tape exactly when T never halts on t

46

Proof (cont.)Proof (cont.)

Let’s imagine adding more instructions to P to create a new machine Q that behaves just like P except that when it reaches this same configuration, it moves forever to the right on the tape instead of halting. (Refer to Figure 10.11)

Example: Pick some state not in P, say 52, and add two new instructions to P:

(9,1,1,52,R)

(52,b,b,52,R)

47

Proof (cont.)Proof (cont.)

T*bt

Q

Input

outputNever halts exactly when T eventually halts on t

Halts with 0 on tape exactly when T never halts on t

48

Proof (cont.)Proof (cont.)

Finally, create a new machine S which first makes a copy of what appears on its input.After S is finished with its copying job, it uses the same instruction as machine Q.What happens when machine S is run on a tape that contains S*, the binary representation of S?S first makes a copy of S* and then turn the computation over to Q, which is now running on a tape containing S* and S*.S running on input S* never halts exactly when S halts running on S*.S running on input S* halts with 0 on the tape exactly when S never halts running on S*.

49

Proof (cont.)Proof (cont.)S*

S*bS*

S

Input

outputNever halts exactly when S eventually halts on S*

Halts with 0 on tape exactly when S never halts on S*

Contradiction!

50

Practical Unsolvable ProblemsPractical Unsolvable Problems

The unsolvability of the halting problem says that is not possible… that such a problem cannot exist.No C++ program can be written to decide whether any given C++ program always stops eventually, no matter what input.No C++ program can be written to decide whether any two C++ programs are equivalent.No C++ program can be written to decide whether any given C++ program on any given input will ever produce some specific output.