Understanding Turing Machines and Computability
Understanding Turing Machines and Computability
S E D G E W I C K / W A Y N E S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
• Context
CS.15.A.Turing.Context
Principia Mathematics
Fundamental questions Mathematics. Any formal system powerful enough to express arithmetic. Peano arithmetic
• What is a general-purpose computer? Zermelo-Fraenkel set theory
.
• Are there limits on the power of digital computers? .
.
• Are there limits on the power of machines we can build? Complete. Can prove truth or falsity of any arithmetic statement.
Consistent. Cannot prove contradictions like 2 + 2 = 5.
Decidable. An algorithm exists to determine truth of every statement.
Pioneering work at Princeton in the 1930s.
David Hilbert Kurt Gödel Alonzo Church Alan Turing Q. (Hilbert's Entscheidungsproblem) Is mathematics decidable?
1862 1943 1906 1978 1903 1995 1912 1954
Image sources
http://en.wikipedia.org/wiki/David_Hilbert#/media/File:Hilbert.jpg
http://en.wikipedia.org/wiki/Kurt_Gödel#/media/File:Kurt_gödel.jpg
http://en.wikipedia.org/wiki/Alonzo_Church#/media/File:Alonzo_Church.jpg
http://en.wikipedia.org/wiki/Alan_Turing#/media/File:Alan_Turing_photo.jpg
15. Turing Machines
• Context
• A simple model of computation
• Universality
• Computability
• Implications
CS.15.A.Turing.Context CS.15.B.Turing.Machine
NO
1 0 1 0
3 1 4 2
Characteristics YES
7 1 8 2
• Discrete.
1 0 3 2 4 • Local.
b b a a b b a b b
• States.
7 8
Previous lecture: DFA details and example This lecture: Turing machines
A DFA is an abstract machine with a finite number of states, each labelled Y or N and A Turing machine (TM) is an abstract model of computation.
transitions between states, each labelled with a symbol. One state is the start state. • A string is specified on a tape (no limit on its length).
• Begin in the start state. • The TM reads and writes characters on the tape, moving left or right.
• Read an input symbol and move to the indicated state. • The TM lights "YES" if it recognizes the string, "NO" otherwise.
• Repeat until the last input symbol has been read. • The TM may halt, leaving the result of the computation on the tape.
• Turn on the "YES" or "NO" light according to the label on the final state.
NO
NO
a a a
Does this DFA recognize HALT
Y b N b N this string?
b YES
YES
... ...
b b a a b b a b b
9 10
A Turing Machine is an abstract machine with a finite number of states, each labeled Y, N, Similarities
H, L, or R and transitions between states, each labeled with a read/write pair of symbols. • Simple model of computation.
• Begin in the start state. • Input on tape is a finite string with symbols from a finite alphabet.
• Read an input symbol, move to the indicated state and write the indicated output. • Finite number of states.
• Move tape head left if new state is labeled L, right if it is labeled R. • State transitions determined by current state and input symbol.
• Repeat until entering a state labeled Y, N, or H ( and turn on associated light).
Differences
NO
1:0 #:1
DFAs TMs
R #:# L H HALT
• Can read input symbols from the tape. • Can read from or write onto the tape.
0:1
• Can only move tape head to the right. • Can move tape head either direction.
YES
• Tape is finite (a string). • Tape does not end (either direction).
• One step per input symbol. • No limit on number of steps.
# # # 1 0 1 1 0 0 1 1 1 # # # • Can recognize (turn on "YES" or "NO"). • Can also compute (with output on tape).
11 12
TM example 1: Binary decrementer TM example 1: Binary decrementer
Note:
x:x self-loops are omitted. NO NO
0:1 0:1 #:#
# # # 0
1 0 1 0 1 0 0 0 0 # # # # # # #
1 # # # # 0 0 0 0 # # #
0:1
YES • Move left to right end of x (left of +) . ... # # 1 0 1 1 + 1 0 0 1 # # ...
# # # 1
0 0 1 0 0 1 1 1 1 # # #
• Increment x. ... # # 1 1 0 0 + 1 0 0 1 # # ...
Output 1 0 1 0 1 0 0 0 0
15 16
TM example 3: Binary adder Simulating an infinite tape with two stacks
19 CS.15.B.Turing.Machine
COMPUTER SCIENCE Representing a Turing machine
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
Turing's key insight. A TM is nothing more than a finite sequence of symbols.
dec.txt
decrementer TM
0:1 #:# 3 01# 0
R 0 0 1 0 1 #
R #:# L 1:0 H L 1 2 2 1 0 #
0 1 2 H 2 2 2 0 1 #
• Universality
input Turing machine
• Computability
• Implications Profound implication. We can use a TM to simulate the operation of any TM.
CS.15.C.Turing.Universality 22
NO
Universal Turing machine. A TM that takes as Java simulator gives a roadmap
input any TM and input for that TM on a TM tape. UTM HALT
run with that input (could loop or end in Y, N, or H). UTM HALT
YES
... # # # 1 0 1 1 1 # # #
Easier for us to think about. Implement Java simulator as a TM. Warning. TM development may be addictive. Amazed that it's only 24 states?
The record is 4 states, 6 symbols.
23 24
Universality Conway's game of life
Conway's Game of Life A complex initial configuration for the game of life
Lesson. Simple rules can lead to complicated behavior Q. What happens with this starting configuration?
Example 1. Glider
Church-Turing thesis. Turing machines can do anything that can be described by any Evidence. Many, many models of computation have turned out to be equivalent (universal).
physically harnessable process of this universe: All computational devices are equivalent.
model of computation description
YES
PCP systems string matching puzzles (stay tuned)
... ...
Example of a universal model: Extended Lindenmayer systems for synthetic plants COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
Image sources
http://astronomy.swin.edu.au/~pbourke/modelling/plants
31 CS.15.C.Turing.Universality
COMPUTER SCIENCE Post's correspondence problem (PCP)
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
PCP. A family of puzzles, each based on a set of cards.
• N types of cards.
• No limit on the number of cards of each type.
• Each card has a top string and bottom string.
Does there exist an arrangement of cards with matching top and bottom strings?
PCP. A family of puzzles, each based on a set of cards. PCP. A family of puzzles, each based on a set of cards.
• N types of cards. • N types of cards.
• No limit on the number of cards of each type. • No limit on the number of cards of each type.
• Each card has a top string and bottom string. • Each card has a top string and bottom string.
Does there exist an arrangement of cards with matching top and bottom strings? Does there exist an arrangement of cards with matching top and bottom strings?
Solution 2 (easy): NO. No way to match even the first character! Challenge for the bored: Find a solution that starts with a card of type 0.
35 36
Post's correspondence problem (PCP) Another impossible problem
PCP. A family of puzzles, each based on a set of cards. Halting problem. Write a Java program that reads in code for a Java static method f()
• N types of cards. and an input x, and decides whether or not f(x) results in an infinite loop.
• No limit on the number of cards of each type.
• Each card has a top string and bottom string.
Does there exist an arrangement of cards with matching top and bottom strings? Example 1 (easy). Example 2 (difficulty unknown).
public void f(int x) public void f(int x)
{ {
while (x != 1) while (x != 1)
{ { Involves Collatz conjecture
(see Recursion lecture)
... if (x % 2 == 0) x = x / 2; if (x % 2 == 0) x = x / 2;
else x = 2*x + 1; else x = 3*x + 1;
} }
} }
0 1 2 3 4 N
f(7): 7 22 11 34 17 52 26 13 40 20 10 5 16 8 4 2 1
Halts only if x is a positive power of 2 f(-17): -17 -50 -25 -74 -37 -110 -55 -164 -82 -41 -122 ... -17 ...
A reasonable idea. Write a program to take N card types as input and solve PCP.
A surprising fact. It is not possible to write such a program. Next. A proof that it is not possible to write such a program.
37 38
Definition. A yes-no problem is undecidable if no Turing machine exists to solve it. Liar paradox (dates back to ancient Greek philosophers).
(A problem is computable if a Turing machine does exist that solves it.) • Divide all statements into two categories: true and false.
• Consider the statement "This statement is false."
• Is it true? If so, then it is false, a contradiction.
• Is it false? If so, then it is true, a contradiction.
Theorem (Turing, 1936). The halting problem is undecidable.
true false
2+2=4 2 + 2 = 99
The earth is round. The earth is flat.
Source of the difficulty: Self-reference. Starfish have no brains. Earthworms have 3 hearts.
Venus rotates clockwise. Saturn rotates clockwise.
Profound implications ... ...
• There exists a problem that no Turing machine can solve. This statement is false.✗ This statement is false. ✗
• There exists a problem that no computer can solve.
• There exist many problems that no computer can solve (stay tuned).
Logical conclusion. Cannot label all statements as true or false.
39 40
Proof of the undecidability of the halting problem Proof of the undecidability of the halting problem
Theorem (Turing, 1936). The halting problem is undecidable. Theorem (Turing, 1936). The halting problem is undecidable.
41 42
CS.15.D.Turing.Computability CS.15.E.Turing.Implications
Implications of undecidability Implications for programming systems
Primary implication. If you know that a problem is undecidable... Q. Why is debugging difficult?
A. All of the following are undecidable.
...don't try to solve it!
Hey, Alice. We came up with a
great idea at our hackathon.
We're going for startup funding. Halting problem. Give a function f, does it halt on a given input x?
What's the idea? Totality problem. Give a function f, does it halt on every input x?
No-input halting problem. Give a function f with no input, does it halt?
An app that you can use to
make sure that any app you Program equivalence. Do two functions f and g always return same value?
download won't hang your phone! Uninitialized variables. Is the variable x initialized before it's used? LE
AB
Dead-code elimination. Does this statement ever get executed? C ID
DE
Umm. I think that's UN
undecidable.
Prove each by reduction from the halting problem: A solution would solve the halting problem.
?
Will your app
work on itself ? Q. Why are program development environments complicated?
??? A. They are programs that manipulate programs.
45 46
Lambda calculus E
BL
• Formulated by Church in the 1930s to address the Entscheidungsproblem. DA
E CI
U ND
• Also the basis of modern functional languages.
Alonso Church
1903 1995
Theorem (Church and Turing, 1936). The Entscheidungsproblem is undecidable. Theorem (Post, 1946). Post's correspondence problem is undecidable.
47 48
Examples of undecidable problems from computational mathematics Examples of undecidable problems from computer science
CI
. . .
E Ex. 2 NO E BreakUmOffASlice.Subject = "Important Message From " &
ND BL Application.UserName
U DA
CI
BreakUmOffASlice.Body = "Here is that document you asked for
E ... don't show anyone else ;-)"
U ND . . .
Universality. Concept of general-purpose programmable computers. — John Hodges, in Alan Turing, the Enigma
UTM
=
HALT
YES
Turing's paper was published in 1936, ten years before Eckert and Mauchly worked on ENIAC (!) ... ...
Computer Science
Computer 15. Turing Machines
Science An Interdisciplinary Approach
ROBERT SEDGEWICK
K E V I N WAY N E
Section 7.4
http://introcs.cs.princeton.edu
CS.15.E.Turing.Implications