Formal Models of Computation

March 21

Here’s a big question:

• Given an arbitrary specification, are we guaranteed that there is a program

that meets the specification?

To answer this question, we need to say what we mean by “specification”

and “program”. This is the topic of formal models of computation.

1 Specification/Problems/Languages
In this class, we’ll take a very simple notion of specification: a specification (or
“problem” or “language”) is just a set of strings of characters from a finite set
(called the “alphabet”). I’ll use Σ to denote the alphabet and Σ∗ to denote the
set of finite length strings. We’ll use  to denote the empty string.
For example, here are some specifications:
• The set of all strings (this is implemented by the program that always
says “yes”)
• The empty set (implemented by the program that always says “no”)
• The set of all strings of balanced parentheses
• The set of all strings of the form “n,m,k” where n = mk.

We’ll consider programs that simply output “yes” or “no”. A program meets
a specification L if it outputs “yes” on input x if and only if x ∈ L. We’ll refer
to the set of strings on which a program P outputs “yes” as the “language of
P ”, denoted L(P ). Formally:

L(P ) = {x ∈ Σ | P outputs “yes” on input x}

We say that P “recognizes” L(P )

1
2 First answer to the computability question
I haven’t yet specified what a “program” is. For a moment, let’s pick a concrete
and intuitively appealing definition: let’s say that a “program” is just a string
containing Java source code for a function that takes a string as input and
returns a boolean. For simplicity, let’s interpret any string that is not valid
Java source code as a program that says “no” on every input - thus every string
can be interpreted as a program.
We can now state the “big question” asked earlier more formally, and eas-
ily provide an answer. We asked if every language L has a program P that
recognizes it.
The set of languages is just the set of sets of strings:

L = P(Σ∗ )

The set of programs is the set of strings:

P = Σ∗

We have the function L(·) that takes a program and gives the language of
the program:

L(·) : Σ∗ → P(Σ∗ )

Asking whether every specification has a program is the same as asking if

L(·) is onto.
The answer is clearly no, because |P(Σ∗ )| > |Σ∗ |, so there are no onto func-
tions from Σ∗ to P(Σ∗ ).

3 Deterministic Finite Automata (DFAs)

This is a very interesting fact with an unenlightening proof. What languages
don’t have corresponding programs? Can we characterize the ones that do
somehow?
We’ll start by studying a very simple model of computation (much simpler
than Java!). Instead of strings containing Java text, we’ll talk about programs
that are built using state machines that process strings one character at a time
in a very simple way.
These machines will have a collection of “states”, and each time they process
a character they will “transition” to a new state. Here is a picture of a machine
that recognizes strings of odd length:

0, 1
q0 q1
0, 1

2
 When processing a string, the machine starts in state q0 (as indicated by the
arrow pointing at q0 . When it is in a given state, it reads a character from the
input, and follows the edge corresponding to that character from the state it is
in to find a new state.
It will say “yes” on the string x if it is in the accept state (designated by the
double circle) after processing x completely.
If we run this machine on the input “010”, it will start in q0 . It will read
“0” which causes it to transition to q1 . It will read “1”, causing it to transition
back to q0 . Finally, it will read “0”, and transition back to q1 . Since q1 is an
accept state, it will say “yes”.

4 Formalism
We can encode all of the parts of the picture symbolically as follows. A machine
M consists of the following:
• A finite set Q of states ({q0 , q1 } in the example)
• An initial state q0 ∈ Q. (q0 in the example)
• A set of “accepting states” A ⊆ Q ({q1 } in the example)
• A transition function δ : Q × Σ → Q. In the example, δ(q0 , 0) = δ(q0 , 1) =
q1 and δ(q1 , 0) = δ(q1 , 1) = q0 .
Using this encoding, we can give a crisp definition of the informal process
described above. We can use δ to define a function δ̂ : Q×Σ∗ → Q that gives the
state of the machine after processing an entire string. For the machine above,
δ̂(010) = q1 .
δ̂ is defined inductively. For any state q,
• δ̂(q, ) = q

• δ̂(q, ax) = δ̂(δ(q, a), x)

Then M accepts x if δ(q0 , x) ∈ A.

5 Nondeterministic Finite Automata (NFAs)

It is also useful to loosen the restrictions on building machines to allow multiple
transitions on the same character, or no transitions at all. This allows us to more
easily express some constructions, but turns out not to add any computational
power (as we will see later).
We will also add “ transitions,” allowing the machine to transition from one
state to another without consuming any input.
A machine will accept the input x if there is ANY path from the start state
to an accepting state that is labeled by x.
Here is an example NFA:

3
 
q1 q3
1
0 1
1
q0 q2 0
1

On the input 010, this machine will operate as follows. It begins in state q0 .
After processing the first 0, it can transition to state q1 , and it can also take
the  transition to state q3 . So δ̂(q0 , 0) = {q1 , q3 }.
It will then process the 1, allowing it to transition to state q0 (from state
q1 ) or to state q2 (from state q3 ). Finally it will process the second 0, allowing
it to end in state q1 , q2 , or q3 . Since at least one of these is an accept state, the
string 010 will be accepted.
Another way to look at the process is that we can insert some s into the
string 010 so that it gives a path from the start state to the accepting state: we
can expand it to 010, and take the path q0 → q1 → q3 → q2 → q2 . Since q2 is
an accepting state, we accept.

6 NFA Formalism
We can extend the formal definitions we gave for DFAs. We need to modify
them in two ways:
• δ now returns a set of possible states: δ : Q × Σ → P(Q)
• We must encode the epsilon transitions. We do this using a function
 : Q → P(Q).
This means it’s a little more effort to define the extended transition function,
but it’s not too bad1 :
• δ̂ on the empty string is allowed to take epsilon transitions:
δ̂(q, ) = {q} ∪ (q)

• To get to a state q 000 from state q on input ax, you can take an epsilon
transition to get to q 0 , then process a to arrive at q 00 then process x to get
to q 000 .
δ̂(q, ax) = {q 000 ∈ δ̂(q 00 , x) | q 00 ∈ δ(q 0 , a) and q 0 ∈ (q)}

The machine will accept the input x if there is any final state q ∈ A reachable
from the start state:
L(M ) = {x ∈ Σ | ∃q ∈ δ̂(q0 , x) such that q ∈ A}

1 this is slightly different then the definition I gave in lecture, which had a small bug

4
7 Summary of notation and definitions
There’s a lot of notation in this lecture. Here’s a summary for quick reference.

7.1 Languages
• Σ is a finite alphabet; Σ∗ is the set of finite strings of Σ
•  is the empty string.
• a language is a subset of Σ∗
• a program P recognizes the language
L(P ) = x ∈ Σ∗ |P outputs “yes” when run on input x

7.2 DFAs
• DFA stands for “deterministic finite automaton”
• A DFA M is represented by a tuple (Q, A, q0 , δ) where
– Q is a finite set of states
– q0 ∈ Q is an initial state
– A ⊆ Q is a set of final states
– δ : Q × Σ → Q is the transition function
• there is an extended transition function δ̂ : Q × Σ∗ → Q defined as follows:
– δ̂(q, ) = q
– δ̂(q, ax) = δ̂(δ(q, a), x)
• M = (Q, A, q0, δ) outputs “yes” on x if δ̂(q0 , x) ∈ A

7.3 NFAs
• NFA stands for “nondeterministic finite automaton”
• An NFA is allowed to transition to multiple states on a given input (or no
states)
• An NFA is represented by a tuple (Q, A, q0 , δ(·), (·)) where
– Q, A, q0 are the same as for a DFA
– δ : Q × Σ → P(Q) defines the set of transitions out of q
–  : Q → P(Q) defines the  transitions
• δ̂ : Q × Σ → P (Q) is like δ̂ for DFAs but accounts for  transitions and
nondeterminism:
– ˆ(S) = ∪q∈S {q} ∪ (q)
– δ̂(q, ax) = {q 000 ∈ δ̂(q 00 , x) | q 00 ∈ δ(q 0 , a) and q 0 ∈ (q)}