Computational Complexity

Frank Stephan
Department of Computer Science
Department of Mathematics
National University of Singapore
[email protected]

Computational Complexity – p. 1
 Overview
Lecture Dates in the Semester. The lectures are on
Monday evenings followed by a tutorial (except Week 1)

13 Jan 2025 20 Jan 2025 27 Jan 2025 3 Feb 2025

10 Feb 2025 17 Feb 2025 3 Mar 2025 10 Mar 2025
17 Mar 2025 24 Mar 2025 7 Apr 2025 14 Apr 2025

Midterm Test: 17 March 2025, Second half of ninth lecture.

30 Marks.
Deadline for Homeworks to present: Last tutorial in
Semester; however, it is strongly encouraged to do each
week one homework.
Final Exam: 6 May 2025, 17.00-19.00 hrs, 60 Marks.
www.comp.nus.edu.sg/~fstephan/complexity.html
Computational Complexity – p. 2
 Homework and Task Rules
Marks: 1 marks per homework. Sum of all homework
marks capped at 10 Marks. There are few tasks, these are
comprehensive homeworks requiring more work and can
be done by two students together, each sharing half of the
load and each receives 1 marks for it.
In the Discussions Forum, students can write up only
homeworks not done by others. Homeworks have in the
thread title the following info (several students possible for
tasks):‘
Homework Number, Student Name
For example:
Task 5.9 N. Immerman and R. Szelepcsényi
Check marks in Canvas in Reading Week.
Computational Complexity – p. 3
 Content 1-4
• Lecture 1 on Slide 7
Machine models, Basic steps, Turing machines, Big and
Little Oh, Counter machines, Addition machines of
Floyd and Knuth.
• Lecture 2 on Slide 30
Diagonalisation, Time hierarchies, Number of registers
in Addition Machines, Computation Problems and
Decision Problems.
• Lecture 3 on Slide 52
Space Complexity Classes: LOGSPACE and its
relationship to PTIME.
• Lecture 4 on Slide 77
Nondeterminism and Alternation for logarithmic space.
Savitch’s Theorem. POLYLOGSPACE. Nick’s Class.
Computational Complexity – p. 4
 Content 5-8
• Lecture 5 on Slide 107
Deterministic and Nondeterministic Linear Space,
Context-Sensitive Languages, the LBA problems and
the Boolean closure of nondeterministic space.
• Lecture 6 on Slide 132
P, NP and CoNP, in particular the world of NP-complete
problems.
• Lecture 7 on Slide 156
SAT-type Problems and Exponential Time Algorithms;
The Exponential Time Hypothesis; Exponential Space
Algorithms.
• Lecture 8 on Slide 182
Counting Classes and Counting Algorithms; The
Theorem of Valiant and Vazirani; Probabilistic
Algorithms and the Class RP.
Computational Complexity – p. 5
 Content 9-12
• Lecture 9 on Slide 207
Conditional lower bounds for polynomial time problems.
Lecture 9 will be short and there will be the Midterm
Test in the second half of the lecture time.
• Lecture 10 on Slide 223
Lecture 10 will be on lower and upper bounds for
deterministic and nondeterministic algorithms for
problems in P.
• Lecture 11 on Slide 249
Lecture 11 will be on the Polynomial Hierarchy,
alternating polynomial time and the relations of these to
PSPACE. A further topic is relativised worlds.
• Lecture 12 on Slide 275
Lecture 12 will discuss the situation at PSPACE and
beyond.
Computational Complexity – p. 6
 Lecture 1
Definition 1.1: Machine models are given by the machine
commands plus the primitive steps which count as one step.
The number of steps is then the runtime of the machine.
One basic model is that of a Turing machine. A Turing
machine consists on one or several infinite tapes, a state, a
finite set of tape symbols and commands. For the ease of
notation, the machine is equipped with variables with
constant range. In each step, it can either read the symbol
of one of these tapes into a variable or scroll a tape in some
direction or go (conditionally or unconditionally) to a line
number (which might depend on the outcome of comparing
a variable or constant with the symbol on the tape) or write
a symbol or variable onto the tape.

Computational Complexity – p. 7
 Turing Machine Example 1.2
Example 1.2. This Turing machine only checks whether
there are an even number of zeores and an even number of
ones in the input word. It has bitvariables a, b and starts at
the beginning of line 1.
Line 1: a=0; b=0;
Line 2: If tapesymbol is blank then
begin Move forward; goto line 2 End;
Line 3: If tapesymbol is 0 then a = 1-a;
Line 4: If tapesymbol is 1 then b = 1-b;
Line 5: If tapesymbol is not blank then
begin Move forward; goto line 3 end;
Line 6: If a=0 and b=0 then accept else reject.
The commands “accept” and “reject” communicate a
decision of the Turing machine.

Computational Complexity – p. 8
 Turing Machine Example 1.3
Tape symbols 0,1,2,...,k and blank; one tape;
Output has to replace nonempty input word;
Variables a ∈ {0, 1, 2, . . . , 4 ∗ k}; b, c ∈ {0, 1, ..., k};
Here k is a fixed constant greater than 5.
Line 1: a=0; b=0; c=0; Goto 2;
Line 2: Move forward;
if tapesymbol is blank then goto 2;
Line 3: Move forward;
if tapesymbol is not blank then goto 3;
Line 4: Move backward once;
Line 5: Let b = max{0,tapesymbol}; %% blank<0
a = a+b*3; c = a mod (k+1);
a = a div (k+1); let tapesymbol = c;
Line 6: Move backward; If a > 0 or
tapesymbol not blank then goto 5;
Line 7: Halt.
Computational Complexity – p. 9
 Notes 1.4
Technically, the state is a tuple holding all variable values,
the current line number and the current position inside the
line (if several instructions are in one line). The Turing
machine program mandates does in each situation the
following: It evaluates or updates the tapesymbol; it moves
on the tape forward or backward (if explicitly said so); it
updates variables; it goes to a new line number. Lines can
have several commands, but a goto command always go to
the beginning of a line. At the end of the line, the Turing
machine goes to the next line.
One-tape Turing machines use the tape for everything:
Finding the input written there; writing the output there and
having necessary additional information on the tape during
computation.

Computational Complexity – p. 10
 Example and Quiz
What does this Turing machine compute? It has the
tapealphabet 0,1,2 and blank; two variables ranging over
these four symbols and output has to replace nonempty
input word. The “ts” is the tapesymbol and “bl” is blank.
Line 1: Move forward; if ts = bl then goto 1;
Line 2: Move forward; if ts != bl then goto 2;
Line 3: Move backward;
Line 4: Switch(ts) begin
ts is 0: Let ts = 1; Goto Line 7;
ts is 1: Let ts = 2; Goto Line 7;
ts is 2: Let ts = 0; Goto Line 3;
ts is blank: Let a=1; Goto Line 5 end;
Line 5: Move forward; Let b = ts;
Let ts = a; Let a=b;
Line 6: If a != bl then goto Line 5;
Line 7: Halt.
Computational Complexity – p. 11
 Ressource-Use of Turing Machine
Facts 1.5.
Size (Number of States) is bounded by product of:
(a) Overall number of instructions;
(b) Number of values for each constant ranged variable;
(c) (Tape alphabet size)Number of Tapes .
Input size: Number of symbols except blanks on tapes
where the input is.
Time used: Overall number of steps until halting.
Space used: Number of different fields visited on
bidirectional tapes which can be modified.
If there are designated input and output tapes which are not
used for computation: Do not count towards space use; the
head on input tapes might be bidirectional, but the head on
output tape goes only forward.
Computational Complexity – p. 12
 Time Usage Examples
Examples 1.6. Let n be length of input.
Multiplication with 3: Linear time, O(n).
Checking whether word is palindrome on one-tape
machine: O(n2 ).
Alternating forward pass and backward pass.
Forward pass: Find first unmarked input symbol and last
unmarked input symbol and compare them, mark both after
reading.
If symbols are equal then start backward pass else halt with
output “not palindrome”.
Backward pass: Move backward until first unmarked symbol
is found and go then to Forward pass.
In the case that forward pass or backward pass do not find
unmarked symbols in the above process then halt with
output “palindrome”.
Computational Complexity – p. 13
 Turing Machine Program
Line 1: Move forward; If tapesymbol blank
then Goto 1;
Line 2: Read tapesymbol into a and mark as read;
Line 3: Move forward; if tapesymbol
neither marked nor blank then goto 3;
Line 4: Move backward one step; if tapesymbol
marked then goto Line 9;
Line 5: If tapesymbol different from a
then goto 10;
Line 6: Mark tapesymbol as read;
Line 7: Move backward; If tapesymbol not marked
then goto 7;
Line 8: Move forward; If tapesymbol not marked
then goto 2;
Line 9: Accept (Is Palindrome); Halt;
Line 10: Reject (Is not Palindrome); Halt.
Computational Complexity – p. 14
 Upper Bound
Turing machine goes on word of length n at most n times
forward and backward, as with each double-pass
“forward-backward”, two symbols get marked.
After n/2 + 1 double-passes, all symbols are marked and
the algorithm has reached a decision, either palindrome or
not palindrome.
If it says “not palindrome” after k forward passes, then the
k-th symbols from front and from end are different, thus the
output is correct – here note that the k-th forward pass
marks the k-th symbols from front and from end after
inspecting them.
If it says “palindrome” then no k has been found such that
the k-th symbol from front and from end are different, thus
the output is also correct.
Computational Complexity – p. 15
 Lower Bound
Suppose a one-tape Turing machine wants to to recognise
the set {u0n u : |u| = n}. How to prove that it needs at least
quadratic time?
The method is that of a crossing sequence. Given i, j with
i + j = n, the crossing sequence is the sequence of all
states when the Turing machine crosses the border
between v0i and 0j w for words v, w of length n in order to
check whether v = w.
Assume now that the crossing sequences for v0n v and
w0n w are the same for some i, j with i + j = n. Then the
machine produces on v0n v and v0n w the same output.
Thus there must be at least 2n different crossing sequences
and each crossing sequence has at least linear length. Due
to n + 1 choices for (i, j), the Turing machine has to cross a
border between v and w Ω(n2 ) times.
Computational Complexity – p. 16
 The O-Calculus (Landau Notation)
Definitions 1.7. Assume a Turing machine uses, in the
worst case, T(n) steps for inputs of length n. Then
T(n) ∈ O(f (n)) iff there is a constant c with T(n) ≤ c · f (n)
for all n ≥ c. Furthermore, T(n) ∈ Ω(f (n)) iff there is a
constant c > 1 with T(n) > f (n)/c for all n ≥ c.
T(n) ∈ o(f (n)) iff for all c > 1 there is a constant d such that
for all n ≥ d, T(n) < f (n)/c. T(n) ∈ ω(f (n)) iff for all c > 1
there is a constant d such that for all n ≥ d, T(n) > f (n) · c.
Alternative definition by Hardy and Littlewood:
T(n) ∈ Ω(f (n)) iff there is a constant c > 1 with
T(n) ≥ f (n)/c for infinitely many n ≥ c and T(n) ∈ ω(f (n))
iff for every constant c > 1 there are infinitely many n with
T(n) > f (n) · c. These versions are referred to as Ω′ and ω ′
from now onwards.
T ∈ Θ(f (n)) iff T(n) ∈ O(f (n)) and T(n) ∈ Ω(f (n)).
Computational Complexity – p. 17

′ ′′
 Examples for O-Calculus
Examples 1.8. Polynomials: nk ∈ O(nh ) iff k ≤ h.
O(p((k + 1)n)) = O(p(n)). n4 ∈ Θ(5n4 + 1881). n7 ∈/ O(n5 ).
O(f (n)) = O(g(n)) ⇔ f (n) ∈ Θ(g(n)).
1551 ∈/ O(1/n). n + n2 + n3 ∈ O(n4 ). n4 ∈
/ O(n).
Examples 1.9. Exponentials and Factorials:
2n+1 ∈ O(2n ), 2n+log n ∈
/ O(2n ), 3n ∈ Ω(2n ), 3n ∈
/ O(2n ).
For all k > 1: nn ∈
/ O(kn ), nn ∈ Ω(kn ), (n + 1)n+1 ∈
/ O(nn ).
n! ∈ O(nn ), (n/4)n ∈ O(n!), (n + 1)! ∈
/ O(n!).
For Euler Number e = 2.71828 . . ., n! ∈ Θ((n/e)n ).
Examples 1.10. Step Functions:
T(0) = 1, T(2n + 1) = T(2n), T(2n + 2) = 2T(2n+1) ,
f (n) = T(n + 1), g(0) = 1, g(n + 1) = T(n):
T(n) ∈ O(f (n)), T(n) ∈ Ω′ (f (n)), T(n) ∈
/ Ω(f (n)).
T(n) ∈ / O(g(n)), T(n) ∈ Ω(g(n)), T(n) ∈ ω ′ (g(n)).
Computational Complexity – p. 18
 Counter and Addition Machines
Definition 1.11: Counter machines can increment (add 1),
decrement (minus 1) and compare with 0 (outcomes
positive, zero, negative) their counters. Besides that they
are written with programs like Turing machines and can
also have additional constant-range variables which one
can compile away into a longer (and less understandable)
program.
Counter machines are Turing complete from the viewpoint
of recursion theory, however, they need exponential time for
adding two n-digit integers.
Definitions 1.12: Addition machines are counter machines
which can also add and subtract and which can compare
either variables or variables and constants (outcome
greater, equal, smaller). For those, polynomial time
coincides with polynomial time of Turing machines.
Computational Complexity – p. 19
 Counter Machine Examples 1.13
The following counter machine reads the inputs x, y and
outputs their sum z. As computing the sum overwrites the
values of x and y, it keeps for book keeping purposes the
copy v of x and w of y.
Line 1: Read x; Read y;
Line 2: Let z=0; Let v=0; Let w=0;
Line 3: If x = 0 then goto Line 4;
Decr x; Incr z; Incr v; Goto Line 3;
Line 4: If y = 0 then goto Line 5;
Decr y; Incr z; Incr w; Goto Line 4;
Line 5: Write z;
Using these techniques, one can indeed implement every
addition machine as a counter machine with sufficiently
many counters.

Computational Complexity – p. 20
Convention 1.14 of Addition Machines
In order to simplify the writing of addition machines, the
following is allowed:
• Constant ranged variables use letters at the beginning
of the alphabet and integer ranged registers use letters
at the end of the alphabet; the possible values of
variables must be listed;
• Integer constants and constant ranged variables can be
used for comparing, subtracting, adding, assignment;
• Assignments can have any number of terms on the right
side, furthermore, one can write x = 3y − 2z in place of
x = y + y + y − z − z;
• One can use “begin” and “end” in if-then-else
statements;
• All goto statements go to the beginning of some line.
Computational Complexity – p. 21
 Counting Trailing Zeroes
Note that one has no direct access to the digits of the input
x, one has to compute them. This construction reads out
the digits of x at the top and counts the trailing zeroes.
Line 1: Read x; If x < 1 Then Goto 1;
Let y=1; Let z=10; Let w=0;
Line 2: Let y = 10y; If y < x+1 Then Goto 2;
Line 3: Let x = 10x; Let z=10z;
Line 4: If x = 0 then Goto 5; If x < y Then
goto 3; Let x = x-y; Goto 4;
Line 5: If z = y Then Goto 6;
Let w = w+1; Let z = 10z; Goto 5;
Line 6: Write w; Halt.
Here x has the input, y is the first power of 10 larger than x.
One uses y to remove the digits of x which are shifted out
on the top. z is a counter which counts by multiplying with
10; w is, at the end, the result. Computational Complexity – p. 22
Multiply with Addition Machines 1.15
Example: Multiply 1234 with 111, Running Sum
with Frontshifts
111 0 + 111 = 111
222 1110 + 222 = 1332
333 13320 + 333 = 13653
444 136530 + 444 = 136974
------
136974
Algorithm: Read digits of one number (here 1234) out at the
top and add product of digit with other number to running
sum times 10.
Use binary numbers to avoid multiplication by digits
2,3,4,...,9 and decide only on adding the second number
(bit 1) or skipping the addition of the second number (bit 0).

Computational Complexity – p. 23
 Example with Coding Digit
Example of Multiplication:
12340 (x) times 111 (y) Digit Running sum
1000000 (Comparator v) - w=0
1234010 1 111
2340100 2 1332
3401000 3 13653
4010000 4 136974
0100000 0 1369740
1000000 Coding digit reached, the End
Algorithm x times y:
1. Append Coding digit 1 to x;
2. Comparator v is power of 10 greater x;
3. Shift x up, while (x > v) do x=x-v;
4. Digit is number of subtractions needed;
5. Shift w up, Add digit times y to w;
6. If x = v then stop else goto 3. Computational Complexity – p. 24
 Program for Multiplication
1. Begin Read x; Read y; Let a = 1;
2. If x < 0 Then Begin Let x = −x; Let a = −a End;
3. If y < 0 Then Begin Let y = −y; Let a = −a End;
4. If y < x Then Begin Let v = x; Let x = y; Let y = v End;
5. Let v = 1; Let w = 0; Let x = x + x; Let x = x + 1;
6. If v > x Then Goto 7;
Let v = v + v; Goto 6;
7. Let x = x + x; If v = x Then Goto 8 Else Let w = w + w;
If x > v Then Begin Let w = w + y; Let x = x − v End;
Goto 7;
8. If a = −1 Then Begin Let w = −w End;
Write w; End.

Computational Complexity – p. 25
 Homeworks
Homework 1.16: Provide a two-tape Turing machine which
checks whether the input (on the first tape) is a palindrome
in linear time (using accept / reject to communicate result).
Homework 1.17: Prove a quadratic lower bound for
palindrome-checking with one-tape Turing machines.
Homework 1.18: Provide a quadratic time one-tape Turing
machine which recognises whether the input is of the form
3u2u2u2u2u4 where u is a binary string and the tape
alphabet is decimals plus blank.
Homework 1.19: Provide a quadratic time one-tape Turing
machines computing the square of the binary number on
the input.
Homework 1.20: Provide a proof that one cannot improve
this algorithm to subquadratic time for squaring on one-tape
Turing machines. Computational Complexity – p. 26
 Homeworks
Homework 1.21: Provide a read-only one-tape Turing
machine which accepts an input word if and only if it has
three 1s and five 2s and arbitrarily many 0s and no other
digits.
Homework 1.22: Assume that f (n) > 0 for all n. Prove that
T(n) ∈ O(f (n)) iff there is a constant c such that, for all n,
T(n) ≤ c · f (n). Explain why the assumption that the domain
is the set of natural numbers is needed.
Homework 1.23: Provide a counter machine which
computes the square of an input number. Provide a lower
bound on the computation time needed (in terms of the
number n of binary digits needed to write down the input
number).

Computational Complexity – p. 27
 Homeworks
Homework 1.24: Provide an addition machine which checks
whether at least one of five input numbers to be read
satisfies that its ternary representation uses all three
ternary digits 0, 1, 2 after a leading digit which is either 1 or
2. Try to keep the number of registers needed minimal.
Homework 1.25: Write a linear time addition machine which
checks whether the input x into a product of two numbers y
and z with y being a power of 2 and w being a power of 3. If
this is the case, the machine should accept and output v
and w. Otherwise the machine should reject.
Homework 1.26: Prove identities in Examples 1.8.
Homework 1.27: Prove identities in Examples 1.9.
Homework 1.28: Prove identities in Examples 1.10.

Computational Complexity – p. 28
 Tasks
Task 1.29: Provide a 2-counter machine which checks
whether the input number x given is a palindrome when
written as a ternary number. Give the program explicitly, but
the usage of constant ranged variables is allowed.
Compute a bound on the state complexity of the counter
machine as follows: Count for each line number the number
of instructions in the line and sum these up; multiply the
resulting number with the number of possible values of
each constant-ranged variable used. Explain the
construction in detail.

Computational Complexity – p. 29
 Lecture 2
This lecture is about Complexity Aspects of Addition
Machines. Diagonalisation, Time and Space hierarchies,
Computation Problems and Decision Problems.
The central topic in this chapter is to compare complexity
classes. Most complexity classes are defined in
dependence of certain parameters:
First: What machine model is used;
Second: Which aspects of the machine is during the
complexity of the computation is evaluated for the time used
or the space used or, in the case of addition machines, the
number of registered needed to carry out the computation
fast.
Third: Size of input - usually number of symbols of word or
binary digits of number.
Computational Complexity – p. 30
 Complexity of Turing Machines
Time: The maximum number of steps carried out by the
Turing machine on an input of n symbols.
Space: The overall number of cells visited on tapes which
can be modified and where the head can move in both
directions.
Tapes: The number of tapes needed to perform a task
under certain side-conditions. All things can be computed
on a one-tape Turing machine, if no further requirements
are given, but, for example, recognising palindromes in
linear time requires two tapes. There is a whole hierarchy of
problems which requires so and so many tapes for
computations which satisfy certain side conditions.

Computational Complexity – p. 31
 Complexity of Counter Machines
Time: In the normal range, uninteresting, as even adding
two numbers requires exponential time. However, for larger
classes with preassigned space constraints (like polynomial
space or linear space), one might try to measure the time
used up by a counter machine.
Space: The space used by a counter machine is the
number of binary digits of the largest register value which
appeared during the computation process. This is a
reasonable but uncommon measure.
Number of Registers (“Counters”): This has been
investigated thoroughly. If there are no further constraints,
two counters are enough.

Computational Complexity – p. 32
 Complexity of Addition Machines
Time: How much time does the machine need to computer
a function, say f (x, y), from x and y? Input size is measured
as number of binary digits needed to write out the whole
input (with blanks separating several input numbers).
Speed is measured in number of additions, subtractions
and comparisons and other commands carried out.
Space: Size of the largest register during running time in
dependence of the input size.
Number of Registers: Registers must hold inputs and
outputs and for one-input functions, two registers are
enough without any side-conditions – see Counter
Machines. Floyd and Knuth [1990] investigated the number
of registers needed for linear time addition machines. Value
depends on the exact constraints which operations are
allowed and which not.
Computational Complexity – p. 33
 Lower Bound for Halving
Theorem 2.1 [Stockmeyer 1976]. An addition machine
needs at least linear time for the mapping x 7→ x/2 where
one downrounds the result in the case that x is odd.
Proof. One considers numbers modulo x. The size of a
number y modulo x is the minimum of the number of binary
digits which are needed to write y and x − y, where
0 ≤ y < x. So one starts with input x and follows a given
algorithm. In each instruction, a new value of a register
might be computed which is the sum or difference of old
values. The number of digits (modulo x) increases at most
by a 1 in each addition or subtraction. Thus one needs
log(x/2) − O(1) basic operations (adding and subtracting),
where O(1) comes from integer constants which form the
start of the induction.
Stockmeyer [1976] obtained the same lower bound for
checking whether the input is even or odd.
Computational Complexity – p. 34
 Upper Bound
This algorithm divides by two. A small adjustment would
then also detect whether the number is even or odd (last
value of a). The algorithm is for positive numbers and zero.
Line 1: Read x; Let x = x+x+1; Let z=0;
Let y = 1; Let b=0 (a,b are bitvariables);
Line 2: Let y = y+y; If y <= x then goto 2;
Line 3: Let x = x+x; If x = y then goto 4;
If y < x then begin Let x = x-y; let a=1; end
else let a=0;
Let z = z+z; Let z = z+b; let b = a; Goto 3;
Line 4: Write z.
For input x, one appends a coding bit and reads out the bits
of x at the top using y. The bits will be copied into z except
for the last bit read, that one will be discarded. The
variables a, b are there to store the bits intermediately.
Computational Complexity – p. 35
 Other Linear Time Operations
Register Complexity: How many registers does one need
for linear time operations?
Multiplication: 4 registers sufficient;
Division: 4 registers sufficient;
Remainder: 3 registers sufficient; in the model of Floyd and
Knuth (usage of constant 1 counted as one additional
register, at most one operation + or - per assignment, only
comparisons of two registers) also three registers needed;
Computing Fixed Remainders (Simulation of Finite
Automata): 2 registers;
Outputting powers of 2 in binary representation: 4 registers
sufficient.
Open Problem: Is there a constant number of registers
sufficient for all linear time operations? Most likely not.
Computational Complexity – p. 36
 Constant Time Operations
Constant time operations carry out only constantly many
steps on an addition machine. They are functions without
loops. Example of sorting three numbers.
Line 1: Read x; Read y; Read z;
Line 2: If x > y then
begin let u = x; let x = y; let y = u end;
Line 3: If y > z then
begin let u = x; let x = y; let y = u end;
Line 4: If x > y then
begin let u = x; let x = y; let y = u end;
Line 5: Write x; Write y; Write z.
Four registers suffice to sort three inputs in constant time.

Computational Complexity – p. 37
 Constant Time Operations
One can do the same even with three registers. Algorithm:
Read x, y, z;
Choose the unique case which applies:
If x <= y and y <= z then Write x,y,z;
If x <= z and z < y then Write x,z,y;
If y < x and x <= z then Write y,x,z;
If y <= z and z < x then write y,z,x;
If z < x and x <= y then write z,x,y;
If z < y and y < x then write z,y,x.
One uses “less or equal” when the order of register names
coincides with their value order and “strictly less” otherwise
to avoid double listings.

Computational Complexity – p. 38
 Hierarchy
Theorem 2.2. One can read in and sort and write out n
variables in constant time if and only if one has at least n
registers. Thus the function mapping n inputs to n outputs
which sorts these n numbers in constant time witnesses
that one needs at least n registers.
Proof. The sufficiency is obtained by generalising the
previous example from 3 to n inputs; note that the inputs
are read in some order and might be output in the inverse
order, thus one needs to memorise all n inputs and one
cannot make any memory manipulation operations (like
storing two numbers in one register) in constant time. So for
constant time, there is a strict hierarchy of what can be
computed and memorised in this time.

Computational Complexity – p. 39
 Addition Machine Simulation
Theorem 2.3. There is a constant k such that, given a
number m of registers, then there is a universal machine
with m + k registers which on inputs a number e and then
simulates the e-th program in some enumeration of all
register machine programs such that, whenever the original
program uses on inputs of size n the time f (n) then the
simulating program uses time O(f (n) · size(e)) to do the
same computation. In particular if f (n) is bounded by a
constant c for all inputs then the simulating machine just
needs O(size(e)) to run the program, independently of the
input size.

Computational Complexity – p. 40
 Proof idea I
One uses in addition to the given m registers k additional
registers which have the same function as accumulators in
central processing units and then one reads from the
program (which can be a machine translation of a human
readable program) what to do, for example which registers
to copy into an accumulator or vice versa and what
operations to do with the accumulators, for example
subtract or add the accumulator B to A and to store the
result in accumulator C or to compare the two
accumulators. It is also part of the program to code integer
constants (say in binary) in order to use them as operands
or as line numbers in conditional or unconditional Goto
commands.

Computational Complexity – p. 41
 Proof Idea II
As the program does not modify the register content except
for loading the registers into accumulators and then do the
corresponding operation (addition or subtraction) with them
in one step and as constants are bounded by the size of e,
the overall simulation of a step is just bounded by a function
proportional in size(e) and not influenced by the register
content in the simulated machine.

Computational Complexity – p. 42
 2.4 Simulating Addition Machines I
Task 2.4: Assume that the following commands are allowed
for a register machine, where x, y, z are registers and y, z
can also be integer constants: x = y + z, x = y − z,
unconditional goto, if-then-else commands in dependence
of conditions y < z, y = z, y > z, reading and writing of a
register. Construct a register program in some suitable
programming language - for example, Python allows
arbitrarily large registers - and use additional constant
range variables to reduce the program size so that the
simulation bounds from the preceding theorem are met.
State the size of the constant k, that is, the number of
additional registers, explicitly, constant size variables do not
count towards this value k.

Computational Complexity – p. 43
2.4 Simulating Addition Machines II
Task 2.4 (continued): For the writing of the program, any
machine coding of the program can be used, as long as it
allows to simulate any step handling registers and
accumulators and integer parameters in O(size t) steps of
the simulating machine where the parameter t is the
maximum of all integer constants involved and the value of
register and line number labels; furthermore, a conditional
or unconditional goto command might also invoke
operations of size O(size e) due to the scrolling of the
program (which is held in constantly many registers read
out at the top in some fixed digital coding).
In the case that Python is used, the program could be
augmented by a translator which translates a program,
given as text, into the number e. For other programming
languages, use string data type and explain how to simulate
it. Computational Complexity – p. 44
 Addition Machine Diagonalisation
Given an input of length n, one can compute two numbers
d, e used for diagonalisation as follows:
Line 1: Read x; Let y=1; Let e=1; Let d=1;
Line 2: If d>e then let e=e+1 else
begin d=d+1; e=1 end;
Line 3: Let y=y+y; If y < x then Goto 2;
Using these parameters, one makes a program which does
the following:
Line 4: Simulate the Universal Machine for
(log d) times n steps on input x using
h lines; for this use double up counting
in the same way as for getting n out of x;
....
Line 5+h: If the simulation stops with result y
then Write y+1 else Write 0.
Computational Complexity – p. 45
 Formal Statement
Theorem 2.5. There is an addition machine simulating the
e-th O(n) k-register register machine for n · log d steps on
infinitely many x and outputting on these x something
different. The computation time of the inputs for fixed d, e is
n · log d · O(size e) and is inside O(n log2 n) in general.
Corollary 2.6. Some addition machine with k + ℓ registers
can compute in time O(n log2 n) a function different
functions computed on k register addition machines in time
O(n).
Compressing several registers into one for storing purposes
uses up linear time and similarly for retrieving or modifying
the value of one of the stored registers. Thus the number of
registers is an essential ingredient for above result.

Computational Complexity – p. 46
 Low Turing Machine Complexities
Theorem One-tape Turing machines either need O(n) steps
or Ω′ (n log n) steps.
The proof method analyses crossing sequences. Let xm be
the length-lexicographically first word with a crossing
sequence of length m. If one cuts xm into three parts
u, v, w such that the longest crossing sequence occurs
inside u and at the borders from u to v and v to w occur the
same crossing seuqences, then the word uw still has the
same longest crossing sequence as uvw and thus uvw is
not xm . So in xm there are at least |xm |/2 − 1/2 different
crossing sequences. If c is the number of states of the
Turing machine and (c + 1)k < |xn |/2 − 1/2 then half or
more of the crossing sequences have length k. Let
n = |x + m|; now k ≥ log n / log(c + 1) − 1. The xm witness
that the runtime is at least Ω′ (n log n).
Computational Complexity – p. 47
Computation and Decision Problems
A computation problem or functional computation problem
is the computational complexity to compute a certain
function. For example, Floyd and Knuth showed that in the
addition machine model for linear time computations, the
number of registers needed is an important complexity
measure (besides the time constraint). The side-condition
“linear time” is important, as otherwise many problems like
multiplication and remainder have trivial algorithms.
Computation Problem: The problem to compute a certain
function, often lower bounds are obtained by analysing
output size.
Decision Problem: The problem to compute a {0, 1}-valued
function, here one cannot drive up the complexity by big
outputs. Such functions can be viewed as characteristic
functions of sets or relations.
Computational Complexity – p. 48
 Homeworks 2.7-2.8
Homework 2.7: Explain how to simulate the following
operations with a register machine: Holding a string in k-ary
alphabet as k-ary numbers in two registers with a third
register larger than these and one symbol in a
constant-sized variable and how to use this string as a
program which is used to read out binary numbers as well
as other symbols and how to scroll it until a certain lable is
found.
Homework 2.8: Construct a timer which reads in one input
x of size n which is the first number with 2n > x and then
counts some time variables for n3 steps until the counting
comes to an end. Between each two steps it carries out
some routine (omitted and to be indicated by dots) and it
aborts the program with output 0 if it halts and the routing
returns output y + 1 if it computes output y within the given
time. Computational Complexity – p. 49
 Homeworks 2.9-2.13
Construct Addition Machines which compute as fast as
possible the following functions, where the input x ≥ 1.
Homeworks 2.9 and 2.10 run linear in the size of the output.
Homework 2.9: x 7→ xx .
Homework 2.10: x 7→ x! where x! is the product of all
natural numbers from 1 to x.
Homework 2.11: x 7→ loglog(x) where the logarithm is the
first number y with 2y ≥ x.
Homework 2.12: x 7→ y where x and y have each nonzero
digit the same number of times in their decimal
representations and the digits of y are ordered ascendingly.
So 203318 is mapped to 12338. Explain the algorithm.
Homework 2.13: The same as 2.12, but with ternary
numbers and give the explicit program.
Computational Complexity – p. 50
 Homeworks 2.14-2.17
Homework 2.14: Construct a Turing machine which uses
one tape and simulates a two-tape Turing machine. Show if
the two-tape Turing machine uses O(f (n)) time than the
one-tape Turing machine uses O((n + f (n)) · f (n)) time.
Provide a reason for the delay.
Homework 2.15: Can an exponential time counter machine
simulate an exponential time Addition machine where all
register stay linear in size of the input during the whole
computation? Explain the result.
Homework 2.16: Provide a function which can be computed
by an exponential time addition machine but not by an
exponential time counter machine.
Homework 2.17: Provide asymptotically fast programs to
compute x 7→ log(x) with log(x) = min{y : 2y ≥ x} on
counter machine, addition machines and Turing machines.
Computational Complexity – p. 51
 Lecture 3
This lecture is about space complexity. Usually it is
measured by the space used on a Turing machine with the
following items:
1. One Bidirectional Input Tape;
2. One Bidirectional Worktape of which only bound(n)
symbols can be used where n is the length of the word
on the input tape (the Turing machine has to take track
of this);
3. One Unidirectional Output Tape. Whenever its head
writes a symbol, it afterwards automatically moves
forward one step and it cannot move otherwise, initially
the full tape was blank.
The Turing machine might have two states “Accept” and
“Reject” for answer 1 and answer 0 instead of an output
tape and it halts upon going into one of these states.
Computational Complexity – p. 52
 Space Complexity Classes
The bound function bound can be from a certain class
BOUND of permitted bound functions and one can use
such classes to define space complexity classes.
LOGSPACE: BOUND is the class of all functions f for
which there is a constant c with f (n) ≤ c log(n) for all n ≥ 2.
LINSPACE: BOUND is the class of all functions f for which
there is a constant c with f (n) ≤ c n for all n ≥ 2.
PSPACE: BOUND is the class of all polynomials in one
variable n.
ESPACE and EXPSPACE: BOUND is the class of
functions 2f (n) where f is either a linear function or a
polynomial, respectively.
Addition machines can only be used for complexity classes
from LINSPACE onwards.
Computational Complexity – p. 53
 Refined Complexity Bounds
For Turing machine complexity, one takes mainly into
account the following three complexity parameters:
1. Number of Worktapes (has some influence);
2. Number of Cells visited on work tapes (space measure);
3. Number of computation steps done (time complexity).
For Addition machines, the complexity measures are the
following:
1. Number of registers;
2. Maximum size of a register during computation (space
measure);
3. Number of computation steps done (time complexity).
If one limits the space then the computation time can
become worse, as the next example shows.
Computational Complexity – p. 54
 Palindrome LOGSPACE Example
Theorem 3.1. Assume that a Turing machine has one
bi-directional input tape (read only), k LOGSPACE work
tapes and special Accept / Reject commands (or states).
Then the time complexity of recognising palindromes on
such a machine is Θ(n2 /log(n)).
Algorithm. The algorithm holds the following numbers on
the work tapes: Number n of cells of the input word (one
forward and backward scanning of for counting all); the
number j = log(n) of digits to store n; the number i · j of
digits from front and back already compared; the current j
digits to be compared.
Each of these numbers can be stored on an own work tape,
but one could also unify all these numbers with some
logarithmic overhead onto one work tape to maintain them.
Computational Complexity – p. 55
 Algorithm Continued
One initialises n and its logarithm j by counting the digits of
the input in a forward-backward scan.
Then, for i = 0, 1, . . . , n/j − 2, one reads digits at positions
i · j, i · j + 1, . . . , i · j + j − 1 and stores them on the
corresponding work tape. Then one goes to positions
n − i · j, n − i · j − 1, . . . , n − i · j − j + 1 and compares the
digits there with those on the work tape.
If all comparisons show coincidence then the word is a
palindrome and one says ACCEPT else it is not a
palindrome and one says REJECT.
Time Analysis. Note that the input tape is only scrolled
(from beginning to end and back) O(n/log n) times and that
each such process of scrolling and comparing takes O(n)
time; thus the overall effort is O(n2 /log n).
Computational Complexity – p. 56
 Lower Bound I
For this, one uses that there are only c log n symbols on
the work tape. Now one incorporates the work tape content
into the Turing machine states and obtains a one-tape
′
c
read-only Turing machine with n states for some
sufficiently large c′ independent of n. Now one considers
words v0n w where v, w are binary words. The word v0n w
has to be accepted iff w is the mirror image of v. If there is
now an m ∈ {0, 1, . . . , n} where the crossing sequence
length n′ satisfies for 2n /(n + 1) words v0n vmi that
′ ′
(n ) < 2n /(n + 1) then there are two such words v0n vmi
c n

and w0n wmi sharing the same crossing sequence and

either one of these inputs is rejected or the input v0n wmi is
accepted, both is a mistake.

Computational Complexity – p. 57
 Lower Bound II
Thus one has for all m and for all but less than 2n /(n + 1)
words v0n vmi that the crossing sequence has a length n′
such that n′ · c′ · log n ≥ n − log n. This condition implies
n′ ≥ n/(2c′ log n). In particular there is one word v0n vmi
satisfying this for all m. This word witnesses that on an
input of length 3n the algorithm runs at least Ω(n2 /log n)
steps.

Computational Complexity – p. 58
 Checking Multiplications I
Definition 3.2: The problem MULTIPLICATION CHECK:
Given a product of two numbers and its supposed result,
check whether the result is correct.
Example: 12345 ∗ 67890 = 838102050.
Theorem 3.3: MULTIPLICATION CHECK is in LOGSPACE.
Note that one does not need to do the full computation for
checking. It is sufficient to check modulo numbers
m = 2, 3, . . . , n2 where n is five plus the number of binary
digits. Accepts iff all modulo tests show equality.
Note that LOGSPACE is enough to hold variables
containing n (input length plus five), any value m ≤ n2 and
i, j, k being the remainders by m of the two factors and the
supposed product, respectively.

Computational Complexity – p. 59
 Checking Multiplications II
The remainder i of x by m for binary x is computed as
follows:
Line 1: Let s = 0; Let i=0;
Line 2: Read digit d of x; Let i=i+i+d;
Line 3: If i+1>m Then Let i=i-m;
Line 4: If x not completely read go to Line 2;
The equation whether i · j = k modulo m can be checked as
follows:
Line 1: Let s = 0; Let h=0;
Line 2: Let s = s+j; Let h=h+1;
Line 3: If s < m Then Goto Line 4 Else s=s-m;
Line 4: If h < i Then Goto Line 2;
Line 5: Now i*j=k Modulo m iff s=k.
All variables involved are in {0, 1, . . . , 2m} and can be
stored in LOGSPACE.
Computational Complexity – p. 60
 Prime Number Condition
Theorem 3.4: If m1 , m2 , . . . , mn do not have a common
factor and are all at least 2 then two numbers x and y below
the product of m1 , m2 , . . . , mn are equal if and only if they
are equal modulo each of m1 , m2 , . . . , mn .
Theorem 3.4 is known as the “Chinese Remainder
Theorem”.
Fact 3.5: There are at least n prime numbers below n2 for
n ≥ 5. The product of these n primes is at least 2n , as each
prime is at least 2.
Theorem 3.4 together with Fact 3.5 are sufficient to prove
that the algorithm above is correct, note that all binary
numbers have at most n digits. Thus MULTIPLICATION
CHECK is in LOGSPACE.

Computational Complexity – p. 61
 Classes in Logarithmic Space
The following classes are also in LOGSPACE (without
proof).
USTCON (Omer Reingold 2008): Given an undirected
graph (V, E) and two nodes s, t ∈ V, is there a path from s
to t?
REPETITION: Given a list of binary numbers separated by
commas, are two of these numbers equal?
CLAUSE SET EVALUATION: List of clauses using
x1 , x2 , . . . , xn plus list of Boolean values of these variables.
Check whether all clauses are satisfied.
BRACKET EXPRESSION CORRECTNESS: Given an
expressions using brackets, are all the bracket placed in a
matching order? One can have even different types of
brackets.
Computational Complexity – p. 62
 Alternative Machine Models
The following machine models are equivalent to
LOGSPACE, that is, recognise exactly the languages in
LOGSPACE.
1. One-Tape Read-Only Turing Machines with Multiple
Reading Heads. The heads can detect whether they are on
the same field and whether they have reached the
beginning or the end of the Input.
2. One-Tape Read-Only Turing Machine with additional
Addition Machine Registers whose values are always
bounded by the input length raised to the power of some
constant. So this is some type of hybrid machine. The
hybrid machine has explict Accept and Reject commands in
its programming language and uses only the symbols on
the Input Tape as input. Registers can be added,
subtracted, assigned and compared; operations with
constants and tape symbols are possible. Computational Complexity – p. 63
 Constant Space
Constant Space Machines have only one Read-Only Turing
tape for input and explicit Accept and Reject commands. It
can be required that the input is read in one direction only.
The languages which can be recognised in CONSTANT
SPACE are exactly the regular languages.
Regular languages can be characterised by grammars
satisfying severe restrictions or by Finite Automata (which
are Read-Only One-Way Turing machines) and regular
expressions.
Regular expressions are either finite set of explicitly listed
words or obtained from other regular expressions by one of
the following constructs: Union, Intersection, Concaten-
ation, Kleene Star, Set Difference. Here Kleene Star L∗ of a
language L is the set of all words obtained by concaten-
ating some words from L.
Computational Complexity – p. 64
 Configurations of Turing machines
Definition 3.6: A configuration of a Turing machine is given
as follows:
1. The state (line number and constant-ranged variables)
of the Turing machine;
2. The position on the input tape;
3. The content of the work tapes.
One can now compute that for an f (n) space machine there
are constants c′ , c such that the machine has at most
(n + 1) · c′ · cf (n) configurations. If one configuration appears
twice in a computation, the computation does not terminate.
Thus the f (n) space Turing machine either runs forever or
terminates after at most (n + 1) · c′ · cf (n) steps.
Thus LOGSPACE is contained in P (=PTIME) and PSPACE
is contained in EXP (=EXPTIME). Computational Complexity – p. 65
 LOGSPACE versus P
Open Problem 3.7: Is P = LOGSPACE? This problem is
open since the 1970ies and is not yet resolved.
However, researchers have identified many P-complete
problems. These are problems with the following property:
If A and B are problems in P and B is P-complete, then
there is a LOGSPACE computable function f with x ∈ A if
and only if f (x) ∈ B.
It is conjectured (though not proven) that no P-complete
problem is in LOGSPACE or some other small space class.
The next slide provides the formal definitions for two
P-complete problems, the CIRCUIT VALUE PROBLEM and
the MONOTONE CIRCUIT VALUE PROBLEM.

Computational Complexity – p. 66
 The Circuit Value Problems
Definition 3.8: A circuit is a list (B0 , B1 , . . . , Bn ) of gates
together with Boolean equations of the following type for
each m = 0, 1, . . . , n: Bm = 0, Bm = 1, Bm = ¬Bk for some
k < m, Bm = Bi ∨ Bj for some i, j < m, Bm = Bi ∧ Bj for
some i, j < m; for each m exactly one equation is given.
The value of Bn (according to the equations) is called the
value of the circuit.
A circuit without equations of the type Bm = ¬Bk is called
monotone.
The CIRCUIT VALUE PROBLEM: Given a circuit,
determine its value.
The MONOTONE CIRCUIT VALUE PROBLEM: Given a
monotone circuit, determine its value.

Computational Complexity – p. 67
 Context-Free Grammars I
A context-free grammar consists of a terminal alphabet Σ, a
nonterminal alphabet ∆, a start symbol S ∈ Σ and rules
A → w where A ∈ ∆ and w is a word of some symbols
from Σ and ∆. The language generated by a context-free
language is the language of all words over the alphabet Σ
which can be obtained by starting with S and by replacing
some member A ∈ ∆ in the current word which also occurs
in a rule A → w by the word w until all digits of the current
word are in Σ.
Example: Σ = {0, 1, 2}, ∆ = {S}, the rules are S → 0S1,
S → 22. Now the words 22, 0221, 002211, 00022111 and so
on are in the language L generated by this grammar. For
example, S ⇒ 0S1 ⇒ 00S11 ⇒ 000S111 → 00022111 is
generated by iteratively applying the above rules.

Computational Complexity – p. 68
 Context-Free Grammars II
Further Examples: Language of all ternary words with as
many 0 as 1, but arbitrarily many 2.
Σ = {0, 1, 2}, ∆ = {S}, the rules are
S → SS|S0S1S|S1S0S|2|ε. One can write S → v|w in place
of S → v, S → w to denote two or more alternatives.
All ternary words of either the form 0n 1n or 0n 2n with n > 2.
Σ = {0, 1, 2}, ∆ = {S, T, U}, the rules are S → T|U,
T → 0T1|000111, U → 0U2|000222. Sample derivations:
S ⇒ T ⇒ 000111, S ⇒ U ⇒ 0U2 ⇒ 00002222.
One writes v ⇒∗ w to indicate that w can be derived from v
in arbitrarily many derivation steps, where the case v = w is
included and means in 0 derivation steps. The language of
a grammar G is the set of all words w ∈ Σ∗ which satisfy
S ⇒∗ w .

Computational Complexity – p. 69
 Example
Theorem 3.9: The following problem is P-complete
(UNIFORM CONTEXT-FREE GRAMMAR MEMBERSHIP):
Given a word w ∈ Σ∗ and a context-free grammar
(Σ, ∆, S, R) – here R is the set of rules –, is the word w
generated by this grammar?
Proof by reducing the MONOTONE CIRCUIT VALUE
PROBLEM to this problem.
Given the circuit (B0 , B1 , . . . , Bn ) and its rules, one creates
grammar (Σ, {B0 , B1 , . . . , Bn }, Bn , R) with the below rules:
Rule in Circuit Rules in Grammar
Bm = 0 none
Bm = 1 Bm → ε
Bm = Bi ∨ Bj Bm → Bi , Bm → Bj
Bm = Bi ∧ Bj Bm → Bi Bj
Computational Complexity – p. 70
 Simple Game Winning Strategy
A SIMPLE GAME is a game where one can move, from a
node n, downwards with one or two choices in each move.
The choices will be on two players, Anke and Boris. As the
game always goes down, there are at most n moves.
Nodes without outgoing edges evaluate either to “Anke” or
to “Boris” to decide the game.
SIMPLE GAME aka SIMPLE GAME WINNING STRATEGY:
Which player has a chance to win a simple game?
Here a winning strategy is a function which, based on
passed choices of the players, advises the player on the
current choices so that the player wins.
Theorem 3.10: (a) The SIMPLE GAME has a polynomial
time algorithm; (b) There is a reduction of the MONOTONE
CIRCUIT VALUE PROBLEM to SIMPLE GAME WINNING
STRATEGY. Computational Complexity – p. 71
 Solving the Simple Game
One marks from level 0 up to level n which player has a
winning strategy. If nodes 0, 1, . . . , m − 1 are marked with
either Anke or Boris, then one does on node m the
following:
If Anke chooses where to go and one successors of m is
marked as an “Anke Node”, then m is an “Anke Node”.
If Anke chooses where to go and all successors of m are
marked as “Boris Node”, then m is a “Boris Node”.
If Boris chooses where to go and one successors of m is
marked as a “Boris Node”, then m is a “Boris Node”.
If Boris chooses where to go and all successors of m are
marked as “Anke Node”, then m is an “Anke Node”.
Once the node n is marked, its name gives the player with
the winning strategy who always moves to own nodes.
Computational Complexity – p. 72
 P-Completeness
One reduces instances (B0 , B1 , . . . , Bn ) of the MONOTONE
CIRCUIT VALUE PROBLEM to the SIMPLE GAME by
translating the circuit rule update for a gate Bm into the
action for node m as follows:
Rule in Circuit Action in Game
Bm = 0 Anke wins
Bm = 1 Boris wins
Bm = Bi ∨ Bj Boris chooses node i or node j
Bm = Bi ∧ Bj Anke chooses node i or node j
The game starts in node n. Which player has a winning
strategy?

Computational Complexity – p. 73
 Homeworks 3.11-13
For these homeworks, recall that multi-head Turing
machines know when two heads stand on the same cell
and one might assume special commands to place one
head at the beginning of the input or the position of another
head. Constant-ranged variables are allowed.
Homework 3.11: Write a Turing program for a four-head
read-only Turing machine recognising REPETITION.
Homework 3.12: Write a Turing program for a four-head
read-only Turing machine recognising BRACKET
EXPRESSION CORRECTNESS with symbol a (not a
bracket) and three types of brackets: {, }, (, ), [, ].
Homework 3.13: Fix a syntax for clause sets and write a
Turing machine program for CLAUSE SET EVALUATION;
use as few heads as possible.
Computational Complexity – p. 74
 Homeworks 3.14-3.16
For these homeworks, use hybrid Turing-Addition Machines
with one input tape two-sided scrollable and as few additon
machine registers as possible.
Homework 3.14: Write a Turing program for a hybrid
machine recognising REPETITION.
Homework 3.15: Write a Turing program for a hybrid
machine recognising BRACKET EXPRESSION
CORRECTNESS with symbol a (not a bracket) and three
types of brackets: {, }, (, ), [, ].
Homework 3.16: Fix a syntax for clause sets and write a
Turing program for a hybrid machine for CLAUSE SET
EVALUATION.

Computational Complexity – p. 75
 Homeworks 3.17-3.19
Homework 3.17: Write a Turing program for a hybrid
machine interpreting a LOGSPACE two-tape Turing
machine; the work tape must be simulated by addition
machine registers. Choose a suitable format for the
program and data on the input tape.
Homework 3.18: Write a Turing program for a multi-head
machine interpreting a LOGSPACE two-tape Turing
machine; explain the construction and the usage of the
heads; assume that the worktape is binary and not longer
than logn symbols. Choose a suitable format for the
program and data on the input tape.
Homework 3.19: Explain how to simulate a k · logn space
LOGSPACE machine with a binary worktape by a
multi-head machine. How does the number of heads
depend on k?
Computational Complexity – p. 76
 Lecture 4
Nondeterminism and Alternation. In the history of
computing and mathematics, the ideas of nondeterminism
and alternation are quite important. Economic theories deal
a lot with two-player games; these are, to a certain degree,
just another framework of alternation. Furthermore,
one-player games are also a mirror image of
nondetermism. Some instance of a problem can be solved
nondeterministically if the solution can be found by some
player where the rules of the game say which moves are
allowed to obtain a solution.
Whether nondeterminism helps with polynomial time
computations is the central question of the P-NP problem;
here solutions can be checked in polynomial time, but
finding them requires a winning strategy for the
corresponding game which might not be in P.
Computational Complexity – p. 77
 Alternating Logarithmic Space
Definition 4.1: ALTERNATING LOGSPACE or ALOGSPACE
[Chandra, Kozen and Stockmeyer 1981] is the complexity
class of problems solvable by LOGSPACE computations in
which two players can, at certain positions, choose how the
computation should go on. More precisely, the Turing
machine has Anke-Choice and Boris-Choice GOTO
commands. A word x on the input tape is than in the given
language L if Boris can do his choices such that the
outcome is “ACCEPT”, irrespective of what Anke does;
furthermore, x ∈/ L if Anke can do her choices such that the
outcome is “REJECT”, irrespective of what Boris does.
Similarly one can define ALTERNATING POLYNOMIAL
TIME (AP) and ALTERNATING POLYNOMIAL SPACE
(APSPACE) and other such complexity classes.

Computational Complexity – p. 78
 Repetition: Simple Game
A SIMPLE GAME is a game where one can move, from a
node n, downwards with one or two choices in each move.
The choices will be on two players, Anke and Boris. As the
game always goes down, there are at most n moves.
Nodes without outgoing edges evaluate either to “Anke” or
to “Boris” to decide the game.
SIMPLE GAME aka SIMPLE GAME WINNING STRATEGY:
Which player has a chance to win a simple game?
Here a winning strategy is a function which, based on
passed choices of the players, advises the player on the
current choices so that the player wins.
Theorem 3.10: (a) The SIMPLE GAME has a polynomial
time algorithm; (b) There is a reduction of the MONOTONE
CIRCUIT VALUE PROBLEM to SIMPLE GAME WINNING
STRATEGY. Computational Complexity – p. 79
 Alternating Logspace
Theorem 4.2: SIMPLE GAME is ALOGSPACE complete.
Proof: (a) SIMPLE GAME is in ALOGSPACE. An instance
of SIMPLE GAME could be represented as a computer
program of numbered lines with four types of commands as
in this example:
1. Anke Choice Goto 2,5;
2. Boris Choice Goto 3,7;
3. Anke Choice Goto 4,6;
4. Anke Choice Goto 5,6;
5. Boris Choice Goto 6,7;
6. Accept and Halt (Boris wins);
7. Reject and Halt (Anke wins).
Computational Complexity – p. 80
 Simple Game Code
One can use single letters for the commands: A (“Anke
Choice Goto”), B (“Boris Choice Goto”), H (“Accept and
Halt”), R (“Reject and Halt”). The input text of the game on
the previous slide is
1A2,5; 2B3,7; 3A4,6; 4A5,6; 5B6,7; 6H; 7R.
where the spaces are optional. Decimal numbers need
LOGSPACE to write down and GOTOs go only in one
direction. A alternating Turing machine would Halt and
Accept iff player Boris has a winning strategy enabling him
to reach a line with the command “Accept and Halt”.

Computational Complexity – p. 81
 Task 4.3
Task 4.3: Write a Turing machine program interpretating
SIMPLE GAME programs as in the short form
1A2,5; 2B3,7; 3A4,6; 4A5,6; 5B6,7; 6H; 7R.
on a two-tape Turing machine with the input in the first tape
having the input and the second tape the label of the
current instruction. For commands “A” and “B”, the players
Anke and Boris have to provide the input whether the first or
the second number should be replacing the current one in
the work tape. This simulation interacts with players (let
them choose the way). Once the program is done, explain
how it proves that SIMPLE GAME is in ALTERNATING
LOGSPACE in some few sentences.

Computational Complexity – p. 82
 Alternating Space Completeness I
Assume that a Turing machine with one input tape and k
work tapes witnesses that the language L is in
ALTERNATING LOGSPACE. Now one considers the set of
configurations of the Turing machine, as defined in
Definition 3.2. As the content of each work tape is bounded
by k′ · logn for some constant k′ where n is the length of the
′′
k
input, there are only n different configurations for some
constant k′′ . Thus if player Boris can enforce that an
alternating Turing reaches an accepting state then it can do
′′
k
it within 2n steps, as the computation should not repeat a
configuration for the same player, as otherwise either player
Boris can enforce a faster way to acception or player Anke
can enforce an infinite loop.

Computational Complexity – p. 83
 Alternating Space Completeness II
Thus one can consider all pairs (i, j) of configurations where
each transition goes from some state (i, j) to (i + 1, j′ ) with
either j′ being the successor configuration of j – if there are
several possible j′ then the configuration j player specified
which player can choose accordingly – or j′ = j in the case
that the game that the simulated Turing machine has
reached one of “Accept and Halt” or “Reject and Halt”.
′′
When defining (i, j) = i · 2n k + j then the above game has
the configuration numbers always going up and thus does
not go in loops. Furthermore, the start state is (0, m) for the
starting configuration m of the simulated machine and (i, j)
is a halting state iff j is one for the simulated machine and
′′
k
i = 2n ; a halting configuration (i, j) is accepting iff j is; the
Turing machine abstains in non-halting configurations of the
′′
k
form (2n , j). (End of Proof of Theorem 4.2) Computational Complexity – p. 84
 Nondeterministic Logspace
An important special case of alternating computations are
nondeterministic computations where only one player is
used. NONDETERMINISTIC LOGSPACE, in short
NLOGSPACE, is the complexity class of all languages L for
which there is a two-tape Turing machine with a
LOGSPACE workspace which uses only one player, say
Boris, such that a word x is in L if and only if this player
finds a way to guide the computation by choices until it
accepts the computation.
Clearly all LOGSPACE problems are in NLOGSPACE, since
on can just make Turing machines which officially use a
player but never ask him for advice; similarly NLOGSPACE
is in ALOGSPACE what equals P.

Computational Complexity – p. 85
 NLOGSPACE Complete Problems
The P-complete problems from before are in NLOGSPACE
if and only if NLOGSPACE = P. Similarly there are
NLOGSPACE complete problems which are in LOGSPACE
if and only if LOGSPACE = NLOGSPACE.
The most famous NLOGSPACE complete problem are
STCON and the bounded version of USTCON.
Definition 4.4: Given a directed graph (V, E) and nodes
s, t ∈ V, STCON is the problem to determine whether there
is a path from s to t in the graph.
Definition 4.5: Given an undirected graph (V, E), nodes s, t
and a number m, is it possible to reach the node t from s
within m steps? This problem is called BUSTCON or
bounded s-t connectivity in an undirected graph.

Computational Complexity – p. 86
 Membership in Logarithmic Space
Consider an instance (V, E, s, t, m) of BSTCON (bounded
s-t connectivity) which just drops the condition “undirected”
from BUSTCON. Then the following NLOGSPACE
algorithm solves this problem:
Line 1: Let v=s (node); Let k=0 (counter);
Line 2: If v=t then Halt and Accept;
Line 3: Let Boris select a successor w of v;
Line 4: Let k=k+1; Let v=w;
Line 5: If k<m+1 Then Goto Line 2;
Line 6: Halt without accepting.
This NLOGSPACE computation accepts the input
(V, E, s, t, m) if and only if Boris can find a way of going
from s to t within m steps. If such a path exists, Boris can
help to find it and the computation accepts; if such a path
does not exist, Boris cannot give an advice to do the
impossible. Computational Complexity – p. 87
 Completeness of BUSTCON I
Proposition 4.6: STCON is LOGSPACE reducible to
BUSTCON, that is, a LOGSPACE Turing machine can
compute an equivalent BUSTCON instance from a STCON
instance.
Constructing the Graph
Given a graph (V, E) where V has n elements, one makes
the following graph (W, F) where W = V · {0, 1, . . . , n} and
F contains all pairs ((x, k), (y, k + 1)), ((y, k + 1), (x, k)) of
nodes in W where either x = y or there is an edge from x
to y in the directed graph (V, E). Note that the graph (W, F)
is undirected by the way it is defined (each edge in W has
also its mirror image in W, thus allowing the transition in
both ways). The idea is that going in (W, F) against the
direction of (V, E) forces to go down in the second
coordinate so that too much time is lost.
Computational Complexity – p. 88
 Completeness of BUSTCON II
Now if one can go from (s, 0) to (t, m) in (W, F) within m
steps, then the second coordinate must in each step be
counted up by one. Thus, except for transitions from (x, k)
to (x, k + 1), each transition is of the form (x, k) to (y, k + 1)
for some (x, y) ∈ E, thus the path in the graph (W, F) is the
image of a directed path in (V, E) of length up to m; the not
needed steps can be used to rest on some x while counting
up the second coordinate.
Note that the second coordinate does not need to be larger
than n as every reachable node in (V, E) can be reached
from s in at most n steps by skipping all loops. Thus t is
reachable from s in (V, E) iff (t, n) is reachable from (s, 0) in
(W, F) within n steps.

Computational Complexity – p. 89
 Savitch’s Theorem
Walter Savitch (1970) found the following theorem:
Theorem 4.7: NLOGSPACE can be solved in deterministic
space (log2 (n)) and the corresponding simulation takes
time nO(log(n)) .
The important part of this result is the space usage, without
that constraint, NLOGSPACE is a subset of P. Again one
sees, as with palindrome solving, that it might happen that
an algorithm saves computation space at the expense of
computation time.
The algorithm is first illustrated for a STCON solver.

Computational Complexity – p. 90
 Example for Savitch’s Theorem
The following algorithm solves STCON. Assume n = 2k .
Start Algorithm with call Connect(s,t,n).
Line 1: Recursive Algo Connect(s’,t’,n’)
Line 2: If s’=t’ or (0<n’ and (s’,t’) in E)
Then Return True;
Line 3: If n’>0 Then n’’=n’/2 Else Return False;
Line 4: For all u’ in V Do Begin
If Connect(s’,u’,n’’) and Connect(u’,t’,n’’)
Then Return True End;
Line 5: Line 4 failed for all u’; Return False.
Depth of Calls is k = log(n). Each call stores local variables
s′ , t′ , n′ , n′′ , u′ . These need 5log(n) space. Overall space
O(log2 (n)). Timebound per call: O(n) times subcalls. Thus
O(n)k = nO(log(n)) .
Computational Complexity – p. 91
 Proof of Savitch’s Theorem
Assume an NLOGSPACE algorithm is given with space
bound c · log(n) on the tape. Then there are at most nd
configurations for some constant d. This is also a bound on
the nondeterministic computation time.
Now let V be the set of all configurations and E be the set
of pairs of configurations (x, y) such that either the Turing
machine can go from one configuration x to the other one y
in at most one step. This graph can be computed in
LOGSPACE, instead of outputting the graph, one can
access for each pair (x, y) of configuration the reduction
and check whether (x, y) is in the set E.
The previous’ slide algorithm solves STCON for (V, E) in
space O(log2 (nd )) = O(log2 (n)). The computation time is
2
2 O(log (n)) = nO(log(n)) .

Computational Complexity – p. 92
 General Space Bounds
A space bound f is called space-constructible iff
f (n) ∈ Ω(log(n)) and there is an algorithm using at most
space f (n) which computes f (n) from an input of length n
on the input tape.
Most common functions like log(n), logk (n) for constant
√
k > 1, n, nk for rational k > 0 and 2n are
space-constructible.
Theorem 4.8: Let f be a space-constructible function. Then
any problem in NSPACE(f (n)) is also in SPACE(f 2 (n)); the
2
computation time is bounded from above by 2 O(f (n)) .

The proof is essentially the same, one studies the graph of

all possible configurations.

Computational Complexity – p. 93
 Polylogarithmic Space
Definition 4.9: POLYLOGSPACE be the class of all
problems solvable in space O(logk (n)) for some k. As for
current knowledge, both possiblitilies P = LOGSPACE and
P = PSPACE are possible (of course only one of them), for
the first possibility, P is a proper subclass of
POLYLOGSPACE, for the second possibility,
POLYLOGSPACE is a proper subclass of P. Furthermore, it
could be that POLYLOGSPACE and P are incomparable.
The general belief is indeed that POLYLOGSPACE and P
are incomparable and that therefore P-complete problems
cannot be solved in POLYLOGSPACE. In particular,
UNIFORM CONTEXT-FREE GRAMMAR MEMBERSHIP is
not believed to be in POLYLOGSPACE. However, there is a
slight variant of context-free grammars. That are those
which are ε-free. For those the uniform membership
problem is in POLYLOGSPACE. Computational Complexity – p. 94
 Epsilon-Free Grammars
Definition 4.10: A context-free grammar (Σ, ∆, S, R) is
ε-free if the empty word can only be derived from a
nonterminal in the first step of the derivation and all other
steps do not shrink the length of the current word. More
precisely, if ε appears on the right side of a rule then this
rule is S → ε and S does not appear on any right side of any
rule.
Definition 4.11: UNIFORM EPSILON-FREE GRAMMAR
MEMBERSHIP is the problem to decide for a given ε-free
context-free grammar (Σ, ∆, S, R) and a word x whether the
grammar generates the word x.
Theorem 4.12: UNIFORM EPSILON-FREE GRAMMAR
MEMBERSHIP is in NSPACE(log2 (n)) and
SPACE(log4 (n)).
Computational Complexity – p. 95
 Proof of Theorem 4.12 I
Only the NSPACE(log2 (n)) algorithm will be given, the
deterministic algorithm is the translation of the
nondeterministic using Savitch’s Theorem.
The parameter n is the maximum of number of rules,
number of symbols, length of right side of any rule, length of
input word x.
The nondeterministic algorithm has a basis routine
Verify(u ⇒∗ y) which checks in nondeterministic
O(log2 (n)) space whether y can be derived from u. For
this, u will be split into halves v, w and then y into halves
yv , yw and the shorter half is done in a recursive call and
the larger half will replace u and y for further processing.
The splittings are nondeterministic and if there is an error, it
leads to an abortion of the computation; similarly when u
consists of one nonterminal then the rule which says which
right side replaces u is taken nondeterministically.
Computational Complexity – p. 96
 Proof of Theorem 4.12 II
Verify(u ⇒∗ y) does the following:
1. If u consists of a terminal word then compare whether
u = y and if yes then return TRUE else return FALSE;
2. If u consists of a single nonterminal A then pick a rule
of the form A → u′ and replace u by u′ and goto 1;
3. Otherwise u consists of several symbols. If |u| > |y|
then return FALSE;
4. Split u into v, w of same length (up to one symbol
difference) and split y into yv , yw ;
5. If |yv | ≤ |yw |
then begin call Verify(v ⇒∗ yv ); if outcome = FALSE
then return FALSE else let u = w; y = yw ; goto 1 end
else begin call Verify(w ⇒∗ yw ); if outcome = FALSE
then return FALSE else let u = v; y = yv ; goto 1 end.
Computational Complexity – p. 97
 Proof of Theorem 4.12 III
Initial call is Verify(S ⇒∗ x).
Local variables in each call: u, y, v, vy , w, wy . u, v, w stored
by rule number and start and end on right side of rule or
nonterminal number: O(log(n)). y, yv , yw stored by start
and end position in x: O(log(n)).
Depth of recursive calls: Each subcall marks of word half as
long as y on right side, thus depth is at most log(n) and
each recursive call has at most one outgoing recursive call
at a time. Overall Space usage is O(log2 (n)).
If u = y then clearly u ⇒∗ y, as derivation is completed; if
|u| > |y| one cannot do any derivation and return-value
FALSE is correct; if u is split into v, w then either one can
split y in yv , yw such that (v ⇒∗ yv and w ⇒∗ yw ) or
u 6⇒∗ y.
Computational Complexity – p. 98
 Problems reducible to CFL
Definition 4.13: LOGCFL is the complexity class of
problems LOGSPACE reducible to the membership of one
context-free language, the choice of the language might
depend on the problem in LOGCFL reduced to it.
So for each language L in LOGCFL there is a context-free
language H and a LOGSPACE computable function f such
that for all x, x ∈ L iff f (x) ∈ H.
Remark: Every context-free language is in LOGCFL and
every language in LOGSPACE is in LOGCFL. Under the
hypothesis P = LOGSPACE, LOGCFL = P. Under the
hypothesis P = PSPACE, LOGCFL is a proper subset of P.
LOGCFL is strictly contained in POLYLOGSPACE.

Computational Complexity – p. 99
 Example
Example 4.14: There are languages in LOGCFL which are
neither context-free nor known to be in LOGSPACE. For
this let L be a context-free language not known to be in
LOGSPACE and not using the digits 0, 1, 2 and H be the
language containing a word w iff w has the same number of
0, 1, 2 and, when deleting the digits 0, 1, 2 from w one
obtains a word v ∈ L. H is in LOGCFL but neither
context-free nor known to be in LOGSPACE.
For seeing that H is in LOGCFL, one first computes with a
LOGSPACE computation whether the number of all 0, 1, 2
are the same and if so, then one outputs v by omitting all
0, 1, 2 from w at the output else one outputs a fixed word
not in L.

Computational Complexity – p. 100

 Nick’s Class
Definition 4.15: A problem L is in the class NC(k) iff there is
a polynomial p and a sequence of circuits of gates with one
or two input-bits such that the n-th circuit is of size p(n) and
determines for each x ∈ {0, 1}n in parallel time O(logk (n))
whether x ∈ L.
Here parallel time O(logk (n)) means that there is a
constant k′ such that in each circuit all initial values are 0
and in each cycle, each value of the gate gets updated to
the current outputs of the gates below and after k′ · logk (n)
steps all gates have stabilised to the final value; in other
words, the depths of the circuit is at most k′ · logk (n). Recall
log(n) = min{m ≥ 1 : n ≤ 2m } for this course.
There is no constraint how the circuits are obtained.
LOGSPACE-uniform NC(k) has a LOGSPACE algorithm
computing the n-th circuit from input of length n. Computational Complexity – p. 101
 Containment I
Theorem 4.17: LOGSPACE-uniform NC(k) is contained in
SPACE(O(logk+1 (n)))
Proof: For a proof, consider first that there is a
LOGSPACE-algorithm which on input m finds the m-th
symbol of the description of the circuit. Furthermore, one
codes each circuit starting at some position m as one
symbol representing one of AND, OR, NOT, INP and the
further positions of the bits from a lower level gate or the
input separated by comma and semicolon.
The evaluation is recursive and starts with a call with
position 0. Each call has subcalls if computations of lower
gates are used or reads out the corresponding bit from the
input. The depths of the calls is O(logk (n)) and the local
memory is O(log(n)) space and thus the overall space
usage is O(logk+1 (n)). Computational Complexity – p. 102
 Containment II
Local Algorithm with input m:
Read from position m onwards OPERAND and positions i
and, in cases of OPERANDS being AND, OR, also j.
SWITCH(OPERAND):
AND: Call algorithm with i and with j and if both results are
1 then return 1 else return 0.
OR: Call algorithm with i and with j and if both results are 0
then return 0 else return 1.
NEG: Call algortihm with i and if result is 1 then return 0
else return 1.
INP: Find position i in the input, read out the bit there and
return it.
LOCAL MEMORY inside CALL is m, i, j and up to three bits
(results of calls and intended return-value). This is
O(log(n)) as used above. Note that positions in circuits of
size p(n) need only log(p(n)) ∈ O(log(n)) bits. Computational Complexity – p. 103
 Homeworks 4.18-4.21
Show that the following problems are in NC; describe the
circuits and if they are LOGSPACE uniform, explain how the
algorithm outputs the circuit and describe the coding.
Though NC is normally used as a class of languages (word
problems), one can make the analogous definition for
functions from words (coding tuples of numbers) to words
(coding numbers):
Homework 4.18: Adding two positive integers in binary;
Homework 4.19: Multiplying two integers in binary (can use
the addition circuits as submodules and using that they
have logarithmic detph).
Homework 4.20: Computing the remainder of two positive
integers in binary (first divided by second).
Homework 4.21: Computing the downrounded integer value
of the fraction of two binary positive integers.
Computational Complexity – p. 104
NC with Two Inputs and Two Outputs
The precise definition of Nick’s Class requires not only that
every input circuit uses only up to two inputs but also that
only up to two outputs use it. One has then five operands:
AND, OR, NEG, INP, COPY where at COPY one input is
copied into two outputs which is needed to deal with
situations where more than two gates use one output or
input-bit. Furthermore, the description of the circuit has, for
a gate at position m, one OPERAND and four parameters
which refer to the positions of the two inputs and the two
outputs; for NEG and INP, the first two positions are equal,
furthermore, if only one output is used, the last two
parameters are equal.
Task 4.22: Using this more precise definition, show that
LOGSPACE-uniform NC(k) is contained in
SPACE(logk (n)). Explain the algorithm in detail and how
the better memorybound is achieved. Computational Complexity – p. 105
 Homeworks 4.23-4.26
Homework 4.23: Show that if LOGSPACE = P then there is
a constant k with NC = NC(k).
Homework 4.24: Show that if P = PSPACE then all levels of
the NC-hierarchy are different and P is a proper superset of
NC.
Homework 4.25: Show that P = POLYLOGSPACE is
impossible. For this, consider a P-complete problem L and
use that there must be a k with L being in SPACE(logk (n)).
Now show that P is contained in SPACE(logk (n)) as every
problem in P is LOGSPACE many-one reducible to L and
thus SPACE(logk+1 (n)) contains a problem outside P.
Homework 4.26: Prove that LOGSPACE is contained in
LOGSPACE uniform NC(2).

Computational Complexity – p. 106

 Lecture 5
LINSPACE and NLINSPACE are the classes of
deterministic and nondeterministic linear space. The class
NLINSPACE is a somewhat special complexity class, as it
coincides with one level of the Chomsky hierarchy from
formal languages. The Chomsky hierarchy classifies
grammars generating languages (= sets of words) into
regular, context-free, context-sensitive and unrestricted (=
recursively enumerable). One names the classes of
languages generated by the corresponding types of
grammars after these classes. The regular languages
coincide with CONSTANTSPACE where the only memory is
the state of the Turing machine recognising the language
and context-sensitive languages coincide with NLINSPACE.
So the context-sensitive languages are not only a complete
problem for NLINSPACE, they coincide with this class.
Computational Complexity – p. 107
 Overview of Lecture 5
The main results of this lecture are the following ones:
• Formalisations of NLINSPACE: Linear bounded
automata and Input-delimited space;
• Characterisation of NLINSPACE as the class of all
context-senstive languages;
• LINSPACE equals determinsitic context-sensitive
languages;
• Context-sensitive languages are closed under union,
intersection and complementation; the same applies to
all nondeterminstic space classes from logarithmic
space onwards (provided that their bound is
space-constructible).

Computational Complexity – p. 108

 Linear Bounded Automata
Definition 5.1: An linear space Turing machine is a Turing
machine allowed to use space linear in the size of the input.
In the case of LINSPACE, the automaton has to be
deterministic; in the case of NLINSPACE, it can be
nondeterministic.
More formally, the Turing machine uses on the work tape
only the cells 0, 1, 2, . . . , k · n for some constant k depending
on the machine. Furthermore, the machine can use on the
work tape an arbitrary, machine-dependent alphabet.
One can, for example, compress binary symbols on the
work alphabet to hexadecimal symbols and so reduce the
length by a factor four. A sufficiently large work alphabet
allows to have k = 1.

Computational Complexity – p. 109

 Linear Bounded Automaton
The easiest formalisation is a one-tape Turing machine with
an input-word where the tape alphabet contains not only
the letters of the input words but also special variants which
denote that a letter is first or last of the word. Furthermore,
in dependence of the machine, an arbitrary but finite
additional amount of letters can be used.
For example, if the input alphabet is binary, one replaces
the first bit by either 2 (if it was 0) or by 3 (if it was 1) and
similarly the last ones by 4 (instead of 0) and 5 (instead of
1). If there is only one digit one uses 6 for 0 and 7 for 1. The
case of the empty word is ignored. Now the Turing machine
starts on 2/3 and has to move such that it never goes onto
a cell which did not have an input symbol at the beginning.

Computational Complexity – p. 110

 Addition Machine Model
An LINSPACE addition machine with input x satisfies the
following constraint: There is a constant k such that no
register in the addition machine during the whole runtime of
the processing of x takes a value larger than (abs(x) + 2)k
where abs(x) is the absolute value of x. Note that the space
bound of an input is logarithmic in abs(x) and thus the k-th
power uses only k times the space which abs(x) + 2 uses.
NLINSPACE is defined by using nondeterministic addition
machines.
So three equivalent models: LBA (= Input-Deliminated
One-Tape Turing Machine), LOGSPACE Turing machine,
Linear Space Addition Machine. Furthermore, for both
deterministic and nondetermistic computations, one has
these three representations. It is conjectured but unproven
that the nondeterministic variant is more powerful than the
deterministic.
Computational Complexity – p. 111
 Context-Sensitive Languages
Definition 5.2: A grammar (Σ, ∆, S, R) for a language L not
containing the empty word ε with R being the set of rules is
called context-sensitive iff every rule is of the form
vAw → vuw with A ∈ ∆, v, w ∈ (Σ ∪ ∆)∗ and
u ∈ (Σ ∪ ∆)+ . That is, some nonterminal A is replaced by a
nonempty word u provided that in the current word, v is
before and w is behind the occurrence of A which will be
replaced. Such a replacement is therefore depending on
the (perhaps empty) context (v, w) of A and only if this
context is there, the replacement is done. Context-free
grammars do not have this context-condition, but are also
therefore less powerful and not closed under intersection
and complement.
An equivalent type of grammar allows all rules v → w
provided that v contains at least one nonterminal and
|v| ≤ |w|. This more liberal version will be used for proofs.
Computational Complexity – p. 112
 5.3: The Chomsky Hierarchy
Level 0: No restriction on grammar. This level contains all
recursively enumerable languages, that is, all languages
which are the range of a function computed by some
algorithm without any space and time limitations.
Level 1: Context-sensitive languages. This level equals
NLINSPACE (shown in this lecture).
Level 2: Context-free languages. This level contains all
languages generated by context-free grammars, that is,
grammars, which have rules allowing to replace some
nonterminal by the given right side, no further restriction.
Level 3: Regular languages. This level equals to the
languages recognised by Constant Space Turing machines
and to those recognised by one-tape linear time Turing
machines. Closure of finite languages under union,
concatenation and Kleene star; also closed under
intersection and set difference.
Computational Complexity – p. 113
 Example of CS Language
Example 5.4: Let L be the set of all nonempty words which
have as many 0 as 1 as 2. Then the grammar has
∆ = {S, U, V, W} and Σ = {0, 1, 2} and the following rules:
S → UVW, S → SUVW, UV → VU, VU → UV,
UW → WU, WU → UW, VW → WV, WV → VW,
U → 0, V → 1, W → 2.
The first two rules generate a word of the form
UVW, UVWUVW, UVWUVWUVW, . . . and the next six
rules allow to change the order of the U, V, W in an
arbitrary way and the last three rules replace every U by 0,
every V by 1 and every W by 2. All words in the language
can be generated this way and all words generated have
the same number of 0, 1, 2.

Computational Complexity – p. 114

 Characterisation
John Myhill introduced linear bounded automata in 1960,
P.S. Landweber proved some connections for the
deterministic case and Sige-Yuki Kuroda proved the below
theorem 1964.
Theorem 5.5 [Kuroda 1964]: A language L is
context-sensitive iff it is in NLINSPACE.
The proof will be given in two parts.
(a) If the language is context-sensitive then an LBA
recognises it.
(b) If an LBA with input-delimited memory (one-tape Turing
machine) recognises the language L then L is context
sensitive.

Computational Complexity – p. 115

 (a) Implication CS ⇒ LBA
An LBA can store in the memory the current word of a
derivation starting with S. In each step, it selects
non-uniformly a symbol A and replaces it by the right side
u, provided that the contexts v, w occur before the
replacement on both sides of A and that the word does not
become longer than the word to be checked. In the case
that the derivation results in a word of length n which is
equal to the input word x then the LBA accepts else the
computation is discarded. Thus a word x is accepted by the
LBA iff there is a derivation for x in the grammar — note
that during the derivation, no intermediate word is longer
than x, as then there is no way back to generate x. Thus
whenever a replacement would make the current word
strictly longer than x then one abstains from applying the
corresponding rule.
Computational Complexity – p. 116
 (b) Implication LBA ⇒ CS
At the beginning, the grammar generates a start
configuration of the LBA with an arbitrary input word x. A
configuration looks like this:

(1, b, a, −)(0, m, b, −)(0, m, c, q)(1, e, d, −).

Here the first components of the configuration code the

input word, here 1001. The next component informs the
Turing machine about currently being at begin (b), middle
(m), end (e) or only symbol of the input word (o). The third
component of each symbol is the current tape symbol of the
Turing machine, here abcd, the tape alphabet can be
anything depending on the machine. The last component
says either that the Turing machine sits on the current (here
third) symbol and has the state q or that it does not sit on
the current symbol (default entry “−”).
Computational Complexity – p. 117
 Initialisation
The initialisation creates a starting condition for the LBA.
For exaple, the LBA has the start state s and starts on the
first symbol and has not yet overwritten the input. Then the
configuration would look like this:

(1, b, 1, s)(0, m, 0, −)(0, m, 0, −)(1, e, 1, −).

So the part of the grammar which produces this situation

has these rules: S → T(1, e, 1, −), S → T(0, e, 0, −),
T → T(0, m, 0, −), T → T(1, m, 1, −), T → (1, b, 1, s),
T → (0, b, 0, s). So the input word is any binary word and
the Turing machine starts with the same word on its Turing
tape and the state s on the first symbol of the input. In the
next slide come the rules for working on the tape. The rules
for the initialisation do not use any context. The
nonterminals S, T are only used in this phase.
Computational Complexity – p. 118
 Main Phase
Assume now that the Turing machine sits on the symbol
(0, m, a, q) with state q and neighbour (1, m, b, −) and goes
to the to a state r on the next symbol after writing a c onto
the Turing tape before leaving. To allow this, the following
rules must be in the rule set of the grammar:

(0, m, a, q)(1, m, b, −) → (0, m, c, −)(1, m, b, r).

Each allowed transition of the LBA can be coded into such

a rule. Furthermore, one assumes that after the
computation, if the LBA wants to accept, it goes to the first
symbol and then takes an accepting state a. The
configuration looks as follows:

(1, b, e, a)(0, m, f , −)(0, m, g, −)(1, e, h, −).

Only the third and fourth component of symbols change

during the simulation. Computational Complexity – p. 119
 Termination
If an accepting state is reached, the Turing machine applies
the following rules.

(1, b, i, a) → 1, (0, b, j, a) → 0.

These rules can have anything at entries i, j. Once this is

done, there are rules to convert from the front to the end
each nonterminal into a terminal, here Σ = {0, 1}.
0(0, m, i, −) → 00, 0(1, m, i, −) → 01,
1(0, m, i, −) → 10, 1(1, m, i, −) → 11,
0(0, e, i, −) → 00, 0(1, e, i, −) → 01,
1(0, e, i, −) → 10, 1(1, e, i, −) → 11.
These rules allow to convert the whole word into a terminal
word equal to the originally guessed input x of the LBA
provided that the simulation of the LBA leads to acceptance.
Computational Complexity – p. 120
 Deterministic CS Grammars
One might ask how to put determinism into this concept. As
the generation of words starting from S has multiple
outcomes, grammars are nondeterministic by definition.
However, one might ask how much information is needed to
know which rules to apply to derive a specific word x. The
following is a possible definition for a deternistic
context-sensitive grammar and language.
Definition 5.6: A context-sensitive grammar is called
deterministic iff there is a LOGSPACE computable function
which computes from inputs x and y where x is the word to
be generated and y is the current word of the derivation,
which rule to apply and at what position of y. A language is
deterministic context-sensitive iff it has a deterministic
context-sensitive grammar.

Computational Complexity – p. 121

 Characterising DCS Languages
Theorem 5.7: A language is deterministic context-sensitive
if and only if a deterministic LBA can recognise it.
The LOGSPACE guidance function tells which rule to apply
in a derivation. Initially y = S and x is the given input word.
In every step, the LOGSPACE function says which rule to
apply to y in order to update the current word to the next
step. If it ever happens that x = y then the LBA accepts. If
the derivation takes longer than (|Σ| + |∆| + 1)n steps
where n = |x| or ends in a rejecting state then the
deterministic LBA rejects.

Computational Complexity – p. 122

 LINSPACE ⇒ DCS
Now assume that a deterministic LBA is given to witness
that the language is in LINSPACE. The grammar
constructed from the LBA is the same as in Theorem 5.1
(b). Now the LOGSPACE guiding function does the
following:
First it applies those rules using the nonterminals S, T to
generate

(x1 , b, x1 , s)(x2 , m, x2 , −) . . . (xn−1 , m, xn−1 , −)(xn , e, xn , −)

from x = x1 x2 . . . xn . Then the LOGSPACE guiding function

determines at every situation which rule of the Turing
machine applies (it is unique) and the position of the rule is
that of the head and one neighbouring field (depending on
the direction of the upcoming move). If then the LBA
accepts then the last phase applies and one can see from
the rules that always exactly one rule applies. Computational Complexity – p. 123
 5.8: The LBA Problems
It is easy to see that if one takes the union or intersection of
two languages recognised by LBAs, one again has a
language recognised by an LBA and if the given two LBAs
are deterministic, so are those for union and intersection.
Kuroda asked [1964] the following two questions.
(LBA 1) Is there, for every nondeterministic LBA, an
equivalent deterministic LBA? In other words, do
context-sensitive and deterministic context-sensitive
languages coincice? Is NLINSPACE = LINSPACE?
(LBA 2) If some LBA recognises a language, can its
complement also be recognised by an LBA? In other words,
are context-sensitive languages closed under complement?
Question (LBA 1) is still open, Question (LBA 2) was open
for over twenty years until Immerman and Szelepcsényi
solved it independetly at the same time. Computational Complexity – p. 124
 Context-Sensitive Languages
Theorem 5.9 [Immerman and Szelepcsényi 1987]
The complement of a context-sensitive language is
context-sensitive.
Representation of Σ∗ − L
The complent of L will be enumerated by an LBA which has
constantly many word-variables of the same length as the
input x and which accepts x if it does not find a derivation
for x in the original grammar and has verified by
nondeterministic counting that all derivations have been
taken care off.
Proof-Method of Nondeterministic Counting
If i strings can be derived in ℓ steps then one can
nondeterministically check which string y can be derived in
ℓ + 1 steps and count their number j.

Computational Complexity – p. 125

 Basic Algorithm
1. For any x ∈ Σ+ , try to verify x ∈ / L as follows;
2. Let u be the length-lexicographically largest string in
(N ∪ Σ)|x| ;
3. Let i = Succll (ε);
4. For ℓ = ε to u Do Begin
5. Let j = ε;
6. For all y ≤ll u Do Begin
7. Derive words w1 , w2 , . . . , wi non-deterministically in
length-lexicographic order in up to ℓ steps each and check:
8. If some wm satisfies wm ⇒ y or wm = y then let
j = Succll (j);
9. If some wm satisfies wm = x or wm ⇒ x then abort
computation (as x ∈ L); End (of For-Loop 6);
10. Let i = j; let j = ε; End (of For-Loop 4);
11. If the algorithm has not been aborted then x ∈ / L.
Computational Complexity – p. 126
 Refined Algorithm I
1: Choose an x ∈ Σ+ and initial all other variables as ε;
2: Let u = (maxll (N ∪ Σ))|x| ;
3: Let i = Succll (ε) and ℓ = ε;
4: While ℓ <ll u Do Begin
5: Let j = ε;
6: Let y = ε;
7: While y <ll u Do Begin
8: Let y = Succll (y);
9: h = ε and w = ε;
10: While h <ll i Do Begin
11: Nondeterministically replace w by w′ with
w <ll w′ ≤ll u;
12: Let v = S;
13: Let k = ε;
Computational Complexity – p. 127
 Refined Algorithm II
14: While (v 6= w) ∧ (k <ll ℓ) Do Begin
15: Nondeterministically replace (k, v) by (k′ , v′ )
with k <ll k′ and v ⇒ v′ End (of While in 14);
16: If v 6= w Then abort the computation;
17: If w = x or w ⇒ x Then abort the computation;
18: If w 6= y and w 6⇒ y
19: Then Let h = Succll (h)
20: Else Let h = i
21: End (of While in 10);
22: If w = y or w ⇒ y Then j = Succll (j)
23: End (of While in 7);
24: Let i = j;
25: Let ℓ = Succll (ℓ) End (of While in 4);

26: If the algorithm has not yet aborted Then generate x.

Computational Complexity – p. 128
 Verification of LBA
For the LBA, numbers are strings in length-lexicographic order.
The LBA accepts x only if the nondeterministic counting
has counted up all words in (Σ ∪ ∆)∗ of length up to n = |x|
and as the word ε does not occur in the derivation, every
longer derivation has a repetition and is thus not the
shortest derivation. If there is a derivation accepting x of
length ℓ then the algorithm would have detected it for any ℓ
up to length n. So x is accepted iff it is verified that such a
derivation does not exist.
The algorithm stores constantly many variables all taking as
value a word up to length n over the alphabet Σ ∪ ∆. One
can use a sufficiently large alphabet – in this alphabet each
symbol is a tuple with one component being the input word
at this position and all other components being the symbol
of the corresponding variable at the corresponding position.

Computational Complexity – p. 129

 Homework 5.10-5.14
Provide Context-sensitive Grammars for the following
languages and show that they are also in LOGSPACE
(most context-sensitive languages are not).
Homework 5.10: Do this with the language of all words
0n 1n 2n where n ≥ 2.
Homework 5.11: Do this with the language of all words of
the form uu where u is some nonempty word over {0, 1, 2}.
Homework 5.12: Do this with the language of all words of
the form uu where u is some nonempty palindrome over
{0, 1, 2}.
Homework 5.13: Do this with the language of all words
0n 12n 23n where n ≥ 2.
Homework 5.14: Do this with the language of all words in
{0, 1, 2}n where there are distinct i, j ∈ {0, 1, 2} which do
not occur in the word the same amount of times.
Computational Complexity – p. 130
 Homework 5.15-5.18
Let φ(x1 , x2 , . . . , xn , y1 , y2 , . . . , yn ) be a formula consisting
of a conjunction of clauses where every clause is a
disjunction of literals and every literal is either constant 0 or
constant 1 or some variable xk or yk or some negated
variablable ¬xk or ¬yk . There are assumed brackets to be
around each clause (as conjunction binds normally more
than disjunction). Write deterministic LINSPACE algorithms
to evaluate the following formulas. Say whether the formula
is definitely in P or NP or CoNP (without any assumption).
Homework 5.15: ∃x1 . . . ∃xn ∃y1 . . . ∃yn [φ(x1 , . . . , yn )].
Homework 5.16: ∃x1 . . . ∃xn ∀y1 . . . ∀yn [φ(x1 , . . . , yn )].
Homework 5.17: ∃x1 ∀y1 ∃x2 ∀y2 . . . ∃xn ∀yn [φ(x1 , . . . , yn )].
Homework 5.18: ∀x1 . . . ∀xn ∃y1 . . . ∃ym [φ(x1 , . . . , ym )]
where m = log(n) = min{m ≥ 1 : 2m ≥ n}.
Computational Complexity – p. 131
 Lecture 6
Lecture 6 will address the nondeterministic time complexity
class NP and its relation to related classes P and its
complement CoNP.
The motivation for the main class NP is that in mathematics
(as one might remember from school) there are many tasks
where it is quite difficult to find a solution while it is easy to
verify that the found solution is correct by testing.
For nondeterministic machines, it might be difficult to find
out whether there is a nondeterministic computation leading
to ACCEPT, but if one knows for each branching which
decision the machine takes to reach ACCEPT then one can
easily verify that the machine accepts the input.
More concrete problems in NP, in particular complete ones,
are provided in this lecture.
Computational Complexity – p. 132
 P, NP and CoNP
Definition 6.1: A Nondeterministic Polynomial Time
Machine M is a Turing machine (or addition machine) with
nondeterministic branchings which on input words x runs on
all computations at most time f (n) for inputs of size n where
f is a polynomial and which might reach on some inputs
ACCEPT or REJECT. Furthermore M is good iff there is no
input on which both ACCEPT and REJECT can be reached.
A problem L is in NP iff some nondeterministic polynomial
time machine M can on every x ∈ L but on no x ∈ / L reach
ACCEPT.
A problem L is in CoNP iff some nondeterministic
polynomial time machine M can on every x ∈/ L but on no
x ∈ L reach REJECT.
A problem L is in NP ∩ CoNP iff there is a good
nondeterministic polynomial time machine M which can
reach on all x ∈ L ACCEPT and on all x ∈ / L REJECT.
Computational Complexity – p. 133
 Complete Problems
Definition 6.2: A problem L is complete for a complexity
class C iff for every further problem H in C there is a
LOGSPACE computable function f such that for all x,
H(x) = L(f (x)), that is, x ∈ H if and only if f (x) ∈ L.
Example 6.3: A complete problem for CoNP is whether a
nondeterministic polynomial time machine is good on some
length n. So the input is a nondeterministic polynomial time
machine M and a time bound (polynomial) f and a length n
(given in unary) and the machine is good at length n iff
there is no input x of length n on which the machine either
has a run taking longer than f (n) steps or has two runs one
of them reaching ACCEPT and one reaching REJECT.
Note that the length n must be part of the input; otherwise
the problem is outside CoNP and even undecidable, that is,
there is no algorithm at all which solves this problem.
Computational Complexity – p. 134
 Complexity Parameters
Main parameters of certain problem types.

Problem Type Parameter Name

Graphs Number of nodes n
Number of edges m
Languages Alphabet Size |Σ|
Length of Word n
Formulas Number of variables n
and Number of clauses / gates m
Circuits Overall number of literals ℓ
Maximal clause length k
Depth or height h
Natural Numbers Size min{k ≥ 1 : x ≤ 2k } n
Tuples and Matrices Size n, m × n
Computational Complexity – p. 135
 Satisfiability
Literal = Variable, Negated Variable, Truth Constant;
Clause = Disjunction (Or) over literals;
Formula = Conjunction (And) over clauses.
Monotone = No use of Negation.
Example: Let φ be
(x1 ∨ x2 ) ∧ (x1 ∨ ¬x3 ) ∧ (x2 ∨ x4 ) ∧ (x1 ∨ x4 ) ∧ (x1 ∨ x3 ) ∧ (0 ∨ 1).
Now n = 4, m = 6, ℓ = 12, k = 2 and φ is not monotone.
The formula is in CNF (Conjunctive Normal Form, always
assumed for formulas) and is satisfiable, as one can set
x1 = 1 and x4 = 1 (makes all clauses true).
SAT = Set of all CNF formulas which are satisfiable (stan-
dard NP-complete problem). 3SAT = Set of all CNF formu-
las which are satisfiable and where k ≤ 3, similarly 2SAT
and 4SAT. XSAT = Set of all CNF formulas which are satis-
fiable in a way that for each clause exactly one literal is true.
Computational Complexity – p. 136
 NP-completeness of 3SAT I
Theorem 6.4: 3SAT is NP-complete.
Given a SAT CNF formula with parameters n and m and ℓ,
one introduces for each clause with k literals, where k ≥ 4,
k − 3 additional new variables to split it into 3SAT clauses
as in the following example:
x1 ∨ x2 ∨ x3 ∨ x4 ∨ x5 is split into
x1 ∨ x2 ∨ y1 , ¬y1 ∨ x3 ∨ y2 and ¬y2 ∨ x4 ∨ x5 .
If now first clause is satisfied, say by x3 is true, then one
can also satisfy the three second clauses by letting y1 and
¬y2 be true. Note that here 5 − 3 = 2 new variables were
inserted. However if the first clause is not satisfied, then
choosing y1 , y2 does not allow to make all three new
clauses true, as among the literals of y1 , y2 , only two can
be true, thus one of x1 ∨ x2 , x3 , x4 ∨ x5 must be made true
to have all three true.
Computational Complexity – p. 137
 NP-Completeness of 3SAT II
The previous slide explained how to replace an n-var
m-clause ℓ-literal SAT formula by a 3SAT formula with up to
n + ℓ − 3m variables and up to ℓ − 2m clauses. If the
previous formula was in kSAT then ℓ ≤ km so that the new
formula has up to n + (k − 3)m variables and up to (k − 2)m
clauses.
The algorithm can be carried out in LOGSPACE, one just
needs a secure method to generate new variable names
and then can translate clause by clause with a three-tape
LOGSPACE Turing machine. The old variable names are
kept and the new ones are just used twice for each splitting.

Computational Complexity – p. 138

 2SAT is in P
Theorem 6.5: 2SAT is in P.
For the polynomial time algorithm on the next slide, note
that in the case of two literals per clause, α, β have both at
most one literal and only new two literal clauses are
created. Furthermore, the number of clauses is bounded by
4n2 + 2n throughout the algorithm, so all data fits in
polynomial space. Each processing of a variable goes in
polynomial time, as one can check each condition with a
scan over the ocurrences of the variables in the input.
Furthermore, the same polynomial time algorithm applies if
every variable occurs at most two times, independently of
the size of the clauses. Thus 2OCCURSAT is also in P.
3OCCURSAT is NP-complete, as one can replace one
variable x occurring k times by y1 , y2 , . . . , yk , respectively,
and then add clauses for y1 → y2 , y2 → y3 , . . . , yk → y1 .
Computational Complexity – p. 139
 Algorithm for 2SAT is in P
Algorithm 6.6: For each variable xk still occuring in some
clause, do the first case which applies; if no clause is left,
return SATISFIABLE.
1. If there are two single literal clauses xk and ¬xk then
return UNSATISFIABLE.
2. If either xk occurs only positively or there is a single
literal clause containing xk then remove all clauses
containing xk and remove ¬xk from all clauses where it
occurs (due to 1. done before, such clauses still have other
literals).
3. Do the action corresponding to 2. if either xk occurs only
negatively or there is a single literal clause ¬xk .
4. For all pairs of clauses α ∨ xk , β ∨ ¬xk put the clause
α ∨ β into the set of clauses and then remove all clauses
containing xk or ¬xk . This is called resolution.
Computational Complexity – p. 140
 Weighted 2SAT
A weight-function assigns to every variable x weights f (x)
and f (¬x) which are nonnegative integers. A
WEIGHTED2SAT instance is given by a set of 2SAT
clauses and a weight function and a target value k. Now a
WEIGHTED2SAT instance with variables x1 , . . . , xn is in
WEIGHTED2SAT iff there is an assignment of binary values
zk to variables xk such that the sum over all f (xk ) with
zk = 1 and f (¬xk ) with zk = 0 is k; for the ease of notation
one writes f (z1 ), . . . , f (zn ) for these values.
Furthermore WEIGHTED2SATc is the same problem with
the additional constraint that all weights are at most c.
Theorem 6.7: WEIGHTED2SAT and, for all c ≥ 1, also
WEIGHTED2SATc are NP-complete.
Membership in NP is trivial; just guess the zk if they exist.
Computational Complexity – p. 141
 NP-hardness Construction
One translates XSAT into WEIGHTED2SAT. For this, one
adds for each clause with literals x1 , x2 , . . . , xh (they can be
negated variables) all conditions ¬xi ∨ ¬xj with
1 ≤ i < j ≤ h into the 2SAT instance to be constructed.
Furthermore, consider a variable x which occurs in i
clauses as x and in j clauses as ¬x. Then one sets f (x) = i
and f (¬x) = j. The number k is the overall number of XSAT
clauses in the XSAT instance.
In the case that one has WEIGHTED2SAT1 , one adds to
each variable x, which occurs i times positively and j times
negatively, h = max{i, j} further variables y1 , . . . , yh and
adds to the 2SAT clauses for x → y1 , y1 → y2 , ..., yh → x
and one defines f (x) = 0, f (¬x) = 0, f (yℓ ) to be 1 if ℓ ≤ i
and to be 0 if ℓ > i, f (¬yℓ ) to be 1 if ℓ ≤ j and to be 0 if ℓ > j.
Computational Complexity – p. 142
 Ideas of Verification
Assume that z1 , . . . , zn is a satisfying assignment of
underlying XSAT instance and that one extends f to
assignments as indicated above. Then z1 , . . . , zn satisfies
the derived 2SAT instance. Furthermore, f (zh ) is the
number of clauses made true by zh and, as the XSAT
assignment is satisfying, the sum of the f (zh ) is k.
For the converse
P way, if z1 , . . . , zn satisfies the derived 2SAT
formula and h f (zh ) = k then the 2SAT formula enforces
that eachPclause has at most one satisfied literal and the
formula h f (zh ) = k enforces that the overall number of
satisfied literals is k; thus each clause has exactly one and
the given assignment is an XSAT assignment.
The third item just tells how to code with additional
variables which are all equal to the main variable in the
case that the weight bound 1 is there.
Computational Complexity – p. 143
 Trivial SAT Algorithm
One can just test out all 2n combinations of the variables
and for each check whether the assignment is satisfying.
Thus the runtime is at most 2n · Poly(n + m). If there are
additional constraints, it goes slightly better.
Theorem 6.8. kSAT can be solved in time
((2k − 1)1/k )n · Poly(n + m).
Algorithm: Pick the shortest clause (with length k′ ) and
branch all the variable-combinations of the variables in this
clause satisfying it. Each branch removes k′ variables; for
all clauses where these variables occur, if the values of
these variables make the clause true then remove the
clause (as satisfied) else remove the literals of these
variables from the clause. If empty unsatisfied clauses
remain, the corresponding possibility is terminated (as not
satisfying). Repeat this until all variables are used up.
Computational Complexity – p. 144
 Evaluation I
′ ′
k′
kSAT: If ≤ k then (2 k − 1) 1/k
≤ (2k − 1)1/k . Thus
algorithm in Time ((2k − 1)1/k )n · Poly(n + m). This is time
1.91294n · Poly(n + m) for 3SAT.
For this one has to verify that all k ≥ 1 satisfy
(2k − 1)1/k < (2k+1 − 1)1/(k+1) . Note that one can raise
both sides to the power k(k + 1) without changing the
relation and one has (2k − 1)k+1 < (2k+1 − 1)k . To see this,
one takes the logarithm which is monotone on positive
numbers (here no rounding is done). So the above holds iff
the following holds: (k + 1) · log(2k − 1) < k · log(2k+1 − 1).
Now one has (k + 1)log(2k − 1) < k · log(2k − 1) + k =
k · (log(2k − 1) + 1) < k · log(2k+1 − 1) as log(2k − 1) < k
and log(2k − 1) + 1 = log(2k+1 − 2) < log(2k+1 − 1).
Throughout the lecture, the logarithm has base 2.
Computational Complexity – p. 145
 Evaluation II
XkSAT: One needs only to consider k in place of 2k − 1
possibilities, as exactly one out of the k literals is 1 and all
others are 0 for satisfying the clause. Clause-length 3 is
worst case for this algorithm and time is
1.44225n · Poly(n + m) for all XkSAT and XSAT in general.
For both problems, much better algorithms are known.
Paturi, Pudlák, Saks and Zane [2005] showed that 3SAT
can be done by a randomised algorithm in time
1.30685n · Poly(n + m) and Liu [2018] provided a
deterministic algorithm in time O(n1.3280 ). Wahlström
[2007] provided an O(1.0984n ) algorithm for X3SAT and
Gordon Hoi (2020) showed that XSAT can be done in time
1.1674n · Poly(n + m) by a deterministic algorithm.

Computational Complexity – p. 146

 Graph Problems
Definition 6.9: The exact clique problem is given as the set
of all (V, E, k) where V is a set of nodes and E is a set of
edges (unordered pairs of nodes) between the nodes of V
and k is an integer such that an instance (V, E, k) is in
EXACTCLIQUE iff the graph (V, E) has a clique with
exactly k nodes. Here a clique is a subset of V such that
each pair of nodes in the subset is connected by an edge.
Theorem 6.10: Exact Clique is NP-complete.
Membership in EXACTCLIQUE can be verified by guessing
a clique of size k (listing out the nodes explicitly) and a
LOGSPACE algorithm then can check that the listed out
subset has k members and each two members of it are
connected.

Computational Complexity – p. 147

 Negation-Free XSAT
Recall that an XSAT instance is given as a set of variables
X and set of clauses Y using these variables X. Now the
instance (X, Y) is in XSAT iff there truth-assignment to the
variables in X which makes in each clause exactly one
literal true. XSAT is NP-complete.
NEGATIONFREEXSAT instances are XSAT instances in
which all literals are only variables and no negated variable
occurs in a clause; NEGATIONFREEXSAT is NP-complete.
To see this, one translates an XSAT instance into a
negation-free XSAT instance as follows: If a variable occurs
only positively, it just remains as it is. If a variable x occurs
only negatively, one replaces each literal ¬x by x without
changing solvability. If a variable x occurs both positively
and negatively, one introduces a new variable y added
replaces in the clauses all ¬x by y and then one adds the
new clause x ∨ y to the instance.
Computational Complexity – p. 148
 Coding XSAT into Cliques
An instance of NEGATIONFREEXSAT is now translated
into an instance of EXACTCLIQUE as follows.
Let V be the set of all pairs (i, j) of numbers such that the
i-th variable occurs in the j-th clause. Let E contain all
unordered pairs (2-element-sets) {(i, j), (i′ , j′ )} where either
i = i′ and j 6= j′ or i 6= i′ and there is no j′′ such that
(i, j′′ ), (i′ , j′′ ) are both in V. Now k is the number of clauses.
Now for every solution of the NEGATIONFREEXSAT
instance with a solution to the variables, the clique of size k
consists of all nodes (i, j) in V where the i-th variable is set
1. The size of this clique is k as for each clause exactly one
node (i, j) goes into this clique. As no clique contains nodes
(i, j), (i′ , j′ ) such that i, i′ are different variables sharing a
clause, each clique codes a partial solution; the coded
solution is a full solution iff its size is k.
Computational Complexity – p. 149
 Knapsack I
Definition 6.11: A list x1 , x2 , . . . , xn , y of natural numbers is
in KNAPSACK iff there is a binary list z1 , z2 , . . . , zn with
P
k=0,...,n xk zk = y. Size parameter n + log max{x1 , . . . , xn , y}.

Theorem 6.12: KNAPSACK is NP-complete.

For membership in NP, one just guesses z1 , . . . , zn . The
homeworks will show that not only XSAT but also X3SAT is
NP-complete. Similarly NEGATIONFREEX3SAT is
NP-complete. Given an instance, let n be the number of its
variables and m be the number of its clauses. Now one
assigns to each variable x̂i of a negation-free X3SAT
instance the number xi = j:x̂i occurs in j−th clause 4j and let
P

y = j=1,2,...,m 4j . The j-th digit of a possible sum is 1 in

P

base 4 iff exactly one xi is selected for the sum where x̂i
occurs in the j-th clause. Thus solvability of the given XSAT
instance equals solvability of the new KNAPSACK instance.
Computational Complexity – p. 150
 Knapsack II
Remark 6.13: KNAPSACK can also be formulated as an
optimisation problem. Given a list xP 1 , x2 , . . . , xn and y, find
the largest possible value the sum k=0,...,n xk zk can take
without becoming strictly larger than y. Solving this problem
is NP-hard. It is, however, unknown whether the problem is
in NP. This is the case if and only if NP = CoNP.
Various optimisation problems are of that form, that solving
them is as hard as NP-solving, but one cannot get the
optimal number without accessing an NP-oracle (= a data
base containing some NP-complete set) more than once.
Note that such an oracle also provides negative evidence,
that is, that something does not have a solution.

Computational Complexity – p. 151

 Function Graphs
A graph of a function f is in NP iff there is a
nondeterministic polynomial time machine which, for all
inputs x, has accepting computations outputting f (x) but not
outputting any other number in an accepting computation.
Example 6.14: Agrawal, Kayal and Saxena [2004] showed
that one can check in polynomial time whether a number is
prime. Thus one can also check whether a factorisation of a
number (a list of numbers whose prodruct is that number)
consists only of prime factors, that is, is the prime
factorisation. The following two functions are in NP:
• Function computing the greatest prime factor;
• Function computing the number of prime factors.
NP algorithm guesses factorisation and verifies all prime; if
so it computes output and accepts computation. These
functions are believed neither to be in P nor to be NP-hard.
Computational Complexity – p. 152
 Homeworks 6.15-6.18
Homework 6.15: Find an algorithm which replaces a
n-variable m-clause kSAT formula by an equivalent
(log(k) + 1)SAT formula with up to n + (log(k) + 1) · m
variables and up to m · k clauses.
√
Homework 6.16: If one splits each k-literal clauses into k
parts instead of k parts and starts with n variables and m
clauses, how many new clauses and variables does one
need in worst case? How is the value k updated?
Homework 6.17: What are the bounds for translating an
n-variable m-clause 4SAT formula into a 3SAT formula?
Why is a further translation into a 2SAT formula impossible?
Homework 6.18: Replace the proof sketch that
3OCCURSAT is NP-complete by a detailed proof that
3OCCUR3SAT is NP-complete, 3OCCUR3SAT consists of
all satisfiable 3SAT formulas in which all variables occur at
most thrice.
Computational Complexity – p. 153
 Homeworks 6.19-6.21
Homework 6.19: Prove that X3SAT is NP-complete. Here
X3SAT is the set of all solvable XSAT instances where each
clause has at most three literals. Show that
NP-completeness also holds if one requires that all literals
are positive, that is, for no variable x there is a literal of the
form ¬x.
Homework 6.20: Prove that whenever A is in both NP and
CoNP, then either NP = CoNP or the problem is not
NP-complete.
Homework 6.21: Consider LOGKNAPSACK where all
numbers xk are in the set {0, 1, . . . , n}. Is LOGKNAPSACK
NP-complete or in P? Prove the answer.

Computational Complexity – p. 154

 Homeworks 6.22-6.26
Consider the following graph problems and prove that they
are either NP-complete or in P.
Homework 6.22: Is there a roundpath visiting each node
once (except that starting node = end node)?
Homework 6.23: Is there a roundpath using each edge
exactly once (nodes might be visited several times)?
Homework 6.24: Can the nodes of the graph be coloured
with two colours such that each neighbouring nodes have
different colours?
Homework 6.25: Can the nodes of the graph be coloured
with three colours such that each neighbouring nodes have
different colours?
Homework 6.26: Can the nodes of the graph be coloured
with four colours such that each neighbouring nodes have
different colours?
Computational Complexity – p. 155
 Lecture 7
Lecture 7 will deepen the knowledge about the complexity
of variants of SAT. As seen in Lecture 6, certain variants like
3SAT and XSAT allow for faster solutions of time O(cn ) with
1 < c < 2 than the usual method to test all combinations of
variables which takes at least 2n steps.
DPLL algorithms are named after Martin Davis, George
Logeman, Donald Loveland and Hilary Putnam who, in two
papers each with some of these authors laid the
fundaments for these algorithms. They solve satisfiability
problems where there are certain constraints on the
clauses, for example, each variable occurring at most k
times or each clause having at most k literals or the overall
number m of clauses being bounded by kn. Under such
additional constraints, the DPLL algorithms provide better
bounds than the trivial algorithm to check all variables.
Computational Complexity – p. 156
Example 7.1: DPLL Algorithm 3SAT
1. Algo(Clause Set F, var x1 , . . . , xn );
2. If there is an empty clause then return(false);
3. If there is no clause then return(true);
4. If there is a clause of form y with y ∈ {xk , ¬xk } then set
y = 1, remove all clauses containing y and remove ¬y
from all remaining clauses and goto 2;
5. If there is a literal y such that ¬y occurs in no clause
then remove all clauses containing y and goto 2;
6. Find variable xk such that (number of 2-literal clauses
with xk , number of 3-literal clauses with xk ) is maximal;
7. If Algo(F ∪ {xk },vars) or Algo(F ∪ {¬xk },vars) is true
then return true else return false.

This is the basic algorithm. Better algorithms are known.

Computational Complexity – p. 157
 Rule Types
It is convenient to see DPLL algorithms as collections of
rules in some order; always the first rule which applies is
done.
Measure is the number of variables (or whatever other
criterion is used to measure the complexity of input).
Simplification rules simplify the formula F by, for example,
setting y = 1 and doing the follow-up in the case that there
is a single literal clause y. The measure is not allowed to go
up in simplification rules.
Branching rules are recursive calls which call several
instances of the main program. Each branch leads to
modifications of the formula which reduce the measure.

Computational Complexity – p. 158

 Branching Factors
If a branching rule has three branches (a), (b), (c) which
reduce the measure by ra , rb , rc respectively, then the
branching factor is the least real number s > 1 such that

s−ra + s−rb + s−rc ≤ 1.

Similarly for branchings with two, four or more outcomes.

Usually there are several different branching rules with
different branching factors in the algorithm. One takes the
maximum of these, call it s̃, and considers the initial
measure, say n. An upper bound on the runtime of the
algorithm is then Poly(n) · s̃n . Here note that m ∈ O(n3 ) for
3SAT; for general satisfiability formulas, this bound is not
there and one would give the bound Poly(n + m) · s̃n .

Computational Complexity – p. 159

 Evaluating the 3SAT Algorithm
Each variable branched occurs both positive and negative.
If it occurs in a 2-literal clause then this on one side of the
branching disappears and on the other side becomes
1-literal clauses which fixes a further variable.
Use the following measure µ: If there is no 2-literal clause
then µ = n else µ = n − 0.21481.
If there is no 2-literal clause, then each branch of a variable
will produce one and the measure decreases on both sides
by 1.21481. Branching factor is s with 2s−1.21841 = 1,
uprounded to 1.7693.
If there is a 2-literal clause, then this might disappear on
both sides of the branching, but on one side of the
branching a further variable will be removed. Branching
factor is s with s−1+0.21481 + s−2+0.21481 = 1, uprounded to
1.7693. Computational Complexity – p. 160
 Modifying the 3SAT Algorithm
Example 7.2: Adding in a new simplification rule, called
resolution: If there is a literal y such that y occurs in 3-literal
clauses but ¬y only in 2-literal clauses, then add for all pairs
of clauses α ∨ y, β ∨ ¬y the new clause α ∨ β (it also has at
most three literals) into F and afterwards remove all clauses
containing y or ¬y.
One uses only the number of current variables as measure
and bills the measure gained by a resolution onto the next
branching rule done; similarly with all other follow-up
savings which are not taken into account.
Each branching rule except for the first and those
immediately after a resolution will create one 2-literal
clause; thus all branching rules except for the first will either
eat up prior savings or remove on at least one side two
variables. Branching factor is s with s−1 + s−2 = 1,
uprounded to 1.6181.
Computational Complexity – p. 161
 State of the Art
For kSAT and deterministic algorithms, the following recent
results are there: Work by Dantsin and coauthors 2002
(branching number 2k/(k + 1)), Moser and Scheder 2011,
Sixue Liu 2018.
Problem 2002 2011 2018
3SAT 1.5 1.3334 1.3280
4SAT 1.6 1.5001 1.4986
5SAT 1.6667 1.6001 1.5995
6SAT 1.7143 1.6667 1.6665
Improvements by continuous refinement of the algorithm
(better case distinctions), usage of new ideas from coding
theory (Uwe Schöning) combined with derandomisation; in
other areas also the use of more sophisticated measures.

Computational Complexity – p. 162

 Example 7.3: An XSAT Algorithm
Exact satisfiability (XSAT) is a problem which has seen a lot
of ongoing research and it is, after plain SAT and kSAT for
constants k ≥ 3 the most investigated variant of SAT in the
field of exponential time algorithms.
Here the algorithms do not need a constraint on the number
of literals per clause; the reason is that the number of
satisfying assignments of a clause with k literals is at most
k, as one literal has to be 1 and the other k − 1 have to be
0; thus one gets k out of 2k possible choices to consider
when one branches the k variables of a k-literal clause.
Note here that a subclause x ∨ x implies that x = 0 and a
subclause x ∨ ¬x implies that all other literals are 0 while for
x it does not matter whether it is 0 or 1; thus under the
assumption that easy simplifications are done, any instance
considered has for all literals of a clause different variables.
Computational Complexity – p. 163
XSAT Algorithm – Simplification Rules
Always do the first of the below possibilities which applies.
• If x appears in a clause of form x ∨ y, x ∨ x ∨ α,
x ∨ y ∨ ¬y ∨ β , x or both x ∨ y ∨ γ, x ∨ ¬y ∨ δ then set x
to ¬y, 0, 0, 1, 0, respectively, and simplify accordingly.
• If a clause is of form x ∨ ¬x or x ∨ y1 ∨ . . . ∨ yk with
y1 , . . . , yk not occurring elsewhere then remove it.
• If there are clauses α and α ∨ β then set all literals in β
to 0.
• If the instance is small and satisfiable then return 1.
• If a subinstance is small and unsatisfiable then return 0.
• If x appears both, positively and negatively, then pick
clauses α ∨ x, β ∨ ¬x and replace all further clauses
γ ∨ x by β ∨ γ and δ ∨ ¬x by α ∨ δ and the two picked
clauses by α ∨ β . Computational Complexity – p. 164
XSAT Algorithm – Branching Rules
• Branch the largest clause x ∨ y ∨ α with x, y occurring
elsewhere as x = 0, y = 0 versus x = ¬y and simplify
accordingly in both branchings.

Number of removed variables in dependence on number of

literals in α.
Literals of α case x = 0, y = 0 case x = ¬y
1 3 3
2 3 4
3 or more 2 5
The worst branching factor is s with 2s−3 = 1 and that
satisfies s < 1.2600. Gordon Hoi constructed in 2020 an
O(1.1674n ) time algorithm.

Computational Complexity – p. 165

7.4: The Exponential Time Hypotheses
All known algorithms for 3SAT, XSAT, X3SAT and other
such problems use time Ω′ (cn ) for some c > 1 and infinitely
many n; this led to the following still unproven hypotheses.
The Exponential Time Hypothesis [Impagliazzo and Paturi
1999]: For each k ≥ 3 there is a constant ck > 1 such that
for every correct kSAT algorithm there are infinitely many n
for which some input with n variables needs at least cnk
computation steps.
The Strong Exponential Time Hypothesis [Calabro,
Impagliazzo and Paturi 2009]: The above ck converge from
below to 2.
Note that 2n · Poly(m + n) is a trivial upper bound for all
SAT problems including all kSAT.

Computational Complexity – p. 166

 Further Properties of ETH
Theorem 7.5 [Impaglazzio and Paturi 1999]: Assuming ETH
with constants c3 , c4 , . . . as above, there is for each k ≥ 3 a
constant dk such that, for all k and all kSAT algorithms,
there are, for infinitely many n, instances of n variables and
up to n · dk clauses for which the kSAT algorithm needs at
least cnk steps.
These constants dk are only shown to exist; they are useful
to prove that ETH, provided it holds, also forces other
problems to use exponential time. This will be done on the
next slides for various examples. The technique presented
here is useful, if one has to prove conditional lower bounds
under the assumption of ETH.

Computational Complexity – p. 167

 ETH applies also to XSAT Part I
Theorem 7.6: Assuming ETH, there are constants
cxsat , dxsat such that every correct XSAT algorithm needs,
for infinitely many n, on some instance with n variables and
up to dxsat n clauses, at least time cnxsat computation steps
to decide the solvability of this instance.
The idea is to translate a 3SAT instance with n variables
and d3 n clauses. For each clause x ∨ y ∨ z one introduces
four new variables t, u, v, w only used for this purpose and
replaces the clause by x ∨ v ∨ w and v → y and w → z; note
that these two implications can be written with the XSAT
clauses v ∨ ¬y ∨ t, w ∨ ¬z ∨ u. If v is 1, so is y; however, t is
needed to make the clause satisfied when both v, ¬y are 0.

Computational Complexity – p. 168

 ETH applies also to XSAT Part II
The so constructed instance has n + 4d3 n variables and up
to 3d3 n clauses.
Assume now that some algorithm solves this in time below
n/(1+4d3 )
c3 . Then this algorithm can be tweeked such that it
also solves the input instances in time cn3 by translating the
input instance with n variables and up to d3 n clauses as
above and then run the algorithm on this larger instance for
(1+4d3 )n/(1+4d3 )
c3 = cn3 not counting the translation time. This
contradicts the choice of the input instances for 3SAT.
Thus the exponential time hypothesis also applies to XSAT.
As all XSAT clauses constructed have only three literals,
one has the following corollary.
Corollary: ETH applies also to X3SAT.
Computational Complexity – p. 169
 ETH with Respect to Clauses I
Theorem 7.7 [Impagliazzo and Paturi 1999]: The problem
3SAT parameterised by the number of clauses m obeys the
Exponential Time Hypothesis, that is, if ETH holds for 3SAT
with respect to n, then ETH also holds for 3SAT with
respect to m.
Proof: By the ETH there are constants c3 > 1, d3 such that
for every correct 3SAT algorithm there are infinitely many
numbers n such that for each of these n there is an
instance with n variables and at most d3 n clauses where
the algorithm needs at least cn3 computation steps. As
m ≤ d3 n, the same algorithm needs, when measured with
1/d 1/d
respect to m, at least (c3 3 )m steps. Note that c3 3 > 1 as
c3 > 1 and 1/d3 is a positive rational number.

Computational Complexity – p. 170

 ETH with Respect to Clauses II
It remains to show that infinitely many different n imply
infinitely many different m.
For this one can use that difficult instances are reduced,
that is, no simplification rule applies directly. Thus every
variable appears at least twice and m ≥ 2/3n.
Now for each n in the above quantification there are only
finitely many matching m, as 2/3n ≤ m ≤ d3 n.
It follows that for every given correct algorithm, there are
infinitely many m for which there is an instance with m
1/d
clauses where the given algorithm uses at least (c3 3 )m
steps. So ETH holds for 3SAT with parameter m.

Computational Complexity – p. 171

 A Nontrivial Algorithm for SAT I
Theorem 7.8: There is a SAT solver using time
O(1.32472m ) provided that every variable appears in a
clause.
Simplification Rules:
• If there is a variable appearing only positive or only
negative, then make all literals of this variable 1 and
remove the corresponding clauses.
• If there is a variable with clauses ¬x ∨ α, x ∨ β1 , . . .,
x ∨ βh then replace these clauses by α ∨ β1 , . . . , α ∨ βk
and remove variable x; clause number goes down by
one (Resolution Step 1).
• If there is a variable occurring as x ∨ α, x ∨ β , ¬x ∨ γ ,
¬x ∨ δ then replace these four clauses by α ∨ γ , α ∨ δ ,
β ∨ γ , β ∨ δ and remove variable x; clause number
unchanged (Resolution Step 2).
Computational Complexity – p. 172
 A Nontrivial Algorithm for SAT II
Branching and Termination Rules:
• If there is no clause, return true; if there is an empty
clause, return false.
• If none of the simplification and termination rules apply,
then every variable appears at least thrice positively
and twice negatively or vice versa. Branch one of the
variables. Branching factor is s with s−2 + s−3 = 1 and
s < 1.32472.
Note that simplification rules capture all variables which do
not have at least five occurrences and for which at least two
occurrences are positive and two occurrences are negative;
thus the above branching rule is the only one needed and
proves that the algorithm runs in time O(1.32472m ). The
state of the art is O(1.2226m ) by Chu, Xiao, Zhang 2021.
Computational Complexity – p. 173
SETH and Other Parameters of SAT
There is also concurrent research with respect to solving
SAT with respect to the number ℓ of literals. Here the state
of the art is O(1.0641ℓ ) by Peng and Xiao [2021].
A SAT algorithm is of course also an algorithm for 3SAT,
4SAT, ...; thus the following corollary holds. Note that for
this algorithm, the difficult cases are where there are less
clauses than variables.
Corollary 7.9: SETH does not apply to kSAT with respect to
the number of clauses or with respect to the number of
literals.

Computational Complexity – p. 174

Exponential Time - Polynomial Space
All the algorithms presented in this lecture use polynomial
space. They are recursive calls where the depth of these
calls is usually the number of variables or less, as each
branching also removes one variable. Furthermore, the
local parts – simplification rules and follow-up of branchings
– can be done in polynomial time and do therefore only
require polynomial sized local memory. Thus, at all stages
of the algorithm and using that the various branches of a
branching are tried out one after another, cause the overall
algorithm only to use polynomial space.
Corollary 7.10 The algorithms of this lecture so far all use
only polyomial space.

Computational Complexity – p. 175

 Exponential Time and Space
However, in some cases, exponential time algorithms also
require exponential space, for example they create a
exponential sized database. Data can be inserted in
polynomial time and also presence of data in it can be
checked, as usually the basic data base operations are in
time logarithmic of the data base size.
An example is the algorithm “Meet in the Middle”, which
branches half of the variables completely and writes all
possible results into the database. Then the other half are
branched equivalently and one checks for each outcome
whether there is a matching entry in the data base. An
example where this is done is if one wants to meet a
specific sum like in the case of Knapsack.

Computational Complexity – p. 176

 Algorithm Meet in the Middle
ALgorithm 7.11 [Horowitz and Sahni 1974]: Consider a
Knapsack Instance with n numbers x1 , x2 , . . . , xn and target
sum y.
In the first round, one runs over all binary z1 , z2 , . . . , zn/2
and computes
y0 = Σh=1,...,n/2 zh xh
and writes each y0 into the data base. In the second round,
one runs over all binary zn/2+1 , zn/2+2 , . . . , zn and computes

y1 = Σh=n/2+1,...,n zh xh

and checks for each y1 whether y − y1 is in the data base,

that is, equal to some y0 . If there is a y1 with matching y0
then there is a solution else there is no solution.
The time complexity is 2n/2 · Poly(n).
Computational Complexity – p. 177
 Homeworks 7.12-7.15
For the following homeworks, improve the corresponding
algorithms in the lecture with respect to computation time
and verify your algorithm. It is not requested to meet the
state of the art (those algorithms are very complicated).
Homework 7.12: Improve Algorithm 7.2 for 3SAT.
Homework 7.13: Provide an algorithm for 4SAT with
nontrivial upper bound.
Homework 7.14: Improve Algorithm 7.3 for XSAT.
Homework 7.15: Improve the algorithm of Theorem 7.8 for
SAT with respect to clauses.

Computational Complexity – p. 178

 Homeworks 7.16-7.18
Homework 7.16: Provide an algorithm in terms of number
of literals ℓ with nontrivial upper bound (O(1.4ℓ ) or better).
Assume for the following homeworks that ETH holds for
SAT when parameterised by n or parameterised by m
(choose what you need).
Homework 7.17: Prove that ETH holds for SAT when
parameterised by ℓ.
Homework 7.18: Prove that ETH holds for SAT when
parameterised by the number of clauses with at least five
literals.

Computational Complexity – p. 179

 Homeworks 7.19-7.21
Homework 7.19: Prove that ETH holds for Knapsack
(decision version) by reducing X3SAT.
Homework 7.20: Read up on data bases and explain in
detail how the data base functions of the exponential size
data base can be realised such that inserting data and
checking of existance of data can both be done in
polynomial time, that is, logarithmic in the size of the data
base.
Homeworks 7.21: Provide an algorithm for XSAT based on
the idea “meet in the middle”. Explain what is stored in the
data base. This algorithm will not be competitive, neither in
time nor space usage, but it is good to understand it.

Computational Complexity – p. 180

 Task 7.22
Task 7.22: Consider the following problem. Given a SAT
instance with n variables and n3 clauses, does this instance
have a satisfying assignment with at most log3 (n) variables
being 1?
Assuming the Exponential Time Hypothesis, prove that this
problem is neither in P nor NP-complete.
Note that the Exponential Time Hypothesis is needed for
both separations.

Computational Complexity – p. 181

 Lecture 8
Lecture 8 will deal with topics related to counting. One is a
promise problem: Given a SAT formula such that it either
has 0 solutions or a number of solutions from some set A
not containing 0. Is detecting a solution for this problem
easier than solving SAT itself?
Furthermore, requiring an odd number of solutions (in place
of just at least one solution) leads to the problem ParityP.
How does this problem relate to NP?
Furthermore, one would like to count the number of
solutions of a SAT formula, kSAT formula and so on. How
difficult is this (perhaps in dependence of k)?

Computational Complexity – p. 182

 Random Polynomial Time
Definition 8.1: A problem L is in Random Polynomial Time
(RP) if there is a probabilistic algorithm running in
polynomial time such that for all words w, if w ∈ L then the
algorithm says with probability at least 1/4 ACCEPT while
for all words w ∈/ L the algorithm always says REJECT.
Example 8.2: Given a multivariate polynomial p over the
rational numbers with integer coefficients, the question
whether this polynomial is nonzero on some inputs is in RP.
Algorithm: Let n be the number of variables and d be the
degree of the polynomial. One draws at random for every
variable a random number between 0 and 3d · n and one
evaluates the polynomial at this number — if the result is 0
then the polynomial is everywhere 0 with probability at least
1/2; if the result is nonzero then the polynomial is not
everywhere zero.
Computational Complexity – p. 183
 Verification of Algorithm I
A polynomial in one variable of degree d either has at most
d zeroes or is everywhere zero. Let n be the number of
variables. Assume that the overall polynomial is not zero
everywhere.
If for one variable x there are more than d values where the
polynomial when fixing x vanishes than the whole
polynomial vanishes, as one has when fixing all variables
besides x a single variable polynomial which vanishes at
more than d places and thus the polynomial is zero
everywhere. Thus when picking values for x1 , x2 , . . . , xn at
random and when the polynomial does not vanish, then the
value chosen for each xk hits with probability at most 1/3n
a value where the polynomial, after x1 , x2 , . . . , xk are fixed,
vanishes.

Computational Complexity – p. 184

 Verification of Algorithm II
It follows that the overall probability to hit a zero of the
polynomial is at most 1/3 and so the procedure returns with
probability 2/3 a nonzero value of that if such a value exists.
A problem is that the polynomial might return a value up to
(3cnd)d where c is the largest coefficient and one might be
interested to have the dependence on the factor d reduced.
This can be achieved by randomly choosing a prime p with
(3cn log(d))2 ≤ p ≤ (3cn log(d))3 and then the computation
of the polynomial value modulo p is polynomial in the
number of digits of p and the degree d. This is in particular
important if the formula of the polynomial is given by an
arithmetic circuit and not an explicit list of all monomial
terms with coefficients.
Remark: The problem whether a multivariate polynomial
given by an arithmetic circuit has a nonzero value is the
main known problem inside RP not known to be in P.
Computational Complexity – p. 185
 Theorem of Valiant and Vazirani I
Theorem 8.3 [Valiant and Vazirani 1986]: Assume that
there is an RP-algorithm which outputs 1 with probability
1/2 if a 3SAT formula φ has exactly one solution and
outputs 0 always if φ has no solution. Then NP = RP.
Algorithm: The idea is to intersect the given problem with
half spaces where each half space is given as randomly
drawn set Wk of variables such that an assignment to
x1 , . . . , xn is in the half space iff an odd number of the
variables in Wk has the value 1 assigned. Now one draws
uniformly half spaces W1 , . . . , Wn and if the φ has a
satisfying assignment then there is with probability 1/4 a k
such that exactly one satisfying assignment is in
W1 ∩ . . . ∩ Wk . So one has n related problems to check.

Computational Complexity – p. 186

 Theorem of Valiant and Vazirani II
By assumption some RP algorithm F returns with
probability 1/2 ACCEPT if there is exactly one assignment
and always returns REJECT if there is no satisfying
assignment.
Thus if φ is satisfiable then drawing random subspaces
W1 , . . . , Wn and checking each with F returns with
probability 1/8 an ACCEPT in at least one of the tasks. If φ
is not satisfiable, F would return also REJECT on all tasks
φ ∩ W1 ∩ . . . ∩ Wk .
For the borderline case that φ has one unique solution, note
that this solution is in φ ∩ W1 with probability 1/2. This fact
is taken into account for computing the overall probability
1/4 that one of the sets φ ∩ W1 ∩ . . . ∩ Wk has a solution.
Furthermore, one has to do one additional test, namely
whether the assignment of all variables to 0 is a solution, as
that one would never go into the subspaces.
Computational Complexity – p. 187
Theorem of Valiant and Vazirani III
The only problem is that φ ∩ W1 ∩ . . . ∩ Wk is not a 3SAT
formula. Thus Valiant and Vazirani introduced for each Wh
given by variables xi1 , . . . , xij new helper variables
y1 , y2 , . . . , yj such that y1 ↔ xi1 and yh+1 ↔ yh ⊕ xih and
yj ⊕ xij is 1. Each clause in this formula can be translated
into a 3SAT clause and so the intersection
φ ∩ W1 ∩ . . . ∩ Wk becomes a formula with n + (nk)/2
variables expectedly, due to randomness, it can be slightly
more or less, but at most n(k + 1) variables.
For example, the clause y1 ⊕ x2 ⊕ y2 would become the
following four clauses: y1 ∨ x2 ∨ y2 , y1 ∨ ¬x2 ∨ ¬y2 ,
¬y1 ∨ x2 ∨ ¬y2 , ¬y1 ∨ ¬x2 ∨ y2 . It are all clauses where an
odd number of literals is positive.

Computational Complexity – p. 188

 Applications of the Theorem
One might ask to which degree can one strengthen the
Theorem of Valiant and Vazirani. For this one formulates
the Random Exponential Time Hypothesis: Every algorithm
which evaluates satisfiable formulas with probability 1/8 to
ACCEPT and which always evaluates unsatisfiable
formulas to REJECT runs on infinitely many instances in
the worst case cn time for some c > 1 where n is the
number of variables.
Due to the quadratic increase in the number of variables,
the Random Exponential Time Hypothesis implies, using
the Theorem of Valiant and Vazirani, only this: Checking
whether a 3SAT formula with at most one satisfying
assignment is indeed satisfiable by√
a randomised algorithm
requires, in the worst case, time c n for some constant
c > 1 independent of the algorithm.
Computational Complexity – p. 189
 Counting 2SAT Solutions I
Algorithm 8.4: Let φ be a 2SAT formula. Count(φ) does the
first case which applies.
1. If formula φ consists of two disjoint parts with disjoint
variable sets X and Y, respectively, then return
Count(φX ) · Count(φY ).
2. If φ has 14 or less variables, solve them by brute-force
and count the solutions in O(1) time.
3. If the formula consists of variables each having at most
two neighbours then they either form a “line” or “circle”
of n variables. Branch one or two of them such that they
split into parts with n/2 − 1/2 variables at most.
4. Branch the variable with the most neighbours (it are at
least three).

Computational Complexity – p. 190

 Counting 2SAT Solutions II
Item one is a common item done in algorithms which
exploits that there are two independent subproblems which
can be solved by itself. The number of the overall solutions
is the product of the number of solutions of each
subproblem.
Item two is the bottom case which, due to n ≤ 14, counts
only O(1).
Item three branches one or two variables. Then solving the
halves has complexity 8 · O(cn/2−1/2 ) for the resulting
halves in all four cases - they are different due to taking
border conditions into account. As n ≥ 15, each item saves
at least 8 variables, much better than O(cn ). The branching
factor itself is in the worst case (n = 15) the s with 8s−8 = 1
what is below 1.2969.
Computational Complexity – p. 191
 Counting 2SAT Solutions III
Item four has branching factor s with either s−1 + s−4 = 1 or
s−2 + s−3 = 1, as each neighbour is forced to take a value in
one of the two branchings.
For example, for a variable x, if it occurs in clauses x ∨ u,
x ∨ v and ¬x ∨ w, one has to set u, v = 1 in the case that
x = 0 and w = 1 in the case that x = 1. Thus one
eliminates in one branch three and in the other branch two
variables. Thus the branching factor is the s with
s−2 + s−3 = 1, that is, s = 1.32472. In the case that x occurs
positively in all clauses (or negatively in all clauses), one
eliminates only x in one branching and also all its
neighbours in the other branching; hence the branching
factor is the s with s−1 + s−4 = 1, that is, s = 1.3803.
The overall branching factor is max{1.2969, 1.32472, 1.3803}
and the algorithm runs in time O(1.3803n ).
Computational Complexity – p. 192
Unique Exponential Time Hypothesis
Definition 8.5: The Unique Exponential Time Hypothesis
(UETH) says that there are constant c3 , d3 such that for
every algorithm there are infinitely many n and instances of
3SAT with n variables, up to d3 n clauses and either zero or
one solutions such that the algorithm either makes a
mistake or runs longer than cn3 steps.
Theorem 8.6: Assuming UETH with c3 , d3 as above, the
runtime complexity to count 2SAT solutions is at least
1/(d +1)
Ω′ ((c3 3 )n ). In particular, for every correct algorithm
counting the number of 2SAT solutions there are infinitely
many n with instances of (1 + d3 )n variables where the
algorithm needs at least time (c3 )n .

Computational Complexity – p. 193

 Construction of Theorem 8.6 I
Let a SAT instance φ with n variables x1 , . . . , xn and m
clauses be given, by assumption m = d3 n and one can use
clause repetition to get the exact bound. Now one takes m
new variables y1 , . . . , ym . For each literal z occurring in the
k-th clause (which is either of the form xh or of the form
¬xh ), one adds the implication z → yk into the 2SAT
formula.
If an assignment to x1 , . . . , xn makes all clauses true, then
all y1 , . . . , yn are forced to be 1. So this assignment
contributes 1 to the number of all solutions of the 2SAT
formula. If an assignment fails to make the k-th clause true,
then there is no condition on the value of yk and therefore
solutions to 2SAT based on that assignment can either
have yk = 0 or yk = 1 without modifying any other variable.
Thus there is an even number of solutions extending this
assignment. Computational Complexity – p. 194
 Construction of Theorem 8.6 II
For example, if the clauses are x1 ∨ x2 ∨ x3 , ¬x1 ∨ ¬x2 ∨ x4
then y1 , y2 are the new variables and the clauses are
x1 → y1 , x2 → y1 , x3 → y1 , ¬x1 → y2 , ¬x2 → y2 , x4 → y2 .
Now (x1 , x2 , x3 , x4 ) = (1, 0, 1, 1) makes both clauses true
and forces y1 = 1, y2 = 1, so it contributes one 2SAT
solution. However, (x1 , x2 , x3 , x4 ) = (0, 0, 0, 0) makes only
the second clause true and thus no condition implies
y1 = 1, thus it contributes two 2SAT solutions.
As the instance φ in the proof has at most one solution, the
overall number of 2SAT solutions is odd iff the instance φ is
satisfiable. Thus one can compute the solvability of the
3SAT instance from the number of solutions of the new
2SAT instance. Furthermore, note that 3SAT is taken, as
there the number of clauses is bounded by d3 n what is not
the case for the general SAT.
Computational Complexity – p. 195
 Related Results
Corollary 8.7: Given a 3SAT instance φ with n variables and
m clauses, one can compute a 2SAT instance ψ with n + m
variables and up to 3m clauses such that φ has an odd
number of solutions iff ψ has an odd number of solutions.
Corollary 8.8: If Parity3SAT requires exponential time, so
does Parity2SAT. In particular Parity2SAT requires
exponential time infinitely often under the assumption
UETH.

Computational Complexity – p. 196

 NP-Hardness of Count2SAT
Corollary 8.9: Counting 2SAT instances is NP-hard.
Given a 3SAT instance φ with n variables and m clauses,
one assigns to the k-th clause a set Yk of n + 1 new
variables and for each variable y ∈ Yk and each literal z of
the k-th clause one puts the implication z → y. Then there
are for each assignment of φ with h unsatisfied clauses
(2n+1 )h many solutions of the new 2SAT instance ψ and
thus the number of solutions of φ is that of ψ modulo 2n+1 ;
note that φ has at most 2n solutions.
Theorem 8.10: Counting 2SAT modulo a fixed number k is
as hard as counting 3SAT modulo a fixed number k,
provided that one allows a polynomial increase of the
number of variables.

Computational Complexity – p. 197

 Random kSAT
The following discussion is based on work of Akmal and
Williams 2022. Consider the following promise problem.
Definition 8.11: Rq kSAT is the promise problem requesting
an algorithm to REJECT in the case that there is no
solution and to ACCEPT in the case that there are at least
q · 2n solutions where q is a fixed positive rational and k a
natural number.
Theorem 8.11: The problem Rq kSAT can be solved in
polynomial time for all fixed q, k.
Note that the problem RP is not known to be in polynomial
time; the main reason that this result does not transfer is
that one uses the normal SAT and the normal SAT can have
much longer and more clauses than kSAT.

Computational Complexity – p. 198

 Proof of Theorem 8.11 I
A set S of clauses is called disjoint, if no two clauses have a
variable in common. Note that a disjoint set of h kSAT
clauses has at most 2n−hk · (2k − 1)h satisfying
assignments. Thus the ratio of satisfying by all assignments
is at most ((2k − 1)/2k )h what is below q for sufficiently high
h. That is, for every q > 0 and k ≥ 1 there is a constant h
where ((2k − 1)/2k )h < q. Here h depends only on k, q and
not on any further parameter of a given kSAT instance.
The proof is done by induction. A polynomial time algorithm
exists for 2SAT. So assume now that Fk (φ) is {0, 1}-valued
where the value 1 is always taken when there are at least
q2n(φ) solutions and 0 is always taken when there is no
solution. Furthermore, assume that Fk runs in polynomial
time. Now one uses Fk to construct Fk+1 .
Computational Complexity – p. 199
 Proof of Theorem 8.11 II
So let h be such that whenever there are h or more disjoint
clauses then there are less then q · 2n solutions. Given an
(k + 1)SAT instance φ, one first constructs with a greedy
algorithm a maximal disjoint set – maximal in the sense that
there is no disjoint superset of clauses. If it has h or more
clauses one returns 0. Otherwise let X be the set of
variables occurring in these clauses and consider, for each
assignment a to these clauses, the formula φa obtained by
setting the variables in X according to the assignment a.
Let n(φa ) be the number of remaining variables. Now one
returns 1 iff there is an assignment a to the variables in X
for which Fk (φa ) is 1.

Computational Complexity – p. 200

 Proof of Theorem 8.11 III
The correctness of Fk+1 follows from that of Fk as now
explained. Note that each clause has one literal with a
variable in X and thus if φ is a (k + 1)SAT formula then all
φa are kSAT formulas.
If φ has no solutions then so do all φa and Fk (φa ) = 0 for all
assignments a to the variables in X and Fk+1 (φ) = 0.
If φ has at least q2n solutions, then at least one φa must
also have at least q2n(φa ) solutions and Fk (φa ) = 1 for this
a. Thus Fk+1 (φ) = 1.
The values k, h, q are O(1) for fixed choices of these, as the
size parameter is n. Thus Fk+1 consists of an Poly(n)
search for the maximal disjoint set and, if this is below size
h, of Poly(n) handling for the variables to be set (they are
constantly many, but the formula needs to be adjusted) and
O(1) excecutions of Fk (φa ), thus Fk+1 runs in Poly(n) time.
Computational Complexity – p. 201
 Results of Akmal and Williams
Theorem 8.13: Given a rational q > 0 and an integer k,
there is a polynomial time algorithm to decide whether a
given kSAT formula with n variables has at least q · 2n
solutions.
Theorem 8.14: Furthermore, there is a polynomial time
algorithm which decides whether a 3SAT formula with n
variables has at least 2n−1 + 1 solutions.
Theorem 8.15: It is NP-complete to decide whether a 4SAT
formula with n variables has at least 2n−1 + 1 solutions.

Computational Complexity – p. 202

 Classes BPP and PP
Definition 8.16: The class PP – Probabilistic Polynomial
Time consists of all problems L for which a probabilistic
polynomial time Turing machine accepts an input of size n
with probability of at least 2−0.5 + 2−f (n) if it is in L and f is a
fixed polynomial and rejects with probability at least 2−0.5 if
it is not in L.
Definition 8.17: The class BPP – Bounded Probabilistic
Polynomial Time consists of all problems L for which
probabilistic Turing machine accepts with probability of at
least 23 the members of L and rejects with probability at
least 23 the nonmembers of L. Here 23 can be replaced by
any fixed threshold strictly greater than 12 .

Computational Complexity – p. 203

 Homeworks 8.18-8.21
Prove that the following problems are hard under
assumption of UETH with constants c3 , d3 to be used for
the conditional lower runtime bounds below.
Homework 8.18: Mod3 2SAT, that is, whether the number of
solutions of a 2SAT formula are nonzero modulo 3 with
′ n/(1+2d3 )
bound Ω (c3 ).
Homework 8.19: Mod5 2SAT, that is, whether the number of
solutions of a 2SAT formula are nonzero modulo 5 with
n/(1+3d3 )
bound Ω′ (c3 ).
Homework 8.20: Mod7 2SAT, that is, whether the number of
solutions of a 2SAT formula are nonzero modulo 7 with
n/(1+4d3 )
bound Ω′ (c3 ).
Homework 8.21: Decision Knapsack with n variables and
size of numbers being polynomial in n (the size is number
of bits in binary representation).
Computational Complexity – p. 204
Task 8.22 and Homeworks 8.23-8.24
Task 8.22: Show that determining whether a SAT formula
has at least 2n−1 + 1 solutions is a problem in PP; note that
this is not a promise problem but that the input should be
rejected if the formula has at most 2n−1 solutions.
Furthermore, show that under the assumption of UETH,
this problem needs on infinitely many instances exponential
time to be solved.
Homework 8.23: Prove that every problem in NP is also in
PP.
Homework 8.24: A problem L is many-one reducible to
RSAT via f , if f computes in polynomial time a satisfiability
formula and for every x ∈ L, more than half of the
assignments of f (x) are solutions and for every x ∈
/ L, f (x)
has no solution. Show that all problems many-one reducible
to RSAT are in BPP and in RP.
Computational Complexity – p. 205
 Homework 8.25-8.27
Homework 8.25: Manders and Adleman [1976] showed that
the following problem is NP-hard: Given three natural
numbers a, b, c with input size n, check whether there are
natural numbers x, y with ax2 + by = c. Provide an
exoponentuial time algorithm for the related problem to
count all solutions (x, y) for this problem using time
Poly(n) · 2n/2 .
Homework 8.26: Determine the complexity of the following
related problem: Given natural numbers a, b, c, d, count
how many numbers x satisfy ax3 + bx2 + cx = d.
Homework 8.27: Determine the complexity of the following
related problem: Given natural numbers a, b, c, d, count all
tuples (x, y, z) such that (a + 2)x + (b + 2)y + (c + 2)z = d.
Under ETH, is the best algorithm in LOGSPACE, in P and
requiring linear space or requiring exponential time? Computational Complexity – p. 206
 Lecture 9
Some Problems like KNAPSACK of m numbers do not have
known algorithms running in time cm/2 for any c < 2, even
when using exponential space. Another problem often
coded is SAT itself. These problems are used to get
conditional lower bounds near the trivial algorithms for three
famous problems inside the class P. These three problems
are 4SUM (Is the sum of four number in S equal to the
target?), OVP (Are two binary vectors in S orthogonal, that
is, disjoint), DVP (Are two ternary vectors in S different in all
coordinates?). Here S is always a set of n objects of the
type mentioned in the questions before.
A related problem where the lower bound does not follow
from known others is 3SUM (Is the sum of three numbers in
S equal to the target?). Here the problem itself became a
benchmark problems to show other conditional lower
bounds.
Computational Complexity – p. 207
 3SUM and 4SUM
Definition 9.1: For k ≥ 2, the k-sum problem is the problem
to decide whether a set of n numbers x1 , x2 , . . . , xn
contains k elements whose sum is a target number y;
usually this target number is 0. For n = 2 the problem is in
O(n log(n)) and the reason is that one can use a database
with logarithmic update and check time and first one writes
all the n numbers into the data base and then one checks,
for each number xk whether y − xk is in the database.
This algorithm is quadratic for k = 3 and for k = 4, in the
latter case, one would write all sums of two numbers into
the data base and then check for all i, j whether y − xi − xj
is in the data base. So both have O(n2 log(n)) upper bound
on their computational complexity.

Computational Complexity – p. 208

 Reducing KNAPSACK to 4SUM
Assume that the given Knapsack problem has 4m numbers
such that a subsum should give the target z. Now group
them in four groups of m numbers and let n = 4 · 2m . Call
the groups V, W, X, Y and let now V′ , W′ , X′ , Y′ contain
each all sums of some of the members of V, W, X, Y,
respectively, so each of V′ , W′ , X′ , Y′ has up to n/4
members. Now let V′′ = {625v + 1 : v ∈ V′ },
W′′ = {625w + 5 : w ∈ W′ }, X′′ = {625x + 25 : x ∈ X′ } and
Y′′ = {625y + 125 : y ∈ Y′ }. Furthermore, let
z′′ = 625z + 156, note that 156 = 125 + 25 + 5 + 1. Thus
four numbers v′′ , w′′ , x′′ , y′′ from V′′ ∪ W′′ ∪ X′′ ∪ Y′′ can
only sum up to z′′ if v′′ ∈ V′′ , w′′ ∈ W′′ , x′′ ∈ X′′ , y′′ ∈ Y′′ .
Now if an 4SUM algorithm uses only time O(nc ) with c < 2
then the KNAPSACK algorithm for the original numbers
cm c/4 4m c/4
√
uses only time O(2 ) = O((2 ) ) where 2 < 2.
Computational Complexity – p. 209
 3SUM and 4SUM
While the proof for 4SUM is based on KNAPSACK
hardness, there is no such proof that 3SUM requires at
least time nc for all c < 2. However, no algorithm which
beats any nc for any fixed c < 2 for infinitely many n has
been found, one conjectures it does not exist.
Statement 9.2: 3SUM Hypothesis: There is no c with
1 < c < 2 and no algorithm solving 3SUM correctly which
runs in time O(nc ). Indeed, it is even conjectured that every
algorithm solving 3SUM has for each c < 2 the minimum
complexity nc for almost all n.
The 3SUM Hypothesis is used in many applications to show
conditional lower bounds; originally one wanted even to
show that an algorithm needs Ω(n2 ) time, but this original
version of the 3SUM conjecture was broken by Grønlund
and Pettie 2014 who showed the upper bound of time
O(n2 · (log log n/log n)2/3 ) for 3SUM.
Computational Complexity – p. 210
SUM, SUM’, SUM* and Convolution
Definition 9.3: In each case, n numbers x1 , x2 , . . . , xn are
given and in the first two cases also a target y. Now an
instance (x1 , x2 , . . . , xn , y) is in the following set if the
corresponding condition is satisfied:
3SUM: There are i, j, k with xi + xj + xk = y.
3SUM’: There are i, j, k with i ∈ X, j ∈ Y, k ∈ Z and
xi + yj + zk = y where X, Y, Z are a partion of the possible
indices (usually one splits the numbers into three equally
sized sets).
3SUM*: There are i, j, k with xi + xj = xk .
3SUM*-Convolution: There are i, j with xi + xj = xi+j .
Theorem 9.4: All these problems are equivalent in the
sense that if there is a c < 2 such that one of them is
solvable in O(nc ) time then this is true for all of them.
Computational Complexity – p. 211
 Sample Proof for SUM*
Recall that the 3SUM hypothesis says that 3SUM cannot be
solved in time O(nc ) for c < 2. Assume the 3SUM
hypothesis holds. Then also 3SUM* cannot be solved in
time O(nc ) for any c < 2.
For this one takes a 3SUM instance with y = 0.
Furthermore, one let r be a number at least 10 times as
large as the largest of the numbers in the set. Now let, for
k = 1, . . . , n, the number z2k−1 = r + xk and z2k = 2r − xk .
If xi + xj + xk = 0 then xi + yj = −xk and zi + zj = zk . If
zi′ + zj′ = zk′ then zi′ , zj′ < 1.5r and zk′ > 1.5r, thus
zi′ = r + xi , zj′ = r + xj and zk′ = 2r − zk for some i, j, k; it
follows that xi + xj = −xk and xi + xj + xk = 0.
If now an algorithm needs time O((2n)c ) for some c < 2 for
3SUM* then this time is also O(nc ) and together with the
linear time translation it decides 3SUM in time O(nc ).
Computational Complexity – p. 212
 The Orthogonal Vector Problem
Definition 9.5: Two m-bit vectors are called orthogonal (or
disjoint) if for every position e, at most one of them is 1. The
Orthogonal Vector Problem (OVP) is the task to check
whether a list of n m-bit vectors has two orthogonal vectors.
Definition 9.6: Two vectors are coordinate-wise distinct iff
there is no coordinate on which both take the same value.
The Distinct Vector Problem (DVP) is the task to check
whether a list of n ternary m-digit vectors contains a pair of
coordinatewise distinct vectors.
Definition 9.7: The Orthogonal Vector Hypothesis says that
there are no c < 2 and no d > 0 such that some algorithm
can solve OVP with n binary m-bit vectors in time O(nc md ).
Definition 9.8: The Distinct Vector Hypothesis says that
there are no c < 2 and no d > 0 such that some algorithm
can solve DVP with n ternary m-bit vectors in time O(nc md ).
Computational Complexity – p. 213
The Problems OVP, DVP are SAT-hard
One starts with the following assumption.
Definition GSETH: This hypothesis says that there are no
c < 2 such that there is an algorithm which can decide SAT
in time O(cn ) · Poly(m + n). Note that here the polynomial
in n + m can be chosen in dependence of the algorithm.
Note that GSETH is more general than SETH and SETH
implies GSETH, but not vice versa. It is consistent with
current knowledge that SETH might fail while GSETH still
holds, the reason is that SETH only applies to kSAT
formulas.
Theorem 9.9: GSETH implies that the problems OVP and
DVP cannot be solved in time O(nc md ) for any c < 2 and
d > 0 with n being the number of vectors and m being the
dimension (number of coordinates of the vectors).
Computational Complexity – p. 214
 Proof of Theorem 9.9 I
Assume that a SAT instance is given. One devides the n
variables into two equal-sized lists x1 , . . . , xn/2 and
xn/2+1 , . . . , xn , here n/2 is down-rounded. Furthermore,
c1 , . . . , cm is the list of clauses in this instance. One
assumes that either OVH or DVH are false and uses the the
corresponding problem is solvable in (2n/2 )d · Poly(m)
where one assumes in addition that m ≥ n, so that a
polynomial in n could be replaced by one in m – note that
SAT instances with less clauses than variables can be
solved in time O(1.2226n ) as there is a O(1.2226m )
algorithm for SAT and m ≤ n; thus they do not need to be
considered.
Now one creates sets V, W of m-vectors which contain for
each assignment of the first n/2 and second n/2 variables
the vectors v, w which have at position k a 1 if the first
respective second half of variables makes the clause true
and a 0 otherwise. Computational Complexity – p. 215
 Proof of Theorem 9.9 II
The next step depends on the problem to be shown to be
hard.
Orthogonal Vector Problem: Let U to be the set of all
vectors (1 − u(1))(1 − u(2)) . . . (1 − u(m))01 : u ∈ V and all
vectors (1 − u(1))(1 − u(2)) . . . (1 − u(m))10 : u ∈ W. Note
that the 01 and 10 at the end enforce that one vector is
taken from V and one from W when having an orthogonal
pair. If these are derived from v, w then v, w cannot have at
the same position a 0 and therefore the assignments used
for v, w combine to an assignment for all variables which
has for every clause ck that either the first half or the
second of of the variables are assigned such that ck is true.
A (2n/2 )d · Poly(m) time algorithm for solving U produces
then a (2d/2 )n · Poly(n + m) time algorithm for SAT, in
contradiction to GSETH; thus GSETH implies OVH.
Computational Complexity – p. 216
 Proof of Theorem 9.9 III
Now the other case.
Distinct Vector Problem: Let U to be the set of all vectors
v(1)v(2) . . . v(m)1 : v ∈ V and all vectors
(2 · w(1))(2 · w(2)) . . . (2 · w(m))2 : w ∈ W. Note that the 1
and 2 at the end enforce a pair of cordinate-wise distinct
vectors in U must stem from vectors in V and W,
respectively. These cannot be 0 in both coordinate, but if
they are 1 in both coordinates, the translated vectors will
have 1 and 2, respectively. So again solving the
coordinate-wise distinct vector problem in time
(2n/2 )d · Poly(m) gives on instance U a runtime of
(2d/2 )n · Poly(n + m) on the corresponding instance of SAT,
in contradiction to GSETH. So GSETH also implies DVH.

Computational Complexity – p. 217

 The Binary Distinctness Case
Proposition 9.10: One can in time O(mn log(n)) check
whether in a set S of n binary m-bit vectors there are two
vectors x, y which are binary distinct.
Proof: Note that two vectors x, y are coordinate-wise
distince iff for all k ∈ {1, 2, . . . , m} the condition
x(k) + y(k) = 1 holds, that is, y is the bit-complement of x.
Thus the idea is again a meet-in-the-middle algorithm: First
one writes of each vector the complement into a data base.
This uses up time O(mn log(n)), where the mlog(n) steps
are needed to adjust the indices of the data base and to
write the number into it. Then in a second phase, one goes
again over the data base and checks for each vector x,
whether it is in the data base, that is, whether it is the
bit-wise complement of another vector y. This takes also
time O(mn log(n)).
Computational Complexity – p. 218
 Homeworks 9.11-9.14
Complete the following items of the proof of Theorem 9.4.
Assume that c < 2.
Homework 9.11: Show that if 3SUM’ can be solved in time
O(nc ) then also 3SUM can be solved in time O(nc ).
Homework 9.12: Show that if 3SUM∗ -Convolution can be
solved in time O(nc ) then also 3SUM can be solved in time
O(nc ).
Homework 9.13: Show that if 3SUM can be solved in time
O(nc ) so can 3SUM∗ -Convolution.
Homework 9.14: Provide an example of a set X such that
four of its members sum to 0 and also two of its members
sum to 0 but not three of its members sum to 0.

Computational Complexity – p. 219

 Homeworks 9.15 and 9.16
Homework 9.15: The numbers 243, 244, . . . , 728 have all,
when written as ternary numbers, six digits. View them as
ternary six-digit vectors. Find the first x > 243 such that the
set of all numbers from 243 to x (inclusively of the borders)
has two component-wise distinct six-digit numbers y, z.
Find these y, z and write them as both, ternary and decimal
numbers.
Homework 9.16: The numbers 81, 82, . . . , 242 have all,
when written as ternary numbers, five digits. View them as
ternary five-digit vectors. Find the largest x < 242 such that
the set of all numbers from x to 242 (inclusively of the
borders) has two component-wise distinct five-digit
numbers y, z. Find these y, z and write them as both,
ternary and decimal numbers.

Computational Complexity – p. 220

 Homeworks 9.17 and 9.18
Homework 9.17: Let X = {00, 01, 02, . . . , 98, 99} be the set
of all two-digit numbers (when needed with leading zeroes).
Find a subset Y as small as possible such that, whenever
someone selects four different digits i, j, k, h the set of all
numbers in Y written only with these digits is nonempty and
contains two component-wise distinct vectors.
Homework 9.18: Assume that

Xm,n = {xk = (k + m)2 : k ∈ {1, 2, . . . , n}}.

Find an example where Xm,n is in 3SUM∗ but not

3SUM∗ -Convolution. Furthermore, find a a function f such
that Xm,f (m) is in SUM∗ -Convolution for all even m ≥ 2.

Computational Complexity – p. 221

 On the Midterm Test
In the second half of this lecture is the Midterm Test. The
scope of the test is Lectures 1 – 7 and questions might
relate to everything in these lectures. They will, however, be
easier than the average homework, as for homeworks you
can consult the internet and see what researchers did
there. If branching factors or something like this are
needed, they are listed out in the question paper.
This is a closed book exam and you should learn the
material of the seven first lectures as good as possible.
One A4 page helpsheet is allowed.

Computational Complexity – p. 222

 Lecture 10
Complexity of Concrete Problems in P. This lecture looks
into P more from the perspective of complete problems.
The lecture differs from Lecture 9 by mainly studying
nontrivial upper bounds in cases where the lower bounds
are either unknown (for matrix multiplication) or almost
trivial (for multiplication).
Multiplication is, in contrast to addition, an operation where
there is no trivial upper bound on the complexity - the
bound O(n) for adding and subtracting n-digit numbers is
trivial as reading the input (or inspecting the input on the
input tape) takes already that amount of time.
Furthermore, nondeterministic deciders are investigated
which can use nondeterminism, but have in both cases
(inside and outside the language) to provide a decision
which is never wrong (but might be absent if wrong
nondeterministic decisions are taken).
Computational Complexity – p. 223
 Analysis of Recurrences
Jon Bentley, Dorothea Blostein and James B. Saxe [1980]
created a unified approach to resolve recurrence relations
in complexity of algorithms with this theorem.
Theorem 10.1: Assume that an algorithm solves problems
of size O(nk) by calling it self h times with subproblems of
size n and furthermore produces local computations of time
O(nℓ · logp (n)). Then the following holds:
• If log(h)/log(k) > ℓ then the time bound is
O(nlog(h)/log(k) );
• If log(h)/log(k) = ℓ then the time bound is
O(nlog(h)/log(k) · logp+1 (n)).
• If log(h)/log(k) < ℓ then the time bound is
O(nℓ logp (n)).
Computational Complexity – p. 224
 Comments on Theorem 10.1
First note that for numbers which are not divisible by k, one
can make the input size a bit larger, for example adding
zero-rows and columns for matrix multiplication or leading
zeroes for integer multiplication.
Note that p = 0 is possible; in this case, the second entry
would have time bound O(nℓ log(n)) and so a logarithmic
factor comes in.
For log(h)/log(k) to be a rational number, one needs that
h = kq for some positive rational number q. Usually
irrational numbers are uprounded at the fourth or fifth digit
after the decimal dot. So if a problem is solvable in
O(nlog(3) ) then it is also solvable in time O(n1.58497 ) due to
uprounding.

Computational Complexity – p. 225

 Strassen’s Algorithm I
Another complexity value to which often is referred is the
amount of time needed to multiply two n × n matrices A, B.
The standard algorithm computes C = A × B by letting
Ci,j = k Ai,k · Bk,j . This algorithm uses time O(n3 ), as it
P
does n multiplications and additions of matrix elements to
compute each Ci,j and there are n2 values to be computed.
Theorem 10.2 [Strassen 1969] Two n × n matrices can be
multiplied with O(nlog(7) ) = O(n2.8074 ) basic operations.
For this, one divides A, B, C into two m × m blocks with
m = n/2 or m = n/2 + 1/2, the number m is an integer and
in the second case one adds a ZERO-row and -column to
A, B. Strassen schowed that then the product of A × B is a
linear combination of 7 matrix multiplications of matrices of
size m × m and one has Time(n) = 7 · Time(n/2) + O(n2 )
instead of Time(n) = 8 · Time(n/2). This recurrrence gives
the time bound O(nlog(7) ). Computational Complexity – p. 226
 Strassen’s Algorithm II
Split A, B, C into A1,1 , A1,2 , A2,1 , A2,2 , B1,1 , B1,2 , B2,1 , B2,2 ,
C1,1 , C1,2 , C2,1 , C2,2 and now let
D1 = (A1,1 + A2,2 ) × (B1,1 + B2,2 ),
D2 = (A1,1 + A2,2 ) × (B1,1 ),
D3 = (A1,1 ) × (B1,2 − B2,2 ),
D4 = (A2,2 ) × (B2,1 − B2,2 ),
D5 = (A1,1 + A1,2 ) × (B2,2 ),
D6 = (A2,1 − A1,1 ) × (B1,1 + B1,2 ),
D7 = (A1,2 − A2,2 ) × (B2,1 + B2,2 ),
C1,1 = D1 + D4 − D5 + D7 ,
C1,2 = D3 + D5 ,
C2,1 = D2 + D4 ,
C2,2 = D1 − D2 + D3 + D6 .
Multiplying in O((n/2)2.8074 ); adding, subtracting in O((n/2)2 ).
Computational Complexity – p. 227
 Strassen’s Algorithm III
The ideas of Strassen’s algorithm were brought a further
step forward by subsequent work and asymptopically better
matrix multiplications were found. Currently, Alman and
Willians hold the record with matrix multiplication in time
O(n2.37286 ) for multiplying two n × n matrices; furthermore,
for certain other rings than rationals or integers like the ring
of a finite field, even slightly better bounds are known.
Valiant [1975] showed that matrix multiplication can be used
to check membership of words in a context-free grammar
and the current bound for this problem is therefore also
O(n2.37286 ). On the other hand, Lee [2002] showed that if
one can recognise context-free grammars in time O(nc )
then one can do matrix multiplication in time O(n3−(3−c)/3 ).
However, there is still a gap between the two
bound-translations and therefore it is an open problem
whether the two complexities coincide.
Computational Complexity – p. 228
 The Karatsuba Algorithm
Theorem 10.3 [Karatsuba 1960]. Multiplication of two n bit
integers can be done in time O(nlog(3) ) = O(n1.58497 ).
Given 2n bit integers x + 2n y and v + 2n w with x, y, v, w all
having n bits, it holds that

(x+2n y)·(v+2n w) = xv+2n ((x+y)(v+w)−xv−yw)+4n yw

and one needs, beyond shifting of digits, three

multiplications vx, yw, (x + y)(v + w) of (n + 1)-bit integers.
Thus F(2n) = 3F(n + 1) + O(n) operations. Ignoring the
one bit length-increase by forming x + y, v + w, one gets
that the overall algorithm is O(nlog(3) ). With this result,
Anatoly Karatsuba disproved the conjecture mentioned by
Andrey Kolmogorov that multiplication requires Ω(n2 ) time.

Computational Complexity – p. 229

 Example 10.4
One can also split an input three-way as follows:

(x+2ny+4nz)·(u+2n v+4n w) = a+2n b+4n c+8n d+16n e,

where a = x · u,
e = z · w,
b = (x + y) · (u + v) − a − y · v,
d = (y + z) · (v + w) − e − y · v,
c = (x+y+z)·(u+v+w)−b−d+y·v.
This splitting is an easy method and it gives
F(3n) = 6 · F(n), as there are six multiplications, one for
each of a, e, three more for b, d which are (x + y) · (u + v),
(y + z) · (v + w), y · v plus the usage of a, e computed before.
The last value c uses (x + y + z) · (u + v + w) and b, d, y · v
all computed before. This gives O(nlog(6)/log(3) ) what is in
O(n1.63093 ) and worse than Karatsuba’s halving method.
Computational Complexity – p. 230
 Improvements by Andrei Toom
Multiplication with constants (independent of n) is O(n).
Note that the result of the multiplication is the value of a
polynomial at input 2n , where the unknowns a, b, c, d, e are
the coefficients of the polynomial. One can compute this
polynomial at i = −2, −1, 0, 1, 2 by evaluating
p(i) = (x + iy + i2 z) · (u + iv + i2 w) for these five values of i;
so one has five multiplications of numbers with up to n + 3
digits, the constant 3 here is ignored.
Now let V be the Vandermonde matrix with Vi,j = ij where
j = 0, 1, 2, 3, 4 (the indices of entries in V are a bit input
driven and nonstandard). Here 00 = 1. As the determinant
of V is the product of all differences between two distinct
values of i and thus nonzero, V has an inverse W which
can be precomputed (it is independent of n and all inputs)
and (a, b, c, d, e) = W · (p(−2), p(−1), p(0), p(1), p(2)). Thus
one can do with five small multiplications.
Computational Complexity – p. 231
 The Multiplication of Toom
Theorem 10.5: One can mulitply two n-bit numbers in time
O(nlog(2k−1)/log(k) ) and this can be brought as close to
O(n) as desired by choosing a big k; note that
log(2k) = log(k) + 1, thus log(2k − 1)/log(k) < 1 + 1/log(k)
and converges to 1 for k going to infinity.
Preliminaries: Multiplying with fixed rational constants, with
multiplying with 2hn for small h (bit shifting) and adding n-bit
numbers are all in O(n). Similarly the multiplication with a
fixed size matrix with fixed rational coefficients with a
constant dimension vector is O(n).
Toom’s Algorithm is a family of algorithms and once one
has fixed k, one considers it to be constant. The larger the
numbers to be handled are, the better is a big k. So from
now on let k be fixed and k ≥ 2.
Computational Complexity – p. 232
 Toom’s Algorithm for Fixed k
Algorithm 10.6: One precomputes the Vandermonde matrix
with entries ij for i = −(k − 1), . . . , (k − 1) and
j = 0, 1, . . . , 2k − 2 and also computes the inverse W which
translates the 2k − 1-vector
(p(1 − k), p(2 − k), . . . , p(k − 2), p(k − 1)) into
z0 , z1 , . . . , z2k−2 so that the result of the planned
multiplication is the sum over all 2jn · zj . The entries of W
are fixed rational numbers; here i0 = 1 for all i including
i = 0.
For multiplying two kn-bit numbers, one splits them into k
blocks x0 , x1 , . . . , xk−1 and y0 , y1 , . . . , yk−1 of length n and
P j P h
computes p(i) = ( j i · xj ) · ( h i · yh ). Then one
multiplies the result with W and obtains z0 , z1 , . . . , z2k−1 .
n
P n j
One outputs p(2 ) = j (2 ) · zj .
Computational Complexity – p. 233
 Performance and Follow-Ups
Additions and multiplications with fixed rational constants
independent of n and inputs are O(n) and additions and
multiplying with 2n (shifting of bits). Thus the complexity
depends on doing the 2k − 1 small multiplications, as those
need time Ω(n). So the time complexity of Toom’s Algorithm
satisfies the recurrence F(kn) = (2k − 1) · F(n) + O(n) and
this gives time O(nlog(2k−1)/log(k) ) for Toom’s algorithm with
parameter k.
Arnold Schönhage and Volker Strassen [1971] constructed
an integer multiplication algorithm running in time
O(n · log(n) · log log(n)). Manfred Fürer [2007] improved
∗
this bound to n · log(n) · 2 O(log (n)) where

log∗ (n) = min{h ≥ 1 : log(h) (n) ≤ 1}.

The state of the art for the complexity of multiplication is
O(n · log(n)) by David Harvey and Joris van der Hoeven [2021].
Computational Complexity – p. 234
 Nondetermistic Deciders I
A nondeterministic polynomial time decider outputs on
x ∈ L on some computation ACCEPT and on x ∈ / L on
some computation REJECT; the machine is, however,
good, that is always run within some polynomial time bound
and never outputs the wrong decision, while it might abstain
on many runs.
The nondeterministic versions NETH and NSETH of ETH
and SETH postulate lower bounds for nondeterministic
deciders.
Definition 10.7: The hypothesis NETH says that there is are
c, d > 1 such that every nondeterministic decider for 3SAT
instances needs at least time cn for infinitely many
instances where the number of clauses is bounded by d · n
for the number n of variables.

Computational Complexity – p. 235

 Nondeterministic Deciders II
Definition 10.8: The hypothesis NSETH by Marco L.
Carmosino, Jiawei Gao, Russell Impagliazzo, Ivan Mihajlin,
Ramamohan Paturi and Stefan Schneider [2016] says that
there is are no constants c < 2, d > 0 such that one can
solve all SAT instances in nondeterministic time
O(cn · (n + m)d ).
Theorem 10.9 [Carmosino, Gao, Impagliazzo, Mihajlin,
Paturi and Schneider 2016]: For any c > 1.5, 3SUM with
numbers bounded by s(n) for some polynomial s has a
nondeterministic decider running in time O(nc ).
Theorem 10.10: If 3SUM has a decider in NTIME(nc ) with
c < 1.5 then KNAPSACK has a decider in NTIME(2(c/3)n ).

Computational Complexity – p. 236

 Proof of Theorem 10.9 I
First note that for nondeterministic deciders, the difficulty is
to make sure that there is no solution when rejecting the
input, that is, saying “NO SOLUTION”. For the positive side,
the nondeterministic decider has only to guess the indices
and to add up the corresponding numbers, what is O(n). So
assume there is no solution, the following algorithm explains
how to verify this. Let Q be the set of the n input numbers
and q the target output (usually 0). Let d = (c + 3/2)/2.
Let P be the set of prime numbers between nd and nd · e,
there are approximately nd /log(n) such primes, the
constant e has to be chosen appropriately for this. Choose
that p ∈ P such that modulo p there are the least number of
triples (x, y, z) with x + y + z = q.

Computational Complexity – p. 237

 Proof of Theorem 10.9 II
For this, one uses the data base approach to make a
polynomial r(u) of degree p such that each coefficient ak of
uk is the number of numbers in Q which are modulo p
equal to k.
Polynomial multiplication satisfies, similar to Toom’s
′
3 d
algorithm, that one can compute r(u) in time O(n ) for
each given d′ > c, in particular for some d′ < c. For this
note that the coefficients are smaller than n and thus, in all
computations, the number of binary digits of the coefficients
is always logarithmic, so that handling these numbers gives
only a polylogarithmic overhead.
The overall number of solutions modulo p is in n3 /|P| and
by choice of the constant e, this is smaller than n3/2 , as
|P| ≥ nd /log(n) and thus is greater than n3/2 .
Computational Complexity – p. 238
 Proof of Theorem 10.9 III
One had chosen that the size of the numbers is bounded by
s(n), thus two distinct numbers v, w between −3s(n) and
3s(n) can coincide only modulo log(6s(n)/log(nd )) + 2
many primes in P what is O(1). Thus the average number
of solutions modulo a random prime in P is

(log(6s(n)/(nd )) + 2) · (n3 · log(n)/(nd ))

and the exact number is the sum of the coefficients of the

polynomial the polynomial r(u) at 0, p, 2p, 3p; this number is
below above average given in the displayed formula. One
guesses these triples (x, y, z) in ascending lexicographic
order and verifies that (a) x + y + z = q modulo p but (b)
x + y + z 6= q (without any modulo). If this goes through,
one goes to REJECT.
Computational Complexity – p. 239
 Discussions of Thms 10.9 & 10.10
Theorem 10.10 is the counterpart of the corresponding
result showing the 4SUM hardness provided that
KNAPSACK cannot be solved in deterministic time O(cn )
√
with c < 2. However, this assumption is quite strong; the
next result, Theorem 10.11, will have a weaker assumption
and therefore also show less.
Indeed, Carmosino, Gao, Impagliazzo, Mihajlin, Paturi and
Scheiner [2016] used their 3SUM result to show that,
assuming NSETH, one cannot reduce problems like
general SAT to show that 3SUM cannot be solved in O(nc )
for c < 2. Such a reduction would then also be usable to
show that a nondeterministic decider for 3SUM translates
into a nontrivial upper bound for nondeterministic deciders
of SAT which would disprove NSETH. Thus assuming
NSETH, one cannot construct an almost quadratic lower
bound for 3SUM using the difficulty of SAT. Computational Complexity – p. 240
 Consequences of NETH I
Theorem 10.11: Assume that NETH holds for parameter ℓ
(number of literals). Then there is no c > 0 such that for all
k the problem kSUM are in NTIME(nc ), that is, kSUM has
no uniform nondeterministic polynomial time decider
dealing with all k.
Proof: One considers solving SAT instance with ℓ literals
and let d > 1 be a constant such that one cannot solve all
SAT instances in time dℓ .
Now one adds to each clause Ci with ji literals ji − 1 new
variables only occurring there and requires that for a
solution that the number of satisfied literal for the i-th clause
is exactly ji . The overall number of literals is now bounded
by 2ℓ for the new instance. The number of variables is also
bounded by 2ℓ.

Computational Complexity – p. 241

 Consequences of NETH II
Solving the new problem is equivalent to solving the old
one, as the new version of clause Ci can have ji literals true
exactly when the old one has at least one literal true – one
makes of the new variables exactly so many true as one
needs to reach ji .
By assumption the new problem cannot be solved in time
√ 2ℓ
d . This new problem is, however, an XSAT style
problem, as every clause is true only when the number of
satisfying literals is an exact value depending on a clause.
Now one selects c, k such that there is a kSUM algorithm
running in time O(nc ) and (22ℓ/k )c < dℓ/4 ; after taking
logarithms one gets 2ℓ/k · c < log(d)ℓ/4 and by isolating k
on the right side one gets 8c/log(d) < k.

Computational Complexity – p. 242

 Consequences of NETH III
Now one has to code this into a KNAPSACK problem. For
this, one distributes the variables onto k pools with each
having at most 2ℓ/k variables. Furthermore, for each clause
having 2h − 1 literals, one reserves a block of h bits in a
binary representation of numbers coding the solvability of
the modified SAT instance. As the original clause has h
members and the sum of all clause-lengths is ℓ, it follows
that the so constructed binary number has ℓ bits and that
the number of the true literals for each clause coded with h
bits is in any assignment between 0 and 2h − 1. Note that
2h − 1 < 2h for all h, this can be verified by checking the
initial values: 2 · 1 − 1 < 21 , 2 · 2 − 1 < 22 , 2 · 3 − 1 < 23 ,
2 · 4 − 1 < 24 and so on.
Thus one gets a KNAPSACK instance (as on Slide 209)
and here the size of the numbers is 2ℓ what is (22ℓ/k )k/2 .
Computational Complexity – p. 243
 Consequences of NETH IV
In this KNAPSACK instance, one evaluates for every block
of 2ℓ/k numbers the corresponding assignments and then
puts down for each clause of 2h − 1 literals in the
corresponding h-bit slot the number of satisfied literals.
This gives 22ℓ/k many numbers for each set.
Thus for the given choice of k, when one distributes the
input of n = 22ℓ/k · k numbers on the corresponding k
subsets, then one gets an instance where each number is
binary and has ℓ bits and the problem whether one can find
k numbers, one from each set, such that there sum is equal
to q where q is the natural number obtained by setting the
value for each clause of of size 2h − 1 to h in its block of h
bits, cannot be solved by a nondeterministic decider in
nondeterministic time O(nc ). Note that the numbers are
bounded by nk , thus for the specific counterexample to the
bound c, the polynomial s(n) used in Theorem 10.9 exists.
Computational Complexity – p. 244
 Homework 10.12-10.14
Homework 10.12: Show that polynomials in one variable
over a ring can be multiplied with O(nlog(3) ) ring operations,
where n is the maximum degree of the polynomials
multiplied.
√
Homework √ 10.13: Assume that one has numbers x + dy
and v + dw represented as pairs (x, y), (v, w) of integers
where d is a small integer constant which is not a square
and it can be considered to be constant. How many integer
multiplications are needed to compute (p, q) with
√ √ √
p + dq = (x + dy) · (v + dw)?
Homework 10.14: Provide an O(n3 · log(n)) algorithm to
compute 5SUM. According to current knowledge, this
algorithm is optimal up to polylogarithmic factors.

Computational Complexity – p. 245

 Homeworks 10.15-10.17
Homework 10.15: Assume that a language L as well as its
complement are both context-free. Note that such
languages have a grammar in Greibach Normal form where
every rule is of the form A → aW where A is a nonterminal,
a a terminal and W a possibly empty word of nonterminals
(one either does not consider the empty word or needs
some exception handling). Prove that L has a
nondeterministic linear time decider which decides the
membership of words in L for all nonempty words.
Homework 10.16: Construct a nondeterministic decider for
5SUM which runs in time O(n2.56 ).
Homework 10.17: Modify Theorem 10.11 and its proof such
that it shows that, assuming ETH, there is no c such that for
every k, kSUM can be solved in time O(nc ).

Computational Complexity – p. 246

 Homeworks 10.18-10.20
Homework 10.18: Determine a time function F such that all
problems kSUM are solvable in time o(F(n)) while,
assuming ETH, 3SAT and KNAPSACK cannot be solved in
time O(F(n)). Provide this bound F(n) explicitly. Note that
here for each kSUM, the minimum n(k) from which
onwards kSUM can be solved in time F(n) or less, might
depend on k.
Homework 10.19: Assume a problem has an algorithm
which uses the recurrence Time(3n) = 8Time(n) + O(n2 ).
What is, according to Theorem 10.1, its complexity.
Upround a noninteger exponent to five digits.
Homework 10.20: Assume a problem has an algorithm
which uses the recurrence Time(3n) = 5Time(n) + O(n2 ).
What is, according to Theorem 10.1, its complexity.
Upround a noninteger exponent to five digits.
Computational Complexity – p. 247
 Homework 10.21 and 10.22
Use Theorem 10.1 for the following two homeworks. Here
one wants to compute a function value f (x) with x ∈ {0, 1}n .
Homework 10.21: Assume a problem has an algorithm
which computes f (x) by either nondeterministically
switching to a O(n3 ) subroutine or by doing 17 subroutines
of size n/4 of itself with an additional processing of O(n2 ).
Can the nondeterministic switching be eliminated by
hardwiring one choice? What is the complexity then?
Homework 10.22: Assume a problem has an algorithm
which computes f (x) by either nondeterministically
choosing between 25 self-calls with data of size n/3 or 60
self-calls of size n/4. In both cases, there is additional
runtime of O(n5/2 ) in the local routine. Can the
nondeterministic choice be eliminated by hardwiring one of
the choices? If yes, which one is better and what is the
overall complexity (uprounded to five digits)?
Computational Complexity – p. 248
 Lecture 11
This lecture deals with the Polynomial hierarchy (PH) and
its relation to PSPACE. It will furthermore introduce and
investigate the role of oracles in computing. This oracles
will in particular be considered for the notions of P, NP,
CoNP, PH and PSPACE. Relationsships between relatived
worlds will be studied.
Furthermore, some concrete problems other than quantified
Boolean formulas will be exhibited for the lower levels of
PH. They are formulated as integer expression problems
where an integer expression is either given as a finite set or
the sum of two expressions or the union of the two sets
given by expressions. In particular inclusion and equality to
an interval of integer expressions are key problems placed
into PH.

Computational Complexity – p. 249

 Computations Relative Oracles
Definition 11.1: A Turing machine can use an oracle A as
follows: It has an oracle tape which is an additional tape
governed by the same size constraints as applies to the
work memory of the machine and the machine has an
oracle-evaluation command with the following constraints:
• Command is of form “If word in Oracle Then Goto X
Else Goto Y” where X and Y are line numbers;
• The command checks whether the word on the oracle
tape is in A;
• If it is in A then the machine erases the content of the
oracle tape, scrolls to its beginning and goes to Line X;
• If it is not in A then the machine erases the content of
the oracle tape, scrolls to its beginning and goes to Line Y.
All other commands are executed as usual and above
conditional jumps are the only interaction with A.
Computational Complexity – p. 250
 The Polynomial Hierarchy
Definition 11.2: One defines inductively the complexity
classes ΣP k , Π P as follows: ΣP is NP and ΠP is CoNP.
k 1 1

ΣPk+1 is NP[Σ P ], that is, NP with an oracle which is a ΣP

k k
complete set. Furthermore, ΠP k is the class of all languages
whose complement is in ΣP k.

Example 11.3: The class ΣP 2 is equal to NP[NP], more

precisely, to the class NP[SAT] where SAT is the
satisfiability problem and might be replaced by any other
NP-complete problem.

Computational Complexity – p. 251

 Alternating Computations
An alternating computation can be viewed as a game
between two players Anke and Boris. One player wants to
get a word accpeted, the other wants to get it rejected.
A language L is in NP if there is a nondeterministic Turing
machine recognising it such that the player Boris has a
possibility to make the nondeterministic choices such that
the word gets accepted.
A one-time alternation is if during the game, there might
arise the situation that from now on the choices are made
by the other player, say Anke. So first it are Boris moves,
but during the game it can happen that the rules say from
now on Anke moves. If Boris can move such that he wins,
independent of what Anke does later, the word is in L;
otherwise it is in the complement of L.

Computational Complexity – p. 252

 One-Time Alternation I
Theorem 11.4: A language L can be computed by a
one-time alternating computation if it is in NP[NP].
Proof one-time alternation ⇒ NP[NP]: If one has one-time
alternation then one can consider subprograms given by a
configuration which is an alternation-point and the question
whether the second player can continue such that the input
will be rejected.
Thus x ∈ L iff there is a nondeterministic polynomial time
computation leading to such a switching point where the
oracle says that the configuration does not lead to a
rejecting decision, whatever the other player does. Thus
one can nondeterministically guess the first computation
and then check with an NP-complete oracle whether after
the switch the other player can force a rejection.

Computational Complexity – p. 253

 One-Time Alternation II
Proof NP[NP] ⇒ one-time alternation: Assume that a
nondetermistic machine with NP-oracle recognises L. Then
one can guess a computation and for all positive answers to
an NP-complete oracle, one also guesses the witness.
Thus one can guess a nondeterministic computation ending
in a condition that all negatively answered queries are
correct. These can be checked by an alternation which then
requires that the other player continues to play and if the
opposing player cannot disprove that any of the negative
NP-answers is incorrect, then the overall computation is
correct. Thus one can transform every computation in
NP[NP] into a one-time alternating computation, where first
Boris makes the decision and, after the alternation point
Anke checks whether she can disprove any of the negative
answers of the NP-oracle. If so, she rejects and provides
the witness.
Computational Complexity – p. 254
 Generalising the Result
Theorem 11.5: A language L is in ΣP k+1 iff there is a k-time
alternating computation starting with player Boris such that
x ∈ L iff Boris can enforce that the computation accepts.
A language L is in ΠP
k+1 iff there is a k-time alternating
computation starting with player Anke such that x ∈ L iff
Anke can enforce that the computation rejects.
Recall that k-time alternating computations are
nondeterministic computations controlled alternately by
players Anke and Boris such that computations end up in
an explicit ACCEPT or REJECT decision. There are at most
k points in the computation where the control alternates
from one player to the other. A word is in L iff player Boris
can enforce that the computation accepts; a word is outside
L iff player Anke can enforce that the computation rejects.
Computational Complexity – p. 255
 Alternation and Quantifiers
One can evaluate languages in NP[NP] (one-time
alternating computation) using quantifiers: if x ∈ L then
Boris can choose the decisions until the alternation point
such that for all subsequent choices by Anke the
computation still ends up in an accepting state. If x ∈
/L
then independently of what Boris did first, Anke can when it
becomes her turn enforce that the computation ends up in
an rejecting state.
In other words: x ∈ L iff there are decisions of Boris such
that the computation either accepts or goes to an
alternating point and if the latter happens, for all
subsequent decisions of Anke, the outcome is still accept.
In a formula: ∃ decisions by Boris ∀ decisions of Anke [the
outcome is accept]. The decisions here are limited in
number by a polynomial for each player and can be written
as a polynomially sized ∃∀ formula.
Computational Complexity – p. 256
 Integer Expressions
Definition 11.6: Additive expression for integers are either
finite sets of natural numbers given as an explicit list of
binary integers or sums of expressions defined with
L + H = {x + y : x ∈ L, y ∈ H} or unions of expressions.
The length of an expression is the number of symbols
needed to write it down.
Example 11.7: The integer expression ({0, 1} + {0, 2}) ∪
({4, 8} + {0, 8}) generates the set {0, 1, 2, 3, 4, 8, 12, 16}.
Every finite set has an integer expression listing its
elements in order; however, there are often more compact
way of writings expressions.
Example 11.8: The set {0, 1, 2, 3, 4, 5, 6, 7} can be written
as {0, 1} + {0, 2} + {0, 4} using six instead of eight
numerical constants. In general, for intervals of length n
one needs only an expression with O(log(n)) numbers.
Computational Complexity – p. 257
 Complete Problems in PH I
Each level of PH has complete problems; however, if one
problem is complete for the whole PH, PH has only finitely
many distinct levels.
Definition 11.9: The problem
∃x1 , x2 , . . . , xn ∀y1 , y2 , . . . , yn ∃z1 , z2 , . . . , zn [φ(x1 , . . . , zn )]
where φ is a Boolean formula and all variables are Boolean
is ΣP3 complete and this method relativises to all levels of
PH.
Proposition 11.10: It is an NP-complete problem to decide
whether a given binary number is a member of a given
additive integer expression.
For this, one translates KNAPSACK instances into additive
expressions. That is, a subset of {x1 , x2 , . . . , xn } can add to
y iff y ∈ {0, x1 } + {0, x2 } + . . . + {0, xn }.
Computational Complexity – p. 258
 Complete Problems in PH II
Theorem 11.11 [Wagner 1987]: The problem whether an
additive integer expression L is a not necessarily proper
subset of a further expression H is Π02 -complete.
The condition is in Π02 as one can for each number formable
by selecting members from possible choices of
number-sets in subexpressions of L find a nondetermistic
choice how to form the same number in H. So one has to
show completeness.
For completeness one starts with a Boolean formula where
variables are first universally and then existentially
quantified. The intermediate values in the formula can also
all be replaced by existentially quantified variables which
takes as values these intermediate values. The variables in
the formula are furthermore connected by assignments to
the or of two variables or to the not of one variable.
Computational Complexity – p. 259
 Complete Problems in PH III
Now consider a conditions of the type

z=x∨y

and one represents them by the following equations –

including a further auxiliary variables u to make it possible
that the sum gets the value 2:

x + y + u + 2 · ¬z = 2.

If x + y ≥ 1, then z must be 1 and u = 2 − x − y; if x + y = 0

then z, u must both be 0. For the k-th such equation, one
reserves the k-th decimal digit from below in the coding.

Computational Complexity – p. 260

 Complete Problems in PH IV
The following example gives how the numbers are coded in:

z = x ∨ y, z′ = ¬x ∨ y, z′′ = ¬x ∨ ¬z.

The corresponding numerical equations are these:

x+y+u+2¬z = 2, ¬x+y+u′ +2¬z′ = 2, ¬x+¬z+u′′ +2¬z′′ = 2.

Now one makes for each variable, say x, two numbers, one
representing the literals x, ¬x as follows:
xpos = 110, xneg = 001. For the case of z, one must take
into account the scaling factor 2 when the variable stands
on the other side of the original equation:
zpos = 000, zneg = 201. As z does not occur in the middle
equation, both zpos , zneg have the middle digit 0.

Computational Complexity – p. 261

 Complete Problems in PH V
The last step is encoding that some variables are chosen
by a second expression. The way to do this is to introduce
additional digits in the number and to code the value into
them as 1 or 2. So when x, y are universally quantified, one
would xpos , xneg by 10110, 20001 on the right side
expression and similarly for y. The left side expression is
then this: {00222} + {10000, 20000} + {01000, 02000}. So
the first two decimal digits code whether x, y have to be
positive or negative and the other three ensure that the
value of the sum is the intended target 2. Thus L ⊆ H holds
iff for every possible choice of x, y on the left side the right
side can be made to sum up to to 222 on the three lower
digits when the same choice of x, y is taken.
This technique allows to code initial universal quantification
into integer expression comparison.
Computational Complexity – p. 262
 Complete Problems in PH VI
Theorem 11.12 [Wagner 1987]: The following problem is
ΣP3 -complete: Given an integer expression H and a number
y, is there an x with {x + 1, x + 2, . . . , x + y} ⊆ H.
Note that here y is given as a binary or decimal number, not
as a unary number.
Remark 11.13: In Proposition 11.10 and Theorems 11.11
the expressions can be formed from finite sets with adding
expressions only, where in Theorem 11.9, one quantifies
over the x existentially and thus goes up from ΠP2 to Σ P . If
3
y = h + 23 with 0 < h ≤ 23 , then one can write the
expression L on the left side as

L = {x + 1} + {0, 1} + {0, 2} + {0, 4} + {0, h}.

This holds for general 2ℓ in place of 23 .

Computational Complexity – p. 263
 PH and Randomness I
Definition 11.14: A language L is in BPP iff there exists a
polynomial time algorithm F which uses random-bit
sequences b = b0 b1 . . . bp(n) such that, for all x and for 2/3
of these random-bit sequences b, F(x, b) = L(x). That is, F
outputs with probability 2/3 the right bit L(x). Here L(x) = 1
for x ∈ L and L(x) = 0 for x ∈/ L.
Remark 11.15: One can increase the threshold to any
constant. The idea is to run the algorithm c times with
random bits; then one takes the majority of the outcomes.
For example, the majority of three outcomes is correct with
probability (8 + 3 · 4)/27 = 20/27 and false with probability
(1 + 3 · 2)/27 = 7/27.
Proposition 11.16: BPP is closed under Boolean operations
(union, intersection, complement).
Computational Complexity – p. 264
 PH and Randomness II
Theorem 11.17 [Lautemann 1983; Gacs (Sipser) 1983]:
BPP is a subclass of ΣP
2 , that is, of NP[NP].
Algorithm: Meyer and Stockmeyer [1975] showed that
L ∈ ΣP 2 iff there is a set H in P and there are polynomials
r, s such that for all words x,
x ∈ L ⇔ ∃y ∈ {0, 1}r(|x|) ∀z ∈ {0, 1}s(|x|) [(x, y, z) ∈ K].
By Remark 11.15, one can assume that the probability of a
false decision is strictly below 2−n ; let s(n) be the uniform
bound on the number of random bits used for the overall
process. Let k(n) be the uprounded value of 2s(n)/n and
r(n) = k(n) · s(n). Let y(n) be a vector of k(n) strings
y1 , . . . , yk(n) of length s(n) and z be just vector of s(n) bits.
Now (x, y, z) ∈ K iff L(x, yh + z) accepts for some
h ∈ {1, . . . , k(n)} where yh + z is the bitwise exclusive or
and L(x, u) is the BPP-algorithm on x with random-bits u. Computational Complexity – p. 265
 PH and Randomness III
Lautemann [1983] used now calculations to show the
following:
If x ∈ L then there are y1 , . . . , yn such that for each z there
is a h such that L(x, yh + z) accepts. If one selects
y1 , . . . , yk(n) at random, then the probabity that a z makes
all of L(yh + z) to reject is at most 2−n·k(n) which is below
2−s(n) , so one might expect that no string rejects all
combinations. Thus a random choice of the y1 , . . . , yk(n)
would make (x, y, z) ∈ K for all z. Lautemann did some
careful calculations in order to replace the probability
arguement by a counting arguement.

Computational Complexity – p. 266

 PH and Randomness IV
If x ∈
/ L then one has to show that for all y1 , . . . , yn there is
a z such that for all h L(x, yh + z) rejects.
Here now the probability that L(x, yh + z) accepts is below
2−n by assumption on the machine. The union probability
such that all L(x, yh + z) accepts for at least one h is below
k(n) · 2−n . As k(n) is a polynomial, this probability is below
1/2 for all sufficiently long inputs x and for no sufficiently
long input x ∈/ L and all y1 , . . . , yk(n) there is a z with
L(x, yh + z) rejecting for all h.
For the small x one can patch K to hardwire that
K(x, y, z) = L(x). Then the formula
x ∈ L ⇔ ∃y ∈ {0, 1}r(n) ∀z ∈ {0, 1}s(n) [K(x, y, z)]
holds for all x and L ∈ ΣP
2.

As BPP is closed under complement, L ∈ ΠP

2 , too.
Computational Complexity – p. 267
 When P[A] = PSPACE[A] I
Theorem 11.18 [Baker, Gill and Solovay]: There is an
oracle A with P[A] = NP[A] = PSPACE[A].
Proof: Let ϕ0 , ϕ1 , . . . be an enumeration of all PSPACE
computable functions with pe be the polynomial bound on
the space used and ke be the number of tape symbols of
the work alphabet of the Turing machine ϕe . The set
A = {1d 0kd ·pd (|x|) 1xb : ϕd (x) = b} is a PSPACE-complete
set. First one shows that P[A] contains all of PSPACE.
This is done by computing, for fixed e, for each input x the
string 1e 0ke ·pe (|x|) xb with b = 0, 1 and then asking which of
these strings is in A. That b where the answer is 1 is then
output.
This algorithm is in polynomial time relative to A, however,
the larger the space bound of ϕe is, the longer the algorithm
takes to write the oracle query.
Computational Complexity – p. 268
 When P[A] = PSPACE[A] II
Furthermore, one can see that A is in LINSPACE: On input
1e 0ke ·pe (|x|) 1xb, the machine can use the space of size of
the input to simulate ϕe , without loss of generality, this
Turing machine does not have more than e states. Thus
one can use the space of size e to store the Turing machine
states and other things; the space of size ke · pe (|x|) can be
used to simulate the content on the Turing machine tape of
the machine ϕe . So A is in LINSPACE. It follows that
PSPACE[A] ⊆ PSPACE and therefore P[A] = PSPACE[A].
As P[A] ⊆ NP[A] ⊆ PSPACE[A], all three are equal.

Computational Complexity – p. 269

 When P[B] and NP[B] Differ
The following gives the summary of further oracle results.
Theorem 11.19 [Stockmeyer 1976]: If ΣP
k [A] = Π P [A] then
k
PH[A] = ΣP
k [A].
Theorem 11.20 [Baker, Gill and Solovay 1975]: There is an
oracle B such that NP[B] 6= CoNP[B] and thus
NP[B] 6= P[B].
Theorem 11.21 [Heller 1984]: There is an oracle C with
ΣP2 [C] 6
= Π P [C], thus P[C], NP[C], NP[NP[C]] are all
2
different.
Theorem 11.22 [Yao 1985; Hastad 1986]: There is an
oracle D relative to which all levels of the polynomial
hierarchy are different. For each such oracle D,
PH[D] ⊂ PSPACE[D].
Computational Complexity – p. 270
 PH and PP Relativised
Repetition: Recall that PP just says that there is a
probabilistic Turing machine which decides L in the
following way: For each x ∈ L the computation accepts with
at least probability greater than 1/2 and for each x ∈
/ L it
accepts with probability strictly smaller than 1/2. Here the
computations use p(n) random bits and steps on an input
of size n and one out of 2p(n) computations abstains from a
decision to avoid a tie (the remaining number is odd).
Theorem 11.23 [Toda 1991]: Let A ∈ PH and B ∈ PP[A].
Then there is a C ∈ PP such that B ∈ P[C].
In particular the class PP is not a subclass of the
polynomial hierarchy unless the latter collapses to a finite
level. Furthermore, B ∈ P[C] is also known as “B is
polynomial time Turing reducible to C.”
Computational Complexity – p. 271
 Homeworks 11.24-11.29
There are two ways of measuring the size of an integer
expression: (a) by the number of symbols used to write it
down; (b) by the number of numerical parameters (binary
numbers) occurring in the expression when writing it down.
This homework asks for finding integer expressions as
small as possible with respect to (b) for the following sets
U, V, W, X, Y, Z:
Homework 11.24: U = {1, 2, 3, 8, 9, 10, 27, 28, 29, 30}.
Homework 11.25: V = {8, 9, 16, 17, 32, 33}.
Homework 11.26: W = {0, 1, 2, 3, 4, 5, 6, 7}.
Homework 11.27: X = {2, 3, 4, 5, 6, 8, 9, 10, 12, 16}.
Homework 11.28: Y = {0, 3, 6, 9, 12, 15, 18, 21, 31, 34}.
Homework 11.29: Z = {0, 1, 2, 3, 27, 32}.
Computational Complexity – p. 272
 Homework 11.30
Homework 11.30: Assume that L is a problem in BPP such
that the BPP algorithm makes on inputs of length n an error
with probability at most 2−2n−25 with s(n) random bits.
A company wants to make a decider for this problem which
uses as few random bits as possible. The idea is to use a
data base and whenever a word x of length n comes up for
the first time (no word of length n before) then the machine
draws s(n) random bits and writes the random string as un
into the data base. All subsequent words x′ of length n will
output L(x′ , un ), that is, the value of the decider with the
random string un .
Calculate the overall probability that this algorithm ever
misclassifies L on any input.

Computational Complexity – p. 273

 Homeworks 11.31 and 11.32
Homework 11.31: Assume that L has a decider M which
uses r(n) random bits in order to decide every input with
error probability bounded by 1/4. Provide a polynomial k(n)
such that, when repeating the computation k(n) times on
inputs of length n, the error probability is at most 2−2n−25 ;
the method then uses k(n) · r(n) random bits on length n.
Homework 11.32: Assume that the algorithm from
Homework 11.30 is fed with inputs in length-lexicographic
order. Is there an input x such that when having processed
ε, 0, 1, 00, 01, 10, 11, 000, . . . , x the amount of random bits
used so far is on average below 0.00001 bits per data item?

Computational Complexity – p. 274

 Lecture 12
This lecture will be about PSPACE and the classes beyond
including undecidable problems.
PSPACE equals alternating polynomial time. Furthermore,
by Savitch’s theorem, a language L in NSPACE(p(n)) for
some polynomial p is also in SPACE(p(n) · p(n)) and thus
in PSAPCE, so NSPACE = PSPACE.
PSPACE has a further characterisation, namely by
interactive proof systems.
Many complexity classes can be characterised by their
complete sets as the class of all problems LOGSPACE
reducible to such a complete set.

Computational Complexity – p. 275

 PSPACE-Complete Problems I
The language taken as oracle A in Theorem 11.11 is a
PSPACE-complete set. This completeness is obtained by
coding. There are more natural problems.
Example 12.1: The set of all optimal moves for all n × n
Reversi situations is a PSPACE-complete set.

. . . . . O O O . O O O
. X X . . O X . . O X .
, ⇒
. O O . . O X . . X X .
. . . . . . . X X . . X

The starting position and sample move in 4 × 4 Reversi.

The game starts in the centre. A player can move if he can
play his new piece on an empty field (.) such that some
pieces of the opponent are between the new and some old
piece vertically, horizontally or diagonally which are turned.
Computational Complexity – p. 276
 PSPACE-Complete Problems II
Reversi players move alternately and pass only if they
cannot move; the game ends when both players are unable
to move. The aim of Reversi is to have at the end more
(and if possible much more) pieces than the opponent.
Theorem 12.2: Some two-player board games have an
optimal strategy which is in PSPACE. Some games like
Reversi satisfy that their strategy is PSPACE complete.
Theorem 12.3 [Chandra, Kozen and Stockmeyer 1981]:
Everything what can be computed in Alternating Polynomial
Time (APOLYTIME or AP) is in PSPACE and vice versa.
Theorem 12.4: Let φ be a Boolean formula, for example a
3CNF formula. Now evaluating formulas of the type
∃x1 ∀x2 ∃x3 ∀x4 . . . ∀x2n [φ(x1 , x2 , . . . , x2n )] is PSPACE
complete. Note the main difference to 3SAT is that one
uses alternating quantifiers instead of existential ones.
Computational Complexity – p. 277
 PSPACE-Complete Problems III
The formula from Theorem 12.4 can be evaluated in
PSPACE by recursive calls. For this one always takes the
first quantified variable in a formula and replaces it by 0 and
1, respectively:
Algorithm Decide(α):
If α = ∃x [β(x)] then Return Decide(β(0)) ∨ Decide(β(1)).
If α = ∀x [β(x)] then Return Decide(β(0)) ∧ Decide(β(1)).
Otherwise β consists of clauses where all literals are
replaced by either 0 or 1. If each clause contains a literal
evaluating to 1 then return 1 else return 0.
The local memory of each call is linear (size of calling
formula plus local variables) and the depth of the calls is
also linear, thus the whole formula can be evaluated in
polynomial space which is sufficient to store all the stacks
and local memories involved with the recursive calls.
Computational Complexity – p. 278
 PSPACE-Complete Problems IV
Assume that L ∈ PSPACE via an algorithm using p(n)
space which uses 2q(n) steps at most for polynomials p, q.
Note that q(n) ∈ O(p(n)) as it only must take care of the
number of different ways to write the tape on the p(n)
positions and to choose the Turing state. Now to verify that
a computation verifying that L(x) = b starting with a start
configuration encoding x, b and taking at most 2q(n) steps
until a unique configuration for halting and accepting one
does the calls the procedure on the next slide with
parameter k = 0, 1, . . . , q(n).
Each round will consist of Anke putting configurations α, β
and then Boris selecting the middle γ and then the round
will either conclude with checking or the game will go into
the next round. The overall number of rounds is q(n) + 1.

Computational Complexity – p. 279

 PSPACE-Complete Problems V
Round k: Given α, β by initial call or choice of Anke in last
round.
If q(n) − k < 1 then one verifies that that one can go in one
step or less from α to β . If so or if player Anke did not stick
to the rules during the game (that is, did in some round not
take α, β from the two choices of the previous round where
applicable) then one accepts else one rejects.
If q(n) − k ≥ 1 then first player Boris guesses a middle
configuration γ .
Second player Anke selects

(α′ , β ′ ) ∈ {(α, γ), (γ, β)}

and passes it to round k + 1 for the new (α, β) there.

Computational Complexity – p. 280
 PSPACE-Complete Problems VI
If the PSPACE-computation goes to accept, then Player
Boris will always select the configuration γ in the middle of
the two configurations α, β of the computation. It is an
invariant that the steps to computation to be covered per
round is 2q(n)−k . Thus, when q(n) − k = 0 then at most one
steps from α to β what is then checked easily.
If the computation starting with α does not lead to the
accepting and halting configuration β within 2q(n) steps then
each selection of Boris will have that either one cannot go
from α to γ in 2q(n)−1 steps or one cannot go from γ to β in
2q(n)−1 steps. By prudent choices, Anke passes this
nonreachability on from round to round until in the last
round, one cannot go from α to β in at most one step which
can be checked easily.
Computational Complexity – p. 281
 PSPACE-Complete Problems VII
The overall space usage of this is 4 · p(n) · (q(n) + 1)
symbols as each local round just stores the local α, β, γ plus
some small extra space for n, k and other ingredients. Thus
the local memory can be translated into s(n) bits for some
polynomial s(n) where Boris selects γ and Anke selects
α′ , β ′ for the next rounds. One quantifies the bits selected
by Anke universally and those selected by Boris
existentially and furthermore evaluates the whole thing with
a large formula to make sure that Anke really takes the
border configurations from the previous two choices and
that in the last step the computations from α to β can be
done in at most one computation step.
Some quantified Boolean formula encodes these conditions
and witnesses PSPACE-hardness of the formula in
Theorem 12.4.
Computational Complexity – p. 282
 Interactive Proofs
Description 12.5: An interactive proof system is given by a
pair (Prover, Verifier) where the prover has no restrictions in
computation time and space and the verifier works in
probabilistic polynomial time. Prover and verifier
communicate by messages. They work on a language L as
follows: For input x the prover proposes a value b for L(x)
and supplies in messages the verifier with proof ideas and
answers to the verifiers queries. If the verifier accepts
correct suggestions and proofs and rejects incorrect
suggestions and proofs both with probability at least 2/3
then L is in IP.
Theorem 12.6 [Adi Shamir 1992]: A language L is in
PSPACE iff it is in IP.
Theorem 12.7 [Jain, Ji, Upadhyay and Watrous 2009]: Even
if one uses a Quantum Computer for the verifier, the
complexity class equal to IP is still PSPACE.
Computational Complexity – p. 283
 12.8 The Elementary Hierarchy
Let F0 (n) = Poly(n) be the collection of all polynomially
bounded functions in one input n and let Fk+1 (n) = 2Fk (n) .
Definition 12.8: The elementary hierarchy is given as
P ⊆ PSPACE ⊆ TIME(2Poly(n) ) ⊆ SPACE(2Poly(n) ) ⊆
Poly(n) Poly(n)
TIME(2 2 ) ⊆ SPACE(2 2 ) ⊆ . . . where the levels
2k, 2k + 1 can be defined using Fk as TIME(Fk (n)) and
SPACE(Fk (n)).
Whether the inclusion between neighbouring levels is
proper, one does not know, but the level k + 2 is always a
proper superclass of the level k and no set inside level k
can be complete for level k + 2 or higher. The level 2 is
called EXPTIME and the level 3 is called EXPSPACE, the
level 4 is called DOUBLEEXPTIME and the level 5 is called
DOUBLEEXPSPACE. Level k + 1 can also be obtained by
using alternating computations from the level k.
Computational Complexity – p. 284
 Elementary Problems
Example 12.9: Presburger Arithmetic. Set of all true
quantified formulas over integers using +, −, <, =, >,
Boolean connectives, integer constants and variables.
Situated between DOUBLEEXPTIME and
DOUBLEEXPSPACE. A formula in Presburger Arithmetic is
∀x ∃y [x = y + y ∨ x = y + y + 1] (“every integer is even or
odd”).
Example 12.10: Checking whether a position in n × n
checkers is winning, losing or draw is EXPTIME-complete.
Example 12.11: Computing the complement of a regular
expression is DOUBLEEXPTIME-hard. This is due to the
size-increase in the formula constructed, just writing it down
takes this time in the worst case.
Example 12.12: Given a deterministic Turing machine M,
an input x and a binary number y, to check whether M(x)
halts within y steps is EXPTIME-complete.
Computational Complexity – p. 285
 The Exponential Hierarchies
The exponential hierarchy differs from the elementary
hierarchy by using oracles from the Polynomial Hierarchy
and allowing exponential time computations relative these
oracles. Here one makes distinctions between allowing
E-time or EXP-time where E-time means the runtime is of
the form 2O(n) while EXP-time means that the runtime is of
the form 2Poly(n) . E-time classes are not closed under
LOGSPACE reductions, as those can increase the size
polynomially; thus one often considers EXP-time classes.
The E-hierarchy is ETIME(P), ETIME(NP), ETIME(NP[NP]),
. . . and the k-th level allows ETIME computation relative to
an oracle of the k-th level of the polynomial hierarcy.
Similarly one defines the EXPTIME-hierarchy as
EXPTIME(P), EXPTIME(NP), EXPTIME(NP[NP]), . . . and
the properties of both are different; both hierarchies
collapse when PH collapses. Computational Complexity – p. 286
The LOOP Programming Language
Definition 12.13: A LOOP program uses registers as an
addition machine and has the following primitive operations:
Set to 0, Increment, Copy value from one variable to
another; Loop x Do . . . End. The data type of LOOP
programs are natural numbers including zero.
Note that when the Loop command reads out the x before
going into the loop and does the number of iterations given
by the entrance value of x; subsequent changes to x are
ignored.
With some tricks one can make operations like “y = 1; Loop
x Do Begin y = 0 End.” This assigns to y the value 1 iff x
has the value 0. This can be generalised to subtraction
which is not predefined and must be defined using loops.

Computational Complexity – p. 287

 Example
While the LOOP programs for adding and multiplying are
standard, those for subtraction are more tricky.
Computing x0 = max{0, x1 − 1}.
x 3 = 0; x 0 = x 3 ;
Loop x1 Do Begin
x0 = x3 ;
x3 = x3 + 1 End.
Computing x0 = max{0, x1 − x2 }.
x4 = x1 ; x0 = x4 ;
Loop x2 Do Begin
x 3 = 0;
Loop x4 Do Begin
x4 = x3 ;
x3 = x3 + 1 End;
x0 = x4 End.
Computational Complexity – p. 288
 The Grzegorczyk Hierarchy
Description 12.14: The Grzegorczyk Hierarchy is the
hierarchy of functions of the natural numbers to natural
numbers which can be computed by LOOP programs and
those which can be written by LOOP programs with k-fold
nesting of loops form the hierarchy level Ek . The Level 0
cannot be explained with LOOP programs without any
Loop-command and the Level 1 are those functions which
add some inputs; the level E2 include also functions like
multiplication. The level E3 includes exponential functions
x
2
and iterated exponential functions like x 7→ 2 and is called
the level of the elementary functions.
A function from natural numbers to natural numbers is
called primitive recursive iff it can be written with LOOP
programs, that is, is on some level of the Grzegorczyk
Hierarchy.
Computational Complexity – p. 289
 General Programming Paradigm
Functions which can be described by a computer program,
for example that of an Addition Machine or Turing Machine,
are called “partial-recursive functions”; the add-on “partial”
means that it might happen that on some input x, the
computer program runs forever and therefore the function
(the program’s output) is undefined. All primitive recursive
functions are also partial-recursive, but not vice versa, as all
primitive recursive functions are total, that is, everywhere
defined. There are also some total functions which are not
primitive recursive, for example the Ackermann function:

y+1 if x = 0;
(
Ack(x, y) = Ack(x − 1, 1) if x > 0 ∧ y = 0;
Ack(x − 1, Ack(x, y − 1)) if x > 0 ∧ y > 0.

A numbering of all primitive recursive functions with one

input variable is another such example. Computational Complexity – p. 290
 Halting Problem
Fact 12.15: There is a computer program u with two inputs
e, x such that for each one-input program p there is an
index e so that u(e, x) = p(x) for all x (with u(e, x) being
undefined when p(x) is undefined).
Theorem 12.16: There is no computer program u′ such that
for all e, x, u′ (e, x) is defined and for all total p there is an e
with u′ (e, x) = p(x).
Assume that u′ exists and then consider p computing
p(x) = u′ (x, x) + 1.
Corollary 12.17: A two-input computer program is either not
total or it cannot capture (as rows) all total one-input
computer programs. In particular, the question whether u
above halts (= is defined) on inputs e, x cannot be decided
by a total computer program.
Computational Complexity – p. 291
 Halting Problem and Polynomials
Definition 12.18: (a) Recursive Enumerable: If an algorithm
can enumerate (in whatever speed and order) the members
of a set, the set is called recursive enumerable. The halting
problem is an example.
(b) Diophantine Sets: A set L of integers is Diophantine iff
there is a polynomial p in k + 1 variables for some k such
that x ∈ L ⇔ ∃y1 , . . . , yk [p(x, y1 , . . . , yk ) = 0].
Example 12.19: The set {x : ∃y, z > 2 [y2 − x · z2 = 1]} is
Diophantine. A Diophantine is recursive enumerable since
one can go through all k + 1-tuples of (x, y1 , . . . , yk ) and
whenver the polynomial evaluates to 0 on this tuple, one
enumerates x.
Theorem 12.20 [Yuri Matiyasevich 1972]: Every recursively
enumerable set of integers is Diophantine.
Computational Complexity – p. 292
 The Arithmetical Hierarchy
One can investigates quantifier hierarchies on polynomial
expressions. Here bounded quantifiers with finite range are
ignored. So one has the following for sets of numbers as
well as for sets of tuples of numbers:
• A set is Σ0 if it can be defined with bounded quantifiers
only and if it is defined by an expression using +, −, ·,
=, <, > and access to variables and number constants.
Such sets coincide with the Π0 sets.
• A set A is Σk+1 iff there is a Πk set B with h + h′
variables such that (x1 , . . . , xh ) ∈ A if and only if
∃y1 . . . ∃yh′ [(x1 , . . . , xh , y1 , . . . , yh′ ) ∈ B].
• A set A is Πk+1 iff there is a Σk set B with h + h′
variables such that (x1 , . . . , xh ) ∈ A if and only if
∀y1 . . . ∀yh′ [(x1 , . . . , xh , y1 , . . . , yh′ ) ∈ B].
Diophantine sets (= recursively enumerable sets) are Σ1 .
Computational Complexity – p. 293
 Usage of Oracles I
Theorem 12.21: A set A is in Σk+1 if and only if it is
recursively enumerable relative to a Σk set B. A set A is in
Πk+1 if and only if its complement is recursively
enumerable relative to a Σk set B.
Example 12.22: The following sets are in Σ0 :
(a) {(x, y) : x2 − 2y2 = 1} – Equation set given by a
two-variable polynomial.
(b) {(x, y) : 0 ≤ x ∧ x2 < y ∧ y < x2 + 2x + 2} – Pairs where
x is the largest nonnegative integer strictly below the
square-root of y.
(c) {x : ∃v ≤ x ∃w ≤ x [x = v2 + w2 ]} – Numbers which are
the sum of two integer squares.
Example 12.23: The following sets are Σ1 :
(a): Halting problem; (b): Range of some recursive function;
(c): All Σ0 sets including ∅; (d): All Diophantine Sets.
Computational Complexity – p. 294
 Usage of Oracles II
Example 12.24: The following sets are Σ2 :
(a): All sets enumerated relative the halting problem as an
oracle;
(b): Set of all computer programs which compute a function
with a finite range;
(c): Set of all computer programs which work only on finitely
many inputs;
(d): Set of all computer programs which do not halt on
some input.
Example 12.25: The following sets are Π1 :
(a): Set of all computer programs which do not halt on input 0;
(b): Complement of a recursively enumerable set;
(c): Set of all computer programs terminating after x2 + 1
steps latest on each input x.

Computational Complexity – p. 295

 Blum Complexity Measures
Definition 12.26 [Manuel Blum 1967]: A pair (ϕ, Φ) is a
Blum complexity measure where ϕ0 , ϕ1 , . . . is a numbering
of all partial-recursive functions and Φe (x) is defined iff
ϕe (x) outputs some value and the set of triples
{(e, x, t) : Φe (x) = t} is decidable, that is, satisfies that
some computer program can check whether Φe (x) = t.
A complexity class C is given by a family f0 , f1 , . . . of total
recursive functions such that for every g ∈ C there there are
indices d, e such that g = ϕd and, for all x, Φd (x) ≤ fe (x).
Example 12:27: The complexity class of all polynomial time
computable functions is given by the family of all polynomials
and Φe (x) being the computation time which ϕe uses to
compute ϕe (x). If Φe (x) is the computation space, then one
obtains the class of all PSPACE-computable functions. The
additional requirement that the functions g are {0, 1}-valued
leads to the classes P (for time) and PSPACE (for space).
Computational Complexity – p. 296
 Properties of Complexity Measures
Theorem 12.28 [Gap Theorem by Trakhtenbrot 1967 and
Borodin 1972]: Given a recursive function g with g(n) ≥ n
for all n, there is a Time bound t such that, for all n,
DTIME(t(n)) = DTIME(g(t(n))). So the extra time
computed by g does not help. This goes also with abstract
Blum complexity.
Theorem 12.29 [Blum Speedup Theorem 1967]: Given a
two-place recursive function f there exists a total recursive
g such that for each program i for g there is a further
program j for g such that, for all but finitely many n,
f (n, Φj (n)) ≤ Φi (n).
Proposition 12.30: Every complexity class has a numbering
of the functions / sets in the class; this numbering is outside
the class, as the bit-wise exclusive or of 1 and the diagonal
function does not occur in the numbering.
Computational Complexity – p. 297
 Homeworks 12.31-12.34
Homework 12.31: Provide two levels in the elementary
hierarchy such that NEXPTIME sits between these levels.
The levels should be as close to each other as possible.
Homework 12.32: If the levels DOUBLEEXPSPACE and
DOUBLEEXPTIME of the elementary hierarchy are the
same, does this imply anything on other levels to be the
same?
Homework 12.33: A language L is called tally iff for every
two x, y of the same length it holds that L(x) = L(y). Prove
the following: A tally language is in EXPSPACE iff the set {
binary n : 0n ∈ L} is in DOUBLEEXPSPACE.
Homework 12.34: Is there a language which is inside the
Elementary Hierarchy and hard for all levels with respect to
LOGSPACE-many-one reductions? Prove the answer.
Computational Complexity – p. 298
 Homeworks 12.35-12.38
A simplified version of the Loop Programming Languages
uses all commands of addition machines except for GOTO
which is replaced by For-loops and If-Then-Else
statements; the borders and the loop variable of a For-loop
cannot be modified inside the loop, the loop variable is
always incremented after each run of the loop and the loop
is not entered when the lower bound is above the upper
bound. Write programs using this programming language
for the following functions – they are thus primitive
recursive. The input x is a positive integer.
x
Homework 12.35: f (x) = x
x .
Homework 12.36: g(x) = x! = 1 · 2 · 3 · . . . · x.
Homework 12.37: h(x) = g(g(g(g(g(x))))).
Homework 12.38: k(x) is the first prime number above 2x .
Computational Complexity – p. 299
 Homeworks 12.39-12.42
Homework 12.39: It there a total recursive two-input
function e, x 7→ ϕe (x) such that for each total recursive
two-input function e, x 7→ ψe (x) there is a d such that ϕd
differs from each ψe on some input x?
Homework 12.40: Provide a partial-recursive function f
such that the set {(x, y) : f (x) = y} is decidable (=
recursive) but the set {x : f (x) is defined} is undecidable.
Homework 12.41: Provide a numbering of all primitive
recursive functions.
Homework 12.42: Explain why there is a number of all
primitive recursive functions (which are all total) but not one
of all total recursive functions.

Computational Complexity – p. 300

 On Final Exam
The Final Examination is on Tuesday 6 May 2025 from
17:00 to 19:00 hrs.
The syllabus comprises all twelve lectures of this term and
includes the material tested in the midterm examination; of
the twelfth lecture in particular the first parts are relevant.
There will be 10 questions each scoring 6 points testing
both general knowledge on computational complexity and
problem solving skills. It is a closed book exam with one
page helpsheet.

Computational Complexity – p. 301