# COMS W3261 Computer Science Theory Lecture 14: November 7, 2012 Algorithms and the Church-Turing Thesis

## Outline

• Definition of algorithm
• Programming techniques for Turing machines
• Variants of Turing machines
• The Church-Turing thesis
• The Diagonalization Language Ld is not RE

## 1. Definition of Algorithm

• Surprisingly, there is no universally agreed-upon definition for the term "algorithm". Informally, we can think of an algorithm as a collection of well-defined instructions for carrying out some task.
• In The Art of Computer Programming, Donald Knuth states that an algorithm should have five properties:
1. Finiteness: An algorithm must always terminate after a finite number of steps.
2. Definiteness: Each step of an algorithm must be precisely defined.
3. Input: An algorithm has zero or more inputs.
4. Output: An algorithm has one or more outputs, quantities which have a specified relation to the inputs.
5. Effectiveness: All of the operations to be performed in an algorithm can be done exactly and in a finite length of time.
• In this course we will use a Turing machine that halts on all inputs as the definition of an algorithm. The term decider is sometimes used for such a Turing machine.
• A language L that can be recognized by an algorithm is said to be recursive.
• If a language L is recursive, we say L is decidable.
• If a language L is not recursive, we say L is undecidable.
• In general, a Turing machine need not halt all inputs. An input on which a Turing machine never halts is not in the language defined by the Turing machine.
• A language L that can be recognized by a Turing machine is said to be recursively enumerable.
• The term Turing-recognizable language is sometimes used for a recursively enumerable language.
• Note that a language may be undecidable because it is not recursive but is recursively enumerable or because it is not recursively enumerable.

## 2. Programming techniques for Turing machines

• Turing machines are exactly as powerful as conventional computers.
• To make the behavior of a Turing machine clearer, we can use the finite-state control of a Turing machine to hold a finite amount of data. One way to do this is to use states with multiple fields, where one field represents a position in the Turing machine program, and the other fields hold data elements. The number of fields in a state is always finite.
• Another way to make the behavior of a Turing machine clearer, is to think of the tape as having several tracks.
• We can also group states into "subroutines". A subroutine has its own start state, and another state which can serve as a "return" state.

## 3. Variants of Turing machines

• There is no standard definition of a Turing machine. In this course we will use the definition in HMU, which assumes that a TM always halts when it enters an accepting state.
• Some books, such as Sipser, define a TM as having a semi-infinite input tape with one accept state and one reject state. The TM halts if it enters either the accept or reject state. A Sipser-style TM is equivalent to an HMU-style TM in computational capability.
• We can generalize a Turing machine to have multiple tapes. This extension does not add any new computational power to a Turing machine.
• We can allow a Turing machine to be nondeterministic by allowing it to make a choice of moves in a given state on a given tape symbol. This extension does not add any new computational power to a Turing machine.
• A Turing machine with a semi-infinite tape (one that is infinite only to the right) is just as powerful as a regular Turing machine.
• A pushdown automaton with two stacks is just as powerful as a regular Turing machine.
• A two-counter machine is just as powerful as a regular Turing machine.

## 4. The Church-Turing Thesis

• A Turing machine can compute a function from an input to an output by reading the input, making a sequence of moves, and then halting, leaving only the output of the function on the tape.
• A recursive function is one that can be computed by a Turing machine that halts on all inputs.
• A partial-recursive function is on that can be computed by a Turing machine that need not halt on all inputs. The output of the function on an input for which the Turing machine does not halt is said to be undefined.
• The Church-Turing thesis says that any general way to compute will allow us to compute only the partial-recursive functions. The Church-Turing thesis is unprovable because there is no precise definition for "any general way to compute."
• An informal way of expressing the Church-Turing thesis is that any function that can be effectively computed can be computed by a Turing machine.

## 5. The Diagonalization Language Ld is not Recursively Enumerable

• We can enumerate all binary strings.
• We can enumerate all Turing machines.
• We define Ld, the diagonalization language, as follows:
1. Let w1, w2, w3,... be an enumeration of all binary strings.
2. Let M1, M2, M3,... be an enumeration of all Turing machines.
3. Let Ld = { wi | wi is not in L(Mi) }.
• Theorem: Ld is not a recursively enumerable language.
• Proof:
• Suppose Ld = L(Mi) for some TM Mi.
• This gives rise to a contradiction. Consider what Mi will do on the input wi.
• If Mi accepts wi, then by definition wi cannot be in Ld.
• If Mi does not accepts wi, then by definition wi is in Ld.
• Since wi can neither be in Ld nor not be in Ld, we must conclude there is no Turing machine that can define Ld.

## 6. Reading Assignment

• HMU: Ch. 8, Sect. 9.1

## 7. Practice Problems

1. Informally describe how a single-tape Turing machine can simulate a two-tape Turing machine.
2. Informally describe how a deterministic Turing machine can simulate a nondeterministic Turing machine.
3. Informally describe how a pushdown automaton with two stacks can simulate a Turing machine.
4. Given a suitable encoding of input strings and Turing machines, show that L = { wi | wi is not accepted by M2i } is not recursively enumerable.

aho@cs.columbia.edu