11/13/2014
ECEG-5501
Introduction to Algorithm
Analysis & Design
Introduction
What is Algorithm?
a sequence of unambiguous instructions for
solving a problem, i.e. for obtaining a required
output for any legitimate input in a finite
amount of time.
Introduction
What is a problem?
Roughly, a problem specifies what set of outputs is
desired for each given set of inputs.
Example: The Sorting Problem
A problem instance is just a specific set of inputs.
Solving a problem instance consists of specifying a
procedure for converting the inputs to an output of the
desired form (called a solution).
An algorithm that is guaranteed to result in a solution
for every instance is said to be correct.
Note that a given instance may have either no
solutions or more than one solution.
11/13/2014
Introduction
The First Algorithm
Euclids Algorithm
Euclid(m,n){
while n does not divide m
r m%n;
mn
nr
end
return n
}
while n 0 do
r m mod n
mn
n r
return m
Simple factoring algorithm
Input: 2 integers m,n
Output: Largest integer that
divides both without a remainder
Algorithm 1:
1. factorize m=m1Xm2XXmP
2. factorize n=n1Xn2XXnk
3. Identify the common factors,
multiply the result and return
Introduction
What is a program?
A program is the expression of an algorithm in
a programming language
A set of instructions which the computer will
follow to solve a problem
Introduction
Important points
Non-ambiguity
Range of inputs
Several algorithms for solving the same problem
11/13/2014
Introduction
How Algorithm is Specified?
An algorithm is specified:
In words (in a free and also a step-by-step form)
In pseudocode, i.e. a mixture of a natural language and
programming language-like construct
if then
if then else
for do
while do
repeat until
Introduction
Pseudo code
C
false
true
true
S1
next statement
S2
next statement
if then
if then else
Introduction
Pseudo code
for k 1 to n do
.
.
.
done n times
11/13/2014
Introduction
Pseudo code
while do
false
C
true
next statement
Introduction
Pseudo code
repeat until
true
C
false
next statement
Introduction
Steps Problem Solving using Algorithm
Understand the problem
Decide on computational means
Exact vs approximate solving,
Algorithm design technique
Design an algorithm
Prove correctness
Analyze the algorithm
Code the algorithm
11/13/2014
Introduction
Fundamentals of Algorithmic Problem Solving
What does it mean to understand the problem?
Read the problems description carefully
Do a few small examples by hand
Think about special cases
Capabilities of the Computational Device
- Random access machines
- Sequential algorithms
- parallel algorithms
- exact vs Approximate algorithms
Introduction
Fundamentals of Algorithmic Problem Solving
Algorithm Design Techniques
Build a computational model of the solving
process
Prove correctness
Correct output for every legitimate input in finite
time
By Mathematical induction
Introduction
Fundamentals of Algorithmic Problem Solving
Analyze the algorithm
Efficiency: time and space
Simplicity
Generality:
range of inputs, it accepts
Problems the algorithm solve
Optimality
(What is the minimum amount of effort any algorithm will need to exert
to solve the problem?)
11/13/2014
Introduction
Fundamentals of Algorithmic Problem Solving
Analyze the algorithm
recognize limitations of various algorithms for
solving a problem
understand relationship between problem size and
running time
how to analyze an algorithm's running time without
coding it
Coding
How the objects and operations in the algorithm are
represented in the chosen programming language?
Introduction
Important Problem Types
Sorting
Searching
String Processing
Combinatorial Problems
Geometric Problems
Numerical Problems
Introduction
Important Problem Types
Sorting
rearrange items of a given list in a specific order, based on some key
Searching
find a given search key in a given set
String Processing
String is a sequence of characters from alphabet
search a word in a text (string matching)
Graph Problems
Collections of vertexes some of them connected by line segments
includes graph traversal problems, shortest-path algorithms, traveling
salesman problem, graph coloring problems (event scheduling)
11/13/2014
Introduction
Important Problem Types
Combinatorial Problems
Use permutation, combination, subset that satisfies certain
constraints(either maximize a value or minimize a cost)
e.g. traveling salesman
Numerical Problems
solve equations & systems of equations, evaluate
functions, etc.[ difficulty: round-off error]
Geometric problems
Deals with geometric objects such as points, lines and
polygon
E.g. Closest pair problem
Introduction
Fundamental Data Structures
Linear Data Structures
Array
Linked list
Stack
Queue
Introduction
Fundamental Data Structures
Non Linear Data Structures
Graphs G=<V, E>
-graph representation: adjacency lists,
adjacency matrix
- weighted graph
Trees : connected graph without cycles
Rooted trees
Ordered trees
Binary trees
Tree representation: as graphs; binary nodes
11/13/2014
Introduction
Fundamental Data Structures
Sets, Bags, Dictionaries
Set: unordered collection of distinct elements
Operations: membership, union, intersection
Representation: bit string; linear structure
Bag: unordered collection, elements may be
repeated
Dictionary: a bag with operations search, add,
delete
Introduction
Algorithm classification
By types of problems: Study sorting algorithms, then
searching algorithms, etc.
By design technique: Study algorithms by design
strategy.
A general approach to solving problems
Examples of design strategies include:
Brute force
Divide-and-conquer
The greedy method
Dynamic programming
Introduction
Example: ALGORITHM Input / Output
Example: ALGORITHM Euclid (m, n)
// compute GCD(m, n) by Euclids algorithm
// input: two non-negative, not-both-zero integers m and n
// output: Greatest Common Divisor of m and n
while n 0 do
r m mod n
m n
n r
return m
Input : m and n
Output: the greatest common divisor of m and n
Time : ?
11/13/2014
Introduction
Example: ALGORITHM Input/Output
Example: sequential search
Algorithm sequential search (A[0..n-1], K)
// searches for a given value in a given array by sequential search input: an array A[0..n-and
//a search key K output: returns the index of the first element of A that matches K
or
//-1 if there are no matching elements
i0
while i n and A[i] K do
ii+ 1
if i n return i
else return -1
Input: A[0..n-1], K output: j such that A[j] = K or 1
Time: ?
Introduction
End of Slide Show
