IS 239 - Algorithms and
Complexity
Unit I: Basic Analysis
• Preliminaries
• Mathematical background
• Algorithm analysis
• Sorting: Insertion sort, selection sort, Quicksort,
mergesort, heapsort, low bound of comparison of keys,
comparison of sorting algorithms
• Binary tree
• Hashing
• Huffman coding
Background
• An algorithm is a method for solving a class of
problems on a computer.
• The complexity of an algorithm is the cost,
measured in running time, or storage, or whatever
units are relevant, of using the algorithm to solve
one of those problems.
• Computational complexity is the study of the
amount of computational effort that is needed in
order to perform certain kinds of computations.
• The time complexity of a calculation is measured
by expressing the running time of the calculation
as a function of some measure of the amount of
data that is needed to describe the problem to the
computer.
Easy and Hard Problems
• Easy computation
– A computation that is guaranteed to take at most cn3
time for input of size n will be thought of as an ‘easy’
computation.
– A computation that needs at most n10 time is also
easy.
• Hard problem
– If a certain calculation on an input of size n were to
require 2n minutes, then that would be a ‘hard’
problem.
• The general rule is that if the running time is
at most a polynomial function ( f(n) = knx )of
the amount of input data, then the calculation
is an easy one, otherwise it’s hard.
• Easy problems
– Many problems in computer science are known to be easy.
– To convince someone that a problem is easy, it is enough to
describe a fast method for solving that problem.
• To convince someone that a “problem is hard” is hard,
because you will have to prove to them that it is
impossible to find a fast way of doing the calculation. It
will not be enough to point to a particular algorithm and
to lament its slowness. After all, that algorithm may be
slow, but maybe there’s a faster way.
– An example of a hard problem is the tiling problem.
• Suppose we are given infinitely many identical floor
tiles, each shaped like a regular hexagon. Then we can
tile the whole plane with them, i.e., we can cover the
plane with no empty spaces left over. This can also be
done if the tiles are identical rectangles, but not if they
are regular pentagons.
Tiling problem
• Computational question: Suppose we are given a
certain polygon, not necessarily regular and not
necessarily convex, and suppose we have infinitely
many identical tiles in that shape. Can we or can we not
succeed in tiling the whole plane?
– That elegant question has been proved to be computationally
unsolvable.
– We know that there is no any fast way to solve that problem
on a computer, and it has been proved that there isn’t any
way to do it.
– What has been proved is that no single method exists that can
guarantee that it will decide this question for every polygon.
• The fact that a computational problem is hard doesn’t
mean that every instance of it has to be hard. The
problem is hard because we cannot devise an algorithm
for which we can give a guarantee of fast performance
for all instances.
