Greedy Algorithms
Overview
• Greedy algorithms are used to solve optimization problems.
• Problems exhibit optimal substructure.
• Problems also exhibit the greedy-choice property.
• When we have a choice to make, make the one that looks best right
now.
• Make a locally optimal choice in hope of getting a globally optimal
solution.
Greedy Solutions to Optimization Problems
• Every two-year-old knows the greedy algorithm.
• In order to get what you want, just start grabbing
what looks best.
• Surprisingly, many important and practical
optimization problems can be solved this way.
Greedy Algorithms
• Many real-world problems are optimization problems in that they
attempt to find an optimal solution among many possible candidate
solutions.
• A familiar scenario is the change-making problem that we often
encounter at a cash register:
• Receiving the fewest numbers of coins to make change after paying
the bill for a purchase.
Example 1: Making Change
Problem:
• Find the minimum no. of quarters, dimes, nickels, and
pennies that total to a given amount.
Formal Algorithm
• Make change for n units using the least possible number of coins.
• MAKE-CHANGE (n)
C ← {100, 25, 10, 5, 1} // constant.
Sol ← {}; // set that will hold the solution set.
Sum ← 0 sum of item in solution set
WHILE sum not = n
x = largest item in set C such that sum + x ≤ n
IF no such item THEN
RETURN "No Solution"
S ← S {value of x}
sum ← sum + x
RETURN S
The Make-Change algorithm
• The purchase is worth $5.27, how many coins and what
coins does a cash register return after paying a $6 bill?
• For a given amount (e.g. $0.73), use as many quarters ($0.25)
as possible without exceeding the amount.
• Use as many dimes ($.10) for the remainder, then use as many
nickels ($.05) as possible.
• Finally, use the pennies ($.01) for the rest.
Make-Change algorithm
• Greedy algorithms determine the minimum number of coins
to give while making change.
• Steps one would take to emulate a greedy algorithm to
represent 36 cents using only coins with values {1, 5, 10, 20}.
• The coin of the highest value, less than the remaining change
owed, is the local optimum.
Hard: Making Change Example
• Problem: Find the minimum # of
4, 3, and 1 cent coins to make up 6 cents.
• Greedy Choice: Start by grabbing a 4-cent coin.
• Consequences:
4+1+1 = 6 mistake
3+3=6 better
Greedy Algorithm does not work!
When Does It Work?
•Greedy Algorithms:
•Easy to understand and to code, but do they work?
• For most optimization problems, all greedy algorithms tried do not work
(i.e. yield sub-optimal solutions)
• But some problems can be solved optimally by a greedy algorithm.
• The proof that they work, however, is subtle.
• As with all iterative algorithms, we use loop invariants.
Greedy Algorithms
• A greedy algorithm is an algorithm that follows the problem solving
heuristic of making the locally optimal choice at each stage with the
hope of finding a global optimum.
• A greedy strategy generally does not produce an optimal solution.
• But nonetheless a greedy heuristic may yield locally optimal solutions
that approximate a global optimal solution in a reasonable time.
• Greedy algorithms are very fast because they only consider each
object once.
Greedy algorithms components
• A candidate set; From which a solution is created.
• A selection function; which chooses the best candidate to be added to
the solution
• A feasibility function; That is used to determine if a candidate can be
used to contribute to a solution
• An objective function; Which assigns a value to a solution, or a partial
solution, and
• A solution function; Which will indicate when we have discovered a
complete solution
Greedy Strategy
• The choice that seems best at the moment is the one we go with.
• The algorithm is greedy because at every stage it chooses the largest coin
without worrying about the consequences.
• It never changes its mind in the sense that once a coin has been included
in the solution set, it remains there.
• The "greedy-choice property" and "optimal substructure" are two
ingredients/elements in the problem that lend to a greedy strategy.
Greedy choice property
• It says that “a globally optimal solution can be arrived at by making a locally
optimal choice”.
• Make whatever choice seems best at the moment and then solve
the sub-problems that arise later.
• The choice made by a greedy algorithm may depend on choices
made so far but not on future choices or all the solutions to the
sub-problem.
Greedy algorithms characteristics
• Greedy algorithms are:
• Simple and straightforward.
• Shortsighted in their approach (take decisions on the basis of
information at hand without worrying about the effect these
decisions may have in the future).
• Easy to invent, implement and most of the time quite
efficient.
Comparison:
• Greedy algorithms iteratively make one greedy choice after another,
reducing each given problem into a smaller one.
• i.e.; A greedy algorithm never reconsiders its choices.
• This is the main difference from dynamic programming, which is
exhaustive and is guaranteed to find the solution.
• Unlike Dynamic Programming, which solves the sub-problems bottom-
up, a greedy strategy usually progresses in a top-down fashion, making
one greedy choice after another, reducing each problem to a smaller one.
Summary:
• The advantage to using a greedy algorithm;
• Solutions to smaller instances of the problem can be
straightforward and easy to understand.
• The disadvantage;
• It is entirely possible that the most optimal short-term
solutions may lead to the worst long-term outcome.
Summary:
• Greedy algorithms are often used in:
• Ad hoc mobile networking to efficiently route packets with
the fewest number of hops and the shortest delay possible.
• They are also used in machine learning, business
intelligence (BI), artificial intelligence (AI) and
programming.
Any questions…??
