12/24/2021

Overview
• Problem classification
CSIT113 • Basic framework for solving problems
Problem Solving • Algorithms
UNIT 1 • Problem solving strategies
INTRODUCTION
Lecturer/Tutor: Dr Tan Hee Beng Kuan
(email: [email protected])

1 2
1

Q Q
Introduction: What is a problem? Introduction: Some definitions.
Is this a problem? • “A problem exists when the goal that is sought is not directly attainable
• What is 3+45? by the performance of a simple act available in the animal’s repertory…”

Is this a problem? Thorndike 1989

• Given the distribution of rain in the last week, what is the probability of flooding • “A person is confronted with a problem when he wants something and
in building 3? does not know immediately what series of actions he can perform to get
Is this a problem? it”
Newell & Simon 1972
• You are sitting in a lecture and you want to be on the beach.
• “Whenever there is a gap between where you are now and where you
want to be, and you don’t know how to cross that gap, you have a
problem”
Hayes, 1980

3 4
 12/24/2021

Q Q A problem exists when there is a ‘gap’ between the

Introduction: Problem components. current state and the goal state!

• All problems have: Goal State

• A start state • Where you begin.
Current State
• A current state • Where you are now.
• A goal state • Where you want to be. Moves:
N, S, E, W
• Operators • To move between states.
• Constraints • Limits on moves or solution. Constraint:
Cannot go through walls

Initial State

5 6

Q Q
Your turn. Your turn.

• Identify the start and goal states, and the operators • Identify the start and goal states, and the operators
and their constraints in the following problem: and their constraints in the following problem:
• What is 3+45? • You are sitting in a lecture and you want to be on the
beach.
Answer: Answer:
start state: the problem as given start state:
goal state: the numerical answer goal state:
operators: addition and multiplication operators:
constraints: the rules of arithmetic constraints:

• What is the ‘gap’ in this problem?

• In this problem, the answer is less clear.
7 8
 12/24/2021

Q Q
Types of problem. Well vs. Ill defined problems.
• Problems can be classified in a number of different ways. 1. If two fair six-sided dice are rolled, what is the probability of obtaining
a total of 12?
• Know what type of problem you have.
2. Plan a meal for a guest.
• This helps you better understand the problem, and what you might
need to do to solve it.
• Or whether it is even worth trying.
• Well defined problem:
• All the information necessary to solve the problem is either explicitly given or can
be easily inferred
• Ill defined problem:
• There is uncertainty in either the initial state, the permissible operations/actions,
or the final state.

9 10

Q Q
Knowledge rich vs. poor problems. Knowledge rich vs. poor problems.
1. Devise an efficient memory management scheme for a multi-processor • Knowledge Rich Problems:
computer with job sizes distributed according to a normal distribution with • Require prior, specialized or domain-specific knowledge to solve.
μ=500MB and σ=128MB. • Knowledge Poor Problems:
2. Starting with the sequence of pieces given below: • Do not require specialized knowledge. Can be solved using strategies and
methods that apply to many types of problems.
• These are also known as domain-specific and domain-general
rearrange them so that they end up in the following sequence: problems.

A dot can move to an empty space adjacent to it or jump over only ONE dot of the other
colour into an empty space. White dots can only move to the left and brown dots to the
right.

11 12
 12/24/2021

Q Q
Hard vs. Easy problems. Important Problem Types

Classified from their characteristics, the most important

1. What are the factors of 9020779199? problem types in CS/IT are:
2. What is 9433195629?
• Sorting
• How hard a problem depends on how much work you need
to do to reach the goal. • Searching
• Graph problems
• Problems have varying levels of complexity which is a
measure of how hard they are to solve. • Combinatorial problems
• Numerical problems
• We will look more closely at this later in the subject.
• String processing
• Geometric problems
13 14

Q
Introduction: What is problem solving? Polya’s* Problem Solving Method.
• “the bridging of the gap between an initial state of affairs and a desired Polya’s method has four steps:
state where no predetermined operator or strategy is known to the 1. Understand the problem: Make sure you understand what the problem is asking.
individual”
2. Plan a strategy for solving the problem.
• Ollinger, M. & Goel, V. “Problem-Solving.” in
V. Goel, & A. von Müller (Eds), Towards a Theory of Thinking. Springer. Goel, V. 3. Execute your strategy, and revise it if necessary.
2010
4. Check and interpret your result.
• … involves pursuing a goal state In this subject we will use/discuss this method, although we will often not
• … finding a way to reach it explicitly refer to it!
• … requires mental activity. * George Polya “How to Solve It”, Princeton University Press, 1945

15 16
 12/24/2021

Q Q
Understanding the problem.
A Simple Problem.
• Read the problem very carefully.
• A butcher is six foot, four inches tall and wears size 14 shoes. • Take note of any data, or information, that is given. E.g. numbers, quantities, names,
• What does he weigh? etc.
• From the data given:
• The butcher weighs meat! • Identify the start state
• Make sure the question is asking what you think it is! • Identify the goal state
• What is it you have to determine? (unknowns)
• Ok, this is a tricky question but it makes an important point.
• Identify any constraints
• What are you not allowed to do?
• Identify any operators
• What can you do?
• If the problem is not clear restate it in different ways to clarify it.
17
• This is just the first step to solving a problem! 18

Q Q
Some practice problems: approach with caution. Algorithms.
1. Anna had six apples and ate all but four of them. How many apples
were left? • A lot of the problems we encounter in computing can be specified using
algorithms.
2. If there are 12 one-cent stamps in a dozen, how many two-cent stamps • An algorithm is a systematic description of how to solve a problem.
are there in a dozen?
• Algorithms have many aliases:
3. If Mr. Howard’s rooster laid an egg in Mr. Bush’s yard, who owns the
egg?
Problem Algorithm
4. A lady did not have her driver’s license with her when she failed to
Knit a sweater Pattern
stop at a stop sign and then went three blocks down a one-way street
the wrong way. A policeman saw her, but he did not stop her. Explain. Bake a cake Recipe

Drive to Sydney Directions

19 20
 12/24/2021

What is an algorithm? Properties of an algorithm.

• A method for accomplishing a task (such as performing a calculation or • An algorithm must be:
solving a problem). • Finite – it should terminate.
• A finite sequence of instructions, each of which has a clear meaning and • Complete – it should specify every step that must be performed
can be performed with a finite amount of effort in a finite length of • Correct or Effective – it should give you the right answer or get to the desired goal.
time. • Precise – it should be clear and specific enough that it can be followed.

• A precise description of the steps required to go from an initial state to a • An algorithm should be:
final state. • Efficient – it should arrive at the answer in a reasonable time or with a reasonable
amount of effort.
• General – it should solve more than just the specific problem being considered.

21 22

Q Q
Correctness vs. Efficiency. Efficiency and Complexity.
• An algorithm is correct if it comes up with the solution. • Can we estimate how efficient an algorithm is?
• Sometimes we have no efficient way to do this. • The efficiency of an algorithm is expressed in terms of complexity
• Time complexity
• In this case we may be satisfied with an algorithm that gives us a ‘good
enough’ or ‘near-enough’ solution. • Space complexity
• Communication complexity
• Lots of real world problems fall into this category.
• The lower bound complexity for an algorithm to solve a particular
problem is the complexity of that problem.
• In general, no algorithm to solve a problem can be less complex than an algorithm
to check a solution.

23 24
 12/24/2021

Q Q
Generality. Classifying algorithms – One Common Way:
• A good algorithm should be general • Algorithms can be categorised by the domain to which they apply:
• An algorithm that works in all cases of a problem is better than an algorithm that • graph algorithms
only works in certain cases. • string algorithms
• An algorithm that solves a set of related problems is better than an algorithm that • sorting algorithms
only solves a specific problem.
• cryptographic algorithms
• We will talk a bit more about generality when we discuss abstraction and
preconditions. • compression algorithms
• etc.

25 26

Q
Classifying algorithms – Another Common Way: Creating algorithms.
• Algorithms can be categorised by the approach which they apply: • So, how do you create an algorithm?
• greedy • Solve the problem.
• divide and conquer • Write down the steps taken in solving the problem.
• recursive • Pseudocode is used to specify (write) algorithms: next, we shall introduce
sequence that will be used in writing algorithms before introducing pseudocode
• branch and bound briefly.
• genetic • The first step may be a bit tricky!
• heuristic
• etc.
• This is the main topic of this subject.
• As we will see in future weeks there are a number of broad principles
and approaches which are useful in solving all sorts of problems.
• We often refer to these as strategies.
27 28
 12/24/2021

What is an sequence
An sequence of Nine Integers
 An sequence is a fixed-size, sequenced collection of elements
of the same data type
• The length or size of an
[0] [1] [2] [3] [4] [5] [6] [7] [8]
element 0 sequence is the number of
elements in the sequence. ages 10 34 9 1 2 10 3 89 3
element 1
element 2 • Each element in the sequence is
identified by an integer called
index, which indicates the
position of the element in the ages[0] = ? ages[0] = 10
sequence. ages[5] = ? ages[5] = 10
element 6
ages[7] = ? ages[7] = 89
numbers • An sequence’s index starts from 0 ages[2] = ? ages[2] = 9
Q: how many elements in this and goes up to the sequence’s ages[9] =? ages[9] =illegal
sequence? length minus 1 ages[11]=? ages[11]=illegal
There are 7, indexed from 0 to 6

29 30 30

Pseudocode Pseudocode Details

Example: find the largest element in an • Control flow: • Some common notation:
• High-level description of an if … … [else …]
algo. sequence  Assignment x= b
• More structured than English while …..…
prose Input sequence A of n integers for … …  Equality testing if x = = b
• Less detailed than a program Output largest element in A
n2 Superscripts and other
• 3 basic control constructs: sequenceMax(A, n) { • Algorithm declaration: mathematical formatting
sequence, selection and Max = A[0] Input … allowed
iteration for (i = 1 to n  1) Output … x The largest integer f
if A[i]  Max then Algorithm_Name(arg, such that f ≤ x
• Control constructs must be
Max = A[i] arg,…) The smallest integer c
reflected clearly by some x
return Max such that c ≥ x
standard conventions (many } • Calling another algo:
options are available) Algo_Called (arg,
• Other Mathematical
0 1 2 3 4 arg,…) formatting are also
A 9 10 25 34 6 Max: 34 • Return value: allowed
return expression 32
31
 12/24/2021

Pseudocode: The return statement Pseudocode: Selection

if (condition) if (condition)
return expression action1 action1
else
• The expression under the return statement can only be a single action2
variable or single expression with the following meaning: If an action has more than one statement, enclose
all the statements by a pair of braces, {}.
 Single variable: The value of the variable upon the execution of the statement is returned. Example:
if (b > x) // if b is larger than x, update x
 Single expression: The value of the expression upon the execution of the statement is x = b
returned. if (c > x) // if c is larger than x, update x
x = c
return x
• Regardless of where the return statement is placed, immediately after the
execution of the statement, the execution STOP and EXIT. No further Example:
if ( x  0) {
execution of any statement in the same procedure containing the return x = x + 1
statement. a = b + c
33
}
34

*Pseudocode: while loop Pseudocode: for loop

while (condition) 0 1 2 3 4 0 1 2 3 4
action for (var = init to limit) -3 20 450 89 90
s -3 20 450 89 90
action s
 If condition is true, action is executed.
 The process is repeated until condition becomes false
 The variable var is first set to the value init. If var  limit, action is executed and 1 is added to var.
//Find the max value in an sequence s of n numbers using a while loop
 The process is repeated until var > limit
sequence_max1(s, n) {
large = s[0] // Find the max value in an sequence s of n numbers using a for
i = 1 loop
while (i < n) { sequence_max2 (s, n) {
if (s[i] > large) // larger value found large = s[0]
large = s[i] for (i = 1 to (n-1) ) {
i = i + 1
if (s[i] > large) // larger value found
} large = s[i]
return large }
} return large
35 } 36
 12/24/2021

Pseudocode: for loop Additional Algorithm Example 1

for (var = init downto limit)
The following algorithm correctly checks the equality of two
action
sequences A and B:
 Action is executed as long as var  limit, and
1 is subtracted from var after each loop iteration
 The process is repeated until var < limit

Write an algorithm that returns the index of the last occurrence of the value key in the
sequence s[0], . . , s[n-1]. If key is not in the sequence, the algorithm returns the value
99999.

find_last_key (s, n, key) {

for (i = (n-1) downto 0 ) {
if (s[i] == key ) // key value found
return i
}
return 99999 37 38
}

Additional Algorithm Example 2 Q

The following algorithm wrongly checks the equality of two Problem Solving Strategies.
sequences A and B: just based on the equality of the last element • Explore the problem space
• Consider potential states of the problem until you find one that is a
solution
• This covers a range of strategies including
• brute-force
• trial and error
• backtracking
• depth-first search
• etc.

39
40
 12/24/2021

Q Q
Problem Solving Strategies. Problem Solving Strategies.
• Analogy • Reduction
Is this like a problem you have solved before? Can you reduce it to a problem you have solved before?
• A tourist wants to convert 100 US dollars to Australian dollars. If the rate is 1.3 • A team of three runners competes in a race, each person runs for 3 minutes. If the
AUD to the USD, how many Australian dollars will he get? first person runs at 3m/s, the second person runs at 4m/s and the third person
• A car stops after 100 seconds. If it travelled at an average speed of 1.3 m/s. How far runs at 5m/s, how far have they run?
has it travelled?
• This is not always a good idea.
• A car moving at an average speed of 100km/h travels for 1.3 hours. How far has it
gone?

41 42

Q Q
Problem Solving Strategies. Problem Solving Strategies.
• Research • Abstraction
Has someone else already solved a problem like this? Are there any high-level patterns or principles?
• There is no need to reinvent the wheel • If you can identify and represent the key aspects of a problem, it is
• Read widely much easier to solve
• Discuss with others • This is an important strategy that covers lots of aspects
• Consider the type of problem you are dealing with for some hints on where to • Symmetry
look
• Note:
• Pattern Recognition
• Typically not an appropriate technique in assignments • Maths
• Always give credit • Notation
• etc.

43 44
 12/24/2021

Q Q
Problem Solving Strategies. Problem Solving Strategies.
• Solve a simpler problem • Solve a simpler problem - reduce the number of constraints
Can solving a simplified problem help solve the original problem? • Informal technique:
• The solution to the simple problem may give you • throw away some constraints, solve the problem, work out how to
• insight into the original problem reintroduce the constraints
• or a partial solution • Formal technique:
• Covers a range of different techniques, depending on how you • Branch and bound
simplify

45 46

Q Q
Problem Solving Strategies. Problem Solving Strategies.
• Solve a simpler problem - Solve part of the problem • Solve a simpler problem - Make the problem smaller
• Start close to the goal, work backwards • Instead of solving a problem involving 1000 people, solve it for 1
• Start from both ends, work towards the middle person, then 2 people, then 3...
• Break the problem into sub-problems and solve each sub-problem • Identify a pattern
• Formal techniques • Use that to solve your original problem
• divide and conquer • Formal techniques:
• functional decomposition / modularisation • Induction
• Recursion

47 48
 12/24/2021

Q Q
How do I learn to solve problems? After you solve a problem.
• Do CSIT113 (hopefully this will help). • Don’t just solve the problem, think about how you solved the problem
• Practice. • This is a critical factor in coming up with an algorithm
• Practice some more. • Think about how you solved it and what type of problem it is
• Practicing will enable you to: • This helps improve your strategy and allows you to more easily
• Identify similarities between problems. recognise strategies and when to use them
• Master techniques and variations of them. • Even if you can’t solve a problem, think about what is stopping you from
• Gain confidence in your ability to solve problems. solving it
• It is less likely you will be put of by something that looks tricky!
• lack of knowledge? sub-problem? not solvable?

49 50

Q Q
After you solve a problem. After you solve a problem.
• Often when solving a problem, you are too busy trying to get the answer • Think about how you solved the problem and then explain it to
to focus on how you are thinking or what you are learning someone else
• Your friend, grandmother, dog, computer
• You should reflect on what strategies you used, how well they worked,
what types of problem you encountered, whether you need more • Make use of tutorials for this
practice, etc. • Trying to explain your thinking helps you understand it better
• Writing an algorithm requires you to clearly express your strategy

51 52
 12/24/2021

Q Q
Some sample problems: What do we know?
Student: All three of my sons celebrate their birthday today. Can you tell me • What does each piece of the story tell us?
how old each of them is?
Lecturer: Sure, but you will have to tell me something about them. Student: All three of my sons celebrate their birthday today. Can you tell
Student: The product of their ages is 36. me how old each of them is?
Lecturer: … I’ll need more information.
There are three sons
Student: The sum of their ages is equal to the number of windows in that
building (13).
Lecturer: Good, but I still need an additional hint to solve the puzzle.
Student: My oldest son has blue eyes.
Lecturer: Ah, the ages of your sons are …

53 54

Q Q
What do we know? What do we know?
• What does each piece of the story tell us? • What does each piece of the story tell us?
Student: The product of their ages is 36. Student: The sum of their ages is equal to the number of windows in
Age of Son 1 Age of Son 2 Age of Son 3
that building (13).
36 1 1 Age of Son 1 Age of Son 2 Age of Son 3 Sum of Ages
36 1 1 38
18 2 1
18 2 1 21
12 3 1
12 3 1 16
9 4 1
9 4 1 14
9 2 2
9 2 2 13
6 6 1 6 6 1 13
6 3 2 6 3 2 11
4 3 3 4 3 3 10
55 56
 12/24/2021

Q Q
What do we know? What do we know?
• What does each piece of the story tell us? • What does each piece of the story tell us?
Lecturer: Good, but I still need an additional hint to solve the puzzle. Student: My oldest son has blue eyes.
• Why can’t he solve it yet? • There is an oldest son!
Age of Son 1 Age of Son 2 Age of Son 3 Sum of Ages Age of Son 1 Age of Son 2 Age of Son 3 Sum of Ages
36 1 1 38 36 1 1 38
18 2 1 21 18 2 1 21
12 3 1 16 12 3 1 16
9 4 1 14 9 4 1 14
9 2 2 13 9 2 2 13
6 6 1 13 6 6 1 13
6 3 2 11 6 3 2 11
4 3 3 10 4 3 3 10
57 58

Q Q
What do we know? Another example:
• We now have the only possible answer! • There are three boxes:
• This solution used a combination of strategies • One contains only apples.
• Brute-force, to get the initial table • One contains only pears.
• Logic, to remove options • One contains both apples and pears.
Age of Son 1 Age of Son 2 Age of Son 3 Sum of Ages
• Each box is labelled:
36 1 1 38
• No box has the correct label.
18 2 1 21
12 3 1 16 • You are allowed to examine one piece of fruit from one box.
9 4 1 14
9 2 2 13 • Can you label all the boxes correctly?
6 6 1 13
6 3 2 11
4 3 3 10
59 60
 12/24/2021

Q Q
What do we know? It’s not all algorithms.
• There are only two possible arrangements that fit the description: • Though algorithms are very important for solving problems in
Computer Science and Information Technology areas, not every
Label “Apples” “Pears” “Apples and
Pears” problem can be solved by constructing an algorithm.
Arrangement Pears Apples and Apples • Often, problems arise because of poor communication.
1 Pears

Arrangement Apples and Apples Pears • Consider the example of the garden swing…
2 Pears

• How can we tell them apart with a single piece of fruit? Try the box
labelled with “Apples and Pears”

61 62

Q Q
The garden swing. The garden swing.

What the customer described… What the project leader heard…

63 64
 12/24/2021

Q Q
The garden swing. The garden swing.

What the analyst designed… What the programmer wrote…

65 66

Q Q
The garden swing.
The garden swing.

What the beta testers got… The documentation…

67 68
 12/24/2021

Q Q
The garden swing. The garden swing.

What was first installed… What was finally installed…

69 70

Q Q
The garden swing. A final problem (for today).
• Consider a bench with enough room on it for three people.
• Two people are sitting on it.
What was needed!
• One is in the middle and one is to the left.
• The question:
• Who sat down first?
• Do we have enough information?
• What is involved?
• Psychology.
• Probability.
• …?

71 72
 12/24/2021

Q
A final thought (for today).
• Clarke’s Laws: (paraphrased)

1. When a distinguished but elderly lecturer states that something is possible, he

is almost certainly right. When he states that something is impossible, he is very
frequently wrong.
2. The only way of discovering the limits of the possible is to venture a little way
past them into the impossible.
3. Any sufficiently advanced technology is indistinguishable from magic.

• If we don’t understand the solution – it’s magic!

• If we do understand the solution – it’s technology!

73