M269: Theoretical definitions in the last two lectures
(i.e., Lec. 10 & Lec. 11)
Paradigms of Algorithms
• Brute force: In computer science, brute-force search or exhaustive search, also
known as generate and test, is a very general problem-solving technique that
consists of systematically enumerating all possible candidates for the solution and
checking whether each candidate satisfies the problem's statement. e.g. Selection
and insertion sort algorithms…naïve pattern matching algorithm.
• Greedy Algorithms: The solution is constructed through a sequence of steps. At
each step the next optimal step is selected locally, without considering whether this
step leads to the final global optimal solution or not. e.g. Kruskal, Dijkstra algorithm.
• Divide-and-Conquer: Divide the problem instance to several smaller sub-instances
then solve each one independently. Finally, the solutions of these sub-instances are
combined to form final solution. e.g. Merge and quick sort algorithm.
• Dynamic Programming: Solve problem of divide-conquer where identical sub-
instances are computed repeatedly. Smallest sub-instances are solved first and
results are placed in table to be used to construct larger sub-instances solution. e.g.
Floyd-Warshall algorithm.
• Optimization Programming: Optimization is the selection of a best element from
set of available alternatives. In general, optimization includes finding "best available"
values of some objective function given a defined domain (or input), including a
variety of different types of objective functions and different types of domains. e.g.
genetic algorithm, ant colony algorithm.
The types of problems: P, NP, NP-Complete, NP-Hard problem
• P problems are questions that have a yes/no answer and can be easily solved by a
computer. For example, determining whether a number is prime is a relatively easy
problem to solve.
• NP problems are questions that have yes/no answers that are easy to verify, but
are hard to solve. And by hard, I mean it would take years, decades or centuries for
your computer to come up with an answer. For example, the travelling salesman
problem is trying to figure out the shortest trip through a bunch of cities, visiting each
city only once. Let's say you ask me the question, can I visit these 500 cities in my
state and spend less than 3000 miles on the road? If I magically show you an
itinerary that takes you to all those 500 cities, you can easily verify whether that
itinerary is less than 3000 miles. BUT, trying to come up with that itinerary in the first
place is really hard for computers.
• NP-complete problems are special kinds of NP problems. You can take any kind of
NP problem and twist and contort it until it looks like an NP-complete problem. For
example, the knapsack problem is NP. It can ask what's the best way to stuff a
knapsack if you had lots of different sized pieces of different precious metals lying on
the ground , and that you can't carry all of them in the bag. Surprisingly, there are
some tricks you can do to convert this problem into a travelling salesman problem. In
Page 1 of 3
fact, any NP problem can be made into a travelling salesman problem, which makes
travelling salesman NP-complete.
• NP-Hard problems are worst than NP problems. Even if someone suggested you a
solution to a NP-Hard problem, it'd still take forever to verify if they were right. For
example, in travelling salesman, trying to figure out the absolute shortest path
through 500 cities in your state would take forever to solve. Even if someone walked
up to you and gave you an itinerary and claimed it was the absolute shortest path,
it'd still take you forever to figure out whether he was a liar or not.
Prepositional Logic: Propositional logic, also known as sentential logic and statement
logic, is the branch of logic that studies ways of joining and/or modifying entire
propositions, statements or sentences to form more complicated propositions,
statements or sentences, as well as the logical relationships and properties that are
derived from these methods of combining or altering statements.
Predicate logic, first-order logic or quantified logic is a formal language in which
propositions are expressed in terms of predicates, variables and quantifiers. It should be
viewed as an extension to propositional logic, which lacks quantifiers. The First-order
logic uses quantified variables over non-logical objects and allows the use of sentences
that contain variables.
Logic programming is a type of programming which is largely based on formal logic.
Any program written in a logic programming language is a set of sentences in logical
form, expressing facts and rules about some problem domain.
Turing machine is a mathematical model of computation that defines an abstract
machine that operates on an infinite memory tape divided into discrete cells. The
machine positions its head over a cell and "reads" (scans) the symbol there. Then, as
per the symbol and its present place in a finite table of user-specified instructions, each
instruction either proceeds to a subsequent instruction or halts the computation.
Finite State Machine (FSM) is a mathematical model of computation that can be in
exactly one of a finite number of states at any given time. The FSM can change from
one state to another in response to some external inputs; the change from one state to
another is called a transition.
The Halting problem: Given a program and an input to the program, determine if the
given program will eventually stop with this particular input. If the program doesn’t stop,
then it is in an infinite loop and this problem is unsolvable.
Computability and Limits of Computation:
• Limits on Arithmetic: There are limitations imposed by the hardware on the
representations of both integer numbers and real numbers. The maximum
number of significant digits that can be represented is limited to a certain value.
• Limits on Components: Although most errors are caused by software,
hardware components do fail
Page 2 of 3
• Limits on Communications: They are the limits that make communication as
such possible, as in, for example, distinctions between signal and noise.
• Limits of software: Commercial software contains errors. Software testing can
demonstrate the presence of bugs but cannot demonstrate their absence. As we
find problems and fix them, we raise our confidence that the software performs
as it should, but we can never guarantee that all bugs have been removed
Page 3 of 3
