Upload
barry-baldwin
View
214
Download
0
Embed Size (px)
Citation preview
Prof. Mario PavoneProf. Mario PavoneComputazione NaturaleComputazione Naturale
CdL Magistrale in InformaticaCdL Magistrale in InformaticaSDAI – DMI – UniCtSDAI – DMI – UniCt
[email protected]://www.dmi.unict.it/mpavone/
Teoria della Complessità:Teoria della Complessità:la classe NPla classe NP
Problem
A problem is a general question to be answered, usually possessing several parameters, or free variables, whose values are left unspecified.
A problem is described by giving:a general description of all its parameters, anda statement of what properties the answer, or
solution, is required to satisfyAn instance of a problem is obtained by
specifying particular values for all the problem parameters
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Search Space
Given a combinatorial problem PP, a search search spacespace associated to a mathematical formulation of PP is defined by a couple (S,(S,f f )) where SS is a finite set of configurations (or nodes
or points) andff a cost functioncost function which associates a real number
to each configurations of SS.For this structure two most common measures
are the minimum and the maximum coststhe minimum and the maximum costs. In this case we have the combinatorial combinatorial optimization problemsoptimization problems.
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Example: K-SATAn instance of the K-SAT problem K-SAT problem consists of a
set V of variables, a collection C of clauses over V such that each clause c C has |c|= K
The problem is to find a satisfying truth assignment for C
The search space search space for the 2-SATfor the 2-SAT with |V|=2 is (S,(S,f f ) ) where S={ (T,T), (T,F), (F,T), (F,F) }S={ (T,T), (T,F), (F,T), (F,F) } andthe cost functionthe cost function for 2-SAT2-SAT computes only the
number of satisfied clauses
ffsatsat (s)= #SatisfiedClauses(F,s), s (s)= #SatisfiedClauses(F,s), s S S
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Example: SEARCH SPACE of K-SAT
Let we consider F = (A B) ( A B)
A BA B ffsatsat(F,s)(F,s)
T T 1
T F 2
F T 1
F F 2
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
SEARCH LANDSCAPE
Given a search space (S,(S,ff)), a search landscapesearch landscape is defined by a triplet (S,n,f)(S,n,f) where nn is a neighborhood functionneighborhood function which verifies
n : S n : S 2 2SS - {0} - {0} This landscape, also called energy landscapeenergy landscape,
can be considered as a neutralneutral one since no search process is involved.
It can be conveniently viewed as weighted graphweighted graph G=(S,n,F)G=(S,n,F) where the weights are defined on the the weights are defined on the nodesnodes, not on the edges.
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
PROCESS LANDSCAPE
Given a search landscape (S,n,f)(S,n,f), a process process landscapelandscape is defined by a quadruplet (S,n,f,(S,n,f,)) where is a search processsearch process
The process landscape represents a particular view of the neutral landscape (S,n,f)(S,n,f) seen by a search algorithm.
Examples of search algorithmsExamples of search algorithms:Local Search AlgorithmsLocal Search AlgorithmsComplete Algorithms (e. g. Davis-Putnam algorithm)Complete Algorithms (e. g. Davis-Putnam algorithm)Evolutionary AlgorithmsEvolutionary Algorithms: Genetic Algorithms, Genetic Algorithms,
Genetic Programming, Evolution Strategies, Evolution Genetic Programming, Evolution Strategies, Evolution Programming, Immune AlgorithmsProgramming, Immune Algorithms
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
ALGORITHMS
Algorithms are general, step-by-step procedures for solving problems
For concreteness, we can think of them simply as being computer programs, written in some precise computer language.
An algorithm is said to An algorithm is said to solvesolve a problem a problem PP if that if that algorithm can be applied to algorithm can be applied to anyany instance instance II of of PP and is and is guaranteedguaranteed alwaysalways to produce a to produce a solution for that instance solution for that instance II
For example:An algorithm does not "solve" the TSP unless it
always constructs an ordering that gives a minimum length tour
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
BIBLIOGRAPHYBIBLIOGRAPHYM. R. Garey and D. S. Johnson,
Computers and Intractability: Computers and Intractability: A Guide to the Theory of NP-A Guide to the Theory of NP-CompletnessCompletness, W. H. Freeman, 1st ed. (1979).
OR: A compendium of NP A compendium of NP optimization problemsoptimization problems at the at the link:link:
http://www.csc.kth.se/~viggo/problemlist/
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Problem: the start point
PROBLEMPROBLEM: to refer to a question such as: "Is a given graph G=(V,E) K-colorable?"
INSTANCEINSTANCE of a problem is a list of arguments, one argument for each parameter of the problem
a problem is a binary relationa problem is a binary relation on a set II of problem instancesproblem instances, and a set SS of problem problem solutionssolutions: Q: I → SQ: I → S
decision problemsdecision problems: Q: I → {0, 1}Q: I → {0, 1}in general, if we can solve an optimization in general, if we can solve an optimization
problem quickly, we can also solve its related problem quickly, we can also solve its related decision problem quicklydecision problem quickly
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Problem: the start point
a problem is UNDECIDABLEUNDECIDABLE if there is no algorithm that takes as input an instance of the problem and determines whether the answer to that instance is "yes""yes" or "no""no"
problems that are solvable by polinomial-time solvable by polinomial-time algorithmsalgorithms as being TRACTABLETRACTABLE; ones that require superpolynomial timesuperpolynomial time as being INTRACTABLEINTRACTABLE
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Problem: the start pointNP-complete problems are intractableNP-complete problems are intractable
If any single NP-complete problem can be If any single NP-complete problem can be solved in polynomial time then every NP-solved in polynomial time then every NP-complete problem has a polynomial-time complete problem has a polynomial-time algorithmalgorithm
If you can establish a problem as NP-complete you provide good evidence for its intractabilityits intractability
You would then do better spending your time developing an approximation algorithmdeveloping an approximation algorithm rather than searching for a fast algorithm that solves the problem exactly
Polynomial-time solvable problems are generally Polynomial-time solvable problems are generally regarded as tractableregarded as tractable
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Complexity Class PAn algorithm solves a problem in time An algorithm solves a problem in time O(T(n))O(T(n)) if,
when it is provided a problem instance problem instance ii of of length length n=|i|n=|i|, the algorithm can produce the the algorithm can produce the solution in at most solution in at most O(T(n))O(T(n)) time time
A problem is polynomial-time solvableproblem is polynomial-time solvable if there exists an algorithm to solve it in time O(nan algorithm to solve it in time O(nkk)) for some constant k
COMPLEXITY CLASS PCOMPLEXITY CLASS P: the set of decision problems that are solvable in polynomial time
A function function ff is polynomial-time computable is polynomial-time computable if there exists a polynomial-time algorithm AA that, given any input x x I I, produces as output f(x)f(x)
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Example: problem in P
Minimum Weight Spanning Tree.
Input: Graph G, integer k.
Decision problem: Does G have a spanning tree of Does G have a spanning tree of weight at most k? weight at most k?
If you are provided with a tree with weight at most k as part of the solution, the answer can be verified in O(n2) time.
Polynomial-Time Algorithms
Are some problems solvable in polynomial time?Of course: every algorithm we’ve studied (Algorithms
1) provides polynomial-time solution to some problemWe define P to be the class of problems solvable in
polynomial timeAre all problems solvable in polynomial time?
No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given
Such problems are clearly intractable, not in P
Complexity Class NP
Some problems are intractable: as they grow large, we are unable to solve them in reasonable time
What constitutes reasonable time? Standard working definition: polynomial timeOn an input of size n the worst-case running time
is O(nk) for some constant kPolynomial time: O(n2), O(n3), O(1), O(n lg n) Not in polynomial time: O(2n), O(nn), O(n!)
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Complexity Class NPA decision problemdecision problem (yes/no question) is in the
class NPclass NP if it has a nondeterministic polynomial time algorithm
Informally, such an algorithm:
1.Guesses a solution (nondeterministically).
2.Checks deterministically in polynomial time that the answer is correct.
Or equivalently, when the answer is "yes", there is a certificate (a solution meeting the criteria) that can be verified in polynomial time (deterministically)
The Complexity Class NPComplexity Class NP is the class of problems that can be verified by a polynomial-time can be verified by a polynomial-time algorithmalgorithm
If Q Q PP then Q Q NPNP => P P NPNPOpen question is:Open question is: P = NP P = NP
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Complexity Classes: P and NP
PP is set of problems that can be solved in polynomial time
NP (NNondeterministic PPolynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computerWhat the hell is that?
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Nondeterminism
Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correctIf a solution exists, computer always guesses itOne way to imagine it: a parallel computer that
can freely spawn an infinite number of processesHave one processor work on each possible solutionAll processors attempt to verify that their solution
worksIf a processor finds it has a working solution
So: NP = problems verifiable in polynomial time
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Review: P and NP
Summary so far:P = problems that can be solved in polynomial
timeNP = problems for which a solution can be
verified in polynomial timeUnknown whether P = NP (most suspect not)
Hamiltonian-cycle problem is in NP:Cannot solve in polynomial timeEasy to verify solution in polynomial time (How?)
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Hamiltonian Cycle Problem
A hamiltonian cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V.
A graph that contains a hamiltonian cycle is said to be hamiltonianNOT ALL GRAPHS ARE HAMILTONIAN
Hamiltonian-cycle problem: “Does a graph G have a hamiltonian cycle?”
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Hamiltonian Cycle Problem
Hamiltonian Cycle Hamiltonian Cycle (input: a graph G)Does G have a Hamiltonian cycle?Does G have a Hamiltonian cycle?
Solution:Solution:
0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 15, 6, 17, 18, 19, 12, 13, 3, 415, 6, 17, 18, 19, 12, 13, 3, 4
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
Hamiltonian Cycle Problem
algorithm: lists all permutations of the vertices and check each permutation to see if it is a hamiltonian paththere are m! Possible permutations of the vertices
CHECK if the provided cycle is hamiltonian: if it is a permutation of the vertices of V and each consecutive edges along the cycle exists in the graph
This verification can be implemented to run in O(n2) time => a hamiltonian cycle exists in a graph can be verified in polynomial timeN is the lenght of the econding of G
NP-Complete Problems
The NP-Complete problems are an interesting class of problems whose status is unknown No polynomial-time algorithm has been
discovered for an NP-Complete problemNo suprapolynomial lower bound has been
proved for any NP-Complete problem, eitherWe call this the P = NP question
The biggest open problem in CS
Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt
NP Complete ProblemsThe class of problems in NP which are the problems in NP which are the
“hardest”“hardest” are called the NP-completeNP-complete problemsproblems
A problem QQ in NPNP is NP-completeNP-complete if the existence of a polynomial time algorithm for QQ implies the existence of a polynomial time existence of a polynomial time algorithm for all problems in algorithm for all problems in NPNP
Steve Cook in 1971 proved that SAT is NP-Steve Cook in 1971 proved that SAT is NP-complete. complete.
NP-Complete Problems
We will see that NP-Complete problems are the “hardest” problems in NP:If any one NP-Complete problem can be solved in
polynomial time……then every NP-Complete problem can be solved
in polynomial time……and in fact every problem in NP can be solved
in polynomial time (which would show P = NP)Thus: solve hamiltonian-cycle in O(n100) time,
you’ve proved that P = NP. Retire rich & famous.
Reduction
The crux of NP-Completeness is reducibilityInformally, a problem P can be reduced to
another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of PWhat do you suppose “easily” means?This rephrasing is called transformation
Intuitively: If P reduces to Q, P is “no harder to solve” than Q
Reducibility
An example:P: Given a set of Booleans, is at least one TRUE?Q: Given a set of integers, is their sum positive?Transformation: (x1, x2, …, xn) = (y1, y2, …, yn)
where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE
Another example: Solving linear equations is reducible to solving
quadratic equationsHow can we easily use a quadratic-equation solver to
solve linear equations?
Using Reductions
If P is polynomial-time reducible to Q, we denote this P p Q
Definition of NP-Complete: If P is NP-Complete, P NP and all problems R
are reducible to PFormally: R p P R NP
If P p Q and P is NP-Complete, Q is also NP-Complete
Ex. of some NP-complete problemsGiven one NP-Complete problem, we can
prove many interesting problems NP-CompleteGraph coloring (= register allocation)Hamiltonian cycleHamiltonian pathKnapsack problemTraveling salesmanJob scheduling with penalitiesMany, many more
An Aside: Terminology
What is the difference between a problem and an instance of that problem?
To formalize things, we will express instances of problems as stringsHow can we express a instance of the
hamiltonian cycle problem as a string?To simplify things, we will worry only about
decision problems with a yes/no answerMany problems are optimization problems, but
we can often re-cast those as decision problems
NP-Hard and NP-Complete
If P is polynomial-time reducible to Q, we denote this P p Q
Definition of NP-Hard and NP-Complete: If all problems R NP are reducible to P, then P
is NP-HardWe say P is NP-Complete if P is NP-Hard
and P NPIf P p Q and P is NP-Complete, Q is also
NP- Complete
Why Prove NP-Completeness?
Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable
Therefore it can be important to prove that a problem is NP-CompleteDon’t need to come up with an efficient
algorithmCan instead work on approximation algorithms
Proving NP-Completeness
What steps do we have to take to prove a problem P is NP-Complete?Pick a known NP-Complete problem QReduce Q to P
Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q
Prove the transformation worksProve it runs in polynomial time
Oh yeah, prove P NP (What if you can’t?)
Review: P And NP Summary
P = set of problems that can be solved in polynomial time
NP = set of problems for which a solution can be verified in polynomial time
P NPOpen question: Does P = NP?
Review: Reduction
A problem P can be reduced to another problem Q if any instance of P can be rephrased to an instance of Q, the solution to which provides a solution to the instance of PThis rephrasing is called a transformation
Intuitively: If P reduces in polynomial time to Q, P is “no harder to solve” than Q
Review: NP-Hard and NP-CompleteIf P is polynomial-time reducible to Q, we
denote this P p Q
Definition of NP-Hard and NP-Complete: If all problems R NP are reducible to P, then P
is NP-HardWe say P is NP-Complete if P is NP-Hard
and P NP
If P p Q and P is NP-Complete, Q is alsoNP- Complete
Review: Proving NP-CompletenessWhat steps do we have to take to prove a
problem Q is NP-Complete?Pick a known NP-Complete problem PReduce P to Q
Describe a transformation that maps instances of P to instances of Q, s.t. “yes” for Q = “yes” for P
Prove the transformation worksProve it runs in polynomial time
Oh yeah, prove Q NP (What if you can’t?)
Directed Hamiltonian Cycle Undirected Hamiltonian CycleWhat was the hamiltonian cycle problem
again?For my next trick, I will reduce the directed
hamiltonian cycle problem to the undirected hamiltonian cycle problem before your eyesWhich variant am I proving NP-Complete?
Draw a directed example on the boardWhat transformation do I need to effect?
Transformation:Directed Undirected Ham. Cycle Transform graph G = (V, E) into G’ = (V’, E’):
Every vertex v in V transforms into 3 vertices v1, v2, v3 in V’ with edges (v1,v2) and (v2,v3) in E’
Every directed edge (v, w) in E transforms into the undirected edge (v3, w1) in E’ (draw it)
Can this be implemented in polynomial time?Argue that a directed hamiltonian cycle in G
implies an undirected hamiltonian cycle in G’Argue that an undirected hamiltonian cycle in G’
implies a directed hamiltonian cycle in G
Undirected Hamiltonian Cycle
Thus we can reduce the directed problem to the undirected problem
What’s left to prove the undirected hamiltonian cycle problem NP-Complete?
Argue that the problem is in NP
Hamiltonian Cycle TSP
The well-known traveling salesman problem:Optimization variant: a salesman must travel to n
cities, visiting each city exactly once and finishing where he begins. How to minimize travel time?
Model as complete graph with cost c(i,j) to go from city i to city j
How would we turn this into a decision problem?A: ask if a TSP with cost < k
Hamiltonian Cycle TSP
The steps to prove TSP is NP-Complete:Prove that TSP NP (Argue this)Reduce the undirected hamiltonian cycle
problem to the TSPSo if we had a TSP-solver, we could use it to solve the
hamilitonian cycle problem in polynomial timeHow can we transform an instance of the hamiltonian
cycle problem to an instance of the TSP?Can we do this in polynomial time?
The TSP
Random asides: TSPs (and variants) have enormous practical
importanceE.g., for shipping and freighting companiesLots of research into good approximation algorithms
Recently made famous as a DNA computing problem