90
NP-complete Problems: Search Problems Alexander S. Kulikov Steklov Institute of Mathematics at St. Petersburg Russian Academy of Sciences Advanced Algorithms and Complexity Data Structures and Algorithms

Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Embed Size (px)

Citation preview

Page 1: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

NP-complete Problems:Search Problems

Alexander S. KulikovSteklov Institute of Mathematics at St. Petersburg

Russian Academy of Sciences

Advanced Algorithms and ComplexityData Structures and Algorithms

Page 2: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 3: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Polynomial vs Exponential

running time: n n2 n3 2n

less than 109: 109 104.5 103 29

Page 4: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Improving Brute Force Search

Usually, an efficient (polynomial) algorithmsearches for a solution among an exponentialnumber of candidates:

there are n! permutations of n objects

there are 2n ways to partition n objectsinto two setsthere are nn−2 spanning trees in acomplete graph on n vertices

Page 5: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Improving Brute Force Search

Usually, an efficient (polynomial) algorithmsearches for a solution among an exponentialnumber of candidates:

there are n! permutations of n objectsthere are 2n ways to partition n objectsinto two sets

there are nn−2 spanning trees in acomplete graph on n vertices

Page 6: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Improving Brute Force Search

Usually, an efficient (polynomial) algorithmsearches for a solution among an exponentialnumber of candidates:

there are n! permutations of n objectsthere are 2n ways to partition n objectsinto two setsthere are nn−2 spanning trees in acomplete graph on n vertices

Page 7: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

This module

For thousands of practically importantproblems we don’t have an efficientalgorithm yet

An efficient algorithm for one suchproblem automatically gives efficientalgorithms for all these problems!$1M prize for constructing such analgorithm or proving that this isimpossible!

Page 8: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

This module

For thousands of practically importantproblems we don’t have an efficientalgorithm yetAn efficient algorithm for one suchproblem automatically gives efficientalgorithms for all these problems!

$1M prize for constructing such analgorithm or proving that this isimpossible!

Page 9: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

This module

For thousands of practically importantproblems we don’t have an efficientalgorithm yetAn efficient algorithm for one suchproblem automatically gives efficientalgorithms for all these problems!$1M prize for constructing such analgorithm or proving that this isimpossible!

Page 10: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 11: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Boolean Formulas

Formula in conjunctive normal form

(x ∨ y ∨ z)(x ∨ y)(y ∨ z)(z ∨ x)(x ∨ y ∨ z)

x , y , z are Boolean variables (values:true/false or 1/0)literals are variables (x , y , z) and theirnegations (x , y , z)clauses are disjunctions (logical or) ofliterals

Page 12: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Satisfiability (SAT)

Input: Formula F in conjunctive normalform (CNF).

Output: An assignment of Boolean values tothe variables of F satisfying allclauses, if exists.

Page 13: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Examples

The formula (x ∨ y)(x ∨ y)(x ∨ y) issatisfiable: set x = 1, y = 0.The formula (x ∨ y ∨ z)(x ∨ y)(y ∨ z) issatisfiable: set x = 1, y = 1, z = 1 orx = 1, y = 0, z = 0.The formula(x∨y ∨z)(x∨y)(y ∨z)(z∨x)(x∨y ∨z)

is unsatisfiable.

Page 14: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Satisfiability

Classical hard problemMany applications: e.g.,hardware/software verification, planning,schedulingMany hard problems are stated in termsof SAT naturallySAT solvers (will see later), SATcompetition

Page 15: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

SAT is a typical search problemSearch problem: given an instance I ,find a solution S or report that noneexistsMain property: one must be able tocheck quickly whether S is indeed asolution for IBy saying quickly, we mean in timepolynomial in the length of I . Inparticular, the length of S should bepolynomial in the length of I

Page 16: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

DefinitionA search problem is defined by analgorithm 𝒞 that takes an instance I anda candidate solution S , and runs in timepolynomial in the length of I . We say that Sis a solution to I iff 𝒞(S , I ) = true.

Page 17: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

For SAT, I is a Boolean formula, S is anassignment of Boolean constants to itsvariables. The corresponding algorithm 𝒞checks whether S satisfies all clauses of I .

Page 18: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Next part

A few practical search problems for whichpolynomial algorithms remain unknown

Page 19: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 20: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 21: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Traveling salesman problem (TSP)

Input: Pairwise distances between n citiesand a budget b.

Output: A cycle that visits each vertexexactly once and has total length atmost b.

Page 22: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Delivery Company

https://simple.wikipedia.org/wiki/Travelling_salesman_problem

Page 23: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Drilling Holes in a Circuit Board

https://developers.google.com/optimization/routing/tsp

Page 24: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

1

2

5

3

4

5

2

4

2

2

1

13

33

Page 25: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

1

2

5

3

4

5

2

4

2

2

1

13

33

length: 15

Page 26: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

1

2

5

3

4

5

2

4

2

2

1

13

33

length: 13

Page 27: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

1

2

5

3

4

5

2

4

2

2

1

13

33

length: 9

Page 28: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Search Problem

TSP is a search problem: given asequence of vertices, it is easy to checkwhether it is a cycle visiting all thevertices of total length at most bTSP is usually stated as an optimizationproblem; we stated its decision versionto guarantee that a candidate solutioncan be efficiently checked for correctness

Page 29: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Algorithms

Check all permutations: about O(n!),extremely slowDynamic programming: O(n22n)No significantly better upper bound isknownThere are heuristic algorithms andapproximation algorithms

Page 30: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Comparing to MSTMSTDecision version:given n cities, connectthem by (n − 1) roadsof minimal totallength

Can be solvedefficiently

TSPDecision version:given n cities, connectthem in a path by(n − 1) roads ofminimal total length

No polynomialalgorithm known!

Page 31: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Comparing to MSTMSTDecision version:given n cities, connectthem by (n − 1) roadsof minimal totallength

Can be solvedefficiently

TSPDecision version:given n cities, connectthem in a path by(n − 1) roads ofminimal total length

No polynomialalgorithm known!

Page 32: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Comparing to MSTMSTDecision version:given n cities, connectthem by (n − 1) roadsof minimal totallength

Can be solvedefficiently

TSPDecision version:given n cities, connectthem in a path by(n − 1) roads ofminimal total length

No polynomialalgorithm known!

Page 33: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Comparing to MSTMSTDecision version:given n cities, connectthem by (n − 1) roadsof minimal totallength

Can be solvedefficiently

TSPDecision version:given n cities, connectthem in a path by(n − 1) roads ofminimal total length

No polynomialalgorithm known!

Page 34: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 35: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Hamiltonian cycle

Input: A graph.Output: A cycle that visits each vertex of

the graph exactly once.

Page 36: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 37: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 38: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Input: A graph.Output: A cycle that visits each edge of the

graph exactly once.

TheoremA graph has an Eulerian cycle if and only if itis connected and the degree of each vertex iseven.

Page 39: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Input: A graph.Output: A cycle that visits each edge of the

graph exactly once.

TheoremA graph has an Eulerian cycle if and only if itis connected and the degree of each vertex iseven.

Page 40: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Non-Eulerian graph

Eulerian graph

Page 41: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Non-Eulerian graph Eulerian graph

Page 42: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Find a cycle visitingeach edge exactlyonce

Can be solvedefficiently

Hamiltonian cycle

Find a cycle visitingeach vertex exactlyonce

No polynomialalgorithm known!

Page 43: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Find a cycle visitingeach edge exactlyonce

Can be solvedefficiently

Hamiltonian cycle

Find a cycle visitingeach vertex exactlyonce

No polynomialalgorithm known!

Page 44: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Find a cycle visitingeach edge exactlyonce

Can be solvedefficiently

Hamiltonian cycle

Find a cycle visitingeach vertex exactlyonce

No polynomialalgorithm known!

Page 45: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Eulerian cycle

Find a cycle visitingeach edge exactlyonce

Can be solvedefficiently

Hamiltonian cycle

Find a cycle visitingeach vertex exactlyonce

No polynomialalgorithm known!

Page 46: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 47: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Longest path

Input: A weighted graph, two vertices s, t,and a budget b.

Output: A simple path (containing norepeated vertices) of total length atleast b.

Page 48: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 49: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 50: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 51: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 52: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Shortest path

Find a simple pathfrom s to t of totallength at most b

Can be solvedefficiently

Longest path

Find a simple pathfrom s to t of totallength at least b

No polynomialalgorithm known!

Page 53: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Shortest path

Find a simple pathfrom s to t of totallength at most b

Can be solvedefficiently

Longest path

Find a simple pathfrom s to t of totallength at least b

No polynomialalgorithm known!

Page 54: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Shortest path

Find a simple pathfrom s to t of totallength at most b

Can be solvedefficiently

Longest path

Find a simple pathfrom s to t of totallength at least b

No polynomialalgorithm known!

Page 55: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Shortest path

Find a simple pathfrom s to t of totallength at most b

Can be solvedefficiently

Longest path

Find a simple pathfrom s to t of totallength at least b

No polynomialalgorithm known!

Page 56: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 57: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Integer linear programming

Input: A set of linear inequalities Ax ≤ b.Output: Integer solution.

Page 58: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

x1 ≥ 0.5−x1 + 8x2 ≥ 0−x1 − 8x2 ≥ −8

Page 59: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

x1 ≥ 0.5−x1 + 8x2 ≥ 0−x1 − 8x2 ≥ −8 x1

x2

Page 60: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

x1 ≥ 0.5−x1 + 8x2 ≥ 0−x1 − 8x2 ≥ −8 x1

x2

Page 61: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

x1 ≥ 0.5−x1 + 8x2 ≥ 0−x1 − 8x2 ≥ −8 x1

x2

Page 62: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

x1 ≥ 0.5−x1 + 8x2 ≥ 0−x1 − 8x2 ≥ −8 x1

x2

Page 63: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

LP(decision version)

Find a realsolution of a system oflinear inequalities

Can be solvedefficiently

ILP

Find an integersolution of a system oflinear inequalities

No polynomialalgorithm known!

Page 64: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

LP(decision version)

Find a realsolution of a system oflinear inequalities

Can be solvedefficiently

ILP

Find an integersolution of a system oflinear inequalities

No polynomialalgorithm known!

Page 65: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

LP(decision version)

Find a realsolution of a system oflinear inequalities

Can be solvedefficiently

ILP

Find an integersolution of a system oflinear inequalities

No polynomialalgorithm known!

Page 66: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

LP(decision version)

Find a realsolution of a system oflinear inequalities

Can be solvedefficiently

ILP

Find an integersolution of a system oflinear inequalities

No polynomialalgorithm known!

Page 67: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 68: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent set

Input: A graph and a budget b.Output: A subset of vertices of size at least

b such that no two of them areadjacent.

Page 69: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 70: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Example

Page 71: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent Sets in a Tree

A maximal independent set in a tree can befound by a simple greedy algorithm: it is safeto take into a solution all the leaves.

Page 72: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent set ina tree

Find an independentset of size at least b ina given tree

Can be solvedefficiently

Independent set ina graph

Find an independentset of size at least b ina given graph

No polynomialalgorithm known!

Page 73: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent set ina tree

Find an independentset of size at least b ina given tree

Can be solvedefficiently

Independent set ina graph

Find an independentset of size at least b ina given graph

No polynomialalgorithm known!

Page 74: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent set ina tree

Find an independentset of size at least b ina given tree

Can be solvedefficiently

Independent set ina graph

Find an independentset of size at least b ina given graph

No polynomialalgorithm known!

Page 75: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Independent set ina tree

Find an independentset of size at least b ina given tree

Can be solvedefficiently

Independent set ina graph

Find an independentset of size at least b ina given graph

No polynomialalgorithm known!

Page 76: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Next part

It turns out that all these hard problems arein a sense a single hard problem:a polynomial time algorithm for any of theseproblems can be used to solve all of them inpolynomial time!

Page 77: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Outline1 Brute Force Search

2 Search Problems

3 Easy and Hard ProblemsTraveling Salesman ProblemHamiltonian Cycle ProblemLongest Path ProblemInteger Linear Programming ProblemIndependent Set Problem

4 P and NP

Page 78: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class NPDefinitionA search problem is defined by analgorithm 𝒞 that takes an instance I anda candidate solution S , and runs in timepolynomial in the length of I . We say that Sis a solution to I iff 𝒞(S , I ) = true.

DefinitionNP is the class of all search problems.

Page 79: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class NPDefinitionA search problem is defined by analgorithm 𝒞 that takes an instance I anda candidate solution S , and runs in timepolynomial in the length of I . We say that Sis a solution to I iff 𝒞(S , I ) = true.

DefinitionNP is the class of all search problems.

Page 80: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

NP stands for “non-deterministicpolynomial time”: one can guess asolution, and then verify its correctnessin polynomial timeIn other words, the class NP containsall problems whose solutions can beefficiently verified

Page 81: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class P

DefinitionP is the class of all search problems that canbe solved in polynomial time.

Page 82: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class PProblems whosesolution can befound efficiently

MSTShortest pathLPIS on trees

Class NPProblems whosesolution can beverified efficiently

TSPLongest pathILPIS on graphs

Page 83: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class PProblems whosesolution can befound efficiently

MSTShortest pathLPIS on trees

Class NPProblems whosesolution can beverified efficiently

TSPLongest pathILPIS on graphs

Page 84: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class PProblems whosesolution can befound efficiently

MSTShortest pathLPIS on trees

Class NPProblems whosesolution can beverified efficiently

TSPLongest pathILPIS on graphs

Page 85: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Class PProblems whosesolution can befound efficiently

MSTShortest pathLPIS on trees

Class NPProblems whosesolution can beverified efficiently

TSPLongest pathILPIS on graphs

Page 86: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

The main open problem in ComputerScience

Is P equal to NP?

Millenium Prize ProblemClay Mathematics Institute: $1M prize forsolving the problem

Page 87: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

The main open problem in ComputerScience

Is P equal to NP?

Millenium Prize ProblemClay Mathematics Institute: $1M prize forsolving the problem

Page 88: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

If P=NP, then all search problems canbe solved in polynomial time.

If P̸=NP, then there exist searchproblems that cannot be solved inpolynomial time.

Page 89: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

If P=NP, then all search problems canbe solved in polynomial time.If P̸=NP, then there exist searchproblems that cannot be solved inpolynomial time.

Page 90: Алгоритмы для NP-трудных задач, осень 2016: Задачи поиска

Next part

We’ll show that the satisfiability problem, thetraveling salesman problem, the independentset problem, the integer linear programmingare the hardest problems in NP.