8/11/2019 Lecture-35-CS210-2012.pptx

1/37

Data Structures and Algorithms

(CS210/ESO207/ESO211)

Lecture 35

Greedy strategy: a new algorithm paradigm

PartIV

1

8/11/2019 Lecture-35-CS210-2012.pptx

2/37

Overview of todays lecture

Reviewtwo algorithms for MST (from last class)

Exploreways to improve their running times

Make clever use of simple data structures

2

8/11/2019 Lecture-35-CS210-2012.pptx

3/37

A useful lessonfrom data structures(hopefully you learnt it by now)

Question:When do you feel the needof using a data structure for solving a problem?

Answer:

Whenever we have an algorithm which performs a large number of operations of same type.

In such a situation, it is worth building a data structure which can perform these operations

efficiently.

3

8/11/2019 Lecture-35-CS210-2012.pptx

4/37

MST - Problem Description

Input:an undirected graph G=(V,E) with : E,

Aim: compute a spanning tree (V,E), E E such that ()

E isminimum.

4

8/11/2019 Lecture-35-CS210-2012.pptx

5/37

Two graph theoretic properties of MST

Cut property

Cycle property

5

Every algorithm till date is based onone of these properties!

8/11/2019 Lecture-35-CS210-2012.pptx

6/37

Cut Property

6

8/11/2019 Lecture-35-CS210-2012.pptx

7/37

Cut Property

Definition: For any subset, such that , cut(,)is defined as:

cut(,)= { (,) | and or and }

Cut-property: The least weight edge of a cut(,) must be in MST.

7

cut(,)

23

13

37

48

67

u

v

8/11/2019 Lecture-35-CS210-2012.pptx

8/37

Cycle Property

8

8/11/2019 Lecture-35-CS210-2012.pptx

9/37

Cycle Property

Let be any cycle in .

Cycle-property:Maximum weight edge of any cycle can not be present in MST.

9

u v

a

b

c

f

r

s

43

24

12

1731

36

19

26

8/11/2019 Lecture-35-CS210-2012.pptx

10/37

Algorithms based on cut Property

10

8/11/2019 Lecture-35-CS210-2012.pptx

11/37

Pick a vertex and start growing MST from it?

11

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

8/11/2019 Lecture-35-CS210-2012.pptx

12/37

Pick a vertex and start growing MST from it?

12

170

33

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

35

w

8/11/2019 Lecture-35-CS210-2012.pptx

13/37

Pick a vertex and start growing MST from it?

13

170

33

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

35

w

x

8/11/2019 Lecture-35-CS210-2012.pptx

14/37

Pick a vertex and start growing MST from it?

14

170

33

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

35

w

x

y

8/11/2019 Lecture-35-CS210-2012.pptx

15/37

Pick a vertex and start growing MST from it?

15

170

33

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

35

w

x

y

8/11/2019 Lecture-35-CS210-2012.pptx

16/37

Pick a vertex and start growing MST from it?

Finally

16

170

33

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6324

53

44

29

64

42

43

u

v70

35

w

x

y

8/11/2019 Lecture-35-CS210-2012.pptx

17/37

An Algorithm based on cut property(discussed in previous class)

Algorithm (Input:graph =(,) with weightson edges)

;

{u};

While ( ?? ) do

{ Compute the least weight edge from cut(,);

Let this edge be (x,y), with x, y;

{(x,y)};

{y};}

Return;

Number of iterations of the While loop: ??

Time spent in one iteration of While loop: ??

Running timeof the algorithm: O()

17

O()

To improve the running time of the algo,

we need to have a closer look at the

computation during an iteration.

8/11/2019 Lecture-35-CS210-2012.pptx

18/37

Analyzinga single iteration ofwhile loop

Let us maintain a data structure for .

What operations to perform on it ?

18

31

45

28

33

71

27

1311

9

a

b

c

d

r

t

69

u

v

y

x

z

Extract-min Observation:Forvertices of set (),

we need to keep onlythe least weight edge

incident from.

The black edgesconstitute cut(,).

Let be the set of

vertices that lie outsideof but have at least

one neighbor in.

In this picture ={u,v,z,y,x}

8/11/2019 Lecture-35-CS210-2012.pptx

19/37

Analyzinga single iteration ofwhile loop

Let us maintain a data structure for .

What operations to perform on it ?

19

31

33

11

9

a

b

c

d

r

t

69

u

v

y

x

z

27

44

56

Extract-min

8/11/2019 Lecture-35-CS210-2012.pptx

20/37

Analyzinga single iteration ofwhile loop

Let us maintain a data structure for .

What operations to perform on it ?

20

31

33

11

9

a

b

c

d

r

t

69

u

v

y

x

z

27

44

56

Extract-min

Insert

8/11/2019 Lecture-35-CS210-2012.pptx

21/37

Analyzinga single iteration ofwhile loop

Let us maintain a data structure for .

What operations to perform on it ?

21

31

33

11

9

a

b

c

d

r

t

u

v

y

x

z

27

56

Insert

Extract-min

Decrease-key

8/11/2019 Lecture-35-CS210-2012.pptx

22/37

Overview of an efficient implementation

Keep a min-heap storing .

For each v (), there will be one entry in storing the following information:

Vertex v,

the edge (as well as its weight) of least weight incident on v from.

For example, if y (), and (x,y) is the edge of least weight incident on y from

, and its weight is 55. Then its entry in will be (x,y,55). Key associated with ywill be 55.

22

8/11/2019 Lecture-35-CS210-2012.pptx

23/37

An efficient Algorithm based on cut property

Algorithm (Input:graph =(,) with : )

;

For each v do[] false;

[u] true;

Createa min-heap;

For each (u,v)do Insert( (v,u,(u,v)),);

For( = to ) do

{ (y,x,(x,y)) Extract_min();

{(x,y)};[y] true;

For each(y,z) and[z]= false

If( ) ??

Else ??

}

Return;

23

Insert((z,y, (y,z),)

Decrease_key((z,y, (y,z),)

O(logn)

O(logn)

O(logn)

O(logn)

If current value of keyof zis greater

than(y,z), we decreaseit to(y,z).

8/11/2019 Lecture-35-CS210-2012.pptx

24/37

Time complexity analysis

Question: During th iteration, if vertex y get added to , how much

computation is carried out during th iteration ?

Answer: (number of edges incident on y)

O(log n)

Time complexity of the algorithm = O(mlog n) time.

This algorithm is called Prims algorithm for MST.

Theorem: MST ofa weighted graph can be computed in O(mlog n) time.

24

8/11/2019 Lecture-35-CS210-2012.pptx

25/37

Algorithm based on cycle Property

25

8/11/2019 Lecture-35-CS210-2012.pptx

26/37

An Algorithm based on cycle property(discussed in the last class)

Algorithm (Input:graph =(,) with weightson edges)

While ( ?? ) do

{ Compute any cycle ;

Let (u,v) be the maximum weightedge of the cycle ;

Remove (u,v) from ;}

Return;

Number of iterations of the While loop: ??

Time spent in one iteration of While loop: ??

Running timeof the algorithm: O()

26

has any cycle

+

O()

8/11/2019 Lecture-35-CS210-2012.pptx

27/37

We shall now discuss an efficient algorithm for MSTbased on

cycle property. The algorithm is called Kruskalsalgorithm

(named after its inventor).

27

8/11/2019 Lecture-35-CS210-2012.pptx

28/37

An Algorithm based on cycle propertyKruskals algorithm

Algorithm (Input:graph =(,) with weightson edges)

Sort()in the increasing order of weights;

Let {,,,} be the sequence of edges in increasing order of weights;

;

For( = to ) do{ If( ?? )

{};}

Return;

Correctness:The result will surely be a spanning tree. So to prove correctness, wejust need to justify discarding of the edges which are not added to .

For this purpose, we execute the algorithm on an example.

28

does not create a cycle upon addition to

8/11/2019 Lecture-35-CS210-2012.pptx

29/37

Executing the algorithm on an example

29

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6344

53

44

29

64

42

43

v70

a

bc

d

u

vx

s

r

t

y

hk

p

8/11/2019 Lecture-35-CS210-2012.pptx

30/37

Executing the algorithm on an example

30

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6344

53

44

29

64

42

43

v70

a

bc

d

u

vx

s

r

t

y

hk

p

8/11/2019 Lecture-35-CS210-2012.pptx

31/37

Executing the algorithm on an example

31

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6344

53

44

29

64

42

43

v70

a

bc

d

u

vx

s

r

t

y

hk

p

8/11/2019 Lecture-35-CS210-2012.pptx

32/37

Executing the algorithm on an example

32

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6344

53

44

29

64

42

43

v70

a

bc

d

u

vx

s

r

t

y

hk

p

8/11/2019 Lecture-35-CS210-2012.pptx

33/37

Executing the algorithm on an example

33

170

33

35

31 41 81

52

42

50

102

50 3060

70

54

57

49

49

80

78

6344

53

44

29

64

42

43

v70

a

bc

d

u

vx

s

r

t

y

hk

p

The algorithm wont add

edge (t,h).

Can you see the reason?

8/11/2019 Lecture-35-CS210-2012.pptx

34/37

Analyzing an iteration of the Kruskalsalgorithm

The edge (u,v) must be the heaviest edge in a cycle.

So using Cycle-property, edge (u,v) can not be present in MST.

Hence every edge discarded during the algorithm is surely not present in MST.

34

v

u

8/11/2019 Lecture-35-CS210-2012.pptx

35/37

KruskalsalgorithmTime complexity analysis

Algorithm (Input:graph =(,) with weightson edges)

Sort()in the increasing order of weights;

Let {,,,} be the sequence of edges in increasing order of weights;

;

For( = to ) do

{ If( ?? )

{};}

Return;

Time spent in Sort(): O( )Number of iterations of For loop:

Time spent in one iteration of Forloop: ??

Running timeof the algorithm: O( )

35

does not create a cycle upon addition to

O(log)

Make use of Set-union

data structure discussed

in Lecture 29 (held on

19thOctober)

8/11/2019 Lecture-35-CS210-2012.pptx

36/37

Homework for those who aim for A* or whose

expectation from the course is more than any grade

(to be submitted during Lecture class on 16thNovember.)

Give precise and concise description as well as analysis of Kruskalsalgorithm based on

the set-uniondata structure.

Note:Doing this homework is necessary (but not sufficient) for a person to get A*.

36

8/11/2019 Lecture-35-CS210-2012.pptx

37/37

Concluding remark on MST for all students

This is one of the most researched problem of computer science. However,there does not exist, till date, an O(+ ) time deterministic algorithm for

this problem. But there exists a randomized algorithm whose average running

time isO(+ ).

37