CSC401 – Analysis of AlgorithmsCSC401 – Analysis of Algorithms Lecture Notes 4 Lecture Notes 4

Trees and Priority QueuesTrees and Priority Queues

Objectives:Objectives:

General Trees and ADTGeneral Trees and ADTProperties of TreesProperties of TreesTree TraversalsTree TraversalsBinary TreesBinary TreesPriority Queues and ADTPriority Queues and ADT

22

The Tree StructureThe Tree Structure

In computer science, a In computer science, a tree is an abstract model tree is an abstract model of a hierarchical of a hierarchical structurestructure

A tree consists of nodes A tree consists of nodes with a parent-child with a parent-child relationrelation

Applications:Applications:– Organization chartsOrganization charts– File systemsFile systems– Programming Programming

environmentsenvironments

Computers”R”Us

Sales R&DManufacturing

Laptops DesktopsUS International

Europe Asia Canada

33

subtree

Tree TerminologyTree TerminologyRoot: node without parent (A)Root: node without parent (A)Internal node: node with at least Internal node: node with at least one child (A, B, C, F)one child (A, B, C, F)External node (a.k.a. leaf ): node External node (a.k.a. leaf ): node without children (E, I, J, K, G, H, D)without children (E, I, J, K, G, H, D)Ancestors of a node: parent, Ancestors of a node: parent, grandparent, grand-grandparent, grandparent, grand-grandparent, etc.etc.Depth of a node: number of Depth of a node: number of ancestorsancestorsHeight of a tree: maximum depth Height of a tree: maximum depth of any node (3)of any node (3)Descendant of a node: child, Descendant of a node: child, grandchild, grand-grandchild, etc.grandchild, grand-grandchild, etc.

A

B DC

G HE F

I J K

Subtree: tree consisting Subtree: tree consisting of a node and its of a node and its descendantsdescendants

44

Tree ADTTree ADTWe use positions to We use positions to abstract nodesabstract nodes

Generic methods:Generic methods:– integer integer sizesize()()– boolean boolean isEmptyisEmpty()()– objectIterator objectIterator elementselements()()– positionIterator positionIterator positionspositions()()

Accessor methods:Accessor methods:– position position rootroot()()– position position parentparent(p)(p)– positionIterator positionIterator childrenchildren(p)(p)

Query methods:Query methods:– boolean boolean isInternalisInternal(p)(p)– boolean boolean isExternalisExternal(p)(p)– boolean boolean isRootisRoot(p)(p)

Update methods:Update methods:– swapElementsswapElements(p, q)(p, q)– object object replaceElementreplaceElement(p, o)(p, o)

Additional update methods Additional update methods may be defined by data may be defined by data structures implementing the structures implementing the Tree ADTTree ADT

55

Depth and HeightDepth and HeightDepth Depth -- the depth of v is -- the depth of v is the number of ancestors, the number of ancestors, excluding v itselfexcluding v itself– the depth of the root is 0the depth of the root is 0– the depth of v other than the the depth of v other than the

root is one plus the depth of root is one plus the depth of its parentits parent

– time efficiency is O(1+d)time efficiency is O(1+d)

Height Height -- the height of a -- the height of a subtree v is the maximum subtree v is the maximum depth of its external nodesdepth of its external nodes– the height of an external the height of an external

node is 0node is 0– the height of an internal the height of an internal

node v is one plus the node v is one plus the maximum height of its maximum height of its childrenchildren

– time efficiency is O(n)time efficiency is O(n)

Algorithm depth(T,v)if T.isRoot(v) then

return 0else return

1+depth(T, T.parent(v))

Algorithm height(T,v)if T.isExternal(v) then

return 0else

h=0;for each

wT.children(v) do h=max(h,

height(T,w)) return 1+h

66

Preorder TraversalPreorder TraversalA traversal visits the nodes of a A traversal visits the nodes of a tree in a systematic mannertree in a systematic manner

In a preorder traversal, a node is In a preorder traversal, a node is visited before its descendants visited before its descendants

The running time is O(n)The running time is O(n)

Application: print a structured Application: print a structured documentdocument

Make Money Fast!

1. Motivations References2. Methods

2.1 StockFraud

2.2 PonziScheme

1.1 Greed 1.2 Avidity2.3 BankRobbery

1

2

3

5

4 6 7 8

9

Algorithm preOrder(v)visit(v)for each child w of v

preorder (w)

77

Postorder TraversalPostorder TraversalIn a postorder traversal, a In a postorder traversal, a node is visited after its node is visited after its descendantsdescendants

The running time is O(n)The running time is O(n)

Application: compute space Application: compute space used by files in a directory and used by files in a directory and its subdirectoriesits subdirectories

Algorithm postOrder(v)for each child w of v

postOrder (w)visit(v)

cs16/

homeworks/todo.txt

1Kprograms/

DDR.java10K

Stocks.java25K

h1c.doc3K

h1nc.doc2K

Robot.java20K

9

3

1

7

2 4 5 6

8

88

Binary TreeBinary TreeA binary tree is a tree with the A binary tree is a tree with the following properties:following properties:– Each internal node has two Each internal node has two

childrenchildren– The children of a node are an The children of a node are an

ordered pairordered pair

We call the children of an We call the children of an internal node left child and internal node left child and right childright childAlternative recursive Alternative recursive definition: a binary tree is definition: a binary tree is eithereither– a tree consisting of a single a tree consisting of a single

node, ornode, or– a tree whose root has an ordered a tree whose root has an ordered

pair of children, each of which is pair of children, each of which is a binary treea binary tree

Applications:Applications:– arithmetic arithmetic

expressionsexpressions– decision processesdecision processes– searchingsearching

A

B C

F GD E

H I

99

Binary Tree ExamplesBinary Tree ExamplesArithmetic expression Arithmetic expression binary treebinary tree– internal nodes: operatorsinternal nodes: operators– external nodes: operandsexternal nodes: operands– Example: arithmetic Example: arithmetic

expression tree for the expression tree for the expression (2expression (2((aa1)1)(3 (3 b))b))

2

a 1

3 b

Decision treeDecision tree– internal nodes: questions with yes/no answerinternal nodes: questions with yes/no answer– external nodes: decisionsexternal nodes: decisions– Example: dining decisionExample: dining decision

Want a fast meal?

How about coffee? On expense account?

Starbucks Spike’s Al Forno Café Paragon

Yes No

Yes No Yes No

1010

Properties of Binary TreesProperties of Binary TreesNotationNotationnn number of nodesnumber of nodes

ee number of external number of external nodesnodes

ii number of internal number of internal nodesnodes

hh heightheight

Properties:Properties:– e e i i 11– n n 22e e 11– h h ii– h h ((n n 1)1)22– h+h+11 e e 22hh

– h h loglog22 ee– h h loglog22 ( (n n 1)1) 11

1111

BinaryTree ADTBinaryTree ADTThe BinaryTree ADT extends the Tree The BinaryTree ADT extends the Tree ADT, i.e., it inherits all the methods of ADT, i.e., it inherits all the methods of the Tree ADTthe Tree ADT

Additional methods:Additional methods:– position position leftChildleftChild(p)(p)– position position rightChildrightChild(p)(p)– position position siblingsibling(p)(p)

Update methods may be defined by data Update methods may be defined by data structures implementing the BinaryTree structures implementing the BinaryTree ADTADT

1212

Inorder TraversalInorder TraversalIn an inorder traversal a In an inorder traversal a node is visited after its left node is visited after its left subtree and before its subtree and before its right subtreeright subtreeTime efficiency is O(n)Time efficiency is O(n)Application: draw a binary Application: draw a binary treetree– x(v) = inorder rank of vx(v) = inorder rank of v– y(v) = depth of vy(v) = depth of v

Algorithm inOrder(v)if isInternal (v)

inOrder (leftChild (v))visit(v)if isInternal (v)

inOrder (rightChild (v))

3

1

2

5

6

7 9

8

4

1313

Print Arithmetic ExpressionsPrint Arithmetic ExpressionsSpecialization of an inorder Specialization of an inorder traversaltraversal– print operand or operator print operand or operator

when visiting nodewhen visiting node– print “(“ before traversing print “(“ before traversing

left subtreeleft subtree– print “)“ after traversing print “)“ after traversing

right subtreeright subtree

Algorithm printExpression(v)if isInternal (v)

print(“(’’)inOrder (leftChild (v))

print(v.element ())if isInternal (v)

inOrder (rightChild (v))print (“)’’)

2

a 1

3 b((2 (a 1)) (3 b))

1414

Evaluate Arithmetic ExpressionsEvaluate Arithmetic ExpressionsSpecialization of a Specialization of a postorder traversalpostorder traversal– recursive method recursive method

returning the value of a returning the value of a subtreesubtree

– when visiting an internal when visiting an internal node, combine the node, combine the values of the subtreesvalues of the subtrees

Algorithm evalExpr(v)if isExternal (v)

return v.element ()else

x evalExpr(leftChild (v))

y evalExpr(rightChild (v))

operator stored at vreturn x y

2

5 1

3 2

1515

Euler Tour TraversalEuler Tour TraversalGeneric traversal of a binary treeGeneric traversal of a binary tree

Includes a special cases the preorder, postorder and inorder Includes a special cases the preorder, postorder and inorder traversalstraversals

Walk around the tree and visit each node three times:Walk around the tree and visit each node three times:– on the left (preorder)on the left (preorder)– from below (inorder)from below (inorder)– on the right (postorder)on the right (postorder)

2

5 1

3 2

LB

R

1616

Template Method PatternTemplate Method PatternGeneric algorithm that Generic algorithm that can be specialized by can be specialized by redefining certain stepsredefining certain stepsImplemented by means Implemented by means of an abstract Java class of an abstract Java class Visit methods that can Visit methods that can be redefined by be redefined by subclassessubclassesTemplate method Template method eulerToureulerTour– Recursively called on Recursively called on

the left and right the left and right childrenchildren

– A A ResultResult object with fields object with fields leftResultleftResult,, rightResult rightResult andand finalResultfinalResult keeps track of keeps track of the output of the the output of the recursive calls to recursive calls to eulerToureulerTour

public abstract class EulerTour {protected BinaryTree tree;protected void visitExternal(Position p, Result r) { }protected void visitLeft(Position p, Result r) { }protected void visitBelow(Position p, Result r) { }

protected void visitRight(Position p, Result r) { } protected Object eulerTour(Position p) {

Result r = new Result();if tree.isExternal(p) { visitExternal(p, r); }

else {visitLeft(p, r);r.leftResult = eulerTour(tree.leftChild(p));visitBelow(p, r);r.rightResult = eulerTour(tree.rightChild(p));visitRight(p, r);return r.finalResult;

} …

1717

Specializations of EulerTourSpecializations of EulerTourWe show how to We show how to specialize class specialize class EulerTour to evaluate EulerTour to evaluate an arithmetic an arithmetic expressionexpression

AssumptionsAssumptions– External nodes store External nodes store

Integer objectsInteger objects– Internal nodes store Internal nodes store

OperatorOperator objects objects supporting methodsupporting method

operation operation (Integer, Integer) (Integer, Integer)

public class EvaluateExpressionextends EulerTour {

protected void visitExternal(Position p, Result r) {r.finalResult = (Integer) p.element();

}

protected void visitRight(Position p, Result r) {Operator op = (Operator) p.element();r.finalResult = op.operation(

(Integer) r.leftResult,(Integer) r.rightResult);

}

…

}

1818

Data Structure for TreesData Structure for TreesA node is represented A node is represented by an object storingby an object storing– ElementElement– Parent nodeParent node– Sequence of children Sequence of children

nodesnodes

Node objects Node objects implement the Position implement the Position ADTADT

B

DA

C E

F

B

A D F

C

E

1919

Data Structure for Binary TreesData Structure for Binary TreesA node is represented A node is represented by an object storingby an object storing– ElementElement– Parent nodeParent node– Left child nodeLeft child node– Right child nodeRight child node

Node objects implement Node objects implement the Position ADTthe Position ADT

B

DA

C E

B

A D

C E

2020

Vector-Based Binary TreeVector-Based Binary TreeLevel numbering of nodes of T: p(v)Level numbering of nodes of T: p(v)– if v is the root of T, p(v)=1if v is the root of T, p(v)=1– if v is the left child of u, p(v)=2p(u)if v is the left child of u, p(v)=2p(u)– if v is the right child of u, p(v)=2p(u)+1if v is the right child of u, p(v)=2p(u)+1

Vector S storing the nodes of T by putting Vector S storing the nodes of T by putting the root at the second position and the root at the second position and following the above level numbering following the above level numbering

Properties: Properties: Let n be the number of nodes of T, N Let n be the number of nodes of T, N be the size of the vector S, and PM be the be the size of the vector S, and PM be the maximum value of p(v) over all the nodes of Tmaximum value of p(v) over all the nodes of T– N=PM+1N=PM+1– N=2^((n+1)/2)N=2^((n+1)/2)

2121

Java ImplementationJava ImplementationTree interfaceTree interface

BinaryTree interface BinaryTree interface extending Treeextending Tree

Classes implementing Classes implementing Tree and BinaryTree Tree and BinaryTree and providingand providing1.1. ConstructorsConstructors

2.2. Update methodsUpdate methods

3.3. Print methodsPrint methods

Examples of updates Examples of updates for binary treesfor binary trees– expandExternalexpandExternal((vv))– removeAboveExternalremoveAboveExternal((ww))

A

expandExternal(v)

A

CB

B

removeAboveExternal(w)

Av v

w

2222

Trees in JDSLTrees in JDSLJDSL is the Library of Data JDSL is the Library of Data Structures in JavaStructures in Java

Tree interfaces in JDSLTree interfaces in JDSL– InspectableBinaryTreeInspectableBinaryTree– InspectableTreeInspectableTree– BinaryTreeBinaryTree– TreeTree

Inspectable versions of the Inspectable versions of the interfaces do not have interfaces do not have update methodsupdate methods

Tree classes in JDSLTree classes in JDSL– NodeBinaryTreeNodeBinaryTree– NodeTreeNodeTree

JDSL was developed at JDSL was developed at Brown’s Center for Brown’s Center for Geometric ComputingGeometric Computing

See the JDSL See the JDSL documentation and documentation and tutorials at tutorials at http://jdsl.orghttp://jdsl.org

InspectableTree

InspectableBinaryTree

Tree

BinaryTree

2323

Priority Queue ADTPriority Queue ADTA priority queue stores A priority queue stores a collection of itemsa collection of items

An item is a pairAn item is a pair(key, element)(key, element)

Main methods of the Main methods of the Priority Queue ADTPriority Queue ADT– insertIteminsertItem(k, o) -- inserts (k, o) -- inserts

an item with key k and an item with key k and element oelement o

– removeMinremoveMin() -- removes () -- removes the item with smallest the item with smallest key and returns its key and returns its elementelement

Additional methodsAdditional methods– minKeyminKey(k, o) -- returns, (k, o) -- returns,

but does not remove, the but does not remove, the smallest key of an itemsmallest key of an item

– minElementminElement() -- returns, () -- returns, but does not remove, the but does not remove, the element of an item with element of an item with smallest keysmallest key

– sizesize(), (), isEmptyisEmpty()()

Applications:Applications:– Standby flyersStandby flyers– AuctionsAuctions– Stock marketStock market

2424

Total Order RelationTotal Order RelationKeys in a priority Keys in a priority queue can be queue can be arbitrary objects arbitrary objects on which an on which an order is definedorder is defined

Two distinct Two distinct items in a items in a priority queue priority queue can have the can have the same keysame key

Mathematical concept Mathematical concept of total order relation of total order relation – Reflexive property:Reflexive property:

x x x x

– Antisymmetric Antisymmetric property:property:x x y y y y x x x x == y y

– Transitive property:Transitive property: x x y y y y z z x x z z

2525

Comparator ADTComparator ADTA comparator A comparator encapsulates the action of encapsulates the action of comparing two objects comparing two objects according to a given total according to a given total order relationorder relation

A generic priority queue A generic priority queue uses an auxiliary uses an auxiliary comparatorcomparator

The comparator is The comparator is external to the keys being external to the keys being comparedcompared

When the priority queue When the priority queue needs to compare two needs to compare two keys, it uses its keys, it uses its comparatorcomparator

Methods of the Methods of the Comparator ADT, all Comparator ADT, all with Boolean return with Boolean return typetype– isLessThanisLessThan(x, y)(x, y)– isLessThanOrEqualToisLessThanOrEqualTo(x,(x,

y)y)– isEqualToisEqualTo(x,y)(x,y)– isGreaterThanisGreaterThan(x, y)(x, y)– isGreaterThanOrEqualToisGreaterThanOrEqualTo

(x,y)(x,y)– isComparableisComparable(x)(x)

2626

Sorting with a Priority QueueSorting with a Priority QueueWe can use a priority We can use a priority queue to sort a set of queue to sort a set of comparable elementscomparable elements– Insert the elements Insert the elements

one by one with a one by one with a series of series of insertIteminsertItem(e, (e, e) operationse) operations

– Remove the elements Remove the elements in sorted order with a in sorted order with a series of series of removeMinremoveMin() () operationsoperations

The running time of The running time of this sorting method this sorting method depends on the depends on the priority queue priority queue implementationimplementation

Algorithm PQ-Sort(S, C)Input sequence S, comparator C for the elements of SOutput sequence S sorted in increasing order according to CP priority queue with

comparator Cwhile S.isEmpty ()

e S.remove (S. first ())

P.insertItem(e, e)while P.isEmpty()

e P.removeMin()S.insertLast(e)

2727

Sequence-based Priority QueueSequence-based Priority QueueImplementation with an Implementation with an unsorted sequenceunsorted sequence– Store the items of the Store the items of the

priority queue in a list-priority queue in a list-based sequence, in based sequence, in arbitrary orderarbitrary order

Performance:Performance:– insertIteminsertItem takes takes OO(1)(1) time time

since we can insert the since we can insert the item at the beginning or item at the beginning or end of the sequenceend of the sequence

– removeMinremoveMin, , minKeyminKey and and minElementminElement take take OO((nn)) time time since we have to traverse since we have to traverse the entire sequence to the entire sequence to find the smallest keyfind the smallest key

Implementation with a Implementation with a sorted sequencesorted sequence– Store the items of the Store the items of the

priority queue in a priority queue in a sequence, sorted by sequence, sorted by keykey

Performance:Performance:– insertIteminsertItem takes takes OO((nn))

time since we have to time since we have to find the place where to find the place where to insert the iteminsert the item

– removeMinremoveMin, , minKeyminKey and and minElementminElement take take OO(1)(1) time since the smallest time since the smallest key is at the beginning key is at the beginning of the sequenceof the sequence

2828

Selection-SortSelection-SortSelection-sort is the variation of PQ-sort Selection-sort is the variation of PQ-sort where the priority queue is implemented with where the priority queue is implemented with an unsorted sequencean unsorted sequence

Running time of Selection-sort:Running time of Selection-sort:– Inserting the elements into the priority queue with Inserting the elements into the priority queue with

nn insertIteminsertItem operations takes operations takes OO((nn) ) timetime– Removing the elements in sorted order from the Removing the elements in sorted order from the

priority queue with priority queue with nn removeMinremoveMin operations takes operations takes time proportional totime proportional to

1 1 2 2 ……nn

Selection-sort runs in Selection-sort runs in OO((nn22) ) time time

2929

Insertion-SortInsertion-SortInsertion-sort is the variation of PQ-sort Insertion-sort is the variation of PQ-sort where the priority queue is implemented where the priority queue is implemented with a sorted sequencewith a sorted sequence

Running time of Insertion-sort:Running time of Insertion-sort:– Inserting the elements into the priority queue with Inserting the elements into the priority queue with

nn insertIteminsertItem operations takes time proportional to operations takes time proportional to 1 1 2 2 ……nn

– Removing the elements in sorted order from the Removing the elements in sorted order from the priority queue with a series of priority queue with a series of nn removeMinremoveMin operations takes operations takes OO((nn) ) timetime

Insertion-sort runs in Insertion-sort runs in OO((nn22) ) time time

3030

In-place Insertion-sortIn-place Insertion-sortInstead of using an Instead of using an external data structure, external data structure, we can implement we can implement selection-sort and selection-sort and insertion-sort in-placeinsertion-sort in-placeA portion of the input A portion of the input sequence itself serves as sequence itself serves as the priority queuethe priority queueFor in-place insertion-sortFor in-place insertion-sort– We keep sorted the initial We keep sorted the initial

portion of the sequenceportion of the sequence– We can use We can use

swapElementsswapElements instead of instead of modifying the sequencemodifying the sequence

5 4 2 3 1

5 4 2 3 1

4 5 2 3 1

2 4 5 3 1

2 3 4 5 1

1 2 3 4 5

1 2 3 4 5