View
0
Download
0
Category
Preview:
Citation preview
Sorting
I Given is a sequence of pairs
(k1, e1), (k2, e2), . . . , (kn, en)
of elements ei with keys ki and an order ≤ on the keys.
4
e1
2
e2
3
e3
1
e4
5
e5
I We search a permutation π of the pairs such that the keyskπ(1) ≤ kπ(2) ≤ . . . ≤ kπ(n) are in ascending order.
1
e4
2
e2
3
e3
4
e1
5
e5
Selection-Sort
Selection-sort takes an unsorted list A and sorts as follows:I search the smallest element A and swap it with the first
I afterwards continue sorting the remainder of A
5 4 3 7 1
1 4 3 7 5
1 3 4 7 5
1 3 4 7 5
1 3 4 5 7
1 3 4 5 7
Unsorted Part
Sorted Part
Minimal Element
Selection-Sort: Properties
I Time complexity (best, average and worst-case) O(n2):
n + (n − 1) + . . .+ 1 =n2 + n
2∈ O(n2)
(caused by searching the minimal element)
I Selection-sort is an in-place sorting algorithm.
In-Place AlgorithmAn algorithm is in-place if apart from space for input data onlya constant amount of space is used: space complexity O(1).
Stable Sorting Algorithms
Stable Sorting AlgorithmA sorting algorithm is called stable if the order of items withequal key is preserved.
Example: not stable
3 2 2
A B C2 2 3
C B A2 2 3
C B AB, C exchanged order,although they have equal keys
Example: stable
3 2 2
A B C2 3 2
B A C2 2 3
B C A
Stable Sorting Algorithms
Applications of stable sorting:I preserving original order of elements with equal key
For example:I we have an alphabetically sorted list of names
I we want to sort by date of birth while keeping alphabeticalorder for persons with same birthday
Selection-sort is stable ifI we always select the first (leftmost) minimal element
Insertion-Sort
Selection-sort takes an unsorted list A and sorts as follows:I distinguishes a sorted and unsorted part of A
I in each step we remove an element from the unsortedpart, and insert it at the correct position in the sorted part
5 3 4 7 1
5 3 4 7 1
3 5 4 7 1
3 4 5 7 1
3 4 5 7 1
1 3 4 5 7
Unsorted Part
Sorted Part
Minimal Element
Insertion-Sort: Complexity
I Time complexity worst-case O(n2):
1 + 2 + . . .+ n =n2 + n
2∈ O(n2)
(searching insertion position together with inserting)I Time complexity best-case O(n):
I if list is already sorted, and
I we start searching insertion position from the end
I More general: time complexity O(n · (n − d + 1))
I if the first d elements are already sorted
I Insertion-sort is an in-place sorting algorithm:I space complexity O(1)
Insertion-Sort: Properties
I Simple implementation.I Efficient for:
I small lists
I big lists of which a large prefix is already sorted
I Insertion-sort is stable if:I we always pick the first element from the unsorted part
I we always insert behind all elements with equal keys
I Insertion-sort can be used online:I does not need all data at once,
I can sort a list while receiving it
Heaps
A heap is a binary tree storing keys at its inner nodes such that:
I if A is a parent of B, then key(A) ≤ key(B)
I the heap is a complete binary tree: let h be the heap heightI for i = 0, . . . , h − 1 there are 2i nodes at depth i
I at depth h − 1 the inner nodes are left of the external nodes
We call the rightmost inner node at depth h − 1 the ‘last node’.
2
5
9 7
6
last node
Height of Heaps
TheoremA heap storing n keys height O(log2 n).
Proof.A heap of height h contains 2i nodes at every depthi = 0, . . . , h − 2 and at least one node at depth h − 1. Thusn ≥ 1 + 2 + 4 + . . . 2h−2 + 1 = 2h−1. Hence h ≤ 1 + log2 n.
10
21
2ii
1h − 1
depth keys
Heaps and Priority Queues
We can use a heap to implement a priority queue:I inner nodes store (key, element) pair
I variable last points to the last node
(2,Sue)(5,Pat) (6,Mark)
(7,Anna)
(9, Jeff )
last
I For convenience, in the sequel, we only show the keys.
Heaps: Insertion
The insertion of key k into the heap consits of 3 steps:I Find the insertion node z (the new last node).
I Expand z to an internal node and store k at z.
I Restore the heap-order property (see following slides).
2
5
9 7
6
insertion node z
Example: insertion of key 1 (without restoring heap-order)
2
5
9 7
6
1z
Heaps: Insertion, Upheap
After insertion of k the heap-order may be violated.
We restore the heap-order using the upheap algorithm:I we swap k upwards along the path to the root
as long as the parent of k has a larger keyTime complexity is O(log2 n) since the heap height is O(log2 n).
2
5
9 7
6
1
Now the heap-order property is restored.
Heaps: Insertion, Upheap
After insertion of k the heap-order may be violated.
We restore the heap-order using the upheap algorithm:I we swap k upwards along the path to the root
as long as the parent of k has a larger keyTime complexity is O(log2 n) since the heap height is O(log2 n).
1
5
9 7
2
6
Now the heap-order property is restored.
Heaps: Insert, Finding the Insertion Position
An algorithm for finding the insertion position (new last node):I start from the current last node
I while the current node is a right child, go to the parent node
I if the current node is a left child, go to the right child
I while the current node has a left child, go to the left childTime complexity is O(log2 n) since the heap height is O(log2 n).(we walk at most at most once completely up and down again)
Heaps: Removal of the Root
The removal of the root consits of 3 steps:I Replace the root key with the key of the last node w.
I Compress w and its children into a leaf.
I Restore the heap-order property (see following slides).
2
5
9 7
6w
Example: removal of the root (without restoring heap-order)
7
5
9
6w
Heaps: Removal, Downheap
Replacing the root key by k may violate the heap-order.
We restore the heap-order using the downheap algorithm:I we swap k with its smallest child
as long as a child of k has a smaller keyTime complexity is O(log2 n) since the heap height is O(log2 n).
7
5
9
6
Now the heap-order property is restored. The new last nodecan be found similar to finding the insertion position (but nowwalk against the clock direction).
Heaps: Removal, Downheap
Replacing the root key by k may violate the heap-order.
We restore the heap-order using the downheap algorithm:I we swap k with its smallest child
as long as a child of k has a smaller keyTime complexity is O(log2 n) since the heap height is O(log2 n).
5
7
9
6
Now the heap-order property is restored. The new last nodecan be found similar to finding the insertion position (but nowwalk against the clock direction).
Heaps: Removal, Finding the New Last Node
After the removal we have to find the new last node:I start from the old last node (which has been remove)
I while the current node is a left child, go to the parent node
I if the current node is a right child, go to the left child
I while the current node has an right child which is not a leaf,go to the right child
Time complexity is O(log2 n) since the heap height is O(log2 n).(we walk at most at most once completely up and down again)
removed
Heap-Sort
We implement a priority queue by means of a heap:I insertItem(k, e) corresponds to adding (k, e) to the heap
I removeMin() corresponds to removing the root of the heapPerformance:
I insertItem(k, e), and removeMin() run in O(log2 n) time
I size(), isEmtpy(), minKey(), and minElement() are O(1)
Heap-sort is O(n log2 n)
Using a heap-based priority queue we can sort a list of nelements in O(n · log2 n) time (n times insert + n times removal).
Thus heap-sort is much faster than quadratic sorting algorithms(e.g. selection sort).
Vector-based Heap Implementation
We can represent a heap with n keys by a vector of size n + 1:
2
5
9 7
6
0 1 2 3 4 52 5 6 9 7
I The root node has rank 1 (cell at rank 0 is not used).I For a node at rank i :
I the left child is at rank 2i
I the right child is at rank 2i + 1
I the parent (if i > 1) is located at rank bi/2c
I Leafs and links between the nodes are not stored explicitly.
Vector-based Heap Implementation, continued
We can represent a heap with n keys by a vector of size n + 1:
2
5
9 7
6
0 1 2 3 4 52 5 6 9 7
I The last element in the heap has rank n, thus:I insertItem corresponds to inserting at rank n + 1
I removeMin corresponds to removing at rank n
I Yields in-place heap-sort (space complexity O(1)):I uses a max-heap (largest element on top)
Merging two Heaps
We are given two heaps h1, h2 and a key k:I create a new heap with root k and h1, h2 as children
I we perform downheap to restore the heap-order
2
6 5
h1 3
4 5
h2 k = 7
7
2
6 5
3
4 5
Merging two Heaps
We are given two heaps h1, h2 and a key k:I create a new heap with root k and h1, h2 as children
I we perform downheap to restore the heap-order
2
6 5
h1 3
4 5
h2 k = 7
2
5
6 7
3
4 5
Bottom-up Heap Construction
We have n keys and want to construct a heap from them.
Possibility one:I start from empty heap and use n times insert
I needs O(n log2 n) time
Possibility two: bottom-up heap constructionI for simplicity we assume n = 2h − 1 (for some h)
I take 2h−1 elements and turn them into heaps of size 1I for phase i = 1, . . . , log2 n:
I merge the heaps of size 2i − 1 to heaps of size 2i+1 − 1
2i − 1 2i − 1merge
2i − 1 2i − 1
2i+1 − 1
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
16 15 4 12 6 9 23 20
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
25
16 15
5
4 12
11
6 9
27
23 20
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
15
16 25
4
5 12
6
11 9
20
23 27
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
7
15
16 25
4
5 12
8
6
11 9
20
23 27
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
4
15
16 25
5
7 12
6
8
11 9
20
23 27
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
10
4
15
16 25
5
7 12
6
8
11 9
20
23 27
We are ready: this is the final heap.
Bottom-up Heap Construction, Example
We construct a heap from the following 24 − 1 = 15 elements:
16,15,4,12,6,9,23,20,25,5,11,27,7,8,10
4
5
15
16 25
7
10 12
6
8
11 9
20
23 27
We are ready: this is the final heap.
Bottom-up Heap Construction, Performance
Visualization of the worst-case of the construction:
I displays the longest possible heapdown paths(may not be the actual path, but maximal length)
I each edge is traversed at most once
I we have 2n edges hence the time complexity is O(n)
I faster than n successive insertions
Recommended