View
218
Download
0
Category
Preview:
Citation preview
1
CSE 326: Data Structures: Sorting
Lecture 17: Wednesday, Feb 19, 2003
2
Today
• Bucket sort (review)
• Radix sort
• Merge sort for external memory
3
Lower Bound
Recall:• Any sorting algorithm based on comparisons
requires (n log n) time– More precisely: there exists a “bad input” for that
algorithm, on which it takes (n log n)
– The theorem can be extended: the average running time is (n log n)
• The next two algorithms (Bucket, Radix) apparently break this theorem !
4
Bucket Sort
• Now let’s sort in O(N)• Assume:
A[0], A[1], …, A[N-1] {0, 1, …, M-1}M = not too big
• Example: sort 1,000,000 person records on the first character of their last names:– Hence M = 128 (in practice: M = 27)
5
Bucket Sortint bucketSort(Array A, int N) { for k = 0 to M-1 Q[k] = new Queue;
for j = 0 to N-1 Q[A[j]].enqueue(A[j]);
Result = new Queue; for k = 0 to M-1 Result = Result.append(Q[k]);
return Result;}
int bucketSort(Array A, int N) { for k = 0 to M-1 Q[k] = new Queue;
for j = 0 to N-1 Q[A[j]].enqueue(A[j]);
Result = new Queue; for k = 0 to M-1 Result = Result.append(Q[k]);
return Result;}
Stablesorting !
6
Bucket Sort• Running time: O(M+N)
• Space: O(M+N)
• Recall that M << N, hence time = O(N)
• What about the Theorem that says sorting takes (N log N) ?? This is not based on
key comparisons, insteadexploits the fact that keys
are small
7
Radix Sort• I still want to sort in time O(N): non-trivial keys
• A[0], A[1], …, A[N-1] are strings– Very common in practice
• Each string is:cd-1cd-2…c1c0, where c0, c1, …, cd-1 {0, 1, …, M-1}M = 128
• Other example: decimal numbers
8
RadixSort• Radix = “The base of a
number system” (Webster’s dictionary)– alternate terminology: radix is
number of bits needed to represent 0 to base-1; can say “base 8” or “radix 3”
• Used in 1890 U.S. census by Hollerith
• Idea: BucketSort on each digit, bottom up.
9
The Magic of RadixSort
• Input list: 126, 328, 636, 341, 416, 131, 328
• BucketSort on lower digit:341, 131, 126, 636, 416, 328, 328
• BucketSort result on next-higher digit:416, 126, 328, 328, 131, 636, 341
• BucketSort that result on highest digit:126, 131, 328, 328, 341, 416, 636
10
Inductive Proof that RadixSort Works
• Keys: d-digit numbers, base B– (that wasn’t hard!)
• Claim: after ith BucketSort, least significant i digits are sorted. – Base case: i=0. 0 digits are sorted.– Inductive step: Assume for i, prove for i+1.
Consider two numbers: X, Y. Say Xi is ith digit of X:• Xi+1 < Yi+1 then i+1th BucketSort will put them in order• Xi+1 > Yi+1 , same thing• Xi+1 = Yi+1 , order depends on last i digits. Induction hypothesis
says already sorted for these digits because BucketSort is stable
11
Radix Sort
int radixSort(Array A, int N) { for k = 0 to d-1 A = bucketSort(A, on position k)}
int radixSort(Array A, int N) { for k = 0 to d-1 A = bucketSort(A, on position k)}
Running time: T = O(d(M+N)) = O(dN) = O(Size)
12
Radix Sort35 53 55 33 52 32 25
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
53 33 35 55 25
52 32 53 33 35 55 25
52 32
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
32 33 3525 52 53 55
25 32 33 35 52 53 55
A=
A=
A=
13
Running time of Radixsort
• N items, d digit keys of max value M
• How many passes?
• How much work per pass?
• Total time?
14
Running time of Radixsort
• N items, d digit keys of max value M
• How many passes? d
• How much work per pass? N + M – just in case M>N, need to account for time to empty out
buckets between passes
• Total time? O( d(N+M) )
15
Radix Sort
• What is the size of the input ? Size = dN
• Radix sort takes time O(Size) !!
cd-1 cD-2 … c0
A[0] ‘S’ ‘m’ ‘i’ ‘t’ ‘h’
A[1] ‘J’ ‘o’ ‘n’ ‘e’ ‘s’
…
A[N-1]
16
Radix Sort
• Variable length strings:
• Can adapt Radix Sort to sort in time O(Size) !– What about our Theorem ??
A[0]
A[1]
A[2]
A[3]
A[4]
17
Radix Sort
• Suppose we want to sort N distinct numbers
• Represent them in decimal:– Need d=log N digits
• Hence RadixSort takes time O(Size) = O(dN) = O(N log N)
• The total Size of N keys is O(N log N) !
• No conflict with theory
18
Sorting HUGE Data Sets• US Telephone Directory:
– 300,000,000 records • 64-bytes per record
– Name: 32 characters– Address: 54 characters– Telephone number: 10 characters
– About 2 gigabytes of data– Sort this on a machine with 128 MB RAM…
• Other examples?
19
Merge Sort Good for Something!
• Basis for most external sorting routines
• Can sort any number of records using a tiny amount of main memory– in extreme case, only need to keep 2 records in
memory at any one time!
20
External MergeSort• Split input into two “tapes” (or areas of disk)• Merge tapes so that each group of 2 records is
sorted• Split again• Merge tapes so that each group of 4 records is
sorted• Repeat until data entirely sorted
log N passes
21
Sorting
• Illustrates the difference in algorithm design when your data is not in main memory:– Problem: sort 8Gb of data with 8Mb of RAM
• We know we can do it in O(n log n) time, but let’s see the number of disk I/O’s
22
2-Way Merge-sort:Requires 3 Buffers
• Pass 1: Read a page, sort it, write it.– only one buffer page is used
• Pass 2, 3, …, etc.:– three buffer pages used.
Main memory buffers
INPUT 1
INPUT 2
OUTPUT
DiskDisk
Buffer size = 8Kb (typically)
23
2-Way Merge-sort• A run = a sequence of sorted elements
• Main property of 2-way merge:– If the minimum run length is L, then after merge the minimum run
length is 2L
• Initially: minimum run length = 1 (why ?)• After one pass = 2• After two passes = 4• . . .
2 40 649 66 70 80 75 65 40 50
run run run
24
Two-Way External Merge Sort
• Each pass we read + write each page in file.
• N pages in the file => the number of passes
• So total cost is:
• Improvement: start with larger runs
• Sort 1GB with 1MB memory in 10 passes
log2 1N
2 12N Nlog
Input file
1-page runs
2-page runs
4-page runs
8-page runs
PASS 0
PASS 1
PASS 2
PASS 3
9
3,4 6,2 9,4 8,7 5,6 3,1 2
3,4 5,62,6 4,9 7,8 1,3 2
2,34,6
4,7
8,91,35,6 2
2,3
4,46,7
8,9
1,23,56
1,22,3
3,4
4,56,6
7,8
25
2-Way Merge-sort
• Hence we need exactly log N passes through the entire data
• How much is N ? N 106 (why ?)• Hence we need to read and write the entire 8GB
data log(106) = 20 times !
• It takes about 1minute to read 1GB of data– even more
• It takes at least160 minutes to sort = 3 hours
26
2-Way Merge-sort: less dumb• Use the 8Mb of main memory better !
• Initial step: Run formation– Read 8Mb of data in main memory– Sort (what algorithm would you use ?)– Write to disk
• Now the runs are 8Mb after one pass !• After subsequent passes the runs are
– 2 8Mb, 4 8Mb, . . .
• Need log(8Gb/8Mb) = log(103) = 10 passes• 1.5h instead of 3h have time to see a movie
27
Can We Do Better ?
• We have more main memory• Used it during all passes
• Multiway merge:• Given M sorted sequence• Merge them
28
Multiway Merge
1 9 27 60 80 . . .
2 3 4 5 94 . . .
10 20 30 40 50 . . .
2 4 32 69 94 . . .
1 2 2 3 4 4 . . .
m
At each step:select the smallestvalue among the M,store in the output
What datastructure shouldwe use here ?
29
Multiway Merge-Sort
• Phase one: load M bytes in memory, sort– Result: runs of length M/B blocks
M bytes of main memory
DiskDisk
. .
.. . .
M/B blocks
B = size of one block = 8Kb typically
30
Pass Two
• Merge m = M/B runs into a new run
• Runs have M/B (M/B – 1) (M/B)2 blocks
M bytes of main memory
DiskDisk
. .
.. . .
Input M/B
Input 1
Input 2. . . .
Output
31
Pass Three
• Merge M/B – 1 runs into a new run
• Runs have now M/R (M/B – 1)2 (M/B)3 blocks
M bytes of main memory
DiskDisk
. .
.. . .
Input M/B
Input 1
Input 2. . . .
Output
32
Multiway Merge-Sort
• Input file has N bytes
• Need logM/B (N/B) = (log(N/B)) / (log(M/B)) complete passes over the file
• N/B = 8Gb / 8Kb = 106
• M/B = 8Mb / 8Kb = 103
• Hence need log(106)/log(103) = 2 passes !• 2 minutes ! Time for two movies
33
Multiway Merge-Sort
• With today’s main memories, we can sort almost any file in two passes
• The file can have (M/B)2 blocks
• XML Toolkit:– xsort sorts using multiway merge, in two passes
Recommended