RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms Chapter 2.11

Analysis of Algorithms

Dr. Ying [email protected]

August 28, 2012

http://www.cse.unl.edu/~ylu/raik283/

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides Most of the lecture notes are based on the slides

from the Textbook’s companion websitefrom the Textbook’s companion website

http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin• Several slides are from Jeff Edmonds of the Several slides are from Jeff Edmonds of the

York UniversityYork University• I have modified them and added new slidesI have modified them and added new slides

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

The Time Complexity of an The Time Complexity of an AlgorithmAlgorithm

The Time Complexity of an The Time Complexity of an AlgorithmAlgorithm

SpecifiesSpecifies how the running time depends how the running time depends on the size of the inputon the size of the input


PurposePurpose


PurposePurpose

To estimate how long a program will run. To estimate how long a program will run. To estimate the largest input that can reasonably be To estimate the largest input that can reasonably be

given to the program. given to the program. To compare the efficiency of different algorithmsTo compare the efficiency of different algorithms.. To help focus on the parts of code that are executed To help focus on the parts of code that are executed

the largest number of times. the largest number of times. To choose an algorithm for an application. To choose an algorithm for an application.


Purpose (Example)Purpose (Example)

Suppose a machine that performs a million floating-Suppose a machine that performs a million floating-point operations per second (10point operations per second (106 6 FLOPS), then how FLOPS), then how long an algorithm will run for an input of size n=50? long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n1) If the algorithm requires n22 such operations: such operations:



Suppose a machine that performs a million floating-Suppose a machine that performs a million floating-point operations per second (10point operations per second (106 6 FLOPS), then how FLOPS), then how long an algorithm will run for an input of size n=50? long an algorithm will run for an input of size n=50? • 1) 1) If the algorithm requires nIf the algorithm requires n22 such operations: such operations:

– 0.0025 second0.0025 second




– 0.0025 second0.0025 second• 2) 2) If the algorithm requires 2If the algorithm requires 2nn such operations: such operations:

– A) Takes a similar amount of time (t < 1 sec)A) Takes a similar amount of time (t < 1 sec)

– B) Takes a little bit longer time (1 sec < t < 1 year)B) Takes a little bit longer time (1 sec < t < 1 year)

– C) Takes a much longer time (1 year < t)C) Takes a much longer time (1 year < t)




– 0.0025 second0.0025 second• 2) 2) If the algorithm requires 2If the algorithm requires 2nn such operations: such operations:

– over 35 years! over 35 years!


Time Complexity Is a FunctionTime Complexity Is a Function

SpecifiesSpecifies how the running time depends on the size of the how the running time depends on the size of the input. input.

AA function mapping function mapping

““size” of inputsize” of input

““time” T(n) executed . time” T(n) executed .


Definition of Time?Definition of Time?


Definition of TimeDefinition of Time

# of seconds (machine, implementation dependent). # of seconds (machine, implementation dependent).

# lines of code executed. # lines of code executed.

# of times a specific operation is performed # of times a specific operation is performed ( (e.g., e.g., addition). addition).


Theoretical analysis of time Theoretical analysis of time efficiencyefficiency

Time efficiency is analyzed by determining the number of Time efficiency is analyzed by determining the number of repetitions of the repetitions of the basic operationbasic operation as a function of as a function of input sizeinput size

Basic operation:Basic operation: the operation that contributes most the operation that contributes most towards the running time of the algorithm.towards the running time of the algorithm.

TT((nn) ≈ ) ≈ ccopopCC((nn))running time execution time

for basic operation

Number of times basic operation is

executed

input size


Input size and basic operation Input size and basic operation examplesexamples

ProblemProblem Input size measureInput size measure Basic operationBasic operation

Search for key in a list of Search for key in a list of nn items items

Multiply two matrices of Multiply two matrices of floating point numbersfloating point numbers

Compute Compute aann

Graph problemGraph problem





Number of items in the Number of items in the list: list: nn

Key comparisonKey comparison








Number of items in the Number of items in the list: list: nn

Key comparisonKey comparison


Dimensions of matricesDimensions of matricesFloating point Floating point multiplicationmultiplication






Search for key in list of Search for key in list of nn items items

Number of items in list Number of items in list nn Key comparisonKey comparison



Compute Compute aann nnFloating point Floating point multiplicationmultiplication





Search for key in list of Search for key in list of nn items items

Number of items in list Number of items in list nn Key comparisonKey comparison



Compute Compute aann nnFloating point Floating point multiplicationmultiplication

Graph problemGraph problem #vertices and/or edges#vertices and/or edgesVisiting a vertex or Visiting a vertex or traversing an edgetraversing an edge


Theoretical analysis of time Theoretical analysis of time efficiencyefficiency

Time efficiency is analyzed by determining the Time efficiency is analyzed by determining the number of repetitions of the number of repetitions of the basic operationbasic operation as a as a function of function of input sizeinput size


Which Input of size n?Which Input of size n?

Efficiency also depends on the particular inputEfficiency also depends on the particular input

For instance: search a key in a list of n letters For instance: search a key in a list of n letters • Problem input: a list of n lettersProblem input: a list of n letters• How many different inputs?How many different inputs?


Which Input of size n?Which Input of size n?

Efficiency also depends on the particular inputEfficiency also depends on the particular input

For instance: search a key in a list of n letters For instance: search a key in a list of n letters

There are 26There are 26n n inputs of size n.inputs of size n.Which do we considerWhich do we consider

for the time efficiency C(n)?for the time efficiency C(n)?


Best-case, average-case, worst-Best-case, average-case, worst-casecase

Worst case: W(Worst case: W(nn) – maximum over inputs of size ) – maximum over inputs of size nn

Best case: B(Best case: B(nn) – minimum over inputs of size ) – minimum over inputs of size nn

Average case: A(Average case: A(nn) – “average” over inputs of size ) – “average” over inputs of size nn• NOT the average of worst and best caseNOT the average of worst and best case

• Under some assumption about the probability distribution of all Under some assumption about the probability distribution of all possible inputs of size possible inputs of size n, n, calculate the weighted sum of expected C(n) calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size (numbers of basic operation repetitions) over all possible inputs of size n. n.


Example: Sequential searchExample: Sequential search

Problem:Problem: Given a list of Given a list of nn elements and a search key elements and a search key KK, find , find an element equal to an element equal to KK, if any., if any.

Algorithm:Algorithm: Scan the list and compare its successive Scan the list and compare its successive elements with elements with KK until either a matching element is found until either a matching element is found ((successful searchsuccessful search) or the list is exhausted () or the list is exhausted (unsuccessful unsuccessful searchsearch))

Worst caseWorst case

Best caseBest case

Average caseAverage case


An exampleAn example

Compute gcd(m, n) by applying Compute gcd(m, n) by applying the algorithm based on the algorithm based on checking consecutive integers from min(m, n) down to checking consecutive integers from min(m, n) down to gcd(m, n)gcd(m, n)

Input size?Input size? Best case?Best case? Worst case?Worst case? Average case?Average case?


Types of formulas for basic Types of formulas for basic operation countoperation count

Exact formulaExact formula

e.g., C(e.g., C(nn) = ) = nn((nn-1)/2-1)/2

Formula indicating order of growth with specific Formula indicating order of growth with specific multiplicative constantmultiplicative constant

e.g., C(e.g., C(nn) ≈ 0.5 ) ≈ 0.5 nn22

Formula indicating order of growth with unknown Formula indicating order of growth with unknown multiplicative constantmultiplicative constant

e.g., C(e.g., C(nn) ≈ ) ≈ cncn22


Types of formulas for basic Types of formulas for basic operation countoperation count

Exact formulaExact formula

e.g., C(e.g., C(nn) = ) = nn((nn-1)/2-1)/2 Formula indicating order of growth with specific Formula indicating order of growth with specific

multiplicative constantmultiplicative constant

e.g., C(e.g., C(nn) ≈ 0.5 ) ≈ 0.5 nn22

Formula indicating order of growth with unknown Formula indicating order of growth with unknown multiplicative constantmultiplicative constant

e.g., C(e.g., C(nn) ≈ ) ≈ cncn22

Most important: Order of growth within a constant Most important: Order of growth within a constant multiple as multiple as nn→∞→∞


Asymptotic growth rateAsymptotic growth rate

A way of comparing functions that ignores constant factors A way of comparing functions that ignores constant factors and small input sizesand small input sizes

O(O(gg((nn)):)):

Θ Θ ((gg((nn)):)):

ΩΩ((gg((nn)):)):


Asymptotic growth rateAsymptotic growth rate

A way of comparing functions that ignores constant factors A way of comparing functions that ignores constant factors and small input sizesand small input sizes

O(O(gg((nn)): class of functions )): class of functions ff((nn) that grow ) that grow no fasterno faster than than gg((nn))

Θ Θ ((gg((nn)): class of functions )): class of functions ff((nn) that grow ) that grow at same rateat same rate as as gg((nn))

ΩΩ((gg((nn)): class of functions )): class of functions ff((nn) that grow ) that grow at least as fastat least as fast as as gg((nn))


Table 2.1Table 2.1

Classifying FunctionsClassifying Functions

GivingGiving an idea of how fast a function an idea of how fast a function grows without going into too much detail. grows without going into too much detail.


Which are more alike?Which are more alike?



Mammals





Dogs


Classifying AnimalsClassifying Animals

Vertebrates

Birds

Mam

mals

Reptiles

Fish

Dogs G

iraffe



n1000 n2 2n



Polynomials

n1000 n2 2n



1000n2 3n2 2n3



Quadratic

1000n2 3n2 2n3


Classifying Functions?Classifying Functions?

Functions



Functions

Constant

Logarithm

ic

Poly L

ogarithmic

Polynom

ial

Exponential

Factorial

(log n)5 n5 25n5 log n5 n!


Classifying Functions?Classifying Functions?

Polynomial



Polynomial

Linear

Quadratic

Cubic

?

5n25n 5n3 5n4


Logarithmic Logarithmic

loglog1010nn = # digits to write n = # digits to write n

loglog22nn = # bits to write n = # bits to write n

= 3.32 log = 3.32 log1010nn

log(nlog(n10001000) = 1000 log(n)) = 1000 log(n)

Differ only by a multiplicative constant


Poly LogarithmicPoly Logarithmic

(log n)5 = log5 n


Which grows faster?Which grows faster?

log1000 n n0.001


Poly Logarithmic << Poly Logarithmic << PolynomialPolynomial

For sufficiently large nFor sufficiently large n

log1000 n << n0.001



10000 n 0.0001 n2


Linear << QuadraticLinear << Quadratic


10000 n << 0.0001 n2



n1000 20.001 n


Polynomial << ExponentialPolynomial << Exponential


n1000 << 20.001 n


Ordering FunctionsOrdering Functions

Functions

Constant

Logarithm

ic

Poly L

ogarithmic

Polynom

ial

Exponential

(log n)5 n5 25n5 log n5

<< << << << <<

<< << << << <<

Factorial

n!


Which Functions are Constant?Which Functions are Constant?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)


Which Functions are Constant?Which Functions are Constant?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)

YesYesYes

No

YesYes


Which Functions are “Constant”?Which Functions are “Constant”?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)

The running time of the algorithm is a “Constant” if it does not depend significantly

on the size of the input.


Which Functions are “Constant”?Which Functions are “Constant”?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)

YesYesYesNo

NoYes

Lie in between

7

9

The running time of the algorithm is a “Constant” It does not depend significantly

on the size of the input.


Which Functions are Quadratic?Which Functions are Quadratic?

• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n



• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n

Lie in between



• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n

Ignore low-order termsIgnore multiplicative constants.

Ignore "small" values of n. Write θ(n2).


ExamplesExamples

f(n) g(n) O(g(n))? Ω(g(n))? Θ(g(n))?

1) ln2n n

2) nk cn

3) nsinn

4) 2n 2n/2

5) nlgc clgn

6) lg(n!) lg(nn)

n

Documents

RAIK 283: Data Structures & Algorithms