3 - lect_3 (1)

8/8/2019 3 - lect_3 (1)

1/14

Department of Computer Science

COMSATS Institute of InformationTechnology, Abbottabad

30.09.2010 CSC511 - Advanced Algorithms Analysis 1

Khizar Hayat

Advanced Algorithms Analysis

(CSC511)

Asymptotic Analysis

After George Bebis

8/8/2019 3 - lect_3 (1)

2/14


Logarithms and properties

In algorithm analysis we often use the notation

log n without specifying the base

nn

nn

elogln

loglg 2

=

= =yxlogBinary logarithm

Natural logarithm

)lg(lglglg

)(lglg

nn

nnkk

=

=

xy log

=xylog yx loglog +

=

y

xlog yx loglog

logb x =

ab

xlog

=x

balog

log

log

a

a

x

b

8/8/2019 3 - lect_3 (1)

3/14


Common Summations

Arithmetic series:

Geometric series:

Special case: |x| < 1:

Harmonic series:

Other important

formulas:

2)1( +nn

=

=+++=

n

k

nk1

...21

( )11

11

+

xx

xn=++++=

=

nn

k

k xxxx ...1 2

0

x1

1=

=0k

kx

nln=

+++=n

k nk1

1...

2

11

1

=

n

kk

1lg nn lg

1

1

1 +

+

pnp

=

+++=n

k

pppp nk1

...21

8/8/2019 3 - lect_3 (1)

4/14


Definition: O(g), at mostorder g

Let f,g are functions RR.

We say that fis at most order g, if:

c,k: f(x) cg(x), x>k Beyond some point k, function fis at most a

constant c times g (i.e., proportional to g).

fis at most order g, or fis O(g), orf=O(g) all just mean that fO(g).

Sometimes the phrase at most isomitted.

witnesses

8/8/2019 3 - lect_3 (1)

5/14


Big-O Visualization

k

8/8/2019 3 - lect_3 (1)

6/14


Points about the definition

Note that fis O(g) as long as anyvalues ofcand kexist that satisfy the definition.

But: The particular c, k, values that make thestatement true are notunique: Any larger

value ofc and/or kwill also work.You are not required to find the smallest c and

kvalues that work. (Indeed, in some cases,there may be no smallest values!)

However, you should prove that the values you choose do work.

8/8/2019 3 - lect_3 (1)

7/14


No Uniqueness

There is no unique set of values for n0

and c in proving

the asymptotic bounds

Prove that 100n + 5 = O(n2)

100n + 5 100n + n = 101n 101n2

for all n 5

n0 = 5 and c = 101is a solution

100n + 5 100n + 5n = 105n 105n2

for all n 1

n0 = 1 and c = 105is also a solution

Must findSOMEconstants c and n0 that satisfy the asymptotic notation

relation

8/8/2019 3 - lect_3 (1)

8/14


Big-O Proof Examples

Show that 30n+8 is O(n). Show c,k: 30n+8 cn, n>k.

Let c=31, k=8. Assume n>k=8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn.

Show that n2+1 is O(n2). Show c,k: n2+1 cn2, n>k:.

Let c=2, k=1. Assume n>1. Then

cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.

8/8/2019 3 - lect_3 (1)

9/14


Note 30n+8 isntless than nanywhere (n>0).

It isnt even

less than 31neverywhere.

But it is less than31n everywhere to

the right ofn=8.n>k=8

Big-O example, graphically

Increasing n

Valueoffun

ction

n

30n+8

cn =

31n

30n+8

O(n)

8/8/2019 3 - lect_3 (1)

10/14


Big-O Visualization

O(g(n)) is the set of

functions with smaller

or same order of

growth as g(n)

8/8/2019 3 - lect_3 (1)

11/14


Order-of-Growth in Expressions

O(f) can be used as a term in an arithmeticexpression .

E.g.: we can write x2+x+1 as x2+O(x)meaning x2 plus some function that is O(x).

Formally, you can think of any such expressionas denoting a set of functions:

x2+O(x) : {g | fO(x): g(x)=x2+f(x)}

8/8/2019 3 - lect_3 (1)

12/14


Useful Facts about Big O

Constants ...c>0, O(cf)=O(f+c)=O(fc)=O(f)

Sums:

- IfgO(f) and hO(f), then g+hO(f).- IfgO(f

1) and hO(f

2), then

g+hO(f1+f

2) =O(max(f

1,f

2))

(Very useful!)

8/8/2019 3 - lect_3 (1)

13/14


More Big-O facts

Products:IfgO(f

1) and hO(f

2), then ghO(f

1f2)

Big O, as a relation, is transitive:fO(g) gO(h) fO(h)

8/8/2019 3 - lect_3 (1)

14/14


More Big O facts

f,g & constants a,bR, with b 0, af= O(f) (e.g. 3x2= O(x2))

f+O(f) = O(f) (e.g. x2

+x = O(x2

)) |f|1-b= O(f) (e.g.x1= O(x))

(logb|f|)a= O(f) (e.g. logx= O(x))

g=O(fg) (e.g.x = O(xlogx))

fg O(g) (e.g.xlogx O(x))

a=O(f) (e.g. 3 = O(x))

Documents

3 - lect_3 (1)