mathcamp_lec1 - kopia (5)

8/11/2019 mathcamp_lec1 - kopia (5)

1/49

Math/Stats Camp (MSc AEF)Day 1: Intro, Calculus, Matrix Algebra I

Yun Liu

Department of Economics

September 1, 2014

1 / 4 9


2/49

Prologue

Course Logistics

Coffee break: 1 hr teaching + 15 mins breaks, OK?

No exercise and exam after the math/stats camp, but materials will comeout in your following econometrics, economics and finance courses.

Course website: http://yunliueconomics.weebly.com/math-camp-2014.html

and CBS LEARN

Also,

Due to time limit, we can only superficially skim through the topics.

Probably because of the content itself, this condensed camp will be taughtin a relatively dry way.

Try to scrutinize these concepts/definitions (from other more comprehensivelecture notes and textbooks) in your following studies.

2 / 4 9
http://yunliueconomics.weebly.com/math-camp-2014.htmlhttp://yunliueconomics.weebly.com/math-camp-2014.html


3/49

Prologue

Course Plan I

Mathematics

Day 1: September 1st

Introduction

Calculus

Matrix algebra I

Day 2: September 3rd

Recap Day 1

Matrix algebra IIUnconstrained optimization

Constrained optimization (Lagrangian method)

3 / 4 9


4/49

Prologue

Course Plan II

Statistics

Day 3: September 5th

Recap Day 2Introduction (notations and basic concepts)

Probability, Random variables

Probability density functions, Probability mass functions, Cumulativedistribution functionsSummarizing distributions (Expectation, Variance, Covariance,Correlation, Conditional expectation)

Day 4: September 8th

Recap Day 3Finite sample properties of estimators (unbiasedness, consistency)Asymptotic properties of estimators

Inference, Hypothesis testing

4 / 4 9


5/49

Prologue

Why we need math? I

In economic theory

Economics is the most mathematical part of the social science.

Economics researchers try to elicit/predict some common patterns of

peoples behavior through their observations (imaginations?). Mathematicsis the most elegant and unambiguous language to express logic relationsamong different objects.

Also, in most cases, we are unable to conduct controlled experiments tounderstand the precise economic impacts of specific policies or changes to

the existing economic structure. As a result, economists resort tomathematical models that provide a rigorous, abstract framework to analyzethe impacts of economic policy.

5 / 4 9

l


6/49

Prologue

Why we need math? II

In empirical works (econometrics)

Obvious . . . You need sufficient statistics and probability knowledge to do

regression, i.e., choose the right model, and interpret the results correctly ina meaningful way.

Economics theory tell you the demand curve is downward slope,econometrics tell you what the slope is.

6 / 4 9

P l


7/49

Prologue

Why we need math? III

Models, like maps, come in all shapes and sizes: they can be complex or simple,highlight many different features of the area being covered or focus on a single

feature. Essentially, they are both abstractions of a more complex reality.

Anonymous

7 / 4 9

F ti


8/49

Function

Function: basic notions I

A set is a collection of items. Each item in a set are known as the elementsof that set.

A function is a relationship (not necessarily an explicit algebraic equation)that relates each element of one set (this set is the domain of the function)

to a single element of another set (that set is known as the range of thatfunction).

A function is said to be a one-to-one function if each element of thedomain is mapped to a unique element of the range.

A function is said to be an onto function if each element of the range ismapped to at least once. Thus a one to one and onto function will haveeach element of the domain mapped uniquely to every single element of therange.

8 / 4 9

Function


9/49

Function

Function: basic notions II

9 / 4 9

Function


10/49

Function

Function: basic notions III

A univariate function is a function that has a single argument. e.g.,y=f(x) =x2

A multivariate function, on the other hand, has more than one argument.e.g., z=f(x, y) =x2 +y2.

Besides dependent and independent variables, a function can also involvevarious constants, which we call parameters. e.g., y=f(x) ax+b,where a and bare parameters.

10/49

Function


11/49

Function

Function: basic notions IV

Note:

Economists often refer to the yvariable in a function of the form y=f(x)as the dependent variable (or the value of the function) and to the xvariable as the independent variable.

In economic models, variables (dependent and independent) that the modelpresents are classified into two categories: endogenous variables andexogenous variables.

Endogenous variables are variables whose values are determined by the

model.Exogenous variables are variable determined outside the model.

11/49

Function


12/49

Function

Function: Monotonicity

A function f(x) can be categorized as (strictly) increasing or decreasing iffor any x1 and x2 where x2 >x1, we say:

The function is increasing if f(x2) f(x1)

The function is strictly increasing iff(x2)>f(x1)

The function is decreasing iff(x2) f(x1)The function is strictly decreasing iff(x2)< f(x1)

A monotonic function is always either a decreasing or increasing function.

A strictly monotonic function is a strictly increasing or decreasing function(one-to-one function).

A non-monotonic function is strictly increasing over some portion andstrictly decreasing over another portion of its range.

12/49

Function


13/49

Monotonicity: Examples

x

f(x)f(x) =x2

x

f(x)

f(x) =x3

Note:

One-to-one function has an inverse function.

E.g., We write the inverse function ofy=f(x) =x3 as f1(y) =y1/3

13/49

Function


14/49

Function: Curvature

The curvature of a function refers to the change in the slope of a function.

In economics, we are mainly interested in two concepts: (strict) concavityand (strict) convexity.

A function f(x) is said to be concave if, for any value of such that0 1, the following identity holds for any two points x1 and x2 in

its domain:

f(x1+ (1 )x2) f(x1) + (1 )f(x2)

Similarly, a function f(x) is said to be convex if the following identityholds for any two points x1 and x2 in its domain:

f(x1+ (1 )x2) f(x1) + (1 )f(x2)

Strict concavity and strict convexity are the same definitions except thatwe use > and < to replace the or signs.

14/49

Function


15/49

Curvature: Examples

x

y

Concave

x

y

Strict concave

x

y

Convex

x

y

Strict convex

15/49

Function


16/49

Function: Continuity

A function f(x) is said to be continuous at x=a if

limxa

f(x) =f(a)

A function is said to be continuous on the interval [a, b], if it is continuous ateach point in the interval.

If f(x) is continuous at x=a then

limxa

f(x) = limxa+

f(x) =f(a)

16/49

Function


17/49

Discontinuity: Example

0 2 4 6 8 10

5

0

5

10

15

17/49

Calculus Derivative for Univariate Function


18/49

Definition

The derivative of a function represents an infinitesimal change in the functionwith respect to one of its variables.

Formally, given a function y=f(x), the derivative ofywith respect to x is

dydx

=f(x) = limx0

f(x+ x) f(x)

x

Given a function we call dy and dx differentials and the relationship between

them is given by,

dy=f(x)dx

18/49



19/49

Geometric interpretation

You may have seen different definitions of derivative written by different natural

languages. Understand how a (mathematical) concept try to describe the world, not

merely recite the words (and formula).19/49



20/49

Example: Given y=f(x) =x2, why f(x) =2x

Given a function f(x) =x2, we know f(x+ x) = (x+ x)2.

For any real number a and b, (a+b)2 =a2 +2ab+b2.

We now have f(x+ x) =x2 +2xx+ (x)2.

Use the definition dydx =f

(x) = limx0 f(x+x)f(x)

x

.

f(x) = limx0

f(x+x)f(x)

x

= lim

0

x2+2xx+(x)2x2

x

= lim

x0[2x+ x]

f(x) =2x, x 0 at limit

You can try to compute other derivative formulas. E.g., d(lnx)dx

=1x

, d(eax)

dx = aex.

Hope there is nothing until this point you have never seen before . . .

20/49

Calculus Derivative for Multivariate Function


21/49

Total Differentials: Definition

Given a multivariate function y=f(x1, x2, . . . , xn), its total differential dyis the changes in ythat are brought through changes in all the xi variables.

yxi

is called the partial derivative, which is the per-unit (marginal)impact of each xi changes on y.

Denote fi= yxi . The total differential (of the function) dycan be written as

dy=f1dx1+f2dx2+ +fndxn

Intuitively, the change in ymay come from any of the changes in each xi.

The magnitude of this change in xibrought to ycan be found bymultiplying the partial derivative y (which measures the impact on yof aninfinitesimally small change in xi) by the actual change dxi.

21/49



22/49

Total Differentials: Example

Consider the function

z=f(x, y) =5x2 +6xy+3y3

We can calculate its partial derivatives

z

x =10x+6y,

z

y =6x+9y2

Use its definition, dy=f1dx1+f2dx2+ +fndxn, the total differential is

dz= z

xdx+

z

ydy= (10x+6y)dx+ (6x+9y2)dy

22/49



23/49

Implicit Function Theorem: Definition

Some multivariate functions we need to deal with have the explicit formy=f(x1, x2, . . . , xn), i.e. the dependent variable (y) can be writtenexplicitly as a function of the independent variables (xi).

But it is not always possible to express yas an explicit function of all the xi.

Another toolkit we have is the implicit function, which writes dependentvariableyand all independent variables xitogether with the formF(y, x1, x2, . . . , xn) =0, The implicit function theorem tells us that

dydxi

= Fxi

Fy

23/49



24/49

Implicit Function Theorem: Example

Consider the function

F(x, y) =y3 2y2x3 +6y2x2 100= 0

Which is very difficult to rewrite in the explicit form y=f(x), and calculatedydx

.

We first calculate its partial derivatives

Fx= 6y2x2 +12y2x, Fy=3y

2 4yx3 +12yx2

Use the implicit function theorem,

dy

dxi =

Fxi

Fy(in this example there is onlyone independent variable x, thus xi=x)

dy

dx =

6y2x2 +12y2x

3y2 4yx3 +12yx2

24/49



25/49

Chain Rule: Definition & Example

Often we run across functions in which the independent variables of thefunction are themselves functions of another variable.

If we have z=f(y) and y=g(x), the chain rule tells us that thederivative ofz(with respect to x) is

dz

dx =

dz

dy

dy

dx

Example: Consider z= (3x+1)2

1 Let y=3x+1, thus z=y2;

2 dydx

=3, dzdy

=2y=2(3x+1);

3 Use the chain rule formula, dzdx

=6(3x+1).

25/49

Calculus


26/49

Epilogue: Calculus

The purpose is to help you recall some results you should have learned in your

undergraduate Calculus course.We only have time to cover the most basic results of differential calculus (wecompletely skip the integral calculus part).

If you are not familiar with these materials, you should at least read the notes byAkeela Weerapana (Lecture 1 - 5, see link in my website, Other resources), or any

undergraduate calculus textbooks.I personally prefer math books written by mathematician rather than economists.But if you would like to have an all-in-one cookbook, you can try to start withthe second reference book Mathematics for Economists by Simon and Blume.

Or, Fundamental Methods of Mathematical Economics by Alpha Chiang

(http://www.drchristiansalas.org.uk/BusinessandEconomics/Economics/FundamentalMethods.pdf)

You can find a dozen alternative books to fill the mathematical gap for studentsmoving from intermediate to more advanced economic studies. I do not knowwhether these two are the best textbooks that fit your demand. I list them onlybecause these are the ones I used during my first-year master.

26/49

Matrix Algerba Intro
http://www.drchristiansalas.org.uk/BusinessandEconomics/Economics/FundamentalMethods.pdfhttp://www.drchristiansalas.org.uk/BusinessandEconomics/Economics/FundamentalMethods.pdfhttp://www.drchristiansalas.org.uk/BusinessandEconomics/Economics/FundamentalMethods.pdfhttp://www.drchristiansalas.org.uk/BusinessandEconomics/Economics/FundamentalMethods.pdf


27/49

We now begin to review the matrix algebra part.

Since you will need sufficient knowledge and understanding of matrix algebrain your econometrics course instructed by Prof. Ralf Wikle, I will go throughthis part in more details.

Basic reading:

Appendix D(?): matrix algebra in Introductory Econometrics: AModern Approach by Jeffrey M. Wooldridge.

Lecture 6 - 7 by Akeela Weerapana.

27/49



28/49

Basic definitions & notations I

A matrix is a rectangular array of numbers.

A m n matrix has m rows and n columns.

The positive integer m is called the row dimension, and n is called thecolumn dimension.

A m n matrix can be written as

A = [aij] =

a11 a12 . . . a1na21 a22 . . . a2n

... ... . . . ...am1 am2 . . . amn

where aij represents the element in the ith row and the jth column.

28/49



29/49

Basic definitions & notations II

A square matrix has the same number of rows and columns m m. Thedimension of a square matrix is its number of rows and columns.

A 1 m matrix is called a row vector (of dimension m) and can be writtenas x= [x1, x2, . . . , xm].

A n 1 matrix is called a column vector

y =

y1y2...

yn

29/49


30/49



31/49

Basic definitions & notations IV

The n n identity matrix, denoted I, or sometimes In to emphasize itsdimension, is the diagonal matrix with unity (one) in each diagonal position,and zero elsewhere

A =

1 0 0 . . . 0

0 1 0 . . . 00 0 1 . . . 0...

......

. . . ...

0 0 0 . . . 1

The m n zero matrix, denoted 0, is the m n matrix with zero for allentries. This need not be a square matrix.

31/49



32/49

Basic definitions & notations V

A symmetric matrix is another special square matrix. A matrix A is said to besymmetric ifAij=Aji for all i and j

A = [aij] =

a11 a12 a13 . . . a1na21 a22 a23 . . . a2n

a31 a32 a33 . . . a3n...

......

. . . ...

an1 an2 an3 . . . ann

32/49


S l


33/49

Some examples

12 1 23 5426 78 351 8720 8 52 5760 67 35 129

1 3 92 4 8

1, 0005, 0009878123

xy

z

x 0 00 y 0

0 0 z

Alice Jens Sofie

1 0 0 00 1 0 00 0 1 00 0 0 1

12 1 201 78 8

20 8 52

33/49

Matrix Algerba Matrix Operations

T i i


34/49

Transposition

Let A= [aij] be a m n matrix. The transpose ofA, denoted A (called A

prime), is the n m matrix obtained by interchanging the rows and columns ofA. We can write this as A = [aji]

Example:

A =

1 3 92 4 8

A =

1 23 49 8

The transpose of a symmetric matrix equals to its prime, A =A.

Example:

A =

12 1 201 78 8

20 8 52

A =

12 1 201 78 8

20 8 52

34/49


M i Addi i


35/49

Matrix Addition

Two matrices Aand B, each having dimension m n, can be added element

by element: A+B= [aij+bij].

A+B =

a11+b11 a12+b12 . . . a1n+b1na21+b21 a22+b22 . . . a2n+b2n

......

. . . ...

am1+bm1 am2+bm2 . . . amn+bmn

Example:

1 3 92 4 8 +

20 5 26 3 9 =

21 8 118 7 17

As a result, matrix addition is legal only when performed with two matricesof identical dimensions, e.g. a 5 6 matrix can only be added to another5 6 matrix.

35/49


S l M lti li ti


36/49

Scalar Multiplication

Anymatrix can be multiplied by a scalar (i.e., a real number), q R.

IfA is a m n matrix and qis a scalar, the matrix that results frommultiplying A by the scalar q is a m n matrix B, where each element in B

is obtained by multiplying the corresponding element in A byq, i.e.Bij=q Aij.

Example:

8

1 3 92 4 8

=

8 24 7216 32 64

36/49


M t i M lti li ti D fi iti I


37/49

Matrix Multiplication: Definition I

Matrix multiplication is more sophisticated compare to the other basicoperations we have introduced.

When we multiply matrix A by matrix B (denoted AB) to obtain matrix C,a given element is obtained by multiplying the ithrowofA by the jth

columnofB.

Important: Multiplication of two matrices is legal only when the ith row ofthe first matrix has the same number of elements as the jth column of thesecond matrix.

The number of columns in A has to equal the number of rows in B.

37/49


Mat i M lti licatio Defi itio II


38/49

Matrix Multiplication: Definition II

Let Abe a m n matrix and let B be a n pmatrix. Then matrix multiplicationC= AB is defined as

C =

n

k=1aikbkj

for eachi=1, . . . ,m, j=1, . . . , p

wheren

k=1

aikbkj=ai1b1j+ai2b2j+ai3b3j+ +ainbnj.

In other words, the (i,j)th element of the new matrix C is obtained bymultiplying each element in the ith row ofA by the corresponding elementin the jth column ofB and adding these n products together.

38/49


Matrix Multiplication: Example


39/49

Matrix Multiplication: Example

Given

A =1 12 1

1 3

, B = 1 1 32 4 5

We can get

AB =

1 1+1 2 1 1+1 4 1 3+1 52 1+1 2 2 1+1 4 2 3+1 5

1 1+3 2 1 1+3 4 1 3+3 5

=

3 5 84 6 11

7 13 18

and

BA =

1 1+1 2+3 1 1 1+1 1+3 32 1+4 2+5 1 2 1+4 1+5 3

=

6 1115 21

39/49


Trace


40/49

Trace

The trace of a matrix is defined only for square matrices.

For any n n matrix A, the trace ofA, denoted tr(A), is the sum of its diagonalelements

tr(A) =

ni=1

aii

Example:

A =

4 32 1

, tr(A) =4+1= 5

40/49


Some rules I


41/49

Some rules I

Matrix Addition (A, B and C are all m n matrices)

A+B= B+A

(A+B) +C= A+ (B+C)

Matrix Multiplication (A is a m n matrix, B is n p, and C is p r)

(AB)C= A(BC)

A(B+C) =AB+AC

(A+B)C= AC+BC

AIn=ImA= A (a matrixA left or right multiply by an identity matrixequal to itself.)

41/49


Some rules II


42/49

Some rules II

Matrix Transpose (A, B arem n matrices, and q is a scalar)

(A+B) =A +B

(AB) =BA

(qA) =qA

Trace (A, B aren n square matrices)

tr(In) =n (In is an identity matrix)

tr(A) =tr(A)

tr(A+B) =tr(A) +tr(B)tr(qA) =q tr(A)

tr(CD) =tr(DC) (C is m n, D is n m )

42/49


Determinant: Definition I


43/49

Determinant: Definition I

The determinant of a square matrix is a unique number associated withthat matrix.

The determinant of a matrix is usually written by the name of the matrixsurrounded by two vertical bars, i.e., the determinant ofA is |A|.

The determinant of a 1 1 matrix (a scalar) is the scalar itself.

The determinant of a 2 2 matrix

A= a bc d

, |A| =ad cd

Question: How to compute matrix with n>2?

43/49


Determinant: Definition II


44/49

Determinant: Definition II

The determinant of a n n matrix is found using a procedure known as aLaplace expansion.

The Laplace expansion is a recursive process: the determinant of a n nmatrix can be expressed as a function of determinants of(n 1) (n 1)

submatrix matrices, each of which in turn are expressed as functions ofseveral (n 2) (n 2) submatrix matrices, and so on so forth.

Such decomposition is stopped when we are able to calculate thedeterminant of the submatrix matrices, such as 2 2 (or 3 3).

In order to write down the Laplace expansion formula, we need to define afew other terms: minor, cofactor.

44/49


Determinant: Definition III


45/49

Determinant: Definition III

The minor of a matrix is the determinant of a submatrix formed from a givenmatrix. In particular, the (i,j)th minor of a matrix, denoted as |Aij| is thedeterminant of a matrix obtained by omitting the ith row and jth column of theoriginal matrix.

Example:

GivenA =

2 1 23 5 41 8 2

, Some of the minors ofA are

A11 =

5 48 2 = 22, A12 =

3 41 2 =2, A13=

3 51 8 =19

Exercise: calculate the rest 6 minors ofA

45/49


Determinant: Definition IV


46/49

Determinant: Definition IV

The (i,j)th cofactor of a matrix is defined as |Cij| = (1)i+j |Aij|.

Note:

Co-factor is similar to the minor, except that it may have a different sign.

If the sum of i and j is an even number then the sign is positive, if the sumis an odd number, the sign is negative.

46/49


Determinant: Definition V


47/49

Determinant: Definition V

We are finally ready to write down the formula for the Laplace expansion of a

matrix A

IfA is a n n (square) matrix, then

|A| =N

k=1

aij|Cij|

When k=j, we say the expansion is along the ith row. Similarly, whenk=i, we say the expansion is along the jth column.

In other words, we can find the determinant of a matrix by picking any row,then multiplying each element of the row by the appropriate cofactor.

47/49


Determinant: Example


48/49

p

ConsiderA =

2 1 23 5 4

1 8 2

, we expanse A along the first row

|A| =2 (1)1+1 |A11| +1 (1)1+2 |A12| +2 (1)

1+3 |A13|

=2 (22) 1 2+2 19

= 8

48/49


Determinant: Properties


49/49

p

|A| = |A| (The determinant ofA is equal to the determinant of its

transpose)

|AB| = |A||B|

|qA| =qn|A|

|A| =0, ifA has a row (or column) that are all 0

If we multiply a row (or column) ofA byq, denote the new matrix B, then|B| =q|A|

If we multiply a row (or column) ofA byqand add it to another row (orcolumn), denote the new matrix B, then |B| = |A|

A is invertible if and only if|A| =0

If any row (or column) of a matrix is a linear combination of one or moreof the other rows of the matrix, then the matrix has a determinant of zero.(So matrices with linearly dependent rows or columns are not invertible.)

49/49

Documents

mathcamp_lec1 - kopia (5)