mathematical methods - lecture 1 - inria · pdf filemathematical methods - lecture 1 ......

Mathematical Methods - Lecture 1

Yuliya Tarabalka

Inria Sophia-Antipolis Mediterranee, Titane team,http://www-sop.inria.fr/members/Yuliya.Tarabalka/

Tel.: +33 (0)4 92 38 77 09email: yuliya.tarabalka@inria.fr

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 1 / 31

Outline

1 Introduction to the Course

2 Linear Algebra - Introduction

3 Gaussian Elimination

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 2 / 31

Introduction to the Course

Methods course details

Course title: Mathematical Methods

Course lecturer: Yuliya Tarabalka (yuliya.tarabalka@inria.fr)

Lectures:

3 hours × 8 timesFrom Sept. 11 to Sep. 22

Assessment at the end

Recommended books:

Mathematical methods for science students. G Stephenson, 1973.Analysis: with an introduction to proof. Steven R Lay, 2005.Mathematical methods for physics and engineering. K. F. Riley, M. P.Hobson and S. J. Bence, 2006.Introduction to differential equations. J. R. Chasnov, 2016.Linear algebra in twenty five lectures. T. Denton and A. Waldron, 2012.Tons of materials online.

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 3 / 31

Introduction to the Course

Goals of the course

Revise and learn the most important mathematical concepts

Maths underpins most computing concepts/applications in datascience & business analystics, e.g.:

stock market modelsinformation search and retrievalcomputer visionneural computing...

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 4 / 31

Linear Algebra - Introduction

What is Linear Algebra?

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 5 / 31

Linear Algebra - Introduction

What is Linear Algebra?

Linear algebra is the branch of maths concerning vector spaces andlinear mappings between such spaces [Wikipedia]

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 6 / 31

Linear Algebra - Introduction

Example 1

n people in this room

I ask everybody to rate the likeability of everybody else on a scalefrom 1 to 10

There are n2 ratings to keep track of

We could arrange these in a square array 9 4 · · ·10 6...

. . .

We can replace this array by an abstract symbol M, called a matrix

M is an example of a more general abstract structure called lineartransformation

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 7 / 31

Linear Algebra - Introduction

Example 1

n people in this room

I ask everybody to rate the likeability of everybody else on a scalefrom 1 to 10

There are n2 ratings to keep track of

We could arrange these in a square array 9 4 · · ·10 6...

. . .

We can replace this array by an abstract symbol M, called a matrix

M is an example of a more general abstract structure called lineartransformation

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 7 / 31

Linear Algebra - Introduction

Example 1

n people in this room

I ask everybody to rate the likeability of everybody else on a scalefrom 1 to 10

There are n2 ratings to keep track of

We could arrange these in a square array 9 4 · · ·10 6...

. . .

We can replace this array by an abstract symbol M, called a matrix

M is an example of a more general abstract structure called lineartransformation

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 7 / 31

Linear Algebra - Introduction

Example 1

The theory of linear transformations is incredibly useful!

Think that we can replace “likebility ratings” with the number of timesinternet websites link to one another

Three main topics in the Linear Algebra course:

Linear SystemsVector SpacesLinear Transformations

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 8 / 31

Linear Algebra - Introduction

Example 2

I have x apples and y oranges

To keep track of the number of apples &oranges:

We put them in the list (x , y)

We call this list a vector

We can represent this vector with a point in a 2-D plane with thecorresponding coordinates

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 9 / 31

Linear Algebra - Introduction

Example 2

I have x apples and y oranges

To keep track of the number of apples &oranges:

We put them in the list (x , y)

We call this list a vector

We can represent this vector with a point in a 2-D plane with thecorresponding coordinates

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 9 / 31

Linear Algebra - Introduction

Example 2

In the plane, we can imagine each point as some combination ofapples & oranges (or parts thereof for non-int coordinates)

Then each point corresponds to some vector

The collection of all such vectors is an example of a vector space

Vector space (also called a linear space) is a collection of objects calledvectors, which may be added together and multiplied by numbers

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 10 / 31

Linear Algebra - Introduction

Example 2

In the plane, we can imagine each point as some combination ofapples & oranges (or parts thereof for non-int coordinates)

Then each point corresponds to some vector

The collection of all such vectors is an example of a vector space

Vector space (also called a linear space) is a collection of objects calledvectors, which may be added together and multiplied by numbers

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 10 / 31

Linear Algebra - Introduction

Exercise

There are 27 pieces of fruit in a barrel, and twice as many oranges asapples. How many oranges and apples in the barrel?

We can re-write the question mathematically:

x + y = 27 (1)

y = 2x (2)

This is an example of a linear systemf1(x1, · · · , xm) = a1

...

fn(x1, · · · , xm) = an

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 11 / 31

Linear Algebra - Introduction

Exercise

There are 27 pieces of fruit in a barrel, and twice as many oranges asapples. How many oranges and apples in the barrel?

We can re-write the question mathematically:

x + y = 27 (1)

y = 2x (2)

This is an example of a linear systemf1(x1, · · · , xm) = a1

...

fn(x1, · · · , xm) = an

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 11 / 31

Linear Algebra - Introduction

Exercise

There are 27 pieces of fruit in a barrel, and twice as many oranges asapples. How many oranges and apples in the barrel?

We can re-write the question mathematically:

x + y = 27 (1)

y = 2x (2)

This is an example of a linear systemf1(x1, · · · , xm) = a1

...

fn(x1, · · · , xm) = an

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 11 / 31

Linear Algebra - Introduction

Linear function

Definition: Let X and Y be vectors and α and β be scalars. Afunction is linear if

f (αX + βY ) = αf (X ) + βf (Y )

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 12 / 31

Linear Algebra - Introduction

Exercise

There are 27 pieces of fruit in a barrel, and twice as many oranges asapples. How many oranges and apples in the barrel?

We can re-write the question mathematically:

x + y = 27

y = 2x

By manipulating the equations:

x + y = 27

−2x + y = 0

⇒ 3x = 27 ⇒ x = 9

y = 18

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 13 / 31

Linear Algebra - Introduction

Exercise

Let us express this set of equations

x + y = 27

2x − y = 0

with a matrix: (1 12 −1

)(xy

)=

(270

)

To get back the linear system, we apply the multiplication rule:(a bc d

)(xy

)=

(ax + bycx + dy

)

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 14 / 31

Linear Algebra - Introduction

Exercise

Let us express this set of equations

x + y = 27

2x − y = 0

with a matrix: (1 12 −1

)(xy

)=

(270

)

To get back the linear system, we apply the multiplication rule:(a bc d

)(xy

)=

(ax + bycx + dy

)

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 14 / 31

Linear Algebra - Introduction

Exercise

Let us express this set of equations

x + y = 27

2x − y = 0

with a matrix: (1 12 −1

)(xy

)=

(270

)

The matrix is an example of a linear transformation: it takes onevector and turns it into another in a linear way

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 15 / 31

Linear Algebra - Introduction

Next task:

Solve linear systems

⇓

We’ll learn a general method called Gaussian Elimination

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 16 / 31

Gaussian Elimination

Notation for linear systems

We’ve found that we can express the linear system

x + y = 27

2x − y = 0

as(

1 12 −1

)(xy

)=

(270

)Likewise, we can write the solution as:

(1 00 1

)(xy

)=

(9

18

)

I =

(1 00 1

)is called the Identity Matrix: Iv = v

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 17 / 31

Gaussian Elimination

Notation for linear systems

We’ve found that we can express the linear system

x + y = 27

2x − y = 0

as(

1 12 −1

)(xy

)=

(270

)Likewise, we can write the solution as:(

1 00 1

)(xy

)=

(9

18

)

I =

(1 00 1

)is called the Identity Matrix: Iv = v

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 17 / 31

Gaussian Elimination

Augmented matrix notation

A useful shorthand for a linear system is an Augmented Matrix:

(1 12 −1

)(xy

)=

(270

) ⇒ (1 1 272 −1 0

)

The solution to the linear system looks like this:

(1 00 1

)(xy

)=

(9

18

) ⇒

(1 0 90 1 18

)

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 18 / 31

Gaussian Elimination

Augmented matrix notation

A useful shorthand for a linear system is an Augmented Matrix:

(1 12 −1

)(xy

)=

(270

) ⇒ (1 1 272 −1 0

)

The solution to the linear system looks like this:

(1 00 1

)(xy

)=

(9

18

) ⇒(

1 0 90 1 18

)

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 18 / 31

Gaussian Elimination

Augmented matrix notation

Another example of an augmented matrix: 1 3 2 0 96 2 0 −2 0−1 0 1 1 3

General case: a1

1 a12 · · · a1

k b1

a21 a2

2 · · · a2k b2

......

......

ar1 ar2 · · · ark br

number of rows r = number of equationsnumber of columns k = number of unknowns

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 19 / 31

Gaussian Elimination

Augmented matrix notation

Another example of an augmented matrix: 1 3 2 0 96 2 0 −2 0−1 0 1 1 3

General case:

a11 a1

2 · · · a1k b1

a21 a2

2 · · · a2k b2

......

......

ar1 ar2 · · · ark br

number of rows r = number of equationsnumber of columns k = number of unknowns

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 19 / 31

Gaussian Elimination

Idea of Gaussian elimination (equivalence relations)

From the example:

(1 1 272 −1 0

) ⇒ (1 0 90 1 18

)

Idea:

Turn a general augmented matrix into a simple augmented matrixconsisting of the identity matrix on the left and a bunch of numbers(solution) on the right

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 20 / 31

Gaussian Elimination

Equivalence relations

From the example:

(1 1 272 −1 0

) ⇒ (1 0 90 1 18

)

Two augmented matrices that actually have solutions are said to be(row) equivalent if they have the same solutions

(1 1 272 −1 0

) ∼ (1 0 90 1 18

)

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 21 / 31

Gaussian Elimination

Reduced Row Echelon Form

Reduced Row Echelon Form (RREF) = canonical form foraugmented matrices:

1 ∗ 0 ∗ 0 · · · 0 b1

0 1 ∗ 0 · · · 0 b2

0 0 1 · · · 0 b3

......

... 0...

1 bk

0 0 0 · · · 0 0...

......

......

0 0 0 · · · 0 0

Pivot = the first non-zero entry in each rowThe asterisks = arbitrary content (1 or several columns long)In RREF, the pivot is always 1The pivot is the only non-zero entry in its columnThe pivot of any row is always to the right of the pivot of the rowabove it

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 22 / 31

Gaussian Elimination

Reduced Row Echelon Form

Reduced Row Echelon Form (RREF) = canonical form foraugmented matrices:

1 ∗ 0 ∗ 0 · · · 0 b1

0 1 ∗ 0 · · · 0 b2

0 0 1 · · · 0 b3

......

... 0...

1 bk

0 0 0 · · · 0 0...

......

......

0 0 0 · · · 0 0

Theorem: Every augmented matrix is row-equivalent to a uniqueaugmented matrix in RREF

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 23 / 31

Gaussian Elimination

Reduced Row Echelon Form: Example

RREF: 1 0 7 0 40 1 3 0 10 0 0 1 20 0 0 0 0

We can read off the solution set directly from the RREF:

xyzw

=

4102

+ λ

−7−310

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 24 / 31

Gaussian Elimination

Gauss-Jourdan (Gaussian) elimination

Idea: Begin with an arbitrary matrix & apply operations that respectrow equivalence until we have RREF

3 elementary row operations:

1 (Row Swap) Exchange any two rows

2 (Scalar Multiplication) Multiply any row by a non-zero constant

3 (Row Sum) Add a multiple of one row to another row

Theorem: Gauss-Jourdan Elimination produces a unique augmentedmatrix in RREF

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 25 / 31

Gaussian Elimination

Gauss-Jourdan (Gaussian) elimination

Idea: Begin with an arbitrary matrix & apply operations that respectrow equivalence until we have RREF

3 elementary row operations:

1 (Row Swap) Exchange any two rows

2 (Scalar Multiplication) Multiply any row by a non-zero constant

3 (Row Sum) Add a multiple of one row to another row

Theorem: Gauss-Jourdan Elimination produces a unique augmentedmatrix in RREF

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 25 / 31

Gaussian Elimination

Gaussian elimination - Example

3x3 = 9x1 + 5x2 − 2x3 = 2

13x1 + 2x2 = 3

∼ 0 0 3 9

1 5 −2 213 2 0 3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 26 / 31

Gaussian Elimination

Gaussian elimination - Example

3x3 = 9x1 + 5x2 − 2x3 = 2

13x1 + 2x2 = 3

∼ 0 0 3 9

1 5 −2 213 2 0 3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 26 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 27 / 31

Gaussian Elimination

Gaussian elimination - Example

R1 ↔ R3

3R1

R2 =R2 − R1

−R2

0 0 3 91 5 −2 213 2 0 3

1 6 0 91 5 −2 20 0 3 9

1 6 0 90 1 2 70 0 3 9

R1 =R1 − 6R2

1/3R3

R1 =R1 + 12R3

R2 =R2 − 2R3

1 0 −12 −330 1 2 70 0 1 3

1 0 0 30 1 0 10 0 1 3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 28 / 31

Gaussian Elimination

Gaussian elimination - Example

3x3 = 9x1 + 5x2 − 2x3 = 2

13x1 + 2x2 = 3

∼ 0 0 3 9

1 5 −2 213 2 0 3

1 0 0 30 1 0 10 0 1 3

⇒Solution:x1 = 3x2 = 1x3 = 3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 29 / 31

Gaussian Elimination

Gaussian elimination - Example

3x3 = 9x1 + 5x2 − 2x3 = 2

13x1 + 2x2 = 3

∼ 0 0 3 9

1 5 −2 213 2 0 3

1 0 0 30 1 0 10 0 1 3

⇒Solution:x1 = 3x2 = 1x3 = 3

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 29 / 31

Gaussian Elimination

Gaussian elimination - Example 2

1 0 −1 2 −11 1 1 −1 20 −1 −2 3 −35 2 −1 4 1

R2 − R1;R4 − 5R1

1 0 −1 2 −10 1 2 −3 30 −1 −2 3 −30 2 4 −6 6

R3 + R2;R4 − 2R3

1 0 −1 2 −10 1 2 −3 30 0 0 0 00 0 0 0 0

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 30 / 31

Gaussian Elimination

Gaussian elimination - Example 2

1 0 −1 2 −10 1 2 −3 30 0 0 0 00 0 0 0 0

Variables x3 and x4 are undetermined

The solution is not unique

Let’s set x3 = λ and x4 = µ. The solution is set is:x1

x2

x3

x4

=

−1300

+ λ

1−210

+ µ

−2301

Yuliya Tarabalka (yuliya.tarabalka@inria.fr) Introduction 31 / 31