EssenceOfLinearAlgebra - 02

03-Matrices and Linear Transformations

#

Linear Transformations

#

You can think of it as a type of function, meaning there are outputs and inputs. You input a vector and output a vector. The intermediate calculations, the transformation process, reflect the motion laws of vectors.

Characteristics

#

1. Lines remain lines after transformation (including diagonals formed by vectors)
2. Origin position remains unchanged
3. Grid lines remain parallel and equidistant

Linear transformation

Non-linear transformation

Example

#

We can describe linear transformations through numerical values

Original basis vectors: $\hat{i} = \begin{bmatrix} 1 \ 0 \end{bmatrix}$, $\hat{j} = \begin{bmatrix} 0 \ 1 \end{bmatrix}$

When considering vector $\vec{v}$ with coordinates (-1, 2): $ \vec{v} = -1 \hat{i} + 2 \hat{j} = -1 \begin{bmatrix} 1 \ 0 \end{bmatrix} + 2 \begin{bmatrix} 0 \ 1 \end{bmatrix} = \begin{bmatrix} -1 \ 2 \end{bmatrix} $

If we apply some transformation, the basis vectors will move together with vector $\vec{v}$

Transformed basis vectors: $\hat{i}’ = \begin{bmatrix} 1 \ -2 \end{bmatrix}$, $\hat{j}’ = \begin{bmatrix} 3 \ 0 \end{bmatrix}$

$ \vec{v}’ = -1 \hat{i}’ + 2 \hat{j}’ = -1 \begin{bmatrix} 1 \ -2 \end{bmatrix} + 2 \begin{bmatrix} 3 \ 0 \end{bmatrix} = \begin{bmatrix} -1 \cdot 1 + 2 \cdot 3 \ -1 \cdot (-2) + 2 \cdot 0 \end{bmatrix} = \begin{bmatrix} 5 \ 2 \end{bmatrix} $

From this, we can conclude that we can infer the landing point of $\vec{v}$ after transformation through $\hat{i}’$ and $\hat{j}'$

A 2D linear transformation is completely determined by just four numbers, which are the coordinates of the transformed $\hat{i}’$ and $\hat{j}’$. Usually, we package these coordinates in a 2×2 grid, called a 2×2 matrix. By just inputting a vector $\begin{bmatrix} -1 \ 2 \end{bmatrix}$, we can obtain the transformed vector $\begin{bmatrix} 5 \ 2 \end{bmatrix}$ through $\begin{bmatrix} 1 & 3 \ -2 & 0 \end{bmatrix}$

Formula

#

$$ \begin{bmatrix} a & b \ c & d \end{bmatrix} \begin{bmatrix} x \ y \end{bmatrix} = x \begin{bmatrix} a \ c \end{bmatrix} + y \begin{bmatrix} b \ d \end{bmatrix} = \begin{bmatrix} ax + by \ cx + dy \end{bmatrix} $$

Common Matrices

#

Counterclockwise rotation 90° $\begin{bmatrix} 0 & -1 \ 1 & 0 \end{bmatrix}$

Shear $\begin{bmatrix} 1 & 1 \ 0 & 1 \end{bmatrix}$

Column linearly dependent $\begin{bmatrix} 2 & -2 \ 1 & -1 \end{bmatrix}$ (meaning one vector is a multiple of the other)

04-Matrix Multiplication and Linear Transformation Composition

#

Composition of Transformations

#

• Core definition: An ordered combination of multiple linear transformations, recording the transformation process by tracking the final positions of basis vectors

• Calculation example: First rotate (matrix R = $\begin{bmatrix} 0 & -1 \ 1 & 0 \end{bmatrix}$) then shear (matrix S = $\begin{bmatrix} 1 & 1 \ 0 & 1 \end{bmatrix}$), composite matrix S·R = $\begin{bmatrix} 1 & -1 \ 1 & 0 \end{bmatrix}$

• Order rule: Matrix multiplication is read from right to left (like f(g(x))), derived from the logical order of function composition

Matrix Multiplication Calculation

#

Let $M_1$ = $\begin{bmatrix} 1 & -2 \ 1 & 0 \end{bmatrix}$, $M_2$ = $\begin{bmatrix} 0 & 2 \ 1 & 0 \end{bmatrix}$

Path of basis vector $\hat{i}’$: $M_1$’s $\hat{i}’$ transformed by $M_2$:

1 × $\begin{bmatrix} 0 \ 1 \end{bmatrix}$ + 1 × $\begin{bmatrix} 2 \ 0 \end{bmatrix}$ = $\begin{bmatrix} 0 & 2 \ 1 & 0 \end{bmatrix}$ = $\begin{bmatrix} 0 + 2 \ 1 + 0 \end{bmatrix}$ = $\begin{bmatrix} 2 \ 1 \end{bmatrix}$

Path of basis vector $\hat{j}’$: $M_1$’s $\hat{j}’$ transformed by $M_2$:

(-2) × $\begin{bmatrix} 0 \ 1 \end{bmatrix}$ + 0 × $\begin{bmatrix} 2 \ 0 \end{bmatrix}$ = $\begin{bmatrix} 0 & 0 \ -2 & 0 \end{bmatrix}$ = $\begin{bmatrix} 0 + 0 \ -2 + 0 \end{bmatrix}$ = $\begin{bmatrix} 0 \ -2 \end{bmatrix}$

Composite matrix: $\begin{bmatrix} 2 & 0 \ 1 & -2 \end{bmatrix}$

General Formula

#

$M_2 = \begin{bmatrix} a & b \ c & d \end{bmatrix}, M_1 = \begin{bmatrix} e & f \ g & h \end{bmatrix}$

$M_2M_1 = \begin{bmatrix} ae + bg & af + bh \ ce + dg & cf + dh \end{bmatrix}$

Operation Properties

#

• Does not satisfy commutativity: $M_1M_2 ≠ M_2M_1$
• Satisfies associativity: (AB)C = A(BC)

Extension in 3D Space

#

Additional basis vector: Reference the basis vector $\hat{k}’$ in the Z-axis direction, coordinates $\begin{bmatrix} 0 \ 0 \ 1 \end{bmatrix}$

• Linear transformation is determined by the positions of $\hat{i}’, \hat{j}’, \hat{k}’$ after transformation

• Vector transformation calculation rules are the same as in 2D

$M_1 \vec{v} = x \begin{bmatrix} a \ b \ c \end{bmatrix} + y \begin{bmatrix} d \ e \ f \end{bmatrix} + z\begin{bmatrix} g \ h \ i \end{bmatrix}$

05-Determinant

#

Geometric Essence of Determinant

#

The determinant describes the scaling ratio of volume or area by linear transformation

The sign represents whether the space is inverted

Right-Hand Rule

#

We usually use the right-hand rule, where the ring finger pointing forward represents $\hat{i}’$, the middle finger pointing sideways represents $\hat{j}’$, and the thumb pointing upward represents $\hat{k}’$. If we use the left hand to represent this space, it means the space orientation is flipped, and the determinant is negative

Special Cases

#

Determinant = 0: Space is compressed to lower dimensions

Determinant = 1: Volume remains unchanged

Formula

#

$$ \det \begin{bmatrix} a & b \ c & d \end{bmatrix} = ad - bc $$

ad: Represents the area of the rectangle formed by scaled basis vectors

bc: Represents how much the parallelogram is stretched or compressed along the diagonal direction

Example

#

$$ \det(M_1M_2) = \det(M_1) \cdot \det(M_2) $$

Assuming we transform a unit square, M1 scales by 2 times, M2 scales by 4 times,

$\det(M_1M_2)$ can be understood as first scaling by 4 times then scaling by 2 times, i.e., 1 × (4 × 2),

$\det(M_1) \cdot \det(M_2)$ can be understood as 1 × 2 × 4,

The two are equal

06-Inverse Matrix, Column Space, and Null Space

#

System of Linear Equations

#

A system of linear equations refers to a collection containing several unknowns and related equations. Each equation has unknowns with only constant coefficients, and these unknowns only undergo addition operations. Such systems can be uniformly represented as vector equations:

Example:

$$ \begin{cases} 2x + 5y + 3z = -3 \ 4x + 0y + 8z = 0 \ 1x + 3y + 0z = 2 \end{cases} $$

Can be written in matrix form:

$$ \begin{bmatrix} 2 & 5 & 3 \ 4 & 0 & 8 \ 1 & 3 & 0 \end{bmatrix} \begin{bmatrix}x \ y \ z\end{bmatrix} = \begin{bmatrix}-3 \ 0 \ 2\end{bmatrix} $$

Where the left matrix is $A$, the unknown vector is $X$, the right side is $V$, i.e., $AX = V$.

Concept of Inverse Matrix

#

• The inverse matrix $A^{-1}$ can be understood as a tool to “restore” transformations.
• If $AX = V$, then $X = A^{-1}V$, meaning we can solve for the original unknowns through the inverse matrix.
• The inverse matrix only exists when $A$ is a square matrix and $\det(A) \neq 0$.
• If the matrix “compresses” the space (reduces dimensions), the inverse matrix doesn’t exist.
• Special case: If after dimension reduction, a certain vector happens to fall on the target space, this vector can be restored by inverse transformation.

Terminology Explanation

#

• Rank: The dimension after space transformation. If transformed to 1D, rank is 1; if 2D, rank is 2; and so on.
• Column Space: The set of all possible output vectors, i.e., the space formed by $Av$, spanned by the column vectors of matrix $A$.
• Difference between Span and Column Space:
  1. Span refers to all possible results of linear combinations of any vector set, while column space is the set of all possible results after multiplying a matrix with any vector.
  2. Span can be applied to any vector set, while column space is directly related to the existence of solutions to the linear system $Ax = v$.
• Full Rank: Rank equals the number of columns.
• Zero Vector: Vector with coordinates (0, 0). When full rank, the zero vector is the origin; when not full rank, multiple vectors become zero vectors.
• Null Space/Kernel: The set of all vectors that land at the origin after transformation.

6-Supplementary Note: Non-Square Matrices

#

• Non-square matrices are matrices where the number of rows and columns are not equal ($m \neq n$).
• Examples:
  • $\begin{bmatrix}2 & -1 & 2 \ 0 & 1 & 1\end{bmatrix}$: 2 rows 3 columns, represents mapping from 3D space to 2D space (e.g., points in 3D space are projected onto a plane).
  • $\begin{bmatrix}3 & 1 \ 1 & 5 \ 4 & 9\end{bmatrix}$: 3 rows 2 columns, represents mapping from 2D space to 3D space (e.g., points on a 2D plane are stretched into 3D space).
  • $[1, 2]$: 1 row 2 columns, represents mapping from 2D space to 1D space (e.g., points in 2D space are compressed onto a line).