Notes on Mathematical Methods for Physicists Chapter2

Notes on Mathematical Methods for Physicists$\S2$

Chapter2 Determinants & Matrices

$\S2.1$Determinants

We begin the study of matrices by solving linear equations that will lead us to determinants and matrices. The concept of determinant and the notation were introduced by the renowned German mathematician and philosopher$Gottfried\,\,\,Wilhelm\,\,\,von\,\,\,Leibniz$.

$\square$Homogeneous Linear Equations

Suppose three unknowns$x_1,x_2,x_3$(or$n$equations with$n$unknowns) :

The problem is to determine under what conditions there is any solution , apart from the trivial one$x_1=x_2=x_3=0$. Using vectors , we have$\vec{a}\cdot\vec{x}=0\,,\,\vec{b}\cdot\vec{y}=0\,,\,\vec{c}\cdot\vec{z}=0$. These three vector equations have the geometrical intepretation that$x$is orthogonal to$\vec{a}\,,\,\vec{b}\,,\,\vec{c}$.

If the volume spanned by$\vec{a}\,,\,\vec{b}\,,\,\vec{c}$given by determinant (or triple scalar product)

is not zero , then there is the only trivial solution$\vec{x}=0$.

Conversely , if the aforementional determinant of the coefficient vanishes , then one of the row vectors is a combination of the other two.$\vec{c}\subseteq plane(\vec{a}\cross\vec{b})$, only ratios of$x_i$are relevant.

This is Cramer's Rule for homogeneous linear equation.

$\square$Inhomogeneous Linear Equation

Simple example :

This is Cramer's Rule for inhomogeneous linear equation.

$\square$Definitions

Before defining a determinant , we need to introduce some related concepts and definitions.

$\bullet$When we write two-dimensional ($2-D$) arrays of items , we identify the item in the$n$th horizontal row and the$m$th vertical column by the index set$n,m$; note that the row index is conventionally written first.

$\bullet$Starting from a set of$n$objects in some reference order (e.g. , the number sequence$1,2,3,\cdots,n$) , we can make a permutation of them to some other order ; the total number of distinct permutations that are possible is$n!$(choose the first object$n$ways , then choose the second in$n-1$ways , etc.).

$\bullet$Every permutation of$n$objects can be reached from the reference order by a succession of pairwise interchanges (e.g. ,$1234\to4132$can be reached by the successive steps$1234\to1432\to4132$). Although the number of pairwise interchanges needed for a given permutation depends on the path (compare the above example with$1234\to1243\to1423\to4123\to4132$) , for a given permutation the number of interchanges will always either be even or odd. Thus a permutation can be identified as having either even or odd parity.

$\bullet$It is convenient to introduce the Levi-Civita symbol , which for an$n$-object system is denoted by$\varepsilon_{ij...}$, where$\varepsilon$has$n$subscripts , each of which identifies one of the objects.

We now define a determinant of order$n$to be an$n\cross n$square array of numbers (or functions) , with the array conventionally written within vertical bars (not parentheses , braces , or any other type of brackets) , as follows :

The determinant$D_n$has a value$D_n=\underset{ij...}{\sum}\varepsilon_{ij...}a_{1i}a_{2j}\cdots$.

$\square$Properties of Determinants

Take determinants of order$3$for example.

$\bullet$Any determinant with two rows equal , or two columns equal , has the value zero. To prove this , interchange the two identical rows or columns ; the determinant both remains the same and changes sign , and therefore must have the value zero.

$\bullet$An extension of the above is that if two rows (or columns) are proportional , the determinant is zero.

$\bullet$The value of a determinant is unchanged if a multiple of one row is added (column by column) to another row or if a multiple of one column is added (row by row) to another column.

$\bullet$If each element in a row or each element in a column is zero , the determinant has the value zero.

$\square$Laplacian Development by Minor

The fact that a determinant of order$n$expands into$n!$terms means that it is important to identify efficient means for determinant evaluation. One approach is to expand in terms of minors. The minor corresponding to$a_{ij}$, denoted$M_{ij}$, or$M_{ij}(a)$if we need to identify$M$as coming from the$a_{ij}$, is the determinant (of order$n-1$) produced by stiking out row$i$and column$j$of the original determinant. And we get

$\square$Linear Equation Systems

For equation

We define

Then we have

This is the Cramer's Rule.

If$D$is nonzero , the above construction of the$x_i$is definitive and unique , so that there will be exactly one solution to the equation set.

$\square$Determinants & Linear Dependence

If the coefficients of$n$linear forms in$n$variables form a nonzero determinant , the forms are linearly independent ; if the determinant of the coefficients is zero , the forms exhibit linear dependence.

$\square$Linearly Dependent Equations

Situation$1$

All the equations are homogeneous (which means all the right hand side quantities$h_i$are zero). Then , one or more of the equations in the set will be equivalent to linear combinations of others , and we will have less than$n$equations in our$n$variables. We can then assign one (or in some cases , more than one) variable an arbitrary value , obtaining the others as functions of the assigned variables. We thus have a manifold (i.e. , a parameterized set) of solutions to our equation system.

Situation$2$

A second case is where we have (or combine equations so that we have) the same linear form in two equations , but with different values of the right-hand quantities$h_i$. In that case the equations are mutually inconsistent , and the equation system has no solution.

Situation$3$

A third , related case , is where we have a duplicated linear form , but with a common value of$h_i$. This also leads to a solution manifold.

$\square$Numerical Evaluation

There are many methods to evaluate determinants , even using computers. We use the Gauss Elimination to calculate determinants , which is a versatile procedure that can be used for evaluating determinants, for solving linear equation systems, and (as we will see later) even for matrix inversion.

Gauss Elimination : make the determinant into a form that all the entries in the lower triangle of the determinant. Then the only effective part is the product of thediagonal elements.

$\S2.2$Matrices

Matrices are$2-D$arrays of numbers or functions that obey the laws that define matrix algebra.

$\square$Basic Definitions

A matrix is a set of numbers or functions in a$2-D$square or rectangular array. A matrix with$m$(horizontal) rows and$n$(vertical) columns is known as an$m\cross n$matrix. When we introduced determinants , when row and column indices or dimensions are mentioned together , it is customary to write the row indicaters first.

A matrix for which$n=m$is termed square; One consisting of a single column (an$m\cross1$matrix) is often called a column vector , while a matrix with only one row (therefore$1\cross n$) is a row vector.

$\square$Equality

If$A$and$B$are matrices ,$A=B$only if$a_{ij}=b_{ij}$for all values of$i$and$j$. A necessary but not sufficient condition for equality is that both matrices have the same dimensions.

$\square$Addition , Subtraction

Addition and subtraction are defined only for matrices$A$and$B$of the same dimensions , in which case$A\pm B=C$, with$c_{ij}=a_{ij}\pm b_{ij}$for all values of$i$and$j$. Addition is commutative ($A+B=B+A$) and also associative ($(A+B)+C=A+(B+C)$). A matrix with all elements zero , called a null matrix or zero matrix , can either be written as$O$or as a simple zero. Thus for all$A$,

$\square$Multiplication (by a Scalar)

Here we have$B=\alpha A$, with$b_{ij}=\alpha a_{ij}$for all values of$i$and$j$. This operation is commutative , with$\alpha A=A\alpha$.

Note that the definition of multiplication by a scalar causes each element of marix$A$to be multiplied by the scalar factor , so there is

$\square$Matrix Multiplication (Inner Product)

Matrix multiplication is not an element-by-element operation like addition or multiplication by a scalar. The inner product of matrices$A$and$B$is defined as

This definition causes the$ij$element of$C$to be formed from the entire$i$th row of$A$and the entire$j$th column of$B$. And as you can realize ,$AB\neq BA$.

It is useful to define the commutator of$A$and$B$,

which , as stated above , will in many cases be nonzero.

But , matrix multiplication is associative , meaning that$A(BC)=(AB)C$.

$\square$Unit Matrix

By direct matrix multiplication , it is possible to show that a square matrix with elements of value unity on its principal diagonal (the elements$(i,j)$with$i=j$) , and zeros everywhere else , will leave unchanged any matrix with which it can be multiplied. For example , the$3\cross3$unit matrix has the form

note that it is not a matrix all of whose elements are unity. Giving such a matrix the name$1$,

Remember that$A$must be$n\cross n$.

The previously introduced null matrices have only zero elements , so it is also obvious that for all$A$,

 

$\square$Diagonal Matrices

If a matrix$D$has nonzero elements$d_{ij}$only for$i=j$, it is said to be diagonal.

$\square$Matrix Inverse

It will often be the case that given a square matrix$A$, there will be a square matrix$B$such that$AB=BA=1$. A matrix$B$with this property is called the inverse of$A$and is given the name$A^{-1}$. If$A^{-1}$exists , it must be unique.

Every nonezero real (or complex) number$\alpha$has a nonzero multiplicative inverse , often written$1/\alpha$. But the corresponding property does not hold for matrices ; there exist nonzero matrices that do not have inverses. To demonstrate this , consider the following :

If$A$has an inverse , we can multiply the equation$AB=O$on the left by$A^{-1}$, thereby obtaining

Since we started with a matrix$B$that was nonzero , this is an inconsistency , and we are forced to conclude that$A^{-1}$does not exist. A matrix without an inverse is said to be singular , so our conclusion is that$A$is singular. Note that in our derivation , we had to be careful to multiply both members of$AB=O$from the left , because multiplication is noncommutative. Alternatively , assuming$B^{-1}$to exist , we could multiply this equation on the right by$B^{-1}$, obtaining

This is inconsistent with the nonzero$A$with which we started ; we conclude that$B$is also singular. Summerizing , there are nonzero matrices that do not have inverses and are identified as singular.

The algebraic properties of real and complex numbers (including the existence of inverses for all nonzero numbers) define what mathematicians call a field. The properties we have identified for matrices are different ; they form what is called a ring.

A closed , but cumber-some formula for the inverse of a matrix exists ; it expresses the elements of$A^{-1}$in terms of the determinants that are the minors of$det(A)$. That formula , the derivation of which is in several of the Additional Readings , is

We describe here a well-known method that is computationally more efficient than the equation above , namely the Gauss-Jordan procedure.

Example Gauss-Jordan Matrix Inversion

The Gauss-Jordan method is based on the fact that there exist matrices$M_L$such that the product$M_LA$will leave an arbitrary matrix$A$unchanged , except with ​ (a) one row multiplied by a constant , or ​ (b) one row replaced by the original row minus a multiple of another row , or ​ (c) the interchange of two rows.

By using these transformations , the rows of a matrix can be altered (by matrix multiplication) in the same way as we did to the elements of determinants. If$A$is nonsingular , we change both side of the equation to reduce$A$to$1$, then we get$A^{-1}$:

What we need to do is to find out how to reduce$A$to$1$using the method which we have used to determinants. Here is a concrete example :

Write , side by side , the matrix$A$and a unit matrix of the same size , and perform the same operations on each until$A$has been converted to a unit matrix , which means that the unit matrix will have been changed into$A^{-1}$:

Multiply the rows as necessary to set to unity all elments of the first column of the left matrix ,

Subtracting the first row from the second an third rows , we obtain

Divide the second row by$\frac{5}{6}$and subtract$\frac{2}{3}$times it from the first row and$\frac{1}{3}$times it from the third row ,

Divide the third row by$\frac{18}{5}$. Then as the last step ,$\frac{1}{5}$times the third row is subtracted from each of the first two rows. Our final pair is

$\square$Derivatives of Determinants

The formula giving the inverse of a matrix in terms of its minors enables us to write a compact formula for the derivative of a determinant$det(A)$where the matrix$A$has elements that depend on some variable$x$. To carry out the differentiation with respect to the$x$dependence of its element$a_{ij}$, we write$det(A)$as its expansion in minors$M_{ij}$about the elements of row$i$, so we have

Applying now the chain rule to allow for the$x$dependence of all elements of$A$, we get

 

$\square$Systems of Linear Equations

Note that if$A$is a$n\cross n$square matrix , and$\vec{x}$and$\vec{h}$are$n\cross1$column vectors. Consider the matrix equation$A\vec{x}=\vec{h}$, which is equivalent to a system of$n$lenear equations.

This tells us two things : ​ (a) that if we can evaluate$A^{-1}$, we can compute the solution$\vec{x}$; ​ (b) and that the existence of$A^{-1}$means that this equation system has a unique solution.

Then the result is important enough to be emphasized : A square matrix$A$is singular if and only if$det(A)=0$.

$\square$Determinant Product Theorem

The Product Theorem is that$det(A\,B)=det(A)\,det(B)$. The proof of this theorem is dull , I would just skip it.

Note that$det(A^{-1})=\frac{1}{det(A)}$.

$\square$Rank of a Matrix

The concept of a matrix singularity can be refined by introducing the notion of the rank of a matrix. If the elements of a matrix are viewed as the coefficients of a set of linear forms , a square matrix is assigned a rank equal to the number of linearly independent forms that its elements describe. Thus , a nonsingular$n\cross n$matrix will have rank$n$, while a$n\cross n$singular matrix will have a rank$r$less$n$. The rank provides a measure of the extent of the singularity ; if$r=n-1$, the matrix describes one linear form that is dependent on the others , etc. Further and more systematical discussion will be seen in Chapter 6.

$\square$Transpose , Adjoint , Trace

Transpose

The transpose of a matrix is the matrix that results from interchanging its row and column indices. This operation corresponds to subjecting the array to reflection about its principal diagonal. If a matrix is not square , its transpose will not even have the same shape as the original matrix. The transpose of$A$, denoted$\widetilde{A}$or sometimes$A^T$, thus has elements

Note that transposition will convert a column vector into a row vector. A matrix that is unchanged by transposition is called symmetric.

Adjoint

The adjoint of a matrix$A$, denoted$A^\dagger$, is obtained by both complex conjugating and transposing it. Thus ,

 

Trace

The trace , a quantity defined for square matrices , is the sum of the elements on the principal diagonal. Thus , for an$n\cross n$matrix$A$,

Some properties of the trace :

The second property holds even if$[A,B]\neq0$, which means$A$and$B$are not commute.

Considering the trace of the matrix product$ABC$, if we group the factors as$A(BC)$, we easily see that

Repeating this process , we also find$trace(ABC)=trace(BCA)$. Note , however , that we cannot equate any of these quantities to$trace(CBA)$or to the trace of any other noncyclic permutation of these matrces.

$\square$Operations on Matrix Products

There are some properties of the determinant and trace :

whether or not$A$and$B$commute. The properties above establish that the trace is a linear operator. Since similar relations do not exist for the determinant , it is not a linear operator.

For other operations on matrix products , there are

 

$\square$Matrix Representation of Vectors

 

I have nothing to say , because it is easy to understand. (I am going to use$\bold{a}$to represent vectors , and may sometimes use$\vec{a}$as well.)

$\square$Orthogonal Matrices

A real matrix is termed orthogonal if its transpose is equal to its inverse. Thus , if$S$is orthogonal , we may write

Since , for$S$orthogonal ,$det(SS^T)=det(S)\,det(S^T)=[det(S)]^2=1$, we see that$set(S)=\pm1$. It is easy to prove that if$S$and$S'$are each orthogonal , then so are$SS'$and$S'S$.

$\square$Unitary Matrices

The definition is matrix which the adjoint is also the inverse is identified as unitary. One way of expressing this relationship is

If all the elements of a unitary matrix are real , the matrix is also orthogonal.

Since for any matrix$det(A^T)=det(A)$, and therefore$det(A^\dagger)=det(A^*)$, application of the determinant product theorem to a unitary matrix$U$leads to

We observe that if$U$and$V$are both unitary , then$UV$and$VU$will be unitary as well. This is a generalization of earlier result that the matrix product of two orthogonal matrices is also orthogonal.

$\square$Hermitian Matrices

A matrix is identified as Hermitian , or , synonymously , self-adjoint , if it is equal to its adjoint. To be self-adjoint , a matrix$H$must be square , and in addition , its elements must satisfy

We see that the principal diagonal elements must be real.

Note that if two matrices$A$and$B$are Hermitian , it is not necessarily true that$AB$or$BA$is Hermitian ; however ,$AB+BA$, if nonzero , will be Hermitian , and$AB-BA$, if nonzero , will be anti-Hermitian , meaning that$(AB-BA)^\dagger=-(AB-BA)$.

$\square$Extraction of a Row or Column

It is useful to define column vectors$\hat{\bold{e}}_i$which are zero except for the