Invariant subspaces
A subspace of is said to be invariant under a linear operator if
In this case, the action of restricted to the subspace gives rise to a linear operator on .
Let be a three-dimensional vector space with basis , and the operator defined by
Let be the subspace of all vectors of the form , where and are arbitrary scalars. This subspace is spanned by and and is invariant under , since
Show that if both and are invariant subspaces of under an operator then so is their intersection and their sum
Suppose and let be a basis of . By Theorem 3.7 this basis can be extended to a basis spanning all of . The invariance of under implies that the first basis vectors are transformed among themselves,
In such a basis, the components of the operator vanish for , and the matrix has the upper block diagonal form
The submatrix is the matrix of components of expressed in the basis , while and are submatrices of orders and , respectively, where , and is the zero matrix.
If is a decomposition with both and invariant under , then choose a basis of such that the first vectors span while the last vectors span . Then whenever and , and the matrix of the operator has block diagonal form
In Example 4.1 set . The vectors form a basis adapted to the invariant subspace spanned by and ,
and the matrix of has the upper block diagonal form
On the other hand, the one-dimensional subspace spanned by is invariant since , and in the basis adapted to the invariant decomposition the matrix takes on block diagonal form
Eigenvectors and eigenvalues
Given an operator , a scalar is said to be an eigenvalue of if there exists a non-zero vector such that
and is called an eigenvector of corresponding to the eigenvalue . Eigenvectors are those non-zero vectors that are ‘stretched’ by an amount on application of the operator . It is important to stipulate since the equation Eq. (4.6) always holds for the zero vector,
For any scalar , let
The set is a vector subspace, for
For every the subspace is invariant under ,
consists of the set of all eigenvectors having eigenvalue , supplemented with the zero vector 0 . If is not an eigenvalue of , then
If is any basis of the vector space , and any vector of , let be the column vector of components
By Eq. (3.8) the matrix equivalent of Eq. (4.6) is
where is the matrix of components of . Under a change of basis Eq. (4.2) we have from Eqs. (3.18) and Eq. (4.5),
Hence, if satisfies Eq. (4.8) then is an eigenvector of with the same eigenvalue ,
This result is not unexpected, since Eq. (4.8) and its primed version are simply representations with respect to different bases of the same basis-independent equation Eq. (4.6).
Define the th power of an operator inductively, by setting and
Thus and , etc. If is any polynomial with coefficients , the operator polynomial is defined in the obvious way,
If is an eigenvalue of and a corresponding eigenvector, then is an eigenvector of any power corresponding to eigenvalue . For
and the proof follows by induction: assume then by linearity
For a polynomial , it follows immediately that is an eigenvector of the operator with eigenvalue ,
Characteristic equation
The matrix equation Eq. (4.8) can be written in the form
A necessary and sufficient condition for this equation to have a non-trivial solution is
called the characteristic equation of . The function is a polynomial of degree in
known as the characteristic polynomial of .
If the field of scalars is the complex numbers, , then the fundamental theorem of algebra implies that there exist complex numbers such that
As some of these roots of the characteristic equation may appear repeatedly, we can write the characteristic polynomial in the form
Since for each there exists a non-zero complex vector solution to the linear set of equations given by (4.10), the eigenvalues of must all come from the set of roots . The positive integer is known as the multiplicity of the eigenvalue .
When the field of scalars is the real numbers there will not in general be real eigenvectors corresponding to complex roots of the characteristic equation. For example, let be the operator on defined by the following action on the standard basis vectors and ,
The characteristic polynomial is
whose roots are . The operator thus has no real eigenvalues and eigenvectors. However, if we regard the field of scalars as being and treat as operating on , then it has complex eigenvectors
It is worth noting that, since and , the operator satisfies its own characteristic equation
This is a simple example of the important Cayley–Hamilton theorem – see Theorem 4.3 below.
Let be a three-dimensional complex vector space with basis , and the operator whose matrix with respect to this basis is
The characteristic polynomial is
Hence the eigenvalues are 1 and 2, and it is trivial to check that the eigenvector corresponding to 2 is . Let be an eigenvector with eigenvalue 1,
then
Hence and . Thus, even though the eigenvalue has multiplicity 2, all corresponding eigenvectors are multiples of .
Note that while is not an eigenvector, it is annihilated by , for
Operators of the form and their powers , where are eigenvalues of , will make regular appearances in what follows. These operators evidently commute with each other and there is no ambiguity in writing them as
Theorem 4.1 · Linear independence of eigenvectors for distinct eigenvalues
Any set of eigenvectors corresponding to distinct eigenvalues of an operator is linearly independent.
Proof
Let be a set of eigenvectors of corresponding to eigenvalues , no pair of which are equal,
and let be scalars such that
If we apply the polynomial ) to this equation, then al terms except the first are annihilated, leaving
Hence
and since and all the factors for , it follows that Similarly, , proving linear independence of .
If the operator has distinct eigenvalues where , then Theorem 4.1 shows the eigenvectors are l.i. and form a basis of . With respect to this basis the matrix of is diagonal and its eigenvalues lie along the diagonal,
Conversely, any operator whose matrix is diagonalizable has a basis of eigenvectors (the eigenvalues need not be distinct for the converse). The more difficult task lies in the classification of those cases such as Example 4.4, where an eigenvalue has multiplicity but there are less than independent eigenvectors corresponding to it.
Minimal annihilating polynomial
The space of linear operators is a vector space of dimension since it can be put into one-to-one correspondence with the space of matrices. Hence the first powers ofany linear operator on cannot be linearly independent since there are operators in all. Thus must satisfy a polynomial equation,
not all of whose coefficients vanish.
Show that the matrix equivalent of any such polynomial equation is basis-independent by showing that any similarity transform of satisfies the same polynomial equation,
Let
be the polynomial equation with leading coefficient 1 of lowest degree , satisfied by . The polynomial is unique, for if
is another such polynomial equation, then on subtracting these two equations we have
which is a polynomial equation of degree satisfied by . Hence . The unique polynomial is called the minimal annihilating polynomial of .
Theorem 4.2 · Eigenvalues and the minimal annihilating polynomial
A scalar is an eigenvalue of an operator over a vector space if and only if it is a root of the minimal annihilatingpolynomial
Proof
If is an eigenvalue of , let be any corresponding eigenvector, Since it follows that
Conversely, if is a root of then there exists a polynomial such that
and since has lower degree than it cannot annihilate
Therefore, there exists a vector such that , and
Hence , and is an eigenvalue of with eigenvector .
It follows from this theorem that the minimal annihilating polynomial of an operator on a complex vector space can be written in the form
where run through all the distinct eigenvalues of . The various factors are called the elementary divisors of . The following theorem shows that the characteristic polynomial is always divisible by the minimal annihilating polynomial; that is, for each the coefficient where is the multiplicity of the th eigenvalue.
Theorem 4.3 · Cayley–Hamilton theorem
(Cayley–Hamilton) Every linear operator over a finite dimensional vec tor space satisfies its own characteristic equation
Equivalently, every matrix satisfies its own characteristic equation
Proof
Let be any basis of , and le be the matrix ofcomponents of with respect to this basis,
This equation can be written as
or alternatively as
where
Set to be the matrix of cofactors of , such that
The components are polynomials of degree in , and multiplying both sides of Eq. (4.15) by gives
Since the span we have the desired result, . The matrix version is simply the component version of this equation.
Let be the matrix operator on the space of complex column vectors given by
Successive powers of are
and it is straightforward to verify that the matrices , , are linearly independent, while
Hence the minimal annihilating polynomial of is
The elementary divisors of are thus and and the eigenvalues are and . Computation of the characteristic polynomial reveals that
which is divisible by in agreement with Theorem 4.3.
Problem
The trace of an matrix is defined as the sum of its diagonal elements,
Show that
(a)
(b)
(c) If is any operator define its trace to be the trace of its matrix with respect to a basis . Show that this definition is independent of the choice of basis, so that there is no ambiguity in writing .
(d) If is the characteristic polynomial of the operator , show that
(e) If has eigenvalues with multiplicities , show that