¨ & ¢ §¥

¦

(

C gB

@ & $% "

(

("

' ©"

¦¦

C

and the two other ones can be obtained from (1.30) and (1.32) by simply relabeling the

eigenvalues increasingly instead of decreasingly. Thus, with our labeling of the eigenvalues

in descending order, (1.32) tells us that the smallest eigenvalue satis¬es

C gg B B ( ¢ ‘

¦

¦

t

¨ ¢ ¢ §¥

¦

(

@ '$

& ©"

¦¦

C

0@

with replaced by if the eigenvalues are relabeled increasingly.

@

‘

In order for all the eigenvalues of a Hermitian matrix to be positive, it is necessary and

suf¬cient that

p S z C x B

‘ 7 8

7

©¦

¦

¦ ¦

( C g B

7

©¦

¦ ¦

Such a matrix is called positive de¬nite. A matrix which satis¬es for any is

(I

said to be positive semide¬nite. In particular, the matrix is semipositive de¬nite for

any rectangular matrix, since

g p ( C x B C g I B

7

¦© ¦ ¦ ¦

¦

(

Similarly, is also a Hermitian semipositive de¬nite matrix. The square roots of the

I

DI

eigenvalues of for a general rectangular matrix are called the singular values of

uY u d¥ £ ¡ ¨ ¢¤¥

n ¦¢¥

¤¤§ ¥ §¢

¡

£ ¡© ¥ ¡ © ¥ ¡ ¥ © ¥ ¥

¥ ¨

and are denoted by . In Section 1.5, we have stated without proof that the 2-norm of

X

0X

any matrix is equal to the largest singular value of . This is now an obvious fact,

because

66 ¦

C g g DBI B §¦¥¢ C x B B ¤¦ ( ¢

x

¦ ¥¢ 66

¤( ¤( 6

¦ ¦ ¦

¦

6 ¦ 0X

'%

&$ & $% '$

&

6 ¦© ¦ ¦

¦

C

C

which results from (1.31).

Another characterization of eigenvalues, known as the Courant characterization, is

stated in the next theorem. In contrast with the min-max theorem, this property is recursive

in nature.

„ §CjG b¤

¥¦ ¥¡ %

U

Q T )

The eigenvalue and the corresponding eigenvector of a Hermi-

@

tian matrix are such that

g B §¦¥¢ C 0 ) 0 B 0

( ¤ © C 0 ) 0 B

) ¦

¦

C gB

@ ) "

¦¦

'$ " ¡

&

C

Q

and for ,

C gx B B ( §¦( ¥¢( ( C ' ' ) ' ' B B

¤

) 0

) ©¦

¦

t

¨ £ ¢ §¥

¦

'@

¢ " &'$ %%$

$$ ' ¥¢£

) & ©

¦¦

¤

C C

In other words, the maximum of the Rayleigh quotient over a subspace that is orthog-

3 )

'@ '

onal to the ¬rst eigenvectors is equal to and is achieved for the eigenvector

'@

associated with . The proof follows easily from the expansion (1.29) of the Rayleigh

quotient.