an approximate minimization may be obtained. In addition, (6.36) is no longer valid. This

equality had been derived from the above equation by exploiting the orthogonality of the

R¦ ™s. It turns out that in practice, remains a reasonably good estimate of the actual¡

0 0

¢

"U

W

R¦

residual norm because the ™s are nearly orthogonal. The following inequality provides an

actual upper bound of the residual norm in terms of computable quantities:

² ! ¡ ©V§ y¨ r±a — „ ®

i

² wW ¡

0 0

¢ ¢

"U

W

y

Here, is to be replaced by when . The proof of this inequality is a consequence

! !

W W

¤¥! ¢ ¢ ¦¤(( £ ¨( ¦

of (6.52). If the unit vector has components , then ¦

¢¡ ¢

"U

W W

W"U

©tA¨ ¢

§² ¬

£ £

¡

0 0

¢ ¢ ¢

¢¢ ¢¢ ¢¢ ¢¢

"UW W"U ¢

W"QfH W"U f

¢¢ ¢¢ R ¦ R ¦ ¢¢ ¢¢ R ¦ R ¦

U

‚0 „

w

¡

0 ¢

¢ ¢ ¢ ¢

£

"U

W £R

R

£ £

W¡ QH

U

¤¢

£

¢

W£

W"QfH "U f

„ £ R ¦ 0 R 0¦

U W

¢R

‚0 w

¡

0 ¢ ¦

"U

W £

R R

W¡ QH

U

¤¢

£ ¤¢

£

¡ ¢

² !

W£ W£

W"QfH W"U f

U ¢R ¢R

‚0 „

w

¡

0 ¢ ¦ W ¦

"U

W £

R R

W QH

U

R¦

Here, the orthogonality of the ¬rst vectors was used and the last term comes wW

y

from using the Cauchy-Schwartz inequality. The desired inequality follows from using the

Cauchy-Schwartz inequality again in the form

² ! ¨’ ² ! £c § £

w c ¨

wW w

W

y y

¢

and from the fact that the vector is of norm unity. Thus, using as a residual ¡

0 0

¢

² ! "U

W

estimate, we would make an error of a factor of at most. In general, this is an wW

y

overestimate and tends to give an adequate estimate for the residual norm.

¡

0 0

¢

’U

W

It is also interesting to observe that with a little bit more arithmetic, it is possible to

actually compute the exact residual vector and norm. This is based on the observation that,

according to (6.52), the residual vector is times the vector which is the last ¡