¡

(V( Y`¬ ! the auxiliary vectors produced by Algorithm 6.16. Then,

(

y §¥

¦ £¬ £

Each residual vector is such that where is a certain scalar.

¨ ¨ ¦

¢ ¢ ¢ ¢ ¢

"U

W

As a result, the residual vectors are orthogonal to each other. T

©¨

¦ ¬ Xu ¥

§ § ¬E

R (R

The auxiliary vectors form an -conjugate set, i.e., , for .

`

Y

¡ ¡ ¢

¡

£ 6 A

£

The ¬rst part of the proposition is an immediate consequence of the relation

§ ¢ ¥

¥

(6.66). For the second part, it must be proved that is a diagonal matrix, where ¢

¬¥ . This follows from ¤

¢ ¢ W “¢

¬ §§ ¥ §

¥ ¤ ¤

“¢ W “¢

¢ ¢

¢

¢

¬ ¤ ¤

“¢ W “¢

¢

¬ ¤ £ ¢

“¢

Now observe that is a lower triangular which is also symmetric since it is equal

¤ £ ¢

“¢

§§ ¢ ¥

¥

to the symmetric matrix . Therefore, it must be a diagonal matrix.

¢

A consequence of the above proposition is that a version of the algorithm can be

derived by imposing the orthogonality and conjugacy conditions. This gives the Conjugate

© u

Gradient algorithm which we now derive. The vector can be expressed as

"U

W a± „ ®

i

© ¬ "U u © u ¡u w u

W

Therefore, the residual vectors must satisfy the recurrence

a± „ ®

i

§ u ² u ¬

u¨ u

¨ T

¡

"U

W Xu u

§u ² ¬

u¨ u¨

If the ™s are to be orthogonal, then it is necessary that and as a

¨( Y

¡

T

result

Xu u T

R±` „ ®

°i

¨( ¨

§¬

u Xu u

¨( ¡

u¡ u¨

Also, it is known that the next search direction is a linear combination of and

’U

W "U

W

u

, and after rescaling the vectors appropriately, it follows that

¡ ¡

a±0 „ ®

i

¬

u u¨ u ¡u S w

¡

"U

W "U

W

Thus, a ¬rst consequence of the above relation is that T T T

§ ¬ Xu¨( u § § ¬ XW “u Xu u

²

u ¡( u uS ¢(

¡ ¡ ¡ ¡ ¡

W“

¨ ¡¡µ£ „ ¢

§|5¥ yq¢| ¢ £ ¥§

„ 5| j C¦£¥

5§

¡ C

"–© ¡"

§

1B

T T

X Xu u

§ ¬ §

u u u u¨( u¨

because is orthogonal to . Then, (6.71) becomes . In

¡(

¡ ¡ ¡

W“ §

u u

addition, writing that as de¬ned by (6.72) is orthogonal to yields

T

¡ ¡

W"U Xu

W§( "u U u T ¨ ² ¬ ¡

uS Xu

’§(W ¡ ¡

Note that from (6.70) T

r±a „ ®

i

Xu

² ¬u § ²

u¨ y ¨

¡

u

W"U

and therefore, T T T

rX u

X X

²

u ¨ (T u ¨ Tu ¨ u¨(

X

¨

¬ ¬