Xf
a±—0 „— ®
i y H
(
©¡ (V( ( W “R
¬
7E tc¨
©§ ² § ¬Rb
R
X
T
The additive projection procedure can be written as
R R § ¬R§
CQH
, and de¬ne for ©u § ¢£¡
¬E ¬
¥ ¥
¥ ¥ ¨
¥C tt ¢ § @ £ ¢ © v¦ © §
w
 y C
5 5r £ y ¦5 ¦£

µ£ „ ¢
5¥ qzl 5 ¢ ¤ ¤¥
„ 5  §
¥C ¡
§ "!
The residual norm in this situation is given by
r± 0 „— ®
i
%R R # X f
¬ ² ¥
¨ (¨
"
’QH
WU H
£R
W
#
considering the single parameter as a particular case. Exercise 14 gives an example of
R#
the choice of which has the effect of producing a sequence with decreasing residual
norms.
( ¬R#
We now return to the generic case, where . A leastsquares option can be E
R y § ¬ R £ R
¥
de¬ned by taking for each of the subproblems . In this situation, becomes an
R (§
orthogonal projector onto , since
T
T
X
X
R § W “ R § R § R (¬ R ¥
§
It is interesting to note that the residual vector obtained after one outer loop is related to
the previous residual by
Xf %R
¬ ² ¥
¨ (¨
"
’QH
WU H
£R
W
¥ R
where the ™s are now orthogonal projectors. In particular, in the ideal situation when
§ ¤
¥
R R
the ™s are orthogonal to each other, and the total rank of the ™s is , then the exact
solution would be obtained in one outer step, since in this situation
Xf
² ¬¥ Y` R
R
W
§
R
Thus, the maximum reduction in the residual norm is achieved when the ™s are 
orthogonal to one another.
Similar to the Jacobi and GaussSeidel iterations, what distinguishes the additive and
multiplicative iterations is that the latter updates the component to be corrected at step
immediately. Then this updated approximate solution is used to compute the residual
E
vector needed to correct the next component. The Jacobi iteration uses the same previous
©
approximation to update all the components of the solution. Thus, the analogue of the
H
block GaussSeidel iteration can be de¬ned as follows.
¡¤¦ ¢
A ´ ˜Q v ¢ 7 £7HTEQ C¢ £S B ¢
¡3 V8 ¡ ¥ 5©E Y8
5 QS@ B Q PT£§H5 ¤ Y8
D
6 DQ S E D
5
' 7$
%# ) 2
1
1. Until convergence, Do:
T
¬vE
2. For Do: (˜ (
( ¢
¡
X
R
bR y
¬§ ty¨
©§ ²
3. Solve
3
bR w
©¬©
4. Set
5. EndDo
6. EndDo
³¥C
y £ w ¢ ¡y ¡
5  §
vcQ—tc™£™
—™ “ ¢
§ ¡
1 Consider the linear system , where is a Symmetric Positive De¬nite matrix.
¤ ¢
£ £ R §¨¦1
©¥
Consider the sequence of onedimensional projection processes with ,
where the sequence of indices is selected in any fashion. Let be a new iterate after
"
% $ § Tt
v
" ¡¦ §