1, and «sy P= pal s Ue Pol for all n> 1 Sec.2.1 ITERATION FoR SoLVING x 0 ence when I< (P), Figure 25 (b) Divergent oscilla aon when g'(P) <1 Graphical Interpretation of Fixed-point Iteration Since we seek a fixed point P t0 g(x) i is necesary thatthe graph of the curve y= g(a) and ie line y — 4 intersect at he point (P, P). Two supe type of ‘converter iteration, monotone and oscillating ar illustrated in Figure 2a) and (), respectively "To visualize the process, sat at py on the Aan nd have verily Wo the point (po, pd) = (po, 8(po)) on the curve y = g(x). Then move horizontally from (po, P1) to the point (py, pi) onthe line y = x. Finally, move vertically downward to p) on the x-ans. Tho rooutsion ces = 6(pr) is wed to constact the pot (Ps Prt) 00 the graph, then a horizontal motion locates (py 41. Py) on the line y = x, and then a vertical moverentends up at py. onthe x-ats. The situation is shown in Figure 2448 Cuiae 2 THe SoLUTION oF NontiNeaK EQUATIONS fix) ~0 If |g'(P)] > 2, then the iteration py 1 = g(pn) produces a sequence that diverges wey from P, The two simple types of divergent iteration, monotone aud oscillating, ae ilustrated in Figure 2.5(a) and (b), respectively. Example24. Consider the iteration p,1 = g(ps) when the function g(x) = +x~27/4 'sused. The ied points can be ound by solving the easton x = (x). The tw slutons (oa piven ebarev — -Pand r~9 The drain ofthe fimston sq) ~ 12/2, fn there are only two cases consider. 2 casei: 20s Sa ih 2 owes tenet 2 Tanaris = daira Covet P: then get slg, me=t ‘Since ig < 4 on [1,3], by Theo- oi Sin ig (21> $on{-3, —1.by The- cram 28 tere il comers 3 |r ‘Theorem 2.3 does not state what will happen when @/(P) = 1. The next example ins been specially consrucied su that the sequence {pe} converges whenever po > and it diverges if we choose po < P. 8(Px) when the function x) = 2¢2— 1/2 Example 25. Consider the iteration py exists. The derivativeis e(2) = 1/(e— 1) forx 2 1 is used. Only one fixed point and g'(2) = 1, a0 Theorem ?3 dace not apply. There ser wo cxses to concider when the starting value lies othe left or right of P = 2. i Case i: Str with py = 15, Case i: Stax with py tener ey = ALOIS den ge 87 pas te7teest P= 240789513. ps Lares 37309914 ps =0.53890832 5aasen84 ps= 2 -awurtony"? 2B, Pe =z, Since pg lies ooece tne domain of ‘Tm sequence 1s converging 10 slowly £13), the tem pe canna be computed (o the vulue P = 2; inded, Phono = ‘oosssria. Ste. 2.1. IrBRaTio FoR Sotvina 4 = 5(2) ” Absolute and Relative Error Considerations In Example 2.5, case (i), the sequence converges Slowly, and after 1000 iterations the three consecutive terms are roo = 2.00398714, pico = 2.00398317, and proms = 2.00397921 This should not be disturbing; after all, we could compute few thousand more terms and find « better approximation! But what about a criterion for stopping the iteration? Notice that if we use the difference between consecutive terms, lero Proce! ~ 12.00398317 2.00397921| ~ 0.000396. Yet the absolute error in the approximation poop ie known ta he |? ~ presa| = |2.00000000 ~ 2.003987141 = 0.00398714. This is about 1000 times larger than |pico1 — pion! and it shows that closeness of consecutive terms does not guarantee that accuracy has been achieved. But itis usually {We only criterion available and is often used to terminate an iterative procedure. Program 2.1 (Fixed-Point Iteration), To approximate a solution to the equation | + = ie) sarin wit te iat 055 py a ‘function Oe,pyorr,P]-fixpt (@,p0, tol mast) f laput ~ 6 ie she sceration function input ae a string “4° ~ pO is the initial guess for the fixed point i tol is the tolerance % ~ maxi is the maximum nunber of iterations Yourput ~ k io tho nunber of iterations that vere carried out = p 4a the approximation to the fixed point x = err is the error in the approximation % ~ P contains the sequence {pn} PAD p05 for ke2:maxt POO=feval (g,PQ1)); erreabe(P(i)-P(z-2)) relerr-err/(abe(P(R))+eps); PPP: Af (arrctot) | (relerretol) ,breaksand rimim number of ‘iterations excceded’) ‘Remark. When using the user-defined function fixpt, it is necessary to input the Mofile g.m as a string: "(see MATLAR Appendix)50 CHAP. 2 THE SOLUTION OF NONLINEAR EQUATIONS. f(x) = 0 Exercises for Iteration for Solving x = g(x) 1. Deteemine rigoroly if ech fnetion has & nique ied point on the given interval (llow Exampie 23) (a) 60) =1~7/40010.1) ) 9s) =2-F on 0,1] (@) ex) = Ix on f05.5.21 2. laveigate the nature ofthe ined pon iteration when aya, (@) Solve g(x) = + and show that P = 2and P = 4 are ined point. {b) Use me staring vale py = 1 and.eompute pi, po, and ps (©) Use the sting vale po ~ 3.8 and compute ps, ps, ad ps (@) Find the errs Ei and relative ers Re forthe vals pin pats) ad (6) (©) What conclusions can be deawn from Theorem 2.3? 4 raph (2) Be ne y = x, andthe gWven xed point # onthe same coordinate system. Using the given starting value po. compute and pa, Consist figures similar to Figures 24 and2.5. Based on your graph, determine geometrically if fixed point tration converges. (@) £02) =6-+2)"?, P=3,and po =7 2s.P 23P (8) ghey 42842, Pe Dando = 4, Let g(x) = x* +1 ~4, Can Bxed-point iteration be used to find the solution(s fo the 22 equation x= a(x)? Why? 5, Let gx) = xeostx), Solve x = g(2) and find all he fixed points of ¢ (there ae in finitely many) Can fxe-pin iertion be weed to fnd te olin) othe equation x= g(a)? Why? 6 Suppose that (x) and («are defined and comtinuouson (a6), Pos pa € (a0) and p; ~ a(po) and Ps ~ gps). Also czar tht there exit 3sorataat sch tage) = K. Show that [p> ~ pti < K'py ~ po. Hint. Use the Mean Value ‘neorem 7. Suppose that g() ang) ase continous on (4,6) and Maa jy’G)| = Lou dis interval. Ifthe fixed point P and ihe inital approximations py and ps lien the itera (2.2), the show that p, = gpa) implies that Ei] =P pil = IP ~ po! = IEol once sateen (1) of Theorem 23s etablished (local divergence) 8 Let 2) = ~0 00014? + x and ao = and consider fixed-point iteration, (a) Show that po > pa >> Pa > Past >> (©) Show mat py > O fora SEC.2.2 BRACKETING METHODS FOR LOCATING 4 ROor st (©Since the sequence {9} is decreasing and bounded below, it has limit. What is the init? 9. Let g(x) = 0.5x + 1.5 und pp = 4, and consider fixed-point teravon, (@) Show that the fixed point is P = 3 (©) Show that |P ~ py = |P — py/2 for (©) Show that 1# — pal = 1" — poi/2" torn 10, Let g() = 1/2, and consider fixed-point iteration (a) Find the quantity p11 pal/|pvatl (©) Discuss what will happen if only the relative error stopping criterion were use in Program 2, 11, For fixed point iteration, aiseuse why iti an advantage to have (P) Pe y i Algorithms and Programs 1, Use Program 2.1 to approximate the fixed points if any) ofeach function. Answers cane a yr of eae Zaeion ai tne that clearly shows any fixed points. (a) gle) =x ~3e8 28 2 ©) g(a) = corcsinte)) (©) etx) sintx + 0.19) er @ a Bracketing Methods for Locating a Root Consider afaniliae topic of interest. Suppose that you save money by making eeu monthly deposits P andthe annual interest rate is 1; then te toal amount A after depositsis (Digs See = +P (i+ 5) +P(i+5) +. sees ‘The fistterm on the right side of equation (1 isthe last payment, Then the nextlast payment, which has eared one period of interest, contibutes P (1+ 45). Te second- ‘from-last payment has earned two periods of interest snd contributes P (1+ 4)*, and soon. Finally the last payment, which hac earned interest for N 1 periods, connvtes (14 4)""" coward the total. Recall chat the formula for the sum of the N terms of a geometric series is 22 Cuar2 THESOLUTION oF NontaNean EQUATIONS fis) =O ‘We can write (1) in the form, e(ia(ep)a(- and se the sobstttion x= (1+ /12)n (2) t0 obtain a+" ‘This canbe simplited to obtain the annuity equation, P ry" ® se ((4d)"=) ‘te following example uses the annuity-due equation and requires @ sequence ot repeated calculations t find an answer A Example 2.6, You save $250 per smo fr 20 years and desie dha he total value of all payments and interest is $250, 000 atthe end ofthe 20 years. What interest rate 7 is Deeded to achieve your goal? If we hold N = 240 fixed, then A is a function of I alone: that A = ACD), We wil tart with two guesses, fo = 01? and fy = 013, and perform sequence ofcaleuations to narrow down te final answer. Starting with fp = 0.12 yields 1) 250 oy Since this value is litle short ofthe goal, we next ry fy = 0.13: a0 gia (+82) viare(('=2) -!) =a ‘This is lite high, so we ty the value inthe mide fy = 0.125: 250 0.125)" 40.125) aisha ((1+ ) -1) “This i wpain high ad we conclude that de desived rate Hes iu the interval (0.12, 0.12 164, 623. The next uss isthe midpoint Is = 0.1225: 250 0.1225) a = 1) =255, 803. nee) = ras (1+ 8) —) asa This is high and the interval is now narrowed wo [0.12, 0.1225]. Our las calculation uses the midpoint approximation fg = 0.12125: 5 1) =2sisie Sec.2.2 BRackETING Merilops Fok Locarina a Root s (ofa (ofa) Pa éfle) ee Nite a > fo) ~——_+ fbn ero (2 11 7.a) and fey nave “opposite signs then squeeze from the right. (80 fee) and fb) have onposite sins then squeeze from the left ‘igure 2.6 The decision process forthe bisection process unter iterations can be done to obtain as many significant digits as required The purpose ofthis example was to find the valuc of { dat produced a specified level L of the Funetion value, tha is to find a solution to AC) = L. Ie is stand practice 10 place the constant Zon the lft and solve the equation A(Z) — . Definition 23 (Root of an Equation, Zero of a Function). Assume that f(x) 18 a continuous function. Any number r for which f(r) = Qis called araot ofthe equation I(x) = 0. Also, we say ris azero ofthe function f(x). 4 For example, the equation 2x? + $x — 3 = 0 has two real roots ry = 0.5 and r2 = 3, whereas the corresponding function f (x) = 2x7-+5x—3 = (2x—1)(r+3) hhas two real eros. 7) = 0.5 and re = —3 ‘The Bisection Method of Bolzano In this section we develop our fist bracketing medhod for using a ze ura continuous function. We must start with an initial inerval fa, b], where f(a) and f(b) have ‘opposite signs. Since the graph y = f(x) of a continuous function is unbroken, it will crosethe x-axis at zero.x = r that lies eomewherein the interval (sce Figure 2.6). The bisection method systematically moves the end points ofthe interval closer and closer together until we obiain an interval of arbitrarily small width dhat brackets the zero. ‘The decision sep for this process of intorval halving is fist to choose the midpointS4CHAR.2 THE SOLUTION oF NONLINEAR EQUATIONS f(x) = 0 (¢= (@-+0)/2 and then to analyze the three possbiltes that might aise: a If Flay and fc} nave opposte signs. & zero Hes in (a. c 6) If Fle) and f(b) have opposite sign, 42270 lis in [e, 5) 6) If Fle) =O. then the zer0 is either ease(4)or(5) occurs, we have found an interval haf as wide asthe original intervel that contains the root, and we are "squeezing down on it” (see Figure 2.6). To wontinue the process, selabel the uew salle inteval fu b] and tepeat the process until the interval is as small as desired, Since the bisection process involves sequences of nestcd intervals and their midpoints, we will use the foliowing notation to keep wack sof the details i the proves jag, Bo} the starung interval and cy = 85% is the midposat. 1-6 ]is the second interval, which brackets the zero, ané i its mudpoint; (7) helinterval (a1, by] is Hallas wide a Lao, Boh tier ariv:ng atthe nth anierval ft, By Which Drackers r and "as Mudpoint 0 the interval [ac boa] i constructed, which also brackets r and is half {8 wide (09, Dal Tt is Heft as an exercise forthe reader to show thatthe sequence of left end points is imereasing and the sequence oP right end puis 5 dcvieasing, hat i, 8 Says eager Se Sb se Sh ho were og = BH, an Flor P Fag < 0.160 lana OF Wdgets Beet = Lens Bad f08 all (9) Laat. bye tT ‘Theorem 2.4 (Biseetion Theorem). Assume the: f € C[a, b] and that there exists a vumberr © fu.d] such that f(r) = 0. IF fa) and £6) have opposite signs’ and ‘er} Fepresents the sequence of midpoints generated by the bisection process of (8) and (9), then for n=0, 1, U0) rae < and therefore the sequetce (¢4]%iq converses 0 the zero x =r; that 1, wy Jim cy =r. Proof Since both the 7era sed the midpoint cx He in the interval [ap Be) the die- tance between cy and r cannot be greater than half the width ofthis interval (see Fig- ure 2.7). Thus «2 W-ols for all n Sec. 2.2 BRACKETING METWODS FOR LOCATING A ROOT s Figure’27 The root rand midooint cy of fs. by) forthe bisection meted (Observe that the successive interval widths form the pattern Itis eft as an exercise forthe reader o use mathernatical inductioe and show thet oy ees Combining (12) and (13) results in aay for all n [Now an argument similar 0 the one given in Theorem 2.3 can be used to show that 114) implies thet the sequence (cy converges tar aod the proof ofthe theorem is complete : Example 2.7. Te funciion n(x) = x sinix) occurs mn We study of undamped forced ‘oscillations. Find the valve of x that ies in the interval (0.21. where tae function takes on the value x) = 1 (the function sin(x) is evalvaed in radians) ‘We use the bisection method o finda zeroof the function f(a) — sina) 1, Stating with ap = Oand by = 2, we compute #0) = -1.000000 and f12 sias9s 50 a root of f(1) — 6 Nes in the interval [0.2]. At the midpoint

0, break,end naxiei+round( (Log (b-a)-log(delta)) /1og(2)) for ket:maxt exCaeh) /25 yerfeval (fc); Af youn yarye: if bra < delta, break, end end c=(aeb) /25 errsabs(b-a) yerfeval (fc),60 Ciiap,2 THE SOLUTION OF NONLINEAR EQUATIONS f(x) = 0 | Program 2.3 (False Position or Regula Falsi Method), To approximate & root of | the equation <2) = 0 in the interval (a. 61. Proceed with the method only if f(x) | iscontinuous and f(a) and f(b) have opposite sign. j function (c,err,yc]=regula(f,a,b,delta,epstlon,maxt) ‘input - f is the function input as a string *f° % 0"), break, for kel:maxt xrybe(o-a) /(yb-ya) cxbrdx; acec-a; ye=teval(t,c); Af yors0 break: elseif ybeyooo axruin(abs(ex) Af abe (dx) *-__ In Exercises 1 and 2, find an approximation forthe interest rate that wll yield the total annaity value A st 240 monthly paymens P are made. Use the wo staring values for 7 and compute the next three approximations using the bisection medhod 1. P =$275, A =$250,000, fy = 0.11, = 0.12 2, P $325, A =$400, 000, lo = 0.13, = 0.14 3. Foreach function, find an interval (a,b so that f(a) and f(b) have opposite signs (@) suy—e 2-4 () for) = costed + 1x (©) fa) =Iny—S+2 (Joys ~ oe $28 tn Exereincs 4 throiagh 7 start wi co.crrea. andes fs, dy] and use the false posit metal to cumple lay, bo] = [-2.4, -1.6] {4 69] = 108, 6} « 8, ap, 9] = (9.24.0) 1. x2 — 108 +23 = 0, ap, bo] = (6.0, 638] ‘8 Denote the intervals that arise in the bisection method by (a9, hs), [01 i}, (ey. (2) Show that ay = a) +++ S dy 3. then fap) f(09) <0. Thus, on the imirval (a9, 25) the bisection method will converge f0 one ofthe hee zeros. If ag < 1 and bo > 3 are selected such that cy = S542 isnot equal to 1,2, oF 3 for any n > 1, then the Disestion method will over converge to which <70(5)° Why? 14, 15, Tf polynomial, f(0), has an old nomber of real rer in the interal [g, Po), and cach of the zeros is of odd multiplicity, then f(a) (ba) < 0, and the bisection into emerge one oe Hm aid ch (snot equal to any ofthe eros of F(x) for any n > f, then the bisection ‘method will never converge to which zero(s)? Why? Algorithms and Programs ia apyronimation aceurew 10 vine pace} fin te inate cate Fit wi Yield oa annuity valu of $50, 000i 240 monthly payments of $300 are made 2. Consider spherical ball of radius ¢ ~ 15 om that i constructed fom a variety of ute oak that has a density of p = 0.710. How much ofthe dal (accurate to 8 decimal places) wil be submerged when itis placed in water? 3. Modify Programs 22 and 23 to ouput « matrix analogous to Tables 2.1 and 22 respectively (ie, the fist row ofthe matix would be [029 cp & feo) 4. Use your prgrunm from Problem 3 « approximate the Hee smallest poste Toots ofx = tan) (accurate to 8 decimal places). 5 A unit phere is cut into two Segments by a plane. One segment has thes times the Yolume ofthe other. Determine the distance x of te plane rom the centr ofthe sphere acct 1 ema places 3 Initial Approximation and Convergence Criteria ‘The bracketing methods depend on finding an interval [a,b] so that f(a) and (6) have ‘opposite signs, Onve de interval ius been found, oo quatter bow lage, re iesations ‘will proceed until a root is found. Hence these methods are called globally convergent. However if f(r) = O has several roots in fa, b), then a different starting inferval mist be used to find cach root, I is not easy to locate these smaller intervals on which f(s) changes sign. Tn Section 2.4 we develop the Newton-Raphson method and the secant method for solving f() — 0. Both of these methods require that a close approximation to the root See.2.3 INITIAL APPROXIAEATION AND CONURROPNCR CRITERIA a be givento guarantee convergence. Hence these methods ae called Locally converges ‘They usually converge more rapidly than do global anes. Some hybrid algorithms start with a globally convergent method and switgh fo a locally convergent method when ‘the iteration gets close f0 a root Ifthe computation of roots is one part of a larger project, then a leiarely pace is suggested and the first thing to do is graph the function. We can view the graph F(X) and make decisions based on what it 10OKS like (concavity, slope, oscillatory ‘behavior. local extrema. inflection points, etc). But more important, i the enordinates ‘of points on the graph are available, they can be analyzed and the approximate location ofsvots determined. These approximations can then be used as starung values in our root-finding algorithms. ‘We mast proceed carefully. Computer software packages use graphics software of vatying sophistication. Sappose thet a compulcr is used w yrapit y = fx) om (a, 2) ‘Typically, the interval is partitioned into N + 1 equally spaced points: @ = xo < x) Oand X41 C48 not Necessary r0 USE the difference quotients. and it will suffice to check to see ifthe differences ve ~ v1.1 and yy) — yy change sign, In this ease, xe is the approximate root. Unfortunately, ‘we cant guarantee tha this starting value will produce « vowvergent sequence. IT de graph of y = f(x) basa local minimum (or maximum) that i extremely close to zero, then itis possible that x, will be reported as an approximate root when f(x4) * 0, although xx may not be closc to 8 root.64 CHAP 2. THESOLUTION OF NONLINEAR EQUATIONS f(s) = 0 “Table?.3._Finting Appeonimate Lovatone for Roots Funcom values | __ Differences im y : Siu tags 4 % murm | imymose “O98 13 e201 1463 | changes sign 4a.) “06 1028 a9 m3 Hiss =0183 _| fangs sign ear, oo 1.000 “0368 03 037 0381 a | 0230 mos) 09 oI 06 | cng sign ear, 12 | 0.088 oss gure 210 The graph ofthe cx bie polynomial y Port! eample 29, Find the approximate location ofthe rnt of 1 — «2 = eg T= onthe itera [-1.2, 1.2] Forillasration, choose N = 8 and lok at Table 23 “Te ee shcise orctakieratnmse ~1,05,~0.3, at 09, Becasne f(x) changes sign on the inerval [=1.2.~03]. the value ~108 is an approximate roe; indesd FC.) = ~0.210. ‘Although te slope changes sign near ~0.3, we Gd that /(-0.3) ~ 1.183; hence =0.3is not neara oo, Finally, the slope changes sign nar 0.9 and/(09} = 0,019,500 {5 an approximate root se Figure 210) . INITIAL APPROXIMATION AND CONVERGENCE CRITERIA 6 Figure 2.11 (2) The horizontal comergence band for locating a solution to Ie) * Figure 2.11 (b)The vertical convergence band for locating a solution to Fx) Checking for Convergence _Agraph can be wed to ae the approximate locaton ofa rot, but an algorithm must be ‘sed fo compute a value py that Is an accepable computer solution. Heaton i often ‘zed to produce a sequence (ps) that converges oa root p, anda termination eitrion cr rtegy mt he Geigned ahead of time an tha the compmter wil top hen an Accurate approximation is reached. Since the gosl ito sve f(x) = Othe final value Ps Should have te propery that |/ pe} <€ ‘The usercan saply a tolerance value « forthe sizeof fp and shen an iterative process proces points Py = (pu, J(pe) unl the ast point Pies inthe horizontal ‘and bounded by the lines y te and y= ~e, as alown in Foguce 2.110). This esiterion is Useful ifthe user is tying to Solve A(z) = L by applying a oot-fnding6 CAP.2. THE SOLUTION OF NONLINEAR EQUATIONS f(x algorithm tthe function f(x) = hla) ~L. “Another termination cierion involves te abscissis, and we can try tod the sequence (74) in cnverging I we dew the vera lines x = p+ 8am! + fon each side of x = p, we could decide o stop the eration when the pst between these two verti lines, as show in Figure 211 (0). The later witcriom is often deste ut itt implement Because i volves che unknown solution p. We adap this idea and terminate further calculatios when the consecutive trates py and py are suflcentl close rif they agree with significant ist Sometimes the user ofan algorithm will be satisfied if ry ~ 75-1 and other ies when (Pq) * 0. Correct logical reasoning is required to understand the cons gpences, If we require that jem p) = 5 and IF Pn ~ «. the put Px wil Tocated inthe retangulr region about the solution (p, 0), sb shown in Figure 2.12( If we stipulate that py ~ pl = 8 oF if(py)| < e:the point Py could be loca anywhere in the region formed bythe io of the horizontal and ver} spe shovin in Figure 2.12(b) ‘The size of the tolerances 8 and are crcil. Tf the to races are chosen too small ilraton may continue forever. They shouldbe chose ston 100 mes larger than 10M where Af isthe umber of decimal digit in he computer's Noting point numbers, The closeness ofthe abscissas is checke® th on: of te entena IPe~pe-il <8 (estimate forthe absolute eror or 21pm — Prot! <4 (estimate forthe relative error). ial ¥ Ipe-1l “The closeuess uf tu urdinte is usually checked by Lf (Pa}i <€ ‘Tronhlesome Functions [A computer solution to f(a) = 0 will almost always he in esrnr de sso and/or instability in the calculations. If the graph y = f(x) is steep nev (p.0), wen the root-finding problem is well conditioned (Le. a solution with sex significant digits is easy to obtain’. Ifthe graph v = f(x) is shallow neae (0). then the root-finding problem ill conditioned (.., the computed root may have ya few siguificen digits). This oeurs when /¢«) ls a muldple ro ap. This scussee further in the next section ‘SEC.2.3 INITIAL APPROXIMATION AND CONVERGENCE CRITERIA, o Figore 212. (e) The rectangular region defined by [x ~ p) < 6 ANDIy| <. yan) Figare212_ (&) The unbounded region defined by [x ~ p| < 8 OR [yl <«68 Cuar.2 THE SoLUTION oF NoNLiNeaK Equarions f(x) —0 [Program 2.4 (Approximate Location of Roots). To roughly estimate the loca. tions ofthe roots of the equatton f(x) = 0 over the interval {a,b}, by using the equally spaced sample points (xs, /()) and the following criteria: © Gr0ON <0.0r i) Lye) <€ and Oe — DOH ~ 9) <0. ‘That is, either f(x4-1) and f (4x) have opposite signs or | f(x] is small and the slope of the curve y = f(s) changes sign near (xa, f(a). = approot C,epettou) = £ ip the object fumction caved ac an M-file aanod fm ~ K is the vector of abscissas = epsilon is the tolerance 4 Output - Rie the vector of approxiaate roots Yet: yrange = nax(¥)-ain(); spsaion2 = yrangerepsilon; nelength(X): Xturs)=x60) 5 Yonts)=¥@); for ke2:n, Af YOCDHYG)<0, weet; Row end fe CVO) Veet) de C#) YO) if (abs(¥(K)) < epsilon2) & (s<=0), ment Rem =X) ; end ond (e-1)#X009/25 Example 2.10, Use approot to find approximate locations for the roots of f(e) = sincos(x?) in the imterval [-2, 2). Furst save f as an M-file named fm. Since the results ‘willbe used a iil approximations for aroot finding algorithm, we will construct X20 that the approximations will be accurate to 4 decimal places. 001 >>approot (X,0.00001) 1.9076 -1.0765 -1.1626 4.1625 1.6765 1.9075 See 22. TWITTAL AppRoxtwaTio aNn CONVERGENCE CRITERIA o Comparing the results with the graph of J, we now have good initial approximations for ‘ane of cert Snting algorithms . Exercises for Initial Approximation In Exercises | through 6 use a computer or graphics calculator to graphically determine ‘he appronicnate location of the ot: of Ge) — 0 in the given interval. In each case, 0 such tbat the sequence {4}? defined by te iteration _ fn Fon yal appronizuaton py © Lp — 8. p+ 81 1) defined hy foros for o p B(P1) = Pea will comenge to p for Remark The function ¢ fey 6 aly Fm is called the Newion-Raphson eration function. Since 1(p) 0, its easy to see ‘that g(p) = p. Thus the Newion-Raphson iteration for finding the root of the equation (2) — Os accomplished by finding a fixed point ofthe function g(a), derstanding why 7g needs to be close to p of why the continuity of "¢x) is essential (Our analysis starts with the Taylor polynomial of degree n = 1 and its remainder term: The Sain Pigare 2.19 aes wo ee a we Fox ~ po}? o Feo) = Spo + F (pode poy + LOS ‘wher lies somewhere between pp and.x, Substiuting x = p into equation (6) and using the fat that fp) = O produces ie ad a 0 Spo) + F (perp ~ py) + HAE POP TT ny is close enough to the last term on the right side of (7) will be small com: ‘ered to the sum of the lirst two terms. Hence i: can be neglected and we can use the approximation 0) US J (po) + Swot? ~ Bod. So’sing for p in equation (8). we get p * po ~ f(Pod/f"(Po)- This i used to define the rex approximation p) tothe root Sow 0 1 = po Le, 2 Pe PF) When ps is used place of pc in equation (9) the general rule (4) established. For ‘most applications this is ull that needs to be understood. However, to fully comprehend72 CHab.2. Tae SOLUTION OF NONLINEAR EQUATIONS f(x) =: what is happening, we need to consider the fixed-point iteration function and apply Theorem 2.2 i our situation. The key isin the analysis of g(x): POF) =fOs"w _ fos" Cor Fon? By hypothesis, (p) = 0: thus g’(p) — 0. Since g'(p) = Oand g(x) is continuous it ‘possible to find a8 > 0 so tha the hypothesis [g"(x)| < 1 of Theorem 2.2is stisied on (p — 5, p +8). Therefore, a sufficient condition for po to initialize a convergent sequence {pa)j2y. which convergesto a root of f(s) ~0,is that py € (p 8, +8) and that 8 be chosen o that Wie fo) FOF dale G9) AL forall ve Gp pas) + Corollary 22 (Newton's eration for Finding Square Koots). Assume that A > U isa real miner and et pp > Ohe an inital approximation ta VA Define the sequence (paleo using the recursive rule 4 mat be for k= 1, 2, an a Thon the eequonce (pi }fiy converges to V/H tha i iby nce Pk — VA Outi of Prof. Stax with he Sncion f(x) = ~ Avan otc tht he oof thecquton © — A= Oare/A. Now ute f(s) andthe derivative f'n formula (3) and wate down the Newton Raphson eration formula a fo This fucmuta can be simplified to obtain 03) ao ‘When g(x) in (13) is used to define the recursive iteration in (4), the results formula (11D, Ie can be proved that the sequence that is generated in (11) will converge for any starting value py > 0. The details arc left forthe exercises . ‘An important point of Corollary 2.2 is the fact shat the iteration function 4s involved only the arithmetic operations +, ~, x, and /. If g(x) had involved the cal: culation ofa square roo, we would be caught in the circular reasoning that being able vw calculate the square root would permit you to recursively define a sequance that will converge to VA. For this reason, f(x) =x? ~ A was chosen, because it involved onls the arithmetic operations. 'S8C:2.4 NBWION-RAPHSON AND SECANT METHODS Example 211, Use Newton's squareroot algorithm ind 5 ‘Suing with pp-~ 2 and using formas (11, compu = 2482 was p= PEHSEB pasitttns Mm zzz a6 1 _ a associat pa = 23898 Sra2seosrs ae Further iterations produce pe ~ 2.236007978 lor k > 9, so we see that convergence accurate 10 nine decimal places has been achieved. . Now fetus tum w a fasniliat problem oun elementary physics aid see why de termining the location ofa root isan important task. Suppose that a projectile i fired from the origin with an angle of elevation by and inital velocity wp. In clementary the height y — 9G} ad de ise eFC ls vest fa neghevied ad tance traveled « = (0), measured in fee, obey the rules ay ty 6? amd ‘where the horizontal and vertical components ofthe initial velocity are v< = v9 cas(bo) and », = vo in(bp), tespectively. The mathematical model expressed by tne rules in (14) is easy 10 work with, but tends 10 give too high an altitude and too long a range for the projectte’s path. If we make the additional assumpyion thatthe air resistance is proportional w the velocity, te equations of moti: become as yas) = ten 4220 (1-8) 004 and a6 rerth=ty (1%), ‘where C = ni/k and & 18 the coetictent of aur resistance and m 1s the mass of the rrojectile A larger value of C will resait in a higher maximnm alftude and a longer ‘ange fo the projectile. The graph of a fight path ofa projectile when ar resistance is considered is shown ts Figure 2.14. This mproved unodel is more wealistic, but requires the use ofa root finding algorithm for solving f(1) = 0 to determine the elapsed time until the projectile hits the ground. The elementary model in (14) does not require & sophisticwted procedure to Gnd the elapsed time.14 CHar.2 THE SOLUTION oF NONLINEAR EQUATIONS (2 300} os 200 | a Pgare 214 Pat of a ysicuite 2) Wad GO 300TH * ae wesunce cone Table 24 Finding the Time Whee te Height f(t) 1s Zero 7 [en pam] To — ormrrio zane 874200041 ~n.onse7s 0.03050700| aairdct 0.900001 =a.09000100| - 74217466 ‘0000000 ‘.ove000 Bxanuple 212, A projet is red wil an angle of elevation ty = 45°. vy = vy 160 sec, and C = 10, Find the elapsed time un rapt and find the range. Using formulas (15) and (18, the equations of motion are y = f(0) = 4800(t 19) 3201 and x — v(t) ~ 16001 ~ e110) Since f(8) = 83.220972 and £9) — ~31.534367, we will se the ial guess py = 8. The derivative #(@) = 4€00°#!9 — 2320, and is Yale f'(po) = J (8) = 108322025 used in formula (8) wo get 3.2009 /200 Tie mone = a 797731010. ‘A sorumary ofthe calculation i ven in Table 2.4, ‘The value pg has eight decimel places of accuracy, and the time until impact is ¢ 8.74217466 seconde. The range can now be computed using r(t) and we get 718.1421 1800)= 1000 (1 -€ 84) we gszapyosoer . ‘The Division-by-Zero Error One obvious pitfall of the Newton-Raphcon method is the possibilty of division by zero in formula (4), which would occut if #"(p4-1) = O. Program 2.5 has a procedure SEC.2.4 NEWTON-RAPHSON AND SECANT METHODS B {o check for this situation, but what use isthe last calculated approximation p—1 in this case? It is quite possible that f (pp) is sufficiently clase to zero and that px ian acceptable approximation tothe root, We now investigate this situation and will, uncover an interesting fact, that is, how fast the iteration converges. Definition 2.4 (Order of « Root). Assume thet fx) and its denivatives (x) . Fe) are defined and continuous on an interval about x = p. We say that Fee) = O has arootof order M atx = p and only if a Jo) erp and J°P(p) £0. £0 A ot of order AY = 1 is often called a simple root, and if M > 1, itis called mulipie root. A root of vider M = 2 is sometines culled 4 double root, ad 50 09, ‘The next esult wil illuminate these concepts, a Lemuma 2.1. If dhe equation f(x) = Oh u root of onder Af at x = then there exists continuous function (x) so that F(x) can be expressed as the product as) Fee) =e pMats), where hip) £0. Example 2.3. The function f(x) = x ~ 3x +2 has a simple root at p = —2 and a double root at p = 1. This can be verified by considering the detivatives f(x) = 3x? — 3 and f"(2) = 6x. At the value p = ~2, we have f(-2) = O and f° in Definition 24; hence p = ~2 sa simple oot. For the value p ~ 1, we have £0) =0. 71) =0. and {"() = 6, 50M = 2 in Definition 2.4: hence p = 1 is double oot, Also, note that f(x) has the factorization f(x) = (x + 2)¢x— 18 . Speed of Convergence ‘The distinguishing property we seck i the following. If pis a simple oot of f(x) [Newton's method will converge rapidly, and dhe stumber of accurate decinal places (oughly) doubles with each iteration. On the other hand, if p is a multiple root, the error in each successive approximation is a fraction of the previous error. To make this precise, we deGine the onder af convergence. This is a measure of how sapidly a sequence comverges. Deftntiin 25 (Order of Convergence). Assume that (pylg converges to p and Set Ey = p— py torn & 0. If two positive constants A # Oand R > Oexist, and a76 Chat? THESOLCTION OF NONLINEAR EQUATIONS fx ‘Table 2.6 Newton's Method Converges Quidraically ot « Simple Root SBC.2.4 NEWTON-RAPHSON AND SECANT METHODS n ‘Table 26 Newton's Method Converpes Linealy at» Double Root then the sequence is said 0 converge 10 p with order of convergence K. The num- ber A is called the asymptotic error constant. The cases RK = 1.2 are given special consideration 20) IF R= 1. the convergence of {7} is called near. en It R = 2, the convergence of [paz called quadratic, a 1f R islarge, the sequence {py} converges rapidly to: thats, relation (19) implies that for large values of » we have the approximation \Fa-.i1 = AlFf. Foc example suppose that R = 2 and |Fn| ~ 10"; then we would expect that |Eysi| © A x 10"! ‘Some sequences converge ata rate that is not an integer and We will see thatthe order of convergence ofthe secant method is R = (1 + /5)/2~ 1.618033989, Example 2.14 (Quadratic Convergence at a Simple Root). Start with pp 1nd use Newton-Raphson iteration 10 find the root p = ~2 of the polynomial fx) = Sx 42, The iteration formula tor computing {74} 15 2phy-2 Bhs 3 2 Pe = etre sing Formula (21) to check for quadratic convergence, We get the valucsin Table25. A detailed look atthe rate of convergence in Example 214 will reveal thatthe error in each suecessve iteration is proportional to the square ofthe error in the previous iteration. That is, Ip pussi® Alp ~ pels where A * 2/3, To check this, we use 0.,000008589 and |p — py \- pal and it is easy to see that lp — psi = 0.000008589 ~ 0.000008621 ~ - (Exot ° 2 aon000000 ‘0323808528 ‘0400000000 "01476190475 0 | 7200000000 |" —0.096969097 ~0:200000000 o.sts15isis 2a era (gare | aceon | creme (| fitom | chemo | caimoms | Sua 2 | =2003s9011 | oonsssve22 | oonssseott 064202613 7 Laszssomo | -omisscon | -naszxsonn assis 7 1 | timer | Cateaen | Coonne | SSnoythe ¢ | Smee | $ | immom | stasis | tamen | oman Example 2.15 (Linear Convergence at » Double Root). Stan with pp = 1.2 and use NNewton-Rephson iteration to find the double root p = I of the polynomial f(x) = x? — Using formula (20) to check for linear convergence, we get the values in Table 2.6. Notice thatthe Newion-Raphson metho is converging to the double root, bu at slow ruc, The values of /(p4) in Example 2.15 0 wo 220 faster than he Vales of #'(pe). 50 the quotient f(70)/f" (ps) in formula (4) is defined when pe * p ‘The sequcnce is converging linearly and the eror is decreasing by a factor of approx. imately 1/2 with each successive teiaion, The following theorem surmmataes the performance of Newton's method on simple and double roots. ‘Theorem 2.6 (Convergence Rate for Newton-Raphson Iteration). Aseume that [Newton-Raphson iteration produces a sequence (9, that converges tothe r00t p of te function (x). IF pis simple roct, convergence s quadratic and 3) lEneil Brune for ifficiently large. Mp isa oot fends Bf caverns a ad 24) JE nil Hat \En| for m sufficiently large. Pitfalls ‘The diyision-by-7ero enor was easy to anticipate but there are ther difcuties that se nt 0 easy to spot Suppose thatthe function ie fa) = x2 ~ dr +; then the sequence (px) of real numbers generated by formula (4) will wander back and forh ‘rom left 10 right and not converge. A simple analysis of te situation reveals that $2) > O and has no rea root.TR Cup? THE SOLUTION OF NOMLINGAR EQUATIONS fis} y asa Figure 215 (a) Newion-Raphson iteration for f(x) = xe can proauce a arvergent sequence Sometimes the initial approximation pp is too far away from the desited root and s small and the tangent line tothe curve y F'00 #2) is nearly horizontal. For example, f f(x) = cost) and we seek the root p = /2 and start with py = 3, calenlation reveals that py = —4.01525255, p ~ —-4,85265757,..... and (pal will converge toa different root ~3x/2 ~ ~4.71238898, ‘Suppose that f(x} 1s positive and monotone decreasing on the unbounded interval (a.00) and po > a: then the sequence (} might diverge to 400. For example, if fle) = xe and po = 2.0, then 19.723549434, 339333333, py and (m} diverges slowly 19 40 (see Figure 2.15(a)). This particular fanetion has ‘another surprising problem, The valve of f (2) goes to zero rapidly asx gets large, for example, 7(p1s) = U.0OXKQOSI6, and its possible that pis could be mistaken for root. For this reason we designed stopping criterion in Program 2.5 to involve the relative enor 2p, pal/(Ipel+10~), and when k = 15, this value is 0.106817, so the tolerance 8 = 10 § wall help guara against reporting a tlse root. Another phenomenon, eseling, accu when the tems in the sequence py} tend to repeat or almost repeat, Forexample, if f(z) = x—x~3 and the inital approximation pa = 1.961538, py = ~1.147176, pg = -0.006579, P= 1.961818, py = ~1.147430, and we are stuck in a cycle where pes ~ px for k = 0 1, ... (see Figure 2.15(b). But if the starting value py is culicietly close to the root p ~ 1.671699881, then (px) Sec. 2.6 NEWTON-RAPHSON AND SECANT METHODS ” faces pe Po 1 os 20 =} = scan produce a eyeic sequence Fo arcane) Figure 218 (©) Newton-Raphinn iteration for fx) = atctan(x) can produce a divergent oscillating sequence. conveiges. If py — 2, the sequence converges: pi — 1.72727272, pz = 1.67369173. ps = 1,671702570, and py = 1.671699881 ‘When |g'(x)/ > 1 on an interval containing the root p, there isa chance of di- vvergent oscillation. Por cxample, let f(r) = arctan(s); then the Newton-Rephson iteration function is g(x) = x — (1 +2*)aretan(x), and g(x) = —2x arctan(x). Irthe starting value po = 1.45 is chosen, then 1889109054, pi = -1,950263297, pr = 1.845951751, ps te cee Figure 2 15(¢)). Rut ifthe starting value is sufficiently close tothe root p = 0,80 Cuan? Tue SOLUTION oF NONLINEAR EQUATIONS f(x) Figure 216. The geomet contistion af fr these. cat method. 0.5, then a convergent sequence results. If p pi = ~0.079559514, pp = 0.000335302, py = 900000000, “The stations shove point tothe fact that we must be hones in reporting an answer. ‘Sometimes the sequence does not converge. Itis not always the case that after V ‘erations a solution s found. The user ofa root-inding algorithm needs to be warned ofthe situation when @ root is not found. If there is other information concerning the context of the problem, then itis less likely that an erroneous root will be found. Someumes f(x) has a detlaie interval in which a root is meaningful. If knowledge of the behavior of the function or an “accurate” graph is available, then itis easier to choose po ‘The Secant Method ‘The Newion-Raphson algoritnm requies the evaluation of two Functions per iteration, ft pe-1)and £1. Traditionally the calculation of derivatives of elementary func: sions could involve considerable effort. But, with modern computer algebra software packages, this has becoue iss of an issue. Still many Functions have uonelemeatary forms (integrals, sums, ec.) and itis desirable to have @ method that convertes almost as fast as Newton's method yet involves only evaluations of f(x) and not of f(x) ‘The secant method will require only one evaluation of (x) per step and ata simple root has an onder of convergence R = 1.618033989. Iris almost as fast as Newton's ‘method, which has order 2 ‘The formula involved in the secant method is the came one that was usod in the regula falsi method, except that the logical decisions regarding how to define each succeeding term are different, Two intial points (po. f (po)) and (pi. F(pi)) near the point (p,0) are needed, as shown in Figure 2.16. Define pe to be the abscissa Se. 2.4 NEWION-RAPHSUN AND SECANI METHODS, a ‘ble 2.7 Convergence ofthe Secant Method ata Simple Root mn * aaron Za. 5 soomony | ~ Seanonaoo | oor : oeume's | ago | oanaarres 2 ors, | atossivas | osrra0012 3 acute | ooarisosi | goaeusia | Gaaxanas 4 | =zonistioe | soowssscr | oamstiose | osnseainis 3 | “Bommazsey | oooomasis | womens? | onions? © | “zone | coomneoze | oooonn2 | =Poninoono | Sonne | oomooato ‘ofthe point of intersection of the line through these two points and the x-axis: then Figure 2.16 shows that pz will be closer top then to either pp or pr. The equation selaing pa, paid pv is fount by considering de slope 0s) mo Le and m= ©_L, Di= Po PP The values of m in (25) are the slope ofthe secant line through the fist two approxi ‘mations and the slope ofthe line through (p\, f(p1)) and (pz, 0), respectively. Set the right hand sides equal in 25) and golve for ps = g(t, po) and get _ £o00N.0.= po) F(p1) ~ f(p0) ‘he general term is given bythe Cwo-point tration formula Flpv(Pe= ps 26) p2= alpi. Po) on Secale on SEP =~ Top Tepid Example 216 (Secant Method at a Simple Roo), Star with pp = —26 sn Pin 7 4and ase the secant mediog to find hero p= =2 of he polynomial function fa—0 342 Tm this cae the iteration formula (27) is (P= 3 P+ 2 = Pat) Fi Pin —3PL+3pe-1 ‘isan be algebraically manipulated to obtain Piper + Papas ~2 oo Pret 4 * Fit PERI * ‘The sequence of iterates is given ia Table 2.7 . oa Pat = (PL PAD Pe (Pe Pe2 CuAR2 THESOLU (ON OF NONLINEAR EQUATIONS (2) =0 ‘There is a relationship between the secant method and Newton's method. For a polynomial function f(x), the secant method two-pount formula pk+i = §(Dk. Pk-) will reduce to Newton's one-point formula psi = gC) if ps is replaced by >. Indeed, if we replace px by Ps in (29), then the right side becomes the same asthe Fight side of (2) in Example 2.14. Proofs about he rate of convergence ofthe secant method can be found in advanced texts on numerical analysis, Let us state that the eror terms satisfy the relationship | Fen) re 27) G0) WBesal * [Eel ‘where the order of convergence is R = (1 + /5)/2 1.618 and the reation in (30) is valid only at simple roots. ‘To check this, we make use of Example 2.16 and the specific values and AFD 2S 2H = (2/3) = 0.778351205. ‘Combine hese and itis easy to see that lp = pst = 0.000022537 ~ 0,000021246 = Alp — pel!*"* Accelerated Convergence We comld hope that there. are rat-finding techniques that converge faster than Finearly when p isa root of order M. Our final result shows that a modification can be made to [Newton's method so that convergence becomes quadratic a a multiple root ‘Theorem 2.7 (Acceleration of Newton-Raphson Iteration). Suppose that the ‘Newion-Raphson algorithm produces x sequence that converges linearly to the root a= pof order Mf > 1. Then the Newton Raphson iteration formula en pe ‘ill produce a sequence (pg that converges quadratically tp. Sec.2.4 NewroN-RAPHION AND SECANT METHODS 8 ‘Table 28 Acceleration of Convergece ata Double Root ial ‘ n win on i iE? 7 Taanonoon | —oua9ae5e4 | ——aaanonoe 1 ocowoses | —o.nwosasis | —o.cosososos | oxesriasr8 2 owns? | “0000887 | “econonsae? a 100000000 omcceeo cxnon000 ‘Table29 Comparison ofthe Speed of Convergence Special Relation berween Method conseraons successive ear ems Bisection Ext © HER egal es Bet. AIL Secant method ‘Maltpe rot Bua © AIEG| ‘newom-penon naiape rt agi ~ aicel Seca met Simple root Beat © Ale SIE Newton Raphton ‘Simple rot Bess © AEE Accelerated Mapa rot Bens © AEE Newton-Rapbson ‘Example 2.17 (Acceleration of Convergence at a Double Root). Start with po = 1.2 spf essed Nevin Rapson tation to fn the double oot 9 = 1 of #0) a 3x42, Since Af — 2, de acceleration formula (31) becomes ‘and we obtain the values in Table 28 . “Table 2.9 compares the speed of convergence of the various root-finding methods that we have studied so far. The value ofthe constant A is different for each method.BA CAR, 2 TH:SOLUIION OF NONLINEAR EQUATIONS f(x) = ‘wen one mital approximation py and Using the iteration, | fee) Function [p0,eF¥ ky) -covton(E af pO delta, pollen maxi) Ainput ~ f is the object function input as # string '{” + d¢ is the derivative of f soput ax string ’af? 1 po sv the taivtel appronionrion £0 8 rere sf 7 = oita s= the tolerance for pO epsilon 6 te tolerance for the tuiction values y = atrt ia the anriaur numer of itarations = p0 is the Nevton-Rapheon approrieation to the zero + ie in the error extiaace for po and p3 Port (> pay? B Let f(x) 207 = 2e = 1 Start with py = 2.6 and py 9. Let fin) =A? 4 = 3. Slat wih py = 17 a ) and") do nt change to approximations (pe) #°(p) and fx) * 1"(p). Now use pat (a 0g 1. Lat 2) — 23 2-42. Set with y= LS and pi = —1.82 ; 1. Cube-root algorithm. Start with f(x) = x3 — A. where A is any real number. und Fae ea erive the recursive formula 419, Suppose mat A 1s a positive real number. (2) Show that A has the eepemeentation A = 672", where 1/4 £0,000062, and E3 = 0.000000, Estimate the asymptotic error constant A and the onder of convergence R of the sequence generate by the iterative method. Algorithms and Programs 1. Modify Programs 25 and 26 to display an appropriate eror message whet (i vision by zero oscurs in (4) or (27, respectively, or (i) the maximum aorber erations, maxi, exceeded 2 1s ofen tmuuctve wpa tetris We sequen genie by (4) atl (Ge. the second column of Table 24). Modify Programs 25 and 26 to display Sequences generated by () and (27), respectively ‘3. Modily Program 29 to use Newton's square-root algorithm approtmate eich the following square rots o 10 decimal places. (a). Start with pp = 3 and approximate V8 (©) Start wih po = 10 and approximate 5. {© Stat with p= andepprosimate VE 4. Mesify Program 25 tn xe the eube-oot algorithm in Exercise 11 1 approxi: cach ofthe folowing cube rots to 10 decimal places. (a) Start win pp = 2 and approximate 7°” (Stan with 7 = 6 nd approximate 200° (©) Start with pp = ~2 and approximate (~7)? Sec.2.4 NEWTON-RAPHSON AND SecaNr MerHons ry ‘5, Modity Program 2.5 0 use the accelerated Newton-Reptison algorithm in. Theo- rem 2.7 to find the root p of onder Mf ofeach ofthe folowing funchons (a) fla) = (x= 2)°. M=5, p = 2; start with po 0) fle) w sine, a 0; lar with po = 1 (©) (02) = Ue) Inte), M =2, p = Ly star with pp = 2 ‘6 Modify Program 2. to use Halley's method in Execcise 2 wo find the simple 2210 of Tha) 50 ~ Se4 2, sing pp = ~2.4, 7. Suppose thatthe equations of motion fata projectile sxe = FA) = 96001 — 615) — 408 arty = 24000 68, (@)_ Find the clapsed time und impact accurate 10 decal plas (h) in the range senate 18 decimal places 8. (a) Find she point on the parabola y 110 decimal places, (@)_ Find the point on the graph of y = sin(x — sin¢x)) that is closest tothe point 2-1,0.3) wocurate to 10 decimal places. that is eloset tothe point (3,1) accurate (6) Find the value of» a which the minimum vertical distance berween the graphs of fe) = x? +2 and g(x) = (4/5) — sin(x) oocurs accurate 1010 decimal pices, ‘9. An oper-top box is consiructed from 2 rectangular piece of sheet met measuring 10, by L6iaches. Squares of what size (accurate 6 0,008000000 inv} sll be a fon the comers if the volume of the box isto be 100 cubic inches? 10, A catenary isthe curve formed by a hanging able. Assume thatthe loiest point i (0,0); then the formula fr the catenary is y = C coshix/C) ~ C, To determine the sotenary tat goes thuough (La, D) we nwsi salve the equation b Ceusha/C) ~€ fore. (Show that the catenary through (2:10, 6) is y= 9.1889 cosh(x/9.1889) 9.1889. (©) Find the eatenary that parses trough ( >99 CHAt.2 THE SOLUTION OF NONLINEAR EQUAHIONS fC) 2.5 Aitken’s Process and Steffensen’s and Muller’s Methods (Optional) In Section 2.4 we saw that Newton's method converged slowly at a multiple root and the sequence of trates {pz} exhibited linear convergence. Theorem 2.7 showed how to speed up convergence, ht it depends on knowing the order af the root in advance Aitken’s Process ‘A technique ealled Aitken’s A? process can be used to speed up convergence of any Sequence that 1s Lnearly convergent. In order to proceed, we will need a definition, Definition 26, Given the sequence {pu define the forward difference Ape by my APe= Poti — pr for n>. Higher powers AK p, are defined recursively hy e Ahr Mea) Theorem 2.8 (Aitken’s Acceleration), Assume tbat the sequence {pq}%9 con verges linearly to the limit p and that p — py #0 forall n 2 0, WF there exists @ real nomber A with JA] < 1 such that a then the sequence (gp}$2 defined by (apn Opn nt Prva 2st + Po “ QP converges top faster than {pa} in the sense that oy a jee =o. ola pe Proof. We will show how to derive formula (4) and will leave the proof of (5) as an cxorste Sings the termi (3) ae approaching imi, we can write 6 EPs and ZOE A whem n is lage Pes P= Past ‘The reltions in (6) imply that o (= Past? © (O = Pas) (P~ Pa) See.2.$ ArrkEN's PROCESS AND STEFFENEEN'S AND MULLER’S Meroe 9 ‘Table2.10 Linealy Convergent Sequence (p4) Ee . ™ Fuamnp | eae 1 | csvessacco | casose7se | —oxecei6co9 2} ossszaoai2 | -o.nn1so1079 | -0.5s6119357 3 | osrs7030%5 | o.o1zssse0s | —o 57300209 4 | oSeonssea8 | -o.con0neees | 05396551 5 | os7it72149 | —ooowozssse | a seoisssss 6 | nscasaur | -oomasmas | —0 sensi ‘Table 211 Desived Sequence [ae) Using ‘Aiken's Process 7 ‘oser9e989 | o00015s6%9 2 | osenvsa | nownsss2 3 | Oserise36e 000016074 4 | oseriassss 005163 s Oo seriauos | onnnni62 6 | _ossniass2s | _aonnonas3s When both sides of (7) are expanded and the terms p? are canceled, the resus 2 Pae2Pn— Fj Tal gy for n=0, 1, bi PO pest — 2m + Oe The formula in (8 is used to define the term dq. I can be rearranged algebraically to obixin formula (4), which has less error propagation When computer calculations are mode . Example 2.18. Show that the sequence (p,.) in Example 2.2 exhibits linese convergence, and show that ihe sequence (ge) obtanea By astken's A? process converges faster. The sequence {9} was obteined by fixed-point iteration using the function ex) e-F and gariing with po = 0:5. After convergence has been achieved, the limit is P= 0.567143290. The values py aad yp ate given in Tables 2.10 and 2.11. Fos Husaliv, Whe valve of quis given by the calculation aan e Pi — 2p +P (-Woo1291 488 96530660 — OSETIA = 0,567298980. .92 CuAR? THE SoLUTION oF NONLINEAR EQUATIONS f(s) = 0 op s00 Figure 2417 The starting approximations po, ps, and py for Muller's method, and the Aifferences ho andy Altnougn the sequence {gq} mn able 2.11 converges linearly, it converges taster than Line} in the sense af Theowem 9-8 and neva Aitken’s method gives m hatter {improvement than this. When Aitken’s process is combined with fixed-point iteration the result is called Seffensen’s accelerasion. The details are given In Program 2.7 and inthe exercises. Mauller’s Method Muller's method is a generalization of the secant method, in the sense that it doe not require the derivative of the function. It isan iterative method that requires thre. starting points (po, f(po))s (Pi. f(p1)). and (po, f(p2)). A parabola is construct that passes through the three points; then the quadratic formula 1s used to find a 10: ff the quadratic for the next approximation Tt hac heen paved that near « simp oot Muller's method converges faster than the secant method and almost as fast a "Newton s method. The method can be used 10 1nd real or complex zeros of & tuncto and can he programmed to use complex arithmetic Without loss of generality, we assume tbat p2 is the best approximation to th uot and wonsider the parabola dough the dave stating values, shown in Figure 2.1 ‘Make the change of variable o and use the difference (1) ho=po-p2 and hy =pi~ pr Consider the quadratic polynomial involving the variable ¢: ay a +or+e. Sec. 25 AITKEN's PROCESS AND STEPFENSEN'S AND MULLER’S METHODS 93 Each point is used to obtain an equation involving a,b, and c: At ho: ah} + bho +6 = fos hy: akb+bhky +e= fi. =0: a +50 +e= fe 2) ‘From the third equation in (12), we see that «ay ‘ Substituting (13) nc the fist two equations in (12) and using the definition e andy = fi ereuls in he fneat system ahh +0h0= fo ak +h fone ay, Solving the linear system for a and b results in ne oh certo hg hoki = fg eo yh — heh ‘The quadratic formula is used to fiod the roots 1 1.22 08 (11) 16) Formula (16) i equivalent to the standard formula for the rots of a quadeatic and is better inthis case because we know that = fy To cnsure stability of the method, we choose the root in (J6) that has the smallest stsolote value. IF ~ 0, use the positive sign with the square root, and if B= 0, se ‘he negative sign. Then ps is shown in Figure 2.17 and is given by an P= pte ‘To update te iterates. choose pn and ps ta be the two values selected from amone ‘po, pty ps that lie closest to py (ie. throw out the one thats farthest away). Then re- place /Athith ps. Alihougi lt of auxiliary calculations are done in Malles’s method, ‘only requites one function evalustion per iteration. If Muller's method is used to find the real roots of (x) = 0, itis possible that ‘onc my encounter complex approximations, because the roots of the quadratic in (16) ‘might be complex (nonzero imaginary components) In these cases the imaginary com ponents will hve a small magnitude and can be set equal to zer0 so that the calculations Proceed with real numbers.94 Cuan 2 THE SoLUTION oF NONLINEAR EQUATIONS f(x) = 0 ‘Table 212 Comparison of Convergences near a Simple Root SeC.2.5 ARTREN'S PROCESS AND STERFENSEN'S AND MULLER's MeTKoDs 9S ‘Table 213 Comparison of Convergence Neat a Double Root Seca ‘Mall's News Seensen Seat Maters Newt's Stetensen t eed ested ‘ethos with Newion k smebod eho rch with Newton 3} 2 nanan | 2 eonnnnti —] 2 anno |= aan 2] heoonnon0 Taop00000 7 20600006 aa 1 | =2acoee00 | -2scon00000 | 2ovsis07s | —2076i9047 1 ¥ 200000000 ¥ 300000000 1103030303 iago30303 2 | “2howssines | “2aoooooom | poossasort | —aemsosort 2 | 1iseseiss8 1 200000000 1092356817 1082386807 3 aemens2 ioaszesze7 | 2000009509 o9es618i43 3 Vomsra7at anna Tomeannaia |p 6toness | -2onisiio% | -2oo0ssane2 | —2.000000000 2oomoioe2 ’ 10093854 103839922 rowzsma | ogeee4so23 5 | crows | CZeeomuzia = auaeet 3 1eozes0156 00002740 toosesssis | ogsazzi213 6 | —zon0000022 | 200000000 —Soo0002388 6 | toro aspesonais Loos32ss7s | 0999999193 7 | =om0oi00 | “2eccencono 1 | grapes? ose! 1.03607 0999993397 3 | toorsszi2e 1000000000 Hovossa0ss | c.eegeeno8 ’ ootsa757 Looosieans | Oseiae8 Comparison of Methods aa Steffensen's method can be used together withthe Newton Raphion fixed poiat func tion g(x) = x ~ f(2)/f"(x)._ In the next two examples we look at the roots of the polynomial fr) = x° — 3x + 2. ‘The Newton-Raphson function is g(x) = (x3 —25/Gx2 = 3). When this fmction is used in Program 2.7, we get the calculs- tions under the Reading Steffensen with Newton in Tables 2.12 and 2.13, For example, staring with pp = ~2.4, we woule compute as) pi = eo) = ~2.076190476, and a9) pa = a(p1) = 2003596011 ‘Then Aitken's improvement will give ps = ~1.982618143 Example 2.19 (Convergence near a Simple Root). This ic = comparison of method! forthe funetion f(1) = 2° — 3x + 2 near the simple root p = ~2. ‘Newton method andthe secant method for tus function were given in Examples 2.14 and 2.16. respectively. Table 2.12 provides a summary of calculations forthe methods. = Example 2.20 (Comvergence near a Double Root). This s «comparison ofthe aicdnes for the function f(x) = x? — 3x + 2 near the double root p = 1. Table 2.13 provides summary of calculations. 7. ‘Newton's method is the best choice for finding a simple root (see Table 2.12). AT double root, cither Muller's method or Steffensen’s method with the Newton Raphso' {ormula is a good choice (see Table 2.13). Note inthe Aitken’s acceleration formula (¢ that division by zero can occur as the sequence {p4) converges. In this case, the la ‘calculated approximation to zero ehould be used as the approximation tothe zero of J with the Newton-Raphson formula, is stored in a matrix Q that has maxct rows and ‘ee columns. The fst column of J contains the initial approximation to the root, and the terms ps, pe,» Pak, -- Generated hy Aitken's acceleration method (4) ‘The second and thid columns of Q contain the terms generated by Newton's method. ‘he stopping criteria inthe program are based on the difference between conseculve terms from the frst column of O. [ Promram 27 (Steffensen’s Acceleration). To quickly finda soktlon ofthe fixed | point equation x — g(x) given an initia! approximation po: where i is aseumed | that both g(x) and gc) are continuous, g"(x) < 1, and that ordinary fixed-point | eration converges slowly (linearly) 0p. function (p,Gl=ster# (f,af,p0,delva, epsilon wax) Minput — f i» the object function input aa @ string "£° i = af is the derivative of £ input as a string ‘df? x = pO ig the initial approximation to a zero of t % Getta ie the tolerance for pO % % ysilon is the tolerance for the function values = maxi is the maxisun number of iterations Yourput - p 42 the Stoffongen approximation to the zero * ~ Q is the matrix containing the Steffensen sequence Initialize the matrix R Rezeros(aaxt,3): RCL. Depo;9% CHAP.2 THESOLUTION OF NONLINEAR EQUATIONS f(x) = C for kel:maxi for j=2:3 ‘Denominator in Nevton-Raphson method is calculated ardenonsfeval (Af Re, -1)); Halculate Nevton-Raphson approximations if nrdenon==0 ‘division by zero in Newton-Rapheon method? brenk ele Rove, $)“R Ue, f-1)~foval RG, }-1))/medenan: end ‘WDenoeinator in Aitken’s Acceleration process calculate ‘aadenoz-R(x,3)-2+R(ie,2)4R (ke, 1) wealculate Aitkea’s Acceleration approximations Af aadenoa==0 Daiwinian hy zara in Aitkan’s Accaleration? break else Rk, 1)*R(K, 1) (RC. 2)-R(e.1))°2/aadenom; and end Wend program if division by zero occurres rf (ardencme=0) | (aadenom==0) break ond ustopping cricaria are evaluated erreabs(R(k, 1)-RG+1,19); relerr=err/(abs(R(x+i,1))“de1v8) yofaval (£ RU: 102 af (erzedelta) | (xelerr 0 dtacesare (b-2-48a%c) ; lee disco; end Wand the smallest root of (17) are dlacedisc: end 2ec/ feds P ‘Sort she entries of P to tind the t¥o closest to p Af abe(p-P(Q)) I and shrinks the vector when le|< 1. This is shown by using equation (10) Cbs tds tetehyl 2 AeXI = ay eR dt tad"? = loll. An important relationship exists between the dot product and norm of 2 vector. I ot sides of equation (10) are squared and equation (9) is used, with ¥ being replaced with. we have ay WP aaa eta x, If X and ¥ are postion vectors that lacate the to points (ej #3, aay) and (i 72..-.. 9») in N-cimensional space, then the displacement vector fom to Y 1s given byte ference a Y—X — (isplacement from position X to positon ¥). Notice that if a particle starts th: position X andl muves through the aisplacement ¥ ~ X, its new position is ¥. This can be obtained by the followin vector sum: CHa gt yexXs+@-4), Using equations (10) and (13), we cen write down the formula for the distance beaween two points in space. ay WX, hi \when the distance between points is computed using formula (15), we say that the points lie in N-dimensionat Euclidean space. (1-10? + 02-0) ++ GW am?)104 Char.J. THe SOLUTION oF Livear SyaTEMS AX = B Example 34. Let X = (2.-3,5,-1) and ¥ above are now ilostraed for vectors in 4-spoce. (6, 1,2, ~4). The concepts mentioned sum X+Y=@,-2.7.-3) Difference KAY =(-4,-4.3,3) Scalar multiple 3X = 6,-9,15,—3) Length Wx 41.9825 1 DY? = 39!?? Dot product X-¥=12-3410+4=23 Displacement from X to ¥ ¥-x=(4,4,-3,-3) Distance from X to ¥ (Y-XY— Ub + 1649491 — sol? is sometinncs useful ts ite vectors as eulamns istead of vows, Fur example, 6) X=] P| and Y= : cxw + dy By clowning ¢ and d appropriately in equation (17), we have the suum LX + 1Y, the difference LX — 1Y, and the scalar multiple cX + OY. We use the superscript" for transpose to indicate that a row vector should be converted toa column vector, and 4 1 08) Gta P| and [77 | = Grate, ‘The set of vectors has a zero element 0, which is defined by as) 0=(0,0,...,0) Theorem 3.1 (Vector Algebra). Suppose that X, ¥, and Z are N-dimansional vee tors and a and bare scalars (real numbers). The following properties of vector addition and scalar multiplication hol: (0) ¥4X=KeyY commutative property @) 0+¥=x+0 additive identity See.3.1 lntnopuction To VecToRS AnD Manices 2) X-X=x4+(-x)=0 additive inverse @3 (LLY) I Z=X |W |Z) associative propery 04) (a+ b)X = aX +bX distributive property for scalars 25) a(X4¥)= aX +a¥ distributive property for vectors (26) abX) = (eb) X associative property for scalars ‘Matrices and Two-dimensional Arrays ‘A matrix is a rectangular array of numbers that is arranged systematically in rows and wolumns, A matrix having M rows and N colurans is called an M >N (read “M by N”) rates. The capital letter A denotes a matrix, and the lowercase subscripted leter 0, denotes one of the numbers forming the matrix. We write on Azlawe forlsisM1sisM, where aus is the number in location (i,j) Ge, stored inthe ith ena an jth enum of the matrix), We refer to a; as the element in location (, j). In expanded form we. wate ax am y+ aw o rows | ain ain ay |e an ane omy omy wl The rows of the M x N matrix A are N-dimensional vectors; 9) vy “The row vectors in (29) can also be viewed as 1 N matrices. Here we nave sficed the MN matrix A into M pieces (submatrices) that are 1 x N matrices. Ti this vase we could express A as an M 2» 1 mautix consisting ofthe 1 > N row snatrices V;; thats, (ayia. escdin) for 1, 2s eens Mt (30) ={Vi We v Vue106 CHan3 THE SoLUHON oF LiveaR Systems AX: Similarly, the columns of the M x N matrix A are M x | matrices: an ayy ‘aw aay any oy On eee al Cael oun amy aun, Inthis case we could express A asa 1 x N matrix consisting of the M x 1 column smamices Cj: 2 aaler Cr €) Oy] Example 3.2, Identify the row and column matrices associated withthe 4 x 3 matrix The fourcow matricesare Vj =[-2 4 9},V2=[5 -7 1,¥s=[0 -3 8} and V4=[-4 6 —3]. The dee column matrices are ysl i “2 s 5 1 i ad Cs=| 3 -4 -5 Let A = layly cy and B = [bijiyyy be two matrices of the same dimension. ‘The two matrices Aand Bare said w be equal if aud only if each corresponding clement is the same; that is, G3) A=B ifand only if ay ‘The sum of the two Af x N matriogs A and B is computed clement by element, using the definition on A+B lai + bike. for < conte of ll zeros G9) 0 (Ohraw ; F i £ i : : forthe matrices 12 tans Aa]o7 3) md ae] = 3-4 oe Using formula (37), we obtain 24 6 9 a4 10) and =| 3-12 6 8. 721 ‘The linear combination 2A ~ 3B is now found “246 4-9] [4 =! za-ap—f ada wy i2}—ln 22 ee | ‘Theorem 3.2 (Matrix Addition). Suppose that A, B, and C are M x N matrices and p and are scalars, The following properties of mattix addition and scalar multi- plication bot 40) B+A~AtB ['7 34 Wa5]- tha ‘When an attempt is made to form the product BA, we discover that the dimensions ze rot compatible in this order beeauze the cas of Mase three-dimasional vetory ao dhe columns ofA are two-dimensional vectors. Hence the dot product of the jth row of B and the kth column of A is pot defined . IF it happens that AB = BA, we say that A and B commute, Most often, even when AB and BA are both defined, the products are not necescarly the same, ‘We now discuss how to use matrices to represent a linear system of equations ‘he hinear equations in (3) can be written as a matrix product. The coefficients ay; sem stored in a matrix A (called the coefficient matrix) of dimension M ~ Nand the ‘unknowns x, are stored in a matrix X of dimension N x 1. The constants by are stored ‘ma mamnx B of dimension Af x |, 111s conventional to use column matrices for both X and H and write an aa ay aw Pay Ph a a 2 ay {fre | |b = : : =|; |=2 ee aaa aun | | ay |] =| by amy ava amy + aun} {ew} [ow The matrix muliplication AY = R in (8) ig reminiscent of the dot product for ordinary vectors, because each element by in B is the result obtained by taking the dot product of row & in mamx A withthe column matnx X Example 3.6, Express the system of linear equations (5) in Example 34 as a matrix product. Use matsix mulipicaion w vesify ist [3 33} is he sedation of (5). v.12 0.200 y.av0)] [x1 2.3] % 0375 0500 o40o| |x| =|4% 0300 0300 900) [3] [28 ‘To verify that (4 3. 3]' isthe solution of (5), we must show that A(4 3. 3] [23 48 29): 0125 0.200 a400] [4] [05406412] [23 0373 0500 o600||3]=|15+15+18)=]as]. 0500 0.300 0.000113] }20+09+00! [29 2152 Cuan.3 The SoLUTION oF LiNkAR SYETEME A ‘Some Special Matrices ‘The Mx N’ matrix whose elen sion M + N and is denoted by sls ar all zero is called the zero matrix of dimen- 119) 0=(0lve¥. ‘When the dimension is clear, we use 0 9 denote the 2ero matrix ‘The Mentey matrix of order N is the square mawix given by 1 when ~ j, fe Ty =Wilven where &, i Med 0 whens 4. is the amiplcative identity, as illustrated in ube next exemple. Example 37, Let A bea? x 3 matrs, Then [2A = Aly = A. Multiplication of A on the left by F2 results in [fan en an] o | Ae au +0 anr0 on t0]_ 4 a) ‘Mulnpiication of A on the right by f3 reswtsin jt ag [t ee) feo) bean eo oro a ‘Some properties of matrix multiplication are given inthe following theorem. ‘Theorem 3.3 (Matrix Maltiptication). Suppove that c i¢ a scalar and that A. # and C are matrices such that the indicated sus and products are defined: then 02 (ABYC = CBO) associativity of matrix multiplication (3) [A= AI~A Identity matrix (14) ABE C)=ABTAC — Seftdisuibasive property (15) (A+ BIC—AC+BC sigh distributive propery (16) $(AB) = (cAVB = A(eB) scalar associative property The Inverse of a Nonsingular Matrix Ie concept of an inverse applies ro matrices, hu special auention must be Kiven.\1 NN snatrix A is called nonsingular or invertible if there exists an N x N wats such that un AB=BA SEC 37 PROPERTIES OF VECTORS Axo MAtRICES us Ino such matrix & can be found, A is sad to be singular. When B can be found and (17) holds, we usually write B= A“! and use dhe familiar elation as) AA 2 ACA if A is nonsingular Its easy to show that at most one matrix B can be found that satisfies relation (17) ‘Suppose that C 1s also an inverse of A (te, AC = CA = 1), Then properties (12) and (13) can he weed en abe CAC = (BANC = BAC B. Determinants ‘The deworminant of a square mateix A isa scalar quantity (cal number) and is denoted by det( A) or [Ai IEA iva N > N matrix ieee [eee tera its castomary to write jan aio ay OWA) Set er te Although he mation or determinant may Took fike a matin, its properties are oun pletely differen. For one, the determinant is a scalar quantity (real number). The definition of det(A) found in most Linear algebra textbooks isnot tractable for compu tation shen NV > 3, We will review how to wompute detevaiaants using the cofactor expansion method. Evaluation of higher-order determinants is done using Gaussian elimination and is meuioned inthe body of Program 3. MA — fay] isa se Late, we define det A) = 211. WA = ai heme here > 2 then let Af he the determinant of the N_ 1x N — 1 submatrix of A obtained by deleting the ith row and jth column of 4. The determinant Mf, is said to be the minor of a;;. The cofactor A,, of 2, is defined ws Ay, — (TIM, Then the determinant of an N x matrix A is given by 9 deta = Si, (ih v0» expansion)(CHAP. 3 THE SOLUTION OF LINEAR SYSTEMS AX = B w 2) deta) = ay ij (ith column expansion). a Appling fom 1, with = 3.40939 manic -[e ai) nits — ans, Te flowing expe lasts owt ase formulas (19) and (20) to recursively reduce the calculation of the determinant of en ‘AV x N maunx fo the calcUlation of a number of 2 x 2 determinants, Wwe see that det A Example 38, Use forms (19) wih / = 1 nd formula 20) with j= 2 tcl be Serna fe mui ael4 3 AL. L7-6 9} {sng ora (39) with = J we ebxin ee aa-al§ Jolt roft J (5-6) ON-364 74+ A= 35 =n Ung Sl 2) wth j = 2. cnn exay=-ai4 real s-col 2 tf =n. i The following theorem gives Sufficient conditions For the existence and uniqueness. ‘of solutions ofthe linear system AX = B for square coefficient matrices. ‘Theorem 3.4, Assume that A is an N > N mauix. The following statements are squivalent (2)) Given any N x 1 matrix B, the linear system AX = B has a unique solution (22) ‘The matrix A is nonsingular (Le., A~! exists). (23) The system of equations Aa’ = O nas the unique sotution (20 desta) #0, SEC.3.2 PROPERTIES OF VECTORS AND MarRices us ‘Theorems 3.3 and 3.4 help relate matrix algebra to ordinary algebra. IF state ment (21) is tue, then statement (22) together with properties (12) and (13) give the following line of reazoning: (2) AX=B implies AT'AK=A™'B, whichimplies X= A~)B. Example 39. Use the inverse mauix aif a ot oL5 andthe reasoning in (28) solve the ner systems AX = 3 1p] _ pe of QE}-B) [4 1P] 22) fos) Ler Rd 5 Ly” wz] Remark. In practice we never numerically calculate the inverse of a nonsingular rmatix ofthe determinant ofa square matrix, These concepts are used as theortceh “ool” to establish the existence and uniqueness of solutions of as a means 10 alge- bratcally express the solution of linear system (asin Example 39), 2. Using (25), we get xeate 1 Plane Rotations Suppose that Aisa 3x3 mauis awd = [x yz} ia3xt maxis; men ne product AU is another 3 1 matrix. This is an example ofa linear transformation, snd spplcations are found in the area of computer graphics. The matrix {i equivalent to the positional vecun B= (a, » <), whinh ecpreseus the eaves of point thre-dimensional space. Consider tee special mates: eco o 4 26) Re(a)=]0 costa) sina) |, 0 sina) — costa) cos(6) 0 | RO) eign e =sin(B) 0+ costs) cos(y) —sinty) 0 28) Re) =| sin(y)—costy) 0 o CeaM6 Cnar.d THeSoLUTION oF Linear SyStEMS AX = B ‘Table 31 Coordinates ofthe Vertices ofa Cube under Successive Rotations @ VERY WR RODE way ‘Ca, 00000, 67 ‘Donon, 6.9000, doo (0.0 {orm ox0ne7 9) {Denes O77 03335507 wo Caran. 0.70707. {0612372 0707107. 0353553) 0,01) (0.000000, 0.000000, 1)" (0.500000, 0.000000, 0.86025)" key {0.00000 1412814. 0) {0.00000 1.414214, 0.00000) ony {ornon o70707 (uanzsm2, 0707107 051272) wun Cornnion onuncr, ay (eoatasig etviiin1 2199197 ay {ocweene, 249 1) {0 Seen, 1.81214, 0866025" ‘Mhese matnces Re(a), Ry(B), and Kp(y) are used to rotate pomts about the and z-axes though the angles , A, andy, respectively ‘The inverkes are Ry (a), Ry(—A), and R,(y) and they rotate space about the x-, y- and zaxes through the angles —a, —p, and —y, respectively. The next example ilustates the situation, and further investigations are lef forthe reader. Example 2:10. A unit cube is situatd inthe fist ostant with one vertex atthe origin. Fir, rotate the cube through an angle /4 about the z-axis; then rotate this image through ‘an angle =/6 about the y-axis. Find the images ofall eight vertices ofthe cube. “The fist mtion ie piven hy the transformation con(§) sinc) 0] fs (aa cox i f oO TLE 0.707107 0.707107 0.000000] [= 0.707107 0.707107 0.000000 || y s.o0000 v.00 F-avaUNN | {Z| ven ‘Then the second rotation is given by : cost) 8 se mee yon gio |y . sin) 0 cost), 0.886025 0.000000 0.509000 ~| 2.000000 1.000000 0 00non0 | ¥ 0.500000 0.000000 0.866025, ‘The composition of the two rotations is : 0612372 0612372 0.500000) [x wen, (2)a(Z)e=| ororicr ar07107 0.0000 |s ~0.353553 0.353553 0.866025 [2 SEC.3.2 PROPERTIES OF VECTORS AND MATRICES 7 r if jag A © » © igure 82 (a) The onginal staring cube. (b) V = K.(7/4)U- Kotaton about the z-axis. (€) W = Ry(/6)¥. Rotation about he y-axis, [Nusnerical computations fr the coordinates of the vertices of the starting cube are gwen in able 3.1 (as positional vectors), and the images of these cubes are shown in Figure 3.2(8) trough () . MATLAB ‘The MATLAB functions det (A) and inv(A) calculate the determinant and inverse (fA is invertible), respectively, ofa square matrix A. ‘Example 3.1. Use MATLAB to solve the linear system in Example 36, Use the inverse matrix method deserted in 2. First we verify that A is nonsingular by showing that det(A) 0 (Theorem 3.4). 0,128 0.200 0.400;0.376 0.600 0.600;0.500 0.200 0.0001; soaee (a) 0.0175 ollowmng the reasoning in (25), the solution of AX = Bis X = AM'a. setesay(a)af2.2 48 2.0)? © ¢.0000 3.0000 3.0000 ‘We can check our solution by verifying that AX = B. >oveant 2.3000 44,2000 2.9000 .N18 Cixp 3. THE SOLUTION OF LINEAR SYSTEMS AX = B Exercises for Properties of Vectors and Matrices ‘The reader is encouraged to carry out the following exercises by hand and with MATLAB, 1. Find AB and BA forthe following matsees afd eb a] 2. Find AB and BL forthe following matrices ; af Beefs °. 3-2 3. Le A Rand be pety 3 ' a-B ae eis fo) Fed AyCand A080 (@) Find (ABy’ and BA’, 4. We use the notation A? = AAA. Find A? and B for the following matrices: anes aoe a-[$ oq} e-[ ss ae asa 5. Find the determinant ofthe following mates it exes ofiq ia! © [34 00. 6 Show that R,(a)R,(—a) Rea); (sce formula (26)). 7. (a) Show that Ryla)R, (A) = costa) ° sins) sin(B)sin(a) —cos(a) — os (f) sina) ~cos(a)sin(B) sinfad —cos(B)costa) (cee formulas (26) and (27). Sec.3.2_ PROPERILES OF VECTORS AND MATRICES 49 () Show that R, (A)Ry (a 0 costa) = sina) ox{A)sin() situ) cust sin(By =sin(a) c0s(2)sinfa) cos(B) costa) 8. IfA and B are nonsingular Nx. matrices and C = AB, show that C~' Hint, Use the associative property of matrix multiplication. 9. Prove statements (13) and (10) of Theorem 5.3 10, Let Abe an M x M mattis and X aa 8 1 main (a) How many muliplcatons ae needed to calculate AX (b) How many adltions are needed to calculate AX? 11, Let A be an M x N matrix, and let B and C be N x P matrices, Prove the left Aisributive law for matic meltiplcation: A(B +0) ~ AB | AC. 12, Let A and B be M x W matrices, and let C be @.N x P matrix. Prove the right N matrix, Example 3.16. Express the following system in sagmented matrix form and fied an equivalent upper-tiangular system and the Solution. stdin t bau = Dey +Oxg +415 +3nq = 28 Au t2n+2ns a= 20 Ait adnan 6 The augmented matrix is prone Fo 2 1 4/13 minz | 20 4 3428 m= 4221/2 mie 3 L313 2] 6128 CHAP.3. THE SOLUTION OF LINEAR SYSTEMS. AX = B ‘The first row is used to eliminate elements in the first column below the diagonal We refer tothe first row as the pivotal row and the element ay; = Lis called the pivotal clement. The values may are the multiples of row 1 that ae to be subtracted from row k for k= 2.3.4. The result aftr elimination ix 121 4 B Genes ie 0 “6 -2 -15]-32 Oia ecla|itas “The second row is sed to eliminate elements in the second column that lie below the diagonal. The second row isthe pivotal row and the values mz are the multiples of row 2 that are tobe subtracted from row & for = 3, 4. The result after elimination ic Tae eas (Olea ees ee pt | 0 0 =$ -75|-35 my=-t9 0 0 93 325) 483 Finally, the multiple mgs = ~1.9 of the third row is subtracted from the fourth row. rt the rests the upper-riangular system img :| B 0-4 2-5) 2 ay 0 0 -5 -75|-35 0 0 0 -9]-18 ‘The hack-substtution algorithm can be use to salve (11), and we get ‘The process described above is called Gaussian elimination and must be modified so that it can he used in most circumstances. If ayy = 0, row k cannot be used to gag im “ oe ee af) a ass ay lla] | al, axa far a ag a TPT, fap. ‘Then we will consuuct an equivalent upper-iiangular system UX = ¥: ay ay a3 ny | fe aye] 0 of of Bll] | oy ux=|° ay ay {]+3] =| aina | av. fo oo otf] fot.) i Step 1. Store the coeficients in the augmented matrix. The superscript on af!” means that this isthe fist ime that a number is stored in loestion (r, ¢) 4 ait) 1 | af) af aff ofa | fan on le oP oP on a | alas rou aa Gy | Suet ahi ak ol om ahaa | Step 2. If necessary, switch rows so that a{') # 0; then eliminate x; in rows 2 through NV. In this process, m,y is the multiple of row 1 that is subtracted from row 1. forr=2:N van al ath Po, fore=2:N 41 ity (Ps end130 Car. 3. The SOLUTION oF LivgaR SveTEMS AX = B The new elements are written a{? to indicate that this is the second time that number has been stored inthe matrix at location (r, ¢), The result after step 2is “0 09 | ah on a Sin | SN4 Or amen iueaw:| eohel Oa ay ms ay | Sarat 0 of a oh [oP | Siep 3, KE necessary, swatch the second row with some row below it so that 28) o£ 0- then elininate x2 in rows 3 through Nn this process. myy is the malt pile of row 2 that is subtracted from row r for =3: “The new elements ae witten af to indicate that this is Ue itd the Usa ber has been stored in the matrix at location (r, c). The result after step 3 is oy eames em (ae ot? ay tn | tna oO 2 | 2 OZ ug oh | Fra 21 oe 0a es ON | oa °) > | 9, 00 ayy vay | ais Step p+ This i the general sep. Wnecessary, switch mou p with come row beneath itso that lf) 4 0; then eliminate x» in rows p+ | through N. Here mp is the multiple of row p that is subtracted tom for. Ske 3.4 GaUssiaN ELmawaTiOn AND PivorINe at force=pHiinet aff? al? mp wai end end ‘The final result after xy.-1 has been eliminated from row N is Ie Ne it ai aig aty | athe Ge eee aa may ay | a my 2 | Qo eg ee ew ae 0 0 6 i | gar “The wpper-cangulasiaton rocen is now complete Since Ais nonsingular, when row operations ae serfomed the scesive mies are also nonsingular. This guarantes that o® #0 for alk in the construction process ence back subttutioncat be wed to sol UX = ¥ for X, andthe theorem is prove Pivoting to Avoid ayy = 0 aif) = 0, row p cannot be used to eliminate the elements in column p below the main diagonal. tis necessary to find row k, where a{2) # O and k > p, and then in: teschange row p and row & so that a nonzero pit eletent 1s obtained. This process s called pivoting. and the criterion for deciding which row to choose i called a pivots strategy. The trivial pivoting strategy is as follows. If a?) +4 0, do not switch rows. aff? = 0, locate the first row below p in which a{?? #0 and switch rows k and p Xn lesan anew clement af! #0, which nonzero pivot element Pivoting to Reduce Frrar HHecamse the computer utes fixed-precision arithmetic, itis possible that a small error will be introduced each time that an arithmetic operation is performed. The following example illustrates how the use of the trivial pivoting strategy in Gaussian elimination can lead to significant eror in the solution of «linear system of equations.132 CHaP.3. THE SOLUTION OF LINEAR SYSTEMS AX = B Example 3.17, The values 11 — sz — 1.000 are the solutions to Lasae1 + 5.281 6.414 ” 2A, 1Axy — 1.2102 = 22.93. Use four-digit avithetic (tee Exercises G and 7 in Section 1.3) and Gaussian elimination with aval pivoting to find a approximate solution othe system, ‘The multiple may = 24,14/1.133 = 21.31 of row 1 is tobe subtracted from row 2 t0 ‘obtain the upper tangular system. Using four digits in the caleulatons, we obtain the new coefficients 2 210213165281) = =1210~ 1125 = -1137 B= 293-2364) = 2295-1367 -1138 ‘The computed upper-tringula system is LIM +578 KALE 113.92 = -1138. Back substitution i used to compute x2 = ~113.8/(-113.7) = 1.001, and.x1 = (6.414 5.281(1.001))/(1.133) = (6.414 ~ 5.286) /(1.133) = 0.9956. . ‘The error in the solution of the linear system (12) is due to the magnitude of the ‘multiplier mai = 2131, In the next example the magnitude of dhe multiplier may is reduced by fist interchanging the first and second equations in the linear system (12) nd then using the trivial pivoting strategy in Gaussian elimination to solve the system. Example 3:18. Use four-digit arithmetic and Gaussian elimination with trivial piveting te sole the linear system 21d ~1.210r = 22.93 Liga + 9.28129 = 6.414 1.133/24.14 = 0.04693 isthe multiple of row 1 that i 1 be subir from tow? The new eeeticiete are a) = 5.281 0,04603( 1.210) = 5.281 4 0.05679 = 8.338 12) — 6.414 —0.0469%02.08) 641M 1076 — 8398 The computed upper triangular system is 2.tde| ~ 1.210 = 2293 5 38Req = 5.338 Bock substitution is used to compute sz = $.338/5.338 — 1.000, and xj = (22.93 1.210(1,000)) (24.14) = 1.000, . Ste. 3.4 GAUSSIAN ELIMINATION AND PIVOTING 1% ‘The purpose of a pivoting sumegy is to move the entry of greatest magnitude tc the main diagonal and then use ito eliminate the remaining entries in the colums. 1 there is more than one nonzero element in column p that lies on or below the mait ‘iagonal, then there is a choice to determine which rows to interchange. The parta. ivoting strategy itustrated in Example 2.18, is the most conimnon one and ie used ir Program 3.2. To reduce the propagation of error, itis suggested that one check the ‘magnitude of all the elements m columm p that lie on of below the main diagonal cate row Fin which the element that has the largest abealute value His, that i, lap} — mmaxtappl lapsuple +++ la ipls Kemps sud then switch row p with row k if k > p. Now, each of tne muplirs niry tor P= p~1,....1V will be less than or equal to 1 in absolute value. This process will usually keep the relative magnitudes of the elements of the matrix U in Theorem 3.9 i sane as Uiose in Uhe origina cueMiciewt mnaliix A. Usually, te ebwive of the larger pivot element in partial pivoting wll result in a smaller erzor being propagated, In Section 3.5 we will find that it takes a total of (4N 4 9N? — 7N)/6 arithmetic ‘operations to solve an Nx system, When N = 20, dhe ial aunuber of ariduuetic ‘operations that must be performed is 5910, and the propagation of error in the compu: ‘ations could result in an erroneous answer. The technique of sealed partial pivoting or equlibrating can be used to forther reduce the effect of error propagation. In scaled Partial pivoting we search all the elements in column p that lie on or below the main diagonal forthe one that is largest relative to the enuies in its row. First search rows ‘through NV forthe largest element in magnitude in each row, s4y (13) 5) maxtlayplelarpeiles- lara ll for r =p, ply “The pivotal row k ts determined by finding (4) ‘etal om [2 Uapzigl lanl Sp! Spei Se Now interchange wow p and A, wiless p ~ A Again, this pivoting process is desigued to keep the relative magnitudes of the elements in the matrix U' in Theorem 3.9 the same as those inthe original coefficient matrix A. Mi conditioning ‘A matrix A is called ill conditioned if thore exists a matt B for hich small pertar bations in the coefficients of A or B will produce lage changes in X = A~'B. The system AX = B is suid to be ill conditioned when A is ill conditioned. In this case, ‘tumerieal methods for computing an approximate solution are prone to have more ‘One cxrcumstance involving ill conditioning accurs when A is “nearly singular” and the determinant of A is close to 2er0. TM conditioning ean also acer in systemeTM Cua 3 THESOLUTION OF LINEAR SYSTEMS AX “ os Se 08) os ii ra2yed oo > deli 21 4:2043;6224;-313 21; >> Bel13 28 20 6)"; >> AvgeTAtow UIA. 9 1Me SULUTION OF LINEAR SYSTEMS AN — og 413 4328 422120 31326 (In the following MATLAB display, a is the element of greatest magnitude in te fi coluun of A and 9 isthe row number. >> fa, sHemaxtabe(A(d-4,))> 3 (©) Let Augup. 27 kogup=[L 2 14 13;0 “4 2 ~b 2:0 0-8 ~ {ULY] be the upper-tiangular matrix in (11). 5 -35;0.0 0 -9 -16) 1.0000 2.0000 1.0000 4.0000 © ~4.0000 2.0000 -5.0000 © 0° =8.0000 =7,5000 Oe oi otooe0 >> Yekugup(1:4,8) 13 2 “35 “18 i [Program 32 (Upper Triangularization Followed by Back Substation), To construct the solution to AX = B. by first reducing the augmented matrix [AIB1 to "upper-iriangular form and then performing back substitution. function X= uptrdk(,B) upul — A is au 8 x N nonsangular matrix * - Bie an Nx 2 matrix Wowrput ~ X is an Wx 1 matrix containing the solution to AX-B Uinitaalize X abd the temporary storage matrix C Te Nias zea) Kezeros(W,1); Crzeros(1, M91); Worm tho augnonted Avge [a 8); ‘Sec.34 GAUSSIAN ELIMINATION AND PIvoriNc for pei:N-1 Wartial pivoting for column p (, jJ=max Cabs (Aug¢p:¥,p))) wintercnange row p and j Crkug(p 3: Aug(p. :)=Aug(j+p-1, 2); nag(sep-1,2)=C;, if Aug(p.p)=0 "A vas singular. Mo unique solution’ bevak ead Welivination process for column p Aug (,p:ll+1) dug (x, p:t+1)-medug(p,p:N+1) 5 ena. and Weack Substitution on (UIY] using Program 3.1 Kebacksub(Aug(1:N, 1:10 ,Aug(4:N,W+1)) 5 ination and Pivoting Exercises for Gaussian In Exercises 1 through 4 show that AX = B is equivalent o the upper-triangular system DY = ¥ ana find the sation 1 2 +4n-6= 4 244 - Oye -4 x1¢5nt35= 10 3n+ 6y= 12 xtintig= 5 3 2 n+ ntone 7 at nt 7 “atta tI 2 de | a 9 Bo2n Hee 10 3 d-Intin= 6 Int B= 13 sttn-dn= 3 & -sntin~9= -1 w40m $35— 5 But 2+6n= 17 5, Find the parabola y = A + Bx + Cx? that passes through (1.4), Tr and @, 14).138 CHAH.3 The SOLUTION OF LINEAR SYSTEMS AX = 2 6, Find the parabola y = A + Bx + Cx? that pases though (1,6), (2, $). and (3,2), 7. Bind the cubic y = A+ Bx + Cx? + Ds that passes through (0,0), (1,1). (2,2). and (3,2). In Exercises 8 through 10, show thar A = Ris equivalent to the upper-tiangularey:tem UX = Y and find the solution, By +8nt4y404= 8 : arto ban sus 4 3x43 —3u= 6 x1 az 1 Tay 42m 10 futdae 12 mi43m+0n—2rs -4 9. Dei + 412 — 435 +04 In t4n- an bOu= 12 3n-3n-3u= 12 4y42te= 0 antag Day Pay Bus 9 $0. xy 122 1083 y= AVRO age 9 2a +3a— H+On wan m4 2ee Ory 4g + 283 — 5 tn tiny SntSm+25—4y= 32 15k |, Fin the solution tothe following linear system. wit d =7 2 +3n— nyt Bay 1 3ay = 10 2ey = dee = 12 12. Find the solution tothe following linear ayeten, nin ee Bi nts 9 Bey arnt 2a = 19 2a + 6x4 13, The Rockmore Cor, is considering the purchase of @ new computer and wil chonse either the DoGood 174 or the MightDo 11, They test both computers” ability to solve the near sytem Mx 59y—21=0 55x 4 89y 4=0. “The DoGood 174 computer gives. 0.11 and y = 0.43, and its check fr accuracy SEC.3.4 GAUSSIAN FLnAINATION AND PIVOTING 130 1s found by substitution: oor 0.00. 34(-0.11) + 550.48) -2 55(—0.11) + 89(0.45)— 3 10.9 and y = 1.01, and its check for accuracy ‘The MighiDo 11 computer gives is found by substitution 34(-0.99) + $5(L.01) ~21 = 0.89 '5(-0.99) + 89(1.01) ~ 34 = 1.44, ‘Which computer gave the hetter answer? Why? 14, Solve the following linear systems using (2) Gaussian climination with partial pivot- ing, and (i) Gaussian elimination wit sealed paral pivoting. (2 Set 101) yt 2-23 + 0.001440 eek Wy; 0001; =0 Oy Sot M— Olu =1 3x1 — 100r2 4+ 0.0123 = 0 Sn + m~100x3— 104 =0 2 100R- y+ -H=O 15, The Hilbert matrix is classical il-conditioned matrix and small changes in ts coet feicats will produce a large change inthe colton to the perturbed system () Find the exact solution of AX = B (leave all numbers as fractions and do exact arithmetic) using the Hilbert matrix of dimension 4x 4: t (0) Now solve AX = B using four-digit rounding antamenc 1.0000 0.5000 0.3333 0.2500 fe 0.5000 0.3333 9.2900 0.2000] yo A=lo3a33 0.2500 0.2000 0.1667 |" ol’ 0.2500 0.2000 0.1667 0.1429 o Note, The coefficient matrix in part (b)is an approximation to the coefficient ‘matrix in part (8).sau Cian.3 Tue SoLUTION oF LineaR Systems AX Algorithms and Programs 1. Many applications involve matrices with many zeros Of practical importance are ‘idioma systems (see Exercises {1 and 12) of the form diy ten aun + da +3 apna tds toate aman a die am 4 beat = Bn n-1aN-1 + dyn = by. Construct a prosram that will scive a tridiagonal system. You may ascume that row interchanges are not needed and that row k can be used to eliminate xy in tow k + 1 2. Use Program 3.210 find the sith-degre polynomial y =a) ape r aus! + aux? + and (6, 1. Use the plot command to plot te polynomial and the given points onthe saune graph Explain any discrepancies in Your grap, 3. Use Program 3.2 solve the linear syst AX = B, where A= laylyew and ay =i", and B = [by lv, where bus = N and by, = 22/61) fort > 2. Use N = 3,7,and 11. Theexactsoluion is X= [1/1 .., 1 1]. Explain any 4 Const progamthatchangesth pivot Seen Poeam 3210s pais ting ‘5. Use your sae paral protng rogram ftom Problem 4 solve the system given in Problem 3 for N = 11. Fxpain any improwements inthe sitions 6, Modify Program 3.2so that it will ficiently solve M linear systems with he sume covfficient mats A but difecat column matices B. The Mf linear syste look like AX, AX = Bp AXw = By. 7. The following discussion is presented for matices of dimension 3 x 3, but the con- cepts apply 10 matrices of dimension Wx N. If A is nonsingular, then A™" exists and AMES T Lat, C5, and C3 be the columns of A~! and Ry, Fs, and B be the columns off. The equation AA™! = I can be represented as A[Cr C2 Cs)=[B E> Bs] This matrix product is equivalent to the three linear systems ACL = Ey. AC2— Bz, and AC = By Sec.3.5 TRIANGULAR FACTORIZATION 11 ‘Thus finding A~" is equivalent to solving the three linear eystems. ‘Using Program 3.20r your program frm Problem 6, ind the inverse ach of the ‘olioing maize, Check your answer by compotng he prodoct AA and also by ting the command invA). Explain any ferences sot 16-120 MO 140 : 120 am 27001680 @) i : H| (©) | 240-2700 6480-4200 a 140 1680 —4200 2800. 35. ‘Triangular Factorization In Section 3.3 we saw how easy it isto solve an uppertriangular system, Now we introduce the concept ot factonzation of given mattis into Ue product of a lower ‘siangular matvix T that has I's along the main diagonal and an upper-riangular ma tix U with nonzero diagonal elements. For ease of notation we illustrate the concepts ‘with matrices of dimension 4 x 4, but they apply to an abitrary system of dimension NeW Definition 3.4. The nonsingular asin A has a triangular factorization if it can be expressed as the product of @ lowerriangular matrix Zand an upper-tiangular matrix U- v. a A In matrix form, this is written as ay a2 ay aie 10 0 OP fay aie ay aie ay an an aul _ [mn 1 0 0} | 0 up ms ws Sis ect tt bata tat ite ait rail ae aie ag | fa ase awd | 0 Od allan “The condition that A is nonsingular implies that uxs # U forall &, The mutation forthe cies in L ie my, and the reason forthe choice of m.; instead of will be pointed out soon Solution of a Linear System Suppose thatthe coefficient matrix A forthe linear system AX = B has «triangular factorization (1); then the solution 0 2 LUX=B142 CHAP.3. THE SoLUTION oF LINEAR SvSTEMS A. e canbe obtained by defining ¥ = UX and then solving two systems ° frstsolve LY = B forY; then solve UX = ¥ for X In equation form, we must fst solve the lower-nangula system ” =H may oo -b muntmmt om = mary + may tmasys+¥4 to obtain y, 9, yay and yg and use them in solving the upper-riangular system 4 wna +x tunes + max = y1 upon + waxy + zara = yo ants + waa 6 a Example 320. Solve 2+ Bert bry bar = 3x 410+ 1 +814 =79 xy + 122 + 103 + 6xy = 82. Lie te wing ectriton method an he ft at baad [room «4 4-1 S$ aln|3 id allo o 2 3 /ae2 sno] biz ieoo (ete arvana sve LY = #: 2mm Sntmt ne Ayr ty +29 +94 = 82. ‘Compute the values y1 = 21, yz = 52— 221) = 10,95 4021) — 1066) = 24, of ¥ © 79 — 301) ~ 10 = 6, ani 106 =24). Next write the 0 bintay+ x= 2 o 4r-254204= 10 Pesta 6x SEC. 3.5. TRIANGULAR FACTORIZATION M3 [Now use back substitution and compute the solution x4 = ~24/(~6) = 4,x3 314))/(-2) = 3, x, = (10 24) + 28))/4=2, and xs = 2-4 4) 20) = 1, [i234] i: ‘Triangular Factorization ‘We now discuss how to obtain the triangular factorization. If row interchanges are not nececeary when using Gaussian elimination, the mulipliers mj, are the cubdiagonal entries in L. [Reample 121. Ulee Ganesion elimination to conetruc the triangular Factorization of the smateix 3-1 43 2 6 ‘The matrix L will he constructed from an identity matrix placed atthe lef. For each row ‘operation used to construct the upper-tiangular matri, the multipliers mi; will be putin to ofa 3-4 a=|o 1 o//-2 -4 5]. norte G Row 1 is used to eliminate the elements of A in column 1 below ayt. The multiples may = 0.5 unl my = 0.25 of sow 1 ave subwacted from rows 2 and 3, respectively. These multipliers ae putin Ube matrix tthe left and the result is 10 0)fs 3 =r -05 1 offo 23 45 2s o 1ffo 125 625 ‘Row 2 is used to eliminate the elements of A in column 2 below azz, The muliple ms2 = 0.5 of the second row i subtracted from row 3, andthe multiplier is entered in te matrix atthe left and we have the desired tiangular factorization of A. 1 00)fs 3 = ~os 1 o|fo -23 45 o2s -05 ij[o 0 85 i ‘Theorem 3.10 (Direct Factortantion A = 1.1!. No Row Interchanges). Suppose ‘hat Gaussian elimination, without row interchanges, can be successfully performed to Solve the general Linear system AX = B. Then the matrix A can be factored as the rreduct ofa lower-triangular matrix L and an wpper-triangular matrix U: ry e AnWw.144 CHARS THE SOLUTION OF LinEaR SySTEMS AX = & Furthermore, L can be constructed to have I's on its diagonal and U will have nonzero diagonal elements. After finding I. and U, the solution X is computed in two steps: 1. Solve LU = B for ¥ using forward substitution. 2. Solve UX = ¥ for X using back substitution. Proof. We will show that, when the Gaussian elimination process is followed and B ic stored in column N {1 of the augmented matrix, the result after the upp ‘eiangularization step is the equivalent upper-siangular system UX = ¥. The matrices £.U, B, aad ¥ will have the form © Toga ommaug com aire g Ba | ote a, my) mya omy Py ea yg ai a “ ato? ay ay olka oO 4a 2 2 ea ay ha 2 2 2 vs[% o a Ow), =| eter at Mea ay faye ‘Remark, To find just L and U, the (N+ 1st colaun is wot uzeded. Step 1. Store the coefficients in the auemented matrix. The superscript on a! ‘tweans tha this isthe frst time that a number is stored in location (7, c) og Qi ant git) an 2 a3 4 | Sine oo WL, > a a of [el al) aS al mt | ayy ah a all oo all Fallon Siep 2 hinnimate x; in rows 2 rough Vand store the muliplier m1, used to eliminate xy in row r, in the matrix at location (r. 1) Su. 3.3 TRIANGULAR FACTORIZATION Ms forr=2:N rma maa fore =2: N+ 22) = of) — mn eal sn 3 The new elements are written a2) to indicate that this is the second time that a ‘number has been stored in the matrix at location (¢,c). The result after step 2 is mar a 8) 0 | ee alee mur a a ah a2... ‘rep 5, Bliminate x2 in sows 3 dough N and store the multiplier mra, used t0 eliminate x2 in row r, in the matrix at location (r, 2) torre 3.4 ee The new elements are writen af?) to indicate that this is the third time that a num: ‘ber has been stored in the matrix at the location (rc). on ata a ma a oP oB | a, eee ee daa) | a | 2), mm mor of oh |aPan Step p +1. This is the general step, Eliminate x» in rows p+ 1 through N and store the mulkipliers atthe location (, p)146 Cuap 3. Tue SouuTION oF LINEAR SYSTEMS AX. forr=ptiiw arp = aD 13D np = 1b fags Op = Pap fore=p+i:N+1 ae" = gle) — end end wal rep +0 “The final result after xy -1 has been eliminated form row Nis oa a “ Pe ale ci of oo oe mom am ol ae mt maa mina off far | “The upper-triangular process is now complete. Notice that one array is used to store the elements of both Zand U. The I's of L are not stored, nor are the 0’s of Land © that ie above and below wie diagonal, respectively. Only the essential coefficients needed to reconstruct L and U are stored! ‘We must now verify that the product LU = A. Suppose that D = LU and consider the ease when r S ¢. Tae dre is 9) dpc = mir! + mya? +--+ Mra? +a, Using the replacement equations in steps | through p-+1 = r, we obtain the following substitutions matt =a? a2, maa = a2) — af ina = ae — ah, uy aoe aes ane Wen the substitutions in (10) ae used in (9), the sesutis ema 0 +0 a4 oa =a a al? The other case, r > ¢, is similar to prove. . See. 4.8 TRiansiitan FacroRizArion w ‘Computational Complexity ‘The process for wiangularizing Is the same for both the Gaussian elimination and tn ‘angular factorization methods. We can count the operations if we look at the first NY columns of the augmented matrix in Theorem 3.10. The outer loop af step p + 1 re- guises WV — p = 0 ~ (p+ 1) + I divisions wo compute the multipliers mrp. inside the loops, but forthe fist N columns only a total of (WV — p)(N ~ p) multiplications and the same number of subtractions are required to compute the new row elements af?*? “Tis process is cared out tor p = 1, 2,...,.W — 1, hus the mangular factorization portion of A = LU requires a : ay Sa-mav-p+n= 22% muipcatons an dvsions, wos uw I=IN (2) Dev = p= p= EA EEN sotractions mi ‘Tocstablish (11), we use the summation formulas MMOD a Sy MoeDaMeD 6 OnkW/the triangular factorization A = LU has been obtained, the solution to the lower-ulangulas systeus LY = B will require 0+ Lp oe NL = (NP = N92 ‘witplications and subtractions: no divisions are required because the diagonal ele- ments of Z are 1's. Then the solution of the upper-tiangular system UX = ¥ requires 1424-004 = (N?-+N)/2 multiplications and divisions and (N? — N)/2 sub- twactions. Therefore, finding the solution to LUX = B requires 1N? multiplications and divisions. and N? — N subtractions.MB Chap. 3 The SoLUTION oF Lincan Systems AX = B ‘We see that the bulk of the calculations lies in the tiangularization portion of the solution. Ifthe linear system is to be solved many times. with the same ecefficien: ‘matrix A but with different column matrices B. ic is not necessary to triangularize the ‘matrix each time ifthe factors are saved. This isthe reason the triangular factorization ‘method is usually chosen over the elimination method. However, if only one linear system is solved, the two methods afe the same, except thatthe triangular factorization ‘method stores the multipliers. Permutation Matrices ‘The A = LU factorization in Theorem 3.10 assumes that there are no row inter: hues. ICis possibie that a nonsingular matnx A cannot be ditecty tactored as Aw LU. 10: Example 3.22. show that me tollowang matrix cannot be directly factored as A fiz 4 [a3 Suppose that A has a direct factorization LU: then 12 6) 4 0 0] fa eu as a3 48 -1/=/m 1 olf 0 uz us 23 5} [mi ma tfLo 0 ass ‘The matrices L and V onthe right-hand side of (13) can be multiplied andcach element of the product compared with the corresponding clement of the matin A. In de fst column, 1 = Jay, then 4 = mayiys = may, and finally ~2 = msquii the second columa, 2 = Taya, then 8 (42) + pe implies that up2 = 0, and finally 3 — msaung + mgauga ~ (=2)Q) 1 my2(0) — A, which ipa contradiction, Therefore, A does not havea LU factorization . A permutation ofthe first N positive integers 1,2,..., V isan arrangement ki, ko, «ky ofthese integers in definite order. For example 1,4, 2, 3,Sisa permutation of the five intogers 1, 2,3, 4,5. The standard base vectors E; —(00--- 01,0 --- Oh are used inthe next definition, Definition 3.8. An Wx permudation matrix P ix a malin with previsely one enuy whose value is 1 in each column and row, and all of whose other entries are 0. The rows of P are a permutation ofthe rows of the identity matrix and can be writen as, (ay P=[E, Bi Bul Spc.3.5 Triancutas Facroarzariow ” ‘he elements of P = [pj] have the form jak, 1 PY 10. otherwise. Forexample, the following 4 x 4 matrix is permutation matrix, 0100 : ooo era as) ra[b 08 Ol Lie ey ay my. . oo10 Theorem 3.1. Suppose that P= (Bi, Ej, ... Ej] is permutation mati Tie product A is a new matrix whose rows consist of the rows of A rearranged in the onder rows, A. to, A, rom, A Beample 825, et A bead mata P eth patna gen them PA icthe mats when ewe rene Pr) ow) A, 104 A, 10W3 A. ‘Computing the product, we have 010 O} fan a2 ax aw] fay ax a2 an 10 0 Ojfoy a a ax|_ as a2 as aw 000 t}Jey a2 a3 au) “fan an a6 ave Oot offen an ax au) Las am ass ase . ‘Theorem 3.12. Uf P isa permutation matrix, then itis nonsingular and P~ ‘Theorem 3.13. If A is ¢ nonsingular mauin, dhen there exists permutation matnx P so thal PA has a tiangular factorization a6), PAWLU, ‘Ihe proots can be found in advanced linear algebra texts Example 3.24. If rows 2 and 3 of the matrix in Example 3.22 are interchanged, then the resulting matrx #A has a triangular Factorization ‘The permutation matrix that switches rows ? and Tie P= [F\ EBL] Comput ing the product PA, we obtain 100 oot 910, P.180 Cuae 3 Tus SoLuTion oF LINEAR SYSTEMS AX = B [Now Gaussian elimination without row interchanges can be used: po [12 6 my=-2}223 $ mie tL 4 8 AL ‘After x has been eliminated from column 2, row 3, we have Tague pvr 10 7 17] =u. ma=0 [0 0 25 i. Extending the Gaussian Elimination Process ‘The following theorem is an extension of Theorem 3.10, which includes the cases ‘wmen row interchanges are required. 1hus tnangular factorization can be used to tin! the solution to any linear system AX’ = B, where A is nonsingular ‘Theorem 3.14 (Indirect Factorization: PA = LU). Let A be a given N x. ‘matrix. Assume that Gaussian elimination can be performed successfully to solve the ‘general linear system AX — B, but that row interchange are required, Then there exists a permutation matrix P so thatthe product PA can be factored as the product of alower-triangular matrix Z and an upper-triangular matrix U: PA=LU. Furthermore, L can be constructed to have 1s om its main diagonal and U will hss nents. The solution X is found in four steps nonzero diagonal 1 Constnict the mattioes 1. U7. and P. 2. Compute the column vector P B. 5. Solve LY = 2° for ¥ using forward substitution 4. Solve HI = ¥ foe ¥ using hack eabituion Remark: Suppose that AX = B is to be solved for a fixed mavix A and several diffe ‘ent column matrices B. Then sep 1 is performed only once and steps 2 through 4 a: used to find the solution X that corresponds to B. Steps 2 through 4 aze a computatio ally efficient way to construct the solution X and require O(N?) operations insteed the O(N5) operations requited by Gaussian elimination. Ste 3.8 TRIANGUTAR Facroar7arIon 1st MATLAB ‘The MATLAB command {L,U,P]=1u(A) creates the lower triangular matrix L. the ‘uppertrangular matrix U (from the triangular factorization of A), and the permutation inatcix P from Theorem 3.14. Example 3.25. Use the MATLAB command (L,U,P]=20(A) on the matrix A in Ex- sample 3.22. Verify that A= PAU (equivalent to showing that Pa = LU). (126 40 -1,-23-51; On) 1.0000 0 9 ~0,5000 1.0000 0 v.z00 0° 1.0000 4.0000 8.0000 -1.0000 (ou ore elasn i o10 oon 100 deny (P)eLed ae 235 . Aspreviously indicated the tiangular factorization meted is often choseu over the ‘iination method. In addition, itis used in the inv(A) and det (A) commands in MATLAB. For example, from the study of linear algebra we know that the determinant of a nonsingular matrix A equals (-1)* det 8, where U isthe uppertriangular matin {from the triangular factorization of A and isthe numberof row interchanges required ‘tw obtain P from the identity matrix £. Since U is an upper-riangular matrix, we ‘know that the determinant of U is just the product of the elements on its main diagonal (Theorem 3.6). ‘The reader should verily in Example 3.25 that; dew(A) = 175 (1PC175) = (1 detw), The Jollowing program implements tne process described in the proof of Theo- rem 3.10. [cis an extension of Program 3.2 and uses pail pivating. The interchang- {og of rows due to partial pivoting is recorded in the matrix R. The matnx R is then ‘uped in the forwasd substitution step ta tine the mann ¥.4152 Ciap.3. THe SOLUTION OF LINEAR SYSTEMS AX = B Program 3.3 (PA = LU: Factorization with Pivoting), “To construct the solu tion tothe linear system AX = B, where A is a nonsingular matrix. function X = lufact (A,B) Aimput - A is an NxM matrix & 8 is an Nx 1 matrix output - X ie an Nx 1 matrix containing the solution to A = B {Initialize X, ¥, the tesporary storage matrix C, and the row 4% permutation information matrix 8 Di wD-aiz0(A); Aezeros (W,1); Yezeros(N, 1); crzeren (1.0); ReiiW; for pet:N-1 APind Ue pivot cow for colum p (maxi, j]=max(abs(A(p:N,p))): interchange rov p and j Ap, = )=AC ApH) AC}p~t 2265 a-RCp) 5 ROp)=RGjtp-1) RCjtp-t) as Af A(p,p)—=0 ’4 ig singular. No unique solution’ break ond Yearevlate multiplier and place in subdiagonal portion of A for kepti:l BULT=ACK,P/ACP PD AGe.p) = mul AC, pet :N) *ACK, pet:N)-multaa(p, pet ND; wud end Wsolve for ¥ ya) # BUR); for kad YQO= B(RGD)-AGK, 1H) AY ed); Yseive for XQD=YQD/ACEND ; ‘Sue. 3.5 TRIANGULAR FACTORIZATION for ket-1:—1:4 XC) =(Y CE) “AC 1) 9X (e611) /A 0 5 and Exercises for Triangular Factorization 1. Solve LY = 8, UX = Y, and verily tha B = AX for(a) B= [-4 10. 5] and (B=[20 49 22] .whew A= LU is is Ola ie OulO]|2ieed Hes 3 |e as aie ol) Obst 6: ese ola ary ogee 2. Solve LY — BUX — ¥, and varity hat B — AN for (a) B~ (7 2 10J/ and @)B=[23 35 7. where A = LU is eee tog) at -t 2 9)=[-1 1 of[o 3 15 i-2oy Li 1 aie oy 3. Find the triangular factorization A — LU forthe matrices, santa 120 es @ | 10 3 m [31 6 atte ager Find the triangular factorization A = LU forthe matrices aaa re @ [2 5 -2 wm |s 2 aes nas 5, Solve LY = B, UX = ¥, and verify that B = AX for(a) B a eg) (l io oLiraga eG 4 -afefp ro elje ss ital) (eal loi0 aia o-2} Lg -p tlooo 1 6, Find the wianglar factorization A = LU forthe matrix 1 etHork aio aa 3026 17. Exablish the formula in (12)