CSC 3323 Analysis of Algorithms Part IT | “Putorch says that Sertarius, inorder to teach his soles that perseveronce and wit are beter than brute force, had two ‘horses brought before them, and set two men to pullout thelr tals, One ofthe men wes a burly Hercules, who tugged and tugged, but elo no purpose; he other was a shorp, weasel. {faced tailor, who plucked one hair at atime, amidst roars of Taughter and soon lef the tall quite bare” Cobham Brewer (asi0-1897) Content * Decrease-by-a-Constant Algorithms ‘Topological Sorting * Decrease-by-2-Constant-Factor Algorithms Fake Coin Problem “Mulupication la Russe + Josephus Problem * Variable Size-Decrease Algorithms + Compatinga median andthe selection problem + Inverpolatin satel + Searching and insertion ina binary search ree Decrease and Conquer © technique based on exploiting the relationship ‘between a solution toa given instance ofa problem and ‘solution to» smaller instanceof he same problem © Once such relationships established, it canbe exploited ether 1 Top down cutie oF + Bottom up (without recursion) © Also reterredto as inductive or incremental approach Decrease by a Constant + The she ofthe instance is reduced bythe same constant con each eration of the algorithm “= Typically this constant is equal to one Decrease by a Constant — Example| * Objective: Compute fin) =o" © Can compute it "top down" fo-ta if nad fin)= ° nea © Can also compute it “bottom up": {Wote: Same as Brute Force algorithm) times 5 © Houssain Kettani, Ph.D.CSC 3323 Analysis of Algorithms Decrease by a Constant Factor * Reduces problem's instance by the same constant actor on each iteration of the algorithm Part IT >=... bya — _ — Example + Objective: Compute fn} =" «= Can compute it “top down (#2 ifnisevenandn> 1 Insertion Sort + Objective: Sort an array Al. “Assume that Alt... 1] is already sorted Thus, Ali] $ AI2] $$ Ala~1] “Hence, we simply need to find the appropriate position of An] among the ather elements “= Typialy, this constants equal to two Ao 5 (dria nis odd anda « Note the dference between this andthe divide and congue dea duel) tava fir) = ° Honea 7 a Variable Size Decrease Content «= Asite reduction pater varies from one iteration of an algorithm to another * Decrease-by-a-Constant Algorithms + ample + nseron Sort + Ek sent for comping the greatest common + Topological Sring sec Reena * Decrease-bya-Constant Factor Algorithms © Based on geal, 1)» godin, m mod n) +* Variable-Size-Decrease Algorithms ‘+ Arguments on RHS are always smaller than those on LHS erable Sian Det went + Smale nether by conan nor by conan ctor 9 10 Insertion Sort (Cont.) ‘There are three ways to insert Ala 1. Sean A(t... 2] from lft to right until the first ‘element 2 A[n] is encountered, then insert Ain} right before that element 2. Sean A(t... 1] from right to left until the first ‘element All “Note the diference between this and selection sort algorithm, Ui] is not in its final position B Insertion Sort — Algorithm Aigorithm insertionson(A 1 ors evenarray by insertion sort ‘input: An are A. nf noderble elervents 1) Out: Ara A. sored nondecreasing oder 14 Insertion Sort — Example 89 |45 68 90 29 34 17 45 99 (68 90 29 34 17 Insertion Sort — Efficiency “= Basle Operation: The key comparison if AU] > v “Worst case: FAU]> All forj=/— 2,1 “In this case, Alf] > v= Al] is executed for every boot + cones «© Note that this is exactly the same count as the ‘worst case of selection sor algorithm fore 2t0ndo 45 68 a9 [90 29 34 17 veal) e/=1 45 68 8 90 [29 34 17 whe)? 1 ang AUn> veo 2 45 68 8 90 [38 17 aust eA 29 34 45 6889 80 17 jet 7 2 38 45 68 8 90 aueney 45 16 Insertion Sort — Efficiency (2) © Best case: If the input array is already sorted in ascending order se MALU =3) SAU} 04) =F Be “In this case, A] > v= Al] is executed only once on every iteration ofthe outer loop 7 © Houssain Kettani, Ph.D.CSC 3323 Analysis of Algorithms Content + Decrease-by-a-Constant Algorithms + Topologieal sorting + Decrease-by-2-Constant-Factor Algorithms + Varlable-Site-Decrease Algorithms Part IT Topological Sorting © A sorting strategy fo digraph. “Lists the vertices in such an order that for every edge inthe graph, the vertex where the edge stars is sted before the vertex where the edge ends. = Two algorithms: 1 DFS based algort 2 Souresrernval igorithn © Both O{|V) + [E|) using adjacency linked lit. Topological Sorting - Example = Consider a set of five required courses (Gx Co Co eh «= Some courses are prerequisite of others. Topological Sorting - Observa «= No solution exist fa digraph has a directed cycle “Thus, forthe solution to exist, the digraph must be 2 sag o alrected acyclic graph “© The topologlcal sorting has a solution ifthe digraph is ag DFS-Based Algorithm DFS traversal noting order vertices are popped off stack Reverse order solves topological sorting 2 Reverse order of dead-ends ‘ack edges encountered?-> NOT a dag! 2 Example, Revisited (1) Running OFS algorithm, we get: 1 OSG 2G 26 969696, 1G © Houssain Kettani, Ph.D. 24CSC 3323 Analysis of Algorithms Part I p icks in DFS — Example 1, D Revisited Topologically Sorting — Ex.1 Rev. [Verex [Onder of [Onder of Visit [Dead End ‘The topologically Sorted list: ci ¢ ——+e, +, +, ( | 2 The popping-off order: Cy Cy Cy Ce Cy 25 26 | a le, Reunied 7 Source Removal Algorithm Rey deleec, leet «eased on a drect implementation of decrease by ‘one and conquer 2 oe —, ef + Repeatecly identify and remove a source vertex, ie, vertex that has no incoming edges * Delete it along with all the edges outgoing from it, Saar ss delete Cy © there ae several sources, break the tie arbitrary = = = + ithere sno source then the graph i nota dag! @ +The order in which he vertices are deleted yields 9 The solution obtained is:¢, G33 ‘solution to the topological sorting problem No 27 28 Content Fake Coin Problem + Decrease-by-a-Constant Algorithms += Decrease-by-a-Constant-Factor Algorithms + Fake Coin Problem + Malkpication la Russe + Josephus Problem © We would like to design an efficient algorithm to «Variable Size-Decrease Algorithms etect the fake coin © We assume thatthe fake coins lighter than the genuine one + Ammong 1 identically looking cons, one is fake in abalance scale, we can compare any two sets of 23 30 © Houssain Kettani, Ph.D.CSC 3323 Analysis of Algorithms Fake Coin Problem — Algorithm * Divide m coins into two piles of Ln/2 Jeoins each * Leaving one extra coin apart ifn is ode “Put the two piles onthe scale “If the ples weigh the same, the coin put aside must be Fake + Otherwise, we can proceed in the same manner with ‘the lighter pile, which must be the one with the fake “This isa decrease by half and conquer algorithm Part IT Fake Coin Problem — Efficiency + Inthe worst case, we have the allowing recuence: Win) = W(L n/2 J) +1 form > 2, WII) =0 + Amostdentcal othe nen tary search {aference nthe inal condton ony) thus, wt) = log | + We can nteatetheefficeney by ddng the coins into 3 piles of | n/3 | coins each and proceed moore * Inthis case, Win) «| torso | 31 Multiplication a la Russe «Also known as the Russion Peasant Method “Let and m be positive integers whose product we ‘want to compute © Let us measure the instance size bythe value of 7 “tf nis eve: instanceof hal he size hast desl with m/2 hen we haven m= n/2 2m if nis ode: ‘then we havea <= (n= 3/2 «2m 4m * Stop when a= 1 (1% m=) “We compute n x m either recursively or iteratively 2 [= Multiplication a la Russe — Ex. © will compute 50 x 65 by Multiplication la Russe: as [330 hs [iso [50 [260 [307 2 [260 Is [sao is [520 fs [uoao fs [xowo [000 Pan P80 2.080 cao 5250 33 Multiplication a la Russe — Efficiency * Algorithm Russet) {frmlements maltipcation 3a russe recursively ‘neu Two postive negers ny and {posto The produc of adm 'fnmod2 =O return RussRecin/2, 2m) fe return Russeelin-2}/2, 2m ln} eetlog, where nm © Cn} £618, heres the misimum numberof bits nn 34 Josephus Problem — Story “© Named after the Jewish historian Flavius Josephus (37-101), “= He partieipated in and chronicled the Jewish revolt of {65-70 AD. against the Romans “He managed to hold the fortress of otapata for 47 days «© After the fall ofthe city, he took refuge with 40, dichards in 2 nearby cave in Masada 35 © Houssain Kettani, Ph.D. 36CSC 3323 Analysis of Algorithms Josephus Problem = Story (2) «© There the rebels voted to perish rather than surrender * Josephus proposed that each man in turn should aispatch hisneighbor “The order sto be determined by casting lots, * Josephus contrived to draw the last lot + As one ofthe two surviving men in the cave, he prevailed upon his intended vietim to surrender to the Romans! Part IT Josephus Problem © Let people numbered 4 ton standin a circle «© Start atthe grim count with person number 1 «© We eliminate every second person until only one survivors lft «The problem isto determine the survivors number J(n) 37 Josephus Problem — Example 6 a 7 * = a 6 3% ; 54 ‘3 ® Instances ofthe Joseph problem fra) = 6 and (8) =. Subsript numbers indieat he pass on whic the person in that postion eliminated The slutone are Ne)= Sand 7) =7. 39 38. Josephus Problem - Effi © Hf nis even (0=24) iency 1 The frst pas ofthe ccleyields an instance of exactly he same problem bt half the intl sie + The oly aiference in postion numbering 1 Ths fora sui 2k =28)—2 ifn 2igodd (n= 24+ 2) ‘+ Te fist pas eliminates peopein all even postions + fe add tothat the elimination ofthe petzonn goon 1 Fah after that we ae lot witha instance fs 1 Thus, fora survivo J2k= 1) = 2081+ 2 Josephus Problem - Efficiency “We would ike to get a closed-form solution tothe tw: “The tial condition ss12)=1 “Tetum out that (a) can be obtained by 2 one bit oft tele shit of nsf = bxample | Thisalso equivalent 0} = 2(0— 25!) +2 rm 40 Jusephus’ Special Cases * Mersenne Numbers ‘+ Name after the Trench scholar Marin Mersenne Us 1648) ‘+The Great Intemet Messenne Prime Search (GIMPS) is project that finds Mersenne Primes (wwwmersenne O78) Mgr atatenan, (G78) + Thus, J) = = JQ") =Joo.-0,) + J") = fto.01) = 0.00 © Houssain Kettani, Ph.D. a2CSC 3323 Analysis of Algorithms Content + Decrease-by-a-Constant Algorithms * Decrease-by-2-Constant-Factor Algorithms * Variable-Size-Decrease Algorithms + Computing median andthe selection problem + Ingerpoation seasch « Searching and intrton ina binary search ree Part IT | Selection Problem & Median “© The selection problem isthe problem of finding the smallest element n alist of n numbers “This number is called the KP order statistic “The medion of a ist of n numbers isthe element that {s greater than one half ofthe lists elements and smaller than the other half * Inother words, «=[n/21 43 <> Computing a Median & the Selection Problem “= Note that we are not asked to sort the array “= We start by parttoning the arrays elements into two subsets: ‘Tre one containing the elements that are less than or ‘equa ta some value p pivoting the partion, ane ‘The elements that are greater than or equal top Partition © We pick the pivot p= All) Select two pointers, /and that go left-right and rignt- left, respectively © We use the pointers to scan and exchange the content as [t's compared tothe pivot, = We increment i and decrement j Partition — Cases Af pointers cross over, le, />/, we have partitioned the aray after exchanging p with Al) ie ‘if pointers stop while pointing to the same element (which Isp), .e,/=J, then we have partitioned the ara. Sie © Thus, swap p with ly] whenever 2 Aleas=tl, Al=g, Ale An 2 le ee [e[aioeee [wae = [Ao [ ata] 5 «6 Partition - Algorithm {Teetcosssnee orate se {equ henry Ao dy loan in Mouiptpcton oA. me et pion era pemmbre ners Tevet € io LunsAline teest/ etal sn a A wns) el Al ana emp ten 2) seo ah eo a7 © Houssain Kettani, Ph.D.CSC 3323 Analysis of Algorithms p ‘Computing a Median & the Selection Problem “We have three cases: Me thon p sles the election problem \f5>4, the # mals elementin the entre et can be founda the females eement inthe let part oF the partioned aay If. we can proceed by searching forthe (k= 5)® seve lement nits ah part “Thus, if we do not solve the problem outright, we reduce is instance toa smaller one which can be solved recursively Part IT Example - Median “© We want to find the median of the following lis: 4,4, 20,9,7,12,8, 2,15, © Note that k=[9/21=5 «= Thus our tacks to find the SP smallest element inthe given array © We use the same version of array partitioning that we used nour discussion of quicksor. = Thus, we select the first element a a pvot and rearange elements by two oppositely directed scans ofthe array 49 Example — Median (Cont.) Soe soSckeS merce ihe epee Sinces=6> k= we proceed withthe pf the bla 7d Sincere. meson.Ta the medlan sb 50 Algorithm’s Efficiency The worst case efficiency is O(n") “It turns out thatthe average case efficiency isthe same asthe case of reducing the size by half «= Thus the average efficiency solves Cte} = cln/2} + (n+ 3}, Ca) =0 © By the Master Theorem, the average efiiency is ©(n) 51 Interpolation Search “© Recall that ina binary search, we always compare 3 ‘search key with the middle value of a given sorted array “Hence we reduce the problem’ instance sie by ha “Interpolation search takes into account the value of the search key in order to find the aray’s element to bbe compared with the search key 52 Interpolation Search (2) © Consider a portion ofthe array with leftmost element All] and the rightmost element Afr) © On the iteration dealing with this array portion, the gorithm assumes that the arrays values increase ineary * Ley along a straight ine through the points (Al) and (7, Al) 53 © Houssain Kettani, Ph.D. 54CSC 3323 Analysis of Algorithms Interpolation Search (3) «The search key value vis compared with the element ‘whose index is computed as the x coordinate of the polnt on the straight line through the points (A) and (Ale) whose y coordinate is equal to the search valuev © Using the standard equation ofa straight line: well -AllieAyeale Alm Part IT Interpolation Search (4) index 55 56 Interpolation Search (5) Interpolation Search — Efficiency “The algorthm searches in the same manner among + On average, tis algorithm performs fewer than the elements indexed tog,log,0 + 1} comparisons when searching na Iist « Setween and = HAD) ce of nrandom keys f Between + Land Ae)» “This function grows 50 stow «The algorithm stops ity = Als] E Sleeisaninie ae * Thus, the ste of the problems instance is reduced, en emacs ties Cee) but we cannot tll aprivi by how much 87 58 Searching & Insertion in a Binary Searching & Insertion in BST — Search Tree Algorithm “+ Recall that a binary search ree binary tree whose nodes contain elements of set of orderableitems Wen searching fran element ofa given value vin such + Far every node al element nthe let subtree are salar bree fae eee ex le anal th elements the right subtree are greater han ¢ the tree emt the search ed in fal er neeees + ifahe tees nat empty we compare v with the tre's root ih HU a ded elements found andthe search can be ‘opoed és + vex cominue eorcing inthe et sobre othe rot + ifv> aU coninue eee inthe igh sabes of the root pease 59 60 © Houssain Kettani, Ph.D.CSC 3323 Analysis of Algorithms | Searching & Insertion in BST (2) ‘Thus, on each iteration ofthe algorithm, the problem of searching in a binary search tree is reduced to searching ina smaller binary search tree “= The measure of size isthe height ofthe tree “= Note that the decrease in a tree's height normally ‘changes from one iteration to another “Thus, this san example of variable size decrease algorithm Part IT Séarching & Insertion in BST — Efficiency + The worst case happens fa trees constructed by ‘econtve ncerton ofan incrantng or decentng Sequence of keys © The worst case efficiency is Of) © The average case efficiency is Zlog(n) © Ollog") “© Note that the prablem of inserting a new key ina binary search tre i almost identical to that of searching there «= Thus, both exemplify the variable size decrease technique and have the same efficiency 63 © Houssain Kettani, Ph.D. 2