15
In thie Section We briefly fl
feme. basic reoulls defined
enparkamt im umdenstand i
convergence. Sf com wlatiGnal algori
camlyatned amd ee alrined a
Hand for Which Aubbicient- ophmali ty
The only functions for Which neccaran,
ilfouk lineariSation, We Shall alse Ama
function sé Which ave ©"
Aubseb I
ianizat
is a collec!
Sia vebbeved fo a9 UG compen
aefinition (8°). An nedimensienal yeah SNe
poration £E
Sm what §
mn mean Z
pevertor.
FAVELLLELELLE ALLELE EEEGLLEELLEGbEE GOO?
hoduce convex am
|. eli ffovenl anpecks
Bans fer
Sohial for understonding. he
Z% is am clemen'
lows We Shall take
Stkerwibe, Stated.
J concawe functins amd Mhaiv
Where veouls ave extrem
ah ‘aplimizatien techmigtes ,' he
fncling
ian, 6) 3
jteria cam be
Cendiiens com be given —
erralizations &
topics | covered in IRIS
a.
oni
timality
fame gem
cheptew.
i 5
soil ecley or Tedimenmional or se Bor omy Imlegval 7
Definikion (n-vester). An n-veckor ™ Ge neers fo thak the veal number
— tion (41% * %m) vector X
Bf the clement 6
Jidiam. space 14 demBled by RM Thee
ie RD amd hemes is @
x tp be am n-dimensional
Column vector umleae
. _pa\ -@% mm) ER |
ie = [% en {
( |
Xr /axt |
we Ax (25 yen oid fo be positive definite iF |
matrix, A= (nen : lin din iaambetel
So frall xeR and i4 oid te be pas ve Bermi definite |
po fal xeR”
m1
. a) fh ara ain 2
age ® “C Sisson "
Gham apy 212m o»
‘ = TD aj%%
ase .
tthe form LAX 18 aalled Quadratic form’ in SDLUEGLEBLGELELELELELELEEGELEEG EOE EEE
Yhen Me partial derivative
jectively Known. 07
ile W * yet
Definition” Gradient of 0 funtion) Let F282.
of F With veospect fo each é the 71-Variablen ove coll
Ihe gradient of f and is Hrillen «
UF
Vex) = f a
OE
25
ze
am / nx!
~( i jiadives 13 eck fo cack. SE
Definition (Hessian Metrix) She Second partied derivatives WilK reepeck Fe
—— (eee e Res R” is a mabvix dendted by H(#) Is called
Hessian matrix 5 f ond is sive bY
ge OF. at
Hi) = Be OHH ‘Oxe%
= af 4F |... oe
an “ae ox
2 a,
FE BE on ens BF
DnOX% BHM Oxf
v
2 "
Exomples 51 Gy fs antes, = anfary t% ore positive definite
aparece forms im two omd Hivee Vowiableo reapeckively?
A 1, 2
i) Re arty 148 43%
= Gyn) 3x
20
is a posihive Aen definite qadvalic form Aince epialily halds
(fo ay =% and Hae
‘ a
WW tye cane? | = 3-9" ave negative clepinite in
4150 Varriablen
Gv) Ke daa is on inde fine qadratis farm Since ibis
positive for ome Value a, 2 Ms ty -1) and negative far
same Eure Vekuer i 1,32 (ib: =o
Hence a apadrabic farm xTAX fer all XER® 16 Anid to be
indefinite if Ie form {4 posihive for Some XER” amd negative for
ome shor XER”
"
\ 3 }
impertamk Reo
A set 4 neceas
‘, ary amd subticien condilions for i
form x74 x fo be positive definile if reprincipad apis rei
all positive
ie a oie en a dy
2 ayo, | 50, fan an ag | do, ------ deb(A) oe
mt a oa 42 9 /
Again if x1Ax is negative definite len 2™GA)x (4 positive.
debinitte - Furthermare We recall tRal- det(-A) = 1)" deta). Shere tee
neceanory aud aubbicient Gmdibion fv xtAx fo be negative, hinite
ov eqpivalently TCA) x te be positive dabirite: axe
ais a
aco, [& Sfpoe [BER on late
oy 222
, oor O52 3) elements of A (Not of -A)
here we Shovld vemember tak aj ore
aye. mek
) Givem FOE) = 8x FART
Here DAZ =(H oC 2)(3)
since ay 2302 [6,08] = [22 [22 Berto Fe)
is positive debimite qpactrastic Form
3
pany tame tows 1 EER
i) Given §(4) = setae se “
Here XAZ = (%% m( 2 :) (=)
1
35
gow, (*
ae 7 |g) ae
ay an O13
a) Gr 3
M3 932 G3
§(Z) by
(1-9) o(2')4 20) aLc-z tax]
G-ayaieaxtet
plaich ineqpallly bdds ame 6
Je Hol
. ih foid to be convex anf, if iF 1 men, sheet ie ne.
this debinition convex function 16 AN! ly snore: ° :
Sorry Ziptitien mE itevatuve im that () He do ee
[rhe be a convex deb Gi) He debine convexity ato point bir .
Genvexity on a deb, Bhid generalization ill allew un te hamclle a femesthak-
igider claao SL. problems laler. Tt fellows jumediately from Me above debinibion
Kak-a numerical funchen @ defined am a Comex
Qis Said to be Atvictly convex if the
ig and only if
an yc epaystanet
wane rt => Gopotat) +n 0) 2ol-n2'4az']
a
.
® i
Ob)
toy
1 e
=f
I
~ or"
Convex funchion » ..§ on wp,
Bibsel Se RMR
canvex function @
oS ee
Definitim (Concave Jumenon, A fumekic 0 debi; 7 .
Concave ob HET {ote 0 setinediok ers” Ud dat be
er
osael S G-n 0Cz!) +20(27) < eLet-anz'+ Ax? ]
Gr) zie axer
6 is Aaid jo be Strictly comeave if the Strict ineyality hélds and
@ iS Aaid te be concave an [7 if ibis cmeawe at each XEL
LULLED DELEKBKDDVEULYBUW YY
ek Lo is Convex an LT CLULLLOLLECLLEE
tt
)
TTT
UTT tT
v1
wa
i
oe
Obviowly Q is concave at XEL (cmcave aml’) if and omly iF —0 is
Comer at 2€1 (convex an [')- Renulks obtained for Convex Functions cam
be changed ink concave funchiond by He appropriate mulkiplicahin by (—1)
ond vice versa. Hence it follows teanieallately from the above definition _
HRal-a numerical Fumction @ debined on a Zemnver Bet P18 Cancawe on
Sif ond only if
2, Behe G-ayo(z!) + aa(z?) < o[G-az'+az?]
oxnel
R
oft-adz's327]
So OP)
ae y+as Ge)
'
Concave Function @ on Aubsel-
KR Concave Function 9 on (= [2.1]
Example 15:I-3 shew that the linear femclion f(z)= SX +a is bSIR convex
end concave om R’
Séution Lek Xe€R" and HER
then [axe (-22i] n( laa ta) + (r9)(ER 44)
fee) + (IEA) For al 2
“Some IMPORTANT THEOREDS
anedimensional vector function cefined
ae be am
Theorem — Lek § (fe ee enn mm-negative lineancmbinralin
per". aff is convex oF EL,
of ile Components fpr t¥% “™ ,
OE) Hb ICE), 420 18 conven a 2
Proof, ber 2el, ”
then Ofir-mz'4>* J
engl amd tah Geayzi4 ax” Ef
= pefera2'4a2"]
< plo-wsala aia
yaa) pECEID FART)
-ayo(z!) +20(2*)
by covery of
a) Pee pre
Hence 0(x') iS Convex ot x! i)
Ba
Fe(Ha)
Corollaery he ypadvatie. function
f(2) = ox +h XAx
is convex on RY if amd only iF
Furliew, if A is posilivedsbivte, Hen $()
Proof the pref a thre Cervollary Blleee Pee Dheavem4 since,
im (fr diol ic the Hessiam matrix
Aié positive Aemicebinite
is dbrichly eamven.
Eacaale a7 delerctine heer the fotoaringy fumalions ave Conver ov Concave
A) $i 8.79) = Meaty 24
SW) $2) =x?
Lei) $e, %) SB
ana hy 421% ~671— HM
Me ped t6x gy 44% - 3H 2 TIT
6 2 -2
SOution (i) Hesstam metric H (1%. 73) = (2 4 :)
ab 2
wow, Ihe leading principal
fy = 161 =6 >
Hy = [6 -2 -2[=16 >0
ve HG %.%) 1 © poditivedabimite matrix. Shur Fls.%,3)
stricHy convex fumeHon in R?
fl) =-32>
Hoo =E'6*)
{H| =|-16l=46 0
woe [pg|stye
wre s ¢ ‘| sild>0
we Fle a4 3a) 18 Abrvetly convex function Im RP
TOURUUUUUUUUUT CUT CULE LLL e ee)’ (4
FNM YU YVUYYYVEVEUELUEUVLELUUVUUEOUUOECCOOHECE EE
Tn tRe previous Section, We dercribe Several minimization poset
imization ,
Hakure anly functional evaluations duvring He Gourde SL oF t
We nem dideusy Same meleds Hat wae Verivativen in determining
Me Search directions
he gemewal uncansirained minimization prdblem is ;
RoblemQUMP) Minimize (2) fr zer
Let ws comider Some campulational melfiods for Sdlving Problemd - By-
the compubational meHiod We mean a procedure for obbalning a Aeqpemce_
ane Me, £3, ple a ASlubioll Prcblems ,
Aucceanive approximations fe
it ae ed by Re reeurence velabion
Hie. approximation being gem
tenting, Foes wn Wilial-opprosienatten Lys Tor Woe erent potrh Zs
Gn is Called Re dircetion Jj. Wnear search ond CR (real number)
is the step lenstt
Definition (Order Sf Convergence) Lek He Seqomee ef Convengen to a petnt—
veg” and ef XE for all Sipticentiy lnge Kk» Uhe quembity
zi | is Colled the evror & ths Ktt approximation % - Suppose
oe
(2a 7 2 tah number pond vito Sushil
lim Weal
Pee Tae
Them fi i called He arden S convergence fehk
@ if Bel omd 7-1 The vohe Sf Conergence if inca
a) ig B=2, the vale Convergence i& qpochratic;s omd if pst
the vale df Convergence is Auperlinear ,
Definition (Descent or Stoble meted. Any meltod, ndt neceanarily Herative.
Defiitin (> generates gerne ealy the Sequence {xx} Sb approximations
Fok
(21) > $(22) > PBI De
is Colled a dlercemt (or Stable) method Jor Probleml.
We Ahall now clidcwss Same meltods for Sélving Problems (ume) EPEAT TUT U TUT U TUT TTT TITT ITT
Sts
P30
Steepest Descent Melhod ;
‘and iteratively move tovard&
In IRis melfod, we Stark from am initial point 2
the optianum poinb according to the rule
BPE)
tutere af id te optimal Alep length caleng He
fo be evaluated by Wie Forenulo
in (te ~ re THER)» Eat Pe
>o
Ba - 9k ve(%)) = i . on
peat clea for —9C2) dives he
Je that in Steepest descent "m Med the vec!
anes pon oe which teaser jal vale Gh deovenne of $id greatest previded
go(%) fe
Feanch diveebon de = -B4C)
2 oa
Example 15°23 Minimize Ce) = my tote tm Te 1 xe Gumek
lanting frem 2 = (20)
ng fran tee,
ssn vo=()=(Qm aon
=- a=)
terctins ve(2=(4) s ea Bed the ephinal atep
To find = 2M BH inimize
lengtR fang die we ee ae
2 gem M) =
te sal ee af ot, He optimal AF
oA
4) = cy) 2G)
co yield >=!
length along 3
Lae A" ig nate optirnm
= ho 0), x is nik
wele) = é)} #(2)
eration vgs wa He) eB +,
: a 7 2
- a aa
dr) 2 (vd ayrl= 8%
Now gare”) i eT ee
db cg 7S? DF lark
an ay Nae s 4
z jemgth along sh)
noted Coptionwn leg man
Bees) Ce
gets) = (53) 7G)» ® ie sar opifete Hewation+ Uhié process han fo be ealinued
2) (tere T4C2") 20,0) ) 6 found
we hove to proceed Ip Ihe mex
wall Iie optimumpoink x" = (
ae
Example 152:4% Minimize f(z) = (iat (a-any seem) ER
; c
Alarling with the poink (0, 3%
< Steepeot-deocem—
SSwhiom We <élve this problem using ee ae oie get
Bhanling wiih i = (3) se anny tions, Re poink Ze =(226 M5)"
im the fal te. Abler eve ie Aral recs Oi
Eo ee gata = 2.01 4 oa he pe
alee ta ‘Bhoton. tn Re figure. below iite:that- the
for (hid problem iA (Em
Bp] Me | eee
Wy |aed [(are,
Tevet] on [yb (an)] al
1 [eer 3) (era, 24)/C4,
“si
2 |erorispfers tall 673, 128) 0-24 [la.sa, 12
3 lesz,reCa9-
+20, °48)}0-N |G-ta, 025)
4 |e43,125)i8,-28)) (18 ,~"28)] 0° 31 (2-37, 1-46)
5 l(a97,1)¢a -29] (20, -2¢)] 0-12 [a.33, 113)}
6 |@33,119)0a,- 12} C708, -*12)] 0-36 l¢r-30, 1-14)
7 aerapas-aay C15, "08/0" 13 (2-28, 115)"
@ (2-28, 1S}(05,-08) Cos, ~108)|
Table: Corpputians far Seepent Descent Meelhod « Figure : Illustration of Steepeat descent meliiod
tthe melfed of Steepeat deacent anay appear fo be le beat unconstrained
aninimization tech nie Since och one ‘dimemsional search Atarts inthe beak—
divection- However, faa. mead 16 ndt really ebbeclive in moat J He problem
due fo the fect that the eC Scent itn iva wee heey and
the Gavergence id very Slow
Summ He Steepest Descemt gorithm
the Steepent descemb algeritfim proceeds by perborming.
Gvien « print %
iver Point oe ire gation ~U(E): A Summeny Sf Ia method
@ line Seach along he
ib given below
Triftalizotion Slop Let €>0 be He leiminatian Acalan. chooks a Sterbiy
poink xr, Ter ko! ‘amd go to Ihe. main Alep
4 II vplzal]o
k by kt! and repeat Jha main Alep P-33
E ; ,
sample 15:25 Minimize cht) = 31-40% 42% rkAyrs Starling from
% = (o0)" ER?
SSlution he Punction 14 a positive ce Pinte. padvotic funalien.
vee) = eae
= na 448
“She resulke we obtain Sir Hs feat Fas Herabions one hewn im
tho fellewing table
“rx [a ls
T) G0) Jeaoy 1%
G40) |¢o,-%) | %
3 }C4,-%) |84, 0)
4 |64,-%) |(o, —la)
S |@lol9,-'la) |Cie/a, 0)
6
7
3421, el) (0, ~Z)
25/7, -24fr) (-22 , 0)
SNR
Fable: Computation of Steepeot Dercenb-mettiod .
eanily Seen thal the globel minimum 5} is 2, -2)"
Discussion: Ot i,
Wi f= 2. Nébe a Alew convergence 4 Iht_opproximnatians
fo He amimimwe point C22)” Jhis i beraune the Aearch clvechms
im the Ateepesk dencent melfad ave orlfogenal oud in owr exeanple
Hene divections ave He two Govcinale axea: his partialay example
tells wr (Rot Steepent dercemh melfied doer nd possess He
qpasratic termination a ~—=
the ok clencem real
the method d Newkon is ceeuuve Ih
nee tr a pce al defleeks tRe Stee
by prommulliplying ik by To fowde 4 The Heaoian Tce hid Toeratian ia mébivabed
by Findingia Ysuitable direction for We qpadvatie approxiznatin Ie He fumehien.
yalhur Ham by finding a linear opproxi mation to Re Han «To métivale fia
procedure, Cemidew the following approxiznation drat a given point 2x
a
Qe) = denyalyeen) & -x) +4 (2-2) Hlex) 2+)
fare H(&x) 6 Re Heaslam amatrix endihien_
: S ninntynum dL. Ha. qpacratic OFF
nclutiely yeb(#x) + Yee) (£24) = 2." sw
exists, he success ov Por”
Zee
She above eqpation giver the veewrsive form [Ro points gene vated by—
Newhon'é relfiod Sr Hee mulbidimensianal Sect feobering tak TP
tak H(x*)is poaitive definite aka lecal minimum xt amd Hal is
eS tiusuoly biice-cif ferembiable, i follews Hak H (ax) I positive aebinite
al- points Llose to at ands hemce, the uecennor poink Zee! iS Well defined.
Wealso Comment hure lek, (05-2.2 b) Com be Viewed an am applicahion Q-
Ve Weaken —Raphsen melfied fo He ZSluhion df Mi & tem SF equahan
y4(%) =2 - Given a pell-dateymvined system & °
Zach iterakion Sf this ametiod adopts a” bivst
ap imation to this bystem ab He current
veoulking. \rneare ae to
hin bo Sige apiok V2") = 9 ak am Mera!
approximakion fe ve( x) is given by ve
eis cape! fo Eero oud sawing, producer
given by (15-2 2b) "d
4x2 Slorting from
Example 15'2-6 VMinimize (4) = wy nmy tam 2%
4 = (0,0)
‘SSlukion gated ~ (sean 208) H(z) . *)-G5)
py 42%
. vl Agen _
Moralins —ve(e")= (*) ju] = Soman” ( ’)
he [ned 4)
=(2)- GG) vvuvuvve
vvvuvuews
vu
vuvvye
nas
Te see UhelRew av ndk aa is ophmun paint; We evaluale yo(e)= (3)
wae ge (a) ia the optimum point. Note thal, Necokon melod
need only ame Mevabien A} cath the optimality for « Paratic function
Important Reoult— Newton Method Sind the minimum of a qpodratic fumctian
in one iWeration only.
Prof [ck He qmadratic function be given by
le) = oF TE +4 0Az tor
alt Z€R"
now vite) = +42
Far minirmum of V(E), YE) =9
pb btAz
> x
Hessian matrix A SY is nen-Singulow,
‘From Newhom melked prenemed corliev,
= 2 -[heoy'v4@)
an Al (AK +b)
ew aan ate
= -Atb
e Capelucion Sf He
Again
_... @sa2d)
yeu immediately Fille,
Fram (15-2:2¢) amd (5-724) th
Abrtey Tess Conergence of the melfed of Newton
Tink generated by We, metfiod Meiten may net Grverge
6 ote HiCiK) may be ete a Teak eat iS ndt—
fF [a@ey exists, p(xet) is ndb mecca
the Atarting point 16 clese enough I a
Fink Ut auch Hal GCS = 2 amd HEY) id Sfp full vomk , them
{fe metRod df Newton ie Welldebined ovnd cenvengen zt. jhis is
ved in the follow: These He clescemt Funchan 2 (4 given
ty a(z)ole-zil
In general the p
the veonan fer Hid
Hell defined. Ever i
lessttam (2K). However, if VYBBVUBLELELELLLEVULYUEUDUVUBUEUBYUVUEYYUY ww
P36
Shewrant Lek cf: BD pR” be conti nuoualy rice differentiable Coviden
eaten'S algori tf defined by Be map M(x) = ¥-[He))v(z)-
Lek xt be such IKak vet) =9 ond [HCH exishs. Let fe
Atanting point 21 be pukbiciently close to 2% Auch Hak Hiame existe
ky, ky Po WIR Kite zi Zt
ag i eZee
giz-zl
function omd Convergence
ie re te€X, the new itowabe Y=2es1
fakiabies | Zev ~2 Ils Bie Mae ="?
at Wehave, atleaot: om ordev-two vale
ig adescent
lehich Lhows thot (2)
te xf, Moreover, Fr any
frodueed by tha algorid
‘from obove’, Since {ref =e
4, Convergence.
dee?
Example 15-2" Minimize the unchan due tp He Hs Rosen brock
F(Z) = loom xp)? Cr
SAucion, Bh endear (a 2h) 20) 5 Bh = s00(s-27)
a. s ay
Bh. geom, » BA = 200, TH = ~sonler ai )42
a7, 2 “aur 2 exer 4 G34) Toke °
[Heo]
. m=
Now 2
Wi(2)
eh
ceckuve tewmi
in feck, a global
Example 18:2"
sSlabion
in te Table
Ager Aix itera
pornt- Hl ubCér) Hl
9 by the
deereane?
core, Bo
He Kave, alvead
we alert
1
weak
q
Hc) Ze) =(re))
(2) = (40, -200)
Connider the fellewing
She Summary SF
below. At
In the example,
eoch-
Eamndt be. wed a2
my
°
(
acess cna, HOD
200 0
ol og 2)’
tere (eo
‘200 400
(
— Hie VG) = 6
makes al the optim
Leninimuem of f.
a
= Boqes (Ae 1202
+
Li Since
problem:
Minimize (2) = 1-2) +C
tha Computations wing
each iteration, K+
Hone, He porn
20-04 aud Hie fp
amalhad ase he
qe observe tral
Me ation. However,
the, value
see. Sak Newton! A
ae point close enous!
J
van point 3 = Ch
Newton
Ky = (1-83, 0-91)
roceduve 16 terminal
ram. in figure below
this will
o. clenceml-fumebian «
eeeliod mdaeel Comey
ho to le optiznal. point
°
)
deo
~4o0
gen
r
widen (2) = (41,0)
)
) tage) = Go
y
y(as) = (0,0), Be
2
2%)
Zk -
is veosh
fed. the points
function
He objective
be ie
a gemerall
ian. In Sheever 7
povided
1) oF $2), Wek
1g method is given
TaCan)! wee)
ed, Ab hia
+
Itevalion]
dio] xe
vem) | AG)
| (H(zx]T
1 10, 3)
2 | (067, 0-38)
3 |G, 0-56)
(vt, 0-70)
(1-61, 0-80)
lina,
Ca, 2a) | (752
0 Ca
\(-9-39,-0-04) eG 23
4-0
(2°34, -0-04)
i)
~40
aro
sea (F
jt
toae4
50
~4.0
618
40
(022, -004)} (3-89
40
oo}, v0) (40
“4-0
L-o-B0, -0°04) 1 (3
(23,05)
(os, -94)
8:
fee 23-23
a fee 40
7% (4-0 150
0 4:0)
E)
0 40
Coat, 0-22) | (vu,
(0-20, 0-14)
(0-20, on2)}
(ons, 007)
VUovVVG
Vv
vuuvuVvUuVUvUUVYvYUeVvUuUuUNYVUY
veuvuvruw~
Sable :
Computions by Nedkon?s Melhod vey
vurv
vuyuvuvvuvVvvvVuve
vvwee
P38
°
a By 3 25 7 3
Figure Ilustration of Newton's ™elfiod.
NaleMope smelted 8 Newton ja mst very useful, in, prostics due Fo He
fellowsing readin Cao obvious frm eet 8)
vetoving of tRe Hessian atria comrute
& Jt veppiver Me 4 ;
(u) Qt becamen Very diffiaulk oud fame times tmpess
The elements a thie Heavlem ‘matrix H
Gi), GE repre the inversion Sf H in each ikeration
Ww) Ob veqpiver He evaluation oF Hie i every itewation
These features make Me imetfod impractical for he lem invdlvi
a complecared objective functien 06 he erge 0 27 provelected toleramcen,
Olfexwise, goto Step?, vepleee ke by Ktl amd vepeak He process,
Exaanple 15+ 2-' 2 2
emp 29Y imine $(z) = y- BF 2H HIB 4x2 by DFP
algorition.
10
oe Jake Zi 20,0) ond wens
SButia gate) (1295 0%
ay = oh, vate) (a8) A) Ct)
Ltevations —ga(z) = (21)
ep length Af" along Search divection i,
De find the minimizing
We compute :
(zit rdi)= pC, %) = An 2M yield af =
= mtn dis (3)#1(1) =(4)
we have te tent Wheliu 2 is optimum» Since
At IRIs Alage
v4(2) = (2!) (3) » 22 Is mA optimum,
‘) Pol
Lhevatin2
2
£4(2) - 2 (51)
= Gi) =e)
® hat ia) 2
wax dO) 460%)
wy (49 )C8) °C)
Tam = G20)(2) 24
yt
g = - ee. (48)
ere] aya (48)
ene aine, =(68) "Gs ayes
=[? ~%
-h ae
da = -H, ¥(®)
= 2)E) 0)
yh
ext bo find Hu aminimiaing Step lengik % along si a
(eth) a PCL Hm) = AHH He ROR
= ay 4:
eet) G2)
‘demti be be opt
Since vt(s) «(2), of =(7is) cam be tdemtified fe be optimum ,
g he Himmel blow funehion +
Bxomple 15-2-10 4 1
ea co iniraiee b(t) = (@ ab RmN) (FR 7)*
Subion amidigeursion he inverse Sf te Heaviam matyix ab Hie
anivimumn point (2, a)" 14 Sound to be H* (2!) =(%0% 9 80
Succenrive iterations of ihe DEP melfod tramiborm am inihal idenbity
matrix. imto He above matrix, Shis Auggest that DFP became von
Aisle fo the melRod Sf NecSten after a few ikeration. Yhe_
advambage Sf tRié Tmlfod over Newhm (A method 16 tol the Inverse
Hessian mabrix eed nét- be Golewlaked, Yhevetore only wit.
r6t order devivativen the algetitfim iA achieved Re Second-ardey
‘Beareh. 153 ConstTRAINED OPTIMIZATION _
Combvained non linear Pregromanlr aie lanl toi Hh Pha problem eee
ality amd Pieri Srokeaint,
Yn. objective function in ee fr seme e
GE all The jumetione ave aan, We 6 tas, ewa a (nean- + SRamwia
ap bah UE pciemt—od
subewA
fF
ihe hte, Ba mozWncar program » 9 Ji develeperneml
yobuat algorithms far linear [he adVemk highs ced
ua Ha eautcation Sf ae Coed pe erent Erente Te the adeentog
ng and amalys 14, hawe. masta
aud profitab gpl naitenbatiat ‘mo del
Wie = ettamt +00) for Aiving- Prulera tn diverse.
re preg an fry
field. onarer irony weabiatic problems, cam nd be
yented or appolimated aa a limes progzam cel mabure
ie “nenli if! Pear he cin i pot
Abwe sails non-| i cae pele
te bles
& the cority af
ototty hewe ‘made. for Ie civring ta past Hvee decaden.
his Jechion Op hid “segleer preent4 Hine ldevelapements Im & logtcat_
Jind Self-contained Ferm
Waeginition ne gemeral malumatical programy! problem 1 dabined
lem Sr deterenining Z =, ming. xn)" Se ao to
er minimize) the objestive funetron.
z-$() |
|
Babjyect Fe He Constraints
dae DY 2 ser
haem,
Wpre fF and Ji ore veal valued functions hz amd bi C=
hiZ problem i+ a Linear preyromaning. pole 1f §@)
ave Conatants «
Giherwise iis Called o
amd 9;(%) ove all linear.
optiezation poblem.
ABLE OPTIMIZATION WITH EQUALITY CONSTRAINTS
15-3-1W/M ULTI VAR.
Bhall be concerned wilh finding the extrema S F
We Al
subject to the eypality conobreutntS
by (2) 9, dnb
lature each hj 1 debined om the danatn Dek F-
4Lagramges metkod
the Lagramgian fumetion aanociated
f(z ays FG v4 Sa )
bofere 9g (aches corm) ewe Lostenne enlipliors
have the foitewing reoubts
ith this problem is
Now We ~ewweree
Ros
Sheorem ,
Theorem ™ Let £ amd hy (24° m) be the continuownly défferentiabl
le
furetions. $i Lyk is @ local anini 7
epee caer We ay toe
Aatiafre [Re in oe e paimk (21 2")
ve &(%-3) = vFlz)+ Su Lhe) =a nose
_ pCa ees ,
be th, nen lat ( HERE) i Baron
eqsal to amin(m,m) ay
7m o-- (53-1b)
Now, Buppese hak mem
tree lokin m £7 ?
in, For Convenience,
Proof She fheavem iA obviously
Then, He rank S He nem Jacobian motrin is
let fa Jacobiam BCh. ha. hn) nde Vamiak at ta local oninimum
=) A), johannine
BC By
xt, shié means Ba tie, vectors hl \
finearly independenh, amd fun form a bonis Lor RY. Heres, vFz)
ambimation df te vectors Why (*),
Com be &*P’ ed an a linear &
wipdy = hs Thebkié, tere eeieh Conskambs A%7 I> dn,
>
Zero Auch Ral y
“
gilt) = COSINE)
Defining Xt ger be my mts we See Hak (2% 2°)
aki shier (15-314) and (rath _ Hemee. the yenulk hélds fora
local minimum x Op the problem Since a local -maxtravm 5
§ th 4 local minimum ah -F Hee veoulk—ié true else fin
fecal anaximum 5f,the problean .
Note oft Each- fecal maximum omd fecal mini:
minimum we om
MeES ON dibiond to eppalion (12°3°1 =) ea an i hee eet
Mare global amaximunn amd amirimnuan macuniiees thel~
Hane avve finite. Further (13°3-1a) amt (13:3.1 8) ee
the Sbetionany pointe Oy Ihe Lagrangiom (x, 2), Sof
(1a.9-18) amd (laeB.1b) ave He quceon ary Ocoll
oP abjecl to hj(#)=2 Se eee Tr
extrema