‘Bayesian Muthvaviate Regression
Ix Explain the concept + of Bayesian © Mulivervice.
Regression model . :
ee Multivariate Regression, renodels
esian, muliveriode weavessiow vnodel"!
ae teal approach hot! models thie! folate
between smultiple dependent | ‘(xesponse) Variables
and one ov, move independent (predictor) vartabtes,
usin Bayesion im inference Unlike. ‘classical wnethods ,
aa incorporates priov dist butions for paramelercs
and uses postextov dishtibublons fox Inference,
Allowing’ tor the’ incorporation of prio knowledge.
and uncevlainly, quontification .
Given a set of independent and. enol ishibuted
veetor- valued observations 9G deb Br p74 OP
possibly covveloted: vondorn, variables , the mpultivariate,
regression ‘enodel on the variable ui is
a : 2 GN
(8/04) =, 8 2 Ob + ‘ef
(Px4) [Px] Geo] [Px]
Uhive! the amabiie of “ieression "doe in venls, ‘6 is
given! by, Ba y
>
which deseribes the velationship between
| Ob aC diy ~, Ug)! and, 2G while, @ is i
P dimensional - Neely dishibuted error.
Note ahaty | ip the.” “coelfictemt snahie B were_
. writen we a Ge Gx) and Bui =Gh uy 1 then
(os eit ut) SME Beh tut ob Eben
xyes CBD Pxqs Camper Ca:
which 1s the. same basic model 8s iy Souvee_
Separation..: birg “heabersagl
More. specifoghy sor ae “egredsion ‘rhode, a
given element os x say the oth Jk
Colby) = wg,
us Drea+2) fered] Can.
where B = Cpe. Sree, Big) + TThis £8 also; vrepresented_
"os Y
6 1a. = 2 Aye Une * Fj all observed. vectors injo.a matnix, the .
Giothervin
“Eqression smodel may be written as:
(|v) =_¥ i
Cov) ca J} Ceaeoxel . (aR)
where dhe, mali. of ‘uepeld emt variables x, dhe.
independent variables UY, and. the madrin. of errors
“Bp.
“ThE yh element of X is the. th yow of O snultiplied
by the jth column of B plus the yh element of E.
the model 72 algo be wvitlen in Jevms of colerenns
by panemnelevizing the dependent variables (observed
amined sources) x, the. independent variables
(observable source) modi U, the smattric. of of regression
coefficients B, and -the smooth of errors |F as
Luz = (fy 4 » Xp) i
v = Gate w-Uq)
B= (Pr, Pos -- Pp)
Es Ce, oo Ep) Where. en 15 an n-dimensional. column iveetor
of ones. °
| This leads +o +the ‘model
C516u) a Uv ey , at aero)
(nx) [axan] (ero) fon)
Which desevibes’ all the. obsevvahons Sor a Stole
microphone in the cocktail Porky ‘problern ot- aly
m Hime, points . . a" 4
*Conslruct the likelihood funchon for Bayesian
muttivartate. megrression med el with, copjunade,
priovs ond. postevioy. Diceuss a method of
Parameters of this ‘model’ LO
[=> Likelihood. funetion:- Let (avi) be pairs are
observed variables, i= Urn, where, xj is a
P-dimensional vector, and’ ur is “a G1 dimen sional
Veetoy ‘cortaintag 1 and q, evvable_ u's )
Un Uigy Ug the. errors of observahbns are.
independent amd “normally distributed vendor
vectors with p-dimensional wnéan o ond pxp
Covariance ~madrin the roulHvari ater p-dimensional normal dishi bution
for the enrors 1 Loon
wes cif ee u : ‘
PEI =) [=| & Helse; ligt
Wheve, €} 16 the. p-dimensionol ‘evvor vector. 1+ Is
Common. in Bayesian: stalistes. to. ornit the,
normalization constant and use. proportionality .
| From hig “multivariate: novinal” ero “Specification. ,
Hthe dishibution of the observation vectors is also
multivariate. normal dighibuded. and given by
P04 18, wi =) Isl Seow = (s = vi).
because “the. Jacobian of The “hongfov mn a: fon from
eG; to 1G is re j
With the. amochine vrepresemtation of the model by
she disvibution, of the Matix of observations is a
male normal dishibuh'on | as f ,
: , ing, ity (x08) =! (x-ue')’,
pele, z.v)
Cony, ugate estimation and inference: \
“Two different ‘methods,ave used +o estimate.
parameters and draw inferences. The. methods of-
estimation are rneaginal poslevioy mean and_
maximum oa posterfor! eshmates |,
ow Mosqinal postenis¥ mean Cmarginalizedion)
GD Maximum a posteriori estimates W Masginatization’ Maxgin al posterfow mean ~*
estimalton ' of pairarneters involves computin
the marginal posterior distribuchon sor each
the parometer cond. then computing mean eshmedes
From these Tnaxgin al posterior dishibuclions .
[ Me manxginal postertor dishibution, for the. maddy
of vegression coefficients g alter integrating wih,
*respect +o = 1s give by ‘ .
P(BIX, Ud 4
let (8-8) (D'+U'v)(a-B) | meee tga)
Where. the molt,
B= i400!) (ot
and By is: the prior mean. while. the (px py med,
tA has been written as
Ga Qtx’x +8,.06~('us 8,5). 0 wy C405)
has been ‘defined. :
“Tre mean and model estimode of the matrix of
vegression coefficients trom 4hys exact ™aginal
posterivy distibution_is @. The marginal postevioy variance of the matrix of
regression coeffict ents is :
vor (B18, X)U) = eed Ae feemea +1) cous
ov equivalent
equivalently, ey ,
van (0 | 5% x,y) 2 nevrPoh bal [en-a-n(o7 ea
NEV
Generalized. priors and postevioy-The. _ fold prior
dishibution (2,5) Jos the parameters B=vec(g)
and =, vee! being the veclovizedion operator that
slacks the columns of its makhe cagument, ts
[given by the product F-the prior dishibutton_
P(B) Jov the regression cnelfictents and that dor
the error covariance ‘mah rep (z),
© PCA) =P(P) P(x) ——-O - -
TRIS prtor dishtbutions ave found from the,
generalized conjugate procedure, given by
P(A) fap ee 4 (A -A.) G'(B-A,)
and p(s) = jxptebtreg
where, the quomtities A,2.,v ond @ ave byper-
Parameters 4o be assessed. the mmadyices A=
and @ are all positive deftnite , By Bay es vule, the. join} posterior dishibutton jor
the anda nodél parameters with specified
generalized corte. priots agin ete
P(A =x, u) & PCz) PP p(x]@ xu)
Which becomes
P(B=zIx,Y) os [af ECPI (A-B) epee
| o's =Cex-v0)/(x-v8') + 4]
After inserting the generalized. conjugate. prion ‘
dishibutions and. the Itkelihood. «
t
3 Pancmeler estimation = \
te estimate parameters as ing, generalized: priovs
and. postertors, there are. two methods for
estimatteg | parameters . may are.
CD Mag singlization ¥d ; bid
W Posterior eonditionals .
We want--to explain ‘posterior conditionals’
estimation procedure . The! posterior conditional dishibubon of and.
= ore: needed for the. algorithm the posterior
Gonditional dishi bution -of Bois found; iby enpsidern
only -those terms in _ the joint postesioy,
dishibutton Which’ involve. 2 and is Sven by
Pls, xu). P(A) P12, VW)
2 'e(0-BY Calta) (PB)
TTRis is is weoggnizable OS A Multivartote Norma
dightbuliots for the vector of “egressfon Coefficient
f& whose mean and mode: ws
B= [al +0u@ =) '[hlarlvves aad)
where the veclor fi is
A= eo ever)”
Which wos found by compiling the aquerre. In
the exponent; The posterior conditional dishibulion of the ertoy
covariance mahix =. is similayly Sound. by considering
only. those. +erms tn the. Joint posterior dishibuHon—
which Involve = and is_given by
PCEIP-XV) 0 PCE) P(K1B, ZX)
|x) O) eh E [ove VOr-ve) +4]
This. ts alveodsg creoggnized as betag os inverted
with wishart disttbuton with mode
ek ~uB') (x-v8/) +9
atv