S. No.
Date
Title
Page
No. Teacher
Sign 's
Remarks
Deep Leaa
Mid sem 35 - 40%
ad sem ’
35-40
Tiun apr 20- 30 Yo
Read Prnt
Brnp 3-4 shuduts
Larer 020than 2020
Sowces: JMLR, TeLR, IeML,
NarIps, TPAMI, CVPR pret)
Icev. Eccv
Clas code
Loss fmehiens
Dezp Layevu/Cmponunt
Applioations
elasifitaton
DCaAN
Auoecodes WqAN
+Agmentatien 4PT
Jnatiw Models J iugle Objcchwe Toske.
’ Gbject detechion -ReNN, FRCNN, Fastin RCNN, Yo LO V2-V3
’ Fe-shot Zoro-shot task Maltiple bjctwe Tasks
Los hmetion
Acolar Valusß
(1) 0-1 Loss
pedichßn
t: dabel
madl
qradiant ducant oill mot wnk
&calau Vaued
Evolationany Aanning olgorithms 4 Capmle met (3)
(sy gnhe alg can wok on
this pc Loss s. do mot
hag do grad. dycnt
. TToik a valid los but cawadt be wed for deup
they grad based ophinigos msdess,
() Sauaned Loss
=wx+b <-geod for nugnusion
blomieoon ?
Ni
db
Note
tet,
49 whn t=| bt
-139 whun t -I jquaned laKS.
* Mti- olas elawificatn
wSk
Dn't rk ; labels ane weed; o cene
dist on't
tran a deep model wsg a MSE Clasifr ?
Yes.; awswes wil mot mot be geod
qrad. ule wil
* Camt trown dep
deup Mmodel
wmedel wwg
wing o- Los grad. deseanr dosnit
() bgutie Loss/Ragrsin
6(z) = l+ez
Zwtxtb
%bettu thon squared loge.
(-2I tr bim elasfction
]
= (4-t)x *e
(*e
(y-). el
(1+e
y-t)x sOx
(9-t). ez
(1+e
uppee for tol,zs -5, callote2 when d 0. 0067
Z=-l0, y =4.5x105 Strmaole tum
traming. gong
(4-y U- ) = (0.oD6 ¥-1) *o.oo63s (- s.ocr)
=-6. G|| X1o
(4.5XI0- )x 4.SX10"((- 45xI0r)
’ radient gong loo (bod)
[due to d) ] Vaihng
* Satohn roolem assoiated ith ineanay radnt problam
"VGP trawng Atops
Sat prob ’ trawng collapsu
x Relu to tackle VGP .
() MAE LoyMean tosdwte Ernor)
model
t : labl
N N: Mo. o Aamplee m batek
* man- ditountiale at
homdle wmg Codaing eneuption
toke qradunt when mo o ; ele skip
penfect matele)
MAE nr towing
tctor-H( beten than Jquanedl wnan)
metwork. reeonotuchion or amg geatue
# As oSS ’0, trawng ons dawn n cose
case of
ike MAE . squaned berj
(5) Hubu Loss
S:thrshstd
Ls(1--s)
Squand
* MAE for te low Vales : trouing slews don for st.
tovtiwow ?
-iite ciseontinites ?
witiaizotion -Xawer
netwvrks traumng vilile mitialization
np ás #
stop wiM tring
Prstslem' yradint trgag
toke
2add bateh,
Aample] afr *
&ealar
label t:
:model
/p
Bunany
CE ()
alasi(muti-
)
categorical crosbmoy
entry
Loss Cres (6)
Lipaclity Ls(rl-)
Mod
Lipsehit, met isLogs Hubu #
L.finite intervad,
no isthr frud Himupuwhive
hu ’
No Liptehit,?
stabilhq
Lla-| ()Il t()- || that Such
costowntL a sthoruLipsebhit,, f:R"’R"
L- tuwetin A
Lipselitg
Tre-franed
Model 2 CModel I weight purious
trained model
Task 2
Trowatn
Leanmng
Task"
tram
the
p r e y
omodel 2
KacE -Hog y-(-t)lag(-)
z= wx+ b igned acthvstien
|+e-z
(-t
() Muti elass clastieation
(Catagrcsl ntrop)
We hawe ndieatur wer one-hot veetor
t= (0,0,..., 1,.o,o)
k-th ntny is l ift elass daal s k
K mo. elasss
(Kxi) Wx +b ’mati (Kxd)
Vetor
(ar)
Pv
Aehivotin
sftman (quunalizcd sigmnid)
Mmtivaiate 2
mtti valud]
Preve that the case
Sottman a geunalied
Itez L= wt+b
# chek
* The o/p otmoa is a taf
’ ony 1 component oill
k= Conti bute to the Loss
t’ one- hot Veetr
X
elinment-uise opunation
res ntrpy distribution a neate to distibuon ow a
given ut s definedA as
H(p.) - pl) log (
pheyricat met wnbomded
enhopy has mo mohon of Atk's like dhst b/w
oistibtHons
ions beter for elasifeotion
Oe actvay eloer
elar in the feane onse
di - log (a) rom itth clay
K#i
k'
e e3
k=i
-
e2e .e
kzi
1
(o e
For k=L,
,Zi
+ heok
For kz i ,
Cowergne
Goung l stop vohen derivativee
’desihed (because ne- ho enerding )
dacs (o.o1, ) =- Jn (o.o1) = H.605 we mwve
Laee (0.o6 ool, 1) > -ln (o.oD oo) = ll.s13
:closz
good Coss
beter;
to actual lael
4Reqrmon Sauaned loss 2 AE Joes.
- E Loss
MMli- lalbed olaui fiotion
con hawe mtiple Aalbels
(medicad inmage elanifieatin, designing deup madel ter
pmf cEdesigndotion)
mll mot wrk
) Bnany Relevamce
Lalbel Poer det
Trect al povnble vaiants as
all those
one elas: amd fran
elaes
2 Oo ol6
K lo o 00
*froblam - No. oy elases wnereats Uponutially
9) Swm BCE
BCE k
Mode 3 k
BCEx
+ Loss wil always be sealar valued
* S3IM loss ( stuctwal nilay Tnden Meoune)
2004 TIP. Wae et. al.
3 Key termns ’ wminanee : (meau) more vaame,
Covtrst :(vanone) more contrast
2signal stuetanes :(eross eovanioucn)
Siqwal Luwminamc
measu
uminamce 3)
S 2
Comanin
Contrat
measwe
Contra
Companison Combine
Shucine
Companin
*Relounee Mehie
mud a te gromd tnth to Comþane the nesnt
# 2 2 sgnal. geuuated by metwk
romd trtn
superviud et
GAN Aettio teni atinA doun't med walowee metie ; udo labe
*global
N ’ total mo. ot tiels
Vaiance’ wmoiased estimatr ot Varianc
famçte
Caloulate
# Lwmnamce Compariloni
+ Ci
s(xY)
N
whee
# SSIM
snal ehoiee
Man. vame of SSIM = 1 + check
" $SIM s bownded
tomdand
Vaues
(K2)
* Hiqt value
val o ssIM all eome wp whn the 2 ignals
hmilan
oun.
tuth growd ieG
Pg 2coefiint "Die
Vome
any’predcted P
þe = D
elose, how N measwus
metic grod
dirurtiabte) not opunations
ore et (:
pnloes as uied dineoty cam't
be
0. Min.
Max.
candinolity -|I0I
Union Iutrsection IoU=
Mean lace
5SIM measw boeks wto
qlabal
meoswe
isSS]M
SsIM moninize st, mn
Lo4s the into dineetiy SSIM se
camot
loss
ssM) a(1- +
uelant iwage Delhagomng *
Gombined)
ps mmboundi
las d bowrded+ "Scal(ing
SSIM 4SE
Mac
G[mam
aligrud)
M. vae
[min ownlap
(" D hoas to be manimiged so
Less can't have D direety)
-2 z8)-g(24)
(*)
* Tvesky Loss
Salei et al. IWML, 2017
Haward Medical cho
+Tonay Celficient / Dndua
|xn|
s(x1) =
spcial Case
T(B) N
L=1
’3-D voma
o Vozel i to be deion
wNon lesion (omplemnt Po)
for unin -volume
Lo 6uise
l for won-lewion volume
Mox valuu of T dapende onehaiee o t f =|
* Chaen Ditane
"ehoee he manet
GT towt to
caleilat lee.
wim
der (s, s) - Xes,Xgesp -4 n
tobe
mùnimized X’ pred drppgthe.ud tom wl
Y ’6T ost
* KL Diwgune
9()
KL- Div. rP
wrt
+ nelatin bw
- log s(). KL div, L
CE
=Hlp.4) - H)
Rroe r dus prove that D(Pl 4) o
Sanaun's Tnaqality
fumchon,
a(t())
P(*)
to mate t
Cowe P(E0)
q(E)
P(E())
Zr) Qa)
X PC9
(E)
- las 1. E)) E()=1
logP)
g().
Dup Comtonanta
- Fllhy Cowectad laues (Fc) [NN pont
[NN pant nn PRMI
-Activations/ nNan- ininitg
- Conwodtion Layes (Flatening) " Tensor
- Poo ling lager mti dimensibnal
motrix.
Racwnent Layo
Residuat ayo
- lerti¡n Xager
Back prep
Dwivaiale/wuti vaniate nchions ith scala ovtpt (hwngle
valued fumchii
Can be Scalarvector
0 Lieor Aetivotion
a) e.x
Wse
ith tve -e vaes
Sigmod Achivation
)= |+e
or binany clawifcation
tanh funchion
() = e voishing grndirt
prdolem
Lstsse
|+e-2a
() (+e)(2) - (I-e")2e)
(te
2e- (1 te +|-e)
(ite)
4e-2a
(1+ey
’ ( ) = |- 44)
Max. valwe oy (2) = |
Min. vale ot () -I
ARuchfied Linan Dnit (Ralu) to miti¡ate the vomishing
gradibt prtdam
detoult choie for hidden layer acivotien
)-maa (0, )
* hoween, mot differewrtiable
. ut side s L- Lipschi
" Hve hde
voihing gradint doroblem
-e yide stil has avamihing grndiint protlem
* Relu wvdues Aponialy to the metwrk
s(w*+)
wXtb ’-Ve. RelU ill Ave o
"praing
* Dyng Relu problerm furthen, oconbibution
it mihoizotion is bad, if you hawe wìll mo ^tßeet
nate
igh
owtx+ b)
-ve bios
l node
nod will go to at vey eany stage of
Laky RelU
tra~iing
('wnat hronabl)
a : 2 for ao akage barauatie
typieally, walue t a is vey small
"L- Lipsitg
wn-dylnertiole ’ codingeception
6 Panametie RelU (PRelu)) rodes but uult
hawere, no. o
for o trauinatte panametou
da for x<o; ao
tainable
Bottphs Activation
Swooth approx? Relu
ReLw for hiah vaue of B
220’ l+e
ac0’l+eIeo
8) Erpontal linean Unit (Eu)
L’ Conctimows tor a=|
1
mn. Vyalue =-1
Condimonsy itnentinde ELU (CELU
graduint in smal,
enplosion vau mot happen
