unitb-4
RecuYYent neuroal netwotk foy NLP
tuhat is RNNCecurent neural nebwork)?
"one kind of arbificial neurol network that makes use
of seqyential of time seIies
data is the YecuYrent neuiol network.
why RNN Came to existence
conentional neural networks ae plert where the inputs a Outputs
They aie many
ate independe nt of one aother. guess the following wotd in a serntera
it is necesay to
- However, in situations when is necessaxy to retain the peccding
necessavy, hence ib
the pieceding uoids are
woids
one Of the Yeasons why RNN Came into Pictuie, and it USeS a
ThUs,this is the
hidden layer to tockle this problem.
states ate called as memoty states.
These bidden
Oxt
unfolding of R
fasic RNN Unit
HOw it workS ? traves in a bop to the middle hidelen layer.
neual networks, data tate:
In
YecUYYent thefomula for calculating the cunent
TA where, state
cUY Yent state hien
A he
hidden state
hi ’ previous state
t’input state cUTYent
furction (tanh)
8 B bimula for applying ativation &)
wah
ht tanh (wmhe t
(Ct-i)
of RNW
Bosic orchitectwte where, yecuTient reUYon
whh ’ ueiqht at
input neuton
Cuyh > weight at
brmua for calcula ting atput :
the
loyer,bhe lnput layey x xceives
slep 1: Before serxirg be data to the middle twotk.
and proceSses the input frorn the neural ne
veval hidden loyers ,eoch with uiqve octiuten
h has se
Ster 2:te ntemediatc byer previous loyar has ro effect on the vaioUS
Tf the
functions, weights, and biases. layers,ic the reutal network has no
hidden
fameters of lhe Subsequent
can Use a current ncural retwotk.
memoy, then yoU variouS acti vatricn funchons
standardize the
netwotk wiil
tee3 ihe ecurvent neuial hiclclen layer has the sAme propibi:s.
every
ucights,and bases,guanteeing bhat buijd just ore and yespeat the la
hiclden laons, ib will
nstead of builcling multipk
yequiYec.
oveY it as mary times as
RNN :
offevence bebween feed forwatd and RecuYYernt neula netwok
network
Feed fotwavd neLwal
a furelormentoal
processing
aRNis having known recuTYent unit or
artficial neural netwoks Unit which is
as
-fFENN ar yecurrent neuon.
without locping nodes.
rnaintain
as mutti-loyer NN b tyansfens the vnit has the unique abil.by to
"known
infot mation unidirectionally. a hidden state, allowing the netuok to
capture sequentiaj dependencies
sequential data zt renembers bhe previous inputs
.les uel as
not there as they cant SorTe of the mproved eYsjors of ANN
Prcessing is
sermgmber pasb irputs * LstM(long shorb --teim memory)
* opplications * GRU Cgated yecurent unib).
** imoge classification
Ansics f Recosenb neul netuorks:
s sjuentol leaiing
. &e fhe t tarn AN ue have to look into hat
si2e uhich moy not
. n the fesd fowavd a CNN the input size is of flxed
haen n
aljec
tive
noun.
noun werb artick
uhat is seqerbial learning )!:
of
"zn moy oications the inputs ae not
the sne size.
independent
*suCCessi ve inputs are aso nob
of each cther
(SMS)
" for exomple : Ato completion tasks Socin
animal
A'and you
*SUPPos you ue givern a character Man
senterce formoton
hae to prcdicb the next chaiacte.
t hagpens when we go for the auto compkte
Serrtence foimati on as weil.
Hwto mdei thbis ?
"one of the method is
thiough RVN y[t-2) Ct-)
one fixed astivation function unt
"
an inbernal
" Every nib possesses
hiclden state.
State known as
given time step, this hidden state 8
" At a information that KCt-)
tepresenbs tte prior
SSes
the netuoik nOw possE Bosc RUN architectje.
output state is updated at ench
the modifications
Ge step to veflect any
to the rebwoks histoty.
mathematicat fotnula foyRNN:
Basic
step there s a fnction.
o each tine netuork.
Let us dicw a sirg e
. Yepiesenbtion
sorme function which can
"ue need ..Basic mathemotical omua
the output function.
ocYOss for one singie cell Of RNN
which ae shodpararmeters
W,U,V, c,b h= sluX+ Wh +t)
he tine steps
Y= olvh +c)
hence,
add a ecurrent connecbion in the nehwotk
of muibiple seclions
S; = (Ux + Wsj- +b)
Y :o(Vs; +c)
9; f(*i,Si, W,U,v)
statei
state of ne buotk at time
"S isthe
netuok ard netwoik pajametes
the same Jioo
computeY yi,y1,ioo1 CcUYternt connection,
an be uSed bo
piopaqotton throvgh
- RNN 's using BeTT CBack fme)
TYaining RNN : a single -time step. determine its
network in
given to the stote,
step : the input is cuent inpub and the pievioUs
2: y utilizinga set of
Step
Present state. willbe ht
stont the
cUIYent bt
the clata from every proY
Step3: At this mert,one can combne
veguite
Dependingon the time steps
as neces SaTy
step 4:
tyavel back as many
ient state aftter all bime
state and cut
computed using the ultimate
output is
step 5: The been
completed.
desived output and find out the erior
steps have wth the
outpub got tion and that is why
compore the updated with the bockprqpogal
step 6
aYe get RNN ovey bime.
Step: the weihts the
key to tain
bockpropogabion isthe
RNN.
see how fo train
Let us now
the forword and the backword stoges.4
ae
tao stages of an RVN colaborates.
everyone
the netwovk's training process,
thioughout vay,though.
calauloting methods
struc tores and
LstA Clong short term mernory) :
" simle A facesa short tem mernory
poblem
LSTM is a srecial wrsion of RNN
" RNN Rmenbers very few urds whch are near by I t fvgets the previous wads
f ue have a vey long sentence . this problern s kroun as vanshing or
cxlodng gradient probler.
" et us take the follbwing two terms.
h>shot term me mory
c’ bng term memory
RNS ae not able to handle very long sequences.
Sequence to seqvence model:
"Amachine kaming architecture called the seguence to-seqience (seg1scg)model
is intended for aplcations involving seguential data.
"After pDCeSsig an input seqUence, it produces an outpub sequence
" An encoder and a decoder are the tuo main patts of the design.
Used for machine brarslatiorn systerns.
Encoder loccoder
Sequence to sequence model-Encodey:
"The encoder block's primary tonction is t pto cess the inpub sequence and
etiact data into a fiued- size context vectot.
" tveceives the loput sequerce.
element
-It uses neuial netwotks [or tvansforener aichitecture) to piocess Goch
of the input sequence
led
zt mantains an ntenal s tate during this operabian, and the fnal corcea
state Set ves as the context vector that contains a condensed epicserttaton
of the conmplete input sguence.
