Interpretatiof Logistic Regression
Uploaded by
Milton EspinozaInterpretatiof Logistic Regression
Uploaded by
Milton EspinozaInterpretation of
Logistic Regression
MNIST Data Set
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Number
0-9
DATA MACHINE MODEL PREDICTION
Images are Encoded as Numbers
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Images are Encoded as Numbers
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Images are Encoded as Numbers
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Images are Encoded as Numbers
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0
0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0
X=
0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Output is Binary
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Learning on MNIST
Training Set Logistic Regression Model
σ(zi) (or “Network”)
zi = (b1 X xi1) + (b2 X xi2 (b2 X xi2) + ꔇ + (bM X xiM) + b0
zi
bM
b1
xi1 xiM
(b0, b1,… bM)
Learned Parameters
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Zooming in on 0/1
subject learned parameters
– 0.2
– 0.1
σ( x – 0.0
– –0.1
)
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Zooming in on 0/1
– 0.2
– 0.1
σ( – 0.0
– –0.1
) = 0.006
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Zooming in on 0/1
negative sections – 0.2
– 0.1
σ( – 0.0
– –0.1
) = 0.006
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Zooming in on 0/1
– 0.2
– 0.1
σ( – 0.0
– –0.1
) = 0.991
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Zooming in on 0/1
positive sections – 0.2
– 0.1
σ( – 0.0
– –0.1
) = 0.991
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Learned Weights for 0/1
– 0.2
– 0.1
σ( x – 0.0
– –0.1
) = 0.006
(6% chance it is a 1)
– –0.2
– 0.2
– 0.1
σ( x – 0.0
– –0.1
) = 0.991
(99.1% chance it is a 1)
– –0.2
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Interpretation of
Logistic Regression
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
xi1 xi2 xiM
add up the M products
xi
M
b
x x x
Σ
m=1
xim X bm
b1 b2 bM
Interpretation of
Logistic Regression
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
xi1 xi2 xiM
add up the M products
xi
M
b
x x x
Σ
m=1
xim X bm
b1 b2 bM
Compact Notation: xi ⊙b
Interpretation of
Logistic Regression
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
xi1 xi2 xiM
add up the M products
xi
M
b
x x x
Σ
m=1
xim X bm
b1 b2 bM
Compact Notation: xi ⊙b
inner product
Interpretation of Model
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
= b0 + xi ⊙ b
– 0.2
– 0.1
– 0.0
examples of class y = 0
– –0.1
– –0.2
filter
examples of class y = 1
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Interpretation of Model
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
= b0 + xi ⊙ b
– 0.2
– 0.1
– 0.0
examples of class y = 0
– –0.1
– –0.2
filter
examples of class y = 1
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Interpretation of Model
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
= b0 + xi ⊙ b
– 0.2
– 0.1
– 0.0
examples of class y = 0
– –0.1
– –0.2
filter b
examples of class y = 1
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Interpretation of Model
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
Outcome of y
= b0 + xi ⊙ b
Strong match between filter
– 0.2
and data indicates p(yi = 1|xi)
is high
– 0.1
Low match between filter
– 0.0 and data indicates p(yi = 1|xi)
examples of class y = 0 is low
– –0.1
– –0.2
filter b
examples of class y = 1
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Interpretation of
Logistic Regression
zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
Sigmoid Function p(yi = 1|xi) = σ(zi)
1
– 0.2
– 0.1
– 0.0
0.5
– –0.1
– –0.2
–6
0 zi filter b
–4 –2 0 2 4 6
Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
Credits
MNIST Dataset of Handwritten Digits (Images)
Yann LeCun (Courant Institue, NYU) and Corinna Cortes (Google Labs, New York) CC-by-SA 3.0
http://yann.lecun.com/exdb/mnist/
You might also like
- Logistic Regression For Binary Classification With Core APIs - TensorFlow CoreNo ratings yetLogistic Regression For Binary Classification With Core APIs - TensorFlow Core22 pages
- Learning Multiple Layers of Features From Tiny Images. Alex KrizhevskyNo ratings yetLearning Multiple Layers of Features From Tiny Images. Alex Krizhevsky60 pages
- Building A Brain in 10 Minutes: Perceptron Research From The 50's & 6 Perceptron Research From The 50's & 6No ratings yetBuilding A Brain in 10 Minutes: Perceptron Research From The 50's & 6 Perceptron Research From The 50's & 614 pages
- DP-100 Microsoft Exam Practice QuestionsNo ratings yetDP-100 Microsoft Exam Practice Questions56 pages
- Logistic Regression With A Neural Network Mindset: 1 - PackagesNo ratings yetLogistic Regression With A Neural Network Mindset: 1 - Packages23 pages
- Time Series Classification: Lab Based ProjectNo ratings yetTime Series Classification: Lab Based Project14 pages
- Neural Networks: and Shape IdentificationNo ratings yetNeural Networks: and Shape Identification95 pages
- MNIST Handwritten Digit Classification GuideNo ratings yetMNIST Handwritten Digit Classification Guide54 pages
- Ieee Guide For The Application, Specification, and Testing of Phase-Shiftingtransformers100% (1)Ieee Guide For The Application, Specification, and Testing of Phase-Shiftingtransformers43 pages
- Realization of Ground Penetrating Radar System For Environmental ExplorationsNo ratings yetRealization of Ground Penetrating Radar System For Environmental Explorations6 pages
- Classification of Ford Motor Data: Joerg D. WichardNo ratings yetClassification of Ford Motor Data: Joerg D. Wichard4 pages
- Sample Exam With Solutions. Econometrics II 2015.No ratings yetSample Exam With Solutions. Econometrics II 2015.15 pages
- 4-1 Introduction To Corrrelation and Its Properties0% (1)4-1 Introduction To Corrrelation and Its Properties14 pages
- تكنولوجيا المعلومات والاتصالات والنمو الاقتصادي في البلاد ال... ة - دراسة قياسية باستخدام نماذج البانل (panel data models) للفترة 2005 - 2018No ratings yetتكنولوجيا المعلومات والاتصالات والنمو الاقتصادي في البلاد ال... ة - دراسة قياسية باستخدام نماذج البانل (panel data models) للفترة 2005 - 201823 pages
- Weighted Regression of Calibration CurveNo ratings yetWeighted Regression of Calibration Curve2 pages