Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit cd08bec

Browse files
antmarakisnorvig
antmarakis
authored and
norvig
committed
Notebook + Implementation: Perceptron (#512)
* Update learning.ipynb * Delete perceptron.png * Add new Perceptron image * Update Perceptron Implementation
1 parent 856e8d9 commit cd08bec

File tree

3 files changed

+74
-74
lines changed

3 files changed

+74
-74
lines changed

‎images/perceptron.png

-1.45 KB
Loading

‎learning.ipynb

Lines changed: 69 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
},
1515
{
1616
"cell_type": "code",
17-
"execution_count": 6,
17+
"execution_count": 1,
1818
"metadata": {
1919
"collapsed": true,
2020
"deletable": true,
@@ -582,7 +582,10 @@
582582
},
583583
{
584584
"cell_type": "markdown",
585-
"metadata": {},
585+
"metadata": {
586+
"deletable": true,
587+
"editable": true
588+
},
586589
"source": [
587590
"## Distance Functions\n",
588591
"\n",
@@ -597,7 +600,9 @@
597600
"cell_type": "code",
598601
"execution_count": 2,
599602
"metadata": {
600-
"collapsed": false
603+
"collapsed": false,
604+
"deletable": true,
605+
"editable": true
601606
},
602607
"outputs": [
603608
{
@@ -619,7 +624,10 @@
619624
},
620625
{
621626
"cell_type": "markdown",
622-
"metadata": {},
627+
"metadata": {
628+
"deletable": true,
629+
"editable": true
630+
},
623631
"source": [
624632
"### Euclidean Distance (`euclidean_distance`)\n",
625633
"\n",
@@ -630,7 +638,9 @@
630638
"cell_type": "code",
631639
"execution_count": 13,
632640
"metadata": {
633-
"collapsed": false
641+
"collapsed": false,
642+
"deletable": true,
643+
"editable": true
634644
},
635645
"outputs": [
636646
{
@@ -652,7 +662,10 @@
652662
},
653663
{
654664
"cell_type": "markdown",
655-
"metadata": {},
665+
"metadata": {
666+
"deletable": true,
667+
"editable": true
668+
},
656669
"source": [
657670
"### Hamming Distance (`hamming_distance`)\n",
658671
"\n",
@@ -663,7 +676,9 @@
663676
"cell_type": "code",
664677
"execution_count": 4,
665678
"metadata": {
666-
"collapsed": false
679+
"collapsed": false,
680+
"deletable": true,
681+
"editable": true
667682
},
668683
"outputs": [
669684
{
@@ -685,7 +700,10 @@
685700
},
686701
{
687702
"cell_type": "markdown",
688-
"metadata": {},
703+
"metadata": {
704+
"deletable": true,
705+
"editable": true
706+
},
689707
"source": [
690708
"### Mean Boolean Error (`mean_boolean_error`)\n",
691709
"\n",
@@ -696,7 +714,9 @@
696714
"cell_type": "code",
697715
"execution_count": 9,
698716
"metadata": {
699-
"collapsed": false
717+
"collapsed": false,
718+
"deletable": true,
719+
"editable": true
700720
},
701721
"outputs": [
702722
{
@@ -718,7 +738,10 @@
718738
},
719739
{
720740
"cell_type": "markdown",
721-
"metadata": {},
741+
"metadata": {
742+
"deletable": true,
743+
"editable": true
744+
},
722745
"source": [
723746
"### Mean Error (`mean_error`)\n",
724747
"\n",
@@ -729,7 +752,9 @@
729752
"cell_type": "code",
730753
"execution_count": 10,
731754
"metadata": {
732-
"collapsed": false
755+
"collapsed": false,
756+
"deletable": true,
757+
"editable": true
733758
},
734759
"outputs": [
735760
{
@@ -751,7 +776,10 @@
751776
},
752777
{
753778
"cell_type": "markdown",
754-
"metadata": {},
779+
"metadata": {
780+
"deletable": true,
781+
"editable": true
782+
},
755783
"source": [
756784
"### Mean Square Error (`ms_error`)\n",
757785
"\n",
@@ -762,7 +790,9 @@
762790
"cell_type": "code",
763791
"execution_count": 11,
764792
"metadata": {
765-
"collapsed": false
793+
"collapsed": false,
794+
"deletable": true,
795+
"editable": true
766796
},
767797
"outputs": [
768798
{
@@ -784,7 +814,10 @@
784814
},
785815
{
786816
"cell_type": "markdown",
787-
"metadata": {},
817+
"metadata": {
818+
"deletable": true,
819+
"editable": true
820+
},
788821
"source": [
789822
"### Root of Mean Square Error (`rms_error`)\n",
790823
"\n",
@@ -795,7 +828,9 @@
795828
"cell_type": "code",
796829
"execution_count": 12,
797830
"metadata": {
798-
"collapsed": false
831+
"collapsed": false,
832+
"deletable": true,
833+
"editable": true
799834
},
800835
"outputs": [
801836
{
@@ -1062,8 +1097,17 @@
10621097
"\n",
10631098
"The Perceptron is a linear classifier. It works the same way as a neural network with no hidden layers (just input and output). First it trains its weights given a dataset and then it can classify a new item by running it through the network.\n",
10641099
"\n",
1065-
"You can think of it as a single neuron. It has *n* synapses, each with its own weight. Each synapse corresponds to one item feature. Perceptron multiplies each item feature with the corresponding synapse weight and then adds them together (aka, the dot product) and checks whether this value is greater than the threshold. If yes, it returns 1. It returns 0 otherwise.\n",
1100+
"Its input layer consists of the the item features, while the output layer consists of nodes (also called neurons). Each node in the output layer has *n* synapses (for every item feature), each with its own weight. Then, the nodes find the dot product of the item features and the synapse weights. These values then pass through an activation function (usually a sigmoid). Finally, we pick the largest of the values and we return its index.\n",
1101+
"\n",
1102+
"Note that in classification problems each node represents a class. The final classification is the class/node with the max output value.\n",
10661103
"\n",
1104+
"Below you can see a single node/neuron in the outer layer. With *f* we denote the item features, with *w* the synapse weights, then inside the node we have the dot product and the activation function, *g*."
1105+
]
1106+
},
1107+
{
1108+
"cell_type": "markdown",
1109+
"metadata": {},
1110+
"source": [
10671111
"![perceptron](images/perceptron.png)"
10681112
]
10691113
},
@@ -1076,50 +1120,20 @@
10761120
"source": [
10771121
"### Implementation\n",
10781122
"\n",
1079-
"First, we train (calculate) the weights given a dataset, using the `BackPropagationLearner` function of `learning.py`. We then return a function, `predict`, which we will use in the future to classify a new item. The function computes the (algebraic) dot product of the item with the calculated weights. If the result is greater than a predefined threshold (usually 0.5, 0 or 1), it returns 1. If it is less than the threshold, it returns 0.\n",
1080-
"\n",
1081-
"NOTE: The current implementation of the algorithm classifies an item into one of two classes. It is a binary classifier and will not work well for multi-class datasets."
1123+
"First, we train (calculate) the weights given a dataset, using the `BackPropagationLearner` function of `learning.py`. We then return a function, `predict`, which we will use in the future to classify a new item. The function computes the (algebraic) dot product of the item with the calculated weights for each node in the outer layer. Then it picks the greatest value and classifies the item in the corresponding class."
10821124
]
10831125
},
10841126
{
10851127
"cell_type": "code",
1086-
"execution_count": 6,
1128+
"execution_count": 2,
10871129
"metadata": {
10881130
"collapsed": true,
10891131
"deletable": true,
10901132
"editable": true
10911133
},
10921134
"outputs": [],
10931135
"source": [
1094-
"def PerceptronLearner(dataset, learning_rate=0.01, epochs=100):\n",
1095-
" \"\"\"Logistic Regression, NO hidden layer\"\"\"\n",
1096-
" i_units = len(dataset.inputs)\n",
1097-
" o_units = 1 # As of now, dataset.target gives only one index.\n",
1098-
" hidden_layer_sizes = []\n",
1099-
" raw_net = network(i_units, hidden_layer_sizes, o_units)\n",
1100-
" learned_net = BackPropagationLearner(dataset, raw_net, learning_rate, epochs)\n",
1101-
"\n",
1102-
" def predict(example):\n",
1103-
" # Input nodes\n",
1104-
" i_nodes = learned_net[0]\n",
1105-
"\n",
1106-
" # Activate input layer\n",
1107-
" for v, n in zip(example, i_nodes):\n",
1108-
" n.value = v\n",
1109-
"\n",
1110-
" # Forward pass\n",
1111-
" for layer in learned_net[1:]:\n",
1112-
" for node in layer:\n",
1113-
" inc = [n.value for n in node.inputs]\n",
1114-
" in_val = dotproduct(inc, node.weights)\n",
1115-
" node.value = node.activation(in_val)\n",
1116-
"\n",
1117-
" # Hypothesis\n",
1118-
" o_nodes = learned_net[-1]\n",
1119-
" pred = [o_nodes[i].value for i in range(o_units)]\n",
1120-
" return 1 if pred[0] >= 0.5 else 0\n",
1121-
"\n",
1122-
" return predict"
1136+
"%psource PerceptronLearner"
11231137
]
11241138
},
11251139
{
@@ -1129,11 +1143,9 @@
11291143
"editable": true
11301144
},
11311145
"source": [
1132-
"The weights are trained from the `BackPropagationLearner`. Note that the perceptron is a one-layer neural network, without any hidden layers. So, in `BackPropagationLearner`, we will pass no hidden layers. From that function we get our network, which is just one node, with the weights calculated.\n",
1133-
"\n",
1134-
"`PerceptronLearner` returns `predict`, a function that can be used to classify a new item.\n",
1146+
"Note that the Perceptron is a one-layer neural network, without any hidden layers. So, in `BackPropagationLearner`, we will pass no hidden layers. From that function we get our network, which is just one layer, with the weights calculated.\n",
11351147
"\n",
1136-
"That function passes the input/example through the network, calculating the dot product of the input and the weights. If that value is greater than or equal to 0.5, it returns 1. Otherwise it returns 0."
1148+
"That function `predict` passes the input/example through the network, calculating the dot product of the input and the weights for each node and returns the class with the max dot product."
11371149
]
11381150
},
11391151
{
@@ -1145,14 +1157,12 @@
11451157
"source": [
11461158
"### Example\n",
11471159
"\n",
1148-
"We will train the Perceptron on the iris dataset. Because, though, the algorithm is a binary classifier (which means it classifies an item in one of two classes) and the iris dataset has three classes, we need to transform the dataset into a proper form, with only two classes. Therefore, we will remove the third and final class of the dataset, *Virginica*.\n",
1149-
"\n",
1150-
"Then, we will try and classify the item/flower with measurements of 5,3,1,0.1."
1160+
"We will train the Perceptron on the iris dataset. Because though the `BackPropagationLearner` works with integer indexes and not strings, we need to convert class names to integers. Then, we will try and classify the item/flower with measurements of 5, 3, 1, 0.1."
11511161
]
11521162
},
11531163
{
11541164
"cell_type": "code",
1155-
"execution_count": 7,
1165+
"execution_count": 10,
11561166
"metadata": {
11571167
"collapsed": false,
11581168
"deletable": true,
@@ -1169,11 +1179,10 @@
11691179
],
11701180
"source": [
11711181
"iris = DataSet(name=\"iris\")\n",
1172-
"iris.remove_examples(\"virginica\")\n",
11731182
"iris.classes_to_numbers()\n",
11741183
"\n",
11751184
"perceptron = PerceptronLearner(iris)\n",
1176-
"print(perceptron([5,3,1,0.1]))"
1185+
"print(perceptron([5, 3, 1, 0.1]))"
11771186
]
11781187
},
11791188
{
@@ -1183,7 +1192,7 @@
11831192
"editable": true
11841193
},
11851194
"source": [
1186-
"The output is 0, which means the item is classified in the first class, *setosa*. This is indeed correct. Note that the Perceptron algorithm is not perfect and may produce false classifications."
1195+
"The output is 0, which means the item is classified in the first class, \"Setosa\". This is indeed correct. Note that the Perceptron algorithm is not perfect and may produce false classifications."
11871196
]
11881197
},
11891198
{

‎learning.py

Lines changed: 5 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -653,24 +653,15 @@ def PerceptronLearner(dataset, learning_rate=0.01, epochs=100):
653653
learned_net = BackPropagationLearner(dataset, raw_net, learning_rate, epochs)
654654

655655
def predict(example):
656-
# Input nodes
657-
i_nodes = learned_net[0]
658-
659-
# Activate input layer
660-
for v, n in zip(example, i_nodes):
661-
n.value = v
656+
o_nodes = learned_net[1]
662657

663658
# Forward pass
664-
for layer in learned_net[1:]:
665-
for node in layer:
666-
inc = [n.value for n in node.inputs]
667-
in_val = dotproduct(inc, node.weights)
668-
node.value = node.activation(in_val)
659+
for node in o_nodes:
660+
in_val = dotproduct(example, node.weights)
661+
node.value = node.activation(in_val)
669662

670663
# Hypothesis
671-
o_nodes = learned_net[-1]
672-
prediction = find_max_node(o_nodes)
673-
return prediction
664+
return find_max_node(o_nodes)
674665

675666
return predict
676667

0 commit comments

Comments
 (0)