Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 01e4fcd

Browse files
antmarakisnorvig
antmarakis
authored and
norvig
committed
Update learning.ipynb (aimacode#628)
1 parent fc1b865 commit 01e4fcd

File tree

1 file changed

+144
-19
lines changed

1 file changed

+144
-19
lines changed

‎learning.ipynb

Lines changed: 144 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@
110110
},
111111
{
112112
"cell_type": "code",
113-
"execution_count": 2,
113+
"execution_count": null,
114114
"metadata": {
115115
"collapsed": true
116116
},
@@ -817,13 +817,13 @@
817817
},
818818
{
819819
"cell_type": "code",
820-
"execution_count": 23,
820+
"execution_count": null,
821821
"metadata": {
822822
"collapsed": true
823823
},
824824
"outputs": [],
825825
"source": [
826-
"%psource PluralityLearner"
826+
"psource(PluralityLearner)"
827827
]
828828
},
829829
{
@@ -909,13 +909,13 @@
909909
},
910910
{
911911
"cell_type": "code",
912-
"execution_count": 25,
912+
"execution_count": null,
913913
"metadata": {
914914
"collapsed": true
915915
},
916916
"outputs": [],
917917
"source": [
918-
"%psource NearestNeighborLearner"
918+
"psource(NearestNeighborLearner)"
919919
]
920920
},
921921
{
@@ -991,19 +991,39 @@
991991
"\n",
992992
"Information Gain is difference between entropy of the parent and weighted sum of entropy of children. The feature used for splitting is the one which provides the most information gain.\n",
993993
"\n",
994+
"#### Pseudocode\n",
995+
"\n",
996+
"You can view the pseudocode by running the cell below:"
997+
]
998+
},
999+
{
1000+
"cell_type": "code",
1001+
"execution_count": null,
1002+
"metadata": {
1003+
"collapsed": true
1004+
},
1005+
"outputs": [],
1006+
"source": [
1007+
"pseudocode(\"Decision Tree Learning\")"
1008+
]
1009+
},
1010+
{
1011+
"cell_type": "markdown",
1012+
"metadata": {},
1013+
"source": [
9941014
"### Implementation\n",
9951015
"The nodes of the tree constructed by our learning algorithm are stored using either `DecisionFork` or `DecisionLeaf` based on whether they are a parent node or a leaf node respectively."
9961016
]
9971017
},
9981018
{
9991019
"cell_type": "code",
1000-
"execution_count": 27,
1020+
"execution_count": null,
10011021
"metadata": {
10021022
"collapsed": true
10031023
},
10041024
"outputs": [],
10051025
"source": [
1006-
"%psource DecisionFork"
1026+
"psource(DecisionFork)"
10071027
]
10081028
},
10091029
{
@@ -1015,13 +1035,13 @@
10151035
},
10161036
{
10171037
"cell_type": "code",
1018-
"execution_count": 28,
1038+
"execution_count": null,
10191039
"metadata": {
10201040
"collapsed": true
10211041
},
10221042
"outputs": [],
10231043
"source": [
1024-
"%psource DecisionLeaf"
1044+
"psource(DecisionLeaf)"
10251045
]
10261046
},
10271047
{
@@ -1033,13 +1053,13 @@
10331053
},
10341054
{
10351055
"cell_type": "code",
1036-
"execution_count": 29,
1056+
"execution_count": null,
10371057
"metadata": {
10381058
"collapsed": true
10391059
},
10401060
"outputs": [],
10411061
"source": [
1042-
"%psource DecisionTreeLearner"
1062+
"psource(DecisionTreeLearner)"
10431063
]
10441064
},
10451065
{
@@ -1142,7 +1162,7 @@
11421162
"source": [
11431163
"### Implementation\n",
11441164
"\n",
1145-
"The implementation of the Naive Bayes Classifier is split in two; Discrete and Continuous. The user can choose between them with the argument `continuous`."
1165+
"The implementation of the Naive Bayes Classifier is split in two; *Learning* and *Simple*. The *learning* classifier takes as input a dataset and learns the needed distributions from that. It is itself split into two, for discrete and continuous features. The *simple* classifier takes as input not a dataset, but already calculated distributions (a dictionary of `CountingProbDist` objects)."
11461166
]
11471167
},
11481168
{
@@ -1237,13 +1257,13 @@
12371257
},
12381258
{
12391259
"cell_type": "code",
1240-
"execution_count": 32,
1260+
"execution_count": null,
12411261
"metadata": {
12421262
"collapsed": true
12431263
},
12441264
"outputs": [],
12451265
"source": [
1246-
"%psource NaiveBayesDiscrete"
1266+
"psource(NaiveBayesDiscrete)"
12471267
]
12481268
},
12491269
{
@@ -1327,13 +1347,42 @@
13271347
},
13281348
{
13291349
"cell_type": "code",
1330-
"execution_count": 35,
1350+
"execution_count": null,
13311351
"metadata": {
13321352
"collapsed": true
13331353
},
13341354
"outputs": [],
13351355
"source": [
1336-
"%psource NaiveBayesContinuous"
1356+
"psource(NaiveBayesContinuous)"
1357+
]
1358+
},
1359+
{
1360+
"cell_type": "markdown",
1361+
"metadata": {},
1362+
"source": [
1363+
"#### Simple\n",
1364+
"\n",
1365+
"The simple classifier (chosen with the argument `simple`) does not learn from a dataset, instead it takes as input a dictionary of already calculated `CountingProbDist` objects and returns a predictor function. The dictionary is in the following form: `(Class Name, Class Probability): CountingProbDist Object`.\n",
1366+
"\n",
1367+
"Each class has its own probability distribution. The classifier given a list of features calculates the probability of the input for each class and returns the max. The only pre-processing work is to create dictionaries for the distribution of classes (named `targets`) and attributes/features.\n",
1368+
"\n",
1369+
"The complete code for the simple classifier:"
1370+
]
1371+
},
1372+
{
1373+
"cell_type": "code",
1374+
"execution_count": null,
1375+
"metadata": {},
1376+
"outputs": [],
1377+
"source": [
1378+
"psource(NaiveBayesSimple)"
1379+
]
1380+
},
1381+
{
1382+
"cell_type": "markdown",
1383+
"metadata": {},
1384+
"source": [
1385+
"This classifier is useful when you already have calculated the distributions and you need to predict future items."
13371386
]
13381387
},
13391388
{
@@ -1385,7 +1434,83 @@
13851434
"cell_type": "markdown",
13861435
"metadata": {},
13871436
"source": [
1388-
"Notice how the Discrete Classifier misclassified the second item, while the Continuous one had no problem."
1437+
"Notice how the Discrete Classifier misclassified the second item, while the Continuous one had no problem.\n",
1438+
"\n",
1439+
"Let's now take a look at the simple classifier. First we will come up with a sample problem to solve. Say we are given three bags. Each bag contains three letters ('a', 'b' and 'c') of different quantities. We are given a string of letters and we are tasked with finding from which bag the string of letters came.\n",
1440+
"\n",
1441+
"Since we know the probability distribution of the letters for each bag, we can use the naive bayes classifier to make our prediction."
1442+
]
1443+
},
1444+
{
1445+
"cell_type": "code",
1446+
"execution_count": 2,
1447+
"metadata": {
1448+
"collapsed": true
1449+
},
1450+
"outputs": [],
1451+
"source": [
1452+
"bag1 = 'a'*50 + 'b'*30 + 'c'*15\n",
1453+
"dist1 = CountingProbDist(bag1)\n",
1454+
"bag2 = 'a'*30 + 'b'*45 + 'c'*20\n",
1455+
"dist2 = CountingProbDist(bag2)\n",
1456+
"bag3 = 'a'*20 + 'b'*20 + 'c'*35\n",
1457+
"dist3 = CountingProbDist(bag3)"
1458+
]
1459+
},
1460+
{
1461+
"cell_type": "markdown",
1462+
"metadata": {},
1463+
"source": [
1464+
"Now that we have the `CountingProbDist` objects for each bag/class, we will create the dictionary. We assume that it is equally probable that we will pick from any bag."
1465+
]
1466+
},
1467+
{
1468+
"cell_type": "code",
1469+
"execution_count": 3,
1470+
"metadata": {
1471+
"collapsed": true
1472+
},
1473+
"outputs": [],
1474+
"source": [
1475+
"dist = {('First', 0.5): dist1, ('Second', 0.3): dist2, ('Third', 0.2): dist3}\n",
1476+
"nBS = NaiveBayesLearner(dist, simple=True)"
1477+
]
1478+
},
1479+
{
1480+
"cell_type": "markdown",
1481+
"metadata": {},
1482+
"source": [
1483+
"Now we can start making predictions:"
1484+
]
1485+
},
1486+
{
1487+
"cell_type": "code",
1488+
"execution_count": 4,
1489+
"metadata": {},
1490+
"outputs": [
1491+
{
1492+
"name": "stdout",
1493+
"output_type": "stream",
1494+
"text": [
1495+
"First\n",
1496+
"Second\n",
1497+
"Third\n"
1498+
]
1499+
}
1500+
],
1501+
"source": [
1502+
"print(nBS('aab')) # We can handle strings\n",
1503+
"print(nBS(['b', 'b'])) # And lists!\n",
1504+
"print(nBS('ccbcc'))"
1505+
]
1506+
},
1507+
{
1508+
"cell_type": "markdown",
1509+
"metadata": {},
1510+
"source": [
1511+
"The results make intuitive sence. The first bag has a high amount of 'a's, the second has a high amount of 'b's and the third has a high amount of 'c's. The classifier seems to confirm this intuition.\n",
1512+
"\n",
1513+
"Note that the simple classifier doesn't distinguish between discrete and continuous values. It just takes whatever it is given. Also, the `simple` option on the `NaiveBayesLearner` overrides the `continuous` argument. `NaiveBayesLearner(d, simple=True, continuous=False)` just creates a simple classifier."
13891514
]
13901515
},
13911516
{
@@ -1423,13 +1548,13 @@
14231548
},
14241549
{
14251550
"cell_type": "code",
1426-
"execution_count": 37,
1551+
"execution_count": null,
14271552
"metadata": {
14281553
"collapsed": true
14291554
},
14301555
"outputs": [],
14311556
"source": [
1432-
"%psource PerceptronLearner"
1557+
"psource(PerceptronLearner)"
14331558
]
14341559
},
14351560
{

0 commit comments

Comments
 (0)