Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 671dea6

Browse files
reachtarunherereachtarunhere
committed
Added section for inference by full joint distributions
1 parent d81be44 commit 671dea6

File tree

1 file changed

+162
-0
lines changed

1 file changed

+162
-0
lines changed

‎probability.ipynb

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,168 @@
263263
"source": [
264264
"j.values('X')"
265265
]
266+
},
267+
{
268+
"cell_type": "markdown",
269+
"metadata": {},
270+
"source": [
271+
"## Inference Using Full Joint Distributions\n",
272+
"\n",
273+
"In this section we use Full Joint Distributions to calculate the posterior distribution given some evidence. We represent evidence by using a python dictionary with variables as dict keys and dict values representing the values.\n",
274+
"\n",
275+
"This is illustrated in **Section 13.3** of the book. The functions **enumerate_joint** and **enumerate_joint_ask** implement this functionality. Under the hood they implement **Equation 13.9** from the book.\n",
276+
"\n",
277+
"$$\\textbf{P}(X | \\textbf{e}) = α \\textbf{P}(X, \\textbf{e}) = α \\sum_{y} \\textbf{P}(X, \\textbf{e}, \\textbf{y})$$\n",
278+
"\n",
279+
"Here **α** is the normalizing factor. **X** is our query variable and **e** is the evidence. According to the equation we enumerate on the remaining variables **y** (not in evidence or query variable) i.e. all possible combinations of **y**\n",
280+
"\n",
281+
"We will be using the same example as the book. Let us create the full joint distribution from **Figure 13.3**. "
282+
]
283+
},
284+
{
285+
"cell_type": "code",
286+
"execution_count": null,
287+
"metadata": {
288+
"collapsed": false
289+
},
290+
"outputs": [],
291+
"source": [
292+
"full_joint = JointProbDist(['Cavity', 'Toothache', 'Catch'])\n",
293+
"full_joint[dict(Cavity=True, Toothache=True, Catch=True)] = 0.108\n",
294+
"full_joint[dict(Cavity=True, Toothache=True, Catch=False)] = 0.012\n",
295+
"full_joint[dict(Cavity=True, Toothache=False, Catch=True)] = 0.016\n",
296+
"full_joint[dict(Cavity=True, Toothache=False, Catch=False)] = 0.064\n",
297+
"full_joint[dict(Cavity=False, Toothache=True, Catch=True)] = 0.072\n",
298+
"full_joint[dict(Cavity=False, Toothache=False, Catch=True)] = 0.144\n",
299+
"full_joint[dict(Cavity=False, Toothache=True, Catch=False)] = 0.008\n",
300+
"full_joint[dict(Cavity=False, Toothache=False, Catch=False)] = 0.576"
301+
]
302+
},
303+
{
304+
"cell_type": "markdown",
305+
"metadata": {},
306+
"source": [
307+
"Let us now look at the **enumerate_joint** function returns the sum of those entries in P consistent with e,provided variables is P's remaining variables (the ones not in e). Here, P refers to the full joint distribution. The function uses a recursive call in its implementation. The first parameter **variables** refers to remaining variables. The function in each recursive call keeps on variable constant while varying others."
308+
]
309+
},
310+
{
311+
"cell_type": "code",
312+
"execution_count": null,
313+
"metadata": {
314+
"collapsed": true
315+
},
316+
"outputs": [],
317+
"source": [
318+
"%psource enumerate_joint"
319+
]
320+
},
321+
{
322+
"cell_type": "markdown",
323+
"metadata": {},
324+
"source": [
325+
"Let us assume we want to find **P(Toothache=True)**. This can be obtained by marginalization (**Equation 13.6**). We can use **enumerate_joint** to solve for this by taking Cavity=True as our evidence. **enumerate_joint** will return the sum of probabilities consistent with evidence i.e. Marginal Probability."
326+
]
327+
},
328+
{
329+
"cell_type": "code",
330+
"execution_count": null,
331+
"metadata": {
332+
"collapsed": false
333+
},
334+
"outputs": [],
335+
"source": [
336+
"evidence = dict(Toothache=True)\n",
337+
"variables = ['Cavity', 'Catch'] # variables not part of evidence\n",
338+
"ans1 = enumerate_joint(variables, evidence, full_joint)\n",
339+
"ans1"
340+
]
341+
},
342+
{
343+
"cell_type": "markdown",
344+
"metadata": {},
345+
"source": [
346+
"You can verify that result from the row in our definition. We can use the same function to find more complex probabilities like **P(Cavity=True and Toothache=True)** "
347+
]
348+
},
349+
{
350+
"cell_type": "code",
351+
"execution_count": null,
352+
"metadata": {
353+
"collapsed": false
354+
},
355+
"outputs": [],
356+
"source": [
357+
"evidence = dict(Cavity=True, Toothache=True)\n",
358+
"variables = ['Catch'] # variables not part of evidence\n",
359+
"ans2 = enumerate_joint(variables, evidence, full_joint)\n",
360+
"ans2"
361+
]
362+
},
363+
{
364+
"cell_type": "markdown",
365+
"metadata": {},
366+
"source": [
367+
"Being able to find sum of probabilities satisfying given evidence allows us to compute conditional probabilities like **P(Cavity=True | Toothache=True)** as we can rewrite this as $$P(Cavity=True | Toothache = True) = \\frac{P(Cavity=True \\ and \\ Toothache=True)}{P(Toothache=True)}$$\n",
368+
"\n",
369+
"We have already calculated both the numerator and denominator."
370+
]
371+
},
372+
{
373+
"cell_type": "code",
374+
"execution_count": null,
375+
"metadata": {
376+
"collapsed": false
377+
},
378+
"outputs": [],
379+
"source": [
380+
"ans2/ans1"
381+
]
382+
},
383+
{
384+
"cell_type": "markdown",
385+
"metadata": {},
386+
"source": [
387+
"We might be interested in the probability distribution of a particular variable conditioned on some evidence. This can involve doing calculations like above for each possible value of the variable. This has been implemented slightly differently using normalization in the function **enumerate_joint_ask** which returns a probability distribution over the values of the variable **X**, given the {var:val} observations **e**, in the **JointProbDist P**. The implementation of this function calls **enumerate_joint** for each value of the query variable and passes **extended evidence** with the new evidence having **X = x<sub>i</sub>**. This is followed by normalization of the obtained distribution."
388+
]
389+
},
390+
{
391+
"cell_type": "code",
392+
"execution_count": null,
393+
"metadata": {
394+
"collapsed": true
395+
},
396+
"outputs": [],
397+
"source": [
398+
"%psource enumerate_joint_ask"
399+
]
400+
},
401+
{
402+
"cell_type": "markdown",
403+
"metadata": {},
404+
"source": [
405+
"Let us find **P(Cavity | Toothache=True)** using **enumerate_joint_ask**."
406+
]
407+
},
408+
{
409+
"cell_type": "code",
410+
"execution_count": null,
411+
"metadata": {
412+
"collapsed": false
413+
},
414+
"outputs": [],
415+
"source": [
416+
"query_variable = 'Cavity'\n",
417+
"evidence = dict(Toothache=True)\n",
418+
"ans = enumerate_joint_ask(query_variable, evidence, full_joint)\n",
419+
"(ans[True], ans[False])"
420+
]
421+
},
422+
{
423+
"cell_type": "markdown",
424+
"metadata": {},
425+
"source": [
426+
"You can verify that the first value is the same as we obtained earlier by manual calculation."
427+
]
266428
}
267429
],
268430
"metadata": {

0 commit comments

Comments
 (0)