| Percept: | \n", @@ -226,117 +461,254 @@ "Action: | \n", "eat | \n", "drink | \n", - "move up | \n", + "\n",
+ "
| \n",
"
\n", + "***Arc-consistency algorithms*** remove all unsupported values from the domains of variables making the CSP *arc-consistent* or decide that a CSP is *inconsistent* by finding that some variable has no supported values in its domain.
\n", + "Heuristics significantly enhance the efficiency of the *arc-consistency algorithms* improving their average performance in terms of *consistency-checks* which can be considered a standard measure of goodness for such algorithms. *Arc-heuristic* operate at arc-level and selects the constraint that will be used for the next check, while *domain-heuristics* operate at domain-level and selects which values will be used for the next support-check." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from csp import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Domain-Heuristics for Arc-Consistency Algorithms\n", + "In [[1]](#cite-van2002domain) are investigated the effects of a *domain-heuristic* based on the notion of a *double-support check* by studying its average time-complexity.\n", + "\n", + "The objective of *arc-consistency algorithms* is to resolve some uncertainty; it has to be know, for each $v_i \\in D_i$ and for each $v_j \\in D_j$, whether it is supported.\n", + "\n", + "A *single-support check*, $(v_i, v_j) \\in C_{ij}$, is one in which, before the check is done, it is already known that either $v_i$ or $v_j$ are supported. \n", + "\n", + "A *double-support check* $(v_i, v_j) \\in C_{ij}$, is one in which there is still, before the check, uncertainty about the support-status of both $v_i$ and $v_j$. \n", + "\n", + "If a *double-support check* is successful, two uncertainties are resolved. If a *single-support check* is successful, only one uncertainty is resolved. A good *arc-consistency algorithm*, therefore, would always choose to do a *double-support check* in preference of a *single-support check*, because the cormer offers the potential higher payback.\n", + "\n", + "The improvement with *double-support check* is that, where possible, *consistency-checks* are used to find supports for two values, one value in the domain of each variable, which were previously known to be unsupported. It is motivated by the insight that *in order to minimize the number of consistency-checks it is necessary to maximize the number of uncertainties which are resolved per check*." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### AC-3b: an improved version of AC-3 with Double-Support Checks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As shown in [[2]](#cite-van2000improving) the idea is to use *double-support checks* to improve the average performance of `AC3` which does not exploit the fact that relations are bidirectional and results in a new general purpose *arc-consistency algorithm* called `AC3b`." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC3\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"[Figure 6.3]\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrevise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC3" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mrevise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Return true if we remove a value.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# If Xi=x conflicts with Xj=y for every possible y, eliminate Xi=x\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if all(not csp.constraints(Xi, x, Xj, y) for y in csp.curr_domains[Xj]):\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource revise" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "At any stage in the process of making 2-variable CSP *arc-consistent* in `AC3b`:\n", + "- there is a set $S_i^+ \\subseteq D_i$ whose values are all known to be supported by $X_j$;\n", + "- there is a set $S_i^? = D_i \\setminus S_i^+$ whose values are unknown, as yet, to be supported by $X_j$.\n", + "\n", + "The same holds if the roles for $X_i$ and $X_j$ are exchanged.\n", + "\n", + "In order to establish support for a value $v_i^? \\in S_i^?$ it seems better to try to find a support among the values in $S_j^?$ first, because for each $v_j^? \\in S_j^?$ the check $(v_i^?,v_j^?) \\in C_{ij}$ is a *double-support check* and it is just as likely that any $v_j^? \\in S_j^?$ supports $v_i^?$ than it is that any $v_j^+ \\in S_j^+$ does. Only if no support can be found among the elements in $S_j^?$, should the elements $v_j^+$ in $S_j^+$ be used for *single-support checks* $(v_i^?,v_j^+) \\in C_{ij}$. After it has been decided for each value in $D_i$ whether it is supported or not, either $S_x^+ = \\emptyset$ and the 2-variable CSP is *inconsistent*, or $S_x^+ \\neq \\emptyset$ and the CSP is *satisfiable*. In the latter case, the elements from $D_i$ which are supported by $j$ are given by $S_x^+$. The elements in $D_j$ which are supported by $x$ are given by the union of $S_j^+$ with the set of those elements of $S_j^?$ which further processing will show to be supported by some $v_i^+ \\in S_x^+$." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC3b\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Si_p values are all known to be supported by Xj\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Sj_p values are all known to be supported by Xi\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Dj - Sj_p = Sj_u values are unknown, as yet, to be supported by Xi\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# or queue -= {(Xj, Xi)} or queue.remove((Xj, Xi))\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference_update\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference_update\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the elements in D_j which are supported by Xi are given by the union of Sj_p with the set of those\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# elements of Sj_u which further processing will show to be supported by some vi_p in Si_p\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_u\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvi_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvj_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC3b" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvi_u\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# now, in order to establish support for a value vi_u in Di it seems better to try to find a support among\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the values in Sj_u first, because for each vj_u in Sj_u the check (vi_u, vj_u) is a double-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# and it is just as likely that any vj_u in Sj_u supports vi_u than it is that any vj_p in Sj_p does...\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_u\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# double-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvi_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvj_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# ... and only if no support can be found among the elements in Sj_u, should the elements vj_p in Sj_p be used\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# for single-support checks (vi_u, vj_p)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# single-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvi_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource partition" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "`AC3b` is a refinement of the `AC3` algorithm which consists of the fact that if, when arc $(i,j)$ is being processed and the reverse arc $(j,i)$ is also in the queue, then consistency-checks can be saved because only support for the elements in $S_j^?$ has to be found (as opposed to support for all the elements in $D_j$ in the\n", + "`AC3` algorithm).
\n", + "`AC3b` inherits all its properties like $\\mathcal{O}(ed^3)$ time-complexity and $\\mathcal{O}(e + nd)$ space-complexity fron `AC3` and where $n$ denotes the number of variables in the CSP, $e$ denotes the number of binary constraints and $d$ denotes the maximum domain-size of the variables." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Arc-Heuristics for Arc-Consistency Algorithms" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "Many *arc-heuristics* can be devised, based on three major features of CSPs:\n", + "- the number of acceptable pairs in each constraint (the *constraint size* or *satisfiability*);\n", + "- the *domain size*;\n", + "- the number of binary constraints that each variable participates in, equal to the *degree* of the node of that variable in the constraint graph. \n", + "\n", + "Simple examples of heuristics that might be expected to improve the efficiency of relaxation are:\n", + "- ordering the list of variable pairs by *increasing* relative *satisfiability*;\n", + "- ordering by *increasing size of the domain* of the variable $v_j$ relaxed against $v_i$;\n", + "- ordering by *descending degree* of node of the variable relaxed.\n", + "\n", + "In [[3]](#cite-wallace1992ordering) are investigated the effects of these *arc-heuristics* in an empirical way, experimenting the effects of them on random CSPs. Their results demonstrate that the first two, later called `sat up` and `dom j up` for n-ary and binary CSPs respectively, significantly reduce the number of *consistency-checks*." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdom_j_up\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSortedSet\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mneg\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dom_j_up" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0msat_up\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSortedSet\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource sat_up" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Experimental Results" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "For the experiments below on binary CSPs, in addition to the two *arc-consistency algorithms* already cited above, `AC3` and `AC3b`, the `AC4` algorithm was used.
\n", + "The `AC4` algorithm runs in $\\mathcal{O}(ed^2)$ worst-case time but can be slower than `AC3` on average cases." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC4\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# construction and initialization of support sets\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# propagation of removed values\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m-=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC4" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Sudoku" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Easy Sudoku" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". . 3 | . 2 . | 6 . .\n", + "9 . . | 3 . 5 | . . 1\n", + ". . 1 | 8 . 6 | 4 . .\n", + "------+-------+------\n", + ". . 8 | 1 . 2 | 9 . .\n", + "7 . . | . . . | . . 8\n", + ". . 6 | 7 . 8 | 2 . .\n", + "------+-------+------\n", + ". . 2 | 6 . 9 | 5 . .\n", + "8 . . | 2 . 3 | . . 9\n", + ". . 5 | . 1 . | 3 . .\n" + ] + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 23.6 ms, sys: 0 ns, total: 23.6 ms\n", + "Wall time: 22.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 11322 consistency-checks'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 7.43 ms, sys: 3.68 ms, total: 11.1 ms\n", + "Wall time: 10.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 8345 consistency-checks'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 56.3 ms, sys: 0 ns, total: 56.3 ms\n", + "Wall time: 55.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 27718 consistency-checks'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.2 ms, sys: 0 ns, total: 17.2 ms\n", + "Wall time: 16.9 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 6925 consistency-checks'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 40.9 ms, sys: 2.47 ms, total: 43.4 ms\n", + "Wall time: 41.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 6278 consistency-checks'" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 38.9 ms, sys: 1.96 ms, total: 40.9 ms\n", + "Wall time: 40.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 9393 consistency-checks'" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 8 3 | 9 2 1 | 6 5 7\n", + "9 6 7 | 3 4 5 | 8 2 1\n", + "2 5 1 | 8 7 6 | 4 9 3\n", + "------+-------+------\n", + "5 4 8 | 1 3 2 | 9 7 6\n", + "7 2 9 | 5 6 4 | 1 3 8\n", + "1 3 6 | 7 9 8 | 2 4 5\n", + "------+-------+------\n", + "3 7 2 | 6 8 9 | 5 1 4\n", + "8 1 4 | 2 5 3 | 7 6 9\n", + "6 9 5 | 4 1 7 | 3 8 2\n" + ] + } + ], + "source": [ + "backtracking_search(sudoku, select_unassigned_variable=mrv, inference=forward_checking)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Harder Sudoku" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 1 7 | 3 6 9 | 8 . 5\n", + ". 3 . | . . . | . . .\n", + ". . . | 7 . . | . . .\n", + "------+-------+------\n", + ". 2 . | . . . | . 6 .\n", + ". . . | . 8 . | 4 . .\n", + ". . . | . 1 . | . . .\n", + "------+-------+------\n", + ". . . | 6 . 3 | . 7 .\n", + "5 . . | 2 . . | . . .\n", + "1 . 4 | . . . | . . .\n" + ] + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.7 ms, sys: 481 µs, total: 18.2 ms\n", + "Wall time: 17.2 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 12837 consistency-checks'" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 24.1 ms, sys: 2.6 ms, total: 26.7 ms\n", + "Wall time: 25.1 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 8864 consistency-checks'" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 63.4 ms, sys: 3.48 ms, total: 66.9 ms\n", + "Wall time: 65.5 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 44213 consistency-checks'" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 9.96 ms, sys: 570 µs, total: 10.5 ms\n", + "Wall time: 10.3 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 7045 consistency-checks'" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 36.1 ms, sys: 0 ns, total: 36.1 ms\n", + "Wall time: 35.5 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 6994 consistency-checks'" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 40.3 ms, sys: 0 ns, total: 40.3 ms\n", + "Wall time: 39.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 19210 consistency-checks'" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 1 7 | 3 6 9 | 8 2 5\n", + "6 3 2 | 1 5 8 | 9 4 7\n", + "9 5 8 | 7 2 4 | 3 1 6\n", + "------+-------+------\n", + "8 2 5 | 4 3 7 | 1 6 9\n", + "7 9 1 | 5 8 6 | 4 3 2\n", + "3 4 6 | 9 1 2 | 7 5 8\n", + "------+-------+------\n", + "2 8 9 | 6 4 3 | 5 7 1\n", + "5 7 3 | 2 9 1 | 6 8 4\n", + "1 6 4 | 8 7 5 | 2 9 3\n" + ] + } + ], + "source": [ + "backtracking_search(sudoku, select_unassigned_variable=mrv, inference=forward_checking)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### 8 Queens" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n" + ] + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "chess.display(chess.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 689 µs, sys: 193 µs, total: 882 µs\n", + "Wall time: 892 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 666 consistency-checks'" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 451 µs, sys: 127 µs, total: 578 µs\n", + "Wall time: 584 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 428 consistency-checks'" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3b(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 8.53 ms, sys: 109 µs, total: 8.64 ms\n", + "Wall time: 8.48 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 4096 consistency-checks'" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC4(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.88 ms, sys: 0 ns, total: 1.88 ms\n", + "Wall time: 1.88 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 666 consistency-checks'" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3(chess, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.21 ms, sys: 326 µs, total: 1.53 ms\n", + "Wall time: 1.54 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 792 consistency-checks'" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3b(chess, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 4.71 ms, sys: 0 ns, total: 4.71 ms\n", + "Wall time: 4.65 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 4096 consistency-checks'" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC4(chess, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". - . - Q - . - 2 2 3 3 0* 1 1 2 \n", + "- Q - . - . - . 1 0* 3 3 2 2 2 2 \n", + ". - . - . Q . - 3 2 3 2 2 0* 3 2 \n", + "Q . - . - . - . 0* 3 1 2 3 3 3 3 \n", + ". - . - . - Q - 2 2 2 2 3 3 0* 2 \n", + "- . - Q - . - . 2 1 3 0* 2 3 2 2 \n", + ". - . - . - . Q 1 3 2 3 3 1 2 0* \n", + "- . Q . - . - . 2 2 0* 2 2 2 2 2 \n" + ] + } + ], + "source": [ + "backtracking_search(chess, select_unassigned_variable=mrv, inference=forward_checking)\n", + "chess.display(chess.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the experiments below on n-ary CSPs, due to the n-ary constraints, the `GAC` algorithm was used.
\n", + "The `GAC` algorithm has $\\mathcal{O}(er^2d^t)$ time-complexity and $\\mathcal{O}(erd)$ space-complexity where $e$ denotes the number of n-ary constraints, $r$ denotes the constraint arity and $d$ denotes the maximum domain-size of the variables." + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + " \u001b[0;32mdef\u001b[0m \u001b[0mGAC\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morig_domains\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msat_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Makes this CSP arc-consistent using Generalized Arc Consistency\u001b[0m\n", + "\u001b[0;34m orig_domains is the original domains\u001b[0m\n", + "\u001b[0;34m to_do is a set of (variable,constraint) pairs\u001b[0m\n", + "\u001b[0;34m returns the reduced domains (an arc-consistent variable:domain dictionary)\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0morig_domains\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0morig_domains\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdomains\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mconst\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0morig_domains\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mother_vars\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mov\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mov\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mov\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mother_vars\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mholds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if const.holds({var: val})}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mother_vars\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mother\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mother_vars\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mother_val\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mother\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mholds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mother\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mother_val\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if any(const.holds({var: val, other: other_val})\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# for other_val in domains[other])}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# general case\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mholds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0many_holds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mother_vars\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mholds\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if self.any_holds(domains, const, {var: val}, other_vars)}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_domain\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0madd_to_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnew_to_do\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m|=\u001b[0m \u001b[0madd_to_do\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource ACSolver.GAC" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Crossword" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[_] [_] [_] [*] [*] \n", + "[_] [*] [_] [*] [*] \n", + "[_] [_] [_] [_] [*] \n", + "[_] [*] [_] [*] [*] \n", + "[*] [*] [_] [_] [_] \n", + "[*] [*] [_] [*] [*] \n" + ] + }, + { + "data": { + "text/plain": [ + "{'ant',\n", + " 'big',\n", + " 'book',\n", + " 'bus',\n", + " 'buys',\n", + " 'car',\n", + " 'ginger',\n", + " 'has',\n", + " 'hold',\n", + " 'lane',\n", + " 'search',\n", + " 'symbol',\n", + " 'syntax',\n", + " 'year'}" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "crossword = Crossword(crossword1, words1)\n", + "crossword.display()\n", + "words1" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1min 20s, sys: 2.02 ms, total: 1min 20s\n", + "Wall time: 1min 20s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 64617645 consistency-checks'" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(crossword).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.19 s, sys: 0 ns, total: 1.19 s\n", + "Wall time: 1.19 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 908015 consistency-checks'" + ] + }, + "execution_count": 42, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "crossword = Crossword(crossword1, words1)\n", + "%time _, _, checks = ACSolver(crossword).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[B] [U] [S] [*] [*] \n", + "[U] [*] [E] [*] [*] \n", + "[Y] [E] [A] [R] [*] \n", + "[S] [*] [R] [*] [*] \n", + "[*] [*] [C] [A] [R] \n", + "[*] [*] [H] [*] [*] \n" + ] + } + ], + "source": [ + "crossword.display(ACSolver(crossword).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Kakuro" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Easy Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t10\\\t13\\\t[*]\t\n", + "\\3\t[_]\t[_]\t13\\\t\n", + "\\12\t[_]\t[_]\t[_]\t\n", + "\\21\t[_]\t[_]\t[_]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro2)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.8 ms, sys: 171 µs, total: 18 ms\n", + "Wall time: 16.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 2752 consistency-checks'" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 8.55 ms, sys: 0 ns, total: 8.55 ms\n", + "Wall time: 8.39 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 1765 consistency-checks'" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro2)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t10\\\t13\\\t[*]\t\n", + "\\3\t[1]\t[2]\t13\\\t\n", + "\\12\t[5]\t[3]\t[4]\t\n", + "\\21\t[4]\t[8]\t[9]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Medium Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t17\\\t28\\\t[*]\t42\\\t22\\\t\n", + "\\9\t[_]\t[_]\t31\\14\t[_]\t[_]\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t\\30\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t22\\24\t[_]\t[_]\t[_]\t[*]\t\n", + "\\25\t[_]\t[_]\t[_]\t[_]\t11\\\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "\\14\t[_]\t[_]\t\\17\t[_]\t[_]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro3)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.96 s, sys: 0 ns, total: 1.96 s\n", + "Wall time: 1.96 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 1290179 consistency-checks'" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 225 ms, sys: 0 ns, total: 225 ms\n", + "Wall time: 223 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 148780 consistency-checks'" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro3)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t17\\\t28\\\t[*]\t42\\\t22\\\t\n", + "\\9\t[8]\t[1]\t31\\14\t[5]\t[9]\t\n", + "\\20\t[9]\t[2]\t[1]\t[3]\t[5]\t\n", + "[*]\t\\30\t[6]\t[9]\t[7]\t[8]\t\n", + "[*]\t22\\24\t[7]\t[8]\t[9]\t[*]\t\n", + "\\25\t[8]\t[4]\t[7]\t[6]\t11\\\t\n", + "\\20\t[5]\t[3]\t[6]\t[4]\t[2]\t\n", + "\\14\t[9]\t[5]\t\\17\t[8]\t[9]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Harder Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t[*]\t[*]\t[*]\t[*]\t4\\\t24\\\t11\\\t[*]\t[*]\t[*]\t11\\\t17\\\t[*]\t[*]\t\n", + "[*]\t[*]\t[*]\t17\\\t11\\12\t[_]\t[_]\t[_]\t[*]\t[*]\t24\\10\t[_]\t[_]\t11\\\t[*]\t\n", + "[*]\t4\\\t16\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t\\20\t[_]\t[_]\t[_]\t[_]\t16\\\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t24\\13\t[_]\t[_]\t16\\\t\\12\t[_]\t[_]\t23\\10\t[_]\t[_]\t\n", + "\\10\t[_]\t[_]\t24\\12\t[_]\t[_]\t16\\5\t[_]\t[_]\t16\\30\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t[*]\t3\\26\t[_]\t[_]\t[_]\t[_]\t\\12\t[_]\t[_]\t4\\\t16\\14\t[_]\t[_]\t[*]\t\n", + "[*]\t\\8\t[_]\t[_]\t\\15\t[_]\t[_]\t34\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t[*]\t\n", + "[*]\t\\11\t[_]\t[_]\t3\\\t17\\\t\\14\t[_]\t[_]\t\\8\t[_]\t[_]\t7\\\t17\\\t[*]\t\n", + "[*]\t[*]\t[*]\t23\\10\t[_]\t[_]\t3\\9\t[_]\t[_]\t4\\\t23\\\t\\13\t[_]\t[_]\t[*]\t\n", + "[*]\t[*]\t10\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t\\7\t[_]\t[_]\t30\\9\t[_]\t[_]\t[*]\t\n", + "[*]\t17\\11\t[_]\t[_]\t11\\\t24\\8\t[_]\t[_]\t11\\21\t[_]\t[_]\t[_]\t[_]\t16\\\t17\\\t\n", + "\\29\t[_]\t[_]\t[_]\t[_]\t[_]\t\\7\t[_]\t[_]\t23\\14\t[_]\t[_]\t3\\17\t[_]\t[_]\t\n", + "\\10\t[_]\t[_]\t3\\10\t[_]\t[_]\t[*]\t\\8\t[_]\t[_]\t4\\25\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t\\16\t[_]\t[_]\t[_]\t[_]\t[*]\t\\23\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t[*]\t\n", + "[*]\t[*]\t\\6\t[_]\t[_]\t[*]\t[*]\t\\15\t[_]\t[_]\t[_]\t[*]\t[*]\t[*]\t[*]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro4)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 76.5 ms, sys: 847 µs, total: 77.4 ms\n", + "Wall time: 77 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 46633 consistency-checks'" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC()\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 64.6 ms, sys: 0 ns, total: 64.6 ms\n", + "Wall time: 63.6 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 36828 consistency-checks'" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro4)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t[*]\t[*]\t[*]\t[*]\t4\\\t24\\\t11\\\t[*]\t[*]\t[*]\t11\\\t17\\\t[*]\t[*]\t\n", + "[*]\t[*]\t[*]\t17\\\t11\\12\t[3]\t[7]\t[2]\t[*]\t[*]\t24\\10\t[2]\t[8]\t11\\\t[*]\t\n", + "[*]\t4\\\t16\\26\t[8]\t[5]\t[1]\t[9]\t[3]\t[*]\t\\20\t[8]\t[1]\t[9]\t[2]\t16\\\t\n", + "\\20\t[3]\t[7]\t[9]\t[1]\t24\\13\t[8]\t[5]\t16\\\t\\12\t[9]\t[3]\t23\\10\t[3]\t[7]\t\n", + "\\10\t[1]\t[9]\t24\\12\t[3]\t[9]\t16\\5\t[1]\t[4]\t16\\30\t[7]\t[5]\t[8]\t[1]\t[9]\t\n", + "[*]\t[*]\t3\\26\t[8]\t[2]\t[7]\t[9]\t\\12\t[3]\t[9]\t4\\\t16\\14\t[9]\t[5]\t[*]\t\n", + "[*]\t\\8\t[1]\t[7]\t\\15\t[8]\t[7]\t34\\26\t[1]\t[7]\t[3]\t[9]\t[6]\t[*]\t[*]\t\n", + "[*]\t\\11\t[2]\t[9]\t3\\\t17\\\t\\14\t[8]\t[6]\t\\8\t[1]\t[7]\t7\\\t17\\\t[*]\t\n", + "[*]\t[*]\t[*]\t23\\10\t[1]\t[9]\t3\\9\t[7]\t[2]\t4\\\t23\\\t\\13\t[4]\t[9]\t[*]\t\n", + "[*]\t[*]\t10\\26\t[6]\t[2]\t[8]\t[1]\t[9]\t\\7\t[1]\t[6]\t30\\9\t[1]\t[8]\t[*]\t\n", + "[*]\t17\\11\t[3]\t[8]\t11\\\t24\\8\t[2]\t[6]\t11\\21\t[3]\t[9]\t[7]\t[2]\t16\\\t17\\\t\n", + "\\29\t[8]\t[2]\t[9]\t[3]\t[7]\t\\7\t[4]\t[3]\t23\\14\t[8]\t[6]\t3\\17\t[9]\t[8]\t\n", + "\\10\t[9]\t[1]\t3\\10\t[2]\t[8]\t[*]\t\\8\t[2]\t[6]\t4\\25\t[8]\t[1]\t[7]\t[9]\t\n", + "[*]\t\\16\t[4]\t[2]\t[1]\t[9]\t[*]\t\\23\t[1]\t[8]\t[3]\t[9]\t[2]\t[*]\t[*]\t\n", + "[*]\t[*]\t\\6\t[1]\t[5]\t[*]\t[*]\t\\15\t[5]\t[9]\t[1]\t[*]\t[*]\t[*]\t[*]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Cryptarithmetic Puzzle" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{array}{@{}r@{}}\n", + " S E N D \\\\\n", + "{} + M O R E \\\\\n", + " \\hline\n", + " M O N E Y\n", + "\\end{array}\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "pycharm": {} + }, + "outputs": [], + "source": [ + "cryptarithmetic = NaryCSP(\n", + " {'S': set(range(1, 10)), 'M': set(range(1, 10)),\n", + " 'E': set(range(0, 10)), 'N': set(range(0, 10)), 'D': set(range(0, 10)),\n", + " 'O': set(range(0, 10)), 'R': set(range(0, 10)), 'Y': set(range(0, 10)),\n", + " 'C1': set(range(0, 2)), 'C2': set(range(0, 2)), 'C3': set(range(0, 2)),\n", + " 'C4': set(range(0, 2))},\n", + " [Constraint(('S', 'E', 'N', 'D', 'M', 'O', 'R', 'Y'), all_diff),\n", + " Constraint(('D', 'E', 'Y', 'C1'), lambda d, e, y, c1: d + e == y + 10 * c1),\n", + " Constraint(('N', 'R', 'E', 'C1', 'C2'), lambda n, r, e, c1, c2: c1 + n + r == e + 10 * c2),\n", + " Constraint(('E', 'O', 'N', 'C2', 'C3'), lambda e, o, n, c2, c3: c2 + e + o == n + 10 * c3),\n", + " Constraint(('S', 'M', 'O', 'C3', 'C4'), lambda s, m, o, c3, c4: c3 + s + m == o + 10 * c4),\n", + " Constraint(('M', 'C4'), eq)])" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 21.7 s, sys: 0 ns, total: 21.7 s\n", + "Wall time: 21.7 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 14080592 consistency-checks'" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(cryptarithmetic).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 939 ms, sys: 0 ns, total: 939 ms\n", + "Wall time: 938 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 573120 consistency-checks'" + ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(cryptarithmetic).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/latex": [ + "\\begin{array}{@{}r@{}} 9567 \\\\ + 1085 \\\\ \\hline 10652 \\end{array}" + ], + "text/plain": [ + "
class CSP(search.Problem):\n",
+ " """This class describes finite-domain Constraint Satisfaction Problems.\n",
+ " A CSP is specified by the following inputs:\n",
+ " variables A list of variables; each is atomic (e.g. int or string).\n",
+ " domains A dict of {var:[possible_value, ...]} entries.\n",
+ " neighbors A dict of {var:[var,...]} that for each variable lists\n",
+ " the other variables that participate in constraints.\n",
+ " constraints A function f(A, a, B, b) that returns true if neighbors\n",
+ " A, B satisfy the constraint when they have values A=a, B=b\n",
+ "\n",
+ " In the textbook and in most mathematical definitions, the\n",
+ " constraints are specified as explicit pairs of allowable values,\n",
+ " but the formulation here is easier to express and more compact for\n",
+ " most cases. (For example, the n-Queens problem can be represented\n",
+ " in O(n) space using this notation, instead of O(N^4) for the\n",
+ " explicit representation.) In terms of describing the CSP as a\n",
+ " problem, that's all there is.\n",
+ "\n",
+ " However, the class also supports data structures and methods that help you\n",
+ " solve CSPs by calling a search function on the CSP. Methods and slots are\n",
+ " as follows, where the argument 'a' represents an assignment, which is a\n",
+ " dict of {var:val} entries:\n",
+ " assign(var, val, a) Assign a[var] = val; do other bookkeeping\n",
+ " unassign(var, a) Do del a[var], plus other bookkeeping\n",
+ " nconflicts(var, val, a) Return the number of other variables that\n",
+ " conflict with var=val\n",
+ " curr_domains[var] Slot: remaining consistent values for var\n",
+ " Used by constraint propagation routines.\n",
+ " The following methods are used only by graph_search and tree_search:\n",
+ " actions(state) Return a list of actions\n",
+ " result(state, action) Return a successor of state\n",
+ " goal_test(state) Return true if all constraints satisfied\n",
+ " The following are just for debugging purposes:\n",
+ " nassigns Slot: tracks the number of assignments made\n",
+ " display(a) Print a human-readable representation\n",
+ " """\n",
+ "\n",
+ " def __init__(self, variables, domains, neighbors, constraints):\n",
+ " """Construct a CSP problem. If variables is empty, it becomes domains.keys()."""\n",
+ " variables = variables or list(domains.keys())\n",
+ " self.variables = variables\n",
+ " self.domains = domains\n",
+ " self.neighbors = neighbors\n",
+ " self.constraints = constraints\n",
+ " self.initial = ()\n",
+ " self.curr_domains = None\n",
+ " self.nassigns = 0\n",
+ "\n",
+ " def assign(self, var, val, assignment):\n",
+ " """Add {var: val} to assignment; Discard the old value if any."""\n",
+ " assignment[var] = val\n",
+ " self.nassigns += 1\n",
+ "\n",
+ " def unassign(self, var, assignment):\n",
+ " """Remove {var: val} from assignment.\n",
+ " DO NOT call this if you are changing a variable to a new value;\n",
+ " just call assign for that."""\n",
+ " if var in assignment:\n",
+ " del assignment[var]\n",
+ "\n",
+ " def nconflicts(self, var, val, assignment):\n",
+ " """Return the number of conflicts var=val has with other variables."""\n",
+ "\n",
+ " # Subclasses may implement this more efficiently\n",
+ " def conflict(var2):\n",
+ " return (var2 in assignment and\n",
+ " not self.constraints(var, val, var2, assignment[var2]))\n",
+ "\n",
+ " return count(conflict(v) for v in self.neighbors[var])\n",
+ "\n",
+ " def display(self, assignment):\n",
+ " """Show a human-readable representation of the CSP."""\n",
+ " # Subclasses can print in a prettier way, or display with a GUI\n",
+ " print('CSP:', self, 'with assignment:', assignment)\n",
+ "\n",
+ " # These methods are for the tree and graph-search interface:\n",
+ "\n",
+ " def actions(self, state):\n",
+ " """Return a list of applicable actions: nonconflicting\n",
+ " assignments to an unassigned variable."""\n",
+ " if len(state) == len(self.variables):\n",
+ " return []\n",
+ " else:\n",
+ " assignment = dict(state)\n",
+ " var = first([v for v in self.variables if v not in assignment])\n",
+ " return [(var, val) for val in self.domains[var]\n",
+ " if self.nconflicts(var, val, assignment) == 0]\n",
+ "\n",
+ " def result(self, state, action):\n",
+ " """Perform an action and return the new state."""\n",
+ " (var, val) = action\n",
+ " return state + ((var, val),)\n",
+ "\n",
+ " def goal_test(self, state):\n",
+ " """The goal is to assign all variables, with all constraints satisfied."""\n",
+ " assignment = dict(state)\n",
+ " return (len(assignment) == len(self.variables)\n",
+ " and all(self.nconflicts(variables, assignment[variables], assignment) == 0\n",
+ " for variables in self.variables))\n",
+ "\n",
+ " # These are for constraint propagation\n",
+ "\n",
+ " def support_pruning(self):\n",
+ " """Make sure we can prune values from domains. (We want to pay\n",
+ " for this only if we use it.)"""\n",
+ " if self.curr_domains is None:\n",
+ " self.curr_domains = {v: list(self.domains[v]) for v in self.variables}\n",
+ "\n",
+ " def suppose(self, var, value):\n",
+ " """Start accumulating inferences from assuming var=value."""\n",
+ " self.support_pruning()\n",
+ " removals = [(var, a) for a in self.curr_domains[var] if a != value]\n",
+ " self.curr_domains[var] = [value]\n",
+ " return removals\n",
+ "\n",
+ " def prune(self, var, value, removals):\n",
+ " """Rule out var=value."""\n",
+ " self.curr_domains[var].remove(value)\n",
+ " if removals is not None:\n",
+ " removals.append((var, value))\n",
+ "\n",
+ " def choices(self, var):\n",
+ " """Return all values for var that aren't currently ruled out."""\n",
+ " return (self.curr_domains or self.domains)[var]\n",
+ "\n",
+ " def infer_assignment(self):\n",
+ " """Return the partial assignment implied by the current inferences."""\n",
+ " self.support_pruning()\n",
+ " return {v: self.curr_domains[v][0]\n",
+ " for v in self.variables if 1 == len(self.curr_domains[v])}\n",
+ "\n",
+ " def restore(self, removals):\n",
+ " """Undo a supposition and all inferences from it."""\n",
+ " for B, b in removals:\n",
+ " self.curr_domains[B].append(b)\n",
+ "\n",
+ " # This is for min_conflicts search\n",
+ "\n",
+ " def conflicted_vars(self, current):\n",
+ " """Return a list of variables in current assignment that are in conflict"""\n",
+ " return [var for var in self.variables\n",
+ " if self.nconflicts(var, current[var], current) > 0]\n",
+ "def different_values_constraint(A, a, B, b):\n",
+ " """A constraint saying two neighboring variables must differ in value."""\n",
+ " return a != b\n",
+ "def MapColoringCSP(colors, neighbors):\n",
+ " """Make a CSP for the problem of coloring a map with different colors\n",
+ " for any two adjacent regions. Arguments are a list of colors, and a\n",
+ " dict of {region: [neighbor,...]} entries. This dict may also be\n",
+ " specified as a string of the form defined by parse_neighbors."""\n",
+ " if isinstance(neighbors, str):\n",
+ " neighbors = parse_neighbors(neighbors)\n",
+ " return CSP(list(neighbors.keys()), UniversalDict(colors), neighbors,\n",
+ " different_values_constraint)\n",
+ "def queen_constraint(A, a, B, b):\n",
+ " """Constraint is satisfied (true) if A, B are really the same variable,\n",
+ " or if they are not in the same row, down diagonal, or up diagonal."""\n",
+ " return A == B or (a != b and A + a != B + b and A - a != B - b)\n",
+ "class NQueensCSP(CSP):\n",
+ " """Make a CSP for the nQueens problem for search with min_conflicts.\n",
+ " Suitable for large n, it uses only data structures of size O(n).\n",
+ " Think of placing queens one per column, from left to right.\n",
+ " That means position (x, y) represents (var, val) in the CSP.\n",
+ " The main structures are three arrays to count queens that could conflict:\n",
+ " rows[i] Number of queens in the ith row (i.e val == i)\n",
+ " downs[i] Number of queens in the \\ diagonal\n",
+ " such that their (x, y) coordinates sum to i\n",
+ " ups[i] Number of queens in the / diagonal\n",
+ " such that their (x, y) coordinates have x-y+n-1 = i\n",
+ " We increment/decrement these counts each time a queen is placed/moved from\n",
+ " a row/diagonal. So moving is O(1), as is nconflicts. But choosing\n",
+ " a variable, and a best value for the variable, are each O(n).\n",
+ " If you want, you can keep track of conflicted variables, then variable\n",
+ " selection will also be O(1).\n",
+ " >>> len(backtracking_search(NQueensCSP(8)))\n",
+ " 8\n",
+ " """\n",
+ "\n",
+ " def __init__(self, n):\n",
+ " """Initialize data structures for n Queens."""\n",
+ " CSP.__init__(self, list(range(n)), UniversalDict(list(range(n))),\n",
+ " UniversalDict(list(range(n))), queen_constraint)\n",
+ "\n",
+ " self.rows = [0] * n\n",
+ " self.ups = [0] * (2 * n - 1)\n",
+ " self.downs = [0] * (2 * n - 1)\n",
+ "\n",
+ " def nconflicts(self, var, val, assignment):\n",
+ " """The number of conflicts, as recorded with each assignment.\n",
+ " Count conflicts in row and in up, down diagonals. If there\n",
+ " is a queen there, it can't conflict with itself, so subtract 3."""\n",
+ " n = len(self.variables)\n",
+ " c = self.rows[val] + self.downs[var + val] + self.ups[var - val + n - 1]\n",
+ " if assignment.get(var, None) == val:\n",
+ " c -= 3\n",
+ " return c\n",
+ "\n",
+ " def assign(self, var, val, assignment):\n",
+ " """Assign var, and keep track of conflicts."""\n",
+ " oldval = assignment.get(var, None)\n",
+ " if val != oldval:\n",
+ " if oldval is not None: # Remove old val if there was one\n",
+ " self.record_conflict(assignment, var, oldval, -1)\n",
+ " self.record_conflict(assignment, var, val, +1)\n",
+ " CSP.assign(self, var, val, assignment)\n",
+ "\n",
+ " def unassign(self, var, assignment):\n",
+ " """Remove var from assignment (if it is there) and track conflicts."""\n",
+ " if var in assignment:\n",
+ " self.record_conflict(assignment, var, assignment[var], -1)\n",
+ " CSP.unassign(self, var, assignment)\n",
+ "\n",
+ " def record_conflict(self, assignment, var, val, delta):\n",
+ " """Record conflicts caused by addition or deletion of a Queen."""\n",
+ " n = len(self.variables)\n",
+ " self.rows[val] += delta\n",
+ " self.downs[var + val] += delta\n",
+ " self.ups[var - val + n - 1] += delta\n",
+ "\n",
+ " def display(self, assignment):\n",
+ " """Print the queens and the nconflicts values (for debugging)."""\n",
+ " n = len(self.variables)\n",
+ " for val in range(n):\n",
+ " for var in range(n):\n",
+ " if assignment.get(var, '') == val:\n",
+ " ch = 'Q'\n",
+ " elif (var + val) % 2 == 0:\n",
+ " ch = '.'\n",
+ " else:\n",
+ " ch = '-'\n",
+ " print(ch, end=' ')\n",
+ " print(' ', end=' ')\n",
+ " for var in range(n):\n",
+ " if assignment.get(var, '') == val:\n",
+ " ch = '*'\n",
+ " else:\n",
+ " ch = ' '\n",
+ " print(str(self.nconflicts(var, val, assignment)) + ch, end=' ')\n",
+ " print()\n",
+ "\n", + "In the start, all the variables of the CSP are _randomly_ initialized. \n", + "
\n", + "The algorithm then randomly selects a variable that has conflicts and violates some constraints of the CSP.\n", + "
\n", + "The selected variable is then assigned a value that _minimizes_ the number of conflicts.\n", + "
\n", + "This is a simple **stochastic algorithm** which works on a principle similar to **Hill-climbing**.\n", + "The conflicting state is repeatedly changed into a state with fewer conflicts in an attempt to reach an approximate solution.\n", + "
\n", + "This algorithm sometimes benefits from having a good initial assignment.\n", + "Using greedy techniques to get a good initial assignment and then using `min_conflicts` to solve the CSP can speed up the procedure dramatically, especially for CSPs with a large state space." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def min_conflicts(csp, max_steps=100000):\n",
+ " """Solve a CSP by stochastic Hill Climbing on the number of conflicts."""\n",
+ " # Generate a complete assignment for all variables (probably with conflicts)\n",
+ " csp.current = current = {}\n",
+ " for var in csp.variables:\n",
+ " val = min_conflicts_value(csp, var, current)\n",
+ " csp.assign(var, val, current)\n",
+ " # Now repeatedly choose a random conflicted variable and change it\n",
+ " for i in range(max_steps):\n",
+ " conflicted = csp.conflicted_vars(current)\n",
+ " if not conflicted:\n",
+ " return current\n",
+ " var = random.choice(conflicted)\n",
+ " val = min_conflicts_value(csp, var, current)\n",
+ " csp.assign(var, val, current)\n",
+ " return None\n",
+ "\n", + "`notebook.py` has a helper function to visualize the solution space." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHwCAYAAADjD7WGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df7QU5Z3v+8932Aii/BDYoALXYJKVs+YYMdKjzqBcYkgICEbvmZuBa8zR3FzOzT2GEHEyI2tlxWSdxBwViBNzJydHBzwnKppxjKgTJTGCASNOwygzmpm7HDURCT+2sAO6TQTmuX/U7tndvetXd1V1d1W/X2vt1d1VTz31bZ+9/fI89TxV5pwTAADI3u+1OwAAALoFSRcAgBYh6QIA0CIkXQAAWoSkCwBAi5B0AQBoEZIuAAAtQtIFAKBFSLpAC5jZe8zsb83ssJntM7M7zKwnpPwEM/vLwbIDZvYPZvYfWxkzgPSRdIHW+H8lHZB0hqTzJP2vkv4fv4JmdpKkn0g6S9IfShov6U8l3WJmK1oSLYBMkHSB1pgp6QHn3G+dc/skPS7p3weUvVrS/yLpf3fOveqcO+ace1zSCkn/xcxOlSQzc2b2vspBZrbBzP5L1efFZva8mfWb2TNmdm7VvjPN7EEzO2hmr1YnczO7ycweMLP/YWZHzexFMytV7f8zM3tjcN8/m9lH0vlPBBQfSRdojW9JWmpmY8xsmqSF8hKvn49K+pFz7u267Q9KGiOv9xvKzD4k6a8k/SdJkyT9N0mbzGyUmf2epEckvSBpmqSPSFppZguqqrhc0kZJEyRtknTHYL0fkHSdpD9wzo2VtEDSa1HxAPCQdIHWeFpez/aIpD2SypJ+GFB2sqRf1290zh2X1CepN8b5lkv6b865Hc65E865uyX9TtJFkv5AUq9z7mvOuXedc69I+u+SllYdv80597fOuROS/qekWYPbT0gaJen3zWykc+4159y/xIgHgEi6QOYGe5aPS/obSafIS6qnSfqvAYf0ybv2W19Pz+CxfTFOe5akVYNDy/1m1i9phqQzB/edWbdvtaSpVcfvq3o/IGm0mfU4516WtFLSTZIOmNlGMzszRjwARNIFWmGivGu0dzjnfuece1PSekmLAsr/RNJCMzulbvt/kPSupB2DnwfkDTdXnF71/nVJX3fOTaj6GeOcu29w36t1+8Y654LiqeGcu9c5d7G85O0U/I8HAHVIukDGnHN9kl6V9Dkz6zGzCZL+o6TdAYf8T3lD0D8YXGo0cvB6619IutU595vBcs9L+j/MbISZfVzejOiK/y7p/zazC81zipldZmZjJT0n6ejghKiTB48/x8z+IOq7mNkHzOxSMxsl6beS3pH0rw3/RwG6FEkXaI3/TdLHJR2U9LKkY5K+6FfQOfc7SfPl9Uh3yEtsj8ubjPXVqqJfkLREUr+kq1R1jdg5V5b0f8mbAHV48JzXDO47IWmxvKVLr8obrr5T3tKkKKMkfXPwmH2Spki6McZxACSZc67dMQAIYWYjJf1I0huSrnH80QK5RU8X6HDOuWPyruf+i6QPtDkcAAnQ0wUAoEXo6QIA0CKBN1xPYvLkye4973lPFlV3hJ07d7Y7hEzNnj273SFkjjbMN9ov/4rehs4589ueyfByqVRy5XI59Xo7hZnvf8vCSO13YmcK/51mZ3P5gzbMN9ov/7qgDX2/IMPLSNf+W71km0bClYbq2r8mnfoAoI1IukjHsTe95LjnS9nUv+cGr/5j+7OpHwBaIJNruugyafVq49g9eKfDjIadASBL9HSRTCsTbiecFwASIOmiObtGtT/x7TTp0Mb2xgAADSDponE7TXLvJq7multSiOXVZe1P/gAQE9d00ZhdoxNXYaWh9995wHt1SVeY7Rolnf+7hJUAQLbo6aIxLjqx9c6X7vmR/77qhBtne2wp9LwBIGskXcQXMYxrJe+nr1/61JeTJ9JKfZWfcz6ZLD4AaDeSLuKJSGjfvt9/e7OJ1++4F1+JcSCJF0AHI+ki2vEDkUVW3NqCOBQziR/vyzwOAGgGSRfRXpiaWlVBE6YST6Sq9kJvipUBQHqYvYxwvx5a1+PXy6wkS1eOP5TsytLRAWncXOnI09LYMfHDWf+Vofdh8WjfOun0L8avGABagJ4uwu39M0nBCXVP1cjznFnD9wf1YCuJNijhBh13zRLv9Vf7/Pf/W5xvXO9fAADaiKSLRGYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixMAOgFJF8ESzgR+I2T+1cuve6+HjgSXCdsXCzOZAXQYki4SWTQneN/0RcH74gjrBS++JFndANAOJF3EMrDdf/tjt7c2jopH1vlvf+eZ1sYBAI0g6cLfsdqZSieP8q6pnjxqaFucZT4bHmnu9A9vjS5Tff4xo73Po0+qK3TsYHMBAEAGSLrwt/sM380D26VjO7z3cZYIXfvV4duOn6j93Nc/vMwVq6Lrrpy/f4v09raAQrunRFcEAC1C0kXDekYkO/6ki2o/985PVt/4U5MdDwCtQtJFInF6u0tX1352Lrz8Z76WznkBoNOQdJG5+zc3Vn79pmziAIB2i5V0zezjZvbPZvaymf151kGh/a5fG79sq3udjZyvke8BAFmLTLpmNkLSdyQtlPT7kpaZ2e9nHRjaa23Kd1H83M3xyqX9tKK0vwcAJBGnp3uBpJedc684596VtFHSJ7INC3mzeGX4/u8+6L1u3eW/f9PT3mvQc3kr6mc1f/qy6NgAoFPESbrTJL1e9XnP4LYaZrbczMpmVj54kLWRRTfzzNrPjwUt2akzb7n/9k/E7JHWr9+922dJEgB0qtQmUjnnvuecKznnSr29PM+06H525/BtC1eEHzMx5LaOknTah8P3r1wTvh8AOl2cpPuGpBlVn6cPbkORzQofrZjmc8+JxyNuwXg44gEG/UfD999+X/h+X+f2NXEQAGQjTtL9O0nvN7OZZnaSpKWSWNRRdD2Tmzosq5nMV97Q5IEjJ6UaBwAk0RNVwDl33Myuk/SEpBGS/so592LmkQFVfril3REAQHKRSVeSnHN/K+lvM44FOTN1orT/UPvOf+E57Ts3ADSDO1Ih2Ozw+zXua/BOU9U++D5p/gXSe6c3X8ezGyIKRMQPAK0Wq6cLBHHl4Ou4i+Yke97uguukzc8GnxcA8oaki3DTb5P2hM9i6t8iTZjnvd+/WZoysXb/NTdJdz8a/5RzZknb7pKeuGNo26t7pbMv997H6mHP+Iv4JwSAFjEX9ciXJpRKJVcuF7crYmbtDiFTw34ndkZ/XysN9T43bpaWrQ4v34h7vy4tWzD8PKEihpa7rg0LhvbLvy5oQ98vSNJtQhf8stRuOHYw1sPg4y4XWjJXunaJNG+2dPio9PPd0jfWSy+9EiO2OL9W5/ZFLhXqujYsGNov/7qgDX2/IMPLiDay+TuMbVrrJdkgp42Tzp4mXbWwdvu256VLPtvkSVmbC6BDkXQRz2wXOcxcmVQ1skd6t24CVCM3zXBl6eLzhnq1Iy+Ujp9IZ1gZANqJpIv4YiReaSjhNnt3qurjTjwnHdsRsy4SLoAOxzpdNGZm9A2QrRScJG9aLh1+yuu1Vn4Gtnvb/Yy4IGbCnfmDGIUAoL2YSNWELpgAEF4goLdbnxyvmCc9dFvzcSxb7c2Erokt6NeqwV5u17dhztF++dcFbcjs5bR0wS9LdKFdYyT3Ts0mK0l9T0qTxtcWHTtXemsg/vknjpPe/Gnttm9ukG68wyfpzrxPmrg0fuWVWGnDXKP98q8L2pDZy0jR+YNZtK7X2zNCmnm59Nre5qs+dKS21/zLR4f3eCVxDRdA7nBNF8lUJT5Xlh7emizh+jlrsbeut6aXS8IFkEMMLzehC4ZFGj/o2CFpdwvWx557ING64QraMN9ov/zrgjb0/YL0dJGOkRO93ueMddnUP+N2r/4UEi4AtAvXdJGuKSu9HynWmt5IDCMDKBB6usjObDf0M+vwsN2r/DrF5/669jgAKBB6umiNngnDkuia77cpFgBoE3q6AAC0CEkXAIAWIekCANAiJF0AAFokk4lUO3fuLPTC56IvXC9y21XQhvlG++VfkduwVAp+NBqzlwGg4sRh6fmJNZtWrZPWfLGu3Ll7pZFntC4uFAZJF0B3i7iJy7CEK0m7z6z9zJpyxMQ1XQDdZ/+tXrJN465p0lBd+9ekUx8Ki6QLoHsce9NLjnu+lE39e27w6j+2P5v6kXsMLwPoDmn1auPYfbr3yrAz6tDTBVB8rUy4nXBedCySLoDi2jWq/Ylvp0mHNrY3BnQMki6AYtppkns3cTXX3ZJCLK8ua3/yR0fgmi6A4tk1OnEVVnV/g+884L26csJKd42Szv9dwkqQZ/R0ARSPi05svfOle37kv88CbigUtD22FHreyDeSLoBiiRjGtZL309cvferLyRNppb7KzzmfTBYfio2kC6A4IhLat+/3395s4vU77sVXYhxI4u1aJF0AxXD8QGSRFbe2IA7FTOLH+zKPA52HpAugGF6YmlpVQROmEk+kqvZCb4qVIS+YvQwg/349tK7Hr5dZSZauHH8o2ZWlowPSuLnSkaelsWPih7P+K0Pvw+LRvnXS6X5PVEBR0dMFkH97/0xScELdUzXyPGfW8P1BPdhKog1KuEHHXbPEe/3VPv/9/xbnG9f7F0BhkXQBFN6MRUPvt91VmyzDhozff6X3OunS4DL1dVV/PmtxY3Gi+Ei6APIt4UzgN0LmX738uvd66EhwmbB9sTCTuauQdAEU3qI5wfumLwreF0dYL3jxJcnqRvGQdAEUxsB2/+2P3d7aOCoeWee//Z1nWhsHOgdJF0B+HaudqXTyKO+a6smjhrbFWeaz4ZHmTv/w1ugy1ecfM9r7PPqkukLHDjYXAHKHpAsgv3af4bt5YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5YlV03ZXz92+R3t4WUGj3lOiKUAgkXQCF1DMi2fEnXVT7uXd+svrGn5rseBQDSRdA4cXp7S5dXfvZufDyn/laOudFd4lMumb2V2Z2wMz+sRUBAUA73L+5sfLrN2UTB4otTk93g6SPZxwHADTs+rXxy7a619nI+Rr5Hsi3yKTrnHta0qEWxAIADVmb8l0UP3dzvHJpP60o7e+BzsU1XQBdY/HK8P3ffdB73brLf/+mp73XoOfyVtTPav70ZdGxoTuklnTNbLmZlc0szYdfAUDTZp5Z+/mxoCU7deYt99/+iZg90vr1u3f7LElCd0ot6TrnvuecKznnmK8HoCP87M7h2xauCD9mYshtHSXptA+H71+5Jnw/uhvDywDya1b4nZym+dxz4vGIWzAejniAQf/R8P233xe+39e5fU0chDyKs2ToPkk/l/QBM9tjZv9n9mEBQAw9k5s6LKuZzFfe0OSBIyelGgc6V09UAefcslYEAgB598Mt7Y4AnY7hZQCFNnVie89/4TntPT86C0kXQL7NDr9f474G7zRV7YPvk+ZfIL13evN1PLshokBE/CiWyOFlAMg7Vw6+jrtoTrLn7S64Ttr8bPB5gWokXQD5N/02aU/4LKb+LdKEed77/ZulKXXDztfcJN39aPxTzpklbbtLeuKOoW2v7pXOvtx7H6uHPeMv4p8QhWAu6lEazVRqVujxkiz+m3USM2t3CJmjDfPNt/12Rn9nKw31PjdulpatDi/fiHu/Li1bMPw8oQKGloveflKx/wZLpZLK5bJvI5J0m1DkXxaJP/giKHob+rbfsYOxHgYfd7nQkrnStUukebOlw0eln++WvrFeeumVGPHFSbjn9gUuFSp6+0nF/hsMS7oMLwMohpG9TR+6aa2XZIOcNk46e5p01cLa7duely75bJMnZW1uVyLpAiiO2S5ymLkyqWpkj/Ru3QSoRm6a4crSxecN9WpHXigdP5FsWBnFR9IFUCwxEq80lHCbvTtV9XEnnpOO7YhZFwm3q7FOF0DxzIy+AbKVgpPkTculw095vdbKz8B2b7ufERfETLgzfxCjEIqMiVRNKPIEAIlJHEVQ9DaM1X4Bvd365HjFPOmh25qPZdlqbyZ0tcAh5pi93KK3n1Tsv0FmL6esyL8sEn/wRVD0NozdfrvGSO6dmk1WkvqelCaNry06dq701kD8GCaOk978ae22b26QbrzDJ+nOvE+auDR23UVvP6nYf4PMXgbQnc4fzKJ1vd6eEdLMy6XX9jZf9aEjtb3mXz46vMcriWu4qME1XQDFV5X4XFl6eGuyhOvnrMXeut6aXi4JF3UYXm5CkYdFJIa2iqDobdh0+x07JO1uwfrYcw8kWjdc9PaTiv03GDa8TE8XQPcYOdHrfc5Yl039M2736k+QcFFsXNMF0H2mrPR+pFhreiMxjIyY6OkC6G6z3dDPrMPDdq/y6xSf++va44CY6OkCQEXPhGFJdM332xQLComeLgAALULSBQCgRUi6AAC0SCbXdGfPnq1yOc7zrfKp6Gvoirx+roI2zDfaL/+K3oZB6OkCANAizF4GAORW4BOdGtDsM5WbQU8XAJArN1w99JzjNFTquv6qdOoLk8m9l0ulkuOabn5xPSn/it6GtF/+NdOGfo9TzMLUj0kHDiWrwznHo/0AAPmUVq82jv2Dj2jMYtiZ4WUAQEdrZcLN+rwkXQBAR/rtM+1LuBWuLP3JR9Orj6QLAOg4riyNOil5PdfdkryOjTenl/y5pgsA6CjvbE9eR/X12O884L0mTZy/fUYa/UfJ6qCnCwDoKKNHRZfpnS/d8yP/fUEToJJOjEqj503SBQB0jKjeqJW8n75+6VNfTp5IK/VVfs75ZLL4opB0AQAdISqhfft+/+3NJl6/4158Jfq4JImXpAsAaLveidFlVtyafRxSvCQ+aXxzdZN0AQBtd2BzenUF9UTTXH7U92RzxzF7GQDQVn969dB7v15mJVm6cvyhZFeWjg5I4+ZKR56Wxo6JH8/6r8SLZ+Uy6Vv3xa9XoqcLAGizW77gvQYl1D0Hht7PmTV8f1APtpJogxJu0HHXLPFef7XPf38lznWr/PeHIekCADrajEVD77fdVZssw4aM33+l9zrp0uAy9XVVfz5rcWNxxkHSBQC0TdLrrG8cCN738uve66EjwWXC9sXRaPwkXQBAR1s0J3jf9EXB++II6wUvviRZ3X5IugCAjjAQcPvHx25vbRwVj6zz3/7OM83XSdIFALTF1Em1n08e5Q3Xnlx1G8g4w7cbHmnu/A9vjS5Tff4xo73Po+tuBzl5QvxzknQBAG2x7wn/7QPbpWM7vPdxlghd+9Xh246fqP3c1z+8zBUxZh9Xzt+/RXp7m3+Zgz+JrqeCpAsA6Dg9I5Idf9JFtZ975yerb/ypyY6vIOkCADpanN7u0tW1n50LL/+Zr6Vz3kaRdAEAuXd/g7eRXL8pmziiRCZdM5thZk+Z2Utm9qKZfaEVgQEAiu36tfHLZtHrTOt8jXyPOD3d45JWOed+X9JFkv6zmf1+/FMAADDc2uvTre9zN8crl/bTihr5HpFJ1zn3a+fcrsH3RyX9QtK0ZoMDAKAZi1eG7//ug97r1l3++zc97b0GPZe3on5W86cvi44troau6ZrZeyR9SNIOn33LzaxsZuWDBw+mEx0AoGvNPLP282MBS3bqzVvuv/0TMXuk9et37/ZZktSs2EnXzE6V9KCklc65YXerdM59zzlXcs6Vent704sQANCVfnbn8G0LV4QfMzHkto6SdNqHw/evXBO+P6lYSdfMRspLuPc45/4m25AAAN1g8kfC90+bMnzb4xG3YDwc8QCD/qPh+29v8Pm4Uvj9m+vFmb1sku6S9AvnXANztAAACPbmb5o7LquZzFfe0NxxjTypKE5Pd46kqyVdambPD/4kfK4DAACd5Ydbsj9HT1QB59w2SZZ9KAAA1Jo6Udp/qH3nv/CcdOvjjlQAgLaJGire1+Cdpqp98H3S/Auk905vvo5nN4Tvb3SoO7KnCwBAO7lycHJbNCfZ83YXXCdtfjb4vGkj6QIA2mrVOmnNF8PL9G+RJszz3u/fLE2ZWLv/mpukux+Nf845s6Rtd0lP3DG07dW90tmXe+/j9LA/38SdrcxFPYqhCaVSyZXLGfwToUN4E7qLK4vfiU5DG+Yb7Zd/9W0Yp1dppaFyGzdLy1aHl2/EvV+Xli0Yfp6oeII453x/SUm6TeAPPv9ow3yj/fKvvg0nT4j3MPi411CXzJWuXSLNmy0dPir9fLf0jfXSS69EHxsn4U66NHypUFDSZXgZANB2ff3NH7tprZdkg5w2Tjp7mnTVwtrt256XLvlsc+dsZG1uNZIuAKAjxBnWrUyqGtkjvVs3AaqRmcSuLF183tD5Rl4oHT+RfFg5CkkXANAx4l5PrSTcZhNg9XEnnpOO7YhXV9K7YbFOFwDQUZbeGF3GSsEJ8Kbl0uGnvORd+RnY7m33M+KCeMn0j78UXSYKE6mawCSO/KMN8432y7+oNgzq7dYnxyvmSQ/d1nwcy1Z7M6GbOXcYZi+niD/4/KMN8432y784bfj2NmnM6LrjSlLfk9Kk8bXbx86V3hqIf/6J46Q3f1q77ZsbpBvvGJ50l94o3f/j+HVLzF4GAOTMKRd7r/VJsGeENPNy6bW9zdd96Ehtz/WXjw7v8UrpP9GIa7oAgI5WnfhcWXp4a7KE6+esxd663uoEn8UjBBlebgJDW/lHG+Yb7Zd/zbThaWOlQ09lEEyd3vnJ1g1LwcPL9HQBALlw+KjX+1y5Jpv6V9w6eM04YcINQ0+3CfwrO/9ow3yj/fIvrTZM40lAWQwj09MFABROZb2ulYaeQlRt1brh205fUHtcKzF7GQBQCL95yz+Jrr2n9bEEoacLAECLkHQBAGgRki4AAC1C0gUAoEUymUi1c+fOQk/pL/p0/iK3XQVtmG+0X/4VuQ1LpeAp0cxe7hQnDkvPT6zZtGqdtOaLdeXO3SuNPKN1cQEAUkPSbaed4f+aHZZwJWn3mbWfZxf3X4sAUDRc0221/bd6yTYi4cZWqWt/RvdFAwCkhqTbKsfe9JLjni9lU/+eG7z6j+3Ppn4AQGIML7dCWr3aOHaf7r0y7AwAHYeebtZamXA74bwAgEAk3azsGtX+xLfTpEMb2xsDAODfkHSzsNMk927iaq67JYVYXl3W/uQPAJDENd307RqduIrqp2R85wHvNfEzI3eNks7/XcJKAABJ0NNNm4tObL3zpXt+5L8v6NmOiZ/5mELPGwCQDEk3TRHDuJUHJvf1S5/6cvJEWv0QZitJ53wyWXwAgGyRdNMSkdC+fb//9mYTr99xL74S40ASLwC0DUk3DccPRBZZcWsL4lDMJH68L/M4AADDkXTT8MLU1KoKmjCVeCJVtRd6U6wMABAXs5eT+vXQuh6/XmYlWbpy/KFkV5aODkjj5kpHnpbGjokfzvqvDL0Pi0f71kmn+z1RAQCQFXq6Se39M0nBCXVP1cjznFnD9wf1YCuJNijhBh13zRLv9Vf7/Pf/W5xvXO9fAACQGZJuxmYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixMAkD2SbhIJZwK/ETL/6uXXvddDR4LLhO2LhZnMANBSJN2MLZoTvG/6ouB9cYT1ghdfkqxuAED6SLopGdjuv/2x21sbR8Uj6/y3v/NMa+MAAAwh6TbrWO1MpZNHeddUTx41tC3OMp8NjzR3+oe3RpepPv+Y0d7n0SfVFTp2sLkAAAANI+k2a/cZvpsHtkvHdnjv4ywRuvarw7cdP1H7ua9/eJkrVkXXXTl//xbp7W0BhXZPia4IAJAKkm4GekYkO/6ki2o/985PVt/4U5MdDwBIB0k3Y3F6u0tX1352Lrz8Z76WznkBAK0VmXTNbLSZPWdmL5jZi2bmMyCKJO7f3Fj59ZuyiQMAkK04Pd3fSbrUOTdL0nmSPm5mF0UcU3jXr41fttW9zkbO18j3AAAkE5l0neetwY8jB38iBkCLb23Kd1H83M3xyqX9tKK0vwcAIFisa7pmNsLMnpd0QNKPnXM7fMosN7OymaX5PJzCWLwyfP93H/Ret+7y37/pae816Lm8FfWzmj99WXRsAIDWiJV0nXMnnHPnSZou6QIzO8enzPeccyXnHFN4JM08s/bzY0FLdurMW+6//RMxe6T163fv5go8AHSMhmYvO+f6JT0l6ePZhFMcP7tz+LaFK8KPmRhyW0dJOu3D4ftXrgnfDwBorzizl3vNbMLg+5MlfVTSP2UdWMebFX4np2k+95x4POIWjIcjHmDQfzR8/+33he/3dW5fEwcBAJoR5yH2Z0i628xGyEvSDzjnHs02rBzomdzUYVnNZL7yhiYPHDkp1TgAAMEik65zbrekD7UgFiTwwy3tjgAAEIU7UmVo6sT2nv/CYdPdAADtRNJNYnb4cuV9Dd5pqtoH3yfNv0B67/Tm63h2Q0SBiPgBAOmKc00XCbhy8HXcRXOSPW93wXXS5meDzwsA6Cwk3aSm3ybtCZ/F1L9FmjDPe79/szSlbtj5mpukuxuYmjZnlrTtLumJO4a2vbpXOvty732sHvaMv4h/QgBAKsxFPdKmmUrNCj1uOey/2U6LPMZKQ73PjZulZavDyzfi3q9LyxYMP0+okKFls+jvk3dZ/N53kqK3Ie2Xf0Vuw1KppHK57NuIJN0mDPtvduxgrIfBx10utGSudO0Sad5s6fBR6ee7pW+sl156JUZscRLuuX2hS4X4g8+/orch7Zd/RW7DsKTL8HIaRvY2feimtV6SDXLaOOnsadJVC2u3b3teuuSzTZ6UtbkA0BYk3bTMdpHDzJVJVSN7pHfrJkA1ctMMV5YuPm+oVzvyQun4ieTDygCAbJF00xQj8UpDCbfZu1NVH3fiOenYjph1kXABoK1Yp5u2mdE3QLZScJK8abl0+Cmv11r5Gdjubfcz4oKYCXfmD2IUAgBkiYlUTYj8bxbQ261PjlfMkx66rfk4lq32ZkLXxBY0xNxAL5dJHPlX9Dak/fKvyG3I7OWUxfpvtmuM5N6p2WQlqe9JadL42qJj50pvDcQ//8Rx0ps/rd32zQ3SjXf4JN2Z90kTl8avXPzBF0HR25D2y78ityGzl9vh/D6YMZgAACAASURBVMEsWtfr7Rkhzbxcem1v81UfOlLba/7lo8N7vJK4hgsAHYZrulmrSnyuLD28NVnC9XPWYm9db00vl4QLAB2H4eUmNPXf7NghaXcL1seeeyDRumGJoa0iKHob0n75V+Q2DBtepqfbKiMner3PGeuyqX/G7V79CRMuACA7XNNttSkrvR8p1preSAwjA0Bu0NNtp9lu6GfW4WG7V/l1is/9de1xAIDcoKfbKXomDEuia77fplgAAJmgpwsAQIuQdAEAaBGSLgAALZLJNd3Zs2erXI7znLl8KvoauiKvn6ugDfON9su/ordhEHq6AAC0CLOXgQQCn+rUgGafqwwgf+jpAg264eqhZx2noVLX9VelUx+AzpXJvZdLpZLjmm5+cT3Jn98jFbMw9WPSgUPJ6ih6G/I3mH9d0IY82g9oVlq92jj2Dz6mkWFnoHgYXgYitDLhdsJ5AWSHpAsE+O0z7U98riz9yUfbGwOA9JB0AR+uLI06KXk9192SvI6NN7c/+QNIB9d0gTrvbE9eR/X12O884L0mTZy/fUYa/UfJ6gDQXvR0gTqjR0WX6Z0v3fMj/31BE6CSToxKo+cNoL1IukCVqN6olbyfvn7pU19Onkgr9VV+zvlksvgAdDaSLjAoKqF9+37/7c0mXr/jXnwl+jgSL5BfJF1AUu/E6DIrbs0+DileEp80Pvs4AKSPpAtIOrA5vbqCeqJp9lD7nkyvLgCtw+xldL0/vXrovV8vs5IsXTn+ULIrS0cHpHFzpSNPS2PHxI9n/VfixbNymfSt++LXC6D96Omi693yBe81KKHuOTD0fs6s4fuDerCVRBuUcIOOu2aJ9/qrff77K3GuW+W/H0DnIukCEWYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixNA5yPpoqslvc76xoHgfS+/7r0eOhJcJmxfHMxkBvKFpAtEWDQneN/0RcH74gjrBS++JFndADoPSRcYNBBw+8fHbm9tHBWPrPPf/s4zrY0DQHpIuuhaUyfVfj55lDdce3LVbSDjDN9ueKS58z+8NbpM9fnHjPY+j667HeTkCc2dH0DrkXTRtfY94b99YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5Isbs48r5+7dIb2/zL3PwJ9H1AOgMJF3AR8+IZMefdFHt5975yeobf2qy4wF0BpIuECFOb3fp6trPzoWX/8zX0jkvgHyJnXTNbISZ/b2ZPZplQEAe3d/gbSTXb8omDgCdrZGe7hck/SKrQIBWu35t/LKt7nU2cr5GvgeA9oqVdM1suqTLJN2ZbThA66y9Pt36PndzvHJpP60o7e8BIDtxe7rfkvQlSf8aVMDMlptZ2czKBw8eTCU4oJMsXhm+/7sPeq9bd/nv3/S09xr0XN6K+lnNn74sOjYA+RCZdM1ssaQDzrmdYeWcc99zzpWcc6Xe3t7UAgTaZeaZtZ8fC1iyU2/ecv/tn4jZI61fv3u3z5IkAPkUp6c7R9LlZvaapI2SLjWz72caFdABfuZzMWXhivBjJobc1lGSTvtw+P6Va8L3A8i3yKTrnLvROTfdOfceSUsl/dQ596nMIwMyNvkj4funTRm+7fGIWzAejniAQf/R8P23N/F83LD7NwPoLKzTRdd68zfNHZfVTOYrb2juuKRPKgLQOj2NFHbObZG0JZNIgC73wy3tjgBA1ujpAiGmTmzv+S88p73nB5Auki66WtRQ8b4G7zRV7YPvk+ZfIL13evN1PLshfD+3igTypaHhZaAbuXJwcls0J9nzdhdcJ21+Nvi8AIqFpIuut2qdtOaL4WX6t0gT5nnv92+WptQNO19zk3R3A3clnzNL2naX9MQdQ9te3Sudfbn3Pk4P+/Mp39kKQPbMRT0OpQmlUsmVy8X9Z7qZtTuETGXxO9Fp6tswTq/SSkPlNm6Wlq0OL9+Ie78uLVsw/DxR8QQpehvyN5h/XdCGvl+QpNuELvhlaXcImatvw8kT4j0MPu411CVzpWuXSPNmS4ePSj/fLX1jvfTSK9HHxkm4ky4NXypU9DbkbzD/uqANfb8gw8uApL7+5o/dtNZLskFOGyedPU26amHt9m3PS5d8trlzsjYXyCeSLjAozrBuZVLVyB7p3boJUI3MJHZl6eLzhs438kLp+Inkw8oAOhtJF6gS93pqJeE2mwCrjzvxnHRsR7y6SLhAvrFOF6iz9MboMlYKToA3LZcOP+Ul78rPwHZvu58RF8RLpn/8pegyADobE6ma0AUTANodQuai2jCot1ufHK+YJz10W/NxLFvtzYRu5txhit6G/A3mXxe0IbOX09IFvyztDiFzcdrw7W3SmNF1x5WkvielSeNrt4+dK701EP/8E8dJb/60dts3N0g33jE86S69Ubr/x/HrlorfhvwN5l8XtCGzl4FGnHKx91qfBHtGSDMvl17b23zdh47U9lx/+ejwHq/ENVygaLimC0SoTnyuLD28NVnC9XPWYm9db3WCJ+ECxcPwchO6YFik3SFkrpk2PG2sdOipDIKp0zs/2bphqfhtyN9g/nVBG/p+QXq6QEyHj3q9z5Vrsql/xa2D14wTJlwAnYuebhO64F9o7Q4hc2m1YRpPAspiGLnobcjfYP51QRvS0wXSVlmva6WhpxBVW7Vu+LbTF9QeB6B7MHsZSMlv3vJPomvvaX0sADoTPV0AAFqEpAsAQIuQdAEAaJFMrunu3Lmz0DPTij6zsMhtV0Eb5hvtl39FbsNSKXiGJD1dAABapGNnL3fq+kcAAJrVUT3dG64eev5oGip1XX9VOvUBAJBEJnekMrOGKvV7zFkWpn5MOnAoeT1FvhYhcT2pCIrehrRf/hW5DUulksrlcmc+2i+tXm0c+wcfncawMwCgHdo6vNzKhNsJ5wUAdLe2JN3fPtP+xOfK0p98tL0xAAC6S8uTritLo05KXs91tySvY+PN7U/+AIDu0dJruu9sT15H9fXY7zzgvSZNnL99Rhr9R8nqAAAgSkt7uqNHRZfpnS/d8yP/fUEToJJOjEqj5w0AQJSWJd2o3mjl2aJ9/dKnvpw8kVY/r9RK0jmfTBYfAABJtSTpRiW0b9/vv73ZxOt33IuvRB9H4gUAZCnzpNs7MbrMiluzjsITJ4lPGp99HACA7pR50j2wOb26gnqiafZQ+55Mry4AAKplOnv5T68eeu/Xy6wkS1eOP5TsytLRAWncXOnI09LYMfHjWf+VePGsXCZ967749QIAEEemPd1bvuC9BiXUPQeG3s+ZNXx/UA+2kmiDEm7Qcdcs8V5/tc9/fyXOdav89wMAkERbbwM5Y9HQ+2131SbLsCHj91/pvU66NLhMfV3Vn89a3FicAACkIbOkm/Q66xsHgve9/Lr3euhIcJmwfXEwkxkAkLa29nQXzQneN31R8L44wnrBiy9JVjcAAM1oSdIdCLj942O3t+Lswz2yzn/7O8+0Ng4AQHfJJOlOnVT7+eRR3nDtyVW3gYwzfLvhkebO//DW6DLV5x8z2vs8uu52kJMnNHd+AAD8ZJJ09z3hv31gu3Rsh/c+zhKha786fNvxE7Wf+/qHl7kixuzjyvn7t0hvb/Mvc/An0fUAABBXy6/p9oxIdvxJF9V+7p2frL7xpyY7HgCAuNo6kSpOb3fp6trPzoWX/8zX0jkvAABpi5V0zew1M/sHM3vezFq6mOb+Bm8juX5TNnEAAJBUIz3dDzvnznPORfYTr18bv9JW9zobOV8j3wMAgCiZDC+vvT7d+j53c7xyaT+tKO3vAQDobnGTrpO02cx2mtlyvwJmttzMys0MPy9eGb7/uw96r1t3+e/f9LT3GvRc3or6Wc2fviw6NgAA0mIuamaSJDOb5px7w8ymSPqxpM87554OPGCnhVZ69uXSq3trt1XWzQYN/0Y9iShsf1DdcdYK+z6NKMZ/szwzs3aHkDnaMN9ov/wrchuWSiWVy2XfRozV03XOvTH4ekDSQ5IuSBLQz+4cvm3hivBjJobc1lGSTvtw+P6Va8L3AwCQtcika2anmNnYyntJH5P0j2HHTP5IeJ3Tpgzf9njELRgPRzzAoP9o+P7bm3g+btj9mwEAaFSch9hPlfTQ4HBHj6R7nXOPhx3w5m+aCyarmcxX3tDccUmfVAQAQLXIpOuce0WSzyPm8+OHW9odAQAAbbwj1dSJ7Tqz58Jz2nt+AED3ySzpRg0V72vwTlPVPvg+af4F0nunN1/HsxvC93OrSABA2uJc081M2DKfRXOSPW93wXXS5meDzwsAQKtlmnRXrZPWfDG8TP8WacI87/3+zdKUumHna26S7n40/jnnzJK23SU9ccfQtlf3emuDpXg97M+nfGcrAACkmDfHaLhSG7o5RtwbUFTKbdwsLVsdXr4R935dWrZg+Hmi4glT5EXdEgvzi6DobUj75V+R2zDs5hiZJ93JE+I9DD7uNdQlc6Vrl0jzZkuHj0o/3y19Y7300ivRx8ZJuJMujV4qVORfFok/+CIoehvSfvlX5DYMS7qZX9Pt62/+2E1rvSQb5LRx0tnTpKsW1m7f9rx0yWebOydrcwEAWWnJRKo4w7qVSVUje6R36yZANTKT2JWli88bOt/IC6XjJ9IZVgYAIImWzV6Oez21knCbTYDVx514Tjq2I15dJFwAQNZaenOMpTdGl7FScAK8abl0+CkveVd+BrZ72/2MuCBeMv3jL0WXAQAgqcwnUtUL6u3WJ8cr5kkP3dZ8DMtWezOhmzl3lCJPAJCYxFEERW9D2i//ityGbZ297OftbdKY0XXHlKS+J6VJ42u3j50rvTUQ/9wTx0lv/rR22zc3SDfeMTzpLr1Ruv/H8euuKPIvi8QffBEUvQ1pv/wrchu2dfayn1Mu9l7rk2DPCGnm5dJre4cfE9ehI7U9118+OrzHK3ENFwDQem174IFUm/hcWXp4a7KE6+esxd663uoET8IFALRDW4aX6502Vjr0VOphDNM7P9m64YoiD4tIDG0VQdHbkPbLvyK3Ydjwclt7uhWHj3q9z5Vrsql/xa2D14xTSLgAADSrI3q6ftJ4ElBWw8hF/heaxL+yi6DobUj75V+R27Dje7p+Kut1rTT0FKJqq9YN33b6gtrjAADoJG19nm5cv3nLP4muvaf1sQAA0KyO7ekCAFA0JF0AAFqEpAsAQItkck139uzZKpdTmH7coYo+s7DIsworaMN8o/3yr+htGISeLgAALULSBQCgRXKxZAgA0KSdKQzjzi7+cHer0NMFgKLZf6uXbNNIuNJQXfszuldvFyHpAkBRHHvTS457vpRN/Xtu8Oo/tj+b+rsAw8sAUARp9Wrj2H2698qwc8Po6QJA3rUy4XbCeXOMpAsAebVrVPsT306TDm1sbww5QtIFgDzaaZJ7N3E1192SQiyvLmt/8s8JrukCQN7sGp24iuont33nAe818XPMd42Szv9dwkqKjZ4uAOSNi05svfOle37kvy/oeeOJn0OeQs+76Ei6AJAnEcO4VvJ++vqlT305eSKt1Ff5OeeTyeLrdiRdAMiLiIT27fv9tzebeP2Oe/GVGAeSeAORdAEgD44fiCyy4tYWxKGYSfx4X+Zx5BFJFwDy4IWpqVUVNGEq8USqai/0plhZcTB7GQA63a+H1vX49TIrydKV4w8lu7J0dEAaN1c68rQ0dkz8cNZ/Zeh9WDzat046/YvxK+4C9HQBoNPt/TNJwQl1T9XI85xZw/cH9WAriTYo4QYdd80S7/VX+/z3/1ucb1zvX6CLkXQBIOdmLBp6v+2u2mQZNmT8/iu910mXBpepr6v681mLG4sTJF0A6GwJZwK/ETL/6uXXvddDR4LLhO2LhZnMNUi6AJBzi+YE75u+KHhfHGG94MWXJKu7G5F0ASAnBrb7b3/s9tbGUfHIOv/t7zzT2jjyhKQLAJ3qWO1MpZNHeddUTx41tC3OMp8NjzR3+oe3RpepPv+Y0d7n0SfVFTp2sLkACoikCwCdavcZvpsHtkvHdnjv4ywRuvarw7cdP1H7ua9/eJkrVkXXXTl//xbp7W0BhXZPia6oS5B0ASCHekYkO/6ki2o/985PVt/4U5Md3y1iJV0zm2Bmf21m/2RmvzCzP8w6MABAPHF6u0tX1352Lrz8Z76WznlRK25P93ZJjzvn/p2kWZJ+kV1IAIC03b+5sfLrN2UTR7eLTLpmNl7SXEl3SZJz7l3nnM/oPwAgTdevjV+21b3ORs7XyPcoujg93ZmSDkpab2Z/b2Z3mtkpGccFAF1vbcp3UfzczfHKpf20orS/R57FSbo9ks6X9JfOuQ9JelvSn9cXMrPlZlY2s/LBg0wPB4BWW7wyfP93H/Ret+7y37/pae816Lm8FfWzmj99WXRs8MRJunsk7XHODU5Q11/LS8I1nHPfc86VnHOl3l4e6QQAWZt5Zu3nx4KW7NSZt9x/+ydi9kjr1+/e7bMkCf4ik65zbp+k183sA4ObPiLppUyjAgBE+tmdw7ctXBF+zMSQ2zpK0mkfDt+/ck34foSL+zzdz0u6x8xOkvSKpGuzCwkAIEmadTD0YfDTfO458XjELRgPRzzAoP9o+P7b7wvf7+vcviYOKqZYSdc597wkVmQBQCv1TG7qsKxmMl95Q5MHjpyUahx5xh2pAACx/HBLuyPIP5IuAOTY1IntPf+F57T3/HlD0gWATjY7/H6N+xq801S1D75Pmn+B9N7pzdfx7IaIAhHxd5u4E6kAAB3KlYOv4y6ak+x5uwuukzY/G3xeNIakCwCdbvpt0p7wWUz9W6QJ87z3+zdLU+qGna+5Sbr70finnDNL2naX9MQdQ9te3Sudfbn3PlYPe8ZfxD9hlzAX9aiJJpRKJVcuF/efQGbW7hAylcXvRKehDfOtK9tvZ/R3ttJQ73PjZmnZ6vDyjbj369KyBcPPEypkaLkL2tD3C5J0m9AFvyztDiFztGG+dWX7HTsY62HwcZcLLZkrXbtEmjdbOnxU+vlu6RvrpZdeiRFfnP+9n9sXulSoC9rQ9wsyvAwAeTCy+dvrblrrJdkgp42Tzp4mXbWwdvu256VLPtvkSVmb64ukCwB5MdtFDjNXJlWN7JHerZsA1chNM1xZuvi8oV7tyAul4yeSDyt3O5IuAORJjMQrDSXcZu9OVX3cieekYzti1kXCDcU6XQDIm5nRN0C2UnCSvGm5dPgpr9da+RnY7m33M+KCmAl35g9iFOpuTKRqQhdMAGh3CJmjDfON9lNgb7c+OV4xT3rotuZjWbbamwldLXCIuYFebhe0IbOX09IFvyztDiFztGG+0X6Ddo2R3Ds1m6wk9T0pTRpfW3TsXOmtgfgxTBwnvfnT2m3f3CDdeIdP0p15nzRxafzK1RVtyOxlACiU8wezaF2vt2eENPNy6bW9zVd96Ehtr/mXjw7v8UriGm6DuKYLAHlXlfhcWXp4a7KE6+esxd663ppeLgm3YQwvN6ELhkXaHULmaMN8o/0CHDsk7W7B+thzDyRaNyx1RRv6fkF6ugBQFCMner3PGeuyqX/G7V79CRNuN+OaLgAUzZSV3o8Ua01vJIaRU0NPFwCKbLYb+pl1eNjuVX6d4nN/XXscUkNPFwC6Rc+EYUl0zffbFEuXoqcLAECLkHQBAGgRki4AAC2SyTXdnTt3FnoNFmsg8482zDfaL/+K3IalUvDTIejpAgDQIsxeBhAo1gPLIzT7PFegiOjpAqhxw9VDz1hNQ6Wu669Kpz4gzzK597KZFXewXsW+FiFxPakImmlDv0e5ZWHqx6QDh5LVQfvlX5HbsFQqqVwu82g/AP7S6tXGsX/w8XAMO6MbMbwMdLlWJtxOOC/QTiRdoEv99pn2Jz5Xlv7ko+2NAWglki7QhVxZGnVS8nquuyV5HRtvbn/yB1qFa7pAl3lne/I6qq/HfucB7zVp4vztM9LoP0pWB9Dp6OkCXWb0qOgyvfOle37kvy9oAlTSiVFp9LyBTkfSBbpIVG/USt5PX7/0qS8nT6SV+io/53wyWXxA3pF0gS4RldC+fb//9mYTr99xL74SfRyJF0VG0gW6QO/E6DIrbs0+DileEp80Pvs4gHYg6QJd4MDm9OoK6omm2UPtezK9uoBOwuxloOD+9Oqh9369zEqydOX4Q8muLB0dkMbNlY48LY0dEz+e9V+JF8/KZdK37otfL5AH9HSBgrvlC95rUELdc2Do/ZxZw/cH9WAriTYo4QYdd80S7/VX+/z3V+Jct8p/P5BnJF2gy81YNPR+2121yTJsyPj9V3qvky4NLlNfV/XnsxY3FidQBCRdoMCSXmd940Dwvpdf914PHQkuE7YvDmYyo2hIukCXWzQneN/0RcH74gjrBS++JFndQB6RdIEuMRBw+8fHbm9tHBWPrPPf/s4zrY0DaCWSLlBQUyfVfj55lDdce3LVbSDjDN9ueKS58z+8NbpM9fnHjPY+j667HeTkCc2dH+hEJF2goPY94b99YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5Isbs48r5+7dIb2/zL3PwJ9H1AHlB0gW6UM+IZMefdFHt5975yeobf2qy44G8IOkCXS5Ob3fp6trPzoWX/8zX0jkvUDSRSdfMPmBmz1f9HDGzla0IDkBnuL/B20iu35RNHEDeRSZd59w/O+fOc86dJ2m2pAFJD2UeGYBErl8bv2yre52NnK+R7wF0ukaHlz8i6V+cc7/MIhgA6Vl7fbr1fe7meOXSflpR2t8DaKdGk+5SSb63IDez5WZWNjPuIQPk0OKIi0bffdB73brLf/+mp73XoOfyVtTPav70ZdGxAUVhLmpGRKWg2UmS9kr69865/RFl41WaU3H/m+WVmbU7hMx1QxtGrcE9+3Lp1b212yrHBA3/Rj2JKGx/UN1x1goPO6YL2q/oityGpVJJ5XLZtxEb6ekulLQrKuECyIef3Tl828IV4cdMDLmtoySd9uHw/SvXhO8Hiq6RpLtMAUPLADrP5I+E7582Zfi2xyNuwXg44gEG/UfD99/exP9Bwu7fDORNrKRrZqdI+qikv8k2HABpefM3zR2X1UzmK29o7rikTyoCOklPnELOubclTYosCAABfril3REA7ccdqYAuNnVie89/4TntPT/QaiRdoMCihor3NXinqWoffJ80/wLpvdObr+PZDeH7uVUkiibW8DKA4gpb5rNoTrLn7S64Ttr8bPB5gW5D0gUKbtU6ac0Xw8v0b5EmzPPe798sTakbdr7mJunuR+Ofc84sadtd0hN3DG17da+3NliK18P+fMp3tgI6QeybYzRUKTfHyDUW5udffRvGvQFFpdzGzdKy1eHlG3Hv16VlC4afJyqeIN3WfkVU5DYMuzkGSbcJRf5lkfiDL4L6Npw8Id7D4ONeQ10yV7p2iTRvtnT4qPTz3dI31ksvvRJ9bJyEO+nS8KVC3dZ+RVTkNgxLugwvA12gr7/5Yzet9ZJskNPGSWdPk65aWLt92/PSJZ9t7pyszUVRkXSBLhFnWLcyqWpkj/Ru3QSoRmYSu7J08XlD5xt5oXT8RPJhZSDvSLpAF4l7PbWScJtNgNXHnXhOOrYjXl0kXBQd63SBLrP0xugyVgpOgDctlw4/5SXvys/Adm+7nxEXxEumf/yl6DJA3jGRqglFngAgMYmjCKLaMKi3W58cr5gnPXRb83EsW+3NhG7m3GG6vf2KoMhtyOzllBX5l0XiD74I4rTh29ukMaPrjitJfU9Kk8bXbh87V3prIP75J46T3vxp7bZvbpBuvGN40l16o3T/j+PXLdF+RVDkNmT2MoBhTrnYe61Pgj0jpJmXS6/tHX5MXIeO1PZcf/no8B6vxDVcdB+u6QJdrjrxubL08NZkCdfPWYu9db3VCZ6Ei27E8HITijwsIjG0VQTNtOFpY6VDT2UQTJ3e+cnWDUu0XxEUuQ3Dhpfp6QKQ5N1ZykrSyjXZ1L/i1sFrxgkTLpBn9HSbUOR/oUn8K7sI0mrDNJ4ElMUwMu2Xf0VuQ3q6AJpSWa9rpaGnEFVbtW74ttMX1B4HYAizlwHE8pu3/JPo2ntaHwuQV/R0AQBoEZIuAAAtQtIFAKBFsrqm2yfplxnV7Wfy4Dlbog0zC1v6/dqg5d+vxW1I+6WMv8HUFb0NW/39zgrakcmSoVYzs7JzrrDzJPl++cb3y7+if0e+X+swvAwAQIuQdAEAaJGiJN3vtTuAjPH98o3vl39F/458vxYpxDVdAADyoCg9XQAAOh5JFwCAFsl10jWzj5vZP5vZy2b25+2OJ21m9ldmdsDM/rHdsWTBzGaY2VNm9pKZvWhmX2h3TGkys9Fm9pyZvTD4/b7a7piyYGYjzOzvzezRdseSNjN7zcz+wcyeN7MUnrnUWcxsgpn9tZn9k5n9wsz+sN0xpcnMPjDYdpWfI2a2sq0x5fWarpmNkPT/SfqopD2S/k7SMufcS20NLEVmNlfSW5L+h3PunHbHkzYzO0PSGc65XWY2VtJOSVcUpQ3NW/1/inPuLTMbKWmbpC84555tc2ipMrPrJZUkjXPOLW53PGkys9cklZxzhbwxhpndLelnzrk7zewkSWOcc4V84vFgznhD0oXOuVbevKlGnnu6F0h62Tn3inPuXUkbJX2izTGlyjn3tKRD7Y4jK865Xzvndg2+PyrpF5KmtTeq9DjPW4MfRw7+5PNfuQHMbLqkyyTd2e5Y0BgzGy9prqS7JMk5925RE+6gj0j6l3YmXCnfSXeapNerPu9Rgf6H3W3M7D2SPiRpR3sjSdfg0Ovzkg5I+rFzrlDfT9K3JH1J0r+2O5CMOEmbzWynmS1vdzApmynpoKT1g5cH7jSzU9odVIaWSrqv3UHkOemiIMzsVEkPSlrpnDvS7njS5Jw74Zw7T9J0SReYWWEuE5jZYkkHnHM72x1Lhi52zp0vaaGk/zx4yacoeiSdL+kvnXMfkvS2pMLNjZGkwaHzyyX9oN2x5DnpviFpRtXnQ6Y6tgAAAXNJREFU6YPbkCOD1zoflHSPc+5v2h1PVgaH7Z6S9PF2x5KiOZIuH7zuuVHSpWb2/faGlC7n3BuDrwckPSTvslZR7JG0p2r05a/lJeEiWihpl3Nuf7sDyXPS/TtJ7zezmYP/ilkqaVObY0IDBica3SXpF865te2OJ21m1mtmEwbfnyxv0t8/tTeq9DjnbnTOTXfOvUfe399PnXOfanNYqTGzUwYn+Glw2PVjkgqzksA5t0/S62b2gcFNH5FUiEmMPpapA4aWpewe7Zc559xxM7tO0hOSRkj6K+fci20OK1Vmdp+keZImm9keSV9xzt3V3qhSNUfS1ZL+YfC6pyStds79bRtjStMZku4enDX5e5IecM4VbllNgU2V9NDgI+h6JN3rnHu8vSGl7vOS7hnsuLwi6do2x5O6wX8wfVTSf2p3LFKOlwwBAJA3eR5eBgAgV0i6AAC0CEkXAIAWIekCANAiJF0AAFqEpAsAQIuQdAEAaJH/H3g7SUIqLC/qAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
\n", + "In the `search.ipynb` notebook, we will see how NQueensProblem can be solved using a **heuristic search method** such as `uniform_cost_search` and `astar_search`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper Functions\n", + "\n", + "We will now implement a few helper functions that will allow us to visualize the Coloring Problem; we'll also make a few modifications to the existing classes and functions for additional record keeping. To begin, we modify the **assign** and **unassign** methods in the **CSP** in order to add a copy of the assignment to the **assignment_history**. We name this new class as **InstruCSP**; it will allow us to see how the assignment evolves over time. " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, "outputs": [], "source": [ - "import csp" + "import copy\n", + "class InstruCSP(CSP):\n", + " \n", + " def __init__(self, variables, domains, neighbors, constraints):\n", + " super().__init__(variables, domains, neighbors, constraints)\n", + " self.assignment_history = []\n", + " \n", + " def assign(self, var, val, assignment):\n", + " super().assign(var,val, assignment)\n", + " self.assignment_history.append(copy.deepcopy(assignment))\n", + " \n", + " def unassign(self, var, assignment):\n", + " super().unassign(var,assignment)\n", + " self.assignment_history.append(copy.deepcopy(assignment))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we define **make_instru** which takes an instance of **CSP** and returns an instance of **InstruCSP**." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "def make_instru(csp):\n", + " return InstruCSP(csp.variables, csp.domains, csp.neighbors, csp.constraints)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will now use a graph defined as a dictionary for plotting purposes in our Graph Coloring Problem. The keys are the nodes and their values are the corresponding nodes they are connected to." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "neighbors = {\n", + " 0: [6, 11, 15, 18, 4, 11, 6, 15, 18, 4], \n", + " 1: [12, 12, 14, 14], \n", + " 2: [17, 6, 11, 6, 11, 10, 17, 14, 10, 14], \n", + " 3: [20, 8, 19, 12, 20, 19, 8, 12], \n", + " 4: [11, 0, 18, 5, 18, 5, 11, 0], \n", + " 5: [4, 4], \n", + " 6: [8, 15, 0, 11, 2, 14, 8, 11, 15, 2, 0, 14], \n", + " 7: [13, 16, 13, 16], \n", + " 8: [19, 15, 6, 14, 12, 3, 6, 15, 19, 12, 3, 14], \n", + " 9: [20, 15, 19, 16, 15, 19, 20, 16], \n", + " 10: [17, 11, 2, 11, 17, 2], \n", + " 11: [6, 0, 4, 10, 2, 6, 2, 0, 10, 4], \n", + " 12: [8, 3, 8, 14, 1, 3, 1, 14], \n", + " 13: [7, 15, 18, 15, 16, 7, 18, 16], \n", + " 14: [8, 6, 2, 12, 1, 8, 6, 2, 1, 12], \n", + " 15: [8, 6, 16, 13, 18, 0, 6, 8, 19, 9, 0, 19, 13, 18, 9, 16], \n", + " 16: [7, 15, 13, 9, 7, 13, 15, 9], \n", + " 17: [10, 2, 2, 10], \n", + " 18: [15, 0, 13, 4, 0, 15, 13, 4], \n", + " 19: [20, 8, 15, 9, 15, 8, 3, 20, 3, 9], \n", + " 20: [3, 19, 9, 19, 3, 9]\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we are ready to create an InstruCSP instance for our problem. We are doing this for an instance of **MapColoringProblem** class which inherits from the **CSP** Class. This means that our **make_instru** function will work perfectly for it." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "coloring_problem = MapColoringCSP('RGBY', neighbors)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "coloring_problem1 = make_instru(coloring_problem)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### CONSTRAINT PROPAGATION\n", + "Algorithms that solve CSPs have a choice between searching and or doing a _constraint propagation_, a specific type of inference.\n", + "The constraints can be used to reduce the number of legal values for another variable, which in turn can reduce the legal values for some other variable, and so on. \n", + "
\n", + "Constraint propagation tries to enforce _local consistency_.\n", + "Consider each variable as a node in a graph and each binary constraint as an arc.\n", + "Enforcing local consistency causes inconsistent values to be eliminated throughout the graph, \n", + "a lot like the `GraphPlan` algorithm in planning, where mutex links are removed from a planning graph.\n", + "There are different types of local consistencies:\n", + "1. Node consistency\n", + "2. Arc consistency\n", + "3. Path consistency\n", + "4. K-consistency\n", + "5. Global constraints\n", + "\n", + "Refer __section 6.2__ in the book for details.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## AC-3\n", + "Before we dive into AC-3, we need to know what _arc-consistency_ is.\n", + "
\n", + "A variable $X_i$ is __arc-consistent__ with respect to another variable $X_j$ if for every value in the current domain $D_i$ there is some value in the domain $D_j$ that satisfies the binary constraint on the arc $(X_i, X_j)$.\n", + "
\n", + "A network is arc-consistent if every variable is arc-consistent with every other variable.\n", + "
\n", + "\n", + "AC-3 is an algorithm that enforces arc consistency.\n", + "After applying AC-3, either every arc is arc-consistent, or some variable has an empty domain, indicating that the CSP cannot be solved.\n", + "Let's see how `AC3` is implemented in the module." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def AC3(csp, queue=None, removals=None, arc_heuristic=dom_j_up):\n",
+ " """[Figure 6.3]"""\n",
+ " if queue is None:\n",
+ " queue = {(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]}\n",
+ " csp.support_pruning()\n",
+ " queue = arc_heuristic(csp, queue)\n",
+ " while queue:\n",
+ " (Xi, Xj) = queue.pop()\n",
+ " if revise(csp, Xi, Xj, removals):\n",
+ " if not csp.curr_domains[Xi]:\n",
+ " return False\n",
+ " for Xk in csp.neighbors[Xi]:\n",
+ " if Xk != Xj:\n",
+ " queue.add((Xk, Xi))\n",
+ " return True\n",
+ "def revise(csp, Xi, Xj, removals):\n",
+ " """Return true if we remove a value."""\n",
+ " revised = False\n",
+ " for x in csp.curr_domains[Xi][:]:\n",
+ " # If Xi=x conflicts with Xj=y for every possible y, eliminate Xi=x\n",
+ " if all(not csp.constraints(Xi, x, Xj, y) for y in csp.curr_domains[Xj]):\n",
+ " csp.prune(Xi, x, removals)\n",
+ " revised = True\n",
+ " return revised\n",
+ "\n", + "If in doing so, $D_i$ is left unchanged, the algorithm just moves to the next arc, \n", + "but if the domain $D_i$ is revised, then we add all the neighboring arcs $(X_k, X_i)$ to the queue.\n", + "
\n", + "We repeat this process and if at any point, the domain $D_i$ is reduced to nothing, then we know the whole CSP has no consistent solution and `AC3` can immediately return failure.\n", + "
\n", + "Otherwise, we keep removing values from the domains of variables until the queue is empty.\n", + "We finally get the arc-consistent CSP which is faster to search because the variables have smaller domains." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see how `AC3` can be used.\n", + "
\n", + "We'll first define the required variables." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "neighbors = parse_neighbors('A: B; B: ')\n", + "domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]}\n", + "constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 and y % 2 != 0\n", + "removals = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We'll now define a `CSP` object." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints)" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "AC3(csp, removals=removals)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This configuration is inconsistent." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "constraints = lambda X, x, Y, y: (x % 2) == 0 and (x + y) == 4\n", + "removals = []\n", + "csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "AC3(csp,removals=removals)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This configuration is consistent." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## BACKTRACKING SEARCH\n", + "\n", + "The main issue with using Naive Search Algorithms to solve a CSP is that they can continue to expand obviously wrong paths; whereas, in **backtracking search**, we check the constraints as we go and we deal with only one variable at a time. Backtracking Search is implemented in the repository as the function **backtracking_search**. This is the same as **Figure 6.5** in the book. The function takes as input a CSP and a few other optional parameters which can be used to speed it up further. The function returns the correct assignment if it satisfies the goal. However, we will discuss these later. For now, let us solve our **coloring_problem1** with **backtracking_search**." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "result = backtracking_search(coloring_problem1)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{0: 'R',\n", + " 1: 'R',\n", + " 2: 'R',\n", + " 3: 'R',\n", + " 4: 'G',\n", + " 5: 'R',\n", + " 6: 'G',\n", + " 7: 'R',\n", + " 8: 'B',\n", + " 9: 'R',\n", + " 10: 'G',\n", + " 11: 'B',\n", + " 12: 'G',\n", + " 13: 'G',\n", + " 14: 'Y',\n", + " 15: 'Y',\n", + " 16: 'B',\n", + " 17: 'B',\n", + " 18: 'B',\n", + " 19: 'G',\n", + " 20: 'B'}" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result # A dictonary of assignments." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let us also check the number of assignments made." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "21" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "coloring_problem1.nassigns" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, let us check the total number of assignments and unassignments, which would be the length of our assignment history. We can see it by using the command below. " + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "21" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(coloring_problem1.assignment_history)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let us explore the optional keyword arguments that the **backtracking_search** function takes. These optional arguments help speed up the assignment further. Along with these, we will also point out the methods in the CSP class that help to make this work. \n", + "\n", + "The first one is **select_unassigned_variable**. It takes in, as a parameter, a function that helps in deciding the order in which the variables will be selected for assignment. We use a heuristic called Most Restricted Variable which is implemented by the function **mrv**. The idea behind **mrv** is to choose the variable with the least legal values left in its domain. The intuition behind selecting the **mrv** or the most constrained variable is that it allows us to encounter failure quickly before going too deep into a tree if we have selected a wrong step before. The **mrv** implementation makes use of another function **num_legal_values** to sort out the variables by the number of legal values left in its domain. This function, in turn, calls the **nconflicts** method of the **CSP** to return such values." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def mrv(assignment, csp):\n",
+ " """Minimum-remaining-values heuristic."""\n",
+ " return argmin_random_tie(\n",
+ " [v for v in csp.variables if v not in assignment],\n",
+ " key=lambda var: num_legal_values(csp, var, assignment))\n",
+ "def num_legal_values(csp, var, assignment):\n",
+ " if csp.curr_domains:\n",
+ " return len(csp.curr_domains[var])\n",
+ " else:\n",
+ " return count(csp.nconflicts(var, val, assignment) == 0\n",
+ " for val in csp.domains[var])\n",
+ " def nconflicts(self, var, val, assignment):\n",
+ " """Return the number of conflicts var=val has with other variables."""\n",
+ "\n",
+ " # Subclasses may implement this more efficiently\n",
+ " def conflict(var2):\n",
+ " return (var2 in assignment and\n",
+ " not self.constraints(var, val, var2, assignment[var2]))\n",
+ "\n",
+ " return count(conflict(v) for v in self.neighbors[var])\n",
+ "def lcv(var, assignment, csp):\n",
+ " """Least-constraining-values heuristic."""\n",
+ " return sorted(csp.choices(var),\n",
+ " key=lambda val: csp.nconflicts(var, val, assignment))\n",
+ "def tree_csp_solver(csp):\n",
+ " """[Figure 6.11]"""\n",
+ " assignment = {}\n",
+ " root = csp.variables[0]\n",
+ " X, parent = topological_sort(csp, root)\n",
+ "\n",
+ " csp.support_pruning()\n",
+ " for Xj in reversed(X[1:]):\n",
+ " if not make_arc_consistent(parent[Xj], Xj, csp):\n",
+ " return None\n",
+ "\n",
+ " assignment[root] = csp.curr_domains[root][0]\n",
+ " for Xi in X[1:]:\n",
+ " assignment[Xi] = assign_value(parent[Xi], Xi, csp, assignment)\n",
+ " if not assignment[Xi]:\n",
+ " return None\n",
+ " return assignment\n",
+ "\n", - "*Date: 12th March 2016*" + "# GAMES OR ADVERSARIAL SEARCH\n", + "\n", + "This notebook serves as supporting material for topics covered in **Chapter 5 - Adversarial Search** in the book *Artificial Intelligence: A Modern Approach.* This notebook uses implementations from [games.py](https://github.com/aimacode/aima-python/blob/master/games.py) module. Let's import required classes, methods, global variables etc., from games module." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## An Introduction" + "# CONTENTS\n", + "\n", + "* Game Representation\n", + "* Game Examples\n", + " * Tic-Tac-Toe\n", + " * Figure 5.2 Game\n", + "* Min-Max\n", + "* Alpha-Beta\n", + "* Players\n", + "* Let's Play Some Games!" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from games import *\n", + "from notebook import psource, pseudocode" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - " Hello all! \n", - " In this IPython notebook, I plan to help you a little so that you will be able to use the [games.py](https://github.com/aimacode/aima-python/blob/master/games.py) module. You might already know that the `games.py` module implements the algorithms in Chapter 5 (Adversarial Search) of the book *Artificial Intelligence: A Modern Approach*. \n", - " \n", - " Before we begin, if you are unsure of how to use the [aima-python](https://github.com/aimacode/aima-python) repository or are not familiar with IPython notebooks you should read the [intro notebook](https://github.com/aimacode/aima-python/blob/master/intro.ipynb) first. Also, if you are more comfortable with Java than you are with Python we also have the [aima-java](https://github.com/aimacode/aima-java) repository.\n", - " \n", - " What we will do to learn to use the code in this module is simply dive in! I feel this is the correct approach as I assume you must have already read Chapter 5 of AIMA. If you haven't, you might want to go back and do that first. If you are tired (or just lazy), at least read the chapter upto Sec. 5.3 because this module covers the algorithms only till that section anyway. So, I will start by explaining what the class `Game` is and then we will immediately start implementing the `TicTacToe` game. After we define the rules of the `TicTacToe` game, we will create AI players who use different search strategies, namely Minimax Search and Alpha-Beta Search. We will make these players play among themselves, and later on we ourselves will play against these AI players (Yay!). \n", - " \n", - "The reason I chose the `TicTacToe` game for demonstration of this module should be obvious to you. Everyone knows it and has played it, it is analyzed in quite some detail in AIMA, and most importantly, it has comparatively few states (fewer than 362,880) so that we can explore the search tree completely. \n", - " \n", - " So let's begin." + "# GAME REPRESENTATION\n", + "\n", + "To represent games we make use of the `Game` class, which we can subclass and override its functions to represent our own games. A helper tool is the namedtuple `GameState`, which in some cases can come in handy, especially when our game needs us to remember a board (like chess)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Implementing TicTacToe" + "## `GameState` namedtuple\n", + "\n", + "`GameState` is a [namedtuple](https://docs.python.org/3.5/library/collections.html#collections.namedtuple) which represents the current state of a game. It is used to help represent games whose states can't be easily represented normally, or for games that require memory of a board, like Tic-Tac-Toe.\n", + "\n", + "`Gamestate` is defined as follows:\n", + "\n", + "`GameState = namedtuple('GameState', 'to_move, utility, board, moves')`\n", + "\n", + "* `to_move`: It represents whose turn it is to move next.\n", + "\n", + "* `utility`: It stores the utility of the game state. Storing this utility is a good idea, because, when you do a Minimax Search or an Alphabeta Search, you generate many recursive calls, which travel all the way down to the terminal states. When these recursive calls go back up to the original callee, we have calculated utilities for many game states. We store these utilities in their respective `GameState`s to avoid calculating them all over again.\n", + "\n", + "* `board`: A dict that stores the board of the game.\n", + "\n", + "* `moves`: It stores the list of legal moves possible from the current position." ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "collapsed": true + }, "source": [ - "To use the code in `games.py` let's import everything from it:" + "## `Game` class\n", + "\n", + "Let's have a look at the class `Game` in our module. We see that it has functions, namely `actions`, `result`, `utility`, `terminal_test`, `to_move` and `display`.\n", + "\n", + "We see that these functions have not actually been implemented. This class is just a template class; we are supposed to create the class for our game, by inheriting this `Game` class and implementing all the methods mentioned in `Game`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": false + "collapsed": true }, "outputs": [], "source": [ - "from games import *" + "%psource Game" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that, here, as the module `games.py` does a `from utils import *`, all the names (global variables, functions etc.) available in `utils.py` are directly available to us now." + "Now let's get into details of all the methods in our `Game` class. You have to implement these methods when you create new classes that would represent your game.\n", + "\n", + "* `actions(self, state)`: Given a game state, this method generates all the legal actions possible from this state, as a list or a generator. Returning a generator rather than a list has the advantage that it saves space and you can still operate on it as a list.\n", + "\n", + "\n", + "* `result(self, state, move)`: Given a game state and a move, this method returns the game state that you get by making that move on this game state.\n", + "\n", + "\n", + "* `utility(self, state, player)`: Given a terminal game state and a player, this method returns the utility for that player in the given terminal game state. While implementing this method assume that the game state is a terminal game state. The logic in this module is such that this method will be called only on terminal game states.\n", + "\n", + "\n", + "* `terminal_test(self, state)`: Given a game state, this method should return `True` if this game state is a terminal state, and `False` otherwise.\n", + "\n", + "\n", + "* `to_move(self, state)`: Given a game state, this method returns the player who is to play next. This information is typically stored in the game state, so all this method does is extract this information and return it.\n", + "\n", + "\n", + "* `display(self, state)`: This method prints/displays the current state of the game." ] }, { "cell_type": "markdown", - "metadata": { - "collapsed": true - }, + "metadata": {}, "source": [ - "### The class `Game` \n", - " \n", - "Let's have a look at the class `Game` in our module. We see that it has six functions, namely `actions`, `result`, `utility`, `terminal_test`, `to_move` and `display`. We see that these functions have not actually been implemented. This class is actually just a template class; we are supposed to create the class for our game, `TicTacToe` by inheriting this `Game` class and implement all the methods mentioned in `Game`. If you forget to implement any one of those, a `NotImplementedError` will be raised. So, in this sense, the `Game` class is what you might call an abstract class in Java: it implements nothing, just tells you all that you are supposed to implement and screams at you if forget to implement what it asks. \n", - " \n", - " Now let's get into some details of all these methods in our `Game` class. You have to implement these methods when you create the new class that would represent your game.\n", - " \n", - "* `__init__(self,
![](\"images/fig_5_2.png\")
\n",
+ "\n",
+ "The states are represented with capital letters inside the triangles (eg. \"A\") while moves are the labels on the edges between states (eg. \"a1\"). Terminal nodes carry utility values. Note that the terminal nodes are named in this example 'B1', 'B2' and 'B2' for the nodes below 'B', and so forth.\n",
+ "\n",
+ "We will model the moves, utilities and initial state like this:"
]
},
{
- "cell_type": "markdown",
- "metadata": {},
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
"source": [
- "## `query_player` and `play_game` \n",
- " \n",
- " The `query_player` function allows you, a human opponent, to play the game. This function requires a `display` method to be implemented in your game class, so that successive game states can be displayed on the terminal, making it easier for you to visualize the game and play accordingly. \n",
- " \n",
- " The `play_game` function will be the one that will actually be used to play the game. You pass as arguments to it, an instance of the game you want to play and the players you want in this game. Use it to play AI vs AI, AI vs human, or even human vs human matches!"
+ "moves = dict(A=dict(a1='B', a2='C', a3='D'),\n",
+ " B=dict(b1='B1', b2='B2', b3='B3'),\n",
+ " C=dict(c1='C1', c2='C2', c3='C3'),\n",
+ " D=dict(d1='D1', d2='D2', d3='D3'))\n",
+ "utils = dict(B1=3, B2=12, B3=8, C1=2, C2=4, C3=6, D1=14, D2=5, D3=2)\n",
+ "initial = 'A'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Examples \n",
- " \n",
- " I will show some code examples below that you can run. The games' classes which I will use are `TicTacToe` and the `Fig52Game`. The `Fig52Game` is already implemented (actually both are) in the module. This is that small game in Fig 5.2 of the book. \n",
- " \n",
- " Have fun executing and modifying these examples!"
+ "In `moves`, we have a nested dictionary system. The outer's dictionary has keys as the states and values the possible moves from that state (as a dictionary). The inner dictionary of moves has keys the move names and values the next state after the move is complete.\n",
+ "\n",
+ "Below is an example that showcases `moves`. We want the next state after move 'a1' from 'A', which is 'B'. A quick glance at the above image confirms that this is indeed the case."
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": 5,
"metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "B\n"
+ ]
+ }
+ ],
"source": [
- "![](\"images/fig_5_2.svg\")
"
+ "print(moves['A']['a1'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Let's start by experimenting with the `Fig52Game` first. For that we'll first create an instance of this game:"
+ "We will now take a look at the functions we need to implement. First we need to create an object of the `Fig52Game` class."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 6,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "from games import *\n",
- "game52 = Fig52Game()"
+ "fig52 = Fig52Game()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "First we try out our `random_player`. Given a game state it will give us a random move every time:"
+ "`actions`: Returns the list of moves one can make from a given state."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "random_player(game52, 'A')"
+ "psource(Fig52Game.actions)"
]
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "['b1', 'b2', 'b3']\n"
+ ]
+ }
+ ],
"source": [
- "random_player(game52, 'A')"
+ "print(fig52.actions('B'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "The `alphabeta_player` will always give us the best move:"
+ "`result`: Returns the next state after we make a specific move."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "print( alphabeta_player(game52, 'A') )\n",
- "print( alphabeta_player(game52, 'B') )\n",
- "print( alphabeta_player(game52, 'C') )"
+ "psource(Fig52Game.result)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "B\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(fig52.result('A', 'a1'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "What the `alphabeta_player` does is, it simply calls the method `alphabeta_full_search`. They both are essentially the same. In the module, both `alphabeta_full_search` and `minimax_decision` have been implemented. They both do the same job and return the same thing, which is, the best move in the current state. It's just that `alphabeta_full_search` is more efficient w.r.t time because it prunes the search tree and hence, explores lesser number of states."
+ "`utility`: Returns the value of the terminal state for a player ('MAX' and 'MIN'). Note that for 'MIN' the value returned is the negative of the utility."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "minimax_decision('A', game52)"
+ "psource(Fig52Game.utility)"
]
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "3\n",
+ "-3\n"
+ ]
+ }
+ ],
"source": [
- "alphabeta_full_search('A', game52)"
+ "print(fig52.utility('B1', 'MAX'))\n",
+ "print(fig52.utility('B1', 'MIN'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now let's play `TicTacToe`. First we initialize the game:"
+ "`terminal_test`: Returns `True` if the given state is a terminal state, `False` otherwise."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "ttt = TicTacToe()"
+ "psource(Fig52Game.terminal_test)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "True\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(fig52.terminal_test('C3'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "We can print a state using the display method:"
+ "`to_move`: Return the player who will move in this state."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "ttt.display(ttt.initial)"
+ "psource(Fig52Game.to_move)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "MAX\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(fig52.to_move('A'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Hmm, so that's the initial state of the game; no X's and no O's. \n",
- " \n",
- " Let us create a new game state by ourselves to experiment:"
+ "As a whole the class `Fig52` that inherits from the class `Game` and overrides its functions:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "my_state = GameState(\n",
- " to_move = 'X',\n",
- " utility = '0',\n",
- " board = {(1,1): 'X', (1,2): 'O', (1,3): 'X',\n",
- " (2,1): 'O', (2,3): 'O',\n",
- " (3,1): 'X',\n",
- " },\n",
- " moves = [(2,2), (3,2), (3,3)]\n",
- " )"
+ "psource(Fig52Game)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "So, how does this game state look like?"
+ "# MIN-MAX\n",
+ "\n",
+ "## Overview\n",
+ "\n",
+ "This algorithm (often called *Minimax*) computes the next move for a player (MIN or MAX) at their current state. It recursively computes the minimax value of successor states, until it reaches terminals (the leaves of the tree). Using the `utility` value of the terminal states, it computes the values of parent states until it reaches the initial node (the root of the tree).\n",
+ "\n",
+ "It is worth noting that the algorithm works in a depth-first manner. The pseudocode can be found below:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/markdown": [
+ "### AIMA3e\n",
+ "__function__ MINIMAX-DECISION(_state_) __returns__ _an action_ \n",
+ " __return__ arg max _a_ ∈ ACTIONS(_s_) MIN\\-VALUE(RESULT(_state_, _a_)) \n",
+ "\n",
+ "---\n",
+ "__function__ MAX\\-VALUE(_state_) __returns__ _a utility value_ \n",
+ " __if__ TERMINAL\\-TEST(_state_) __then return__ UTILITY(_state_) \n",
+ " _v_ ← −∞ \n",
+ " __for each__ _a_ __in__ ACTIONS(_state_) __do__ \n",
+ " _v_ ← MAX(_v_, MIN\\-VALUE(RESULT(_state_, _a_))) \n",
+ " __return__ _v_ \n",
+ "\n",
+ "---\n",
+ "__function__ MIN\\-VALUE(_state_) __returns__ _a utility value_ \n",
+ " __if__ TERMINAL\\-TEST(_state_) __then return__ UTILITY(_state_) \n",
+ " _v_ ← ∞ \n",
+ " __for each__ _a_ __in__ ACTIONS(_state_) __do__ \n",
+ " _v_ ← MIN(_v_, MAX\\-VALUE(RESULT(_state_, _a_))) \n",
+ " __return__ _v_ \n",
+ "\n",
+ "---\n",
+ "__Figure__ ?? An algorithm for calculating minimax decisions. It returns the action corresponding to the best possible move, that is, the move that leads to the outcome with the best utility, under the assumption that the opponent plays to minimize utility. The functions MAX\\-VALUE and MIN\\-VALUE go through the whole game tree, all the way to the leaves, to determine the backed\\-up value of a state. The notation argmax _a_ ∈ _S_ _f_(_a_) computes the element _a_ of set _S_ that has maximum value of _f_(_a_)."
+ ],
+ "text/plain": [
+ "\n",
+ "\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "\n",
+ "\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "\n",
+ "\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "
+
+
+
+
+''' # noqa
+
+with open('js/continuousworld.js', 'r') as js_file:
+ _JS_CONTINUOUS_WORLD = js_file.read()
+
+
+class ContinuousWorldView:
+ """ View for continuousworld Implementation in agents.py """
+
+ def __init__(self, world, fill="#AAA"):
+ self.time = time.time()
+ self.world = world
+ self.width = world.width
+ self.height = world.height
+
+ def object_name(self):
+ globals_in_main = {x: getattr(__main__, x) for x in dir(__main__)}
+ for x in globals_in_main:
+ if isinstance(globals_in_main[x], type(self)):
+ if globals_in_main[x].time == self.time:
+ return x
+
+ def handle_add_obstacle(self, vertices):
+ """ Vertices must be a nestedtuple. This method
+ is called from kernel.execute on completion of
+ a polygon. """
+ self.world.add_obstacle(vertices)
+ self.show()
+
+ def handle_remove_obstacle(self):
+ return NotImplementedError
+
+ def get_polygon_obstacles_coordinates(self):
+ obstacle_coordiantes = []
+ for thing in self.world.things:
+ if isinstance(thing, PolygonObstacle):
+ obstacle_coordiantes.append(thing.coordinates)
+ return obstacle_coordiantes
+
+ def show(self):
+ clear_output()
+ total_html = _CONTINUOUS_WORLD_HTML.format(self.width, self.height, self.object_name(),
+ str(self.get_polygon_obstacles_coordinates()),
+ _JS_CONTINUOUS_WORLD)
+ display(HTML(total_html))
+
+
+# ______________________________________________________________________________
+# Grid environment
+
+_GRID_WORLD_HTML = '''
+
+
+
+
+'''
+
+with open('js/gridworld.js', 'r') as js_file:
+ _JS_GRID_WORLD = js_file.read()
+
+
+class GridWorldView:
+ """ View for grid world. Uses XYEnviornment in agents.py as model.
+ world: an instance of XYEnviornment.
+ block_size: size of individual blocks in pixes.
+ default_fill: color of blocks. A hex value or name should be passed.
+ """
+
+ def __init__(self, world, block_size=30, default_fill="white"):
+ self.time = time.time()
+ self.world = world
+ self.labels = defaultdict(str) # locations as keys
+ self.representation = {"default": {"type": "color", "source": default_fill}}
+ self.block_size = block_size
+
+ def object_name(self):
+ globals_in_main = {x: getattr(__main__, x) for x in dir(__main__)}
+ for x in globals_in_main:
+ if isinstance(globals_in_main[x], type(self)):
+ if globals_in_main[x].time == self.time:
+ return x
+
+ def set_label(self, coordinates, label):
+ """ Add lables to a particular block of grid.
+ coordinates: a tuple of (row, column).
+ rows and columns are 0 indexed.
+ """
+ self.labels[coordinates] = label
+
+ def set_representation(self, thing, repr_type, source):
+ """ Set the representation of different things in the
+ environment.
+ thing: a thing object.
+ repr_type : type of representation can be either "color" or "img"
+ source: Hex value in case of color. Image path in case of image.
+ """
+ thing_class_name = thing.__class__.__name__
+ if repr_type not in ("img", "color"):
+ raise ValueError('Invalid repr_type passed. Possible types are img/color')
+ self.representation[thing_class_name] = {"type": repr_type, "source": source}
+
+ def handle_click(self, coordinates):
+ """ This method needs to be overidden. Make sure to include a
+ self.show() call at the end. """
+ self.show()
+
+ def map_to_render(self):
+ default_representation = {"val": "default", "tooltip": ""}
+ world_map = [[copy.deepcopy(default_representation) for _ in range(self.world.width)]
+ for _ in range(self.world.height)]
+
+ for thing in self.world.things:
+ row, column = thing.location
+ thing_class_name = thing.__class__.__name__
+ if thing_class_name not in self.representation:
+ raise KeyError('Representation not found for {}'.format(thing_class_name))
+ world_map[row][column]["val"] = thing.__class__.__name__
+
+ for location, label in self.labels.items():
+ row, column = location
+ world_map[row][column]["tooltip"] = label
+
+ return json.dumps(world_map)
+
+ def show(self):
+ clear_output()
+ total_html = _GRID_WORLD_HTML.format(
+ self.object_name(), self.map_to_render(),
+ self.block_size, json.dumps(self.representation), _JS_GRID_WORLD)
+ display(HTML(total_html))
diff --git a/js/canvas.js b/js/canvas.js
new file mode 100644
index 000000000..d9d313d2e
--- /dev/null
+++ b/js/canvas.js
@@ -0,0 +1,135 @@
+/*
+ JavaScript functions that are executed by running the corresponding methods of a Canvas object
+ Donot use these functions by making a js file. Instead use the python Canvas class.
+ See canvas.py for help on how to use the Canvas class to draw on the HTML Canvas
+*/
+
+
+//Manages the output of code executed in IPython kernel
+function output_callback(out, block){
+ console.log(out);
+ //Handle error in python
+ if(out.msg_type == "error"){
+ console.log("Error in python script!");
+ console.log(out.content);
+ return ;
+ }
+ script = out.content.data['text/html'];
+ script = script.substr(8, script.length - 17);
+ eval(script)
+}
+
+//Handles mouse click by calling mouse_click of Canvas object with the co-ordinates as arguments
+function click_callback(element, event, varname){
+ var rect = element.getBoundingClientRect();
+ var x = event.clientX - rect.left;
+ var y = event.clientY - rect.top;
+ var kernel = IPython.notebook.kernel;
+ var exec_str = varname + ".mouse_click(" + String(x) + ", " + String(y) + ")";
+ console.log(exec_str);
+ kernel.execute(exec_str,{'iopub': {'output': output_callback}}, {silent: false});
+}
+
+function rgbToHex(r,g,b){
+ var hexValue=(r<<16) + (g<<8) + (b<<0);
+ var hexString=hexValue.toString(16);
+ hexString ='#' + Array(7-hexString.length).join('0') + hexString; //Add 0 padding
+ return hexString;
+}
+
+function toRad(x){
+ return x*Math.PI/180;
+}
+
+//Canvas class to store variables
+function Canvas(id){
+ this.canvas = document.getElementById(id);
+ this.ctx = this.canvas.getContext("2d");
+ this.WIDTH = this.canvas.width;
+ this.HEIGHT = this.canvas.height;
+ this.MOUSE = {x:0,y:0};
+}
+
+//Sets the fill color with which shapes are filled
+Canvas.prototype.fill = function(r, g, b){
+ this.ctx.fillStyle = rgbToHex(r,g,b);
+}
+
+//Set the stroke color
+Canvas.prototype.stroke = function(r, g, b){
+ this.ctx.strokeStyle = rgbToHex(r,g,b);
+}
+
+//Set width of the lines/strokes
+Canvas.prototype.strokeWidth = function(w){
+ this.ctx.lineWidth = w;
+}
+
+//Draw a rectangle with top left at (x,y) with 'w' width and 'h' height
+Canvas.prototype.rect = function(x, y, w, h){
+ this.ctx.fillRect(x,y,w,h);
+}
+
+//Draw a line with (x1, y1) and (x2, y2) as end points
+Canvas.prototype.line = function(x1, y1, x2, y2){
+ this.ctx.beginPath();
+ this.ctx.moveTo(x1, y1);
+ this.ctx.lineTo(x2, y2);
+ this.ctx.stroke();
+}
+
+//Draw an arc with (x, y) as centre, 'r' as radius from angles start to stop
+Canvas.prototype.arc = function(x, y, r, start, stop){
+ this.ctx.beginPath();
+ this.ctx.arc(x, y, r, toRad(start), toRad(stop));
+ this.ctx.stroke();
+}
+
+//Clear the HTML canvas
+Canvas.prototype.clear = function(){
+ this.ctx.clearRect(0, 0, this.WIDTH, this.HEIGHT);
+}
+
+//Change font, size and style
+Canvas.prototype.font = function(font_str){
+ this.ctx.font = font_str;
+}
+
+//Draws "filled" text on the canvas
+Canvas.prototype.fill_text = function(text, x, y){
+ this.ctx.fillText(text, x, y);
+}
+
+//Write text on the canvas
+Canvas.prototype.stroke_text = function(text, x, y){
+ this.ctx.strokeText(text, x, y);
+}
+
+
+//Test if the canvas functions are working
+Canvas.prototype.test_run = function(){
+ var dbg = false;
+ if(dbg)
+ alert("1");
+ this.clear();
+ if(dbg)
+ alert("2");
+ this.fill(0, 200, 0);
+ if(dbg)
+ alert("3");
+ this.rect(this.MOUSE.x, this.MOUSE.y, 100, 200);
+ if(dbg)
+ alert("4");
+ this.stroke(0, 0, 50);
+ if(dbg)
+ alert("5");
+ this.line(0, 0, 100, 100);
+ if(dbg)
+ alert("6");
+ this.stroke(200, 200, 200);
+ if(dbg)
+ alert("7");
+ this.arc(200, 100, 50, 0, 360);
+ if(dbg)
+ alert("8");
+}
diff --git a/js/continuousworld.js b/js/continuousworld.js
new file mode 100644
index 000000000..ab589f6d1
--- /dev/null
+++ b/js/continuousworld.js
@@ -0,0 +1,71 @@
+var latest_output_area ="NONE"; // Jquery object for the DOM element of output area which was used most recently
+function handle_output(out, block){
+ var output = out.content.data["text/html"];
+ latest_output_area.html(output);
+}
+function polygon_complete(canvas, vertices){
+ latest_output_area = $(canvas).parents('.output_subarea');
+ var world_object_name = canvas.dataset.world_name;
+ var command = world_object_name + ".handle_add_obstacle(" + JSON.stringify(vertices) + ")";
+ console.log("Executing Command: " + command);
+ var kernel = IPython.notebook.kernel;
+ var callbacks = { 'iopub' : {'output' : handle_output}};
+ kernel.execute(command,callbacks);
+}
+var canvas , ctx;
+function drawPolygon(array) {
+ ctx.fillStyle = '#f00';
+ ctx.beginPath();
+ ctx.moveTo(array[0][0],array[0][1]);
+ for(var i = 1;i
+
+
+\n", + "$ Background ∧ Hypothesis ∧ Descriptions \\vDash Classifications $\n", + "\n", + "for the __unknown__ $Hypothesis$, given the $Background$ knowledge described by $Descriptions$ and $Classifications$.\n", + "\n", + "\n", + "\n", + "The first approach to ILP works by starting with a very general rule and gradually specializing\n", + "it so that it fits the data.
\n", + "This is essentially what happens in decision-tree learning, where a\n", + "decision tree is gradually grown until it is consistent with the observations.
To do ILP we\n", + "use first-order literals instead of attributes, and the $Hypothesis$ is a set of clauses (set of first order rules, where each rule is similar to a Horn clause) instead of a decision tree.
\n", + "\n", + "\n", + "The FOIL algorithm learns new rules, one at a time, in order to cover all given positive and negative examples.
\n", + "More precicely, FOIL contains an inner and an outer while loop.
\n", + "- __outer loop__: (function __foil()__) add rules until all positive examples are covered.
\n", + " (each rule is a conjuction of literals, which are chosen inside the inner loop)\n", + " \n", + " \n", + "- __inner loop__: (function __new_clause()__) add new literals until all negative examples are covered, and some positive examples are covered.
\n", + " - In each iteration, we select/add the most promising literal, according to an estimate of its utility. (function __new_literal()__)
\n", + " \n", + " - The evaluation function to estimate utility of adding literal $L$ to a set of rules $R$ is (function __gain()__) : \n", + " \n", + " $$ FoilGain(L,R) = t \\big( \\log_2{\\frac{p_1}{p_1+n_1}} - \\log_2{\\frac{p_0}{p_0+n_0}} \\big) $$\n", + " where: \n", + " \n", + " $p_0: \\text{is the number of possitive bindings of rule R } \\\\ n_0: \\text{is the number of negative bindings of R} \\\\ p_1: \\text{is the is the number of possitive bindings of rule R'}\\\\ n_0: \\text{is the number of negative bindings of R'}\\\\ t: \\text{is the number of possitive bindings of rule R that are still covered after adding literal L to R}$\n", + " \n", + " - Calculate the extended examples for the chosen literal (function __extend_example()__)
\n", + " (the set of examples created by extending example with each possible constant value for each new variable in literal)\n", + " \n", + "- Finally, the algorithm returns a disjunction of first order rules (= conjuction of literals)\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class FOIL_container(FolKB):\n",
+ " """Hold the kb and other necessary elements required by FOIL."""\n",
+ "\n",
+ " def __init__(self, clauses=None):\n",
+ " self.const_syms = set()\n",
+ " self.pred_syms = set()\n",
+ " FolKB.__init__(self, clauses)\n",
+ "\n",
+ " def tell(self, sentence):\n",
+ " if is_definite_clause(sentence):\n",
+ " self.clauses.append(sentence)\n",
+ " self.const_syms.update(constant_symbols(sentence))\n",
+ " self.pred_syms.update(predicate_symbols(sentence))\n",
+ " else:\n",
+ " raise Exception("Not a definite clause: {}".format(sentence))\n",
+ "\n",
+ " def foil(self, examples, target):\n",
+ " """Learn a list of first-order horn clauses\n",
+ " 'examples' is a tuple: (positive_examples, negative_examples).\n",
+ " positive_examples and negative_examples are both lists which contain substitutions."""\n",
+ " clauses = []\n",
+ "\n",
+ " pos_examples = examples[0]\n",
+ " neg_examples = examples[1]\n",
+ "\n",
+ " while pos_examples:\n",
+ " clause, extended_pos_examples = self.new_clause((pos_examples, neg_examples), target)\n",
+ " # remove positive examples covered by clause\n",
+ " pos_examples = self.update_examples(target, pos_examples, extended_pos_examples)\n",
+ " clauses.append(clause)\n",
+ "\n",
+ " return clauses\n",
+ "\n",
+ " def new_clause(self, examples, target):\n",
+ " """Find a horn clause which satisfies part of the positive\n",
+ " examples but none of the negative examples.\n",
+ " The horn clause is specified as [consequent, list of antecedents]\n",
+ " Return value is the tuple (horn_clause, extended_positive_examples)."""\n",
+ " clause = [target, []]\n",
+ " # [positive_examples, negative_examples]\n",
+ " extended_examples = examples\n",
+ " while extended_examples[1]:\n",
+ " l = self.choose_literal(self.new_literals(clause), extended_examples)\n",
+ " clause[1].append(l)\n",
+ " extended_examples = [sum([list(self.extend_example(example, l)) for example in\n",
+ " extended_examples[i]], []) for i in range(2)]\n",
+ "\n",
+ " return (clause, extended_examples[0])\n",
+ "\n",
+ " def extend_example(self, example, literal):\n",
+ " """Generate extended examples which satisfy the literal."""\n",
+ " # find all substitutions that satisfy literal\n",
+ " for s in self.ask_generator(subst(example, literal)):\n",
+ " s.update(example)\n",
+ " yield s\n",
+ "\n",
+ " def new_literals(self, clause):\n",
+ " """Generate new literals based on known predicate symbols.\n",
+ " Generated literal must share atleast one variable with clause"""\n",
+ " share_vars = variables(clause[0])\n",
+ " for l in clause[1]:\n",
+ " share_vars.update(variables(l))\n",
+ " for pred, arity in self.pred_syms:\n",
+ " new_vars = {standardize_variables(expr('x')) for _ in range(arity - 1)}\n",
+ " for args in product(share_vars.union(new_vars), repeat=arity):\n",
+ " if any(var in share_vars for var in args):\n",
+ " # make sure we don't return an existing rule\n",
+ " if not Expr(pred, args) in clause[1]:\n",
+ " yield Expr(pred, *[var for var in args])\n",
+ "\n",
+ "\n",
+ " def choose_literal(self, literals, examples): \n",
+ " """Choose the best literal based on the information gain."""\n",
+ "\n",
+ " return max(literals, key = partial(self.gain , examples = examples))\n",
+ "\n",
+ "\n",
+ " def gain(self, l ,examples):\n",
+ " """\n",
+ " Find the utility of each literal when added to the body of the clause. \n",
+ " Utility function is: \n",
+ " gain(R, l) = T * (log_2 (post_pos / (post_pos + post_neg)) - log_2 (pre_pos / (pre_pos + pre_neg)))\n",
+ "\n",
+ " where: \n",
+ " \n",
+ " pre_pos = number of possitive bindings of rule R (=current set of rules)\n",
+ " pre_neg = number of negative bindings of rule R \n",
+ " post_pos = number of possitive bindings of rule R' (= R U {l} )\n",
+ " post_neg = number of negative bindings of rule R' \n",
+ " T = number of possitive bindings of rule R that are still covered \n",
+ " after adding literal l \n",
+ "\n",
+ " """\n",
+ " pre_pos = len(examples[0])\n",
+ " pre_neg = len(examples[1])\n",
+ " post_pos = sum([list(self.extend_example(example, l)) for example in examples[0]], []) \n",
+ " post_neg = sum([list(self.extend_example(example, l)) for example in examples[1]], []) \n",
+ " if pre_pos + pre_neg ==0 or len(post_pos) + len(post_neg)==0:\n",
+ " return -1\n",
+ " # number of positive example that are represented in extended_examples\n",
+ " T = 0\n",
+ " for example in examples[0]:\n",
+ " represents = lambda d: all(d[x] == example[x] for x in example)\n",
+ " if any(represents(l_) for l_ in post_pos):\n",
+ " T += 1\n",
+ " value = T * (log(len(post_pos) / (len(post_pos) + len(post_neg)) + 1e-12,2) - log(pre_pos / (pre_pos + pre_neg),2))\n",
+ " return value\n",
+ "\n",
+ "\n",
+ " def update_examples(self, target, examples, extended_examples):\n",
+ " """Add to the kb those examples what are represented in extended_examples\n",
+ " List of omitted examples is returned."""\n",
+ " uncovered = []\n",
+ " for example in examples:\n",
+ " represents = lambda d: all(d[x] == example[x] for x in example)\n",
+ " if any(represents(l) for l in extended_examples):\n",
+ " self.tell(subst(example, target))\n",
+ " else:\n",
+ " uncovered.append(example)\n",
+ "\n",
+ " return uncovered\n",
+ "\n", + "\n", + "
\n", + "Given some positive and negative examples of the relation 'Parent(x,y)', we want to find a set of rules that satisfies all the examples.
\n", + "\n", + "A definition of Parent is $Parent(x,y) \\Leftrightarrow Mother(x,y) \\lor Father(x,y)$, which is the result that we expect from the algorithm. " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "A, B, C, D, E, F, G, H, I, x, y, z = map(expr, 'ABCDEFGHIxyz')" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "small_family = FOIL_container([expr(\"Mother(Anne, Peter)\"),\n", + " expr(\"Mother(Anne, Zara)\"),\n", + " expr(\"Mother(Sarah, Beatrice)\"),\n", + " expr(\"Mother(Sarah, Eugenie)\"),\n", + " expr(\"Father(Mark, Peter)\"),\n", + " expr(\"Father(Mark, Zara)\"),\n", + " expr(\"Father(Andrew, Beatrice)\"),\n", + " expr(\"Father(Andrew, Eugenie)\"),\n", + " expr(\"Father(Philip, Anne)\"),\n", + " expr(\"Father(Philip, Andrew)\"),\n", + " expr(\"Mother(Elizabeth, Anne)\"),\n", + " expr(\"Mother(Elizabeth, Andrew)\"),\n", + " expr(\"Male(Philip)\"),\n", + " expr(\"Male(Mark)\"),\n", + " expr(\"Male(Andrew)\"),\n", + " expr(\"Male(Peter)\"),\n", + " expr(\"Female(Elizabeth)\"),\n", + " expr(\"Female(Anne)\"),\n", + " expr(\"Female(Sarah)\"),\n", + " expr(\"Female(Zara)\"),\n", + " expr(\"Female(Beatrice)\"),\n", + " expr(\"Female(Eugenie)\"),\n", + "])\n", + "\n", + "target = expr('Parent(x, y)')\n", + "\n", + "examples_pos = [{x: expr('Elizabeth'), y: expr('Anne')},\n", + " {x: expr('Elizabeth'), y: expr('Andrew')},\n", + " {x: expr('Philip'), y: expr('Anne')},\n", + " {x: expr('Philip'), y: expr('Andrew')},\n", + " {x: expr('Anne'), y: expr('Peter')},\n", + " {x: expr('Anne'), y: expr('Zara')},\n", + " {x: expr('Mark'), y: expr('Peter')},\n", + " {x: expr('Mark'), y: expr('Zara')},\n", + " {x: expr('Andrew'), y: expr('Beatrice')},\n", + " {x: expr('Andrew'), y: expr('Eugenie')},\n", + " {x: expr('Sarah'), y: expr('Beatrice')},\n", + " {x: expr('Sarah'), y: expr('Eugenie')}]\n", + "examples_neg = [{x: expr('Anne'), y: expr('Eugenie')},\n", + " {x: expr('Beatrice'), y: expr('Eugenie')},\n", + " {x: expr('Mark'), y: expr('Elizabeth')},\n", + " {x: expr('Beatrice'), y: expr('Philip')}]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[Parent(x, y), [Father(x, y)]], [Parent(x, y), [Mother(x, y)]]]\n" + ] + } + ], + "source": [ + "# run the FOIL algorithm \n", + "clauses = small_family.foil([examples_pos, examples_neg], target)\n", + "print (clauses)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Indeed the algorithm returned the rule: \n", + "
$Parent(x,y) \\Leftrightarrow Mother(x,y) \\lor Father(x,y)$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Suppose that we have some positive and negative results for the relation 'GrandParent(x,y)' and we want to find a set of rules that satisfies the examples.
\n", + "One possible set of rules for the relation $Grandparent(x,y)$ could be:
\n", + "\n", + "
\n", + "Or, if $Background$ included the sentence $Parent(x,y) \\Leftrightarrow [Mother(x,y) \\lor Father(x,y)]$ then: \n", + "\n", + "$$Grandparent(x,y) \\Leftrightarrow \\exists \\: z \\quad Parent(x,z) \\land Parent(z,y)$$\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[Grandparent(x, y), [Parent(x, v_6), Parent(v_6, y)]]]\n" + ] + } + ], + "source": [ + "target = expr('Grandparent(x, y)')\n", + "\n", + "examples_pos = [{x: expr('Elizabeth'), y: expr('Peter')},\n", + " {x: expr('Elizabeth'), y: expr('Zara')},\n", + " {x: expr('Elizabeth'), y: expr('Beatrice')},\n", + " {x: expr('Elizabeth'), y: expr('Eugenie')},\n", + " {x: expr('Philip'), y: expr('Peter')},\n", + " {x: expr('Philip'), y: expr('Zara')},\n", + " {x: expr('Philip'), y: expr('Beatrice')},\n", + " {x: expr('Philip'), y: expr('Eugenie')}]\n", + "examples_neg = [{x: expr('Anne'), y: expr('Eugenie')},\n", + " {x: expr('Beatrice'), y: expr('Eugenie')},\n", + " {x: expr('Elizabeth'), y: expr('Andrew')},\n", + " {x: expr('Elizabeth'), y: expr('Anne')},\n", + " {x: expr('Elizabeth'), y: expr('Mark')},\n", + " {x: expr('Elizabeth'), y: expr('Sarah')},\n", + " {x: expr('Philip'), y: expr('Anne')},\n", + " {x: expr('Philip'), y: expr('Andrew')},\n", + " {x: expr('Anne'), y: expr('Peter')},\n", + " {x: expr('Anne'), y: expr('Zara')},\n", + " {x: expr('Mark'), y: expr('Peter')},\n", + " {x: expr('Mark'), y: expr('Zara')},\n", + " {x: expr('Andrew'), y: expr('Beatrice')},\n", + " {x: expr('Andrew'), y: expr('Eugenie')},\n", + " {x: expr('Sarah'), y: expr('Beatrice')},\n", + " {x: expr('Mark'), y: expr('Elizabeth')},\n", + " {x: expr('Beatrice'), y: expr('Philip')}, \n", + " {x: expr('Peter'), y: expr('Andrew')}, \n", + " {x: expr('Zara'), y: expr('Mark')},\n", + " {x: expr('Peter'), y: expr('Anne')},\n", + " {x: expr('Zara'), y: expr('Eugenie')}, ]\n", + "\n", + "clauses = small_family.foil([examples_pos, examples_neg], target)\n", + "\n", + "print(clauses)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Indeed the algorithm returned the rule: \n", + "
$Grandparent(x,y) \\Leftrightarrow \\exists \\: v \\: \\: Parent(x,v) \\land Parent(v,y)$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example Network\n", + "\n", + "Suppose that we have the following directed graph and we want to find a rule that describes the reachability between two nodes (Reach(x,y)).
\n", + "Such a rule could be recursive, since y can be reached from x if and only if there is a sequence of adjacent nodes from x to y: \n", + "\n", + "$$ Reach(x,y) \\Leftrightarrow \\begin{cases} \n", + " Conn(x,y), \\: \\text{(if there is a directed edge from x to y)} \\\\\n", + " \\lor \\quad \\exists \\: z \\quad Reach(x,z) \\land Reach(z,y) \\end{cases}$$\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "\"\"\"\n", + "A H\n", + "|\\ /|\n", + "| \\ / |\n", + "v v v v\n", + "B D-->E-->G-->I\n", + "| / |\n", + "| / |\n", + "vv v\n", + "C F\n", + "\"\"\"\n", + "small_network = FOIL_container([expr(\"Conn(A, B)\"),\n", + " expr(\"Conn(A ,D)\"),\n", + " expr(\"Conn(B, C)\"),\n", + " expr(\"Conn(D, C)\"),\n", + " expr(\"Conn(D, E)\"),\n", + " expr(\"Conn(E ,F)\"),\n", + " expr(\"Conn(E, G)\"),\n", + " expr(\"Conn(G, I)\"),\n", + " expr(\"Conn(H, G)\"),\n", + " expr(\"Conn(H, I)\")])\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[Reach(x, y), [Conn(x, y)]], [Reach(x, y), [Reach(x, v_12), Reach(v_14, y), Reach(v_12, v_16), Reach(v_12, y)]], [Reach(x, y), [Reach(x, v_20), Reach(v_20, y)]]]\n" + ] + } + ], + "source": [ + "target = expr('Reach(x, y)')\n", + "examples_pos = [{x: A, y: B},\n", + " {x: A, y: C},\n", + " {x: A, y: D},\n", + " {x: A, y: E},\n", + " {x: A, y: F},\n", + " {x: A, y: G},\n", + " {x: A, y: I},\n", + " {x: B, y: C},\n", + " {x: D, y: C},\n", + " {x: D, y: E},\n", + " {x: D, y: F},\n", + " {x: D, y: G},\n", + " {x: D, y: I},\n", + " {x: E, y: F},\n", + " {x: E, y: G},\n", + " {x: E, y: I},\n", + " {x: G, y: I},\n", + " {x: H, y: G},\n", + " {x: H, y: I}]\n", + "nodes = {A, B, C, D, E, F, G, H, I}\n", + "examples_neg = [example for example in [{x: a, y: b} for a in nodes for b in nodes]\n", + " if example not in examples_pos]\n", + "clauses = small_network.foil([examples_pos, examples_neg], target)\n", + "\n", + "print(clauses)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The algorithm produced something close to the recursive rule: \n", + " $$ Reach(x,y) \\Leftrightarrow [Conn(x,y)] \\: \\lor \\: [\\exists \\: z \\: \\: Reach(x,z) \\, \\land \\, Reach(z,y)]$$\n", + " \n", + "This happened because the size of the example is small. " + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.5" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} \ No newline at end of file diff --git a/knowledge_current_best.ipynb b/knowledge_current_best.ipynb new file mode 100644 index 000000000..5da492cd0 --- /dev/null +++ b/knowledge_current_best.ipynb @@ -0,0 +1,662 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# KNOWLEDGE\n", + "\n", + "The [knowledge](https://github.com/aimacode/aima-python/blob/master/knowledge.py) module covers **Chapter 19: Knowledge in Learning** from Stuart Russel's and Peter Norvig's book *Artificial Intelligence: A Modern Approach*.\n", + "\n", + "Execute the cell below to get started." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from knowledge import *\n", + "\n", + "from notebook import pseudocode, psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "\n", + "* Overview\n", + "* Current-Best Learning" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## OVERVIEW\n", + "\n", + "Like the [learning module](https://github.com/aimacode/aima-python/blob/master/learning.ipynb), this chapter focuses on methods for generating a model/hypothesis for a domain; however, unlike the learning chapter, here we use prior knowledge to help us learn from new experiences and find a proper hypothesis.\n", + "\n", + "### First-Order Logic\n", + "\n", + "Usually knowledge in this field is represented as **first-order logic**; a type of logic that uses variables and quantifiers in logical sentences. Hypotheses are represented by logical sentences with variables, while examples are logical sentences with set values instead of variables. The goal is to assign a value to a special first-order logic predicate, called **goal predicate**, for new examples given a hypothesis. We learn this hypothesis by infering knowledge from some given examples.\n", + "\n", + "### Representation\n", + "\n", + "In this module, we use dictionaries to represent examples, with keys being the attribute names and values being the corresponding example values. Examples also have an extra boolean field, 'GOAL', for the goal predicate. A hypothesis is represented as a list of dictionaries. Each dictionary in that list represents a disjunction. Inside these dictionaries/disjunctions we have conjunctions.\n", + "\n", + "For example, say we want to predict if an animal (cat or dog) will take an umbrella given whether or not it rains or the animal wears a coat. The goal value is 'take an umbrella' and is denoted by the key 'GOAL'. An example:\n", + "\n", + "`{'Species': 'Cat', 'Coat': 'Yes', 'Rain': 'Yes', 'GOAL': True}`\n", + "\n", + "A hypothesis can be the following:\n", + "\n", + "`[{'Species': 'Cat'}]`\n", + "\n", + "which means an animal will take an umbrella if and only if it is a cat.\n", + "\n", + "### Consistency\n", + "\n", + "We say that an example `e` is **consistent** with an hypothesis `h` if the assignment from the hypothesis for `e` is the same as `e['GOAL']`. If the above example and hypothesis are `e` and `h` respectively, then `e` is consistent with `h` since `e['Species'] == 'Cat'`. For `e = {'Species': 'Dog', 'Coat': 'Yes', 'Rain': 'Yes', 'GOAL': True}`, the example is no longer consistent with `h`, since the value assigned to `e` is *False* while `e['GOAL']` is *True*." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "## CURRENT-BEST LEARNING\n", + "\n", + "### Overview\n", + "\n", + "In **Current-Best Learning**, we start with a hypothesis and we refine it as we iterate through the examples. For each example, there are three possible outcomes: the example is consistent with the hypothesis, the example is a **false positive** (real value is false but got predicted as true) and the example is a **false negative** (real value is true but got predicted as false). Depending on the outcome we refine the hypothesis accordingly:\n", + "\n", + "* Consistent: We do not change the hypothesis and move on to the next example.\n", + "\n", + "* False Positive: We **specialize** the hypothesis, which means we add a conjunction.\n", + "\n", + "* False Negative: We **generalize** the hypothesis, either by removing a conjunction or a disjunction, or by adding a disjunction.\n", + "\n", + "When specializing or generalizing, we should make sure to not create inconsistencies with previous examples. To avoid that caveat, backtracking is needed. Thankfully, there is not just one specialization or generalization, so we have a lot to choose from. We will go through all the specializations/generalizations and we will refine our hypothesis as the first specialization/generalization consistent with all the examples seen up to that point." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Pseudocode" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "### AIMA3e\n", + "__function__ Current-Best-Learning(_examples_, _h_) __returns__ a hypothesis or fail \n", + " __if__ _examples_ is empty __then__ \n", + " __return__ _h_ \n", + " _e_ ← First(_examples_) \n", + " __if__ _e_ is consistent with _h_ __then__ \n", + " __return__ Current-Best-Learning(Rest(_examples_), _h_) \n", + " __else if__ _e_ is a false positive for _h_ __then__ \n", + " __for each__ _h'_ __in__ specializations of _h_ consistent with _examples_ seen so far __do__ \n", + " _h''_ ← Current-Best-Learning(Rest(_examples_), _h'_) \n", + " __if__ _h''_ ≠ _fail_ __then return__ _h''_ \n", + " __else if__ _e_ is a false negative for _h_ __then__ \n", + " __for each__ _h'_ __in__ generalizations of _h_ consistent with _examples_ seen so far __do__ \n", + " _h''_ ← Current-Best-Learning(Rest(_examples_), _h'_) \n", + " __if__ _h''_ ≠ _fail_ __then return__ _h''_ \n", + " __return__ _fail_ \n", + "\n", + "---\n", + "__Figure ??__ The current-best-hypothesis learning algorithm. It searches for a consistent hypothesis that fits all the examples and backtracks when no consistent specialization/generalization can be found. To start the algorithm, any hypothesis can be passed in; it will be specialized or generalized as needed." + ], + "text/plain": [ + "
def current_best_learning(examples, h, examples_so_far=None):\n",
+ " """ [Figure 19.2]\n",
+ " The hypothesis is a list of dictionaries, with each dictionary representing\n",
+ " a disjunction."""\n",
+ " if not examples:\n",
+ " return h\n",
+ "\n",
+ " examples_so_far = examples_so_far or []\n",
+ " e = examples[0]\n",
+ " if is_consistent(e, h):\n",
+ " return current_best_learning(examples[1:], h, examples_so_far + [e])\n",
+ " elif false_positive(e, h):\n",
+ " for h2 in specializations(examples_so_far + [e], h):\n",
+ " h3 = current_best_learning(examples[1:], h2, examples_so_far + [e])\n",
+ " if h3 != 'FAIL':\n",
+ " return h3\n",
+ " elif false_negative(e, h):\n",
+ " for h2 in generalizations(examples_so_far + [e], h):\n",
+ " h3 = current_best_learning(examples[1:], h2, examples_so_far + [e])\n",
+ " if h3 != 'FAIL':\n",
+ " return h3\n",
+ "\n",
+ " return 'FAIL'\n",
+ "\n",
+ "\n",
+ "def specializations(examples_so_far, h):\n",
+ " """Specialize the hypothesis by adding AND operations to the disjunctions"""\n",
+ " hypotheses = []\n",
+ "\n",
+ " for i, disj in enumerate(h):\n",
+ " for e in examples_so_far:\n",
+ " for k, v in e.items():\n",
+ " if k in disj or k == 'GOAL':\n",
+ " continue\n",
+ "\n",
+ " h2 = h[i].copy()\n",
+ " h2[k] = '!' + v\n",
+ " h3 = h.copy()\n",
+ " h3[i] = h2\n",
+ " if check_all_consistency(examples_so_far, h3):\n",
+ " hypotheses.append(h3)\n",
+ "\n",
+ " shuffle(hypotheses)\n",
+ " return hypotheses\n",
+ "\n",
+ "\n",
+ "def generalizations(examples_so_far, h):\n",
+ " """Generalize the hypothesis. First delete operations\n",
+ " (including disjunctions) from the hypothesis. Then, add OR operations."""\n",
+ " hypotheses = []\n",
+ "\n",
+ " # Delete disjunctions\n",
+ " disj_powerset = powerset(range(len(h)))\n",
+ " for disjs in disj_powerset:\n",
+ " h2 = h.copy()\n",
+ " for d in reversed(list(disjs)):\n",
+ " del h2[d]\n",
+ "\n",
+ " if check_all_consistency(examples_so_far, h2):\n",
+ " hypotheses += h2\n",
+ "\n",
+ " # Delete AND operations in disjunctions\n",
+ " for i, disj in enumerate(h):\n",
+ " a_powerset = powerset(disj.keys())\n",
+ " for attrs in a_powerset:\n",
+ " h2 = h[i].copy()\n",
+ " for a in attrs:\n",
+ " del h2[a]\n",
+ "\n",
+ " if check_all_consistency(examples_so_far, [h2]):\n",
+ " h3 = h.copy()\n",
+ " h3[i] = h2.copy()\n",
+ " hypotheses += h3\n",
+ "\n",
+ " # Add OR operations\n",
+ " if hypotheses == [] or hypotheses == [{}]:\n",
+ " hypotheses = add_or(examples_so_far, h)\n",
+ " else:\n",
+ " hypotheses.extend(add_or(examples_so_far, h))\n",
+ "\n",
+ " shuffle(hypotheses)\n",
+ " return hypotheses\n",
+ "def version_space_learning(examples):\n",
+ " """ [Figure 19.3]\n",
+ " The version space is a list of hypotheses, which in turn are a list\n",
+ " of dictionaries/disjunctions."""\n",
+ " V = all_hypotheses(examples)\n",
+ " for e in examples:\n",
+ " if V:\n",
+ " V = version_space_update(V, e)\n",
+ "\n",
+ " return V\n",
+ "\n",
+ "\n",
+ "def version_space_update(V, e):\n",
+ " return [h for h in V if is_consistent(e, h)]\n",
+ "def all_hypotheses(examples):\n",
+ " """Build a list of all the possible hypotheses"""\n",
+ " values = values_table(examples)\n",
+ " h_powerset = powerset(values.keys())\n",
+ " hypotheses = []\n",
+ " for s in h_powerset:\n",
+ " hypotheses.extend(build_attr_combinations(s, values))\n",
+ "\n",
+ " hypotheses.extend(build_h_combinations(hypotheses))\n",
+ "\n",
+ " return hypotheses\n",
+ "\n",
+ "\n",
+ "def values_table(examples):\n",
+ " """Build a table with all the possible values for each attribute.\n",
+ " Returns a dictionary with keys the attribute names and values a list\n",
+ " with the possible values for the corresponding attribute."""\n",
+ " values = defaultdict(lambda: [])\n",
+ " for e in examples:\n",
+ " for k, v in e.items():\n",
+ " if k == 'GOAL':\n",
+ " continue\n",
+ "\n",
+ " mod = '!'\n",
+ " if e['GOAL']:\n",
+ " mod = ''\n",
+ "\n",
+ " if mod + v not in values[k]:\n",
+ " values[k].append(mod + v)\n",
+ "\n",
+ " values = dict(values)\n",
+ " return values\n",
+ "def build_attr_combinations(s, values):\n",
+ " """Given a set of attributes, builds all the combinations of values.\n",
+ " If the set holds more than one attribute, recursively builds the\n",
+ " combinations."""\n",
+ " if len(s) == 1:\n",
+ " # s holds just one attribute, return its list of values\n",
+ " k = values[s[0]]\n",
+ " h = [[{s[0]: v}] for v in values[s[0]]]\n",
+ " return h\n",
+ "\n",
+ " h = []\n",
+ " for i, a in enumerate(s):\n",
+ " rest = build_attr_combinations(s[i+1:], values)\n",
+ " for v in values[a]:\n",
+ " o = {a: v}\n",
+ " for r in rest:\n",
+ " t = o.copy()\n",
+ " for d in r:\n",
+ " t.update(d)\n",
+ " h.append([t])\n",
+ "\n",
+ " return h\n",
+ "\n",
+ "\n",
+ "def build_h_combinations(hypotheses):\n",
+ " """Given a set of hypotheses, builds and returns all the combinations of the\n",
+ " hypotheses."""\n",
+ " h = []\n",
+ " h_powerset = powerset(range(len(hypotheses)))\n",
+ "\n",
+ " for s in h_powerset:\n",
+ " t = []\n",
+ " for i in s:\n",
+ " t.extend(hypotheses[i])\n",
+ " h.append(t)\n",
+ "\n",
+ " return h\n",
+ "def minimal_consistent_det(E, A):\n",
+ " """Return a minimal set of attributes which give consistent determination"""\n",
+ " n = len(A)\n",
+ "\n",
+ " for i in range(n + 1):\n",
+ " for A_i in combinations(A, i):\n",
+ " if consistent_det(A_i, E):\n",
+ " return set(A_i)\n",
+ "def consistent_det(A, E):\n",
+ " """Check if the attributes(A) is consistent with the examples(E)"""\n",
+ " H = {}\n",
+ "\n",
+ " for e in E:\n",
+ " attr_values = tuple(e[attr] for attr in A)\n",
+ " if attr_values in H and H[attr_values] != e['GOAL']:\n",
+ " return False\n",
+ " H[attr_values] = e['GOAL']\n",
+ "\n",
+ " return True\n",
+ "- \n",
+ "
- If the input at the current step has no training data we return the mode of classes of input data received in the parent step (previous level of recursion). \n", + "
- If all values in training data belong to the same class it returns a `DecisionLeaf` whose class label is the class which all the data belongs to. \n", + "
- If the data has no attributes that can be tested we return the class with highest plurality value in the training data. \n", + "
- We choose the attribute which gives the highest amount of entropy gain and return a `DecisionFork` which splits based on this attribute. Each branch recursively calls `decision_tree_learning` to construct the sub-tree. \n", + "
def AdaBoost(L, K):\n",
+ " """[Figure 18.34]"""\n",
+ " def train(dataset):\n",
+ " examples, target = dataset.examples, dataset.target\n",
+ " N = len(examples)\n",
+ " epsilon = 1. / (2 * N)\n",
+ " w = [1. / N] * N\n",
+ " h, z = [], []\n",
+ " for k in range(K):\n",
+ " h_k = L(dataset, w)\n",
+ " h.append(h_k)\n",
+ " error = sum(weight for example, weight in zip(examples, w)\n",
+ " if example[target] != h_k(example))\n",
+ " # Avoid divide-by-0 from either 0% or 100% error rates:\n",
+ " error = clip(error, epsilon, 1 - epsilon)\n",
+ " for j, example in enumerate(examples):\n",
+ " if example[target] == h_k(example):\n",
+ " w[j] *= error / (1. - error)\n",
+ " w = normalize(w)\n",
+ " z.append(math.log((1. - error) / error))\n",
+ " return WeightedMajority(h, z)\n",
+ " return train\n",
+ "\n", - "*Date: 23rd March 2016*\n", - "\n", - "---" + "# Logic" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## An Introduction\n", - "\n", - "Hello reader.
\n", - "In this IPython notebook, I will help you a little so that you will become more comfortable with using the [logic.py](https://github.com/aimacode/aima-python/blob/master/logic.py) module. The `logic.py` module implements the algorithms given in Chapter 6 (Logical Agents), Chapter 7 (First-Order Logic) and Chapter 8 (Inference in First-Order Logic) of the book *Artificial Intelligence: A Modern Approach*.\n", - "\n", - "Before we begin, if you are unsure of how to use the [aima-python](https://github.com/aimacode/aima-python) repository or are not familiar with IPython notebooks you should read the [intro notebook](https://github.com/aimacode/aima-python/blob/master/intro.ipynb) first. Also, if you are more comfortable with Java than you are with Python we also have the [aima-java](https://github.com/aimacode/aima-java) repository.\n", - "\n", - "I am assuming that you have read at least Chapter 7 (Logical Agents). You really want to do this if you intend to make sense of anything I tell you in this notebook, or any code in the `logic.py` module, for that matter. If you haven't you should go back and read this chapter first, at least upto Sec. 7.5. As a side note, be sure to keep the `logic.py` module open and keep referring to it as you read this notebook. The docstrings of most classes and functions are well-written and will give you more insight and in some cases, even examples, of how to use that particular class or function.\n", + "This Jupyter notebook acts as supporting material for topics covered in __Chapter 6 Logical Agents__, __Chapter 7 First-Order Logic__ and __Chapter 8 Inference in First-Order Logic__ of the book *[Artificial Intelligence: A Modern Approach](http://aima.cs.berkeley.edu)*. We make use of the implementations in the [logic.py](https://github.com/aimacode/aima-python/blob/master/logic.py) module. See the [intro notebook](https://github.com/aimacode/aima-python/blob/master/intro.ipynb) for instructions.\n", "\n", - "To briefly outline how I will proceed in this notebook, I will start by telling you more about the classes `KB` and `ProbKB`, the classes for the Knowledge Bases that we will be using. Next, we will begin with Propositional Logic; only after we are mostly done with it, we will be getting into First-Order Logic. In Propositional Logic, we will have a look at the class `Expr` and the `expr` function, and try to get more comfortable with using them to create and manipulate logical expressions. We will also play a little with other utility functions created to make working with statements easy. Then, we will construct a knowledge base of a specific situation in the Wumpus World. We will next go through the `tt_entails` function and experiment with it a bit. The `pl_resolution` and `pl_fc_entails` functions will come next.\n", - "\n", - "So let's get started." + "Let's first import everything from the `logic` module." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from utils import *\n", + "from logic import *\n", + "from notebook import psource" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Knowledge Bases: `KB` and `PropKB`\n", - "\n", - "The class `KB` is just a template class which you have to inherit to create a knowledge base class that you plan to use. This class reminds you to implement all the methods mentioned here and will scream at you if you forget to. It is, what you might call in Java, an abstract class. The class `PropKB` has been derived from the class `KB` and all the methods have been implemented in here. Let's have a look at these classes in somewhat more detail.\n", - "\n", - "We see that the class `KB` has four methods, apart from `__init__`. A point to note here: the `ask` method simply calls the `ask_generator` method. Thus, this one has already been implemented and what you'll have to actually implement when you create your own knowledge base class (if you want to, though I doubt you'll ever need to; just use the ones we've created for you), will be the `ask_generator` function and not the `ask` function itself.\n", - "\n", - "The class `PropKB` now.\n", - "* `__init__(self, sentence=None)` : The constructor `__init__` creates a single field `clauses` which will be a list of all the sentences of the knowledge base. Note that each one of these sentences will be a 'clause' i.e. a sentence which is made up of only literals and `or`s.\n", - "* `tell(self, sentence)` : When you want to add a sentence to the KB, you use the `tell` method. This method takes a sentence, converts it to its CNF, extracts all the clauses, and adds all these clauses to the `clauses` field. So, you need not worry about `tell`ing only clauses to the knowledge base. You can `tell` the knowledge base a sentence in any form that you wish; converting it to CNF and adding the resulting clauses will be handled by the `tell` method.\n", - "* `ask_generator(self, query)` : The `ask_generator` function is used by the `ask` function. It calls the `tt_entails` function, which in turn returns `True` if the knowledge base entails query and `False` otherwise. The `ask_generator` itself returns an empty dict `{}` if the knowledge base entails query and `None` otherwise. This might seem a little bit weird to you. After all, it makes more sense just to return a `True` or a `False` instead of the `{}` or `None` But this is done to maintain consistency with the way things are in First-Order Logic, where, an `ask_generator` function, is supposed to return all the substitutions that make the query true. Hence the dict, to return all these substitutions. I will be mostly be using the `ask` function which returns a `{}` or a `False`, but if you don't like this, you can always use the `ask_if_true` function which returns a `True` or a `False`.\n", - "* `retract(self, sentence)` : This function removes all the clauses of the sentence given, from the knowledge base. Like the `tell` function, you don't have to pass clauses to remove them from the knowledge base; any sentence will do fine. The function will take care of converting that sentence to clauses and then remove those." + "## CONTENTS\n", + "- Logical sentences\n", + " - Expr\n", + " - PropKB\n", + " - Knowledge-based agents\n", + " - Inference in propositional knowledge base\n", + " - Truth table enumeration\n", + " - Proof by resolution\n", + " - Forward and backward chaining\n", + " - DPLL\n", + " - WalkSAT\n", + " - SATPlan\n", + " - FolKB\n", + " - Inference in first order knowledge base\n", + " - Unification\n", + " - Forward chaining algorithm\n", + " - Backward chaining algorithm" ] }, { @@ -54,110 +60,124 @@ "collapsed": true }, "source": [ - "## Getting started with Propositional Logic" + "## Logical Sentences" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### The `Expr` class\n", - "\n", - "The `Expr` class is the one that enables us to work with propositional logic. This class, combined with the `expr` function will enable us to work with propositional logic with much ease.\n", - "\n", - "An instance of the `Expr` class, an `Expr` object represents a symbolic mathematical expression. Truth be told, this class can handle not just Propositional Logic but also First-Order Logic. (As a matter of fact, you can also do arithmetic using this class but you would just be introducing unnecessary complication for a simple task). For the case of our Propositional Logic, an `Expr` object represents a propositional sentence. If you will have a look at its `__init__`, you will see that an `Expr` object just stores the operator and the arguments of a propositional sentence. This is important to note. The `Expr` class does not define the *logic* of Propositional Logic; nor will we be defining it ourselves. It just gives you a way to *represent* expressions. You won't be able to do any propositional math using `Expr`; you won't be be able assign a value of `True` to `P` and `False` to `Q` and then do a `P` ∧ `Q` to get `False`. No, you won't be able to do that. What you will be able to do is to create a representation of sentence and assign it to `P`. Something like,\n", - "\n", - "```python\n", - "sent = Expr(\"==>\", \"A & B\", \"C\")\n", - "```\n", - "\n", - "which is represents the sentence\n", - "\n", - "> (A ∧ B) → C\n", - "\n", - "That's not much, you say. We can create representations of sentences using strings, you continue. Well, we manipulate the `Expr` objects to convert a sentence to its CNF (`to_cnf`), check satisfiability of a sentence (`dpll_satisfiable`), use resolution to find out if a knowledge base entails a sentence (`pl_resolution`) and whatnot. Best of luck doing that with your string representations!\n", - "\n", - "So, the point to take away from the last two paragraphs: The `Expr` class just allows you to create good, easily manipulable representations of propositional sentences. It does a little more than that though. Before I get into that let us create a few expressions of our own to experiment with them later on." + "The `Expr` class is designed to represent any kind of mathematical expression. The simplest type of `Expr` is a symbol, which can be defined with the function `Symbol`:" ] }, { "cell_type": "code", "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "x" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Symbol('x')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Or we can define multiple symbols at the same time with the function `symbols`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ - "from logic import *\n", - "\n", - "P = Expr(\"P\")\n", - "Q = Expr(\"Q\")\n", - "R1 = Expr(\"&\", \"A\", \"B\")\n", - "R2 = Expr(\"==>\", \"C | D\", \"E\")" + "(x, y, P, Q, f) = symbols('x, y, P, Q, f')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note here that you can create expressions that have no operators (like the literals `P` and `Q`), simple expressions of literals (like the sentence R1 which represents `A` ∧ `B`) and also expressions that have, as their arguments, complex sentences represented as strings. But, these strings that are allowed as arguments, can use only certain symbols in them. This is the list of symbols that you should use when you want to put complex sentences as arguments to the `Expr` constructor:\n", - "\n", - "| Operation | Propositional Symbol | Operator to use in Code |\n", - "|--------------------------|----------------------|-------------------------|\n", - "| Negation | ¬ | ~ |\n", - "| And | ∧ | & |\n", - "| Or | ∨ | | |\n", - "| Implies | → | >> or ==> |\n", - "| Biconditional | ↔ | % or <=> |\n", - "| **Some additional ones** | | |\n", - "| Inequality (Xor) | (Dunno) | =/= or ^ |\n", - "| Reverse Implication | ← | << or <== |\n", - "\n", - "Also, this is the precedence sequence with which the operators will be evaluated in code. The highest precedence operators are at the top:\n", - "\n", - " ~\n", - " % <=>\n", - " << <== >> ==>\n", - " &\n", - " ^\n", - " |\n", - " \n", - "Note that the `<=>` and the implication operators are quite at the top. So make sure to use parenthesis correctly when using them with others like `&`, `^` and `|`. You might note that the precedence of these operators is the same as that in Python language. This is not just a coincidence. More about this later.\n", - "\n", - "Getting back to the `Expr` class and the expressions that we have created, lets create a more complex expression from the ones we have already created:" + "We can combine `Expr`s with the regular Python infix and prefix operators. Here's how we would form the logical sentence \"P and not Q\":" ] }, { "cell_type": "code", "execution_count": 4, - "metadata": { - "collapsed": true - }, - "outputs": [], + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(P & ~Q)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "# Cell: Creating complicated sentences\n", - "R3 = Expr(\"<=>\", R1, Q)\n", - "R4 = Expr(\"==>\", R2, P & ~Q)" + "P & ~Q" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "So, these are the expressions that we've created now. To display these expressions in a nice, intuitive form, the `__repr__` method has been implemented accordingly. It called when we put the variable in the interpreter or when we use the `print` function. Let's try both:" + "This works because the `Expr` class overloads the `&` operator with this definition:\n", + "\n", + "```python\n", + "def __and__(self, other): return Expr('&', self, other)```\n", + " \n", + "and does similar overloads for the other operators. An `Expr` has two fields: `op` for the operator, which is always a string, and `args` for the arguments, which is a tuple of 0 or more expressions. By \"expression,\" I mean either an instance of `Expr`, or a number. Let's take a look at the fields for some `Expr` examples:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'&'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sentence = P & ~Q\n", + "\n", + "sentence.op" ] }, { "cell_type": "code", "execution_count": 6, - "metadata": { - "collapsed": false - }, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "P" + "(P, ~Q)" ] }, "execution_count": 6, @@ -166,20 +186,18 @@ } ], "source": [ - "P" + "sentence.args" ] }, { "cell_type": "code", "execution_count": 7, - "metadata": { - "collapsed": false - }, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(A & B)" + "'P'" ] }, "execution_count": 7, @@ -188,20 +206,38 @@ } ], "source": [ - "R1" + "P.op" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "()" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "P.args" ] }, { "cell_type": "code", "execution_count": 9, - "metadata": { - "collapsed": false - }, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(((C | D) ==> E) ==> (P & ~Q))" + "'P'" ] }, "execution_count": 9, @@ -210,119 +246,4738 @@ } ], "source": [ - "R4" + "Pxy = P(x, y)\n", + "\n", + "Pxy.op" ] }, { "cell_type": "code", "execution_count": 10, - "metadata": { - "collapsed": false - }, + "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "P\n", - "(A & B)\n", - "(((C | D) ==> E) ==> (P & ~Q))\n" - ] + "data": { + "text/plain": [ + "(x, y)" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ - "print(P)\n", - "print(R1)\n", - "print(R4)" + "Pxy.args" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "So, that's how it works. Now scroll above a little and have a look at the cell titled \"Creating complicated sentences\". Do you notice something amiss? Now note that, the third argument in the 2nd line is `P & ~Q`. Now, how is that done? It's a statement, for sure, but it's not in the form of a string. As a matter of fact, `P` and `Q` are both `Expr`s themselves.\n", - "\n", - "This is made possible because the `Expr` class overloads many operators. (It actually overloads mostly all the operators available in Python, but don't use them all; not, at least, for Propositional Logic.) Hence, you can do things like `P & ~Q` to *create `Expr`s by directly combining existing `Exprs`*. You might not immediately recognize the power and ease that this grants you, but I'll explain. Once, you have created some small, rudimentary expressions, you can use these overloaded operators to directly get your desired expressions; no need of using the `Expr` constructor each time. See how simple doing all that we did above becomes:" + "It is important to note that the `Expr` class does not define the *logic* of Propositional Logic sentences; it just gives you a way to *represent* expressions. Think of an `Expr` as an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree). Each of the `args` in an `Expr` can be either a symbol, a number, or a nested `Expr`. We can nest these trees to any depth. Here is a deply nested `Expr`:" ] }, { "cell_type": "code", "execution_count": 11, - "metadata": { - "collapsed": false - }, + "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "P\n", - "(A & B)\n", - "(((C | D) >> E) >> (P & ~Q))\n" - ] + "data": { + "text/plain": [ + "(((3 * f(x, y)) + (P(y) / 2)) + 1)" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ - "# Some simple, rudimentary sentences\n", - "P = Expr(\"P\")\n", - "Q = Expr(\"Q\")\n", - "A = Expr(\"A\")\n", - "B = Expr(\"B\")\n", - "C = Expr(\"C\")\n", - "D = Expr(\"D\")\n", - "E = Expr(\"E\")\n", + "3 * f(x, y) + P(y) / 2 + 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Operators for Constructing Logical Sentences\n", "\n", - "# Now for our complex expressions\n", - "R1 = A & B\n", - "R2 = (C | D) >> E\n", + "Here is a table of the operators that can be used to form sentences. Note that we have a problem: we want to use Python operators to make sentences, so that our programs (and our interactive sessions like the one here) will show simple code. But Python does not allow implication arrows as operators, so for now we have to use a more verbose notation that Python does allow: `|'==>'|` instead of just `==>`. Alternately, you can always use the more verbose `Expr` constructor forms:\n", "\n", - "# And the more complex expressions\n", - "R3 = R1 % Q\n", - "R4 = R2 >> (P & ~Q)\n", + "| Operation | Book | Python Infix Input | Python Output | Python `Expr` Input\n", + "|--------------------------|----------------------|-------------------------|---|---|\n", + "| Negation | ¬ P | `~P` | `~P` | `Expr('~', P)`\n", + "| And | P ∧ Q | `P & Q` | `P & Q` | `Expr('&', P, Q)`\n", + "| Or | P ∨ Q | `P` | `Q`| `P` | `Q` | `Expr('`|`', P, Q)`\n", + "| Inequality (Xor) | P ≠ Q | `P ^ Q` | `P ^ Q` | `Expr('^', P, Q)`\n", + "| Implication | P → Q | `P` |`'==>'`| `Q` | `P ==> Q` | `Expr('==>', P, Q)`\n", + "| Reverse Implication | Q ← P | `Q` |`'<=='`| `P` |`Q <== P` | `Expr('<==', Q, P)`\n", + "| Equivalence | P ↔ Q | `P` |`'<=>'`| `Q` |`P <=> Q` | `Expr('<=>', P, Q)`\n", "\n", - "# Let's print them and see if they are the same as before\n", - "print(P)\n", - "print(R1)\n", - "print(R4)" + "Here's an example of defining a sentence with an implication arrow:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(~(P & Q) ==> (~P | ~Q))" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "~(P & Q) |'==>'| (~P | ~Q)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Yes, yes they are. We cannot use `==>` and `<=>` when we use operator overloading, because those are not operators in Python. Instead, we have to use `>>` and `%` operators in their place. (Actually, `==>` and `<=>` are converted to `>>` and `%` internally.) Did you just cringe at using `%` for biconditionals? I am not too happy with that either. Ugly, I know, but for many reasons we *had to* implement it that way. But hey, it works like a charm.\n", - "\n", - "Before we move on, I would like to point out something that might cause you some confusion. The `==` and `!=` operators for `Expr`s. They do not logically evaluate two expressions and then check if they are equal. So don't do something like\n", + "## `expr`: a Shortcut for Constructing Sentences\n", "\n", - "```python\n", - "A & (B | C) == (A & B) | (A & C)\n", - "```\n", + "If the `|'==>'|` notation looks ugly to you, you can use the function `expr` instead:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(~(P & Q) ==> (~P | ~Q))" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "expr('~(P & Q) ==> (~P | ~Q)')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`expr` takes a string as input, and parses it into an `Expr`. The string can contain arrow operators: `==>`, `<==`, or `<=>`, which are handled as if they were regular Python infix operators. And `expr` automatically defines any symbols, so you don't need to pre-define them:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "sqrt(((b ** 2) - ((4 * a) * c)))" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "expr('sqrt(b ** 2 - 4 * a * c)')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For now that's all you need to know about `expr`. If you are interested, we explain the messy details of how `expr` is implemented and how `|'==>'|` is handled in the appendix." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Propositional Knowledge Bases: `PropKB`\n", "\n", - "or even\n", + "The class `PropKB` can be used to represent a knowledge base of propositional logic sentences.\n", "\n", - "```python\n", - "A & B == B & A\n", - "```\n", + "We see that the class `KB` has four methods, apart from `__init__`. A point to note here: the `ask` method simply calls the `ask_generator` method. Thus, this one has already been implemented, and what you'll have to actually implement when you create your own knowledge base class (though you'll probably never need to, considering the ones we've created for you) will be the `ask_generator` function and not the `ask` function itself.\n", "\n", - "and expect it to return `True`. That's not how the `==` operator is intended to work. If you to know what it is supposed to do, have a look at the implementation of `__eq__`; that should tell you enough." + "The class `PropKB` now.\n", + "* `__init__(self, sentence=None)` : The constructor `__init__` creates a single field `clauses` which will be a list of all the sentences of the knowledge base. Note that each one of these sentences will be a 'clause' i.e. a sentence which is made up of only literals and `or`s.\n", + "* `tell(self, sentence)` : When you want to add a sentence to the KB, you use the `tell` method. This method takes a sentence, converts it to its CNF, extracts all the clauses, and adds all these clauses to the `clauses` field. So, you need not worry about `tell`ing only clauses to the knowledge base. You can `tell` the knowledge base a sentence in any form that you wish; converting it to CNF and adding the resulting clauses will be handled by the `tell` method.\n", + "* `ask_generator(self, query)` : The `ask_generator` function is used by the `ask` function. It calls the `tt_entails` function, which in turn returns `True` if the knowledge base entails query and `False` otherwise. The `ask_generator` itself returns an empty dict `{}` if the knowledge base entails query and `None` otherwise. This might seem a little bit weird to you. After all, it makes more sense just to return a `True` or a `False` instead of the `{}` or `None` But this is done to maintain consistency with the way things are in First-Order Logic, where an `ask_generator` function is supposed to return all the substitutions that make the query true. Hence the dict, to return all these substitutions. I will be mostly be using the `ask` function which returns a `{}` or a `False`, but if you don't like this, you can always use the `ask_if_true` function which returns a `True` or a `False`.\n", + "* `retract(self, sentence)` : This function removes all the clauses of the sentence given, from the knowledge base. Like the `tell` function, you don't have to pass clauses to remove them from the knowledge base; any sentence will do fine. The function will take care of converting that sentence to clauses and then remove those." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### The `expr` function\n", - "\n" + "## Wumpus World KB\n", + "Let us create a `PropKB` for the wumpus world with the sentences mentioned in `section 7.4.3`." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "wumpus_kb = PropKB()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We define the symbols we use in our clauses.
\n", + "$P_{x, y}$ is true if there is a pit in `[x, y]`.
\n", + "$B_{x, y}$ is true if the agent senses breeze in `[x, y]`.
" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], - "source": [] + "source": [ + "P11, P12, P21, P22, P31, B11, B21 = expr('P11, P12, P21, P22, P31, B11, B21')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we tell sentences based on `section 7.4.3`.
\n", + "There is no pit in `[1,1]`." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "wumpus_kb.tell(~P11)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A square is breezy if and only if there is a pit in a neighboring square. This has to be stated for each square but for now, we include just the relevant squares." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "wumpus_kb.tell(B11 | '<=>' | ((P12 | P21)))\n", + "wumpus_kb.tell(B21 | '<=>' | ((P11 | P22 | P31)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we include the breeze percepts for the first two squares leading up to the situation in `Figure 7.3(b)`" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "wumpus_kb.tell(~B11)\n", + "wumpus_kb.tell(B21)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can check the clauses stored in a `KB` by accessing its `clauses` variable" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[~P11,\n", + " (~P12 | B11),\n", + " (~P21 | B11),\n", + " (P12 | P21 | ~B11),\n", + " (~P11 | B21),\n", + " (~P22 | B21),\n", + " (~P31 | B21),\n", + " (P11 | P22 | P31 | ~B21),\n", + " ~B11,\n", + " B21]" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "wumpus_kb.clauses" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We see that the equivalence $B_{1, 1} \\iff (P_{1, 2} \\lor P_{2, 1})$ was automatically converted to two implications which were inturn converted to CNF which is stored in the `KB`.
\n", + "$B_{1, 1} \\iff (P_{1, 2} \\lor P_{2, 1})$ was split into $B_{1, 1} \\implies (P_{1, 2} \\lor P_{2, 1})$ and $B_{1, 1} \\Longleftarrow (P_{1, 2} \\lor P_{2, 1})$.
\n", + "$B_{1, 1} \\implies (P_{1, 2} \\lor P_{2, 1})$ was converted to $P_{1, 2} \\lor P_{2, 1} \\lor \\neg B_{1, 1}$.
\n", + "$B_{1, 1} \\Longleftarrow (P_{1, 2} \\lor P_{2, 1})$ was converted to $\\neg (P_{1, 2} \\lor P_{2, 1}) \\lor B_{1, 1}$ which becomes $(\\neg P_{1, 2} \\lor B_{1, 1}) \\land (\\neg P_{2, 1} \\lor B_{1, 1})$ after applying De Morgan's laws and distributing the disjunction.
\n", + "$B_{2, 1} \\iff (P_{1, 1} \\lor P_{2, 2} \\lor P_{3, 2})$ is converted in similar manner." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Knowledge based agents" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A knowledge-based agent is a simple generic agent that maintains and handles a knowledge base.\n", + "The knowledge base may initially contain some background knowledge.\n", + "
\n", + "The purpose of a KB agent is to provide a level of abstraction over knowledge-base manipulation and is to be used as a base class for agents that work on a knowledge base.\n", + "
\n", + "Given a percept, the KB agent adds the percept to its knowledge base, asks the knowledge base for the best action, and tells the knowledge base that it has in fact taken that action.\n", + "
\n", + "Our implementation of `KB-Agent` is encapsulated in a class `KB_AgentProgram` which inherits from the `KB` class.\n", + "
\n", + "Let's have a look." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def KB_AgentProgram(KB):\n",
+ " """A generic logical knowledge-based agent program. [Figure 7.1]"""\n",
+ " steps = itertools.count()\n",
+ "\n",
+ " def program(percept):\n",
+ " t = next(steps)\n",
+ " KB.tell(make_percept_sentence(percept, t))\n",
+ " action = KB.ask(make_action_query(t))\n",
+ " KB.tell(make_action_sentence(action, t))\n",
+ " return action\n",
+ "\n",
+ " def make_percept_sentence(percept, t):\n",
+ " return Expr("Percept")(percept, t)\n",
+ "\n",
+ " def make_action_query(t):\n",
+ " return expr("ShouldDo(action, {})".format(t))\n",
+ "\n",
+ " def make_action_sentence(action, t):\n",
+ " return Expr("Did")(action[expr('action')], t)\n",
+ "\n",
+ " return program\n",
+ "def tt_check_all(kb, alpha, symbols, model):\n",
+ " """Auxiliary routine to implement tt_entails."""\n",
+ " if not symbols:\n",
+ " if pl_true(kb, model):\n",
+ " result = pl_true(alpha, model)\n",
+ " assert result in (True, False)\n",
+ " return result\n",
+ " else:\n",
+ " return True\n",
+ " else:\n",
+ " P, rest = symbols[0], symbols[1:]\n",
+ " return (tt_check_all(kb, alpha, rest, extend(model, P, True)) and\n",
+ " tt_check_all(kb, alpha, rest, extend(model, P, False)))\n",
+ "\n", + "If symbols are defined, the routine recursively constructs every combination of truth values for the symbols and then, \n", + "it checks whether `model` is consistent with `kb`.\n", + "The given models correspond to the lines in the truth table,\n", + "which have a `true` in the KB column, \n", + "and for these lines it checks whether the query evaluates to true\n", + "
\n", + "`result = pl_true(alpha, model)`.\n", + "
\n", + "
\n", + "In short, `tt_check_all` evaluates this logical expression for each `model`\n", + "
\n", + "`pl_true(kb, model) => pl_true(alpha, model)`\n", + "
\n", + "which is logically equivalent to\n", + "
\n", + "`pl_true(kb, model) & ~pl_true(alpha, model)` \n", + "
\n", + "that is, the knowledge base and the negation of the query are logically inconsistent.\n", + "
\n", + "
\n", + "`tt_entails()` just extracts the symbols from the query and calls `tt_check_all()` with the proper parameters.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def tt_entails(kb, alpha):\n",
+ " """Does kb entail the sentence alpha? Use truth tables. For propositional\n",
+ " kb's and sentences. [Figure 7.10]. Note that the 'kb' should be an\n",
+ " Expr which is a conjunction of clauses.\n",
+ " >>> tt_entails(expr('P & Q'), expr('Q'))\n",
+ " True\n",
+ " """\n",
+ " assert not variables(alpha)\n",
+ " symbols = list(prop_symbols(kb & alpha))\n",
+ " return tt_check_all(kb, alpha, symbols, {})\n",
+ "\n", + "This technique corresponds to proof by contradiction, a standard mathematical proof technique. We assume $\\alpha$ to be false and show that this leads to a contradiction with known axioms in $\\text{KB}$. We obtain a contradiction by making valid inferences using inference rules. In this proof we use a single inference rule, resolution which states $(l_1 \\lor \\dots \\lor l_k) \\land (m_1 \\lor \\dots \\lor m_n) \\land (l_i \\iff \\neg m_j) \\implies l_1 \\lor \\dots \\lor l_{i - 1} \\lor l_{i + 1} \\lor \\dots \\lor l_k \\lor m_1 \\lor \\dots \\lor m_{j - 1} \\lor m_{j + 1} \\lor \\dots \\lor m_n$. Applying the resolution yields us a clause which we add to the KB. We keep doing this until:\n", + "\n", + "* There are no new clauses that can be added, in which case $\\text{KB} \\nvDash \\alpha$.\n", + "* Two clauses resolve to yield the empty clause, in which case $\\text{KB} \\vDash \\alpha$.\n", + "\n", + "The empty clause is equivalent to False because it arises only from resolving two complementary\n", + "unit clauses such as $P$ and $\\neg P$ which is a contradiction as both $P$ and $\\neg P$ can't be True at the same time." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There is one catch however, the algorithm that implements proof by resolution cannot handle complex sentences. \n", + "Implications and bi-implications have to be simplified into simpler clauses. \n", + "We already know that *every sentence of a propositional logic is logically equivalent to a conjunction of clauses*.\n", + "We will use this fact to our advantage and simplify the input sentence into the **conjunctive normal form** (CNF) which is a conjunction of disjunctions of literals.\n", + "For eg:\n", + "
\n", + "$$(A\\lor B)\\land (\\neg B\\lor C\\lor\\neg D)\\land (D\\lor\\neg E)$$\n", + "This is equivalent to the POS (Product of sums) form in digital electronics.\n", + "
\n", + "Here's an outline of how the conversion is done:\n", + "1. Convert bi-implications to implications\n", + "
\n", + "$\\alpha\\iff\\beta$ can be written as $(\\alpha\\implies\\beta)\\land(\\beta\\implies\\alpha)$\n", + "
\n", + "This also applies to compound sentences\n", + "
\n", + "$\\alpha\\iff(\\beta\\lor\\gamma)$ can be written as $(\\alpha\\implies(\\beta\\lor\\gamma))\\land((\\beta\\lor\\gamma)\\implies\\alpha)$\n", + "
\n", + "2. Convert implications to their logical equivalents\n", + "
\n", + "$\\alpha\\implies\\beta$ can be written as $\\neg\\alpha\\lor\\beta$\n", + "
\n", + "3. Move negation inwards\n", + "
\n", + "CNF requires atomic literals. Hence, negation cannot appear on a compound statement.\n", + "De Morgan's laws will be helpful here.\n", + "
\n", + "$\\neg(\\alpha\\land\\beta)\\equiv(\\neg\\alpha\\lor\\neg\\beta)$\n", + "
\n", + "$\\neg(\\alpha\\lor\\beta)\\equiv(\\neg\\alpha\\land\\neg\\beta)$\n", + "
\n", + "4. Distribute disjunction over conjunction\n", + "
\n", + "Disjunction and conjunction are distributive over each other.\n", + "Now that we only have conjunctions, disjunctions and negations in our expression, \n", + "we will distribute disjunctions over conjunctions wherever possible as this will give us a sentence which is a conjunction of simpler clauses, \n", + "which is what we wanted in the first place.\n", + "
\n", + "We need a term of the form\n", + "
\n", + "$(\\alpha_{1}\\lor\\alpha_{2}\\lor\\alpha_{3}...)\\land(\\beta_{1}\\lor\\beta_{2}\\lor\\beta_{3}...)\\land(\\gamma_{1}\\lor\\gamma_{2}\\lor\\gamma_{3}...)\\land...$\n", + "
\n", + "
\n", + "The `to_cnf` function executes this conversion using helper subroutines." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def to_cnf(s):\n",
+ " """Convert a propositional logical sentence to conjunctive normal form.\n",
+ " That is, to the form ((A | ~B | ...) & (B | C | ...) & ...) [p. 253]\n",
+ " >>> to_cnf('~(B | C)')\n",
+ " (~B & ~C)\n",
+ " """\n",
+ " s = expr(s)\n",
+ " if isinstance(s, str):\n",
+ " s = expr(s)\n",
+ " s = eliminate_implications(s) # Steps 1, 2 from p. 253\n",
+ " s = move_not_inwards(s) # Step 3\n",
+ " return distribute_and_over_or(s) # Step 4\n",
+ "\n", + "`eliminate_implications` converts bi-implications and implications to their logical equivalents.\n", + "
\n", + "`move_not_inwards` removes negations from compound statements and moves them inwards using De Morgan's laws.\n", + "
\n", + "`distribute_and_over_or` distributes disjunctions over conjunctions.\n", + "
\n", + "Run the cell below for implementation details." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def eliminate_implications(s):\n",
+ " """Change implications into equivalent form with only &, |, and ~ as logical operators."""\n",
+ " s = expr(s)\n",
+ " if not s.args or is_symbol(s.op):\n",
+ " return s # Atoms are unchanged.\n",
+ " args = list(map(eliminate_implications, s.args))\n",
+ " a, b = args[0], args[-1]\n",
+ " if s.op == '==>':\n",
+ " return b | ~a\n",
+ " elif s.op == '<==':\n",
+ " return a | ~b\n",
+ " elif s.op == '<=>':\n",
+ " return (a | ~b) & (b | ~a)\n",
+ " elif s.op == '^':\n",
+ " assert len(args) == 2 # TODO: relax this restriction\n",
+ " return (a & ~b) | (~a & b)\n",
+ " else:\n",
+ " assert s.op in ('&', '|', '~')\n",
+ " return Expr(s.op, *args)\n",
+ "def move_not_inwards(s):\n",
+ " """Rewrite sentence s by moving negation sign inward.\n",
+ " >>> move_not_inwards(~(A | B))\n",
+ " (~A & ~B)"""\n",
+ " s = expr(s)\n",
+ " if s.op == '~':\n",
+ " def NOT(b):\n",
+ " return move_not_inwards(~b)\n",
+ " a = s.args[0]\n",
+ " if a.op == '~':\n",
+ " return move_not_inwards(a.args[0]) # ~~A ==> A\n",
+ " if a.op == '&':\n",
+ " return associate('|', list(map(NOT, a.args)))\n",
+ " if a.op == '|':\n",
+ " return associate('&', list(map(NOT, a.args)))\n",
+ " return s\n",
+ " elif is_symbol(s.op) or not s.args:\n",
+ " return s\n",
+ " else:\n",
+ " return Expr(s.op, *list(map(move_not_inwards, s.args)))\n",
+ "def distribute_and_over_or(s):\n",
+ " """Given a sentence s consisting of conjunctions and disjunctions\n",
+ " of literals, return an equivalent sentence in CNF.\n",
+ " >>> distribute_and_over_or((A & B) | C)\n",
+ " ((A | C) & (B | C))\n",
+ " """\n",
+ " s = expr(s)\n",
+ " if s.op == '|':\n",
+ " s = associate('|', s.args)\n",
+ " if s.op != '|':\n",
+ " return distribute_and_over_or(s)\n",
+ " if len(s.args) == 0:\n",
+ " return False\n",
+ " if len(s.args) == 1:\n",
+ " return distribute_and_over_or(s.args[0])\n",
+ " conj = first(arg for arg in s.args if arg.op == '&')\n",
+ " if not conj:\n",
+ " return s\n",
+ " others = [a for a in s.args if a is not conj]\n",
+ " rest = associate('|', others)\n",
+ " return associate('&', [distribute_and_over_or(c | rest)\n",
+ " for c in conj.args])\n",
+ " elif s.op == '&':\n",
+ " return associate('&', list(map(distribute_and_over_or, s.args)))\n",
+ " else:\n",
+ " return s\n",
+ "def pl_resolution(KB, alpha):\n",
+ " """Propositional-logic resolution: say if alpha follows from KB. [Figure 7.12]"""\n",
+ " clauses = KB.clauses + conjuncts(to_cnf(~alpha))\n",
+ " new = set()\n",
+ " while True:\n",
+ " n = len(clauses)\n",
+ " pairs = [(clauses[i], clauses[j])\n",
+ " for i in range(n) for j in range(i+1, n)]\n",
+ " for (ci, cj) in pairs:\n",
+ " resolvents = pl_resolve(ci, cj)\n",
+ " if False in resolvents:\n",
+ " return True\n",
+ " new = new.union(set(resolvents))\n",
+ " if new.issubset(set(clauses)):\n",
+ " return False\n",
+ " for c in new:\n",
+ " if c not in clauses:\n",
+ " clauses.append(c)\n",
+ "\n", + "#### Horn Clauses\n", + "Horn clauses can be defined as a *disjunction* of *literals* with **at most** one positive literal. \n", + "
\n", + "A Horn clause with exactly one positive literal is called a *definite clause*.\n", + "
\n", + "A Horn clause might look like \n", + "
\n", + "$\\neg a\\lor\\neg b\\lor\\neg c\\lor\\neg d... \\lor z$\n", + "
\n", + "This, coincidentally, is also a definite clause.\n", + "
\n", + "Using De Morgan's laws, the example above can be simplified to \n", + "
\n", + "$a\\land b\\land c\\land d ... \\implies z$\n", + "
\n", + "This seems like a logical representation of how humans process known data and facts. \n", + "Assuming percepts `a`, `b`, `c`, `d` ... to be true simultaneously, we can infer `z` to also be true at that point in time. \n", + "There are some interesting aspects of Horn clauses that make algorithmic inference or *resolution* easier.\n", + "- Definite clauses can be written as implications:\n", + "
\n", + "The most important simplification a definite clause provides is that it can be written as an implication.\n", + "The premise (or the knowledge that leads to the implication) is a conjunction of positive literals.\n", + "The conclusion (the implied statement) is also a positive literal.\n", + "The sentence thus becomes easier to understand.\n", + "The premise and the conclusion are conventionally called the *body* and the *head* respectively.\n", + "A single positive literal is called a *fact*.\n", + "- Forward chaining and backward chaining can be used for inference from Horn clauses:\n", + "
\n", + "Forward chaining is semantically identical to `AND-OR-Graph-Search` from the chapter on search algorithms.\n", + "Implementational details will be explained shortly.\n", + "- Deciding entailment with Horn clauses is linear in size of the knowledge base:\n", + "
\n", + "Surprisingly, the forward and backward chaining algorithms traverse each element of the knowledge base at most once, greatly simplifying the problem.\n", + "
\n", + "
\n", + "The function `pl_fc_entails` implements forward chaining to see if a knowledge base `KB` entails a symbol `q`.\n", + "
\n", + "Before we proceed further, note that `pl_fc_entails` doesn't use an ordinary `KB` instance. \n", + "The knowledge base here is an instance of the `PropDefiniteKB` class, derived from the `PropKB` class, \n", + "but modified to store definite clauses.\n", + "
\n", + "The main point of difference arises in the inclusion of a helper method to `PropDefiniteKB` that returns a list of clauses in KB that have a given symbol `p` in their premise." + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def clauses_with_premise(self, p):\n",
+ " """Return a list of the clauses in KB that have p in their premise.\n",
+ " This could be cached away for O(1) speed, but we'll recompute it."""\n",
+ " return [c for c in self.clauses\n",
+ " if c.op == '==>' and p in conjuncts(c.args[0])]\n",
+ "def pl_fc_entails(KB, q):\n",
+ " """Use forward chaining to see if a PropDefiniteKB entails symbol q.\n",
+ " [Figure 7.15]\n",
+ " >>> pl_fc_entails(horn_clauses_KB, expr('Q'))\n",
+ " True\n",
+ " """\n",
+ " count = {c: len(conjuncts(c.args[0]))\n",
+ " for c in KB.clauses\n",
+ " if c.op == '==>'}\n",
+ " inferred = defaultdict(bool)\n",
+ " agenda = [s for s in KB.clauses if is_prop_symbol(s.op)]\n",
+ " while agenda:\n",
+ " p = agenda.pop()\n",
+ " if p == q:\n",
+ " return True\n",
+ " if not inferred[p]:\n",
+ " inferred[p] = True\n",
+ " for c in KB.clauses_with_premise(p):\n",
+ " count[c] -= 1\n",
+ " if count[c] == 0:\n",
+ " agenda.append(c.args[1])\n",
+ " return False\n",
+ "\n", + "
\n", + "`count` initially stores the number of symbols in the premise of each sentence in the knowledge base.\n", + "
\n", + "The `conjuncts` helper function separates a given sentence at conjunctions.\n", + "
\n", + "`inferred` is initialized as a *boolean* defaultdict. \n", + "This will be used later to check if we have inferred all premises of each clause of the agenda.\n", + "
\n", + "`agenda` initially stores a list of clauses that the knowledge base knows to be true.\n", + "The `is_prop_symbol` helper function checks if the given symbol is a valid propositional logic symbol.\n", + "
\n", + "
\n", + "We now iterate through `agenda`, popping a symbol `p` on each iteration.\n", + "If the query `q` is the same as `p`, we know that entailment holds.\n", + "
\n", + "The agenda is processed, reducing `count` by one for each implication with a premise `p`.\n", + "A conclusion is added to the agenda when `count` reaches zero. This means we know all the premises of that particular implication to be true.\n", + "
\n", + "`clauses_with_premise` is a helpful method of the `PropKB` class.\n", + "It returns a list of clauses in the knowledge base that have `p` in their premise.\n", + "
\n", + "
\n", + "Now that we have an idea of how this function works, let's see a few examples of its usage, but we first need to define our knowledge base. We assume we know the following clauses to be true." + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses = ['(B & F)==>E', \n", + " '(A & E & F)==>G', \n", + " '(B & C)==>F', \n", + " '(A & B)==>D', \n", + " '(E & F)==>H', \n", + " '(H & I)==>J',\n", + " 'A', \n", + " 'B', \n", + " 'C']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will now `tell` this information to our knowledge base." + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "definite_clauses_KB = PropDefiniteKB()\n", + "for clause in clauses:\n", + " definite_clauses_KB.tell(expr(clause))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can now check if our knowledge base entails the following queries." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pl_fc_entails(definite_clauses_KB, expr('G'))" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pl_fc_entails(definite_clauses_KB, expr('H'))" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pl_fc_entails(definite_clauses_KB, expr('I'))" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pl_fc_entails(definite_clauses_KB, expr('J'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Effective Propositional Model Checking\n", + "\n", + "The previous segments elucidate the algorithmic procedure for model checking. \n", + "In this segment, we look at ways of making them computationally efficient.\n", + "
\n", + "The problem we are trying to solve is conventionally called the _propositional satisfiability problem_, abbreviated as the _SAT_ problem.\n", + "In layman terms, if there exists a model that satisfies a given Boolean formula, the formula is called satisfiable.\n", + "
\n", + "The SAT problem was the first problem to be proven _NP-complete_.\n", + "The main characteristics of an NP-complete problem are:\n", + "- Given a solution to such a problem, it is easy to verify if the solution solves the problem.\n", + "- The time required to actually solve the problem using any known algorithm increases exponentially with respect to the size of the problem.\n", + "
\n", + "
\n", + "Due to these properties, heuristic and approximational methods are often applied to find solutions to these problems.\n", + "
\n", + "It is extremely important to be able to solve large scale SAT problems efficiently because \n", + "many combinatorial problems in computer science can be conveniently reduced to checking the satisfiability of a propositional sentence under some constraints.\n", + "
\n", + "We will introduce two new algorithms that perform propositional model checking in a computationally effective way.\n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 1. DPLL (Davis-Putnam-Logeman-Loveland) algorithm\n", + "This algorithm is very similar to Backtracking-Search.\n", + "It recursively enumerates possible models in a depth-first fashion with the following improvements over algorithms like `tt_entails`:\n", + "1. Early termination:\n", + "
\n", + "In certain cases, the algorithm can detect the truth value of a statement using just a partially completed model.\n", + "For example, $(P\\lor Q)\\land(P\\lor R)$ is true if P is true, regardless of other variables.\n", + "This reduces the search space significantly.\n", + "2. Pure symbol heuristic:\n", + "
\n", + "A symbol that has the same sign (positive or negative) in all clauses is called a _pure symbol_.\n", + "It isn't difficult to see that any satisfiable model will have the pure symbols assigned such that its parent clause becomes _true_.\n", + "For example, $(P\\lor\\neg Q)\\land(\\neg Q\\lor\\neg R)\\land(R\\lor P)$ has P and Q as pure symbols\n", + "and for the sentence to be true, P _has_ to be true and Q _has_ to be false.\n", + "The pure symbol heuristic thus simplifies the problem a bit.\n", + "3. Unit clause heuristic:\n", + "
\n", + "In the context of DPLL, clauses with just one literal and clauses with all but one _false_ literals are called unit clauses.\n", + "If a clause is a unit clause, it can only be satisfied by assigning the necessary value to make the last literal true.\n", + "We have no other choice.\n", + "
\n", + "Assigning one unit clause can create another unit clause.\n", + "For example, when P is false, $(P\\lor Q)$ becomes a unit clause, causing _true_ to be assigned to Q.\n", + "A series of forced assignments derived from previous unit clauses is called _unit propagation_.\n", + "In this way, this heuristic simplifies the problem further.\n", + "
\n", + "The algorithm often employs other tricks to scale up to large problems.\n", + "However, these tricks are currently out of the scope of this notebook. Refer to section 7.6 of the book for more details.\n", + "
\n", + "
\n", + "Let's have a look at the algorithm." + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def dpll(clauses, symbols, model):\n",
+ " """See if the clauses are true in a partial model."""\n",
+ " unknown_clauses = [] # clauses with an unknown truth value\n",
+ " for c in clauses:\n",
+ " val = pl_true(c, model)\n",
+ " if val is False:\n",
+ " return False\n",
+ " if val is not True:\n",
+ " unknown_clauses.append(c)\n",
+ " if not unknown_clauses:\n",
+ " return model\n",
+ " P, value = find_pure_symbol(symbols, unknown_clauses)\n",
+ " if P:\n",
+ " return dpll(clauses, removeall(P, symbols), extend(model, P, value))\n",
+ " P, value = find_unit_clause(clauses, model)\n",
+ " if P:\n",
+ " return dpll(clauses, removeall(P, symbols), extend(model, P, value))\n",
+ " if not symbols:\n",
+ " raise TypeError("Argument should be of the type Expr.")\n",
+ " P, symbols = symbols[0], symbols[1:]\n",
+ " return (dpll(clauses, symbols, extend(model, P, True)) or\n",
+ " dpll(clauses, symbols, extend(model, P, False)))\n",
+ "\n", + "The `dpll_satisfiable` helper function converts the input clauses to _conjunctive normal form_ and calls the `dpll` function with the correct parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def dpll_satisfiable(s):\n",
+ " """Check satisfiability of a propositional sentence.\n",
+ " This differs from the book code in two ways: (1) it returns a model\n",
+ " rather than True when it succeeds; this is more useful. (2) The\n",
+ " function find_pure_symbol is passed a list of unknown clauses, rather\n",
+ " than a list of all clauses and the model; this is more efficient."""\n",
+ " clauses = conjuncts(to_cnf(s))\n",
+ " symbols = list(prop_symbols(s))\n",
+ " return dpll(clauses, symbols, {})\n",
+ "\n", + "The symbol to be flipped is decided by an evaluation function that counts the number of unsatisfied clauses.\n", + "Sometimes, symbols are also flipped randomly to avoid local optima. A subtle balance between greediness and randomness is required. Alternatively, some versions of the algorithm restart with a completely new random assignment if no solution has been found for too long as a way of getting out of local minima of numbers of unsatisfied clauses.\n", + "
\n", + "
\n", + "Let's have a look at the algorithm." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def WalkSAT(clauses, p=0.5, max_flips=10000):\n",
+ " """Checks for satisfiability of all clauses by randomly flipping values of variables\n",
+ " """\n",
+ " # Set of all symbols in all clauses\n",
+ " symbols = {sym for clause in clauses for sym in prop_symbols(clause)}\n",
+ " # model is a random assignment of true/false to the symbols in clauses\n",
+ " model = {s: random.choice([True, False]) for s in symbols}\n",
+ " for i in range(max_flips):\n",
+ " satisfied, unsatisfied = [], []\n",
+ " for clause in clauses:\n",
+ " (satisfied if pl_true(clause, model) else unsatisfied).append(clause)\n",
+ " if not unsatisfied: # if model satisfies all the clauses\n",
+ " return model\n",
+ " clause = random.choice(unsatisfied)\n",
+ " if probability(p):\n",
+ " sym = random.choice(list(prop_symbols(clause)))\n",
+ " else:\n",
+ " # Flip the symbol in clause that maximizes number of sat. clauses\n",
+ " def sat_count(sym):\n",
+ " # Return the the number of clauses satisfied after flipping the symbol.\n",
+ " model[sym] = not model[sym]\n",
+ " count = len([clause for clause in clauses if pl_true(clause, model)])\n",
+ " model[sym] = not model[sym]\n",
+ " return count\n",
+ " sym = argmax(prop_symbols(clause), key=sat_count)\n",
+ " model[sym] = not model[sym]\n",
+ " # If no solution is found within the flip limit, we return failure\n",
+ " return None\n",
+ "\n", + "1. The `clauses` we want to satisfy.\n", + "
\n", + "2. The probability `p` of randomly changing a symbol.\n", + "
\n", + "3. The maximum number of flips (`max_flips`) the algorithm will run for. If the clauses are still unsatisfied, the algorithm returns `None` to denote failure.\n", + "
\n", + "The algorithm is identical in concept to Hill climbing and the code isn't difficult to understand.\n", + "
\n", + "
\n", + "Let's see a few examples of usage." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "A, B, C, D = expr('A, B, C, D')" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{A: True, B: True, C: False, D: True}" + ] + }, + "execution_count": 57, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "WalkSAT([A, B, ~C, D], 0.5, 100)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is a simple case to show that the algorithm converges." + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{A: True, B: True, C: True}" + ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "WalkSAT([A & B, A & C], 0.5, 100)" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{A: True, B: True, C: True, D: True}" + ] + }, + "execution_count": 59, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "WalkSAT([A & B, C & D, C & B], 0.5, 100)" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "WalkSAT([A & B, C | D, ~(D | B)], 0.5, 1000)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This one doesn't give any output because WalkSAT did not find any model where these clauses hold. We can solve these clauses to see that they together form a contradiction and hence, it isn't supposed to have a solution." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "One point of difference between this algorithm and the `dpll_satisfiable` algorithms is that both these algorithms take inputs differently. \n", + "For WalkSAT to take complete sentences as input, \n", + "we can write a helper function that converts the input sentence into conjunctive normal form and then calls WalkSAT with the list of conjuncts of the CNF form of the sentence." + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def WalkSAT_CNF(sentence, p=0.5, max_flips=10000):\n", + " return WalkSAT(conjuncts(to_cnf(sentence)), 0, max_flips)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can call `WalkSAT_CNF` and `DPLL_Satisfiable` with the same arguments." + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{A: True, B: True, C: False, D: True}" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "WalkSAT_CNF((A & B) | (C & ~A) | (B & ~D), 0.5, 1000)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It works!\n", + "
\n", + "Notice that the solution generated by WalkSAT doesn't omit variables that the sentence doesn't depend upon. \n", + "If the sentence is independent of a particular variable, the solution contains a random value for that variable because of the stochastic nature of the algorithm.\n", + "
\n", + "
\n", + "Let's compare the runtime of WalkSAT and DPLL for a few cases. We will use the `%%timeit` magic to do this." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "sentence_1 = A |'<=>'| B\n", + "sentence_2 = (A & B) | (C & ~A) | (B & ~D)\n", + "sentence_3 = (A | (B & C)) |'<=>'| ((A | B) & (A | C))" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1.55 ms ± 64.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "dpll_satisfiable(sentence_1)\n", + "dpll_satisfiable(sentence_2)\n", + "dpll_satisfiable(sentence_3)" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1.02 ms ± 6.92 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "WalkSAT_CNF(sentence_1)\n", + "WalkSAT_CNF(sentence_2)\n", + "WalkSAT_CNF(sentence_3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "On an average, for solvable cases, `WalkSAT` is quite faster than `dpll` because, for a small number of variables, \n", + "`WalkSAT` can reduce the search space significantly. \n", + "Results can be different for sentences with more symbols though.\n", + "Feel free to play around with this to understand the trade-offs of these algorithms better." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### SATPlan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this section we show how to make plans by logical inference. The basic idea is very simple. It includes the following three steps:\n", + "1. Constuct a sentence that includes:\n", + " 1. A colection of assertions about the initial state.\n", + " 2. The successor-state axioms for all the possible actions at each time up to some maximum time t.\n", + " 3. The assertion that the goal is achieved at time t.\n", + "2. Present the whole sentence to a SAT solver.\n", + "3. Assuming a model is found, extract from the model those variables that represent actions and are assigned true. Together they represent a plan to achieve the goals.\n", + "\n", + "\n", + "Lets have a look at the algorithm" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def SAT_plan(init, transition, goal, t_max, SAT_solver=dpll_satisfiable):\n",
+ " """Converts a planning problem to Satisfaction problem by translating it to a cnf sentence.\n",
+ " [Figure 7.22]"""\n",
+ "\n",
+ " # Functions used by SAT_plan\n",
+ " def translate_to_SAT(init, transition, goal, time):\n",
+ " clauses = []\n",
+ " states = [state for state in transition]\n",
+ "\n",
+ " # Symbol claiming state s at time t\n",
+ " state_counter = itertools.count()\n",
+ " for s in states:\n",
+ " for t in range(time+1):\n",
+ " state_sym[s, t] = Expr("State_{}".format(next(state_counter)))\n",
+ "\n",
+ " # Add initial state axiom\n",
+ " clauses.append(state_sym[init, 0])\n",
+ "\n",
+ " # Add goal state axiom\n",
+ " clauses.append(state_sym[goal, time])\n",
+ "\n",
+ " # All possible transitions\n",
+ " transition_counter = itertools.count()\n",
+ " for s in states:\n",
+ " for action in transition[s]:\n",
+ " s_ = transition[s][action]\n",
+ " for t in range(time):\n",
+ " # Action 'action' taken from state 's' at time 't' to reach 's_'\n",
+ " action_sym[s, action, t] = Expr(\n",
+ " "Transition_{}".format(next(transition_counter)))\n",
+ "\n",
+ " # Change the state from s to s_\n",
+ " clauses.append(action_sym[s, action, t] |'==>'| state_sym[s, t])\n",
+ " clauses.append(action_sym[s, action, t] |'==>'| state_sym[s_, t + 1])\n",
+ "\n",
+ " # Allow only one state at any time\n",
+ " for t in range(time+1):\n",
+ " # must be a state at any time\n",
+ " clauses.append(associate('|', [state_sym[s, t] for s in states]))\n",
+ "\n",
+ " for s in states:\n",
+ " for s_ in states[states.index(s) + 1:]:\n",
+ " # for each pair of states s, s_ only one is possible at time t\n",
+ " clauses.append((~state_sym[s, t]) | (~state_sym[s_, t]))\n",
+ "\n",
+ " # Restrict to one transition per timestep\n",
+ " for t in range(time):\n",
+ " # list of possible transitions at time t\n",
+ " transitions_t = [tr for tr in action_sym if tr[2] == t]\n",
+ "\n",
+ " # make sure at least one of the transitions happens\n",
+ " clauses.append(associate('|', [action_sym[tr] for tr in transitions_t]))\n",
+ "\n",
+ " for tr in transitions_t:\n",
+ " for tr_ in transitions_t[transitions_t.index(tr) + 1:]:\n",
+ " # there cannot be two transitions tr and tr_ at time t\n",
+ " clauses.append(~action_sym[tr] | ~action_sym[tr_])\n",
+ "\n",
+ " # Combine the clauses to form the cnf\n",
+ " return associate('&', clauses)\n",
+ "\n",
+ " def extract_solution(model):\n",
+ " true_transitions = [t for t in action_sym if model[action_sym[t]]]\n",
+ " # Sort transitions based on time, which is the 3rd element of the tuple\n",
+ " true_transitions.sort(key=lambda x: x[2])\n",
+ " return [action for s, action, time in true_transitions]\n",
+ "\n",
+ " # Body of SAT_plan algorithm\n",
+ " for t in range(t_max):\n",
+ " # dictionaries to help extract the solution from model\n",
+ " state_sym = {}\n",
+ " action_sym = {}\n",
+ "\n",
+ " cnf = translate_to_SAT(init, transition, goal, t)\n",
+ " model = SAT_solver(cnf)\n",
+ " if model is not False:\n",
+ " return extract_solution(model)\n",
+ " return None\n",
+ "\n", + "The law says that it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American.
\n", + "The first step is to extract the facts and convert them into first-order definite clauses. Extracting the facts from data alone is a challenging task. Fortunately, we have a small paragraph and can do extraction and conversion manually. We'll store the clauses in list aptly named `clauses`." + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "“... it is a crime for an American to sell weapons to hostile nations”
\n", + "The keywords to look for here are 'crime', 'American', 'sell', 'weapon' and 'hostile'. We use predicate symbols to make meaning of them.\n", + "\n", + "* `Criminal(x)`: `x` is a criminal\n", + "* `American(x)`: `x` is an American\n", + "* `Sells(x ,y, z)`: `x` sells `y` to `z`\n", + "* `Weapon(x)`: `x` is a weapon\n", + "* `Hostile(x)`: `x` is a hostile nation\n", + "\n", + "Let us now combine them with appropriate variable naming to depict the meaning of the sentence. The criminal `x` is also the American `x` who sells weapon `y` to `z`, which is a hostile nation.\n", + "\n", + "$\\text{American}(x) \\land \\text{Weapon}(y) \\land \\text{Sells}(x, y, z) \\land \\text{Hostile}(z) \\implies \\text{Criminal} (x)$" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"(American(x) & Weapon(y) & Sells(x, y, z) & Hostile(z)) ==> Criminal(x)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"The country Nono, an enemy of America\"
\n", + "We now know that Nono is an enemy of America. We represent these nations using the constant symbols `Nono` and `America`. the enemy relation is show using the predicate symbol `Enemy`.\n", + "\n", + "$\\text{Enemy}(\\text{Nono}, \\text{America})$" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"Enemy(Nono, America)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Nono ... has some missiles\"
\n", + "This states the existence of some missile which is owned by Nono. $\\exists x \\text{Owns}(\\text{Nono}, x) \\land \\text{Missile}(x)$. We invoke existential instantiation to introduce a new constant `M1` which is the missile owned by Nono.\n", + "\n", + "$\\text{Owns}(\\text{Nono}, \\text{M1}), \\text{Missile}(\\text{M1})$" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"Owns(Nono, M1)\"))\n", + "clauses.append(expr(\"Missile(M1)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "\"All of its missiles were sold to it by Colonel West\"
\n", + "If Nono owns something and it classifies as a missile, then it was sold to Nono by West.\n", + "\n", + "$\\text{Missile}(x) \\land \\text{Owns}(\\text{Nono}, x) \\implies \\text{Sells}(\\text{West}, x, \\text{Nono})$" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"(Missile(x) & Owns(Nono, x)) ==> Sells(West, x, Nono)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"West, who is American\"
\n", + "West is an American.\n", + "\n", + "$\\text{American}(\\text{West})$" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"American(West)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We also know, from our understanding of language, that missiles are weapons and that an enemy of America counts as “hostile”.\n", + "\n", + "$\\text{Missile}(x) \\implies \\text{Weapon}(x), \\text{Enemy}(x, \\text{America}) \\implies \\text{Hostile}(x)$" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "clauses.append(expr(\"Missile(x) ==> Weapon(x)\"))\n", + "clauses.append(expr(\"Enemy(x, America) ==> Hostile(x)\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we have converted the information into first-order definite clauses we can create our first-order logic knowledge base." + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "crime_kb = FolKB(clauses)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `subst` helper function substitutes variables with given values in first-order logic statements.\n", + "This will be useful in later algorithms.\n", + "It's implementation is quite simple and self-explanatory." + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def subst(s, x):\n",
+ " """Substitute the substitution s into the expression x.\n",
+ " >>> subst({x: 42, y:0}, F(x) + y)\n",
+ " (F(42) + 0)\n",
+ " """\n",
+ " if isinstance(x, list):\n",
+ " return [subst(s, xi) for xi in x]\n",
+ " elif isinstance(x, tuple):\n",
+ " return tuple([subst(s, xi) for xi in x])\n",
+ " elif not isinstance(x, Expr):\n",
+ " return x\n",
+ " elif is_var_symbol(x.op):\n",
+ " return s.get(x, x)\n",
+ " else:\n",
+ " return Expr(x.op, *[subst(s, arg) for arg in x.args])\n",
+ "def fol_fc_ask(KB, alpha):\n",
+ " """A simple forward-chaining algorithm. [Figure 9.3]"""\n",
+ " # TODO: Improve efficiency\n",
+ " kb_consts = list({c for clause in KB.clauses for c in constant_symbols(clause)})\n",
+ " def enum_subst(p):\n",
+ " query_vars = list({v for clause in p for v in variables(clause)})\n",
+ " for assignment_list in itertools.product(kb_consts, repeat=len(query_vars)):\n",
+ " theta = {x: y for x, y in zip(query_vars, assignment_list)}\n",
+ " yield theta\n",
+ "\n",
+ " # check if we can answer without new inferences\n",
+ " for q in KB.clauses:\n",
+ " phi = unify(q, alpha, {})\n",
+ " if phi is not None:\n",
+ " yield phi\n",
+ "\n",
+ " while True:\n",
+ " new = []\n",
+ " for rule in KB.clauses:\n",
+ " p, q = parse_definite_clause(rule)\n",
+ " for theta in enum_subst(p):\n",
+ " if set(subst(theta, p)).issubset(set(KB.clauses)):\n",
+ " q_ = subst(theta, q)\n",
+ " if all([unify(x, q_, {}) is None for x in KB.clauses + new]):\n",
+ " new.append(q_)\n",
+ " phi = unify(q_, alpha, {})\n",
+ " if phi is not None:\n",
+ " yield phi\n",
+ " if not new:\n",
+ " break\n",
+ " for clause in new:\n",
+ " KB.tell(clause)\n",
+ " return None\n",
+ "def fol_bc_or(KB, goal, theta):\n",
+ " for rule in KB.fetch_rules_for_goal(goal):\n",
+ " lhs, rhs = parse_definite_clause(standardize_variables(rule))\n",
+ " for theta1 in fol_bc_and(KB, lhs, unify(rhs, goal, theta)):\n",
+ " yield theta1\n",
+ "def fol_bc_and(KB, goals, theta):\n",
+ " if theta is None:\n",
+ " pass\n",
+ " elif not goals:\n",
+ " yield theta\n",
+ " else:\n",
+ " first, rest = goals[0], goals[1:]\n",
+ " for theta1 in fol_bc_or(KB, subst(theta, first), theta):\n",
+ " for theta2 in fol_bc_and(KB, rest, theta1):\n",
+ " yield theta2\n",
+ "\n",
+ "\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "class MDP:\n",
+ "\n",
+ " """A Markov Decision Process, defined by an initial state, transition model,\n",
+ " and reward function. We also keep track of a gamma value, for use by\n",
+ " algorithms. The transition model is represented somewhat differently from\n",
+ " the text. Instead of P(s' | s, a) being a probability number for each\n",
+ " state/state/action triplet, we instead have T(s, a) return a\n",
+ " list of (p, s') pairs. We also keep track of the possible states,\n",
+ " terminal states, and actions for each state. [page 646]"""\n",
+ "\n",
+ " def __init__(self, init, actlist, terminals, transitions = {}, reward = None, states=None, gamma=.9):\n",
+ " if not (0 < gamma <= 1):\n",
+ " raise ValueError("An MDP must have 0 < gamma <= 1")\n",
+ "\n",
+ " if states:\n",
+ " self.states = states\n",
+ " else:\n",
+ " ## collect states from transitions table\n",
+ " self.states = self.get_states_from_transitions(transitions)\n",
+ " \n",
+ " \n",
+ " self.init = init\n",
+ " \n",
+ " if isinstance(actlist, list):\n",
+ " ## if actlist is a list, all states have the same actions\n",
+ " self.actlist = actlist\n",
+ " elif isinstance(actlist, dict):\n",
+ " ## if actlist is a dict, different actions for each state\n",
+ " self.actlist = actlist\n",
+ " \n",
+ " self.terminals = terminals\n",
+ " self.transitions = transitions\n",
+ " if self.transitions == {}:\n",
+ " print("Warning: Transition table is empty.")\n",
+ " self.gamma = gamma\n",
+ " if reward:\n",
+ " self.reward = reward\n",
+ " else:\n",
+ " self.reward = {s : 0 for s in self.states}\n",
+ " #self.check_consistency()\n",
+ "\n",
+ " def R(self, state):\n",
+ " """Return a numeric reward for this state."""\n",
+ " return self.reward[state]\n",
+ "\n",
+ " def T(self, state, action):\n",
+ " """Transition model. From a state and an action, return a list\n",
+ " of (probability, result-state) pairs."""\n",
+ " if(self.transitions == {}):\n",
+ " raise ValueError("Transition model is missing")\n",
+ " else:\n",
+ " return self.transitions[state][action]\n",
+ "\n",
+ " def actions(self, state):\n",
+ " """Set of actions that can be performed in this state. By default, a\n",
+ " fixed list of actions, except for terminal states. Override this\n",
+ " method if you need to specialize by state."""\n",
+ " if state in self.terminals:\n",
+ " return [None]\n",
+ " else:\n",
+ " return self.actlist\n",
+ "\n",
+ " def get_states_from_transitions(self, transitions):\n",
+ " if isinstance(transitions, dict):\n",
+ " s1 = set(transitions.keys())\n",
+ " s2 = set([tr[1] for actions in transitions.values() \n",
+ " for effects in actions.values() for tr in effects])\n",
+ " return s1.union(s2)\n",
+ " else:\n",
+ " print('Could not retrieve states from transitions')\n",
+ " return None\n",
+ "\n",
+ " def check_consistency(self):\n",
+ " # check that all states in transitions are valid\n",
+ " assert set(self.states) == self.get_states_from_transitions(self.transitions)\n",
+ " # check that init is a valid state\n",
+ " assert self.init in self.states\n",
+ " # check reward for each state\n",
+ " #assert set(self.reward.keys()) == set(self.states)\n",
+ " assert set(self.reward.keys()) == set(self.states)\n",
+ " # check that all terminals are valid states\n",
+ " assert all([t in self.states for t in self.terminals])\n",
+ " # check that probability distributions for all actions sum to 1\n",
+ " for s1, actions in self.transitions.items():\n",
+ " for a in actions.keys():\n",
+ " s = 0\n",
+ " for o in actions[a]:\n",
+ " s += o[0]\n",
+ " assert abs(s - 1) < 0.001\n",
+ "![](\"files/images/mdp-a.png\")
"
+ ]
+ },
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 3,
"metadata": {
- "collapsed": false
+ "collapsed": true
},
"outputs": [],
"source": [
- "import mdp"
+ "# Transition Matrix as nested dict. State -> Actions in state -> List of (Probability, State) tuples\n",
+ "t = {\n",
+ " \"A\": {\n",
+ " \"X\": [(0.3, \"A\"), (0.7, \"B\")],\n",
+ " \"Y\": [(1.0, \"A\")]\n",
+ " },\n",
+ " \"B\": {\n",
+ " \"X\": {(0.8, \"End\"), (0.2, \"B\")},\n",
+ " \"Y\": {(1.0, \"A\")}\n",
+ " },\n",
+ " \"End\": {}\n",
+ "}\n",
+ "\n",
+ "init = \"A\"\n",
+ "\n",
+ "terminals = [\"End\"]\n",
+ "\n",
+ "rewards = {\n",
+ " \"A\": 5,\n",
+ " \"B\": -10,\n",
+ " \"End\": 100\n",
+ "}"
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
- "source": []
+ "source": [
+ "class CustomMDP(MDP):\n",
+ " def __init__(self, init, terminals, transition_matrix, reward = None, gamma=.9):\n",
+ " # All possible actions.\n",
+ " actlist = []\n",
+ " for state in transition_matrix.keys():\n",
+ " actlist.extend(transition_matrix[state])\n",
+ " actlist = list(set(actlist))\n",
+ " MDP.__init__(self, init, actlist, terminals, transition_matrix, reward, gamma=gamma)\n",
+ "\n",
+ " def T(self, state, action):\n",
+ " if action is None:\n",
+ " return [(0.0, state)]\n",
+ " else: \n",
+ " return self.t[state][action]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Finally we instantize the class with the parameters for our MDP in the picture."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "our_mdp = CustomMDP(init, terminals, t, rewards, gamma=.9)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "With this we have successfully represented our MDP. Later we will look at ways to solve this MDP."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## GRID MDP\n",
+ "\n",
+ "Now we look at a concrete implementation that makes use of the MDP as base class. The GridMDP class in the mdp module is used to represent a grid world MDP like the one shown in in **Fig 17.1** of the AIMA Book. We assume for now that the environment is _fully observable_, so that the agent always knows where it is. The code should be easy to understand if you have gone through the CustomMDP example."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " class GridMDP(MDP):\n",
+ "\n",
+ " """A two-dimensional grid MDP, as in [Figure 17.1]. All you have to do is\n",
+ " specify the grid as a list of lists of rewards; use None for an obstacle\n",
+ " (unreachable state). Also, you should specify the terminal states.\n",
+ " An action is an (x, y) unit vector; e.g. (1, 0) means move east."""\n",
+ "\n",
+ " def __init__(self, grid, terminals, init=(0, 0), gamma=.9):\n",
+ " grid.reverse() # because we want row 0 on bottom, not on top\n",
+ " reward = {}\n",
+ " states = set()\n",
+ " self.rows = len(grid)\n",
+ " self.cols = len(grid[0])\n",
+ " self.grid = grid\n",
+ " for x in range(self.cols):\n",
+ " for y in range(self.rows):\n",
+ " if grid[y][x] is not None:\n",
+ " states.add((x, y))\n",
+ " reward[(x, y)] = grid[y][x]\n",
+ " self.states = states\n",
+ " actlist = orientations\n",
+ " transitions = {}\n",
+ " for s in states:\n",
+ " transitions[s] = {}\n",
+ " for a in actlist:\n",
+ " transitions[s][a] = self.calculate_T(s, a)\n",
+ " MDP.__init__(self, init, actlist=actlist,\n",
+ " terminals=terminals, transitions = transitions, \n",
+ " reward = reward, states = states, gamma=gamma)\n",
+ "\n",
+ " def calculate_T(self, state, action):\n",
+ " if action is None:\n",
+ " return [(0.0, state)]\n",
+ " else:\n",
+ " return [(0.8, self.go(state, action)),\n",
+ " (0.1, self.go(state, turn_right(action))),\n",
+ " (0.1, self.go(state, turn_left(action)))]\n",
+ " \n",
+ " def T(self, state, action):\n",
+ " if action is None:\n",
+ " return [(0.0, state)]\n",
+ " else:\n",
+ " return self.transitions[state][action]\n",
+ " \n",
+ " def go(self, state, direction):\n",
+ " """Return the state that results from going in this direction."""\n",
+ " state1 = vector_add(state, direction)\n",
+ " return state1 if state1 in self.states else state\n",
+ "\n",
+ " def to_grid(self, mapping):\n",
+ " """Convert a mapping from (x, y) to v into a [[..., v, ...]] grid."""\n",
+ " return list(reversed([[mapping.get((x, y), None)\n",
+ " for x in range(self.cols)]\n",
+ " for y in range(self.rows)]))\n",
+ "\n",
+ " def to_arrows(self, policy):\n",
+ " chars = {\n",
+ " (1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'}\n",
+ " return self.to_grid({s: chars[a] for (s, a) in policy.items()})\n",
+ "def value_iteration(mdp, epsilon=0.001):\n",
+ " """Solving an MDP by value iteration. [Figure 17.4]"""\n",
+ " U1 = {s: 0 for s in mdp.states}\n",
+ " R, T, gamma = mdp.R, mdp.T, mdp.gamma\n",
+ " while True:\n",
+ " U = U1.copy()\n",
+ " delta = 0\n",
+ " for s in mdp.states:\n",
+ " U1[s] = R(s) + gamma * max([sum([p * U[s1] for (p, s1) in T(s, a)])\n",
+ " for a in mdp.actions(s)])\n",
+ " delta = max(delta, abs(U1[s] - U[s]))\n",
+ " if delta < epsilon * (1 - gamma) / gamma:\n",
+ " return U\n",
+ "Value Iteration starts with arbitrary initial values for the utilities, calculates the right side of the Bellman equation and plugs it into the left hand side, thereby updating the utility of each state from the utilities of its neighbors. \n", + "This is repeated until equilibrium is reached. \n", + "It works on the principle of _Dynamic Programming_ - using precomputed information to simplify the subsequent computation. \n", + "If $U_i(s)$ is the utility value for state $s$ at the $i$ th iteration, the iteration step, called Bellman update, looks like this:\n", + "\n", + "$$ U_{i+1}(s) \\leftarrow R(s) + \\gamma \\max_{a \\epsilon A(s)} \\sum_{s'} P(s'\\ |\\ s,a)U_{i}(s') $$\n", + "\n", + "As you might have noticed, `value_iteration` has an infinite loop. How do we decide when to stop iterating? \n", + "The concept of _contraction_ successfully explains the convergence of value iteration. \n", + "Refer to **Section 17.2.3** of the book for a detailed explanation. \n", + "In the algorithm, we calculate a value $delta$ that measures the difference in the utilities of the current time step and the previous time step. \n", + "\n", + "$$\\delta = \\max{(\\delta, \\begin{vmatrix}U_{i + 1}(s) - U_i(s)\\end{vmatrix})}$$\n", + "\n", + "This value of delta decreases as the values of $U_i$ converge.\n", + "We terminate the algorithm if the $\\delta$ value is less than a threshold value determined by the hyperparameter _epsilon_.\n", + "\n", + "$$\\delta \\lt \\epsilon \\frac{(1 - \\gamma)}{\\gamma}$$\n", + "\n", + "To summarize, the Bellman update is a _contraction_ by a factor of $gamma$ on the space of utility vectors. \n", + "Hence, from the properties of contractions in general, it follows that `value_iteration` always converges to a unique solution of the Bellman equations whenever $gamma$ is less than 1.\n", + "We then terminate the algorithm when a reasonable approximation is achieved.\n", + "In practice, it often occurs that the policy $pi$ becomes optimal long before the utility function converges. For the given 4 x 3 environment with $gamma = 0.9$, the policy $pi$ is optimal when $i = 4$ (at the 4th iteration), even though the maximum error in the utility function is stil 0.46. This can be clarified from **figure 17.6** in the book. Hence, to increase computational efficiency, we often use another method to solve MDPs called Policy Iteration which we will see in the later part of this notebook. \n", + "
For now, let us solve the **sequential_decision_environment** GridMDP using `value_iteration`." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{(0, 0): 0.2962883154554812,\n", + " (0, 1): 0.3984432178350045,\n", + " (0, 2): 0.5093943765842497,\n", + " (1, 0): 0.25386699846479516,\n", + " (1, 2): 0.649585681261095,\n", + " (2, 0): 0.3447542300124158,\n", + " (2, 1): 0.48644001739269643,\n", + " (2, 2): 0.7953620878466678,\n", + " (3, 0): 0.12987274656746342,\n", + " (3, 1): -1.0,\n", + " (3, 2): 1.0}" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "value_iteration(sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The pseudocode for the algorithm:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "### AIMA3e\n", + "__function__ VALUE-ITERATION(_mdp_, _ε_) __returns__ a utility function \n", + " __inputs__: _mdp_, an MDP with states _S_, actions _A_(_s_), transition model _P_(_s′_ | _s_, _a_), \n", + " rewards _R_(_s_), discount _γ_ \n", + " _ε_, the maximum error allowed in the utility of any state \n", + " __local variables__: _U_, _U′_, vectors of utilities for states in _S_, initially zero \n", + " _δ_, the maximum change in the utility of any state in an iteration \n", + "\n", + " __repeat__ \n", + " _U_ ← _U′_; _δ_ ← 0 \n", + " __for each__ state _s_ in _S_ __do__ \n", + " _U′_\\[_s_\\] ← _R_(_s_) + _γ_ max_a_ ∈ _A_(_s_) Σ _P_(_s′_ | _s_, _a_) _U_\\[_s′_\\] \n", + " __if__ | _U′_\\[_s_\\] − _U_\\[_s_\\] | > _δ_ __then__ _δ_ ← | _U′_\\[_s_\\] − _U_\\[_s_\\] | \n", + " __until__ _δ_ < _ε_(1 − _γ_)/_γ_ \n", + " __return__ _U_ \n", + "\n", + "---\n", + "__Figure ??__ The value iteration algorithm for calculating utilities of states. The termination condition is from Equation (__??__)." + ], + "text/plain": [ + "
def expected_utility(a, s, U, mdp):\n",
+ " """The expected utility of doing a in state s, according to the MDP and U."""\n",
+ " return sum([p * U[s1] for (p, s1) in mdp.T(s, a)])\n",
+ "def policy_iteration(mdp):\n",
+ " """Solve an MDP by policy iteration [Figure 17.7]"""\n",
+ " U = {s: 0 for s in mdp.states}\n",
+ " pi = {s: random.choice(mdp.actions(s)) for s in mdp.states}\n",
+ " while True:\n",
+ " U = policy_evaluation(pi, U, mdp)\n",
+ " unchanged = True\n",
+ " for s in mdp.states:\n",
+ " a = argmax(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp))\n",
+ " if a != pi[s]:\n",
+ " pi[s] = a\n",
+ " unchanged = False\n",
+ " if unchanged:\n",
+ " return pi\n",
+ "Fortunately, it is not necessary to do _exact_ policy evaluation. \n", + "The utilities can instead be reasonably approximated by performing some number of simplified value iteration steps.\n", + "The simplified Bellman update equation for the process is\n", + "\n", + "$$U_{i+1}(s) \\leftarrow R(s) + \\gamma\\sum_{s'}P(s'\\ |\\ s,\\pi_i(s))U_{i}(s')$$\n", + "\n", + "and this is repeated _k_ times to produce the next utility estimate. This is called _modified policy iteration_." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def policy_evaluation(pi, U, mdp, k=20):\n",
+ " """Return an updated utility mapping U from each state in the MDP to its\n",
+ " utility, using an approximation (modified policy iteration)."""\n",
+ " R, T, gamma = mdp.R, mdp.T, mdp.gamma\n",
+ " for i in range(k):\n",
+ " for s in mdp.states:\n",
+ " U[s] = R(s) + gamma * sum([p * U[s1] for (p, s1) in T(s, pi[s])])\n",
+ " return U\n",
+ "This is the environment for our agent.\n", + "We assume for now that the environment is _fully observable_, so that the agent always knows where it is.\n", + "We also assume that the transitions are **Markovian**, that is, the probability of reaching state $s'$ from state $s$ depends only on $s$ and not on the history of earlier states.\n", + "Almost all stochastic decision problems can be reframed as a Markov Decision Process just by tweaking the definition of a _state_ for that particular problem.\n", + "
\n", + "However, the actions of our agent in this environment are unreliable. In other words, the motion of our agent is stochastic. \n", + "
\n", + "More specifically, the agent may - \n", + "* move correctly in the intended direction with a probability of _0.8_, \n", + "* move $90^\\circ$ to the right of the intended direction with a probability 0.1\n", + "* move $90^\\circ$ to the left of the intended direction with a probability 0.1\n", + "
\n", + "The agent stays put if it bumps into a wall.\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These properties of the agent are called the transition properties and are hardcoded into the GridMDP class as you can see below." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def T(self, state, action):\n",
+ " if action is None:\n",
+ " return [(0.0, state)]\n",
+ " else:\n",
+ " return self.transitions[state][action]\n",
+ "\n", + "Such a solution is known as a **policy** and is usually denoted by $\\pi$.\n", + "
\n", + "The **optimal policy** is the policy that yields the highest expected utility an is usually denoted by $\\pi^*$.\n", + "
\n", + "The `GridMDP` class has a useful method `to_arrows` that outputs a grid showing the direction the agent should move, given a policy.\n", + "We will use this later to better understand the properties of the environment." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def to_arrows(self, policy):\n",
+ " chars = {\n",
+ " (1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'}\n",
+ " return self.to_grid({s: chars[a] for (s, a) in policy.items()})\n",
+ " def to_grid(self, mapping):\n",
+ " """Convert a mapping from (x, y) to v into a [[..., v, ...]] grid."""\n",
+ " return list(reversed([[mapping.get((x, y), None)\n",
+ " for x in range(self.cols)]\n",
+ " for y in range(self.rows)]))\n",
+ "\n", + "\n", + "
\n", + "Notice that, because the cost of taking a step is fairly small compared with the penalty for ending up in `(4, 2)` by accident, the optimal policy is conservative. \n", + "In state `(3, 1)` it recommends taking the long way round, rather than taking the shorter way and risking getting a large negative reward of -1 in `(4, 2)`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Case 2\n", + "---\n", + "R(s) = -0.4 in all states except in terminal states" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "sequential_decision_environment = GridMDP([[-0.4, -0.4, -0.4, +1],\n", + " [-0.4, None, -0.4, -1],\n", + " [-0.4, -0.4, -0.4, -0.4]],\n", + " terminals=[(3, 2), (3, 1)])" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "> > > .\n", + "^ None ^ .\n", + "^ > ^ <\n" + ] + } + ], + "source": [ + "pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .001))\n", + "from utils import print_table\n", + "print_table(sequential_decision_environment.to_arrows(pi))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is exactly the output we expected\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As the reward for each state is now more negative, life is certainly more unpleasant.\n", + "The agent takes the shortest route to the +1 state and is willing to risk falling into the -1 state by accident." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Case 3\n", + "---\n", + "R(s) = -4 in all states except terminal states" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "sequential_decision_environment = GridMDP([[-4, -4, -4, +1],\n", + " [-4, None, -4, -1],\n", + " [-4, -4, -4, -4]],\n", + " terminals=[(3, 2), (3, 1)])" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "> > > .\n", + "^ None > .\n", + "> > > ^\n" + ] + } + ], + "source": [ + "pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .001))\n", + "from utils import print_table\n", + "print_table(sequential_decision_environment.to_arrows(pi))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is exactly the output we expected\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The living reward for each state is now lower than the least rewarding terminal. Life is so _painful_ that the agent heads for the nearest exit as even the worst exit is less painful than any living state." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Case 4\n", + "---\n", + "R(s) = 4 in all states except terminal states" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "sequential_decision_environment = GridMDP([[4, 4, 4, +1],\n", + " [4, None, 4, -1],\n", + " [4, 4, 4, 4]],\n", + " terminals=[(3, 2), (3, 1)])" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "> > < .\n", + "> None < .\n", + "> > > v\n" + ] + } + ], + "source": [ + "pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .001))\n", + "from utils import print_table\n", + "print_table(sequential_decision_environment.to_arrows(pi))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case, the output we expect is\n", + "\n", + "
\n", + "As life is positively enjoyable and the agent avoids _both_ exits.\n", + "Even though the output we get is not exactly what we want, it is definitely not wrong.\n", + "The scenario here requires the agent to anything but reach a terminal state, as this is the only way the agent can maximize its reward (total reward tends to infinity), and the program does just that.\n", + "
\n", + "Currently, the GridMDP class doesn't support an explicit marker for a \"do whatever you like\" action or a \"don't care\" condition.\n", + "You can however, extend the class to do so.\n", + "
\n", + "For in-depth knowledge about sequential decision problems, refer **Section 17.1** in the AIMA book." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## POMDP\n", + "---\n", + "Partially Observable Markov Decision Problems\n", + "\n", + "In retrospect, a Markov decision process or MDP is defined as:\n", + "- a sequential decision problem for a fully observable, stochastic environment with a Markovian transition model and additive rewards.\n", + "\n", + "An MDP consists of a set of states (with an initial state $s_0$); a set $A(s)$ of actions\n", + "in each state; a transition model $P(s' | s, a)$; and a reward function $R(s)$.\n", + "\n", + "The MDP seeks to make sequential decisions to occupy states so as to maximise some combination of the reward function $R(s)$.\n", + "\n", + "The characteristic problem of the MDP is hence to identify the optimal policy function $\\pi^*(s)$ that provides the _utility-maximising_ action $a$ to be taken when the current state is $s$.\n", + "\n", + "### Belief vector\n", + "\n", + "**Note**: The book refers to the _belief vector_ as the _belief state_. We use the latter terminology here to retain our ability to refer to the belief vector as a _probability distribution over states_.\n", + "\n", + "The solution of an MDP is subject to certain properties of the problem which are assumed and justified in [Section 17.1]. One critical assumption is that the agent is **fully aware of its current state at all times**.\n", + "\n", + "A tedious (but rewarding, as we will see) way of expressing this is in terms of the **belief vector** $b$ of the agent. The belief vector is a function mapping states to probabilities or certainties of being in those states.\n", + "\n", + "Consider an agent that is fully aware that it is in state $s_i$ in the statespace $(s_1, s_2, ... s_n)$ at the current time.\n", + "\n", + "Its belief vector is the vector $(b(s_1), b(s_2), ... b(s_n))$ given by the function $b(s)$:\n", + "\\begin{align*}\n", + "b(s) &= 0 \\quad \\text{if }s \\neq s_i \\\\ &= 1 \\quad \\text{if } s = s_i\n", + "\\end{align*}\n", + "\n", + "Note that $b(s)$ is a probability distribution that necessarily sums to $1$ over all $s$.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "### POMDPs - a conceptual outline\n", + "\n", + "The POMDP really has only two modifications to the **problem formulation** compared to the MDP.\n", + "\n", + "- **Belief state** - In the real world, the current state of an agent is often not known with complete certainty. This makes the concept of a belief vector extremely relevant. It allows the agent to represent different degrees of certainty with which it _believes_ it is in each state.\n", + "\n", + "- **Evidence percepts** - In the real world, agents often have certain kinds of evidence, collected from sensors. They can use the probability distribution of observed evidence, conditional on state, to consolidate their information. This is a known distribution $P(e\\ |\\ s)$ - $e$ being an evidence, and $s$ being the state it is conditional on.\n", + "\n", + "Consider the world we used for the MDP. \n", + "\n", + "\n", + "\n", + "#### Using the belief vector\n", + "An agent beginning at $(1, 1)$ may not be certain that it is indeed in $(1, 1)$. Consider a belief vector $b$ such that:\n", + "\\begin{align*}\n", + " b((1,1)) &= 0.8 \\\\\n", + " b((2,1)) &= 0.1 \\\\\n", + " b((1,2)) &= 0.1 \\\\\n", + " b(s) &= 0 \\quad \\quad \\forall \\text{ other } s\n", + "\\end{align*}\n", + "\n", + "By horizontally catenating each row, we can represent this as an 11-dimensional vector (omitting $(2, 2)$).\n", + "\n", + "Thus, taking $s_1 = (1, 1)$, $s_2 = (1, 2)$, ... $s_{11} = (4,3)$, we have $b$:\n", + "\n", + "$b = (0.8, 0.1, 0, 0, 0.1, 0, 0, 0, 0, 0, 0)$ \n", + "\n", + "This fully represents the certainty to which the agent is aware of its state.\n", + "\n", + "#### Using evidence\n", + "The evidence observed here could be the number of adjacent 'walls' or 'dead ends' observed by the agent. We assume that the agent cannot 'orient' the walls - only count them.\n", + "\n", + "In this case, $e$ can take only two values, 1 and 2. This gives $P(e\\ |\\ s)$ as:\n", + "\\begin{align*}\n", + " P(e=2\\ |\\ s) &= \\frac{1}{7} \\quad \\forall \\quad s \\in \\{s_1, s_2, s_4, s_5, s_8, s_9, s_{11}\\}\\\\\n", + " P(e=1\\ |\\ s) &= \\frac{1}{4} \\quad \\forall \\quad s \\in \\{s_3, s_6, s_7, s_{10}\\} \\\\\n", + " P(e\\ |\\ s) &= 0 \\quad \\forall \\quad \\text{ other } s, e\n", + "\\end{align*}\n", + "\n", + "Note that the implications of the evidence on the state must be known **a priori** to the agent. Ways of reliably learning this distribution from percepts are beyond the scope of this notebook." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### POMDPs - a rigorous outline\n", + "\n", + "A POMDP is thus a sequential decision problem for for a *partially* observable, stochastic environment with a Markovian transition model, a known 'sensor model' for inferring state from observation, and additive rewards. \n", + "\n", + "Practically, a POMDP has the following, which an MDP also has:\n", + "- a set of states, each denoted by $s$\n", + "- a set of actions available in each state, $A(s)$\n", + "- a reward accrued on attaining some state, $R(s)$\n", + "- a transition probability $P(s'\\ |\\ s, a)$ of action $a$ changing the state from $s$ to $s'$\n", + "\n", + "And the following, which an MDP does not:\n", + "- a sensor model $P(e\\ |\\ s)$ on evidence conditional on states\n", + "\n", + "Additionally, the POMDP is now uncertain of its current state hence has:\n", + "- a belief vector $b$ representing the certainty of being in each state (as a probability distribution)\n", + "\n", + "\n", + "#### New uncertainties\n", + "\n", + "It is useful to intuitively appreciate the new uncertainties that have arisen in the agent's awareness of its own state.\n", + "\n", + "- At any point, the agent has belief vector $b$, the distribution of its believed likelihood of being in each state $s$.\n", + "- For each of these states $s$ that the agent may **actually** be in, it has some set of actions given by $A(s)$.\n", + "- Each of these actions may transport it to some other state $s'$, assuming an initial state $s$, with probability $P(s'\\ |\\ s, a)$\n", + "- Once the action is performed, the agent receives a percept $e$. $P(e\\ |\\ s)$ now tells it the chances of having perceived $e$ for each state $s$. The agent must use this information to update its new belief state appropriately.\n", + "\n", + "#### Evolution of the belief vector - the `FORWARD` function\n", + "\n", + "The new belief vector $b'(s')$ after an action $a$ on the belief vector $b(s)$ and the noting of evidence $e$ is:\n", + "$$ b'(s') = \\alpha P(e\\ |\\ s') \\sum_s P(s'\\ | s, a) b(s)$$ \n", + "\n", + "where $\\alpha$ is a normalising constant (to retain the interpretation of $b$ as a probability distribution.\n", + "\n", + "This equation is just counts the sum of likelihoods of going to a state $s'$ from every possible state $s$, times the initial likelihood of being in each $s$. This is multiplied by the likelihood that the known evidence actually implies the new state $s'$. \n", + "\n", + "This function is represented as `b' = FORWARD(b, a, e)`\n", + "\n", + "#### Probability distribution of the evolving belief vector\n", + "\n", + "The goal here is to find $P(b'\\ |\\ b, a)$ - the probability that action $a$ transforms belief vector $b$ into belief vector $b'$. The following steps illustrate this -\n", + "\n", + "The probability of observing evidence $e$ when action $a$ is enacted on belief vector $b$ can be distributed over each possible new state $s'$ resulting from it:\n", + "\\begin{align*}\n", + " P(e\\ |\\ b, a) &= \\sum_{s'} P(e\\ |\\ b, a, s') P(s'\\ |\\ b, a) \\\\\n", + " &= \\sum_{s'} P(e\\ |\\ s') P(s'\\ |\\ b, a) \\\\\n", + " &= \\sum_{s'} P(e\\ |\\ s') \\sum_s P(s'\\ |\\ s, a) b(s)\n", + "\\end{align*}\n", + "\n", + "The probability of getting belief vector $b'$ from $b$ by application of action $a$ can thus be summed over all possible evidences $e$:\n", + "\\begin{align*}\n", + " P(b'\\ |\\ b, a) &= \\sum_{e} P(b'\\ |\\ b, a, e) P(e\\ |\\ b, a) \\\\\n", + " &= \\sum_{e} P(b'\\ |\\ b, a, e) \\sum_{s'} P(e\\ |\\ s') \\sum_s P(s'\\ |\\ s, a) b(s)\n", + "\\end{align*}\n", + "\n", + "where $P(b'\\ |\\ b, a, e) = 1$ if $b' = $ `FORWARD(b, a, e)` and $= 0$ otherwise.\n", + "\n", + "Given initial and final belief states $b$ and $b'$, the transition probabilities still depend on the action $a$ and observed evidence $e$. Some belief states may be achievable by certain actions, but have non-zero probabilities for states prohibited by the evidence $e$. Thus, the above condition thus ensures that only valid combinations of $(b', b, a, e)$ are considered.\n", + "\n", + "#### A modified rewardspace\n", + "\n", + "For MDPs, the reward space was simple - one reward per available state. However, for a belief vector $b(s)$, the expected reward is now:\n", + "$$\\rho(b) = \\sum_s b(s) R(s)$$\n", + "\n", + "Thus, as the belief vector can take infinite values of the distribution over states, so can the reward for each belief vector vary over a hyperplane in the belief space, or space of states (planes in an $N$-dimensional space are formed by a linear combination of the axes)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we know the basics, let's have a look at the `POMDP` class." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class POMDP(MDP):\n",
+ "\n",
+ " """A Partially Observable Markov Decision Process, defined by\n",
+ " a transition model P(s'|s,a), actions A(s), a reward function R(s),\n",
+ " and a sensor model P(e|s). We also keep track of a gamma value,\n",
+ " for use by algorithms. The transition and the sensor models\n",
+ " are defined as matrices. We also keep track of the possible states\n",
+ " and actions for each state. [page 659]."""\n",
+ "\n",
+ " def __init__(self, actions, transitions=None, evidences=None, rewards=None, states=None, gamma=0.95):\n",
+ " """Initialize variables of the pomdp"""\n",
+ "\n",
+ " if not (0 < gamma <= 1):\n",
+ " raise ValueError('A POMDP must have 0 < gamma <= 1')\n",
+ "\n",
+ " self.states = states\n",
+ " self.actions = actions\n",
+ "\n",
+ " # transition model cannot be undefined\n",
+ " self.t_prob = transitions or {}\n",
+ " if not self.t_prob:\n",
+ " print('Warning: Transition model is undefined')\n",
+ " \n",
+ " # sensor model cannot be undefined\n",
+ " self.e_prob = evidences or {}\n",
+ " if not self.e_prob:\n",
+ " print('Warning: Sensor model is undefined')\n",
+ " \n",
+ " self.gamma = gamma\n",
+ " self.rewards = rewards\n",
+ "\n",
+ " def remove_dominated_plans(self, input_values):\n",
+ " """\n",
+ " Remove dominated plans.\n",
+ " This method finds all the lines contributing to the\n",
+ " upper surface and removes those which don't.\n",
+ " """\n",
+ "\n",
+ " values = [val for action in input_values for val in input_values[action]]\n",
+ " values.sort(key=lambda x: x[0], reverse=True)\n",
+ "\n",
+ " best = [values[0]]\n",
+ " y1_max = max(val[1] for val in values)\n",
+ " tgt = values[0]\n",
+ " prev_b = 0\n",
+ " prev_ix = 0\n",
+ " while tgt[1] != y1_max:\n",
+ " min_b = 1\n",
+ " min_ix = 0\n",
+ " for i in range(prev_ix + 1, len(values)):\n",
+ " if values[i][0] - tgt[0] + tgt[1] - values[i][1] != 0:\n",
+ " trans_b = (values[i][0] - tgt[0]) / (values[i][0] - tgt[0] + tgt[1] - values[i][1])\n",
+ " if 0 <= trans_b <= 1 and trans_b > prev_b and trans_b < min_b:\n",
+ " min_b = trans_b\n",
+ " min_ix = i\n",
+ " prev_b = min_b\n",
+ " prev_ix = min_ix\n",
+ " tgt = values[min_ix]\n",
+ " best.append(tgt)\n",
+ "\n",
+ " return self.generate_mapping(best, input_values)\n",
+ "\n",
+ " def remove_dominated_plans_fast(self, input_values):\n",
+ " """\n",
+ " Remove dominated plans using approximations.\n",
+ " Resamples the upper boundary at intervals of 100 and\n",
+ " finds the maximum values at these points.\n",
+ " """\n",
+ "\n",
+ " values = [val for action in input_values for val in input_values[action]]\n",
+ " values.sort(key=lambda x: x[0], reverse=True)\n",
+ "\n",
+ " best = []\n",
+ " sr = 100\n",
+ " for i in range(sr + 1):\n",
+ " x = i / float(sr)\n",
+ " maximum = (values[0][1] - values[0][0]) * x + values[0][0]\n",
+ " tgt = values[0]\n",
+ " for value in values:\n",
+ " val = (value[1] - value[0]) * x + value[0]\n",
+ " if val > maximum:\n",
+ " maximum = val\n",
+ " tgt = value\n",
+ "\n",
+ " if all(any(tgt != v) for v in best):\n",
+ " best.append(tgt)\n",
+ "\n",
+ " return self.generate_mapping(best, input_values)\n",
+ "\n",
+ " def generate_mapping(self, best, input_values):\n",
+ " """Generate mappings after removing dominated plans"""\n",
+ "\n",
+ " mapping = defaultdict(list)\n",
+ " for value in best:\n",
+ " for action in input_values:\n",
+ " if any(all(value == v) for v in input_values[action]):\n",
+ " mapping[action].append(value)\n",
+ "\n",
+ " return mapping\n",
+ "\n",
+ " def max_difference(self, U1, U2):\n",
+ " """Find maximum difference between two utility mappings"""\n",
+ "\n",
+ " for k, v in U1.items():\n",
+ " sum1 = 0\n",
+ " for element in U1[k]:\n",
+ " sum1 += sum(element)\n",
+ " sum2 = 0\n",
+ " for element in U2[k]:\n",
+ " sum2 += sum(element)\n",
+ " return abs(sum1 - sum2)\n",
+ "\n", + "
\n", + "`remove_dominated_plans`, `remove_dominated_plans_fast`, `generate_mapping` and `max_difference` are helper methods for `pomdp_value_iteration` which will be explained shortly." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To understand how we can model a partially observable MDP, let's take a simple example.\n", + "Let's consider a simple two state world.\n", + "The states are labelled 0 and 1, with the reward at state 0 being 0 and at state 1 being 1.\n", + "
\n", + "There are two actions:\n", + "
\n", + "`Stay`: stays put with probability 0.9 and\n", + "`Go`: switches to the other state with probability 0.9.\n", + "
\n", + "For now, let's assume the discount factor `gamma` to be 1.\n", + "
\n", + "The sensor reports the correct state with probability 0.6.\n", + "
\n", + "This is a simple problem with a trivial solution.\n", + "Obviously the agent should `Stay` when it thinks it is in state 1 and `Go` when it thinks it is in state 0.\n", + "
\n", + "The belief space can be viewed as one-dimensional because the two probabilities must sum to 1." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's model this POMDP using the `POMDP` class." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "# transition probability P(s'|s,a)\n", + "t_prob = [[[0.9, 0.1], [0.1, 0.9]], [[0.1, 0.9], [0.9, 0.1]]]\n", + "# evidence function P(e|s)\n", + "e_prob = [[[0.6, 0.4], [0.4, 0.6]], [[0.6, 0.4], [0.4, 0.6]]]\n", + "# reward function\n", + "rewards = [[0.0, 0.0], [1.0, 1.0]]\n", + "# discount factor\n", + "gamma = 0.95\n", + "# actions\n", + "actions = ('0', '1')\n", + "# states\n", + "states = ('0', '1')" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have defined our `POMDP` object." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## POMDP VALUE ITERATION\n", + "Defining a POMDP is useless unless we can find a way to solve it. As POMDPs can have infinitely many belief states, we cannot calculate one utility value for each state as we did in `value_iteration` for MDPs.\n", + "
\n", + "Instead of thinking about policies, we should think about conditional plans and how the expected utility of executing a fixed conditional plan varies with the initial belief state.\n", + "
\n", + "If we bound the depth of the conditional plans, then there are only finitely many such plans and the continuous space of belief states will generally be divided inte _regions_, each corresponding to a particular conditional plan that is optimal in that region. The utility function, being the maximum of a collection of hyperplanes, will be piecewise linear and convex.\n", + "
\n", + "For the one-step plans `Stay` and `Go`, the utility values are as follows\n", + "
\n", + "
\n", + "$$\\alpha_{|Stay|}(0) = R(0) + \\gamma(0.9R(0) + 0.1R(1)) = 0.1$$\n", + "$$\\alpha_{|Stay|}(1) = R(1) + \\gamma(0.9R(1) + 0.1R(0)) = 1.9$$\n", + "$$\\alpha_{|Go|}(0) = R(0) + \\gamma(0.9R(1) + 0.1R(0)) = 0.9$$\n", + "$$\\alpha_{|Go|}(1) = R(1) + \\gamma(0.9R(0) + 0.1R(1)) = 1.1$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The utility function can be found by `pomdp_value_iteration`.\n", + "
\n", + "To summarize, it generates a set of all plans consisting of an action and, for each possible next percept, a plan in U with computed utility vectors.\n", + "The dominated plans are then removed from this set and the process is repeated till the maximum difference between the utility functions of two consecutive iterations reaches a value less than a threshold value." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "### AIMA3e\n", + "__function__ POMDP-VALUE-ITERATION(_pomdp_, _ε_) __returns__ a utility function \n", + " __inputs__: _pomdp_, a POMDP with states _S_, actions _A_(_s_), transition model _P_(_s′_ | _s_, _a_), \n", + " sensor model _P_(_e_ | _s_), rewards _R_(_s_), discount _γ_ \n", + " _ε_, the maximum error allowed in the utility of any state \n", + " __local variables__: _U_, _U′_, sets of plans _p_ with associated utility vectors _αp_ \n", + "\n", + " _U′_ ← a set containing just the empty plan \\[\\], with _α\\[\\]_(_s_) = _R_(_s_) \n", + " __repeat__ \n", + " _U_ ← _U′_ \n", + " _U′_ ← the set of all plans consisting of an action and, for each possible next percept, \n", + " a plan in _U_ with utility vectors computed according to Equation(__??__) \n", + " _U′_ ← REMOVE\\-DOMINATED\\-PLANS(_U′_) \n", + " __until__ MAX\\-DIFFERENCE(_U_, _U′_) < _ε_(1 − _γ_) ⁄ _γ_ \n", + " __return__ _U_ \n", + "\n", + "---\n", + "__Figure ??__ A high\\-level sketch of the value iteration algorithm for POMDPs. The REMOVE\\-DOMINATED\\-PLANS step and MAX\\-DIFFERENCE test are typically implemented as linear programs." + ], + "text/plain": [ + "
def pomdp_value_iteration(pomdp, epsilon=0.1):\n",
+ " """Solving a POMDP by value iteration."""\n",
+ "\n",
+ " U = {'':[[0]* len(pomdp.states)]}\n",
+ " count = 0\n",
+ " while True:\n",
+ " count += 1\n",
+ " prev_U = U\n",
+ " values = [val for action in U for val in U[action]]\n",
+ " value_matxs = []\n",
+ " for i in values:\n",
+ " for j in values:\n",
+ " value_matxs.append([i, j])\n",
+ "\n",
+ " U1 = defaultdict(list)\n",
+ " for action in pomdp.actions:\n",
+ " for u in value_matxs:\n",
+ " u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), [[1], [1]])\n",
+ " u1 = Matrix.add(Matrix.scalar_multiply(pomdp.gamma, Matrix.transpose(u1)), [pomdp.rewards[int(action)]])\n",
+ " U1[action].append(u1[0])\n",
+ "\n",
+ " U = pomdp.remove_dominated_plans_fast(U1)\n",
+ " # replace with U = pomdp.remove_dominated_plans(U1) for accurate calculations\n",
+ " \n",
+ " if count > 10:\n",
+ " if pomdp.max_difference(U, prev_U) < epsilon * (1 - pomdp.gamma) / pomdp.gamma:\n",
+ " return U\n",
+ "\n", + "Consider the problem of a user listening to voicemails.\n", + "At the end of each message, they can either _save_ or _delete_ a message.\n", + "This forms the unobservable state _S = {save, delete}_.\n", + "It is the task of the POMDP solver to guess which goal the user has.\n", + "
\n", + "The belief space has two elements, _b(s = save)_ and _b(s = delete)_.\n", + "For example, for the belief state _b = (1, 0)_, the left end of the line segment indicates _b(s = save) = 1_ and _b(s = delete) = 0_.\n", + "The intermediate points represent varying degrees of certainty in the user's goal.\n", + "
\n", + "The machine has three available actions: it can _ask_ what the user wishes to do in order to infer his or her current goal, or it can _doSave_ or _doDelete_ and move to the next message.\n", + "If the user says _save_, then an error may occur with probability 0.2, whereas if the user says _delete_, an error may occur with a probability 0.3.\n", + "
\n", + "The machine receives a large positive reward (+5) for getting the user's goal correct, a very large negative reward (-20) for taking the action _doDelete_ when the user wanted _save_, and a smaller but still significant negative reward (-10) for taking the action _doSave_ when the user wanted _delete_. \n", + "There is also a small negative reward for taking the _ask_ action (-1).\n", + "The discount factor is set to 0.95 for this example.\n", + "
\n", + "Let's define the POMDP." + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [], + "source": [ + "# transition function P(s'|s,a)\n", + "t_prob = [[[0.65, 0.35], [0.65, 0.35]], [[0.65, 0.35], [0.65, 0.35]], [[1.0, 0.0], [0.0, 1.0]]]\n", + "# evidence function P(e|s)\n", + "e_prob = [[[0.5, 0.5], [0.5, 0.5]], [[0.5, 0.5], [0.5, 0.5]], [[0.8, 0.2], [0.3, 0.7]]]\n", + "# reward function\n", + "rewards = [[5, -10], [-20, 5], [-1, -1]]\n", + "\n", + "gamma = 0.95\n", + "actions = ('0', '1', '2')\n", + "states = ('0', '1')\n", + "\n", + "pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have defined the `POMDP` object.\n", + "Let's run `pomdp_value_iteration` to find the utility function." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "utility = pomdp_value_iteration(pomdp, epsilon=0.1)" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXd81dX9/5+fm733JiEBEkYYYRNkrwABW9yjWq2tP7WuVq24v0WtiKNaF1K0al3ggNIEhIDKEhlhRCBkD7L33rnn98eH+ykrZN2bm3Gej0cekuQzziee+359zjnv83orQggkEolEMjDRmbsBEolEIjEfUgQkEolkACNFQCKRSAYwUgQkEolkACNFQCKRSAYwUgQkEolkACNFQCKRSAYwUgQkEolkACNFQCKRSAYwluZuwPl4enqK4OBgczdDIpFI+hTx8fElQgivrpzbq0QgODiYI0eOmLsZEolE0qdQFCWrq+fK6SCJRCIZwEgRkEgkkgGMFAGJRCIZwEgRkEgkkgGMFAGJRCIZwEgRkEgkkgGMFAGJRCIZwEgRkEgkkgGMFAGJRCIZwEgRkEgkkgHMgBaBF198kfDwcMaOHUtERAQHDx40d5MkfYBNmzahKApnzpy54nGOjo491CJJW1hYWBAREUF4eDjjxo3j9ddfR6/XX/GczMxMRo8e3e4xn3/+uTGbajYGrAgcOHCAmJgYjh49SkJCAjt37iQwMNDczZL0Ab744gtmzJjBl19+ae6mSNrBzs6O48ePc+rUKeLi4ti6dSt//etfu31dKQL9gPz8fDw9PbGxsQHA09MTf39/Vq1axeTJkxk9ejR33303QggSExOZMmWKdm5mZiZjx44FID4+ntmzZzNx4kSioqLIz883y/NIeoaamhr279/PBx98oIlAfn4+s2bNIiIigtGjR7N3794LzikpKSEyMpLY2FhzNFlyDm9vb9atW8fbb7+NEILW1lYee+wxJk+ezNixY3n//fcvOaetY1auXMnevXuJiIjg73//e4eu1WsRQvSar4kTJ4qeorq6WowbN06EhoaKe++9V/z4449CCCFKS0u1Y37zm9+ILVu2CCGEGDdunEhLSxNCCLF69Wrx/PPPi6amJhEZGSmKioqEEEJ8+eWX4s477+yxZ5D0PP/+97/F7373OyGEEJGRkSI+Pl68+uqr4oUXXhBCCNHS0iKqqqqEEEI4ODiIgoICMWXKFLFjxw6ztXkg4+DgcMnPXF1dRUFBgXj//ffF888/L4QQoqGhQUycOFGkp6eLjIwMER4eLoQQbR7zww8/iOjoaO2abR3XUwBHRBfjrlGspBVF+RBYBhQJIUaf+5k7sAEIBjKBG4QQ5ca4nzFwdHQkPj6evXv38sMPP3DjjTeyevVqnJycWLNmDXV1dZSVlREeHs7y5cu54YYb2LhxIytXrmTDhg1s2LCBpKQkTp48ycKFCwH1rcHPz8/MTyYxJV988QUPP/wwADfddBNffPEFy5cv53e/+x3Nzc38+te/JiIiAoDm5mbmz5/PO++8w+zZs83ZbMl5qDETduzYQUJCAl9//TUAlZWVpKSkEBYWph3b1jHW1tYXXLOt40JCQnrikbpHV9Xj/C9gFjABOHnez9YAK8/9eyXwcnvXCRsdJhpbGk2ilO3x1VdfiQULFghvb2+RnZ0thBDiueeeE88995wQQojU1FQxfvx4kZSUJCZMmCCEECIhIUFMmzbNLO2V9DwlJSXC1tZWBAUFicGDB4tBgwaJwMBAodfrRW5urli3bp0YPXq0+Pjjj4UQQtjb24vbb79dPPHEE2Zu+cDl4pFAWlqacHd3F3q9XlxzzTXiu+++u+Sc80cCbR1z8UigreN6gvz87o0EjLImIITYA5Rd9ONfAR+f+/fHwK/bu05yaTKeazy5buN1fHT8IwprCo3RvMuSlJRESkqK9v3x48cZPnw4oK4P1NTUaKoOMHToUCwsLHj++ee58cYbARg+fDjFxcUcOHAAUN/8Tp06ZbI2S8zL119/ze23305WVhaZmZmcPXuWkJAQ9uzZg7e3N3/4wx+46667OHr0KACKovDhhx9y5swZVq9ebebWS4qLi7nnnnu4//77URSFqKgo3nvvPZqbmwFITk6mtrb2gnPaOsbJyYnq6up2jzMlubnw0EPQ3cGGKSuL+Qgh8gGEEPmKoni3d8JQ96HMGz2P2JRYvkn8BgWFyQGTWRa6jOiwaMb7jkdRFKM0rqamhgceeICKigosLS0ZNmwY69atw9XVlTFjxhAcHMzkyZMvOOfGG2/kscceIyMjAwBra2u+/vprHnzwQSorK2lpaeHhhx8mPDzcKG2U9C6++OILVq5cecHPrr32Wu644w4cHBywsrLC0dGRTz75RPu9hYUFX375JcuXL8fZ2Zn77ruvp5s9oKmvryciIoLm5mYsLS257bbb+POf/wzA73//ezIzM5kwYQJCCLy8vNi8efMF57d1zNixY7G0tGTcuHHccccdPPTQQ+1ey1hkZ8PLL8P69aDXw223wb/+1fXrKeLc/Fh3URQlGIgR/1sTqBBCuJ73+3IhhNtlzrsbuBsgKChoYlZWFkIIjhccJyY5htiUWA7lHkIg8HfyZ+mwpSwLW8aCIQtwsHYwStslEomkt5OZCS+99L+Af+edsHKlOhJQFCVeCDGpK9c1pQgkAXPOjQL8gB+FEMOvdI1JkyaJy9UYLqwpZFvqNmJTYtmeup3qpmpsLGyYEzyHZWHLiA6NJsStDyzAmIGdZeos3QJ3dzO3RCLpOgO5H6elwd/+Bp98Ajod/P738PjjEBT0v2N6qwi8ApQKIVYrirIScBdC/OVK12hLBM6nqbWJfdn7iEmOISY5hpQydV5/lNcookOjWRa2jOmB07HUmXKmq+8w59gxAH4cP97MLZFIus5A7MdJSWrw/+wzsLKCu++Gv/wFAgIuPdbsIqAoyhfAHMATKASeAzYDG4EgIBu4Xghx8eLxBXREBC4muTSZ2ORYYlNi2Z21mxZ9C662riwetphloctYPGwxHvYeXXiq/sFA/PBI+h8DqR+fPg0vvghffgk2NnDvvfDoo3Cl7PPuiIBRXpeFEDe38av5xrj+lQjzCCMsMow/Rf6JqsYq4tLiiEmJYWvKVr48+SU6RUfkoEhtlDDae7TRFpclEonEWPzyC7zwAnz1Fdjbq4H/kUfAu92Umu7Rr+ZMnG2cuXbUtVw76lr0Qs+RvCPa4vKT3z/Jk98/SZBLkCYIc4PnYmdlZ+5mSySSAczx4/D88/Dtt+DkBE88AX/6E3h69sz9e5UItOfu1xl0io4pAVOYEjCFVXNXkVedx9aUrcQkx/DxiY9578h72FnaMX/IfC0FdZDzIKPdXyKRSK7EkSNq8N+yBVxc4Nln1bz/nl77NtrCsDFQFEUA6HQ6fH19iYqK4qmnnmLo0KFGvU9DSwO7M3eri8spMWRWZAIwzmecNkqYEjAFC52FUe9rDpLq6gAYbm9v5pZIJF2nP/Xjn3+GVatg2zZwc1Pf+h94AFxd2z+3Lcy+MGwsDCLQFs7OzkRERPDII4+wfPlyo8ztCyFILEnUpo32Z++nVbTiae/JkmFLWBa2jEVDF+Fq243/QxKJZMCzb58a/OPiwMNDnfO/7z5wdu7+tfuNCLi6uopp06aRnZ1NWloaTU1N7Z5jZWVFSEgIt9xyC3/6059w7uZftLy+nO1p24lJjmFb6jbK6suwUCyYOXimNkoY7jG8zywu/7ekBIDlPTXBKJGYgL7aj4WA3bvV4P/DD+oi72OPwT33gDFrDvUbETh/JGBjY4OXlxe+vr5YW1tTX19PRkYGlZWVtNdmRVHw9PRk9uzZPPfcc+1WCWqLVn0rP+f8TGxKLDHJMfxS9AsAQ9yGaOsIswfPxsbSpkvX7wkGUmqdpP/S1/qxELBrlxr89+4FX191g9fdd6uZP8am34hASEiIWLRoEQkJCWRkZFBWVqYZMhmwtrbGw8MDV1dXdDod5eXlFBYW0tra2u717ezsGD16NA888AC33HILFhadm/PPrswmNjmWmJQYvs/4noaWBhysHFg0dBHRodEsDV2Kn1PvspLuax8eieRy9JV+LARs364G/wMH1I1dK1fCXXeBnQkTEfuNCFxus1hZWRlxcXHs2bOH48ePk5GRQUlJySXiYGVlhYuLC/b29uj1ekpLS6mvr2/3nhYWFvj7+3Pttdfy7LPP4uZ2ib3RZalrruP7jO81UcipygFgot9EloUtY1nYMib4TUCnmLd4W1/58EgkV6K392MhIDZWDf6HD6uWDk88ofr72PTAREG/FoG2qKqqYufOnezZs4djx46RlpZGSUkJjY2NFxxnaWmJg4MDNjY21NfXU1tb26FUVFdXVyIjI3n22WeZNm3aFY8VQpBQmKBNG/2c8zMCgY+DD9Gh0USHRbNwyEKcbJw69GzGpLd/eCSSjtBb+7Fer6Z4rloFx46pZm5PPgm33w4X1Z0xKQNSBNqitraW77//nt27d3P06FFSU1MpLi6moaHhguMsLCywtbVFp9NRX19PS0tLu9e2sbFh2LBh3H333dx3331YWl5+m0VxbTHfpX5HbEos36V+R2VjJVY6K+YEz9EWl4e6GzfttS1664dHIukMva0f6/Xq5q7nn4eEBBg2DJ56Cm69VfX56WmkCHSAhoYGdu/ezY8//kh8fDwpKSkUFhZeMmWk0+mwsrJCr9dfMuV0OXQ6Hd7e3ixdupQXXnjhkvKSza3N/HT2J21PwpmSMwAM9xiuOaDOCJqBlYVpes7Zc+IXaGtrkutLJD1Bb+nHra2wcaNq73D6NAwfDk8/DTfdBG28E/YIUgS6QVNTE/v27eOHH37gyJEjJCcnU1BQQN25zSkGFEVBp9Oh1+vbzU4CcHBwYMKECTz11FNERUVpP08rSyM2RTW8+zHzR5pam3C2cSZqaBTLwpaxZNgSvBy8jP6cEomk67S0wBdfqMZuSUkwahQ88wxcfz10Mr/EJPQbEXBxcRGPP/44V199NeHh4WbNxW9paeHgwYPs3LmTw4cPk5ycTF5e3mVLximK0iFhsLS0JCgoiN/+9rc8/vjjNNHEzvSdmigU1BSgoDB10FQtBXWcz7hu/R02FBUBcKOpXagkEhNirn7c3AyffqoG/7Q0GDtWDf7XXKN6+/cW+o0IXGnHsJWVFa6urvj7+zN27FiWL1/OwoULce3OXusuoNfriY+PZ+fOnRw8eJAzZ86Ql5d3Qb3RzqAoCi4uLsybN49bH7qVX/S/EJMSw5E8dUQ0yHmQurgcGs38IfOxt+pcknFvm0uVSLpCT/fjpib4+GPVzz8zEyZMUIP/1Vf3ruBvYECIQEfQ6XTY2dnh6elJSEgIM2bMYMWKFYwbN67TewI6i16vJyEhgbi4OA4ePEhiYiI5OTlUV1d3aJRwMVbWVvgM9sF3iS9nfM9Q01SDraUtc4PnamsJg10Ht3sdKQKS/kBP9ePGRvjwQ7WM49mzMHkyPPccLF0KvdkkoF+JgCFrx9bWFmtray0Dp7GxkZqaGhobGzu0MexKGAqC+/v7M3r0aBYvXszVV1+Nuwns+4QQJCYmsmPHDn7++WdOnz7N2bNnqaqq6rxrqgI6Bx36kXqYD6ODRmvTRtMGTbtsNTUpApL+gKn7cX29Wrj95ZchNxciI9Xgv2hR7w7+BvqNCDg4OAhfX18qKyupra2lqanpioHS0tISKyurC8SiqamJhoYGWlpauvQGbkBRFGxtbXFzc2PIkCFMnTqVa6+9lsmTJ7eZGtpZkpOTiYuL48CBA5w6dYrs7GwqKio6Lw6WoAvUMf+e+dy55E6ihkXhbqcKmhQBSX/AVP24rg7efx/WrIGCApg5Uw3+8+b1jeBvoN+IwOWygxobG0lOTubMmTOkpKSQlZVFTk4OhYWFlJaWUlVVpQnGlZ7FwsICS0tLLYC3tLTQ0tLS7VGFhYUFjo6O+Pn5MXLkSKKiorjmmmvw8up6hk9WVhbbt29n//79nDx5kuzsbMrLyzvXVh24ertif+2vCPntH9k3eXKX2yORmBtji0BNDbz3Hrz6KhQVqUH/2Wdh9myjXL7H6dci0Bnq6+s5c+YMiYmJpKWlkZmZSV5eHgUFBZSVlVFVVUVdXR3Nzc3tCoZhDaG1tbXDaaFtoSgKVlZWuLm5MXjwYCZPnsy1117LzJkzOzWqyM3NJS4ujn379pGQkEBmZiZlZWWdEgdnZ2eioqJ4++238ZYZQ5I+Qsk5R2HPbm7DraqCd96B116D0lJ1uueZZ2DGDGO00nxIEegC1dXVJCYmkpSURGpqKtnZ2eTk5FBUVERZWRnV1dWaYFwJ3blUASFEt4TCcC07Ozt8fHwYOXIk8+bN44YbbmDQoCtXPCsqKiIuLo69e/eSkJCgWWh0dFrJysqK4cOHs3r1aqKjo7v1DBJJb6SiAt56C/7+dygvVxd6n3kG2nGE6bVUVFTw1VdfsWvXLk6fPs0vv/wiRcCUlJeXc/r0aZKSkkhLSyM7O5u8vDyKioooLy+nurqa+vr6Du0wNgaWlpY4OzszePBgJkyYwIoVK4iKirpkVFFWVsZTGzeSfPAg9clJnDx9kuqKjqeyenp6cvPNN/Paa69hZY698BLJOT7KzwfgDr/OufSWlcGbb6pflZVqiuczz8CkLoXLnqG5uZm9e/eyefNm4uPjtenghoaGK436pQj0FoqLizXBSE9P1wSjuLhYEwzDwrUpURQFGxsbWp2csAsO5rGrr+bWW28lJCSEyspK/vXtv/h629ecOHqCmuwa6KB+2djYMHHiRN577z3Gjh1r0meQSAx0dk2gpER963/rLaiuVjd3Pf009Ib8iIyMDDZu3MjevXtJSUmhqKiI2traDiezWFpaYm9vj4eHB8HBwQwdOpT169f3DxFQFEVYWVlhZWWFjY0Ntra22Nvb4+joiKOjIy4uLri6uuLm5oanpydeXl54e3vj6+uLv78/fn5+2NnZ9YmqX0IICgoKNMHIyMjQBKOkpITy8nJqamraU/9uY2FhgZ2dHU4eTuj8dRRZFdHc2AzpQCnQgRklRVHw8fHhkUce4dFHHzVZWyUDl46KQFGROt//zjtq5s/116vBf8yYnmgl1NXVsWXLFrZv384vv/xCTk4OlZWV7WY6GtDpdNjY2ODg4ICzs7MW7ywsLCgtLaW4uJiKigrq6uoufpHsHyJga2srvL29qa+vp7Gxkaampi5l8Oh0OiwsLC4REwcHh0vExMPDAy8vL3x8fPD19cXPzw8/Pz8cHBx6jZjo9XrOnj3L6dOnSUlJISMjg7Nnz5Kfn09JSQkVFRVG20PRJjrQWejQt+ihg13G2tqa6dOn8/nnn19irCeRdIb2RCA/H155BdauVTd83XST6uo5apTx2iCE4MiRI3z77bccOnSI9PR0rW5JR0b2Bv8xQ1q7ra0tlpaWtLS0UFtb290Xvt4rAoqiZALVQCvQcqWGXmk6qLm5mZKSEvLz88nPz6ewsJDi4mJKSkooLS2lvLycyspKqqurqampoa6ujrq6ugvEpLNZPoqiXLAX4XJi4uLigru7uyYm3t7e+Pj44OfnR0BAAI6Ojj0qJnq9noyMDE0wXjt0iMbCQkIbGjTBqK2tpbGxsfP7EYyAoih4eHjw2GOP8eijj2oL6xLJlWhLBHJy1Bz/detUk7dbb1WDf1hY5+9RXFzMV199xffff09iYiKFhYVUV1e3m01owBDkDZ93vV5v9M+Y4QXX1tYWJycnfHx8GDZsGF999VWvF4FJQoiS9o7tiTWBlpYWSktLLxCToqIiSkpKKCsr08SkqqqKmpoaamtrLxiZNDc3d0lMzh+Z2NjYtDkycXd3v0BM/P398ff3x9nZuUticqU3qJaWFlJTU0lMTCQlJYXMzExycnIoKCigtLSUyspKampq2t2DYQysrKwYO3Ysr7/+OjNmzJDiILmAi/txdjasXg0ffKB6+99+u1rJa9iwy5/f0tJCXFwcMTExHD16VNuY2dDQYJaXofOxsrLC1tYWZ2dnfHx8CA0NZcKECUyePJkJEybg4uLS7jV6dYpobxMBY6HX6ykrKyMvL4+8vLwLRiYGMamoqLhETBoaGi6Y5uqqmBhGJnZ2djg4OODk5ISTk9Ml01wuHh54+/gQHBDAoEGDcHFx6ZKYNDU1kZycTGJiIqmpqWRkZJCYnkja2TRKS0tpqmlSF5dN8Hny9vbm1ltv5emnnzaJtYek91N3bpqkMNuCl16Cjz5Sf/6736k1fJubU9iwYQM//fQTKSkpFBcXa/PmPT3lbWFhgb29Pc7Ozvj6+hIaGsqkSZOYOnUqERERODo6Gv2evV0EMoBy1Jnk94UQ69o6ti+JgLHQ6/WUl5dfMjIpLi6+YJqrLTFpbm7usphYWlpqIxODmDg6Ol6wIOXh4YGnp+cF01z+/v64u7tfICZFtUVsS9lGbEos2xK3UZNbg0WZBcH6YLyavbCutaaqrIrCwkLKy8tpbGw02ofTwcEBb29vJk+ezF133cX8+fNNbhgo6TlqampYu3Yna9e6k5Y2HWjFwuJftLb+DThr8vsbpl9cXFzw8/MjNDSUqVOnEhkZybhx47DtBQWbersI+Ash8hRF8QbigAeEEHvO+/3dwN0AQUFBE7Oyskzanv6KEILKykry8vI0QSkuLmZ7Rga15eX4NzVdMDKpqakxqphYW1trYmLvYA/WUKfUUSbKqNHVgD14eXoxMXQi88bMY96YeQwOHIy1tTVJSUnaLm+DLUhqaio5OTlGW+jW6XQ4OzsTHBzMwoULeeihhwgICDDKtSVdRwjBgQMH2LRpE4cOHSIlJYWysrLzpiDDgKeAW4Em4H3gFSCvy/c0vPwYgnpYWBjTp0/nqquuIjw8HOueLA5sJHq1CFxwM0X5P6BGCPHq5X4/EEcCpqaz+dVCCKqrq8nNzb1kzcSQutqemHR2CH5x1oQmJufSg52dnXFxccHOzk6zBbm4LKixsLS0xMXFhbFjx3Lbbbdxyy23YGNjY5J7DQSys7N588032bJlCzk5OZ0YAY5CDf43AfXAe8CrQOElRxrSKl1cXPD399eC+rx58xg+fLjRDB97M71WBBRFcQB0Qojqc/+OA1YJIb673PFSBIyPuVxEhRDU1tZqI5OCggKKiorIyc/hRMYJknOTyS3KpamuCRrBRthgrbeGFmhtbtVGJuZetGsLa2tr3N3dmTJlCo8//jjTp083d5N6jMrKSj788EO2bt1KQkICZWVlRtz8OAZ4GrgOqMPLayPTp//ML2GuuE6bxpEVK3pN6nZvojeLwBBg07lvLYHPhRAvtnW8FAHj05utpPVCT3xePDHJMcSmxBKfHw9AoHOgVjhnXsg89E168vPzNTNAw8iktLSUsrIyKioqtPTg2tpa6urqqK+vp6amxuQ7s6+EoihYW1vj5eVFZGQkDz/8MNOmTet1mU/19fVs2bKF//znPxw/fpzCwkItjdiYWFtb4+Pjw4QJE5g7dy7R0dEMHToURVE4dgyefx42bQInJ3jwQXj4YfD0VM/tzf24N9BrRaCzSBEwPn3pw5NXncfWlK3EpsQSlxZHbXMtdpZ2zAuZp4lCoEtgl6+vLjCu5b333iMrK8ukO7E7i8GSfPjw4cyZM4fIyEjGjx+Pv79/p32b6urq2L59O9u3byc+Pl4rYtTU1GT0Z9bpdDg4ODBixAjuvfdebrvttk5Nvxw+rAb///4XXFzUwP/QQ+DmduFxfakfmwMpApI26asfnsaWRnZn7SYmOYaY5BgyKjIAGOszVqumNjVgKha67mcBnTp1imeffZbvv/+eysrKTq1nGIKgYa+Hs7MzDg4OVFRUkJaWRlVVlVlHI11Fp9Ph6OjIoEGDCA8PZ86cOVx//fXdqpNxPgcOqMF/2zY14P/5z/DAA6oQXI6+2o97CikCkn6NEIIzJWe0aaN92ftoFa142HmwJHQJy0KXETUsCldbV6Pds7a2ljVr1vDvf/+bs2fPdjqQW1hY4O7uTnBwMGPHjuWqq65i0aJFBAQEUFVVRVxcHDt37uTHH38kPT2dpnN++T2JYUe8g4MDnp6e+Pn5aenBHh4eF+yC9/X1JSAgAF9f326lRO7dC6tWwc6d6lTPI4/AH/+oTgFJuo4UAcmAory+nB1pO4hJiWFrylbK6suwUCyYETRDmzYa4TnC6AuIQgj++9//smbNGo4dO0ZdXZ1Rr9+X0Ol0mqWKwZ/Lzs7uArNHNze3c7vgPaioiGDXrumcOuWNh0cLjzwieOABK0ywb2pAIkVA0iavZmcD8GhQkJlbYhpa9a0czD2ojRISChMAGOI2RJs2mj14NjaWHUvzNLi77tu3jz179mhOkGVlZVoZU3Nx/pu7t7c3YWFhzJw5kxtuuIHAwEAOHjzIp59+yp49e8jJyaG6utpo2VXn+2hZWFhc4o/T2trahtnjAuBZYCZqbv/LwD9R0z6vbPbo5OSkjUyyrKywd3Pj6qFD8fb21jYt+vv7Y29vb5Rn7MtIEZC0yUCbS82uzGZrylZikmPYlbGLhuYGbOtsCW8Mx7nAmfrceory1WJABk8oY38GDEHSWAZijo6OBAcHEx4ezrRp01i4cCGjRo3q0kinoqKCr7/+mi1btnDixAmKi4tpaGgw2t9ADejW2Nr+moaGx2hoGI+1dSEhIRsJCfkea2v171FdXX2B2aNhr0lzc3OX/LnOF5OLzR7PF5OLnYPP3wXf02aPxkSKgKRN+psI6PV60tLSOHToED/99BMnT54kNzeXsrIyzSvGmBkwhk1shsVfRVEuCVodCfSGdFFbW1taW1s7VVhIUZRLAqKiKDg5OTFo0CBGjhzJlClTWLhwIePGjet2CqoQgmPHjrFhwwZ2795NWloalZWVHayctwz1zX8ykAX8DfgIdbfvhZz/d/Xw8NDWHYKDgxk2bBhhYWF4enpSUlLCbfv20Vxayl329hQXF3fI7LGr/lyWlpYXWKpcSUwutlQZNGgQTk5OPS4mUgQkbdLbRaClpYWkpCQOHTrE4cOHOXXqFDk5OZSXl2s1no2+YUwHWAD2YOliqQYe72DsmuwoLfnqFwU+AAAgAElEQVRf4Y6uVnsaNmwY06ZN47rrrmPMmDHtBoS8vDxeeukltmzZQm5ubodEzGBZfPGxiqLg6OiIv78/I0aMYMqUKSxYsIBJkyYZfX9CbW3tuf0F/2XPHjcKC/+AXh+BWpHoReDfdLhk3RVQFAWsrLCwsyPo3IK1j48PgwYNIiQkhGHDhjFy5EhCQ0Mvm54qhLjA7LGgoMDsZo+GXfDni4mhpolBDDtj9ihFQNImPS0CjY2NnDp1imPHjhEfH8+pU6c4e/asVg3JFEFdURTtrdLDw0ObMw4MDCQkJISAgABycnKIj4/n5MmT5OXlUVlZ2eGpIIMtgaurK4MGDSIiIoKlS5eyZMkSk1lKNDc38/HHH7N27VpOnz7dIZsMw9/BcP7Fz+bg4ICfnx/Dhw9n0qRJLFiwgMjIyC6b7en18M03aqrnL7+oNs5PPw233AKGrQ1CCJKSkti4cSO7d+8mMTHR6AaC53N+ZS4XFxc8PT3x9fUlMDCQ4OBgQkNDGTVqFEOGDOm0KAohKC8v13bBn2+pYti42BNmj/b29tjb21/gHPz1119LEZBcniUJ6kLpti7WA66vrychIYFjx45x4sSJC97U6+vrTfOmzv9qJDs6OuLu7q6lMBre/sLCwhg5ciSBgYHEx8dfUO2prKysU9WerKyssLGzwcrZiibXJmr8aiAchg8ZTnRoNMvCljEjaAZWFp3btGUKDh8+zOrVq9mzZw+lpaUdCiiGRdeWlpbLBl97e3t8fX0JCwtj4sSJzJ07l1mzZrW5Sa21FTZuhBdegNOnYcQINfjfeCN01aanvr6eHTt28J///Efz+6+urjbpHgsLCwtNMFxdXfHy8tIEw9DHRo0aRWBgoFFHURebPRYWFlJYWHhJgSxDPQ/DLvh2xESKgKRj1NTUcPz4cY4dO8bJkyc5ffo0OTk5VFRUmDSow//e0gxvL+cH9uDgYO1DFxQUpH3oioqKtLnpxMRECgoKqKmp6XC1J0MNZXd3dwYPHszkyZP51a9+dcXCNenl6cQmxxKTEsOPmT/S1NqEs40zUUOjWBa2jCXDluDlYJxNU8agpKSEN954gw0bNpCVldWhuXuD572VlRVNTU3U1dVd8v/d1tYWX19fhg0bxsSJE5k5cy6FhXN5+WVrkpMhPByeeQauuw56yrk7NTWVb7/9lh9//JEzZ85QWFjYI4VhDHbSjo6OmmD4+fkRFBRESEgIw4cPZ+TIkfj5+fWYLcj5Zo+jRo2SIjAQEUJQUVGhvaWfPn2axMRE8vLyTP6mbkCn013w4bg4sIeGhhIeHn5BYDfQ0tLC9u3biY2N5dixY52u9mRYtHV2dsbf35/Ro0ezYMECVqxY0aFqTB2hpqmGnek7iU2OJTYllvyafBQUpg6aqo0SxvmM63VZJS0tLXz77be89dZbnDhxgurq6g6dZyhwDurbeXV1Na2tCvAbVFfPYShKAu7u7zBmTCoTJkQwZ84c5s2bh4ODg8mepzM0NDSwZ88eNm/ezJEjR8jKyqKioqLH0nstLS21z4RhFOvv709QUBBDhw4lNDSU0aNHG233Ncg1gX6DEIKSkhKOHj3KiRMnOHPmDMnJyeTm5mpz6oZayabk/GGym5ubtmgVFBSkBfaRI0cSHBx8xbee06dP880331xQ7ckwTdNevzNkadjb2+Pp6cmwYcO46qqruO666xgxwvgbwTqCXug5XnBcs7I4nHcYgACnAE0Q5oXMw8G6dwTDy3Hy5EleeeUV4uLiKCgoaOf/gxXwW+BJIAQbm5O4uPyD1tbNVFaWXzJVYzDLGzJkCOPHj2fWrFksXLhQE5Xu8HxmJgDPBAd3+1oGzp49y5YtW4iLi9NGmbW1td3OLju/b7bXzw2lJZ2cnHBzc8Pb2xt/f38GDx7MkCFDGD58OOHh4bhdbKZ06T2lCPRG9Ho9BQUFxMfH88svv3DmzBnS0tK0oG4IiD1hl3y5+U8fHx9t/rOjgd1AZWUlmzdvJi4ujpMnT5Kfn6+ZlHXkeQztcXV1JSgoiHHjxrF8+XIWLFjQZ/z7C2oKtGpq29O2U9NUg42FDfNC5hEdGk10WDTBrsHmbma7VFVVsXbtWj755BPS0tJoaBDA74CVQBBwEFgFbNXOURQFNzc3fH19sbGxoampidLSUkpLSy+ZjrKyssLT05OQkBAiIiKYOXMmixYt6lSpUHNluTU3N7N//37+85//cPDgQTIyMqioqDDKwrZOp9M+a0KIdsXHysoKOzs7nJ2dtVrkhpTaVatWSRHoCVpbW8nNzeXIkSMkJCSQmppKeno6ubm5VFZWatMvPfU3vTiwn58JYQjsf9XrsfX3Z8/EiR2+bmtrq9bxjxw5QmZmprbY2pG3JMNiq6OjIz4+PowaNUozIPPx8enOI/damlqb2JO1R1tLSC1LBSDcK5xlYctYFraMaYOmYanrvQVO6uvhn/+El1+GvDwYObIMO7tXSE9/n4qK8g5dw9bWlkGDBuHj44O1tTW1tbXk5uZSXFx8yXSMpaUlHh4emr/SjBkziIqKumwf6e2pznl5eWzbto0dO3ZoGWjGsDI/P9UU1BfLNvbCSBHoCk1NTWRlZWmpjKmpqWRmZpKXl3fBm3pP/o2uFNjPT3Hr6Bv75T48Z8+e5ZtvvtEWW4uKiqitre2wgBnmPD08PAgJCWHKlCmsWLGCKVOm9DqvfHORXJqsWVnsydpDi74FN1s3loQuITo0msXDFuNu1/E3YVNSWwvvvw9r1kBhIcyaBc89B3PnwsWzbqmpqbz++uvExsaSn5/foUVoRVHw9PQkNDQULy8vLC0tKSkpIT09naKiokvqFhjM9wYPHsyYMWO46qqrWB8QgI2PT68VgY7Q3NzM4cOHiYmJ4aeffiI9PZ3S0lLq6+uNEWOkCIC6IJSenk58fDynT58mPT2drKwsLagbdmka45k7M+/XkcBumIrpboH0hoYGtm3bxrZt2zhx4gTH0tJoqalB6eACsU6nw9raGmdnZwICAhgzZgxRUVEsX74cJ2n12CUqGyqJS48jJlk1vCuuK0an6Lgq8CptLWGUV9dsILpDTQ28+y68+ioUF8P8+Wq2z+zZnbtObW0tH3/8MR988AFJSUnU1tZ26DxbW1sGDx5MeHg4vr6+6PV6UlJSSE1N1bJ+LkCnw8PNjaCgIMaMGcP06dOJiooi2IjrBL2B4uJidu7cydatWzlx4gT5+fkd2bHdP0WgtraW1NRULahnZGSQk5Oj/VGM+aZuqHMLalA3fLVFW4H9/F2Mhjf27gZ2A0IIjh8/zubNmzlw4ACpqamUlpZqC8YdeUbDYqvBgGz69OnccMMNWoUniWnRCz2Hcw9ro4RjBepIbbDLYG3aaE7wHGwtu27X3B5VVfD22/D661BaClFRavC/6irj3UMIwe7du3nzzTfZu3cv5eXlHX4J8fDwICwsjAkTJuDn50dlZSX/2rePmqwsOGcPcvE5Li4uBAYGEh4eTmRkJIsWLWL48OHGe6BeRktLCydOnCA2Npa9e/eyc+fO/iECOp1OQPtv1h28lra13mDm1d5uPUO6oyGwG/xMLjcVY6zAfj6lpaVs2rSJXbt2cerUKfLz86murqapqanDOfG2tra4ubkRGBjIhAkTuPrqq5k9e3afWWwdaORU5WjV1Ham76SuuQ57K3sWDFmgLi6HRhPgHGCUe1VUwD/+AX//u/rv6Gg1+E+dapTLd4js7GzefvttNm3aRHZ2dofTNm1tbbURwLRp0wgICCApKYnDhw+TnJxMfn7+JSMQRVFwcXHR/JWmTZvGokWLCA8P73cvPP0mO0hRlMs2xvCWbmFhcYFDY2tra7vFyA0blAx57BcHdsMbe0hIiEkCu4GWlhZ2797Nli1biI+PJysri/LychoaGjq12Ork5ISvry/h4eHMnTuXa665Bm9vb5O1W9JzNLQ08GPmj1oKalZlFgARvhGaLfZk/8mdrqZWVgZvvAFvvqmOAn71KzX4dyJXwKQ0NjbyxRdfsH79ehISEqipqemwnYe7uzuhoaFMmjSJefPm4e7uzt69ezl06BBnzpzRFmjPx2C+FxAQwIgRI5g6dSoLFy4kIiKiz65p9RsRsLCwEJaWlu2mTbYV2C82lBoyZIhJA/v5pKWl8c0337Bv3z6SkpK0xdbOGJDZ2dnh4eHB0KFDmTp1Ktdccw3jx4/vVsd8Ij0dgJeGDOnyNSQ9jxCC08WntWmj/Wf3oxd6vOy9WBq6lOjQaBYNXYSLbdub4kpK1Cmft95S5/+vvVa1d4iI6MEH6SJCCA4cOMC7777Lrl27KCwqQnQwldqQoRQeHs6MGTOIjo6mvr6enTt38vPPP3PmzBlyc3Oprq6+5LPp5OSEn5+fZr43f/58Jk+e3GNxpKv0GxHQ6XTC09PzAuMncwZ2A7W1tcTGxrJt2zYSEhK0lNCO5sQbFlsNQ9Nx48axePFilixZgqOJSyv19tQ6Sccoqy9je+p2YlJi2JayjfKGcix1lswMmqmtJYR5hAFqhs9rr6mLvnV1cMMNavAfPdrMD9ENDP34Sz8/1q5dy1dffUV6evqli8dtoCgK7u7uDB06lEmTJjF//nwWLVpEZmYmO3bs4Oeff9YsVKqqqi5rvufr68uIESO08yMjIy/rWmoO+o0ImGOfQGtrK0ePHmXTpk0cPHjwgrStzhiQOTg44OXlxfDhw5k1axbXXHMNISEhZp97lCLQ/2jRt/Bzzs/aKOFk0UkAgi0icTv6N05vm0lzk46bb1Z46ikYOdLMDTYCV+rHTU1NbNq0iX/+85/Ex8dTWVnZ4XVFW1tb/P39GTVqFNOnT2f58uWEh4eTnJysicOpU6fIzs6msrLykpc+e3t7fHx8NPO9efPmMXPmTKytrbv/0J1AisAVKCgoYNOmTfzwww+cOnWKwsJCampqurTYGhwczMSJE1m+fPkVXRZ7E1IE+j8/n8ph5V/L2Lt5BPpWHYz5FIf5/yBqagjLQpexNHQpPo59e5NeZ/uxoTDO2rVr+e6778jPz+9UER83NzdCQkK0wL5kyRKcnZ1JT09nx44dWkEjg9/Vxet6tra2+Pj4aOZ7c+bMYc6cOdjZ2XXuwTvIgBWB5uZmdu7cSUxMDMeOHdOMohobGzu82GrIiff19dUMyK6++mo8PT278yi9BikC/ZesLFi9Gj78UPX2/+1v4aFH6sjU7dJGCbnVuQBM9p/MsrBlRIdGM95vPDqlby2AGqsfl5WVsX79er788kuSkpI6tVHLxsYGPz8/Ro4cSWRkJMuWLdMquZ09e5YdO3awf/9+fvnlF7KysigrK7skDtnY2ODt7c3QoUOZMGECs2fPZt68ed2eFu7VIqAoymLgTdRaTuuFEKvbOvZ8ERBCcObMGTZt2sS+ffs0AzJDTnxnqj0ZDMimTZvGihUrGDNmTK9f6DEWvzl9GoBPR40yc0skxiI9HV56CT76SN3R+7vfwcqVcPGeKSEEJwpPaFYWB3MOIhD4OfqxNHQpy8KWsWDIAhytTbsuZQxM2Y9bW1v573//ywcffMCBAwc6vKcB/peGGhISwvjx45k3bx5Lly7VDN8KCgqIi4tj7969JCQkkJmZSWlp6RXN9yIiIpg1axYLFizA1dW1o+3onSKgKIoFkAwsBHKAw8DNQojTlztep9MJQ/pnexhy+g2LrREREURHRzN//nyTL7ZKJOYgJQX+9jf497/V4i2//z08/jgEBnbs/OLaYralqoZ336V+R1VjFdYW1swJnqOloA5xk1lkBk6dOsW6deuIiYnh7NmzHayxrGJtba0tJEdGRrJ06dILSnyWlpZq4nD8+HEyMjIoKSm5rPmeh4cHQ4YMYdy4ccycOZOFCxdeMlPRm0UgEvg/IUTUue+fABBCvNTG8eLcfzUDMm9vb0aMGMHs2bO55pprCAwMNPtiq0TSk5w5Ay++CJ9/DtbWcM898Nhj4O/f9Ws2tzazL3sfsSmxxCTHkFSaBMBIz5GalcX0wOm9oppab6KqqopPPvmEzz77jJMnT1JbW9vh6SRFUXB2diY4OJhx48Yxd+5coqOjL6grUFlZyc6dO9mzZw/Hjh0jPT29TfM9d3d3zXxv/fr1vVYErgMWCyF+f+7724CpQoj7L3f8xIkTRXx8vMnaMxB5OCUFgDdCQ83cEklnOXVKLeG4YQPY2cF998Ejj4Cvr/HvlVqWqk0b7c7cTbO+GVdbV62a2uJhi/G0N986WW/ux3q9nri4OD788EN2795NSUlJp2oSWFlZaRlGkZGRLF68mOnTp1+wP6i2tpZdu3axe/dujh07RmpqKsXFxeenyPZaEbgeiLpIBKYIIR4475i7gbsBgoKCJmZlZZmsPQMRuTDc9zhxQg3+X38Njo5w//3w5z+DEQtRXZHqxmri0uO0amqFtYXoFB3TBk3Tpo3GeI/p0RF5X+zHaWlprF+/ns2bN5ORkXGJW2p7ODs7a7U2Zs+ezdVXX32JzXZDQwM//PADS5cu7bUi0KnpoN5eT6Av0hc/PAOVo0fh+edh82ZwdoYHH4SHHwYPD/O1SS/0xOfFa9NG8fnqSD3QOfCCamp2VqZJfTTQX/pxXV0dn3/+OZ999hlHjx697K7lK2FlZYWXlxdhYWFMnTqVqKgoZs6ciZWVVa8VAUvUheH5QC7qwvAtQohTlzteioDx6S8fnv7MoUNq8I+JAVdXNfA/+CC0U1HQLORX52uGdzvSdlDbXIutpS3zQ+Zr1dSCXIKMft/+3I/1ej379+9n/fr1qkVGYWFXitF0WQRMuudZCNGiKMr9wHbUFNEP2xIAiWSgceAArFoF330H7u7qFND994NL23ZAZsfPyY+7JtzFXRPuorGlkT1Ze1TDuxR1XwJbYazPWG2UMDVgaqcN7wYaOp2OmTNnMnPmzAt+npOTwwcffMCmTZtITk6mvr7eJPfv05vFJO1zd5Ka9bGuH3ur9zX27FHf/HfuBE9PePRRddG3L9fsEUKQVJqkbVLbm7WXVtGKh52HVk0tamgUbnZdG97IfqxSX1/P119/zWeffcbhw4cpLy83TCf1zumgziJFQNJfEQJ+/BH++lfYvRt8fNQ0z3vuAQcHc7fO+FQ0VLA9dTuxKbFsTdlKaX0pFooFM4JmaKOEEZ4jZLq3ERBCoNPppAhIJL0RIdQ3/lWrYN8+8PNTN3j94Q9gb2/u1vUMrfpWDuUe0kYJJwpPABDiGqI5oM4ePBsbS1n4qKv02s1inUWKgPGRw2jzIARs26YG/4MHYdAg1drhrrvA1nSVI/sEZyvPEpuipp/uSt9FfUs9DlYOLBy6kOjQaJaGLsXf6cKdcLIfX5nuiEDvMMOWmIzki+qxSkyLEPDf/6rBPz4eBg+G999Xzd1khU+VQJdA7pl0D/dMuof65np+yPxBGyVsPrMZgAl+E7Q9CZP8J8l+bEKkCEgkRkCvV/P7n38ejh+HIUPggw/gttugDziOmw07KzuWhi5laehShBCcLDqp7Ul4Ye8LrNqzCh8HHxT3qbj7zKRq1FCcbZzN3ex+hZwO6uf05/zq3kBrK3zzjRr8T56E0FC1itctt6gmb5KuU1pXynep3xGTEsPXSbG0NFdjpbNi1uBZ2lrCMPdh5m5mr0CuCUjaRIqAaWhtVT19XngBEhNhxAi1ePuNN8IAcSnvUWbHH6ay/ASLRDKxKbGcLlaNiMM8wrRpoxlBM7C26NmKXr0FuSYgaZMIaattVFpaVDfPF1+E5GS1bu+GDWoRdxn8Tcd4Z1dwns2a0N+zZuEaMsoztGmjtw+/zes/v46zjTNRQ6OIDo1mSegSvB28zd3sPoEcCUgkHaC5WfXxf/FFtajLuHHw7LPw61+Drm8V6ep31DTVsCt9lyYK+TX5KChMCZiiVVOL8I3o13sS5HSQRGIiGhvh44/VYi5ZWTBxohr8ly9Xq3pJehdCCI4VHNNssQ/nHkYgCHAK0LyN5ofMx8G6f+3QkyIgaRNZXrJrNDSotXtXr4azZ2HqVDX4L1kig7856Go/LqwpZFvqNmKSY9iRtoPqpmpsLGyYGzJXW0sIdg02QYt7FrkmIGmTnE56mA906uth3TpYswby8uCqq2D9eli4UAZ/c9LVfuzj6MMdEXdwR8QdNLU2sTdrrzZtdP+2+7l/2/2Ee4VrVhaRgZFY6gZWWBxYTyuRtEFtLaxdC6+8AoWFMHs2fPopzJkjg39/wdrCmvlD5jN/yHxej3qd5NJkbdro9Z9fZ81Pa3CzdWPxsMVaNTV3O3dzN9vkSBGQDGiqq+Hdd+HVV6GkBObPV7N9Zs82d8skpibMI4ywyDD+FPknKhsq1WpqKbHEJsfyxckv0Ck6pgdO16aNwr3C++XishQByYCkshLefhtefx3KymDxYjXPf/p0c7dMYg5cbF24btR1XDfqOvRCz5G8I2qdhOQYVu5aycpdKxnsMlibNpobMhdby/5hAiVFoJ8T2ZsrlJiB8nL4xz/gjTegogKWLVOD/5Qp5m6Z5Er0ZD/WKTqmBExhSsAUVs1dRW5VrlZN7aMTH/HukXext7Jnfsh8LQU1wDmgx9pnbGR2kGRAUFqqBv5//AOqqtT8/qefVlM+JZKO0tDSwO7M3Vo1tcyKTAAifCO0UcJk/8k9Xk1NpohKJG1QXKxO+bz9NtTUwHXXqcF/3Dhzt0zS1xFCkFiSqDmg7s/eT6toxcveiyWhS1gWuoxFQxfhYmv6UYwUAUmbXHvyJADfjB5t5pb0LIWF6mLvu++qaZ833ghPPaXaPEj6Hn2hH5fVl2nV1LalbqOsvgxLnSUzg2Zqo4QwjzCTLC7LfQKSNiltbjZ3E3qUvDw1zXPtWmhqUt08n3pKNXiT9F36Qj92t3Pn5jE3c/OYm2nVt/Jzzs/aKOHRuEd5NO5RhroN1RxQZw2e1SsM76QISPoFOTnw8svwz3+qJm+33QZPPqlaO0skPY2FzoKrgq7iqqCreGnBS2RVZLE1ZSsxKTG8H/8+bx58E0drRxYNXaRVU/N19DVLW6UISPo0WVmqtcOHH6qFXe64A554Qi3qIpH0Fga7Dubeyfdy7+R7qWuu4/uM77VRwreJ3wIw2X+yNm003m88OqVnnAmlCEj6JOnpqqnbxx+rO3rvukut4Tt4sLlbJpFcGXsre21KSAhBQmGCJgh/3f1X/m/3/+Hr6KsJwoIhC3C0Np0lvBSBfs58NzdzN8GopKSods6ffqpW7rrnHvjLXyAw0Nwtk5iS/taPDSiKwjjfcYzzHcdTs56iuLb4f9XUTn/NB8c+wNrCmjnBc1QX1NBohroPNW4bTJUdpCjK/wF/AIrP/ehJIcTWK50js4MkbZGYqAb/L75QC7b/v/8Hjz0G/v7mbplEYhqaW5vZf3a/5m90puQMACM8R2hWFlcFXoWVhVXvTBE9JwI1QohXO3qOFAHJxZw8qZZw3LgR7Ozgj3+ERx4BHx9zt0wi6VnSytI0B9TdWbtpam3CxcaFxcMWs+H6DTJFVHJ5liQkALBt7Fgzt6RznDihFm//5htwdFTn+//0J/DyMnfLJOagr/ZjYzLUfSgPTn2QB6c+SHVjNTvTd6qGdymx3bquqUXgfkVRbgeOAI8IIcpNfD/JRdS3tpq7CZ0iPl4N/v/5Dzg7q74+Dz0EHh7mbpnEnPS1fmxqnGycWDFyBStGrkAv9Fg82nWbim7lICmKslNRlJOX+foV8B4wFIgA8oHX2rjG3YqiHFEU5UhxcfHlDpEMAA4eVM3cJk2C3bvhr39V0z9XrZICIJFcie6mknZrJCCEWNCR4xRF+ScQ08Y11gHrQF0T6E57JH2Pn35SA/327eDuri7+3n+/OgqQSCSmx2S7ERRF8Tvv2xXASVPdS9L32LMHFixQyzceParu9s3MVHf5SgGQSHoOU64JrFEUJQIQQCbw/0x4L0kbLOtFcylCwA8/qG/+u3erGT6vvaamezo4mLt1kt5Mb+rH/Q3pIioxOUJAXJwa/PfvV3P7H38c/vAHNe1TIpF0j+7sE+gZcwrJgEQI2LoVIiMhKkpd6H3nHUhLgwcflAIgkfQGpAj0c+YcO8acY8d69J5CqCmekydDdDQUFMD770NqKtx3H9j2j9Kskh7EHP14oCBFQGI09Hp1c9f48Wr5xooK1d0zJQXuvlu1e5BIJL0LKQKSbtPaChs2wNixavnGujrV3fPMGbjzTrCyMncLJRJJW0gRkHSZlhb47DO1ZONNN6kjgc8+U83ebr9ddfmUSCS9GykCkk7T0qK+6Y8aBb/5jRrsN25Uzd5uuQUsur6DXSKR9DDyXa2fc4O3t9Gu1dQE//63WswlPR0iIuDbb+FXvwKdfJ2QmBBj9mPJhUgR6OfcFxDQ7Ws0NsJHH8FLL6lpnpMmwRtvqF4/itL9Nkok7WGMfiy5PPL9rZ9T19pKXRcdGBsa1Lz+YcPUCl6+vmre/6FDsHy5FABJz9Gdfiy5MnIk0M9Zes6H/cfx4zt8Tl0d/POfqp9Pfr7q7/Phh6rXjwz8EnPQlX4s6RhSBCQatbWwdi288goUFsKcOWq2z5w5MvhLJP0VKQISqqvVaZ/XXoOSEvWNf+NGmDXL3C2TSCSmRorAAKayEt56C/7+dygrg8WL1Upe06ebu2USiaSnkCIwACkvhzffVL8qKtRF3qefhilTzN0yiUTS00gR6Ofc4eur/bu0VH3rf+stqKqCFSvU4D9hghkbKJF0gPP7scS4SBHo59zh50dxMaxcqc7719aq/j5PP616/UgkfYE7/PzaP0jSJaQI9GMKCuD5l1v5aJ2O+nqFm26Cp56C8HBzt0wi6RwlTU0AeFpbm7kl/Q8pAv2QvDxYs0b18G9o0uGzpJz4V90ZMcLcLZNIusZ1p04Bcp+AKZAi0I84e1bd4LV+vWrydvvtcOpXp7ELamLECHdzN08ikfRCpG1EPyAzU2czTncAAA2TSURBVLV1GDpUffu//XZITlZ3+doFNZm7eRKJpBcjRwJ9mLQ01dTt449VF8/f/14t4D54sLlbJpFI+gpSBPogycnw4ouqpYOlJdx7L/zlLzBokLlbJpFI+hpSBPoQiYlq8P/iC7Ve74MPwmOPwZWy5+6VFrySfoDsx6ZDikAf4Jdf4IUX4KuvwN4eHnlE/fLxaf/cG2UxDkk/QPZj09GthWFFUa5XFOWUoih6RVEmXfS7JxRFSVUUJUlRlKjuNXNgcvw4XHutuqlr2zZ44gl1EXjNmo4JAMDZhgbONjSYtJ0SiamR/dh0dHckcBK4Bnj//B8qijIKuAkIB/yBnYqihAkhZFWIDnDkCDz/PGzZAi4u8Oyz8NBD4N6FLM/bEhMBmV8t6dvIfmw6uiUCQohEAOVSs/lfAV8KIRqBDEVRUoEpwIHu3K+/c/AgrFqlVu9yc1P//cAD4Opq7pZJJJL+iqnWBAKAn8/7PufczySXYf9+NeDv2AEeHmoh9z/+EZydzd0yiUTS32lXBBRF2QlczsLvKSHEf9o67TI/E21c/27gboCgoKD2mtOv2L1bDf7ffw9eXupc/733gqOjuVsmkUgGCu2KgBBiQReumwMEnvf9ICCvjeuvA9YBTJo06bJC0Z8QQg36q1bBnj1q8fbXX4e77wYHB3O3TiKRDDRMNR20BfhcUZTXUReGQ4FDJrpXn0AIdbpn1Sr46Sfw94d//EPd5WtnZ7r7PhIY2P5BEkkvR/Zj09EtEVAUZQXwFuAFxCqKclwIESWEOKUoykbgNNAC/HGgZgYJoS70rloFhw5BYCC8+y7ceSfY2pr+/ss9PU1/E4nExMh+bDq6mx20CdjUxu9eBF7szvX7MkKoKZ6rVsHRoxAcDOvWwW9/Cz1piZ5UVwfAcHv7nrupRGJkZD82HXLHsJHR6+Hbb9UdvidOqM6eH34Iv/kNWFn1fHv+X1ISIPOrJX0b2Y9Nh7SSNhKtrfDll+ru3uuvh/p6+OQTOHNGnfoxhwBIJBJJe0gR6CYtLfDpp2rJxptvVkcCn38Op0/DbbepLp8SiUTSW5Ei0EWam+Gjj2DkSDXYW1vDxo1w8qQqBhYW5m6hRCKRtI98T+0kTU3qNM/f/gYZGTB+PGzaBFdfrRZ2kUgkkr6EFIEO0tgI//qXWskrOxsmTVLz/KOj4VLrpN7D07LMmKQfIPux6ZAi0A4NDWrh9tWrITcXpk1T6/hGRfXu4G9gQVesRyWSXobsx6ZDikAb1NWpef1r1kB+PsyYoa4BzJ/fN4K/gePV1QBEODmZuSUSSdeR/dh0SBG4iJoaWLsWXnkFiopg7lw122f27L4V/A08nJoKyPxqSd9G9mPTIUXgHNXV8M478NprUFICCxfCM8/AzJnmbplEIpGYjgEvApWV8NZb8Pe/Q1kZLFmiBv/ISHO3TCKRSEzPgBWB8nJ44w14801VCJYvV4P/5MnmbplEIpH0HANOBEpL1bf+f/xDnQJasUIN/nKqUSKRDEQGjAgUFanz/e+8o2b+XH89PP00jBlj7paZlr8NGWLuJkgk3Ub2Y9PR70WgoEDN9HnvPXXD1003wVNPwahR5m5ZzzDdxcXcTZBIuo3sx6aj34pAbq6a479unWr18JvfwJNPwvDh5m5Zz/JTZSUgP0SSvo3sx6aj34lAdja8/LK6y1evh9tvhyeegGHDzN0y8/Bkejog86slfRvZj01HvxGBzEzV1+df/1K/v/NOWLkSQkLM2iyJRCLp1fR5EUhLUx09P/lEdfH8wx/g8cchKMjcLZNIJJLeT58VgaQkNfh/9plateu+++Avf4GAAHO3TCKRSPoOfU4ETp+GF19USzna2MBDD8Gjj/L/27vfGCuuMo7j31+pQAh/w9qUWBAaoeFPiVZC2jdVQ6MNUYhNVUwaW20kFOUFEqMNSW3AvrFpNEZrwdigjVr+NBTQEixarTFuBUNKKRUC2BYQslIUX7Si4OOLmXY3ZHfv7M7OzN6Z3ychmd2Ze+6Th3Pvs3PmzBmmTKk6MjOz9tM2ReCll5KHt2/dCmPGJF/8a9bANddUHdnw9p2mXhG3WnE/Ls6wLwIHDsD69cnTu8aNS2b6rF4NHR1VR9YevPSu1YH7cXGGbRHYty/58t+1CyZMgAceSIZ+/GyJgdl7/jzgh3JYe3M/Lk6uIiDpU8CDwGxgYUTsT38/HXgFOJIe2hkRK7K02dkJ69bB7t0waVKyvWoVTJyYJ9Lm+uZrrwH+8Fh7cz8uTt4zgUPAHcCGXvYdj4j3D6Sxo0eTJZwnT07m/K9cCePH54zQzMz6lKsIRMQrABqiR2699Vayzs+KFTB27JA0aWZm/biqwLZnSDog6XeSMj2f68Ybk1k/LgBmZuVoeSYgaS9wbS+71kbEjj5edgaYFhFvSPog8LSkuRHxr17aXw4sB5jm23zNzErVsghExG0DbTQiLgIX0+0/SzoOzAL293LsRmAjwIIFC2Kg72X929C0ZVOtltyPi1PIFFFJ7wbOR8RlSdcDM4ETRbyX9e+GMWOqDsEsN/fj4uS6JiDpk5JOAbcAv5S0J911K3BQ0ovANmBFRJzPF6oNxq5z59h17lzVYZjl4n5cnLyzg7YD23v5/VPAU3natqHxyMmTAHzCt1hbG3M/Lk6Rs4PMzGyYcxEwM2swFwEzswZzETAza7Bhu4qoDY0nZs+uOgSz3NyPi+MiUHNTR4+uOgSz3NyPi+PhoJrb3NXF5q6uqsMwy8X9uDg+E6i5H5w+DcBn/BxOa2Pux8XxmYCZWYO5CJiZNZiLgJlZg7kImJk1mC8M19y2uXOrDsEsN/fj4rgI1FzHyJFVh2CWm/txcTwcVHObzpxh05kzVYdhlov7cXFcBGpu09mzbDp7tuowzHJxPy6Oi4CZWYO5CJiZNZiLgJlZg7kImJk1mKeI1twz8+dXHYJZbu7HxXERqLkxI0ZUHYJZbu7HxfFwUM09evo0j6bL8Jq1K/fj4rgI1NyWri62+GEc1ubcj4uTqwhIeljSXyQdlLRd0sQe++6XdEzSEUkfyx+qmZkNtbxnAs8C8yJiPnAUuB9A0hxgGTAXuB14VJIH9czMhplcRSAifhURl9IfO4Hr0u2lwJMRcTEi/gocAxbmeS8zMxt6Q3lN4AvA7nT7PcDJHvtOpb8zM7NhpOUUUUl7gWt72bU2Inakx6wFLgE/fftlvRwffbS/HFie/nhR0qFWMTVEB3BuqBrr7T+kjQxpLtpco3NxRT9udC6ucMNgX9iyCETEbf3tl3Q38HFgUUS8/UV/Cpja47DrgL/10f5GYGPa1v6IWJAh7tpzLro5F92ci27ORTdJ+wf72ryzg24HvgYsiYg3e+zaCSyTNErSDGAm8Kc872VmZkMv7x3D3wNGAc9KAuiMiBUR8bKkLcBhkmGiL0XE5ZzvZWZmQyxXEYiI9/Wz7yHgoQE2uTFPPDXjXHRzLro5F92ci26DzoW6h/HNzKxpvGyEmVmDVVIEJN2eLidxTNLXe9k/StLmdP8LkqaXH2U5MuTiK5IOp0tz/FrSe6uIswytctHjuDslhaTazgzJkgtJn077xsuSflZ2jGXJ8BmZJuk5SQfSz8niKuIsmqTHJXX1NY1eie+meToo6aZMDUdEqf+AEcBx4HpgJPAiMOeKY1YCj6Xby4DNZcc5jHLxEWBMun1fk3ORHjcOeJ7kDvUFVcddYb+YCRwAJqU/X1N13BXmYiNwX7o9B3i16rgLysWtwE3AoT72Lya5YVfAzcALWdqt4kxgIXAsIk5ExH+AJ0mWmehpKfDjdHsbsEjp9KOaaZmLiHguuqff9lyao26y9AuA9cC3gH+XGVzJsuTii8D3I+IfABFR1yU2s+QigPHp9gT6uCep3UXE88D5fg5ZCvwkEp3ARElTWrVbRRHIsqTEO8dEsjbRBWByKdGVa6DLa9xL99IcddMyF5I+AEyNiF+UGVgFsvSLWcAsSX+Q1Jnes1NHWXLxIHCXpFPAM8CqckIbdga1XE8VTxbLsqRE5mUn2txAlte4C1gAfKjQiKrTby4kXQV8G7inrIAqlKVfXE0yJPRhkrPD30uaFxH/LDi2smXJxWeBTRHxiKRbgCfSXPyv+PCGlUF9b1ZxJpBlSYl3jpF0NckpXn+nQe0q0/Iakm4D1pLcmX2xpNjK1ioX44B5wG8lvUoy5rmzpheHs35GdkTEfyNZqfcISVGomyy5uBfYAhARfwRGk6wr1DSZl+vpqYoisA+YKWmGpJEkF353XnHMTuDudPtO4DeRXvmomZa5SIdANpAUgLqO+0KLXETEhYjoiIjpETGd5PrIkogY9Jopw1iWz8jTJJMGkNRBMjx0otQoy5ElF68DiwAkzSYpAn8vNcrhYSfwuXSW0M3AhYg40+pFpQ8HRcQlSV8G9pBc+X88kmUm1gH7I2In8COSU7pjJGcAy8qOswwZc/EwMBbYml4bfz0illQWdEEy5qIRMuZiD/BRSYeBy8BXI+KN6qIuRsZcrAF+KGk1yfDHPXX8o1HSz0mG/zrS6x/fAN4FEBGPkVwPWUzy/JY3gc9nareGuTIzs4x8x7CZWYO5CJiZNZiLgJlZg7kImJk1mIuAmVmDuQiYmTWYi4CZWYO5CJiZNdj/AfYEjbWN5IUkAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
\n", + "Here's a brief tutorial about how to use it\n", + "
\n", + "Let us use it to solve `Case 2` above\n", + "1. Run `python gui/grid_mdp.py` from the master directory.\n", + "2. Enter the dimensions of the grid (3 x 4 in this case), and click on `'Build a GridMDP'`\n", + "3. Click on `Initialize` in the `Edit` menu.\n", + "4. Set the reward as -0.4 and click `Apply`. Exit the dialog. \n", + "\n", + "
\n", + "5. Select cell (1, 1) and check the `Wall` radio button. `Apply` and exit the dialog.\n", + "\n", + "
\n", + "6. Select cells (4, 1) and (4, 2) and check the `Terminal` radio button for both. Set the rewards appropriately and click on `Apply`. Exit the dialog. Your window should look something like this.\n", + "\n", + "
\n", + "7. You are all set up now. Click on `Build and Run` in the `Build` menu and watch the heatmap calculate the utility function.\n", + "\n", + "
\n", + "Green shades indicate positive utilities and brown shades indicate negative utilities. \n", + "The values of the utility function and arrow diagram will pop up in separate dialogs after the algorithm converges." + ] } ], "metadata": { @@ -37,9 +2988,2496 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.1" + "version": "3.6.4" + }, + "widgets": { + "state": { + "001e6c8ed3fc4eeeb6ab7901992314dd": { + "views": [] + }, + "00f29880456846a8854ab515146ec55b": { + "views": [] + }, + "010f52f7cde545cba25593839002049b": { + "views": [] + }, + "01473ad99aa94acbaca856a7d980f2b9": { + "views": [] + }, + "021a4a4f35da484db5c37c5c8d0dbcc2": { + "views": [] + }, + "02229be5d3bc401fad55a0378977324a": { + "views": [] + }, + "022a5fdfc8e44fb09b21c4bd5b67a0db": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "025c3b0250b94d4c8d9b33adfdba4c15": { + "views": [] + }, + "028f96abfed644b8b042be1e4b16014d": { + "views": [] + }, + "0303bad44d404a1b9ad2cc167e42fcb7": { + "views": [] + }, + "031d2d17f32347ec83c43798e05418fe": { + "views": [] + }, + "03de64f0c2fd43f1b3b5d84aa265aeb7": { + "views": [] + }, + "03fdd484675b42ad84448f64c459b0e0": { + "views": [] + }, + "044cf74f03fd44fd840e450e5ee0c161": { + "views": [] + }, + "054ae5ba0a014a758de446f1980f1ba5": { + "views": [] + }, + "0675230fb92f4539bc257b768fb4cd10": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "06c93b34e1f4424aba9a0b172c428260": { + "views": [] + }, + "077a5ea324be46c3ad0110671a0c6a12": { + "views": [] + }, + "0781138d150142a08775861a69beaec9": { + "views": [] + }, + "0783e74a8c2b40cc9b0f5706271192f4": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "07c7678b73634e728085f19d7b5b84f7": { + "views": [] + }, + "07febf1d15a140d8adb708847dd478ec": { + "views": [] + }, + "08299b681cd9477f9b19a125e186ce44": { + "views": [] + }, + "083af89d82e445aab4abddfece61d700": { + "views": [] + }, + "08a1129a8bd8486bbfe2c9e49226f618": { + "views": [] + }, + "08a2f800c0d540fdb24015156c7ffc15": { + "views": [] + }, + "097d8d0feccc4c76b87bbcb3f1ecece7": { + "views": [] + }, + "098f12158d844cdf89b29a4cd568fda0": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "09e96f9d5d32453290af60fbd29ca155": { + "views": [] + }, + "0a2ec7c49dcd4f768194483c4f2e8813": { + "views": [] + }, + "0b1d6ed8fe4144b8a24228e1befe2084": { + "views": [] + }, + "0b299f8157d24fa9830653a394ef806a": { + "views": [] + }, + "0b2a4ac81a244ff1a7b313290465f8f4": { + "views": [] + }, + "0b52cfc02d604bc2ae42f4ba8c7bca4f": { + "views": [] + }, + "0b65fb781274495ab498ad518bc274d4": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "0b865813de0841c49b41f6ad5fb85c6a": { + "views": [] + }, + "0c2070d20fb04864aeb2008a6f2b8b30": { + "views": [] + }, + "0cf5319bcde84f65a1a91c5f9be3aa28": { + "views": [] + }, + "0d721b5be85f4f8aafe26b3597242d60": { + "views": [] + }, + "0d9f29e197ad45d6a04bbb6864d3be6d": { + "views": [] + }, + "0e03c7e2c0414936b206ed055e19acba": { + "views": [] + }, + "0e2265aa506a4778bfc480d5e48c388b": { + "views": [] + }, + "0e4e3d0b6afc413e86970ec4250df678": { + "views": [] + }, + "0e6a5fe6423542e6a13e30f8929a8b02": { + "views": [] + }, + "0e7b2f39c94343c3b0d3b6611351886e": { + "views": [] + }, + "0eb5005fa34440988bcf3be231d31511": { + "views": [] + }, + "104703ad808e41bc9106829bb0396ece": { + "views": [] + }, + "109c376b28774a78bf90d3da4587d834": { + "views": [] + }, + "10b24041718843da976ac616e77ea522": { + "views": [] + }, + "11516bb6db8b45ef866bd9be8bb59312": { + "views": [] + }, + "1203903354fa467a8f38dbbad79cbc81": { + "views": [] + }, + "124ecbe68ada40f68d6a1807ad6bcdf9": { + "views": [] + }, + "1264becdbb63455183aa75f236a3413e": { + "views": [] + }, + "13061cc21693480a8380346277c1b877": { + "views": [] + }, + "130dd4d2c9f04ad28d9a6ac40045a329": { + "views": [] + }, + "1350a087b5a9422386c3c5f04dd5d1c9": { + "views": [] + }, + "139bd19be4a4427a9e08f0be6080188e": { + "views": [] + }, + "13f9f589d36c477f9b597dda459efd16": { + "views": [] + }, + "140917b5c77348ec82ea45da139a3045": { + "views": [] + }, + "145419657bb1401ba934e6cea43d5fd1": { + "views": [] + }, + "15d748f1629d4da1982cd62cfbcb1725": { + "views": [] + }, + "17ad015dbc744ac6952d2a6da89f0289": { + "views": [] + }, + "17b6508f32e4425e9f43e5407eb55ed3": { + "views": [] + }, + "185598d8e5fc4dffae293f270a6e7328": { + "views": [] + }, + "196473b25f384f3895ee245e8b7874e9": { + "views": [] + }, + "19c0f87663a0431285a62d4ad6748046": { + "views": [] + }, + "1a00a7b7446d4ad8b08c9a2a9ea9c852": { + "views": [] + }, + "1a97f5b88cdc4ae0871578c06bbb9965": { + "views": [] + }, + "1a9a07777b0c4a45b33e25a70ebdc290": { + "views": [] + }, + "1af711fe8e4f43f084cef6c89eec40ae": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "1aff6a6e15b34bb89d7579d445071230": { + "views": [] + }, + "1b1ea7e915d846aea9efeae4381b2c48": { + "views": [] + }, + "1ba02ae1967740b0a69e07dbe95635cb": { + "views": [] + }, + "1c5c913acbde4e87a163abb2e24e6e38": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "1cfca0b7ef754c459e1ad97c1f0ceb3b": { + "views": [] + }, + "1d8f6a4910e649589863b781aab4c4d4": { + "views": [] + }, + "1e64b8f5a1554a22992693c194f7b971": { + "views": [] + }, + "1e8f0a2bf7614443a380e53ed27b48c0": { + "views": [] + }, + "1f4e6fa4bacc479e8cd997b26a5af733": { + "views": [] + }, + "1fdf09158eb44415a946f07c6aaba620": { + "views": [] + }, + "200e3ebead3d4858a47e2f6d345ca395": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "2050d4b462474a059f9e6493ba06ac58": { + "views": [] + }, + "20b5c21a6e6a427ba3b9b55a0214f75e": { + "views": [] + }, + "20b99631feba4a9c98c9d5f74c620273": { + "views": [] + }, + "20bcff5082854ab89a7977ae56983e30": { + "views": [] + }, + "20d708bf9b7845fa946f5f37c7733fee": { + "views": [] + }, + "210b36ea9edf4ee49ae1ae3fe5005282": { + "views": [] + }, + "21415393cb2d4f72b5c3f5c058aeaf66": { + "views": [] + }, + "2186a18b6ed8405a8a720bae59de2ace": { + "views": [] + }, + "220dc13e9b6942a7b9ed9e37d5ede7ba": { + "views": [] + }, + "221a735fa6014a288543e6f8c7e4e2ef": { + "views": [] + }, + "2288929cec4d4c8faad411029f5e21fa": { + "views": [] + }, + "22b86e207ea6469d85d8333870851a86": { + "views": [] + }, + "23283ad662a140e3b5e8677499e91d64": { + "views": [] + }, + "23a7cc820b63454ca6be3dcfd2538ac1": { + "views": [] + }, + "240ed02d576546028af3edfab9ea8558": { + "views": [] + }, + "24678e52a0334cb9a9a56f92c29750be": { + "views": [] + }, + "247820f6d83f4dd9b68f5df77dbda4b7": { + "views": [] + }, + "24b6a837fbd942c9a68218fb8910dcd5": { + "views": [] + }, + "24ee3204f26348bca5e6a264973e5b56": { + "views": [] + }, + "262c7bb5bd7447f791509571fe74ae44": { + "views": [] + }, + "263595f22d0d45e2a850854bcefe4731": { + "views": [] + }, + "2640720aa6684c5da6d7870abcbc950b": { + "views": [] + }, + "265ca1ec7ad742f096bb8104d0cf1550": { + "views": [] + }, + "26bf66fba453464fac2f5cd362655083": { + "views": [] + }, + "29769879478f49e8b4afd5c0b4662e87": { + "views": [] + }, + "29a13bd6bc8d486ca648bf30c9e4c2a6": { + "views": [] + }, + "29c5df6267584654b76205fc5559c553": { + "views": [] + }, + "29ce25045e7248e5892e8aafc635c416": { + "views": [] + }, + "2a17207c43c9424394299a7b52461794": { + "views": [] + }, + "2a777941580945bc83ddb0c817ed4122": { + "views": [] + }, + "2ae1844e2afe416183658d7a602e5963": { + "views": [] + }, + "2afa2938b41944cf8c14e41a431e3969": { + "views": [] + }, + "2bdc5f9b161548e3aab8ea392b5af1a1": { + "views": [] + }, + "2c26b2bcfc96473584930a4b622d268e": { + "views": [] + }, + "2ca2a914a5f940b18df0b5cde2b79e4b": { + "views": [] + }, + "2ca2c532840548a9968d1c6b2f0acdd8": { + "views": [] + }, + "2d17c32bfea143babe2b114d8777b15d": { + "views": [] + }, + "2d3acd8872c342eab3484302cac2cb05": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "2dc514cc2f5547aeb97059a5070dc9e3": { + "views": [] + }, + "2e1351ad05384d058c90e594bc6143c1": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "2e9b80fa18984615933e41c1c1db2171": { + "views": [] + }, + "2ef17ee6b7c74a4bbbbbe9b1a93e4fb6": { + "views": [] + }, + "2f5438f1b34046a597a467effd43df11": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "2f8d22417f3e421f96027fca40e1554f": { + "views": [] + }, + "2fb0409cfb49469d89a32597dc3edba9": { + "views": [] + }, + "303ccef837984c97b7e71f2988c737a4": { + "views": [] + }, + "3058b0808dca48a0bba9a93682260491": { + "views": [] + }, + "306b65493c28411eb10ad786bbf85dc5": { + "views": [] + }, + "30f5d30cf2d84530b3199015c5ff00eb": { + "views": [] + }, + "310b1ac518bd4079bdb7ecaf523a6809": { + "views": [] + }, + "313eca81d9d24664bcc837db54d59618": { + "views": [] + }, + "31413caf78c14548baa61e3e3c9edc55": { + "views": [] + }, + "317fbd3cb6324b2fbdfd6aa46a8d1192": { + "views": [] + }, + "319425ba805346f5ba366c42e220f9c6": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "31fc8165275e473f8f75c6215b5184ff": { + "views": [] + }, + "329f12edaa0c44d2a619450f188e8777": { + "views": [] + }, + "32edf057582f4a6ca30ce3cb685bf971": { + "views": [] + }, + "330e74773ba148e18674cfa3e63cd6cc": { + "views": [] + }, + "332a89c03bfb49c2bb291051d172b735": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3347dfda0aca450f89dd9b39ca1bec7d": { + "views": [] + }, + "336e8bcfd7cc4a85956674b0c7bffff2": { + "views": [] + }, + "3376228b3b614d4ab2a10b2fd0f484fd": { + "views": [] + }, + "3380a22bc67c4be99c61050800f93395": { + "views": [] + }, + "34b5c16cbea448809c2ccbce56f8d5a5": { + "views": [] + }, + "34bb050223504afc8053ce931103f52c": { + "views": [] + }, + "34c28187175d49198b536a1ab13668c4": { + "views": [] + }, + "3521f32644514ecf9a96ddfa5d80fb9b": { + "views": [] + }, + "36511bd77ed74f668053df749cc735d4": { + "views": [] + }, + "36541c3490bd4268b64daf20d8c24124": { + "views": [] + }, + "37aa1dd4d76a4bac98857b519b7b523a": { + "views": [] + }, + "37aa3cfa3f8f48989091ec46ac17ae48": { + "views": [] + }, + "386991b0b1424a9c816dac6a29e1206b": { + "views": [] + }, + "386cf43742234dda994e35b41890b4d8": { + "views": [] + }, + "388571e8e0314dfab8e935b7578ba7f9": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3974e38e718547efaf0445da2be6a739": { + "views": [] + }, + "398490e0cc004d22ac9c4486abec61e1": { + "views": [] + }, + "399875994aba4c53afa8c49fae8d369e": { + "views": [] + }, + "39b64aa04b1d4a81953e43def0ef6e10": { + "views": [] + }, + "39ffc3dd42d94a27ba7240d10c11b565": { + "views": [] + }, + "3a21291c8e7249e3b04417d31b0447cf": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3a377d9f46704d749c6879383c89f5d3": { + "views": [] + }, + "3a44a6f1f62742849e96d957033a0039": { + "views": [] + }, + "3b22d68709b046e09fe70f381a3944cd": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3b329209c8f547acae1925dc3eb4af77": { + "views": [] + }, + "3c1b2ec10a9041be8a3fad9da78ff9f6": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3c2be3c85c6d41268bb4f9d63a43e196": { + "views": [] + }, + "3c6796eff7c54238a7b7776e88721b08": { + "views": [] + }, + "3cbca3e11edf439fb7f8ba41693b4824": { + "views": [] + }, + "3d4b6b7c0b0c48ff8c4b8d78f58e0f1c": { + "views": [] + }, + "3de1faf0d2514f49a99b3d60ea211495": { + "views": [] + }, + "3df60d9ac82b42d9b885d895629e372e": { + "views": [] + }, + "3e5b9fd779574270bf58101002c152ce": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3e80f34623c94659bfab5b3b56072d9a": { + "views": [] + }, + "3e8bb05434cb4a0291383144e4523840": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "3ea1c8e4f9b34161928260e1274ee048": { + "views": [] + }, + "3f32f0915bc6469aaaf7170eff1111e3": { + "views": [] + }, + "3fe69a26ae7a46fda78ae0cb519a0f8b": { + "views": [] + }, + "4000ecdd75d9467e9dffd457b35aa65f": { + "views": [] + }, + "402d346f8b68408faed2fd79395cf3fb": { + "views": [] + }, + "402f4116244242148fdc009bb399c3bd": { + "views": [] + }, + "4049e0d7c0d24668b7eae2bb7169376e": { + "views": [] + }, + "4088c9ed71b0467b9b9417d5b04eda0e": { + "views": [] + }, + "40d70faa07654b6cb13496c32ba274b3": { + "views": [] + }, + "4146be21b7614abe827976787ec570f1": { + "views": [] + }, + "4198c08edda440dd93d1f6ce3e4efa62": { + "views": [] + }, + "42023d7d3c264f9d933d4cee4362852b": { + "views": [] + }, + "421ad8c67f754ce2b24c4fa3a8e951cf": { + "views": [] + }, + "4263fe0cef42416f8d344c1672f591f9": { + "views": [] + }, + "428e42f04a1e4347a1f548379c68f91b": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "42a47243baf34773943a25df9cf23854": { + "views": [] + }, + "4343b72c91d04a7c9a6080f30fc63d7d": { + "views": [] + }, + "43488264fc924c01a30fa58604074b07": { + "views": [] + }, + "4379175239b34553bf45c8ef9443ac55": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "43859798809a4a289c58b4bd5e49d357": { + "views": [] + }, + "43ad406a61a34249b5622aba9450b23d": { + "views": [] + }, + "4421c121414d464bb3bf1b5f0e86c37b": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "445cc08b4da44c2386ac9379793e3506": { + "views": [] + }, + "447cff7e256c434e859bb7ce9e5d71c8": { + "views": [] + }, + "44af7da9d8304f07890ef7d11a9f95fe": { + "views": [] + }, + "45021b6f05db4c028a3b5572bc85217f": { + "views": [] + }, + "457768a474844556bf9b215439a2f2e9": { + "views": [] + }, + "45d5689de53646fe9042f3ce9e281acc": { + "views": [] + }, + "461aa21d57824526a6b61e3f9b5af523": { + "views": [] + }, + "472ca253aab34b098f53ed4854d35f23": { + "views": [] + }, + "4731208453424514b471f862804d9bb8": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "47dfef9eaf0e433cb4b3359575f39480": { + "views": [] + }, + "48220a877d494a3ea0cc9dae19783a13": { + "views": [] + }, + "4882c417949b4b6788a1c3ec208fb1ac": { + "views": [] + }, + "49f5c38281984e3bad67fe3ea3eb6470": { + "views": [] + }, + "4a0d39b43eee4e818d47d382d87d86d1": { + "views": [] + }, + "4a470bf3037047f48f4547b594ac65fa": { + "views": [] + }, + "4abab5bca8334dfbb0434be39eb550db": { + "views": [] + }, + "4b48e08fd383489faa72fc76921eac4e": { + "views": [] + }, + "4b9439e6445c4884bd1cde0e9fd2405e": { + "views": [] + }, + "4b9fa014f9904fcf9aceff00cc1ebf44": { + "views": [] + }, + "4bdc63256c3f4e31a8fa1d121f430518": { + "views": [] + }, + "4bebb097ddc64bbda2c475c3a0e92ab5": { + "views": [] + }, + "4c201df21ca34108a6e7b051aa58b7f6": { + "views": [] + }, + "4ced8c156fd941eca391016fc256ce40": { + "views": [] + }, + "4d281cda33fa489d86228370e627a5b0": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "4d85e68205d94965bdb437e5441b10a1": { + "views": [] + }, + "4e0e6dd34ba7487ba2072d352fe91bf5": { + "views": [] + }, + "4e82b1d731dd419480e865494f932f80": { + "views": [] + }, + "4e9f52dea051415a83c4597c4f7a6c00": { + "views": [] + }, + "4ec035cba73647358d416615cf4096ee": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "4f09442f99aa4a9e9f460f82a50317c4": { + "views": [] + }, + "4f80b4e6b074475698efbec6062e3548": { + "views": [] + }, + "4f905a287b4f4f0db64b9572432b0139": { + "views": [] + }, + "50a339306cd549de86fbe5fa2a0a3503": { + "views": [] + }, + "51068697643243e18621c888a6504434": { + "views": [] + }, + "51333b89f44b41aba813aef099bdbb42": { + "views": [] + }, + "5141ae07149b46909426208a30e2861e": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "515606cb3b3a4fccad5056d55b262db4": { + "views": [] + }, + "51aa6d9f5a90481db7e3dd00d77d4f09": { + "views": [] + }, + "524091ea717d427db2383b46c33ef204": { + "views": [] + }, + "524d1132c88f4d91b15344cc427a9565": { + "views": [] + }, + "52f70e249adc4edb8dca28b883a5d4f4": { + "views": [] + }, + "531c080221f64b8ca50d792bbaa6f31e": { + "views": [] + }, + "53349c544b54450f8e2af9b8ba176d78": { + "views": [] + }, + "53a8b8e7b7494d02852a0dc5ccca51a2": { + "views": [] + }, + "53c963469eee41b59479753201626f18": { + "views": [] + }, + "5436516c280a49828c1c2f4783d9cf0e": { + "views": [] + }, + "55a1b0b794f44ac796bc75616f65a2a1": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "55ebf735de4c4b5ba2f09bc51d3593fd": { + "views": [] + }, + "56007830e925480e94a12356ff4fb6a4": { + "views": [] + }, + "56def8b3867843f990439b33dab3da58": { + "views": [] + }, + "5719bb596a5649f6af38c11c3daae6e9": { + "views": [] + }, + "572245b145014b6e91a3b5fe55e4cf78": { + "views": [] + }, + "5728da2e2d5a4c5595e1f49723151dca": { + "views": [] + }, + "579673c076da4626bc34a34370702bd4": { + "views": [] + }, + "57c2148f18314c3789c3eb9122a85c86": { + "views": [] + }, + "58066439757048b98709d3b3f99efdf8": { + "views": [] + }, + "58108da85e9443ea8ba884e8adda699e": { + "views": [] + }, + "583f252174d9450196cdc7c1ebab744f": { + "views": [] + }, + "58b92095873e4d22895ee7dde1f8e09a": { + "views": [] + }, + "58be1833a5b344fb80ec86e08e8326da": { + "views": [] + }, + "58ee0f251d7c4aca82fdace15ff52414": { + "views": [] + }, + "590f2f9f8dc342b594dc9e79990e641f": { + "views": [] + }, + "593c6f6b541e49be95095be63970f335": { + "views": [] + }, + "593d3f780c1a4180b83389afdb9fecfe": { + "views": [] + }, + "5945f05889be40019f93a90ecd681125": { + "views": [] + }, + "595c537ed2514006ac823b4090cf3b4b": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "599cfb7471ec4fd29d835d2798145a54": { + "views": [] + }, + "5a8d17dc45d54463a6a49bad7a7d87ac": { + "views": [] + }, + "5bb323bde7e4454e85aa18fda291e038": { + "views": [] + }, + "5bc5e0429c1e4863adc6bd1ff2225b6d": { + "views": [] + }, + "5bd0fafc4ced48a5889bbcebc9275e40": { + "views": [] + }, + "5ccf965356804bc38c94b06698a2c254": { + "views": [] + }, + "5d1f96bedebf489cac8f820c783f7a14": { + "views": [] + }, + "5d3fc58b96804b57aad1d67feb26c70a": { + "views": [] + }, + "5d41872e720049198a319adc2f476276": { + "views": [] + }, + "5d7a630da5f14cd4969b520c77bc5bc5": { + "views": [] + }, + "5da153e0261e43af8fd1c3c5453cace0": { + "views": [] + }, + "5dde90afb01e44888d3c92c32641d4e2": { + "views": [] + }, + "5de2611543ff4475869ac16e9bf406fd": { + "views": [] + }, + "5e03db9b91124e79b082f7e3e031a7d3": { + "views": [] + }, + "5e576992ccfe4bb383c88f80d9746c1d": { + "views": [] + }, + "5e91029c26c642a9a8c90186f3acba8e": { + "views": [] + }, + "5ea2a6c21b9845d18f72757ca5af8340": { + "views": [] + }, + "5ef08dc24584438c8bc6c618763f0bc8": { + "views": [] + }, + "5f823979d2ce4c34ba18b4ca674724e4": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "5fc7b070fc1a4e809da4cda3a40fc6d9": { + "views": [] + }, + "601ca9a27da94a6489d62ac26f2805a9": { + "views": [] + }, + "605cbb1049a4462e9292961e62e55cee": { + "views": [] + }, + "60addd9bec3f4397b20464fdbcf66340": { + "views": [] + }, + "60e17d6811c64dc8a69b342abe20810a": { + "views": [] + }, + "611840434d9046488a028618769e4b86": { + "views": [] + }, + "627ab7014bbf404ba8190be17c22e79d": { + "views": [] + }, + "633aa1edce474560956be527039800e7": { + "views": [] + }, + "63b6e287d1aa48efad7c8154ddd8f9c4": { + "views": [] + }, + "63dcfdb9749345bab675db257bda4b81": { + "views": [] + }, + "640ba8cc905a4b47ad709398cc41c4e3": { + "views": [] + }, + "644dcff39d7c47b7b8b729d01f59bee5": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "6455faf9dbc6477f8692528e6eb90c9a": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "64ca99573d5b48d2ba4d5815a50e6ffe": { + "views": [] + }, + "65d7924ba8c44d3f98a1d2f02dc883f1": { + "views": [] + }, + "665ed2b201144d78a5a1f57894c2267c": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "66742844c1cd47ddbbe9aacf2e805f36": { + "views": [] + }, + "6678811915f14d0f86660fe90f63bd60": { + "views": [] + }, + "66a04a5cf76e429cadbebfc527592195": { + "views": [] + }, + "66e5c563ffe94e29bab82fdecbd1befa": { + "views": [] + }, + "673066e0bb0b40e288e6750452c52bf6": { + "views": [] + }, + "67ae0fb9621d488f879d0e3c458e88e9": { + "views": [] + }, + "687702eca5f74e458c8d43447b3b9ed5": { + "views": [] + }, + "68a4135d6f0a4bae95130539a2a44b3c": { + "views": [] + }, + "68c3a74e9ea74718b901c812ed179f47": { + "views": [] + }, + "694bd01e350449c2a40cd4ffc5d5a873": { + "views": [] + }, + "6981c38c44ad4b42bfb453b36d79a0e6": { + "views": [] + }, + "69e08ffffce9464589911cc4d2217df2": { + "views": [] + }, + "6a28f605a5d14589907dba7440ede2fc": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "6a74dc52c2a54837a64ad461e174d4e0": { + "views": [] + }, + "6ad1e0bf705141b3b6e6ab7bd6f842ea": { + "views": [] + }, + "6b37935db9f44e6087d1d262a61d54ac": { + "views": [] + }, + "6b402f0f3afb4d0dad0e2fa8b71aa890": { + "views": [] + }, + "6bc95be59a054979b142d2d4a8900cf2": { + "views": [] + }, + "6ce0ea52c2fc4a18b1cce33933df2be4": { + "views": [] + }, + "6d7effd6bc4c40a4b17bf9e136c5814c": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "6d9a639e949c4d1d8a7826bdb9e67bb5": { + "views": [] + }, + "6e18fafd95744f689c06c388368f1d21": { + "views": [] + }, + "6e2bc4a1e3424e2085d0363b7f937884": { + "views": [] + }, + "6e30c494930c439a996ba7c77bf0f721": { + "views": [] + }, + "6e682d58cc384145adb151652f0e3d15": { + "views": [] + }, + "6f08def65d27471b88fb14e9b63f9616": { + "views": [] + }, + "6f20c1dc00ef4a549cd9659a532046bf": { + "views": [] + }, + "6f605585550d4879b2f27e2fda0192be": { + "views": [] + }, + "706dd4e39f194fbbba6e34acd320d1c3": { + "views": [] + }, + "70f21ab685dc4c189f00a17a1810bbad": { + "views": [] + }, + "7101b67c47a546c881fdaf9c934c0264": { + "views": [] + }, + "71b0137b5ed741be979d1896762e5c75": { + "views": [] + }, + "7223df458fdf4178af0b9596e231c09c": { + "views": [] + }, + "7262519db6f94e2a9006c68c20b79d29": { + "views": [] + }, + "72dfe79a3e52429da1cf4382e78b2144": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "72e8d31709eb4e3ea28af5cb6d072ab2": { + "views": [] + }, + "73647a1287424ee28d2fb3c4471d720c": { + "views": [] + }, + "739c5dde541a41e1afae5ba38e4b8ee3": { + "views": [] + }, + "74187cc424a347a5aa73b8140772ec68": { + "views": [] + }, + "7418edf751a6486c9fae373cde30cb74": { + "views": [] + }, + "744302ec305b4405894ed1459b9d41d0": { + "views": [] + }, + "74dfbaa15be44021860f7ba407810255": { + "views": [] + }, + "750a30d80fd740aaabc562c0564f02a7": { + "views": [] + }, + "75e344508b0b45d1a9ae440549d95b1a": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "766efd1cfee542d3ba068dfa1705c4eb": { + "views": [] + }, + "7738084e8820466f9f763d49b4bf7466": { + "views": [] + }, + "781855043f1147679745947ff30308fa": { + "views": [] + }, + "78e2cfb79878452fa4f6e8baea88f822": { + "views": [] + }, + "796027b3dd6b4b888553590fecd69b29": { + "views": [] + }, + "7a302f58080c4420b138db1a9ed8103e": { + "views": [] + }, + "7a3c362499f54884b68e951a1bcfc505": { + "views": [] + }, + "7a4ee63f5f674454adf660bfcec97162": { + "views": [] + }, + "7ac2c18126414013a1b2096233c88675": { + "views": [] + }, + "7b1e3c457efa4f92ab8ff225a1a2c45e": { + "views": [] + }, + "7b8897b4f8094eef98284f5bb1ed5d51": { + "views": [] + }, + "7bbfd7b13dd242f0ac15b36bb437eb22": { + "views": [] + }, + "7d3c88bc5a0f4b428174ff33d5979cfd": { + "views": [] + }, + "7d4f53bd14d44f3f80342925f5b0b111": { + "views": [] + }, + "7d95ca693f624336a91c3069e586ef1b": { + "views": [] + }, + "7dcdc07b114e4ca69f75429ec042fabf": { + "views": [] + }, + "7e79b941d7264d27a82194c322f53b80": { + "views": [] + }, + "7f2f98bbffc0412dbb31c387407a9fed": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "7f4688756da74b369366c22fd99657f4": { + "views": [] + }, + "7f7ed281359f4a55bbe75ce841dd1453": { + "views": [] + }, + "7fdf429182a740a097331bddad58f075": { + "views": [] + }, + "81b312df679f4b0d8944bc680a0f517e": { + "views": [] + }, + "82036e8fa76544ae847f2c2fc3cf72c2": { + "views": [] + }, + "821f1041188a43a4be4bdaeb7fa2f201": { + "views": [] + }, + "827358a9b4ce49de802df37b7b673aea": { + "views": [] + }, + "82db288a0693422cbd846cc3cb5f0415": { + "views": [] + }, + "82e2820c147a4dff85a01bcddbad8645": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "82f795491023435e8429ea04ff4dc60a": { + "views": [] + }, + "8317620833b84ccebc4020d90382e134": { + "views": [] + }, + "8346e26975524082af27967748792444": { + "views": [] + }, + "83f8ed39d0c34dce87f53f402d6ee276": { + "views": [] + }, + "844ac22a0ebe46db84a6de7472fe9175": { + "views": [] + }, + "849948fe6e3144e1b05c8df882534d5a": { + "views": [] + }, + "85058c7c057043b185870da998e4be61": { + "views": [] + }, + "85443822f3714824bec4a56d4cfed631": { + "views": [] + }, + "8566379c7ff943b0bb0f9834ed4f0223": { + "views": [] + }, + "85a3c6f9a0464390be7309edd36c323c": { + "views": [] + }, + "85d7a90fbac640c9be576f338fa25c81": { + "views": [] + }, + "85f31444b4e44e11973fd36968bf9997": { + "views": [] + }, + "867875243ad24ff6ae39b311efb875d3": { + "views": [] + }, + "8698bede085142a29e9284777f039c93": { + "views": [] + }, + "86bf40f5107b4cb6942800f3930fdd41": { + "views": [] + }, + "874c486c4ebb445583bd97369be91d9b": { + "views": [] + }, + "87c469625bda412185f8a6c803408064": { + "views": [] + }, + "87d4bd76591f4a9f991232ffcff3f73b": { + "views": [] + }, + "87df3737c0fc4e848fe4100b97d193df": { + "views": [] + }, + "886b599c537b467ab49684d2c2f8fb78": { + "views": [] + }, + "889e19694e8043e289d8efc269eba934": { + "views": [] + }, + "88c628983ad1475ea3a9403f6fea891c": { + "views": [] + }, + "88c807c411d34103ba2e31b2df28b947": { + "views": [] + }, + "895ddca8886b4c06ad1d71326ca2f0af": { + "views": [] + }, + "899cc011a1bd4046ac798bc5838c2150": { + "views": [] + }, + "89d0e7a3090c47df9689d8ca28914612": { + "views": [] + }, + "89ea859f8bbd48bb94b8fa899ab69463": { + "views": [] + }, + "8a600988321e4e489450d26dedaa061f": { + "views": [] + }, + "8adcca252aff41a18cca5d856c17e42f": { + "views": [] + }, + "8b2fe9e4ea1a481089f73365c5e93d8b": { + "views": [] + }, + "8b5acd50710c4ca185037a73b7c9b25c": { + "views": [] + }, + "8bbdba73a1454cac954103a7b1789f75": { + "views": [] + }, + "8cffde5bdb3d4f7597131b048a013929": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "8db2abcad8bc44df812d6ccf2d2d713c": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "8dd5216b361c44359ba1233ee93683a4": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "8e13719438804be4a0b74f73e25998cd": { + "views": [] + }, + "8eb4ff3279fe4d43a9d8ee752c78a956": { + "views": [] + }, + "8f577d437d4743fd9399fefcd8efc8cb": { + "views": [] + }, + "8f8fbe8fd1914eae929069aeeac16b6d": { + "views": [] + }, + "8f9b8b5f7dd6425a9e8e923464ab9528": { + "views": [] + }, + "8f9e3422db114095a72948c37e98dd3e": { + "views": [] + }, + "8fd325068289448d990b045520bad521": { + "views": [] + }, + "9039bc40a5ad4a1c87272d82d74004e2": { + "views": [] + }, + "90bf5e50acbb4bccad380a6e33df7e40": { + "views": [] + }, + "91028fc3e4bc4f6c8ec752b89bcf3139": { + "views": [] + }, + "9274175be7fb47f4945e78f96d39a7a6": { + "views": [] + }, + "929245675b174fe5bfa102102b8db897": { + "views": [] + }, + "92be1f7fb2794c9fb25d7bbb5cbc313d": { + "views": [] + }, + "933904217b6045c1b654b7e5749203f5": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "936bc7eb12e244c196129358a16e14bb": { + "views": [] + }, + "936c09f4dde8440b91e9730a0212497c": { + "views": [] + }, + "9406b6ae7f944405a0e8a22f745a39b2": { + "views": [] + }, + "942a96eea03740719b28fcc1544284d4": { + "views": [] + }, + "94840e902ffe4bbba5b374ff4d26f19f": { + "views": [] + }, + "948d01f0901545d38e05f070ce4396e4": { + "views": [] + }, + "94e2a0bc2d724f7793bb5b6d25fc7088": { + "views": [] + }, + "94f2b877a79142839622a61a3a081c03": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "94f30801a94344129363c8266bf2e1f8": { + "views": [] + }, + "95b127e8aff34a76a813783a6a3c6369": { + "views": [] + }, + "95d44119bf714e42b163512d9a15bbc5": { + "views": [] + }, + "95f016e9ea9148a4a3e9f04cb8f5132d": { + "views": [] + }, + "968e9e9de47646409744df3723e87845": { + "views": [] + }, + "97207358fc65430aa196a7ed78b252f0": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "9768d539ee4044dc94c0bd5cfb827a18": { + "views": [] + }, + "98587702cc55456aa881daf879d2dc8d": { + "views": [] + }, + "986c6c4e92964759903d6eb7f153df8a": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "987d808edd63404f8d6f2ce42efff33a": { + "views": [] + }, + "9895c26dfb084d509adc8abc3178bad3": { + "views": [] + }, + "994bc7678f284a24a8700b2a69f09f8d": { + "views": [] + }, + "99eee4e3d9c34459b12fe14cee543c28": { + "views": [] + }, + "9a5c0b0805034141a1c96ddd57995a3c": { + "views": [] + }, + "9a7862bb66a84b4f897924278a809ef3": { + "views": [] + }, + "9b812f733f6a4b60ba4bf725959f7913": { + "views": [] + }, + "9bb5ae9ff9c94fe7beece9ce43f519af": { + "views": [] + }, + "9bfde7b437fb4e76a16a49574ea5b7ec": { + "views": [] + }, + "9c1d14484b6d4ab3b059731f17878d14": { + "views": [] + }, + "9c7a66ead55e48c8b92ef250a5a464b7": { + "views": [] + }, + "9ce50a53aafe439ebb19fff363c1bfe2": { + "views": [] + }, + "9d5e9658af264ad795f6a5f3d8c3c30f": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "9d7aa65511b6482d9587609ad7898f54": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "9d87f94baf454bd4b529e55e0792a696": { + "views": [] + }, + "9de4bd9c6a7b4f3dbd401df15f0b9984": { + "views": [] + }, + "9dfd6b08a2574ed89f0eb084dae93f73": { + "views": [] + }, + "9e1dffcb1d9d48aaafa031da2fb5fed9": { + "views": [] + }, + "9efb46d2bb0648f6b109189986f4f102": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "9f1439500d624f769dd5e5c353c46866": { + "views": [] + }, + "9f27ba31ccc947b598dc61aefca16a7f": { + "views": [] + }, + "9f31a58b6e8e4c79a92cf65c497ee000": { + "views": [] + }, + "9f43f85a0fb9464e9b7a25a85f6dba9c": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "9f4970dc472946d48c14e93e7f4d4b70": { + "views": [] + }, + "9f5dd25217a84799b72724b2a37281ea": { + "views": [] + }, + "9faa50b44e1842e0acac301f93a129c4": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "a0202917348d4c41a176d9871b65b168": { + "views": [] + }, + "a058f021f4ca4daf8ab830d8542bf90b": { + "views": [] + }, + "a0a2dded995543a6b68a67cd91baa252": { + "views": [] + }, + "a0e170b3ea484fd984985d2607f90ef3": { + "views": [] + }, + "a168e79f4cbb44c8ac7214db964de5f2": { + "views": [] + }, + "a182b774272b48238b55e3c4d40e6152": { + "views": [] + }, + "a1840ca22d834df2b145151baf6d8241": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "a1bb2982e88e4bb1a2729cc08862a859": { + "views": [] + }, + "a1d897a6094f483d8fc9a3638fbc179d": { + "views": [] + }, + "a231ee00d2b7404bb0ff4e303c6b04ee": { + "views": [] + }, + "a29fdc2987f44e69a0343a90d80c692c": { + "views": [] + }, + "a2de3ac1f4fe423997c5612b2b21c12f": { + "views": [] + }, + "a30ba623acec4b03923a2576bcfcbdf5": { + "views": [] + }, + "a3357d5460c5446196229eae087bb19e": { + "views": [] + }, + "a358d9ecd754457db178272315151fa3": { + "views": [] + }, + "a35aec268ac3406daa7fe4563f83f948": { + "views": [] + }, + "a38c5ed35b9945008341c2d3c0ef1470": { + "views": [] + }, + "a39cfb47679c4d2895cda12c6d9d2975": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "a55227f2fd5d42729fc4fd39a8c11914": { + "views": [] + }, + "a65af2c8506d47ec803c15815e2ab445": { + "views": [] + }, + "a6d2366540004eeaab760c8be196f10a": { + "views": [] + }, + "a709f15a981a468b9471a0f672f961a7": { + "views": [] + }, + "a7258472ad944d038cd227de28d9155f": { + "views": [] + }, + "a72eb43242c34ef19399c52a77da8830": { + "views": [] + }, + "a7568aed621548649e37cfa6423ca198": { + "views": [] + }, + "a83f7f5c09a845ecb3f5823c1d178a54": { + "views": [] + }, + "a87c651448f14ce4958d73c2f1e413e1": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "a8e78f5bc64e412ab44eb9c293a7e63b": { + "views": [] + }, + "a996d507452241e0b99aabe24eecbdd9": { + "views": [] + }, + "a9a4b7a2159e40f8aa93a50f11048342": { + "views": [] + }, + "a9cc48370b964a888f8414e1742d6ff2": { + "views": [] + }, + "a9dcbe9e9a4445bf9cf8961d4c1214a6": { + "views": [] + }, + "aab29dfddb98416ea815475d6c6a3eed": { + "views": [] + }, + "ab89783a86bc4939a5f78957f4019553": { + "views": [] + }, + "abaee5bb577d4a68b6898d637a4c7898": { + "views": [] + }, + "abecb04251e04260860074b8bdad088a": { + "views": [] + }, + "acc07b8cf2cf4d50ae1bceef2254637f": { + "views": [] + }, + "ae3ee1ee05a2443c8bf2f79cd9e86e56": { + "views": [] + }, + "ae4e85e2bceb4ec783dbfaaf3a174ea7": { + "views": [] + }, + "aec1a51db98f470cb0854466f3461fc1": { + "views": [] + }, + "afc5dccd3db64a1592ee0b2fd516b71d": { + "views": [] + }, + "afe28f5bae8941b19717e3d7285ddc61": { + "views": [] + }, + "b00516b171544bca9113adc99ed528a1": { + "views": [] + }, + "b005d7f2afbe479eb02678447a079a1a": { + "views": [] + }, + "b020ad1a7750461bb79fe4e74b9384f6": { + "views": [] + }, + "b07d0aab375142978e1261a6a4c94b10": { + "views": [] + }, + "b2c18df5c51649cdbdaf64092fc945b3": { + "views": [] + }, + "b410c14ee52d4af49c08da115db85ac7": { + "views": [] + }, + "b41220079b2b49c2ba6f59dcfe9e7757": { + "views": [] + }, + "b445a187ca6943bbb465782a67288ce5": { + "views": [] + }, + "b4dfb435038645dc9673ea4257fc26f3": { + "views": [] + }, + "b5633708bd8b4abdaec77a96aca519bb": { + "views": [] + }, + "b59b2622026d4ec582354d919e16f658": { + "views": [] + }, + "b635f31747e14f989c7dee2ba5d5caa5": { + "views": [] + }, + "b63dfdde813a4f019998e118b5168943": { + "views": [] + }, + "b6c3d440986d44ed88a9471a69b70e05": { + "views": [] + }, + "b6ee195c9bfd48ee8526b8cf0f3322b9": { + "views": [] + }, + "b7064dd21c9949d79f40c73fee431dff": { + "views": [] + }, + "b7537298609f4d64b8e36692b84f376c": { + "views": [] + }, + "b755013f41fa4dce8e2bab356d85d26d": { + "views": [] + }, + "b7cd4bfabc2e40fe9f30de702ae63716": { + "views": [] + }, + "b7e4c497ff5c4173961ffdc3bd3821a9": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "b821a13ce3e8453d85f07faccc95fee1": { + "views": [] + }, + "b86ea9c1f1ee45a380e35485ad4e2fac": { + "views": [] + }, + "b87f4d4805944698a0011c10d626726c": { + "views": [] + }, + "b8e173c7c8be41df9161cbbe2c4c6c86": { + "views": [] + }, + "b9322adcd8a241478e096aa1df086c78": { + "views": [] + }, + "b9ad471398784b6889ce7a1d2ef5c4c0": { + "views": [] + }, + "b9c138598fce460692cc12650375ee52": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "ba146eb955754db88ba6c720e14ea030": { + "views": [] + }, + "ba48cba009e8411ea85c7e566a47a934": { + "views": [] + }, + "bb2793de83a64688b61a2007573a8110": { + "views": [] + }, + "bb53891d7f514a17b497f699484c9aed": { + "views": [] + }, + "bbe5dea9d57d466ba4e964fce9af13cf": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "bbe88faf528d44a0a9083377d733d66a": { + "views": [] + }, + "bc0525d022404722a921132e61319e46": { + "views": [] + }, + "bc320fb35f5744cc82486b85f7a53b6f": { + "views": [] + }, + "bc900e9562c546f9ae3630d5110080ec": { + "views": [] + }, + "bcbf6b3ff19d4eb5aa1b8a57672d7f6f": { + "views": [] + }, + "bccf183ccb0041e380732005f2ca2d0a": { + "views": [] + }, + "bd0d18e3441340a7a56403c884c87a8e": { + "views": [] + }, + "bd21e4fe92614c22a76ae515077d2d11": { + "views": [] + }, + "bd5b05203cfd402596a6b7f076c4a8f8": { + "views": [] + }, + "beb0c9b29d8d4d69b3147af666fa298b": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "bf0d147a6a1346799c33807404fa1d46": { + "views": [] + }, + "c03d4477fa2a423dba6311b003203f62": { + "views": [] + }, + "c05697bcb0a247f78483e067a93f3468": { + "views": [] + }, + "c09c3d0e94ca4e71b43352ca91b1a88a": { + "views": [] + }, + "c0d015a0930e4ddf8f10bbace07c0b24": { + "views": [] + }, + "c15edd79a0fd4e24b06d1aae708a38c4": { + "views": [] + }, + "c20b6537360f4a70b923e6c5c2ba7d9b": { + "views": [] + }, + "c21fff9912924563b28470d32f62cd44": { + "views": [] + }, + "c2482621d28542268a2b0cbf4596da37": { + "views": [] + }, + "c25bd0d8054b4508a6b427447b7f4576": { + "views": [] + }, + "c301650ac4234491af84937a8633ad76": { + "views": [] + }, + "c333a0964b1e43d0817e73cb47cf0317": { + "views": [] + }, + "c36213b1566843ceb05b8545f7d3325c": { + "views": [] + }, + "c37d0add29fa4f41a47caf6538ec6685": { + "views": [] + }, + "c409a01effb945c187e08747e383463c": { + "views": [] + }, + "c4e104a7b731463688e0a8f25cf50246": { + "views": [] + }, + "c54f609af4e94e93b57304bc55e02eba": { + "views": [] + }, + "c576bf6d24184f3a9f31d4f40231ce87": { + "views": [] + }, + "c58ab80a895344008b5aadd8b8c628a4": { + "views": [] + }, + "c5d28bea41da447e88f4cec9cfaaf197": { + "views": [] + }, + "c74bbd55a8644defa3fcef473002a626": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "c856e77b213b400599b6e026baaa4c85": { + "views": [] + }, + "c894f9e350a1473abb28ff651443ae6f": { + "views": [] + }, + "c8e3827ae28b45bc9768a8c3e35cc8b1": { + "views": [] + }, + "c95bf1935b71400e98c63722b77caa08": { + "views": [] + }, + "c9e5129d30ea4b78b846e8e92651b0e9": { + "views": [] + }, + "ca2123c7b103485c851815cbcb4a6c17": { + "views": [] + }, + "ca34917db02148168daf0c30ceed7466": { + "views": [] + }, + "caa6adf7b0d243da8229c317c7482fe3": { + "views": [] + }, + "cb924475ebb64e76964f88e830979d38": { + "views": [] + }, + "cba1473ccaee4b2a89aba4d2b4b1e648": { + "views": [] + }, + "cbd735eb8eb446069ee912d795ccaf14": { + "views": [] + }, + "cc0ee37900ef40069515c79e99a9a875": { + "views": [] + }, + "cc564bca35c743b89697f5cfd4ecccc2": { + "views": [] + }, + "cc5a47588e2b4c8eb5deff560a0256c2": { + "views": [] + }, + "ccc64ac3a8a84ae9815ff9e8bdc3279d": { + "views": [] + }, + "cd02a06cec7342438f8585af6227db96": { + "views": [] + }, + "cd236465e91d4a90a2347e6baab6ab71": { + "views": [] + }, + "cd9a0aa1700a4407ab445053029dca18": { + "views": [] + }, + "cdd6c6a945a74c568d611b42e4ba8a1a": { + "views": [] + }, + "cdf0323ea1324c0b969f49176ecee1c2": { + "views": [] + }, + "ce3a0e82e80d48b9b2658e0c52196644": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "ce6ad0459f654b6785b3a71ccdf05063": { + "views": [] + }, + "ce8d3cd3535b459c823da2f49f3cc526": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "cf8c8f791d0541ffa4f635bb07389292": { + "views": [] + }, + "cfed29ab68f244e996b0d571c31020ec": { + "views": [] + }, + "d034cbd7b06a448f98b3f11b68520c08": { + "views": [] + }, + "d13135f5facc4c5996549a85974145a1": { + "views": [] + }, + "d18c7c17fa93493ebc622fe3d2c0d44e": { + "views": [] + }, + "d23b743d7d0342aca257780f2df758d6": { + "views": [] + }, + "d2fe43f4a2064078a6c8da47f8afb903": { + "views": [] + }, + "d34f626ca035456bb9e0c9ad2a9dced1": { + "views": [] + }, + "d359911be08f4342b20e86a954cd060f": { + "views": [] + }, + "d4d76a1c09a342e79cd6733886626459": { + "views": [] + }, + "d58d12f54e2b426fba4ca611b0ffc68f": { + "views": [] + }, + "d5e2a77d429d4ca0969e1edec5dc2690": { + "views": [] + }, + "d5f4bbe3242245f0a2c3b18a284e55f8": { + "views": [] + }, + "d6c325f3069a4186b3022619f4280c37": { + "views": [] + }, + "d6d46520bbcf495bad20bcd266fe1357": { + "views": [] + }, + "d72b7c8058324d1bb56b6574090ccda6": { + "views": [] + }, + "d73bbb49a33d49e187200fa7c8f23aaa": { + "views": [] + }, + "d80e4f8eb9a54aef8b746e38d8c3ef1b": { + "views": [] + }, + "d819255bc7104ee8b9466b149dba5bff": { + "views": [] + }, + "d819fcff913441d39a41982518127af5": { + "views": [] + }, + "d8295021db704345a63c9ff9d692b761": { + "views": [] + }, + "d83329fe36014f85bb5d0247d3ae4472": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "d88a0305cc224037a14e5040ed8e13af": { + "views": [] + }, + "d89b81d63c6048ff800d3380bf921ac0": { + "views": [] + }, + "d8d8667ab50944e4b066d648aa3c8e2a": { + "views": [] + }, + "d8fd2b5ef6e24628b2b5102d3cd375f3": { + "views": [] + }, + "d9579a126d5f44a3bc0a731e0ad55f24": { + "views": [] + }, + "da51bd4d4fd848699919e3973b2fabc2": { + "views": [] + }, + "dba5a5a8fec346b2bcdc88f4ce294550": { + "views": [] + }, + "dc201c38ac434cb8a424553f1fa5a791": { + "views": [] + }, + "dc631df85ae84ffc964acd7a76e399ce": { + "views": [] + }, + "dc7376a2272e44179f237e5a1c7f6a49": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "dc8a45203a0a457c927f582f9d576e5d": { + "views": [] + }, + "dcc0e1ea9e994fc0827d9d7f648e4ad9": { + "views": [] + }, + "dce6f4cb98094ee1b06c0dd0ff8f488a": { + "views": [] + }, + "dcfc688de41b4ed7a8f89ae84089d5c0": { + "views": [] + }, + "dd486b2cbda84c83ace5ceaee8a30ff8": { + "views": [] + }, + "ddcfbf7b97714357920ba9705e8d4ab0": { + "views": [] + }, + "ddd4485714564c65b70bd865783076af": { + "views": [] + }, + "de7738417f1040b1a06ad25e485eb91d": { + "views": [] + }, + "df4cada92e484fd4ae75026eaf1845e2": { + "views": [] + }, + "dfb3707b4a01441c8a0a1751425b8e1c": { + "views": [] + }, + "e03b701a52d948aab86117c928cbe275": { + "views": [] + }, + "e0a614fe085c4d3c835c78d6ada60a40": { + "views": [] + }, + "e138e0c7d5a4471d99bbdac50de00fe1": { + "views": [] + }, + "e154289ce1774450a9a51ac45a1d5725": { + "views": [] + }, + "e25c1d2c78c94c9a805920df36268508": { + "views": [] + }, + "e281172ebc7f48b5ae6545b16da79477": { + "views": [] + }, + "e2862bd7efac4bc0b23532705f5e46c4": { + "views": [] + }, + "e2cd9bb21f254e08885f43fd6e968879": { + "views": [] + }, + "e2f4acecaf194351b8e67439440a9966": { + "views": [] + }, + "e3198c124ac841a79db062efa81f6812": { + "views": [] + }, + "e36f3009f61a4f5ba047562e70330add": { + "views": [] + }, + "e3765274f28b4a55a82d9115ded151de": { + "views": [] + }, + "e37e3fba3b40413180cd30e594bf62bd": { + "views": [] + }, + "e3f9760867fa410fbdc4611aef1cee18": { + "views": [] + }, + "e4331c134ab24f9cae99d476dfa04c89": { + "views": [] + }, + "e46db59e121045169a1ea5313b1748b7": { + "views": [] + }, + "e475d1e00f9d48edadac886fb53c2a20": { + "views": [] + }, + "e48449d21c2d4360b851169468066470": { + "views": [] + }, + "e4c26b8a42b54e959b276a174f2c2795": { + "views": [] + }, + "e4e55dabd92f4c17b78ed4b6881842e8": { + "views": [] + }, + "e4e5dd3dc28d4aa3ab8f8f7c4a475115": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "e516fd8ebfc6478c95130d6edec77c88": { + "views": [] + }, + "e5afb8d0e8a94c4dac18f2bbf1d042ce": { + "views": [] + }, + "e5bcb13bf2e94afc857bcbb37f6d4d87": { + "views": [] + }, + "e64ab85e80184b70b69d01a9c6851943": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "e66b26fb788944ba83b7511d79b85dc5": { + "views": [] + }, + "e73434cfcc854429ac27ddc9c9b07f5e": { + "views": [] + }, + "e7a8244ea5a84493b3b5bdeaf92a50b4": { + "views": [] + }, + "e81ed2c281df4f06bc1d4e6b67c574b4": { + "views": [] + }, + "e85ff7ccdc034c268df9cb0e95e9b850": { + "views": [] + }, + "e8a198bff55a437eab56887563cd9a6e": { + "views": [] + }, + "e92ede4cfc96436b84e63809bcb22385": { + "views": [] + }, + "e949474f6aa64c5dada603476ea6cabd": { + "views": [] + }, + "e98e59c3156c49c1bb27be7a478c3654": { + "views": [] + }, + "e9ea6f88d1334fbcab7f9c9a11cf4a50": { + "views": [] + }, + "ea09e5da878c42f2b533856dc3149e3e": { + "views": [] + }, + "ea74036074054593b1cc31fec030d2a2": { + "views": [] + }, + "ea8d97fb8c0d499095cceb133e4d7d9c": { + "views": [] + }, + "eafbea5bce1f4ab4bcbb0aa08598af0f": { + "views": [] + }, + "ec01e6cdc5a54f068f1bb033415b4a06": { + "views": [] + }, + "ec2d1f18f2e841b184f5d4cd15979d46": { + "views": [] + }, + "ec923af478b94ad99bdfd3257f48cb06": { + "views": [] + }, + "ed02e2272e844678979bd6a3c00f5cb3": { + "views": [] + }, + "ed80296f5f5e42e694dfc5cc7fd3acee": { + "views": [] + }, + "ee4df451ca9d4ed48044b25b19dc3f3f": { + "views": [] + }, + "ee77219007884e089fc3c1479855c469": { + "views": [] + }, + "ef372681937b4e90a04b0d530b217edb": { + "views": [] + }, + "ef452efe39d34db6b4785cb816865ca3": { + "views": [] + }, + "efcb07343f244ff084ea49dbc7e3d811": { + "views": [] + }, + "f083a8e4c8574fe08f5eb0aac66c1e71": { + "views": [] + }, + "f09d7c07bec64811805db588515af7f6": { + "views": [] + }, + "f0ef654c93974add9410a6e243e0fbf2": { + "views": [] + }, + "f20d7c2fcf144f5da875c6af5ffd35cb": { + "views": [] + }, + "f234eb38076146b9a640f44b7ef30892": { + "views": [] + }, + "f24d087598434ed1bb7f5ae3b0b4647a": { + "views": [] + }, + "f262055f3f1b48029f9e2089f752b0b8": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "f2d40a380f884b1b95992ccc7c3df04e": { + "views": [] + }, + "f2e2e2e5177542aa9e5ca3d69508fb89": { + "views": [] + }, + "f31914f694384908bec466fc2945f1c7": { + "views": [] + }, + "f31cbea99df94f2281044c369ef1962d": { + "views": [] + }, + "f32c6c5551f540709f7c7cd9078f1aad": { + "views": [] + }, + "f337eb824d654f0fbd688e2db3c5bf7b": { + "views": [] + }, + "f36f776a7767495cbda2f649c2b3dd48": { + "views": [] + }, + "f3cef080253c46989413aad84b478199": { + "views": [] + }, + "f3df35ce53e0466e81a48234b36a1430": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "f3fa0f8a41ab4ede9c4e20f16e35237d": { + "views": [] + }, + "f42e4f996f254a1bb7fe6f4dfc49aba3": { + "views": [] + }, + "f437babcddc64a8aa238fc7013619fbb": { + "views": [] + }, + "f44a5661ed1f4b5d97849cf4bb5e862e": { + "views": [] + }, + "f44d24e28afa475da40628b4fd936922": { + "views": [] + }, + "f44d5e6e993745b8b12891d1f3af3dc3": { + "views": [] + }, + "f457cb5e76be46a29d9f49ba0dc135f1": { + "views": [] + }, + "f4691cbe84534ef6b7d3fca530cf1704": { + "views": [] + }, + "f4ca26fbbdbf49dda5d1b8affdecfa3e": { + "views": [] + }, + "f54998361fe84a8a95b2607fbe367d52": { + "views": [] + }, + "f54bdb1d3bfb47af9e7aaabb4ed12eff": { + "views": [] + }, + "f54c28b82f7d498b83bf6908e19b6d1b": { + "views": [] + }, + "f5cc05fcee4d4c3e80163c6e9c072b6e": { + "views": [] + }, + "f621b91a209e4997a47cf458f8a5027f": { + "views": [] + }, + "f665bf176eb443f6867cef8fdd79b4e5": { + "views": [] + }, + "f6e27824f5e84bd8b4671e9eb030b20f": { + "views": [] + }, + "f6f162ac0811434ea95875f6335bd484": { + "views": [] + }, + "f6f629e6fb164c97acdc50c25d1354ee": { + "views": [] + }, + "f71adee125f74ddd8302aa2796646d67": { + "views": [] + }, + "f731d66445aa4543800a6bb3e9267936": { + "views": [] + }, + "f8f8e8c27fff45afa309a849d1655e29": { + "views": [] + }, + "f913752b9e86487cb197f894d667d432": { + "views": [] + }, + "f92cde8d24064ae5afd4cd577eaa895a": { + "views": [] + }, + "f944674b7ca345a582de627055614499": { + "views": [] + }, + "f9458080ed534d25856c67ce8f93d5a1": { + "views": [ + { + "cell_index": 27 + } + ] + }, + "f986f98d05dd4b9fa8a3c1111c1cea9b": { + "views": [] + }, + "f9f7bc097f654e41b68f2d849c99a1a1": { + "views": [] + }, + "fa00693458bc45669e2ed4ee536e98d6": { + "views": [] + }, + "fa2f219e60ff453da3842df62a371813": { + "views": [] + }, + "fa6cbfe76fff48848dc08a9344de84ff": { + "views": [] + }, + "fb3b6d5e405d4e1b87e82bcc8ae3df0f": { + "views": [] + }, + "fbe27ee7dc93467292b67f68935ae6f0": { + "views": [] + }, + "fc494b2bcade4c3a890f08386dd8aab0": { + "views": [] + }, + "fd98ac9b76cc44f09bc3b684caf1882d": { + "views": [] + }, + "feb9bf5d951c40d4a87d57a4de5e819a": { + "views": [] + }, + "fedfd679505d409fa74ccaa52b87fcce": { + "views": [] + }, + "fef0278d4386407f96c44b4affe437b8": { + "views": [] + }, + "ff29b06d50b048d6bbcbdb5a8665dcde": { + "views": [] + }, + "ff3c868e31c0430dbf5b85415da9a24b": { + "views": [] + }, + "ff8a91a101044f4fba19cdfffc39e0d3": { + "views": [] + }, + "ffbca26ec77b492bbbda1be40b044d8e": { + "views": [] + }, + "fff5f5bc334942bd851ac24f782f4f3c": { + "views": [] + } + }, + "version": "1.1.1" } }, "nbformat": 4, - "nbformat_minor": 0 + "nbformat_minor": 1 } diff --git a/mdp.py b/mdp.py index 442a92de6..1003e26b5 100644 --- a/mdp.py +++ b/mdp.py @@ -1,150 +1,250 @@ -"""Markov Decision Processes (Chapter 17) +""" +Markov Decision Processes (Chapter 17) First we define an MDP, and the special case of a GridMDP, in which -states are laid out in a 2-dimensional grid. We also represent a policy -as a dictionary of {state:action} pairs, and a Utility function as a -dictionary of {state:number} pairs. We then define the value_iteration -and policy_iteration algorithms.""" +states are laid out in a 2-dimensional grid. We also represent a policy +as a dictionary of {state: action} pairs, and a Utility function as a +dictionary of {state: number} pairs. We then define the value_iteration +and policy_iteration algorithms. +""" -from utils import * # noqa +import random +from collections import defaultdict +import numpy as np -class MDP: +from utils import vector_add, orientations, turn_right, turn_left + +class MDP: """A Markov Decision Process, defined by an initial state, transition model, and reward function. We also keep track of a gamma value, for use by algorithms. The transition model is represented somewhat differently from - the text. Instead of P(s' | s, a) being a probability number for each + the text. Instead of P(s' | s, a) being a probability number for each state/state/action triplet, we instead have T(s, a) return a - list of (p, s') pairs. We also keep track of the possible states, - terminal states, and actions for each state. [page 646]""" + list of (p, s') pairs. We also keep track of the possible states, + terminal states, and actions for each state. [Page 646]""" + + def __init__(self, init, actlist, terminals, transitions=None, reward=None, states=None, gamma=0.9): + if not (0 < gamma <= 1): + raise ValueError("An MDP must have 0 < gamma <= 1") + + # collect states from transitions table if not passed. + self.states = states or self.get_states_from_transitions(transitions) - def __init__(self, init, actlist, terminals, gamma=.9): self.init = init - self.actlist = actlist + + if isinstance(actlist, list): + # if actlist is a list, all states have the same actions + self.actlist = actlist + + elif isinstance(actlist, dict): + # if actlist is a dict, different actions for each state + self.actlist = actlist + self.terminals = terminals - if not (0 <= gamma < 1): - raise ValueError("An MDP must have 0 <= gamma < 1") + self.transitions = transitions or {} + if not self.transitions: + print("Warning: Transition table is empty.") + self.gamma = gamma - self.states = set() - self.reward = {} + + self.reward = reward or {s: 0 for s in self.states} + + # self.check_consistency() def R(self, state): - "Return a numeric reward for this state." + """Return a numeric reward for this state.""" + return self.reward[state] def T(self, state, action): - """Transition model. From a state and an action, return a list + """Transition model. From a state and an action, return a list of (probability, result-state) pairs.""" - raise NotImplementedError + + if not self.transitions: + raise ValueError("Transition model is missing") + else: + return self.transitions[state][action] def actions(self, state): - """Set of actions that can be performed in this state. By default, a + """Return a list of actions that can be performed in this state. By default, a fixed list of actions, except for terminal states. Override this method if you need to specialize by state.""" + if state in self.terminals: return [None] else: return self.actlist + def get_states_from_transitions(self, transitions): + if isinstance(transitions, dict): + s1 = set(transitions.keys()) + s2 = set(tr[1] for actions in transitions.values() + for effects in actions.values() + for tr in effects) + return s1.union(s2) + else: + print('Could not retrieve states from transitions') + return None -class GridMDP(MDP): + def check_consistency(self): - """A two-dimensional grid MDP, as in [Figure 17.1]. All you have to do is + # check that all states in transitions are valid + assert set(self.states) == self.get_states_from_transitions(self.transitions) + + # check that init is a valid state + assert self.init in self.states + + # check reward for each state + assert set(self.reward.keys()) == set(self.states) + + # check that all terminals are valid states + assert all(t in self.states for t in self.terminals) + + # check that probability distributions for all actions sum to 1 + for s1, actions in self.transitions.items(): + for a in actions.keys(): + s = 0 + for o in actions[a]: + s += o[0] + assert abs(s - 1) < 0.001 + + +class MDP2(MDP): + """ + Inherits from MDP. Handles terminal states, and transitions to and from terminal states better. + """ + + def __init__(self, init, actlist, terminals, transitions, reward=None, gamma=0.9): + MDP.__init__(self, init, actlist, terminals, transitions, reward, gamma=gamma) + + def T(self, state, action): + if action is None: + return [(0.0, state)] + else: + return self.transitions[state][action] + + +class GridMDP(MDP): + """A two-dimensional grid MDP, as in [Figure 17.1]. All you have to do is specify the grid as a list of lists of rewards; use None for an obstacle - (unreachable state). Also, you should specify the terminal states. + (unreachable state). Also, you should specify the terminal states. An action is an (x, y) unit vector; e.g. (1, 0) means move east.""" def __init__(self, grid, terminals, init=(0, 0), gamma=.9): grid.reverse() # because we want row 0 on bottom, not on top - MDP.__init__(self, init, actlist=orientations, - terminals=terminals, gamma=gamma) - self.grid = grid + reward = {} + states = set() self.rows = len(grid) self.cols = len(grid[0]) + self.grid = grid for x in range(self.cols): for y in range(self.rows): - self.reward[x, y] = grid[y][x] - if grid[y][x] is not None: - self.states.add((x, y)) - - def T(self, state, action): - if action is None: - return [(0.0, state)] - else: + if grid[y][x]: + states.add((x, y)) + reward[(x, y)] = grid[y][x] + self.states = states + actlist = orientations + transitions = {} + for s in states: + transitions[s] = {} + for a in actlist: + transitions[s][a] = self.calculate_T(s, a) + MDP.__init__(self, init, actlist=actlist, + terminals=terminals, transitions=transitions, + reward=reward, states=states, gamma=gamma) + + def calculate_T(self, state, action): + if action: return [(0.8, self.go(state, action)), (0.1, self.go(state, turn_right(action))), (0.1, self.go(state, turn_left(action)))] + else: + return [(0.0, state)] + + def T(self, state, action): + return self.transitions[state][action] if action else [(0.0, state)] def go(self, state, direction): - "Return the state that results from going in this direction." + """Return the state that results from going in this direction.""" + state1 = vector_add(state, direction) - return (state1 if state1 in self.states else state) + return state1 if state1 in self.states else state def to_grid(self, mapping): """Convert a mapping from (x, y) to v into a [[..., v, ...]] grid.""" + return list(reversed([[mapping.get((x, y), None) for x in range(self.cols)] for y in range(self.rows)])) def to_arrows(self, policy): - chars = { - (1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'} - return self.to_grid( - dict([(s, chars[a]) for (s, a) in list(policy.items())])) + chars = {(1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'} + return self.to_grid({s: chars[a] for (s, a) in policy.items()}) + # ______________________________________________________________________________ -Fig[17, 1] = GridMDP([[-0.04, -0.04, -0.04, +1], - [-0.04, None, -0.04, -1], - [-0.04, -0.04, -0.04, -0.04]], - terminals=[(3, 2), (3, 1)]) + +""" [Figure 17.1] +A 4x3 grid environment that presents the agent with a sequential decision problem. +""" + +sequential_decision_environment = GridMDP([[-0.04, -0.04, -0.04, +1], + [-0.04, None, -0.04, -1], + [-0.04, -0.04, -0.04, -0.04]], + terminals=[(3, 2), (3, 1)]) + # ______________________________________________________________________________ def value_iteration(mdp, epsilon=0.001): - "Solving an MDP by value iteration. [Fig. 17.4]" - U1 = dict([(s, 0) for s in mdp.states]) + """Solving an MDP by value iteration. [Figure 17.4]""" + + U1 = {s: 0 for s in mdp.states} R, T, gamma = mdp.R, mdp.T, mdp.gamma while True: U = U1.copy() delta = 0 for s in mdp.states: - U1[s] = R(s) + gamma * max([sum([p * U[s1] for (p, s1) in T(s, a)]) - for a in mdp.actions(s)]) + U1[s] = R(s) + gamma * max(sum(p * U[s1] for (p, s1) in T(s, a)) + for a in mdp.actions(s)) delta = max(delta, abs(U1[s] - U[s])) - if delta < epsilon * (1 - gamma) / gamma: + if delta <= epsilon * (1 - gamma) / gamma: return U def best_policy(mdp, U): """Given an MDP and a utility function U, determine the best policy, - as a mapping from state to action. (Equation 17.4)""" + as a mapping from state to action. [Equation 17.4]""" + pi = {} for s in mdp.states: - pi[s] = argmax( - mdp.actions(s), lambda a: expected_utility(a, s, U, mdp)) + pi[s] = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) return pi def expected_utility(a, s, U, mdp): - "The expected utility of doing a in state s, according to the MDP and U." - return sum([p * U[s1] for (p, s1) in mdp.T(s, a)]) + """The expected utility of doing a in state s, according to the MDP and U.""" + + return sum(p * U[s1] for (p, s1) in mdp.T(s, a)) + # ______________________________________________________________________________ def policy_iteration(mdp): - "Solve an MDP by policy iteration [Fig. 17.7]" - U = dict([(s, 0) for s in mdp.states]) - pi = dict([(s, random.choice(mdp.actions(s))) for s in mdp.states]) + """Solve an MDP by policy iteration [Figure 17.7]""" + + U = {s: 0 for s in mdp.states} + pi = {s: random.choice(mdp.actions(s)) for s in mdp.states} while True: U = policy_evaluation(pi, U, mdp) unchanged = True for s in mdp.states: - a = argmax( - mdp.actions(s), lambda a: expected_utility(a, s, U, mdp)) + a = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) if a != pi[s]: pi[s] = a unchanged = False @@ -155,25 +255,238 @@ def policy_iteration(mdp): def policy_evaluation(pi, U, mdp, k=20): """Return an updated utility mapping U from each state in the MDP to its utility, using an approximation (modified policy iteration).""" + R, T, gamma = mdp.R, mdp.T, mdp.gamma for i in range(k): for s in mdp.states: - U[s] = R(s) + gamma * sum([p * U[s1] for (p, s1) in T(s, pi[s])]) + U[s] = R(s) + gamma * sum(p * U[s1] for (p, s1) in T(s, pi[s])) return U + +class POMDP(MDP): + """A Partially Observable Markov Decision Process, defined by + a transition model P(s'|s,a), actions A(s), a reward function R(s), + and a sensor model P(e|s). We also keep track of a gamma value, + for use by algorithms. The transition and the sensor models + are defined as matrices. We also keep track of the possible states + and actions for each state. [Page 659].""" + + def __init__(self, actions, transitions=None, evidences=None, rewards=None, states=None, gamma=0.95): + """Initialize variables of the pomdp""" + + if not (0 < gamma <= 1): + raise ValueError('A POMDP must have 0 < gamma <= 1') + + self.states = states + self.actions = actions + + # transition model cannot be undefined + self.t_prob = transitions or {} + if not self.t_prob: + print('Warning: Transition model is undefined') + + # sensor model cannot be undefined + self.e_prob = evidences or {} + if not self.e_prob: + print('Warning: Sensor model is undefined') + + self.gamma = gamma + self.rewards = rewards + + def remove_dominated_plans(self, input_values): + """ + Remove dominated plans. + This method finds all the lines contributing to the + upper surface and removes those which don't. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [values[0]] + y1_max = max(val[1] for val in values) + tgt = values[0] + prev_b = 0 + prev_ix = 0 + while tgt[1] != y1_max: + min_b = 1 + min_ix = 0 + for i in range(prev_ix + 1, len(values)): + if values[i][0] - tgt[0] + tgt[1] - values[i][1] != 0: + trans_b = (values[i][0] - tgt[0]) / (values[i][0] - tgt[0] + tgt[1] - values[i][1]) + if 0 <= trans_b <= 1 and trans_b > prev_b and trans_b < min_b: + min_b = trans_b + min_ix = i + prev_b = min_b + prev_ix = min_ix + tgt = values[min_ix] + best.append(tgt) + + return self.generate_mapping(best, input_values) + + def remove_dominated_plans_fast(self, input_values): + """ + Remove dominated plans using approximations. + Resamples the upper boundary at intervals of 100 and + finds the maximum values at these points. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [] + sr = 100 + for i in range(sr + 1): + x = i / float(sr) + maximum = (values[0][1] - values[0][0]) * x + values[0][0] + tgt = values[0] + for value in values: + val = (value[1] - value[0]) * x + value[0] + if val > maximum: + maximum = val + tgt = value + + if all(any(tgt != v) for v in best): + best.append(np.array(tgt)) + + return self.generate_mapping(best, input_values) + + def generate_mapping(self, best, input_values): + """Generate mappings after removing dominated plans""" + + mapping = defaultdict(list) + for value in best: + for action in input_values: + if any(all(value == v) for v in input_values[action]): + mapping[action].append(value) + + return mapping + + def max_difference(self, U1, U2): + """Find maximum difference between two utility mappings""" + + for k, v in U1.items(): + sum1 = 0 + for element in U1[k]: + sum1 += sum(element) + sum2 = 0 + for element in U2[k]: + sum2 += sum(element) + return abs(sum1 - sum2) + + +class Matrix: + """Matrix operations class""" + + @staticmethod + def add(A, B): + """Add two matrices A and B""" + + res = [] + for i in range(len(A)): + row = [] + for j in range(len(A[0])): + row.append(A[i][j] + B[i][j]) + res.append(row) + return res + + @staticmethod + def scalar_multiply(a, B): + """Multiply scalar a to matrix B""" + + for i in range(len(B)): + for j in range(len(B[0])): + B[i][j] = a * B[i][j] + return B + + @staticmethod + def multiply(A, B): + """Multiply two matrices A and B element-wise""" + + matrix = [] + for i in range(len(B)): + row = [] + for j in range(len(B[0])): + row.append(B[i][j] * A[j][i]) + matrix.append(row) + + return matrix + + @staticmethod + def matmul(A, B): + """Inner-product of two matrices""" + + return [[sum(ele_a * ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A] + + @staticmethod + def transpose(A): + """Transpose a matrix""" + + return [list(i) for i in zip(*A)] + + +def pomdp_value_iteration(pomdp, epsilon=0.1): + """Solving a POMDP by value iteration.""" + + U = {'': [[0] * len(pomdp.states)]} + count = 0 + while True: + count += 1 + prev_U = U + values = [val for action in U for val in U[action]] + value_matxs = [] + for i in values: + for j in values: + value_matxs.append([i, j]) + + U1 = defaultdict(list) + for action in pomdp.actions: + for u in value_matxs: + u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], + Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), + [[1], [1]]) + u1 = Matrix.add(Matrix.scalar_multiply(pomdp.gamma, Matrix.transpose(u1)), [pomdp.rewards[int(action)]]) + U1[action].append(u1[0]) + + U = pomdp.remove_dominated_plans_fast(U1) + # replace with U = pomdp.remove_dominated_plans(U1) for accurate calculations + + if count > 10: + if pomdp.max_difference(U, prev_U) < epsilon * (1 - pomdp.gamma) / pomdp.gamma: + return U + + __doc__ += """ ->>> pi = best_policy(Fig[17,1], value_iteration(Fig[17,1], .01)) +>>> pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .01)) ->>> Fig[17,1].to_arrows(pi) +>>> sequential_decision_environment.to_arrows(pi) [['>', '>', '>', '.'], ['^', None, '^', '.'], ['^', '>', '^', '<']] ->>> print_table(Fig[17,1].to_arrows(pi)) +>>> from utils import print_table + +>>> print_table(sequential_decision_environment.to_arrows(pi)) > > > . ^ None ^ . ^ > ^ < ->>> print_table(Fig[17,1].to_arrows(policy_iteration(Fig[17,1]))) +>>> print_table(sequential_decision_environment.to_arrows(policy_iteration(sequential_decision_environment))) > > > . ^ None ^ . ^ > ^ < +""" # noqa + +""" +s = { 'a' : { 'plan1' : [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], + 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }, + } """ diff --git a/mdp4e.py b/mdp4e.py new file mode 100644 index 000000000..f8871bdc9 --- /dev/null +++ b/mdp4e.py @@ -0,0 +1,516 @@ +""" +Markov Decision Processes (Chapter 16) + +First we define an MDP, and the special case of a GridMDP, in which +states are laid out in a 2-dimensional grid. We also represent a policy +as a dictionary of {state: action} pairs, and a Utility function as a +dictionary of {state: number} pairs. We then define the value_iteration +and policy_iteration algorithms. +""" + +import random +from collections import defaultdict + +import numpy as np + +from utils4e import vector_add, orientations, turn_right, turn_left + + +class MDP: + """A Markov Decision Process, defined by an initial state, transition model, + and reward function. We also keep track of a gamma value, for use by + algorithms. The transition model is represented somewhat differently from + the text. Instead of P(s' | s, a) being a probability number for each + state/state/action triplet, we instead have T(s, a) return a + list of (p, s') pairs. We also keep track of the possible states, + terminal states, and actions for each state. [Page 646]""" + + def __init__(self, init, actlist, terminals, transitions=None, reward=None, states=None, gamma=0.9): + if not (0 < gamma <= 1): + raise ValueError("An MDP must have 0 < gamma <= 1") + + # collect states from transitions table if not passed. + self.states = states or self.get_states_from_transitions(transitions) + + self.init = init + + if isinstance(actlist, list): + # if actlist is a list, all states have the same actions + self.actlist = actlist + + elif isinstance(actlist, dict): + # if actlist is a dict, different actions for each state + self.actlist = actlist + + self.terminals = terminals + self.transitions = transitions or {} + if not self.transitions: + print("Warning: Transition table is empty.") + + self.gamma = gamma + + self.reward = reward or {s: 0 for s in self.states} + + # self.check_consistency() + + def R(self, state): + """Return a numeric reward for this state.""" + + return self.reward[state] + + def T(self, state, action): + """Transition model. From a state and an action, return a list + of (probability, result-state) pairs.""" + + if not self.transitions: + raise ValueError("Transition model is missing") + else: + return self.transitions[state][action] + + def actions(self, state): + """Return a list of actions that can be performed in this state. By default, a + fixed list of actions, except for terminal states. Override this + method if you need to specialize by state.""" + + if state in self.terminals: + return [None] + else: + return self.actlist + + def get_states_from_transitions(self, transitions): + if isinstance(transitions, dict): + s1 = set(transitions.keys()) + s2 = set(tr[1] for actions in transitions.values() + for effects in actions.values() + for tr in effects) + return s1.union(s2) + else: + print('Could not retrieve states from transitions') + return None + + def check_consistency(self): + + # check that all states in transitions are valid + assert set(self.states) == self.get_states_from_transitions(self.transitions) + + # check that init is a valid state + assert self.init in self.states + + # check reward for each state + assert set(self.reward.keys()) == set(self.states) + + # check that all terminals are valid states + assert all(t in self.states for t in self.terminals) + + # check that probability distributions for all actions sum to 1 + for s1, actions in self.transitions.items(): + for a in actions.keys(): + s = 0 + for o in actions[a]: + s += o[0] + assert abs(s - 1) < 0.001 + + +class MDP2(MDP): + """ + Inherits from MDP. Handles terminal states, and transitions to and from terminal states better. + """ + + def __init__(self, init, actlist, terminals, transitions, reward=None, gamma=0.9): + MDP.__init__(self, init, actlist, terminals, transitions, reward, gamma=gamma) + + def T(self, state, action): + if action is None: + return [(0.0, state)] + else: + return self.transitions[state][action] + + +class GridMDP(MDP): + """A two-dimensional grid MDP, as in [Figure 16.1]. All you have to do is + specify the grid as a list of lists of rewards; use None for an obstacle + (unreachable state). Also, you should specify the terminal states. + An action is an (x, y) unit vector; e.g. (1, 0) means move east.""" + + def __init__(self, grid, terminals, init=(0, 0), gamma=.9): + grid.reverse() # because we want row 0 on bottom, not on top + reward = {} + states = set() + self.rows = len(grid) + self.cols = len(grid[0]) + self.grid = grid + for x in range(self.cols): + for y in range(self.rows): + if grid[y][x]: + states.add((x, y)) + reward[(x, y)] = grid[y][x] + self.states = states + actlist = orientations + transitions = {} + for s in states: + transitions[s] = {} + for a in actlist: + transitions[s][a] = self.calculate_T(s, a) + MDP.__init__(self, init, actlist=actlist, + terminals=terminals, transitions=transitions, + reward=reward, states=states, gamma=gamma) + + def calculate_T(self, state, action): + if action: + return [(0.8, self.go(state, action)), + (0.1, self.go(state, turn_right(action))), + (0.1, self.go(state, turn_left(action)))] + else: + return [(0.0, state)] + + def T(self, state, action): + return self.transitions[state][action] if action else [(0.0, state)] + + def go(self, state, direction): + """Return the state that results from going in this direction.""" + + state1 = tuple(vector_add(state, direction)) + return state1 if state1 in self.states else state + + def to_grid(self, mapping): + """Convert a mapping from (x, y) to v into a [[..., v, ...]] grid.""" + + return list(reversed([[mapping.get((x, y), None) + for x in range(self.cols)] + for y in range(self.rows)])) + + def to_arrows(self, policy): + chars = {(1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'} + return self.to_grid({s: chars[a] for (s, a) in policy.items()}) + + +# ______________________________________________________________________________ + + +""" [Figure 16.1] +A 4x3 grid environment that presents the agent with a sequential decision problem. +""" + +sequential_decision_environment = GridMDP([[-0.04, -0.04, -0.04, +1], + [-0.04, None, -0.04, -1], + [-0.04, -0.04, -0.04, -0.04]], + terminals=[(3, 2), (3, 1)]) + + +# ______________________________________________________________________________ +# 16.1.3 The Bellman equation for utilities + + +def q_value(mdp, s, a, U): + if not a: + return mdp.R(s) + res = 0 + for p, s_prime in mdp.T(s, a): + res += p * (mdp.R(s) + mdp.gamma * U[s_prime]) + return res + + +# TODO: DDN in figure 16.4 and 16.5 + +# ______________________________________________________________________________ +# 16.2 Algorithms for MDPs +# 16.2.1 Value Iteration + + +def value_iteration(mdp, epsilon=0.001): + """Solving an MDP by value iteration. [Figure 16.6]""" + + U1 = {s: 0 for s in mdp.states} + R, T, gamma = mdp.R, mdp.T, mdp.gamma + while True: + U = U1.copy() + delta = 0 + for s in mdp.states: + # U1[s] = R(s) + gamma * max(sum(p * U[s1] for (p, s1) in T(s, a)) + # for a in mdp.actions(s)) + U1[s] = max(q_value(mdp, s, a, U) for a in mdp.actions(s)) + delta = max(delta, abs(U1[s] - U[s])) + if delta <= epsilon * (1 - gamma) / gamma: + return U + + +# ______________________________________________________________________________ +# 16.2.2 Policy Iteration + + +def best_policy(mdp, U): + """Given an MDP and a utility function U, determine the best policy, + as a mapping from state to action.""" + + pi = {} + for s in mdp.states: + pi[s] = max(mdp.actions(s), key=lambda a: q_value(mdp, s, a, U)) + return pi + + +def expected_utility(a, s, U, mdp): + """The expected utility of doing a in state s, according to the MDP and U.""" + + return sum(p * U[s1] for (p, s1) in mdp.T(s, a)) + + +def policy_iteration(mdp): + """Solve an MDP by policy iteration [Figure 17.7]""" + + U = {s: 0 for s in mdp.states} + pi = {s: random.choice(mdp.actions(s)) for s in mdp.states} + while True: + U = policy_evaluation(pi, U, mdp) + unchanged = True + for s in mdp.states: + a_star = max(mdp.actions(s), key=lambda a: q_value(mdp, s, a, U)) + # a = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) + if q_value(mdp, s, a_star, U) > q_value(mdp, s, pi[s], U): + pi[s] = a_star + unchanged = False + if unchanged: + return pi + + +def policy_evaluation(pi, U, mdp, k=20): + """Return an updated utility mapping U from each state in the MDP to its + utility, using an approximation (modified policy iteration).""" + + R, T, gamma = mdp.R, mdp.T, mdp.gamma + for i in range(k): + for s in mdp.states: + U[s] = R(s) + gamma * sum(p * U[s1] for (p, s1) in T(s, pi[s])) + return U + + +# ___________________________________________________________________ +# 16.4 Partially Observed MDPs + + +class POMDP(MDP): + """A Partially Observable Markov Decision Process, defined by + a transition model P(s'|s,a), actions A(s), a reward function R(s), + and a sensor model P(e|s). We also keep track of a gamma value, + for use by algorithms. The transition and the sensor models + are defined as matrices. We also keep track of the possible states + and actions for each state. [Page 659].""" + + def __init__(self, actions, transitions=None, evidences=None, rewards=None, states=None, gamma=0.95): + """Initialize variables of the pomdp""" + + if not (0 < gamma <= 1): + raise ValueError('A POMDP must have 0 < gamma <= 1') + + self.states = states + self.actions = actions + + # transition model cannot be undefined + self.t_prob = transitions or {} + if not self.t_prob: + print('Warning: Transition model is undefined') + + # sensor model cannot be undefined + self.e_prob = evidences or {} + if not self.e_prob: + print('Warning: Sensor model is undefined') + + self.gamma = gamma + self.rewards = rewards + + def remove_dominated_plans(self, input_values): + """ + Remove dominated plans. + This method finds all the lines contributing to the + upper surface and removes those which don't. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [values[0]] + y1_max = max(val[1] for val in values) + tgt = values[0] + prev_b = 0 + prev_ix = 0 + while tgt[1] != y1_max: + min_b = 1 + min_ix = 0 + for i in range(prev_ix + 1, len(values)): + if values[i][0] - tgt[0] + tgt[1] - values[i][1] != 0: + trans_b = (values[i][0] - tgt[0]) / (values[i][0] - tgt[0] + tgt[1] - values[i][1]) + if 0 <= trans_b <= 1 and trans_b > prev_b and trans_b < min_b: + min_b = trans_b + min_ix = i + prev_b = min_b + prev_ix = min_ix + tgt = values[min_ix] + best.append(tgt) + + return self.generate_mapping(best, input_values) + + def remove_dominated_plans_fast(self, input_values): + """ + Remove dominated plans using approximations. + Resamples the upper boundary at intervals of 100 and + finds the maximum values at these points. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [] + sr = 100 + for i in range(sr + 1): + x = i / float(sr) + maximum = (values[0][1] - values[0][0]) * x + values[0][0] + tgt = values[0] + for value in values: + val = (value[1] - value[0]) * x + value[0] + if val > maximum: + maximum = val + tgt = value + + if all(any(tgt != v) for v in best): + best.append(np.array(tgt)) + + return self.generate_mapping(best, input_values) + + def generate_mapping(self, best, input_values): + """Generate mappings after removing dominated plans""" + + mapping = defaultdict(list) + for value in best: + for action in input_values: + if any(all(value == v) for v in input_values[action]): + mapping[action].append(value) + + return mapping + + def max_difference(self, U1, U2): + """Find maximum difference between two utility mappings""" + + for k, v in U1.items(): + sum1 = 0 + for element in U1[k]: + sum1 += sum(element) + sum2 = 0 + for element in U2[k]: + sum2 += sum(element) + return abs(sum1 - sum2) + + +class Matrix: + """Matrix operations class""" + + @staticmethod + def add(A, B): + """Add two matrices A and B""" + + res = [] + for i in range(len(A)): + row = [] + for j in range(len(A[0])): + row.append(A[i][j] + B[i][j]) + res.append(row) + return res + + @staticmethod + def scalar_multiply(a, B): + """Multiply scalar a to matrix B""" + + for i in range(len(B)): + for j in range(len(B[0])): + B[i][j] = a * B[i][j] + return B + + @staticmethod + def multiply(A, B): + """Multiply two matrices A and B element-wise""" + + matrix = [] + for i in range(len(B)): + row = [] + for j in range(len(B[0])): + row.append(B[i][j] * A[j][i]) + matrix.append(row) + + return matrix + + @staticmethod + def matmul(A, B): + """Inner-product of two matrices""" + + return [[sum(ele_a * ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A] + + @staticmethod + def transpose(A): + """Transpose a matrix""" + + return [list(i) for i in zip(*A)] + + +def pomdp_value_iteration(pomdp, epsilon=0.1): + """Solving a POMDP by value iteration.""" + + U = {'': [[0] * len(pomdp.states)]} + count = 0 + while True: + count += 1 + prev_U = U + values = [val for action in U for val in U[action]] + value_matxs = [] + for i in values: + for j in values: + value_matxs.append([i, j]) + + U1 = defaultdict(list) + for action in pomdp.actions: + for u in value_matxs: + u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], + Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), + [[1], [1]]) + u1 = Matrix.add(Matrix.scalar_multiply(pomdp.gamma, Matrix.transpose(u1)), [pomdp.rewards[int(action)]]) + U1[action].append(u1[0]) + + U = pomdp.remove_dominated_plans_fast(U1) + # replace with U = pomdp.remove_dominated_plans(U1) for accurate calculations + + if count > 10: + if pomdp.max_difference(U, prev_U) < epsilon * (1 - pomdp.gamma) / pomdp.gamma: + return U + + +__doc__ += """ +>>> pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .01)) + +>>> sequential_decision_environment.to_arrows(pi) +[['>', '>', '>', '.'], ['^', None, '^', '.'], ['^', '>', '^', '<']] + +>>> from utils import print_table + +>>> print_table(sequential_decision_environment.to_arrows(pi)) +> > > . +^ None ^ . +^ > ^ < + +>>> print_table(sequential_decision_environment.to_arrows(policy_iteration(sequential_decision_environment))) +> > > . +^ None ^ . +^ > ^ < +""" # noqa + +""" +s = { 'a' : { 'plan1' : [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], + 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }, + } +""" diff --git a/mdp_apps.ipynb b/mdp_apps.ipynb new file mode 100644 index 000000000..da3ae7b06 --- /dev/null +++ b/mdp_apps.ipynb @@ -0,0 +1,1825 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# APPLICATIONS OF MARKOV DECISION PROCESSES\n", + "---\n", + "In this notebook we will take a look at some indicative applications of markov decision processes. \n", + "We will cover content from [`mdp.py`](https://github.com/aimacode/aima-python/blob/master/mdp.py), for **Chapter 17 Making Complex Decisions** of Stuart Russel's and Peter Norvig's book [*Artificial Intelligence: A Modern Approach*](http://aima.cs.berkeley.edu/).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from mdp import *\n", + "from notebook import psource, pseudocode" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "- Simple MDP\n", + " - State dependent reward function\n", + " - State and action dependent reward function\n", + " - State, action and next state dependent reward function\n", + "- Grid MDP\n", + " - Pathfinding problem\n", + "- POMDP\n", + " - Two state POMDP" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## SIMPLE MDP\n", + "---\n", + "### State dependent reward function\n", + "\n", + "Markov Decision Processes are formally described as processes that follow the Markov property which states that \"The future is independent of the past given the present\". \n", + "MDPs formally describe environments for reinforcement learning and we assume that the environment is *fully observable*. \n", + "Let us take a toy example MDP and solve it using the functions in `mdp.py`.\n", + "This is a simple example adapted from a [similar problem](http://www0.cs.ucl.ac.uk/staff/D.Silver/web/Teaching_files/MDP.pdf) by Dr. David Silver, tweaked to fit the limitations of the current functions.\n", + "\n", + "\n", + "Let's say you're a student attending lectures in a university.\n", + "There are three lectures you need to attend on a given day.\n", + "
\n", + "Attending the first lecture gives you 4 points of reward.\n", + "After the first lecture, you have a 0.6 probability to continue into the second one, yielding 6 more points of reward.\n", + "
\n", + "But, with a probability of 0.4, you get distracted and start using Facebook instead and get a reward of -1.\n", + "From then onwards, you really can't let go of Facebook and there's just a 0.1 probability that you will concentrate back on the lecture.\n", + "
\n", + "After the second lecture, you have an equal chance of attending the next lecture or just falling asleep.\n", + "Falling asleep is the terminal state and yields you no reward, but continuing on to the final lecture gives you a big reward of 10 points.\n", + "
\n", + "From there on, you have a 40% chance of going to study and reach the terminal state, \n", + "but a 60% chance of going to the pub with your friends instead. \n", + "You end up drunk and don't know which lecture to attend, so you go to one of the lectures according to the probabilities given above.\n", + "
\n", + "We now have an outline of our stochastic environment and we need to maximize our reward by solving this MDP.\n", + "
\n", + "
\n", + "We first have to define our Transition Matrix as a nested dictionary to fit the requirements of the MDP class." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "t = {\n", + " 'leisure': {\n", + " 'facebook': {'leisure':0.9, 'class1':0.1},\n", + " 'quit': {'leisure':0.1, 'class1':0.9},\n", + " 'study': {},\n", + " 'sleep': {},\n", + " 'pub': {}\n", + " },\n", + " 'class1': {\n", + " 'study': {'class2':0.6, 'leisure':0.4},\n", + " 'facebook': {'class2':0.4, 'leisure':0.6},\n", + " 'quit': {},\n", + " 'sleep': {},\n", + " 'pub': {}\n", + " },\n", + " 'class2': {\n", + " 'study': {'class3':0.5, 'end':0.5},\n", + " 'sleep': {'end':0.5, 'class3':0.5},\n", + " 'facebook': {},\n", + " 'quit': {},\n", + " 'pub': {},\n", + " },\n", + " 'class3': {\n", + " 'study': {'end':0.6, 'class1':0.08, 'class2':0.16, 'class3':0.16},\n", + " 'pub': {'end':0.4, 'class1':0.12, 'class2':0.24, 'class3':0.24},\n", + " 'facebook': {},\n", + " 'quit': {},\n", + " 'sleep': {}\n", + " },\n", + " 'end': {}\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now need to define the reward for each state." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "rewards = {\n", + " 'class1': 4,\n", + " 'class2': 6,\n", + " 'class3': 10,\n", + " 'leisure': -1,\n", + " 'end': 0\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This MDP has only one terminal state." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "terminals = ['end']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now set the initial state to Class 1." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "init = 'class1'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will write a CustomMDP class to extend the MDP class for the problem at hand. \n", + "This class will implement the `T` method to implement the transition model. This is the exact same class as given in [`mdp.ipynb`](https://github.com/aimacode/aima-python/blob/master/mdp.ipynb#MDP)." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class CustomMDP(MDP):\n", + "\n", + " def __init__(self, transition_matrix, rewards, terminals, init, gamma=.9):\n", + " # All possible actions.\n", + " actlist = []\n", + " for state in transition_matrix.keys():\n", + " actlist.extend(transition_matrix[state])\n", + " actlist = list(set(actlist))\n", + " print(actlist)\n", + "\n", + " MDP.__init__(self, init, actlist, terminals=terminals, gamma=gamma)\n", + " self.t = transition_matrix\n", + " self.reward = rewards\n", + " for state in self.t:\n", + " self.states.add(state)\n", + "\n", + " def T(self, state, action):\n", + " if action is None:\n", + " return [(0.0, state)]\n", + " else: \n", + " return [(prob, new_state) for new_state, prob in self.t[state][action].items()]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now need an instance of this class." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['quit', 'sleep', 'study', 'pub', 'facebook']\n" + ] + } + ], + "source": [ + "mdp = CustomMDP(t, rewards, terminals, init, gamma=.9)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The utility of each state can be found by `value_iteration`." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'class1': 16.90340650279542,\n", + " 'class2': 14.597383430869879,\n", + " 'class3': 19.10533144728953,\n", + " 'end': 0.0,\n", + " 'leisure': 13.946891353066082}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "value_iteration(mdp)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we can compute the utility values, we can find the best policy." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pi = best_policy(mdp, value_iteration(mdp, .01))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pi` stores the best action for each state." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'class2': 'sleep', 'class3': 'pub', 'end': None, 'class1': 'study', 'leisure': 'quit'}\n" + ] + } + ], + "source": [ + "print(pi)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can confirm that this is the best policy by verifying this result against `policy_iteration`." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'class1': 'study',\n", + " 'class2': 'sleep',\n", + " 'class3': 'pub',\n", + " 'end': None,\n", + " 'leisure': 'quit'}" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "policy_iteration(mdp)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "Everything looks perfect, but let us look at another possibility for an MDP.\n", + "
\n", + "Till now we have only dealt with rewards that the agent gets while it is **on** a particular state.\n", + "What if we want to have different rewards for a state depending on the action that the agent takes next. \n", + "The agent gets the reward _during its transition_ to the next state.\n", + "
\n", + "For the sake of clarity, we will call this the _transition reward_ and we will call this kind of MDP a _dynamic_ MDP. \n", + "This is not a conventional term, we just use it to minimize confusion between the two.\n", + "
\n", + "This next section deals with how to create and solve a dynamic MDP." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### State and action dependent reward function\n", + "Let us consider a very similar problem, but this time, we do not have rewards _on_ states, \n", + "instead, we have rewards on the transitions between states. \n", + "This state diagram will make it clearer.\n", + "\n", + "\n", + "A very similar scenario as the previous problem, but we have different rewards for the same state depending on the action taken.\n", + "
\n", + "To deal with this, we just need to change the `R` method of the `MDP` class, but to prevent confusion, we will write a new similar class `DMDP`." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class DMDP:\n", + "\n", + " \"\"\"A Markov Decision Process, defined by an initial state, transition model,\n", + " and reward model. We also keep track of a gamma value, for use by\n", + " algorithms. The transition model is represented somewhat differently from\n", + " the text. Instead of P(s' | s, a) being a probability number for each\n", + " state/state/action triplet, we instead have T(s, a) return a\n", + " list of (p, s') pairs. The reward function is very similar.\n", + " We also keep track of the possible states,\n", + " terminal states, and actions for each state.\"\"\"\n", + "\n", + " def __init__(self, init, actlist, terminals, transitions={}, rewards={}, states=None, gamma=.9):\n", + " if not (0 < gamma <= 1):\n", + " raise ValueError(\"An MDP must have 0 < gamma <= 1\")\n", + "\n", + " if states:\n", + " self.states = states\n", + " else:\n", + " self.states = set()\n", + " self.init = init\n", + " self.actlist = actlist\n", + " self.terminals = terminals\n", + " self.transitions = transitions\n", + " self.rewards = rewards\n", + " self.gamma = gamma\n", + "\n", + " def R(self, state, action):\n", + " \"\"\"Return a numeric reward for this state and this action.\"\"\"\n", + " if (self.rewards == {}):\n", + " raise ValueError('Reward model is missing')\n", + " else:\n", + " return self.rewards[state][action]\n", + "\n", + " def T(self, state, action):\n", + " \"\"\"Transition model. From a state and an action, return a list\n", + " of (probability, result-state) pairs.\"\"\"\n", + " if(self.transitions == {}):\n", + " raise ValueError(\"Transition model is missing\")\n", + " else:\n", + " return self.transitions[state][action]\n", + "\n", + " def actions(self, state):\n", + " \"\"\"Set of actions that can be performed in this state. By default, a\n", + " fixed list of actions, except for terminal states. Override this\n", + " method if you need to specialize by state.\"\"\"\n", + " if state in self.terminals:\n", + " return [None]\n", + " else:\n", + " return self.actlist" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The transition model will be the same" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "t = {\n", + " 'leisure': {\n", + " 'facebook': {'leisure':0.9, 'class1':0.1},\n", + " 'quit': {'leisure':0.1, 'class1':0.9},\n", + " 'study': {},\n", + " 'sleep': {},\n", + " 'pub': {}\n", + " },\n", + " 'class1': {\n", + " 'study': {'class2':0.6, 'leisure':0.4},\n", + " 'facebook': {'class2':0.4, 'leisure':0.6},\n", + " 'quit': {},\n", + " 'sleep': {},\n", + " 'pub': {}\n", + " },\n", + " 'class2': {\n", + " 'study': {'class3':0.5, 'end':0.5},\n", + " 'sleep': {'end':0.5, 'class3':0.5},\n", + " 'facebook': {},\n", + " 'quit': {},\n", + " 'pub': {},\n", + " },\n", + " 'class3': {\n", + " 'study': {'end':0.6, 'class1':0.08, 'class2':0.16, 'class3':0.16},\n", + " 'pub': {'end':0.4, 'class1':0.12, 'class2':0.24, 'class3':0.24},\n", + " 'facebook': {},\n", + " 'quit': {},\n", + " 'sleep': {}\n", + " },\n", + " 'end': {}\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The reward model will be a dictionary very similar to the transition dictionary with a reward for every action for every state." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "r = {\n", + " 'leisure': {\n", + " 'facebook':-1,\n", + " 'quit':0,\n", + " 'study':0,\n", + " 'sleep':0,\n", + " 'pub':0\n", + " },\n", + " 'class1': {\n", + " 'study':-2,\n", + " 'facebook':-1,\n", + " 'quit':0,\n", + " 'sleep':0,\n", + " 'pub':0\n", + " },\n", + " 'class2': {\n", + " 'study':-2,\n", + " 'sleep':0,\n", + " 'facebook':0,\n", + " 'quit':0,\n", + " 'pub':0\n", + " },\n", + " 'class3': {\n", + " 'study':10,\n", + " 'pub':1,\n", + " 'facebook':0,\n", + " 'quit':0,\n", + " 'sleep':0\n", + " },\n", + " 'end': {\n", + " 'study':0,\n", + " 'pub':0,\n", + " 'facebook':0,\n", + " 'quit':0,\n", + " 'sleep':0\n", + " }\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The MDP has only one terminal state" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "terminals = ['end']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now set the initial state to Class 1." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "init = 'class1'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will write a CustomDMDP class to extend the DMDP class for the problem at hand.\n", + "This class will implement everything that the previous CustomMDP class implements along with a new reward model." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class CustomDMDP(DMDP):\n", + " \n", + " def __init__(self, transition_matrix, rewards, terminals, init, gamma=.9):\n", + " actlist = []\n", + " for state in transition_matrix.keys():\n", + " actlist.extend(transition_matrix[state])\n", + " actlist = list(set(actlist))\n", + " print(actlist)\n", + " \n", + " DMDP.__init__(self, init, actlist, terminals=terminals, gamma=gamma)\n", + " self.t = transition_matrix\n", + " self.rewards = rewards\n", + " for state in self.t:\n", + " self.states.add(state)\n", + " \n", + " \n", + " def T(self, state, action):\n", + " if action is None:\n", + " return [(0.0, state)]\n", + " else:\n", + " return [(prob, new_state) for new_state, prob in self.t[state][action].items()]\n", + " \n", + " def R(self, state, action):\n", + " if action is None:\n", + " return 0\n", + " else:\n", + " return self.rewards[state][action]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "One thing we haven't thought about yet is that the `value_iteration` algorithm won't work now that the reward model is changed.\n", + "It will be quite similar to the one we currently have nonetheless." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Bellman update equation now is defined as follows\n", + "\n", + "$$U(s)=\\max_{a\\epsilon A(s)}\\bigg[R(s, a) + \\gamma\\sum_{s'}P(s'\\ |\\ s,a)U(s')\\bigg]$$\n", + "\n", + "It is not difficult to see that the update equation we have been using till now is just a special case of this more generalized equation. \n", + "We also need to max over the reward function now as the reward function is action dependent as well.\n", + "
\n", + "We will use this to write a function to carry out value iteration, very similar to the one we are familiar with." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def value_iteration_dmdp(dmdp, epsilon=0.001):\n", + " U1 = {s: 0 for s in dmdp.states}\n", + " R, T, gamma = dmdp.R, dmdp.T, dmdp.gamma\n", + " while True:\n", + " U = U1.copy()\n", + " delta = 0\n", + " for s in dmdp.states:\n", + " U1[s] = max([(R(s, a) + gamma*sum([(p*U[s1]) for (p, s1) in T(s, a)])) for a in dmdp.actions(s)])\n", + " delta = max(delta, abs(U1[s] - U[s]))\n", + " if delta < epsilon * (1 - gamma) / gamma:\n", + " return U" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We're all set.\n", + "Let's instantiate our class." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['quit', 'sleep', 'study', 'pub', 'facebook']\n" + ] + } + ], + "source": [ + "dmdp = CustomDMDP(t, r, terminals, init, gamma=.9)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Calculate utility values by calling `value_iteration_dmdp`." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'class1': 2.0756895004431364,\n", + " 'class2': 5.772550326127298,\n", + " 'class3': 12.827904448229472,\n", + " 'end': 0.0,\n", + " 'leisure': 1.8474896554396596}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "value_iteration_dmdp(dmdp)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These are the expected utility values for our new MDP.\n", + "
\n", + "As you might have guessed, we cannot use the old `best_policy` function to find the best policy.\n", + "So we will write our own.\n", + "But, before that we need a helper function to calculate the expected utility value given a state and an action." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def expected_utility_dmdp(a, s, U, dmdp):\n", + " return dmdp.R(s, a) + dmdp.gamma*sum([(p*U[s1]) for (p, s1) in dmdp.T(s, a)])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we write our modified `best_policy` function." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from utils import argmax\n", + "def best_policy_dmdp(dmdp, U):\n", + " pi = {}\n", + " for s in dmdp.states:\n", + " pi[s] = argmax(dmdp.actions(s), key=lambda a: expected_utility_dmdp(a, s, U, dmdp))\n", + " return pi" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Find the best policy." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'class2': 'sleep', 'class3': 'study', 'end': None, 'class1': 'facebook', 'leisure': 'quit'}\n" + ] + } + ], + "source": [ + "pi = best_policy_dmdp(dmdp, value_iteration_dmdp(dmdp, .01))\n", + "print(pi)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From this, we can infer that `value_iteration_dmdp` tries to minimize the negative reward. \n", + "Since we don't have rewards for states now, the algorithm takes the action that would try to avoid getting negative rewards and take the lesser of two evils if all rewards are negative.\n", + "You might also want to have state rewards alongside transition rewards. \n", + "Perhaps you can do that yourself now that the difficult part has been done.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### State, action and next-state dependent reward function\n", + "\n", + "For truly stochastic environments, \n", + "we have noticed that taking an action from a particular state doesn't always do what we want it to. \n", + "Instead, for every action taken from a particular state, \n", + "it might be possible to reach a different state each time depending on the transition probabilities. \n", + "What if we want different rewards for each state, action and next-state triplet? \n", + "Mathematically, we now want a reward function of the form R(s, a, s') for our MDP. \n", + "This section shows how we can tweak the MDP class to achieve this.\n", + "
\n", + "\n", + "Let's now take a different problem statement. \n", + "The one we are working with is a bit too simple.\n", + "Consider a taxi that serves three adjacent towns A, B, and C.\n", + "Each time the taxi discharges a passenger, the driver must choose from three possible actions:\n", + "1. Cruise the streets looking for a passenger.\n", + "2. Go to the nearest taxi stand.\n", + "3. Wait for a radio call from the dispatcher with instructions.\n", + "
\n", + "Subject to the constraint that the taxi driver cannot do the third action in town B because of distance and poor reception.\n", + "\n", + "Let's model our MDP.\n", + "
\n", + "The MDP has three states, namely A, B and C.\n", + "
\n", + "It has three actions, namely 1, 2 and 3.\n", + "
\n", + "Action sets:\n", + "
\n", + "$K_{a}$ = {1, 2, 3}\n", + "
\n", + "$K_{b}$ = {1, 2}\n", + "
\n", + "$K_{c}$ = {1, 2, 3}\n", + "
\n", + "\n", + "We have the following transition probability matrices:\n", + "
\n", + "
\n", + "Action 1: Cruising streets \n", + "
\n", + "$\\\\\n", + " P^{1} = \n", + " \\left[ {\\begin{array}{ccc}\n", + " \\frac{1}{2} & \\frac{1}{4} & \\frac{1}{4} \\\\\n", + " \\frac{1}{2} & 0 & \\frac{1}{2} \\\\\n", + " \\frac{1}{4} & \\frac{1}{4} & \\frac{1}{2} \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "Action 2: Waiting at the taxi stand \n", + "
\n", + "$\\\\\n", + " P^{2} = \n", + " \\left[ {\\begin{array}{ccc}\n", + " \\frac{1}{16} & \\frac{3}{4} & \\frac{3}{16} \\\\\n", + " \\frac{1}{16} & \\frac{7}{8} & \\frac{1}{16} \\\\\n", + " \\frac{1}{8} & \\frac{3}{4} & \\frac{1}{8} \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "Action 3: Waiting for dispatch \n", + "
\n", + "$\\\\\n", + " P^{3} =\n", + " \\left[ {\\begin{array}{ccc}\n", + " \\frac{1}{4} & \\frac{1}{8} & \\frac{5}{8} \\\\\n", + " 0 & 1 & 0 \\\\\n", + " \\frac{3}{4} & \\frac{1}{16} & \\frac{3}{16} \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "For the sake of readability, we will call the states A, B and C and the actions 'cruise', 'stand' and 'dispatch'.\n", + "We will now build the transition model as a dictionary using these matrices." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "t = {\n", + " 'A': {\n", + " 'cruise': {'A':0.5, 'B':0.25, 'C':0.25},\n", + " 'stand': {'A':0.0625, 'B':0.75, 'C':0.1875},\n", + " 'dispatch': {'A':0.25, 'B':0.125, 'C':0.625}\n", + " },\n", + " 'B': {\n", + " 'cruise': {'A':0.5, 'B':0, 'C':0.5},\n", + " 'stand': {'A':0.0625, 'B':0.875, 'C':0.0625},\n", + " 'dispatch': {'A':0, 'B':1, 'C':0}\n", + " },\n", + " 'C': {\n", + " 'cruise': {'A':0.25, 'B':0.25, 'C':0.5},\n", + " 'stand': {'A':0.125, 'B':0.75, 'C':0.125},\n", + " 'dispatch': {'A':0.75, 'B':0.0625, 'C':0.1875}\n", + " }\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The reward matrices for the problem are as follows:\n", + "
\n", + "
\n", + "Action 1: Cruising streets \n", + "
\n", + "$\\\\\n", + " R^{1} = \n", + " \\left[ {\\begin{array}{ccc}\n", + " 10 & 4 & 8 \\\\\n", + " 14 & 0 & 18 \\\\\n", + " 10 & 2 & 8 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "Action 2: Waiting at the taxi stand \n", + "
\n", + "$\\\\\n", + " R^{2} = \n", + " \\left[ {\\begin{array}{ccc}\n", + " 8 & 2 & 4 \\\\\n", + " 8 & 16 & 8 \\\\\n", + " 6 & 4 & 2\\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "Action 3: Waiting for dispatch \n", + "
\n", + "$\\\\\n", + " R^{3} = \n", + " \\left[ {\\begin{array}{ccc}\n", + " 4 & 6 & 4 \\\\\n", + " 0 & 0 & 0 \\\\\n", + " 4 & 0 & 8\\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "
\n", + "
\n", + "We now build the reward model as a dictionary using these matrices." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "r = {\n", + " 'A': {\n", + " 'cruise': {'A':10, 'B':4, 'C':8},\n", + " 'stand': {'A':8, 'B':2, 'C':4},\n", + " 'dispatch': {'A':4, 'B':6, 'C':4}\n", + " },\n", + " 'B': {\n", + " 'cruise': {'A':14, 'B':0, 'C':18},\n", + " 'stand': {'A':8, 'B':16, 'C':8},\n", + " 'dispatch': {'A':0, 'B':0, 'C':0}\n", + " },\n", + " 'C': {\n", + " 'cruise': {'A':10, 'B':2, 'C':18},\n", + " 'stand': {'A':6, 'B':4, 'C':2},\n", + " 'dispatch': {'A':4, 'B':0, 'C':8}\n", + " }\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "The Bellman update equation now is defined as follows\n", + "\n", + "$$U(s)=\\max_{a\\epsilon A(s)}\\sum_{s'}P(s'\\ |\\ s,a)(R(s'\\ |\\ s,a) + \\gamma U(s'))$$\n", + "\n", + "It is not difficult to see that all the update equations we have used till now is just a special case of this more generalized equation. \n", + "If we did not have next-state-dependent rewards, the first term inside the summation exactly sums up to R(s, a) or the state-reward for a particular action and we would get the update equation used in the previous problem.\n", + "If we did not have action dependent rewards, the first term inside the summation sums up to R(s) or the state-reward and we would get the first update equation used in `mdp.ipynb`.\n", + "
\n", + "For example, as we have the same reward regardless of the action, let's consider a reward of **r** units for a particular state and let's assume the transition probabilities to be 0.1, 0.2, 0.3 and 0.4 for 4 possible actions for that state.\n", + "We will further assume that a particular action in a state leads to the same state every time we take that action.\n", + "The first term inside the summation for this case will evaluate to (0.1 + 0.2 + 0.3 + 0.4)r = r which is equal to R(s) in the first update equation.\n", + "
\n", + "There are many ways to write value iteration for this situation, but we will go with the most intuitive method.\n", + "One that can be implemented with minor alterations to the existing `value_iteration` algorithm.\n", + "
\n", + "Our `DMDP` class will be slightly different.\n", + "More specifically, the `R` method will have one more index to go through now that we have three levels of nesting in the reward model.\n", + "We will call the new class `DMDP2` as I have run out of creative names." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class DMDP2:\n", + "\n", + " \"\"\"A Markov Decision Process, defined by an initial state, transition model,\n", + " and reward model. We also keep track of a gamma value, for use by\n", + " algorithms. The transition model is represented somewhat differently from\n", + " the text. Instead of P(s' | s, a) being a probability number for each\n", + " state/state/action triplet, we instead have T(s, a) return a\n", + " list of (p, s') pairs. The reward function is very similar.\n", + " We also keep track of the possible states,\n", + " terminal states, and actions for each state.\"\"\"\n", + "\n", + " def __init__(self, init, actlist, terminals, transitions={}, rewards={}, states=None, gamma=.9):\n", + " if not (0 < gamma <= 1):\n", + " raise ValueError(\"An MDP must have 0 < gamma <= 1\")\n", + "\n", + " if states:\n", + " self.states = states\n", + " else:\n", + " self.states = set()\n", + " self.init = init\n", + " self.actlist = actlist\n", + " self.terminals = terminals\n", + " self.transitions = transitions\n", + " self.rewards = rewards\n", + " self.gamma = gamma\n", + "\n", + " def R(self, state, action, state_):\n", + " \"\"\"Return a numeric reward for this state, this action and the next state_\"\"\"\n", + " if (self.rewards == {}):\n", + " raise ValueError('Reward model is missing')\n", + " else:\n", + " return self.rewards[state][action][state_]\n", + "\n", + " def T(self, state, action):\n", + " \"\"\"Transition model. From a state and an action, return a list\n", + " of (probability, result-state) pairs.\"\"\"\n", + " if(self.transitions == {}):\n", + " raise ValueError(\"Transition model is missing\")\n", + " else:\n", + " return self.transitions[state][action]\n", + "\n", + " def actions(self, state):\n", + " \"\"\"Set of actions that can be performed in this state. By default, a\n", + " fixed list of actions, except for terminal states. Override this\n", + " method if you need to specialize by state.\"\"\"\n", + " if state in self.terminals:\n", + " return [None]\n", + " else:\n", + " return self.actlist\n", + " \n", + " def actions(self, state):\n", + " \"\"\"Set of actions that can be performed in this state. By default, a\n", + " fixed list of actions, except for terminal states. Override this\n", + " method if you need to specialize by state.\"\"\"\n", + " if state in self.terminals:\n", + " return [None]\n", + " else:\n", + " return self.actlist" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Only the `R` method is different from the previous `DMDP` class.\n", + "
\n", + "Our traditional custom class will be required to implement the transition model and the reward model.\n", + "
\n", + "We call this class `CustomDMDP2`." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class CustomDMDP2(DMDP2):\n", + " \n", + " def __init__(self, transition_matrix, rewards, terminals, init, gamma=.9):\n", + " actlist = []\n", + " for state in transition_matrix.keys():\n", + " actlist.extend(transition_matrix[state])\n", + " actlist = list(set(actlist))\n", + " print(actlist)\n", + " \n", + " DMDP2.__init__(self, init, actlist, terminals=terminals, gamma=gamma)\n", + " self.t = transition_matrix\n", + " self.rewards = rewards\n", + " for state in self.t:\n", + " self.states.add(state)\n", + " \n", + " def T(self, state, action):\n", + " if action is None:\n", + " return [(0.0, state)]\n", + " else:\n", + " return [(prob, new_state) for new_state, prob in self.t[state][action].items()]\n", + " \n", + " def R(self, state, action, state_):\n", + " if action is None:\n", + " return 0\n", + " else:\n", + " return self.rewards[state][action][state_]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can finally write value iteration for this problem.\n", + "The latest update equation will be used." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def value_iteration_taxi_mdp(dmdp2, epsilon=0.001):\n", + " U1 = {s: 0 for s in dmdp2.states}\n", + " R, T, gamma = dmdp2.R, dmdp2.T, dmdp2.gamma\n", + " while True:\n", + " U = U1.copy()\n", + " delta = 0\n", + " for s in dmdp2.states:\n", + " U1[s] = max([sum([(p*(R(s, a, s1) + gamma*U[s1])) for (p, s1) in T(s, a)]) for a in dmdp2.actions(s)])\n", + " delta = max(delta, abs(U1[s] - U[s]))\n", + " if delta < epsilon * (1 - gamma) / gamma:\n", + " return U" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These algorithms can be made more pythonic by using cleverer list comprehensions.\n", + "We can also write the variants of value iteration in such a way that all problems are solved using the same base class, regardless of the reward function and the number of arguments it takes.\n", + "Quite a few things can be done to refactor the code and reduce repetition, but we have done it this way for the sake of clarity.\n", + "Perhaps you can try this as an exercise.\n", + "
\n", + "We now need to define terminals and initial state." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "terminals = ['end']\n", + "init = 'A'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's instantiate our class." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['stand', 'dispatch', 'cruise']\n" + ] + } + ], + "source": [ + "dmdp2 = CustomDMDP2(t, r, terminals, init, gamma=.9)" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'A': 124.4881543573768, 'B': 137.70885410461636, 'C': 129.08041190693115}" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "value_iteration_taxi_mdp(dmdp2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These are the expected utility values for the states of our MDP.\n", + "Let's proceed to write a helper function to find the expected utility and another to find the best policy." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def expected_utility_dmdp2(a, s, U, dmdp2):\n", + " return sum([(p*(dmdp2.R(s, a, s1) + dmdp2.gamma*U[s1])) for (p, s1) in dmdp2.T(s, a)])" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from utils import argmax\n", + "def best_policy_dmdp2(dmdp2, U):\n", + " pi = {}\n", + " for s in dmdp2.states:\n", + " pi[s] = argmax(dmdp2.actions(s), key=lambda a: expected_utility_dmdp2(a, s, U, dmdp2))\n", + " return pi" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Find the best policy." + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'C': 'cruise', 'A': 'stand', 'B': 'stand'}\n" + ] + } + ], + "source": [ + "pi = best_policy_dmdp2(dmdp2, value_iteration_taxi_mdp(dmdp2, .01))\n", + "print(pi)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have successfully adapted the existing code to a different scenario yet again.\n", + "The takeaway from this section is that you can convert the vast majority of reinforcement learning problems into MDPs and solve for the best policy using simple yet efficient tools." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## GRID MDP\n", + "---\n", + "### Pathfinding Problem\n", + "Markov Decision Processes can be used to find the best path through a maze. Let us consider this simple maze.\n", + "\n", + "\n", + "This environment can be formulated as a GridMDP.\n", + "
\n", + "To make the grid matrix, we will consider the state-reward to be -0.1 for every state.\n", + "
\n", + "State (1, 1) will have a reward of -5 to signify that this state is to be prohibited.\n", + "
\n", + "State (9, 9) will have a reward of +5.\n", + "This will be the terminal state.\n", + "
\n", + "The matrix can be generated using the GridMDP editor or we can write it ourselves." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "grid = [\n", + " [None, None, None, None, None, None, None, None, None, None, None], \n", + " [None, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, None, +5.0, None], \n", + " [None, -0.1, None, None, None, None, None, None, None, -0.1, None], \n", + " [None, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, None], \n", + " [None, -0.1, None, None, None, None, None, None, None, None, None], \n", + " [None, -0.1, None, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, None], \n", + " [None, -0.1, None, None, None, None, None, -0.1, None, -0.1, None], \n", + " [None, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, None, -0.1, None], \n", + " [None, None, None, None, None, -0.1, None, -0.1, None, -0.1, None], \n", + " [None, -5.0, -0.1, -0.1, -0.1, -0.1, None, -0.1, None, -0.1, None], \n", + " [None, None, None, None, None, None, None, None, None, None, None]\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have only one terminal state, (9, 9)" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "terminals = [(9, 9)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We define our maze environment below" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [], + "source": [ + "maze = GridMDP(grid, terminals)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To solve the maze, we can use the `best_policy` function along with `value_iteration`." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pi = best_policy(maze, value_iteration(maze))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is the heatmap generated by the GridMDP editor using `value_iteration` on this environment\n", + "
\n", + "\n", + "
\n", + "Let's print out the best policy" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "None None None None None None None None None None None\n", + "None v < < < < < < None . None\n", + "None v None None None None None None None ^ None\n", + "None > > > > > > > > ^ None\n", + "None ^ None None None None None None None None None\n", + "None ^ None > > > > v < < None\n", + "None ^ None None None None None v None ^ None\n", + "None ^ < < < < < < None ^ None\n", + "None None None None None ^ None ^ None ^ None\n", + "None > > > > ^ None ^ None ^ None\n", + "None None None None None None None None None None None\n" + ] + } + ], + "source": [ + "from utils import print_table\n", + "print_table(maze.to_arrows(pi))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can infer, we can find the path to the terminal state starting from any given state using this policy.\n", + "All maze problems can be solved by formulating it as a MDP." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## POMDP\n", + "### Two state POMDP\n", + "Let's consider a problem where we have two doors, one to our left and one to our right.\n", + "One of these doors opens to a room with a tiger in it, and the other one opens to an empty hall.\n", + "
\n", + "We will call our two states `0` and `1` for `left` and `right` respectively.\n", + "
\n", + "The possible actions we can take are as follows:\n", + "
\n", + "1. __Open-left__: Open the left door.\n", + "Represented by `0`.\n", + "2. __Open-right__: Open the right door.\n", + "Represented by `1`.\n", + "3. __Listen__: Listen carefully to one side and possibly hear the tiger breathing.\n", + "Represented by `2`.\n", + "\n", + "
\n", + "The possible observations we can get are as follows:\n", + "
\n", + "1. __TL__: Tiger seems to be at the left door.\n", + "2. __TR__: Tiger seems to be at the right door.\n", + "\n", + "
\n", + "The reward function is as follows:\n", + "
\n", + "We get +10 reward for opening the door to the empty hall and we get -100 reward for opening the other door and setting the tiger free.\n", + "
\n", + "Listening costs us -1 reward.\n", + "
\n", + "We want to minimize our chances of setting the tiger free.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Our transition probabilities can be defined as:\n", + "
\n", + "
\n", + "Action `0` (Open left door)\n", + "$\\\\\n", + " P(0) = \n", + " \\left[ {\\begin{array}{cc}\n", + " 0.5 & 0.5 \\\\\n", + " 0.5 & 0.5 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "Action `1` (Open right door)\n", + "$\\\\\n", + " P(1) = \n", + " \\left[ {\\begin{array}{cc}\n", + " 0.5 & 0.5 \\\\\n", + " 0.5 & 0.5 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "Action `2` (Listen)\n", + "$\\\\\n", + " P(2) = \n", + " \\left[ {\\begin{array}{cc}\n", + " 1.0 & 0.0 \\\\\n", + " 0.0 & 1.0 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "
\n", + "
\n", + "Our observation probabilities can be defined as:\n", + "
\n", + "
\n", + "$\\\\\n", + " O(0) = \n", + " \\left[ {\\begin{array}{ccc}\n", + " Open left & TL & TR \\\\\n", + " Tiger: left & 0.5 & 0.5 \\\\\n", + " Tiger: right & 0.5 & 0.5 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "\n", + "$\\\\\n", + " O(1) = \n", + " \\left[ {\\begin{array}{ccc}\n", + " Open right & TL & TR \\\\\n", + " Tiger: left & 0.5 & 0.5 \\\\\n", + " Tiger: right & 0.5 & 0.5 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "\n", + "$\\\\\n", + " O(2) = \n", + " \\left[ {\\begin{array}{ccc}\n", + " Listen & TL & TR \\\\\n", + " Tiger: left & 0.85 & 0.15 \\\\\n", + " Tiger: right & 0.15 & 0.85 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + "\n", + "
\n", + "
\n", + "The rewards of this POMDP are defined as:\n", + "
\n", + "
\n", + "$\\\\\n", + " R(0) = \n", + " \\left[ {\\begin{array}{cc}\n", + " Openleft & Reward \\\\\n", + " Tiger: left & -100 \\\\\n", + " Tiger: right & +10 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "$\\\\\n", + " R(1) = \n", + " \\left[ {\\begin{array}{cc}\n", + " Openright & Reward \\\\\n", + " Tiger: left & +10 \\\\\n", + " Tiger: right & -100 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "$\\\\\n", + " R(2) = \n", + " \\left[ {\\begin{array}{cc}\n", + " Listen & Reward \\\\\n", + " Tiger: left & -1 \\\\\n", + " Tiger: right & -1 \\\\\n", + " \\end{array}}\\right] \\\\\n", + " \\\\\n", + " $\n", + " \n", + "
\n", + "Based on these matrices, we will initialize our variables." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's first define our transition state." + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [], + "source": [ + "t_prob = [[[0.5, 0.5], \n", + " [0.5, 0.5]], \n", + " \n", + " [[0.5, 0.5], \n", + " [0.5, 0.5]], \n", + " \n", + " [[1.0, 0.0], \n", + " [0.0, 1.0]]]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Followed by the observation model." + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [], + "source": [ + "e_prob = [[[0.5, 0.5], \n", + " [0.5, 0.5]], \n", + " \n", + " [[0.5, 0.5], \n", + " [0.5, 0.5]], \n", + " \n", + " [[0.85, 0.15], \n", + " [0.15, 0.85]]]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the reward model." + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [], + "source": [ + "rewards = [[-100, 10], \n", + " [10, -100], \n", + " [-1, -1]]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now define our states, observations and actions.\n", + "
\n", + "We will use `gamma` = 0.95 for this example.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [], + "source": [ + "# 0: open-left, 1: open-right, 2: listen\n", + "actions = ('0', '1', '2')\n", + "# 0: left, 1: right\n", + "states = ('0', '1')\n", + "\n", + "gamma = 0.95" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have all the required variables to instantiate an object of the `POMDP` class." + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [], + "source": [ + "pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can now find the utility function by running `pomdp_value_iteration` on our `pomdp` object." + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "defaultdict(list,\n", + " {'0': [array([-83.05169196, 26.94830804])],\n", + " '1': [array([ 26.94830804, -83.05169196])],\n", + " '2': [array([23.55049363, -0.76359097]),\n", + " array([23.55049363, -0.76359097]),\n", + " array([23.55049363, -0.76359097]),\n", + " array([23.55049363, -0.76359097]),\n", + " array([23.24120177, 1.56028929]),\n", + " array([23.24120177, 1.56028929]),\n", + " array([23.24120177, 1.56028929]),\n", + " array([20.0874279 , 15.03900771]),\n", + " array([20.0874279 , 15.03900771]),\n", + " array([20.0874279 , 15.03900771]),\n", + " array([20.0874279 , 15.03900771]),\n", + " array([17.91696135, 17.91696135]),\n", + " array([17.91696135, 17.91696135]),\n", + " array([17.91696135, 17.91696135]),\n", + " array([17.91696135, 17.91696135]),\n", + " array([17.91696135, 17.91696135]),\n", + " array([15.03900771, 20.0874279 ]),\n", + " array([15.03900771, 20.0874279 ]),\n", + " array([15.03900771, 20.0874279 ]),\n", + " array([15.03900771, 20.0874279 ]),\n", + " array([ 1.56028929, 23.24120177]),\n", + " array([ 1.56028929, 23.24120177]),\n", + " array([ 1.56028929, 23.24120177]),\n", + " array([-0.76359097, 23.55049363]),\n", + " array([-0.76359097, 23.55049363]),\n", + " array([-0.76359097, 23.55049363]),\n", + " array([-0.76359097, 23.55049363])]})" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "utility = pomdp_value_iteration(pomdp, epsilon=3)\n", + "utility" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "%matplotlib inline\n", + "\n", + "def plot_utility(utility):\n", + " open_left = utility['0'][0]\n", + " open_right = utility['1'][0]\n", + " listen_left = utility['2'][0]\n", + " listen_right = utility['2'][-1]\n", + " left = (open_left[0] - listen_left[0]) / (open_left[0] - listen_left[0] + listen_left[1] - open_left[1])\n", + " right = (open_right[0] - listen_right[0]) / (open_right[0] - listen_right[0] + listen_right[1] - open_right[1])\n", + " \n", + " colors = ['g', 'b', 'k']\n", + " for action in utility:\n", + " for value in utility[action]:\n", + " plt.plot(value, color=colors[int(action)])\n", + " plt.vlines([left, right], -10, 35, linestyles='dashed', colors='c')\n", + " plt.ylim(-10, 35)\n", + " plt.xlim(0, 1)\n", + " plt.text(left/2 - 0.35, 30, 'open-left')\n", + " plt.text((right + left)/2 - 0.04, 30, 'listen')\n", + " plt.text((right + 1)/2 + 0.22, 30, 'open-right')\n", + " plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXlcVNX7xz9nZthXWRSU1Q0UJBREQITUci/XMpc0yg03yi3zq4ZkmltuZWaWmfnT1EpLLS0zFTUVRRQXElkURUUUBVkHnt8fA8Q4wzZzZwHO+/WalzJz7znPXc793HOec56HERE4HA6H0zgR6doADofD4egOLgIcDofTiOEiwOFwOI0YLgIcDofTiOEiwOFwOI0YLgIcDofTiFFbBBhjxoyxs4yxeMbYFcbYorLvv2WMpTDGLpZ9fNU3l8PhcDhCIhGgjEIAPYgolzFmACCGMfZb2W+ziWiPAHVwOBwORwOoLQIkW22WW/anQdmHr0DjcDicegATYsUwY0wM4DyA1gA+J6L3GWPfAgiCrKdwBMBcIipUsu8EABMAwMzMzM/T01NtexoKiXl5AAAPU1MdW8Lh6Ae8TSjn/PnzD4nIXpV9BRGBisIYswbwM4BpALIA3ANgCGATgJtEFF3d/v7+/hQbGyuYPfWdF+PiAAB/d+yoY0s4HP2AtwnlMMbOE5G/KvsKOjuIiLIB/A2gDxFlkIxCAFsABAhZF4fD4XDUR22fAGPMHkAxEWUzxkwAvARgGWPMkYgyGGMMwCAACerW1diY7+qqaxM4HL2CtwnhEWJ2kCOArWV+ARGAXUS0nzH2V5lAMAAXAUwSoK5GxUs2Nro2gcPRK3ibEB4hZgddAqAwQEdEPdQtu7FzMScHAOBrYaFjSzgc/YC3CeERoifA0RDvJiUB4E4wDqcc3iaEh4eN4HA4nEYMFwEOh8NpxHAR4HA4nEYMFwEOh8NpxHDHsB6zpGVLXZvA4egVvE0IDxcBPSbYykrXJnA4egVvE8LDh4P0mFNPnuDUkye6NoPD0Rt4mxAe3hPQY+YlJwPgc6I5nHJ4mxAe3hPgcDicRgwXAQ6Hw2nEcBFQg7feegt79lSfPfP69evw9fVFx44dcf78eWzYsEFL1nGqw9zcHABw9+5dDBs2rMrtsrOz+TXjqMTGjRvx3XffVbvNt99+i6lTpyr9bcmSJZowSwEuAhpm7969GDhwIOLi4mBra8sfKHpG8+bNqxVyLgIcVZBKpZg0aRLGjBmjchlcBFTk008/hbe3N7y9vbFmzRqkpqbC09MTY8eOhY+PD4YNG4a8shR158+fR1hYGPz8/NC7d29kZGQAAF588UW8//77CAgIQNu2bXHixIka61VW1sGDB7FmzRps3rwZ3bt3x9y5c3Hz5k34+vpi9uzZNZa5pnVrrGndWr0TwqmW1NRUeHt7AwCuXLmCgIAA+Pr6wsfHBzdu3FB6zVasWIHOnTvDx8cHH374YUU57dq1w/jx4+Hl5YVevXohPz9fZ8fVUKlrm9Dm8+DFF1/EvHnzEBYWhrVr1yIqKgorV64EAJw7dw4+Pj4ICgrC7NmzK+45QNYb7dOnD9q0aYM5c+YAAObOnYv8/Hz4+vpi1KhRKp2rWkNEevPx8/MjdYiNjSVvb2/Kzc2lnJwcat++PV24cIEAUExMDBERhYeH04oVK6ioqIiCgoLowYMHRES0c+dOCg8PJyKisLAwmjFjBhERHThwgHr27Km0vrFjx9Lu3burLevDDz+kFStWEBFRSkoKeXl5qXWMHGEwMzMjIvlrMnXqVPr++++JiKiwsJDy8vIUrtmhQ4do/PjxVFpaSiUlJdS/f386duwYpaSkkFgspri4OCIieu2112jbtm1aPipOZbT9PAgLC6OIiIiKvyu3fS8vLzp58iQREb3//vsV99SWLVvI3d2dsrOzKT8/n1xcXOjWrVtE9N89WhsAxJKKz10hMosZAzgOwAiyKad7iOhDxpg7gJ0AbABcAPAmERWpW191xMTEYPDgwTAzMwMADBkyBCdOnICzszO6du0KABg9ejTWrVuHPn36ICEhAS+//DIAoKSkBI6OjhVlDRkyBADg5+eH1NTUautNTEystixV+fPRIwA8kYa2CAoKwscff4z09HQMGTIEbdq0Udjm8OHDOHz4MDqWTVHMzc3FjRs34OLiAnd3d/j6+gKo3X3DqTt1aRO6eB4MHz5c4bvs7Gzk5OQgODgYADBy5Ejs37+/4veePXvCqmwRXPv27ZGWlgZnZ+caj08ohFgnUAigBxHlMsYMAMQwxn4DMAPAaiLayRjbCOAdAF8IUF+VyARREVmGS/m/iQheXl44ffq00n2MjIwAAGKxGFKpFAAQHh6OuLg4NG/eHAcPHpSrt7qyVGVxWhoALgLaYuTIkejSpQsOHDiA3r17Y/PmzWj5XJgCIsIHH3yAiRMnyn2fmppacc8AsvuGDwcJT13ahC6eB+WCUxs7ni/7+fK1hdo+gbLeSG7ZnwZlHwLQA0C5x20rZHmGNUpoaCj27t2LvLw8PHv2DD///DO6deuGW7duVVzcHTt2ICQkBB4eHsjMzKz4vri4GFeuXKm2/C1btuDixYtyAgCg1mVZWFggpywzEkf/SE5ORsuWLTF9+nS8+uqruHTpksI16927N7755hvk5spu+Tt37uDBgwe6MplTDbp6HjxPkyZNYGFhgX/++QcAsHPnzlrZb2BggOLi4lptqw6COIYZY2LG2EUADwD8AeAmgGwiKpe0dAAthKirOjp16oS33noLAQEB6NKlC8aNG4cmTZqgXbt22Lp1K3x8fPDo0SNERETA0NAQe/bswfvvv48XXngBvr6+OHXqlEr11rYsW1tbdO3aFd7e3rVyDHO0yw8//ABvb2/4+vri+vXrGDNmjMI169WrF0aOHImgoCB06NABw4YN48Kup+jqeaCMr7/+GhMmTEBQUBCIqGL4pzomTJgAHx8fjTuGWU1dlToVxpg1gJ8BLASwhYhal33vDOAgEXVQss8EABMAwMXFxS+trLsnFKmpqRgwYAASEhIELVcbvBgXB4AvkedwylG3TejqeZCbm1uxNuWTTz5BRkYG1q5dK1j5jLHzROSvyr6CThElomwAfwMIBGDNGCv3OTgBuFvFPpuIyJ+I/O3t7YU0h8PhcPSCAwcOwNfXF97e3jhx4gTmz5+va5MqULsnwBizB1BMRNmMMRMAhwEsAzAWwI+VHMOXiKjaVTf+/v4UGxurlj0NicSy+csepqY6toTD0Q94m1COOj0BIWYHOQLYyhgTQ9az2EVE+xljVwHsZIwtBhAH4GsB6mpU8Budw5GHtwnhUVsEiOgSAIUBOiJKBhCgbvmNmV8fPgQAvGJnp2NLOBz9gLcJ4eH5BPSYVbdvA+A3PIdTDm8TwtPgYgdxOBwOp/ZwEeBwOJxGDBcBDofDacRwEeBwOJxGDHcM6zHb2rXTtQkcjl7B24TwcBHQY5yNjXVtAoejV/A2ITx8OEiP+eHBA/zAI1RyOBXwNiE8etUTuHdP1xboF1/cuQMAGN60qY4t4XD0A94mFPm/y/+n1v561RO4cweoRTpfDofD4QC4m3MXEQci1CpDr0TA0BCYMAEoLNS1JRwOh6P/RP4eiUKpeg9MvRIBFxfg+nVg2TJdW8LhcDj6zf5/92PP1T1YGLZQrXL0SgSsrIA33gA+/hhITNS1NRwOh6Of5BblYvKByfCy98Ks4FlqlaVXjmEAWLMG+P13YOJE4OhR4Lmc0I2KPV5eujaBw9EreJuQseCvBbj99DZOvn0ShmJDtcrSq54AADRrBixfDhw7BmzZomtrdIudoSHsDNW7wBxOQ4K3CeD83fNYd3YdJvlNQrBzsNrl6Z0IAMA77wDdugGzZgGNeUrwtxkZ+DYjQ9dmcDh6Q2NvE9JSKcb/Oh7NzJph6UtLBSlTbRFgjDkzxo4yxq4xxq4wxiLLvo9ijN1hjF0s+/SrtVEi4MsvgdxcYMYMdS2sv3x77x6+5YsnOJwKGnubWHdmHeLuxWFd33WwNrYWpEwhegJSADOJqB1kCeanMMbal/22moh8yz4H61Jou3bABx8A27cDhw4JYCWHw+HUY1KzU7Hg6AIMaDsAQ9sNFaxctUWAiDKI6ELZ/3MAXAPQQt1yAZkItG0LREQAZfmlORwOp9FBRJhycAoYGD7v9zmYgDNmBPUJMMbcIMs3fKbsq6mMsUuMsW8YY02q2GcCYyyWMRabmZkp95uxMbBpE5CSAkRHC2kph8Ph1B92X92NgzcOYnGPxXCxchG0bMFEgDFmDuBHAO8S0VMAXwBoBcAXQAaAVcr2I6JNRORPRP729vYKv4eFAW+/DaxcCVy6JJS1HA6HUz94nP8Y03+bDj9HP0wLmCZ4+YyI1C+EMQMA+wEcIqJPlfzuBmA/EXlXV46/vz/FxsYqfP/oEeDpCbi7A6dOAWKx2ibXC/JKSgAApo3lgDmcGmiMbWLirxOxOW4zzo0/h06OnZRuwxg7T0T+qpQvxOwgBuBrANcqCwBjzLHSZoMBJKhah40NsHo1cPYs8MUXqtta3zAVixvVzc7h1ERjaxMxt2Kw6cImvNvl3SoFQF3U7gkwxkIAnABwGUBp2dfzAIyAbCiIAKQCmEhE1U7wraonAABEQJ8+wOnTwLVrQAtBXM/6zYaysLmTG8PBcji1oDG1iaKSInT8siOeFT1DwuQEmBuaV7mtOj0BtcNGEFEMAGWu6jpNCQWA/Pz8Kn9jTNYL8PYGpk0DfvqprqXXP3aVrZRrDDc8h1MbGlObWH5yOa5mXsWBkQeqFYCHDx+qVY9erRi+evUqJBIJunbtintKFoS0bAl8+CHw88/A3r06MJDD4XC0wL9Z/2Lx8cV43et19GujuM62oKAAw4cPh7GxMZRNqKkLeiUCAFBSUoJTp07B0dERhoaGGDx4MHJzcyt+nzED8PEBpk4Fnj7VoaEcDoejAYgIk/ZPgrHEGGv7rK34XiqV4t1334W5uTlMTEywa9cuFAqQfEXvRKAyxcXF2Lt3LywsLGBqaoqIiAgwJsVXXwF37wLz5+vaQg6HwxGWrfFbcTT1KJa/vBwO5g5Ys2YNbG1tYWBggLVr1+LZs2eC1qdXIuDp6YnQ0FAYGRkp/Jafn4+NGzfCwMAAvXs3QUDAx/jsM9mMIQ6Hw2kIZD7LxMzDM+Fp6omoV6PAGMN7772HR48eKWwrkUjQoUMH7Nq1S606BVknIBSVZwfdunULM2bMwOHDh5GTk1PlPmJxM3z11XKEh4/RlpkcDocjOKdPn0bvL3sjxyUH2AggU3EbIyMjdOnSBcuWLUNgYGDF9zpdJ6ApXFxcsGfPHjx9+hQ5OTmYOnUq7OzsFLYrKbmPt98eC5FIhFatWuH48eM6sJbD4XDqTmpqKgICAiAWixE8Ohg57jlADOQEwMLCAkOHDkVaWhoKCgpw7NgxOQFQF70VgcqYm5tj/fr1yMzMhFQqxSeffAJXV1e5IEpEhOTkZISFhYExBl9fXyTW8xyVK2/dwspbt3RtBoejNzSENpGdnY0+ffrAwMAA7u7uOHfuHEpFpcAAAFkATgB2dnaYOnUqcnJy8PTpU+zZswcuLsLGDCqnXohAZcRiMd5//32kpqaitLQUn322E4x5A5BfRRgfHw9PT08wxtCjRw+159Lqgv1ZWdiflaVrMzgcvaG+tgmpVIoxY8bAxMQETZo0waFDhyCVSv/bIAyADTDeYTykBVJkZmZi/fr1MDeven2AUNQ7EXieKVOGY+3aywCkWLToNEJCQmD4XPq5o0ePwt7eHgYGBhg+fDgKCgp0YyyHw2lU/O9//4OlpSUMDAywbds2hWePt7c3ln+7HJIwCd7yfQub5m2CWMthMeq9CADA5MlAQADw2WeB2LfvBAoLC5GcnIxBgwbBzMysYjupVIpdu3bBxMQEJiYmmDlzprwaczgcjpps2rQJtra2YIxhyZIlchNbxGIxQkJCcPr0aRAR4i/F4yfpT7A2tsbKl1fqxN4GIQJiMfDVV7Joo3PmyL5zd3fHzz//jNzcXGRnZyMiIgLW1v+lYysoKMCnn34KAwODCp8Dh8PhqMLvv/8OBwcHMMYwceJEuSmdRkZGGDRoEJKTkyGVSnHixIkKx+7G2I34J/0frO69GramtjqxvUGIACBbRTxzJvD118CxY/K/WVlZYcOGDXj8+DGkUik++ugjuZlGz549w/Tp08EYg4WFBXbv3q1l65VjIhbDpBFFTORwakKf2sSlS5fg7OwMxhj69u2L+/fvV/xWvrg1OzsbBQUF+Pnnn+Hu7i63/52nd/DBkQ/wcsuXMarDKG2b/x9EpDcfPz8/Uodnz4jc3Yk8PIgKCmq3z7Zt28jBwYEgi3Yq97G0tKQ//vhDLZs4HE7DIT09nVxdXZU+L8zMzOjDDz8kqVRaq7KG/DCEjBcbU1JWktp2AYglFZ+7DaYnAACmprJIo4mJwNKltdtn9OjRyMjIABHh2LFjaFEpOuHTp0/x8ssvgzGGJk2a4OTJkxqynMPh6Cv37t2Du7s7GGNwcnJCWlpaxW8WFhb46quvQETIzc1FVFRUrRy7vyT+gp+u/YSFoQvRyqaVJs2vGVXVQxMfdXsC5YwcSWRoSHTtmuplXL9+nZycnJQqvrW1NR0+fFgQW6sjOiWFolNSNF4Ph1Nf0FabuHnzJrm5uVU5QrBv3z6Vy35a8JScPnUi7w3eVCQtEsRe6LInwBhzZowdZYxdY4xdYYxFln1vwxj7gzF2o+xfpYnmNcHq1YCZGTBhAlBaWvP2yvDw8MDt27dBREhNTYWTk1PFb9nZ2ejVq1dFD2Hnzp0CWS7PkcePceTxY42UzeHURzTZJs6ePQs3NzcwxtCqVSukpqZW/GZpaYnDhw+DiPDkyRO8+uqrKtez4OgC3Hl6B1+98hUMxAYCWK4eQgwHSQHMJKJ2AAIBTGGMtQcwF8ARImoD4EjZ39Vy/fp1DBs2DPPmzcPOnTuV5hSoDU2bAitWACdOAN98o1IRcri6ulYIwvXr1+Ho+F/mzOzsbIwYMQKMMVhbW2P58uUoKcuDyuFw9Jvdu3fDyckJjDF06dJFYajnl19+qXjwv/zyy2rXd+7OOaw7sw4R/hEIdFIt9EN2djb27duHRYsWYcSIEejWrZtaNgkeQI4xtg/AZ2WfF4kooyzf8N9E5FHDvtUawxiDSCSCRCKBkZERTE1NYWVlhWbNmsHZ2Rnt2rVDQEAAgoKCYGZmju7dgfh44Pp1oFkz4Y6xnNOnT2Pw4MFyswLKMTc3x1tvvYWlS5eqvOrvxbg4AMDfHTuqZSeH01BQt02UlJRgzZo1WLFihdJ2a2Zmhs8//xxjx45Vy05lSEul6PxVZ9zPvY9rU67BCEaIjY3FmTNncOXKFdy6dQv37t3D48eP8ezZMxQWFqK4uBilpaWoxXNa5QBygooAY8wNwHEA3gBuEZF1pd8eE5HCkBBjbAKACQBgbm7u161bN9y7dw9ZWVnIyclBQUEBiouLUVJSUpsTUZVlEItFMDAwgLGxMczMzGBjYwNHR0e4urqiQ4cOCAwMRMeOHSGRqJZxc8+ePQrzg8sxMTFB//79sWbNGjnHc01wEeBw5FGlTeTn52Pu3Ln4/vvvq2yf//vf//C///1PJZukUikSExNx8uRJXL58GcnJycjIyEBWVhZyc3ORn5+P4uJiSAOkQC8APwC4Vrc6yl+An3+GNWvWDO7u7vjqq690LwKMMXMAxwB8TEQ/McayayMClaku0fzzFBQU4OzZszhz5gyuXbuG1NRU3L9/H9nZ2cjNzUVhYSGkUilKSkoh8+fUHZFIBLFYXHHiLS0tYWNjAycnJ7i7u8PX1xchISFo3bq13H6rVq3CRx99hCdPniiUaWhoiKCgICxfvhwBAQHV1j80IQEA8KO3t0r2czgNjdq2iTt37uDdd9/FwYMHkZeXp/C7kZERwsPDsX79erkXv3v37uHvv//GpUuX8O+//+L27dvIysrCkydPkJ+fj6KiIpSUlKC0rs5GawCTASQDot0iSMSy0QwzMzNYWlqiWbNmcHFxgYeHBzp37oyQkJA6jSCoE0paEBFgjBkA2A/gEBF9WvZdIuo4HFQXEagthYWAry9QUAAkJAD5+Q9x8uRJnD9/Hjdu3MCtW7eQmZmJJ0+eIC8vD0VFRZBKpRWe87rCGANjDBKJBAYGBigtLUV+fr7SbcViMby8vLBw4UIMHTpU3UPlcBo158+fx+zZs3HixIkqw8EYGBhAIpGgtLQUUqm0tkMtSil/STQ0NISJiQksLS1hZ2cHZ2dntGrVCh07dkRISAhatGiBfv/XDzG3YnB18lU4Wzmrc5hK0akIMFk8560AHhHRu5W+XwEgi4g+YYzNBWBDRHOqK0sTIgAAx48DYWHA7NnA8uWqlZGUlIRTp04hPj4eSUlJuHv3Lh4+fIicnJyK7p5KbwjPIZFIYGpqCjMzMzRp0gT29vZwc3ODl5cX/Pz8EBwcDGNjY7Xq4HDqC1KpFHFxcfjnn39w+fJlpKWlISMjA48ePcKzZ8/w7NkzFBcXq1UHY0yux29hYQFbW1s4ODigVatWFcPF7du3V2m4eGfCToz4cQTW9F6DyMBItWytCl2LQAiAEwAuAyh/As4DcAbALgAuAG4BeI2IFAfkKqEpEQCA8eOBLVuA2FhZz0AblN/AZ8+eRUJCApKTk3Hv3j08fPgQ9+/fF2QW0fM3sLm5OWxsbNC8eXO4ubnBx8cHwcHBKt/AHI5QJCUlISYmBhcvXkRKSgrS09PlfH9FRUUoLS1V+0UKQMX0bUdHRzRt2rTiRapz584ICAjQ2ovU4/zH8PzcEy5WLvjnnX8gFmkm5IXOh4OEQpMi8Pgx4OkJuLoCp0/Lgs7pA1euXMGoUaNw+fLlKm9+kUgEMzMzGBsbo7CwsMLfIWRX1sLCAnZ2dnBycqroyoaGhsqtj+BwAODhw4c4fvw4Ll68iMTERNy+fRsPHz5EdnZ2xbi5UEOqhoaGMDIyQklJCXJzc6uN+uvi4oLPP/8cAwYMUOfwBGXCrxPwTdw3iJ0QC18Hzb19chGoJTt2ACNHAmvXAtOna6walfnrr78wbtw4pKamVtl4jI2NERwcjFWrVsH3uS7NvXv3EBMTgwsXLuDGjRtIT0/Hw4cP5fwd6gxZlc9QKBcPMzMzWFlZoWnTpnBxcUHbtm3h5+eHkJAQuYitHP2koKAAp06dwvnz53HlyhWkpqYiMzNTboqiUC8bBgYGCi8bLVu2xAsvvIDg4GCFyRX37t3Du+++i99++w1Pnz6tsnw7Ozt89NFHmDRpkkr2aZITaScQ+m0oZgXNwopeKzRaFxeBWkIE9OsHxMQAV68CzsL7ZwTju+++wzvvvQepkilt5UgkEnh7e2PRokUqr2CUSqW4efMmYmJicOnSJSQnJ1f4OypPb1Nnim7l6W3lMyLKp7e5ubnB29sb/v7+CAwM5ENWKiCVSnH16lWcOnUKly5dQmpqKu7evYtHjx4hNzdXkGnWz09RLB92dHBwQMuWLeHt7Y2AgAC1pllfvHgRM2fOxKlTp6pN/CQyNcX7kZFYsmSJSvVog0JpIXy/9EV+cT6uTL4CM0OzmndSAy4CdSAlBfDyAnr1Avbu1WhValM+J7rn/v1YtWqV0imn5YhEIri5uWHatGmYNm2axrMTFRQUIC4uDmfOnEFCQgLS0tJw7949PHr0CHl5eRUPHnXeIiv7O0xMTGBubg47Ozs4ODigdevW8PHxQWBgIDw8PBqMeKSnp+P48eOIi4vDzZs3K3pz5RMQhOrNVV5wWT4BwdXVFe3bt4efnx+CgoK0ktpw//79WLhwIS5fvlztUI+xsTFee+01JE+bBolEovdrZ6KPRePDvz/EwZEH0bdNX43X12BEoKYVwxwOh6P32AKIgGxB2I9aq1VlEWhQoaQ5HA5H57wCoBjA77o2pHbolQj4+flpLWz12bMExghTpug+hLa6n9u3byMoKEhhCKi8218ZY2Nj9OzZE/Hx8Tq3W9Of4uJixMfH44svvsCkSZMqFu6Ym5tDLBZDtsRFeMpntpiamsLBwQEBAQEIDw/H6tWrcerUKeTn5+v83Gj6k5GRgZEjR8LKykrpuXn+u7Zt2+LUqVM6t1vdz9cXvgbcgE2vbQLlaq9ete5XdQsQEm34BCoTGQmsXw+cOgUEqhbQT6O8e+MGAGBNmza13ufChQt48803ce3aNbmbo3ylZOXVyxKJBD4+Pli0aJFeTaurCWWzoDIzM/H06VPBZkEp+xBRhY9D3bns1YUkad26NTp06KA0JIk+c+nSJcycORMnT56Uu88MDAwgFosVnL3NmzfH2rVrMWzYsFrXoUqb0BYPnj2A52ee8G7qjb/f+hsipr137AbjE9C2COTkAO3bA02aAOfPAwa6D+0th7oB5Pbv34/Jkyfj9u3bct+bmZnB0NAQjyvFZReJRGjZsiUiIyMRERGhcccyAOTm5uL06dM4e/YsEhMTkZaWhvv371dMadXEeghLS0vY2trCyckJbdq0qZii6ObmpvbxPHz4X0iSxMTECnHS1Px5ExMTWFtbw97eHk5OTvDw8ICfnx+6du0ql0Nbkxw8eBALFizApUuX5By75b2tp0+fyh2rjY0N5s2bh5kzZ6pUnz4HVRz902jsurIL8ZPi0c6+nVbr5iKgBvv2AYMGydJRzq0x44F2EfKG37hxIxYsWICHDx9WfMcYg52dHYyMjHD37l25t1sHBwe8+eabWLx4MQwNDastu6qV0c9PURRqplBDWxldvpL28uXLciFJnj59Kje9U6gZQeUhScqDlnl5eaFLly61WklbUlKCTZs24dNPP0VycrKcTfb29jA0NMS9e/fkVsObmZlh3LhxWLlypdrXRl9F4FDSIfTZ3gcLQxdiUfdFWq+fi4CaDB0KHDwoCzDXSsfpPiujqRv+/fffx4YNG5Cbm1vxnUgkQquyg09KSpJ7WDPGKrr0ssis+vFAasxUjqlz9erVivDFjx8/Flx4RSIRGGMV5VX+3dHREba2trh+/bpcDB8jIyMMHDgQW7duFfQ66qMI5BXnwXvenOb2AAAgAElEQVSDNwzEBoifFA9jifbvW3VEoH69MmmIdeuAP/4AIiKAQ4cADfkLtUp2djaOHTtWbbRUkUhU0ahLS0txo2y89XmICEVFRQrflz/MKw9NlEdR9PDwgK+vL0JDQ7U2NNGYkEgk6Ny5Mzp37lyn/Wo7BFe+sKy6uftEhLt37+Lu3bsKv5WWluLAgQNwdnZWOgTXkEKSRB+LRkp2Cv4e+7dOBEBdeE+gjM8/B6ZOBb7/Hhg1SicmKDAhMRHSggK89eRJjXkT1B03LxeE59/wRSIROnXqBAcHB5w4cUJuwZqBgQF8fHwQHR2Nfv36qXWsHN2TkJCAWbNm4fjx43KOXRMTE/j7+8PMzAxHjx5FYWGh3H7ls6wYYxoLSeLp6YmOHTtil709jK2tscmj2qj0WuPS/Uvo9GUnjH1hLL4e+LXO7ODDQQJQUgJ07QrcvClLR2lrK2z55Uv7y0Pi3rx5U/AMalVlH6prBrWkpCSMGDECFy5ckGvUlpaWiIiIQGlpKbZt2yaXA7p8OGnmzJmYOHGiSvZztM+hQ4cwf/58xMfHyw3nWFlZ4ZVXXkFwcDCio6PlrjVjDG5ubti8eTN69OhRZdnlGbf++ecfXLp0CUlJSRVRdDURksTY2BimpqYV4SxcXV01GpKkpLQEwd8EI+VxCq5PvQ4bExtBy68LXAQE4tIlwM8PePPN6hPUp6enIyYmBnFxcbhx4wbu3LlT4cjT1NL+qnIpa3ppf0xMDN5++20FP4G9vT0WL16MoqIirF27VsFJ6OjoiLFjx2LRokU1OpY52uXLL7/EqlWrcPPmTaWTAXr27InJkycjJSVF7po3a9YMy5cvx5gxYzRqX0FBAWJjYxEbG4uEhISK3m95DoE65t5VSlUhSZo3b46WLVvCx8cHISEhaNWqVZXi8dnZzzDtt2n4fvD3GOWj2+EDLgIqkpubi5iYGJw7dw6JiYm4desWrly5j0ePnsLU9Bmk0sKKh7lQUxStrKxga2sLZ2dntG3btuJmUzY+OiExEQD0puu7Z88eREZGKowBu7i44Msvv4RUKlUaB8ba2hqvvvoqVq9eDRsb3b0tNVaKioqwaNEifPvtt3LXrvK04B49emD06NGIj4+XEwYrKyvMnDkTCxYs0IXpClTXJrKzsytezq5fv45bt27hwYMHePLkCZ49eyb3cqZue5bYSFDwTgGMHxrD55IPnJ2c0aZNG3Tq1AkhISFwcHBQ6zjris5FgDH2DYABAB4QkXfZd1EAxgPILNtsHhEdrK4cVUVAKpXi3LlzOHPmDK5evYqUlBTcu3evIiSupoKZ2drawtHREa1bt4aXlxe6du0qaDAzfZwJUc6qVavw8ccfy601YIzBy8sL27dvR2lpKWbNmoWTJ0/KLRIyNTVFaGgoPv30U7Rrp9251I2JR48eYcaMGfjll1/krpFEIkGHDh0QHR2NwMBAvP766zh+/LjclE4TExOMHj0aGzZs0LvptppoE6mpqTh16lTdgvYNB9AawAYAj6souIyagvZ5eHio3bPXBxEIBZAL4LvnRCCXiFbWthx/f3/avn17xZzpmzdv4u7duwrZh4QOa9ykSRM4ODhUTFEUibpgxgx/LFhgjOholaoRBH0WgXKkUilmzZqFzZs349mzZxXfi8ViBAUFYffu3QCAGTNm4ODBgwqOZV9fX3z88cd4+eWXtW57Q+PatWuYMWMGjh8/Lpdc3djYGF27dsXKlSvh6emJ8PBw/Pzzz3IOXgMDA/Tt2xfbt2/XSvRQVdGHNrH3+l4M/mEwIr0i4fnQUy/CdxsYGKgsAkLGrnADkFDp7ygAs+pYBtX2wxgjkUhEhoaGZG5uTk2bNqW2bdtSSEgIvfHGGxQVFUW//vorPX78mFRh9GgiAwOiK1dU2l0Qwi5coLALF3RnQB3Jz8+nYcOGkaGhody1MjQ0pMGDB1N+fj7l5eXRe++9R82aNZPbRiQSUdu2bWnz5s26Pox6xZEjRyggIEDhnFtZWdGIESMoIyODiouLacaMGWRmZia3jVgspqCgILp9+7auD6PW6LpNPCl4Qi1WtSCfL3yoSFpU5/3z8/PpyJEjtHz5cho7dix1796d2rdvT46OjmRpaUlGRkYkFoupLKJyXT6xpOqzW9UdFQpSLgKpAC4B+AZAkyr2mwAgFkAsY4xsbW3J3d2d/P39adCgQTRr1izaunUrpaSk1PmEq8ODB0Q2NkQhIUQlJVqtugJd3/DqkJmZSWFhYSQWi+VuVlNTU4qIiKDi4mKSSqW0evVqatmyJYlEIrntWrRoQfPnz6fCwkJdH4resXnzZvLw8FA4Z82aNaP33nuP8vLyiIho3bp1ZGNjo/Dy1L59ezp//ryOj0I1dN0mph2cRiyK0T+3/9FqvZmZmfTjjz/SggUL6PXXX6egoCBq06YN2dvbk7m5ud6KQDMAYsgilX4M4JuayvDz89PUOVSJb76RnaFNm3RTf+S//1Lkv//qpnIBSUhIIB8fH4W3G2tra1q6dGnFdj/99BP5+vqSRCKR265JkyYUHh5OWVlZOjwK3VFYWEgLFy6kFi1aKPSeWrZsSatXryapVEpERD/++CM1b95c4U3R2dmZfv31Vx0fifrosk2cST9DLIrR1ANTdVJ/deilCNT2t8offROB0lKiF18ksrIiysjQtTUNgyNHjpC7u7uCIDg4OND27dsrtouNjaXu3buTsbGxQk+iX79+lJiYqMOj0DxZWVn09ttvK7zJSyQS8vX1pZ9++qli27Nnz1Lbtm0VzqmdnR198cUXOjyKhkORtIh8vvChFqta0JOCJ7o2RwG9FAEAjpX+/x6AnTWVoW8iQESUmEhkZEQ0fLiuLWl4bNmyhZo2baowXNGqVSs6duxYxXbp6en0+uuvk6WlpYKvITAwkI4cOaLDoxCOxMRE6t+/P5mamsodp7GxMXXv3p1iY2Mrtk1JSaHOnTsrDAmZm5vTnDlzdHgUDZNlMcsIUaCfrv5U88Y6QOciAGAHgAzI8umkA3gHwDYAl8t8Ar9UFoWqPvooAkRE0dGyM3XwoHbrHXXlCo3SpWdai0RHR5OVlZXCcEfHjh3p+vXrFdvl5eVRZGSkgniIRCLy9PSkLVu26O4gVODo0aMUGBio4Ni1tLSk119/ndLT0yu2zcnJob59+5KBgYGCSLz55ptUXFyswyPRDrpoE8mPkslksQkN2jlIq/XWBZ2LgFAffRWBwkKidu2IXF2JcnO1V6+unWC6oLi4mMaPH08mJiYKwyA9e/akzMzMim2lUimtXLmS3Nzc5IZCGGPUokULWrhwYcVYuT6xdetW8vT0VHCaN23alCIjIyscu0Sy8zF27FiFYTGJREK9evVSefZbfUXbbaK0tJR6b+tNFkss6PYT/Z1FxUVAC5w4ITtbM2dqr87GKAKVycnJoQEDBii8+RoZGdEbb7xB+fn5ctvv2bOHfHx8FBzLNjY2NG7cOMrOztbJcUilUoqKiiInJycFsXJzc6OVK1cqiNWCBQsUhr9EIhH5+/vTjRs3dHIc+oC228T2S9sJUaB1/6zTWp2qwEVAS0yYQCQWE2nrHmzsIlCZ27dvU1BQkMLbs7m5Oc2aNUth+zNnzlBYWBgZGRnJbW9mZkYDBgygpKQkjdqbnZ1N48aNU+rY9fHxoT179ijss3nzZrK3t1fwkbRt25ZOnTqlUXvrC9psE1l5WWS/3J4CvgogaYn+9Sgrw0VASzx6RNSsGZG/P5E2Rhm4CCjn/Pnz1K5dO4XZMDY2NrRuneIbW1paGg0bNowsLCwUHMvBwcFyTmh1SEpKoldffVVhUZaRkRGFhYXRmTNnFPb57bffyMXFRWFKZ/PmzWn37t2C2NWQ0GabeGffOyReJKaLGRe1Up86cBHQIjt3ys7amjWar2vuzZs09+ZNzVdUj/n111/J2dlZ4SHaokUL2rt3r8L2OTk5NHXqVIU3brFYTO3ataNt27bVqf5jx45RcHCwgmPXwsKChg0bRmlpaQr7xMfHk7e3t4KINWnShFauXKnyuWgMaKtN/J3yNyEKNOdw/ZhpxUVAi5SWEvXtS2RmRqSkfXN0yGeffUa2trZKh1POnj2rsL1UKqVly5YpdSw7OztTdHS0Usfytm3bqF27dgpDU/b29jR16lTKyclR2CcjI4NCQkIU9jEzM6PIyMhGMbOnvlBQXEAe6z3IfY07PSt6pmtzagUXAS2TkkJkakr0yisyUeDoH3PmzClfTi/nWA0ICKgyBMmuXbvI29tb4UFtbW1N/v7+1KJFC6WO3WXLlikVi/z8fBo8eLDSWErDhg1TcGxz9IMPj35IiAL9fuN3XZtSa7gI6ICVK2VnT4l/TzCGXL5MQy5f1lwFjYDi4mJ68803FaZYGhgYUN++fZW+tRMR/fHHH+Tg4KAwzFQ+1LN+/foq64uIiFBY8CUWiyksLExuiiun7mi6TVzLvEaGHxnSyB9HaqwOTcBFQAcUFxN17Ejk6EikqZmH3DEsLI8fP6ZevXopTCE1Njam8PBwunHjBg0cOFChB2FoaEi2trYKaxcMDQ2pa9euFBMTQ0uXLiVra2uFoSgfHx9KSEjQ9aE3GDTZJkpKS6jbN92oySdN6H7ufY3UoSm4COiIc+eIRCKiiAjNlM9FQHPcuHGD/P39lb7pl7/tDxkyhJKTk+X2y8nJoYiICLKzs6tyX3d39wYTykLf0GSb+Or8V4Qo0Obz9S+cuToiIAJHZfz9genTgY0bgdOndW0Np7b88MMPGDx4MOLKEpQoIycnBydOnMBff/0l9/2FCxdw+PBhZGVlVblvamoqwsPDsXTpUrmMXRz95X7ufcz+YzZCXUPxdse3dW2OVuEioCYffQQ4OQETJgDFxbq2hqOMkpISLFu2DK6urhCJRHjjjTeQkJCA0tJSuLi4YMmSJZBKpSAi7NixA46OjgCAzMxMjBs3DowxGBoaQiQSISwsDDdv3gQRwcrKCtHR0RVvVNu3b4e3tzdEIhFu3bqFefPmQSKRwN7eHpMnT0Zubq6OzwSnKt479B7yivPw5YAvwRjTtTnaRdUuhCY+9W04qJxffpENrC1ZImy50SkpFK3lZDoNhZycHJoyZYrCsI1YLCZvb2+5sNXKyMzMpFatWikd7rG2tq4xKUtMTAx17dpV6fqBIUOGKF0/wKkZTbSJ3278RogCRR2NErRcbQLuE9A9w4bJQk434rAuOictLY2GDBmidGVwuQO3OvLz8+mNN95QCDVhYGBAbm5uCt+LxWLq1q0bZdSQbCIpKUmpw9nIyIi6detGp0+fFvI0cOpAbmEuua1xI8/PPKmguEDX5qgMFwE94M4dIktLop49+doBbXL69Gnq1q2bwgPa3NycBg4cWKsYQbNmzVJ4QFeVf7c2eZSrIzs7myZOnKiwqK28h7Jr1y61zgenbsw+PJsQBTqWKkzoEF3BRUBP2LBBdka/+06Y8vrEx1Of+HhhCmtA7Nq1izp06KAw1dPW1pYmTpxYq2ihVeXfbdeuXa3z72ZkZFBoaKjSPMpTp06tcRWwVCql6OhocnZ2rtMitMaMkG0iLiOOxIvENG7fOEHK0yU6FwHIEsk/gHxmMRsAfwC4Ufav0kTzlT/1XQRKSoiCgojs7IiEWBPEp4jKUDW8w/Ps3btXIU8vIEz+3drmUa6OrVu31jkcRWNDqDYhLZFS502dqemKpvQo75EAlukWfRCBUACdnhOB5QDmlv1/LoBlNZVT30WAiOjyZSKJhGjsWPXLaswiUFOgt61bt9aqHF3k3z18+DC5ubkpiI2DgwPt2LGjVmUcPXq0ysB0Q4cObbSOZaHaxNp/1hKiQP936f8EsEr36FwEZDYo5BhORFlKSQCOABJrKqMhiAAR0bx5sjOr7nqhxiYC6enpNGzYMKW5hIODg+no0aO1KiclJYUCAgL0Iv9ubfMoV4cqIaobKkK0iVvZt8h8iTn13tabShuIA09fRSD7ud8fV7HfBACxAGJdXFw0dY60Sl4eUevWRG3aEKkTI6wxiEBsbGyVyV9effXVWid/qQ/5d6OiopRmC+vUqVOts4WpkqymISFEmxi4YyCZLDah5EfJNW9cT6jXIlD501B6AkREf/4pO7vz56texoq0NFrRALv9QqWBLC4upvDw8HqXf7e4uJjeeeedKvMo19ZuVdJW1nfUbRM/Xf2JEAVaHrNcQKt0j76KQKMdDipnzBiZf6Cxxw+TSqW0atUqcnd3V3hYOTk5UVRUVJ0eVg0p/251eZRHjhxZpx7Mli1byNPTU2EYTFkC+8ZIdn42NV/VnF744gUqkhbp2hxB0VcRWPGcY3h5TWU0NBHIzCSytSUKDpbNHGpM5OXlUWRkpMJ4uEgkIk9PT9qyZUudyqsq/26bNm3oxIkTmjkILVPXPMrVceTIEQoMDFRwLFtaWtLrr79O6enpGjoK/WXKgSnEohidSW94PhSdiwCAHQAyABQDSAfwDgBbAEcgmyJ6BIBNTeU0NBEgIvr2W9lZ3rix7vvWN59Aeno6vf7660odu4GBgXWOrHn48GFydXVVmGXj6OhY61k29ZXq8ih/9tlndSorMTGR+vfvr5DjwNjYmLp3705xcXEaOgrhUbVNnL59mlgUo+kHp2vAKt2jcxEQ6tMQRaC0lKhHDyIrK6K7d+u2b30Qgbi4OOrRo4fCuLypqSn179+fEhMT61Redfl3ly9vWOO4taWueZSrIysri8LDw6lJkyYK/ghfX1/at2+fho5CGFRpE0XSIuqwoQM5fepETwueasgy3cJFQM/5919ZXKHXXqvbfvoqAvv27SNfX18Fx26TJk0oPDycsrKy6lReRkYGdevWTWn+3enTp+vFzB59oao8yh4eHkrzKFdHYWEhzZ8/X2EBnUgkopYtW9Lq1av1zrGsSptYemIpIQq091rdBLM+wUWgHrB4sexs799f+330RQSkUimtW7eOWrZsqeB0bNGiBc2fP58KCwvrVCbPv6s+yvIoi8XiavMoV8emTZvIw8ND4Ro3a9aMZs6cqReO5bq2iaSsJDJebEyDdw7WoFW6h4tAPaCwkMjLi8jFhai2q/91KQJ5eXk0c+ZMatasmcJbooeHB23atKnOZRYXF9PUqVOV5t8NDQ2tMRonRznFxcU0atQopXmU+/Xrp1K4icOHD1Pnzp0VZi1ZWVnRiBEjdHat6tImSktL6eXvXiaLJRaU/qRhO8K5CNQTYmJkZ3zGjNpt/3l6On2uxVkcGRkZNGLECLKyslJ4mHTu3JkOHz6sUrk8/672qCqPsomJCYWHh6s0tHb16lXq06ePUsdyz549tepYrkub+D7+e0IU6LMzdXOk10e4CNQjJk2S5SWOjdW1JTLi4+OpZ8+eSh27ffr0oatXr6pU7o4dO8jBwUHBmenm5qaymHDqxo0bN6hTp04KwzuWlpa0YMEClcp88OABjR07VqljuWPHjmoH4hOKh88ekt1yO+ryVReSluiXX0MTcBGoRzx+TOTgQNSpE1FNL2XPpFJ6pgHH3K+//kqdOnVS6tgdO3YsPXjwQKVyjx07Rq1atVKY2dO0adM6rwvgCMuJEyeodevWCtfG3t6eNm9WLbF6YWEhzZ07lxwdHRWGDFu3bk3r168X3LFc2zYRvjecJNESir/XOEKxcxGoZ+zaJTvzn35a/XaChc2VSmn9+vXUunVrhbdCR0dHmjt3bp0du+VU97YZFRWltu0c4dmxY4fCgxsAubq6qtVL27hxI7Vp00bhXnBwcKDZs2erfI9VpjZt4mjKUUIUaO4fc9Wur77ARaCeUVpK1L8/kZkZUWpq1dupIwKFhYU0e/ZshSEZkUhEbdq0oY2qrF4r4/Hjx9SzZ0+l487jx4/nUzrrEcuXL1fqr+nQoYNa/prff/+d/P39lTqWR48erXJvs6Y2kV+cT23Xt6WWa1tSXpHuZzNpCy4C9ZDUVJkI9O9fdTrKuorAgwcPaPTo0QqN2sDAgPz9/en3339X2d7i4mIaOXKk0vy7AwYM4AlP6jnFxcU0ffp0hXDVtc2jXB2XL1+m3r17KwTMMzExoV69etHly5drXVZNbWLBXwsIUaDDSY3L78RFoJ7y6aeyK1BVWtnaiMDly5epV69eShtY796969TAlFGX/LuchkF+fj4NHTpU6RqOoUOHqrWGo7oXFT8/Pzpw4EC1+1fXJq48uEIG0QY06sdRKttXX+EiUE8pLpY5iB0cZA7j56nqhj9w4AD5+fkpdLWtra3V6mqXI0T+XU7DQN08ytVRWFhIc+bMUepYrmrIsqo2UVJaQiHfhJDNMhu6n3tfZZvqK1wE6jHnz8umjE6apPjblrt3aUtZwKGqnG6Ojo40Z84ctZ1umsy/y2kYJCQkUIcOHZTmUVY3rpNUKqUNGzbUOHmhcpuozKbYTYQo0DcXvlHLjvoKF4F6zowZsisRE/PfdzVNv9uwYYPa0+/Onj1LHh4eCo3a1ta2zpEqOY0LIfIoV0dV05itra1pzJgxcr3djJwMslpqRS9++2KDSRdZV7gI1HNycmThJDw8smjUqDEK46USiYQ6deokyBu5PuXf5TQMqsr10Lp1a0FyPcTHx9NLL71Upd+r79d9yfAjQ7qeeV2Ao6mfcBGox5QvyTcyMlW4wa27dCG/nTvVriMnJ4f69eunNP/uqFGj+JROjmAIkUe5OoIOHSL7Pn3+C23SGoQokKi7SK3QJvUdvRYBAKkALgO4WJOhjUUEqgrOJZFYEWMj6ORJ2XQ8ddYJlOffVZbHVp/z73IaBtXlUX7ppZdUvv8qt4nM7EyyXGhJ4uliglhedNq2batSkMP6Sn0QAbvabNuQRWDz5s3Utm3basP03r0rSz7Tvbts7YAqIlBV/l2h3sQ4nLoiZB7lym1i5qGZhCjQibQTFeHOW7VqpdDGmjdvrlK48/oEFwE9pLqEHa1ataJ169Ypdexu3Ci7Kt9+W3sR0PSYLIcjFLdv36bAwECleZRr45MqbxMX7l4g8SIxjf9lvNLthE58pO/ouwikALgA4DyACUp+nwAgFkCsi4uLps6RVhAidV9JCVHXrmUJ6o/EVykCjTn/LqdhcPbs2TrnUQ67cIFCz58j/03+1GxFM3qU96jGeqpLgdqvX786p0DVR/RdBJqX/dsUQDyA0Kq2rY89gcTEROrXr5/SWOs9evRQKdZ6QgKRgQFR6Ot5tPP+/Urfa26eNoejS/bu3UtOTk4KLzVOTk5yeZR33r9PY/74iBAF2nm57pMm0tPTafjw4QpDpoaGhhQYGEhHjhwR8rC0hl6LgFxlQBSAWVX9Xl9E4MiRIxQYGKiwrN7S0pKGDx9O6QIkgpk/X3Z1duzg+Xc5jYvq8ij/cuwXMvvYjPp+31ftNQF5eXkUGRlJTZs2VRiy9fT0rFfhz/VWBACYAbCo9P9TAPpUtb0+i8CWLVvI09NTwenUtGlTioyMFDT/an5+Pg0aNJQA4WO3cDj1CYXYVSNAmAfy7e4raOwqqVRKq1atInd3d7meNmOMWrRoQQsXLhQ8N4KQ6LMItCwbAooHcAXA/6rbXp9EQCqV0sKFC6lFixYKN4W7uzutWrVK0Juiqvy7gJicnXn+XU7jpri4mLpN7EaIAiFIPvCcqnmUq2PPnj30wgsvKDiWbWxsaNy4cZSdnS1ofeqityJQ14+uRSA7O5vGjRunEDxNIpHQCy+8QHv27BG8zuriufvt3k3NXnlIEgmRmsFAOZx6TXZ+NjmudCTz1Z4UfOQPeumll5Tms3jnnXcEHyI9c+YMhYWFKYRRNzMzowEDBlBSUpKg9akCFwE1SEpKogEDBijEUTcyMqKwsDA6c+aM4HXWNv9u2IULFHwknuzsiIKCZDOHOJzGSMT+CBItElGnv76XmzGn7cx2aWlpNGzYMKWO5aCgIDp69KjgddYGLgJ15NixYxQcHKzUsTts2DBKS0sTvE5VcryWz4n+7jvZldqwQXCzOBy959StU8SiGEX+Flnt2pnq2pgmnLw5OTk0ffp0BceyWCymdu3a0datWwWvsyq4CNSCrVu3Urt27RRm2TRt2pSmT5+ukcxY6r6llN/wpaVEPXsSWVoS3bkjuJkcjt5SJC0i7w3e5PypMz0teFrrBZS17W0LhVQqpWXLlpGbm5uCD9HZ2Zmio6M16ljmIqAEqVRK0dHR5OzsrHBR3NzcaNmyZRq5KI8fPxZsvPKXzEz6JTOTiIhu3CAyNiYaNkxwkzkcvWXJ8SWEKNAv138hIvk2UVs0lUe5Onbt2kUdOnRQeA7Y2trS+PHjBXcscxEoIzs7m8aPH68wx1gikVCHDh1oV1V5HNVEW/l3lyyRXbFffhGkOA5Hr7mRdYOMPjKioT8MFaS88hl4yvIoh4aGUmYdxaW2nD59mkJDQxWeD+bm5jRw4EBKTk5Wu45GLQLJyck0cOBAhTy4RkZGFBoaSqdPn65zmbVlzpw5SvPvBgYGCjKH+fqzZ3T92bOKv4uKiLy9iZydZTkIOJyGSmlpKfXc2pMsl1rSnaf/jYE+3yZURZN5lKsjLS2NhgwZQhYWFgr1du3alWIqZ5aqA41OBGJiYqhr164KF9DCwoKGDBmiEcduOZ999lmV+XfPnj0raF3Kxj9PnSJijOjddwWtisPRK767+B0hCrThrPxsCHXCq1dFRoZuVuXn5OTQlClTyM7OTuFF0svLi7Zv317rshqFCGzfvp28vLwULpSdnR1NmTJFI47dcmob10RoqrrhIyJkeYnPndNY1RyOzsh8lkl2y+0oaHMQlZTKz4vWhAhURlfxuaRSKX3yySfk6uqq4MN0cXGhJUuWVOvDbJAiIJVKacmSJeTi4qJwUlxdXemTTz7RqLddH/LvVnXDZ2cTOToS+foS8dBBnIbG2J/HkiRaQpfvK66Q1LQIVEaXkXp37txJ3t7eSl96IyIiFF56G4wIdFLhXSoAABI+SURBVOzYkSIiIpR2j7y9vWmnAKkWq6O6WOezZs3SaN3KqO6G37NHdvVWrtSyURyOBjmSfIQQBfrgzw+U/q5NEaiMLnN2xMTEUEhIiFLH8qBBgyg5ObnhiMDzjt2QkBCVHSW1RZ/z71Z3w5eWEr3yCpGpKVFKinbt4nA0QX5xPrVZ14ZarW1FeUXKAzLqSgQqo8vsfcnJyTRo0CCFCSkNRgREIlGFsmkSTeU/FZo/srLoj2oyIKWlEZmZEfXtKxMFDqc+M//IfEIU6M+bf1a5TU1tQpvo+jmSnZ1NERER5VPiG4YIaDpsRFRUVIPLv7tmjewqanikjMPRKAn3E0gSLaE3f3pT16aohK5HFLgIVMOWLVvqbf7duKdPKe7p02q3kUqJ/P2JmjUjelRzpj0OR+8oKS2h4K+DyXaZLT3IfVDttrVpE7pG3TzKqsBF4DkOHz5Mbm5uOvHqC0ltxz8vXCASi4kmTNCCURyOwGw8t5EQBfo27tsat9UHn0Bd0NYsQ3VEQAQNwxjrwxhLZIwlMcbmaqqeK1euwMfHByKRCL169UJqaioAwNraGsuXLwcR4e7du3jjjTc0ZYLO6NgReO89YNMmICZG19ZwOLUnIycD7//5Pnq498CYF8bo2hzB6dy5M65fv47S0lLs3bsXTk5OAICsrCxMnToVjDE4Oztj3759OrNRoyLAGBMD+BxAXwDtAYxgjLUXqvyHDx8iLCwMEokE3t7euHz5MogIZmZmmDp1KoqLi/H48WPMnj1bqCr1lqgowNUVmDABKCzUtTUcTu2I/D0SBdICbOy/EYwxXZujUQYOHIjbt2+DiLBu3TrY2NgAANLT0zFo0CCIRCK0b98eFy5c0Kpdmu4JBABIIqJkIioCsBPAQHUKLCgowLBhw2BkZAR7e3scP34cJSUlMDQ0xNChQ5Gfn4/c3FysX78eEolEkIOoD5iZAV98AVy7BixfrmtrOJyaOfDvAey+uhvzQ+ejjW0bXZujVaZNm4asrCwQEWbNmgVzc3MQEa5duwY/Pz9IJBIEBQUhPT1d47ZoWgRaALhd6e/0su8qYIxNYIzFMsZiMzMzlRYilUoxbdo0mJubw8TEBD/++COKioogFovRrVs3ZGRkoLCwEHv27IGxsbHmjkbP6dsXGD4c+Phj4N9/dW0Nh1M1uUW5mHxwMtrbt8ecrnN0bY5OWbFiBXJyclBcXIyRI0fCyMgIJSUl+Oeff+Ds7AxDQ0O88soryM3N1YwBqjoTavMB8BqAzZX+fhPA+qq2f94xrIs44PrEyexsOlnHuOMZGUTW1kQvvsjXDnD0lxm/zyBEgWLS6rYYVJU2UR/JzMyknj171jovCfR1dhCAIACHKv39AYAPqtrez8+PduzYQY6Ojgoze1xdXTWSEaghsmmT7Mp+842uLeFwFIm9E0uiRSKa+OtEXZtSL6hNhkJ1RIDJ9tcMjDEJgH8B9ARwB8A5ACOJ6EoV28sZY29vj6VLl+Kdd97RmI36zKknTwAAwVZWddqvtBQICwOuXgWuXwfs7TVhHYdTd6SlUnTZ3AV3c+7i2pRrsDa2rtP+qraJhsLx48fx9ttvIzk5Gc89u88Tkb8qZWrUJ0BEUgBTARwCcA3ArqoEoBxLS0ssWLAARIQHDx40WgEAgHnJyZiXnFzn/UQi2XTRnBxgxgwNGMbhqMj6M+txIeMC1vVZV2cBAFRvEw2F0NBQJCUlobS0FNu3b4eDg4PaZWq0J1BX/Pz86Pz587o2Q294MS4OAPB3x44q7f/hh0B0NHD4MPDyy0JaxuHUnbTsNHht8MKLbi/i1xG/qjQlVN020VBhjOlnT6CuNPR5wtrmgw+Atm2BSZOAvDxdW8NpzBARphycAgLh836f87auR+iVCHCExdgY+PJLIDkZ+OgjXVvDaczsuboHB24cwEfdP4KrtauuzeFUgotAA+fFF4HwcGDlSuDyZV1bw2mMZBdkY/rv09HJsROmd5mua3M4z9F4ltTWQ9a0bi1IOStWAPv3A+PHAydPAmKxIMVyOLXigz8/wINnD3Bg5AFIROo9coRqE5z/4D0BPcbXwgK+FhZql2NrC6xeDZw5A2zcKIBhHE4tOXnrJDae34jILpHo5NhJ7fKEahOc/9Cr2UH+/v4UGxurazP0hj8fPQIAvFQWaEodiIA+fYDTp2XxhVq0qHkfDkcdikqK0PHLjsgtysWVyVdgbmiudplCtomGRIOZHcSRZ3FaGhanpQlSFmPAhg1AcTEwnQ/LcrTAipMrcDXzKjb02yCIAADCtgmODC4CjYhWrWQhp3/6CdBh+HJOI+BG1g18dPwjvNb+NfRv21/X5nCqgYtAI2PGDMDHB5g6VbaimMMRGiLCxP0TYSwxxto+a3VtDqcGuAg0MgwMZCEl7twB5s/XtTWchsh38d/haOpRfPLSJ3C0cNS1OZwa4CLQCOnSBZg8GVi/Hjh3TtfWcBoSD/MeYubhmQh2DsYEvwm6NodTC/g6AT3mSw8PjZW9ZAmwd69s7cC5c7IeAoejLjMPz8TTwqfYNGATREz4d0xNtonGCu8J6DEepqbwMDXVSNmWlrKeQHw8sGaNRqrgNDL+TP4T38V/hzld58CrqZdG6tBkm2is8HUCesyvDx8CAF6xs9NYHYMGyaKMXrkCuLtrrBpOAye/OB8dvugAxhguTboEEwMTjdSjjTZRH+HrBBooq27fxqrbt2veUA3Wr5eFkZg8WbagjMNRhcXHF+Pm45v4csCXGhMAQDttorGhMRFgjEUxxu4wxi6Wffppqi6O6jg7y/wDv/8O/PCDrq3h1EcSHiRg+anlGPvCWPRw76Frczh1RNM9gdVE5Fv2OajhujgqMnkyEBAAREYCZavyOZxaUUqlmPDrBFgZWWFlr5W6NoejAnw4iPP/7d17cBX1FcDx7zEYGFCoEkvRgtApVMFHFWSktlQeUygMMFSpMKMtFc1AgVFsOgYZrRYYlPoYsbyCj9Q6KMgfCCjyUBF0CJSpaYAgDgJieDSGolWp4eHpH7+tydCLWXNz97d37/nMZLibu3f3cNi9h939PcjLc30HjhyBu+/2HY3JJgu2LmBT1SYeHfgoBS3tPn02ynQRmCgiFSLytIicl+F9mTRceaXrTfzkk7Bxo+9oTDY4+OlBil8rpn/n/txyxS2+wzGNlFbrIBFZB6Sa6XgqUAbUAApMA9qr6q0ptlEIFAJ07Nixxwc2ONRXPvziCwA6tGgRyf4+/xwuvxyaN4fycvenMWcy8sWRrHxvJdvGb+P750czzn/U50S2SKd1UFqdxVR1QJj1RGQhsPIM2ygBSsA1EU0nnqSJ+kBv1QrmzXNDTj/4oJuo3phUVuxawdLKpczoNyOyAgD25Z8JmWwdVH/QkBHA9kztK6kWV1ezuLo60n0OHAijR7sWQ+++G+muTZb47PhnTHhlAt0v6E7Rj4oi3bePcyLpMvlMYJaIbBORCqAvMDmD+0qkeQcOMO/Agcj3+9hj0LIljBtnfQfM/7v39Xup+ncVC4cuJD8vP9J9+zonkixjRUBVb1HVy1X1ClUdpqqHMrUv07TatXMT07/5JjzzjO9oTJxsPbiV2VtmM67nOHp36O07HNMErImoSenWW6FPHygqArv6NgAnvzzJ7Stup12rdszsP9N3OKaJWBEwKYnAggWuxdBku5FngMfLHqf8cDmzfz6bNi3a+A7HNBErAuaMLrkEpkyBRYtg9Wrf0Rif9n28j/vW38fQrkO54dIbfIdjmpCNIhpjNcePA1CQH+3Dt/pqa11HsuPHYft298DY5BZVZciiIWz4YAOVEyrp2Kajt1jicE7EkY0imlAF+fneD/bmzd2QEnv3wgMPeA3FeLJkxxJW7V7F9H7TvRYAiMc5kTRWBGKs9NAhSg/5b1TVpw+MHQuPPOImoTG54+h/jnLHq3fQo30PJvWa5Duc2JwTSWJFIMZKDx+m9PBh32EAMGsWtG0LhYVw6pTvaExUitcVU3OshoVDF5J3Vp7vcGJ1TiSFFQETyvnnu2kot2xxQ0uY5Htr/1uU/L2EO6+9k6vaX+U7HJMhVgRMaKNGuWElpkyBqirf0ZhMqj1ZS+GKQi5uczEPXG8Pg5LMioAJTcRdBZw6BZP83x42GTTr7VnsrNnJ3CFzaZXfync4JoOsCJhvpHNnuP9+WLbM/Zjkee/Ie8zYOIObut/E4C42K2zSWT+BGDsWPIFtmef/gVx9J07ANddATQ1UVkLr1r4jMk1FVen3bD/KD5ezc8JOvnNOqulC/InrOeGb9RNIqJZ5ebE82M8+2/UdOHgQpk71HY1pSqXlpazft56HBjwUuwIA8T0nspkVgRibe+AAc2M6bG6vXjBxIsyZA5s3+47GNIWPPv+IorVFXNfhOm67+jbf4aQU53MiW1kRiLEl1dUsifEQntOnw4UXur4DJ074jsak6641d/Fp7aeUDC3hLInnV0Pcz4lsFM9/aZMVWrd2VwIVFW4iGpO91r6/lucqnqP4x8V0u6Cb73BMhNIqAiIyUkR2iMiXItLztPemiMhuEdklIgPTC9PE1fDhMGKEazG0Z4/vaExjHDtxjHEvj6Nr267c85N7fIdjIpbulcB24BfAhvq/FJFuwCigOzAImCsi9jQnoZ54Apo1g/HjbTrKbDTtzWnsObqH+UPm06KZTeSea9IqAqq6U1V3pXhrOPCCqtaq6l5gN9ArnX2Z+LroIpg5E9asgeef9x2N+Sa2/XMbD296mDE/HEPfzn19h2M8aJJ+AiKyHihS1a3B8p+BMlV9Llh+ClilqktTfLYQKAwWL8NdXRgoAGp8BxETlos6los6los6P1DVcxvzwWYNrSAi64BUDYanqupLZ/pYit+lrDaqWgKUBPva2tgOD0ljuahjuahjuahjuagjIo3uZdtgEVDVAY3YbhXQod7yd4GDjdiOMcaYDMpUE9HlwCgRaS4inYEuwJYM7csYY0wjpdtEdISIVAG9gZdFZDWAqu4AlgCVwKvABFUNMxVJSTrxJIzloo7loo7loo7lok6jcxGrAeSMMcZEy3oMG2NMDrMiYIwxOcxLERCRQcFwErtFpDjF+81FZHHw/mYR6RR9lNEIkYu7RKRSRCpE5DURudhHnFFoKBf11rtRRPT0oUqSJEwuROSXwbGxQ0QWRR1jVEKcIx1F5A0ReSc4TxI5E46IPC0i1SKSsi+VOLODPFWIyNWhNqyqkf4AecD7wPeAfOAfQLfT1vktMD94PQpYHHWcMcpFX6Bl8Hp8LuciWO9c3DAlZUBP33F7PC66AO8A5wXL3/Ydt8dclADjg9fdgH2+485QLvoAVwPbz/D+YGAVrp/WtcDmMNv1cSXQC9itqntU9TjwAm6YifqGA38JXi8F+otIqg5o2a7BXKjqG6p6LFgsw/W5SKIwxwXANGAW8EWUwUUsTC5uB+ao6lEAVU3q+MphcqHA/+a3a0NC+ySp6gbgX1+zynDgWXXKgG+JSPuGtuujCFwEfFhvuSr4Xcp1VPUk8AnQNpLoohUmF/WNxVX6JGowFyJyFdBBVVdGGZgHYY6LrkBXEXlbRMpEZFBk0UUrTC7uB24Omqu/AkyKJrTY+abfJ0CIHsMZEGZIidDDTmS50H9PEbkZ6An8NKMR+fO1uRCRs4DHgDFRBeRRmOOiGe6W0PW4q8ONInKZqn6c4diiFiYXo4FSVX1ERHoDfw1y8WXmw4uVRn1v+rgSCDOkxFfriEgz3CXe110GZatQw2uIyABgKjBMVWsjii1qDeXiXNwAg+tFZB/unufyhD4cDnuOvKSqJ9SN1LsLVxSSJkwuxuI6p6Kqm4AWuMHlck2jhuvxUQT+BnQRkc4iko978Lv8tHWWA78OXt8IvK7Bk4+EaTAXwS2QBbgCkNT7vtBALlT1E1UtUNVOqtoJ93xkmAYj1yZMmHNkGa7RACJSgLs9lMRpfcLkYj/QH0BELsUVgY8ijTIelgO/CloJXQt8oqqHGvpQ5LeDVPWkiEwEVuOe/D+tqjtE5I/AVlVdDjyFu6TbjbsCGBV1nFEImYs/AecALwbPxver6jBvQWdIyFzkhJC5WA38TEQqgVPA71X1iL+oMyNkLn4HLBSRybjbH2OS+J9GEXked/uvIHj+8QfgbABVnY97HjIYN3/LMeA3obabwFwZY4wJyXoMG2NMDrMiYIwxOcyKgDHG5DArAsYYk8OsCBhjTA6zImCMMTnMioAxxuSw/wJvKdH74RNWdQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
def NeuralNetLearner(dataset, hidden_layer_sizes=None,\n",
+ " learning_rate=0.01, epochs=100, activation = sigmoid):\n",
+ " """Layered feed-forward network.\n",
+ " hidden_layer_sizes: List of number of hidden units per hidden layer\n",
+ " learning_rate: Learning rate of gradient descent\n",
+ " epochs: Number of passes over the dataset\n",
+ " """\n",
+ "\n",
+ " hidden_layer_sizes = hidden_layer_sizes or [3] # default value\n",
+ " i_units = len(dataset.inputs)\n",
+ " o_units = len(dataset.values[dataset.target])\n",
+ "\n",
+ " # construct a network\n",
+ " raw_net = network(i_units, hidden_layer_sizes, o_units, activation)\n",
+ " learned_net = BackPropagationLearner(dataset, raw_net,\n",
+ " learning_rate, epochs, activation)\n",
+ "\n",
+ " def predict(example):\n",
+ " # Input nodes\n",
+ " i_nodes = learned_net[0]\n",
+ "\n",
+ " # Activate input layer\n",
+ " for v, n in zip(example, i_nodes):\n",
+ " n.value = v\n",
+ "\n",
+ " # Forward pass\n",
+ " for layer in learned_net[1:]:\n",
+ " for node in layer:\n",
+ " inc = [n.value for n in node.inputs]\n",
+ " in_val = dotproduct(inc, node.weights)\n",
+ " node.value = node.activation(in_val)\n",
+ "\n",
+ " # Hypothesis\n",
+ " o_nodes = learned_net[-1]\n",
+ " prediction = find_max_node(o_nodes)\n",
+ " return prediction\n",
+ "\n",
+ " return predict\n",
+ "def BackPropagationLearner(dataset, net, learning_rate, epochs, activation=sigmoid):\n",
+ " """[Figure 18.23] The back-propagation algorithm for multilayer networks"""\n",
+ " # Initialise weights\n",
+ " for layer in net:\n",
+ " for node in layer:\n",
+ " node.weights = random_weights(min_value=-0.5, max_value=0.5,\n",
+ " num_weights=len(node.weights))\n",
+ "\n",
+ " examples = dataset.examples\n",
+ " '''\n",
+ " As of now dataset.target gives an int instead of list,\n",
+ " Changing dataset class will have effect on all the learners.\n",
+ " Will be taken care of later.\n",
+ " '''\n",
+ " o_nodes = net[-1]\n",
+ " i_nodes = net[0]\n",
+ " o_units = len(o_nodes)\n",
+ " idx_t = dataset.target\n",
+ " idx_i = dataset.inputs\n",
+ " n_layers = len(net)\n",
+ "\n",
+ " inputs, targets = init_examples(examples, idx_i, idx_t, o_units)\n",
+ "\n",
+ " for epoch in range(epochs):\n",
+ " # Iterate over each example\n",
+ " for e in range(len(examples)):\n",
+ " i_val = inputs[e]\n",
+ " t_val = targets[e]\n",
+ "\n",
+ " # Activate input layer\n",
+ " for v, n in zip(i_val, i_nodes):\n",
+ " n.value = v\n",
+ "\n",
+ " # Forward pass\n",
+ " for layer in net[1:]:\n",
+ " for node in layer:\n",
+ " inc = [n.value for n in node.inputs]\n",
+ " in_val = dotproduct(inc, node.weights)\n",
+ " node.value = node.activation(in_val)\n",
+ "\n",
+ " # Initialize delta\n",
+ " delta = [[] for _ in range(n_layers)]\n",
+ "\n",
+ " # Compute outer layer delta\n",
+ "\n",
+ " # Error for the MSE cost function\n",
+ " err = [t_val[i] - o_nodes[i].value for i in range(o_units)]\n",
+ "\n",
+ " # The activation function used is relu or sigmoid function\n",
+ " if node.activation == sigmoid:\n",
+ " delta[-1] = [sigmoid_derivative(o_nodes[i].value) * err[i] for i in range(o_units)]\n",
+ " else:\n",
+ " delta[-1] = [relu_derivative(o_nodes[i].value) * err[i] for i in range(o_units)]\n",
+ "\n",
+ " # Backward pass\n",
+ " h_layers = n_layers - 2\n",
+ " for i in range(h_layers, 0, -1):\n",
+ " layer = net[i]\n",
+ " h_units = len(layer)\n",
+ " nx_layer = net[i+1]\n",
+ "\n",
+ " # weights from each ith layer node to each i + 1th layer node\n",
+ " w = [[node.weights[k] for node in nx_layer] for k in range(h_units)]\n",
+ "\n",
+ " if activation == sigmoid:\n",
+ " delta[i] = [sigmoid_derivative(layer[j].value) * dotproduct(w[j], delta[i+1])\n",
+ " for j in range(h_units)]\n",
+ " else:\n",
+ " delta[i] = [relu_derivative(layer[j].value) * dotproduct(w[j], delta[i+1])\n",
+ " for j in range(h_units)]\n",
+ "\n",
+ " # Update weights\n",
+ " for i in range(1, n_layers):\n",
+ " layer = net[i]\n",
+ " inc = [node.value for node in net[i-1]]\n",
+ " units = len(layer)\n",
+ " for j in range(units):\n",
+ " layer[j].weights = vector_add(layer[j].weights,\n",
+ " scalar_vector_product(\n",
+ " learning_rate * delta[i][j], inc))\n",
+ "\n",
+ " return net\n",
+ "
+
+
+
+
+""" # noqa
+
+
+class Canvas:
+ """Inherit from this class to manage the HTML canvas element in jupyter notebooks.
+ To create an object of this class any_name_xyz = Canvas("any_name_xyz")
+ The first argument given must be the name of the object being created.
+ IPython must be able to reference the variable name that is being passed."""
+
+ def __init__(self, varname, width=800, height=600, cid=None):
+ self.name = varname
+ self.cid = cid or varname
+ self.width = width
+ self.height = height
+ self.html = _canvas.format(self.cid, self.width, self.height, self.name)
+ self.exec_list = []
+ display_html(self.html)
+
+ def mouse_click(self, x, y):
+ """Override this method to handle mouse click at position (x, y)"""
+ raise NotImplementedError
+
+ def mouse_move(self, x, y):
+ raise NotImplementedError
+
+ def execute(self, exec_str):
+ """Stores the command to be executed to a list which is used later during update()"""
+ if not isinstance(exec_str, str):
+ print("Invalid execution argument:", exec_str)
+ self.alert("Received invalid execution command format")
+ prefix = "{0}_canvas_object.".format(self.cid)
+ self.exec_list.append(prefix + exec_str + ';')
+
+ def fill(self, r, g, b):
+ """Changes the fill color to a color in rgb format"""
+ self.execute("fill({0}, {1}, {2})".format(r, g, b))
+
+ def stroke(self, r, g, b):
+ """Changes the colors of line/strokes to rgb"""
+ self.execute("stroke({0}, {1}, {2})".format(r, g, b))
+
+ def strokeWidth(self, w):
+ """Changes the width of lines/strokes to 'w' pixels"""
+ self.execute("strokeWidth({0})".format(w))
+
+ def rect(self, x, y, w, h):
+ """Draw a rectangle with 'w' width, 'h' height and (x, y) as the top-left corner"""
+ self.execute("rect({0}, {1}, {2}, {3})".format(x, y, w, h))
+
+ def rect_n(self, xn, yn, wn, hn):
+ """Similar to rect(), but the dimensions are normalized to fall between 0 and 1"""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ w = round(wn * self.width)
+ h = round(hn * self.height)
+ self.rect(x, y, w, h)
+
+ def line(self, x1, y1, x2, y2):
+ """Draw a line from (x1, y1) to (x2, y2)"""
+ self.execute("line({0}, {1}, {2}, {3})".format(x1, y1, x2, y2))
+
+ def line_n(self, x1n, y1n, x2n, y2n):
+ """Similar to line(), but the dimensions are normalized to fall between 0 and 1"""
+ x1 = round(x1n * self.width)
+ y1 = round(y1n * self.height)
+ x2 = round(x2n * self.width)
+ y2 = round(y2n * self.height)
+ self.line(x1, y1, x2, y2)
+
+ def arc(self, x, y, r, start, stop):
+ """Draw an arc with (x, y) as centre, 'r' as radius from angles 'start' to 'stop'"""
+ self.execute("arc({0}, {1}, {2}, {3}, {4})".format(x, y, r, start, stop))
+
+ def arc_n(self, xn, yn, rn, start, stop):
+ """Similar to arc(), but the dimensions are normalized to fall between 0 and 1
+ The normalizing factor for radius is selected between width and height by
+ seeing which is smaller."""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ r = round(rn * min(self.width, self.height))
+ self.arc(x, y, r, start, stop)
+
+ def clear(self):
+ """Clear the HTML canvas"""
+ self.execute("clear()")
+
+ def font(self, font):
+ """Changes the font of text"""
+ self.execute('font("{0}")'.format(font))
+
+ def text(self, txt, x, y, fill=True):
+ """Display a text at (x, y)"""
+ if fill:
+ self.execute('fill_text("{0}", {1}, {2})'.format(txt, x, y))
+ else:
+ self.execute('stroke_text("{0}", {1}, {2})'.format(txt, x, y))
+
+ def text_n(self, txt, xn, yn, fill=True):
+ """Similar to text(), but with normalized coordinates"""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ self.text(txt, x, y, fill)
+
+ def alert(self, message):
+ """Immediately display an alert"""
+ display_html(''.format(message))
+
+ def update(self):
+ """Execute the JS code to execute the commands queued by execute()"""
+ exec_code = ""
+ self.exec_list = []
+ display_html(exec_code)
+
+
+def display_html(html_string):
+ display(HTML(html_string))
+
+
+################################################################################
+
+
+class Canvas_TicTacToe(Canvas):
+ """Play a 3x3 TicTacToe game on HTML canvas"""
+
+ def __init__(self, varname, player_1='human', player_2='random',
+ width=300, height=350, cid=None):
+ valid_players = ('human', 'random', 'alpha_beta')
+ if player_1 not in valid_players or player_2 not in valid_players:
+ raise TypeError("Players must be one of {}".format(valid_players))
+ super().__init__(varname, width, height, cid)
+ self.ttt = TicTacToe()
+ self.state = self.ttt.initial
+ self.turn = 0
+ self.strokeWidth(5)
+ self.players = (player_1, player_2)
+ self.font("20px Arial")
+ self.draw_board()
+
+ def mouse_click(self, x, y):
+ player = self.players[self.turn]
+ if self.ttt.terminal_test(self.state):
+ if 0.55 <= x / self.width <= 0.95 and 6 / 7 <= y / self.height <= 6 / 7 + 1 / 8:
+ self.state = self.ttt.initial
+ self.turn = 0
+ self.draw_board()
+ return
+
+ if player == 'human':
+ x, y = int(3 * x / self.width) + 1, int(3 * y / (self.height * 6 / 7)) + 1
+ if (x, y) not in self.ttt.actions(self.state):
+ # Invalid move
+ return
+ move = (x, y)
+ elif player == 'alpha_beta':
+ move = alpha_beta_player(self.ttt, self.state)
+ else:
+ move = random_player(self.ttt, self.state)
+ self.state = self.ttt.result(self.state, move)
+ self.turn ^= 1
+ self.draw_board()
+
+ def draw_board(self):
+ self.clear()
+ self.stroke(0, 0, 0)
+ offset = 1 / 20
+ self.line_n(0 + offset, (1 / 3) * 6 / 7, 1 - offset, (1 / 3) * 6 / 7)
+ self.line_n(0 + offset, (2 / 3) * 6 / 7, 1 - offset, (2 / 3) * 6 / 7)
+ self.line_n(1 / 3, (0 + offset) * 6 / 7, 1 / 3, (1 - offset) * 6 / 7)
+ self.line_n(2 / 3, (0 + offset) * 6 / 7, 2 / 3, (1 - offset) * 6 / 7)
+
+ board = self.state.board
+ for mark in board:
+ if board[mark] == 'X':
+ self.draw_x(mark)
+ elif board[mark] == 'O':
+ self.draw_o(mark)
+ if self.ttt.terminal_test(self.state):
+ # End game message
+ utility = self.ttt.utility(self.state, self.ttt.to_move(self.ttt.initial))
+ if utility == 0:
+ self.text_n('Game Draw!', offset, 6 / 7 + offset)
+ else:
+ self.text_n('Player {} wins!'.format("XO"[utility < 0]), offset, 6 / 7 + offset)
+ # Find the 3 and draw a line
+ self.stroke([255, 0][self.turn], [0, 255][self.turn], 0)
+ for i in range(3):
+ if all([(i + 1, j + 1) in self.state.board for j in range(3)]) and \
+ len({self.state.board[(i + 1, j + 1)] for j in range(3)}) == 1:
+ self.line_n(i / 3 + 1 / 6, offset * 6 / 7, i / 3 + 1 / 6, (1 - offset) * 6 / 7)
+ if all([(j + 1, i + 1) in self.state.board for j in range(3)]) and \
+ len({self.state.board[(j + 1, i + 1)] for j in range(3)}) == 1:
+ self.line_n(offset, (i / 3 + 1 / 6) * 6 / 7, 1 - offset, (i / 3 + 1 / 6) * 6 / 7)
+ if all([(i + 1, i + 1) in self.state.board for i in range(3)]) and \
+ len({self.state.board[(i + 1, i + 1)] for i in range(3)}) == 1:
+ self.line_n(offset, offset * 6 / 7, 1 - offset, (1 - offset) * 6 / 7)
+ if all([(i + 1, 3 - i) in self.state.board for i in range(3)]) and \
+ len({self.state.board[(i + 1, 3 - i)] for i in range(3)}) == 1:
+ self.line_n(offset, (1 - offset) * 6 / 7, 1 - offset, offset * 6 / 7)
+ # restart button
+ self.fill(0, 0, 255)
+ self.rect_n(0.5 + offset, 6 / 7, 0.4, 1 / 8)
+ self.fill(0, 0, 0)
+ self.text_n('Restart', 0.5 + 2 * offset, 13 / 14)
+ else: # Print which player's turn it is
+ self.text_n("Player {}'s move({})".format("XO"[self.turn], self.players[self.turn]),
+ offset, 6 / 7 + offset)
+
+ self.update()
+
+ def draw_x(self, position):
+ self.stroke(0, 255, 0)
+ x, y = [i - 1 for i in position]
+ offset = 1 / 15
+ self.line_n(x / 3 + offset, (y / 3 + offset) * 6 / 7, x / 3 + 1 / 3 - offset, (y / 3 + 1 / 3 - offset) * 6 / 7)
+ self.line_n(x / 3 + 1 / 3 - offset, (y / 3 + offset) * 6 / 7, x / 3 + offset, (y / 3 + 1 / 3 - offset) * 6 / 7)
+
+ def draw_o(self, position):
+ self.stroke(255, 0, 0)
+ x, y = [i - 1 for i in position]
+ self.arc_n(x / 3 + 1 / 6, (y / 3 + 1 / 6) * 6 / 7, 1 / 9, 0, 360)
+
+
+class Canvas_min_max(Canvas):
+ """MinMax for Fig52Extended on HTML canvas"""
+
+ def __init__(self, varname, util_list, width=800, height=600, cid=None):
+ super.__init__(varname, width, height, cid)
+ self.utils = {node: util for node, util in zip(range(13, 40), util_list)}
+ self.game = Fig52Extended()
+ self.game.utils = self.utils
+ self.nodes = list(range(40))
+ self.l = 1 / 40
+ self.node_pos = {}
+ for i in range(4):
+ base = len(self.node_pos)
+ row_size = 3 ** i
+ for node in [base + j for j in range(row_size)]:
+ self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2,
+ self.l / 2 + (self.l + (1 - 5 * self.l) / 3) * i)
+ self.font("12px Arial")
+ self.node_stack = []
+ self.explored = {node for node in self.utils}
+ self.thick_lines = set()
+ self.change_list = []
+ self.draw_graph()
+ self.stack_manager = self.stack_manager_gen()
+
+ def min_max(self, node):
+ game = self.game
+ player = game.to_move(node)
+
+ def max_value(node):
+ if game.terminal_test(node):
+ return game.utility(node, player)
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ max_a = max(game.actions(node), key=lambda x: min_value(game.result(node, x)))
+ max_node = game.result(node, max_a)
+ self.utils[node] = self.utils[max_node]
+ x1, y1 = self.node_pos[node]
+ x2, y2 = self.node_pos[max_node]
+ self.change_list.append(('l', (node, max_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return self.utils[node]
+
+ def min_value(node):
+ if game.terminal_test(node):
+ return game.utility(node, player)
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ min_a = min(game.actions(node), key=lambda x: max_value(game.result(node, x)))
+ min_node = game.result(node, min_a)
+ self.utils[node] = self.utils[min_node]
+ x1, y1 = self.node_pos[node]
+ x2, y2 = self.node_pos[min_node]
+ self.change_list.append(('l', (node, min_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return self.utils[node]
+
+ return max_value(node)
+
+ def stack_manager_gen(self):
+ self.min_max(0)
+ for change in self.change_list:
+ if change[0] == 'a':
+ self.node_stack.append(change[1])
+ elif change[0] == 'e':
+ self.explored.add(change[1])
+ elif change[0] == 'h':
+ yield
+ elif change[0] == 'l':
+ self.thick_lines.add(change[1])
+ elif change[0] == 'p':
+ self.node_stack.pop()
+
+ def mouse_click(self, x, y):
+ try:
+ self.stack_manager.send(None)
+ except StopIteration:
+ pass
+ self.draw_graph()
+
+ def draw_graph(self):
+ self.clear()
+ # draw nodes
+ self.stroke(0, 0, 0)
+ self.strokeWidth(1)
+ # highlight for nodes in stack
+ for node in self.node_stack:
+ x, y = self.node_pos[node]
+ self.fill(200, 200, 0)
+ self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5)
+ for node in self.nodes:
+ x, y = self.node_pos[node]
+ if node in self.explored:
+ self.fill(255, 255, 255)
+ else:
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.l, self.l)
+ self.line_n(x, y, x + self.l, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.l, y + self.l, x + self.l, y)
+ self.line_n(x + self.l, y + self.l, x, y + self.l)
+ self.fill(0, 0, 0)
+ if node in self.explored:
+ self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10)
+ # draw edges
+ for i in range(13):
+ x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l
+ for j in range(3):
+ x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1]
+ if i in [1, 2, 3]:
+ self.stroke(200, 0, 0)
+ else:
+ self.stroke(0, 200, 0)
+ if (i, j) in self.thick_lines:
+ self.strokeWidth(3)
+ else:
+ self.strokeWidth(1)
+ self.line_n(x1, y1, x2, y2)
+ self.update()
+
+
+class Canvas_alpha_beta(Canvas):
+ """Alpha-beta pruning for Fig52Extended on HTML canvas"""
+
+ def __init__(self, varname, util_list, width=800, height=600, cid=None):
+ super().__init__(varname, width, height, cid)
+ self.utils = {node: util for node, util in zip(range(13, 40), util_list)}
+ self.game = Fig52Extended()
+ self.game.utils = self.utils
+ self.nodes = list(range(40))
+ self.l = 1 / 40
+ self.node_pos = {}
+ for i in range(4):
+ base = len(self.node_pos)
+ row_size = 3 ** i
+ for node in [base + j for j in range(row_size)]:
+ self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2,
+ 3 * self.l / 2 + (self.l + (1 - 6 * self.l) / 3) * i)
+ self.font("12px Arial")
+ self.node_stack = []
+ self.explored = {node for node in self.utils}
+ self.pruned = set()
+ self.ab = {}
+ self.thick_lines = set()
+ self.change_list = []
+ self.draw_graph()
+ self.stack_manager = self.stack_manager_gen()
+
+ def alpha_beta_search(self, node):
+ game = self.game
+ player = game.to_move(node)
+
+ # Functions used by alpha_beta
+ def max_value(node, alpha, beta):
+ if game.terminal_test(node):
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ self.change_list.append(('p',))
+ return game.utility(node, player)
+ v = -np.inf
+ self.change_list.append(('a', node))
+ self.change_list.append(('ab', node, v, beta))
+ self.change_list.append(('h',))
+ for a in game.actions(node):
+ min_val = min_value(game.result(node, a), alpha, beta)
+ if v < min_val:
+ v = min_val
+ max_node = game.result(node, a)
+ self.change_list.append(('ab', node, v, beta))
+ if v >= beta:
+ self.change_list.append(('h',))
+ self.pruned.add(node)
+ break
+ alpha = max(alpha, v)
+ self.utils[node] = v
+ if node not in self.pruned:
+ self.change_list.append(('l', (node, max_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return v
+
+ def min_value(node, alpha, beta):
+ if game.terminal_test(node):
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ self.change_list.append(('p',))
+ return game.utility(node, player)
+ v = np.inf
+ self.change_list.append(('a', node))
+ self.change_list.append(('ab', node, alpha, v))
+ self.change_list.append(('h',))
+ for a in game.actions(node):
+ max_val = max_value(game.result(node, a), alpha, beta)
+ if v > max_val:
+ v = max_val
+ min_node = game.result(node, a)
+ self.change_list.append(('ab', node, alpha, v))
+ if v <= alpha:
+ self.change_list.append(('h',))
+ self.pruned.add(node)
+ break
+ beta = min(beta, v)
+ self.utils[node] = v
+ if node not in self.pruned:
+ self.change_list.append(('l', (node, min_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return v
+
+ return max_value(node, -np.inf, np.inf)
+
+ def stack_manager_gen(self):
+ self.alpha_beta_search(0)
+ for change in self.change_list:
+ if change[0] == 'a':
+ self.node_stack.append(change[1])
+ elif change[0] == 'ab':
+ self.ab[change[1]] = change[2:]
+ elif change[0] == 'e':
+ self.explored.add(change[1])
+ elif change[0] == 'h':
+ yield
+ elif change[0] == 'l':
+ self.thick_lines.add(change[1])
+ elif change[0] == 'p':
+ self.node_stack.pop()
+
+ def mouse_click(self, x, y):
+ try:
+ self.stack_manager.send(None)
+ except StopIteration:
+ pass
+ self.draw_graph()
+
+ def draw_graph(self):
+ self.clear()
+ # draw nodes
+ self.stroke(0, 0, 0)
+ self.strokeWidth(1)
+ # highlight for nodes in stack
+ for node in self.node_stack:
+ x, y = self.node_pos[node]
+ # alpha > beta
+ if node not in self.explored and self.ab[node][0] > self.ab[node][1]:
+ self.fill(200, 100, 100)
+ else:
+ self.fill(200, 200, 0)
+ self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5)
+ for node in self.nodes:
+ x, y = self.node_pos[node]
+ if node in self.explored:
+ if node in self.pruned:
+ self.fill(50, 50, 50)
+ else:
+ self.fill(255, 255, 255)
+ else:
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.l, self.l)
+ self.line_n(x, y, x + self.l, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.l, y + self.l, x + self.l, y)
+ self.line_n(x + self.l, y + self.l, x, y + self.l)
+ self.fill(0, 0, 0)
+ if node in self.explored and node not in self.pruned:
+ self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10)
+ # draw edges
+ for i in range(13):
+ x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l
+ for j in range(3):
+ x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1]
+ if i in [1, 2, 3]:
+ self.stroke(200, 0, 0)
+ else:
+ self.stroke(0, 200, 0)
+ if (i, j) in self.thick_lines:
+ self.strokeWidth(3)
+ else:
+ self.strokeWidth(1)
+ self.line_n(x1, y1, x2, y2)
+ # display alpha and beta
+ for node in self.node_stack:
+ if node not in self.explored:
+ x, y = self.node_pos[node]
+ alpha, beta = self.ab[node]
+ self.text_n(alpha, x - self.l / 2, y - self.l / 10)
+ self.text_n(beta, x + self.l, y - self.l / 10)
+ self.update()
+
+
+class Canvas_fol_bc_ask(Canvas):
+ """fol_bc_ask() on HTML canvas"""
+
+ def __init__(self, varname, kb, query, width=800, height=600, cid=None):
+ super().__init__(varname, width, height, cid)
+ self.kb = kb
+ self.query = query
+ self.l = 1 / 20
+ self.b = 3 * self.l
+ bc_out = list(self.fol_bc_ask())
+ if len(bc_out) == 0:
+ self.valid = False
+ else:
+ self.valid = True
+ graph = bc_out[0][0][0]
+ s = bc_out[0][1]
+ while True:
+ new_graph = subst(s, graph)
+ if graph == new_graph:
+ break
+ graph = new_graph
+ self.make_table(graph)
+ self.context = None
+ self.draw_table()
+
+ def fol_bc_ask(self):
+ KB = self.kb
+ query = self.query
+
+ def fol_bc_or(KB, goal, theta):
+ for rule in KB.fetch_rules_for_goal(goal):
+ lhs, rhs = parse_definite_clause(standardize_variables(rule))
+ for theta1 in fol_bc_and(KB, lhs, unify_mm(rhs, goal, theta)):
+ yield ([(goal, theta1[0])], theta1[1])
+
+ def fol_bc_and(KB, goals, theta):
+ if theta is None:
+ pass
+ elif not goals:
+ yield ([], theta)
+ else:
+ first, rest = goals[0], goals[1:]
+ for theta1 in fol_bc_or(KB, subst(theta, first), theta):
+ for theta2 in fol_bc_and(KB, rest, theta1[1]):
+ yield (theta1[0] + theta2[0], theta2[1])
+
+ return fol_bc_or(KB, query, {})
+
+ def make_table(self, graph):
+ table = []
+ pos = {}
+ links = set()
+ edges = set()
+
+ def dfs(node, depth):
+ if len(table) <= depth:
+ table.append([])
+ pos = len(table[depth])
+ table[depth].append(node[0])
+ for child in node[1]:
+ child_id = dfs(child, depth + 1)
+ links.add(((depth, pos), child_id))
+ return (depth, pos)
+
+ dfs(graph, 0)
+ y_off = 0.85 / len(table)
+ for i, row in enumerate(table):
+ x_off = 0.95 / len(row)
+ for j, node in enumerate(row):
+ pos[(i, j)] = (0.025 + j * x_off + (x_off - self.b) / 2, 0.025 + i * y_off + (y_off - self.l) / 2)
+ for p, c in links:
+ x1, y1 = pos[p]
+ x2, y2 = pos[c]
+ edges.add((x1 + self.b / 2, y1 + self.l, x2 + self.b / 2, y2))
+
+ self.table = table
+ self.pos = pos
+ self.edges = edges
+
+ def mouse_click(self, x, y):
+ x, y = x / self.width, y / self.height
+ for node in self.pos:
+ xs, ys = self.pos[node]
+ xe, ye = xs + self.b, ys + self.l
+ if xs <= x <= xe and ys <= y <= ye:
+ self.context = node
+ break
+ self.draw_table()
+
+ def draw_table(self):
+ self.clear()
+ self.strokeWidth(3)
+ self.stroke(0, 0, 0)
+ self.font("12px Arial")
+ if self.valid:
+ # draw nodes
+ for i, j in self.pos:
+ x, y = self.pos[(i, j)]
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.b, self.l)
+ self.line_n(x, y, x + self.b, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.b, y, x + self.b, y + self.l)
+ self.line_n(x, y + self.l, x + self.b, y + self.l)
+ self.fill(0, 0, 0)
+ self.text_n(self.table[i][j], x + 0.01, y + self.l - 0.01)
+ # draw edges
+ for x1, y1, x2, y2 in self.edges:
+ self.line_n(x1, y1, x2, y2)
+ else:
+ self.fill(255, 0, 0)
+ self.rect_n(0, 0, 1, 1)
+ # text area
+ self.fill(255, 255, 255)
+ self.rect_n(0, 0.9, 1, 0.1)
+ self.strokeWidth(5)
+ self.stroke(0, 0, 0)
+ self.line_n(0, 0.9, 1, 0.9)
+ self.font("22px Arial")
+ self.fill(0, 0, 0)
+ self.text_n(self.table[self.context[0]][self.context[1]] if self.context else "Click for text", 0.025, 0.975)
+ self.update()
+
+
+############################################################################################################
+
+##################### Functions to assist plotting in search.ipynb ####################
+
+############################################################################################################
+
+
+def show_map(graph_data, node_colors=None):
+ G = nx.Graph(graph_data['graph_dict'])
+ node_colors = node_colors or graph_data['node_colors']
+ node_positions = graph_data['node_positions']
+ node_label_pos = graph_data['node_label_positions']
+ edge_weights = graph_data['edge_weights']
+
+ # set the size of the plot
+ plt.figure(figsize=(18, 13))
+ # draw the graph (both nodes and edges) with locations from romania_locations
+ nx.draw(G, pos={k: node_positions[k] for k in G.nodes()},
+ node_color=[node_colors[node] for node in G.nodes()], linewidths=0.3, edgecolors='k')
+
+ # draw labels for nodes
+ node_label_handles = nx.draw_networkx_labels(G, pos=node_label_pos, font_size=14)
+
+ # add a white bounding box behind the node labels
+ [label.set_bbox(dict(facecolor='white', edgecolor='none')) for label in node_label_handles.values()]
+
+ # add edge lables to the graph
+ nx.draw_networkx_edge_labels(G, pos=node_positions, edge_labels=edge_weights, font_size=14)
+
+ # add a legend
+ white_circle = lines.Line2D([], [], color="white", marker='o', markersize=15, markerfacecolor="white")
+ orange_circle = lines.Line2D([], [], color="orange", marker='o', markersize=15, markerfacecolor="orange")
+ red_circle = lines.Line2D([], [], color="red", marker='o', markersize=15, markerfacecolor="red")
+ gray_circle = lines.Line2D([], [], color="gray", marker='o', markersize=15, markerfacecolor="gray")
+ green_circle = lines.Line2D([], [], color="green", marker='o', markersize=15, markerfacecolor="green")
+ plt.legend((white_circle, orange_circle, red_circle, gray_circle, green_circle),
+ ('Un-explored', 'Frontier', 'Currently Exploring', 'Explored', 'Final Solution'),
+ numpoints=1, prop={'size': 16}, loc=(.8, .75))
+
+ # show the plot. No need to use in notebooks. nx.draw will show the graph itself.
+ plt.show()
+
+
+# helper functions for visualisations
+
+def final_path_colors(initial_node_colors, problem, solution):
+ "Return a node_colors dict of the final path provided the problem and solution."
+
+ # get initial node colors
+ final_colors = dict(initial_node_colors)
+ # color all the nodes in solution and starting node to green
+ final_colors[problem.initial] = "green"
+ for node in solution:
+ final_colors[node] = "green"
+ return final_colors
+
+
+def display_visual(graph_data, user_input, algorithm=None, problem=None):
+ initial_node_colors = graph_data['node_colors']
+ if user_input is False:
+ def slider_callback(iteration):
+ # don't show graph for the first time running the cell calling this function
+ try:
+ show_map(graph_data, node_colors=all_node_colors[iteration])
+ except:
+ pass
+
+ def visualize_callback(visualize):
+ if visualize is True:
+ button.value = False
+
+ global all_node_colors
+
+ iterations, all_node_colors, node = algorithm(problem)
+ solution = node.solution()
+ all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution))
+
+ slider.max = len(all_node_colors) - 1
+
+ for i in range(slider.max + 1):
+ slider.value = i
+ # time.sleep(.5)
+
+ slider = widgets.IntSlider(min=0, max=1, step=1, value=0)
+ slider_visual = widgets.interactive(slider_callback, iteration=slider)
+ display(slider_visual)
+
+ button = widgets.ToggleButton(value=False)
+ button_visual = widgets.interactive(visualize_callback, visualize=button)
+ display(button_visual)
+
+ if user_input is True:
+ node_colors = dict(initial_node_colors)
+ if isinstance(algorithm, dict):
+ assert set(algorithm.keys()).issubset({"Breadth First Tree Search",
+ "Depth First Tree Search",
+ "Breadth First Search",
+ "Depth First Graph Search",
+ "Best First Graph Search",
+ "Uniform Cost Search",
+ "Depth Limited Search",
+ "Iterative Deepening Search",
+ "Greedy Best First Search",
+ "A-star Search",
+ "Recursive Best First Search"})
+
+ algo_dropdown = widgets.Dropdown(description="Search algorithm: ",
+ options=sorted(list(algorithm.keys())),
+ value="Breadth First Tree Search")
+ display(algo_dropdown)
+ elif algorithm is None:
+ print("No algorithm to run.")
+ return 0
+
+ def slider_callback(iteration):
+ # don't show graph for the first time running the cell calling this function
+ try:
+ show_map(graph_data, node_colors=all_node_colors[iteration])
+ except:
+ pass
+
+ def visualize_callback(visualize):
+ if visualize is True:
+ button.value = False
+
+ problem = GraphProblem(start_dropdown.value, end_dropdown.value, romania_map)
+ global all_node_colors
+
+ user_algorithm = algorithm[algo_dropdown.value]
+
+ iterations, all_node_colors, node = user_algorithm(problem)
+ solution = node.solution()
+ all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution))
+
+ slider.max = len(all_node_colors) - 1
+
+ for i in range(slider.max + 1):
+ slider.value = i
+ # time.sleep(.5)
+
+ start_dropdown = widgets.Dropdown(description="Start city: ",
+ options=sorted(list(node_colors.keys())), value="Arad")
+ display(start_dropdown)
+
+ end_dropdown = widgets.Dropdown(description="Goal city: ",
+ options=sorted(list(node_colors.keys())), value="Fagaras")
+ display(end_dropdown)
+
+ button = widgets.ToggleButton(value=False)
+ button_visual = widgets.interactive(visualize_callback, visualize=button)
+ display(button_visual)
+
+ slider = widgets.IntSlider(min=0, max=1, step=1, value=0)
+ slider_visual = widgets.interactive(slider_callback, iteration=slider)
+ display(slider_visual)
+
+
+# Function to plot NQueensCSP in csp.py and NQueensProblem in search.py
+def plot_NQueens(solution):
+ n = len(solution)
+ board = np.array([2 * int((i + j) % 2) for j in range(n) for i in range(n)]).reshape((n, n))
+ im = Image.open('images/queen_s.png')
+ height = im.size[1]
+ im = np.array(im).astype(np.float) / 255
+ fig = plt.figure(figsize=(7, 7))
+ ax = fig.add_subplot(111)
+ ax.set_title('{} Queens'.format(n))
+ plt.imshow(board, cmap='binary', interpolation='nearest')
+ # NQueensCSP gives a solution as a dictionary
+ if isinstance(solution, dict):
+ for (k, v) in solution.items():
+ newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1)
+ newax.imshow(im)
+ newax.axis('off')
+ # NQueensProblem gives a solution as a list
+ elif isinstance(solution, list):
+ for (k, v) in enumerate(solution):
+ newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1)
+ newax.imshow(im)
+ newax.axis('off')
+ fig.tight_layout()
+ plt.show()
+
+
+# Function to plot a heatmap, given a grid
+def heatmap(grid, cmap='binary', interpolation='nearest'):
+ fig = plt.figure(figsize=(7, 7))
+ ax = fig.add_subplot(111)
+ ax.set_title('Heatmap')
+ plt.imshow(grid, cmap=cmap, interpolation=interpolation)
+ fig.tight_layout()
+ plt.show()
+
+
+# Generates a gaussian kernel
+def gaussian_kernel(l=5, sig=1.0):
+ ax = np.arange(-l // 2 + 1., l // 2 + 1.)
+ xx, yy = np.meshgrid(ax, ax)
+ kernel = np.exp(-(xx ** 2 + yy ** 2) / (2. * sig ** 2))
+ return kernel
+
+
+# Plots utility function for a POMDP
+def plot_pomdp_utility(utility):
+ save = utility['0'][0]
+ delete = utility['1'][0]
+ ask_save = utility['2'][0]
+ ask_delete = utility['2'][-1]
+ left = (save[0] - ask_save[0]) / (save[0] - ask_save[0] + ask_save[1] - save[1])
+ right = (delete[0] - ask_delete[0]) / (delete[0] - ask_delete[0] + ask_delete[1] - delete[1])
+
+ colors = ['g', 'b', 'k']
+ for action in utility:
+ for value in utility[action]:
+ plt.plot(value, color=colors[int(action)])
+ plt.vlines([left, right], -20, 10, linestyles='dashed', colors='c')
+ plt.ylim(-20, 13)
+ plt.xlim(0, 1)
+ plt.text(left / 2 - 0.05, 10, 'Save')
+ plt.text((right + left) / 2 - 0.02, 10, 'Ask')
+ plt.text((right + 1) / 2 - 0.07, 10, 'Delete')
+ plt.show()
diff --git a/notebook4e.py b/notebook4e.py
new file mode 100644
index 000000000..5b03081c6
--- /dev/null
+++ b/notebook4e.py
@@ -0,0 +1,1158 @@
+import time
+from collections import defaultdict
+from inspect import getsource
+
+import ipywidgets as widgets
+import matplotlib.pyplot as plt
+import networkx as nx
+import numpy as np
+from IPython.display import HTML
+from IPython.display import display
+from PIL import Image
+from matplotlib import lines
+from matplotlib.colors import ListedColormap
+
+from games import TicTacToe, alpha_beta_player, random_player, Fig52Extended
+from learning import DataSet
+from logic import parse_definite_clause, standardize_variables, unify_mm, subst
+from search import GraphProblem, romania_map
+
+
+# ______________________________________________________________________________
+# Magic Words
+
+
+def pseudocode(algorithm):
+ """Print the pseudocode for the given algorithm."""
+ from urllib.request import urlopen
+ from IPython.display import Markdown
+
+ algorithm = algorithm.replace(' ', '-')
+ url = "https://raw.githubusercontent.com/aimacode/aima-pseudocode/master/md/{}.md".format(algorithm)
+ f = urlopen(url)
+ md = f.read().decode('utf-8')
+ md = md.split('\n', 1)[-1].strip()
+ md = '#' + md
+ return Markdown(md)
+
+
+def psource(*functions):
+ """Print the source code for the given function(s)."""
+ source_code = '\n\n'.join(getsource(fn) for fn in functions)
+ try:
+ from pygments.formatters import HtmlFormatter
+ from pygments.lexers import PythonLexer
+ from pygments import highlight
+
+ display(HTML(highlight(source_code, PythonLexer(), HtmlFormatter(full=True))))
+
+ except ImportError:
+ print(source_code)
+
+
+def plot_model_boundary(dataset, attr1, attr2, model=None):
+ # prepare data
+ examples = np.asarray(dataset.examples)
+ X = np.asarray([examples[:, attr1], examples[:, attr2]])
+ y = examples[:, dataset.target]
+ h = 0.1
+
+ # create color maps
+ cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA', '#00AAFF'])
+ cmap_bold = ListedColormap(['#FF0000', '#00FF00', '#00AAFF'])
+
+ # calculate min, max and limits
+ x_min, x_max = X[0].min() - 1, X[0].max() + 1
+ y_min, y_max = X[1].min() - 1, X[1].max() + 1
+ # mesh the grid
+ xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
+ np.arange(y_min, y_max, h))
+ Z = []
+ for grid in zip(xx.ravel(), yy.ravel()):
+ # put them back to the example
+ grid = np.round(grid, decimals=1).tolist()
+ Z.append(model(grid))
+ # Put the result into a color plot
+ Z = np.asarray(Z)
+ Z = Z.reshape(xx.shape)
+ plt.figure()
+ plt.pcolormesh(xx, yy, Z, cmap=cmap_light)
+
+ # Plot also the training points
+ plt.scatter(X[0], X[1], c=y, cmap=cmap_bold)
+ plt.xlim(xx.min(), xx.max())
+ plt.ylim(yy.min(), yy.max())
+ plt.show()
+
+
+# ______________________________________________________________________________
+# Iris Visualization
+
+
+def show_iris(i=0, j=1, k=2):
+ """Plots the iris dataset in a 3D plot.
+ The three axes are given by i, j and k,
+ which correspond to three of the four iris features."""
+
+ plt.rcParams.update(plt.rcParamsDefault)
+
+ fig = plt.figure()
+ ax = fig.add_subplot(111, projection='3d')
+
+ iris = DataSet(name="iris")
+ buckets = iris.split_values_by_classes()
+
+ features = ["Sepal Length", "Sepal Width", "Petal Length", "Petal Width"]
+ f1, f2, f3 = features[i], features[j], features[k]
+
+ a_setosa = [v[i] for v in buckets["setosa"]]
+ b_setosa = [v[j] for v in buckets["setosa"]]
+ c_setosa = [v[k] for v in buckets["setosa"]]
+
+ a_virginica = [v[i] for v in buckets["virginica"]]
+ b_virginica = [v[j] for v in buckets["virginica"]]
+ c_virginica = [v[k] for v in buckets["virginica"]]
+
+ a_versicolor = [v[i] for v in buckets["versicolor"]]
+ b_versicolor = [v[j] for v in buckets["versicolor"]]
+ c_versicolor = [v[k] for v in buckets["versicolor"]]
+
+ for c, m, sl, sw, pl in [('b', 's', a_setosa, b_setosa, c_setosa),
+ ('g', '^', a_virginica, b_virginica, c_virginica),
+ ('r', 'o', a_versicolor, b_versicolor, c_versicolor)]:
+ ax.scatter(sl, sw, pl, c=c, marker=m)
+
+ ax.set_xlabel(f1)
+ ax.set_ylabel(f2)
+ ax.set_zlabel(f3)
+
+ plt.show()
+
+
+# ______________________________________________________________________________
+# MNIST
+
+
+def load_MNIST(path="aima-data/MNIST/Digits", fashion=False):
+ import os, struct
+ import array
+ import numpy as np
+
+ if fashion:
+ path = "aima-data/MNIST/Fashion"
+
+ plt.rcParams.update(plt.rcParamsDefault)
+ plt.rcParams['figure.figsize'] = (10.0, 8.0)
+ plt.rcParams['image.interpolation'] = 'nearest'
+ plt.rcParams['image.cmap'] = 'gray'
+
+ train_img_file = open(os.path.join(path, "train-images-idx3-ubyte"), "rb")
+ train_lbl_file = open(os.path.join(path, "train-labels-idx1-ubyte"), "rb")
+ test_img_file = open(os.path.join(path, "t10k-images-idx3-ubyte"), "rb")
+ test_lbl_file = open(os.path.join(path, 't10k-labels-idx1-ubyte'), "rb")
+
+ magic_nr, tr_size, tr_rows, tr_cols = struct.unpack(">IIII", train_img_file.read(16))
+ tr_img = array.array("B", train_img_file.read())
+ train_img_file.close()
+ magic_nr, tr_size = struct.unpack(">II", train_lbl_file.read(8))
+ tr_lbl = array.array("b", train_lbl_file.read())
+ train_lbl_file.close()
+
+ magic_nr, te_size, te_rows, te_cols = struct.unpack(">IIII", test_img_file.read(16))
+ te_img = array.array("B", test_img_file.read())
+ test_img_file.close()
+ magic_nr, te_size = struct.unpack(">II", test_lbl_file.read(8))
+ te_lbl = array.array("b", test_lbl_file.read())
+ test_lbl_file.close()
+
+ # print(len(tr_img), len(tr_lbl), tr_size)
+ # print(len(te_img), len(te_lbl), te_size)
+
+ train_img = np.zeros((tr_size, tr_rows * tr_cols), dtype=np.int16)
+ train_lbl = np.zeros((tr_size,), dtype=np.int8)
+ for i in range(tr_size):
+ train_img[i] = np.array(tr_img[i * tr_rows * tr_cols: (i + 1) * tr_rows * tr_cols]).reshape((tr_rows * te_cols))
+ train_lbl[i] = tr_lbl[i]
+
+ test_img = np.zeros((te_size, te_rows * te_cols), dtype=np.int16)
+ test_lbl = np.zeros((te_size,), dtype=np.int8)
+ for i in range(te_size):
+ test_img[i] = np.array(te_img[i * te_rows * te_cols: (i + 1) * te_rows * te_cols]).reshape((te_rows * te_cols))
+ test_lbl[i] = te_lbl[i]
+
+ return (train_img, train_lbl, test_img, test_lbl)
+
+
+digit_classes = [str(i) for i in range(10)]
+fashion_classes = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat",
+ "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"]
+
+
+def show_MNIST(labels, images, samples=8, fashion=False):
+ if not fashion:
+ classes = digit_classes
+ else:
+ classes = fashion_classes
+
+ num_classes = len(classes)
+
+ for y, cls in enumerate(classes):
+ idxs = np.nonzero([i == y for i in labels])
+ idxs = np.random.choice(idxs[0], samples, replace=False)
+ for i, idx in enumerate(idxs):
+ plt_idx = i * num_classes + y + 1
+ plt.subplot(samples, num_classes, plt_idx)
+ plt.imshow(images[idx].reshape((28, 28)))
+ plt.axis("off")
+ if i == 0:
+ plt.title(cls)
+
+ plt.show()
+
+
+def show_ave_MNIST(labels, images, fashion=False):
+ if not fashion:
+ item_type = "Digit"
+ classes = digit_classes
+ else:
+ item_type = "Apparel"
+ classes = fashion_classes
+
+ num_classes = len(classes)
+
+ for y, cls in enumerate(classes):
+ idxs = np.nonzero([i == y for i in labels])
+ print(item_type, y, ":", len(idxs[0]), "images.")
+
+ ave_img = np.mean(np.vstack([images[i] for i in idxs[0]]), axis=0)
+ # print(ave_img.shape)
+
+ plt.subplot(1, num_classes, y + 1)
+ plt.imshow(ave_img.reshape((28, 28)))
+ plt.axis("off")
+ plt.title(cls)
+
+ plt.show()
+
+
+# ______________________________________________________________________________
+# MDP
+
+
+def make_plot_grid_step_function(columns, rows, U_over_time):
+ """ipywidgets interactive function supports single parameter as input.
+ This function creates and return such a function by taking as input
+ other parameters."""
+
+ def plot_grid_step(iteration):
+ data = U_over_time[iteration]
+ data = defaultdict(lambda: 0, data)
+ grid = []
+ for row in range(rows):
+ current_row = []
+ for column in range(columns):
+ current_row.append(data[(column, row)])
+ grid.append(current_row)
+ grid.reverse() # output like book
+ fig = plt.imshow(grid, cmap=plt.cm.bwr, interpolation='nearest')
+
+ plt.axis('off')
+ fig.axes.get_xaxis().set_visible(False)
+ fig.axes.get_yaxis().set_visible(False)
+
+ for col in range(len(grid)):
+ for row in range(len(grid[0])):
+ magic = grid[col][row]
+ fig.axes.text(row, col, "{0:.2f}".format(magic), va='center', ha='center')
+
+ plt.show()
+
+ return plot_grid_step
+
+
+def make_visualize(slider):
+ """Takes an input a sliderand returns callback function
+ for timer and animation."""
+
+ def visualize_callback(visualize, time_step):
+ if visualize is True:
+ for i in range(slider.min, slider.max + 1):
+ slider.value = i
+ time.sleep(float(time_step))
+
+ return visualize_callback
+
+
+# ______________________________________________________________________________
+
+
+_canvas = """
+
+
+
+
+
+
+""" # noqa
+
+
+class Canvas:
+ """Inherit from this class to manage the HTML canvas element in jupyter notebooks.
+ To create an object of this class any_name_xyz = Canvas("any_name_xyz")
+ The first argument given must be the name of the object being created.
+ IPython must be able to reference the variable name that is being passed."""
+
+ def __init__(self, varname, width=800, height=600, cid=None):
+ self.name = varname
+ self.cid = cid or varname
+ self.width = width
+ self.height = height
+ self.html = _canvas.format(self.cid, self.width, self.height, self.name)
+ self.exec_list = []
+ display_html(self.html)
+
+ def mouse_click(self, x, y):
+ """Override this method to handle mouse click at position (x, y)"""
+ raise NotImplementedError
+
+ def mouse_move(self, x, y):
+ raise NotImplementedError
+
+ def execute(self, exec_str):
+ """Stores the command to be executed to a list which is used later during update()"""
+ if not isinstance(exec_str, str):
+ print("Invalid execution argument:", exec_str)
+ self.alert("Received invalid execution command format")
+ prefix = "{0}_canvas_object.".format(self.cid)
+ self.exec_list.append(prefix + exec_str + ';')
+
+ def fill(self, r, g, b):
+ """Changes the fill color to a color in rgb format"""
+ self.execute("fill({0}, {1}, {2})".format(r, g, b))
+
+ def stroke(self, r, g, b):
+ """Changes the colors of line/strokes to rgb"""
+ self.execute("stroke({0}, {1}, {2})".format(r, g, b))
+
+ def strokeWidth(self, w):
+ """Changes the width of lines/strokes to 'w' pixels"""
+ self.execute("strokeWidth({0})".format(w))
+
+ def rect(self, x, y, w, h):
+ """Draw a rectangle with 'w' width, 'h' height and (x, y) as the top-left corner"""
+ self.execute("rect({0}, {1}, {2}, {3})".format(x, y, w, h))
+
+ def rect_n(self, xn, yn, wn, hn):
+ """Similar to rect(), but the dimensions are normalized to fall between 0 and 1"""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ w = round(wn * self.width)
+ h = round(hn * self.height)
+ self.rect(x, y, w, h)
+
+ def line(self, x1, y1, x2, y2):
+ """Draw a line from (x1, y1) to (x2, y2)"""
+ self.execute("line({0}, {1}, {2}, {3})".format(x1, y1, x2, y2))
+
+ def line_n(self, x1n, y1n, x2n, y2n):
+ """Similar to line(), but the dimensions are normalized to fall between 0 and 1"""
+ x1 = round(x1n * self.width)
+ y1 = round(y1n * self.height)
+ x2 = round(x2n * self.width)
+ y2 = round(y2n * self.height)
+ self.line(x1, y1, x2, y2)
+
+ def arc(self, x, y, r, start, stop):
+ """Draw an arc with (x, y) as centre, 'r' as radius from angles 'start' to 'stop'"""
+ self.execute("arc({0}, {1}, {2}, {3}, {4})".format(x, y, r, start, stop))
+
+ def arc_n(self, xn, yn, rn, start, stop):
+ """Similar to arc(), but the dimensions are normalized to fall between 0 and 1
+ The normalizing factor for radius is selected between width and height by
+ seeing which is smaller."""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ r = round(rn * min(self.width, self.height))
+ self.arc(x, y, r, start, stop)
+
+ def clear(self):
+ """Clear the HTML canvas"""
+ self.execute("clear()")
+
+ def font(self, font):
+ """Changes the font of text"""
+ self.execute('font("{0}")'.format(font))
+
+ def text(self, txt, x, y, fill=True):
+ """Display a text at (x, y)"""
+ if fill:
+ self.execute('fill_text("{0}", {1}, {2})'.format(txt, x, y))
+ else:
+ self.execute('stroke_text("{0}", {1}, {2})'.format(txt, x, y))
+
+ def text_n(self, txt, xn, yn, fill=True):
+ """Similar to text(), but with normalized coordinates"""
+ x = round(xn * self.width)
+ y = round(yn * self.height)
+ self.text(txt, x, y, fill)
+
+ def alert(self, message):
+ """Immediately display an alert"""
+ display_html(''.format(message))
+
+ def update(self):
+ """Execute the JS code to execute the commands queued by execute()"""
+ exec_code = ""
+ self.exec_list = []
+ display_html(exec_code)
+
+
+def display_html(html_string):
+ display(HTML(html_string))
+
+
+################################################################################
+
+
+class Canvas_TicTacToe(Canvas):
+ """Play a 3x3 TicTacToe game on HTML canvas"""
+
+ def __init__(self, varname, player_1='human', player_2='random',
+ width=300, height=350, cid=None):
+ valid_players = ('human', 'random', 'alpha_beta')
+ if player_1 not in valid_players or player_2 not in valid_players:
+ raise TypeError("Players must be one of {}".format(valid_players))
+ super().__init__(varname, width, height, cid)
+ self.ttt = TicTacToe()
+ self.state = self.ttt.initial
+ self.turn = 0
+ self.strokeWidth(5)
+ self.players = (player_1, player_2)
+ self.font("20px Arial")
+ self.draw_board()
+
+ def mouse_click(self, x, y):
+ player = self.players[self.turn]
+ if self.ttt.terminal_test(self.state):
+ if 0.55 <= x / self.width <= 0.95 and 6 / 7 <= y / self.height <= 6 / 7 + 1 / 8:
+ self.state = self.ttt.initial
+ self.turn = 0
+ self.draw_board()
+ return
+
+ if player == 'human':
+ x, y = int(3 * x / self.width) + 1, int(3 * y / (self.height * 6 / 7)) + 1
+ if (x, y) not in self.ttt.actions(self.state):
+ # Invalid move
+ return
+ move = (x, y)
+ elif player == 'alpha_beta':
+ move = alpha_beta_player(self.ttt, self.state)
+ else:
+ move = random_player(self.ttt, self.state)
+ self.state = self.ttt.result(self.state, move)
+ self.turn ^= 1
+ self.draw_board()
+
+ def draw_board(self):
+ self.clear()
+ self.stroke(0, 0, 0)
+ offset = 1 / 20
+ self.line_n(0 + offset, (1 / 3) * 6 / 7, 1 - offset, (1 / 3) * 6 / 7)
+ self.line_n(0 + offset, (2 / 3) * 6 / 7, 1 - offset, (2 / 3) * 6 / 7)
+ self.line_n(1 / 3, (0 + offset) * 6 / 7, 1 / 3, (1 - offset) * 6 / 7)
+ self.line_n(2 / 3, (0 + offset) * 6 / 7, 2 / 3, (1 - offset) * 6 / 7)
+
+ board = self.state.board
+ for mark in board:
+ if board[mark] == 'X':
+ self.draw_x(mark)
+ elif board[mark] == 'O':
+ self.draw_o(mark)
+ if self.ttt.terminal_test(self.state):
+ # End game message
+ utility = self.ttt.utility(self.state, self.ttt.to_move(self.ttt.initial))
+ if utility == 0:
+ self.text_n('Game Draw!', offset, 6 / 7 + offset)
+ else:
+ self.text_n('Player {} wins!'.format("XO"[utility < 0]), offset, 6 / 7 + offset)
+ # Find the 3 and draw a line
+ self.stroke([255, 0][self.turn], [0, 255][self.turn], 0)
+ for i in range(3):
+ if all([(i + 1, j + 1) in self.state.board for j in range(3)]) and \
+ len({self.state.board[(i + 1, j + 1)] for j in range(3)}) == 1:
+ self.line_n(i / 3 + 1 / 6, offset * 6 / 7, i / 3 + 1 / 6, (1 - offset) * 6 / 7)
+ if all([(j + 1, i + 1) in self.state.board for j in range(3)]) and \
+ len({self.state.board[(j + 1, i + 1)] for j in range(3)}) == 1:
+ self.line_n(offset, (i / 3 + 1 / 6) * 6 / 7, 1 - offset, (i / 3 + 1 / 6) * 6 / 7)
+ if all([(i + 1, i + 1) in self.state.board for i in range(3)]) and \
+ len({self.state.board[(i + 1, i + 1)] for i in range(3)}) == 1:
+ self.line_n(offset, offset * 6 / 7, 1 - offset, (1 - offset) * 6 / 7)
+ if all([(i + 1, 3 - i) in self.state.board for i in range(3)]) and \
+ len({self.state.board[(i + 1, 3 - i)] for i in range(3)}) == 1:
+ self.line_n(offset, (1 - offset) * 6 / 7, 1 - offset, offset * 6 / 7)
+ # restart button
+ self.fill(0, 0, 255)
+ self.rect_n(0.5 + offset, 6 / 7, 0.4, 1 / 8)
+ self.fill(0, 0, 0)
+ self.text_n('Restart', 0.5 + 2 * offset, 13 / 14)
+ else: # Print which player's turn it is
+ self.text_n("Player {}'s move({})".format("XO"[self.turn], self.players[self.turn]),
+ offset, 6 / 7 + offset)
+
+ self.update()
+
+ def draw_x(self, position):
+ self.stroke(0, 255, 0)
+ x, y = [i - 1 for i in position]
+ offset = 1 / 15
+ self.line_n(x / 3 + offset, (y / 3 + offset) * 6 / 7, x / 3 + 1 / 3 - offset, (y / 3 + 1 / 3 - offset) * 6 / 7)
+ self.line_n(x / 3 + 1 / 3 - offset, (y / 3 + offset) * 6 / 7, x / 3 + offset, (y / 3 + 1 / 3 - offset) * 6 / 7)
+
+ def draw_o(self, position):
+ self.stroke(255, 0, 0)
+ x, y = [i - 1 for i in position]
+ self.arc_n(x / 3 + 1 / 6, (y / 3 + 1 / 6) * 6 / 7, 1 / 9, 0, 360)
+
+
+class Canvas_min_max(Canvas):
+ """MinMax for Fig52Extended on HTML canvas"""
+
+ def __init__(self, varname, util_list, width=800, height=600, cid=None):
+ super().__init__(varname, width, height, cid)
+ self.utils = {node: util for node, util in zip(range(13, 40), util_list)}
+ self.game = Fig52Extended()
+ self.game.utils = self.utils
+ self.nodes = list(range(40))
+ self.l = 1 / 40
+ self.node_pos = {}
+ for i in range(4):
+ base = len(self.node_pos)
+ row_size = 3 ** i
+ for node in [base + j for j in range(row_size)]:
+ self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2,
+ self.l / 2 + (self.l + (1 - 5 * self.l) / 3) * i)
+ self.font("12px Arial")
+ self.node_stack = []
+ self.explored = {node for node in self.utils}
+ self.thick_lines = set()
+ self.change_list = []
+ self.draw_graph()
+ self.stack_manager = self.stack_manager_gen()
+
+ def min_max(self, node):
+ game = self.game
+ player = game.to_move(node)
+
+ def max_value(node):
+ if game.terminal_test(node):
+ return game.utility(node, player)
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ max_a = max(game.actions(node), key=lambda x: min_value(game.result(node, x)))
+ max_node = game.result(node, max_a)
+ self.utils[node] = self.utils[max_node]
+ x1, y1 = self.node_pos[node]
+ x2, y2 = self.node_pos[max_node]
+ self.change_list.append(('l', (node, max_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return self.utils[node]
+
+ def min_value(node):
+ if game.terminal_test(node):
+ return game.utility(node, player)
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ min_a = min(game.actions(node), key=lambda x: max_value(game.result(node, x)))
+ min_node = game.result(node, min_a)
+ self.utils[node] = self.utils[min_node]
+ x1, y1 = self.node_pos[node]
+ x2, y2 = self.node_pos[min_node]
+ self.change_list.append(('l', (node, min_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return self.utils[node]
+
+ return max_value(node)
+
+ def stack_manager_gen(self):
+ self.min_max(0)
+ for change in self.change_list:
+ if change[0] == 'a':
+ self.node_stack.append(change[1])
+ elif change[0] == 'e':
+ self.explored.add(change[1])
+ elif change[0] == 'h':
+ yield
+ elif change[0] == 'l':
+ self.thick_lines.add(change[1])
+ elif change[0] == 'p':
+ self.node_stack.pop()
+
+ def mouse_click(self, x, y):
+ try:
+ self.stack_manager.send(None)
+ except StopIteration:
+ pass
+ self.draw_graph()
+
+ def draw_graph(self):
+ self.clear()
+ # draw nodes
+ self.stroke(0, 0, 0)
+ self.strokeWidth(1)
+ # highlight for nodes in stack
+ for node in self.node_stack:
+ x, y = self.node_pos[node]
+ self.fill(200, 200, 0)
+ self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5)
+ for node in self.nodes:
+ x, y = self.node_pos[node]
+ if node in self.explored:
+ self.fill(255, 255, 255)
+ else:
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.l, self.l)
+ self.line_n(x, y, x + self.l, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.l, y + self.l, x + self.l, y)
+ self.line_n(x + self.l, y + self.l, x, y + self.l)
+ self.fill(0, 0, 0)
+ if node in self.explored:
+ self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10)
+ # draw edges
+ for i in range(13):
+ x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l
+ for j in range(3):
+ x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1]
+ if i in [1, 2, 3]:
+ self.stroke(200, 0, 0)
+ else:
+ self.stroke(0, 200, 0)
+ if (i, j) in self.thick_lines:
+ self.strokeWidth(3)
+ else:
+ self.strokeWidth(1)
+ self.line_n(x1, y1, x2, y2)
+ self.update()
+
+
+class Canvas_alpha_beta(Canvas):
+ """Alpha-beta pruning for Fig52Extended on HTML canvas"""
+
+ def __init__(self, varname, util_list, width=800, height=600, cid=None):
+ super().__init__(varname, width, height, cid)
+ self.utils = {node: util for node, util in zip(range(13, 40), util_list)}
+ self.game = Fig52Extended()
+ self.game.utils = self.utils
+ self.nodes = list(range(40))
+ self.l = 1 / 40
+ self.node_pos = {}
+ for i in range(4):
+ base = len(self.node_pos)
+ row_size = 3 ** i
+ for node in [base + j for j in range(row_size)]:
+ self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2,
+ 3 * self.l / 2 + (self.l + (1 - 6 * self.l) / 3) * i)
+ self.font("12px Arial")
+ self.node_stack = []
+ self.explored = {node for node in self.utils}
+ self.pruned = set()
+ self.ab = {}
+ self.thick_lines = set()
+ self.change_list = []
+ self.draw_graph()
+ self.stack_manager = self.stack_manager_gen()
+
+ def alpha_beta_search(self, node):
+ game = self.game
+ player = game.to_move(node)
+
+ # Functions used by alpha_beta
+ def max_value(node, alpha, beta):
+ if game.terminal_test(node):
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ self.change_list.append(('p',))
+ return game.utility(node, player)
+ v = -np.inf
+ self.change_list.append(('a', node))
+ self.change_list.append(('ab', node, v, beta))
+ self.change_list.append(('h',))
+ for a in game.actions(node):
+ min_val = min_value(game.result(node, a), alpha, beta)
+ if v < min_val:
+ v = min_val
+ max_node = game.result(node, a)
+ self.change_list.append(('ab', node, v, beta))
+ if v >= beta:
+ self.change_list.append(('h',))
+ self.pruned.add(node)
+ break
+ alpha = max(alpha, v)
+ self.utils[node] = v
+ if node not in self.pruned:
+ self.change_list.append(('l', (node, max_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return v
+
+ def min_value(node, alpha, beta):
+ if game.terminal_test(node):
+ self.change_list.append(('a', node))
+ self.change_list.append(('h',))
+ self.change_list.append(('p',))
+ return game.utility(node, player)
+ v = np.inf
+ self.change_list.append(('a', node))
+ self.change_list.append(('ab', node, alpha, v))
+ self.change_list.append(('h',))
+ for a in game.actions(node):
+ max_val = max_value(game.result(node, a), alpha, beta)
+ if v > max_val:
+ v = max_val
+ min_node = game.result(node, a)
+ self.change_list.append(('ab', node, alpha, v))
+ if v <= alpha:
+ self.change_list.append(('h',))
+ self.pruned.add(node)
+ break
+ beta = min(beta, v)
+ self.utils[node] = v
+ if node not in self.pruned:
+ self.change_list.append(('l', (node, min_node - 3 * node - 1)))
+ self.change_list.append(('e', node))
+ self.change_list.append(('p',))
+ self.change_list.append(('h',))
+ return v
+
+ return max_value(node, -np.inf, np.inf)
+
+ def stack_manager_gen(self):
+ self.alpha_beta_search(0)
+ for change in self.change_list:
+ if change[0] == 'a':
+ self.node_stack.append(change[1])
+ elif change[0] == 'ab':
+ self.ab[change[1]] = change[2:]
+ elif change[0] == 'e':
+ self.explored.add(change[1])
+ elif change[0] == 'h':
+ yield
+ elif change[0] == 'l':
+ self.thick_lines.add(change[1])
+ elif change[0] == 'p':
+ self.node_stack.pop()
+
+ def mouse_click(self, x, y):
+ try:
+ self.stack_manager.send(None)
+ except StopIteration:
+ pass
+ self.draw_graph()
+
+ def draw_graph(self):
+ self.clear()
+ # draw nodes
+ self.stroke(0, 0, 0)
+ self.strokeWidth(1)
+ # highlight for nodes in stack
+ for node in self.node_stack:
+ x, y = self.node_pos[node]
+ # alpha > beta
+ if node not in self.explored and self.ab[node][0] > self.ab[node][1]:
+ self.fill(200, 100, 100)
+ else:
+ self.fill(200, 200, 0)
+ self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5)
+ for node in self.nodes:
+ x, y = self.node_pos[node]
+ if node in self.explored:
+ if node in self.pruned:
+ self.fill(50, 50, 50)
+ else:
+ self.fill(255, 255, 255)
+ else:
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.l, self.l)
+ self.line_n(x, y, x + self.l, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.l, y + self.l, x + self.l, y)
+ self.line_n(x + self.l, y + self.l, x, y + self.l)
+ self.fill(0, 0, 0)
+ if node in self.explored and node not in self.pruned:
+ self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10)
+ # draw edges
+ for i in range(13):
+ x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l
+ for j in range(3):
+ x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1]
+ if i in [1, 2, 3]:
+ self.stroke(200, 0, 0)
+ else:
+ self.stroke(0, 200, 0)
+ if (i, j) in self.thick_lines:
+ self.strokeWidth(3)
+ else:
+ self.strokeWidth(1)
+ self.line_n(x1, y1, x2, y2)
+ # display alpha and beta
+ for node in self.node_stack:
+ if node not in self.explored:
+ x, y = self.node_pos[node]
+ alpha, beta = self.ab[node]
+ self.text_n(alpha, x - self.l / 2, y - self.l / 10)
+ self.text_n(beta, x + self.l, y - self.l / 10)
+ self.update()
+
+
+class Canvas_fol_bc_ask(Canvas):
+ """fol_bc_ask() on HTML canvas"""
+
+ def __init__(self, varname, kb, query, width=800, height=600, cid=None):
+ super().__init__(varname, width, height, cid)
+ self.kb = kb
+ self.query = query
+ self.l = 1 / 20
+ self.b = 3 * self.l
+ bc_out = list(self.fol_bc_ask())
+ if len(bc_out) == 0:
+ self.valid = False
+ else:
+ self.valid = True
+ graph = bc_out[0][0][0]
+ s = bc_out[0][1]
+ while True:
+ new_graph = subst(s, graph)
+ if graph == new_graph:
+ break
+ graph = new_graph
+ self.make_table(graph)
+ self.context = None
+ self.draw_table()
+
+ def fol_bc_ask(self):
+ KB = self.kb
+ query = self.query
+
+ def fol_bc_or(KB, goal, theta):
+ for rule in KB.fetch_rules_for_goal(goal):
+ lhs, rhs = parse_definite_clause(standardize_variables(rule))
+ for theta1 in fol_bc_and(KB, lhs, unify_mm(rhs, goal, theta)):
+ yield ([(goal, theta1[0])], theta1[1])
+
+ def fol_bc_and(KB, goals, theta):
+ if theta is None:
+ pass
+ elif not goals:
+ yield ([], theta)
+ else:
+ first, rest = goals[0], goals[1:]
+ for theta1 in fol_bc_or(KB, subst(theta, first), theta):
+ for theta2 in fol_bc_and(KB, rest, theta1[1]):
+ yield (theta1[0] + theta2[0], theta2[1])
+
+ return fol_bc_or(KB, query, {})
+
+ def make_table(self, graph):
+ table = []
+ pos = {}
+ links = set()
+ edges = set()
+
+ def dfs(node, depth):
+ if len(table) <= depth:
+ table.append([])
+ pos = len(table[depth])
+ table[depth].append(node[0])
+ for child in node[1]:
+ child_id = dfs(child, depth + 1)
+ links.add(((depth, pos), child_id))
+ return (depth, pos)
+
+ dfs(graph, 0)
+ y_off = 0.85 / len(table)
+ for i, row in enumerate(table):
+ x_off = 0.95 / len(row)
+ for j, node in enumerate(row):
+ pos[(i, j)] = (0.025 + j * x_off + (x_off - self.b) / 2, 0.025 + i * y_off + (y_off - self.l) / 2)
+ for p, c in links:
+ x1, y1 = pos[p]
+ x2, y2 = pos[c]
+ edges.add((x1 + self.b / 2, y1 + self.l, x2 + self.b / 2, y2))
+
+ self.table = table
+ self.pos = pos
+ self.edges = edges
+
+ def mouse_click(self, x, y):
+ x, y = x / self.width, y / self.height
+ for node in self.pos:
+ xs, ys = self.pos[node]
+ xe, ye = xs + self.b, ys + self.l
+ if xs <= x <= xe and ys <= y <= ye:
+ self.context = node
+ break
+ self.draw_table()
+
+ def draw_table(self):
+ self.clear()
+ self.strokeWidth(3)
+ self.stroke(0, 0, 0)
+ self.font("12px Arial")
+ if self.valid:
+ # draw nodes
+ for i, j in self.pos:
+ x, y = self.pos[(i, j)]
+ self.fill(200, 200, 200)
+ self.rect_n(x, y, self.b, self.l)
+ self.line_n(x, y, x + self.b, y)
+ self.line_n(x, y, x, y + self.l)
+ self.line_n(x + self.b, y, x + self.b, y + self.l)
+ self.line_n(x, y + self.l, x + self.b, y + self.l)
+ self.fill(0, 0, 0)
+ self.text_n(self.table[i][j], x + 0.01, y + self.l - 0.01)
+ # draw edges
+ for x1, y1, x2, y2 in self.edges:
+ self.line_n(x1, y1, x2, y2)
+ else:
+ self.fill(255, 0, 0)
+ self.rect_n(0, 0, 1, 1)
+ # text area
+ self.fill(255, 255, 255)
+ self.rect_n(0, 0.9, 1, 0.1)
+ self.strokeWidth(5)
+ self.stroke(0, 0, 0)
+ self.line_n(0, 0.9, 1, 0.9)
+ self.font("22px Arial")
+ self.fill(0, 0, 0)
+ self.text_n(self.table[self.context[0]][self.context[1]] if self.context else "Click for text", 0.025, 0.975)
+ self.update()
+
+
+############################################################################################################
+
+##################### Functions to assist plotting in search.ipynb ####################
+
+############################################################################################################
+
+
+def show_map(graph_data, node_colors=None):
+ G = nx.Graph(graph_data['graph_dict'])
+ node_colors = node_colors or graph_data['node_colors']
+ node_positions = graph_data['node_positions']
+ node_label_pos = graph_data['node_label_positions']
+ edge_weights = graph_data['edge_weights']
+
+ # set the size of the plot
+ plt.figure(figsize=(18, 13))
+ # draw the graph (both nodes and edges) with locations from romania_locations
+ nx.draw(G, pos={k: node_positions[k] for k in G.nodes()},
+ node_color=[node_colors[node] for node in G.nodes()], linewidths=0.3, edgecolors='k')
+
+ # draw labels for nodes
+ node_label_handles = nx.draw_networkx_labels(G, pos=node_label_pos, font_size=14)
+
+ # add a white bounding box behind the node labels
+ [label.set_bbox(dict(facecolor='white', edgecolor='none')) for label in node_label_handles.values()]
+
+ # add edge lables to the graph
+ nx.draw_networkx_edge_labels(G, pos=node_positions, edge_labels=edge_weights, font_size=14)
+
+ # add a legend
+ white_circle = lines.Line2D([], [], color="white", marker='o', markersize=15, markerfacecolor="white")
+ orange_circle = lines.Line2D([], [], color="orange", marker='o', markersize=15, markerfacecolor="orange")
+ red_circle = lines.Line2D([], [], color="red", marker='o', markersize=15, markerfacecolor="red")
+ gray_circle = lines.Line2D([], [], color="gray", marker='o', markersize=15, markerfacecolor="gray")
+ green_circle = lines.Line2D([], [], color="green", marker='o', markersize=15, markerfacecolor="green")
+ plt.legend((white_circle, orange_circle, red_circle, gray_circle, green_circle),
+ ('Un-explored', 'Frontier', 'Currently Exploring', 'Explored', 'Final Solution'),
+ numpoints=1, prop={'size': 16}, loc=(.8, .75))
+
+ # show the plot. No need to use in notebooks. nx.draw will show the graph itself.
+ plt.show()
+
+
+# helper functions for visualisations
+
+def final_path_colors(initial_node_colors, problem, solution):
+ """Return a node_colors dict of the final path provided the problem and solution."""
+
+ # get initial node colors
+ final_colors = dict(initial_node_colors)
+ # color all the nodes in solution and starting node to green
+ final_colors[problem.initial] = "green"
+ for node in solution:
+ final_colors[node] = "green"
+ return final_colors
+
+
+def display_visual(graph_data, user_input, algorithm=None, problem=None):
+ initial_node_colors = graph_data['node_colors']
+ if user_input is False:
+ def slider_callback(iteration):
+ # don't show graph for the first time running the cell calling this function
+ try:
+ show_map(graph_data, node_colors=all_node_colors[iteration])
+ except:
+ pass
+
+ def visualize_callback(visualize):
+ if visualize is True:
+ button.value = False
+
+ global all_node_colors
+
+ iterations, all_node_colors, node = algorithm(problem)
+ solution = node.solution()
+ all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution))
+
+ slider.max = len(all_node_colors) - 1
+
+ for i in range(slider.max + 1):
+ slider.value = i
+ # time.sleep(.5)
+
+ slider = widgets.IntSlider(min=0, max=1, step=1, value=0)
+ slider_visual = widgets.interactive(slider_callback, iteration=slider)
+ display(slider_visual)
+
+ button = widgets.ToggleButton(value=False)
+ button_visual = widgets.interactive(visualize_callback, visualize=button)
+ display(button_visual)
+
+ if user_input is True:
+ node_colors = dict(initial_node_colors)
+ if isinstance(algorithm, dict):
+ assert set(algorithm.keys()).issubset({"Breadth First Tree Search",
+ "Depth First Tree Search",
+ "Breadth First Search",
+ "Depth First Graph Search",
+ "Best First Graph Search",
+ "Uniform Cost Search",
+ "Depth Limited Search",
+ "Iterative Deepening Search",
+ "Greedy Best First Search",
+ "A-star Search",
+ "Recursive Best First Search"})
+
+ algo_dropdown = widgets.Dropdown(description="Search algorithm: ",
+ options=sorted(list(algorithm.keys())),
+ value="Breadth First Tree Search")
+ display(algo_dropdown)
+ elif algorithm is None:
+ print("No algorithm to run.")
+ return 0
+
+ def slider_callback(iteration):
+ # don't show graph for the first time running the cell calling this function
+ try:
+ show_map(graph_data, node_colors=all_node_colors[iteration])
+ except:
+ pass
+
+ def visualize_callback(visualize):
+ if visualize is True:
+ button.value = False
+
+ problem = GraphProblem(start_dropdown.value, end_dropdown.value, romania_map)
+ global all_node_colors
+
+ user_algorithm = algorithm[algo_dropdown.value]
+
+ iterations, all_node_colors, node = user_algorithm(problem)
+ solution = node.solution()
+ all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution))
+
+ slider.max = len(all_node_colors) - 1
+
+ for i in range(slider.max + 1):
+ slider.value = i
+ # time.sleep(.5)
+
+ start_dropdown = widgets.Dropdown(description="Start city: ",
+ options=sorted(list(node_colors.keys())), value="Arad")
+ display(start_dropdown)
+
+ end_dropdown = widgets.Dropdown(description="Goal city: ",
+ options=sorted(list(node_colors.keys())), value="Fagaras")
+ display(end_dropdown)
+
+ button = widgets.ToggleButton(value=False)
+ button_visual = widgets.interactive(visualize_callback, visualize=button)
+ display(button_visual)
+
+ slider = widgets.IntSlider(min=0, max=1, step=1, value=0)
+ slider_visual = widgets.interactive(slider_callback, iteration=slider)
+ display(slider_visual)
+
+
+# Function to plot NQueensCSP in csp.py and NQueensProblem in search.py
+def plot_NQueens(solution):
+ n = len(solution)
+ board = np.array([2 * int((i + j) % 2) for j in range(n) for i in range(n)]).reshape((n, n))
+ im = Image.open('images/queen_s.png')
+ height = im.size[1]
+ im = np.array(im).astype(np.float) / 255
+ fig = plt.figure(figsize=(7, 7))
+ ax = fig.add_subplot(111)
+ ax.set_title('{} Queens'.format(n))
+ plt.imshow(board, cmap='binary', interpolation='nearest')
+ # NQueensCSP gives a solution as a dictionary
+ if isinstance(solution, dict):
+ for (k, v) in solution.items():
+ newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1)
+ newax.imshow(im)
+ newax.axis('off')
+ # NQueensProblem gives a solution as a list
+ elif isinstance(solution, list):
+ for (k, v) in enumerate(solution):
+ newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1)
+ newax.imshow(im)
+ newax.axis('off')
+ fig.tight_layout()
+ plt.show()
+
+
+# Function to plot a heatmap, given a grid
+def heatmap(grid, cmap='binary', interpolation='nearest'):
+ fig = plt.figure(figsize=(7, 7))
+ ax = fig.add_subplot(111)
+ ax.set_title('Heatmap')
+ plt.imshow(grid, cmap=cmap, interpolation=interpolation)
+ fig.tight_layout()
+ plt.show()
+
+
+# Generates a gaussian kernel
+def gaussian_kernel(l=5, sig=1.0):
+ ax = np.arange(-l // 2 + 1., l // 2 + 1.)
+ xx, yy = np.meshgrid(ax, ax)
+ kernel = np.exp(-(xx ** 2 + yy ** 2) / (2. * sig ** 2))
+ return kernel
+
+
+# Plots utility function for a POMDP
+def plot_pomdp_utility(utility):
+ save = utility['0'][0]
+ delete = utility['1'][0]
+ ask_save = utility['2'][0]
+ ask_delete = utility['2'][-1]
+ left = (save[0] - ask_save[0]) / (save[0] - ask_save[0] + ask_save[1] - save[1])
+ right = (delete[0] - ask_delete[0]) / (delete[0] - ask_delete[0] + ask_delete[1] - delete[1])
+
+ colors = ['g', 'b', 'k']
+ for action in utility:
+ for value in utility[action]:
+ plt.plot(value, color=colors[int(action)])
+ plt.vlines([left, right], -20, 10, linestyles='dashed', colors='c')
+ plt.ylim(-20, 13)
+ plt.xlim(0, 1)
+ plt.text(left / 2 - 0.05, 10, 'Save')
+ plt.text((right + left) / 2 - 0.02, 10, 'Ask')
+ plt.text((right + 1) / 2 - 0.07, 10, 'Delete')
+ plt.show()
diff --git a/notebooks/chapter19/Learners.ipynb b/notebooks/chapter19/Learners.ipynb
new file mode 100644
index 000000000..c6f3d1e4f
--- /dev/null
+++ b/notebooks/chapter19/Learners.ipynb
@@ -0,0 +1,508 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Learners\n",
+ "\n",
+ "In this section, we will introduce several pre-defined learners to learning the datasets by updating their weights to minimize the loss function. when using a learner to deal with machine learning problems, there are several standard steps:\n",
+ "\n",
+ "- **Learner initialization**: Before training the network, it usually should be initialized first. There are several choices when initializing the weights: random initialization, initializing weights are zeros or use Gaussian distribution to init the weights.\n",
+ "\n",
+ "- **Optimizer specification**: Which means specifying the updating rules of learnable parameters of the network. Usually, we can choose Adam optimizer as default.\n",
+ "\n",
+ "- **Applying back-propagation**: In neural networks, we commonly use back-propagation to pass and calculate gradient information of each layer. Back-propagation needs to be integrated with the chosen optimizer in order to update the weights of NN properly in each epoch.\n",
+ "\n",
+ "- **Iterations**: Iterating over the forward and back-propagation process of given epochs. Sometimes the iterating process will have to be stopped by triggering early access in case of overfitting.\n",
+ "\n",
+ "We will introduce several learners with different structures. We will import all necessary packages before that:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Using TensorFlow backend.\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os, sys\n",
+ "sys.path = [os.path.abspath(\"../../\")] + sys.path\n",
+ "from deep_learning4e import *\n",
+ "from notebook4e import *\n",
+ "from learning4e import *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Perceptron Learner\n",
+ "\n",
+ "### Overview\n",
+ "\n",
+ "The Perceptron is a linear classifier. It works the same way as a neural network with no hidden layers (just input and output). First, it trains its weights given a dataset and then it can classify a new item by running it through the network.\n",
+ "\n",
+ "Its input layer consists of the item features, while the output layer consists of nodes (also called neurons). Each node in the output layer has *n* synapses (for every item feature), each with its own weight. Then, the nodes find the dot product of the item features and the synapse weights. These values then pass through an activation function (usually a sigmoid). Finally, we pick the largest of the values and we return its index.\n",
+ "\n",
+ "Note that in classification problems each node represents a class. The final classification is the class/node with the max output value.\n",
+ "\n",
+ "Below you can see a single node/neuron in the outer layer. With *f* we denote the item features, with *w* the synapse weights, then inside the node we have the dot product and the activation function, *g*."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Implementation\n",
+ "\n",
+ "Perceptron learner is actually a neural network learner with only one hidden layer which is pre-defined in the algorithm of `perceptron_learner`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "raw_net = [InputLayer(input_size), DenseLayer(input_size, output_size)]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Where `input_size` and `output_size` are calculated from dataset examples. In the perceptron learner, the gradient descent optimizer is used to update the weights of the network. we return a function `predict` which we will use in the future to classify a new item. The function computes the (algebraic) dot product of the item with the calculated weights for each node in the outer layer. Then it picks the greatest value and classifies the item in the corresponding class."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example\n",
+ "\n",
+ "Let's try the perceptron learner with the `iris` dataset examples, first let's regulate the dataset classes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "iris = DataSet(name=\"iris\")\n",
+ "classes = [\"setosa\", \"versicolor\", \"virginica\"]\n",
+ "iris.classes_to_numbers(classes)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "epoch:50, total_loss:14.089098023560856\n",
+ "epoch:100, total_loss:12.439240091345326\n",
+ "epoch:150, total_loss:11.848151059704785\n",
+ "epoch:200, total_loss:11.283665595671044\n",
+ "epoch:250, total_loss:11.153290841913241\n",
+ "epoch:300, total_loss:11.00747536734494\n",
+ "epoch:350, total_loss:10.871093050365419\n",
+ "epoch:400, total_loss:10.838400319844233\n",
+ "epoch:450, total_loss:10.687417928867456\n",
+ "epoch:500, total_loss:10.650371951865573\n"
+ ]
+ }
+ ],
+ "source": [
+ "pl = perceptron_learner(iris, epochs=500, learning_rate=0.01, verbose=50)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can see from the printed lines that the final total loss is converged to around 10.50. If we check the error ratio of perceptron learner on the dataset after training, we will see it is much higher than randomly guess:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.046666666666666634\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(err_ratio(pl, iris))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If we test the trained learner with some test cases:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "1\n"
+ ]
+ }
+ ],
+ "source": [
+ "tests = [([5.0, 3.1, 0.9, 0.1], 0),\n",
+ " ([5.1, 3.5, 1.0, 0.0], 0),\n",
+ " ([4.9, 3.3, 1.1, 0.1], 0),\n",
+ " ([6.0, 3.0, 4.0, 1.1], 1),\n",
+ " ([6.1, 2.2, 3.5, 1.0], 1),\n",
+ " ([5.9, 2.5, 3.3, 1.1], 1),\n",
+ " ([7.5, 4.1, 6.2, 2.3], 2),\n",
+ " ([7.3, 4.0, 6.1, 2.4], 2),\n",
+ " ([7.0, 3.3, 6.1, 2.5], 2)]\n",
+ "print(grade_learner(pl, tests))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It seems the learner is correct on all the test examples.\n",
+ "\n",
+ "Now let's try perceptron learner on a more complicated dataset: the MNIST dataset, to see what the result will be. First, we import the dataset to make the examples a `Dataset` object:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "length of training dataset: 60000\n",
+ "length of test dataset: 10000\n"
+ ]
+ }
+ ],
+ "source": [
+ "train_img, train_lbl, test_img, test_lbl = load_MNIST(path=\"../../aima-data/MNIST/Digits\")\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "train_examples = [np.append(train_img[i], train_lbl[i]) for i in range(len(train_img))]\n",
+ "test_examples = [np.append(test_img[i], test_lbl[i]) for i in range(len(test_img))]\n",
+ "print(\"length of training dataset:\", len(train_examples))\n",
+ "print(\"length of test dataset:\", len(test_examples))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's train the perceptron learner on the first 1000 examples of the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "epoch:1, total_loss:423.8627535296463\n",
+ "epoch:2, total_loss:341.31697581698995\n",
+ "epoch:3, total_loss:328.98647291325443\n",
+ "epoch:4, total_loss:327.8999700915627\n",
+ "epoch:5, total_loss:310.081065570072\n",
+ "epoch:6, total_loss:268.5474616202945\n",
+ "epoch:7, total_loss:259.0999998773958\n",
+ "epoch:8, total_loss:259.09999987481393\n",
+ "epoch:9, total_loss:259.09999987211944\n",
+ "epoch:10, total_loss:259.0999998693056\n"
+ ]
+ }
+ ],
+ "source": [
+ "mnist = DataSet(examples=train_examples[:1000])\n",
+ "pl = perceptron_learner(mnist, epochs=10, verbose=1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.893\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(err_ratio(pl, mnist))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It looks like we have a near 90% error ratio on training data after the network is trained on it. Then we can investigate the model's performance on the test dataset which it never has seen before:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.92\n"
+ ]
+ }
+ ],
+ "source": [
+ "test_mnist = DataSet(examples=test_examples[:100])\n",
+ "print(err_ratio(pl, test_mnist))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It seems a single layer perceptron learner cannot simulate the structure of the MNIST dataset. To improve accuracy, we may not only increase training epochs but also consider changing to a more complicated network structure."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Neural Network Learner\n",
+ "\n",
+ "Although there are many different types of neural networks, the dense neural network we implemented can be treated as a stacked perceptron learner. Adding more layers to the perceptron network could add to the non-linearity to the network thus model will be more flexible when fitting complex data-target relations. Whereas it also adds to the risk of overfitting as the side effect of flexibility.\n",
+ "\n",
+ "By default we use dense networks with two hidden layers, which has the architecture as the following:\n",
+ "\n",
+ "![](\"images/nn.png\")
\n",
+ "\n",
+ "In our code, we implemented it as:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# initialize the network\n",
+ "raw_net = [InputLayer(input_size)]\n",
+ "# add hidden layers\n",
+ "hidden_input_size = input_size\n",
+ "for h_size in hidden_layer_sizes:\n",
+ " raw_net.append(DenseLayer(hidden_input_size, h_size))\n",
+ " hidden_input_size = h_size\n",
+ "raw_net.append(DenseLayer(hidden_input_size, output_size))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Where hidden_layer_sizes are the sizes of each hidden layer in a list which can be specified by user. Neural network learner uses gradient descent as default optimizer but user can specify any optimizer when calling `neural_net_learner`. The other special attribute that can be changed in `neural_net_learner` is `batch_size` which controls the number of examples used in each round of update. `neural_net_learner` also returns a `predict` function which calculates prediction by multiplying weight to inputs and applying activation functions.\n",
+ "\n",
+ "### Example\n",
+ "\n",
+ "Let's also try `neural_net_learner` on the `iris` dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "epoch:10, total_loss:15.931817841643683\n",
+ "epoch:20, total_loss:8.248422285412149\n",
+ "epoch:30, total_loss:6.102968668275\n",
+ "epoch:40, total_loss:5.463915043272969\n",
+ "epoch:50, total_loss:5.298986288420822\n",
+ "epoch:60, total_loss:4.032928400456889\n",
+ "epoch:70, total_loss:3.2628899927346855\n",
+ "epoch:80, total_loss:6.01336701367312\n",
+ "epoch:90, total_loss:5.412020420311795\n",
+ "epoch:100, total_loss:3.1044027319850773\n"
+ ]
+ }
+ ],
+ "source": [
+ "nn = neural_net_learner(iris, epochs=100, learning_rate=0.15, optimizer=gradient_descent, verbose=10)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Similarly we check the model's accuracy on both training and test dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "error ration on training set: 0.033333333333333326\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(\"error ration on training set:\",err_ratio(nn, iris))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "accuracy on test set: 1\n"
+ ]
+ }
+ ],
+ "source": [
+ "tests = [([5.0, 3.1, 0.9, 0.1], 0),\n",
+ " ([5.1, 3.5, 1.0, 0.0], 0),\n",
+ " ([4.9, 3.3, 1.1, 0.1], 0),\n",
+ " ([6.0, 3.0, 4.0, 1.1], 1),\n",
+ " ([6.1, 2.2, 3.5, 1.0], 1),\n",
+ " ([5.9, 2.5, 3.3, 1.1], 1),\n",
+ " ([7.5, 4.1, 6.2, 2.3], 2),\n",
+ " ([7.3, 4.0, 6.1, 2.4], 2),\n",
+ " ([7.0, 3.3, 6.1, 2.5], 2)]\n",
+ "print(\"accuracy on test set:\",grade_learner(nn, tests))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can see that the error ratio on the training set is smaller than the perceptron learner. As the error ratio is relatively small, let's try the model on the MNIST dataset to see whether there will be a larger difference. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "epoch:10, total_loss:89.0002153455983\n",
+ "epoch:20, total_loss:87.29675663038348\n",
+ "epoch:30, total_loss:86.29591779319225\n",
+ "epoch:40, total_loss:83.78091780128402\n",
+ "epoch:50, total_loss:82.17091581738829\n",
+ "epoch:60, total_loss:83.8434277386084\n",
+ "epoch:70, total_loss:83.55209905561495\n",
+ "epoch:80, total_loss:83.106898191118\n",
+ "epoch:90, total_loss:83.37041170165992\n",
+ "epoch:100, total_loss:82.57013813500876\n"
+ ]
+ }
+ ],
+ "source": [
+ "nn = neural_net_learner(mnist, epochs=100, verbose=10)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.784\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(err_ratio(nn, mnist))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After the model converging, the model's error ratio on the training set is still high. We will introduce the convolutional network in the following chapters to see how it helps improve accuracy on learning this dataset."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/chapter19/Loss Functions and Layers.ipynb b/notebooks/chapter19/Loss Functions and Layers.ipynb
new file mode 100644
index 000000000..25676e899
--- /dev/null
+++ b/notebooks/chapter19/Loss Functions and Layers.ipynb
@@ -0,0 +1,398 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Loss Function\n",
+ "\n",
+ "Loss functions evaluate how well specific algorithm models the given data. Commonly loss functions are used to compare the target data and model's prediction. If predictions deviate too much from actual targets, loss function would output a large value. Usually, loss functions can help other optimization functions to improve the accuracy of the model.\n",
+ "\n",
+ "However, there’s no one-size-fits-all loss function to algorithms in machine learning. For each algorithm and machine learning projects, specifying certain loss functions could assist the user in getting better model performance. Here we will demonstrate two loss functions: `mse_loss` and `cross_entropy_loss`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Min Square Error\n",
+ "\n",
+ "Min square error(MSE) is the most commonly used loss function in machine learning. The intuition of MSE is straight forward: the distance between two points represents the difference between them. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$MSE = -\\sum_i{(y_i-t_i)^2/n}$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Where $y_i$ is the prediction of the ith example and $t_i$ is the target of the ith example. And n is the total number of examples.\n",
+ "\n",
+ "Below is a plot of an MSE function where the true target value is 100, and the predicted values range between -10,000 to 10,000. The MSE loss (Y-axis) reaches its minimum value at prediction (X-axis) = 100."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![](\"images/mse_plot.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Cross-Entropy\n",
+ "\n",
+ "For most deep learning applications, we can get away with just one loss function: cross-entropy loss function. We can think of most deep learning algorithms as learning probability distributions and what we are learning is a distribution of predictions $P(y|x)$ given a series of inputs. \n",
+ "\n",
+ "To associate input examples x with output examples y, the parameters that maximize the likelihood of the training set should be:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\\theta^* = argmax_\\theta \\prod_{i=0}^n p(y^{(i)}/x^{(i)})$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Maxmizing the above formula equals to minimizing the negative log form of it:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\\theta^* = argmin_\\theta -\\sum_{i=0}^n logp(y^{(i)}/x^{(i)})$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It can be proven that the above formula equals to minimizing MSE loss.\n",
+ "\n",
+ "The majority of deep learning algorithms use cross-entropy in some way. Classifiers that use deep learning calculate the cross-entropy between categorical distributions over the output class. For a given class, its contribution to the loss is dependent on its probability in the following trend:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![](\"images/corss_entropy_plot.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Examples\n",
+ "\n",
+ "First let's import necessary packages."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Using TensorFlow backend.\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os, sys\n",
+ "sys.path = [os.path.abspath(\"../../\")] + sys.path\n",
+ "from deep_learning4e import *\n",
+ "from notebook4e import *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Neural Network Layers\n",
+ "\n",
+ "Neural networks may be conveniently described using data structures of computational graphs. A computational graph is a directed graph describing how many variables should be computed, with each variable by computed by applying a specific operation to a set of other variables. \n",
+ "\n",
+ "In our code, we provide class `NNUnit` as the basic structure of a neural network. The structure of `NNUnit` is simple, it only stores the following information:\n",
+ "\n",
+ "- **val**: the value of the current node.\n",
+ "- **parent**: parents of the current node.\n",
+ "- **weights**: weights between parent nodes and current node. It should be in the same size as parents.\n",
+ "\n",
+ "There is another class `Layer` inheriting from `NNUnit`. A `Layer` object holds a list of nodes that represents all the nodes in a layer. It also has a method `forward` to pass a value through the current layer. Here we will demonstrate several pre-defined types of layers in a Neural Network."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Output Layers\n",
+ "\n",
+ "Neural networks need specialized output layers for each type of data we might ask them to produce. For many problems, we need to model discrete variables that have k distinct values instead of just binary variables. For example, models of natural language may predict a single word from among of vocabulary of tens of thousands or even more choices. To represent these distributions, we use a softmax layer:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$P(y=i|x)=softmax(h(x)^TW+b)_i$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "where $W$ is matrix of learned weights of output layer $b$ is a vector of learned biases, and the softmax function is:\n",
+ "\n",
+ "$$softmax(z_i)=exp(z_i)/\\sum_i exp(z_i)$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is simple to create a output layer and feed an example into it:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[0.03205860328008499, 0.08714431874203257, 0.23688281808991013, 0.6439142598879722]\n"
+ ]
+ }
+ ],
+ "source": [
+ "layer = OutputLayer(size=4)\n",
+ "example = [1,2,3,4]\n",
+ "print(layer.forward(example))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The output can be treated like normalized probability when the input of output layer is calculated by probability."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Input Layers\n",
+ "\n",
+ "Input layers can be treated like a mapping layer that maps each element of the input vector to each input layer node. The input layer acts as a storage of input vector information which can be used when doing forward propagation.\n",
+ "\n",
+ "In our realization of input layers, the size of the input vector and input layer should match."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[1, 2, 3]\n"
+ ]
+ }
+ ],
+ "source": [
+ "layer = InputLayer(size=3)\n",
+ "example = [1,2,3]\n",
+ "print(layer.forward(example))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Hidden Layers\n",
+ "\n",
+ "While processing an input vector x of the neural network, it performs several intermediate computations before producing the output y. We can think of these intermediate computations as the state of memory during the execution of a multi-step program. We call the intermediate computations hidden because the data does not specify the values of these variables.\n",
+ "\n",
+ "Most neural network hidden layers are based on a linear transformation followed by the application of an elementwise nonlinear function called the activation function g:\n",
+ "\n",
+ "$$h=g(W+b)$$\n",
+ "\n",
+ "where W is a learned matrix of weights and b is a learned set of bias parameters.\n",
+ "\n",
+ "Here we pre-defined several activation functions in `utils.py`: `sigmoid`, `relu`, `elu`, `tanh` and `leaky_relu`. They are all inherited from the `Activation` class. You can get the value of the function or its derivative at a certain point of x:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Sigmoid at 0: 0.5\n",
+ "Deriavation of sigmoid at 0: 0\n"
+ ]
+ }
+ ],
+ "source": [
+ "s = sigmoid()\n",
+ "print(\"Sigmoid at 0:\", s.f(0))\n",
+ "print(\"Deriavation of sigmoid at 0:\", s.derivative(0))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To create a hidden layer object, there are several attributes need to be specified:\n",
+ "\n",
+ "- **in_size**: the input vector size of each hidden layer node.\n",
+ "- **out_size**: the size of the output vector of the hidden layer. Thus each node will hide the weight of the size of (in_size). The weights will be initialized randomly.\n",
+ "- **activation**: the activation function used for this layer.\n",
+ "\n",
+ "Now let's demonstrate how a dense hidden layer works briefly:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[0.21990266877137224, 0.2038864498984756, 0.5543443697256466]\n"
+ ]
+ }
+ ],
+ "source": [
+ "layer = DenseLayer(in_size=4, out_size=3, activation=sigmoid())\n",
+ "example = [1,2,3,4]\n",
+ "print(layer.forward(example))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This layer mapped input of size 4 to output of size 3. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Convolutional Layers\n",
+ "\n",
+ "The convolutional layer is similar to the hidden layer except they use a different forward strategy. The convolutional layer takes an input of multiple channels and does convolution on each channel with a pre-defined kernel function. Thus the output of the convolutional layer will still be with the same number of channels. If we image each input as an image, then channels represent its color model such as RGB. The output will still have the same color model as the input.\n",
+ "\n",
+ "Now let's try the one-dimensional convolution layer:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[array([3.9894228, 3.9894228, 3.9894228]), array([3.9894228, 3.9894228, 3.9894228]), array([3.9894228, 3.9894228, 3.9894228])]\n"
+ ]
+ }
+ ],
+ "source": [
+ "layer = ConvLayer1D(size=3, kernel_size=3)\n",
+ "example = [[1]*3 for _ in range(3)]\n",
+ "print(layer.forward(example))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Which can be deemed as a one-dimensional image with three channels."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Pooling Layers\n",
+ "\n",
+ "Pooling layers can be treated as a special kind of convolutional layer that uses a special kind of kernel to extract a certain value in the kernel region. Here we use max-pooling to report the maximum value in each group."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[3, 4], [4, 4], [4, 4]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "layer = MaxPoolingLayer1D(size=3, kernel_size=3)\n",
+ "example = [[1,2,3,4], [2,3,4,1],[3,4,1,2]]\n",
+ "print(layer.forward(example))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can see that each time kernel picks up the maximum value in its region."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/chapter19/Optimizer and Backpropagation.ipynb b/notebooks/chapter19/Optimizer and Backpropagation.ipynb
new file mode 100644
index 000000000..5194adc7a
--- /dev/null
+++ b/notebooks/chapter19/Optimizer and Backpropagation.ipynb
@@ -0,0 +1,311 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Optimization Algorithms\n",
+ "\n",
+ "Training a neural network consists of modifying the network’s parameters to minimize the cost function on the training set. In principle, any kind of optimization algorithm could be used. In practice, modern neural networks are almost always trained with some variant of stochastic gradient descent(SGD). Here we will provide two optimization algorithms: SGD and Adam optimizer.\n",
+ "\n",
+ "## Stochastic Gradient Descent\n",
+ "\n",
+ "The goal of an optimization algorithm is to find the value of the parameter to make loss function very low. For some types of models, an optimization algorithm might find the global minimum value of loss function, but for neural network, the most efficient way to converge loss function to a local minimum is to minimize loss function according to each example.\n",
+ "\n",
+ "Gradient descent uses the following update rule to minimize loss function:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\\theta^{(t+1)} = \\theta^{(t)}-\\alpha\\nabla_\\theta L(\\theta^{(t)})$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "where t is the time step of the algorithm and $\\alpha$ is the learning rate. But this rule could be very costly when $L(\\theta)$ is defined as a sum across the entire training set. Using SGD can accelerate the learning process as we can use only a batch of examples to update the parameters. \n",
+ "\n",
+ "We implemented the gradient descent algorithm, which can be viewed with the following code:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Using TensorFlow backend.\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os, sys\n",
+ "sys.path = [os.path.abspath(\"../../\")] + sys.path\n",
+ "from deep_learning4e import *\n",
+ "from notebook4e import *"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " def gradient_descent(dataset, net, loss, epochs=1000, l_rate=0.01, batch_size=1):\n",
+ " """\n",
+ " gradient descent algorithm to update the learnable parameters of a network.\n",
+ " :return: the updated network.\n",
+ " """\n",
+ " # init data\n",
+ " examples = dataset.examples\n",
+ "\n",
+ " for e in range(epochs):\n",
+ " total_loss = 0\n",
+ " random.shuffle(examples)\n",
+ " weights = [[node.weights for node in layer.nodes] for layer in net]\n",
+ "\n",
+ " for batch in get_batch(examples, batch_size):\n",
+ "\n",
+ " inputs, targets = init_examples(batch, dataset.inputs, dataset.target, len(net[-1].nodes))\n",
+ " # compute gradients of weights\n",
+ " gs, batch_loss = BackPropagation(inputs, targets, weights, net, loss)\n",
+ " # update weights with gradient descent\n",
+ " weights = vector_add(weights, scalar_vector_product(-l_rate, gs))\n",
+ " total_loss += batch_loss\n",
+ " # update the weights of network each batch\n",
+ " for i in range(len(net)):\n",
+ " if weights[i]:\n",
+ " for j in range(len(weights[i])):\n",
+ " net[i].nodes[j].weights = weights[i][j]\n",
+ "\n",
+ " if (e+1) % 10 == 0:\n",
+ " print("epoch:{}, total_loss:{}".format(e+1,total_loss))\n",
+ " return net\n",
+ "![](\"images/backprop.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Applying optimizers and back-propagation algorithm together, we can update the weights of a neural network to minimize the loss function with alternatively doing forward and back-propagation process. Here is a figure form [here](https://medium.com/datathings/neural-networks-and-backpropagation-explained-in-a-simple-way-f540a3611f5e) describing how a neural network updates its weights:\n",
+ "\n",
+ "![](\"images/nn_steps.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In our implementation, all the steps are integrated into the optimizer objects. The forward-backward process of passing information through the whole neural network is put into the method `BackPropagation`. You can view the code with:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "psource(BackPropagation)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The demonstration of optimizers and back-propagation algorithm will be made together with neural network learners."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/chapter19/RNN.ipynb b/notebooks/chapter19/RNN.ipynb
new file mode 100644
index 000000000..b6971b36a
--- /dev/null
+++ b/notebooks/chapter19/RNN.ipynb
@@ -0,0 +1,487 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# RNN\n",
+ "\n",
+ "## Overview\n",
+ "\n",
+ "When human is thinking, they are thinking based on the understanding of previous time steps but not from scratch. Traditional neural networks can’t do this, and it seems like a major shortcoming. For example, imagine you want to do sentimental analysis of some texts. It will be unclear if the traditional network cannot recognize the short phrase and sentences.\n",
+ "\n",
+ "Recurrent neural networks address this issue. They are networks with loops in them, allowing information to persist.\n",
+ "\n",
+ "![](\"images/rnn_unit.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A recurrent neural network can be thought of as multiple copies of the same network, each passing a message to a successor. Consider what happens if we unroll the above loop:\n",
+ " \n",
+ "![](\"images/rnn_units.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As demonstrated in the book, recurrent neural networks may be connected in many different ways: sequences in the input, the output, or in the most general case both.\n",
+ "\n",
+ "![](\"images/rnn_connections.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Implementation\n",
+ "\n",
+ "In our case, we implemented rnn with modules offered by the package of `keras`. To use `keras` and our module, you must have both `tensorflow` and `keras` installed as a prerequisite. `keras` offered very well defined high-level neural networks API which allows for easy and fast prototyping. `keras` supports many different types of networks such as convolutional and recurrent neural networks as well as user-defined networks. About how to get started with `keras`, please read the [tutorial](https://keras.io/).\n",
+ "\n",
+ "To view our implementation of a simple rnn, please use the following code:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Using TensorFlow backend.\n"
+ ]
+ }
+ ],
+ "source": [
+ "import warnings\n",
+ "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n",
+ "import os, sys\n",
+ "sys.path = [os.path.abspath(\"../../\")] + sys.path\n",
+ "from deep_learning4e import *\n",
+ "from notebook4e import *"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " def SimpleRNNLearner(train_data, val_data, epochs=2):\n",
+ " """\n",
+ " RNN example for text sentimental analysis.\n",
+ " :param train_data: a tuple of (training data, targets)\n",
+ " Training data: ndarray taking training examples, while each example is coded by embedding\n",
+ " Targets: ndarray taking targets of each example. Each target is mapped to an integer.\n",
+ " :param val_data: a tuple of (validation data, targets)\n",
+ " :param epochs: number of epochs\n",
+ " :return: a keras model\n",
+ " """\n",
+ "\n",
+ " total_inputs = 5000\n",
+ " input_length = 500\n",
+ "\n",
+ " # init data\n",
+ " X_train, y_train = train_data\n",
+ " X_val, y_val = val_data\n",
+ "\n",
+ " # init a the sequential network (embedding layer, rnn layer, dense layer)\n",
+ " model = Sequential()\n",
+ " model.add(Embedding(total_inputs, 32, input_length=input_length))\n",
+ " model.add(SimpleRNN(units=128))\n",
+ " model.add(Dense(1, activation='sigmoid'))\n",
+ " model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
+ "\n",
+ " # train the model\n",
+ " model.fit(X_train, y_train, validation_data=(X_val, y_val), epochs=epochs, batch_size=128, verbose=2)\n",
+ "\n",
+ " return model\n",
+ "![](\"images/autoencoder.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Autoencoders are learned automatically from data examples. It means that it is easy to train specialized instances of the algorithm that will perform well on a specific type of input and that it does not require any new engineering, only the appropriate training data.\n",
+ "\n",
+ "Autoencoders have different architectures for different kinds of data. Here we only provide a simple example of a vanilla encoder, which means they're only one hidden layer in the network:\n",
+ "\n",
+ "![](\"images/vanilla.png\")
\n",
+ "\n",
+ "You can view the source code by:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " def AutoencoderLearner(inputs, encoding_size, epochs=200):\n",
+ " """\n",
+ " Simple example of linear auto encoder learning producing the input itself.\n",
+ " :param inputs: a batch of input data in np.ndarray type\n",
+ " :param encoding_size: int, the size of encoding layer\n",
+ " :param epochs: number of epochs\n",
+ " :return: a keras model\n",
+ " """\n",
+ "\n",
+ " # init data\n",
+ " input_size = len(inputs[0])\n",
+ "\n",
+ " # init model\n",
+ " model = Sequential()\n",
+ " model.add(Dense(encoding_size, input_dim=input_size, activation='relu', kernel_initializer='random_uniform',\n",
+ " bias_initializer='ones'))\n",
+ " model.add(Dense(input_size, activation='relu', kernel_initializer='random_uniform', bias_initializer='ones'))\n",
+ "\n",
+ " # update model with sgd\n",
+ " sgd = optimizers.SGD(lr=0.01)\n",
+ " model.compile(loss='mean_squared_error', optimizer=sgd, metrics=['accuracy'])\n",
+ "\n",
+ " # train the model\n",
+ " model.fit(inputs, inputs, epochs=epochs, batch_size=10, verbose=2)\n",
+ "\n",
+ " return model\n",
+ "![](\"images/mdp.png\")
Now we define actions and a policy similar to **Fig 21.1** in the book."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Action Directions\n",
+ "north = (0, 1)\n",
+ "south = (0,-1)\n",
+ "west = (-1, 0)\n",
+ "east = (1, 0)\n",
+ "\n",
+ "policy = {\n",
+ " (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None,\n",
+ " (0, 1): north, (2, 1): north, (3, 1): None,\n",
+ " (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, \n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This enviroment will be extensively used in the following demonstrations."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Direct Utility Estimation (DUE)\n",
+ " \n",
+ " The first, most naive method of estimating utility comes from the simplest interpretation of the above definition. We construct an agent that follows the policy until it reaches the terminal state. At each step, it logs its current state, reward. Once it reaches the terminal state, it can estimate the utility for each state for *that* iteration, by simply summing the discounted rewards from that state to the terminal one.\n",
+ "\n",
+ " It can now run this 'simulation' $n$ times and calculate the average utility of each state. If a state occurs more than once in a simulation, both its utility values are counted separately.\n",
+ " \n",
+ " Note that this method may be prohibitively slow for very large state-spaces. Besides, **it pays no attention to the transition probability $P(s'\\ |\\ s, a)$.** It misses out on information that it is capable of collecting (say, by recording the number of times an action from one state led to another state). The next method addresses this issue.\n",
+ " \n",
+ "### Examples\n",
+ "\n",
+ "The `PassiveDEUAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveDEUAgent` sums over rewards to find the estimated utility for each state. It thus requires the running of several iterations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%psource PassiveDUEAgent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's try the `PassiveDEUAgent` on the newly defined `sequential_decision_environment`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "DUEagent = PassiveDUEAgent(policy, sequential_decision_environment)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can try passing information through the markove model for 200 times in order to get the converged utility value:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "for i in range(200):\n",
+ " run_single_trial(DUEagent, sequential_decision_environment)\n",
+ " DUEagent.estimate_U()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's print our estimated utility for each position:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(0, 1):0.7956939931414414\n",
+ "(1, 2):0.9162054322837863\n",
+ "(3, 2):1.0\n",
+ "(0, 0):0.734717308253083\n",
+ "(2, 2):0.9595117143816332\n",
+ "(0, 2):0.8481387156375687\n",
+ "(1, 0):0.4355860415209706\n",
+ "(2, 1):-0.550079982553143\n",
+ "(3, 1):-1.0\n"
+ ]
+ }
+ ],
+ "source": [
+ "print('\\n'.join([str(k)+':'+str(v) for k, v in DUEagent.U.items()]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Adaptive Dynamic Programming (ADP)\n",
+ " \n",
+ " This method makes use of knowledge of the past state $s$, the action $a$, and the new perceived state $s'$ to estimate the transition probability $P(s'\\ |\\ s,a)$. It does this by the simple counting of new states resulting from previous states and actions.\n", + " The program runs through the policy a number of times, keeping track of:\n", + " - each occurrence of state $s$ and the policy-recommended action $a$ in $N_{sa}$\n", + " - each occurrence of $s'$ resulting from $a$ on $s$ in $N_{s'|sa}$.\n", + " \n", + " It can thus estimate $P(s'\\ |\\ s,a)$ as $N_{s'|sa}/N_{sa}$, which in the limit of infinite trials, will converge to the true value.
\n", + " Using the transition probabilities thus estimated, it can apply `POLICY-EVALUATION` to estimate the utilities $U(s)$ using properties of convergence of the Bellman functions.\n", + " \n", + "### Examples\n", + "\n", + "The `PassiveADPAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveADPAgent` uses state transition and occurrence counts to estimate $P$, and then $U$. Go through the source below to understand the agent." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We instantiate a `PassiveADPAgent` below with the `GridMDP` shown and train it for 200 steps. The `rl` module has a simple implementation to simulate a single step of the iteration. The function is called `run_single_trial`." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Warning: Transition table is empty.\n" + ] + } + ], + "source": [ + "ADPagent = PassiveADPAgent(policy, sequential_decision_environment)\n", + "for i in range(200):\n", + " run_single_trial(ADPagent, sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The utilities are calculated as :" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 0):0.3014408531958584\n", + "(0, 1):0.40583863351329275\n", + "(1, 2):0.6581480346627065\n", + "(3, 2):1.0\n", + "(3, 0):0.0\n", + "(3, 1):-1.0\n", + "(2, 1):0.5341859348580892\n", + "(2, 0):0.0\n", + "(2, 2):0.810403779650285\n", + "(1, 0):0.23129676787627254\n", + "(0, 2):0.5214746706094832\n" + ] + } + ], + "source": [ + "print('\\n'.join([str(k)+':'+str(v) for k, v in ADPagent.U.items()]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When comparing to the result of `PassiveDUEAgent`, they both have -1.0 for utility at (3,1) and 1.0 at (3,2). Another point to notice is that the spot with the highest utility for both agents is (2,2) beside the terminal states, which is easy to understand when referring to the map." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Temporal-difference learning (TD)\n", + " \n", + " Instead of explicitly building the transition model $P$, the temporal-difference model makes use of the expected closeness between the utilities of two consecutive states $s$ and $s'$.\n", + " For the transition $s$ to $s'$, the update is written as:\n", + "$$U^{\\pi}(s) \\leftarrow U^{\\pi}(s) + \\alpha \\left( R(s) + \\gamma U^{\\pi}(s') - U^{\\pi}(s) \\right)$$\n", + " This model implicitly incorporates the transition probabilities by being weighed for each state by the number of times it is achieved from the current state. Thus, over a number of iterations, it converges similarly to the Bellman equations.\n", + " The advantage of the TD learning model is its relatively simple computation at each step, rather than having to keep track of various counts.\n", + " For $n_s$ states and $n_a$ actions the ADP model would have $n_s \\times n_a$ numbers $N_{sa}$ and $n_s^2 \\times n_a$ numbers $N_{s'|sa}$ to keep track of. The TD model must only keep track of a utility $U(s)$ for each state.\n", + " \n", + "### Examples\n", + "\n", + "`PassiveTDAgent` uses temporal differences to learn utility estimates. We learn the difference between the states and back up the values to previous states. Let us look into the source before we see some usage examples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource PassiveTDAgent" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In creating the `TDAgent`, we use the **same learning rate** $\\alpha$ as given in the footnote of the book: $\\alpha(n)=60/(59+n)$" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "TDagent = PassiveTDAgent(policy, sequential_decision_environment, alpha = lambda n: 60./(59+n))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we run **200 trials** for the agent to estimate Utilities." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(200):\n", + " run_single_trial(TDagent,sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The calculated utilities are:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 1):0.36652562797696076\n", + "(1, 2):0.6584162739552614\n", + "(3, 2):1\n", + "(0, 0):0.27775491505339645\n", + "(3, 0):0.0\n", + "(3, 1):-1\n", + "(2, 1):0.6097040420148784\n", + "(2, 0):0.0\n", + "(2, 2):0.7936759402770092\n", + "(1, 0):0.19085842384266813\n", + "(0, 2):0.5258782999305713\n" + ] + } + ], + "source": [ + "print('\\n'.join([str(k)+':'+str(v) for k, v in TDagent.U.items()]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When comparing to previous agents, the result of `PassiveTDAgent` is closer to `PassiveADPAgent`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter21/images/mdp.png b/notebooks/chapter21/images/mdp.png new file mode 100644 index 000000000..e874130ee Binary files /dev/null and b/notebooks/chapter21/images/mdp.png differ diff --git a/notebooks/chapter22/Grammar.ipynb b/notebooks/chapter22/Grammar.ipynb new file mode 100644 index 000000000..3c1a2a005 --- /dev/null +++ b/notebooks/chapter22/Grammar.ipynb @@ -0,0 +1,526 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Grammar\n", + "\n", + "Languages can be represented by a set of grammar rules over a lexicon of words. Different languages can be represented by different types of grammar, but in Natural Language Processing we are mainly interested in context-free grammars.\n", + "\n", + "## Context-Free Grammar\n", + "\n", + "A lot of natural and programming languages can be represented by a **Context-Free Grammar (CFG)**. A CFG is a grammar that has a single non-terminal symbol on the left-hand side. That means a non-terminal can be replaced by the right-hand side of the rule regardless of context. An example of a CFG:\n", + "\n", + "```\n", + "S -> aSb | ε\n", + "```\n", + "\n", + "That means `S` can be replaced by either `aSb` or `ε` (with `ε` we denote the empty string). The lexicon of the language is comprised of the terminals `a` and `b`, while with `S` we denote the non-terminal symbol. In general, non-terminals are capitalized while terminals are not, and we usually name the starting non-terminal `S`. The language generated by the above grammar is the language anbn for n greater or equal than 1." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Probabilistic Context-Free Grammar\n", + "\n", + "While a simple CFG can be very useful, we might want to know the chance of each rule occurring. Above, we do not know if `S` is more likely to be replaced by `aSb` or `ε`. **Probabilistic Context-Free Grammars (PCFG)** are built to fill exactly that need. Each rule has a probability, given in brackets, and the probabilities of a rule sum up to 1:\n", + "\n", + "```\n", + "S -> aSb [0.7] | ε [0.3]\n", + "```\n", + "\n", + "Now we know it is more likely for `S` to be replaced by `aSb` than by `ε`.\n", + "\n", + "An issue with *PCFGs* is how we will assign the various probabilities to the rules. We could use our knowledge as humans to assign the probabilities, but that is laborious and prone to error task. Instead, we can *learn* the probabilities from data. Data is categorized as labeled (with correctly parsed sentences, usually called a **treebank**) or unlabeled (given only lexical and syntactic category names).\n", + "\n", + "With labeled data, we can simply count the occurrences. For the above grammar, if we have 100 `S` rules and 30 of them are of the form `S -> ε`, we assign a probability of 0.3 to the transformation.\n", + "\n", + "With unlabeled data, we have to learn both the grammar rules and the probability of each rule. We can go with many approaches, one of them the **inside-outside** algorithm. It uses a dynamic programming approach, that first finds the probability of a substring being generated by each rule and then estimates the probability of each rule." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Chomsky Normal Form\n", + "\n", + "Grammar is in Chomsky Normal Form (or **CNF**, not to be confused with *Conjunctive Normal Form*) if its rules are one of the three:\n", + "\n", + "* `X -> Y Z`\n", + "* `A -> a`\n", + "* `S -> ε`\n", + "\n", + "Where *X*, *Y*, *Z*, *A* are non-terminals, *a* is a terminal, *ε* is the empty string and *S* is the start symbol (the start symbol should not be appearing on the right-hand side of rules). Note that there can be multiple rules for each left-hand side non-terminal, as long they follow the above. For example, a rule for *X* might be: `X -> Y Z | A B | a | b`.\n", + "\n", + "Of course, we can also have a *CNF* with probabilities.\n", + "\n", + "This type of grammar may seem restrictive, but it can be proven that any context-free grammar can be converted to CNF." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lexicon\n", + "\n", + "The lexicon of a language is defined as a list of allowable words. These words are grouped into the usual classes: `verbs`, `nouns`, `adjectives`, `adverbs`, `pronouns`, `names`, `articles`, `prepositions` and `conjunctions`. For the first five classes, it is impossible to list all words since words are continuously being added in the classes. Recently \"google\" was added to the list of verbs, and words like that will continue to pop up and get added to the lists. For that reason, these first five categories are called **open classes**. The rest of the categories have much fewer words and much less development. While words like \"thou\" were commonly used in the past but have declined almost completely in usage, most changes take many decades or centuries to manifest, so we can safely assume the categories will remain static for the foreseeable future. Thus, these categories are called **closed classes**.\n", + "\n", + "An example lexicon for a PCFG (note that other classes can also be used according to the language, like `digits`, or `RelPro` for relative pronoun):\n", + "\n", + "```\n", + "Verb -> is [0.3] | say [0.1] | are [0.1] | ...\n", + "Noun -> robot [0.1] | sheep [0.05] | fence [0.05] | ...\n", + "Adjective -> good [0.1] | new [0.1] | sad [0.05] | ...\n", + "Adverb -> here [0.1] | lightly [0.05] | now [0.05] | ...\n", + "Pronoun -> me [0.1] | you [0.1] | he [0.05] | ...\n", + "RelPro -> that [0.4] | who [0.2] | which [0.2] | ...\n", + "Name -> john [0.05] | mary [0.05] | peter [0.01] | ...\n", + "Article -> the [0.35] | a [0.25] | an [0.025] | ...\n", + "Preposition -> to [0.25] | in [0.2] | at [0.1] | ...\n", + "Conjunction -> and [0.5] | or [0.2] | but [0.2] | ...\n", + "Digit -> 1 [0.3] | 2 [0.2] | 0 [0.2] | ...\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Grammer Rules\n", + "\n", + "With grammars we combine words from the lexicon into valid phrases. A grammar is comprised of **grammar rules**. Each rule transforms the left-hand side of the rule into the right-hand side. For example, `A -> B` means that `A` transforms into `B`. Let's build a grammar for the language we started building with the lexicon. We will use a PCFG.\n", + "\n", + "```\n", + "S -> NP VP [0.9] | S Conjunction S [0.1]\n", + "\n", + "NP -> Pronoun [0.3] | Name [0.1] | Noun [0.1] | Article Noun [0.25] |\n", + " Article Adjs Noun [0.05] | Digit [0.05] | NP PP [0.1] |\n", + " NP RelClause [0.05]\n", + "\n", + "VP -> Verb [0.4] | VP NP [0.35] | VP Adjective [0.05] | VP PP [0.1]\n", + " VP Adverb [0.1]\n", + "\n", + "Adjs -> Adjective [0.8] | Adjective Adjs [0.2]\n", + "\n", + "PP -> Preposition NP [1.0]\n", + "\n", + "RelClause -> RelPro VP [1.0]\n", + "```\n", + "\n", + "Some valid phrases the grammar produces: \"`mary is sad`\", \"`you are a robot`\" and \"`she likes mary and a good fence`\".\n", + "\n", + "What if we wanted to check if the phrase \"`mary is sad`\" is actually a valid sentence? We can use a **parse tree** to constructively prove that a string of words is a valid phrase in the given language and even calculate the probability of the generation of the sentence.\n", + "\n", + "\n", + "\n", + "The probability of the whole tree can be calculated by multiplying the probabilities of each individual rule transormation: `0.9 * 0.1 * 0.05 * 0.05 * 0.4 * 0.05 * 0.3 = 0.00000135`.\n", + "\n", + "To conserve space, we can also write the tree in linear form:\n", + "\n", + "[S [NP [Name **mary**]] [VP [VP [Verb **is**]] [Adjective **sad**]]]\n", + "\n", + "Unfortunately, the current grammar **overgenerates**, that is, it creates sentences that are not grammatically correct (according to the English language), like \"`the fence are john which say`\". It also **undergenerates**, which means there are valid sentences it does not generate, like \"`he believes mary is sad`\"." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implementation\n", + "\n", + "In the module, we have implemented both probabilistic and non-probabilistic grammars. Both of these implementations follow the same format. There are functions for the lexicon and the rules which can be combined to create a grammar object.\n", + "\n", + "### Non-Probabilistic\n", + "\n", + "Execute the cell below to view the implementations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from nlp4e import *\n", + "from notebook4e import psource" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(Lexicon, Rules, Grammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's build a lexicon and a grammar for the above language:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Lexicon {'Verb': ['is', 'say', 'are'], 'Noun': ['robot', 'sheep', 'fence'], 'Adjective': ['good', 'new', 'sad'], 'Adverb': ['here', 'lightly', 'now'], 'Pronoun': ['me', 'you', 'he'], 'RelPro': ['that', 'who', 'which'], 'Name': ['john', 'mary', 'peter'], 'Article': ['the', 'a', 'an'], 'Preposition': ['to', 'in', 'at'], 'Conjunction': ['and', 'or', 'but'], 'Digit': ['1', '2', '0']}\n", + "\n", + "Rules: {'S': [['NP', 'VP'], ['S', 'Conjunction', 'S']], 'NP': [['Pronoun'], ['Name'], ['Noun'], ['Article', 'Noun'], ['Article', 'Adjs', 'Noun'], ['Digit'], ['NP', 'PP'], ['NP', 'RelClause']], 'VP': [['Verb'], ['VP', 'NP'], ['VP', 'Adjective'], ['VP', 'PP'], ['VP', 'Adverb']], 'Adjs': [['Adjective'], ['Adjective', 'Adjs']], 'PP': [['Preposition', 'NP']], 'RelClause': [['RelPro', 'VP']]}\n" + ] + } + ], + "source": [ + "lexicon = Lexicon(\n", + " Verb = \"is | say | are\",\n", + " Noun = \"robot | sheep | fence\",\n", + " Adjective = \"good | new | sad\",\n", + " Adverb = \"here | lightly | now\",\n", + " Pronoun = \"me | you | he\",\n", + " RelPro = \"that | who | which\",\n", + " Name = \"john | mary | peter\",\n", + " Article = \"the | a | an\",\n", + " Preposition = \"to | in | at\",\n", + " Conjunction = \"and | or | but\",\n", + " Digit = \"1 | 2 | 0\"\n", + ")\n", + "\n", + "print(\"Lexicon\", lexicon)\n", + "\n", + "rules = Rules(\n", + " S = \"NP VP | S Conjunction S\",\n", + " NP = \"Pronoun | Name | Noun | Article Noun \\\n", + " | Article Adjs Noun | Digit | NP PP | NP RelClause\",\n", + " VP = \"Verb | VP NP | VP Adjective | VP PP | VP Adverb\",\n", + " Adjs = \"Adjective | Adjective Adjs\",\n", + " PP = \"Preposition NP\",\n", + " RelClause = \"RelPro VP\"\n", + ")\n", + "\n", + "print(\"\\nRules:\", rules)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Both the functions return a dictionary with keys to the left-hand side of the rules. For the lexicon, the values are the terminals for each left-hand side non-terminal, while for the rules the values are the right-hand sides as lists.\n", + "\n", + "We can now use the variables `lexicon` and `rules` to build a grammar. After we've done so, we can find the transformations of a non-terminal (the `Noun`, `Verb` and the other basic classes do **not** count as proper non-terminals in the implementation). We can also check if a word is in a particular class." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "How can we rewrite 'VP'? [['Verb'], ['VP', 'NP'], ['VP', 'Adjective'], ['VP', 'PP'], ['VP', 'Adverb']]\n", + "Is 'the' an article? True\n", + "Is 'here' a noun? False\n" + ] + } + ], + "source": [ + "grammar = Grammar(\"A Simple Grammar\", rules, lexicon)\n", + "\n", + "print(\"How can we rewrite 'VP'?\", grammar.rewrites_for('VP'))\n", + "print(\"Is 'the' an article?\", grammar.isa('the', 'Article'))\n", + "print(\"Is 'here' a noun?\", grammar.isa('here', 'Noun'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Chomsky Normal Form\n", + "If the grammar is in **Chomsky Normal Form**, we can call the class function `cnf_rules` to get all the rules in the form of `(X, Y, Z)` for each `X -> Y Z` rule. Since the above grammar is not in *CNF* though, we have to create a new one." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "E_Chomsky = Grammar(\"E_Prob_Chomsky\", # A Grammar in Chomsky Normal Form\n", + " Rules(\n", + " S = \"NP VP\",\n", + " NP = \"Article Noun | Adjective Noun\",\n", + " VP = \"Verb NP | Verb Adjective\",\n", + " ),\n", + " Lexicon(\n", + " Article = \"the | a | an\",\n", + " Noun = \"robot | sheep | fence\",\n", + " Adjective = \"good | new | sad\",\n", + " Verb = \"is | say | are\"\n", + " ))" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[('S', 'NP', 'VP'), ('NP', 'Article', 'Noun'), ('NP', 'Adjective', 'Noun'), ('VP', 'Verb', 'NP'), ('VP', 'Verb', 'Adjective')]\n" + ] + } + ], + "source": [ + "print(E_Chomsky.cnf_rules())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can generate random phrases using our grammar. Most of them will be complete gibberish, falling under the overgenerated phrases of the grammar. That goes to show that in the grammar the valid phrases are much fewer than the overgenerated ones." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'a fence is 2 at 0 at he at john the fence at a good new sheep in the new sad robot which is who is a good robot which are good sad new now lightly sad at 2 and me are'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "grammar.generate_random('S')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Probabilistic\n", + "\n", + "The probabilistic grammars follow the same approach. They take as input a string, are assembled from grammar and a lexicon and can generate random sentences (giving the probability of the sentence). The main difference is that in the lexicon we have tuples (terminal, probability) instead of strings and for the rules, we have a list of tuples (list of non-terminals, probability) instead of the list of lists of non-terminals.\n", + "\n", + "Execute the cells to read the code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(ProbLexicon, ProbRules, ProbGrammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's build a lexicon and rules for the probabilistic grammar:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Lexicon {'Verb': [('is', 0.5), ('say', 0.3), ('are', 0.2)], 'Noun': [('robot', 0.4), ('sheep', 0.4), ('fence', 0.2)], 'Adjective': [('good', 0.5), ('new', 0.2), ('sad', 0.3)], 'Adverb': [('here', 0.6), ('lightly', 0.1), ('now', 0.3)], 'Pronoun': [('me', 0.3), ('you', 0.4), ('he', 0.3)], 'RelPro': [('that', 0.5), ('who', 0.3), ('which', 0.2)], 'Name': [('john', 0.4), ('mary', 0.4), ('peter', 0.2)], 'Article': [('the', 0.5), ('a', 0.25), ('an', 0.25)], 'Preposition': [('to', 0.4), ('in', 0.3), ('at', 0.3)], 'Conjunction': [('and', 0.5), ('or', 0.2), ('but', 0.3)], 'Digit': [('0', 0.35), ('1', 0.35), ('2', 0.3)]}\n", + "\n", + "Rules: {'S': [(['NP', 'VP'], 0.6), (['S', 'Conjunction', 'S'], 0.4)], 'NP': [(['Pronoun'], 0.2), (['Name'], 0.05), (['Noun'], 0.2), (['Article', 'Noun'], 0.15), (['Article', 'Adjs', 'Noun'], 0.1), (['Digit'], 0.05), (['NP', 'PP'], 0.15), (['NP', 'RelClause'], 0.1)], 'VP': [(['Verb'], 0.3), (['VP', 'NP'], 0.2), (['VP', 'Adjective'], 0.25), (['VP', 'PP'], 0.15), (['VP', 'Adverb'], 0.1)], 'Adjs': [(['Adjective'], 0.5), (['Adjective', 'Adjs'], 0.5)], 'PP': [(['Preposition', 'NP'], 1.0)], 'RelClause': [(['RelPro', 'VP'], 1.0)]}\n" + ] + } + ], + "source": [ + "lexicon = ProbLexicon(\n", + " Verb = \"is [0.5] | say [0.3] | are [0.2]\",\n", + " Noun = \"robot [0.4] | sheep [0.4] | fence [0.2]\",\n", + " Adjective = \"good [0.5] | new [0.2] | sad [0.3]\",\n", + " Adverb = \"here [0.6] | lightly [0.1] | now [0.3]\",\n", + " Pronoun = \"me [0.3] | you [0.4] | he [0.3]\",\n", + " RelPro = \"that [0.5] | who [0.3] | which [0.2]\",\n", + " Name = \"john [0.4] | mary [0.4] | peter [0.2]\",\n", + " Article = \"the [0.5] | a [0.25] | an [0.25]\",\n", + " Preposition = \"to [0.4] | in [0.3] | at [0.3]\",\n", + " Conjunction = \"and [0.5] | or [0.2] | but [0.3]\",\n", + " Digit = \"0 [0.35] | 1 [0.35] | 2 [0.3]\"\n", + ")\n", + "\n", + "print(\"Lexicon\", lexicon)\n", + "\n", + "rules = ProbRules(\n", + " S = \"NP VP [0.6] | S Conjunction S [0.4]\",\n", + " NP = \"Pronoun [0.2] | Name [0.05] | Noun [0.2] | Article Noun [0.15] \\\n", + " | Article Adjs Noun [0.1] | Digit [0.05] | NP PP [0.15] | NP RelClause [0.1]\",\n", + " VP = \"Verb [0.3] | VP NP [0.2] | VP Adjective [0.25] | VP PP [0.15] | VP Adverb [0.1]\",\n", + " Adjs = \"Adjective [0.5] | Adjective Adjs [0.5]\",\n", + " PP = \"Preposition NP [1]\",\n", + " RelClause = \"RelPro VP [1]\"\n", + ")\n", + "\n", + "print(\"\\nRules:\", rules)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's use the above to assemble our probabilistic grammar and run some simple queries:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "How can we rewrite 'VP'? [(['Verb'], 0.3), (['VP', 'NP'], 0.2), (['VP', 'Adjective'], 0.25), (['VP', 'PP'], 0.15), (['VP', 'Adverb'], 0.1)]\n", + "Is 'the' an article? True\n", + "Is 'here' a noun? False\n" + ] + } + ], + "source": [ + "grammar = ProbGrammar(\"A Simple Probabilistic Grammar\", rules, lexicon)\n", + "\n", + "print(\"How can we rewrite 'VP'?\", grammar.rewrites_for('VP'))\n", + "print(\"Is 'the' an article?\", grammar.isa('the', 'Article'))\n", + "print(\"Is 'here' a noun?\", grammar.isa('here', 'Noun'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we have a grammar in *CNF*, we can get a list of all the rules. Let's create a grammar in the form and print the *CNF* rules:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "E_Prob_Chomsky = ProbGrammar(\"E_Prob_Chomsky\", # A Probabilistic Grammar in CNF\n", + " ProbRules(\n", + " S = \"NP VP [1]\",\n", + " NP = \"Article Noun [0.6] | Adjective Noun [0.4]\",\n", + " VP = \"Verb NP [0.5] | Verb Adjective [0.5]\",\n", + " ),\n", + " ProbLexicon(\n", + " Article = \"the [0.5] | a [0.25] | an [0.25]\",\n", + " Noun = \"robot [0.4] | sheep [0.4] | fence [0.2]\",\n", + " Adjective = \"good [0.5] | new [0.2] | sad [0.3]\",\n", + " Verb = \"is [0.5] | say [0.3] | are [0.2]\"\n", + " ))" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[('S', 'NP', 'VP', 1.0), ('NP', 'Article', 'Noun', 0.6), ('NP', 'Adjective', 'Noun', 0.4), ('VP', 'Verb', 'NP', 0.5), ('VP', 'Verb', 'Adjective', 0.5)]\n" + ] + } + ], + "source": [ + "print(E_Prob_Chomsky.cnf_rules())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Lastly, we can generate random sentences from this grammar. The function `prob_generation` returns a tuple (sentence, probability)." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a good good new good sheep that say a good good robot the sad robot to 1 to me you to sheep are\n", + "5.511240000000004e-26\n" + ] + } + ], + "source": [ + "sentence, prob = grammar.generate_random('S')\n", + "print(sentence)\n", + "print(prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As with the non-probabilistic grammars, this one mostly overgenerates. You can also see that the probability is very, very low, which means there are a ton of generate able sentences (in this case infinite, since we have recursion; notice how `VP` can produce another `VP`, for example)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter22/Introduction.ipynb b/notebooks/chapter22/Introduction.ipynb new file mode 100644 index 000000000..0905b91a9 --- /dev/null +++ b/notebooks/chapter22/Introduction.ipynb @@ -0,0 +1,92 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# NATURAL LANGUAGE PROCESSING\n", + "\n", + "The notebooks in this folder cover chapters 23 of the book *Artificial Intelligence: A Modern Approach*, 4th Edition. The implementations of the algorithms can be found in [nlp.py](https://github.com/aimacode/aima-python/blob/master/nlp4e.py).\n", + "\n", + "Run the below cell to import the code from the module and get started!" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from nlp4e import *\n", + "from notebook4e import psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## OVERVIEW\n", + "\n", + "**Natural Language Processing (NLP)** is a field of AI concerned with understanding, analyzing and using natural languages. This field is considered a difficult yet intriguing field of study since it is connected to how humans and their languages work.\n", + "\n", + "Applications of the field include translation, speech recognition, topic segmentation, information extraction and retrieval, and a lot more.\n", + "\n", + "Below we take a look at some algorithms in the field. Before we get right into it though, we will take a look at a very useful form of language, **context-free** languages. Even though they are a bit restrictive, they have been used a lot in research in natural language processing.\n", + "\n", + "Below is a summary of the demonstration files in this chapter." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "\n", + "- Introduction: Introduction to the field of nlp and the table of contents.\n", + "- Grammars: Introduction to grammar rules and lexicon of words of a language.\n", + " - Context-free Grammar\n", + " - Probabilistic Context-Free Grammar\n", + " - Chomsky Normal Form\n", + " - Lexicon\n", + " - Grammar Rules\n", + " - Implementation of Different Grammars\n", + "- Parsing: The algorithms parsing sentences according to a certain kind of grammar.\n", + " - Chart Parsing\n", + " - CYK Parsing\n", + " - A-star Parsing\n", + " - Beam Search Parsing\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter22/Parsing.ipynb b/notebooks/chapter22/Parsing.ipynb new file mode 100644 index 000000000..50a4264fb --- /dev/null +++ b/notebooks/chapter22/Parsing.ipynb @@ -0,0 +1,522 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Parsing\n", + "\n", + "## Overview\n", + "\n", + "Syntactic analysis (or **parsing**) of a sentence is the process of uncovering the phrase structure of the sentence according to the rules of grammar. \n", + "\n", + "There are two main approaches to parsing. *Top-down*, start with the starting symbol and build a parse tree with the given words as its leaves, and *bottom-up*, where we start from the given words and build a tree that has the starting symbol as its root. Both approaches involve \"guessing\" ahead, so it may take longer to parse a sentence (the wrong guess mean a lot of backtracking). Thankfully, a lot of effort is spent in analyzing already analyzed substrings, so we can follow a dynamic programming approach to store and reuse these parses instead of recomputing them. \n", + "\n", + "In dynamic programming, we use a data structure known as a chart, thus the algorithms parsing a chart is called **chart parsing**. We will cover several different chart parsing algorithms." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Chart Parsing\n", + "\n", + "### Overview\n", + "\n", + "The chart parsing algorithm is a general form of the following algorithms. Given a non-probabilistic grammar and a sentence, this algorithm builds a parse tree in a top-down manner, with the words of the sentence as the leaves. It works with a dynamic programming approach, building a chart to store parses for substrings so that it doesn't have to analyze them again (just like the CYK algorithm). Each non-terminal, starting from S, gets replaced by its right-hand side rules in the chart until we end up with the correct parses.\n", + "\n", + "### Implementation\n", + "\n", + "A parse is in the form `[start, end, non-terminal, sub-tree, expected-transformation]`, where `sub-tree` is a tree with the corresponding `non-terminal` as its root and `expected-transformation` is a right-hand side rule of the `non-terminal`.\n", + "\n", + "The chart parsing is implemented in a class, `Chart`. It is initialized with grammar and can return the list of all the parses of a sentence with the `parses` function.\n", + "\n", + "The chart is a list of lists. The lists correspond to the lengths of substrings (including the empty string), from start to finish. When we say 'a point in the chart', we refer to a list of a certain length.\n", + "\n", + "A quick rundown of the class functions:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* `parses`: Returns a list of parses for a given sentence. If the sentence can't be parsed, it will return an empty list. Initializes the process by calling `parse` from the starting symbol.\n", + "\n", + "\n", + "* `parse`: Parses the list of words and builds the chart.\n", + "\n", + "\n", + "* `add_edge`: Adds another edge to the chart at a given point. Also, examines whether the edge extends or predicts another edge. If the edge itself is not expecting a transformation, it will extend other edges and it will predict edges otherwise.\n", + "\n", + "\n", + "* `scanner`: Given a word and a point in the chart, it extends edges that were expecting a transformation that can result in the given word. For example, if the word 'the' is an 'Article' and we are examining two edges at a chart's point, with one expecting an 'Article' and the other a 'Verb', the first one will be extended while the second one will not.\n", + "\n", + "\n", + "* `predictor`: If an edge can't extend other edges (because it is expecting a transformation itself), we will add to the chart rules/transformations that can help extend the edge. The new edges come from the right-hand side of the expected transformation's rules. For example, if an edge is expecting the transformation 'Adjective Noun', we will add to the chart an edge for each right-hand side rule of the non-terminal 'Adjective'.\n", + "\n", + "\n", + "* `extender`: Extends edges given an edge (called `E`). If `E`'s non-terminal is the same as the expected transformation of another edge (let's call it `A`), add to the chart a new edge with the non-terminal of `A` and the transformations of `A` minus the non-terminal that matched with `E`'s non-terminal. For example, if an edge `E` has 'Article' as its non-terminal and is expecting no transformation, we need to see what edges it can extend. Let's examine the edge `N`. This expects a transformation of 'Noun Verb'. 'Noun' does not match with 'Article', so we move on. Another edge, `A`, expects a transformation of 'Article Noun' and has a non-terminal of 'NP'. We have a match! A new edge will be added with 'NP' as its non-terminal (the non-terminal of `A`) and 'Noun' as the expected transformation (the rest of the expected transformation of `A`).\n", + "\n", + "You can view the source code by running the cell below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(Chart)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "We will use the grammar `E0` to parse the sentence \"the stench is in 2 2\".\n", + "\n", + "First, we need to build a `Chart` object:" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [], + "source": [ + "chart = Chart(E0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And then we simply call the `parses` function:" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[0, 6, 'S', [[0, 2, 'NP', [('Article', 'the'), ('Noun', 'stench')], []], [2, 6, 'VP', [[2, 3, 'VP', [('Verb', 'is')], []], [3, 6, 'PP', [('Preposition', 'in'), [4, 6, 'NP', [('Digit', '2'), ('Digit', '2')], []]], []]], []]], []]]\n" + ] + } + ], + "source": [ + "print(chart.parses('the stench is in 2 2'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see which edges get added by setting the optional initialization argument `trace` to true." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "chart_trace = Chart(nlp.E0, trace=True)\n", + "chart_trace.parses('the stench is in 2 2')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's try and parse a sentence that is not recognized by the grammar:" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[]\n" + ] + } + ], + "source": [ + "print(chart.parses('the stench 2 2'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An empty list was returned." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CYK Parse\n", + "\n", + "The *CYK Parsing Algorithm* (named after its inventors, Cocke, Younger, and Kasami) utilizes dynamic programming to parse sentences of grammar in *Chomsky Normal Form*.\n", + "\n", + "The CYK algorithm returns an *M x N x N* array (named *P*), where *N* is the number of words in the sentence and *M* the number of non-terminal symbols in the grammar. Each element in this array shows the probability of a substring being transformed from a particular non-terminal. To find the most probable parse of the sentence, a search in the resulting array is required. Search heuristic algorithms work well in this space, and we can derive the heuristics from the properties of the grammar.\n", + "\n", + "The algorithm in short works like this: There is an external loop that determines the length of the substring. Then the algorithm loops through the words in the sentence. For each word, it again loops through all the words to its right up to the first-loop length. The substring will work on in this iteration is the words from the second-loop word with the first-loop length. Finally, it loops through all the rules in the grammar and updates the substring's probability for each right-hand side non-terminal." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "The implementation takes as input a list of words and a probabilistic grammar (from the `ProbGrammar` class detailed above) in CNF and returns the table/dictionary *P*. An item's key in *P* is a tuple in the form `(Non-terminal, the start of a substring, length of substring)`, and the value is a `Tree` object. The `Tree` data structure has two attributes: `root` and `leaves`. `root` stores the value of current tree node and `leaves` is a list of children nodes which may be terminal states(words in the sentence) or a sub tree.\n", + "\n", + "For example, for the sentence \"the monkey is dancing\" and the substring \"the monkey\" an item can be `('NP', 0, 2):
![](\"images/gradients.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Implementation\n",
+ "\n",
+ "We implemented an edge detector using a gradient method as `gradient_edge_detector` in `perceptron.py`. There are two filters defined as $[[1, -1]], [[1], [-1]]$ to extract edges in x and y directions respectively. The filters are applied to an image using `convolve2d` method in `scipy.single` package. The image passed into the function needs to be in the form of `numpy.ndarray` or an iterable object that can be transformed into a `ndarray`.\n",
+ "\n",
+ "To view the detailed implementation, please execute the following block"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "psource(gradient_edge_detector)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example\n",
+ "\n",
+ "Now let's try the detector for real case pictures. First, we will show the original picture before edge detection:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![](\"images/stapler.png\")
\n",
+ "\n",
+ "We will use `matplotlib` to read the image as a numpy ndarray:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "image height: 590\n",
+ "image width: 787\n"
+ ]
+ }
+ ],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "import numpy as np\n",
+ "import matplotlib.image as mpimg\n",
+ "\n",
+ "im =mpimg.imread('images/stapler.png')\n",
+ "print(\"image height:\", len(im))\n",
+ "print(\"image width:\", len(im[0]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The code shows we get an image with a size of $787*590$. `gaussian_derivative_edge_detector` can extract images in both x and y direction and then put them together in a ndarray:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "image height: 590\n",
+ "image width: 787\n"
+ ]
+ }
+ ],
+ "source": [
+ "edges = gradient_edge_detector(im)\n",
+ "print(\"image height:\", len(edges))\n",
+ "print(\"image width:\", len(edges[0]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The edges are in the same shape of the original image. Now we will try print out the image, we implemented a `show_edges` function to do this:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9WW9jSXa1vTlooERNpays6a3uKtgGjL7yb/KP9bUBXzQaNrpt1FyZlalMSRQpkuJ3IT/B5ywdMY3yxQcDGYAgiTwnTsSOPay9YjiDzWZTH8vH8rF8LP8Xy/D/7wZ8LB/Lx/Kx/N7y0YF9LB/Lx/J/tnx0YB/Lx/Kx/J8tHx3Yx/KxfCz/Z8tHB/axfCwfy//ZMt715T//8z9vqqr+/Oc/13j8eOlyuayDg4OazWZVVXVwcFCr1aoODw9rsVjUZrOpyWRSq9Wq5vN5nZyc1PX1dQ2Hw9rf36/NZlODwaDG43Hd3d3V3t5eLZfLGo/HtdlsarVa1d7eXq3X6xqNRrXZbGq9XtfBwUEtFov293q9LmZQ9/b2arVa1Wg0aveNRqO6v7+v1WpVBwcHVVW1Xq9rtVq1vozH41qtVlVVNRqNajAYtOfxma/n8yz0abPZtDrH43Fri+8dDof18PDw5F6e5+dyHW2kHdxXVfXu3bt68eJFHRwc1HK5rIeHh04fqJ86+fvVq1f1/fff15/+9Kc6PDys+Xxel5eXdXh4WFdXV7Ver+vh4aEGg0EtFos6PDxs41NVrc3ZHvrKeO7t7bX2MD6MScpzOBzWarWq/f399hz34eHhoV2DPKiTQpvdPu5nXFwfdQ6Hw3b/w8NDkzX9sm4wTnw/HA7bd9Q/Go1qOBzWcrls+p5yQF8Gg0Etl8uOntM/X7PZbGqxWLS2jUajWiwWTZb8/9z3FF/XtwoBWaVNWHYpd/SL6xjPh4eHJ+OFPjEeyMi2Np1O61/+5V9qvV7XyclJvXr1avCkofUBBHZ1dVWj0ahubm6acQwGg5rNZnV4eNiMdG9vr66vr2t/f78ODg7q6uqqNebu7q6Ojo46ij+fz+vm5qaqqubzeTO+qqr9/f32G+Gu1+u6v7+vo6Oj9szDw8NW32w2a0rFYM9ms44jwUkeHBw0x2gj4RqUx87L19nJoPQInUGgvqqqxWLRcUb8RvF9D3XgPCjj8fiJQdmZLZfLzjPsgK2kKN7d3V3t7+/Xcrlsjp77uBalWi6XzWngAHn+ZrPp1M3vo6OjzljbuDDIxWLRnAeyvL+/b87AZW9vr+nEcDhs8sCIHh4eWh37+/udsbHBHR4eNplbtrShqtp9Lu6b5Uvb3N7hcFjr9bo5RZwXxp16h0yoG51DRxnndAJuv+szEEDuq9WqjZWdkXUK28L5Wsb0gb8p1I1sBoNBs8uqbTA4ODh40g90jyCHbTK+yC4DfpadDuzs7KyWy2UdHx83g6PhODAGb39/vxaLRSdaj8fj5pCm02nN5/OqquZE+E3EQ3lxJDQeoRC9uYa22MD4DoGgVDiK9XrdQSfcT3tz0Hguz+KHAUJZ+gYbhdrb2+sYBu1hYKgTY7QT5fo+w6LwXDtUHPd6vW5KR38JKIyrgwHPwmlYgfjMkZj//bkdoRWX4IKBoPQ434ODg3afn4vSz+fzJnOPC07j4eGh5vN53d/fNyO106GezWZTDw8PrT8gG7fTz0A30NmDg4MOyrMc7LBAOegmtuDgwxgbablvjKGdlh2o9db/M6Z+Fm2mLuRB8OBv6liv100uXG+5GG2tVqteZ8PY4KzcZ+xxtVo1BIotL5fL5nAT7Xfqf/abqvrpp5+aR6RhCPnq6qopOg6DTk+n03bPw8NDzWazhrTm83lTYCDzfD6vwWBQ9/f3tVwuWwqJwdFxkBaKhxBtgFZEp5UJl+2c/MNnOEgrIQPqaEq/5/N5UwieTfF3DLLbAuqqqoZCHDkxSKdC9JNxsUNECa0ooCfS6qotckPBHh4emhIZ1ldVc9ROiWkPyoaztNN0H7mXvxlnI1gK6CzHLI3Af+NgCK6MoQMVzzHCd7DKtthA6SOyzzGk2Hnz98HBQd3f3zcnkobpIGM581zoE/eVwAVdYx3O+9y2PnrB16H/2KnTyaQ7nK6iO+iSi3WCoG79QiamhfARuxbb73RgL168qOFw2NAQ6AYFwRDv7u466OT+/r4zEOPxuI6OjmqxWLTOoQQoMAZmSEwHN5tNSw3gu6gro7+f54i4WCw6XJCFNhgMOtE/Oa39/f0nENrPNiRP9AYS7DNS14FxHR4edmA4dfFjmTLQmZbiZHkWiCFTDdJD+gAyxGkvl8va399v/aeQ9riNyPY5h8MzqqqTqjOW6Aty537/b7nRV4yP+pfLZcdxUQ8ozwZGn6kTHUPnGE/SRI+FUQr3gDbM0/ahBzsOj3Efkkd3yCY8hvDQpO3YX/KUo9GoyQQ9Tw6L59upo9dwks4+PA4eO2im59JybJH7+hAov5HpruxjpwOD16p6VI7JZFKTyaRxYKCn09PTxmnw2cPDQx0dHbW8GPIfARqiuuEMPAYE/zWbzWpvb6/m83ltNpsnRonAgaT85vpMZewIGEzqw2iA1UQnlNXPMq9h5GWU4+hqBenjzFarVUNJTj/4LhXP8BzOiejF9Tg0p74ohw2WgmxJ622IOalhp+/IbqV2/bQTmgE9QuZOpfsmDNAV84QYN06IvlvxHbySk0Rf7u/vGwXisTEazfFwv13IStz3RFXIECRssp3nGPVbj3GSyBSdSV0D7cBLEshN4uNQUreqqoMaPRaMAc80eMmJJE+U8FxPNlCHbZbP+gKYy04Htr+/X/P5vHUUIZGnMtg4AfgHlGO5XDbey8VRhIG1ITunJwI5Qjk6UN9yuWxRiAKySsjsdjgdALkliuLaPkEi6D7Ogf9JpYn8tJGoaDn4GY56OJG9vb32mTkcjxmyTJjPOJnTYhIGuTrioVCMv50QwSadPoHJfTI/41TBqTroj346grs4dcVJ+nnJcyILj0um40bfOX7omXUiddD0AIgF5+I60SE7BJwlz6ra6i1ytw1kv+y0zC86KGCDfWm3ucLk0tAx2wLIl6yoT1agW/pOu0ndLZNEtE6Rq+pJKpplpwNDwVerVUsBE8FUVd3d3TUjHo/HHU7q8PCwzXotFouGpCzo29vbDkTG4CGb3UELzp/t7e01rsikO4KzIVES8eHQiIi0hz45J6cNOB0jLReUNQlOOwz+N1fH99SB0WP4JqXbYP63YrG0xUaGfFGibA+IhkCFQSOTTDeMWJ0KGVFn2pW8klPyvun2nDTpky16gFxsAPSB7CCNyrqcHJbrsbwdVLx8gRlOAr5TWWSVwZM60xHwTF+TjjoDcwY/zyzznQONZWEdSAftZ9AGgjDfeWYX3SUQPjw8NDTqe7kfAEH2wJjs7++3e3eVnevAmCUE1iE0ZgZxXFbkPmRU9ejk/LkHylzC/f19e87d3V1HKBjE/f19axd1elYyI1pCYyMxRzQbprmRRBSkGwieOjuC/e+60snwzDRKf9ZH7mYKRzvsJJAXfbATMjmaXIVTtZwA4D4UEmPCSHONkdfcpbHYMO3gnIbwzAxELn3yoTj1oD0YXl5r4/U4ZxrFd0Yt2R5TEuhnOiyuTdlmfe5D8oiUPgeUOu3vPR7OTFK/MxD36QfBIFNEtx8kxn3wjfzdh3gdNPqQc1/ZicBubm5aWrVarVqqQUe9PseDzKwU3hyI7uhlfgBvy+B6kB0hPaXr73iO4a6d5GKxaG00DM/U0jxMttNRw8XGzuBl3dRDX+GV7Egd6WmzFTeRCNfxPNIEjMkDj0Owg6I9RmE4NpPcnvH1dU7VEq0wrjzXjpT7E2FWVUPmfZySi5EccjG36O+sH5Zj9od2mpg2Kqd99CNTLcuaa5BDOizzP/zP95YL7UfO2JAJcAdIAwGnYxTrIfXYTtwnfqfTt77YaWWQz+tpm/Xejp/nY+dkbfnMLDsdGLNUDw8PbR0XC1pBPHBi9rA4sKrqpB809O7urnUCGJmroT2TZk7EcNhpEtfTJgYDY2FhK8J0OkAx6ey0YReaovSlj6nMCct5VhajWEd9f45zYNCd7tM/Pwe5AdPdVytiprnphHBwJm4tA+qlXZm624BMavOb2WX/ZGrJve6HESd9tZEYbVjmmXo53TMyNAdo/eGe5LbW6y3JTp8sD5AOBDvf2zm5mHIxr+SshvTVAdr9M/plvNxv64z7w8SIgzFysp6iG3Z82C1Lpaqqw+M6XaWe0Wi7VrHPrlx2OrAvvvii5vN5zWaz5swYBKaWR6NRzWazlveS0xOtptNpGyjSv5OTk85gOsofHh7W/f19TafTzroxZkPMsSCIqurMyBiB8RvI7FnJvmhnRXJxvVaujFpuk9OAVMhEDI5gXrZgDsTpWN/AegbuuZkbAoUdoNNl/uY3DpGx3dvb6wQLI66qerLlKPk6pyiptMjTxmSD8no6y4vn2+G6TXZijJHlS0GnPV4OhOkQ7Cxoo/XRjtNZhX8zM2enkw7InJ3XIVr3SAvpN4jc8nNAJSuy04Q/pb8OnkZs5tecFXmBthEtVIMDVlIRVVuwMx6P27q25/S4PXPXl7/88kudnp42Ah7FZ5DxwhDo3uYBEoMQpzMoIaQt6Gg8Htft7W2LIldXV01J+lI6BGEjgMy3t/fAQ8bboBhoBpW29nE0fm5GbXNpoBorjQfaEcfX8L/X99Bvzyr5Htq12WxXkzvAmLyl346UOCaUxuukcFh2EChY31o1xtqLFLnXq9f7Upw+TodCkMtU0OOLLhplWO9s7DYgxtipNcZj3rWqmu7QB/javjbbyTyH6u0MjLxMITDmHlvaAzlOe5yi22Egd8sfQGA0avohsw50ygjQ/Ch9tpPEB5gygS8zWjVgQE7o0K41YFUfcGCffvppcyTHx8e1t7fXUhXQEmV/f7+lkTguowEadnx83EnLbESsVQEqIzwvejR6ykiPIOz0mHxgQOfzeVssas4hyU4/w7Oqvt5ciqfNiUL0pS+dsAEaTdkZJEeR7U15eIMsdfv3aLSdLfMOC773DggKK7LN43gngbkuO1iPY6LU5CCdbtipePwxbgcK/+9ITl95RhqpZcn9zGAzltZZ2uTZadprns3pc5a+oGfDddrqtiILxtiOHwfh+rJYh2iHn4teJidHe422TNX4fhAXumkb9FIf/IODt69zYLRz3lV2OrD379+3LRer1aoN4Hj8OONl4fH7/v6+PRjB403NUSTPgyOzInggud45vwfAvAglOQKuz0F97nr+N6eX16dR2tD7hJ/T388VR6jxeNyJmCiDURkKhgycIvYpvJXQ8nB09304LaJikuDuc59sfR31Wx+qnpK1XqDq51FXH2rDYbrv6VBs7H4G36Uz8Ho3X8fJEbTPqNNBzgg05UX7s41OXX0tz+azvu+zH0Zjz8nCsvSzJ5PJk7Hum6E3LeC/7djskI12TUkQANlhsosOqfqAA5tOp61xNNSzUyZXSf3oGJ9VdZc49K3ExQNTh1GHCdQ0Dl8LN0OEzZTOKW9VtX2cSST3EctJVrqYK/B9nlEz55GkMYNpDs6IxcaY0diBwxuw7Rjov1EHMD63IOFUGBsbZcrEgclt5bdlD7dhxadYRjhSG5mN1Gm4n+dUMfnC5KmyJOLMsfL90CWJ+h280nn3PduBgfqwERyOEVdV9xQK2ma9zGDpdBTkxGeWqduLzLIfyDSvR/eHw2FnLZuvgQOj/r5ZbVAXz/EJHw7kfeWDyygmk0lTdq8FciOGw2GbSXFUd2Qaj8edtAXOoS/PRfDeHoGyOiXAwwPvnZcb4hoNUh8zqQjHgjVH42iSCJK/06n6M1+LItkJmZ8yemLw7Iid5vp/OCsfSeTgwv1+rsl8r8x3muyx8MQMwckIog95sfjX7baToh9G5HbYicRzdTvf8zncXqZDzxWjzhxn82R5T3JARmUYo4OVuSc/t6radjm+R+/NHVNAJEaC1rPxePxkpTv1Gpl6bPtk6oAC0vSOG+qlzw8PD53TLJJysKO1/GibU3/awGL2D5WdDozFqqvV48wi6SG5qhctzmazur29bXslE5Ki8DiryWTSlAtin0MRKScnJ03QPuTNHFdCYxuSicHVatVmNI3KzDVxPcrTR0o6VaPv5lX4zL+NqHzAH/VaSY1qnouOyIR783QExqmqu+aIfvrQPiNSHCEBwaklRkVgcuo1m806zswpTk5eoMxOgb3mzUgMFJspBEaW9VZ1T+egPUZTRjQE3eQSkYsN2ZkAupRjSduMMD07bkRopGq5Ge1Z93D2dhyWDSiOXRTm7tDlPookERXtRQ+ZwMMmjLixIe8m4Zgi2gVYMZeaMvJY0X/PxO8i8ncuc10sFjWZTDpC5uE3Nzct2rBQdL1ed7gTH7VDw82ngOb6TktIJU20kp1nQPtycistdRgd+hmkOim0Pu6gahthMp3I62hTKqHr7UuRq56iODuz8Xh7EJw5BjvPbBOIOuWNsnkPLEqeG41tCE4faB9t8GRMpgJGsfSd1LQP/Tw3Fs9t43JJNIdxowvmF2mrEYv1ij6h29l39y1lZsdlotpB11yZ0//83AHIDi+5NlMticxTJhT3I+XtdjoY+hmWre2MdqczpE2WG4vbd1EAOxEYs4IgJCLRYrGo6XTapkgdUVerVYtsw+GwkYA5Y0TquL+/3440Ho/HHYfpweYZmaZWdVcs02lHTaM0p4aGuZm3JyLgJ5FfDkIihvzf0Sf5NKeuRg5GXpnGcjRNwm2e48BhBJlbhqw8PpPNzzLqSkNwMSJFeftSqExzUlnT8bk9li2y8hh5vDNQ2WAsa7c/xyXHCv6KdrndpNweK/qGHSWnm07K97mtFC9v4XtTLfSTttkmLEOjSn/v1N7BnO8TwduGnMYnerZDsxyYELEjRM6/G4HNZrPa39/vHBHs1Cv5CmbKvCePTgwG243dIADun81mzfmZ33JkrNrO4GWEcZTw1LZJaNIjp3wZVe39jT4wXsP4VDa3xZ+7rqquI3ZxKpB1u2QkM+znefl/8hHIyVyI0YOjuSG/z93qi4xWfitpOiBKLmpNB9d3br7rSTSbz8GobGQ5Ns+No69P9JYcUaI29kJmitiHOuiHg2pfpmGnn3rovroPu/qSmYptzeiTa2wv2SZk4lNMKLlPMreu0V+fQYcTxk+kE3TZicAg5qqqnVtvbiUVnUGzV2dw3r17165LJ5OHtXnrhetK3shOAAXhhzoQlFEK3EwOvlGbHYl5C67PSJeoy/yDI6+NNFOWdKDun1MMGwJ9y8kVBwMHnoz8NkIjK0N9kB4FmdBGnxnntqUR8zy+s94wJmkU1iPzPGxrg49KvgcdyP4YsaE32R6PSaJ1nuNV40aYXinv8UxS3lSA9SvRHO3tc/TJbVrWyd1xf/KK1l1f55lDozyyLssReyYby/od8DwZmKc8bzbdnQAfcl5VH3Bgl5eXHaMfDB6PvGDz9Wq1assfcjq4apuCDofDOj8/r9Vq1RayGhkx6HSMRbMoTA4UMyqOSgjUkwyOlMfHx3V4eFij0eOSjcPDw86xPskn+H9KThTYiVnJrdjmDXE2ri+dsvuTKRPPof/Aa7jIyWTSOwNJvTh1p+Y+GgVUTKACIfWhRZwEium2U9br/ul7HB399ypzoydWxPvdAfQZrsxplh0/MmLBctV2xi+RBA7JsnawsWF7ZtVBgOfzbMvAxm5DzfGxY3XweQ4J+bDJTBEZG4AGfUu9SvTJc5EtZD798Nl2LP716bYEDvcJ30A7Qf99gc9j4xcJPVc+uJB1Mpm0xrI/cTab1XQ6rc1m094W9P79+44hDQaP+x5RIKLk9fV1Jwp7zdHR0VEHrSAAp4hGWiZDrQjmYBwN2WLkKO00kYFHYZN3svK7XSiP0QP3JTIzCnqOIOVZVri+wwtRqGzfaDTqcIn0JaOeEaodFs8yx+M6jFq8LaQvWvZxW/6sD4EyHjzHBy6at6IcHR09WTu1Xq/bxBJt90tCCJrwfSz29AyunTTBOsfZaNwOwfpX9TRjcdrE59Yt6uAZ5uGcdlFSj+zQ0sFyfQZaZOlA7HZxPdfC8zEmTg+9AD4zBsYXe/T5fp5Rzy1HfWUnBzadTtv+RIh2hMDx0ZzA6uNiOSfMKcn+/n5HEfhNYeM2nextrHgEpz728HSe/z0YLiYoE2VWbfmZ5EgywuXgunCfEVXW5771tc/GZBkYQfXVl840r7HTxgkyHpZHn4NxnaYZ+q5xX6zE5nrMddmpPier5HgcWByAkm8hc0g+xwaeXFXKLIObuUjrgumHvn5VbfXCKSjXWl/62uSATjv6OK8cAxyp5UWbq6qD2CzjlIO/T10jC+gLarSLtBMQZL22jPv0yWUnAru9ve2kV/a+eVIi6SKNzGgBwU/Ewzi8Yp30sA+hIbiMhtRpwabHp91WCq7zEc+OnuYMzC/RFhy5I1pGZ6drqUy+LzkDUs4+XiYVlTU2FIzB6JI6gPvZFhSa53kG0u31SvpMnxIZuq15tlnyeEbdOKOsl3sSVVRVJ/1D1qPRqK37o3iNmI+s9ro/1ivyWXKijHO+bgwZWZfM7xkd8r3l69QfQyaFQ098fZ98kKWDkekDxtYUAm332Jg/s6Op6r7iz6jcxRvUE0WanjGVwGoEbJvnPBcUKR88kRWhOPVCEI6+9ugMBo3gEL/7+/uaTCbtdNbFovvW4PF43IHwNiq+J0pagTN60Q6ndnZOTlEw7OeQhnN+95P7kp/y/1zrGVs+x/kaotPHqu4ECnXz28rKb8bI9RoBmIdz34zyCCq5qyEjqTkytw/DoF1MjTvtNGndJ8M+xJyo02lhIl/66edkwZF5HDL981g6LU15+XOjSooJ7HyGeUCPIbb2XCaS/XcxGKCNfcjMIMF1mFczaqadpiCwce+T9RgaBSeKNK8HCjPa9MtWkoN12YnASA19SoQjXJ6dbljrwafROAKnd+Z60jERUe3x8eQWrpGVnZMdXqZ8GR1opyMSqTKRMCMsA+EI5etQWNdrXscDSTuIQH5W34mdNpg8w8m8hNsBZ8ZYpDKRUuOc0jhyD6WdJs93kMOBZoTOiRn6AkVhjvL+/r7tBuF6SsrOTpHn4JTzPvrJ+DCBkc5wvX482txjYELabcjgapSeFIKdUKIg6591hud7TCl27paTbaav2EEZeVGHj0bydxyf5e1+HncKzo1+g3KzrzmetOl/ReJzuCCppBVruVw2sp6CYqKIfMYPpGpGMIoH1x0yJE0nk/daKE5LvGHUTsDRBqHZELgfg37OkIx0HFHtyM3VeIO76zEqMwdStT1NwgSnnbNTdQeWjLAUy95biYxO3VecFw4z992xjcTyt25Q6IeDUHJgOBP2YGbwzPIcmvV9vtbOrS+V5LqqaiS/P3OAsS4k92Mds3O1U2MMvZzI+ml9f+5zZxEeE6dslnPfrKvtjns4ydizkVVbPUK+BIBMKb3ecLPZzr46hec7nD+y+Z8cqfPB90KypIHD7qoeHZvfLmLyLfe+cegbxkFhYgDICu9CneksxuPHAw9tmJREAjZ+Cw+Be+A4Kz95KRQuuQuKX3ji+vI6O3CXVGiMJHk6L/WwU2UtDs+08+B7vjMqsDNi2xcTLKQISX6DUJClF7Q6FfHsJW32LJSdi9GiHR4ONz/fbLYz2Xd3d52DKH04JgXnlJMf3MPYMQFklErBgRJ8jIAXi0XnjTqWrcfLY2R52UmkTjsI0AcvgeHz1DWebTu0jmXfzO1ZTknHZGZUtaVWfIKvA5mvd1oKp02gpG+0Dx0iyDx3lFXrx7PfVLWNnBiEp6FHo1FnxsKeHmWhQSAoKxEbt3Og7cTMZaxWq85O/4xk5rb8WjHanfA00R0CdHTO9Mi/OUU2Oat0VE5ls9iAmPXMaNOXGiW0RkZ8lsS9r6FOf59ps1dU2wCdblgWz/EwvravJPrFiWQ6lJ8n92XHTkkeiutoE/phHg99zUkhczMm9rPQRu8wSOdmGfEMc5aj0ejJ3k4HL9MjTmeNxPx/Ik/bIIEJNIQuuD1eKeCCXVlvrPsGLNbrTJlB9M7Y+I52/W4ERmfhDKjUHACF19D7wTgO8wAI3g7Lb45O4/L7F41kPEDj8bjjDD0IJhG9gdkDyf+gKvMOTg0pyeEkFO5LcVLpURAbkJ1gPtftdjDht4/H8efcw/ONSjE2jkNhbIiUPBOjYBW2nRxcSHI9KbMsdgY2vkTDNmz+x1kxY0hb/VJl6gdFkZI4OHl8q7bn2O/t7dX+/n6bfMLJ2pH0OfTxuLuX1+PB96ArE9TJCaczdh+RsakKZGKeyqjL6bIRK22yPNxW5AgKst0+PDx09nsmJZIBJO0J2/bsK+1Fdn2HKmTZicD8QCrHY97d3bXV7jQYA+R3enUPljtpiOoOemBwVI5cjoqgHEe89Pref5nkqNMKR6CMcP6dXB198v+OdhkN05D42xGpL5J7KYoDi7e3+Nn0weewMZaOsLkeztGSzzIgeDU1n3u8XJCZ0bVl1TdxkKm25ZZGQfGWloODg3r37l2nrbSddjNmtMvkNLppfUzk52DB0UKpj0lxuFjnn0N3GZx5dupKttF/M8bpGIza+1B1Xx/SsRjF+X6j16puIM4TUYyASVE/dCbY/+hEVufbHlSUaLPZnhufjTXqsgEYMZmD4TsvuAMyo2A4A5SKLUImAM1hYaQ+mykNNAeN5/lsoyx9aYGRZtXThYBGaHawqWhZj4u30IDG7Hwo3tJS9RSVVW2NHf6t76ww6vX9ufshHYK5P1MOoIm+lPbLL7+s09PThoRTHkZYlpGDHwaMM/zll1/q1atX7bP5fF7z+fzJIXwY1O3tbXu2qQicoYl2JheOjo7q8PCwJpNJnZyc1MuXL+vLL7+sP/zhD/XZZ5/VF198UZ9++mlnEssOM7MK5JhjxZjndTiTzHBMrVC4FvTExnPuZ7wMVsimMrPwi3zcFusK44HjtPM1wqefyZfaZ/SVnQjsxx9/rG+//bYtLBsMBnV8fFy3t7d1dHTUFgru7e21tV2kkjg6D5Y3gvoFFPBJHJqIAiXByUwNsBa4nmlLojgGhGtJhxI7BQYAACAASURBVMwtmetypMT46Mfd3V1n5X9flB4MBu14oD6eKTmtJFkxDuo3csn0bLFY1CeffNKUk90RJmapm5XoboP/p262i/E557U9d0aaFZn25jlRntzxixvclvv7+zaLys4PorSX8xgtjMfjzktQcbqTyaTNYk0mk87xM5vNpukr4zccDms+n9fR0VFdX193jki3TDwWrn9/f79NMt3e3tbV1VXN5/O21Y46Pv300/rpp58a7cHnfVwl32ETuevBpDeoCvlZ76jfiJ/rGIdcRuL7fe6/Sy7wfi5Agn651hOAi8WibTfkedgN9f/u43Sm02m9ffu2Dfzx8XHzspPJpC2XYAbx6OioIatcZ0N0MO8Dj4Hz4lgdBhJBcI0dEwPHKROguKp+lOhtUNRr0tUFDsgowJMC3J8DaviOcTjlwyFZoRKx0kaCg1EoTteRdjKZPFn0akTHW5n6NjLTb0N5Ip6dc57qauSNPmBg0ApOTZ1241ydlrifRgw4MoyLWUhPzZsnzTQF1GRODdmYHuAenP54PK7j4+OOg7c+MHu7WCzq9va2bm9vW0B2n1ldfnJyUuv1uv7617/W9fV1/fDDDzWdTutPf/pTc3hkHegHssZponOMIzrmoMdnnBHnNNaUi20jl4ckb7lerzsOhuuchjrdQ0ZG833H6SToMIq2o+vLQFx2frtarWo6nTaF8QkUoCauI0Ixc4nwbDAoJ9EQJ4JiJxnObwRC5/zSUKMcnsFzPavj65xmGSkxYNTjNJOSZ1RZwCiEnW/Vdr0Un/l+2um01vyOOT8+x8GYf2HVO8VrtjB6w3hka7iPk2Ua3MrE/Xbijuo4MrjGHPM+bjD5Pfqe6BF0lMbXl9o7pfHxyl6zSHroZyMP80hGMtZ1xhPDZjmQj1LH4cGl3d3d1fn5eV1cXLQJk8Vi0V4cnbyeZZLy4n/rbKZdyQ/iYL0onHt5DmPoWXujaRf/77/7OExfY2dmrjD7TFAEhT9XdjowKgI1GR1ZkebzeZ2ennaWVeDsSBdubm4ap0bjq546AP9N1K3qHmaYkZPBBDnRdtpqktIIyMVpHM8G0tqR2TnzHORBW9wXOwHXzzMTZbmt9CHPR4O/wPkYyrs4FfV19AXei+fc39835OGgkOkAdVsXXGgbz4aCAE0kMW90bmSQqZpf0IKTAwmBCBhvk8pGKqRZlpGNFwfsCQ6na5alJx0Wi0Xd3Ny0e//4xz92TluYTCZ1enpav/zyS5sAI3sAKDg9PTo6qvF4XL/99ltL0a2XTtctw9S/1Pu81g67apveMbZ8brSVjscBLTnfdGBJiaCjbiv2ANL93SnkYDBoZ/5MJpM263V/f99BFazYx8ujfCgEUJqUxXwJL8ZEQQzj6YCjCErTx5exOLaqu//QENw5O3878sI5pJIwOCZ2q6rDD5hgzXTISMXRMdOAXI3thX+MCXX69AiMFydj5zUejzszw4yB03/agzL7eozaM56ejfSSCsbeiMUI2zSAEbdlilNi/GijeSmclBGlN2+jdyB1zvFaLpctLfXY0i/0ijbSHvNFlqmRMgF6OBzW69ev6927d3V4eFiXl5dVVfX69es6PDysv/u7v2sLqDm92OnhfD5vu1+Oj49rOp3WbDbrbN3L1BFn49e+JenOcyxvB1TuAdkTLGwzznD6shZs2X8bLTuoeKyyHVWPmQvyea58EIFl9B0Oh81hkbbk6+dRqpubmzo9PW3KxObt2WzWuJK+QUFwwGwMFeK1qrsA1fyEoxKDZOVD4MnlYDR8lwjLxD7ludlJrjd6ozjfxwjNMVEvzpqB9uQDs192KuaJ7LgNzz3BkXIARXtyw3yl0wgIYKeAKCXBxU4RA8C5+R2IzEoaSYB8U6n9khhm0NC34fDxIExPwcN7cj3tINAhQxv1eLydteYeAoNfWOM1e9TrID0YDOrFixe1Wq2a48IRL5fLTto4n8/r5uamxuNxM9jPPvusBoNBc2TT6bST3r958+aJfjJedjqWqbMeo3sHUuwnZWKdIfit10+XcVA/bcWRJc3znO3wPfV54qmvfPA4HchI3hFpo/Eh/Dibh4eHms1mLdLxGYoCUU9KCvlub2/FITJ5X5QdHkLEUdAO9lxZUOPx4wmd3hfp6MoAcZ1L8hPJSSB481iZDhqZmPszf5GcmGF/VbWZV/gM82Buj98EbX4NOVjJuJ5UhQBkJ7+3t9dSQQwxSfOmVP99vxdfVlXjg9xHp8f8Nj9aVW3pg/fEemsNr3yz0yQFOz4+7siVVNScIgEMXYYvG48fJ5d46xN9ZTkHY8J2I+8BPj09ba8ePD8/r729vbq5ualff/211WPU//XXX9fR0VFdXl7W119/3caWSbT5fF5v376t29vbWi6X9cknn9SLFy86wS11kVTeKbllkcE4dRq5OI0jC6MYIZlW8GsUsX/oBNuGg6ULwb1Pv1x2IjCEa+IyPT7XVW0jLMZjKA4flhuAzZslr+IowbQ6kSyXQZgjOjw8bHwLioUjYFXxev04a4oi+pnU55kr2mdH6ZzdUc5IkL/T2XMf9dt581r14XBYt7e3nXVKpO9EfgwNx418bWDm2aqqEaMYL0rG3+YejTwxCsYjFYv+YTj5Ehb3n9TUaM31ZFrP2IH2efWbjdMBzTpbtSWxjQSsx5YlKJaUlDrtbEFS3OuUHN2nDQQJjlXnxTbX19d1d3dXn376aV1fXzduc39/vy4uLmq5XNZ0Oq2Tk5P65Zdf6vr6uiaTSf3888/14sWLphMnJye1t7dXt7e3nUkU2ydy93HwDu59kwSU+/v7Dp9pWRmZ43QS1SZx73Hy54w9ss0Mqq/sdGAXFxf19u3bJ05js3mcvr+5uenwVTgBiPSTk5PWEAvFXASKVFWdNBMDZfEgjsmDQV2LxaKtt0HprWjpdFF6H+lLfeYWMhpVbZ1ZrhOycdqIXHBuSWBnQKBu1nXRZ9ARhuN1VjyL2WE7dORgR+10AV7z9PS0XYNBuj0YMH0zmetUtmp74oQdfa4XQ6dyv53Hy2SxdxIwZugd35n8x7i8dpA+8wwMMhEKz/YEAwaLbHGK1v3keeGBva6NCYbpdNqyFOqBWrm9vW39QP4vX76s5XLZ0NpsNqv5fN5S1MlkUkdHRzWZTNpaNgcrHKv1AIdmZ2en5CBQ1f8OTuszz3K6TZ995hcHRBiooA9pj7sc2M4U8rfffquzs7N6//593d7ets8PDg7q5uampWlEUVbEk6ZBPFZt824i78nJSVv/glfmWrgopyF4ZZxjOjEmGfjOu9gtAITrejBmOwKnnzy/qnvuvK8z0vCzuBdZMdhGb/xG6b0oE0eFkZNyg+5yDRgol2fA7TmFJY2q2iI13vRTtSVjDfetVIwZz8CwWajclEtOEGOu2gYDnudjbMw/GqE6gJgyIAh6lT46hvL7NFLSQ6fUpIVcm4GJoJGcmwMS4zKZTFqwub+/bzOT9JVTRzzuBwcH7dSXzWZTb9++rXfv3rXFxOgQyG2z2dQPP/zQaJjpdFq//vprnZ2d1du3b+unn36q1WpVL168qPPz8zo6OmryoJ+5di77mycR+3sKji4zFoI/tlLVPU+fFJNxxXZA1IwhiPR3L6O4vLzsvKyD4kWqnjFCmT1r5Qhs5wAhj7J5QI34PJuIQVGHDY57qev09LStHTK5SuEFInZG3t6EwM2Tua0WdBLmJoxzUK0IRq8MlGf67CBoR86Q2SkYbfm5yA8lY5bLywSYxk/kZA6La9kL65cxOOXDAXK9DYEUFGX2rKkXTFpfqKeqWiDd29ur4+Pjuru7q4eHhzo6Oqr7+/tOxoA+8J3lS6oGGgKdOHCZmwE1gCI9RqA8xouZdfruVM1jYzS8Xq/r7Oysk3KBrpjdxNYGg0FDY0ym/f3f/317dSFp6s3NTdMXeMvJZNJZeEuwM4J0m9EdX8tYeybXjsZrBj1+th87QtuQC/z5704hUSo6grBzGwZ8kmc6mI52LoxiApNRUENH7jNpb0IfIRiiUnwPLxaxQpq8tJHwPYPqtiJ8rm2CE7dl58X1dny+PlPRvmN0UCRkZ0NxwOA7HBOR/vj4uHNWFWggOSgcjmf0GA8fBZ3OeL1eP0ktHIC4FmPI6XS33csxMi21knt8OJsO50VfxuNxnZ6etuC2WCya86raHqFN3d5O437TfhbQOoCATNGlXNBp3s/9A63YkTP+zCrTLtKry8vLpuvehQB6Qfdubm5qMBjUzz//XFXbSQVOe8AJMCYsvHX70O/kZ20LHkdSRU+kcK8XpTPmVd3FvX1vIvKEBPbdl9m4/I+O03E6ZqdlLovIYO+KJ3f6g3HZkHivpI0weRtzQSYATSx6+tuCQwikuCh8cmB2UGxBYeDNn1VVm2XxYDvS+r1/o9GoM22Og0OxIM9xznd3d+1vFIdi9IKMGQs4IB8Qh8KYkyMKZvREqXCe5kKs2HZCFHNqOAmnkzynz3k5HfeSHCMdy8POE6dEXwlA6BaGdnh42HnxDHWZo0Wvjo6OOgtHU+e5zmmRObFMo0kbfZSOZWLHxup8c7TMPONUV6tVW15xdXVVt7e39fr1687eTB/rY44RXcUmjHirtjPFBDZS9hxvnBj1YFek9bYVp/xV3Zl000BuX+4YeK7sRGCsDTKpSgFSIgQrHhHk5OSkoTUUNjksKwkD6tNOiU5OwZz+WaDJy2CkpJB9DsSpS9XT7Q1uqz9z1Od/nk1dKBtK7dTKzt1paSpK9p02egYMWTltsaJmnXbipFqkkKRIOGgjCKeCKR+uSYRBagAKol19vIapB1/Db+qkn17UbFLaqJeAOJ/P2yygjcmODmdmhMvyEfQHRNc3yQCiM+Kl3UaK9IFreFl0jikAwbN6o9GoXr582YAFy0QODw/r9PS0s0cYu7NzxeFmcEraI3UO+Rph01ZkkDpBH5hYMh9qh52rAdCTPg4uywcXsjIoNB5kdHJyUrPZ7Enqw+D4RANIUARhISVacgch9hwBUHLzB0m2e4qfuiBI4c2IVIvFojOLZB6NwbFzdtTNwetDAXnaJMaJEtFOnsd9Nl6no1ZkDMMGm2+KypQMoxgMBvX//t//ay8ornpENm/fvm0G79SYdWJVXaQJUsIZgwyQlWfR6D+GRf+437yjuSlkgq4kL2g0l+m4lzDwco6zs7M22121TXns6AnQOBCQlM9iA61gfCx/2Ww2HY7JM3/IC96OAIqcqZ9AD3okaNE224hpkqQJTJAjdxA5z3EWwLjk9h76C7GeqV2CHDtvZwzOaniuddb1kEb+bg6MQUXIKLUdT1X3JE0iOANKjuvUw4pib2yvj6L689Fo1FmNTbEymtdxpKXtfVwUEWo2m3UQHzMhdgR+poWN8cGR2FkZETGojixEGoyWAkz30oBcg5XHlPg318Of2TET4a6vrzvLM/iB+0SRHX2tnBSe4QmfjNg4T/SDgONJE/gtEIv1Cwdh5Dyfz5sTNsq206adpHLv3r1r6aSdJtfliSggUq9HY38jxDhvTmIpEGmmZ+Ft9JlKI+fDw8N2LA/9vrq6aujSOkcQs9OwXVqPWDfHcVAHBwdt3Zjlbx2l/6y/XK/X9f79+6ravg3dugwadtClrRkQrRtca5kz/jn5lmWnA7u/v28zIMw6ofxwJsBgIDGK5XTAg+foUVUduF7V3cCMgjK4KJH5K0fOhKVA/SedjuiEgB11cEJ9qRJ1VG0VEBl46QDRmP6no0si1e2p6jpxvqcddnROCYzqSIlAajxzMHg8140xct9Go8f1SZD4dpB2XEkZIFN2WlhuufLdMoYvMbqyolufrFP007pjLhC58xynYaenp7VeP661AmWNx9tjg0DUnh0DBYFUuNZptvUc2gLnQTaCgXttFsjZa8Zot+2CIIEesaczJwqMXv28vb29hvqur687lAv6wMwmsicIz2azGg6HHR5xMpk050r/zXu5jpyN9VloGajgTz80A1n1AQd2dnbWBA9ywonYcVVtN/K2iqXsjoQoIo13+sU9DA5QHOXmepSD9UiktXSWvZp98BOlRlhOE1E4OwcG2CiIAg8D5+fI7T5zn6MjSpazkObSSEG9Atz/pwPCYTklwrkwFukgGbsc39y3SNs88UAdOB2iJuPpCJsTHXZkTt1znDCsPnSBE8DILH+T5TgdkISdx3C4PQYnTwb2eFRVB10gN/TW/eMwRVALTspOCB3NPoCsTMdwDh9yoO67u7vWV3TDqSr3W1+NNI2eWJ4yHo87a9Lo993dXW02m0YR2Ma4hsk5y9BgJTeaJ4eWCC3XDfaVnQ7sb3/7W33zzTdNsc/OzqqqWm6Kst/d3TXlsWICqYkwnFjBLBKpHwbh0xmtfMBx82I4lSSy/Xw+w3nQdpwVjgzj6CMjq56emMkAcL/vZUC9UJRi9FDVJSpJvZzGoLTmDYwyR6PHY1dIFVESz8baEOwknE67nSZwjbSqqo0fcuR55iD533LN9XUep6QT7PT5znUiX7IA2pWbkqnPbfOSoM1m09AlJDInsqacucdtt8ytR+Z7TbFg8EbL0C7eAsZYoKdGU94ORfCmjTiv5JOMIglyw+Gwrq6u6vj4uM3MozOJfjlZ9uTkpKoeucSqqqurq7bHk0Dq4I9+Ovjapj2mdmIU9pV6Rrev7HRg3377bd3c3NR6va6Li4v2ksuq7pE4k8mkoQmTnsfHx41n4QWlOQvJDAUOiQH3Z0ZmKFMKGoXJNAThUSeO17NQroNUgr9x3j4dAqFTcEImUNOxoTheLJm8iu9LPpA22rBBW7S1b90Rf6fjSgQKf2lE4NTIi1udnvWlfDxn12GRdswu1A9qsbPzc3DeObFiRE7BkHnmcrls/BV6jHNm6w0G6eBlBOgForQ5t0XRf1CzOVGCu6kEE/xepsFvr32kv+gJvJuR/nrdPRnXSzTOz887C3lBhpyKAQrf29uri4uLDsqz/dB/xp3f5mlNEfC/gQX984QHTtVv8e4rO93bL7/8Up9++mmtVqu2NwvBcfqqyWCioo8OYfuD07aqbu7rzts4QFieJLABorCcPMAguxB1+J77bDh2Rj7JAiXkt+/BWRp5wi+R9yMbIqn7RkSt6q74TqO2A3AqCKJ1imhHZ8NMrsj9pYBIHPE9PiDnRG6JlukDqNlLbSjmvrie+00eY7DMErOOijqJ/Ci/F/VafqtVd8ExPOV8Pn+yaZj67aAxKHOVyNuptPfw0nbSrqrtMczm43Aw3t9atSWxCWhGw76/qtrWPdAjYw6gqKqW+r18+bLOz8/r7OysHh4e6u7urk0aDIfDuri4qLOzs86WNQI4Y+EZV2Rtu7SOGWxYf7nGmY3XSuLsLfe+shOBnZyc1Pv371sjnA4Au2mID8+D5KTghHI2zgS+0w3+T9KaLRo4JAbdz7JC2OvbSOyIrYxGUTjNTGsc2b1OxY5pMNhuS/HyEa7ByFAM6jCyQy6WB7Jg5o0Xq7hflqWVxo7HKJHvQSC0P+XkfXjI1Olj3wRDciGURFiemcMReL2cUayDoBcTG2FzDanZYPC4zs3LEAiQIBfGIXlUdOn29rY++eSTDhqDUE99NnJC9pvNpp2D50mfqmrZCXrtpT4U2sZ9Rs84bmR1fHzc2sW2oel0WldXV61eNn8bieIcCWY+ogi7g49jH6YdjdNFj7VlZhRvrjuDHIExeecsOxHY+/fvW4TnhwcicJTMvAHrw5iupYMecKKNEUYfj1G13Xrk86QQBobp9NFOwIZvo0w+xsLlPiKQNxtTjOJ8vzksitEW6A4DMuqkLqNTOx5IUvNQOMOckmYGijqNct0GnlFVbSY2USwpV6YBXsVOO73i3+dj4fyRuVEKKYkdn9NBb4syb8IYIheegzF5CxZby0yQ39zcNPl4I7oXNhtp4JAdyDB+vjcvheOiPcgGFMZPktRs0AZZYoOe4TZiM79pR4feffrppw39XV5e1vHxcduu57PNaK/bhTMzj4UOQOzTX9sCBVslWBB0kjrAWXK9fcPvJvF5m4rTORQo9y5WdY9ENvSnYyamq6rDYaDAHIsD0khSvy8VynTHKMrCIq0xYiNSWhEdnT0w5gvoPwbP5+aWeF4anJ9L+3OdmtEPA08AsXJ4ywmycb0pE+Tsl7JUVWdNj1EYbe1LQ+3MMn3kPu4BlSOzqu45VdQHCqB4GxvpuJergMzNs+E87u7u2okoRnYsDeK3I/1qtaqbm5uWSnIdnCWOl3TN8iYNM8I1GkdOpPYOIjhcO72UEfK1I4Gq8T2evTNnNZ1Oa29vr53Jj5NwtkKbQYHr9eNyk9PT02ZnrCXjhSTMhlKoyyja9uag56VO1N9X13PlgwcaovBegGhCG2WBkDRJDUREQEk4e8EfEcrnjnFEMOuRcp0MCmcPjZKmMTtNQEmsVBYWCsbftJkdABY20dlrvqgrc3630w4AOYM2bPiLxaKur6/r7OzsyWRBErpWEOrjc08S2IEbVeaMj5eW9CkSyu9XwFVtSXrXz/1u92rVnQEEJVHQKeTr9VMYB4YEkiMQ8bO/v98ckB26xwDOxbwqhmydANGwhMHr37gehJVoxkjOBoocaQ9jzN9kMYwNW/OMUJGHJ4noP9whdkR7QW0gVIr7jK1z7l9VF3Ey5hzPTbup31mPnSrPzEkn5MAkhm3oubLz299++61Nn3oRJEpNh40gGACMEuTA/wjCJwFgtF7chrHhNFhbA6zlGV5LRt1wOeZ7Dg4OnixYRGlAPiYXbXAYqq8zXLd8uM/twklTEq2gLP7cZ25dXFx0vnP67FTF8uM5yMRr1QgCyINxNWdDGonhwrvZ+Jl1g1xGTn1KZ/QBgX56eto2rg8Gg7q8vOykY69evaqqarPfPse+qppugoa8kyANBqMyAqAfFOTo4IQxJYVwdHTU1iLStnROjC2OxJyVjdfUxmKx6CBUdJU+TafTjlOynu7t7XXO+SIQIxN0zP87YKxWqzo9PW0BBTl8/vnnHb1FX+zs0EMHUqNvbITimVdPrOAjBoNBa8suBLaTA2PNx3q9bm8WQrlM/nnAvFLYHBEHvdlQUAS8rfksK5WVzqiPXe3Ux9Qvg8T9FAva5KIJYvJ6c1Dci9KZR7NCWvlZHJmIgz7wXJPQoMTBYNDu59o+FMS6OTtar5czn4cDRtFBK6AE/qY+85f0ge0y5jFs2B5fox2npnyGI1qtVm1Ly+3tbTvDar1e18uXL+vly5c1nU5rOp226yeTSZ2cnNR0Om3ohOeAWngObccxVm0PfTT1wBjY4ZgyMFda9cjvTSaT5lTRH6PtDDYeS+zGzt7cMjI16sexoN+2FezRyzIoPB9UV7V9Byjy8ub9m5ubhqyRkXUvAzB9ZFLHDph2Uw/PR3dM89jhTSaTThr+XNmJwByxmIVgMOmsG0jDHMm5F8Nh5TOGRUrD/31kOZ4ZpSIVNDFrRGRH4nvtDIlGvt+KwACY/3G76LdTRwbD98PPOMXgf/cNfuvNmze1WCzq/Py84yCA/8ia/zFaFMMLdau6fBh8Jf0BwRkh+yUj/t7podNUO6aUmbkwzw7j9Pkuz2GDfzFKGQ6HdXJyUuPx9qyvX375pZMu+oBG9ImgZZRCffzmOqc9Od5uC/1izRTtN5pgrJJ7NE/qtXrJHVKsK6ZLvLaP9BT9zfVitJlTXKoe0/Pr6+v2HC8KNzBYr9d1c3PTdLJqu9TIARSny0LenKHmmvF43OHNs/3mC+24nysfJPFfv35dVd1pTaakIRDt7Z1mIXgjF6dTfdtWjDoypTKhn167qrsUwtHBPJfvtUIRhVar7fsEvRzE5CsRyegKpcy/GXBD6/yePr5+/bopkNucvA/yxZiRi9GloyVyof+WqRdqJv/Bs4zcTCjzmVee0z+UEMSe13C/jdhr2pKYtmNzekJ5eHhoa514NwKfJ2fj5RhOl3HyRog+ngZ0m6jJCzut23bUdvSgdf42yqJYtxxgGRscgJd7MMGDEyPY3d3d1fHxcb17966Wy2VbTsHaMS9SxhHZoR4cHLQz9pG/f6dOmJ/jf/SUsUxg5GuRCTL63SnkDz/8UOfn520Zg5XZAjdpR27t6OU0BXgOGWli28rtNJS6EG4qsGdEva4q00T+9iC5DqKpkVVG6PybduAQcv0bvxN++x2Xg8GgbYo9Pz9v63hMGlOMboxG7TyqtukT8vPSABP8iSp9Ppn3XHpLEbKjbvePa7Iddk7oCYhvNps1ngu0hFOjnTyPyOzUjNTEJ5C+ffu2MwteteVuNpvH/YoEYMbUskIWfG7nlVmCdW25XLZXjeV+3dSLqi53aueNTZkjZfwZw0xd7+/v6+7urqX6oCDQKU75+Pi4vvrqq3b8EUdxw5d5HOkzbWIxMTZtHUlZ2raxdUCC0SOBCufvNWkfKjsd2DfffNMiHxCTAfQhchCPHFWL0FnRbMXAkCBe+wYIZWAKGMdiJ2DCD0XmxRRGFWmkDKjPYDJaRHBOmxw9vT7G7ed+Rx8PIBwDysaufhSeiQuQlB2AZ2jsnP3SB+TA6bhuC9czfl7nBSXQd/pGX/rsfhsZ4EiyMDY8i+uM5uDViMbZZ8vRqb8jNEY6GAzauVXz+bzpBDtJIOUJeui1U0rkAyGO46PNHiujKuqy3meajoxtnJli4RzMo/Ec+EEHT/SKvoIIvS0IPYFbImg4sDvgoacHBwfNbuFBCTA+Ftr0CRwd7TT/SiDiPsba/KEDclIUT/Tr2W/qcSErhpY8h72o3+3HYOPYgISOXBx1YxIc2ItioQB2RjaQ5Lccke28INQd3fySDxylD6uzw/PhdbQVhTK5i+A928hn6/X2nCTSGdZ2QZY7RTAfYWiPXDK18jQ2/eY3bbLD8XIM2kV6kGuq+O36ab+fxXXmymwcHr90ZBj8mzdv2kuUOZttsVjU559/3sbJYw4yMNey2Ww3aROkvA8Vg2EcQW/QCOg3dERSOAAAIABJREFU6IKlD4xNLocwN4VM/HpAuCefk5cv0sDwzQUBEoxyzXviBGlTVdW//uu/1j/90z91HDoTJDhTHKOpEWyHvxlXxna5XLbtgVXVQV3IxbaXY4NOmA+0HvSdBEMw8nP6yk4Exrsf6SicEFyQIZ8b6KM1lstle8lsVT0xJtCGV21TktcxscwgG7au1+sG2U0K82wGKhfYGuajjOb1fGwIEcRpMs/yZIIdPPJisGmn7+E7il855lSYaEa6SttJG/gxV0ihDqek5tU4t4q25SQHcs6jk5CBi9Gtr3GkNkFvRAKPxfVXV1cNLbCq3vJ3tMZhMW6k2lwP4Q/qM88D+gKRmj4wlUF9XM81BCf2qCYFwt5L0nOQCnqVSBo54mQ9keZ24JRx3BwKmejZM6Ec0kDfnGbygw7gXAeDQdvXyRvLbWNul/tEP5yO9uk81zHOjN2ushOBXV1d1cXFRTsAzmgBBaPAgXF2UFW13f2kjfa8/O89YEQKIxzQCB01mZ9H7RqhEBFwgFYQb9Y2Z0UhUtHWRBl+lu9x31DwRDrIjDPoHR3twHH0TltQOK+hwwF7gsOK6v4hT0N698HIoo+zSSftIODrqcMGRPtctwMAJySYK2Mf7s3NTW02m3rz5k3d3t7WyclJ3d/ft7PhkY3RuutnsWxfuofDQk4sGEanTDVY3z3zi+MwwmYCwFwOOoATQLfQU9A+smVJkNdRpT56Iu3s7KzW68dJtvPz84a+mH1ETzJYABRMObAbgVX2oGT6zjh6wSzjnvU7m0guL4Ns8nr+rq98cBYSaD+dThvxTL5dtfWyXkFs5bDCQ9QBr+mAHRQDmySx0wSM1ttoDOlRAi+uMy9nB2VDN+eTMyCOYqlEtCs5kb5noJh+hu93HzyAdnJGgThSo2Onbk4n+c5Q3sZn+aJIPMfjaOdZ1V064eNbfFIHY0Ib/DkFtEnKwn5A6ry5uamTk5PmbF69elXr9bo+++yz1jYcR6Iyj48dkdMWdPbNmzc1Ho9rOp029M0kAYG1art3FBm4TusMcnJARdagJZwd+ko6yec4CsYajtDk//n5eX311Vc1GAzq3bt3HXLdNumFtUZJRlzomNeL+XvQH0EZ6iiDInUlxeExsM5uNpuGfpFBZmYuOx3YcrlsnteOgKlbk9dc6xy+arsdhQaRfjjFsCKnAJLI6zNCFxu8o5udnF9OQb2elOB+X2MlTD6Q//2byEk0B005ImVdNjrXZ3lWPXWWRqyWEcVK6tQLbq9q63CNFBPe9/U7n039fcGH69LY3V6+T66KZzML5kWmfis2HI/T/iweS39GkGO9E/yp02U7cYKGDzVAdsnz8LcdEK8ZPDh4PKcfBGbKwHKCPLeNWTaj0eNSHMs0ETXLK1iIC93jYOQglJSLwQZOkHSXDMxIns8tb6fiJvCpG/0wzfJc2enAjo6OGurCKyIEHu4IjRf2oIHImB1J53VwcNA4MqIhRmzkljyMZ0ZzZXiiIwbCvJQRHryFB7HPeewSJCUVCrLep4HaiBMVIB9/nugLhXQfGK9MiY3Wqrobr5mp47nJXfjaLJadZWO5OdhUdXceWEYgN/ps9Oo1cMyEUTdnot3e3tZyuWwryNHP8/PzJ0Ei++Ctbpa7Ax/6ikNlVg5nQpBiEihTPOqiX/SJTeHMHHM21+XlZedlvFXbBcp9XBOcGtu6KDyPjMNIGOfDxAYoajabtR0Oq9WqXr9+XdfX13V8fFzT6bQzNixbMVqvenoENJ970a63qjnzQB9t88+NX9UHHJgdFp7WCpkCJKUzdKWRQGHy8KrtiaImDau2bxByypmEpA3d6JAfKxG8D8oFeqROzzQZnrsd1McmXvqDseTAEpn39/fbK+Or6olzdwCgID/QKYQv93ON/0aWVd2junPfKv1KBEta4JXQ/i6fiVz53kHJXE7O6jpNNzJ1yty3s8PEL4HSlMDBwUE7WhqDvLm56ThATp9wO5AdY+zJjzQuO4F37941fgknxt69qu0Bgh5zEJTfls2qdztdOxWvSTOpbQQP2nfqavRK1vHw8NCQ6dXVVZ2enraN7l7H+ebNm/rkk0+aLn/77bftfqNR+sAMK89EZ5EV/WZW2fwhsqYNZEqMEc99ruyk+N+9e1fHx8dtILx/Dj7HEcVKjULgPFBGZhSNQOgI11hB6WjfOiUrI+0y75J80Gaz6RwiSBriwXfEcx1V1TE26uM0WJyjz/gyqjC6o02OlvANPIf+c6/r8KvekB2KbL6jj69aLpedo3M8XZ3jlxE1eTWcijkU2uGAY5TmtU600SlK8lVGarQDDof+uo7JZFLT6bQuLi7q/Py8Hfw4Ho/r9evX9d1333XQ3WAw6JyPlmkyz0Y2GFRmCp6gWa/XLTUE2XpSg9lEXkr75s2bTv9Ho8fjsk9PT5uecj/yPTw8bEteTE0Y4ZozcwpqfWNl/vv372u5fDz37fz8vC4vL9u91m2XzebxtFkDGQdxns//m82mrR2zz4Drsgwpu/ivqg8gsOPj44ZCrHA+jZWGez2RZx2dLzuCOy2q2jowH6OBYjqKo/wm091JDNj8gJ3GeDxuJ2PiiEFZCBBlyHY7Gpvnof2Qp35xCU6zL611n1y/5eJtFvTHb4ry/TgjoxU7G/63kpioh5/08+2ISbXdJsbMY2MEnKjN51f1pURE8eR7LD87xVR89GU4fHwnA+9sZEzW68d3G2JAe3t7dXx83HHilqtTTOgQ6wV9XK1W9f79+9b38/Pzlg5NJpMO5YIT96ZtApTTTGQLLeAjyK13XmPowEYfcz8ryO/k5KRubm5qMpl0bNpvIbq4uGjo0EEJpGQ/gHwANubhQH+0tY/3ts9Yr7fnuj1HY1R9wIG9evWq/vjHP7aji+EgvBGTxtiofD4UgvZ79Wg0SsxsJakonXFdTB6gpGw5yfQGxehzOkTOTBOqqgm36ikH5pQop+oNjauqGYOdnBFH5vPpUMwJ8Zvrkrj37/ysj6h2Ks53tI9xtDLRdyOLqqfLVRhPO/eUYZ8jywKPRFpY1T1H3mgMw3AKQorF8dcggPH48ZhlCH4iPvr266+/1tHRUTs7zC9wtsNy3ywHIyt0/ubmptENLDlij+tisWipI21ChrTPzzo5OWmnQ3iirGq7W4C6jArdNnQRRwdq4ogk2m/C30EkJ+fQJ5/T51NAsp20DWSIbXqJCDwnIInx/90O7JNPPqnFYtFeweTZEZTIC81QFiuvTyRFUE4RiEYIxAZBfdTjaIW3NhIyr2Xy22kUA8RAUogSfaSvDTIhrdfA9BH/dhiWG39bIZzC8HnOSDklpS+Wq59TVb2fZbtANLQnl7nkBEJVdbg1TyogZ7eVa/y/73dxGp1kvmet3DfGBVl5fFnMav3DSZDqsX5qPB43Up11UMlZpgxdQI6k8AQzZjJ5mSz6RFDG+fs5Rrm0jYDul46kTmba6+LJCNphPfCyDduTHZGzmqpqkxm5Y4V7sI8cO4K5J25wVg5GXnbSV3ZyYEBHc0EgGGAiZTTaHsELvMzOApGpj6iE0bDuzB7X34/H45ZLAzEp5ohcUDLSB88c5WCj2C5e7+aoiLGnEqMYmV7wtxGU6/Lbe9JBJ9flsXBqQL3J/SGXRET8JC/X51iTd0NhUTAU1TJPB2WZoANWTgyfa5yG2Jnlmij6nQiXdpBSwoWxLebo6Kg5LF6g+9tvv9Xbt2/r7u6uXr9+Xb/++mvd3t62WW+PBfLweJsDBJmw3xU95n7TJcxswm2BwvyiFfYx8izWzRlROfVHVrQLztc6xfifnZ21lBoHybNpo8eMDAj+cLFYdLIYp/i5mt7ywd6cAntc/1cIbDwet/e0Oa1xtLXyOLXk/uR/QF10ns2h4/G4reBnIAaDQfsMx5b5vAsKayRHm5l5JKoQgXmWuTPqQpDpsJ3qZrGD4X9+J3ozP+fPfH0GANrGZ7mLwWjUxuZxeq495gC5jzqZvbKTcv/Na2Sa7Wtcb6aV1h8/x+Sw68l0h/4lYuT61E9kkg6SWeaTk5POzNvh4WG9f/++EeAOJC4OUlyDfAhUqb/wYdYHp6Wk15YN9WQmRB/6EDpB2o6tqtopEcgc2wWBPReMaItTxpSHx4lnZsrqQGS79AqBvrLTgc3n83acsR9ixGIexWuKaNjx8XGL0nYU8DF4YL/UAgUB4ZAP8+ovBOw0xJxXoh8vzvSKagaLSGiew8/xAOLUiWZEVD+zautEQTmZIiTSs7NBDsnpMaiJ/pzC5d8OIo6mTmNZ2sLKcsYrnUw6pTSMPofuscni++x03ffk07zWkPY5mDz3LAcBozUvHeF76/pyuWwH/719+7ZNAKzX6/riiy8aanUaaOoiqY5Mxzab7RIGZhR9dLZRJw7YRDgzh+xh9dYzByn2Z/7nf/5nffnll/XZZ5+1NpCqJW+22Ww675scjUZt5wCcN3K1jqR+IweehTxA2Z688TKT5If7ygcRmCEgAkS5aIAHJReqYhSGgaAhOmWjNVoYDocdDs1O0mmoIyHRCGeJ0B2tDKPNJaGwqWiObrPZrHMUUBptpm3mtZIrw6HCSTgNTgRio8yIVtVdfOrPkmgH7qOgVlqnjk5Ds247SnNbfY7D/U5Uyt+JNJNn8/W5Jshj4CAG8nBJxARn42f5Hk86YXiff/75kyUAjCEpnftgWTnoIlMcclInXrPlQFVVLSuiYBPejE1xSrder9vx3NiUUaudOO1ika4PDjDCfPXqVb18+bITFHIxbdod424ujnSb/nmd566y+5Uf/y0UEAreneNovNIXQpKIAPxECCAVQ33PtNERBpPBtvIDi71Mw4Qj9/roG0hPG5gjfVWXs3EOnlPEq9WqTk5OOoruSIuSpdCdbtkxEX2tFFa+JLqNItIppGz5Pvd4cr+5LBsI/IcREHUbRVOPzxZjPIzw7Ez60n6KnajlWVUdh4s+ofBJ9NNW62amu9Tp9V+JShkb0kn+Jp1er9dtZpCf1WrVVqwzu86MqN9VYLnSLztxnyZB1lFVnaOtTO7Td9eJA7y6uqrRaFRnZ2d1dXXV+D7AAEDBtshSFAcP84nOVkD3OG/qc3ZmX8HeW9rptN6B0k5yp948+81/N5YTHlnLUrX1+JREC0ZJ6/W6DXi+/dg5N6keHj+jJwpg+AmRSEHpMWAiJSjSBtjHnzjXtvOgLW4/deX/CXttkH1t5fn53JwA6eNN/Ix0aGm4NlQrCumpp7ApHp8+xIfM+ji8vvY9VxyZ85hr2u66Gdu+AEAxT5Z8DTRGpvU8w8/ebDaNh0WvE6lNp9M6OzvrtIEV9ayqx7gz2PWhWnSGrTrm8tB76oDOIFB7cmg8HrcN8YPB4x7P8XjcTsNAtl6lj+3ZBm0X3sVR9XjgA/phfXMW46UwqV/038GnqjrHDO0qOx0Y77+DyPQ6GsM+GsgAsJXGsJGG0EFy+uSO8mAzSqYG1GkvT3GU4PlOe30Yn08D8MI5nPXV1VWdn593CNY+h+HpXkco8392Ih7w55ybFb3vPHbX5Vml1WrV4YVQZjtpO+3kGxhjxsI7ClwyWrqPljtpViIxo8VMoekLQSKX0fga6wffO1U3uexnMU4OJsnlgUT8PeNhNIW8uObnn39uevr27du6uLjoLH8gZbMDRX52rE6PV6tVO5wT+6K+8/PzzrsUWIjqt0y5HqeJ5tb4306OuvpOGjGaoy2gLD93tVo9Qco+WNTIGsCR29D6yk4Hdnt722A2a1lsLObHHAlBRpmCJCFrA3UEohhNUAf/0+FMm7xEIlORTPu8MNBw3Ijq7Oysc48Now8l+P6qehLp87f7lX9TnH5QcobRiu/n2+mbcE/jNh+WiNGOzX3PtlKHv/fSA6dJiSj7xp0+54tHfE8iO7cvHZadQkb45KyyP04xfT26A2IhUBD0j46O6vz8vIbDYUsF2eCfAc0608d9bjbbrTiTyaSNNXYFT4UTYukLQTv5w77i/uNgHWQdJHICggKoQT481+lu3+Jk82FV1QElz5Wd68AgwL3AkxQvCxwBnUNJxuNxW1s0Ho87kdRIxINvGIxAvTrbe7KIiBRvT/EMk1NHojpRCKFXVXvlEyfR8ow+/ii5Lk9IZHridLWqe6gj9xlhGVE5kjn9TY6CH9rp39zrYoXy/0nu813f6nkX+pxpmdMfxoji/XJ+ViIUy4z7CJpeB5VUAPXRF/NNcKp2mKwuz3u9fsuycrAD7ezv79fR0VFNJpOmq4vFot68eVO//vpr/fDDD/Xdd9/Vf/3XfzV+KJ1tHmnDs1JWOE7Q2Xr9+Bo07Or4+LjtMqAvJvo5R5+3b3MIIvIydw1KRM5VXeDC2NuuMjCRWgI+/CJenCHX8PO7OTDenkx66C0YVd1D+4B7OAtzLJnn53orUJBTr5zZQtGcAiYKcBoK95WQ3AIkfaraTk2/f//+yVuW3Y7kvFzsnPK0jCx9HFofSuWZfXXgWB0xKZbbczwVzzaxmpEUpcxAkTNK7hMliXMrPaUvQvchPuo3cvTizXTmqTtup/vnZ/iIZF9vnUQeTpNpu3Wa34nK9vf36/z8vF68eNF0jtXsAIREXbTTdZvHdNsGg0E77Xe9Xjekh6657+Px9i1cfWOGI0K+ffrlJUnou+sDSDBm9iP8z1gSzLgn17b1lZ1XcF6UPTLKbC/rGUSvf/IgjEbbM4DoBAabyzBQIkdzE5eu3zNS6/W6rYVxusB3Rl5AbpTzt99+q+vr687RxxQbMYPIYOHsTNR6gImmJr35PtO8qqdrxHiOI1+mGp0BjbV61OHrIHuRDw4/jdN9deqLkqVS5/+kPDn54bTVsk0Zp0PLgOHn9qXmFDsbj1Efr4cO+7l2DhnM8jQPAgL2YTTJqn9Om2Wc37x5U2/evKn5fF7v3r2r7777rn7++ed69epVB7knpUIbDBz8nYMkM47IlHTWW664x7J02u+CzXsyxfw2v5Ghl0zQJiNy18HfyHZX6rsTgS0Wi0Y+JnnvyGlOKzkFE3EWRBpJn2HbiTm1JFJRUJREJDynqnt2laMmDm4ymdTx8XEbsOcQVnIlPoCtD+raISSnYdK2DYgCRR9ysKL2IdA+7oT+O8hQzJGNRt03c7skmnFJXtMcF+uj8l4jbMvGKXoabCI69zsdvZ9jWaecLGfvAUVmuePChmzEbJrAJ/7ybE944VB4tjOC/f39Ojk5qapqKSCnqNJu+rJabRdcr1ZPDxrYhYYTBTuVw077dNVjDTdu8t926VUAyMtj2RfEccaeNNxVdjowKnEHqrYOw5HUaVsu1rOS3N/ft5NDaayPjnX6lamEFc4pw2w2a051Mpl0PLw3JVNARff39/XTTz/Vixcveo8qqdo6YJOXWfpSCX+WBkN77ChQDn+e5DOKZSWhbhTCiNPbO5zmYagofiosPKADQgYF8yjmaLw5n3FwdKZYn4xoLGfk6F0ZaaROudx3GwrI0KmdEa/H2XXbofl0VnOFGYBc0G1PjHiGzTN71lWWX1RVe+EshzO+fPmy7u/v6/T0tINOMpil8zG/6rQw0Vz2x85qtVq1F6xcX1/XyclJO9PsuWKkj+yQhe3bOk0m9Rz90pHxri/ZYsJiOhtSoioIUQ9uQlpKrgvzEgEfMQ1Cg6icTqd1c3PT2oTD5IW6Jh2tDPBsVY+DdHR0VD/++GPt7++3dw7yHcoOKnCk7DsoMHkgpzs4WQzDEd2IxVwh8uxDYlYESqZBVpJcd7O3t9cQI+nPbDZrTr+qi2ZYGpDOkesYM/qYDibT2+Rt/HleQ18zEFD3cySxHavl1OdYzVP6oMXsb/JMlrvb6nag/zkLncgTA8dgIbc5yQKSHa7q7u6u3rx50+HOjKD8DIKVZYITRc/y+CoHAL7/7bffmtN7eHioo6Oj+uSTT+rs7KyzOh85mfdGh0ChiYRTh00NfYjAr/qAA5tMJm2tFgNBQ5imtTHTaX9mRXDKyDn5Nuo8OsN/HxwctCNOMD5HawvNhpaobX9/v/7617/WF1988Wy0cbqUaUcaMnLxy32rttEsrzPPZFRpY+DZdormg4yW2MOYfCFBwQiCcaQ+9oXSJxQNhff2Gm8AdhT1KQ2MIdc49TcfkilnOgGnK5Y5xYsioTeSwHfJIIkemhOyvhj9oiM2cK5zfZZ7VXeLEd8RcHk+aIw6vSfTDpeXlTAeX331Vc3n87q6umop/8XFRTvby/0Zj8cNUftz+pCOF+d+dXVVDw8P9dtvv7XlIC9evGh1Xlxc1GQyqTdv3nRk4smIqu3C16pt2urZxdFo1PEnOFg+N2fZV3Y6sPv7+xYJrBR4fwSaq2zNe0HeJ0fkiG8S19HOHJBTIuq0grUOyXD6VmnPZrP6+uuvnzij3JPYR5imQ/Nz801HfXyX+0JdWU+mJWm8OZHRl1aZ3Mf4cvkKsrCjSOMxgezx7ZN/piPcx/OSz+tLyZ0K+j6nqFXdZR99a4lyFsxIKmWdbXCwM9HsurKd/v859OlnE8gwZMveBwUY/RuNMxHAIlLqu7y8rPfv39fd3V17Gzi8Jg4C20y9I9tiKyBB7NNPP33St6pH3elzLH3cmlfkJyHPuOQBiBS2NT1Xdjqw2WzWTqMAxhIxMFT/BhXZ65po9Hsaqx6VxW/tdkQgsj9HtlZ1N46jcJ4ZRWmOj4/r+++/r5OTk7q4uGgnZfZxW30DbAWj+CjqLCgUbU/HkZMdJnvz3PGq7sD3GbwRg1N3o7z8Hvn1oUqjEHNgTjGyX0beRi/ez8qWNCN5rnd/ciIojTjb6pSIunNcjPCN3u0gfTY/8nFW4HTWekIG4N0LLslToj+JUhlrb/B2+1kMy+vYjABns1n98MMPLe2bzWb1/v37Oj09rfv7+7aY9u7urubzef34448NsXG8dNUj2ttsNu2I6aOjo9psNp2XKVOY2aSPBA9znunQkgbwotaqbgDPseorOx3Yixcv6t27d7VYLNpRuxCQ5rB4UQQoxDN+XIsjg3fJdM/G7WlsrvO79xBYRiZ/j2JOp9P67rvv6sWLF21vJ8K3QpoL4Nn+HIGaH8AoMp1I55W8kPtqdJvLIkza4zSSL8joDyLmt2eMbRB2XImQ/IxEjkZQjJ37hNMySkOOTrPdFvN+fGdHmzyOnbKn7OlPbkEx1+Mxzz7YUVm3kt/kXqe49NcO0fVnUOjTO/OJmcrSP2TidZNsOKdMJpM6PDysi4uLzvHVP/zwQzuy+49//GNbMoSzHI/H7SUkq9Xj5AtIyzKwjvtcPY8nbXQhJcxVDAYetgc4t1082E4H9uOPP9Y333xT0+m0bm9v6/T0tNbrxxkCzvlGSTNiG5F5KtqvNMvNw5CO8DpWdtJWpztWbKcOjiCvXr1qb4hJHiAjpme0ntv/1wQ37h6qaCdlJR6Pt/vETNq6z8gLZewjrZMUtSPJv73q3byDnUI6qef66pTVaMTOr6+4D88tRXAabT0wggetols+ssYksBct98k20z1kmusWLXMjVZ+Rxr2JhjO7wAkaZWIr6Gxyl66jqjve3JNrH13QS3OgOAcm4gwe/KpENnWzbCPH37xc8nc4G2zVGY5tlUKQdWC2c+YVeF6w3Fd2OrCXL1+2WavpdNpWzpK/I1g2b+OY/CYhhOiUI2fZcgBZFsFgcqYYAkmkwnYM6p5MJvX69evWbpZMOOVB0LTD62mIpii2N2RXPd0aZCI6ERvOlgFKZ5OcDI6GOpL89qxg8mOJFF0vEy58x3256fs5jsSFqOu0wakUbUE+fWmxURtyIU1Kx22URGDJXSCO1Clb6uS5Hjdfz3OS3Hc6lxMuRk8pzz5yv298+AxHmQGeceWzTNUeHrZv/GFd1nA4bM8kqE2n0yYjiH3kitPy2FpfkzayrtEHp753d3edNN4BPZ1fonGe3TfTnGXnSnzPaODIaGRVd6YFIQHh6QiLQ1NZ/DIDhEhjmRLGSMn7UzEZIHJ0IPN//ud/1ieffNK8+Gr1OFvFe/qqtq9Yt1GQciJUnuXobCEbURGRWU9G8fd2dIb8VpZEEibHPbg2MOrM929SqBeZ2vnjpI2cUwesfFZgO3LDfXNltM/pD22yY+ZZ5rD6lgawewJHz1EyJt1zJwbtY4yMHjn9FCO2QZpvI2Vcr9dNzpY97fbpvk7T3X9zecnH8Vo77MMoEmK/akuBIAOQFnJYr9d1fX1dt7e37Tx+eGhOm+D4HOQ7HA7bsggHhL7JMPptYECw98GnBN2qLeeVOrhYLDqy9wTH/yqFdJqUM3vuBMVbVBCKnaANE5I+I23frFKmURYOQtnb26urq6tarVb18uXLJ/cYGWVxP0x826n2RU1klGjH19gxOJL7mUYzWaib5ydvYMcE0jDUp1gJkyTv6xclxyD7acSRiypzgsR9Teefz0+DT56M+x4eHpoeQQEgFwwBJGxDwzn4sxxLdDbRItcjByM/oxbrja9L5J3F12dQoY1ul7OG5C6TXPf49TmG5KdAnKmHOX583vc6tVydnzSIT4QZjUZt3en/ZB3Y/wiBkbbRQW/cJsUbjbZvpnH0cfplFMZg53YBrw9xLk97iIY+N/vy8rK9yv3s7KzDAfG85GAwclIA+tbnRGgrhueZI3NKfSXXtjm9sBFUbVGcn5WO32NjZOUUljSANvK8jHz+jlQ/HZ/bg+w99kac6ZwHg0EHYYAi0JE+p0R7/PdzyNcTNnxf1eVnQNkOHvztmeTkbqu6KJlnsM+R/jgggGbNYzkdS1TICSsGCk4nq7Y6ac6NvtnB+XBP6sSGvFEaxMnx088FE/OC6UTRe/eJseVv6IC+GXQjd8bLY8NnBKDfnULSqNVq+1Yf8m08/mr1OA27XC4bdES5x+PHWY1cB2ZuAiXnefnqKl4egNF4hT1n0//5z39uDhPl9GGJcAvA4/V63U6YRTFMBHsQLFTa4XSEwe9bq2JF9sFuThkT5VppiLR9xL9RR9U7/LoVAAAgAElEQVR2c66vt9LDn6QD4ToMOF8KbP4J2duhum2gQvpgnsOkrR1nppI2AqdWFGb67LwxFAdEo4A+1LvZbA/7y7FgjJhIsjNiBt0OA9qDgO1tchmEIMOrti8arurO2HF2lw0YZ5zXIkPGHf32b/PDfsuYHVEWDnFAv7ER+uCgwm9nJZa1MwM700RmyVt+CBxUfcCBIQDWhvBABsuRzE6LQoftoOigowQDicElCcp1RBqM8f379/W3v/2tXr58WV988UVnYMxdMINJ5EFxyd9tSI6YRhzJPVFodx+y8TXUhVxRZPMhyNQpDvKxTIxy/Qxklam9Uw2ucVs90WDERb/MB9oZpPInsWty36g4U1jrhQME1/Sla3aUDjAgCz63LN1n5GcjxYHSj4ODgzZBRfu8oj8zh6pqeua01frO/2yARrbuA1kN8uG9FC59KS31OTWmbi9h8DimDK132AjOy4GOMci03NQBqagdn/nyvJd7vDRm1wxk1QccGFOskKRWOKePLFkgNQMa574/GwTRyhHZRp1RgRy5qtqWhvl8Xt9880176zHPyRMw7ED424cyJq9hxUaR3R4bQ6Y16YSt5FZSO9nkK8ypVFVDpTmYfYPreincn7yHUXGfUbrYIFer7bHBbosVd5fiZdrrtjgYoBsei3REnlQyyiPKm7+iGPFWVdND2uQMwDPV5tGqtkttHDjddp6Fo/epHIkq08lmkFksFh2nzeceVyMqHyJqWzAa41qj3RwX9xUU5X5R2EHgwE8fcFzmSOHLDFJAZaYsPF595YMnhq3X687bUPgMJ8CeRjoOvAYleXAZBENdjB3nZYiK86BuXkrwH//xH/X+/fv24gS/eaeq+4YWhENJSFzVTRftUEibrLjpXDMlsyI71eQ5lqsjoe/B6HDaduqOfukkzIM5peDePOUz+bk8TibJbiMxyHO4T9pq/pKxtiG5zcwyZvSnDX63IUqdiyMHg0HHqPL75BmRHfIZjUaN1zLCwFkYtUIV0G70ximjHaAdYXJI6AqcZQZa15EImT75BGRskPt4FZqfZd1N5GyUaNBhx2ruCllYf7yX0WOBjJE9bcdOSW25HmD0HCftstOBvX37tr00E6HgAOxkckD70sDBYNCWQ+DIII2NwniRCI4TI7u8vKzvv/++vv/++/r888/r/Pz8CdfC3jA2xSbiAfUl8eqoz6vdzY9UbZdsOEWhTit6Ctyy4XkUO1kG3VE22+j/zS35+nRKVU/3U9oh5UGR7I90SuJ++1kodJ9TQW6sFzTiNVJzP5Gn67fMbDzeu+rjts0TYSB2/L7XAfnh4aHm83nnFWAZ+Ql6XiaQuwS85tEkOWPA58vlsnPCC/20EzFHOxqNnrw4xIH34OCgczSP092qp8S50TaO1CjTOmH0aF6XOgkIzDDSL+sFDqmqOkui0B/sy3IfDodPkH6Wncsovvjii/ZWYhrvbUUIGUUhVenjZjBknCBeFqTGoLhevPLR0VE7P/wf/uEfmrCJ+Lky2UrjOs250R87HsNwK6AHy2edOZrZQThFIcImn8b3Jj4xOga/Ly0jwvEWJae3yUHu4q9oZx70SJpAnRxdZCdBv7zQsmpLvLJEhjYhUy9MzCUgOCI7MaNk82tGusjFxg7Sc0RPAhw5g0zhSa27tMvjiY71cVLmdCjsWjFKq9oebYQd0BbGKlNfjzGySZrEdIcdhAv3ONvxJERe35fKsWh2NBq11BFHhFzRzRx77qc4pfWYowt9S4JcdiKwH374oc7Ozprz4Y25d3d3rVL4MYSCUZBajMfjOjk5aRGOqE6kx3nZ+dCJzWZTFxcX9Ze//KWqqr766qu6vr7uzGS4c34BAYNotGSEYmhdtVUQT61bYV2fSUkE30cor1bdY39MvPo6Dy7XohR2Zk6ncCDJFWb66rbRV1IWR2AiNuNzdHT0JG2xItvweKaNlOd7M3dV95ROE/vmmZI3tCH6e/TM7zBknB3t7dDs2JAzbWTm2lxRn/E4taVPdl5GdsgCR+HFpmQguXyFlAt07R/qRTaWn5e0pCNCVk6THaC9fMnZkx0pNJDPlaOYHgKUmEJBXs4yCEDJjz1Hu/SVnQ7s9PS0k4sSdaw0OStRtXVGKOv19XWHXF6tHpderFarzlncHgDe7v3dd9/Vl19+2V6HjuF4epmOInD+pi3m0WyopLUogNPiJFq5p2prwIkWKJ7kSELzuX1sEMYYvZcFMNj035yhi2XOwENQJ0djZIFMcd4mtbkmUyrrAxHW9TkNs5Oo2u4YgDfzBE06Kwp7II2+PDvotuJIHd2dGtpYuI8N0ckzeoxwxEarXEc7nIHgDLIOj01ftgKFYk6UoOXASXCjTYyhx8ht4tnJM2dxnciUoAfCg99mrI2uB4NBZ98xSDOPeYKWMILGt5B97SLwqz7gwFhrZXTEw/0/3p/lFkRA5+nmNgaDQdsrxT5FnseRKw8PjxuxOY/I6Slw18riqMFEgslRRx2ucxRxW83xOe1iUC0HowAKjsLKYZLb13kFMs/KlCfb7QjvYscFwsJBIz8HCSt28jg8C6Scqa/RoKOkZ4AZV7gMBwCeY8V1MZnPPZDGGAmnLCAHxtMHLDqNBoEhawKJ+Tye7dQU2eb6JTvxlGVVdYK80asPl7RDJIUlWHuRd2YrfdkDSzhoD0HOAddjn46Yuh1QkgMjMDjg8mzLCwTpJVToP7oN5+jP0rE5U+grOx3Y3t5eez8is42Q4IbE8/m8zQayCM/5PMSp0RIC5Tu8+uXlZf37v/97vXnzpv7whz80gbF3kS0G1E8kQDmMAlJp+MwLbqu6U/gZnRzl7TDg8py20jcrCcbgpQXUQ9/Me1R1lyEkLwSCHY/HT84jt5OlTbQn0Q919ykIjgmk7M9zgsFoF76RtpMa0WZk4XQJI3AbnGpzL/1Dn3iWxyQdkbkW2ml058g/Ho87KSTXeMkHi7i9tCLXQDr9dEaSgcv8qQ0VOxmPx212FPLeM43OGPjMn/MMnD5tsM7jkA0uaAPjgWPGESGnvmzFz0BeRsKegMCWcY7myrwY+UNLcnY6MBwEA4VnNCpBqY0U3BFzOhRv9ubzk5OTms/n9W//9m/17bfftgPYIFg9WEQvIpxPUGAgM6JzMJsFn2mg0aKjGsWpnJ1dokv+znOpHNkSOboYgSFfQ3Tuz7dLV3WNluvM/3kpRZKzfp5LpkCWj8lb2m6eCUXEEDBQc1lcw/1eAc8GfHNA6CD9oh700+mxv0vU6KN5uMZcH7K1k7AxcpiBKRP304ia+91uZJDOLNF30hAgfqf6DlBG/w4Ath87B/TLz0+HS5sZF+9ltC3Qz+QEh8Ptm8mxDcvGky+ACfPhz5WdDgzegBkHKjVxzcOcIgCrcyuG4Sh5MgPzl7/8pfb39+sf//Efm+cm4hnu+zVmKFzyOYbXjuZJlCaHxK547nuOVLaBe9CsTI5ITdhyQjmLg2PFsNxOp55WvuT/LHuUyMjSwcgBhrbwk/2gYMAZEY2AmT22TJk1pQ8oM5HeqCVJY3M/ONdMneys7HwoOY7oLZwXesazGAeW0ngMc0IlJwYYH56LPNDLqu1xROiSU+VEb+i901Y4KvOqfmdFVXcvow88dH9MBYA+c8IJWzLy9+fppJGhwQG8Lm3LpTVOO42+PsR/VX3AgSF8vLONwp7VcDQRCQNZVZ39iVzPG4JOTk6acmIk1GukAOzE2SQ5aT4JJ2WHg1Ml9bOBMvNWVW2dDoNjwjXJ5UynnosYhsN20o6yJs0pKJmVmh/uTz4MwpX22vBJu0ER9M0kMdfZ8M1V5LOQk9uHk/TyBHQJBEmEzZlBFxuJg4KXDFQ9nUlOJGaSn+eAdr1Jm3ZBn2Co6FUaljMFOzT6RdpFXVVbgh95oEMO2PTJDs5O27pkWWGnjDvpJ5/5TLk8np0glsfb4Ig8juboqJ+6klNErsjR40IdyNlLiey8+8pOB3Z1ddU5IsdpkQ0OPsKCQsj8dkNAIFdXV/W3v/2t/vCHP9Tl5WWLyAyiIz6K4KgNP2ClsgP1rAk/8DJc53SYKGxFMGzGcTsNw7k493fqnOlYOiYjVyMLBtLK/fDw8GShqnksE7NWCPrqae5Eqm6rx5fr7JSNMJAVnzvg5Vq2qm4QI0D5jUep3K7fER+ZWwf53IHHxYQ8442OobfmibyI0pwO3znAsozABT0BgZr3hdN1ACJlNvdmboxJg+Fw2GYqk1flHj/fdog+mH6wU/Vz3Q9P7KBPnkyj0DefpsFY5RY/p/WbzaZDF9mm+oJmG9Nnv6mqr7/+umazWVuMB/SzAQA/q7abXemISWki8Xq9rrOzs3r9+nWNx+P65ptvWlp4e3vbGo5CGHaidAwq0cTkpmeDzNFRTAKbMzO359LHByV34u/sGJ1WPlefDTejD87bUcpGWNVdNEt0yxXYyAZDyvaY+OU5VliTuO4Dz3NUdpDCEJ2Wp4wTPZizYezMPWXQQReo2wdlut7/r70725Eruc42HDWITbLVkjXBMmDAPvXd+OoNwzpow4AB2QIkWU0Wh67hPyDezCcXI7ParZPfAAMoVFXm3jGsWMO3vogde7YlSri/Pz2R14DluOrX999/v+7u7g4Ui9zpnMP0zG0gId+cvHKuLnmwfud4Hh8/vcBZPZmpc/VN/mz3uJ4yc1U0ZD71fepDwEI57xYOtI+Ayg7VSn1Me5nlWQQWYunkhlDN3MPiM4Mq6VdffXVwPt9///365ptv1r/8y7+s+/v7w7OMefJy+aJVEDglMq8XfU1jlixMcAl0ClZFb2NmdQmxbdu2/NxJlreyaOzdE8FuRJ0LEfJVGrlbFeqvyEnFCsnKc9XH3f8ig9r22jnmeItdFG/syVzi32dp5cSaA+s36Kx1yuPIH9qvuTdKHVprHV4nlgPM4UxSPV2OF/Z9i/W7TENSf63jCS7JpR/5HgNvTjteSJSnvuhwHaOZRAg3tCv48GmK6igIyrPplJorx5oTza7u7u4O2Y+cbnNdPyZA0dntqJFZLjqwn//85+vt27cnHECCNA1wYMLVJq2l9Zubm/W73/1u/cM//MPhFenCRfkRByZ0Vwlm1DEtMgJ1n2eEN44Ub3r6HQqoPg1zpoqT/7Ou+ti4ur6+abgZaYsd9XGOWfJ7rU883uTLehTIlKW+yz8YMCZaEmmLGrq2uZvp9ST9M4ScYU7D7/2tTEXkRnZlNUtpl2nmHGsOJLokh9Bc60z9PCSSM2psyaZ07+np6XCsc30wvW6Ocp7pmwgtOUUj+ELimZK7QKOTzI60i9BVAEPH0RjTPbMZ+Tcd3/QVM/C/fv365F51I6B0c3Nz8jzlJTL/WQTWSzTn84RN2FxxmJP09PTpDUH/8z//sz5+/Lj+8R//8fCizbU+P7E0RyeR2edyLh4KJwJqMlxpskjyF53qu+eQNyYfzpUYtz4nPCWakSNllYtqLI1NRTONKArvkE0GIOmp4q91+g7DiSJDLkbltU73QNXHqWw6yhxj94j8TG+tvx/3c3lqgYbpeA1QjqXfjqUnPkTWfZ/uqEtrHfcr+TiMTjQnVn19Vt8zurKUjx8/Hl4Wm741nzpsF6JyRF6jDpa5dK+66LWeeefiT/NWxjNPljD9tQgsmpuZestRyx2KvlskmEhu+pLJoc7y7HlgEfR1PqMWuSTwnEJe/+XLl+ubb75Z//3f/72++eabQ/rTpERgq/QZck4uYYsGREHzrC5T3Lk62HXC64mo5t9B4Ev5uFGpCZrpn5GqezLGFL5Jlxeb6VZ1KOucR+lOK2qSsslVzs9Fkvpfu5NfcYWt1EkuKn2xHlFsiDdZmJIqP2Xn/fJSom/nb/J+NzfHU1VFXTrz9EX0bpruPOrs1JPQqVsFDFAvXrw40BNlI81f+xNrU6JfPUw+2ZZHqte3UsTmVzQ20/H0uY256kJzPMeobc9gONH6dDrt26weeS/tNB1ID/4qDsyJzuCurq4Ox+tMb1wnnp4+neL65z//+bAxNWVJsKaFDSiPG5wXeczzjuqTKV4EqQI1r55CnemYBtMkKWih9+4+jU9eYHJHOayp7F0vKqv+nELEsQhJrmKuhIWS23flezmn4U9nJ//T36781YeMx7F5XQaSceog1zruU6quGdQmP9M1/dYx+1Ppfon2UrwcnHqjrjZv0hqV9K75Clm9ePHiME/pT+mpAaonW0JmEtciu+Q7tyGJXpJ9+jOfF/ZQ0mTr4oh6oM4JBPpMuVfPzumajZmhhLDqk1TEHM9fxYF1DMgk1HQc04t++PBh/c3f/M16//79+u6779Y//dM/Hd6GncDn2VIO1OsUbnyC3ltEpVHqiIT7Mx+fKY6rNSmtKzCT9PdzI1CfPT4+Ho6j8fPpqLo/BdUxZGizzRz9VCZTBx9z8YzzomyIKrmI7qaim/q2ipaDSiYerWJa7SbWtY7Kbdqbk5tE/ORtciRzIUIn5XzskExoMNogx9E1/e0pFx3qea7sHK/kuwFjopCrq6v19u3bgwx3KLQxNmYd5DT05kpEmzzmkdumn2udpoaPj48HCsXUvvkr41Bmyr7PClrNnTTOzF4mx1j958rF88C+++679Ytf/OKE+5qEakaYsfzsZz9b33777frlL3+5fvvb3x6gsxOYUnX8tNHNlDQhTeLZenQ43e//tbsj10sNZ0pZmYJrvPbDFR+LDsUyuZO5jUD+0DHqDBpT3+UATD/kakzBu+bx8fGE2Pdzjd5o2O/r6+vDqp3p+dwa0KqdYzQVnRxaBpXcq8+50AnsaAHnX13x/2SnTlbXbv7nql7fzbmVn7Lk3OxjCMlgNus6l+EkI1HsLAYor9llEF7n766Xb/T+ApkAJLnOYK/tNL5JNSTTgmP686M5sHLkYJ9vS9kR2j//+c/XX/7yl/Wzn/3sMGiXV40KCbnBO+iiSmhBMlFo3/Up57mBGs2mUC06SSfEOvrfiKMj7RodqLl8KMJVJpViGuI5518fJ+FrP2Z0TZbyV66G9YiRiMm3Ue3S9wzNgKasdFL1QXnM7+z3ROHO7yTx+z7Es9Y6Iaado6lLs6508ebmeNx0OtZ1bgRtLkTDBiN1Vhn4DsUQiW+vUhaVHFe2IVLf6X/yO0cx1LYBqnZcHKo4jvms7+4N7OlHMu36DoBIPzyBwvsuoa+1nnFgnbwaBC6Nk8O5v78/nN31u9/97sC15Fhawm/ydwLMGc5BlgpNIaQwXTcdRk6jCfVxCQU+Uz+5tOqtTolJf5eKzFMeqq+VrJzW69evD8ee+FSA3I4pdOOvmBK4gtp9BRn7KXpxhbMxyZ108oGKL+Jb6/ToZtN1g1VjCeXV965tT9Pk0CKQZzv1xXlK5r4cwl3j9S1COj3o99PT6QLV5GJ8RVp1TzI8uebkQvU+JhXiqk/qbSvfV1efzrUv1fcVZLOkj+q5QX4XHAIiczGovuSszLRM352jyuQZW/EUfRUAkulc6c9WTNfngsGlctGB/fu///t6/fr1ZydQlj5EUH/33Xfr/v7TG7HXWiePhtze3p4QpN47BTkjsBxTzkBl8pjjrps8QxxSQp1chg+Hi+bkQeSIZr7faQk6Qo/B1RnriHPuplM5gdqKg6h/ySTnM5FZ/QlR2O8MUidUv5Jn/RUtd7+8ylTc6ik91MBML0xP5iNUj4+Ph5XnFNx9VJWbm5uDk24FL9k1zklH5ISSoYGjlVMNa4f4c0JyV2YC6l2/rWOH8poTg/fcQKrMZ0AycE8+S1nf3Nwc0vGZCk5nVZ3qbSv9ZgtzAcUgqdOLWsp5Zhs6NdPiifwb57lykQP77W9/e1CI9oNFat/d3a1f/epX6+7ubn333Xfr17/+9aGzOhR5iniXJktHlhDa32L6MfNwI6D8QSsmppazGBnkKnJcRuHycElYHYBpocbob8ld0YRcTelZk+8zeDmTFF3OprmRB0z55vYSX80mem1+5LJM7bsmQ5gkrI/FJDfh/26u04WKz8DphDLAZDB5qMlv1i/RaP2ZD0TLhTp/OsLa6u/0v2vev39/+KxrTR2rL8PUFhxHaNTsRnSabkx7eXh4OElDLa5aNza5UBFvc61+G5hqWx2eepZz7zN52/ow6QDnLV8hBzzBxK48eyKrjwK9efPmkDL97d/+7fr222/Xn/70p/XrX//6M4JbnmStY47vgXcaSDuLc14Zm8u2Oozqa6+aZLdKInKxP3d3dydGGpxNCUNmObm5ymI6aiTy2opp1Ty3f63j8TIZcYikOluaD7VdX1+fbP6c3J7KMPmqc31MbpLoGo2BSSfhufzTyctpaNQ5pubWc/Obk5k+uTKXzOqLeiR6kI8T+db/+iUhL+coQqifIjLvrw3bypgr3jsfvSttVm45l7lwEqLKqSiLify9rmv6PLu+vr4+BDhTzMnbZuOOYYKJdHwHHpKtCznd0zxKKTRPlxzYRQT24sWL9e7du8N2gLzsy5cv17fffnt4T+Ps7O6zJtpjO/zthFXH5MRER3ln4fskbOXNIqjz/J7kKp/Q/3MM9d/PUvy5QjbH3W9J9toTmquQLsNr/DteTAfVNY13Kpgks87eeWheRMGixOqIVyo9cF7milL9mgsLKug5hDZX0vzecccbzbE4X7Zt0fh389hYzAKmDtcHg62Oea3T48Z1otPBh9p12pM0T87SEPJf1dN8O4dytjkUUaLt2Nb8vBLyrI6JYtO1HKX3zYWcZJMuXSoXEVgQfK11eJHs/f39+sMf/rB+9atffbb6Yc7f/92fkExjimoKfA5orU+OVI5CRyYXsVNQjWytdXjno6Sh11ZqZxLmftY9RUTL7NPcj7PWafrbNd0jkSkaSd5GaTlDFVflr21TvCJdixHKI7RXvaKWjHn2e85/DrN5N2XXccmXiuBsd3JI6kGlelL8UiaRRLKbn691Su7bho656w0Gpsp9pnw8QNJX+jkvfR4iMigaQHYOZKasyii5iNQKpgZVA6qydC52n810vj70qFBBUN2Ygb8ij+cTD5ec2LMO7N27d+vrr79ev/nNb9a333677u+PLx2YHclz+wIA30BTXtv//RZ5ROTW6Y4PKbJJDCcor005Jw9XHzsIUbi/1ufnIhUxamc66XntRIlOuBNvVIpDqh8phIbSffbXh9Krez7LNoOA3NxEhSFjSf/Sets2cLx///6z/hqcvK65nRRDv3PGorLKDEYimB3yr5/1y/4r84kWGrOkc/KavJhHRtVOiH5mAckiCmBuNWilU2eZbPteY/ZluJPWUAbusRLdy8f95Cc/ObyIZ2YcM5WUa0vXTLFLiQ06Dw8PB9SYXHvkqu8Lwtqb6PW5cjGFvLr69OaXt2/frj/84Q/r7//+70/2EKUc5fo+2K0xNcAeHcj7m8rlaX0HZcJqIpy0STbmgHypan3TIIqSGkP37lJfU9uEvktLKpNQzqnOKGgkzXHLf8mZidJSgpREx66RJWP5qsYgKkx29lteJ/TZhuScZn1SSePq4k6dJ8euQ5lpbN/LjyiDEErXy9vopEWBGreIyejum4Ls40y5J4orGKr/of7+7jtTMHm2mZ7N9GuX1qUHcaei5SkndS8uTd5SPew+A+Pd3d2J49tROv0fmowqsn7rNu3d7XVrl4FHIe3KRQf26tWr9ebNm/X09LR+85vffKYolTkQU5QclYZu2lNd07BdRfHxh12U9r4mW6d4LqLLZ8zvKnP1UAQpZ1epvSanFbq5RcQVtlDoTPkyDBHAfHrANCwFmatXO/nbF9OGKYMdMrR+l9bXOh6J7H05qbiy6p0o0XlLPvGvtuFYditwfi9yEE3rvKQhZsBThiJGg2P/64w06p2Drm77O/ukXGZprueufw9ZNOMxyMyAbtDzwIW11snD51M2k/+T+vDadFtKofbO8cZlK9N+Z7mYQv7xj39cL168WN98881BaFauwekpbbitEw2wjhtF1zoaj0odchH5FHm8d/ci1tlmRe5B/k6UsuNbdKymh/XRFFMn0p6lHU/kcd0T8nuuu+2r5KYr9qV6puI7XvsjwmyzZp+7SKJhWsdcZJDjNCWbc9T9zrn6MLmreU06t3NiyWPnCM6NR75G7jODlD+cPFB2If9rMJ3o3tTVel69enXilLQ5KYh0UK52fnZ9fdxvJ63j/IkIp6zVcZGd+md//J2emsbvFgF6f2T1GZzr4yUO7CIC01n0f56/HdumY3XCiVM4KnECEtGYDigsCcb7+/sTbqb09ebm+NjTWqf7X2rbYtq1i5AuPkhWNhGuElbf/N56KhrRJM37fKLNZKXTMbqaJjq+adgqUYo+UUT1ORc7xGz0nYiooGOavtbplgPrnP2cemDKYdSefM+5vs7PLepADiqZTPTUmE0/SyOlNiY3rAxM3Saf2Zt6OsOs9FM98P9zjru0bDogEf2Um3pqgDHoze+d+5lqe51lBsBseDcWg9i58ux5YObiEvJ1MIKvYuTXk+uAditlIrUZFeQwaq/o9fOf//zAA6x1FOokWkNw5uWSkEXEndD9TOflGCsRnPP00ooTqALIL3WfCEteqD7PVLB6+i5HMgOJAcD27ZfOZZfiTkThXM9V0elo/H9GcfVg1tf183Vc/a0hXFL+iQLmG6oMIPKjjvnp6enA0ezojIl+5ZCaa51in/l0ydPT0+HpCfnAHf87Uy1lIzpqdbD57GeWyb3V39n2HOO0feufWc319fUhQxHlrXX6NM+lcvGKN2/eHCqOi5pCeXh4OEE+5/iJBuaxHJMQLwqa5rnqlAA8D/zt27frzZs3n62uuQF1viLKFZMUa/ZlGpufG92m8uYYHOdM7SrTYWXA3ZNy10e3mUyElbOYiGgS8msdV2tVmAnbpxxmGjdlMtPC/p4cYu3bbrLaLS7MhQ9RrUgi2dj2JI3rvyuBGnPy2unkw8PDyWpg8pv62ne+jFe6ZZLS2lHXllWIxutDnxuY5FwzcEYAACAASURBVN9mKqecRE7Jsp/dHJn+Tqc836k6U3zb6jMdoMFCWUY/tPtg1j3LRQcWAVr0nw5pPqipUUjYTkMVfT09HV+saj0KQS+egXa9E6NiT2/ug7yS9kYfU90fIjyRm5855tvb2xNDdoWnPtT3xr1rezqV2W5KGHLLeEVqjU2+pnq8xofRNRRRoWnoHNfO6Yu6NBgXQvqsbTSTt9FAk3F17NoWESivuVAklbFDU2YY9WM6ua61jUlu79LVFnl0dA8PDyeOzleydc3koSe/tnsNWqhnrXUyppubm8+eaGneI/99h0F9mrZRn0yn1/qco56627UF8JkVXCLxL3JgdtxIWMl763zkPSQ9J0zNsK6urj5bSanu+pCwhK9zRcQiJ3Rzc3wId63TEylmGuQpCDNqV3QktT+JUfuvkfj9DAgZRRPY7z7LEU4e0bpqI9nqNEUYjXv2v3t08Mmzvu94t5nmTQcx08sMqXsa20z7JgLr/x1a2M1Fc9Q8ipSUzUzBlW888DkOTYQb2jdd9BpXKSdpfi6dN8U27TSwzHvMJiaHVzH1rb4pw8YjF65d1ofuDTnO1UPHbXCeKe9c1JkLI7vy7Imsu1RJ2OzSaw32HKEK6llidXa+AktEJVnqG7gnT7GDpcHwPqu/cQoq3IT/osbGsFPy6aj8zt/yV441WYTUWnVMKUSDO+MxmPhqtZyTfFf9FOlI7BoQRLvOvc68/nb0i/Mq+s1JzJRgpiwqtfvmKpMTTS7KqD6LLEojm38Rjn20jr4zpek6DUoUJpc6jU3HIMr2x9XIrnORJNlMDq7/60tzPIN9/ZhZiXa8O99s9kU5FRBEWz0X6ziT/3wms36nNzt+9LkVyLV+AAfmSoRGmMCurz+9JbgIY94u1DW/rYNd028fP1hrnazQdM0kWCd09noFXpRxQoX5EzHNdHKma7PNynTKKfXkDCryN3Jec5z1q7YnWZ3SiYJFPI1ZFCe570KHjs40VKfuK8Dsl0o9U8yKS+cWUYoObi4C1J7ZQboYWlrrk5G6wCNHqHOafKyIojmUi7TU377XuHUoax15pSnn5qBxybs5d9Y1F5bq5/zOlXw/L/jsitdN+8hmZmaVE/cag2M/OvDqvcSbTq53losOzNXDSsJwhbJJE+3c3Hz+VLqdK0VyY6aEevdO3qN2XI2rD0UZ0wUV077OVbXGKWq0GKn7fvbNSGK0d8Kmght5Jgmt/KcTSBlEU1MhVQLH2qM0cV0zlZ3O+uHheHZZ16x1POBvtpnjnGggGc000OA4ebsZnXXmIs/6qhz6W+er/KrzHMJrPB7IudPFZLHWMTWb6XfB13mY6Ezd1KHJO4qSJ/qvXed2Iqf+76Fqg6bj0SkaINXTMhqvt83kZ9Euq9uMZq3TRSFteVcucmCWmVpJWBox+tzvD40BIeucT9gb9TxVc3JV1TWFIoSf11dSgmns83qh+twaYD2T0/K3crGouD4KZARtUg0Moicjv46wIp+1cxbniGtTDY3EHe3Oo47VxRpTL+tXNqby1XcOFey+271Sz5L86oPzOHkwU0YdkO0atGZKJrm+49HUJz+b+xWrt7E1/zMo74rB5zn9lsM6l82ItKZdzZT23Lx5Vlz3xZWZxc3Alo1MOc/y7Jn4VarXtTMJVodVhJ9potEjpZpQdMJvJ9doUJs6lF1qZ0paCb1NvmqOy+i/W45f66jUrqBZX9zfXGlLphnNXAAxgvvmahGNCLU65mJBgaB+75xGRWQzOZScZvP7+Ph4eGyoMcz0tr9V7mQlzWD7u8WNyR9W0qe5c93o3blXtavsdg5tOq+JSEWzIiW/n8R5uiw9EHJJf1zlm+lsv135m4jKa+WpdD6WEJ9vl7K+9puJ/rS/q6urwzOPU07a5U6WX3/99Wf98VnOgshfzYF55vg0gP5uILe3t4fnsCZXMPPnh4eHz96Ic2mlQbLUU18znt0kmbvrxa+urg57dGZ+P8eV8fl/dUwkZpozHent7e2JE1rrlOCWp5rk6Awg0+mIcDp80pJT26VRc9Gke3vVmOmYqLVNhhmdBL2lduWbZmowKQZl6fX+bZquY21MIsXpPHUI8WXNgRytbe6cVO3vXmRRMcDMtNzAUF+qc5cyFbB3trRr1342dn93bzLZLZIJLtJJbW0n38nD9b87DibAmVyfOr+bE8uzHFgpnhFWZPX09HRyvM6Mvg0qhRVh5BhNi3IWCWymqvPFHJUm3snXyVSExlNRXCRonLtIMvkqZeL/yU8UYrqSjGr7/v7+cFihfU+OOoFQpIhr5zhFdvJq3WMQmGWiyfog0poLHDogHYn1R+I3BymqAWonSxV+p9hzJcsDDuOy5KGURU4sfdxRFPOz+jupArlh73EXfW3OolPfoe0QW/OW3k/K5txiwxyTczDttDLrNqsxCH748OHAvZlqy1v20h9tsrGIuLKBSUHMctGBeYzKWqcpRt7aCOVEp2DB1BxT54jf3h6PbXZin56eTo77EKIGievXWkfvbU6dkia8eR66SqGBmpOvdSRwcwBOXs7DlND0ts98w0+fpXiumPW5m3oncpvoReRaPaGw+BW5Sd/h14pvz9/lQKq/uubblppvV08tRvEchAsOax1TeJ1uXN1Ml5Op6KxAuVudzFnJAZriPz4+HvTtxYsXJ6R79ySTEKaLESJyA6oLAZ46Og9JtKhX1e3Bhs55f8/XvOk4227U/CrD5kDbcM6qP0ekg2qsAomCwdTJ9Ky2RPHX19eH16lNrlh5N0+XHFflogN79+7dyQpFA9Kj6niMHBp/Ci/Br0CrN+EI0TMy8/uMTaWSd1BZVDijTgKdUVOeqz6LKoXdog3htimJkz8dmYGh/6+vrw/jDbm5ClRbjUEH+fT0tN6+fXtwHFPRRVAdsKdcdgsmzrcvtLi/P758ZCqxjsZl/F0aXKCT80hf5JDSAxcNGs/OOa21DoFSfUnOcWfJtu0WIr24wAzVN2q5Sij34747aY5+ZlpXIM6ZGSjV+fpvgJgLA83XjsIRtRfQknWyrS0XR+qf16aLHiaqLvm+hsq5R4PU+5ni78Y3y0UHNldPJJvlgybZPvP0Otp9KVQeu6g7oert7elpGA0q8njnoZtgo0qTpTHXJ9OOuTJnhJCEb+xO0kyr+rv25n0+//f4+HgS7afjlBeYRQOO31P2yclUtO+9X3Q00+GUU4OyDhF4Tiz0Y4pSEaHO1Fd0N+U7U9OdTGor5z/R0VpHI89JFqD7/5whnVtMqIhW4lnr+y51zGmdQ2eWaWPWVT1+P9Neg67/ZxcCiElF2PbkwGamJGem3tjutL9kL02SDKYznuVZDizIasVGqZub49HDKuqE9b3jz1zX9HMX9bt3F4kSRgNd64jYipIJy35NUlaeZubrfe+BilMOIqMdLzP3SmlMItvdxkLJ6oopqk5B5yhx3v86F2Vkn6dj8nVfa62T44x3fOFcMNEp6oBEG/XHdqajnI5DVDWL8u6aiTanQ1FupUfdL+ro944Drb+iIx2fXOeOdqlM3Z6r6z2xYVB2TvrteyBevnx5chCDMuj3uYfMu37qhsi4fmmPU5fV2eahLRY5su6Zdn0JgT17pHSPjNS43zlh8ghrfX4ShTxEn8VjOGnxNHpxvfmMjk6G0UIi1zpndBFh+l3tTCdlMU9XSZNVqEcHdu49fjocCcz6EqoREagks31XR7tHVBna9Q1Hzp/Bpvrqk8Y8OTxlKPkuag8dTaTXnIpm/W2gsqgTU8cm/THToK4/54BNp+b3s32pA+fEeawu9X7yXP6ee93SWU/qnSml3yevmSHNVFUHo3ynne8cd59PPrkxS2Hk3LTp7FYqxfZ/dArZ4WqleCKNySs1iNlgkdqXTuQUQm5CUl9yYc5e+zMNnATkhJtXV1fr7u7upD8Zlf30vuvr6xMy2RRSEl9eSSRnKRWaxlg7E9GpWCISU+/d/XOlTq6wdpWZc2FK56qWzsf+tLKavHN8Mw3LOLpmpqnTkU+nYsrn2Ezn5fuUi/3MgF340XHnvHaGIpmunNJzU7fQW2PvLeOV6lA2OZN0Shplku21sdb5Y3m8rjrnG7OSqci78czFg9BcRXufDtnFBPXQQOmug0nFqK/ya5fKsynk4+PjCdFYx4vENl6UTtlzPu/fvz+8fWXHbQiXPdTQlYzKjCYJaEJxFac3r0z+zlVAI7MIIYF7RHUIbyKBxmN9/b3jYWon+c1Jm0ZVPaKePpP49ycjmlyDXIXznUHWP+c+PjJCdvIlte/KnpyJDnnnvJPtfMOS47m+vj5ZVfa7OffN76tXr9Za60BjOOc6zFaAp2GLwnWWrYZLT3hYQK/ws6SD6kmf9WYo0zTf+h0V4/+iy1mm/ShTF8YEAZXJ06nrBTjrjgKR3+3+HH5ZRHXNI3oKMs1Zcp+2bbnowH76058enE6DnpOYADSqfuYKY3yaQumAtjrphDm5lSbZaOqqh5xXv4XMORadmMJ2goTkIq1+JOhDCabFOYlWw5TFXLAoQmUwTbppnNyMfQ7hzlUbOcB4joy6Oma6NtGkiwI68Zn6G5j6vEjsXHW/jrv7XWRwzirWa6o2FzxmeisNUTvNSXMrApkI0AUotzlMhJB+W9yG1BwazJKjdEmfzflzLP3d/3K+zZ3XKn+Lz0zu6qmu6tFGZirab1eR5TtFd09Px0NHAwn5g1Z/s4cfnUL2OqUic84mCC5CmMStqU3l/v7+s3OF2ruSo5qcxfT+XWskcqXFF9aqZPPN3Gt9Ijc9+C2FauIV/nTMk78TIYi+hOWW+lrffTpAJzJXiJr4lDpnkNJMB5nMG4cP4Zsat7teo82J184MVLtFj7XWCeJpjvztvLVIUj0ilh2qMFg05mRn4LD+r7/+er148eLwjkrl01MHtplD0bgbY/u+JPRFfOri5KJqt82cbkNpDiPDfZnIDKQ6JgO0gcGgsSP662NjaUNuNu7+tXjkm5ubg32VKlf3T37yk5OzwLQzUbxO2cMEQuqNpf7sFsYsz6aQOigdQgPXKdQpjVruyNUx0xWJd51TcNSIU9sNVGchYpirJimm0U3klnJMtJBiTXI4xTGKCNVnyphz63eIqf77Snr5pxlJQ4+mKi0MPD4+niDY+hPXOJ2rvFmKbtqcfHUa1W2Ri1rrlANJLrVR0djcuHzunYUZtfWYhoeukmmGdn19vf785z+vP/3pT+unP/3pSWA1ta8N21bXk/3kXE1xdGa7hZrms5M9Zpai/kp5zHZFayIzDX0XCOTU3Gya43Lc2lyo0cWgAsBcGIsCmKug1dvLgPQNkzOvzdq/VC46MJ1JnZvEpw3XYHDbo3CNivIia31+8oSku9eafho9hNF3d3cn3IxePwWznkl+a5xyfY7bFHUS5/W5x2XkSzL0jKxxZPAZb/U6yaXv1TF3LWfcRjZfgOv/yeX+/v7kOdeM6/r6+LKFjhqeK4i7qOi8J6u1jsZU/3IwyUwuzoAj8tThVZdviM6ZXV1dHdLDAtXV1acd7MnZdNU6TUO7vzrNOJqr+tZTDSE869UhXlp99cUiOtbZx/n/TCmTademX9bdd/6vA5w8pI5chF+/d5lPtubnfdd87NJO/Uc/cyHDctGB9ahFTsKKyvdT/Enoeo+rN3Vo8iF69IQmknHiMsra8H4VX35IQTrpprwapXxUk7lDEqIqI+oknq17OmBX+1Is29A5aawiqRCIaWApikrT36LR6u45xMfHx8MjSb1NvaJDr62+Twkz1lJkz4zSQTcXolcXcSJ0uz70OVNqjS80m3zUAee7eRCdNMciuuYup9hY0w/5zRYJSqfUY3V758Amqq3fLho1nkm+73grsxoX3xqrv/vbwO5LPbThybFpc7e3tyeOaTpCaRrtrgWM7CC7kBY4Vy46sL/85S+frZisdfS6VqxzUbgpglCzwc63vEx+a6Kv6p9p31dfffXZAW5GClODHexd6/QYj5CTixdF4BRDtCQy6neKnzFXl3ILmahIU+mc9LWOjlHkqQPoO9FDDjF5GlFVUtMO4b/z2LhCr24c1TFJzicXI7xFKkKdagOqejA5HLnCqWONMSQgZyfq1PAzJkt986j0/q/+h4eHk82czfs5NKasRGe1V72S67tS/53XybnpUHb2IwcWmura7DIg06KElJJBMjtpfuqDaae0jMFH9G1wuoTALm5k1RnIa/S234TStaYWRos4G8nwrvFvuSXTPduZijoHvtbpyqKkrM7Y8UkWCl9d9p9vpjGS7wQ8EcB08LuUqHHlTEMxOUxX2JLnJJJFl6LS7ptcibI8t8Ug+Xld/VFmfadcko3yqr6irY5NhKfyRhyn+M5Xvz19IsOd+pPBRTiXVqdzBrvamVxr8+pJLela91Vfbab7c6FkrXXyKJ3yUUdEYvaxOT+nh9MB6EyqMw7MPhusG3MyEN05d+/fvz/U07XxXnd3dwd05uEBzafy0Ef8VST+zc2nLQCtRlp5T+xXhNMqS0S5D9bWMdMBI5npZ+mH5+UnoBQ3YVZev359+LvUUUPQce2E4wpXRacmmqh/pnH9NBZfxCrRbNt97u8mX76usYQSciYquvyjjiJEZhpgwFGBdD7JSj5NZ6rcNP7msWNWkmeGX4o4FTan64KQfbUeHetME5OJNMDj4+PJo0UFY/ckGWSsT0dt/51Dx57sG69Ox5Rvt+FV2eeks7lzxPZOlzsZQ2eyy0AECNmIqC7ZdO38+/7+/vBSD6//8OHDSfAJLDg/Ux4ucj1XftBxOj4P2SRN6J1yTC/eMnUTngIFmU0BJPOtdw6kCJoRJRjTBceQ8e3SR4/20AA6nWAaqJtf1zo98bT0s2tzwBpCCEHiWm5kh2BcRNBxJkPhfvf78lq3XUyUMJGf6CYkaLuOY/IulfpR/aHq7otXM02f/UoHcsTygvJV6Vj9VQbOp5uEp9Opr9VR8JrOun4ml9l/nwnO+UodeAyUiyE5z12qONHSrpg6znQsFD5pIO81yBqkdGLWYT1zDtb6ZFMhXDMpeUFlkCxrt6A80fOu/KATWVMWU5sEq/D0rDkNCWUjzuy0ApL4E5EVNbtWnqCBWmd9Kcefu5r7W+egcKuzSdbJzYlITtMg5O66tnpU7sbaNSpIshZB9YCu5LcBoXs+fvx4QBkT/icfuaAM9+HhdPl8OuZdMT1xjDpoU2uVdyp5RmUwcq7td7zMTMudH+chvdEZ+WBxbeSANP7Qktf1nUhfVNy81Y/GXR3v378/OILkITJ2n5u8lTJXl8uc0ou4OTkwnYOLbSJ5A/+7d+8ObajX876bm5uTM+3sV+2qDwWB7NuVe5H5uXLRga31KR3L+6fIEnJGJxXj9vZ4FE6TI2qrczvYG2dmvf5dpHPVRIfn/wm5a1XylEln2wRNwaV0KXl1yQ1OhU1eM30w4iUHHamnX8xn3mqnSTc1ru7GJioTpYYypsOcPKNF7kaH1v0qnkS5gUnn1RMVoaOQWfeXdhgolcnV1fF44uqegWCmlhpJK7Iugsw03dQ9vYoKkcvRWL3fIDH5XukIOUr5th3XWzBKL+Y8Nc6JwMyKduh/IkznoWDW/fFdcqP21WzMfqy1DhlBbalftXt3d3eY+92KreXZjaxv3749UfqMs87MDgnnFcpa6+Q87LU+OcfdW7mvrq7W69evD2mlfEh5dX1oQqeTM8rUVyerqDTPMfK+lDNhZoB9rpwapxPW567uZQxTNl7vsco5a2WzS9tmGuEqabJpeV/yWMXx+TgdViWHU/31xxM25PAcl6nqzc3Nevv27cmYHh8/bd3wZRc5tEoBsjn7+uuvT9DguWgdreAWjWRWX+V9crrNRzvs5UYzavngbCTUbno5s5XpeLIr7SEHMAl7nfluv9Zapxu5+5mLMSInrysw5/xMKysBG9vS4eXgXKCwveiY9CnbyqFr4z+axO/5vAYgwepWAT16AyiFyJkldIURvzbhZhM1rxWhNDBX6pqAhDkXGeQIdpFLPq7/4+4q1ZkDlBfM+V1fXx8chcpj2jDRie3rtES4RqPJaRg5J/qr7pyaxLnc3URLykk47yqhpK8p7iT856rS7uW2GphGnvMNpagTBibpDYNMfXIxRe5Q/aivZRrNk0g5Z+XKev3NLtJjF5p2qd7kGLMvnZ+6airY/+m1pHvzWH0Gvh3Xppyau2Si81X/svWJeKUkpG2yj8fHxwONdHV1dcJzem99ulQuOrCf/exnB8U3osp1yInsnMMlGJjCxU+l3BqGBhpRGsFXlDZXdyHAFZI+s1+mYPJfFSNyY4hwlVeqf8pgpi8S8k2kaVvyWOv0BR1GcudAp9Mc9FnOsvFlVK0EunBR0Qh1HBOR9lsjnal1shTNJAMfo5m64SqmjildSEecR1Nv53Ot0xViHUaymCnvWsfFjzkHk5tKF/tuOhSdSg7esTYeOcv6Khqv7NLhyVmJzJKfNpl8pvym3Trn2sBMLWdqOudA52Zw9QkKF1qmH/H1fufKsw9zv379+qD4RqMdDE0gooMmRJ4l4X748OFkR7+5vfXZTnA+x5UQhdo5mSD9dDLznXwJVodU/6tnGnvX2l/J/hTbz/09l87X+vxRkNrqPpU6Q5ltiyJSiBxG8zBTUB1k81Dgurm5OTg85WFU1qE1/66yNeY4nMZiqpl+OR/qwOPj8YUfOv/qMJo3FklikZuBYKZp9bM+1f/mSz7IYFERac25nenfLrCb8Uy9cK5DgBp99lMb6cLUFecpG5pyaO7mm5T6bq11wkHOur3H9rPb5GyG0Pgl89XjXbnowF6+fLnevXt34nndhGY+LWLSQFSkJjyhttRaJx8eHg5K2u83b96stY5H6Ta5kp4qZ5OiIskTKOApZNtpvKa3Xdcyb/VVVGyJzOoWeZRCzXRjrdOHodc6TqoGP9FS9+/4MZ2qxqmjdaUsJ5eTmajANLTvpA50SqIvuQ6dt8hTJFL9EvWNs/mfafxMKZW96aHOJI5ThDZlIgrXeYuAXAyYhqeOuqdRB5huS37PosPpHm3KOtR55306s35bR2Pqc1FZ8prHpdcfuTPT1z4LYGQnBkLRV/Pxozmw9+/fr5/85Ccny+k6D1dLEsIkLDOeSbBOdGC6d3Nzc+C82hwX2sl4clJzmX2ig+rq+7liMqObRPNMI6ojI9BhyK9YNJjqr56MYh5kuLteZ6qcMxwPizRomAaaYpdGVTy+pTkRYWjg9bu6q0uFn0UuVAqivtRe10ruplteI09TmZF/l45N45efnM6hOiaCaV6tu9/Zinag3cyNpPKD8ok76iUdeHh4OByWWFtPT0+fnbw6edI+25H3875dJtLczHtyepMuMEiUPdW2Y579zE7bT3apXHRgGUYvdxDm3d8fl7mFnyKwHdKZPM5ax5QvUraJ7z55hwTR6mSObJKPMx00glxdXZ2c02QE65o56dU30WXXCYsjKdf6/Fy0DL77W4Bwp3oOwzSriY7DEgnWr9rXqcq3Tbif0k8C1XnKkc+d0Y3dCL1LOZS5xHAotHm3zmQtEa3DF42a2qaDptw629JAFyZyTqbH9VMkJXrcoVz7Im+UgzK4No+iL1GSSFYbmMauzWWTkwPrmt3znY43mUwHO23QOWmRzvnvJ73UrvIhUlLWL2rt+ea5cXyWiw7MaKpxhcxMSXIgU3h9NyOsiwApRntsJFibnKKLfNf9/f3J7t5zzsdXXN3f3x8ewo2LMtVMAXWa3XcO1q+1Tu4pBalfOw6t72vbQwe9VvK4FL4FBB1VRuqTD7Wf4+oRGpXx48ePJw7EucyImss+qy0J++SnstY3T6Oo3R6QV8bJsWvm0cRzDgyqkvPqRz+R89NwNA45TZ3adFoFqbaPGJCUuwhQ6kNnUXra/GszymM6zXRCRL5bSAh16/h2nFrjjqMyNXbc6qLHaU8eWqemHOub78NoDrN/D0ZMrufKxYe5dwPQoFReicCEMh92NWUwcjowHZ85tg8zGyFqy/RQI1jrE6/11VdfnTzv1hvCjURNkCjJNHlOfH2XOK8UYVNkjW2t02cFG/9uoUNCM4XUmRkBq1cZi6RUytCZe9REvrXlfqAUUr6psTdHIaScXUhLhKfSu4BgKpnD0iDUxRCqVET9DC0a3bu2z6yruUpX5aukDFwQuL6+PslMkpW63HhF+LWto1K+ORuph5nWptsS8c1x9esA5XUrs91zjlKn2BjSt/r69HQ8qUPf0JwZMM0IJjhyH2K2eYn/WusHPMydojRYeYYmzsghVJ0EpcR1hlkaZGopN6FQ7+/vP1ullLAvrdQp1bbpRdHKNlTY7tER7pxX350I9Pr0GTodm8qgQhipWpXNCCciqy+mCypWjtE0dvJ69aHnPUtl5cxEOyl5DknnPOXcb9uZJHjXz4UQnZHUgE7ZdiSrr6+PL/uYjs5iXc1xaU312KeJJqUS+hFN979/1568j3pqEDR4Ordd53cziE9nlvMqHSuT0SlMR++4dWrqtPZU6XrHojNf6zQzkqpIP0xlpwzOlWc3strpPjOvlQexs0aH6om7qcMJIgcparN4JveEwwqjCclZTkg+YXLXWxTWOac1eS1Lxmckl/9a65iSzCjU4y0i2Rx6xt7/IjJl4Oc+kC6aq70WSrreXeDt0hd5zu00zeHcujCdhKg2JyjfY9pmEb1OB3Dud7SAvOAMTqLd+jUPQZzkv8b4+Hg86DEDLAXse4NfdaozM0WbSHIubFSXMhJVW2fXpiv+nb26OTv7DZlPCkiH1+duaZnzWQpYX1ts8B4pE7MK+UYXks6VZ0+jqCPlrSml0bCHaSUQVXqNSnJa8j2Ho3ORELU/tfPy5ctDnxTY/G2krs/97AjZ7vHaSgrqNXOVq+t0XvM8sXlf450krA6n/ppihpR2PIEGv+PgMlyd2OSYvCfFcnGhfuskJcn7viKqNi1MzhLvylfD3a3O9btV84xWi5SRVAAAIABJREFUxJRDLBiKwGp7Oq7qVV9De1PupdDVIxrWuYq+01sDmhzqRNKVmfpJr4giZ0BuLGZKBlYXWqSFqts+5tSnXuasAjkF36en061TrponX4N66PtHI7A3b96ccBaeqW4auSNYp4FpHAksgtlXtwlFNQh5sdoupfz48ePhtff9SBAbeZqcXVpoWmTpTKPalCcyXer7tU73PRVx5GLiT5Kh8qjfGohk6Jx858h+iPx0jCmGnE59dBOwZRLWa53ucE+Zpwzra2WnjKJ1x6Zxhv5EYnIr3aMOumv9/v7+QJpPpy8yNPA0Z/XF7ED9LgCb8ubopuwz/LXWyeJShLh8ovLRiU75z7RXdFk/ldVap5u5nQPL7n/ft5BcG2NjmauT6Ygbbxur96n76eVzPNhFB/aLX/zi0Emf7VOQrrAoDGFlEz3z77xtRlNkqO6cUM+cff/994cVxLU+PwRtrj5l9N1vJMnhWeTOfDvR7e3tyZEmOotpkPM7lWo6jJzqXJHMeCby07BsL9kYlXU4L1++PKADebLuT1bym8lupvhyFpPTapyzeG3lHIqqjsmd+GbnHJl8nG0VHHIYE9HZnroi6ljrSCSbhjWH8YbprimaMiirEF26cFH9PhtqNtE4pjOxnbmoUrvKw7FPHnjOYU5vykkqo+9Mz0VRBqC1jsFEvnutY9rvfDr/c1P3LBcd2Nu3bw9LmyKYhDa9fIPqM7kyO7tbQaoOl9w7WVV47TYBlW3m1L6TcL6fsEmSwOyz2jZqTmEbVTNwlU+jyugnX9GkTQ4jY5M3kqh2HHI5IsEQY8YehzgVp7Y9SdPfk98y2k+jr5QiVNIHV5QnWvQFEgYkUwj3oeVATM+q10ChA6x+nXD9FBUp1zkv9dVUyvRwps6uWNZOemGaGYI0kBj4znGxU3d3mUCgwHRPp2OmUz2Cj+oITblZtvtnXwQPytVFFp2tKL2/Zxp5rjx7nE6IZ8LnJqcOlQJMUt98ujo1+CZ4Rpmiro6xFEvuSgEVIVKuhKDjrO4JoR2HdRqBd8IXPZkS5nByQqYcTYibXVPanI98RGWS4zmFZNhqmrKpLk9ldd50rPKb00k6Ry5QOCYXMNY6HuqnwTcOZdTP5Dgdw45bEyntFlZccHBVt2vmW+Ez5kn6Z9CSzD3TaarjVgKDj0S91Is6MPXY4vgs6obUS2OrPDw8HLIQOUfR0kzT6mtbMPreM9uaLxGdTjJ78r50dqa+Bp61Tncw/GgObK2jcXX0TRXWYdOu0E4w071XM80QGhZhjcCR+iIChZIx7RyOR/dahLm7RxRctdHYdyS07RlFHaOpWtcow84r11m3wU8Ep4PLKI36rR6tdTxNwXvjtnL+a51uvG2e5ziSrylbRquRq4wzbdDJ7pCZMmvOXHEVfYq0RDM50MnDzvoNRH1v0FGmE0X7v1xgOplckpX8p4sKpqG3t8fz8+rLNG51Sf2bKZ5o03HapsWFMf+fXOl8s/isZx4W2f8iM8n7eMVkICXioosZxY/mwOSijOJrne7+rjw9PZ28Xt2XgZrfZnQSpwnp3bt3J0StAtXoZiSxf99///3J0cQ62tLRkI6kfPvIRAwTNk/i8Ry8nwpon/u+saTkKn2G2PEz9VUDjesx8hZVMwaVR7I8Jer/SUC3nF7quYPxGotpskEoGYm4qqv/5VAdS85AZVam6ZTUgbKv/uZ4FnmW5sH+plciAw0zZ9O1bYNxT11ZQ/NRgMp4DXLN7+Q4u1an5ftau85UTznVj+q/ubk5CRYCBx8FSzdEbnP+07ns01XtUFp9a9zutH98fDw5bl47aB4ucWAXd+I3Of3doFLwSkY+lSQFaYAZh6mER/sa/eS5dqmobdvWWkfiMoNyVcQU1LRGpyG3U/vWteOtTFEbm/1T2VLAHJdONKRR1I6MV77Vk+wqpkwRx/PxKCH8VH63AMwVUr9LRn3vfCmDjltJiT3FVW5wysjVUx2retl4dK739/cnJx2oq6a8c9w6kFmms3H+RKbJrLlqA2mGaNDzWmWo05O/dB6mvs50t/7VJx/ZqWhHLuJYpAfm4oc23WfZbXan05po7+7u7qD781ip/i4bmSh6lh+UQqa4EtdF5hTOhhWCRtdg+mzyajlMV2EUQMKTb0t407GZhzsJ87P64l6VHYnf9VPJu+5cnq4xVkwlVczJDfp9TlVFs84MormYRrlbbbJ/54rG69g1yGmk861Q9cn0qHodg1yeZcq8+de5rnXc6lGf5AcNwHNOJlGuDu/Qp6jEa+W9cgw5JhFH+iUfOudnZit+t0P9orMdpzXt0wxiOihlNduuCAji2N69e3eQy66P2bW6r8437nQ9Wf1oDqyla1dg5IXM1VPS6RjODUShNLGetCrHYKqSIxVFyYXZt7WOEUayUchcmSuUOiZfItvPbhHB/yehH4qT9JVL6l45v+4THU4k0vL7Ln3q/why0yWN1986UFGk8pHjUfkkfa0rfZjOZBrvWqfnQF1SXOuv7MYy2+6z7s2JzCBZMR1XnsnUh8S9xsCbUU/UZZky2aEObdFMJqeZ4cvjzvsM5DvdFaT027TZ1U0XNSoz8E9y3yyhuc4mQm35gLkiuSsXHZgIJ35Iw9VJyRfU8TqlIGd+7sQblYTGveBjrU8vE5DDSlgTOc0xODESr0abCbWre24NKPKYtk5OrElJueT5RCKOvf06RrEdWtRw+87TDIT5KmJOM3nIr/QT0qoPOn35KMeZYrahuCIn6dyn/HJgztVMW6ZRJG/l6RycG0sGkWxy/AYZU+nSGOe576tbZ15fnZ84zPo/na78nv2uXEKjycW2TYvTaeU2aZKKXGj9enp6Onn8R6fZPFVElbYVOgtRJY8ZSEVccr1yxbvy7EbWtlFMIwwNCXONdpXp5UUyCn9OmoK+u7s7GJLbH4wmsxgBzME1ksZgSttY4srcf6QDn9B2hzS9Zipb5xxF2LpCKNk6Edda6zOHM+Wqo5skdqtKbSicBpNRejTxWscFgxRNRNO4vvrqq8PCzVrHbS0GvfTFvWqVUM2M6qbZyci9Y3JrXi93Oh2HNISOP3k0Rh3VnINdmee4mybXH1FIOiKFkNEaNLQj0WNFqqPfBujKDLRmLZZ0PbAwU33/ru/q2e3tcfO3uxEKCqLgrpmLGqbc58rVJe/2z//8z09fffXV+o//+I/PuCije5M0SdIJxxVin3uvxmCUa8BGFutyNWRX5qTtihD2hxQj3DkHmiE+lwpZdnzNJRJzyls0MPuzQ8JTnumDfdZ465NOxvbm3Pb9lMNON+TGpmPbfT6/n3VP7nYGyRyFqEX5KQsdVuMxwEz00f2O2f8v6cW5eVSez5WZRu445R9y/3P6vSvTV+QgDXjpz7mxvHr1av3rv/7rYfvW73//+21jFxFYe1SC9MFpXym21jGNqLNt8uu6mUrqpT1QUEeWwH1vo9DSiLNLbediwkQUIhV5Ju+rHiOU6epa62SLxUSbcyLll/p7clfB68bf+Gaa0d9TSSea0ggLOjoYV8RaGZroqvkXfU86QDpBx6YzcJHB6yZfKhqpHn9XREnqlosIu5StuZBgzylblym0lEaowFVJUdpsxzS2Mvc91vdQq46xupLnDv1Nu1CWM9Vba53svdrVY+YgjVAbOzvqeq+9uro6eRdp/XKMk7et7BZQZrnowO7u7k54JVdQNJb+V8El4xTiVOR56qb1eH3tTMexi+o74/KRqIkyrGP394483a3u2eZcpVJeKv8OPU4nJSc0nVSfda0OsZ3mKrV15/xEI7tnz/pOp6uhZ/wz1VH+19fXh1XCGWQmsdwYn1NeH0+azq/P1JnmyNXC9EXawnn0RJbHx0/bftoWoh7pXAxcBYaJKCevWVZT32aqPLnjxjNXFR3rWqdHt/u5KZ2fe8Jxpa0oM6Pob+tPVn7mvriZgmvfO0Q+aZRZLjowVwJEIZ63pJCmYmvUkzyf3te8v7pMDZzUuf9Jx1e/5vaM+tAWkAl/Jb1doNhdt+Me5uqmzvwzocN19L8cmGOeq51GbOdFRciQXCnqmhzEXOyorxpm7c7UWm5iprxyIgUwHZt/TwV1FcoyEWH3miY6F82fiDP9m8hcdFGf+zv+x/lNh9yW0BHrNzfHzdTv378/GZfFhY+ZpqX39UH+bs6LgXTq6URMft79a51/s3f6Egm/c4ITPc4Fs5Bp6LTvJq87M6Zk1PeXUt5nVyElGlNIjwiu4068TqxB6FTMxxtc9ekAvE64LnqZxOHDw8PhuOg+u78/vlrex3dm9Go88nkJV8Nz4jJQI6r37EpBIQdUPUYnUdo5wri+d33nzOfgVaRO5VSGtmEf5pJ7L1GR/yoqJ1u5DfufIWsQ8k0PD8cnHSbqyphFOFMfHh4eDpRFY3P+0hNRmQ6hx85qIz1Z6xOFEo1iMNbZPDwcj1bu713gMZD3v/X4+JTEtaDAN9Jf4qesP1v1c3VHBxgQ0AbWOn2usrmXilAm2d/8Ph3Qec/+ivJ1XDrWWZ5FYHVkGrkkoTuOu0bomnObz0WamhZ5Vbbum29KcTe9z2qlBCKo/s/A1zqekV8bprsiBpeBLxWNsLpS6EvCt0wU1r05aZ32jJRyCD4HGuzX0Zw7niTl0cmEXDIix+jbYuZ8pXyPj48nJ7vWhsioNtdahzZEQNNAHWt1uRA0VwwnJ1PRQAy+GtI07vqafohwJfZF6Mnel9JU0tsQjvd2Rt7kIuvPWp+jp8lnrfWJBjp3QohZwqRE6qfB0LR0d8JL9nt7e3sCICrNr/qsE1bG6e1fhcA8omZGjgav4wp91YGMwfy+Qelk5AmEmCrlXFXa8RWVmZOvdcoxGJmNfLVjpM1B19Y5VDW5sMlJnCvyLxq59+7I4drSiZgS6YT7LoPWmA0mPfojgjD9XuvzR1okmKejzSlrKKIJKYpK8jcNtoRO/NzVcPsWpyf30nWzXlPrnE51yNvO4KTDu7q6Ojj26dx3PE6yk4qRa3XM0wEbYBtzezUtPgfbeHY2MznedMxtPn3eOP2+urWtfht0JkXj32Zz19fXh6d9frQDa7LkCVR4YXJC0bimQIO0Rmjh5PTm1SMX5gtUZ+5uRPXvnJN5vxA5RTJ9EumY9jhOI2YrtFMelRkdU1wdomndLprW3jSGJtwxTHSjIdYfHWNz4yNi1d3Yk2F9r36Vci70XF1dnaSu9qk+VFftTS6wudhxSY5BJGwqKa820aeOuXu9Zp7X7/xPvtOAPOcvJ+T4RNlTH3YrsJPymMFUTkyUaJ9dOLDs6tYhmSJ6nRRKp4/IV6tz7iebXLhtFcB2jnaWiw9zz9xZuKxDmClmHfRFBxr7jjScKdS8NqfTRO7y78rs2/x+ktlOyuTTzvEH08DndpDJT+yQ285J5XgmEVp/i+6S6JPjqcyXi07npVHMvtR/EY99mESsvNNMdeaCikhzbpFw7vq+1cud4akPppDTyft5js9swX56/ySqC467VVIRcHM3sxOdcfqsjUyezf7pRGfQFlxU1FHvq5/q53TQEvc7BKSsSil3+lz7zeOswzHPlVqzinPlIgJLWRO0XlPDSBhNkkZoyjG5JSNG/0+DTFEmyadw1to/puGkCp13xjYdmbxX9ZxbpekarxXJSfArA++ZSGw64JlmOd5QZoaZgxBNzUin49DIRANyGucUbcq7YqpbH3fKmAxDZqJn5dC4vS8Z+zKNxrbj+WbaKXJ0QWiHCtY65c3mAkuy2p0a4vXpgqc22L/unfsW6599Sc7eP+fKAGXKOxcZrCfdya6VTfIqaE07qK3mI90rtXbxqrprr3uzv+fSx7V+wEZWo2mOp+cRPYFSj2xnJmrJIBNiAjeqeT64QlVhdVzVrUDqj/vFJvdj/zwDP0XQGU0y0bP7bWeefySJrJxc8dtxXdNZ6LCc/KK6Cy7KeK2jM6kPcxtH/fAdBGsdH+Z3kaX2+5lpqAFHg1OP2sRoIPS7eZ/OKQczdS6nZUC16KiTm2munJ/RX8SZIXtyScip8/Ez1l1pLPGq1TWDrDqn45r8pVxc36UDO1rF316zQzg52nRL0r46XKHNjs0ezBbSL1Pp6hPIVKSWfjQC+/Of/3xikGsd8/urq+P59CqGKKyO9N3T03Elsk6L5DxeZ66GyOuEFJ6ejk8FVKZj8Axzn4V0MnQuO9K/a43IGXlKkwOc/bcop36/f//+hDtxzNPJNPkZwVyhajw6vX5n1Mm6uaov33///cmr3ZVpjka0nHwcp+m+ZRLabXvIGZnOz7l/eno6KL5UhahAGSSbHKHjmA8mz5Rl8k+VZJvuvnr16jD+fiScnRfLzc3xjCupjeQ4V9tzaL2Axjp1Wt2v09ahVSTnRXpzcUsgoKyvrq4OWySenp4OutupHL5drC0pOTTJ+eo0iNnv+nQuEFiefRby+vp6/ed//udJR3Q6eXINXn7lHBEn11SZ/IppZIP3YdkdvAy2ioIm+moid32Tp/khJGKl6+d9/5v6dtzN7jtX0XZ8xuR33GbgNWutZ697rr/V0W8dg7ycFEN6JHLy95TVOZlOeYma+v8cJaFTnTI4Nwc7DrDPRLvWP+sqEOz4UVMzi5zZlMG5ck5m874fop/ec67eS/ec69Ouz2t9cm4vX75c//Zv/3bYGPxf//Vf//tnISPhQy4zndRL63GbtCZbIrq/Q0gp945fUdlEW0U9lSihuXBQX4u4O1hev6Zjtr+TyO23TlHObEaU+jFTuknsa3jzOyPVLvWyZIi1lWFX96zTYDSjuIi1ceuwRBv9L4LMYfm9r4Drc7+fQTU9qrhaOFNYkXGIwfHPtFfUKzeoM9Xpqgs5RykG06UdOv3w4cPJSunkhAuCIVT5pXTx5ubmM5Q79WkGAGXk5+r77vrukaoRjbfq6L3JW8S1C0rp1qROyu7kR8+Vi6uQbQ1olaEGJp9T2UXUKZSHh+PO5XPIob9T9rkQMI0qotD0QcHZT9PA/rZPE0nN++VNqkNoXn+6v3FK5F9Cfmud3zNk9Pca07HaT17nIt68dq3TE3INNClvbRvJ2x+01uevVAv5TDJ7pjwT9eyciGM3tfE69aI+ekprnytPdWDKZ6ZoyUSnbBrdWKejU87KU+TmNc6fqbo2N23nErKZiG/eMxHYDmHNldLsy/mfgciFpdm/h4eHk6cEtJG1Tt9idA4Rr/UDHiWKD5m7hWelRrY8r52fENjOeriZxphwjLQ5DxHY9fX1evXq1WcpTNdPHkA01vjKueV5GvcsOkFR0DklmG9qafx+phP0usm5eWJp3+fAd8XI7JwUzUU9OoD5hnDl2tw9Pj6evDHbMfeZr45zfCLY2p+Bz7RWWUsc6/ziY6pfuUyi2TpzfrPorKwnpzgNfTqu3XyutU4cZ3unlItjtJ9zsUHnN9uYqGvqW38LLGy/72cQ738DxlwgqGi3jdeFO+c6EFGdjf05SuPZjaxv3rxZj4+PhxXJILAE8FxJk3xz4C7x7tIsBd5vFWWXXniPUSKhiR5n/ZMwffXq1UlfvK5rJtRvTI1xEsdrre2zYVM+poQW04enp0+bg90QuDO8Wb8RMsWpr+fQssGqKOiCxySP+z4HEUqUgJ5ckavKBiTHrswMDo1D5yO62aVKUiA5pebOFVbTo3SkR2p8VEuH0v8GohnMplNpDlqJDCScG4MroPbZuZZHMtDO9uczkum2qWlOpb5MLq6/k1eLLc6DY/GzeSLr7e3t4WUfM8P60Q6sXPb6+tMLbl1ClVfq0DGv7/5Kg8qRuSIo7J7pXo6zfSQZg6toreapKBlWXt0fJ6giD1EdOQz7XKmNflqhSj7TQRs5va7/Ne7p6BqLqVqy0mAyvrlgsdaRrDfS7qKxUc/o3xykiDrcUood6ulRotBM9faz43N0LDvuTp7l+vr6YGS7FHoiQ/++vb09OZm2e5wbA4VpaTLoYfX+99Ep21nr9Awsx6Tjm3roXBo8GovptG9YbxydfOsD52sdkbz11kf1I9Cy1ilFUOYiB+aTHAWIgoAO2EUV03MD5lpHfu0caFnrB+zENxoryAxFwRkZ1zpVNHkbHVkCTcBzI+DHjx8/O62gOp3QhJ7Sud+lyTQ6hQB0AkaTacTm+n7ffc/tXH6uTGQ6nZPOpeIYa7dxFhAeHh4OJyyk8PXda/pu9l2iujloPk3fdinsrq75nXzKWvuVuMmB5SQndzb5xXOrmsp5UhMW9VVUJfo0DU82kycS+Wmk1pUz1UEpJ5H0LoOwHWUx9Wl+P+Xt3+lFOtUOADOQ6J4oAO2ndqc85iqu8p+UxaVV4bWeQWA5ExXcMlfhVIoi3+QKigrdP/P+hDDhrSmIaMKoPtPNwyBR/L73pSCVooD8TJ9NJz7Hf44HOMdNzJUfJ77J1CDP9dU6RS3TCQjLp5HUH52fq7JrnR5103dTsYqWppJ9LqFbHRNxpg9zFdaA5CpncyIfaGpaOxMRpUs6fB10bUeFZEwuJvnbMRUQXGVX7pPUzmALkNIxfW//pC7mnCfDykyP/X7qqd8bTMsestlQbbqh41N2Zks7BDWD1tTv5k20tisXHdjvf//79eLFi8MGwGlUwcXSy5yCXnOinHk6o5MeCjNfdoJ6kDtFLZWaAtqtak0+KyElwBxHfd2lcSpj/W9rh+jNuoXHlRTkkrNTkWtL5dPoZ13yJDsiuHpCAd7b/xrgLCq07YjOpjyMtNfX14dNs31WZE8nDETp0Fw88m03ax0Rky/5EHkXmNwYW/u+QLbr3Gm+MyL1rnkuBSuV9MXFE5UULHQY2sdap0+IOJ+7VHPqz0RdOaSpE7v76ufkkdMLX8qSoxK01GfnVLRrMGizvCXKZi7KzXLRgf3d3/3devv27WFvVUqmV00Z44s0kCJvDjBFTQj93SRqFKYYkoVGyhTrHPSvxFkk4F5lnhHJS01OqT5MYrbrMnYRTtxA/zuuHGVvbLEdHapKNIuKlUw1erdDNLZ2S+s8e3t39zrOc23vSvM4kZFzlUKG3CZ6zfGV7hbBnefpRDo8UKSXwRhARQ/xoqLNtY679A2icwFkBkqdRzSKKVAynKhKPiq5udDUok/9ukRJpAsFm5lymqLqUMx6qkN6IVvO7tW5tdb6+uuvT57SeXx8PFlxVjYGSVF9/WyedMLK7EcjsD/96U/rm2++OUnr8ohN8FrHZ+bmA9BCyj5rb5dIZxLas4gCJEmnZ7+9PW6cc4LkT+pvdZ1L41KoIv+MdO/fvz85akVHNlFExjGj5FrHqJgyzAne7ZlTKZODiFUDSi5zkURit+AzkaQOdZYcsKnEdJprndIJ9sf0IJmEsHskpXmbL4CQT5lL8fXBrSEFqqkblnQi1NR16bs6MNMfZVXx/16M3PwkCwOYDkjj9XnjKVvTfFN/dcEAZ306N9vOkalTyiW9/vDhwwG1yn1NzlEnWD3KLsrGk3C1nUsE/lo/4GHujHTmsxrJXIHzO4VXZPTUzR3H5j1zqXsKQ+OQ5J/GnaAymCY+dCb/1gQ0uRmAxjr70rjP/Z0B6PQ+fPhwwgeaxoYSVIRkMJ2BvFMp7Q497dLF6lZBTZN1nLbZPfVXOSTPDL92Uvh2xxsM58JM85l+yUv6nfXmoGdKGaLJSaUnIc7Zf1GBf/fdHPPMJpKPvGhotzpc/NHx7gw2/ZkpuRmDyNK5cTzagbSE+jMdR+NpN8DLly9PHHryrt3r6+sThz2Div3f8bST1rmEvtZ6xoF5TEkGPdOMGpaf8bpzRO8kH/u7dnQ4CnUikQzAtHWS+Sprf3sSa44qJVCoRrbJgVl2PFVjmumSf5tO6Kib+Jm6zPRERdBBhahmP6dDnPM4of7uuo6mnnOevOU4TCcKYCp37eZUlE9z55zPVUfHUv+n0utIvK8+TELcotE71omI/K1jnES6lEhO3PRJjrf7al9ubFIHGfzUP+dRdKeOp6P1TXRWW6JyAcLcKtVvAUh1zaDeuER/1pOvuITCnl2FTLESnNBeIRS956GEOipJ1oQiATg5IRHOTOWqs/sT0PT2rpwVOeRNJG3X2j/eoeJUv6uufj4dVuOYJKXjkn/ouxysZ6mLUEs7d5FbxyN3mHzmgsKuX93vfiGNpzHpFAoos4jeZ+pn++qW98p1Wk//zzI/k49zbBLpIpbpsOaCRfJ0y0864mEDIZqJnERgydM0fKInN7imn7vAdHNz3FM3OdtdCqcsZv/kq/1OGWtHyUz0W3+qR/AS+kxu0ij1NXByiQN89kDDGQE6CmWHsExvJsJq0kVV7W7++PHjye79JtDOTyLUlM9oP43Cvq21Di+ZKH3p+klm7tJB+akgtX26uTm+Nr3/J1oyfTX65IwnWpAYntG+yGa/krEOsQ2IboA15VReGo5zZX0q9UxVJ+cpyet7IUXDXdvYdryWhtE1fq5c/OycExGJaFwGor6Pw6yfX3311QmP2+el4cl4OgFRsplGfbq5uTlwi6auE8V1/6Qw7GPXhcSds8Zp2z51UGCK8sgxWc+0u+kY3YLlizwMCC0UxCmvdeQLS1svrUCu9YwDi0xtopo0vXswv53yrfBpCAlaxKRR9ln1NokTvq91JO6noc8J3K0arXVMf+ZrrHI8CW8+fjSNdxLdjfnu7u7knhTZNK/PTUWa7PYDTW7gnLI2TxO9mhrJy9hvjal7fbHpDjnOsWUMGfxE3aIR35ko/yIPZv0Gzvrk/+fQQ/2SE6wu05k+LzDrLNY6Hpw5g8ckvCvJS2epzHVMlqenpwMX1vYEgcNMDU1R7bOntVTPdLJ9bz/v749PskiX1J5z05MH6WMZ2tPT02fHqrtfs7ZzUFEnLYZNZNp3f5UDCyHNKC9J57VNbh7WZXMnQYMx320g07C8toP3GuTuNWHuMZlKJsyeBvr4+HjitGcRodUfnxUrgtZf+z6dz3SQlmkUGYER23QyeeRMZmpzdXV1OHxuvmhll3o5Mj7dAAAELUlEQVS2Gjh5kl06MR8VaW9ZJWMzEr98+fJANTQfBUJTinOUwVzdrvS6PMcYejKyu51ERyzarb2pB85/sqr4CjX77YLMRK89W5w+2HfRUO2EVJJ9Y3jx4sXJiSqvX78+XKP8nHP1SkBQX0wTk7sApSwmfZinFE89aXwuhKSfPj5U/wJEP3ofWM84zry8CZJvKorO975JVkrUTiXtf/fsOIntY1prHQjkBr9bKJhRMGGKKN1d7emRbnwVYfW/xl1/U94dST75ku6bqGDep+ObvNW8zjTDKCvKEPl1nwacce+c92wnXkanpfF7re1KC7gaKFepHLuuOucWHhG6CKw2Q1EajuNRttNpmmLtOCQRlePLyZjK72gJt+qo7wZ4EWV9VE593kkq3ZOTr81zNEBjlM/KCXqOfaW5ag5KvXeLac5dY2zc6VvXF2B9FrP6fvQ+sF/+8pfr3bt3h/9dls7j1pEcQx1UyYwaCaEJqiT4BOtu8r6fPNjMu9c6EsbxGkX53fWTPzFlmasrKmjtNFY5ld2YgujTqZWKnCumOipi8un+maYqdw3COtc6oifnqzFomPGeM7WQL5zp3lxUub29PXlL9xz3bqOq17jNQlLduiatYKrks6rTSShT09X0MSc9+SidtXMvgppZy6QcPGzA/ldn1zRWg5rtFtT6maln13ddTi6H1NMJ2U5ofT4JYDpZOzNF9bOZqZXeVvw+FFYqGsi4ZCMXHdgf//jHk0c+8sbuVl7rFL3IXWXYM11ygnWAT0/HJ/1nGjKFsEMsE5kY2et/pbYsjUlkVrqTgQm/p8NobEa+169fn6RZte3vnGcKqjHMdGSOfzp1ZaKTST72003Bfid6Sy71pWunrOXpRH2eYGBgmCmgXMqUR33wO0v6NN/hqMPwyYf+njxjv51P+bo2W9bePEUj5DL71/86PJ2JtMPNzc0JjzqdoT+h3v5Wr/xflOVpEdfXn07yKC0sczHjmkFBRDZ1r2C3Q2ei2dLi+ikYurr6tDfTx8Em/WK56MC++eabA7TL84ZqgueTG+k691tJOq51XC5vMKUtL168OJw+sSOPVSzThUrEX4hAVJXCTlQ20ZbOV6JUfsg+7NCd0crXY03D2qE55ZnRiIymcZhKzdUqyfSZDqWkM7obeObcyhX5MHz6EcoR5U3eRxnteCxT94lqRVCNybbmQoULIlNPmgvnqvrloWZA8hwrubLukwMWXU2Oa1IUjfHDhw/r1atXJ7qxy1hMxXTyyeTjx48ni0aVNhE3jnbUByIK6tlgn08klh75GFfOMbRcQJqLM/VfpCiVYR3PlYsv9fhSvpQv5Uv5/7lc3qf/pXwpX8qX8v9x+eLAvpQv5Uv5P1u+OLAv5Uv5Uv7Pli8O7Ev5Ur6U/7PliwP7Ur6UL+X/bPniwL6UL+VL+T9b/h95WY89BzyAGgAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "![](\"images/derivative_of_gaussian.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Implementation\n",
+ "\n",
+ "In our implementation, we initialize Gaussian filters by applying the 2D Gaussian function on a given size of the grid which is the same as the kernel size. Then the x and y direction image filters are calculated as the convolution of the Gaussian filter and the gradient filter:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "x_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1, -1]]), 'same')\n",
+ "y_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1], [-1]]), 'same')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Then both of the filters are applied to the input image to extract the x and y direction edges. For detailed implementation, please view by:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "psource(gaussian_derivative_edge_detector)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example\n",
+ "\n",
+ "Now let's try again on the stapler image and plot the extracted edges:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOyd224kyXW1d1YVz2w2+zTTI2E0smDA8Av5cf0EvjYsyxYkSJbm0D3dzVMVWaf/gviCXy4GqwXr4oeBDoAgWZUZGbEPa6+9IzJz2G639aV9aV/al/Z/sU3+fw/gS/vSvrQv7X/bvgDYl/alfWn/Z9sXAPvSvrQv7f9s+wJgX9qX9qX9n21fAOxL+9K+tP+zbbbry3/5l3/Znp+f1x/+8IeazWY1mUxqvV7XwcFB3d7e1na7rf39/dput7W3t1er1ao2m00dHR3Ver2uxWJRz549q5ubmxqGofb392uz2dR0Oq3pdFq3t7ftvOl0WlVV6/W6ZrNZrVarms3uh7darerg4KBWq1Wt1+va29ur7Xbbfvb29mqz2dQwDLXdbmsymdR0Oq3lctmOr6rabDbt+lXV5jMMQ/uhz6qqYRhqvV63459aseXa7nM6nY6utdlsuudut9vRuKuqptNprVar9t1qtaphGEbj4Of9+/f18uXLOj4+rtVqVZPJpHutyWTS+qmq+v777+s///M/69tvv629vb1aLBb19u3bevPmTd3d3dVms2nzvb29rYODg9putyN5eQ6eB3PYbDY1m80efY+OU57IezabtTlwLvNCNsMw1GQyeTTfHIc/22w2NZlMRv3xHfPx9axzzudv902fq9WqfT6bzWoYhloulzWbzdqcfG3mg26Q193dXbNh5MGc7u7u2v/YBn3TD2OfTCZ1d3fX/MVyZly2d+yU8U0mkzZXbNry6vnKdrsd2T76tkw3m03t7e3Ver0e2SU4MAxDnZ6e1r/+67/WTz/9VJPJpBaLxYPQbde9D2lXV1c1nU7r5uamlstlU+B8Pq+9vb0mgNlsVvP5vGazWe3v79f19XUbJMaPwiaTSS2Xy1osFlVVtVwua39/v00S0MJhENBqtarDw8P2/f7+flMSfVVVE8Bisajlctmcfrlc1mQyGRnDer0eGeJqtWrAYaUxNpSHwuiHa6JElIShrNfrESBst9sR6HkcNgSOBfA5FmPgfMbtuSAbAyhjRCc2HPqh8TdGxhzsHJaJgYGAgR64Nk4KCBkQkBX9MI9hGJrO+Y28DULIw985qEyn09rb22ug5+/44X/rwr8dQKx7BxQHR65pG9psNo/kbFvAbwBM7BVZMQcasvT/XAdZ83cGfdvG/v5+A1PGzv/0a9khYwc05utAgQ+hO/sz+pvNZiP95Dh2bfXaCWAnJye1Wq3q6OioKYWB7+/v12w2a863t7dXd3d3I0CaTqe1v79fwzC0vtzPwcHByBAQ1mq1qtvb2xHjw7FRhvtKJYHk7t9Rz8p0VLDS7PyOMjgI56AkK5Y+DD42Ovp3v3ZGMwzLq8cW+J7fNojNZjMCR+Z4eHhYd3d3dXBwUFUPwYBm0PC48zODQNUD68KBsIuqGsko+0CvZl6O8Hy2XC5HoGkGZdklKKAjZGHHQi7L5XI0TjuagxRsKuXO/DmGPtGJMwGuaxn6GtYt87fsbDcEfQdmwMB67AVe5IHebHOQE+SCbMyMexmGfY3rWRduDvb8bT+17p9qOwHs/fv3zQEZGIK6urqqYRhaCsh3AA4CWK1WtVgsarFYtNQQ9Ofvu7u7qnpgEkZrGzVMy8rC8Jk8qe16vW4gZzrbJi5QMDByPOfaCFGoGRXysdPwPf1zjo3HinGEXS6Xj9gcqTPOmnOz4mkJzk4xbDhmRT4v2ZnTx6p6NJ5M9ZyW+4exYi/oOFmT2S7NOjQAmhHMZrNmX5naG5js2A5WZnAwFc/XqaLBk76Qq9kY1yM1dFBNlpdyxnbIgAzs1gVpv+0CEE3AS4ZLqmn7xw5hj5a3/Yg5YgfWc0/uyMLM0nI0c/wc+6r6DICdn583FoRRL5fLBlAIdLFYjIwSByF/J+ozQRuBc2sml7UCjJz0ZL1e1+3tbRMQgvbEDw8PmzFm3cQOhmO7TmDnT1aWaUkaAr99PjJIA3Af9Etqzmd2MObo83xspmeOpuiPYx140INTOddfPP6qB1Zix7A8aTZQ5MS1nLaSZmbdC/316mFmo4wHlmWmhQ5sU5xvQMAG6A+bN9CiM7NrzkV+ZltZckjAMoOzn1nnZDro2UyJwD8MQ8t0Us/JXu23LlVkWm8ZYwvOJDITMHPNtNw+A4kxCFsHZo+ex1NtJ4DN5/NWpKcGRQGfKDefz0fp4f7+fkPzw8PD9hlO4RyZiOHUBwETsThmsVi0wj+RMalqGjLnGpBc0zHYIMy7u7umeOgzfSTwudaA4TqKca4VnWmgDZ/zzJjcp5kTzcECowZskC3M1nUJIqwNnjGahSDjTLW5ZoJLb65Z06uq5uD7+/vN+WwXmVagY6cs9G8GjpM5WBpos4Rg4EROaVtmXHl8D6Bt1wYFjjGbMwt2EDXLB1AzLXamYtDlfGcIlFMYF5kFtpCpHw37TaboudhWnM3YNjiP62cKb3u+u7sbEY5dLGwngAEiCKqXAx8dHTUw48JpEEZkGyr/54qUDXFvb+/RggGG6XEBsK7ZcH5GWrPATDOPj49HrM5Ak+yC87gO13RzkTeZg50sz03jMEuw3HpplqMov5mPr+Fg4RUmjnFkzboE7ABjtCxwCOTeK347HUuwTBZr2/DfDlB2fKfo/DaDZC7YYrL+dFJAwONCf54/bHt/f78ODw+bnu2omV4D3g5iMBX0jn5sSwTXXprMGJA1Qdm6t207hU9QYRxeUaThWw7CjBkdGNgcbLh2LhTYNpPR99pnAQwDOTk5qeVy2WpUTj9ub2+bMiaTSSsKY+QwtuVyWbe3t6OIOZvNarFYNAAAANfrdVsMsCHhPDYgxpqrfQYJxmNjT0YI+jvaUbdglcaG7ojr6zBet6wdZDrhdMRzNFD6x5Gc/qw31xVsmMzb47cT7mIIfG9AzbTHDIa+uV7qgbkzP4IJx6cMkk2ZFWZKaHkzfo4zGJAeuTnImLXBcviclXJSchgwtp61smTe/GSth7Fjh84oDHxmfslUPCf6J1iYHeOL9OOSiQMNJMHBGAywnrA3bBq5IWenmNb5MAwjJmiWv6vt3AdGSgfaIzSofxac7XwYNwO8vb0dpTZWlJkJq5gInOPSwblmRgbn1DacpPo2fo/DKSnRr9d/pjemyTaaBLKn6LDTNANbOjpzTibjKGdDpW87r+WRKSTf2eGSQZppmb1gM+7D3yFfM89kXcjILKOX2ni8lpXHjh566Zfn5mv0or5B0TpMZkjtq1fMTvtxMEwWbeaN/rIho96Kpo9P+3RAs+3TTEyyP1/LqaIZNscnk3Ogc7B04E2byPn12k4GxsohBfi9vb1GTZ3D2imcbvAd7Mroa8bkwr+pLQpwfcffcR6ACd11f1UP4GnFpcBt0E55cYB0ZloaoNmLQRS5eI9P0v8EFYMpyrcTGbTS2a34HAfAbBlwHXTAdwCSGabH6jmYOfBdpnecbz07WPTAqsdmLZcMPACEZcGcXMYweFlGBgenqS5Z2OYNAq5JeqUWO6uqxupprrEmM8/A6yzDzNnzSPn6ewcRrsl3Ls94vBlEHQBovfIK55Faey7WuwET2wd7evofzenJb6pGkfLo6KhthyA62kA8qbu7u2YALP86QlOIR+AUnrOm5Z25CAKlYiDug+twHkKB0hsk6SPrPTir04Ze/p/Nykx2xFjp29HYaSDjSDZmMPB4DVR2Ngw7nRq9IV/YWjqTC8DIiWuaFSewwsoMLICJdYhMkgV7/D1wMijZDpJNJWOy7DwfBxaag1nWIJPt5uKHUx8Dnut8wzC0Egx/u17ma9APY/WKtsGKcfc2AufYE3ATHA1sjMnbUlwPY/yWretptivGVvVQV3N5wW0ymdTh4WGXIWbbCWAvX75sdSscGWMkzRuGYcRwNpvNaMf80dHRSAjr9bpOTk5aP6yQmIJTkAd4QGXAkXNdxHZNwAyMMbLCQ57tqIzQXHRMoWGQGSWTSfg2j56SHOU5BwNM9sDxTgHzPDcb9q6Uy/TeUa5Xr3JJAAbpeojrclU10qXH7/SW5hQKmRjg3C9BygsGjNGA66BhmXqfHraYzoEzWgZmbpny9JgbY8JWPAbOTTZ4e3vb5ulgb3tkDhlgLAtnNS7yW94AsxedHCDJEHxdAgv9ekMxWzmQjQMu9XKu5+ysZxNmg57nrrYTwD5+/FgnJyftliAiDxM1InszK7+Xy+Wjovh8Ph8VMCeTSbsNaT6f13K5rL29vbq6uqqqcZqQ6WEWsXHIxWIxoqcsFU8mk9HKSVWNdmCn0TE/rzbZOVCcKbkZlh2Q62DMNtjsy7VFO0QaG/NmPLnPK6Nuj7liQOjLjoZMnOojU3bxcz1YBccxLp/rORkkDMxZEzHz8H40g4FTNTNpA2SuBNop7aiwSLMXdOC9XcPwsF8ui/DILsGf8ZCC2lbQP9ciozBztkywKcvRzNIAnWzZZZFk1AZN+wKlIPaAunzhQJJZkLfvpIwchPkuS0nuv9d2AtizZ8/q6uqqttttY0QocbVatXpY1UPEQZBJZ5nY8fHxqH5mug5bY/US4RtkiDxZXHcqS4SdTqej9BTj3N/fb/Nw/QwFomz681hcS/P8iGjMic+h3t7RbZBgXD7XNbBMkZyyeRc9c8iUvmrs7L4fENBh7mYwjN8rTTg97MvRFsM3++Ma9Js1FQMMc3df6DuvlyCFzLJOxzV69RkvOjFPM3hkn8CTfTE3gN5gacfzjnhk4OszB+ZpEEGOTr2YY+5R7DWzXjIZrosfGGQoMxg4uQbZGOfYD13isU0hb/sqLf2AHQzGiF1tJ4Dd3Ny0AWJIZjleJkbYRPGq8b1hHrjzaf73KkXevoNQnPaZCfJ/KotzAEgMzcDH+PnMqRsA19uMSP/+2wqxY7o5WveiEi1Tjh7r9F4fX9OO1gsUgJ9TIssNoLejePw4eM7TAczXzBTK10+ZZUqXjNX9OqjY5pAxn2eNxd/5xnOPIdmv58VxXom3nWJHZnYO9paXZUq/qTeDv5mlATvtyPPwthZvm+A4fA1Z+treymT59gDXtogN2qd61/VxPs+481Q5pOozAHZ8fDwSooGAWlXVw2M7UDTRgtt9vL8HRmSQQsjU0kwjzQoccfmevr03zcyolzIx5p6DpKPzOzfg2fmsIMbuFR0DQc8Ze0ZU9VCjMrsxC3SKwbE5NgMe/QDsBiEfS2riFJPzM5iY6XCOU9DVajV6kkmmBJ4X87DOneI5YFpmGH2mre4rm0E/WZB1ZTlhB07/bWtcv3edXqM/Mhb68FMZvEjjsfknZZqpY4/FOnBlQM55uObpzw1c3mmQQcgLRWZVBndfP1cs/9cp5Hw+b7vbGbipuj87ODgYDY4o54n0Il5GUJQKE4PyOg1JI3HBMBlR7jJm7CgVA0zB2jAc9WwUHJsRyuDisTDOHghxjNmTUww7stmiQZm5UeezA3Mdr2J5Xh47f2PMrkHxGSvJHAvAGFSdyjMHAAZgyBIAnydLR5fM18BO/+jRunTtKFtPp4yJIrLZpPXglMvf4Yypk7xGBljOQY5O4xw0HZCtP+vJDJMfkwH7GkV7jsn6H3rknkzGwW+uf3t7O5qrdYKNMN9kzLnhlz7/bgA7ODhoE8IpMEoKes6NF4tFu1fSAEKB2BM/PDxskZ5CP8+oonFbjydpCktfBiAziaT1vv8LQTqycA4G5J3Hdsxe2mRQQwGZxhFd/F3WVczcemluT9Gu5xiYq8YbEx2Jmb+B2mwDfXG85VE1LmjP5/MRezQop3G6tojzuPjLsSwg9Awa/ZmJeY7YgR3JTsw4uH724wK4g4htgU28vqYZO2MA0MySXcvlOC8g8Zn1y+8ERsvEK4TJmlxbxC5sh4wfvfoWIsDUwIlteA6QE/pg/PSB/J1iuznwcOcL43qq7QQw5/cMHMF564QLwTAMJuJVDhRhJuInVtrYN5tN20+GEMxGnG44sqQBcwyfOdoZfCxA03j6yRw/f9NnRjiPN5kZ83JqgKGl8yVgJyA8peg0Eo8JWdhJbIjM34ZvtohOuYXMY6T1NjB6HvRjoHDNtFf/yDk50PR+qsY1GYOQ5WKGyJiSjVSNGWemcAa7ZJjM0U9bQMbWhefpwNmTFde2fvIzp9EpY9tMruZyvOtmPj51Zr9MpuqyAr6cpCZLMdbdU+2zDAyGZENar9eNQTF4Iip7uJggT2N1rQTh4iywtun04a55JuUagI2J761QG7wVkTTaoMSYLExfwwzASnGUpQ+ajc4Rvqq6Rp/G6d8JigYvB4MExtQXfTm19PgdQPy4YcveK1YGPJqB23KzUyGD3vEGx95xlpNl42BlOZjt2fESiJ7SKc3f+TOz5dShQSNTTo/TZZgcp8flQIe+eiw0yw62W8smgyv2b7tKfViHll9mDMMwjGTDZ+m/XL/qgeyYiTKeXauRn2Vgm83DY3GM7CBpsrPJZDJ6QCHpJGDlpXHS0uvr6/a3H0joCef1rHSPwcZjZ7OQU5BQfQSGgzu9wYBMx6v6t1kkyDq9yaKwlWsnzLTJRuQ5VY1XlgAaU/RMr/hsl8PmvGDajDFZie3DINtLsWm5k91zy3m50b9ZS9aVnE4/Fcn9vR2Ghh7cB5/b0R0wbCMOQgn01r0L7AYy25CZIWNwn8zFnydrtmw9XsDY4Jf67QVS26jBK7/rgabT9aqHnfq5+t1j7247b+Z2Cgfi8zuL5k41XEcBja+ururZs2cj1OZ7lph5/phT0owyKJx+MbLZ7OH5YbAJanSTyWT0FAwbq4VkxmeFAIim2F4QsGHaKOnbdSEDcjIQA61B2IZkMOfc5XLZnhZSNX5Olft2FHb65BTXCzRERdcMPV6nBwYNG3sv4ia4pmzNajNYGNx9voNoBgb0ju1m0d+AZBn12KU3aOZCDHq3jAzoBhfrw1sGMig4WFi+2JUBxM8T8zFe8IEouH7FOWZbfp+E5+MV6iQK+AWpc6aYrmNbBtY3W1N6jLTXdjKw09PTkSFst9v26BuiBhf043BobEydTCb17Nmzlnoa6GBsTq3YNNuL9jyuJLdQIFBviEMAXPfg4KBFG54xZnruiGmgRcn+DDDnOyvCxxronDKgyKT7VY+jXzIixs54kb036HIs4yFNpi8vJiST9QKHHdkOttlsGtM2qCT79I5yp03IxP04YK3X67bXMBdUiNzWG2NC7r4jBHZCkduOSp/exkBfBiBkgA3lijZ9OX1KxuH+sGHLi7G7pOH+nSojL4/TKSJ/uyadWQf9JWNmfMwz2TPH+j5nxsRGZ9u4wct2aR9PZlj18DjtXe2zT6NAYavV/cs9eL7X8fFxE/T+/n59+vSpTdpgBpAwGK9YQRkxBvp0zQ2hg9qkoUQurmOluRaAgNfrdXuaBQ6Uhd2qx9sYrDAzJy/3c60sxDpqojgzBs8ro00aVNYUemzEwOAXdRik7SyOsE4RDQLIlWMMaN4Z3mMrnJMszHO0cyN77/Zm7gYkxsB1WOLHWWAGzhCcqt3e3tZ8Pm8voWFV3UwV2Tv1ytU1ZGO24NqibYInUDhg5aqe60EJMJY7/RiMnS7yO4Ojx2oWZf15O5TPR1/oAv1jS6ljMzGDEnaGHpgvzfdwGvSfajtTyOPj4/Z6MrY42NG8E9k3XvPbObejS1J5HI5I7OVtKz2jnp3b9JyIu8t5bBj87wjm61iAKD7TZUd/xsDfXqHx5x6XHdLjszHxf9J/j4PmiEd/7t+1PAMF35lpMT6DJ306wic78G/PO1fx0CsOzfV8fs7FvzMAJROkP/dpXTvdJ0D2mvs2o+6xlKxReRwp0x74JKu0bCwf24fHYLDytQDSlBfNd9K4+Zg8xxkGNpoppAmFAyHkxtkWx9ufnmp/061ETof427R/s9mMbu7lczsXEyL1QgDe6+PCfjK0qvF9XF7lwVkxPCvb4zZrqRrXfuxgBhPfuFr1+OkTdkrmkEaT4JHKdgqHQh19bQhZdyOiue+qx3UNg76Pca0xAdfXsEx8fA/UaHyWaZvtgf/v7u5aKQEwy9QG0E2wsAN5tc7OSF85H659e3tbt7e37R2oBG76z7cC5eJAysggh2Oa9fIdP5a5mZlZjnWcgdUsKetqyKIX7L1PkH4MrAn8HoOzHV/L4IVf0K/7cyZFWYdzfI0Mzm47AcxMyPtnEASAksjvG1tRHK9Pg2kBTp44fWZxkOZl7xSkv8/akql8AkOCh43B5/dYUwrWffv7NBLkyd8JFkQwp78GjwQNpzT528aSsvOc/T+LKJaxj3Uhm/PsVHbiHvBnaoAevHnSunXEtu6sdzuPZekxcbyd3amft9g4VXKKmXp2CaFnE7bh1LOL6y5p8J39zT+ue6VdmBlyzQwqtqGnUr9k9bmVwd/nmBiP02n7JHqy/xG4bDOWx1NtJ4AxINiVEXQ6nY523Bt4zHiS7mdx2ALNKOPr8T+CSlCxcgFRb7vopXw9YTqae3wumrulQbh+wfFOfVGGC5iWDwzVDmuwM3MYhqGx1pRzKj2dyOM26FFHyvcJZv/0lX2wQm255aZJsy+DCKzn7u6uPTzz5uamrq+vGwPy+N2n++F/O3Ev8AA+2B02Q6MgfXNzM1pQoJbam3um2rYj7DMZTjJFmu0wdZsA5jQ2QSTLG2kXBuLUjQHRPseY/ex/PyAgQZkGm82sh+9o9tveuNv4n/ymqhmyX0oLm2Flz06N89n4fTyO7HTOtZxkFEzcj/RAmMkk7Ez8dvTyUyucZmStwNHc5+d+oarH+1ro30BqcE5K7z64hmsFmWoBWAbZvIMh6zcJUDZOjNvRGACyXpPhIUszlGQkCc5PFXLT4bEHg4sZmY+1syeTwHkdCBIQEtjMeq0bGFjKNcfhwPIUO/GxfEfdzY9rpw9sIJl96jrnkpmLx1L1+Pl0zMOgMQxDW1DLQr2ZauqMsTMGj8UlKc+J822jri8+1XYC2O3tbXu6qot/+Swts68sHrMA4KgAcvMo3aqHVzSZViZb4AWoZiUGtAQvFwxRkGmthW4jtyJwoFwtZE7JrJL5GIwYrw0ABboG4LnDrjJVMdBmFPU4ONcOx/mkit7EaKBxBHa09EoRgG3AMIO0A2ZKkkwa8ExnZAV7sVjU7e1tXV9f183NTYvmTj8s9wSXrJHaln1eApfrU67H5vYOs8DUr4M2srUM7Dt+/A3n+H2leb4bx7hkYobo8dg26Td9wYBsG0pbcW3LQOssAqzwmDabzQgzfHO5a6FPtZ0A5khp5aKkXMbt1ahgZKA4A+JG8adSGjsejmBnTppsttADiaSijmIowyzLCwN2KubGuwFs0PSbMnRa0ktdzfDSIA1odqKntmykAXuOBiP0wfcGQa6RkdmO3XNMfjjHAQld8dt6ppkh+XOXEKbT6aMnI2Q66vFm/4zZ9+LaQbwY5eK9WUGvlOCA6dSV5mt43nZyZIW9eX5sR7IN2E7TXyw7Wm8RweUTxmN7NKCkjmxT/G/f9bUyi/B4XW/NBanUYbadAIaiqSnRqUHLF2bvUQKHBZSrI5vNZvTYHBSAQCwEO4cnxTh9nhEcYTlFyKLiZDJprKpnDKbrpMqM30JOA7IyDKLIxQrLVVaO8++k8mZhZmhJzc1qDfB+6oTn7OfPI0OzXubo1eiM1DmWBFunqskisw7D3wZkHuGyWCxqPp/XfD4f1dGoD/JQPvZ9ZU2R5k3Zh4eHdXx83MASZmm7ZKxOpf1OAOZpUPULMhycnXnYxvEJUrnMTlIu6RsOzJznOiVj8TkOZMgtMwMIRdqu7d12kKup+BDpc/pMLwD12k4Aq3rY5pCT86uh+O3H1SDIFGCme1yDSTsSuAaAsjP1wPncp4HQgGGDcVTl+1z54/OMILkiY7DIVCnHz/EJaBTPiUhOfdKJvbWklyIYAPjcoNUzrAwqnmeCuVlDPsguU1vPwf9jvHZWM2Cn3J6L5WHG70UWs7W9vb26vr6u29vb9oiWHmPnehyzXq/bRkvm53sNPS6XEQw0jNfjtpzp13LNYEjzYoj1mNmIwZI+vOHYNWAznCQlvXQ/s55svTkyp9zqlAyS67GDIUsAT7WdG1mpUSUi+34lWARRbjp9ePZPsikoO5teeVIFBoMh+J4pKymdgAlCr71QYOWQxqbDoNC8A97Xgd310p2MXHaKXmTzd+6PvhyBzcQyLfUbXtJozN6Yd9V41zvzctoCECGDvJE6jYkn61p/HmvWQtlG4+tnMPn2229rf3+/bm5u6ubmZsTmqh7uzzOIGeicMWCTf/rTn+ri4qLOz89bf7zbwfLAFpbLZT179mwEqsjz8PBwBLBHR0ftntuqB/Db29trt9GZvX78+LGur6/buEnP8KWsVRqkDJLWuedgXdAgG2aRtg3s/ykGh48yHo4hkOYz1czUM3txOSUXm5hXZhz5d7adAPbhw4f69ttvR9Ge90NikAglHxVNgRijNvoDgvyPYXv/mA2VSTg1BACdezsVQhimvgYV7zHJ1NS/mSPGxTyTRRmsMEqc1ODMXPI6NDOZZJoGZQzg7u6uzs7O2nVxzKoapeXJ7Dx3rsvfPN6b6/QeF4wxWn8OMhg61/TN43nfrI15sVhU1f1dIFdXV6NVUZ47lozWDspnfpY7AWo+n7d5oEevMGLXnz59ansWOX82m9XNzc2I6QNe5+fn7ZFQ9L1YLBrzOz09bYD3zTff1I8//ljD8LDC59u+DFjI6qnAbAbNOLOUgv56YMcxmU5WjVM9lxeyJaPCNpxZmCTYfowT9t9kuL2STJvzk9/U/c3Yl5eXzQB5UsQw3N9wfX19PRK8H2BISuTGwLwszFMiZrP7F4VQc7BgzPYAJfqmyOqVG7MZBMGenixSZ8rLeS40oiinZlmvoLmeAYglqHoFyPUQs5GnQAGjROnoJNMWp5UYX6ZnXpzIVV07iFNWp2+cy9xw6KyPIO8em92VavEYJ4ya+xczzQK82JuEUzDOw8PDEVtk1QZIOCsAACAASURBVJJx2Xb4jHsCM5Bhc9PptO1T495g2wD9PHv2rE5PT2uxWNQf//jHev78ef3www+1v79f//zP/9zmdX193V456PtemZMzCezCAcmgjq171z12gC3yeT7i3XZA3/v7+yMCQfNeRmzBPuSgaHaVcrJNZmbRS1XddgIYBulIihEsFouRUfOZ0zXTzaS5Rtqk6gYFRwanfPTt2lHm/tm350Dzo259XdN1MwQbDQKnmalxLIr28b5GppL8cFymlY6ymSrk+PncBpH1McvbIJtMwHO0EZtFcnzOyUvoNAexnD868hK6V8WclmXKk0HHgcINObh+ljVB92tb8Dai6XTamKGd0Cn1xcVFffz4sU5PT+vs7KwFDhgbm2QZR6/1nD9rZ/YBvmM++XYh+0nvGgarTOXRj/WF7DOg9/q2rJ6yGY7LxYFsOwHMVI4VHRogNAxDi0CkSmY1rLrwghAEYIBJp+PapCdEdKd/CS4GK85Pp8yaVwrYAG2W5GsZJK0AK8VzSQVZuRg5feNkOQeDvSMrzMlpsMdmJmLZ0Y+jL+w5AaCqRk5tQOEzgwNzNJNwqskxmb7SB/LH4YZhaIExgYLVa1gK+k1WlyybJ86SPvHcu2EY6vLysumMjdoEB2pdDsIA7Wq1Gr3o5Lvvvmv6mE6n9fXXX9fZ2Vn99NNPdX19XbPZ/YucKcUcHh6OHgHFtX7++eeWVkMkshziQGDGw1hdGkhfcGBzP7Yp94eP2Ab9Xa85aGem4RTSY3fp4O8CMJSEQOnc6SFFeYNMOuzBwcGovuVCK8boR+WQOg7Dw6Nh8lVu7q/qfuMtWyHs6Bxj1oKR8j1CxnFcEO0p1umEI0cyQC/ZJ+NwxOVavmOAeWR0d/QyODEGzjezoHlxJRmH2Z73F/UKrpyT16RfA7MjqYu1ZnNZ3/E1Dw8P26qgU3hWMuljsViMnIXarIGdrAIdWzdO/WFGMHdsh3kwB+xqOp3W0dFRu9YPP/xQV1dXdXx8XK9fv67Dw8P69OlTHR0d1T/+4z+2wE6pABkxrvl83lLQ58+f19XVVWNqdmjXsghEDurWF3+bfCTo8Bn6h/EDntafGaDPRS70vYs99hgYzfXcp9pOAOMCRnPyZiI4wONCIBNZLBZ1fHw8AhMizmw2a899qnpMSzEoPyyR5+ZnPu5H+vhzM7XN5uH5SVYQ4GdnI6ryeVJoxsuYHSX43M5tkE0qDSMwi3GqlBHIqUbS/F6Nw8ZhluexbLfbVrT3rUoOYG4GU+ZoALTxOg11pPVxdkrm5OgLSABIGf1hZjwogEyAv81syRhydQ17gS2aSXIeNk8fyBn7AxiZB8X7v/zlL3V8fFxHR0d1dHTUxoaM7u7u6uLioqrunwAzDEN98803o1LN8+fP68WLF411vn///lHQ8RzzrUkGCKeYWULxgkmWZXo2YJBBZjBvs2HrluOMLW6MD59+itlV/Q2PlMaAbm9v6/z8fFT7MhWn9uVCIw8/5HgKlhSeWXZ2VHNkREAYiCdsgzfTQnneuWy094KBGSDNtD9TQoOG+zXoZsPoM331fB0JzWCZfxacncoa+D0eF4LNMCwHj5fnvXkuZjLIxiUCF4bRL79xbIOB93khBz8D3foz0JJ6Yhs4f26fwUa8wIOu0ZcDGrLAaQFby8IBjSe6unZGult1zwAZA2wMVr3ZbOr9+/e12Wzqq6++aosLOOkvf/nLury8rJOTk+a0q9Wq3r17V8+ePRsF8+l0Wm/fvq1hGOrDhw/NT+3oDiLI1Wze99Bid7lSbrt3YHZQghU7LSS7guz4abdm49ZJBkmz9dyS4/ZZBuZn1OM0GJJ3ozNY0k2ilPNZHnnMAKvG+4Wy6OooyDI1q0NO/7Ke5gctonjXdegTI0OgMJuMWB4rArcSnRIhIy8c4KgYTa+gjJzW6/Wo/sNmSq6HQ/A3BuV6AmPKCGrGasaGvgzeyA8nSCbDylTOw4aeWwD47bqc65H5Gdf3HjU7BLUkv5/UwM6+LYMScvIWDqedfI/deAwG5M1m09gSDKaq2s5/p1JV9+API7u8vKxhGOrdu3e1XC7r1atXzbb39/fr4OCgjo+Pa7lc1tHRUZ2entaPP/5Ynz59qoODg7q4uGhpKds4ZrNZSzP9OOosmwBsXiBBDta37Z80FzvLYGDZmYVZvujPmUr6AZ/hLw6oT7XPbqO4uLioo6OjZtQM4vDwsBUtnSeT12+32/aIaNNqp3eueVU95NuODtyhDzCaVXE90h4iWjoEwjFT8RNkze44Pumuo9l2u22pjMHBzuclbLMrMyMzMCvaWyiyoG5ANigwbvYymT3glIyBMRlMXMNkTPTJeLJew3iqHr+ooapGoE0/rkWalSJDflu/NIIM84eNoScHIDsNxXjXw5yiw5KTkbgvGJeBAXvE/vARbBuwMPDm+ewvY7vS6elpnZyctD1rk8mkAdfbt29rtVrV2dlZK82wOXZ/f79OTk7q+Pi4gdzV1VVdX1+PanWwWfpmjGZeZkS+A2C73T5izLZ9znXtmPNMRhywDXo+z0FuV9sJYFdXV/UP//APLdIdHx+3aOQagCO5gYbVE0coJnFyclLX19cjtAYE/axs00/vITIjcGRxsdVFRxpGRj8GLCvQ9aSsl9mxnFbSn/N7FO6UDZBwAZr+vCXEaSbzc2rrAiggCZuyQTEPpz++KwGdmLWiV9f47JCurxn4MkLDZrxJmbH61jMbu42eBoAfHR21uTFGakoHBwcNOBgLTjubzdp+MN+g7eCCrj1Ws28vslhvyJAAhd2TUuamWp+DbCeTSdv8enFx0VYlT09Pm+xdGvnhhx/aOyo2m039+OOP9erVqwZoe3t79Ytf/KI+fvxY8/m8Pn782IKJ972ZgQIc+GSv9uvA6e/Tvw08BqZeuok8sRXOcbB/qu0EsLOzs7q5uXn05mX+dormHJ3Cb7IaK5HobIpLnxgaRmNWYzqcubWdgdeMZb7PHFC8HcWFYzsw83D/jj5mDK73MDb6Npuwo6fCPQY7t2WdBVI7Ps2prdnnarWq+XzeAgf1GBu1U1YDCfP1TvVkbQkgzJ/xeYUUufF4JesEmRnMr6+v27gODw9bUONFy1dXV3V0dDR6DA6rmNhCProYMM97TGlZq7E8HLCcxnuzrIPiZrMZZRV2/JOTk3bOer2uT58+1dXVVZ2entbx8fHodqw3b960LSSHh4f1m9/8pj0hhfTr48eP7ZYsxrG3t1eLxeLR28Hc7JNOBV1X47fLKlwjg7t91pkEzf26+YECT7WdAEY9BsUzANgXEWm1WrXVlaTM1NAANaM7BmomAUglwMHwkolZmE7rKC7a6Lhm0liamUQvtWGMBrMERv62A8NSkI2VbFaEAfTqWTYCF0ETQNlTlICIoRmADZ5ebTRbY96ZUuXiR9YzDGLJGJ2K5XfJ/FyDw25gOF7c4djT09PGuhaLRZ2eno507LphgqzZE5/5YZ4OHPTheiJ2z5jNYqjrVD2sFGKDvkWKhZq7u7tWM+M5aK4rAmAGm++//7751DAMo9qmg7RrdC6RbLfbR3rFLjjOoGf7NntifmZqtnXvLOBz+xCZixftnmo7AQzFoYScgGnt7e1tGxQC4jjf9+i6FxTRiwSkkKbdAKXpKRGXyNa7WTujJd8jeM738ZxvAbtWRX+9p3FkDYw+AVQAyY5gWVY93KfX23Fv6u0aG30mS3VaZOBz7cf1DRd4TfUNJq5XmP5nAPKYXOfwuOnbgMb5jviWE3rM53hhr3zOlgsclgBsNoLtOT2GiWJn2KGL+Ni4a3ouOzjtyRLGer1uK/n2E8bv+4uxadJL2vX1dWPNV1dXNQxDe9DjwcFBq61hs17UMljZ5mkAtu3KunKzbbhea7k6A+L6vrc2a8UGMca1C8Q+eyuRC6M0O6DrII6g6/X9ex57Gwl7Cvb+JTt/1jQwMpRg9OYzRyGzRMbBdXA0mBfnJ1U2MPGZ01aub3lwHM5oJ2bezI8GQPZSTR+DHBPc0gAMoBxjZ5tMJm2D8unp6ei+SoAgQcwvZOnNx6Bm5zAL5n+a5eSW/7sUsd1u28ZVv4EdUGBM2+223a94dnbW9OsaFsd5QQVg5x5H33NrVuLyhDMDBwfbjOugvPWLHfmulwGk9ImNzmazOj8/r9VqVVdXV00mrv9VVSMBNJ6c4ccFMWbGYx155dElDNsl+ve8TDKG4eEOD9uAgd51PeyN/nspe7bPbqNw6lL1YJiHh4etSMnAKbxhAN4X42Vr+sMQLRi+M3MhCqWQMjLjBCwm+DqMn1uTZrNZMxbXyugH40nWCEAnqBr8AFUvLgCYzCNTWAMwyua301Yr2ADFMWaKPpaxoYvJZFK/+tWv6uXLl218BwcHdXV11V5xZr15E2o25siKmufcW+ww8PMdc/ViBee6+O95uaxgQ7escBQWjTabTT1//nwEMPSfhehkpGy7AUyoOTF2s0/qc16hd8pf9XB3is+Ddd3c3FTVPdPjurYH21+PqZrlbDabBpLYNduReDFPvi/SaaYDie+ocfBGL7Y5rm9987fl4NVfB0LfyP5U++ytRDaOpKGuvfC3ayeudbULxsqgWYSdjyiG0vjMqZUF4H1qNhYzPIzUQES/FDdd65tMHpbbMRpapnXMzWwU5VY9fvVX0mPP0SkW5zo6wpQ8PrMhswmcx3ufvArnms9isWjghXPDhh0IvIpGS+Zr/VgWXuVzTdFOmYsn6BXgIM0irTs7O2tPZ3VKY0eGnQ3DUB8/fqyjo6NRMDFbzBU41wSdBjkYcH3qV2xncTptndsuHDxJB52WU3P2vaD04aBicLctcB2eXLu3t9d2EphY2L5dOsIv1ut1XV5eVlXV8+fP27alTBfN2O3b9hczuyxZMG4v4DzVPptCYgB3d3ejGtJ2u211AO8cdiRysdI1BxzQxfIERFa5fEtEbki1AaJIfnuFygrKFJXfXvFywdgKtsNaqJkGMwYUaKUmi/DxbmaVCQQJ4GZejBP5OwV0X0RSpzzMhVW7TAGQfRoqf/s2MR+PfRCAevpzemHDxwaxFRu4WZMdiOZb3qx39ideXV21epFBM9Nxsy7mZfYDkDFOAAK5ARbICPB33RKA5BjbDVtEzIZIMx1IzIwAOAOnt3CQfuKj+PJyuRzdjpWB9ujoqOHB2dlZXV1djVaUuab9lPPtP7lKb+bmsscu8Kr6DICdnJy0aDeZPNyL5tUkFOwVRQaCIfaOrarR8rYZDiyjqlp9A+MHnBA0Bs9k7XR2cm/uzDTWmzmTPQAGGHCmPhiuFzF6YOSUESW5FkFzam3GyXiyuI8jEBC43QXnxjidrjKvTGNxRBu6n4NlVpgG53l+7m+uVfV4g67naSd1NAbwXf9E78yNvjl+Nps1HRkoq6ptBKUe2GPHvTGbTWBDw/CwyZtAzbnYEPbthQjbG77GNUjnAMHlclnHx8f16dOn5nMwTM5BVhlo+Q57YaHDfkudjLa/v99ux7q+vm7bnKhBMj8/tJK+bM8GdTNIl4ws65R/r+0EsO+//77evHnTnPv09LSqHvY0sbKGsiys7Xb7iK6u1+vRSxJgb0wkC48WtFka1Jec3lE5ncasA+PxKpady6mRQcQ3idvgYDkGJafPXNPG6VQyI7znQP8ev+cDqPHoYgyMVMNGVPXgdI7YZmzJ5ojcTlHNcAx6lr9Xlawbmtkn51sGWQ9Dlqkr25prdrAZF+cZr5+wipyXy2ULktyLaF2aKWRpwgtSBl6yExwYwGGMaau9bAKbdzoMC6Pwb5BKX8gyDjLF7j9+/NjqYIeHh83+DDjL5bLevXtXt7e3dXJyUsMw1KtXr2o2m7W7A/JuGuzb2Q8Bw3sBrVPbFw286NmQ204Ae/PmTauRnJ6eNmpp2r9arUYbT512sYLDsnGmAoARrMJMAeMiGniyXuYFRABFpyN2AuoIRA4Q3quVAAyRBCaSt+44V7dynFrY8M18cAzYDTLJ9LPXsn6wWq1arcXsFyNxVHN90SySMTJOy4yW6bb79Vj5HsfKupGvb6flXObo9NCyM7DgeDTX5rhe1pw8B4ISjsuGUFgOqSaOhkw9T2yEMTv44MSTyaSOj4/bvY5mzHb+lIMZv4OxbQRGw4/3V2G7DphXV1c1n8/r4OCgnj171uyG+azX67q6umpBkA3B3GAOO3P24OcAVj2khvZ1Bznm2JsLcjEL3QVeVX/DrUS/+c1varVajZ4hXlU1n8/b0izH5uOEnUPbgZKJZCERhmR2ZgBhoplmYIiO0hiI06JkR3YS0kX647amnmNjKIwP4/OmPsDBgIZBulbkWkFGUoM6RkG9ItN2Agjz5d63dEQDBnLxahZj4Xg/+SGDBNc28Di1ckGY79OouY7H52BAKpzjRX4Jyg4wLlG4hpuBKsdPcAAwE8joxw6HA/o63gOIw2KzXnEkEGLz7t92QmmBMVDvBfS4HrYzm93f6D2ZTOr169fNJnkNHT43nU7rxYsXI/tjTs+ePWtyxZ8yuFSNbw+rGr//wAtiBGDbke+19dgzNXX77FuJQGQuArCQurgQicBhT46WueLEpMyETHFdTIWdkQK4DjeZjG9Q9ebYXj4N2/GtJhnhHDkcxcxkcA7XOBiPU72sD5nBslDh9M51LWRkA+FpA7y3EIA1czMrsKE7LbWBIGPvSaNP5gaDzXQpa3j0zXnehmFGgVxh1NiCU0/rDRnaVvJ9hU5BAT7qpb6vFpChDmXnZ45szubm7PV6XS9evBiBrhkZjBpQNeukT1gYsuF80jNqgOyddMD03A3wZrXMnUxmGO4ft3N0dNQeqEhN7ezsrLFM+yV9mBSgN1Lh1WrVbjF0EPZDCGwPZsS5rSTl7ubyzFPt6W+q2ltYLCAUYGQ03Z9MJm0ZGNBBMM6VneIZvYnaGCHCRBiO5ozJhUyDAH35M8bumoZZFA5FfzghhoxhAJT0x2cGTTc/VdardgAAn6Uxut4C60I+Lu4boK0T5OjVK+RFsOBaflhg9olBZz3Oesi0z9tnDOTog7EAXtSNSFW4jtmbzyWAUBPFSQABxt6L7tS8HFi4np8+nDW+ZIlmY/7fdUSA3gyyVwvE7ghOs9ms1ZDRkeWXDNDBydd++fJlO//Vq1cNtOjH10/bQy+M2aDNM/Vgq7Yx692lGT++241+0JcXBRPY3D77OB2UxSQ98clk/MAyp0YI1I/c8VLwMAztyQJ2gru7u5ZX85YiG62fBmBHw4AcyTkvU1evrDn/p7FsTXTGQG3spIlmDzi0jQyj9THZH9c0AHgVdbVaNSbsTbqAr2+poiVzcYBhk7GPz9uwnM44fadl9DfbdmBzwdy3ySB3p0w0O5a35tiJzPy9agbTmUzu97VdXV2NUsWqavbl24YMNMgeIOMmazbCbjabOjk5afMw+yXYAHyuBSJTb5PwPEkHXScDHDJbgCQ4UzH7QZ4soE0mkzo9Pa2jo6O6urpq5RnGYIDMmqV92DXEy8vL9vSLZKYG/txGQ7/pe1laQK4Jdm6f3Ynv1INO7TR+LRqD8U3R7Dfix4KA1fihiQDien3/wEFABLA0KLi2RCNyp0OQTrh4iFGZwnq+rjVkbcrOZGCyAszSckxZ3ESheXP33d1dffr0qT1WxWluj8VmKst1zcxcU0i25TF6C0cvapqdug+nyujHmx3Nfp89e9YWIy4uLkZMLvcdegneKQkrVjc3N6MUnRqTswDXvnKnt+unaVfeX7bdbtt2gqyfMT+zI2zEvw3+yZ6ZK1kMQWC73bY7HSwbN5dlbB/DMDT2z90LWV6w/fL/s2fPWjmINgxDe8BoVTXmnI8pAlgBW+zZxOIpGaMPlyF6bSeAffr0qV3EgjDLQeBmNdB5106YOFEkGQ97zKrGaRbXzpdt0I/fWef9ZVnv8SZLol/V+DYS03+ie0ZTfp4SKv35cT1mjsir1wyObIu4u7trb4pGhgm0djrXTsx2YUYcC8N0ncnpkJ0bxsYGUK5F0GHeTgezJXNaLBbtnr7pdNpefmGb+umnn5peN5v7bQRkBVXVtvXwyGVSSQCKgGlgQLa91gtifM4Y+I4nfrCd4ejoaGSzZg+2+wQX24TtDV34/8lkUmdnZ6Ogjw5Ikb2Axrj9FnUH4awzbjabOj4+blnKdnt/3/GvfvWrVg8007dfZNDkeweIZFMORHxP8KPOe3d314Cy13YCGFHfkQxFsnPX6ZkNFOFUPdScvFGRBiuyAFz3MEtCKEzIr2mremBfXN/Cqxo/Vsbf29j8lnDO8fkujjvquk7jdMlposGEc2im21XVdjj3CrkoH2B0TYjoZzZXNWZAMGiPDTn6NiWnLRg1x1l2Pp9rpNHaHugfpkGK5JrLbDart2/fVlXVxcVFu81puVy2RyjzzDfmYsZMgOFzs2TAx8HUTmimy7wc9OiX7OL29vbRhl/bi3WYrMlyMpv3MSlP7Mf2jrNn+lb1sErrVWvAkuwJ9juZTFpKCGvzwxCsZ48bGXtrkO3LrJR5gxnYpJm+meeuthPA6DjrEQgnaSAKszNxHIKlNuF8lwhB6omzZX2FCWM4LsxyXRAcQ8w+cqk7o6Gf9e40tbfiZgUY3FyrMBAY0BxtnDJ9+vSprq+vW/DYbDajyJdGQb+WddY1MBhvaWBsLAr0UnJ041Q4mV8PhB3EcNh8AYdTKvROkZdnzRsAeDop9rNcLuvDhw+tD4M9fWJ37BhHNpa9deNFImRDc42TOWHHBGE/GywfGOjgkjUn+s/g6M8Zb5Yc/HQJZw22PY5nX+Zkcn9j+ocPH0Y1RmRxcnLSbGmz2bS3hrO6yebftCPmjlwT8FyD7dVJuaZr2Jz/VNsJYMfHx/Xx48cR2oL2fv4WF+U4BrTZPDwOx6kAxoxzABrUvnxLDsbOVgqAyuzMTuoHIRqkTFMdlR0BMHRTecAui/AoJCOjvzcTMgA4zTBr+emnn0abEZlrGgSNiOc0OlkRzfKnIWsDOWNkHsjSzuEU1XP2/AyyVY/rTVW1k+3SvxmyX3BiYKiqthoJeLgMQb8UrtmQ3dsr5zkmg8WmPE6AC/tnbDzx1rLCLpwmZxrl+fp6GaiwSYLVdrttwMO+MH54o9h8Pq+ff/65rq+vq+p+kY7Ntdg6jNJ2ykIA13dmhQzwJZcwHLB9143rv16sMo64frir7dxG8f79+5YTOzIjMP4m2lFwB9SIaAzE6ZtfUupm2o7SuDb9u/5W9ZAeUaB0+oBzEU0QmlMEjHpvb68plHOsKIzJzMlGBgC7ltQEPRlvSnRhcxiG+vTpU00mk3r27Nlob06OyUAGs3EkdNQyUPo5Wa5tGdCqHnaAI/cEL+SC7JKVmr1a7rlSyPeLxaJubm7q6uqq2ZD7dd3RbNm6oE8zu0+fPjW74DhAbRiGtsuc8oiDHk8yJfhyrhk+4zPrBjBYTLi9vR1tL3gquNjRXSvMAMwckYOZFzLzEyZYZXXhntrSN998M7qB++rqqo0fMNlut+1uGNcDzR7zMTx8n7Uu5O5tULY3xmnmjR1m2cltJwP7+uuvW0emmC44ujietSuci4FTgL29vW2Fdr9k1EohQnpVxStvXMPpYjK8qvEGShtHvoSUYwG3qvHO9oyk/j5rXfmZa2t8xxL9fD4fsQGYqKk18k7Hyccd04dv4aHZGFx8R79+Nprn52LwU5+jt6rHgJZFYvQL89ls7gvh19fXTfbs+vaN/DSDsJuDCc+h415cxn58fFzDMLSVT2TgzarOIkipncoSsAmOZqDoz4X0YRhaScAszPNxiuUCvtkec7O8sw7FPaHc/rNYLNqr2tDZ2dlZe3mI33zk0o3rebPZ/ctQuF3Q76A0sHl/on3V9sln9O10PIEappf2lG0ngLGR1U7L4OxMTiWJdigfw/I9f9QMiH5EO4yCibu+xaTMWvidk+Z419xc3CUFNsg5/cT4mJtBLOVho8PIs+blVA2DIVImmJox0J/l0hSnPTnclGsgTrBwHcW/XYesekg/PE8zMKcb1oNlVlWP2AGf2Y4IirPZrL37cDabtYcq4vTffPPNCBgSvFwewKYoOWBnwzDUzc3NyFaQWwYzxp3bB5AX8uEz7/BH/k7teZoDsj45OXnkvIyb8SJz5Gq/Ys7oA7n827/9W/3TP/1TA4nDw8O2SuqXtuAT+HKyZmzQhXXS06oalWmwS78NCxtJv80FCj5HhsjX51i+vbYTwPb39+v6+nqUe/u2DDMdWMLd3V2dnJyMHNSDdjHcKxVMNiOua09M0A+LM1IbaJzmcG2YDhEXg3B+jgGnoaDgTF0BboONU7eqxy/JgD2RrthJmL9l588xWG8ehgUzV9dr3JI9M1+Dpu/9pD/6RK7YgAEr62C9yNk7PlcmOR8g4/P9/f168eLFiJE6rWSM9FX18EZvR3wa7A+Gwooecs3n1bsOmgEhU3HYtlNu+t5sNnV0dDTa2gCbym0wXNd2Y//IVMt74th862Bj1n15edmCB3NwiouuATlAnnsq0f98Pm/jN2C5D9ufa2XMwXOyHHfVdGk7Aezy8rLevn1bJycnLSIwcLMoBLler1u0qarR27Sz1gKK44xMzKkXE7YzskseRyZPz7qOC7agOUbCd7AuCwgDclQ3gGdERvCu7xBBHV1JB9nV7F3gnJ+ggczNopiDr5spIaDOeJJJ2aHSWTw3GtcC2M1S8lgfjyzNQNOg0el0Oq2XL1+OygUXFxc1nU7r5uam7u7u6uLiom5vb+vFixdVVfXVV181XTvo0LiWX9eGDLmXl20YyPTs7Gxk48iXoGO5Y1/efO0MxYHVQHtxcVFVD2/xJsA5C7FN+IZofATHtt+cn5/Xdnu/J+38/Lx+/vnnka8ul8vG/vAZmBM2BCDO5/NRWcbsj89dj0s2if7xkUyPe/bplHMXaLl9dhWSPSI83DBzcddDyMO93EokYsIAXNYJXB8iYtC3wx/sPgAAIABJREFUi34ImKjhp2SgSK7ZA0AXaxEk486UizFgPPyfTMxprBXn+lLVfUrOErSBg3nxO+sgXMPXxii8H80sxikp83a0Zi6kW2a9Xvm1o2RKaBnwvfdK8XeuVrvG6LqIAebg4KABGq/su7m5qb29vXYPHmzgF7/4RavJkDo6PbYNcZ1kjDjjYrFoL4E9OztrNusaG0GPVU/XAHtObb35uuwfI/Dn00WcVtoG6Nulgqqq169f11dffVXT6bQBkPWHHQ/D0Jj26elps12yAvu2bSjLRrbdDHI0ZJ7g5Tk5czG79lyfajsBzHm6FYGwUT5AYhrKAJ2ukL75LSTOlxGw9x25CMj17FCepA2U8QKkBjLAyNTVTIfWAyunFBauWYdZJDm8nx4L+JjBuf/PpV92xty3hQFknz7OYOtalp0ha25PNcbhVNclgl1zs2xtpKRartdVPTCWDCY8wwtATqDP4rkBnObVXRYRtttte1orYOBg6vqoAcoO7Xnxues8h4eHdXJyUpeXl49KMsjVaZmb7Ri9OzU0yDF+MgBeblv1cHuPSwUGSViqSw3ol/SScXhuzMUBy7Zi/zGzNBb0mLXbZ98LCTA5MsCsXOvxs7syb4d1QbfNNnyzKT8AGgpyWgEIGJQcCbN+RXMqYHAxy7DwGJ/pNkpwSyCzsWAwPESOcdgJ3I/BxzKyLPNvgxO3B2XfTwERQYDrOkgAYMkEc8y0jNbu/6lImgyO47wvzKUC30Y0mUzaC18vLy9rtbp/EzXX29/fr9evX7cyRQYa/nZt0gGNtI5jPE5W5ViUqnr8VFj6o09AwnUibBi9LRaLdouVA7xtvxc0XRohJeRzg7gZKSklm4JhZLe3t3V8fNzsireD84SMtJlk67aLp4J7b+WWz13M90rrU+2zbyVyhPELark4QuJx0CgFQfE9kdMbYK0YszCzFfp0aufI4l3HRnwr0TeL079TX9e5nJr53XgYOWDJ9anlJViy/2c2m7UaYtX4wYhmIwa97Aulul5ApMvietWYxRLFbDwOQGl0rhFm3c9BizE7VeZY5EFfrj851QQgsg7naGzWY+Dn2icnJ+3FHAAaJYs///nPNZlMmuOdnZ2NQBI7SdafjC1ZNY69Xq8bqHKnAOBwfX09qgnlPYowNZ5qQTD3M8o2m01jSVlWsM0aABhn1mAJbNj0zz//XJPJ/b5D7jfcbO5fdPLhw4c6Pz9vvvDmzZsR68xm+8RffIcHY/bnztSsc2clPRab7bPbKH75y1+2wbFTHeNksEQ95+wYCUDBwDDWrBu5FkIzMHkflNPDXNkgssF4XLdBkQcHB60gm07qAjfzw5gYk8dMgdiG7mf2M1c7f6bXNObRAwm+x9DNCjIt8x6mBGbAA8PJGhhj7rEmxuVxZ23Jhul9fJmiGBSsc4OHdZNpdxbGCXY8JZj0j5dWzOfzev/+fS2Xy/r222/ba9XMmHoprceG3M1UYVbe/DoMQz1//rzJAaaDvAGYk5OT2t/fr6urq3KDNVY9vCnbLAsde6UwmVkGxLTB1er+Ju+PHz+O7gAhKJyfn9fl5WUjALngg/xZVPPnpPJOJTnXsvYeTa/ouwa2C7yq/oYnshq0GLDvqWIiCBwm5giRq5Wca0PBIZi4GZb3xyCIdAbadvuwwY4xAAoIhzECxFXjfUuOAgkOBgDXZ5DPxcVF7e/vjyKVjd4tr80409ByHNB9A5sBg3E6lc/CucfgucI2DaLImAjaq73YWayn1I9XQb2wwNh6tRD68/9mF4yd6yN7xsuGTa797t27qqpWf4KZ0SxX686LFAZegpa3spydnbX/WQyDDXsLhQEbOVo/7JZn/6LlRGbiDMAB0AHdtuC5X15e1mQyqefPn7fzDg4OGjDzFFoHT2dWtjOvpFpH/HZ5x/qzfg20ucu/13YC2MePH+vXv/5129zI6o53lbuOhTH70bq+fw2hVj0sCcMAAA2noAAmDkoxlVTUk2M8Zh5WqPNuDNCrp+7PtZCqGjEWPybHwufxN0TiXG17quW4mAcGj8NkmurG58nczL5gkl4QMcvJAIMsk21ZJh4f4O/UzPMxS+b8bNSUcsOq9Wxwo2/GznUtdz+zCwC5vLxsY/n48WO9e/eujo6O6vT0tM7Pz0cZBeyuZ0OWg8F7tVrV5eVlC5S+R3EY7l+s6zKK2Sg+BCAMw1Cnp6eNxeUKneXIeM1QnbJantgTK+KwUZc43BcN5k86atDNe5g9Lgc1jnEQ4H/Ih/1vVx1sJ4Cdn5/Xcrmsm5ub9vRUF48RNsYFHbdiXNDMepELmQjHf7OaUlUj5LfSkwlZYVZC1fgFHDZSjkNYKTDA1g7uOfZqOVzfaQnXMeNIUPU1uU5vVbIHNslYMpp7ngZv5u3ree9Rz4FzVdNzzHTF9TuO8/luZq4GBeZkhtFL67N+BUvxSqEBjfsduQZ2x+1u1kmuiGXg4frMgYB/eXnZ9kSakZDGuaEzQNFZAUSBVC9tcpejex6wG4LuMAxtqwrAyr26uQveYLter1s2lj5sJu2x9eyN4/BPszWPu9d2ApiXTjOiGjxQHpM324IRoQADEDeWshfm9PS0sSsmeXBw0GoEZmCmrozNqRjnIxDqF67Z9Bwgd1XzWGvGDVObTCbNABK0nQI7LfJYMSSu7xSW1gM4H+valltGTbME953HMm5qZU4t/UwojA0H9LYYF9rt7AZZxs21ub7vW6QBNLY7O4XZcNZ9sA+YBeNxwd4lhsViUT/88ENVPTywkTRqb2+vXr58OUpVASGDuwOEa5X0ZwLARtlcQAAcLOfJZNI2cBPUWd3uyTL9FZmTQTE3jj89Pa2bm5t207cDBqDpYOIaMfNDtmlXCazWTQY3ZJm18qfaTgCjBoBge3uyuECvHoSwTHMBIADNO+X95EUmyWde2YLyOu/HaBCmazDU0hgzqY9ZUkYz+nBUsIP4WWRuWb+xPJ6KJsko8nNa1rMYp+fi6/mY/Nws1udlfYP5eLUzZUxD1pnuJigbuHusM2WQRsxnCWgGBx+T9pEO47SI883+ARoveLgm5/E5fTKwmUUhKzNL+1naKmNHH1ljMgO276Xd2Y69VaXq4UWy+CZBKc/ls2RQDuI0B4dkYAlcybaRRfaZbSeAed+W2YMNhlSF2lfVAxMjyqB057k+FgGC6BgPxkXx0imoDdjCsmKz/uKVN88h01rn4ln3GIah3SBrg8400ds7HAW5JqzOlDkN0EHD6RNj6tV8kElvuwIyM/vimjZ4O5xBJpleL13POgefZYrnvw2Wnof7thG7RuqAmKlr2qod26tiDkS5YsotTJvNpt6/f992/282m/rVr35VVdVeNWZ/sD2YnfHbqSOMazK5f9Agj6c2owVcvKiFH3HPsu8RZiGJ8znv97//fZ2dndWbN29GQYzH6vgJE1X3LzRxHZF3WF5fX7cVVtsGAOzSUbKx9AeaiRG29bn22X1gRncP1rUKT9ibBmFQftV91fjhcAiXqOctC0Q9hOItDdQBXMsxqJlJeNUOI8u6FoZusE3mtdls2qY+A0xGCSvHjpkgZjabdNtMMGXcc+pevcCg5Ov7N3J2H4AzASdB3P/nQoWZUrICy9vXT5bluWfK6AjNb4NOr0TQk4evBQha3tgWKRsZwsuXL5veb25u2tu8/Y5EMxeDN+PrpZz8tr4B2kzL3X9mNMx7Pp/XarWqk5OTBtCsZPJYIet0Nps1P7WcsVM/624YHl4Q8v3337d7U92X9W39ucRjxmWfs1/tSh+r/gYAA0RQ5Hr9cGe6nR+jhzERLUj3sugLa4MlICiE41Ujb/QjuqBcogBC8yooxVhv8LPwcFwrxkvlNCLU3d1di0Y29HSIFLojkp3ThmjlZ8qTcuuBQB5DH960amfBOHxrDgbn+p3ZAeP3NgjPA3309rN57MlW+W0mZRlShsgUuDd398nf1qnl4udk+Q4Rro/tsxGZW4xIt9hH5frW6elpvXjxor0ABfm7vpeg5lRrs3l48iysixSWG9AZf2YGjB+d3t7e1ocPH2oymdTLly/bFp9nz561BQHGgI8DYs5AzBi96jwMD4+gyjTcgd+pr8sdxoVe6aXnS9k+WwPDcSnumTZmHgzI8T2KQsEAH+cBHGZYXDONyQZgGu60wEbhIr+/cxTwWDjW6SbXZjzeO2bnzpTNYG0DfSqXz9rbUy1TeOspV22YpwHCaSHG6e0jmT64/pGUP9P0bAbAHJ/7MSNm/rmQ4sIw55htZ582en/n4OWHaGaKY11ttw8buJPdAfaZhs7n83Y7jj+3nzgQmKVYRgRh5AeQu2zBHLyCz3n7+/tt68Yw3G+upS82cnuPnwOZ2bHtncfnQBJOTk5GK6Zpm87grLve35aVwf7veh6YU4qkwtBXFy1hQb7Z26mSVxdwnmEY2goLINAThGsTZneTyeRRBLAh2lB87Gz28ILXVCCK/fDhQ3ttF9/ZgFE0CjUt9tzzvOxrV3Mq0asx0QykT606WnecT+DxfqbccNprvZqYa3ZmtlkH87hzO0I6syN3sjDr2kHKdShHcss7UzGaZezr0tdms2llBHSMU2+39yt0f/rTn5o9XV1d1fn5eZ2entbJyUm3TgaT9XiRG2PihSc4tbMhXlqLPOfzectahmEYPaHDCxFOQZEFAAfIshHWfg4j5m4CgC33cfqcZOTeoG798/9Tdue284j5fF5v3rxpdSTfdc5vhODc1dsvMufP6ObfRmR/nswDo00khx1lfSFzbM5lkYJjHP1QLGkDLWtdNBu9j4FN9Obwt7ZcZXRzrcWFzwwABlzLgEbh3vr0d09dl0bfmY5mem1HyfpfXs91TI+blsDlebuGk6l3Ljx4DDlejsMpDX7IjIDhm/6rqm1xOD09rdls1p7D7/1crvNxrhmo7ZjvqmoESIyLp0ZUPawSMyavKDP+nizxE66bT/ewfyDjHKMJQK9k0/P3TDcJDH93CslNnt664DqHPwOVMzLl/XW+SXo6nTb0ZlXFBm6Ki6LI8bPew0qLx8hvLw5k/75ZfDqd1uXlZYuyZlPIxOllAoVXY/IGXhSVY8DJHJVoyMy3org5lWXe/tyAkUCRqVNGS2+bcb3wqQZz4zj+Rz/o0g7K3JgDdpaAYuDIsWeKaJswG8OZM5Wm0V/eLlM1XpEnZXQKulqt2mokdTMam0avrq5ajYtd9W/fvm1FcIAAeWetyH7FHNjbOAwPj8rhlp7pdNpeRedVa2TAKissi4wLP8hSUcrKpMD2mFlBMn5k6LkZ1LAHfOKp0kvVZwCMZwb16hy5+sdFXcB1+ja66OzhXkIUTp+eQEZegCSXoN1vsjAf4/TEKD8MQzNansvUYwqM0dH8qeaicI9tmZ36s5SX0/SsISVj7bHDZBw+Dh14pYvruH/+f2ol0tfrXYeWaRjzzbTWYJXzs9MQPA0kHreZeY8lZEaQLMtsxZu0rQfXWmGfrtkwBq/ETyb3T4HAD66vr0d7zjLVcu3XY0uQ9TwYG31n6uwHgTpAebHJAc39e3zekpR69vz9nTHCtoBseoHoqbYTwNhoaidgMAwcoSJI351udAbRcWyvDNqwvHsZNCeqIOAe7Xd0xdgAQ54EYAXDZlDi+/fva7O5X2lK0OzRbo6h8O36UYIbwMr1rLSeszhlzN805mlnsVxcc0pH9QZis1RH9QwqfmyzndVAZqbIXOjn5uZmtLzeG6fH4jE5WPYagcmARXN2kOla2h4y8qqn007maLBGV2Z61rHtiPsOOQ825lfArdfrtlVnf3+/vvrqq9ELMwygMBSDucdj3dA/1/VDSS3vDBicn8HE6aPHxnkcbz3aJ61768Zjxtd3gdhnbyXKWws8oYwABiachsG72ZkzemCQbmY8RnRfz7Ulpxw4k2809dhcjyD6ub+cn8e42WxGNNvn+Tgc38ZPfwl2VnquLNIHn/fYGt9l3cnGmKs6dmbvg7NezGZ3Mc+sR5lBJCNlDk5Pc5Eij88g4vEzh16dNaO8dWKgn81mo5JH3nHgmk4CiZlqpr1mjqRusGuApapa+uniPc2+5OzFdTrbRA98XHLw+K0Dl4UykFvGk8mk3TOKTJ9iW+mfDphuXN+3p+1qn30iKwMlEgMamaey/A1bwvEdHWBhvHzWqZ4nw6Tdp3NrM77tdlvz+byNkeVxb+5zbYjGU2J//PHHOj8/b/l/KtxsqOc82Qw8WY/iexSVss7UiutkZDPoO3pmkLG8cBBWsygge878nfvccr7IgXEBBDikAZP6Rq4ocb2cqxt69v2ZuXJlsHVtje8Zi69jR3O0dzZh5lb1+KZ2sw4HLjuca4gOIGY6fvsRY+DJE+v1uv7617/Wer1ujwR6+/ZtTSaTevXq1ajPtKdM77PkYxsxQegFVOa0WCzqw4cPtVrdv4/g9PS0ZSyZrWQ67+zH+ncq7PH6Mde72k4Aw1CWy2UdHR2NDNa3p6AsHCgjuJXrgiKg4aX36XTa9t0wYcCO55Pl7TV+6YJTzaoaASXC2t/fr++//76GYRjtIkYBGBO1AtNzBOxIZoO0M9IXQMEKrZWSTDH7d72gx+LQh89zUMD4clWK4NB7a5TB1rUjg6NTIWTngIPtZAT1PNwvLQE4A4HTVf+2ffVKDHZggxNytj3Z2ZyOOcgafJOxoy9SaAOdGZz/94ZOv+ZtsVjUfD5vcv348WN7zPPJyUlbJUSHZqDYHvJm3A7I3uJkooKd3dzc1Lt37xprZDP3wcFB217k+aVMssSQ+k89JpBlRpPts/vAvPoB8DBxorgHb7bjQSUroA7jnfMsA7smgsD39vbam1YAEzNC1zdSiI4us9ms/vjHP9arV68e1T/cXOOhoVxTeQzUVN4OQz9emMgo12Nl/t6pldMdjJ29NxwDkCSDwKFcy7KD9WTIWKhfpcOiQ4yN6zwFtp5bD4gBGLOgZJ2WAbrIiJ+69MLRMAyj+3Wd3iFjBwP+T2ekv8wY0HcCFMf5mk5nU7akUqzOw8xev35dd3d37b2Z+/v79fLly3r9+vVIZ4AbPsY4GL+vhS5YIb28vKy7u7t69+5dTaf3z+0/Oztrcnjx4kU9f/68rq+vH9X6HJwMkDTruJdFID8D3FNtJ4AZvalXoUweg4MjGLjSmZ2OOJdmoM7JOccFSkf7jFh8l0VGj9XHXV1d1ddffz1Kbbzp0w7g/pICJwh5I22yMfr13wazpP9ZgO/VhQAswMVsp7cI4A2Z9MdtVpne2hFzzMzdOvXnPQBJWfX+99ws654+HOB8vu2G5npLMqCsXXls6MVycF8ca/Cz3hKcbfNmSiYDfIdj+zYw9Owf31qErUMyeL7XZDJpG2CPj49HvmkZXFxc1Gw2a7cbTafTtjkW/7esuFXJIOhm37XO/J1LDi70u7/ckJ3ts0+jYEWRfSI0pzcoARrrDX2kfpvNpqWGfmmEd8I7TfMLQ3F0JgtL860vGJwNtOregU9OTuoPf/hDe+EnfaPskUBm4yeK0lcKEfDuveTADpBRm/ZUbSnvmdzVPEf67KV/Zhd2pASIZBfu0wzEdSYbdY7ZtbTpdNrSVacpHk+vdueGXhL06StXeN1g+GZMABrH2n48/l4mYdaJIzrltM1guw5EsOZhGEaB2fVOB3wY3dHR0aNtPlX3m87/8pe/tPoRe8940cj5+XlNJvcv1b2+vq7/+Z//abXo/f39Bm5kVQBX790SjI+FiAwCZpkGK2SRaTrnuC/rNB/66LYTwE5PT+v6+rqBEMDkpdjtdttWImA1VpQLsNwIDXLbSOy43veCIOjb0Y9+HQVt/NPptI6Pj+vPf/5zPX/+vD0V08Knue6UqY9rBHZ+oqTPN6tyrclOmkqj5SqSr++tCVlv4Vz+7hU+XXswK07ATKBOgLUMHP1Tn06fqx6ncT2Q8sqbdY3MnmJKBmAzU7Nqbjr2Nc2okukZnJ9i1O4n02fLnDkzJ8DIpQazPfdrMHb5hv6QJfdekgkBXrDs+XxeP/zwQ9si8+rVq0ZIWI1nBdYyyWDDXFwSMWjbLnpBuFcD9BzN6Ho11GyffSb+d9991x7yD5rzxInt9uHJjKw4eXUShuRnZxPRQP4cPEyOOwBsMH4qpAvSdiqnuOv1un766ae2RcJ7bUz5ASIDi0EnW47XOT9j9aJG7xjO99g5z6m7QYnvd6Wm2SyXqvHSuo/pscycsx8XzPhzDrRkxhksGIMXV3wdZGXQdIDpAYXTO8bXA3h/7xoufWYaaGaVTIIxJtiZuTpdpF+CpOtAyRCr6lGd2XPl9iXfXsR4sHcyHL73ajSMK+eYhXxIi1Ndy8BAxLVYsOIzM07Xv6xP7NCZyOdAbCeAPX/+vD1o0DdboyTYB6le3hLCLTpQXgZpup1F0Kr7TY+uvfnFHq6JYTQuglZVHR0d1bt371otACNwOuMovV4/3n7h4mMWIi2DZCEc68jqiGbjxpjdzG7o0+e62Jn1sTRCX8cvT8GoPN9mELPxzdi9hr4N+BhnLn48VSOxXDz+Xj3MaXCmIBi7ncJycx9mdWZZfObgi/15FZJ+bWtZmLc8zVqyTmkwol9vbSFgu1SDndqhvZEaUIMh5YMRqW1VVUsf6bd3TbNlg61Bq8dgTS7Mqpm3bd26MaPspeK99jdvoyAfzqKic2MjMMBkpZhCs0PehsR5AIijEEDmdNHKWa1W9eLFi/rpp5/qr3/9a3399dftODs647cz+6mzR0dHDTBN8e1IZlPJXvL5WnZGf5YMz6mwjSeVnsVlO5JrWJlSWkdOd/m+qh7tC+N4SgW+8b0HkvTTAz8DJ//3AoOPd+rMmL1S6qDhmpTZEfLynM0kfPuZ2YCZGzqm5WZfZJIOnYDn1Ar/cH3MzMM2YL1CFFxT84tJrD+eKGF50hf3TXqOXqE2+CTjtg34lW3Wj4HZtUvkZJt2JsV4LNddbecuMRyXwSeLAOAYUKKsFZkU3EaEAHpMBaUYkf2Z09XLy8taLpf16tWrUaTlepnqZAQm3XXkpI8810DQS4/4u5c2Mh7PNaOMU6hkWj3QQIYunDvtyjHYAT2OHE8v/XHL1Ugz4t547Qwpm6oasYHePNImq8YFdcZqFpZz8HiyxmOHSabhaztzsL37/3R4y5/xGLQtY8/d35vx5UqzQdqssKc7Ew/82LLygobnYDkaoDmPUpHHbBtN+aM/A55LPbvYV9VnAMwO6FzZq4NmRgwMSktkcwpAv5vNpr0k1UCXTIw6GUqlcO807fz8vC4uLmq9vn/NkyOZBe15OTVIw8tm5Zq5GPyear0UDUfMdIl5Ox2CbRkUPH7OxeCdbrtv1xkYu69pZpzfp6O4qE7dhfNTXi72ehx23Nyy0QM2Pk/2lfLLFNLB02NwbTLZQaY9zMlAxhw5lzoR9tFbqEnGCyAaGC23HLP/TmD2fs3ZbNbqXdS58FvuseQHnRuIPBeDbQKZg4Zlb9nl+T6P1gv0+Ege+8gmnvymHnJU6kcIjC0EoDhpk9M+p115v6CXohG4B42QWE3x9b0xk9fD/+53vxvdAO5aQNXDZlPoLnQZY7TwrKRe3u50xEaVTshY8vus//k8s7lUusFrOn14W5KBnj78OZ+t1+u2sZF+POdeas24kVmuEtpgnV7bqb1x0oyafpwm8pnnn4bt2if9GtQyGBogzCoAXAcMxsw8sBfryLafgAiQ+AEIZqZ85mCTgQYboV+PN4Osx8z1ez+9NN2BLFlj1cMjqy0f+7aDSaaVPfaUNTWfn8Et+9iVRn72kYekZ45qfjor2yKM2Ln5zAVMGwvGCguz01nQNgQEe3h4WJeXl/Xu3bv2tEueOoHRmYa75uQFiarHK0aZ8jBnR4NMueyE2ZyiMTevNjqiG+hms4fHCZu5caxB1tsiDN5mqnYMN3/v6M64PWeul2kt/dqxvZDRA+M0TKdFPcaYKQ2fp6M4yidbcTNT5H9sGMAfhoe3w9sOM31kfLaT1BH90pfLKsmI/ahv1+kMSj2ZMJdkl5ZVjt8poetx9uWebxhwptNpw4FM5bGdBEMzWNdTMwjuap+tgQEaNk4EOJvNaj6fj+4c9ws2sjhXVW1VcrO5f5QIaWHVw+qW6yCus3Hcy5cvq6rq06dP9c0339Th4WG7zQgl9sDJOX9vKwLnm2VhyK45JStItoPsdgkfwzajsOFkxEtHTeUyBvrKFK03ltRLApKbV8Kc1u5qKZOqMfC7DpOpOb+TiWZqYjkYTJBnpq/JYKrGixdmabnqbXtErk6nPL9MqwAHvrPNOYD3ZOcUMRmPA6/lRfDLDbeZFibQeJw0+ktG7bKQMzSzRbN72wBytc/62Ax8T7WdDIxB8l5Gp4MMlPu0cBqAq+rhTnsbCIZhwWOUpt5MlJ3bd3d37d6r3/72t3VyclIvXrwY1cRgLUmLjez0m58hMDslYGuQcT3F/dC8NL2rGZzS+HuUP5ujoaOunyaaNN6slPvjzEzZg+fnRBGQ3Bfz84qZmUEvDcb4LS/r2d/zWW7LMRD5d7KoHtCb4ThAZCnAKWQGEWwT2TrIcYwDJ2Ox7Nw/tuLN3pnO2SZwegDAdwrkxmyIAQ86SBactpR/G5jzPli+s/48xiwFAaaM1QzSMvQCBRvOd6WPVZ9hYFdXV21TadXD7Q8eFH8zEde+GAwT9K0KVdUKiEZybvKG+cDiXr9+Xd9//3396U9/qjdv3jx6Vv16vW77XrwUjCFktKIxXsbIC0FRvGtB/E7Q6aUntLzVyPWUHgu0QxmAzVDMfly3yXpS1fiJA4yR4/jtz9m5bT1aV66DGSjdchsFizrogmslkFkO7iNrkXYe+jKL8NzNvg2ynMc1qf/ZSZOt9dJa13WG4aH47dIH57kMALi4f47v1ZjIeAwCnosXYNCfHxFlG3AAZg53d3et1uysyAyQ/9NnMgA4jeTzxWLR5opPWKYZzLiW59BrOxnYixcv6ubmZmS7DVtOAAAgAElEQVQYd3d3dXx8PHphLdGI2ph3DrMbl0H7diSE5pUskBfn2mw2dXZ2Vn/84x9rsVjUd9991wAuqXFuosTIXMNgvK5ncFwP6GzkGELSYgsdxaAA79vhWMaRhfik2Vk7cXT0LVjJfGgG8mS2zQDEwMwMOJenXWTdArZnXXMdmB1jdyTNmk2Oxey1x4CQgRmQA4p1iYPmewZS3wYW5megyj7NSuyAGVSwF993a8DBLtktb12ZDHgsBgrLCYZlQuB0M1NM5Mc8epkLfee1sbteZgMQ2waSUDhops0yX/Ai6+nZdjKwd+/e1enp6cjx/QTT7XbbdupXjXfmm+Wcnp5WVbWbMieTSXsIodNMFx4RxtnZWf3ud7+r1WpVX3/9dXuwG07vPLpXAHbURsg9dsj3sEyDiqO3QSBXELM2xjF2QsbZc2D/z7EeK/OC0TzlmLlS5+v6f59DvQT5s8KbEZs2DEPTPbL2yhW6ZCXPzmQGadZBuujmAOU5VNUoPXIzGOSmT+br/pFXriwms+DHq5f0lwza9uIgZVmyEn5wcDDaguKyCsf7fDNm6zgZnRvs2bJmHp6fFysc+Gx7XhDq1cBMSiwT6ylZs/3TY/+7UsiTk5O2slD1YDDURnKVyQjN4Pb29urq6mqUbqzX61Fti0ECZpvNpo6Pj+vm5qb+8Ic/1OvXr9ubjn1tRwDXGzIH9zPyJ5OHp2EMwzB6fDAKXa/XjTnaAOyEGIENN483M0iDBBQts6ypOEo5qmJkudmxV1dD3mYY6cBEzCzE8v1ThuQ0287O+NyH64u+jovjyDFZr23KgOP3ilrGvbdW9yK5GTwA3gt8yTCQq+3OYOXVxQySDoY5Tv72Oxn5zfEOwE7Tttv7nff5whsHYffpWt5TbNNzx/5dNmLOZqL87S1L1MjzxclZ10Re9r9d7KvqMwC2Wq3aTaEusGb6BvpT7DdLsYCrHtCYnBjnqbo3vNvb2zo6Oqrlclnv3r1rT0z1Xi4bWtJ2HACGgvECwj7Oz4a34QJ6diYAM9MyQM+1J4wGOZm6o6w0Lq8YGcTox7UnO5eZYoIojLnqIT3z3Q8GkDQm+vZbqbNG4zkxVrMry8qs3GmU5+V+kIebt+oMw8OjXmjYEU7mWoud3nbCGD0PsyanYAQ5O6zfyoOuzXjRnf0FX0Im2B71nqOjo5HdcH2XWsxo0Pnh4eFIvwS5XokC3fcWZ+wLyCIXpzKgwSYdTGCrtkeXQYwjZmLJAnexsM++FxKgIWe30yB8P1IGuu6aEc2rUAlqPLb68PCw/uu//qtOT0/b/YywvMlk0p4ygXMCHnY++jQzJCdHaBzDb6O9Adi1sGQEXCNTMhsXBkBztHNNB2NNMDLDrRo/oNC1tTy+x2IYF8aVxXY3zvPLjB2RPUenzvnOT2QEuBFMckOvr5nytz4ZS9Xjl5NgU9ifbc+ypi8DGce7dmfZpT1jd1kcN9vKtDcZsnXN92ZzTltd83OGwXm2VevLdUXL2Trjf5iT/Zdzfb8w5MDX8fjN9O3rKSMA1umj+2FO/+saGFEio7QN07mwU0oGmIqpqkfspqrq7Oys7u7u6j/+4z/qm2++qefPn7drWaH0gRPgGEbyBMiqao//MdOgbwsuU0M/PC4djWtaIe47ATPPRcb+n7l5Dr1jzPDcchXSaZyfIFH19Bu3rRfkwndmkz4ugdXnJGB7zE6nOd8FaZ5TZXsxc7OenbbbkfnOc3Q6V/WgL4/dAGdWynwzDbMj91Jb25cB2UwqGQfysw/Y96wTA13K32Bo5m8ikmzROnImZR832JiJ2rbQpV9gYv0hawNpyvCp9tkUEhBj4nYAK7hX3M48FtCB6pvR/fa3v631el2//vWvm1IzfXI+jWHzuQVBlCAacIwpbqZ9jMeOSFR29Koapzt2jKxxJEg1oSsV8ZYSlJiMEhmmnLPo7JS2Z0TUHX3NqmqPWcnaleeRTM5zye+8749aiHeok1rkudYl13QNLx3fQFP1kLL1ADg/d1qJXfGb/gheyVacQpo106/BA3k4uAGSjMF3umTQyjTXwdIsLbcj2fm9xcI+6w3O9jP7ja/rcWBvLlEY3M26yLC8SOf0NQlFL0A+1T57BPuCmIQHkaiOoDK1QLDenIZRHx8f11/+8pf23C4bGoJ2LYXHU7OLP+mnz7MwaYCqb5GgsfJWNX5aBlSXYw0MXuECJHsposfGd6437GJkVY9vvvZxCVYc41oVDKzqYR8OKRMFfAAcY/cqZ6bRPSbpOqNTSD9jirlQerB8Mgj5d9XjRxCl/visV1Jww3Fcg+rdJ8qKN2N3Xcb9ulaU13QtyjUkznGAotbkQEtQMrt7qiSBDA046B27d4DcbDYjm3CgzOCYTIvPmA8LEmaZLjcwpiy32PYNfA7C/+sU8ubmpm0mRZDcuM1g6TydwANMtjMMQ3v8ze9///v66quv6sWLF23wvX1EWWzkc0ciFOgo5adZMBYM0QK1MunH/fn/LC47ZTYN30WBHXUMtsyPvtxf1fhZVMmIOderk5mG0wfX9QZTf0/QsSP5Gh6vX0hBfx47hs7YnIrAHjIN8XWe0pdBw2PyyvJTMmelizmSDZgFJfAYPAA8O3KyLwf+6XQ62tTcC14c488ydYXp8MTiDCyepzMnMzPmj84zW+G6PQbn62VAsizM5qxD1xC9AJLptetpBuxsOwHs5cuXDbxgPr077e343s+CI2C0UOWzs7P66aef6vb2tr7++utmSDc3N23TJGmmC9i+lqMMzC7ZyK6UwsLFQDLC2+lpdtJsrgG6kOmWrMHA68hlA/O1k43k+TCLHlOCXZlpuK7ptMHskxqGQZnfzDdTdOvOAE9/Tl2QtW+pyfl6DozTQY2+Waq3jA16/ozz1+uH+3O94OAMo6pGgWE+n492xafz2fFgNFwvF7rMbgxW2LhBDJkeHByMrpF26zQc2XJtBw+XFjx+fI1xZzAzeDkj8Fgzq8BuGTO4YHtyEPhb0sjPMjA69kZTdmfbULwL38LDMDj++Pi4fvvb39bt7W29fPmyGQvCcz3GzAfHtID439GBsfh/11WSodkJqK9xrgHThWP3nZ/TnhK+Uwhfy3vbaO6Xv0nrcCY/RsisgnPsgMzJRmKG63kZfDjed164pVytNzulAZ7+mTs2xhgcXBgnc8p02tfpLUwwB9uO2SuPV55Op+2pw2amOK/TLLIRA7CZQ6Z6uTjg8XCOa30Oqswr39Dl420rTivpg+t5QyupM4GN4zjH7N1j5Bqea6b/XjByqQemawKEjpxCem672k4A460mZiLr9cNTSx11vdEwFc6tEtvttv77v/+73rx5017Oaafi78zrmZCNPtPSqnoEfBbAZrMZUfhkUgk2Xl3NNIjz/dvyqeo/Q4zjHOX4nbv6uTbnw3Zy64DlNplMGlN2HcNplcfoaxisGI8jq1NTy9vR3fUz5mFmh7ycThG0+Nty8RhTFlzfcu4FDDMH99EDqFwBTV0ydrNkB4lk1iwCkFF4Xukr6Mgy53hs3mQgFwI4hv89zrQ1+xIBb7VajepYZoVOG82UmIf1i805zXRAOjg4GLFNgznBbTKZtLsT/i4Au7i4aDdgc/uQc1sE4Ojh+gkAcnJyUh8/fqybm5v65ptvRkzBu6k9IYTgyTkvNv23sdrQqvqrhAjV9Syzxen0YROlnbNqXLOycyN4A9RI0JPxdgFHyJyHHYVrZkRPZ8UoCCQGWMsrjZF+Pe4MJHkXgseUdRA7km2F6zq9xz7M5p3iZh2F5tqa5cH/1in72MwQGI9ZrMsWfG5g5beDpFMx5mu26wUjPyE15WVGjEx6DA2deh+d7cLszemqQcjzH4ahbX5lVd9gnWPz52aTLtukzrFH10bpx+eayWF3n7sPsuozAIbyYWFOC1ECv22kDPTg4KCOjo7qz3/+cx0cHIzeHWljfUqRjpJG8YzC2fzIazuV6Srn9upZGf1Ji9NAaU4HewyMMfZSQqcLBsCqxw+C5Lwe+NowSBMMLpa703bPBYPKdCudghJCFv298x1ZMHaAyoVt64RxOIgl2FnmfOZ587dTUZhPpijIEjZoAId95a1a2bfZTaY7mcJxHlkHYHZ0dNR0TbDJYIU8bWcZYG0TDkgGVuuWseWN1ZyT90karK2jtGdapsu2ZdtwytIA6jT/qbYTwFC4GdBkMhndq4Vhc1GM/fj4uC4uLurf//3f69tvv20TMGtJ50XAVrxzZwuGY/hNPxiKUzo7cm9+NBtsT9BZYHY/WYuwM+UeHVoaP06LzDNqZb9WctYBTfEpUpsBeauBUwsf43naiRwImHPPuB1lLRsHg8lkvKePzzwG+jfIW8cG+QT8vJ4ZNoHODMFzsfNk6m5ZMC5k7e0nyK2nn9vb2xZozX6SKdkefL7tEV2RxnuFlYyHY5J4pK75m2saVFIu7jNBt3ctf+6HO/A7x/B3pZA83M7CclQ2K0KBt7e39ezZs5rP5/X+/fv67rvvaj6fj6KgjYTJ0TCCjLhp4E6xUtiMs5eKZZ9ch/4diYnCdsAeezCA0LJAnDJDDgYg16Ac8cy03L/rB5ma+jaXYXh4rBEG6Foh1/DtIq7bGdxwdhe0uTa2Yp0afNySOTmgJVhgN+jBiwHYBKyTYrQBIXWCQ/dYo8/5f+2dS28kx7FGk90zQ3IsSzIM////5YW98MKAF5auLGhIDh93oXuap05HNQV7cw0wAYLd9chHZMQXX0RmVVsGMKUWOziPvY7RfW1q4HA4nN6558UUX8u8NYqgnQl8DLDoi7d/lHnTd+qFjVoHDWZ2SMaBpljsnGmD8ZgR166cCtkrF5+F9K/62oPRIGh/dfXrw9kfP35c33333frLX/6yvvvuu/WnP/3pFDbYe7hzsCwbbQGp8bbrcCg2TWgZk72HGZcNdyoObwxGvq9GagAwy3Q/3Uf33Z8LNrSF/MweWpdzJVY6lNO0viBoam+Zk98xq3ZfHOZ44cM5nSq5c2cNXajf/SqI2bHYWDwXZgEOiV1qZFzrdIZlbWMzU3e9lSP2UB0qW2lOiHMUO83Ok9mwj0+LF2Vivp6+2VaYo69fv24eNXQbnlfLwfpWG6V+Lw7Vplre3InvjYdsUjUtRrhXV1frm2++WT/88MO6vb09S7RPVLX5FXukKtCE8A5PEFZj87W2+aYKywLq9z3hGZiYCIeJbsfA9vz8fGJGsArv8DfT8XinsNVMy4ZTQOc4dcFwzDKRHZTeitMnFuzRJwa8t8Gz8+M5dh02Vo+7gFewtuz9SmzrSxmDWYCLN/p64crg2wfFKXY2ZjNmzHz3j+OYRRroHfVMYWV1vrpPP8oS/bkgx7E6E39H580svf+w7SNT9NvbQJCHo7IJN6ZyEcD4lWoeRYBtGbm/fv26bm5u1t3d3frzn/98mnQbqHdqW1ksOCYTY0G5mv/xCpA91hQ6UTBMK8xa6wzhq3yuE6AsG7m/v9/kGKiX+jz2tX4NKdnrxsqUJxDAKgA6z+LrXTfzsQd4djweC8DqJy0skwIJymemM4V1V1dXmxfbtZ/WJa9eeTO0i6+Z8laVuRkP88h/zvWpBcvn4eHh7PnYRgFrbUMonEOB3xuzDfxOMXz9+vXUnlfn22ZX8ih19naCJiK+tiGrnQtt7AFJgdsryT7ulEV/Y8LO187UC3yXykUA+8c//nEyNjqM0a7166R8/Phx/fTTT+vLly/rj3/840kZ7eH80PZeSGOgApwQkB86xqCdd7Ki2qMjIJLEJLM9Ad0IataAoRlkPJGAkGW01mvins/UzzgBQ7fNmC03K4PHhVI1TMILIl8UB4M0EJu5WsYoejc/mgVYXgZrpwIAGPpZ1tUHiWkbp0cfWmB6z8+vPzbjvJhZkkHFLMSbOf3cp0GsOnw4HDavCuIadGxvocjOhrlHNs/Pz6cXhiIjFlua9GecBRgDEv8bGiJvz3mjoJ7DFnyv27NDtR5h440QAGE/tmSmyb2Mw4sHl0DsIoB9++23J2/FD10g3Lu7u/Xtt9+ul5eX9dNPP60//OEPGwEYYR0KOnHuvAvCcnLbjGytreexEdMOYOGkN39d0auXLrtCyfml74afVo561UkGnSB7Jd8HKDiUxGhRpg8fPpweJSkbwzic34IFA2YGW+cqzH4NPpwz0E45L8bl/I3ns16ctpxIJwStvJGN60OPnBP0mMwkvbXEBsf8deEHQOF6swfvG+N+7x2bmIXZjeVD/82mfA19RLbI36kbg0jrMGiWXDQUtR40ldLIaa11Gr9tD71w6Nm8rOef/9UPh8+XwsiLSXwYC5PmV9r+8Y9/XH/961/X4XBY33///SbXYw9YGs+D2hWY98H4uIVoI2/IZQOz4jtmd66E35FkjF6kuLp6fXeWV9aoz8yhk+K+2vPUs9j4r66uTr/+NOWWYFOM7dOnTycW3HbXOn+cg3s9H82JmL67YMw2ZCutQ07mz2003Pfc1gEA3NRj4Kcvrsf64r5SGhIZ4Gz4ZisO/8zKDWqc2wuvLKfJWUACIAc4oQJN5d0UjEFrDxBhlnaIEAm3zw+0VD9sT9xHKUOu7dTGrV+MpRvZGY/7cKlcvOLTp0+nPV+m2Dc3N+uvf/3r+uabb9Y333xzlhNoOOHOkYdqvskMwIbguqzIa73uzanX8TUomNnY09Ovm2w/fvx4CkNsZL5n8hoO3zr2lhqu2YND4OaQDM4TReezx8p1AC9joY6GyJx3ewDKFKo4LHdux/J3/Yxx8qDul8fVUGWt+bU5bs9sp+crP+TkUN7XA1AtTgtQx2RoXa10P+2Auxo49bFEwKzajsE2VibGdU3VNJrgnoK7AcfOpv21QzTTq3ymhZXJxvbmvOUigPGrM09PT+tf//rX+vz583p4eFh///vf17fffnsWVjgO57+Ny5PC/064QwgKe5QcLnLcbdi4PCmOp3kioIqN0Cx0JsH5No73mb96I+euDORmhOQD19qucKEoZSCuzzuwyzoKSg4zPWfuD69DogBENggzSq/OWQ78L7shlPW1VVyHwp0T55DoT4HGdQK2DpktF4c5Tl9M/TE7p57qO063n9eaf4qv/W44ZWdsuVe3XQo67p9ZbwHQTqv315HQR9qz87A8/Syx+2d5tbi9zsVeuQhgDw8P68uXL+vz58/r+++/X3/729/W169fTz+T1klH6R1SQEv9x7U2WgTZX8eBSUwbal0/15QhOObnGkJhC9dgaoV3LO5zfryI7z7vuh0WOk/Hjmn6YkaK/BiLldcLEU5GN29BHYyF/2U5h8PhlLuxwdrjd75ZffVSOteYVTnt4NykQyKvbpfZFixoz3PSOh2CmbGY1Tp0M2PAqVgHygiIIqy7HCvw0Ac/+tN+m/HCnie2RqhXtsNxrrG+23nWGR0Oh9PjggU56xfycbrHoArDwxYdLvrtKXZCDnvt5CxT17VXLubADofDur29XT///PP6xz/+cXp3VxWbCXLsa+/OQLx07pASASKwtbZgRB1G/HoZBHh3d7dhQ1NYAViRB3CeZ0oIO7RzQtJ5m4YxZlLOQ5WRuu/8d1/7Hvvj8XhaDeM89XmBhL7YoPZydlamJlwBF/JzBUnaBSTIpZQJNbShbsvBzsey874sh2722AZEz6HD9mmcFPTNKYvqDTI08GK0AI+BkPEcDq8rmICVbYQxeYXOMpmYEfbmX+uybCfmBDh5bH6/nvvH34cPH07v2ysDo+/0Fz13xGNmZ330HE862cWmvXIRwD58+LB+/PHH9fj4uL777rtNoxYIAqBBTxDK4ESgO9yQwArZ857YxtcTA5rQ2wZTmlvmw7HmnNZaG+PqBPhejLQg0jbMemz03e1uwHJbVgobh412yrN0XD6HJ3x6en2+j+MN9Sh2YjA05t+JXYfPnRvaNbvZA5S9fl8a11TK3Pfqdjjr9AcGXOdtgKpOV9ena/fGa7B2qX51jmsXdtpmTFwL8E56NDknz59ttDpf+7U8PJYea7kYQv7444/reDyengNzGGOjpDHK9GMedLoewANqZ/GcZi1OUCMUwhkL5q3XCpvdNVwx4+J7aX8ZYz0mfWm46j01ZiDOlxAaNxRp/+sRnXeZktc4ACsa9+Apva/LckL2ZrcOzyjMcQG+OSyU3HPmduvxJ51xqsL/fZ3rmLx9wYP5Zqz8IS/3m+LFmDpN11m22ZwfIZ03RXsbRENsMyV0lDHzv/k7y8jhq2XpebFOUI/H0NXr5rysH0Q+U92eL4e9/xGAGSzcKA07P2FFIJ/iAZh52cBtvBUewrKQYAMcJ2S8vb3dxPZVbo4XFGyoZoEFZwNVQ1uXCn9SaO53uOaVTY5ZfpORNy83rRBNsjWVL2ibSVR+/l4PSzGrcIjk41ZU19W+2inVMXjck1Na63XHvT1/rzeLsEyaZ6VfdcAG6b0QDtn6mBmcjz08PKzr6+tNisW6uRdW7bFry88yK0P3anHZ2pSGsTyoz/ZkrGiUZjBuntnXu6298uY2Cq/cuEEaajjRifIgzDZ8PwOavMUkfHbWH4/H9f33359+yZh7DofXpOZaW6ZjoPCD0BVWP08Mi/asLChClXMK+cwI8MaTcthT8rlh6WTcBR2zKDuGyYnUY3fcgG6va/rAhtLQznktK7Hfx9+0AnIrmPg4ZW+VtHnOfneyf62to7Axk4dy/XbYnnsbpgGp8u0LDw3Cnp+WttGogTptU9WZgq51og57AhbLcMrBTazROUTnxn9rDuwigH358mWzyuaVLgvUeaA9L2Qq7fMeFAZmwRbQDofXl9Q9P//6uuq7u7sNcL68bF/J4nzL8Xg8bb5sItXeuJPrUsUoA8GjcW6SWxPsZhxO+ttJwGIdykyMqDKnHxil5eG63Meyk8lwzKb2PCXy8L19ENpzYxmVxXZ7iGWIbhWYzFoculO8Sj0ZqQ2+DNlhuPXYc17Zls04r0TkQO7QDr6pg4az1YcyMsuz/UD3GnVQzwTmHUdD/AkUC3CdMzNz5NA6Wt4MIe39PUiMFGNzktaggOCsaKWKXmK+RPcR1MT8XE+Fa6ZlwdBn2tzztHvKgPG5b80fWGmq5O4rSt/rKd2qYCVGwVBEzwfj8Zwyf76f9ixXOxO3zf0dH8eb4DeITPV5Pp6fnzevYi5TLIjwfwpnbXjug4GQ/9UZ2nt+3v6Wgs+1DTsG7jVTRF/sqK1/3j40PQZl/ZpAxzbFVhuP2zk3b+e4uro6vU7dkRBs2KwY+XY7CNcbdMoaDcRTLpXPnsvJMbtcBLAqppW1MSwTWa/B5LUTRt8pgYhQbIgWQnNG7i9gwGfvFq8xN4yjHnvj5lZ6jfNaFMbiSTDQTJPi8ND3mqU45K6D4Lyva8hvj884zD4tR89tPSr1dQ44ZmfAfQU7jmMwzJsfnrdM+e4+rPX6sHbHMcnBMio7tJx9//Q2VtpvrmnarFuWhGxpk747irERoxcN+8pu7HC6lYG2KNOYHKLTRn8xioKtey4Ae8bUaIxrHCHZNjqvv6W8mQNrKAWi29O40y8vL6e3T1gwXplk4FxnJSrlR3goYA3DA29Yy2ezijKZadWL7wYI57p8rgZZwPZ9LgZGxmggKjjZcXg+PCazsYZL9o7ID8NoCGSw53jzS4+PjxvgMKOcHNb0iI1lwbhgMdN1nbuGlHZclj36Wjbk6xymc34K/dtvZLxnfJWn+0qbzfc4XWA9a/rFc2vG5/HVoVHaZlf2qz914NZP+uDtLiYvXs21Hhvk7Fzdj0vh41q/IQeG4OxBERgNwnBMiz1oMxR7I0+uQcFJdgupBu4cxSTktbb5JNdRAVVQza/0HntL+uP7NkI+nO/XoU3X7wktGywDKlAYvA2ME9uz56RvZioTC2yo7Nft2NDKmmirBuo59V83SdPHSXaWkZ2dZehtK44M9sJqy695nRbAcZo3MyeKV4k5NzEy67b1zNfX2XIvbSN3Mzzf493+raORhP+XxVqGXrmdoiXbVNkZfZiii0sg9mYOrBOJ4ImZ3Ygnn9DQnp3PFij3mErSjgXhYiWwoGzkZlxmD845VAmsRM3j2JszgZaLwyDadVjtPIRZjRW/IYblNlFy8hMA/UTzGVsZm8dig6EvHfe0RQOG4mP0c1oi9zzacTgcKZC7717qd5887wWNGmTDKue/7HzcTnOTrq/Odm+jse3CfZkchGXq/jPWPVbo42WxE+B4PhoON53A/OEUCjTth3WX+j2v1U3rh22z89dycSe+B1eAmVaAGJATlVPYZ4Uzi/KA7Tmpr0Kirg7QYMS1ZUjT4y7+7MmdlqS530pfRoXyT32n365nz4NyzPU2zKjX7FaAyqH5P8vU4+48I1+H7VOYYgcw3W/5+76Hh4czVtMUwVTnNEY7yrJQ2vAWChu9HXL1sMddT/tgOVJwOk9PT2ePAFHQGzOaMq0WRwa9pt+tq5ciGDsS6rE8DNjTOCbbeXp63cvp+TFWOFKaohfKRQZGpV4dMYtyxz2BMC+EM4UL9iau0/VyvxXFhlxPu7dC1clrIr3G5kmawiEbrRWNsbsOvwWVMZh5YjQGbi9QHI/HzSqYQa3hQdnIWmvz8K8ZcpXNMjKT86Zh582sF9RnJ9FQ18zMrMYOEabqlSyz1BoJ7RI2Wq5cb7aAXM1+JydZGRcUDHgTmLROL4Ywx330jTnu86G2na6GToyO4+7HBCzI3mSjzLKAaDaG3MviK7+JwfEKK/pRG7ccL4WPa70BYAzCm+tKLe1NrUj2kFVeT75DGbOXqVhoFvolJZpocUMCU1cb+Vrbd471Ouo3IDhEpY49EGr73Ybiayt31+VxVX5N8Pc+Aynj8XzbKfQ3DL1pd8+TO3GNAtsouc86YEAsQ7RTRD8NUp6bPfbLdV7ocR6wrNKsiz4i248fP2767DJtfeG45Y1s19r+SEv77felce8l3S8rX+scIBhXF2nsWDtuz7NxgPr2cOLq6mrzm2CwfwgAACAASURBVA1llJZRdWWvXAQwlLV7O6qANzc3G0rKtf5so2ag/iEP2rMioKRmSJNSesBWPJ9DUA69LBzGZa/q82V3zZFV0EyGf+CE4wWJtbZPOhjcLXvLyfmfOgfaMbvtSiP9hqEZ6Ch9HpN6+96wMtg6Lzs3xuo59pyZ7ft4AYD+W4Zl1HaoAJ11BBC2Xq51/mMvlDry4/F1JzkydD11Gvywh/ve8pZDtkzsLE0ELANK9225Pf57vuoEK4+JIbLZvZGKQYn9aWbW1k/30RiyV94EMJ5Gd2f76lnv/6Iz9pYoLIPkWUm2UZSN2EC8QsS1puBmEA1P6F83zTWc8zgMmPawBgCHNg1vy5b6Opy1tmGiJ9chJcvOBnmDJMyG/tDG9fX1Cfhgq/TXzohxfv369ex1NfTFbyYo6CIvzy3X+e0dXOdnZ53gb6jZbRBNLaBnBc6J1ZlBui7/qC19Nfg8Pv76y1b9RSkzFoMifed+vyLKYDSxiY7XDquFOek8mcU5vN+bKztg98k26jYnZl1nUCD1PBkU/dOMnaduE/kt5SKA+VUaFD9cSseZqKKnabLZlCccz2BP6boBKwvRIcda2zcv1IOZHXV7hxmj8xZepfNPtNOW++G6zPB6z8T4KNxLfYzXeSZkipx83vLiLboAnPtrZfYL+DxPfrcX8kBZ+3P1fc2N81h2dtRtZ2RZ0A/PBfXYk0/bCKawjT7x1tjWxXcnoFlVt5ECdBgq9sC+RHTeuS6Pyey6rIZiVjaFoRz386tNj1gmdSjWLW9zsDPxPNFej9fx21H5PuTW0ldoW2cLkpzfY6mbNi+dbGLTwrAHMJOgU/w3qJWt2TOsdU5Vnadg8M/P5y9va9tVJnvZvXyS+8NnlHPvvj3QbDjtHI1ZJH3HkxsEC5S+3214lQewYazUO/UZGZbheENtnZT1wvPr9j0mh+oOAytHK/S0g5xrplXsgqHlbUO1HtiRNQltQ7Ie+D6f93fPR59l3CsTqHmMfN7bCMz4C35NpZTFTzo6tQ/wTfZmkOlrcezYO+dlbVw3yWkCdJc3VyHpjMMVH/eklZG4837PlDvafArXWBhWqks0s6xure3PaTXXwT1up4qAMjo8dPwPYDJWe+a11ikEbXvUZeMqIFDMfJqDdCnD8PUONQE6wMJjsPctyF1fX2/m1zrh+XBYaM/vflunpvGY2RTQyqgN7jYk+ri3HWTP6fT9V5U7dZYJ2fEQJk/XeN73AMz9mhZSPOaClR0xIW0Bxvc75HS7vt6g48Jxp5U8567LfWVurBt7dV8qb+bAyAd0lYLJdl6gDMGfPUAfa36q7MMoXW/WUBIh2IvzuTkr6rNnMPhUmJOicKyMwsZZFrqXeyi49j4bz941LlaMCWRgenUgZVoGoz5cjUypv84BuXSryMvLax6qelKlrbFbpgUH6mAcDZMcRvt65FsdJzqYDHHqq68xg7MsvD2Gdi+V5k+nPFPlwPnmxihmm/6b2sPG7Oin4jpcig+WkRl62ZjHe5HB7p5Zvxo/Pz9WxSwL8+DKIq6uXld7+I5HtyL0/UpdWeOzw4m9nJTL3d3dRmAUG0EBpb/gXVC1kpSK872Jatrkf/uLnCqvidX6uxWyoNHwHEP1vDmBT8LdjIF6qLu/Y+Ck7NRGHYd/h2BSzuZfqtiWuVmW5QKTYkyHw+vzuM7fOL826UW3oXi+uHdPdl595tqJVTrMNwO0zKs/7u9UDPKTTUws0GBveyg7m+RkIuKFCMbch7f3HJCJxyUHfRrHpZM01LyIcwtuwINnUrmf1S4LyUlFjjuHQz1uo/mYtbZeo4nNtdbpl1c6md506k15DsUYj8MKlHZSIiuOH/GxIgN+lsHV1dUpVDSt93i6D8iyJNyzLLln2luEfBqu853//IAu/QOc6ly4BnbVpHm96sSeuQbwNrvzfPYZWesQMvb8sjLrpP7EFgDCSTYOs7mGH6qZdAqdaW4JMG10Qurj6up1XyWsl754fmobE0gVAP3Zea1pjsokret8t+PmHX1rvaZNPD9uH92cFtXAGr/V+VIoeRHAbm9vTwZA5YBMJ7cxsr2xJ9SejEFMq1Ze/rdgUQqv3vjBVPqGorHMTX8dXlL/xHT63jDH6f6jnw6T8LLIxbvhnT+xMU1bEuh/vVwZyfF43LBk5+rs9f2TdZQafdkkbMvhvsHHc9OQkLbsfS1fh3o4RetP86VmZjagMrkyvpeXl837rmA5ZtNuwwzKuoGjo46yd37hvWzTzJjcYyMJM9g+gO5ramvUwWeztzrsyoT+0I5JR4HS+juFe9UD9xubtB0B3A7HwRYvWFgfp3IRwPg5JQMQHakHdsc6IM7jUcwyzOYQpBXDObG1XnMxZgr1gAYee2P3Za1XhaMe5zwMtGw5sOewsjBpnggnuD0e52W8x84ASB3c65Wy5+fnTSL3cDhsDNpeDmVDXgDSFOI0IW/52UjKiNwmCvfp06eNfBiPGQNAzjyg8P6l9OZpOO6Q1w7CoRCO7unpaX3+/Hnd3Nxs3uTLvbRNMWABJGWHlgPhEfPQcKwbSJ+fn096Z2NFr61jDk2ttwYQyxbHYH33WD0+gz91EDW4Ho8DZzQ5F87brtEzyIb7gHwpJgjItLsSpvJmmt+VoPAOK22kptd0iuNlYw7LMD6MZPrhViaD6znnbQ8os4VVZmGva+ZGX703iL9uozA4TOyhQOV+cN6gyPX02wrpMTLmtc73V9GHbhVYaxtydZOwjdOGYmZoQJ8ofWVsWdnT2/A9Zr8/zszMYTwhm0HQ7WE46Cgs53g8rh9//HH985//XLe3t6dxlE1Tr1lsnQlzYVCzgwHEpmJGZaDyvNaoucft8t3tVr8859YFxs35RgPoAPKnXhyB2ZbtmmLbdv6yDNEyrtPEPtufvXIRwEy96fhESTtgT6xpqY3A9zfR7PDQgnNoaO9hgXh3vwse2cKZkqSTcRZ8uM/9mibKoGxmh5GZIaHYNVozSHtjezOz4+PxeHoRZZP1DqU5byC1ogIkzH/Dj8qXOjtWgIfzXOMwweyvim8n6DZxVg5xaJNcF/2G7TRPiDzr7CzfghjzXYZydXV1prcGxbISM0izN3SjUYeLHU1DSved/tqJNjrxOTsGg6PrsRO3zprceHyNfCbGjy5aZ92vRnMuv2kbRVdO1vo1MQ49RtGKpDU2rjH9ZpIdFnKtwc6DN5szS7IHoJiqe/IclrmP7r/zFo79qcftMCY/fuL43srh1bGCqJWdcUPPCQH7JgfL1izAAGk257wd84FRGtxfXs4fhUJRG7JMyjflNOijQ2b6j6yRAy/K5K8M0IZs1gj42tEwBjtP98HynIDGeUzrCGC51utTKt535XbszMr0mltCRtVBlwK7++eoxjri68qe3E+DZu2J/wZgZO8IyDbEPSYdDc2ZA4esHXPLxfeBsf2goZsT4RQPBiM0k3BohKD8GluM2gZv0LTAPPkevENEg1YN1WOheHJtCABCvRDnDAo2BhuS8yAGT9iNjRTDoyALOwk7AcsDmZiWmxljKPQBGWBsfvYR+fkXuWF4HgPAZ2CxV23Y6D7x2QzR5xxueLx2apZFAbIhfB2fZd18jnWMNg3Ma50/MmMdmxhU7aCs0/PonKb1rcXjsQ5QvFnZulAZUL/TQ3YUsHXnX0tqug2q+mlWankbJ9Y63+t4qbz5QkOzDoPJWtu9JPWmZUP27s6fuSBMK33ZX0EMwdCv5nmaRC1QcX+9V0OIGiHny4Q6FhtW8xbI1OzB8kOZDofDSXFsTPbOTgY7lMWLM4YC95SToJ6mA8yAu31gYpsuNVo+ex4NJpzzPPgheMuWfnoLA8cmD24H52NlJHXKHDcjbI4GR2FwhMFZdz0XLHq4Heu7++Sow23uFeusv6N7yKrpFSIE63v11UBzOBzW/f39ZtXa7Oru7u4U8pvIID+Hw43kLpWLZz98+LBubm7Wly9fNsaFAfVXu6d3IxEj+w0RNlYrbWP/siCK43XXh3DZ90X7fb7On80cqNssA4Vx2NjcFMWTYZBxQrmsoGyCOjiHTL3sj9yYcO8PQwG658lhpdnTSRECRgXmhrs17LZjkGELBrIDPKZnWs06uqG2/WqO1dczN8iXvKCdosf9WzaPMmcY/f39/UYXbIANj5ojcptOb7i0H+jyW6zEBQdOv7xa6GLArK73GjsG5urp6WljdwY4Ox87yebHqo+186lcBLDj8bgeHh42+7esUH2+yiEgwvBDyl2CtbAQbr1yWUOpq/uEInSljTapH0E6X7PW9hd8LHDawUCnnECZKIZm5sR5A0iBwkn2MjJkaplYVnYSTbg6/0BxmGLwNEg3j+E+INcyaUDSgGPdYPuImWKNxQzK/aFN65BXrf3iSL/+xzpRBt/8U0Mo7uHapiMKuDicygzgRu/ok1eJkVGd6ltMhOsoyNL5tbXmHfy1A9vWlNZwPbRjuZM3tY0hn7JB5+rc7t5q7lnfL51sjqRxqT2KhexEr72jPc7UaUon38o9xeb1DBYS/eBJAOq3QgI6HksVaZrIKRdnsDAld+wPQ1rr/Pk6j6vH3CcSxRiT81fur/Nz9MsyNCigYAAk40FO1NtC35pbY8yVoefPQIaSc41/is/Xo0MFRK6pPK2bBi8DTn/YlfvKAroFAt3hz47OfQH4Oo7j8fwxGwPO8XjcsHDniaw/jI3/ZpReRKk8+O46LSfmhxVWxtEUisNa2L/n39c7PWA9tA5Q/iMGttb5b0OatpvyuxMYl5eazUo4Nikb1zKpZgW0wdK/DdN9mMKAThh98YrVtDpJcfxvA61ReYKt4A4rME7LzAppgLVym7XWsMp+aZt+YFBcx/iow8zS/TETNJjVIxt8zJocUrm+T58+na6nbb++m6SxAYI2+0iT5eMcTdmQHS7gZefscAjZOSrAcVim1GWb8AKP5eY8oOVvmzJrLytFv9x2HQr1tM2ySesJc8p3629lzxi9d6+65OKneAzgjTAY68vLy7q/v9+w0slpnsaye+b/BnV3d7cxtjIeGzRMoMbkibThXV9fnz3aw+fb29u11mtYZGBw7sRhYduZjMyCdd6EiXY9tAFbMt113c11ICsfLyNoGGmHgKLa+Ax+ZpmeeI/dLIExmQUbKBlXc5pVSMsXj0wfDYz8Nfz3OO/u7jb5M8CEuT0cDqccEwVdYww3Nzcbh2iQdyF35HlkHvjzKl8XX3wOkGEOOe/cX0Gkq5IGLZc6RIf1nLdsqXuKClxH9cHXOP3iJzq6E74gwhMTHoN1BvZMXqxpAsa6xyD3QLzlzUeJLCB7IJTMgLbW9j1Ips2m13SU/Npa56/xcNK5oGW2Yg/enFXjfRuMAaAeqWyQB5oZH/1jTGYxjBHQclK+rMSTaBBCbmZtFFPwssHTpAaoud8JYO6vo6k39OcqmwGr7U8Mzn1rMtmysbHRBzMl6uTRIOvVFLajw87Beq4Yu+eIPgGYjBX2Y6dH8Zi76GIdQ6ctFzOdhpBmxm6rYbOLdd06yb2OKCiOZqpnk1Nw2NgIB/ZqEK9TYFy0B3aY9V5iX2u9AWCfP3/exPwvLy8nZIYpOG9idG3IMHUEBbMHc7GBMrFOhOIp3G7zdVYGA9Nar4zKntgTYgEzsXhxvlcpmxdrzoD2620twz3wMGC3X5Y3/2tgXcaeZFIgMvugPx07MvTKo8MQAwLMt+2ZHTgMtGJ7U/Wka5ORUTBOA1bDKHTC4OzFDy8WTBtLXYeBvU6eaww8lqW3qFh3Pd/UWQfWPFYZD/d4Li0f2rQTsEN2vSY37SfHTGA6Bjs6zjm8tpPdKxcB7OHhYd3e3m5yGg4XrGCT0FBGo7iRldUKT15XN6pcKLEfEO1kkyMDAEuDnbuoB+mKTb1cmZHDLrNOew8zFMbonBT1OhR3W2YZrtN5BYOiv7s0Ie9+c75eEofhuaH/niOcB/NPv82k0B+DteVvZfVY8eTe5W0jpM/MffWnc8bnKZ9Y3cZQWQR6fp5/TYs+Wx8re4NaWatLF7dqwA77qvveHG5nYHD3Pe6P9RFd8Dx7LLRf4lGnVRCanj/mOrO+5tr3ypv7wL58+XL6jgelkwYjhGGFpZOeWJ+fdlqjHCjiL7/8chamUF/bQhBNEjopbgXrfTCOeh8L0+xtrbVhBM5XTYoHSHllacqFFMg8rq5s2UDdB8+Zw2vXzTEzRzOPhrqWm0MCwLRhrUHMYIOcO0aDgoGmRlJQnEIh5q91u88O2bh+j1VNoRx9c/jaHNkUPptJ2oC5vuzMbZbldqwGJ+dZm/fiOvSlzJvx2c6pz3npCVwMOg5bp9XU9pvjDfH3ykUA+/r16+YNABaWcwAYiemtBVwUrofjOLkJKwbJQrOdhkkUMwQKCxEI1gpWz4TgTfmd8DXD8TgwSIOm7zdAoHywT8ba8Md5INNt/3Vl1MzHygfgkMSnXoMd57oIYVkY0Kmf663ULp5n+ugFE2Ti8Rv40A0/aO6xWd6txw6WYl1GhgZEJ9y5zoBinfVjNTZYjN1bKRyKOTeMrD1uO1zrLH2mn58+fToBGvdVh1z/pI+T7pUdVufN1DjGGC1n6kIWyMvJezNgM7v7+/v18PCwCdun8uY+ML9Pu3GqQ0uooVfxJm9mUHOd1MGgJ1rPRHmZ2spY8DC6l8F11zbFK23uM/fZW3OdvShK20l2WOjPjKUvPbSc+fMKEePxRtnn5+fNw9dQfJSmISeMzYDunAR9gRGbeTBWr2p6DrluUmpkwL2kA1BigwufnW+tPLxQUsDwf+dvPUbGwmfk31yfdcC6YGbn+W2ivX2knzgT+jAlyG3k1gnrqqMjz93EzCkFn+Zyy3B9vfchIjvaM+ADrtRjZuWw0fp1c3OzPn36dLZxvOUigHUPGOXh4eHM0JvbMNIaaBp2Gmx4uZw9EApnquuJswE1H4By16M5Du9zkwZgezVPCqUAOHkirpvA2Nc6H9h2AG4MuS/w88Zgv0qHAqj4wWyucR7D/XeI5hDSDqMG7L1VZl0FNebAK81VUnJe7pPDVO6zc7m6ujr9WC2g5M/tVwHW4apBws6SvqFX1jXPhcdqtmjgY14Zm+fezLvX9jvsGjkaSCbbmRwr7fp3A3yea5xWsLN07tf39Zjnrjhwc3NzCjXdh6ZiXC4+zG2jM7U2sjtnwufmuFxPB+cwwMli7reAS9PdT4cCnRSUl6fp2W/kxx+4zuN0TsxxO2OhLdPyghf1TJ9rtPbOAIEn3PkPG5vH6vpr4L4fIOuPl6BMgCwKjcwbcrs9jLDHfa/ZjUM+OyD6PrG5AhZjNvt3mOz6uM+sv/qHLOsMDeqMg+011If+2tl5FbNzzXnLrVs2anv0hbEaVFwMnhOL8Wp+iYRLWbMjKDND5qo44JC795e5equU5X2pXGRgKI0ZQPMMZQ0OG00vqacsDMNyGFZKabbX0Mz1wLSa85jqKwBQphDEuRjXyxgdqlpuNgKO7YU/tM1TDPawbtthSBOw9dIei5kUfeJXoRzKmn1aKQ1ATgE01GH8E+Ps3JiBGZgMEmWCDekYL2G4AalOrvpiXUS3qcvMYmK6XOMopdHDXgrF8ui82WYKOmaqZeqeD64F4LzlY3pG08U6ZRm4T46CKCUOZfHWdVISBjfnVanP5GevXAQw6LdDQ3sTe4IirZGY3IsH4AH63oldeWe6laIAxzErko0Yg/QO7z0B2QgvleYjyqI8Dht6wYY+G+CYQIdw/HeyGBmgCBz3UxE2VMvDzMwMhOfZzFy6i98LA83/+B6zMivzWuuU3DUomqFXoa3U/o987cn9rCPzUUPqsaYuDKzU8fLycnr+kvvQUWzGTLdMnfqp13Uz9w3b+d4FKutYgaOLPGZ7Zre2Hz9j6tw3xUyR7567tV5TMRwjrPfbanhZpR2S533q21Te3EbhpKFfDWzF6hsXrGgYgUHLSo6C45HYXW3l4npyIrRj4ymIOBdQj+a+18tRyhKneNwMteMpuPp9UFPo6H67LgqyazK0St361toai+W61mvYx1zZ4MwybfRVWMu5SXIDt5nslAeiXTN1xj5tpWBMDTsArj4ZYYMqw+Z7V5OptxEGsrI+2A56Pdf1aQjydE1RUA/X2Plwnn4b+N1vszz6UACyfhhIaM+AWkLQvpqIoO9d2Tbb5lrrpHXT9+6ViwB2d3e3of5O2hVkakgWlnNEVui1Xn81mEGZwjt8MiBSCHNQAAvVG/oQvHfptx/0efqMVzLbahhoZXOh3W4B8WKBx+OwyquGZrIFKAzRytu8S0MzyxIAgXlZRtzrl+61WHkrO/piR1I5o6gOjwvmXrhwyG3m1Tn1WLxianZL/ydHZ+ftbSYNP9d6dTZmxma3Lnz3LyJ5PxmAbWB3HmwK31wvfZ/kSJ3WM/e784Y8GCvjNFsvuNop2ZFbJxvGuz2z7UnfNmO/dPLz588bT2YjomGU1+GdJ89MoejrlSCUzzkFztP+4+Pjenh42EzklBOwUgNy1AMjnN5x7omeXnHi3IfvaRh4Eu5h+2iTx+8QoYzHit9QlnG4Hj8EDWCa8Xk3Pe1abg5tPT8Oh71S2RzOBPwNAwqmVlzrQEM1PrfP1r0yfifTJ3CfwrbJMRQ0qd8A6nzqxMpYjKAfZlrMj3WLes0KK0/Ph8Nz7rXMab+huHNkHjPF1zsU7mJCwcegbn028LWt6kSd9l6UtNZvYGB+75RXBZunacM+ZkUz+JlyMmBi47VeQ0a8w+Hw668s1xsgGCbFynA4bJ9fLG11KYVnLPTf46WNMi5/t2GgLJafJxuP6WRmcxB2ENRhT0Vd3ovGd2+hqLJ7u4IZtUPWysqsx3pBOwYbrllrnSnpWtul+4b3BiGHsa7b6QmHqwYk7kUufKbYIXtsyI++ePWPdjo/U8hMG762Ty8gAwNs62Dcnn+PsZ9h1Q7PTEbqkD2/FOtGWZ774nunaAT7czTg+iewtsOYyps5MH7N2Rvt7B1p2KBg5mAPUfA6dUIMjAHY6zZnYGVtLI7xeuWE/tpw7cE8jk6Sx+d2bUQWPu05BOZ/PbQfpyFp7tCh9Zb5NkHOnFFM5afwn/oxarPAblB1OoB7fL8dA7L/+vXrhv3Wk3osOBo7DDsOy7HADmunHsuO/jjE47wf57JeeqXVBm9HTJ1mG37pn/Wp80gBlKcQzIWtLh6bx2oW19VGZOaFMNuoichUOm/9IR7PjW284T5bTuygGwXUUb4VPq71BoA5BIGNWXj1EighhultDR4YddcjudNWuoYODS9coKv9xZi1tquqrdvjbf7Inq2exjIwkNSrUE8T/ihQV84ccnLcCWHXD1ABFBOIwYxsAO6fV40bunEdhgIoINPKn3o+fvx4yvMAPPS97Pvl5eX0kjyzBwOAZWEQ9f+9BYuJ+bltO9A6jeaRkF+dOP/dzocPH05vezXL5j/gZBCwnVgO1j8DrOfMqRqu92pkS8HS+tuQ1+cdiVAaFVmG1nXPf+fR4LcHqpv+Xjx5eE0wAgoYhEM2OgnSN5a3EjkJaq9KZ/uzTGttPZnDUIqNCRZmD2Sw9QqVDZRzjMlssHktg+20SlLPYS9dwG6dKAXjJN8Fs7TReZWt3oyCQnvu2g4yNmNDPg4vWwyyzKVlX6Cply5rQv715DUi999MtMwLuVZursMyrnEWtD23nLfz4DrbAMBMXZZnGT8A1y0OXGOHxnwa+HBQHqMZlO2WNprMt6wa1k/zX8AuY5vsyW+LcbTiOTRgT+yV8uZGVocGDLCg5UloHGumZcS2J17rlV15MyLC9iAueSjTca/YWcBmOhaMPcTk+U8CO5yvjPJ5mugqq8dmcDR7sJw476V/s6auqNFHxuI8IvWZFft6O4kqXsfesAPw8jWPj4+nt5kwJ14McHvIxh7ax6uHNhzAlrm1jKmzaQT6TN/o9wSU7o/zigY+rvOqs/vteXMqhjY4z2onxYtJzS2amRk0kY/1x8VM3KkCF2xsz3kV1D9+/HjSNeuFox6cDfswyxT9ea1Xnb/ExC4CmL1Mjc2hjge6R6VrZAjBnTMdNr00g6I0D9WJssJ44jleYGsoNLEoC5PvDX33hG3vWCB3m80D1BvaaXQ1FplNjxhNTx0gDxfnDj0PBQ/YkoEFcLTx4pCcMG744/bqMGAbBibLwWNof8xQLM86WeQzsSzkW9bvUNzGx3evhvKZuuir57D64XYKuh6TQXgKKSlu1/dNEYQZW2Xl+5hXnAebe2s71eGu4ttWPf9OB+yVN3fiUwEob4G4YRp04r3epx0zrcfDGIyo217InqYI7lyLldrGX6W2R3OxkL2NxJMw0dtpRceTY1ZRBmPA5j9tVdHsQQ0MDntoE2brxQvanpbRkaGvqTcug+a7XzHjYs/qOpCvdaOAY8O2zrSu6VxlbpnYEdiwzb691cXjBcSdHOde2jTIuI7WZblMDrB6Qvuuv0DYiKD6bR2ZjlU/kGttt86t4G+2b7xgAcBza303Ifi3AYyJhSIaXRtKIdB6hA6gRmADcOhjA7+9vT1de319fQJTG+MEqIyBvqGQNnRKFWcKL1sv/Z48les0QJ8EL0V03qlKVDCzzDlnhuNQbFJs5FiQs3zsJflshZzGwr3ex9e+eN7RFXtgs0UfL2ujzQnUm5cBYMqeXbfnwec4b+fthLhBgu/ThmoDq2VjJu552AO0guvkdL3laALDaasF3wvyLy8vp0WYMiXrhMHJxMXzaUBEnmX4nQfGvBfGrvUGgP3ud79b9/f34yoIAqhXKrX3OXdoz6NaOWjny5cvJ6X3A8ZlUS71ANxjwy8rsPIBCH6cYfJabW9iVfUqa72+PscLD1xr6tzJ5HhDUOTakNxjXWttmNges6J/yKwAOYW4Xixwn8xQuJ76zLyYX89B9cRzaUApmBbkpygAOZfpWod9bUOv6jWy6kZNA+0U8lonCpq0a/nZKdKW27NTn3S1DGsvIkhsWgAAGKRJREFU9UG7Zu2U6ovr8LYK53HtKBwtMDfe8G1deYuBXXydzsPDw/r9739/VgHe04M20yl6Fl2N3vWsFgoo7p/DmrxOVz782V7Syuqx+Np6v0ve0McmhdhjCZyv4dmImFzLyvKhPc6bLSM/hz9W/GmrxlrbFSQUkHN9tY/nDXBzbpR58OqWgal7qLgewCs7rwFP17hdrumik+fSoRZj8TXMT1Mak+zMoB1NuFQX7FjsHLzi6HsMeG53Cvc8/2a37UPnoCkFAxTFc97jlmHnraH4lBqwA2K8U57uNM7dM2udnn/zRBpluxWAxvGipomTd3MIxCALgkZmA4o9vz2zz7n0gVgLjf7vhTpTnQYcvndRwpNW5afebtHAs045jbZvZaCPZUdWUBTZsnC4zkoS11O4pkZQ42POGSO601SD++B69hZsrDN1Tvb4PdZxOFTmfx2g22BurTOAjg3NjtltGYiYL8977/Gc0I639hjwrENTmGUwn5xoc6G9j7rd/6ltsyTGbIdwOLw+CeM59BxZZhTqRY/2ypuPEpnymcabuje08CTXA1ix8I4NQRlckdz9mDaiToKg8EiUBTddN1F6X4vCeTvAWtvd/g4Zuc+yqud3/WYYlC6euH9uFyqO8ninPoZtI5oWDPyDohSU2Abt7ygqIaDl6LH213walpUtOkVhgOF/xzexiQIx8rIRNfy2c0Rn7MSbU0OG1OuIwjkzitka381IcIYlAZPuTKGsy8ReAEbfNwE+xYzL/ZjsqCuH1qnJtqb0heW0l7I5XXPppGmsN1KSvzEj8wS4UYTv5DmT7Guc+zJY1cDXOk+qlyWZCdlg6H/3gK31yg4IYfaKvSD30YfSZHuPvdC19L+TjKIZPAxI1GHDpJ6yM+q0ArvNhoCcm2i8x+U2DKYoYENbA1eBoMzCsrGxOn9S5mlGBADUWVqWBoOuLvupEsvZ7fEDzTZ05/cIvw20Dq3dXnW5Ye8UlvaY62mp02jO0fW6z2VcjMFzYdZVNjWxMzv8yUExl5fKm9so7C35IxFccKoH8ncbwQReKEZDTX9mQPaQZif01b/4jacFdL31oobiejk+0X8rB3WaXtcYprIXErr+PYbg/lghefaQjZQOk2CgHZeNAiOctrFcX19vnJBDHT/szH185/lAhyyACWNyiGbH0+Q/pbrnzZ/NGZrp0187NB59Q+bIb61X1o5umm3Tb5yB+1L98Hx7Pm1bhNTeumTmyfzsAVML7WGrlh3nbZNluXWGlqfBnOL57yuyPFetj2LnR32/ZawXr+gv0ZhVOIxgAv1nRXE8vWlcwEFdDjVtaBUQ9Tks3EPtsojuJUMZvaeK66tULfbua23zGPWeBYz2sbmYtc4TvV097SpaVza5hlU/h1VmjO6fw3gn1b1r2z8L11yRAez6+nojZ4NfQ6CmBszs7RCdmuB/tzkU5Bs+W//o2wR8vs7MznU6UnB4yBh4vIq+lEH7cR76AIOzTD3vAKfHM+nV/f392eNgZf+uty85wLbdNy+s2YaRWR2rSYhtvGTHerDWq51fYmEXAaw/2UVhYkypXerdO8kON82GUI7SSNNTe76pXQvGx3ocYdfIrVj0008g7HmFerm9cKhsjsK19vK+jn57JcoKWOdgBuh6ylDMJrx4gOJyn3N+VmSDlWVAu31/m8Mq60DZzcRUaL8pCNqysXN8Wi1riI9hUY/l4Otoo2M1S/dbP5oj9pw5vPb1zWlNoST3d8uGWTbFOVHLq8WAZfbDvONcDcxrne9zaxvd02fb8Bxzv53C9AzsWb8vnTTqlz3AWLhuSt75R3E517jZwuqydMNIBOwEuZWfe2zkExuCWZohOtyi/zYWAyf3+Pv0BgiX5qscilGs5BOQWc6nCTwcNkrniXffDehWJOrgPudrOMe1ZsMOx71HzAyIvju8qyE1Z+Kxmg26vwVQ6iozsIFYV60vdlrTc4P9rVLkbx3rXFV+Hq9ZE3oJgzeY2d6sM+hddYexmUFNNrvngPdA3uyTMVAc5vdRsTpjs3rq8zW0aec0LUCc9fvNK/6vmD0xkaWNDIrr7LV7nwHCCuk2LEhAwStCbsv1lGpzvnR3rW1S3AKdAKMMw+Oenj+s3KjX/ajyc02T9h5vix+YtVcm/LNhtK9eLJm2Nrh/VS7nLQy0Dg/tZLxS5/9m4mX69HHP8OoErV/U4e8AroG0jNbfy5i9IDGxQDvGyZH14WY+ozMlAvRnYio18LbV8ewl/Kd7/RNne6kTh5Z9w0QJwgS4TZm8xbamchHA3AmHgs35MEhf6wny+cbyZiacN+20wKiH77CG3u97bJATjbdH9XEb+STYaUHCxYnpjpH77V0nb9q223/Ax4lmA5KTy82VUY+VzswMpuDEdJWyffUcuP/Vl47JTMpys6Pp/DE++tzw5pL8kD/9ar1lLb63xsx3nos0i5v0hjH5rSjuE6W5KPpbkC/DK9iXWNhBu/i7galy4Ttybv7ZusS1U8ht8gNDNpO7lPDf9PvSSb+ccK1Xo7u5uVmPj4+nF9A5obnW9hUd9nwNCWEGBbauRFKPJ9OMzhPTjZRdeaNwL/1jtaxsjH5OsrGAHSYVGG309J0691aWCpBmA76GMVxiqF4pNcV3XQ61+64o5N+lfxS1ITznrYT9xR7L1szTcwIrdOqBcwZts/Sycs+3S/XWffAf+ur7ahMYHg6je7xo3yAI4NVRTgBgXaI0PCzL6nGzqYa4ZZO0i7wtH+cu/bN91iuHuLZ1+tYICDCsrAu+U7kIYL/88ssGjBwKXF1dnR7cpQFe3EZC3B21oSHIJoGncMkD5XtXRDxAv9wOJsVkd48TQraA7Sk8Nq5ryIiyYPQFDwNvWSdgUeVxHsgycB2WB3U631SPSt8ZE9cbbMiBNXnKPFnWXalFXu5bx2S2wpw0DWHjYyx9ooNjDscsAy9IWLaWTcM7G07B0jmqp6ens1dYc0/lUObC/DjE9hz5x15tvBx3sV7xvfJGTtNx20/1bIqS6JN/9pDjXQACEJ34r6MwybCzwWadgrpULj4LeXt7e1IeK91a24R7wxIUxGGHlcwC5HwBwiyIgR2Px43BG/ych/HkMhlXV1ebuL7hbcfU+lyXJ5vPDs+sFJZPc4U2dJ8ziNeTG7B8re/1UvXeWLw8bllPMvR8UR+Kh8ycWjDIdWw2bq41i7Is6YNZg+Xg+y1vh6zMWftpJ2FAdz6q47cxcg1MqsV5ub1inenceazkMqektuf0t5TqpsteaFrn4+tsG9YLzwnzYVvsdQVtMKdAfzb+S4P1GxnsBelU3+OEcApORtW9AYHGDTP5/Pz8vO7u7kbhGegKEs6NWEGdNKV/TIQVGkNpKa11uNUwhHotE3/3eAqqLvaMrX+SyVrbH1Fo/507goXVSVnh3b7Zk/NNHbOB2mE+QODj1R+Px2PwiqLbxklRN17cjs6657YdGnmslqMB2iyuaYpPnz6dvSnCKQ+nDcr8Wzdy5lo/G9n53mNkHZeP28583KU2zVwVdAw2nhuniNqnOgnq4HcE/iMA86NCftWzBdbvRt69eNYbX43Cvt6I7mvdRhGb+xpWFiQmcPQvv1TgZTMeW0Ndt+F2mzcpo7AMqbchTmk4bTU34j5VOSuTht/+bCYz9XGt131AhDo4Njsns0WU1nmihjCeSxvZFEb5uMGKsQM4jRw8x3agntemKsyO7IDs/PjeEN9ypj/YFtfZOTSHync/GlVZ7DGxPdZlcJ2283QuWiw3rrWs/dl2xznG3PvWes1Jv7UX7E0GhmFTIcKssKwIDmem0GKtrbetQTOg5sios6EWiG1wtND9IwlmLkZ6rutqyRQeeByeWOdBbLR9ntF97zhap43TcuO+GqhLDc3/ndifru8bK+w4vHrH6pHZmUufevCYzB6q4FwHiyqTaXhJ8aq0E8OMxX1tysJg7/u7UsmWgYaaBnzas0zt5JEHOcc6IkpzstPLNd2H6XtTI77G43ffKAac/nfYbRDcY3l2SHVK2K7TPHt6vRnnxZOHw/ry5cuJoeBdndBEQDTocGyteZtEqebk5TlXGt5BmUk1z2T26OL2zBb4bUZKWVRpMlTfCjx5i/5cGHXRR495otluswnTevAWszaDf0MAt40MHDYZzDz3DZ3RDcbHDxET1hVA11pnCyDuT1MCyMP5L+thDdXG5Pqb+6vOcA06Z0Zqhzhtfm1I3bDUDMUrkdNPAVKwK9tW2XDHwTUFa/o96faeLVKnQWqKMuyIGumY+eKI3c+vX7+ebTJ/q1wEMHeWRySsGAiQhh1Sco0Fa5ZVr8ZxKxkeHmbkNs34Hh8fN8l959y6vcB5O/+fjJ98BnV2cqmTvIbHXE/XCZkmqHKyolvueO6J3gMSBqtSeReuZW7NQnqfd6VbF+qV3X8eJfLcIDMU2ivHZVoUy7566JVy/69jrUxxBh4jc1W2VMeCDOxMPLYyGwMj1/qz27vEOsriywzbT+yiD2B7dR5ZrDVv38H2eKDf/bDsyc9Z3h6b7cWAZnLTBZ+3gOwigJUiUvAcKOz0gxFG2z3DmGiugQAhwIwaXriN5kvKpAAgPCnCN7X1xHGfQ8Iqltus9/ytHsSloSSfJ4Ay/XbfJioPc5iMykZe5V/rlYVZEZtnc998f70338sayloZn+vpeJ30bthXZtF2AU+3a5lUFwy+lpPtw8XOuXV2LJO8Gw66D23H93ZPmcPm3s95g8ha52kMO6s+92gAdD8LTpVVQ0zLxMza8torbzIwwKodZRB0ti88YzAWFMBiT049/m5PQTH9br0+V8rquj0Ge4KWCtBeuddZVv5fFnASeIyXUo9HqUdsXQaKhk1TPfRpCtes7JeW23uuYXAV1Xmh3mfQsbHUqRiUDRAOZeyk2kbZnNmfw1nLz2+JcF1T2A3DwCmSG3Sk4DEwZlhcd95T/OwvsmecEzh7DNbJ2u6Ub1trC0xuv29kcXTl7+5HU0YlILbR4ovTA5fIwEUA++GHH9aHDx9OYVRjeIdnhG1eimeyPXh+O84My0yrg3XnvSXgcHj9JWcLpfF2BeZj/rk2C87t2wDKUK6uXn+gtue7TaMhwyUvzFin45a7++eXNNYYUIYqF2zUYUA9ZYGO9glNPH8UlNHJ/8qlhuliZXZu08zL+tax9S0NZhPPz+e/yk70YAdoffRcdaweF/pAH5w789gBri6QMCZvs4D1VL5eDTX4U5qO4VjZZR2znVvnkfOMhXYnp9Q2pucezboMaJ73CXw3Y9o9s9b6wx/+sO7u7s7Y1WRQ9jpmH8/Pz5uf2sJ7MBAEDUU1wjvUrPezAOoNCg7OF6GYxPRlMBVyWQ3FQNrVJtqcqH2ZhsvEmAquPmdD76twCqjs5au8HQbYCGyoE8hQHIqarU6GYVlVKS2rKvTUD0Dbcza163Fz3O/mYm5w0sjOjrfz0D6vtU4rk1MY2lDPq5jWIcbBbneuvST/PZbt6MastXpMP+yEuR7ZTk78+vr6bO+c89S1y+aI20/3z3Y3OfrNHOyeWWv9/PPP6+bmZgMgRkQ6XO9TIPFAuN5MZ48Gd5C06Xo9KQUgT5AF7Wf9proRKMdMY12Hc34oob0thuofNTibgLDJyTlQ9hhb2V3fgYXMDUoAQL038mIMnnvPhRdtkG1B03NvGfohc8boENY5KrNAriGnV7mYSVHssCgTeDJWr7q6b056T3ra7Tbug/eFNXw06JpdUti+NIV87ovnqfkjy9q5WtsRdmhH5jGYbLy8vGz0nzE2ZHY/qj+eB1JQDrvfAi7KmxtZvWq31jkNnoDIArDwoL4Ngaz8/PkeA5SF7PwXguCYAcvCsMIQRtFe6+Jae8H2ZQobO1EGQY/b+8489peXl3Vzc3MKYVxXFcvgBHg5h+dS45tWfCdF7JjsPBjbxJB9rsrfXJadT9vzkr/nlOI9azYa2kbOzScatKZ5tEPyuKyzLlOuCzk6hHJfYc6Xnvubnus027UulzV5PE2POAfV0NKyctRUnfQc0S870DJop3Ka4kGfjBtvgdibG1ldeRmPz1lxEIAVtF5hCs0mcPB1BiiEV4X2da6ztJQJ8gOna80JT09YWU/ZX+XgYz5OW8ioY8Xg2hfnI5yvch0oc3NDjGWvTJtOp/L09LTx5A4f19rO9cRWOic2sLLQKX+3145DGF/rcKilgDP1d1qEuGRYBsbqSfNibtP/3/psoDGgTvo3/bk07bMX0Tils8f+zeToZx0Yx80Q7Syru/92Dmx6ZUmRncaZlMnACnp4THul5jxQZlPg5oWa85iW0C14My0rggGvINlJMvBNxwtYa52/idRj6KRyzvt3qMP3EnZOSXbLZpKly2SwHr/nc8rrGID3wHECwxqw2VzHYVZtOdmRte0yhIndOZGOjpVFwigqH3bQU7pah7zcN8/5FFY1GuGe5lDdjx6vrKfQzWUKrZugn2yi93J/AcrOwXP58vKy7u/vN7rlfWQGtH87B7bW1viOx+NpZ7W9jDu8x1ScQ0DpoaNsRC3a21uYflqAHuRbFHWt7YPATJANY6LdDdmopxSXCSuTpB6+17O5Dq+00g/LmT5Mhu3xeSwYqVcqYSUFYu7xSmaNwHM7AY7bt+PzIzNOMTQ0cuF82/GxCWDNJHs/xmIZTvOBjkxPcuy9ywsjLGs3mFlv3XfkYpk3OvA90xYd1308HjeLC5Vpx4Q8zFid4K/8KXaUzXvZ3usQWNw7Ho/r/v5+rbVOuxQOh8Pp8cBL5SKAsYTrRx1YuUIQVZAq5+Pj40kBSpfLhhCa9yNNjMaMzpNdj+TvZhRmaggZhfZqme87CSzMzUzi+fn5lHR13VbaspUarBcGXM8U2tI3g5BBoeG86y3boBjk9sIuj22trdw7p/QFoz8ej6f3xnWcBUv0qADgfEsNyQzGjKhbDMgVMufOH5mJlFW1zy7oTcHWsp4MEsZNLgynxZx7UcMyKYBjr2a0zb3Z6VlXuLepi4mImKwY3JpTdfTD5z4hA6YgZ88zj+BdKm+GkEyijZoGjbgGHfIjeKPmcroQwP3el1XvwrGnp9dnGx0S2nNO+TBPCH33mFzfFGrZMOxN3H9/9/1TfdMzdPTZxauhBQb+/AgJTMv3X11dnRS0e+cYN8VjqKG2Xo6hkM7v2Mg4x/zxuhkzUrNy6nOOz/2rIVNYyfIzrfSZfhhILUPrsce9F6JXJhhnV0jXWqf+2PEiX/rs/k59Ri4GU7Ntxo1MsBHPWZ0cYNGcVl9XZMBDz52W8H3OvXGcFBP98d7Jkpb29z96G8X9/f1GIc2ySnMdIpnh2Jv4fjrrhYLD4XD2W3AOObr9wWBR8JrCPoczje2dzDezIBfl+j3xXNs3WTR87UR0pWYqBV7GZYWz7Av8ToY2HzO1gXHvGa0Bwxs2HU5Pr8nxKpsByQ7By/zNhU0hTJmvQx7GwJ9zXU0NWBe4nzKFap4rh0PuX7cqTKEyYA/r6Vs0LHPX0UiC/hDSAqSTHXj+HSY3jEYm08PlE9u2HU12x7VgA2+AMSP15mKz4Ia/LRcB7Lvvvlv39/cbAa61/WFZdwTBmTWs9SsQOjSYkvFMJOBmim/6XA9WwTbPZIOx0nN/QaIgQB2d5Bprww/61K0DJ8En/HGxXKYcBCy34SAKhFGxIXLP+M0efdyyxbt3tdRj8T4nb3loKIKSGoSnv8rIwOnx0UaBxvMHcE3PCboO53UtA0AdZmSHcSmUpJ2G+QYQGCf923vtkhmZ9dX24XbrhP3n/KmT9XbU6Lt/E6P6zxhqoz3X3JtTRx4r43SYWUAcZX3p5P/8z/9sNgFSoR+CXuv1J5is2Cgj1NYC9oT4OHU35JyEUy9vukup12r41TDOXovPZhdNvk4Mir4xtuvr67PEfBmXcwIeE+1PYaUVoOc7Xoq/V85TQthysbzor4tDRerjz4C81jaHyXcDJox9b/VtLy9ZNtQH961/zSk6rVGmbMdgeU/6M+kFdlFj5LhTIsfjcd3d3Z2NDZn5j/Yq1z3W5pCMKIpHBR0alv24jmlczEmZZCMerkNGPsdiA9HKpVX9jQx3z6y1Pn/+fGJbBgwQHq8yLZPTOZDdRg0LQ2jQYbxRH58oPTWjqqfxO4nsTeizBdhJKItDeA3LfG0/Mz766ZCY482zIAMnpxlPx9hiJ1AmUjbj/35Wj3psgO4L/bc3bmIbhmUAN5iVZeEUyixpr2zIfXE/zTZshIyR7/b67lvnnHrM5jxfZlKVAeyT4w4RzVA9rq5kPz4+ruvr682Yp5AefXEUYec6bWhGHz1XhJ5mXfTTqQ+zv0Yi/DexMStrusNzbXa4t+J5qVxdQrf38l7ey3v5/1ze3Af2Xt7Le3kv/1/LO4C9l/fyXv5ryzuAvZf38l7+a8s7gL2X9/Je/mvLO4C9l/fyXv5ryzuAvZf38l7+a8v/As0toE27V8j6AAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "![](\"images/laplacian.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Implementation\n",
+ "\n",
+ "There are two commonly used small Laplacian kernels:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![](\"images/laplacian_kernels.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In our implementation, we used the first one as the default kernel and convolve it with the original image using packages provided by `scipy`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example\n",
+ "\n",
+ "Now let's use the Laplacian edge detector to extract edges of the staple example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9SY9cR3a+f3KunGsixaGpHiC50W4b6I3hjT+CF4YX/speeGXYbrRa3S3JIsUia8qxcs78Ler/RD43mEzB8uIPA7wAQbIq896IE2d4z3tOxC3tdrv4dH26Pl2frv+LV/n/7wF8uj5dn65P10+9PjmwT9en69P1f/b65MA+XZ+uT9f/2euTA/t0fbo+Xf9nr08O7NP16fp0/Z+9qsd++Y//+I+7arUab968iXq9HuVyOVarVdRqtZjP57Fer+Pk5CTW63V0Op1Yr9fpZ7vdLsbjcfT7/RiPx1GtVqNarcZ2u41yuRy1Wi1ms1mUy+X0s1KpFJvNJqrVx2Gt1+v9QKvVwv232236Xrm898P+2Xa7TePlfrvdLiqVSmy328J48mu9Xke1Wi18f7vdRkREqVQKV2+5x3K5jJOTk/Qd33u9XqdxeV4RkcbKvD3Xcrkcm80mNptNWgPmUavVYjweR6PRiHq9HpVKJTabTZKNZefx1+v1uLq6ivl8Hp999lk0Go1YLBaxWq3i1atXcXNzE6VS6aAMLEOPn5/7e/mzkYXlw5j43Xa7jUajEavVqiAby9DfqVaraS02m81BmSLv9XodjUaj8LntdhulUilKpVJUq9VYLpeFtcgv5sjfPAvbqFarSYfL5XLM5/P071wOlhPzrVQqsVwuC7KrVqtpbefzedKDXJY8j99vt9ukk54v62VZ5hdjQ/8iImq1WkGPkd1ut0vz99z5DN+vVCrpcycnJ7HZbKJSqcR8Pk/zrtfrERHRaDRiNBrFV199FZ1OJ96+fVs6NM6jCOz6+jpqtVoMBoNYLpdJcR8eHuLk5CQajUZst9vodrtxf38flUolms1m/PDDD0mQDw8PcXp6WjDI1WoV4/E4IiLm83k0m800Se5ZqVSi0WgkQ5nNZtHr9aLdbsd6vY5Wq5UE9/DwkJQUQ1sulzGfz9Ni7Ha7aDQaUavVCkblP6vVKlarVTIi7rdarZLwcbT8GyOKeHQMLOp6vY7tdhuz2SyNLXdeOPXlcvmBwaFA/Ix78/9KpRLr9To2m00sl8v0bxuQgwbPr9VqMZlMUuAplUqxXC7TfXCWzJnPIB8Uularpc9zb/7d7XbTzwh0rDv3m8/nybhQeORgJ7xer9Pcy+VyVCqVqFaryeGs1+tYLBbp/81mMyIiOReeHRHR6XQKwSe/Z+54HdjsMHAw6BTz3mw2ae0jHg0WXbGjsONGR2u1WloDjNjj4RmMC/3CqfB7gvt8Pk/PQO58zvq42+1ivV5Hu91O9+UZPLtSqRQCltd8sVgUAMjJyUkhUDNG1gOHxrw3m03yJScnJ1EqlVIwZizHWr2OIrAnT57EbDaL09PTiNgjj5OTk6jVaknpZrNZdLvdWCwWsV6v4+nTpwXlwOMiUJxMo9GI3W6XIldu+CC5er0e1Wo1FotFzOfzKJVK8fDwkAyLReRCaVEE7ovD47OOtEQyDAwjQnEdRfg39wEN5fdcr9dRq9UKzgclNILEOa7X64KRoyT8O3dGKKDRAs9kLL6QVbfbjXfv3kWn04mIR8MejUZJaXFuyCtfN6Inn0O5+T9oIiIKxsjnHJ35PMrP2tl5YozIiyDBevAzDBcnXq/XE6KKeHQWlidjwykRTEDRNiIcTI5mQRGsD/q63W5ToOf+7Xb7A0TPd1erVRoPF5kCMuNvZxm+H3KuVqsJxVqmeaaC/tbr9ZQN8Xnm4PkZ+TMGbDh3Ng4cyIZ1rtfryZE6MDA/1sy29rHrKAL77rvvCt7XghgOh2lyy+UyISa8Oc5gt9vFdDqN+Xye0FylUolWqxWLxSIJgwnhzNbrdSyXy3TP7XYb0+m0AFF3u11st9s4OTlJivzw8FCA64zDaaeVjUW0oCxcxs3nQBTb7TY5D1AaPzfqqFar8fDwELPZrADfcZA4HAx8s9mk+/p+oCLGRASz8vB/K3Q+H1JF5gJ6c8qNQoHO8shtA3AU5v8eT57qg94i9mjb42VN+Rw6FLFHuJaPEU61Wo1Wq5VQnZESaJRxcC/kb+SF8WDQjDs3NtawVqulgO45oxfoKAiNoG4DzxEKThi95D52ZLVaLTkcnDx6sVgsUhZip+JgQEaF3XgNcIwg7dz5YbdGx8vlMn2HZwB6HFgt19xBczUajQKq/9h11IG9fPkyKSiLO5/PE+eC4BEGURkjQRGq1Wq02+2Yz+fRbrdjs9nEw8NDAR57IigmSlgqlaLRaCR0QHTjMygGisLi8js4OxtRRBQcc71ePwhZcY6G01YgFpSx+A+fr9frScHzSOjnMs92u53GRfpoR+nUsFarRaVSSYaPw7SB8GyUm7FzfzsJO+3VahWNRiNarVbBQDECfwdnbSOzwiInKAiM06k+v8cp5fK30aGPjAe9eXh4SP/P55qn8kbLoBZ0xugIpGMkakfMPOyEDiGfHLUZDZKKogcYOXLnWUa3rHO1Wo1utxvlcjlarVZ6LrIDxfI55m3n0Ol0UuBFb7l3vV5P/ybo4NDRM/42lYMtgeSRMwEZeTj4WV+RPTI6dB11YG/evIlarRaNRiPm83mcnJzE6elp4q3Ozs6SU2Iw9Xo9KXi73Y5Wq5UMkogRsY84GAkOsd/vF1Ko1WoV8/k8xuNxNJvNNHkTkygAThTEAFKxEEgNEDhKaxK0VqulKAQqcTqWKySGBYJkQSMeI6HTUBQaKE10w6A2m00KACcnJwklRUSB7DRxbUObzWZRqVQKhCv3AIFst9vEG5mbstOI2HM4niepzm63K/Afdmo4JZPa5okYX6VSiXa7XYjGBBin+XaK5quQMUZFgDP6Mvo7VBxCrnBpq9UqWq1WGiufh2w2yjH3xIW8yB7yORiBcw/mYiSKDI3oCVjIyk7Na8j81ut10u9Wq5XWHV0llTNf5nkjG/TV9st6OMUGhdkOc+djbhwHZxvgM6SRecEsv446sH6/n6IlOTUOZDabxf39fVIYHmxhzGazuL29TYaJcFC83DBQvHq9nlJHnAzIDgI7oljhwsFaYNzHysx34Tem02laECIYv0cZneJyWdhEu4g96sh5K6c9PM/Q3JUcvo9M4EKYD4jOBoJ8gfSWD5HYn+f/u90uISzIUyNLc1eMC/TR6XSS8yJFIn0wL2dZmtdZLBYF+TuQ5NVLp7ncHxkZseEQUHqQqauoOAPWD1QPXeJn8izSTsuJtAibID1sNBoFcj8n3svlcpIp5LntAX1iXNgY8oFewZ48ZtabFAzemPsyT+sbsgZocF/WgcCwWq0SReM0G5rhUADCuTE3dND24sojz3awPXYddWCOJrVaLabTaYFsxEin02nsdrvo9/vRbDaTE1iv19FsNmM8HqeFnkwmyfjMEYF6zKe4mmVOyQIxnF+tVim6WJheCC4UE3hsZGLilwjTarWSAi2Xy2g2m+m+REvDXhSEyJRHEiMxpwh2ACha3h7B+ECTjBWjAKkyT3gJFArj58JpYtzj8TjdEw6Tf5vEZVyMAYdC8YXPkE7k1S3kz7MZh9MmVwP9Xf7PHO20jQgImMi+UqkkhwA3ChphLcx32WmCJLk3RnhychLNZjNqtVoMh8MkB/TQxs447IAwXObCM0Fevg+2APXAmGzslUolIX8jdKeO5u/MO9br9Q+AAJfHYSTotWUNQMpkFE4nnSI6JZ5Op2lNms1mIe3/2HW0Cvn8+fNk0LVaLd2UqICR4J2n02kBZVQqlRiPx1Gv12OxWCTheNCuFjWbzZhOp0kxRqPRfqCKyE4DicDk+BFR6JeKOJzy8VkWmUhsfsqLRhrJYpnzwAj8HVI4R1X6jZxOECVz5MOF8mFci8UiBQiUr16vp9Qd0tUpNsbEWHHyTq0xMuQC4kah+JnRLkUYvp/zPcjWqClin0bklT8XVJxCHuLXGL8dOM4dp+hnImOnqszLbRX52rB2OGiCq8eDLkZE4qDQE8ZpXg00Zzvxc/i+K/d2Xu5jdGrN/W1fyNV0AvdGT3On6cKdA6bHiA65EGZ9sx9gvVzZNFeW63rEY/bm+X3sOvrbr7/+OiqVSpyfn6cWCS54MC6jLiYKDAZOu+eFiLtcLqPf76fo5ChNtTKvmhmtRERaEPq8csTGovjZKGtuuF78iH3LgFsdWEDP1yllEu7/p7iQ8yifjdAoZbVapchJUyAXcvFzgfTr9brg7Pke83MlitTHBsRnnYbivEAHjJ2/jTCYq9fVMjbHAYLHiHme52ijZcz+mauYJou513a7TdwPOpU33nreNlbmSzAwj4oDt3MFBeWpDhU/c4tc3JvPTafTJB8yE+63XC7THz+byzQEOsRaOFPhZ3Zodmx83+Q+a8VzPS4HRV92/MyPNVgulwl0sNamnJAJVftc5w9dRxEYUWSxWMTp6WlCO0zIEZpWCpfsc04H4nY6nSbB1Ov1GAwGCZ1g9IvFIkFjk9J4dv6NchN5DWtRTtAJ/A3RPye7HcFcngeS55eVHyO0oRC14QyRjxtjIW/Nl2CkTm0N3SP2qKHdbqfqkB1iHkhQJNpYmLOViXt4/Rmv0yIjAyMu903xGTuqiEjIje+ydozBKRoXTh05kZKgIzyLQFOtVgvpdd57ZcRh+UbsnT+Zgg2YMeP8vavD6JB5MC6ej02QkbA2dqBG+tanPE1j3B5f3kJDb5fHxJzcx2cnZsoChN9oNKLZbCaqJ6+Gu6UHh8O/QbmsB9V6LnhFp7oRj4Do7u6uoOuHrqMI7NWrVzEej2M4HBZgIdwFBgOJ6Uja7XZjvV5Hv98vLO5sNovLy8tC2wGLu1qtUhrZ7Xbj5OQkkYYoLdULd3CjcCiAldKob7fbpbE6KrOA8BI0vjoN9f8dZbiHOQXLKe9ByvksKy9GBLIgGBCBHfkPcRTIBCN0asT/jXJw4hih21GYm1NIdAAZMnaiqBXWcuNZcJxWYPMldoJUMRn3ZrOJ4XD4AZmP7DFIZOZSfalUislkkr6DfPkucmo2mx/oD4UO6wCB1QjHa4O+mWw38uZCH9yblyM8HAnoj90P6Brj4ZnYgDlZrvV63xhMqus1IqXE2WALDmLb7TZxopVKJSaTSVpHbMyIFH3xWAlCIE4C2ng8Tj8jo/CWrUPXUQf23//933F2dhYXFxcxHo8LvVmkZ65CQuCaRKRBjgVfLpcJcSHAyWSS4OR4PI5WqxXX19cR8eiJvfju9j7EFcHN4WRwBKSR7rFizCZ0HYFJtXB4Rmi5I8sbHUEBVh6UC+dvzgMZrVarmM1mSZauAjHOQ+nKZrOJ6XRa4IxQPtYLCI9zNQeHornYYaQTsW8uJGqa2+JzGD3Pw/jdbMxcjRQPcTo4VlIfZGCeCkeBYeDASINx2lAcrA9jjYik1yAlHDI8IONuNpsFHgt5kVlERMEWGAc6YyTNWtpxee2MxEgz+T/PYl3cWhKxb1cyl5cjRGcU5oNxbk6l/TlSc9/DgYz/M5bZbFaojKMP5rQte4KcC3jHrh9FYO/fv0+bfnkwyKJW2+9xIkKUSqUUURAI13w+j06nkyAmC4jxUAmibQLvzj28qAjNXdH87SZCBIPQp9NpWmCnZFT8HF3zlKZcLm7r4QLKO+XkMl+DI2U83MdR3cbBvV0UaDabBYKYiNVsNqPT6RQQBmjMZepms5kaY0GUVFShAnJ0ikMyckMZc77KXBqNmFZW/saoeRapx3a7b1YlEM1msxREiMgggohI9IENCF2Ac/HzncrlfXsR++Zi5gHqAdn4qtVqqafOgSgvRLCLxDrD+rjR03sEkT96cSiN9DOcziMHo0B0yQUe7onDsvMEyTKOiEeHxLMjHgEGsuEeBA2cN87ZRQPW2e0U3kvJARDH0FfEjziwq6ur6PV6sVqtYjgcRsS+/DuZTFKPGJG2Wq3GaDSKbrebFoAOYQu60WhEqVRKnt6l7X6/n5wMDbIorI2BezFBoiHC55k5Z+QUx1HPFUX2d0bs00l/lohrzoIFsiKadyHlYgsVDXx59QgngRNG0ani0teDHPkem8YxeBAGToQx8PO8X4/xm2w1OW5DYE5cGKI3pXvNuYzAjIBwzjyLVAUagjUnwBBU7Cgj9nwf36fqzXPthHm2kZkREvfk3zhUowkoDuSEI6TQQZ8cduBqI/KBGwZ9WKagMXQul71Rv50j4+D3rq7jnL1W1mPkh45CAfneRkWmXRiH19+y9NgBD4PB4ING6nK5nA5tOJRx+DrqwM7OzmK5XKYuXrzkfD5PEC/ikUimhwMeodvtpmoCUL7VasXDw0NSBhyQIaj3/TmXtqNCAOZzut1uodoYsc+/Scdyghrj5jmgSiKXU5VckVAYb3MyckI++cJgrERUkAdR2xVRvkMKlCMbp7bNZrOgKNwPZwXSoRXGkTxPe1B8xkkqaSNxmoIhmrh3wALduHjgi/uD7pgz6IQgwucwUOaC8wSF4QztsJzy5z9HD41iuMgskLv7k/w9I3YQk5GGLzt50lQCNM9oNpupIZZA7JYHgq5Pb+HiuS5SUYlG38gYXMVEf3kmFAQ0kNE39ofONZvNQrBzQIQi8tyd7rqBFRsmc3CT8qHrqAO7vb2NXq+X0hUTt4bYTMA8kXuEECRREcOLiFR9dO8JTiTfa8dl/ogxPDw8FDaWOpKan3AEYPHcg0VENRfE54n+vnCSfh4oiN/bmOEEMNLZbFZoXjVkd+pFA7B5NjhFkCcpN0bmJloUkjEZDSETHLFhO+O0wzKR7+/lAYLU0w3RbNEyB8S6kHYwHhwo4/CG/EqlkqpsLpycnJwU+sCcGhphRexbEBykrGOH0k+QMI4jR3VGg274NQrMkbqRFM8ARfM7xkJwZ22RAX/cIO7KNrxlRLFS7sDO2Nx4ut1uo9PpFO5rR8+a+gQJ246dpYslrkz7GCWn5Hm6fug66sCePn0at7e3cXd3F6enp8m4WUAWixSEMit9LZVKJSk3itBsNtPRIiAuDAKkh3DOz88L6ASB2zgMj/Pob16M1IvfY/iMHeUg6jhtRMiu0Pn+jnREXsZkREXaZ47CPFK5XE5FC8vMz7STJ+WcTCbp36wHzySiRuxTsIh9oYHnMgdQD2gMWXpLionrcrmcqtSslcfrbSw4TgczH5eTV+pYY77PxZgwJvcDWrY5d2QU02g04vT0NKEQ7uPP2/mBSkB7i8Ui8Zp5iuO0DuLffZQ4RzuUHPV7LubD8qOeGJOLRH6Osxn0hmDCOFizWq2WdpwQyBqNRkwmk0LKz72928SICX0DSRml73a7FLSRlVsucO74F2zyY1fpWI/F3/3d3+1+9atfxVdffZUcEV3T7gmhx8kGEVEsA0dEGjipGigDKA5SsIJaMUkTECRR1ydh4HQcGa1YVJXyvhOPm7H7b+6VozjPxc/j3z6yJL/foefY2Vmx3D/G2BuNRozH42QAbr3wnBz1UbK3b9/GfD6Pp0+fpnXd7Xbx6tWrmEwmafuX19AEuhsx83l7zSOKp4163sjTPVs4Gm+C597mvaw7PNdpov/NHPIepFxOEXtOx8bEhf4RpI3c8zk5+BGgzb+iu/4sPzOdgA4x9hzl+rJTY944WVdTbSO2Fd8DPfMaIycKKA7AjIUgzty8n5g+sIh9YHWrj2XRbDZjNpvF73//+6jX63F9ff0/P5H15cuX6SwvJtbtdmM8Hsf5+XkatM912m636TwwEAVVR2Cpy7SNRiO63W6aWK/XO+gMDYutpDiQXBHsGIzgHImt4DlB7fSI7zulPQTHDeFtjHaiJk/9PcbG/ZCPx+6UBEWaz+cpjbDjN8/Fv/m5e7FMFkfsj4bBKTNHEJObWvm+52A+xoR2/rOIfarr9Ip7REShBSf/rlE265HvNfUeQzcmmzR2ZS9PY/l5nmISLPOtPozLvI7Xlmf7Zx6HUTH6mbcdcE8+7+Dgi++6kdyf8/3gnshQttttYZO9HR/3IBjaYZrQBz05w7DDo6GZ9ev1egX+jXsfAhqFeX70N/F4nA7leYjAh4eHaDabiW/B+EkJIx65M5rSKIG7cjSdTtO+ydlsFtfX1wkCTyaTAhJjsXOPbuUwx3ToMp9EOsnP+Z1PTTV/Qfmb6IGBMn5HsYj9QXKOdN5Dap6OORBlGZ8V0/1ujJVeJZTC24IwIKNRrxPjyJWCaqbnyfj4A9ImHcidMM4AFGinYiVmrJTKufLeI86u8lpzufxOOu0WkxxRmLvBmWF8XnePx3ykMwPk53Sce2CYtBUQ3Nl2Z2cAlXGo/YP0Lnf+DkqMgYCL7iBzV8FNBeRpuVNnbA75glzdAUCQRkdbrVZqNeGopRwpem7ICjS7Wq3i7u4uHbPj9o//1XE6FowjA44HRWBwCMHGMZvNCqkOzsJw2CTsZrNJHA5H8LIAh1Ijvgu6w0jNvZTL5VQW5jK/YRK+XC6e404vCwuxXC7ThnO+B2rB2LzRfblcpv2Khu05SZwf3cPl7moUl61Y5t6YI45mtdqf3U50NDfh0wp4Jm0aXnNkCdoldYCHjNgjcOblVNGVs4Li6f/MCzlZ2W0o3pfJGiIPHLRPv/X7CFarVepBdCc+DsEFEhsY46NlhTXmmUbzoKLcsbEmNl4T9w6ufN/onc+5pcc8pAn0vKrvrMTP8t5KttnZ0fNd92vyPQco5IVMaZEy/0zAQx8d+Bkzgc8nYThIfOw66sBevHjxQaUJ7otWCAbJxUQh+kFdFxcXsVgsotfrJWNwAx9GWCqVotvtRrVaLRy65+qn822jAPqsqtVqgcPYbh/bLHCc/J5/s5+Qz7rRMwlKykYqBcphAZwGEcHyVM1bPMyfeC6uoqF8riDmp2aw6N1ut4B6chQGb0NliOjmdAOHCMpzNEYxMUQqrUYglpsVP0+bqRhGFDu4jdhwaKQidkhumcERY/heD9Zqs9nEZDJJzt3ku52g5Zf38WEHpjO8duiC02/vfPDJH+6U52c4Q7cZ5ZxVHghyfcEBkYa5nSRHpcgnR53YgFG1K5UugBnZVSqPp884rcVeDvGNlqH1i/GAJn/yVqL3798nQ8f5tFqtGA6HyVGMx+N48uRJDAaDFMloZF0sFunFEexFu7u7SxNzeX82m6UUwOgI4ZkYBLk4DWBhUEoEb96ESgqIhn+7wuKKFPeMKPa2ROybL3kOn7GC+Myn9XpdKISgLB/L77mHv+v52aEjQ9BTRBT2oDJGOwhvB2ItkLvPuWKcyJTiAdGTcThtMArBSdp5OQ1ibd1ywZhBNjhbxs18WA8qVkbj/Dyi2Obh3jXG1Gw208GCvEwGrhAZErBcJAEx0QNlJ8E6s9ZG0Pzf63+o55F75W0WrIm3A5kQ9/ec6qPXTpX5LLICPNhx2pFa332AgflpZIrtgfa5r7Mk+t5Ae+7zRB+OXUcd2JMnT2I8HsdkMknlVDz7arWKXq8XZ2dnMRwOC2nTYDBIUR5n4z2TOERXZdrtdmFTK0J1pYazrPIKl4/1ySEnSp0TwlY0R3BvZbDD4js5Me7UCUNlsd03xZxNUPJ7nmv04gZGNryCaI0IjGSQS6lUSq0sVjwqvXzPTYl83ymbU2fm7GKGnV6pVEr8Xu6UXTCJiIKxs56W3SG6IJc338WIMBan21z0btkRcoESQU5U7XIU7tM4WE9ONXEV1fqAwyeDcFrKv12g8Vr5fvBEvtAd1hwExOXKpfUA1MS87aDL5ce9y+gDDskFLmTG2KAjcidHvyM6bCDgI45wxOg0fgRf8b9yYN999116ywt5csQe+vFSgfV6nTw3F8pMGogBWrl2u12hAZaufy6iHRM2sY3RViqVlGrQsmDejOjgrSEoDf08eHpzDkRb0jMQBpHckBpnBR/hi14gjB6FthMGzRGVjHh4Bq0iOQkL8cnlaGguMGLfF8SYzXmYJ/FeQKcbGAjyQF5GpcwZdA0Ssbzc9W7+lLExB+5PdCalcDU2Yp/2YKzoKbwrDb/cx6Q1cuEgAaMa0xWsX6PRKFTsqtV98zFcGf9G73BCzInLwS03chwa93NGYCTHGXkO3NvtNlX/8855gwj01cEFO3ABwYcc8F3G6pQf9EwaSMuVCwuWAQi2VqvF+fl52k2DLTHeYzzYUQf2/Pnz5AyI3obiwMN2u51SyNVqlU58gDitVB73hZEm5uc5OR93kxuL5P1ikH7mMSqVxy0yKKtTI8bB521ApFE4P0d9lIbPk+bAq6EAKGLOX/liTkYFH2uApDwPEc34nULjmJGf00icvFPFnNvz9hwrIP/21h7k4DFSUeblxMwL1MfnKcJEFDlAN/NS4WUtarXaBzJk3fljXs1pJ5cRu1GTix2W/6H0j7/ReRdT2u12Wg8HYGSAPFk7ZMaYCcjmrCi6MA7rrAOmv4fN5PJijibB7QQO8ZHWfXPPeVrL511ZR6d4BrZk1I6d2UmztvCTVKWRB/7lJ7dRDIfD1KfFTUzymvNhkQ2ZI4q5uSMVykjPEZHFQsKgcwjN4rkkfWiTcsR+yw0I0txSjoIQHsppgTuyMAanvKSg7j1i4cntMSqewYKDYlzdMQ/hcr7TPBQEdEmFzemr1wLFSItf3p8AAZLISWjmZkTm3yEz1ipHfUY6EfsDDanqIbeISDsK/Fo+7uc0iH97fcwPMrfJZFKoCLpVxqf3ohd+OzXtDbPZLI3LqTS6x3l1jCXfD8ozWH+CkeVj3sd2gH7ahpzBuCqac7b8zfocao1hParV/RvNQT2Mn5/nhQ3aiMicnPKax+SeBIn1ep129OSOKadfkPFPJvF/8YtfxGg0infv3iXnNBqN4uTkJKbTaUJaGDYLsVgs0tnt9FHNZrPodDqFtAhuwo7BJCCL5BYIw+d88uYciLgIlZMBnKY4OufKwxEzVjxzBrQWsKDIJ99ywZhwKKSA+XnyJkHz6hYOyumiORsiN3vo+HfEHn1ygWAJEJ4zxsd62MFZrjiqfD/gZDIptJ0gL1IF5or83LhpBGj5eYO60wobGn9Il9AP3rbkVI7nMEH0x8IAACAASURBVCacWUQkZ8SpE6yZCx44WHq8QBCgIDeDkvYaMTE3DN68Dxv2HUSxJ+uZiwhGs8jWAdCkvR2tq+EPDw+F3kgQYrlcjvv7+5SSoleeC/f2JnwuxknHAbp86PRi9NvBFLv/yQ7sD3/4Qzx58iQuLy/j5uYmRXhXA4Hp5Pnu4KXS0mq1otvtFjis0WiUoLjRBEpGQ6hT2LyvhwVwCmiEYGLdkcqeHijsqIRygypweE5tT05OUs7OfSP2jo97wVEYLRy6OE0iV8bcSYBYnEqxJvCQLhQ4pbZMWL9qdd/Pxb0dIPg+vBaOAgRq9N3pdFIXNvfFWeQkt9M+jNJprdE16GE6naajjiaTScH50UwdUXx5qlG/m4VBlHwWROisws6Nz4MMhsNhSqHd5IncvZfPa2RESHBzzx3zdioJgnfwNvLPHSMcXU78W8/RNTIU7IY+LvY0evuVgx2Iyk6Hz7HWOVJkPtgc9u2DAYza8qLRoevomfhPnz6N+/v7mEwm6bA8Jo3xLpfLgnPi/C826XK+FoPCGTQajQRD7VQcVVgYJkW5FcE4AvBZnAvjcZrIZZ7GfW0myFlYKlT5IoLeUFh6ooyQyuVygQ9EBtzTvWS8vclONB+/USZOhM/BH/I7H5OCXByFiWw4GO6LIiKPpCjVasF5OUqbp/D68PP8d/474pEj7Ha7KVKzRtaJiD1CrNX2B2my/lQP+ZmLPNvttnAoogOFdYJUyPdjPUFJUA+eU0Rxl4T5WRPWnjNBwSmldYvf49wZs/dj+rnMgUDDxXhAdegla4tMyEKM3HFijInnsb7QP0b1XhP3edpGjbx3u/2hhjkPhz38ZARGdCAy8MDcK65Wq7TFKKJYXrfH5j6bzabwIgdKuiwAjsSoB9jripvzfO6H54YMxuma/GfMJscj9mfRRxSNJWL/9maehVPL08WIfUQ2YnIlzOmA50oJGwW2E0FxfPZYRPFt3yg9MuUzfIc1pLLaarXSvDAMHBMtH1Zey9iVXFJZOxvWCx3I/1gH8oKBAxprZEfAM0jv1+t1ejep0zcMvlwupyIPa+/n0SXuog7n7/O86XSajNSVPVdqQTAEN+TPZ43UfPy30y8jTheYxuNxQoDMAb0wR2rngPPG8aLnbNwHLSMH7NppKJwVc2BOtM3gD5iDm6yRDYHXXBlOmbP5GTdzceZz7DqKwFBwFM6TheAH7YC0XNHg9NGcaDyEihCwOQ9zQUROUidKtCymT6mw0QHJ7YidGuRRHv4DJ5IrIj8jQueIwfwScyCCeG7ufEdZD8mFexv1ReydpY8CppxOVQ8E5Fe1gQDNtSAHHLPTZyJxXqWlN41Ii6OlYAKHxPq4Cst88qjNFi0HLZfmuS/I3pwkz8I4XcwAPXFfPxvUys8JMqS/BNzVapV6IZ1S8xkCGu97wPhxFDR2cpkrcm8Y8sOhclHtRIfRKfaLsrY4DAIwMjOl4QBoFGyqwXaBQ7XugTbZ9occXMBhvPgG0nh8CTrnKjEHoaJvh/jugm0c+yULaQPOy6MIE4PGQDAc9uCxYOPx+IP8PPe08/k8hsPhB+dP2RGaNAUBMF7GdXKyf1ckXc9ETaMKQ3sWm3O5SFuJCChKXpnL0VFejUSeyAYSnvk74oI46CHjuzgB3gJtB+FeIHqd3FPnOUbsESaku+dnfooWEiPuPN0GpfAMnAxjcnAArTuIoFcvX74sBA22ePHMnCJAJ0BcXKVSKR02sF6v09FBjMH7+Rg/usA42JaFsZ2fn0e9Xk9b0pBZtfrY0DqdTgt9kRGRTmXp9/up1Wg0GqV1BOnwXDsMV6lBQ+iLtyRRMEM2plIYu9eGdUavKEA4LTSX7DYR5out+kBJH2IA+kUO+bYo+Ow8ja9WqymgoF8GJoeuowjs22+/jb/927+NarUanU4n8V23t7fx5MmTmM1miaQfj8epOgVJF/F4LDWDRVibzSYdL00krNfrhR4xFozIbgLfipxDaldsgM7uqgZ15N7dqZ7huVFbuVxO80SRGDvKlH8uj3yH9nuh0EZKoAQ7ioji0TLMl+1ajor39/cFxTRKJQDl5D6pUS4D0AORFqfN85gbUZRxuoKE495sNum47bzKhMGyZe3169cFfqXf7xd4MmToQGe+1HKERPf2KH+ewNtqteL29jZGo1GqXOOkJpNJcua1Wi21V6zX6+h0OtFsNpPzIbDizOv1elxeXkaz2Yz/+I//iHa7nZxDzsmRSlWr1fQ+xkqlknjWiD1Zb76O7AQ0hE7WarV0D2yAJl/zfKBmdBpHb8oE++N5dBjAg7M+Rnk5T8cf7JDvOoMCkOQZUH4ddWCvXr2KN2/epJ6YanW/mRtldnm83W7HcDj8oM+FQRF12fDJ7yqVSnKCCDlif7IozyaFZFMyikjUR0gIjDy90WjEw8NDAT7zh1TIaRIGDAGdoz9DYBCqnZ17f2h9wJnB+6AETrlAEigiz8GJEDAoRTt1RyEwWuQxn8/Ty1FIKZz2uSGRCweBIqP8OD8UNGKfIoN6ut1uImD5vCuikOnoBGk48oCPssNdrR63rdGSA6pwRZxxc7/lchnNZjMd9UIgtMMkXUSWNrLT09M0//V6nVpEvH8Xw2u321Gv1+P29jYiHvsnQbGTySR+/etfR6fTie+//z5RHycnJ/HFF1/EfD6P77//vpARcMjker1OzZ3oqwOvz8xyRXqxWMR0Oo1ms5n0Bd31/1kT5AICcs8YrzijWoge5NVCnB2VS2dqXNYz855+0xGBhN/9r7YSXV9fx7Nnz6LVaiXoaycBETwejxMyi3jcC0kEmkwmqQzM4PJ3vhEh+FnO9xjJoNigAIjAHKWgkNw7T7m43E2O4wGl2YEArZ3W8B0jQlCjDYtIjjF7UTEgb0ECNTBenFe1Wk2kJ87O/TWG9+YfQCgoqZWOn6OURlEYDsZIMOD/kK9ES+bBXF3IAA3k3IsdPlQAa4TykmaBBFB4HCRzBhW4OMQLUk9PT5M+VKuPJ5C0Wq2o1+vRbrfT/12FNYpwjxw8oVPBSuVxtwkpI5uUeVEz68e4eAkODq3X6yW6hZdg+Ngp7IE1x0EYBGy325TttNvtVOEjHUfm1n8CBJkBTprPuJcTWcAPEhQjiu/WNKeVUw/WC+ucdcK652ccuo4isPPz84SWUMS8fFyrPb4aCacEbxTx2FJAf81kMol+v1+oqmGQdlbmRvJSMc80AnLVwtGJzxNxQDn5ZvC8yoHzg1RnIeFdTAZjQCiI2w5AUlSl+LwrOW4PsOHn82CLDhcHTILSjFLzUyuQo0vzGAGpJ4Y4GAwKXIubMEEuFAbykreRDQbmvizmkcvEFVDGyXehE0BCOQcEUuP3bMGyIwYh+3w2AhRGaB0zj+S/PV/WdjQaJdTNoZzL5TJ6vV58+eWXMRgMUoC7u7uL8/PzuLu7S47zhx9+iO12G4PBIKWYIKhutxvL5TLevXuXUlMcq/sm0SscoavAIK6ISFRP3iyKw3L6hr3sdrsULLgv623U61SeMebkO9yXdxTwc3ogObGGViBA0rEU8igCu7+/T0fjdDqdRBjCF1Wr1RgOh3F6epo4H++AJ3+OeKyW3N/fp9QMjsIkL07DZVtXVyDkIcCJQCAfw3OjN/M1XgjG6Gqne3xQPhPJLK5TMsZJdCF9xRmiYLkhgB5BlcjKB+kxJjfqsrgEDZSI73nPJjIF1oN4vNma+8KlgJYYQ8S+r80RlLnyHBxJxL5nDyVEX0y8R0RCQyg18qZHkM/MZrPEgzI2qq088/7+Pn2ehsnFYpEQOI4ApMSzOGUFZ4V8803yrJErqjhN0Ck0x9dffx3ffPNNrFarePbsWUrFT09Po9frRafTibOzs+h2u/Hs2bP42c9+Fs1mM8kJ1Pby5csolUrx8PAQV1dXqf2CtfGZbeiK1w26wWmkC1NUbU2roKNG2uiKewVtU9wPOwJtud2IIAeSRo6r1arQmIw92uF97DrqwD777LPCK80ZCE2sVLpIVYhMCMpKhCGdnJyk43YgEM0/wAvZ8LgfztNQFcXHgdowiNAWlFPgiEjpKNyO+8TYs+XPg0AwYKIFSuN+OSua95NiwPwew0H5KZYQjZwWumfIzYgR++qgeSXv1yQq8x33A7EPkiBlAha5UMFDJ0zYo+BGhKwbnwXx+dyyiH1PFCkaz3WJnfckmmLAwdDj5XPoIJcj9rwQ/BvGgr50Op0PAgi8LGPASEkzsQlvXwLhwEnxVq0//elPMZ/P4+LiIqWwcLnI5u7uLu7v7xPvBbUBQf7ixYv45S9/mdDr1dVVQT/dRwXyIrAi1/V6nSrBXt98ozZzc0B0hkPQc1EkpwQI4nzWqAqdMcrN74Ef8O8PXUcd2J/+9KdoNBpxfn4eV1dXKafnLC/OvbfxYDCQmygZlcfhcJjeuusTCKzsfB+hI3gT1VQ8mCi5PguMghL9jR5MspvXwynimHEgRnBEFQzUhshlp+O+GIzbVRqUyfwEjiAi0g4IFtgn2jr9zU9UII3Hubg4AvKzQ6VaRoTlQpGpqLGFimeCDAlwbmFhDZEbuoCMXcAwyjXRXK0+vu2dAg9OeLfbpXFAPJvjM5dn8hhEgj65bQN+EtkgF2TorS/cDwTofarVajU+++yzpFu0s3z33Xfx5z//OZ4/fx69Xi/a7XbSzX6/H+VyOZ3Ycn5+Hqenp3F2dlY4RaRer8fZ2Vk8e/Ys8WWkqoyHebp6CNAwIIgobhR3xQ8ddlDmMsobjUZJR9DL9Xqd3mvJOkDwd7vdlOoSFOyocGpuHcnTUV9HOTCIyIeHh7RT3z0lTCJi76Dy84ccoTAAtyv4/B8jCnguDMGtGodK9+YH3PflyotRzyHOAGWlodLbUyL2XA8wnOhm5Afv4ndosjcNefj9l3zPvBBparVaTYURFG0ymSSZuJ0ARGZ+z86Fz8G54ExRLNbL1ULu4QDjSihj5zk+ugfOxTsuWB+cE3yVuUTGYUTmfjYCJ6kISu42G4wYrpCKJD1GPA99IyAxTtYOHoiCCuvIBnp+h4NzgOSMOusJhxl88803SW/n83n86le/SuN4eHiIVquVOMD7+/vodDoxGAxiPB6nqvHp6WlqMSElpppJQHfRCrSE7iAvOycjafO53JM02r2MyNqVZa+hbZN72TZdgEHH0A2KCz95K9GrV6/i+vo6SqVSPHv2LB4eHgpbIICF5gyIggzQJVsW2WQrqRKGZB4L74yzc45NlQUH+vDwkBwjxup/YxAYs6syroCZTwDpWQkYj3tyzB/hFPr9fgG9gWxw1iwWY/Bx15DD5XI5vYKOBYbfgnjFEBgfr8Fzp3jEPqVl0zPfYWwoFfdljKVSKZ1OOx6Pk3wcLc1zuEfMBLIJYebNQY9OM5gj60wEXq1WKTU0SrUzRm8c+SM+PN7YztFozikq62oSmUCB/lD5dfc+jo2UlmKBx8PFeG9ubuLdu3fpjVzsE6YZly552iKQKQ6TSv+zZ8/il7/8ZXzxxRcR8dgNAF2DPfKSWuac0zRcBCW/lTvnBbkANtgJaMxNqHQiYMetVismk0lhf6n1inVBjz92HXVgf/nLX+Lp06exXq/jzZs3hQV1o55PUmy1WslrOirifGgI5D1ww+EwOUCcGhGCkwcM272AQM/dbpeiNIpK71q+dYIKCaVh0hyclaMGwjcKwwjghfxcmhj5P8bntARnnCNY/m3Hi7xxEKBSnCGOx7xFq9VK6QIOAH6L75fL5XRqBs7B3JYrzibqXfklvTcCRW7Myc7AaQJpM6mfq9z+PakrrRsQwf1+P92PN73jXNvtdjo11esVEWkNkA3OZT6fx/39feofM7rACK2foLc8YHqrE0gOvs0bxT2OPHWeTCbptYS1Wi0uLi6SfpbL5Tg7O4uzs7PUqFyr1eLJkyfxhz/8IVqtVgwGg3j//n10u9348ssv4/z8PI2fv1lL7Apdw+k4DacyiC66QMb3zJl5XymOHVvyXuPxeJx8BffjJBt0hwzpGAd2NIX8/PPP4927dykqoDQ4I4z14eGhQNAeKjsjKFIgogaK6pSKnprtdpuUnMXiMximu9oh/8rlcpyfn8fDw0NSHDe/RkTiH3yZ8EcpQWDmtWyg/O1UGkW2gtpp8R2qV6QsTqW4cD6kck5r4ESYMykLVbeI+MBIIh4jOs2pOMxOp5POw3JVlNTDiMttKTyDy602oDPSDuuCtxExL3SIy1tTGM9kMomLi4vo9XoxGo0SomAP3fv376PRaMTp6Wm6DyQ+c0HWdo6VyuMROaRPLiqxJqenp2k9WCO3beDcORnE+o8srEfICnSMzqOz4/E4rq+v4+XLl0kX6f5/8eJFAW3/4he/+AB5LpfLODs7S3oHNUJLEwiLLMrca8SeVsH5sb8WJ4jDhzrwqTOsv52mT0zBgTqoGzR4/Y8hsKMObDqdRr/fLwwI2G6oDqlvQ3IuzP/peMcAQAQsJsq2Wq0+6H1yhYrqGsbhNIwICenLuIyMgKYR+96ViH1q42gUUTz5kufzHb6PcUdEYUw5wnLEYtFNmqPkdpw8E8dlUhrjAr3BodiAUH5HfVehXDBBDgQi1s99Xiiz5+x5uvpkxc6bg/m/+VJHbPr20A/kNJvN4unTp/H111+nVI6Uw/waOyDcCI08/Gyus7OzRHzTrwaaMhrnkALu6abOHKFE7PcXon84VLf+mH/CuHG2t7e3hb455k9qig59//33CWyAKt2fyNjM/QIs0A1vLCeYYT/MgfUBdWGPLo5Zx7gfOgk/il0BLtxH6NTzJ6eQ/X4/tTpE7F/2QMqIknY6nQRxqR74pEj6TuAF8jyaCWGI7sqn0uXTIyDKUSjGkUd1RxP3i/F/kIbHyxj47KH+J/gBowrzAOXyfmM33yedpkSPE3KLBFwNe+5QGC8m0Rlewn1XzJt0CoMg3XZBxfeiSmTS18oH5+Z5mi9hjN4mheOxo3WA82UnCMdnfs1Erp0UDs1pPQgFJ0TK1ul0EifltJKiBJuRaZ9gDLRdODvgtGGcME4Dgt2kNXpD9Q0jb7fbcXp6mjIL1t+pP06MajrIstVqRbPZjOfPn0ez2YyLi4vkyOkxwym6j8pFjjxDgicjiHDlvC22w/1o2WF9yuVyOhOQ/0NX8D2KWAYPH7t+jMQ/+u23b9/GF198kY7V8WZjJokXJfeNeDSm8XicHJsb09jB7q0iQEt+TuQhpYvYp6aQ0CApkBifx0BQJLY5GcGhhPBe/X6/EOHyCorRC9+HXMybG0nxqNj4mBkT2xg5/XNcOEtkQ4Tld3BBNkAXRXBuIF2QDJcJXc6Y4pnIrd1uJ+QREYn0Z/z5SZ0gAfQAx23EdogL89YVnKNTCf547lSo3QcIsqBfiXHhDO/u7mI+n8eTJ0/SfJMBKHBgUPC57BVFB1arx21w0+k0/Z+2IsbDvafTaeJHKWCYT9xuH7vz0X3SMKM9k/buw+r1ejEej1NQpDrP/E1PeEM/usu8jbgj9hvusQOcDjLhO06FQblOq+FUmevd3V3h9GLvlZzP56kdBr/C/L0/+GPXUQf2/PnzuLq6Sk14lPJRhsFgkBwYhkiqCO+B4gNd3RfFRUpnhISg+D5K6S51LwzOjr9dLYootikA21kYoi/PYSFBWkYbNng7B4Sebx1CLu6E32w2qYhhBzebzQr9XZZdXqkjurrlBCeLEwD1kDIgX8b37NmzAtSHP8v5GxyoSfe8UrnZbFLgQo4OBibpTRK7VG6E5vUGQbqyxRojL7eU2CEz9svLyxiPx/Hw8BAvXrxIqMrtOzZ2UBaOCg4YHokgAtp1+mi+DAeHLjEHCkAODASrfr+fCG0jM2/Dy3eboKek/wQSUxysGbZFr5jbJNBb97+xbnCsFFoc0LFB7IAxlUqllB0c4p2RNTK0TTtwf+z60bcSUYWcz+cxGAxSJzqog4fgSR2F2F5BRANddbvdtCmaaI3C23i4r4UFf8DlxTP0dRoSUaxwGE1F7B0NlUvuRWsH9+NewF//7ZQjj3xuMgWV4Mj5OTyDq20YSn7WO8hit9ul37nkzD0pTRsJubs9LxqAFECR5jyMqkgrcnjvk0iQBw7A6SqRFwIe/SI4Wq44W2gGjNBv0KaaBalNDxY9dzjKarUa5+fn8fbt25SugtpB9xizW2Xc1+U1oOjAGqDn2IqzFR88ib553UDP/X4/ta2wxYnOfKr+OHM3HeOMuZdTefSPai0tGdhnv99Pm8AZK+Ol/5NtT6A+0yQAAFeuWU9zbW5LwjH7pdTmoI1Ej13HE9DY72uaTCZxenpa8KBwRUSx4XCYBoIy8H/ug4I4ZXAf0SG4Cyx2pzPKYu6GyEx0MwLk9yy0o+Zq9bgtxY2YRHenncwnIj5wUJTB2byenyfPwnDiAPIwJ2QH5KjnBXWaxZyMzkBmIBFkwLPdmGmob2UBGbiJFcfBeJ12kCr1er24u7srNLniSDFkUAaICQSLPD0v+BXmRKsDDoLxUbBhnUC7jCNHr/1+PyIibm9vPyD4WWe+h76CHnCYHFfjNY6IVF3v9Xqp1YImbHTZLThGfAQQqAc+a94PRAT/iU6aAmBt2G+IrAlobsjdbB6Py768vExyd1sNbTqj0ShRD6BPtko5aFr2EfsTjqmWk7EBFmh7oc8PvTUqO3YddWCnp6fJaXS73QJXQQREGV2Zci7N/jo+G7E//wdYjmfOjYUuY9IkHCGKA79m/sAEKM8gKtmYjEKIuDgSnBzCM8TOlX2xWKTGQ58LjwGg9CZ2USj3wNhx+mfuyuf/jAek4gIHgcPIAsdg5xBRrMbB5+UnYeZI1tU4k9oeP87HaMytA8gOJM/caKaN2O9GYL52bqSMrCtr7UDmwACRDF/Dz9rtdpRKpRiNRkm2zIN7GBUQnAgg8LDIkrF5JwBO2L1soGpQj5tJ0QvWFPmASEFSTv9MjiPvQ4bPWuN0/BJeTkB2es/9SIcrlUq8fv06Go1G2hbo/ZLz+bxwaIH11UGTsbADwqk3c2QXxY85saMp5O9///tk3IPBIDqdTkI29KlUq/vjXowUgPAY+Hw+j+l0muArBCVOC4V1KkQkxGAhlvl5qVRK8N3wGTQUsTcYFC3fqJxvfeLnGI2NKs/9WYzFYlHo7I8ovmqLy8Q34zQ6yNOBiOIZ5ia/GUOn00lbVLjHyclJqnb6tAZ4JkdkyweZIl87F3NaGC9BBefuRk+itR1JtfrhiSBGsmw1w2nYmeUokX+TurgdgN+5p49udu/7Qy69Xq9ARXjrEmhks9kkVEL6dKigACriZ7wCjm1CDorVajUFDQwaPcZhkabhUBqNRtoPyxyRG5V0Aq2LBg5IfosY+gLKQ3dII8vlclxdXcVyuUyct7dH+eQTgqA7EEyVsO5uqMaufXQ96TM9d8euow7sd7/7XeqI//nPfx6j0ajw2jCqU5xISVMmUabZbEa73U6enhaK2WwWw+EwarVa3NzcRLlcTt4c48BQjZosEEdfFAmjoYiAMEx84wC5nO6wEMB0oDCGaqN05Q0DsWMAXTmCM37G4P4cE7T+vJ0Xc8Wg+TkG4gqT+2qQD6SqDdtOyOS87+emXKcpoGt3iuPky+VygYhmLZxG8n/Pl2rgdrtNnBeKz32QHTwOAYd141l5BRjZwtcyTrdljMfjGAwG6f84CPg+kBH35twu1setNfRG8co4B06f2c/lVgT00pV60nqvM/Lg5bTOELBNnAuZC2f8cVF0AFQ8PDzE/f19cr69Xi9lQ9iHe8rgwbAH1gZ9yHvu+L9tza05+Xr/5D6wr776Kl68eBHlcjlev36dlAGviSBrtVoiHBE+6ROdtygPyuQo50qJq2bASAzGFTMbmNMCUAACMxGMcIyEXDFECU18EwU4QZPnEJGdpvGdvBCBEhF1QZ5GaMzLc4yIFJkwaKdm7uViDhiqq1AYEimPHQZKRMCBz8DRcYGaXWHk4l7cG6ft/iGiP+kcMjFKAOlw+U3Z6IDXlZMbzs7O4vT09AOi3ByZdYztbKQ8Rrn8DXnudUE/eT4pMJ8xHWJnQvXWLR7wjFT24cAcbOycTUl4FwQB3rKmMLLb7WI4HEa5XI5er1c4L+3zzz9P7Qvz+Txubm5iPB6nVgzWutPpxGg0SojTugHix9G7qGf+G3txFoH+OAOCJwTlUTg6VLnkOsqBffbZZ/H+/fukwCiT+7QwTFIBohV5b97B7WohToqfOxUgpUSxKWVThbTX94mg9trmZHCK7kHJ+Rp3HJO24hy5XI10ZAFVsDAoGYaGMbmogCyNAPk9z3CXMs51MpkUXrN1aFykgO4Bw2j4ec5Z5V3PjAVnHVE8nSJHVvz+UKS1U86JZx9JTXrmdJF+QtoD8oMXMYi8CAOSIzVCt5iLCxnoLTKFo2JNJ5NJPH/+PI0dwzIvasRt5J2vd7VaLbTb8PYiuGKfS29USwbgthyn90YufGYwGESv1ys0yaIDu92u8LJo7BnC3dVXPkPhgSDpdQKs5HypsyB0kzlxhhvj9lgoYORUjK+jCOz29jYd98pN4LP8MgiUnj1Vo9Eo9QUxIRyDS7Amo3GO8AL+rHkFFMWOwPm1kYUnjqK76sZCWhn5vR0X0RxlBq3YkPPxuMJEhOLfGBfPcCMsXIbJd28fArnhdFwBwtn7u3zHvVTwTKRxKJc3HiNjSHo33NpJ24iIwowN1JpHXRAFDoW2AZCH71+tVtO4GD9r7UIG8jENALLBASFjOtXhXM25USSiOdUcj3XCe35zB03QYj3gZL3FjJTXHNt8Pk/naDFW1ttVX1L2iD23y/O9GR09vri4SOk42Q8bzafTaTrscb1eJ4qHcaHXeYsDetBoNFIxpNlspsMh86zk5OQkpblGk74Xa4Ze88dOMr9+9Ex8JsXmWHJwvLX5KefG5XI5HXCIMP3qJcrL3Ac0tVqtCmjvEArI+7hwchgKC2l0w8UzWRCf+CK1QAAAIABJREFUZ0S0ZmFQMJ7p3+enfaI0TjGstF54FtTEr8lOowKnQwQIxsM8XIny88xJuFufyOqCgpXFzp+UHnQRsUdc3ivq6qJ/T/CqVCqJAHfhxqmSiw1U7vg5yu41hVci5bEewlPiFKh40TuHfvB5EAupTLVajV6vFxGR+r04En06ncbp6WkhJTLqtP64JYJn+dDNnLoA9VNlxDaYH+uEc3f1PC8SuS+Nl5rAg3ltTOg7cwKJ3t3dJbtnVw62fHV1lYAAMjffFREJNRJkkBFtIzyTOThIucP/0HUUgUHgkbYYBWA8OKhms5lOlXQ0MHdluG94CTJy86bf+IsQXLmI2PND5iFyHoTLSMDwm/u55A/iQxkqlUriBpwSsNmYhcPIURLG4BSHBckdBbwCFRlvlgbOYwROh5ENLSlGtP4dhlOtVlNKy7hBKZYZyu6igjlAy5qtNSieFTmieEKHyXtO9KWplfeEMkfzPKSm6BSGzKZg0g0CC6/Ri3hsA+j3+2k/pJ2lUS48lbknWh3Qj2azGZeXlwnREmy5l3uZWFvGjswdfPmeq7+LxeNr0bzRn8DvXQOgR+vQfD5PdACZAfbrObNOPNPZiP+/Wq1iOBwWzttjPag6ViqVdMIsxT32ebrP7+TkJDWwH6JUsLN6vZ4oElMgh67SsX1Gv/vd73ZffvllfPPNN4XqF97TA8BoUFw8Nj9HaZzXRxT5CAoDjvQYKhHIqMM/57MR8QFSwFExVhCLf5YjSS8086C65zYGX8gGp0gkyrkSE5x2sii50ZzlZs7LvIrTUjcb2lgckZEhRs9cjSQi9nwX6Ml7SnN+BwX0kS65DmAAoNPtdn9G1Hq9Tigf4+Q0Wg7KRGZEf29fw+n4jc/WM+RL0OUiTaaqnXO2Rt3m2fjubrdLPYB+obP5PyPGQ3yOkS/66J4tMpBKpVIIXgACy501JZhy1FBExGg0il6vl5yh+U70A9oHB82Z/PV6PVFK2A/PMf3ji7nnPYB5yn1oqx3yvbq6ioiI77///mAp8igCoyeGFMkDdC+JB+e/baBeXCMEdtE7L3a7gvm1iCJEB+nwDBAIkzfEx4Hg+KzkEVGI3BiLK352SigZEQJSlnnVarXCiypAXczf97FDx2HkzbdwhsyfOa/Xjz09OAxvM+HzyMnOhjQAx4+82CjPvHxYY6VSSS9gcZQ3onZFF/lhxEaM5iFBZyAxaIf1el04nBHkSMEobwzl7DeeTbe5X8qLjJ2WR+x5JNaGih5oDJRPKsvz4EFdUDL348AHyuPCAfA34wb9u60GXYFXnM/nMRqNYrFYpP5Kt1kwBjeYcy9fVGCRfX6xdxQ0yIXO2qZAWBQGcj2I2PO/Bi9G6rl/Qa+PtVEc5cDI1YGhnjgKiRd2Vzl/s9j2ruy+j9gjnLwDGMMyEY2yulJkwtoErvmInDB0v5ORFotN64YdnA2Iy07O90HRiCTusSJSYSDMn7lyoiaHPlp2GBCRNSfEcXqMFR7HCARymj452gl4FijYRQNSPyNITgMxie3LyDViX0rPCwLcm/vAUTF2dAl5vXz5MsrlcuG4ZTtvDNV0A9xho9FIXFjEfgsTnKirlnzO72iwHpLGkvJSSXe3POmysxMjaMZhhGz0zgUIwBatU/yeOURE4dSH7XbfS0fLCRdtFbme8wxz1hH7tyNZF0Fy6CMpKNuhXPU3yjS/Z94OeVj/8DEfu44iMN5GZPQwn8/j8vIyRRx3k0dEqlp6iwHKUCqVUl8K96RREMeD8HE6LBItFeaVQCZOAXBU/NukYUTx3YNETBSZnQNEZzgFFidXQJo4WRyjVDgVFh8HYKKaZ282m6R4KAUR206JxSXqU9lhrkaPpBNEzuVymTZqg1Qmk0mqKjsQMSZHbyM1/k+VzqmnK582NJAbn4E/Ya1AfTgfk9EYA++GZL1YPwIVaxWxL4ZQZcPhIgM+S0WP0ykwGDeLUrGFQ2R8rIMDKa9/Q7bQAUbbRtFOR7nYK+ktZzhH1oc+tkrl8QW7cMbYiZHbbDaLTqeT9Gs+n8ft7W2akzlet+BY1rvdLumX38tgPpeAYa4ZcMK4mCvzYDxGgbbxY84r4kc4sL/5m7/Z/eY3v4lvvvkmdrtdAc478ubKDWfA58zXADGZGAvsASNESH2TzEzQCpBHLhbWaZmjPYICSeB8THzaieWLlPM/jM1IMueqzJ2BIthjVq3uX+RqRJJHJuTB/HAe7oInQjNHkIB5HAzf6IyxWtn8vRzNfQx92SitG14z/w3aWa8fD8b0epkPzcdgXi1ij2jQp9lslhyOnVGpVEpv2UJ26F/eiGskna8zso0opsR5JRp7QGbojWXJd8xh8TMjHJ7JuE1vOB027YAuoxfD4TC22238/Oc/T+koDgt+1Pcdj8fx+eefx/X1dUREOsqbixM/zCtbZsgTJEebR94/h04CaghYP/zwQ1QqlZ/Ggf31X/91ROwP4EcJgPUm42x8LJIrJJCuZ2dnaSEcPfMIzkT8soa8goPTMp9C4yf38P39M8hQLm9IdroJimGu5o1YIJ7vPXt2YtvtvuKGgjEfnpE7K6exyIWFxcB4MYnl5c3f3kCPvNyywQWHkc8NhT7kOJya5xHYl1Eu34/Yb+YGsbdarTg7O4vLy8vo9/vpLUEuNrA2HosrYzgo9NOcGEGQtR0Oh8l58sJb2lT8wg6f5eVCCXrFXKrVaiLAT05O0vixG79ExcjK+gYysV46nSSosNa2FXTNXCntD+Xy4ympUAegMbYFOhgTPJCdd19EPBYCeJuUgy3yIw2FenLRhIBg+3QPH880EiMt/dj1oyeyPnny5ANEwMMcge2BEYT7rRAy5+kbvQFj3RjplgEm75/zXYwOpYDP4bNU0Nw06h4slJ1+Lj6Hw0NRnDq6Gx0+AOdmhTtUvSKqsuhUd0EOLFx+MKKdRMT+mG6+Txk+Yo8MvNWD75P2oUw8E6UxJwGisNNibVH+vPPaVUd+ZgRrpAInutk8nijy/v37hJDYRrRcLuPZs2fpuRHFtzrbKTA+1hIOh1QIJ+Y007JkKxV6BbK1TOBImZsLMuaEqExiN/Blq9UqHc1Dip4jeuiJnHzP1wZbIRD+27/9W3zxxRdJxryxCVtAns1mM66urhJ9w8Vz0HmeNx6P037n8Xhc6HtEX/kbZ4TdI0OvlTMZ98c5iINGke/HrqMI7PLyMu7u7pJh5Nt7mEB+xDBbP2jI9FEzDJiUzZUtK6e9stM2UEFE8WWYwF4TmFTLnDJ5R0DEvonUAs7TEyA9bRs0/3kcRGZ3MHO5uTdij3gYmxsJ+QxVTKIh43L/nc9rAuHB18HtGBHhONhTyditYL6nf0egYfzIm/vmHA4/N2LKHTJjRG5s3anX62lz8enpaWrshPfKX8eGDjB/Ir6LQTzXPCStAhjyer2O4XAYk8kkIQqcU76TBLTgwgB0A9kGfCXpPnpPW0NEsWXHXCvfh1IB7Vp3I/aVRKNnZwDIiQZeUmo7aL7j55h3Yz7lcjm9Zg3ODMftIG906nQaJ2XbQf4GFM4+jqGviB9BYK9fv44XL14UzsIH4fgt3KAKFIL9U6Aw0jVHTgwIaJvzAwg234vIiZJMPmJ/GByfN1/kLt80aUFiBMnf5hJQIIQJucrYiHxEHC5XZ7mfUZGdsaOv/83v8ooflTzuiYL5wD3PDyfh1ArC18flGCHlUZLLuyVcWTYKY12MPJ3meW1xxlTIQPsYy+3tbUIAm80m3r59G5PJJDqdTgyHw/jiiy8KG5stz4h9MAOJ8XPmWak8vkoNI18sFnFxcZGiPimsN8MjFyqzfI5g6yo3a0pKCqq5vr5O67TZbNKbg5AVBuxTQNw/54ZU70x4+vRpcvCXl5cpCFH15cRc1t72wIGe6IjfTk4ApU8vD7qMjfnmbSkRe4SFXaGfoGPzlMwfp3aMpz/qwF68eBGlUilubm6i1+slz2sOxZ4eItpbO9zZu91u4/z8PB2l420HNmAbBm0HOFCMgdeHOXUlimAcpCE4Qcbqz/Nzn1ZgYyQCOuVDKfLobgMhShviR0Q6NsgpIgvshY7Yvxcv76nLnQU/g6v0WHCYENrMF7n5WGU/v1wuvh4L+XuLlrdh5f1nPMfjzYl485BweoyNNMZ9WdVqNXGojUYjyef8/Dxms1nae4kDZ11xKKwXOuWeQzv9H374IT0LpBgRqYWCizPGcHLoCHM1ikAHmZt5oHK5nMhxHCvByfsyI6LgOJ1Ogoh4+xG6TwBjxwJzZj8kF9umWCvvZ0R/jaBZ84j9wQVw1k4FvcMB+3UmZ1oEWeDofBz4x66jDuz9+/fx8uXLZPAoMacxwh1wDliv10uDQTg0wjIJzq5ylGTyGIgdjVEU9wDxmKTnuSgQDhDlQqjeKcCzuC+oEdRlWGzUh/FbAc1huJrlZkJHTyMbIzpkjBIzF6difB9EiKKYqzT3ZLLdcwb+82yiL8aeb+MwAZunKSAuBwMQmJGeoyr3IECAQFwZ4/7MHcdASglZ3e1245e//GWiDTjTC341Ys9XGtWDthyk2AO82WzSUermA0EnyA9HihMEzXwM/SM/O/u7u7s4Pz9PaA2ZOQ1Gh+z0GTstEaPRKBHsXi/kzBltVME5EJRtQ25rQq/Ql1arFaPRqAA4VqtV6g/j8FKca+7QHVSsczzfNkgV0tTLoeuoA/vyyy/j9evXUSqV0ivK6a1hQyxGZ6NAMSIiwW/gLxtWWZjz8/P0codarZYE5iNq80oW6MYEP87BEyYCrdfrQhOnS80+jwhDj9hDWhxe7hiMWnLFxKGDRNkE7MoRabOb/SKKaZAJ6nK5nEhNO0tkuV7vX4fGZcfutNrPclRE6fJtJiiUUStIBjLXL4JgvZwG2+hAtQ463mYGOmKsvi9jbLfbMZlM0nsgp9NpKvUPh8N49epVasHhcqAsl8vJoH36CRfHhBMAWYt2u51eIkInOYEMRIYD4MBP1teoCWdEI2pExM3NTdze3sZvfvObwvl6Efu9kG6jYW2gFUjTkSUVTbgrjtWmn4t+OGdOb9++jU6nExcXF6mgQtsE53VxdTqdQpEnotjOwr7evJLNGhK0sBk7NN7MZKR36DrqwOCbIDcRNKQ9h7ExWAbGYEE65g3YyY4iAZtRbpwD0Ys3oWA0hwydpsiI/d42k8zmffr9fmGbRo5++D/K7XQk4hFZktZsNpvCSZQsEL0xKArO3sjFnJYdkheYn9Mdn5fuTYZGREK6DhTePeC0HKMlYCBHp5A2HmTkyxVbpxf5rgD/zhybkUVEcXeDZeFWFv5mbUgpaTZlnQeDQaGxdbfbRafTScaA4YBeXBHl+XnPFagQvaYYAyI8OztL60qlLk+XaZkpl8vJhswvv3r1KkajUUJa/J7ARUB3HyX6bAdveyJgcu12u5Ry9/v9tEZujGXM3j+7Xq8L54HhoIyeADOsjwOf+VN+lm9RsiPLqZ5D19Eq5M3NTZyfnycF8w5/zi3CUWEEzp1ZLOe7EVGYBBHMkR1UErFHQnn64WhpB0K0h5vgvhgHMNfVRjtQjMGNr+7MzlGeXw9Xqz2eTkBhgrTETad5lPccOLaFcYESvEWmXC6nNgH3gDFHHCYOzuvA/DEmHAyKAupB3sw5L2ObmHUKhR743tyL+RpRgeLY08ezcofJvXAefvZms4nxeBzT6TTt8iBN5I3cFxcXcXZ2Fuv1Om5ubuLZs2dxeXmZDhLE2bvJ0nv00FUIcrIN9sGSAYAqXOzAVuic3+126fwtEOL9/X2h4hnxiEDOzs5iPp8nJIfe7Xa76Ha7qe2Do7WRi1Eu3KcRNY6I0zkoclSr1XTP8/Pz1EtHMcPbs1jv4XBYCL4uWiGfarVa0GHW2eDAfX7WuWPOK+JHOvH/6Z/+abdcLuPNmzdp4yoIajqdpmobaAjU4pSBCGEHxs+t6KQuRiR5CoRSE+Xdge7J2qH53qQqGCqKhxHnbQP8fejfng8OtVarpb2F/A4HY+7LcvAYrQieE8gD4/K/LROn5sz3UPrNPJwq4mxRMAcV942Zk2EcOcQ/NEee7bX0yar8n21XH9sryH18f+aB46HRkhQKIh+Hx+ZkgiLFKp5pVE4g87rn8onYB2U+xysISfvp7SKgO81zMHWK661ooDzzoawhgTpvgjXi3m63cXl5GYPBIJrNZrx79y45SKfrpJvn5+cRsacoIvYnyqJb3Afum3PLGJuLDW4GxgHmes6FHaxWq/juu+8iIuLt27cH+ymOppDfffdd/NVf/VUBNpKOucGMVISoQvc8EBQBmU8gB3fbgwl5DIN3/hkdbLf7ipWdVkSk1M2CsRIRMV3qjti/iRlojFM2mnNPWo4OOMcIJchTJByAfxex32CNwjEWHDQKkXfh54GHiOjoS4rCOrmKi2xAjj4LCqWlxO/vI2s7T5Q7Yr91ybsycA5eK6fqXPSyEfGdbpvUzp0I80Um3m8bsSfvoUNwdqTYy+Uyzs7Okn7d3d2lceWEP3pDUAINe08oesO5a/CgzOX+/r5Q9eN+ODccFaiMVprJZBL9fj/JlDFhTybPcfagR/cS8r1KpRJPnjxJOo8OkRJCP3AvV50JZm6PcXM3js6FLfQAZ0bGhv0DhuxQnYYfuo46sOfPn8d2u43RaJR4KyMFhAYyQ2FcvUJQ6/U68UAImdzaKQmLah4CRQKx8ZxDfArlY+7DhRDgiYyK/BkL1YiITuq885zNu4PB4IPvsJguXxMI7DgZP1xXRPGVbZ7feDwuOHI+m1dHc/4qn6dPFDB3sV6vE7HN+uROO2LfBMozmCNrZe4I5eSZXoccfVI1xEjzijCG7R0e3gmB/Bgv0Z45o4sYES0FjLXdbsevf/3rdJTMcDhMxmTnb0To56HXzIXvPXv2LIbDYQGpwf86iPM7AjftSfV6PZ1WwjHOrE2eslsHGSPjIK3zqw2RBWm5q53ujwSRkvY5oCIPc+ERxXPRcurHCNdy5D6MO+defR11YPf39+lkRZPDDJIBMiGXUFFsQ1M8Msd6DIfD9Frz1WqVohKLR8sGhCNVPRBLvqkWodhhsqBsq0CYfAZOCcHlVZOHh4dE/oIgSWFbrVbc3t6m88+53LtGJMd58X/zVjh5O2iiEHs7CRJOI7if0QxOFRmRllClckoGSmE94VOMYBkr97SToiSfb/dCKY2ukLOdAL9H9rxImTm44ZY5umDjFgmcOI4Aw3L6D5o0+jGivbm5iaurq9RLBx3w7Nmz5LCperoB2sEJWfjtWJDydM1fXFzEdvu4oRoOMHc+Xh8HectiMpnExcVFcnSsGUicrIO2CveS0WaCbHjlIYjS6ItMwOPCqfKH442wdxfyGJ95bBw96+wMjGq9q+8fu446sIuLi3Q+FS9PxduaZ8J7e48gzouFZGDb7f5NMvSIoYygAVcraHtgYblfzr0YibllAQMgonNvtmpwT5eoMQSMDkcJGkCB6HUzr8EiMiYQg6umThmoZPpAxIh9lRGFABkyv4h9scHODZSGvLyhOecRI6Kwnjh4NyjiIHDSTnPdksH4vFvDvJFTa1Alc0SGlicHEYIIGL85Hp8CAQJ1tdZ8HwR8jsqNGtwLViqV4vLyMsm5VCrFxcVF4WDD3HmZ67OseE65XE6FGgIGa+b5ErRASKwp98a+IiIFdsbOvZ3FOO2mzcP6TFUyX1OcV552I08COY7e+sS/I4oHg3IdSnXh8Jwt5cg/v446sDdv3sRvfvOb1NmLsM3/sFA+MM6keN6xvlwuU9UDR8N3MR7uR4WLvWOr1SptuwCZIQg+b2IUPsZ7+PKKjCM3iI+Ig3KQekTsN/6ywRXHnfM6RCg4MJTHShcR6R6gFxTT6bX7lnIkReS1Apl/4sXC/j3K7uoewYb0AW7skCLaIaKsrLWVzRyl03qnvqx5nv5iLEZ0DpDm+XBk+TNzrjLnJo3m0FnOdWdcs9ks/vjHP6Yeqbu7u6RDv/3tb9M7J51uORixrk6F+DcoiHVEd16+fJkaSwm4pmlAw/T9GcWwzjg/9GQwGMQXX3wRg8EgFotF4RVxBGBsYLVaxeXlZXKAzWYzneLrvbIEPusWVAi8uVN5924yNxfO+Bw2cYz7Sjp27JfPnz9PaYIdF7wGxDKVR3iUiP32EvgP4KCPbTECitifPolxwcP4eBPGg+CN2FBK7gvcpqPXXeGka9zXCA+FgAuL2Kd6OE3mwSKiDAjeaTTwGbRkhGYS11EbGeX9RDkf5XUBhXGxPi5IVCqVGI/HhX4bywXZ5309OBgXb7icTnExTlfmmKc/gxyMoLhwvO7ZYi1MBtsJ2VDcaGvnh5wh2o0ieQ76gY6wVYi3EUHMl8vlePLkSWy321QQwPmwBiBO/9vz4GKNSflYIxqhaWY1zUCWYroB1N/v91MzKgCE4OWX3HJRsbR9oAc4Lz6Hfd3d3cWLFy8KhS9sgwDEGrMm9JGyLqAudAS+2YH9Y9dRBzYajeJXv/pV6pdhIAgITgPh0XTK+feUkev1elpcR1kTtzgalItSOAKFuDVJmvMcGCDGZiP0a7dQFrgxBIYBgMDMSbXb7bSFxA7FFTlHXSMDUJp5N6dWdkiu6vhkDRcs8gXl+66CYoigF8N4qq15lZLvwAH53Z3c1zLEII26KaWzlq6M5QUO0w7u2bMcIiLtezWy8lE1NnwjXrhbUnSeWa1W075RI2fWDc4FZMvLVwnc8/k8IZ7NZhM3NzeJj2o0GvH8+fMYDAapP819dtAS6Gh+bNJyuYzhcJjGwwt1d7td9Pv9FNj8smQ7c7KW4XAY19fXqWfs8vIyZTmVSiVt9iZo0J8GYl+tVslpE/QI1gThJ0+exHfffZf0w3tnvZbMdb1ep0MgrMeuqAJWOLYod/L5dbQP7J//+Z93y+Uyvv7663j69GlCICaiXVkzOmBw/J/+MfcR8TepBK0IPqqHCXhyefXKqMECMwHIQvN894J5nO5bi4i022AymRTISMbv8fAceIOI+MCB2Vm5ggvK4HPcFxn7uzkKysfu59pRIiPPkTHkRL05i7wtIqJY4bVz8r+Rjx1gfg/zVyC0vDrtOWJwXv+PodNDTvPQZV7S3fggBQcZ9BEDL5fLaRsWRYVarRYXFxdRrT6+7ANnZp1ATy0b5mC95vO0Y6AHFL6oXKM3Lj5RyMrTvOVymU6/dZDzDg1sw1xypVJJQQE06HcqOKgzJ2dE6LLTxnwrn+WwWq3i22+/jUqlEm/evPmf94GRGxOZGbRJdkM8FItesBypmNMBTkbsjwGml8yLDJw1YQo6MYLD4ODQrMCr1Sr1riEsxk2UcSmbMTcajbi+vk5Vq4gotAPYUbOQeaWPNIax20BNmPvzXnTPy04HA+CZtFeYz2BO+SGOKAgpnreLsI52Fu4FswOy43DRg6oayklPEIoP2iLQUKjBQLjQOcZvzsaB91B6aj2ws+ce9AvSA2dkDSLkPszVbSe0IVBwYA242JeJblxcXMSrV69iPB7H/f19IsWtqyBIIw/bAfsSOVEV51kqlRJicXDy71lr+GxXbNkUT7XdPZKVSiW9R4BMymvUarWi2Wymo48IwqxHu91OSBWUx4Xjw3Hl6M10yMeuow7M79gzf2MjN8fhwZlvcApkb48yL5fLgpE4ghvZWZm85YWIGREJvTkl5DIScoXUXcI4zJOTk/QOPY8JBMQ4IvYENgvImBinkQCfsQN3BMsJUVduzd1g/N6Ia8UwJ8cY+AwFBsuCNc6LEcyP8RhJGqk5mjJHNz6iJ+ZZmIvpCa81+sT4vKZu1+H/9DnhHI2Sc4dEgGX+Rm6MJaL4Dk+qtKAyno3uQ5/QMtHpdKJer8erV68SLdHr9aJer8fd3V2SjZ0YOsD4nbqXSqV0MgbVTLIi7s8mcE4+5ufo+ng8Luw4cDbk9hR4yYh9s7XRPJQMJ7w6E2PsDrp2iOa/HNywd+yQ1oxj11EHhvEbhm82m0LKgyBM4jFBFhQlgnPKS/IgE6cGTJRnUwVBoESJQ7CVZ5hjMVIk4mKsRCNHjLu7u0LjLZGJQgLVSVfeIFo5bsikc17a96kRNN+SfviARuTLz+3IkJ0bET2n7XZ/IKKDjquijMVkuZGTN4ZHRGHcKB3zY6y5TuTpNHtF+RnOjxTO8/Pl+0A18H/vD835NNIh5uZmTusRc6Pb3UgbhI2TNFIC9bDf8vT0NM2lVCrF7e1tbDabdDBjuVyOu7u7+NnPfhaXl5dRqezfLLRerwtv2jIXaceOLroIgUyXy2Wcn59Hu92OdrtdOK2Xfi4cB/dFB0ejUaqiQtrDUTtjMhcKuEHHDUAc1OHY8At2uk7RkW1EFALVoesoB/YP//APuxcvXsS///u/R6vVSpHQBsnVbDYLXeJOF0njaIEgogDVIfh5fZMhLGiGaqQ5pYgPkZ4rhR5fzi3kQkGYQGY7JTsfOz9+h6BJvRgXjugQZ2fOiDnmMs3HaAdoBeHf/jvnpHJuyjJzOpkTpoe+x3dzPsqRNn9GRKSGRwh5c0A+wSIngbmfe65M7jot9fz5v/+dFy28O6DRaKT2GPdOISN3oec8FZ9ztoETYqyj0ajwchyOqLq8vEzBZjAYpDO34LhAUi4WeR0YG4jSSJLUMKJ43Lqr9qyjU82I4rsrHZhy1MbxUZY/c8xtje84ELL+PJ/PPDw8xLfffhuNRiNev379P38r0dOnT+Pdu3epJAv/BSLhMDSgqRcUY2QRyZ8RBI6O7zlNgB/A85N2WDAQ+wiAJkjux2KjsAjY3Nlq9bi/jDldXV2lqMNCUA3jO7SLcNHvwjhYZCItlb78915o95nlDgNl4/M2LCOIiOIOf8+B6MvPcCCgXhwAMmXdQHcEBozF7wV0as3a+Go0GomfcTrrC94uovhaMAKYT6rAeTmtNVrLZR2xRwjr9fqDky94BuOv1+sxnU4LTsFO0fMF0fB95MHpqP5/tVrvRPBoAAAgAElEQVSNp0+fRqfTiUajEd1uN0ajUQwGg/jP//zP+Oqrr+KPf/xjfP/993Fzc5MQ0atXrwpVWuTvrT0UyJCPdWu1WsVwOIy3b98mSohewuVymQAFKCiiaIMOkrkTozPADdXWAWcGpMe0mDhboHDCz6j0UnDyfPLrKAL7+7//+92rV6/iz3/+c5TL5cJxxU7zMFg3kHpRnXoxYHfXeyuFy7ou8xuyOt+m8uLobWFzT2Bxzr9EPL68BGI1v9xbZY7I9/fnjBQcjQ2lDyHAfJGcophwBarnVUi+g5zylBUn4ec69eP7vheyQ96sPfP3fPLnRETa7Fwul9OxK0ZTEcXXhuXyMIdlZ5lXNDE4CiOmN3B26KR1Alnmuxmcqhplcn/km1fXcjSObdFK5P2ndsAcggCF0O/3E5fbarXSW7QXi0Xq54qI1LKyWCwSnXFIN4wkXcFGNpZljlhNu1iu6Ah7pHmjVF45PoTIsG3bkVsr8DWj0Si+/vrrqNfr8cMPP/zPq5AvX75MR48AfR1pMES3VxiFuYuc/BzPTveySXQW342eefXRx+igRBy8CP8Euc0GWAQUUXzX4cnJSdzf38f19fUHgmeu5oWc0uYNivACTmmdbtoR2pmASpgXP4dbwkjseEA/PNOtCkYa8I8R+6NVPJb1el0wNkd49xlZEZkXYzYCItC4oABCzNsw3GBrZ+n58n92RyBfEAiOzHoIme/eIgzL/Vbok2WJ3JH1oeCFLJGzix7MxWmcv+fnuQJnQr5er8fNzU1yfnd3d3FychK3t7fR6/Xi888/T2+sQt7OEEy2H1o3ZwugQ9bD9hWxR8MREWdnZ0mHarVa3NzcxHq9jtvb28QpGulz4STzYlZEFDoDkCPywXbyAxfy62gK+fXXX8fZ2VlUq/t9ajgwHgpnhCPyYPzSAH7ulggmbW/c7XYLWyBQQKodLtEzjlarFd1uNyk6OwPYp0XJmef5cDscHgpmI65UKoW9hBybbAU1l2NyE+V0ShZR3IBuxQPe04fjjfGMi/vAFbBXjMuoD8Xg2GO2X3l8vP6OdWQNIP1dDUPOvrxh3FVlnpEXWhhPxOESuauMRkHIBuSQO6mI4lvh3buEHDxWj4OUlN+bH/M+VBue50ibilNefu7KL2tuOSAXAuRms0l63uv1CocNcnLqN998E3/4wx+iUqnEz3/+84S68oIbaAsnhH461d9ut4WtQVASoLJWqxX9fj851sViEdfX1/H69esYDAYFrtlpP7qIs+NneTaC88z1m/X3CTcfu46mkP/yL/+yu7m5ie+//z6Vgpko21HcbMmgvPeOBXe3OxfRGrQE0vKeKVcoybnzEjqKjCK4WdOIECHf3NzE6elpQaCG3nYgFrCNylFus9kUnANzy9stcA4YgBUKFOt03EjA5DHRFD4r30JkhIVDYLsJ88NJe8z5z/P7gDpZZ3roKM6AmniWEVdOePvi/hiQlZw1QB4Yi1tf3CXPDhGjh8lkEmdnZyn9cqWSz7gFKOLjzbs5YjRyN40AZ8i9nZqB0vL3IoB43bxdLpcL7wjNnSkvHeHoK95pabRjJ27d8lz4XLfbTRlSo9GIv/zlLwmI0L3PeqIDvnIZeccOl6vzDiAuEuIwv/3229jtdh9NIX+0Cvmzn/0s/uu//isuLi7Slgx3qmPYwMjpdJqUJZ8Ihm/uxEQ0nAwNshYKf+NcXIb35SZJ7oMT5OhdFgQHSvqUL6h3D3BZAfMufu6Z9zUduuykrWyuqPnezJ3P59VePouc7Ehy8ty8oru4CQ55Vc/7CCOKp+zaoHIEytzd4uDKHCjHvJ3XnPtZBl4HuC8bVb5O/M7pML83P5Y/02tEYORQRDuInBPyOlg2eXHGKR72Y53xesEbsaYcDVWtVlMRqVKpxG9/+9t0vNNut0vvvEQfvLWNijvB//T0NDqdTtzd3UWpVEqHOhLIfEYYMu/1eikbsqNnLRxATRfxfX+GC3/U6/ViOBzGn//856hUfmInPohgPp+nvWP5YqzX63SWULn8+H67wWBQUAoqFY5EEfuKD4oLaU965AiIMhIdjFDMq/io4G63W9jDNRgMUiMgz/H3I4rGWa1WE0GaH7rGq7MMg2307sgHLWEMvo8dn/k/HEtuAHZeGLGdr4lYlNeIxKkTY0Qp3eeTp0BGjhgZqZz5PnOEoEo6v73LgO+BrkFPlomLQSAQjBvny2d3u10yFLcwIEvGXiqVCkETxx0Rhf4q1itvT0BGRnDMk0IVvGlu1CBKuvapYDJWZxak8cyvVCqllgWqmNgnsvnXf/3XNA9SW7r/5/N5PHv2LNEGNzc3UalU0okYdNKzlhQdWq1W2jvrTfO0PvF/xoj8qVQbdaPDbk2yHbhQxdxcdDh0/eiLba+urmK9XqdUxdEBTsj9PBHFA+4YsI1hMpkUNtzSRkBEg/NwXwr3tnE5ArKQJgXZ9kATIuX8iP0Jq0Qmxm1UFREHnRdvYgFxMgejBL+VxQ4HOeCMbLD0umGIyBZ5M3bPOSe9zWU5tYbz8rlrOGnkaMVlzjzLXB4RHcNzusXPKMC42OC00vdn/nkzLM/mPu4jRLYm65EDDo5obrRp9I6jRwfszHiuq4sRxWNvcP55ddJIMc9YuBeycHUP/o5jqSDWQS3m6uBicSy1Wi0dW+22FV72u1wu4+rqKqrVx5anfr+fqA/0g7mxA4Vxudo/n88LlXEQmAMGyA6/4UDi7/JZ/Ad66AILAe5j11GG7I9//GP0+/3UZkC3trkH99UYEVGaLpVKySFxIiVd7Jx9/v9I+7ceu5Ijv/9O1oFk8dAkW2q1NJbGsA3DV77whTEYwHPhAXxhwG/BL9qABXtgW+PWwfKoW+qmWMVTnf4XfD5rf3dws/pxTwJEFXfttVZmZGTEL34RmasT683VRWw8UXmRbvjtW4b6/GfPnu15Dd+h+D1GZq21ZxgdFT0XGoXHs1UhS17fv39/exGKcLpvITJBHRNCHWlq8RdpzBdhzN36JUT7b621vX+zqKDXlHfs3zREs4W71tqoA/LTWsDb7TLGWrSsJklDKkM8LWOoUtdI894Mpzku6jk6Ovpo/DgjiNW9DiHMmXVsyImD80yGGl1BXuU86xAYAiee0Nn+7idjw0CjTB4+/HByhmOoIP6HDx+uH//4x5vjVDgLjJydna3nz5+vtdbeC1GMVSTA2PqbdUwfSgGphQMUoFvypTdNzOD4Li4utvIbyPpT7U4D9q/+1b/alMbplASmHR8f750ywepb3LIqrLmOQRuMEasMtvsc2qlXhkh4KcrfCt8f/ehH6+XLl+vNmzfb1hCe6PT0dDsux6J9+vTpBmMvLy/X69evN8WXyWMAPa9/t2h6iikoT1llbWYWdsrVApqLWGamBLGFcnt7u/e6Mn11zxK6FswsEK7TOTo6+mjDOkW1HUU46DuT67IAGs4yWBwIQ4EUpvj6a4FZtM2EMjhz+5GFYnGYYwR5WxdkE0NFiX2G+9B3zqalFRpnROcaUpb28LtEFgfse/SHvtIbztfrDlUEcDLdHlbEfX19ve3TJH9Z/bnhmpytt/KR5pje0YfHjx9vRwJBmnTg5ORkkxdkx5iRt/Vtbn4wAvv666+37NrLly/3BF0ICEJ28y1l6ndxSrJBFj5ysgRmF7r7TeUgCPzJ06dP15dffrn+9Kc/bSEcD2krFGVGdLoXNPf555/vLcCGSuXutB6a5/41LmRUI3B8fLyNycKvQZvkJqWzINwbGrG4GdamrtsP4zlUid5MbEMgRnqtD29iJk+LrYikBqPHpPDOe4o3khJC/u4smLV5t7e32zEx7rvWBxQFJVkQFmfR+dxfenR0tPFQr1+/3jN8/k6ORZ9QHGfX+5bnND/VlzomKKZGjf44hRgKkWwoF2mhP378eFvsa+0yqp5vLNUDxrSo8vj4eK8ivw2POHnnoi8hprdCcaCoJ8bSeWp06fXr13uRkPHZmH6XAfveM/G9Xdrhb8fHx3vp6JLCjuOFwBoWMSJrrb2tAs3Ute5mZm0YFI3yW0hOAHj37t3eQsNzKHZ1PwLt89da29iEsCaP4ashYmzds8anylKSvSStvjEojGER2DZRuQ6aKYKrk0CaV9HW2lWIN5xrfycxrk2jUmfCUcxFAg2XAJ+y59G1SXo3JJ/z0JDxyZMn2wIQcsiY1zGUMysNwjj0hS6eKyJoQqkhaTOV+lSk0nKTtXa7H6bxns0Z+WTWjDqniofGl/ZdCJMzrg5B8j4rf0VWU/dmZtpY9E9ipGvWHLf0yXcdhAnBCrdLDX2f8VrrexCYKtvyFJeXl3unm3aPo6xiyfXGuLgwoUs3hddDEurMQiiWFIpCXp9//vn6F//iX2wGp0S29HMX9L1797asqdjehOj3hNENBed3D/Vf86aXTvRau7f0aBOVFT30nYMMAq83DZj6nfJ5ntfvtd9k8OjRo70+8coMnPk3xxCEe/nZM+TojdC74e2s6etzy2n6DNqxyBiRjolBkvXE0ay1T/RblPrBEKA8hGDl0nyPEZ9hJoRcpIzANxed37Ozs+3N7q2NAhgmEqphpLMQtbpMkUlrqnxWbvfx48fbi3rq1MimyLwcrTVF1r43gYCz0sxHW9dWd4BM4+v6HxxCOsL27du3e2/OoaAU5NmzZ3ulFsK6o6Oj7QhZHVprP7NHCLwnLyit31olfAgh3Lv34c0xv/nNb9b/+B//Y621y4A0TS6D6jMozcQ1jc2TUta1duFGa5x8bmFWabugcH1CFen2KidOyD3dg0drZpQBxudwMNCtTcRFhxTMwjPOQnahfjNm9YRClW6D2ZQoaLZckmubXaSsWhMGs0Ec5fHevXu3bcBn5LrQen99oWvl+ziM9okcEOl9E5D/I+Xt63Vd10ar2ckFyiATnGh3ADAU/t7wvPVw1a/yd8h8rWU4HCzqpPs0i5rJDQ/NyTDmdRzVITpL3j1LkPzYDc65yY4ias1YfjCJ/8c//nGdnZ2tzz//fM+L1avz3iy5B1t06qW2B6bzvILspL9beK3DITzGdK0PVv7Xv/71un///vr5z3++N/girpubD++9UwPUEKFnGbmesWr21CRLQrj/LJPoYvV70adFeHR0tE3oWvuV6v5GMcmw6KPJhLV2W2NKzLt3+YW5G4LSU8pyFxyB+eume2FRDZljaBoy0guZKX1oKCJRVMPueo5TvxDTx8fH2ykPa328lcjYoJW5jQf32vmGMFAZjx492itcVaXOIPm8svYd4+KEGk0gvmeoTv7Hx8db9T0E7hQUTncCALJVf1d5io4axpHRzIYaB8fecFVSzbw3aQa1di78bFKBo4d2q1/ty8y+f6rd+ddf/OIX6+XLl+sPf/jD3qbWtT4oJoWVJbK4zs7ONiK35x+ZVB1zfHShaBVprf2aMkL5yU9+sl68eLH+8Ic/rBcvXmxvLPb8HgBXY+SZNRad5HorBqQcB6/pO2utPRSH6KUAJgl/N/kx4VX5PIrcMFwhYQ1PjZJrkKRk7/pyNPXqrqvnngak/YHE3r59ux4/frwng6Ojow0VQx91KIjo/v2QY6inFibjgywcaKIZvI6rIYgxF5lNJDgzsb7THQJv3rzZQxXVTToL3R/iC8nN75Nzqs53POYUKpxZOf0spdCXmEDwjRLcU1+urq72nGH7N/mzFvJWl52e2sQSSqSRCRlB9Naf9VEj79k/OIR0gx/96EebsGolGYO+BslE4XuagSopN6F+Pb/n9O8s+C9+8Yv11Vdfrd/85jfr2bNnmzC61QGpT8BQ48z0eZ5QhQK4zlaiOcFVYvdF+JJT66Rc08Xh2Yc2+FI2RqdKX6QxSW/IdL5o1L/uPlCf1qY2b1OOcC1k3O82za2vZNKkS3mSzn/3DV5fX+9lsfFr5ZIa4jTkbihZLpIBWGufsyx353meI+SnNxyWk3qdhb/WzvH2ZJWW3Mi+6U8XIud1fn6+d4ZZax8ZpmYc19rftO9erUWjuzKKDflqKDkMWUyfqX80pzX09Bqy7JhaFMuwQoKojhZeG5v1Rcfv37+/dzjqXe1OA/brX/96vXjxYr18+XK9fft2e5ecB/udgaDkIK7OlNyuAdEa/xP+DFdM4C9/+cv1+eefr5/+9Kd7fX3/fncGOY7EAmHQ/NSEIUIl/I6jeJWG4JrWWhsJ6pkN08oPFv77/8ykTq+31k6p1tonnfV3rY9LUyxUi6nlB57T71Gulju4r+vxOjUa7Zt5JDt9t/CL6JrVqnGZIWiNIJlULvTE4plckUVQI2Zeyvkg+H1uAeF0D2Xu1lrbWCGxm5ubvVftrbUrIeJwheLNkuLTlN10DbkXoyqU0pfSM3SkRoqsTk9P15MnTzaZlq6ovNfaGSZjMnfWUPlYYXt3yXRdK6YtYEBruE6VAJmWp6yxn4b7ULvTgP3zf/7P1x/+8Ietg1988cVa64OlbSp/rR0/wmCYZMfZEKyfBgAtgI8WPQX/7rvv1meffbYeP368fv/7369/8k/+yVpr7WU+1/p4/57JbpwtO0NpC81b//T27YeXeJYngOyEEp5dbqYKb/Kc/21SLJpJsttcWzKzNUaUpRtjeXuKd3p6uleD18wXZa2R1YeGKmutrTbHIYSHPD/qoNyIeWs6v4uy894tOfpqcUyFpWscwDSQRXxFiRIInBRjUBR6ff2hMPfevXtb7ZVFZQ58HzXBCJFfIwjzqR+ymkVJa63trdr0rWU11kWNd+dYUz/JIdV51UA3oulRVdYL2dXI1di1H4wxWTaspluMPIPlvpzl69evt8ipDkIzThX8d/Fg33se2Oeff749xKui1IYd2rBpG1C5L8dRK0J05M5aa/MSJmStXSr8/fv368svv1z/8A//sL766qv1s5/9bMsMzT1wJo2gfaccmglca5ehoThNOuC0KtByFoxGXzTSZ1hsrYGzkD3bdwq3uz2GtysKa/kE79QMZdPbwlPefK1dSt+ug1lnBXlReN4daukJIMfHx5uBE/55wUS3ihWl6ptrZMMsohLrXczlDNtXaNgWKdeRoYUgCSIi6DFLJycn26Gd9rW2lRfl9CanwyjUYPg7WUw0Zf6g3CaFJJu6E6LhcB1KSXlHvK+1v/ugPyXO1LbJjsvyCyFL+7TmkyzRRjWY5W4/++yzPZ65ZRo18OQ60WHlfle78zid//Af/sPt/+9A/b3snQng1XAzPGDJ2ZK1freYKWwLEQkNfP0//+f/rOfPn6/Xr1/vQeK52djP7uOCZuYWj55P1LICi1qI0UxIiV9GWEjK2DCGPX7G4mz6vB7VZPX/7Zc+tP+f+v4Mz8i7xYGzgNg8ddfDPAds1tG138bYxsjrd1FV5WixqgecbZbbNCs15VyZzfPRuuD1q/PvPg2j5gJqISkHNsfVvvnM8/vdRirN2EJo5UD9dN8mVCZ6MYddGwppW3NZx1DjYQyTI57r0s/ajvLLRaMihm5Ud+8i/xYYm5fLy8v193//9+vk5OST54HdicDOz8+3424nhwFtNBz67LPPNq6nStK0rgkDd8/OzjY0VuLzxz/+8fruu++28gcKaXLB0daYdQErIJ2V7T3CpLUuPZGCIrXkoOQnL+L782C3Lp7yBSatymXcPJ9ryLdHEDVkr5GiBC36paz1xMfHx3vZw/IdJWP7shae1jWzeJGTgFq7UDkuBl4/KfhMl8+F1HozZRg1FD1Nd60dArf9DWLT34aDE5W2HouR6iLu6Sk1LGRVZAy5kk+TJ3ahkIs1dHl5uSHWx48fb7xcnw8hFR1pUFhrGk9OTrZxVTdcC1W1uNo9GdieXgK5+V4dEDSsv9bZBA/00rPtwWy/oFJz/YPrwF68eLGdfS0bR2HL/4DR3U/mzScmjWcv7HS/etEvvvhi/eEPf1j/83/+z7XW+mhwJebXWh+l8y02SkwIILN3K7b+ZG4faSglfq9SrrW2UJYRbJxuwsmimc3WabXosN/vRM+9ckKuk5OT9fTp071F/+7duz0erWEqmXWfXAnWwvqS5baQeb656ni7cFoTCLXLrjFglJuTK7ldnkw/3bP9omM+g5iLnLszg9HRNyi+G4jpp7HIQK61tt/71qurq6ttVwJnOWVjnsu7MX76UXTUuZGUOjs723gj48IXNltpHOrXOEmGBx+41s7hM5qzcTA1hvQRb2xuzaMQvC/CrbH3Xkx6zBm0vMQ8l076wRzYt99+u372s5+t29vbjcxv1fZMV+uArKHwQ6vyi8WbPXn+/Pn6+7//+/Xll1+uL774YtufNrfonJycbAQyL3GIM+nAnUjRvx2qnK8XMWEWVb1Ji22LroQox8fH21uaNUYKPOZZi8rIpN6y4yiXN4tGPaP3k1mDQKCYLpjyPodI3MqjnEWfI6EwQzx9cJ0izdbMNXGx1j6BLSvcViOw1u7kVEmMomXPmAmCIhyEsZ0ePrdIG6rhfTlE/WgZAI6NQS5ynsa5W2msn4ZT9vc2qlHqwXkyNGutbaM02ZQsL+9pLoV4PmspBhlOvmoeRd69zRzMPFjg6Ohoe08FxzHXa9chu1Bq5FC704AZ3Onp6ZahIRSCZmWbBrW4Fbc1PLi5udkObHv79sM77c7OztY333yzzs/P109+8pO11n5ZQuP+VpyXS9Aar3cRQlld8H3RA6OgNqqclWdMfsGEd7Genp5u5HQ3AbdvjFshuQmVwamiGx85FInMGqOpbPfu3dvbxkLxW3ndKvzJUdVQVVb9f8PWWXDao470E79ZBZ6hdY1GS1PqRMopkVMXgQXQQtM6MCU+To29urraKxgVWrVBI6UFWlIwT6MoaierGl/lKozG5IfMRyMRxrW1WzhL9++cQoieNwn10hHl5UoFMLbTwNLvlvYwZrPA1vyoDMAfM3KtQWzZy13t7jr9tdvEXIPAu/AKFUjLGkrw46W6OZzgGToks1DQMyChVvVWGBMdlF+YFc74tqaTKYF33K31weDZvsHQWATlnlrfUk9GHh8JPN6mCk5BLIYaZTyDZ1Iwxqxopg167CZqxqOJF166By+SOThfRe8Y6ySEhZo5Rj+QCcM7ZWCum7Toc8pVlc/rPEK5+C/y7JlcPi/HaRFxrKenp+v8/HyvJmytfbRqbuh9w/uG9lNuvt+QnZNpwqt/a7lJDdxa+7V1DHadDyAAsUHiRcezJrC0ylo7rrkke20CPsuato4b6ZBr+WrAgENumU1171Pte0n8KbR2dmYPoAf1KQ0ZG9r4/vn5+To/P1+np6fryy+/XGvtCiTLi6k3aXaRIjAyRTldVK0ONlG+x8gSXreBFFlAmC1vKMIxqSWNa7g1fWjTnxaENhz2O4TnFI21dkXEUJ/vV9ksFmixfaXcrrWgelZaZdRWw1jEVCUU+nRLTve2utb1RTdTRuVT5tzw9sYoBJ+7OcjEnDXxMmvjGNRZh0QO5fG8N0G/m6V0TSMGz4ZORCXQSI32zPZBrn/+85+3PjbJwjBPKkFkxPl1b64+lDhvnd7jx4/3Eib0SakUg4lCgcAk2yrXJt3U31nPdu7Qbett6t6eHn7yL2utf/kv/+WGPIpOLAyT1ZqsGq2SnWut9fnnn6/z8/P17NmzrSj2yZMnG09EEdZaWxjFI9/e3m58QBWBZ5lhqt+LHqqATafXg030U0Pq7xcXF3thnZ81vDME1aps5UbW2lXYUwTf12/wfq0dCcsYlKBu9TIj4/qZFKBYrqeIDe8OVfLrg79bTGvt71tlmI+Pj/cyaa6f4T9UxvAxRp1fPBRn0d0D9o2WoyHPzgOjbwE5NcW/yc/QY/d+9erV3pg5r+5cKCKDMuhkeU6GhK6Tw+3t7d47EhoJqGZHV0wivtwc9M5wmveChZaQGCfSXdSiza1uZCVJ1qQdHTFOOq7uzub86kL7d6i8pu3OOrC//uu/vv2n//Sfrv/6X//rdvyMzjp8TBjVhUHZDE5R47t379aLFy/W7373u3V0dLSdfrrWfhGoRsFM4NyfNY1SFa6I0e8Up9fXMPDauJbyJg39GA6Q3eS3yHEarSIkk88zq27mPacS+ryp9jYwvXvPJgowFw1T+7yGG7P/+gKRdI6LphpGt2+994MHD7bwQqPYNfYl3Ofva+2QnvsaK97RXDS75dnmuH2SZezWmfKZjTQmDzTHWuJ56kTHN//fYuZeZwylRhgH4+gzO3cTaDRT+ubNm72ayM5H10tl7ffK7xBKmocgzKhhorMiLn19+/bt+uqrr9bJyQ+sA3M8s3CwNSGtdRJmFI3YviHOfvXq1frFL36xvv322/XkyZP15ZdfbpxY+Q+TVaK28byJLClZBXKNSS5vNav2ZeiEGp1I17Uo9+3b3RtZ2jfPLvKoIS4JbQyF/FLK+tGwobxP69Xabm5utlDHYZOebxE3a+v5DaGur6/3kKwMcJ/RGqC1djVu5QCLXBhnxrTGQLiCk2MIEcWd02bRtKJ995566HoIk6Oa82MsXmhBV3B1DH0RW9GV5whnfd+cXl1dba8hc61r/M7JG6f7NwNvrTBeJf79XpTaZt5lSUUykjqMtuu7JivT0glNmnDy0BkEPeUsGQidl8czvpbglKY61O5EYP/23/7b25///Ofr17/+9WbAyid5sHKBhj31mmvtQqCeG1TlaTxPIIcWK0G1apmlJ4xD22xcO5GGzJz7UkpoowimfEOf5xmfQoX6OInsepw52RDNXDj9e/tAJpSySRb9rzdufU3lMcdS5LbWLjyc2TJOh1ExNuPUyJHyCwfLudQQTjlb1HVGdIHR8RMy7RHbHSODNtGEAu4uQnOuxsnC6q6GLv6GU2ofIfuiWLsNqiNTr2Yj185nxzCR9NHR0YZwiyYb0lUv/N513LAWiiv66q6J9sO13c1RfSxCLvo6Pf3wjs1/VCX+F198sS4vd2/o8eBHjx5tFr/ZHCRkrf+XX365vv766z0UQJBi3xLG3eA6J7Zx5AwAACAASURBVK2hqUlo6Ooe3QrUcEHrwmvBXcNCE7HWjkfzzFlcV0PQ++sXJWlMX5QhPOYM6vFaDAtNKEhsGMK7qouDBPSrxs/RNMZRrpFSVvkorgUvjIUYORr1Wp4rLKakjo6hS94ZWONFNshgrcRwM2Dlobq3s3yT7/fv5hNqVR/39u3brQi6YdxaH1CNMptmy6CihnqVI+TMeEF7Jycn2za6+/fvb0aJcZ3Io84Cb9WdAfTD/92LzOaa6oGE1pT7eOO3xtnQEcivRcI14I1mimjds2sBci7Fw7h2nRxqdxqwV69ebWd2l1P505/+tFdl30P5bm5uNtLv6Oho/fKXv1x/+Zd/+dFZTjxwLX2teENU8brnlQugMAQ2t07I7hCkVt7EM6rgNVIETUkPFY/O+5e8bW2W0KDZl8pWHwqfmwnDH3V/G2Vtur2GTWHxWvtbsfAmZFayt7+vtUs+mDvyauZQyOXfRA89mBF6XmtXZFp04L7GLdNoriZ6nP/ayLYoiKPt6QlCQMatBcdql4peioSKhlrC0cr3Ih1yf/PmzVZAy6Ca05YS0Qe67z7Vb2E+Y6WPCHn30I+iuPavjl242tC7un+ofMcYej8AociQ8WzW2PP079D995511x9/8YtfrO+++24rddBm2hpkfvTo0Xr9+vX64osv1o9//ON1cXGxfvrTn26LrfCfcs7QqDxYLbBtM4wHZSzRXa+71uFNtJNH4R2ExxCC4kdClMGrh+0CpKjQpO9aJJSnyqA0QyammbNm7njDLk7It16act67d297VyDE1pM6mtI/Pj7eO85ord2bnrQmUciVvMyfejRz2oXSBdakjTF3e5PvHdIH82IcDVUsbP8Y35amFKn1oMIWfZKHa8l31pxNbqf1iebH2HCtNc7lytZa2xl2ypPKVWp1eo1iyhm7fxMNRUvV1TodjrnOCxUjBG1pirXV/lnbjXoa3pa0lyCpAb293X8nBr79B3Ng//pf/+vbf/Nv/s36u7/7u21x44Z4UNkdaMeCBJunJ19rx3NcXV19dGDZDNXqmYuaLIYWZRLEWh9nJicP1v/7fiek6Gzei1ErBO5Cash6SOHJg1LMDGzHcohDkGlD/Epy1ChBEOVbhJfGfyjrqfavXh+f454zwzn5n87DlIFFamwNi9oqy87VlO/knZo9PsQRWlwQaMsgWurgOQzS5NBmlrqc6USAkDTk5VkWcLPgmpooci+HSb6HqJH+rCyLxubn5tt663yVkytCZpCca2Z8QEHpjUZZ5sqWrbXW9qIbf8eBXV5err/7u7/74RzYz3/+8/WnP/1p7zwlXv3o6GjjPB4+fLi++eab9eLFi/XTn/50e1GlhVml5hlMHgMKzlpUJXJngSmDgVM4RHRS9k5wkwL6NUlpSEJoTElnNTEjTRnnHkMLg1fp4pvZvHpaPye0LxdVQ+X/5TsY+yofpbi62r2dvNty1lp7J9l2QTF2lV0XU+WqL/o6DVPnVV99p6E5D1wDVARGZ/QPEnn16tV69erVxm3VCBYZ4EfNy8XFxXa/i4uLzfH2hNVuSVIHxjDa1N0w1XOPj483brNGX8jEaXfbGb0WVh7aO9iiV7I+tBbW2tXO1WhX12aG1tz00IZyZGvtXuICTfWt6V2vBRh0jPw4SPwae2A8E+nOdudfZVxAYTyLSaTgFxcX6/nz5+u3v/3t+uabbzZjd3Nzs1fc19Cu9S6e1Ti5cN/EMYizXIGV91lj/Mba5VEoPW6IYpu0hmh4JMIujG/SgUx6vA4jCw4j2I+OdqUm5VDmGHoAo7E3VPJ9MvCcGsjKbK39F0n0eba3dH56n3J4/VdZk+NcAJ13z4ZI6tjKG5VHY4CLGj2zr79zpHHHx5AxFg1RSmwzaM6/f/v27Z6DFPaVg9U35LnTeluicnR0tJ1eAnm5FolPT+o824pEGQQGvpk788tIW0c9IbVOpDpSA8UBKeydhLq1rU/6z9kaXzdv39zcbIXrM9zHLzNi08h+qt1pwH75y1+up0+frqdPn27KpKM2oF5eXq4vvvhivX//4Ux6ghPi3Lu3e+W9UKipW8pHqQi4Ropna7jZs+lNZhe1vlJqoRMvQcDdQlJD6Jkmr7yb+9dDzxqzEq0zLHaPTjiFgaJ8No+q9uyixxoCUBxStNjIqE6gsL7ndVWxZip9ekSoqaivW5naTwvDwnKNJJBTKIRN5N70PudwdXW1JQV6f4uqRZ2vXr3aDBW9gDS9Bk0o1ORC0cjJycmm96435z05t6i64VhpE4sfkd8z6Mi39ETnndygpupYnVTnEoLjcIvayrtNh1Yd13zeGr9Jk1xdXW36p4SFUeco8b7Vl0Zn8+W3n2p3cmD/8T/+x9vj4+P1v//3/17Pnj3brCer/OzZs7XWh6OmvQS3YdvkleYgTXDj71kBPbmlTqY4vJti11oHDU2fz2tDVYX7BNYtQQjfyfWY2ENIsKUYxtViw8l3USZyqZdca+0R8vpcDsZY19qvgu48tJV/mp9PPqbzxGDUyJYXo9juLzSpISo/Qj79/5RJedTJU87+VxZ1Pne1zl/He3t7+1H20331qxuXEeWcFGQxTwhp6cNaa9ONzlGRkrVQB1DDWAc8yzva9M28Q251EH3e5FPpmznwwubKmu5VH6eO6iM06rntC8P6q1/9ap2env4wDuzy8nJ99tlna60PYaKNxG/evFl/+Zd/uS4vL9fXX3+9nj9/vnlMi4+XKuyFDDqgepSrq483cfMgRVbudXp6umWHSoQ2/p6Lw08nLzSMsChbR1ZPsNYu5V+vaXwgMKWqkRV6CDHW2hmLk5MP9UCtg/F3Dfq8uLjYtr4wJkVvNzc3Gz/XOq/JJ3TBus7/6/XNGcUuahOG4w2baSP7lqMUMTREtSuiNWMNd3w+5VrOqcavnKtxQqAW0NzBIXSjh1BlOVD3rVEsd0QP/by8vPzoLDP0ATBAZxw6YOyeoV/6osBWOF2HQB5NEMg6lj89OjraajmNyWnK+n56errnmITSPb31yZMnezoqMdKMdxsZQmIc3KHymUZTPzgL+Z/+03+6XWut//W//tf62c9+tiGRZ8+erTdv3mxvFCpRSnEOoQzbXOpJi6JY/pYstO6m9zuU2SgRz5j5TjNQlK0IqoiqC9t92yZ/YJEfyqjyJG1FPhMJzedRBinwTxG19ex3NYu8YXwN0uyjBTRrdKp0TVhAoTVQGm4DF+U5Fh8agcFYa5/0rxM6hL54/1bQk40xfEo+RRZF/d2fCPXqd7mcOmqGltG1cFsjttbaCzv77KKoylk0UFl1/uqkXOv/rT9shFGKZtZMWkPWJf571jbWmBtr951Wt4rE6OJcN/oEgd2/f3/97ne/+39HYP/wD/+wzs7O1meffbZ+85vfrJ/+9Kfryy+/XP/3//7fDS3pRFHSDEEI0ELuNVW0FurhwWTL8B0VGMHPwlLPKB/UhTGTChBVwz79gLjsIsC7QU4Qziy87aLTvwcPHmyQW+xfo9OwAgJsjRZuSJ/rpSba8VkVQj8awljs5+fney+6vbraHe7X8MLfZllBnw2FMn6UE+KYfSz61sciseqMRT75OGjQc2oIOs7WMrWkAm8jtHJvOtSsuho7qLSV9kJ3SKUV6JzAWrtjnvSvyZqi+NIO5YWLSrquWg9ZB9JwdK3d28T18fXr15uOkzuH5FlFcfqDW4a+6DXj5Tn6MndvlDLo7/3sB3Ngf/M3f3P79OnT9fvf/359+eWX66uvvtogbLMEk59qPUg/o9Syb5TN4oe+6iHKcRXlGdzksBTCVbm7aMpzTQ82Y/V65UPeYnI202Bo7XO9e/tWZV5rfRROal3wZGCxkt2hLNO8N0PRcdbQ9V6T7+sYex1dKgIuUp6G2nX4yDm/s99F7BPdMWCHrp/0wRynzxm4SVrPcHGir7X2nXbn36J/9OjRVt3ee1r4a62PkFojjTneidzRHWTQ0Nn3qgsy7TWOk//s/Xv9rB+kq5zu7G8N5XRMHFHXFN14//79+u1vf7uOj49/+FuJHj9+vF68eLF++9vfrp/97Gfb4WayVmutzVOXO9A5/549e7YXKjZsq9JRaB7R9pgS1v1H2fx/clbbQIPaWpNTIU94S8l75r5+Tx5vKowJafVx0Zh7TOOKq2kWTf/JsidkqFXrW3XW2oUMMsKtEyrHRSbtN46Ct0VUF2lP42URdN8nHkt/5kKixBcXF9u9e6Kv+8xyh+rXXGD9d3OzK3OBvsmfzOb3Jo1Qwyf8FYJzlk46Lb+LKmGMutuAY6C/rdGbYSY9hcboqkJQL83tS1MsfugFh6jPEBMZkH+TA+bGu0/La6k+MJ4afOPofOhT9aLzMFH5Wrv3y94FsNb6HgT2n//zf769uLhY/+W//Jf1i1/8YiuJ8GACVpFbfmB6oZLBFL8Zq3IABOiEi55N7jsQXQdOyLIhnYhDXrS8xfQaRQtVYp93Mvx/rR3SrNLNSmvGhDK5r6r6QwiuC7+oZn63JRUTFRvrId6QjCh3jYP+ty/1wH2JxURRHTeUZvHiY8qb9Rr3syiKcKD4iUQPUQkdXx1njRbj4fnVAYaBHA5luefP6txaO5TimhqkOpTytoeim0P3L3dmXfk/w/n06dOtzqxJjKJw+l/H6tnk1fKXynQ6/3m6Rr83w8a2SaP86le/Wg8fPvxhHNh/+2//bd2/f3/9xV/8xbq4uNg2vEIIhHB8fLxt4Pb3GWocMmhVbIawoZW3mLhvBVZe6fb2dp2fn++dM/748eO9M6c0Mbjr53lglJoBbDaowvV5CexDMT4FwmsYK26PHPzkPTmDps/JhRwsgG4yp1xeUuEaC6PFofXQzcLh3SibBWGu3ctP7VBYY2ytr6ox7TNc05KCm5ubvfOr1lp7NYju1+ynBQm9+7/FZWE9fPjwo3Pt2icnRtANaKkFnHRKX429n9V4lZejA/TtwYMH28mwniFbjoOD8qbxpj+tBri9vd043IuLi63P9+7d28qhIH46xHj5na4zuJzvNErTecjSNztOV2ssHQnf66eu/2AO7N//+39/e3Z2tr799tsN3kuVHx8f720xmts0evJlBdHB40gmh2OiqyTd80dwkwCfHq+ebXqrcmXlGtwXypwLQ2vY1cV9dbUrpbDoymN8Cp2YsBpb32HEWq9TQ+H/nlekYkH02ZeXu9er1XtORZpK5e/QQY2uMRvrrGGaaKb9nKiiMur/6drJyclHxyDNRIM+T7Q6ERnDVJlVvg0Jm7mrLCCTGveZJadLZNcjiBhiyHc6/Pa3Y2sf11p7KLFocXJ97tlop+toytH1h0K9+fzagYk6q0vk/PDhwy37OdfrP/pEVidAXF1dbSQd5WTQjo6Otu/xNlMxIQ9CcE+Leq1dzZfvmWRepAv39PR0+8zvU7DN8kxDOj0INAl9FJIfWszlCmqALAYLbBojW0607usrr9TPLH6LRH9rjCgEI6KP5DI99pMnT/bCBPPgHlUw81XuomFCCd0ar85rES/UrI/mu0iv8+nZrr93797ecU2dg4kcGio1s2jcrsMBTQTpGRbiDB1PTk42J14k2kQKOVYOnW99kB1vlAIZ1eA5/qfkt37hO8l1ht7v37/fSqEO6VvHXmPt8Ab9te60rjOgoM7feuoRP9AvWone+S70+33tTgT2V3/1V7cOJDRoE1RSlufQ2QkFO5GsfRW9fFW9QS365GFKHK+1j2YmSc2wHCLq28+1dpNojFWCKdDyOQT/+PHjjWeAfA5xQ2vtsjlHRx9S832JL4Ws4en/J6cwxzUXUT2l8Gl68f597mH0bP+f3NlEJfri/+0Pg1C96BwUdVbf3E8Rp0XS0LmITBhZfZvconloCQCDOUsYypPSsU9xlhC+BJQC074/onyUeWuo2/sVoU5DSyY9AXYiTd/tzoDq/Kzxk5DBb1tbxtJ6sBr5ysJn5mNeU93yOcdKT/7+7/9+HR0drd///vf/7wjs+fPnm9W2KCzmKkoVsfH7bJSkROX19fV6/vz5dnDiWrvsUonGTnK9RDNLk1PQh4cPH+69D/IQXKYYmjHyOF1MUBBP2BeTNrtVg1yU1EXh86dPn+4ZQyEhuelTq6mbMYPE9Hse7SKsqdcltyLNIqTKybMZ5KI3Y70rDGyb58MxGkWVlRnPbO5bmU8P9BMaaijGQJljhqoRQDlO6AxXxGAWDVuMNdztE77YfEBTRbAPHz7c6huNWx+bvdYn97FuGEifc350suNhGBmvGbU4q77RCCK+NV1kXZ6ULHqsPKdxc3OzbWQ/Pj7eeyuVz1xjnou26e6n2p0GDDy+vd293mmikMk92Djts6nsvKVO37t3b52fn+8ZFvCZsjQMMMl9YYfJd+3R0dHe+yIvLi72yP3p5dzHpDRkImTXWBxS31X2tXY8Bm+EG+kCPZSNoYAgflFmvSPCvaESchlhi8i1EHhlcuTdzJ0arKJaygeNeX7R8vS4Dc2gS0ru99vb3ZvbGVz98pyWYhjzRJoNxRDQ7d9a+yFma8fmUeZ1ChOdlA6wsCfC7/O6LW1uV+ox3/rb+Z7hZR0JfW0yoKiyDtz66ckpNRrv37/fnKx2c7PLBDOSULojtjseutOC67U+PtiRTD2rR29D1jPULC0xE0UfyfyTf1kfzrOf1ctXV1fb21sKiSGAy8vLraK7YSOBlVBtlmGmjlU34xkYKILpizxq7XmEGjTKdH29e8ddQ+ca5iK3tXbkK+/LKBXtNDSxANyfskzOo97VxFog+l1Z1OjNUBsJWgKX4VJ4rO/u3RDOC0eL0GYIZ261t2/frj//+c97+lID2Fo/zqbPL6qFZKCU29vbzXHWyJNJD0SEQiz+vnfBc9X9cQTqtsikoVYzpXSyKKdb0nyP0aaHk5sjl/KAjTZOTk72SoUa0XCuwkS6o7/mQqusjH2t/Qwr42hXiHtwtL0PZ6cvfQ5nY80xYo8ePdrsxdu3bzfHSufIvTrQ9VNj+4+qA/vrv/7r2xcvXqw//vGPG79jcfR44XIejF3htHZ9vTsrqwQko2grh7Cm/ALlK3Lpgmlt1+QGwO2pANq8jnfAnR1K+5cnsFDB7hoBHm2t9RFiadaS8rcuysszJqc0+85g4BAaGmvlo/Ay3RVRb9x784bz2cZKZhSvi2n2uxzZIY5urbWhx24MJ7PJo3aeG250rBxhn0UGzcyVEpick8Xp+1MO+kg3auT839x2TeDK2o9D8+u7h+an2faiqKurD2VIfXdCG73C1TVryWi21tL4etKJ9TETVpMT7HjNS0l+cmzfrflf//rXP5wDcxCcwbYGRlZyKimLX9RVI+m19QZycXGxLeSiDSc2Ekh5IBZcX+r11tr3Wj3Z0v1qiPS92bfWa/l+s43gOUUoPzNDrRLWlM0it7exaKyLm4KTQUs66kRqhCiecbVejyI6Rsii1NdZg2XcrRnqQponntYIQl76wRgy+LNsYa3doYWM10QBM6xq6Kivwubqm/msHray3Pc7Fy3WnBX30GX730MTy3NpM6T07PK5xlGjO9FhQ2m62jkh16dPn26nxLq+Drzn/vsblGl9FGmutb+tj+O8uLjY+26pCHOED2yIXkqq+gapNuv5qZB9rbXuzFO+efNmff7559vvNjzXW7W+pVzDJHYPdeTmZndCo0lWqgGGvnr16qPtRmvtQjUCa6sC1euvtf+arUOkc4VvK8P0gsLaWe/TZ661C9XmIq1x9DeTTRn0sbIpilAmUkWuXDtPzb4WgTQcrrKQTRFl72/c3d5FDs1Wdnx0oGOgwM2s4gJL7NYAUXwbp2fIP8slWnIyEV1RYMP89rt6UrTYmjeIVptIqoagCJ2D7byJGBiYnrNVpFXd9RnE1PGQe59xe3u7d3hjD45sqFr5Tae21gcaxysXe0xUdag7Htzz+vp6vXjxYiu2Nhc11sCNOf1UuxOBvXjxYp2fn297rnilTrKw8O3bt9tpp1VOHSqRWytOcDV2JteroHy3ZRd+Tq5gG1iQQbN9a63tunrITnA9uOfXSPVUiI7Z3/osp4xCUF3QFLkGvwgFOSq5MPtZ4zf7TVZ2JFCC7h31PePqvOpDFVc4/ebNm72w59Ci50V59c6NsQpP6+xkJbtIkdXuX7TJyM5xa7Jx7Z/vtUyixmkiG1lI90OWi0KKlNzH/JSPs6BnqF2j0c9OTnZ1iRBcxzANZbnAzkt3JDC+zoyT/RY+0oW524axIQuRxKNHj/aywRB7x9HyiAcPHqzPPvtsj7sUFZQXVHv2fe1OA/b1119vD3RgXTNyBHxxcbEePny4/vznP28CqpL5iXdhECG2Q0Vrc3IatvJOPdP+kAJ0Yk18Mya+U46m+wCVRFAGC60nUvA8PB/CUnMGfl+NttbaXrfl+SarxaBkOLmI1kTVWbQId60dkbvWx+hjrbXnSOoFHQ0+5XhxcbEV407jqwnnm9JvcqLohUGcyKLPriybpeu8NUtaY9x5n/domF4Z6ENpBlHG5ASvrq7WkydP1lofZ+M1Mm3E4Hs1nocOIZh9M39r7ZcD1WB3Phry15n3/61Ra6Kl4WbLk9ru37+/GWYOr/WdpXvoS59xdLQ7ZrpJOX2kNz8YgSmJADObGiXUy8vL9fnnn29EfIU0M32qjddaewezEfYUFHRTb9gjfjtJTQDMZ/MwVZqZ7XBtjVtDoVmmQeD6Jtwsr3B8fLzV+eDioCETWu7v7du3m+dpGURr0ZDPeAWes8XFJr73rwHRGj5UqdwHvzFDJvKBMKCp2Shw0+gMIEPCm3s2x9R556lbcGps/b2Gmcw7ZkaNUW2YCvWZ9775fKIqzgTlsdYuQSNEOznZnXpibmp0qkt1OmRrIRfNNJnw+PHjdXZ2tl6/fr0XsjXErcOcut5yDDrUOfR/eiBkJw/66LP+jdw8q0YdWJH15tzL2zYROMn+2b6XxG8Y56eyhpLR7QBDQQCFv1CcI3OEn50kijDDmtZrSeF7VtFHDaHfm50qQqm3nyEoQ9Kx9zBEHsT/CbpGGIlcXgFJ62/Co25eJzdoqEXEDS2hNEYO4dwQs2R4kRtFKi9GgW5uPmSdzYV7QyStKSM/rYjWM5oQcSpCjaZ6qBZY1pO3Nqi0g+cX8RV1dQuOe9Il3t3Y6Sh0zBF4q9Tz58+3e5Aljsd4W6oh4lhrh4D10xirKw4CbH+L3CQ5ZAovLy+3Gi3zLMLpAYlFevPVZ+az89fSn7V2JUtKI25ubvbus9baxnpzc7O9cAeyKwf38uXL7bV3DLvi9xq6XnNXu9OAvXr1ag8y88CyXhZfM0qNW8X7JWV5azxSi1513O9OivDmlq3TR0frs88+24O9tfgd/Hy9lZ8mtAt68kr6U4LRJPHkRRNrrY9CBTLqGPXB/1teMavM8X0tCi13YOItRg7nU9zU5LuEcDi0orY6IH2RIZyV/tP4NwxnAJu4qMwmei8KY7hxLjVqWrOWPYL76dOnm974WSTJMK71Ae0XVci4M0gPHjxY33333YZyaiSh2Ovr670DPxm4IuKizbXWtpiPjo6201TMceVZ51EHbN40b/sxdw2T1WiW47q8vNycntIdReCuIeP5ex1QebSXL1/uncJCPxpK+6z8d8dtPlv0fqh972butfZfDVYyWlpcRmOt/QXfg9BaS3V7e7tBycvLy41HaB3OWrv0eZV4rV318iR3K6iGFCab8AmrnMuEvvra9PwkuT3Hd5pe7/UTJbhXM3Ldb9aSjrX2ObzWXpnwGZ7N8K4h4tym01CK/BoSWjSV5aHTWcmxWUB9Mtd1YmRaHs/v5qaIUxa8PKBF1OfVmEMiNbLCk6K3WeRpoTZcmoZ6cksQknEpq5jZtV7DaNUBk+c0dJ23hsTq/8pn6reQUz8YmDqU8rz37t3baIz2x/5J/TT/UyeAjUOcZ8dW2XcsHIrv15h9qt1pwN68ebOePHmyx2F0wROmTMbJyclm/T0cWnvy5Mm2MCifNG4hOK6sFrl1Xq1p0uoBxfeMiX2KFvY8gqfhlXuV4/A3Fd6b4I72RVf01MnviRBrHTZia30gyLtb/xCKpNiMRMNN11HUclvl/4QhXtDQvpGNaviimbV27+LsXE/k2/op98fX1dPimKacNQultWkMmtM6y5H6WQRs3tUm1qO3BIKxI6+GsS0UbT/LXR3qvxo1z6nROjraPxO+P30HyoeYuj7W2kdEDef1p1vwyl9Vv/qvp0L4u/uXAhGBlIdea+2dPzez7QUM5k5/OdLj4+OP6BhjvQuB3VmJ/7d/+7e3Jycn6+uvv14nJyfr6dOne/F2j7mpkEt8dgANTXimGsObm5u9e7Z5ngEWQa21e5lsiWbtEBKrABk9NWht5ZRc07Ea+ySVZz1UZUQmhzKDDcfVerXWrminnnpWoE9jd4hTbL9wVTKDLW8g86bijUHrQm0mqtyVZ3k+B9WQzn272Pvsyo4eFcnSgeoQNCJUV1fV8VlADP9ETeXMWv5RiqD3cO1EiL03NDj1ot+FarQmXmbYPpv7Novd+ZsZalz1fF8qzm3qUWVqXbQgl21R+A0R1pFAgtUB/Xjz5s361a9+tY6OfmAl/ps3b9bz58+38+xL1h8yXjyTzwjl6OjDe+imh+1RJ67vJtOrq6vNsvPwjIkJ0codUHzGcW5vqQctKvTz+Ph4ex/mrCvSd3u81tovnAWvm2qnRIcUrh5PSG48ri0BXAPESJr0ohl/ayX/NLLti35Q6h7SV55iZrfcuyHn/L1h37179zaurcWwxlelFsoIg8ydzGydr/HQq+Pj4w154cjqfJsEIOvqlH6X84ReW/pRnVcrRxe6U6BjbKRAt0UMDFRR7KHWnQS+12vKJ7YQ3NzN2jbrBD3TsM9prnUqnfO26+tddT6dJgu62tqvJv86jz0a6K72vVnI169fr2+++WYLOyhOjZcHNxxrDdPbt2/Xd999t5HAXczCAouncXP9JwAAIABJREFUZCWSvM8pYbk3kBiZm5uPic2SjoyCvsiMdVHbhtExlYS1B6wGuAu7nldGqsaghHYntai1nplxN6mUnoE/VO1MSSzqtfbroHokimeaO55S7VeznSWPi+Q8+/z8fI/TqgGYHIdnMwqMRJFbn0n2a60NKaAiXG87DB3wctkaX/rg37179/Yya7NkiI4KYcnIYqUfynn0E49chM3hTZ5TzaBwthSHdWEeGfUZRtawdM1O/q3vgOzxT56L92ufXMuId7y2m631gTvv9jQURstbRHQt18DBHR0drWfPnn1yT3XbnQbM26IpU1O7c5+URtGk3eu9CRTS4B002S3Cb9apSnFysjvj++rqas/7UKYiskLbcgmNu016UQ0jxBAbXxMSjNyhze2eAX1WecHvetCixumRyMoz9bdH0EyURx7GgKCdc2UsjDEkdHZ2tkfY19iVp2xhrT7TFwR8jdAMd2vs2rciEfJroqTHYvte+093yLahkjmpzJrR7nUMML6WUy5qM9edu7V2WcY5DqjcOijnw2AWWR/aqmSd+Tt5CsEmzVH91ya6Zbi7XnBgXYe93ne0nuCK0y4Y4AwnsqPjylbW2j8v7FC704AVDay1tleJy1xUMNrkYYpq1IqYQKEY5a0HIJxyDGvtKpAtrAcPHmzfmV6tC4JA+mwL1ATOmhnKiRdyvxlGMErTOLZ8ZPJAiFMhHsNco9l71ZAqQel3J4fSe5+fn29Gjjcln5Z4NDQw3mlUmp5vWFny3vYSzoqhmYaZg2NMyHpybOa14TI5mI8a9YaHFpdseZGEPvRZFvHp6eneW6OM2wLjsFpiVJ6WvNyj/8ijvB+DRU8fPXq0t+ODHPzU3zrorp3u8ihHVYfWjdvWFcPV/bPKmBhF/++9Hz58uJ3v3/KUm5ubdX5+vierGtmuW98/PT3dIot/VBlFvRGh1MhQSp3QqWYPes4QpZhGAmRmiHpOUV+CSxlNNMLRtScnu3OVTNI81bTI0T5FY72+vl6vXr3aM4SQi0Xv2RS66MLi0kcKarGVXFcAeXFxse12IJNuZ5ohAFgNalsQDe/WWttRKjc3H7YGQRqMNHmR8fSE5pYM5nYdMvMTh7bW2ja6WzT6Xw7PIrm9vd08MSRR423ePZd8mlnWR2FtS1/o4Pn5+Xrx4sWevuKc+gwLjezwdJzRoajDNcJXMvR514oiZp81Iyd8Jhvh6exf0WEdcmkddW30w3Xm+vHjx3tFtEXaDUMvLy/3jroie88vshJhPHjwYDsTkHEzr3am9DktvSE7TmpSFLPdacDevXu3vQDCzXE1wqDp+WdYZQFeXe1eHd8TDhCfhHV8fLzBbp918TbEE6Z24TmGhzGYxlZGhHEo+a2Qr+ipntmiL9leQ9BK7sLn6+vrDb2Sx7t37zYlqqKutQuNJE7Ioa2GgOIYC4NWRawna7mEtxaRkUWAt8MLWczlliZS4hzwPmQvIdKsaw12s2rm7sGDBxsZT851qCrS1Zh1K8vcqsTwvXr1ai9DaFzu30avyrHNZJPrObQiPEbY/fGFE1H5HsTkOy0zmte0RIIuVH9mJrD86HwBckP/kv4FAmq78NI1wnPngXUI8UJ2paGOjj4UorfMo8mLrv3vI/K/dy/ky5cvN8PlphY/5FBexyQ0XKjhY3QsLjD59evXH3FCU7hIW0pvQroVQSWy58qw8AAWYFHdWvsV2pSiY/GsyZvUY/TUDvf2t6KV3o8MGKw3b95s/Eo5sWb2ZrU2R9CQkie05cT36nR4xiIbyJIH50kPwfjW+0A/UCnlfvDgwbq4uNjjEMnXM3t/Sv3mzZvtOBXjp2fm4Pb2w8kYNQynp6dbEkHjqMy9edaHop8aEM+uUdZX8+HN2J6z1i6rxwD2OZ/KatchVRbmuhzUPLjSjoHJQ+GqOX764f/0SX+sndPT0w0IFEBIshnrzc0uOaTWzPP0c+45di0uETDggCTXXDsLrWe704D96U9/2tCAyfdwgtcYsk+FF+C1392HUB89erTtQUNWtxFK62agAvsC2whaWtj9cE3gtVZFs1jKFxgP78c7WEytdZF5KUotLCaXp0+ffmTYKEIRQnkDfS4fVFlVCSyWhrhVJt69sllrv1gZAhMSGxflKnfIKBSZtSix3FRLVtbaz442fBYuMxqSFaUhyO/+/fubk2khr9emdVEYM8RNdkVzdVLd6wfZQMhQxqH5qGNBgZQ+0ffyQJOHKjKezXh6Yqzwt/fs3OOppl7T1devX2/rRF/ovQNIi5Z7HwT8IT7OfFv3bErr/OiQOac/n2p3HmjYt+1U2UpUNgshNKToTfeDmiUim2GoF7i52Z0YWc/kOpPqXgTo/ofO85oJgnqeiRjbH7/j4hhNqMd4jbNhUiFxjdhaHxSvh8D1ma2f8bzyQA2RzcdMqjR0Lo+IFzKvxj0XFSWfIZw+VMZdiA3d5pimXHtPz2qJjs98r9tgijTdC6IVkhStkhmnwtnMAzgPtRpk/2Zf9aHzVbQ09aMy6RYpetL56rNLfHeeOw8Nw3y3xqtozPfnOXENn805GeGdm2Q4OjrajtVqWI1vU5bEcTFULWMqd16Z/mAE9u7du/Xo0aO9bQItdCwnAQaWO2C4hI1d7MIP3+dpK+xOIKvOCzoJElIqD9U3EtWTNsyZyqExTnOCeSxjmNxFkVrrpnxORj6Hfibxy8CUkIds26/WLNXrrbW2WqWinXpJSuEn48UD+tkFN1FeF7FFW2RbjoXzKvq4vr7ezo8zXobp+Ph4yxAfSut7Zo/SljkkR5/ra7O8xloqgn7SI32d4U9Pt5CsqI5wbHSSEYMmqmc+76GIdaxduNaImq2J0P1efWbwjIeMhMptTW65f8td8MOa3837+/fvtxdc0xd1pFO//X0ayTo/Cbp/VCHrj370o3V+fr6RpTIIJd7rMUDt7k1kTEyUrAal7eDm6aaua4P0Hj16tKXoC4fxAfrm7xRqxv49kE3af61dOOX5ru3OAN8r/J1GU1+aMrdQeLCe7lGe5FAq3wJs+YnvtpwAX4En4u0a4pm38jXuN9FCEV/74zNjK69RT0tvhBBF2Wvtv01In/GDc89ldzsIxT2bDJoRxrFqFiudgd7meV3TeFUvlGaYD+FW/7/WflW82kbyYgydh1V07Rnmif6Wc9RKQ9CdeVxREdEhZ4SDpp9+WhulTWqMi7Ks5YuLi/XkyZNtA/7V1dVG8zQZZp2UehCZ2Xbk759qdxqwb7/9diPm3r9/v16/fr23qVmas2GjRUDhhTZFYI7paZapxGRLLeq1bm5uNmKTAjZlb7LqTTy/2zraatRMZscwhWcS+n3JhRqbGZa1tqre130on+u6S2Ct3VE2nt26HF7eC1PWWlsB6bt37zZvOTNXJeFPTk4+OourSKXIZCIEi2vSBOrnKKdnlsD33Pfv32+H9Al7Efnmoouf83nw4MF69erVJuOWP2i4W/2X0DEmhtJcuNda+yePXF5ebkkWKNt3uhg5a/LQGjY1YqA/HEr5XI6Ow53Z8Y4NH3Z8fLz3ouhZEF7Kx//JTtKnCPThw4dbeVG3IJnncoqePddNv8so+n/BB96137mr3bmZ+9/9u393+xd/8Rfr17/+9fawtXa7xSkr62whmoCGa53IIpS1Puab1E2plTLphK7GqVD50P0OhYjzOyarPEjrvMpNyIhQAGFyjXdDABPSwkfPLrFLPjUm/l6jNb/fBTPHXYdQZSiSmuUsrSfjJetpyzmVgyp/1f51vt2Dx24yqI3Hhkw9X7+hBQW5zRo35CGLGpq5IbvhS0N+aIksy8OUpGfk19pxcJ1H925pEXlNvWxSpv3v7w1/9f1QmDvXw/xsfn/yf/1/11BDXC+9blg+G0dsTiFkcjBfHT85Pn36dL169Wp99dVX6/T0dP3ud7/7f9/MXU+ihmMOVCbm6Oho4y0ak9fbzKyX2qhJWh8fH2/xNMTjZAYDNAmySH0mb9mXhZqEho5r7TyCfjJUhNr6qoaiFsUk9CGneun5NnATWGNi8TBaTVMbtzHObUMUpRwaUrTPazEq2RrXWruyDuFhQ40an4auHBnZzAVVpXd95U+Ga+3eSrXWh5CyR1rf3t6u8/PzbeGstTZPjz8rX7PWrg6Kd4cezNFaa89AcmRNLhTpQRozg9qdKY0YoKrez1x3Hnx3rV3WXL/Nq7610n1GK5V9ERqqpnPUvpSfqo5O9N2I4dmzZ3uhZu9XGme+FlCtnHloZrfrsI7zLpB1ZxayyGOt3cIp/O2m0G6YXmtXZ9QCyoZFPCmB18PMheDs77nob25uNoPWibEdobUxM6zr52utPaNAmBRqQnbGvPCZwTF2hqVEqjEW7RUFIODrpcu1UIQaHWEG5MjLFVVI+3csM9tTlNVQxWKarUgashHCz7R/qYaZQKgzKA9UJNrQxeI1N+X1itQmaijCZbCKmMoz1cBAgp0D/Z0lGVBEjX15QPSBqGUio4ZY5r4GZWaPK+deV93p3/u9iZqm8TP+ltCU2nHQoWv0r5zWHMfJyckWhjfJwXFbQ+7xjyLxf/KTn6z379+v8/PzPcvMu8+K4CoBw1diuYugKWebjKeHr0E8Pj7ekgkW/cnJyXr58uUeydnTLCzm8kjliKo4PFUNqL81LIQEkffG1SORm8FrYsCibYg3YfRau1KHGQq3hkzfus/QpAuViup8t9xH0/HQivsUcVV59ePkZJfBIzPGFJ0wEU836NdAH5oTMoLQr66uNo5vZj79n46SY6vA3R+iqDEtgYx4VmLRkA+irxGq3mt1aIcMpyQL1FNOqRniGjf60MLQItnylV1jU39xgGTeJFiNfrcfWcPQoPXFgXe9WEP607lpGdbR0dEeqV8DTW4Kou9qdxqwb775Zr148WI9f/58G0AJb62Gqe/hm6c3ts7Dwm/WCMQ0If1pwdnWgFR0YmyLCRGZNWRr7b+Qg5CKLlvz0nDWeMufUEQv5tBH32lldGG6Ce82IZPdDNY0bhbcRE+tiSpC6zMbRkF4NdDmrydVyDRZJEVhFmCNkIwwWZH98fHxll1+8+bNtvWnMm4Yql9Q5MOHD7fs99OnT9e9e/e2/8tods6cZ2++Ss6Ti3H4vZ9VZ9SKQUsW8HwxRou9J9qrQe7WGDKqgSTb8sszsjDnqB1G3lx5j6v+zKiIbPFS1lOTG0dHu8RUk1+ds+vr63V+fr53fNBauyytz7v29bnhNj2jz5wKubILn2p3GrBnz56t7777btuuQVg8ueyRhzsQjSL34SaupGo9ncl++vTppohrrfXdd99t925TI8Qg1DNNaN4Qw2SVAykUbwlGUc1a+2d36TOU0c219+59OAe+L7mdkFoWca39dDneo+Ufhe360JKM3oNxgA5n1qh1X35n0HxGkci2h/TpZ8l7CwGKa2u9GyPIIJAF5DXDTv3lJHx/bsAuHdC/9b76Tz4WpVKV8oNFdmRQZGM9NJHT5zdcbKP7xqFvddbdT9sxVvbVm6JA5U0cNRRcTsl9PLeHcs5tO2o7y1G658OHD9eTJ08+iUJFKOTKwbln5eH3Jj1alHwXB3anAXv58uU6OzvbXkJbgUq79mwwkLCevCii4YJQYCIm1+KvPvvss00ZGZsuHjzIWmtLAfNIjEbT7xR5LpZmszpZ0/vNFLwFAXGpvaryd3O1PrVSfJ6YUTm0f0pIPNMYhFdgOIPR0P34+PgjhW+yohkli68hvf41/Cp532OJqied88pQaFkD2jC1mT6yrD55Xs9h615azxXW1qFxNsbbGrZGEJ5dakATMuuPvkOH5TEruyJ9rWit/S2vVwNs/EJy89G5neS8vjBk1lL3yhr3dFYdB8RUR1802XCxe3Bvb2+3WrMiQvraOXc/6PCudqcBe/DgwfrjH/+4FbN2MlqwenV1tVeZb8FOklAni1TW2oVHLcWYmaV6LwOrMIo09LG/Nxxca0ey6zPBzYQExbdwW+zXui5jo/QND+tpGAbP84xu96lxrEExViijfFcVTt/61uMqauVTDqLcy+THis58VkpBP2eleDkpTf8ZIZu2m2jQJ4jfQm0NYMPnjrfEf7PI5nTyNkJ6DpXxq0MpCpmvAiSPhoaNIqoPE2l2rnuSRRc2HaxTaORQzrOclnm5vb396Gyxoh33rCOaznsamYlY9UVmUQhJRyEvBdZkOo3ZfHYrDg61Ow3Y9fX13stWLQ5v6uUxeB4PFwI1dq6gyguwzhRjGopmXXhcAlOta8GVmK1Q6pltHsWZQUIVdjNw2uQShDf62DCTN7LI6oko+iFvDv73OeXEKG4TCJyJZxaRaJSp9TiU/OpqV6XesLQZoBbMmjPfabNghR0UlLcun1jPzohyFJ1P97TAGYIiRH1sn8jUHM9N9r7jJx1gGMioXCmZdj71zYLs6RrGT15aryt32bmfHHAjAjoNvdZQ1zi655w/rWjXHHEmXRfGQ/cYeXrVzDcZkzN5apB3eW99dbBl5/wQB9h2pwFzWkLrvxqDF3n4WRLZxFDMub3BgNyPYPqc7oZ33YSVBFXCsQiCwjnf2/X+Tyk80+IhPM+sJ/a3EuJFHkV7DFbraGr0anQsuoYAjFOzqhAr5FCuB3qYBqaEa+fVK+8qtyJo8isy6O/k19e/FU26D8Wtk+oCmckQfaZX3WFhkZQ0N/8We+Xd+rYuiC62GtWJkiuPogOOSt9cp8915MbZZEJLZG5ubrbz+62Z6sFsRZYTXTdRovaqIXGTJEVT5c+KylA2khprfajDM36y757VtXbJmIbDs5n3Ho3UhNBd7c46MGGimqr79++vs7OzLb1JEL57797uxapNUVMkh+dNq4o/sgh7TeNrELVbLRqvT+G1JqeL0HaoZiO1GfrW61kAPOshj9aJmiQv3qgnrl5dXe2dcqlMpJ6qCt4QfPJCvuOaGgyLRV3a9fX19nbzclEWXuE9JcRFWYAWY4noQ2HAWvsvnCUr47JtqDKZqLZbr9x/OsoWUZPLbP7eZ621W7juRabTQTUSKFqC2KtLoovyQ/reOrgaKkhxJsCmblXvJs/c7zqeqki9/S5CagjaNVZkr0+9bqJFme4+CxJmrBj8OqfOUZ3YD85C8n4eeHJysveSgsbdFlw7YFGWmK/wqiyutYCbgQNdGUithZnTUqucbraQl5ivbtsTSBSw9+znk9+poOutjWH+ffJ3/uazchhkZRwloOtAGJYWy5b36CLR7CNlgJo4kP3Rdyi0RqPKOPnM1o71iHCL0Xiurq72soxTN4q0yLooxjyVkOd4a1R7f8aZ4Z7lFZVh65Jcd3Nzs7c5fK1dQXHnrOuEkWm00fEwcKIU+tNQvn0s5aEZTyMK4290g1Os7Kw1OgC56WvHwii6V/VzrbX33gX/FyJeX19vIIjeyE4y9PScvt6VhbwTgfHUDJHjpWvhj4+PN5KfEmk3N/sZOANpaMZgmXSv8aIUzVBCEATeDc0z1jaBLQisp6wnbSv0bWu/24o+/F7laajZNyBR4NYBUeQmSSxWMhd2kd/kl2afa7gYgaLMhvz61WwgLqhjqWctcmaAycBYq8zGXWRjjEVoLblp6QEZ1xjP5Mda+85N5u/169d7CMVCn0i9C7oylGVu1o7M8F+uFV6Wc1tr7R3B03KYhpJzDotIioobzhk/edUwae7juiJCsvK9tdYevTKRPF0kC1y0/9uvPBHto0ePtoLkyq1z0tCV8/lUuxOBOUO8k1KLCAZTsHIs5b1svxEqdVJwV+7ZbQSu92zWugbCDnpJhKI2Fr2K36zZNEb+X/5krX1vgEtwj7mYJsrhdVv42TBym4ij3am3NQYlZSlHwzz3E9oyiu/evdtC5RrfenVGBNIynpYGdF4oovH57tHR0UHOwr07hoYDDf3pRdP9NWjd0VAjyYCUq6KbXYBC5/J1natZx8c5VH5HR0d7ZTv+/+DBg624FV80DbD5KeF+dna2ya2GrYbAc2u8psFpCGfuZjTUM/fX2u1S6Ekl1f8a+K4F31f3Z23MQ04nart3b/eaPc/zs06/z/7/B4HdacC++OKL7UYtOyhpX6PRFyt0EizYKuFauyLHWT1tAN2awoh5g43JqLGgbMIDBgGv0O87p2xPGEEdDXdbfHdoC0nv0zoiE9Q4v8aVl9ZKYlvAzdzx6H0xAw5pcnI84MOHH153Ve8+620auvm8WThzRZE+xRN+6v+Qn9+70LT+vWGd56kLbEgONRVFFCEVgUPe+lbutk6YLG9vd697q1zU+dEzYZGEyOvXr/dCb/1oSN0sejPwLW0oEvXsNske89f7rrWP6KB58sVRlTaYdZGauSLTWZ/Y+k062uytvpOj9yPUGJNBQ0jfsVY+1b73PLBHjx5taIC11G5ubjbUBVL6O8JuhitzsbhPSUyDf/78+SYUn/X43SouYycml81pGrkG4dCEeUYriKcA8QwlgQ8hP5PQEGFyLkVvNdiTzK7HnQuhxZrau3fv9hQNQd6wlpE/Pj7eDBwZQGrmicxaU9Xwr9635HZD0VkdXkcmi9Zwn6FiePq255avlMtScA2VrbXjHX3H2P1DWJdU7rxNntdBi3WiZMpBlWtr63X6xbm13KYotxFL7znDLQ7b/5sR1i9/b8HpIQSK51prl8BpmN7wuqGfNg/b1HoYZI1bHSqdNtYZgs9253lgf/u3f3u71gdD9uTJk+0h80FdPMKfcgsEbkAlmycP4rszg+Zn0eCM2U9PT/cOUGwRHKNjUhvLV0CynJ3g+Z05rrYu3oZ1DJfdCya4W6BAf+OetU6T1G1oDYaTc8c6ZVhZM9LmZGZ8zHPnqTVb1Ylm1tZaG9cynVXlUmNeHWmbWdup2GTVULT9b3hfQ16ZVSZzzFq5sGajZ1jc8dUQVgYannf2q3KofpTjajuEUg7p9aFsZPV7rikyI1+60lCysmxyos7Z9YyjdVX06/y/Gshf/epX6+Tk5IedB/bq1avtDcGIehbcthbelCI8evRo8ySPHj3aTm+ooIRoXUStcF9r7b1pqLE0QTVM6UTjY/quxWmEILnZSsI3W9SJLiHdyTau6TllF9dae4W3t7e7F7MKc/XLIoUGeGT8SfmCen1hCYUrqoWMQPyJFGS9unDrHXlNY5le+NCis3G7zm6tHcJoqEEn1to/faNOrw5zhoXCOCelIMihzGkkyLDOxRzq36ec+xzLWjvqwPowh6ICz6sRPT093YBB71laYvJS5qgIrGuhxpu+om9KV7Twt2urzz45OdnLKHJSc46rMxNl6a/QvpvvhdA+KxduvUzdme17N3O/e/dufffdd1utzlr7r4riEbTyOs79MSBCYMFnKLDWh7dls8g8bSeykzP/zxvgfj7VKLtxCC9Mqnt1Itfar2xuKNL7HurfrDejMCanYe7V1dXGX3lmq+aLWkv06vdc6MYFxTRMmUWW6sCMrQZ9Gra19mkCC5KM6gSqhDWczZr5f40hVAjZVZFrdFwzs26cmUTP1JfyfOWtyNg9fe47+qt/mjmEdjyjB0UWqXhWx0cfGaYZbTCwHJW/tWC0Dt440D0cZd9AT66+4zPzBLm1XV9ff8R7dd30zUX0jx431L+5udl7CU8Lxju/n2p3GjC1Gi0itOAoUGF1lWNCVEphEnlIi8aEP3nyZK+MYsb89RJzsspP9U0vDTcsDBwZJNSXSqhc7iLsT0inY6R0a+0fAVSv63km+Pp6d4yz/iI6TSq5Nhs8F7p/5Mao2Hfms9vb221rWA3HLAC2EOactuSkDqWhfps9ceSoT0WRlJ8u1EBLHHiuRVJj0LDSnFQvmyFnXLVmBGe5x6eMDP3qrpKrq6vNSHEUL1++3OZ78mRd9HX+9+7d24wFnXnz5s1ets+ctLQE0qF/6gwZQHNU7qs62jIZTclI1+1cD70OV14ZOT7H+O/fv78ePny4Efnmex5IenPzoeyovOmn2p0G7MmTJ9vxrzc3u9dY8TAgOiF0awhh83w8DONFCEUTk6tox01UN3zjWAp7OwEUufc2cfZzep53NDKms5JYSGo8JrAvVNDPhniTi+I9+94ABqBIpIvVJHbBNGS1CFsjxUApF2jmuB63oaP+l9sqB9JMZHk6/ZpFjS2H8Z2O5/j4eM9ouG+97tHR7mC7GqyGgPrRN77PUMi1Laep4dO/WW5Tfo9Ra1aP7N3Pz9Y3qm73LIagCGhyf+YARXNIDybSqjEju7V2Z3FVvn2Bi/DSOD27hdgQn8+sPfd9/fr1BnhcT6cbqnqe8JbOGH+zj2zL5GXb7jRgf/rTn9bjx4+3VyZ5QwxYPGGqkLGTeagGyD1Ya4rRavx+v0peYpgA2wdK0spe8LV9cU49hSV0nq3n6V9eXu59H4lfg02htHq8Tl6NCYNeZMrYtvL+7Ozsozoez20dTwsnIQYKSUkqr2kUKVENxAznGjZNgtwcnpyc7CGdtXbhdV8hxxh6HhRdhNm5nOMvMV49EYa1n9XBEvYcdO/ZLGTDMvNqoZ2cfCibcI3FNpFrHZM5aYlK55NjmsmIzrfQuCiqzh1wKPpu8WkdVNdfkZsIZO5kacG4NeD9Dz4TxeD/jo6O9rYL1tC3XAZPZm3r1w82YCU5z87O9rZnFApTEPVSUAyE0kVe/qNe3z0p31q7Km4G9Obmw4GH0u71RDzDTNErMWj2pt69b+/xt5l1q6dByncyZ0jruvJJRUz1UIX/PVPMdcLN7vMrurAA/YQOZIssFCF/n1sFp+ST/2n4W7kyWP0uspjiU+KSwK27Mg7XNyw3Pn1ba3+zf/vg3vrpGv3r7oUaJ4a2iL/cn4Xk/vrgp1CniFZNozGWmO615rEcY2vaZqLF94WqdR76hu4pH2XufW8i8uotZ0E2+ClNeOu57t8z6qBOpyS/e/duq+Gr7L2puw6i3F+3zP1gEv/nP//5Oj8//4jbWGtXnV6inWA/5VkIpgiOsE0gJawntr2UaDJ9AAAgAElEQVSoZRJzC0Wf1ed5z6AFS0iMT5EfwyH0Oj093cui1uvN5EWVZK39UgvX1tsUencLDmPEg07+gVczXvIv/zBT9+UxhK7dGVAjoH89dVR/eU1zMKvjhU2da4roH6dYB6Df9+7d2+M9OJyiQwulVfINw/WlIRdZQSZ+kkEztEVYa+2Okamu1aiutX/eXPdINrSvQa5j6Ib8GnWL3bhqnH2/i3/SC+Q+F/+kWzjQ1pDVKckcStLQBfrUqKZ2gjH3prKGwtXhtXZHXDNypQo6l4fanXVgf/M3f3N7dna2Xr58uVlaiiCj4PNDB85NHqtektIV+QhRKFNruqCLuYfLQCGl/q01KZ5/qJlAKEe9y6wJ6318p4u1z2DAmm2ZZ32ttUsi9FRWiiwMwp+0oJfy9gC7bg3qAihvORFi54ATaW0OZSry5IRqvLSGxEV9LceAHlxnjBb2LDae92xYXurAPSqfmUBhLJvl6h7djt21Dc3nWCujUgPkUsOlf0Ui1eOpS1O2dKOoqtfVMHH284DD2e+5TttfBv5QXWfXVvdkWov+Zp0IDYsyJzLVj+vr6/Xo0aP17t279d//+39fZ2dn67e//e3/ex3Y06dPN2EfHx9vx+XKIvAm0IJwAVJ4/fr1XjbPAmZlCyE7EJ5e1gJcV/4AQa212zNX48X7EKr71RNOTqxK7Bklbik0j89bQXjd4KsxQELVNpNloiyCtXZvEbcAz87ONu/VeizyqyK3NMAYa3whZ3NaL25vnvlpogFq0qfj4+M9T1zerGR/ZbfWjhYolWAeoNKGU80gux4/VALc9y8vL7cawxpDfTEXFpgdEr0/eZojRqPzO9GNvljYntN71HjhWBuilverfPwjyxoqfWkSwjwX9U4U3PdPdkx99wQ9u7ra7UTQui2uRrXn7TUK0jd1aXSwtoD8UR9dn59qd55G8e23364XL15sXkjWwMBN4oMHDzYyU8PFIOZm2QHF4SE0fy+vMGEtOFsFnXVmnRQK07CxGUQe1rPL79Uj6YeTJcob1PhSFIrcEK7jrLE1eUVNmgmfz6nxqow0NVBkAfWST1GP67thv5yZn0080IVmmrSpfPPvh8bQ+dU3RqdIXbhcA14DWTnN6vwagLmhfqL2ojMyfPz48bboGT4hbQ0Qx1tH5LknJyd7We/OX/WuIS4ZaRO5zXWAj2yJQsGI7WMz1H78+PGWafW8s7Oz9erVq02uRWGHSnCguu5JNe4i4iZv5r09u5UKh9qdCOz58+cb2moHeV4PA1VL2Lfux2Qj+Zs+L0paa2e43KvZGkJRs6XNbCckWOH4XSg1YfMkTRvWTo8HgZV785boQ2FODZr+TQQqNNNXSMNmYeRwQyTzgTMzHtuVKEQXermTytu4yi0yErM+y+8NrcjMfOA5JofTgtHKAP/FMFah67yKoA/RFOX5ytm1JKDzyiGZMwbPta9fv976dHR0tGXaa7x6GCMZytjXMHtGw05jMc6WN5Qf01q7N8PwtsePH299bIKrRPzJyckWQWilhuiUU2kawupDkZ817vw31IgoiY2gQ+USZ6KmJ1f84Czkq1evtvKBFqj1PXj1WgbWUyjW2iltiX9NR+sxS+BNiwzRQQktWNWUPdQjdqtOoatWHqjPK1ehTYVfa3+XfhMTXdw1kjUIxoWj8azyDr0XPmqt/SppxuzJkydbiGDyuzWrJRYzI9Z5a6mBhEKNaCmCGpImB2QdZ4jTZ5uT6hM5z1ILTrGouaUBDeebBCDPSQyfnJxsBdPmho49efJkOyZnhm/k5fnG2JNda6QmAuzcNznVl7nSUX2jU4xyUf1EKSKgFpwWxWp98/yMEBi6uavFuA8hwMvLy/Xq1auP0Oft7e26uLjY47usw6kXU59+cBby+fPn2wDViHShetBM8RMAoZhghq/eq4ZwcmLCS9+rB+NFKVi9kVBstnI3DICFCS10/yXD19Mr24rumqHsFhqtyjxDqbXWHiqlqIpHmyWivMbajdgWlcyxhW7ReulpeQbPmuEA+RXy9x2BvGq9aENMc9nXrUFm7ltnVk/cMKgGowulWeAmRyZZfXR0tM3x7IPv1PhZmE4tLRpiEA/NpcXYI6U4FDxm+bcai5YM6D/HbF6bqdVX66kUwWwXFxcfnZCKoNf/hsOyqAxxuWVr8e3bt+uzzz7b/l45inBEDnXIa60tdC1FZD47Z8Yzs/2z3WnAfve7321v5Z7nHE3rX+82i1R1kNAeP368KbvrWkpQT97SjGbGTPqh+Lh97O523tbkXF5eblxE0+oT6dSotYShiwu6bHj1KQ/Zz423YQVlUZzKWPtOU9BNvVN0hlgGqImELtgSzM28Qll1JjUMJc0pWMsOmq2EbswFlHJ6erolJ9zPfJe3I18yZazLnc2QvtlRTou+kWsN3uT7Li93x6DXURwf77bX+Dd3eZyfn2/z4LOiQjLr+MwPg7/WrqTHWPWvIeTcY/vw4cP14MGDPV5ZDaWo4fp693Z76xNFcXm5Oxi0BxBAU/Tx/v372+m26rkYQM+xroo4cXLVrUYPngdRP3r0aG9uDrU7DdgXX3yxXr58ua6urrYMZLOMQhEKbHEITxgq7fT0dG9iWufiegu/Rw53wt2zpyLIbNRQEYyJmW81Kt9RRZoCna01QUdHu2LBmdyYB78VXfk3M5P1SgxhwzEFqhNdNmy5d2/3YhUK5PvdUCv0a2shpHC6yFbFNUNYQ8EQCRvna/XMeVFGCyHJv98vioFqGMOGGnWEsmUTlVXHPKuonhFgJFsiYN45Fn2kh/pTRNuwVR+mE5u60bXSUhqNwxF+FdnoC6csYqrh7JqtfpvHSde0oJysPLfyL3Fv3dY5ciIXFxd75ULkSc/olAgD/XBXu9OA/fnPf/5oT6CaL8rqX8lfg+7LPHljpF7DpBLTsiZTaSl534ikdeMnwXeCJQ+01rbUExEoYVbB+6/Iz0S1wLbhtN+bJeX1q9RFmpOPIgfoaPIwDXGFneTPK/uu5+pbF83c3VADL2QkP8rMWWnQTRXVGegN/7X+3nolzm2tte1NPcRdyrY1NCWDot/yMT2U8BDqZyCgDuOY3G55OeEeFMlY1yi7hiw9c75Nqc3irlObc9UIonN66CQYAGTyau7jGRcXFwezn5rici/P8QrGGmz9UEZhN4/mO7O2Dfqi6z+YA3ODvqnEDQtvCdrPQl5WvOUFthjg1Eq0Wqi+SzkYpSqM71OKCsFk1YP2bzM0mkfLNDlgsXZLTE9aqOGDkGqUKY3v9gQK1xYmVwmnJ6+sGvboS5XDM7tHrjJrPVX/3zBT+AHR+F4XfMPuetMiw6KvQ6F3k0T0RbhZeoJMje/29nbjaZvJqpGDFvp+04az5TqrQxwsudd5dA6MFbda+mQW+3YnQJGMMfZsrBaI0v/yrMJ1+gmBet7x8fFe9Tz91U86MbPJa32oAeXERDgNv4WDQlJzMMNqYzZ3OEHPppuAElvz5z//eY8r+1S704D13B4LEYF3fX2999IPyEkadSp7vUhflInkbSshPluJQsJaa6e0BK3Gx/1xLPVgBH5zc7Nevny554GULRR613sUFdzc3GzcgQMgKYrsH+VqdbMM4oTuNVI+L6GrXxCJEJZhFwKReevRalQod+v19NtPcitH1xC2pPgkrlvQ6jML3aK1kNpHDvD29nbbaiXkrvHyPQuh+2DJpUi399dfBmVu5/KTIWBc6sRub3fbjN6/f78hjdZeFZUwOGgM95zoCyclBCTv8sjV4+vr623PIkDQdWeOGdomSYqm6bqIgkGhw6WK1lobrSRB13Pg/Ox89ngsKJdOVVe6FsqvHmp3GrCvv/56Iy6b7gTBnz17timmgROe1ord6+v9CuS+eso9OkH1yFU+hsOEtbnftOglSGcI1nu2Cr/kZPclUhwCvrq62g6I6zswjYtM/K2GoqllC6snYXQiy1VAqL5DlhRD6+IomV1+SF8gLfs/8UEtXq2ym5MaiyY9GKaGVLztNIyer0+uWWt39FF3NiDaeXb3aDrffTxXJhDCNBZj5WhKAzCOJycf3mDO2HE+9HuendXnrLULIdVITaPVkIvsRAGeVaPa0iXGoTpxqLynTms6q667AgHfgar8/v79++3FN48ePVoXFxebznuWMHRy3M24F1UaK8dZvvlT7XvPxL+9vd24MJXdFXSNSjmjLhTw1LNYZgqPO/Dd1iY1kwElWHStjyo57L6e0exlDRllU4oAXejTFGArtyd3U68zD2gzoRR8osvG/pWB8ddwUljyMuHdBM1wNpwrX8IoldtkrLpH1fctlo6pxrMhkJR9kzCdP4c1MrjmpuPv86Hh8jmlD2oMqg+4uTpKC77zWMNXB1F+r/NDn5rdLVpuYqvJpxo2+uaen2oN02dFuiik+4PbZv2XOeEE6AADX32H9kuPzP62b4wrWujo6Gidn5/vHYRwiBYh5yb01lp7+22/+uqrdXR09MPOxP/222+33eTn5+d7e/Z69Gw5hk7S5ILAYh3kDRra1Xi5f2HmIf6qC1SoRmCIRtlLk16P0n2QDRdAfWNqHRSl8B2hQMfc2jd9c68ap8pI6+Kc3J5nzIwP2SiLaPhLyfRrhgOI7y4oxohB972SvtOgHx8fb4rLqPZNUhS9RLnrGO9ZI9R6rOvr670yEnNm3DNz5k3yDZt6PVm3GLb9IS+0CBla0E1ukM/9+/fXxcXFXiLBHOGfKruGmW3lXY21pDlnVeM4IwR/M34hJL21dmbNW+9XAFAHeXS02wvcc9U4nfZ7AgxgSClNHfT9+/fXs2fPtvFPiqntexHYycmH43EL70FjN693r4fuYuEpSuIyVo8ePdo7VsaEm4gS3DPk0ywKtT6M0vTmWsfNC2k4pZub/aOjFSpaeDVAUNBETBSp0J3xaAZ0ZvOKKqvoHQ8595rva0VMlenbt2+34sOJ/iC+osh68In0yLaorWOprFoz2HDZz3JXjNs09tULujHr0zjJ8mTGouK/4279UXWS41KuMREGpNM1cWiu3HfqYZFt57WOit5XXw4hdeNuGA3x9Dtrra1wd61dhDQ5QmtsolgGjY4U2UG3pTY6j3hEv3dM79+/X7///e/X7e3tD0NgFxcX68WLF5vnMFBkvkmYsNLbaHAQJfYoSKHsmzdv9rgG93J9kYTnNUxa68OuAcLXL5M5a7B43Hv37m0nbhwfH++9zPby8nLzolUMC4En9qwarypdEUlDkSJNSHEqbw0W1IofmFnL6cWLyCp7fYdGNYuvi5aRbRmLe1M05G0RCgRYDtEzbm5utu1prSubHn6Oba1dNlGTlq+DefLkyRb2tNxnrbXtVy2CFC00077W2vSy/WZonz59upeh1qdZouC6GhXjMZ8I8nKEHGcTDsZPb2cxrPtrE0w0otAnOtzwr06jVEa5tGkkOQz6ZPz0g74bbyOOhuHe/Vm5TgAz250G7PPPP1+/+93vtt31R0dH6+XLl5uBmm/uMQndz1WE0a0PFkTDmLkoixbqqWS4CLPX1TMgGrtQy5fwMGt9MEzexOy587VuNTpaXw/vmV20NzcfMpwNbQr9Pd8G5sqzXpXB7mvX6limd3eNUL73sjB6PE5bw6TyeXP7FUdEjpIPveeTJ0+2sMW8UXBZrDdv3uzxM4eQ5HyW+jJhRjkuhskcdyN2naH7+U51i6HVnNLw8OHDLSytrPSRAa8DORSh0C2GVgFw0V9DQvxvX348w3drhDOTkJi6wYF3jR1KIFn3zTSXb6MPNryfn59v97dmIXpOiUMhe+uCXDmk7jf+wQZM1hA6YWVbBiAt/ujRo72KaoapxKiaD167BbBVzoYvtkL0Ba4l/XhZwmsoA/1QyhoXnM8MjVrSwGBALY6zNoGqnt++fbspWHf7UxByqrJ5TvnDkq79bmE4xdE/E07xyxs0SVIkAiVQtC6IIooin/fv36+Li4tNduXUzFnrfVzf19fpLwNhzvEo9AbyrmEt91LOqjpJ5uYVUmkdXEsScHyTpK7jtBChMY7N83Cf5NnMqOshlHK6E1np66zKbyjYUJzRIVu61+jg6dOnW9aQ4zMvrdVimNfavWi3xu3i4mLb3M6RM+jsA3vAAEJdHLPx+1fDPkl9DrKntH6q3XkemMVDuPUK79+/32Dl48eP9wxILW7rs3Ta/xm1mcKu0uIbTL57t+hyeqxZrV1v1Xt7KWczbw0leg0k1/7VoOLw6jGm55vPr+dW82MMJfxN5CE+rgbDcyUqKLti1FZrHx8fbwkM/ZnZJ/fs/tGGxv5PWVvMXA7JYu8R1BAe4110MrmdhszCmUNeuQanzrMIWt/ItmHkpzKOzZDVwNC/Gf7OjF3De+UgEIbfFYZ27G0twi36K43Sou179+7tZVftc4Tg8JDWoetaEWCd6yukVRLf39+8ebPOzs42Q69/dKd6CvSU0unco5QOcb6zfW8lfssRTIRFeigcab0KwwANrfWBsPeig+kVTXC9Uzm0KuBa+1X/NRaNwfUdOuiue96yXAXFbcUw71K43XKOJihaM1cDdej3FkU2VGMUypl0xwIuAZLqZJuDmZpuoWersDVo2JxyOM0sFZmVg4EEfLfyopQztOYAy6cYS41TkSR5F020HR8fb1k6iMU8Vi/pyL179/Zqw8i/hp8MGf9mY7WeRjyJ/UmzuL8Gzamh6vFIa+3e6eB4KIZAyMuBFjDQi74TkiMij6I1TrmlF2Tv757rXrjL29vbLbNaDnuOuWF1eXP6M7ONtTl3AYE7DdhXX321Hj58uJ1C2Th2rd15TTgMC83kNdYWVuEhGjr6jlNTG5YwLNBOicfugetich2lLXLDa1QZupfQQm+IdCidbZE29COXbqL9VKNE/f9HkxMPZVF5tj2lvCsov9baC2v0r4Wh2ty/qpFrZVQZ86Cz1dj1EEFzVwTeHQrTYXEOENHp6emmNzVudSJr7bY1+Yz+NYPu56H3ZZYLurq62o5qIqsmABrmM3ZQbUNLaLjjLJdaPYEuyzO9evVqG6t+kS/ZTPRKh3tsFKfuOJsaJOMpx8opGpt1Qw4SeYeykfpSIw/59b50WalWI5DpbD/V7jRg/+yf/bONgMZZ6UQRi2MvSmgSyuvXr7drdbwnN7TjhZ9NwfPChwj/eiyLtAKcz6No9+/f3yalZx/VCMxzwLpYKEZr20rMu5eJLkHvurV2IXcTCkU7rejvhnYIFp/UTfIWXjNDLUJuGOF5rcXTrzoDn2nlCt2/Bt+ROQ3JGMbJ/5TXRCPw9L1f+9frJnruM+7du7fxt3SH8YSarq6u9k4RRVv0jP5puMwnowpJ2dRM7nShiLRo8vT0dG/3wlofsv+e8/jx4z3HM9dYObfq5tu3bzdjhaMyX+oZGS1Gp8ABipMx9hzh3bNnz7aDAtb6sDboaus7yY7utqqggMff9PHBgwdbP38wAvvmm2/W559/vil7QzmxPA5DzF1isPxNs2iEQcEPbfGZJB9BNPykIA09QNVa/8vLy726Hd5rnjtVEv358+d7mbOpgC9fvlyvX7/ejM48f4shgzbWWnsGo89Ffh7KGOEtZmuxYeWx1m4vHhlyEhYl5RFGIHH1C9qc5Sf9vcWYFJ8+1OuXB1prZ1QtHE4H0QzRm2dOp+OjDzWiSOImjeiB7LJMeIlx15MZY+Zv+tTTNXB+7Uv5p/7fZzM5VKM0SeqW5Lx8+XKveLfGWfRiDoTp7s8hNCyE6HpdnVWpAPwfIGDerq+vtzny3W4NamPQCgIKMBqtSACY7373U+1OA/ajH/1oy/JZwA3xeEBWWhhZ6Co97O8lhNfahW2TY5nbPdbaWeaJFkrQCl0bY5eMbSi61u7U1z6XUH2XcbKIjcEG1xpaCjFLKSzQGrHXr1/vpdCNCQIw8c0QMljlg8iI520msgiqmSp9WGu32M/OzvZI26LZyhu6pGQMl6Zv+tmEwtHR0fYCGEkU6Mw1EAmyvoilaBu3wnBxtDUY19fXWxZNWNvxqGucYV3Ddk6WoWOk3INu0X+ois6Yl/Ky+j1PPSk6LcFdB2FezT2AgO5hiDof/glzOWeymJyUnxyfM/afPXu2RSfuVb6tIazxtphb3wGOrj2h9tu3b7f3kt5VQrHW91Ti/9Vf/dXt559/vr799tttV3o9aif46GhXu9F9YpMrWGttIaVJ7ELDVcmOzGyX32e4Y8FNEpJC8vQN13jSSXivtTN6nqe/NXgl3MkEMuhzKRSjPvtuwt2D0vN2vSeFaLU0+deL9u+HnukaY+hnPe1jZoE4ln6/WefO+bzvDAGnty5f2b4K9eZ3O67pqT8VdkBlUz+F6JUlg8jxNOt7SJ79XVlNEUa5xNm/SWx3TDi1WSx9SO79rM8qumy5C13pnsoZ/Qj9qyvTQZY+WGuHKumL+W6ENiv+G0LiPX/729+utdYPq8R//vz5XgU+D8NYWLCMVy1z0dpa+3H/zABCFSwzYyJMJZBJjHtu0/UlcGtMZZT0uWHAXEi8irHypicnH3bY8zwlwY1F/8nI+Jo5q/L6rEoOkZEJFFLiHPlZctf1zZTxaA0xagwZCL/rt3s4IrlOivL3TeZFMWvtG5TOsWdZaC0CZrAtVNf67kTWE2H2sy4GfZ3yIYciSH+HVtQhNhqAkNt3ffMWr7V2vGNlU8NNv+oEO8f6BjnTK4t/opOjow/8Z5HORHQtEzLXa+3zT+0D5Fnjg0M2T3UmDJH1UD7WXLofOuP4+EO9pgL5ho9z18lsdxqwP/7xj9smaDwJpe2GXJ3volhrf1f7mzdv9lK3a+1exlAuqTCboAhgGgzCrtdpuNWJKelOmBMtgdnKKIp43F8Y/PTp03VxcfER4ugrwXiStXZZGK18EwM5DYt/lT8Falq/ix167dhdh7xWkLrWbsG4x6wpmhufez8LqkZBOz4+3sJBYy2qM6YaYIaupQd1SuVEGrqSURHlIV6Q3E9PT7dxFT1cXv5/7d1bbhs5EIVhZgDlwVEuC8j+N5Yl2EDg2AnmYfC1flUUzWsMsIAgsmyp2WRdTx2yX66OsPFeMTtwirJ0ZlJt6mhKCbLNkHq/t+bW+panN0vSmX3PexV0Z3aqHO31y8BX0tMlgaVYHN1tGbrWxTGy9VsHBNC5QjN8gGDx/v37gzh7T+4SWSmPCRFdgMGzjJRqilIMrcCj7tJal6fOMPLWz67f0sgiFvDlBFsK9/VMXUs9KH7y6dOn4/l+6vNpADVm3ZgC/HCtlop/evyV1LzgeXGeifGJ5uawkVx54zNVbNeiRJOw2e5eywmGPcfR/wsDFDPyORiH7+LEkR77HVVyc6+0aPZOz2YJ47rdjqQEBCRXT9roKP9pUkE4sYnbzq7orDRKrG7553XXiKEXWpklbvV5QgikSUAPSqzuteFV4mpLZI7WZzsOHfDSo9pUcF8lNtNBVVQ7ro+Pj1c+AVVFMLlVblfuZmB1PD19VDbVL9aGbslSIK9YWbtqOj6UrYowjafcFpNk0ijMxMy6EL7DolvIlji3ukjmAlPafQKbLaR7LkZXY690Efu7mTlwmpzwxElankrh63g40wKpzWQ7TqWj02UnhjWPselOBOvXErNYUukN5RqVflFH6G85s2ajNRTjV9aL6uYJVUbjoIC6OZtgdhsqUwfpXOekOo6T18+oXDi+4lXtVNOf6m35UGutI+s3d+ZAF5tefvny5bc165jMT3cWmGP8zpaVxlRox+4O4ylvbq1L2djrnU6ng9Ffx27OYMwqkgbmW3LXgYk8ohFFEY1EKBfF3Xh9fT2ejDIjPQUuZiDFplxTaXSselKjyawy+F9WUGyje//mViOKwRBbVswsseXXWut4vJRxz6hnUaTOFH9un/H3kyxZbliFgvt9FeRY3H8uHLPpeH2ngGOcLX163WYkrssJdMNvQdkaNoJmx8oZTFwVQdI93GJxtwRTKchO/N3r6+txmqsdIBXzXJih1ICW8w1Eri87m9kzqoK/t47deQE68Hu24nvQkjzhuxia+Wwi0GeF6sR6v2stEJav1Yqnc9yDBa2byquMAtl2pT9bO+vjdJY6p8fHx6NTbF4bvP4kdx3Y169fj6cvW7SfPy/PlTufz1el4vPz87Gx+XS6nEw6Faf1usylXJO11pUB+FlG1zT/VtuZABpFD4aCXzRb6i2HvJbRzQd+yjREnVuM4ToIDxj1fjtexu5fDWhiDBShStHsdirgFMqx1uU4ms+fP19lmGtdIvOPHz8OHlWNoDwr+IimAydJ2UV1D2oo49q/W91mv1vr+uRS9wUzEa0LbPvshw8fjuZSsTvS7mY77cbZUk22KGNyb5NsKavgpIxZ6Vs9QpxlU+6z5E/jlpF67T45Jfrfknhua2NvBdEFVutaUi9MsHuPbZcri6D67poyqep4qRddV0nKXJPy0m7JXQf27du39fDwsM7n85FhUc46GHvATqfTkX6SpoETMKQka13jHiazxluQeCpL32+DgAF1LI3oLc8YooXivIppMNouyFrXWRBFkll8/Pjxt9RaZlgntNY1A7ugaTOyGvVa66qEmRmDwNKNsy2lv3//fnS5ZplWjIfzZrxVym79KYF1rUvTgMOxZWRmVda75b6svtiJzLkNgkodZiEHY+HsbzkynzVfXZMaXrdszSdZtXSuUddOXNdr9tHxmG/jnlibNTLnhVTa2Zx0Jusxn+LknDSZM5IqG7Fesmx8Os6+2XG5kYJlEyCwSwN+78c9dy7vUb3+l8jqkD9ZCQXWPaSQf3IkQGzYRd+D3yhbHh4ejpttK306CotRRfD+09PTVaQooCtFLjdrRm6L1izMvdjA2jFRhCoykWY/Pz9f7cG0xaN438vLy/EUdMfsKiPm9TrP82EOa13O6xKJi7ettQ6nVWOGSRBOquuw1mWfJYPvxnqf5zjMi5KBg5DFtQtWPpPrNNtop7mvZ0NC4AJPwPVkEsUkhCUAAACcSURBVNayTmHumGBgpXS4N8bbkpqT4PS7D7I0iH737OS2IYMKMaXZlLUrp8sc/Pr1XzOpeB2HVmjn3bt3h7NCYtYcO5/PR0LSTeASlzbvev+lQ1m/VhtK3Y5pYpFz29Y9uUtk3bJly5a/We5mYFu2bNnyN8t2YFu2bHmzsh3Yli1b3qxsB7Zly5Y3K9uBbdmy5c3KdmBbtmx5s/IvSjj13Upl9AQAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "![](\"images/stapler_bbox.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Some of the bounding boxes do have the stapler or at least most of it in the box, which can assist the classification process."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### R-CNN and Faster R-CNN\n",
+ "\n",
+ "[Ross Girshick et al.](https://arxiv.org/pdf/1311.2524.pdf) proposed a method where they use selective search to extract just 2000 regions from the image. Then the regions in bounding boxes are feed into a convolutional neural network to perform classification. The brief architecture can be shown as:\n",
+ "\n",
+ "![](\"images/RCNN.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The problem with R-CNN is that one must pass each box independently through an image classifier thus it takes a huge amount of time to train the network. And meanwhile, the selective search is not that stable and sometimes may generate bad examples.\n",
+ "\n",
+ "Faster R-CNN solved the drawbacks of R-CNN by applying a faster object detection algorithm. Instead of feeding the region proposals to the CNN, we feed the input image to the CNN to generate a convolutional feature map. Then we identify the region of interests on the feature map and then reshape them into a fixed size with an ROI pooling layer so it can be put into another classifier. \n",
+ "\n",
+ "This algorithm is faster than R-CNN as the image is not frequently fed into the CNN to extract feature maps."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Implementation\n",
+ "\n",
+ "For an ROI pooling layer, we implemented a simple demo of it as `pool_rois`. We can fake a simple feature map with `numpy`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "feature_maps_shape = (200, 100, 1)\n",
+ "feature_map = np.ones(feature_maps_shape, dtype='float32')\n",
+ "feature_map[200 - 1, 100 - 3, 0] = 50"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note that the fake feature map is all 1 except for one spot with a value of 50. Now let's generate some regio of interests:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "roiss = np.asarray([[0.5, 0.2, 0.7, 0.4], [0.0, 0.0, 1.0, 1.0]])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we only made up two regions of interest. The first only crops some part of the image where all pixels are '1' which ranges from 0.5-0.7 of the length of the horizontal edge and 0.2-0.4 of verticle edge. The range of the second region is the whole image. Now let's pool a 3x7 area out of each region of interest."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[array([[1., 1., 1., 1., 1., 1., 1.],\n",
+ " [1., 1., 1., 1., 1., 1., 1.],\n",
+ " [1., 1., 1., 1., 1., 1., 1.]], dtype=float32),\n",
+ " array([[ 1., 1., 1., 1., 1., 1., 1.],\n",
+ " [ 1., 1., 1., 1., 1., 1., 1.],\n",
+ " [ 1., 1., 1., 1., 1., 1., 50.]], dtype=float32)]"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "pool_rois(feature_map, roiss, 3, 7)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "What we are expecting is that the second pooled region is different from the first one as there is an artificial feature-the '50' in its input. The printed result is exactly the same as we expected."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In order to try the whole algorithm of the Faster R-CNN, you can refer to [this GitHub repository](https://github.com/endernewton/tf-faster-rcnn) for more detailed guidance."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/chapter24/images/RCNN.png b/notebooks/chapter24/images/RCNN.png
new file mode 100644
index 000000000..273021fbe
Binary files /dev/null and b/notebooks/chapter24/images/RCNN.png differ
diff --git a/notebooks/chapter24/images/derivative_of_gaussian.png b/notebooks/chapter24/images/derivative_of_gaussian.png
new file mode 100644
index 000000000..0be575529
Binary files /dev/null and b/notebooks/chapter24/images/derivative_of_gaussian.png differ
diff --git a/notebooks/chapter24/images/gradients.png b/notebooks/chapter24/images/gradients.png
new file mode 100644
index 000000000..ae57bdf3b
Binary files /dev/null and b/notebooks/chapter24/images/gradients.png differ
diff --git a/notebooks/chapter24/images/laplacian.png b/notebooks/chapter24/images/laplacian.png
new file mode 100644
index 000000000..6d7e6916a
Binary files /dev/null and b/notebooks/chapter24/images/laplacian.png differ
diff --git a/notebooks/chapter24/images/laplacian_kernels.png b/notebooks/chapter24/images/laplacian_kernels.png
new file mode 100644
index 000000000..faca3321c
Binary files /dev/null and b/notebooks/chapter24/images/laplacian_kernels.png differ
diff --git a/notebooks/chapter24/images/stapler.png b/notebooks/chapter24/images/stapler.png
new file mode 100644
index 000000000..e550d83f9
Binary files /dev/null and b/notebooks/chapter24/images/stapler.png differ
diff --git a/notebooks/chapter24/images/stapler_bbox.png b/notebooks/chapter24/images/stapler_bbox.png
new file mode 100644
index 000000000..c5a7c7af0
Binary files /dev/null and b/notebooks/chapter24/images/stapler_bbox.png differ
diff --git a/obsolete_search4e.ipynb b/obsolete_search4e.ipynb
new file mode 100644
index 000000000..72981d49b
--- /dev/null
+++ b/obsolete_search4e.ipynb
@@ -0,0 +1,3835 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "*Note: This is not yet ready, but shows the direction I'm leaning in for Fourth Edition Search.*\n",
+ "\n",
+ "# State-Space Search\n",
+ "\n",
+ "This notebook describes several state-space search algorithms, and how they can be used to solve a variety of problems. We start with a simple algorithm and a simple domain: finding a route from city to city. Later we will explore other algorithms and domains.\n",
+ "\n",
+ "## The Route-Finding Domain\n",
+ "\n",
+ "Like all state-space search problems, in a route-finding problem you will be given:\n",
+ "- A start state (for example, `'A'` for the city Arad).\n",
+ "- A goal state (for example, `'B'` for the city Bucharest).\n",
+ "- Actions that can change state (for example, driving from `'A'` to `'S'`).\n",
+ "\n",
+ "You will be asked to find:\n",
+ "- A path from the start state, through intermediate states, to the goal state.\n",
+ "\n",
+ "We'll use this map:\n",
+ "\n",
+ "![](\"http://robotics.cs.tamu.edu/dshell/cs625/images/map.jpg\")
\n",
+ "\n",
+ "A state-space search problem can be represented by a *graph*, where the vertices of the graph are the states of the problem (in this case, cities) and the edges of the graph are the actions (in this case, driving along a road).\n",
+ "\n",
+ "We'll represent a city by its single initial letter. \n",
+ "We'll represent the graph of connections as a `dict` that maps each city to a list of the neighboring cities (connected by a road). For now we don't explicitly represent the actions, nor the distances\n",
+ "between cities."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "romania = {\n",
+ " 'A': ['Z', 'T', 'S'],\n",
+ " 'B': ['F', 'P', 'G', 'U'],\n",
+ " 'C': ['D', 'R', 'P'],\n",
+ " 'D': ['M', 'C'],\n",
+ " 'E': ['H'],\n",
+ " 'F': ['S', 'B'],\n",
+ " 'G': ['B'],\n",
+ " 'H': ['U', 'E'],\n",
+ " 'I': ['N', 'V'],\n",
+ " 'L': ['T', 'M'],\n",
+ " 'M': ['L', 'D'],\n",
+ " 'N': ['I'],\n",
+ " 'O': ['Z', 'S'],\n",
+ " 'P': ['R', 'C', 'B'],\n",
+ " 'R': ['S', 'C', 'P'],\n",
+ " 'S': ['A', 'O', 'F', 'R'],\n",
+ " 'T': ['A', 'L'],\n",
+ " 'U': ['B', 'V', 'H'],\n",
+ " 'V': ['U', 'I'],\n",
+ " 'Z': ['O', 'A']}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "Suppose we want to get from `A` to `B`. Where can we go from the start state, `A`?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['Z', 'T', 'S']"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "romania['A']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "We see that from `A` we can get to any of the three cities `['Z', 'T', 'S']`. Which should we choose? *We don't know.* That's the whole point of *search*: we don't know which immediate action is best, so we'll have to explore, until we find a *path* that leads to the goal. \n",
+ "\n",
+ "How do we explore? We'll start with a simple algorithm that will get us from `A` to `B`. We'll keep a *frontier*—a collection of not-yet-explored states—and expand the frontier outward until it reaches the goal. To be more precise:\n",
+ "\n",
+ "- Initially, the only state in the frontier is the start state, `'A'`.\n",
+ "- Until we reach the goal, or run out of states in the frontier to explore, do the following:\n",
+ " - Remove the first state from the frontier. Call it `s`.\n",
+ " - If `s` is the goal, we're done. Return the path to `s`.\n",
+ " - Otherwise, consider all the neighboring states of `s`. For each one:\n",
+ " - If we have not previously explored the state, add it to the end of the frontier.\n",
+ " - Also keep track of the previous state that led to this new neighboring state; we'll need this to reconstruct the path to the goal, and to keep us from re-visiting previously explored states.\n",
+ " \n",
+ "# A Simple Search Algorithm: `breadth_first`\n",
+ " \n",
+ "The function `breadth_first` implements this strategy:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "from collections import deque # Doubly-ended queue: pop from left, append to right.\n",
+ "\n",
+ "def breadth_first(start, goal, neighbors):\n",
+ " \"Find a shortest sequence of states from start to the goal.\"\n",
+ " frontier = deque([start]) # A queue of states\n",
+ " previous = {start: None} # start has no previous state; other states will\n",
+ " while frontier:\n",
+ " s = frontier.popleft()\n",
+ " if s == goal:\n",
+ " return path(previous, s)\n",
+ " for s2 in neighbors[s]:\n",
+ " if s2 not in previous:\n",
+ " frontier.append(s2)\n",
+ " previous[s2] = s\n",
+ " \n",
+ "def path(previous, s): \n",
+ " \"Return a list of states that lead to state s, according to the previous dict.\"\n",
+ " return [] if (s is None) else path(previous, previous[s]) + [s]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "A couple of things to note: \n",
+ "\n",
+ "1. We always add new states to the end of the frontier queue. That means that all the states that are adjacent to the start state will come first in the queue, then all the states that are two steps away, then three steps, etc.\n",
+ "That's what we mean by *breadth-first* search.\n",
+ "2. We recover the path to an `end` state by following the trail of `previous[end]` pointers, all the way back to `start`.\n",
+ "The dict `previous` is a map of `{state: previous_state}`. \n",
+ "3. When we finally get an `s` that is the goal state, we know we have found a shortest path, because any other state in the queue must correspond to a path that is as long or longer.\n",
+ "3. Note that `previous` contains all the states that are currently in `frontier` as well as all the states that were in `frontier` in the past.\n",
+ "4. If no path to the goal is found, then `breadth_first` returns `None`. If a path is found, it returns the sequence of states on the path.\n",
+ "\n",
+ "Some examples:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['A', 'S', 'F', 'B']"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('A', 'B', romania)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['L', 'T', 'A', 'S', 'F', 'B', 'U', 'V', 'I', 'N']"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('L', 'N', romania)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['N', 'I', 'V', 'U', 'B', 'F', 'S', 'A', 'T', 'L']"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('N', 'L', romania)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['E']"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('E', 'E', romania)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "Now let's try a different kind of problem that can be solved with the same search function.\n",
+ "\n",
+ "## Word Ladders Problem\n",
+ "\n",
+ "A *word ladder* problem is this: given a start word and a goal word, find the shortest way to transform the start word into the goal word by changing one letter at a time, such that each change results in a word. For example starting with `green` we can reach `grass` in 7 steps:\n",
+ "\n",
+ "`green` → `greed` → `treed` → `trees` → `tress` → `cress` → `crass` → `grass`\n",
+ "\n",
+ "We will need a dictionary of words. We'll use 5-letter words from the [Stanford GraphBase](http://www-cs-faculty.stanford.edu/~uno/sgb.html) project for this purpose. Let's get that file from aimadata."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "from search import *\n",
+ "sgb_words = open_data(\"EN-text/sgb-words.txt\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "We can assign `WORDS` to be the set of all the words in this file:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "5757"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "WORDS = set(sgb_words.read().split())\n",
+ "len(WORDS)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "And define `neighboring_words` to return the set of all words that are a one-letter change away from a given `word`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def neighboring_words(word):\n",
+ " \"All words that are one letter away from this word.\"\n",
+ " neighbors = {word[:i] + c + word[i+1:]\n",
+ " for i in range(len(word))\n",
+ " for c in 'abcdefghijklmnopqrstuvwxyz'\n",
+ " if c != word[i]}\n",
+ " return neighbors & WORDS"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'cello', 'hallo', 'hells', 'hullo', 'jello'}"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "neighboring_words('hello')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'would'}"
+ ]
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "neighboring_words('world')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "Now we can create `word_neighbors` as a dict of `{word: {neighboring_word, ...}}`: "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "word_neighbors = {word: neighboring_words(word)\n",
+ " for word in WORDS}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "Now the `breadth_first` function can be used to solve a word ladder problem:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['green', 'greed', 'treed', 'trees', 'treys', 'trays', 'grays', 'grass']"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('green', 'grass', word_neighbors)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['smart',\n",
+ " 'start',\n",
+ " 'stars',\n",
+ " 'sears',\n",
+ " 'bears',\n",
+ " 'beans',\n",
+ " 'brans',\n",
+ " 'brand',\n",
+ " 'braid',\n",
+ " 'brain']"
+ ]
+ },
+ "execution_count": 15,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('smart', 'brain', word_neighbors)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['frown',\n",
+ " 'flown',\n",
+ " 'flows',\n",
+ " 'slows',\n",
+ " 'slots',\n",
+ " 'slits',\n",
+ " 'spits',\n",
+ " 'spite',\n",
+ " 'smite',\n",
+ " 'smile']"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "breadth_first('frown', 'smile', word_neighbors)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "# More General Search Algorithms\n",
+ "\n",
+ "Now we'll embelish the `breadth_first` algorithm to make a family of search algorithms with more capabilities:\n",
+ "\n",
+ "1. We distinguish between an *action* and the *result* of an action.\n",
+ "3. We allow different measures of the cost of a solution (not just the number of steps in the sequence).\n",
+ "4. We search through the state space in an order that is more likely to lead to an optimal solution quickly.\n",
+ "\n",
+ "Here's how we do these things:\n",
+ "\n",
+ "1. Instead of having a graph of neighboring states, we instead have an object of type *Problem*. A Problem\n",
+ "has one method, `Problem.actions(state)` to return a collection of the actions that are allowed in a state,\n",
+ "and another method, `Problem.result(state, action)` that says what happens when you take an action.\n",
+ "2. We keep a set, `explored` of states that have already been explored. We also have a class, `Frontier`, that makes it efficient to ask if a state is on the frontier.\n",
+ "3. Each action has a cost associated with it (in fact, the cost can vary with both the state and the action).\n",
+ "4. The `Frontier` class acts as a priority queue, allowing the \"best\" state to be explored next.\n",
+ "We represent a sequence of actions and resulting states as a linked list of `Node` objects.\n",
+ "\n",
+ "The algorithm `breadth_first_search` is basically the same as `breadth_first`, but using our new conventions:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def breadth_first_search(problem):\n",
+ " \"Search for goal; paths with least number of steps first.\"\n",
+ " if problem.is_goal(problem.initial): \n",
+ " return Node(problem.initial)\n",
+ " frontier = FrontierQ(Node(problem.initial), LIFO=False)\n",
+ " explored = set()\n",
+ " while frontier:\n",
+ " node = frontier.pop()\n",
+ " explored.add(node.state)\n",
+ " for action in problem.actions(node.state):\n",
+ " child = node.child(problem, action)\n",
+ " if child.state not in explored and child.state not in frontier:\n",
+ " if problem.is_goal(child.state):\n",
+ " return child\n",
+ " frontier.add(child)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next is `uniform_cost_search`, in which each step can have a different cost, and we still consider first one os the states with minimum cost so far."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def uniform_cost_search(problem, costfn=lambda node: node.path_cost):\n",
+ " frontier = FrontierPQ(Node(problem.initial), costfn)\n",
+ " explored = set()\n",
+ " while frontier:\n",
+ " node = frontier.pop()\n",
+ " if problem.is_goal(node.state):\n",
+ " return node\n",
+ " explored.add(node.state)\n",
+ " for action in problem.actions(node.state):\n",
+ " child = node.child(problem, action)\n",
+ " if child.state not in explored and child not in frontier:\n",
+ " frontier.add(child)\n",
+ " elif child in frontier and frontier.cost[child] < child.path_cost:\n",
+ " frontier.replace(child)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Finally, `astar_search` in which the cost includes an estimate of the distance to the goal as well as the distance travelled so far."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def astar_search(problem, heuristic):\n",
+ " costfn = lambda node: node.path_cost + heuristic(node.state)\n",
+ " return uniform_cost_search(problem, costfn)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "# Search Tree Nodes\n",
+ "\n",
+ "The solution to a search problem is now a linked list of `Node`s, where each `Node`\n",
+ "includes a `state` and the `path_cost` of getting to the state. In addition, for every `Node` except for the first (root) `Node`, there is a previous `Node` (indicating the state that lead to this `Node`) and an `action` (indicating the action taken to get here)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "class Node(object):\n",
+ " \"\"\"A node in a search tree. A search tree is spanning tree over states.\n",
+ " A Node contains a state, the previous node in the tree, the action that\n",
+ " takes us from the previous state to this state, and the path cost to get to \n",
+ " this state. If a state is arrived at by two paths, then there are two nodes \n",
+ " with the same state.\"\"\"\n",
+ "\n",
+ " def __init__(self, state, previous=None, action=None, step_cost=1):\n",
+ " \"Create a search tree Node, derived from a previous Node by an action.\"\n",
+ " self.state = state\n",
+ " self.previous = previous\n",
+ " self.action = action\n",
+ " self.path_cost = 0 if previous is None else (previous.path_cost + step_cost)\n",
+ "\n",
+ " def __repr__(self): return \"\n", + "\n", + "**Planning in the real world**\n", + "- Problem\n", + "- HLA\n", + "- Planning Problems\n", + " * Job shop problem\n", + " * Double tennis problem\n", + "- Solving Planning Problems\n", + " * Hierarchical Search\n", + " * Angelic Search" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PlanningProblem\n", + "\n", + "PDDL stands for Planning Domain Definition Language.\n", + "The `PlanningProblem` class is used to represent planning problems in this module. The following attributes are essential to be able to define a problem:\n", + "* an initial state\n", + "* a set of goals\n", + "* a set of viable actions that can be executed in the search space of the problem\n", + "\n", + "View the source to see how the Python code tries to realise these." ] }, { "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, + "execution_count": 80, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class PlanningProblem:\n",
+ " """\n",
+ " Planning Domain Definition Language (PlanningProblem) used to define a search problem.\n",
+ " It stores states in a knowledge base consisting of first order logic statements.\n",
+ " The conjunction of these logical statements completely defines a state.\n",
+ " """\n",
+ "\n",
+ " def __init__(self, init, goals, actions):\n",
+ " self.init = self.convert(init)\n",
+ " self.goals = self.convert(goals)\n",
+ " self.actions = actions\n",
+ "\n",
+ " def convert(self, clauses):\n",
+ " """Converts strings into exprs"""\n",
+ " if not isinstance(clauses, Expr):\n",
+ " if len(clauses) > 0:\n",
+ " clauses = expr(clauses)\n",
+ " else:\n",
+ " clauses = []\n",
+ " try:\n",
+ " clauses = conjuncts(clauses)\n",
+ " except AttributeError:\n",
+ " clauses = clauses\n",
+ "\n",
+ " new_clauses = []\n",
+ " for clause in clauses:\n",
+ " if clause.op == '~':\n",
+ " new_clauses.append(expr('Not' + str(clause.args[0])))\n",
+ " else:\n",
+ " new_clauses.append(clause)\n",
+ " return new_clauses\n",
+ "\n",
+ " def goal_test(self):\n",
+ " """Checks if the goals have been reached"""\n",
+ " return all(goal in self.init for goal in self.goals)\n",
+ "\n",
+ " def act(self, action):\n",
+ " """\n",
+ " Performs the action given as argument.\n",
+ " Note that action is an Expr like expr('Remove(Glass, Table)') or expr('Eat(Sandwich)')\n",
+ " """ \n",
+ " action_name = action.op\n",
+ " args = action.args\n",
+ " list_action = first(a for a in self.actions if a.name == action_name)\n",
+ " if list_action is None:\n",
+ " raise Exception("Action '{}' not found".format(action_name))\n",
+ " if not list_action.check_precond(self.init, args):\n",
+ " raise Exception("Action '{}' pre-conditions not satisfied".format(action))\n",
+ " self.init = list_action(self.init, args).clauses\n",
+ "\n", + "The `goals` attribute is an expression that indicates the goals to be reached by the problem.\n", + "
\n", + "Lastly, `actions` contains a list of `Action` objects that may be executed in the search space of the problem.\n", + "
\n", + "The `goal_test` method checks if the goal has been reached.\n", + "
\n", + "The `act` method acts out the given action and updates the current state.\n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## ACTION\n", + "\n", + "To be able to model a planning problem properly, it is essential to be able to represent an Action. Each action we model requires at least three things:\n", + "* preconditions that the action must meet\n", + "* the effects of executing the action\n", + "* some expression that represents the action" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The module models actions using the `Action` class" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class Action:\n",
+ " """\n",
+ " Defines an action schema using preconditions and effects.\n",
+ " Use this to describe actions in PlanningProblem.\n",
+ " action is an Expr where variables are given as arguments(args).\n",
+ " Precondition and effect are both lists with positive and negative literals.\n",
+ " Negative preconditions and effects are defined by adding a 'Not' before the name of the clause\n",
+ " Example:\n",
+ " precond = [expr("Human(person)"), expr("Hungry(Person)"), expr("NotEaten(food)")]\n",
+ " effect = [expr("Eaten(food)"), expr("Hungry(person)")]\n",
+ " eat = Action(expr("Eat(person, food)"), precond, effect)\n",
+ " """\n",
+ "\n",
+ " def __init__(self, action, precond, effect):\n",
+ " if isinstance(action, str):\n",
+ " action = expr(action)\n",
+ " self.name = action.op\n",
+ " self.args = action.args\n",
+ " self.precond = self.convert(precond)\n",
+ " self.effect = self.convert(effect)\n",
+ "\n",
+ " def __call__(self, kb, args):\n",
+ " return self.act(kb, args)\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return '{}({})'.format(self.__class__.__name__, Expr(self.name, *self.args))\n",
+ "\n",
+ " def convert(self, clauses):\n",
+ " """Converts strings into Exprs"""\n",
+ " if isinstance(clauses, Expr):\n",
+ " clauses = conjuncts(clauses)\n",
+ " for i in range(len(clauses)):\n",
+ " if clauses[i].op == '~':\n",
+ " clauses[i] = expr('Not' + str(clauses[i].args[0]))\n",
+ "\n",
+ " elif isinstance(clauses, str):\n",
+ " clauses = clauses.replace('~', 'Not')\n",
+ " if len(clauses) > 0:\n",
+ " clauses = expr(clauses)\n",
+ "\n",
+ " try:\n",
+ " clauses = conjuncts(clauses)\n",
+ " except AttributeError:\n",
+ " pass\n",
+ "\n",
+ " return clauses\n",
+ "\n",
+ " def substitute(self, e, args):\n",
+ " """Replaces variables in expression with their respective Propositional symbol"""\n",
+ "\n",
+ " new_args = list(e.args)\n",
+ " for num, x in enumerate(e.args):\n",
+ " for i, _ in enumerate(self.args):\n",
+ " if self.args[i] == x:\n",
+ " new_args[num] = args[i]\n",
+ " return Expr(e.op, *new_args)\n",
+ "\n",
+ " def check_precond(self, kb, args):\n",
+ " """Checks if the precondition is satisfied in the current state"""\n",
+ "\n",
+ " if isinstance(kb, list):\n",
+ " kb = FolKB(kb)\n",
+ " for clause in self.precond:\n",
+ " if self.substitute(clause, args) not in kb.clauses:\n",
+ " return False\n",
+ " return True\n",
+ "\n",
+ " def act(self, kb, args):\n",
+ " """Executes the action on the state's knowledge base"""\n",
+ "\n",
+ " if isinstance(kb, list):\n",
+ " kb = FolKB(kb)\n",
+ "\n",
+ " if not self.check_precond(kb, args):\n",
+ " raise Exception('Action pre-conditions not satisfied')\n",
+ " for clause in self.effect:\n",
+ " kb.tell(self.substitute(clause, args))\n",
+ " if clause.op[:3] == 'Not':\n",
+ " new_clause = Expr(clause.op[3:], *clause.args)\n",
+ "\n",
+ " if kb.ask(self.substitute(new_clause, args)) is not False:\n",
+ " kb.retract(self.substitute(new_clause, args))\n",
+ " else:\n",
+ " new_clause = Expr('Not' + clause.op, *clause.args)\n",
+ "\n",
+ " if kb.ask(self.substitute(new_clause, args)) is not False: \n",
+ " kb.retract(self.substitute(new_clause, args))\n",
+ "\n",
+ " return kb\n",
+ "def air_cargo():\n",
+ " """\n",
+ " [Figure 10.1] AIR-CARGO-PROBLEM\n",
+ "\n",
+ " An air-cargo shipment problem for delivering cargo to different locations,\n",
+ " given the starting location and airplanes.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> ac = air_cargo()\n",
+ " >>> ac.goal_test()\n",
+ " False\n",
+ " >>> ac.act(expr('Load(C2, P2, JFK)'))\n",
+ " >>> ac.act(expr('Load(C1, P1, SFO)'))\n",
+ " >>> ac.act(expr('Fly(P1, SFO, JFK)'))\n",
+ " >>> ac.act(expr('Fly(P2, JFK, SFO)'))\n",
+ " >>> ac.act(expr('Unload(C2, P2, SFO)'))\n",
+ " >>> ac.goal_test()\n",
+ " False\n",
+ " >>> ac.act(expr('Unload(C1, P1, JFK)'))\n",
+ " >>> ac.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK) & Cargo(C1) & Cargo(C2) & Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)', \n",
+ " goals='At(C1, JFK) & At(C2, SFO)',\n",
+ " actions=[Action('Load(c, p, a)', \n",
+ " precond='At(c, a) & At(p, a) & Cargo(c) & Plane(p) & Airport(a)',\n",
+ " effect='In(c, p) & ~At(c, a)'),\n",
+ " Action('Unload(c, p, a)',\n",
+ " precond='In(c, p) & At(p, a) & Cargo(c) & Plane(p) & Airport(a)',\n",
+ " effect='At(c, a) & ~In(c, p)'),\n",
+ " Action('Fly(p, f, to)',\n",
+ " precond='At(p, f) & Plane(p) & Airport(f) & Airport(to)',\n",
+ " effect='At(p, to) & ~At(p, f)')])\n",
+ "def spare_tire():\n",
+ " """[Figure 10.2] SPARE-TIRE-PROBLEM\n",
+ "\n",
+ " A problem involving changing the flat tire of a car\n",
+ " with a spare tire from the trunk.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> st = spare_tire()\n",
+ " >>> st.goal_test()\n",
+ " False\n",
+ " >>> st.act(expr('Remove(Spare, Trunk)'))\n",
+ " >>> st.act(expr('Remove(Flat, Axle)'))\n",
+ " >>> st.goal_test()\n",
+ " False\n",
+ " >>> st.act(expr('PutOn(Spare, Axle)'))\n",
+ " >>> st.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='Tire(Flat) & Tire(Spare) & At(Flat, Axle) & At(Spare, Trunk)',\n",
+ " goals='At(Spare, Axle) & At(Flat, Ground)',\n",
+ " actions=[Action('Remove(obj, loc)',\n",
+ " precond='At(obj, loc)',\n",
+ " effect='At(obj, Ground) & ~At(obj, loc)'),\n",
+ " Action('PutOn(t, Axle)',\n",
+ " precond='Tire(t) & At(t, Ground) & ~At(Flat, Axle)',\n",
+ " effect='At(t, Axle) & ~At(t, Ground)'),\n",
+ " Action('LeaveOvernight',\n",
+ " precond='',\n",
+ " effect='~At(Spare, Ground) & ~At(Spare, Axle) & ~At(Spare, Trunk) & \\\n",
+ " ~At(Flat, Ground) & ~At(Flat, Axle) & ~At(Flat, Trunk)')])\n",
+ "\n", + "Another possible solution is" + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, "outputs": [], - "source": [] + "source": [ + "spareTire = spare_tire()\n", + "\n", + "solution = [expr('Remove(Spare, Trunk)'),\n", + " expr('Remove(Flat, Axle)'),\n", + " expr('PutOn(Spare, Axle)')]\n", + "\n", + "for action in solution:\n", + " spareTire.act(action)" + ] + }, + { + "cell_type": "code", + "execution_count": 101, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n" + ] + } + ], + "source": [ + "print(spareTire.goal_test())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that both solutions work, which means that the problem can be solved irrespective of the order in which the `Remove` actions take place, as long as both `Remove` actions take place before the `PutOn` action." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have successfully mounted a spare tire onto the axle." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Three Block Tower Problem" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This problem's domain consists of a set of cube-shaped blocks sitting on a table. \n", + "The blocks can be stacked, but only one block can fit directly on top of another.\n", + "A robot arm can pick up a block and move it to another position, either on the table or on top of another block. \n", + "The arm can pick up only one block at a time, so it cannot pick up a block that has another one on it. \n", + "The goal will always be to build one or more stacks of blocks. \n", + "In our case, we consider only three blocks.\n", + "The particular configuration we will use is called the Sussman anomaly after Prof. Gerry Sussman." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's take a look at the definition of `three_block_tower()` in the module." + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def three_block_tower():\n",
+ " """\n",
+ " [Figure 10.3] THREE-BLOCK-TOWER\n",
+ "\n",
+ " A blocks-world problem of stacking three blocks in a certain configuration,\n",
+ " also known as the Sussman Anomaly.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> tbt = three_block_tower()\n",
+ " >>> tbt.goal_test()\n",
+ " False\n",
+ " >>> tbt.act(expr('MoveToTable(C, A)'))\n",
+ " >>> tbt.act(expr('Move(B, Table, C)'))\n",
+ " >>> tbt.goal_test()\n",
+ " False\n",
+ " >>> tbt.act(expr('Move(A, Table, B)'))\n",
+ " >>> tbt.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='On(A, Table) & On(B, Table) & On(C, A) & Block(A) & Block(B) & Block(C) & Clear(B) & Clear(C)',\n",
+ " goals='On(A, B) & On(B, C)',\n",
+ " actions=[Action('Move(b, x, y)',\n",
+ " precond='On(b, x) & Clear(b) & Clear(y) & Block(b) & Block(y)',\n",
+ " effect='On(b, y) & Clear(x) & ~On(b, x) & ~Clear(y)'),\n",
+ " Action('MoveToTable(b, x)',\n",
+ " precond='On(b, x) & Clear(b) & Block(b)',\n",
+ " effect='On(b, Table) & Clear(x) & ~On(b, x)')])\n",
+ "def simple_blocks_world():\n",
+ " """\n",
+ " SIMPLE-BLOCKS-WORLD\n",
+ "\n",
+ " A simplified definition of the Sussman Anomaly problem.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> sbw = simple_blocks_world()\n",
+ " >>> sbw.goal_test()\n",
+ " False\n",
+ " >>> sbw.act(expr('ToTable(A, B)'))\n",
+ " >>> sbw.act(expr('FromTable(B, A)'))\n",
+ " >>> sbw.goal_test()\n",
+ " False\n",
+ " >>> sbw.act(expr('FromTable(C, B)'))\n",
+ " >>> sbw.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='On(A, B) & Clear(A) & OnTable(B) & OnTable(C) & Clear(C)',\n",
+ " goals='On(B, A) & On(C, B)',\n",
+ " actions=[Action('ToTable(x, y)',\n",
+ " precond='On(x, y) & Clear(x)',\n",
+ " effect='~On(x, y) & Clear(y) & OnTable(x)'),\n",
+ " Action('FromTable(y, x)',\n",
+ " precond='OnTable(y) & Clear(y) & Clear(x)',\n",
+ " effect='~OnTable(y) & ~Clear(x) & On(y, x)')])\n",
+ "def shopping_problem():\n",
+ " """\n",
+ " SHOPPING-PROBLEM\n",
+ "\n",
+ " A problem of acquiring some items given their availability at certain stores.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> sp = shopping_problem()\n",
+ " >>> sp.goal_test()\n",
+ " False\n",
+ " >>> sp.act(expr('Go(Home, HW)'))\n",
+ " >>> sp.act(expr('Buy(Drill, HW)'))\n",
+ " >>> sp.act(expr('Go(HW, SM)'))\n",
+ " >>> sp.act(expr('Buy(Banana, SM)'))\n",
+ " >>> sp.goal_test()\n",
+ " False\n",
+ " >>> sp.act(expr('Buy(Milk, SM)'))\n",
+ " >>> sp.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='At(Home) & Sells(SM, Milk) & Sells(SM, Banana) & Sells(HW, Drill)',\n",
+ " goals='Have(Milk) & Have(Banana) & Have(Drill)', \n",
+ " actions=[Action('Buy(x, store)',\n",
+ " precond='At(store) & Sells(store, x)',\n",
+ " effect='Have(x)'),\n",
+ " Action('Go(x, y)',\n",
+ " precond='At(x)',\n",
+ " effect='At(y) & ~At(x)')])\n",
+ "def socks_and_shoes():\n",
+ " """\n",
+ " SOCKS-AND-SHOES-PROBLEM\n",
+ "\n",
+ " A task of wearing socks and shoes on both feet\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> ss = socks_and_shoes()\n",
+ " >>> ss.goal_test()\n",
+ " False\n",
+ " >>> ss.act(expr('RightSock'))\n",
+ " >>> ss.act(expr('RightShoe'))\n",
+ " >>> ss.act(expr('LeftSock'))\n",
+ " >>> ss.goal_test()\n",
+ " False\n",
+ " >>> ss.act(expr('LeftShoe'))\n",
+ " >>> ss.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='',\n",
+ " goals='RightShoeOn & LeftShoeOn',\n",
+ " actions=[Action('RightShoe',\n",
+ " precond='RightSockOn',\n",
+ " effect='RightShoeOn'),\n",
+ " Action('RightSock',\n",
+ " precond='',\n",
+ " effect='RightSockOn'),\n",
+ " Action('LeftShoe',\n",
+ " precond='LeftSockOn',\n",
+ " effect='LeftShoeOn'),\n",
+ " Action('LeftSock',\n",
+ " precond='',\n",
+ " effect='LeftSockOn')])\n",
+ "def have_cake_and_eat_cake_too():\n",
+ " """\n",
+ " [Figure 10.7] CAKE-PROBLEM\n",
+ "\n",
+ " A problem where we begin with a cake and want to \n",
+ " reach the state of having a cake and having eaten a cake.\n",
+ " The possible actions include baking a cake and eating a cake.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> cp = have_cake_and_eat_cake_too()\n",
+ " >>> cp.goal_test()\n",
+ " False\n",
+ " >>> cp.act(expr('Eat(Cake)'))\n",
+ " >>> cp.goal_test()\n",
+ " False\n",
+ " >>> cp.act(expr('Bake(Cake)'))\n",
+ " >>> cp.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='Have(Cake)',\n",
+ " goals='Have(Cake) & Eaten(Cake)',\n",
+ " actions=[Action('Eat(Cake)',\n",
+ " precond='Have(Cake)',\n",
+ " effect='Eaten(Cake) & ~Have(Cake)'),\n",
+ " Action('Bake(Cake)',\n",
+ " precond='~Have(Cake)',\n",
+ " effect='Have(Cake)')])\n",
+ "\n", + "`hierarchical_search` and `angelic_search` are also built into the `Problem` class to solve such planning problems." + ] + }, + { + "cell_type": "code", + "execution_count": 129, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class Problem(PlanningProblem):\n",
+ " """\n",
+ " Define real-world problems by aggregating resources as numerical quantities instead of\n",
+ " named entities.\n",
+ "\n",
+ " This class is identical to PDLL, except that it overloads the act function to handle\n",
+ " resource and ordering conditions imposed by HLA as opposed to Action.\n",
+ " """\n",
+ " def __init__(self, init, goals, actions, jobs=None, resources=None):\n",
+ " super().__init__(init, goals, actions)\n",
+ " self.jobs = jobs\n",
+ " self.resources = resources or {}\n",
+ "\n",
+ " def act(self, action):\n",
+ " """\n",
+ " Performs the HLA given as argument.\n",
+ "\n",
+ " Note that this is different from the superclass action - where the parameter was an\n",
+ " Expression. For real world problems, an Expr object isn't enough to capture all the\n",
+ " detail required for executing the action - resources, preconditions, etc need to be\n",
+ " checked for too.\n",
+ " """\n",
+ " args = action.args\n",
+ " list_action = first(a for a in self.actions if a.name == action.name)\n",
+ " if list_action is None:\n",
+ " raise Exception("Action '{}' not found".format(action.name))\n",
+ " self.init = list_action.do_action(self.jobs, self.resources, self.init, args).clauses\n",
+ "\n",
+ " def refinements(hla, state, library): # refinements may be (multiple) HLA themselves ...\n",
+ " """\n",
+ " state is a Problem, containing the current state kb\n",
+ " library is a dictionary containing details for every possible refinement. eg:\n",
+ " {\n",
+ " 'HLA': [\n",
+ " 'Go(Home, SFO)',\n",
+ " 'Go(Home, SFO)',\n",
+ " 'Drive(Home, SFOLongTermParking)',\n",
+ " 'Shuttle(SFOLongTermParking, SFO)',\n",
+ " 'Taxi(Home, SFO)'\n",
+ " ],\n",
+ " 'steps': [\n",
+ " ['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'],\n",
+ " ['Taxi(Home, SFO)'],\n",
+ " [],\n",
+ " [],\n",
+ " []\n",
+ " ],\n",
+ " # empty refinements indicate a primitive action\n",
+ " 'precond': [\n",
+ " ['At(Home) & Have(Car)'],\n",
+ " ['At(Home)'],\n",
+ " ['At(Home) & Have(Car)'],\n",
+ " ['At(SFOLongTermParking)'],\n",
+ " ['At(Home)']\n",
+ " ],\n",
+ " 'effect': [\n",
+ " ['At(SFO) & ~At(Home)'],\n",
+ " ['At(SFO) & ~At(Home)'],\n",
+ " ['At(SFOLongTermParking) & ~At(Home)'],\n",
+ " ['At(SFO) & ~At(SFOLongTermParking)'],\n",
+ " ['At(SFO) & ~At(Home)']\n",
+ " ]\n",
+ " }\n",
+ " """\n",
+ " e = Expr(hla.name, hla.args)\n",
+ " indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+ " for i in indices:\n",
+ " actions = []\n",
+ " for j in range(len(library['steps'][i])):\n",
+ " # find the index of the step [j] of the HLA \n",
+ " index_step = [k for k,x in enumerate(library['HLA']) if x == library['steps'][i][j]][0]\n",
+ " precond = library['precond'][index_step][0] # preconditions of step [j]\n",
+ " effect = library['effect'][index_step][0] # effect of step [j]\n",
+ " actions.append(HLA(library['steps'][i][j], precond, effect))\n",
+ " yield actions\n",
+ "\n",
+ " def hierarchical_search(problem, hierarchy):\n",
+ " """\n",
+ " [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical\n",
+ " Forward Planning Search'\n",
+ " The problem is a real-world problem defined by the problem class, and the hierarchy is\n",
+ " a dictionary of HLA - refinements (see refinements generator for details)\n",
+ " """\n",
+ " act = Node(problem.actions[0])\n",
+ " frontier = deque()\n",
+ " frontier.append(act)\n",
+ " while True:\n",
+ " if not frontier:\n",
+ " return None\n",
+ " plan = frontier.popleft()\n",
+ " print(plan.state.name)\n",
+ " hla = plan.state # first_or_null(plan)\n",
+ " prefix = None\n",
+ " if plan.parent:\n",
+ " prefix = plan.parent.state.action # prefix, suffix = subseq(plan.state, hla)\n",
+ " outcome = Problem.result(problem, prefix)\n",
+ " if hla is None:\n",
+ " if outcome.goal_test():\n",
+ " return plan.path()\n",
+ " else:\n",
+ " print("else")\n",
+ " for sequence in Problem.refinements(hla, outcome, hierarchy):\n",
+ " print("...")\n",
+ " frontier.append(Node(plan.state, plan.parent, sequence))\n",
+ "\n",
+ " def result(state, actions):\n",
+ " """The outcome of applying an action to the current problem"""\n",
+ " for a in actions: \n",
+ " if a.check_precond(state, a.args):\n",
+ " state = a(state, a.args).clauses\n",
+ " return state\n",
+ " \n",
+ "\n",
+ " def angelic_search(problem, hierarchy, initialPlan):\n",
+ " """\n",
+ "\t[Figure 11.8] A hierarchical planning algorithm that uses angelic semantics to identify and\n",
+ "\tcommit to high-level plans that work while avoiding high-level plans that don’t. \n",
+ "\tThe predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression\n",
+ "\tof refinements. \n",
+ "\tAt top level, call ANGELIC -SEARCH with [Act ] as the initialPlan .\n",
+ "\n",
+ " initialPlan contains a sequence of HLA's with angelic semantics \n",
+ "\n",
+ " The possible effects of an angelic HLA in initialPlan are : \n",
+ " ~ : effect remove\n",
+ " $+: effect possibly add\n",
+ " $-: effect possibly remove\n",
+ " $$: possibly add or remove\n",
+ "\t"""\n",
+ " frontier = deque(initialPlan)\n",
+ " while True: \n",
+ " if not frontier:\n",
+ " return None\n",
+ " plan = frontier.popleft() # sequence of HLA/Angelic HLA's \n",
+ " opt_reachable_set = Problem.reach_opt(problem.init, plan)\n",
+ " pes_reachable_set = Problem.reach_pes(problem.init, plan)\n",
+ " if problem.intersects_goal(opt_reachable_set): \n",
+ " if Problem.is_primitive( plan, hierarchy ): \n",
+ " return ([x for x in plan.action])\n",
+ " guaranteed = problem.intersects_goal(pes_reachable_set) \n",
+ " if guaranteed and Problem.making_progress(plan, plan):\n",
+ " final_state = guaranteed[0] # any element of guaranteed \n",
+ " #print('decompose')\n",
+ " return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set)\n",
+ " (hla, index) = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive.\n",
+ " prefix = plan.action[:index-1]\n",
+ " suffix = plan.action[index+1:]\n",
+ " outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+ " for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+ " frontier.append(Angelic_Node(outcome.init, plan, prefix + sequence+ suffix, prefix+sequence+suffix))\n",
+ "\n",
+ "\n",
+ " def intersects_goal(problem, reachable_set):\n",
+ " """\n",
+ " Find the intersection of the reachable states and the goal\n",
+ " """\n",
+ " return [y for x in list(reachable_set.keys()) for y in reachable_set[x] if all(goal in y for goal in problem.goals)] \n",
+ "\n",
+ "\n",
+ " def is_primitive(plan, library):\n",
+ " """\n",
+ " checks if the hla is primitive action \n",
+ " """\n",
+ " for hla in plan.action: \n",
+ " indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+ " for i in indices:\n",
+ " if library["steps"][i]: \n",
+ " return False\n",
+ " return True\n",
+ " \n",
+ "\n",
+ "\n",
+ " def reach_opt(init, plan): \n",
+ " """\n",
+ " Finds the optimistic reachable set of the sequence of actions in plan \n",
+ " """\n",
+ " reachable_set = {0: [init]}\n",
+ " optimistic_description = plan.action #list of angelic actions with optimistic description\n",
+ " return Problem.find_reachable_set(reachable_set, optimistic_description)\n",
+ " \n",
+ "\n",
+ " def reach_pes(init, plan): \n",
+ " """ \n",
+ " Finds the pessimistic reachable set of the sequence of actions in plan\n",
+ " """\n",
+ " reachable_set = {0: [init]}\n",
+ " pessimistic_description = plan.action_pes # list of angelic actions with pessimistic description\n",
+ " return Problem.find_reachable_set(reachable_set, pessimistic_description)\n",
+ "\n",
+ " def find_reachable_set(reachable_set, action_description):\n",
+ " """\n",
+ "\tFinds the reachable states of the action_description when applied in each state of reachable set.\n",
+ "\t"""\n",
+ " for i in range(len(action_description)):\n",
+ " reachable_set[i+1]=[]\n",
+ " if type(action_description[i]) is Angelic_HLA:\n",
+ " possible_actions = action_description[i].angelic_action()\n",
+ " else: \n",
+ " possible_actions = action_description\n",
+ " for action in possible_actions:\n",
+ " for state in reachable_set[i]:\n",
+ " if action.check_precond(state , action.args) :\n",
+ " if action.effect[0] :\n",
+ " new_state = action(state, action.args).clauses\n",
+ " reachable_set[i+1].append(new_state)\n",
+ " else: \n",
+ " reachable_set[i+1].append(state)\n",
+ " return reachable_set\n",
+ "\n",
+ " def find_hla(plan, hierarchy):\n",
+ " """\n",
+ " Finds the the first HLA action in plan.action, which is not primitive\n",
+ " and its corresponding index in plan.action\n",
+ " """\n",
+ " hla = None\n",
+ " index = len(plan.action)\n",
+ " for i in range(len(plan.action)): # find the first HLA in plan, that is not primitive\n",
+ " if not Problem.is_primitive(Node(plan.state, plan.parent, [plan.action[i]]), hierarchy):\n",
+ " hla = plan.action[i] \n",
+ " index = i\n",
+ " break\n",
+ " return (hla, index)\n",
+ "\t\n",
+ " def making_progress(plan, initialPlan):\n",
+ " """ \n",
+ " Not correct\n",
+ "\n",
+ " Normally should from infinite regression of refinements \n",
+ " \n",
+ " Only case covered: when plan contains one action (then there is no regression to be done) \n",
+ " """\n",
+ " if (len(plan.action)==1):\n",
+ " return False\n",
+ " return True \n",
+ "\n",
+ " def decompose(hierarchy, s_0, plan, s_f, reachable_set):\n",
+ " solution = [] \n",
+ " while plan.action_pes: \n",
+ " action = plan.action_pes.pop()\n",
+ " i = max(reachable_set.keys())\n",
+ " if (i==0): \n",
+ " return solution\n",
+ " s_i = Problem.find_previous_state(s_f, reachable_set,i, action) \n",
+ " problem = Problem(s_i, s_f , plan.action)\n",
+ " j=0\n",
+ " for x in Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])]):\n",
+ " solution.insert(j,x)\n",
+ " j+=1\n",
+ " s_f = s_i\n",
+ " return solution\n",
+ "\n",
+ "\n",
+ " def find_previous_state(s_f, reachable_set, i, action):\n",
+ " """\n",
+ " Given a final state s_f and an action finds a state s_i in reachable_set \n",
+ " such that when action is applied to state s_i returns s_f. \n",
+ " """\n",
+ " s_i = reachable_set[i-1][0]\n",
+ " for state in reachable_set[i-1]:\n",
+ " if s_f in [x for x in Problem.reach_pes(state, Angelic_Node(state, None, [action],[action]))[1]]:\n",
+ " s_i =state\n",
+ " break\n",
+ " return s_i\n",
+ "class HLA(Action):\n",
+ " """\n",
+ " Define Actions for the real-world (that may be refined further), and satisfy resource\n",
+ " constraints.\n",
+ " """\n",
+ " unique_group = 1\n",
+ "\n",
+ " def __init__(self, action, precond=None, effect=None, duration=0,\n",
+ " consume=None, use=None):\n",
+ " """\n",
+ " As opposed to actions, to define HLA, we have added constraints.\n",
+ " duration holds the amount of time required to execute the task\n",
+ " consumes holds a dictionary representing the resources the task consumes\n",
+ " uses holds a dictionary representing the resources the task uses\n",
+ " """\n",
+ " precond = precond or [None]\n",
+ " effect = effect or [None]\n",
+ " super().__init__(action, precond, effect)\n",
+ " self.duration = duration\n",
+ " self.consumes = consume or {}\n",
+ " self.uses = use or {}\n",
+ " self.completed = False\n",
+ " # self.priority = -1 # must be assigned in relation to other HLAs\n",
+ " # self.job_group = -1 # must be assigned in relation to other HLAs\n",
+ "\n",
+ " def do_action(self, job_order, available_resources, kb, args):\n",
+ " """\n",
+ " An HLA based version of act - along with knowledge base updation, it handles\n",
+ " resource checks, and ensures the actions are executed in the correct order.\n",
+ " """\n",
+ " # print(self.name)\n",
+ " if not self.has_usable_resource(available_resources):\n",
+ " raise Exception('Not enough usable resources to execute {}'.format(self.name))\n",
+ " if not self.has_consumable_resource(available_resources):\n",
+ " raise Exception('Not enough consumable resources to execute {}'.format(self.name))\n",
+ " if not self.inorder(job_order):\n",
+ " raise Exception("Can't execute {} - execute prerequisite actions first".\n",
+ " format(self.name))\n",
+ " kb = super().act(kb, args) # update knowledge base\n",
+ " for resource in self.consumes: # remove consumed resources\n",
+ " available_resources[resource] -= self.consumes[resource]\n",
+ " self.completed = True # set the task status to complete\n",
+ " return kb\n",
+ "\n",
+ " def has_consumable_resource(self, available_resources):\n",
+ " """\n",
+ " Ensure there are enough consumable resources for this action to execute.\n",
+ " """\n",
+ " for resource in self.consumes:\n",
+ " if available_resources.get(resource) is None:\n",
+ " return False\n",
+ " if available_resources[resource] < self.consumes[resource]:\n",
+ " return False\n",
+ " return True\n",
+ "\n",
+ " def has_usable_resource(self, available_resources):\n",
+ " """\n",
+ " Ensure there are enough usable resources for this action to execute.\n",
+ " """\n",
+ " for resource in self.uses:\n",
+ " if available_resources.get(resource) is None:\n",
+ " return False\n",
+ " if available_resources[resource] < self.uses[resource]:\n",
+ " return False\n",
+ " return True\n",
+ "\n",
+ " def inorder(self, job_order):\n",
+ " """\n",
+ " Ensure that all the jobs that had to be executed before the current one have been\n",
+ " successfully executed.\n",
+ " """\n",
+ " for jobs in job_order:\n",
+ " if self in jobs:\n",
+ " for job in jobs:\n",
+ " if job is self:\n",
+ " return True\n",
+ " if not job.completed:\n",
+ " return False\n",
+ " return True\n",
+ "\n", + "Let's look at how the `job_shop_problem` has been defined on the module." + ] + }, + { + "cell_type": "code", + "execution_count": 138, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def job_shop_problem():\n",
+ " """\n",
+ " [Figure 11.1] JOB-SHOP-PROBLEM\n",
+ "\n",
+ " A job-shop scheduling problem for assembling two cars,\n",
+ " with resource and ordering constraints.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> p = job_shop_problem()\n",
+ " >>> p.goal_test()\n",
+ " False\n",
+ " >>> p.act(p.jobs[1][0])\n",
+ " >>> p.act(p.jobs[1][1])\n",
+ " >>> p.act(p.jobs[1][2])\n",
+ " >>> p.act(p.jobs[0][0])\n",
+ " >>> p.act(p.jobs[0][1])\n",
+ " >>> p.goal_test()\n",
+ " False\n",
+ " >>> p.act(p.jobs[0][2])\n",
+ " >>> p.goal_test()\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ " resources = {'EngineHoists': 1, 'WheelStations': 2, 'Inspectors': 2, 'LugNuts': 500}\n",
+ "\n",
+ " add_engine1 = HLA('AddEngine1', precond='~Has(C1, E1)', effect='Has(C1, E1)', duration=30, use={'EngineHoists': 1})\n",
+ " add_engine2 = HLA('AddEngine2', precond='~Has(C2, E2)', effect='Has(C2, E2)', duration=60, use={'EngineHoists': 1})\n",
+ " add_wheels1 = HLA('AddWheels1', precond='~Has(C1, W1)', effect='Has(C1, W1)', duration=30, use={'WheelStations': 1}, consume={'LugNuts': 20})\n",
+ " add_wheels2 = HLA('AddWheels2', precond='~Has(C2, W2)', effect='Has(C2, W2)', duration=15, use={'WheelStations': 1}, consume={'LugNuts': 20})\n",
+ " inspect1 = HLA('Inspect1', precond='~Inspected(C1)', effect='Inspected(C1)', duration=10, use={'Inspectors': 1})\n",
+ " inspect2 = HLA('Inspect2', precond='~Inspected(C2)', effect='Inspected(C2)', duration=10, use={'Inspectors': 1})\n",
+ "\n",
+ " actions = [add_engine1, add_engine2, add_wheels1, add_wheels2, inspect1, inspect2]\n",
+ "\n",
+ " job_group1 = [add_engine1, add_wheels1, inspect1]\n",
+ " job_group2 = [add_engine2, add_wheels2, inspect2]\n",
+ "\n",
+ " return Problem(init='Car(C1) & Car(C2) & Wheels(W1) & Wheels(W2) & Engine(E2) & Engine(E2) & ~Has(C1, E1) & ~Has(C2, E2) & ~Has(C1, W1) & ~Has(C2, W2) & ~Inspected(C1) & ~Inspected(C2)',\n",
+ " goals='Has(C1, W1) & Has(C1, E1) & Inspected(C1) & Has(C2, W2) & Has(C2, E2) & Inspected(C2)',\n",
+ " actions=actions,\n",
+ " jobs=[job_group1, job_group2],\n",
+ " resources=resources)\n",
+ "\n", + "
\n", + "**Has(x, y)**: Car **'x'** _has_ **'y'** where **'y'** can be an Engine or a Wheel.\n", + "\n", + "**~Has(x, y)**: Car **'x'** does _not have_ **'y'** where **'y'** can be an Engine or a Wheel.\n", + "\n", + "**Inspected(c)**: Car **'c'** has been _inspected_.\n", + "\n", + "**~Inspected(c)**: Car **'c'** has _not_ been inspected.\n", + "\n", + "In the initial state, `C1` and `C2` are cars and neither have an engine or wheels and haven't been inspected.\n", + "`E1` and `E2` are engines.\n", + "`W1` and `W2` are wheels.\n", + "
\n", + "Our goal is to have engines and wheels on both cars and to get them inspected. We will discuss how to achieve this.\n", + "
\n", + "Let's define an object of the `job_shop_problem`." + ] + }, + { + "cell_type": "code", + "execution_count": 139, + "metadata": {}, + "outputs": [], + "source": [ + "jobShopProblem = job_shop_problem()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before taking any actions, we will check if `jobShopProblem` has reached its goal." + ] + }, + { + "cell_type": "code", + "execution_count": 140, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "False\n" + ] + } + ], + "source": [ + "print(jobShopProblem.goal_test())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now define a possible solution that can help us reach the goal. \n", + "The actions are then carried out on the `jobShopProblem` object." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following actions are available to us:\n", + "\n", + "**AddEngine1**: Adds an engine to the car C1. Takes 30 minutes to complete and uses an engine hoist.\n", + " \n", + "**AddEngine2**: Adds an engine to the car C2. Takes 60 minutes to complete and uses an engine hoist.\n", + "\n", + "**AddWheels1**: Adds wheels to car C1. Takes 30 minutes to complete. Uses a wheel station and consumes 20 lug nuts.\n", + "\n", + "**AddWheels2**: Adds wheels to car C2. Takes 15 minutes to complete. Uses a wheel station and consumes 20 lug nuts as well.\n", + "\n", + "**Inspect1**: Gets car C1 inspected. Requires 10 minutes of inspection by one inspector.\n", + "\n", + "**Inspect2**: Gets car C2 inspected. Requires 10 minutes of inspection by one inspector." + ] + }, + { + "cell_type": "code", + "execution_count": 141, + "metadata": {}, + "outputs": [], + "source": [ + "solution = [jobShopProblem.jobs[1][0],\n", + " jobShopProblem.jobs[1][1],\n", + " jobShopProblem.jobs[1][2],\n", + " jobShopProblem.jobs[0][0],\n", + " jobShopProblem.jobs[0][1],\n", + " jobShopProblem.jobs[0][2]]\n", + "\n", + "for action in solution:\n", + " jobShopProblem.act(action)" + ] + }, + { + "cell_type": "code", + "execution_count": 142, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n" + ] + } + ], + "source": [ + "print(jobShopProblem.goal_test())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is a valid solution and one of many correct ways to solve this problem." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Double tennis problem\n", + "This problem is a simple case of a multiactor planning problem, where two agents act at once and can simultaneously change the current state of the problem. \n", + "A correct plan is one that, if executed by the actors, achieves the goal.\n", + "In the true multiagent setting, of course, the agents may not agree to execute any particular plan, but atleast they will know what plans _would_ work if they _did_ agree to execute them.\n", + "
\n", + "In the double tennis problem, two actors A and B are playing together and can be in one of four locations: `LeftBaseLine`, `RightBaseLine`, `LeftNet` and `RightNet`.\n", + "The ball can be returned only if a player is in the right place.\n", + "Each action must include the actor as an argument.\n", + "
\n", + "Let's first look at the definition of the `double_tennis_problem` in the module." + ] + }, + { + "cell_type": "code", + "execution_count": 172, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def double_tennis_problem():\n",
+ " """\n",
+ " [Figure 11.10] DOUBLE-TENNIS-PROBLEM\n",
+ "\n",
+ " A multiagent planning problem involving two partner tennis players\n",
+ " trying to return an approaching ball and repositioning around in the court.\n",
+ "\n",
+ " Example:\n",
+ " >>> from planning import *\n",
+ " >>> dtp = double_tennis_problem()\n",
+ " >>> goal_test(dtp.goals, dtp.init)\n",
+ " False\n",
+ " >>> dtp.act(expr('Go(A, RightBaseLine, LeftBaseLine)'))\n",
+ " >>> dtp.act(expr('Hit(A, Ball, RightBaseLine)'))\n",
+ " >>> goal_test(dtp.goals, dtp.init)\n",
+ " False\n",
+ " >>> dtp.act(expr('Go(A, LeftNet, RightBaseLine)'))\n",
+ " >>> goal_test(dtp.goals, dtp.init)\n",
+ " True\n",
+ " >>>\n",
+ " """\n",
+ "\n",
+ " return PlanningProblem(init='At(A, LeftBaseLine) & At(B, RightNet) & Approaching(Ball, RightBaseLine) & Partner(A, B) & Partner(B, A)',\n",
+ " goals='Returned(Ball) & At(a, LeftNet) & At(a, RightNet)',\n",
+ " actions=[Action('Hit(actor, Ball, loc)',\n",
+ " precond='Approaching(Ball, loc) & At(actor, loc)',\n",
+ " effect='Returned(Ball)'),\n",
+ " Action('Go(actor, to, loc)', \n",
+ " precond='At(actor, loc)',\n",
+ " effect='At(actor, to) & ~At(actor, loc)')])\n",
+ "\n", + "The algorithms input is: problem, hierarchy and initialPlan\n", + "- problem is of type Problem \n", + "- hierarchy is a dictionary consisting of all the actions. \n", + "- initialPlan is an approximate description(optimistic and pessimistic) of the agents choices for the implementation.
\n", + " initialPlan contains a sequence of HLA's with angelic semantics" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import * \n", + "from notebook import psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Angelic search algorithm consists of three parts. \n", + "- Search using angelic semantics\n", + "- Decompose\n", + "- a search in the space of refinements, in a similar way with hierarchical search\n", + "\n", + "### Searching using angelic semantics\n", + "- Find the reachable set (optimistic and pessimistic) of the sequence of angelic HLA in initialPlan\n", + " - If the optimistic reachable set doesn't intersect the goal, then there is no solution\n", + " - If the pessimistic reachable set intersects the goal, then we call decompose, in order to find the sequence of actions that lead us to the goal. \n", + " - If the optimistic reachable set intersects the goal, but the pessimistic doesn't we do some further refinements, in order to see if there is a sequence of actions that achieves the goal. \n", + " \n", + "### Search in space of refinements\n", + "- Create a search tree, that has root the action and children it's refinements\n", + "- Extend frontier by adding each refinement, so that we keep looping till we find all primitive actions\n", + "- If we achieve that we return the path of the solution (search tree), else there is no solution and we return None.\n", + "\n", + " \n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def angelic_search(problem, hierarchy, initialPlan):\n",
+ " """\n",
+ "\t[Figure 11.8] A hierarchical planning algorithm that uses angelic semantics to identify and\n",
+ "\tcommit to high-level plans that work while avoiding high-level plans that don’t. \n",
+ "\tThe predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression\n",
+ "\tof refinements. \n",
+ "\tAt top level, call ANGELIC -SEARCH with [Act ] as the initialPlan .\n",
+ "\n",
+ " initialPlan contains a sequence of HLA's with angelic semantics \n",
+ "\n",
+ " The possible effects of an angelic HLA in initialPlan are : \n",
+ " ~ : effect remove\n",
+ " $+: effect possibly add\n",
+ " $-: effect possibly remove\n",
+ " $$: possibly add or remove\n",
+ "\t"""\n",
+ " frontier = deque(initialPlan)\n",
+ " while True: \n",
+ " if not frontier:\n",
+ " return None\n",
+ " plan = frontier.popleft() # sequence of HLA/Angelic HLA's \n",
+ " opt_reachable_set = Problem.reach_opt(problem.init, plan)\n",
+ " pes_reachable_set = Problem.reach_pes(problem.init, plan)\n",
+ " if problem.intersects_goal(opt_reachable_set): \n",
+ " if Problem.is_primitive( plan, hierarchy ): \n",
+ " return ([x for x in plan.action])\n",
+ " guaranteed = problem.intersects_goal(pes_reachable_set) \n",
+ " if guaranteed and Problem.making_progress(plan, initialPlan):\n",
+ " final_state = guaranteed[0] # any element of guaranteed \n",
+ " #print('decompose')\n",
+ " return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set)\n",
+ " (hla, index) = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive.\n",
+ " prefix = plan.action[:index]\n",
+ " suffix = plan.action[index+1:]\n",
+ " outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+ " for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+ " frontier.append(Angelic_Node(outcome.init, plan, prefix + sequence+ suffix, prefix+sequence+suffix))\n",
+ " def decompose(hierarchy, s_0, plan, s_f, reachable_set):\n",
+ " solution = [] \n",
+ " i = max(reachable_set.keys())\n",
+ " while plan.action_pes: \n",
+ " action = plan.action_pes.pop()\n",
+ " if (i==0): \n",
+ " return solution\n",
+ " s_i = Problem.find_previous_state(s_f, reachable_set,i, action) \n",
+ " problem = Problem(s_i, s_f , plan.action)\n",
+ " angelic_call = Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])])\n",
+ " if angelic_call:\n",
+ " for x in angelic_call: \n",
+ " solution.insert(0,x)\n",
+ " else: \n",
+ " return None\n",
+ " s_f = s_i\n",
+ " i-=1\n",
+ " return solution\n",
+ "\n", + "Those two actions have some preconditions and some effects. \n", + "If you get the taxi, you need to have cash, whereas if you drive you need to have a car.
\n", + "Thus we define the following hierarchy of possible actions.\n", + "\n", + "##### hierarchy" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "library = {\n", + " 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', 'Taxi(Home, SFO)'],\n", + " 'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []],\n", + " 'precond': [['At(Home) & Have(Car)'], ['At(Home)'], ['At(Home) & Have(Car)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n", + " 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(LongTermParking)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']] }\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "the possible actions are the following:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "go_SFO = HLA('Go(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)')\n", + "taxi_SFO = HLA('Taxi(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home) & ~Have(Cash)')\n", + "drive_SFOLongTermParking = HLA('Drive(Home, SFOLongTermParking)', 'At(Home) & Have(Car)','At(SFOLongTermParking) & ~At(Home)' )\n", + "shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'At(SFOLongTermParking)', 'At(SFO) & ~At(LongTermParking)')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Suppose that (our preconditionds are that) we are Home and we have cash and car and our goal is to get to SFO and maintain our cash, and our possible actions are the above.
\n", + "##### Then our problem is: " + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "prob = Problem('At(Home) & Have(Cash) & Have(Car)', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An agent gives us some approximate information about the plan we will follow:
\n", + "(initialPlan is an Angelic Node, where: \n", + "- state is the initial state of the problem, \n", + "- parent is None \n", + "- action: is a list of actions (Angelic HLA's) with the optimistic estimators of effects and \n", + "- action_pes: is a list of actions (Angelic HLA's) with the pessimistic approximations of the effects\n", + "##### InitialPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "angelic_opt_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & $-At(Home)' ) \n", + "angelic_pes_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & ~At(Home)' )\n", + "\n", + "initialPlan = [Angelic_Node(prob.init, None, [angelic_opt_description], [angelic_pes_description])] \n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We want to find the optimistic and pessimistic reachable set of initialPlan when applied to the problem:\n", + "##### Optimistic/Pessimistic reachable set" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[At(Home), Have(Cash), Have(Car)], [Have(Cash), Have(Car), At(SFO), NotAt(Home)], [Have(Cash), Have(Car), NotAt(Home)], [At(Home), Have(Cash), Have(Car), At(SFO)], [At(Home), Have(Cash), Have(Car)]] \n", + "\n", + "[[At(Home), Have(Cash), Have(Car)], [Have(Cash), Have(Car), At(SFO), NotAt(Home)], [Have(Cash), Have(Car), NotAt(Home)]]\n" + ] + } + ], + "source": [ + "opt_reachable_set = Problem.reach_opt(prob.init, initialPlan[0])\n", + "pes_reachable_set = Problem.reach_pes(prob.init, initialPlan[0])\n", + "print([x for y in opt_reachable_set.keys() for x in opt_reachable_set[y]], '\\n')\n", + "print([x for y in pes_reachable_set.keys() for x in pes_reachable_set[y]])\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### Refinements" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))]\n", + "[{'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'args': (Home, SFOLongTermParking), 'uses': {}, 'consumes': {}, 'name': 'Drive', 'completed': False, 'precond': [At(Home), Have(Car)]}, {'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'args': (SFOLongTermParking, SFO), 'uses': {}, 'consumes': {}, 'name': 'Shuttle', 'completed': False, 'precond': [At(SFOLongTermParking)]}] \n", + "\n", + "[HLA(Taxi(Home, SFO))]\n", + "[{'duration': 0, 'effect': [At(SFO), NotAt(Home), NotHave(Cash)], 'args': (Home, SFO), 'uses': {}, 'consumes': {}, 'name': 'Taxi', 'completed': False, 'precond': [At(Home)]}] \n", + "\n" + ] + } + ], + "source": [ + "for sequence in Problem.refinements(go_SFO, prob, library):\n", + " print (sequence)\n", + " print([x.__dict__ for x in sequence ], '\\n')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Run the angelic search\n", + "##### Top level call" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))] \n", + "\n", + "[{'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'args': (Home, SFOLongTermParking), 'uses': {}, 'consumes': {}, 'name': 'Drive', 'completed': False, 'precond': [At(Home), Have(Car)]}, {'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'args': (SFOLongTermParking, SFO), 'uses': {}, 'consumes': {}, 'name': 'Shuttle', 'completed': False, 'precond': [At(SFOLongTermParking)]}]\n" + ] + } + ], + "source": [ + "plan= Problem.angelic_search(prob, library, initialPlan)\n", + "print (plan, '\\n')\n", + "print ([x.__dict__ for x in plan])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 2" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "library_2 = {\n", + " 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)' , 'Metro(MetroStop, SFO)', 'Metro1(MetroStop, SFO)', 'Metro2(MetroStop, SFO)' ,'Taxi(Home, SFO)'],\n", + " 'steps': [['Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)'], ['Taxi(Home, SFO)'], [], ['Metro1(MetroStop, SFO)'], ['Metro2(MetroStop, SFO)'],[],[],[]],\n", + " 'precond': [['At(Home)'], ['At(Home)'], ['At(Home)'], ['At(MetroStop)'], ['At(MetroStop)'],['At(MetroStop)'], ['At(MetroStop)'] ,['At(Home) & Have(Cash)']],\n", + " 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(MetroStop) & ~At(Home)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'] , ['At(SFO) & ~At(MetroStop)'] ,['At(SFO) & ~At(Home) & ~Have(Cash)']] \n", + " }" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Bus(Home, MetroStop)), HLA(Metro1(MetroStop, SFO))] \n", + "\n", + "[{'duration': 0, 'effect': [At(MetroStop), NotAt(Home)], 'args': (Home, MetroStop), 'uses': {}, 'consumes': {}, 'name': 'Bus', 'completed': False, 'precond': [At(Home)]}, {'duration': 0, 'effect': [At(SFO), NotAt(MetroStop)], 'args': (MetroStop, SFO), 'uses': {}, 'consumes': {}, 'name': 'Metro1', 'completed': False, 'precond': [At(MetroStop)]}]\n" + ] + } + ], + "source": [ + "plan_2 = Problem.angelic_search(prob, library_2, initialPlan)\n", + "print(plan_2, '\\n')\n", + "print([x.__dict__ for x in plan_2])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 3 \n", + "\n", + "Sometimes there is no plan that achieves the goal!" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "library_3 = {\n", + " 'HLA': ['Shuttle(SFOLongTermParking, SFO)', 'Go(Home, SFOLongTermParking)', 'Taxi(Home, SFOLongTermParking)', 'Drive(Home, SFOLongTermParking)', 'Drive(SFOLongTermParking, Home)', 'Get(Cash)', 'Go(Home, ATM)'],\n", + " 'steps': [['Get(Cash)', 'Go(Home, SFOLongTermParking)'], ['Taxi(Home, SFOLongTermParking)'], [], [], [], ['Drive(SFOLongTermParking, Home)', 'Go(Home, ATM)'], []],\n", + " 'precond': [['At(SFOLongTermParking)'], ['At(Home)'], ['At(Home) & Have(Cash)'], ['At(Home)'], ['At(SFOLongTermParking)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n", + " 'effect': [['At(SFO)'], ['At(SFO)'], ['At(SFOLongTermParking) & ~Have(Cash)'], ['At(SFOLongTermParking)'] ,['At(Home) & ~At(SFOLongTermParking)'], ['At(Home) & Have(Cash)'], ['Have(Cash)'] ]\n", + " }\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'Have(Cash) & At(SFOLongTermParking)', 'At(SFO)')\n", + "prob_3 = Problem('At(SFOLongTermParking) & Have(Cash)', 'At(SFO) & Have(Cash)', [shuttle_SFO])\n", + "# optimistic/pessimistic descriptions\n", + "angelic_opt_description = Angelic_HLA('Shuttle(SFOLongTermParking, SFO)', precond = 'At(SFOLongTermParking)', effect ='$+At(SFO) & $-At(SFOLongTermParking)' ) \n", + "angelic_pes_description = Angelic_HLA('Shuttle(SFOLongTermParking, SFO)', precond = 'At(SFOLongTermParking)', effect ='$+At(SFO) & ~At(SFOLongTermParking)' ) \n", + "# initial Plan\n", + "initialPlan_3 = [Angelic_Node(prob.init, None, [angelic_opt_description], [angelic_pes_description])] " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "None\n" + ] + } + ], + "source": [ + "plan_3 = prob_3.angelic_search(library_3, initialPlan_3)\n", + "print(plan_3)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_graphPlan.ipynb b/planning_graphPlan.ipynb new file mode 100644 index 000000000..bffecb937 --- /dev/null +++ b/planning_graphPlan.ipynb @@ -0,0 +1,1066 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## SOLVING PLANNING PROBLEMS\n", + "----\n", + "### GRAPHPLAN\n", + "
\n", + "The GraphPlan algorithm is a popular method of solving classical planning problems.\n", + "Before we get into the details of the algorithm, let's look at a special data structure called **planning graph**, used to give better heuristic estimates and plays a key role in the GraphPlan algorithm." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Planning Graph\n", + "A planning graph is a directed graph organized into levels. \n", + "Each level contains information about the current state of the knowledge base and the possible state-action links to and from that level.\n", + "The first level contains the initial state with nodes representing each fluent that holds in that level.\n", + "This level has state-action links linking each state to valid actions in that state.\n", + "Each action is linked to all its preconditions and its effect states.\n", + "Based on these effects, the next level is constructed.\n", + "The next level contains similarly structured information about the next state.\n", + "In this way, the graph is expanded using state-action links till we reach a state where all the required goals hold true simultaneously.\n", + "We can say that we have reached our goal if none of the goal states in the current level are mutually exclusive.\n", + "This will be explained in detail later.\n", + "
\n", + "Planning graphs only work for propositional planning problems, hence we need to eliminate all variables by generating all possible substitutions.\n", + "
\n", + "For example, the planning graph of the `have_cake_and_eat_cake_too` problem might look like this\n", + "\n", + "
\n", + "The black lines indicate links between states and actions.\n", + "
\n", + "In every planning problem, we are allowed to carry out the `no-op` action, ie, we can choose no action for a particular state.\n", + "These are called 'Persistence' actions and are represented in the graph by the small square boxes.\n", + "In technical terms, a persistence action has effects same as its preconditions.\n", + "This enables us to carry a state to the next level.\n", + "
\n", + "
\n", + "The gray lines indicate mutual exclusivity.\n", + "This means that the actions connected bya gray line cannot be taken together.\n", + "Mutual exclusivity (mutex) occurs in the following cases:\n", + "1. **Inconsistent effects**: One action negates the effect of the other. For example, _Eat(Cake)_ and the persistence of _Have(Cake)_ have inconsistent effects because they disagree on the effect _Have(Cake)_\n", + "2. **Interference**: One of the effects of an action is the negation of a precondition of the other. For example, _Eat(Cake)_ interferes with the persistence of _Have(Cake)_ by negating its precondition.\n", + "3. **Competing needs**: One of the preconditions of one action is mutually exclusive with a precondition of the other. For example, _Bake(Cake)_ and _Eat(Cake)_ are mutex because they compete on the value of the _Have(Cake)_ precondition." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the module, planning graphs have been implemented using two classes, `Level` which stores data for a particular level and `Graph` which connects multiple levels together.\n", + "Let's look at the `Level` class." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class Level:\n",
+ " """\n",
+ " Contains the state of the planning problem\n",
+ " and exhaustive list of actions which use the\n",
+ " states as pre-condition.\n",
+ " """\n",
+ "\n",
+ " def __init__(self, kb):\n",
+ " """Initializes variables to hold state and action details of a level"""\n",
+ "\n",
+ " self.kb = kb\n",
+ " # current state\n",
+ " self.current_state = kb.clauses\n",
+ " # current action to state link\n",
+ " self.current_action_links = {}\n",
+ " # current state to action link\n",
+ " self.current_state_links = {}\n",
+ " # current action to next state link\n",
+ " self.next_action_links = {}\n",
+ " # next state to current action link\n",
+ " self.next_state_links = {}\n",
+ " # mutually exclusive actions\n",
+ " self.mutex = []\n",
+ "\n",
+ " def __call__(self, actions, objects):\n",
+ " self.build(actions, objects)\n",
+ " self.find_mutex()\n",
+ "\n",
+ " def separate(self, e):\n",
+ " """Separates an iterable of elements into positive and negative parts"""\n",
+ "\n",
+ " positive = []\n",
+ " negative = []\n",
+ " for clause in e:\n",
+ " if clause.op[:3] == 'Not':\n",
+ " negative.append(clause)\n",
+ " else:\n",
+ " positive.append(clause)\n",
+ " return positive, negative\n",
+ "\n",
+ " def find_mutex(self):\n",
+ " """Finds mutually exclusive actions"""\n",
+ "\n",
+ " # Inconsistent effects\n",
+ " pos_nsl, neg_nsl = self.separate(self.next_state_links)\n",
+ "\n",
+ " for negeff in neg_nsl:\n",
+ " new_negeff = Expr(negeff.op[3:], *negeff.args)\n",
+ " for poseff in pos_nsl:\n",
+ " if new_negeff == poseff:\n",
+ " for a in self.next_state_links[poseff]:\n",
+ " for b in self.next_state_links[negeff]:\n",
+ " if {a, b} not in self.mutex:\n",
+ " self.mutex.append({a, b})\n",
+ "\n",
+ " # Interference will be calculated with the last step\n",
+ " pos_csl, neg_csl = self.separate(self.current_state_links)\n",
+ "\n",
+ " # Competing needs\n",
+ " for posprecond in pos_csl:\n",
+ " for negprecond in neg_csl:\n",
+ " new_negprecond = Expr(negprecond.op[3:], *negprecond.args)\n",
+ " if new_negprecond == posprecond:\n",
+ " for a in self.current_state_links[posprecond]:\n",
+ " for b in self.current_state_links[negprecond]:\n",
+ " if {a, b} not in self.mutex:\n",
+ " self.mutex.append({a, b})\n",
+ "\n",
+ " # Inconsistent support\n",
+ " state_mutex = []\n",
+ " for pair in self.mutex:\n",
+ " next_state_0 = self.next_action_links[list(pair)[0]]\n",
+ " if len(pair) == 2:\n",
+ " next_state_1 = self.next_action_links[list(pair)[1]]\n",
+ " else:\n",
+ " next_state_1 = self.next_action_links[list(pair)[0]]\n",
+ " if (len(next_state_0) == 1) and (len(next_state_1) == 1):\n",
+ " state_mutex.append({next_state_0[0], next_state_1[0]})\n",
+ " \n",
+ " self.mutex = self.mutex + state_mutex\n",
+ "\n",
+ " def build(self, actions, objects):\n",
+ " """Populates the lists and dictionaries containing the state action dependencies"""\n",
+ "\n",
+ " for clause in self.current_state:\n",
+ " p_expr = Expr('P' + clause.op, *clause.args)\n",
+ " self.current_action_links[p_expr] = [clause]\n",
+ " self.next_action_links[p_expr] = [clause]\n",
+ " self.current_state_links[clause] = [p_expr]\n",
+ " self.next_state_links[clause] = [p_expr]\n",
+ "\n",
+ " for a in actions:\n",
+ " num_args = len(a.args)\n",
+ " possible_args = tuple(itertools.permutations(objects, num_args))\n",
+ "\n",
+ " for arg in possible_args:\n",
+ " if a.check_precond(self.kb, arg):\n",
+ " for num, symbol in enumerate(a.args):\n",
+ " if not symbol.op.islower():\n",
+ " arg = list(arg)\n",
+ " arg[num] = symbol\n",
+ " arg = tuple(arg)\n",
+ "\n",
+ " new_action = a.substitute(Expr(a.name, *a.args), arg)\n",
+ " self.current_action_links[new_action] = []\n",
+ "\n",
+ " for clause in a.precond:\n",
+ " new_clause = a.substitute(clause, arg)\n",
+ " self.current_action_links[new_action].append(new_clause)\n",
+ " if new_clause in self.current_state_links:\n",
+ " self.current_state_links[new_clause].append(new_action)\n",
+ " else:\n",
+ " self.current_state_links[new_clause] = [new_action]\n",
+ " \n",
+ " self.next_action_links[new_action] = []\n",
+ " for clause in a.effect:\n",
+ " new_clause = a.substitute(clause, arg)\n",
+ "\n",
+ " self.next_action_links[new_action].append(new_clause)\n",
+ " if new_clause in self.next_state_links:\n",
+ " self.next_state_links[new_clause].append(new_action)\n",
+ " else:\n",
+ " self.next_state_links[new_clause] = [new_action]\n",
+ "\n",
+ " def perform_actions(self):\n",
+ " """Performs the necessary actions and returns a new Level"""\n",
+ "\n",
+ " new_kb = FolKB(list(set(self.next_state_links.keys())))\n",
+ " return Level(new_kb)\n",
+ "\n", + "
\n", + "The `find_mutex` method finds the mutex links according to the points given above.\n", + "
\n", + "The `build` method populates the data structures storing the state and action information.\n", + "Persistence actions for each clause in the current state are also defined here. \n", + "The newly created persistence action has the same name as its state, prefixed with a 'P'." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now look at the `Graph` class." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class Graph:\n",
+ " """\n",
+ " Contains levels of state and actions\n",
+ " Used in graph planning algorithm to extract a solution\n",
+ " """\n",
+ "\n",
+ " def __init__(self, planningproblem):\n",
+ " self.planningproblem = planningproblem\n",
+ " self.kb = FolKB(planningproblem.init)\n",
+ " self.levels = [Level(self.kb)]\n",
+ " self.objects = set(arg for clause in self.kb.clauses for arg in clause.args)\n",
+ "\n",
+ " def __call__(self):\n",
+ " self.expand_graph()\n",
+ "\n",
+ " def expand_graph(self):\n",
+ " """Expands the graph by a level"""\n",
+ "\n",
+ " last_level = self.levels[-1]\n",
+ " last_level(self.planningproblem.actions, self.objects)\n",
+ " self.levels.append(last_level.perform_actions())\n",
+ "\n",
+ " def non_mutex_goals(self, goals, index):\n",
+ " """Checks whether the goals are mutually exclusive"""\n",
+ "\n",
+ " goal_perm = itertools.combinations(goals, 2)\n",
+ " for g in goal_perm:\n",
+ " if set(g) in self.levels[index].mutex:\n",
+ " return False\n",
+ " return True\n",
+ "\n", + "The `expand_graph` method generates a new level of the graph.\n", + "This method is invoked when the goal conditions haven't been met in the current level or the actions that lead to it are mutually exclusive.\n", + "The `non_mutex_goals` method checks whether the goals in the current state are mutually exclusive.\n", + "
\n", + "
\n", + "Using these two classes, we can define a planning graph which can either be used to provide reliable heuristics for planning problems or used in the `GraphPlan` algorithm.\n", + "
\n", + "Let's have a look at the `GraphPlan` class." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class GraphPlan:\n",
+ " """\n",
+ " Class for formulation GraphPlan algorithm\n",
+ " Constructs a graph of state and action space\n",
+ " Returns solution for the planning problem\n",
+ " """\n",
+ "\n",
+ " def __init__(self, planningproblem):\n",
+ " self.graph = Graph(planningproblem)\n",
+ " self.nogoods = []\n",
+ " self.solution = []\n",
+ "\n",
+ " def check_leveloff(self):\n",
+ " """Checks if the graph has levelled off"""\n",
+ "\n",
+ " check = (set(self.graph.levels[-1].current_state) == set(self.graph.levels[-2].current_state))\n",
+ "\n",
+ " if check:\n",
+ " return True\n",
+ "\n",
+ " def extract_solution(self, goals, index):\n",
+ " """Extracts the solution"""\n",
+ "\n",
+ " level = self.graph.levels[index] \n",
+ " if not self.graph.non_mutex_goals(goals, index):\n",
+ " self.nogoods.append((level, goals))\n",
+ " return\n",
+ "\n",
+ " level = self.graph.levels[index - 1] \n",
+ "\n",
+ " # Create all combinations of actions that satisfy the goal \n",
+ " actions = []\n",
+ " for goal in goals:\n",
+ " actions.append(level.next_state_links[goal]) \n",
+ "\n",
+ " all_actions = list(itertools.product(*actions)) \n",
+ "\n",
+ " # Filter out non-mutex actions\n",
+ " non_mutex_actions = [] \n",
+ " for action_tuple in all_actions:\n",
+ " action_pairs = itertools.combinations(list(set(action_tuple)), 2) \n",
+ " non_mutex_actions.append(list(set(action_tuple))) \n",
+ " for pair in action_pairs: \n",
+ " if set(pair) in level.mutex:\n",
+ " non_mutex_actions.pop(-1)\n",
+ " break\n",
+ " \n",
+ "\n",
+ " # Recursion\n",
+ " for action_list in non_mutex_actions: \n",
+ " if [action_list, index] not in self.solution:\n",
+ " self.solution.append([action_list, index])\n",
+ "\n",
+ " new_goals = []\n",
+ " for act in set(action_list): \n",
+ " if act in level.current_action_links:\n",
+ " new_goals = new_goals + level.current_action_links[act]\n",
+ "\n",
+ " if abs(index) + 1 == len(self.graph.levels):\n",
+ " return\n",
+ " elif (level, new_goals) in self.nogoods:\n",
+ " return\n",
+ " else:\n",
+ " self.extract_solution(new_goals, index - 1)\n",
+ "\n",
+ " # Level-Order multiple solutions\n",
+ " solution = []\n",
+ " for item in self.solution:\n",
+ " if item[1] == -1:\n",
+ " solution.append([])\n",
+ " solution[-1].append(item[0])\n",
+ " else:\n",
+ " solution[-1].append(item[0])\n",
+ "\n",
+ " for num, item in enumerate(solution):\n",
+ " item.reverse()\n",
+ " solution[num] = item\n",
+ "\n",
+ " return solution\n",
+ "\n",
+ " def goal_test(self, kb):\n",
+ " return all(kb.ask(q) is not False for q in self.graph.planningproblem.goals)\n",
+ "\n",
+ " def execute(self):\n",
+ " """Executes the GraphPlan algorithm for the given problem"""\n",
+ "\n",
+ " while True:\n",
+ " self.graph.expand_graph()\n",
+ " if (self.goal_test(self.graph.levels[-1].kb) and self.graph.non_mutex_goals(self.graph.planningproblem.goals, -1)):\n",
+ " solution = self.extract_solution(self.graph.planningproblem.goals, -1)\n",
+ " if solution:\n",
+ " return solution\n",
+ " \n",
+ " if len(self.graph.levels) >= 2 and self.check_leveloff():\n",
+ " return None\n",
+ "\n", + "Once a goal is found, `extract_solution` is called.\n", + "This method recursively finds the path to a solution given a planning graph.\n", + "In the case where `extract_solution` fails to find a solution for a set of goals as a given level, we record the `(level, goals)` pair as a **no-good**.\n", + "Whenever `extract_solution` is called again with the same level and goals, we can find the recorded no-good and immediately return failure rather than searching again. \n", + "No-goods are also used in the termination test.\n", + "
\n", + "The `check_leveloff` method checks if the planning graph for the problem has **levelled-off**, ie, it has the same states, actions and mutex pairs as the previous level.\n", + "If the graph has already levelled off and we haven't found a solution, there is no point expanding the graph, as it won't lead to anything new.\n", + "In such a case, we can declare that the planning problem is unsolvable with the given constraints.\n", + "
\n", + "
\n", + "To summarize, the `GraphPlan` algorithm calls `expand_graph` and tests whether it has reached the goal and if the goals are non-mutex.\n", + "
\n", + "If so, `extract_solution` is invoked which recursively reconstructs the solution from the planning graph.\n", + "
\n", + "If not, then we check if our graph has levelled off and continue if it hasn't." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's solve a few planning problems that we had defined earlier." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Air cargo problem\n", + "In accordance with the summary above, we have defined a helper function to carry out `GraphPlan` on the `air_cargo` problem.\n", + "The function is pretty straightforward.\n", + "Let's have a look." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def air_cargo_graphplan():\n",
+ " """Solves the air cargo problem using GraphPlan"""\n",
+ " return GraphPlan(air_cargo()).execute()\n",
+ "\n", + "There are similar helper functions for some other planning problems.\n", + "
\n", + "Lets' try solving the spare tire problem." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "spareTireG = spare_tire_graphplan()\n", + "linearize(spareTireG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution for the cake problem" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Eat(Cake), Bake(Cake)]" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cakeProblemG = have_cake_and_eat_cake_too_graphplan()\n", + "linearize(cakeProblemG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution for the Sussman's Anomaly configuration of three blocks." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sussmanAnomalyG = three_block_tower_graphplan()\n", + "linearize(sussmanAnomalyG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution of the socks and shoes problem" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[RightSock, LeftSock, RightShoe, LeftShoe]" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "socksShoesG = socks_and_shoes_graphplan()\n", + "linearize(socksShoesG)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_hierarchical_search.ipynb b/planning_hierarchical_search.ipynb new file mode 100644 index 000000000..18e57b23b --- /dev/null +++ b/planning_hierarchical_search.ipynb @@ -0,0 +1,546 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Hierarchical Search \n", + "\n", + "Hierarchical search is a a planning algorithm in high level of abstraction.
\n", + "Instead of actions as in classical planning (chapter 10) (primitive actions) we now use high level actions (HLAs) (see planning.ipynb)
\n", + "\n", + "## Refinements\n", + "\n", + "Each __HLA__ has one or more refinements into a sequence of actions, each of which may be an HLA or a primitive action (which has no refinements by definition).
\n", + "For example:\n", + "- (a) the high level action \"Go to San Fransisco airport\" (Go(Home, SFO)), might have two possible refinements, \"Drive to San Fransisco airport\" and \"Taxi to San Fransisco airport\". \n", + "
\n", + "- (b) A recursive refinement for navigation in the vacuum world would be: to get to a\n", + "destination, take a step, and then go to the destination.\n", + "
\n", + "\n", + "
\n", + "- __implementation__: An HLA refinement that contains only primitive actions is called an implementation of the HLA\n", + "- An implementation of a high-level plan (a sequence of HLAs) is the concatenation of implementations of each HLA in the sequence\n", + "- A high-level plan __achieves the goal__ from a given state if at least one of its implementations achieves the goal from that state\n", + "
\n", + "\n", + "The refinements function input is: \n", + "- __hla__: the HLA of which we want to compute its refinements\n", + "- __state__: the knoweledge base of the current problem (Problem.init)\n", + "- __library__: the hierarchy of the actions in the planning problem\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import * \n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def refinements(hla, state, library): # refinements may be (multiple) HLA themselves ...\n",
+ " """\n",
+ " state is a Problem, containing the current state kb\n",
+ " library is a dictionary containing details for every possible refinement. eg:\n",
+ " {\n",
+ " 'HLA': [\n",
+ " 'Go(Home, SFO)',\n",
+ " 'Go(Home, SFO)',\n",
+ " 'Drive(Home, SFOLongTermParking)',\n",
+ " 'Shuttle(SFOLongTermParking, SFO)',\n",
+ " 'Taxi(Home, SFO)'\n",
+ " ],\n",
+ " 'steps': [\n",
+ " ['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'],\n",
+ " ['Taxi(Home, SFO)'],\n",
+ " [],\n",
+ " [],\n",
+ " []\n",
+ " ],\n",
+ " # empty refinements indicate a primitive action\n",
+ " 'precond': [\n",
+ " ['At(Home) & Have(Car)'],\n",
+ " ['At(Home)'],\n",
+ " ['At(Home) & Have(Car)'],\n",
+ " ['At(SFOLongTermParking)'],\n",
+ " ['At(Home)']\n",
+ " ],\n",
+ " 'effect': [\n",
+ " ['At(SFO) & ~At(Home)'],\n",
+ " ['At(SFO) & ~At(Home)'],\n",
+ " ['At(SFOLongTermParking) & ~At(Home)'],\n",
+ " ['At(SFO) & ~At(SFOLongTermParking)'],\n",
+ " ['At(SFO) & ~At(Home)']\n",
+ " ]\n",
+ " }\n",
+ " """\n",
+ " e = Expr(hla.name, hla.args)\n",
+ " indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+ " for i in indices:\n",
+ " actions = []\n",
+ " for j in range(len(library['steps'][i])):\n",
+ " # find the index of the step [j] of the HLA \n",
+ " index_step = [k for k,x in enumerate(library['HLA']) if x == library['steps'][i][j]][0]\n",
+ " precond = library['precond'][index_step][0] # preconditions of step [j]\n",
+ " effect = library['effect'][index_step][0] # effect of step [j]\n",
+ " actions.append(HLA(library['steps'][i][j], precond, effect))\n",
+ " yield actions\n",
+ "\n", + "The algorithms input is: problem and hierarchy\n", + "- __problem__: is of type Problem \n", + "- __hierarchy__: is a dictionary consisting of all the actions and the order in which they are performed. \n", + "
\n", + "\n", + "In top level call, initialPlan contains [act] (i.e. is the action to be performed) " + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
def hierarchical_search(problem, hierarchy):\n",
+ " """\n",
+ " [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical\n",
+ " Forward Planning Search'\n",
+ " The problem is a real-world problem defined by the problem class, and the hierarchy is\n",
+ " a dictionary of HLA - refinements (see refinements generator for details)\n",
+ " """\n",
+ " act = Node(problem.init, None, [problem.actions[0]])\n",
+ " frontier = deque()\n",
+ " frontier.append(act)\n",
+ " while True:\n",
+ " if not frontier:\n",
+ " return None\n",
+ " plan = frontier.popleft()\n",
+ " (hla, index) = Problem.find_hla(plan, hierarchy) # finds the first non primitive hla in plan actions\n",
+ " prefix = plan.action[:index]\n",
+ " outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+ " suffix = plan.action[index+1:]\n",
+ " if not hla: # hla is None and plan is primitive\n",
+ " if outcome.goal_test():\n",
+ " return plan.action\n",
+ " else:\n",
+ " for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+ " frontier.append(Node(outcome.init, plan, prefix + sequence+ suffix))\n",
+ "\n", + "Those two actions have some preconditions and some effects. \n", + "If you get the taxi, you need to have cash, whereas if you drive you need to have a car.
\n", + "Thus we define the following hierarchy of possible actions.\n", + "\n", + "##### hierarchy" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "library = {\n", + " 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', 'Taxi(Home, SFO)'],\n", + " 'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []],\n", + " 'precond': [['At(Home) & Have(Car)'], ['At(Home)'], ['At(Home) & Have(Car)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n", + " 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(LongTermParking)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']] }\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "the possible actions are the following:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "go_SFO = HLA('Go(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)')\n", + "taxi_SFO = HLA('Taxi(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home) & ~Have(Cash)')\n", + "drive_SFOLongTermParking = HLA('Drive(Home, SFOLongTermParking)', 'At(Home) & Have(Car)','At(SFOLongTermParking) & ~At(Home)' )\n", + "shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'At(SFOLongTermParking)', 'At(SFO) & ~At(LongTermParking)')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Suppose that (our preconditionds are that) we are Home and we have cash and car and our goal is to get to SFO and maintain our cash, and our possible actions are the above.
\n", + "##### Then our problem is: " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "prob = Problem('At(Home) & Have(Cash) & Have(Car)', 'At(SFO) & Have(Cash)', [go_SFO])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### Refinements\n", + "\n", + "The refinements of the action Go(Home, SFO), are defined as:
\n", + "['Drive(Home,SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)']" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))]\n", + "[{'completed': False, 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'uses': {}, 'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'consumes': {}, 'precond': [At(Home), Have(Car)]}, {'completed': False, 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'consumes': {}, 'precond': [At(SFOLongTermParking)]}] \n", + "\n", + "[HLA(Taxi(Home, SFO))]\n", + "[{'completed': False, 'args': (Home, SFO), 'name': 'Taxi', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(Home), NotHave(Cash)], 'consumes': {}, 'precond': [At(Home)]}] \n", + "\n" + ] + } + ], + "source": [ + "for sequence in Problem.refinements(go_SFO, prob, library):\n", + " print (sequence)\n", + " print([x.__dict__ for x in sequence ], '\\n')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Run the hierarchical search\n", + "##### Top level call" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))] \n", + "\n", + "[{'completed': False, 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'uses': {}, 'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'consumes': {}, 'precond': [At(Home), Have(Car)]}, {'completed': False, 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'consumes': {}, 'precond': [At(SFOLongTermParking)]}]\n" + ] + } + ], + "source": [ + "plan= Problem.hierarchical_search(prob, library)\n", + "print (plan, '\\n')\n", + "print ([x.__dict__ for x in plan])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 2" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "library_2 = {\n", + " 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)' , 'Metro(MetroStop, SFO)', 'Metro1(MetroStop, SFO)', 'Metro2(MetroStop, SFO)' ,'Taxi(Home, SFO)'],\n", + " 'steps': [['Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)'], ['Taxi(Home, SFO)'], [], ['Metro1(MetroStop, SFO)'], ['Metro2(MetroStop, SFO)'],[],[],[]],\n", + " 'precond': [['At(Home)'], ['At(Home)'], ['At(Home)'], ['At(MetroStop)'], ['At(MetroStop)'],['At(MetroStop)'], ['At(MetroStop)'] ,['At(Home) & Have(Cash)']],\n", + " 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(MetroStop) & ~At(Home)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'] , ['At(SFO) & ~At(MetroStop)'] ,['At(SFO) & ~At(Home) & ~Have(Cash)']] \n", + " }" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[HLA(Bus(Home, MetroStop)), HLA(Metro1(MetroStop, SFO))] \n", + "\n", + "[{'completed': False, 'args': (Home, MetroStop), 'name': 'Bus', 'uses': {}, 'duration': 0, 'effect': [At(MetroStop), NotAt(Home)], 'consumes': {}, 'precond': [At(Home)]}, {'completed': False, 'args': (MetroStop, SFO), 'name': 'Metro1', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(MetroStop)], 'consumes': {}, 'precond': [At(MetroStop)]}]\n" + ] + } + ], + "source": [ + "plan_2 = Problem.hierarchical_search(prob, library_2)\n", + "print(plan_2, '\\n')\n", + "print([x.__dict__ for x in plan_2])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_partial_order_planner.ipynb b/planning_partial_order_planner.ipynb new file mode 100644 index 000000000..4b1a98bb3 --- /dev/null +++ b/planning_partial_order_planner.ipynb @@ -0,0 +1,850 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### PARTIAL ORDER PLANNER\n", + "A partial-order planning algorithm is significantly different from a total-order planner.\n", + "The way a partial-order plan works enables it to take advantage of _problem decomposition_ and work on each subproblem separately.\n", + "It works on several subgoals independently, solves them with several subplans, and then combines the plan.\n", + "
\n", + "A partial-order planner also follows the **least commitment** strategy, where it delays making choices for as long as possible.\n", + "Variables are not bound unless it is absolutely necessary and new actions are chosen only if the existing actions cannot fulfil the required precondition.\n", + "
\n", + "Any planning algorithm that can place two actions into a plan without specifying which comes first is called a **partial-order planner**.\n", + "A partial-order planner searches through the space of plans rather than the space of states, which makes it perform better for certain problems.\n", + "
\n", + "
\n", + "Let's have a look at the `PartialOrderPlanner` class." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class PartialOrderPlanner:\n",
+ "\n",
+ " def __init__(self, planningproblem):\n",
+ " self.planningproblem = planningproblem\n",
+ " self.initialize()\n",
+ "\n",
+ " def initialize(self):\n",
+ " """Initialize all variables"""\n",
+ " self.causal_links = []\n",
+ " self.start = Action('Start', [], self.planningproblem.init)\n",
+ " self.finish = Action('Finish', self.planningproblem.goals, [])\n",
+ " self.actions = set()\n",
+ " self.actions.add(self.start)\n",
+ " self.actions.add(self.finish)\n",
+ " self.constraints = set()\n",
+ " self.constraints.add((self.start, self.finish))\n",
+ " self.agenda = set()\n",
+ " for precond in self.finish.precond:\n",
+ " self.agenda.add((precond, self.finish))\n",
+ " self.expanded_actions = self.expand_actions()\n",
+ "\n",
+ " def expand_actions(self, name=None):\n",
+ " """Generate all possible actions with variable bindings for precondition selection heuristic"""\n",
+ "\n",
+ " objects = set(arg for clause in self.planningproblem.init for arg in clause.args)\n",
+ " expansions = []\n",
+ " action_list = []\n",
+ " if name is not None:\n",
+ " for action in self.planningproblem.actions:\n",
+ " if str(action.name) == name:\n",
+ " action_list.append(action)\n",
+ " else:\n",
+ " action_list = self.planningproblem.actions\n",
+ "\n",
+ " for action in action_list:\n",
+ " for permutation in itertools.permutations(objects, len(action.args)):\n",
+ " bindings = unify(Expr(action.name, *action.args), Expr(action.name, *permutation))\n",
+ " if bindings is not None:\n",
+ " new_args = []\n",
+ " for arg in action.args:\n",
+ " if arg in bindings:\n",
+ " new_args.append(bindings[arg])\n",
+ " else:\n",
+ " new_args.append(arg)\n",
+ " new_expr = Expr(str(action.name), *new_args)\n",
+ " new_preconds = []\n",
+ " for precond in action.precond:\n",
+ " new_precond_args = []\n",
+ " for arg in precond.args:\n",
+ " if arg in bindings:\n",
+ " new_precond_args.append(bindings[arg])\n",
+ " else:\n",
+ " new_precond_args.append(arg)\n",
+ " new_precond = Expr(str(precond.op), *new_precond_args)\n",
+ " new_preconds.append(new_precond)\n",
+ " new_effects = []\n",
+ " for effect in action.effect:\n",
+ " new_effect_args = []\n",
+ " for arg in effect.args:\n",
+ " if arg in bindings:\n",
+ " new_effect_args.append(bindings[arg])\n",
+ " else:\n",
+ " new_effect_args.append(arg)\n",
+ " new_effect = Expr(str(effect.op), *new_effect_args)\n",
+ " new_effects.append(new_effect)\n",
+ " expansions.append(Action(new_expr, new_preconds, new_effects))\n",
+ "\n",
+ " return expansions\n",
+ "\n",
+ " def find_open_precondition(self):\n",
+ " """Find open precondition with the least number of possible actions"""\n",
+ "\n",
+ " number_of_ways = dict()\n",
+ " actions_for_precondition = dict()\n",
+ " for element in self.agenda:\n",
+ " open_precondition = element[0]\n",
+ " possible_actions = list(self.actions) + self.expanded_actions\n",
+ " for action in possible_actions:\n",
+ " for effect in action.effect:\n",
+ " if effect == open_precondition:\n",
+ " if open_precondition in number_of_ways:\n",
+ " number_of_ways[open_precondition] += 1\n",
+ " actions_for_precondition[open_precondition].append(action)\n",
+ " else:\n",
+ " number_of_ways[open_precondition] = 1\n",
+ " actions_for_precondition[open_precondition] = [action]\n",
+ "\n",
+ " number = sorted(number_of_ways, key=number_of_ways.__getitem__)\n",
+ " \n",
+ " for k, v in number_of_ways.items():\n",
+ " if v == 0:\n",
+ " return None, None, None\n",
+ "\n",
+ " act1 = None\n",
+ " for element in self.agenda:\n",
+ " if element[0] == number[0]:\n",
+ " act1 = element[1]\n",
+ " break\n",
+ "\n",
+ " if number[0] in self.expanded_actions:\n",
+ " self.expanded_actions.remove(number[0])\n",
+ "\n",
+ " return number[0], act1, actions_for_precondition[number[0]]\n",
+ "\n",
+ " def find_action_for_precondition(self, oprec):\n",
+ " """Find action for a given precondition"""\n",
+ "\n",
+ " # either\n",
+ " # choose act0 E Actions such that act0 achieves G\n",
+ " for action in self.actions:\n",
+ " for effect in action.effect:\n",
+ " if effect == oprec:\n",
+ " return action, 0\n",
+ "\n",
+ " # or\n",
+ " # choose act0 E Actions such that act0 achieves G\n",
+ " for action in self.planningproblem.actions:\n",
+ " for effect in action.effect:\n",
+ " if effect.op == oprec.op:\n",
+ " bindings = unify(effect, oprec)\n",
+ " if bindings is None:\n",
+ " break\n",
+ " return action, bindings\n",
+ "\n",
+ " def generate_expr(self, clause, bindings):\n",
+ " """Generate atomic expression from generic expression given variable bindings"""\n",
+ "\n",
+ " new_args = []\n",
+ " for arg in clause.args:\n",
+ " if arg in bindings:\n",
+ " new_args.append(bindings[arg])\n",
+ " else:\n",
+ " new_args.append(arg)\n",
+ "\n",
+ " try:\n",
+ " return Expr(str(clause.name), *new_args)\n",
+ " except:\n",
+ " return Expr(str(clause.op), *new_args)\n",
+ " \n",
+ " def generate_action_object(self, action, bindings):\n",
+ " """Generate action object given a generic action andvariable bindings"""\n",
+ "\n",
+ " # if bindings is 0, it means the action already exists in self.actions\n",
+ " if bindings == 0:\n",
+ " return action\n",
+ "\n",
+ " # bindings cannot be None\n",
+ " else:\n",
+ " new_expr = self.generate_expr(action, bindings)\n",
+ " new_preconds = []\n",
+ " for precond in action.precond:\n",
+ " new_precond = self.generate_expr(precond, bindings)\n",
+ " new_preconds.append(new_precond)\n",
+ " new_effects = []\n",
+ " for effect in action.effect:\n",
+ " new_effect = self.generate_expr(effect, bindings)\n",
+ " new_effects.append(new_effect)\n",
+ " return Action(new_expr, new_preconds, new_effects)\n",
+ "\n",
+ " def cyclic(self, graph):\n",
+ " """Check cyclicity of a directed graph"""\n",
+ "\n",
+ " new_graph = dict()\n",
+ " for element in graph:\n",
+ " if element[0] in new_graph:\n",
+ " new_graph[element[0]].append(element[1])\n",
+ " else:\n",
+ " new_graph[element[0]] = [element[1]]\n",
+ "\n",
+ " path = set()\n",
+ "\n",
+ " def visit(vertex):\n",
+ " path.add(vertex)\n",
+ " for neighbor in new_graph.get(vertex, ()):\n",
+ " if neighbor in path or visit(neighbor):\n",
+ " return True\n",
+ " path.remove(vertex)\n",
+ " return False\n",
+ "\n",
+ " value = any(visit(v) for v in new_graph)\n",
+ " return value\n",
+ "\n",
+ " def add_const(self, constraint, constraints):\n",
+ " """Add the constraint to constraints if the resulting graph is acyclic"""\n",
+ "\n",
+ " if constraint[0] == self.finish or constraint[1] == self.start:\n",
+ " return constraints\n",
+ "\n",
+ " new_constraints = set(constraints)\n",
+ " new_constraints.add(constraint)\n",
+ "\n",
+ " if self.cyclic(new_constraints):\n",
+ " return constraints\n",
+ " return new_constraints\n",
+ "\n",
+ " def is_a_threat(self, precondition, effect):\n",
+ " """Check if effect is a threat to precondition"""\n",
+ "\n",
+ " if (str(effect.op) == 'Not' + str(precondition.op)) or ('Not' + str(effect.op) == str(precondition.op)):\n",
+ " if effect.args == precondition.args:\n",
+ " return True\n",
+ " return False\n",
+ "\n",
+ " def protect(self, causal_link, action, constraints):\n",
+ " """Check and resolve threats by promotion or demotion"""\n",
+ "\n",
+ " threat = False\n",
+ " for effect in action.effect:\n",
+ " if self.is_a_threat(causal_link[1], effect):\n",
+ " threat = True\n",
+ " break\n",
+ "\n",
+ " if action != causal_link[0] and action != causal_link[2] and threat:\n",
+ " # try promotion\n",
+ " new_constraints = set(constraints)\n",
+ " new_constraints.add((action, causal_link[0]))\n",
+ " if not self.cyclic(new_constraints):\n",
+ " constraints = self.add_const((action, causal_link[0]), constraints)\n",
+ " else:\n",
+ " # try demotion\n",
+ " new_constraints = set(constraints)\n",
+ " new_constraints.add((causal_link[2], action))\n",
+ " if not self.cyclic(new_constraints):\n",
+ " constraints = self.add_const((causal_link[2], action), constraints)\n",
+ " else:\n",
+ " # both promotion and demotion fail\n",
+ " print('Unable to resolve a threat caused by', action, 'onto', causal_link)\n",
+ " return\n",
+ " return constraints\n",
+ "\n",
+ " def convert(self, constraints):\n",
+ " """Convert constraints into a dict of Action to set orderings"""\n",
+ "\n",
+ " graph = dict()\n",
+ " for constraint in constraints:\n",
+ " if constraint[0] in graph:\n",
+ " graph[constraint[0]].add(constraint[1])\n",
+ " else:\n",
+ " graph[constraint[0]] = set()\n",
+ " graph[constraint[0]].add(constraint[1])\n",
+ " return graph\n",
+ "\n",
+ " def toposort(self, graph):\n",
+ " """Generate topological ordering of constraints"""\n",
+ "\n",
+ " if len(graph) == 0:\n",
+ " return\n",
+ "\n",
+ " graph = graph.copy()\n",
+ "\n",
+ " for k, v in graph.items():\n",
+ " v.discard(k)\n",
+ "\n",
+ " extra_elements_in_dependencies = _reduce(set.union, graph.values()) - set(graph.keys())\n",
+ "\n",
+ " graph.update({element:set() for element in extra_elements_in_dependencies})\n",
+ " while True:\n",
+ " ordered = set(element for element, dependency in graph.items() if len(dependency) == 0)\n",
+ " if not ordered:\n",
+ " break\n",
+ " yield ordered\n",
+ " graph = {element: (dependency - ordered) for element, dependency in graph.items() if element not in ordered}\n",
+ " if len(graph) != 0:\n",
+ " raise ValueError('The graph is not acyclic and cannot be linearly ordered')\n",
+ "\n",
+ " def display_plan(self):\n",
+ " """Display causal links, constraints and the plan"""\n",
+ "\n",
+ " print('Causal Links')\n",
+ " for causal_link in self.causal_links:\n",
+ " print(causal_link)\n",
+ "\n",
+ " print('\\nConstraints')\n",
+ " for constraint in self.constraints:\n",
+ " print(constraint[0], '<', constraint[1])\n",
+ "\n",
+ " print('\\nPartial Order Plan')\n",
+ " print(list(reversed(list(self.toposort(self.convert(self.constraints))))))\n",
+ "\n",
+ " def execute(self, display=True):\n",
+ " """Execute the algorithm"""\n",
+ "\n",
+ " step = 1\n",
+ " self.tries = 1\n",
+ " while len(self.agenda) > 0:\n",
+ " step += 1\n",
+ " # select <G, act1> from Agenda\n",
+ " try:\n",
+ " G, act1, possible_actions = self.find_open_precondition()\n",
+ " except IndexError:\n",
+ " print('Probably Wrong')\n",
+ " break\n",
+ "\n",
+ " act0 = possible_actions[0]\n",
+ " # remove <G, act1> from Agenda\n",
+ " self.agenda.remove((G, act1))\n",
+ "\n",
+ " # For actions with variable number of arguments, use least commitment principle\n",
+ " # act0_temp, bindings = self.find_action_for_precondition(G)\n",
+ " # act0 = self.generate_action_object(act0_temp, bindings)\n",
+ "\n",
+ " # Actions = Actions U {act0}\n",
+ " self.actions.add(act0)\n",
+ "\n",
+ " # Constraints = add_const(start < act0, Constraints)\n",
+ " self.constraints = self.add_const((self.start, act0), self.constraints)\n",
+ "\n",
+ " # for each CL E CausalLinks do\n",
+ " # Constraints = protect(CL, act0, Constraints)\n",
+ " for causal_link in self.causal_links:\n",
+ " self.constraints = self.protect(causal_link, act0, self.constraints)\n",
+ "\n",
+ " # Agenda = Agenda U {<P, act0>: P is a precondition of act0}\n",
+ " for precondition in act0.precond:\n",
+ " self.agenda.add((precondition, act0))\n",
+ "\n",
+ " # Constraints = add_const(act0 < act1, Constraints)\n",
+ " self.constraints = self.add_const((act0, act1), self.constraints)\n",
+ "\n",
+ " # CausalLinks U {<act0, G, act1>}\n",
+ " if (act0, G, act1) not in self.causal_links:\n",
+ " self.causal_links.append((act0, G, act1))\n",
+ "\n",
+ " # for each A E Actions do\n",
+ " # Constraints = protect(<act0, G, act1>, A, Constraints)\n",
+ " for action in self.actions:\n",
+ " self.constraints = self.protect((act0, G, act1), action, self.constraints)\n",
+ "\n",
+ " if step > 200:\n",
+ " print('Couldn\\'t find a solution')\n",
+ " return None, None\n",
+ "\n",
+ " if display:\n",
+ " self.display_plan()\n",
+ " else:\n",
+ " return self.constraints, self.causal_links \n",
+ "\n", + "
\n", + "Let's briefly glance over the helper functions before going into the actual algorithm.\n", + "
\n", + "**`expand_actions`**: generates all possible actions with variable bindings for use as a heuristic of selection of an open precondition.\n", + "
\n", + "**`find_open_precondition`**: finds a precondition from the agenda with the least number of actions that fulfil that precondition.\n", + "This heuristic helps form mandatory ordering constraints and causal-links to further simplify the problem and reduce the probability of encountering a threat.\n", + "
\n", + "**`find_action_for_precondition`**: finds an action that fulfils the given precondition along with the absolutely necessary variable bindings in accordance with the principle of _least commitment_.\n", + "In case of multiple possible actions, the action with the least number of effects is chosen to minimize the chances of encountering a threat.\n", + "
\n", + "**`cyclic`**: checks if a directed graph is cyclic.\n", + "
\n", + "**`add_const`**: adds `constraint` to `constraints` if the newly formed graph is acyclic and returns `constraints` otherwise.\n", + "
\n", + "**`is_a_threat`**: checks if the given `effect` negates the given `precondition`.\n", + "
\n", + "**`protect`**: checks if the given `action` poses a threat to the given `causal_link`.\n", + "If so, the threat is resolved by either promotion or demotion, whichever generates acyclic temporal constraints.\n", + "If neither promotion or demotion work, the chosen action is not the correct fit or the planning problem cannot be solved altogether.\n", + "
\n", + "**`convert`**: converts a graph from a list of edges to an `Action` : `set` mapping, for use in topological sorting.\n", + "
\n", + "**`toposort`**: a generator function that generates a topological ordering of a given graph as a list of sets.\n", + "Each set contains an action or several actions.\n", + "If a set has more that one action in it, it means that permutations between those actions also produce a valid plan.\n", + "
\n", + "**`display_plan`**: displays the `causal_links`, `constraints` and the partial order plan generated from `toposort`.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The **`execute`** method executes the algorithm, which is summarized below:\n", + "
\n", + "1. An open precondition is selected (a sub-goal that we want to achieve).\n", + "2. An action that fulfils the open precondition is chosen.\n", + "3. Temporal constraints are updated.\n", + "4. Existing causal links are protected. Protection is a method that checks if the causal links conflict\n", + " and if they do, temporal constraints are added to fix the threats.\n", + "5. The set of open preconditions is updated.\n", + "6. Temporal constraints of the selected action and the next action are established.\n", + "7. A new causal link is added between the selected action and the owner of the open precondition.\n", + "8. The set of new causal links is checked for threats and if found, the threat is removed by either promotion or demotion.\n", + " If promotion or demotion is unable to solve the problem, the planning problem cannot be solved with the current sequence of actions\n", + " or it may not be solvable at all.\n", + "9. These steps are repeated until the set of open preconditions is empty." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A partial-order plan can be used to generate different valid total-order plans.\n", + "This step is called **linearization** of the partial-order plan.\n", + "All possible linearizations of a partial-order plan for `socks_and_shoes` looks like this.\n", + "
\n", + "\n", + "
\n", + "Linearization can be carried out in many ways, but the most efficient way is to represent the set of temporal constraints as a directed graph.\n", + "We can easily realize that the graph should also be acyclic as cycles in constraints means that the constraints are inconsistent.\n", + "This acyclicity is enforced by the `add_const` method, which adds a new constraint only if the acyclicity of the existing graph is not violated.\n", + "The `protect` method also checks for acyclicity of the newly-added temporal constraints to make a decision between promotion and demotion in case of a threat.\n", + "This property of a graph created from the temporal constraints of a valid partial-order plan allows us to use topological sort to order the constraints linearly.\n", + "A topological sort may produce several different valid solutions for a given directed acyclic graph." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we know how `PartialOrderPlanner` works, let's solve a few problems using it." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(PutOn(Spare, Axle)), At(Spare, Axle), Action(Finish))\n", + "(Action(Start), Tire(Spare), Action(PutOn(Spare, Axle)))\n", + "(Action(Remove(Flat, Axle)), NotAt(Flat, Axle), Action(PutOn(Spare, Axle)))\n", + "(Action(Start), At(Flat, Axle), Action(Remove(Flat, Axle)))\n", + "(Action(Remove(Spare, Trunk)), At(Spare, Ground), Action(PutOn(Spare, Axle)))\n", + "(Action(Start), At(Spare, Trunk), Action(Remove(Spare, Trunk)))\n", + "(Action(Remove(Flat, Axle)), At(Flat, Ground), Action(Finish))\n", + "\n", + "Constraints\n", + "Action(Remove(Flat, Axle)) < Action(PutOn(Spare, Axle))\n", + "Action(Start) < Action(Finish)\n", + "Action(Remove(Spare, Trunk)) < Action(PutOn(Spare, Axle))\n", + "Action(Start) < Action(Remove(Spare, Trunk))\n", + "Action(Start) < Action(Remove(Flat, Axle))\n", + "Action(Remove(Flat, Axle)) < Action(Finish)\n", + "Action(PutOn(Spare, Axle)) < Action(Finish)\n", + "Action(Start) < Action(PutOn(Spare, Axle))\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(Remove(Flat, Axle)), Action(Remove(Spare, Trunk))}, {Action(PutOn(Spare, Axle))}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "st = spare_tire()\n", + "pop = PartialOrderPlanner(st)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that in the given partial order plan, Remove(Flat, Axle) and Remove(Spare, Trunk) are in the same set.\n", + "This means that the order of performing these actions does not affect the final outcome.\n", + "That aside, we also see that the PutOn(Spare, Axle) action has to be performed after both the Remove actions are complete, which seems logically consistent." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(FromTable(C, B)), On(C, B), Action(Finish))\n", + "(Action(FromTable(B, A)), On(B, A), Action(Finish))\n", + "(Action(Start), OnTable(B), Action(FromTable(B, A)))\n", + "(Action(Start), OnTable(C), Action(FromTable(C, B)))\n", + "(Action(Start), Clear(C), Action(FromTable(C, B)))\n", + "(Action(Start), Clear(A), Action(FromTable(B, A)))\n", + "(Action(ToTable(A, B)), Clear(B), Action(FromTable(C, B)))\n", + "(Action(Start), On(A, B), Action(ToTable(A, B)))\n", + "(Action(ToTable(A, B)), Clear(B), Action(FromTable(B, A)))\n", + "(Action(Start), Clear(A), Action(ToTable(A, B)))\n", + "\n", + "Constraints\n", + "Action(Start) < Action(FromTable(C, B))\n", + "Action(FromTable(B, A)) < Action(FromTable(C, B))\n", + "Action(Start) < Action(FromTable(B, A))\n", + "Action(Start) < Action(ToTable(A, B))\n", + "Action(Start) < Action(Finish)\n", + "Action(FromTable(B, A)) < Action(Finish)\n", + "Action(FromTable(C, B)) < Action(Finish)\n", + "Action(ToTable(A, B)) < Action(FromTable(B, A))\n", + "Action(ToTable(A, B)) < Action(FromTable(C, B))\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(ToTable(A, B))}, {Action(FromTable(B, A))}, {Action(FromTable(C, B))}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "sbw = simple_blocks_world()\n", + "pop = PartialOrderPlanner(sbw)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "We see that this plan does not have flexibility in selecting actions, ie, actions should be performed in this order and this order only, to successfully reach the goal state." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(RightShoe), RightShoeOn, Action(Finish))\n", + "(Action(LeftShoe), LeftShoeOn, Action(Finish))\n", + "(Action(LeftSock), LeftSockOn, Action(LeftShoe))\n", + "(Action(RightSock), RightSockOn, Action(RightShoe))\n", + "\n", + "Constraints\n", + "Action(LeftSock) < Action(LeftShoe)\n", + "Action(RightSock) < Action(RightShoe)\n", + "Action(Start) < Action(RightShoe)\n", + "Action(Start) < Action(Finish)\n", + "Action(LeftShoe) < Action(Finish)\n", + "Action(Start) < Action(RightSock)\n", + "Action(Start) < Action(LeftShoe)\n", + "Action(Start) < Action(LeftSock)\n", + "Action(RightShoe) < Action(Finish)\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(LeftSock), Action(RightSock)}, {Action(LeftShoe), Action(RightShoe)}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "ss = socks_and_shoes()\n", + "pop = PartialOrderPlanner(ss)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "This plan again doesn't have constraints in selecting socks or shoes.\n", + "As long as both socks are worn before both shoes, we are fine.\n", + "Notice however, there is one valid solution,\n", + "
\n", + "LeftSock -> LeftShoe -> RightSock -> RightShoe\n", + "
\n", + "that the algorithm could not find as it cannot be represented as a general partially-ordered plan but is a specific total-order solution." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Runtime differences\n", + "Let's briefly take a look at the running time of all the three algorithms on the `socks_and_shoes` problem." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "ss = socks_and_shoes()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "198 µs ± 3.53 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "GraphPlan(ss).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "844 µs ± 23.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "Linearize(ss).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "258 µs ± 4.03 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "PartialOrderPlanner(ss).execute(display=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that `GraphPlan` is about 4 times faster than `Linearize` because `Linearize` essentially runs a `GraphPlan` subroutine under the hood and then carries out some transformations on the solved planning-graph.\n", + "
\n", + "We also find that `GraphPlan` is slightly faster than `PartialOrderPlanner`, but this is mainly due to the `expand_actions` method in `PartialOrderPlanner` that slows it down as it generates all possible permutations of actions and variable bindings.\n", + "
\n", + "Without heuristic functions, `PartialOrderPlanner` will be atleast as fast as `GraphPlan`, if not faster, but will have a higher tendency to encounter threats and conflicts which might take additional time to resolve.\n", + "
\n", + "Different planning algorithms work differently for different problems." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_total_order_planner.ipynb b/planning_total_order_planner.ipynb new file mode 100644 index 000000000..b94941ece --- /dev/null +++ b/planning_total_order_planner.ipynb @@ -0,0 +1,341 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### TOTAL ORDER PLANNER\n", + "\n", + "In mathematical terminology, **total order**, **linear order** or **simple order** refers to a set *X* which is said to be totally ordered under ≤ if the following statements hold for all *a*, *b* and *c* in *X*:\n", + "
\n", + "If *a* ≤ *b* and *b* ≤ *a*, then *a* = *b* (antisymmetry).\n", + "
\n", + "If *a* ≤ *b* and *b* ≤ *c*, then *a* ≤ *c* (transitivity).\n", + "
\n", + "*a* ≤ *b* or *b* ≤ *a* (connex relation).\n", + "\n", + "
\n", + "In simpler terms, a total order plan is a linear ordering of actions to be taken to reach the goal state.\n", + "There may be several different total-order plans for a particular goal depending on the problem.\n", + "
\n", + "
\n", + "In the module, the `Linearize` class solves problems using this paradigm.\n", + "At its core, the `Linearize` uses a solved planning graph from `GraphPlan` and finds a valid total-order solution for it.\n", + "Let's have a look at the class." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + "
class Linearize:\n",
+ "\n",
+ " def __init__(self, planningproblem):\n",
+ " self.planningproblem = planningproblem\n",
+ "\n",
+ " def filter(self, solution):\n",
+ " """Filter out persistence actions from a solution"""\n",
+ "\n",
+ " new_solution = []\n",
+ " for section in solution[0]:\n",
+ " new_section = []\n",
+ " for operation in section:\n",
+ " if not (operation.op[0] == 'P' and operation.op[1].isupper()):\n",
+ " new_section.append(operation)\n",
+ " new_solution.append(new_section)\n",
+ " return new_solution\n",
+ "\n",
+ " def orderlevel(self, level, planningproblem):\n",
+ " """Return valid linear order of actions for a given level"""\n",
+ "\n",
+ " for permutation in itertools.permutations(level):\n",
+ " temp = copy.deepcopy(planningproblem)\n",
+ " count = 0\n",
+ " for action in permutation:\n",
+ " try:\n",
+ " temp.act(action)\n",
+ " count += 1\n",
+ " except:\n",
+ " count = 0\n",
+ " temp = copy.deepcopy(planningproblem)\n",
+ " break\n",
+ " if count == len(permutation):\n",
+ " return list(permutation), temp\n",
+ " return None\n",
+ "\n",
+ " def execute(self):\n",
+ " """Finds total-order solution for a planning graph"""\n",
+ "\n",
+ " graphplan_solution = GraphPlan(self.planningproblem).execute()\n",
+ " filtered_solution = self.filter(graphplan_solution)\n",
+ " ordered_solution = []\n",
+ " planningproblem = self.planningproblem\n",
+ " for level in filtered_solution:\n",
+ " level_solution, planningproblem = self.orderlevel(level, planningproblem)\n",
+ " for element in level_solution:\n",
+ " ordered_solution.append(element)\n",
+ "\n",
+ " return ordered_solution\n",
+ "\n", + "The `orderlevel` method finds a valid total-ordering of a specified level of the planning-graph, given the state of the graph after the previous level.\n", + "
\n", + "The `execute` method sequentially calls `orderlevel` for all the levels in the planning-graph and returns the final total-order solution.\n", + "
\n", + "
\n", + "Let's look at some examples." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Unload(C2, P2, SFO),\n", + " Unload(C1, P1, JFK)]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for air_cargo problem\n", + "Linearize(air_cargo()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for spare_tire problem\n", + "Linearize(spare_tire()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for three_block_tower problem\n", + "Linearize(three_block_tower()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[ToTable(A, B), FromTable(B, A), FromTable(C, B)]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for simple_blocks_world problem\n", + "Linearize(simple_blocks_world()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[RightSock, LeftSock, RightShoe, LeftShoe]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for socks_and_shoes problem\n", + "Linearize(socks_and_shoes()).execute()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/probabilistic_learning.py b/probabilistic_learning.py new file mode 100644 index 000000000..1138e702d --- /dev/null +++ b/probabilistic_learning.py @@ -0,0 +1,154 @@ +"""Learning probabilistic models. (Chapters 20)""" + +import heapq + +from utils import weighted_sampler, product, gaussian + + +class CountingProbDist: + """ + A probability distribution formed by observing and counting examples. + If p is an instance of this class and o is an observed value, then + there are 3 main operations: + p.add(o) increments the count for observation o by 1. + p.sample() returns a random element from the distribution. + p[o] returns the probability for o (as in a regular ProbDist). + """ + + def __init__(self, observations=None, default=0): + """ + Create a distribution, and optionally add in some observations. + By default this is an unsmoothed distribution, but saying default=1, + for example, gives you add-one smoothing. + """ + if observations is None: + observations = [] + self.dictionary = {} + self.n_obs = 0 + self.default = default + self.sampler = None + + for o in observations: + self.add(o) + + def add(self, o): + """Add an observation o to the distribution.""" + self.smooth_for(o) + self.dictionary[o] += 1 + self.n_obs += 1 + self.sampler = None + + def smooth_for(self, o): + """ + Include o among the possible observations, whether or not + it's been observed yet. + """ + if o not in self.dictionary: + self.dictionary[o] = self.default + self.n_obs += self.default + self.sampler = None + + def __getitem__(self, item): + """Return an estimate of the probability of item.""" + self.smooth_for(item) + return self.dictionary[item] / self.n_obs + + # (top() and sample() are not used in this module, but elsewhere.) + + def top(self, n): + """Return (count, obs) tuples for the n most frequent observations.""" + return heapq.nlargest(n, [(v, k) for (k, v) in self.dictionary.items()]) + + def sample(self): + """Return a random sample from the distribution.""" + if self.sampler is None: + self.sampler = weighted_sampler(list(self.dictionary.keys()), list(self.dictionary.values())) + return self.sampler() + + +def NaiveBayesLearner(dataset, continuous=True, simple=False): + if simple: + return NaiveBayesSimple(dataset) + if continuous: + return NaiveBayesContinuous(dataset) + else: + return NaiveBayesDiscrete(dataset) + + +def NaiveBayesSimple(distribution): + """ + A simple naive bayes classifier that takes as input a dictionary of + CountingProbDist objects and classifies items according to these distributions. + The input dictionary is in the following form: + (ClassName, ClassProb): CountingProbDist + """ + target_dist = {c_name: prob for c_name, prob in distribution.keys()} + attr_dists = {c_name: count_prob for (c_name, _), count_prob in distribution.items()} + + def predict(example): + """Predict the target value for example. Calculate probabilities for each + class and pick the max.""" + + def class_probability(target_val): + attr_dist = attr_dists[target_val] + return target_dist[target_val] * product(attr_dist[a] for a in example) + + return max(target_dist.keys(), key=class_probability) + + return predict + + +def NaiveBayesDiscrete(dataset): + """ + Just count how many times each value of each input attribute + occurs, conditional on the target value. Count the different + target values too. + """ + + target_vals = dataset.values[dataset.target] + target_dist = CountingProbDist(target_vals) + attr_dists = {(gv, attr): CountingProbDist(dataset.values[attr]) for gv in target_vals for attr in dataset.inputs} + for example in dataset.examples: + target_val = example[dataset.target] + target_dist.add(target_val) + for attr in dataset.inputs: + attr_dists[target_val, attr].add(example[attr]) + + def predict(example): + """ + Predict the target value for example. Consider each possible value, + and pick the most likely by looking at each attribute independently. + """ + + def class_probability(target_val): + return (target_dist[target_val] * product(attr_dists[target_val, attr][example[attr]] + for attr in dataset.inputs)) + + return max(target_vals, key=class_probability) + + return predict + + +def NaiveBayesContinuous(dataset): + """ + Count how many times each target value occurs. + Also, find the means and deviations of input attribute values for each target value. + """ + means, deviations = dataset.find_means_and_deviations() + + target_vals = dataset.values[dataset.target] + target_dist = CountingProbDist(target_vals) + + def predict(example): + """Predict the target value for example. Consider each possible value, + and pick the most likely by looking at each attribute independently.""" + + def class_probability(target_val): + prob = target_dist[target_val] + for attr in dataset.inputs: + prob *= gaussian(means[target_val][attr], deviations[target_val][attr], example[attr]) + return prob + + return max(target_vals, key=class_probability) + + return predict diff --git a/probability.ipynb b/probability.ipynb index 446fc11fb..fe9643a83 100644 --- a/probability.ipynb +++ b/probability.ipynb @@ -1,24 +1,6515 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Probability \n", + "\n", + "This IPy notebook acts as supporting material for topics covered in **Chapter 13 Quantifying Uncertainty**, **Chapter 14 Probabilistic Reasoning**, **Chapter 15 Probabilistic Reasoning over Time**, **Chapter 16 Making Simple Decisions** and parts of **Chapter 25 Robotics** of the book* Artificial Intelligence: A Modern Approach*. This notebook makes use of the implementations in probability.py module. Let us import everything from the probability module. It might be helpful to view the source of some of our implementations. Please refer to the Introductory IPy file for more details on how to do so." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from probability import *\n", + "from utils import print_table\n", + "from notebook import psource, pseudocode, heatmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "- Probability Distribution\n", + " - Joint probability distribution\n", + " - Inference using full joint distributions\n", + "
\n", + "- Bayesian Networks\n", + " - BayesNode\n", + " - BayesNet\n", + " - Exact Inference in Bayesian Networks\n", + " - Enumeration\n", + " - Variable elimination\n", + " - Approximate Inference in Bayesian Networks\n", + " - Prior sample\n", + " - Rejection sampling\n", + " - Likelihood weighting\n", + " - Gibbs sampling\n", + "
\n", + "- Hidden Markov Models\n", + " - Inference in Hidden Markov Models\n", + " - Forward-backward\n", + " - Fixed lag smoothing\n", + " - Particle filtering\n", + "
\n", + "
\n", + "- Monte Carlo Localization\n", + "- Decision Theoretic Agent\n", + "- Information Gathering Agent" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PROBABILITY DISTRIBUTION\n", + "\n", + "Let us begin by specifying discrete probability distributions. The class **ProbDist** defines a discrete probability distribution. We name our random variable and then assign probabilities to the different values of the random variable. Assigning probabilities to the values works similar to that of using a dictionary with keys being the Value and we assign to it the probability. This is possible because of the magic methods **_ _getitem_ _** and **_ _setitem_ _** which store the probabilities in the prob dict of the object. You can keep the source window open alongside while playing with the rest of the code to get a better understanding." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
class ProbDist:\n",
+ " """A discrete probability distribution. You name the random variable\n",
+ " in the constructor, then assign and query probability of values.\n",
+ " >>> P = ProbDist('Flip'); P['H'], P['T'] = 0.25, 0.75; P['H']\n",
+ " 0.25\n",
+ " >>> P = ProbDist('X', {'lo': 125, 'med': 375, 'hi': 500})\n",
+ " >>> P['lo'], P['med'], P['hi']\n",
+ " (0.125, 0.375, 0.5)\n",
+ " """\n",
+ "\n",
+ " def __init__(self, varname='?', freqs=None):\n",
+ " """If freqs is given, it is a dictionary of values - frequency pairs,\n",
+ " then ProbDist is normalized."""\n",
+ " self.prob = {}\n",
+ " self.varname = varname\n",
+ " self.values = []\n",
+ " if freqs:\n",
+ " for (v, p) in freqs.items():\n",
+ " self[v] = p\n",
+ " self.normalize()\n",
+ "\n",
+ " def __getitem__(self, val):\n",
+ " """Given a value, return P(value)."""\n",
+ " try:\n",
+ " return self.prob[val]\n",
+ " except KeyError:\n",
+ " return 0\n",
+ "\n",
+ " def __setitem__(self, val, p):\n",
+ " """Set P(val) = p."""\n",
+ " if val not in self.values:\n",
+ " self.values.append(val)\n",
+ " self.prob[val] = p\n",
+ "\n",
+ " def normalize(self):\n",
+ " """Make sure the probabilities of all values sum to 1.\n",
+ " Returns the normalized distribution.\n",
+ " Raises a ZeroDivisionError if the sum of the values is 0."""\n",
+ " total = sum(self.prob.values())\n",
+ " if not isclose(total, 1.0):\n",
+ " for val in self.prob:\n",
+ " self.prob[val] /= total\n",
+ " return self\n",
+ "\n",
+ " def show_approx(self, numfmt='{:.3g}'):\n",
+ " """Show the probabilities rounded and sorted by key, for the\n",
+ " sake of portable doctests."""\n",
+ " return ', '.join([('{}: ' + numfmt).format(v, p)\n",
+ " for (v, p) in sorted(self.prob.items())])\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return "P({})".format(self.varname)\n",
+ "class JointProbDist(ProbDist):\n",
+ " """A discrete probability distribute over a set of variables.\n",
+ " >>> P = JointProbDist(['X', 'Y']); P[1, 1] = 0.25\n",
+ " >>> P[1, 1]\n",
+ " 0.25\n",
+ " >>> P[dict(X=0, Y=1)] = 0.5\n",
+ " >>> P[dict(X=0, Y=1)]\n",
+ " 0.5"""\n",
+ "\n",
+ " def __init__(self, variables):\n",
+ " self.prob = {}\n",
+ " self.variables = variables\n",
+ " self.vals = defaultdict(list)\n",
+ "\n",
+ " def __getitem__(self, values):\n",
+ " """Given a tuple or dict of values, return P(values)."""\n",
+ " values = event_values(values, self.variables)\n",
+ " return ProbDist.__getitem__(self, values)\n",
+ "\n",
+ " def __setitem__(self, values, p):\n",
+ " """Set P(values) = p. Values can be a tuple or a dict; it must\n",
+ " have a value for each of the variables in the joint. Also keep track\n",
+ " of the values we have seen so far for each variable."""\n",
+ " values = event_values(values, self.variables)\n",
+ " self.prob[values] = p\n",
+ " for var, val in zip(self.variables, values):\n",
+ " if val not in self.vals[var]:\n",
+ " self.vals[var].append(val)\n",
+ "\n",
+ " def values(self, var):\n",
+ " """Return the set of possible values for a variable."""\n",
+ " return self.vals[var]\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return "P({})".format(self.variables)\n",
+ "def enumerate_joint(variables, e, P):\n",
+ " """Return the sum of those entries in P consistent with e,\n",
+ " provided variables is P's remaining variables (the ones not in e)."""\n",
+ " if not variables:\n",
+ " return P[e]\n",
+ " Y, rest = variables[0], variables[1:]\n",
+ " return sum([enumerate_joint(rest, extend(e, Y, y), P)\n",
+ " for y in P.values(Y)])\n",
+ "def enumerate_joint_ask(X, e, P):\n",
+ " """Return a probability distribution over the values of the variable X,\n",
+ " given the {var:val} observations e, in the JointProbDist P. [Section 13.3]\n",
+ " >>> P = JointProbDist(['X', 'Y'])\n",
+ " >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[2,1] = 0.125\n",
+ " >>> enumerate_joint_ask('X', dict(Y=1), P).show_approx()\n",
+ " '0: 0.667, 1: 0.167, 2: 0.167'\n",
+ " """\n",
+ " assert X not in e, "Query variable must be distinct from evidence"\n",
+ " Q = ProbDist(X) # probability distribution for X, initially empty\n",
+ " Y = [v for v in P.variables if v != X and v not in e] # hidden variables.\n",
+ " for xi in P.values(X):\n",
+ " Q[xi] = enumerate_joint(Y, extend(e, X, xi), P)\n",
+ " return Q.normalize()\n",
+ "class BayesNode:\n",
+ " """A conditional probability distribution for a boolean variable,\n",
+ " P(X | parents). Part of a BayesNet."""\n",
+ "\n",
+ " def __init__(self, X, parents, cpt):\n",
+ " """X is a variable name, and parents a sequence of variable\n",
+ " names or a space-separated string. cpt, the conditional\n",
+ " probability table, takes one of these forms:\n",
+ "\n",
+ " * A number, the unconditional probability P(X=true). You can\n",
+ " use this form when there are no parents.\n",
+ "\n",
+ " * A dict {v: p, ...}, the conditional probability distribution\n",
+ " P(X=true | parent=v) = p. When there's just one parent.\n",
+ "\n",
+ " * A dict {(v1, v2, ...): p, ...}, the distribution P(X=true |\n",
+ " parent1=v1, parent2=v2, ...) = p. Each key must have as many\n",
+ " values as there are parents. You can use this form always;\n",
+ " the first two are just conveniences.\n",
+ "\n",
+ " In all cases the probability of X being false is left implicit,\n",
+ " since it follows from P(X=true).\n",
+ "\n",
+ " >>> X = BayesNode('X', '', 0.2)\n",
+ " >>> Y = BayesNode('Y', 'P', {T: 0.2, F: 0.7})\n",
+ " >>> Z = BayesNode('Z', 'P Q',\n",
+ " ... {(T, T): 0.2, (T, F): 0.3, (F, T): 0.5, (F, F): 0.7})\n",
+ " """\n",
+ " if isinstance(parents, str):\n",
+ " parents = parents.split()\n",
+ "\n",
+ " # We store the table always in the third form above.\n",
+ " if isinstance(cpt, (float, int)): # no parents, 0-tuple\n",
+ " cpt = {(): cpt}\n",
+ " elif isinstance(cpt, dict):\n",
+ " # one parent, 1-tuple\n",
+ " if cpt and isinstance(list(cpt.keys())[0], bool):\n",
+ " cpt = {(v,): p for v, p in cpt.items()}\n",
+ "\n",
+ " assert isinstance(cpt, dict)\n",
+ " for vs, p in cpt.items():\n",
+ " assert isinstance(vs, tuple) and len(vs) == len(parents)\n",
+ " assert all(isinstance(v, bool) for v in vs)\n",
+ " assert 0 <= p <= 1\n",
+ "\n",
+ " self.variable = X\n",
+ " self.parents = parents\n",
+ " self.cpt = cpt\n",
+ " self.children = []\n",
+ "\n",
+ " def p(self, value, event):\n",
+ " """Return the conditional probability\n",
+ " P(X=value | parents=parent_values), where parent_values\n",
+ " are the values of parents in event. (event must assign each\n",
+ " parent a value.)\n",
+ " >>> bn = BayesNode('X', 'Burglary', {T: 0.2, F: 0.625})\n",
+ " >>> bn.p(False, {'Burglary': False, 'Earthquake': True})\n",
+ " 0.375"""\n",
+ " assert isinstance(value, bool)\n",
+ " ptrue = self.cpt[event_values(event, self.parents)]\n",
+ " return ptrue if value else 1 - ptrue\n",
+ "\n",
+ " def sample(self, event):\n",
+ " """Sample from the distribution for this variable conditioned\n",
+ " on event's values for parent_variables. That is, return True/False\n",
+ " at random according with the conditional probability given the\n",
+ " parents."""\n",
+ " return probability(self.p(True, event))\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return repr((self.variable, ' '.join(self.parents)))\n",
+ "![](\"files/images/bayesnet.png\")
\n",
+ "\n",
+ "The alarm node can be made as follows: "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "alarm_node = BayesNode('Alarm', ['Burglary', 'Earthquake'], \n",
+ " {(True, True): 0.95,(True, False): 0.94, (False, True): 0.29, (False, False): 0.001})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is possible to avoid using a tuple when there is only a single parent. So an alternative format for the **cpt** is"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "john_node = BayesNode('JohnCalls', ['Alarm'], {True: 0.90, False: 0.05})\n",
+ "mary_node = BayesNode('MaryCalls', 'Alarm', {(True, ): 0.70, (False, ): 0.01}) # Using string for parents.\n",
+ "# Equivalant to john_node definition."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The general format used for the alarm node always holds. For nodes with no parents we can also use. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "burglary_node = BayesNode('Burglary', '', 0.001)\n",
+ "earthquake_node = BayesNode('Earthquake', '', 0.002)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is possible to use the node for lookup function using the **p** method. The method takes in two arguments **value** and **event**. Event must be a dict of the type {variable:values, ..} The value corresponds to the value of the variable we are interested in (False or True).The method returns the conditional probability **P(X=value | parents=parent_values)**, where parent_values are the values of parents in event. (event must assign each parent a value.)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.09999999999999998"
+ ]
+ },
+ "execution_count": 26,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "john_node.p(False, {'Alarm': True, 'Burglary': True}) # P(JohnCalls=False | Alarm=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "With all the information about nodes present it is possible to construct a Bayes Network using **BayesNet**. The **BayesNet** class does not take in nodes as input but instead takes a list of **node_specs**. An entry in **node_specs** is a tuple of the parameters we use to construct a **BayesNode** namely **(X, parents, cpt)**. **node_specs** must be ordered with parents before children."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ " class BayesNet:\n",
+ " """Bayesian network containing only boolean-variable nodes."""\n",
+ "\n",
+ " def __init__(self, node_specs=None):\n",
+ " """Nodes must be ordered with parents before children."""\n",
+ " self.nodes = []\n",
+ " self.variables = []\n",
+ " node_specs = node_specs or []\n",
+ " for node_spec in node_specs:\n",
+ " self.add(node_spec)\n",
+ "\n",
+ " def add(self, node_spec):\n",
+ " """Add a node to the net. Its parents must already be in the\n",
+ " net, and its variable must not."""\n",
+ " node = BayesNode(*node_spec)\n",
+ " assert node.variable not in self.variables\n",
+ " assert all((parent in self.variables) for parent in node.parents)\n",
+ " self.nodes.append(node)\n",
+ " self.variables.append(node.variable)\n",
+ " for parent in node.parents:\n",
+ " self.variable_node(parent).children.append(node)\n",
+ "\n",
+ " def variable_node(self, var):\n",
+ " """Return the node for the variable named var.\n",
+ " >>> burglary.variable_node('Burglary').variable\n",
+ " 'Burglary'"""\n",
+ " for n in self.nodes:\n",
+ " if n.variable == var:\n",
+ " return n\n",
+ " raise Exception("No such variable: {}".format(var))\n",
+ "\n",
+ " def variable_values(self, var):\n",
+ " """Return the domain of var."""\n",
+ " return [True, False]\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return 'BayesNet({0!r})'.format(self.nodes)\n",
+ "def enumerate_all(variables, e, bn):\n",
+ " """Return the sum of those entries in P(variables | e{others})\n",
+ " consistent with e, where P is the joint distribution represented\n",
+ " by bn, and e{others} means e restricted to bn's other variables\n",
+ " (the ones other than variables). Parents must precede children in variables."""\n",
+ " if not variables:\n",
+ " return 1.0\n",
+ " Y, rest = variables[0], variables[1:]\n",
+ " Ynode = bn.variable_node(Y)\n",
+ " if Y in e:\n",
+ " return Ynode.p(e[Y], e) * enumerate_all(rest, e, bn)\n",
+ " else:\n",
+ " return sum(Ynode.p(y, e) * enumerate_all(rest, extend(e, Y, y), bn)\n",
+ " for y in bn.variable_values(Y))\n",
+ "def enumeration_ask(X, e, bn):\n",
+ " """Return the conditional probability distribution of variable X\n",
+ " given evidence e, from BayesNet bn. [Figure 14.9]\n",
+ " >>> enumeration_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary\n",
+ " ... ).show_approx()\n",
+ " 'False: 0.716, True: 0.284'"""\n",
+ " assert X not in e, "Query variable must be distinct from evidence"\n",
+ " Q = ProbDist(X)\n",
+ " for xi in bn.variable_values(X):\n",
+ " Q[xi] = enumerate_all(bn.variables, extend(e, X, xi), bn)\n",
+ " return Q.normalize()\n",
+ "def make_factor(var, e, bn):\n",
+ " """Return the factor for var in bn's joint distribution given e.\n",
+ " That is, bn's full joint distribution, projected to accord with e,\n",
+ " is the pointwise product of these factors for bn's variables."""\n",
+ " node = bn.variable_node(var)\n",
+ " variables = [X for X in [var] + node.parents if X not in e]\n",
+ " cpt = {event_values(e1, variables): node.p(e1[var], e1)\n",
+ " for e1 in all_events(variables, bn, e)}\n",
+ " return Factor(variables, cpt)\n",
+ "def all_events(variables, bn, e):\n",
+ " """Yield every way of extending e with values for all variables."""\n",
+ " if not variables:\n",
+ " yield e\n",
+ " else:\n",
+ " X, rest = variables[0], variables[1:]\n",
+ " for e1 in all_events(rest, bn, e):\n",
+ " for x in bn.variable_values(X):\n",
+ " yield extend(e1, X, x)\n",
+ " def pointwise_product(self, other, bn):\n",
+ " """Multiply two factors, combining their variables."""\n",
+ " variables = list(set(self.variables) | set(other.variables))\n",
+ " cpt = {event_values(e, variables): self.p(e) * other.p(e)\n",
+ " for e in all_events(variables, bn, {})}\n",
+ " return Factor(variables, cpt)\n",
+ "def pointwise_product(factors, bn):\n",
+ " return reduce(lambda f, g: f.pointwise_product(g, bn), factors)\n",
+ " def sum_out(self, var, bn):\n",
+ " """Make a factor eliminating var by summing over its values."""\n",
+ " variables = [X for X in self.variables if X != var]\n",
+ " cpt = {event_values(e, variables): sum(self.p(extend(e, var, val))\n",
+ " for val in bn.variable_values(var))\n",
+ " for e in all_events(variables, bn, {})}\n",
+ " return Factor(variables, cpt)\n",
+ "def sum_out(var, factors, bn):\n",
+ " """Eliminate var from all factors by summing over its values."""\n",
+ " result, var_factors = [], []\n",
+ " for f in factors:\n",
+ " (var_factors if var in f.variables else result).append(f)\n",
+ " result.append(pointwise_product(var_factors, bn).sum_out(var, bn))\n",
+ " return result\n",
+ "def elimination_ask(X, e, bn):\n",
+ " """Compute bn's P(X|e) by variable elimination. [Figure 14.11]\n",
+ " >>> elimination_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary\n",
+ " ... ).show_approx()\n",
+ " 'False: 0.716, True: 0.284'"""\n",
+ " assert X not in e, "Query variable must be distinct from evidence"\n",
+ " factors = []\n",
+ " for var in reversed(bn.variables):\n",
+ " factors.append(make_factor(var, e, bn))\n",
+ " if is_hidden(var, X, e):\n",
+ " factors = sum_out(var, factors, bn)\n",
+ " return pointwise_product(factors, bn).normalize()\n",
+ "\n", + "Of course, for more complicated networks, variable elimination will be significantly faster and runtime will drop not just by a constant factor, but by a polynomial factor proportional to the number of nodes, due to the reduction in repeated calculations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Approximate Inference in Bayesian Networks\n", + "\n", + "Exact inference fails to scale for very large and complex Bayesian Networks. This section covers implementation of randomized sampling algorithms, also called Monte Carlo algorithms." + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def sample(self, event):\n",
+ " """Sample from the distribution for this variable conditioned\n",
+ " on event's values for parent_variables. That is, return True/False\n",
+ " at random according with the conditional probability given the\n",
+ " parents."""\n",
+ " return probability(self.p(True, event))\n",
+ "def prior_sample(bn):\n",
+ " """Randomly sample from bn's full joint distribution. The result\n",
+ " is a {variable: value} dict. [Figure 14.13]"""\n",
+ " event = {}\n",
+ " for node in bn.nodes:\n",
+ " event[node.variable] = node.sample(event)\n",
+ " return event\n",
+ "![](\"files/images/sprinklernet.jpg\")
\n",
+ "\n",
+ "Traversing the graph in topological order is important.\n",
+ "There are two possible topological orderings for this particular directed acyclic graph.\n",
+ "\n", + "1. `Cloudy -> Sprinkler -> Rain -> Wet Grass`\n", + "2. `Cloudy -> Rain -> Sprinkler -> Wet Grass`\n", + "
\n", + "
\n", + "We can follow any of the two orderings to sample from the network.\n", + "Any ordering other than these two, however, cannot be used.\n", + "
\n", + "One way to think about this is that `Cloudy` can be seen as a precondition of both `Rain` and `Sprinkler` and just like we have seen in planning, preconditions need to be satisfied before a certain action can be executed.\n", + "
\n", + "We store the samples on the observations. Let us find **P(Rain=True)** by taking 1000 random samples from the network." + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": {}, + "outputs": [], + "source": [ + "N = 1000\n", + "all_observations = [prior_sample(sprinkler) for x in range(N)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we filter to get the observations where Rain = True" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [], + "source": [ + "rain_true = [observation for observation in all_observations if observation['Rain'] == True]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can find **P(Rain=True)**" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.503\n" + ] + } + ], + "source": [ + "answer = len(rain_true) / N\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Sampling this another time might give different results as we have no control over the distribution of the random samples" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.519\n" + ] + } + ], + "source": [ + "N = 1000\n", + "all_observations = [prior_sample(sprinkler) for x in range(N)]\n", + "rain_true = [observation for observation in all_observations if observation['Rain'] == True]\n", + "answer = len(rain_true) / N\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To evaluate a conditional distribution. We can use a two-step filtering process. We first separate out the variables that are consistent with the evidence. Then for each value of query variable, we can find probabilities. For example to find **P(Cloudy=True | Rain=True)**. We have already filtered out the values consistent with our evidence in **rain_true**. Now we apply a second filtering step on **rain_true** to find **P(Rain=True and Cloudy=True)**" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.8265895953757225\n" + ] + } + ], + "source": [ + "rain_and_cloudy = [observation for observation in rain_true if observation['Cloudy'] == True]\n", + "answer = len(rain_and_cloudy) / len(rain_true)\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Rejection Sampling\n", + "\n", + "Rejection Sampling is based on an idea similar to what we did just now. \n", + "First, it generates samples from the prior distribution specified by the network. \n", + "Then, it rejects all those that do not match the evidence. \n", + "
\n", + "Rejection sampling is advantageous only when we know the query beforehand.\n", + "While prior sampling generally works for any query, it might fail in some scenarios.\n", + "
\n", + "Let's say we have a generic Bayesian network and we have evidence `e`, and we want to know how many times a state `A` is true, given evidence `e` is true.\n", + "Normally, prior sampling can answer this question, but let's assume that the probability of evidence `e` being true in our actual probability distribution is very small.\n", + "In this situation, it might be possible that sampling never encounters a data-point where `e` is true.\n", + "If our sampled data has no instance of `e` being true, `P(e) = 0`, and therefore `P(A | e) / P(e) = 0/0`, which is undefined.\n", + "We cannot find the required value using this sample.\n", + "
\n", + "We can definitely increase the number of sample points, but we can never guarantee that we will encounter the case where `e` is non-zero (assuming our actual probability distribution has atleast one case where `e` is true).\n", + "To guarantee this, we would have to consider every single data point, which means we lose the speed advantage that approximation provides us and we essentially have to calculate the exact inference model of the Bayesian network.\n", + "
\n", + "
\n", + "Rejection sampling will be useful in this situation, as we already know the query.\n", + "
\n", + "While sampling from the network, we will reject any sample which is inconsistent with the evidence variables of the given query (in this example, the only evidence variable is `e`).\n", + "We will only consider samples that do not violate **any** of the evidence variables.\n", + "In this way, we will have enough data with the required evidence to infer queries involving a subset of that evidence.\n", + "
\n", + "
\n", + "The function **rejection_sampling** implements the algorithm described by **Figure 14.14**" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def rejection_sampling(X, e, bn, N=10000):\n",
+ " """Estimate the probability distribution of variable X given\n",
+ " evidence e in BayesNet bn, using N samples. [Figure 14.14]\n",
+ " Raises a ZeroDivisionError if all the N samples are rejected,\n",
+ " i.e., inconsistent with e.\n",
+ " >>> random.seed(47)\n",
+ " >>> rejection_sampling('Burglary', dict(JohnCalls=T, MaryCalls=T),\n",
+ " ... burglary, 10000).show_approx()\n",
+ " 'False: 0.7, True: 0.3'\n",
+ " """\n",
+ " counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.14]\n",
+ " for j in range(N):\n",
+ " sample = prior_sample(bn) # boldface x in [Figure 14.14]\n",
+ " if consistent_with(sample, e):\n",
+ " counts[sample[X]] += 1\n",
+ " return ProbDist(X, counts)\n",
+ "def consistent_with(event, evidence):\n",
+ " """Is event consistent with the given evidence?"""\n",
+ " return all(evidence.get(k, v) == v\n",
+ " for k, v in event.items())\n",
+ "def weighted_sample(bn, e):\n",
+ " """Sample an event from bn that's consistent with the evidence e;\n",
+ " return the event and its weight, the likelihood that the event\n",
+ " accords to the evidence."""\n",
+ " w = 1\n",
+ " event = dict(e) # boldface x in [Figure 14.15]\n",
+ " for node in bn.nodes:\n",
+ " Xi = node.variable\n",
+ " if Xi in e:\n",
+ " w *= node.p(e[Xi], event)\n",
+ " else:\n",
+ " event[Xi] = node.sample(event)\n",
+ " return event, w\n",
+ "def likelihood_weighting(X, e, bn, N=10000):\n",
+ " """Estimate the probability distribution of variable X given\n",
+ " evidence e in BayesNet bn. [Figure 14.15]\n",
+ " >>> random.seed(1017)\n",
+ " >>> likelihood_weighting('Burglary', dict(JohnCalls=T, MaryCalls=T),\n",
+ " ... burglary, 10000).show_approx()\n",
+ " 'False: 0.702, True: 0.298'\n",
+ " """\n",
+ " W = {x: 0 for x in bn.variable_values(X)}\n",
+ " for j in range(N):\n",
+ " sample, weight = weighted_sample(bn, e) # boldface x, w in [Figure 14.15]\n",
+ " W[sample[X]] += weight\n",
+ " return ProbDist(X, W)\n",
+ "def gibbs_ask(X, e, bn, N=1000):\n",
+ " """[Figure 14.16]"""\n",
+ " assert X not in e, "Query variable must be distinct from evidence"\n",
+ " counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.16]\n",
+ " Z = [var for var in bn.variables if var not in e]\n",
+ " state = dict(e) # boldface x in [Figure 14.16]\n",
+ " for Zi in Z:\n",
+ " state[Zi] = random.choice(bn.variable_values(Zi))\n",
+ " for j in range(N):\n",
+ " for Zi in Z:\n",
+ " state[Zi] = markov_blanket_sample(Zi, state, bn)\n",
+ " counts[state[X]] += 1\n",
+ " return ProbDist(X, counts)\n",
+ "\n", + "Likelihood weighting is the fastest out of all as it doesn't involve rejecting samples, but also has a quite high variance." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## HIDDEN MARKOV MODELS" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Often, we need to carry out probabilistic inference on temporal data or a sequence of observations where the order of observations matter.\n", + "We require a model similar to a Bayesian Network, but one that grows over time to keep up with the latest evidences.\n", + "If you are familiar with the `mdp` module or Markov models in general, you can probably guess that a Markov model might come close to representing our problem accurately.\n", + "
\n", + "A Markov model is basically a chain-structured Bayesian Network in which there is one state for each time step and each node has an identical probability distribution.\n", + "The first node, however, has a different distribution, called the prior distribution which models the initial state of the process.\n", + "A state in a Markov model depends only on the previous state and the latest evidence and not on the states before it.\n", + "
\n", + "A **Hidden Markov Model** or **HMM** is a special case of a Markov model in which the state of the process is described by a single discrete random variable.\n", + "The possible values of the variable are the possible states of the world.\n", + "
\n", + "But what if we want to model a process with two or more state variables?\n", + "In that case, we can still fit the process into the HMM framework by redefining our state variables as a single \"megavariable\".\n", + "We do this because carrying out inference on HMMs have standard optimized algorithms.\n", + "A HMM is very similar to an MDP, but we don't have the option of taking actions like in MDPs, instead, the process carries on as new evidence appears.\n", + "
\n", + "If a HMM is truncated at a fixed length, it becomes a Bayesian network and general BN inference can be used on it to answer queries.\n", + "\n", + "Before we start, it will be helpful to understand the structure of a temporal model. We will use the example of the book with the guard and the umbrella. In this example, the state $\\textbf{X}$ is whether it is a rainy day (`X = True`) or not (`X = False`) at Day $\\textbf{t}$. In the sensor or observation model, the observation or evidence $\\textbf{U}$ is whether the professor holds an umbrella (`U = True`) or not (`U = False`) on **Day** $\\textbf{t}$. Based on that, the transition model is \n", + "\n", + "| $X_{t-1}$ | $X_{t}$ | **P**$(X_{t}| X_{t-1})$| \n", + "| ------------- |------------- | ----------------------------------|\n", + "| ***${False}$*** | ***${False}$*** | 0.7 |\n", + "| ***${False}$*** | ***${True}$*** | 0.3 |\n", + "| ***${True}$*** | ***${False}$*** | 0.3 |\n", + "| ***${True}$*** | ***${True}$*** | 0.7 |\n", + "\n", + "And the the sensor model will be,\n", + "\n", + "| $X_{t}$ | $U_{t}$ | **P**$(U_{t}|X_{t})$| \n", + "| :-------------: |:-------------: | :------------------------:|\n", + "| ***${False}$*** | ***${True}$*** | 0.2 |\n", + "| ***${False}$*** | ***${False}$*** | 0.8 |\n", + "| ***${True}$*** | ***${True}$*** | 0.9 |\n", + "| ***${True}$*** | ***${False}$*** | 0.1 |\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "HMMs are implemented in the **`HiddenMarkovModel`** class.\n", + "Let's have a look." + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
class HiddenMarkovModel:\n",
+ " """A Hidden markov model which takes Transition model and Sensor model as inputs"""\n",
+ "\n",
+ " def __init__(self, transition_model, sensor_model, prior=None):\n",
+ " self.transition_model = transition_model\n",
+ " self.sensor_model = sensor_model\n",
+ " self.prior = prior or [0.5, 0.5]\n",
+ "\n",
+ " def sensor_dist(self, ev):\n",
+ " if ev is True:\n",
+ " return self.sensor_model[0]\n",
+ " else:\n",
+ " return self.sensor_model[1]\n",
+ "\n", + "The basic inference tasks that must be solved are:\n", + "1. **Filtering**: Computing the posterior probability distribution over the most recent state, given all the evidence up to the current time step.\n", + "2. **Prediction**: Computing the posterior probability distribution over the future state.\n", + "3. **Smoothing**: Computing the posterior probability distribution over a past state. Smoothing provides a better estimation as it incorporates more evidence.\n", + "4. **Most likely explanation**: Finding the most likely sequence of states for a given observation\n", + "5. **Learning**: The transition and sensor models can be learnt, if not yet known, just like in an information gathering agent\n", + "
\n", + "
\n", + "\n", + "There are three primary methods to carry out inference in Hidden Markov Models:\n", + "1. The Forward-Backward algorithm\n", + "2. Fixed lag smoothing\n", + "3. Particle filtering\n", + "\n", + "Let's have a look at how we can carry out inference and answer queries based on our umbrella HMM using these algorithms." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### FORWARD-BACKWARD\n", + "This is a general algorithm that works for all Markov models, not just HMMs.\n", + "In the filtering task (inference) we are given evidence **U** in each time **t** and we want to compute the belief $B_{t}(x)= P(X_{t}|U_{1:t})$. \n", + "We can think of it as a three step process:\n", + "1. In every step we start with the current belief $P(X_{t}|e_{1:t})$\n", + "2. We update it for time\n", + "3. We update it for evidence\n", + "\n", + "The forward algorithm performs the step 2 and 3 at once. It updates, or better say reweights, the initial belief using the transition and the sensor model. Let's see the umbrella example. On **Day 0** no observation is available, and for that reason we will assume that we have equal possibilities to rain or not. In the **`HiddenMarkovModel`** class, the prior probabilities for **Day 0** are by default [0.5, 0.5]. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The observation update is calculated with the **`forward()`** function. Basically, we update our belief using the observation model. The function returns a list with the probabilities of **raining or not** on **Day 1**." + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def forward(HMM, fv, ev):\n",
+ " prediction = vector_add(scalar_vector_product(fv[0], HMM.transition_model[0]),\n",
+ " scalar_vector_product(fv[1], HMM.transition_model[1]))\n",
+ " sensor_dist = HMM.sensor_dist(ev)\n",
+ "\n",
+ " return normalize(element_wise_product(sensor_dist, prediction))\n",
+ "def backward(HMM, b, ev):\n",
+ " sensor_dist = HMM.sensor_dist(ev)\n",
+ " prediction = element_wise_product(sensor_dist, b)\n",
+ "\n",
+ " return normalize(vector_add(scalar_vector_product(prediction[0], HMM.transition_model[0]),\n",
+ " scalar_vector_product(prediction[1], HMM.transition_model[1])))\n",
+ "\n", + "Additionally, the forward pass stores $t$ vectors of size $S$ which makes the auxiliary space requirement equivalent to $O(St)$.\n", + "
\n", + "
\n", + "Is there any way we can improve the time or space complexity?\n", + "
\n", + "Fortunately, the matrix representation of HMM properties allows us to do so.\n", + "
\n", + "If $f$ and $b$ represent the forward and backward messages respectively, we can modify the smoothing algorithm by first\n", + "running the standard forward pass to compute $f_{t:t}$ (forgetting all the intermediate results) and then running\n", + "backward pass for both $b$ and $f$ together, using them to compute the smoothed estimate at each step.\n", + "This optimization reduces auxlilary space requirement to constant (irrespective of the length of the sequence) provided\n", + "the transition matrix is invertible and the sensor model has no zeros (which is sometimes hard to accomplish)\n", + "
\n", + "
\n", + "Let's look at another algorithm, that carries out smoothing in a more optimized way." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### FIXED LAG SMOOTHING\n", + "The matrix formulation allows to optimize online smoothing with a fixed lag.\n", + "
\n", + "Since smoothing can be done in constant, there should exist an algorithm whose time complexity is independent of the length of the lag.\n", + "For smoothing a time slice $t - d$ where $d$ is the lag, we need to compute $\\alpha f_{1:t-d}$ x $b_{t-d+1:t}$ incrementally.\n", + "
\n", + "As we already know, the forward equation is\n", + "
\n", + "$$f_{1:t+1} = \\alpha O_{t+1}{T}^{T}f_{1:t}$$\n", + "
\n", + "and the backward equation is\n", + "
\n", + "$$b_{k+1:t} = TO_{k+1}b_{k+2:t}$$\n", + "
\n", + "where $T$ and $O$ are the transition and sensor models respectively.\n", + "
\n", + "For smoothing, the forward message is easy to compute but there exists no simple relation between the backward message of this time step and the one at the previous time step, hence we apply the backward equation $d$ times to get\n", + "
\n", + "$$b_{t-d+1:t} = \\left ( \\prod_{i=t-d+1}^{t}{TO_i} \\right )b_{t+1:t} = B_{t-d+1:t}1$$\n", + "
\n", + "where $B_{t-d+1:t}$ is the product of the sequence of $T$ and $O$ matrices.\n", + "
\n", + "Here's how the `probability` module implements `fixed_lag_smoothing`.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def fixed_lag_smoothing(e_t, HMM, d, ev, t):\n",
+ " """[Figure 15.6]\n",
+ " Smoothing algorithm with a fixed time lag of 'd' steps.\n",
+ " Online algorithm that outputs the new smoothed estimate if observation\n",
+ " for new time step is given."""\n",
+ " ev.insert(0, None)\n",
+ "\n",
+ " T_model = HMM.transition_model\n",
+ " f = HMM.prior\n",
+ " B = [[1, 0], [0, 1]]\n",
+ " evidence = []\n",
+ "\n",
+ " evidence.append(e_t)\n",
+ " O_t = vector_to_diagonal(HMM.sensor_dist(e_t))\n",
+ " if t > d:\n",
+ " f = forward(HMM, f, e_t)\n",
+ " O_tmd = vector_to_diagonal(HMM.sensor_dist(ev[t - d]))\n",
+ " B = matrix_multiplication(inverse_matrix(O_tmd), inverse_matrix(T_model), B, T_model, O_t)\n",
+ " else:\n",
+ " B = matrix_multiplication(B, T_model, O_t)\n",
+ " t += 1\n",
+ "\n",
+ " if t > d:\n",
+ " # always returns a 1x2 matrix\n",
+ " return [normalize(i) for i in matrix_multiplication([f], B)][0]\n",
+ " else:\n",
+ " return None\n",
+ "\n", + "`vector_to_diagonal`, `matrix_multiplication` and `inverse_matrix` are matrix manipulation functions to simplify the implementation.\n", + "
\n", + "`normalize` is used to normalize the output before returning it." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here's how we can use `fixed_lag_smoothing` for inference on our umbrella HMM." + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": {}, + "outputs": [], + "source": [ + "umbrella_transition_model = [[0.7, 0.3], [0.3, 0.7]]\n", + "umbrella_sensor_model = [[0.9, 0.2], [0.1, 0.8]]\n", + "hmm = HiddenMarkovModel(umbrella_transition_model, umbrella_sensor_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given evidence T, F, T, F and T, we want to calculate the probability distribution for the fourth day with a fixed lag of 2 days.\n", + "
\n", + "Let `e_t = False`" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[0.1111111111111111, 0.8888888888888888]" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "e_t = F\n", + "evidence = [T, F, T, F, T]\n", + "fixed_lag_smoothing(e_t, hmm, d=2, ev=evidence, t=4)" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[0.9938650306748466, 0.006134969325153394]" + ] + }, + "execution_count": 83, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "e_t = T\n", + "evidence = [T, T, F, T, T]\n", + "fixed_lag_smoothing(e_t, hmm, d=1, ev=evidence, t=4)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We cannot calculate probability distributions when $t$ is less than $d$" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": {}, + "outputs": [], + "source": [ + "fixed_lag_smoothing(e_t, hmm, d=5, ev=evidence, t=4)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As expected, the output is `None`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### PARTICLE FILTERING\n", + "The filtering problem is too expensive to solve using the previous methods for problems with large or continuous state spaces.\n", + "Particle filtering is a method that can solve the same problem but when the state space is a lot larger, where we wouldn't be able to do these computations in a reasonable amount of time as fast, as time goes by, and we want to keep track of things as they happen.\n", + "
\n", + "The downside is that it is a sampling method and hence isn't accurate, but the more samples we're willing to take, the more accurate we'd get.\n", + "
\n", + "In this method, instead of keping track of the probability distribution, we will drop particles in a similar proportion at the required regions.\n", + "The internal representation of this distribution is usually a list of particles with coordinates in the state-space.\n", + "A particle is just a new name for a sample.\n", + "\n", + "Particle filtering can be divided into four steps:\n", + "1. __Initialization__: \n", + "If we have some idea about the prior probability distribution, we drop the initial particles accordingly, or else we just drop them uniformly over the state space.\n", + "\n", + "2. __Forward pass__: \n", + "As time goes by and measurements come in, we are going to move the selected particles into the grid squares that makes the most sense in terms of representing the distribution that we are trying to track.\n", + "When time goes by, we just loop through all our particles and try to simulate what could happen to each one of them by sampling its next position from the transition model.\n", + "This is like prior sampling - samples' frequencies reflect the transition probabilities.\n", + "If we have enough samples we are pretty close to exact values.\n", + "We work through the list of particles, one particle at a time, all we do is stochastically simulate what the outcome might be.\n", + "If we had no dimension of time, and we had no new measurements come in, this would be exactly the same as what we did in prior sampling.\n", + "\n", + "3. __Reweight__:\n", + "As observations come in, don't sample the observations, fix them and downweight the samples based on the evidence just like in likelihood weighting.\n", + "$$w(x) = P(e/x)$$\n", + "$$B(X) \\propto P(e/X)B'(X)$$\n", + "
\n", + "As before, the probabilities don't sum to one, since most have been downweighted.\n", + "They sum to an approximation of $P(e)$.\n", + "To normalize the resulting distribution, we can divide by $P(e)$\n", + "
\n", + "Likelihood weighting wasn't the best thing for Bayesian networks, because we were not accounting for the incoming evidence so we were getting samples from the prior distribution, in some sense not the right distribution, so we might end up with a lot of particles with low weights. \n", + "These samples were very uninformative and the way we fixed it then was by using __Gibbs sampling__.\n", + "Theoretically, Gibbs sampling can be run on a HMM, but as we iterated over the process infinitely many times in a Bayesian network, we cannot do that here as we have new incoming evidence and we also need computational cycles to propagate through time.\n", + "
\n", + "A lot of samples with very low weight and they are not representative of the _actual probability distribution_.\n", + "So if we keep running likelihood weighting, we keep propagating the samples with smaller weights and carry out computations for that even though these samples have no significant contribution to the actual probability distribution.\n", + "Which is why we require this last step.\n", + "\n", + "4. __Resample__:\n", + "Rather than tracking weighted samples, we _resample_.\n", + "We choose from our weighted sample distribution as many times as the number of particles we initially had and we replace these particles too, so that we have a constant number of particles.\n", + "This is equivalent to renormalizing the distribution.\n", + "The samples with low weight are rarely chosen in the new distribution after resampling.\n", + "This newer set of particles after resampling is in some sense more representative of the actual distribution and so we are better allocating our computational cycles.\n", + "Now the update is complete for this time step, continue with the next one.\n", + "\n", + "
\n", + "Let's see how this is implemented in the module." + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def particle_filtering(e, N, HMM):\n",
+ " """Particle filtering considering two states variables."""\n",
+ " dist = [0.5, 0.5]\n",
+ " # Weight Initialization\n",
+ " w = [0 for _ in range(N)]\n",
+ " # STEP 1\n",
+ " # Propagate one step using transition model given prior state\n",
+ " dist = vector_add(scalar_vector_product(dist[0], HMM.transition_model[0]),\n",
+ " scalar_vector_product(dist[1], HMM.transition_model[1]))\n",
+ " # Assign state according to probability\n",
+ " s = ['A' if probability(dist[0]) else 'B' for _ in range(N)]\n",
+ " w_tot = 0\n",
+ " # Calculate importance weight given evidence e\n",
+ " for i in range(N):\n",
+ " if s[i] == 'A':\n",
+ " # P(U|A)*P(A)\n",
+ " w_i = HMM.sensor_dist(e)[0] * dist[0]\n",
+ " if s[i] == 'B':\n",
+ " # P(U|B)*P(B)\n",
+ " w_i = HMM.sensor_dist(e)[1] * dist[1]\n",
+ " w[i] = w_i\n",
+ " w_tot += w_i\n",
+ "\n",
+ " # Normalize all the weights\n",
+ " for i in range(N):\n",
+ " w[i] = w[i] / w_tot\n",
+ "\n",
+ " # Limit weights to 4 digits\n",
+ " for i in range(N):\n",
+ " w[i] = float("{0:.4f}".format(w[i]))\n",
+ "\n",
+ " # STEP 2\n",
+ "\n",
+ " s = weighted_sample_with_replacement(N, s, w)\n",
+ "\n",
+ " return s\n",
+ "\n", + "This implementation considers two state variables with generic names 'A' and 'B'.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here's how we can use `particle_filtering` on our umbrella HMM, though it doesn't make much sense using particle filtering on a problem with such a small state space.\n", + "It is just to get familiar with the syntax." + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": {}, + "outputs": [], + "source": [ + "umbrella_transition_model = [[0.7, 0.3], [0.3, 0.7]]\n", + "umbrella_sensor_model = [[0.9, 0.2], [0.1, 0.8]]\n", + "hmm = HiddenMarkovModel(umbrella_transition_model, umbrella_sensor_model)" + ] + }, { "cell_type": "code", - "execution_count": null, + "execution_count": 87, "metadata": { - "collapsed": false + "scrolled": false }, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']" + ] + }, + "execution_count": 87, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "particle_filtering(T, 10, hmm)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We got 5 samples from state `A` and 5 samples from state `B`" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['A', 'B', 'A', 'B', 'B', 'B', 'B', 'B', 'B', 'B']" + ] + }, + "execution_count": 88, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "particle_filtering([F, T, F, F, T], 10, hmm)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This time we got 2 samples from state `A` and 8 samples from state `B`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Comparing runtimes for these algorithms will not be useful, as each solves the filtering task efficiently for a different scenario.\n", + "
\n", + "`forward_backward` calculates the exact probability distribution.\n", + "
\n", + "`fixed_lag_smoothing` calculates an approximate distribution and its runtime will depend on the value of the lag chosen.\n", + "
\n", + "`particle_filtering` is an efficient method for approximating distributions for a very large or continuous state space." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## MONTE CARLO LOCALIZATION\n", + "In the domain of robotics, particle filtering is used for _robot localization_.\n", + "__Localization__ is the problem of finding out where things are, in this case, we want to find the position of a robot in a continuous state space.\n", + "
\n", + "__Monte Carlo Localization__ is an algorithm for robots to _localize_ using a _particle filter_.\n", + "Given a map of the environment, the algorithm estimates the position and orientation of a robot as it moves and senses the environment.\n", + "
\n", + "Initially, particles are distributed uniformly over the state space, ie the robot has no information of where it is and assumes it is equally likely to be at any point in space.\n", + "
\n", + "When the robot moves, it analyses the incoming evidence to shift and change the probability to better approximate the probability distribution of its position.\n", + "The particles are then resampled based on their weights.\n", + "
\n", + "Gradually, as more evidence comes in, the robot gets better at approximating its location and the particles converge towards the actual position of the robot.\n", + "
\n", + "The pose of a robot is defined by its two Cartesian coordinates with values $x$ and $y$ and its direction with value $\\theta$.\n", + "We use the kinematic equations of motion to model a deterministic state prediction.\n", + "This is our motion model (or transition model).\n", + "
\n", + "Next, we need a sensor model.\n", + "There can be two kinds of sensor models, the first assumes that the sensors detect _stable_, _recognizable_ features of the environment called __landmarks__.\n", + "The robot senses the location and bearing of each landmark and updates its belief according to that.\n", + "We can also assume the noise in measurements to be Gaussian, to simplify things.\n", + "
\n", + "Another kind of sensor model is used for an array of range sensors, each of which has a fixed bearing relative to the robot.\n", + "These sensors provide a set of range values in each direction.\n", + "This will also be corrupted by Gaussian noise, but we can assume that the errors for different beam directions are independent and identically distributed.\n", + "
\n", + "After evidence comes in, the robot updates its belief state and reweights the particle distribution to better aproximate the actual distribution.\n", + "
\n", + "
\n", + "Let's have a look at how this algorithm is implemented in the module" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def monte_carlo_localization(a, z, N, P_motion_sample, P_sensor, m, S=None):\n",
+ " """Monte Carlo localization algorithm from Fig 25.9"""\n",
+ "\n",
+ " def ray_cast(sensor_num, kin_state, m):\n",
+ " return m.ray_cast(sensor_num, kin_state)\n",
+ "\n",
+ " M = len(z)\n",
+ " W = [0]*N\n",
+ " S_ = [0]*N\n",
+ " W_ = [0]*N\n",
+ " v = a['v']\n",
+ " w = a['w']\n",
+ "\n",
+ " if S is None:\n",
+ " S = [m.sample() for _ in range(N)]\n",
+ "\n",
+ " for i in range(N):\n",
+ " S_[i] = P_motion_sample(S[i], v, w)\n",
+ " W_[i] = 1\n",
+ " for j in range(M):\n",
+ " z_ = ray_cast(j, S_[i], m)\n",
+ " W_[i] = W_[i] * P_sensor(z[j], z_)\n",
+ "\n",
+ " S = weighted_sample_with_replacement(N, S_, W_)\n",
+ " return S\n",
+ "\n", + "`a` stores the `v` and `w` components of the robot's velocity.\n", + "
\n", + "`z` is a range scan.\n", + "
\n", + "`P_motion_sample` is the motion or transition model.\n", + "
\n", + "`P_sensor` is the range sensor noise model.\n", + "
\n", + "`m` is the 2D map of the environment\n", + "
\n", + "`S` is a vector of samples of size N" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, "source": [ - "import probability" + "We'll now define a simple 2D map to run Monte Carlo Localization on.\n", + "
\n", + "Let's say this is the map we want\n", + "
" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 90, "metadata": { - "collapsed": true + "scrolled": true }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFaCAYAAADhKw9uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASOUlEQVR4nO3df4ztd13n8dd779hAKSwlvaj9oaVaUJao0JGARFYpxIJIMbvZBcUUf6SJP6AQFIsmaGI0ZDWoiQZTC7aJDailArqKdPEHmrDVuQWEclEaiu2FSoclCLrGWnz7x5yScXrnzvSc750zn9PHI7mZ8+M75/v+3Dszz/s958w51d0BAMbyn5Y9AADw4Ak4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOBwiFXVx6vq2Tsue2lV/cUEt91V9dWL3g6wHAIOAAMScBhYVZ1bVW+tqs2quqOqXr7tuqdW1Xur6rNVdXdV/UpVnTG77j2zzT5QVf9YVf+zqr6lqk5U1aur6p7Z57ywqp5XVX9bVZ+pqp/Yz+3Pru+qenlVfayqPl1VP19VfubARHwzwaBmMfy9JB9Icl6SS5O8oqq+bbbJF5K8Msk5SZ4+u/6HkqS7nznb5uu7+6zu/q3Z+S9L8rDZ7b02ya8neUmSS5J8c5LXVtVFe93+Nt+ZZD3JU5JcnuT7plg7kJTXQofDq6o+nq1A3rft4jOS3JrkVUl+p7u/Ytv2r0ny+O7+3pPc1iuS/Nfu/s7Z+U5ycXffPjv/LUn+MMlZ3f2Fqnpkks8leVp33zLb5liSn+nut+3z9p/b3e+cnf+hJP+tuy9d4K8EmFlb9gDAnl7Y3f/n/jNV9dIkP5DkK5OcW1Wf3bbtkSR/Ptvu8Ulen60j4DOz9f1+bI99/b/u/sLs9D/PPn5q2/X/nOSsB3H7d207/XdJzt1j/8A+uQsdxnVXkju6+9Hb/jyyu583u/4NST6SraPsRyX5iSQ14f73c/sXbDv9FUk+OeH+4SFNwGFcf5nkc1X141X18Ko6UlVPqqpvnF1//13g/1hVX5PkB3d8/qeSXJT57XX7SfJjVXV2VV2Q5Kokv3WSbYA5CDgManZX93ck+YYkdyT5dJJrk/zn2SY/muS7knw+W09G2xnPn05y/exZ5P9jjhH2uv0keXu27lZ/f5L/neSNc+wHOAlPYgNOi51PkgOm5QgcAAYk4AAwIHehA8CAHIEDwIAO9IVczjnnnL7wwgsPcpfAijh2bK/XoGE/LrnkkmWPcFoc9NfHQf49Hjt27NPdfXTn5Qd6F/r6+npvbGwc2P6A1VE15WvQPHSt6sOmB/31cZB/j1V1rLvXd17uLnQAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0EIBr6rLqupvqur2qrp6qqEAgFObO+BVdSTJryZ5bpInJnlxVT1xqsEAgN0tcgT+1CS3d/fHuvveJG9Jcvk0YwEAp7JIwM9Lcte28ydml/0HVXVlVW1U1cbm5uYCuwMA7rdIwE/21i8PeHuW7r6mu9e7e/3o0Qe8GxoAMIdFAn4iyQXbzp+f5JOLjQMA7MciAf+rJBdX1eOq6owkL0ryjmnGAgBOZW3eT+zu+6rqR5L8UZIjSd7U3bdNNhkAsKu5A54k3f0HSf5golkAgH3ySmwAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgBb6PXAA2E3Vyd4yg6k4AgeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAyouvvgdlZ1cDuDh6iD/J4+SFW17BFWwgH/zD+wfR20A/57PNbd6zsvdwQOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABjQ3AGvqguq6k+q6nhV3VZVV005GACwu7UFPve+JK/q7lur6pFJjlXVzd394YlmAwB2MfcReHff3d23zk5/PsnxJOdNNRgAsLtFjsC/qKouTPLkJLec5Lork1w5xX4AgC0Lv51oVZ2V5M+S/Gx337THtqv5PodwiHg7UU7F24lOY/i3E62qL0ny1iQ37BVvAGA6izwLvZK8Mcnx7n79dCMBAHtZ5Aj8GUm+J8mzqur9sz/Pm2guAOAU5n4SW3f/RZLVfYADAA4xr8QGAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAk7wb2X5dcskl2djYOMhdAsBKcgQOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0NqyBzhdqmrZIwDAaeMIHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwoIUDXlVHqup9VfX7UwwEAOxtiiPwq5Icn+B2AIB9WijgVXV+km9Pcu004wAA+7HoEfgvJXl1kn/bbYOqurKqNqpqY3Nzc8HdAQDJAgGvqucnuae7j51qu+6+prvXu3v96NGj8+4OANhmkSPwZyR5QVV9PMlbkjyrqn5zkqkAgFOaO+Dd/ZruPr+7L0zyoiR/3N0vmWwyAGBXfg8cAAa0NsWNdPefJvnTKW4LANibI3AAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQJP8Hvhh1N3LHgGYUFUtewQ4VByBA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGihgFfVo6vqxqr6SFUdr6qnTzUYALC7tQU//5eTvLO7/3tVnZHkzAlmAgD2MHfAq+pRSZ6Z5KVJ0t33Jrl3mrEAgFNZ5C70i5JsJvmNqnpfVV1bVY/YuVFVXVlVG1W1sbm5ucDuAID7LRLwtSRPSfKG7n5ykn9KcvXOjbr7mu5e7+71o0ePLrA7AOB+iwT8RJIT3X3L7PyN2Qo6AHCazR3w7v77JHdV1RNmF12a5MOTTAUAnNKiz0J/WZIbZs9A/1iS7118JABgLwsFvLvfn2R9olkAgH3ySmwAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAa06CuxkaSqlj0Ch1x3L3sEYMU4AgeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxobdkDAOxHdy97BB6kg/w3q6oD29dh4QgcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADCghQJeVa+sqtuq6kNV9eaqethUgwEAu5s74FV1XpKXJ1nv7iclOZLkRVMNBgDsbtG70NeSPLyq1pKcmeSTi48EAOxl7oB39yeS/EKSO5PcneQfuvtdO7erqiuraqOqNjY3N+efFAD4okXuQj87yeVJHpfk3CSPqKqX7Nyuu6/p7vXuXj969Oj8kwIAX7TIXejPTnJHd292978muSnJN00zFgBwKosE/M4kT6uqM2vrndQvTXJ8mrEAgFNZ5DHwW5LcmOTWJB+c3dY1E80FAJzC2iKf3N0/leSnJpoFANgnr8QGAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGih3wNnS3cvewSAQ2frRTo5XRyBA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADCgtWUPsAqqatkjcMh197JHGJ7vs2kc5NfiQe7rofj14QgcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABrRnwKvqTVV1T1V9aNtlj6mqm6vqo7OPZ5/eMQGA7fZzBH5dkst2XHZ1knd398VJ3j07DwAckD0D3t3vSfKZHRdfnuT62enrk7xw4rkAgFOY9zHwL+3uu5Nk9vGxu21YVVdW1UZVbWxubs65OwBgu9P+JLbuvqa717t7/ejRo6d7dwDwkDBvwD9VVV+eJLOP90w3EgCwl3kD/o4kV8xOX5Hk7dOMAwDsx35+jezNSd6b5AlVdaKqvj/J65I8p6o+muQ5s/MAwAFZ22uD7n7xLlddOvEsAMA+eSU2ABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADqu4+uJ1VbSb5uwf5aeck+fRpGOcwWNW1req6ktVd26quK1ndta3qupLVXdu86/rK7n7Au4EdaMDnUVUb3b2+7DlOh1Vd26quK1ndta3qupLVXduqritZ3bVNvS53oQPAgAQcAAY0QsCvWfYAp9Gqrm1V15Ws7tpWdV3J6q5tVdeVrO7aJl3XoX8MHAB4oBGOwAGAHQQcAAZ0qANeVZdV1d9U1e1VdfWy55lCVV1QVX9SVcer6raqumrZM02tqo5U1fuq6veXPctUqurRVXVjVX1k9m/39GXPNJWqeuXsa/FDVfXmqnrYsmeaR1W9qaruqaoPbbvsMVV1c1V9dPbx7GXOOK9d1vbzs6/Hv66q362qRy9zxnmcbF3brvvRquqqOmcZsy1qt7VV1ctmXbutqv7XIvs4tAGvqiNJfjXJc5M8McmLq+qJy51qEvcleVV3f22SpyX54RVZ13ZXJTm+7CEm9stJ3tndX5Pk67Mi66uq85K8PMl6dz8pyZEkL1ruVHO7LsllOy67Osm7u/viJO+enR/RdXng2m5O8qTu/rokf5vkNQc91ASuywPXlaq6IMlzktx50ANN6LrsWFtVfWuSy5N8XXf/lyS/sMgODm3Akzw1ye3d/bHuvjfJW7K18KF1993dfevs9OezFYLzljvVdKrq/CTfnuTaZc8ylap6VJJnJnljknT3vd392eVONam1JA+vqrUkZyb55JLnmUt3vyfJZ3ZcfHmS62enr0/ywgMdaiInW1t3v6u775ud/b9Jzj/wwRa0y79ZkvxiklcnGfZZ1rus7QeTvK67/2W2zT2L7OMwB/y8JHdtO38iKxS6JKmqC5M8Ockty51kUr+UrW+8f1v2IBO6KMlmkt+YPTRwbVU9YtlDTaG7P5Gto4A7k9yd5B+6+13LnWpSX9rddydb/3lO8tglz3O6fF+SP1z2EFOoqhck+UR3f2DZs5wGj0/yzVV1S1X9WVV94yI3dpgDXie5bNj/je1UVWcleWuSV3T355Y9zxSq6vlJ7unuY8ueZWJrSZ6S5A3d/eQk/5Rx74r9D2aPCV+e5HFJzk3yiKp6yXKn4sGoqp/M1kNzNyx7lkVV1ZlJfjLJa5c9y2myluTsbD18+mNJfruqTta6fTnMAT+R5IJt58/PoHft7VRVX5KteN/Q3Tcte54JPSPJC6rq49l6yONZVfWbyx1pEieSnOju++8puTFbQV8Fz05yR3dvdve/JrkpyTcteaYpfaqqvjxJZh8XusvysKmqK5I8P8l392q8qMdXZes/kx+Y/Rw5P8mtVfVlS51qOieS3NRb/jJb91TO/SS9wxzwv0pycVU9rqrOyNYTa96x5JkWNvvf1huTHO/u1y97nil192u6+/zuvjBb/15/3N3DH811998nuauqnjC76NIkH17iSFO6M8nTqurM2dfmpVmRJ+jNvCPJFbPTVyR5+xJnmVRVXZbkx5O8oLv//7LnmUJ3f7C7H9vdF85+jpxI8pTZ9+AqeFuSZyVJVT0+yRlZ4F3XDm3AZ0/O+JEkf5StHyi/3d23LXeqSTwjyfdk6+j0/bM/z1v2UOzpZUluqKq/TvINSX5uyfNMYnavwo1Jbk3ywWz9TBjyZSyr6s1J3pvkCVV1oqq+P8nrkjynqj6arWc1v26ZM85rl7X9SpJHJrl59nPk15Y65Bx2WddK2GVtb0py0exXy96S5IpF7jnxUqoAMKBDewQOAOxOwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKB/B24h+wUcnnY9AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
\n", + "To make choices between different possible plans in a certain situation in a given environment, an agent must have _preference_ between the possible outcomes of the various plans.\n", + "
\n", + "__Utility theory__ is used to represent and reason with preferences.\n", + "The agent prefers states with a higher _utility_.\n", + "While constructing multi-agent systems, one major element in the design is the mechanism the agents use for making decisions about which actions to adopt in order to achieve their goals.\n", + "What is usually required is a mechanism which ensures that the actions adopted lead to benefits for both individual agents, and the community of which they are part.\n", + "The utility of a state is _relative_ to an agent.\n", + "
\n", + "Preferences, as expressed by utilities, are combined with probabilities in the general theory of rational decisions called __decision theory__.\n", + "
\n", + "An agent is said to be _rational_ if and only if it chooses the action that yields the highest expected utility, averaged over all the possible outcomes of the action." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we'll see how a decision-theoretic agent is implemented in the module." + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
def DTAgentProgram(belief_state):\n",
+ " """A decision-theoretic agent. [Figure 13.1]"""\n",
+ " def program(percept):\n",
+ " belief_state.observe(program.action, percept)\n",
+ " program.action = argmax(belief_state.actions(),\n",
+ " key=belief_state.expected_outcome_utility)\n",
+ " return program.action\n",
+ " program.action = None\n",
+ " return program\n",
+ "\n", + "It encapsulates a function `program` that takes in an observation or a `percept`, updates its `belief_state` and returns the action that maximizes the `expected_outcome_utility`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## INFORMATION GATHERING AGENT\n", + "Before we discuss what an information gathering agent is, we'll need to know what decision networks are.\n", + "For an agent in an environment, a decision network represents information about the agent's current state, its possible actions, the state that will result from the agent's action, and the utility of that state.\n", + "Decision networks have three primary kinds of nodes which are:\n", + "1. __Chance nodes__: These represent random variables, just like in Bayesian networks.\n", + "2. __Decision nodes__: These represent points where the decision-makes has a choice between different actions and the decision maker tries to find the optimal decision at these nodes with regard to the cost, safety and resulting utility.\n", + "3. __Utility nodes__: These represent the agent's utility function.\n", + "A description of the agent's utility as a function is associated with a utility node.\n", + "
\n", + "
\n", + "To evaluate a decision network, we do the following:\n", + "1. Initialize the evidence variables according to the current state.\n", + "2. Calculate posterior probabilities for each possible value of the decision node and calculate the utility resulting from that action.\n", + "3. Return the action with the highest utility.\n", + "
\n", + "Let's have a look at the implementation of the `DecisionNetwork` class." + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
class DecisionNetwork(BayesNet):\n",
+ " """An abstract class for a decision network as a wrapper for a BayesNet.\n",
+ " Represents an agent's current state, its possible actions, reachable states\n",
+ " and utilities of those states."""\n",
+ "\n",
+ " def __init__(self, action, infer):\n",
+ " """action: a single action node\n",
+ " infer: the preferred method to carry out inference on the given BayesNet"""\n",
+ " super(DecisionNetwork, self).__init__()\n",
+ " self.action = action\n",
+ " self.infer = infer\n",
+ "\n",
+ " def best_action(self):\n",
+ " """Return the best action in the network"""\n",
+ " return self.action\n",
+ "\n",
+ " def get_utility(self, action, state):\n",
+ " """Return the utility for a particular action and state in the network"""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def get_expected_utility(self, action, evidence):\n",
+ " """Compute the expected utility given an action and evidence"""\n",
+ " u = 0.0\n",
+ " prob_dist = self.infer(action, evidence, self).prob\n",
+ " for item, _ in prob_dist.items():\n",
+ " u += prob_dist[item] * self.get_utility(action, item)\n",
+ "\n",
+ " return u\n",
+ "\n", + "`best_action` returns the best action in the network.\n", + "
\n", + "`get_utility` is an abstract method which is supposed to return the utility of a particular action and state in the network.\n", + "
\n", + "`get_expected_utility` computes the expected utility, given an action and evidence.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before we proceed, we need to know a few more terms.\n", + "
\n", + "Having __perfect information__ refers to a state of being fully aware of the current state, the cost functions and the outcomes of actions.\n", + "This in turn allows an agent to find the exact utility value of each state.\n", + "If an agent has perfect information about the environment, maximum expected utility calculations are exact and can be computed with absolute certainty.\n", + "
\n", + "In decision theory, the __value of perfect information__ (VPI) is the price that an agent would be willing to pay in order to gain access to _perfect information_.\n", + "VPI calculations are extensively used to calculate expected utilities for nodes in a decision network.\n", + "
\n", + "For a random variable $E_j$ whose value is currently unknown, the value of discovering $E_j$, given current information $e$ must average over all possible values $e_{jk}$ that we might discover for $E_j$, using our _current_ beliefs about its value.\n", + "The VPI of $E_j$ is then given by:\n", + "
\n", + "
\n", + "$$VPI_e(E_j) = \\left(\\sum_{k}P(E_j=e_{jk}\\ |\\ e) EU(\\alpha_{e_{jk}}\\ |\\ e, E_j=e_{jk})\\right) - EU(\\alpha\\ |\\ e)$$\n", + "
\n", + "VPI is _non-negative_, _non-additive_ and _order-indepentent_." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An information gathering agent is an agent with certain properties that explores decision networks as and when required with heuristics driven by VPI calculations of nodes.\n", + "A sensible agent should ask questions in a reasonable order, should avoid asking irrelevant questions, should take into account the importance of each piece of information in relation to its cost and should stop asking questions when that is appropriate.\n", + "_VPI_ is used as the primary heuristic to consider all these points in an information gathering agent as the agent ultimately wants to maximize the utility and needs to find the optimal cost and extent of finding the required information.\n", + "
\n", + "As an overview, an information gathering agent works by repeatedly selecting the observations with the highest information value, until the cost of the next observation is greater than its expected benefit.\n", + "
\n", + "The `InformationGatheringAgent` class is an abstract class that inherits from `Agent` and works on the principles discussed above.\n", + "Let's have a look.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "
class InformationGatheringAgent(Agent):\n",
+ " """A simple information gathering agent. The agent works by repeatedly selecting\n",
+ " the observation with the highest information value, until the cost of the next\n",
+ " observation is greater than its expected benefit. [Figure 16.9]"""\n",
+ "\n",
+ " def __init__(self, decnet, infer, initial_evidence=None):\n",
+ " """decnet: a decision network\n",
+ " infer: the preferred method to carry out inference on the given decision network\n",
+ " initial_evidence: initial evidence"""\n",
+ " self.decnet = decnet\n",
+ " self.infer = infer\n",
+ " self.observation = initial_evidence or []\n",
+ " self.variables = self.decnet.nodes\n",
+ "\n",
+ " def integrate_percept(self, percept):\n",
+ " """Integrate the given percept into the decision network"""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def execute(self, percept):\n",
+ " """Execute the information gathering algorithm"""\n",
+ " self.observation = self.integrate_percept(percept)\n",
+ " vpis = self.vpi_cost_ratio(self.variables)\n",
+ " j = argmax(vpis)\n",
+ " variable = self.variables[j]\n",
+ "\n",
+ " if self.vpi(variable) > self.cost(variable):\n",
+ " return self.request(variable)\n",
+ "\n",
+ " return self.decnet.best_action()\n",
+ "\n",
+ " def request(self, variable):\n",
+ " """Return the value of the given random variable as the next percept"""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def cost(self, var):\n",
+ " """Return the cost of obtaining evidence through tests, consultants or questions"""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def vpi_cost_ratio(self, variables):\n",
+ " """Return the VPI to cost ratio for the given variables"""\n",
+ " v_by_c = []\n",
+ " for var in variables:\n",
+ " v_by_c.append(self.vpi(var) / self.cost(var))\n",
+ " return v_by_c\n",
+ "\n",
+ " def vpi(self, variable):\n",
+ " """Return VPI for a given variable"""\n",
+ " vpi = 0.0\n",
+ " prob_dist = self.infer(variable, self.observation, self.decnet).prob\n",
+ " for item, _ in prob_dist.items():\n",
+ " post_prob = prob_dist[item]\n",
+ " new_observation = list(self.observation)\n",
+ " new_observation.append(item)\n",
+ " expected_utility = self.decnet.get_expected_utility(variable, new_observation)\n",
+ " vpi += post_prob * expected_utility\n",
+ "\n",
+ " vpi -= self.decnet.get_expected_utility(variable, self.observation)\n",
+ " return vpi\n",
+ "\n", + "The `request` method returns the value of the given random variable as the next percept.\n", + "
\n", + "The `vpi_cost_ratio` method returns a list of VPI divided by cost for each variable in the `variables` list provided to it.\n", + "
\n", + "The `vpi` method calculates the VPI for a given variable\n", + "
\n", + "And finally, the `execute` method executes the general information gathering algorithm, as described in __figure 16.9__ in the book.\n", + "
\n", + "Our agent implements a form of information gathering that is called __myopic__ as the VPI formula is used shortsightedly here.\n", + "It calculates the value of information as if only a single evidence variable will be acquired.\n", + "This is similar to greedy search, where we do not look at the bigger picture and aim for local optimizations to hopefully reach the global optimum.\n", + "This often works well in practice but a myopic agent might hastily take an action when it would have been better to request more variables before taking an action.\n", + "A _conditional plan_, on the other hand might work better for some scenarios.\n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With this we conclude this notebook." + ] } ], "metadata": { @@ -37,9 +6528,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.1" + "version": "3.6.9" } }, "nbformat": 4, - "nbformat_minor": 0 + "nbformat_minor": 2 } diff --git a/probability.py b/probability.py index b73ddfb09..e1e77d224 100644 --- a/probability.py +++ b/probability.py @@ -1,32 +1,32 @@ -"""Probability models. (Chapter 13-15) -""" +"""Probability models (Chapter 13-15)""" -from utils import * # noqa -from logic import extend - -import random from collections import defaultdict from functools import reduce -# ______________________________________________________________________________ +from agents import Agent +from utils import * def DTAgentProgram(belief_state): - "A decision-theoretic agent. [Fig. 13.1]" + """ + [Figure 13.1] + A decision-theoretic agent. + """ + def program(percept): belief_state.observe(program.action, percept) - program.action = argmax(belief_state.actions(), - belief_state.expected_outcome_utility) + program.action = max(belief_state.actions(), key=belief_state.expected_outcome_utility) return program.action + program.action = None return program + # ______________________________________________________________________________ class ProbDist: - - """A discrete probability distribution. You name the random variable + """A discrete probability distribution. You name the random variable in the constructor, then assign and query probability of values. >>> P = ProbDist('Flip'); P['H'], P['T'] = 0.25, 0.75; P['H'] 0.25 @@ -35,26 +35,26 @@ class ProbDist: (0.125, 0.375, 0.5) """ - def __init__(self, varname='?', freqs=None): - """If freqs is given, it is a dictionary of value: frequency pairs, - and the ProbDist then is normalized.""" + def __init__(self, var_name='?', freq=None): + """If freq is given, it is a dictionary of values - frequency pairs, + then ProbDist is normalized.""" self.prob = {} - self.varname = varname + self.var_name = var_name self.values = [] - if freqs: - for (v, p) in list(freqs.items()): + if freq: + for (v, p) in freq.items(): self[v] = p self.normalize() def __getitem__(self, val): - "Given a value, return P(value)." + """Given a value, return P(value).""" try: return self.prob[val] except KeyError: return 0 def __setitem__(self, val, p): - "Set P(val) = p." + """Set P(val) = p.""" if val not in self.values: self.values.append(val) self.prob[val] = p @@ -62,29 +62,23 @@ def __setitem__(self, val, p): def normalize(self): """Make sure the probabilities of all values sum to 1. Returns the normalized distribution. - Raises a ZeroDivisionError if the sum of the values is 0. - >>> P = ProbDist('Flip'); P['H'], P['T'] = 35, 65 - >>> P = P.normalize() - >>> print '%5.3f %5.3f' % (P.prob['H'], P.prob['T']) - 0.350 0.650 - """ - total = float(sum(self.prob.values())) - if not (1.0-epsilon < total < 1.0+epsilon): + Raises a ZeroDivisionError if the sum of the values is 0.""" + total = sum(self.prob.values()) + if not np.isclose(total, 1.0): for val in self.prob: self.prob[val] /= total return self - def show_approx(self, numfmt='%.3g'): + def show_approx(self, numfmt='{:.3g}'): """Show the probabilities rounded and sorted by key, for the sake of portable doctests.""" - return ', '.join([('%s: ' + numfmt) % (v, p) - for (v, p) in sorted(self.prob.items())]) + return ', '.join([('{}: ' + numfmt).format(v, p) for (v, p) in sorted(self.prob.items())]) -epsilon = 0.001 + def __repr__(self): + return "P({})".format(self.var_name) class JointProbDist(ProbDist): - """A discrete probability distribute over a set of variables. >>> P = JointProbDist(['X', 'Y']); P[1, 1] = 0.25 >>> P[1, 1] @@ -99,12 +93,12 @@ def __init__(self, variables): self.vals = defaultdict(list) def __getitem__(self, values): - "Given a tuple or dict of values, return P(values)." + """Given a tuple or dict of values, return P(values).""" values = event_values(values, self.variables) return ProbDist.__getitem__(self, values) def __setitem__(self, values, p): - """Set P(values) = p. Values can be a tuple or a dict; it must + """Set P(values) = p. Values can be a tuple or a dict; it must have a value for each of the variables in the joint. Also keep track of the values we have seen so far for each variable.""" values = event_values(values, self.variables) @@ -114,15 +108,15 @@ def __setitem__(self, values, p): self.vals[var].append(val) def values(self, var): - "Return the set of possible values for a variable." + """Return the set of possible values for a variable.""" return self.vals[var] def __repr__(self): - return "P(%s)" % self.variables + return "P({})".format(self.variables) def event_values(event, variables): - """Return a tuple of the values of variables variables in event. + """Return a tuple of the values of variables in event. >>> event_values ({'A': 10, 'B': 9, 'C': 8}, ['C', 'A']) (8, 10) >>> event_values ((1, 2), ['C', 'A']) @@ -133,12 +127,15 @@ def event_values(event, variables): else: return tuple([event[var] for var in variables]) + # ______________________________________________________________________________ def enumerate_joint_ask(X, e, P): - """Return a probability distribution over the values of the variable X, - given the {var:val} observations e, in the JointProbDist P. [Section 13.3] + """ + [Section 13.3] + Return a probability distribution over the values of the variable X, + given the {var:val} observations e, in the JointProbDist P. >>> P = JointProbDist(['X', 'Y']) >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[2,1] = 0.125 >>> enumerate_joint_ask('X', dict(Y=1), P).show_approx() @@ -158,20 +155,20 @@ def enumerate_joint(variables, e, P): if not variables: return P[e] Y, rest = variables[0], variables[1:] - return sum([enumerate_joint(rest, extend(e, Y, y), P) - for y in P.values(Y)]) + return sum([enumerate_joint(rest, extend(e, Y, y), P) for y in P.values(Y)]) + # ______________________________________________________________________________ class BayesNet: + """Bayesian network containing only boolean-variable nodes.""" - "Bayesian network containing only boolean-variable nodes." - - def __init__(self, node_specs=[]): - "nodes must be ordered with parents before children." + def __init__(self, node_specs=None): + """Nodes must be ordered with parents before children.""" self.nodes = [] self.variables = [] + node_specs = node_specs or [] for node_spec in node_specs: self.add(node_spec) @@ -180,7 +177,7 @@ def add(self, node_spec): net, and its variable must not.""" node = BayesNode(*node_spec) assert node.variable not in self.variables - assert every(lambda parent: parent in self.variables, node.parents) + assert all((parent in self.variables) for parent in node.parents) self.nodes.append(node) self.variables.append(node.variable) for parent in node.parents: @@ -193,24 +190,115 @@ def variable_node(self, var): for n in self.nodes: if n.variable == var: return n - raise Exception("No such variable: %s" % var) + raise Exception("No such variable: {}".format(var)) def variable_values(self, var): - "Return the domain of var." + """Return the domain of var.""" return [True, False] def __repr__(self): - return 'BayesNet(%r)' % self.nodes + return 'BayesNet({0!r})'.format(self.nodes) -class BayesNode: +class DecisionNetwork(BayesNet): + """An abstract class for a decision network as a wrapper for a BayesNet. + Represents an agent's current state, its possible actions, reachable states + and utilities of those states.""" + + def __init__(self, action, infer): + """action: a single action node + infer: the preferred method to carry out inference on the given BayesNet""" + super(DecisionNetwork, self).__init__() + self.action = action + self.infer = infer + + def best_action(self): + """Return the best action in the network""" + return self.action + + def get_utility(self, action, state): + """Return the utility for a particular action and state in the network""" + raise NotImplementedError + + def get_expected_utility(self, action, evidence): + """Compute the expected utility given an action and evidence""" + u = 0.0 + prob_dist = self.infer(action, evidence, self).prob + for item, _ in prob_dist.items(): + u += prob_dist[item] * self.get_utility(action, item) + return u + + +class InformationGatheringAgent(Agent): + """ + [Figure 16.9] + A simple information gathering agent. The agent works by repeatedly selecting + the observation with the highest information value, until the cost of the next + observation is greater than its expected benefit.""" + + def __init__(self, decnet, infer, initial_evidence=None): + """decnet: a decision network + infer: the preferred method to carry out inference on the given decision network + initial_evidence: initial evidence""" + self.decnet = decnet + self.infer = infer + self.observation = initial_evidence or [] + self.variables = self.decnet.nodes + + def integrate_percept(self, percept): + """Integrate the given percept into the decision network""" + raise NotImplementedError + + def execute(self, percept): + """Execute the information gathering algorithm""" + self.observation = self.integrate_percept(percept) + vpis = self.vpi_cost_ratio(self.variables) + j = max(vpis) + variable = self.variables[j] + + if self.vpi(variable) > self.cost(variable): + return self.request(variable) + + return self.decnet.best_action() + + def request(self, variable): + """Return the value of the given random variable as the next percept""" + raise NotImplementedError + + def cost(self, var): + """Return the cost of obtaining evidence through tests, consultants or questions""" + raise NotImplementedError + + def vpi_cost_ratio(self, variables): + """Return the VPI to cost ratio for the given variables""" + v_by_c = [] + for var in variables: + v_by_c.append(self.vpi(var) / self.cost(var)) + return v_by_c + + def vpi(self, variable): + """Return VPI for a given variable""" + vpi = 0.0 + prob_dist = self.infer(variable, self.observation, self.decnet).prob + for item, _ in prob_dist.items(): + post_prob = prob_dist[item] + new_observation = list(self.observation) + new_observation.append(item) + expected_utility = self.decnet.get_expected_utility(variable, new_observation) + vpi += post_prob * expected_utility + + vpi -= self.decnet.get_expected_utility(variable, self.observation) + return vpi + + +class BayesNode: """A conditional probability distribution for a boolean variable, P(X | parents). Part of a BayesNet.""" def __init__(self, X, parents, cpt): """X is a variable name, and parents a sequence of variable - names or a space-separated string. cpt, the conditional + names or a space-separated string. cpt, the conditional probability table, takes one of these forms: * A number, the unconditional probability P(X=true). You can @@ -241,12 +329,12 @@ def __init__(self, X, parents, cpt): elif isinstance(cpt, dict): # one parent, 1-tuple if cpt and isinstance(list(cpt.keys())[0], bool): - cpt = dict(((v,), p) for v, p in list(cpt.items())) + cpt = {(v,): p for v, p in cpt.items()} assert isinstance(cpt, dict) - for vs, p in list(cpt.items()): + for vs, p in cpt.items(): assert isinstance(vs, tuple) and len(vs) == len(parents) - assert every(lambda v: isinstance(v, bool), vs) + assert all(isinstance(v, bool) for v in vs) assert 0 <= p <= 1 self.variable = X @@ -264,7 +352,7 @@ def p(self, value, event): 0.375""" assert isinstance(value, bool) ptrue = self.cpt[event_values(event, self.parents)] - return (ptrue if value else 1 - ptrue) + return ptrue if value else 1 - ptrue def sample(self, event): """Sample from the distribution for this variable conditioned @@ -276,25 +364,27 @@ def sample(self, event): def __repr__(self): return repr((self.variable, ' '.join(self.parents))) -# Burglary example [Fig. 14.2] + +# Burglary example [Figure 14.2] T, F = True, False -burglary = BayesNet([ - ('Burglary', '', 0.001), - ('Earthquake', '', 0.002), - ('Alarm', 'Burglary Earthquake', - {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}), - ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}), - ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01}) -]) +burglary = BayesNet([('Burglary', '', 0.001), + ('Earthquake', '', 0.002), + ('Alarm', 'Burglary Earthquake', + {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}), + ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}), + ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01})]) + # ______________________________________________________________________________ def enumeration_ask(X, e, bn): - """Return the conditional probability distribution of variable X - given evidence e, from BayesNet bn. [Fig. 14.9] + """ + [Figure 14.9] + Return the conditional probability distribution of variable X + given evidence e, from BayesNet bn. >>> enumeration_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary ... ).show_approx() 'False: 0.716, True: 0.284'""" @@ -320,11 +410,14 @@ def enumerate_all(variables, e, bn): return sum(Ynode.p(y, e) * enumerate_all(rest, extend(e, Y, y), bn) for y in bn.variable_values(Y)) + # ______________________________________________________________________________ def elimination_ask(X, e, bn): - """Compute bn's P(X|e) by variable elimination. [Fig. 14.11] + """ + [Figure 14.11] + Compute bn's P(X|e) by variable elimination. >>> elimination_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary ... ).show_approx() 'False: 0.716, True: 0.284'""" @@ -338,7 +431,7 @@ def elimination_ask(X, e, bn): def is_hidden(var, X, e): - "Is var a hidden variable when querying P(X|e)?" + """Is var a hidden variable when querying P(X|e)?""" return var != X and var not in e @@ -348,8 +441,8 @@ def make_factor(var, e, bn): is the pointwise product of these factors for bn's variables.""" node = bn.variable_node(var) variables = [X for X in [var] + node.parents if X not in e] - cpt = dict((event_values(e1, variables), node.p(e1[var], e1)) - for e1 in all_events(variables, bn, e)) + cpt = {event_values(e1, variables): node.p(e1[var], e1) + for e1 in all_events(variables, bn, e)} return Factor(variables, cpt) @@ -358,7 +451,7 @@ def pointwise_product(factors, bn): def sum_out(var, factors, bn): - "Eliminate var from all factors by summing over its values." + """Eliminate var from all factors by summing over its values.""" result, var_factors = [], [] for f in factors: (var_factors if var in f.variables else result).append(f) @@ -367,42 +460,37 @@ def sum_out(var, factors, bn): class Factor: - - "A factor in a joint distribution." + """A factor in a joint distribution.""" def __init__(self, variables, cpt): self.variables = variables self.cpt = cpt def pointwise_product(self, other, bn): - "Multiply two factors, combining their variables." + """Multiply two factors, combining their variables.""" variables = list(set(self.variables) | set(other.variables)) - cpt = dict((event_values(e, variables), self.p(e) * other.p(e)) - for e in all_events(variables, bn, {})) + cpt = {event_values(e, variables): self.p(e) * other.p(e) for e in all_events(variables, bn, {})} return Factor(variables, cpt) def sum_out(self, var, bn): - "Make a factor eliminating var by summing over its values." + """Make a factor eliminating var by summing over its values.""" variables = [X for X in self.variables if X != var] - cpt = dict((event_values(e, variables), - sum(self.p(extend(e, var, val)) - for val in bn.variable_values(var))) - for e in all_events(variables, bn, {})) + cpt = {event_values(e, variables): sum(self.p(extend(e, var, val)) for val in bn.variable_values(var)) + for e in all_events(variables, bn, {})} return Factor(variables, cpt) def normalize(self): - "Return my probabilities; must be down to one variable." + """Return my probabilities; must be down to one variable.""" assert len(self.variables) == 1 - return ProbDist(self.variables[0], - dict((k, v) for ((k,), v) in list(self.cpt.items()))) + return ProbDist(self.variables[0], {k: v for ((k,), v) in self.cpt.items()}) def p(self, e): - "Look up my value tabulated for e." + """Look up my value tabulated for e.""" return self.cpt[event_values(e, self.variables)] def all_events(variables, bn, e): - "Yield every way of extending e with values for all variables." + """Yield every way of extending e with values for all variables.""" if not variables: yield e else: @@ -411,34 +499,42 @@ def all_events(variables, bn, e): for x in bn.variable_values(X): yield extend(e1, X, x) + # ______________________________________________________________________________ -# Fig. 14.12a: sprinkler network +# [Figure 14.12a]: sprinkler network + + +sprinkler = BayesNet([('Cloudy', '', 0.5), + ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}), + ('Rain', 'Cloudy', {T: 0.80, F: 0.20}), + ('WetGrass', 'Sprinkler Rain', + {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})]) -sprinkler = BayesNet([ - ('Cloudy', '', 0.5), - ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}), - ('Rain', 'Cloudy', {T: 0.80, F: 0.20}), - ('WetGrass', 'Sprinkler Rain', - {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})]) # ______________________________________________________________________________ def prior_sample(bn): - """Randomly sample from bn's full joint distribution. The result - is a {variable: value} dict. [Fig. 14.13]""" + """ + [Figure 14.13] + Randomly sample from bn's full joint distribution. + The result is a {variable: value} dict. + """ event = {} for node in bn.nodes: event[node.variable] = node.sample(event) return event + # _________________________________________________________________________ -def rejection_sampling(X, e, bn, N): - """Estimate the probability distribution of variable X given - evidence e in BayesNet bn, using N samples. [Fig. 14.14] +def rejection_sampling(X, e, bn, N=10000): + """ + [Figure 14.14] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn, using N samples. Raises a ZeroDivisionError if all the N samples are rejected, i.e., inconsistent with e. >>> random.seed(47) @@ -446,44 +542,47 @@ def rejection_sampling(X, e, bn, N): ... burglary, 10000).show_approx() 'False: 0.7, True: 0.3' """ - counts = dict((x, 0) - for x in bn.variable_values(X)) # bold N in Fig. 14.14 + counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.14] for j in range(N): - sample = prior_sample(bn) # boldface x in Fig. 14.14 + sample = prior_sample(bn) # boldface x in [Figure 14.14] if consistent_with(sample, e): counts[sample[X]] += 1 return ProbDist(X, counts) def consistent_with(event, evidence): - "Is event consistent with the given evidence?" - return all(evidence.get(k, v) == v - for k, v in list(event.items())) + """Is event consistent with the given evidence?""" + return all(evidence.get(k, v) == v for k, v in event.items()) + # _________________________________________________________________________ -def likelihood_weighting(X, e, bn, N): - """Estimate the probability distribution of variable X given - evidence e in BayesNet bn. [Fig. 14.15] +def likelihood_weighting(X, e, bn, N=10000): + """ + [Figure 14.15] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn. >>> random.seed(1017) >>> likelihood_weighting('Burglary', dict(JohnCalls=T, MaryCalls=T), ... burglary, 10000).show_approx() 'False: 0.702, True: 0.298' """ - W = dict((x, 0) for x in bn.variable_values(X)) + W = {x: 0 for x in bn.variable_values(X)} for j in range(N): - sample, weight = weighted_sample(bn, e) # boldface x, w in Fig. 14.15 + sample, weight = weighted_sample(bn, e) # boldface x, w in [Figure 14.15] W[sample[X]] += weight return ProbDist(X, W) def weighted_sample(bn, e): - """Sample an event from bn that's consistent with the evidence e; + """ + Sample an event from bn that's consistent with the evidence e; return the event and its weight, the likelihood that the event - accords to the evidence.""" + accords to the evidence. + """ w = 1 - event = dict(e) # boldface x in Fig. 14.15 + event = dict(e) # boldface x in [Figure 14.15] for node in bn.nodes: Xi = node.variable if Xi in e: @@ -492,21 +591,16 @@ def weighted_sample(bn, e): event[Xi] = node.sample(event) return event, w + # _________________________________________________________________________ -def gibbs_ask(X, e, bn, N): - """[Fig. 14.16] - >>> random.seed(1017) - >>> gibbs_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary, 1000 - ... ).show_approx() - 'False: 0.738, True: 0.262' - """ +def gibbs_ask(X, e, bn, N=1000): + """[Figure 14.16]""" assert X not in e, "Query variable must be distinct from evidence" - counts = dict((x, 0) - for x in bn.variable_values(X)) # bold N in Fig. 14.16 + counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.16] Z = [var for var in bn.variables if var not in e] - state = dict(e) # boldface x in Fig. 14.16 + state = dict(e) # boldface x in [Figure 14.16] for Zi in Z: state[Zi] = random.choice(bn.variable_values(Zi)) for j in range(N): @@ -525,26 +619,22 @@ def markov_blanket_sample(X, e, bn): Q = ProbDist(X) for xi in bn.variable_values(X): ei = extend(e, X, xi) - # [Equation 14.12:] - Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei) - for Yj in Xnode.children) + # [Equation 14.12] + Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei) for Yj in Xnode.children) # (assuming a Boolean variable here) return probability(Q.normalize()[True]) + # _________________________________________________________________________ class HiddenMarkovModel: + """A Hidden markov model which takes Transition model and Sensor model as inputs""" - """ A Hidden markov model which takes Transition model and Sensor model as inputs""" - - def __init__(self, transition_model, sensor_model, prior= [0.5, 0.5]): + def __init__(self, transition_model, sensor_model, prior=None): self.transition_model = transition_model self.sensor_model = sensor_model - self.prior = prior - - def transition_model(self): - return self.transition_model + self.prior = prior or [0.5, 0.5] def sensor_dist(self, ev): if ev is True: @@ -558,140 +648,146 @@ def forward(HMM, fv, ev): scalar_vector_product(fv[1], HMM.transition_model[1])) sensor_dist = HMM.sensor_dist(ev) - return(normalize(element_wise_product(sensor_dist, prediction))) + return normalize(element_wise_product(sensor_dist, prediction)) def backward(HMM, b, ev): sensor_dist = HMM.sensor_dist(ev) prediction = element_wise_product(sensor_dist, b) - return(normalize(vector_add(scalar_vector_product(prediction[0], HMM.transition_model[0]), - scalar_vector_product(prediction[1], HMM.transition_model[1])))) + return normalize(vector_add(scalar_vector_product(prediction[0], HMM.transition_model[0]), + scalar_vector_product(prediction[1], HMM.transition_model[1]))) -def forward_backward(HMM, ev, prior): - """[Fig. 15.4] +def forward_backward(HMM, ev): + """ + [Figure 15.4] Forward-Backward algorithm for smoothing. Computes posterior probabilities of a sequence of states given a sequence of observations. - - umbrella_evidence = [T, T, F, T, T] - umbrella_prior = [0.5, 0.5] - umbrella_transition = [[0.7, 0.3], [0.3, 0.7]] - umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]] - umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) - - >>> forward_backward(umbrellaHMM, umbrella_evidence, umbrella_prior) - [[0.6469, 0.3531], [0.8673, 0.1327], [0.8204, 0.1796], - [0.3075, 0.6925], [0.8204, 0.1796], [0.8673, 0.1327]] """ t = len(ev) ev.insert(0, None) # to make the code look similar to pseudo code - fv = [[0.0, 0.0] for i in range(len(ev))] + fv = [[0.0, 0.0] for _ in range(len(ev))] b = [1.0, 1.0] - bv = [b] # we don't need bv; but we will have a list of all backward messages here - sv = [[0, 0] for i in range(len(ev))] + sv = [[0, 0] for _ in range(len(ev))] - fv[0] = prior + fv[0] = HMM.prior for i in range(1, t + 1): fv[i] = forward(HMM, fv[i - 1], ev[i]) for i in range(t, -1, -1): sv[i - 1] = normalize(element_wise_product(fv[i], b)) b = backward(HMM, b, ev[i]) - bv.append(b) sv = sv[::-1] - return(sv) + return sv + + +def viterbi(HMM, ev): + """ + [Equation 15.11] + Viterbi algorithm to find the most likely sequence. Computes the best path and the + corresponding probabilities, given an HMM model and a sequence of observations. + """ + t = len(ev) + ev = ev.copy() + ev.insert(0, None) + + m = [[0.0, 0.0] for _ in range(len(ev) - 1)] + + # the recursion is initialized with m1 = forward(P(X0), e1) + m[0] = forward(HMM, HMM.prior, ev[1]) + # keep track of maximizing predecessors + backtracking_graph = [] + + for i in range(1, t): + m[i] = element_wise_product(HMM.sensor_dist(ev[i + 1]), + [max(element_wise_product(HMM.transition_model[0], m[i - 1])), + max(element_wise_product(HMM.transition_model[1], m[i - 1]))]) + backtracking_graph.append([np.argmax(element_wise_product(HMM.transition_model[0], m[i - 1])), + np.argmax(element_wise_product(HMM.transition_model[1], m[i - 1]))]) + + # computed probabilities + ml_probabilities = [0.0] * (len(ev) - 1) + # most likely sequence + ml_path = [True] * (len(ev) - 1) + + # the construction of the most likely sequence starts in the final state with the largest probability, and + # runs backwards; the algorithm needs to store for each xt its predecessor xt-1 maximizing its probability + i_max = np.argmax(m[-1]) + + for i in range(t - 1, -1, -1): + ml_probabilities[i] = m[i][i_max] + ml_path[i] = True if i_max == 0 else False + if i > 0: + i_max = backtracking_graph[i - 1][i_max] + + return ml_path, ml_probabilities + # _________________________________________________________________________ def fixed_lag_smoothing(e_t, HMM, d, ev, t): - """[Fig. 15.6] + """ + [Figure 15.6] Smoothing algorithm with a fixed time lag of 'd' steps. Online algorithm that outputs the new smoothed estimate if observation for new time step is given. - - umbrella_evidence = [T, T, F, T, T] - e_t = T - t = 4 - d = 3 - umbrella_transition = [[0.7, 0.3], [0.3, 0.7]] - umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]] - umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) - - >>> fixed_lag_smoothing(T, umbrellaHMM, d) """ ev.insert(0, None) T_model = HMM.transition_model f = HMM.prior B = [[1, 0], [0, 1]] - evidence = [] - evidence.append(e_t) - O_t = vector_to_diagonal(HMM.sensor_dist(e_t)) + O_t = np.diag(HMM.sensor_dist(e_t)) if t > d: f = forward(HMM, f, e_t) - O_tmd = vector_to_diagonal(HMM.sensor_dist(ev[t- d])) - B = matrix_multiplication(inverse_matrix(O_tmd), inverse_matrix(T_model), B, T_model, O_t) + O_tmd = np.diag(HMM.sensor_dist(ev[t - d])) + B = matrix_multiplication(np.linalg.inv(O_tmd), np.linalg.inv(T_model), B, T_model, O_t) else: B = matrix_multiplication(B, T_model, O_t) - t = t + 1 + t += 1 if t > d: # always returns a 1x2 matrix - return([normalize(i) for i in matrix_multiplication([f], B)][0]) + return [normalize(i) for i in matrix_multiplication([f], B)][0] else: return None + # _________________________________________________________________________ def particle_filtering(e, N, HMM): - """ - Particle filtering considering two states variables - N = 10 - umbrella_evidence = T - umbrella_prior = [0.5, 0.5] - umbrella_transition = [[0.7, 0.3], [0.3, 0.7]] - umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]] - umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) - - >>> particle_filtering(umbrella_evidence, N, umbrellaHMM) - ['A', 'A', 'A', 'B', 'A', 'A', 'B', 'A', 'A', 'A', 'B'] - - NOTE: Output is an probabilistic answer, therfore can vary - """ - s = [] + """Particle filtering considering two states variables.""" dist = [0.5, 0.5] - # State Initialization - s = ['A' if probability(dist[0]) else 'B' for i in range(N)] # Weight Initialization - w = [0 for i in range(N)] + w = [0 for _ in range(N)] # STEP 1 # Propagate one step using transition model given prior state dist = vector_add(scalar_vector_product(dist[0], HMM.transition_model[0]), scalar_vector_product(dist[1], HMM.transition_model[1])) # Assign state according to probability - s = ['A' if probability(dist[0]) else 'B' for i in range(N)] + s = ['A' if probability(dist[0]) else 'B' for _ in range(N)] w_tot = 0 # Calculate importance weight given evidence e for i in range(N): if s[i] == 'A': # P(U|A)*P(A) - w_i = HMM.sensor_dist(e)[0]*dist[0] + w_i = HMM.sensor_dist(e)[0] * dist[0] if s[i] == 'B': # P(U|B)*P(B) - w_i = HMM.sensor_dist(e)[1]*dist[1] + w_i = HMM.sensor_dist(e)[1] * dist[1] w[i] = w_i w_tot += w_i # Normalize all the weights for i in range(N): - w[i] = w[i]/w_tot + w[i] = w[i] / w_tot # Limit weights to 4 digits for i in range(N): @@ -699,55 +795,76 @@ def particle_filtering(e, N, HMM): # STEP 2 s = weighted_sample_with_replacement(N, s, w) + return s -def weighted_sample_with_replacement(N, s, w): +# _________________________________________________________________________ +# TODO: Implement continuous map for MonteCarlo similar to Fig25.10 from the book + + +class MCLmap: + """Map which provides probability distributions and sensor readings. + Consists of discrete cells which are either an obstacle or empty""" + + def __init__(self, m): + self.m = m + self.nrows = len(m) + self.ncols = len(m[0]) + # list of empty spaces in the map + self.empty = [(i, j) for i in range(self.nrows) for j in range(self.ncols) if not m[i][j]] + + def sample(self): + """Returns a random kinematic state possible in the map""" + pos = random.choice(self.empty) + # 0N 1E 2S 3W + orient = random.choice(range(4)) + kin_state = pos + (orient,) + return kin_state + + def ray_cast(self, sensor_num, kin_state): + """Returns distance to nearest obstacle or map boundary in the direction of sensor""" + pos = kin_state[:2] + orient = kin_state[2] + # sensor layout when orientation is 0 (towards North) + # 0 + # 3R1 + # 2 + delta = ((sensor_num % 2 == 0) * (sensor_num - 1), (sensor_num % 2 == 1) * (2 - sensor_num)) + # sensor direction changes based on orientation + for _ in range(orient): + delta = (delta[1], -delta[0]) + range_count = 0 + while 0 <= pos[0] < self.nrows and 0 <= pos[1] < self.nrows and not self.m[pos[0]][pos[1]]: + pos = vector_add(pos, delta) + range_count += 1 + return range_count + + +def monte_carlo_localization(a, z, N, P_motion_sample, P_sensor, m, S=None): """ - Performs Weighted sampling over the paricles given weights of each particle. - We keep on picking random states unitll we fill N number states in new distribution + [Figure 25.9] + Monte Carlo localization algorithm """ - s_wtd = [] - cnt = 0 - while (cnt <= N): - # Generate a random number from 0 to N-1 - i = random.randint(0, N-1) - if (probability(w[i])): - s_wtd.append(s[i]) - cnt += 1 - return s_wtd -# _________________________________________________________________________ -__doc__ += """ -# We can build up a probability distribution like this (p. 469): ->>> P = ProbDist() ->>> P['sunny'] = 0.7 ->>> P['rain'] = 0.2 ->>> P['cloudy'] = 0.08 ->>> P['snow'] = 0.02 - -# and query it like this: (Never mind this ELLIPSIS option -# added to make the doctest portable.) ->>> P['rain'] #doctest:+ELLIPSIS -0.2... - -# A Joint Probability Distribution is dealt with like this (Fig. 13.3): # noqa ->>> P = JointProbDist(['Toothache', 'Cavity', 'Catch']) ->>> T, F = True, False ->>> P[T, T, T] = 0.108; P[T, T, F] = 0.012; P[F, T, T] = 0.072; P[F, T, F] = 0.008 ->>> P[T, F, T] = 0.016; P[T, F, F] = 0.064; P[F, F, T] = 0.144; P[F, F, F] = 0.576 - ->>> P[T, T, T] -0.108 - -# Ask for P(Cavity|Toothache=T) ->>> PC = enumerate_joint_ask('Cavity', {'Toothache': T}, P) ->>> PC.show_approx() -'False: 0.4, True: 0.6' - ->>> 0.6-epsilon < PC[T] < 0.6+epsilon -True - ->>> 0.4-epsilon < PC[F] < 0.4+epsilon -True -""" + def ray_cast(sensor_num, kin_state, m): + return m.ray_cast(sensor_num, kin_state) + + M = len(z) + S_ = [0] * N + W_ = [0] * N + v = a['v'] + w = a['w'] + + if S is None: + S = [m.sample() for _ in range(N)] + + for i in range(N): + S_[i] = P_motion_sample(S[i], v, w) + W_[i] = 1 + for j in range(M): + z_ = ray_cast(j, S_[i], m) + W_[i] = W_[i] * P_sensor(z[j], z_) + + S = weighted_sample_with_replacement(N, S_, W_) + return S diff --git a/probability4e.ipynb b/probability4e.ipynb new file mode 100644 index 000000000..e148e929e --- /dev/null +++ b/probability4e.ipynb @@ -0,0 +1,1381 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "button": false, + "deletable": true, + "new_sheet": false, + "run_control": { + "read_only": false + } + }, + "source": [ + "# Probability and Bayesian Networks\n", + "\n", + "Probability theory allows us to compute the likelihood of certain events, given assumptioons about the components of the event. A Bayesian network, or Bayes net for short, is a data structure to represent a joint probability distribution over several random variables, and do inference on it. \n", + "\n", + "As an example, here is a network with five random variables, each with its conditional probability table, and with arrows from parent to child variables. The story, from Judea Pearl, is that there is a house burglar alarm, which can be triggered by either a burglary or an earthquake. If the alarm sounds, one or both of the neighbors, John and Mary, might call the owwner to say the alarm is sounding.\n", + "\n", + "
![](\"http://norvig.com/ipython/burglary2.jpg\")
\n",
+ "\n",
+ "We implement this with the help of seven Python classes:\n",
+ "\n",
+ "\n",
+ "## `BayesNet()`\n",
+ "\n",
+ "A `BayesNet` is a graph (as in the diagram above) where each node represents a random variable, and the edges are parent→child links. You can construct an empty graph with `BayesNet()`, then add variables one at a time with the method call `.add(`*variable_name, parent_names, cpt*`)`, where the names are strings, and each of the `parent_names` must already have been `.add`ed.\n",
+ "\n",
+ "## `Variable(`*name, cpt, parents*`)`\n",
+ "\n",
+ "A random variable; the ovals in the diagram above. The value of a variable depends on the value of the parents, in a probabilistic way specified by the variable's conditional probability table (CPT). Given the parents, the variable is independent of all the other variables. For example, if I know whether *Alarm* is true or false, then I know the probability of *JohnCalls*, and evidence about the other variables won't give me any more information about *JohnCalls*. Each row of the CPT uses the same order of variables as the list of parents.\n",
+ "We will only allow variables with a finite discrete domain; not continuous values. \n",
+ "\n",
+ "## `ProbDist(`*mapping*`)`
`Factor(`*mapping*`)`\n",
+ "\n",
+ "A probability distribution is a mapping of `{outcome: probability}` for every outcome of a random variable. \n",
+ "You can give `ProbDist` the same arguments that you would give to the `dict` initializer, for example\n",
+ "`ProbDist(sun=0.6, rain=0.1, cloudy=0.3)`.\n",
+ "As a shortcut for Boolean Variables, you can say `ProbDist(0.95)` instead of `ProbDist({T: 0.95, F: 0.05})`. \n",
+ "In a probability distribution, every value is between 0 and 1, and the values sum to 1.\n",
+ "A `Factor` is similar to a probability distribution, except that the values need not sum to 1. Factors\n",
+ "are used in the variable elimination inference method.\n",
+ "\n",
+ "## `Evidence(`*mapping*`)`\n",
+ "\n",
+ "A mapping of `{Variable: value, ...}` pairs, describing the exact values for a set of variables—the things we know for sure.\n",
+ "\n",
+ "## `CPTable(`*rows, parents*`)`\n",
+ "\n",
+ "A conditional probability table (or *CPT*) describes the probability of each possible outcome value of a random variable, given the values of the parent variables. A `CPTable` is a a mapping, `{tuple: probdist, ...}`, where each tuple lists the values of each of the parent variables, in order, and each probability distribution says what the possible outcomes are, given those values of the parents. The `CPTable` for *Alarm* in the diagram above would be represented as follows:\n",
+ "\n",
+ " CPTable({(T, T): .95,\n",
+ " (T, F): .94,\n",
+ " (F, T): .29,\n",
+ " (F, F): .001},\n",
+ " [Burglary, Earthquake])\n",
+ " \n",
+ "How do you read this? Take the second row, \"`(T, F): .94`\". This means that when the first parent (`Burglary`) is true, and the second parent (`Earthquake`) is fale, then the probability of `Alarm` being true is .94. Note that the .94 is an abbreviation for `ProbDist({T: .94, F: .06})`.\n",
+ " \n",
+ "## `T = Bool(True); F = Bool(False)`\n",
+ "\n",
+ "When I used `bool` values (`True` and `False`), it became hard to read rows in CPTables, because the columns didn't line up:\n",
+ "\n",
+ " (True, True, False, False, False)\n",
+ " (False, False, False, False, True)\n",
+ " (True, False, False, True, True)\n",
+ " \n",
+ "Therefore, I created the `Bool` class, with constants `T` and `F` such that `T == True` and `F == False`, and now rows are easier to read:\n",
+ "\n",
+ " (T, T, F, F, F)\n",
+ " (F, F, F, F, T)\n",
+ " (T, F, F, T, T)\n",
+ " \n",
+ "Here is the code for these classes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "button": false,
+ "collapsed": true,
+ "deletable": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "from collections import defaultdict, Counter\n",
+ "import itertools\n",
+ "import math\n",
+ "import random\n",
+ "\n",
+ "class BayesNet(object):\n",
+ " \"Bayesian network: a graph of variables connected by parent links.\"\n",
+ " \n",
+ " def __init__(self): \n",
+ " self.variables = [] # List of variables, in parent-first topological sort order\n",
+ " self.lookup = {} # Mapping of {variable_name: variable} pairs\n",
+ " \n",
+ " def add(self, name, parentnames, cpt):\n",
+ " \"Add a new Variable to the BayesNet. Parentnames must have been added previously.\"\n",
+ " parents = [self.lookup[name] for name in parentnames]\n",
+ " var = Variable(name, cpt, parents)\n",
+ " self.variables.append(var)\n",
+ " self.lookup[name] = var\n",
+ " return self\n",
+ " \n",
+ "class Variable(object):\n",
+ " \"A discrete random variable; conditional on zero or more parent Variables.\"\n",
+ " \n",
+ " def __init__(self, name, cpt, parents=()):\n",
+ " \"A variable has a name, list of parent variables, and a Conditional Probability Table.\"\n",
+ " self.__name__ = name\n",
+ " self.parents = parents\n",
+ " self.cpt = CPTable(cpt, parents)\n",
+ " self.domain = set(itertools.chain(*self.cpt.values())) # All the outcomes in the CPT\n",
+ " \n",
+ " def __repr__(self): return self.__name__\n",
+ " \n",
+ "class Factor(dict): \"An {outcome: frequency} mapping.\"\n",
+ "\n",
+ "class ProbDist(Factor):\n",
+ " \"\"\"A Probability Distribution is an {outcome: probability} mapping. \n",
+ " The values are normalized to sum to 1.\n",
+ " ProbDist(0.75) is an abbreviation for ProbDist({T: 0.75, F: 0.25}).\"\"\"\n",
+ " def __init__(self, mapping=(), **kwargs):\n",
+ " if isinstance(mapping, float):\n",
+ " mapping = {T: mapping, F: 1 - mapping}\n",
+ " self.update(mapping, **kwargs)\n",
+ " normalize(self)\n",
+ " \n",
+ "class Evidence(dict): \n",
+ " \"A {variable: value} mapping, describing what we know for sure.\"\n",
+ " \n",
+ "class CPTable(dict):\n",
+ " \"A mapping of {row: ProbDist, ...} where each row is a tuple of values of the parent variables.\"\n",
+ " \n",
+ " def __init__(self, mapping, parents=()):\n",
+ " \"\"\"Provides two shortcuts for writing a Conditional Probability Table. \n",
+ " With no parents, CPTable(dist) means CPTable({(): dist}).\n",
+ " With one parent, CPTable({val: dist,...}) means CPTable({(val,): dist,...}).\"\"\"\n",
+ " if len(parents) == 0 and not (isinstance(mapping, dict) and set(mapping.keys()) == {()}):\n",
+ " mapping = {(): mapping}\n",
+ " for (row, dist) in mapping.items():\n",
+ " if len(parents) == 1 and not isinstance(row, tuple): \n",
+ " row = (row,)\n",
+ " self[row] = ProbDist(dist)\n",
+ "\n",
+ "class Bool(int):\n",
+ " \"Just like `bool`, except values display as 'T' and 'F' instead of 'True' and 'False'\"\n",
+ " __str__ = __repr__ = lambda self: 'T' if self else 'F'\n",
+ " \n",
+ "T = Bool(True)\n",
+ "F = Bool(False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "And here are some associated functions:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def P(var, evidence={}):\n",
+ " \"The probability distribution for P(variable | evidence), when all parent variables are known (in evidence).\"\n",
+ " row = tuple(evidence[parent] for parent in var.parents)\n",
+ " return var.cpt[row]\n",
+ "\n",
+ "def normalize(dist):\n",
+ " \"Normalize a {key: value} distribution so values sum to 1.0. Mutates dist and returns it.\"\n",
+ " total = sum(dist.values())\n",
+ " for key in dist:\n",
+ " dist[key] = dist[key] / total\n",
+ " assert 0 <= dist[key] <= 1, \"Probabilities must be between 0 and 1.\"\n",
+ " return dist\n",
+ "\n",
+ "def sample(probdist):\n",
+ " \"Randomly sample an outcome from a probability distribution.\"\n",
+ " r = random.random() # r is a random point in the probability distribution\n",
+ " c = 0.0 # c is the cumulative probability of outcomes seen so far\n",
+ " for outcome in probdist:\n",
+ " c += probdist[outcome]\n",
+ " if r <= c:\n",
+ " return outcome\n",
+ " \n",
+ "def globalize(mapping):\n",
+ " \"Given a {name: value} mapping, export all the names to the `globals()` namespace.\"\n",
+ " globals().update(mapping)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Sample Usage\n",
+ "\n",
+ "Here are some examples of using the classes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# Example random variable: Earthquake:\n",
+ "# An earthquake occurs on 0.002 of days, independent of any other variables.\n",
+ "Earthquake = Variable('Earthquake', 0.002)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.998, T: 0.002}"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# The probability distribution for Earthquake\n",
+ "P(Earthquake)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.002"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Get the probability of a specific outcome by subscripting the probability distribution\n",
+ "P(Earthquake)[T]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "F"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Randomly sample from the distribution:\n",
+ "sample(P(Earthquake))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Counter({F: 99793, T: 207})"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Randomly sample 100,000 times, and count up the results:\n",
+ "Counter(sample(P(Earthquake)) for i in range(100000))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# Two equivalent ways of specifying the same Boolean probability distribution:\n",
+ "assert ProbDist(0.75) == ProbDist({T: 0.75, F: 0.25})"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'lose': 0.15, 'tie': 0.1, 'win': 0.75}"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Two equivalent ways of specifying the same non-Boolean probability distribution:\n",
+ "assert ProbDist(win=15, lose=3, tie=2) == ProbDist({'win': 15, 'lose': 3, 'tie': 2})\n",
+ "ProbDist(win=15, lose=3, tie=2)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'a': 1, 'b': 2, 'c': 3, 'd': 4}"
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# The difference between a Factor and a ProbDist--the ProbDist is normalized:\n",
+ "Factor(a=1, b=2, c=3, d=4)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'a': 0.1, 'b': 0.2, 'c': 0.3, 'd': 0.4}"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ProbDist(a=1, b=2, c=3, d=4)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Example: Alarm Bayes Net\n",
+ "\n",
+ "Here is how we define the Bayes net from the diagram above:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "alarm_net = (BayesNet()\n",
+ " .add('Burglary', [], 0.001)\n",
+ " .add('Earthquake', [], 0.002)\n",
+ " .add('Alarm', ['Burglary', 'Earthquake'], {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001})\n",
+ " .add('JohnCalls', ['Alarm'], {T: 0.90, F: 0.05})\n",
+ " .add('MaryCalls', ['Alarm'], {T: 0.70, F: 0.01})) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[Burglary, Earthquake, Alarm, JohnCalls, MaryCalls]"
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Make Burglary, Earthquake, etc. be global variables\n",
+ "globalize(alarm_net.lookup) \n",
+ "alarm_net.variables"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.999, T: 0.001}"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Probability distribution of a Burglary\n",
+ "P(Burglary)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.06000000000000005, T: 0.94}"
+ ]
+ },
+ "execution_count": 15,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Probability of Alarm going off, given a Burglary and not an Earthquake:\n",
+ "P(Alarm, {Burglary: T, Earthquake: F})"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{(F, F): {F: 0.999, T: 0.001},\n",
+ " (F, T): {F: 0.71, T: 0.29},\n",
+ " (T, F): {F: 0.06000000000000005, T: 0.94},\n",
+ " (T, T): {F: 0.050000000000000044, T: 0.95}}"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Where that came from: the (T, F) row of Alarm's CPT:\n",
+ "Alarm.cpt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "deletable": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "# Bayes Nets as Joint Probability Distributions\n",
+ "\n",
+ "A Bayes net is a compact way of specifying a full joint distribution over all the variables in the network. Given a set of variables {*X*1, ..., *X**n*}, the full joint distribution is:\n",
+ "\n",
+ "P(*X*1=*x*1, ..., *X**n*=*x**n*) = Π*i* P(*X**i* = *x**i* | parents(*X**i*))\n",
+ "\n",
+ "For a network with *n* variables, each of which has *b* values, there are *bn* rows in the joint distribution (for example, a billion rows for 30 Boolean variables), making it impractical to explicitly create the joint distribution for large networks. But for small networks, the function `joint_distribution` creates the distribution, which can be instructive to look at, and can be used to do inference. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def joint_distribution(net):\n",
+ " \"Given a Bayes net, create the joint distribution over all variables.\"\n",
+ " return ProbDist({row: prod(P_xi_given_parents(var, row, net)\n",
+ " for var in net.variables)\n",
+ " for row in all_rows(net)})\n",
+ "\n",
+ "def all_rows(net): return itertools.product(*[var.domain for var in net.variables])\n",
+ "\n",
+ "def P_xi_given_parents(var, row, net):\n",
+ " \"The probability that var = xi, given the values in this row.\"\n",
+ " dist = P(var, Evidence(zip(net.variables, row)))\n",
+ " xi = row[net.variables.index(var)]\n",
+ " return dist[xi]\n",
+ "\n",
+ "def prod(numbers):\n",
+ " \"The product of numbers: prod([2, 3, 5]) == 30. Analogous to `sum([2, 3, 5]) == 10`.\"\n",
+ " result = 1\n",
+ " for x in numbers:\n",
+ " result *= x\n",
+ " return result"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{(F, F, F, F, F),\n",
+ " (F, F, F, F, T),\n",
+ " (F, F, F, T, F),\n",
+ " (F, F, F, T, T),\n",
+ " (F, F, T, F, F),\n",
+ " (F, F, T, F, T),\n",
+ " (F, F, T, T, F),\n",
+ " (F, F, T, T, T),\n",
+ " (F, T, F, F, F),\n",
+ " (F, T, F, F, T),\n",
+ " (F, T, F, T, F),\n",
+ " (F, T, F, T, T),\n",
+ " (F, T, T, F, F),\n",
+ " (F, T, T, F, T),\n",
+ " (F, T, T, T, F),\n",
+ " (F, T, T, T, T),\n",
+ " (T, F, F, F, F),\n",
+ " (T, F, F, F, T),\n",
+ " (T, F, F, T, F),\n",
+ " (T, F, F, T, T),\n",
+ " (T, F, T, F, F),\n",
+ " (T, F, T, F, T),\n",
+ " (T, F, T, T, F),\n",
+ " (T, F, T, T, T),\n",
+ " (T, T, F, F, F),\n",
+ " (T, T, F, F, T),\n",
+ " (T, T, F, T, F),\n",
+ " (T, T, F, T, T),\n",
+ " (T, T, T, F, F),\n",
+ " (T, T, T, F, T),\n",
+ " (T, T, T, T, F),\n",
+ " (T, T, T, T, T)}"
+ ]
+ },
+ "execution_count": 18,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# All rows in the joint distribution (2**5 == 32 rows)\n",
+ "set(all_rows(alarm_net))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# Let's work through just one row of the table:\n",
+ "row = (F, F, F, F, F)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.999, T: 0.001}"
+ ]
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# This is the probability distribution for Alarm\n",
+ "P(Alarm, {Burglary: F, Earthquake: F})"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.999"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Here's the probability that Alarm is false, given the parent values in this row:\n",
+ "P_xi_given_parents(Alarm, row, alarm_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{(F, F, F, F, F): 0.9367427006190001,\n",
+ " (F, F, F, F, T): 0.009462047481000001,\n",
+ " (F, F, F, T, F): 0.04930224740100002,\n",
+ " (F, F, F, T, T): 0.0004980024990000002,\n",
+ " (F, F, T, F, F): 2.9910060000000004e-05,\n",
+ " (F, F, T, F, T): 6.979013999999999e-05,\n",
+ " (F, F, T, T, F): 0.00026919054000000005,\n",
+ " (F, F, T, T, T): 0.00062811126,\n",
+ " (F, T, F, F, F): 0.0013341744900000002,\n",
+ " (F, T, F, F, T): 1.3476510000000005e-05,\n",
+ " (F, T, F, T, F): 7.021971000000001e-05,\n",
+ " (F, T, F, T, T): 7.092900000000001e-07,\n",
+ " (F, T, T, F, F): 1.7382600000000002e-05,\n",
+ " (F, T, T, F, T): 4.0559399999999997e-05,\n",
+ " (F, T, T, T, F): 0.00015644340000000006,\n",
+ " (F, T, T, T, T): 0.00036503460000000007,\n",
+ " (T, F, F, F, F): 5.631714000000006e-05,\n",
+ " (T, F, F, F, T): 5.688600000000006e-07,\n",
+ " (T, F, F, T, F): 2.9640600000000033e-06,\n",
+ " (T, F, F, T, T): 2.9940000000000035e-08,\n",
+ " (T, F, T, F, F): 2.8143600000000003e-05,\n",
+ " (T, F, T, F, T): 6.56684e-05,\n",
+ " (T, F, T, T, F): 0.0002532924000000001,\n",
+ " (T, F, T, T, T): 0.0005910156000000001,\n",
+ " (T, T, F, F, F): 9.40500000000001e-08,\n",
+ " (T, T, F, F, T): 9.50000000000001e-10,\n",
+ " (T, T, F, T, F): 4.9500000000000054e-09,\n",
+ " (T, T, F, T, T): 5.0000000000000066e-11,\n",
+ " (T, T, T, F, F): 5.7e-08,\n",
+ " (T, T, T, F, T): 1.3299999999999996e-07,\n",
+ " (T, T, T, T, F): 5.130000000000002e-07,\n",
+ " (T, T, T, T, T): 1.1970000000000001e-06}"
+ ]
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# The full joint distribution:\n",
+ "joint_distribution(alarm_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[Burglary, Earthquake, Alarm, JohnCalls, MaryCalls]\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "0.00062811126"
+ ]
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Probability that \"the alarm has sounded, but neither a burglary nor an earthquake has occurred, \n",
+ "# and both John and Mary call\" (page 514 says it should be 0.000628)\n",
+ "\n",
+ "print(alarm_net.variables)\n",
+ "joint_distribution(alarm_net)[F, F, T, T, T]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Inference by Querying the Joint Distribution\n",
+ "\n",
+ "We can use `P(variable, evidence)` to get the probability of aa variable, if we know the vaues of all the parent variables. But what if we don't know? Bayes nets allow us to calculate the probability, but the calculation is not just a lookup in the CPT; it is a global calculation across the whole net. One inefficient but straightforward way of doing the calculation is to create the joint probability distribution, then pick out just the rows that\n",
+ "match the evidence variables, and for each row check what the value of the query variable is, and increment the probability for that value accordningly:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "def enumeration_ask(X, evidence, net):\n",
+ " \"The probability distribution for query variable X in a belief net, given evidence.\"\n",
+ " i = net.variables.index(X) # The index of the query variable X in the row\n",
+ " dist = defaultdict(float) # The resulting probability distribution over X\n",
+ " for (row, p) in joint_distribution(net).items():\n",
+ " if matches_evidence(row, evidence, net):\n",
+ " dist[row[i]] += p\n",
+ " return ProbDist(dist)\n",
+ "\n",
+ "def matches_evidence(row, evidence, net):\n",
+ " \"Does the tuple of values for this row agree with the evidence?\"\n",
+ " return all(evidence[v] == row[net.variables.index(v)]\n",
+ " for v in evidence)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.9931237539265789, T: 0.006876246073421024}"
+ ]
+ },
+ "execution_count": 25,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# The probability of a Burgalry, given that John calls but Mary does not: \n",
+ "enumeration_ask(Burglary, {JohnCalls: F, MaryCalls: T}, alarm_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.03368899586522123, T: 0.9663110041347788}"
+ ]
+ },
+ "execution_count": 26,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# The probability of an Alarm, given that there is an Earthquake and Mary calls:\n",
+ "enumeration_ask(Alarm, {MaryCalls: T, Earthquake: T}, alarm_net)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Variable Elimination\n",
+ "\n",
+ "The `enumeration_ask` algorithm takes time and space that is exponential in the number of variables. That is, first it creates the joint distribution, of size *bn*, and then it sums out the values for the rows that match the evidence. We can do better than that if we interleave the joining of variables with the summing out of values.\n",
+ "This approach is called *variable elimination*. The key insight is that\n",
+ "when we compute\n",
+ "\n",
+ "P(*X*1=*x*1, ..., *X**n*=*x**n*) = Π*i* P(*X**i* = *x**i* | parents(*X**i*))\n",
+ "\n",
+ "we are repeating the calculation of, say, P(*X**3* = *x**4* | parents(*X**3*))\n",
+ "multiple times, across multiple rows of the joint distribution.\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# TODO: Copy over and update Variable Elimination algorithm. Also, sampling algorithms."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "button": false,
+ "deletable": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "source": [
+ "# Example: Flu Net\n",
+ "\n",
+ "In this net, whether a patient gets the flu is dependent on whether they were vaccinated, and having the flu influences whether they get a fever or headache. Here `Fever` is a non-Boolean variable, with three values, `no`, `mild`, and `high`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "button": false,
+ "collapsed": false,
+ "deletable": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "flu_net = (BayesNet()\n",
+ " .add('Vaccinated', [], 0.60)\n",
+ " .add('Flu', ['Vaccinated'], {T: 0.002, F: 0.02})\n",
+ " .add('Fever', ['Flu'], {T: ProbDist(no=25, mild=25, high=50),\n",
+ " F: ProbDist(no=97, mild=2, high=1)})\n",
+ " .add('Headache', ['Flu'], {T: 0.5, F: 0.03}))\n",
+ "\n",
+ "globalize(flu_net.lookup)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.9616440110625343, T: 0.03835598893746573}"
+ ]
+ },
+ "execution_count": 29,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# If you just have a headache, you probably don't have the Flu.\n",
+ "enumeration_ask(Flu, {Headache: T, Fever: 'no'}, flu_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "button": false,
+ "collapsed": false,
+ "deletable": true,
+ "new_sheet": false,
+ "run_control": {
+ "read_only": false
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.9914651882096696, T: 0.008534811790330398}"
+ ]
+ },
+ "execution_count": 30,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Even more so if you were vaccinated.\n",
+ "enumeration_ask(Flu, {Headache: T, Fever: 'no', Vaccinated: T}, flu_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.9194016377587207, T: 0.08059836224127925}"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# But if you were not vaccinated, there is a higher chance you have the flu.\n",
+ "enumeration_ask(Flu, {Headache: T, Fever: 'no', Vaccinated: F}, flu_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.1904145077720207, T: 0.8095854922279793}"
+ ]
+ },
+ "execution_count": 32,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# And if you have both headache and fever, and were not vaccinated, \n",
+ "# then the flu is very likely, especially if it is a high fever.\n",
+ "enumeration_ask(Flu, {Headache: T, Fever: 'mild', Vaccinated: F}, flu_net)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{F: 0.055534567434831886, T: 0.9444654325651682}"
+ ]
+ },
+ "execution_count": 33,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "enumeration_ask(Flu, {Headache: T, Fever: 'high', Vaccinated: F}, flu_net)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Entropy\n",
+ "\n",
+ "We can compute the entropy of a probability distribution:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 34,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "def entropy(probdist):\n",
+ " \"The entropy of a probability distribution.\"\n",
+ " return - sum(p * math.log(p, 2)\n",
+ " for p in probdist.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 35,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "1.0"
+ ]
+ },
+ "execution_count": 35,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "entropy(ProbDist(heads=0.5, tails=0.5))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.011397802630112312"
+ ]
+ },
+ "execution_count": 36,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "entropy(ProbDist(yes=1000, no=1))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 37,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.8687212463394045"
+ ]
+ },
+ "execution_count": 37,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "entropy(P(Alarm, {Earthquake: T, Burglary: F}))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 38,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.011407757737461138"
+ ]
+ },
+ "execution_count": 38,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "entropy(P(Alarm, {Earthquake: F, Burglary: F}))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For non-Boolean variables, the entropy can be greater than 1 bit:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 39,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "1.5"
+ ]
+ },
+ "execution_count": 39,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "entropy(P(Fever, {Flu: T}))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# Unknown Outcomes: Smoothing\n",
+ "\n",
+ "So far we have dealt with discrete distributions where we know all the possible outcomes in advance. For Boolean variables, the only outcomes are `T` and `F`. For `Fever`, we modeled exactly three outcomes. However, in some applications we will encounter new, previously unknown outcomes over time. For example, we could train a model on the distribution of words in English, and then somebody could coin a brand new word. To deal with this, we introduce\n",
+ "the `DefaultProbDist` distribution, which uses the key `None` to stand as a placeholder for any unknown outcome(s)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 40,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "class DefaultProbDist(ProbDist):\n",
+ " \"\"\"A Probability Distribution that supports smoothing for unknown outcomes (keys).\n",
+ " The default_value represents the probability of an unknown (previously unseen) key. \n",
+ " The key `None` stands for unknown outcomes.\"\"\"\n",
+ " def __init__(self, default_value, mapping=(), **kwargs):\n",
+ " self[None] = default_value\n",
+ " self.update(mapping, **kwargs)\n",
+ " normalize(self)\n",
+ " \n",
+ " def __missing__(self, key): return self[None] "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 41,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import re\n",
+ "\n",
+ "def words(text): return re.findall(r'\\w+', text.lower())\n",
+ "\n",
+ "english = words('''This is a sample corpus of English prose. To get a better model, we would train on much\n",
+ "more text. But this should give you an idea of the process. So far we have dealt with discrete \n",
+ "distributions where we know all the possible outcomes in advance. For Boolean variables, the only \n",
+ "outcomes are T and F. For Fever, we modeled exactly three outcomes. However, in some applications we \n",
+ "will encounter new, previously unknown outcomes over time. For example, when we could train a model on the \n",
+ "words in this text, we get a distribution, but somebody could coin a brand new word. To deal with this, \n",
+ "we introduce the DefaultProbDist distribution, which uses the key `None` to stand as a placeholder for any \n",
+ "unknown outcomes. Probability theory allows us to compute the likelihood of certain events, given \n",
+ "assumptions about the components of the event. A Bayesian network, or Bayes net for short, is a data \n",
+ "structure to represent a joint probability distribution over several random variables, and do inference on it.''')\n",
+ "\n",
+ "E = DefaultProbDist(0.1, Counter(english))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 42,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.052295177222545036"
+ ]
+ },
+ "execution_count": 42,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 'the' is a common word:\n",
+ "E['the']"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.005810575246949448"
+ ]
+ },
+ "execution_count": 43,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 'possible' is a less-common word:\n",
+ "E['possible']"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.0005810575246949449"
+ ]
+ },
+ "execution_count": 44,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 'impossible' was not seen in the training data, but still gets a non-zero probability ...\n",
+ "E['impossible']"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.0005810575246949449"
+ ]
+ },
+ "execution_count": 45,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# ... as do other rare, previously unseen words:\n",
+ "E['llanfairpwllgwyngyll']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note that this does not mean that 'impossible' and 'llanfairpwllgwyngyll' and all the other unknown words\n",
+ "*each* have probability 0.004.\n",
+ "Rather, it means that together, all the unknown words total probability 0.004. With that\n",
+ "interpretation, the sum of all the probabilities is still 1, as it should be. In the `DefaultProbDist`, the\n",
+ "unknown words are all represented by the key `None`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.0005810575246949449"
+ ]
+ },
+ "execution_count": 46,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "E[None]"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.5.1"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/probability4e.py b/probability4e.py
new file mode 100644
index 000000000..d413a55ae
--- /dev/null
+++ b/probability4e.py
@@ -0,0 +1,776 @@
+"""Probability models (Chapter 12-13)"""
+
+import copy
+import random
+from collections import defaultdict
+from functools import reduce
+
+import numpy as np
+
+from utils4e import product, probability, extend
+
+
+# ______________________________________________________________________________
+# Chapter 12 Qualifying Uncertainty
+# 12.1 Acting Under Uncertainty
+
+
+def DTAgentProgram(belief_state):
+ """A decision-theoretic agent. [Figure 12.1]"""
+
+ def program(percept):
+ belief_state.observe(program.action, percept)
+ program.action = max(belief_state.actions(), key=belief_state.expected_outcome_utility)
+ return program.action
+
+ program.action = None
+ return program
+
+
+# ______________________________________________________________________________
+# 12.2 Basic Probability Notation
+
+
+class ProbDist:
+ """A discrete probability distribution. You name the random variable
+ in the constructor, then assign and query probability of values.
+ >>> P = ProbDist('Flip'); P['H'], P['T'] = 0.25, 0.75; P['H']
+ 0.25
+ >>> P = ProbDist('X', {'lo': 125, 'med': 375, 'hi': 500})
+ >>> P['lo'], P['med'], P['hi']
+ (0.125, 0.375, 0.5)
+ """
+
+ def __init__(self, varname='?', freqs=None):
+ """If freqs is given, it is a dictionary of values - frequency pairs,
+ then ProbDist is normalized."""
+ self.prob = {}
+ self.varname = varname
+ self.values = []
+ if freqs:
+ for (v, p) in freqs.items():
+ self[v] = p
+ self.normalize()
+
+ def __getitem__(self, val):
+ """Given a value, return P(value)."""
+ try:
+ return self.prob[val]
+ except KeyError:
+ return 0
+
+ def __setitem__(self, val, p):
+ """Set P(val) = p."""
+ if val not in self.values:
+ self.values.append(val)
+ self.prob[val] = p
+
+ def normalize(self):
+ """Make sure the probabilities of all values sum to 1.
+ Returns the normalized distribution.
+ Raises a ZeroDivisionError if the sum of the values is 0."""
+ total = sum(self.prob.values())
+ if not np.isclose(total, 1.0):
+ for val in self.prob:
+ self.prob[val] /= total
+ return self
+
+ def show_approx(self, numfmt='{:.3g}'):
+ """Show the probabilities rounded and sorted by key, for the
+ sake of portable doctests."""
+ return ', '.join([('{}: ' + numfmt).format(v, p)
+ for (v, p) in sorted(self.prob.items())])
+
+ def __repr__(self):
+ return "P({})".format(self.varname)
+
+
+# ______________________________________________________________________________
+# 12.3 Inference Using Full Joint Distributions
+
+
+class JointProbDist(ProbDist):
+ """A discrete probability distribute over a set of variables.
+ >>> P = JointProbDist(['X', 'Y']); P[1, 1] = 0.25
+ >>> P[1, 1]
+ 0.25
+ >>> P[dict(X=0, Y=1)] = 0.5
+ >>> P[dict(X=0, Y=1)]
+ 0.5"""
+
+ def __init__(self, variables):
+ self.prob = {}
+ self.variables = variables
+ self.vals = defaultdict(list)
+
+ def __getitem__(self, values):
+ """Given a tuple or dict of values, return P(values)."""
+ values = event_values(values, self.variables)
+ return ProbDist.__getitem__(self, values)
+
+ def __setitem__(self, values, p):
+ """Set P(values) = p. Values can be a tuple or a dict; it must
+ have a value for each of the variables in the joint. Also keep track
+ of the values we have seen so far for each variable."""
+ values = event_values(values, self.variables)
+ self.prob[values] = p
+ for var, val in zip(self.variables, values):
+ if val not in self.vals[var]:
+ self.vals[var].append(val)
+
+ def values(self, var):
+ """Return the set of possible values for a variable."""
+ return self.vals[var]
+
+ def __repr__(self):
+ return "P({})".format(self.variables)
+
+
+def event_values(event, variables):
+ """Return a tuple of the values of variables in event.
+ >>> event_values ({'A': 10, 'B': 9, 'C': 8}, ['C', 'A'])
+ (8, 10)
+ >>> event_values ((1, 2), ['C', 'A'])
+ (1, 2)
+ """
+ if isinstance(event, tuple) and len(event) == len(variables):
+ return event
+ else:
+ return tuple([event[var] for var in variables])
+
+
+def enumerate_joint_ask(X, e, P):
+ """Return a probability distribution over the values of the variable X,
+ given the {var:val} observations e, in the JointProbDist P. [Section 12.3]
+ >>> P = JointProbDist(['X', 'Y'])
+ >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[2,1] = 0.125
+ >>> enumerate_joint_ask('X', dict(Y=1), P).show_approx()
+ '0: 0.667, 1: 0.167, 2: 0.167'
+ """
+ assert X not in e, "Query variable must be distinct from evidence"
+ Q = ProbDist(X) # probability distribution for X, initially empty
+ Y = [v for v in P.variables if v != X and v not in e] # hidden variables.
+ for xi in P.values(X):
+ Q[xi] = enumerate_joint(Y, extend(e, X, xi), P)
+ return Q.normalize()
+
+
+def enumerate_joint(variables, e, P):
+ """Return the sum of those entries in P consistent with e,
+ provided variables is P's remaining variables (the ones not in e)."""
+ if not variables:
+ return P[e]
+ Y, rest = variables[0], variables[1:]
+ return sum([enumerate_joint(rest, extend(e, Y, y), P)
+ for y in P.values(Y)])
+
+
+# ______________________________________________________________________________
+# 12.4 Independence
+
+
+def is_independent(variables, P):
+ """
+ Return whether a list of variables are independent given their distribution P
+ P is an instance of JoinProbDist
+ >>> P = JointProbDist(['X', 'Y'])
+ >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[1,0] = 0.125
+ >>> is_independent(['X', 'Y'], P)
+ False
+ """
+ for var in variables:
+ event_vars = variables[:]
+ event_vars.remove(var)
+ event = {}
+ distribution = enumerate_joint_ask(var, event, P)
+ events = gen_possible_events(event_vars, P)
+ for e in events:
+ conditional_distr = enumerate_joint_ask(var, e, P)
+ if conditional_distr.prob != distribution.prob:
+ return False
+ return True
+
+
+def gen_possible_events(vars, P):
+ """Generate all possible events of a collection of vars according to distribution of P"""
+ events = []
+
+ def backtrack(vars, P, temp):
+ if not vars:
+ events.append(temp)
+ return
+ var = vars[0]
+ for val in P.values(var):
+ temp[var] = val
+ backtrack([v for v in vars if v != var], P, copy.copy(temp))
+
+ backtrack(vars, P, {})
+ return events
+
+
+# ______________________________________________________________________________
+# Chapter 13 Probabilistic Reasoning
+# 13.1 Representing Knowledge in an Uncertain Domain
+
+
+class BayesNet:
+ """Bayesian network containing only boolean-variable nodes."""
+
+ def __init__(self, node_specs=None):
+ """
+ Nodes must be ordered with parents before children.
+ :param node_specs: an nested iterable object, each element contains (variable name, parents name, cpt)
+ for each node
+ """
+
+ self.nodes = []
+ self.variables = []
+ node_specs = node_specs or []
+ for node_spec in node_specs:
+ self.add(node_spec)
+
+ def add(self, node_spec):
+ """
+ Add a node to the net. Its parents must already be in the
+ net, and its variable must not.
+ Initialize Bayes nodes by detecting the length of input node specs
+ """
+ if len(node_spec) >= 5:
+ node = ContinuousBayesNode(*node_spec)
+ else:
+ node = BayesNode(*node_spec)
+ assert node.variable not in self.variables
+ assert all((parent in self.variables) for parent in node.parents)
+ self.nodes.append(node)
+ self.variables.append(node.variable)
+ for parent in node.parents:
+ self.variable_node(parent).children.append(node)
+
+ def variable_node(self, var):
+ """
+ Return the node for the variable named var.
+ >>> burglary.variable_node('Burglary').variable
+ 'Burglary'
+ """
+ for n in self.nodes:
+ if n.variable == var:
+ return n
+ raise Exception("No such variable: {}".format(var))
+
+ def variable_values(self, var):
+ """Return the domain of var."""
+ return [True, False]
+
+ def __repr__(self):
+ return 'BayesNet({0!r})'.format(self.nodes)
+
+
+class BayesNode:
+ """
+ A conditional probability distribution for a boolean variable,
+ P(X | parents). Part of a BayesNet.
+ """
+
+ def __init__(self, X, parents, cpt):
+ """
+ :param X: variable name,
+ :param parents: a sequence of variable names or a space-separated string. Representing the names of parent nodes
+ :param cpt: the conditional probability table, takes one of these forms:
+
+ * A number, the unconditional probability P(X=true). You can
+ use this form when there are no parents.
+
+ * A dict {v: p, ...}, the conditional probability distribution
+ P(X=true | parent=v) = p. When there's just one parent.
+
+ * A dict {(v1, v2, ...): p, ...}, the distribution P(X=true |
+ parent1=v1, parent2=v2, ...) = p. Each key must have as many
+ values as there are parents. You can use this form always;
+ the first two are just conveniences.
+
+ In all cases the probability of X being false is left implicit,
+ since it follows from P(X=true).
+
+ >>> X = BayesNode('X', '', 0.2)
+ >>> Y = BayesNode('Y', 'P', {T: 0.2, F: 0.7})
+ >>> Z = BayesNode('Z', 'P Q',
+ ... {(T, T): 0.2, (T, F): 0.3, (F, T): 0.5, (F, F): 0.7})
+ """
+ if isinstance(parents, str):
+ parents = parents.split()
+
+ # We store the table always in the third form above.
+ if isinstance(cpt, (float, int)): # no parents, 0-tuple
+ cpt = {(): cpt}
+ elif isinstance(cpt, dict):
+ # one parent, 1-tuple
+ if cpt and isinstance(list(cpt.keys())[0], bool):
+ cpt = {(v,): p for v, p in cpt.items()}
+
+ assert isinstance(cpt, dict)
+ for vs, p in cpt.items():
+ assert isinstance(vs, tuple) and len(vs) == len(parents)
+ assert all(isinstance(v, bool) for v in vs)
+ assert 0 <= p <= 1
+
+ self.variable = X
+ self.parents = parents
+ self.cpt = cpt
+ self.children = []
+
+ def p(self, value, event):
+ """
+ Return the conditional probability
+ P(X=value | parents=parent_values), where parent_values
+ are the values of parents in event. (event must assign each
+ parent a value.)
+ >>> bn = BayesNode('X', 'Burglary', {T: 0.2, F: 0.625})
+ >>> bn.p(False, {'Burglary': False, 'Earthquake': True})
+ 0.375
+ """
+ assert isinstance(value, bool)
+ ptrue = self.cpt[event_values(event, self.parents)]
+ return ptrue if value else 1 - ptrue
+
+ def sample(self, event):
+ """
+ Sample from the distribution for this variable conditioned
+ on event's values for parent_variables. That is, return True/False
+ at random according with the conditional probability given the
+ parents.
+ """
+ return probability(self.p(True, event))
+
+ def __repr__(self):
+ return repr((self.variable, ' '.join(self.parents)))
+
+
+# Burglary example [Figure 13 .2]
+
+
+T, F = True, False
+
+burglary = BayesNet([
+ ('Burglary', '', 0.001),
+ ('Earthquake', '', 0.002),
+ ('Alarm', 'Burglary Earthquake',
+ {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}),
+ ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}),
+ ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01})
+])
+
+
+# ______________________________________________________________________________
+# Section 13.2. The Semantics of Bayesian Networks
+# Bayesian nets with continuous variables
+
+
+def gaussian_probability(param, event, value):
+ """
+ Gaussian probability of a continuous Bayesian network node on condition of
+ certain event and the parameters determined by the event
+ :param param: parameters determined by discrete parent events of current node
+ :param event: a dict, continuous event of current node, the values are used
+ as parameters in calculating distribution
+ :param value: float, the value of current continuous node
+ :return: float, the calculated probability
+ >>> param = {'sigma':0.5, 'b':1, 'a':{'h1':0.5, 'h2': 1.5}}
+ >>> event = {'h1':0.6, 'h2': 0.3}
+ >>> gaussian_probability(param, event, 1)
+ 0.2590351913317835
+ """
+
+ assert isinstance(event, dict)
+ assert isinstance(param, dict)
+ buff = 0
+ for k, v in event.items():
+ # buffer varianle to calculate h1*a_h1 + h2*a_h2
+ buff += param['a'][k] * v
+ res = 1 / (param['sigma'] * np.sqrt(2 * np.pi)) * np.exp(-0.5 * ((value - buff - param['b']) / param['sigma']) ** 2)
+ return res
+
+
+def logistic_probability(param, event, value):
+ """
+ Logistic probability of a discrete node in Bayesian network with continuous parents,
+ :param param: a dict, parameters determined by discrete parents of current node
+ :param event: a dict, names and values of continuous parent variables of current node
+ :param value: boolean, True or False
+ :return: int, probability
+ """
+
+ buff = 1
+ for _, v in event.items():
+ # buffer variable to calculate (value-mu)/sigma
+
+ buff *= (v - param['mu']) / param['sigma']
+ p = 1 - 1 / (1 + np.exp(-4 / np.sqrt(2 * np.pi) * buff))
+ return p if value else 1 - p
+
+
+class ContinuousBayesNode:
+ """ A Bayesian network node with continuous distribution or with continuous distributed parents """
+
+ def __init__(self, name, d_parents, c_parents, parameters, type):
+ """
+ A continuous Bayesian node has two types of parents: discrete and continuous.
+ :param d_parents: str, name of discrete parents, value of which determines distribution parameters
+ :param c_parents: str, name of continuous parents, value of which is used to calculate distribution
+ :param parameters: a dict, parameters for distribution of current node, keys corresponds to discrete parents
+ :param type: str, type of current node's value, either 'd' (discrete) or 'c'(continuous)
+ """
+
+ self.parameters = parameters
+ self.type = type
+ self.d_parents = d_parents.split()
+ self.c_parents = c_parents.split()
+ self.parents = self.d_parents + self.c_parents
+ self.variable = name
+ self.children = []
+
+ def continuous_p(self, value, c_event, d_event):
+ """
+ Probability given the value of current node and its parents
+ :param c_event: event of continuous nodes
+ :param d_event: event of discrete nodes
+ """
+ assert isinstance(c_event, dict)
+ assert isinstance(d_event, dict)
+
+ d_event_vals = event_values(d_event, self.d_parents)
+ if len(d_event_vals) == 1:
+ d_event_vals = d_event_vals[0]
+ param = self.parameters[d_event_vals]
+ if self.type == "c":
+ p = gaussian_probability(param, c_event, value)
+ if self.type == "d":
+ p = logistic_probability(param, c_event, value)
+ return p
+
+
+# harvest-buy example. Figure 13.5
+
+
+harvest_buy = BayesNet([
+ ('Subsidy', '', 0.001),
+ ('Harvest', '', 0.002),
+ ('Cost', 'Subsidy', 'Harvest',
+ {True: {'sigma': 0.5, 'b': 1, 'a': {'Harvest': 0.5}},
+ False: {'sigma': 0.6, 'b': 1, 'a': {'Harvest': 0.5}}}, 'c'),
+ ('Buys', '', 'Cost', {T: {'mu': 0.5, 'sigma': 0.5}, F: {'mu': 0.6, 'sigma': 0.6}}, 'd')])
+
+
+# ______________________________________________________________________________
+# 13.3 Exact Inference in Bayesian Networks
+# 13.3.1 Inference by enumeration
+
+
+def enumeration_ask(X, e, bn):
+ """
+ Return the conditional probability distribution of variable X
+ given evidence e, from BayesNet bn. [Figure 13.10]
+ >>> enumeration_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary
+ ... ).show_approx()
+ 'False: 0.716, True: 0.284'
+ """
+
+ assert X not in e, "Query variable must be distinct from evidence"
+ Q = ProbDist(X)
+ for xi in bn.variable_values(X):
+ Q[xi] = enumerate_all(bn.variables, extend(e, X, xi), bn)
+ return Q.normalize()
+
+
+def enumerate_all(variables, e, bn):
+ """
+ Return the sum of those entries in P(variables | e{others})
+ consistent with e, where P is the joint distribution represented
+ by bn, and e{others} means e restricted to bn's other variables
+ (the ones other than variables). Parents must precede children in variables.
+ """
+
+ if not variables:
+ return 1.0
+ Y, rest = variables[0], variables[1:]
+ Ynode = bn.variable_node(Y)
+ if Y in e:
+ return Ynode.p(e[Y], e) * enumerate_all(rest, e, bn)
+ else:
+ return sum(Ynode.p(y, e) * enumerate_all(rest, extend(e, Y, y), bn)
+ for y in bn.variable_values(Y))
+
+
+# ______________________________________________________________________________
+# 13.3.2 The variable elimination algorithm
+
+
+def elimination_ask(X, e, bn):
+ """
+ Compute bn's P(X|e) by variable elimination. [Figure 13.12]
+ >>> elimination_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary
+ ... ).show_approx()
+ 'False: 0.716, True: 0.284'
+ """
+ assert X not in e, "Query variable must be distinct from evidence"
+ factors = []
+ for var in reversed(bn.variables):
+ factors.append(make_factor(var, e, bn))
+ if is_hidden(var, X, e):
+ factors = sum_out(var, factors, bn)
+ return pointwise_product(factors, bn).normalize()
+
+
+def is_hidden(var, X, e):
+ """Is var a hidden variable when querying P(X|e)?"""
+ return var != X and var not in e
+
+
+def make_factor(var, e, bn):
+ """
+ Return the factor for var in bn's joint distribution given e.
+ That is, bn's full joint distribution, projected to accord with e,
+ is the pointwise product of these factors for bn's variables.
+ """
+ node = bn.variable_node(var)
+ variables = [X for X in [var] + node.parents if X not in e]
+ cpt = {event_values(e1, variables): node.p(e1[var], e1)
+ for e1 in all_events(variables, bn, e)}
+ return Factor(variables, cpt)
+
+
+def pointwise_product(factors, bn):
+ return reduce(lambda f, g: f.pointwise_product(g, bn), factors)
+
+
+def sum_out(var, factors, bn):
+ """Eliminate var from all factors by summing over its values."""
+ result, var_factors = [], []
+ for f in factors:
+ (var_factors if var in f.variables else result).append(f)
+ result.append(pointwise_product(var_factors, bn).sum_out(var, bn))
+ return result
+
+
+class Factor:
+ """A factor in a joint distribution."""
+
+ def __init__(self, variables, cpt):
+ self.variables = variables
+ self.cpt = cpt
+
+ def pointwise_product(self, other, bn):
+ """Multiply two factors, combining their variables."""
+ variables = list(set(self.variables) | set(other.variables))
+ cpt = {event_values(e, variables): self.p(e) * other.p(e)
+ for e in all_events(variables, bn, {})}
+ return Factor(variables, cpt)
+
+ def sum_out(self, var, bn):
+ """Make a factor eliminating var by summing over its values."""
+ variables = [X for X in self.variables if X != var]
+ cpt = {event_values(e, variables): sum(self.p(extend(e, var, val))
+ for val in bn.variable_values(var))
+ for e in all_events(variables, bn, {})}
+ return Factor(variables, cpt)
+
+ def normalize(self):
+ """Return my probabilities; must be down to one variable."""
+ assert len(self.variables) == 1
+ return ProbDist(self.variables[0],
+ {k: v for ((k,), v) in self.cpt.items()})
+
+ def p(self, e):
+ """Look up my value tabulated for e."""
+ return self.cpt[event_values(e, self.variables)]
+
+
+def all_events(variables, bn, e):
+ """Yield every way of extending e with values for all variables."""
+ if not variables:
+ yield e
+ else:
+ X, rest = variables[0], variables[1:]
+ for e1 in all_events(rest, bn, e):
+ for x in bn.variable_values(X):
+ yield extend(e1, X, x)
+
+
+# ______________________________________________________________________________
+# 13.3.4 Clustering algorithms
+# [Figure 13.14a]: sprinkler network
+
+
+sprinkler = BayesNet([
+ ('Cloudy', '', 0.5),
+ ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}),
+ ('Rain', 'Cloudy', {T: 0.80, F: 0.20}),
+ ('WetGrass', 'Sprinkler Rain',
+ {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})])
+
+
+# ______________________________________________________________________________
+# 13.4 Approximate Inference for Bayesian Networks
+# 13.4.1 Direct sampling methods
+
+
+def prior_sample(bn):
+ """
+ Randomly sample from bn's full joint distribution. The result
+ is a {variable: value} dict. [Figure 13.15]
+ """
+ event = {}
+ for node in bn.nodes:
+ event[node.variable] = node.sample(event)
+ return event
+
+
+# _________________________________________________________________________
+
+
+def rejection_sampling(X, e, bn, N=10000):
+ """
+ [Figure 13.16]
+ Estimate the probability distribution of variable X given
+ evidence e in BayesNet bn, using N samples.
+ Raises a ZeroDivisionError if all the N samples are rejected,
+ i.e., inconsistent with e.
+ >>> random.seed(47)
+ >>> rejection_sampling('Burglary', dict(JohnCalls=T, MaryCalls=T),
+ ... burglary, 10000).show_approx()
+ 'False: 0.7, True: 0.3'
+ """
+ counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 13.16]
+ for j in range(N):
+ sample = prior_sample(bn) # boldface x in [Figure 13.16]
+ if consistent_with(sample, e):
+ counts[sample[X]] += 1
+ return ProbDist(X, counts)
+
+
+def consistent_with(event, evidence):
+ """Is event consistent with the given evidence?"""
+ return all(evidence.get(k, v) == v
+ for k, v in event.items())
+
+
+# _________________________________________________________________________
+
+
+def likelihood_weighting(X, e, bn, N=10000):
+ """
+ [Figure 13.17]
+ Estimate the probability distribution of variable X given
+ evidence e in BayesNet bn.
+ >>> random.seed(1017)
+ >>> likelihood_weighting('Burglary', dict(JohnCalls=T, MaryCalls=T),
+ ... burglary, 10000).show_approx()
+ 'False: 0.702, True: 0.298'
+ """
+
+ W = {x: 0 for x in bn.variable_values(X)}
+ for j in range(N):
+ sample, weight = weighted_sample(bn, e) # boldface x, w in [Figure 14.15]
+ W[sample[X]] += weight
+ return ProbDist(X, W)
+
+
+def weighted_sample(bn, e):
+ """
+ Sample an event from bn that's consistent with the evidence e;
+ return the event and its weight, the likelihood that the event
+ accords to the evidence.
+ """
+
+ w = 1
+ event = dict(e) # boldface x in [Figure 13.17]
+ for node in bn.nodes:
+ Xi = node.variable
+ if Xi in e:
+ w *= node.p(e[Xi], event)
+ else:
+ event[Xi] = node.sample(event)
+ return event, w
+
+
+# _________________________________________________________________________
+# 13.4.2 Inference by Markov chain simulation
+
+
+def gibbs_ask(X, e, bn, N=1000):
+ """[Figure 13.19]"""
+ assert X not in e, "Query variable must be distinct from evidence"
+ counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.16]
+ Z = [var for var in bn.variables if var not in e]
+ state = dict(e) # boldface x in [Figure 14.16]
+ for Zi in Z:
+ state[Zi] = random.choice(bn.variable_values(Zi))
+ for j in range(N):
+ for Zi in Z:
+ state[Zi] = markov_blanket_sample(Zi, state, bn)
+ counts[state[X]] += 1
+ return ProbDist(X, counts)
+
+
+def markov_blanket_sample(X, e, bn):
+ """
+ Return a sample from P(X | mb) where mb denotes that the
+ variables in the Markov blanket of X take their values from event
+ e (which must assign a value to each). The Markov blanket of X is
+ X's parents, children, and children's parents.
+ """
+ Xnode = bn.variable_node(X)
+ Q = ProbDist(X)
+ for xi in bn.variable_values(X):
+ ei = extend(e, X, xi)
+ # [Equation 13.12:]
+ Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei)
+ for Yj in Xnode.children)
+ # (assuming a Boolean variable here)
+ return probability(Q.normalize()[True])
+
+
+# _________________________________________________________________________
+# 13.4.3 Compiling approximate inference
+
+
+class complied_burglary:
+ """compiled version of burglary network"""
+
+ def Burglary(self, sample):
+ if sample['Alarm']:
+ if sample['Earthquake']:
+ return probability(0.00327)
+ else:
+ return probability(0.485)
+ else:
+ if sample['Earthquake']:
+ return probability(7.05e-05)
+ else:
+ return probability(6.01e-05)
+
+ def Earthquake(self, sample):
+ if sample['Alarm']:
+ if sample['Burglary']:
+ return probability(0.0020212)
+ else:
+ return probability(0.36755)
+ else:
+ if sample['Burglary']:
+ return probability(0.0016672)
+ else:
+ return probability(0.0014222)
+
+ def MaryCalls(self, sample):
+ if sample['Alarm']:
+ return probability(0.7)
+ else:
+ return probability(0.01)
+
+ def JongCalls(self, sample):
+ if sample['Alarm']:
+ return probability(0.9)
+ else:
+ return probability(0.05)
+
+ def Alarm(self, sample):
+ raise NotImplementedError
diff --git a/pytest.ini b/pytest.ini
new file mode 100644
index 000000000..1561b6fe6
--- /dev/null
+++ b/pytest.ini
@@ -0,0 +1,5 @@
+[pytest]
+filterwarnings =
+ ignore::DeprecationWarning
+ ignore::UserWarning
+ ignore::RuntimeWarning
diff --git a/reinforcement_learning.ipynb b/reinforcement_learning.ipynb
new file mode 100644
index 000000000..ee3b6a5eb
--- /dev/null
+++ b/reinforcement_learning.ipynb
@@ -0,0 +1,644 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Reinforcement Learning\n",
+ "\n",
+ "This Jupyter notebook acts as supporting material for **Chapter 21 Reinforcement Learning** of the book* Artificial Intelligence: A Modern Approach*. This notebook makes use of the implementations in `rl.py` module. We also make use of implementation of MDPs in the `mdp.py` module to test our agents. It might be helpful if you have already gone through the Jupyter notebook dealing with Markov decision process. Let us import everything from the `rl` module. It might be helpful to view the source of some of our implementations. Please refer to the Introductory Jupyter notebook for more details."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "from reinforcement_learning import *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## CONTENTS\n",
+ "\n",
+ "* Overview\n",
+ "* Passive Reinforcement Learning\n",
+ " - Direct Utility Estimation\n",
+ " - Adaptive Dynamic Programming\n",
+ " - Temporal-Difference Agent\n",
+ "* Active Reinforcement Learning\n",
+ " - Q learning"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "## OVERVIEW\n",
+ "\n",
+ "Before we start playing with the actual implementations let us review a couple of things about RL.\n",
+ "\n",
+ "1. Reinforcement Learning is concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. \n",
+ "\n",
+ "2. Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Further, there is a focus on on-line performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).\n",
+ "\n",
+ "-- Source: [Wikipedia](https://en.wikipedia.org/wiki/Reinforcement_learning)\n",
+ "\n",
+ "In summary we have a sequence of state action transitions with rewards associated with some states. Our goal is to find the optimal policy $\\pi$ which tells us what action to take in each state."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## PASSIVE REINFORCEMENT LEARNING\n",
+ "\n",
+ "In passive Reinforcement Learning the agent follows a fixed policy $\\pi$. Passive learning attempts to evaluate the given policy $pi$ - without any knowledge of the Reward function $R(s)$ and the Transition model $P(s'\\ |\\ s, a)$.\n",
+ "\n",
+ "This is usually done by some method of **utility estimation**. The agent attempts to directly learn the utility of each state that would result from following the policy. Note that at each step, it has to *perceive* the reward and the state - it has no global knowledge of these. Thus, if a certain the entire set of actions offers a very low probability of attaining some state $s_+$ - the agent may never perceive the reward $R(s_+)$.\n",
+ "\n",
+ "Consider a situation where an agent is given a policy to follow. Thus, at any point it knows only its current state and current reward, and the action it must take next. This action may lead it to more than one state, with different probabilities.\n",
+ "\n",
+ "For a series of actions given by $\\pi$, the estimated utility $U$:\n",
+ "$$U^{\\pi}(s) = E(\\sum_{t=0}^\\inf \\gamma^t R^t(s')$$)\n",
+ "Or the expected value of summed discounted rewards until termination.\n",
+ "\n",
+ "Based on this concept, we discuss three methods of estimating utility:\n",
+ "\n",
+ "1. **Direct Utility Estimation (DUE)**\n",
+ " \n",
+ " The first, most naive method of estimating utility comes from the simplest interpretation of the above definition. We construct an agent that follows the policy until it reaches the terminal state. At each step, it logs its current state, reward. Once it reaches the terminal state, it can estimate the utility for each state for *that* iteration, by simply summing the discounted rewards from that state to the terminal one.\n",
+ "\n",
+ " It can now run this 'simulation' $n$ times, and calculate the average utility of each state. If a state occurs more than once in a simulation, both its utility values are counted separately.\n",
+ " \n",
+ " Note that this method may be prohibitively slow for very large statespaces. Besides, **it pays no attention to the transition probability $P(s'\\ |\\ s, a)$.** It misses out on information that it is capable of collecting (say, by recording the number of times an action from one state led to another state). The next method addresses this issue.\n",
+ " \n",
+ "2. **Adaptive Dynamic Programming (ADP)**\n",
+ " \n",
+ " This method makes use of knowledge of the past state $s$, the action $a$, and the new perceived state $s'$ to estimate the transition probability $P(s'\\ |\\ s,a)$. It does this by the simple counting of new states resulting from previous states and actions.
\n",
+ " The program runs through the policy a number of times, keeping track of:\n",
+ " - each occurrence of state $s$ and the policy-recommended action $a$ in $N_{sa}$\n",
+ " - each occurrence of $s'$ resulting from $a$ on $s$ in $N_{s'|sa}$.\n",
+ " \n",
+ " It can thus estimate $P(s'\\ |\\ s,a)$ as $N_{s'|sa}/N_{sa}$, which in the limit of infinite trials, will converge to the true value.
\n",
+ " Using the transition probabilities thus estimated, it can apply `POLICY-EVALUATION` to estimate the utilities $U(s)$ using properties of convergence of the Bellman functions.\n",
+ "\n",
+ "3. **Temporal-difference learning (TD)**\n",
+ " \n",
+ " Instead of explicitly building the transition model $P$, the temporal-difference model makes use of the expected closeness between the utilities of two consecutive states $s$ and $s'$.\n",
+ " For the transition $s$ to $s'$, the update is written as:\n",
+ "$$U^{\\pi}(s) \\leftarrow U^{\\pi}(s) + \\alpha \\left( R(s) + \\gamma U^{\\pi}(s') - U^{\\pi}(s) \\right)$$\n",
+ " This model implicitly incorporates the transition probabilities by being weighed for each state by the number of times it is achieved from the current state. Thus, over a number of iterations, it converges similarly to the Bellman equations.\n",
+ " The advantage of the TD learning model is its relatively simple computation at each step, rather than having to keep track of various counts.\n",
+ " For $n_s$ states and $n_a$ actions the ADP model would have $n_s \\times n_a$ numbers $N_{sa}$ and $n_s^2 \\times n_a$ numbers $N_{s'|sa}$ to keep track of. The TD model must only keep track of a utility $U(s)$ for each state."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Demonstrating Passive agents\n",
+ "\n",
+ "Passive agents are implemented in `rl.py` as various `Agent-Class`es.\n",
+ "\n",
+ "To demonstrate these agents, we make use of the `GridMDP` object from the `MDP` module. `sequential_decision_environment` is similar to that used for the `MDP` notebook but has discounting with $\\gamma = 0.9$.\n",
+ "\n",
+ "The `Agent-Program` can be obtained by creating an instance of the relevant `Agent-Class`. The `__call__` method allows the `Agent-Class` to be called as a function. The class needs to be instantiated with a policy ($\\pi$) and an `MDP` whose utility of states will be estimated."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "from mdp import sequential_decision_environment"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The `sequential_decision_environment` is a GridMDP object as shown below. The rewards are **+1** and **-1** in the terminal states, and **-0.04** in the rest. ![](\"files/images/mdp.png\")
Now we define actions and a policy similar to **Fig 21.1** in the book."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "# Action Directions\n",
+ "north = (0, 1)\n",
+ "south = (0,-1)\n",
+ "west = (-1, 0)\n",
+ "east = (1, 0)\n",
+ "\n",
+ "policy = {\n",
+ " (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None,\n",
+ " (0, 1): north, (2, 1): north, (3, 1): None,\n",
+ " (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, \n",
+ "}\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Direction Utility Estimation Agent\n",
+ "\n",
+ "The `PassiveDEUAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveDEUAgent` sums over rewards to find the estimated utility for each state. It thus requires the running of a number of iterations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "%psource PassiveDUEAgent"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "DUEagent = PassiveDUEAgent(policy, sequential_decision_environment)\n",
+ "for i in range(200):\n",
+ " run_single_trial(DUEagent, sequential_decision_environment)\n",
+ " DUEagent.estimate_U()\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The calculated utilities are:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('\\n'.join([str(k)+':'+str(v) for k, v in DUEagent.U.items()]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Adaptive Dynamic Programming Agent\n",
+ "\n",
+ "The `PassiveADPAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveADPAgent` uses state transition and occurrence counts to estimate $P$, and then $U$. Go through the source below to understand the agent."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "%psource PassiveADPAgent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We instantiate a `PassiveADPAgent` below with the `GridMDP` shown and train it over 200 iterations. The `rl` module has a simple implementation to simulate iterations. The function is called **run_single_trial**."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "ADPagent = PassiveADPAgent(policy, sequential_decision_environment)\n",
+ "for i in range(200):\n",
+ " run_single_trial(ADPagent, sequential_decision_environment)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The calculated utilities are:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "print('\\n'.join([str(k)+':'+str(v) for k, v in ADPagent.U.items()]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Passive Temporal Difference Agent\n",
+ "\n",
+ "`PassiveTDAgent` uses temporal differences to learn utility estimates. We learn the difference between the states and backup the values to previous states. Let us look into the source before we see some usage examples."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "%psource PassiveTDAgent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In creating the `TDAgent`, we use the **same learning rate** $\\alpha$ as given in the footnote of the book on **page 837**."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "TDagent = PassiveTDAgent(policy, sequential_decision_environment, alpha = lambda n: 60./(59+n))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we run **200 trials** for the agent to estimate Utilities."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "for i in range(200):\n",
+ " run_single_trial(TDagent,sequential_decision_environment)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The calculated utilities are:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('\\n'.join([str(k)+':'+str(v) for k, v in TDagent.U.items()]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Comparison with value iteration method\n",
+ "\n",
+ "We can also compare the utility estimates learned by our agent to those obtained via **value iteration**.\n",
+ "\n",
+ "**Note that value iteration has a priori knowledge of the transition table $P$, the rewards $R$, and all the states $s$.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "from mdp import value_iteration"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The values calculated by value iteration:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "U_values = value_iteration(sequential_decision_environment)\n",
+ "print('\\n'.join([str(k)+':'+str(v) for k, v in U_values.items()]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Evolution of utility estimates over iterations\n",
+ "\n",
+ "We can explore how these estimates vary with time by using plots similar to **Fig 21.5a**. We will first enable matplotlib using the inline backend. We also define a function to collect the values of utilities at each iteration."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "%matplotlib inline\n",
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "def graph_utility_estimates(agent_program, mdp, no_of_iterations, states_to_graph):\n",
+ " graphs = {state:[] for state in states_to_graph}\n",
+ " for iteration in range(1,no_of_iterations+1):\n",
+ " run_single_trial(agent_program, mdp)\n",
+ " for state in states_to_graph:\n",
+ " graphs[state].append((iteration, agent_program.U[state]))\n",
+ " for state, value in graphs.items():\n",
+ " state_x, state_y = zip(*value)\n",
+ " plt.plot(state_x, state_y, label=str(state))\n",
+ " plt.ylim([0,1.2])\n",
+ " plt.legend(loc='lower right')\n",
+ " plt.xlabel('Iterations')\n",
+ " plt.ylabel('U')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here is a plot of state $(2,2)$."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n))\n",
+ "graph_utility_estimates(agent, sequential_decision_environment, 500, [(2,2)])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is also possible to plot multiple states on the same plot. As expected, the utility of the finite state $(3,2)$ stays constant and is equal to $R((3,2)) = 1$."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "graph_utility_estimates(agent, sequential_decision_environment, 500, [(2,2), (3,2)])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "## ACTIVE REINFORCEMENT LEARNING\n",
+ "\n",
+ "Unlike Passive Reinforcement Learning in Active Reinforcement Learning we are not bound by a policy pi and we need to select our actions. In other words the agent needs to learn an optimal policy. The fundamental tradeoff the agent needs to face is that of exploration vs. exploitation. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### QLearning Agent\n",
+ "\n",
+ "The QLearningAgent class in the rl module implements the Agent Program described in **Fig 21.8** of the AIMA Book. In Q-Learning the agent learns an action-value function Q which gives the utility of taking a given action in a particular state. Q-Learning does not required a transition model and hence is a model free method. Let us look into the source before we see some usage examples."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "%psource QLearningAgent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The Agent Program can be obtained by creating the instance of the class by passing the appropriate parameters. Because of the __ call __ method the object that is created behaves like a callable and returns an appropriate action as most Agent Programs do. To instantiate the object we need a mdp similar to the PassiveTDAgent.\n",
+ "\n",
+ " Let us use the same GridMDP object we used above. **Figure 17.1 (sequential_decision_environment)** is similar to **Figure 21.1** but has some discounting as **gamma = 0.9**. The class also implements an exploration function **f** which returns fixed **Rplus** until agent has visited state, action **Ne** number of times. This is the same as the one defined on page **842** of the book. The method **actions_in_state** returns actions possible in given state. It is useful when applying max and argmax operations."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let us create our object now. We also use the **same alpha** as given in the footnote of the book on **page 837**. We use **Rplus = 2** and **Ne = 5** as defined on page 843. **Fig 21.7** "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, \n",
+ " alpha=lambda n: 60./(59+n))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now to try out the q_agent we make use of the **run_single_trial** function in rl.py (which was also used above). Let us use **200** iterations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "for i in range(200):\n",
+ " run_single_trial(q_agent,sequential_decision_environment)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let us see the Q Values. The keys are state-action pairs. Where different actions correspond according to:\n",
+ "\n",
+ "north = (0, 1)\n",
+ "south = (0,-1)\n",
+ "west = (-1, 0)\n",
+ "east = (1, 0)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "q_agent.Q"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The Utility **U** of each state is related to **Q** by the following equation.\n",
+ "\n",
+ "**U (s) = max a Q(s, a)**\n",
+ "\n",
+ "Let us convert the Q Values above into U estimates.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "U = defaultdict(lambda: -1000.) # Very Large Negative Value for Comparison see below.\n",
+ "for state_action, value in q_agent.Q.items():\n",
+ " state, action = state_action\n",
+ " if U[state] < value:\n",
+ " U[state] = value"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "U"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let us finally compare these estimates to value_iteration results."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print(value_iteration(sequential_decision_environment))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
+ },
+ "pycharm": {
+ "stem_cell": {
+ "cell_type": "raw",
+ "source": [],
+ "metadata": {
+ "collapsed": false
+ }
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
\ No newline at end of file
diff --git a/reinforcement_learning.py b/reinforcement_learning.py
new file mode 100644
index 000000000..4cb91af0f
--- /dev/null
+++ b/reinforcement_learning.py
@@ -0,0 +1,337 @@
+"""Reinforcement Learning (Chapter 21)"""
+
+import random
+from collections import defaultdict
+
+from mdp import MDP, policy_evaluation
+
+
+class PassiveDUEAgent:
+ """
+ Passive (non-learning) agent that uses direct utility estimation
+ on a given MDP and policy.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveDUEAgent(policy, sequential_decision_environment)
+ for i in range(200):
+ run_single_trial(agent,sequential_decision_environment)
+ agent.estimate_U()
+ agent.U[(0, 0)] > 0.2
+ True
+ """
+
+ def __init__(self, pi, mdp):
+ self.pi = pi
+ self.mdp = mdp
+ self.U = {}
+ self.s = None
+ self.a = None
+ self.s_history = []
+ self.r_history = []
+ self.init = mdp.init
+
+ def __call__(self, percept):
+ s1, r1 = percept
+ self.s_history.append(s1)
+ self.r_history.append(r1)
+ ##
+ ##
+ if s1 in self.mdp.terminals:
+ self.s = self.a = None
+ else:
+ self.s, self.a = s1, self.pi[s1]
+ return self.a
+
+ def estimate_U(self):
+ # this function can be called only if the MDP has reached a terminal state
+ # it will also reset the mdp history
+ assert self.a is None, 'MDP is not in terminal state'
+ assert len(self.s_history) == len(self.r_history)
+ # calculating the utilities based on the current iteration
+ U2 = {s: [] for s in set(self.s_history)}
+ for i in range(len(self.s_history)):
+ s = self.s_history[i]
+ U2[s] += [sum(self.r_history[i:])]
+ U2 = {k: sum(v) / max(len(v), 1) for k, v in U2.items()}
+ # resetting history
+ self.s_history, self.r_history = [], []
+ # setting the new utilities to the average of the previous
+ # iteration and this one
+ for k in U2.keys():
+ if k in self.U.keys():
+ self.U[k] = (self.U[k] + U2[k]) / 2
+ else:
+ self.U[k] = U2[k]
+ return self.U
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)"""
+ return percept
+
+
+class PassiveADPAgent:
+ """
+ [Figure 21.2]
+ Passive (non-learning) agent that uses adaptive dynamic programming
+ on a given MDP and policy.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveADPAgent(policy, sequential_decision_environment)
+ for i in range(100):
+ run_single_trial(agent,sequential_decision_environment)
+
+ agent.U[(0, 0)] > 0.2
+ True
+ agent.U[(0, 1)] > 0.2
+ True
+ """
+
+ class ModelMDP(MDP):
+ """Class for implementing modified Version of input MDP with
+ an editable transition model P and a custom function T."""
+
+ def __init__(self, init, actlist, terminals, gamma, states):
+ super().__init__(init, actlist, terminals, states=states, gamma=gamma)
+ nested_dict = lambda: defaultdict(nested_dict)
+ # StackOverflow:whats-the-best-way-to-initialize-a-dict-of-dicts-in-python
+ self.P = nested_dict()
+
+ def T(self, s, a):
+ """Return a list of tuples with probabilities for states
+ based on the learnt model P."""
+ return [(prob, res) for (res, prob) in self.P[(s, a)].items()]
+
+ def __init__(self, pi, mdp):
+ self.pi = pi
+ self.mdp = PassiveADPAgent.ModelMDP(mdp.init, mdp.actlist,
+ mdp.terminals, mdp.gamma, mdp.states)
+ self.U = {}
+ self.Nsa = defaultdict(int)
+ self.Ns1_sa = defaultdict(int)
+ self.s = None
+ self.a = None
+ self.visited = set() # keeping track of visited states
+
+ def __call__(self, percept):
+ s1, r1 = percept
+ mdp = self.mdp
+ R, P, terminals, pi = mdp.reward, mdp.P, mdp.terminals, self.pi
+ s, a, Nsa, Ns1_sa, U = self.s, self.a, self.Nsa, self.Ns1_sa, self.U
+
+ if s1 not in self.visited: # Reward is only known for visited state.
+ U[s1] = R[s1] = r1
+ self.visited.add(s1)
+ if s is not None:
+ Nsa[(s, a)] += 1
+ Ns1_sa[(s1, s, a)] += 1
+ # for each t such that Ns′|sa [t, s, a] is nonzero
+ for t in [res for (res, state, act), freq in Ns1_sa.items()
+ if (state, act) == (s, a) and freq != 0]:
+ P[(s, a)][t] = Ns1_sa[(t, s, a)] / Nsa[(s, a)]
+
+ self.U = policy_evaluation(pi, U, mdp)
+ ##
+ ##
+ self.Nsa, self.Ns1_sa = Nsa, Ns1_sa
+ if s1 in terminals:
+ self.s = self.a = None
+ else:
+ self.s, self.a = s1, self.pi[s1]
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+class PassiveTDAgent:
+ """
+ [Figure 21.4]
+ The abstract class for a Passive (non-learning) agent that uses
+ temporal differences to learn utility estimates. Override update_state
+ method to convert percept to state and reward. The mdp being provided
+ should be an instance of a subclass of the MDP Class.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n))
+ for i in range(200):
+ run_single_trial(agent,sequential_decision_environment)
+
+ agent.U[(0, 0)] > 0.2
+ True
+ agent.U[(0, 1)] > 0.2
+ True
+ """
+
+ def __init__(self, pi, mdp, alpha=None):
+
+ self.pi = pi
+ self.U = {s: 0. for s in mdp.states}
+ self.Ns = {s: 0 for s in mdp.states}
+ self.s = None
+ self.a = None
+ self.r = None
+ self.gamma = mdp.gamma
+ self.terminals = mdp.terminals
+
+ if alpha:
+ self.alpha = alpha
+ else:
+ self.alpha = lambda n: 1 / (1 + n) # udacity video
+
+ def __call__(self, percept):
+ s1, r1 = self.update_state(percept)
+ pi, U, Ns, s, r = self.pi, self.U, self.Ns, self.s, self.r
+ alpha, gamma, terminals = self.alpha, self.gamma, self.terminals
+ if not Ns[s1]:
+ U[s1] = r1
+ if s is not None:
+ Ns[s] += 1
+ U[s] += alpha(Ns[s]) * (r + gamma * U[s1] - U[s])
+ if s1 in terminals:
+ self.s = self.a = self.r = None
+ else:
+ self.s, self.a, self.r = s1, pi[s1], r1
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+class QLearningAgent:
+ """
+ [Figure 21.8]
+ An exploratory Q-learning agent. It avoids having to learn the transition
+ model because the Q-value of a state can be related directly to those of
+ its neighbors.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60./(59+n))
+ for i in range(200):
+ run_single_trial(q_agent,sequential_decision_environment)
+
+ q_agent.Q[((0, 1), (0, 1))] >= -0.5
+ True
+ q_agent.Q[((1, 0), (0, -1))] <= 0.5
+ True
+ """
+
+ def __init__(self, mdp, Ne, Rplus, alpha=None):
+
+ self.gamma = mdp.gamma
+ self.terminals = mdp.terminals
+ self.all_act = mdp.actlist
+ self.Ne = Ne # iteration limit in exploration function
+ self.Rplus = Rplus # large value to assign before iteration limit
+ self.Q = defaultdict(float)
+ self.Nsa = defaultdict(float)
+ self.s = None
+ self.a = None
+ self.r = None
+
+ if alpha:
+ self.alpha = alpha
+ else:
+ self.alpha = lambda n: 1. / (1 + n) # udacity video
+
+ def f(self, u, n):
+ """Exploration function. Returns fixed Rplus until
+ agent has visited state, action a Ne number of times.
+ Same as ADP agent in book."""
+ if n < self.Ne:
+ return self.Rplus
+ else:
+ return u
+
+ def actions_in_state(self, state):
+ """Return actions possible in given state.
+ Useful for max and argmax."""
+ if state in self.terminals:
+ return [None]
+ else:
+ return self.all_act
+
+ def __call__(self, percept):
+ s1, r1 = self.update_state(percept)
+ Q, Nsa, s, a, r = self.Q, self.Nsa, self.s, self.a, self.r
+ alpha, gamma, terminals = self.alpha, self.gamma, self.terminals,
+ actions_in_state = self.actions_in_state
+
+ if s in terminals:
+ Q[s, None] = r1
+ if s is not None:
+ Nsa[s, a] += 1
+ Q[s, a] += alpha(Nsa[s, a]) * (r + gamma * max(Q[s1, a1]
+ for a1 in actions_in_state(s1)) - Q[s, a])
+ if s in terminals:
+ self.s = self.a = self.r = None
+ else:
+ self.s, self.r = s1, r1
+ self.a = max(actions_in_state(s1), key=lambda a1: self.f(Q[s1, a1], Nsa[s1, a1]))
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+def run_single_trial(agent_program, mdp):
+ """Execute trial for given agent_program
+ and mdp. mdp should be an instance of subclass
+ of mdp.MDP """
+
+ def take_single_action(mdp, s, a):
+ """
+ Select outcome of taking action a
+ in state s. Weighted Sampling.
+ """
+ x = random.uniform(0, 1)
+ cumulative_probability = 0.0
+ for probability_state in mdp.T(s, a):
+ probability, state = probability_state
+ cumulative_probability += probability
+ if x < cumulative_probability:
+ break
+ return state
+
+ current_state = mdp.init
+ while True:
+ current_reward = mdp.R(current_state)
+ percept = (current_state, current_reward)
+ next_action = agent_program(percept)
+ if next_action is None:
+ break
+ current_state = take_single_action(mdp, current_state, next_action)
diff --git a/reinforcement_learning4e.py b/reinforcement_learning4e.py
new file mode 100644
index 000000000..eaaba3e5a
--- /dev/null
+++ b/reinforcement_learning4e.py
@@ -0,0 +1,353 @@
+"""Reinforcement Learning (Chapter 21)"""
+
+import random
+from collections import defaultdict
+
+from mdp4e import MDP, policy_evaluation
+
+
+# _________________________________________
+# 21.2 Passive Reinforcement Learning
+# 21.2.1 Direct utility estimation
+
+
+class PassiveDUEAgent:
+ """
+ Passive (non-learning) agent that uses direct utility estimation
+ on a given MDP and policy.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveDUEAgent(policy, sequential_decision_environment)
+ for i in range(200):
+ run_single_trial(agent,sequential_decision_environment)
+ agent.estimate_U()
+ agent.U[(0, 0)] > 0.2
+ True
+ """
+
+ def __init__(self, pi, mdp):
+ self.pi = pi
+ self.mdp = mdp
+ self.U = {}
+ self.s = None
+ self.a = None
+ self.s_history = []
+ self.r_history = []
+ self.init = mdp.init
+
+ def __call__(self, percept):
+ s1, r1 = percept
+ self.s_history.append(s1)
+ self.r_history.append(r1)
+ ##
+ ##
+ if s1 in self.mdp.terminals:
+ self.s = self.a = None
+ else:
+ self.s, self.a = s1, self.pi[s1]
+ return self.a
+
+ def estimate_U(self):
+ # this function can be called only if the MDP has reached a terminal state
+ # it will also reset the mdp history
+ assert self.a is None, 'MDP is not in terminal state'
+ assert len(self.s_history) == len(self.r_history)
+ # calculating the utilities based on the current iteration
+ U2 = {s: [] for s in set(self.s_history)}
+ for i in range(len(self.s_history)):
+ s = self.s_history[i]
+ U2[s] += [sum(self.r_history[i:])]
+ U2 = {k: sum(v) / max(len(v), 1) for k, v in U2.items()}
+ # resetting history
+ self.s_history, self.r_history = [], []
+ # setting the new utilities to the average of the previous
+ # iteration and this one
+ for k in U2.keys():
+ if k in self.U.keys():
+ self.U[k] = (self.U[k] + U2[k]) / 2
+ else:
+ self.U[k] = U2[k]
+ return self.U
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)"""
+ return percept
+
+
+# 21.2.2 Adaptive dynamic programming
+
+
+class PassiveADPAgent:
+ """
+ [Figure 21.2]
+ Passive (non-learning) agent that uses adaptive dynamic programming
+ on a given MDP and policy.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveADPAgent(policy, sequential_decision_environment)
+ for i in range(100):
+ run_single_trial(agent,sequential_decision_environment)
+
+ agent.U[(0, 0)] > 0.2
+ True
+ agent.U[(0, 1)] > 0.2
+ True
+ """
+
+ class ModelMDP(MDP):
+ """Class for implementing modified Version of input MDP with
+ an editable transition model P and a custom function T."""
+
+ def __init__(self, init, actlist, terminals, gamma, states):
+ super().__init__(init, actlist, terminals, states=states, gamma=gamma)
+ nested_dict = lambda: defaultdict(nested_dict)
+ # StackOverflow:whats-the-best-way-to-initialize-a-dict-of-dicts-in-python
+ self.P = nested_dict()
+
+ def T(self, s, a):
+ """Return a list of tuples with probabilities for states
+ based on the learnt model P."""
+ return [(prob, res) for (res, prob) in self.P[(s, a)].items()]
+
+ def __init__(self, pi, mdp):
+ self.pi = pi
+ self.mdp = PassiveADPAgent.ModelMDP(mdp.init, mdp.actlist,
+ mdp.terminals, mdp.gamma, mdp.states)
+ self.U = {}
+ self.Nsa = defaultdict(int)
+ self.Ns1_sa = defaultdict(int)
+ self.s = None
+ self.a = None
+ self.visited = set() # keeping track of visited states
+
+ def __call__(self, percept):
+ s1, r1 = percept
+ mdp = self.mdp
+ R, P, terminals, pi = mdp.reward, mdp.P, mdp.terminals, self.pi
+ s, a, Nsa, Ns1_sa, U = self.s, self.a, self.Nsa, self.Ns1_sa, self.U
+
+ if s1 not in self.visited: # Reward is only known for visited state.
+ U[s1] = R[s1] = r1
+ self.visited.add(s1)
+ if s is not None:
+ Nsa[(s, a)] += 1
+ Ns1_sa[(s1, s, a)] += 1
+ # for each t such that Ns′|sa [t, s, a] is nonzero
+ for t in [res for (res, state, act), freq in Ns1_sa.items()
+ if (state, act) == (s, a) and freq != 0]:
+ P[(s, a)][t] = Ns1_sa[(t, s, a)] / Nsa[(s, a)]
+
+ self.U = policy_evaluation(pi, U, mdp)
+ ##
+ ##
+ self.Nsa, self.Ns1_sa = Nsa, Ns1_sa
+ if s1 in terminals:
+ self.s = self.a = None
+ else:
+ self.s, self.a = s1, self.pi[s1]
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+# 21.2.3 Temporal-difference learning
+
+
+class PassiveTDAgent:
+ """
+ [Figure 21.4]
+ The abstract class for a Passive (non-learning) agent that uses
+ temporal differences to learn utility estimates. Override update_state
+ method to convert percept to state and reward. The mdp being provided
+ should be an instance of a subclass of the MDP Class.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n))
+ for i in range(200):
+ run_single_trial(agent,sequential_decision_environment)
+
+ agent.U[(0, 0)] > 0.2
+ True
+ agent.U[(0, 1)] > 0.2
+ True
+ """
+
+ def __init__(self, pi, mdp, alpha=None):
+
+ self.pi = pi
+ self.U = {s: 0. for s in mdp.states}
+ self.Ns = {s: 0 for s in mdp.states}
+ self.s = None
+ self.a = None
+ self.r = None
+ self.gamma = mdp.gamma
+ self.terminals = mdp.terminals
+
+ if alpha:
+ self.alpha = alpha
+ else:
+ self.alpha = lambda n: 1 / (1 + n) # udacity video
+
+ def __call__(self, percept):
+ s1, r1 = self.update_state(percept)
+ pi, U, Ns, s, r = self.pi, self.U, self.Ns, self.s, self.r
+ alpha, gamma, terminals = self.alpha, self.gamma, self.terminals
+ if not Ns[s1]:
+ U[s1] = r1
+ if s is not None:
+ Ns[s] += 1
+ U[s] += alpha(Ns[s]) * (r + gamma * U[s1] - U[s])
+ if s1 in terminals:
+ self.s = self.a = self.r = None
+ else:
+ self.s, self.a, self.r = s1, pi[s1], r1
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+# __________________________________________
+# 21.3. Active Reinforcement Learning
+# 21.3.2 Learning an action-utility function
+
+
+class QLearningAgent:
+ """
+ [Figure 21.8]
+ An exploratory Q-learning agent. It avoids having to learn the transition
+ model because the Q-value of a state can be related directly to those of
+ its neighbors.
+
+ import sys
+ from mdp import sequential_decision_environment
+ north = (0, 1)
+ south = (0,-1)
+ west = (-1, 0)
+ east = (1, 0)
+ policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
+ (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
+ q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60./(59+n))
+ for i in range(200):
+ run_single_trial(q_agent,sequential_decision_environment)
+
+ q_agent.Q[((0, 1), (0, 1))] >= -0.5
+ True
+ q_agent.Q[((1, 0), (0, -1))] <= 0.5
+ True
+ """
+
+ def __init__(self, mdp, Ne, Rplus, alpha=None):
+
+ self.gamma = mdp.gamma
+ self.terminals = mdp.terminals
+ self.all_act = mdp.actlist
+ self.Ne = Ne # iteration limit in exploration function
+ self.Rplus = Rplus # large value to assign before iteration limit
+ self.Q = defaultdict(float)
+ self.Nsa = defaultdict(float)
+ self.s = None
+ self.a = None
+ self.r = None
+
+ if alpha:
+ self.alpha = alpha
+ else:
+ self.alpha = lambda n: 1. / (1 + n) # udacity video
+
+ def f(self, u, n):
+ """Exploration function. Returns fixed Rplus until
+ agent has visited state, action a Ne number of times.
+ Same as ADP agent in book."""
+ if n < self.Ne:
+ return self.Rplus
+ else:
+ return u
+
+ def actions_in_state(self, state):
+ """Return actions possible in given state.
+ Useful for max and argmax."""
+ if state in self.terminals:
+ return [None]
+ else:
+ return self.all_act
+
+ def __call__(self, percept):
+ s1, r1 = self.update_state(percept)
+ Q, Nsa, s, a, r = self.Q, self.Nsa, self.s, self.a, self.r
+ alpha, gamma, terminals = self.alpha, self.gamma, self.terminals,
+ actions_in_state = self.actions_in_state
+
+ if s in terminals:
+ Q[s, None] = r1
+ if s is not None:
+ Nsa[s, a] += 1
+ Q[s, a] += alpha(Nsa[s, a]) * (r + gamma * max(Q[s1, a1]
+ for a1 in actions_in_state(s1)) - Q[s, a])
+ if s in terminals:
+ self.s = self.a = self.r = None
+ else:
+ self.s, self.r = s1, r1
+ self.a = max(actions_in_state(s1), key=lambda a1: self.f(Q[s1, a1], Nsa[s1, a1]))
+ return self.a
+
+ def update_state(self, percept):
+ """To be overridden in most cases. The default case
+ assumes the percept to be of type (state, reward)."""
+ return percept
+
+
+def run_single_trial(agent_program, mdp):
+ """Execute trial for given agent_program
+ and mdp. mdp should be an instance of subclass
+ of mdp.MDP """
+
+ def take_single_action(mdp, s, a):
+ """
+ Select outcome of taking action a
+ in state s. Weighted Sampling.
+ """
+ x = random.uniform(0, 1)
+ cumulative_probability = 0.0
+ for probability_state in mdp.T(s, a):
+ probability, state = probability_state
+ cumulative_probability += probability
+ if x < cumulative_probability:
+ break
+ return state
+
+ current_state = mdp.init
+ while True:
+ current_reward = mdp.R(current_state)
+ percept = (current_state, current_reward)
+ next_action = agent_program(percept)
+ if next_action is None:
+ break
+ current_state = take_single_action(mdp, current_state, next_action)
diff --git a/requirements.txt b/requirements.txt
index e69de29bb..dd6b1be8a 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -0,0 +1,18 @@
+cvxopt
+image
+ipython
+ipythonblocks
+ipywidgets
+jupyter
+keras
+matplotlib
+networkx
+numpy
+opencv-python
+pandas
+pillow
+pytest-cov
+qpsolvers
+scipy
+sortedcontainers
+tensorflow
\ No newline at end of file
diff --git a/rl.ipynb b/rl.ipynb
deleted file mode 100644
index cc0c0b59e..000000000
--- a/rl.ipynb
+++ /dev/null
@@ -1,354 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "# Reinforcement Learning\n",
- "\n",
- "This IPy notebook acts as supporting material for **Chapter 21 Reinforcement Learning** of the book* Artificial Intelligence: A Modern Approach*. This notebook makes use of the implementations in rl.py module. We also make use of implementation of MDPs in the mdp.py module to test our agents. It might be helpful if you have already gone through the IPy notebook dealing with Markov decision process. Let us import everything from the rl module. It might be helpful to view the source of some of our implementations. Please refer to the Introductory IPy file for more details."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "from rl import *"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
- "source": [
- "## Review\n",
- "Before we start playing with the actual implementations let us review a couple of things about RL.\n",
- "\n",
- "1. Reinforcement Learning is concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. \n",
- "\n",
- "2. Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Further, there is a focus on on-line performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).\n",
- "\n",
- "-- Source: [Wikipedia](https://en.wikipedia.org/wiki/Reinforcement_learning)\n",
- "\n",
- "In summary we have a sequence of state action transitions with rewards associated with some states. Our goal is to find the optimal policy (pi) which tells us what action to take in each state."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Passive Reinforcement Learning\n",
- "\n",
- "In passive Reinforcement Learning the agent follows a fixed policy and tries to learn the Reward function and the Transition model (if it is not aware of that).\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Passive Temporal Difference Agent\n",
- "\n",
- "The PassiveTDAgent class in the rl module implements the Agent Program (notice the usage of word Program) described in **Fig 21.4** of the AIMA Book. PassiveTDAgent uses temporal differences to learn utility estimates. In simple terms we learn the difference between the states and backup the values to previous states while following a fixed policy. Let us look into the source before we see some usage examples."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "%psource PassiveTDAgent"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The Agent Program can be obtained by creating the instance of the class by passing the appropriate parameters. Because of the __ call __ method the object that is created behaves like a callable and returns an appropriate action as most Agent Programs do. To instantiate the object we need a policy(pi) and a mdp whose utility of states will be estimated. Let us import a GridMDP object from the mdp module. **Fig[17, 1]** is similar to **Fig[21, 1]** but has some discounting as **gamma = 0.9**."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "from mdp import Fig"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "data": {
- "text/plain": [
- " Now we define a policy similar to **Fig 21.1** in the book."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "# Action Directions\n",
- "north = (0, 1)\n",
- "south = (0,-1)\n",
- "west = (-1, 0)\n",
- "east = (1, 0)\n",
- "\n",
- "policy = {\n",
- " (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None,\n",
- " (0, 1): north, (2, 1): north, (3, 1): None,\n",
- " (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, \n",
- "}\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Let us create our object now. We also use the **same alpha** as given in the footnote of the book on **page 837**."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "our_agent = PassiveTDAgent(policy, Fig[17,1], alpha=lambda n: 60./(59+n))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The rl module also has a simple implementation to simulate iterations. The function is called **run_single_trial**. Now we can try our implementation. We can also compare the utility estimates learned by our agent to those obtained via **value iteration**.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "from mdp import value_iteration"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The values calculated by value iteration:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{(0, 1): 0.3984432178350045, (1, 2): 0.649585681261095, (3, 2): 1.0, (0, 0): 0.2962883154554812, (3, 0): 0.12987274656746342, (3, 1): -1.0, (2, 1): 0.48644001739269643, (2, 0): 0.3447542300124158, (2, 2): 0.7953620878466678, (1, 0): 0.25386699846479516, (0, 2): 0.5093943765842497}\n"
- ]
- }
- ],
- "source": [
- "print(value_iteration(Fig[17,1]))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now the values estimated by our agent after **200 trials**."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 9,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{(0, 1): 0.43655093803808254, (1, 2): 0.7111433090760988, (3, 2): 1, (0, 0): 0.3220542204171776, (2, 0): 0.0, (3, 0): 0.0, (1, 0): 0.20098994088292488, (3, 1): 0.0, (2, 2): 0.8560074788087413, (2, 1): 0.6639270026362584, (0, 2): 0.5629080090683166}\n"
- ]
- }
- ],
- "source": [
- "for i in range(200):\n",
- " run_single_trial(our_agent,Fig[17,1])\n",
- "print(our_agent.U)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "We can also explore how these estimates vary with time by using plots similar to **Fig 21.5a**. To do so we define a function to help us with the same. We will first enable matplotlib using the inline backend."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%matplotlib inline\n",
- "import matplotlib.pyplot as plt\n",
- "\n",
- "def graph_utility_estimates(agent_program, mdp, no_of_iterations, states_to_graph):\n",
- " graphs = {state:[] for state in states_to_graph}\n",
- " for iteration in range(1,no_of_iterations+1):\n",
- " run_single_trial(agent_program, mdp)\n",
- " for state in states_to_graph:\n",
- " graphs[state].append((iteration, agent_program.U[state]))\n",
- " for state, value in graphs.items():\n",
- " state_x, state_y = zip(*value)\n",
- " plt.plot(state_x, state_y, label=str(state))\n",
- " plt.ylim([0,1.2])\n",
- " plt.legend(loc='lower right')\n",
- " plt.xlabel('Iterations')\n",
- " plt.ylabel('U')"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Here is a plot of state (2,2)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 11,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEPCAYAAACp/QjLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYFNW5h99hFhZngAFZIrs77pq4RFGHEFFxTWJUXJMY\nlyQmepMYNLkxaG5y45WYiMZd401cuCpxQVFj1EEjAjGyyaYoIJswCMgOM1D3j2+OVd1VvU73LN2/\n93n66e6q6urTp6vO73zLOQeEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhChIHgJWAbMT7L8QmAnM\nAt4CDmmmcgkhhGhBjgcOJ7E4fBno0vj6FGBKcxRKCCFEyzOQxOIQpBpYlt+iCCGESId2LV2AAJcB\nE1u6EEIIIZqHgaS2HIYCczHrQQghRAtT1tIFwILQ92Mxh3VRBxx66KHezJkzm7VQQghRAMwEDsvm\ngy3tVuoP/A24CFiY6KCZM2fieZ4ensevfvWrFi9Da3moLlQXqovkD+DQbBvnfFsOjwMnArsDS4Ff\nAeWN++4FbsRcSXc3bqsHjspzmYQQQqQg3+IwMsX+7zY+hBBCtCJa2q0kMqSmpqali9BqUF34qC58\nVBe5oaSlC5AmXqP/TAghRJqUlJRAlu28LAchhBAhJA5CCCFCSByEEEKEkDgIIYQIIXEQQggRQuIg\nhBAihMRBCCFECImDEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQgh\nRAiJgxBCiBASByGEECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUQIiYMQQogQ\nEgchhBAhJA5CCCFC5FscHgJWAbOTHDMW+ACYCRye5/IIIYRIg3yLw5+BU5LsHwHsDewDXAHcnefy\nCCGESIN8i8ObwLok+88E/rfx9VSgK9Arz2USQgiRgpaOOfQBlgbeLwP6tlBZhBBCNNLS4gBQEvfe\na5FSCCGE+JyyFv7+5UC/wPu+jdtCjB49+vPXNTU11NTU5LNcQgjR5qitraW2tjYn54rvteeDgcAE\n4OCIfSOAqxufjwH+2Pgcj+d5MiiEECITSkpKIMt2Pt+Ww+PAicDuWGzhV0B54757gYmYMCwENgPf\nznN5hBBCpEFzWA65QJaDEEJkSFMsh9YQkBZCCNHKkDgIIYQIIXEQQggRQuIghBAihMRBCCFECImD\nEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQghRAiJgxBCiBASByGE\nECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUSIspYuQLqMGQPf/z785jeweDHs\n2gVlZXDXXVBV5R/nebavtDR8jsmT4aOP4KKLwvveew9eeMH2b98ODQ328Dz/mHPPhW98I+c/rSjZ\nsQMWLoT582H1ali/3h6bNsHOnf6jocF/HfwvvvpV+O53W678QhQ6JS1dgDTx+vb1uPlmeOAB+N73\nrPG/7jp45RUYPNg/8MgjYf/94b77oGNHf/vjj1tjsmULrFoFt99uQuP2XXstXHAB7LOPfa683L6j\nXaNt9frrUF8PDz4YXcAVK2CPPZL/iH/+08o+fboJWxTbtsHatanPBdZggpWzvh7+9Cd7TJ0K3bql\n/nxz43lWj3fcAa++Cr1723+1xx7Qtas9dtvNfk9pqdWRex38L2bNgn//G15+uWV/jxCtnZKSEmg7\n7XxWeNXVnjdihOc99pj3OYce6nkzZvjv16zxPGuC7OF5nrdyped9/LHn7bWX5735pud16uR555zj\n79+yxfO6d489TxR/+YvnXXRR9L6XXvLPl4gPPvC8Hj3suMWLo4/Ztcvz+vTxvM6dY7d//eue98IL\nsdvWr7dz/fa3ntfQ4HlnneV5Q4d6Xs+envf++8nL0hJs2GD1vu++nnf//Z5XV5f9ud54w/OGDMld\n2URx0NDgeRMnet4vf2n3Wkuya5fnTZ7seT/9qecdeaTnrViRn+8BvJStawLaTMxhwwZ46y34ylf8\nbRUV5gJyzJoFlZX++02b4Pe/h29+044dMgQGDICnnvKPGT/erI1DD03+/RUV5gqJ4sYbU5f/xhvN\nOhkyBBYtij7m0Udh+XI48EB/24QJ8Le/wbvvxh7705/a86JFcOutVj8vvww9e8LWranL05xs3Agn\nnmiWwcyZZsHtvnv25+vUySxAUTzs3Anvv594X6rPPvKI3Vc33gi33QZr1uS+jOlQX29ejUMOgW99\ny7wUDQ3WdrU22ow4dO9uLpeePf1t8Q32kiXmi3ZUVcEf/2hulsMPt239+9tz5872/NZbcMopqb8/\nkTgsXw7z5iX/bF0dTJwIV18NgwaFxeG66+DFF+GWW8zVVV9v2z0PbroJDjgg9viFC+GZZ0zYpkwx\ncXjgAXOFdezY+sTh+9+3+r/vPujQoenna43isH07fO1rFj8pVBYssPtr7drm/d5Fi6wDt99+sXGn\nbdvgJz+Bfv0Sf3buXPjyl+Huu83lOm2auaE//DB8bEODxTb79MnsHtq+HX79a4tpJuP55+Ggg+DJ\nJ82tPX8+3Hyz/TZXnoYG//5vadqMOPToYQ17ScB71r59rOWwZIn1DoL++oYGe3a98QEDzEpwvY1Z\ns1JbDZBYHF58EU47LbZc8bz0EgwdaoIULw47dljD/sc/wubNdq5t22zf3LkWHxk50vY57r4bLrsM\nvvhFmD0bTj0V9tzT9nXo4H++NTBpksVaxo5NXkeZkK44PPUUnHNO8mOWL4dPP216ma691gR78eKm\nn6s1EN/h+fe/4YQTLF62YEHzleONN+Dooy2JpHNnWLfOtq9ebffUokX2OqpBHT/eLNbLLrNrcNgw\nuwb33NMST4KsXWv30QsvWELLsmXplW/hQvjSl+DOOxPHwLZsMWv5P/7DROHvfzcPiLsf9trLxGHG\nDBOPMWPS++5802bEoWfPcA8hynIYMMACzAMG2LayMnNnOHHYd1+7yOvr7SKYPdtMvFQkEofp0+Go\no/xIRxQvvwwjRtjrgQNjxeHNN8399fe/w1lnWcPnei3jx1vjVlnpi4PnwdNPm2C4+jj3XP98ySyH\ns86CUaNS/9amsHkz/OpX/vvRo8362W233H1Hx46pxaG+3tyJ48cnPsbzLPvs/vubVp6XXzbLcP/9\n7ZpqCzzyiFl0UYwbZ9aqE7qPPoIzzoB777UGdskS+5+nTMlvGV991f6fxx6DH//Y7p0lS8wSP/54\nK8v48dCli2W6BXn4YbjmGuuYXXllbMfENcaOTz6B446zduCVV6yNWL7c9r33nlkSmzaFy/f22/Y5\nl0X58cfhY+rqTAi2bLG24pRTwp2kvfaCZ5+F4cOtA1xXl01t5Z42Iw7OcgjSvr012B9/DKef7ovD\nCSdYJkxZGZx/vjVOxx9vn7n2WlPm+nq7ACor08vsSSQOM2ea5dGuXeKGYfp0Mx3Bvuuzz/x9U6b4\nwnHiibGN++uvw8knW8PqxGHePLOGDjnEvvO118zacCQTh+eey79vc9w4M5U9D+bMMdN55Mjcfkc6\nlsOjj9pN3ru3vf/Tn6wXGmTCBHM5ut5oNtTXWwbaAw9AdXVq/3drYNUquPhiuOee8L5Fi+CHP7T7\nYskS+33nnAPXXw9nn2331+LF1uCedFJ0o5kL3HXz5JO+q7h/f9t+6qkm/P/1X9bQdusW+x8+8wz8\n/OcmLl/8Yvjce+7pi8P69SYyI0dafLKsDPr2Ncth8WK7/1asCN83//qXdbb+/Gf7//v3h6VLY49Z\nu9aEYehQux6D8dAgBx9s9TxxotV1vuo0U/ItDqcA84EPgKg+6+7AS8AM4D3gW4lO1LNnWBxcQHrV\nKgtWBdNJu3eHI46Av/4VfvQjXwBKS+1zACtX+o1HKqLEwfPM8nDi4BqGJ580cxOsfB995Kfbtm8f\n6/Z55x047zwrx/HH+26hnTvNlD/qKF8cHnzQGt6hQ/3ex9ChsWM6ErmVnLWSjpXUFJ580p63brXe\n6SWXWCwklzgBTGSpeZ656X7/ews8bttm8Z4//Sn2mF/+0oS5KeIwbpw1mCedZP9Dc4nD5MnRjXs6\n/OQn1ugddFDsds+DSy816/LMM63T9dvf2rX5wx/aMQMHWq986lQ49tjY5I5kZFIv27ZZ/Oa//xtq\navztAwZY527ffc3H76iu9v/D99+Hyy+3nvh++0Wf/4ADzGW7cydceKE14MGkkr59fREaNcpcQs89\nZ4kVV1xhscGzzzaL03Xs+vePtRx27LA6HD7c6jCZS3XffU2IvvQlE5CNG9Ovq3yST3EoBe7EBOIA\nYCQwOO6Yq4HpwGFADfB7EgzMu+46U+ggrsHets0eW7f67otu3ZI3/OXlJiY9eqT3Y6LEYfVqO0/3\n7tYwOMvh3HMtEwHsIhs0yEQBwnGSd96BY44xK6Z7d7/hmzMHvvAF+x1OHB57DP7v/+wiSkQiy8EF\ny1wMJh9s2mQB/qoq6zU980x+Bg06gd+2zXpks2fH7n/nHbvBRoywuvvLX2x79+7+MW+/bZ+/+OKw\nSyIVs2b5LqTf/Q5uuMEvV3OJw6hRFu8K8r3vQW2tP3gwKjj+5pv2GDPGMtyCPP201duPf2yN3Ztv\nWqzo/vv9xm3AAIs53HOP1e+0aSbAyepwwwbYe2/4xz/S+2033WTCddllsdsHDbLG8557Yhvb6mq7\n3hoazAJwQd5EHHyw3V+uDm67LXZ/376W5PHlL1vH8oADTBCOOMKskeuvhx/8wCwHR79+8MEHfpl/\n9jO73saMSS/W5o6pqioOy+EoYCGwGKgHxgFnxR2zEmjMG6Iz8CkQ2Xz17x9u7F1D68Rh2zY/G6Z7\n99TisHJl+imV8WmzYJ/v08dex7uVpk+357lzY1NTO3Twz7NxowVDBw3yB3h17Gi/Y/ZsOOww2+bE\nwTWC6YjD88/HNlRTp9oFnCgdNxe89poFD/v3N6tn/frkZW0KLjZz0UXWGAR54gkb0NiunYn/7bdb\nL/CTT/xjHn4YvvOdsEsiHe67zxrISZNMEE46ybZnKw7PPJNZauWbb5rYBxuRhQstJjBpkjVaFRXQ\nq1fs5xoarBf7n/9pHY+gODQ0wC9+YfvbtbP/8IEH4Kqr/GsczMd+zz3mihk82Nyc551nIpmI//kf\nu9/coNNkzJljFvKdd4b3XXml/W6XaehwlsNdd9m+q65K/h1VVdY2/OY3dh3EW7b77msW0tix9v68\n8+x1v34myP/6l98hcLhO6UMPWVD72Wft3JkmYbQmyyGf02f0AYJeuGXA0XHH3A+8BqwAqoBzyQDX\nm9++3R7t2vni8KUvJU8Jy0Yc4hvWFSvsJgPfrbRzp712PvFly2ID6UHLYfFiuwiDF1C7dla2OXNs\ntDbYhffRR3bu0aP9tNwoOnSw7z7jDOvhHnywbZ82zdxWuRSHW24xP6u7kSdNMhP9pZeswaup8UUv\n13TqZNkdYI3D5s1WT55n40JcILqqyoT6llus4QOrx2eftXhPXV1m4uB59lkw19X55/v/XzbisGWL\nuVD+8hezYlatsg5DfPpykDFjzDINWky332698+eeM3+65/mN3osv2m995BET1Keftv9lwwY7rqTE\n6qx7dz+te8AAa2jdeBpHdbU10mDiMHmyCYuzjONZu9ay655/Hr7+9cS/adcu60iNHm297nhhA2s4\no/z23bpZr/2OO0w402mQTzjBOl977RXeN3y41a1zP++xh7nVnGstETt2WEfw4ostKaa6OnU54qms\nbD2WQz7FIZ2ReT/H4g01wF7AK8ChQEg7R48e/fnrmpoaampqQm6loDhEzZ8UxImDy2pKRSJxcDEO\n51ZaudIu7E8+sYZi+fLE4rBokVkN8XToYBenc8nstpsFB084ITYTKIqOHf14h6sLz7Osi/POswB6\nrrj+ent24vD229YbmzrVxME1xvmgUydrpHv1skZv6VJrBD/80BpAl548d649Dx7sWw5vv22iPmiQ\nNWyZuJVcozFkiDXEt97q78tGHJzQuLjRrbdag/rQQ9HHf/qpuY7+/nffdblpk9XBI49YYsaoUdY5\nGDnSft/ll/vX4U9/6l8XZWV233TsaFbHD3/oN6zDh1vDn6yB69vX6uJHPzJ3ZxR33GH++cMPt9/l\nxCieCRNMPHr18t2A6VJdbUJ97rmxU+kk46GHEotISYkvDJlQXm6W1ZYtFtPJhqa6lWpra6mtrc3+\nBAHyKQ7LgWDyaT/MeghyLOCMzQ+BRcB+wDvxJwuKgyPoVtqxw/7URD2YeFzMISqbIYoocVi5Mmw5\nLF1qN+Gnn5rlsmKFxRTiywwmDgMHhr+rY0drhFzaqTNZo4Qk6rPOpeUaqhUr7By9ejXNcoi6sZ3L\nYccOE54jj7Se3Pr15rPNF506mRh861tmFbjRs6++atktQR/54MG+YHuepSu6mzcYzEyHiRPN1755\ns/W8993X35etOJSXWwabS1MOXi/xjB9vvfsvfMF3P0ycaJ8ZOtQa/iuuMLfJ6tX2W/fYw0R75MhY\nF0rnzvYbli2zzsPXvubvKytLbr2A1fFzz/mDzNavt7Rxx44dlgTwz39aucrKrN6iev9/+IMlDZx8\nsv23mVBdbXXhOivpkKsxN/Hcfnv6bVAU6bqVEoms6zg7brrppqzLks+YwzvAPsBAoAI4D3gu7pj5\ngBvT3AsThrjhKYkJupXALvx03RiZupVc2iyYn3Xr1mjLwYlDWZmfLhv02ca7laIa/I4dzVLYe297\n78QhPlsrig4dfHFwwef337fMjWRTgKTDBRdYui34QW/32+bNs4a4stLvmaZqXJpCx47mSrjkEnu/\ndKk1zJMmWSPpmDHDMqgqK+0/2rjRBGTYMNvfpYs1zOmOT3jxRROHYcOsMQuSqTjU15sFcMEFVoZZ\ns8x9mOw/euwxa+SDPUw32K9TJ7ve9tzTXpeXW6rl+edbFlJFRWyD4sThwQf9/ZniBOnQQ8NTvDz/\nvF0DTkC7d48ecDh9ulm7Y8b42T+ZcOCBNsAsnc5TvuncuenikI7lcPPN5oKLx43PgPSm9UlGPsWh\nActGehmYC/wfMA+4svEB8FvgS8BM4B/Az4C0B+cHLQfIbGqGbGMOnmcX/OOPhy2HXbsssNijh50/\nkTi48johicfdwG6qECcO6bjA3Fwt4DdUCxbYDdpUcRg3zh8r4EbJutjKnDl+aqS7QLNpbNLFTeEw\neLC5Yqqr7f+YOjW25+1m2AXrTS9ZYo3RccfZNjflSDo35Pr19tmaGkuBvPzy2P2ZisPkyebz3m8/\nO/fTT9v7+MQHR12did2pp/o9zC1bbBCey5wJjtnp1cvce2efHX0+Jw7jxzd9LMoxx4QHxf35z/Dt\nb/vvE4nDAw9YHCPblOdTT41177VlnDgkStMG60jcdpu5TD3P2oVZsywhpF8/c5ktWGDWXFPI93oO\nLzY+gtwbeL0GOCPbk1dUWEVmKw6ffhrOfEj2XTt2+D7s3Xc3d4RLj3RupXXrzLQuL7fjV66Mnc4j\naDnU1cXOFeVwWSTOCnJmdt++qcsZrAPXUC1caMHtpooD+EK3YIGNwVi0yJ/IzGVl7bVX+uNHssW5\nkUpKzI8+frylsK5Zkzi/vXdvC5bvu2/siG1nPaS6FiZPNrdZcCr4IJmKw/PP2wDGLl0sR/6118wC\ncIH2eGprLanA9UzLy80tdeSR0Z2cXr3sd7qpVeKpqrLMm23b/My4bDnmGIt5OFatMnfSuHH+tm7d\nwvMyNTSY5ZNqXqJioazM7tOtW+2+/8EPrN6C4zruuss6pR9/bANlP/7YsgMff9xcg088YXG1q6+O\nti7Spc2MkI4i3q2UiThUVFjPPtGNHo9r7N96y943NFgj7hoU51Zav956seXlJhQVFbHlCqayrlkT\nfVMHR1C7c0O0kMQT/D3Ogli2zA8ebt9uvvdVq6I/P3ducn+sE6gFCyxtta7O0vree88Xh9//Pjxa\nNB8EM00GDrRG5otfTOxa7N3bGtOj43Lm0hXNKVOSx1EyFYd//MN87F26WCdizhxLOkhkObz2WqzL\nrKrKfP4ulTaenj0TWw1g1+4jj1jwuqk++CFDLFPIZQg+/7w/ut/RvbsFnZcs8be9/rr1fKOyhoqV\nykqLXXztayYEwZH99fUW5L/tNrvH7rjD7skJE8x6uPVWs56feio8LixT2rQ4NNWtBOmLg1tsxgUv\n6+tjxSFoOThxWLMmNkAHvijt2mUNa9QgvKjUwCuvTC8Tw/2evn39hmr5cnvv4ib77Wc3cxRO/OJx\nPT53sy9Y4Afzd+ywnorrsbdrl3gxo1zxhz/YeAPHAQdYw59sEsXeva2c8WMvnAswGR9+aA1xsmBx\nJuKwZYtZP0ccYdfI669b2nGXLomF6rXXYqesr6oyS+iEE6KP//WvLZMoEZ072/995pnplTkZLvvL\nWQAvvBA7rQvY79q40QLojnHjLCYifKqqLKZw8MEmnMFY48SJZgkOH27tx6RJNmjw6adtJPegQXbv\nnX12ep3JZLRpcQimskJ24pCpteEayXhxcJZDUBzq6sLi4NLk3IpvwVG7Qbp0iX1/zz3pldVZGfHi\n0KePX18bNiTOiAgGtIK4SdjcHE/OVeVYs6Z5A4LXXhvbUB56qLkYgwMO4+nd2/4jN/bDUV6eeuT4\nmDHWALqAfBSZiMO771pZ27e3/3rDBouDRA22BLP+1qyJnf6kqsqOTZRxd9BBia8v8OMmQWukKQwb\nZtbD9u0mZPFT4btU4meftWt/1y4/hVX4nH66Nf433WQpukFPwv33mwiUlZnb8NxzrbPTrp1lqZWU\n2Aj3TDK3EtFm1pCOwt1I2biVMrUc3PclEgcXkE4lDmANwief2M2dKAiXbiwkHpezX1Hhr4G9YoWJ\nw+bNfq800fkTicPHH5vrxgVuP/44NkDeo0fiicWaA+czj58vKIiLg8QLiMssS8asWTZuI7heeTyZ\niMO0aRazAb8jcOyxsVlxQd54IzyosKrK3FzZBv5HjMguOygRfftaBtmbb5qVG28Vjx9v19ell9qM\nqT/5id0fUencxYwbmQ1WP04cVqwwy8yNKTn7bBsNPniwpSw7CyN+9Ha2tGlxcDdSc7iVIFYcXEaB\nc/9EuZWSicOyZcnndco2oLv//vadZWVWnjVrzBXUsWOsbz1TcVi61J8uYft2C+YHy9jSaYT9+pnv\nPZU47LlnWMRSuZXc1O7xFkc8mYqDa5jdNXLssdYQRFkO774bni+osjK/Y0kypUcPc1O9/LJlEMXT\np489Xn/drtN99kkcLxGGS5YAG8V++um+a/eOO/zjglZ0rihqt5KbqiKT71u71rII1q6NbWCjAtJ1\ndWH3EKQWh1mz0p/tMp6jjjKBKi01yyGYSht0WWQrDps22TF77GHf8eGHNmq7pcWhpMTGDCSzXo49\nNnohlVTisGSJ1Veqqd2ztRx69zYXwhe+kNhymDEjnFF04YWty1/fs6dd82+95U+RH0Xv3iZ0t94q\ncUhFcK2K8ePzM5FlItq05eAau2zFoUOHzLI0nDhUV4fTYKNSWdesiZ5+oH17fxbWKFL1UNPBNVTB\noHdFhZ9nnuh3r1wZvX3pUuudbNpkLiU3PmPPPc3v2VJr8mZCdXXsKGBHUBzq6836ePttXwzmz09v\nQF+64vDpp/Zwg8PKy/0BS1ExB8+z8RXx4nDxxam/qznp2dOukxUrks+KCjbw7/XXcxfvKFSc5VBX\nZ9fA8OHN991tWhxcL2v7dnOjZCoOmbiUILk4lJZaw7Jpk/2hznJINHfSmjXRVkWucG6lDRv8dNn2\n7f0xFMFlR4MkWjJz8WLLRtqxw14HB+/lcyR0c1BW5gekH3zQsojq6nxxeP/92GkyEpGuOMybZ1ZY\nVMptlOWwbJldT/keO9JUevSwRIUjjkgdf/rmN0308nkPFAIuWWHsWBOGTNusptCmxSHoVurSpXnE\nYelSa/CjLIdNm+yczl2VLOawZk3y4GZTcW6lNWt8C8UFLhNN7rVrlz3ib+zgFOK77WaNZToD8toK\nQcvBpWIGG/n33zcfeSrSFYcFCxIP1IuyHKJcSq0R1wk59tjUx1ZW2pTpIjllZXZPjh0bnp4k37Tp\nmENwnEM24pDJ8e77tmyxBj+ROLjRzGVlLS8OO3daOd1N68Shf/9ocVi71j4X73KaOtUyfDp39qcP\nj5pSua0SFAe38Hy8OOTSckgmDkHL4ec/t+yftiIOZWXWEUlHHERmnHxy8w8UbNPi0LGjNX7btlkj\n3ByWAyR2KznLwZ0/keuofXsTjnyKg3Mrffpp2HIYMCBaHFav9scCBJk2zc+Kqay0KTOaOsCmNREv\nDrvvHjvu4YMP/EkQk5GuOLiJEKNw1vBHH9kymZMm+euUtwW+8Q3FEfJBSwTu27Q4HHigNVZTpzaf\nWwn83OPg1ADt2tnAMmc5lJebVRM8xtFSbiU3QK5PH18cgv7tujoTh/gGbvVqf36oykpruApNHBoa\nzCpcu9YsKycODQ0WpI+aIDGeZOLw3nsW5IbklkNJiZXHLVYE6bu1WgP33tv6YyNtjUWLbOBbc9Om\nxaG01J95sFu3zKbKzUYc3Ays1dXWuAbFyFkOQXGA6O9oDnEIWg7OreTcRT16WE/ZTdPx73/b9tWr\n7TfGWw5BgenZ094XklvJDYJbvNisKjeAECyrrFev9AaaBdcRj+f4483d0tBg35PMEmnf3oLWJSWW\n/fbhh5p7qJiJXy2yuWjTAWkwd8eiRXZTZpqWmmnMYcgQm/nQzX8TFKN27SwDKOhWguiFSzp0sJu+\nuS0HR1WVWQAuc8n1dhNZDkGBOeggG09QaJZDfb0/6nvLFl8cnGCkQzLLYf16uz4XLTIBTnbtVVRY\n+uwRR9hzp07Zj5gXIlvatOXgGDjQ8u0zGYiVjeXg/O6uFxm8waPcShD9HW7cQXMEpNeuDQ/ecmvx\nujUZXFlXrUptObi5fZKN7m5rOHFws9cGU1vdOt/pEC8OvXrFTr/dq1dyl5LDWQ5HHGFWXTrxDiFy\nTUGIQzbGT/TOAAAVUElEQVRkIw6HH24pna4xDVoOUQFpiP4O579uDrfShg3hoLgTB7dAvRODJUus\nl+x5sYuNBC2Hgw/2R4AXClHi4Bp5VyfpEBSHzz4zN11w6vKePcMTFkZRUWGWxuGH2zTecimJlqCo\nxSFTtxKYWyVqRtf4VNZkbqXmEAfnVtqwIfw9lZUWKJ81y967Bs25Vdwkgo5gxtNhh8UGSwsBZylE\nWQ5uuvN0CIqDm4N/+3Z/wGHnzvYdqZZ7bd/eLLN+/awcEgfREhS1OGQ72tCtVRBlOaTjVmoucdi8\n2Z6D5bzpJkuLq6y0XimEe8luKhC3z80XBbav0FIVk7mV3DxS6RCsNycOGzbYecFiGcuWxS4bG0VF\nhblJ3RgZiYNoCdp8QDpbmpJtk8xycL74lrYcysos6B0fyHRz+FRWmosDrEHbtcsarn79/B7wE0+Y\nz71z5/wv3tOSJBOHFSvSF4eg5TBtmqVaf/aZnaNHDxNr9x3JaN9e4iBangK+5ZNz2WXZfzZdt1JJ\nSXR6res55ttyWLs28XdUVlqjBSYMn3xi1kGHDr5baeZMW5c51WykbR0nDitXWkDeueTAXwsjHZw4\nNDTYVAff+Y4vDvvsY267bdtSi0O85aCAtGgJitat1BTSDUgnmvW1osIygzIZl5EpiSwHR2Wl7wt3\ns7e69FTXyH32mQVUC31ytPJyW9B9wwaLrbiAdH29CWy6abuu3hYtMkuhf/9Ycdi0yQQolSXiLIfu\n3W2Vt0JKGxZtB4lDFqRrOUS5lBz5vuFLS1OLg2PnTkvDdVaGG8y1YUNxiENZmTXgu+/ur3/d0GAC\n3qOHP7I8FU4cnLXhplt24rB+vVkDqToFw4fbgLmOHc1ya4kBUEIUrVupKURZDlET7zXn9LrxOHFI\n1Et14tCpkwlBUBxcYPWzz+w3FfoArPJyiwW4OJQTh5UrM5sKwomDc0+56ZY3b7aFfUpK0nNR5WqZ\nRyGagiyHLHDB2ajpM4JupZYUh3TcSmA9WTcewh0btByg8C2HROKwenVmiQtR4uAshz59LH24kKY6\nF4WNxCEL0rEcUrmV8o2zHJIFpMGC0PFupaDlAMVrOaxendlI8GTisMceEgfRtpA4ZEFUzME1DK3F\ncigtNR93oobdzRYbJQ7OcnDiUOiWQ1mZjUFw4uCylYJB+nRIJA7ufadOEgfRdpA4ZEEiyyG4raXF\nwbm+UlkOXbokjjk4t1IxWA7gC4HLVmqq5dCrl40TKS83MZblINoSEocsSJStFNzXGtxKkHgt38pK\nE6/27aMth6BbqdAtB/efufmjnFspW8vBxSqqqy2m45ICunZNPXWGEK2FfIvDKcB84ANgVIJjaoDp\nwHtAbZ7LkxMSjXMAf8bWiorWYTkkEqjKShMD16AF52Bq187cLCUlJgzFYjm4+aOCMYdsxCE4i+0B\nB/ji8MQTcMIJuSu3EPkkn6mspcCdwFeB5cC/gOeAeYFjugJ/Ak4GlgG757E8OSMqWynecjj1VH96\n65bAiVUycejc2W/QNm6MzVZat85fXa9YLId4cairy86tFJyocPBgq1vwF4sSoi2QTBx+EvfeA+qA\nfwKL0jj3UcBCYHHj+3HAWcSKwwXAeEwYANakcd4WJx3LoVu3lp12wpUnkfVy4IHwu9/B3/4WHXNw\nmU4dOxa+5eDEPricqlsoafcMuiulpWaBlZb69T5ihImFEG2NZG6lKqAy8KgCjgReAkamce4+QGA2\ne5Y1bguyD9ANeB14B7g4rVK3MOnEHFqaVG6lDh1sMfig5RCMOWzcaNbFzTfbNN2FTCLLYe3a8Cp6\nySgtNVdU8DOnnw6XXpq7sgrRXCSzHEYn2N4NeBV4PMW5vRT7AcqBI4BhQCfgbWAKFqOILcxovzg1\nNTXU1NSkcfr8UF7uT7PgcOKQzlrDzUEqt1LwuPiYg+sBd+oEX/96fsvZGnB15aYlLyuzUc1bt2Y2\nOaITB7mPREtRW1tLbW1tTs6VTcxhbZrHLQf6Bd73w3cfOZZirqStjY83gENJIQ4tTfv2/jgBh2tg\nWovlkMqtFDxu1y5rDN1vckuexv/GQsVNQOjEvqzM4g3V1ZnNa1Raaq6ogw7KfRmFSIf4jvNNN92U\n9bmyyVYaCqxL47h3MLfRQKACOA8LSAd5FhiCBa87AUcDc7MoU7NSVQXTp8dua22WQyq3ksONadi6\n1RcD51YqFnFw4zkcZWVmAWQaM3JCm4krSojWSjLLYXbEtmpgJXBJGuduAK4GXsYa/wexYPSVjfvv\nxdJcXwJmAbuA+2kD4gDhBVhaq+WQrltpyxbfyig2yyF+TeemiANIHERhkEwczoh77wGfApsyOP+L\njY8g98a9H9P4aNO0VsshHbeSEwcnJMGYQzFw3HHgBSJkLnZw8MGZnUfiIAqJZOKwuLkKUQi0tmyl\nTC2HbdvClkMm01UXEi7mkK3lUOgr54niQNNn5Ii26lZyo6ErKnyBK7aYQzxlZRaDkVtJFDMShxzR\n2iwHR6ryOCEIikixxRzicS45l9qaLhIHUUhIHHKE81mnu6Rkvtmxw55TpWI6cQjGJlzModjFoWvX\nzD4ncRCFhMQhR+zc2dIliGX79vSOi1+kCIovIB2Pa+QznVNK4iAKCYlDjmht4uAsh1S45U3lVvJx\nlkOmc0pJHEQhIXHIEW1ZHKLcShKH7CyHkpLM3VFCtEYkDjmitYlDulNNJ7IcduyQOGQjDtXVrSfu\nJERTkDjkiNYmDmeeaWtIp8LFHOItByjemEO24tC7N3z3u7kvjxAtgcQhR7Q2cXCruKUikeUAEodM\nYw5VVXDLLbkvjxAtgcQhR7Q2cUiXqHEO6c7oWqhkm60kRCEhccgRbVkc4t1KznIILmZUTLj/slh/\nvxAgccgZbVUc2rWzqSKiLIdibRzd+g5CFDMShxzRVsUhyoVU7JaDxEEIiUPOaGho6RJkR5SVUOyW\nw4EHFm8arxAOiUOO2LWrpUuQHc5KaN/e3+bmYwpuKyYOO8ziMEIUMxKHHNHW3UpBIaivt+dM1k8W\nQhQWEoccUYjiIIQoXiQOOaKQxCHdeZmEEIWLxCFHtNWAdFTMQZaDECLZGtIiA+66Cz74oKVLkTmy\nHIQQUUgccsTee9ujrRGVtirLQQght1KRI8tBCBGFxKHIUcxBCBGFxKHIkeUghIhC4lDkaJyDECIK\niUORI7eSECIKiUORI7eSECKKfIvDKcB84ANgVJLjjgQagK/nuTwiDrmVhBBR5FMcSoE7MYE4ABgJ\nDE5w3C3AS4CmemtmZDkIIaLIpzgcBSwEFgP1wDjgrIjjfgg8BdTlsSwiAVExh8GDYc89W6Y8QojW\nQT5HSPcBlgbeLwOOjjjmLOArmGvJy2N5RARRlsOrr7bd9SmEELkhn+KQTkP/R+D6xmNLSOJWGj16\n9Oeva2pqqKmpaVrpRAxlgSshuGSoEKLtUFtbS21tbU7OlU8f/zHAaCzmAHADsAuLLzg+CpRhd2AL\ncDnwXNy5PM+TUZEPXnsNhg0DVa8QhUeJrdiVVTufT8vhHWAfYCCwAjgPC0oHCXq2/wxMICwMIo+0\n1anGhRD5JZ/i0ABcDbyMZSQ9CMwDrmzcf28ev1ukicRBCBFFW0kdlVspTzz7LJx9ttxKQhQiTXEr\naYR0kSPLQQgRhcShyPnqV+E3v2npUgghWhtyKwkhRIEit5IQQoicInEQQggRQuIghBAihMRBCCFE\nCImDEEKIEBIHIYQQISQOQgghQkgchBBChJA4CCGECCFxEEIIEULiIIQQIoTEQQghRAiJgxBCiBAS\nByGEECEkDkIIIUJIHIQQQoSQOAghhAghcRBCCBFC4iCEECKExEEIIUQIiYMQQogQEgchhBAhJA5C\nCCFCSByEEEKEkDgIIYQI0RzicAowH/gAGBWx/0JgJjALeAs4pBnKJIQQIgkleT5/KbAA+CqwHPgX\nMBKYFzjmy8Bc4DNMSEYDx8Sdx/M8L89FFUKIwqKkpASybOfzbTkcBSwEFgP1wDjgrLhj3saEAWAq\n0DfPZRJCCJGCfItDH2Bp4P2yxm2JuAyYmNcSCSGESElZns+fiS9oKPAd4Lg8lUUIIUSa5FsclgP9\nAu/7YdZDPIcA92Mxh3VRJxo9evTnr2tqaqipqclVGYUQoiCora2ltrY2J+fKd0C6DAtIDwNWANMI\nB6T7A68BFwFTEpxHAWkhhMiQpgSk8205NABXAy9jmUsPYsJwZeP+e4EbgWrg7sZt9VggWwghRAuR\nb8shV8hyEEKIDGnNqaxCCCHaIBIHIYQQISQOQgghQkgchBBChMh3tpIQQmRFt27dWLcuctiTiKO6\nupq1a9fm9JzKVhJCtEpKSkrQfZ8eiepK2UpCCCFyisRBCCFECImDEEKIEBIHIYQQISQOQgiRJTfc\ncAO333573r9nwoQJnH/++Xn/niASByGEyIK6ujr++te/ctVVVwEwZcoUTjrpJLp3707Pnj0599xz\n+eSTT9I+18iRI+nTpw9du3ZlyJAhTJs27fP9Z5xxBnPmzGH27Nl5+S1RSByEECILHn74YU477TTa\nt28PwPr167nqqqtYsmQJS5Ysoaqqim9/+9tpnWvTpk0cffTRvPvuu6xbt45LL72U0047jc2bN39+\nzMiRI7nvvvvy8lui0DgHIUSrpLWPcxg2bBiXXXYZF1xwQeT+d999l5qaGjZs2JDV+bt06UJtbS2H\nH344AJMnT+aiiy7io48+Ch2rcQ5CCNFKmD17Nvvtt1/C/W+88QYHHXRQVueeMWMGO3bsYO+99/58\n2/7778/ixYvZtGlTVufMFE2fIYRos5TkyPeRjYGyfv16qqqqIvfNmjWLX//61zz33HMZn3fDhg1c\nfPHFjB49Oub87vX69euprKzMvMAZInEQQrRZWtLrVF1dzcaNG0PbFy5cyIgRIxg7dizHHXdcRufc\nunUrZ5xxBsceeyyjRo2K2ee+q2vXrtkXOgPkVhJCiCw45JBDWLBgQcy2JUuWcNJJJ3HjjTdy4YUX\nZnS+7du3c/bZZ9O/f3/uvffe0P558+YxcODAZrEaQOIghBBZMWLECCZNmvT5++XLl/OVr3yFq6++\nmiuuuCJ0/MMPP8ygQYMiz1VfX88555xDp06dePjhhyOPmTRpEiNGjMhJ2dNB4iCEEFlwySWXMHHi\nRLZt2wbAAw88wKJFiz6PFVRVVdG5c+fPj1+6dClDhgyJPNfkyZN54YUXeOWVV+jatevnn3/rrbc+\nP2bcuHFceeWV+f1RAZTKKoRolbT2VFaAX/ziF/Ts2ZNrrrkm5bEnn3wyY8eOTZrhlIgJEybw6KOP\nMm7cuMj9+UhllTgIIVolbUEcWgsa5yCEEKJZkDgIIYQIIXEQQggRQuIghBAihMRBCCFECE2fIYRo\nlVRXV7tsG5GC6urqnJ8z3zV/CvBHoBR4ALgl4pixwKnAFuBbwPSIY5TKKoQQGdJaU1lLgTsxgTgA\nGAkMjjtmBLA3sA9wBXB3HstTENTW1rZ0EVoNqgsf1YWP6iI35FMcjgIWAouBemAccFbcMWcC/9v4\neirQFeiVxzK1eXTh+6gufFQXPqqL3JBPcegDLA28X9a4LdUxffNYJiGEEGmQT3FIN0gQ7w9TcEEI\nIVqYfAakjwFGYzEHgBuAXcQGpe8BajGXE8B84ERgVdy5FgJ75amcQghRqHyIxXVbFWVYwQYCFcAM\nogPSExtfHwNMaa7CCSGEaDlOBRZgPf8bGrdd2fhw3Nm4fyZwRLOWTgghhBBCCFEYnILFIT4ARqU4\nthB4CIu3zA5s6wa8ArwP/B1L93XcgNXNfGB4M5WxuegHvA7MAd4DftS4vRjrowOW6j0DmAv8d+P2\nYqwLRyk2YHZC4/tirYvFwCysLqY1biv4uijF3E0DgXKiYxaFxvHA4cSKw/8AP2t8PQr4XePrA7A6\nKcfqaCGFNVdWb+CwxteVmHtyMMVbH50an8uw2NwQircuAH4MPAo81/i+WOtiESYGQQq+Lr4MvBR4\nf33jo9AZSKw4zMcfGNi78T1YDyBoTb2EBfULlWeAr6L66AT8CziQ4q2LvsA/gKH4lkOx1sUioHvc\ntpzURWtWjXQG0RUDvfBTe1fh/+l7YHXiKOT6GYhZVFMp3vpoh/X6VuG724q1Lv4AXIelxjuKtS48\nTCjfAS5v3JaTumjNs7JqMFwYj+T1Uoh1VgmMB64BNsbtK6b62IW52boAL2O95iDFUhenA6sxH3tN\ngmOKpS4AjgNWAj2wOMP8uP1Z10VrthyWY0FJRz9iVa9YWIWZhgBfwG4MCNdP38ZthUQ5Jgx/xdxK\nUNz1AfAZ8ALwRYqzLo7F5mRbBDwOfAW7PoqxLsCEAaAOeBqb067g6yKdQXSFyEDCAWnnJ7yecHCp\nAhiE1VUhTX5fAvwFcyEEKcb62B0/46Qj8AYwjOKsiyAn4sccirEuOgFVja93A97CMpCKoi6iBtEV\nMo8DK4AdWLzl21gmwj+ITkv7OVY384GTm7Wk+WcI5kqZgbkQpmOpzcVYHwcD72J1MQvzt0Nx1kWQ\nE/GzlYqxLgZh18QMLN3btZHFWBdCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgjRltjU+DwAGJnj\nc/887v1bOT6/EEKIPOHmZKrBH1GbLqnmH4uf70kIIUQbwTXgU4D12Gjra7C5xW7FFkmZCVzReFwN\n8CbwLP5EZs9gM1++hz/75e+Ahsbz/bVxm7NSShrPPRsb1Xxu4Ny1wJPAPOCRQDl/h822OrPxs0II\nIfKIE4fgXDxgYvCLxtftsXUSBmIN+CbMDeWobnzuiDX47n285eDefwObuqAE6AkswSZDq8EEao/G\nfZOxmTW7EzujZud0f5wQ+aA1z8oqRK6Jn2RsOHAJ1vOfgs1Js3fjvmlYg+64BpvD5m1sZst9UnzX\nEOAxbErk1cAk4MjG99OwObS8xnMOwARjG/Ag8DVga6Y/TohcInEQxc7V2EJChwN7YROWAWwOHFOD\nzYJ6DLamwnRsXedkeITFyM2dvz2wbSc2NflObLrlp7A1C15CiBZE4iCKiY34UxyDLZrzffyg8774\nazUH6Qysw3r2+xO7tGI90UHrN4HzsHusB3ACZjEkmiJ5N2z2zBex9ZEPTflrhMgjrXklOCFyheux\nz8R66DOAPwNjsRjDu1ijvRpz6cSvnvUScBUwF5tC/u3AvvuwgPO/gYsDn3saWwd9ZuO26xrPP5jw\n6lseJlrPYhZJCfAfWf9aIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQohC5v8BJFxH\nUX/IRW0AAAAASUVORK5CYII=\n",
- "text/plain": [
- "\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "class Problem(object):\n",
+ "\n",
+ " """The abstract class for a formal problem. You should subclass\n",
+ " this and implement the methods actions and result, and possibly\n",
+ " __init__, goal_test, and path_cost. Then you will create instances\n",
+ " of your subclass and solve them with the various search functions."""\n",
+ "\n",
+ " def __init__(self, initial, goal=None):\n",
+ " """The constructor specifies the initial state, and possibly a goal\n",
+ " state, if there is a unique goal. Your subclass's constructor can add\n",
+ " other arguments."""\n",
+ " self.initial = initial\n",
+ " self.goal = goal\n",
+ "\n",
+ " def actions(self, state):\n",
+ " """Return the actions that can be executed in the given\n",
+ " state. The result would typically be a list, but if there are\n",
+ " many actions, consider yielding them one at a time in an\n",
+ " iterator, rather than building them all at once."""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def result(self, state, action):\n",
+ " """Return the state that results from executing the given\n",
+ " action in the given state. The action must be one of\n",
+ " self.actions(state)."""\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def goal_test(self, state):\n",
+ " """Return True if the state is a goal. The default method compares the\n",
+ " state to self.goal or checks for state in self.goal if it is a\n",
+ " list, as specified in the constructor. Override this method if\n",
+ " checking against a single self.goal is not enough."""\n",
+ " if isinstance(self.goal, list):\n",
+ " return is_in(state, self.goal)\n",
+ " else:\n",
+ " return state == self.goal\n",
+ "\n",
+ " def path_cost(self, c, state1, action, state2):\n",
+ " """Return the cost of a solution path that arrives at state2 from\n",
+ " state1 via action, assuming cost c to get up to state1. If the problem\n",
+ " is such that the path doesn't matter, this function will only look at\n",
+ " state2. If the path does matter, it will consider c and maybe state1\n",
+ " and action. The default method costs 1 for every step in the path."""\n",
+ " return c + 1\n",
+ "\n",
+ " def value(self, state):\n",
+ " """For optimization problems, each state has a value. Hill-climbing\n",
+ " and related algorithms try to maximize this value."""\n",
+ " raise NotImplementedError\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "class Node:\n",
+ "\n",
+ " """A node in a search tree. Contains a pointer to the parent (the node\n",
+ " that this is a successor of) and to the actual state for this node. Note\n",
+ " that if a state is arrived at by two paths, then there are two nodes with\n",
+ " the same state. Also includes the action that got us to this state, and\n",
+ " the total path_cost (also known as g) to reach the node. Other functions\n",
+ " may add an f and h value; see best_first_graph_search and astar_search for\n",
+ " an explanation of how the f and h values are handled. You will not need to\n",
+ " subclass this class."""\n",
+ "\n",
+ " def __init__(self, state, parent=None, action=None, path_cost=0):\n",
+ " """Create a search tree Node, derived from a parent by an action."""\n",
+ " self.state = state\n",
+ " self.parent = parent\n",
+ " self.action = action\n",
+ " self.path_cost = path_cost\n",
+ " self.depth = 0\n",
+ " if parent:\n",
+ " self.depth = parent.depth + 1\n",
+ "\n",
+ " def __repr__(self):\n",
+ " return "<Node {}>".format(self.state)\n",
+ "\n",
+ " def __lt__(self, node):\n",
+ " return self.state < node.state\n",
+ "\n",
+ " def expand(self, problem):\n",
+ " """List the nodes reachable in one step from this node."""\n",
+ " return [self.child_node(problem, action)\n",
+ " for action in problem.actions(self.state)]\n",
+ "\n",
+ " def child_node(self, problem, action):\n",
+ " """[Figure 3.10]"""\n",
+ " next_state = problem.result(self.state, action)\n",
+ " next_node = Node(next_state, self, action,\n",
+ " problem.path_cost(self.path_cost, self.state,\n",
+ " action, next_state))\n",
+ " return next_node\n",
+ " \n",
+ " def solution(self):\n",
+ " """Return the sequence of actions to go from the root to this node."""\n",
+ " return [node.action for node in self.path()[1:]]\n",
+ "\n",
+ " def path(self):\n",
+ " """Return a list of nodes forming the path from the root to this node."""\n",
+ " node, path_back = self, []\n",
+ " while node:\n",
+ " path_back.append(node)\n",
+ " node = node.parent\n",
+ " return list(reversed(path_back))\n",
+ "\n",
+ " # We want for a queue of nodes in breadth_first_graph_search or\n",
+ " # astar_search to have no duplicated states, so we treat nodes\n",
+ " # with the same state as equal. [Problem: this may not be what you\n",
+ " # want in other contexts.]\n",
+ "\n",
+ " def __eq__(self, other):\n",
+ " return isinstance(other, Node) and self.state == other.state\n",
+ "\n",
+ " def __hash__(self):\n",
+ " return hash(self.state)\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "class GraphProblem(Problem):\n",
+ "\n",
+ " """The problem of searching a graph from one node to another."""\n",
+ "\n",
+ " def __init__(self, initial, goal, graph):\n",
+ " Problem.__init__(self, initial, goal)\n",
+ " self.graph = graph\n",
+ "\n",
+ " def actions(self, A):\n",
+ " """The actions at a graph node are just its neighbors."""\n",
+ " return list(self.graph.get(A).keys())\n",
+ "\n",
+ " def result(self, state, action):\n",
+ " """The result of going to a neighbor is just that neighbor."""\n",
+ " return action\n",
+ "\n",
+ " def path_cost(self, cost_so_far, A, action, B):\n",
+ " return cost_so_far + (self.graph.get(A, B) or infinity)\n",
+ "\n",
+ " def find_min_edge(self):\n",
+ " """Find minimum value of edges."""\n",
+ " m = infinity\n",
+ " for d in self.graph.graph_dict.values():\n",
+ " local_min = min(d.values())\n",
+ " m = min(m, local_min)\n",
+ "\n",
+ " return m\n",
+ "\n",
+ " def h(self, node):\n",
+ " """h function is straight-line distance from a node's state to goal."""\n",
+ " locs = getattr(self.graph, 'locations', None)\n",
+ " if locs:\n",
+ " if type(node) is str:\n",
+ " return int(distance(locs[node], locs[self.goal]))\n",
+ "\n",
+ " return int(distance(locs[node.state], locs[self.goal]))\n",
+ " else:\n",
+ " return infinity\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "class SimpleProblemSolvingAgentProgram:\n",
+ "\n",
+ " """Abstract framework for a problem-solving agent. [Figure 3.1]"""\n",
+ "\n",
+ " def __init__(self, initial_state=None):\n",
+ " """State is an abstract representation of the state\n",
+ " of the world, and seq is the list of actions required\n",
+ " to get to a particular state from the initial state(root)."""\n",
+ " self.state = initial_state\n",
+ " self.seq = []\n",
+ "\n",
+ " def __call__(self, percept):\n",
+ " """[Figure 3.1] Formulate a goal and problem, then\n",
+ " search for a sequence of actions to solve it."""\n",
+ " self.state = self.update_state(self.state, percept)\n",
+ " if not self.seq:\n",
+ " goal = self.formulate_goal(self.state)\n",
+ " problem = self.formulate_problem(self.state, goal)\n",
+ " self.seq = self.search(problem)\n",
+ " if not self.seq:\n",
+ " return None\n",
+ " return self.seq.pop(0)\n",
+ "\n",
+ " def update_state(self, state, percept):\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def formulate_goal(self, state):\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def formulate_problem(self, state, goal):\n",
+ " raise NotImplementedError\n",
+ "\n",
+ " def search(self, problem):\n",
+ " raise NotImplementedError\n",
+ "
\n",
+ "However, RBFS still suffers from excessive node regeneration.\n",
+ "
\n",
+ "Let's have a look at the implementation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def recursive_best_first_search(problem, h=None):\n",
+ " """[Figure 3.26] Recursive best-first search (RBFS) is an\n",
+ " informative search algorithm. Like A*, it uses the heuristic\n",
+ " f(n) = g(n) + h(n) to determine the next node to expand, making\n",
+ " it both optimal and complete (iff the heuristic is consistent).\n",
+ " To reduce memory consumption, RBFS uses a depth first search\n",
+ " and only retains the best f values of its ancestors."""\n",
+ " h = memoize(h or problem.h, 'h')\n",
+ "\n",
+ " def RBFS(problem, node, flimit):\n",
+ " if problem.goal_test(node.state):\n",
+ " return node, 0 # (The second value is immaterial)\n",
+ " successors = node.expand(problem)\n",
+ " if len(successors) == 0:\n",
+ " return None, infinity\n",
+ " for s in successors:\n",
+ " s.f = max(s.path_cost + h(s), node.f)\n",
+ " while True:\n",
+ " # Order by lowest f value\n",
+ " successors.sort(key=lambda x: x.f)\n",
+ " best = successors[0]\n",
+ " if best.f > flimit:\n",
+ " return None, best.f\n",
+ " if len(successors) > 1:\n",
+ " alternative = successors[1].f\n",
+ " else:\n",
+ " alternative = infinity\n",
+ " result, best.f = RBFS(problem, best, min(flimit, alternative))\n",
+ " if result is not None:\n",
+ " return result, best.f\n",
+ "\n",
+ " node = Node(problem.initial)\n",
+ " node.f = h(node)\n",
+ " result, bestf = RBFS(problem, node, infinity)\n",
+ " return result\n",
+ "
\n",
+ "Let's define our goal state."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 39,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "goal = [1, 2, 3, 4, 5, 6, 7, 8, 0]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Heuristics :-\n",
+ "\n",
+ "1) Manhattan Distance:- For the 8 puzzle problem Manhattan distance is defined as the distance of a tile from its goal state( for the tile numbered '1' in the initial configuration Manhattan distance is 4 \"2 for left and 2 for upward displacement\").\n",
+ "\n",
+ "2) No. of Misplaced Tiles:- The heuristic calculates the number of misplaced tiles between the current state and goal state.\n",
+ "\n",
+ "3) Sqrt of Manhattan Distance:- It calculates the square root of Manhattan distance.\n",
+ "\n",
+ "4) Max Heuristic:- It assign the score as the maximum between \"Manhattan Distance\" and \"No. of Misplaced Tiles\"."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 40,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Heuristics for 8 Puzzle Problem\n",
+ "import math\n",
+ "\n",
+ "def linear(node):\n",
+ " return sum([1 if node.state[i] != goal[i] else 0 for i in range(8)])\n",
+ "\n",
+ "def manhattan(node):\n",
+ " state = node.state\n",
+ " index_goal = {0:[2,2], 1:[0,0], 2:[0,1], 3:[0,2], 4:[1,0], 5:[1,1], 6:[1,2], 7:[2,0], 8:[2,1]}\n",
+ " index_state = {}\n",
+ " index = [[0,0], [0,1], [0,2], [1,0], [1,1], [1,2], [2,0], [2,1], [2,2]]\n",
+ " x, y = 0, 0\n",
+ " \n",
+ " for i in range(len(state)):\n",
+ " index_state[state[i]] = index[i]\n",
+ " \n",
+ " mhd = 0\n",
+ " \n",
+ " for i in range(8):\n",
+ " for j in range(2):\n",
+ " mhd = abs(index_goal[i][j] - index_state[i][j]) + mhd\n",
+ " \n",
+ " return mhd\n",
+ "\n",
+ "def sqrt_manhattan(node):\n",
+ " state = node.state\n",
+ " index_goal = {0:[2,2], 1:[0,0], 2:[0,1], 3:[0,2], 4:[1,0], 5:[1,1], 6:[1,2], 7:[2,0], 8:[2,1]}\n",
+ " index_state = {}\n",
+ " index = [[0,0], [0,1], [0,2], [1,0], [1,1], [1,2], [2,0], [2,1], [2,2]]\n",
+ " x, y = 0, 0\n",
+ " \n",
+ " for i in range(len(state)):\n",
+ " index_state[state[i]] = index[i]\n",
+ " \n",
+ " mhd = 0\n",
+ " \n",
+ " for i in range(8):\n",
+ " for j in range(2):\n",
+ " mhd = (index_goal[i][j] - index_state[i][j])**2 + mhd\n",
+ " \n",
+ " return math.sqrt(mhd)\n",
+ "\n",
+ "def max_heuristic(node):\n",
+ " score1 = manhattan(node)\n",
+ " score2 = linear(node)\n",
+ " return max(score1, score2)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can solve the puzzle using the `astar_search` method."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 41,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "True"
+ ]
+ },
+ "execution_count": 41,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Solving the puzzle \n",
+ "puzzle = EightPuzzle((2, 4, 3, 1, 5, 6, 7, 8, 0))\n",
+ "puzzle.check_solvability((2, 4, 3, 1, 5, 6, 7, 8, 0)) # checks whether the initialized configuration is solvable or not"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This case is solvable, let's proceed.\n",
+ "
\n",
+ "The default heuristic function returns the number of misplaced tiles."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 42,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['UP', 'LEFT', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'RIGHT', 'DOWN']"
+ ]
+ },
+ "execution_count": 42,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "astar_search(puzzle).solution()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the following cells, we use different heuristic functions.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['UP', 'LEFT', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'RIGHT', 'DOWN']"
+ ]
+ },
+ "execution_count": 43,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "astar_search(puzzle, linear).solution()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'RIGHT']"
+ ]
+ },
+ "execution_count": 44,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "astar_search(puzzle, manhattan).solution()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'RIGHT']"
+ ]
+ },
+ "execution_count": 45,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "astar_search(puzzle, sqrt_manhattan).solution()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'RIGHT']"
+ ]
+ },
+ "execution_count": 46,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "astar_search(puzzle, max_heuristic).solution()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "And here's how `recursive_best_first_search` can be used to solve this problem too."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 47,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'UP', 'DOWN', 'RIGHT']"
+ ]
+ },
+ "execution_count": 47,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "recursive_best_first_search(puzzle, manhattan).solution()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Even though all the heuristic functions give the same solution, the difference lies in the computation time.\n",
+ "
\n",
+ "This might make all the difference in a scenario where high computational efficiency is required.\n",
+ "
\n",
+ "Let's define a few puzzle states and time `astar_search` for every heuristic function.\n",
+ "We will use the %%timeit magic for this."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 48,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "puzzle_1 = EightPuzzle((2, 4, 3, 1, 5, 6, 7, 8, 0))\n",
+ "puzzle_2 = EightPuzzle((1, 2, 3, 4, 5, 6, 0, 7, 8))\n",
+ "puzzle_3 = EightPuzzle((1, 2, 3, 4, 5, 7, 8, 6, 0))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The default heuristic function is the same as the `linear` heuristic function, but we'll still check both."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 49,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "3.24 ms ± 190 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "astar_search(puzzle_1)\n",
+ "astar_search(puzzle_2)\n",
+ "astar_search(puzzle_3)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 50,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "3.68 ms ± 368 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "astar_search(puzzle_1, linear)\n",
+ "astar_search(puzzle_2, linear)\n",
+ "astar_search(puzzle_3, linear)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 51,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "3.12 ms ± 88.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "astar_search(puzzle_1, manhattan)\n",
+ "astar_search(puzzle_2, manhattan)\n",
+ "astar_search(puzzle_3, manhattan)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 52,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "22.7 ms ± 1.69 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "astar_search(puzzle_1, sqrt_manhattan)\n",
+ "astar_search(puzzle_2, sqrt_manhattan)\n",
+ "astar_search(puzzle_3, sqrt_manhattan)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 53,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "3.91 ms ± 434 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "astar_search(puzzle_1, max_heuristic)\n",
+ "astar_search(puzzle_2, max_heuristic)\n",
+ "astar_search(puzzle_3, max_heuristic)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can infer that the `manhattan` heuristic function works the fastest.\n",
+ "
\n",
+ "`sqrt_manhattan` has an extra `sqrt` operation which makes it quite a lot slower than the others.\n",
+ "
\n",
+ "`max_heuristic` should have been a bit slower as it calls two functions, but in this case, those values were already calculated which saved some time.\n",
+ "Feel free to play around with these functions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For comparison, this is how RBFS performs on this problem."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 54,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "140 ms ± 9.89 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "recursive_best_first_search(puzzle_1, linear)\n",
+ "recursive_best_first_search(puzzle_2, linear)\n",
+ "recursive_best_first_search(puzzle_3, linear)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is quite a lot slower than `astar_search` as we can see."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## HILL CLIMBING\n",
+ "\n",
+ "Hill Climbing is a heuristic search used for optimization problems.\n",
+ "Given a large set of inputs and a good heuristic function, it tries to find a sufficiently good solution to the problem. \n",
+ "This solution may or may not be the global optimum.\n",
+ "The algorithm is a variant of generate and test algorithm. \n",
+ "
\n",
+ "As a whole, the algorithm works as follows:\n",
+ "- Evaluate the initial state.\n",
+ "- If it is equal to the goal state, return.\n",
+ "- Find a neighboring state (one which is heuristically similar to the current state)\n",
+ "- Evaluate this state. If it is closer to the goal state than before, replace the initial state with this state and repeat these steps.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 55,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def hill_climbing(problem):\n",
+ " """From the initial node, keep choosing the neighbor with highest value,\n",
+ " stopping when no neighbor is better. [Figure 4.2]"""\n",
+ " current = Node(problem.initial)\n",
+ " while True:\n",
+ " neighbors = current.expand(problem)\n",
+ " if not neighbors:\n",
+ " break\n",
+ " neighbor = argmax_random_tie(neighbors,\n",
+ " key=lambda node: problem.value(node.state))\n",
+ " if problem.value(neighbor.state) <= problem.value(current.state):\n",
+ " break\n",
+ " current = neighbor\n",
+ " return current.state\n",
+ "
\n",
+ "We need to define a class for this problem.\n",
+ "
\n",
+ "`Problem` will be used as a base class."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 56,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class TSP_problem(Problem):\n",
+ "\n",
+ " \"\"\" subclass of Problem to define various functions \"\"\"\n",
+ "\n",
+ " def two_opt(self, state):\n",
+ " \"\"\" Neighbour generating function for Traveling Salesman Problem \"\"\"\n",
+ " neighbour_state = state[:]\n",
+ " left = random.randint(0, len(neighbour_state) - 1)\n",
+ " right = random.randint(0, len(neighbour_state) - 1)\n",
+ " if left > right:\n",
+ " left, right = right, left\n",
+ " neighbour_state[left: right + 1] = reversed(neighbour_state[left: right + 1])\n",
+ " return neighbour_state\n",
+ "\n",
+ " def actions(self, state):\n",
+ " \"\"\" action that can be excuted in given state \"\"\"\n",
+ " return [self.two_opt]\n",
+ "\n",
+ " def result(self, state, action):\n",
+ " \"\"\" result after applying the given action on the given state \"\"\"\n",
+ " return action(state)\n",
+ "\n",
+ " def path_cost(self, c, state1, action, state2):\n",
+ " \"\"\" total distance for the Traveling Salesman to be covered if in state2 \"\"\"\n",
+ " cost = 0\n",
+ " for i in range(len(state2) - 1):\n",
+ " cost += distances[state2[i]][state2[i + 1]]\n",
+ " cost += distances[state2[0]][state2[-1]]\n",
+ " return cost\n",
+ "\n",
+ " def value(self, state):\n",
+ " \"\"\" value of path cost given negative for the given state \"\"\"\n",
+ " return -1 * self.path_cost(None, None, None, state)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We will use cities from the Romania map as our cities for this problem.\n",
+ "
\n",
+ "A list of all cities and a dictionary storing distances between them will be populated."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 57,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "['Arad', 'Bucharest', 'Craiova', 'Drobeta', 'Eforie', 'Fagaras', 'Giurgiu', 'Hirsova', 'Iasi', 'Lugoj', 'Mehadia', 'Neamt', 'Oradea', 'Pitesti', 'Rimnicu', 'Sibiu', 'Timisoara', 'Urziceni', 'Vaslui', 'Zerind']\n"
+ ]
+ }
+ ],
+ "source": [
+ "distances = {}\n",
+ "all_cities = []\n",
+ "\n",
+ "for city in romania_map.locations.keys():\n",
+ " distances[city] = {}\n",
+ " all_cities.append(city)\n",
+ " \n",
+ "all_cities.sort()\n",
+ "print(all_cities)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we need to populate the individual lists inside the dictionary with the manhattan distance between the cities."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 58,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "for name_1, coordinates_1 in romania_map.locations.items():\n",
+ " for name_2, coordinates_2 in romania_map.locations.items():\n",
+ " distances[name_1][name_2] = np.linalg.norm(\n",
+ " [coordinates_1[0] - coordinates_2[0], coordinates_1[1] - coordinates_2[1]])\n",
+ " distances[name_2][name_1] = np.linalg.norm(\n",
+ " [coordinates_1[0] - coordinates_2[0], coordinates_1[1] - coordinates_2[1]])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The way neighbours are chosen currently isn't suitable for the travelling salespersons problem.\n",
+ "We need a neighboring state that is similar in total path distance to the current state.\n",
+ "
\n",
+ "We need to change the function that finds neighbors."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 59,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def hill_climbing(problem):\n",
+ " \n",
+ " \"\"\"From the initial node, keep choosing the neighbor with highest value,\n",
+ " stopping when no neighbor is better. [Figure 4.2]\"\"\"\n",
+ " \n",
+ " def find_neighbors(state, number_of_neighbors=100):\n",
+ " \"\"\" finds neighbors using two_opt method \"\"\"\n",
+ " \n",
+ " neighbors = []\n",
+ " \n",
+ " for i in range(number_of_neighbors):\n",
+ " new_state = problem.two_opt(state)\n",
+ " neighbors.append(Node(new_state))\n",
+ " state = new_state\n",
+ " \n",
+ " return neighbors\n",
+ "\n",
+ " # as this is a stochastic algorithm, we will set a cap on the number of iterations\n",
+ " iterations = 10000\n",
+ " \n",
+ " current = Node(problem.initial)\n",
+ " while iterations:\n",
+ " neighbors = find_neighbors(current.state)\n",
+ " if not neighbors:\n",
+ " break\n",
+ " neighbor = argmax_random_tie(neighbors,\n",
+ " key=lambda node: problem.value(node.state))\n",
+ " if problem.value(neighbor.state) <= problem.value(current.state):\n",
+ " \"\"\"Note that it is based on negative path cost method\"\"\"\n",
+ " current.state = neighbor.state\n",
+ " iterations -= 1\n",
+ " \n",
+ " return current.state"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "An instance of the TSP_problem class will be created."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 60,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "tsp = TSP_problem(all_cities)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can now generate an approximate solution to the problem by calling `hill_climbing`.\n",
+ "The results will vary a bit each time you run it."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 50,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['Arad',\n",
+ " 'Timisoara',\n",
+ " 'Lugoj',\n",
+ " 'Mehadia',\n",
+ " 'Drobeta',\n",
+ " 'Craiova',\n",
+ " 'Pitesti',\n",
+ " 'Giurgiu',\n",
+ " 'Bucharest',\n",
+ " 'Urziceni',\n",
+ " 'Eforie',\n",
+ " 'Hirsova',\n",
+ " 'Vaslui',\n",
+ " 'Iasi',\n",
+ " 'Neamt',\n",
+ " 'Fagaras',\n",
+ " 'Rimnicu',\n",
+ " 'Sibiu',\n",
+ " 'Oradea',\n",
+ " 'Zerind']"
+ ]
+ },
+ "execution_count": 50,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "hill_climbing(tsp)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The solution looks like this.\n",
+ "It is not difficult to see why this might be a good solution.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## SIMULATED ANNEALING\n",
+ "\n",
+ "The intuition behind Hill Climbing was developed from the metaphor of climbing up the graph of a function to find its peak. \n",
+ "There is a fundamental problem in the implementation of the algorithm however.\n",
+ "To find the highest hill, we take one step at a time, always uphill, hoping to find the highest point, \n",
+ "but if we are unlucky to start from the shoulder of the second-highest hill, there is no way we can find the highest one. \n",
+ "The algorithm will always converge to the local optimum.\n",
+ "Hill Climbing is also bad at dealing with functions that flatline in certain regions.\n",
+ "If all neighboring states have the same value, we cannot find the global optimum using this algorithm.\n",
+ "
\n",
+ "
\n",
+ "Let's now look at an algorithm that can deal with these situations.\n",
+ "
\n",
+ "Simulated Annealing is quite similar to Hill Climbing, \n",
+ "but instead of picking the _best_ move every iteration, it picks a _random_ move. \n",
+ "If this random move brings us closer to the global optimum, it will be accepted, \n",
+ "but if it doesn't, the algorithm may accept or reject the move based on a probability dictated by the _temperature_. \n",
+ "When the `temperature` is high, the algorithm is more likely to accept a random move even if it is bad.\n",
+ "At low temperatures, only good moves are accepted, with the occasional exception.\n",
+ "This allows exploration of the state space and prevents the algorithm from getting stuck at the local optimum.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 62,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def simulated_annealing(problem, schedule=exp_schedule()):\n",
+ " """[Figure 4.5] CAUTION: This differs from the pseudocode as it\n",
+ " returns a state instead of a Node."""\n",
+ " current = Node(problem.initial)\n",
+ " for t in range(sys.maxsize):\n",
+ " T = schedule(t)\n",
+ " if T == 0:\n",
+ " return current.state\n",
+ " neighbors = current.expand(problem)\n",
+ " if not neighbors:\n",
+ " return current.state\n",
+ " next_choice = random.choice(neighbors)\n",
+ " delta_e = problem.value(next_choice.state) - problem.value(current.state)\n",
+ " if delta_e > 0 or probability(math.exp(delta_e / T)):\n",
+ " current = next_choice\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def exp_schedule(k=20, lam=0.005, limit=100):\n",
+ " """One possible schedule function for simulated annealing"""\n",
+ " return lambda t: (k * math.exp(-lam * t) if t < limit else 0)\n",
+ "
\n",
+ "Let's also time it."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 72,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "problem = PeakFindingProblem(initial, grid, directions8)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "533 ms ± 51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "solutions = {problem.value(simulated_annealing(problem)) for i in range(100)}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "9"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "max(solutions)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The peak is at 1.0 which is how gaussian distributions are defined.\n",
+ "
\n",
+ "This could also be solved by Hill Climbing as follows."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "206 µs ± 21.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "solution = problem.value(hill_climbing(problem))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "1.0"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "solution = problem.value(hill_climbing(problem))\n",
+ "solution"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As you can see, Hill-Climbing is about 24 times faster than Simulated Annealing.\n",
+ "(Notice that we ran Simulated Annealing for 100 iterations whereas we ran Hill Climbing only once.)\n",
+ "
\n",
+ "Simulated Annealing makes up for its tardiness by its ability to be applicable in a larger number of scenarios than Hill Climbing as illustrated by the example below.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's define a 2D surface as a matrix."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 73,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grid = [[0, 0, 0, 1, 4], \n",
+ " [0, 0, 2, 8, 10], \n",
+ " [0, 0, 2, 4, 12], \n",
+ " [0, 2, 4, 8, 16], \n",
+ " [1, 4, 8, 16, 32]]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 74,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeAAAAHwCAYAAAB+ArwOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztvX/MdV1a13et93ned4AMZZpIUueHjka0NSRCO1Ia0taMNB2RiqZJiwYTf2WSWuPQ0FLxD9umfzVNiH+UNHkLRBONaIttLdUaGiGUhCIzCAYcNRMYwxTiSA2BSWfeee9ndv84977vfda5fq9rrb32Ptc3eXKfvda11tr3eZ7nfM73Wj92WZYFUqlUKpVKjdVre99AKpVKpVL3qARwKpVKpVI7KAGcSqVSqdQOSgCnUqlUKrWDEsCpVCqVSu2gBHAqlUqlUjsoAZxKpVKp1A5KAKdSg1RK+WQp5eursj9SSvnRgL6XUspvae0nlUqNUwI4lUqlUqkdlABOpSZRKeXdpZTvL6X8s1LKz5dS/vSm7mtKKT9WSvmVUsovlVL+u1LKG491P/IY9tOllM+UUv7DUsrvKqV8qpTy7aWUTz+2+f2llG8opfzjUso/L6X8WU3/j/VLKeVPl1J+rpTyy6WU/7aUkp8fqVSD8j9QKjWBHmH2vwHATwPAewDgdwPAt5ZS/t3HkFcA8J8AwK8DgH/jsf5PAgAsy/JvPcb8jmVZ3rksy199vP6XAOCLHvv7cwDwPwDAtwDAvwYA/yYA/LlSym+W+t/oDwDABwDgXwWAbwKAPxbxu6dS96qSZ0GnUmNUSvkkXAD3sCl+AwB+EgC+DQD+x2VZfsMm/jsA4Lcuy/JHkb6+FQD+7WVZ/sDj9QIAX7Esyycer38XAPwtAHjnsiyvSilfCgC/CgBfuyzLjz/GfAwA/utlWf4XZf+/Z1mW/+Px+k8CwL+/LMvvbnhLUqm71su9byCVujP9/mVZ/s/1opTyRwDgTwDAbwSAd5dSfmUT+wIA/q/HuN8KAN8JFwf6JXD5v/sxYaz/d1mWV4+vP/v4859u6j8LAO809P8Lm9f/BADeLYyfSqUYZQo6lZpDvwAAP78sy7s2f750WZZveKz/7wHgH8LF5f4LAPBnAaAEjq/p/32b178BAH4xcPxU6u6UAE6l5tDfBYBfLaX856WULy6lvCilfGUp5Xc+1q8p5M+UUv5lAPiPqvb/FAB+M/gl9Q8A8J+VUv7FUsr7AOAjAPBXkZhUKqVUAjiVmkCPqeJ/DwC+CgB+HgB+GQC+GwC+7DHkPwWAPwQAvwaXxVQ1/P5LAPiLj6uY/wPHLUj9AwD8r3BJS/8UAPzvAPA9jnFSqdSjchFWKpUSVS/ySqVS7UoHnEqlUqnUDkoAp1KpVCq1gzIFnUqlUqnUDkoHnEqlUqnUDupyEEcpX7IAvKtH16lUKnUA1VuoLdfaOq4N1r61X889cvfDxWvLvGNGxVL6JCzLL4sddToJ610A8OE+XadSqdT0er26rj9qufrXiXKuTtPGM6am3FKH1dfXWBsqjovXtrX2pdHvlEMgU9CpVCp1YlnAM4usAIyE71glgFOpVGpX7X0k/7yAGq+xfxcJ4FQqlbo77QV9L+zP+SUhAZxKpVKhOics+qj3ezV67temBHAqlUodQtxCqyhxi6k0bSx1qQRwKpVKdZW0CjjlVxTg9/mikABOpVKpaeRxuTM549YvF5btR1Grn/dz6QngVCqVCpMVQD0//F8nXu+lGe6h1r4p8gRwKpVKdVNU+jkSXp55Xk1fljrt2L3d777KGfJUKpUKUeuH/t4p01Yw94S5RaPfR//RlQngVCqV6iLL8ZNSW0qaoyIjwTjSwXvc7wj4RpwVfVGmoFOpVGp3jUqZas+PtvZlleXLSLTmgC9AOuBUKpUKUE+Aah+I0Hs8i0akyWtp3g/PfcVCd6sEcCqVSoXL+/QjqU4zHpV+bv24124/sj79yPrkI2l8Stbfvx94VyWAU6lUqknWx+95Fel+90w/e9UypqVtf/CuSgCnUqmUWx4QjnK/HrUeBCLVed1vS+pZ+3uMA++qBHAqlUq5pAFF1GIjD2Q1c8fRp221ztFa4Htc8K7KVdCpVCplltWladSaCp7d/daawf/tB1+AOd6BVCqVOpA8DhSAd8eWOuvYVHvP+c49n3wU5X7nd76rEsCpVCqllvZwiJaFWR4Yag/esK4WjrhP7FrzfvSC7/7gXZUATqVSKZV6wVfrKj2uWNM+YrEXVzcLfOcB76oEcCqVSrGyfOi3Pg2pdR6WGr/nimoLRjSxVvgey/VulQBOpVIpVFa35VkV7anzuF+q3DO/25J6luKtccdzvVslgFOpVOpGI+DrSelyfVpTzK1zzVJdROrZm3KeG7yrchtSKpVKXSlihW2Ped9aniMnW1dEa/ryxkb1dwz4AqQDTqVSqUd59622LrqKTgNr+m4d03reM3df1D140s7HgS9AAjiVSt29Wg6MiIQv17f2HkadeEW1wa6lLw4Rx1EeC7yrEsCpVOpOFQleLD7qIA5u7EjIRhwMYj16837hC5AATqVSdyXrQRTa9r3gG7HoqnUvsLZNxP5nC8CPC95VCeBUKnUH6gVerN2IU7Ba9/VGrMDeqkcqnhrv+OBdlQBOpVInVit4pT72OAVLGxd92hUo6xK+WiWAU6nUyRS1jaZ1L3CPfcD1dS+HG3E/Uj9YPRaz6lzwBUgAp1Kp02gv8GJtZoFvrehTsEbA93zgXZUATqVSB1bkoRE9wCvF9IZvrzaWPjz1qw4C3/r2H3zNUqlUanJpoQvQ5na59qO3Id0bfCcDbydSJoBTqdTksgAXoN3tcn2M3obUc/529JzvQVLOA6mYAE6lUhNqD+hy/XjBK8UkfJ+1E3h3pGACOJVKTaQe4NX02+PwjTrG4oqtMMMU3caTuqbGngC+UfTD+nk19hZSqVTKIStwAcZAl2sfvQWpro8A3V5zvhPP93pp15GSCeBUKjVQHuACxEFX6qt17y8W1yPFOyrt7F3BrakfAF4r4QYTUfU84FLKh0op/6iU8olSyp/pfVOpVOoser36Y9HLzR/tON6+qPZUOyy+jqtjsHouth4Pq9P272nTsn1qR/ha/tm8BFt8sMQhSykvAOC7AODfAYBPAcBPlFL+xrIs/6D3zaVSqaPJ63BXRTpdqT+r26XatKakW0DXq03EPdV1AN3BGxET0e7zcd1+DQB8YlmWnwMAKKV8HwB8EwAkgFOpFLRB1/rJ1ppilvoYuf1Iit8rHd1rvrcTfKW/7onT0Jqh3gMAv7C5/hQA/Ot1UCnlwwDw4cvVlwXcWiqVmlejnK5lrD3Bi8Va4BsNWK7uJPCNAu+OK6E0Q2Pv3HJTsCxvAsCbAAClvPumPpVKHVUjYWsZrwW4XHsLdLH4HuD11mnbHCTlHAHdVuB6qens6lMA8L7N9XsB4Bd13adSqePpqMCV+vLMB7fMBXvBW9f3hLIlbkfXy/3VRaegvW06DfMTAPAVpZTfBAD/DwB8MwD8oa53lUqlBqoVuADHc7lc20joYmXRKWWuLiLdbBkbIAy8I6C7Y/pZNfyyLA+llD8FAH8bAF4AwPcuy/Kz3e8slUp1VLpcXTvvqucW9xjhhrl2k6ebqb+KPdPP1naBKWhYluVvAsDfNN5CKpWaRqOBaxnzqNDFYkeA11vXAlcO/B3B2+KCtTGe2CDtbMBTqVRfndXpSn1Ywduy2pkqiwJcRN3ErtfqeCPnfGdPQadSqaMpoatrExHfulCp98Ksg4HX63hnSj/P0XUqlRqrFvCeaREV167V6WLlmoVYo93wxOlmC3i9Llgb44kNVAI4lTq8vOBN6OpjJWhpYnpB8aCOd4/UcwTxNH1ELsJKpVKzaZTbPSJ0e6SWtWNZY6LqJgWv1u16oNsDuIOJmABOpQ6lEW53ZuiOnM+l2nv2A/eY+43sZyLwtkBX+8+8B/m2faYDTqXOpFnAe1ToalPLlvYt0JXqpbYR88RYfTB4ezpgTb02JqLNvMOkUimfZgGvps/ovbpndLvW+hOlmkelnidPO08ydCqVojULeI8I3T3cribmZI43Os3she6ItLO1baagU6kj6gjg9aaZZ4XurG44atV0XT+R251h8VU64FTq3nV08PZ2u9pY7VizQre+nsTtYt3NAt2RC6+0faQDTqWOoBnA2yPN3Au6Z3S61nhL3Q6LqiKgO9IBe+KDlABOpXbRiH28ezjeiLSxNrYndD0xPaFb1x/Y7Vrnf6X6GR3wPt2lUilZR3a9MzjekVDVxCR4xRht24g6S0xEmwYlgFOpYTqy6+3leM8K3Tqm5x7hidPMoxZe7b3oyvlXkgBOpborweuP2wuonhjp/qPnkQO3EHmg2wrX3sBNB5xK3bNGgFczztHAO2ofriam1elKfe4IXaxsr7neVuCOnAfW9JcOOJXaU7O7XusCq2jw9nK7e6xgrssmcrpYd73mea1w7QVjbUxLfJASwKlUqGYHL9d+VvDuAVWp3vpl4Y6gGwncSNhG0U7TTzrgVGqk7iXdHDUHq2k3W711DvjOodsDuCdyvzsPnUqdRQlfW0zvudmZtg1p6ncG797zvKPmf62xLX2mA06lRuhs8G1Zsbx32jYS2gndZrgexQF72wQoAZxKudQCXoB953tbXG8P8PbcI9vidhO6zWVceUudpt4bO6KfPt2lUvegdL14zMjVwZFQ7pnO7rhtKAqoe6egpbromJb4YCWAUymT7mGVsxUyWEwUIKOgfFDoRgC1N4StsS11lhhLXHT7nANOpSJ1NteLxUe43hGu1Vt3Euj2grC2naU/Tz+9YlriO2mS20ilZlbrfO9s42jHOit8I1PdO4J3hhS0p7ylzhJjiYvuIx1wKhWhe3S+1rT0aLh6U83ePiaG7mgIR5ZLddExLfGdNMltpFIz6mzwjZjvjXK9PcEbPS7A1NCNdLR7gVhTr42xxEW3XZUOOJVqUcJ3zHxqJJSjoe6ErhWIo+Z4ZwJuBIwtcdbYFr2EBHAq5VeudO4D31ZnawWvB8hBTneP1LIXsBaA7gXbns53RwomgFOpK50dvhJ46xjvQRWRgI0Cb2e3e8ZU80jXGxnjiY3sJx1wKmXV7PDl2kbAV6rvlTo+GHhnh+4RgHvHrnerSW4jldpbs8PX2kazx5eTxflq+pgBvpOCd2S6WdtXZHlUvTXOGjuinzHdplJH0hHgG5l2rmM41xo5p6spnwy8FnBGxWLX3pgeZVLd6FRzOuBU6qiaHb6te3ylmJ7wjSifALxakJ4h3Tw61dzDzbZSLYKKOQecSkm6B/hqnClW3wO+1HgW1zsBePeE8N7A7QXiXnHeeKvq/hPAqRSnUft8vX3NAl8NNK3lra43GLytoG2BqMfl7g3hljpLTI+4qHZBSgCn7lCj4CuNE/XfT7PPl6vrAV8vRLV9DQLvDC7YG2Mp85RLdZp6bUzv2Og+0gGnUnuqBb69D9nQ1mnisHLt4qy6TNvXAPhGuN+90s0WkI52vzPBdgL6TXALqdRIzTDvuyd8OSBa5lcj5oFbUs6B4J3Z/Y5IN/eAbY/U8VHcL0A64FTqVglfvM664CoS1No2ja43GrwjoTsKuHu73jtwvLUmvKVUqofuDb7WeizOA1RLe63DbXC9reCNSEVb+tG2by3zlLfUWWI8sZ741nYBSgCnUqyOCl/PXt/Z4Iv14Ug3t4I3Iv0c4XJngXBEvTXOEx9NN0t/mYJOpVZ53e+I/x7W7UaauB7wpcaMgC8X3+B6PZCNTD/PlnqOLNfWa2N6x0a066CJbiWVOqr23G5kiWmBr7RCmWvfMt+7KgC+HuBSUNwbvHtBdy/na20TSTZPX+mAUymA/u53ptSzZcWzJE86OhK+RucbBVmvE7bUea61Mdq2UnlLnSXGEzu6TUdNdjup1L1oJHy5/lrTya3w5cZ3ut4e7rcVyBHXrWWecqlOU2+NGxUf3X6rdMCp1MzuVxvvWfHM9eFNMU8IXw6aLRBuAe+RIdxSZ4mxxLW260k4ru8EcCrl0Z6pZ400kKbAGrHiWRNnga9zpbMGoNY6bz0XH3HtjfGWS3Waem1MS7y3TUu7YE1yG6lUtFog17Pv1tSzFKOBL9Ve+lTHnGsdxzn2TvBthbFUpqm31HmutTHathF1mnprXGu7KKINImMCOJV6Uu//vXvO+0pxEmgtW4cGOt8oCLc64vp1j+vWMq68pc4S0xI/epwBmvjWUimv9nS/vT4drP1GzPtyZVh/VJ3GOU8GXwuQufiIa2+MVN5Sp6m3xrW2m8n95hxw6j6156EbXB+W+2pNPVOi4Mt9mmv36dbul4Nvw4KrmSCsfR1xrY2JLNfWa2Na4lvaTUy5iW8tlRol7X8DL9xHp541c79YW+scb13GtafGEOCrgaYHtFrgWsBsqfNcW8o85VKdpl4bE9GmpV1rW0npgFOpker1X0mCb63RqWcOvpIb7gjf3u53NghbY1vqLDGe2Na2kf8VvX0lgFP3J49Dndn9Wj8FvaddtZxSRSkIvhT8ejljT1n92lKnuW4t48pb6jxx3viIthPSbsJbSqWOpohPn+h5X8tpV1x/HHy1874Hge9MEPbGeMulOkuMJW7vdlHta6UDTqUk7eV+NbL+1/QAXCrTAJ7bXhQkDHxY+SwO2ALbSPD2gG4klL3xEe0nJd2kt5VKWdVz6xGnI7lfa1ndnwWyQe43+qc31lJvrdNca2M85VKdJcYSt3e7qPbHHDqVOoIi3W/UJ1hdbz3xSlOmgW3n1POon9Y67WtLnebaUsaVt9RZYlriI9r3pJvUd6agU/ejXouvog/d8IwhxVgPyeDKWlY9HxS+VuC2QliK1bS3lHHlUp2m3hrX2mbvtsGa6FZSqSOJ+6+jdb/Rx01aFl55z3XWyNhub+j2cMKWOs21NsZTHlXvjY1sH0W01n7SAadSlCLcb3S7vceKdL91O+bTiGqSENbHaNtq6zT12piW+Kg+JqbcxLeWSmm0x+KriE+eVvdb1/VyvxJ8pXYOK3AUCHtee661MZ5yqU5Tb42bqW1Ee0zpgFOp0dprJXarWu67oS0HOWubnvDlxudeW+o015YyrrylzhPnjY/sY1LSTXpbqVQvaf7JR4LU+2kWOffL9YuNQbncDu63BZDePlqd72gH3Apirs7ypScqbqa2Ee0xpQNOpWr1/Are8rQj6315j5zEylpWYTdstWoBqqfN3hC21LXEeMu19dqYlvjZxvX0nQBOnV89Ur4zpJFncr/WbUerFAuvNIDiYGht0wJfL4QtdZprSxlXLtVp6q1xUe0i2k9CvkluI5U6gqzuF4u3ut+6vof7jbYoyq//2yYRLtfTZpQD7gHeSPc7q/P19BFJtc6ETACnDqoIEI1W66EbXGyL+61jWt0vIyvwtLEREPbGaMeX6jTXljJPubZeGxPZrrWPGT4CKk14S6nUXtpj8ZVFmiMnJfX+L+9wv1wdBWUuVtO/d0yvA7bUSWN7yrhybb02xhMb2U/0P3FPfzkHnEpFypp+9vTpXZxFuV/OoWrTz9R+Y0ca2+N+ufYW59r6UyrTvrbUYdeWMq5cqtPUW+Na20S0n4x4k91OKtVDo/+ZY+O1uusId869D1JaWfMeDnC/HpfaA74zgDcy7Tyj+90zzdzaTzrgVMoi60MNWtViPzTbglpgK2mA+21xu9y4e8B3Rgcc4X73cqF7gjlYk95WKsUpeqtQ70cORi++ksbz9luDtRXURPdeJ+Z1zVR7y/1o4euF7ewOeKT7PRLYKaUDTqV6auTc74yLrwynXlkAh5VFO2MJUB5XzL221HmuqTJPubZeGxPRprWPiSk38a2lUhGa6WuxR9rUeHT6uWGxFSYrzDR1dYxlXE8qmrun1lS059pSxpVLdZYYS1xk+9n+m6cDTqVGy5t+7qWW9LNFhsVX2JDeOi+wLWNZxuReS2NGwJgq75163iNlvJf7DtaEt5RKjdTei68ijp2U2rfAlbsPxfujdbqaeGudFOuN04zbKx2tjfGUS3Wa+tb4vceL6mO+oVKpCM14VvPe47Tej/aTujH9zNV5nSNX1wJlb6pc+1qqk+K1baTyljpPXGubvcbrJPGWSinfCwDfCACfXpblK/vfUioVpZH/4zTp55H3Y3nwQt2GKsf6aFx8pS3rAWWLtI47XbA/NqL96PEoKWdlXlPE/AUA+FDDraRSk2p0+lka35N+ltLSvdLPRmndI1fG9eVJG1vcr0Yj4PtSEcPFYeVSnaa+jpFiuTbesTRfHLzjdZI45LIsP1JKeX//W0mljqDZ0s+9FbT6WVunjfeAlpMF+NJ97OWCLWVcuVSnqffGHmksTqNXQZdSPgwAH75cfVlUt6nUQTULQL3awQ7sqQc41q9s/fLS2rdnHO/9eNqNahOssFtYluVNAHgTAKCUdy9R/aZS86vVYo1UnZKe5b5SXWR1vy2uWBvTEj96HG/fuQ84lTqLrNuPWtz3QZz76lgx58rV9b6fvRSRep49Jd3beUe1n3OoVCr1rD1XSE+svUGmkeUeZ/t9IqGsqbfGzdZ3Z4mroEspfwUAfgwAflsp5VOllD/e/7ZSKUpv730DEynSrXZyvg99ujWp9R4eqp/e8fZ+L0bDt3U19F59W/vC/kSloJdl+YO+O02lUj5FwXCSr/lR4tzkXq50Noe7qtU9jkhH93KuIxd/NUqzDziVSgHAYeZHj6LRjrAej7q2Ot570l7wtbhia5+trrhBM353S6VS9y6vs/QswIpysdt+qD6pmJmcdC8H2WOh1ojFXx37muWvPJWaSK3/LaxOmRsvXfeNLJCNhis2tjSG5x6igWxZJW3pQ9NPNEx7r4YeSMVMQadSpxIG7AkgzqV1vXV1jGV8aaGUd9yWtPUMKe8jLNSK6q+O3yEVnQ44lWpSz1UhKZNanGaPFLAm3awdd6YU9apZFmr1dsSevgIfxpBKTabcijSFq5Xkda2j3K5WXH89F2zN4IgB7OlrqS7a7WpivIu+qD9BSgCnUsPlhWdPCyR9qelEg6jUdAuYpZ9acYD2vNaMpb32KmqeWNtXBHg9ae2d0tAJ4FQqtVGH7EIUDDyQjZa2/9kc8Qg33ZqSjgKvRjtuPdoqAZxKHV49PkmoT+wa0I7nrkSnmC2x3kVYWocuLczSuF2PI45Qr7nUqJR0Sx8tDvflYv+Tc8CpVKpdlCN2kMGSJvXEemBlWe2sgTFVZoF0Sz97ybNKunX+NjIVDUDDtKMSwKkTa6ZPqBQpC9h6zvW2zCtTZZZUdetccK954K0s7rHXQq2oVPRA0JK3sMuoqVSz3oY5VwLPeE87qt4+o9lOo2lTbx/yjGMRNx5XhvWhfa1pO4OioNxar4WuVy8N32iKbpx0wKnUVDopwD2rdiPbRLpgrWO3rIrGFOGCuf56Ombvth9PvdjW6HJfPuB/OigBnErdtXZeWtuSYg68DdV4VhhjcZr6yMVZe8zCWOeDe7peD3QHKgGcSqUErQuxHqrrVR3mz1rmY3vMAXudcc/FWTOodeV0D9erdbxR0H356vZProJOpVIX7XxymDa92xOyUnttPRfrKZP6bn3N9bu3WhZpoeWdoIsBdvunQQng1IGVR1L6pXnvBn1aW+dtqXaWMTT1Hmes7VPqj3stjaFt20PWYyu1fazlHHzZPg3QDQSspARwKpWqZPliY/x07wHTnqlnaQyqThOP9U/VU7Ga8paxeilqnlhyvRrwDgRurQRwKrWrZtpP0pJREByIFhpR0LSmmKPngaV70brfKHfsgbMk7wlWVJkHvuS4AnidwH3t5SvVH+26iARwKnVX8loeaSFWgCLcWIs77OWIPWXYWFQ9FTujwtLRBOC04FUIB2usEsCpVGqMvO511E9OFhhr4y1lrWCObEvJm8yJcr7B4B2hBHAq1VUzpZg94j6da1e8w3F+kuONgLsnPS3dS09HbE1V91LLQxai4Suop8tlxx06WiqVmlDYh1enFea9XKvkZjVzuZ75X01MzzIgylpS8b3UvGcYgS+XchZcrxe6L16+Ev+UPIoydR/KrUix4t7P2vEO0p6pZo2jbYFwXRdVpqm3fFnpIcuWJQq+aKwOvBphcI1UAjiVUmnmM5q9dkf75SU4DT0CmD1SzVrQUjHWPrG+qDKtS6a0pzOWZIUvIS14e8EWvafuI6RSp9XR53e32vkTeKSbbUk1a2OjXLK2LzCU7fFXXf9XUW9PaoevBrwR0H3x8uHpTz4NKZW6S41KyXOf4g4X3HILFjhHttH2YxlDqx7paW8iJUId4cvJC90tbJ+g61ACOJW6C23BrP2wwGDOlTV+Uvd0rL3aRI1prQOkzlJmAbOnj5HJIQd8LeCNgi2mBHAqlaqEQdZqr4JdcC+Xq4FwFGCl2Mg6qWwmtbhfBKJcylkL3h6wxZQATqWGarbFXJ6UNdfG+YEV6VyjY6UYi6NudcK1IvjQG9Ca+d+r+jb4UpLAOwq6WyWAUyfXjF/5ZxX2XkmAXttgcQYXjJW1uF2uzupO9wC+5r48ddrxNPWj1AhfyfVaofvy5SvxT8nnAadSKb+0aWhMxk9ub4o4Eoxcf5r2lrGi7ivSGY+CseSGa/cbAF9KWvDWcI1UAjiVSgmSFmNhLngtUx5PGeXwvDDnYnqAFZMlVtPG8p5IZbMoAL4a8PYCbq0EcCp1SLVuN/J+yg78dG51eFFOVtO3B/hRoLXICuM91TAXy8FXHHbgedAJ4FTq1Np+4FDQ9mxRovoLdMFS3V7p5Ii0cWQK2ut0veqWjhb+nSjdrwe+HsebZ0GnUilEUYdxvE28xiR9KjekoqPT01ydBnhSe21MT2fscfFcjLdeK83cbyUtfLmUswW6Pc6ETgCnUmGabYuRRRZoS67a66iZYeqyFtDu4YA1/Xp+euSFcqSLtqoCngW+ZJcTnAudAE6l7lbUhxMFWIsLdm5LqrvRwBEr88ZLgLWAUXq910+rwlxuTDet8JVcbz6MIZUK05kemBChqPQ0BVvJOg2GcKQ7toCOGkeCYXS51YGPEvffsgan4jzn2zIavlw/vnOhkUcW5j7gVCoVIwra0qd2YypIM4rRAAAgAElEQVRaYrk1XRoZT8VIwNOOEVVe188obvGVIvVcywtfrSLnghPAqdSV7tkxa4BpXUndkIredmVN52rLPC5Zajc6lWyFq/bLgOaLx86qAWiFr/5s6D5p6QRwKuXSHqD2fiL2/CS1WC5HKnrbzALhnmAeCd/6/rgyb5+WfwK9wbsFqOB+tTDk4MtpxFxwAjiVujtp54EpJ6txwVT5IAhjZRQ4rQ54JHwfkGtrHxHy9tfpe6p23tcD31ELsAASwKlU6kqeRVoUqDXlnSDc0wFT8S0/6365Oo07tt6nFK+RB9LS4RuPkuZ+tfDl4Gp6RvCLV+yfovx3nQBO3amOvGd3L1m2JK3SQNggDpDUa48D5oA1g9OtX0uxknqnljVi0s9baeZ9KfhS/anmgTeAjVICOJUSdURYew/WsNRZFl5xMrjgeiisTFsvxVniW1LDHpBaIDtberrz8gkrfCVFQ3erBHAq1U1HXVHNAZSrG5SK3jaVnLCmnoqLdsLbsVtAbfkiwKk3mIO0TT9L7jcSvj3BuyoBnEoNU7STbnG5ddsIF8zVdYTw9rUmHa1J63pgLf1sSTNb6ywx2jY9AU2kn7m5X82TjTzw9YD3BTxc/dH+W04Ap1IpRByg6zrt3O5ACFuATLXxuGLuJ3bfUfDl+sbUAlMPkAckg2r3i6+UZhZhKcFbw/ZFw5uZAE6lUg55UtFcXTCEt68lIEeUaYGqAbi2DpAybTxVNzINrVwBverFVSpaTj1z7a/KFeCNgC2mBHAqFaIzLNSypqm5VdGTQXj7utXtetPP2P22wNTqdDVjTiTNsZOUbueKafiy/XSA7lYJ4FTqSUddNKVV70/ZCSDckoZuKYtOKUuQjEhL90pRc8L+iym2H1ncbwR8W8D7El5pn8WQAE6dWb2AenZQc5JcMifPgi0jhCmNhDA3fi9wtrha6/gTy3KQBlln+GVfwqubPxYlgFN3qFnSxb1B3nqqlacPS9paW7eA+cQsyglLALSknLkyTapaWxb15QGEsonEbT3aSpr7RRdiEfDVuN4W2GJKAKcOrt4wnQXWvaQBrHWuuAeEAboszrLWa2HNtbP0ifUl9UcpwjnvDO5t+tmTesbgK4E3Eri1EsCp1LTqAX/NJ6jnU7YVwg/Kus4rpKMcsPQzGrTS70nF7iVsBbThEA0pzgJfSr2gu1UCOJXqorPNE2NAjXTCdT1XN8HiLE06m/vpGZN7jSna9WN9t8IcOUyDWv3Mud/ruBj4WvQCXl39yYcxpFKHU6vj9X4aYmlorC/PnHJPCA84tMPigDXQle4Fq+faePs6kTRHSt6UEW+KxvXWsH3R4JITwKmTyupAsXgtEDVxZ51LloDqaaOFsENeyNVl1G1409XSPVjgq9FsgBbSz173a4Uvp1bYYkoApw4sD9RaQXiG1LLW3XrjpFS0pk0N4aAV0nXzCCdsSTdHpqDBWO5JWWvG2FHaOeKneOSXkFxvD/CuSgCnUofRkeAfDeG6vq7baYV0K3wl8FP3TPXLlWP9TAZV6fSrl8z2JMn9UvCl1BO8qxLAqdShwNZTkS4Yi6udLNauZfX0TmdIa+Z6LcCV4N1SvqcM/824k6+keIA2+GrBi80FP7fNRVipU2uP9HOkRkHfs3CqVnTKOhLCdX3HFdIaCEt1dUwdZ4Ev16c0lqcfr6h/6itYHWc+S48TvLo2wpcdN2Dh1VYJ4NQJFQG3FlgfwVFHfMqOhvAO88KtZaPmdFthOgrGAKanIGkWX1nngTH4clCNhu5WCeBUSq2WldJHEeWYqU/iaAhzkMX67jwvjLli6/ywta0lNV2/xq4xWcE6IIWtffpRi/ul4Ev2l3PAqVRPjXCrZ4O0Rl4IY9oJwly9NPcquVOre/XAdwSYOwgDrMb9RsLX6ni3zwt+AQ/5NKTUmcUBjQLqKAh6gT7TnHaEC6biPeloTUyHxVnaOWHPXC9Xr32tGctap5E2vvG7rbT4ypJ6tsJXUg1crxLAqdSNZnCsnnvYc+659bGEnqMuLfWd54Rb5ou519Q9adu0yNsn98+wnv99hGhr+plzv1r4Sq43Ari1EsCplEqR87+zLNLyfJBE2SVtOvpAEJbKpL6xWCnlrZHXGfeUYmvRU6jxIQ0SICn4cv1FQnerBHDqYPKkn6Pi70mco41IRVPxXgg/VPXcCmmDJLhSsdr23GstfGeBKiflf7UVrFj6WeN+pXlfC3yt4N0+tjAfxpBKPcniVKO3H93bOdEjIexdnBUI4WhnzJVjdVHOeFJZ3e9VWwG+VMpZA94tbFseW5gATqVcGg3NPdy6xwVz7aIhzI0ljR+wOpors6SgqZhWHQCwreLc71WcEpBa8EYpAZw6kPZKP8+Yqp7BNc8IYSwdTbVvWB2tKa/LNCllb50US5VRioI3uyDregGWNf2MPenoedgtmLWLsPBfutXlckoAp06uGUC1lQbmI++59ZN2BIRbtykFQrjuoiUFTcVY67BrSQdIVVvSz9Kq5+vY2xXTnqck8co54NSpFOl+PWNIcWed/209SzpqT3HrNqUgCGtdZuRcrwRADSBHQFT6b7huQTKsgAaQ3S8HX2ze9/qadr3ifbEPY9ApAZw6sSi47ZV+3iOVHQF4CcLSB2ovCGN9tDxXWKnWFDTXB9WPN1Zq71HgP2Mq/dyy+Oop3gFf7bOB82EMqTvSXu53lrnfWe6Dk/dTfTSEubaOdLRnrrV3qjmijVbkk44UMQp53a8Xvug95MMYUimrItyvtt/IQzqOfrgHpch9xa3nR3PtlRDWQC5yDpgaQ7qvCIUmgR5v0OpmMRAHwZdyvR7orm3yLOjUSTRijnSvedjRkG6RBnreVDTXVjsHbSUP9wCHhiGj5oBbYiPaeRUIa+vKZ7QP5Zyv1I6K8c79rkoApyaWBJojbT3y9je7s7UqCsLRx1YGbE+K1GypZo+u0tD0e6qZ/71po3S/VJu1nefxhDkHnEqxYOp18lXv9HO09rgPzSf8TBDW1ima9FqENaP7DZjjtT6AQXK/XOpZ+2jCqMcTaiUCuJTyvlLKD5VSPl5K+dlSykfC7yKVulF0enY294vd51nnf61qhbAUo5kPHnhmdMRK5mjYtvyTErcjYSucr93vi+rnVaxiz68HvlQ/nucCa8+C1rzNDwDwbcuy/GQp5UsB4GOllB9cluUfqO8qlQrVLNt5ervflv61cZb38m1lvw+KfrV9aVT3pRmfUkvbgG41cZ1ukZUIVUW54QELN3GI+6VSzxb4ep8JHPV0JNEBL8vyS8uy/OTj618DgI8DwHtCRk+lUHm3HVnd70xbj2ZJW0ep9QMqelHWABeMDSvVj14kRcnzz17bZjv/q3j+r8b9auB7cxvKJyNJT0fa7XnApZT3A8BXA8CPI3UfLqV8tJTyUYD/L+buUneoGUHU4n5nniO2ynLfZ0uLM/LsBU51V69n+EZKDeBSyjsB4PsB4FuXZfnVun5ZljeXZfnAsiwfAPiSyHtM3Y1aVj1HuN+9tBfg7wiSKbvqfx6WNLThn5a0+tm69QjTyOcCW6QCcCnldbjA9y8vy/LXu9xJKsUqatUzFx8NvZndrxW+6X6nkTr129B2gOxHSyJzyMq5X7lv+SEPWl1gH7QIq5RSAOB7AODjy7J8p/luUilRvaB01EcO9gZ8T/hGaaa/j46K+jV7vV0tLviq3OcgpcVXGlkf0PBcrrvnlscUahzw1wHAHwaAD5ZSfurxzze4R0ylrtQKjJ4Lr0a439Fw6w1fbf+zZAICZHGaM6xeHvV8EcM4mu1HtTzuV7ulSIJv1DOCxbdoWZYfBVAfbZlKGTQDfK19zyKP+50Fvi2K+ns5uMNuBWtL+8a3TnsAx1beOVjrs4E1Y7VCd6s8CSs1sSLnfT1jaGJb9uWOnDeeCTjRXxQ0fcz+hepRPf6aev/Va9PUzAEc+qH88Is4ySoSvpf+Uqnh6uV8uXYj3W/kf6s99ip7fv89P0pa/74C4TzybTiCC2bOgI4SlX6Odr/R8L30mUoN1Uzw7eF+W+Z+W/47Rj0zuTd8e2w166XAmbegudLw8TUQ1tyf8Xfg5ng1Zz97HhMotafg2wO8qzIFnRqoHit0ve16LLyKVgSkR+cfOY0CaM+DjANkXUkc9Vce/baoXfAz2LD533oBVi39IwT9e3X3gO+l/1Squ6IOhvA42b1Tz3u63yM5X6k/qr2m3zpm0Mde5C64nrcctVLaeRCHvnv94RncIwq17tkC37pPbe4kAZzqrN7w9bQZlXrec4wzwdeinfY4a+BjAdSM25Wa/on7FmBJh2+gQwnP89WMoYFvzKKuVKqLIo9DHDHv2yP1PGLfbwuoRyy2ioBvpPvV3scEOy81AOy939gK4asvGfoFWPX+X83xky0nX7W3i0lN5xxwqoMiIRO1uIiLH7XqeaanL3m0h7tsgS+m4Pd7tJPtsfK5RY7+rEdQXobxAc8DSs/qaa+O9L8/Nb2izwvey/lS8aPmk6NP4NK0k9QLvpGrnq00fJ2pc6j1+6AWrpayHq7Z+aWDW4ClVWT62Zp6jgTv83ipVLOiPyi9W1VmPpxjxLGWPeDr+YjoDd/eK9WD0889tiC1grX1k187n+08AxpAf6QkFRcJzB7wBUgAp5o1C3w9bUadCz3DKUwzuV5P354xuPE6Hb7xEinjbsNaL8W2QtgLa+NfJ7cAK+Lxgzd9NrjfXvC9jJlKmdXrw7zHIQ0RbrnH3O1I93sU1yv11Wu6wLn4ygtLKfXcuvrYCmFsvJYFYDdxG7eKrobm9wGvYLSkn63QjIDvNrZEPY4wlXpWTxc1A3xHrXpumWyz/h30hu+oef+Wv7OO7hcbgnLBIyFsVYTLNa6Ats7/rrKufpbcbyt8WxxyAjil1D3Cl1LkfGNE+6g58Vnha9EIt+9v1jRGj7lcy/gNsj4BaYWkZ/UzB0T9qVq5Dzg1hXpu92id7+0N39bU84iFV5axKc0O3sgsAjeWY/GVtBipdQGWF8LWFdDe8VmXr1+AZdn/+9RG8eCFllXT9Li5DSnVXb33WfZwvVy70XPEvVfqRvTZc65X2791tbOlLwm+A+Z+63hLWtoyvgW4rXAWIYynn5/neh9uyiS9UM4D8320u9/oBVkJ4FSlEQccHBW+PQDaMs5I+PZw1F74en5vCc4OYXO9mvlfqh8prm7TA7gtZZic87yXIXSpZGrxlcf9joTv5R5SqS7zj95xRsHXOoblQ7/3Xl5LXGu7HnOqnpQz13crXAPcrwVSUiwG79YlCty4mvGs97zRa1eroB/ndpGV0Wv6uXaqEeDTLLyixsltSKlOOgp4pT488LUAdRR8KWljPe9R65iW/jV99pgqCEo9Y11YwGRxuh4ItkjTt+aen177D+CgJIHQ4n61fUf1QykBfJcaBV7tWDPANyK2ddFWr9Rz9JclT99e18uNoXlvAuFrAZCnPy0EvWXUGJzMEObnf6Wymxjh6UW6pxbx7rcFvq3uOAF8d7oH+HrGaz0Vq8fBHhHzy73gG9nvHvBtUEt6VlzA5LyH1jKLtE7eMP9bp581YPXM/WpXPec2pFSgeqy0bR2vBbxc+97zxK2QGuWSLf3u5XqlmIgvS1R9gPv1zvtaIdw679zqmqWxb/p5dpza+V9JXthJK59xh51PQ0qFaEbwavr1ut7eK6T3nvfdG757g5dqZ3W+DfCl5n69Md5rTlZga8a2uHyFPI8kBMCB6nW/VvjmKuiUQbOlmrX9nhW+lEa5ZM0YLf1Z+uz5dzUAvt56S91aX/c/cq63w/wvdvwklX7m9v5a535b4JuroFNKjZ4ztIzZK+UstY2Ar2Xc1rRpD5fs7T96/jhyWkHz3jXCl5LV/WqdrzYF3JJStqSZ6/vDrsl2FfCQVdFWF2xxv9pDN7C2mvIoJYBPo6OmmzV9eeAb2Ua76CpyzlKK7QnfmcFLxXeAbwtYLbGY06X6kcpHwJm9V3z+VxK199frfrn424cztD6M4Rb2+TSku9LIdLNlvJ4pZ8t9aNq0wtc6nja21SVb40emm7nxJoQvVu8Fryb9zF17pE1Va8ZmIYwvsnopPHbw0hUFwxj3G/mEI6vLxpQAPrTS9ca0i4DvrPO+0a63N3i5tjuknSUXzA1tLdc44x5zwJr5XhHCOsdXy/LwhcuQ/njtvK+8CjrukJEE8CF1ZPBq+hs132ttEwHwkfO+EdvBLH1p4nqBF4sLPOVK64IpSG/LOahqU8C908yeexO2H11e37pgy+Kr5zay+5VgbV8FHX+6VwL4cJpxdbO275YPZ6l9b/hG9GGBb4tLPhp4ufY7wVeq5xyiJYaqk5wv1gcVaymTJEGZSC9zq58laQ/nsKSetfPAUr+tSgAfRkd2va1AmGFr0lEWXUXBd0bwUvEd4Kt1txqX3DsFrZXm/rRjMOnn14S5Xqyccr/efb9c6lm/CEsP3vrLQS7COpVmdL0j0s1S+6h0ptSmF3xboOr5rxvpekd/qdK+B53hi7XZOwVtSTVbvgxYwEw8fEF69q8WctLTizSPJ9Rca+/LOhdN95OaWGd2vS3g5dr3dr3WfkbCt3WB2IzgpdoEud66K9HtCT/rMq1bjkhBv0RiW+FscdzC6udt+Tb9TG0T4tyvZ95XA99R4H3uL3VSWf5qjzTXq70HbZue8KXU+sXK0ucR0s1c253ga3GP2vrW/jz3tI2L6k+Rfq6lcb+adLJFLY8njLoHud/UpPJ+SM/sejX9zDrf6+mnh6PV9sn1q2lridkLvADd4atJM2vipdR0HYfVczEWp2rtT8wKVCB9mgMm0tLI4qtbd3q78Mrifq0PVKCcby/wPvefmlAj4Hsm15vw1fcrtbPGjPp7CXS9dXceF9ySgpbSxFoAasDpTWfXZeh4G/erWP3MLb56ijGcSuWZ97WknXXnTbc669Rk8sD3zK5Xat97sRXXV/SeXE+/1ri9HS/XfoDrrbtscb6aWCyeKueAisGU6lsDbK+TvrnW7/0FuH3wwuW17H6pWKneA1/P/mGvEsBT6UjwnRW8XDuuzR7wbXHJVkjv7Xi5tpbfJSjlXF974Outt7rdWl6Ycn1Z7uklkO5XWnx13Y3N/VoXXnHwtbreXg9lSABPoZlSziNc7yxzilJdLzdL9R2519cSv9fcveW9CXS99bUFpJo4rcO21GmcrhamFjhz9xDofuuVz/W+4K00qWcrfKPAu8bnPuDTa1b49nS9Uvs94Wv5++jx3877ZWGmdDMVPxi+UqymP+0YotNkrrE2lDzpZW3MozTuV3uq1U3fwlORtCueo+Gbc8CHV++081ng22N1dG/49lhIFeWS91hgxbXrAN6628gUdE8IS/+NeqSgpb6urunFV5L7xeRxv1r4ck9J8oA3OhWdAN5VM8B3dvBK7T0f9r3TpSNje8DXC16urfXLyyD4asqpWGkMrp5LQdf1VNo4MgVtdd8vH57g+9rLVy73a32EoHRIRit8W58L7FECeDclfMeDl6uz9jcDqK3wne2L0ADXW197X2vdcA1N7rVUV/cttbe00/bFud9KGvcrPfFIcyY0DnD9nG/L1idKt18Ocg74ZNL+VY1a5dyzfY+2vV2vNX4kfO8EvFjXo+Bbg9QyDlanAaPkbDXtNOMFul9MlscRtsBX63rzecCnl9X9RsK3t+ud7cPe2+c9wHcG8AJ0gW8LiLX1GkCvr6Vy6p6tDpXrx+x26zb0sZOt7pdzpdp9wRb4zvAsYIAE8AEU7Xxb+2n5J5Pwtd1Ly4EcVHttPwdyvXX3rS6Yq8dAKsVqyjEnTckLZqovSzvieEkAm/ul5nU1W4Z0q5Nj4OsFr/ZfdgJ4uCygPIvzbfmwbxk3ar6Xa7O386Vie3yZmdz11tfRLtjqbuv6+l64txODstaxcu2ofiRnXq183j7z1+N+r4ePm/e1zvfm4wjvSrPCd8a0cUvbURAZuThL22+C9+Za40StEObqLUDWALO+d6y9Fs5cO/H6ee4XwLbv9/KaBqwmfdwC39ZHEebTkA6vM8J3Rsc8m+u1xrfM+Ub+7t52ncGLDRHlerevW4G8LZOcM9a2h5P1tjPM/UrP+6X2/Nb1dVscuDjora434klIdZ95EtbpNTN87831RsUfAb4Tgbcua4FvC5AxWHJtJAcMVX19rYEs1sbV7hEkiPtd4fuyWgm9PXKyJfUcDd8I8OZBHIdUtPvtDd8ebXumuaPv5+zwPZnrra81r7fXWvh6HTD1WqqTwKxpY4Uz2ub6zOeI1HNv+Fpd70joXo+bmkizw3d0O6ntyPsZPT/cAt/oL0EHAW997X3d6oCxsggH3ORkkXHEsfCFVwCX1PPLzUKsp/IXr9jjIr0PYvDC1wve1nOgMwU9jaK2B2nV0/l62iV842I1GuH4uTYHhy/X/7YMA3PLeFp3zMVIYMYkjVVtO8IeK7iWY+c96/fzao6i7A/fkedAX+4hNYki3O9ZnK8XvFy/e6ape8RGOd8JXS82jBZeXGwP51v/pNpJ5dK1xrVyfXjc7wpfwf1KTzvCDtywH0WpHyMSvB7oWtokgLtK637vCb735Hqt8Vhsr7RzFKwHgle6toK4Bcga+FLu0upePVBtgfVVDL7war1+ek3s+W2Z942Gb9QZ0JFOOAG8u44I35nAK/V9lG1J0fDt/aVjMHjrsijXS732QJgro0BN1Wmu63G4NhZYr+6XOe/5xdYBC6uePfO+rfCNOwmr3wIsgARwR42a+z06fEeDl2vXG1qtsa3wjfoCcWDXu329lwP21PV0wFf98Ht+rauet9D0wtd2KIf+MA5L2XW9fDxlLsI6hCLcr7ftCAfoHadXn7O5Xio+Gr4nBG99PaMDll5b6rbX0bC++R0u7pc7bnJ74AYH2bX8qe0g+FrBu8eDGAASwJNrpoMyPP9UEr72+ISv+ZpzxNr4CAdsuTfp95F+JyuIqTGv2mwWXj2K2vMLgM/7AvBbjq6HHw/fqLOg63FqpQPeVb336krtZ04731PK2Ro/I3w7gxcr04KKi/W4XazMCmGqHeeGLe7YDFVFPy8BLHt+63nfp7hq0dWl62tXbHW+ONjtc715FvTdaMTc75lAJNVxfUptR71PR4HvSV0vV6d1qBEOmCvD+rPUuaAK16JgrThu8vkaXxRFLZaithtty6LhawFvyznQEQu0EsC7qMX93hN8vb+rt21vp2+J9UL1AK63F4j3dsB1mfTaUre99sKaBfbtcZPcWc9ah7uWX35yW5H4uWEshhqnfo1f51GUJ1SE++3hoD1/zb3hO5Pr7Q1eKt7y3tTtD+B6sSF6uFyubk8HrC2n6iwOV9OmHgcAsAM3uLOeuXlfDJ4R8G3bB0yvkK77ofqw1gPkHPDEannLjwQXLn6WRVbediPhq23bCt87c73b1xEO2ApiCpQRoFW3ked9uf2+1IpnDKyt8PU8H/j5euxJWBYlgIdKervP4uyoeO/v19J2RIqdatP63kTDdwfwYmU9XHAUcLVlGuDW9VjMaBd8c41vOeL2+0rpZaocg69mvtd2AAfteFsO5MD645UOeLBGHbwRMe4Mzk4zbmvb6Dlxzzia2JZ/OwnfrvD11FkcsFRX9yO1V7e5Pu0K4Bq06zW23xcAX/EslWM/a/kXZPUBb889wAAJ4IHqBRlrmyPBd5Z2o7MCJ3C+kddWKHuA3OKApf5ncMBXZbenXT2nmh/QRVfPQ8grnqlyW2ral3JuOYyjbl/LshVJ+78rATyFjgQMa9+R6fHR7Xq/973hS93LwEcGSjFncMCUg9W8puq8oKVinsqutxxh8F0lzftqVkK3wNfrem0HcfTZhpSLsIaqZUVv9JgJ35h2vTMIkfDdwfVaQWu9bgFxtAOWYinIamK0YMbirTEIfFdJh21wK54t8NUutuK3IdnAq4EuBdxchHV6zXYgBBXfCzBSPzO1S/ii3fe8jn7tdcCan1SZBGWsfQucWWeMH7ahge86v2uFLwdXTcq5F3itW5BsME4HPEi93G/vvxorNLSxkYd0eAHqbdvbKbd+sZHg2xm82BBRcPXWeRxwDwhHu2APnFlAX5/zvD1so4bvqtHw5ff/0qCmYuv4uk0dqynX1ucc8CG05wrm1r4t/Y6E78h2rfC1jKdxvrUmhi/Xd0/4Yv1GOGBuDO6epN+bi1e34Q/bAADypCuAa1heutTvAb68bn8kIRZ7W0873lmfCZwAnlJRfy0RQNXGzuB8Z3S9VLm2D+37X8ftuNCqhwuOeK0BLVfXwwFTdRKo1a7YftgGt7KZ2+srzfl64RvxVKQ6DruWyuv+OeUirCHqkQb1jGf5a9wbvrOA19uuF3wj44LgK4HWeu2NjXTAvSC8vqbgzMFaE1vXm4BNH7YhwZdatWxNO9tWQceBV14FTUG47XGEOQd8WFk/4C3xCV+5XZRTbn1fNHFYzCD4WuHcCtvtdZTrxcq8UKbiuPvzulx1m+fDNqSTrrC9vq3wxQDrfRyhNB9MxWqusb7qPjXa9ptzwN012v1aZF2BrInV3nNvsO3RrpfrtcRK8D3IKucernf7ugeE19eacovT5eq8cDbAd7viOdr5ahdbtYI38jGEe2xFSgBPpQh4RS/2oeIitipZ47k2PdqNdr1Y7I4p50iXy8Va46LqPU5Y44Kp19iYFhijoMXK8GMmsWf7RsK35WlImm1Iaz/bmNt63aKsug+qjbauVs4Bd9XILTAjF1KNgu9IiHLtItuMeu93gm/LtQfE3tfWsigYe143u1ysnX67kXTEpLTVyLLYCk9H6xZjcTHc6227bdvrehuE6z5pJYBPLMtf255/xQlfXWzCtxt8sbG5f0peByy95hwxV68tewmg3W6E7fWtYUo5zFtQ43VyOlq7Ejr2TOg6jiur+9FoHSvngHfTCPfbIzba/UakqLk2Pdp52hwYvhykPNd7gngv57v+1Dhdiwvm6g3MUpMAACAASURBVNG42+1G0gMWJPhiKWQKol74jjgPuo7Dr2MewmBVAtis3guoJLW637PDd5RTHrUqHYsZAF9rfStsqbojQFh6TdVpXLEKyPIZz9ijBTXw9aSdW07Fouq3P29f38Kai9+22SpyEVbYHHAp5YsA4EcA4B2P8f/Tsiz/hfmO7kIzud/Z4XvPrlcbOwi+kde9nS71OhrC2nItiDWwxa7VMfTTjVrgSzlYySljbdb6y69gT0djsdt4Kvb52r8Iq9dKaI2degsAPrgsy2dKKa8DwI+WUv7Wsiz/d5c7mlozul/tPZ0ZvmdzvVRcI3wlV2u9jnS61Os9IOz5aYWvxuVKMFY8WpCC76oe8OVWOUedBd3rHGh+FbR+PjjMAS/LsgDAZx4vX3/8o+v9rhQJjh77eL0aBd9oiEaPtSd8B7jeumwkiC2Qldp5wdwLvsDUUfVYTA3sRvjWoJXdMA1Z76Eccp2clqbint9GzhFjAI6ZCw7dhlRKeQEAHwOA3wIA37Usy48jMR8GgA9frr5MeZupNo10pj37PxN8oxdlnTDlvL2m/jqOAt9tn1I5d611vVtVe30BAOpTrgBk+D7FbWC6jXu+boevfvWzdTGW9yhK/Vww1UeLVABeluUVAHxVKeVdAPA/l1K+clmWn6li3gSANwEASnn3CR3ynqcrRTpaqj/LwRJSTAR8jwZeKr4ldjL4RoO41e1i9Xs5YK0jphytKeb6oI16ry92vjPlcuWyW+fLwVWa65UXYN3GrOXXP+d+HnCXgziWZfmVUsoPA8CHAOBnhPATadTc76iFV5oY7xeBFofMxUvtotvsfQBK8Hzv0UDc4oAtdVIbKX5H+FIPV6AWXGlgqpnvleaHL7eqB69tLljniLexdTweG7P4ah2zwBdU8eInainlywHg7Uf4fjEAfD0A/Demuzq1ot2YJjYy9eyFb48tSVT8yDYHX2iFddsLtlxs9OuIsh4/LfDVwJZtcw1fas53PeGqhiq2pcgz36txyWvdtvw2/rYe63OVdv7Xsh2JLuMXXElp6siDOH49APzFx3ng1wDgry3L8gPK/k8gr/u1ttsr9Rx1H63w3dv1Wvu6M9dbX2tee9pY4YrV94axFcQt8H0qv4XverZzDV89aHGYavcDa+Z5ow7iwCAqwZlq93zdby9w5Crovw8AX20a/TSSIOoBpOXDHFPkgQ5Ri65aoLM3eLk6S18tsQdyvVGA1by2Aper88LYGtMCW6yMWO3MPdXIstKZc728E8ahvMbj5Zo54Fsob1/bUtD4Iq26PdaWKsOEgTwfxrCrIuaMW/5qerr2yIVZnM4IXyzuQPDlxu3xmqvn4jwxLW3X19L7iMVLZQA38N0Kc74ANvg+t7mO3ZZR/W7L6njt/PDabq27Ltc74tvXEXPAur2/df8J4GYd1f16nWdU33s4X+7voudcLxW/03zvWVxwRJnWAUswtThf6rXVCT+V6+d8JedLOVxubrgue+7bcoa0DF4rdCPngK3bkKypaEkJYFQt8N174ZVGHvhiioZv5Pt64hXOWLezwnZ73RPCHhh7fmpATIFVE8PAl5vzlVPM+GIry0Ir7dzwtp9tHVa+7evyuu04Si1sex5D+QJexa2CTlkUCYmo8TWQlNpg7TyA5sa3vke9wUvFt8Z2hu8eYI54HQnhKBh7QKyFLdamEb60y+WBKjlkrLwuq/u9/Ho+8GqgG3Uwh6UM65tSPo7QrR6pZ4u8zsrrYj1zuh5AjziIY5Z0MxY74Nm9vcBsHYOK8QA5CsJUmygwd4Yvt+BKcqhSGplOQ9PpaWnrUn0P2z7WOiy+/sm5XCtwpXlfCqyWIyhXpQN2aWTqudc+Wk4R874e+FKKAKDU5g5cb30dFeuBsgeylnoPhFt+auta4SusduZOt2p1s1pIY22x2G3Z5de1zAnbFmVhMfxrec6Xd786GKcDnlotqWcvpD2p517umIrjxrTGn2SRFdZ1D9hysVEgbgEuVtYLwtY6C2yxstX1ApDw5U63srhZ7aIsy57h27Fo8GJAtUJ35Epo/yrodMBGHWXhlbeviL49/1wSvk2SoKmN7V3XCl9sjL0csKWuLudgXJdtnS8ACd9VUtp5Kw6qT/0xK6Iv9T74tmxPWsuuf/pWQ2/bcPF1m/p9lISBOx2wSdGLoaxjaf8aNEDQxESknkdsSZoJvFR8J/ha3Gp9HRXb87W2vgW0UozXDXMuWOuEt/Bl0s5W5yutdLY4ZMu88GXM2O1Ja911OT3/q4GtdCBHHa8px5T7gNXyrgqW2re+tVFfCnqlnq19pOtVqwdALbF7OOBoCK+vPS7YA2QOtliZEr7cuc6eeVzLPmDPyul63Ou28opnaVFW3X4VB+Y6to7Druv2t3U8jBPAYfLA1xLf8oHvWfwUsZ3Iu+hKuheuP+v2ol7gpWI7wHcEiKMAq3kdAWEPjHtBd/uauq7LjPCV4EhBFUADbQq0Nkg/t5PBq0lP4z/1Z0NrD+LAAIuB1bcKOgGsUK8tR1aAeORZIBWxqKpXahqLk+JPssIZ6zrCvVrqtH1Gwnc0hDEoSrEcdKnXXJkRvtK2IgmqFjdribv8apq0NJeGjlmUhcXcvn5Ay7dtqXpt3VYJYFGt8N3zxCtPTK95X0k94Huifb1Y1xHQ9NZp2mjKI4HL1UltPIBuAXEwfD3p5MjUtG5fMe/A6/aXt+YW9GubtX6tu/5pe0rStp47kMOaggYAePmKhnECuKsiUs97K+Kv3gpw671Y3reR8A2UBFht7BHhq+kfq6PeI+09RsEXqvJG+FLywBfvIzo1LT+iUOeEr/tay65/6t0wBl0JuOj+YAawLx6EOeAlAcxoZOrZEjvS/Upjt7rjFud7hynnumwkbLVx3tdWSEc4YC90ubrOzhfAttrZ4ny51LFvNTTvrJ+v9eDFoFq7Vg641lXQNWAxqL540O3nfVHxu+ia3SOAR6eeo7cdSTERq56j4Gq5B6mvXk5254VW0rU3djSIPfVRdRbY1tcW6G7LpL6N8PWudo6MsY5/iZcd83WcvCDr8jbiQN5em1ZBb2C7BS0G2Bqmz33g5ZiKzgDfG4BbU8E9U88el+rptxWe0j8ZDeQi4HtC11tfRzvdPVxvSxkVI7XRwJark8q41zWgAVTw9a52HhEDQK+OvtTZU9PPfV7Hb8vX+LWsjtn+3MZqYFtDtoZr4aZ/NYdjJYBr9drvK7XTxHpdcoQTbe3TWm9xoXfqeqXrVsB6+tvbAff4GQ3iGr7MgxU0q53fAZ9nIPcK3gFv3bR5Ca/gDfj8pjwC0PqFXJe3g4f0tv02vi6XoFsDV4KtCFkMrJzr5UCcKeitWuHraaeFYetpUZa+JbiOrsdiuHLLe90aO2iVs3TdE8RU3BkhbK3zvH76077aeYUvBrU34K2bNu+At1DobfvZguwN+DwB1pgFWnw5nX7moKsB7tbZrsC9Am0NzRqwGFR1x0Ff95UOeFUEfKNWMXvf7oiFV0eBb6859hO5Xq4uEsqt8PVCuBXK3pgWEG/hq3iwAvcsXws0+6Wl7XuHKfDWZXV/6/Xl7SRccgXdGrhbd4sC9xVSRl1TrlcLYoAEsF4tb0EPKGju5wx/ba1bjA6ecra2p+q0wG1p73G72HhaByy11wBbM04kfG/GtW01Arhd9ftc/gylOv42rl453DflrJkTrsu2/UWB9wm627eEgi4VU9dpyqnYBDBAjHPde89vxIlXs7nf1rRzK3w7ghfrfhbna+2Lio92w1EOOLJOA9yb1/KcL0D7VqOtEwaAx+t459vijp/LcRi/gFdQu+G1TgNd1uVqYMulpSnYGlZBJ4C7p54tb11P9+vZ82vpr1YP+J5kvpdzYJprD2y1cdGvR0O45ae2TgPlBvh6tho9L6i6hW9k2vmNxwVd2oVeXL/Y77iWYeDdul0VdCngSrDlQIvBtcUN3/cirL3g29v9ev66rIDl2h8FvpY+JzhYQ7rW9DMSypHA5epaYFtfc1DlylSv9audL0308N22aYHvukgLAFu01edgj3WsLXhv3DHidl9c3koAMECXc78aF4xd1/G1OBDfrwOeDb5eQMyYeo7smyprBWqmnE39asq1cMXqLaDVxGjdrKZOUy/G4/ClpHWgeJpZB8p3bLYhbVdMx6WptWlpHXhX6AKA7HQtwI2aA5YcL1Z/nw54phXPmrEscaP/qjxfCCglfNHrvUHsfR1Rpo3hHK32Z4jLrV4LW40AgF3xXF9jaVkrBOutSNpU9xvVfmNsFTY1z7y2qfslfycEvKjbfQWyy7W4YA60rU4Ya3t/Djg6/YvJ8sFPKerEq97ul1OLc6bGOhh8sa6tMPbU7f26B3yl9tg1VhfhfKXXV9cbelSywHe7FWcLtrVs+1rqY9vGts2oTidf168x8sEb1673CtKPc7zb+d0bx6txu9s6qn5bvi3jXtdtqBhN/X0BWAuPFvdrfat6ut/Rc82RcI6Grzb139H1YmVaN2uJnRm4WP1eDri7C35OOwOAaa+vZ7UzAGxcqexkrc43avU0l24mU81bWFIgXn+2zAPXryPngrF29wPgEfCNahflfqU2rQ7V2/cM8O3oerHuI697g5jqs6cD9jjhlp/R8L26vk47A/SDrxWEEkytMF+vb+emr+eUAa5XN9fpZtLx1uClQMxBl1vpzDlkqh6Ieq6M0vkB3HqQg6Uvqn2P/ahSjPVeW+Ac6YwTvs2A3V5HwFTz+ggQ1tZZQczA97Wred54+N4unrpts24R0sI3ymlz6WZunpcEL5Z2plywdh4Yq6Pq69d13FZaCJ97EdYM8LXEe6GiGTtycZTld7XAOfr333m+FxvKC1sudtTrnkDuBWFrHVWmKScWXAFAJ+dbQw2HLwdvALhZTFXHS/Xciu31HgFuF1mR4K1BW19TLpgq06aksbq6vq7DrqXyrc7rgCPh29J+xKIvacw9U8+WcT3xB4WvF7ZcXTRkqfII4GJlURD2xjS9Hg9faTWzBF/pUA9pDzBfL6ebb1Y2b1c1U+ClFl9h0PXMAXtPwfI64fM54NYTnqx9euCrcXXauOi/mhY4WwDb+iWBupeDp5zr6x4g7gltC0S5Om2bKDC3whd5sAKADF8A+5OFtoCLhK8uRa378nC5h+pJTZLrrV8DU+ZJR9flgMRzr7kFWVQZ1narczngHm5zDwdLacS2I04tgLV8UYn853YS+HLjWl9H9ucpo+q4WM0YUr91mQbSYvktfFdt9/pSeoF8Mr/clHHAe46Rtithe3Jv66i+rNfXzvd2rvcdbwnzvBhUI+eBe5yG5XXA5wHwaOcr9bHHwqtWRcKZkzWjEJkdOCh8R7jWiD68btgb63XA2r6MzhfgdsHVU5lrr6+8+tgKSGpOt80Z61POrOv1LMDSzAN7FmVR9XUMdr0V9x3s+AD2ONSIXycKvlrN7H6lfl8ydVw/WLxmPKqvSeE7GsSRsdp6D4QjfnrKuNdP1zr4tj7ZyFtPPRO4Fb7U/DB2L+946/O3e3o/B/SWIu08cN1mew1CmWbhlXcFtOR2sfrjzgF7U8PaX2VU6nkv99sC5yhnLPWjcbSTL7aqr71gjgJxbzjPBGELaKV6Ab7UnC9A22MF/fUyfFeY1vCl5o8pqK/zvU99eV2vZgGWNA/csgraOvdrnQfG+jueA24BYxR8e7pfLWSkmMh5Va4tB26uLmJl9uTwtcCWa9sC397AlepHQdgacwD4Pm/fsW0P4oFKp5WpPcUm17yB781c71ugn+eVwCuloS1zwBhsKdBq5n41EF51DABHuNEZ4Rvl4jUnSXH1Un+963rBd9KUs3RtBbGmfLTbxcp6Qdha1xm+q8bs9eXT0tHwJV3zq8fDPj73dpvrresAqQfiWjsHrElD168tC7LqeE5zAzgqDTwCvla1nmFskeW+W7Ydaeui0tva8RplhWlLf9q+qDisvJfz1ZRp/om0/pTqsPvUwvcpXr/auV5wRelF9clNAfq57rr+cpu30Nz2tb7exq5j0yujOcg/IOM+PD2n9/HtiYEvNefbejAH54Bb54GxGEoPMNMccK8511Hw7bHwStOf9X2LPHQjok4aX/OlBIsZlHbu5Xy5uh7O1VKPxba64b0csOq1f7UzAL8dKGLLjzxXK6eOsaMl6362J1ttnS8731svumoFb+14tfPA2lXQEozrGOyaKsO0rwMu0Hex0yjjHpF69oImoo2n7Qj3q/mycVD4jgBxD/h6QcvVzQ7fp/uPg699xbPlmk8tS/DF0s7YSmcSvutcrwTYtx7fVwnMFHip+V4OxBHzwFR8LQ2E505Be2W93R7zvlHxLX1JAJvJ/SZ8TXWj4DuzAx4FX+Z4yVVb+D6VGeBbp469cK6BW/eznRNeH9Jgge+TK6YWW9Xg/BxRrnHE2pXQEogx6HLApcBrnQeu22CaJwUdpVnga7mPXu7XMqZl7663Ttrzq6njYgLgi3XbC7aW2FHO1VPfWtcTvm4Q4081AriGb33E5KU5BVYavtyiKmq7EbYtSNpOxKWvw+C7dcAUYNfUNFR1WLmUmt5eg1C+lgFSbnXB2n3AnBM+lwOOcnSe/jTtZnK/o6V1yd4vIyeCLxWnKef6sbTxOlpr/CjHi5UZ4btNOz9f8/CtF2FJK54BtIuh8GstYLm08/Y1Bd83PveF6/neNZ1cwxfbeqRJN2vAS7ldbsUzBmmqvq6ryzGwco4Xiz+HA97j9nqu0Pb03XI/PeZ+o7IDmt8rcLvRVi3wtfTtebulsSQgauM0X0qwuh7OVxoHuz/LlxQEvqs4+D43vwbpU9vKDT+X4+74+le57u9FNQbW9nYO2l631l/Bf+N8ARDnC4DDMwK+1i1IGvC2zgOnAwbw3Vqr+/XARQtIz9GLkedgc2N5XbXX/Q7c69sK22i3a30d7WhbynrAd4QDRuC7zvu+ePn8Kbo9aAPgeksPVkalli9D49uCuNSzdtEVtopZV3e92nl7utWN86VWOnPOdxsPQKesAenDuyBLgm6UA+Zgi7nj4wLYe0s9U89Wacbx3IsFlD2cfIT7PQl8I+oSvreKhu9TvzR8pVOuAPBFVJdheqWeuUM7dHVu+GKQxVZAf+7xvcXiPe4YiDLOFXNlgJRvy7jXGFQ5CNcxx0tBt9xKBHyj3C8WG3HkpKQe7pfrs2XuF5j6hK+7/ojw5Rywpkx8vaDwfepKccQkAP1owFXauVztvC+3L3gL0eex6ZXSbvhSK525cit4gagHJgaQOAAaupj7xcBMxWwlrX42amcARww/E3y16u1+e8i77Yjro9M/v3uGbzS494Kv1+0q4as533kVBV/t4RtYH5cyDXCv221XQFNbmqg9wutxkyh8a4eLQRaLAei7IIuCMQbXiFXQyvnfRXLDczvgqGFng+8o92vZW9vb/XJxHJixdh1WPB8NvlIfUllL2zPC91GW852fyhCQrore76t1yZQrpsB8DePrBVcq+NaQpVLU27S0dzEWIPXbMoBbEFNlGhdMpKVruD7UUH7U2wyEvzDHHHCv7qPcX6SL1P6uEe6XUw9nzD3tKGrshO/VawucNW1bQcvVeX9SdRb4Alauf7iC5qCN63J+v+/lVm5XMtvT0rb0NQbim/lgbJ+vBF/NIizqhCwqdQ1MHFbPXQPyU5uShmvYbkGLwfWBAW4d/4V9HXDp13XToRQWWd2vN7YVmFFwtszbamRxv532+lrqLfHauj1ccJRrpupGQFgqq+uvyq+PmNxqu+L5cv3KvN1IuyWpTj1jsdsxsTrMQdf3WW83em537ZAB4Op4yavHCW4hCJvXlGOlnKvG+daulhoDmPjtNcDtfddl22t4hi4H3Bq2lNul3LHSAPd2wNGKhG/UftZoRT5SUEuKiCcVafsYBF8LkHs4YQmsmngtnK1uWBungaclXut2sX5VDljeblTP+wLotxvVwgCIbUnCYRmReqaOqqzPd36Ad7z1+auznUmXW28dqh3sKyIOK9fOCUtgBriFKwZYAroccLegvYFwBde3QdYaozTARwLwKPh62mHjefb99lTE7xwx9ztAUUC1xHohq+nH6mCtfXjgq/l9te01YObaXJXL8F1l2W50GYpOPWOuVZNSvrRp35pErXh+nvfdPFhhC1Jq7vatzU8AGb7c3G+dugakvQW8igVYNXRr4G5hewVmuBYGXSYTbdZBADwq7expr43XxFndrxaC3jrPgRyWcSeY942ItYLY0o+1zFvHxffsRypT1+u2GwHcPt1Igq98zjPubum0MQ5PyjFvX79RHSeJpZvred83Pvf29VONapBqXDAGX8rtYtDWOGIg6qAqA6QNXKArAZeC7fZ1DVjK+XIgPokDjtwLq+mTaz/b3K+lv4iFU545Yq7NBPD1wlgLXwm6s8BXWwdIXB1rccCa+5DaAAC24nmVZtEVwG06+bmchi8d87Dp8xaQmpQyB2ntwq3nRVeI86VAyK12luDLPaxhWwdAg5lbkAVVDFw7XQm6GHAfkLK6HKvfCos9AYCPAF+sjfZwDqnd2dzvVpPBtyWWksUhW+rrMk2dBPKWdhI8rf1KDvimXF7xvIpbdMWtgq4XXNVx122eHe+lXD6j2TPve/vEJATy9XYjKsXMAVMDX8xBAxLLwZZKNTPg3bpdCbo1cCnYeiFc6+CLsHpvpTmboue0I/qPXlXd0J3F+VrGsrpiC8hb4cn1KZVJfUs/sb6tcLfe60sckgC6M54v3V9D9KoPBpScG962rftaX2NjUwCn2nDzvmULQCR1S8IQc8sYTKEqr6+18KXuB56vKcdrBa8Gut7537fhsA44YjVudPvoQzewuFHHTnr7pOJ2cr+1IgGrdcY9X3tdsBd4mrrWn9a6ugx9jR+2IS262sqTeqaAWgPS4n7reWhN6plMSVepZxSqq2vd/sTiuMcQWh/WULtjLq76osA5Xg66rc7XM/8LcDgH3PtYxxHzvp7+tW16PiIRU+t4A92v5ToqttWBa/tsdbpR0NY6XW18XYfFU/eHwbdKPQMAmnrGjpC8hSQN2lo8YLUPUKAXb1GLq9gY7KQrzNFyZdRWJC98LQ9rgOtrDLyY29VAFwOrNv3MpZ6xugM44FFw65F6nsn9Riy+0n4JodyvVo3utwW+1r6pOo8rtsS2uERrDHZvmlisXANhz++Glt/Cd1XEs33xgzP4hyfUfdcLseoxqPljzZwwt98XPWyD+sPt2aXqNfDFjq/UwNcA3hq68HhNQdcD4d5bkACGA3iPox+9fVhSz57+tW2itmB53K8nfU0Be2f49nC7FqBa+vC6YatjtjhXyf1q+8FiOVjflOMnXWmf7buV7mQrft4XBy73IAbu4A5r6pnf73sFRmoRFVVXQ7FOWT8A/jCGliclAVzBVwNei/PlgHuybUgF+i568sDE0of13nsuZurRnydF3Op+d1QUfK3A1bpcbCypzANWbXttf5rfwxKL1aPtVsLw+30Bbh+ygEHtKfYGiLdzxlthgL3c6isUktt2FGTX/p7jArYcaRdY1S4XA3I9h1xDtu7b8KQkK3gxsGoh7FkFLa2A3sYfIAXtVQR8e4+tje259ajjPCw5bicwt7rd6DptWlbbVgtnL3SlGIuLjYKwywG37fel5nQpID8Pz7nhGqz1eDRM1/J6DGqh17af+ktC87yvpo5LKXNPUKqBDNf9UenmFbwAz8DlwKuFbl2+LavL6zoqptbBFmFpFXW7ke5XqyO4Rk/6mdL2PQ5c+Sz9ExgBY02cFrTe/rd1FuhqxpLaSIDF4rwO+On1der5tat53uuPyK0DvnTDO91a3HzwNgbrn5oX5l5fQ5sCM/7Epcvb8djPA/KQhfon5Xg1dVwf29fA9A3VawB0SxHneqm5Xyt4W5wvl34GOCWA93S+3PhRB29YdSfpZwtwRzjjltfavr0OVxMbkYKuy7l+re6eff2cegYAVep5KyqlzLna53bY3O6tw63jsXHrOO7Eq9vFWrz7JeFJpYIpmGJ11FYl7VhVOZZy1rjeGrbYa20qelvGldd1terYk6WgI29zVvd7pvRzkPttAaqlL22dNQ0tvda6QQ9srRCPSD17xqnL0Nf4fl8A32lXVEp5K2nVM+VKNc7YcuIVNw564IbWzVpSz9KCKmkRF5K65uDbCl4JuhYI13VUTK2TOGDr7R3F/bYq4mxnrp3nC8NgjXa7lnux9ql1qNbUsKYNN2Y0jLF40QHrnu9rPe0KEwVbPPahase7X6x8vZ/t/W7r+BXQxIEblJuVAFvP4WKglRZiOeD72c/xrpeCcA1R76KsbRlX3ksTA7gHfPdwv5qxWtyvd1zKvdbSxHVw3L3crhSrifM6ZEs9VacBcg+n622P9SHd0/p6+3zfR2lXPW+FzedqQXuJxVPU3JwuNl/r2XZ0fb/XK6rRhVcA/J5b7o8mhS3t7RXg+/bjIqzPfs7mejHHq90HrNkDPAK2mCYF8EzwbXW/vcEeQZetImHqTD9b3a3lLeDA4AWrJ56rj3CtUW3qtpoY7ZcFqv6q/PEjEjlwYyvPgRu18FQz/zQkzP3W/VErmuttR2sdte3o6h65s545KFKQtYAag2u9P7gac1md7oOcctakoKX0NBDXox2upAkBPOEtTaXeD144uCJSzxFjcbER7lc7ngeyLS65BcJX94LPonHuF0A+cONqCMTVWoRBddvv+horr1PRVIqactwvHvftPP36mPuF6vWrqryOrfvZxtVldR/1WEh9BHw1h3BY5n731kS0897KLO5XK2v6uaVvSpHOODj93NP9euqiXnPjaAEmxbS4YCm+pR9LClrpfrmFVwDynl9M2hOv+JiHq/625VvYUwdqrPGXt+M2VX15O57dLwDI7reOkeaL1z9bJ0sdS4k54nWv72bO97Nv2eFbQ1i7EItLSc+mCQDccgszLRbqlX723r+2XS9n3OGpIel29gAAFONJREFUR7V6wNgCUK0kF6gZL9LRemJb+9CmoAHActzkVprTrZ6Hu00rU5LcMeZSMZhu72lbh9/XA12+PfEKAyKAnI7GoKuJwR7UwKS41wVXEnw/+3jbGISlvcAa9zurdgTwBOwf7n5b5V2c5TkqU7I+HRTpdj3A5RThfq1lkWljT5ueKWjq9VPsA2iOm8ROvAKgockdrmHZdiTPBVPzt5TLvYau5H4BqrlfgGsIAshOdxujATMG4hq+G9e8wpdbcPVZwCG6hS9XD8zrI2gHCkYM2TPV6pHW/fZc/dzzi4F19fMO7jcitsX99gBxXWcBs7YPawraC2HJAT+V8cdNbkWdeIWtRqYXY92ukNaoXji1HZ8qx8Bc3+v6egtqyv2anS0QMRo3LI1FON+3Gee7ha/W9VLp5aOBd9UAQs3gdDHtPfe7l7a/2yz3BMd3v56xrA5Sau9xw9pxLW2pdtrUNAC0nHgFAGQZB1qL+73c8u2iqnqcOt28ff0Sheyrq7ZXC7pq90stgOKgu42VQKyFb/WH2ue7QhdAhi8FXA7CR1QnOpZ+XU/nfveSljKR6Wcqfqud3S8Hjkgwex2v95+lFqYtaWuq3nIPaqeLtbGdeHXdnTTPe+1YMbBaVC/Oql+v19c/6ScePbe5TVev5Vcrn7ffLziHS0G1XqSFtQEgIUv1WZ9wVTvfbbca+GpTz0fVa3vfgE1Rjs37mEJtfMTiMCk9bR3PqwHzwi3uN2pMTVzkfWjhpe1HC1HrPVHlmr8j75eRlzQMa/eLxiDu9zKcZjGVbu53jcf6xg/hoMH8PDY1n/zq9oELANcQBLgGJ2xioKp/qOrqfjCIA/ITAzrAzeMEsT28ALd19wZfgNPaxB6/Vu90bVT/Fnt4QPVIN1tTzJpyD5Couoi/NmtaWhuvSSdTcVdtrvf9ag/dsB45KaWVPdK42fqe6vQzdt83bnl94AIAvc1o+5pKFwNTL80XMylrasUz5Xa3aWguDoiyM+hAn8hRqedI96vtPxre0fd5oLlgS9vWOE0bayJDKusx5+tJS2tiqWuqP/L15shJ5dwvJmw7kSXNTLnf535wh7zWX34tvLwGM97/A9Judb6vbvf9wuYaSyXXIIUqhuurLpcWeQGQi64ouGrgu97CGeELYEhBl1JelFL+XinlB3reEK69AUGN3zqhJ/WvrY9sp5k/Dpz/bQFshNmPyqq3uN+odLQkq6sd5X4raVc+U4uqNAutLJJS19iRktfl13PEWDnWzxPwscVXAPSpV1gMVk+5X+oaNj+3b8mj++Xmfan5Xs1hHGeFL4BtDvgjAPDxXjdCywKRlk+rmZIBlnsZtSit0/yvZ0isbhSMLaC1ppw5tbpfj7O13AcXw76+feACwK37vap7Uad6uYVWOKS5eV2qreSQsXvh4CqV36TVKZiu1/UcMAZSyv1SKW0ulf1YVqeeuf25np9nhS+AEsCllPcCwO8FgO/uezu1op3vHvO4I9PP2k/3vTMKj2pJN1v6peqivq/1SkP3+G7TA+Ct7ldx5GT9uMFLF7j75eZ+OVFOV2qLwbTeosSlpbfbkOrxto8cvFp8haWUMZhSgKaASjloDOKPP+vUM8Ctc/XCd7sn+IzSOuA/DwDfDgBfoAJKKR8upXy0lPJRgM8E3JoVEr2cWHT6eTZNAuNaUQ7X4357mX2vI9bCcNQ/yRb32yjszGdTe2Qe13wPW0CK88o01LF6bBHXlarU781cbZ0qrl+vfTxU11vVfTA/l8fXD48/t7AF4vXZoWqRCOBSyjcCwKeXZfkYF7csy5vLsnxgWZYPALwz7Abj1HvxlVbW+d+RGjz/61VvGHvbtECn5d68c7jS2B4XbClrPPP5+brP4iv5Gb3btPV1Cnn7unbGz28F7pCpZ/4+qYYuwK27XX9iKWUsBa1xzOs4jz+3e34Bnt3v1sEC8rqOkVZAn1kaB/x1APD7SimfBIDvA4APllL+Ute7umv3GwXoyKcUdbBcPeZwveMbFwmFpZ97pKE9jlpzX9o+NGVY+hmuF18BAHvwBn5LtKtt3Wp02x8PXCzNfH0vt+X19XbvLwDgbhdzsxRMAW5Ty1g/2lT1GrKZ+926W2wRFoAewvcgEcDLsnzHsizvXZbl/QDwzQDwd5Zl+ZZ+t9TDEe7lfo+yd3hntcwFj5jv1bSXxmr9gmGdo40CuaYf1xww7X634o6d9EBV42yp/uutR5Sb3cZvf8pnRSOOmlv9DNVPzBnXLrcuoxZura9rMG9+YnO/9RA1bKX9vfcEX4DpTsI6MlBG3HuETYza/7vTHPio+d6Itj1A3KqWxVZUH1IdmUlA5jjhduvR7RB8SnpbZoG0tPqZbnftZjFI12CWti4BwPWDF1bV8Fx/Ui54206CLQX5bf3jn+22IwB8WxFUr6VDNgDuC74Axv/yy7L8MAD8cJc7cWt0+rlF1nsd+YVk4PxvD4fbMgbWxgtpT0qWqmt1ra3yThNwEL7p8/KJL6WfNYuvsO1I13U82LWqH0O4/Sk99xdzwNghHGv6+UlY+rku4+aBOUDXq6glMMOz492+rud8LRDetr8nTeSAj+J+I4Hf8vhBS78tGux0e8/39uizpf+W9LPUhzVNTfVnTTtLqo6dXMWlnwFu3aJnTpd75i96T0zK2zunzH0RuOp/m35ehQ2JwRnglmgYtLEYaqzHuu3K5+15z3VzDLzYrd9j6nnVJAD2AkPzv3+W1c/ROslToSwuloPe6PSzJ7anY49oZ/kyoK0zLr7SPPP3qp5ID3uecERBWeoLP8GKdsZ1ObcV6WXlPAEAX1RFlWPzwnU5l36uob0CdzP3yzleawr63jQBgI/ifDmNOIDjJBo5z+ltM1P6uefiKU8/XCxVZhwbO/nq6hqFlvTwBX4e13R/V4C9TTNv4+oYbBvTWne7/agamHO6NYjrcmx1NADuchkwb0+9WiUttMJWQN+z691qZwDPCqkZ5n8t7VsXYFkt0bY8cP9vj/neqFRxa59WEFOxrYAdtQiLbGtf/QzAp23rOM1Z0NRDFbi+6nlk6jCNerw6pj71qtZ2/rdQaWMszawpX0VtNZIWYcH1vt968ZQmBU3NFd+jdgRwK+T2SD/3nP+11ke3k9p3+FLSG7jasTXfM6IWjkXE76WIRVgA3dPPlzq726XmhLk5Ynwrkm4hGHV0JQDcHr4BQLtXbDEWVc7BHCurwPx01OTDdTgAn3bevqbmg+9ROwF4BHyPJsvvNGvm4ASKnFedLf2sTXRo2kmQxcoUi69mTz/X/T6XXaeit6+p+d/69drPi6slxkCvYoaGcixtLSzQoo6dpB6akCloWTsA+GzwmHn+d/vpN8E9RS9CsvQ5Kv3ckpKWYrWrlq39WfoJex+ZtK3i1CvcYcrpZ6vqOVv8yUr1lqTtNqVbMG/LsfYAcDv/C0ADEksXa8uxhVvMIqw6/Uy5XyrdXJfduwYDeCQE9krhplTqmdK19Lf3orCW8Ufee4dFWPXe363q5/5aJM3Jco8V3I55nVJud9DYnDJ3z6UmFhDXEeWUc96GIOnnbZMaqNS8MHdL96ZBudwD70sNUc8FWB5FLcDqpN7zv5r2Fkj3cL1e9XD6lhjxvcI/erHTr7bpZ2qrUaQsJ2fRz/W9hSp33vPNSVivXj3t/wUAfLvRes0truLmfzVzwNvXr/D08zaFTC2sytOueHX+SJjVTe69AEvq2/K+jVgYFnACVk9wjurT0n/0/K3Uv3UeN2IVNFWGzg/Lq5/r+V8A/iAMqmxb3mv+F0D3BQADc90WO5YSTQ7U87ZSOcBtGpoqr1dBI2Cun3r0VL5psl5j6WmAnPut1SkFXWBe+PaW5vfu9d7c2Xs+2nnu6YSj5389as4wyO635/yv72CO23ld6n6kgzuohzFsX98swNKsUpbKt3WYk66FgLlOP1OQpcx0rnzGNcFBHBZpPwHOPP97hHsU1GP+d4btR1JfVN2ssyrRq6A30s7/XrrQp58l0HLP/q1jtn3S/fFjYTHYAqyres0CLNhcc+XaxVxUynobtoEvNhy1wrkeKt3vsw4G4Nl0Ahi6FXgAx1ajU9Wtfc84/6tNL3vS0C73rjv7+apO8fCFaFlcMrW1iFowxp1j/eSc6wM4MHHlmjS0Zv63ilk219iDF7BrQOoTvLdKAJ9Wk21BotQbfD00q2PVqueqcucY2PyvJK8zbdF2xbRusZbuiwQKbm4B1iqtK9b2R/S7XflMpZupLrC9wamLTghgCTazLcA6kga/DxFwHr3lqGWcqAVYkbKmycV58gpIzPyv5ulH1KlTa/v66UdWaZ5+hD2akLrX+v5uYrYroAH4NLMEVGoBFtcfsg0J236EQXV7q9wWpNSzDgTgo0Jwti1Imr6Dx5zxry56LljrAvdcgOUZk6uzvCfE8ZPybcjQwuKsklZXSwuwuHvSrIC+qteuXN6Kmu+t64R53qfyR2Hbj1ZRJ1xB9TrTz7QOBODemjhNS8rzCWzdahT8vsy+5Wi0Sx6RDraUR2cdNE2ZhzLgQ/FbjzRttQuw1jJsLOyaArXmOMoX9f4eaZEVpofqj6U/wa7W8791KHf0ZP06dVECOFQeWLXsAY5oN5EiFv+MdNs9XW+rtAuv6nJNn6b70D39aKt6AZZV3vlf6wIs3xwwvgJ6XYB1swJ6uy93Kw60WPta2KrnepzH7Uea+V9u6pkrv3cdBMAz5jDvGJRRurcFWD2des/MQss2pACNWAGtFeeGr5/7K7tmqsy8AhqABu3azroAqz5sg4Aw52qpldGpZx0EwFr1WIA1k+4I3hF/VXsswPLKunWIuj6QIhZgkX3D7SP+XPco7Nm19qHuR7OlqBYH2rV++1MoX6prKp3MHTuZzpfXyQDcQwf+hEvJ6rUAK1p7p9Y1dWQb38ewbi5W7tt7JKVmBbS2PR9H/A6at80CWq7dA1EOzwuwuC6lc57TBeNKAKcc6nQIxyhFLP5ugfJe3+mk3ztySsBxApZuKP2pWBZx+3W5/rEtSLr+K1f88Op2CxIAf4SkBrSeum26eV0Fjfxa0vGS6YJlHQDA9+xAIz4RT562PlKauVbv+7UuxOL6sI6pDRcWZuldpN3dYmlqbdrae1+qLUiUWkBrXVHN9CdlsKlV0KlbHQDAI9QLUkchgmYLUsDv4nWFR3kbvSugo36/XlubIvpsdL5bcS5TAiMXoz2oQ3PsJAZX9UEgEY6W29+rXGlNPfVIuo2UXglgl07uKiN1FHhqdKbfRVLEFwflGdDba+0WJM1DEHoIByu/B5iKr19fqQXC3EIsqg/CIT880CdgbRdbcSdfpQumNTmAj3jCwlE06BSsaE1+e6T2OrVq1HhHngroLAnM24cwXEkDUOm7hmfrEjHfi71OtWlyAFu0tyvde3yNJrrH3innMwPhoL+PdgGWa9vOAFmdtfVhEehjCAFkyEpvj3deWanchuTXiQDcQzN/0s18bwfX3quUPWctR99DZ71wbk0aLcupV5zEuWlsBTSA7ulGVL22H0T1HmBD05RBCeBdlRDdVSHznBE3otTewB24B7iWdQ9w9CEcdMztKVgtIk/BApCdrlfEHuC3mXvJ/b4xSgBPpYlSxCN1dIj1HiMCkr2+SHR6PyP29gLQ242sfUSp9XAPVpIjZg7b4BzydjW05QzohLGsBPBQ9QJsgntoX0dKXPRaazfJQyfiQG2bd9ZvWXJkAFrmbLXDKeM8LjilVwI4lRqpUZA6wJcEz2MIe4g7BStSve7/Rq0rpJFyyx7gBLNeEwN41CfInbrH1H3K4/R3gnmvU7Ci2nJA7XoOtOWWB66cyj3Adk0M4JRPB7A+PXSnv3bKL+5QDU5SzDCnCzDF0uSErF8J4FQqQp3PTD60Bh1DaetnP3LtOfaNqIO45th6fXolgFNjNNkq29T9ioK117lKT0LqIgvDg3ifTjdeCeBUqpfOtr3qBNrjVC33mNy2odQplABOpVLxIh7EkLrVi9Z87w4Z7YmS6IdWAvgwytXaqVQqdSaVZYn/plpK+WcA8E/CO+6rXwcAv7z3TZxc+R6PUb7PY5Tv8xgd8X3+jcuyfLkU1AXAR1Qp5aPLsnxg7/s4s/I9HqN8n8co3+cxOvP7nCnoVCqVSqV2UAI4lUqlUqkdlAB+1pt738AdKN/jMcr3eYzyfR6j077POQecSqVSqdQOSgecSqVSqdQOSgCnUqlUKrWD7h7ApZQPlVL+USnlE6WUP7P3/ZxRpZTvLaV8upTyM3vfy5lVSnlfKeWHSikfL6X8bCnlI3vf0xlVSvmiUsrfLaX89OP7/F/tfU9nVSnlRSnl75VSfmDve+mhuwZwKeUFAHwXAPweAPjtAPAHSym/fd+7OqX+AgB8aO+buAM9AMC3LcvyrwDA1wLAf5z/nrvoLQD44LIsvwMAvgoAPlRK+dqd7+ms+ggAfHzvm+iluwYwAHwNAHxiWZafW5bl8wDwfQDwTTvf0+m0LMuPAMA/3/s+zq5lWX5pWZaffHz9a3D54HrPvnd1Pi0Xfebx8vXHP7maNVillPcCwO8FgO/e+1566d4B/B4A+IXN9acgP7BSJ1Ap5f0A8NUA8OP73sk59Zga/SkA+DQA/OCyLPk+x+vPA8C3A8AX9r6RXrp3ABekLL/Jpg6tUso7AeD7AeBbl2X51b3v54xaluXVsixfBQDvBYCvKaV85d73dCaVUr4RAD69LMvH9r6Xnrp3AH8KAN63uX4vAPziTveSSjWrlPI6XOD7l5dl+et738/ZtSzLrwDAD0OucYjW1wHA7yulfBIuU4MfLKX8pX1vKV73DuCfAICvKKX8plLKGwDwzQDwN3a+p1TKpVJKAYDvAYCPL8vynXvfz1lVSvnyUsq7Hl9/MQB8PQD8w33v6lxaluU7lmV577Is74fL5/LfWZblW3a+rXDdNYCXZXkAgD8FAH8bLgtW/tqyLD+7712dT6WUvwIAPwYAv62U8qlSyh/f+55Oqq8DgD8MF7fwU49/vmHvmzqhfj0A/FAp5e/D5Uv8Dy7LcsptMqm+yqMoU6lUKpXaQXftgFOpVCqV2ksJ4FQqlUqldlACOJVKpVKpHZQATqVSqVRqByWAU6lUKpXaQQngVCqVSqV2UAI4lUqlUqkd9P8DnGSSkMm/7/MAAAAASUVORK5CYII=",
+ "text/plain": [
+ "
\n",
+ "The region at the upper left corner is planar."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's instantiate `PeakFindingProblem` one last time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 75,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "problem = PeakFindingProblem(initial, grid, directions8)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Solution by Hill Climbing"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "solution = problem.value(hill_climbing(problem))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "solution"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Solution by Simulated Annealing"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "32"
+ ]
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "solutions = {problem.value(simulated_annealing(problem)) for i in range(100)}\n",
+ "max(solutions)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Notice that even though both algorithms started at the same initial state, \n",
+ "Hill Climbing could never escape from the planar region and gave a locally optimum solution of **0**,\n",
+ "whereas Simulated Annealing could reach the peak at **32**.\n",
+ "
\n",
+ "A very similar situation arises when there are two peaks of different heights.\n",
+ "One should carefully consider the possible search space before choosing the algorithm for the task."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## GENETIC ALGORITHM\n",
+ "\n",
+ "Genetic algorithms (or GA) are inspired by natural evolution and are particularly useful in optimization and search problems with large state spaces.\n",
+ "\n",
+ "Given a problem, algorithms in the domain make use of a *population* of solutions (also called *states*), where each solution/state represents a feasible solution. At each iteration (often called *generation*), the population gets updated using methods inspired by biology and evolution, like *crossover*, *mutation* and *natural selection*."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Overview\n",
+ "\n",
+ "A genetic algorithm works in the following way:\n",
+ "\n",
+ "1) Initialize random population.\n",
+ "\n",
+ "2) Calculate population fitness.\n",
+ "\n",
+ "3) Select individuals for mating.\n",
+ "\n",
+ "4) Mate selected individuals to produce new population.\n",
+ "\n",
+ " * Random chance to mutate individuals.\n",
+ "\n",
+ "5) Repeat from step 2) until an individual is fit enough or the maximum number of iterations is reached."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Glossary\n",
+ "\n",
+ "Before we continue, we will lay the basic terminology of the algorithm.\n",
+ "\n",
+ "* Individual/State: A list of elements (called *genes*) that represent possible solutions.\n",
+ "\n",
+ "* Population: The list of all the individuals/states.\n",
+ "\n",
+ "* Gene pool: The alphabet of possible values for an individual's genes.\n",
+ "\n",
+ "* Generation/Iteration: The number of times the population will be updated.\n",
+ "\n",
+ "* Fitness: An individual's score, calculated by a function specific to the problem."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Crossover\n",
+ "\n",
+ "Two individuals/states can \"mate\" and produce one child. This offspring bears characteristics from both of its parents. There are many ways we can implement this crossover. Here we will take a look at the most common ones. Most other methods are variations of those below.\n",
+ "\n",
+ "* Point Crossover: The crossover occurs around one (or more) point. The parents get \"split\" at the chosen point or points and then get merged. In the example below we see two parents get split and merged at the 3rd digit, producing the following offspring after the crossover.\n",
+ "\n",
+ "\n",
+ "\n",
+ "* Uniform Crossover: This type of crossover chooses randomly the genes to get merged. Here the genes 1, 2 and 5 were chosen from the first parent, so the genes 3, 4 were added by the second parent.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Mutation\n",
+ "\n",
+ "When an offspring is produced, there is a chance it will mutate, having one (or more, depending on the implementation) of its genes altered.\n",
+ "\n",
+ "For example, let's say the new individual to undergo mutation is \"abcde\". Randomly we pick to change its third gene to 'z'. The individual now becomes \"abzde\" and is added to the population."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Selection\n",
+ "\n",
+ "At each iteration, the fittest individuals are picked randomly to mate and produce offsprings. We measure an individual's fitness with a *fitness function*. That function depends on the given problem and it is used to score an individual. Usually the higher the better.\n",
+ "\n",
+ "The selection process is this:\n",
+ "\n",
+ "1) Individuals are scored by the fitness function.\n",
+ "\n",
+ "2) Individuals are picked randomly, according to their score (higher score means higher chance to get picked). Usually the formula to calculate the chance to pick an individual is the following (for population *P* and individual *i*):\n",
+ "\n",
+ "$$ chance(i) = \\dfrac{fitness(i)}{\\sum_{k \\, in \\, P}{fitness(k)}} $$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Implementation\n",
+ "\n",
+ "Below we look over the implementation of the algorithm in the `search` module.\n",
+ "\n",
+ "First the implementation of the main core of the algorithm:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 51,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def genetic_algorithm(population, fitness_fn, gene_pool=[0, 1], f_thres=None, ngen=1000, pmut=0.1):\n",
+ " """[Figure 4.8]"""\n",
+ " for i in range(ngen):\n",
+ " population = [mutate(recombine(*select(2, population, fitness_fn)), gene_pool, pmut)\n",
+ " for i in range(len(population))]\n",
+ "\n",
+ " fittest_individual = fitness_threshold(fitness_fn, f_thres, population)\n",
+ " if fittest_individual:\n",
+ " return fittest_individual\n",
+ "\n",
+ "\n",
+ " return argmax(population, key=fitness_fn)\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def recombine(x, y):\n",
+ " n = len(x)\n",
+ " c = random.randrange(0, n)\n",
+ " return x[:c] + y[c:]\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def mutate(x, gene_pool, pmut):\n",
+ " if random.uniform(0, 1) >= pmut:\n",
+ " return x\n",
+ "\n",
+ " n = len(x)\n",
+ " g = len(gene_pool)\n",
+ " c = random.randrange(0, n)\n",
+ " r = random.randrange(0, g)\n",
+ "\n",
+ " new_gene = gene_pool[r]\n",
+ " return x[:c] + [new_gene] + x[c+1:]\n",
+ "
\n",
+ "\n",
+ "\n"
+ ],
+ "text/plain": [
+ "def init_population(pop_number, gene_pool, state_length):\n",
+ " """Initializes population for genetic algorithm\n",
+ " pop_number : Number of individuals in population\n",
+ " gene_pool : List of possible values for individuals\n",
+ " state_length: The length of each individual"""\n",
+ " g = len(gene_pool)\n",
+ " population = []\n",
+ " for i in range(pop_number):\n",
+ " new_individual = [gene_pool[random.randrange(0, g)] for j in range(state_length)]\n",
+ " population.append(new_individual)\n",
+ "\n",
+ " return population\n",
+ "
\n",
+ "For this particular problem, two strings from the population will be chosen as parents and will be split at a random index and recombined as described in the `recombine` function to create a child. This child string will then be added to the new generation.\n",
+ "\n",
+ "\n",
+ "* **Variation**: There must be a variety of traits present in the population or a means with which to introduce variation.
If there is no variation in the sample space, we might never reach the global optimum. To ensure that there is enough variation, we can initialize a large population, but this gets computationally expensive as the population gets larger. Hence, we often use another method called mutation. In this method, we randomly change one or more characters of some strings in the population based on a predefined probability value called the mutation rate or mutation probability as described in the `mutate` function. The mutation rate is usually kept quite low. A mutation rate of zero fails to introduce variation in the population and a high mutation rate (say 50%) is as good as a coin flip and the population fails to benefit from the previous recombinations. An optimum balance has to be maintained between population size and mutation rate so as to reduce the computational cost as well as have sufficient variation in the population.\n",
+ "\n",
+ "\n",
+ "* **Selection**: There must be some mechanism by which some members of the population have the opportunity to be parents and pass down their genetic information and some do not. This is typically referred to as \"survival of the fittest\".
\n",
+ "There has to be some way of determining which phrases in our population have a better chance of eventually evolving into the target phrase. This is done by introducing a fitness function that calculates how close the generated phrase is to the target phrase. The function will simply return a scalar value corresponding to the number of matching characters between the generated phrase and the target phrase."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Before solving the problem, we first need to define our target phrase."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 55,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "target = 'Genetic Algorithm'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "We then need to define our gene pool, i.e the elements which an individual from the population might comprise of. Here, the gene pool contains all uppercase and lowercase letters of the English alphabet and the space character."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 56,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "# The ASCII values of uppercase characters ranges from 65 to 91\n",
+ "u_case = [chr(x) for x in range(65, 91)]\n",
+ "# The ASCII values of lowercase characters ranges from 97 to 123\n",
+ "l_case = [chr(x) for x in range(97, 123)]\n",
+ "\n",
+ "gene_pool = []\n",
+ "gene_pool.extend(u_case) # adds the uppercase list to the gene pool\n",
+ "gene_pool.extend(l_case) # adds the lowercase list to the gene pool\n",
+ "gene_pool.append(' ') # adds the space character to the gene pool"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We now need to define the maximum size of each population. Larger populations have more variation but are computationally more expensive to run algorithms on."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 57,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "max_population = 100"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As our population is not very large, we can afford to keep a relatively large mutation rate."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 58,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "mutation_rate = 0.07 # 7%"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Great! Now, we need to define the most important metric for the genetic algorithm, i.e the fitness function. This will simply return the number of matching characters between the generated sample and the target phrase."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 59,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def fitness_fn(sample):\n",
+ " # initialize fitness to 0\n",
+ " fitness = 0\n",
+ " for i in range(len(sample)):\n",
+ " # increment fitness by 1 for every matching character\n",
+ " if sample[i] == target[i]:\n",
+ " fitness += 1\n",
+ " return fitness"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Before we run our genetic algorithm, we need to initialize a random population. We will use the `init_population` function to do this. We need to pass in the maximum population size, the gene pool and the length of each individual, which in this case will be the same as the length of the target phrase."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 60,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "population = init_population(max_population, gene_pool, len(target))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We will now define how the individuals in the population should change as the number of generations increases. First, the `select` function will be run on the population to select *two* individuals with high fitness values. These will be the parents which will then be recombined using the `recombine` function to generate the child."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 61,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "parents = select(2, population, fitness_fn) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 62,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "# The recombine function takes two parents as arguments, so we need to unpack the previous variable\n",
+ "child = recombine(*parents)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we need to apply a mutation according to the mutation rate. We call the `mutate` function on the child with the gene pool and mutation rate as the additional arguments."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 63,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "child = mutate(child, gene_pool, mutation_rate)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The above lines can be condensed into\n",
+ "\n",
+ "`child = mutate(recombine(*select(2, population, fitness_fn)), gene_pool, mutation_rate)`\n",
+ "\n",
+ "And, we need to do this `for` every individual in the current population to generate the new population."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 64,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "population = [mutate(recombine(*select(2, population, fitness_fn)), gene_pool, mutation_rate) for i in range(len(population))]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The individual with the highest fitness can then be found using the `max` function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 65,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "current_best = max(population, key=fitness_fn)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's print this out"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 66,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "['J', 'y', 'O', 'e', ' ', 'h', 'c', 'r', 'C', 'W', 'H', 'o', 'r', 'R', 'y', 'P', 'U']\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(current_best)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We see that this is a list of characters. This can be converted to a string using the join function"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 67,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "JyOe hcrCWHorRyPU\n"
+ ]
+ }
+ ],
+ "source": [
+ "current_best_string = ''.join(current_best)\n",
+ "print(current_best_string)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We now need to define the conditions to terminate the algorithm. This can happen in two ways\n",
+ "1. Termination after a predefined number of generations\n",
+ "2. Termination when the fitness of the best individual of the current generation reaches a predefined threshold value.\n",
+ "\n",
+ "We define these variables below"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 68,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "ngen = 1200 # maximum number of generations\n",
+ "# we set the threshold fitness equal to the length of the target phrase\n",
+ "# i.e the algorithm only terminates whne it has got all the characters correct \n",
+ "# or it has completed 'ngen' number of generations\n",
+ "f_thres = len(target)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "To generate `ngen` number of generations, we run a `for` loop `ngen` number of times. After each generation, we calculate the fitness of the best individual of the generation and compare it to the value of `f_thres` using the `fitness_threshold` function. After every generation, we print out the best individual of the generation and the corresponding fitness value. Lets now write a function to do this."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 69,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def genetic_algorithm_stepwise(population, fitness_fn, gene_pool=[0, 1], f_thres=None, ngen=1200, pmut=0.1):\n",
+ " for generation in range(ngen):\n",
+ " population = [mutate(recombine(*select(2, population, fitness_fn)), gene_pool, pmut) for i in range(len(population))]\n",
+ " # stores the individual genome with the highest fitness in the current population\n",
+ " current_best = ''.join(max(population, key=fitness_fn))\n",
+ " print(f'Current best: {current_best}\\t\\tGeneration: {str(generation)}\\t\\tFitness: {fitness_fn(current_best)}\\r', end='')\n",
+ " \n",
+ " # compare the fitness of the current best individual to f_thres\n",
+ " fittest_individual = fitness_threshold(fitness_fn, f_thres, population)\n",
+ " \n",
+ " # if fitness is greater than or equal to f_thres, we terminate the algorithm\n",
+ " if fittest_individual:\n",
+ " return fittest_individual, generation\n",
+ " return max(population, key=fitness_fn) , generation "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The function defined above is essentially the same as the one defined in `search.py` with the added functionality of printing out the data of each generation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 70,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "