From a0cbf2818e5e4cb6127ac7f7e389e83199d7c34b Mon Sep 17 00:00:00 2001 From: Marianna Date: Mon, 23 Jul 2018 15:28:56 +0300 Subject: [PATCH 1/5] created notebooks for GraphPlan, Total Order Planner and Partial Order Planner --- planning_graphPlan.ipynb | 1066 ++++++++++++++++++++++++++ planning_partial_order_planner.ipynb | 850 ++++++++++++++++++++ planning_total_order_planner.ipynb | 341 ++++++++ 3 files changed, 2257 insertions(+) create mode 100644 planning_graphPlan.ipynb create mode 100644 planning_partial_order_planner.ipynb create mode 100644 planning_total_order_planner.ipynb diff --git a/planning_graphPlan.ipynb b/planning_graphPlan.ipynb new file mode 100644 index 000000000..bffecb937 --- /dev/null +++ b/planning_graphPlan.ipynb @@ -0,0 +1,1066 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## SOLVING PLANNING PROBLEMS\n", + "----\n", + "### GRAPHPLAN\n", + "
\n", + "The GraphPlan algorithm is a popular method of solving classical planning problems.\n", + "Before we get into the details of the algorithm, let's look at a special data structure called **planning graph**, used to give better heuristic estimates and plays a key role in the GraphPlan algorithm." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Planning Graph\n", + "A planning graph is a directed graph organized into levels. \n", + "Each level contains information about the current state of the knowledge base and the possible state-action links to and from that level.\n", + "The first level contains the initial state with nodes representing each fluent that holds in that level.\n", + "This level has state-action links linking each state to valid actions in that state.\n", + "Each action is linked to all its preconditions and its effect states.\n", + "Based on these effects, the next level is constructed.\n", + "The next level contains similarly structured information about the next state.\n", + "In this way, the graph is expanded using state-action links till we reach a state where all the required goals hold true simultaneously.\n", + "We can say that we have reached our goal if none of the goal states in the current level are mutually exclusive.\n", + "This will be explained in detail later.\n", + "
\n", + "Planning graphs only work for propositional planning problems, hence we need to eliminate all variables by generating all possible substitutions.\n", + "
\n", + "For example, the planning graph of the `have_cake_and_eat_cake_too` problem might look like this\n", + "![title](images/cake_graph.jpg)\n", + "
\n", + "The black lines indicate links between states and actions.\n", + "
\n", + "In every planning problem, we are allowed to carry out the `no-op` action, ie, we can choose no action for a particular state.\n", + "These are called 'Persistence' actions and are represented in the graph by the small square boxes.\n", + "In technical terms, a persistence action has effects same as its preconditions.\n", + "This enables us to carry a state to the next level.\n", + "
\n", + "
\n", + "The gray lines indicate mutual exclusivity.\n", + "This means that the actions connected bya gray line cannot be taken together.\n", + "Mutual exclusivity (mutex) occurs in the following cases:\n", + "1. **Inconsistent effects**: One action negates the effect of the other. For example, _Eat(Cake)_ and the persistence of _Have(Cake)_ have inconsistent effects because they disagree on the effect _Have(Cake)_\n", + "2. **Interference**: One of the effects of an action is the negation of a precondition of the other. For example, _Eat(Cake)_ interferes with the persistence of _Have(Cake)_ by negating its precondition.\n", + "3. **Competing needs**: One of the preconditions of one action is mutually exclusive with a precondition of the other. For example, _Bake(Cake)_ and _Eat(Cake)_ are mutex because they compete on the value of the _Have(Cake)_ precondition." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the module, planning graphs have been implemented using two classes, `Level` which stores data for a particular level and `Graph` which connects multiple levels together.\n", + "Let's look at the `Level` class." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
class Level:\n",
+       "    """\n",
+       "    Contains the state of the planning problem\n",
+       "    and exhaustive list of actions which use the\n",
+       "    states as pre-condition.\n",
+       "    """\n",
+       "\n",
+       "    def __init__(self, kb):\n",
+       "        """Initializes variables to hold state and action details of a level"""\n",
+       "\n",
+       "        self.kb = kb\n",
+       "        # current state\n",
+       "        self.current_state = kb.clauses\n",
+       "        # current action to state link\n",
+       "        self.current_action_links = {}\n",
+       "        # current state to action link\n",
+       "        self.current_state_links = {}\n",
+       "        # current action to next state link\n",
+       "        self.next_action_links = {}\n",
+       "        # next state to current action link\n",
+       "        self.next_state_links = {}\n",
+       "        # mutually exclusive actions\n",
+       "        self.mutex = []\n",
+       "\n",
+       "    def __call__(self, actions, objects):\n",
+       "        self.build(actions, objects)\n",
+       "        self.find_mutex()\n",
+       "\n",
+       "    def separate(self, e):\n",
+       "        """Separates an iterable of elements into positive and negative parts"""\n",
+       "\n",
+       "        positive = []\n",
+       "        negative = []\n",
+       "        for clause in e:\n",
+       "            if clause.op[:3] == 'Not':\n",
+       "                negative.append(clause)\n",
+       "            else:\n",
+       "                positive.append(clause)\n",
+       "        return positive, negative\n",
+       "\n",
+       "    def find_mutex(self):\n",
+       "        """Finds mutually exclusive actions"""\n",
+       "\n",
+       "        # Inconsistent effects\n",
+       "        pos_nsl, neg_nsl = self.separate(self.next_state_links)\n",
+       "\n",
+       "        for negeff in neg_nsl:\n",
+       "            new_negeff = Expr(negeff.op[3:], *negeff.args)\n",
+       "            for poseff in pos_nsl:\n",
+       "                if new_negeff == poseff:\n",
+       "                    for a in self.next_state_links[poseff]:\n",
+       "                        for b in self.next_state_links[negeff]:\n",
+       "                            if {a, b} not in self.mutex:\n",
+       "                                self.mutex.append({a, b})\n",
+       "\n",
+       "        # Interference will be calculated with the last step\n",
+       "        pos_csl, neg_csl = self.separate(self.current_state_links)\n",
+       "\n",
+       "        # Competing needs\n",
+       "        for posprecond in pos_csl:\n",
+       "            for negprecond in neg_csl:\n",
+       "                new_negprecond = Expr(negprecond.op[3:], *negprecond.args)\n",
+       "                if new_negprecond == posprecond:\n",
+       "                    for a in self.current_state_links[posprecond]:\n",
+       "                        for b in self.current_state_links[negprecond]:\n",
+       "                            if {a, b} not in self.mutex:\n",
+       "                                self.mutex.append({a, b})\n",
+       "\n",
+       "        # Inconsistent support\n",
+       "        state_mutex = []\n",
+       "        for pair in self.mutex:\n",
+       "            next_state_0 = self.next_action_links[list(pair)[0]]\n",
+       "            if len(pair) == 2:\n",
+       "                next_state_1 = self.next_action_links[list(pair)[1]]\n",
+       "            else:\n",
+       "                next_state_1 = self.next_action_links[list(pair)[0]]\n",
+       "            if (len(next_state_0) == 1) and (len(next_state_1) == 1):\n",
+       "                state_mutex.append({next_state_0[0], next_state_1[0]})\n",
+       "        \n",
+       "        self.mutex = self.mutex + state_mutex\n",
+       "\n",
+       "    def build(self, actions, objects):\n",
+       "        """Populates the lists and dictionaries containing the state action dependencies"""\n",
+       "\n",
+       "        for clause in self.current_state:\n",
+       "            p_expr = Expr('P' + clause.op, *clause.args)\n",
+       "            self.current_action_links[p_expr] = [clause]\n",
+       "            self.next_action_links[p_expr] = [clause]\n",
+       "            self.current_state_links[clause] = [p_expr]\n",
+       "            self.next_state_links[clause] = [p_expr]\n",
+       "\n",
+       "        for a in actions:\n",
+       "            num_args = len(a.args)\n",
+       "            possible_args = tuple(itertools.permutations(objects, num_args))\n",
+       "\n",
+       "            for arg in possible_args:\n",
+       "                if a.check_precond(self.kb, arg):\n",
+       "                    for num, symbol in enumerate(a.args):\n",
+       "                        if not symbol.op.islower():\n",
+       "                            arg = list(arg)\n",
+       "                            arg[num] = symbol\n",
+       "                            arg = tuple(arg)\n",
+       "\n",
+       "                    new_action = a.substitute(Expr(a.name, *a.args), arg)\n",
+       "                    self.current_action_links[new_action] = []\n",
+       "\n",
+       "                    for clause in a.precond:\n",
+       "                        new_clause = a.substitute(clause, arg)\n",
+       "                        self.current_action_links[new_action].append(new_clause)\n",
+       "                        if new_clause in self.current_state_links:\n",
+       "                            self.current_state_links[new_clause].append(new_action)\n",
+       "                        else:\n",
+       "                            self.current_state_links[new_clause] = [new_action]\n",
+       "                   \n",
+       "                    self.next_action_links[new_action] = []\n",
+       "                    for clause in a.effect:\n",
+       "                        new_clause = a.substitute(clause, arg)\n",
+       "\n",
+       "                        self.next_action_links[new_action].append(new_clause)\n",
+       "                        if new_clause in self.next_state_links:\n",
+       "                            self.next_state_links[new_clause].append(new_action)\n",
+       "                        else:\n",
+       "                            self.next_state_links[new_clause] = [new_action]\n",
+       "\n",
+       "    def perform_actions(self):\n",
+       "        """Performs the necessary actions and returns a new Level"""\n",
+       "\n",
+       "        new_kb = FolKB(list(set(self.next_state_links.keys())))\n",
+       "        return Level(new_kb)\n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(Level)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each level stores the following data\n", + "1. The current state of the level in `current_state`\n", + "2. Links from an action to its preconditions in `current_action_links`\n", + "3. Links from a state to the possible actions in that state in `current_state_links`\n", + "4. Links from each action to its effects in `next_action_links`\n", + "5. Links from each possible next state from each action in `next_state_links`. This stores the same information as the `current_action_links` of the next level.\n", + "6. Mutex links in `mutex`.\n", + "
\n", + "
\n", + "The `find_mutex` method finds the mutex links according to the points given above.\n", + "
\n", + "The `build` method populates the data structures storing the state and action information.\n", + "Persistence actions for each clause in the current state are also defined here. \n", + "The newly created persistence action has the same name as its state, prefixed with a 'P'." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now look at the `Graph` class." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
class Graph:\n",
+       "    """\n",
+       "    Contains levels of state and actions\n",
+       "    Used in graph planning algorithm to extract a solution\n",
+       "    """\n",
+       "\n",
+       "    def __init__(self, planningproblem):\n",
+       "        self.planningproblem = planningproblem\n",
+       "        self.kb = FolKB(planningproblem.init)\n",
+       "        self.levels = [Level(self.kb)]\n",
+       "        self.objects = set(arg for clause in self.kb.clauses for arg in clause.args)\n",
+       "\n",
+       "    def __call__(self):\n",
+       "        self.expand_graph()\n",
+       "\n",
+       "    def expand_graph(self):\n",
+       "        """Expands the graph by a level"""\n",
+       "\n",
+       "        last_level = self.levels[-1]\n",
+       "        last_level(self.planningproblem.actions, self.objects)\n",
+       "        self.levels.append(last_level.perform_actions())\n",
+       "\n",
+       "    def non_mutex_goals(self, goals, index):\n",
+       "        """Checks whether the goals are mutually exclusive"""\n",
+       "\n",
+       "        goal_perm = itertools.combinations(goals, 2)\n",
+       "        for g in goal_perm:\n",
+       "            if set(g) in self.levels[index].mutex:\n",
+       "                return False\n",
+       "        return True\n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(Graph)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The class stores a problem definition in `pddl`, \n", + "a knowledge base in `kb`, \n", + "a list of `Level` objects in `levels` and \n", + "all the possible arguments found in the initial state of the problem in `objects`.\n", + "
\n", + "The `expand_graph` method generates a new level of the graph.\n", + "This method is invoked when the goal conditions haven't been met in the current level or the actions that lead to it are mutually exclusive.\n", + "The `non_mutex_goals` method checks whether the goals in the current state are mutually exclusive.\n", + "
\n", + "
\n", + "Using these two classes, we can define a planning graph which can either be used to provide reliable heuristics for planning problems or used in the `GraphPlan` algorithm.\n", + "
\n", + "Let's have a look at the `GraphPlan` class." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
class GraphPlan:\n",
+       "    """\n",
+       "    Class for formulation GraphPlan algorithm\n",
+       "    Constructs a graph of state and action space\n",
+       "    Returns solution for the planning problem\n",
+       "    """\n",
+       "\n",
+       "    def __init__(self, planningproblem):\n",
+       "        self.graph = Graph(planningproblem)\n",
+       "        self.nogoods = []\n",
+       "        self.solution = []\n",
+       "\n",
+       "    def check_leveloff(self):\n",
+       "        """Checks if the graph has levelled off"""\n",
+       "\n",
+       "        check = (set(self.graph.levels[-1].current_state) == set(self.graph.levels[-2].current_state))\n",
+       "\n",
+       "        if check:\n",
+       "            return True\n",
+       "\n",
+       "    def extract_solution(self, goals, index):\n",
+       "        """Extracts the solution"""\n",
+       "\n",
+       "        level = self.graph.levels[index]    \n",
+       "        if not self.graph.non_mutex_goals(goals, index):\n",
+       "            self.nogoods.append((level, goals))\n",
+       "            return\n",
+       "\n",
+       "        level = self.graph.levels[index - 1]    \n",
+       "\n",
+       "        # Create all combinations of actions that satisfy the goal    \n",
+       "        actions = []\n",
+       "        for goal in goals:\n",
+       "            actions.append(level.next_state_links[goal])    \n",
+       "\n",
+       "        all_actions = list(itertools.product(*actions))    \n",
+       "\n",
+       "        # Filter out non-mutex actions\n",
+       "        non_mutex_actions = []    \n",
+       "        for action_tuple in all_actions:\n",
+       "            action_pairs = itertools.combinations(list(set(action_tuple)), 2)        \n",
+       "            non_mutex_actions.append(list(set(action_tuple)))        \n",
+       "            for pair in action_pairs:            \n",
+       "                if set(pair) in level.mutex:\n",
+       "                    non_mutex_actions.pop(-1)\n",
+       "                    break\n",
+       "    \n",
+       "\n",
+       "        # Recursion\n",
+       "        for action_list in non_mutex_actions:        \n",
+       "            if [action_list, index] not in self.solution:\n",
+       "                self.solution.append([action_list, index])\n",
+       "\n",
+       "                new_goals = []\n",
+       "                for act in set(action_list):                \n",
+       "                    if act in level.current_action_links:\n",
+       "                        new_goals = new_goals + level.current_action_links[act]\n",
+       "\n",
+       "                if abs(index) + 1 == len(self.graph.levels):\n",
+       "                    return\n",
+       "                elif (level, new_goals) in self.nogoods:\n",
+       "                    return\n",
+       "                else:\n",
+       "                    self.extract_solution(new_goals, index - 1)\n",
+       "\n",
+       "        # Level-Order multiple solutions\n",
+       "        solution = []\n",
+       "        for item in self.solution:\n",
+       "            if item[1] == -1:\n",
+       "                solution.append([])\n",
+       "                solution[-1].append(item[0])\n",
+       "            else:\n",
+       "                solution[-1].append(item[0])\n",
+       "\n",
+       "        for num, item in enumerate(solution):\n",
+       "            item.reverse()\n",
+       "            solution[num] = item\n",
+       "\n",
+       "        return solution\n",
+       "\n",
+       "    def goal_test(self, kb):\n",
+       "        return all(kb.ask(q) is not False for q in self.graph.planningproblem.goals)\n",
+       "\n",
+       "    def execute(self):\n",
+       "        """Executes the GraphPlan algorithm for the given problem"""\n",
+       "\n",
+       "        while True:\n",
+       "            self.graph.expand_graph()\n",
+       "            if (self.goal_test(self.graph.levels[-1].kb) and self.graph.non_mutex_goals(self.graph.planningproblem.goals, -1)):\n",
+       "                solution = self.extract_solution(self.graph.planningproblem.goals, -1)\n",
+       "                if solution:\n",
+       "                    return solution\n",
+       "            \n",
+       "            if len(self.graph.levels) >= 2 and self.check_leveloff():\n",
+       "                return None\n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(GraphPlan)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given a planning problem defined as a PlanningProblem, `GraphPlan` creates a planning graph stored in `graph` and expands it till it reaches a state where all its required goals are present simultaneously without mutual exclusivity.\n", + "
\n", + "Once a goal is found, `extract_solution` is called.\n", + "This method recursively finds the path to a solution given a planning graph.\n", + "In the case where `extract_solution` fails to find a solution for a set of goals as a given level, we record the `(level, goals)` pair as a **no-good**.\n", + "Whenever `extract_solution` is called again with the same level and goals, we can find the recorded no-good and immediately return failure rather than searching again. \n", + "No-goods are also used in the termination test.\n", + "
\n", + "The `check_leveloff` method checks if the planning graph for the problem has **levelled-off**, ie, it has the same states, actions and mutex pairs as the previous level.\n", + "If the graph has already levelled off and we haven't found a solution, there is no point expanding the graph, as it won't lead to anything new.\n", + "In such a case, we can declare that the planning problem is unsolvable with the given constraints.\n", + "
\n", + "
\n", + "To summarize, the `GraphPlan` algorithm calls `expand_graph` and tests whether it has reached the goal and if the goals are non-mutex.\n", + "
\n", + "If so, `extract_solution` is invoked which recursively reconstructs the solution from the planning graph.\n", + "
\n", + "If not, then we check if our graph has levelled off and continue if it hasn't." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's solve a few planning problems that we had defined earlier." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Air cargo problem\n", + "In accordance with the summary above, we have defined a helper function to carry out `GraphPlan` on the `air_cargo` problem.\n", + "The function is pretty straightforward.\n", + "Let's have a look." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
def air_cargo_graphplan():\n",
+       "    """Solves the air cargo problem using GraphPlan"""\n",
+       "    return GraphPlan(air_cargo()).execute()\n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(air_cargo_graphplan)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's instantiate the problem and find a solution using this helper function." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[[[Load(C2, P2, JFK),\n", + " PAirport(SFO),\n", + " PAirport(JFK),\n", + " PPlane(P2),\n", + " PPlane(P1),\n", + " Fly(P2, JFK, SFO),\n", + " PCargo(C2),\n", + " Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " PCargo(C1)],\n", + " [Unload(C2, P2, SFO), Unload(C1, P1, JFK)]]]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "airCargoG = air_cargo_graphplan()\n", + "airCargoG" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each element in the solution is a valid action.\n", + "The solution is separated into lists for each level.\n", + "The actions prefixed with a 'P' are persistence actions and can be ignored.\n", + "They simply carry certain states forward.\n", + "We have another helper function `linearize` that presents the solution in a more readable format, much like a total-order planner, but it is _not_ a total-order planner." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Unload(C2, P2, SFO),\n", + " Unload(C1, P1, JFK)]" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "linearize(airCargoG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Indeed, this is a correct solution.\n", + "
\n", + "There are similar helper functions for some other planning problems.\n", + "
\n", + "Lets' try solving the spare tire problem." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "spareTireG = spare_tire_graphplan()\n", + "linearize(spareTireG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution for the cake problem" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Eat(Cake), Bake(Cake)]" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cakeProblemG = have_cake_and_eat_cake_too_graphplan()\n", + "linearize(cakeProblemG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution for the Sussman's Anomaly configuration of three blocks." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sussmanAnomalyG = three_block_tower_graphplan()\n", + "linearize(sussmanAnomalyG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solution of the socks and shoes problem" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[RightSock, LeftSock, RightShoe, LeftShoe]" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "socksShoesG = socks_and_shoes_graphplan()\n", + "linearize(socksShoesG)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_partial_order_planner.ipynb b/planning_partial_order_planner.ipynb new file mode 100644 index 000000000..4b1a98bb3 --- /dev/null +++ b/planning_partial_order_planner.ipynb @@ -0,0 +1,850 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### PARTIAL ORDER PLANNER\n", + "A partial-order planning algorithm is significantly different from a total-order planner.\n", + "The way a partial-order plan works enables it to take advantage of _problem decomposition_ and work on each subproblem separately.\n", + "It works on several subgoals independently, solves them with several subplans, and then combines the plan.\n", + "
\n", + "A partial-order planner also follows the **least commitment** strategy, where it delays making choices for as long as possible.\n", + "Variables are not bound unless it is absolutely necessary and new actions are chosen only if the existing actions cannot fulfil the required precondition.\n", + "
\n", + "Any planning algorithm that can place two actions into a plan without specifying which comes first is called a **partial-order planner**.\n", + "A partial-order planner searches through the space of plans rather than the space of states, which makes it perform better for certain problems.\n", + "
\n", + "
\n", + "Let's have a look at the `PartialOrderPlanner` class." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
class PartialOrderPlanner:\n",
+       "\n",
+       "    def __init__(self, planningproblem):\n",
+       "        self.planningproblem = planningproblem\n",
+       "        self.initialize()\n",
+       "\n",
+       "    def initialize(self):\n",
+       "        """Initialize all variables"""\n",
+       "        self.causal_links = []\n",
+       "        self.start = Action('Start', [], self.planningproblem.init)\n",
+       "        self.finish = Action('Finish', self.planningproblem.goals, [])\n",
+       "        self.actions = set()\n",
+       "        self.actions.add(self.start)\n",
+       "        self.actions.add(self.finish)\n",
+       "        self.constraints = set()\n",
+       "        self.constraints.add((self.start, self.finish))\n",
+       "        self.agenda = set()\n",
+       "        for precond in self.finish.precond:\n",
+       "            self.agenda.add((precond, self.finish))\n",
+       "        self.expanded_actions = self.expand_actions()\n",
+       "\n",
+       "    def expand_actions(self, name=None):\n",
+       "        """Generate all possible actions with variable bindings for precondition selection heuristic"""\n",
+       "\n",
+       "        objects = set(arg for clause in self.planningproblem.init for arg in clause.args)\n",
+       "        expansions = []\n",
+       "        action_list = []\n",
+       "        if name is not None:\n",
+       "            for action in self.planningproblem.actions:\n",
+       "                if str(action.name) == name:\n",
+       "                    action_list.append(action)\n",
+       "        else:\n",
+       "            action_list = self.planningproblem.actions\n",
+       "\n",
+       "        for action in action_list:\n",
+       "            for permutation in itertools.permutations(objects, len(action.args)):\n",
+       "                bindings = unify(Expr(action.name, *action.args), Expr(action.name, *permutation))\n",
+       "                if bindings is not None:\n",
+       "                    new_args = []\n",
+       "                    for arg in action.args:\n",
+       "                        if arg in bindings:\n",
+       "                            new_args.append(bindings[arg])\n",
+       "                        else:\n",
+       "                            new_args.append(arg)\n",
+       "                    new_expr = Expr(str(action.name), *new_args)\n",
+       "                    new_preconds = []\n",
+       "                    for precond in action.precond:\n",
+       "                        new_precond_args = []\n",
+       "                        for arg in precond.args:\n",
+       "                            if arg in bindings:\n",
+       "                                new_precond_args.append(bindings[arg])\n",
+       "                            else:\n",
+       "                                new_precond_args.append(arg)\n",
+       "                        new_precond = Expr(str(precond.op), *new_precond_args)\n",
+       "                        new_preconds.append(new_precond)\n",
+       "                    new_effects = []\n",
+       "                    for effect in action.effect:\n",
+       "                        new_effect_args = []\n",
+       "                        for arg in effect.args:\n",
+       "                            if arg in bindings:\n",
+       "                                new_effect_args.append(bindings[arg])\n",
+       "                            else:\n",
+       "                                new_effect_args.append(arg)\n",
+       "                        new_effect = Expr(str(effect.op), *new_effect_args)\n",
+       "                        new_effects.append(new_effect)\n",
+       "                    expansions.append(Action(new_expr, new_preconds, new_effects))\n",
+       "\n",
+       "        return expansions\n",
+       "\n",
+       "    def find_open_precondition(self):\n",
+       "        """Find open precondition with the least number of possible actions"""\n",
+       "\n",
+       "        number_of_ways = dict()\n",
+       "        actions_for_precondition = dict()\n",
+       "        for element in self.agenda:\n",
+       "            open_precondition = element[0]\n",
+       "            possible_actions = list(self.actions) + self.expanded_actions\n",
+       "            for action in possible_actions:\n",
+       "                for effect in action.effect:\n",
+       "                    if effect == open_precondition:\n",
+       "                        if open_precondition in number_of_ways:\n",
+       "                            number_of_ways[open_precondition] += 1\n",
+       "                            actions_for_precondition[open_precondition].append(action)\n",
+       "                        else:\n",
+       "                            number_of_ways[open_precondition] = 1\n",
+       "                            actions_for_precondition[open_precondition] = [action]\n",
+       "\n",
+       "        number = sorted(number_of_ways, key=number_of_ways.__getitem__)\n",
+       "        \n",
+       "        for k, v in number_of_ways.items():\n",
+       "            if v == 0:\n",
+       "                return None, None, None\n",
+       "\n",
+       "        act1 = None\n",
+       "        for element in self.agenda:\n",
+       "            if element[0] == number[0]:\n",
+       "                act1 = element[1]\n",
+       "                break\n",
+       "\n",
+       "        if number[0] in self.expanded_actions:\n",
+       "            self.expanded_actions.remove(number[0])\n",
+       "\n",
+       "        return number[0], act1, actions_for_precondition[number[0]]\n",
+       "\n",
+       "    def find_action_for_precondition(self, oprec):\n",
+       "        """Find action for a given precondition"""\n",
+       "\n",
+       "        # either\n",
+       "        #   choose act0 E Actions such that act0 achieves G\n",
+       "        for action in self.actions:\n",
+       "            for effect in action.effect:\n",
+       "                if effect == oprec:\n",
+       "                    return action, 0\n",
+       "\n",
+       "        # or\n",
+       "        #   choose act0 E Actions such that act0 achieves G\n",
+       "        for action in self.planningproblem.actions:\n",
+       "            for effect in action.effect:\n",
+       "                if effect.op == oprec.op:\n",
+       "                    bindings = unify(effect, oprec)\n",
+       "                    if bindings is None:\n",
+       "                        break\n",
+       "                    return action, bindings\n",
+       "\n",
+       "    def generate_expr(self, clause, bindings):\n",
+       "        """Generate atomic expression from generic expression given variable bindings"""\n",
+       "\n",
+       "        new_args = []\n",
+       "        for arg in clause.args:\n",
+       "            if arg in bindings:\n",
+       "                new_args.append(bindings[arg])\n",
+       "            else:\n",
+       "                new_args.append(arg)\n",
+       "\n",
+       "        try:\n",
+       "            return Expr(str(clause.name), *new_args)\n",
+       "        except:\n",
+       "            return Expr(str(clause.op), *new_args)\n",
+       "        \n",
+       "    def generate_action_object(self, action, bindings):\n",
+       "        """Generate action object given a generic action andvariable bindings"""\n",
+       "\n",
+       "        # if bindings is 0, it means the action already exists in self.actions\n",
+       "        if bindings == 0:\n",
+       "            return action\n",
+       "\n",
+       "        # bindings cannot be None\n",
+       "        else:\n",
+       "            new_expr = self.generate_expr(action, bindings)\n",
+       "            new_preconds = []\n",
+       "            for precond in action.precond:\n",
+       "                new_precond = self.generate_expr(precond, bindings)\n",
+       "                new_preconds.append(new_precond)\n",
+       "            new_effects = []\n",
+       "            for effect in action.effect:\n",
+       "                new_effect = self.generate_expr(effect, bindings)\n",
+       "                new_effects.append(new_effect)\n",
+       "            return Action(new_expr, new_preconds, new_effects)\n",
+       "\n",
+       "    def cyclic(self, graph):\n",
+       "        """Check cyclicity of a directed graph"""\n",
+       "\n",
+       "        new_graph = dict()\n",
+       "        for element in graph:\n",
+       "            if element[0] in new_graph:\n",
+       "                new_graph[element[0]].append(element[1])\n",
+       "            else:\n",
+       "                new_graph[element[0]] = [element[1]]\n",
+       "\n",
+       "        path = set()\n",
+       "\n",
+       "        def visit(vertex):\n",
+       "            path.add(vertex)\n",
+       "            for neighbor in new_graph.get(vertex, ()):\n",
+       "                if neighbor in path or visit(neighbor):\n",
+       "                    return True\n",
+       "            path.remove(vertex)\n",
+       "            return False\n",
+       "\n",
+       "        value = any(visit(v) for v in new_graph)\n",
+       "        return value\n",
+       "\n",
+       "    def add_const(self, constraint, constraints):\n",
+       "        """Add the constraint to constraints if the resulting graph is acyclic"""\n",
+       "\n",
+       "        if constraint[0] == self.finish or constraint[1] == self.start:\n",
+       "            return constraints\n",
+       "\n",
+       "        new_constraints = set(constraints)\n",
+       "        new_constraints.add(constraint)\n",
+       "\n",
+       "        if self.cyclic(new_constraints):\n",
+       "            return constraints\n",
+       "        return new_constraints\n",
+       "\n",
+       "    def is_a_threat(self, precondition, effect):\n",
+       "        """Check if effect is a threat to precondition"""\n",
+       "\n",
+       "        if (str(effect.op) == 'Not' + str(precondition.op)) or ('Not' + str(effect.op) == str(precondition.op)):\n",
+       "            if effect.args == precondition.args:\n",
+       "                return True\n",
+       "        return False\n",
+       "\n",
+       "    def protect(self, causal_link, action, constraints):\n",
+       "        """Check and resolve threats by promotion or demotion"""\n",
+       "\n",
+       "        threat = False\n",
+       "        for effect in action.effect:\n",
+       "            if self.is_a_threat(causal_link[1], effect):\n",
+       "                threat = True\n",
+       "                break\n",
+       "\n",
+       "        if action != causal_link[0] and action != causal_link[2] and threat:\n",
+       "            # try promotion\n",
+       "            new_constraints = set(constraints)\n",
+       "            new_constraints.add((action, causal_link[0]))\n",
+       "            if not self.cyclic(new_constraints):\n",
+       "                constraints = self.add_const((action, causal_link[0]), constraints)\n",
+       "            else:\n",
+       "                # try demotion\n",
+       "                new_constraints = set(constraints)\n",
+       "                new_constraints.add((causal_link[2], action))\n",
+       "                if not self.cyclic(new_constraints):\n",
+       "                    constraints = self.add_const((causal_link[2], action), constraints)\n",
+       "                else:\n",
+       "                    # both promotion and demotion fail\n",
+       "                    print('Unable to resolve a threat caused by', action, 'onto', causal_link)\n",
+       "                    return\n",
+       "        return constraints\n",
+       "\n",
+       "    def convert(self, constraints):\n",
+       "        """Convert constraints into a dict of Action to set orderings"""\n",
+       "\n",
+       "        graph = dict()\n",
+       "        for constraint in constraints:\n",
+       "            if constraint[0] in graph:\n",
+       "                graph[constraint[0]].add(constraint[1])\n",
+       "            else:\n",
+       "                graph[constraint[0]] = set()\n",
+       "                graph[constraint[0]].add(constraint[1])\n",
+       "        return graph\n",
+       "\n",
+       "    def toposort(self, graph):\n",
+       "        """Generate topological ordering of constraints"""\n",
+       "\n",
+       "        if len(graph) == 0:\n",
+       "            return\n",
+       "\n",
+       "        graph = graph.copy()\n",
+       "\n",
+       "        for k, v in graph.items():\n",
+       "            v.discard(k)\n",
+       "\n",
+       "        extra_elements_in_dependencies = _reduce(set.union, graph.values()) - set(graph.keys())\n",
+       "\n",
+       "        graph.update({element:set() for element in extra_elements_in_dependencies})\n",
+       "        while True:\n",
+       "            ordered = set(element for element, dependency in graph.items() if len(dependency) == 0)\n",
+       "            if not ordered:\n",
+       "                break\n",
+       "            yield ordered\n",
+       "            graph = {element: (dependency - ordered) for element, dependency in graph.items() if element not in ordered}\n",
+       "        if len(graph) != 0:\n",
+       "            raise ValueError('The graph is not acyclic and cannot be linearly ordered')\n",
+       "\n",
+       "    def display_plan(self):\n",
+       "        """Display causal links, constraints and the plan"""\n",
+       "\n",
+       "        print('Causal Links')\n",
+       "        for causal_link in self.causal_links:\n",
+       "            print(causal_link)\n",
+       "\n",
+       "        print('\\nConstraints')\n",
+       "        for constraint in self.constraints:\n",
+       "            print(constraint[0], '<', constraint[1])\n",
+       "\n",
+       "        print('\\nPartial Order Plan')\n",
+       "        print(list(reversed(list(self.toposort(self.convert(self.constraints))))))\n",
+       "\n",
+       "    def execute(self, display=True):\n",
+       "        """Execute the algorithm"""\n",
+       "\n",
+       "        step = 1\n",
+       "        self.tries = 1\n",
+       "        while len(self.agenda) > 0:\n",
+       "            step += 1\n",
+       "            # select <G, act1> from Agenda\n",
+       "            try:\n",
+       "                G, act1, possible_actions = self.find_open_precondition()\n",
+       "            except IndexError:\n",
+       "                print('Probably Wrong')\n",
+       "                break\n",
+       "\n",
+       "            act0 = possible_actions[0]\n",
+       "            # remove <G, act1> from Agenda\n",
+       "            self.agenda.remove((G, act1))\n",
+       "\n",
+       "            # For actions with variable number of arguments, use least commitment principle\n",
+       "            # act0_temp, bindings = self.find_action_for_precondition(G)\n",
+       "            # act0 = self.generate_action_object(act0_temp, bindings)\n",
+       "\n",
+       "            # Actions = Actions U {act0}\n",
+       "            self.actions.add(act0)\n",
+       "\n",
+       "            # Constraints = add_const(start < act0, Constraints)\n",
+       "            self.constraints = self.add_const((self.start, act0), self.constraints)\n",
+       "\n",
+       "            # for each CL E CausalLinks do\n",
+       "            #   Constraints = protect(CL, act0, Constraints)\n",
+       "            for causal_link in self.causal_links:\n",
+       "                self.constraints = self.protect(causal_link, act0, self.constraints)\n",
+       "\n",
+       "            # Agenda = Agenda U {<P, act0>: P is a precondition of act0}\n",
+       "            for precondition in act0.precond:\n",
+       "                self.agenda.add((precondition, act0))\n",
+       "\n",
+       "            # Constraints = add_const(act0 < act1, Constraints)\n",
+       "            self.constraints = self.add_const((act0, act1), self.constraints)\n",
+       "\n",
+       "            # CausalLinks U {<act0, G, act1>}\n",
+       "            if (act0, G, act1) not in self.causal_links:\n",
+       "                self.causal_links.append((act0, G, act1))\n",
+       "\n",
+       "            # for each A E Actions do\n",
+       "            #   Constraints = protect(<act0, G, act1>, A, Constraints)\n",
+       "            for action in self.actions:\n",
+       "                self.constraints = self.protect((act0, G, act1), action, self.constraints)\n",
+       "\n",
+       "            if step > 200:\n",
+       "                print('Couldn\\'t find a solution')\n",
+       "                return None, None\n",
+       "\n",
+       "        if display:\n",
+       "            self.display_plan()\n",
+       "        else:\n",
+       "            return self.constraints, self.causal_links                \n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(PartialOrderPlanner)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will first describe the data-structures and helper methods used, followed by the algorithm used to find a partial-order plan." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each plan has the following four components:\n", + "\n", + "1. **`actions`**: a set of actions that make up the steps of the plan.\n", + "`actions` is always a subset of `pddl.actions` the set of possible actions for the given planning problem. \n", + "The `start` and `finish` actions are dummy actions defined to bring uniformity to the problem. The `start` action has no preconditions and its effects constitute the initial state of the planning problem. \n", + "The `finish` action has no effects and its preconditions constitute the goal state of the planning problem.\n", + "The empty plan consists of just these two dummy actions.\n", + "2. **`constraints`**: a set of temporal constraints that define the order of performing the actions relative to each other.\n", + "`constraints` does not define a linear ordering, rather it usually represents a directed graph which is also acyclic if the plan is consistent.\n", + "Each ordering is of the form A < B, which reads as \"A before B\" and means that action A _must_ be executed sometime before action B, but not necessarily immediately before.\n", + "`constraints` stores these as a set of tuples `(Action(A), Action(B))` which is interpreted as given above.\n", + "A constraint cannot be added to `constraints` if it breaks the acyclicity of the existing graph.\n", + "3. **`causal_links`**: a set of causal-links. \n", + "A causal link between two actions _A_ and _B_ in the plan is written as _A_ --_p_--> _B_ and is read as \"A achieves p for B\".\n", + "This imples that _p_ is an effect of _A_ and a precondition of _B_.\n", + "It also asserts that _p_ must remain true from the time of action _A_ to the time of action _B_.\n", + "Any violation of this rule is called a threat and must be resolved immediately by adding suitable ordering constraints.\n", + "`causal_links` stores this information as tuples `(Action(A), precondition(p), Action(B))` which is interpreted as given above.\n", + "Causal-links can also be called **protection-intervals**, because the link _A_ --_p_--> _B_ protects _p_ from being negated over the interval from _A_ to _B_.\n", + "4. **`agenda`**: a set of open-preconditions.\n", + "A precondition is open if it is not achieved by some action in the plan.\n", + "Planners will work to reduce the set of open preconditions to the empty set, without introducing a contradiction.\n", + "`agenda` stored this information as tuples `(precondition(p), Action(A))` where p is a precondition of the action A.\n", + "\n", + "A **consistent plan** is a plan in which there are no cycles in the ordering constraints and no conflicts with the causal-links.\n", + "A consistent plan with no open preconditions is a **solution**.\n", + "
\n", + "
\n", + "Let's briefly glance over the helper functions before going into the actual algorithm.\n", + "
\n", + "**`expand_actions`**: generates all possible actions with variable bindings for use as a heuristic of selection of an open precondition.\n", + "
\n", + "**`find_open_precondition`**: finds a precondition from the agenda with the least number of actions that fulfil that precondition.\n", + "This heuristic helps form mandatory ordering constraints and causal-links to further simplify the problem and reduce the probability of encountering a threat.\n", + "
\n", + "**`find_action_for_precondition`**: finds an action that fulfils the given precondition along with the absolutely necessary variable bindings in accordance with the principle of _least commitment_.\n", + "In case of multiple possible actions, the action with the least number of effects is chosen to minimize the chances of encountering a threat.\n", + "
\n", + "**`cyclic`**: checks if a directed graph is cyclic.\n", + "
\n", + "**`add_const`**: adds `constraint` to `constraints` if the newly formed graph is acyclic and returns `constraints` otherwise.\n", + "
\n", + "**`is_a_threat`**: checks if the given `effect` negates the given `precondition`.\n", + "
\n", + "**`protect`**: checks if the given `action` poses a threat to the given `causal_link`.\n", + "If so, the threat is resolved by either promotion or demotion, whichever generates acyclic temporal constraints.\n", + "If neither promotion or demotion work, the chosen action is not the correct fit or the planning problem cannot be solved altogether.\n", + "
\n", + "**`convert`**: converts a graph from a list of edges to an `Action` : `set` mapping, for use in topological sorting.\n", + "
\n", + "**`toposort`**: a generator function that generates a topological ordering of a given graph as a list of sets.\n", + "Each set contains an action or several actions.\n", + "If a set has more that one action in it, it means that permutations between those actions also produce a valid plan.\n", + "
\n", + "**`display_plan`**: displays the `causal_links`, `constraints` and the partial order plan generated from `toposort`.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The **`execute`** method executes the algorithm, which is summarized below:\n", + "
\n", + "1. An open precondition is selected (a sub-goal that we want to achieve).\n", + "2. An action that fulfils the open precondition is chosen.\n", + "3. Temporal constraints are updated.\n", + "4. Existing causal links are protected. Protection is a method that checks if the causal links conflict\n", + " and if they do, temporal constraints are added to fix the threats.\n", + "5. The set of open preconditions is updated.\n", + "6. Temporal constraints of the selected action and the next action are established.\n", + "7. A new causal link is added between the selected action and the owner of the open precondition.\n", + "8. The set of new causal links is checked for threats and if found, the threat is removed by either promotion or demotion.\n", + " If promotion or demotion is unable to solve the problem, the planning problem cannot be solved with the current sequence of actions\n", + " or it may not be solvable at all.\n", + "9. These steps are repeated until the set of open preconditions is empty." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A partial-order plan can be used to generate different valid total-order plans.\n", + "This step is called **linearization** of the partial-order plan.\n", + "All possible linearizations of a partial-order plan for `socks_and_shoes` looks like this.\n", + "
\n", + "![title](images/pop.jpg)\n", + "
\n", + "Linearization can be carried out in many ways, but the most efficient way is to represent the set of temporal constraints as a directed graph.\n", + "We can easily realize that the graph should also be acyclic as cycles in constraints means that the constraints are inconsistent.\n", + "This acyclicity is enforced by the `add_const` method, which adds a new constraint only if the acyclicity of the existing graph is not violated.\n", + "The `protect` method also checks for acyclicity of the newly-added temporal constraints to make a decision between promotion and demotion in case of a threat.\n", + "This property of a graph created from the temporal constraints of a valid partial-order plan allows us to use topological sort to order the constraints linearly.\n", + "A topological sort may produce several different valid solutions for a given directed acyclic graph." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we know how `PartialOrderPlanner` works, let's solve a few problems using it." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(PutOn(Spare, Axle)), At(Spare, Axle), Action(Finish))\n", + "(Action(Start), Tire(Spare), Action(PutOn(Spare, Axle)))\n", + "(Action(Remove(Flat, Axle)), NotAt(Flat, Axle), Action(PutOn(Spare, Axle)))\n", + "(Action(Start), At(Flat, Axle), Action(Remove(Flat, Axle)))\n", + "(Action(Remove(Spare, Trunk)), At(Spare, Ground), Action(PutOn(Spare, Axle)))\n", + "(Action(Start), At(Spare, Trunk), Action(Remove(Spare, Trunk)))\n", + "(Action(Remove(Flat, Axle)), At(Flat, Ground), Action(Finish))\n", + "\n", + "Constraints\n", + "Action(Remove(Flat, Axle)) < Action(PutOn(Spare, Axle))\n", + "Action(Start) < Action(Finish)\n", + "Action(Remove(Spare, Trunk)) < Action(PutOn(Spare, Axle))\n", + "Action(Start) < Action(Remove(Spare, Trunk))\n", + "Action(Start) < Action(Remove(Flat, Axle))\n", + "Action(Remove(Flat, Axle)) < Action(Finish)\n", + "Action(PutOn(Spare, Axle)) < Action(Finish)\n", + "Action(Start) < Action(PutOn(Spare, Axle))\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(Remove(Flat, Axle)), Action(Remove(Spare, Trunk))}, {Action(PutOn(Spare, Axle))}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "st = spare_tire()\n", + "pop = PartialOrderPlanner(st)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that in the given partial order plan, Remove(Flat, Axle) and Remove(Spare, Trunk) are in the same set.\n", + "This means that the order of performing these actions does not affect the final outcome.\n", + "That aside, we also see that the PutOn(Spare, Axle) action has to be performed after both the Remove actions are complete, which seems logically consistent." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(FromTable(C, B)), On(C, B), Action(Finish))\n", + "(Action(FromTable(B, A)), On(B, A), Action(Finish))\n", + "(Action(Start), OnTable(B), Action(FromTable(B, A)))\n", + "(Action(Start), OnTable(C), Action(FromTable(C, B)))\n", + "(Action(Start), Clear(C), Action(FromTable(C, B)))\n", + "(Action(Start), Clear(A), Action(FromTable(B, A)))\n", + "(Action(ToTable(A, B)), Clear(B), Action(FromTable(C, B)))\n", + "(Action(Start), On(A, B), Action(ToTable(A, B)))\n", + "(Action(ToTable(A, B)), Clear(B), Action(FromTable(B, A)))\n", + "(Action(Start), Clear(A), Action(ToTable(A, B)))\n", + "\n", + "Constraints\n", + "Action(Start) < Action(FromTable(C, B))\n", + "Action(FromTable(B, A)) < Action(FromTable(C, B))\n", + "Action(Start) < Action(FromTable(B, A))\n", + "Action(Start) < Action(ToTable(A, B))\n", + "Action(Start) < Action(Finish)\n", + "Action(FromTable(B, A)) < Action(Finish)\n", + "Action(FromTable(C, B)) < Action(Finish)\n", + "Action(ToTable(A, B)) < Action(FromTable(B, A))\n", + "Action(ToTable(A, B)) < Action(FromTable(C, B))\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(ToTable(A, B))}, {Action(FromTable(B, A))}, {Action(FromTable(C, B))}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "sbw = simple_blocks_world()\n", + "pop = PartialOrderPlanner(sbw)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "We see that this plan does not have flexibility in selecting actions, ie, actions should be performed in this order and this order only, to successfully reach the goal state." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Causal Links\n", + "(Action(RightShoe), RightShoeOn, Action(Finish))\n", + "(Action(LeftShoe), LeftShoeOn, Action(Finish))\n", + "(Action(LeftSock), LeftSockOn, Action(LeftShoe))\n", + "(Action(RightSock), RightSockOn, Action(RightShoe))\n", + "\n", + "Constraints\n", + "Action(LeftSock) < Action(LeftShoe)\n", + "Action(RightSock) < Action(RightShoe)\n", + "Action(Start) < Action(RightShoe)\n", + "Action(Start) < Action(Finish)\n", + "Action(LeftShoe) < Action(Finish)\n", + "Action(Start) < Action(RightSock)\n", + "Action(Start) < Action(LeftShoe)\n", + "Action(Start) < Action(LeftSock)\n", + "Action(RightShoe) < Action(Finish)\n", + "\n", + "Partial Order Plan\n", + "[{Action(Start)}, {Action(LeftSock), Action(RightSock)}, {Action(LeftShoe), Action(RightShoe)}, {Action(Finish)}]\n" + ] + } + ], + "source": [ + "ss = socks_and_shoes()\n", + "pop = PartialOrderPlanner(ss)\n", + "pop.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "This plan again doesn't have constraints in selecting socks or shoes.\n", + "As long as both socks are worn before both shoes, we are fine.\n", + "Notice however, there is one valid solution,\n", + "
\n", + "LeftSock -> LeftShoe -> RightSock -> RightShoe\n", + "
\n", + "that the algorithm could not find as it cannot be represented as a general partially-ordered plan but is a specific total-order solution." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Runtime differences\n", + "Let's briefly take a look at the running time of all the three algorithms on the `socks_and_shoes` problem." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "ss = socks_and_shoes()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "198 µs ± 3.53 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "GraphPlan(ss).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "844 µs ± 23.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "Linearize(ss).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "258 µs ± 4.03 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + ] + } + ], + "source": [ + "%%timeit\n", + "PartialOrderPlanner(ss).execute(display=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We observe that `GraphPlan` is about 4 times faster than `Linearize` because `Linearize` essentially runs a `GraphPlan` subroutine under the hood and then carries out some transformations on the solved planning-graph.\n", + "
\n", + "We also find that `GraphPlan` is slightly faster than `PartialOrderPlanner`, but this is mainly due to the `expand_actions` method in `PartialOrderPlanner` that slows it down as it generates all possible permutations of actions and variable bindings.\n", + "
\n", + "Without heuristic functions, `PartialOrderPlanner` will be atleast as fast as `GraphPlan`, if not faster, but will have a higher tendency to encounter threats and conflicts which might take additional time to resolve.\n", + "
\n", + "Different planning algorithms work differently for different problems." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/planning_total_order_planner.ipynb b/planning_total_order_planner.ipynb new file mode 100644 index 000000000..b94941ece --- /dev/null +++ b/planning_total_order_planner.ipynb @@ -0,0 +1,341 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### TOTAL ORDER PLANNER\n", + "\n", + "In mathematical terminology, **total order**, **linear order** or **simple order** refers to a set *X* which is said to be totally ordered under ≤ if the following statements hold for all *a*, *b* and *c* in *X*:\n", + "
\n", + "If *a* ≤ *b* and *b* ≤ *a*, then *a* = *b* (antisymmetry).\n", + "
\n", + "If *a* ≤ *b* and *b* ≤ *c*, then *a* ≤ *c* (transitivity).\n", + "
\n", + "*a* ≤ *b* or *b* ≤ *a* (connex relation).\n", + "\n", + "
\n", + "In simpler terms, a total order plan is a linear ordering of actions to be taken to reach the goal state.\n", + "There may be several different total-order plans for a particular goal depending on the problem.\n", + "
\n", + "
\n", + "In the module, the `Linearize` class solves problems using this paradigm.\n", + "At its core, the `Linearize` uses a solved planning graph from `GraphPlan` and finds a valid total-order solution for it.\n", + "Let's have a look at the class." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

\n", + "\n", + "
class Linearize:\n",
+       "\n",
+       "    def __init__(self, planningproblem):\n",
+       "        self.planningproblem = planningproblem\n",
+       "\n",
+       "    def filter(self, solution):\n",
+       "        """Filter out persistence actions from a solution"""\n",
+       "\n",
+       "        new_solution = []\n",
+       "        for section in solution[0]:\n",
+       "            new_section = []\n",
+       "            for operation in section:\n",
+       "                if not (operation.op[0] == 'P' and operation.op[1].isupper()):\n",
+       "                    new_section.append(operation)\n",
+       "            new_solution.append(new_section)\n",
+       "        return new_solution\n",
+       "\n",
+       "    def orderlevel(self, level, planningproblem):\n",
+       "        """Return valid linear order of actions for a given level"""\n",
+       "\n",
+       "        for permutation in itertools.permutations(level):\n",
+       "            temp = copy.deepcopy(planningproblem)\n",
+       "            count = 0\n",
+       "            for action in permutation:\n",
+       "                try:\n",
+       "                    temp.act(action)\n",
+       "                    count += 1\n",
+       "                except:\n",
+       "                    count = 0\n",
+       "                    temp = copy.deepcopy(planningproblem)\n",
+       "                    break\n",
+       "            if count == len(permutation):\n",
+       "                return list(permutation), temp\n",
+       "        return None\n",
+       "\n",
+       "    def execute(self):\n",
+       "        """Finds total-order solution for a planning graph"""\n",
+       "\n",
+       "        graphplan_solution = GraphPlan(self.planningproblem).execute()\n",
+       "        filtered_solution = self.filter(graphplan_solution)\n",
+       "        ordered_solution = []\n",
+       "        planningproblem = self.planningproblem\n",
+       "        for level in filtered_solution:\n",
+       "            level_solution, planningproblem = self.orderlevel(level, planningproblem)\n",
+       "            for element in level_solution:\n",
+       "                ordered_solution.append(element)\n",
+       "\n",
+       "        return ordered_solution\n",
+       "
\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(Linearize)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `filter` method removes the persistence actions (if any) from the planning graph representation.\n", + "
\n", + "The `orderlevel` method finds a valid total-ordering of a specified level of the planning-graph, given the state of the graph after the previous level.\n", + "
\n", + "The `execute` method sequentially calls `orderlevel` for all the levels in the planning-graph and returns the final total-order solution.\n", + "
\n", + "
\n", + "Let's look at some examples." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Unload(C2, P2, SFO),\n", + " Unload(C1, P1, JFK)]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for air_cargo problem\n", + "Linearize(air_cargo()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for spare_tire problem\n", + "Linearize(spare_tire()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for three_block_tower problem\n", + "Linearize(three_block_tower()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[ToTable(A, B), FromTable(B, A), FromTable(C, B)]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for simple_blocks_world problem\n", + "Linearize(simple_blocks_world()).execute()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[RightSock, LeftSock, RightShoe, LeftShoe]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# total-order solution for socks_and_shoes problem\n", + "Linearize(socks_and_shoes()).execute()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} From a1f091c22bc82b86ef4261556b0d1fe1fb114ab7 Mon Sep 17 00:00:00 2001 From: Marianna Date: Mon, 23 Jul 2018 15:52:10 +0300 Subject: [PATCH 2/5] Added hierarchical search and tests --- planning.ipynb | 2813 ++++++---------------------------------- planning.py | 22 +- tests/test_planning.py | 27 + 3 files changed, 422 insertions(+), 2440 deletions(-) diff --git a/planning.ipynb b/planning.ipynb index 82be3da14..7b05b3c20 100644 --- a/planning.ipynb +++ b/planning.ipynb @@ -28,10 +28,8 @@ }, { "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": true - }, + "execution_count": 79, + "metadata": {}, "outputs": [], "source": [ "from planning import *\n", @@ -88,7 +86,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 80, "metadata": {}, "outputs": [ { @@ -282,7 +280,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 81, "metadata": {}, "outputs": [ { @@ -505,10 +503,8 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": true - }, + "execution_count": 82, + "metadata": {}, "outputs": [], "source": [ "from utils import *\n", @@ -536,10 +532,8 @@ }, { "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": true - }, + "execution_count": 83, + "metadata": {}, "outputs": [], "source": [ "knowledge_base.extend([\n", @@ -558,7 +552,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 84, "metadata": {}, "outputs": [ { @@ -576,7 +570,7 @@ " At(Sibiu)]" ] }, - "execution_count": 6, + "execution_count": 84, "metadata": {}, "output_type": "execute_result" } @@ -596,10 +590,8 @@ }, { "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": true - }, + "execution_count": 85, + "metadata": {}, "outputs": [], "source": [ "#Sibiu to Bucharest\n", @@ -642,10 +634,8 @@ }, { "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": true - }, + "execution_count": 86, + "metadata": {}, "outputs": [], "source": [ "#Drive\n", @@ -663,10 +653,8 @@ }, { "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": true - }, + "execution_count": 87, + "metadata": {}, "outputs": [], "source": [ "goals = 'At(Bucharest)'" @@ -681,10 +669,8 @@ }, { "cell_type": "code", - "execution_count": 10, - "metadata": { - "collapsed": true - }, + "execution_count": 88, + "metadata": {}, "outputs": [], "source": [ "def goal_test(kb):\n", @@ -700,10 +686,8 @@ }, { "cell_type": "code", - "execution_count": 11, - "metadata": { - "collapsed": true - }, + "execution_count": 89, + "metadata": {}, "outputs": [], "source": [ "prob = PlanningProblem(knowledge_base, goals, [fly_s_b, fly_b_s, fly_s_c, fly_c_s, fly_b_c, fly_c_b, drive])" @@ -730,7 +714,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 90, "metadata": {}, "outputs": [ { @@ -900,10 +884,8 @@ }, { "cell_type": "code", - "execution_count": 13, - "metadata": { - "collapsed": true - }, + "execution_count": 91, + "metadata": {}, "outputs": [], "source": [ "airCargo = air_cargo()" @@ -918,7 +900,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 92, "metadata": {}, "outputs": [ { @@ -954,10 +936,8 @@ }, { "cell_type": "code", - "execution_count": 15, - "metadata": { - "collapsed": true - }, + "execution_count": 93, + "metadata": {}, "outputs": [], "source": [ "solution = [expr(\"Load(C1 , P1, SFO)\"),\n", @@ -980,7 +960,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 94, "metadata": {}, "outputs": [ { @@ -1019,7 +999,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 95, "metadata": {}, "outputs": [ { @@ -1175,10 +1155,8 @@ }, { "cell_type": "code", - "execution_count": 18, - "metadata": { - "collapsed": true - }, + "execution_count": 96, + "metadata": {}, "outputs": [], "source": [ "spareTire = spare_tire()" @@ -1193,7 +1171,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 97, "metadata": {}, "outputs": [ { @@ -1228,10 +1206,8 @@ }, { "cell_type": "code", - "execution_count": 20, - "metadata": { - "collapsed": true - }, + "execution_count": 98, + "metadata": {}, "outputs": [], "source": [ "solution = [expr(\"Remove(Flat, Axle)\"),\n", @@ -1244,7 +1220,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 99, "metadata": {}, "outputs": [ { @@ -1270,10 +1246,8 @@ }, { "cell_type": "code", - "execution_count": 22, - "metadata": { - "collapsed": true - }, + "execution_count": 100, + "metadata": {}, "outputs": [], "source": [ "spareTire = spare_tire()\n", @@ -1288,7 +1262,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 101, "metadata": {}, "outputs": [ { @@ -1346,7 +1320,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 102, "metadata": {}, "outputs": [ { @@ -1503,10 +1477,8 @@ }, { "cell_type": "code", - "execution_count": 25, - "metadata": { - "collapsed": true - }, + "execution_count": 103, + "metadata": {}, "outputs": [], "source": [ "threeBlockTower = three_block_tower()" @@ -1521,7 +1493,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 104, "metadata": {}, "outputs": [ { @@ -1553,10 +1525,8 @@ }, { "cell_type": "code", - "execution_count": 27, - "metadata": { - "collapsed": true - }, + "execution_count": 105, + "metadata": {}, "outputs": [], "source": [ "solution = [expr(\"MoveToTable(C, A)\"),\n", @@ -1576,7 +1546,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 106, "metadata": {}, "outputs": [ { @@ -1609,7 +1579,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 107, "metadata": {}, "outputs": [ { @@ -1767,10 +1737,8 @@ }, { "cell_type": "code", - "execution_count": 30, - "metadata": { - "collapsed": true - }, + "execution_count": 108, + "metadata": {}, "outputs": [], "source": [ "simpleBlocksWorld = simple_blocks_world()" @@ -1785,7 +1753,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 109, "metadata": {}, "outputs": [ { @@ -1794,7 +1762,7 @@ "False" ] }, - "execution_count": 31, + "execution_count": 109, "metadata": {}, "output_type": "execute_result" } @@ -1820,10 +1788,8 @@ }, { "cell_type": "code", - "execution_count": 32, - "metadata": { - "collapsed": true - }, + "execution_count": 110, + "metadata": {}, "outputs": [], "source": [ "solution = [expr('ToTable(A, B)'),\n", @@ -1843,7 +1809,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 111, "metadata": {}, "outputs": [ { @@ -1883,7 +1849,7 @@ }, { "cell_type": "code", - "execution_count": 34, + "execution_count": 112, "metadata": {}, "outputs": [ { @@ -2037,10 +2003,8 @@ }, { "cell_type": "code", - "execution_count": 35, - "metadata": { - "collapsed": true - }, + "execution_count": 113, + "metadata": {}, "outputs": [], "source": [ "shoppingProblem = shopping_problem()" @@ -2055,7 +2019,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 114, "metadata": {}, "outputs": [ { @@ -2091,10 +2055,8 @@ }, { "cell_type": "code", - "execution_count": 37, - "metadata": { - "collapsed": true - }, + "execution_count": 115, + "metadata": {}, "outputs": [], "source": [ "solution = [expr('Go(Home, SM)'),\n", @@ -2117,7 +2079,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 116, "metadata": {}, "outputs": [ { @@ -2126,7 +2088,7 @@ "True" ] }, - "execution_count": 38, + "execution_count": 116, "metadata": {}, "output_type": "execute_result" } @@ -2159,7 +2121,7 @@ }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 117, "metadata": {}, "outputs": [ { @@ -2318,10 +2280,8 @@ }, { "cell_type": "code", - "execution_count": 40, - "metadata": { - "collapsed": true - }, + "execution_count": 118, + "metadata": {}, "outputs": [], "source": [ "socksShoes = socks_and_shoes()" @@ -2336,7 +2296,7 @@ }, { "cell_type": "code", - "execution_count": 41, + "execution_count": 119, "metadata": {}, "outputs": [ { @@ -2345,7 +2305,7 @@ "False" ] }, - "execution_count": 41, + "execution_count": 119, "metadata": {}, "output_type": "execute_result" } @@ -2364,10 +2324,8 @@ }, { "cell_type": "code", - "execution_count": 42, - "metadata": { - "collapsed": true - }, + "execution_count": 120, + "metadata": {}, "outputs": [], "source": [ "solution = [expr('RightSock'),\n", @@ -2378,7 +2336,7 @@ }, { "cell_type": "code", - "execution_count": 43, + "execution_count": 121, "metadata": {}, "outputs": [ { @@ -2387,7 +2345,7 @@ "True" ] }, - "execution_count": 43, + "execution_count": 121, "metadata": {}, "output_type": "execute_result" } @@ -2423,7 +2381,7 @@ }, { "cell_type": "code", - "execution_count": 44, + "execution_count": 122, "metadata": {}, "outputs": [ { @@ -2574,10 +2532,8 @@ }, { "cell_type": "code", - "execution_count": 45, - "metadata": { - "collapsed": true - }, + "execution_count": 123, + "metadata": {}, "outputs": [], "source": [ "cakeProblem = have_cake_and_eat_cake_too()" @@ -2592,7 +2548,7 @@ }, { "cell_type": "code", - "execution_count": 46, + "execution_count": 124, "metadata": {}, "outputs": [ { @@ -2628,10 +2584,8 @@ }, { "cell_type": "code", - "execution_count": 47, - "metadata": { - "collapsed": true - }, + "execution_count": 125, + "metadata": {}, "outputs": [], "source": [ "solution = [expr(\"Eat(Cake)\"),\n", @@ -2650,7 +2604,7 @@ }, { "cell_type": "code", - "execution_count": 48, + "execution_count": 126, "metadata": {}, "outputs": [ { @@ -2682,7 +2636,7 @@ }, { "cell_type": "code", - "execution_count": 49, + "execution_count": 128, "metadata": {}, "outputs": [ { @@ -2690,11 +2644,11 @@ "evalue": "Action 'Bake(Cake)' pre-conditions not satisfied", "output_type": "error", "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mException\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[1;32min\u001b[0m \u001b[0msolution\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m \u001b[0mcakeProblem\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mact\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maction\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[1;32m~\\Documents\\Python\\Data Science\\Machine Learning\\Aima\\planning.py\u001b[0m in \u001b[0;36mact\u001b[1;34m(self, action)\u001b[0m\n\u001b[0;32m 58\u001b[0m \u001b[1;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Action '{}' not found\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maction_name\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 59\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mlist_action\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcheck_precond\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 60\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Action '{}' pre-conditions not satisfied\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maction\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 61\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlist_action\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 62\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mException\u001b[0m: Action 'Bake(Cake)' pre-conditions not satisfied" + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mException\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mcakeProblem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mact\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[0;32m~/aima-python/planning.py\u001b[0m in \u001b[0;36mact\u001b[0;34m(self, action)\u001b[0m\n\u001b[1;32m 58\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Action '{}' not found\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 59\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mlist_action\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheck_precond\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 60\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Action '{}' pre-conditions not satisfied\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 61\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minit\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist_action\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 62\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mException\u001b[0m: Action 'Bake(Cake)' pre-conditions not satisfied" ] } ], @@ -2722,62 +2676,24 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## SOLVING PLANNING PROBLEMS\n", - "----\n", - "### GRAPHPLAN\n", - "
\n", - "The GraphPlan algorithm is a popular method of solving classical planning problems.\n", - "Before we get into the details of the algorithm, let's look at a special data structure called **planning graph**, used to give better heuristic estimates and plays a key role in the GraphPlan algorithm." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Planning Graph\n", - "A planning graph is a directed graph organized into levels. \n", - "Each level contains information about the current state of the knowledge base and the possible state-action links to and from that level.\n", - "The first level contains the initial state with nodes representing each fluent that holds in that level.\n", - "This level has state-action links linking each state to valid actions in that state.\n", - "Each action is linked to all its preconditions and its effect states.\n", - "Based on these effects, the next level is constructed.\n", - "The next level contains similarly structured information about the next state.\n", - "In this way, the graph is expanded using state-action links till we reach a state where all the required goals hold true simultaneously.\n", - "We can say that we have reached our goal if none of the goal states in the current level are mutually exclusive.\n", - "This will be explained in detail later.\n", - "
\n", - "Planning graphs only work for propositional planning problems, hence we need to eliminate all variables by generating all possible substitutions.\n", - "
\n", - "For example, the planning graph of the `have_cake_and_eat_cake_too` problem might look like this\n", - "![title](images/cake_graph.jpg)\n", - "
\n", - "The black lines indicate links between states and actions.\n", - "
\n", - "In every planning problem, we are allowed to carry out the `no-op` action, ie, we can choose no action for a particular state.\n", - "These are called 'Persistence' actions and are represented in the graph by the small square boxes.\n", - "In technical terms, a persistence action has effects same as its preconditions.\n", - "This enables us to carry a state to the next level.\n", - "
\n", + "## PLANNING IN THE REAL WORLD\n", + "---\n", + "## PROBLEM\n", + "The `Problem` class is a wrapper for `PlanningProblem` with some additional functionality and data-structures to handle real-world planning problems that involve time and resource constraints.\n", + "The `Problem` class includes everything that the `PlanningProblem` class includes.\n", + "Additionally, it also includes the following attributes essential to define a real-world planning problem:\n", + "- a list of `jobs` to be done\n", + "- a dictionary of `resources`\n", + "\n", + "It also overloads the `act` method to call the `do_action` method of the `HLA` class, \n", + "and also includes a new method `refinements` that finds refinements or primitive actions for high level actions.\n", "
\n", - "The gray lines indicate mutual exclusivity.\n", - "This means that the actions connected bya gray line cannot be taken together.\n", - "Mutual exclusivity (mutex) occurs in the following cases:\n", - "1. **Inconsistent effects**: One action negates the effect of the other. For example, _Eat(Cake)_ and the persistence of _Have(Cake)_ have inconsistent effects because they disagree on the effect _Have(Cake)_\n", - "2. **Interference**: One of the effects of an action is the negation of a precondition of the other. For example, _Eat(Cake)_ interferes with the persistence of _Have(Cake)_ by negating its precondition.\n", - "3. **Competing needs**: One of the preconditions of one action is mutually exclusive with a precondition of the other. For example, _Bake(Cake)_ and _Eat(Cake)_ are mutex because they compete on the value of the _Have(Cake)_ precondition." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In the module, planning graphs have been implemented using two classes, `Level` which stores data for a particular level and `Graph` which connects multiple levels together.\n", - "Let's look at the `Level` class." + "`hierarchical_search` and `angelic_search` are also built into the `Problem` class to solve such planning problems." ] }, { "cell_type": "code", - "execution_count": 50, + "execution_count": 129, "metadata": {}, "outputs": [ { @@ -2869,135 +2785,269 @@ "\n", "

\n", "\n", - "
class Level:\n",
+       "
class Problem(PlanningProblem):\n",
        "    """\n",
-       "    Contains the state of the planning problem\n",
-       "    and exhaustive list of actions which use the\n",
-       "    states as pre-condition.\n",
-       "    """\n",
-       "\n",
-       "    def __init__(self, kb):\n",
-       "        """Initializes variables to hold state and action details of a level"""\n",
-       "\n",
-       "        self.kb = kb\n",
-       "        # current state\n",
-       "        self.current_state = kb.clauses\n",
-       "        # current action to state link\n",
-       "        self.current_action_links = {}\n",
-       "        # current state to action link\n",
-       "        self.current_state_links = {}\n",
-       "        # current action to next state link\n",
-       "        self.next_action_links = {}\n",
-       "        # next state to current action link\n",
-       "        self.next_state_links = {}\n",
-       "        # mutually exclusive actions\n",
-       "        self.mutex = []\n",
-       "\n",
-       "    def __call__(self, actions, objects):\n",
-       "        self.build(actions, objects)\n",
-       "        self.find_mutex()\n",
-       "\n",
-       "    def separate(self, e):\n",
-       "        """Separates an iterable of elements into positive and negative parts"""\n",
-       "\n",
-       "        positive = []\n",
-       "        negative = []\n",
-       "        for clause in e:\n",
-       "            if clause.op[:3] == 'Not':\n",
-       "                negative.append(clause)\n",
-       "            else:\n",
-       "                positive.append(clause)\n",
-       "        return positive, negative\n",
-       "\n",
-       "    def find_mutex(self):\n",
-       "        """Finds mutually exclusive actions"""\n",
+       "    Define real-world problems by aggregating resources as numerical quantities instead of\n",
+       "    named entities.\n",
        "\n",
-       "        # Inconsistent effects\n",
-       "        pos_nsl, neg_nsl = self.separate(self.next_state_links)\n",
+       "    This class is identical to PDLL, except that it overloads the act function to handle\n",
+       "    resource and ordering conditions imposed by HLA as opposed to Action.\n",
+       "    """\n",
+       "    def __init__(self, init, goals, actions, jobs=None, resources=None):\n",
+       "        super().__init__(init, goals, actions)\n",
+       "        self.jobs = jobs\n",
+       "        self.resources = resources or {}\n",
        "\n",
-       "        for negeff in neg_nsl:\n",
-       "            new_negeff = Expr(negeff.op[3:], *negeff.args)\n",
-       "            for poseff in pos_nsl:\n",
-       "                if new_negeff == poseff:\n",
-       "                    for a in self.next_state_links[poseff]:\n",
-       "                        for b in self.next_state_links[negeff]:\n",
-       "                            if {a, b} not in self.mutex:\n",
-       "                                self.mutex.append({a, b})\n",
+       "    def act(self, action):\n",
+       "        """\n",
+       "        Performs the HLA given as argument.\n",
        "\n",
-       "        # Interference will be calculated with the last step\n",
-       "        pos_csl, neg_csl = self.separate(self.current_state_links)\n",
+       "        Note that this is different from the superclass action - where the parameter was an\n",
+       "        Expression. For real world problems, an Expr object isn't enough to capture all the\n",
+       "        detail required for executing the action - resources, preconditions, etc need to be\n",
+       "        checked for too.\n",
+       "        """\n",
+       "        args = action.args\n",
+       "        list_action = first(a for a in self.actions if a.name == action.name)\n",
+       "        if list_action is None:\n",
+       "            raise Exception("Action '{}' not found".format(action.name))\n",
+       "        self.init = list_action.do_action(self.jobs, self.resources, self.init, args).clauses\n",
        "\n",
-       "        # Competing needs\n",
-       "        for posprecond in pos_csl:\n",
-       "            for negprecond in neg_csl:\n",
-       "                new_negprecond = Expr(negprecond.op[3:], *negprecond.args)\n",
-       "                if new_negprecond == posprecond:\n",
-       "                    for a in self.current_state_links[posprecond]:\n",
-       "                        for b in self.current_state_links[negprecond]:\n",
-       "                            if {a, b} not in self.mutex:\n",
-       "                                self.mutex.append({a, b})\n",
+       "    def refinements(hla, state, library):  # refinements may be (multiple) HLA themselves ...\n",
+       "        """\n",
+       "        state is a Problem, containing the current state kb\n",
+       "        library is a dictionary containing details for every possible refinement. eg:\n",
+       "        {\n",
+       "        'HLA': [\n",
+       "            'Go(Home, SFO)',\n",
+       "            'Go(Home, SFO)',\n",
+       "            'Drive(Home, SFOLongTermParking)',\n",
+       "            'Shuttle(SFOLongTermParking, SFO)',\n",
+       "            'Taxi(Home, SFO)'\n",
+       "            ],\n",
+       "        'steps': [\n",
+       "            ['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'],\n",
+       "            ['Taxi(Home, SFO)'],\n",
+       "            [],\n",
+       "            [],\n",
+       "            []\n",
+       "            ],\n",
+       "        # empty refinements indicate a primitive action\n",
+       "        'precond': [\n",
+       "            ['At(Home) & Have(Car)'],\n",
+       "            ['At(Home)'],\n",
+       "            ['At(Home) & Have(Car)'],\n",
+       "            ['At(SFOLongTermParking)'],\n",
+       "            ['At(Home)']\n",
+       "            ],\n",
+       "        'effect': [\n",
+       "            ['At(SFO) & ~At(Home)'],\n",
+       "            ['At(SFO) & ~At(Home)'],\n",
+       "            ['At(SFOLongTermParking) & ~At(Home)'],\n",
+       "            ['At(SFO) & ~At(SFOLongTermParking)'],\n",
+       "            ['At(SFO) & ~At(Home)']\n",
+       "            ]\n",
+       "        }\n",
+       "        """\n",
+       "        e = Expr(hla.name, hla.args)\n",
+       "        indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+       "        for i in indices:\n",
+       "            actions = []\n",
+       "            for j in range(len(library['steps'][i])):\n",
+       "                # find the index of the step [j]  of the HLA \n",
+       "                index_step = [k for k,x in enumerate(library['HLA']) if x == library['steps'][i][j]][0]\n",
+       "                precond = library['precond'][index_step][0] # preconditions of step [j]\n",
+       "                effect = library['effect'][index_step][0] # effect of step [j]\n",
+       "                actions.append(HLA(library['steps'][i][j], precond, effect))\n",
+       "            yield actions\n",
        "\n",
-       "        # Inconsistent support\n",
-       "        state_mutex = []\n",
-       "        for pair in self.mutex:\n",
-       "            next_state_0 = self.next_action_links[list(pair)[0]]\n",
-       "            if len(pair) == 2:\n",
-       "                next_state_1 = self.next_action_links[list(pair)[1]]\n",
+       "    def hierarchical_search(problem, hierarchy):\n",
+       "        """\n",
+       "        [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical\n",
+       "        Forward Planning Search'\n",
+       "        The problem is a real-world problem defined by the problem class, and the hierarchy is\n",
+       "        a dictionary of HLA - refinements (see refinements generator for details)\n",
+       "        """\n",
+       "        act = Node(problem.actions[0])\n",
+       "        frontier = deque()\n",
+       "        frontier.append(act)\n",
+       "        while True:\n",
+       "            if not frontier:\n",
+       "                return None\n",
+       "            plan = frontier.popleft()\n",
+       "            print(plan.state.name)\n",
+       "            hla = plan.state  # first_or_null(plan)\n",
+       "            prefix = None\n",
+       "            if plan.parent:\n",
+       "                prefix = plan.parent.state.action  # prefix, suffix = subseq(plan.state, hla)\n",
+       "            outcome = Problem.result(problem, prefix)\n",
+       "            if hla is None:\n",
+       "                if outcome.goal_test():\n",
+       "                    return plan.path()\n",
        "            else:\n",
-       "                next_state_1 = self.next_action_links[list(pair)[0]]\n",
-       "            if (len(next_state_0) == 1) and (len(next_state_1) == 1):\n",
-       "                state_mutex.append({next_state_0[0], next_state_1[0]})\n",
-       "        \n",
-       "        self.mutex = self.mutex + state_mutex\n",
+       "                print("else")\n",
+       "                for sequence in Problem.refinements(hla, outcome, hierarchy):\n",
+       "                    print("...")\n",
+       "                    frontier.append(Node(plan.state, plan.parent, sequence))\n",
        "\n",
-       "    def build(self, actions, objects):\n",
-       "        """Populates the lists and dictionaries containing the state action dependencies"""\n",
+       "    def result(state, actions):\n",
+       "        """The outcome of applying an action to the current problem"""\n",
+       "        for a in actions: \n",
+       "            if a.check_precond(state, a.args):\n",
+       "                state = a(state, a.args).clauses\n",
+       "        return state\n",
+       "    \n",
        "\n",
-       "        for clause in self.current_state:\n",
-       "            p_expr = Expr('P' + clause.op, *clause.args)\n",
-       "            self.current_action_links[p_expr] = [clause]\n",
-       "            self.next_action_links[p_expr] = [clause]\n",
-       "            self.current_state_links[clause] = [p_expr]\n",
-       "            self.next_state_links[clause] = [p_expr]\n",
+       "    def angelic_search(problem, hierarchy, initialPlan):\n",
+       "        """\n",
+       "\t[Figure 11.8] A hierarchical planning algorithm that uses angelic semantics to identify and\n",
+       "\tcommit to high-level plans that work while avoiding high-level plans that don’t. \n",
+       "\tThe predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression\n",
+       "\tof refinements. \n",
+       "\tAt top level, call ANGELIC -SEARCH with [Act ] as the initialPlan .\n",
+       "\n",
+       "        initialPlan contains a sequence of HLA's with angelic semantics \n",
+       "\n",
+       "        The possible effects of an angelic HLA in initialPlan are : \n",
+       "        ~ : effect remove\n",
+       "        $+: effect possibly add\n",
+       "        $-: effect possibly remove\n",
+       "        $$: possibly add or remove\n",
+       "\t"""\n",
+       "        frontier = deque(initialPlan)\n",
+       "        while True: \n",
+       "            if not frontier:\n",
+       "                return None\n",
+       "            plan = frontier.popleft() # sequence of HLA/Angelic HLA's \n",
+       "            opt_reachable_set = Problem.reach_opt(problem.init, plan)\n",
+       "            pes_reachable_set = Problem.reach_pes(problem.init, plan)\n",
+       "            if problem.intersects_goal(opt_reachable_set): \n",
+       "                if Problem.is_primitive( plan, hierarchy ): \n",
+       "                    return ([x for x in plan.action])\n",
+       "                guaranteed = problem.intersects_goal(pes_reachable_set) \n",
+       "                if guaranteed and Problem.making_progress(plan, plan):\n",
+       "                    final_state = guaranteed[0] # any element of guaranteed \n",
+       "                    #print('decompose')\n",
+       "                    return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set)\n",
+       "                (hla, index) = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive.\n",
+       "                prefix = plan.action[:index-1]\n",
+       "                suffix = plan.action[index+1:]\n",
+       "                outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+       "                for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+       "                    frontier.append(Angelic_Node(outcome.init, plan, prefix + sequence+ suffix, prefix+sequence+suffix))\n",
+       "\n",
+       "\n",
+       "    def intersects_goal(problem, reachable_set):\n",
+       "        """\n",
+       "        Find the intersection of the reachable states and the goal\n",
+       "        """\n",
+       "        return [y for x in list(reachable_set.keys()) for y in reachable_set[x] if all(goal in y for goal in problem.goals)] \n",
        "\n",
-       "        for a in actions:\n",
-       "            num_args = len(a.args)\n",
-       "            possible_args = tuple(itertools.permutations(objects, num_args))\n",
        "\n",
-       "            for arg in possible_args:\n",
-       "                if a.check_precond(self.kb, arg):\n",
-       "                    for num, symbol in enumerate(a.args):\n",
-       "                        if not symbol.op.islower():\n",
-       "                            arg = list(arg)\n",
-       "                            arg[num] = symbol\n",
-       "                            arg = tuple(arg)\n",
+       "    def is_primitive(plan,  library):\n",
+       "        """\n",
+       "        checks if the hla is primitive action \n",
+       "        """\n",
+       "        for hla in plan.action: \n",
+       "            indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+       "            for i in indices:\n",
+       "                if library["steps"][i]: \n",
+       "                    return False\n",
+       "        return True\n",
+       "             \n",
        "\n",
-       "                    new_action = a.substitute(Expr(a.name, *a.args), arg)\n",
-       "                    self.current_action_links[new_action] = []\n",
        "\n",
-       "                    for clause in a.precond:\n",
-       "                        new_clause = a.substitute(clause, arg)\n",
-       "                        self.current_action_links[new_action].append(new_clause)\n",
-       "                        if new_clause in self.current_state_links:\n",
-       "                            self.current_state_links[new_clause].append(new_action)\n",
-       "                        else:\n",
-       "                            self.current_state_links[new_clause] = [new_action]\n",
-       "                   \n",
-       "                    self.next_action_links[new_action] = []\n",
-       "                    for clause in a.effect:\n",
-       "                        new_clause = a.substitute(clause, arg)\n",
+       "    def reach_opt(init, plan): \n",
+       "        """\n",
+       "        Finds the optimistic reachable set of the sequence of actions in plan \n",
+       "        """\n",
+       "        reachable_set = {0: [init]}\n",
+       "        optimistic_description = plan.action #list of angelic actions with optimistic description\n",
+       "        return Problem.find_reachable_set(reachable_set, optimistic_description)\n",
+       " \n",
+       "\n",
+       "    def reach_pes(init, plan): \n",
+       "        """ \n",
+       "        Finds the pessimistic reachable set of the sequence of actions in plan\n",
+       "        """\n",
+       "        reachable_set = {0: [init]}\n",
+       "        pessimistic_description = plan.action_pes # list of angelic actions with pessimistic description\n",
+       "        return Problem.find_reachable_set(reachable_set, pessimistic_description)\n",
        "\n",
-       "                        self.next_action_links[new_action].append(new_clause)\n",
-       "                        if new_clause in self.next_state_links:\n",
-       "                            self.next_state_links[new_clause].append(new_action)\n",
-       "                        else:\n",
-       "                            self.next_state_links[new_clause] = [new_action]\n",
+       "    def find_reachable_set(reachable_set, action_description):\n",
+       "        """\n",
+       "\tFinds the reachable states of the action_description when applied in each state of reachable set.\n",
+       "\t"""\n",
+       "        for i in range(len(action_description)):\n",
+       "            reachable_set[i+1]=[]\n",
+       "            if type(action_description[i]) is Angelic_HLA:\n",
+       "                possible_actions = action_description[i].angelic_action()\n",
+       "            else: \n",
+       "                possible_actions = action_description\n",
+       "            for action in possible_actions:\n",
+       "                for state in reachable_set[i]:\n",
+       "                    if action.check_precond(state , action.args) :\n",
+       "                        if action.effect[0] :\n",
+       "                            new_state = action(state, action.args).clauses\n",
+       "                            reachable_set[i+1].append(new_state)\n",
+       "                        else: \n",
+       "                            reachable_set[i+1].append(state)\n",
+       "        return reachable_set\n",
+       "\n",
+       "    def find_hla(plan, hierarchy):\n",
+       "        """\n",
+       "        Finds the the first HLA action in plan.action, which is not primitive\n",
+       "        and its corresponding index in plan.action\n",
+       "        """\n",
+       "        hla = None\n",
+       "        index = len(plan.action)\n",
+       "        for i in range(len(plan.action)): # find the first HLA in plan, that is not primitive\n",
+       "            if not Problem.is_primitive(Node(plan.state, plan.parent, [plan.action[i]]), hierarchy):\n",
+       "                hla = plan.action[i] \n",
+       "                index = i\n",
+       "                break\n",
+       "        return (hla, index)\n",
+       "\t\n",
+       "    def making_progress(plan, initialPlan):\n",
+       "        """ \n",
+       "        Not correct\n",
+       "\n",
+       "        Normally should from infinite regression of refinements \n",
+       "        \n",
+       "        Only case covered: when plan contains one action (then there is no regression to be done)  \n",
+       "        """\n",
+       "        if (len(plan.action)==1):\n",
+       "            return False\n",
+       "        return True \n",
+       "\n",
+       "    def decompose(hierarchy, s_0, plan, s_f, reachable_set):\n",
+       "        solution = [] \n",
+       "        while plan.action_pes: \n",
+       "            action = plan.action_pes.pop()\n",
+       "            i = max(reachable_set.keys())\n",
+       "            if (i==0): \n",
+       "                return solution\n",
+       "            s_i = Problem.find_previous_state(s_f, reachable_set,i, action) \n",
+       "            problem = Problem(s_i, s_f , plan.action)\n",
+       "            j=0\n",
+       "            for x in Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])]):\n",
+       "                solution.insert(j,x)\n",
+       "                j+=1\n",
+       "            s_f = s_i\n",
+       "        return solution\n",
        "\n",
-       "    def perform_actions(self):\n",
-       "        """Performs the necessary actions and returns a new Level"""\n",
        "\n",
-       "        new_kb = FolKB(list(set(self.next_state_links.keys())))\n",
-       "        return Level(new_kb)\n",
+       "    def find_previous_state(s_f, reachable_set, i, action):\n",
+       "        """\n",
+       "        Given a final state s_f and an action finds a state s_i in reachable_set \n",
+       "        such that when action is applied to state s_i returns s_f.  \n",
+       "        """\n",
+       "        s_i = reachable_set[i-1][0]\n",
+       "        for state in reachable_set[i-1]:\n",
+       "            if s_f in [x for x in Problem.reach_pes(state, Angelic_Node(state, None, [action],[action]))[1]]:\n",
+       "                s_i =state\n",
+       "                break\n",
+       "        return s_i\n",
        "
\n",
        "\n",
        "\n"
@@ -3011,39 +3061,20 @@
     }
    ],
    "source": [
-    "psource(Level)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Each level stores the following data\n",
-    "1. The current state of the level in `current_state`\n",
-    "2. Links from an action to its preconditions in `current_action_links`\n",
-    "3. Links from a state to the possible actions in that state in `current_state_links`\n",
-    "4. Links from each action to its effects in `next_action_links`\n",
-    "5. Links from each possible next state from each action in `next_state_links`. This stores the same information as the `current_action_links` of the next level.\n",
-    "6. Mutex links in `mutex`.\n",
-    "
\n",
-    "
\n",
-    "The `find_mutex` method finds the mutex links according to the points given above.\n",
-    "
\n",
-    "The `build` method populates the data structures storing the state and action information.\n",
-    "Persistence actions for each clause in the current state are also defined here. \n",
-    "The newly created persistence action has the same name as its state, prefixed with a 'P'."
+    "psource(Problem)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let's now look at the `Graph` class."
+    "## HLA\n",
+    "To be able to model a real-world planning problem properly, it is  essential to be able to represent a _high-level action (HLA)_ that can be hierarchically reduced to primitive actions."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 51,
+   "execution_count": 130,
    "metadata": {},
    "outputs": [
     {
@@ -3135,2094 +3166,30 @@
        "\n",
        "
\n",
        "\n",
-       "
class Graph:\n",
+       "
class HLA(Action):\n",
        "    """\n",
-       "    Contains levels of state and actions\n",
-       "    Used in graph planning algorithm to extract a solution\n",
+       "    Define Actions for the real-world (that may be refined further), and satisfy resource\n",
+       "    constraints.\n",
        "    """\n",
+       "    unique_group = 1\n",
        "\n",
-       "    def __init__(self, pddl):\n",
-       "        self.pddl = pddl\n",
-       "        self.kb = FolKB(pddl.init)\n",
-       "        self.levels = [Level(self.kb)]\n",
-       "        self.objects = set(arg for clause in self.kb.clauses for arg in clause.args)\n",
-       "\n",
-       "    def __call__(self):\n",
-       "        self.expand_graph()\n",
-       "\n",
-       "    def expand_graph(self):\n",
-       "        """Expands the graph by a level"""\n",
-       "\n",
-       "        last_level = self.levels[-1]\n",
-       "        last_level(self.pddl.actions, self.objects)\n",
-       "        self.levels.append(last_level.perform_actions())\n",
-       "\n",
-       "    def non_mutex_goals(self, goals, index):\n",
-       "        """Checks whether the goals are mutually exclusive"""\n",
-       "\n",
-       "        goal_perm = itertools.combinations(goals, 2)\n",
-       "        for g in goal_perm:\n",
-       "            if set(g) in self.levels[index].mutex:\n",
-       "                return False\n",
-       "        return True\n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(Graph)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The class stores a problem definition in `pddl`, \n",
-    "a knowledge base in `kb`, \n",
-    "a list of `Level` objects in `levels` and \n",
-    "all the possible arguments found in the initial state of the problem in `objects`.\n",
-    "
\n",
-    "The `expand_graph` method generates a new level of the graph.\n",
-    "This method is invoked when the goal conditions haven't been met in the current level or the actions that lead to it are mutually exclusive.\n",
-    "The `non_mutex_goals` method checks whether the goals in the current state are mutually exclusive.\n",
-    "
\n",
-    "
\n",
-    "Using these two classes, we can define a planning graph which can either be used to provide reliable heuristics for planning problems or used in the `GraphPlan` algorithm.\n",
-    "
\n",
-    "Let's have a look at the `GraphPlan` class."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 52,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
class GraphPlan:\n",
-       "    """\n",
-       "    Class for formulation GraphPlan algorithm\n",
-       "    Constructs a graph of state and action space\n",
-       "    Returns solution for the planning problem\n",
-       "    """\n",
-       "\n",
-       "    def __init__(self, pddl):\n",
-       "        self.graph = Graph(pddl)\n",
-       "        self.nogoods = []\n",
-       "        self.solution = []\n",
-       "\n",
-       "    def check_leveloff(self):\n",
-       "        """Checks if the graph has levelled off"""\n",
-       "\n",
-       "        check = (set(self.graph.levels[-1].current_state) == set(self.graph.levels[-2].current_state))\n",
-       "\n",
-       "        if check:\n",
-       "            return True\n",
-       "\n",
-       "    def extract_solution(self, goals, index):\n",
-       "        """Extracts the solution"""\n",
-       "\n",
-       "        level = self.graph.levels[index]    \n",
-       "        if not self.graph.non_mutex_goals(goals, index):\n",
-       "            self.nogoods.append((level, goals))\n",
-       "            return\n",
-       "\n",
-       "        level = self.graph.levels[index - 1]    \n",
-       "\n",
-       "        # Create all combinations of actions that satisfy the goal    \n",
-       "        actions = []\n",
-       "        for goal in goals:\n",
-       "            actions.append(level.next_state_links[goal])    \n",
-       "\n",
-       "        all_actions = list(itertools.product(*actions))    \n",
-       "\n",
-       "        # Filter out non-mutex actions\n",
-       "        non_mutex_actions = []    \n",
-       "        for action_tuple in all_actions:\n",
-       "            action_pairs = itertools.combinations(list(set(action_tuple)), 2)        \n",
-       "            non_mutex_actions.append(list(set(action_tuple)))        \n",
-       "            for pair in action_pairs:            \n",
-       "                if set(pair) in level.mutex:\n",
-       "                    non_mutex_actions.pop(-1)\n",
-       "                    break\n",
-       "    \n",
-       "\n",
-       "        # Recursion\n",
-       "        for action_list in non_mutex_actions:        \n",
-       "            if [action_list, index] not in self.solution:\n",
-       "                self.solution.append([action_list, index])\n",
-       "\n",
-       "                new_goals = []\n",
-       "                for act in set(action_list):                \n",
-       "                    if act in level.current_action_links:\n",
-       "                        new_goals = new_goals + level.current_action_links[act]\n",
-       "\n",
-       "                if abs(index) + 1 == len(self.graph.levels):\n",
-       "                    return\n",
-       "                elif (level, new_goals) in self.nogoods:\n",
-       "                    return\n",
-       "                else:\n",
-       "                    self.extract_solution(new_goals, index - 1)\n",
-       "\n",
-       "        # Level-Order multiple solutions\n",
-       "        solution = []\n",
-       "        for item in self.solution:\n",
-       "            if item[1] == -1:\n",
-       "                solution.append([])\n",
-       "                solution[-1].append(item[0])\n",
-       "            else:\n",
-       "                solution[-1].append(item[0])\n",
-       "\n",
-       "        for num, item in enumerate(solution):\n",
-       "            item.reverse()\n",
-       "            solution[num] = item\n",
-       "\n",
-       "        return solution\n",
-       "\n",
-       "    def goal_test(self, kb):\n",
-       "        return all(kb.ask(q) is not False for q in self.graph.pddl.goals)\n",
-       "\n",
-       "    def execute(self):\n",
-       "        """Executes the GraphPlan algorithm for the given problem"""\n",
-       "\n",
-       "        while True:\n",
-       "            self.graph.expand_graph()\n",
-       "            if (self.goal_test(self.graph.levels[-1].kb) and self.graph.non_mutex_goals(self.graph.pddl.goals, -1)):\n",
-       "                solution = self.extract_solution(self.graph.pddl.goals, -1)\n",
-       "                if solution:\n",
-       "                    return solution\n",
-       "            \n",
-       "            if len(self.graph.levels) >= 2 and self.check_leveloff():\n",
-       "                return None\n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(GraphPlan)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Given a planning problem defined as a PlanningProblem, `GraphPlan` creates a planning graph stored in `graph` and expands it till it reaches a state where all its required goals are present simultaneously without mutual exclusivity.\n",
-    "
\n",
-    "Once a goal is found, `extract_solution` is called.\n",
-    "This method recursively finds the path to a solution given a planning graph.\n",
-    "In the case where `extract_solution` fails to find a solution for a set of goals as a given level, we record the `(level, goals)` pair as a **no-good**.\n",
-    "Whenever `extract_solution` is called again with the same level and goals, we can find the recorded no-good and immediately return failure rather than searching again. \n",
-    "No-goods are also used in the termination test.\n",
-    "
\n",
-    "The `check_leveloff` method checks if the planning graph for the problem has **levelled-off**, ie, it has the same states, actions and mutex pairs as the previous level.\n",
-    "If the graph has already levelled off and we haven't found a solution, there is no point expanding the graph, as it won't lead to anything new.\n",
-    "In such a case, we can declare that the planning problem is unsolvable with the given constraints.\n",
-    "
\n",
-    "
\n",
-    "To summarize, the `GraphPlan` algorithm calls `expand_graph` and tests whether it has reached the goal and if the goals are non-mutex.\n",
-    "
\n",
-    "If so, `extract_solution` is invoked which recursively reconstructs the solution from the planning graph.\n",
-    "
\n",
-    "If not, then we check if our graph has levelled off and continue if it hasn't."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's solve a few planning problems that we had defined earlier."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "#### Air cargo problem\n",
-    "In accordance with the summary above, we have defined a helper function to carry out `GraphPlan` on the `air_cargo` problem.\n",
-    "The function is pretty straightforward.\n",
-    "Let's have a look."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 53,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
def air_cargo_graphplan():\n",
-       "    """Solves the air cargo problem using GraphPlan"""\n",
-       "    return GraphPlan(air_cargo()).execute()\n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(air_cargo_graphplan)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let's instantiate the problem and find a solution using this helper function."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 54,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[[[Load(C2, P2, JFK),\n",
-       "   Fly(P2, JFK, SFO),\n",
-       "   Load(C1, P1, SFO),\n",
-       "   Fly(P1, SFO, JFK),\n",
-       "   PCargo(C1),\n",
-       "   PAirport(JFK),\n",
-       "   PPlane(P2),\n",
-       "   PAirport(SFO),\n",
-       "   PPlane(P1),\n",
-       "   PCargo(C2)],\n",
-       "  [Unload(C2, P2, SFO), Unload(C1, P1, JFK)]]]"
-      ]
-     },
-     "execution_count": 54,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "airCargoG = air_cargo_graphplan()\n",
-    "airCargoG"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Each element in the solution is a valid action.\n",
-    "The solution is separated into lists for each level.\n",
-    "The actions prefixed with a 'P' are persistence actions and can be ignored.\n",
-    "They simply carry certain states forward.\n",
-    "We have another helper function `linearize` that presents the solution in a more readable format, much like a total-order planner, but it is _not_ a total-order planner."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 55,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Load(C2, P2, JFK),\n",
-       " Fly(P2, JFK, SFO),\n",
-       " Load(C1, P1, SFO),\n",
-       " Fly(P1, SFO, JFK),\n",
-       " Unload(C2, P2, SFO),\n",
-       " Unload(C1, P1, JFK)]"
-      ]
-     },
-     "execution_count": 55,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "linearize(airCargoG)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Indeed, this is a correct solution.\n",
-    "
\n",
-    "There are similar helper functions for some other planning problems.\n",
-    "
\n",
-    "Lets' try solving the spare tire problem."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 56,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]"
-      ]
-     },
-     "execution_count": 56,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "spareTireG = spare_tire_graphplan()\n",
-    "linearize(spareTireG)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Solution for the cake problem"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 57,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Eat(Cake), Bake(Cake)]"
-      ]
-     },
-     "execution_count": 57,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "cakeProblemG = have_cake_and_eat_cake_too_graphplan()\n",
-    "linearize(cakeProblemG)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Solution for the Sussman's Anomaly configuration of three blocks."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 58,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]"
-      ]
-     },
-     "execution_count": 58,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "sussmanAnomalyG = three_block_tower_graphplan()\n",
-    "linearize(sussmanAnomalyG)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Solution of the socks and shoes problem"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 59,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[LeftSock, RightSock, LeftShoe, RightShoe]"
-      ]
-     },
-     "execution_count": 59,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "socksShoesG = socks_and_shoes_graphplan()\n",
-    "linearize(socksShoesG)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### TOTAL ORDER PLANNER\n",
-    "\n",
-    "In mathematical terminology, **total order**, **linear order** or **simple order** refers to a set *X* which is said to be totally ordered under ≤ if the following statements hold for all *a*, *b* and *c* in *X*:\n",
-    "
\n",
-    "If *a* ≤ *b* and *b* ≤ *a*, then *a* = *b* (antisymmetry).\n",
-    "
\n",
-    "If *a* ≤ *b* and *b* ≤ *c*, then *a* ≤ *c* (transitivity).\n",
-    "
\n",
-    "*a* ≤ *b* or *b* ≤ *a* (connex relation).\n",
-    "\n",
-    "
\n",
-    "In simpler terms, a total order plan is a linear ordering of actions to be taken to reach the goal state.\n",
-    "There may be several different total-order plans for a particular goal depending on the problem.\n",
-    "
\n",
-    "
\n",
-    "In the module, the `Linearize` class solves problems using this paradigm.\n",
-    "At its core, the `Linearize` uses a solved planning graph from `GraphPlan` and finds a valid total-order solution for it.\n",
-    "Let's have a look at the class."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 60,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
class Linearize:\n",
-       "\n",
-       "    def __init__(self, pddl):\n",
-       "        self.pddl = pddl\n",
-       "\n",
-       "    def filter(self, solution):\n",
-       "        """Filter out persistence actions from a solution"""\n",
-       "\n",
-       "        new_solution = []\n",
-       "        for section in solution[0]:\n",
-       "            new_section = []\n",
-       "            for operation in section:\n",
-       "                if not (operation.op[0] == 'P' and operation.op[1].isupper()):\n",
-       "                    new_section.append(operation)\n",
-       "            new_solution.append(new_section)\n",
-       "        return new_solution\n",
-       "\n",
-       "    def orderlevel(self, level, pddl):\n",
-       "        """Return valid linear order of actions for a given level"""\n",
-       "\n",
-       "        for permutation in itertools.permutations(level):\n",
-       "            temp = copy.deepcopy(pddl)\n",
-       "            count = 0\n",
-       "            for action in permutation:\n",
-       "                try:\n",
-       "                    temp.act(action)\n",
-       "                    count += 1\n",
-       "                except:\n",
-       "                    count = 0\n",
-       "                    temp = copy.deepcopy(pddl)\n",
-       "                    break\n",
-       "            if count == len(permutation):\n",
-       "                return list(permutation), temp\n",
-       "        return None\n",
-       "\n",
-       "    def execute(self):\n",
-       "        """Finds total-order solution for a planning graph"""\n",
-       "\n",
-       "        graphplan_solution = GraphPlan(self.pddl).execute()\n",
-       "        filtered_solution = self.filter(graphplan_solution)\n",
-       "        ordered_solution = []\n",
-       "        pddl = self.pddl\n",
-       "        for level in filtered_solution:\n",
-       "            level_solution, pddl = self.orderlevel(level, pddl)\n",
-       "            for element in level_solution:\n",
-       "                ordered_solution.append(element)\n",
-       "\n",
-       "        return ordered_solution\n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(Linearize)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The `filter` method removes the persistence actions (if any) from the planning graph representation.\n",
-    "
\n",
-    "The `orderlevel` method finds a valid total-ordering of a specified level of the planning-graph, given the state of the graph after the previous level.\n",
-    "
\n",
-    "The `execute` method sequentially calls `orderlevel` for all the levels in the planning-graph and returns the final total-order solution.\n",
-    "
\n",
-    "
\n",
-    "Let's look at some examples."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 61,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Load(C2, P2, JFK),\n",
-       " Fly(P2, JFK, SFO),\n",
-       " Load(C1, P1, SFO),\n",
-       " Fly(P1, SFO, JFK),\n",
-       " Unload(C2, P2, SFO),\n",
-       " Unload(C1, P1, JFK)]"
-      ]
-     },
-     "execution_count": 61,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# total-order solution for air_cargo problem\n",
-    "Linearize(air_cargo()).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 62,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]"
-      ]
-     },
-     "execution_count": 62,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# total-order solution for spare_tire problem\n",
-    "Linearize(spare_tire()).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 63,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]"
-      ]
-     },
-     "execution_count": 63,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# total-order solution for three_block_tower problem\n",
-    "Linearize(three_block_tower()).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 64,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[ToTable(A, B), FromTable(B, A), FromTable(C, B)]"
-      ]
-     },
-     "execution_count": 64,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# total-order solution for simple_blocks_world problem\n",
-    "Linearize(simple_blocks_world()).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 65,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[LeftSock, RightSock, LeftShoe, RightShoe]"
-      ]
-     },
-     "execution_count": 65,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# total-order solution for socks_and_shoes problem\n",
-    "Linearize(socks_and_shoes()).execute()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### PARTIAL ORDER  PLANNER\n",
-    "A partial-order planning algorithm is significantly different from a total-order planner.\n",
-    "The way a partial-order plan works enables it to take advantage of _problem decomposition_ and work on each subproblem separately.\n",
-    "It works on several subgoals independently, solves them with several subplans, and then combines the plan.\n",
-    "
\n",
-    "A partial-order planner also follows the **least commitment** strategy, where it delays making choices for as long as possible.\n",
-    "Variables are not bound unless it is absolutely necessary and new actions are chosen only if the existing actions cannot fulfil the required precondition.\n",
-    "
\n",
-    "Any planning algorithm that can place two actions into a plan without specifying which comes first is called a **partial-order planner**.\n",
-    "A partial-order planner searches through the space of plans rather than the space of states, which makes it perform better for certain problems.\n",
-    "
\n",
-    "
\n",
-    "Let's have a look at the `PartialOrderPlanner` class."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 66,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
class PartialOrderPlanner:\n",
-       "\n",
-       "    def __init__(self, pddl):\n",
-       "        self.pddl = pddl\n",
-       "        self.initialize()\n",
-       "\n",
-       "    def initialize(self):\n",
-       "        """Initialize all variables"""\n",
-       "        self.causal_links = []\n",
-       "        self.start = Action('Start', [], self.pddl.init)\n",
-       "        self.finish = Action('Finish', self.pddl.goals, [])\n",
-       "        self.actions = set()\n",
-       "        self.actions.add(self.start)\n",
-       "        self.actions.add(self.finish)\n",
-       "        self.constraints = set()\n",
-       "        self.constraints.add((self.start, self.finish))\n",
-       "        self.agenda = set()\n",
-       "        for precond in self.finish.precond:\n",
-       "            self.agenda.add((precond, self.finish))\n",
-       "        self.expanded_actions = self.expand_actions()\n",
-       "\n",
-       "    def expand_actions(self, name=None):\n",
-       "        """Generate all possible actions with variable bindings for precondition selection heuristic"""\n",
-       "\n",
-       "        objects = set(arg for clause in self.pddl.init for arg in clause.args)\n",
-       "        expansions = []\n",
-       "        action_list = []\n",
-       "        if name is not None:\n",
-       "            for action in self.pddl.actions:\n",
-       "                if str(action.name) == name:\n",
-       "                    action_list.append(action)\n",
-       "        else:\n",
-       "            action_list = self.pddl.actions\n",
-       "\n",
-       "        for action in action_list:\n",
-       "            for permutation in itertools.permutations(objects, len(action.args)):\n",
-       "                bindings = unify(Expr(action.name, *action.args), Expr(action.name, *permutation))\n",
-       "                if bindings is not None:\n",
-       "                    new_args = []\n",
-       "                    for arg in action.args:\n",
-       "                        if arg in bindings:\n",
-       "                            new_args.append(bindings[arg])\n",
-       "                        else:\n",
-       "                            new_args.append(arg)\n",
-       "                    new_expr = Expr(str(action.name), *new_args)\n",
-       "                    new_preconds = []\n",
-       "                    for precond in action.precond:\n",
-       "                        new_precond_args = []\n",
-       "                        for arg in precond.args:\n",
-       "                            if arg in bindings:\n",
-       "                                new_precond_args.append(bindings[arg])\n",
-       "                            else:\n",
-       "                                new_precond_args.append(arg)\n",
-       "                        new_precond = Expr(str(precond.op), *new_precond_args)\n",
-       "                        new_preconds.append(new_precond)\n",
-       "                    new_effects = []\n",
-       "                    for effect in action.effect:\n",
-       "                        new_effect_args = []\n",
-       "                        for arg in effect.args:\n",
-       "                            if arg in bindings:\n",
-       "                                new_effect_args.append(bindings[arg])\n",
-       "                            else:\n",
-       "                                new_effect_args.append(arg)\n",
-       "                        new_effect = Expr(str(effect.op), *new_effect_args)\n",
-       "                        new_effects.append(new_effect)\n",
-       "                    expansions.append(Action(new_expr, new_preconds, new_effects))\n",
-       "\n",
-       "        return expansions\n",
-       "\n",
-       "    def find_open_precondition(self):\n",
-       "        """Find open precondition with the least number of possible actions"""\n",
-       "\n",
-       "        number_of_ways = dict()\n",
-       "        actions_for_precondition = dict()\n",
-       "        for element in self.agenda:\n",
-       "            open_precondition = element[0]\n",
-       "            possible_actions = list(self.actions) + self.expanded_actions\n",
-       "            for action in possible_actions:\n",
-       "                for effect in action.effect:\n",
-       "                    if effect == open_precondition:\n",
-       "                        if open_precondition in number_of_ways:\n",
-       "                            number_of_ways[open_precondition] += 1\n",
-       "                            actions_for_precondition[open_precondition].append(action)\n",
-       "                        else:\n",
-       "                            number_of_ways[open_precondition] = 1\n",
-       "                            actions_for_precondition[open_precondition] = [action]\n",
-       "\n",
-       "        number = sorted(number_of_ways, key=number_of_ways.__getitem__)\n",
-       "        \n",
-       "        for k, v in number_of_ways.items():\n",
-       "            if v == 0:\n",
-       "                return None, None, None\n",
-       "\n",
-       "        act1 = None\n",
-       "        for element in self.agenda:\n",
-       "            if element[0] == number[0]:\n",
-       "                act1 = element[1]\n",
-       "                break\n",
-       "\n",
-       "        if number[0] in self.expanded_actions:\n",
-       "            self.expanded_actions.remove(number[0])\n",
-       "\n",
-       "        return number[0], act1, actions_for_precondition[number[0]]\n",
-       "\n",
-       "    def find_action_for_precondition(self, oprec):\n",
-       "        """Find action for a given precondition"""\n",
-       "\n",
-       "        # either\n",
-       "        #   choose act0 E Actions such that act0 achieves G\n",
-       "        for action in self.actions:\n",
-       "            for effect in action.effect:\n",
-       "                if effect == oprec:\n",
-       "                    return action, 0\n",
-       "\n",
-       "        # or\n",
-       "        #   choose act0 E Actions such that act0 achieves G\n",
-       "        for action in self.pddl.actions:\n",
-       "            for effect in action.effect:\n",
-       "                if effect.op == oprec.op:\n",
-       "                    bindings = unify(effect, oprec)\n",
-       "                    if bindings is None:\n",
-       "                        break\n",
-       "                    return action, bindings\n",
-       "\n",
-       "    def generate_expr(self, clause, bindings):\n",
-       "        """Generate atomic expression from generic expression given variable bindings"""\n",
-       "\n",
-       "        new_args = []\n",
-       "        for arg in clause.args:\n",
-       "            if arg in bindings:\n",
-       "                new_args.append(bindings[arg])\n",
-       "            else:\n",
-       "                new_args.append(arg)\n",
-       "\n",
-       "        try:\n",
-       "            return Expr(str(clause.name), *new_args)\n",
-       "        except:\n",
-       "            return Expr(str(clause.op), *new_args)\n",
-       "        \n",
-       "    def generate_action_object(self, action, bindings):\n",
-       "        """Generate action object given a generic action andvariable bindings"""\n",
-       "\n",
-       "        # if bindings is 0, it means the action already exists in self.actions\n",
-       "        if bindings == 0:\n",
-       "            return action\n",
-       "\n",
-       "        # bindings cannot be None\n",
-       "        else:\n",
-       "            new_expr = self.generate_expr(action, bindings)\n",
-       "            new_preconds = []\n",
-       "            for precond in action.precond:\n",
-       "                new_precond = self.generate_expr(precond, bindings)\n",
-       "                new_preconds.append(new_precond)\n",
-       "            new_effects = []\n",
-       "            for effect in action.effect:\n",
-       "                new_effect = self.generate_expr(effect, bindings)\n",
-       "                new_effects.append(new_effect)\n",
-       "            return Action(new_expr, new_preconds, new_effects)\n",
-       "\n",
-       "    def cyclic(self, graph):\n",
-       "        """Check cyclicity of a directed graph"""\n",
-       "\n",
-       "        new_graph = dict()\n",
-       "        for element in graph:\n",
-       "            if element[0] in new_graph:\n",
-       "                new_graph[element[0]].append(element[1])\n",
-       "            else:\n",
-       "                new_graph[element[0]] = [element[1]]\n",
-       "\n",
-       "        path = set()\n",
-       "\n",
-       "        def visit(vertex):\n",
-       "            path.add(vertex)\n",
-       "            for neighbor in new_graph.get(vertex, ()):\n",
-       "                if neighbor in path or visit(neighbor):\n",
-       "                    return True\n",
-       "            path.remove(vertex)\n",
-       "            return False\n",
-       "\n",
-       "        value = any(visit(v) for v in new_graph)\n",
-       "        return value\n",
-       "\n",
-       "    def add_const(self, constraint, constraints):\n",
-       "        """Add the constraint to constraints if the resulting graph is acyclic"""\n",
-       "\n",
-       "        if constraint[0] == self.finish or constraint[1] == self.start:\n",
-       "            return constraints\n",
-       "\n",
-       "        new_constraints = set(constraints)\n",
-       "        new_constraints.add(constraint)\n",
-       "\n",
-       "        if self.cyclic(new_constraints):\n",
-       "            return constraints\n",
-       "        return new_constraints\n",
-       "\n",
-       "    def is_a_threat(self, precondition, effect):\n",
-       "        """Check if effect is a threat to precondition"""\n",
-       "\n",
-       "        if (str(effect.op) == 'Not' + str(precondition.op)) or ('Not' + str(effect.op) == str(precondition.op)):\n",
-       "            if effect.args == precondition.args:\n",
-       "                return True\n",
-       "        return False\n",
-       "\n",
-       "    def protect(self, causal_link, action, constraints):\n",
-       "        """Check and resolve threats by promotion or demotion"""\n",
-       "\n",
-       "        threat = False\n",
-       "        for effect in action.effect:\n",
-       "            if self.is_a_threat(causal_link[1], effect):\n",
-       "                threat = True\n",
-       "                break\n",
-       "\n",
-       "        if action != causal_link[0] and action != causal_link[2] and threat:\n",
-       "            # try promotion\n",
-       "            new_constraints = set(constraints)\n",
-       "            new_constraints.add((action, causal_link[0]))\n",
-       "            if not self.cyclic(new_constraints):\n",
-       "                constraints = self.add_const((action, causal_link[0]), constraints)\n",
-       "            else:\n",
-       "                # try demotion\n",
-       "                new_constraints = set(constraints)\n",
-       "                new_constraints.add((causal_link[2], action))\n",
-       "                if not self.cyclic(new_constraints):\n",
-       "                    constraints = self.add_const((causal_link[2], action), constraints)\n",
-       "                else:\n",
-       "                    # both promotion and demotion fail\n",
-       "                    print('Unable to resolve a threat caused by', action, 'onto', causal_link)\n",
-       "                    return\n",
-       "        return constraints\n",
-       "\n",
-       "    def convert(self, constraints):\n",
-       "        """Convert constraints into a dict of Action to set orderings"""\n",
-       "\n",
-       "        graph = dict()\n",
-       "        for constraint in constraints:\n",
-       "            if constraint[0] in graph:\n",
-       "                graph[constraint[0]].add(constraint[1])\n",
-       "            else:\n",
-       "                graph[constraint[0]] = set()\n",
-       "                graph[constraint[0]].add(constraint[1])\n",
-       "        return graph\n",
-       "\n",
-       "    def toposort(self, graph):\n",
-       "        """Generate topological ordering of constraints"""\n",
-       "\n",
-       "        if len(graph) == 0:\n",
-       "            return\n",
-       "\n",
-       "        graph = graph.copy()\n",
-       "\n",
-       "        for k, v in graph.items():\n",
-       "            v.discard(k)\n",
-       "\n",
-       "        extra_elements_in_dependencies = _reduce(set.union, graph.values()) - set(graph.keys())\n",
-       "\n",
-       "        graph.update({element:set() for element in extra_elements_in_dependencies})\n",
-       "        while True:\n",
-       "            ordered = set(element for element, dependency in graph.items() if len(dependency) == 0)\n",
-       "            if not ordered:\n",
-       "                break\n",
-       "            yield ordered\n",
-       "            graph = {element: (dependency - ordered) for element, dependency in graph.items() if element not in ordered}\n",
-       "        if len(graph) != 0:\n",
-       "            raise ValueError('The graph is not acyclic and cannot be linearly ordered')\n",
-       "\n",
-       "    def display_plan(self):\n",
-       "        """Display causal links, constraints and the plan"""\n",
-       "\n",
-       "        print('Causal Links')\n",
-       "        for causal_link in self.causal_links:\n",
-       "            print(causal_link)\n",
-       "\n",
-       "        print('\\nConstraints')\n",
-       "        for constraint in self.constraints:\n",
-       "            print(constraint[0], '<', constraint[1])\n",
-       "\n",
-       "        print('\\nPartial Order Plan')\n",
-       "        print(list(reversed(list(self.toposort(self.convert(self.constraints))))))\n",
-       "\n",
-       "    def execute(self, display=True):\n",
-       "        """Execute the algorithm"""\n",
-       "\n",
-       "        step = 1\n",
-       "        self.tries = 1\n",
-       "        while len(self.agenda) > 0:\n",
-       "            step += 1\n",
-       "            # select <G, act1> from Agenda\n",
-       "            try:\n",
-       "                G, act1, possible_actions = self.find_open_precondition()\n",
-       "            except IndexError:\n",
-       "                print('Probably Wrong')\n",
-       "                break\n",
-       "\n",
-       "            act0 = possible_actions[0]\n",
-       "            # remove <G, act1> from Agenda\n",
-       "            self.agenda.remove((G, act1))\n",
-       "\n",
-       "            # For actions with variable number of arguments, use least commitment principle\n",
-       "            # act0_temp, bindings = self.find_action_for_precondition(G)\n",
-       "            # act0 = self.generate_action_object(act0_temp, bindings)\n",
-       "\n",
-       "            # Actions = Actions U {act0}\n",
-       "            self.actions.add(act0)\n",
-       "\n",
-       "            # Constraints = add_const(start < act0, Constraints)\n",
-       "            self.constraints = self.add_const((self.start, act0), self.constraints)\n",
-       "\n",
-       "            # for each CL E CausalLinks do\n",
-       "            #   Constraints = protect(CL, act0, Constraints)\n",
-       "            for causal_link in self.causal_links:\n",
-       "                self.constraints = self.protect(causal_link, act0, self.constraints)\n",
-       "\n",
-       "            # Agenda = Agenda U {<P, act0>: P is a precondition of act0}\n",
-       "            for precondition in act0.precond:\n",
-       "                self.agenda.add((precondition, act0))\n",
-       "\n",
-       "            # Constraints = add_const(act0 < act1, Constraints)\n",
-       "            self.constraints = self.add_const((act0, act1), self.constraints)\n",
-       "\n",
-       "            # CausalLinks U {<act0, G, act1>}\n",
-       "            if (act0, G, act1) not in self.causal_links:\n",
-       "                self.causal_links.append((act0, G, act1))\n",
-       "\n",
-       "            # for each A E Actions do\n",
-       "            #   Constraints = protect(<act0, G, act1>, A, Constraints)\n",
-       "            for action in self.actions:\n",
-       "                self.constraints = self.protect((act0, G, act1), action, self.constraints)\n",
-       "\n",
-       "            if step > 200:\n",
-       "                print('Couldn\\'t find a solution')\n",
-       "                return None, None\n",
-       "\n",
-       "        if display:\n",
-       "            self.display_plan()\n",
-       "        else:\n",
-       "            return self.constraints, self.causal_links                \n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(PartialOrderPlanner)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We will first describe the data-structures and helper methods used, followed by the algorithm used to find a partial-order plan."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Each plan has the following four components:\n",
-    "\n",
-    "1. **`actions`**: a set of actions that make up the steps of the plan.\n",
-    "`actions` is always a subset of `pddl.actions` the set of possible actions for the given planning problem. \n",
-    "The `start` and `finish` actions are dummy actions defined to bring uniformity to the problem. The `start` action has no preconditions and its effects constitute the initial state of the planning problem. \n",
-    "The `finish` action has no effects and its preconditions constitute the goal state of the planning problem.\n",
-    "The empty plan consists of just these two dummy actions.\n",
-    "2. **`constraints`**: a set of temporal constraints that define the order of performing the actions relative to each other.\n",
-    "`constraints` does not define a linear ordering, rather it usually represents a directed graph which is also acyclic if the plan is consistent.\n",
-    "Each ordering is of the form A < B, which reads as \"A before B\" and means that action A _must_ be executed sometime before action B, but not necessarily immediately before.\n",
-    "`constraints` stores these as a set of tuples `(Action(A), Action(B))` which is interpreted as given above.\n",
-    "A constraint cannot be added to `constraints` if it breaks the acyclicity of the existing graph.\n",
-    "3. **`causal_links`**: a set of causal-links. \n",
-    "A causal link between two actions _A_ and _B_ in the plan is written as _A_ --_p_--> _B_ and is read as \"A achieves p for B\".\n",
-    "This imples that _p_ is an effect of _A_ and a precondition of _B_.\n",
-    "It also asserts that _p_ must remain true from the time of action _A_ to the time of action _B_.\n",
-    "Any violation of this rule is called a threat and must be resolved immediately by adding suitable ordering constraints.\n",
-    "`causal_links` stores this information as tuples `(Action(A), precondition(p), Action(B))` which is interpreted as given above.\n",
-    "Causal-links can also be called **protection-intervals**, because the link _A_ --_p_--> _B_ protects _p_ from being negated over the interval from _A_ to _B_.\n",
-    "4. **`agenda`**: a set of open-preconditions.\n",
-    "A precondition is open if it is not achieved by some action in the plan.\n",
-    "Planners will work to reduce the set of open preconditions to the empty set, without introducing a contradiction.\n",
-    "`agenda` stored this information as tuples `(precondition(p), Action(A))` where p is a precondition of the action A.\n",
-    "\n",
-    "A **consistent plan** is a plan in which there are no cycles in the ordering constraints and no conflicts with the causal-links.\n",
-    "A consistent plan with no open preconditions is a **solution**.\n",
-    "
\n",
-    "
\n",
-    "Let's briefly glance over the helper functions before going into the actual algorithm.\n",
-    "
\n",
-    "**`expand_actions`**: generates all possible actions with variable bindings for use as a heuristic of selection of an open precondition.\n",
-    "
\n",
-    "**`find_open_precondition`**: finds a precondition from the agenda with the least number of actions that fulfil that precondition.\n",
-    "This heuristic helps form mandatory ordering constraints and causal-links to further simplify the problem and reduce the probability of encountering a threat.\n",
-    "
\n",
-    "**`find_action_for_precondition`**: finds an action that fulfils the given precondition along with the absolutely necessary variable bindings in accordance with the principle of _least commitment_.\n",
-    "In case of multiple possible actions, the action with the least number of effects is chosen to minimize the chances of encountering a threat.\n",
-    "
\n",
-    "**`cyclic`**: checks if a directed graph is cyclic.\n",
-    "
\n",
-    "**`add_const`**: adds `constraint` to `constraints` if the newly formed graph is acyclic and returns `constraints` otherwise.\n",
-    "
\n",
-    "**`is_a_threat`**: checks if the given `effect` negates the given `precondition`.\n",
-    "
\n",
-    "**`protect`**: checks if the given `action` poses a threat to the given `causal_link`.\n",
-    "If so, the threat is resolved by either promotion or demotion, whichever generates acyclic temporal constraints.\n",
-    "If neither promotion or demotion work, the chosen action is not the correct fit or the planning problem cannot be solved altogether.\n",
-    "
\n",
-    "**`convert`**: converts a graph from a list of edges to an `Action` : `set` mapping, for use in topological sorting.\n",
-    "
\n",
-    "**`toposort`**: a generator function that generates a topological ordering of a given graph as a list of sets.\n",
-    "Each set contains an action or several actions.\n",
-    "If a set has more that one action in it, it means that permutations between those actions also produce a valid plan.\n",
-    "
\n",
-    "**`display_plan`**: displays the `causal_links`, `constraints` and the partial order plan generated from `toposort`.\n",
-    "
"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The **`execute`** method executes the algorithm, which is summarized below:\n",
-    "
\n",
-    "1. An open precondition is selected (a sub-goal that we want to achieve).\n",
-    "2. An action that fulfils the open precondition is chosen.\n",
-    "3. Temporal constraints are updated.\n",
-    "4. Existing causal links are protected. Protection is a method that checks if the causal links conflict\n",
-    "   and if they do, temporal constraints are added to fix the threats.\n",
-    "5. The set of open preconditions is updated.\n",
-    "6. Temporal constraints of the selected action and the next action are established.\n",
-    "7. A new causal link is added between the selected action and the owner of the open precondition.\n",
-    "8. The set of new causal links is checked for threats and if found, the threat is removed by either promotion or demotion.\n",
-    "   If promotion or demotion is unable to solve the problem, the planning problem cannot be solved with the current sequence of actions\n",
-    "   or it may not be solvable at all.\n",
-    "9. These steps are repeated until the set of open preconditions is empty."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "A partial-order plan can be used to generate different valid total-order plans.\n",
-    "This step is called **linearization** of the partial-order plan.\n",
-    "All possible linearizations of a partial-order plan for `socks_and_shoes` looks like this.\n",
-    "
\n",
-    "![title](images/pop.jpg)\n",
-    "
\n",
-    "Linearization can be carried out in many ways, but the most efficient way is to represent the set of temporal constraints as a directed graph.\n",
-    "We can easily realize that the graph should also be acyclic as cycles in constraints means that the constraints are inconsistent.\n",
-    "This acyclicity is enforced by the `add_const` method, which adds a new constraint only if the acyclicity of the existing graph is not violated.\n",
-    "The `protect` method also checks for acyclicity of the newly-added temporal constraints to make a decision between promotion and demotion in case of a threat.\n",
-    "This property of a graph created from the temporal constraints of a valid partial-order plan allows us to use topological sort to order the constraints linearly.\n",
-    "A topological sort may produce several different valid solutions for a given directed acyclic graph."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Now that we know how `PartialOrderPlanner` works, let's solve a few problems using it."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 67,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Causal Links\n",
-      "(Action(PutOn(Spare, Axle)), At(Spare, Axle), Action(Finish))\n",
-      "(Action(Start), Tire(Spare), Action(PutOn(Spare, Axle)))\n",
-      "(Action(Remove(Flat, Axle)), NotAt(Flat, Axle), Action(PutOn(Spare, Axle)))\n",
-      "(Action(Start), At(Flat, Axle), Action(Remove(Flat, Axle)))\n",
-      "(Action(Remove(Spare, Trunk)), At(Spare, Ground), Action(PutOn(Spare, Axle)))\n",
-      "(Action(Start), At(Spare, Trunk), Action(Remove(Spare, Trunk)))\n",
-      "(Action(Remove(Flat, Axle)), At(Flat, Ground), Action(Finish))\n",
-      "\n",
-      "Constraints\n",
-      "Action(Start) < Action(Finish)\n",
-      "Action(Start) < Action(Remove(Spare, Trunk))\n",
-      "Action(Remove(Flat, Axle)) < Action(PutOn(Spare, Axle))\n",
-      "Action(Remove(Flat, Axle)) < Action(Finish)\n",
-      "Action(Remove(Spare, Trunk)) < Action(PutOn(Spare, Axle))\n",
-      "Action(Start) < Action(PutOn(Spare, Axle))\n",
-      "Action(Start) < Action(Remove(Flat, Axle))\n",
-      "Action(PutOn(Spare, Axle)) < Action(Finish)\n",
-      "\n",
-      "Partial Order Plan\n",
-      "[{Action(Start)}, {Action(Remove(Flat, Axle)), Action(Remove(Spare, Trunk))}, {Action(PutOn(Spare, Axle))}, {Action(Finish)}]\n"
-     ]
-    }
-   ],
-   "source": [
-    "st = spare_tire()\n",
-    "pop = PartialOrderPlanner(st)\n",
-    "pop.execute()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We observe that in the given partial order plan, Remove(Flat, Axle) and Remove(Spare, Trunk) are in the same set.\n",
-    "This means that the order of performing these actions does not affect the final outcome.\n",
-    "That aside, we also see that the PutOn(Spare, Axle) action has to be performed after both the Remove actions are complete, which seems logically consistent."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 68,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Causal Links\n",
-      "(Action(FromTable(B, A)), On(B, A), Action(Finish))\n",
-      "(Action(FromTable(C, B)), On(C, B), Action(Finish))\n",
-      "(Action(Start), Clear(C), Action(FromTable(C, B)))\n",
-      "(Action(Start), Clear(A), Action(FromTable(B, A)))\n",
-      "(Action(Start), OnTable(C), Action(FromTable(C, B)))\n",
-      "(Action(Start), OnTable(B), Action(FromTable(B, A)))\n",
-      "(Action(ToTable(A, B)), Clear(B), Action(FromTable(C, B)))\n",
-      "(Action(Start), On(A, B), Action(ToTable(A, B)))\n",
-      "(Action(ToTable(A, B)), Clear(B), Action(FromTable(B, A)))\n",
-      "(Action(Start), Clear(A), Action(ToTable(A, B)))\n",
-      "\n",
-      "Constraints\n",
-      "Action(Start) < Action(FromTable(B, A))\n",
-      "Action(Start) < Action(FromTable(C, B))\n",
-      "Action(Start) < Action(ToTable(A, B))\n",
-      "Action(ToTable(A, B)) < Action(FromTable(C, B))\n",
-      "Action(Start) < Action(Finish)\n",
-      "Action(ToTable(A, B)) < Action(FromTable(B, A))\n",
-      "Action(FromTable(C, B)) < Action(Finish)\n",
-      "Action(FromTable(B, A)) < Action(Finish)\n",
-      "Action(FromTable(B, A)) < Action(FromTable(C, B))\n",
-      "\n",
-      "Partial Order Plan\n",
-      "[{Action(Start)}, {Action(ToTable(A, B))}, {Action(FromTable(B, A))}, {Action(FromTable(C, B))}, {Action(Finish)}]\n"
-     ]
-    }
-   ],
-   "source": [
-    "sbw = simple_blocks_world()\n",
-    "pop = PartialOrderPlanner(sbw)\n",
-    "pop.execute()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "collapsed": true
-   },
-   "source": [
-    "We see that this plan does not have flexibility in selecting actions, ie, actions should be performed in this order and this order only, to successfully reach the goal state."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 69,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Causal Links\n",
-      "(Action(RightShoe), RightShoeOn, Action(Finish))\n",
-      "(Action(LeftShoe), LeftShoeOn, Action(Finish))\n",
-      "(Action(LeftSock), LeftSockOn, Action(LeftShoe))\n",
-      "(Action(RightSock), RightSockOn, Action(RightShoe))\n",
-      "\n",
-      "Constraints\n",
-      "Action(Start) < Action(RightSock)\n",
-      "Action(Start) < Action(LeftSock)\n",
-      "Action(RightSock) < Action(RightShoe)\n",
-      "Action(RightShoe) < Action(Finish)\n",
-      "Action(Start) < Action(LeftShoe)\n",
-      "Action(LeftSock) < Action(LeftShoe)\n",
-      "Action(Start) < Action(RightShoe)\n",
-      "Action(Start) < Action(Finish)\n",
-      "Action(LeftShoe) < Action(Finish)\n",
-      "\n",
-      "Partial Order Plan\n",
-      "[{Action(Start)}, {Action(LeftSock), Action(RightSock)}, {Action(RightShoe), Action(LeftShoe)}, {Action(Finish)}]\n"
-     ]
-    }
-   ],
-   "source": [
-    "ss = socks_and_shoes()\n",
-    "pop = PartialOrderPlanner(ss)\n",
-    "pop.execute()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "collapsed": true
-   },
-   "source": [
-    "This plan again doesn't have constraints in selecting socks or shoes.\n",
-    "As long as both socks are worn before both shoes, we are fine.\n",
-    "Notice however, there is one valid solution,\n",
-    "
\n",
-    "LeftSock -> LeftShoe -> RightSock -> RightShoe\n",
-    "
\n",
-    "that the algorithm could not find as it cannot be represented as a general partially-ordered plan but is a specific total-order solution."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Runtime differences\n",
-    "Let's briefly take a look at the running time of all the three algorithms on the `socks_and_shoes` problem."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 70,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "ss = socks_and_shoes()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 71,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "333 µs ± 8.86 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
-     ]
-    }
-   ],
-   "source": [
-    "%%timeit\n",
-    "GraphPlan(ss).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 72,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "1.29 ms ± 43.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
-     ]
-    }
-   ],
-   "source": [
-    "%%timeit\n",
-    "Linearize(ss).execute()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 73,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "425 µs ± 17 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
-     ]
-    }
-   ],
-   "source": [
-    "%%timeit\n",
-    "PartialOrderPlanner(ss).execute(display=False)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We observe that `GraphPlan` is about 4 times faster than `Linearize` because `Linearize` essentially runs a `GraphPlan` subroutine under the hood and then carries out some transformations on the solved planning-graph.\n",
-    "
\n",
-    "We also find that `GraphPlan` is slightly faster than `PartialOrderPlanner`, but this is mainly due to the `expand_actions` method in `PartialOrderPlanner` that slows it down as it generates all possible permutations of actions and variable bindings.\n",
-    "
\n",
-    "Without heuristic functions, `PartialOrderPlanner` will be atleast as fast as `GraphPlan`, if not faster, but will have a higher tendency to encounter threats and conflicts which might take additional time to resolve.\n",
-    "
\n",
-    "Different planning algorithms work differently for different problems."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## PLANNING IN THE REAL WORLD\n",
-    "---\n",
-    "## PROBLEM\n",
-    "The `Problem` class is a wrapper for `PlanningProblem` with some additional functionality and data-structures to handle real-world planning problems that involve time and resource constraints.\n",
-    "The `Problem` class includes everything that the `PlanningProblem` class includes.\n",
-    "Additionally, it also includes the following attributes essential to define a real-world planning problem:\n",
-    "- a list of `jobs` to be done\n",
-    "- a dictionary of `resources`\n",
-    "\n",
-    "It also overloads the `act` method to call the `do_action` method of the `HLA` class, \n",
-    "and also includes a new method `refinements` that finds refinements or primitive actions for high level actions.\n",
-    "
\n",
-    "`hierarchical_search` and `angelic_search` are also built into the `Problem` class to solve such planning problems."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 74,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
class Problem(PlanningProblem):\n",
-       "    """\n",
-       "    Define real-world problems by aggregating resources as numerical quantities instead of\n",
-       "    named entities.\n",
-       "\n",
-       "    This class is identical to PDLL, except that it overloads the act function to handle\n",
-       "    resource and ordering conditions imposed by HLA as opposed to Action.\n",
-       "    """\n",
-       "    def __init__(self, init, goals, actions, jobs=None, resources=None):\n",
-       "        super().__init__(init, goals, actions)\n",
-       "        self.jobs = jobs\n",
-       "        self.resources = resources or {}\n",
-       "\n",
-       "    def act(self, action):\n",
-       "        """\n",
-       "        Performs the HLA given as argument.\n",
-       "\n",
-       "        Note that this is different from the superclass action - where the parameter was an\n",
-       "        Expression. For real world problems, an Expr object isn't enough to capture all the\n",
-       "        detail required for executing the action - resources, preconditions, etc need to be\n",
-       "        checked for too.\n",
-       "        """\n",
-       "        args = action.args\n",
-       "        list_action = first(a for a in self.actions if a.name == action.name)\n",
-       "        if list_action is None:\n",
-       "            raise Exception("Action '{}' not found".format(action.name))\n",
-       "        self.init = list_action.do_action(self.jobs, self.resources, self.init, args).clauses\n",
-       "\n",
-       "    def refinements(hla, state, library):  # TODO - refinements may be (multiple) HLA themselves ...\n",
-       "        """\n",
-       "        state is a Problem, containing the current state kb\n",
-       "        library is a dictionary containing details for every possible refinement. eg:\n",
-       "        {\n",
-       "        'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', 'Taxi(Home, SFO)'],\n",
-       "        'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []],\n",
-       "        # empty refinements ie primitive action\n",
-       "        'precond': [['At(Home), Have(Car)'], ['At(Home)'], ['At(Home)', 'Have(Car)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n",
-       "        'effect': [['At(SFO)'], ['At(SFO)'], ['At(SFOLongTermParking)'], ['At(SFO)'], ['At(SFO)'], ['~At(Home)'], ['~At(Home)'], ['~At(Home)'], ['~At(SFOLongTermParking)'], ['~At(Home)']]\n",
-       "        }\n",
-       "        """\n",
-       "        e = Expr(hla.name, hla.args)\n",
-       "        indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
-       "        for i in indices:\n",
-       "            # TODO multiple refinements\n",
-       "            precond = []\n",
-       "            for p in library['precond'][i]:\n",
-       "                if p[0] == '~':\n",
-       "                    precond.append(expr('Not' + p[1:]))\n",
-       "                else:\n",
-       "                    precond.append(expr(p))\n",
-       "            effect = []\n",
-       "            for e in library['effect'][i]:\n",
-       "                if e[0] == '~':\n",
-       "                    effect.append(expr('Not' + e[1:]))\n",
-       "                else:\n",
-       "                    effect.append(expr(e))\n",
-       "            action = HLA(library['steps'][i][0], precond, effect)\n",
-       "            if action.check_precond(state.init, action.args):\n",
-       "                yield action\n",
-       "\n",
-       "    def hierarchical_search(problem, hierarchy):\n",
-       "        """\n",
-       "        [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical\n",
-       "        Forward Planning Search'\n",
-       "        The problem is a real-world problem defined by the problem class, and the hierarchy is\n",
-       "        a dictionary of HLA - refinements (see refinements generator for details)\n",
-       "        """\n",
-       "        act = Node(problem.actions[0])\n",
-       "        frontier = deque()\n",
-       "        frontier.append(act)\n",
-       "        while True:\n",
-       "            if not frontier:\n",
-       "                return None\n",
-       "            plan = frontier.popleft()\n",
-       "            print(plan.state.name)\n",
-       "            hla = plan.state  # first_or_null(plan)\n",
-       "            prefix = None\n",
-       "            if plan.parent:\n",
-       "                prefix = plan.parent.state.action  # prefix, suffix = subseq(plan.state, hla)\n",
-       "            outcome = Problem.result(problem, prefix)\n",
-       "            if hla is None:\n",
-       "                if outcome.goal_test():\n",
-       "                    return plan.path()\n",
-       "            else:\n",
-       "                print("else")\n",
-       "                for sequence in Problem.refinements(hla, outcome, hierarchy):\n",
-       "                    print("...")\n",
-       "                    frontier.append(Node(plan.state, plan.parent, sequence))\n",
-       "\n",
-       "    def result(problem, action):\n",
-       "        """The outcome of applying an action to the current problem"""\n",
-       "        if action is not None:\n",
-       "            problem.act(action)\n",
-       "            return problem\n",
-       "        else:\n",
-       "            return problem\n",
-       "
\n",
-       "\n",
-       "\n"
-      ],
-      "text/plain": [
-       ""
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "psource(Problem)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## HLA\n",
-    "To be able to model a real-world planning problem properly, it is  essential to be able to represent a _high-level action (HLA)_ that can be hierarchically reduced to primitive actions."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 75,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "  Codestin Search App\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "\n",
-       "
\n",
-       "\n",
-       "
class HLA(Action):\n",
-       "    """\n",
-       "    Define Actions for the real-world (that may be refined further), and satisfy resource\n",
-       "    constraints.\n",
-       "    """\n",
-       "    unique_group = 1\n",
-       "\n",
-       "    def __init__(self, action, precond=None, effect=None, duration=0,\n",
-       "                 consume=None, use=None):\n",
-       "        """\n",
-       "        As opposed to actions, to define HLA, we have added constraints.\n",
-       "        duration holds the amount of time required to execute the task\n",
-       "        consumes holds a dictionary representing the resources the task consumes\n",
-       "        uses holds a dictionary representing the resources the task uses\n",
-       "        """\n",
-       "        precond = precond or [None]\n",
-       "        effect = effect or [None]\n",
-       "        super().__init__(action, precond, effect)\n",
-       "        self.duration = duration\n",
-       "        self.consumes = consume or {}\n",
-       "        self.uses = use or {}\n",
-       "        self.completed = False\n",
-       "        # self.priority = -1 #  must be assigned in relation to other HLAs\n",
-       "        # self.job_group = -1 #  must be assigned in relation to other HLAs\n",
+       "    def __init__(self, action, precond=None, effect=None, duration=0,\n",
+       "                 consume=None, use=None):\n",
+       "        """\n",
+       "        As opposed to actions, to define HLA, we have added constraints.\n",
+       "        duration holds the amount of time required to execute the task\n",
+       "        consumes holds a dictionary representing the resources the task consumes\n",
+       "        uses holds a dictionary representing the resources the task uses\n",
+       "        """\n",
+       "        precond = precond or [None]\n",
+       "        effect = effect or [None]\n",
+       "        super().__init__(action, precond, effect)\n",
+       "        self.duration = duration\n",
+       "        self.consumes = consume or {}\n",
+       "        self.uses = use or {}\n",
+       "        self.completed = False\n",
+       "        # self.priority = -1 #  must be assigned in relation to other HLAs\n",
+       "        # self.job_group = -1 #  must be assigned in relation to other HLAs\n",
        "\n",
        "    def do_action(self, job_order, available_resources, kb, args):\n",
        "        """\n",
@@ -5326,7 +3293,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 76,
+   "execution_count": 138,
    "metadata": {},
    "outputs": [
     {
@@ -5503,10 +3470,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 77,
-   "metadata": {
-    "collapsed": true
-   },
+   "execution_count": 139,
+   "metadata": {},
    "outputs": [],
    "source": [
     "jobShopProblem = job_shop_problem()"
@@ -5521,7 +3486,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 78,
+   "execution_count": 140,
    "metadata": {},
    "outputs": [
     {
@@ -5565,10 +3530,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 79,
-   "metadata": {
-    "collapsed": true
-   },
+   "execution_count": 141,
+   "metadata": {},
    "outputs": [],
    "source": [
     "solution = [jobShopProblem.jobs[1][0],\n",
@@ -5584,7 +3547,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 80,
+   "execution_count": 142,
    "metadata": {},
    "outputs": [
     {
@@ -5624,7 +3587,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 81,
+   "execution_count": 172,
    "metadata": {},
    "outputs": [
     {
@@ -5739,7 +3702,7 @@
        "    """\n",
        "\n",
        "    return PlanningProblem(init='At(A, LeftBaseLine) & At(B, RightNet) & Approaching(Ball, RightBaseLine) & Partner(A, B) & Partner(B, A)',\n",
-       "                             goals='Returned(Ball) & At(x, LeftNet) & At(y, RightNet)',\n",
+       "                             goals='Returned(Ball) & At(a, LeftNet) & At(a, RightNet)',\n",
        "                             actions=[Action('Hit(actor, Ball, loc)',\n",
        "                                             precond='Approaching(Ball, loc) & At(actor, loc)',\n",
        "                                             effect='Returned(Ball)'),\n",
@@ -5781,10 +3744,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 82,
-   "metadata": {
-    "collapsed": true
-   },
+   "execution_count": 173,
+   "metadata": {},
    "outputs": [],
    "source": [
     "doubleTennisProblem = double_tennis_problem()"
@@ -5799,7 +3760,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 83,
+   "execution_count": 174,
    "metadata": {},
    "outputs": [
     {
@@ -5811,7 +3772,7 @@
     }
    ],
    "source": [
-    "print(goal_test(doubleTennisProblem.goals, doubleTennisProblem.init))"
+    "print(doubleTennisProblem.goal_test())"
    ]
   },
   {
@@ -5842,10 +3803,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 84,
-   "metadata": {
-    "collapsed": true
-   },
+   "execution_count": 175,
+   "metadata": {},
    "outputs": [],
    "source": [
     "solution = [expr('Go(A, RightBaseLine, LeftBaseLine)'),\n",
@@ -5858,22 +3817,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 85,
+   "execution_count": 178,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "True"
+       "False"
       ]
      },
-     "execution_count": 85,
+     "execution_count": 178,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "goal_test(doubleTennisProblem.goals, doubleTennisProblem.init)"
+    "doubleTennisProblem.goal_test()"
    ]
   },
   {
diff --git a/planning.py b/planning.py
index 2913c2c2e..b92e6e276 100644
--- a/planning.py
+++ b/planning.py
@@ -1308,27 +1308,23 @@ def hierarchical_search(problem, hierarchy):
         The problem is a real-world problem defined by the problem class, and the hierarchy is
         a dictionary of HLA - refinements (see refinements generator for details)
         """
-        act = Node(problem.actions[0])
+        act = Node(problem.init, None, [problem.actions[0]])
         frontier = deque()
         frontier.append(act)
         while True:
             if not frontier:
                 return None
             plan = frontier.popleft()
-            print(plan.state.name)
-            hla = plan.state  # first_or_null(plan)
-            prefix = None
-            if plan.parent:
-                prefix = plan.parent.state.action  # prefix, suffix = subseq(plan.state, hla)
-            outcome = Problem.result(problem, prefix)
-            if hla is None:
+            (hla, index) = Problem.find_hla(plan, hierarchy) # finds the first non primitive hla in plan actions
+            prefix = plan.action[:index]
+            outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )
+            suffix = plan.action[index+1:]
+            if not hla: # hla is None and plan is primitive
                 if outcome.goal_test():
-                    return plan.path()
+                    return plan.action
             else:
-                print("else")
-                for sequence in Problem.refinements(hla, outcome, hierarchy):
-                    print("...")
-                    frontier.append(Node(plan.state, plan.parent, sequence))
+                for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements
+                    frontier.append(Node(outcome.init, plan, prefix + sequence+ suffix))
 
     def result(state, actions):
         """The outcome of applying an action to the current problem"""
diff --git a/tests/test_planning.py b/tests/test_planning.py
index 08e59ae2e..4d633e688 100644
--- a/tests/test_planning.py
+++ b/tests/test_planning.py
@@ -335,6 +335,33 @@ def test_refinements():
     assert(result[1][0].effect == taxi_SFO.effect)
 
 
+def test_hierarchical_search(): 
+
+    #test_1
+    prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO])
+
+    solution = Problem.hierarchical_search(prob_1, library_1)
+
+    assert( len(solution) == 2 )
+
+    assert(solution[0].name == drive_SFOLongTermParking.name)
+    assert(solution[0].args == drive_SFOLongTermParking.args) 
+
+    assert(solution[1].name == shuttle_SFO.name)
+    assert(solution[1].args == shuttle_SFO.args)
+    
+    #test_2
+    solution_2 = Problem.hierarchical_search(prob_1, library_2)
+
+    assert( len(solution_2) == 2 )
+
+    assert(solution_2[0].name == 'Bus')
+    assert(solution_2[0].args == (expr('Home'), expr('MetroStop'))) 
+
+    assert(solution_2[1].name == 'Metro1')
+    assert(solution_2[1].args == (expr('MetroStop'), expr('SFO')))
+
+
 def test_convert_angelic_HLA():
     """ 
     Converts angelic HLA's into expressions that correspond to their actions

From 24125e2625934cbae2ed4fb4b8f6800f627a95f7 Mon Sep 17 00:00:00 2001
From: Marianna 
Date: Tue, 24 Jul 2018 19:11:24 +0300
Subject: [PATCH 3/5] Created notebook planning_hierarchical_search.ipynb

---
 planning_hierarchical_search.ipynb | 546 +++++++++++++++++++++++++++++
 tests/test_planning.py             |   2 +-
 2 files changed, 547 insertions(+), 1 deletion(-)
 create mode 100644 planning_hierarchical_search.ipynb

diff --git a/planning_hierarchical_search.ipynb b/planning_hierarchical_search.ipynb
new file mode 100644
index 000000000..18e57b23b
--- /dev/null
+++ b/planning_hierarchical_search.ipynb
@@ -0,0 +1,546 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Hierarchical Search \n",
+    "\n",
+    "Hierarchical search is a a planning algorithm in high level of abstraction. 
\n",
+    "Instead of actions as in classical planning (chapter 10) (primitive actions) we now use high level actions (HLAs) (see planning.ipynb) 
\n",
+    "\n",
+    "## Refinements\n",
+    "\n",
+    "Each __HLA__ has one or more refinements into a sequence of actions, each of which may be an HLA or a primitive action (which has no refinements by definition).
\n",
+    "For example:\n",
+    "-  (a) the high level action \"Go to San Fransisco airport\" (Go(Home, SFO)), might have two possible refinements, \"Drive to San Fransisco airport\" and \"Taxi to San Fransisco airport\". \n",
+    "
\n",
+    "-  (b) A recursive refinement for navigation in the vacuum world would be: to get to a\n",
+    "destination, take a step, and then go to the destination.\n",
+    "
\n",
+    "![title](images/refinement.png)\n",
+    "
\n",
+    "-  __implementation__: An HLA refinement that contains only primitive actions is called an implementation of the HLA\n",
+    "-  An implementation of a high-level plan (a sequence of HLAs) is the concatenation of implementations of each HLA in the sequence\n",
+    "- A high-level plan __achieves the goal__ from a given state if at least one of its implementations achieves the goal from that state\n",
+    "
\n",
+    "\n",
+    "The refinements function input is: \n",
+    "-  __hla__: the HLA of which we want to compute its refinements\n",
+    "- __state__: the knoweledge base of the current problem (Problem.init)\n",
+    "- __library__: the hierarchy of the actions in the planning problem\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from planning import * \n",
+    "from notebook import psource"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "  Codestin Search App\n",
+       "  \n",
+       "  \n",
+       "\n",
+       "\n",
+       "
\n",
+       "\n",
+       "
    def refinements(hla, state, library):  # refinements may be (multiple) HLA themselves ...\n",
+       "        """\n",
+       "        state is a Problem, containing the current state kb\n",
+       "        library is a dictionary containing details for every possible refinement. eg:\n",
+       "        {\n",
+       "        'HLA': [\n",
+       "            'Go(Home, SFO)',\n",
+       "            'Go(Home, SFO)',\n",
+       "            'Drive(Home, SFOLongTermParking)',\n",
+       "            'Shuttle(SFOLongTermParking, SFO)',\n",
+       "            'Taxi(Home, SFO)'\n",
+       "            ],\n",
+       "        'steps': [\n",
+       "            ['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'],\n",
+       "            ['Taxi(Home, SFO)'],\n",
+       "            [],\n",
+       "            [],\n",
+       "            []\n",
+       "            ],\n",
+       "        # empty refinements indicate a primitive action\n",
+       "        'precond': [\n",
+       "            ['At(Home) & Have(Car)'],\n",
+       "            ['At(Home)'],\n",
+       "            ['At(Home) & Have(Car)'],\n",
+       "            ['At(SFOLongTermParking)'],\n",
+       "            ['At(Home)']\n",
+       "            ],\n",
+       "        'effect': [\n",
+       "            ['At(SFO) & ~At(Home)'],\n",
+       "            ['At(SFO) & ~At(Home)'],\n",
+       "            ['At(SFOLongTermParking) & ~At(Home)'],\n",
+       "            ['At(SFO) & ~At(SFOLongTermParking)'],\n",
+       "            ['At(SFO) & ~At(Home)']\n",
+       "            ]\n",
+       "        }\n",
+       "        """\n",
+       "        e = Expr(hla.name, hla.args)\n",
+       "        indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name]\n",
+       "        for i in indices:\n",
+       "            actions = []\n",
+       "            for j in range(len(library['steps'][i])):\n",
+       "                # find the index of the step [j]  of the HLA \n",
+       "                index_step = [k for k,x in enumerate(library['HLA']) if x == library['steps'][i][j]][0]\n",
+       "                precond = library['precond'][index_step][0] # preconditions of step [j]\n",
+       "                effect = library['effect'][index_step][0] # effect of step [j]\n",
+       "                actions.append(HLA(library['steps'][i][j], precond, effect))\n",
+       "            yield actions\n",
+       "
\n",
+       "\n",
+       "\n"
+      ],
+      "text/plain": [
+       ""
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "psource(Problem.refinements)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Hierarchical search \n",
+    "\n",
+    "Hierarchical search is a breadth-first implementation of hierarchical forward planning search in the space of refinements. (i.e. repeatedly choose an HLA in the current plan and replace it with one of its refinements, until the plan achieves the goal.) \n",
+    "\n",
+    "
\n",
+    "The algorithms input is: problem and hierarchy\n",
+    "-  __problem__: is of type Problem \n",
+    "-  __hierarchy__: is a dictionary consisting of all the actions and the order in which they are performed. \n",
+    "
\n",
+    "\n",
+    "In top level call, initialPlan contains [act] (i.e. is the action to be performed)   "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "  Codestin Search App\n",
+       "  \n",
+       "  \n",
+       "\n",
+       "\n",
+       "
\n",
+       "\n",
+       "
    def hierarchical_search(problem, hierarchy):\n",
+       "        """\n",
+       "        [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical\n",
+       "        Forward Planning Search'\n",
+       "        The problem is a real-world problem defined by the problem class, and the hierarchy is\n",
+       "        a dictionary of HLA - refinements (see refinements generator for details)\n",
+       "        """\n",
+       "        act = Node(problem.init, None, [problem.actions[0]])\n",
+       "        frontier = deque()\n",
+       "        frontier.append(act)\n",
+       "        while True:\n",
+       "            if not frontier:\n",
+       "                return None\n",
+       "            plan = frontier.popleft()\n",
+       "            (hla, index) = Problem.find_hla(plan, hierarchy) # finds the first non primitive hla in plan actions\n",
+       "            prefix = plan.action[:index]\n",
+       "            outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+       "            suffix = plan.action[index+1:]\n",
+       "            if not hla: # hla is None and plan is primitive\n",
+       "                if outcome.goal_test():\n",
+       "                    return plan.action\n",
+       "            else:\n",
+       "                for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+       "                    frontier.append(Node(outcome.init, plan, prefix + sequence+ suffix))\n",
+       "
\n",
+       "\n",
+       "\n"
+      ],
+      "text/plain": [
+       ""
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "psource(Problem.hierarchical_search)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example\n",
+    "\n",
+    "Suppose that somebody wants to get to the airport. \n",
+    "The possible ways to do so is either get a taxi, or drive to the airport. 
\n",
+    "Those two actions have some preconditions and some effects. \n",
+    "If you get the taxi, you need to have cash, whereas if you drive you need to have a car. 
\n",
+    "Thus we define the following hierarchy of possible actions.\n",
+    "\n",
+    "##### hierarchy"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "library = {\n",
+    "        'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', 'Taxi(Home, SFO)'],\n",
+    "        'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []],\n",
+    "        'precond': [['At(Home) & Have(Car)'], ['At(Home)'], ['At(Home) & Have(Car)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n",
+    "        'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(LongTermParking)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']] }\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "the possible actions are the following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "go_SFO = HLA('Go(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)')\n",
+    "taxi_SFO = HLA('Taxi(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home) & ~Have(Cash)')\n",
+    "drive_SFOLongTermParking = HLA('Drive(Home, SFOLongTermParking)', 'At(Home) & Have(Car)','At(SFOLongTermParking) & ~At(Home)' )\n",
+    "shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'At(SFOLongTermParking)', 'At(SFO) & ~At(LongTermParking)')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Suppose that (our preconditionds are that) we are Home and we have cash and car and  our goal is to get to SFO and maintain our cash, and our possible actions are the above. 
\n",
+    "##### Then our problem is: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prob = Problem('At(Home) & Have(Cash) & Have(Car)', 'At(SFO) & Have(Cash)', [go_SFO])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "##### Refinements\n",
+    "\n",
+    "The refinements of the action Go(Home, SFO), are defined as: 
\n",
+    "['Drive(Home,SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)']"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))]\n",
+      "[{'completed': False, 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'uses': {}, 'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'consumes': {}, 'precond': [At(Home), Have(Car)]}, {'completed': False, 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'consumes': {}, 'precond': [At(SFOLongTermParking)]}] \n",
+      "\n",
+      "[HLA(Taxi(Home, SFO))]\n",
+      "[{'completed': False, 'args': (Home, SFO), 'name': 'Taxi', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(Home), NotHave(Cash)], 'consumes': {}, 'precond': [At(Home)]}] \n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "for sequence in Problem.refinements(go_SFO, prob, library):\n",
+    "    print (sequence)\n",
+    "    print([x.__dict__ for x in sequence ], '\\n')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Run the hierarchical search\n",
+    "##### Top level call"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))] \n",
+      "\n",
+      "[{'completed': False, 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'uses': {}, 'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'consumes': {}, 'precond': [At(Home), Have(Car)]}, {'completed': False, 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'consumes': {}, 'precond': [At(SFOLongTermParking)]}]\n"
+     ]
+    }
+   ],
+   "source": [
+    "plan= Problem.hierarchical_search(prob, library)\n",
+    "print (plan, '\\n')\n",
+    "print ([x.__dict__ for x in plan])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example 2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "library_2 = {\n",
+    "        'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)' , 'Metro(MetroStop, SFO)', 'Metro1(MetroStop, SFO)', 'Metro2(MetroStop, SFO)'  ,'Taxi(Home, SFO)'],\n",
+    "        'steps': [['Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)'], ['Taxi(Home, SFO)'], [], ['Metro1(MetroStop, SFO)'], ['Metro2(MetroStop, SFO)'],[],[],[]],\n",
+    "        'precond': [['At(Home)'], ['At(Home)'], ['At(Home)'], ['At(MetroStop)'], ['At(MetroStop)'],['At(MetroStop)'], ['At(MetroStop)'] ,['At(Home) & Have(Cash)']],\n",
+    "        'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(MetroStop) & ~At(Home)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'] , ['At(SFO) & ~At(MetroStop)'] ,['At(SFO) & ~At(Home) & ~Have(Cash)']] \n",
+    "        }"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[HLA(Bus(Home, MetroStop)), HLA(Metro1(MetroStop, SFO))] \n",
+      "\n",
+      "[{'completed': False, 'args': (Home, MetroStop), 'name': 'Bus', 'uses': {}, 'duration': 0, 'effect': [At(MetroStop), NotAt(Home)], 'consumes': {}, 'precond': [At(Home)]}, {'completed': False, 'args': (MetroStop, SFO), 'name': 'Metro1', 'uses': {}, 'duration': 0, 'effect': [At(SFO), NotAt(MetroStop)], 'consumes': {}, 'precond': [At(MetroStop)]}]\n"
+     ]
+    }
+   ],
+   "source": [
+    "plan_2 = Problem.hierarchical_search(prob, library_2)\n",
+    "print(plan_2, '\\n')\n",
+    "print([x.__dict__ for x in plan_2])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/tests/test_planning.py b/tests/test_planning.py
index 4d633e688..a1e2bad60 100644
--- a/tests/test_planning.py
+++ b/tests/test_planning.py
@@ -338,7 +338,7 @@ def test_refinements():
 def test_hierarchical_search(): 
 
     #test_1
-    prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO])
+    prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO])
 
     solution = Problem.hierarchical_search(prob_1, library_1)
 

From 89b4589fcdb2d9659dfa5a9810b7a2b3ae61307a Mon Sep 17 00:00:00 2001
From: Marianna 
Date: Thu, 26 Jul 2018 12:52:48 +0300
Subject: [PATCH 4/5] Added making progress and tests, minor changes in
 decompose

---
 planning.py                   |  29 +--
 planning_angelic_search.ipynb | 362 ++++++++++++++++++++++++++++++++--
 tests/test_planning.py        |  20 +-
 3 files changed, 376 insertions(+), 35 deletions(-)

diff --git a/planning.py b/planning.py
index b92e6e276..cb2f53307 100644
--- a/planning.py
+++ b/planning.py
@@ -1361,12 +1361,12 @@ def angelic_search(problem, hierarchy, initialPlan):
                 if Problem.is_primitive( plan, hierarchy ): 
                     return ([x for x in plan.action])
                 guaranteed = problem.intersects_goal(pes_reachable_set) 
-                if guaranteed and Problem.making_progress(plan, plan):
+                if guaranteed and Problem.making_progress(plan, initialPlan):
                     final_state = guaranteed[0] # any element of guaranteed 
                     #print('decompose')
                     return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set)
                 (hla, index) = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive.
-                prefix = plan.action[:index-1]
+                prefix = plan.action[:index]
                 suffix = plan.action[index+1:]
                 outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )
                 for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements
@@ -1446,30 +1446,33 @@ def find_hla(plan, hierarchy):
 	
     def making_progress(plan, initialPlan):
         """ 
-        Not correct
+        Prevents from infinite regression of refinements  
 
-        Normally should from infinite regression of refinements 
-        
-        Only case covered: when plan contains one action (then there is no regression to be done)  
+        (infinite regression of refinements happens when the algorithm finds a plan that 
+        its pessimistic reachable set intersects the goal inside a call to decompose on the same plan, in the same circumstances)  
         """
-        if (len(plan.action)==1):
-            return False
+        for i in range(len(initialPlan)):
+            if (plan == initialPlan[i]):
+                return False
         return True 
 
     def decompose(hierarchy, s_0, plan, s_f, reachable_set):
         solution = [] 
+        i = max(reachable_set.keys())
         while plan.action_pes: 
             action = plan.action_pes.pop()
-            i = max(reachable_set.keys())
             if (i==0): 
                 return solution
             s_i = Problem.find_previous_state(s_f, reachable_set,i, action) 
             problem = Problem(s_i, s_f , plan.action)
-            j=0
-            for x in Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])]):
-                solution.insert(j,x)
-                j+=1
+            angelic_call = Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])])
+            if angelic_call:
+                for x in angelic_call: 
+                    solution.insert(0,x)
+            else: 
+                return None
             s_f = s_i
+            i-=1
         return solution
 
 
diff --git a/planning_angelic_search.ipynb b/planning_angelic_search.ipynb
index 20400cd49..7d42fbae3 100644
--- a/planning_angelic_search.ipynb
+++ b/planning_angelic_search.ipynb
@@ -17,11 +17,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 66,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from planning import * "
+    "from planning import * \n",
+    "from notebook import psource"
    ]
   },
   {
@@ -47,6 +48,153 @@
     "  \n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "  Codestin Search App\n",
+       "  \n",
+       "  \n",
+       "\n",
+       "\n",
+       "
\n",
+       "\n",
+       "
    def angelic_search(problem, hierarchy, initialPlan):\n",
+       "        """\n",
+       "\t[Figure 11.8] A hierarchical planning algorithm that uses angelic semantics to identify and\n",
+       "\tcommit to high-level plans that work while avoiding high-level plans that don’t. \n",
+       "\tThe predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression\n",
+       "\tof refinements. \n",
+       "\tAt top level, call ANGELIC -SEARCH with [Act ] as the initialPlan .\n",
+       "\n",
+       "        initialPlan contains a sequence of HLA's with angelic semantics \n",
+       "\n",
+       "        The possible effects of an angelic HLA in initialPlan are : \n",
+       "        ~ : effect remove\n",
+       "        $+: effect possibly add\n",
+       "        $-: effect possibly remove\n",
+       "        $$: possibly add or remove\n",
+       "\t"""\n",
+       "        frontier = deque(initialPlan)\n",
+       "        while True: \n",
+       "            if not frontier:\n",
+       "                return None\n",
+       "            plan = frontier.popleft() # sequence of HLA/Angelic HLA's \n",
+       "            opt_reachable_set = Problem.reach_opt(problem.init, plan)\n",
+       "            pes_reachable_set = Problem.reach_pes(problem.init, plan)\n",
+       "            if problem.intersects_goal(opt_reachable_set): \n",
+       "                if Problem.is_primitive( plan, hierarchy ): \n",
+       "                    return ([x for x in plan.action])\n",
+       "                guaranteed = problem.intersects_goal(pes_reachable_set) \n",
+       "                if guaranteed and Problem.making_progress(plan, initialPlan):\n",
+       "                    final_state = guaranteed[0] # any element of guaranteed \n",
+       "                    #print('decompose')\n",
+       "                    return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set)\n",
+       "                (hla, index) = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive.\n",
+       "                prefix = plan.action[:index]\n",
+       "                suffix = plan.action[index+1:]\n",
+       "                outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions )\n",
+       "                for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements\n",
+       "                    frontier.append(Angelic_Node(outcome.init, plan, prefix + sequence+ suffix, prefix+sequence+suffix))\n",
+       "
\n",
+       "\n",
+       "\n"
+      ],
+      "text/plain": [
+       ""
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "psource(Problem.angelic_search)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -59,6 +207,134 @@
     "   \n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "  Codestin Search App\n",
+       "  \n",
+       "  \n",
+       "\n",
+       "\n",
+       "
\n",
+       "\n",
+       "
    def decompose(hierarchy, s_0, plan, s_f, reachable_set):\n",
+       "        solution = [] \n",
+       "        i = max(reachable_set.keys())\n",
+       "        while plan.action_pes: \n",
+       "            action = plan.action_pes.pop()\n",
+       "            if (i==0): \n",
+       "                return solution\n",
+       "            s_i = Problem.find_previous_state(s_f, reachable_set,i, action) \n",
+       "            problem = Problem(s_i, s_f , plan.action)\n",
+       "            angelic_call = Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])])\n",
+       "            if angelic_call:\n",
+       "                for x in angelic_call: \n",
+       "                    solution.insert(0,x)\n",
+       "            else: \n",
+       "                return None\n",
+       "            s_f = s_i\n",
+       "            i-=1\n",
+       "        return solution\n",
+       "
\n",
+       "\n",
+       "\n"
+      ],
+      "text/plain": [
+       ""
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "psource(Problem.decompose)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -76,7 +352,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 67,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -98,7 +374,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 68,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -118,7 +394,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 69,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -140,7 +416,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 70,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -160,7 +436,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 71,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
@@ -189,7 +465,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 72,
+   "execution_count": 9,
    "metadata": {},
    "outputs": [
     {
@@ -197,10 +473,10 @@
      "output_type": "stream",
      "text": [
       "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))]\n",
-      "[{'consumes': {}, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'uses': {}, 'completed': False, 'precond': [At(Home), Have(Car)], 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'duration': 0}, {'consumes': {}, 'effect': [At(SFO), NotAt(LongTermParking)], 'uses': {}, 'completed': False, 'precond': [At(SFOLongTermParking)], 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'duration': 0}] \n",
+      "[{'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'args': (Home, SFOLongTermParking), 'uses': {}, 'consumes': {}, 'name': 'Drive', 'completed': False, 'precond': [At(Home), Have(Car)]}, {'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'args': (SFOLongTermParking, SFO), 'uses': {}, 'consumes': {}, 'name': 'Shuttle', 'completed': False, 'precond': [At(SFOLongTermParking)]}] \n",
       "\n",
       "[HLA(Taxi(Home, SFO))]\n",
-      "[{'consumes': {}, 'effect': [At(SFO), NotAt(Home), NotHave(Cash)], 'uses': {}, 'completed': False, 'precond': [At(Home)], 'args': (Home, SFO), 'name': 'Taxi', 'duration': 0}] \n",
+      "[{'duration': 0, 'effect': [At(SFO), NotAt(Home), NotHave(Cash)], 'args': (Home, SFO), 'uses': {}, 'consumes': {}, 'name': 'Taxi', 'completed': False, 'precond': [At(Home)]}] \n",
       "\n"
      ]
     }
@@ -221,7 +497,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 73,
+   "execution_count": 10,
    "metadata": {},
    "outputs": [
     {
@@ -230,7 +506,7 @@
      "text": [
       "[HLA(Drive(Home, SFOLongTermParking)), HLA(Shuttle(SFOLongTermParking, SFO))] \n",
       "\n",
-      "[{'consumes': {}, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'uses': {}, 'completed': False, 'precond': [At(Home), Have(Car)], 'args': (Home, SFOLongTermParking), 'name': 'Drive', 'duration': 0}, {'consumes': {}, 'effect': [At(SFO), NotAt(LongTermParking)], 'uses': {}, 'completed': False, 'precond': [At(SFOLongTermParking)], 'args': (SFOLongTermParking, SFO), 'name': 'Shuttle', 'duration': 0}]\n"
+      "[{'duration': 0, 'effect': [At(SFOLongTermParking), NotAt(Home)], 'args': (Home, SFOLongTermParking), 'uses': {}, 'consumes': {}, 'name': 'Drive', 'completed': False, 'precond': [At(Home), Have(Car)]}, {'duration': 0, 'effect': [At(SFO), NotAt(LongTermParking)], 'args': (SFOLongTermParking, SFO), 'uses': {}, 'consumes': {}, 'name': 'Shuttle', 'completed': False, 'precond': [At(SFOLongTermParking)]}]\n"
      ]
     }
    ],
@@ -249,7 +525,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 74,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -263,7 +539,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 75,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [
     {
@@ -272,7 +548,7 @@
      "text": [
       "[HLA(Bus(Home, MetroStop)), HLA(Metro1(MetroStop, SFO))] \n",
       "\n",
-      "[{'consumes': {}, 'effect': [At(MetroStop), NotAt(Home)], 'uses': {}, 'completed': False, 'precond': [At(Home)], 'args': (Home, MetroStop), 'name': 'Bus', 'duration': 0}, {'consumes': {}, 'effect': [At(SFO), NotAt(MetroStop)], 'uses': {}, 'completed': False, 'precond': [At(MetroStop)], 'args': (MetroStop, SFO), 'name': 'Metro1', 'duration': 0}]\n"
+      "[{'duration': 0, 'effect': [At(MetroStop), NotAt(Home)], 'args': (Home, MetroStop), 'uses': {}, 'consumes': {}, 'name': 'Bus', 'completed': False, 'precond': [At(Home)]}, {'duration': 0, 'effect': [At(SFO), NotAt(MetroStop)], 'args': (MetroStop, SFO), 'uses': {}, 'consumes': {}, 'name': 'Metro1', 'completed': False, 'precond': [At(MetroStop)]}]\n"
      ]
     }
    ],
@@ -281,6 +557,62 @@
     "print(plan_2, '\\n')\n",
     "print([x.__dict__ for x in plan_2])"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example 3 \n",
+    "\n",
+    "Sometimes there is no plan that achieves the goal!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "library_3 = {\n",
+    "        'HLA': ['Shuttle(SFOLongTermParking, SFO)', 'Go(Home, SFOLongTermParking)', 'Taxi(Home, SFOLongTermParking)', 'Drive(Home, SFOLongTermParking)', 'Drive(SFOLongTermParking, Home)', 'Get(Cash)', 'Go(Home, ATM)'],\n",
+    "        'steps': [['Get(Cash)', 'Go(Home, SFOLongTermParking)'], ['Taxi(Home, SFOLongTermParking)'], [], [], [], ['Drive(SFOLongTermParking, Home)', 'Go(Home, ATM)'], []],\n",
+    "        'precond': [['At(SFOLongTermParking)'], ['At(Home)'], ['At(Home) & Have(Cash)'], ['At(Home)'],  ['At(SFOLongTermParking)'], ['At(SFOLongTermParking)'], ['At(Home)']],\n",
+    "        'effect': [['At(SFO)'], ['At(SFO)'], ['At(SFOLongTermParking) & ~Have(Cash)'], ['At(SFOLongTermParking)'] ,['At(Home) & ~At(SFOLongTermParking)'], ['At(Home) & Have(Cash)'], ['Have(Cash)'] ]\n",
+    "        }\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'Have(Cash) & At(SFOLongTermParking)', 'At(SFO)')\n",
+    "prob_3 = Problem('At(SFOLongTermParking) & Have(Cash)', 'At(SFO) & Have(Cash)', [shuttle_SFO])\n",
+    "# optimistic/pessimistic descriptions\n",
+    "angelic_opt_description = Angelic_HLA('Shuttle(SFOLongTermParking, SFO)', precond = 'At(SFOLongTermParking)', effect ='$+At(SFO) & $-At(SFOLongTermParking)' ) \n",
+    "angelic_pes_description = Angelic_HLA('Shuttle(SFOLongTermParking, SFO)', precond = 'At(SFOLongTermParking)', effect ='$+At(SFO) & ~At(SFOLongTermParking)' ) \n",
+    "# initial Plan\n",
+    "initialPlan_3 = [Angelic_Node(prob.init, None, [angelic_opt_description], [angelic_pes_description])] "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "plan_3 = prob_3.angelic_search(library_3, initialPlan_3)\n",
+    "print(plan_3)"
+   ]
   }
  ],
  "metadata": {
diff --git a/tests/test_planning.py b/tests/test_planning.py
index a1e2bad60..3223fcc61 100644
--- a/tests/test_planning.py
+++ b/tests/test_planning.py
@@ -312,6 +312,11 @@ def test_job_shop_problem():
 plan2 = Angelic_Node('At(Home)', None, [taxi_SFO])
 plan3 = Angelic_Node('At(Home)', None, [drive_SFOLongTermParking, shuttle_SFO])
 
+# Problems
+prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO])
+
+initialPlan = [Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description])] 
+
 
 def test_refinements():
     
@@ -467,19 +472,19 @@ def test_making_progress():
     """
     function not yet implemented
     """
-    assert(True)
+    
+    intialPlan_1 = [Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description]), 
+            Angelic_Node(prob_1.init, None, [angelic_pes_description], [angelic_pes_description]) ]
+
+    plan_1 = Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description])
+
+    assert(not Problem.making_progress(plan_1, initialPlan))
 
 def test_angelic_search(): 
     """
     Test angelic search for problem, hierarchy, initialPlan
     """
     #test_1
-    prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO])
-
-    angelic_opt_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & $-At(Home)' ) 
-    angelic_pes_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & ~At(Home)' ) 
-
-    initialPlan = [Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description])] 
     solution = Problem.angelic_search(prob_1, library_1, initialPlan)
 
     assert( len(solution) == 2 )
@@ -490,6 +495,7 @@ def test_angelic_search():
     assert(solution[1].name == shuttle_SFO.name)
     assert(solution[1].args == shuttle_SFO.args)
     
+
     #test_2
     solution_2 = Problem.angelic_search(prob_1, library_2, initialPlan)
 

From 44d650328ef11f37dead295679e30d41d10c4eac Mon Sep 17 00:00:00 2001
From: MariannaSpyrakou 
Date: Thu, 26 Jul 2018 13:00:16 +0300
Subject: [PATCH 5/5] image for planning_hierarchical_search.ipynb

---
 images/refinement.png | Bin 0 -> 63029 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 images/refinement.png

diff --git a/images/refinement.png b/images/refinement.png
new file mode 100644
index 0000000000000000000000000000000000000000..8270d81d0e322eca1a29dc59af4c358f19404356
GIT binary patch
literal 63029
zcmc$`cRbg9-#`38AyHPMlr7n_Xi&&bh{#SzRAxpQ$%urIWMw6*$jpi)S=p;nlu?Mt
zDpK9gZ=L6PU)OnE*Y9!P*Zs%skE7$Lj_>y~-tX7zxnA#39c@)Qnr$=$LC~qIDeDmg
zc`!kcrBjpRca{#`jKMDyu8QjV)YR02ACA4pf3mq9F>*WZXzS*2%Eg8_bHVYvjfm@M
z7aN-ku6B-YvlO-R1i?wDD<9VPOqx!1Fg@PbKSxCo5?mExd`nPSod3wxGC^}8)#?%&
z`v#LEIgLWb=NnE4^&Pi6ctj^XcAiqzt)w3GpLw9-nH#$=>_vlJ&RV{YIV
z)|Z@zw-Y)hCO=xmYH3UO|NOnRQE>^0mBpT8y2jfHPR3B0jT=98q$u3fI7RxLg^tnX
znZXMeE>u=l67kj*rrK#bZ{NP9G?w`D$-hnpUc2Vu?#>-^!>9T0*U~&ve}S)}wbi+D
zRPxWa2aunOmLUBh8}mP}Bb7+B{__Ep>tFGnCVi2Vz*BxU(zV_>Ot$C$)kj&1Csd<&xuoNEZ%
z`taeyF|7(y-|?0$)TdH%Lh$8|8{Yin{WjT?b@=dMzqPN|l9Q8zf{rLD1w=%&e@gMY
zaQ19Y#Yl!~BTe0xu+tBA-d_Mx4nwrv-u5Dvu%Ne>#e|Nvf
zmtQ}B?Afztwt68#jrUEFO;}`P$9QYpWx?X?YyoNMTaBYf?}*nmH#fg~cW~dnef<0?
zyo_ykBu>`{)4995`@AhQj7mxI`|xc2{{8z4jUGIFn0)u{&MjNk6HRSxbelK(ElzaZ
zQ;&HOv120_b6A6ym)G{a7uKeWJ9Nw{OxtaX6B85TweIT(j4iDA`uf`1+7bj!&@*dE
zG4pCQX1u05el+yYl9H0%Re99c@JmS@*VNP$X31!sn3#|_{i3(O-#$BCN8sGKbL+^-
zD&5@OwT~T}b!{-Gic%Y!n$j@s@9p*U^vo979i?{Sq4CAl#j`<*{w~k03ghEndkp(z
zRfOhS-_OdTAOaP+qKbWfEC+4ex?H~5r*7d>N1UkHo1viyQM|Z;hbF3TeZJ3e5vfH*
z(M=m@Xsn7~3y4IiVWmY!M~77D>gsOXD#}Yqqv$!-6qTNy{%eMUw8Y7il9Lk?5_Bp#
zEKFT1Oj*K$Zp#x_&i337Rpz!ftmNLcOI20%ns!!3>-X>9*`#b2Q-Ysxa&eKZPf&UE
zdirSMo*b5#)^W`^(d2y>r|urSTvAe^b3Z;lJ|kmvZDm1bZ&$3AXk4qMxUjHr2>YgM
zqtnxG!_2GYYieqC?%avjH4)ux?fd=1wdZD{ahh2@!(N|ua9?Y-Ha0P#AolIwkJW>_
z)DWE=8Hv7g=T36+(Xn&yD*GB6vxckuvDyMceWjb(H=P-(_Ihq(Z$I_uRK1b#?O?&l
z$w}9bFDDgZ%{ew*Q!2~Zy?eLor*;Do_o?2T)zwR4Vq%%z8#jhwnL1oZ7mC+7YG`P<
zb<36~-A}(UojiGR$BrG(GozxqMjFCGHRpRjjg1jR$nTQ&B)OiUp)X&)Xy3k{m*-$<
zc{46fb?nBD{bfZ(MNghQvAF!_ny&wv=W89G9~&EcX!1C{wYyu-&+m6rlUhlWkcbG~
z#*K-x&R7zQ^Yh(3JpuH0<$YGOQhc{5_^y2$_S;Md2naa2xxFnu6R9sqM@^lblOyLj
z_a2|5b@b?QJ-zn32Q|}ls6SYrJ%9du_F5E=>d5G5j=sI3;1Pl$<^8>|E&YPgY3H@A-8k=<(ynwY3UeG{n@RIb0`D(9)`){0i9z+FyWuOTX
zX8HWO_zeBC2M>g)sHoZ`f3GbZBc6V!zvlD%YJ-)}VwY;d`kHnfJDr`10#RzFW@b@x
z9xg8SXU}q7;8LBBv+%8O7!Z|`D!y}vH8UzFH&-wE#Rn&X&>6w55gSiC9}l2Zx*s@9NB583&e>j>o7)rKRf$Yg=1M)*Gkl
zgD>*Y#rl1F=h5ZM?SIJ9)^?BAyftq>Dr9f3zN=a3d9p2z`HE>eKJ~Q375Z*F>3NZD
z;V;Hr%uG#b>F9PdU)LAJ4Vl&Y9;{4^i`(nF=7}9hCmGAWpb;^azYc4{O{RFR;D`wE
z^yyQ}*`X(zw}o@^@_6zz1j?+NqZxQq=pP#!8;AIjQ(H^EjukSD*s)(_XfG9!sw3do
zdTkOdyl2;9tKdn|xU75k=p(Gtbc~K4&nPHBO%M{w&=DAX?6~Vd-MZJWUk9FH{`sKR
zdj&=3-P^ZGYeF1MOrsw^p0KbGi)+QUGCpfffh!Mn!*ieN%iF$fTS7ttFQdBQ#lA8b
zmoc?X*A|urEGP&p$pdG*d0kx>@bde@xPfLQ5pXQcL%E`K+BC=d$+3
zb>zg;rYN4zQ=}g}{LpA;XXhhFjwp*@OWykU)l=$Bd$rpnqjA1-g{c*VuD5q}SXe7i
z!L`jCB!qEdTg8EE^;`oNFJ24}3F++V`K)c`VzRWfgfA^AEL7Lf$j!|qCpzs)&UD<(
zarbLWkZB5Gl*8-Y>o~;wGA%PR^X}cdifIJF%*>2BzV=Ds_u{(_C;^r*GJ=+}zNsm@
zvU2rhyp$GyPh;b?Hrri%eCyY*KkML-VfdUNc1AvS`k;8r9K{bn7)~{G;gO$1c
z1yvQcZQEvE>-#gWpr#U^e&NCwAVTdFd7s4z-CetO=|6q;%xbvC2au?CZC+DHM@LVO
z@f1U>wIX#Shl20>r(`}pKi|K%j8a=1YmQO6wyEL8i|Y*|H*elFr14HpYieoP=R6Xq
z#>@RJqH(m#uH@;{L-HwpOH&343bk`{P9h>A)M+UxDRFUIV>N7wUvJ)dK*whfFa$Rb
z&%uKSrKF^AC6}gO6#|qzf393KoXn}9;8%Bby}-j0OhCmeRb)SA7DXlTLce`^$D=(T
zt&wS9U@+E`kdT;oR(hM9sxV7T#}n@PaThN$vmGbPbERFpT#VCn1a@%TugKRkL95U>
zIy+ojo43#aEP^k@e$LYmO3lyDucg&`=g0EQ_BF}NM~#eFS1(P`xdNY}kkzGb09;!Xa-2BSc7dCj1`nBi`}WT>
zGY2~>7g+&i;-${;^YYTV#6QWnckg*ii$-7T;4t;Ns}T{Ko+%u>l0f1viT7%2G{4A8
z*}k0n_KgFZ58q8?95?j-<3|=27XDa`KHp;pJgt|>aT`_vecP2p7RiXl$4b%2bkaa!X4LIU|KXPvIVmi;Ck##JEa{c40vU{ZGgZcq9
z0cH(t?e;^vmFBQJapi(GAT~e`461%j)8AF+3q*+vd%i
z`_>j0-H0nX0=zC|j>9!hAD)>@e_cJ`_se^-J9B1cW}OABHWZm
zxueP}Dx&HoRgN4HF)hF8O}=4DcwhQG1zER=&JKm&H;ms_UKC=Ok8T`2KV0)f!C3#&
zO=WI|SdF7ckIn=MI6BUvg6y`bY3HHhI)L^CGBZ16C4T4bJpWZG_Y_nWa8W5%TH@)K
zcU2A!4nc}(PoDgIa`{`QZnyQ2-)|qv0B1brsVRFG5RYHGQoLP$|MCHfCapQ&Pstwa
zJ9g-r?gr@LdG+?~0ladon@r8PWkmzgxX+Ju$Lg+lluhBSr2rZ>FcG@vs>g`H$T_clPXSjus_qROt{-V^oL6h#dmw
z&z-xW7_^M08`ogPu(^ApE8`m7_S?5^-DAF6l$frsFyWW1x`}vn@pFFw)n*lCcYkd>25
zKHw$E&u`n04F=$cDuL2M5DOpM7QTHurlpl6=W6Th{55a`Q%KBhh1Ch3B
zR0Zfot|LE8+KQ7smj$yjGx_%Kuj+_p-|v17DDv{<%fr_jEX50~-y1fgs+v@}KYa4U
z$Ifmd*C6lZ%a=Nyl$$q4+O*y#BZBF+V*}VaIx5(-Fpo*~pqT94`?##^uBS}3enVP=ElC)SE
z&odQ0rma0x4?ckuGqe-%W@|V24(i(jJP&)ryE6~=Q
zoE(#~b6hU6>=9sj40(n3@AJ#b-ciZeuDj=|8|9aY!?yJ{PxLZkF#eduWu2WaNFnA9A+hRSHW*5p#yNkz))vZcyj|o
zSi`XI50S*S6H}?6+!~pI;i}07pWpO$wqf}VRbFiE?Ue=voPJ##s3{{RHVbA{SXz4C
z&aSho>ouq<%3}z9JNu@_rlyd&=NW-hcG>NvrlPZ03m@mwsY0n$cE`2)czeHn_b&8~
z5V`;RmRKQ_B9{u&G@WyUWk+XvND9izpsW@)MqKN=qB75P*B7hPQ5;LT|)tav|Ll5rfW{`uou
z#v=IFDw>EA^%k1-$=ZIiPrgCH34D~6=6{okfuYd8Pq6RRhTIx!bPek!dgl9~oWjCd
znwoiKWdJG<($lQ~M*t$}pJ5k~I6iPV@ZqrcLK`3$XCP`A{?o!|DK;gABT!sU?saqX
z21O%NQ?0BXUE1KAH|d`hURwBgv~*z}vKcwCi9>$+$5vsMu(>NVL5eJ4OVh8Vs85|Z
zaXTP@qMIq_v2EKnH$CaF1{V*H+3D#(daT#^6>t3~LN?Nze`#v7^Z!6oM_7t_fQ~e}
zW!X0oSM0W4EW3YSCFjn^3zbz>8KiPB-;;GrHx?8DTPjDNhHGss=2*?@ln`J1VA*+-
z8gDr{)fv3KrCYm?SISk>D=uoR)c*cCTjBZ%ujIhNgZ1_Gy6V)YbalHf&px@fQ)=th
zt@rM|$=awmF*5_v>chQz_io&{fqy3N_Ygej@?_`D(ZYIeKEPIfm1bE@j$ZMip
zfwYvPC~blCE!IV1wQG!uviL|KiNgZSD4$FLe%tmjHPc)1K{`%)#nMx<#06|b=H%fZ1aAwCIKEe^>LEy+tq)J-X=
zs3-_@sw1KqUa=2!1b~{F_+R(+$?V&=3PPmu@m_K90!lHISG<|mLP|8b31g88zwQ|M
z@qouIE-i%~x(;m+3{t~IQc@C~PD|R*$Ve7c+Rf|tub(E5FF=3N6}a{O{d*LbhxhIU
zJ}|Pg%YKv6fNC0Zjrs^f`^%Ss^flfqjD(J<>5U5w&z~EeJjpNx@%XYpGFZ9ui1MxW
zwzks3!oz3jTC$I%=^XH!yOLCF)j%_1xKpv*x|xyt=|EfJ-ex|9AJbA;(0u-236zwS
zK;S$cEFyFpynTFRJYt%y)7>;QT1w9L=v3zCZz4c&$hpv%y9Ne!)e8I5pEz-X9Gc(<
z>B!(IQ;##A3>;WwsOYKtnl#cM_XI}gitNF?aE0mJ=;-v4<_WBGl&r}cY*L+>Z_L#2H{3lu(AOUp6j1riTvQzD
zQH|V*J1=&D!Q3c0S`S0SoE39+hYCa$!Oga2`>79HYD)yt9I
zM)&S|XecN+S1*q}7&kZ8@smqSQ^CQMQEK-Nvz0Qlu-IB#-*iRMk^lalLeXG8Rm7qu
z$+Ww_pJk?spXm?j-=H8gCG4f&)7?D;mACC3Z%gCz=fH)10|Q|VBaM+9fII0uy&Ur1
zO_t)&nVOD5kS^i@TskGK|EY3+{l;IHZZ`F(@{5Ueyn1!#!5AulriMmuSqOVGct1A9
zuMgKa2;$h?17uS>(E*yEJTwn}lmPKLb>(PLY3b6?lglXzei5RXdS{OxXY#5QC5WH1
zsrpe#N@+Tfwa~Z?gMM_O7r4(0*?T|uwZu3cG$h0e>0ET2cah8V7!qfZ>dh6?z^VQP{UbVg7@
z!Wnw3=KU-Uxbs;rJRekvhwr(hbA5gH$qpt}S$+-|m18SQ(p~yaT$}%*?}6M?B}pn)3_`rY0xXQ&QSHI6&_o=S~ex{~LH8s9YPILKYRZ#cf(pa574Py2VU0zj%hZX$RW?MK5
zD=Rz_4(Ox}5P+^-%dl8Mha9fF7^aa~^!nVHGiT=aoC}1{W^H{hE9=u{g*8hnE5&&cyo()~|O-Ia3rTP|iT`Kp1n9@edRd>>M5Anys-MpwYd1TX1iAQM9dh
zU?9&HG&|E9`u53_Ib-*63qUlvHG_b1Of^$qjvE?g<{^C01b0P~oc;kH}Q0RPA~
zsZ6E#^{;3%r!ibCNJmG>c3)Z02nO%-C!+
zB|#7K^CM)Lj+P#AiBRU|+Vgxjzzge5JRXC*6H%?%+6O_vgebd~^ABr^X}S1uGShsU
z8Qj}xyyo}p*|VEDo}YCDnA6hICenUGjy23TmFwI_C!RnxhJ}ZHIF|I-V@l#qUB~-<
zI!qa4{}OrVKdI3&^YQVab_d3Nn@O*!85$fMtgB0_u3lSR`iu{xc-Tc`oH_(@q^e3)
z$VF6-ACIi7HhZriwPbhp?7JGD%i`p
zw$Jg?+X`2Qf&7DB^Y^6gt8M&e7Qp^u-!oR)j~Acr&%dnV>-!6P4cr-)DJ%(8PJMlS
zW8*g@?=&MLz)H17{v9*09ZBfn;mI(xr~W_rzPySY|BYJ1v;2#B%N96N^PbfpF1S>z
zj)7ON%8#vT-hcFH?x{ZoiuT8kA3-`9L!ZN@;f{hDcq~Q!XQE@xN`F(a<8GG93wHSyGD#-mM8*jUK;Z`lu#d;bVegnt1%nu&rW@d>$yk@D3h5
z5;s_{zY#TbfP7>&x~lE3`>T;O^#1ebstfP--CAB={?aA!lRK)Zug^fKU1APXJvP;O-)KtzriQ(P2zP7e-_HYS7LHiHSg2s+Z!h{3?54`C+0ssLXFJtG%VX{g}yo|sz__^DnyWa+=3DqB}#5Bc`-(o-a4S*eUk>6FMMx*Pa
zJEk
zik32i>|FHaLnltKtVr3w2xI2T_#-F`c6Am0Nbe11ihR3uMDP$#H%2^s@ZcOIuL%Bwg4jmVSA3ceLPYv9^rWh!
z#I51_7Po!LM%MQ-Gf$eBsH&(y#wS_4SX=B8R)LCX<(^-y;DF@ieFk(T2$XR-z-x8&
zX2@aKbGCMNo{;Kb+`}6d7)Q5&B*37V2jSt-qeoz-AeV5*4<3AitEHfz=+j{{J4^kG
zCf4)Unip}1mkoT``;GMPfv{(
zDzMIafW&;S5%7ucJd4}n`~J30D-%O8M;
zU4((}R&e^mha#MuskyoK;G0l51YXR2Y)jOw!is~PlBJh}P6eV=<@#xp1`HymFin8s
zq$I7tfLpEO^78WVekaPM{SP@Irvd@A!Zcq;08l-6UuN$JAfaxyvS4gNQc@Behs4Co
z*mD0-iD}E|>btTya&|hVLS^eOhUYb77kK`#>#l-|m>6U~Hnx0Q
zUWQRU@!l8v+rsbX7DZO{VI!mOeTEi0AlfUET0P9pO*Eo^{pZ)JdAER3ezxr$PIYs3
zmPpoCt)oy>RJ{3n7B^|1^+1U3gGGA-o{`Hyz8O?12?>ekB8!7&1_qhD;q@N~f;w_E
z#qZY!0#O0o^M(Su9#aoI#CsChGis)%TB>3J@UrzsYe+{K3oa&@(mm%gUy{
zI?BF@N)e0!AIuqOSYXcQ?r2u&c6R1Xsdc=D9M)Kk$LvlGfXCn0IDF}8g3zP(yH6jA
zdH@@s;KMFG10|Bx#DT{?37~efp2_Rdr3h9DMT2&17chbldLet(=Ms4~z;&
z6n?L4)_{hwnN4z?NY4I!``}Q*kD=v)`L+Yq#?@6S(CHsUoQg!m-&FpSi03N#C*VrC
zXHRp#IYN2tKgj@c+w=|{x&prT=9DQkm9(e0-jG}RgWQ2mU>xS=<_C60Ho;}239u)M4?X_#Q`f=brIGq;?-lj66r${)jinFMBzxJRt!<*vZn0CnxUgbu`B^!IfWfIO=X>@6OBP-xLKkEh#$M>fAYmgwCHi
zp)ii?p=e@N+XpNAKz>#4)GPnH3tU=3~d+Ukb{FzWaIIRbteoAqINW!
z(XF>1dFL_AhbJc4b%Fn1#yxS!*S7||wJ>Jmnnj9G{Oal|B!jkwhV>*-|8{(QDCyB9
zNS;1D3PJ`jnD)|Z;bR(KIrK7J-HXyXCI#&Dl8%=7WCYR?qHzv70a}qX=&<=u`72ER
zjnEQ@jExhXJUO`U?jrUiI&*(l7nlJV!Mkf$N8;YdR|f!-*HAGJtu}bCeV2$-|^rIf9kA)VdznY3kv
z9;ts}Vc(SX=+Vt`VL?HJodmAzfFJKJPvWoT%$*l2I8M8KF1559~a~AH#6)VRrW1;v$wP9|4<9@7S?pX*w`{
zXlPtqTtajd;jFxJ52XAwKi(?6?R;;xF47wGiWU|Yr6%QY$srJF%A058rRn^bWT-Gh
zuB^Wf;ukl!;<00!2{3gY7^s`V(4T^W*6Ypp<{XFQ$i4|ybKd!=N2T=75J52j56R3V
zBQ6Zr%uG#Pn(XEijD6E*N6R7qw!6D1kL-g>1uS)q%B63nJXkcH4#8hKGye5RQ2BJo?8D3{+N`MTFW|egAMk7;A0OTz2oc@|42%1F;L10c5X+f;PE>Ar0QE_ohb-kL5Dt8nDI`naRIVHKd*Wc0v
zI^kmNI9?0MO!NPXg$CX|nEp3(uscRrLo`lksj{NtBHZTf-KF_YTB_bW?lLdZmO)mpMJ$7$FlJy|@Z27PK
z|Kg~NtU-zl+qPZ5TMHC(al9p#L@F0r$Tn=;$m3pCD-p5tV8ZR&DyH^(nl*wT6Gw6;
zT6^m^4PFllYCu*6%d9hv#c?Cn733RuOW5XkbDQbtcG!hevbIrRomtt~WR8u`IR53S
zACkV*W2vi1y^337A*Gqj&GYxjj;7imCI+?(-4pS}+Fz?e47O7oVIRrhFLQIWE>WV2
zsjI6$r#whJh2yY^T{hK?_p%`_>E-9t^<2p9e)#YKVPuMje?kf3zlB*ujG4D@539M?
zkoR9A)T}r7*%Xbi=M4;&%GkRgOUbltG6Qpc^(v&BRQ$MIZ^BRMtPprNy_Jpn19}1S
zAIRPZZ0;OEP{bpQ+ywGT9lB$`I4l^}-6!I1)G^P9dc
zJS6yzk&RHSuExZ~pgdzeAswN{n}FC4`fAgeV>N8^3k$ISEdknC!oW9rdU`U_(+L97
z8Xim}-lFu`ZbT8d0<8*V#l#$;%qd%{{x{iYyE9Bdil-#wk-7vC^HX8QLQSZT?Tx`A8jEb73?}$E$OD
zG&Fe`_wL>6!u~WnI~!s^bY!*c-;luXWisU9DajlK7@6Z*Z3>N^tM2Hzf0QXyxhdc6
zh_*KBGq{^-+}E#Pk%^g5pu2onQIUMbfqqZole0bieO4|Nq+A1N?z`eMn;AIkQ0%(8
zbZ4U4tt+FksbKc3yssyt-8TFA^XCsA{I4znNonz~nC$oyX2uobzxGj7{IUDJ@JVCy
zqN1Xqp`kGUmD?e2qT;ST*-pyt-DHv>*_jeiVXc<
zm;IQJwlRDQt0}kR?IZC64e3YeR?@{}Pl#C*{WY$U%cv9<
z5n1~7E%L?<<*@4i!V?cte;}n(RwuK*jy6I@`$k-WQHVqt%D-A>H~Iit%)dLr%6B;V
z{y*iH{q^2j5l^xCsuA?~`7`tcHTL}#ZhC?M$;4>ES?PpBoJNRhx=?!X6=!7d;JLQm
z7E#`GtzJT0{L$mbyIl1DhE)$0Mps__H+o$?hV=zZvYEh&3JD3ZEHn}z9{vB2T&CQy
z>7`X69B`;1T!C1w-txa=k>NBjHYbS=(6DAp*r
z2{yj~42dgHe9OVEC=@m6M%ycd;RgF;D$uYeNf
z4n5q=>>UC3jspm}`O8@5A?e1EDDIoO<
zEgT^)_5+^c`}SSJ7Y63K?fVIWYPYkweWQGeg4Od7)K&rYuQrsRb5-o1$PvTomwtAf^o8@}xsWul{FX=4*K6!-Ddr$}Tm
ze!6!g$b@x#GBGwz|6UG{l$P=|>VTQL(l|^6vyv1wPWn6X@kj3-U<;K&v6GSMGc`PK
za;PCOPe(w1Se%0343yp$?j5F*0e=qMR7pt*A@NuZ;3#z6$S)tq$HO@k%uP&80<`!V
ztD|`;g!~Jnn9bUR*Bo>=65`_GuSh{fK210TO}UKMMNSr3qwhX{h{`^aLXgL6xdQ?~
zo!Xt8u9c=UbbE6T29oX{n}{egtNgvy`
zZH#=QBO|1_2V?9*fws}Fw(Uu%|9mB;qb?QCDbv4y_e7u`O&m0f1PC4?ehNO3kHGUb
z28Oh(tbydpv9hVppDk*<&z?RVbL*DAfx&trwaX52%<-9DU_>_=_*}Ma-;T73`W+=m
zu8743DUy+0Z3Jgb)|3
zpUB9~iEY{%8ems&fmI4ZU?uI0r2bh2$egBw$ij8@Jp9J@*sPvOpgUyl()2Kzvud9i
z)fyA$2LJ0pem>k+X$gr*Qh1x6pC9!PLLdHe(<7|CP-Sw12w(h#YTuu$h88{-A!kFd
zfMJk~v<(~-NW$4y07elRMGVDvWjro5H5IJ#8Mk0_AU$as1HwJN5e11mY63ZlYZ!nD
z58o2jfOuR1Gs3-Sq!&k?Z-`P;{<0&oF{8^4(F#;xBzZv60G|HvLj?r|2&RIuEw;rR
z2Ru_(&%x_cR#AZy0j&;KvWV^qfPt=WUAzA<{EDmVH|iKaB#jO(cu*`4PMTTcJ&27q(!5)W0>=I!l%%5`k$QY)HpxzWY8Jr!P{{!~-$!ocqWtL$=DmG!L
z*3ZxIQ&?$P8NLv5reO}vVMyyfK2PjxDB}}!h2|~>lPliA8p5t@-z_F~jTBz^_3Oth
zx&NUYZx8s6@110krQ-XXp=@I_j&z)~!=Lv|(X>B*_@;V4WZwoz0~MxUVC_+9Tr)?n
zAsJcUZI%1%)RKhdTWe-YE+(BcCF>XHWm&;N{edlM>*42<~gC
zsj-epURG6C4~>nD&5e6p%Rs#jiGo1COT?de^j>v8pSZU^wlD8|n4ML?p2K%tjb8Qd_xj@A?p4$r7)R;+-5~p6%=@6DngXgVdEkZMAqQy
zT@OdqqTH#jy
z;@8v26hV_>rdjc>fjAx?&-PCSz^g%`(a2;%HO!^SW+6>N)^2c0t<^**m^`NWC9Ljq
z=V)nY(8MvSM3G5bhvx#$sL>89?oa4f`^76#>X$X@Z|XN|9@0F}?He#H(Z57NKP>a#
zM;}WnYDSO(gQR!f(NQ%Hu?(11JYUiu-N0H>*4ELXPi#{;8>&3=@uNz_Q%LWnm|hrp
zgyI|l50AkEm6B9*WBEy(mG+~^_|bJ|rsc8WfhBFlS^orN!P!w1FJJzFOnX|!z2@rQ
zzkVPI1{D?6eGC-j=P~(}F^r`Gln|*8AO39LKP$U$UwS!ao|#o1RaAIEzyR0BMb1P;
z(?3H@hP#(BuJv8vX(|oeWX6@~gOHU{8b=x8&h5DjMv~i5ZYm0eIc0D22PaA=sj`BC
zf@{~PXt0beCF94M=pjvMXE7BD>ypK5Q4$l=)0iY(UR>ne^qP_Y$a!E^wP*kS%NTAX
zt{?^ulSQ1HJAHq`VzxLIc=ZfdD17xHnX_13Np0
zhH4=Ch>C6^-j9vBL1D(cC7h}@=>L$y&a@|y5ib+>b_@?IAQ$rSWjxv|6h}^N<6pp)
z=&-O37eH&Fp+Csdz=$lL=IzuHHD1i^@ox&dU~jKSvpBdBq`yg*kNVcL!-5Ug-V4??
zHc`o+Ra8{)^!Y`bb+aJE5d>01Zu@o+PWTdlIS&tyB5Bk!3W|X0>H}_a?$fV;3Q2ya
zT{lS@qg*&4K9@HJtMZ{)RsYbCytK3%CM{441r7vUyVh7_(@GGCq#<*UoHI1p7CfOp
zUji6`q#ioWI!ppdGY1)}s-6MZ$WRF{!X7UxYj0{I5lIb=>stJp>gpTl=#Y!Rn&6|7
zL>wwKEUXG8;G>vDWB7K}b*tk}vbd2zj;(y_$m=lO1p++gQlXf3D2)a6#El*$V?Li1L3)5P(IcZfS#y`h*_Z(a;>|>ymXCT;A$ucK|w)aZ6jfp
z)iX!_W$xufS```z#zvn%5Ys&7BnwSCp>N^{JcaX#?NoHzrhc7u(4Ixb){EU?n*A3{%00oAHv`0_-*l0s+yW%
zMs;ZFMl)Ud;MbVb>iFdMdnM{8Dbz8xu9MlwE+5ynGJ!|dDM1bRC7JO5r|VC
zJ{02H;Q|)atl9(kLtzya5I9tbaZ4o1RnE>QW8!`(Mbez3sry$nF6qNOw4_v^sMVk(
zgw3nVks$!QMd4J$`nR>^FXZhb84iyhdjL*_Z{LetZj=~Hn5&CRbWDsKd%I)pPfDV*
zyZd@%q(zmx2&f&@6O4@=V;;f)G<2U!_e3DKvp0uOP;J6^Ip+d!+LitYrC{2VK(R#X
zqeq>st*sdGlaOE_#O-?pkO>84gYCp-;D@o`Hvv+AMmxU)B!njYAxfYBL8pp4upqya
zMnFGdY+eP4!qKll`2QVUd;aReVA%`=)$)#3u-SFBIQ|h;4W1r}SoBtWR
zuj?QRqLBaq1R^7_*#XA@+0-GMqqvvlQG=er7&6~BJvPD6b$h4
z^11Xi+(_pOQ7TL$V%(;;X3wrFTmS1kCk11%%o+E$BcApmk^|w(gBYU3yfNNSdhFvv
zoZ?ba@LwefjH?a5ear4+iSg5av)Eqa^aJ||*}$9MYa9R59I1omMA?6sBh~-ii0ML7
zXvlq|0iD>)sy@l
zmoO&sHsOyTPpVLl)N!jrD0rxFU77lV=k4ve9Y?^V3QBJYM!O;}nbHv%8akQl1c;0A
z8=Xrnw{|1oLrQ5&VucJ0JpMtm3nOF6`{Sn75a=z%*6HrX!4&&0w!i_e|A07*r5
zRqQ%;mtGFE13z6|T^e6m!0;+LzUHWsV2|yb)YFf4%%BwAj*COS1Kmy5XVpzH1TkBz
zU{E7F={L~3z475dU+E^O$@o-w0SR%TfBYeI@y`PVCMQp>eE$wDD!o)PrrFKOsT>0w
zP_b@{m@FgIf?%zfSOErWNERWolZt6`=nN|>D}kF6>nsmoFXA`tc63;Z^D=G-q@N_X
z79mxJg(2j#dDEt$QinJ#{wfq!#FVhrw=*(cJ@EQp_7f@T8P|Fg*)}cV$i^SYA?oad
zBJkbA8fi&jJ*)!IPJ)0pjj;l&LU38UON3DbQHe;*kc3UX#r_2&2*1ms*?S(Axp2Qi55JSP?h}isf&DN0SM797a31m^>ni*Kce){yu<2O_3PLdUK
z=P@DMndN~qIH*q@KYkqctQP<0NPf&l*+`y*js&KFZRU}ieRn9644Fq=U9PxRu0Tk@
z6Ha-+G;iJ*Q=fv?10S@a+Ef&1a@QxYbp^zF65FB>4m!@bo^;xYeGhWRIm(=WXlKYm
zF*V90UQ3f4`XT)@{M%n2EBk~O-@gng>)0j`NhD{lm-l*Kiqj)u0WVE9(<~1tdfNzb
z@9%TwN&bCDVMTPtl_RD)yLknx0n*WtvmRr
z-hs-o#1ofUNdnlL5-3kz16zov7%^X4O?dyh_>6{Q8elaUfwN#XtvKcHJY{Lw*_U|!
z`R`xf6y|L`+x`W2*F2g5<^N?wQ5rK4guw3IfB<^)^UHh5{Zo+*bkhEI>d(cSJmo)m
z5SRGaao9Jejt!%l;DA(0PAU+^4g8#HF@=iYN|W=da8KZ@|r+{?cLH55F{C!Yp@-%_07DiSB?&y>bt=J8`l
zQvpTfcNpMLKzdSWm-+s8*YN-7@F8T@n(3cGSjo*5tT2`DtaJ+lL{rb~_U})3qMCu}T3rK9q>Bdc>aVC0tMd(@8)z!Y;j|#BUAZ;TS#C?Wk>(-1f
zjW1rLW4g!r{ZoIqZNO%7m=@ehhk@YbZ@&e`YOr8mChz}-xi1V8Pv(#Yx7(p*{ajs|
zAT>}Du7IZU(YT<;%J60)js*HUW$VMojwil)`D0Bdw--_B|;KPSv_4g-rj4t-qmCE=J4H9
z^YS!Ji~gSRBLcIlBS|ji?p=W8t1imiT}bg@h*<0mhBrQa^~B3AC~!5jo6r4l{7=X9NRi|XNj^RsC=2$Q$gj%o-w(l@
zl<2qrTq9BWIzPB(O<}oxzv%Tmhcyp(GTrOt-3(WS^#auxyw@?%yG}9%pcW-t>+aCxvCqb#v)E$*}e5t=0;5m%go}cg&(6ev);>(+qN8*9-6g~
z@@SX)_SV1SZpBiy7^dtRtd>rTzvmIh%fm}T*Gh9==YNPZaPnA<^~v{NJppR)n{#EL
zq`bUi*j~{4<}mno4($}30|_%7oYmq?I`9RHSwdQR4MPh^V^P+l9d~p98O}g4gE@{>
zR^#_uK1fj=;Xyh&zREI}d$t_8%Gyb(5}4Y6(AQ>SHH!Tx3`mo9q=}UMn?X4=ja7cGJ-gy(RFe02(Tj5Vs7pW
zq7i-W`l|5qI0bR9Wc@Qs@$`(0wwEt4);8gG$;s&v^Qmp&wwOb}(E_`6VSpd%&#L@y
zoGsHiG=vklGPAQ`X;fg4l0POMaFChV8gBN9Uy#e-e`;%Az;-9DU^6G~b=E5|IY_aVh8!Cw;)ih*4}d{P7Da=6%T#r(R+I6qrAOg;5Ebjwbj$q}duLKJ4AI
zXGGSQ6n7r8R&0Io;v@dv+$GG{Vq}XdViXh^GCI)J*TqG|1!N;Ilu&c;o^rH|qA!5(
zft{NZmhdMo6|xZs`+Xc6t3Gac&J21JfZ`+e#;^D;^-GRyY;2t?h;`G}OG-&Ovtu|z
z$lKV|bnWjZeJ@)B@`{)yTO$b+kfx_F)(Lmuh4TS;lMsw#BS`EBZxSDO5Rg7%6pQ7}
zo63g|=^C8@G_r-7?`L;Jo7dQxUP^{2VHnyVa
z*IZh%2l*ooVJ56M*7|tw8FeuW4HKjk0h}-tDLwS|?exgVe0%LLh+*$_)?qGr*k(aC
z0tgE
zOM817f^>Wr5Dtdm_)Z4QV~}`Dtj2l*!9|=7fd#xXp|7|1o{m8G#fJX%%Dxj&aPQR(
z;_Nf57|)pxA26%x$IHi8xAmEAG2R;vlUPS!sLTty0y!;$z~;jw93JW&>9dRkCl3#W
zN)5(Bx-h1Xj4I~7wK3&h?t}y9s;clVmCV%l1bN?{pU*5A$3P=C!}Ek0k9IBeCom)!
zbPPi*%*<*unbA+hX>(yvA|}mvpZAobW5=fj$9I*b=Uz(^hmcW8*2ZaC(Y(m(Da47)
zV&R2Z=_pD@Hi853_?G7(*N&)#{RfQSNXC<#pYi?5@=3%Pjwh(lrwh13@mFG*(5B5-IZ3K8ND$u0s*kM7xGqm+b5R_
zCvG;V$50Y|f$+=U?p9E+IC}IvqUppHtXxdtK$>`J@VfK-ggPdVKP*E
ztQ(lIk|X}J_-8Y@FElmGWZSjk0BR^
zv53Ii3r-7COvBPS#Tq>N9UK}NLmaU~yMmFaM`wm{x&ch#b6%I`-uvU_p}yhwOG`>p
zzaU2#dW`DNvgx(neI7$7sGj!r&puB~*g?kuR{@NwkH(+?bkDWCS%VKw8>B
zht+iqeT=+3Jg0%Uh%4pgvgPRyv$C$elHYaWaa!63oLh8t0A(mgSEt@0I3pQUx+2*zw1bWXX8Kv5EekHHD+~9L}CyffRQa8jYOX3eqyDiB2J`1agU?
zUv7nkX@X*Tchv++|b>#lv
zv)>04j~Iub!ji_+MHd^Kt8vN*4lilF;>?ai3Z(b!VG%xi?i}xh4M*4;y$+bWx)!H8
z>pEPWzdC@xMox~pyIEQ#OnaTaU%!6gNJqUyQA&tuAKSKz4TU=tPL--#+my-M-U
zviS8?K8Q`&IS30R?>XYba=z$Q8Fjv2@C!&WsfW`%?j$8)^p>mlDVABCd#LDTmOjP@
zg*bVr3};XQ`D~~V%pJrrJ)agZy}KRnILg7*-z?%8DAw%m~Kpa@ZbP8
z>H
z-@n6>sSCB9{Bwiz`v=J}0b(Z^FTA)30#C@1>A>p$i@7%s=W<{FzAq_M5|SvCLX%8Q
zk|jhX5>jc9A;7ZBH&wX3#p)Za6q9~*y^ny9EKp5@_!0$CIoO|==y`kaZm#Ejhp_d<(1ws1p5*3j@Oe-r>D
zGx0{ke|-_C-sL0pqk$0$^7FH6h7Fk&!9S77>nqWrNQUKV9-x>Kv+2)2%68YDw>=gJ
z&i_I;!JW$(IGLl{YPWQrLp^mnWlGKa_oBM`NTB#_^ushiYEzMC!ZD!GpdG?kL5&ZM
zU!LY68GlQCdO>sG*YDp+E^r!CiL8&VLUHVtKH{3{V~}^vOL1e{;uYsmu~V9)r#m`1
zO(%Q}33-2|@qvMV3Bh(c98DvPjb-+35fX;kMx=02?%A9P?E
z0S`+{!3T1=kvetSj8JT|>R_pz&Q6RGZewDXXk!CZkUy$Onp@HNH!GZIM5xUibqaXzl
zIhB+I+7A+Rc8m|ZDOcrXrTUndJfQ}$uvl+@SUl0k+c((*3ZmzQ+=k$sdUATcml16v8DE%^
z4$d!`KLnyijG%xLH8`5K0{n90MjMhEsn0{Qjn{lVvMF$pBX$JfO(JwEQm6AY2thQ(
zFWYiLll_sIFi~3zaFR-nPJ9$fY8sTTH`Ai4{E=^0)I6|!_2vx?s+u6*v!|Pr(`P&*
z@>>mLNBUF<0T0nfd3m)F5Y{2hV@Zh}kU)$8fp_Y_FMKxu4co1|G4^$A-~N){hwil-
z8XFq0^ha+l;6=b%!oE39nnWtfGE$ZJD)y!y-oJlJlI_x^rw>Fy;OQm2tH+ju5MWz;
zeCLiG*OI2dLJIo*?Jgy60ECdDQfp2`8O=!c3ypA(KhoO`ap=~>NFW4;
z2i11S3P{T#)eG<54Zq0U%3u3D1I)^?&tk#9{P5W`y94b)3vbtuhYDrZD2z*s$%HlmG8u=K)Xcme*8Oodp=1W+^hA?k{Bgr)Vq0mEWpFH9@_x
zln^C@z;EFFUe2vwF^aFqUCCmL+6g&*?&3wnA`fa-l-zM-&$88fY^h=shih}^*U8T=
zG5d(;hvgThQSWc6a$E-P6iZ1}b+z>4)ixxsJty<{@@+7FIMGPencX*EpwX%s;FM*s
z7<^p_CNKc6wEu5=3a=|G-%s(MkP2wx%uk16)GA`ZW$o~;yFk3_-+!MEl4COwbIZAb
z8dXKOS5{_o_(9U1J-_C4;%IMv9xy<2nj-Zh1^)gAHIzUk#jgJ;5yo$W_6;yg&aEW6
zkNWNPn>SJ36}u?$B65$OJgIHDmhKuV16fX(*6_upF-b`%q|G4iY_{MKK{@&J*RSzE
z(0RF$8iAo+xcB(+4}cw9qU0=kW`A##n>iXAHKeb*{sofzZpK^CuE41x{n{9Z*H%=y
z`qa&gBw~|YI{zx%+Jy^Kd^9Qp)#n=RLZ!EE>2LNH#02+!+<|BgSvRB6J>x2sz0!Nc
zdd!to+_(bl5PR+^x6EdY239?3?=9o(bmq;O(^W}{lW#5>c@axO#{~zR;sTbJ6V{RE
z11V3QNmn&B+4JOgF7fg@x9s(J#}EV^{jp^oKiw{A{bT5Lnt_#mezVRm?5bOYrH3|<
zEfuyK6!17D(rF+7+pG<
ztZ^!Kedeqai2|m+21f|-uSd`ynFi`dNC*`4c|QsOat}>R*1Mu7-oTEszT>W=yz=)q
z7TmG=piJ4OGoiPJww4wUB83RR4UizIKj>{}Ks3%IuPpdS3$S9GtLty}A>b9SSwcc$
zrrt$vKN_Kw6d^Yag1fDhlxTY5LgASg_+kK4DLn0Zg*}@_7|Fna$GLrh5+KjA%3i}x
zHzA;Q9`N_ATilN}78aVrhWX($zPC$M%}_}2^$^DQ-Bv_1$4!j1z-sSvRs}USlu>He
zS>QV)1JJR6n7O%?9j(ps$--`MnXxo2mU
zZ0I;78HD=uO`i@K1^mx>aoQ~t!vtY`pOP=i1NTgx{5QCka&%X9^+34Y>PSpBB&N
z^F^vZ1DZj{_)=B{msgv<;J;%N$k&TeK5*@tD-Fxecf2z!6ly8TZ3^lsba`%kFpaH`
ziIhTMt~x_=JViM^3;OM3QPCATBTXy2s-Asjb$D}S2fzM}hkVrLrMqy#`850*O&QJ#
zrus$a%BQ8*HIP>2&GyP_N#5i8R{E55d&P}fHNk@@Sp&Q2uisR!(o`<-+_R^L%9uS0
zV`%!dG$aL?mHeT8@A&2*LbIOX~nBgWPGf7^6QtZ1k8td(yONMP5b`
zPZV0K)f=+EquwJ`r%5fy$!Q~?SbPIEOLIdZP3KIApr!}Q1Hb}Vme}9ygwMKi<%*a!
zQs=yo%avNjU+Amk&ym^ERNMj%MR58h>UNPTEuG@Ob}bOuNF$?V?sr;0;w_K2Ym2lL
zYhYllHblVSAf;0i2mx(Jzm8rGp%F2T{CYkI`3^$X|6B#O5e`jAR8~|(I@5(JMrWI-
z>0(Ru*qub(8fN*E;tJ4`n!>c=q{`sAKy{n9H;rp8(#OqvJK9L?C~IrW=FRPdBgc;y
zCm2c!X?`E3b2#%|D{~__vOwk<8yj=)p8I$Gras(zPkMSiRi?){2y1eyq4DuE_87J}
z;m?c-2?6GWzFJHt8$X`h1chfh=y|Tpy|Y_l+MYQJ7TD}BF?%K_E{f}R;bvOE>0<1g
zN;tYp{TUZdFwtPAzDt_$BpK;qm-7;Bbc3^ZO+e@6gZ;30PC{=NNag|^X5CA_L=!;aCCe1ssOs%V>mm!
zBGMOmKSDr27sz%o2w%uD&^0u4M9gsG2Dy(aa+^joR-$W{E~8($H_OA=j2@jE@syTD
zCNI87KEewZ$LnPh0*E>K85;iMq>Xpz5NG|kq@*4Z3`N&xDuuAcoMHf{kP#GeS(Ydu
z0rlXBkQeD<6fn4;KwcU_h!t~@^0uiDGsx2%F?_ffB%eq6*y-?af?vaGl?`NPoblRt
z^l1Nx-1XX=tIwY!kFJ472+@Y|y>{&y$w|e<;jU;DCAbyTON@EM
zJOyw@tCN9*HF$Qe;$XjBROIOF96)!MnApQGrIR2^IM|<~u4+P}!2cg?JBGJym8tsc
z!4>385&})4%GDE>*HUnI!B6=(A-cx104QgzT}e)
zIS8e5)3_F8wCAyr4!_-{WvZbMPA0}b6!ujE>MYFSzx1aX2LKCy61{%?d|Mg~n9PdB
zYcfZ`_>xLWr>5t+SXiVqW9Zg`E5N|+dk#kNqa}0P!(Q>o2F!Aaf|^LpM6ru8<7h45H-TOxX9Cf!Ud&;7FW}RlG7_{9Jwoh
z9}ztQA-^~GF5St4p{*iAE$jE}Px(OkBuZYUtn&ZY(e8IEqgPL-{aS60>uf7x?+}_Q
zIY)fDQaiId6ak)$YN{JOWy&M5F{4?*V2HD4HRmDNsM9$gHR^lukOgxyg+s=3NBwQ_
zIW6en-+#9{`p61v0H(C+JebnqCn7!6da*KbR=vYQL+P_fP^b9(aT1azkjxPN>iO}k
zme%=$J;lp{txK$6?u1UdKz-cMb~?+CsEbE}q3p}7m8e~5bio&{oR4O{1R8vl+L*cGNgq+r(i@o|sSTVUB;#t;2}lIImrD^K3stYkmkby%{=~fa(SjQf#yri$T`efvra)Q})z
zpT%4pk5X(xaIqcQ;S!+q^Y--2sj7#Y!eE1TEi$s7zCJQhUY5vx5hn?|fjkq{C+-0@
zcBS+L%qqYvx5t9;WP0RE*@)O-<7WxM=}8+VRDs(_>9_|*pIrM6VjScXmHl)+*Co;Rvi044U7
zrmV_%9S151q$or{VrsnQ@S#I@DLM-3xnxA{kRpk$AlG+KwmZ1R#q;MYjp+vDR%(|i
zeVcawR<|X$=z+;ns-(fX0CzTI9ab=I?xLX8Y6ZSTJ%Jk0>c2GpNDjZcx+?-1AoH5(
z8-|keY_XEiNt&b!7cWA5!U?5EfuSMmqa)$6O7o&k0?Lngyl*m1$tTh%cYc(mkO)wt
zaKQ#Faawv&#qKft>Cixr{nxA!eQW~;L|OeRzf3A5c5GSz!le?oZL50wR^}EJBM-sj
zb9=Moj9T$l9;?NdpWLWd{66&FRs?LH
zmgeB*){+FZwxLg!)h7azE#{}5cmGnEa*b983I)_|$&!;$$pM$q{(wvzJ9<BmVQJOq#UKGIiYJt~0vH|4gm^@-t3qDJP6a_&^~T2Ls8nu}r^llS#Y-
zQSWwczJicK$|FCOj8X}~E^!#aCP((~pN%zA2u3KY@{@fy<}kT#)X9s7jl{44DoAE+HCa+t5N+>lThyR!+A8K*#{oKEESQ%bM
ztI<#qKSl77qO2rWY2F3ZhAS|G9o7!s;L(ewnjZ?;2<{y>G`sT#tnM#=(I_Kpz*NDG
zby|`CNV|dWC?ljCJ&HxfDzW(WvPvr>vh0X-7wx$|mnM-1M4CE~i-bTm$pfd-2|j)%
z_36EBsC05fp8(NLXn3ij?dYl;rdnISP8!xe`QX(6nFSbzIDbAz9Dn6nOdUtmm#?p3s`ZJIgAZx|3^-}Ro@ek3
zJnb+T`;sHY72bnvH&&Ai+!(XD9N2$iJ_InbvTOEbLR%IsIu5MM0*%gZBVg-^-rR-r
zLG~=#F8Qs55d6WDDvkIS6iL$dSrtic62;>*vo;rp+q-{FU~Vm&Yc9
zN``Z=c#eHkFKgnJyB4&3nCiQYuU2TFgzl5EpI3UANlk>kdox@Udeim>{ODX!pK5uaBLC
zPiet|{+mRpuup!M5bQBb>cWWc4I2JdVvc3{COhGJ(wzhualgAr0H#>
zkE04Z{h(S~@uc_MY+QWMslA_Q7afWzj@zw!VNlzD@FLzU
z^$qtj(j-yLtG;y7$9)ndUUF6^}
zSJ?ow78+Kcq5dR_&75v$$`P;dRr+%F+_UX$Mbqkz9oSe#Ef
zA(o#=!oNhXbYwk?Y{Lgd*YCQ=v|VPS%vp>Dm(IS+vdPXDFJ0PITz%Gs$2Wjo>-lrv
z{J%jUpvLc?6e)|bRkKfIC|gdOwwmplXEO^#83NBo?<{34?K1}_HdLiO&z+}E?F;uT
z2pg?7ygw_`_TQ(=zi-Le6{LAyY3OWB@BvqtB!D7A-c5v
zYE=C-V<$&1Y;VLbD>VV{GWj&{z|+#wBPUO~PMLCO|NiAbSPT+FY%=Z)DJ0rKK|y7I
z_Qs8S^EMnibxMz0JUY6gjEu@cw--J5f%IyKGqyT(OS~f|BZJJ3zVjMaEZES%c<}%&RGt&Fn>+C8vr}V^*3bsh$kPZ^t8|=S9Y^P+K9i>hkoP;OU
z_ChhkOxcV
zUzHQv%E*arl;T=3fCepd&7V~m&ORfg^?&~0!QgXu1A5U0p=%O?!Qn
z5ohO-Z_qyW!siEl(VK|!$Tzn&#{)oeMVjUxgw`%A`w1w2|Dx%}*E9#5t#EeC=?{{^wW}Eft-COFGBl}O^GjjB28hh>=Di+4Qp}G4?u>xGg
z;31z&&r(+|PMN=C389Alw-$sPo<_pPMA~&xJ9y*9=(m%R9{8#}8VR961G#NuA4}>)
zYFK^Lf1Ov&2JZ%q%gcU~EW;5-#deR72Yd%F`s>t`e!N>bsq8D0jO
zvE%CO#`1Bf-2uR;ln``HOfuPFnM%;Iq`-tATH271kcuJ|U~5t8Fy!Eg6Sb7Qr_o1J
zENxE~b24NoT=|2*lG$TvDZvWaulMiWBWv&lQh)c=j10T&_7yZzQgEM}%5hDxz=vir
zAqad6+^?}?ytcjb&Ng^6AY+l(RWKDniC@UC89%DtH;oWyF|5qh4An3tc-=1^05_LL
zzlj{ZX6GeB0Ix(^o^t~|r}FA)yDL6?pqTy$G`rqMe!Y3qqMxtc#Cb&Q&&Uw_yuVOJ
zAar0Ypr)X`(J=Klk7Sr^6%8B6JLMKLH2Bdb;BW!k;fdb9EhQdS)M+nzWHokdmjYd~
zK2VXzk3UGc`j|@n`Saf#(RC%_T!h?QSQn%sokX4w3k0V;Wsx9Y-nPa&YGnnhLAD!}
zC}|)AhYydhI}BcneCo&`x?n^&RtA0UJ$TSq9Mn8GSpP%d>UkTiE!;1R?fuw;47l5~
ze^(Mxqa1Dhs{2p1*!1?Wn}`Z)$QCu8r70g-yd9B3bepq7`E^BP*Uz8N+gm>hNd4!mlo12dt
z7x{KO*yKo$@Z(&H9B7DLDX7J)Nh)ymJv}NgBTWAf9vzAa7)+GJyXYAmO2tpVw*Mtk`Sq*$E&RM3nR_Fl5({_fSyKJ-Q3H0HBp?
zgG|Z`z~_Rwo!xD|F1wyE0(M$8B2iW?%umf$K>&K#%f=<*hwrdIH8mD5z@$lo*M6_1
zg9F1bVb%`_bQ{5L_mCnaz=X#@LPXTNI4cO`?lB8R6!gNlNrhN=9VZpC_x0SJk>TIxcT$l5X2~tJ6G$8B<>Uby5$a;LW7aZ?
zw8T5ZjEn~8>Vl1F-z22aevdI9dlDEmi=pF!Kae?wqmOJzCRqqE98PR=mbiss>3Tf>
zP*)q0RAh63;i>D|ik-O_xd7QZe#8JKHuR1q2M|)Y(^3j4{%Y4wMfL_vQ#+Dbk!S3)
zj&}D)MFUVldLtzp$QMw`kReL{);7<#y4gnvX6$1`#Mp@wi!rmIFyJYEVe&Qmn_*ZI
zf*4Q2MjUNpyl_I^AV+H3Zxlea%HTZy}dW4$`vNhv$BL!Yu5fF
zTLysA01-j@JjnI(kU#A-iqe-p#v8u3i*MV}UMqTlr^=01d9tX~6yARPxUT+XJGFNJ
zWIqG?2j2Tv`}*JuJ<(0eG*Fw2y>g6Te~7)eVom
zj7ES_ZW$puP{7gh>RnJ-U0Kz+z!&V2`i--&
z11Am8Il+_JaeNEUqTyZ?l{59aothGY0ZVPvIyIg5Lx~h}cE*Y!2vQ-jNlCxUTbjE?
z9b_o(@{3~dg7P4KOn1}RnfLnut7SBGh$ANSalpdsvj5fsJRVN{ky8l?X&hTL_pE4z
zw6jAz4@!B&lXm+*jWEsr#9c;A4ngYCMwB>n^INqz$#d|4^7x}vJ4yX?`0$U0`q^(s
zl+1~sd6)0$LShL0IK8i>l=XS9)Af>-_5PMgPnw6WPoaDHHN=!cR@PVlC#?g_nh=cc
zoZ!|DesjdOSauA@09YojP)|=!^wJbQoz+baoI16R1W*JUa7GMj!qA2cH715RFu@+B
zK-Fy|B`F}#FW8Z8P?d4`a9=g=YdlbM^DKxacqK}c=`&^|lh->ho-m?1EGAAzojERm
zqT96#W1D)m+~-30%@(J#q`M~70Z|G97=}CcE@qIWKYn|l<@6j|nn=zdd=qEkOKG-~
zK);Cjr75T*0pHvvN&Wg`0*xkE8ha*?b`JmEEHS4PzMH#oDaAsM}u+9{}wiX
zXUlRx*V$UjI1VaXR6Cr?B#4ka3I}9NK>wAswO66D*9~7!R8$4Oyyr_5sEHpTL2Z7<
zNlR*Mor-Xed*)ZbldUEPei5zdo-O
z1aoc>AEoJWfkwNU?5$;UBbVO^4HXs3sK24XqL;*`^xkDhtv{sx2)G8xtd==WW7cRd
zD4JCj{2&w4w4Uo_t!te()4WwoO0XRhFbiM;0V`6`Yw=S+V`5_!4DXIQGCjm9Gl{7`
zweR0I#+;lYGP0H18q!Ax%+iHaKXXP=XI`E`EyV37yB+iCUUluZn45i@03~pk#_u}_Bi8DC&Vx$
zDX*N*TnV0?Er0${=+Z^9Z-R8)hfY$o5pLKvxuOq7cN#OW?Qw=sbNv*wlRFaoD&dG0
z4}0ooo6uLVqayM0>lgJrdvfQUxt-6CP5Gxqv|5-CXTi^B8$d68CV>o)`%LM^s3>}h
zTIe|hcwQ$UguKhn11&JiUI&p<&pTqL=iW0dfZ_16xfSax_bpu-
zgY*X0{FBmJ-Vnm!jkBMSRD=%n#m}X~R8GpD*lnkSsemRUGH^iTpAT&GRdYn|-kw}C
ztl`e>+Y`yjP&`_tzeW>dT?1|EE>(m*B-Lw@XXTflD5{`5AC{DeF;!Rn({;OIBncp8
zLrt!)mR8%HG2;#}wy;B(qK1aJcdze-zJ|=V{l2t~1$dm7@0HZlN@_$zy6?w!{app*
zj?Q-a87~`W=Z7Q1s&DU_y>Q`xHMKuuxJdtqU8EhjkOHpD{&v)4l6=D8d0WWV7Tg&|
z%Ha$`B%%>LN6k%lvCM%%p}(LZRUBUn$4)!i%%PP%>F8_XjDU6}_Bd4V;9I_oYJt=p
z!6T#t`*gkWRL@hKV4Hf5dcijx^An(!lm{|k-hqkmhP&N9)O0rrg(Ab|7bnybF*##~
zsAQbN?`1YA{CYfde!ZVxw$}15y}Ej^ow5J_q^8j^9zTL^jz$d=v;8ik{GuY!
zE_x^+Vc3%C(3t#UCOstBy#XYI1zI}a4|Ytc!)v_EoJ8+`IOcBHnBawIxXaj8*W$gY
zjZq9z)tax6D8Kk_z2%KH^dWnalBCO%)?plW(Va3%O2w?rKMkb1Ie<<5E~Bs4UcsqT
zXXd~lbAf@l@nR{~1%U`EKYS`e@IL|^G|$g7gL(=KLOG3>(ym4lwHMEU>WxF!LmZ{+12Id%+U@mtb0X1AFow5R$0->k5jlU!U7%dUje
z*4SDAg%l6~b!ug0wR$xwsN)FH@i?$rX>sK>G>{_WmT&?RIg-f02z7N%|J(;Xm`JMV=Tr0>WQrz>DC}v?k@pbKwkl9O)CLnRQ{M+dM!6E
zV?*c8eM16Yf*rl2C)mb;ZPS2e|ERO!%-agcf3JY4kfV^-fej6W-+z~Rqo%l=#~
zN&VuyTz5o>{A?;}K_Hr49(x(egVDM|Ts;mjE(v~Jd&4w?WA+kldM-t7cUQ1SyRhD%n(Pyr)jEmYJ*k9&B8Ae>p;RV%_1ftaqbLt@wYqSKI!rxbk?tM6WHA#~X3ig^Y?cfk
z`W}4e1N;=T8X}_%ciqm^G7otDwFe4rWU?7P0J-GWX2zRrsp0^pX*S+6SY;Dc8WrD;
z`1o9l?jn2frZ_pMr8zKlc>jNqsxkRp%Cun$s|#C|XOcf^f%2GUM?Tjn_GrI;NjrOZfB+eSz`xsaA9$XEssbUKZhM658WS9>2qmI?6704@npx6E~B$e}J|
zVZ~$T!WOvSt$|4eOosHy4s-SCzT?hZWF0xRnOFr@m;}<&#>qxq9&|K4z|8ID;~W2y
zxzBkTH_MuAmZY|k>$f!nQ^ToBN^1xLroA}Jqgn>CA@A^3Awdc4Zv$pd(iOr35E$A2
zDB87(P}8YiJW{0hXH)y@A*N#*wQ@63JudZ@#`2aa&d!D261i!l?9mI$MM#beY8BME
zI_QwdM8#=gkzHIokwiOU$C1+AFPI)yvFsAq-M7!5dyl6yIfP2m(a}O@8@pWpva-p*
zfzd7#t@f7?g4RyxH(&rVwr#T%49zz{%unKY#fG{dZ%FddH`8JUrO`
zG8qOv{7!qUV8%;G$4%&|-_+EUJ$z`TvyG5sF}QXm93+g|0TrbcGn{5qpsLD-A5eZ>
zT}^kLGs|qxR+5Ar3_g1V@2)vvK@)?8+K%Mcm`
zMm=cR5Py|(Cr=1wnWKR9B&9Gelui$Gsh6+0pLk=9Fv42(KM4liJ?Cj!XDb@*BEz>!
zj~)YU8lbDdST{T-Qxwmq)tU;VLolD$A51h!LJMfGJ766L2)z1E-D#fg?!6Kg7}Izp
zyj%P8vv$dFu&belh6plrF07VyGpMkh{BP_{eF2OIE=jJ^QT$er@BS^_lRCtvqdGmkcSJ-
zHy)7P)7Vv{VkPVE_2Kby>9+%^<~7L;K!$=AdVab{DU|`Ot)RTHaB_H
zI8v3kqS32i1UGc||35UZ_<{%zEpMI5^E0`nDZ;?=pc7mNpM4HdaQwRN-H)ZFO7vu%
zqFZIod7L)=r3_`eETiz5wsBVwlQ~_XgIMLXvy>#*7
z)H+Ah1Sx$3;)XEv>(v7#E&cEo=wA6x02Y4lCqha&pEm|@W{YZ
zR~Iw;aIECm9r3(`O}BJQW5hVI6xp6SYQQVEF+2_ENW<0HS@E-Q-burk8W1m_y58Ik6aY%+S-7n#OP|s=0b-#
zU>60j`!T-l5IQIn8j!%_K6NBDAn?^n6aF{JqJv1ezgvaHs;4AlUa43)^AVZ`
zX8WkYQnl~gc@*@k!^jYT1w`!mAX&%&E2GOP^Xr9vT7SCwjdlIQz0?ZI77rsK
zbLL&SO&d27o7Q8YzLeCyLC_`CPt>(Fyt)EP4d{#7yy86=7<-zRi9FFDAii*wh-Y~QYxkoc4T73b6o4t398Yljh|dZ?$A
zfN!0a*LsLWm~KD(>oqI<7k`ufrQ5#$k{W5c#eJ6!j%tUD_~24i(Z!hRDL>jM_D8qR
zP*<#}E@g`OaeMcgA*0S-UX!bEb5H^TwO9M(nVP!}4hD%-WK@*m@y3#v^jZCvtacL#
z)ckh^?pb5qZzmX1t88e0L0kb{Jkqvw0OaM{8jR{Cr2G^7WYB&hg4h3F2#Eh$j(ni=
z8<(&D_vORqT3s8l@$vZCLqyqSbRg(_|0V13<&oFH{X~IllBB^+XPq!+Kfn;)@d
zf_wG4W;4mrA?kIXBc=OF<$JqZD>*848sN7*Z|k6OL%Q!z|KXA*Z>8H$s1>Fc*p&A
zixE^;`n_-K0Bq+yazV95AX@ehC3My
zAHK@hcfCa~+7;*kliRjOkEf)VQ~7PpkXKT&swTB*{`>%F3eK&%O382!QQ$ij7p+)y
z@9&GM0HO^m1{DkJ8UMWiLM}HA^BAVY*(%>FvO}mC>9dLCE>;PrAf=cFD+3fj;``-(
zk=9eY>VRd_A}zY%TA@%$sY5FU#d!a*X}Cuz0m91qvt)BknO6KPGbmYecbkJZsPl4a
zp>}|Df<})HId#gLpyD|<=%Ju`@Ft2L)4d~nDzd1JDh#K6LoFae|3*E;gZkR&rJ+d-
z8GuBGJLB9XyyOzS==Jt#R2yf{F2^$lhQ-6+kyQNTvJh3Q-(34Cfmdd&cIIlMyQ9?O
zJ0bg`>16wHXqhbV|3!IcGj5zrSPqmN{VfLVR0q-(qQ=U&7up`}cvq;}p9_Z_8}=yg
zWvF>6q6oc8aviqSgBP%i
zqcGe95`(XY@|;71$|S|EoZxlBc*rf>xP7<%br2xb28Zp$gr8X{(pS8itUqJurF#G{
zY7MiZ=cnbSF(O)HWKgrI#%`Pj){I@byPupjbUV3bi|I}QgK1+8Og?d`A-@5bw(HyJ
zkQmeYY_%~d1U5DcIN%&k@N71D=f%W3b?f!l+b~Pb=dV660mVHO?J-JyM%%*}tdQAS
zz!cb{NlEHL3Wq&FFEk%#-pwb|Eru+7rahF;L)M90%b5T@S(hBJKeVJ
zJ1K9jRQ2HP+I&V6lkRS3pI0Ulbn*@Y$~0po;#JqFpz?bCw?L;A7oRUw5Pg{KfqH9cn(!vakRKWlQM
z{ej-#>MT7Ej=%>yS_u^|xM8rFK)j!p{g|vK0@YqMrQ^6v)2`HP)$^vIH(s;eoj3kID&IAQTLkMsuv7
zS>0PvUHuwXa>1NteP#)fxKTl7V|QI-e}Nd)8TacKxmu03bZddP=LIp^am}`mpW<*f
z0r}daL4_C2-Fu_7)c@_hRDLD;qrrxTBM7&}l%uGu?8huz%4uV3Ss`-s=KP3BC_Wi=
z;JskMdVOPdK)q%aUk05AvJ@t5hs&y3_TQYq*dwVR^Q|g|ge}fwnNis7xoYJ~n+EK_
zio%ndR!mUS5nhmiX^z@-?GZYt3ku(Cd>
z`r#e`l?*IsFmjg_Cy0|@vT+`qTzydeCOF}?8&IU|^EorbbIr?ihkm3nzO3$A!hW=N
z#-=wIEo}N*C2j%45`;s2UeXiHt(Rb9WkH@q2faZz^y&NLKdhe%BK>j~2MK|yyqcB$
zwk;lfsc`!oO@g9wsDD$<6?)>j0h@|eO3TPdaMAmGX5KQrePcN8ZTrg0h{Ud)cJ)bI
zpB=*Zv_$aX5NP;peMUUSq@|_jE8kA;vTV&uuKn)Vk{ecAuShRjtnfF)LuU
zY2|2sXS?_mqWKgru7LM~M(I89B>y(4n+CFCND5MEfmD{m=;Xrm<95x7TcXqLQtlb^
zN*|NmcjiVnPUdG(j&=Mhb{1a^B!L!(t1$RVm_r;6ju$W3MP-9_ini6^
zA>VH14IK=okhkkbraEN<^>YaS-QpU9(E3#vc1(YM6}&2pow8oeh-r_UB`u5Wx_
zgxJV0KXtl9CUc$6#oqfCnk5IZZ51m2ss=Jbb;&Mqra@=AFy0II{dM$T4f3=i>{631CYyPO8&}+8MCj)amO+>rj0mD7~9o(
zFjT;t)7{o$yTS9FY_c*+bHa~>wSHDn{DO5YzgKN~7sA-n_427(ue_pw(rD?)-{Jiu
zdd&X4Ez6)5K`;X3`Y&G?oDj|+2y*^3E}UuUqOLAx9>f1s&O^^u;F->q2Wv(}1XSkL
z{Awdom>3}IlQSfq9IY8w-$7illy2eEy6nZX#isv_8*qD#Ju9nrmi#=(DBu!=I}~a6RA5OsJ_E+ik!srpinAkwu{lxex3Tfr
zEw@LSeILTvEh!}{KPr&CF*@XmdTxm`tb~tew2X|-#P^lO$ePRwTlHzy<+kXTrt`QN
z#m+5n+$O(&4iz_dE}2M4cgwIbH_!QcWmS<+7L{K8o#EvTzkX3*qn9zGaljS8eVMh5
zxjc=~{BZJAV!KRMa=X9d>Z&*hg9L#`83M)#7XKZVy_8wV*NW@bHlq%0WFHM8r9G|r{t>8I2T2F$U7lqyf;X(?qD
z`rpEQ<8P|*VpkME`;(WrNc;5!&>`Sy4ctA$*lg6=iy6$L@h
zwI|%}J%7Oh&Sd;EE8}Oe=e8c$I=0Bm{aiTp-FV{21`fn)NM;!c%!bsm;(|$Ym#{-9
zbdWpT-CdYn&+^OpOF`5dal#NFM676nXbW1oe7U56R$WwBz)~CRtm7_5t~Lrnn;L;T
zjy#uLyLV&XA-+Hh3B9=QGO=#WX69U3`8o
zuUA~%o?f&fZqkduUNEQ
zwY`h-V8Vpo=tz-3CjoPS8>v_h7(5uq#!kI6MlAo=ynpU%Rws4Jw=ZAz+XC&3*`l@@
zy4=P_G;NEGSIG3Jtqt9;sMklk|I-eS3`aDc+2{A1?R)54R8?1Vl<;jS!J@57^9@d<
zqy%R)o|8e4m!{7MaDp?w^DXLab(P=WOC4S3WNqC}BeuQZP7ubCBX9$O4X4pEv+5ij
z9K^5^IccKcXSG>h!P2jDTea#(^15FejeCfZ*;tI=!ljFyA1+n&BcGWV?B}w=NQ^@jvCwQuye1q;
zf}k^DH#JM|L4!n|wzTIqE@93eiYg+MX2JXMolU3DS+mAO<55f;LPGu#5{D5ZN3u0Y
z?=t!-&UUx8U4$%>5E0%S(Q$@ni!*1`E|Vyg4l^#l*Z)gvz}%H4T9r+5ZhWPJ+Q~!-dXB~Jm$MG*B5nXoSeUZhIy~x7Gk#OxUzfpljm%XEmV8SL*yIRmrwlj
z5NqPN<~Vr@t5>dA!N$kJ!aFT`o(POe&t@olHa&fPs|`@+#;a6DLld_VQm9Q~f4{&k
z$>bY2F-m8jq;8VDfVABGZF3E%RnDk#u6+6a{S>&FtSnRRL;5o60l5$YhQoHd6npeI
zzJI^u)N6q>hV%ktH3&jDu^JON*0>J;G~u+#s73nR?5o#a=lVFEd0y!*^uCl}ToyGgk$rv)KIx{lz|1QQp8olKX8%T{#g>H&xrs%`0J^3nWav-mY7(
z8GkANSPIAL%9FMqofQab@|L6)ak%ex*~AT+M3FPdOg*EKhl<>rRyttTmm
zQ^Cp}J`8MyzYZKabhEIqBH5Bq(9*K9?Af02s)S}Chtb$dRZNRoqsI?`R7ZdF^3x9M
zS6wQ7jD459{{B%&#m7hF{=f%>43sH`qWXnAup`h{2kvzpfmoCmg4|z>hi|61
zp(rET%;A)M@!h*_>UemgO(
zDE4|#yzD58LrwCkdRtJ@W~_X#%nUw;P1SeRuTOx@*V4)wQH=tCi?VaM29;n+N-x=5
zqCDPJR4{x_)$}o9HHK|({q)6L$)Cy*6?2}7|IA5~I7-+r*NCq7e?!mT5Vy1twGx`~
znJ1<2OA^|lB>RY!GkE^JUa1azd7=rnZf*G#)Y5TVSVTnCCvUipIrHYNYW%(=qtA%v
z+Pnf}%!+Di&tANMLD}2`IhU-P*JuB$bV3j(v?M!r?u;7@@BLp|>UAU>60+fAyoc*q
z?7}^$k}e(U#r7Z^t*(KipAMU!fO#W)XVqxGNFHCa2QbJomrikIil)!i!5%KpGR&&Dsfb4SeY=0HVyF?NgE`t~#{2zWkLS3^PqeIO+Z
zU$q%=4mU$+0f6;gVs)lYjgO7>xV{3oz-`^n{AbThwYBq+5(7;UdryH*HL-qP#fpm<
zcy{cV3|))n0s{>i^Km!83q4fkdU(`R572=_FOw@VkS@L169pXC*`WY^L4cQJzAcKN
zQ^$`#AbZ&eH|X0EiWLcg5R_k}&!I@Ih*TDc;alY4p&&0GvttKvLffFDk>=u?)9a$3
zz*G8yT$vJ)`)S&Y8I*?^x2&@eBQ&E(%DiDfaaAn+Ub;SREr$p87bcI{UC*ojf>{Q&
z_(&xu93_V?V^9Lix%mYR8>t@LLw(%nav@!Nim1i
zv~?Sm>esK;jeUYO;~Tp7$Ikcp+qXIPqsKmu3ifbe(-1uTcsrbD(6BuJqTjz=m&M02
zndGTELBwcBqWX9-)zb4=4`ZKRug7__N}+R7ZdqSC7TItzo;%yHQ?9lUOxNkFyxZO^
z@y5bx~
z*%u$2?=91I0~2-wK4|b;4YG0!bTu_Asndvuwq!FwTjlz?k1f})p>t<#(i-K*9O^sX;^Dx5HKu8S
zosuTM+79KlbbRL9IJRLWusxk5-3hN1XW8`&GEg~`2Mny}()lD(w`*9=xNlC7x77G4
zvmDZ!CWyQmYa?+4z3dFt%%GMRH*U;D2ID$R7w~eo0w1`Ti;81`A4H$R~SjjO}Nwh$o7g#2A8Br3hWESD`%O
zIS#Na*P~P~V$_(_9s>)CPVhC<6*?35B4@eE=q0^`?kKdqy|;XNZ1_ic$zBb3IFii-
zye=08rxPRg#EQ9t2HP4lK5_HrPO|NoEk1A{!6qFHSA;f1&K|7p`K0NVex^Za=sR7s
z3}_%jVh6u^`l>m)=y>pn4aFVAJj5S_w
z+(p@sZFd!|ZiBo-)79tE32Pvo(B@yU?f=a0)RhAg!g!v#t?da#dc&ttpnT+2*8va!GOSj>Pdb+>f2+qgan4OZBrxi52vP9@>(fBU`lfl
zPV<1Ntya>jUJjhm^RG3|)29=~bqQUGki}xa73EV~YIrjb>RC-LLeqYJHz)3wW}EV@
zaTsC0e|~MN9E~n71L~JGk@Y2Qln#pc!oCs$jqdgbhv`D7sRV(FgtkK@bp38L%y1Dy
zifrjPjQw=9w16J)s0v|^V0`TIU$C)FIt3DtCQf4M~}{z_l-L7ozI5ha12{YHttaezzYdbwpU0gusc2zGI;0;H0wcZTc@^NyXX4Z2}ZV
zKA|A6jc;4DN2dmFP0#nbwwA($BWa$41x0$K9KDKkVZJO
zv7y|w^9%o`8_uthL}&+Q47VH|eTydrHe>tjBYo&f4qOiV^@E=>q08m$+)kTbli^su
zS%o+f6!l(n=12&r7X001jU{ytsXByFj>6+}Xo0AV+2o)8hc0fIfmcuN7_MJKO19>v
z5x1$%JQgk-@bAYfPObLy104?h{K|%Qdrz`Bg{0%j{t6w!C^tEL9oDhS`O$?7sDK;a
z1X)>I)Bj%6tlMGxb%fd$$m5yDcdEazO-**F=Q1~TFmRh}E;3r+M@m2zVTF<$1G2x<
z!H9`QdD~We1|gvP&P%sy;{M(jbicrrwog5|SX!KZNOefbzOIp5pFN&mv}n$pl~kLE
zjF{tNVKtY)DqNlb&1OBx{Zm&^AS-<%2*jXiXAX4K@3T=qj&lK^S!(Q_7@FH$Yp4H^
zyoKH@f7GC@ql0LVi9q2DZYyx{A${QXd6|x53`jhLhde~$rNAsE6
znT^%-`c}8tdz-np{Cs_JkaMPDR3%3z5~L#arq=B79}n9H7E4PIhW
zVUq|~a85GS)Xg=i7Y69ULK(SOczbvQN~#A#Q6@UlQ8+k*{y|UTj=pV&uv@60r3Pl@
zr={4^NbiT#+amGuFb(MMZDtzU95#!eUg*>!FQ!MQr~5)=<_+0CaPh`@gj|u7Wo|CG
zz6AO~#+XGY&?Fv)si}{ZRUSRE$+^L;thYD!T3tB@tPspN4t5!9s-g(yQ$)5&Fatpc
zEZ0@0;palB$TgFT2+=oPhgl#g$rY?)_Ut|+4IMqII%1g@jAqkE+O}uK#akDTK?zZx
zIF}RW54|05u>(>KR{N1(981)TYH|ccD~e*^c0pk*eXKgUXln$rA9ShcqF#gg_EJl_7bJ0
znyG(`+U3|&CcQ|0pnjARM1knEUF3~5Xe8+^1#@jq>?3m98?bF%6*mgrLl|7RVbB_$Bj`J5_BvYP
zQ7u0^>*Z0TJbilQ(xtL{_v*NYxnQSGnGzn4)`Cb2;ssT7rgGe>q6%Z=0~j&3R=x{Z
zEQc+>AR;t$@$<`Fbp{U`cK6PmzghH;AFo=z{EB~lYx;%{&P0Q3fKh-hVJlAyB_ZAl
zf*C_S68>w;{a@SVy7)YmaCOYjC@qm}@QOpf6bk~{oQ6gLBFXcAXv)V(F?iUR!OXz&
zd`s|8>%|(zJ}D#$lr1({Pm*rl8wg`HlqlrM8V(;$AP_Sh)rKxqa%OnvW)GEv5zt1!
zlN&-pXiOnBHgDP_x?}0*<(^C!i>J&Kn6+O&;yXO}Rve=~mV0Ym0OervDQl^GP#sZ$
z#vP1ZuD~2HrfR}!;C~QdcUC3nI_0d27;oXo)!w}O66P9u4Aq-)d-mwQnYZ@Oi&m1&
z1<}iF{5bN&R_gU!gA?iL;?yZtSYKy87Ht8ql+uf@=6ixXt?D
zf+~dv9Gg&S;x*TN>-92pLF;zhdi}BharUueQ3=C!b#+-^YLl$-d$I}O1#{EbRd}0U
z9)t+r*U2lfXzqjdT2Dzd=7fbTa%>mw0bsK!Dk`U?YyF_@pb)Zi6rWq#a8Tg`Ti#u&
za7)(?VN@1YV?9(@9O9_m@R@l|jDUWt2l_B;){E2|oEIGKP)V(M3J-}TJACoL0rWWQ
zKMqic6}#3?Imiv_Bh1TBXM{)ZIB@Xbujt`t|E&e!4G}-N#xe}v0~Ji-9pdD|7Hzq{
zm$%CvmM$;qD1f6gFaXCrL}BHI{j3L3Mtyyplz?Y?ld+T9u*E9tZqGf-WxMe+`?s>-
zQBqjGCxlq`-xZjL^3)T6fEamDA~k=o5S+~d@=BBtD3n=_y(K-_;Z#iqJ!I!?h5soT
zw0)C`p(nfY_5zh&BX{zH7A`beI{D5_buk$GBlvr-%sX9+Fclmz7c!qSteyd+A#WLz7&h7tc?n~pb+}n0<5;7FZRAelYxRDa2LJ~rx%yY8Ll{r%~
zW|lEYL{b?d$~=cCR1_goA?AT*rA+3D8np`*N2+F_y;A$zgYXj6s~`#JpFefp8!z)gWLa~{n7286-0PH
zP8*l+f&>P`w5CyBqXwh_&cIsoLpvcfXaraj5FkaM)0AE3Zx2QM{ateZ`b!ehg8$jC
z|Nra<1t~~4=82)IuCD*asbW)7!g$nBHXXzO9GZ<>6LQ!8B3S!bwrs(5DY$13qghgf
zdGZn7(D3jnyN3V5w}Acd@y)H7^5bX0ra%h@cJx)w86v-(a;^ZSq$T*-7>j}%z(hlI*EKuhP5%cn_W$}vaaDJTh8ij^9R4H{P5`J2P?+O3Ciw$r
zh3)~q%8?VE&QKr;V{hZf-kwqAJ0tB?2BsJ5X+XIY;EdhgWU7J2%3$4p~Ju&KHmw
zN@A|>3OVOlCI^GNHotgS;Ak!}d1(PYNr2E~H0Hoz@X8((bWU#U;Xv#ErQJ1*v=Rm@7Y4<)c=&3n|q+GOB~h0erTi6Hu(9i0btzNaUJFi
zj9q}`(*QM@4mON0Oo}a1ddvHvc>~^#t^%QX3K@kt(j!gwj*jxU?LkhdP=ZH-j1iT~a^Mom
zBBy};>n>=!sU?IJoZk0A7H!1^#XK^jNnIHm*)kbIb^&QKG!3V^^!>GIRnQ)qL7AwIIN^V>4MuzPhTH0
z4hbt9l$MY5@W6~;rzR%44c7c+03Ykpk@SF9lG)4@;2osDpL!FPcnZC1wO}uz6h~tPJ1F)viPS*e7_xx1hg=zC9za8PsjMIT9_mRns1Fgo
z3H{%8Fp`ucFwes^0Hwg)5WL80&^APEgp4}Z-;B6>cjrg-^#h*FaNgl9<3rXwXC6Uw
z?FGCZ5?Ue@<#6w-S8>V7p_@9tf7$k34{VxI8o}&1hb_kzgGx|5w<1<+*d
zOo&dSA=H6RgIMR`O+|QVB_%IfTC}vae*hQ6^TZOMCt>;mB6CoM{sI$!N+N-8pGC;j%x{EWqP+zY06GG5
z$Fs63eSRWT{ly33%MTm@&Ju_+f#QQ%hf-l?ze69Wa)`K>N1EvWgxo>G^(>la(|UB(
zh@!xH#Ck=uzFC}tgnffJCN!3ad?04OBc@6
zKzNx}e*UCczdp?B0G2q^i|E`4a|3erfl;sS*^<4th;dAcAG>1JKVMb+!vlqrw
ziFfKbq3;;LI$t^z_aLH3tVN!Wj?Wg+5}3LN?`+5sUi1zA&CXq|fL+bQgmtwL68sp$
zGO!yVi&tDwvc>H2?$%b!q;&!E3t;%7yE|dzG#=c@eBq*%RaSZV58#95NlOS@Md!7m
zcivIK%57%w%lNnu{**)iz&2-xgz<4H3myNS^2`u1cJ||#ziE-OQ8rQkuKaYKuh~4
z&j}{Wsq
z!lkOOrx$Ae4tPEwFzOD)8K`hK1StT$M~nmhfuRBZp$Z^J$<_eUar`_bMgu`U8z+v@
zcu3wN(2p75oQSN!Xvzt}@lT&@ZESFWnsK-R8I9HMJkda&#;3zx$hw4?V1rnsx5Y3;
zj4d3Be>I^<`*qp6jTX2(Fz-kaP*qSpnm=GuGFc0Yi%sXZOdix!0to^V3KIsfrFZNI
z67$Is3D+*r62=4(fW3T`3D`>L>pO22tV2m(5&c17F+l^U+IE}y;cuZQqya^!^ZO3LiZ``n022Es!jeFrZ
z()?7(@A8fe-UHp%?VIROpeIZc%MRTOvDF@^!vJV;1Vc!Lv|}@lGbSd4_~wHb48lky
z1Zi)1$eX^ovGJyyd(1YCO$22#_8C)t?cdo}LG;KH+Tt
z>Gi{Rqz_~H0RN&5ZM?pc#dtF-4ypO1!d`pQ5#&v&Y&~*g?8=%iX2PR90e6CijO35Y
z1N0x{GwVqB$MQ$C*u44x#-Wk|Ty4~W!2%UNyka=gaTnp|Nd7?BfvJHb!&iJ_q8J!}
z^w$k_4Hi(L;O+oxG(pHYyy<9vvt$N(V94G*>Z^af~v7Qy!7m%`r*&
zxRk4Z4w7oKdkt3tcmtxxC(3%>J=>88hP3<6j~5<7}j5((I3{yp}b4a}Sf>cAAE?Cf)Z?f?M{
zZU<~IWb_~|j<85WZlUtbu_g#?-&4>tvU0o9TD44-7B~z96uDz3PcJ|^Y=SY^fIWbu
zSfOJ*eOhVU;eunLW5%+40E;=EDeC3y%EumaBM^=H{$53)w^6JXLkts=E#?fltJl?%
z+X0kem=&ZaK;Z_=me+`Roh^MTU<^?CLF(alhmDy?q?($-2csZCQ;vJVSf|;#_>o5b
zG|X|p+ks1t8Uh~1x(nhMm|1NU2Fx|W5ir8536%F1`KhcMAa
zduuPX^#k4&PGDD98}rxCa&;)GHEMQ{zG`0>tBvBMoLP^q#MyTd`@8FAo9fZ`_Iy$n^1>tUm^~9ouNC{ZgAH!>$
zno?6&9v?${fyih=Q-!)8Fge36PSj#Dkn{rO6H-qm`R&A2>1G>wVR^7VUQ4#vZt<9X
z)&9EkgrRocM@?aU-vk4KBP1i3{-
zmM}ISip@Lp(Ng%uq$A48!T1m;zwre>`rG%b7~U=OsT39WcOGY@d`xU#+dIX#H1k1R
zKwuQ^L|;Af#Cl_hyUnOBlsz0h;2q<*w;aog`6(WEr+IT
z*_=E?+3X-BC1karxuFHBsQ8KI3z`QBsO$?)AGHRFhw}lt+6hPuz|;V$t&2fMSI~;E
ze%4{?Z^XiXP*byps{xt+$vUr2nV2}9J68f`54SXvpCk;7Lp;p*%OhCKh*w|b@55fe
zOKO)5$Y#?q1}h4b=E~aC@Gwpt_|`VrR1VmEvrUQ+r}Ga7UblCa(vpMP2QoXfy}&Os
z;eZ$(esp0}8On*(W$By6SZeSYc1~iI&9dCw`)u0;UNIrwGBqW-ADrLBC-j*0e2%#611Bc99g
z$gS&1deA?>DS#|ntaPORgQ@a9Bn}GFkbC=(U)cDrXbQpx0Z7|tOPBsz~7uP|{XAEeiWdE_+
zCY;AINAg`oZ$*eTK>rIerj3V%rc~=B?68j1p_{fhspJIIC-FugQ16^Y>1N5sU>cJ1
zg$r;BU_1?QH*lz6H<3y-%YN`GU9iDzIeFw=qt3X5PmdJ1sHeV>!`m}-j*eTQ7Xx>X
zqjIGrE4lmV817j-SR|_OG64jK)APy^py8)YO=nGjrt|+@MT3o)9|zC$Hf$XiY9it@E<;d<)nYRom@X*s^KzJ~uoTq3D{8
z!_XrXyrFEcv_X8>iwDXPjQ_lqkb1ISol`VLx3$ZXdjvvIz6KRcV1iHUR$%awMY
z3wR0)tywWJ3B(nQSPOO$BtZQ_&(HA^;m&Bhk#tIoBM4fBYM)}DymAV!7&UI844fwy
zoSpS>4dUgl22+ira)Bkne&74fPV%2%V<<7PyumN0AMO~P8;uqitO;YJ0@0A$s
zi{9Rqkh#Fiiw|B1ISCraSV?VF+++lOP8x#tK&T+4p4<)qqQBo86&JX110?Q$Wm3gr
zX*sWSa2CXSB)1&O`(*(3bKkx_%+G&DM(Xr)#DjXVmIq9G^21XF)&l^r4^i=+bj>Id
z{s@rXyVjoq*8-Xuxc7>1NyW#XM)Qs6eK}3U_%&?mZS|b|9QBR?^N^a2qQ^w;Jfttu
zRH~I85kbj=pBZ|D@qp+tFO4!^$#>zG{yIENK{9)!M+C<_dxoMIvxw`Ylp38lkiNqI
zp)yVGr1k)x%@H&$yiyuBdxcd1`%O%pc+ilXfj-O
zrkTWEGn+k-rdPfsps@ff9y}AyKu{*c6agIAi0q_p()KcJ-`OP{+E(G^+EfBndH~|;
z|AxzeQEAXMgD>IgSuBSk{&#rK8<+KL933HV4i2ZT2tqZBZSys5X%SKaI4pF2_f2fY
ztz%m4ZLR|H9JX=Ucm!?)DwyNQ$LGH4X`$MHMh;^BN+9uM1U&C|YBIv8bkg}kjL&pT5XI+2G
zgq>Af8Zz6JmrQc*7nPN9uBmg0UEfIuWL3%dkUKrlX%uAIcV{|Nj309i
zs|!0OI0h9}S*K;DMzOMvM0W{iv^#)&Ao8zW+{F?60*4`zP-jJ|*Okf`ns3^=_g>3|kdtr^H*40Mw
z{GkuV(u>j;8#}`HualUHRX=Z+*greFV}ifg_{`ByXc6(#0QP7|6Q4g*lJLJDl8hHq
zoU}rxA))h10)2M>hQ=HweU<3JdK7#p+H%xCWJpz|G))4nwe@
zI>YZq$Ah~Kg3sj=V?Q;omXx(pSqm^QF)_|x`lO#So>|Ml0gZfaYz%zwtj~rKQxqre
zL*uEitl*R|H%+~|A_RU0GFk*v*+2k|uy}ITR
zYCM1V%Q*EPsN7HI|A2sdIW;X}AEHWc8Ao#D6WkMyXCavWLb#VbOf_G(eXb6MXbyfi
zobR}A7Xw$nUOcwH{F|K;k?#QZ2E~)}I3pD|TwLggfZOmh?~&^sL@|nLZs-x_)!|^P
zFl&8#n$vg}Q`*Rk$J=k$&wI>7#v!o@s1Y6y&N3LK_*v`<#K?eP)Un&#H*6WUGde12zrXx3
zeWtS-G`Nku2dIdtL7+Z?LB=km**c-}L8n1xm$~>ORb>zjceSD@rHy%
z*M|@3Tpn3ECfA~)-JP9bXDCC@g~Cuc{8@~#(%F9_MOTA|&pb5}>tc;T0fhlDK;uzW
zpr3gTh{jfETRJ!(_IQq413bl!$#ziDB;onN^@T4SZ@G8wEI{Z4{0T_eNhIdB
zwW&4MBl(e$5p{tZu0ZT64~BIm#$Yrw#Wf5gqoajX*>bAFuFB{t=ZklCi)_A9mOw{E
z8SB?NpHf9-bL*H@ReZ-I4c3sl>+y^4&AVhD2IcEg^jgO23;$B~G>uswmv+KimE2#N
z=5W}UVydZdv-AEqnEVP1O6ZiMVaKPhM2U*X#4aS^A=Mq7xW%Kv8b5@CKr0K4ZLqmQ
z29d9Y96M*4S(rKDYQzu;48sBB&LMcas7PB)Z8tFVhCzFC2tF9ec>6vxL??i%G|reJi0hEomIm^Q
zT?VQt>;QQ$QA9S!kAogTT?Q>Y;8F<&R8+4(G~K^nPbTuQv7pA7WKq$kOIT7L
z#8k{DaJ4mKV>qku`5IygVC1UIPPfZ<1VM$7oJSfrtP~{c?!1`leKKJ<+j<}CCPcir
z5V-(MISDLl+1JHKc4Ue@SIv2&<{K5J&&4IHDfz}JYh|l
z5#F;WXif*Z0Nhvz_PDtTpv0BAa!kHEj|@$q?gR=7RD^j;sMFE>%*-6SzwdHjbN4!6bVe7JU@Ru){)IByauhM%-ykV{FuG~QKQWgz@w<`hkk%|#Dyj6
z%HKPA`*Gk)HFD^-?fKmR2&=1aiC!0z%gzVk2%)wg07!tTKy5Z@i4=Xmj1unVMi4aE
zXjnPtoCTOOxnvtA#$Jb6#56G6B9dl&Wt>4=K|%t6l5_-R1PH^YM)}ZesVJ+8pJGeT
z-_gKZXR#fS7*KGaCM*1Bjvp6-FBPAVWcDv4isow6C=neGt`f2joL3;6Fu4q2?x;fX
zDxH(opm}>)sw?q6Rpy&-!f@zo)v(Ui^H1+giljYEnm^E!JJRVHm*FcXy0uub+-SX0
zG`Flv=*^&pVXYsGo&D_mx<}l0uGDPedj7uDPGdsUhs`aZ$pYhpV2IlOdv}q;KY(xn
z9U<=MAQTHg)nbc*sWL!a3i)We9VLX8^$iVY^9;^8IufcUAm>`vNUQ&g33akOWjQ(4
zU65QVVPAJ2#Fc;mTs&~UE9GdcavL=*YeN!hU%ldaVQjQ_mn+Dnx^7Q!il|EnoOeqR
z{J{{#<)`q*~6BiLNeq)7>k=>lGo{hC^D;wK()NMQ;i}J*}|Q`vRAMr+1g`ecM>)
zcY_%tU!N2e^?_@Kz*E+UBQqldV)HBFChW9S{D(848v+FgXksIeJi@YmVN}L*QCw7%
zS9rm6-3nYhxpDg4uJRhl_j!;;0%RQj(%so_Krrs*EvaOsy@sm*}vTk?5x-)VOb)0ngq8j95p6>Y=q0B9I?
zr_R5{fP>obAE+a7Z2s{=qF{y3vr85zWoZL(I7d*w;c7Qr;ANipPeUUCQwuAmzk1p4+8b^b0=eAD>d4@P1YY
z+&>JrX-`nG(C_9CdD##@-?VE>?`eLQ#Gz(L&xM7Bfoozc4pd0ONH-
z^k-Wu?r~9L?L3!Jp-+0@!>7I8>U`Ym7$?Iyr|$)nQ+Xeap|8a~w3k$i7K5KZz+(~y
z4_p!0VNjpckso3jF<>fGLa3gwv~a*cL+}f|7X^uEEn8YvzD;79>n-5@Pz>BpdP)VkRhlNQt)%1pNrx6S%8!ly6wPV3K%~
z0$%OQEXVhhjcTa+7H|w>`E0cr2@h;i?Q>=IF6gck-zsB%)rlBwyG}t*W9#MU}!+
zJFz#|hWC!}sv|+SKfo#l$9pM)cD)Cz4`6Nuf2@Ds%|M3)R|$%JLV1i!Z;r+
zY_#k!xgdq}9o0n{y9cYR7TG&w!bO#DZQoJqA$d1jJGb$Kg2|Kq@h@io{1Y}O#Ts5_
zOKn+=fqrtYI{16eMXl$Mv4
zpN(wVX}|0o{&u|~`Gr|!S+9BJv3rV=(@E3LfO;?`5s>@az8L8tsN~Ugq7V}B`~>Nx
zc=gH%(7d-n-XuMDocIv-#uhJ--iDN$&fj?Pv|5wQ!2pe*$H|+l`R0$eZ=k1dLr^))
z2iLCMNKb!jvlq+sfsBc)fjsG<9H?f-q}m_rgRPpOX)x5*4q%c6Le4%O8)(WS<&w`Eg$Crc!61}`u0iuQ9d*ebdm8S>@*d0sQ@I
zR_6&~6fV#lWzA1hC+q1l867g>;qaTLz_Y3MfGHg&p5n?b57z=N*ZhIVOjbQ)~>%dp=?n0uKwKax80s=v6=liYe
zcq3hpn29l0A3Kb*przr25(%*tqVn6}oD&mn#s(wpxr+&A483jN_(`Oj{!<%US;D9a
zVyCE}8*>^chZ=>Ihi47osg7w@#tmkaZX6sFGE#ypCva}7t1~EIPT2`T7Ix2s-Yy?z
zY(@A3^7^{!O!>HWjmaiAeFwj2>a)n|sq*DqG4U&|>Ns&J-#L|`#T#fP+1uDiRbm4r
z8?I{L6f4d+X9QVV3F&YK#+}mzoq2GKcDQ3}B=uPVT1xSc*Om>A(_%OyJfv_S+WOQ=?_8Kfvm4`_J#G-gT7eok5CvupwU{BJo>VLVMjEy$qF)Ss$&fZnKiCGU?zj
zFU}t;{a_`DkDIOCy3!GnV*4|oKs#4vKICiL=eta9er|3(W@ONwrw;b|B#JP68DjtlY&Gjx2H{R>G&xq@bO<7Xy&4ZjwJj3w1pmRR98b*HG63
zw8F3f1ZlozH}&r7?FDKGZ{V>f8PIVf6vE7GDAlPNiO4h4(@M02y$20EUWh2${=Iv(
zrgUEL?%%-4!-GpsSX9&-I=0pyT=ys4BztCNW-w8@$fSTh9GtQY3PuqTTYLKm0Y?`;
zKhO__B!qXFD{t(}s#v6)aBvvJG>hOGvK)TNo=WpaH7@3$v9VQ+s
z4w{uqW@ctCE{{zLWfU<|nHfXL
zKhbSClp7AGq0$>ReYOvRT4`xkcJ{ZJwT%4#rAwT_{pH=$#YcEO=39lZK+)g9^8m#@
zI3;``+1u3bm+Sm_Lvi?E&(|NhMVNp?o@V_e9
zKxPiq4egBmzIV*$FNW*O5=-o&&;|$@9`>qfLDg};60%j&R59B>IFB!29Q4T*rNg{2tJ1p)3&<{q*s^vj76%}%Y
zORiH~Ov5^#o3#+C;arH(^gg*$=lF4&5=3;9w?-1pIn#$}2;W2vZ03U4!tbBe^Xm?J
z*TqIN43r;!4}epEkTD<`OX!U(m+^^am~>oR-k6c~9w*BEKJ!DNgQr|PN{x>tO(Un<
z7Dci{Bs*$tNFc`ARFv<`p&BUdGxNF0tai_#yCqE!imjn(&bMPHLssEPf7oS%DNtQD
z(DCgXe7KEoY;+VpE=S1}s1q>k1d8FKOo8eNdHT4d?_h{sP+%|GEl~RWf7j)a=
zGJIm+tI+-X^Me8egxmx{$^)g`0SSq@+?-nqv^Z}&tRy7{qwiT4mYJQBE@3_HBQ^Y{
zrf2uLhxR)=drGaCpm7Ddg@p|DD6DrFI^BJ;G~a5=&px{X=s$*@IMN2hsAgkn_s=~M
zU!moDwi?^^(YjhjIEYDM#gjte9`bU^#&0#$Y&UEl1icBp(=u|~ZUTEoJP%+@AVV+B
zujXCYr>0h__5ObBcD+C?m;vF1d(Oc{BK@WremH52s7)z_i7=sC_+3OA`w`xEP=$jV
z1-G-}U7eG29&vGcqFA*$%F5J7nVhN?6OjyvlnjGBc}&Pax*sDW(iBlg4`5_D$|VPf
zJp7ciGX}UwotY!TMB%4{s|02sc#AUM&*Gke(hA%oM(#X8sgshHwu2TEt434K-#SVQ
zoHRmSD%6%bmA?Pay@9FL5M+B3Bn;c%c*;HooX$6(-rK+w?e=X{&^gKpdVIT6OKd`lHG)d*q&
z>xhlY9Hs=~6$p_1++oNnx$BzApETXw_Ybfi&(eZ{2U`WAdt4KUAoQJUeLmf!8NjSi
zS6AnUiAN!frO(QWi#K=VBJ}pvq{|yExKo-Qw(N|IxPr}!%%*Q1l%yl5*(>={t|C)+
zKJDzm>(_Z{t4t0{Nh$W65=qz^8O%%j%0(HaILeyDT54aZZq@hg?RpuX6W*A*^7TB~
zan%e%XLnv#zJsL@!N^$=_Z|}+{xM21EID?dzq)fFt5dQY|YU3!VwO)fU#f?i96a64Y1CfsWdD?MC&gYzBuC&EF~+xm9ZU*(h>rsqK*w
zw^8s<6^vSCfb=1b0^eM_!=I0nD0z)2IsR{d;H*f?8L6CiMh+BJ&okniYk#eGJ