diff --git a/.coveragerc b/.coveragerc new file mode 100644 index 000000000..2398f62e3 --- /dev/null +++ b/.coveragerc @@ -0,0 +1,3 @@ +[report] +omit = + tests/* \ No newline at end of file diff --git a/.gitignore b/.gitignore index 84d9a0eea..58e83214e 100644 --- a/.gitignore +++ b/.gitignore @@ -44,6 +44,7 @@ nosetests.xml coverage.xml *,cover .hypothesis/ +*.pytest_cache # Translations *.mo diff --git a/.travis.yml b/.travis.yml index e374eff1f..e465e8e4c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,23 +1,19 @@ -language: - - python +language: python python: - - "3.4" + - 3.5 + - 3.6 + - 3.7 + - 3.8 before_install: - git submodule update --remote install: - - pip install six - - pip install flake8 - - pip install ipython - - pip install matplotlib - - pip install networkx - - pip install ipywidgets - - pip install Pillow + - pip install --upgrade -r requirements.txt script: - - py.test + - py.test --cov=./ - python -m doctest -v *.py after_success: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index df8b94881..f92643700 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,7 +1,7 @@ How to Contribute to aima-python ========================== -Thanks for considering contributing to `aima-python`! Whether you are an aspiring [Google Summer of Code](https://summerofcode.withgoogle.com/organizations/5674023002832896/) student, or an independent contributor, here is a guide on how you can help. +Thanks for considering contributing to `aima-python`! Whether you are an aspiring [Google Summer of Code](https://summerofcode.withgoogle.com/organizations/5431334980288512/) student, or an independent contributor, here is a guide on how you can help. First of all, you can read these write-ups from past GSoC students to get an idea about what you can do for the project. [Chipe1](https://github.com/aimacode/aima-python/issues/641) - [MrDupin](https://github.com/aimacode/aima-python/issues/632) @@ -23,7 +23,7 @@ In more detail: ## Port to Python 3; Pythonic Idioms -- Check for common problems in [porting to Python 3](http://python3porting.com/problems.html), such as: `print` is now a function; `range` and `map` and other functions no longer produce `list`s; objects of different types can no longer be compared with `<`; strings are now Unicode; it would be nice to move `%` string formatting to `.format`; there is a new `next` function for generators; integer division now returns a float; we can now use set literals. +- Check for common problems in [porting to Python 3](http://python3porting.com/problems.html), such as: `print` is now a function; `range` and `map` and other functions no longer produce `list`; objects of different types can no longer be compared with `<`; strings are now Unicode; it would be nice to move `%` string formatting to `.format`; there is a new `next` function for generators; integer division now returns a float; we can now use set literals. - Replace old Lisp-based idioms with proper Python idioms. For example, we have many functions that were taken directly from Common Lisp, such as the `every` function: `every(callable, items)` returns true if every element of `items` is callable. This is good Lisp style, but good Python style would be to use `all` and a generator expression: `all(callable(f) for f in items)`. Eventually, fix all calls to these legacy Lisp functions and then remove the functions. ## New and Improved Algorithms @@ -84,6 +84,8 @@ Patch Rules without your patch. - Follow the style guidelines described above. +- Refer the issue you have fixed. +- Explain in brief what changes you have made with affected files name. # Choice of Programming Languages diff --git a/README.md b/README.md index 9a29ac4a6..17f1d6085 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,42 @@ -
-

-
+ # `aima-python` [![Build Status](https://travis-ci.org/aimacode/aima-python.svg?branch=master)](https://travis-ci.org/aimacode/aima-python) [![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/aimacode/aima-python) Python code for the book *[Artificial Intelligence: A Modern Approach](http://aima.cs.berkeley.edu).* You can use this in conjunction with a course on AI, or for study on your own. We're looking for [solid contributors](https://github.com/aimacode/aima-python/blob/master/CONTRIBUTING.md) to help. +# Updates for 4th Edition + +The 4th edition of the book as out now in 2020, and thus we are updating the code. All code here will reflect the 4th edition. Changes include: + +- Move from Python 3.5 to 3.7. +- More emphasis on Jupyter (Ipython) notebooks. +- More projects using external packages (tensorflow, etc.). -## Structure of the Project -When complete, this project will have Python implementations for all the pseudocode algorithms in the book, as well as tests and examples of use. For each major topic, such as `nlp` (natural language processing), we provide the following files: +# Structure of the Project -- `nlp.py`: Implementations of all the pseudocode algorithms, and necessary support functions/classes/data. -- `tests/test_nlp.py`: A lightweight test suite, using `assert` statements, designed for use with [`py.test`](http://pytest.org/latest/), but also usable on their own. -- `nlp.ipynb`: A Jupyter (IPython) notebook that explains and gives examples of how to use the code. -- `nlp_apps.ipynb`: A Jupyter notebook that gives example applications of the code. +When complete, this project will have Python implementations for all the pseudocode algorithms in the book, as well as tests and examples of use. For each major topic, such as `search`, we provide the following files: +- `search.ipynb` and `search.py`: Implementations of all the pseudocode algorithms, and necessary support functions/classes/data. The `.py` file is generated automatically from the `.ipynb` file; the idea is that it is easier to read the documentation in the `.ipynb` file. +- `search_XX.ipynb`: Notebooks that show how to use the code, broken out into various topics (the `XX`). +- `tests/test_search.py`: A lightweight test suite, using `assert` statements, designed for use with [`py.test`](http://pytest.org/latest/), but also usable on their own. -## Python 3.4 and up +# Python 3.7 and up -This code requires Python 3.4 or later, and does not run in Python 2. You can [install Python](https://www.python.org/downloads) or use a browser-based Python interpreter such as [repl.it](https://repl.it/languages/python3). +The code for the 3rd edition was in Python 3.5; the current 4th edition code is in Python 3.7. It should also run in later versions, but does not run in Python 2. You can [install Python](https://www.python.org/downloads) or use a browser-based Python interpreter such as [repl.it](https://repl.it/languages/python3). You can run the code in an IDE, or from the command line with `python -i filename.py` where the `-i` option puts you in an interactive loop where you can run Python functions. All notebooks are available in a [binder environment](http://mybinder.org/repo/aimacode/aima-python). Alternatively, visit [jupyter.org](http://jupyter.org/) for instructions on setting up your own Jupyter notebook environment. +Features from Python 3.6 and 3.7 that we will be using for this version of the code: +- [f-strings](https://docs.python.org/3.6/whatsnew/3.6.html#whatsnew36-pep498): all string formatting should be done with `f'var = {var}'`, not with `'var = {}'.format(var)` nor `'var = %s' % var`. +- [`typing` module](https://docs.python.org/3.7/library/typing.html): declare functions with type hints: `def successors(state) -> List[State]:`; that is, give type declarations, but omit them when it is obvious. I don't need to say `state: State`, but in another context it would make sense to say `s: State`. +- Underscores in numerics: write a million as `1_000_000` not as `1000000`. +- [`dataclasses` module](https://docs.python.org/3.7/library/dataclasses.html#module-dataclasses): replace `namedtuple` with `dataclass`. + + +[//]: # (There is a sibling [aima-docker]https://github.com/rajatjain1997/aima-docker project that shows you how to use docker containers to run more complex problems in more complex software environments.) + ## Installation Guide @@ -31,10 +44,16 @@ To download the repository: `git clone https://github.com/aimacode/aima-python.git` -You also need to fetch the datasets from the [`aima-data`](https://github.com/aimacode/aima-data) repository: +Then you need to install the basic dependencies to run the project on your system: ``` cd aima-python +pip install -r requirements.txt +``` + +You also need to fetch the datasets from the [`aima-data`](https://github.com/aimacode/aima-data) repository: + +``` git submodule init git submodule update ``` @@ -168,7 +187,7 @@ Here is a table of the implemented data structures, the figure, name of the impl # Acknowledgements -Many thanks for contributions over the years. I got bug reports, corrected code, and other support from Darius Bacon, Phil Ruggera, Peng Shao, Amit Patil, Ted Nienstedt, Jim Martin, Ben Catanzariti, and others. Now that the project is on GitHub, you can see the [contributors](https://github.com/aimacode/aima-python/graphs/contributors) who are doing a great job of actively improving the project. Many thanks to all contributors, especially @darius, @SnShine, @reachtarunhere, @MrDupin, @Chipe1, @ad71 and @MariannaSpyrakou. +Many thanks for contributions over the years. I got bug reports, corrected code, and other support from Darius Bacon, Phil Ruggera, Peng Shao, Amit Patil, Ted Nienstedt, Jim Martin, Ben Catanzariti, and others. Now that the project is on GitHub, you can see the [contributors](https://github.com/aimacode/aima-python/graphs/contributors) who are doing a great job of actively improving the project. Many thanks to all contributors, especially [@darius](https://github.com/darius), [@SnShine](https://github.com/SnShine), [@reachtarunhere](https://github.com/reachtarunhere), [@antmarakis](https://github.com/antmarakis), [@Chipe1](https://github.com/Chipe1), [@ad71](https://github.com/ad71) and [@MariannaSpyrakou](https://github.com/MariannaSpyrakou). [agents]:../master/agents.py diff --git a/agents.ipynb b/agents.ipynb index 5ce0502da..636df75e3 100644 --- a/agents.ipynb +++ b/agents.ipynb @@ -4,32 +4,120 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "# Intelligent Agents #\n", "\n", - "# AGENT #\n", + "This notebook serves as supporting material for topics covered in **Chapter 2 - Intelligent Agents** from the book *Artificial Intelligence: A Modern Approach.* This notebook uses implementations from [agents.py](https://github.com/aimacode/aima-python/blob/master/agents.py) module. Let's start by importing everything from agents module." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from agents import *\n", + "from notebook import psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "\n", + "* Overview\n", + "* Agent\n", + "* Environment\n", + "* Simple Agent and Environment\n", + "* Agents in a 2-D Environment\n", + "* Wumpus Environment\n", "\n", - "An agent, as defined in 2.1 is anything that can perceive its environment through sensors, and act upon that environment through actuators based on its agent program. This can be a dog, robot, or even you. As long as you can perceive the environment and act on it, you are an agent. This notebook will explain how to implement a simple agent, create an environment, and create a program that helps the agent act on the environment based on its percepts.\n", + "## OVERVIEW\n", "\n", - "Before moving on, review the Agent and Environment classes in [agents.py](https://github.com/aimacode/aima-python/blob/master/agents.py).\n", + "An agent, as defined in 2.1, is anything that can perceive its environment through sensors, and act upon that environment through actuators based on its agent program. This can be a dog, a robot, or even you. As long as you can perceive the environment and act on it, you are an agent. This notebook will explain how to implement a simple agent, create an environment, and implement a program that helps the agent act on the environment based on its percepts.\n", "\n", - "Let's begin by importing all the functions from the agents.py module and creating our first agent - a blind dog." + "## AGENT\n", + "\n", + "Let us now see how we define an agent. Run the next cell to see how `Agent` is defined in agents module." ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Can't find a valid program for BlindDog, falling back to default.\n" - ] - } - ], + "outputs": [], "source": [ - "from agents import *\n", + "psource(Agent)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `Agent` has two methods.\n", + "* `__init__(self, program=None)`: The constructor defines various attributes of the Agent. These include\n", + "\n", + " * `alive`: which keeps track of whether the agent is alive or not \n", + " \n", + " * `bump`: which tracks if the agent collides with an edge of the environment (for eg, a wall in a park)\n", + " \n", + " * `holding`: which is a list containing the `Things` an agent is holding, \n", + " \n", + " * `performance`: which evaluates the performance metrics of the agent \n", + " \n", + " * `program`: which is the agent program and maps an agent's percepts to actions in the environment. If no implementation is provided, it defaults to asking the user to provide actions for each percept.\n", + " \n", + "* `can_grab(self, thing)`: Is used when an environment contains things that an agent can grab and carry. By default, an agent can carry nothing.\n", + "\n", + "## ENVIRONMENT\n", + "Now, let us see how environments are defined. Running the next cell will display an implementation of the abstract `Environment` class." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(Environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Environment` class has lot of methods! But most of them are incredibly simple, so let's see the ones we'll be using in this notebook.\n", + "\n", + "* `thing_classes(self)`: Returns a static array of `Thing` sub-classes that determine what things are allowed in the environment and what aren't\n", + "\n", + "* `add_thing(self, thing, location=None)`: Adds a thing to the environment at location\n", + "\n", + "* `run(self, steps)`: Runs an environment with the agent in it for a given number of steps.\n", + "\n", + "* `is_done(self)`: Returns true if the objective of the agent and the environment has been completed\n", + "\n", + "The next two functions must be implemented by each subclasses of `Environment` for the agent to recieve percepts and execute actions \n", + "\n", + "* `percept(self, agent)`: Given an agent, this method returns a list of percepts that the agent sees at the current time\n", "\n", + "* `execute_action(self, agent, action)`: The environment reacts to an action performed by a given agent. The changes may result in agent experiencing new percepts or other elements reacting to agent input." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## SIMPLE AGENT AND ENVIRONMENT\n", + "\n", + "Let's begin by using the `Agent` class to creating our first agent - a blind dog." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "class BlindDog(Agent):\n", " def eat(self, thing):\n", " print(\"Dog: Ate food at {}.\".format(self.location))\n", @@ -49,17 +137,9 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "True\n" - ] - } - ], + "outputs": [], "source": [ "print(dog.alive)" ] @@ -76,17 +156,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# ENVIRONMENT #\n", + "### ENVIRONMENT - Park\n", "\n", - "A park is an example of an environment because our dog can perceive and act upon it. The Environment class in agents.py is an abstract class, so we will have to create our own subclass from it before we can use it. The abstract class must contain the following methods:\n", - "\n", - "
  • percept(self, agent) - returns what the agent perceives
  • \n", - "
  • execute_action(self, agent, action) - changes the state of the environment based on what the agent does.
  • " + "A park is an example of an environment because our dog can perceive and act upon it. The Environment class is an abstract class, so we will have to create our own subclass from it before we can use it." ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -132,32 +209,15 @@ }, { "cell_type": "markdown", - "metadata": { - "collapsed": true - }, + "metadata": {}, "source": [ - "# PROGRAM - BlindDog #\n", - "Now that we have a Park Class, we need to implement a program module for our dog. A program controls how the dog acts upon it's environment. Our program will be very simple, and is shown in the table below.\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
    Percept: Feel Food Feel WaterFeel Nothing
    Action: eatdrinkmove down
    \n" + "### PROGRAM - BlindDog\n", + "Now that we have a Park Class, we re-implement our BlindDog to be able to move down and eat food or drink water only if it is present.\n" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -177,10 +237,39 @@ " ''' returns True upon success or False otherwise'''\n", " if isinstance(thing, Water):\n", " return True\n", - " return False\n", + " return False" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now its time to implement a program module for our dog. A program controls how the dog acts upon its environment. Our program will be very simple, and is shown in the table below.\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", " \n", + "
    Percept: Feel Food Feel WaterFeel Nothing
    Action: eatdrinkmove down
    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "def program(percepts):\n", - " '''Returns an action based on it's percepts'''\n", + " '''Returns an action based on the dog's percepts'''\n", " for p in percepts:\n", " if isinstance(p, Food):\n", " return 'eat'\n", @@ -198,21 +287,9 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "BlindDog decided to move down at location: 1\n", - "BlindDog decided to move down at location: 2\n", - "BlindDog decided to move down at location: 3\n", - "BlindDog decided to move down at location: 4\n", - "BlindDog ate Food at location: 5\n" - ] - } - ], + "outputs": [], "source": [ "park = Park()\n", "dog = BlindDog(program)\n", @@ -236,19 +313,9 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "BlindDog decided to move down at location: 5\n", - "BlindDog decided to move down at location: 6\n", - "BlindDog drank Water at location: 7\n" - ] - } - ], + "outputs": [], "source": [ "park.run(5)" ] @@ -262,25 +329,9 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "BlindDog decided to move down at location: 7\n", - "BlindDog decided to move down at location: 8\n", - "BlindDog decided to move down at location: 9\n", - "BlindDog decided to move down at location: 10\n", - "BlindDog decided to move down at location: 11\n", - "BlindDog decided to move down at location: 12\n", - "BlindDog decided to move down at location: 13\n", - "BlindDog decided to move down at location: 14\n", - "BlindDog drank Water at location: 15\n" - ] - } - ], + "outputs": [], "source": [ "park.add_thing(water, 15)\n", "park.run(10)" @@ -290,22 +341,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This is how to implement an agent, its program, and environment. However, this was a very simple case. Let's try a 2-Dimentional environment now with multiple agents.\n", + "Above, we learnt to implement an agent, its program, and an environment on which it acts. However, this was a very simple case. Let's try to add complexity to it by creating a 2-Dimensional environment!\n", "\n", "\n", - "# 2D Environment #\n", - "To make our Park 2D, we will need to make it a subclass of XYEnvironment instead of Environment. Please note that our park is indexed in the 4th quadrant of the X-Y plane.\n", + "## AGENTS IN A 2D ENVIRONMENT\n", "\n", - "We will also eventually add a person to pet the dog." + "For us to not read so many logs of what our dog did, we add a bit of graphics while making our Park 2D. To do so, we will need to make it a subclass of GraphicEnvironment instead of Environment. Parks implemented by subclassing GraphicEnvironment class adds these extra properties to it:\n", + "\n", + " - Our park is indexed in the 4th quadrant of the X-Y plane.\n", + " - Every time we create a park subclassing GraphicEnvironment, we need to define the colors of all the things we plan to put into the park. The colors are defined in typical [RGB digital 8-bit format](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations), common across the web.\n", + " - Fences are added automatically to all parks so that our dog does not go outside the park's boundary - it just isn't safe for blind dogs to be outside the park by themselves! GraphicEnvironment provides `is_inbounds` function to check if our dog tries to leave the park.\n", + " \n", + "First let us try to upgrade our 1-dimensional `Park` environment by just replacing its superclass by `GraphicEnvironment`. " ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "class Park2D(XYEnvironment):\n", + "class Park2D(GraphicEnvironment):\n", " def percept(self, agent):\n", " '''return a list of things that are in our agent's location'''\n", " things = self.list_things_at(agent.location)\n", @@ -355,56 +411,23 @@ " ''' returns True upon success or False otherwise'''\n", " if isinstance(thing, Water):\n", " return True\n", - " return False\n", - " \n", - "def program(percepts):\n", - " '''Returns an action based on it's percepts'''\n", - " for p in percepts:\n", - " if isinstance(p, Food):\n", - " return 'eat'\n", - " elif isinstance(p, Water):\n", - " return 'drink'\n", - " return 'move down'" + " return False" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's test this new park with our same dog, food and water" + "Now let's test this new park with our same dog, food and water. We color our dog with a nice red and mark food and water with orange and blue respectively." ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "BlindDog decided to move down at location: [0, 1]\n", - "BlindDog decided to move down at location: [0, 2]\n", - "BlindDog decided to move down at location: [0, 3]\n", - "BlindDog decided to move down at location: [0, 4]\n", - "BlindDog ate Food at location: [0, 5]\n", - "BlindDog decided to move down at location: [0, 5]\n", - "BlindDog decided to move down at location: [0, 6]\n", - "BlindDog drank Water at location: [0, 7]\n", - "BlindDog decided to move down at location: [0, 7]\n", - "BlindDog decided to move down at location: [0, 8]\n", - "BlindDog decided to move down at location: [0, 9]\n", - "BlindDog decided to move down at location: [0, 10]\n", - "BlindDog decided to move down at location: [0, 11]\n", - "BlindDog decided to move down at location: [0, 12]\n", - "BlindDog decided to move down at location: [0, 13]\n", - "BlindDog decided to move down at location: [0, 14]\n", - "BlindDog drank Water at location: [0, 15]\n" - ] - } - ], + "outputs": [], "source": [ - "park = Park2D(5,20) # park width is set to 5, and height to 20\n", + "park = Park2D(5,20, color={'BlindDog': (200,0,0), 'Water': (0, 200, 200), 'Food': (230, 115, 40)}) # park width is set to 5, and height to 20\n", "dog = BlindDog(program)\n", "dogfood = Food()\n", "water = Water()\n", @@ -413,6 +436,7 @@ "park.add_thing(water, [0,7])\n", "morewater = Water()\n", "park.add_thing(morewater, [0,15])\n", + "print(\"BlindDog starts at (1,1) facing downwards, lets see if he can find any food!\")\n", "park.run(20)" ] }, @@ -420,11 +444,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This works, but our blind dog doesn't make any use of the 2 dimensional space available to him. Let's make our dog more energetic so that he turns and moves forward, instead of always moving down. We'll also need to make appropriate changes to our environment to be able to handle this extra motion.\n", + "Adding some graphics was a good idea! We immediately see that the code works, but our blind dog doesn't make any use of the 2 dimensional space available to him. Let's make our dog more energetic so that he turns and moves forward, instead of always moving down. In doing so, we'll also need to make some changes to our environment to be able to handle this extra motion.\n", "\n", - "# PROGRAM - EnergeticBlindDog #\n", + "### PROGRAM - EnergeticBlindDog\n", "\n", - "Let's make our dog turn or move forwards at random - except when he's at the edge of our park - in which case we make him change his direction explicitly by turning to avoid trying to leave the park. Our dog is blind, however, so he wouldn't know which way to turn - he'd just have to try arbitrarily.\n", + "Let's make our dog turn or move forwards at random - except when he's at the edge of our park - in which case we make him change his direction explicitly by turning to avoid trying to leave the park. However, our dog is blind so he wouldn't know which way to turn - he'd just have to try arbitrarily.\n", "\n", "\n", " \n", @@ -458,22 +482,19 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from random import choice\n", "\n", - "turn = False # global variable to remember to turn if our dog hits the boundary\n", "class EnergeticBlindDog(Agent):\n", " location = [0,1]\n", " direction = Direction(\"down\")\n", " \n", " def moveforward(self, success=True):\n", - " '''moveforward possible only if success (ie valid destination location)'''\n", - " global turn\n", + " '''moveforward possible only if success (i.e. valid destination location)'''\n", " if not success:\n", - " turn = True # if edge has been reached, remember to turn\n", " return\n", " if self.direction.direction == Direction.R:\n", " self.location[0] += 1\n", @@ -501,17 +522,17 @@ " \n", "def program(percepts):\n", " '''Returns an action based on it's percepts'''\n", - " global turn\n", + " \n", " for p in percepts: # first eat or drink - you're a dog!\n", " if isinstance(p, Food):\n", " return 'eat'\n", " elif isinstance(p, Water):\n", " return 'drink'\n", - " if turn: # then recall if you were at an edge and had to turn\n", - " turn = False\n", - " choice = random.choice((1,2));\n", - " else:\n", - " choice = random.choice((1,2,3,4)) # 1-right, 2-left, others-forward\n", + " if isinstance(p,Bump): # then check if you are at an edge and have to turn\n", + " turn = False\n", + " choice = random.choice((1,2));\n", + " else:\n", + " choice = random.choice((1,2,3,4)) # 1-right, 2-left, others-forward\n", " if choice == 1:\n", " return 'turnright'\n", " elif choice == 2:\n", @@ -525,19 +546,33 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "### ENVIRONMENT - Park2D\n", + "\n", "We also need to modify our park accordingly, in order to be able to handle all the new actions our dog wishes to execute. Additionally, we'll need to prevent our dog from moving to locations beyond our park boundary - it just isn't safe for blind dogs to be outside the park by themselves." ] }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "class Park2D(XYEnvironment):\n", + "class Park2D(GraphicEnvironment):\n", " def percept(self, agent):\n", " '''return a list of things that are in our agent's location'''\n", " things = self.list_things_at(agent.location)\n", + " loc = copy.deepcopy(agent.location) # find out the target location\n", + " #Check if agent is about to bump into a wall\n", + " if agent.direction.direction == Direction.R:\n", + " loc[0] += 1\n", + " elif agent.direction.direction == Direction.L:\n", + " loc[0] -= 1\n", + " elif agent.direction.direction == Direction.D:\n", + " loc[1] += 1\n", + " elif agent.direction.direction == Direction.U:\n", + " loc[1] -= 1\n", + " if not self.is_inbounds(loc):\n", + " things.append(Bump())\n", " return things\n", " \n", " def execute_action(self, agent, action):\n", @@ -549,21 +584,8 @@ " print('{} decided to {} at location: {}'.format(str(agent)[1:-1], action, agent.location))\n", " agent.turn(Direction.L)\n", " elif action == 'moveforward':\n", - " loc = copy.deepcopy(agent.location) # find out the target location\n", - " if agent.direction.direction == Direction.R:\n", - " loc[0] += 1\n", - " elif agent.direction.direction == Direction.L:\n", - " loc[0] -= 1\n", - " elif agent.direction.direction == Direction.D:\n", - " loc[1] += 1\n", - " elif agent.direction.direction == Direction.U:\n", - " loc[1] -= 1\n", - " if self.is_inbounds(loc):# move only if the target is a valid location\n", - " print('{} decided to move {}wards at location: {}'.format(str(agent)[1:-1], agent.direction.direction, agent.location))\n", - " agent.moveforward()\n", - " else:\n", - " print('{} decided to move {}wards at location: {}, but couldn\\'t'.format(str(agent)[1:-1], agent.direction.direction, agent.location))\n", - " agent.moveforward(False)\n", + " print('{} decided to move {}wards at location: {}'.format(str(agent)[1:-1], agent.direction.direction, agent.location))\n", + " agent.moveforward()\n", " elif action == \"eat\":\n", " items = self.list_things_at(agent.location, tclass=Food)\n", " if len(items) != 0:\n", @@ -587,735 +609,20 @@ " return dead_agents or no_edibles\n" ] }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "dog started at [0,0], facing down. Let's see if he found any food or water!\n", - "EnergeticBlindDog decided to turnright at location: [0, 0]\n", - "EnergeticBlindDog decided to move leftwards at location: [0, 0], but couldn't\n", - "EnergeticBlindDog decided to turnright at location: [0, 0]\n", - "EnergeticBlindDog decided to turnright at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to move upwards at location: [0, 0], but couldn't\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to turnright at location: [0, 0]\n", - "EnergeticBlindDog decided to turnright at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [0, 0]\n", - "EnergeticBlindDog decided to move rightwards at location: [0, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [1, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [1, 0]\n", - "EnergeticBlindDog decided to turnleft at location: [1, 0]\n", - "EnergeticBlindDog decided to move downwards at location: [1, 0]\n", - "EnergeticBlindDog decided to move downwards at location: [1, 1]\n", - "EnergeticBlindDog ate Food at location: [1, 2]\n" - ] - } - ], - "source": [ - "park = Park2D(3,3)\n", - "dog = EnergeticBlindDog(program)\n", - "dogfood = Food()\n", - "water = Water()\n", - "park.add_thing(dog, [0,0])\n", - "park.add_thing(dogfood, [1,2])\n", - "park.add_thing(water, [2,1])\n", - "morewater = Water()\n", - "park.add_thing(morewater, [0,2])\n", - "print(\"dog started at [0,0], facing down. Let's see if he found any food or water!\")\n", - "park.run(20)" - ] - }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This is good, but it still lacks graphics. What if we wanted to visualize our park as it changed? To do that, all we have to do is make our park a subclass of GraphicEnvironment instead of XYEnvironment. Let's see how this looks." + "Now that our park is ready for the 2D motion of our energetic dog, lets test it!" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "class GraphicPark(GraphicEnvironment):\n", - " def percept(self, agent):\n", - " '''return a list of things that are in our agent's location'''\n", - " things = self.list_things_at(agent.location)\n", - " return things\n", - " \n", - " def execute_action(self, agent, action):\n", - " '''changes the state of the environment based on what the agent does.'''\n", - " if action == 'turnright':\n", - " print('{} decided to {} at location: {}'.format(str(agent)[1:-1], action, agent.location))\n", - " agent.turn(Direction.R)\n", - " elif action == 'turnleft':\n", - " print('{} decided to {} at location: {}'.format(str(agent)[1:-1], action, agent.location))\n", - " agent.turn(Direction.L)\n", - " elif action == 'moveforward':\n", - " loc = copy.deepcopy(agent.location) # find out the target location\n", - " if agent.direction.direction == Direction.R:\n", - " loc[0] += 1\n", - " elif agent.direction.direction == Direction.L:\n", - " loc[0] -= 1\n", - " elif agent.direction.direction == Direction.D:\n", - " loc[1] += 1\n", - " elif agent.direction.direction == Direction.U:\n", - " loc[1] -= 1\n", - " if self.is_inbounds(loc):# move only if the target is a valid location\n", - " print('{} decided to move {}wards at location: {}'.format(str(agent)[1:-1], agent.direction.direction, agent.location))\n", - " agent.moveforward()\n", - " else:\n", - " print('{} decided to move {}wards at location: {}, but couldn\\'t'.format(str(agent)[1:-1], agent.direction.direction, agent.location))\n", - " agent.moveforward(False)\n", - " elif action == \"eat\":\n", - " items = self.list_things_at(agent.location, tclass=Food)\n", - " if len(items) != 0:\n", - " if agent.eat(items[0]):\n", - " print('{} ate {} at location: {}'\n", - " .format(str(agent)[1:-1], str(items[0])[1:-1], agent.location))\n", - " self.delete_thing(items[0])\n", - " elif action == \"drink\":\n", - " items = self.list_things_at(agent.location, tclass=Water)\n", - " if len(items) != 0:\n", - " if agent.drink(items[0]):\n", - " print('{} drank {} at location: {}'\n", - " .format(str(agent)[1:-1], str(items[0])[1:-1], agent.location))\n", - " self.delete_thing(items[0])\n", - " \n", - " def is_done(self):\n", - " '''By default, we're done when we can't find a live agent, \n", - " but to prevent killing our cute dog, we will stop before itself - when there is no more food or water'''\n", - " no_edibles = not any(isinstance(thing, Food) or isinstance(thing, Water) for thing in self.things)\n", - " dead_agents = not any(agent.is_alive() for agent in self.agents)\n", - " return dead_agents or no_edibles\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "That is the only change we make. The rest of our code stays the same. There is a slight difference in usage though. Every time we create a GraphicPark, we need to define the colors of all the things we plan to put into the park. The colors are defined in typical [RGB digital 8-bit format](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations), common across the web." - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "dog started at [0,0], facing down. Let's see if he found any food or water!\n" - ] - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move downwards at location: [0, 0]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog drank Water at location: [0, 1]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move downwards at location: [0, 1]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move downwards at location: [0, 2]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move downwards at location: [0, 3]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnleft at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move downwards at location: [0, 4], but couldn't\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move leftwards at location: [0, 4], but couldn't\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnleft at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move upwards at location: [0, 4]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move upwards at location: [0, 3]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnleft at location: [0, 2]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnleft at location: [0, 2]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 2]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to move leftwards at location: [0, 2], but couldn't\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "EnergeticBlindDog decided to turnright at location: [0, 2]\n" - ] - }, - { - "data": { - "text/html": [], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "park = GraphicPark(5,5, color={'EnergeticBlindDog': (200,0,0), 'Water': (0, 200, 200), 'Food': (230, 115, 40)})\n", + "park = Park2D(5,5, color={'EnergeticBlindDog': (200,0,0), 'Water': (0, 200, 200), 'Food': (230, 115, 40)})\n", "dog = EnergeticBlindDog(program)\n", "dogfood = Food()\n", "water = Water()\n", @@ -1347,7 +654,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1389,30 +696,9 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
    " - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[[], [None], [], [None], [None]]\n", - "Forward\n" - ] - } - ], + "outputs": [], "source": [ "step()" ] @@ -1441,7 +727,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.6.4" } }, "nbformat": 4, diff --git a/agents.py b/agents.py index f7ccb255b..d29b0c382 100644 --- a/agents.py +++ b/agents.py @@ -1,4 +1,5 @@ -"""Implement Agents and Environments (Chapters 1-2). +""" +Implement Agents and Environments. (Chapters 1-2) The class hierarchies are as follows: @@ -23,24 +24,21 @@ EnvToolbar ## contains buttons for controlling EnvGUI EnvCanvas ## Canvas to display the environment of an EnvGUI - """ -# TO DO: -# Implement grabbing correctly. -# When an object is grabbed, does it still have a location? -# What if it is released? -# What if the grabbed or the grabber is deleted? -# What if the grabber moves? -# +# TODO # Speed control in GUI does not have any effect -- fix it. from utils import distance_squared, turn_heading from statistics import mean +from ipythonblocks import BlockGrid +from IPython.display import HTML, display, clear_output +from time import sleep import random import copy import collections +import numbers # ______________________________________________________________________________ @@ -69,26 +67,25 @@ def display(self, canvas, x, y, width, height): class Agent(Thing): - """An Agent is a subclass of Thing with one required slot, - .program, which should hold a function that takes one argument, the - percept, and returns an action. (What counts as a percept or action + """An Agent is a subclass of Thing with one required instance attribute + (aka slot), .program, which should hold a function that takes one argument, + the percept, and returns an action. (What counts as a percept or action will depend on the specific environment in which the agent exists.) - Note that 'program' is a slot, not a method. If it were a method, - then the program could 'cheat' and look at aspects of the agent. - It's not supposed to do that: the program can only look at the - percepts. An agent program that needs a model of the world (and of - the agent itself) will have to build and maintain its own model. - There is an optional slot, .performance, which is a number giving - the performance measure of the agent in its environment.""" + Note that 'program' is a slot, not a method. If it were a method, then the + program could 'cheat' and look at aspects of the agent. It's not supposed + to do that: the program can only look at the percepts. An agent program + that needs a model of the world (and of the agent itself) will have to + build and maintain its own model. There is an optional slot, .performance, + which is a number giving the performance measure of the agent in its + environment.""" def __init__(self, program=None): self.alive = True self.bump = False self.holding = [] self.performance = 0 - if program is None or not isinstance(program, collections.Callable): - print("Can't find a valid program for {}, falling back to default.".format( - self.__class__.__name__)) + if program is None or not isinstance(program, collections.abc.Callable): + print("Can't find a valid program for {}, falling back to default.".format(self.__class__.__name__)) def program(percept): return eval(input('Percept={}; action? '.format(percept))) @@ -110,23 +107,29 @@ def new_program(percept): action = old_program(percept) print('{} perceives {} and does {}'.format(agent, percept, action)) return action + agent.program = new_program return agent + # ______________________________________________________________________________ def TableDrivenAgentProgram(table): - """This agent selects an action based on the percept sequence. + """ + [Figure 2.7] + This agent selects an action based on the percept sequence. It is practical only for tiny domains. To customize it, provide as table a dictionary of all - {percept_sequence:action} pairs. [Figure 2.7]""" + {percept_sequence:action} pairs. + """ percepts = [] def program(percept): percepts.append(percept) action = table.get(tuple(percepts)) return action + return program @@ -143,26 +146,37 @@ def RandomAgentProgram(actions): """ return lambda percept: random.choice(actions) + # ______________________________________________________________________________ def SimpleReflexAgentProgram(rules, interpret_input): - """This agent takes action based solely on the percept. [Figure 2.10]""" + """ + [Figure 2.10] + This agent takes action based solely on the percept. + """ + def program(percept): state = interpret_input(percept) rule = rule_match(state, rules) action = rule.action return action + return program def ModelBasedReflexAgentProgram(rules, update_state, model): - """This agent takes action based on the percept and state. [Figure 2.12]""" + """ + [Figure 2.12] + This agent takes action based on the percept and state. + """ + def program(percept): program.state = update_state(program.state, program.action, percept, model) rule = rule_match(program.state, rules) action = rule.action return action + program.state = program.action = None return program @@ -173,6 +187,7 @@ def rule_match(state, rules): if rule.matches(state): return rule + # ______________________________________________________________________________ @@ -192,7 +207,14 @@ def RandomVacuumAgent(): def TableDrivenVacuumAgent(): - """[Figure 2.3]""" + """Tabular approach towards vacuum world as mentioned in [Figure 2.3] + >>> agent = TableDrivenVacuumAgent() + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + True + """ table = {((loc_A, 'Clean'),): 'Right', ((loc_A, 'Dirty'),): 'Suck', ((loc_B, 'Clean'),): 'Left', @@ -202,13 +224,14 @@ def TableDrivenVacuumAgent(): ((loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck', ((loc_B, 'Dirty'), (loc_B, 'Clean')): 'Left', ((loc_A, 'Dirty'), (loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', - ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck' - } + ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck'} return Agent(TableDrivenAgentProgram(table)) def ReflexVacuumAgent(): - """A reflex agent for the two-state vacuum environment. [Figure 2.8] + """ + [Figure 2.8] + A reflex agent for the two-state vacuum environment. >>> agent = ReflexVacuumAgent() >>> environment = TrivialVacuumEnvironment() >>> environment.add_thing(agent) @@ -216,6 +239,7 @@ def ReflexVacuumAgent(): >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} True """ + def program(percept): location, status = percept if status == 'Dirty': @@ -224,6 +248,7 @@ def program(percept): return 'Right' elif location == loc_B: return 'Left' + return Agent(program) @@ -250,8 +275,10 @@ def program(percept): return 'Right' elif location == loc_B: return 'Left' + return Agent(program) + # ______________________________________________________________________________ @@ -318,8 +345,11 @@ def run(self, steps=1000): def list_things_at(self, location, tclass=Thing): """Return all things exactly at a given location.""" + if isinstance(location, numbers.Number): + return [thing for thing in self.things + if thing.location == location and isinstance(thing, tclass)] return [thing for thing in self.things - if thing.location == location and isinstance(thing, tclass)] + if all(x == y for x, y in zip(thing.location, location)) and isinstance(thing, tclass)] def some_things_at(self, location, tclass=Thing): """Return true if at least one of the things at location @@ -389,22 +419,22 @@ def __add__(self, heading): True """ if self.direction == self.R: - return{ + return { self.R: Direction(self.D), self.L: Direction(self.U), }.get(heading, None) elif self.direction == self.L: - return{ + return { self.R: Direction(self.U), self.L: Direction(self.D), }.get(heading, None) elif self.direction == self.U: - return{ + return { self.R: Direction(self.R), self.L: Direction(self.L), }.get(heading, None) elif self.direction == self.D: - return{ + return { self.R: Direction(self.L), self.L: Direction(self.R), }.get(heading, None) @@ -420,15 +450,17 @@ def move_forward(self, from_location): >>> l1 (1, 0) """ + # get the iterable class to return + iclass = from_location.__class__ x, y = from_location if self.direction == self.R: - return (x + 1, y) + return iclass((x + 1, y)) elif self.direction == self.L: - return (x - 1, y) + return iclass((x - 1, y)) elif self.direction == self.U: - return (x, y - 1) + return iclass((x, y - 1)) elif self.direction == self.D: - return (x, y + 1) + return iclass((x, y + 1)) class XYEnvironment(Environment): @@ -459,7 +491,7 @@ def things_near(self, location, radius=None): radius2 = radius * radius return [(thing, radius2 - distance_squared(location, thing.location)) for thing in self.things if distance_squared( - location, thing.location) <= radius2] + location, thing.location) <= radius2] def percept(self, agent): """By default, agent perceives things within a default radius.""" @@ -473,17 +505,24 @@ def execute_action(self, agent, action): agent.direction += Direction.L elif action == 'Forward': agent.bump = self.move_to(agent, agent.direction.move_forward(agent.location)) -# elif action == 'Grab': -# things = [thing for thing in self.list_things_at(agent.location) -# if agent.can_grab(thing)] -# if things: -# agent.holding.append(things[0]) + elif action == 'Grab': + things = [thing for thing in self.list_things_at(agent.location) if agent.can_grab(thing)] + if things: + agent.holding.append(things[0]) + print("Grabbing ", things[0].__class__.__name__) + self.delete_thing(things[0]) elif action == 'Release': if agent.holding: - agent.holding.pop() + dropped = agent.holding.pop() + print("Dropping ", dropped.__class__.__name__) + self.add_thing(dropped, location=agent.location) def default_location(self, thing): - return (random.choice(self.width), random.choice(self.height)) + location = self.random_location_inbounds() + while self.some_things_at(location, Obstacle): + # we will find a random location with no obstacles + location = self.random_location_inbounds() + return location def move_to(self, thing, destination): """Move a thing to a new location. Returns True on success or False if there is an Obstacle. @@ -499,10 +538,12 @@ def move_to(self, thing, destination): t.location = destination return thing.bump - def add_thing(self, thing, location=(1, 1), exclude_duplicate_class_items=False): + def add_thing(self, thing, location=None, exclude_duplicate_class_items=False): """Add things to the world. If (exclude_duplicate_class_items) then the item won't be added if the location has at least one item of the same class.""" - if (self.is_inbounds(location)): + if location is None: + super().add_thing(thing) + elif self.is_inbounds(location): if (exclude_duplicate_class_items and any(isinstance(t, thing.__class__) for t in self.list_things_at(location))): return @@ -511,14 +552,14 @@ def add_thing(self, thing, location=(1, 1), exclude_duplicate_class_items=False) def is_inbounds(self, location): """Checks to make sure that the location is inbounds (within walls if we have walls)""" x, y = location - return not (x < self.x_start or x >= self.x_end or y < self.y_start or y >= self.y_end) + return not (x < self.x_start or x > self.x_end or y < self.y_start or y > self.y_end) def random_location_inbounds(self, exclude=None): """Returns a random location that is inbounds (within walls if we have walls)""" location = (random.randint(self.x_start, self.x_end), random.randint(self.y_start, self.y_end)) if exclude is not None: - while(location == exclude): + while location == exclude: location = (random.randint(self.x_start, self.x_end), random.randint(self.y_start, self.y_end)) return location @@ -526,10 +567,7 @@ def random_location_inbounds(self, exclude=None): def delete_thing(self, thing): """Deletes thing, and everything it is holding (if thing is an agent)""" if isinstance(thing, Agent): - for obj in thing.holding: - super().delete_thing(obj) - for obs in self.observers: - obs.thing_deleted(obj) + del thing.holding super().delete_thing(thing) for obs in self.observers: @@ -540,7 +578,7 @@ def add_walls(self): for x in range(self.width): self.add_thing(Wall(), (x, 0)) self.add_thing(Wall(), (x, self.height - 1)) - for y in range(self.height): + for y in range(1, self.height - 1): self.add_thing(Wall(), (0, y)) self.add_thing(Wall(), (self.width - 1, y)) @@ -571,15 +609,8 @@ class Obstacle(Thing): class Wall(Obstacle): pass -# ______________________________________________________________________________ - -try: - from ipythonblocks import BlockGrid - from IPython.display import HTML, display - from time import sleep -except: - pass +# ______________________________________________________________________________ class GraphicEnvironment(XYEnvironment): @@ -605,7 +636,7 @@ def get_world(self): for x in range(x_start, x_end): row = [] for y in range(y_start, y_end): - row.append(self.list_things_at([x, y])) + row.append(self.list_things_at((x, y))) result.append(row) return result @@ -638,16 +669,16 @@ def run(self, steps=1000, delay=1): def update(self, delay=1): sleep(delay) - if self.visible: - self.conceal() - self.reveal() - else: - self.reveal() + self.reveal() def reveal(self): """Display the BlockGrid for this world - the last thing to be added at a location defines the location color.""" self.draw_world() + # wait for the world to update and + # apply changes to the same grid instead + # of making a new one. + clear_output(1) self.grid.show() self.visible = True @@ -687,6 +718,7 @@ def __init__(self, coordinates): super().__init__() self.coordinates = coordinates + # ______________________________________________________________________________ # Vacuum environment @@ -696,7 +728,6 @@ class Dirt(Thing): class VacuumEnvironment(XYEnvironment): - """The environment of [Ex. 2.12]. Agent perceives dirty or clean, and bump (into obstacle) or not; 2D discrete world of unknown size; performance measure is 100 for each dirt cleaned, and -1 for @@ -715,10 +746,11 @@ def percept(self, agent): Unlike the TrivialVacuumEnvironment, location is NOT perceived.""" status = ('Dirty' if self.some_things_at( agent.location, Dirt) else 'Clean') - bump = ('Bump' if agent.bump else'None') - return (status, bump) + bump = ('Bump' if agent.bump else 'None') + return status, bump def execute_action(self, agent, action): + agent.bump = False if action == 'Suck': dirt_list = self.list_things_at(agent.location, Dirt) if dirt_list != []: @@ -733,7 +765,6 @@ def execute_action(self, agent, action): class TrivialVacuumEnvironment(Environment): - """This environment has two locations, A and B. Each can be Dirty or Clean. The agent perceives its location and the location's status. This serves as an example of how to implement a simple @@ -745,12 +776,11 @@ def __init__(self): loc_B: random.choice(['Clean', 'Dirty'])} def thing_classes(self): - return [Wall, Dirt, ReflexVacuumAgent, RandomVacuumAgent, - TableDrivenVacuumAgent, ModelBasedVacuumAgent] + return [Wall, Dirt, ReflexVacuumAgent, RandomVacuumAgent, TableDrivenVacuumAgent, ModelBasedVacuumAgent] def percept(self, agent): """Returns the agent's location, and the location status (Dirty/Clean).""" - return (agent.location, self.status[agent.location]) + return agent.location, self.status[agent.location] def execute_action(self, agent, action): """Change agent's location and/or location's status; track performance. @@ -770,6 +800,7 @@ def default_location(self, thing): """Agents start in either location at random.""" return random.choice([loc_A, loc_B]) + # ______________________________________________________________________________ # The Wumpus World @@ -779,6 +810,7 @@ class Gold(Thing): def __eq__(self, rhs): """All Gold are equal""" return rhs.__class__ == Gold + pass @@ -828,6 +860,7 @@ def can_grab(self, thing): class WumpusEnvironment(XYEnvironment): pit_probability = 0.2 # Probability to spawn a pit in a location. (From Chapter 7.2) + # Room should be 4x4 grid of rooms. The extra 2 for walls def __init__(self, agent_program, width=6, height=6): @@ -926,24 +959,10 @@ def execute_action(self, agent, action): if isinstance(agent, Explorer) and self.in_danger(agent): return - + agent.bump = False - if action == 'TurnRight': - agent.direction += Direction.R - agent.performance -= 1 - elif action == 'TurnLeft': - agent.direction += Direction.L - agent.performance -= 1 - elif action == 'Forward': - agent.bump = self.move_to(agent, agent.direction.move_forward(agent.location)) - agent.performance -= 1 - elif action == 'Grab': - things = [thing for thing in self.list_things_at(agent.location) - if agent.can_grab(thing)] - if len(things): - print("Grabbing", things[0].__class__.__name__) - if len(things): - agent.holding.append(things[0]) + if action in ['TurnRight', 'TurnLeft', 'Forward', 'Grab']: + super().execute_action(agent, action) agent.performance -= 1 elif action == 'Climb': if agent.location == (1, 1): # Agent can only climb out of (1,1) @@ -953,7 +972,7 @@ def execute_action(self, agent, action): """The arrow travels straight down the path the agent is facing""" if agent.has_arrow: arrow_travel = agent.direction.move_forward(agent.location) - while(self.is_inbounds(arrow_travel)): + while self.is_inbounds(arrow_travel): wumpus = [thing for thing in self.list_things_at(arrow_travel) if isinstance(thing, Wumpus)] if len(wumpus): @@ -983,12 +1002,12 @@ def is_done(self): print("Death by {} [-1000].".format(explorer[0].killed_by)) else: print("Explorer climbed out {}." - .format( - "with Gold [+1000]!" if Gold() not in self.things else "without Gold [+0]")) + .format("with Gold [+1000]!" if Gold() not in self.things else "without Gold [+0]")) return True - # TODO: Arrow needs to be implemented + + # ______________________________________________________________________________ @@ -1000,9 +1019,9 @@ def compare_agents(EnvFactory, AgentFactories, n=10, steps=1000): >>> environment = TrivialVacuumEnvironment >>> agents = [ModelBasedVacuumAgent, ReflexVacuumAgent] >>> result = compare_agents(environment, agents) - >>> performance_ModelBasedVacummAgent = result[0][1] - >>> performance_ReflexVacummAgent = result[1][1] - >>> performance_ReflexVacummAgent <= performance_ModelBasedVacummAgent + >>> performance_ModelBasedVacuumAgent = result[0][1] + >>> performance_ReflexVacuumAgent = result[1][1] + >>> performance_ReflexVacuumAgent <= performance_ModelBasedVacuumAgent True """ envs = [EnvFactory() for i in range(n)] @@ -1020,13 +1039,16 @@ def test_agent(AgentFactory, steps, envs): >>> result == 5 True """ + def score(env): agent = AgentFactory() env.add_thing(agent) env.run(steps) return agent.performance + return mean(map(score, envs)) + # _________________________________________________________________________ diff --git a/agents4e.py b/agents4e.py new file mode 100644 index 000000000..75369a69a --- /dev/null +++ b/agents4e.py @@ -0,0 +1,1089 @@ +""" +Implement Agents and Environments. (Chapters 1-2) + +The class hierarchies are as follows: + +Thing ## A physical object that can exist in an environment + Agent + Wumpus + Dirt + Wall + ... + +Environment ## An environment holds objects, runs simulations + XYEnvironment + VacuumEnvironment + WumpusEnvironment + +An agent program is a callable instance, taking percepts and choosing actions + SimpleReflexAgentProgram + ... + +EnvGUI ## A window with a graphical representation of the Environment + +EnvToolbar ## contains buttons for controlling EnvGUI + +EnvCanvas ## Canvas to display the environment of an EnvGUI +""" + +# TODO +# Implement grabbing correctly. +# When an object is grabbed, does it still have a location? +# What if it is released? +# What if the grabbed or the grabber is deleted? +# What if the grabber moves? +# Speed control in GUI does not have any effect -- fix it. + +from utils4e import distance_squared, turn_heading +from statistics import mean +from ipythonblocks import BlockGrid +from IPython.display import HTML, display, clear_output +from time import sleep + +import random +import copy +import collections +import numbers + + +# ______________________________________________________________________________ + + +class Thing: + """This represents any physical object that can appear in an Environment. + You subclass Thing to get the things you want. Each thing can have a + .__name__ slot (used for output only).""" + + def __repr__(self): + return '<{}>'.format(getattr(self, '__name__', self.__class__.__name__)) + + def is_alive(self): + """Things that are 'alive' should return true.""" + return hasattr(self, 'alive') and self.alive + + def show_state(self): + """Display the agent's internal state. Subclasses should override.""" + print("I don't know how to show_state.") + + def display(self, canvas, x, y, width, height): + """Display an image of this Thing on the canvas.""" + # Do we need this? + pass + + +class Agent(Thing): + """An Agent is a subclass of Thing with one required slot, + .program, which should hold a function that takes one argument, the + percept, and returns an action. (What counts as a percept or action + will depend on the specific environment in which the agent exists.) + Note that 'program' is a slot, not a method. If it were a method, + then the program could 'cheat' and look at aspects of the agent. + It's not supposed to do that: the program can only look at the + percepts. An agent program that needs a model of the world (and of + the agent itself) will have to build and maintain its own model. + There is an optional slot, .performance, which is a number giving + the performance measure of the agent in its environment.""" + + def __init__(self, program=None): + self.alive = True + self.bump = False + self.holding = [] + self.performance = 0 + if program is None or not isinstance(program, collections.abc.Callable): + print("Can't find a valid program for {}, falling back to default.".format(self.__class__.__name__)) + + def program(percept): + return eval(input('Percept={}; action? '.format(percept))) + + self.program = program + + def can_grab(self, thing): + """Return True if this agent can grab this thing. + Override for appropriate subclasses of Agent and Thing.""" + return False + + +def TraceAgent(agent): + """Wrap the agent's program to print its input and output. This will let + you see what the agent is doing in the environment.""" + old_program = agent.program + + def new_program(percept): + action = old_program(percept) + print('{} perceives {} and does {}'.format(agent, percept, action)) + return action + + agent.program = new_program + return agent + + +# ______________________________________________________________________________ + + +def TableDrivenAgentProgram(table): + """ + [Figure 2.7] + This agent selects an action based on the percept sequence. + It is practical only for tiny domains. + To customize it, provide as table a dictionary of all + {percept_sequence:action} pairs. + """ + percepts = [] + + def program(percept): + percepts.append(percept) + action = table.get(tuple(percepts)) + return action + + return program + + +def RandomAgentProgram(actions): + """An agent that chooses an action at random, ignoring all percepts. + >>> list = ['Right', 'Left', 'Suck', 'NoOp'] + >>> program = RandomAgentProgram(list) + >>> agent = Agent(program) + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1, 0): 'Clean' , (0, 0): 'Clean'} + True + """ + return lambda percept: random.choice(actions) + + +# ______________________________________________________________________________ + + +def SimpleReflexAgentProgram(rules, interpret_input): + """ + [Figure 2.10] + This agent takes action based solely on the percept. + """ + + def program(percept): + state = interpret_input(percept) + rule = rule_match(state, rules) + action = rule.action + return action + + return program + + +def ModelBasedReflexAgentProgram(rules, update_state, transition_model, sensor_model): + """ + [Figure 2.12] + This agent takes action based on the percept and state. + """ + + def program(percept): + program.state = update_state(program.state, program.action, percept, transition_model, sensor_model) + rule = rule_match(program.state, rules) + action = rule.action + return action + + program.state = program.action = None + return program + + +def rule_match(state, rules): + """Find the first rule that matches state.""" + for rule in rules: + if rule.matches(state): + return rule + + +# ______________________________________________________________________________ + + +loc_A, loc_B = (0, 0), (1, 0) # The two locations for the Vacuum world + + +def RandomVacuumAgent(): + """Randomly choose one of the actions from the vacuum environment. + >>> agent = RandomVacuumAgent() + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + True + """ + return Agent(RandomAgentProgram(['Right', 'Left', 'Suck', 'NoOp'])) + + +def TableDrivenVacuumAgent(): + """Tabular approach towards vacuum world as mentioned in [Figure 2.3] + >>> agent = TableDrivenVacuumAgent() + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + True + """ + table = {((loc_A, 'Clean'),): 'Right', + ((loc_A, 'Dirty'),): 'Suck', + ((loc_B, 'Clean'),): 'Left', + ((loc_B, 'Dirty'),): 'Suck', + ((loc_A, 'Dirty'), (loc_A, 'Clean')): 'Right', + ((loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', + ((loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck', + ((loc_B, 'Dirty'), (loc_B, 'Clean')): 'Left', + ((loc_A, 'Dirty'), (loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', + ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck'} + return Agent(TableDrivenAgentProgram(table)) + + +def ReflexVacuumAgent(): + """ + [Figure 2.8] + A reflex agent for the two-state vacuum environment. + >>> agent = ReflexVacuumAgent() + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + True + """ + + def program(percept): + location, status = percept + if status == 'Dirty': + return 'Suck' + elif location == loc_A: + return 'Right' + elif location == loc_B: + return 'Left' + + return Agent(program) + + +def ModelBasedVacuumAgent(): + """An agent that keeps track of what locations are clean or dirty. + >>> agent = ModelBasedVacuumAgent() + >>> environment = TrivialVacuumEnvironment() + >>> environment.add_thing(agent) + >>> environment.run() + >>> environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + True + """ + model = {loc_A: None, loc_B: None} + + def program(percept): + """Same as ReflexVacuumAgent, except if everything is clean, do NoOp.""" + location, status = percept + model[location] = status # Update the model here + if model[loc_A] == model[loc_B] == 'Clean': + return 'NoOp' + elif status == 'Dirty': + return 'Suck' + elif location == loc_A: + return 'Right' + elif location == loc_B: + return 'Left' + + return Agent(program) + + +# ______________________________________________________________________________ + + +class Environment: + """Abstract class representing an Environment. 'Real' Environment classes + inherit from this. Your Environment will typically need to implement: + percept: Define the percept that an agent sees. + execute_action: Define the effects of executing an action. + Also update the agent.performance slot. + The environment keeps a list of .things and .agents (which is a subset + of .things). Each agent has a .performance slot, initialized to 0. + Each thing has a .location slot, even though some environments may not + need this.""" + + def __init__(self): + self.things = [] + self.agents = [] + + def thing_classes(self): + return [] # List of classes that can go into environment + + def percept(self, agent): + """Return the percept that the agent sees at this point. (Implement this.)""" + raise NotImplementedError + + def execute_action(self, agent, action): + """Change the world to reflect this action. (Implement this.)""" + raise NotImplementedError + + def default_location(self, thing): + """Default location to place a new thing with unspecified location.""" + return None + + def exogenous_change(self): + """If there is spontaneous change in the world, override this.""" + pass + + def is_done(self): + """By default, we're done when we can't find a live agent.""" + return not any(agent.is_alive() for agent in self.agents) + + def step(self): + """Run the environment for one time step. If the + actions and exogenous changes are independent, this method will + do. If there are interactions between them, you'll need to + override this method.""" + if not self.is_done(): + actions = [] + for agent in self.agents: + if agent.alive: + actions.append(agent.program(self.percept(agent))) + else: + actions.append("") + for (agent, action) in zip(self.agents, actions): + self.execute_action(agent, action) + self.exogenous_change() + + def run(self, steps=1000): + """Run the Environment for given number of time steps.""" + for step in range(steps): + if self.is_done(): + return + self.step() + + def list_things_at(self, location, tclass=Thing): + """Return all things exactly at a given location.""" + if isinstance(location, numbers.Number): + return [thing for thing in self.things + if thing.location == location and isinstance(thing, tclass)] + return [thing for thing in self.things + if all(x == y for x, y in zip(thing.location, location)) and isinstance(thing, tclass)] + + def some_things_at(self, location, tclass=Thing): + """Return true if at least one of the things at location + is an instance of class tclass (or a subclass).""" + return self.list_things_at(location, tclass) != [] + + def add_thing(self, thing, location=None): + """Add a thing to the environment, setting its location. For + convenience, if thing is an agent program we make a new agent + for it. (Shouldn't need to override this.)""" + if not isinstance(thing, Thing): + thing = Agent(thing) + if thing in self.things: + print("Can't add the same thing twice") + else: + thing.location = location if location is not None else self.default_location(thing) + self.things.append(thing) + if isinstance(thing, Agent): + thing.performance = 0 + self.agents.append(thing) + + def delete_thing(self, thing): + """Remove a thing from the environment.""" + try: + self.things.remove(thing) + except ValueError as e: + print(e) + print(" in Environment delete_thing") + print(" Thing to be removed: {} at {}".format(thing, thing.location)) + print(" from list: {}".format([(thing, thing.location) for thing in self.things])) + if thing in self.agents: + self.agents.remove(thing) + + +class Direction: + """A direction class for agents that want to move in a 2D plane + Usage: + d = Direction("down") + To change directions: + d = d + "right" or d = d + Direction.R #Both do the same thing + Note that the argument to __add__ must be a string and not a Direction object. + Also, it (the argument) can only be right or left.""" + + R = "right" + L = "left" + U = "up" + D = "down" + + def __init__(self, direction): + self.direction = direction + + def __add__(self, heading): + """ + >>> d = Direction('right') + >>> l1 = d.__add__(Direction.L) + >>> l2 = d.__add__(Direction.R) + >>> l1.direction + 'up' + >>> l2.direction + 'down' + >>> d = Direction('down') + >>> l1 = d.__add__('right') + >>> l2 = d.__add__('left') + >>> l1.direction == Direction.L + True + >>> l2.direction == Direction.R + True + """ + if self.direction == self.R: + return { + self.R: Direction(self.D), + self.L: Direction(self.U), + }.get(heading, None) + elif self.direction == self.L: + return { + self.R: Direction(self.U), + self.L: Direction(self.D), + }.get(heading, None) + elif self.direction == self.U: + return { + self.R: Direction(self.R), + self.L: Direction(self.L), + }.get(heading, None) + elif self.direction == self.D: + return { + self.R: Direction(self.L), + self.L: Direction(self.R), + }.get(heading, None) + + def move_forward(self, from_location): + """ + >>> d = Direction('up') + >>> l1 = d.move_forward((0, 0)) + >>> l1 + (0, -1) + >>> d = Direction(Direction.R) + >>> l1 = d.move_forward((0, 0)) + >>> l1 + (1, 0) + """ + # get the iterable class to return + iclass = from_location.__class__ + x, y = from_location + if self.direction == self.R: + return iclass((x + 1, y)) + elif self.direction == self.L: + return iclass((x - 1, y)) + elif self.direction == self.U: + return iclass((x, y - 1)) + elif self.direction == self.D: + return iclass((x, y + 1)) + + +class XYEnvironment(Environment): + """This class is for environments on a 2D plane, with locations + labelled by (x, y) points, either discrete or continuous. + + Agents perceive things within a radius. Each agent in the + environment has a .location slot which should be a location such + as (0, 1), and a .holding slot, which should be a list of things + that are held.""" + + def __init__(self, width=10, height=10): + super().__init__() + + self.width = width + self.height = height + self.observers = [] + # Sets iteration start and end (no walls). + self.x_start, self.y_start = (0, 0) + self.x_end, self.y_end = (self.width, self.height) + + perceptible_distance = 1 + + def things_near(self, location, radius=None): + """Return all things within radius of location.""" + if radius is None: + radius = self.perceptible_distance + radius2 = radius * radius + return [(thing, radius2 - distance_squared(location, thing.location)) + for thing in self.things if distance_squared( + location, thing.location) <= radius2] + + def percept(self, agent): + """By default, agent perceives things within a default radius.""" + return self.things_near(agent.location) + + def execute_action(self, agent, action): + agent.bump = False + if action == 'TurnRight': + agent.direction += Direction.R + elif action == 'TurnLeft': + agent.direction += Direction.L + elif action == 'Forward': + agent.bump = self.move_to(agent, agent.direction.move_forward(agent.location)) + # elif action == 'Grab': + # things = [thing for thing in self.list_things_at(agent.location) + # if agent.can_grab(thing)] + # if things: + # agent.holding.append(things[0]) + elif action == 'Release': + if agent.holding: + agent.holding.pop() + + def default_location(self, thing): + location = self.random_location_inbounds() + while self.some_things_at(location, Obstacle): + # we will find a random location with no obstacles + location = self.random_location_inbounds() + return location + + def move_to(self, thing, destination): + """Move a thing to a new location. Returns True on success or False if there is an Obstacle. + If thing is holding anything, they move with him.""" + thing.bump = self.some_things_at(destination, Obstacle) + if not thing.bump: + thing.location = destination + for o in self.observers: + o.thing_moved(thing) + for t in thing.holding: + self.delete_thing(t) + self.add_thing(t, destination) + t.location = destination + return thing.bump + + def add_thing(self, thing, location=None, exclude_duplicate_class_items=False): + """Add things to the world. If (exclude_duplicate_class_items) then the item won't be + added if the location has at least one item of the same class.""" + if location is None: + super().add_thing(thing) + elif self.is_inbounds(location): + if (exclude_duplicate_class_items and + any(isinstance(t, thing.__class__) for t in self.list_things_at(location))): + return + super().add_thing(thing, location) + + def is_inbounds(self, location): + """Checks to make sure that the location is inbounds (within walls if we have walls)""" + x, y = location + return not (x < self.x_start or x > self.x_end or y < self.y_start or y > self.y_end) + + def random_location_inbounds(self, exclude=None): + """Returns a random location that is inbounds (within walls if we have walls)""" + location = (random.randint(self.x_start, self.x_end), + random.randint(self.y_start, self.y_end)) + if exclude is not None: + while location == exclude: + location = (random.randint(self.x_start, self.x_end), + random.randint(self.y_start, self.y_end)) + return location + + def delete_thing(self, thing): + """Deletes thing, and everything it is holding (if thing is an agent)""" + if isinstance(thing, Agent): + for obj in thing.holding: + super().delete_thing(obj) + for obs in self.observers: + obs.thing_deleted(obj) + + super().delete_thing(thing) + for obs in self.observers: + obs.thing_deleted(thing) + + def add_walls(self): + """Put walls around the entire perimeter of the grid.""" + for x in range(self.width): + self.add_thing(Wall(), (x, 0)) + self.add_thing(Wall(), (x, self.height - 1)) + for y in range(1, self.height - 1): + self.add_thing(Wall(), (0, y)) + self.add_thing(Wall(), (self.width - 1, y)) + + # Updates iteration start and end (with walls). + self.x_start, self.y_start = (1, 1) + self.x_end, self.y_end = (self.width - 1, self.height - 1) + + def add_observer(self, observer): + """Adds an observer to the list of observers. + An observer is typically an EnvGUI. + + Each observer is notified of changes in move_to and add_thing, + by calling the observer's methods thing_moved(thing) + and thing_added(thing, loc).""" + self.observers.append(observer) + + def turn_heading(self, heading, inc): + """Return the heading to the left (inc=+1) or right (inc=-1) of heading.""" + return turn_heading(heading, inc) + + +class Obstacle(Thing): + """Something that can cause a bump, preventing an agent from + moving into the same square it's in.""" + pass + + +class Wall(Obstacle): + pass + + +# ______________________________________________________________________________ + + +class GraphicEnvironment(XYEnvironment): + def __init__(self, width=10, height=10, boundary=True, color={}, display=False): + """Define all the usual XYEnvironment characteristics, + but initialise a BlockGrid for GUI too.""" + super().__init__(width, height) + self.grid = BlockGrid(width, height, fill=(200, 200, 200)) + if display: + self.grid.show() + self.visible = True + else: + self.visible = False + self.bounded = boundary + self.colors = color + + def get_world(self): + """Returns all the items in the world in a format + understandable by the ipythonblocks BlockGrid.""" + result = [] + x_start, y_start = (0, 0) + x_end, y_end = self.width, self.height + for x in range(x_start, x_end): + row = [] + for y in range(y_start, y_end): + row.append(self.list_things_at((x, y))) + result.append(row) + return result + + """ + def run(self, steps=1000, delay=1): + "" "Run the Environment for given number of time steps, + but update the GUI too." "" + for step in range(steps): + sleep(delay) + if self.visible: + self.reveal() + if self.is_done(): + if self.visible: + self.reveal() + return + self.step() + if self.visible: + self.reveal() + """ + + def run(self, steps=1000, delay=1): + """Run the Environment for given number of time steps, + but update the GUI too.""" + for step in range(steps): + self.update(delay) + if self.is_done(): + break + self.step() + self.update(delay) + + def update(self, delay=1): + sleep(delay) + self.reveal() + + def reveal(self): + """Display the BlockGrid for this world - the last thing to be added + at a location defines the location color.""" + self.draw_world() + # wait for the world to update and + # apply changes to the same grid instead + # of making a new one. + clear_output(1) + self.grid.show() + self.visible = True + + def draw_world(self): + self.grid[:] = (200, 200, 200) + world = self.get_world() + for x in range(0, len(world)): + for y in range(0, len(world[x])): + if len(world[x][y]): + self.grid[y, x] = self.colors[world[x][y][-1].__class__.__name__] + + def conceal(self): + """Hide the BlockGrid for this world""" + self.visible = False + display(HTML('')) + + +# ______________________________________________________________________________ +# Continuous environment + +class ContinuousWorld(Environment): + """Model for Continuous World""" + + def __init__(self, width=10, height=10): + super().__init__() + self.width = width + self.height = height + + def add_obstacle(self, coordinates): + self.things.append(PolygonObstacle(coordinates)) + + +class PolygonObstacle(Obstacle): + + def __init__(self, coordinates): + """Coordinates is a list of tuples.""" + super().__init__() + self.coordinates = coordinates + + +# ______________________________________________________________________________ +# Vacuum environment + + +class Dirt(Thing): + pass + + +class VacuumEnvironment(XYEnvironment): + """The environment of [Ex. 2.12]. Agent perceives dirty or clean, + and bump (into obstacle) or not; 2D discrete world of unknown size; + performance measure is 100 for each dirt cleaned, and -1 for + each turn taken.""" + + def __init__(self, width=10, height=10): + super().__init__(width, height) + self.add_walls() + + def thing_classes(self): + return [Wall, Dirt, ReflexVacuumAgent, RandomVacuumAgent, + TableDrivenVacuumAgent, ModelBasedVacuumAgent] + + def percept(self, agent): + """The percept is a tuple of ('Dirty' or 'Clean', 'Bump' or 'None'). + Unlike the TrivialVacuumEnvironment, location is NOT perceived.""" + status = ('Dirty' if self.some_things_at( + agent.location, Dirt) else 'Clean') + bump = ('Bump' if agent.bump else 'None') + return status, bump + + def execute_action(self, agent, action): + agent.bump = False + if action == 'Suck': + dirt_list = self.list_things_at(agent.location, Dirt) + if dirt_list != []: + dirt = dirt_list[0] + agent.performance += 100 + self.delete_thing(dirt) + else: + super().execute_action(agent, action) + + if action != 'NoOp': + agent.performance -= 1 + + +class TrivialVacuumEnvironment(Environment): + """This environment has two locations, A and B. Each can be Dirty + or Clean. The agent perceives its location and the location's + status. This serves as an example of how to implement a simple + Environment.""" + + def __init__(self): + super().__init__() + self.status = {loc_A: random.choice(['Clean', 'Dirty']), + loc_B: random.choice(['Clean', 'Dirty'])} + + def thing_classes(self): + return [Wall, Dirt, ReflexVacuumAgent, RandomVacuumAgent, TableDrivenVacuumAgent, ModelBasedVacuumAgent] + + def percept(self, agent): + """Returns the agent's location, and the location status (Dirty/Clean).""" + return agent.location, self.status[agent.location] + + def execute_action(self, agent, action): + """Change agent's location and/or location's status; track performance. + Score 10 for each dirt cleaned; -1 for each move.""" + if action == 'Right': + agent.location = loc_B + agent.performance -= 1 + elif action == 'Left': + agent.location = loc_A + agent.performance -= 1 + elif action == 'Suck': + if self.status[agent.location] == 'Dirty': + agent.performance += 10 + self.status[agent.location] = 'Clean' + + def default_location(self, thing): + """Agents start in either location at random.""" + return random.choice([loc_A, loc_B]) + + +# ______________________________________________________________________________ +# The Wumpus World + + +class Gold(Thing): + + def __eq__(self, rhs): + """All Gold are equal""" + return rhs.__class__ == Gold + + pass + + +class Bump(Thing): + pass + + +class Glitter(Thing): + pass + + +class Pit(Thing): + pass + + +class Breeze(Thing): + pass + + +class Arrow(Thing): + pass + + +class Scream(Thing): + pass + + +class Wumpus(Agent): + screamed = False + pass + + +class Stench(Thing): + pass + + +class Explorer(Agent): + holding = [] + has_arrow = True + killed_by = "" + direction = Direction("right") + + def can_grab(self, thing): + """Explorer can only grab gold""" + return thing.__class__ == Gold + + +class WumpusEnvironment(XYEnvironment): + pit_probability = 0.2 # Probability to spawn a pit in a location. (From Chapter 7.2) + + # Room should be 4x4 grid of rooms. The extra 2 for walls + + def __init__(self, agent_program, width=6, height=6): + super().__init__(width, height) + self.init_world(agent_program) + + def init_world(self, program): + """Spawn items in the world based on probabilities from the book""" + + "WALLS" + self.add_walls() + + "PITS" + for x in range(self.x_start, self.x_end): + for y in range(self.y_start, self.y_end): + if random.random() < self.pit_probability: + self.add_thing(Pit(), (x, y), True) + self.add_thing(Breeze(), (x - 1, y), True) + self.add_thing(Breeze(), (x, y - 1), True) + self.add_thing(Breeze(), (x + 1, y), True) + self.add_thing(Breeze(), (x, y + 1), True) + + "WUMPUS" + w_x, w_y = self.random_location_inbounds(exclude=(1, 1)) + self.add_thing(Wumpus(lambda x: ""), (w_x, w_y), True) + self.add_thing(Stench(), (w_x - 1, w_y), True) + self.add_thing(Stench(), (w_x + 1, w_y), True) + self.add_thing(Stench(), (w_x, w_y - 1), True) + self.add_thing(Stench(), (w_x, w_y + 1), True) + + "GOLD" + self.add_thing(Gold(), self.random_location_inbounds(exclude=(1, 1)), True) + + "AGENT" + self.add_thing(Explorer(program), (1, 1), True) + + def get_world(self, show_walls=True): + """Return the items in the world""" + result = [] + x_start, y_start = (0, 0) if show_walls else (1, 1) + + if show_walls: + x_end, y_end = self.width, self.height + else: + x_end, y_end = self.width - 1, self.height - 1 + + for x in range(x_start, x_end): + row = [] + for y in range(y_start, y_end): + row.append(self.list_things_at((x, y))) + result.append(row) + return result + + def percepts_from(self, agent, location, tclass=Thing): + """Return percepts from a given location, + and replaces some items with percepts from chapter 7.""" + thing_percepts = { + Gold: Glitter(), + Wall: Bump(), + Wumpus: Stench(), + Pit: Breeze()} + + """Agents don't need to get their percepts""" + thing_percepts[agent.__class__] = None + + """Gold only glitters in its cell""" + if location != agent.location: + thing_percepts[Gold] = None + + result = [thing_percepts.get(thing.__class__, thing) for thing in self.things + if thing.location == location and isinstance(thing, tclass)] + return result if len(result) else [None] + + def percept(self, agent): + """Return things in adjacent (not diagonal) cells of the agent. + Result format: [Left, Right, Up, Down, Center / Current location]""" + x, y = agent.location + result = [] + result.append(self.percepts_from(agent, (x - 1, y))) + result.append(self.percepts_from(agent, (x + 1, y))) + result.append(self.percepts_from(agent, (x, y - 1))) + result.append(self.percepts_from(agent, (x, y + 1))) + result.append(self.percepts_from(agent, (x, y))) + + """The wumpus gives out a loud scream once it's killed.""" + wumpus = [thing for thing in self.things if isinstance(thing, Wumpus)] + if len(wumpus) and not wumpus[0].alive and not wumpus[0].screamed: + result[-1].append(Scream()) + wumpus[0].screamed = True + + return result + + def execute_action(self, agent, action): + """Modify the state of the environment based on the agent's actions. + Performance score taken directly out of the book.""" + + if isinstance(agent, Explorer) and self.in_danger(agent): + return + + agent.bump = False + if action == 'TurnRight': + agent.direction += Direction.R + agent.performance -= 1 + elif action == 'TurnLeft': + agent.direction += Direction.L + agent.performance -= 1 + elif action == 'Forward': + agent.bump = self.move_to(agent, agent.direction.move_forward(agent.location)) + agent.performance -= 1 + elif action == 'Grab': + things = [thing for thing in self.list_things_at(agent.location) + if agent.can_grab(thing)] + if len(things): + print("Grabbing", things[0].__class__.__name__) + if len(things): + agent.holding.append(things[0]) + agent.performance -= 1 + elif action == 'Climb': + if agent.location == (1, 1): # Agent can only climb out of (1,1) + agent.performance += 1000 if Gold() in agent.holding else 0 + self.delete_thing(agent) + elif action == 'Shoot': + """The arrow travels straight down the path the agent is facing""" + if agent.has_arrow: + arrow_travel = agent.direction.move_forward(agent.location) + while self.is_inbounds(arrow_travel): + wumpus = [thing for thing in self.list_things_at(arrow_travel) + if isinstance(thing, Wumpus)] + if len(wumpus): + wumpus[0].alive = False + break + arrow_travel = agent.direction.move_forward(agent.location) + agent.has_arrow = False + + def in_danger(self, agent): + """Check if Explorer is in danger (Pit or Wumpus), if he is, kill him""" + for thing in self.list_things_at(agent.location): + if isinstance(thing, Pit) or (isinstance(thing, Wumpus) and thing.alive): + agent.alive = False + agent.performance -= 1000 + agent.killed_by = thing.__class__.__name__ + return True + return False + + def is_done(self): + """The game is over when the Explorer is killed + or if he climbs out of the cave only at (1,1).""" + explorer = [agent for agent in self.agents if isinstance(agent, Explorer)] + if len(explorer): + if explorer[0].alive: + return False + else: + print("Death by {} [-1000].".format(explorer[0].killed_by)) + else: + print("Explorer climbed out {}." + .format("with Gold [+1000]!" if Gold() not in self.things else "without Gold [+0]")) + return True + + # TODO: Arrow needs to be implemented + + +# ______________________________________________________________________________ + + +def compare_agents(EnvFactory, AgentFactories, n=10, steps=1000): + """See how well each of several agents do in n instances of an environment. + Pass in a factory (constructor) for environments, and several for agents. + Create n instances of the environment, and run each agent in copies of + each one for steps. Return a list of (agent, average-score) tuples. + >>> environment = TrivialVacuumEnvironment + >>> agents = [ModelBasedVacuumAgent, ReflexVacuumAgent] + >>> result = compare_agents(environment, agents) + >>> performance_ModelBasedVacuumAgent = result[0][1] + >>> performance_ReflexVacuumAgent = result[1][1] + >>> performance_ReflexVacuumAgent <= performance_ModelBasedVacuumAgent + True + """ + envs = [EnvFactory() for i in range(n)] + return [(A, test_agent(A, steps, copy.deepcopy(envs))) + for A in AgentFactories] + + +def test_agent(AgentFactory, steps, envs): + """Return the mean score of running an agent in each of the envs, for steps + >>> def constant_prog(percept): + ... return percept + ... + >>> agent = Agent(constant_prog) + >>> result = agent.program(5) + >>> result == 5 + True + """ + + def score(env): + agent = AgentFactory() + env.add_thing(agent) + env.run(steps) + return agent.performance + + return mean(map(score, envs)) + + +# _________________________________________________________________________ + + +__doc__ += """ +>>> a = ReflexVacuumAgent() +>>> a.program((loc_A, 'Clean')) +'Right' +>>> a.program((loc_B, 'Clean')) +'Left' +>>> a.program((loc_A, 'Dirty')) +'Suck' +>>> a.program((loc_A, 'Dirty')) +'Suck' + +>>> e = TrivialVacuumEnvironment() +>>> e.add_thing(ModelBasedVacuumAgent()) +>>> e.run(5) + +""" diff --git a/arc_consistency_heuristics.ipynb b/arc_consistency_heuristics.ipynb new file mode 100644 index 000000000..fb2241819 --- /dev/null +++ b/arc_consistency_heuristics.ipynb @@ -0,0 +1,1999 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "# Constraint Satisfaction Problems\n", + "---\n", + "# Heuristics for Arc-Consistency Algorithms\n", + "\n", + "## Introduction\n", + "A ***Constraint Satisfaction Problem*** is a triple $(X,D,C)$ where: \n", + "- $X$ is a set of variables $X_1, …, X_n$;\n", + "- $D$ is a set of domains $D_1, …, D_n$, one for each variable and each of which consists of a set of allowable values $v_1, ..., v_k$;\n", + "- $C$ is a set of constraints that specify allowable combinations of values.\n", + "\n", + "A CSP is called *arc-consistent* if every value in the domain of every variable is supported by all the neighbors of the variable while, is called *inconsistent*, if it has no solutions.
    \n", + "***Arc-consistency algorithms*** remove all unsupported values from the domains of variables making the CSP *arc-consistent* or decide that a CSP is *inconsistent* by finding that some variable has no supported values in its domain.
    \n", + "Heuristics significantly enhance the efficiency of the *arc-consistency algorithms* improving their average performance in terms of *consistency-checks* which can be considered a standard measure of goodness for such algorithms. *Arc-heuristic* operate at arc-level and selects the constraint that will be used for the next check, while *domain-heuristics* operate at domain-level and selects which values will be used for the next support-check." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from csp import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Domain-Heuristics for Arc-Consistency Algorithms\n", + "In [[1]](#cite-van2002domain) are investigated the effects of a *domain-heuristic* based on the notion of a *double-support check* by studying its average time-complexity.\n", + "\n", + "The objective of *arc-consistency algorithms* is to resolve some uncertainty; it has to be know, for each $v_i \\in D_i$ and for each $v_j \\in D_j$, whether it is supported.\n", + "\n", + "A *single-support check*, $(v_i, v_j) \\in C_{ij}$, is one in which, before the check is done, it is already known that either $v_i$ or $v_j$ are supported. \n", + "\n", + "A *double-support check* $(v_i, v_j) \\in C_{ij}$, is one in which there is still, before the check, uncertainty about the support-status of both $v_i$ and $v_j$. \n", + "\n", + "If a *double-support check* is successful, two uncertainties are resolved. If a *single-support check* is successful, only one uncertainty is resolved. A good *arc-consistency algorithm*, therefore, would always choose to do a *double-support check* in preference of a *single-support check*, because the cormer offers the potential higher payback.\n", + "\n", + "The improvement with *double-support check* is that, where possible, *consistency-checks* are used to find supports for two values, one value in the domain of each variable, which were previously known to be unsupported. It is motivated by the insight that *in order to minimize the number of consistency-checks it is necessary to maximize the number of uncertainties which are resolved per check*." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### AC-3b: an improved version of AC-3 with Double-Support Checks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As shown in [[2]](#cite-van2000improving) the idea is to use *double-support checks* to improve the average performance of `AC3` which does not exploit the fact that relations are bidirectional and results in a new general purpose *arc-consistency algorithm* called `AC3b`." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC3\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"[Figure 6.3]\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrevise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC3" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mrevise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Return true if we remove a value.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# If Xi=x conflicts with Xj=y for every possible y, eliminate Xi=x\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if all(not csp.constraints(Xi, x, Xj, y) for y in csp.curr_domains[Xj]):\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource revise" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "At any stage in the process of making 2-variable CSP *arc-consistent* in `AC3b`:\n", + "- there is a set $S_i^+ \\subseteq D_i$ whose values are all known to be supported by $X_j$;\n", + "- there is a set $S_i^? = D_i \\setminus S_i^+$ whose values are unknown, as yet, to be supported by $X_j$.\n", + "\n", + "The same holds if the roles for $X_i$ and $X_j$ are exchanged.\n", + "\n", + "In order to establish support for a value $v_i^? \\in S_i^?$ it seems better to try to find a support among the values in $S_j^?$ first, because for each $v_j^? \\in S_j^?$ the check $(v_i^?,v_j^?) \\in C_{ij}$ is a *double-support check* and it is just as likely that any $v_j^? \\in S_j^?$ supports $v_i^?$ than it is that any $v_j^+ \\in S_j^+$ does. Only if no support can be found among the elements in $S_j^?$, should the elements $v_j^+$ in $S_j^+$ be used for *single-support checks* $(v_i^?,v_j^+) \\in C_{ij}$. After it has been decided for each value in $D_i$ whether it is supported or not, either $S_x^+ = \\emptyset$ and the 2-variable CSP is *inconsistent*, or $S_x^+ \\neq \\emptyset$ and the CSP is *satisfiable*. In the latter case, the elements from $D_i$ which are supported by $j$ are given by $S_x^+$. The elements in $D_j$ which are supported by $x$ are given by the union of $S_j^+$ with the set of those elements of $S_j^?$ which further processing will show to be supported by some $v_i^+ \\in S_x^+$." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC3b\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Si_p values are all known to be supported by Xj\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Sj_p values are all known to be supported by Xi\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Dj - Sj_p = Sj_u values are unknown, as yet, to be supported by Xi\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# or queue -= {(Xj, Xi)} or queue.remove((Xj, Xi))\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference_update\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference_update\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the elements in D_j which are supported by Xi are given by the union of Sj_p with the set of those\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# elements of Sj_u which further processing will show to be supported by some vi_p in Si_p\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_u\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvi_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvj_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC3b" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvi_u\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# now, in order to establish support for a value vi_u in Di it seems better to try to find a support among\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the values in Sj_u first, because for each vj_u in Sj_u the check (vi_u, vj_u) is a double-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# and it is just as likely that any vj_u in Sj_u supports vi_u than it is that any vj_p in Sj_p does...\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_u\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# double-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvi_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvj_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# ... and only if no support can be found among the elements in Sj_u, should the elements vj_p in Sj_p be used\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# for single-support checks (vi_u, vj_p)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvj_p\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# single-support check\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvi_u\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvj_p\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvi_u\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSi_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSj_u\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mSj_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource partition" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "`AC3b` is a refinement of the `AC3` algorithm which consists of the fact that if, when arc $(i,j)$ is being processed and the reverse arc $(j,i)$ is also in the queue, then consistency-checks can be saved because only support for the elements in $S_j^?$ has to be found (as opposed to support for all the elements in $D_j$ in the\n", + "`AC3` algorithm).
    \n", + "`AC3b` inherits all its properties like $\\mathcal{O}(ed^3)$ time-complexity and $\\mathcal{O}(e + nd)$ space-complexity fron `AC3` and where $n$ denotes the number of variables in the CSP, $e$ denotes the number of binary constraints and $d$ denotes the maximum domain-size of the variables." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Arc-Heuristics for Arc-Consistency Algorithms" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "Many *arc-heuristics* can be devised, based on three major features of CSPs:\n", + "- the number of acceptable pairs in each constraint (the *constraint size* or *satisfiability*);\n", + "- the *domain size*;\n", + "- the number of binary constraints that each variable participates in, equal to the *degree* of the node of that variable in the constraint graph. \n", + "\n", + "Simple examples of heuristics that might be expected to improve the efficiency of relaxation are:\n", + "- ordering the list of variable pairs by *increasing* relative *satisfiability*;\n", + "- ordering by *increasing size of the domain* of the variable $v_j$ relaxed against $v_i$;\n", + "- ordering by *descending degree* of node of the variable relaxed.\n", + "\n", + "In
    [[3]](#cite-wallace1992ordering) are investigated the effects of these *arc-heuristics* in an empirical way, experimenting the effects of them on random CSPs. Their results demonstrate that the first two, later called `sat up` and `dom j up` for n-ary and binary CSPs respectively, significantly reduce the number of *consistency-checks*." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdom_j_up\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSortedSet\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mneg\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dom_j_up" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0msat_up\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSortedSet\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource sat_up" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Experimental Results" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "For the experiments below on binary CSPs, in addition to the two *arc-consistency algorithms* already cited above, `AC3` and `AC3b`, the `AC4` algorithm was used.
    \n", + "The `AC4` algorithm runs in $\\mathcal{O}(ed^2)$ worst-case time but can be slower than `AC3` on average cases." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mAC4\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdom_j_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariables\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mneighbors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupport_pruning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# construction and initialization of support sets\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# propagation of removed values\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvariable_value_pairs_supported\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m-=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msupport_counter\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mXj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprune\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremovals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrevised\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munsupported_variable_value_pairs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrevised\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurr_domains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mXi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is inconsistent\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;31m# CSP is satisfiable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource AC4" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Sudoku" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Easy Sudoku" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". . 3 | . 2 . | 6 . .\n", + "9 . . | 3 . 5 | . . 1\n", + ". . 1 | 8 . 6 | 4 . .\n", + "------+-------+------\n", + ". . 8 | 1 . 2 | 9 . .\n", + "7 . . | . . . | . . 8\n", + ". . 6 | 7 . 8 | 2 . .\n", + "------+-------+------\n", + ". . 2 | 6 . 9 | 5 . .\n", + "8 . . | 2 . 3 | . . 9\n", + ". . 5 | . 1 . | 3 . .\n" + ] + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 23.6 ms, sys: 0 ns, total: 23.6 ms\n", + "Wall time: 22.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 11322 consistency-checks'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 7.43 ms, sys: 3.68 ms, total: 11.1 ms\n", + "Wall time: 10.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 8345 consistency-checks'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 56.3 ms, sys: 0 ns, total: 56.3 ms\n", + "Wall time: 55.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 27718 consistency-checks'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.2 ms, sys: 0 ns, total: 17.2 ms\n", + "Wall time: 16.9 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 6925 consistency-checks'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 40.9 ms, sys: 2.47 ms, total: 43.4 ms\n", + "Wall time: 41.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 6278 consistency-checks'" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 38.9 ms, sys: 1.96 ms, total: 40.9 ms\n", + "Wall time: 40.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 9393 consistency-checks'" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(easy1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 8 3 | 9 2 1 | 6 5 7\n", + "9 6 7 | 3 4 5 | 8 2 1\n", + "2 5 1 | 8 7 6 | 4 9 3\n", + "------+-------+------\n", + "5 4 8 | 1 3 2 | 9 7 6\n", + "7 2 9 | 5 6 4 | 1 3 8\n", + "1 3 6 | 7 9 8 | 2 4 5\n", + "------+-------+------\n", + "3 7 2 | 6 8 9 | 5 1 4\n", + "8 1 4 | 2 5 3 | 7 6 9\n", + "6 9 5 | 4 1 7 | 3 8 2\n" + ] + } + ], + "source": [ + "backtracking_search(sudoku, select_unassigned_variable=mrv, inference=forward_checking)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Harder Sudoku" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 1 7 | 3 6 9 | 8 . 5\n", + ". 3 . | . . . | . . .\n", + ". . . | 7 . . | . . .\n", + "------+-------+------\n", + ". 2 . | . . . | . 6 .\n", + ". . . | . 8 . | 4 . .\n", + ". . . | . 1 . | . . .\n", + "------+-------+------\n", + ". . . | 6 . 3 | . 7 .\n", + "5 . . | 2 . . | . . .\n", + "1 . 4 | . . . | . . .\n" + ] + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.7 ms, sys: 481 µs, total: 18.2 ms\n", + "Wall time: 17.2 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 12837 consistency-checks'" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 24.1 ms, sys: 2.6 ms, total: 26.7 ms\n", + "Wall time: 25.1 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 8864 consistency-checks'" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 63.4 ms, sys: 3.48 ms, total: 66.9 ms\n", + "Wall time: 65.5 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 44213 consistency-checks'" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 9.96 ms, sys: 570 µs, total: 10.5 ms\n", + "Wall time: 10.3 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 7045 consistency-checks'" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 36.1 ms, sys: 0 ns, total: 36.1 ms\n", + "Wall time: 35.5 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 6994 consistency-checks'" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC3b(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 40.3 ms, sys: 0 ns, total: 40.3 ms\n", + "Wall time: 39.7 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 19210 consistency-checks'" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sudoku = Sudoku(harder1)\n", + "%time _, checks = AC4(sudoku, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 1 7 | 3 6 9 | 8 2 5\n", + "6 3 2 | 1 5 8 | 9 4 7\n", + "9 5 8 | 7 2 4 | 3 1 6\n", + "------+-------+------\n", + "8 2 5 | 4 3 7 | 1 6 9\n", + "7 9 1 | 5 8 6 | 4 3 2\n", + "3 4 6 | 9 1 2 | 7 5 8\n", + "------+-------+------\n", + "2 8 9 | 6 4 3 | 5 7 1\n", + "5 7 3 | 2 9 1 | 6 8 4\n", + "1 6 4 | 8 7 5 | 2 9 3\n" + ] + } + ], + "source": [ + "backtracking_search(sudoku, select_unassigned_variable=mrv, inference=forward_checking)\n", + "sudoku.display(sudoku.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### 8 Queens" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n", + ". - . - . - . - 0 0 0 0 0 0 0 0 \n", + "- . - . - . - . 0 0 0 0 0 0 0 0 \n" + ] + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "chess.display(chess.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 689 µs, sys: 193 µs, total: 882 µs\n", + "Wall time: 892 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 needs 666 consistency-checks'" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC3 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 451 µs, sys: 127 µs, total: 578 µs\n", + "Wall time: 584 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b needs 428 consistency-checks'" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3b(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC3b needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 8.53 ms, sys: 109 µs, total: 8.64 ms\n", + "Wall time: 8.48 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 needs 4096 consistency-checks'" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC4(chess, arc_heuristic=no_arc_heuristic)\n", + "f'AC4 needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.88 ms, sys: 0 ns, total: 1.88 ms\n", + "Wall time: 1.88 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3 with DOM J UP arc heuristic needs 666 consistency-checks'" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3(chess, arc_heuristic=dom_j_up)\n", + "f'AC3 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.21 ms, sys: 326 µs, total: 1.53 ms\n", + "Wall time: 1.54 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP arc heuristic needs 792 consistency-checks'" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC3b(chess, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 4.71 ms, sys: 0 ns, total: 4.71 ms\n", + "Wall time: 4.65 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC4 with DOM J UP arc heuristic needs 4096 consistency-checks'" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chess = NQueensCSP(8)\n", + "%time _, checks = AC4(chess, arc_heuristic=dom_j_up)\n", + "f'AC4 with DOM J UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". - . - Q - . - 2 2 3 3 0* 1 1 2 \n", + "- Q - . - . - . 1 0* 3 3 2 2 2 2 \n", + ". - . - . Q . - 3 2 3 2 2 0* 3 2 \n", + "Q . - . - . - . 0* 3 1 2 3 3 3 3 \n", + ". - . - . - Q - 2 2 2 2 3 3 0* 2 \n", + "- . - Q - . - . 2 1 3 0* 2 3 2 2 \n", + ". - . - . - . Q 1 3 2 3 3 1 2 0* \n", + "- . Q . - . - . 2 2 0* 2 2 2 2 2 \n" + ] + } + ], + "source": [ + "backtracking_search(chess, select_unassigned_variable=mrv, inference=forward_checking)\n", + "chess.display(chess.infer_assignment())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the experiments below on n-ary CSPs, due to the n-ary constraints, the `GAC` algorithm was used.
    \n", + "The `GAC` algorithm has $\\mathcal{O}(er^2d^t)$ time-complexity and $\\mathcal{O}(erd)$ space-complexity where $e$ denotes the number of n-ary constraints, $r$ denotes the constraint arity and $d$ denotes the maximum domain-size of the variables." + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + " \u001b[0;32mdef\u001b[0m \u001b[0mGAC\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morig_domains\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msat_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Makes this CSP arc-consistent using Generalized Arc Consistency\u001b[0m\n", + "\u001b[0;34m orig_domains is the original domains\u001b[0m\n", + "\u001b[0;34m to_do is a set of (variable,constraint) pairs\u001b[0m\n", + "\u001b[0;34m returns the reduced domains (an arc-consistent variable:domain dictionary)\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0morig_domains\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0morig_domains\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdomains\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mconst\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstraints\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0morig_domains\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_do\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mother_vars\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mov\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mov\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscope\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mov\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mother_vars\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mholds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if const.holds({var: val})}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mother_vars\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mother\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mother_vars\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mother_val\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mother\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mholds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mother\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mother_val\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if any(const.holds({var: val, other: other_val})\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# for other_val in domains[other])}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# general case\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mholds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0many_holds\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mother_vars\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mchecks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mholds\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# new_domain = {val for val in domains[var]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if self.any_holds(domains, const, {var: val}, other_vars)}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_domain\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mnew_domain\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0madd_to_do\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnew_to_do\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconst\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_do\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mto_do\u001b[0m \u001b[0;34m|=\u001b[0m \u001b[0madd_to_do\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchecks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource ACSolver.GAC" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Crossword" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[_] [_] [_] [*] [*] \n", + "[_] [*] [_] [*] [*] \n", + "[_] [_] [_] [_] [*] \n", + "[_] [*] [_] [*] [*] \n", + "[*] [*] [_] [_] [_] \n", + "[*] [*] [_] [*] [*] \n" + ] + }, + { + "data": { + "text/plain": [ + "{'ant',\n", + " 'big',\n", + " 'book',\n", + " 'bus',\n", + " 'buys',\n", + " 'car',\n", + " 'ginger',\n", + " 'has',\n", + " 'hold',\n", + " 'lane',\n", + " 'search',\n", + " 'symbol',\n", + " 'syntax',\n", + " 'year'}" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "crossword = Crossword(crossword1, words1)\n", + "crossword.display()\n", + "words1" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1min 20s, sys: 2.02 ms, total: 1min 20s\n", + "Wall time: 1min 20s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 64617645 consistency-checks'" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(crossword).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.19 s, sys: 0 ns, total: 1.19 s\n", + "Wall time: 1.19 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 908015 consistency-checks'" + ] + }, + "execution_count": 42, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "crossword = Crossword(crossword1, words1)\n", + "%time _, _, checks = ACSolver(crossword).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[B] [U] [S] [*] [*] \n", + "[U] [*] [E] [*] [*] \n", + "[Y] [E] [A] [R] [*] \n", + "[S] [*] [R] [*] [*] \n", + "[*] [*] [C] [A] [R] \n", + "[*] [*] [H] [*] [*] \n" + ] + } + ], + "source": [ + "crossword.display(ACSolver(crossword).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Kakuro" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Easy Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t10\\\t13\\\t[*]\t\n", + "\\3\t[_]\t[_]\t13\\\t\n", + "\\12\t[_]\t[_]\t[_]\t\n", + "\\21\t[_]\t[_]\t[_]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro2)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 17.8 ms, sys: 171 µs, total: 18 ms\n", + "Wall time: 16.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 2752 consistency-checks'" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 8.55 ms, sys: 0 ns, total: 8.55 ms\n", + "Wall time: 8.39 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 1765 consistency-checks'" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro2)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t10\\\t13\\\t[*]\t\n", + "\\3\t[1]\t[2]\t13\\\t\n", + "\\12\t[5]\t[3]\t[4]\t\n", + "\\21\t[4]\t[8]\t[9]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Medium Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t17\\\t28\\\t[*]\t42\\\t22\\\t\n", + "\\9\t[_]\t[_]\t31\\14\t[_]\t[_]\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t\\30\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t22\\24\t[_]\t[_]\t[_]\t[*]\t\n", + "\\25\t[_]\t[_]\t[_]\t[_]\t11\\\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "\\14\t[_]\t[_]\t\\17\t[_]\t[_]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro3)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.96 s, sys: 0 ns, total: 1.96 s\n", + "Wall time: 1.96 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 1290179 consistency-checks'" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 225 ms, sys: 0 ns, total: 225 ms\n", + "Wall time: 223 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 148780 consistency-checks'" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro3)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t17\\\t28\\\t[*]\t42\\\t22\\\t\n", + "\\9\t[8]\t[1]\t31\\14\t[5]\t[9]\t\n", + "\\20\t[9]\t[2]\t[1]\t[3]\t[5]\t\n", + "[*]\t\\30\t[6]\t[9]\t[7]\t[8]\t\n", + "[*]\t22\\24\t[7]\t[8]\t[9]\t[*]\t\n", + "\\25\t[8]\t[4]\t[7]\t[6]\t11\\\t\n", + "\\20\t[5]\t[3]\t[6]\t[4]\t[2]\t\n", + "\\14\t[9]\t[5]\t\\17\t[8]\t[9]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "#### Harder Kakuro" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t[*]\t[*]\t[*]\t[*]\t4\\\t24\\\t11\\\t[*]\t[*]\t[*]\t11\\\t17\\\t[*]\t[*]\t\n", + "[*]\t[*]\t[*]\t17\\\t11\\12\t[_]\t[_]\t[_]\t[*]\t[*]\t24\\10\t[_]\t[_]\t11\\\t[*]\t\n", + "[*]\t4\\\t16\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t\\20\t[_]\t[_]\t[_]\t[_]\t16\\\t\n", + "\\20\t[_]\t[_]\t[_]\t[_]\t24\\13\t[_]\t[_]\t16\\\t\\12\t[_]\t[_]\t23\\10\t[_]\t[_]\t\n", + "\\10\t[_]\t[_]\t24\\12\t[_]\t[_]\t16\\5\t[_]\t[_]\t16\\30\t[_]\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t[*]\t3\\26\t[_]\t[_]\t[_]\t[_]\t\\12\t[_]\t[_]\t4\\\t16\\14\t[_]\t[_]\t[*]\t\n", + "[*]\t\\8\t[_]\t[_]\t\\15\t[_]\t[_]\t34\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t[*]\t\n", + "[*]\t\\11\t[_]\t[_]\t3\\\t17\\\t\\14\t[_]\t[_]\t\\8\t[_]\t[_]\t7\\\t17\\\t[*]\t\n", + "[*]\t[*]\t[*]\t23\\10\t[_]\t[_]\t3\\9\t[_]\t[_]\t4\\\t23\\\t\\13\t[_]\t[_]\t[*]\t\n", + "[*]\t[*]\t10\\26\t[_]\t[_]\t[_]\t[_]\t[_]\t\\7\t[_]\t[_]\t30\\9\t[_]\t[_]\t[*]\t\n", + "[*]\t17\\11\t[_]\t[_]\t11\\\t24\\8\t[_]\t[_]\t11\\21\t[_]\t[_]\t[_]\t[_]\t16\\\t17\\\t\n", + "\\29\t[_]\t[_]\t[_]\t[_]\t[_]\t\\7\t[_]\t[_]\t23\\14\t[_]\t[_]\t3\\17\t[_]\t[_]\t\n", + "\\10\t[_]\t[_]\t3\\10\t[_]\t[_]\t[*]\t\\8\t[_]\t[_]\t4\\25\t[_]\t[_]\t[_]\t[_]\t\n", + "[*]\t\\16\t[_]\t[_]\t[_]\t[_]\t[*]\t\\23\t[_]\t[_]\t[_]\t[_]\t[_]\t[*]\t[*]\t\n", + "[*]\t[*]\t\\6\t[_]\t[_]\t[*]\t[*]\t\\15\t[_]\t[_]\t[_]\t[*]\t[*]\t[*]\t[*]\t\n" + ] + } + ], + "source": [ + "kakuro = Kakuro(kakuro4)\n", + "kakuro.display()" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 76.5 ms, sys: 847 µs, total: 77.4 ms\n", + "Wall time: 77 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 46633 consistency-checks'" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(kakuro).GAC()\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 64.6 ms, sys: 0 ns, total: 64.6 ms\n", + "Wall time: 63.6 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 36828 consistency-checks'" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "kakuro = Kakuro(kakuro4)\n", + "%time _, _, checks = ACSolver(kakuro).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[*]\t[*]\t[*]\t[*]\t[*]\t4\\\t24\\\t11\\\t[*]\t[*]\t[*]\t11\\\t17\\\t[*]\t[*]\t\n", + "[*]\t[*]\t[*]\t17\\\t11\\12\t[3]\t[7]\t[2]\t[*]\t[*]\t24\\10\t[2]\t[8]\t11\\\t[*]\t\n", + "[*]\t4\\\t16\\26\t[8]\t[5]\t[1]\t[9]\t[3]\t[*]\t\\20\t[8]\t[1]\t[9]\t[2]\t16\\\t\n", + "\\20\t[3]\t[7]\t[9]\t[1]\t24\\13\t[8]\t[5]\t16\\\t\\12\t[9]\t[3]\t23\\10\t[3]\t[7]\t\n", + "\\10\t[1]\t[9]\t24\\12\t[3]\t[9]\t16\\5\t[1]\t[4]\t16\\30\t[7]\t[5]\t[8]\t[1]\t[9]\t\n", + "[*]\t[*]\t3\\26\t[8]\t[2]\t[7]\t[9]\t\\12\t[3]\t[9]\t4\\\t16\\14\t[9]\t[5]\t[*]\t\n", + "[*]\t\\8\t[1]\t[7]\t\\15\t[8]\t[7]\t34\\26\t[1]\t[7]\t[3]\t[9]\t[6]\t[*]\t[*]\t\n", + "[*]\t\\11\t[2]\t[9]\t3\\\t17\\\t\\14\t[8]\t[6]\t\\8\t[1]\t[7]\t7\\\t17\\\t[*]\t\n", + "[*]\t[*]\t[*]\t23\\10\t[1]\t[9]\t3\\9\t[7]\t[2]\t4\\\t23\\\t\\13\t[4]\t[9]\t[*]\t\n", + "[*]\t[*]\t10\\26\t[6]\t[2]\t[8]\t[1]\t[9]\t\\7\t[1]\t[6]\t30\\9\t[1]\t[8]\t[*]\t\n", + "[*]\t17\\11\t[3]\t[8]\t11\\\t24\\8\t[2]\t[6]\t11\\21\t[3]\t[9]\t[7]\t[2]\t16\\\t17\\\t\n", + "\\29\t[8]\t[2]\t[9]\t[3]\t[7]\t\\7\t[4]\t[3]\t23\\14\t[8]\t[6]\t3\\17\t[9]\t[8]\t\n", + "\\10\t[9]\t[1]\t3\\10\t[2]\t[8]\t[*]\t\\8\t[2]\t[6]\t4\\25\t[8]\t[1]\t[7]\t[9]\t\n", + "[*]\t\\16\t[4]\t[2]\t[1]\t[9]\t[*]\t\\23\t[1]\t[8]\t[3]\t[9]\t[2]\t[*]\t[*]\t\n", + "[*]\t[*]\t\\6\t[1]\t[5]\t[*]\t[*]\t\\15\t[5]\t[9]\t[1]\t[*]\t[*]\t[*]\t[*]\t\n" + ] + } + ], + "source": [ + "kakuro.display(ACSolver(kakuro).domain_splitting())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "### Cryptarithmetic Puzzle" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{array}{@{}r@{}}\n", + " S E N D \\\\\n", + "{} + M O R E \\\\\n", + " \\hline\n", + " M O N E Y\n", + "\\end{array}\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "pycharm": {} + }, + "outputs": [], + "source": [ + "cryptarithmetic = NaryCSP(\n", + " {'S': set(range(1, 10)), 'M': set(range(1, 10)),\n", + " 'E': set(range(0, 10)), 'N': set(range(0, 10)), 'D': set(range(0, 10)),\n", + " 'O': set(range(0, 10)), 'R': set(range(0, 10)), 'Y': set(range(0, 10)),\n", + " 'C1': set(range(0, 2)), 'C2': set(range(0, 2)), 'C3': set(range(0, 2)),\n", + " 'C4': set(range(0, 2))},\n", + " [Constraint(('S', 'E', 'N', 'D', 'M', 'O', 'R', 'Y'), all_diff),\n", + " Constraint(('D', 'E', 'Y', 'C1'), lambda d, e, y, c1: d + e == y + 10 * c1),\n", + " Constraint(('N', 'R', 'E', 'C1', 'C2'), lambda n, r, e, c1, c2: c1 + n + r == e + 10 * c2),\n", + " Constraint(('E', 'O', 'N', 'C2', 'C3'), lambda e, o, n, c2, c3: c2 + e + o == n + 10 * c3),\n", + " Constraint(('S', 'M', 'O', 'C3', 'C4'), lambda s, m, o, c3, c4: c3 + s + m == o + 10 * c4),\n", + " Constraint(('M', 'C4'), eq)])" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 21.7 s, sys: 0 ns, total: 21.7 s\n", + "Wall time: 21.7 s\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC needs 14080592 consistency-checks'" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(cryptarithmetic).GAC(arc_heuristic=no_heuristic)\n", + "f'GAC needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 939 ms, sys: 0 ns, total: 939 ms\n", + "Wall time: 938 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'GAC with SAT UP arc heuristic needs 573120 consistency-checks'" + ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, _, checks = ACSolver(cryptarithmetic).GAC(arc_heuristic=sat_up)\n", + "f'GAC with SAT UP arc heuristic needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/latex": [ + "\\begin{array}{@{}r@{}} 9567 \\\\ + 1085 \\\\ \\hline 10652 \\end{array}" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "assignment = ACSolver(cryptarithmetic).domain_splitting()\n", + "\n", + "from IPython.display import Latex\n", + "display(Latex(r'\\begin{array}{@{}r@{}} ' + '{}{}{}{}'.format(assignment['S'], assignment['E'], assignment['N'], assignment['D']) + r' \\\\ + ' + \n", + " '{}{}{}{}'.format(assignment['M'], assignment['O'], assignment['R'], assignment['E']) + r' \\\\ \\hline ' + \n", + " '{}{}{}{}{}'.format(assignment['M'], assignment['O'], assignment['N'], assignment['E'], assignment['Y']) + ' \\end{array}'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## References\n", + "\n", + "
    [[1]](#ref-1) Van Dongen, Marc RC. 2002. _Domain-heuristics for arc-consistency algorithms_.\n", + "\n", + "[[2]](#ref-2) Van Dongen, MRC and Bowen, JA. 2000. _Improving arc-consistency algorithms with double-support checks_.\n", + "\n", + "[[3]](#ref-3) Wallace, Richard J and Freuder, Eugene Charles. 1992. _Ordering heuristics for arc consistency algorithms_." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.5rc1" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/classical_planning_approaches.ipynb b/classical_planning_approaches.ipynb new file mode 100644 index 000000000..b3373b367 --- /dev/null +++ b/classical_planning_approaches.ipynb @@ -0,0 +1,2402 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Classical Planning\n", + "---\n", + "# Classical Planning Approaches\n", + "\n", + "## Introduction \n", + "***Planning*** combines the two major areas of AI: *search* and *logic*. A planner can be seen either as a program that searches for a solution or as one that constructively proves the existence of a solution.\n", + "\n", + "Currently, the most popular and effective approaches to fully automated planning are:\n", + "- searching using a *planning graph*;\n", + "- *state-space search* with heuristics;\n", + "- translating to a *constraint satisfaction (CSP) problem*;\n", + "- translating to a *boolean satisfiability (SAT) problem*." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from planning import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Planning as Planning Graph Search\n", + "\n", + "A *planning graph* is a directed graph organized into levels each of which contains information about the current state of the knowledge base and the possible state-action links to and from that level. \n", + "\n", + "The first level contains the initial state with nodes representing each fluent that holds in that level. This level has state-action links linking each state to valid actions in that state. Each action is linked to all its preconditions and its effect states. Based on these effects, the next level is constructed and contains similarly structured information about the next state. In this way, the graph is expanded using state-action links till we reach a state where all the required goals hold true simultaneously.\n", + "\n", + "In every planning problem, we are allowed to carry out the *no-op* action, ie, we can choose no action for a particular state. These are called persistence actions and has effects same as its preconditions. This enables us to carry a state to the next level.\n", + "\n", + "Mutual exclusivity (*mutex*) between two actions means that these cannot be taken together and occurs in the following cases:\n", + "- *inconsistent effects*: one action negates the effect of the other;\n", + "- *interference*: one of the effects of an action is the negation of a precondition of the other;\n", + "- *competing needs*: one of the preconditions of one action is mutually exclusive with a precondition of the other.\n", + "\n", + "We can say that we have reached our goal if none of the goal states in the current level are mutually exclusive." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mGraph\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Contains levels of state and actions\u001b[0m\n", + "\u001b[0;34m Used in graph planning algorithm to extract a solution\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mFolKB\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mLevel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobjects\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0marg\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mexpand_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Expands the graph by a level\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mlast_level\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mlast_level\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mactions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobjects\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlast_level\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperform_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mnon_mutex_goals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Checks whether the goals are mutually exclusive\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoal_perm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcombinations\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mg\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mgoal_perm\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource Graph" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mLevel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Contains the state of the planning problem\u001b[0m\n", + "\u001b[0;34m and exhaustive list of actions which use the\u001b[0m\n", + "\u001b[0;34m states as pre-condition.\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Initializes variables to hold state and action details of a level\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkb\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# current state\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# current action to state link\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# current state to action link\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# current action to next state link\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# next state to current action link\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# mutually exclusive actions\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mobjects\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuild\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mactions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mobjects\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfind_mutex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mseparate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Separates an iterable of elements into positive and negative parts\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mpositive\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnegative\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'Not'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnegative\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mpositive\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mpositive\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnegative\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mfind_mutex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Finds mutually exclusive actions\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Inconsistent effects\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mpos_nsl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mneg_nsl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseparate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnegeff\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mneg_nsl\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_negeff\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnegeff\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mnegeff\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mposeff\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mpos_nsl\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_negeff\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mposeff\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mposeff\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mb\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnegeff\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m}\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Interference will be calculated with the last step\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mpos_csl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mneg_csl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseparate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Competing needs\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mpos_precond\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mpos_csl\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mneg_precond\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mneg_csl\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_neg_precond\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mneg_precond\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mneg_precond\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_neg_precond\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mpos_precond\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mpos_precond\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mb\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mneg_precond\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m}\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Inconsistent support\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate_mutex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mpair\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnext_state_0\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpair\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpair\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnext_state_1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpair\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnext_state_1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpair\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnext_state_0\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnext_state_1\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate_mutex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mnext_state_0\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnext_state_1\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstate_mutex\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mbuild\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mobjects\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Populates the lists and dictionaries containing the state action dependencies\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mp_expr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'P'\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mp_expr\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mp_expr\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mp_expr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mp_expr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnum_args\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mpossible_args\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpermutations\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobjects\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0marg\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mpossible_args\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheck_precond\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbol\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mislower\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0marg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnum\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0marg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_action\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubstitute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecond\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_clause\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubstitute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_clause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_clause\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubstitute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnew_clause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnew_clause\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mnew_action\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mperform_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Performs the necessary actions and returns a new Level\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_kb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mFolKB\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mLevel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_kb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource Level" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A *planning graph* can be used to give better heuristic estimates which can be applied to any of the search techniques. Alternatively, we can search for a solution over the space formed by the planning graph, using an algorithm called `GraphPlan`.\n", + "\n", + "The `GraphPlan` algorithm repeatedly adds a level to a planning graph. Once all the goals show up as non-mutex in the graph, the algorithm runs backward from the last level to the first searching for a plan that solves the problem. If that fails, it records the (level , goals) pair as a *no-good* (as in constraint learning for CSPs), expands another level and tries again, terminating with failure when there is no reason to go on. " + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mGraphPlan\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Class for formulation GraphPlan algorithm\u001b[0m\n", + "\u001b[0;34m Constructs a graph of state and action space\u001b[0m\n", + "\u001b[0;34m Returns solution for the planning problem\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mGraph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mno_goods\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msolution\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcheck_leveloff\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Checks if the graph has levelled off\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcheck\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_state\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcheck\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mextract_solution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Extracts the solution\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mlevel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnon_mutex_goals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mno_goods\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mlevel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mindex\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Create all combinations of actions that satisfy the goal\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mgoal\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext_state_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mall_actions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mproduct\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mactions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Filter out non-mutex actions\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnon_mutex_actions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction_tuple\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mall_actions\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0maction_pairs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcombinations\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_tuple\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnon_mutex_actions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_tuple\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mpair\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction_pairs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpair\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmutex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnon_mutex_actions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Recursion\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction_list\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mnon_mutex_actions\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0maction_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msolution\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msolution\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0maction_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_goals\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mact\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mact\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnew_goals\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_goals\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_action_links\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mact\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mabs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_goals\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mno_goods\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextract_solution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_goals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Level-Order multiple solutions\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mitem\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msolution\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mitem\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mitem\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msolution\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mitem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreverse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnum\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitem\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mgoal_test\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mask\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mq\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mFalse\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mq\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Executes the GraphPlan algorithm for the given problem\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoal_test\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkb\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnon_mutex_goals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msolution\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextract_solution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0msolution\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlevels\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0;36m2\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheck_leveloff\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource GraphPlan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Planning as State-Space Search" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The description of a planning problem defines a search problem: we can search from the initial state through the space of states, looking for a goal. One of the nice advantages of the declarative representation of action schemas is that we can also search backward from the goal, looking for the initial state. \n", + "\n", + "However, neither forward nor backward search is efficient without a good heuristic function because the real-world planning problems often have large state spaces. A heuristic function $h(s)$ estimates the distance from a state $s$ to the goal and, if it is admissible, ie if does not overestimate, then we can use $A^∗$ search to find optimal solutions.\n", + "\n", + "Planning uses a factored representation for states and action schemas which makes it possible to define good domain-independent heuristics to prune the search space.\n", + "\n", + "An admissible heuristic can be derived by defining a relaxed problem that is easier to solve. The length of the solution of this easier problem then becomes the heuristic for the original problem. Assume that all goals and preconditions contain only positive literals, ie that the problem is defined according to the *Stanford Research Institute Problem Solver* (STRIPS) notation: we want to create a relaxed version of the original problem that will be easier to solve by ignoring delete lists from all actions, ie removing all negative literals from effects. As shown in [[1]](#cite-hoffmann2001ff) the planning graph of a relaxed problem does not contain any mutex relations at all (which is the crucial thing when building a planning graph) and for this reason GraphPlan will never backtrack looking for a solution: for this reason the **ignore delete lists** heuristic makes it possible to find the optimal solution for relaxed problem in polynomial time through `GraphPlan` algorithm." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "from search import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Forward State-Space Search" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Forward search through the space of states, starting in the initial state and using the problem’s actions to search forward for a member of the set of goal states." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mForwardPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msearch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mProblem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Section 10.2.1]\u001b[0m\n", + "\u001b[0;34m Forward state-space search\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpanded_actions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpanded_actions\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpre\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mpre\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecond\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mgoal_test\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoal\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mgoal\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Computes ignore delete lists heuristic by creating a relaxed version of the original problem (we can do that\u001b[0m\n", + "\u001b[0;34m by removing the delete lists from all actions, i.e. removing all negative literals from effects) that will be\u001b[0m\n", + "\u001b[0;34m easier to solve through GraphPlan and where the length of the solution will serve as a good heuristic.\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrelaxed_planning_problem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrelaxed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mactions\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlinearize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mGraphPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrelaxed_planning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mexcept\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'inf'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource ForwardPlan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Backward Relevant-States Search" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Backward search through sets of relevant states, starting at the set of states representing the goal and using the inverse of the actions to search backward for the initial state." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mBackwardPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msearch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mProblem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Section 10.2.2]\u001b[0m\n", + "\u001b[0;34m Backward relevant-states search\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpanded_actions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Returns True if the action is relevant to the subgoal, i.e.:\u001b[0m\n", + "\u001b[0;34m - the action achieves an element of the effects\u001b[0m\n", + "\u001b[0;34m - the action doesn't delete something that needs to be achieved\u001b[0m\n", + "\u001b[0;34m - the preconditions are consistent with other subgoals that need to be achieved\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mnegate_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msubgoal\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpanded_actions\u001b[0m \u001b[0;32mif\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mprop\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnegate_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msubgoal\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mprop\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnegate_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msubgoal\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mnegate_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mprop\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecond\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msubgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# g' = (g - effects(a)) + preconds(a)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdifference\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munion\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecond\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mgoal_test\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoal\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mgoal\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msubgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Computes ignore delete lists heuristic by creating a relaxed version of the original problem (we can do that\u001b[0m\n", + "\u001b[0;34m by removing the delete lists from all actions, i.e. removing all negative literals from effects) that will be\u001b[0m\n", + "\u001b[0;34m easier to solve through GraphPlan and where the length of the solution will serve as a good heuristic.\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrelaxed_planning_problem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msubgoal\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrelaxed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mactions\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlinearize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mGraphPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrelaxed_planning_problem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mexcept\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'inf'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource BackwardPlan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Planning as Constraint Satisfaction Problem" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In forward planning, the search is constrained by the initial state and only uses the goal as a stopping criterion and as a source for heuristics. In regression planning, the search is constrained by the goal and only uses the start state as a stopping criterion and as a source for heuristics. By converting the problem to a constraint satisfaction problem (CSP), the initial state can be used to prune what is not reachable and the goal to prune what is not useful. The CSP will be defined for a finite number of steps; the number of steps can be adjusted to find the shortest plan. One of the CSP methods can then be used to solve the CSP and thus find a plan.\n", + "\n", + "To construct a CSP from a planning problem, first choose a fixed planning *horizon*, which is the number of time steps over which to plan. Suppose the horizon is \n", + "$k$. The CSP has the following variables:\n", + "\n", + "- a *state variable* for each feature and each time from 0 to $k$. If there are $n$ features for a horizon of $k$, there are $n \\cdot (k+1)$ state variables. The domain of the state variable is the domain of the corresponding feature;\n", + "- an *action variable*, $Action_t$, for each $t$ in the range 0 to $k-1$. The domain of $Action_t$, represents the action that takes the agent from the state at time $t$ to the state at time $t+1$.\n", + "\n", + "There are several types of constraints:\n", + "\n", + "- a *precondition constraint* between a state variable at time $t$ and the variable $Actiont_t$ constrains what actions are legal at time $t$;\n", + "- an *effect constraint* between $Action_t$ and a state variable at time $t+1$ constrains the values of a state variable that is a direct effect of the action;\n", + "- a *frame constraint* among a state variable at time $t$, the variable $Action_t$, and the corresponding state variable at time $t+1$ specifies when the variable that does not change as a result of an action has the same value before and after the action;\n", + "- an *initial-state constraint* constrains a variable on the initial state (at time 0). The initial state is represented as a set of domain constraints on the state variables at time 0;\n", + "- a *goal constraint* constrains the final state to be a state that satisfies the achievement goal. These are domain constraints on the variables that appear in the goal;\n", + "- a *state constraint* is a constraint among variables at the same time step. These can include physical constraints on the state or can ensure that states that violate maintenance goals are forbidden. This is extra knowledge beyond the power of the feature-based or PDDL representations of the action.\n", + "\n", + "The PDDL representation gives precondition, effect and frame constraints for each time \n", + "$t$ as follows:\n", + "\n", + "- for each $Var = v$ in the precondition of action $A$, there is a precondition constraint:\n", + "$$ Var_t = v \\leftarrow Action_t = A $$\n", + "that specifies that if the action is to be $A$, $Var_t$ must have value $v$ immediately before. This constraint is violated when $Action_t = A$ and $Var_t \\neq v$, and thus is equivalent to $\\lnot{(Var_t \\neq v \\land Action_t = A)}$;\n", + "- or each $Var = v$ in the effect of action $A$, there is a effect constraint:\n", + "$$ Var_{t+1} = v \\leftarrow Action_t = A $$\n", + "which is violated when $Action_t = A$ and $Var_{t+1} \\neq v$, and thus is equivalent to $\\lnot{(Var_{t+1} \\neq v \\land Action_t = A)}$;\n", + "- for each $Var$, there is a frame constraint, where $As$ is the set of actions that include $Var$ in the effect of the action:\n", + "$$ Var_{t+1} = Var_t \\leftarrow Action_t \\notin As $$\n", + "which specifies that the feature $Var$ has the same value before and after any action that does not affect $Var$.\n", + "\n", + "The CSP representation assumes a fixed planning horizon (ie a fixed number of steps). To find a plan over any number of steps, the algorithm can be run for a horizon of $k = 0, 1, 2, \\dots$ until a solution is found." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "from csp import *" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mCSPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msolution_length\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mCSP_solver\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mac_search_solver\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msat_up\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Section 10.4.3]\u001b[0m\n", + "\u001b[0;34m Planning as Constraint Satisfaction Problem\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Returns a string for the var-stage pair that can be used as a variable\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\"_\"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mif_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mv2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"If the second argument is v2, the first argument must be v1\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mif_fun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mx1\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mv1\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mx2\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mv2\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mif_fun\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"if the second argument is \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv2\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\" then the first argument is \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv1\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\" \"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mif_fun\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0meq_if_not_in_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mactset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"First and third arguments are equal if action is not in actset\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0meq_if_not_in\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mx1\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mx2\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mactset\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meq_if_not_in\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"first and third arguments are equal if action is not in \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mactset\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\" \"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0meq_if_not_in\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mexpanded_actions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mfluent_values\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_fluents\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mhorizon\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msolution_length\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mact_vars\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'action'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mav\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexpanded_actions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mav\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mact_vars\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomains\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m}\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mfluent_values\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# initial state constraints\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfluent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfluent\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfluent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mfluent_values\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# goal state constraints\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfluent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfluent\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# precondition constraints\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'action'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mif_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# st(var, stage) == val if st('action', stage) == act\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstrps\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mexpanded_actions\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfluent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfluent\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstrps\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecond\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# effect constraints\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'action'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mif_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# st(var, stage + 1) == val if st('action', stage) == act\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstrps\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mexpanded_actions\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfluent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfluent\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'Not'\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mfluent\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstrps\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# frame constraints\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconstraints\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mConstraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'action'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meq_if_not_in_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mact\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mact\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mexpanded_actions\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Not'\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meffect\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mvar\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mfluent_values\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstage\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhorizon\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcsp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mNaryCSP\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdomains\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconstraints\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msol\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCSP_solver\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcsp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marc_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0marc_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msol\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0msol\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mact_vars\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource CSPlan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Planning as Boolean Satisfiability Problem\n", + "\n", + "As shown in [[2]](cite-kautz1992planning) the translation of a *Planning Domain Definition Language* (PDDL) description into a *Conjunctive Normal Form* (CNF) formula is a series of straightforward steps:\n", + "- *propositionalize the actions*: replace each action schema with a set of ground actions formed by substituting constants for each of the variables. These ground actions are not part of the translation, but will be used in subsequent steps;\n", + "- *define the initial state*: assert $F^0$ for every fluent $F$ in the problem’s initial state, and $\\lnot{F}$ for every fluent not mentioned in the initial state;\n", + "- *propositionalize the goal*: for every variable in the goal, replace the literals that contain the variable with a disjunction over constants;\n", + "- *add successor-state axioms*: for each fluent $F$, add an axiom of the form\n", + "\n", + "$$ F^{t+1} \\iff ActionCausesF^t \\lor (F^t \\land \\lnot{ActionCausesNotF^t}) $$\n", + "\n", + "where $ActionCausesF$ is a disjunction of all the ground actions that have $F$ in their add list, and $ActionCausesNotF$ is a disjunction of all the ground actions that have $F$ in their delete list;\n", + "- *add precondition axioms*: for each ground action $A$, add the axiom $A^t \\implies PRE(A)^t$, that is, if an action is taken at time $t$, then the preconditions must have been true;\n", + "- *add action exclusion axioms*: say that every action is distinct from every other action.\n", + "\n", + "A propositional planning procedure implements the basic idea just given but, because the agent does not know how many steps it will take to reach the goal, the algorithm tries each possible number of steps $t$, up to some maximum conceivable plan length $T_{max}$ . In this way, it is guaranteed to find the shortest plan if one exists. Because of the way the propositional planning procedure searches for a solution, this approach cannot be used in a partially observable environment, ie WalkSAT, but would just set the unobservable variables to the values it needs to create a solution." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from logic import *" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mSATPlan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msolution_length\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSAT_solver\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcdcl_satisfiable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Section 10.4.1]\u001b[0m\n", + "\u001b[0;34m Planning as Boolean satisfiability\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mexpand_transitions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheck_precond\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mact\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'Not'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_strips\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstate\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mstate\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mexpand_transitions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtransition\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mexpand_transitions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_actions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mSAT_plan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplanning_problem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgoals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msolution_length\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSAT_solver\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mSAT_solver\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource SATPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mSAT_plan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt_max\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSAT_solver\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcdcl_satisfiable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Converts a planning problem to Satisfaction problem by translating it to a cnf sentence.\u001b[0m\n", + "\u001b[0;34m [Figure 7.22]\u001b[0m\n", + "\u001b[0;34m >>> transition = {'A': {'Left': 'A', 'Right': 'B'}, 'B': {'Left': 'A', 'Right': 'C'}, 'C': {'Left': 'B', 'Right': 'C'}}\u001b[0m\n", + "\u001b[0;34m >>> SAT_plan('A', transition, 'C', 1) is None\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Functions used by SAT_plan\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mtranslate_to_SAT\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstates\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mstate\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mstate\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Symbol claiming state s at time t\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate_counter\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcount\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstates\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtime\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"S{}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate_counter\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Add initial state axiom\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Add goal state axiom\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfirst\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mclause\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstate_sym\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0missuperset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \\\n", + " \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# All possible transitions\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtransition_counter\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcount\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstates\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0maction\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0ms_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Action 'action' taken from state 's' at time 't' to reach 's_'\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mExpr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"T{}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtransition_counter\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Change the state from s to s_\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0;34m'==>'\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0;34m'==>'\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Allow only one state at any time\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtime\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# must be a state at any time\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'|'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstates\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstates\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mstates\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mstates\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# for each pair of states s, s_ only one is possible at time t\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mstate_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Restrict to one transition per timestep\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# list of possible transitions at time t\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtransitions_t\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mtr\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mtr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction_sym\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mtr\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# make sure at least one of the transitions happens\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'|'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtr\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mtr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransitions_t\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mtr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransitions_t\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mtr_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtransitions_t\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtransitions_t\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# there cannot be two transitions tr and tr_ at time t\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtr\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0;34m~\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtr_\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Combine the clauses to form the cnf\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'&'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mextract_solution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtrue_transitions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maction_sym\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0maction_sym\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Sort transitions based on time, which is the 3rd element of the tuple\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mtrue_transitions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msort\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0maction\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maction\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtime\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtrue_transitions\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# Body of SAT_plan algorithm\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt_max\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# dictionaries to help extract the solution from model\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mstate_sym\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0maction_sym\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mcnf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtranslate_to_SAT\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransition\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgoal\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSAT_solver\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnf\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mextract_solution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource SAT_plan" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Experimental Results" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Blocks World" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mthree_block_tower\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Figure 10.3] THREE-BLOCK-TOWER\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m A blocks-world problem of stacking three blocks in a certain configuration,\u001b[0m\n", + "\u001b[0;34m also known as the Sussman Anomaly.\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m Example:\u001b[0m\n", + "\u001b[0;34m >>> from planning import *\u001b[0m\n", + "\u001b[0;34m >>> tbt = three_block_tower()\u001b[0m\n", + "\u001b[0;34m >>> tbt.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> tbt.act(expr('MoveToTable(C, A)'))\u001b[0m\n", + "\u001b[0;34m >>> tbt.act(expr('Move(B, Table, C)'))\u001b[0m\n", + "\u001b[0;34m >>> tbt.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> tbt.act(expr('Move(A, Table, B)'))\u001b[0m\n", + "\u001b[0;34m >>> tbt.goal_test()\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m >>>\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(A, Table) & On(B, Table) & On(C, A) & Clear(B) & Clear(C)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(A, B) & On(B, C)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Move(b, x, y)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(b, x) & Clear(b) & Clear(y)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(b, y) & Clear(x) & ~On(b, x) & ~Clear(y)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Block(b) & Block(y)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'MoveToTable(b, x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(b, x) & Clear(b)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'On(b, Table) & Clear(x) & ~On(b, x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Block(b) & Block(x)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Block(A) & Block(B) & Block(C)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource three_block_tower" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### GraphPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 4.46 ms, sys: 124 µs, total: 4.59 ms\n", + "Wall time: 4.48 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = GraphPlan(three_block_tower()).execute()\n", + "linearize(blocks_world_solution)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "14 paths have been expanded and 28 paths remain in the frontier\n", + "CPU times: user 91 ms, sys: 0 ns, total: 91 ms\n", + "Wall time: 89.8 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = uniform_cost_search(ForwardPlan(three_block_tower()), display=True).solution()\n", + "blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution))\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3 paths have been expanded and 9 paths remain in the frontier\n", + "CPU times: user 81.3 ms, sys: 3.11 ms, total: 84.5 ms\n", + "Wall time: 83 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = astar_search(ForwardPlan(three_block_tower()), display=True).solution()\n", + "blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution))\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "116 paths have been expanded and 289 paths remain in the frontier\n", + "CPU times: user 266 ms, sys: 718 µs, total: 267 ms\n", + "Wall time: 265 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = uniform_cost_search(BackwardPlan(three_block_tower()), display=True).solution()\n", + "blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution))\n", + "blocks_world_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 paths have been expanded and 20 paths remain in the frontier\n", + "CPU times: user 477 ms, sys: 450 µs, total: 477 ms\n", + "Wall time: 476 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = astar_search(BackwardPlan(three_block_tower()), display=True).solution()\n", + "blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution))\n", + "blocks_world_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 172 ms, sys: 4.52 ms, total: 176 ms\n", + "Wall time: 175 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = CSPlan(three_block_tower(), 3, arc_heuristic=no_heuristic)\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan with SAT UP Arc Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 267 ms, sys: 0 ns, total: 267 ms\n", + "Wall time: 266 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = CSPlan(three_block_tower(), 3, arc_heuristic=sat_up)\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SATPlan with DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 34.9 s, sys: 15.9 ms, total: 34.9 s\n", + "Wall time: 34.9 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = SATPlan(three_block_tower(), 4, SAT_solver=dpll_satisfiable)\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SATPlan with CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.15 s, sys: 4.01 ms, total: 1.15 s\n", + "Wall time: 1.15 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[MoveToTable(C, A), Move(B, Table, C), Move(A, Table, B)]" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time blocks_world_solution = SATPlan(three_block_tower(), 4, SAT_solver=cdcl_satisfiable)\n", + "blocks_world_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Spare Tire" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mspare_tire\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Figure 10.2] SPARE-TIRE-PROBLEM\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m A problem involving changing the flat tire of a car\u001b[0m\n", + "\u001b[0;34m with a spare tire from the trunk.\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m Example:\u001b[0m\n", + "\u001b[0;34m >>> from planning import *\u001b[0m\n", + "\u001b[0;34m >>> st = spare_tire()\u001b[0m\n", + "\u001b[0;34m >>> st.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> st.act(expr('Remove(Spare, Trunk)'))\u001b[0m\n", + "\u001b[0;34m >>> st.act(expr('Remove(Flat, Axle)'))\u001b[0m\n", + "\u001b[0;34m >>> st.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> st.act(expr('PutOn(Spare, Axle)'))\u001b[0m\n", + "\u001b[0;34m >>> st.goal_test()\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m >>>\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(Flat, Axle) & At(Spare, Trunk)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(Spare, Axle) & At(Flat, Ground)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Remove(obj, loc)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(obj, loc)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(obj, Ground) & ~At(obj, loc)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Tire(obj)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'PutOn(t, Axle)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(t, Ground) & ~At(Flat, Axle)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(t, Axle) & ~At(t, Ground)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Tire(t)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'LeaveOvernight'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m''\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'~At(Spare, Ground) & ~At(Spare, Axle) & ~At(Spare, Trunk) & \\\u001b[0m\n", + "\u001b[0;34m ~At(Flat, Ground) & ~At(Flat, Axle) & ~At(Flat, Trunk)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Tire(Flat) & Tire(Spare)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource spare_tire" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### GraphPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 4.24 ms, sys: 1 µs, total: 4.24 ms\n", + "Wall time: 4.16 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = GraphPlan(spare_tire()).execute()\n", + "linearize(spare_tire_solution)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "11 paths have been expanded and 9 paths remain in the frontier\n", + "CPU times: user 10.3 ms, sys: 0 ns, total: 10.3 ms\n", + "Wall time: 9.89 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = uniform_cost_search(ForwardPlan(spare_tire()), display=True).solution()\n", + "spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution))\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "5 paths have been expanded and 8 paths remain in the frontier\n", + "CPU times: user 20.4 ms, sys: 1 µs, total: 20.4 ms\n", + "Wall time: 19.4 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = astar_search(ForwardPlan(spare_tire()), display=True).solution()\n", + "spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution))\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "29 paths have been expanded and 22 paths remain in the frontier\n", + "CPU times: user 22.2 ms, sys: 7 µs, total: 22.2 ms\n", + "Wall time: 21.3 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = uniform_cost_search(BackwardPlan(spare_tire()), display=True).solution()\n", + "spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution))\n", + "spare_tire_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3 paths have been expanded and 11 paths remain in the frontier\n", + "CPU times: user 13 ms, sys: 0 ns, total: 13 ms\n", + "Wall time: 12.5 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 33, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = astar_search(BackwardPlan(spare_tire()), display=True).solution()\n", + "spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution))\n", + "spare_tire_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 94.7 ms, sys: 0 ns, total: 94.7 ms\n", + "Wall time: 93.2 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = CSPlan(spare_tire(), 3, arc_heuristic=no_heuristic)\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan with SAT UP Arc Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 119 ms, sys: 0 ns, total: 119 ms\n", + "Wall time: 118 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = CSPlan(spare_tire(), 3, arc_heuristic=sat_up)\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SATPlan with DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 9.01 s, sys: 3.98 ms, total: 9.01 s\n", + "Wall time: 9.01 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = SATPlan(spare_tire(), 4, SAT_solver=dpll_satisfiable)\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SATPlan with CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 630 ms, sys: 6 µs, total: 630 ms\n", + "Wall time: 628 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Remove(Spare, Trunk), Remove(Flat, Axle), PutOn(Spare, Axle)]" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time spare_tire_solution = SATPlan(spare_tire(), 4, SAT_solver=cdcl_satisfiable)\n", + "spare_tire_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Shopping Problem" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mshopping_problem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m SHOPPING-PROBLEM\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m A problem of acquiring some items given their availability at certain stores.\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m Example:\u001b[0m\n", + "\u001b[0;34m >>> from planning import *\u001b[0m\n", + "\u001b[0;34m >>> sp = shopping_problem()\u001b[0m\n", + "\u001b[0;34m >>> sp.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> sp.act(expr('Go(Home, HW)'))\u001b[0m\n", + "\u001b[0;34m >>> sp.act(expr('Buy(Drill, HW)'))\u001b[0m\n", + "\u001b[0;34m >>> sp.act(expr('Go(HW, SM)'))\u001b[0m\n", + "\u001b[0;34m >>> sp.act(expr('Buy(Banana, SM)'))\u001b[0m\n", + "\u001b[0;34m >>> sp.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> sp.act(expr('Buy(Milk, SM)'))\u001b[0m\n", + "\u001b[0;34m >>> sp.goal_test()\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m >>>\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(Home) & Sells(SM, Milk) & Sells(SM, Banana) & Sells(HW, Drill)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Have(Milk) & Have(Banana) & Have(Drill)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Buy(x, store)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(store) & Sells(store, x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Have(x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Store(store) & Item(x)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Go(x, y)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(y) & ~At(x)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Place(x) & Place(y)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Place(Home) & Place(SM) & Place(HW) & Store(SM) & Store(HW) & '\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m'Item(Milk) & Item(Banana) & Item(Drill)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource shopping_problem" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### GraphPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 5.08 ms, sys: 3 µs, total: 5.08 ms\n", + "Wall time: 5.03 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, HW), Go(Home, SM), Buy(Milk, SM), Buy(Drill, HW), Buy(Banana, SM)]" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = GraphPlan(shopping_problem()).execute()\n", + "linearize(shopping_problem_solution)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "167 paths have been expanded and 257 paths remain in the frontier\n", + "CPU times: user 187 ms, sys: 4.01 ms, total: 191 ms\n", + "Wall time: 190 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, SM), Buy(Banana, SM), Buy(Milk, SM), Go(SM, HW), Buy(Drill, HW)]" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = uniform_cost_search(ForwardPlan(shopping_problem()), display=True).solution()\n", + "shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution))\n", + "shopping_problem_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "9 paths have been expanded and 22 paths remain in the frontier\n", + "CPU times: user 101 ms, sys: 3 µs, total: 101 ms\n", + "Wall time: 100 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, SM), Buy(Banana, SM), Buy(Milk, SM), Go(SM, HW), Buy(Drill, HW)]" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = astar_search(ForwardPlan(shopping_problem()), display=True).solution()\n", + "shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution))\n", + "shopping_problem_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "176 paths have been expanded and 7 paths remain in the frontier\n", + "CPU times: user 109 ms, sys: 2 µs, total: 109 ms\n", + "Wall time: 107 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, HW), Buy(Drill, HW), Go(HW, SM), Buy(Milk, SM), Buy(Banana, SM)]" + ] + }, + "execution_count": 48, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = uniform_cost_search(BackwardPlan(shopping_problem()), display=True).solution()\n", + "shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution))\n", + "shopping_problem_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "18 paths have been expanded and 28 paths remain in the frontier\n", + "CPU times: user 235 ms, sys: 9 µs, total: 235 ms\n", + "Wall time: 234 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, SM), Buy(Banana, SM), Buy(Milk, SM), Go(SM, HW), Buy(Drill, HW)]" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = astar_search(BackwardPlan(shopping_problem()), display=True).solution()\n", + "shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution))\n", + "shopping_problem_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 194 ms, sys: 6 µs, total: 194 ms\n", + "Wall time: 192 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, HW), Buy(Drill, HW), Go(HW, SM), Buy(Banana, SM), Buy(Milk, SM)]" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = CSPlan(shopping_problem(), 5, arc_heuristic=no_heuristic)\n", + "shopping_problem_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan with SAT UP Arc Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 235 ms, sys: 7 µs, total: 235 ms\n", + "Wall time: 233 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, HW), Buy(Drill, HW), Go(HW, SM), Buy(Banana, SM), Buy(Milk, SM)]" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = CSPlan(shopping_problem(), 5, arc_heuristic=sat_up)\n", + "shopping_problem_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SATPlan with CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1min 29s, sys: 36 ms, total: 1min 29s\n", + "Wall time: 1min 29s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Go(Home, HW), Buy(Drill, HW), Go(HW, SM), Buy(Banana, SM), Buy(Milk, SM)]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time shopping_problem_solution = SATPlan(shopping_problem(), 5, SAT_solver=cdcl_satisfiable)\n", + "shopping_problem_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Air Cargo" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mair_cargo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Figure 10.1] AIR-CARGO-PROBLEM\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m An air-cargo shipment problem for delivering cargo to different locations,\u001b[0m\n", + "\u001b[0;34m given the starting location and airplanes.\u001b[0m\n", + "\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m Example:\u001b[0m\n", + "\u001b[0;34m >>> from planning import *\u001b[0m\n", + "\u001b[0;34m >>> ac = air_cargo()\u001b[0m\n", + "\u001b[0;34m >>> ac.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Load(C2, P2, JFK)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Load(C1, P1, SFO)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Fly(P1, SFO, JFK)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Fly(P2, JFK, SFO)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Unload(C2, P2, SFO)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.goal_test()\u001b[0m\n", + "\u001b[0;34m False\u001b[0m\n", + "\u001b[0;34m >>> ac.act(expr('Unload(C1, P1, JFK)'))\u001b[0m\n", + "\u001b[0;34m >>> ac.goal_test()\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m >>>\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mPlanningProblem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minitial\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mgoals\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(C1, JFK) & At(C2, SFO)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mactions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Load(c, p, a)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(c, a) & At(p, a)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'In(c, p) & ~At(c, a)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Cargo(c) & Plane(p) & Airport(a)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Unload(c, p, a)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'In(c, p) & At(p, a)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(c, a) & ~In(c, p)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Cargo(c) & Plane(p) & Airport(a)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mAction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Fly(p, f, to)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mprecond\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(p, f)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0meffect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'At(p, to) & ~At(p, f)'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Plane(p) & Airport(f) & Airport(to)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdomain\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'Cargo(C1) & Cargo(C2) & Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource air_cargo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### GraphPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 9.06 ms, sys: 3 µs, total: 9.06 ms\n", + "Wall time: 8.94 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Unload(C1, P1, JFK),\n", + " Unload(C2, P2, SFO)]" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = GraphPlan(air_cargo()).execute()\n", + "linearize(air_cargo_solution)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "838 paths have been expanded and 1288 paths remain in the frontier\n", + "CPU times: user 3.56 s, sys: 4 ms, total: 3.57 s\n", + "Wall time: 3.56 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Unload(C2, P2, SFO),\n", + " Load(C1, P2, SFO),\n", + " Fly(P2, SFO, JFK),\n", + " Unload(C1, P2, JFK)]" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = uniform_cost_search(ForwardPlan(air_cargo()), display=True).solution()\n", + "air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution))\n", + "air_cargo_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### ForwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "17 paths have been expanded and 54 paths remain in the frontier\n", + "CPU times: user 716 ms, sys: 0 ns, total: 716 ms\n", + "Wall time: 717 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Unload(C2, P2, SFO),\n", + " Load(C1, P2, SFO),\n", + " Fly(P2, SFO, JFK),\n", + " Unload(C1, P2, JFK)]" + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = astar_search(ForwardPlan(air_cargo()), display=True).solution()\n", + "air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution))\n", + "air_cargo_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "506 paths have been expanded and 65 paths remain in the frontier\n", + "CPU times: user 970 ms, sys: 0 ns, total: 970 ms\n", + "Wall time: 971 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Load(C2, P1, JFK),\n", + " Unload(C1, P1, JFK),\n", + " Fly(P1, JFK, SFO),\n", + " Unload(C2, P1, SFO)]" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = uniform_cost_search(BackwardPlan(air_cargo()), display=True).solution()\n", + "air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution))\n", + "air_cargo_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BackwardPlan with Ignore Delete Lists Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "23 paths have been expanded and 50 paths remain in the frontier\n", + "CPU times: user 1.19 s, sys: 2 µs, total: 1.19 s\n", + "Wall time: 1.2 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C2, P2, JFK),\n", + " Fly(P2, JFK, SFO),\n", + " Unload(C2, P2, SFO),\n", + " Load(C1, P2, SFO),\n", + " Fly(P2, SFO, JFK),\n", + " Unload(C1, P2, JFK)]" + ] + }, + "execution_count": 42, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = astar_search(BackwardPlan(air_cargo()), display=True).solution()\n", + "air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution))\n", + "air_cargo_solution[::-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 6.5 s, sys: 0 ns, total: 6.5 s\n", + "Wall time: 6.51 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Load(C2, P1, JFK),\n", + " Unload(C1, P1, JFK),\n", + " Fly(P1, JFK, SFO),\n", + " Unload(C2, P1, SFO)]" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = CSPlan(air_cargo(), 6, arc_heuristic=no_heuristic)\n", + "air_cargo_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSPlan with SAT UP Arc Heuristic" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 13.6 s, sys: 7.98 ms, total: 13.7 s\n", + "Wall time: 13.7 s\n" + ] + }, + { + "data": { + "text/plain": [ + "[Load(C1, P1, SFO),\n", + " Fly(P1, SFO, JFK),\n", + " Load(C2, P1, JFK),\n", + " Unload(C1, P1, JFK),\n", + " Fly(P1, JFK, SFO),\n", + " Unload(C2, P1, SFO)]" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time air_cargo_solution = CSPlan(air_cargo(), 6, arc_heuristic=sat_up)\n", + "air_cargo_solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## References\n", + "\n", + "[[1]](#ref-1) Hoffmann, Jörg. 2001. _FF: The fast-forward planning system_.\n", + "\n", + "[[2]](#ref-2) Kautz, Henry A and Selman, Bart and others. 1992. _Planning as Satisfiability_." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.5rc1" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/csp.ipynb b/csp.ipynb index 411d6f55c..5d490846b 100644 --- a/csp.ipynb +++ b/csp.ipynb @@ -16,7 +16,7 @@ "outputs": [], "source": [ "from csp import *\n", - "from notebook import psource, pseudocode, plot_NQueens\n", + "from notebook import psource, plot_NQueens\n", "%matplotlib inline\n", "\n", "# Hide warnings in the matplotlib sections\n", @@ -183,7 +183,6 @@ " def __init__(self, variables, domains, neighbors, constraints):\n", " """Construct a CSP problem. If variables is empty, it becomes domains.keys()."""\n", " variables = variables or list(domains.keys())\n", - "\n", " self.variables = variables\n", " self.domains = domains\n", " self.neighbors = neighbors\n", @@ -206,10 +205,12 @@ "\n", " def nconflicts(self, var, val, assignment):\n", " """Return the number of conflicts var=val has with other variables."""\n", + "\n", " # Subclasses may implement this more efficiently\n", " def conflict(var2):\n", " return (var2 in assignment and\n", " not self.constraints(var, val, var2, assignment[var2]))\n", + "\n", " return count(conflict(v) for v in self.neighbors[var])\n", "\n", " def display(self, assignment):\n", @@ -607,7 +608,9 @@ { "data": { "text/plain": [ - "(, , )" + "(,\n", + " ,\n", + " )" ] }, "execution_count": 7, @@ -616,7 +619,7 @@ } ], "source": [ - "australia, usa, france" + "australia_csp, usa_csp, france_csp" ] }, { @@ -868,16 +871,16 @@ " CSP.__init__(self, list(range(n)), UniversalDict(list(range(n))),\n", " UniversalDict(list(range(n))), queen_constraint)\n", "\n", - " self.rows = [0]*n\n", - " self.ups = [0]*(2*n - 1)\n", - " self.downs = [0]*(2*n - 1)\n", + " self.rows = [0] * n\n", + " self.ups = [0] * (2 * n - 1)\n", + " self.downs = [0] * (2 * n - 1)\n", "\n", " def nconflicts(self, var, val, assignment):\n", " """The number of conflicts, as recorded with each assignment.\n", " Count conflicts in row and in up, down diagonals. If there\n", " is a queen there, it can't conflict with itself, so subtract 3."""\n", " n = len(self.variables)\n", - " c = self.rows[val] + self.downs[var+val] + self.ups[var-val+n-1]\n", + " c = self.rows[val] + self.downs[var + val] + self.ups[var - val + n - 1]\n", " if assignment.get(var, None) == val:\n", " c -= 3\n", " return c\n", @@ -1074,7 +1077,7 @@ "

    \n", "\n", "
    def min_conflicts(csp, max_steps=100000):\n",
    -       "    """Solve a CSP by stochastic hillclimbing on the number of conflicts."""\n",
    +       "    """Solve a CSP by stochastic Hill Climbing on the number of conflicts."""\n",
            "    # Generate a complete assignment for all variables (probably with conflicts)\n",
            "    csp.current = current = {}\n",
            "    for var in csp.variables:\n",
    @@ -1137,12 +1140,14 @@
        "outputs": [
         {
          "data": {
    -      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+4FNWd7/vPd9gbEMOvDRtMgGtgkifnTowY2SPOELnEkDEgGD137gxco0dzczk39xiC4GRGnmeemDwnmqsCIXHu5OTIgOeMAc04RtREiUYwYNTZMMrEZOY+BkxE5McWdkC3icBZ94/a7e7uXVVdu7uqq6vq/Xqefrq7atVaq3ux+fZatWqVOecEAABa2++lXQEAAFAbARsAgAwgYAMAkAEEbAAAMoCADQBABhCwAQDIAAI2AAAZQMAGACADCNhAizGz95vZD8zsuJkdMrO7zKwtJP04M/vb/rR9ZvYvZvYfmllnAMkjYAOt5/+VdETSeyVdIOl/kfR/+yU0s+GSnpB0rqQ/kjRW0l9Iut3MljeltgCagoANtJ7pku53zv3WOXdI0mOSPhyQ9hpJ/5Ok/805t985d8o595ik5ZL+s5mNliQzc2b2gdJBZrbJzP5z2ftFZvaCmfWa2TNmdn7ZvveZ2QNmdtTM9pf/EDCzW8zsfjP7b2Z20sxeMrOusv1/aWav9e/7NzP7RDxfEVA8BGyg9ayXtMTMRpnZFEkL5AVtP5+U9EPn3FtV2x+QNErSxbUKM7MLJf2dpP8oaYKk/yJpq5mNMLPfk/SwpBclTZH0CUkrzOyysiyukLRF0jhJWyXd1Z/vhyTdIOkPnXOjJV0m6ZVa9QHgj4ANtJ4d8nrUJyQdkNQt6fsBaSdKer16o3PutKQeSZ0Ryvs/Jf0X59xzzrkzzrl7JP1OXrD/Q0mdzrmvOufecc7tk/RfJS0pO36nc+4Hzrkzkv67pJn9289IGiHpD8ys3Tn3inPulxHqA8AHARtoIf092scl/aOks+UF5PGS/p+AQ3rkneuuzqet/9ijEYo9V9Kq/uHwXjPrlTRN0vv6972vat9qSZPLjj9U9rpP0kgza3POvSxphaRbJB0xsy1m9r4I9QHgg4ANtJYOecHyLufc75xzb0jaKGlhQPonJC0ws7Ortv+vkk5Jer7/fZ+8IfKSc8pevyrpa865cWWPUc65zf379lftG+2cC6pPBefcd51zH5MX+J2Cf3gAqIGADbQQ51yPpP2SPm9mbWY2TtJ/kHcO2c9/lzds/r3+y8Ha+88vf1PS7c653/Sne0HS/25mw8zsU/Jmnpf8V0n/l5nNNs/ZZnZ5/4S15yWd6J88dlb/8eeZ2R/W+ixm9iEzu9TMRkj6raS35Q2TA6gDARtoPf9e0qfkDWe/LOm0pBv9Ejrnfidpvrye8HPyguJjkr4h6StlSb8oabGkXklXq+ycuHOuW9557LskHe8v87r+fWf6j7tA3g+JHkl3y7t8rJYRkr7ef8whSZPkDacDqIM559KuA4CYmFm7pB9Kek3SdY4/cCA36GEDOeKcOyXv/PUvJX0o5eoAiBE9bAAAMoAeNgAAGRB4Q4FmmThxonv/+9+fdjUSs3v37rSrkKhZs2alXYXE0YbZRvtlX97bUFKPc67mIkepD4l3dXW57u7uVOuQJDNLuwqJSvvfTzPE1YYuhn/mA6t0xyfvbcjfYPblvQ0l7XbO1fzrZkgcSNBN13iBOo5gLQ3ktfLqePIDkB0EbCABHWO8wHrHF5PJf82NXv6TOpLJH0DrSf0cNpA3cfWmozi8zXtOYqgcQGuhhw3EqJnBuhXKBdA8BGwgBr99Jv2g6bqlP/9kunUAkBwCNtAg1y2NGN54Pjfc3ngeW25L/4cDgGRwDhtowNu7Gs+j/Pzz39zvPTcadH/7jDTyjxvLA0BroYcNNGDkiNppOudL9/7Qf1/QZLFGJ5HF0eMH0FoI2ECdavWCrct79PRKn/nrxoNwKb/S47w/a6x+ALKFgA3UoVYw/NZ9/tvrDdp+x720r/ZxBG0gPwjYwBB1RlisZPkdyddDivYDYMLY5OsBIHkEbGCIjmyLL6+gHnCcPeOeJ+PLC0B6mCUODMFfXDPw2q93Wwq0rjv68Lfrlk72SWPmSieelkaPil6fjV+OVp8VS6VvbI6eL4DWQw8bGILb+9cGDwrGB44MvJ4zc/D+oJ5zKUgHBeug465b7D3/+pD//lI9163y3w8gOwjYQIymLRx4vXNDZaANG+b+4FXe84RLg9NU51X+/txFQ6sngOwhYAMRNXpe+bUjwfteftV7PnYiOE3YviiYMQ5kGwEbiNHCOcH7pi4M3hdFWO970SWN5Q2g9RGwgTr0BSxJ+uj65taj5OF1/tvffqa59QCQHAI2EMHkCZXvzxrhDTGfVbY0aZQh500P11f+Qztqpykvf9RI7/3IqiVKJ46rr3wA6SNgAxEcetx/e98u6dRz3usol3Fd/5XB206fqXzf0zs4zZURZnmXyu/dLr210z/N0Sdq5wOgNRGwgQa1DWvs+OEXV77vnN9YfmPf09jxAFoTARuIUZRe9pLVle+dC0//2a/GUy6AbCNgA0123xCXNt24NZl6AMiWRAK2mX3KzP7NzF42s79KogygmVaujZ622b3doZQ3lM8BoLXEHrDNbJikv5G0QNIfSFpqZn8QdzlAM61dGW9+n78tWrq47/oV9+cA0DxJ9LAvkvSyc26fc+4dSVskfTqBcoCWtWhF+P5vP+A979jjv3/r095z0H21S6pnj197ee26AcimJAL2FEmvlr0/0L/tXWa2zMy6zaz76NGjCVQBaK7p76t8/2jAZVXV5i3z3/7piD3h6uuz7/G5bAxAPiQRsM1nW8U8WOfcd5xzXc65rs7OzgSqADTXT+4evG3B8vBjOkKWGpWk8R8P379iTfh+APmSRMA+IGla2fupkg4mUA7QNBM/Eb5/yqTB2x6rsSzo8Ro38+g9Gb5/fR33tw5bjxxAa0siYP+TpA+a2XQzGy5piSQuTEGmvfGb+o5Lasb4VTfVd1yjd/wCkJ62uDN0zp02sxskPS5pmKS/c869FHc5QJF9f3vaNQDQbLEHbElyzv1A0g+SyBtoVZM7pMPH0it/9nnplQ0geax0BkRUa3j70BBXMCv3kQ9I8y+Sfn9q/Xk8uyl8P8uXAtmWSA8bKCrXHRwYF85p7H7Zl90gbXs2uFwA+UbABoZg1TppzY3haXq3S+Pmea8Pb5MmdVTuv+4W6Z5Hopc5Z6a0c4P0+F0D2/YflGZc4b2O0rP/QswrpgFoPnO1bhWUsK6uLtfdnd/ugZnfZen5kfa/n2aobsMovVnrGki3ZZu0dHV4+qH47tekpZcNLqdWfYLkvQ35G8y+vLehpN3OuZonrQjYCcv7P7S0//00Q3UbThwnHX0iwnERzxkvnitdv1iaN0s6flL66V7p1o3Sz/fVPjZKsJ5wafjlXHlvQ/4Gsy/vbaiIAZshcWCIenrrP3brWi9ABxk/RpoxRbp6QeX2nS9Il3yuvjK59hrIBwI2UIcoQ9GlCWjtbdI7VZPFhjJj23VLH7tgoLz22dLpM40PhQPIFgI2UKeo549Lwbre4Fl+3JnnpVPPRcuLYA3kC9dhAw1YcnPtNNYVHDxvWSYdf8oL/KVH3y5vu59hF0ULxH/6pdppAGQLk84SlvfJEmn/+2mGWm0Y1MuuDqxXzpMevLP+eixd7c04r6fsMHlvQ/4Gsy/vbSgmnQHNYV3SWzulUSMH7+t5UpowtnLb6LnSm33R8+8YI73xY2nzrd5Dkr6+Sbr5rsFpl9ws3fej6HkDyA4CNhCDsz/mPVf3eNuGSdOvkF5p4Aazx05U9ph/9cjgnrbEOWsg7ziHDcSoPGi6bumhHY0Faz/nLvKu2y7/cUCwBvKPHjYQM+uSxo+Wjj0lXXu590hK5/zGrgsHkB30sIEEHD/pBe4Va5LJf/kdXv4Ea6A46GEDCVq/2XtI8dxRi6FvoLjoYQNNUroe27oG7uZVbtW6wdvOuazyOADFRQ8bSMFv3vQPwGvvbX5dAGQDPWwAADKAgA0AQAYQsAEAyAACNgAAGZD6zT/MLNcr16f9/SatAIvy04YZR/tlXwHakJt/5NqZ49ILHRWbVq2T1txYle78g1L7e5tXLwBAIuhhJyzW73d3DL+kZ8X7dfPrPvvy3oa0X/YVoA0j9bA5h93qDt/hBeo4grU0kNfhhNbMBAAkgh52wur+fk+9Ie2dGG9l/Jx/SGqfXPfh/LrPvry3Ie2XfQVoQ85hZ1Zcveko9p7jPcc8VA4AiBdD4q2mmcG6FcoFAERCwG4Ve0akHzR3m3RsS7p1AAD4ImC3gt0muXcazuaG22Ooy/6l6f9wAAAMwqSzhNX8fveMlNzvGirD765PDd972YZLF9auFxNesi/vbUj7ZV8B2pDLujIhQrDunC/d+0P/fUH3SG743skx9PgBAPGhh52w0O+3xtBzlJ5zWGCulfbDM6Sf3R9ahZqzx/l1n315b0PaL/sK0Ib0sFtajWD9rfv8t9fbc/Y77qV9EQ7kfDYAtAQCdhpOH6mZZPkdTaiHIv4AON2TeD0AAOEI2Gl4sf6VxaoFTS5reNJZuRc7Y8wMAFAPVjprttcHrr0KO0ftuqMPf7tu6WSfNGaudOJpafSo6NXZ+OWB16HnzA+tk86pvhUYAKBZ6GE328G/lBQcjA+UjZbPmTl4f1DPuRSkg4J10HHXLfaef33If/+79XxtpX8CAEBTELBbzLSFA693bqgMtGHD3B+8ynuecGlwmuq8yt+fu2ho9QQANBcBu5kanHH9WshctZdf9Z6PnQhOE7YvEmaMA0BqCNgtZuGc4H1TFwbviyKs973oksbyBgAki4Cdkr5d/tsfXd/cepQ8vM5/+9vPNLceAAB/BOxmOVU5q+usEd455LNGDGyLcinWpofrK/6hHbXTlJc/aqT3fuTwqkSnjtZXAQBAQ1iaNGHvfr8h539Pn5HaZ/en9wna1TPKq9OUHy9JR5+QJo4bWh7laXq3S2PfE1jdiuVKWRYx+/LehrRf9hWgDVmaNCvahjV2/PCLK993zm8sv9BgDQBIBQG7xURZLGXJ6sr3tX58fvar8ZQLAEhP7AHbzP7OzI6Y2c/izhue+7YNLf3GrcnUAwDQPEn0sDdJ+lQC+WbayrXR0za7tzuU8obyOQAA8Yk9YDvnnpZ0LO58s25tzCt7fv62aOnivutX3J8DABAN57Bb1KIV4fu//YD3vGOP//6tT3vPQffVLrlyVeX7ay+vXTcAQPOlErDNbJmZdZtZnDeBzLTp76t8/+jOaMfNW+a//dMRe8LV12ff85VoxwEAmiuVgO2c+45zrivKdWdF8ZO7B29bsDz8mI6QpUYlafzHw/evWBO+HwDQOhgSb5aZ4SuETZk0eNtjNZYFPV7jZh69J8P3r98cvt/X+T11HAQAaFQSl3VtlvRTSR8yswNm9n/EXUYmtU2s67CkZoxfdVOdB7ZPiLUeAIBo2uLO0Dm3NO48Eb/vb0+7BgCAoWBIvIVM7ki3/NnnpVs+ACAYN/9I2KDvN+QmIFL9Q+Af+YAX8PcflH55oL48at4tbNbgpuLGA9mX9zak/bKvAG0Y6eYfsQ+JozGuOzhoL5zT2P2yL7tB2vZscLkAgNZFwG62qXdKB8JnfPVul8bN814f3iZNqhoqv+4W6Z5Hohc5Z6a0c4P0+F0D2/YflGZc4b0+FGVt8mnfjF4gACB2DIknzPf7rTEsLnm97FKvd8s2aenq8PRD8d2vSUsvG1xOKJ/hcInhuDzIexvSftlXgDaMNCROwE6Y7/d76qi01+fC6ypRz2cvnitdv1iaN0s6flL66V7p1o3Sz/dFqF+UYH1+T+DlXPxnkX15b0PaL/sK0Iacw25Z7Z11H7p1rRegg4wfI82YIl29oHL7zhekSz5XZ6Fcew0AqaOHnbDQ7zfi0Hh7m/TOs4O3R65DVS+6fbZ0+kxjQ+Hv1oNf95mX9zak/bKvAG1ID7vlzXKRgnYpWNd7yVf5cWeel049FzGvGsEaANA8LJyStum1F/S2ruAAe8sy6fhTXm+59Ojb5W33M+yiiMF6+vciJAIANAtD4gmL9P0G9LKrA+uV86QH76y/LktXezPOywUOi0fsXTMcl315b0PaL/sK0IbMEm8Fkb/fPaMk93bFJuuSep6UJoytTDp6rvRmX/Q6dIyR3vhx5bavb5JuvssnYE/fLHUsiZw3/1lkX97bkPbLvgK0IeewM+XC/ghc1dtuGyZNv0J65WD9WR87Udlb/9Ujg3vakjhnDQAtjHPYraYsaLpu6aEdjQVrP+cu8q7bruhdE6wBoKUxJJ6wur/fU8ekvU24/vn8Iw1dF85wXPblvQ1pv+wrQBtGGhKnh92q2ju8Xu+0dcnkP229l38DwRoA0Dz0sBMW6/cb4ZrtmmIe+ubXffblvQ1pv+wrQBvSw86dWW7gMfP4oN2r/Drj579eeRwAIJPoYScs7e83afy6z768tyHtl30FaEN62AAA5AUBGwCADCBgAwCQAamvdDZr1ix1d0e5z2M25f38Ut7PLUm0YdbRftmX9zaMih42AAAZkHoPGwCAZgm8Q+EQRLpFcQLoYQMAcu2ma7xAHUewlgbyWnl1PPlFRcAGAORSxxgvsN7xxWTyX3Ojl/+kjmTyr8aQOAAgd+LqTUdxuP92xUkPldPDBgDkSjODdTPLJWADAHLht8+kF6xLXLf0559MJm8CNgAg81y3NGJ44/nccHvjeWy5LZkfDpzDBgBk2tu7Gs+j/Pzz39zvPTcadH/7jDTyjxvLoxw9bABApo0cUTtN53zp3h/67wuaLNboJLI4evzlCNgAgMyq1Qu2Lu/R0yt95q8bD8Kl/EqP8/6ssfoNBQEbAJBJtYLht+7z315v0PY77qV9tY+LK2gTsAEAmdMZYbGS5XckXw8p2g+ACWMbL4eADQDInCPb4ssrqAcc53B2z5ON58EscQBApvzFNQOv/Xq3pUDruqMPf7tu6WSfNGaudOJpafSo6PXZ+OVo9VmxVPrG5uj5VqOHDQDIlNv71wYPCsYHjgy8njNz8P6gnnMpSAcF66DjrlvsPf/6kP/+Uj3XrfLfHxUBGwCQK9MWDrzeuaEy0IYNc3/wKu95wqXBaarzKn9/7qKh1XOoCNgAgMxo9Lzya0eC9738qvd87ERwmrB9UTRSfwI2ACBXFs4J3jd1YfC+KMJ634suaSzvWgjYAIBM6gtYkvTR9c2tR8nD6/y3v/1MPPkTsAEAmTB5QuX7s0Z4Q8xnlS1NGmXIedPD9ZX/0I7aacrLHzXSez+yaonSiePqK5+ADQDIhEOP+2/v2yWdes57HeUyruu/Mnjb6TOV73t6B6e5MsIs71L5vdult3b6pzn6RO18/BCwAQCZ1zasseOHX1z5vnN+Y/mNfU9jx/shYAMAciVKL3vJ6sr3zoWn/+xX4ym3EQRsAEDh3DfEpU03bk2mHkMRe8A2s2lm9pSZ/cLMXjKzL8ZdBgCgeFaujZ426d5uI+UN5XOUS6KHfVrSKufc/yzpYkn/ycz+IIFyAAAFsnZlvPl9/rZo6eK+61e9nyP2gO2ce905t6f/9UlJv5A0Je5yAAAIs2hF+P5vP+A979jjv3/r095z0H21S6pnj197ee261SPRc9hm9n5JH5X0XNX2ZWbWbWbdR48eTbIKAICCmP6+yvePBlxWVW3eMv/tn47YE66+Pvsen8vG4pBYwDaz90h6QNIK51zF6qvOue8457qcc12dnZ1JVQEAUCA/uXvwtgXLw4/pCFlqVJLGfzx8/4o14fvjlEjANrN2ecH6XufcPyZRBgCgWCZ+Inz/lEmDtz1WY1nQ4zVu5tF7Mnz/+jrubx22HnmYJGaJm6QNkn7hnKtzLhwAAJXe+E19xyU1Y/yqm+o7rt47fiXRw54j6RpJl5rZC/2PBu+PAgBAa/n+9uaW1xZ3hs65nZIs7nwBAKhlcod0+Fh65c8+L7m8WekMAJAZtYa3Dw1xBbNyH/mANP8i6fen1p/Hs5vC9zcyPB97DxsAgDS57uDAuHBOY/fLvuwGaduzweUmiYANAMiUVeukNTeGp+ndLo2b570+vE2a1FG5/7pbpHseiV7mnJnSzg3S43cNbNt/UJpxhfc6Ss/+Cw2umGau1i1KEtbV1eW6uxP+WZIib9J8fqX976cZaMNso/2yz68No/RmrWsg3ZZt0tLV4emH4rtfk5ZeNricWvUJsNs5V3OwnICdMP6zyD7aMNtov+zza8OJ46SjT0Q4NuI548VzpesXS/NmScdPSj/dK926Ufr5vtrHRgnWEy4NvZwrUsBmSBwAkDk9vfUfu3WtF6CDjB8jzZgiXb2gcvvOF6RLPldfmfVee12OgA0AyKQoQ9GlCWjtbdI7VZPFhjJj23VLH7tgoLz22dLpMw0PhQ8JARsAkFlRzx+XgnW9wbP8uDPPS6eei5ZXnKuscR02ACDTltxcO411BQfPW5ZJx5/yAn/p0bfL2+5n2EXRAvGffql2mqFg0lnCmPCSfbRhttF+2RelDYN62dWB9cp50oN31l+Xpau9Gef1lB2CSWcAgGKwLumtndKokYP39TwpTRhbuW30XOnNvuj5d4yR3vixtPlW7yFJX98k3XzX4LRLbpbu+1H0vKMiYAMAcuHsj3nP1T3etmHS9CukVw7Wn/exE5U95l89MrinLSV3ZzCJc9gAgJwpD5quW3poR2PB2s+5i7zrtst/HCQZrCV62ACAHLIuafxo6dhT0rWXe4+kdM5v7LrwqOhhAwBy6fhJL3CvWJNM/svv8PJvRrCW6GEDAHJu/WbvIcVzR62kh76D0MMGABRG6Xps6xq4m1e5VesGbzvnssrj0kIPGwBQSL950z8Ar723+XWJgh42AAAZQMAGACADCNgAAGQAARsAgAxI/eYfZpbrlevT/n6TlvcbK0i0YdbRftlXgDaMdPMPetgAEjFudOXtCl23tPLqwdvOmZB2TYFsoIedsLS/36Tx6z774mzDVlyUgvbLvgK0IT1sAMm76ZqB3nIcynvjAAbQw05Y2t9v0vh1n331tmHp/sBJm/wn0pFj9R9P+2VfAdowUg+blc4ADFlcvekoDvffczjNJSGBVsCQOIAhaWawboVygVZBwAYQyW+fST9oum7pzz+Zbh2AtBCwAdTkuqURwxvP54bbG89jy23p/3AA0sCks4Sl/f0mjQkv2VerDd/eJY0c0WAZPuefGw26v3tHGvnHtdMVvf3yoABtyGVdABoXJVh3zpfu/aH/vqDJYo1OIoujxw9kCT3shKX9/SaNX/fZF9aGtXrBUXrOYYG5VtoPz5B+dv/Q61BRRoHbLy8K0Ib0sAHUr1aw/tZ9/tvr7Tn7HffSvtrHcT4bRUHABjBIZ0ftNMvvSL4eUrQfABPGJl8PIG0EbACDHNkWX15BPeA4e8Y9T8aXF9CqWOkMQIW/uGbgddg5atcdffjbdUsn+6Qxc6UTT0ujR0Wvz8YvR6vPiqXSNzZHzxfIGnrYACrc/kXvOSgYHzgy8HrOzMH7g3rOpSAdFKyDjrtusff860P++0v1XLfKfz+QFwRsAEMybeHA650bKgNt2DD3B6/ynidcGpymOq/y9+cuGlo9gbwhYAN4V6PnlV87Erzv5Ve952MngtOE7YuCGePIMwI2gCFZOCd439SFwfuiCOt9L7qksbyBrCNgA/DVt8t/+6Prm1uPkofX+W9/+5nm1gNICwEbgCRp8oTK92eN8IaYzypbmjTKkPOmh+sr/6EdtdOUlz9qpPd+ZNUSpRPH1Vc+0OpYmjRhaX+/SWNZxOwrtWFYMD59RmqfrcB01TPKq9OUHy9JR58YHFhr5VGepne7NPY9wfUtz6so7ZdnBWhDliYFEI+2YY0dP/ziyved8xvLLyxYA3lFwAYwJFEWS1myuvJ9rQ7SZ78aT7lAnsUesM1spJk9b2YvmtlLZvaVuMsA0NruG+LSphu3JlMPIE+S6GH/TtKlzrmZki6Q9Ckzu7jGMQBStnJt9LTN7u0OpbyhfA4gS2IP2M7zZv/b9v5HvmcMADmwdmW8+X3+tmjp4r7rV9yfA2gViZzDNrNhZvaCpCOSfuSce65q/zIz6zYz1iUCMmrRivD9337Ae96xx3//1qe956D7apdcWbVG+LWX164bkEeJXtZlZuMkPSjpC865nwWkyXXvuwCXI6RdhcQVpQ1rXWM94wpp/8HKbaVjgoasa93RK2x/UN5RrgXnsq58KUAbpn9Zl3OuV9J2SZ9KshwAyfvJ3YO3LVgefkxHyFKjkjT+4+H7V6wJ3w8USRKzxDv7e9Yys7MkzZf0r3GXAyBeEz8Rvn/KpMHbHquxLOjxGjfz6D0Zvn99Hfe3DluPHMiytgTyfK+ke8xsmLwfBPc75x5JoBwAMXrjN/Udl9SM8atuqu+4Ru/4BbSq2AO2c26vpI/GnS+AYvn+9rRrALQWVjoDENnkjnTLn31euuUDaeLmHwlL+/tNGjNUs6+6DWvNwq53CPwjH/AC/v6D0i8P1JdHPXUrWvvlUQHaMNIs8STOYQPIsbBLsRbOaex+2ZfdIG17NrhcoMgI2AAqrFonrbkxPE3vdmncPO/14W3SpKqh8utuke4ZwlTTOTOlnRukx+8a2Lb/oHfttyQdirA2+RdiXjENaDUMiScs7e83aQzHZZ9fG0ZdnKSUbss2aenq8PRD8d2vSUsvG1xOrfr4KWL75U0B2jDSkDgBO2Fpf79J4z+L7PNrw4njpKNPRDg24vnsxXOl6xdL82ZJx09KP90r3bpR+vm+2sdGCdYTLg2+nKuI7Zc3BWhDzmEDqE9Pb/3Hbl3rBegg48dIM6ZIVy+o3L7zBemSz9VXJtdeowjoYScs7e83afy6z76wNow6FN3eJr3z7ODtUVWX0z5bOn2msaHwd/MucPvlRQHakB42gMZEPX9cCtb1XvJVftyZ56VTz0XLq9n35QbSxMIpAEItubl2GusKDp63LJOOP+UF/tKjb5e33c+wi6IF4j/9Uu00QJ4wJJ6wtL/fpDEcl31R2jCol10dWK+cJz14Z/3uE0VqAAAgAElEQVR1Wbram3FeT9lBaL/sK0AbMku8FaT9/SaN/yyyL2obvrVTGjWy6tguqedJacLYyu2j50pv9kWvQ8cY6Y0fV277+ibp5rsGB+wlN0v3/Sh63rRf9hWgDTmHDSA+Z3/Me64OoG3DpOlXSK8crD/vYycqe8y/emRwT1vinDWKjXPYAIakPGi6bumhHY0Faz/nLvKu2y7/cUCwRtExJJ6wtL/fpDEcl331tuH40dKxp2KujI/O+Y1dF077ZV8B2jDSkDg9bAB1OX7S6/WuWJNM/svv6D9H3kCwBvKEHnbC0v5+k8av++yLsw3juKNW3EPftF/2FaAN6WEDaK7S9djWNXA3r3Kr1g3eds5llccB8EcPO2Fpf79J49d99uW9DWm/7CtAG9LDBgAgLwjYAABkAAEbAIAMSH2ls1mzZqm7O4appS0q7+eX8n5uSaINs472y768t2FU9LABAMiA1HvYAIAWsjuG3uys/Pf600APGwCK7vAdXqCOI1hLA3kdTmgZvIIiYANAUZ16wwusB76UTP4HbvLyP3U4mfwLhiFxACiiuHrTUew9x3tmqLwh9LABoGiaGaxbodycIGADQFHsGZF+0Nxt0rEt6dYhowjYAFAEu01y7zSczQ23x1CX/UvT/+GQQZzDBoC82zOy4SzK76T2N/d7zw3fTnXPCOnC3zWYSXHQwwaAvHO1g2LnfOneH/rvC7rtacO3Q42hx18kBGwAyLMaQ8+l+5D39Eqf+evGg3D5vc2tSzrvzxqrHwYQsAEgr2oEw2/d57+93qDtd9xL+yIcSNCOhIANAHl0+kjNJMvvaEI9FPEHwOmexOuRdQRsAMijFyfHllXQ5LKGJ52Ve7EzxszyiVniAJA3rw9ce+XXuy0FWtcdffjbdUsn+6Qxc6UTT0ujR0WvzsYvD7wOq48OrZPOuTF6xgVDDxsA8ubgX0oKDsYHykbL58wcvD+o51wK0kHBOui46xZ7z78+5L//3Xq+ttI/ASQRsAGgcKYtHHi9c0NloA0b5v7gVd7zhEuD01TnVf7+3EVDqycqEbABIE8anHH9WshctZdf9Z6PnQhOE7YvEmaMByJgA0DBLJwTvG/qwuB9UYT1vhdd0ljeRUfABoCc6tvlv/3R9c2tR8nD6/y3v/1Mc+uRVQRsAMiLU5Wzus4a4Z1DPmvEwLYol2Jteri+4h/aUTtNefmjRnrvRw6vSnTqaH0VyDkCNgDkxd73+m7u2yWdes57HeUyruu/Mnjb6TOV73t6B6e5clXtvEvl926X3toZkGjvpNoZFRABGwAKoG1YY8cPv7jyfef8xvIb+57Gji8iAjYAFEyUXvaS1ZXvnQtP/9mvxlMugiUSsM1smJn9s5k9kkT+AIBk3bdtaOk3bk2mHhiQVA/7i5J+kVDeAAAfK9dGT9vs3u5QyhvK5yiS2AO2mU2VdLmku+POGwAQbG3MK3t+/rZo6eK+61fcnyMvkuhhf0PSlyT9j6AEZrbMzLrNrPvoUabvA0AaFq0I3//tB7znHXv892992nsOuq92SfXs8Wsvr103DBZrwDazRZKOOOd2h6Vzzn3HOdflnOvq7OSWagDQDNPfV/n+0aDLqqrMW+a//dMRe8LV12ff43PZGGqLu4c9R9IVZvaKpC2SLjWzv4+5DABAHX7ic6JywfLwYzpClhqVpPEfD9+/Yk34fkQXa8B2zt3snJvqnHu/pCWSfuyc+0ycZQAAAswMP8U4xWc9ksdqLAt6vMbNPHpPhu9fvzl8v6/ze+o4KP+4DhsA8qJtYl2HJTVj/Kqb6jywfUKs9ciLtqQyds5tl7Q9qfwBAK3t+9vTrkG+0MMGgAKZ3JFu+bPPS7f8LCNgA0CezApfQ/TQEFcwK/eRD0jzL5J+f2r9eTy7qUaCGvUvssSGxAEArcl1B5+3XjinsftlX3aDtO3Z4HJRPwI2AOTN1DulA+Ezvnq3S+Pmea8Pb5MmVQ2VX3eLdM8Q7gYxZ6a0c4P0+F0D2/YflGZc4b2O1LOf9s3oBRYQQ+IAkDeTa9+YunR7S9ftBest27xed+kxlGAtSbterDx+8+PeQi2lXnWkc+eTvjC0QgvGXK17piWsq6vLdXfnd5zEzNKuQqLS/vfTDLRhthW2/U4dlfb6XHhdJeolXYvnStcvlubNko6flH66V7p1o/TzfRHqGOW/+PN7Ai/nynsbStrtnKvZEgyJA0Aetde/7PPWtV6ADjJ+jDRjinT1gsrtO1+QLvlcnYVy7XVNBGwAyKtZTtod3jstTUBrb5PeqZosNpQFVVy39LELBnrT7bOl02ci9q6ZGR4JARsA8ixC0JYGgnW9q56VH3fmeenUcxHzIlhHxqQzAMi76bUX9C5NFvNzyzLp+FNeb7n06Nvlbfcz7KKIwXr69yIkQgmTzhKW98kSaf/7aQbaMNtov34BvezqwHrlPOnBO+uvz9LV3ozzcoHD4hF713lvQzHpDADwrllO2jNKcm8P2tXzpDRhbOW20XOlN/uiZ98xRnrjx9LmW72HJH19k3TzXT6Jp2+WOpZEzxySCNgAUBwX9kfgqt522zBp+hXSKwfrz/rYicre+q8eGdzTlsQ56wZwDhsAiqYsaLpu6aEdjQVrP+cu8q7brhgOJ1g3hB42ABTRLCedOibtnaBrL5euvTzBss4/0tB14fDQwwaAomrv8AL3tHXJ5D9tvZc/wToW9LABoOgmrfAeUqRrtmti6DsR9LABAANmuYHHzOODdq/y64yf/3rlcUgEPWwAgL+2cYMC8Jq/T6kuoIcNAEAWELABAMgAAjYAABmQ+lriZpbrGQppf79JK8Aav7RhxtF+2VeANoy0ljg9bAAAMiA3s8Qj3SS9hnrvAwsAQNIy3cO+6ZqBe7PGoZTXyqvjyQ8AgLhk8hx26TZuSZv8J9KRY43lkfb3mzTOn2Vf3tuQ9su+ArRhPu+HHVdvOorD/beGY6gcAJC2TA2JNzNYt0K5AACUZCJg//aZ9IOm65b+/JPp1gEAUFwtH7BdtzRieOP53HB743lsuS39Hw4AgGJq6Ulnb++SRo5oMH+f88+NBt3fvSON/ONoadP+fpPGhJfsy3sb0n7ZV4A2zP7CKVGCded86d4f+u8LmizW6CSyOHr8AAAMRcv2sGv1gqP0nMMCc620H54h/ez+oddhUDn5/2WYdhUSRxtmG+2XfQVow+z2sGsF62/d57+93p6z33Ev7at9HOezAQDN0nIBu7OjdprldyRfDynaD4AJY5OvBwAALRewj2yLL6+gHnCcPeOeJ+PLCwCAIC210tlfXDPwOuwcteuOPvztuqWTfdKYudKJp6XRo6LXZ+OXo9VnxVLpG5uj5wsAwFC1VA/79i96z0HB+MCRgddzZg7eH9RzLgXpoGAddNx1i73nXx/y31+q57pV/vsBAIhLSwXsWqYtHHi9c0NloA0b5v7gVd7zhEuD01TnVf7+3EVDqycAAHFrmYDd6Hnl144E73v5Ve/52IngNGH7omDGOAAgSS0TsKNYOCd439SFwfuiCOt9L7qksbwBAGhUSwbsvl3+2x9d39x6lDy8zn/72880tx4AgOJqiYA9eULl+7NGeEPMZ5UtTRplyHnTw/WV/9CO2mnKyx810ns/smqJ0onj6isfAIBaWmJp0rBgfPqM1D7be+2XrnpGeXWa8uMl6egTgwNrrTzK0/Rul8a+J7i+g/LK/5J6aVchcbRhttF+2VeANszu0qTl2oY1dvzwiyvfd85vLL+wYA0AQFJaPmCXi7JYypLVle9r/TD77FfjKRcAgCQlErDN7BUz+xcze8HMmnrB031DXNp049Zk6gEAQJyS7GF/3Dl3QZRx+ZVro2fa7N7uUMobyucAAGAoWmJIfO3KePP7/G3R0sV916+4PwcAACVJBWwnaZuZ7TazZdU7zWyZmXXXO1y+aEX4/m8/4D3v2OO/f+vT3nPQfbVLrqxaI/zay2vXDQCAJCRyWZeZvc85d9DMJkn6kaQvOOeeDkgbelmXJM24Qtp/sHJb6ZigIetad/QK2x+Ud5RrwbmsK39ow2yj/bKvAG2Y3mVdzrmD/c9HJD0o6aJG8vvJ3YO3LVgefkxHyFKjkjT+4+H7V6wJ3w8AQDPFHrDN7GwzG116LelPJP0s7JiJnwjPc8qkwdseq7Es6PEaN/PoPRm+f30d97cOW48cAIBGtCWQ52RJD/YP07RJ+q5z7rGwA974TX0FJTVj/Kqb6juu0Tt+AQAQJPaA7ZzbJ2lm3Pk20/e3p10DAAAqtcRlXVFM7ki3/NnnpVs+AKDYWuLmH6XXtWZh1zsE/pEPeAF//0Hplwfqy6PeuqX9/SaNGarZl/c2pP2yrwBtGGmWeBLnsBMTdinWwjmN3S/7shukbc8GlwsAQJpaKmCvWietuTE8Te92adw87/XhbdKkqqHy626R7nkkeplzZko7N0iP3zWwbf9B79pvSToUYW3yL8S8YhoAANVaakhcir44SSndlm3S0tXh6Yfiu1+Tll42uJxa9QmS9vebNIbjsi/vbUj7ZV8B2jDSkHjLBeyJ46SjT0Q4LuL57MVzpesXS/NmScdPSj/dK926Ufr5vtrHRgnWEy4Nv5wr7e83afxnkX15b0PaL/sK0IbZPIfd01v/sVvXegE6yPgx0owp0tULKrfvfEG65HP1lcm11wCAZmi5HnZJ1KHo9jbpnWcHb4+qupz22dLpM40Phb+bf/5/GaZdhcTRhtlG+2VfAdowmz3skqjnj0vBut5LvsqPO/O8dOq5aHk1+77cAIBia+mFU5bcXDuNdQUHz1uWScef8gJ/6dG3y9vuZ9hF0QLxn36pdhoAAOLUskPiJUG97OrAeuU86cE766/H0tXejPN6yg6T9vebNIbjsi/vbUj7ZV8B2jCbs8T9vLVTGjWy6rguqedJacLYyu2j50pv9kUvv2OM9MaPK7d9fZN0812DA/aSm6X7fhQ9b6kQ/9DSrkLiaMNso/2yrwBtmO1z2OXO/pj3XB1A24ZJ06+QXjlYf97HTlT2mH/1yOCetsQ5awBAulr6HHa18qDpuqWHdjQWrP2cu8i7brv8xwHBGgCQtkwMiVcbP1o69lQStanUOb+x68KlQgzlpF2FxNGG2Ub7ZV8B2jDSkHimetglx096vd4Va5LJf/kd/efIGwzWAADEJZM9bD9x3FEriaHvtL/fpPHrPvvy3oa0X/YVoA3z28P2U7oe27oG7uZVbtW6wdvOuazyOAAAWlVuetitKu3vN2n8us++vLch7Zd9BWjDYvWwAQDIMwI2AAAZQMAGACADUl/pbNasWerujmGKd4vK+/mlvJ9bkmjDrKP9si/vbRgVPWwAADKAgA0AQAakPiQOvGt3DMNes/I/PAigmOhhI12H7/ACdRzBWhrI63BC69YCQEoI2EjHqTe8wHrgS8nkf+AmL/9Th5PJHwCajCFxNF9cveko9p7jPTNUDiDj6GGjuZoZrFuhXACICQEbzbFnRPpBc7dJx7akWwcAqBMBG8nbbZJ7p+Fsbrg9hrrsX5r+DwcAqAPnsJGsPSMbzqL81qd/c7/33PD9z/eMkC78XYOZAEDz0MNGslztoNg5X7r3h/77gu5T3vD9y2Po8QNAMxGwkZwaQ8/W5T16eqXP/HXjQbiUX+lx3p81Vj8AaCUEbCSjRjD81n3+2+sN2n7HvbQvwoEEbQAZQcBG/E4fqZlk+R1NqIci/gA43ZN4PQCgUQRsxO/FybFlFTS5rOFJZ+Ve7IwxMwBIBrPEEa/XB6698uvdlgKt644+/O26pZN90pi50omnpdGjoldn45cHXofVR4fWSefcGD1jAGgyetiI18G/lBQcjA+UjZbPmTl4f1DPuRSkg4J10HHXLfaef33If/+79XxtpX8CAGgRBGw01bSFA693bqgMtGHD3B+8ynuecGlwmuq8yt+fu2ho9QSAVkPARnwanHH9WshctZdf9Z6PnQhOE7YvEmaMA2hhBGw01cI5wfumLgzeF0VY73vRJY3lDQBpI2AjEX27/Lc/ur659Sh5eJ3/9refaW49AKBeBGzE41TlrK6zRnjnkM8aMbAtyqVYmx6ur/iHdtROU17+qJHe+5HDqxKdOlpfBQAgYQRsxGPve3039+2STj3nvY5yGdf1Xxm87fSZyvc9vYPTXLmqdt6l8nu3S2/tDEi0d1LtjAAgBQRsJK5tWGPHD7+48n3n/MbyG/uexo4HgDQkErDNbJyZ/YOZ/auZ/cLM/iiJcpA9UXrZS1ZXvncuPP1nvxpPuQDQypLqYa+X9Jhz7t9JminpFwmVgxy6b9vQ0m/cmkw9AKCVxB6wzWyMpLmSNkiSc+4d55zPWUfkycq10dM2u7c7lPKG8jkAoJmS6GHPkHRU0kYz+2czu9vMzk6gHLSQtTGv7Pn526Kli/uuX3F/DgCISxIBu03ShZL+1jn3UUlvSfqr8gRmtszMus2s++hRLqMpokUrwvd/+wHvecce//1bn/aeg+6rXVI9e/zay2vXDQBaURIB+4CkA865/ot59A/yAvi7nHPfcc51Oee6Oju5tWERTH9f5ftHgy6rqjJvmf/2T0fsCVdfn32Pz2VjAJAFsQds59whSa+a2Yf6N31C0s/jLgfZ8pO7B29bsDz8mI6QpUYlafzHw/evWBO+HwCyJKn7YX9B0r1mNlzSPknXJ1QOWsXMo9KLwaMlU3zWI3msxrKgx2vczKP3ZPj+9ZvD9/s6v6eOgwAgeYkEbOfcC5K48rVI2ibWdVhSM8avuqnOA9snxFoPAIgLK50hl76/Pe0aAEC8CNhomskd6ZY/+7x0yweARhCwEZ9Z4WuIHhriCmblPvIBaf5F0u9PrT+PZzfVSFCj/gCQpqQmnQG+XHfweeuFcxq7X/ZlN0jbng0uFwCyjICNeE29UzoQPuOrd7s0bp73+vA2aVLVUPl1t0j3PBK9yDkzpZ0bpMfvGti2/6A04wrvdaSe/bRvRi8QAFLAkDjiNbn2jalLt7d03V6w3rLN63WXHkMJ1pK068XK4zc/7i3UUupVRzp3PukLQysUAJrMXK17Fyasq6vLdXfnd7zSzNKuQqJ8//2cOirt9bnwukrUS7oWz5WuXyzNmyUdPyn9dK9060bp5/si1C/KP63ze0Iv5ypkG+YI7Zd9eW9DSbudczX/R2RIHPFrr3+52a1rvQAdZPwYacYU6eoFldt3viBd8rk6C+XaawAZQMBGMmY5aXf4r+LSBLT2NumdqsliQ1lQxXVLH7tgoDfdPls6fSZi75qZ4QAygoCN5EQI2tJAsK531bPy4848L516LmJeBGsAGcKkMyRreu0FvUuTxfzcskw6/pTXWy49+nZ52/0MuyhisJ7+vQiJAKB1MOksYXmfLBHp309AL7s6sF45T3rwzvrrsnS1N+O8XOCw+BB617RhttF+2Zf3NhSTztAyZjlpzyjJvT1oV8+T0oSxldtGz5Xe7IuefccY6Y0fS5tv9R6S9PVN0s13+SSevlnqWBI9cwBoEQRsNMeF/RG4qrfdNkyafoX0ysH6sz52orK3/qtHBve0JXHOGkCmcQ4bzVUWNF239NCOxoK1n3MXeddtVwyHE6wBZBw9bDTfLCedOibtnaBrL5euvTzBss4/0tB14QDQKuhhIx3tHV7gnrYumfynrffyJ1gDyAl62EjXpBXeQ4p0zXZNDH0DyCl62Ggds9zAY+bxQbtX+XXGz3+98jgAyCl62GhNbeMGBeA1f59SXQCgBdDDBgAgAwjYAABkAAEbAIAMSH0tcTPL9UyhtL/fpBVgjV/aMONov+wrQBtGWkucHjYAABnALHEAiIq1ApAietgAEObwHV6gjiNYSwN5HV4TT34oDM5hJyzt7zdpnD/Lvry3Yd3td+oNae/EeCvj5/xDUvvkug/Pe/tJhfgb5H7YAFCXuHrTUew9x3tmqBw1MCQOAOWaGaxboVxkBgEbACRpz4j0g+Zuk45tSbcOaFkEbADYbZJ7p+Fsbrg9hrrsX5r+Dwe0JCadJSzt7zdpTHjJvry3Yc322zNScr9rqAzzmS7kuhvKUrLh0oW165X39pMK8TfIwikAUFOEYN05X7r3h/77/IJ12PbIYujxI1/oYScs7e83afy6z768t2Fo+9UYeo7Scw4LzLXSfniG9LP7Q6tQc/Z43ttPKsTfID1sAAhUI1h/6z7/7fX2nP2Oe2lfhAM5n41+BGwAxXP6SM0ky+9oQj0U8QfA6Z7E64HWR8AGUDwv1r+yWLWgyWUNTzor92JnjJkhq1jpDECxvD5w7VXYOWrXHX3423VLJ/ukMXOlE09Lo0dFr87GLw+8Dj1nfmiddM6N0TNG7tDDBlAsB/9SUnAwPlA2Wj5n5uD9QT3nUpAOCtZBx1232Hv+9SH//e/W87WV/glQGARsACgzbeHA650bKgNt2DD3B6/ynidcGpymOq/y9+cuGlo9UTwEbADF0eCM69dC5qq9/Kr3fOxEcJqwfZEwY7zQCNgAUGbhnOB9UxcG74sirPe96JLG8kb+EbABFFLfLv/tj65vbj1KHl7nv/3tZ5pbD7QuAjaAYjhVOavrrBHeOeSzRgxsi3Ip1qaH6yv+oR2105SXP2qk937k8KpEp47WVwFkHkuTJizt7zdpLIuYfXlvw3fbL+T87+kzUvvs/vQ+Qbt6Rnl1mvLjJenoE9LEcUPLozxN73Zp7HsCq1uxXGne208qxN8gS5MCQBRtwxo7fvjFle875zeWX2iwRmERsAGgTJTFUpasrnxfqwP42a/GUy6KLfaAbWYfMrMXyh4nzGxF3OUAQFru2za09Bu3JlMPFEvsAds592/OuQuccxdImiWpT9KDcZcDAEOxcm30tM3u7Q6lvKF8DuRL0kPin5D0S+fcrxIuBwBCrY15Zc/P3xYtXdx3/Yr7cyA7kg7YSyRtrt5oZsvMrNvM4ryfDQDEZlGNE3nffsB73rHHf//Wp73noPtql1y5qvL9tZfXrhuKKbHLusxsuKSDkj7snDscki7X8/ULcDlC2lVIHG2YbVEu65KkGVdI+w9WHdvfpQgasq51R6+w/UF5R7otJ5d15UorXNa1QNKesGANAK3iJ3cP3rZgefgxHSFLjUrS+I+H71+xJnw/UC7JgL1UPsPhAJCKmeErhE2ZNHjbYzWWBT1e42YevSfD96+v53/I83vqOAh5kEjANrNRkj4p6R+TyB8AhqxtYl2HJTVj/Kqb6jywfUKs9UB2tCWRqXOuTxL/qgAgwPe3p10DZA0rnQFAv8kd6ZY/+7x0y0dr4+YfCUv7+00aM1SzL+9tOKj9aswWr3cI/CMf8AL+/oPSLw/Ul0fNGeKzBv9bzHv7SYX4G4w0SzyRIXEAyKqwS7EWzmnsftmX3SBteza4XCAMARtAsUy9UzoQPuOrd7s0bp73+vA2aVLVUPl1t0j3PBK9yDkzpZ0bpMfvGti2/6B37bckHYqyNvm0b0YvELnEkHjC0v5+k8ZwXPblvQ1926/GsLjk9bJLvd4t26Slq8PTD8V3vyYtvWxwOaF8hsOl/LefVIi/wUhD4gTshKX9/SaN/yyyL+9t6Nt+p45Ke30uvK4S9Xz24rnS9YulebOk4yeln+6Vbt0o/XxfhPpFCdbn9wRezpX39pMK8TfIOWwA8NXeWfehW9d6ATrI+DHSjCnS1Qsqt+98Qbrkc3UWyrXXED3sxKX9/SaNX/fZl/c2DG2/iEPj7W3SO88O3h65DlW96PbZ0ukzjQ2Fv1uPnLefVIi/QXrYABBqlosUtEvBut5LvsqPO/O8dOq5iHnVCNYoFhZOAVBs02sv6G1dwQH2lmXS8ae83nLp0bfL2+5n2EURg/X070VIhCJhSDxhaX+/SWM4Lvvy3oaR2i+gl10dWK+cJz14Z/11Wbram3FeLnBYPGLvOu/tJxXib5BZ4q0g7e83afxnkX15b8PI7bdnlOTerthkXVLPk9KEsZVJR8+V3uyLXoeOMdIbP67c9vVN0s13+QTs6ZuljiWR8857+0mF+BvkHDYARHZhfwSu6m23DZOmXyG9crD+rI+dqOyt/+qRwT1tSZyzRijOYQNAubKg6bqlh3Y0Fqz9nLvIu267ondNsEYNDIknLO3vN2kMx2Vf3tuw7vY7dUza24Trn88/0tB14XlvP6kQf4ORhsTpYQOAn/YOr9c7bV0y+U9b7+XfQLBGsdDDTlja32/S+HWffXlvw1jbL8I12zXFPPSd9/aTCvE3SA8bAGI1yw08Zh4ftHuVX2f8/NcrjwPqRA87YWl/v0nj13325b0Nab/sK0Ab0sMGACAvCNgAAGQAARsAgAxohZXOeiT9qonlTewvsylSOr/U1M+Ygry3Ie0XI9ovdk3/fAVow3OjJEp90lmzmVl3lJP7WZb3z8jnyzY+X7bl/fNJrfsZGRIHACADCNgAAGRAEQP2d9KuQBPk/TPy+bKNz5dtef98Uot+xsKdwwYAIIuK2MMGACBzCNgAAGRAoQK2mX3KzP7NzF42s79Kuz5xMrO/M7MjZvaztOuSBDObZmZPmdkvzOwlM/ti2nWKm5mNNLPnzezF/s/4lbTrFDczG2Zm/2xmj6RdlySY2Stm9i9m9oKZdaddn7iZ2Tgz+wcz+9f+v8U/SrtOcTGzD/W3W+lxwsxWpF2vcoU5h21mwyT9f5I+KemApH+StNQ59/NUKxYTM5sr6U1J/805d17a9Ymbmb1X0nudc3vMbLSk3ZKuzEv7SZJ5q0Oc7Zx708zaJe2U9EXn3LMpVy02ZrZSUpekMc65RWnXJ25m9oqkLudcLhdOMbN7JP3EOXe3mQ2XNMo515t2veLWHy9ekzTbOdfMhb1CFamHfZGkl51z+5xz70jaIunTKdcpNs65pyUdS7seSXHOve6c29P/+qSkX0iakm6t4uU8b/a/be9/5OYXtZlNlXS5pLvTrguGzszGSD53FtgAAAJLSURBVJoraYMkOefeyWOw7vcJSb9spWAtFStgT5H0atn7A8rZf/hFYWbvl/RRSc+lW5P49Q8ZvyDpiKQfOefy9Bm/IelLkv5H2hVJkJO0zcx2m9mytCsTsxmSjkra2H9a424zOzvtSiVkiaTNaVeiWpECtt9itLnpvRSFmb1H0gOSVjjnTqRdn7g558445y6QNFXSRWaWi9MbZrZI0hHn3O6065KwOc65CyUtkPSf+k9V5UWbpAsl/a1z7qOS3pKUq7lAktQ/1H+FpO+lXZdqRQrYByRNK3s/VdLBlOqCOvSf131A0r3OuX9Muz5J6h9q3C7pUylXJS5zJF3Rf453i6RLzezv061S/JxzB/ufj0h6UN6puLw4IOlA2ajPP8gL4HmzQNIe59zhtCtSrUgB+58kfdDMpvf/gloiaWvKdUJE/ROyNkj6hXNubdr1SYKZdZrZuP7XZ0maL+lf061VPJxzNzvnpjrn3i/vb+/HzrnPpFytWJnZ2f0TItU/VPwnknJz1YZz7pCkV83sQ/2bPiEpN5M+yyxVCw6HS61xe82mcM6dNrMbJD0uaZikv3POvZRytWJjZpslzZM00cwOSPqyc25DurWK1RxJ10j6l/5zvJK02jn3gxTrFLf3Srqnf4bq70m63zmXy8ufcmqypAf7bwXZJum7zrnH0q1S7L4g6d7+Ts8+SdenXJ9YmdkoeVcS/ce06+KnMJd1AQCQZUUaEgcAILMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAGELABAMiA/x8yMOc/us4UiAAAAABJRU5ErkJggg==\n",
    +      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHwCAYAAADjD7WGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df7QU5Z3v+8932Aii/BDYoALXYJKVs+YYMdKjzqBcYkgICEbvmZuBa8zR3FzOzT2GEHEyI2tlxWSdxBwViBNzJydHBzwnKppxjKgTJTGCASNOwygzmpm7HDURCT+2sAO6TQTmuX/U7tndvetXd1V1d1W/X2vt1d1VTz31bZ+9/fI89TxV5pwTAADI3u+1OwAAALoFSRcAgBYh6QIA0CIkXQAAWoSkCwBAi5B0AQBoEZIuAAAtQtIFAKBFSLpAC5jZe8zsb83ssJntM7M7zKwnpPwEM/vLwbIDZvYPZvYfWxkzgPSRdIHW+H8lHZB0hqTzJP2vkv4fv4JmdpKkn0g6S9IfShov6U8l3WJmK1oSLYBMkHSB1pgp6QHn3G+dc/skPS7p3weUvVrS/yLpf3fOveqcO+ace1zSCkn/xcxOlSQzc2b2vspBZrbBzP5L1efFZva8mfWb2TNmdm7VvjPN7EEzO2hmr1YnczO7ycweMLP/YWZHzexFMytV7f8zM3tjcN8/m9lH0vlPBBQfSRdojW9JWmpmY8xsmqSF8hKvn49K+pFz7u267Q9KGiOv9xvKzD4k6a8k/SdJkyT9N0mbzGyUmf2epEckvSBpmqSPSFppZguqqrhc0kZJEyRtknTHYL0fkHSdpD9wzo2VtEDSa1HxAPCQdIHWeFpez/aIpD2SypJ+GFB2sqRf1290zh2X1CepN8b5lkv6b865Hc65E865uyX9TtJFkv5AUq9z7mvOuXedc69I+u+SllYdv80597fOuROS/qekWYPbT0gaJen3zWykc+4159y/xIgHgEi6QOYGe5aPS/obSafIS6qnSfqvAYf0ybv2W19Pz+CxfTFOe5akVYNDy/1m1i9phqQzB/edWbdvtaSpVcfvq3o/IGm0mfU4516WtFLSTZIOmNlGMzszRjwARNIFWmGivGu0dzjnfuece1PSekmLAsr/RNJCMzulbvt/kPSupB2DnwfkDTdXnF71/nVJX3fOTaj6GeOcu29w36t1+8Y654LiqeGcu9c5d7G85O0U/I8HAHVIukDGnHN9kl6V9Dkz6zGzCZL+o6TdAYf8T3lD0D8YXGo0cvB6619IutU595vBcs9L+j/MbISZfVzejOiK/y7p/zazC81zipldZmZjJT0n6ejghKiTB48/x8z+IOq7mNkHzOxSMxsl6beS3pH0rw3/RwG6FEkXaI3/TdLHJR2U9LKkY5K+6FfQOfc7SfPl9Uh3yEtsj8ubjPXVqqJfkLREUr+kq1R1jdg5V5b0f8mbAHV48JzXDO47IWmxvKVLr8obrr5T3tKkKKMkfXPwmH2Spki6McZxACSZc67dMQAIYWYjJf1I0huSrnH80QK5RU8X6HDOuWPyruf+i6QPtDkcAAnQ0wUAoEXo6QIA0CKBN1xPYvLkye4973lPFlV3hJ07d7Y7hEzNnj273SFkjjbMN9ov/4rehs4589ueyfByqVRy5XI59Xo7hZnvf8vCSO13YmcK/51mZ3P5gzbMN9ov/7qgDX2/IMPLSNf+W71km0bClYbq2r8mnfoAoI1IukjHsTe95LjnS9nUv+cGr/5j+7OpHwBaIJNruugyafVq49g9eKfDjIadASBL9HSRTCsTbiecFwASIOmiObtGtT/x7TTp0Mb2xgAADSDponE7TXLvJq7multSiOXVZe1P/gAQE9d00ZhdoxNXYaWh9995wHt1SVeY7Rolnf+7hJUAQLbo6aIxLjqx9c6X7vmR/77qhBtne2wp9LwBIGskXcQXMYxrJe+nr1/61JeTJ9JKfZWfcz6ZLD4AaDeSLuKJSGjfvt9/e7OJ1++4F1+JcSCJF0AHI+ki2vEDkUVW3NqCOBQziR/vyzwOAGgGSRfRXpiaWlVBE6YST6Sq9kJvipUBQHqYvYxwvx5a1+PXy6wkS1eOP5TsytLRAWncXOnI09LYMfHDWf+Vofdh8WjfOun0L8avGABagJ4uwu39M0nBCXVP1cjznFnD9wf1YCuJNijhBh13zRLv9Vf7/Pf/W5xvXO9fAADaiKSLRGYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixMAOgFJF8ESzgR+I2T+1cuve6+HjgSXCdsXCzOZAXQYki4SWTQneN/0RcH74gjrBS++JFndANAOJF3EMrDdf/tjt7c2jopH1vlvf+eZ1sYBAI0g6cLfsdqZSieP8q6pnjxqaFucZT4bHmnu9A9vjS5Tff4xo73Po0+qK3TsYHMBAEAGSLrwt/sM380D26VjO7z3cZYIXfvV4duOn6j93Nc/vMwVq6Lrrpy/f4v09raAQrunRFcEAC1C0kXDekYkO/6ki2o/985PVt/4U5MdDwCtQtJFInF6u0tX1352Lrz8Z76WznkBoNOQdJG5+zc3Vn79pmziAIB2i5V0zezjZvbPZvaymf151kGh/a5fG79sq3udjZyvke8BAFmLTLpmNkLSdyQtlPT7kpaZ2e9nHRjaa23Kd1H83M3xyqX9tKK0vwcAJBGnp3uBpJedc684596VtFHSJ7INC3mzeGX4/u8+6L1u3eW/f9PT3mvQc3kr6mc1f/qy6NgAoFPESbrTJL1e9XnP4LYaZrbczMpmVj54kLWRRTfzzNrPjwUt2akzb7n/9k/E7JHWr9+922dJEgB0qtQmUjnnvuecKznnSr29PM+06H525/BtC1eEHzMx5LaOknTah8P3r1wTvh8AOl2cpPuGpBlVn6cPbkORzQofrZjmc8+JxyNuwXg44gEG/UfD999+X/h+X+f2NXEQAGQjTtL9O0nvN7OZZnaSpKWSWNRRdD2Tmzosq5nMV97Q5IEjJ6UaBwAk0RNVwDl33Myuk/SEpBGS/so592LmkQFVfril3REAQHKRSVeSnHN/K+lvM44FOTN1orT/UPvOf+E57Ts3ADSDO1Ih2Ozw+zXua/BOU9U++D5p/gXSe6c3X8ezGyIKRMQPAK0Wq6cLBHHl4Ou4i+Yke97uguukzc8GnxcA8oaki3DTb5P2hM9i6t8iTZjnvd+/WZoysXb/NTdJdz8a/5RzZknb7pKeuGNo26t7pbMv997H6mHP+Iv4JwSAFjEX9ciXJpRKJVcuF7crYmbtDiFTw34ndkZ/XysN9T43bpaWrQ4v34h7vy4tWzD8PKEihpa7rg0LhvbLvy5oQ98vSNJtQhf8stRuOHYw1sPg4y4XWjJXunaJNG+2dPio9PPd0jfWSy+9EiO2OL9W5/ZFLhXqujYsGNov/7qgDX2/IMPLiDay+TuMbVrrJdkgp42Tzp4mXbWwdvu256VLPtvkSVmbC6BDkXQRz2wXOcxcmVQ1skd6t24CVCM3zXBl6eLzhnq1Iy+Ujp9IZ1gZANqJpIv4YiReaSjhNnt3qurjTjwnHdsRsy4SLoAOxzpdNGZm9A2QrRScJG9aLh1+yuu1Vn4Gtnvb/Yy4IGbCnfmDGIUAoL2YSNWELpgAEF4goLdbnxyvmCc9dFvzcSxb7c2Erokt6NeqwV5u17dhztF++dcFbcjs5bR0wS9LdKFdYyT3Ts0mK0l9T0qTxtcWHTtXemsg/vknjpPe/Gnttm9ukG68wyfpzrxPmrg0fuWVWGnDXKP98q8L2pDZy0jR+YNZtK7X2zNCmnm59Nre5qs+dKS21/zLR4f3eCVxDRdA7nBNF8lUJT5Xlh7emizh+jlrsbeut6aXS8IFkEMMLzehC4ZFGj/o2CFpdwvWx557ING64QraMN9ov/zrgjb0/YL0dJGOkRO93ueMddnUP+N2r/4UEi4AtAvXdJGuKSu9HynWmt5IDCMDKBB6usjObDf0M+vwsN2r/DrF5/669jgAKBB6umiNngnDkuia77cpFgBoE3q6AAC0CEkXAIAWIekCANAiJF0AAFokk4lUO3fuLPTC56IvXC9y21XQhvlG++VfkduwVAp+NBqzlwGg4sRh6fmJNZtWrZPWfLGu3Ll7pZFntC4uFAZJF0B3i7iJy7CEK0m7z6z9zJpyxMQ1XQDdZ/+tXrJN465p0lBd+9ekUx8Ki6QLoHsce9NLjnu+lE39e27w6j+2P5v6kXsMLwPoDmn1auPYfbr3yrAz6tDTBVB8rUy4nXBedCySLoDi2jWq/Ylvp0mHNrY3BnQMki6AYtppkns3cTXX3ZJCLK8ua3/yR0fgmi6A4tk1OnEVVnV/g+884L26csJKd42Szv9dwkqQZ/R0ARSPi05svfOle37kv88CbigUtD22FHreyDeSLoBiiRjGtZL309cvferLyRNppb7KzzmfTBYfio2kC6A4IhLat+/3395s4vU77sVXYhxI4u1aJF0AxXD8QGSRFbe2IA7FTOLH+zKPA52HpAugGF6YmlpVQROmEk+kqvZCb4qVIS+YvQwg/349tK7Hr5dZSZauHH8o2ZWlowPSuLnSkaelsWPih7P+K0Pvw+LRvnXS6X5PVEBR0dMFkH97/0xScELdUzXyPGfW8P1BPdhKog1KuEHHXbPEe/3VPv/9/xbnG9f7F0BhkXQBFN6MRUPvt91VmyzDhozff6X3OunS4DL1dVV/PmtxY3Gi+Ei6APIt4UzgN0LmX738uvd66EhwmbB9sTCTuauQdAEU3qI5wfumLwreF0dYL3jxJcnqRvGQdAEUxsB2/+2P3d7aOCoeWee//Z1nWhsHOgdJF0B+HaudqXTyKO+a6smjhrbFWeaz4ZHmTv/w1ugy1ecfM9r7PPqkukLHDjYXAHKHpAsgv3af4bt5YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5YlV03ZXz92+R3t4WUGj3lOiKUAgkXQCF1DMi2fEnXVT7uXd+svrGn5rseBQDSRdA4cXp7S5dXfvZufDyn/laOudFd4lMumb2V2Z2wMz+sRUBAUA73L+5sfLrN2UTB4otTk93g6SPZxwHADTs+rXxy7a619nI+Rr5Hsi3yKTrnHta0qEWxAIADVmb8l0UP3dzvHJpP60o7e+BzsU1XQBdY/HK8P3ffdB73brLf/+mp73XoOfyVtTPav70ZdGxoTuklnTNbLmZlc0szYdfAUDTZp5Z+/mxoCU7deYt99/+iZg90vr1u3f7LElCd0ot6TrnvuecKznnmK8HoCP87M7h2xauCD9mYshtHSXptA+H71+5Jnw/uhvDywDya1b4nZym+dxz4vGIWzAejniAQf/R8P233xe+39e5fU0chDyKs2ToPkk/l/QBM9tjZv9n9mEBQAw9k5s6LKuZzFfe0OSBIyelGgc6V09UAefcslYEAgB598Mt7Y4AnY7hZQCFNnVie89/4TntPT86C0kXQL7NDr9f474G7zRV7YPvk+ZfIL13evN1PLshokBE/CiWyOFlAMg7Vw6+jrtoTrLn7S64Ttr8bPB5gWokXQD5N/02aU/4LKb+LdKEed77/ZulKXXDztfcJN39aPxTzpklbbtLeuKOoW2v7pXOvtx7H6uHPeMv4p8QhWAu6lEazVRqVujxkiz+m3USM2t3CJmjDfPNt/12Rn9nKw31PjdulpatDi/fiHu/Li1bMPw8oQKGloveflKx/wZLpZLK5bJvI5J0m1DkXxaJP/giKHob+rbfsYOxHgYfd7nQkrnStUukebOlw0eln++WvrFeeumVGPHFSbjn9gUuFSp6+0nF/hsMS7oMLwMohpG9TR+6aa2XZIOcNk46e5p01cLa7duely75bJMnZW1uVyLpAiiO2S5ymLkyqWpkj/Ru3QSoRm6a4crSxecN9WpHXigdP5FsWBnFR9IFUCwxEq80lHCbvTtV9XEnnpOO7YhZFwm3q7FOF0DxzIy+AbKVgpPkTculw095vdbKz8B2b7ufERfETLgzfxCjEIqMiVRNKPIEAIlJHEVQ9DaM1X4Bvd365HjFPOmh25qPZdlqbyZ0tcAh5pi93KK3n1Tsv0FmL6esyL8sEn/wRVD0NozdfrvGSO6dmk1WkvqelCaNry06dq701kD8GCaOk978ae22b26QbrzDJ+nOvE+auDR23UVvP6nYf4PMXgbQnc4fzKJ1vd6eEdLMy6XX9jZf9aEjtb3mXz46vMcriWu4qME1XQDFV5X4XFl6eGuyhOvnrMXeut6aXi4JF3UYXm5CkYdFJIa2iqDobdh0+x07JO1uwfrYcw8kWjdc9PaTiv03GDa8TE8XQPcYOdHrfc5Yl039M2736k+QcFFsXNMF0H2mrPR+pFhreiMxjIyY6OkC6G6z3dDPrMPDdq/y6xSf++va44CY6OkCQEXPhGFJdM332xQLComeLgAALULSBQCgRUi6AAC0SCbXdGfPnq1yOc7zrfKp6Gvoirx+roI2zDfaL/+K3oZB6OkCANAizF4GAORW4BOdGtDsM5WbQU8XAJArN1w99JzjNFTquv6qdOoLk8m9l0ulkuOabn5xPSn/it6GtF/+NdOGfo9TzMLUj0kHDiWrwznHo/0AAPmUVq82jv2Dj2jMYtiZ4WUAQEdrZcLN+rwkXQBAR/rtM+1LuBWuLP3JR9Orj6QLAOg4riyNOil5PdfdkryOjTenl/y5pgsA6CjvbE9eR/X12O884L0mTZy/fUYa/UfJ6qCnCwDoKKNHRZfpnS/d8yP/fUEToJJOjEqj503SBQB0jKjeqJW8n75+6VNfTp5IK/VVfs75ZLL4opB0AQAdISqhfft+/+3NJl6/4158Jfq4JImXpAsAaLveidFlVtyafRxSvCQ+aXxzdZN0AQBtd2BzenUF9UTTXH7U92RzxzF7GQDQVn969dB7v15mJVm6cvyhZFeWjg5I4+ZKR56Wxo6JH8/6r8SLZ+Uy6Vv3xa9XoqcLAGizW77gvQYl1D0Hht7PmTV8f1APtpJogxJu0HHXLPFef7XPf38lznWr/PeHIekCADrajEVD77fdVZssw4aM33+l9zrp0uAy9XVVfz5rcWNxxkHSBQC0TdLrrG8cCN738uve66EjwWXC9sXRaPwkXQBAR1s0J3jf9EXB++II6wUvviRZ3X5IugCAjjAQcPvHx25vbRwVj6zz3/7OM83XSdIFALTF1Em1n08e5Q3Xnlx1G8g4w7cbHmnu/A9vjS5Tff4xo73Po+tuBzl5QvxzknQBAG2x7wn/7QPbpWM7vPdxlghd+9Xh246fqP3c1z+8zBUxZh9Xzt+/RXp7m3+Zgz+JrqeCpAsA6Dg9I5Idf9JFtZ975yerb/ypyY6vIOkCADpanN7u0tW1n50LL/+Zr6Vz3kaRdAEAuXd/g7eRXL8pmziiRCZdM5thZk+Z2Utm9qKZfaEVgQEAiu36tfHLZtHrTOt8jXyPOD3d45JWOed+X9JFkv6zmf1+/FMAADDc2uvTre9zN8crl/bTihr5HpFJ1zn3a+fcrsH3RyX9QtK0ZoMDAKAZi1eG7//ug97r1l3++zc97b0GPZe3on5W86cvi44troau6ZrZeyR9SNIOn33LzaxsZuWDBw+mEx0AoGvNPLP282MBS3bqzVvuv/0TMXuk9et37/ZZktSs2EnXzE6V9KCklc65YXerdM59zzlXcs6Vent704sQANCVfnbn8G0LV4QfMzHkto6SdNqHw/evXBO+P6lYSdfMRspLuPc45/4m25AAAN1g8kfC90+bMnzb4xG3YDwc8QCD/qPh+29v8Pm4Uvj9m+vFmb1sku6S9AvnXANztAAACPbmb5o7LquZzFfe0NxxjTypKE5Pd46kqyVdambPD/4kfK4DAACd5Ydbsj9HT1QB59w2SZZ9KAAA1Jo6Udp/qH3nv/CcdOvjjlQAgLaJGire1+Cdpqp98H3S/Auk905vvo5nN4Tvb3SoO7KnCwBAO7lycHJbNCfZ83YXXCdtfjb4vGkj6QIA2mrVOmnNF8PL9G+RJszz3u/fLE2ZWLv/mpukux+Nf845s6Rtd0lP3DG07dW90tmXe+/j9LA/38SdrcxFPYqhCaVSyZXLGfwToUN4E7qLK4vfiU5DG+Yb7Zd/9W0Yp1dppaFyGzdLy1aHl2/EvV+Xli0Yfp6oeII453x/SUm6TeAPPv9ow3yj/fKvvg0nT4j3MPi411CXzJWuXSLNmy0dPir9fLf0jfXSS69EHxsn4U66NHypUFDSZXgZANB2ff3NH7tprZdkg5w2Tjp7mnTVwtrt256XLvlsc+dsZG1uNZIuAKAjxBnWrUyqGtkjvVs3AaqRmcSuLF183tD5Rl4oHT+RfFg5CkkXANAx4l5PrSTcZhNg9XEnnpOO7YhXV9K7YbFOFwDQUZbeGF3GSsEJ8Kbl0uGnvORd+RnY7m33M+KCeMn0j78UXSYKE6mawCSO/KMN8432y7+oNgzq7dYnxyvmSQ/d1nwcy1Z7M6GbOXcYZi+niD/4/KMN8432y784bfj2NmnM6LrjSlLfk9Kk8bXbx86V3hqIf/6J46Q3f1q77ZsbpBvvGJ50l94o3f/j+HVLzF4GAOTMKRd7r/VJsGeENPNy6bW9zdd96Ehtz/WXjw7v8UrpP9GIa7oAgI5WnfhcWXp4a7KE6+esxd663uoEn8UjBBlebgJDW/lHG+Yb7Zd/zbThaWOlQ09lEEyd3vnJ1g1LwcPL9HQBALlw+KjX+1y5Jpv6V9w6eM04YcINQ0+3CfwrO/9ow3yj/fIvrTZM40lAWQwj09MFABROZb2ulYaeQlRt1brh205fUHtcKzF7GQBQCL95yz+Jrr2n9bEEoacLAECLkHQBAGgRki4AAC1C0gUAoEUymUi1c+fOQk/pL/p0/iK3XQVtmG+0X/4VuQ1LpeAp0cxe7hQnDkvPT6zZtGqdtOaLdeXO3SuNPKN1cQEAUkPSbaed4f+aHZZwJWn3mbWfZxf3X4sAUDRc0221/bd6yTYi4cZWqWt/RvdFAwCkhqTbKsfe9JLjni9lU/+eG7z6j+3Ppn4AQGIML7dCWr3aOHaf7r0y7AwAHYeebtZamXA74bwAgEAk3azsGtX+xLfTpEMb2xsDAODfkHSzsNMk927iaq67JYVYXl3W/uQPAJDENd307RqduIrqp2R85wHvNfEzI3eNks7/XcJKAABJ0NNNm4tObL3zpXt+5L8v6NmOiZ/5mELPGwCQDEk3TRHDuJUHJvf1S5/6cvJEWv0QZitJ53wyWXwAgGyRdNMSkdC+fb//9mYTr99xL74S40ASLwC0DUk3DccPRBZZcWsL4lDMJH68L/M4AADDkXTT8MLU1KoKmjCVeCJVtRd6U6wMABAXs5eT+vXQuh6/XmYlWbpy/KFkV5aODkjj5kpHnpbGjokfzvqvDL0Pi0f71kmn+z1RAQCQFXq6Se39M0nBCXVP1cjznFnD9wf1YCuJNijhBh13zRLv9Vf7/Pf/W5xvXO9fAACQGZJuxmYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixMAkD2SbhIJZwK/ETL/6uXXvddDR4LLhO2LhZnMANBSJN2MLZoTvG/6ouB9cYT1ghdfkqxuAED6SLopGdjuv/2x21sbR8Uj6/y3v/NMa+MAAAwh6TbrWO1MpZNHeddUTx41tC3OMp8NjzR3+oe3RpepPv+Y0d7n0SfVFTp2sLkAAAANI+k2a/cZvpsHtkvHdnjv4ywRuvarw7cdP1H7ua9/eJkrVkXXXTl//xbp7W0BhXZPia4IAJAKkm4GekYkO/6ki2o/985PVt/4U5MdDwBIB0k3Y3F6u0tX1352Lrz8Z76WznkBAK0VmXTNbLSZPWdmL5jZi2bmMyCKJO7f3Fj59ZuyiQMAkK04Pd3fSbrUOTdL0nmSPm5mF0UcU3jXr41fttW9zkbO18j3AAAkE5l0neetwY8jB38iBkCLb23Kd1H83M3xyqX9tKK0vwcAIFisa7pmNsLMnpd0QNKPnXM7fMosN7OymaX5PJzCWLwyfP93H/Ret+7y37/pae816Lm8FfWzmj99WXRsAIDWiJV0nXMnnHPnSZou6QIzO8enzPeccyXnHFN4JM08s/bzY0FLdurMW+6//RMxe6T163fv5go8AHSMhmYvO+f6JT0l6ePZhFMcP7tz+LaFK8KPmRhyW0dJOu3D4ftXrgnfDwBorzizl3vNbMLg+5MlfVTSP2UdWMebFX4np2k+95x4POIWjIcjHmDQfzR8/+33he/3dW5fEwcBAJoR5yH2Z0i628xGyEvSDzjnHs02rBzomdzUYVnNZL7yhiYPHDkp1TgAAMEik65zbrekD7UgFiTwwy3tjgAAEIU7UmVo6sT2nv/CYdPdAADtRNJNYnb4cuV9Dd5pqtoH3yfNv0B67/Tm63h2Q0SBiPgBAOmKc00XCbhy8HXcRXOSPW93wXXS5meDzwsA6Cwk3aSm3ybtCZ/F1L9FmjDPe79/szSlbtj5mpukuxuYmjZnlrTtLumJO4a2vbpXOvty732sHvaMv4h/QgBAKsxFPdKmmUrNCj1uOey/2U6LPMZKQ73PjZulZavDyzfi3q9LyxYMP0+okKFls+jvk3dZ/N53kqK3Ie2Xf0Vuw1KppHK57NuIJN0mDPtvduxgrIfBx10utGSudO0Sad5s6fBR6ee7pW+sl156JUZscRLuuX2hS4X4g8+/orch7Zd/RW7DsKTL8HIaRvY2feimtV6SDXLaOOnsadJVC2u3b3teuuSzTZ6UtbkA0BYk3bTMdpHDzJVJVSN7pHfrJkA1ctMMV5YuPm+oVzvyQun4ieTDygCAbJF00xQj8UpDCbfZu1NVH3fiOenYjph1kXABoK1Yp5u2mdE3QLZScJK8abl0+Cmv11r5Gdjubfcz4oKYCXfmD2IUAgBkiYlUTYj8bxbQ261PjlfMkx66rfk4lq32ZkLXxBY0xNxAL5dJHPlX9Dak/fKvyG3I7OWUxfpvtmuM5N6p2WQlqe9JadL42qJj50pvDcQ//8Rx0ps/rd32zQ3SjXf4JN2Z90kTl8avXPzBF0HR25D2y78ityGzl9vh/D6YMZgAACAASURBVMEsWtfr7Rkhzbxcem1v81UfOlLba/7lo8N7vJK4hgsAHYZrulmrSnyuLD28NVnC9XPWYm9db00vl4QLAB2H4eUmNPXf7NghaXcL1seeeyDRumGJoa0iKHob0n75V+Q2DBtepqfbKiMner3PGeuyqX/G7V79CRMuACA7XNNttSkrvR8p1preSAwjA0Bu0NNtp9lu6GfW4WG7V/l1is/9de1xAIDcoKfbKXomDEuia77fplgAAJmgpwsAQIuQdAEAaBGSLgAALZLJNd3Zs2erXI7znLl8KvoauiKvn6ugDfON9su/ordhEHq6AAC0CLOXgQQCn+rUgGafqwwgf+jpAg264eqhZx2noVLX9VelUx+AzpXJvZdLpZLjmm5+cT3Jn98jFbMw9WPSgUPJ6ih6G/I3mH9d0IY82g9oVlq92jj2Dz6mkWFnoHgYXgYitDLhdsJ5AWSHpAsE+O0z7U98riz9yUfbGwOA9JB0AR+uLI06KXk9192SvI6NN7c/+QNIB9d0gTrvbE9eR/X12O884L0mTZy/fUYa/UfJ6gDQXvR0gTqjR0WX6Z0v3fMj/31BE6CSToxKo+cNoL1IukCVqN6olbyfvn7pU19Onkgr9VV+zvlksvgAdDaSLjAoKqF9+37/7c0mXr/jXnwl+jgSL5BfJF1AUu/E6DIrbs0+DileEp80Pvs4AKSPpAtIOrA5vbqCeqJp9lD7nkyvLgCtw+xldL0/vXrovV8vs5IsXTn+ULIrS0cHpHFzpSNPS2PHxI9n/VfixbNymfSt++LXC6D96Omi693yBe81KKHuOTD0fs6s4fuDerCVRBuUcIOOu2aJ9/qrff77K3GuW+W/H0DnIukCEWYsGnq/7a7aZBk2ZPz+K73XSZcGl6mvq/rzWYsbixNA5yPpoqslvc76xoHgfS+/7r0eOhJcJmxfHMxkBvKFpAtEWDQneN/0RcH74gjrBS++JFndADoPSRcYNBBw+8fHbm9tHBWPrPPf/s4zrY0DQHpIuuhaUyfVfj55lDdce3LVbSDjDN9ueKS58z+8NbpM9fnHjPY+j667HeTkCc2dH0DrkXTRtfY94b99YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5Isbs48r5+7dIb2/zL3PwJ9H1AOgMJF3AR8+IZMefdFHt5975yeobf2qy4wF0BpIuECFOb3fp6trPzoWX/8zX0jkvgHyJnXTNbISZ/b2ZPZplQEAe3d/gbSTXb8omDgCdrZGe7hck/SKrQIBWu35t/LKt7nU2cr5GvgeA9oqVdM1suqTLJN2ZbThA66y9Pt36PndzvHJpP60o7e8BIDtxe7rfkvQlSf8aVMDMlptZ2czKBw8eTCU4oJMsXhm+/7sPeq9bd/nv3/S09xr0XN6K+lnNn74sOjYA+RCZdM1ssaQDzrmdYeWcc99zzpWcc6Xe3t7UAgTaZeaZtZ8fC1iyU2/ecv/tn4jZI61fv3u3z5IkAPkUp6c7R9LlZvaapI2SLjWz72caFdABfuZzMWXhivBjJobc1lGSTvtw+P6Va8L3A8i3yKTrnLvROTfdOfceSUsl/dQ596nMIwMyNvkj4funTRm+7fGIWzAejniAQf/R8P23N/F83LD7NwPoLKzTRdd68zfNHZfVTOYrb2juuKRPKgLQOj2NFHbObZG0JZNIgC73wy3tjgBA1ujpAiGmTmzv+S88p73nB5Auki66WtRQ8b4G7zRV7YPvk+ZfIL13evN1PLshfD+3igTypaHhZaAbuXJwcls0J9nzdhdcJ21+Nvi8AIqFpIuut2qdtOaL4WX6t0gT5nnv92+WptQNO19zk3R3A3clnzNL2naX9MQdQ9te3Sudfbn3Pk4P+/Mp39kKQPbMRT0OpQmlUsmVy8X9Z7qZtTuETGXxO9Fp6tswTq/SSkPlNm6Wlq0OL9+Ie78uLVsw/DxR8QQpehvyN5h/XdCGvl+QpNuELvhlaXcImatvw8kT4j0MPu411CVzpWuXSPNmS4ePSj/fLX1jvfTSK9HHxkm4ky4NXypU9DbkbzD/uqANfb8gw8uApL7+5o/dtNZLskFOGyedPU26amHt9m3PS5d8trlzsjYXyCeSLjAozrBuZVLVyB7p3boJUI3MJHZl6eLzhs438kLp+Inkw8oAOhtJF6gS93pqJeE2mwCrjzvxnHRsR7y6SLhAvrFOF6iz9MboMlYKToA3LZcOP+Ul78rPwHZvu58RF8RLpn/8pegyADobE6ma0AUTANodQuai2jCot1ufHK+YJz10W/NxLFvtzYRu5txhit6G/A3mXxe0IbOX09IFvyztDiFzcdrw7W3SmNF1x5WkvielSeNrt4+dK701EP/8E8dJb/60dts3N0g33jE86S69Ubr/x/HrlorfhvwN5l8XtCGzl4FGnHKx91qfBHtGSDMvl17b23zdh47U9lx/+ejwHq/ENVygaLimC0SoTnyuLD28NVnC9XPWYm9db3WCJ+ECxcPwchO6YFik3SFkrpk2PG2sdOipDIKp0zs/2bphqfhtyN9g/nVBG/p+QXq6QEyHj3q9z5Vrsql/xa2D14wTJlwAnYuebhO64F9o7Q4hc2m1YRpPAspiGLnobcjfYP51QRvS0wXSVlmva6WhpxBVW7Vu+LbTF9QeB6B7MHsZSMlv3vJPomvvaX0sADoTPV0AAFqEpAsAQIuQdAEAaJFMrunu3Lmz0DPTij6zsMhtV0Eb5hvtl39FbsNSKXiGJD1dAABapGNnL3fq+kcAAJrVUT3dG64eev5oGip1XX9VOvUBAJBEJnekMrOGKvV7zFkWpn5MOnAoeT1FvhYhcT2pCIrehrRf/hW5DUulksrlcmc+2i+tXm0c+wcfncawMwCgHdo6vNzKhNsJ5wUAdLe2JN3fPtP+xOfK0p98tL0xAAC6S8uTritLo05KXs91tySvY+PN7U/+AIDu0dJruu9sT15H9fXY7zzgvSZNnL99Rhr9R8nqAAAgSkt7uqNHRZfpnS/d8yP/fUEToJJOjEqj5w0AQJSWJd2o3mjl2aJ9/dKnvpw8kVY/r9RK0jmfTBYfAABJtSTpRiW0b9/vv73ZxOt33IuvRB9H4gUAZCnzpNs7MbrMiluzjsITJ4lPGp99HACA7pR50j2wOb26gnqiafZQ+55Mry4AAKplOnv5T68eeu/Xy6wkS1eOP5TsytLRAWncXOnI09LYMfHjWf+VePGsXCZ967749QIAEEemPd1bvuC9BiXUPQeG3s+ZNXx/UA+2kmiDEm7Qcdcs8V5/tc9/fyXOdav89wMAkERbbwM5Y9HQ+2131SbLsCHj91/pvU66NLhMfV3Vn89a3FicAACkIbOkm/Q66xsHgve9/Lr3euhIcJmwfXEwkxkAkLa29nQXzQneN31R8L44wnrBiy9JVjcAAM1oSdIdCLj942O3t+Lswz2yzn/7O8+0Ng4AQHfJJOlOnVT7+eRR3nDtyVW3gYwzfLvhkebO//DW6DLV5x8z2vs8uu52kJMnNHd+AAD8ZJJ09z3hv31gu3Rsh/c+zhKha786fNvxE7Wf+/qHl7kixuzjyvn7t0hvb/Mvc/An0fUAABBXy6/p9oxIdvxJF9V+7p2frL7xpyY7HgCAuNo6kSpOb3fp6trPzoWX/8zX0jkvAABpi5V0zew1M/sHM3vezFq6mOb+Bm8juX5TNnEAAJBUIz3dDzvnznPORfYTr18bv9JW9zobOV8j3wMAgCiZDC+vvT7d+j53c7xyaT+tKO3vAQDobnGTrpO02cx2mtlyvwJmttzMys0MPy9eGb7/uw96r1t3+e/f9LT3GvRc3or6Wc2fviw6NgAA0mIuamaSJDOb5px7w8ymSPqxpM87554OPGCnhVZ69uXSq3trt1XWzQYN/0Y9iShsf1DdcdYK+z6NKMZ/szwzs3aHkDnaMN9ov/wrchuWSiWVy2XfRozV03XOvTH4ekDSQ5IuSBLQz+4cvm3hivBjJobc1lGSTvtw+P6Va8L3AwCQtcika2anmNnYyntJH5P0j2HHTP5IeJ3Tpgzf9njELRgPRzzAoP9o+P7bm3g+btj9mwEAaFSch9hPlfTQ4HBHj6R7nXOPhx3w5m+aCyarmcxX3tDccUmfVAQAQLXIpOuce0WSzyPm8+OHW9odAQAAbbwj1dSJ7Tqz58Jz2nt+AED3ySzpRg0V72vwTlPVPvg+af4F0nunN1/HsxvC93OrSABA2uJc081M2DKfRXOSPW93wXXS5meDzwsAQKtlmnRXrZPWfDG8TP8WacI87/3+zdKUumHna26S7n40/jnnzJK23SU9ccfQtlf3emuDpXg97M+nfGcrAACkmDfHaLhSG7o5RtwbUFTKbdwsLVsdXr4R935dWrZg+Hmi4glT5EXdEgvzi6DobUj75V+R2zDs5hiZJ93JE+I9DD7uNdQlc6Vrl0jzZkuHj0o/3y19Y7300ivRx8ZJuJMujV4qVORfFok/+CIoehvSfvlX5DYMS7qZX9Pt62/+2E1rvSQb5LRx0tnTpKsW1m7f9rx0yWebOydrcwEAWWnJRKo4w7qVSVUje6R36yZANTKT2JWli88bOt/IC6XjJ9IZVgYAIImWzV6Oez21knCbTYDVx514Tjq2I15dJFwAQNZaenOMpTdGl7FScAK8abl0+CkveVd+BrZ72/2MuCBeMv3jL0WXAQAgqcwnUtUL6u3WJ8cr5kkP3dZ8DMtWezOhmzl3lCJPAJCYxFEERW9D2i//ityGbZ297OftbdKY0XXHlKS+J6VJ42u3j50rvTUQ/9wTx0lv/rR22zc3SDfeMTzpLr1Ruv/H8euuKPIvi8QffBEUvQ1pv/wrchu2dfayn1Mu9l7rk2DPCGnm5dJre4cfE9ehI7U9118+OrzHK3ENFwDQem174IFUm/hcWXp4a7KE6+esxd663uoET8IFALRDW4aX6502Vjr0VOphDNM7P9m64YoiD4tIDG0VQdHbkPbLvyK3Ydjwclt7uhWHj3q9z5Vrsql/xa2D14xTSLgAADSrI3q6ftJ4ElBWw8hF/heaxL+yi6DobUj75V+R27Dje7p+Kut1rTT0FKJqq9YN33b6gtrjAADoJG19nm5cv3nLP4muvaf1sQAA0KyO7ekCAFA0JF0AAFqEpAsAQItkck139uzZKpdTmH7coYo+s7DIsworaMN8o/3yr+htGISeLgAALULSBQCgRXKxZAgA0KSdKQzjzi7+cHer0NMFgKLZf6uXbNNIuNJQXfszuldvFyHpAkBRHHvTS457vpRN/Xtu8Oo/tj+b+rsAw8sAUARp9Wrj2H2698qwc8Po6QJA3rUy4XbCeXOMpAsAebVrVPsT306TDm1sbww5QtIFgDzaaZJ7N3E1192SQiyvLmt/8s8JrukCQN7sGp24iuont33nAe818XPMd42Szv9dwkqKjZ4uAOSNi05svfOle37kvy/oeeOJn0OeQs+76Ei6AJAnEcO4VvJ++vqlT305eSKt1Ff5OeeTyeLrdiRdAMiLiIT27fv9tzebeP2Oe/GVGAeSeAORdAEgD44fiCyy4tYWxKGYSfx4X+Zx5BFJFwDy4IWpqVUVNGEq8USqai/0plhZcTB7GQA63a+H1vX49TIrydKV4w8lu7J0dEAaN1c68rQ0dkz8cNZ/Zeh9WDzat046/YvxK+4C9HQBoNPt/TNJwQl1T9XI85xZw/cH9WAriTYo4QYdd80S7/VX+/z3/1ucb1zvX6CLkXQBIOdmLBp6v+2u2mQZNmT8/iu910mXBpepr6v681mLG4sTJF0A6GwJZwK/ETL/6uXXvddDR4LLhO2LhZnMNUi6AJBzi+YE75u+KHhfHGG94MWXJKu7G5F0ASAnBrb7b3/s9tbGUfHIOv/t7zzT2jjyhKQLAJ3qWO1MpZNHeddUTx41tC3OMp8NjzR3+oe3RpepPv+Y0d7n0SfVFTp2sLkACoikCwCdavcZvpsHtkvHdnjv4ywRuvarw7cdP1H7ua9/eJkrVkXXXTl//xbp7W0BhXZPia6oS5B0ASCHekYkO/6ki2o/985PVt/4U5Md3y1iJV0zm2Bmf21m/2RmvzCzP8w6MABAPHF6u0tX1352Lrz8Z76WznlRK25P93ZJjzvn/p2kWZJ+kV1IAIC03b+5sfLrN2UTR7eLTLpmNl7SXEl3SZJz7l3nnM/oPwAgTdevjV+21b3ORs7XyPcoujg93ZmSDkpab2Z/b2Z3mtkpGccFAF1vbcp3UfzczfHKpf20orS/R57FSbo9ks6X9JfOuQ9JelvSn9cXMrPlZlY2s/LBg0wPB4BWW7wyfP93H/Ret+7y37/pae816Lm8FfWzmj99WXRs8MRJunsk7XHODU5Q11/LS8I1nHPfc86VnHOl3l4e6QQAWZt5Zu3nx4KW7NSZt9x/+ydi9kjr1+/e7bMkCf4ik65zbp+k183sA4ObPiLppUyjAgBE+tmdw7ctXBF+zMSQ2zpK0mkfDt+/ck34foSL+zzdz0u6x8xOkvSKpGuzCwkAIEmadTD0YfDTfO458XjELRgPRzzAoP9o+P7b7wvf7+vcviYOKqZYSdc597wkVmQBQCv1TG7qsKxmMl95Q5MHjpyUahx5xh2pAACx/HBLuyPIP5IuAOTY1IntPf+F57T3/HlD0gWATjY7/H6N+xq801S1D75Pmn+B9N7pzdfx7IaIAhHxd5u4E6kAAB3KlYOv4y6ak+x5uwuukzY/G3xeNIakCwCdbvpt0p7wWUz9W6QJ87z3+zdLU+qGna+5Sbr70finnDNL2naX9MQdQ9te3Sudfbn3PlYPe8ZfxD9hlzAX9aiJJpRKJVcuF/efQGbW7hAylcXvRKehDfOtK9tvZ/R3ttJQ73PjZmnZ6vDyjbj369KyBcPPEypkaLkL2tD3C5J0m9AFvyztDiFztGG+dWX7HTsY62HwcZcLLZkrXbtEmjdbOnxU+vlu6RvrpZdeiRFfnP+9n9sXulSoC9rQ9wsyvAwAeTCy+dvrblrrJdkgp42Tzp4mXbWwdvu256VLPtvkSVmb64ukCwB5MdtFDjNXJlWN7JHerZsA1chNM1xZuvi8oV7tyAul4yeSDyt3O5IuAORJjMQrDSXcZu9OVX3cieekYzti1kXCDcU6XQDIm5nRN0C2UnCSvGm5dPgpr9da+RnY7m33M+KCmAl35g9iFOpuTKRqQhdMAGh3CJmjDfON9lNgb7c+OV4xT3rotuZjWbbamwldLXCIuYFebhe0IbOX09IFvyztDiFztGG+0X6Ddo2R3Ds1m6wk9T0pTRpfW3TsXOmtgfgxTBwnvfnT2m3f3CDdeIdP0p15nzRxafzK1RVtyOxlACiU8wezaF2vt2eENPNy6bW9zVd96Ehtr/mXjw7v8UriGm6DuKYLAHlXlfhcWXp4a7KE6+esxd663ppeLgm3YQwvN6ELhkXaHULmaMN8o/0CHDsk7W7B+thzDyRaNyx1RRv6fkF6ugBQFCMner3PGeuyqX/G7V79CRNuN+OaLgAUzZSV3o8Ua01vJIaRU0NPFwCKbLYb+pl1eNjuVX6d4nN/XXscUkNPFwC6Rc+EYUl0zffbFEuXoqcLAECLkHQBAGgRki4AAC2SyTXdnTt3FnoNFmsg8482zDfaL/+K3IalUvDTIejpAgDQIsxeBhAo1gPLIzT7PFegiOjpAqhxw9VDz1hNQ6Wu669Kpz4gzzK597KZFXewXsW+FiFxPakImmlDv0e5ZWHqx6QDh5LVQfvlX5HbsFQqqVwu82g/AP7S6tXGsX/w8XAMO6MbMbwMdLlWJtxOOC/QTiRdoEv99pn2Jz5Xlv7ko+2NAWglki7QhVxZGnVS8nquuyV5HRtvbn/yB1qFa7pAl3lne/I6qq/HfucB7zVp4vztM9LoP0pWB9Dp6OkCXWb0qOgyvfOle37kvy9oAlTSiVFp9LyBTkfSBbpIVG/USt5PX7/0qS8nT6SV+io/53wyWXxA3pF0gS4RldC+fb//9mYTr99xL74SfRyJF0VG0gW6QO/E6DIrbs0+DileEp80Pvs4gHYg6QJd4MDm9OoK6omm2UPtezK9uoBOwuxloOD+9Oqh9369zEqydOX4Q8muLB0dkMbNlY48LY0dEz+e9V+JF8/KZdK37otfL5AH9HSBgrvlC95rUELdc2Do/ZxZw/cH9WAriTYo4QYdd80S7/VX+/z3V+Jct8p/P5BnJF2gy81YNPR+2121yTJsyPj9V3qvky4NLlNfV/XnsxY3FidQBCRdoMCSXmd940Dwvpdf914PHQkuE7YvDmYyo2hIukCXWzQneN/0RcH74gjrBS++JFndQB6RdIEuMRBw+8fHbm9tHBWPrPPf/s4zrY0DaCWSLlBQUyfVfj55lDdce3LVbSDjDN9ueKS58z+8NbpM9fnHjPY+j667HeTkCc2dH+hEJF2goPY94b99YLt0bIf3Ps4SoWu/Onzb8RO1n/v6h5e5Isbs48r5+7dIb2/zL3PwJ9H1AHlB0gW6UM+IZMefdFHt5975yeobf2qy44G8IOkCXS5Ob3fp6trPzoWX/8zX0jkvUDSRSdfMPmBmz1f9HDGzla0IDkBnuL/B20iu35RNHEDeRSZd59w/O+fOc86dJ2m2pAFJD2UeGYBErl8bv2yre52NnK+R7wF0ukaHlz8i6V+cc7/MIhgA6Vl7fbr1fe7meOXSflpR2t8DaKdGk+5SSb63IDez5WZWNjPuIQPk0OKIi0bffdB73brLf/+mp73XoOfyVtTPav70ZdGxAUVhLmpGRKWg2UmS9kr69865/RFl41WaU3H/m+WVmbU7hMx1QxtGrcE9+3Lp1b212yrHBA3/Rj2JKGx/UN1x1goPO6YL2q/oityGpVJJ5XLZtxEb6ekulLQrKuECyIef3Tl828IV4cdMDLmtoySd9uHw/SvXhO8Hiq6RpLtMAUPLADrP5I+E7582Zfi2xyNuwXg44gEG/UfD99/exP9Bwu7fDORNrKRrZqdI+qikv8k2HABpefM3zR2X1UzmK29o7rikTyoCOklPnELOubclTYosCAABfril3REA7ccdqYAuNnVie89/4TntPT/QaiRdoMCihor3NXinqWoffJ80/wLpvdObr+PZDeH7uVUkiibW8DKA4gpb5rNoTrLn7S64Ttr8bPB5gW5D0gUKbtU6ac0Xw8v0b5EmzPPe798sTakbdr7mJunuR+Ofc84sadtd0hN3DG17da+3NliK18P+fMp3tgI6QeybYzRUKTfHyDUW5udffRvGvQFFpdzGzdKy1eHlG3Hv16VlC4afJyqeIN3WfkVU5DYMuzkGSbcJRf5lkfiDL4L6Npw8Id7D4ONeQ10yV7p2iTRvtnT4qPTz3dI31ksvvRJ9bJyEO+nS8KVC3dZ+RVTkNgxLugwvA12gr7/5Yzet9ZJskNPGSWdPk65aWLt92/PSJZ9t7pyszUVRkXSBLhFnWLcyqWpkj/Ru3QSoRmYSu7J08XlD5xt5oXT8RPJhZSDvSLpAF4l7PbWScJtNgNXHnXhOOrYjXl0kXBQd63SBLrP0xugyVgpOgDctlw4/5SXvys/Adm+7nxEXxEumf/yl6DJA3jGRqglFngAgMYmjCKLaMKi3W58cr5gnPXRb83EsW+3NhG7m3GG6vf2KoMhtyOzllBX5l0XiD74I4rTh29ukMaPrjitJfU9Kk8bXbh87V3prIP75J46T3vxp7bZvbpBuvGN40l16o3T/j+PXLdF+RVDkNmT2MoBhTrnYe61Pgj0jpJmXS6/tHX5MXIeO1PZcf/no8B6vxDVcdB+u6QJdrjrxubL08NZkCdfPWYu9db3VCZ6Ei27E8HITijwsIjG0VQTNtOFpY6VDT2UQTJ3e+cnWDUu0XxEUuQ3Dhpfp6QKQ5N1ZykrSyjXZ1L/i1sFrxgkTLpBn9HSbUOR/oUn8K7sI0mrDNJ4ElMUwMu2Xf0VuQ3q6AJpSWa9rpaGnEFVbtW74ttMX1B4HYAizlwHE8pu3/JPo2ntaHwuQV/R0AQBoEZIuAAAtQtIFAKBFsrqm2yfplxnV7Wfy4Dlbog0zC1v6/dqg5d+vxW1I+6WMv8HUFb0NW/39zgrakcmSoVYzs7JzrrDzJPl++cb3y7+if0e+X+swvAwAQIuQdAEAaJGiJN3vtTuAjPH98o3vl39F/458vxYpxDVdAADyoCg9XQAAOh5JFwCAFsl10jWzj5vZP5vZy2b25+2OJ21m9ldmdsDM/rHdsWTBzGaY2VNm9pKZvWhmX2h3TGkys9Fm9pyZvTD4/b7a7piyYGYjzOzvzezRdseSNjN7zcz+wcyeN7MUnrnUWcxsgpn9tZn9k5n9wsz+sN0xpcnMPjDYdpWfI2a2sq0x5fWarpmNkPT/SfqopD2S/k7SMufcS20NLEVmNlfSW5L+h3PunHbHkzYzO0PSGc65XWY2VtJOSVcUpQ3NW/1/inPuLTMbKWmbpC84555tc2ipMrPrJZUkjXPOLW53PGkys9cklZxzhbwxhpndLelnzrk7zewkSWOcc4V84vFgznhD0oXOuVbevKlGnnu6F0h62Tn3inPuXUkbJX2izTGlyjn3tKRD7Y4jK865Xzvndg2+PyrpF5KmtTeq9DjPW4MfRw7+5PNfuQHMbLqkyyTd2e5Y0BgzGy9prqS7JMk5925RE+6gj0j6l3YmXCnfSXeapNerPu9Rgf6H3W3M7D2SPiRpR3sjSdfg0Ovzkg5I+rFzrlDfT9K3JH1J0r+2O5CMOEmbzWynmS1vdzApmynpoKT1g5cH7jSzU9odVIaWSrqv3UHkOemiIMzsVEkPSlrpnDvS7njS5Jw74Zw7T9J0SReYWWEuE5jZYkkHnHM72x1Lhi52zp0vaaGk/zx4yacoeiSdL+kvnXMfkvS2pMLNjZGkwaHzyyX9oN2x5DnpviFpRtXnQ6Y6tgAAAXNJREFU6YPbkCOD1zoflHSPc+5v2h1PVgaH7Z6S9PF2x5KiOZIuH7zuuVHSpWb2/faGlC7n3BuDrwckPSTvslZR7JG0p2r05a/lJeEiWihpl3Nuf7sDyXPS/TtJ7zezmYP/ilkqaVObY0IDBica3SXpF865te2OJ21m1mtmEwbfnyxv0t8/tTeq9DjnbnTOTXfOvUfe399PnXOfanNYqTGzUwYn+Glw2PVjkgqzksA5t0/S62b2gcFNH5FUiEmMPpapA4aWpewe7Zc559xxM7tO0hOSRkj6K+fci20OK1Vmdp+keZImm9keSV9xzt3V3qhSNUfS1ZL+YfC6pyStds79bRtjStMZku4enDX5e5IecM4VbllNgU2V9NDgI+h6JN3rnHu8vSGl7vOS7hnsuLwi6do2x5O6wX8wfVTSf2p3LFKOlwwBAJA3eR5eBgAgV0i6AAC0CEkXAIAWIekCANAiJF0AAFqEpAsAQIuQdAEAaJH/H3g7SUIqLC/qAAAAAElFTkSuQmCC\n",
           "text/plain": [
            "
    " ] }, - "metadata": {}, + "metadata": { + "needs_background": "light" + }, "output_type": "display_data" } ], @@ -1164,12 +1169,14 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+4FNWd7/vP97A3IIZfGzaYANfAJE/unRhxZI84Q+QSQ8aAYPTeuTNwjR7NzeXc3GMIipMZeZ55YvKcaK4KhIlzJydHBjxnDGjGMaJOlGgEA0adDaNMTGbuY8BERH5sYQcUE4Gz7h+1293du6q6uruqq6vq/Xqefrq7atVaq3ux+fZatWqVOecEAADa279LuwIAAKA2AjYAABlAwAYAIAMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAGELCBNmNmHzSzfzSzY2Z20MzuNrOOkPTjzOxvBtKeNLN/MbN/38o6A0geARtoP/+vpMOS3i/pAkn/s6T/2y+hmQ2X9KSkcyX9gaSxkv5M0h1mtrwltQXQEgRsoP1Ml/SAc+43zrmDkh6X9NGAtNdI+h8k/W/OuX3OuVPOucclLZf0n8xstCSZmTOzD5UOMrONZvafyt4vMrMXzazfzJ41s/PL9n3AzB40syNmtq/8h4CZ3WpmD5jZfzWzE2b2spn1lO3/czN7fWDfv5nZJ+P5ioDiIWAD7WedpCVmNsrMpkhaIC9o+/mUpB84596u2v6gpFGSLq5VmJldKOlvJf0HSRMk/WdJW8xshJn9O0mPSHpJ0hRJn5S0wswuK8viCkmbJY2TtEXS3QP5fkTSDZJ+3zk3WtJlkl6tVR8A/gjYQPvZLq9HfVzSfkm9kr4fkHaipDeqNzrnTkvqk9Qdobz/U9J/ds4975w745y7V9Jv5QX735fU7Zz7mnPuXefcXkn/RdKSsuN3OOf+0Tl3RtJ/kzRzYPsZSSMk/a6ZdTrnXnXO/SJCfQD4IGADbWSgR/uEpH+QdLa8gDxe0v8TcEifvHPd1fl0DBx7JEKx50paOTAc3m9m/ZKmSfrAwL4PVO1bJWly2fEHy16flDTSzDqcc69IWiHpVkmHzWyzmX0gQn0A+CBgA+2lS16wvNs591vn3JuSNkhaGJD+SUkLzOzsqu3/q6RTkl4YeH9S3hB5yTllr1+T9HXn3Liyxyjn3KaBffuq9o12zgXVp4Jz7rvOuY/LC/xOwT88ANRAwAbaiHOuT9I+SV8wsw4zGyfp38s7h+znv8kbNv/ewOVgnQPnl/9K0h3OuV8PpHtR0v9uZsPM7NPyZp6X/BdJ/5eZzTbP2WZ2+cCEtRckHR+YPHbWwPHnmdnv1/osZvYRM7vUzEZI+o2kd+QNkwNoAAEbaD//i6RPyxvOfkXSaUk3+iV0zv1W0nx5PeHn5QXFxyV9U9JXy5J+SdJiSf2SrlbZOXHnXK+889h3Szo2UOZ1A/vODBx3gbwfEn2S7pF3+VgtIyR9Y+CYg5ImyRtOB9AAc86lXQcAMTGzTkk/kPS6pOscf+BAbtDDBnLEOXdK3vnrX0j6SMrVARAjetgAAGQAPWwAADIg8IYCrTJx4kT3wQ9+MO1qJGbXrl1pVyFRs2bNSrsKiaMNs432y768t6GkPudczUWOUh8S7+npcb29vanWIUlmlnYVEpX2v59WiKsNXQz/zAdX6Y5P3tuQv8Hsy3sbStrlnKv5182QOJCgm6/xAnUcwVoazOumq+PJD0B2ELCBBHSN8QLrnV9KJv/VN3r5T+pKJn8A7Sf1c9hA3sTVm47i0FbvOYmhcgDthR42EKNWBut2KBdA6xCwgRj85tn0g6brlf70U+nWAUByCNhAk1yvNGJ48/nccEfzeWy+Pf0fDgCSwTlsoAnv7Gw+j/Lzz3/9gPfcbND9zbPSyD9sLg8A7YUeNtCEkSNqp+meL933A/99QZPFmp1EFkePH0B7IWADDarVC7Ye79HXL332L5sPwqX8So/z/qS5+gHIFgI20IBawfBb9/tvbzRo+x338t7axxG0gfwgYAN16o6wWMnyO5OvhxTtB8CEscnXA0DyCNhAnQ5vjS+voB5wnD3jvqfiywtAepglDtThz64ZfO3Xuy0FWtcbffjb9UonTkpj5krHn5FGj4penw1fiVafFUulb26Kni+A9kMPG6jDHQNrgwcF4/2HB1/PmTl0f1DPuRSkg4J10HHXLfaef3XQf3+pnmtX+u8HkB0EbCBG0xYOvt6xvjLQhg1zf/gq73nCpcFpqvMqf3/uovrqCSB7CNhARM2eV379cPC+V17zno8eD04Tti8KZowD2UbABmK0cE7wvqkLg/dFEdb7XnRJc3kDaH8EbKABJwOWJH1sXWvrUfLIWv/t7zzb2noASA4BG4hg8oTK92eN8IaYzypbmjTKkPPGRxor/+HttdOUlz9qpPd+ZNUSpRPHNVY+gPQRsIEIDj7hv/3kTunU897rKJdxXf/VodtOn6l839c/NM2VEWZ5l8rv3ya9vcM/zZEna+cDoD0RsIEmdQxr7vjhF1e+757fXH5j39fc8QDaEwEbiFGUXvaSVZXvnQtP/7mvxVMugGwjYAMtdn+dS5tu2JJMPQBkSyIB28w+bWb/ZmavmNlfJFEG0Eo3rYmettW93XrKq+dzAGgvsQdsMxsm6a8lLZD0u5KWmtnvxl0O0Eprboo3vy/cHi1d3Hf9ivtzAGidJHrYF0l6xTm31zn3rqTNkj6TQDlA21q0Inz/tx/0nrfv9t+/5RnvOei+2iXVs8evvbx23QBkUxIBe4qk18re7x/Y9h4zW2ZmvWbWe+TIkQSqALTW9A9Uvn8s4LKqavOW+W//TMSecPX12ff6XDYGIB+SCNjms61iHqxz7jvOuR7nXE93d3cCVQBa68f3DN22YHn4MV0hS41K0vhPhO9fsTp8P4B8SSJg75c0rez9VEkHEigHaJmJnwzfP2XS0G2P11gW9FiNm3n0nwjfv66B+1uHrUcOoL0lEbD/SdKHzWy6mQ2XtEQSF6Yg0978dWPHJTVj/KqbGzuu2Tt+AUhPR9wZOudOm9kNkp6QNEzS3zrnXo67HKDIvr8t7RoAaLXYA7YkOef+UdI/JpE30K4md0mHjqZX/uzz0isbQPJY6QyIqNbw9sE6VzAr97EPSfMvkn5nauN5PLcxfD/LlwLZlkgPGygq1xscGBfOae5+2ZfdIG19LrhcAPlGwAbqsHKttPrG8DT926Rx87zXh7ZKk7oq9193q3Tvo9HLnDNT2rFeeuLuwW37DkgzrvBeR+nZfzHmFdMAtJ65WrcKSlhPT4/r7c1v98DM77L0/Ej7308rVLdhlN6s9Qym27xVWroqPH09vvt1aellQ8upVZ8geW9D/gazL+9tKGmXc67mSSsCdsLy/g8t7X8/rVDdhhPHSUeejHBcxHPGi+dK1y+W5s2Sjp2QfrJHum2D9LO9tY+NEqwnXBp+OVfe25C/wezLexsqYsBmSByoU19/48duWeMF6CDjx0gzpkhXL6jcvuNF6ZLPN1Ym114D+UDABhoQZSi6NAGts0N6t2qyWD0ztl2v9PELBsvrnC2dPtP8UDiAbCFgAw2Kev64FKwbDZ7lx515QTr1fLS8CNZAvnAdNtCEJbfUTmM9wcHz1mXSsae9wF96nNzpbfcz7KJogfiPv1w7DYBsYdJZwvI+WSLtfz+tUKsNg3rZ1YH1ynnSQ3c1Xo+lq7wZ542UHSbvbcjfYPblvQ3FpDOgNaxHenuHNGrk0H19T0kTxlZuGz1Xeutk9Py7xkhv/kjadJv3kKRvbJRuuXto2iW3SPf/MHreALKDgA3E4OyPe8/VPd6OYdL0K6RXm7jB7NHjlT3mXz46tKctcc4ayDvOYQMxKg+arld6eHtzwdrPuYu867bLfxwQrIH8o4cNxMx6pPGjpaNPS9de7j2S0j2/uevCAWQHPWwgAcdOeIF7xepk8l9+p5c/wRooDnrYQILWbfIeUjx31GLoGyguethAi5Sux7aewbt5lVu5dui2cy6rPA5AcdHDBlLw67f8A/Ca+1pfFwDZQA8bAIAMIGADAJABBGwAADKAgA0AQAakfvMPM8v1yvVpf79JK8Ci/LRhxtF+2VeANuTmHwCQmDPHpBe7KjatXCutvrEq3fkHpM73t65eyC162AlL+/tNGr/usy/vbRhr++2K4buaFe+/p7y3n1SIv8FIPWzOYQNAmEN3eoE6jmAtDeZ1KKF1a5Fb9LATlvb3mzR+3Wdf3tuw4fY79aa0Z2K8lfFz/kGpc3LDh+e9/aRC/A1yDhsAGhJXbzqKPed4zzEPlSN/GBIHgHKtDNbtUC4yg4ANAJK0e0T6QXOXSUc3p1sHtC0CNgDsMsm923Q2N9wRQ132LU3/hwPaEpPOEpb295s0JrxkX97bsGb77R4pud82VYbfndeavv+5DZcurF2vvLefVIi/QS7rAoCaIgTr7vnSfT/w3xd0n/Km718eQ48f+UIPO2Fpf79J49d99uW9DUPbr8bQc5Sec1hgrpX2ozOknz4QWoWas8fz3n5SIf4G6WEDQKAawfpb9/tvb7Tn7Hfcy3sjHMj5bAwgYAMontOHayZZfmcL6qGIPwBO9yVeD7Q/AjaA4nmp8ZXFqgVNLmt60lm5l7pjzAxZxUpnAIrljcFrr8LOUbve6MPfrlc6cVIaM1c6/ow0elT06mz4yuDr0HPmB9dK51TfCgxFQg8bQLEc+HNJwcF4f9lo+ZyZQ/cH9ZxLQTooWAcdd91i7/lXB/33v1fP12/yT4DCIGADQJlpCwdf71hfGWjDhrk/fJX3POHS4DTVeZW/P3dRffVE8RCwARRHkzOuXw+Zq/bKa97z0ePBacL2RcKM8UIjYANAmYVzgvdNXRi8L4qw3veiS5rLG/lHwAZQSCd3+m9/bF1r61HyyFr/7e8829p6oH0RsAEUw6nKWV1njfDOIZ81YnBblEuxNj7SWPEPb6+dprz8USO99yOHVyU6daSxCiDzWJo0YWl/v0ljWcTsy3sbvtd+Ied/T5+ROmcPpPcJ2tUzyqvTlB8vSUeelCaOqy+P8jT926Sx7wusbsVypXlvP6kQf4MsTQoAUXQMa+744RdXvu+e31x+ocEahUXABoAyURZLWbKq8n2tDuDnvhZPuSi22AO2mf2tmR02s5/GnTcAtIP7t9aXfsOWZOqBYkmih71R0qcTyBcAGnbTmuhpW93brae8ej4H8iX2gO2ce0bS0bjzBYBmrIl5Zc8v3B4tXdx3/Yr7cyA7OIcNAD4WrQjf/+0Hveftu/33b3nGew66r3bJlSsr3197ee26oZhSCdhmtszMes0szhvQAUDDpn+g8v1jO6IdN2+Z//bPROwJV1+ffe9Xox2H4kklYDvnvuOc64ly3RkAtMKP7xm6bcHy8GO6QpYalaTxnwjfv2J1+H6gHEPiAIphZvgKYVMmDd32eI1lQY/VuJlH/4nw/es2he/3dX5fAwchD5K4rGuTpJ9I+oiZ7Tez/yPuMgCgbh0TGzosqRnjV93c4IGdE2KtB7KjI+4MnXNL484TAPLm+9vSrgGyhiFxABgwuSvd8mefl275aG/c/CNhaX+/SePGA9mX9zYc0n4hNwGRGh8C/9iHvIC/74D0i/2N5VHzbmGzhv5bzHv7SYX4G4x084/Yh8QBIMtcb3DQXjinuftlX3aDtPW54HKBMARsAMUy9S5pf/iMr/5t0rh53utDW6VJVUPl190q3fto9CLnzJR2rJeeuHtw274D0owrvNcHo6xNPu2voheIXGJIPGFpf79JYzgu+/Lehr7tV2NYXPJ62aVe7+at0tJV4enr8d2vS0svG1pOKJ/hcCn/7ScV4m8w0pA4ATthaX+/SeM/i+zLexv6tt+pI9Ienwuvq0Q9n714rnT9YmneLOnYCekne6TbNkg/2xuhflGC9fl9gZdz5b39pEL8DXIOGwB8dXY3fOiWNV6ADjJ+jDRjinT1gsrtO16ULvl8g4Vy7TVEDztxaX+/SePXffblvQ1D2y/i0Hhnh/Tuc0O3R65DVS+6c7Z0+kxzQ+Hv1SPn7ScV4m+QHjYAhJrlIgXtUrBu9JKv8uPOvCCdej5iXjWCNYqFhVMAFNv02gt6W09wgL11mXTsaa+3XHqc3Olt9zPsoojBevr3IiRCkTAknrC0v9+kMRyXfXlvw0jtF9DLrg6sV86THrqr8bosXeXNOC8XOCwesXed9/aTCvE3yCzxdpD295s0/rPIvry3YeT22z1Kcu9UbLIeqe8pacLYyqSj50pvnYxeh64x0ps/qtz2jY3SLXf7BOzpm6SuJZHzznv7SYX4G+QcNgBEduFABK7qbXcMk6ZfIb16oPGsjx6v7K3/8tGhPW1JnLNGKM5hA0C5sqDpeqWHtzcXrP2cu8i7bruid02wRg0MiScs7e83aQzHZV/e27Dh9jt1VNrTguufzz/c1HXheW8/qRB/g5GGxOlhA4Cfzi6v1zttbTL5T1vn5d9EsEax0MNOWNrfb9L4dZ99eW/DWNsvwjXbNcU89J339pMK8TdIDxsAYjXLDT5mHhuye6VfZ/z8NyqPAxpEDzthaX+/SePXffblvQ1pv+wrQBvSwwYAIC8I2AAAZAABGwCADEh9pbNZs2aptzfKPeayKe/nl/J+bkmiDbOO9su+vLdhVPSwAQDIgNR72AAAtErg3dHq0Oh90ZtFDxsAkGs3XzN4r/I4lPK66ep48ouKgA0AyKWuMV5gvfNLyeS/+kYv/0ldyeRfjSFxAEDuxNWbjuLQwK1Skx4qp4cNAMiVVgbrVpZLwAYA5MJvnk0vWJe4XulPP5VM3gRsAEDmuV5pxPDm87nhjubz2Hx7Mj8cOIcNAMi0d3Y2n0f5+ee/fsB7bjbo/uZZaeQfNpdHOXrYAIBMGzmidpru+dJ9P/DfFzRZrNlJZHH0+MsRsAEAmVWrF2w93qOvX/rsXzYfhEv5lR7n/Ulz9asHARsAkEm1guG37vff3mjQ9jvu5b21j4sraBOwAQCZ0x1hsZLldyZfDynaD4AJY5svh4ANAMicw1vjyyuoBxzncHbfU83nwSxxAECm/Nk1g6/9erelQOt6ow9/u17pxElpzFzp+DPS6FHR67PhK9Hqs2Kp9M1N0fOtRg8bAJApdwysDR4UjPcfHnw9Z+bQ/UE951KQDgrWQcddt9h7/tVB//2leq5d6b8/KgI2ACBXpi0cfL1jfWWgDRvm/vBV3vOES4PTVOdV/v7cRfXVs14EbABAZjR7Xvn1w8H7XnnNez56PDhN2L4omqk/ARsAkCsL5wTvm7oweF8UYb3vRZc0l3ctBGwAQCadDFiS9LF1ra1HySNr/be/82w8+ROwAQCZMHlC5fuzRnhDzGeVLU0aZch54yONlf/w9tppyssfNdJ7P7JqidKJ4xorn4ANAMiEg0/4bz+5Uzr1vPc6ymVc13916LbTZyrf9/UPTXNlhFnepfL7t0lv7/BPc+TJ2vn4IWADADKvY1hzxw+/uPJ99/zm8hv7vuaO90PABgDkSpRe9pJVle+dC0//ua/FU24zCNgAgMK5v86lTTdsSaYe9Yg9YJvZNDN72sx+bmYvm9mX4i4DAFA8N62Jnjbp3m4z5dXzOcol0cM+LWmlc+5/knSxpP9oZr+bQDkAgAJZc1O8+X3h9mjp4r7rV6OfI/aA7Zx7wzm3e+D1CUk/lzQl7nIAAAizaEX4/m8/6D1v3+2/f8sz3nPQfbVLqmePX3t57bo1ItFz2Gb2QUm/J+n5qu3LzKzXzHqPHDmSZBUAAAUx/QOV7x8LuKyq2rxl/ts/E7EnXH199r0+l43FIbGAbWbvk/SgpBXOuYrVV51z33HO9Tjnerq7u5OqAgCgQH58z9BtC5aHH9MVstSoJI3/RPj+FavD98cpkYBtZp3ygvV9zrl/SKIMAECxTPxk+P4pk4Zue7zGsqDHatzMo/9E+P51DdzfOmw98jBJzBI3Sesl/dw51+BcOAAAKr3568aOS2rG+FU3N3Zco3f8SqKHPUfSNZIuNbMXBx5N3h8FAID28v1trS2vI+4MnXM7JFnc+QIAUMvkLunQ0fTKn31ecnmz0hkAIDNqDW8frHMFs3If+5A0/yLpd6Y2nsdzG8P3NzM8H3sPGwCANLne4MC4cE5z98u+7AZp63PB5SaJgA0AyJSVa6XVN4an6d8mjZvnvT60VZrUVbn/ululex+NXuacmdKO9dITdw9u23dAmnGF9zpKz/6LTa6YZq7WLUoS1tPT43p7E/5ZkiJv0nx+pf3vpxVow2yj/bLPrw2j9GatZzDd5q3S0lXh6evx3a9LSy8bWk6t+gTY5ZyrOVhOwE4Y/1lkH22YbbRf9vm14cRx0pEnIxwb8Zzx4rnS9YulebOkYyekn+yRbtsg/Wxv7WOjBOsJl4ZezhUpYDMkDgDInL7+xo/dssYL0EHGj5FmTJGuXlC5fceL0iWfb6zMRq+9LkfABgBkUpSh6NIEtM4O6d2qyWL1zNh2vdLHLxgsr3O2dPpM00PhdSFgAwAyK+r541KwbjR4lh935gXp1PPR8opzlTWuwwYAZNqSW2qnsZ7g4HnrMunY017gLz1O7vS2+xl2UbRA/Mdfrp2mHkw6SxgTXrKPNsw22i/7orRhUC+7OrBeOU966K7G67J0lTfjvJGyQzDpDABQDNYjvb1DGjVy6L6+p6QJYyu3jZ4rvXUyev5dY6Q3fyRtus17SNI3Nkq33D007ZJbpPt/GD3vqAjYAIBcOPvj3nN1j7djmDT9CunVA43nffR4ZY/5l48O7WlLyd0ZTOIcNgAgZ8qDpuuVHt7eXLD2c+4i77rt8h8HSQZriR42ACCHrEcaP1o6+rR07eXeIynd85u7LjwqetgAgFw6dsIL3CtWJ5P/8ju9/FsRrCV62ACAnFu3yXtI8dxRK+mh7yD0sAEAhVG6Htt6Bu/mVW7l2qHbzrms8ri00MMGABTSr9/yD8Br7mt9XaKghw0AQAYQsAEAyAACNgAAGUDABgAgA1K/+YeZ5Xrl+rS/36Tl/cYKEm2YdbRf9hWgDSPd/IMeNtrSuNGVt7pzvdJNVw/dds6EtGsKAK1BDzthaX+/SYvz1327LmhAG2Yb7Zd9BWhDethofzdfM9hbjkN5bxwA8oQedsLS/n6T1uiv+9K9ZZM2+Y+kw0eby4M2zDbaL/sK0IaRetisdIaWi6s3HcWhgfvVprmcIADEgSFxtFQrg3U7lAsAcSFgoyV+82z6QdP1Sn/6qXTrAACNImAjca5XGjG8+XxuuKP5PDbfnv4PBwBoBJPOEpb295u0WhNe3tkpjRzRZBk+55+bDbq/fVca+YfR0ha9DbOO9su+ArQhl3UhfVGCdfd86b4f+O8LmizW7CSyOHr8ANBK9LATlvb3m7SwX/e1esFRes5hgblW2o/OkH76QP11GFJOgdswD2i/7CtAG9LDRnpqBetv3e+/vdGes99xL++tfRznswFkBQEbsevuqp1m+Z3J10OK9gNgwtjk6wEAzSJgI3aHt8aXV1APOM6ecd9T8eUFAElhpTPE6s+uGXwddo7a9UYf/na90omT0pi50vFnpNGjotdnw1ei1WfFUumbm6LnCwCtRg8bsbrjS95zUDDef3jw9ZyZQ/cH9ZxLQTooWAcdd91i7/lXB/33l+q5dqX/fgBoFwRstNS0hYOvd6yvDLRhw9wfvsp7nnBpcJrqvMrfn7uovnoCQLshYCM2zZ5Xfv1w8L5XXvOejx4PThO2LwpmjANoZwRstNTCOcH7pi4M3hdFWO970SXN5Q0AaSNgIxEnd/pvf2xda+tR8sha/+3vPNvaegBAowjYiMXkCZXvzxrhDTGfVbY0aZQh542PNFb+w9trpykvf9RI7/3IqiVKJ45rrHwASBpLkyYs7e83aaVlEcOC8ekzUudsBaarnlFenab8eEk68uTQwForj/I0/dukse8Lru+QvArShnlF+2VfAdqQpUnRHjqGNXf88Isr33fPby6/sGANAO2KgI2WirJYypJVle9r/bj+3NfiKRcA2lnsAdvMRprZC2b2kpm9bGZfjbsM5Nv9dS5tumFLMvUAgHaSRA/7t5Iudc7NlHSBpE+b2cU1jkHG3bQmetpW93brKa+ezwEArRR7wHaetwbedg488j1jAFpzU7z5feH2aOnivutX3J8DAOKSyDlsMxtmZi9KOizph86556v2LzOzXjNjbamCWrQifP+3H/Set+/237/lGe856L7aJVdWrRF+7eW16wYA7SjRy7rMbJykhyR90Tn304A0ue59F+ByBEm1r7GecYW070DlttIxQUPWte7oFbY/KO8o14JzWVe+0H7ZV4A2TP+yLudcv6Rtkj6dZDlofz++Z+i2BcvDj+kKWWpUksZ/Inz/itXh+wEgS5KYJd490LOWmZ0lab6kf427HLSXiZ8M3z9l0tBtj9dYFvRYjZt59J8I37+ugftbh61HDgBp6kggz/dLutfMhsn7QfCAc+7RBMpBG3nz140dl9SM8atubuy4Zu/4BQBJiT1gO+f2SPq9uPMF6vH9bWnXAADixUpnaJnJXemWP/u8dMsHgGZw84+Epf39Jq16hmqtWdiNDoF/7ENewN93QPrF/sbyaLRuRWvDvKH9sq8AbRhplngS57CBQGGXYi2c09z9si+7Qdr6XHC5AJBlBGzEauVaafWN4Wn6t0nj5nmvD22VJlUNlV93q3RvHdMU58yUdqyXnrh7cNu+A96135J0MMLa5F+MecU0AIgbQ+IJS/v7TZrfcFzUxUlK6TZvlZauCk9fj+9+XVp62dByatUnSBHbME9ov+wrQBtGGhInYCcs7e83aX7/WUwcJx15MsKxEc9nL54rXb9YmjdLOnZC+ske6bYN0s/21j42SrCecGn45VxFbMM8of2yrwBtyDlspKOvv/Fjt6zxAnSQ8WOkGVOkqxdUbt/xonTJ5xsrk2uvAWQBPeyEpf39Ji3s133UoejODund54Zuj6q6nM7Z0ukzzQ+Fv5d/gdswD2i/7CtAG9LDRrqinj8uBetGL/kqP+7MC9Kp56Pl1er7cgNAM1g4BYlackvtNNYTHDxVmMDUAAAgAElEQVRvXSYde9oL/KXHyZ3edj/DLooWiP/4y7XTAEA7YUg8YWl/v0mLMhwX1MuuDqxXzpMeuqvxuixd5c04b6TsMLRhttF+2VeANmSWeDtI+/tNWtT/LN7eIY0aWXVsj9T3lDRhbOX20XOlt05Gr0PXGOnNH1Vu+8ZG6Za7hwbsJbdI9/8wet4SbZh1tF/2FaANOYeN9nH2x73n6gDaMUyafoX06oHG8z56vLLH/MtHh/a0Jc5ZA8g2zmGjpcqDpuuVHt7eXLD2c+4i77rt8h8HBGsAWceQeMLS/n6T1uhw3PjR0tGnY66Mj+75zV0XLtGGWUf7ZV8B2jDSkDg9bKTi2Amv17tidTL5L79z4Bx5k8EaANoFPeyEpf39Ji3OX/dx3FEriaFv2jDbaL/sK0Ab0sNGtpSux7aewbt5lVu5dui2cy6rPA4A8ooedsLS/n6Txq/77Mt7G9J+2VeANqSHDQBAXhCwAQDIAAI2AAAZkPpKZ7NmzVJvbwzTg9tU3s8v5f3ckkQbZh3tl315b8Oo6GEDAJABqfewY7Mrhl9gs/L/SxUAkE3Z7mEfutML1HEEa2kwr0MJLb8FAECDshmwT73pBdb9X04m//03e/mfOpRM/gAA1Cl7Q+Jx9aaj2HOO98xQOQAgZdnqYbcyWLdDuQAADMhGwN49Iv2gucuko5vTrQMAoLDaP2DvMsm923Q2N9wRQ132LU3/hwMAoJDa+xz27pFNZ1F+B6e/fsB7bvo2jrtHSBf+tslMAACIrr172K52UOyeL933A/99QbdbbPo2jDH0+AEAqEf7BuwaQ8+l+x/39Uuf/cvmg3D5PZWtRzrvT5qrHwAAcWrPgF0jGH7rfv/tjQZtv+Ne3hvhQII2AKBF2i9gnz5cM8nyO1tQD0X8AXC6L/F6AADQfgH7pcmxZRU0uazpSWflXuqOMTMAAPy11yzxNwavvfLr3ZYCreuNPvzteqUTJ6Uxc6Xjz0ijR0WvzoavDL4Oq48OrpXOuTF6xgAA1Km9etgH/lxScDDeXzZaPmfm0P1BPedSkA4K1kHHXbfYe/7VQf/979Xz9Zv8EwAAEJP2Ctg1TFs4+HrH+spAGzbM/eGrvOcJlwanqc6r/P25i+qrJwAAcWufgN3kjOvXQ+aqvfKa93z0eHCasH2RMGMcAJCg9gnYESycE7xv6sLgfVGE9b4XXdJc3gAANKstA/bJnf7bH1vX2nqUPLLWf/s7z7a2HgCA4mqPgH2qclbXWSO8c8hnjRjcFuVSrI2PNFb8w9trpykvf9RI7/3I4VWJTh1prAIAANTQHgF7z/t9N5/cKZ163nsd5TKu6786dNvpM5Xv+/qHprlyZe28S+X3b5Pe3hGQaM+k2hkBANCA9gjYITqGNXf88Isr33fPby6/se9r7ngAABrR9gG7XJRe9pJVle+dC0//ua/FUy4AAElKJGCb2TAz+2czezSJ/MPcv7W+9Bu2JFMPAADilFQP+0uSfh418U1romfc6t5uPeXV8zkAAKhH7AHbzKZKulzSPVGPWRPzyp5fuD1aurjv+hX35wAAoCSJHvY3JX1Z0n8PSmBmy8ys18x6jxyp/1KoRSvC93/7Qe95+27//Vue8Z6D7qtdUj17/NrLa9cNAIAkxBqwzWyRpMPOuV1h6Zxz33HO9Tjnerq7a9+ecvoHKt8/FnRZVZV5y/y3fyZiT7j6+ux7fS4bAwCgFeLuYc+RdIWZvSpps6RLzezvms30xz6D6wuWhx/TFbLUqCSN/0T4/hWrw/cDANBKsQZs59wtzrmpzrkPSloi6UfOuc/WPHBm+LD4FJ/1SB6vsSzosRo38+g/Eb5/3abw/b7O72vgIAAAamuP67A7JjZ0WFIzxq+6ucEDOyfEWg8AAEo6ksrYObdN0rak8k/S97elXQMAACq1Rw87gsld6ZY/+7x0ywcAFFv7BOxZ4WuIHqxzBbNyH/uQNP8i6XemNp7HcxtrJKhRfwAAmpHYkHgSXG/weeuFc5q7X/ZlN0hbnwsuFwCANLVXwJ56l7Q/fMZX/zZp3Dzv9aGt0qSqofLrbpXurWMF8zkzpR3rpSfuHty274A04wrvdaSe/bS/il4gAAANaJ8hcUmaXPvG1KXbW7peL1hv3ur1ukuPeoK1JO18qfL4TU94C7WUetWRzp1P+mJ9hQIAUCdzte4/mbCenh7X21s25nzqiLTH58LrKlEv6Vo8V7p+sTRvlnTshPSTPdJtG6Sf7a19bKSh8PP7Qi/nMrNoFc2otP/9tAJtmG20X/blvQ0l7XLO1Yxq7TUkLkmdtZcqDbJljRegg4wfI82YIl29oHL7jhelSz7fYKFcew0AaIH2C9iSN+N6V/gvqtIEtM4O6d2qyWL1LKjieqWPXzDYm+6cLZ0+E7F3zcxwAECLtGfAliIFbWkwWDe66ln5cWdekE49HzEvgjUAoIXaa9JZtem1F/QuTRbzc+sy6djTXm+59Di509vuZ9hFEYP19O9FSAQAQHzab9JZtYBednVgvXKe9NBdjddj6Spvxnm5wGHxOnrXeZ8skfa/n1agDbON9su+vLehMjvprNosJ+0eJbl3huzqe0qaMLZy2+i50lsno2ffNUZ680fSptu8hyR9Y6N0y90+iadvkrqWRM8cAICYtH/AlqQLByJwVW+7Y5g0/Qrp1QONZ330eGVv/ZePDu1pS+KcNQAgVe19DrtaWdB0vdLD25sL1n7OXeRdt10xHE6wBgCkLBs97HKznHTqqLRngq69XLr28gTLOv9wU9eFAwAQl2z1sEs6u7zAPW1tMvlPW+flT7AGALSJ7PWwy01a4T2kSNds18TQNwCgTWWzh+1nlht8zDw2ZPdKv874+W9UHgcAQJvKdg87SMe4IQF49d+lVBcAAGKQnx42AAA5RsAGACADCNgAAGRA6muJm1muZ3ul/f0mrQBr/NKGGUf7ZV8B2jDSWuL0sAEAyIB8zhIHADQk8C6FdYh0m2LUjR42ABTczdd4gTqOYC0N5nXT1fHkBw/nsBOW9vebNM6fZV/e25D2C1a6vXDSJv+RdPho48cXoA1zcj9sAEDs4upNR3Fo4JbFDJU3hyFxACiYVgbrdig3LwjYAFAQv3k2/aDpeqU//VS6dcgqAjYAFIDrlUYMbz6fG+5oPo/Nt6f/wyGLmHSWsLS/36TlfcKSRBtmHe0nvbNTGjmiyXJ8zj83G3R/+6408g9rpytAG7JwCgAgWrDuni/d9wP/fUGTxZqdRBZHj79I6GEnLO3vN2l5751JtGHWFb39avWCo/ScwwJzrbQfnSH99IH661BRRv7bkB42ABRZrWD9rfv9tzfac/Y77uW9tY/jfHY0BGwAyKHurtpplt+ZfD2kaD8AJoxNvh5ZR8AGgBw6vDW+vIJ6wHH2jPueii+vvGKlMwDImT+7ZvB12Dlq1xt9+Nv1SidOSmPmSsefkUaPil6fDV+JVp8VS6Vvboqeb9HQwwaAnLnjS95zUDDef3jw9ZyZQ/cH9ZxLQTooWAcdd91i7/lXB/33l+q5dqX/fngI2ABQMNMWDr7esb4y0IYNc3/4Ku95wqXBaarzKn9/7qL66olKBGwAyJFmzyu/fjh43yuvec9HjwenCdsXBTPGgxGwAaBgFs4J3jd1YfC+KMJ634suaS7voiNgA0BOndzpv/2xda2tR8kja/23v/Nsa+uRVQRsAMiJyRMq3581whtiPqtsadIoQ84bH2ms/Ie3105TXv6okd77kVVLlE4c11j5ecfSpAlL+/tNWt6XtZRow6wrUvuFBePTZ6TO2cHpqmeUV6cpP16Sjjw5NLDWyqM8Tf82aez7gutbnlcB2pClSQEAno5hzR0//OLK993zm8svLFjDHwEbAAomymIpS1ZVvq/Vyf3c1+IpF8ESCdhm9qqZ/YuZvWhmTNIHgIy5v86lTTdsSaYeGJRkD/sTzrkLoozLAwCad9Oa6Glb3dutp7x6PkeRMCQOADmx5qZ48/vC7dHSxX3Xr7g/R14kFbCdpK1mtsvMllXvNLNlZtbLcDkApGfRivD9337Qe96+23//lme856D7apdcWbVG+LWX164bhkrksi4z+4Bz7oCZTZL0Q0lfdM49E5A21/P1C3A5QtpVSBxtmG1Far9a11jPuELad6ByW+mYoCHrWnf0CtsflHeUa8G5rGuoRHrYzrkDA8+HJT0k6aIkygEARPfje4ZuW7A8/JiukKVGJWn8J8L3r1gdvh/RxR6wzexsMxtdei3pjyT9NO5yAACVJn4yfP+USUO3PV5jWdBjNW7m0X8ifP+6Bu5vHbYeeZF1JJDnZEkPDQzTdEj6rnPu8QTKAQCUefPXjR2X1Izxq25u7Lhm7/iVV7EHbOfcXkk+t0QHABTJ97elXYN84bIuACiQyV3plj/7vHTLzzJu/pGwtL/fpOV9hrFEG2ZdEduv1izsRofAP/YhL+DvOyD9Yn9jeTRStwK0YaRZ4kmcwwYAtLGwS7EWzmnuftmX3SBtfS64XDSOgA0AObNyrbT6xvA0/dukcfO814e2SpOqhsqvu1W699HoZc6ZKe1YLz1x9+C2fQe8a78l6WCEtcm/GPOKaXnDkHjC0v5+k5b34VSJNsy6orZf1MVJSuk2b5WWrgpPX4/vfl1aetnQcmrVx08B2jDSkDgBO2Fpf79Jy/t/9hJtmHVFbb+J46QjT0Y4PuL57MVzpesXS/NmScdOSD/ZI922QfrZ3trHRgnWEy4NvpyrAG3IOWwAKKq+/saP3bLGC9BBxo+RZkyRrl5QuX3Hi9Iln2+sTK69ro0edsLS/n6TlvfemUQbZl3R2y/qUHRnh/Tuc0O3R1VdTuds6fSZ5obC38s7/21IDxsAii7q+eNSsG70kq/y4868IJ16Plperb4vd5axcAoA5NySW2qnsZ7g4HnrMunY017gLz1O7vS2+xl2UbRA/Mdfrp0GgxgST1ja32/S8j6cKtGGWUf7eYJ62dWB9cp50kN3NV6fpau8GeeNlB2kAG3ILPF2kPb3m7S8/2cv0YZZR/sNenuHNGpk1fE9Ut9T0oSxldtHz5XeOhm9Hl1jpDd/VLntGxulW+4eGrCX3CLd/8PoeRegDTmHDQAYdPbHvefqANoxTJp+hfTqgcbzPnq8ssf8y0eH9rQlzlk3g3PYAFAw5UHT9UoPb28uWPs5d5F33Xb5jwOCdXMYEk9Y2t9v0vI+nCrRhllH+wUbP1o6+nSMlQnQPb+568IL0IaRhsTpYQNAQR074fV6V6xOJv/ldw6cI28iWGMQPeyEpf39Ji3vvTOJNsw62q8+cdxRK+6h7wK0IT1sAEB9StdjW8/g3bzKrVw7dNs5l1Ueh2TQw05Y2t9v0vLeO5Now6yj/bKvAG1IDxsAgLwgYAMAkAEEbAAAMiD1lc5mzZql3t4YpiW2qbyfX8r7uSWJNsw62i/78t6GUdHDBgAgAwjYAABkQOpD4gByZFcMQ5ez8j/ECzSCHjaA5hy60wvUcQRraTCvQwmtlwlkFAEbQGNOvekF1v1fTib//Td7+Z86lEz+QMYwJA6gfnH1pqPYc473zFA5Co4eNoD6tDJYt0O5QJsgYAOIZveI9IPmLpOObk63DkBKCNgAattlknu36WxuuCOGuuxbmv4PByAFnMMGEG73yKazKL/l4l8/4D03fd/l3SOkC3/bZCZAdtDDBhDO1Q6K3fOl+37gvy/o/shN3zc5hh4/kCUEbADBagw9W4/36OuXPvuXzQfhUn6lx3l/0lz9gDwhYAPwVyMYfut+/+2NBm2/417eG+FAgjYKgoANYKjTh2smWX5nC+qhiD8ATvclXg8gbQRsAEO9NDm2rIImlzU96azcS90xZga0J2aJA6j0xuC1V36921Kgdb3Rh79dr3TipDRmrnT8GWn0qOjV2fCVwddh9dHBtdI5N0bPGMgYetgAKh34c0nBwXh/2Wj5nJlD9wf1nEtBOihYBx133WLv+VcH/fe/V8/Xb/JPAOQEARtAXaYtHHy9Y31loA0b5v7wVd7zhEuD01TnVf7+3EX11RPIGwI2gEFNzrh+PWSu2iuvec9HjwenCdsXCTPGkWMEbAB1WTgneN/UhcH7ogjrfS+6pLm8gawjYAPwdXKn//bH1rW2HiWPrPXf/s6zra0HkBYCNgDPqcpZXWeN8M4hnzVicFuUS7E2PtJY8Q9vr52mvPxRI733I4dXJTp1pLEKAG2OgA3As+f9vptP7pROPe+9jnIZ1/VfHbrt9JnK9339Q9NcubJ23qXy+7dJb+8ISLRnUu2MgAwiYAOoqWNYc8cPv7jyfff85vIb+77mjgeyKJGAbWbjzOzvzexfzeznZvYHSZQDoPWi9LKXrKp871x4+s99LZ5ygTxLqoe9TtLjzrn/UdJMST9PqBwAbej+rfWl37AlmXoAeRJ7wDazMZLmSlovSc65d51zPmesALSTm9ZET9vq3m495dXzOYAsSaKHPUPSEUkbzOyfzeweMzs7gXIAxGhNzCt7fuH2aOnivutX3J8DaBdJBOwOSRdK+hvn3O9JelvSX5QnMLNlZtZrZr1HjnAJBpBFi1aE7//2g97z9t3++7c84z0H3Ve7pHr2+LWX164bkEdJBOz9kvY75wYuBNHfywvg73HOfcc51+Oc6+nu5rZ4QBZM/0Dl+8eCLquqMm+Z//bPROwJV1+ffa/PZWNAEcQesJ1zByW9ZmYfGdj0SUk/i7scAK3143uGbluwPPyYrpClRiVp/CfC969YHb4fKJKk7of9RUn3mdlwSXslXZ9QOQDiMvOI9FLwiNcUn/VIHq+xLOixGjfz6D8Rvn/dpvD9vs7va+AgoP0lErCdcy9K4qpJIEs6JjZ0WFIzxq+6ucEDOyfEWg+gXbDSGYC29P1tadcAaC8EbACRTe5Kt/zZ56VbPpAmAjaAQbPC1xA9WOcKZuU+9iFp/kXS70xtPI/nNtZIUKP+QJYlNekMQE653uDz1gvnNHe/7MtukLY+F1wuUGQEbACVpt4l7Q+f8dW/TRo3z3t9aKs0qWqo/LpbpXsfjV7knJnSjvXSE3cPbtt3QJpxhfc6Us9+2l9FLxDIIIbEAVSaXPvG1KXbW7peL1hv3ur1ukuPeoK1JO18qfL4TU94C7WUetWRzp1P+mJ9hQIZY67Wfe8S1tPT43p78zvWZWZpVyFRaf/7aYVCtuGpI9Ienwuvq0S9pGvxXOn6xdK8WdKxE9JP9ki3bZB+tjdC/aL893B+X+DlXIVsv5zJextK2uWcq/nXxJA4gKE6G18yeMsaL0AHGT9GmjFFunpB5fYdL0qXfL7BQrn2GgVAwAbgb5aTdoX3bEoT0Do7pHerJovVs6CK65U+fsFgb7pztnT6TMTeNTPDURAEbADBIgRtaTBYN7rqWflxZ16QTj0fMS+CNQqESWcAwk2vvaB3abKYn1uXScee9nrLpcfJnd52P8Muihisp38vQiIgP5h0lrC8T5ZI+99PK9CGCuxlVwfWK+dJD93VeF2WrvJmnJcLHBaP2Lum/bIv720oJp0BiM0sJ+0eJbl3huzqe0qaMLZy2+i50lsno2ffNUZ680fSptu8hyR9Y6N0y90+iadvkrqWRM8cyAkCNoBoLhyIwFW97Y5h0vQrpFcPNJ710eOVvfVfPjq0py2Jc9YoNM5hA6hPWdB0vdLD25sL1n7OXeRdt10xHE6wRsHRwwZQv1lOOnVU2jNB114uXXt5gmWdf7ip68KBvKCHDaAxnV1e4J62Npn8p63z8idYA5LoYQNo1qQV3kOKdM12TQx9A77oYQOIzyw3+Jh5bMjulX6d8fPfqDwOgC962ACS0TFuSABe/Xcp1QXIAXrYAABkAAEbAIAMIGADAJABqa8lbma5nmWS9vebtAKs8UsbZhztl30FaMNIa4nTwwYAIAOYJZ4lXOMKAIVFD7vdHbrTC9RxBGtpMK9Dq+PJDwDQEpzDTljD3++pN6U9E+OtjJ/zD0qdkxs+nPNn2Zf3NqT9sq8Abcj9sDMrrt50FHvO8Z4ZKgeAtsaQeLtpZbBuh3IBAJEQsNvF7hHpB81dJh3dnG4dAAC+CNjtYJdJ7t2ms7nhjhjqsm9p+j8cAABDMOksYTW/390jJffbpsown6kKrrepLCUbLl1Yu15MeMm+vLch7Zd9BWhDFk7JhAjBunu+dN8P/Pf5Beuw7ZHF0OMHAMSHHnbCQr/fGkPPUXrOYYG5VtqPzpB++kBoFWrOHufXffblvQ1pv+wrQBvSw25rNYL1t+73395oz9nvuJf3RjiQ89kA0BYI2Gk4fbhmkuV3tqAeivgD4HRf4vUAAIQjYKfhpcZXFqsWNLms6Uln5V7qjjEzAEAjWOms1d4YvPYq7By1640+/O16pRMnpTFzpePPSKNHRa/Ohq8Mvg49Z35wrXTOjdEzBgDEih52qx34c0nBwXh/2Wj5nJlD9wf1nEtBOihYBx133WLv+VcH/fe/V8/Xb/JPAABoCQJ2m5m2cPD1jvWVgTZsmPvDV3nPEy4NTlOdV/n7cxfVV08AQGsRsFupyRnXr4fMVXvlNe/56PHgNGH7ImHGOACkhoDdZhbOCd43dWHwvijCet+LLmkubwBAsgjYKTm503/7Y+taW4+SR9b6b3/n2dbWAwDgj4DdKqcqZ3WdNcI7h3zWiMFtUS7F2vhIY8U/vL12mvLyR4303o8cXpXo1JHGKgAAaApLkybsve835Pzv6TNS5+yB9D5Bu3pGeXWa8uMl6ciT0sRx9eVRnqZ/mzT2fYHVrViulGURsy/vbUj7ZV8B2pClSbOiY1hzxw+/uPJ99/zm8gsN1gCAVBCw20yUxVKWrKp8X+vH5+e+Fk+5AID0xB6wzewjZvZi2eO4ma2Iu5wiu39rfek3bEmmHgCA1ok9YDvn/s05d4Fz7gJJsySdlPRQ3OVkzU1roqdtdW+3nvLq+RwAgPgkPST+SUm/cM79MuFy2t6amFf2/MLt0dLFfdevuD8HACCapAP2Ekmbqjea2TIz6zWzOO8plSuLapxE+PaD3vP23f77tzzjPQfdV7vkypWV76+9vHbdAACtl9hlXWY2XNIBSR91zh0KSZfr+fpRLuuSpBlXSPsOVB078HMmaMi61h29wvYH5R3ptpxc1pUreW9D2i/7CtCGqV/WtUDS7rBgjUE/vmfotgXLw4/pCllqVJLGfyJ8/4rV4fsBAO0jyYC9VD7D4YU1M3yFsCmThm57vMayoMdq3Myj/0T4/nWNtM75fQ0cBABoViIB28xGSfqUpH9IIv9M6pjY0GFJzRi/6uYGD+ycEGs9AADRdCSRqXPupCT+Z29j39+Wdg0AAPVgpbM2Mrkr3fJnn5du+QCAYNz8I2FDvt8as8UbHQL/2Ie8gL/vgPSL/Y3lUXOG+KyhTcUM1ezLexvSftlXgDaMNEs8kSFxNC7sUqyFc5q7X/ZlN0hbnwsuFwDQvgjYrTb1Lml/+Iyv/m3SuHne60NbpUlVQ+XX3Srd+2j0IufMlHasl564e3DbvgPetd+SdDDK2uTT/ip6gQCA2DEknjDf77fGsLjk9bJLvd7NW6Wlq8LT1+O7X5eWXja0nFA+w+ESw3F5kPc2pP2yrwBtGGlInICdMN/v99QRaY/PhddVop7PXjxXun6xNG+WdOyE9JM90m0bpJ/tjVC/KMH6/L7Ay7n4zyL78t6GtF/2FaANOYfdtjq7Gz50yxovQAcZP0aaMUW6ekHl9h0vSpd8vsFCufYaAFJHDzthod9vxKHxzg7p3eeGbo9ch6pedOds6fSZ5obC36sHv+4zL+9tSPtlXwHakB5225vlIgXtUrBu9JKv8uPOvCCdej5iXjWCNQCgdVg4JW3Tay/obT3BAfbWZdKxp73eculxcqe33c+wiyIG6+nfi5AIANAqDIknLNL3G9DLrg6sV86THrqr8bosXeXNOC8XOCwesXfNcFz25b0Nab/sK0AbMku8HUT+fnePktw7FZusR+p7SpowtjLp6LnSWyej16FrjPTmjyq3fWOjdMvdPgF7+iapa0nkvPnPIvvy3oa0X/YVoA05h50pFw5E4KredscwafoV0qsHGs/66PHK3vovHx3a05bEOWsAaGOcw243ZUHT9UoPb28uWPs5d5F33XZF75pgDQBtjSHxhDX8/Z46Ku1pwfXP5x9u6rpwhuOyL+9tSPtlXwHaMNKQOD3sdtXZ5fV6p61NJv9p67z8mwjWAIDWoYedsFi/3wjXbNcU89A3v+6zL+9tSPtlXwHakB527sxyg4+Zx4bsXunXGT//jcrjAACZRA87YWl/v0nj13325b0Nab/sK0Ab0sMGACAvCNgAAGQAARsAgAxoh5XO+iT9soXlTRwosyVSOr/U0s+Ygry3Ie0XI9ovdi3/fAVow3OjJEp90lmrmVlvlJP7WZb3z8jnyzY+X7bl/fNJ7fsZGRIHACADCNgAAGRAEQP2d9KuQAvk/TPy+bKNz5dtef98Upt+xsKdwwYAIIuK2MMGACBzCNgAAGRAoQK2mX3azP7NzF4xs79Iuz5xMrO/NbPDZvbTtOuSBDObZmZPm9nPzexlM/tS2nWKm5mNNLMXzOylgc/41bTrFDczG2Zm/2xmj6ZdlySY2atm9i9m9qKZ9aZdn7iZ2Tgz+3sz+9eBv8U/SLtOcTGzjwy0W+lx3MxWpF2vcoU5h21mwyT9f5I+JWm/pH+StNQ597NUKxYTM5sr6S1J/9U5d17a9Ymbmb1f0vudc7vNbLSkXZKuzEv7SZJ5q0Oc7Zx7y8w6Je2Q9CXn3HMpVy02ZnaTpB5JY5xzi9KuT9zM7FVJPc65XC6cYmb3Svqxc+4eMxsuaZRzrj/tesVtIF68Lmm2c66VC3uFKlIP+yJJrzjn9jrn3pW0WdJnUq5TbJxzz0g6mnY9kuKce8M5t3vg9QlJP5c0Jd1axct53hp42znwyF6IdlEAAAJeSURBVM0vajObKulySfekXRfUz8zGSJorab0kOefezWOwHvBJSb9op2AtFStgT5H0Wtn7/crZf/hFYWYflPR7kp5PtybxGxgyflHSYUk/dM7l6TN+U9KXJf33tCuSICdpq5ntMrNlaVcmZjMkHZG0YeC0xj1mdnbalUrIEkmb0q5EtSIFbL/FaHPTeykKM3ufpAclrXDOHU+7PnFzzp1xzl0gaaqki8wsF6c3zGyRpMPOuV1p1yVhc5xzF0paIOk/DpyqyosOSRdK+hvn3O9JeltSruYCSdLAUP8Vkr6Xdl2qFSlg75c0rez9VEkHUqoLGjBwXvdBSfc55/4h7fokaWCocZukT6dclbjMkXTFwDnezZIuNbO/S7dK8XPOHRh4PizpIXmn4vJiv6T9ZaM+fy8vgOfNAkm7nXOH0q5ItSIF7H+S9GEzmz7wC2qJpC0p1wkRDUzIWi/p5865NWnXJwlm1m1m4wZenyVpvqR/TbdW8XDO3eKcm+qc+6C8v70fOec+m3K1YmVmZw9MiNTAUPEfScrNVRvOuYOSXjOzjwxs+qSk3Ez6LLNUbTgcLrXH7TVbwjl32sxukPSEpGGS/tY593LK1YqNmW2SNE/SRDPbL+krzrn16dYqVnMkXSPpXwbO8UrSKufcP6ZYp7i9X9K9AzNU/52kB5xzubz8KacmS3po4FaQHZK+65x7PN0qxe6Lku4b6PTslXR9yvWJlZmNkncl0X9Iuy5+CnNZFwAAWVakIXEAADKLgA0AQAYQsAEAyAACNgAAGUDABgAgAwjYAABkAAEbAIAM+P8BYrfnP4SxJKkAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHwCAYAAADjD7WGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df7QU5Z3v+8932Aii/BDYYASuwSQrZ80xYqRHM4NyiSEhIBi9Z24GrjFHc3M5N/cYQsTJjKyVFZN1EnNUIE7MnZwcHfCcUdGMY0SdKIkKBow4DaPMaGbuctRERH5sYQd0mwjMc/+o3dndvetXd1V1d1W/X2v1qu6qp576Ns/efPfz1FNV5pwTAADI3u+1OwAAALoFSRcAgBYh6QIA0CIkXQAAWoSkCwBAi5B0AQBoEZIuAAAtQtIFAKBFSLpAC5jZe83s78zssJntM7PbzKwnpPwEM/vLwbIDZvaPZvYfWxkzgPSRdIHW+H8lHZD0HknnSvpfJf0/fgXN7CRJP5V0pqQ/lDRe0p9KusnMVrQkWgCZIOkCrTFT0n3Oud845/ZJelTSvw8oe6Wk/0XS/+6ce8U5d8w596ikFZL+i5mdKklm5szs/ZWdzGyDmf2Xqs+Lzew5M+s3s6fN7JyqbWeY2f1mdtDMXqlO5mZ2g5ndZ2b/w8yOmtkLZlaq2v5nZvb64LZ/MbOPpfNPBBQfSRdoje9IWmpmY8xsmqSF8hKvn49L+rFz7u269fdLGiOv9xvKzD4s6a8k/SdJkyT9N0mbzGyUmf2epIckPS9pmqSPSVppZguqqrhU0kZJEyRtknTbYL0flHSNpD9wzo2VtEDSq1HxAPCQdIHWeEpez/aIpD2SypJ+FFB2sqQ36lc6545L6pPUG+N4yyX9N+fcDufcCefcnZJ+K+kjkv5AUq9z7hvOuXedcy9L+u+Sllbtv80593fOuROS/qekWYPrT0gaJen3zWykc+5V59y/xogHgEi6QOYGe5aPSvpbSafIS6qnSfqvAbv0yTv3W19Pz+C+fTEOe6akVYNDy/1m1i9phqQzBredUbdttaSpVfvvq3o/IGm0mfU4516StFLSDZIOmNlGMzsjRjwARNIFWmGivHO0tznnfuuce1PSekmLAsr/VNJCMzulbv1/kPSupB2DnwfkDTdXnF71/jVJ33TOTah6jXHO3TO47ZW6bWOdc0Hx1HDO3e2cu1Be8nYK/uMBQB2SLpAx51yfpFckfcHMesxsgqT/KGl3wC7/U94Q9A8HLzUaOXi+9S8k3eyc+/Vgueck/R9mNsLMPilvRnTFf5f0f5vZBeY5xcwuMbOxkp6VdHRwQtTJg/ufbWZ/EPVdzOyDZnaxmY2S9BtJ70j6t4b/UYAuRdIFWuN/k/RJSQclvSTpmKQv+xV0zv1W0nx5PdId8hLbo/ImY329quiXJC2R1C/pClWdI3bOlSX9X/ImQB0ePOZVg9tOSFos79KlV+QNV98u79KkKKMkfXtwn32Spki6PsZ+ACSZc67dMQAIYWYjJf1Y0uuSrnL80gK5RU8X6HDOuWPyzuf+q6QPtjkcAAnQ0wUAoEXo6QIA0CKBN1xPYvLkye69731vFlV3hJ07d7Y7hEzNnj273SFkjjbMN9ov/4rehs4581ufyfByqVRy5XI59Xo7hZnvv2VhdMMph7Ta0KXwYz50V+P0FL0N+R3Mvy5oQ98vyPAy0KDrrvSSbRoJVxqq69or0qkPQOeip9uELvgLrd0hZK6ZNpw4TnrziQyCqTP1E9KBQ8nqKHob8juYf13Qhr5fMJNzukDRpNWrjWP/Zm+ZxbAzgPZieBmI0MqE2wnHBZAdki4Q4DdPtz/xubL0Jx9vbwwA0kPSBXy4sjTqpOT1XHNT8jo23tj+5A8gHZzTBeq8sz15HdXnY793n7dMmjh/87Q0+o+S1QGgvejpAnVGj4ou0ztfuuvH/tuCJkAlnRiVRs8bQHuRdIEqUb1RK3mvvn7pM19Nnkgr9VVeZ386WXwAOhtJFxgUldC+e6//+mYTr99+L7wcvR+JF8gvki4gqXdidJkVN2cfhxQviU+K87h5AB2HpAtIOrA5vbqCeqJp9lD7Hk+vLgCtw+xldL0/vXLovV8vs5IsXTn+ULIrS0cHpHFzpSNPSWPHxI9n/dfixbNymfSde+LXC6D96Omi6930JW8ZlFD3HBh6P2fW8O1BPdhKog1KuEH7XbXEW/5qn//2SpzrVvlvB9C5SLpAhBmLht5vu6M2WYYNGX/gcm856eLgMvV1VX8+c3FjcQLofCRddLWk51lfPxC87aXXvOWhI8FlwrbFwUxmIF9IukCERXOCt01fFLwtjrBe8OKLktUNoPOQdIFBAwG3f3zk1tbGUfHQOv/17zzd2jgApIeki641dVLt55NHecO1J1fdBjLO8O2Gh5o7/oNbo8tUH3/MaO/z6LrbQU6e0NzxAbQeSRdda99j/usHtkvHdnjv41widPXXh687fqL2c1//8DKXxZh9XDl+/xbp7W3+ZQ7+NLoeAJ2BpAv46BmRbP+TPlL7uXd+svrGn5psfwCdgaQLRIjT2126uvazc+HlP/eNdI4LIF9IukAK7m3wNpLrN2UTB4DOFivpmtknzexfzOwlM/vzrIMCWuHatfHLtrrX2cjxGvkeANorMuma2QhJ35O0UNLvS1pmZr+fdWBA1tZem259X7gxXrm0n1aU9vcAkJ04Pd3zJb3knHvZOfeupI2SPpVtWEDnWbwyfPv37/eWW3f5b9/0lLcMei5vRf2s5s9eEh0bgHyIk3SnSXqt6vOewXU1zGy5mZXNrHzw4MG04gPaZuYZtZ8fCbhkp9685f7rPxWzR1p//e6dPpckAcin1CZSOed+4JwrOedKvb29aVULtM3Pbh++buGK8H0mhtzWUZJO+2j49pVrwrcDyLc4Sfd1STOqPk8fXAfk2uSPhW+fNmX4ukcjbsF4OOIBBv1Hw7ff2sTzccPu3wygs8RJun8v6QNmNtPMTpK0VBIXPCD33vx1c/tlNZP58uua2y/pk4oAtE5PVAHn3HEzu0bSY5JGSPor59wLmUcGdJkfbWl3BACyFpl0Jck593eS/i7jWICOM3WitP9Q+45/wdntOzaA9HFHKnS1qKHifQ3eaarah94vzT9fet/05ut4ZkP4dm4VCeRLrJ4u0M1cOTi5LZqT7Hm7C66RNj8TfFwAxULSRddbtU5a8+XwMv1bpAnzvPf7N0tTJtZuv+oG6c6H4x9zzixp2x3SY7cNrXtlr3TWpd77OD3sL6Z8ZysA2TMX9TiUJpRKJVcuF/fPdDNrdwiZyuJnotPUt2GcXqWVhspt3CwtWx1evhF3f1NatmD4caLiCVL0NuR3MP+6oA19vyBJtwld8MPS7hAyV9+GkyfEexh83HOoS+ZKVy+R5s2WDh+Vfr5b+tZ66cWXo/eNk3AnXRx+qVDR25Dfwfzrgjb0/YIMLwOS+vqb33fTWi/JBjltnHTWNOmKhbXrtz0nXfT55o7JtblAPpF0gUFxhnUrk6pG9kjv1k2AamQmsStLF547dLyRF0jHTyQfVgbQ2Ui6QJW451MrCbfZBFi934lnpWM74tVFwgXyjet0gTpLr48uY6XgBHjDcunwk17yrrwGtnvr/Yw4P14y/eOvRJcB0NmYSNWELpgA0O4QMhfVhkG93frkeNk86YFbmo9j2WpvJnQzxw5T9DbkdzD/uqANmb2cli74YWl3CJmL04Zvb5PGjK7bryT1PS5NGl+7fuxc6a2B+MefOE5684nadd/eIF1/2/Cku/R66d6fxK9bKn4b8juYf13QhsxeBhpxyoXesj4J9oyQZl4qvbq3+boPHantuf7y4eE9XolzuEDRcE4XiFCd+FxZenBrsoTr58zF3nW91QmehAsUD8PLTeiCYZF2h5C5ZtrwtLHSoSczCKZO7/xk1w1LxW9Dfgfzrwva0PcL0tMFYjp81Ot9rlyTTf0rbh48Z5ww4QLoXPR0m9AFf6G1O4TMpdWGaTwJKIth5KK3Ib+D+dcFbUhPF0hb5XpdKw09hajaqnXD152+oHY/AN2D2ctASn79ln8SXXtX62MB0Jno6QIA0CIkXQAAWoSkCwBAi5B0AQBokUwmUu3cubPQ08GLPp2/yG1XQRvmG+2Xf0Vuw1Ip+LIEZi93ihOHpecm1qxatU5a8+W6cufslUa+p3VxAQBSQ9Jtp53hf80OS7iStPuM2s+zi/vXIgAUDed0W23/zV6yjUi4sVXq2p/RvQkBAKkh6bbKsTe95LjnK9nUv+c6r/5j+7OpHwCQGMPLrZBWrzaO3ad7S4adAaDj0NPNWisTbiccFwAQiKSblV2j2p/4dpp0aGN7YwAA/A5JNws7TXLvJq7mmptSiOWVZe1P/gAASZzTTd+u0YmrqH5Szffu85aJn9u6a5R03m8TVgIASIKebtpcdGLrnS/d9WP/bUHPV0383NUUet4AgGRIummKGMatPLS8r1/6zFeTJ9LqB6FbSTr708niAwBki6SbloiE9t17/dc3m3j99nvh5Rg7kngBoG1Iumk4fiCyyIqbWxCHYibx432ZxwEAGI6km4bnp6ZWVdCEqcQTqao935tiZQCAuJi9nNQbQ9f1+PUyK8nSleMPJbuydHRAGjdXOvKUNHZM/HDWf23ofVg82rdOOt3viQoAgKzQ001q759JCk6oe6pGnufMGr49qAdbSbRBCTdov6uWeMtf7fPf/rs4X7/WvwAAIDMk3YzNWDT0ftsdtckybMj4A5d7y0kXB5epr6v685mLG4sTAJA9km4SCWcCvx4y/+ql17zloSPBZcK2xcJMZgBoKZJuxhbNCd42fVHwtjjCesGLL0pWNwAgfSTdlAxs91//yK2tjaPioXX+6995urVxAACGkHSbdax2ptLJo7xzqiePGloX5zKfDQ81d/gHt0aXqT7+mNHe59En1RU6drC5AAAADSPpNmv3e3xXD2yXju3w3se5ROjqrw9fd/xE7ee+/uFlLlsVXXfl+P1bpLe3BRTaPSW6IgBAKki6GegZkWz/kz5S+7l3frL6xp+abH8AQDpIuhmL09tdurr2s3Ph5T/3jXSOCwBorcika2Z/ZWYHzOyfWhFQN7p3c2Pl12/KJg4AQLbi9HQ3SPpkxnHkzrVr45dtda+zkeM18j0AAMlEJl3n3FOSDrUgllxZm/JdFL9wY7xyaT+tKO3vAQAIxjndFlm8Mnz79+/3llt3+W/f9JS3DHoub0X9rObPXhIdGwCgNVJLuma23MzKZpbmQ+hya+YZtZ8fCbpkp8685f7rPxWzR1p//e6dPpckAQDaI7Wk65z7gXOu5Jxj3qykn90+fN3CFeH7TAy5raMknfbR8O0r14RvBwC0F8PLzZoVfienaT73nHg04haMhyMeYNB/NHz7rfeEb/d1Tl8TOwEAmhHnkqF7JP1c0gfNbI+Z/Z/Zh5UDPZOb2i2rmcyXX9fkjiMnpRoHACBYT1QB59yyVgSCZH60pd0RAACiMLycoakT23v8C85u7/EBALVIuknMDr9f474G7zRV7UPvl+afL71vevN1PLMhokBE/ACAdEUOLyMZVw4+j7toTrLn7S64Rtr8TPBxAQCdhaSb1PRbpD3hs5j6t0gT5nnv92+WptQNO191g3Tnw/EPOWeWtO0O6bHbhta9slc661Lvfawe9oy/iH9AAEAqzEU90qaZSs0KPW457N9sp0XuY6Wh3ufGzdKy1eHlG3H3N6VlC4YfJ1TI0LJZ9PfJuyx+7jtJ0duQ9su/IrdhqVRSuVz2bUSSbhOG/ZsdOxjrYfBxLxdaMle6eok0b7Z0+Kj0893St9ZLL74cI7Y4CfecvtBLhfiFz7+ityHtl39FbsOwpMvwchpG9ja966a1XpINcto46axp0hULa9dve0666PNNHpRrcwGgLUi6aZntIoeZK5OqRvZI79ZNgGrkphmuLF147lCvduQF0vETyYeVAQDZIummKUbilYYSbrN3p6re78Sz0rEdMesi4QJAW3GdbtpmRt8A2UrBSfKG5dLhJ71ea+U1sN1b72fE+TET7swfxigEAMgSE6maEPlvFtDbrU+Ol82THril+TiWrfZmQtfEFjTE3EAvl0kc+Vf0NqT98q/Ibcjs5ZTF+jfbNUZy79SsspLU97g0aXxt0bFzpbcG4h9/4jjpzSdq1317g3T9bT5Jd+Y90sSl8SsXv/BFUPQ2pP3yr8htyOzldjhvMIvW9Xp7RkgzL5Ve3dt81YeO1Paaf/nw8B6vJM7hAkCH4Zxu1qoSnytLD25NlnD9nLnYu663ppdLwgWAjsPwchOa+jc7dkja3YLrY885kOi6YYmhrSIoehvSfvlX5DYMG16mp9sqIyd6vc8Z67Kpf8atXv0JEy4AIDuc0221KSu9lxTrmt5IDCMDQG7Q022n2W7oNevwsM2r/DrF57xRux8AIDfo6XaKngnDkuiav25TLACATNDTBQCgRUi6AAC0CEkXAIAWyeSc7uzZs1Uux3nOXD4V/Rq6Il8/V0Eb5hvtl39Fb8Mg9HQBAGgRZi8DAHIr8MlqDWj22ebNoKcLAMiV664cet54Gip1XXtFOvWFyeTey6VSyXFON784n5R/RW9D2i//mmlDv8eaZmHqJ6QDh5LV4Zzj0X4AgHxKq1cbx/7BR6VmMezM8DIAoKO1MuFmfVySLgCgI/3m6fYl3ApXlv7k4+nVR9IFAHQcV5ZGnZS8nmtuSl7HxhvTS/6c0wUAdJR3tievo/p87Pfu85ZJE+dvnpZG/1GyOujpAgA6yuhR0WV650t3/dh/W9AEqKQTo9LoeZN0AQAdI6o3aiXv1dcvfearyRNppb7K6+xPJ4svCkkXANARohLad+/1X99s4vXb74WXo/dLknhJugCAtuudGF1mxc3ZxyHFS+KTxjdXN0kXANB2BzanV1dQTzTNy4/6Hm9uP2YvAwDa6k+vHHrv18usJEtXjj+U7MrS0QFp3FzpyFPS2DHx41n/tXjxrFwmfeee+PVK9HQBAG1205e8ZVBC3XNg6P2cWcO3B/VgK4k2KOEG7XfVEm/5q33+2ytxrlvlvz0MSRcA0NFmLBp6v+2O2mQZNmT8gcu95aSLg8vU11X9+czFjcUZB0kXANA2Sc+zvn4geNtLr3nLQ0eCy4Rti6PR+Em6AICOtmhO8Lbpi4K3xRHWC158UbK6/ZB0AQAdYSDg9o+P3NraOCoeWue//p2nm6+TpAsAaIupk2o/nzzKG649ueo2kHGGbzc81NzxH9waXab6+GNGe59H190OcvKE+Mck6QIA2mLfY/7rB7ZLx3Z47+NcInT114evO36i9nNf//Ayl8WYfVw5fv8W6e1t/mUO/jS6ngqSLgCg4/SMSLb/SR+p/dw7P1l9409Ntn8FSRcA0NHi9HaXrq797Fx4+c99I53jNoqkCwDIvXsbvI3k+k3ZxBElMuma2Qwze9LMXjSzF8zsS60IDABQbNeujV82i15nWsdr5HvE6ekel7TKOff7kj4i6T+b2e/HPwQAAMOtvTbd+r5wY7xyaT+tqJHvEZl0nXNvOOd2Db4/KukXkqY1GxwAAM1YvDJ8+/fv95Zbd/lv3/SUtwx6Lm9F/azmz14SHVtcDZ3TNbP3SvqwpB0+25abWdnMygcPHkwnOgBA15p5Ru3nRwIu2ak3b7n/+k/F7JHWX797p88lSc2KnXTN7FRJ90ta6ZwbdrdK59wPnHMl51ypt7c3vQgBAF3pZ7cPX7dwRfg+E0Nu6yhJp300fPvKNeHbk4qVdM1spLyEe5dz7m+zDQkA0A0mfyx8+7Qpw9c9GnELxsMRDzDoPxq+/dYGn48rhd+/uV6c2csm6Q5Jv3DONTBHCwCAYG/+urn9sprJfPl1ze3XyJOK4vR050i6UtLFZvbc4Cvhcx0AAOgsP9qS/TF6ogo457ZJsuxDAQCg1tSJ0v5D7Tv+BWenWx93pAIAtE3UUPG+Bu80Ve1D75fmny+9b3rzdTyzIXx7o0PdkT1dAADayZWDk9uiOcmet7vgGmnzM8HHTRtJFwDQVqvWSWu+HF6mf4s0YZ73fv9macrE2u1X3SDd+XD8Y86ZJW27Q3rstqF1r+yVzrrUex+nh/3FJu5sZS7qUQxNKJVKrlzO4E+EDuFN6C6uLH4mOg1tmG+0X/7Vt2GcXqWVhspt3CwtWx1evhF3f1NatmD4caLiCeKc8/0hJek2gV/4/KMN8432y7/6Npw8Id7D4OOeQ10yV7p6iTRvtnT4qPTz3dK31ksvvhy9b5yEO+ni8EuFgpIuw8sAgLbr629+301rvSQb5LRx0lnTpCsW1q7f9px00eebO2Yj1+ZWI+kCADpCnGHdyqSqkT3Su3UToBqZSezK0oXnDh1v5AXS8RPJh5WjkHQBAB0j7vnUSsJtNgFW73fiWenYjnh1Jb0bFtfpAgA6ytLro8tYKTgB3rBcOvykl7wrr4Ht3no/I86Pl0z/+CvRZaIwkaoJTOLIP9ow32i//Itqw6Debn1yvGye9MAtzcexbLU3E7qZY4dh9nKK+IXPP9ow32i//IvThm9vk8aMrtuvJPU9Lk0aX7t+7FzprYH4x584Tnrzidp1394gXX/b8KS79Hrp3p/Er1ti9jIAIGdOudBb1ifBnhHSzEulV/c2X/ehI7U9118+PLzHK6X/RCPO6QIAOlp14nNl6cGtyRKunzMXe9f1Vif4LB4hyPByExjayj/aMN9ov/xrpg1PGysdejKDYOr0zk923bAUPLxMTxcAkAuHj3q9z5Vrsql/xc2D54wTJtww9HSbwF/Z+Ucb5hvtl39ptWEaTwLKYhiZni4AoHAq1+taaegpRNVWrRu+7vQFtfu1ErOXAQCF8Ou3/JPo2rtaH0sQeroAALQISRcAgBYh6QIA0CIkXQAAWiSTiVQ7d+4s9JT+ok/nL3LbVdCG+Ub75V+R27BUCp4STU8XQCwTxtY+Ks2VpWuvGL7u9EntjhToXFwyBCBQ1I0H1nx5+Lo3Hqv93OrrIIFORk8XQI3rrhzqtaahulcMdLtMbgNpZsUdrFexz0VInE8qgmba0O/5olmY+gnpwKFkddB++VfkNiyVSiqXyzxPF4C/tHq1cewffGYpw87oRgwvA12ulQm3E44LtBNJF+hSv3m6/YnPlaU/+Xh7YwBaiaQLdCFXlkadlLyea25KXsfGG9uf/IFW4Zwu0GXe2Z68jurzsd+7z1smTZy/eVoa/UfJ6gA6HT1doMuMHhVdpne+dNeP/bcFTYBKOjEqjZ430OlIukAXieqNVh7q3dcvfearyRNp9YPCrSSd/elk8QF5R9IFukRUQvvuvf7rm028fvu98HL0fiReFBlJF+gCvROjy6y4Ofs4pHhJfNL47OMA2oGkC3SBA5vTqyuoJ5pmD7Xv8fTqAjoJs5eBgvvTK4fe+/UyK8nSleMPJbuydHRAGjdXOvKUNHZM/HjWfy1ePCuXSd+5J369QB7Q0wUK7qYvecughLrnwND7ObOGbw/qwVYSbVDCDdrvqiXe8lf7/LdX4ly3yn87kGckXaDLzVg09H7bHbXJMmzI+AOXe8tJFweXqa+r+vOZixuLEygCki5QYEnPs75+IHjbS695y0NHgsuEbYuDmcwoGpIu0OUWzQneNn1R8LY4wnrBiy9KVjeQRyRdoEsMBNz+8ZFbWxtHxUPr/Ne/83Rr4wBaiaQLFNTUSbWfTx7lDdeeXHUbyDjDtxseau74D26NLlN9/DGjvc+j624HOXlCc8cHOhFJFyiofY/5rx/YLh3b4b2Pc4nQ1V8fvu74idrPff3Dy1wWY/Zx5fj9W6S3t/mXOfjT6HqAvCDpAl2oZ0Sy/U/6SO3n3vnJ6ht/arL9gbwg6QJdLk5vd+nq2s/OhZf/3DfSOS5QNJFJ18xGm9mzZva8mb1gZj6DTQCK7N4GbyO5flM2cQB5F6en+1tJFzvnZkk6V9InzewjEfsAaLNr18Yv2+peZyPHa+R7AJ0uMuk6z1uDH0cOviIGlwC029pr063vCzfGK5f204rS/h5AO8U6p2tmI8zsOUkHJP3EObfDp8xyMyubGfeQAXJo8crw7d+/31tu3eW/fdNT3jLoubwV9bOaP3tJdGxAUcRKus65E865cyVNl3S+mZ3tU+YHzrmSc47pEUAOzDyj9vMjAZfs1Ju33H/9p2L2SOuv372TWSLoIg3NXnbO9Ut6UtInswkHQKv87Pbh6xauCN9nYshtHSXptI+Gb1+5Jnw7UHRxZi/3mtmEwfcnS/q4pH/OOjAAyUz+WPj2aVOGr3s04haMhyMeYNB/NHz7rU08Hzfs/s1A3sR5iP17JN1pZiPkJen7nHMPZxsWgKTe/HVz+2U1k/ny65rbL+mTioBOEpl0nXO7JX24BbEAKLAfbWl3BED7cUcqoItNndje418wbEomUGwkXaDAooaK9zV4p6lqH3q/NP986X3Tm6/jmQ3h27lVJIomzjldAAXmysHJbdGcZM/bXXCNtPmZ4OMC3YakCxTcqnXSmi+Hl+nfIk2Y573fv1maUjfsfNUN0p0NTJ+cM0vadof02G1D617ZK511qfc+Tg/7iynf2QroBOaiHhfSTKVmhb5NZBb/Zp3EzNodQua6rQ3j9CqtNFRu42Zp2erw8o24+5vSsgXDjxMVT5Bua78iKnIblkollctl30Yk6TahyD8sEr/wRVDfhpMnxHsYfNxzqEvmSlcvkebNlg4flX6+W/rWeunFl6P3jZNwJ10cfqlQt7VfERW5DcOSLsPLQBfo629+301rvSQb5LRx0lnTpCsW1q7f9px00eebOybX5qKoSLpAl4gzrFuZVDWyR3q3bgJUIzOJXVm68Nyh4428QDp+IvmwMpB3JF2gi8Q9n1pJuM0mwOr9TjwrHdsRry4SLoqO63SBLrP0+ugyVgpOgDcslw4/6SXvymtgu7fez4jz4yXTP/5KdBkg75hI1YQiTwCQmMRRBFFtGNTbrU+Ol82THril+TiWrfZmQjdz7DDd3n5FUOQ2ZPZyyor8wyLxC18Ecdrw7W3SmNF1+5WkvselSeNr14+dK701EP/4E8dJbz5Ru+7bG6TrbweDeHgAACAASURBVBuedJdeL937k/h1S7RfERS5DZm9DGCYUy70lvVJsGeENPNS6dW9zdd96Ehtz/WXDw/v8Uqcw0X34Zwu0OWqE58rSw9uTZZw/Zy52LuutzrBk3DRjRhebkKRh0UkhraKoJk2PG2sdOjJDIKp0zs/2XXDEu1XBEVuw7DhZXq6ACR5d5aykrRyTTb1r7h58JxxwoQL5Bk93SYU+S80ib+yiyCtNkzjSUBZDCPTfvlX5DakpwugKZXrda009BSiaqvWDV93+oLa/QAMYfYygFh+/ZZ/El17V+tjAfKKni4AAC1C0gUAoEVIugAAtEgm53Rnz56tcjmFaY8dqugzC4s8q7CCNsw32i//it6GQejpAgDQIsxeBoAi25lCj3J28XverUJPFwCKZv/NXrJNI+FKQ3Xtz+h2ZV2EpAsARXHsTS857vlKNvXvuc6r/9j+bOrvAgwvA0ARpNWrjWP36d6SYeeG0dMFgLxrZcLthOPmGEkXAPJq16j2J76dJh3a2N4YcoSkCwB5tNMk927iaq65KYVYXlnW/uSfE5zTBYC82TU6cRXVD6/43n3eMvGjHHeNks77bcJKio2eLgDkjYtObL3zpbt+7L8t6JGLiR/FmELPu+hIugCQJxHDuJXnGPf1S5/5avJEWv1sZCtJZ386WXzdjqQLAHkRkdC+e6//+mYTr99+L7wcY0cSbyCSLgDkwfEDkUVW3NyCOBQziR/vyzyOPCLpAkAePD81taqCJkwlnkhV7fneFCsrDmYvA0Cne2Pouh6/XmYlWbpy/KFkV5aODkjj5kpHnpLGjokfzvqvDb0Pi0f71kmnfzl+xV2Ani4AdLq9fyYpOKHuqRp5njNr+PagHmwl0QYl3KD9rlriLX+1z3/77+J8/Vr/Al2MpAsAOTdj0dD7bXfUJsuwIeMPXO4tJ10cXKa+rurPZy5uLE6QdAGgsyWcCfx6yPyrl17zloeOBJcJ2xYLM5lrkHQBIOcWzQneNn1R8LY4wnrBiy9KVnc3IukCQE4MbPdf/8itrY2j4qF1/uvfebq1ceQJSRcAOtWx2plKJ4/yzqmePGpoXZzLfDY81NzhH9waXab6+GNGe59Hn1RX6NjB5gIoIJIuAHSq3e/xXT2wXTq2w3sf5xKhq78+fN3xE7Wf+/qHl7lsVXTdleP3b5He3hZQaPeU6Iq6BEkXAHKoZ0Sy/U/6SO3n3vnJ6ht/arL9uwVJFwByLk5vd+nq2s/OhZf/3DfSOS5qxU66ZjbCzP7BzB7OMiAAQPru3dxY+fWbsomj2zXS0/2SpF9kFQgAoNa1a+OXbXWvs5HjNfI9ii5W0jWz6ZIukXR7tuEAACrWpnwXxS/cGK9c2k8rSvt75Fncnu53JH1F0r8FFTCz5WZWNrPywYNMDweAVlu8Mnz79+/3llt3+W/f9JS3DHoub0X9rObPXhIdGzyRSdfMFks64JzbGVbOOfcD51zJOVfq7eWRTgCQtZln1H5+JOiSnTrzlvuv/1TMHmn99bt3+lySBH9xerpzJF1qZq9K2ijpYjP760yjAgBE+pnPCb+FK8L3mRhyW0dJOu2j4dtXrgnfjnCRSdc5d71zbrpz7r2Slkp6wjn3mcwjA4BuNyv8VN00n3tOPBpxC8bDEQ8w6D8avv3We8K3+zqnr4mdionrdAGgU/VMbmq3rGYyX35dkzuOnJRqHHnW00hh59wWSVsyiQQA0NF+tKXdEeQfPV0AyLGpE9t7/AvObu/x84akCwCdbHb4/Rr3NXinqWofer80/3zpfdObr+OZDREFIuLvNg0NLwMAOo8rB5/HXTQn2fN2F1wjbX4m+LhoDEkXADrd9FukPeGzmPq3SBPmee/3b5am1A07X3WDdGcDd86fM0vadof02G1D617ZK511qfc+Vg97xl/EP2CXMBf1qIkmlEolVy4X908gM2t3CJnK4mei09CG+daV7bcz+jtbaaj3uXGztGx1ePlG3P1NadmC4ccJFTK03AVt6PsFSbpN6IIflnaHkDnaMN+6sv2OHYz1MPi4lwstmStdvUSaN1s6fFT6+W7pW+ulF1+OEV+c/97P6Qu9VKgL2tD3CzK8DAB5MLL52+tuWusl2SCnjZPOmiZdsbB2/bbnpIs+3+RBuTbXF0kXAPJitoscZq5MqhrZI71bNwGqkZtmuLJ04blDvdqRF0jHTyQfVu52JF0AyJMYiVcaSrjN3p2qer8Tz0rHdsSsi4Qbiut0ASBvZkbfANlKwUnyhuXS4Se9XmvlNbDdW+9nxPkxE+7MH8Yo1N2YSNWELpgA0O4QMkcb5hvtp8Debn1yvGye9MAtzceybLU3E7pa4BBzA73cLmhDZi+npQt+WNodQuZow3yj/QbtGiO5d2pWWUnqe1yaNL626Ni50lsD8WOYOE5684nadd/eIF1/m0/SnXmPNHFp/MrVFW3I7GUAKJTzBrNoXa+3Z4Q081Lp1b3NV33oSG2v+ZcPD+/xSuIcboM4pwsAeVeV+FxZenBrsoTr58zF3nW9Nb1cEm7DGF5uQhcMi7Q7hMzRhvlG+wU4dkja3YLrY885kOi6Yakr2tD3C9LTBYCiGDnR633OWJdN/TNu9epPmHC7Ged0AaBopqz0XlKsa3ojMYycGnq6AFBks93Qa9bhYZtX+XWKz3mjdj+khp4uAHSLngnDkuiav25TLF2Kni4AAC1C0gUAoEVIugAAtEgm53R37txZ6GuwuAYy/2jDfKP98q/IbVgqBT8dgp4uAAAt0rGzl2M9KDlCs8+RBAAgCx3V073uyqFnO6ahUte1V6RTHwAASWRy72Uza6hSv0dIZWHqJ6QDh5LXU+RzERLnk4qg6G1I++VfkduwVCqpXC535qP90urVxrF/8LFUDDsDANqhrcPLrUy4nXBcAEB3a0vS/c3T7U98riz9ycfbGwMAoLu0POm6sjTqpOT1XHNT8jo23tj+5A8A6B4tPaf7zvbkdVSfj/3efd4yaeL8zdPS6D9KVgcAAFFa2tMdPSq6TO986a4f+28LmgCVdGJUGj1vAACitCzpRvVGreS9+vqlz3w1eSKt1Fd5nf3pZPEBAJBUS5JuVEL77r3+65tNvH77vfBy9H4kXgBAljJPur0To8usuDnrKDxxkvik8dnHAQDoTpkn3QOb06srqCeaZg+17/H06gIAoFqms5f/9Mqh9369zEqydOX4Q8muLB0dkMbNlY48JY0dEz+e9V+LF8/KZdJ37olfLwAAcWTa073pS94yKKHuOTD0fs6s4duDerCVRBuUcIP2u2qJt/zVPv/tlTjXrfLfDgBAEm29DeSMRUPvt91RmyzDhow/cLm3nHRxcJn6uqo/n7m4sTgBAEhDZkk36XnW1w8Eb3vpNW956EhwmbBtcTCTGQCQtrb2dBfNCd42fVHwtjjCesGLL0pWNwAAzWhJ0h0IuP3jI7e24ujDPbTOf/07T7c2DgBAd8kk6U6dVPv55FHecO3JVbeBjDN8u+Gh5o7/4NboMtXHHzPa+zy67naQkyc0d3wAAPxkknT3Pea/fmC7dGyH9z7OJUJXf334uuMnaj/39Q8vc1mM2ceV4/dvkd7e5l/m4E+j6wEAIK6Wn9PtGZFs/5M+Uvu5d36y+safmmx/AADiautEqji93aWraz87F17+c99I57gAAKQtVtI1s1fN7B/N7Dkza+nFNPc2eBvJ9ZuyiQMAgKQa6el+1Dl3rnMusp947dr4lba619nI8Rr5HgAARMlkeHnttenW94Ub45VL+2lFaX8PAEB3i5t0naTNZrbTzJb7FTCz5WZWbmb4efHK8O3fv99bbt3lv33TU94y6Lm8FfWzmj97SXRsAACkxVzUzCRJZjbNOfe6mU2R9BNJX3TOPRW4w04LrfSsS6VX9tauq1w3GzT8G/UkorDtQXXHuVbY92lEMf7N8szM2h1C5mjDfKP98q/IbVgqlVQul30bMVZP1zn3+uDygKQHJJ2fJKCf3T583cIV4ftMDLmtoySd9tHw7SvXhG8HACBrkUnXzE4xs7GV95I+IemfwvaZ/LHwOqdNGb7u0YhbMB6OeIBB/9Hw7bc28XzcsPs3AwDQqDgPsZ8q6YHB4Y4eSXc75x4N2+HNXzcXTFYzmS+/rrn9kj6pCACAapFJ1zn3siSfR8znx4+2tDsCAADaeEeqqRPbdWTPBWe39/gAgO6TWdKNGire1+Cdpqp96P3S/POl901vvo5nNoRv51aRAIC0xTmnm5mwy3wWzUn2vN0F10ibnwk+LgAArZZp0l21Tlrz5fAy/VukCfO89/s3S1Pqhp2vukG68+H4x5wzS9p2h/TYbUPrXtnrXRssxethfzHlO1sBACDFvDlGw5Xa0M0x4t6AolJu42Zp2erw8o24+5vSsgXDjxMVT5giX9QtcWF+ERS9DWm//CtyG4bdHCPzpDt5QryHwcc9h7pkrnT1EmnebOnwUennu6VvrZdefDl63zgJd9LF0ZcKFfmHReIXvgiK3oa0X/4VuQ3Dkm7m53T7+pvfd9NaL8kGOW2cdNY06YqFteu3PSdd9Pnmjsm1uQCArLRkIlWcYd3KpKqRPdK7dROgGplJ7MrShecOHW/kBdLxE+kMKwMAkETLZi/HPZ9aSbjNJsDq/U48Kx3bEa8uEi4AIGstvTnG0uujy1gpOAHesFw6/KSXvCuvge3eej8jzo+XTP/4K9FlAABIKvOJVPWCerv1yfGyedIDtzQfw7LV3kzoZo4dpcgTACQmcRRB0duQ9su/IrdhW2cv+3l7mzRmdN0+JanvcWnS+Nr1Y+dKbw3EP/bEcdKbT9Su+/YG6frbhifdpddL9/4kft0VRf5hkfiFL4KityHtl39FbsO2zl72c8qF3rI+CfaMkGZeKr26d/g+cR06Uttz/eXDw3u8EudwAQCt17YHHki1ic+VpQe3Jku4fs5c7F3XW53gSbgAgHZoy/ByvdPGSoeeTD2MYXrnJ7tuuKLIwyISQ1tFUPQ2pP3yr8htGDa83NaebsXho17vc+WabOpfcfPgOeMUEi4AAM3qiJ6unzSeBJTVMHKR/0KT+Cu7CIrehrRf/hW5DTu+p+uncr2ulYaeQlRt1brh605fULsfAACdpK3P043r12/5J9G1d7U+FgAAmtWxPV0AAIqGpAsAQIuQdAEAaJFMzunOnj1b5XIK0487VNFnFhZ5VmEFbZhvtF/+Fb0Ng9DTBQCgRUi6AAC0SC4uGUJO7Uxh+Gh28YfZAHQPerpI1/6bvWSbRsKVhuran9E9QgGghUi6SMexN73kuOcr2dS/5zqv/mP7s6kfAFqA4WUkl1avNo7dp3tLhp0B5BA9XSTTyoTbCccFgARIumjOrlHtT3w7TTq0sb0xAEADSLpo3E6T3LuJq7nmphRieWVZ+5M/AMTEOV00ZtfoxFVUPzHqe/d5y8TPT941SjrvtwkrAYBs0dNFY1x0YuudL931Y/9tQc85Tvz84xR63gCQNZIu4osYxrWS9+rrlz7z1eSJtFJf5XX2p5PFBwDtRtJFPBEJ7bv3+q9vNvH67ffCyzF2JPEC6GAkXUQ7fiCyyIqbWxCHYibx432ZxwEAzSDpItrzU1OrKmjCVOKJVNWe702xMgBID7OXEe6Noet6/HqZlWTpyvGHkl1ZOjogjZsrHXlKGjsmfjjrvzb0Piwe7Vsnnf7l+BUDQAvQ00W4vX8mKTih7qkaeZ4za/j2oB5sJdEGJdyg/a5a4i1/tc9/++/ifP1a/wIA0EYkXSQyY9HQ+2131CbLsCHjD1zuLSddHFymvq7qz2cubixOAOgEJF0ESzgT+PWQ+VcvveYtDx0JLhO2LRZmMgPoMCRdJLJoTvC26YuCt8UR1gtefFGyugGgHUi6iGVgu//6R25tbRwVD63zX//O062NAwAaQdKFv2O1M5VOHuWdUz151NC6OJf5bHioucM/uDW6TPXxx4z2Po8+qa7QsYPNBQAAGSDpwt/u9/iuHtguHdvhvY9zidDVXx++7viJ2s99/cPLXLYquu7K8fu3SG9vCyi0e0p0RQDQIiRdNKxnRLL9T/pI7efe+cnqG39qsv0BoFViJV0zm2Bmf2Nm/2xmvzCzP8w6MORDnN7u0tW1n50LL/+5b6RzXADoNHF7urdKetQ59+8kzZL0i+xCQtHcu7mx8us3ZRMHALRbZNI1s/GS5kq6Q5Kcc+8653zOwqFIrl0bv2yre52NHK+R7wEAWYvT050p6aCk9Wb2D2Z2u5mdknFcaLO1Kd9F8Qs3xiuX9tOK0v4eAJBEnKTbI+k8SX/pnPuwpLcl/Xl9ITNbbmZlMysfPMhlGt1m8crw7d+/31tu3eW/fdNT3jLoubwV9bOaP3tJdGwA0CniJN09kvY45wYvFNHfyEvCNZxzP3DOlZxzpd5eHq1WdDPPqP38SNAlO3XmLfdf/6mYPdL663fv9LkkCQA6VWTSdc7tk/SamX1wcNXHJL2YaVToeD+7ffi6hSvC95kYcltHSTrto+HbV64J3w4AnS7u83S/KOkuMztJ0suSrs4uJHSEWQdDHwY/zeeeE49G3ILxcMQDDPqPhm+/9Z7w7b7O6WtiJwDIRqyk65x7ThJXRnaTnslN7ZbVTObLr2tyx5GTUo0DAJLgjlTIhR9taXcEAJAcSRdNmzqxvce/4Oz2Hh8AGkXSRbDZ4fdr3Nfgnaaqfej90vzzpfdNb76OZzZEFIiIHwBaLe5EKsCXKwefx100J9nzdhdcI21+Jvi4AJA3JF2Em36LtCd8FlP/FmnCPO/9/s3SlLph56tukO58OP4h58yStt0hPXbb0LpX9kpnXeq9j9XDnvEX8Q8IAC1iLuqRL00olUquXC5uV8TM2h1Cpob9TOyM/r5WGup9btwsLVsdXr4Rd39TWrZg+HFCRQwtd10bFgztl39d0Ia+X5Ck24Qu+GGpXXHsYKyHwce9XGjJXOnqJdK82dLho9LPd0vfWi+9+HKM2OL8WJ3TF3mpUNe1YcHQfvnXBW3o+wUZXka0kc3f1nPTWi/JBjltnHTWNOmKhbXrtz0nXfT5Jg/KtbkAOhRJF/HMdpHDzJVJVSN7pHfrJkA1ctMMV5YuPHeoVzvyAun4iXSGlQGgnUi6iC9G4pWGEm6zd6eq3u/Es9KxHTHrIuEC6HBcp4vGzIy+AbKVgpPkDculw096vdbKa2C7t97PiPNjJtyZP4xRCADai4lUTeiCCQDhBQJ6u/XJ8bJ50gO3NB/HstXeTOia2IJ+rBrs5XZ9G+Yc7Zd/XdCGzF5OSxf8sEQX2jVGcu/UrLKS1Pe4NGl8bdGxc6W3BuIff+I46c0natd9e4N0/W0+SXfmPdLEpfErr8RKG+Ya7Zd/XdCGzF5Gis4bzKJ1vd6eEdLMS6VX9zZf9aEjtb3mXz48vMcriXO4AHKHc7pIpirxubL04NZkCdfPmYu963prerkkXAA5xPByE7pgWKTxnY4dkna34PrYcw4kum64gjbMN9ov/7qgDX2/ID1dpGPkRK/3OWNdNvXPuNWrP4WECwDtwjldpGvKSu8lxbqmNxLDyAAKhJ4usjPbDb1mHR62eZVfp/icN2r3A4ACoaeL1uiZMCyJrvnrNsUCAG1CTxcAgBYh6QIA0CIkXQAAWiSTc7o7d+4s9DVYRb+GrshtV0Eb5hvtl39FbsNSKfgpLfR0AQBoEWYvA+huXE+OFqKnC6D77L/ZS7ZpJFxpqK79a9KpD4VF0gXQPY696SXHPV/Jpv4913n1H9ufTf3IPYaXAXSHtHq1cew+3Vsy7Iw69HQBFF8rE24nHBcdi6QLoLh2jWp/4ttp0qGN7Y0BHYOkC6CYdprk3k1czTU3pRDLK8van/zRETinC6B4do1OXIVV3d/ge/d5S1dOWOmuUdJ5v01YCfKMni6A4nHRia13vnTXj/23WcANhYLWx5ZCzxv5RtIFUCwRw7hW8l59/dJnvpo8kVbqq7zO/nSy+FBsJF0AxRGR0L57r//6ZhOv334vvBxjRxJv1yLpAiiG4wcii6y4uQVxKGYSP96XeRzoPCRdAMXw/NTUqgqaMJV4IlW153tTrAx5wexlAPn3xtB1PX69zEqydOX4Q8muLB0dkMbNlY48JY0dEz+c9V8beh8Wj/atk07/cvyKkXv0dAHk394/kxScUPdUjTzPmTV8e1APtpJogxJu0H5XLfGWv9rnv/13cb5+rX8BFBZJF0DhzVg09H7bHbXJMmzI+AOXe8tJFweXqa+r+vOZixuLE8VH0gWQbwlnAr8eMv/qpde85aEjwWXCtsXCTOauQtIFUHiL5gRvm74oeFscYb3gxRclqxvFQ9IFUBgD2/3XP3Jra+OoeGid//p3nm5tHOgcJF0A+XWsdqbSyaO8c6onjxpaF+cynw0PNXf4B7dGl6k+/pjR3ufRJ9UVOnawuQCQOyRdAPm1+z2+qwe2S8d2eO/jXCJ09deHrzt+ovZzX//wMpetiq67cvz+LdLb2wIK7Z4SXREKgaQLoJB6RiTb/6SP1H7unZ+svvGnJtsfxUDSBVB4cXq7S1fXfnYuvPznvpHOcdFdIpOumX3QzJ6reh0xs5WtCA4AWuXezY2VX78pmzhQbJFJ1zn3L865c51z50qaLWlA0gOZRwYAEa5dG79sq3udjRyvke+BfGt0ePljkv7VOffLLIIBgEasTfkuil+4MV65tJ9WlPb3QOdqNOkulXSP3wYzW25mZTNL8zkcAJCaxREnxr5/v7fcust/+6anvGXQc3kr6mc1f/aS6NjQHcxFzRaoFDQ7SdJeSf/eObc/omy8SnMq7r9ZXpkV/7Z0tGG+/a79Im6heNal0it76/Yd7BYEDf9GPYkobHtQ3bEeCTh76Gey6O0nFft3sFQqqVwu+zZiIz3dhZJ2RSVcAOgUP7t9+LqFK8L3mRhyW0dJOu2j4dtXrgnfju7WSNJdpoChZQBoi1nhd3Ka5nPPiUcjbsF4OOIBBv1Hw7ff2sz/kuf0NbET8ihW0jWzUyR9XNLfZhsOADSgZ3JTu2U1k/ny65rcceSkVONA5+qJU8g597YkfioAIMSPtrQ7AnQ67kgFoNCmTmzv8S84u73HR2ch6QLIt9nhs2D3NXinqWofer80/3zpfdObr+OZDREFIuJHscQaXgaAPAu7zGfRnGTP211wjbT5meDjAtVIugDyb/ot0p7wWUz9W6QJ87z3+zdLU+qGna+6Qbrz4fiHnDNL2naH9NhtQ+te2etdGyzF7GHP+Iv4B0QhxL45RkOVcnOMXOPC/Pwrehv6tl/EjTIkr7db6X1u3CwtWx1evhF3f1NatmD4cUIFDC0Xvf2kYv8Oht0cg6TbhCL/sEj8whdB0dvQt/2OHYz1MPi4lwstmStdvUSaN1s6fFT6+W7pW+ulF1+OEV+chHtOX+ClQkVvP6nYv4NhSZfhZQDFMLK36V03rfWSbJDTxklnTZOuWFi7fttz0kWfb/KgXJvblUi6AIpjtoscZq5MqhrZI71bNwGqkZtmuLJ04blDvdqRF0jHTyQbVkbxkXQBFEuMxCsNJdxm705Vvd+JZ6VjO2LWRcLtalynC6B4ZkbfANlKwUnyhuXS4Se9XmvlNbDdW+9nxPkxE+7MH8YohCJjIlUTijwBQGISRxEUvQ1jtV9Ab7c+OV42T3rgluZjWbbamwldLXCIOWYvt+jtJxX7d5DZyykr8g+LxC98ERS9DWO3364xknunZpWVpL7HpUnja4uOnSu9NRA/honjpDefqF337Q3S9bf5JN2Z90gTl8auu+jtJxX7d5DZywC603mDWbSu19szQpp5qfTqXp99Yjp0pLbX/MuHh/d4JXEOFzU4pwug+KoSnytLD25NlnD9nLnYu663ppdLwkUdhpebUORhEYmhrSIoehs23X7HDkm7W3B97DkHEl03XPT2k4r9Oxg2vExPF0D3GDnR633OWJdN/TNu9epPkHBRbJzTBdB9pqz0XlKsa3ojMYyMmOjpAuhus93Qa9bhYZtX+XWKz3mjdj8gJnq6AFDRM2FYEl3z122KBYVETxcAgBYh6QIA0CIkXQAAWiSrc7p9kn6ZUd1+Jg8esyXacA1dS79fG7T8+7W4DWm/lPE7mLqit2Grv9+ZQRsyuTlGq5lZ2TnX5AO6Oh/fL9/4fvlX9O/I92sdhpcBAGgRki4AAC1SlKT7g3YHkDG+X77x/fKv6N+R79cihTinCwBAHhSlpwsAQMcj6QIA0CK5Trpm9kkz+xcze8nM/rzd8aTNzP7KzA6Y2T+1O5YsmNkMM3vSzF40sxfM7EvtjilNZjbazJ41s+cHv9/X2x1TFsxshJn9g5k93O5Y0mZmr5rZP5rZc2ZWjt4jX8xsgpn9jZn9s5n9wsz+sN0xpcnMPjjYdpXXETNb2daY8npO18xGSPr/JH1c0h5Jfy9pmXPuxbYGliIzmyvpLUn/wzl3drvjSZuZvUfSe5xzu8xsrKSdki4rShuad/X/Kc65t8xspKRtkr7knHumzaGlysyulVSSNM45t7jd8aTJzF6VVHLOFfLGGGZ2p6SfOeduN7OTJI1xzvW3O64sDOaM1yVd4Jxr5c2bauS5p3u+pJeccy87596VtFHSp9ocU6qcc09JOtTuOLLinHvDObdr8P1RSb+QNK29UaXHed4a/Dhy8JXPv3IDmNl0SZdIur3dsaAxZjZe0lxJd0iSc+7doibcQR+T9K/tTLhSvpPuNEmvVX3eowL9h91tzOy9kj4saUd7I0nX4NDrc5IOSPqJc65Q30/SdyR9RdK/tTuQjDhJm81sp5ktb3cwKZsp6aCk9YOnB243s1PaHVSGlkq6p91B5DnpoiDM7FRJ90ta6Zw70u540uScO+GcO1fSdEnnm1lhThOY2WJJB5xzO9sdS4YudM6dJ2mhpP88eMqnKHoknSfpL51zH5b0tqTCzY2RpMGh80sl/bDd9TB9QgAAAXtJREFUseQ56b4uaUbV5+mD65Ajg+c675d0l3Pub9sdT1YGh+2elPTJdseSojmSLh0877lR0sVmVqhHvjvnXh9cHpD0gLzTWkWxR9KeqtGXv5GXhItooaRdzrn97Q4kz0n37yV9wMxmDv4Vs1TSpjbHhAYMTjS6Q9IvnHNr2x1P2sys18wmDL4/Wd6kv39ub1Tpcc5d75yb7px7r7zfvyecc59pc1ipMbNTBif4aXDY9ROSCnMlgXNun6TXzOyDg6s+JqkQkxh9LFMHDC1L2T3aL3POueNmdo2kxySNkPRXzrkX2hxWqszsHknzJE02sz2Svuacu6O9UaVqjqQrJf3j4HlPSVrtnPu7NsaUpvdIunNw1uTvSbrPOVe4y2oKbKqkBwYfQdcj6W7n3KPtDSl1X5R012DH5WVJV7c5ntQN/sH0cUn/qd2xSDm+ZAgAgLzJ8/AyAAC5QtIFAKBFSLoAALQISRcAgBYh6QIA0CIkXQAAWoSkCwBAi/z/WKZTYdgmYdwAAAAASUVORK5CYII=\n", "text/plain": [ "
    " ] }, - "metadata": {}, + "metadata": { + "needs_background": "light" + }, "output_type": "display_data" } ], @@ -1434,11 +1441,12 @@ "\n", "

    \n", "\n", - "
    def AC3(csp, queue=None, removals=None):\n",
    +       "
    def AC3(csp, queue=None, removals=None, arc_heuristic=dom_j_up):\n",
            "    """[Figure 6.3]"""\n",
            "    if queue is None:\n",
    -       "        queue = [(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]]\n",
    +       "        queue = {(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]}\n",
            "    csp.support_pruning()\n",
    +       "    queue = arc_heuristic(csp, queue)\n",
            "    while queue:\n",
            "        (Xi, Xj) = queue.pop()\n",
            "        if revise(csp, Xi, Xj, removals):\n",
    @@ -1446,7 +1454,7 @@
            "                return False\n",
            "            for Xk in csp.neighbors[Xi]:\n",
            "                if Xk != Xj:\n",
    -       "                    queue.append((Xk, Xi))\n",
    +       "                    queue.add((Xk, Xi))\n",
            "    return True\n",
            "
    \n", "\n", @@ -2156,10 +2164,12 @@ "\n", "
        def nconflicts(self, var, val, assignment):\n",
            "        """Return the number of conflicts var=val has with other variables."""\n",
    +       "\n",
            "        # Subclasses may implement this more efficiently\n",
            "        def conflict(var2):\n",
            "            return (var2 in assignment and\n",
            "                    not self.constraints(var, val, var2, assignment[var2]))\n",
    +       "\n",
            "        return count(conflict(v) for v in self.neighbors[var])\n",
            "
    \n", "\n", @@ -2318,8 +2328,8 @@ "metadata": {}, "outputs": [], "source": [ - "solve_simple = copy.deepcopy(usa)\n", - "solve_parameters = copy.deepcopy(usa)" + "solve_simple = copy.deepcopy(usa_csp)\n", + "solve_parameters = copy.deepcopy(usa_csp)" ] }, { @@ -2330,54 +2340,54 @@ { "data": { "text/plain": [ - "{'NJ': 'R',\n", - " 'DE': 'G',\n", - " 'PA': 'B',\n", - " 'MD': 'R',\n", - " 'NY': 'G',\n", - " 'WV': 'G',\n", - " 'VA': 'B',\n", - " 'OH': 'R',\n", - " 'KY': 'Y',\n", - " 'IN': 'G',\n", - " 'IL': 'R',\n", - " 'MO': 'G',\n", - " 'TN': 'R',\n", - " 'AR': 'B',\n", - " 'OK': 'R',\n", + "{'SD': 'R',\n", + " 'MN': 'G',\n", + " 'ND': 'B',\n", + " 'MT': 'G',\n", " 'IA': 'B',\n", - " 'NE': 'R',\n", - " 'MI': 'B',\n", - " 'TX': 'G',\n", - " 'NM': 'B',\n", - " 'LA': 'R',\n", - " 'KA': 'B',\n", - " 'NC': 'G',\n", - " 'GA': 'B',\n", - " 'MS': 'G',\n", - " 'AL': 'Y',\n", - " 'CO': 'G',\n", + " 'WI': 'R',\n", + " 'NE': 'G',\n", + " 'MO': 'R',\n", + " 'IL': 'G',\n", " 'WY': 'B',\n", - " 'SC': 'R',\n", - " 'FL': 'R',\n", - " 'UT': 'R',\n", - " 'ID': 'G',\n", - " 'SD': 'G',\n", - " 'MT': 'R',\n", - " 'ND': 'B',\n", - " 'DC': 'G',\n", + " 'ID': 'R',\n", + " 'KA': 'B',\n", + " 'UT': 'G',\n", " 'NV': 'B',\n", - " 'OR': 'R',\n", - " 'MN': 'R',\n", - " 'CA': 'G',\n", - " 'AZ': 'Y',\n", + " 'OK': 'G',\n", + " 'CO': 'R',\n", + " 'OR': 'G',\n", + " 'KY': 'B',\n", + " 'AZ': 'R',\n", + " 'CA': 'Y',\n", + " 'IN': 'R',\n", + " 'OH': 'G',\n", " 'WA': 'B',\n", - " 'WI': 'G',\n", - " 'CT': 'R',\n", - " 'MA': 'B',\n", - " 'VT': 'R',\n", - " 'NH': 'G',\n", - " 'RI': 'G',\n", + " 'MI': 'B',\n", + " 'AR': 'B',\n", + " 'NM': 'B',\n", + " 'TN': 'G',\n", + " 'TX': 'R',\n", + " 'MS': 'R',\n", + " 'AL': 'B',\n", + " 'VA': 'R',\n", + " 'WV': 'Y',\n", + " 'PA': 'R',\n", + " 'LA': 'G',\n", + " 'GA': 'R',\n", + " 'MD': 'G',\n", + " 'NC': 'B',\n", + " 'DC': 'B',\n", + " 'DE': 'B',\n", + " 'SC': 'G',\n", + " 'FL': 'G',\n", + " 'NJ': 'G',\n", + " 'NY': 'B',\n", + " 'MA': 'R',\n", + " 'CT': 'G',\n", + " 'RI': 'B',\n", + " 'VT': 'G',\n", + " 'NH': 'B',\n", " 'ME': 'R'}" ] }, @@ -2760,7 +2770,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "26b425b8fade4789a075632715b1afcd", + "model_id": "1882dd95ddd0465c8ec91d93a8a7224f", "version_major": 2, "version_minor": 0 }, @@ -2774,7 +2784,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "179048eb3f8e41a1afc1ec22343dece4", + "model_id": "3967e7c0226d434e8c08c7f4a59e2b2a", "version_major": 2, "version_minor": 0 }, @@ -2822,7 +2832,7 @@ " ''' Mark grid with queens that are under conflict. '''\n", " for col, row in assignment.items(): # check each queen for conflict\n", " conflicts = {temp_col:temp_row for temp_col,temp_row in assignment.items() \n", - " if (temp_row == row and temp_col != col\n", + " if (temp_row == row and temp_col != col)\n", " or (temp_row+temp_col == row+col and temp_col != col)\n", " or (temp_row-temp_col == row-col and temp_col != col)}\n", " \n", @@ -2909,7 +2919,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "fa243795d27f47c0af2cd12cbefa5e52", + "model_id": "582e8f9b8d2e4a31aa7d45de68fd5b7c", "version_major": 2, "version_minor": 0 }, @@ -2923,7 +2933,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "bdea801600cb441697ea3a810cb747a9", + "model_id": "bb0f50b970764cb4bbebeb69cd4fbd19", "version_major": 2, "version_minor": 0 }, @@ -2993,12 +3003,12 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "3bf64b599e5e4f128da23ecce08f3f53", + "model_id": "409c4961f6e04fbea5d07a01cb1797ea", "version_major": 2, "version_minor": 0 }, "text/plain": [ - "interactive(children=(IntSlider(value=0, description='iteration', max=52, step=0), Output()), _dom_classes=('w…" + "interactive(children=(IntSlider(value=0, description='iteration', max=27, step=0), Output()), _dom_classes=('w…" ] }, "metadata": {}, @@ -3007,7 +3017,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "e4ccaba569f34a78857f2de8af4f01f2", + "model_id": "a55b1b50a9a44085a484b357aa26b50f", "version_major": 2, "version_minor": 0 }, @@ -3032,6 +3042,13 @@ "a = widgets.interactive(visualize_callback, Visualize = visualize_button, time_step=time_select)\n", "display(a)" ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -3050,9 +3067,18 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.5" + "version": "3.6.8" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 1 -} +} \ No newline at end of file diff --git a/csp.py b/csp.py index d5f96f80b..46ae07dd5 100644 --- a/csp.py +++ b/csp.py @@ -1,14 +1,17 @@ -"""CSP (Constraint Satisfaction Problems) problems and solvers. (Chapter 6).""" - -from utils import argmin_random_tie, count, first -import search - -from collections import defaultdict -from functools import reduce +"""CSP (Constraint Satisfaction Problems) problems and solvers. (Chapter 6)""" import itertools -import re import random +import re +import string +from collections import defaultdict, Counter +from functools import reduce +from operator import eq, neg + +from sortedcontainers import SortedSet + +import search +from utils import argmin_random_tie, count, first, extend class CSP(search.Problem): @@ -24,9 +27,9 @@ class CSP(search.Problem): In the textbook and in most mathematical definitions, the constraints are specified as explicit pairs of allowable values, but the formulation here is easier to express and more compact for - most cases. (For example, the n-Queens problem can be represented - in O(n) space using this notation, instead of O(N^4) for the - explicit representation.) In terms of describing the CSP as a + most cases (for example, the n-Queens problem can be represented + in O(n) space using this notation, instead of O(n^4) for the + explicit representation). In terms of describing the CSP as a problem, that's all there is. However, the class also supports data structures and methods that help you @@ -50,13 +53,12 @@ class CSP(search.Problem): def __init__(self, variables, domains, neighbors, constraints): """Construct a CSP problem. If variables is empty, it becomes domains.keys().""" + super().__init__(()) variables = variables or list(domains.keys()) - self.variables = variables self.domains = domains self.neighbors = neighbors self.constraints = constraints - self.initial = () self.curr_domains = None self.nassigns = 0 @@ -74,21 +76,22 @@ def unassign(self, var, assignment): def nconflicts(self, var, val, assignment): """Return the number of conflicts var=val has with other variables.""" + # Subclasses may implement this more efficiently def conflict(var2): - return (var2 in assignment and - not self.constraints(var, val, var2, assignment[var2])) + return var2 in assignment and not self.constraints(var, val, var2, assignment[var2]) + return count(conflict(v) for v in self.neighbors[var]) def display(self, assignment): """Show a human-readable representation of the CSP.""" # Subclasses can print in a prettier way, or display with a GUI - print('CSP:', self, 'with assignment:', assignment) + print(assignment) # These methods are for the tree and graph-search interface: def actions(self, state): - """Return a list of applicable actions: nonconflicting + """Return a list of applicable actions: non conflicting assignments to an unassigned variable.""" if len(state) == len(self.variables): return [] @@ -153,35 +156,186 @@ def conflicted_vars(self, current): return [var for var in self.variables if self.nconflicts(var, current[var], current) > 0] + # ______________________________________________________________________________ -# Constraint Propagation with AC-3 +# Constraint Propagation with AC3 + + +def no_arc_heuristic(csp, queue): + return queue + +def dom_j_up(csp, queue): + return SortedSet(queue, key=lambda t: neg(len(csp.curr_domains[t[1]]))) -def AC3(csp, queue=None, removals=None): + +def AC3(csp, queue=None, removals=None, arc_heuristic=dom_j_up): """[Figure 6.3]""" if queue is None: - queue = [(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]] + queue = {(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]} csp.support_pruning() + queue = arc_heuristic(csp, queue) + checks = 0 while queue: (Xi, Xj) = queue.pop() - if revise(csp, Xi, Xj, removals): + revised, checks = revise(csp, Xi, Xj, removals, checks) + if revised: if not csp.curr_domains[Xi]: - return False + return False, checks # CSP is inconsistent for Xk in csp.neighbors[Xi]: if Xk != Xj: - queue.append((Xk, Xi)) - return True + queue.add((Xk, Xi)) + return True, checks # CSP is satisfiable -def revise(csp, Xi, Xj, removals): +def revise(csp, Xi, Xj, removals, checks=0): """Return true if we remove a value.""" revised = False for x in csp.curr_domains[Xi][:]: # If Xi=x conflicts with Xj=y for every possible y, eliminate Xi=x - if all(not csp.constraints(Xi, x, Xj, y) for y in csp.curr_domains[Xj]): + # if all(not csp.constraints(Xi, x, Xj, y) for y in csp.curr_domains[Xj]): + conflict = True + for y in csp.curr_domains[Xj]: + if csp.constraints(Xi, x, Xj, y): + conflict = False + checks += 1 + if not conflict: + break + if conflict: csp.prune(Xi, x, removals) revised = True - return revised + return revised, checks + + +# Constraint Propagation with AC3b: an improved version +# of AC3 with double-support domain-heuristic + +def AC3b(csp, queue=None, removals=None, arc_heuristic=dom_j_up): + if queue is None: + queue = {(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]} + csp.support_pruning() + queue = arc_heuristic(csp, queue) + checks = 0 + while queue: + (Xi, Xj) = queue.pop() + # Si_p values are all known to be supported by Xj + # Sj_p values are all known to be supported by Xi + # Dj - Sj_p = Sj_u values are unknown, as yet, to be supported by Xi + Si_p, Sj_p, Sj_u, checks = partition(csp, Xi, Xj, checks) + if not Si_p: + return False, checks # CSP is inconsistent + revised = False + for x in set(csp.curr_domains[Xi]) - Si_p: + csp.prune(Xi, x, removals) + revised = True + if revised: + for Xk in csp.neighbors[Xi]: + if Xk != Xj: + queue.add((Xk, Xi)) + if (Xj, Xi) in queue: + if isinstance(queue, set): + # or queue -= {(Xj, Xi)} or queue.remove((Xj, Xi)) + queue.difference_update({(Xj, Xi)}) + else: + queue.difference_update((Xj, Xi)) + # the elements in D_j which are supported by Xi are given by the union of Sj_p with the set of those + # elements of Sj_u which further processing will show to be supported by some vi_p in Si_p + for vj_p in Sj_u: + for vi_p in Si_p: + conflict = True + if csp.constraints(Xj, vj_p, Xi, vi_p): + conflict = False + Sj_p.add(vj_p) + checks += 1 + if not conflict: + break + revised = False + for x in set(csp.curr_domains[Xj]) - Sj_p: + csp.prune(Xj, x, removals) + revised = True + if revised: + for Xk in csp.neighbors[Xj]: + if Xk != Xi: + queue.add((Xk, Xj)) + return True, checks # CSP is satisfiable + + +def partition(csp, Xi, Xj, checks=0): + Si_p = set() + Sj_p = set() + Sj_u = set(csp.curr_domains[Xj]) + for vi_u in csp.curr_domains[Xi]: + conflict = True + # now, in order to establish support for a value vi_u in Di it seems better to try to find a support among + # the values in Sj_u first, because for each vj_u in Sj_u the check (vi_u, vj_u) is a double-support check + # and it is just as likely that any vj_u in Sj_u supports vi_u than it is that any vj_p in Sj_p does... + for vj_u in Sj_u - Sj_p: + # double-support check + if csp.constraints(Xi, vi_u, Xj, vj_u): + conflict = False + Si_p.add(vi_u) + Sj_p.add(vj_u) + checks += 1 + if not conflict: + break + # ... and only if no support can be found among the elements in Sj_u, should the elements vj_p in Sj_p be used + # for single-support checks (vi_u, vj_p) + if conflict: + for vj_p in Sj_p: + # single-support check + if csp.constraints(Xi, vi_u, Xj, vj_p): + conflict = False + Si_p.add(vi_u) + checks += 1 + if not conflict: + break + return Si_p, Sj_p, Sj_u - Sj_p, checks + + +# Constraint Propagation with AC4 + +def AC4(csp, queue=None, removals=None, arc_heuristic=dom_j_up): + if queue is None: + queue = {(Xi, Xk) for Xi in csp.variables for Xk in csp.neighbors[Xi]} + csp.support_pruning() + queue = arc_heuristic(csp, queue) + support_counter = Counter() + variable_value_pairs_supported = defaultdict(set) + unsupported_variable_value_pairs = [] + checks = 0 + # construction and initialization of support sets + while queue: + (Xi, Xj) = queue.pop() + revised = False + for x in csp.curr_domains[Xi][:]: + for y in csp.curr_domains[Xj]: + if csp.constraints(Xi, x, Xj, y): + support_counter[(Xi, x, Xj)] += 1 + variable_value_pairs_supported[(Xj, y)].add((Xi, x)) + checks += 1 + if support_counter[(Xi, x, Xj)] == 0: + csp.prune(Xi, x, removals) + revised = True + unsupported_variable_value_pairs.append((Xi, x)) + if revised: + if not csp.curr_domains[Xi]: + return False, checks # CSP is inconsistent + # propagation of removed values + while unsupported_variable_value_pairs: + Xj, y = unsupported_variable_value_pairs.pop() + for Xi, x in variable_value_pairs_supported[(Xj, y)]: + revised = False + if x in csp.curr_domains[Xi][:]: + support_counter[(Xi, x, Xj)] -= 1 + if support_counter[(Xi, x, Xj)] == 0: + csp.prune(Xi, x, removals) + revised = True + unsupported_variable_value_pairs.append((Xi, x)) + if revised: + if not csp.curr_domains[Xi]: + return False, checks # CSP is inconsistent + return True, checks # CSP is satisfiable + # ______________________________________________________________________________ # CSP Backtracking Search @@ -196,17 +350,16 @@ def first_unassigned_variable(assignment, csp): def mrv(assignment, csp): """Minimum-remaining-values heuristic.""" - return argmin_random_tie( - [v for v in csp.variables if v not in assignment], - key=lambda var: num_legal_values(csp, var, assignment)) + return argmin_random_tie([v for v in csp.variables if v not in assignment], + key=lambda var: num_legal_values(csp, var, assignment)) def num_legal_values(csp, var, assignment): if csp.curr_domains: return len(csp.curr_domains[var]) else: - return count(csp.nconflicts(var, val, assignment) == 0 - for val in csp.domains[var]) + return count(csp.nconflicts(var, val, assignment) == 0 for val in csp.domains[var]) + # Value ordering @@ -218,8 +371,8 @@ def unordered_domain_values(var, assignment, csp): def lcv(var, assignment, csp): """Least-constraining-values heuristic.""" - return sorted(csp.choices(var), - key=lambda val: csp.nconflicts(var, val, assignment)) + return sorted(csp.choices(var), key=lambda val: csp.nconflicts(var, val, assignment)) + # Inference @@ -241,17 +394,16 @@ def forward_checking(csp, var, value, assignment, removals): return True -def mac(csp, var, value, assignment, removals): +def mac(csp, var, value, assignment, removals, constraint_propagation=AC3b): """Maintain arc consistency.""" - return AC3(csp, [(X, var) for X in csp.neighbors[var]], removals) + return constraint_propagation(csp, {(X, var) for X in csp.neighbors[var]}, removals) + # The search, proper -def backtracking_search(csp, - select_unassigned_variable=first_unassigned_variable, - order_domain_values=unordered_domain_values, - inference=no_inference): +def backtracking_search(csp, select_unassigned_variable=first_unassigned_variable, + order_domain_values=unordered_domain_values, inference=no_inference): """[Figure 6.5]""" def backtrack(assignment): @@ -274,12 +426,13 @@ def backtrack(assignment): assert result is None or csp.goal_test(result) return result + # ______________________________________________________________________________ -# Min-conflicts hillclimbing search for CSPs +# Min-conflicts Hill Climbing search for CSPs def min_conflicts(csp, max_steps=100000): - """Solve a CSP by stochastic hillclimbing on the number of conflicts.""" + """Solve a CSP by stochastic Hill Climbing on the number of conflicts.""" # Generate a complete assignment for all variables (probably with conflicts) csp.current = current = {} for var in csp.variables: @@ -299,8 +452,8 @@ def min_conflicts(csp, max_steps=100000): def min_conflicts_value(csp, var, current): """Return the value that will give var the least number of conflicts. If there is a tie, choose at random.""" - return argmin_random_tie(csp.domains[var], - key=lambda val: csp.nconflicts(var, val, current)) + return argmin_random_tie(csp.domains[var], key=lambda val: csp.nconflicts(var, val, current)) + # ______________________________________________________________________________ @@ -356,7 +509,7 @@ def build_topological(node, parent, neighbors, visited, stack, parents): visited[node] = True for n in neighbors[node]: - if(not visited[n]): + if not visited[n]: build_topological(n, node, neighbors, visited, stack, parents) parents[node] = parent @@ -366,15 +519,15 @@ def build_topological(node, parent, neighbors, visited, stack, parents): def make_arc_consistent(Xj, Xk, csp): """Make arc between parent (Xj) and child (Xk) consistent under the csp's constraints, by removing the possible values of Xj that cause inconsistencies.""" - #csp.curr_domains[Xj] = [] + # csp.curr_domains[Xj] = [] for val1 in csp.domains[Xj]: - keep = False # Keep or remove val1 + keep = False # Keep or remove val1 for val2 in csp.domains[Xk]: if csp.constraints(Xj, val1, Xk, val2): # Found a consistent assignment for val1, keep it keep = True break - + if not keep: # Remove val1 csp.prune(Xj, val1, None) @@ -393,8 +546,9 @@ def assign_value(Xj, Xk, csp, assignment): # No consistent assignment available return None + # ______________________________________________________________________________ -# Map-Coloring Problems +# Map Coloring CSP Problems class UniversalDict: @@ -424,11 +578,10 @@ def MapColoringCSP(colors, neighbors): specified as a string of the form defined by parse_neighbors.""" if isinstance(neighbors, str): neighbors = parse_neighbors(neighbors) - return CSP(list(neighbors.keys()), UniversalDict(colors), neighbors, - different_values_constraint) + return CSP(list(neighbors.keys()), UniversalDict(colors), neighbors, different_values_constraint) -def parse_neighbors(neighbors, variables=None): +def parse_neighbors(neighbors): """Convert a string of the form 'X: Y Z; Y: Z' into a dict mapping regions to neighbors. The syntax is a region name followed by a ':' followed by zero or more region names, followed by ';', repeated for @@ -446,27 +599,27 @@ def parse_neighbors(neighbors, variables=None): return dic -australia = MapColoringCSP(list('RGB'), - 'SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: ') - -usa = MapColoringCSP(list('RGBY'), - """WA: OR ID; OR: ID NV CA; CA: NV AZ; NV: ID UT AZ; ID: MT WY UT; - UT: WY CO AZ; MT: ND SD WY; WY: SD NE CO; CO: NE KA OK NM; NM: OK TX AZ; - ND: MN SD; SD: MN IA NE; NE: IA MO KA; KA: MO OK; OK: MO AR TX; - TX: AR LA; MN: WI IA; IA: WI IL MO; MO: IL KY TN AR; AR: MS TN LA; - LA: MS; WI: MI IL; IL: IN KY; IN: OH KY; MS: TN AL; AL: TN GA FL; - MI: OH IN; OH: PA WV KY; KY: WV VA TN; TN: VA NC GA; GA: NC SC FL; - PA: NY NJ DE MD WV; WV: MD VA; VA: MD DC NC; NC: SC; NY: VT MA CT NJ; - NJ: DE; DE: MD; MD: DC; VT: NH MA; MA: NH RI CT; CT: RI; ME: NH; - HI: ; AK: """) - -france = MapColoringCSP(list('RGBY'), - """AL: LO FC; AQ: MP LI PC; AU: LI CE BO RA LR MP; BO: CE IF CA FC RA - AU; BR: NB PL; CA: IF PI LO FC BO; CE: PL NB NH IF BO AU LI PC; FC: BO - CA LO AL RA; IF: NH PI CA BO CE; LI: PC CE AU MP AQ; LO: CA AL FC; LR: - MP AU RA PA; MP: AQ LI AU LR; NB: NH CE PL BR; NH: PI IF CE NB; NO: - PI; PA: LR RA; PC: PL CE LI AQ; PI: NH NO CA IF; PL: BR NB CE PC; RA: - AU BO FC PA LR""") +australia_csp = MapColoringCSP(list('RGB'), """SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: """) + +usa_csp = MapColoringCSP(list('RGBY'), + """WA: OR ID; OR: ID NV CA; CA: NV AZ; NV: ID UT AZ; ID: MT WY UT; + UT: WY CO AZ; MT: ND SD WY; WY: SD NE CO; CO: NE KA OK NM; NM: OK TX AZ; + ND: MN SD; SD: MN IA NE; NE: IA MO KA; KA: MO OK; OK: MO AR TX; + TX: AR LA; MN: WI IA; IA: WI IL MO; MO: IL KY TN AR; AR: MS TN LA; + LA: MS; WI: MI IL; IL: IN KY; IN: OH KY; MS: TN AL; AL: TN GA FL; + MI: OH IN; OH: PA WV KY; KY: WV VA TN; TN: VA NC GA; GA: NC SC FL; + PA: NY NJ DE MD WV; WV: MD VA; VA: MD DC NC; NC: SC; NY: VT MA CT NJ; + NJ: DE; DE: MD; MD: DC; VT: NH MA; MA: NH RI CT; CT: RI; ME: NH; + HI: ; AK: """) + +france_csp = MapColoringCSP(list('RGBY'), + """AL: LO FC; AQ: MP LI PC; AU: LI CE BO RA LR MP; BO: CE IF CA FC RA + AU; BR: NB PL; CA: IF PI LO FC BO; CE: PL NB NH IF BO AU LI PC; FC: BO + CA LO AL RA; IF: NH PI CA BO CE; LI: PC CE AU MP AQ; LO: CA AL FC; LR: + MP AU RA PA; MP: AQ LI AU LR; NB: NH CE PL BR; NH: PI IF CE NB; NO: + PI; PA: LR RA; PC: PL CE LI AQ; PI: NH NO CA IF; PL: BR NB CE PC; RA: + AU BO FC PA LR""") + # ______________________________________________________________________________ # n-Queens Problem @@ -479,12 +632,13 @@ def queen_constraint(A, a, B, b): class NQueensCSP(CSP): - """Make a CSP for the nQueens problem for search with min_conflicts. + """ + Make a CSP for the nQueens problem for search with min_conflicts. Suitable for large n, it uses only data structures of size O(n). Think of placing queens one per column, from left to right. That means position (x, y) represents (var, val) in the CSP. The main structures are three arrays to count queens that could conflict: - rows[i] Number of queens in the ith row (i.e val == i) + rows[i] Number of queens in the ith row (i.e. val == i) downs[i] Number of queens in the \ diagonal such that their (x, y) coordinates sum to i ups[i] Number of queens in the / diagonal @@ -503,26 +657,26 @@ def __init__(self, n): CSP.__init__(self, list(range(n)), UniversalDict(list(range(n))), UniversalDict(list(range(n))), queen_constraint) - self.rows = [0]*n - self.ups = [0]*(2*n - 1) - self.downs = [0]*(2*n - 1) + self.rows = [0] * n + self.ups = [0] * (2 * n - 1) + self.downs = [0] * (2 * n - 1) def nconflicts(self, var, val, assignment): """The number of conflicts, as recorded with each assignment. Count conflicts in row and in up, down diagonals. If there is a queen there, it can't conflict with itself, so subtract 3.""" n = len(self.variables) - c = self.rows[val] + self.downs[var+val] + self.ups[var-val+n-1] + c = self.rows[val] + self.downs[var + val] + self.ups[var - val + n - 1] if assignment.get(var, None) == val: c -= 3 return c def assign(self, var, val, assignment): """Assign var, and keep track of conflicts.""" - oldval = assignment.get(var, None) - if val != oldval: - if oldval is not None: # Remove old val if there was one - self.record_conflict(assignment, var, oldval, -1) + old_val = assignment.get(var, None) + if val != old_val: + if old_val is not None: # Remove old val if there was one + self.record_conflict(assignment, var, old_val, -1) self.record_conflict(assignment, var, val, +1) CSP.assign(self, var, val, assignment) @@ -560,6 +714,7 @@ def display(self, assignment): print(str(self.nconflicts(var, val, assignment)) + ch, end=' ') print() + # ______________________________________________________________________________ # Sudoku @@ -585,7 +740,8 @@ def flatten(seqs): class Sudoku(CSP): - """A Sudoku problem. + """ + A Sudoku problem. The box grid is a 3x3 array of boxes, each a 3x3 array of cells. Each cell holds a digit in 1..9. In each box, all digits are different; the same for each row and column as a 9x9 grid. @@ -602,8 +758,9 @@ class Sudoku(CSP): . . 2 | 6 . 9 | 5 . . 8 . . | 2 . 3 | . . 9 . . 5 | . 1 . | 3 . . - >>> AC3(e); e.display(e.infer_assignment()) - True + >>> AC3(e) # doctest: +ELLIPSIS + (True, ...) + >>> e.display(e.infer_assignment()) 4 8 3 | 9 2 1 | 6 5 7 9 6 7 | 3 4 5 | 8 2 1 2 5 1 | 8 7 6 | 4 9 3 @@ -618,7 +775,7 @@ class Sudoku(CSP): >>> h = Sudoku(harder1) >>> backtracking_search(h, select_unassigned_variable=mrv, inference=forward_checking) is not None True - """ # noqa + """ R3 = _R3 Cell = _CELL @@ -646,9 +803,12 @@ def show_cell(cell): return str(assignment.get(cell, '.')) def abut(lines1, lines2): return list( map(' | '.join, list(zip(lines1, lines2)))) + print('\n------+-------+------\n'.join( '\n'.join(reduce( abut, map(show_box, brow))) for brow in self.bgrid)) + + # ______________________________________________________________________________ # The Zebra Puzzle @@ -670,7 +830,7 @@ def Zebra(): Spaniard: Dog; Kools: Yellow; Chesterfields: Fox; Norwegian: Blue; Winston: Snails; LuckyStrike: OJ; Ukranian: Tea; Japanese: Parliaments; Kools: Horse; - Coffee: Green; Green: Ivory""", variables) + Coffee: Green; Green: Ivory""") for type in [Colors, Pets, Drinks, Countries, Smokes]: for A in type: for B in type: @@ -716,6 +876,7 @@ def zebra_constraint(A, a, B, b, recurse=0): (A in Smokes and B in Smokes)): return not same raise Exception('error') + return CSP(variables, domains, neighbors, zebra_constraint) @@ -729,3 +890,546 @@ def solve_zebra(algorithm=min_conflicts, **args): print(var, end=' ') print() return ans['Zebra'], ans['Water'], z.nassigns, ans + + +# ______________________________________________________________________________ +# n-ary Constraint Satisfaction Problem + +class NaryCSP: + """ + A nary-CSP consists of: + domains : a dictionary that maps each variable to its domain + constraints : a list of constraints + variables : a set of variables + var_to_const: a variable to set of constraints dictionary + """ + + def __init__(self, domains, constraints): + """Domains is a variable:domain dictionary + constraints is a list of constraints + """ + self.variables = set(domains) + self.domains = domains + self.constraints = constraints + self.var_to_const = {var: set() for var in self.variables} + for con in constraints: + for var in con.scope: + self.var_to_const[var].add(con) + + def __str__(self): + """String representation of CSP""" + return str(self.domains) + + def display(self, assignment=None): + """More detailed string representation of CSP""" + if assignment is None: + assignment = {} + print(assignment) + + def consistent(self, assignment): + """assignment is a variable:value dictionary + returns True if all of the constraints that can be evaluated + evaluate to True given assignment. + """ + return all(con.holds(assignment) + for con in self.constraints + if all(v in assignment for v in con.scope)) + + +class Constraint: + """ + A Constraint consists of: + scope : a tuple of variables + condition: a function that can applied to a tuple of values + for the variables. + """ + + def __init__(self, scope, condition): + self.scope = scope + self.condition = condition + + def __repr__(self): + return self.condition.__name__ + str(self.scope) + + def holds(self, assignment): + """Returns the value of Constraint con evaluated in assignment. + + precondition: all variables are assigned in assignment + """ + return self.condition(*tuple(assignment[v] for v in self.scope)) + + +def all_diff_constraint(*values): + """Returns True if all values are different, False otherwise""" + return len(values) is len(set(values)) + + +def is_word_constraint(words): + """Returns True if the letters concatenated form a word in words, False otherwise""" + + def isw(*letters): + return "".join(letters) in words + + return isw + + +def meet_at_constraint(p1, p2): + """Returns a function that is True when the words meet at the positions (p1, p2), False otherwise""" + + def meets(w1, w2): + return w1[p1] == w2[p2] + + meets.__name__ = "meet_at(" + str(p1) + ',' + str(p2) + ')' + return meets + + +def adjacent_constraint(x, y): + """Returns True if x and y are adjacent numbers, False otherwise""" + return abs(x - y) == 1 + + +def sum_constraint(n): + """Returns a function that is True when the the sum of all values is n, False otherwise""" + + def sumv(*values): + return sum(values) is n + + sumv.__name__ = str(n) + "==sum" + return sumv + + +def is_constraint(val): + """Returns a function that is True when x is equal to val, False otherwise""" + + def isv(x): + return val == x + + isv.__name__ = str(val) + "==" + return isv + + +def ne_constraint(val): + """Returns a function that is True when x is not equal to val, False otherwise""" + + def nev(x): + return val != x + + nev.__name__ = str(val) + "!=" + return nev + + +def no_heuristic(to_do): + return to_do + + +def sat_up(to_do): + return SortedSet(to_do, key=lambda t: 1 / len([var for var in t[1].scope])) + + +class ACSolver: + """Solves a CSP with arc consistency and domain splitting""" + + def __init__(self, csp): + """a CSP solver that uses arc consistency + * csp is the CSP to be solved + """ + self.csp = csp + + def GAC(self, orig_domains=None, to_do=None, arc_heuristic=sat_up): + """ + Makes this CSP arc-consistent using Generalized Arc Consistency + orig_domains: is the original domains + to_do : is a set of (variable,constraint) pairs + returns the reduced domains (an arc-consistent variable:domain dictionary) + """ + if orig_domains is None: + orig_domains = self.csp.domains + if to_do is None: + to_do = {(var, const) for const in self.csp.constraints for var in const.scope} + else: + to_do = to_do.copy() + domains = orig_domains.copy() + to_do = arc_heuristic(to_do) + checks = 0 + while to_do: + var, const = to_do.pop() + other_vars = [ov for ov in const.scope if ov != var] + new_domain = set() + if len(other_vars) == 0: + for val in domains[var]: + if const.holds({var: val}): + new_domain.add(val) + checks += 1 + # new_domain = {val for val in domains[var] + # if const.holds({var: val})} + elif len(other_vars) == 1: + other = other_vars[0] + for val in domains[var]: + for other_val in domains[other]: + checks += 1 + if const.holds({var: val, other: other_val}): + new_domain.add(val) + break + # new_domain = {val for val in domains[var] + # if any(const.holds({var: val, other: other_val}) + # for other_val in domains[other])} + else: # general case + for val in domains[var]: + holds, checks = self.any_holds(domains, const, {var: val}, other_vars, checks=checks) + if holds: + new_domain.add(val) + # new_domain = {val for val in domains[var] + # if self.any_holds(domains, const, {var: val}, other_vars)} + if new_domain != domains[var]: + domains[var] = new_domain + if not new_domain: + return False, domains, checks + add_to_do = self.new_to_do(var, const).difference(to_do) + to_do |= add_to_do + return True, domains, checks + + def new_to_do(self, var, const): + """ + Returns new elements to be added to to_do after assigning + variable var in constraint const. + """ + return {(nvar, nconst) for nconst in self.csp.var_to_const[var] + if nconst != const + for nvar in nconst.scope + if nvar != var} + + def any_holds(self, domains, const, env, other_vars, ind=0, checks=0): + """ + Returns True if Constraint const holds for an assignment + that extends env with the variables in other_vars[ind:] + env is a dictionary + Warning: this has side effects and changes the elements of env + """ + if ind == len(other_vars): + return const.holds(env), checks + 1 + else: + var = other_vars[ind] + for val in domains[var]: + # env = dict_union(env, {var:val}) # no side effects + env[var] = val + holds, checks = self.any_holds(domains, const, env, other_vars, ind + 1, checks) + if holds: + return True, checks + return False, checks + + def domain_splitting(self, domains=None, to_do=None, arc_heuristic=sat_up): + """ + Return a solution to the current CSP or False if there are no solutions + to_do is the list of arcs to check + """ + if domains is None: + domains = self.csp.domains + consistency, new_domains, _ = self.GAC(domains, to_do, arc_heuristic) + if not consistency: + return False + elif all(len(new_domains[var]) == 1 for var in domains): + return {var: first(new_domains[var]) for var in domains} + else: + var = first(x for x in self.csp.variables if len(new_domains[x]) > 1) + if var: + dom1, dom2 = partition_domain(new_domains[var]) + new_doms1 = extend(new_domains, var, dom1) + new_doms2 = extend(new_domains, var, dom2) + to_do = self.new_to_do(var, None) + return self.domain_splitting(new_doms1, to_do, arc_heuristic) or \ + self.domain_splitting(new_doms2, to_do, arc_heuristic) + + +def partition_domain(dom): + """Partitions domain dom into two""" + split = len(dom) // 2 + dom1 = set(list(dom)[:split]) + dom2 = dom - dom1 + return dom1, dom2 + + +class ACSearchSolver(search.Problem): + """A search problem with arc consistency and domain splitting + A node is a CSP""" + + def __init__(self, csp, arc_heuristic=sat_up): + self.cons = ACSolver(csp) + consistency, self.domains, _ = self.cons.GAC(arc_heuristic=arc_heuristic) + if not consistency: + raise Exception('CSP is inconsistent') + self.heuristic = arc_heuristic + super().__init__(self.domains) + + def goal_test(self, node): + """Node is a goal if all domains have 1 element""" + return all(len(node[var]) == 1 for var in node) + + def actions(self, state): + var = first(x for x in state if len(state[x]) > 1) + neighs = [] + if var: + dom1, dom2 = partition_domain(state[var]) + to_do = self.cons.new_to_do(var, None) + for dom in [dom1, dom2]: + new_domains = extend(state, var, dom) + consistency, cons_doms, _ = self.cons.GAC(new_domains, to_do, self.heuristic) + if consistency: + neighs.append(cons_doms) + return neighs + + def result(self, state, action): + return action + + +def ac_solver(csp, arc_heuristic=sat_up): + """Arc consistency (domain splitting interface)""" + return ACSolver(csp).domain_splitting(arc_heuristic=arc_heuristic) + + +def ac_search_solver(csp, arc_heuristic=sat_up): + """Arc consistency (search interface)""" + from search import depth_first_tree_search + solution = None + try: + solution = depth_first_tree_search(ACSearchSolver(csp, arc_heuristic=arc_heuristic)).state + except: + return solution + if solution: + return {var: first(solution[var]) for var in solution} + + +# ______________________________________________________________________________ +# Crossword Problem + + +csp_crossword = NaryCSP({'one_across': {'ant', 'big', 'bus', 'car', 'has'}, + 'one_down': {'book', 'buys', 'hold', 'lane', 'year'}, + 'two_down': {'ginger', 'search', 'symbol', 'syntax'}, + 'three_across': {'book', 'buys', 'hold', 'land', 'year'}, + 'four_across': {'ant', 'big', 'bus', 'car', 'has'}}, + [Constraint(('one_across', 'one_down'), meet_at_constraint(0, 0)), + Constraint(('one_across', 'two_down'), meet_at_constraint(2, 0)), + Constraint(('three_across', 'two_down'), meet_at_constraint(2, 2)), + Constraint(('three_across', 'one_down'), meet_at_constraint(0, 2)), + Constraint(('four_across', 'two_down'), meet_at_constraint(0, 4))]) + +crossword1 = [['_', '_', '_', '*', '*'], + ['_', '*', '_', '*', '*'], + ['_', '_', '_', '_', '*'], + ['_', '*', '_', '*', '*'], + ['*', '*', '_', '_', '_'], + ['*', '*', '_', '*', '*']] + +words1 = {'ant', 'big', 'bus', 'car', 'has', 'book', 'buys', 'hold', + 'lane', 'year', 'ginger', 'search', 'symbol', 'syntax'} + + +class Crossword(NaryCSP): + + def __init__(self, puzzle, words): + domains = {} + constraints = [] + for i, line in enumerate(puzzle): + scope = [] + for j, element in enumerate(line): + if element == '_': + var = "p" + str(j) + str(i) + domains[var] = list(string.ascii_lowercase) + scope.append(var) + else: + if len(scope) > 1: + constraints.append(Constraint(tuple(scope), is_word_constraint(words))) + scope.clear() + if len(scope) > 1: + constraints.append(Constraint(tuple(scope), is_word_constraint(words))) + puzzle_t = list(map(list, zip(*puzzle))) + for i, line in enumerate(puzzle_t): + scope = [] + for j, element in enumerate(line): + if element == '_': + scope.append("p" + str(i) + str(j)) + else: + if len(scope) > 1: + constraints.append(Constraint(tuple(scope), is_word_constraint(words))) + scope.clear() + if len(scope) > 1: + constraints.append(Constraint(tuple(scope), is_word_constraint(words))) + super().__init__(domains, constraints) + self.puzzle = puzzle + + def display(self, assignment=None): + for i, line in enumerate(self.puzzle): + puzzle = "" + for j, element in enumerate(line): + if element == '*': + puzzle += "[*] " + else: + var = "p" + str(j) + str(i) + if assignment is not None: + if isinstance(assignment[var], set) and len(assignment[var]) == 1: + puzzle += "[" + str(first(assignment[var])).upper() + "] " + elif isinstance(assignment[var], str): + puzzle += "[" + str(assignment[var]).upper() + "] " + else: + puzzle += "[_] " + else: + puzzle += "[_] " + print(puzzle) + + +# ______________________________________________________________________________ +# Kakuro Problem + + +# difficulty 0 +kakuro1 = [['*', '*', '*', [6, ''], [3, '']], + ['*', [4, ''], [3, 3], '_', '_'], + [['', 10], '_', '_', '_', '_'], + [['', 3], '_', '_', '*', '*']] + +# difficulty 0 +kakuro2 = [ + ['*', [10, ''], [13, ''], '*'], + [['', 3], '_', '_', [13, '']], + [['', 12], '_', '_', '_'], + [['', 21], '_', '_', '_']] + +# difficulty 1 +kakuro3 = [ + ['*', [17, ''], [28, ''], '*', [42, ''], [22, '']], + [['', 9], '_', '_', [31, 14], '_', '_'], + [['', 20], '_', '_', '_', '_', '_'], + ['*', ['', 30], '_', '_', '_', '_'], + ['*', [22, 24], '_', '_', '_', '*'], + [['', 25], '_', '_', '_', '_', [11, '']], + [['', 20], '_', '_', '_', '_', '_'], + [['', 14], '_', '_', ['', 17], '_', '_']] + +# difficulty 2 +kakuro4 = [ + ['*', '*', '*', '*', '*', [4, ''], [24, ''], [11, ''], '*', '*', '*', [11, ''], [17, ''], '*', '*'], + ['*', '*', '*', [17, ''], [11, 12], '_', '_', '_', '*', '*', [24, 10], '_', '_', [11, ''], '*'], + ['*', [4, ''], [16, 26], '_', '_', '_', '_', '_', '*', ['', 20], '_', '_', '_', '_', [16, '']], + [['', 20], '_', '_', '_', '_', [24, 13], '_', '_', [16, ''], ['', 12], '_', '_', [23, 10], '_', '_'], + [['', 10], '_', '_', [24, 12], '_', '_', [16, 5], '_', '_', [16, 30], '_', '_', '_', '_', '_'], + ['*', '*', [3, 26], '_', '_', '_', '_', ['', 12], '_', '_', [4, ''], [16, 14], '_', '_', '*'], + ['*', ['', 8], '_', '_', ['', 15], '_', '_', [34, 26], '_', '_', '_', '_', '_', '*', '*'], + ['*', ['', 11], '_', '_', [3, ''], [17, ''], ['', 14], '_', '_', ['', 8], '_', '_', [7, ''], [17, ''], '*'], + ['*', '*', '*', [23, 10], '_', '_', [3, 9], '_', '_', [4, ''], [23, ''], ['', 13], '_', '_', '*'], + ['*', '*', [10, 26], '_', '_', '_', '_', '_', ['', 7], '_', '_', [30, 9], '_', '_', '*'], + ['*', [17, 11], '_', '_', [11, ''], [24, 8], '_', '_', [11, 21], '_', '_', '_', '_', [16, ''], [17, '']], + [['', 29], '_', '_', '_', '_', '_', ['', 7], '_', '_', [23, 14], '_', '_', [3, 17], '_', '_'], + [['', 10], '_', '_', [3, 10], '_', '_', '*', ['', 8], '_', '_', [4, 25], '_', '_', '_', '_'], + ['*', ['', 16], '_', '_', '_', '_', '*', ['', 23], '_', '_', '_', '_', '_', '*', '*'], + ['*', '*', ['', 6], '_', '_', '*', '*', ['', 15], '_', '_', '_', '*', '*', '*', '*']] + + +class Kakuro(NaryCSP): + + def __init__(self, puzzle): + variables = [] + for i, line in enumerate(puzzle): + # print line + for j, element in enumerate(line): + if element == '_': + var1 = str(i) + if len(var1) == 1: + var1 = "0" + var1 + var2 = str(j) + if len(var2) == 1: + var2 = "0" + var2 + variables.append("X" + var1 + var2) + domains = {} + for var in variables: + domains[var] = set(range(1, 10)) + constraints = [] + for i, line in enumerate(puzzle): + for j, element in enumerate(line): + if element != '_' and element != '*': + # down - column + if element[0] != '': + x = [] + for k in range(i + 1, len(puzzle)): + if puzzle[k][j] != '_': + break + var1 = str(k) + if len(var1) == 1: + var1 = "0" + var1 + var2 = str(j) + if len(var2) == 1: + var2 = "0" + var2 + x.append("X" + var1 + var2) + constraints.append(Constraint(x, sum_constraint(element[0]))) + constraints.append(Constraint(x, all_diff_constraint)) + # right - line + if element[1] != '': + x = [] + for k in range(j + 1, len(puzzle[i])): + if puzzle[i][k] != '_': + break + var1 = str(i) + if len(var1) == 1: + var1 = "0" + var1 + var2 = str(k) + if len(var2) == 1: + var2 = "0" + var2 + x.append("X" + var1 + var2) + constraints.append(Constraint(x, sum_constraint(element[1]))) + constraints.append(Constraint(x, all_diff_constraint)) + super().__init__(domains, constraints) + self.puzzle = puzzle + + def display(self, assignment=None): + for i, line in enumerate(self.puzzle): + puzzle = "" + for j, element in enumerate(line): + if element == '*': + puzzle += "[*]\t" + elif element == '_': + var1 = str(i) + if len(var1) == 1: + var1 = "0" + var1 + var2 = str(j) + if len(var2) == 1: + var2 = "0" + var2 + var = "X" + var1 + var2 + if assignment is not None: + if isinstance(assignment[var], set) and len(assignment[var]) == 1: + puzzle += "[" + str(first(assignment[var])) + "]\t" + elif isinstance(assignment[var], int): + puzzle += "[" + str(assignment[var]) + "]\t" + else: + puzzle += "[_]\t" + else: + puzzle += "[_]\t" + else: + puzzle += str(element[0]) + "\\" + str(element[1]) + "\t" + print(puzzle) + + +# ______________________________________________________________________________ +# Cryptarithmetic Problem + +# [Figure 6.2] +# T W O + T W O = F O U R +two_two_four = NaryCSP({'T': set(range(1, 10)), 'F': set(range(1, 10)), + 'W': set(range(0, 10)), 'O': set(range(0, 10)), 'U': set(range(0, 10)), 'R': set(range(0, 10)), + 'C1': set(range(0, 2)), 'C2': set(range(0, 2)), 'C3': set(range(0, 2))}, + [Constraint(('T', 'F', 'W', 'O', 'U', 'R'), all_diff_constraint), + Constraint(('O', 'R', 'C1'), lambda o, r, c1: o + o == r + 10 * c1), + Constraint(('W', 'U', 'C1', 'C2'), lambda w, u, c1, c2: c1 + w + w == u + 10 * c2), + Constraint(('T', 'O', 'C2', 'C3'), lambda t, o, c2, c3: c2 + t + t == o + 10 * c3), + Constraint(('F', 'C3'), eq)]) + +# S E N D + M O R E = M O N E Y +send_more_money = NaryCSP({'S': set(range(1, 10)), 'M': set(range(1, 10)), + 'E': set(range(0, 10)), 'N': set(range(0, 10)), 'D': set(range(0, 10)), + 'O': set(range(0, 10)), 'R': set(range(0, 10)), 'Y': set(range(0, 10)), + 'C1': set(range(0, 2)), 'C2': set(range(0, 2)), 'C3': set(range(0, 2)), + 'C4': set(range(0, 2))}, + [Constraint(('S', 'E', 'N', 'D', 'M', 'O', 'R', 'Y'), all_diff_constraint), + Constraint(('D', 'E', 'Y', 'C1'), lambda d, e, y, c1: d + e == y + 10 * c1), + Constraint(('N', 'R', 'E', 'C1', 'C2'), lambda n, r, e, c1, c2: c1 + n + r == e + 10 * c2), + Constraint(('E', 'O', 'N', 'C2', 'C3'), lambda e, o, n, c2, c3: c2 + e + o == n + 10 * c3), + Constraint(('S', 'M', 'O', 'C3', 'C4'), lambda s, m, o, c3, c4: c3 + s + m == o + 10 * c4), + Constraint(('M', 'C4'), eq)]) diff --git a/deep_learning4e.py b/deep_learning4e.py new file mode 100644 index 000000000..9f5b0a8f7 --- /dev/null +++ b/deep_learning4e.py @@ -0,0 +1,584 @@ +"""Deep learning. (Chapters 20)""" + +import random +import statistics + +import numpy as np +from keras import Sequential, optimizers +from keras.layers import Embedding, SimpleRNN, Dense +from keras.preprocessing import sequence + +from utils4e import (conv1D, gaussian_kernel, element_wise_product, vector_add, random_weights, + scalar_vector_product, map_vector, mean_squared_error_loss) + + +class Node: + """ + A single unit of a layer in a neural network + :param weights: weights between parent nodes and current node + :param value: value of current node + """ + + def __init__(self, weights=None, value=None): + self.value = value + self.weights = weights or [] + + +class Layer: + """ + A layer in a neural network based on a computational graph. + :param size: number of units in the current layer + """ + + def __init__(self, size): + self.nodes = np.array([Node() for _ in range(size)]) + + def forward(self, inputs): + """Define the operation to get the output of this layer""" + raise NotImplementedError + + +class Activation: + + def function(self, x): + return NotImplementedError + + def derivative(self, x): + return NotImplementedError + + def __call__(self, x): + return self.function(x) + + +class Sigmoid(Activation): + + def function(self, x): + return 1 / (1 + np.exp(-x)) + + def derivative(self, value): + return value * (1 - value) + + +class ReLU(Activation): + + def function(self, x): + return max(0, x) + + def derivative(self, value): + return 1 if value > 0 else 0 + + +class ELU(Activation): + + def __init__(self, alpha=0.01): + self.alpha = alpha + + def function(self, x): + return x if x > 0 else self.alpha * (np.exp(x) - 1) + + def derivative(self, value): + return 1 if value > 0 else self.alpha * np.exp(value) + + +class LeakyReLU(Activation): + + def __init__(self, alpha=0.01): + self.alpha = alpha + + def function(self, x): + return max(x, self.alpha * x) + + def derivative(self, value): + return 1 if value > 0 else self.alpha + + +class Tanh(Activation): + + def function(self, x): + return np.tanh(x) + + def derivative(self, value): + return 1 - (value ** 2) + + +class SoftMax(Activation): + + def function(self, x): + return np.exp(x) / np.sum(np.exp(x)) + + def derivative(self, x): + return np.ones_like(x) + + +class SoftPlus(Activation): + + def function(self, x): + return np.log(1. + np.exp(x)) + + def derivative(self, x): + return 1. / (1. + np.exp(-x)) + + +class Linear(Activation): + + def function(self, x): + return x + + def derivative(self, x): + return np.ones_like(x) + + +class InputLayer(Layer): + """1D input layer. Layer size is the same as input vector size.""" + + def __init__(self, size=3): + super().__init__(size) + + def forward(self, inputs): + """Take each value of the inputs to each unit in the layer.""" + assert len(self.nodes) == len(inputs) + for node, inp in zip(self.nodes, inputs): + node.value = inp + return inputs + + +class OutputLayer(Layer): + """1D softmax output layer in 19.3.2.""" + + def __init__(self, size=3): + super().__init__(size) + + def forward(self, inputs, activation=SoftMax): + assert len(self.nodes) == len(inputs) + res = activation().function(inputs) + for node, val in zip(self.nodes, res): + node.value = val + return res + + +class DenseLayer(Layer): + """ + 1D dense layer in a neural network. + :param in_size: (int) input vector size + :param out_size: (int) output vector size + :param activation: (Activation object) activation function + """ + + def __init__(self, in_size=3, out_size=3, activation=Sigmoid): + super().__init__(out_size) + self.out_size = out_size + self.inputs = None + self.activation = activation() + # initialize weights + for node in self.nodes: + node.weights = random_weights(-0.5, 0.5, in_size) + + def forward(self, inputs): + self.inputs = inputs + res = [] + # get the output value of each unit + for unit in self.nodes: + val = self.activation.function(np.dot(unit.weights, inputs)) + unit.value = val + res.append(val) + return res + + +class ConvLayer1D(Layer): + """ + 1D convolution layer of in neural network. + :param kernel_size: convolution kernel size + """ + + def __init__(self, size=3, kernel_size=3): + super().__init__(size) + # init convolution kernel as gaussian kernel + for node in self.nodes: + node.weights = gaussian_kernel(kernel_size) + + def forward(self, features): + # each node in layer takes a channel in the features + assert len(self.nodes) == len(features) + res = [] + # compute the convolution output of each channel, store it in node.val + for node, feature in zip(self.nodes, features): + out = conv1D(feature, node.weights) + res.append(out) + node.value = out + return res + + +class MaxPoolingLayer1D(Layer): + """ + 1D max pooling layer in a neural network. + :param kernel_size: max pooling area size + """ + + def __init__(self, size=3, kernel_size=3): + super().__init__(size) + self.kernel_size = kernel_size + self.inputs = None + + def forward(self, features): + assert len(self.nodes) == len(features) + res = [] + self.inputs = features + # do max pooling for each channel in features + for i in range(len(self.nodes)): + feature = features[i] + # get the max value in a kernel_size * kernel_size area + out = [max(feature[i:i + self.kernel_size]) + for i in range(len(feature) - self.kernel_size + 1)] + res.append(out) + self.nodes[i].value = out + return res + + +class BatchNormalizationLayer(Layer): + """Batch normalization layer.""" + + def __init__(self, size, eps=0.001): + super().__init__(size) + self.eps = eps + # self.weights = [beta, gamma] + self.weights = [0, 0] + self.inputs = None + + def forward(self, inputs): + # mean value of inputs + mu = sum(inputs) / len(inputs) + # standard error of inputs + stderr = statistics.stdev(inputs) + self.inputs = inputs + res = [] + # get normalized value of each input + for i in range(len(self.nodes)): + val = [(inputs[i] - mu) * self.weights[0] / np.sqrt(self.eps + stderr ** 2) + self.weights[1]] + res.append(val) + self.nodes[i].value = val + return res + + +def init_examples(examples, idx_i, idx_t, o_units): + """Init examples from dataset.examples.""" + + inputs, targets = {}, {} + for i, e in enumerate(examples): + # input values of e + inputs[i] = [e[i] for i in idx_i] + + if o_units > 1: + # one-hot representation of e's target + t = [0 for i in range(o_units)] + t[e[idx_t]] = 1 + targets[i] = t + else: + # target value of e + targets[i] = [e[idx_t]] + + return inputs, targets + + +def stochastic_gradient_descent(dataset, net, loss, epochs=1000, l_rate=0.01, batch_size=1, verbose=False): + """ + Gradient descent algorithm to update the learnable parameters of a network. + :return: the updated network + """ + examples = dataset.examples # init data + + for e in range(epochs): + total_loss = 0 + random.shuffle(examples) + weights = [[node.weights for node in layer.nodes] for layer in net] + + for batch in get_batch(examples, batch_size): + inputs, targets = init_examples(batch, dataset.inputs, dataset.target, len(net[-1].nodes)) + # compute gradients of weights + gs, batch_loss = BackPropagation(inputs, targets, weights, net, loss) + # update weights with gradient descent + weights = [x + y for x, y in zip(weights, [np.array(tg) * -l_rate for tg in gs])] + total_loss += batch_loss + + # update the weights of network each batch + for i in range(len(net)): + if weights[i].size != 0: + for j in range(len(weights[i])): + net[i].nodes[j].weights = weights[i][j] + + if verbose: + print("epoch:{}, total_loss:{}".format(e + 1, total_loss)) + + return net + + +def adam(dataset, net, loss, epochs=1000, rho=(0.9, 0.999), delta=1 / 10 ** 8, + l_rate=0.001, batch_size=1, verbose=False): + """ + [Figure 19.6] + Adam optimizer to update the learnable parameters of a network. + Required parameters are similar to gradient descent. + :return the updated network + """ + examples = dataset.examples + + # init s,r and t + s = [[[0] * len(node.weights) for node in layer.nodes] for layer in net] + r = [[[0] * len(node.weights) for node in layer.nodes] for layer in net] + t = 0 + + # repeat util converge + for e in range(epochs): + # total loss of each epoch + total_loss = 0 + random.shuffle(examples) + weights = [[node.weights for node in layer.nodes] for layer in net] + + for batch in get_batch(examples, batch_size): + t += 1 + inputs, targets = init_examples(batch, dataset.inputs, dataset.target, len(net[-1].nodes)) + + # compute gradients of weights + gs, batch_loss = BackPropagation(inputs, targets, weights, net, loss) + + # update s,r,s_hat and r_gat + s = vector_add(scalar_vector_product(rho[0], s), + scalar_vector_product((1 - rho[0]), gs)) + r = vector_add(scalar_vector_product(rho[1], r), + scalar_vector_product((1 - rho[1]), element_wise_product(gs, gs))) + s_hat = scalar_vector_product(1 / (1 - rho[0] ** t), s) + r_hat = scalar_vector_product(1 / (1 - rho[1] ** t), r) + + # rescale r_hat + r_hat = map_vector(lambda x: 1 / (np.sqrt(x) + delta), r_hat) + + # delta weights + delta_theta = scalar_vector_product(-l_rate, element_wise_product(s_hat, r_hat)) + weights = vector_add(weights, delta_theta) + total_loss += batch_loss + + # update the weights of network each batch + for i in range(len(net)): + if weights[i]: + for j in range(len(weights[i])): + net[i].nodes[j].weights = weights[i][j] + + if verbose: + print("epoch:{}, total_loss:{}".format(e + 1, total_loss)) + + return net + + +def BackPropagation(inputs, targets, theta, net, loss): + """ + The back-propagation algorithm for multilayer networks in only one epoch, to calculate gradients of theta. + :param inputs: a batch of inputs in an array. Each input is an iterable object + :param targets: a batch of targets in an array. Each target is an iterable object + :param theta: parameters to be updated + :param net: a list of predefined layer objects representing their linear sequence + :param loss: a predefined loss function taking array of inputs and targets + :return: gradients of theta, loss of the input batch + """ + + assert len(inputs) == len(targets) + o_units = len(net[-1].nodes) + n_layers = len(net) + batch_size = len(inputs) + + gradients = [[[] for _ in layer.nodes] for layer in net] + total_gradients = [[[0] * len(node.weights) for node in layer.nodes] for layer in net] + + batch_loss = 0 + + # iterate over each example in batch + for e in range(batch_size): + i_val = inputs[e] + t_val = targets[e] + + # forward pass and compute batch loss + for i in range(1, n_layers): + layer_out = net[i].forward(i_val) + i_val = layer_out + batch_loss += loss(t_val, layer_out) + + # initialize delta + delta = [[] for _ in range(n_layers)] + + previous = np.array([layer_out[i] - t_val[i] for i in range(o_units)]) + h_layers = n_layers - 1 + + # backward pass + for i in range(h_layers, 0, -1): + layer = net[i] + derivative = np.array([layer.activation.derivative(node.value) for node in layer.nodes]) + delta[i] = previous * derivative + # pass to layer i-1 in the next iteration + previous = np.matmul([delta[i]], theta[i])[0] + # compute gradient of layer i + gradients[i] = [scalar_vector_product(d, net[i].inputs) for d in delta[i]] + + # add gradient of current example to batch gradient + total_gradients = vector_add(total_gradients, gradients) + + return total_gradients, batch_loss + + +def get_batch(examples, batch_size=1): + """Split examples into multiple batches""" + for i in range(0, len(examples), batch_size): + yield examples[i: i + batch_size] + + +class NeuralNetworkLearner: + """ + Simple dense multilayer neural network. + :param hidden_layer_sizes: size of hidden layers in the form of a list + """ + + def __init__(self, dataset, hidden_layer_sizes, l_rate=0.01, epochs=1000, batch_size=10, + optimizer=stochastic_gradient_descent, loss=mean_squared_error_loss, verbose=False, plot=False): + self.dataset = dataset + self.l_rate = l_rate + self.epochs = epochs + self.batch_size = batch_size + self.optimizer = optimizer + self.loss = loss + self.verbose = verbose + self.plot = plot + + input_size = len(dataset.inputs) + output_size = len(dataset.values[dataset.target]) + + # initialize the network + raw_net = [InputLayer(input_size)] + # add hidden layers + hidden_input_size = input_size + for h_size in hidden_layer_sizes: + raw_net.append(DenseLayer(hidden_input_size, h_size)) + hidden_input_size = h_size + raw_net.append(DenseLayer(hidden_input_size, output_size)) + self.raw_net = raw_net + + def fit(self, X, y): + self.learned_net = self.optimizer(self.dataset, self.raw_net, loss=self.loss, epochs=self.epochs, + l_rate=self.l_rate, batch_size=self.batch_size, verbose=self.verbose) + return self + + def predict(self, example): + n_layers = len(self.learned_net) + + layer_input = example + layer_out = example + + # get the output of each layer by forward passing + for i in range(1, n_layers): + layer_out = self.learned_net[i].forward(np.array(layer_input).reshape((-1, 1))) + layer_input = layer_out + + return layer_out.index(max(layer_out)) + + +class PerceptronLearner: + """ + Simple perceptron neural network. + """ + + def __init__(self, dataset, l_rate=0.01, epochs=1000, batch_size=10, optimizer=stochastic_gradient_descent, + loss=mean_squared_error_loss, verbose=False, plot=False): + self.dataset = dataset + self.l_rate = l_rate + self.epochs = epochs + self.batch_size = batch_size + self.optimizer = optimizer + self.loss = loss + self.verbose = verbose + self.plot = plot + + input_size = len(dataset.inputs) + output_size = len(dataset.values[dataset.target]) + + # initialize the network, add dense layer + self.raw_net = [InputLayer(input_size), DenseLayer(input_size, output_size)] + + def fit(self, X, y): + self.learned_net = self.optimizer(self.dataset, self.raw_net, loss=self.loss, epochs=self.epochs, + l_rate=self.l_rate, batch_size=self.batch_size, verbose=self.verbose) + return self + + def predict(self, example): + layer_out = self.learned_net[1].forward(np.array(example).reshape((-1, 1))) + return layer_out.index(max(layer_out)) + + +def keras_dataset_loader(dataset, max_length=500): + """ + Helper function to load keras datasets. + :param dataset: keras data set type + :param max_length: max length of each input sequence + """ + # init dataset + (X_train, y_train), (X_val, y_val) = dataset + if max_length > 0: + X_train = sequence.pad_sequences(X_train, maxlen=max_length) + X_val = sequence.pad_sequences(X_val, maxlen=max_length) + return (X_train[10:], y_train[10:]), (X_val, y_val), (X_train[:10], y_train[:10]) + + +def SimpleRNNLearner(train_data, val_data, epochs=2, verbose=False): + """ + RNN example for text sentimental analysis. + :param train_data: a tuple of (training data, targets) + Training data: ndarray taking training examples, while each example is coded by embedding + Targets: ndarray taking targets of each example. Each target is mapped to an integer + :param val_data: a tuple of (validation data, targets) + :param epochs: number of epochs + :param verbose: verbosity mode + :return: a keras model + """ + + total_inputs = 5000 + input_length = 500 + + # init data + X_train, y_train = train_data + X_val, y_val = val_data + + # init a the sequential network (embedding layer, rnn layer, dense layer) + model = Sequential() + model.add(Embedding(total_inputs, 32, input_length=input_length)) + model.add(SimpleRNN(units=128)) + model.add(Dense(1, activation='sigmoid')) + model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) + + # train the model + model.fit(X_train, y_train, validation_data=(X_val, y_val), epochs=epochs, batch_size=128, verbose=verbose) + + return model + + +def AutoencoderLearner(inputs, encoding_size, epochs=200, verbose=False): + """ + Simple example of linear auto encoder learning producing the input itself. + :param inputs: a batch of input data in np.ndarray type + :param encoding_size: int, the size of encoding layer + :param epochs: number of epochs + :param verbose: verbosity mode + :return: a keras model + """ + + # init data + input_size = len(inputs[0]) + + # init model + model = Sequential() + model.add(Dense(encoding_size, input_dim=input_size, activation='relu', kernel_initializer='random_uniform', + bias_initializer='ones')) + model.add(Dense(input_size, activation='relu', kernel_initializer='random_uniform', bias_initializer='ones')) + + # update model with sgd + sgd = optimizers.SGD(lr=0.01) + model.compile(loss='mean_squared_error', optimizer=sgd, metrics=['accuracy']) + + # train the model + model.fit(inputs, inputs, epochs=epochs, batch_size=10, verbose=verbose) + + return model diff --git a/games.ipynb b/games.ipynb index 51a2015b4..edf955be8 100644 --- a/games.ipynb +++ b/games.ipynb @@ -82,7 +82,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": { "collapsed": true }, @@ -135,11 +135,18 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": { "collapsed": true }, - "outputs": [], + "outputs": [ + { + "output_type": "stream", + "text": "\u001b[1;32mclass\u001b[0m \u001b[0mTicTacToe\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mGame\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"Play TicTacToe on an h x v board, with Max (first player) playing 'X'.\n A state has the player to move, a cached utility, a list of moves in\n the form of a list of (x, y) positions, and a board, in the form of\n a dict of {(x, y): Player} entries, where Player is 'X' or 'O'.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mh\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mv\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mk\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mh\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mh\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mv\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mk\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mmoves\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mh\u001b[0m \u001b[1;33m+\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mv\u001b[0m \u001b[1;33m+\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minitial\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mGameState\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mto_move\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'X'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mutility\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m{\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmoves\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mactions\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"Legal moves are any square not yet taken.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mmove\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mstate\u001b[0m \u001b[1;31m# Illegal move has no effect\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mboard\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mmove\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto_move\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mmoves\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mmoves\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmove\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mGameState\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mto_move\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'O'\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto_move\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'X'\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;34m'X'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mutility\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcompute_utility\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto_move\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmoves\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mutility\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"Return the value to player; 1 for win, -1 for loss, 0 otherwise.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mutility\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mplayer\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'X'\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;33m-\u001b[0m\u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mutility\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mterminal_test\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"A state is terminal if it is won or there are no empty squares.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mutility\u001b[0m \u001b[1;33m!=\u001b[0m \u001b[1;36m0\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmoves\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mdisplay\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mboard\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mstate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mh\u001b[0m \u001b[1;33m+\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0my\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mv\u001b[0m \u001b[1;33m+\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'.'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mend\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m' '\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mcompute_utility\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"If 'X' wins with this move, return 1; if 'O' wins return -1; else return 0.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk_in_row\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mor\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk_in_row\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mor\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk_in_row\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mor\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk_in_row\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[1;33m+\u001b[0m\u001b[1;36m1\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mplayer\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'X'\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m\n\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mk_in_row\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdelta_x_y\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;34m\"\"\"Return true if there is a line through move on board for player.\"\"\"\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mdelta_x\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdelta_y\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdelta_x_y\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mn\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m0\u001b[0m \u001b[1;31m# n is number of moves in row\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mwhile\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m==\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mn\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mx\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0mdelta_x\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0mdelta_y\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmove\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mwhile\u001b[0m \u001b[0mboard\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m==\u001b[0m \u001b[0mplayer\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mn\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mx\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mdelta_x\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mdelta_y\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[0mn\u001b[0m \u001b[1;33m-=\u001b[0m \u001b[1;36m1\u001b[0m \u001b[1;31m# Because we counted move itself twice\u001b[0m\u001b[1;33m\n\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mn\u001b[0m \u001b[1;33m>=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "metadata": {}, + "execution_count": 4 + } + ], "source": [ "%psource TicTacToe" ] @@ -849,6 +856,9 @@ "## alphabeta_player\n", "The `alphabeta_player`, on the other hand, calls the `alphabeta_search` function, which returns the best move in the current game state. Thus, the `alphabeta_player` always plays the best move given a game state, assuming that the game tree is small enough to search entirely.\n", "\n", + "## minimax_player\n", + "The `minimax_player`, on the other hand calls the `minimax_search` function which returns the best move in the current game state.\n", + "\n", "## play_game\n", "The `play_game` function will be the one that will actually be used to play the game. You pass as arguments to it an instance of the game you want to play and the players you want in this game. Use it to play AI vs AI, AI vs human, or even human vs human matches!" ] @@ -1651,9 +1661,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.3" + "version": "3.8.2-final" } }, "nbformat": 4, "nbformat_minor": 1 -} +} \ No newline at end of file diff --git a/games.py b/games.py index 6aded01d5..d22b2e640 100644 --- a/games.py +++ b/games.py @@ -1,20 +1,23 @@ -"""Games, or Adversarial Search (Chapter 5)""" +"""Games or Adversarial Search (Chapter 5)""" -from collections import namedtuple -import random -import itertools import copy -from utils import argmax, vector_add +import itertools +import random +from collections import namedtuple + +import numpy as np + +from utils import vector_add -infinity = float('inf') GameState = namedtuple('GameState', 'to_move, utility, board, moves') StochasticGameState = namedtuple('StochasticGameState', 'to_move, utility, board, moves, chance') + # ______________________________________________________________________________ -# Minimax Search +# MinMax Search -def minimax_decision(state, game): +def minmax_decision(state, game): """Given a state in a game, calculate the best move by searching forward all the way to the terminal states. [Figure 5.3]""" @@ -23,7 +26,7 @@ def minimax_decision(state, game): def max_value(state): if game.terminal_test(state): return game.utility(state, player) - v = -infinity + v = -np.inf for a in game.actions(state): v = max(v, min_value(game.result(state, a))) return v @@ -31,31 +34,34 @@ def max_value(state): def min_value(state): if game.terminal_test(state): return game.utility(state, player) - v = infinity + v = np.inf for a in game.actions(state): v = min(v, max_value(game.result(state, a))) return v - # Body of minimax_decision: - return argmax(game.actions(state), - key=lambda a: min_value(game.result(state, a))) + # Body of minmax_decision: + return max(game.actions(state), key=lambda a: min_value(game.result(state, a))) + # ______________________________________________________________________________ -def expectiminimax(state, game): - """Return the best move for a player after dice are thrown. The game tree - includes chance nodes along with min and max nodes. [Figure 5.11]""" +def expect_minmax(state, game): + """ + [Figure 5.11] + Return the best move for a player after dice are thrown. The game tree + includes chance nodes along with min and max nodes. + """ player = game.to_move(state) def max_value(state): - v = -infinity + v = -np.inf for a in game.actions(state): v = max(v, chance_node(state, a)) return v def min_value(state): - v = infinity + v = np.inf for a in game.actions(state): v = min(v, chance_node(state, a)) return v @@ -76,22 +82,21 @@ def chance_node(state, action): sum_chances += util * game.probability(chance) return sum_chances / num_chances - # Body of expectiminimax: - return argmax(game.actions(state), - key=lambda a: chance_node(state, a), default=None) + # Body of expect_minmax: + return max(game.actions(state), key=lambda a: chance_node(state, a), default=None) -def alphabeta_search(state, game): +def alpha_beta_search(state, game): """Search game to determine best action; use alpha-beta pruning. As in [Figure 5.7], this version searches all the way to the leaves.""" player = game.to_move(state) - # Functions used by alphabeta + # Functions used by alpha_beta def max_value(state, alpha, beta): if game.terminal_test(state): return game.utility(state, player) - v = -infinity + v = -np.inf for a in game.actions(state): v = max(v, min_value(game.result(state, a), alpha, beta)) if v >= beta: @@ -102,7 +107,7 @@ def max_value(state, alpha, beta): def min_value(state, alpha, beta): if game.terminal_test(state): return game.utility(state, player) - v = infinity + v = np.inf for a in game.actions(state): v = min(v, max_value(game.result(state, a), alpha, beta)) if v <= alpha: @@ -110,9 +115,9 @@ def min_value(state, alpha, beta): beta = min(beta, v) return v - # Body of alphabeta_search: - best_score = -infinity - beta = infinity + # Body of alpha_beta_search: + best_score = -np.inf + beta = np.inf best_action = None for a in game.actions(state): v = min_value(game.result(state, a), best_score, beta) @@ -122,20 +127,19 @@ def min_value(state, alpha, beta): return best_action -def alphabeta_cutoff_search(state, game, d=4, cutoff_test=None, eval_fn=None): +def alpha_beta_cutoff_search(state, game, d=4, cutoff_test=None, eval_fn=None): """Search game to determine best action; use alpha-beta pruning. This version cuts off search and uses an evaluation function.""" player = game.to_move(state) - # Functions used by alphabeta + # Functions used by alpha_beta def max_value(state, alpha, beta, depth): if cutoff_test(state, depth): return eval_fn(state) - v = -infinity + v = -np.inf for a in game.actions(state): - v = max(v, min_value(game.result(state, a), - alpha, beta, depth + 1)) + v = max(v, min_value(game.result(state, a), alpha, beta, depth + 1)) if v >= beta: return v alpha = max(alpha, v) @@ -144,23 +148,20 @@ def max_value(state, alpha, beta, depth): def min_value(state, alpha, beta, depth): if cutoff_test(state, depth): return eval_fn(state) - v = infinity + v = np.inf for a in game.actions(state): - v = min(v, max_value(game.result(state, a), - alpha, beta, depth + 1)) + v = min(v, max_value(game.result(state, a), alpha, beta, depth + 1)) if v <= alpha: return v beta = min(beta, v) return v - # Body of alphabeta_cutoff_search starts here: + # Body of alpha_beta_cutoff_search starts here: # The default test cuts off at depth d or at a terminal state - cutoff_test = (cutoff_test or - (lambda state, depth: depth > d or - game.terminal_test(state))) + cutoff_test = (cutoff_test or (lambda state, depth: depth > d or game.terminal_test(state))) eval_fn = eval_fn or (lambda state: game.utility(state, player)) - best_score = -infinity - beta = infinity + best_score = -np.inf + beta = np.inf best_action = None for a in game.actions(state): v = min_value(game.result(state, a), best_score, beta, 1) @@ -169,6 +170,7 @@ def min_value(state, alpha, beta, depth): best_action = a return best_action + # ______________________________________________________________________________ # Players for Games @@ -195,11 +197,17 @@ def random_player(game, state): """A player that chooses a legal move at random.""" return random.choice(game.actions(state)) if game.actions(state) else None -def alphabeta_player(game, state): - return alphabeta_search(state, game) -def expectiminimax_player(game, state): - return expectiminimax(state, game) +def alpha_beta_player(game, state): + return alpha_beta_search(state, game) + + +def minmax_player(game,state): + return minmax_decision(state,game) + + +def expect_minmax_player(game, state): + return expect_minmax(state, game) # ______________________________________________________________________________ @@ -253,6 +261,7 @@ def play_game(self, *players): self.display(state) return self.utility(state, self.to_move(self.initial)) + class StochasticGame(Game): """A stochastic game includes uncertain events which influence the moves of players at each state. To create a stochastic game, subclass @@ -268,7 +277,7 @@ def outcome(self, state, chance): raise NotImplementedError def probability(self, chance): - """Return the probability of occurence of a chance.""" + """Return the probability of occurrence of a chance.""" raise NotImplementedError def play_game(self, *players): @@ -284,6 +293,7 @@ def play_game(self, *players): self.display(state) return self.utility(state, self.to_move(self.initial)) + class Fig52Game(Game): """The game represented in [Figure 5.2]. Serves as a simple test case.""" @@ -316,7 +326,7 @@ def to_move(self, state): class Fig52Extended(Game): """Similar to Fig52Game but bigger. Useful for visualisation""" - succs = {i:dict(l=i*3+1, m=i*3+2, r=i*3+3) for i in range(13)} + succs = {i: dict(l=i * 3 + 1, m=i * 3 + 2, r=i * 3 + 3) for i in range(13)} utils = dict() def actions(self, state): @@ -337,6 +347,7 @@ def terminal_test(self, state): def to_move(self, state): return 'MIN' if state in {1, 2, 3} else 'MAX' + class TicTacToe(Game): """Play TicTacToe on an h x v board, with Max (first player) playing 'X'. A state has the player to move, a cached utility, a list of moves in @@ -417,7 +428,13 @@ def __init__(self, h=7, v=6, k=4): def actions(self, state): return [(x, y) for (x, y) in state.moves - if y == 1 or (x, y - 1) in state.board] + if x == self.h or (x + 1 , y ) in state.board] + +class Gomoku(TicTacToe): + """Also known as Five in a row.""" + + def __init__(self, h=15, v=16, k=5): + TicTacToe.__init__(self, h, v, k) class Backgammon(StochasticGame): @@ -427,14 +444,14 @@ class Backgammon(StochasticGame): def __init__(self): """Initial state of the game""" - point = {'W' : 0, 'B' : 0} + point = {'W': 0, 'B': 0} board = [point.copy() for index in range(24)] board[0]['B'] = board[23]['W'] = 2 board[5]['W'] = board[18]['B'] = 5 board[7]['W'] = board[16]['B'] = 3 board[11]['B'] = board[12]['W'] = 5 - self.allow_bear_off = {'W' : False, 'B' : False} - self.direction = {'W' : -1, 'B' : 1} + self.allow_bear_off = {'W': False, 'B': False} + self.direction = {'W': -1, 'B': 1} self.initial = StochasticGameState(to_move='W', utility=0, board=board, @@ -481,7 +498,7 @@ def get_all_moves(self, board, player): taken_points = [index for index, point in enumerate(all_points) if point[player] > 0] if self.checkers_at_home(board, player) == 1: - return [(taken_points[0], )] + return [(taken_points[0],)] moves = list(itertools.permutations(taken_points, 2)) moves = moves + [(index, index) for index, point in enumerate(all_points) if point[player] >= 2] @@ -498,7 +515,7 @@ def display(self, state): def compute_utility(self, board, move, player): """If 'W' wins with this move, return 1; if 'B' wins return -1; else return 0.""" - util = {'W' : 1, 'B' : -1} + util = {'W': 1, 'B': -1} for idx in range(0, 24): if board[idx][player] > 0: return 0 @@ -569,5 +586,5 @@ def outcome(self, state, chance): moves=state.moves, chance=dice) def probability(self, chance): - """Return the probability of occurence of a dice roll.""" - return 1/36 if chance[0] == chance[1] else 1/18 + """Return the probability of occurrence of a dice roll.""" + return 1 / 36 if chance[0] == chance[1] else 1 / 18 diff --git a/games4e.ipynb b/games4e.ipynb new file mode 100644 index 000000000..5b619f7ed --- /dev/null +++ b/games4e.ipynb @@ -0,0 +1,1667 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Game Tree Search\n", + "\n", + "We start with defining the abstract class `Game`, for turn-taking *n*-player games. We rely on, but do not define yet, the concept of a `state` of the game; we'll see later how individual games define states. For now, all we require is that a state has a `state.to_move` attribute, which gives the name of the player whose turn it is. (\"Name\" will be something like `'X'` or `'O'` for tic-tac-toe.) \n", + "\n", + "We also define `play_game`, which takes a game and a dictionary of `{player_name: strategy_function}` pairs, and plays out the game, on each turn checking `state.to_move` to see whose turn it is, and then getting the strategy function for that player and applying it to the game and the state to get a move." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from collections import namedtuple, Counter, defaultdict\n", + "import random\n", + "import math\n", + "import functools \n", + "cache = functools.lru_cache(10**6)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": {}, + "outputs": [], + "source": [ + "class Game:\n", + " \"\"\"A game is similar to a problem, but it has a terminal test instead of \n", + " a goal test, and a utility for each terminal state. To create a game, \n", + " subclass this class and implement `actions`, `result`, `is_terminal`, \n", + " and `utility`. You will also need to set the .initial attribute to the \n", + " initial state; this can be done in the constructor.\"\"\"\n", + "\n", + " def actions(self, state):\n", + " \"\"\"Return a collection of the allowable moves from this state.\"\"\"\n", + " raise NotImplementedError\n", + "\n", + " def result(self, state, move):\n", + " \"\"\"Return the state that results from making a move from a state.\"\"\"\n", + " raise NotImplementedError\n", + "\n", + " def is_terminal(self, state):\n", + " \"\"\"Return True if this is a final state for the game.\"\"\"\n", + " return not self.actions(state)\n", + " \n", + " def utility(self, state, player):\n", + " \"\"\"Return the value of this final state to player.\"\"\"\n", + " raise NotImplementedError\n", + " \n", + "\n", + "def play_game(game, strategies: dict, verbose=False):\n", + " \"\"\"Play a turn-taking game. `strategies` is a {player_name: function} dict,\n", + " where function(state, game) is used to get the player's move.\"\"\"\n", + " state = game.initial\n", + " while not game.is_terminal(state):\n", + " player = state.to_move\n", + " move = strategies[player](game, state)\n", + " state = game.result(state, move)\n", + " if verbose: \n", + " print('Player', player, 'move:', move)\n", + " print(state)\n", + " return state" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Minimax-Based Game Search Algorithms\n", + "\n", + "We will define several game search algorithms. Each takes two inputs, the game we are playing and the current state of the game, and returns a a `(value, move)` pair, where `value` is the utility that the algorithm computes for the player whose turn it is to move, and `move` is the move itself.\n", + "\n", + "First we define `minimax_search`, which exhaustively searches the game tree to find an optimal move (assuming both players play optimally), and `alphabeta_search`, which does the same computation, but prunes parts of the tree that could not possibly have an affect on the optimnal move. " + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [], + "source": [ + "def minimax_search(game, state):\n", + " \"\"\"Search game tree to determine best move; return (value, move) pair.\"\"\"\n", + "\n", + " player = state.to_move\n", + "\n", + " def max_value(state):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = -infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = min_value(game.result(state, a))\n", + " if v2 > v:\n", + " v, move = v2, a\n", + " return v, move\n", + "\n", + " def min_value(state):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = +infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = max_value(game.result(state, a))\n", + " if v2 < v:\n", + " v, move = v2, a\n", + " return v, move\n", + "\n", + " return max_value(state)\n", + "\n", + "infinity = math.inf\n", + "\n", + "def alphabeta_search(game, state):\n", + " \"\"\"Search game to determine best action; use alpha-beta pruning.\n", + " As in [Figure 5.7], this version searches all the way to the leaves.\"\"\"\n", + "\n", + " player = state.to_move\n", + "\n", + " def max_value(state, alpha, beta):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = -infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = min_value(game.result(state, a), alpha, beta)\n", + " if v2 > v:\n", + " v, move = v2, a\n", + " alpha = max(alpha, v)\n", + " if v >= beta:\n", + " return v, move\n", + " return v, move\n", + "\n", + " def min_value(state, alpha, beta):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = +infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = max_value(game.result(state, a), alpha, beta)\n", + " if v2 < v:\n", + " v, move = v2, a\n", + " beta = min(beta, v)\n", + " if v <= alpha:\n", + " return v, move\n", + " return v, move\n", + "\n", + " return max_value(state, -infinity, +infinity)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# A Simple Game: Tic-Tac-Toe\n", + "\n", + "We have the notion of an abstract game, we have some search functions; now it is time to define a real game; a simple one, tic-tac-toe. Moves are `(x, y)` pairs denoting squares, where `(0, 0)` is the top left, and `(2, 2)` is the bottom right (on a board of size `height=width=3`)." + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [], + "source": [ + "class TicTacToe(Game):\n", + " \"\"\"Play TicTacToe on an `height` by `width` board, needing `k` in a row to win.\n", + " 'X' plays first against 'O'.\"\"\"\n", + "\n", + " def __init__(self, height=3, width=3, k=3):\n", + " self.k = k # k in a row\n", + " self.squares = {(x, y) for x in range(width) for y in range(height)}\n", + " self.initial = Board(height=height, width=width, to_move='X', utility=0)\n", + "\n", + " def actions(self, board):\n", + " \"\"\"Legal moves are any square not yet taken.\"\"\"\n", + " return self.squares - set(board)\n", + "\n", + " def result(self, board, square):\n", + " \"\"\"Place a marker for current player on square.\"\"\"\n", + " player = board.to_move\n", + " board = board.new({square: player}, to_move=('O' if player == 'X' else 'X'))\n", + " win = k_in_row(board, player, square, self.k)\n", + " board.utility = (0 if not win else +1 if player == 'X' else -1)\n", + " return board\n", + "\n", + " def utility(self, board, player):\n", + " \"\"\"Return the value to player; 1 for win, -1 for loss, 0 otherwise.\"\"\"\n", + " return board.utility if player == 'X' else -board.utility\n", + "\n", + " def is_terminal(self, board):\n", + " \"\"\"A board is a terminal state if it is won or there are no empty squares.\"\"\"\n", + " return board.utility != 0 or len(self.squares) == len(board)\n", + "\n", + " def display(self, board): print(board) \n", + "\n", + "\n", + "def k_in_row(board, player, square, k):\n", + " \"\"\"True if player has k pieces in a line through square.\"\"\"\n", + " def in_row(x, y, dx, dy): return 0 if board[x, y] != player else 1 + in_row(x + dx, y + dy, dx, dy)\n", + " return any(in_row(*square, dx, dy) + in_row(*square, -dx, -dy) - 1 >= k\n", + " for (dx, dy) in ((0, 1), (1, 0), (1, 1), (1, -1)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "States in tic-tac-toe (and other games) will be represented as a `Board`, which is a subclass of `defaultdict` that in general will consist of `{(x, y): contents}` pairs, for example `{(0, 0): 'X', (1, 1): 'O'}` might be the state of the board after two moves. Besides the contents of squares, a board also has some attributes: \n", + "- `.to_move` to name the player whose move it is; \n", + "- `.width` and `.height` to give the size of the board (both 3 in tic-tac-toe, but other numbers in related games);\n", + "- possibly other attributes, as specified by keywords. \n", + "\n", + "As a `defaultdict`, the `Board` class has a `__missing__` method, which returns `empty` for squares that have no been assigned but are within the `width` × `height` boundaries, or `off` otherwise. The class has a `__hash__` method, so instances can be stored in hash tables." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [], + "source": [ + "class Board(defaultdict):\n", + " \"\"\"A board has the player to move, a cached utility value, \n", + " and a dict of {(x, y): player} entries, where player is 'X' or 'O'.\"\"\"\n", + " empty = '.'\n", + " off = '#'\n", + " \n", + " def __init__(self, width=8, height=8, to_move=None, **kwds):\n", + " self.__dict__.update(width=width, height=height, to_move=to_move, **kwds)\n", + " \n", + " def new(self, changes: dict, **kwds) -> 'Board':\n", + " \"Given a dict of {(x, y): contents} changes, return a new Board with the changes.\"\n", + " board = Board(width=self.width, height=self.height, **kwds)\n", + " board.update(self)\n", + " board.update(changes)\n", + " return board\n", + "\n", + " def __missing__(self, loc):\n", + " x, y = loc\n", + " if 0 <= x < self.width and 0 <= y < self.height:\n", + " return self.empty\n", + " else:\n", + " return self.off\n", + " \n", + " def __hash__(self): \n", + " return hash(tuple(sorted(self.items()))) + hash(self.to_move)\n", + " \n", + " def __repr__(self):\n", + " def row(y): return ' '.join(self[x, y] for x in range(self.width))\n", + " return '\\n'.join(map(row, range(self.height))) + '\\n'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Players\n", + "\n", + "We need an interface for players. I'll represent a player as a `callable` that will be passed two arguments: `(game, state)` and will return a `move`.\n", + "The function `player` creates a player out of a search algorithm, but you can create your own players as functions, as is done with `random_player` below:" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": {}, + "outputs": [], + "source": [ + "def random_player(game, state): return random.choice(list(game.actions(state)))\n", + "\n", + "def player(search_algorithm):\n", + " \"\"\"A game player who uses the specified search algorithm\"\"\"\n", + " return lambda game, state: search_algorithm(game, state)[1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Playing a Game\n", + "\n", + "We're ready to play a game. I'll set up a match between a `random_player` (who chooses randomly from the legal moves) and a `player(alphabeta_search)` (who makes the optimal alpha-beta move; practical for tic-tac-toe, but not for large games). The `player(alphabeta_search)` will never lose, but if `random_player` is lucky, it will be a tie." + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Player X move: (0, 0)\n", + "X . .\n", + ". . .\n", + ". . .\n", + "\n", + "Player O move: (1, 1)\n", + "X . .\n", + ". O .\n", + ". . .\n", + "\n", + "Player X move: (1, 2)\n", + "X . .\n", + ". O .\n", + ". X .\n", + "\n", + "Player O move: (0, 1)\n", + "X . .\n", + "O O .\n", + ". X .\n", + "\n", + "Player X move: (2, 1)\n", + "X . .\n", + "O O X\n", + ". X .\n", + "\n", + "Player O move: (2, 0)\n", + "X . O\n", + "O O X\n", + ". X .\n", + "\n", + "Player X move: (2, 2)\n", + "X . O\n", + "O O X\n", + ". X X\n", + "\n", + "Player O move: (0, 2)\n", + "X . O\n", + "O O X\n", + "O X X\n", + "\n" + ] + }, + { + "data": { + "text/plain": [ + "-1" + ] + }, + "execution_count": 74, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "play_game(TicTacToe(), dict(X=random_player, O=player(alphabeta_search)), verbose=True).utility" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The alpha-beta player will never lose, but sometimes the random player can stumble into a draw. When two optimal (alpha-beta or minimax) players compete, it will always be a draw:" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Player X move: (0, 1)\n", + ". . .\n", + "X . .\n", + ". . .\n", + "\n", + "Player O move: (0, 0)\n", + "O . .\n", + "X . .\n", + ". . .\n", + "\n", + "Player X move: (2, 0)\n", + "O . X\n", + "X . .\n", + ". . .\n", + "\n", + "Player O move: (2, 1)\n", + "O . X\n", + "X . O\n", + ". . .\n", + "\n", + "Player X move: (1, 2)\n", + "O . X\n", + "X . O\n", + ". X .\n", + "\n", + "Player O move: (0, 2)\n", + "O . X\n", + "X . O\n", + "O X .\n", + "\n", + "Player X move: (1, 0)\n", + "O X X\n", + "X . O\n", + "O X .\n", + "\n", + "Player O move: (1, 1)\n", + "O X X\n", + "X O O\n", + "O X .\n", + "\n", + "Player X move: (2, 2)\n", + "O X X\n", + "X O O\n", + "O X X\n", + "\n" + ] + }, + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 75, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "play_game(TicTacToe(), dict(X=player(alphabeta_search), O=player(minimax_search)), verbose=True).utility" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Connect Four\n", + "\n", + "Connect Four is a variant of tic-tac-toe, played on a larger (7 x 6) board, and with the restriction that in any column you can only play in the lowest empty square in the column." + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": {}, + "outputs": [], + "source": [ + "class ConnectFour(TicTacToe):\n", + " \n", + " def __init__(self): super().__init__(width=7, height=6, k=4)\n", + "\n", + " def actions(self, board):\n", + " \"\"\"In each column you can play only the lowest empty square in the column.\"\"\"\n", + " return {(x, y) for (x, y) in self.squares - set(board)\n", + " if y == board.height - 1 or (x, y + 1) in board}" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Player X move: (2, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . . .\n", + "\n", + "Player O move: (1, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O X . . . .\n", + "\n", + "Player X move: (5, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O X . . X .\n", + "\n", + "Player O move: (4, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O X . O X .\n", + "\n", + "Player X move: (4, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O X . O X .\n", + "\n", + "Player O move: (2, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . O . X . .\n", + ". O X . O X .\n", + "\n", + "Player X move: (2, 3)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . . .\n", + ". . O . X . .\n", + ". O X . O X .\n", + "\n", + "Player O move: (1, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . . .\n", + ". O O . X . .\n", + ". O X . O X .\n", + "\n", + "Player X move: (0, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . . .\n", + ". O O . X . .\n", + "X O X . O X .\n", + "\n", + "Player O move: (5, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . . .\n", + ". O O . X O .\n", + "X O X . O X .\n", + "\n", + "Player X move: (5, 3)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . X .\n", + ". O O . X O .\n", + "X O X . O X .\n", + "\n", + "Player O move: (6, 5)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . X . . X .\n", + ". O O . X O .\n", + "X O X . O X O\n", + "\n", + "Player X move: (1, 3)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". X X . . X .\n", + ". O O . X O .\n", + "X O X . O X O\n", + "\n", + "Player O move: (6, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". X X . . X .\n", + ". O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (5, 2)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + ". X X . . X .\n", + ". O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player O move: (0, 4)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + ". X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (0, 3)\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + "X X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player O move: (0, 2)\n", + ". . . . . . .\n", + ". . . . . . .\n", + "O . . . . X .\n", + "X X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (0, 1)\n", + ". . . . . . .\n", + "X . . . . . .\n", + "O . . . . X .\n", + "X X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player O move: (0, 0)\n", + "O . . . . . .\n", + "X . . . . . .\n", + "O . . . . X .\n", + "X X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (5, 1)\n", + "O . . . . . .\n", + "X . . . . X .\n", + "O . . . . X .\n", + "X X X . . X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player O move: (4, 3)\n", + "O . . . . . .\n", + "X . . . . X .\n", + "O . . . . X .\n", + "X X X . O X .\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (6, 3)\n", + "O . . . . . .\n", + "X . . . . X .\n", + "O . . . . X .\n", + "X X X . O X X\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player O move: (5, 0)\n", + "O . . . . O .\n", + "X . . . . X .\n", + "O . . . . X .\n", + "X X X . O X X\n", + "O O O . X O O\n", + "X O X . O X O\n", + "\n", + "Player X move: (3, 5)\n", + "O . . . . O .\n", + "X . . . . X .\n", + "O . . . . X .\n", + "X X X . O X X\n", + "O O O . X O O\n", + "X O X X O X O\n", + "\n", + "Player O move: (1, 2)\n", + "O . . . . O .\n", + "X . . . . X .\n", + "O O . . . X .\n", + "X X X . O X X\n", + "O O O . X O O\n", + "X O X X O X O\n", + "\n", + "Player X move: (1, 1)\n", + "O . . . . O .\n", + "X X . . . X .\n", + "O O . . . X .\n", + "X X X . O X X\n", + "O O O . X O O\n", + "X O X X O X O\n", + "\n", + "Player O move: (3, 4)\n", + "O . . . . O .\n", + "X X . . . X .\n", + "O O . . . X .\n", + "X X X . O X X\n", + "O O O O X O O\n", + "X O X X O X O\n", + "\n" + ] + }, + { + "data": { + "text/plain": [ + "-1" + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "play_game(ConnectFour(), dict(X=random_player, O=random_player), verbose=True).utility" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Transposition Tables\n", + "\n", + "By treating the game tree as a tree, we can arrive at the same state through different paths, and end up duplicating effort. In state-space search, we kept a table of `reached` states to prevent this. For game-tree search, we can achieve the same effect by applying the `@cache` decorator to the `min_value` and `max_value` functions. We'll use the suffix `_tt` to indicate a function that uses these transisiton tables." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "def minimax_search_tt(game, state):\n", + " \"\"\"Search game to determine best move; return (value, move) pair.\"\"\"\n", + "\n", + " player = state.to_move\n", + "\n", + " @cache\n", + " def max_value(state):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = -infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = min_value(game.result(state, a))\n", + " if v2 > v:\n", + " v, move = v2, a\n", + " return v, move\n", + "\n", + " @cache\n", + " def min_value(state):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = +infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = max_value(game.result(state, a))\n", + " if v2 < v:\n", + " v, move = v2, a\n", + " return v, move\n", + "\n", + " return max_value(state)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For alpha-beta search, we can still use a cache, but it should be based just on the state, not on whatever values alpha and beta have." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": {}, + "outputs": [], + "source": [ + "def cache1(function):\n", + " \"Like lru_cache(None), but only considers the first argument of function.\"\n", + " cache = {}\n", + " def wrapped(x, *args):\n", + " if x not in cache:\n", + " cache[x] = function(x, *args)\n", + " return cache[x]\n", + " return wrapped\n", + "\n", + "def alphabeta_search_tt(game, state):\n", + " \"\"\"Search game to determine best action; use alpha-beta pruning.\n", + " As in [Figure 5.7], this version searches all the way to the leaves.\"\"\"\n", + "\n", + " player = state.to_move\n", + "\n", + " @cache1\n", + " def max_value(state, alpha, beta):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = -infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = min_value(game.result(state, a), alpha, beta)\n", + " if v2 > v:\n", + " v, move = v2, a\n", + " alpha = max(alpha, v)\n", + " if v >= beta:\n", + " return v, move\n", + " return v, move\n", + "\n", + " @cache1\n", + " def min_value(state, alpha, beta):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " v, move = +infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = max_value(game.result(state, a), alpha, beta)\n", + " if v2 < v:\n", + " v, move = v2, a\n", + " beta = min(beta, v)\n", + " if v <= alpha:\n", + " return v, move\n", + " return v, move\n", + "\n", + " return max_value(state, -infinity, +infinity)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 593 ms, sys: 52 ms, total: 645 ms\n", + "Wall time: 655 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "O X X\n", + "X O O\n", + "O X X" + ] + }, + "execution_count": 81, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time play_game(TicTacToe(), {'X':player(alphabeta_search_tt), 'O':player(minimax_search_tt)})" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.07 s, sys: 30.7 ms, total: 3.1 s\n", + "Wall time: 3.15 s\n" + ] + }, + { + "data": { + "text/plain": [ + "O X X\n", + "X O O\n", + "O X X" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time play_game(TicTacToe(), {'X':player(alphabeta_search), 'O':player(minimax_search)})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Heuristic Cutoffs" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [], + "source": [ + "def cutoff_depth(d):\n", + " \"\"\"A cutoff function that searches to depth d.\"\"\"\n", + " return lambda game, state, depth: depth > d\n", + "\n", + "def h_alphabeta_search(game, state, cutoff=cutoff_depth(6), h=lambda s, p: 0):\n", + " \"\"\"Search game to determine best action; use alpha-beta pruning.\n", + " As in [Figure 5.7], this version searches all the way to the leaves.\"\"\"\n", + "\n", + " player = state.to_move\n", + "\n", + " @cache1\n", + " def max_value(state, alpha, beta, depth):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " if cutoff(game, state, depth):\n", + " return h(state, player), None\n", + " v, move = -infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = min_value(game.result(state, a), alpha, beta, depth+1)\n", + " if v2 > v:\n", + " v, move = v2, a\n", + " alpha = max(alpha, v)\n", + " if v >= beta:\n", + " return v, move\n", + " return v, move\n", + "\n", + " @cache1\n", + " def min_value(state, alpha, beta, depth):\n", + " if game.is_terminal(state):\n", + " return game.utility(state, player), None\n", + " if cutoff(game, state, depth):\n", + " return h(state, player), None\n", + " v, move = +infinity, None\n", + " for a in game.actions(state):\n", + " v2, _ = max_value(game.result(state, a), alpha, beta, depth + 1)\n", + " if v2 < v:\n", + " v, move = v2, a\n", + " beta = min(beta, v)\n", + " if v <= alpha:\n", + " return v, move\n", + " return v, move\n", + "\n", + " return max_value(state, -infinity, +infinity, 0)" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 367 ms, sys: 7.9 ms, total: 375 ms\n", + "Wall time: 375 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "O X X\n", + "X O O\n", + "O X X" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time play_game(TicTacToe(), {'X':player(h_alphabeta_search), 'O':player(h_alphabeta_search)})" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . X\n", + ". . . . . X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O X\n", + ". . . . . X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O X\n", + ". . . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . . O X\n", + ". . . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . X O X\n", + ". . . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . X O X\n", + ". O . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". X . . X O X\n", + ". O . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O O\n", + ". X . . X O X\n", + ". O . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X O O\n", + ". X . . X O X\n", + ". O . . X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + ". . . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . . X .\n", + ". . . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . . X .\n", + ". X . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". X . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + ". O . O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". X . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + ". O O O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". X . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . . .\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . . X\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . . X O O\n", + ". X . . X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . . X\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . . X O O\n", + "O X . . X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . . X O O\n", + "O X . . X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . . X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . . O O\n", + ". X . . . X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . . O O\n", + ". X . . O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . . O O\n", + ". X . X O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . . O O O\n", + ". X . X O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . . X X\n", + ". . . X O O O\n", + ". X . X O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . . O X X\n", + ". . . X O O O\n", + ". X . X O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + ". . . X O X X\n", + ". . . X O O O\n", + ". X . X O X O\n", + ". X . X X O O\n", + "O X . O X O X\n", + "X O O O X X O\n", + "\n", + "CPU times: user 8.82 s, sys: 146 ms, total: 8.96 s\n", + "Wall time: 9.19 s\n" + ] + }, + { + "data": { + "text/plain": [ + "1" + ] + }, + "execution_count": 60, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time play_game(ConnectFour(), {'X':player(h_alphabeta_search), 'O':random_player}, verbose=True).utility" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . . X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . . . X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . O .\n", + ". . O . X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". . O . X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". . O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . X O .\n", + ". X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". X . . . . .\n", + ". O . . X O .\n", + ". X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . . . .\n", + ". X . . . . .\n", + ". O . . X O .\n", + ". X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . . . .\n", + ". X . . . . .\n", + ". O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . . . .\n", + ". X . . . . .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . . . .\n", + ". X . . X . .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . . . .\n", + ". O . . O . .\n", + ". X . . X . .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O . . O . .\n", + ". X . . X . .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O . . O . .\n", + ". X . . X O .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O . . O X .\n", + ". X . . X O .\n", + "O O . . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O . . O X .\n", + ". X . . X O .\n", + "O O O . X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X . .\n", + ". O . . O X .\n", + ". X . . X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". O . . O X .\n", + ". X . . X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". O . . O X .\n", + ". X . X X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . . X O .\n", + ". O . O O X .\n", + ". X . X X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . . . . .\n", + ". . . X X O .\n", + ". O . O O X .\n", + ". X . X X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . O . . .\n", + ". . . X X O .\n", + ". O . O O X .\n", + ". X . X X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + ". . . O . . .\n", + ". . . X X O .\n", + ". O . O O X .\n", + ". X X X X O .\n", + "O O O X X O .\n", + "X X O O X X .\n", + "\n", + "CPU times: user 12.1 s, sys: 237 ms, total: 12.4 s\n", + "Wall time: 12.9 s\n" + ] + }, + { + "data": { + "text/plain": [ + "1" + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time play_game(ConnectFour(), {'X':player(h_alphabeta_search), 'O':player(h_alphabeta_search)}, verbose=True).utility" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Result states: 6,589; Terminal tests: 3,653; for alphabeta_search_tt\n", + "Result states: 25,703; Terminal tests: 25,704; for alphabeta_search\n", + "Result states: 4,687; Terminal tests: 2,805; for h_alphabeta_search\n", + "Result states: 16,167; Terminal tests: 5,478; for minimax_search_tt\n" + ] + } + ], + "source": [ + "class CountCalls:\n", + " \"\"\"Delegate all attribute gets to the object, and count them in ._counts\"\"\"\n", + " def __init__(self, obj):\n", + " self._object = obj\n", + " self._counts = Counter()\n", + " \n", + " def __getattr__(self, attr):\n", + " \"Delegate to the original object, after incrementing a counter.\"\n", + " self._counts[attr] += 1\n", + " return getattr(self._object, attr)\n", + " \n", + "def report(game, searchers):\n", + " for searcher in searchers:\n", + " game = CountCalls(game)\n", + " searcher(game, game.initial)\n", + " print('Result states: {:7,d}; Terminal tests: {:7,d}; for {}'.format(\n", + " game._counts['result'], game._counts['is_terminal'], searcher.__name__))\n", + " \n", + "report(TicTacToe(), (alphabeta_search_tt, alphabeta_search, h_alphabeta_search, minimax_search_tt))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Monte Carlo Tree Search" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "class Node:\n", + " def __init__(self, parent, )\n", + "def mcts(state, game, N=1000):" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Heuristic Search Algorithms" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "t = CountCalls(TicTacToe())\n", + " \n", + "play_game(t, dict(X=minimax_player, O=minimax_player), verbose=True)\n", + "t._counts" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for tactic in (three, fork, center, opposite_corner, corner, any):\n", + " for s in squares:\n", + " if tactic(board, s,player): return s\n", + " for s ins quares:\n", + " if tactic(board, s, opponent): return s" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "\n", + "def ucb(U, N, C=2**0.5, parentN=100):\n", + " return round(U/N + C * math.sqrt(math.log(parentN)/N), 2)\n", + "\n", + "{C: (ucb(60, 79, C), ucb(1, 10, C), ucb(2, 11, C)) \n", + " for C in (1.4, 1.5)}\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def ucb(U, N, parentN=100, C=2):\n", + " return U/N + C * math.sqrt(math.log(parentN)/N)\n", + "\n", + "\n", + "C = 1.4 \n", + "\n", + "class Node:\n", + " def __init__(self, name, children=(), U=0, N=0, parent=None, p=0.5):\n", + " self.__dict__.update(name=name, U=U, N=N, parent=parent, children=children, p=p)\n", + " for c in children:\n", + " c.parent = self\n", + " \n", + " def __repr__(self):\n", + " return '{}:{}/{}={:.0%}{}'.format(self.name, self.U, self.N, self.U/self.N, self.children)\n", + " \n", + "def select(n):\n", + " if n.children:\n", + " return select(max(n.children, key=ucb))\n", + " else:\n", + " return n\n", + " \n", + "def back(n, amount):\n", + " if n:\n", + " n.N += 1\n", + " n.U += amount\n", + " back(n.parent, 1 - amount)\n", + " \n", + " \n", + "def one(root): \n", + " n = select(root)\n", + " amount = int(random.uniform(0, 1) < n.p)\n", + " back(n, amount)\n", + " \n", + "def ucb(n): \n", + " return (float('inf') if n.N == 0 else\n", + " n.U / n.N + C * math.sqrt(math.log(n.parent.N)/n.N))\n", + "\n", + "\n", + "tree = Node('root', [Node('a', p=.8, children=[Node('a1', p=.05), \n", + " Node('a2', p=.25,\n", + " children=[Node('a2a', p=.7), Node('a2b')])]),\n", + " Node('b', p=.5, children=[Node('b1', p=.6,\n", + " children=[Node('b1a', p=.3), Node('b1b')]), \n", + " Node('b2', p=.4)]),\n", + " Node('c', p=.1)])\n", + "\n", + "for i in range(100):\n", + " one(tree); \n", + "for c in tree.children: print(c)\n", + "'select', select(tree), 'tree', tree\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "us = (100, 50, 25, 10, 5, 1)\n", + "infinity = float('inf')\n", + "\n", + "@lru_cache(None)\n", + "def f1(n, denom):\n", + " return (0 if n == 0 else\n", + " infinity if n < 0 or not denom else\n", + " min(1 + f1(n - denom[0], denom),\n", + " f1(n, denom[1:])))\n", + " \n", + "@lru_cache(None)\n", + "def f2(n, denom):\n", + " @lru_cache(None)\n", + " def f(n):\n", + " return (0 if n == 0 else\n", + " infinity if n < 0 else\n", + " 1 + min(f(n - d) for d in denom))\n", + " return f(n)\n", + "\n", + "@lru_cache(None)\n", + "def f3(n, denom):\n", + " return (0 if n == 0 else\n", + " infinity if n < 0 or not denom else\n", + " min(k + f2(n - k * denom[0], denom[1:]) \n", + " for k in range(1 + n // denom[0])))\n", + " \n", + "\n", + "def g(n, d=us): return f1(n, d), f2(n, d), f3(n, d)\n", + " \n", + "n = 12345\n", + "%time f1(n, us)\n", + "%time f2(n, us)\n", + "%time f3(n, us)\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/games4e.py b/games4e.py new file mode 100644 index 000000000..aba5b0eb3 --- /dev/null +++ b/games4e.py @@ -0,0 +1,635 @@ +"""Games or Adversarial Search (Chapter 5)""" + +import copy +import itertools +import random +from collections import namedtuple + +import numpy as np + +from utils4e import vector_add, MCT_Node, ucb + +GameState = namedtuple('GameState', 'to_move, utility, board, moves') +StochasticGameState = namedtuple('StochasticGameState', 'to_move, utility, board, moves, chance') + + +# ______________________________________________________________________________ +# MinMax Search + + +def minmax_decision(state, game): + """Given a state in a game, calculate the best move by searching + forward all the way to the terminal states. [Figure 5.3]""" + + player = game.to_move(state) + + def max_value(state): + if game.terminal_test(state): + return game.utility(state, player) + v = -np.inf + for a in game.actions(state): + v = max(v, min_value(game.result(state, a))) + return v + + def min_value(state): + if game.terminal_test(state): + return game.utility(state, player) + v = np.inf + for a in game.actions(state): + v = min(v, max_value(game.result(state, a))) + return v + + # Body of minmax_decision: + return max(game.actions(state), key=lambda a: min_value(game.result(state, a))) + + +# ______________________________________________________________________________ + + +def expect_minmax(state, game): + """ + [Figure 5.11] + Return the best move for a player after dice are thrown. The game tree + includes chance nodes along with min and max nodes. + """ + player = game.to_move(state) + + def max_value(state): + v = -np.inf + for a in game.actions(state): + v = max(v, chance_node(state, a)) + return v + + def min_value(state): + v = np.inf + for a in game.actions(state): + v = min(v, chance_node(state, a)) + return v + + def chance_node(state, action): + res_state = game.result(state, action) + if game.terminal_test(res_state): + return game.utility(res_state, player) + sum_chances = 0 + num_chances = len(game.chances(res_state)) + for chance in game.chances(res_state): + res_state = game.outcome(res_state, chance) + util = 0 + if res_state.to_move == player: + util = max_value(res_state) + else: + util = min_value(res_state) + sum_chances += util * game.probability(chance) + return sum_chances / num_chances + + # Body of expect_min_max: + return max(game.actions(state), key=lambda a: chance_node(state, a), default=None) + + +def alpha_beta_search(state, game): + """Search game to determine best action; use alpha-beta pruning. + As in [Figure 5.7], this version searches all the way to the leaves.""" + + player = game.to_move(state) + + # Functions used by alpha_beta + def max_value(state, alpha, beta): + if game.terminal_test(state): + return game.utility(state, player) + v = -np.inf + for a in game.actions(state): + v = max(v, min_value(game.result(state, a), alpha, beta)) + if v >= beta: + return v + alpha = max(alpha, v) + return v + + def min_value(state, alpha, beta): + if game.terminal_test(state): + return game.utility(state, player) + v = np.inf + for a in game.actions(state): + v = min(v, max_value(game.result(state, a), alpha, beta)) + if v <= alpha: + return v + beta = min(beta, v) + return v + + # Body of alpha_beta_search: + best_score = -np.inf + beta = np.inf + best_action = None + for a in game.actions(state): + v = min_value(game.result(state, a), best_score, beta) + if v > best_score: + best_score = v + best_action = a + return best_action + + +def alpha_beta_cutoff_search(state, game, d=4, cutoff_test=None, eval_fn=None): + """Search game to determine best action; use alpha-beta pruning. + This version cuts off search and uses an evaluation function.""" + + player = game.to_move(state) + + # Functions used by alpha_beta + def max_value(state, alpha, beta, depth): + if cutoff_test(state, depth): + return eval_fn(state) + v = -np.inf + for a in game.actions(state): + v = max(v, min_value(game.result(state, a), alpha, beta, depth + 1)) + if v >= beta: + return v + alpha = max(alpha, v) + return v + + def min_value(state, alpha, beta, depth): + if cutoff_test(state, depth): + return eval_fn(state) + v = np.inf + for a in game.actions(state): + v = min(v, max_value(game.result(state, a), alpha, beta, depth + 1)) + if v <= alpha: + return v + beta = min(beta, v) + return v + + # Body of alpha_beta_cutoff_search starts here: + # The default test cuts off at depth d or at a terminal state + cutoff_test = (cutoff_test or (lambda state, depth: depth > d or game.terminal_test(state))) + eval_fn = eval_fn or (lambda state: game.utility(state, player)) + best_score = -np.inf + beta = np.inf + best_action = None + for a in game.actions(state): + v = min_value(game.result(state, a), best_score, beta, 1) + if v > best_score: + best_score = v + best_action = a + return best_action + + +# ______________________________________________________________________________ +# Monte Carlo Tree Search + + +def monte_carlo_tree_search(state, game, N=1000): + def select(n): + """select a leaf node in the tree""" + if n.children: + return select(max(n.children.keys(), key=ucb)) + else: + return n + + def expand(n): + """expand the leaf node by adding all its children states""" + if not n.children and not game.terminal_test(n.state): + n.children = {MCT_Node(state=game.result(n.state, action), parent=n): action + for action in game.actions(n.state)} + return select(n) + + def simulate(game, state): + """simulate the utility of current state by random picking a step""" + player = game.to_move(state) + while not game.terminal_test(state): + action = random.choice(list(game.actions(state))) + state = game.result(state, action) + v = game.utility(state, player) + return -v + + def backprop(n, utility): + """passing the utility back to all parent nodes""" + if utility > 0: + n.U += utility + # if utility == 0: + # n.U += 0.5 + n.N += 1 + if n.parent: + backprop(n.parent, -utility) + + root = MCT_Node(state=state) + + for _ in range(N): + leaf = select(root) + child = expand(leaf) + result = simulate(game, child.state) + backprop(child, result) + + max_state = max(root.children, key=lambda p: p.N) + + return root.children.get(max_state) + + +# ______________________________________________________________________________ +# Players for Games + + +def query_player(game, state): + """Make a move by querying standard input.""" + print("current state:") + game.display(state) + print("available moves: {}".format(game.actions(state))) + print("") + move = None + if game.actions(state): + move_string = input('Your move? ') + try: + move = eval(move_string) + except NameError: + move = move_string + else: + print('no legal moves: passing turn to next player') + return move + + +def random_player(game, state): + """A player that chooses a legal move at random.""" + return random.choice(game.actions(state)) if game.actions(state) else None + + +def alpha_beta_player(game, state): + return alpha_beta_search(state, game) + + +def expect_min_max_player(game, state): + return expect_minmax(state, game) + + +def mcts_player(game, state): + return monte_carlo_tree_search(state, game) + + +# ______________________________________________________________________________ +# Some Sample Games + + +class Game: + """A game is similar to a problem, but it has a utility for each + state and a terminal test instead of a path cost and a goal + test. To create a game, subclass this class and implement actions, + result, utility, and terminal_test. You may override display and + successors or you can inherit their default methods. You will also + need to set the .initial attribute to the initial state; this can + be done in the constructor.""" + + def actions(self, state): + """Return a list of the allowable moves at this point.""" + raise NotImplementedError + + def result(self, state, move): + """Return the state that results from making a move from a state.""" + raise NotImplementedError + + def utility(self, state, player): + """Return the value of this final state to player.""" + raise NotImplementedError + + def terminal_test(self, state): + """Return True if this is a final state for the game.""" + return not self.actions(state) + + def to_move(self, state): + """Return the player whose move it is in this state.""" + return state.to_move + + def display(self, state): + """Print or otherwise display the state.""" + print(state) + + def __repr__(self): + return '<{}>'.format(self.__class__.__name__) + + def play_game(self, *players): + """Play an n-person, move-alternating game.""" + state = self.initial + while True: + for player in players: + move = player(self, state) + state = self.result(state, move) + if self.terminal_test(state): + self.display(state) + return self.utility(state, self.to_move(self.initial)) + + +class StochasticGame(Game): + """A stochastic game includes uncertain events which influence + the moves of players at each state. To create a stochastic game, subclass + this class and implement chances and outcome along with the other + unimplemented game class methods.""" + + def chances(self, state): + """Return a list of all possible uncertain events at a state.""" + raise NotImplementedError + + def outcome(self, state, chance): + """Return the state which is the outcome of a chance trial.""" + raise NotImplementedError + + def probability(self, chance): + """Return the probability of occurrence of a chance.""" + raise NotImplementedError + + def play_game(self, *players): + """Play an n-person, move-alternating stochastic game.""" + state = self.initial + while True: + for player in players: + chance = random.choice(self.chances(state)) + state = self.outcome(state, chance) + move = player(self, state) + state = self.result(state, move) + if self.terminal_test(state): + self.display(state) + return self.utility(state, self.to_move(self.initial)) + + +class Fig52Game(Game): + """The game represented in [Figure 5.2]. Serves as a simple test case.""" + + succs = dict(A=dict(a1='B', a2='C', a3='D'), + B=dict(b1='B1', b2='B2', b3='B3'), + C=dict(c1='C1', c2='C2', c3='C3'), + D=dict(d1='D1', d2='D2', d3='D3')) + utils = dict(B1=3, B2=12, B3=8, C1=2, C2=4, C3=6, D1=14, D2=5, D3=2) + initial = 'A' + + def actions(self, state): + return list(self.succs.get(state, {}).keys()) + + def result(self, state, move): + return self.succs[state][move] + + def utility(self, state, player): + if player == 'MAX': + return self.utils[state] + else: + return -self.utils[state] + + def terminal_test(self, state): + return state not in ('A', 'B', 'C', 'D') + + def to_move(self, state): + return 'MIN' if state in 'BCD' else 'MAX' + + +class Fig52Extended(Game): + """Similar to Fig52Game but bigger. Useful for visualisation""" + + succs = {i: dict(l=i * 3 + 1, m=i * 3 + 2, r=i * 3 + 3) for i in range(13)} + utils = dict() + + def actions(self, state): + return sorted(list(self.succs.get(state, {}).keys())) + + def result(self, state, move): + return self.succs[state][move] + + def utility(self, state, player): + if player == 'MAX': + return self.utils[state] + else: + return -self.utils[state] + + def terminal_test(self, state): + return state not in range(13) + + def to_move(self, state): + return 'MIN' if state in {1, 2, 3} else 'MAX' + + +class TicTacToe(Game): + """Play TicTacToe on an h x v board, with Max (first player) playing 'X'. + A state has the player to move, a cached utility, a list of moves in + the form of a list of (x, y) positions, and a board, in the form of + a dict of {(x, y): Player} entries, where Player is 'X' or 'O'.""" + + def __init__(self, h=3, v=3, k=3): + self.h = h + self.v = v + self.k = k + moves = [(x, y) for x in range(1, h + 1) + for y in range(1, v + 1)] + self.initial = GameState(to_move='X', utility=0, board={}, moves=moves) + + def actions(self, state): + """Legal moves are any square not yet taken.""" + return state.moves + + def result(self, state, move): + if move not in state.moves: + return state # Illegal move has no effect + board = state.board.copy() + board[move] = state.to_move + moves = list(state.moves) + moves.remove(move) + return GameState(to_move=('O' if state.to_move == 'X' else 'X'), + utility=self.compute_utility(board, move, state.to_move), + board=board, moves=moves) + + def utility(self, state, player): + """Return the value to player; 1 for win, -1 for loss, 0 otherwise.""" + return state.utility if player == 'X' else -state.utility + + def terminal_test(self, state): + """A state is terminal if it is won or there are no empty squares.""" + return state.utility != 0 or len(state.moves) == 0 + + def display(self, state): + board = state.board + for x in range(1, self.h + 1): + for y in range(1, self.v + 1): + print(board.get((x, y), '.'), end=' ') + print() + + def compute_utility(self, board, move, player): + """If 'X' wins with this move, return 1; if 'O' wins return -1; else return 0.""" + if (self.k_in_row(board, move, player, (0, 1)) or + self.k_in_row(board, move, player, (1, 0)) or + self.k_in_row(board, move, player, (1, -1)) or + self.k_in_row(board, move, player, (1, 1))): + return +1 if player == 'X' else -1 + else: + return 0 + + def k_in_row(self, board, move, player, delta_x_y): + """Return true if there is a line through move on board for player.""" + (delta_x, delta_y) = delta_x_y + x, y = move + n = 0 # n is number of moves in row + while board.get((x, y)) == player: + n += 1 + x, y = x + delta_x, y + delta_y + x, y = move + while board.get((x, y)) == player: + n += 1 + x, y = x - delta_x, y - delta_y + n -= 1 # Because we counted move itself twice + return n >= self.k + + +class ConnectFour(TicTacToe): + """A TicTacToe-like game in which you can only make a move on the bottom + row, or in a square directly above an occupied square. Traditionally + played on a 7x6 board and requiring 4 in a row.""" + + def __init__(self, h=7, v=6, k=4): + TicTacToe.__init__(self, h, v, k) + + def actions(self, state): + return [(x, y) for (x, y) in state.moves + if y == 1 or (x, y - 1) in state.board] + + +class Backgammon(StochasticGame): + """A two player game where the goal of each player is to move all the + checkers off the board. The moves for each state are determined by + rolling a pair of dice.""" + + def __init__(self): + """Initial state of the game""" + point = {'W': 0, 'B': 0} + board = [point.copy() for index in range(24)] + board[0]['B'] = board[23]['W'] = 2 + board[5]['W'] = board[18]['B'] = 5 + board[7]['W'] = board[16]['B'] = 3 + board[11]['B'] = board[12]['W'] = 5 + self.allow_bear_off = {'W': False, 'B': False} + self.direction = {'W': -1, 'B': 1} + self.initial = StochasticGameState(to_move='W', + utility=0, + board=board, + moves=self.get_all_moves(board, 'W'), chance=None) + + def actions(self, state): + """Return a list of legal moves for a state.""" + player = state.to_move + moves = state.moves + if len(moves) == 1 and len(moves[0]) == 1: + return moves + legal_moves = [] + for move in moves: + board = copy.deepcopy(state.board) + if self.is_legal_move(board, move, state.chance, player): + legal_moves.append(move) + return legal_moves + + def result(self, state, move): + board = copy.deepcopy(state.board) + player = state.to_move + self.move_checker(board, move[0], state.chance[0], player) + if len(move) == 2: + self.move_checker(board, move[1], state.chance[1], player) + to_move = ('W' if player == 'B' else 'B') + return StochasticGameState(to_move=to_move, + utility=self.compute_utility(board, move, player), + board=board, + moves=self.get_all_moves(board, to_move), chance=None) + + def utility(self, state, player): + """Return the value to player; 1 for win, -1 for loss, 0 otherwise.""" + return state.utility if player == 'W' else -state.utility + + def terminal_test(self, state): + """A state is terminal if one player wins.""" + return state.utility != 0 + + def get_all_moves(self, board, player): + """All possible moves for a player i.e. all possible ways of + choosing two checkers of a player from the board for a move + at a given state.""" + all_points = board + taken_points = [index for index, point in enumerate(all_points) + if point[player] > 0] + if self.checkers_at_home(board, player) == 1: + return [(taken_points[0],)] + moves = list(itertools.permutations(taken_points, 2)) + moves = moves + [(index, index) for index, point in enumerate(all_points) + if point[player] >= 2] + return moves + + def display(self, state): + """Display state of the game.""" + board = state.board + player = state.to_move + print("current state : ") + for index, point in enumerate(board): + print("point : ", index, " W : ", point['W'], " B : ", point['B']) + print("to play : ", player) + + def compute_utility(self, board, move, player): + """If 'W' wins with this move, return 1; if 'B' wins return -1; else return 0.""" + util = {'W': 1, 'B': -1} + for idx in range(0, 24): + if board[idx][player] > 0: + return 0 + return util[player] + + def checkers_at_home(self, board, player): + """Return the no. of checkers at home for a player.""" + sum_range = range(0, 7) if player == 'W' else range(17, 24) + count = 0 + for idx in sum_range: + count = count + board[idx][player] + return count + + def is_legal_move(self, board, start, steps, player): + """Move is a tuple which contains starting points of checkers to be + moved during a player's turn. An on-board move is legal if both the destinations + are open. A bear-off move is the one where a checker is moved off-board. + It is legal only after a player has moved all his checkers to his home.""" + dest1, dest2 = vector_add(start, steps) + dest_range = range(0, 24) + move1_legal = move2_legal = False + if dest1 in dest_range: + if self.is_point_open(player, board[dest1]): + self.move_checker(board, start[0], steps[0], player) + move1_legal = True + else: + if self.allow_bear_off[player]: + self.move_checker(board, start[0], steps[0], player) + move1_legal = True + if not move1_legal: + return False + if dest2 in dest_range: + if self.is_point_open(player, board[dest2]): + move2_legal = True + else: + if self.allow_bear_off[player]: + move2_legal = True + return move1_legal and move2_legal + + def move_checker(self, board, start, steps, player): + """Move a checker from starting point by a given number of steps""" + dest = start + steps + dest_range = range(0, 24) + board[start][player] -= 1 + if dest in dest_range: + board[dest][player] += 1 + if self.checkers_at_home(board, player) == 15: + self.allow_bear_off[player] = True + + def is_point_open(self, player, point): + """A point is open for a player if the no. of opponent's + checkers already present on it is 0 or 1. A player can + move a checker to a point only if it is open.""" + opponent = 'B' if player == 'W' else 'W' + return point[opponent] <= 1 + + def chances(self, state): + """Return a list of all possible dice rolls at a state.""" + dice_rolls = list(itertools.combinations_with_replacement([1, 2, 3, 4, 5, 6], 2)) + return dice_rolls + + def outcome(self, state, chance): + """Return the state which is the outcome of a dice roll.""" + dice = tuple(map((self.direction[state.to_move]).__mul__, chance)) + return StochasticGameState(to_move=state.to_move, + utility=state.utility, + board=state.board, + moves=state.moves, chance=dice) + + def probability(self, chance): + """Return the probability of occurrence of a dice roll.""" + return 1 / 36 if chance[0] == chance[1] else 1 / 18 diff --git a/gui/eight_puzzle.py b/gui/eight_puzzle.py index 82acced03..5733228d7 100644 --- a/gui/eight_puzzle.py +++ b/gui/eight_puzzle.py @@ -1,138 +1,151 @@ -# author ad71 -from tkinter import * +import os.path +import random +import time from functools import partial +from tkinter import * -import time -import random -import numpy as np +from search import astar_search, EightPuzzle -import sys -import os.path sys.path.append(os.path.join(os.path.dirname(__file__), '..')) -from search import astar_search, EightPuzzle -import utils - root = Tk() state = [1, 2, 3, 4, 5, 6, 7, 8, 0] puzzle = EightPuzzle(tuple(state)) solution = None -b = [None]*9 +b = [None] * 9 + # TODO: refactor into OOP, remove global variables def scramble(): - """ Scrambles the puzzle starting from the goal state """ + """Scrambles the puzzle starting from the goal state""" + + global state + global puzzle + possible_actions = ['UP', 'DOWN', 'LEFT', 'RIGHT'] + scramble = [] + for _ in range(60): + scramble.append(random.choice(possible_actions)) - global state - global puzzle - possible_actions = ['UP', 'DOWN', 'LEFT', 'RIGHT'] - scramble = [] - for _ in range(60): - scramble.append(random.choice(possible_actions)) + for move in scramble: + if move in puzzle.actions(state): + state = list(puzzle.result(state, move)) + puzzle = EightPuzzle(tuple(state)) + create_buttons() - for move in scramble: - if move in puzzle.actions(state): - state = list(puzzle.result(state, move)) - puzzle = EightPuzzle(tuple(state)) - create_buttons() def solve(): - """ Solves the puzzle using astar_search """ + """Solves the puzzle using astar_search""" + + return astar_search(puzzle).solution() - return astar_search(puzzle).solution() def solve_steps(): - """ Solves the puzzle step by step """ - - global puzzle - global solution - global state - solution = solve() - print(solution) - - for move in solution: - state = puzzle.result(state, move) - create_buttons() - root.update() - root.after(1, time.sleep(0.75)) + """Solves the puzzle step by step""" + + global puzzle + global solution + global state + solution = solve() + print(solution) + + for move in solution: + state = puzzle.result(state, move) + create_buttons() + root.update() + root.after(1, time.sleep(0.75)) + def exchange(index): - """ Interchanges the position of the selected tile with the zero tile under certain conditions """ - - global state - global solution - global puzzle - zero_ix = list(state).index(0) - actions = puzzle.actions(state) - current_action = '' - i_diff = index//3 - zero_ix//3 - j_diff = index%3 - zero_ix%3 - if i_diff == 1: - current_action += 'DOWN' - elif i_diff == -1: - current_action += 'UP' - - if j_diff == 1: - current_action += 'RIGHT' - elif j_diff == -1: - current_action += 'LEFT' - - if abs(i_diff) + abs(j_diff) != 1: - current_action = '' - - if current_action in actions: - b[zero_ix].grid_forget() - b[zero_ix] = Button(root, text=f'{state[index]}', width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, zero_ix)) - b[zero_ix].grid(row=zero_ix//3, column=zero_ix%3, ipady=40) - b[index].grid_forget() - b[index] = Button(root, text=None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, index)) - b[index].grid(row=index//3, column=index%3, ipady=40) - state[zero_ix], state[index] = state[index], state[zero_ix] - puzzle = EightPuzzle(tuple(state)) + """Interchanges the position of the selected tile with the zero tile under certain conditions""" + + global state + global solution + global puzzle + zero_ix = list(state).index(0) + actions = puzzle.actions(state) + current_action = '' + i_diff = index // 3 - zero_ix // 3 + j_diff = index % 3 - zero_ix % 3 + if i_diff == 1: + current_action += 'DOWN' + elif i_diff == -1: + current_action += 'UP' + + if j_diff == 1: + current_action += 'RIGHT' + elif j_diff == -1: + current_action += 'LEFT' + + if abs(i_diff) + abs(j_diff) != 1: + current_action = '' + + if current_action in actions: + b[zero_ix].grid_forget() + b[zero_ix] = Button(root, text=f'{state[index]}', width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, zero_ix)) + b[zero_ix].grid(row=zero_ix // 3, column=zero_ix % 3, ipady=40) + b[index].grid_forget() + b[index] = Button(root, text=None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, index)) + b[index].grid(row=index // 3, column=index % 3, ipady=40) + state[zero_ix], state[index] = state[index], state[zero_ix] + puzzle = EightPuzzle(tuple(state)) + def create_buttons(): - """ Creates dynamic buttons """ - - # TODO: Find a way to use grid_forget() with a for loop for initialization - b[0] = Button(root, text=f'{state[0]}' if state[0] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 0)) - b[0].grid(row=0, column=0, ipady=40) - b[1] = Button(root, text=f'{state[1]}' if state[1] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 1)) - b[1].grid(row=0, column=1, ipady=40) - b[2] = Button(root, text=f'{state[2]}' if state[2] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 2)) - b[2].grid(row=0, column=2, ipady=40) - b[3] = Button(root, text=f'{state[3]}' if state[3] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 3)) - b[3].grid(row=1, column=0, ipady=40) - b[4] = Button(root, text=f'{state[4]}' if state[4] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 4)) - b[4].grid(row=1, column=1, ipady=40) - b[5] = Button(root, text=f'{state[5]}' if state[5] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 5)) - b[5].grid(row=1, column=2, ipady=40) - b[6] = Button(root, text=f'{state[6]}' if state[6] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 6)) - b[6].grid(row=2, column=0, ipady=40) - b[7] = Button(root, text=f'{state[7]}' if state[7] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 7)) - b[7].grid(row=2, column=1, ipady=40) - b[8] = Button(root, text=f'{state[8]}' if state[8] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), command=partial(exchange, 8)) - b[8].grid(row=2, column=2, ipady=40) + """Creates dynamic buttons""" + + # TODO: Find a way to use grid_forget() with a for loop for initialization + b[0] = Button(root, text=f'{state[0]}' if state[0] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 0)) + b[0].grid(row=0, column=0, ipady=40) + b[1] = Button(root, text=f'{state[1]}' if state[1] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 1)) + b[1].grid(row=0, column=1, ipady=40) + b[2] = Button(root, text=f'{state[2]}' if state[2] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 2)) + b[2].grid(row=0, column=2, ipady=40) + b[3] = Button(root, text=f'{state[3]}' if state[3] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 3)) + b[3].grid(row=1, column=0, ipady=40) + b[4] = Button(root, text=f'{state[4]}' if state[4] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 4)) + b[4].grid(row=1, column=1, ipady=40) + b[5] = Button(root, text=f'{state[5]}' if state[5] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 5)) + b[5].grid(row=1, column=2, ipady=40) + b[6] = Button(root, text=f'{state[6]}' if state[6] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 6)) + b[6].grid(row=2, column=0, ipady=40) + b[7] = Button(root, text=f'{state[7]}' if state[7] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 7)) + b[7].grid(row=2, column=1, ipady=40) + b[8] = Button(root, text=f'{state[8]}' if state[8] != 0 else None, width=6, font=('Helvetica', 40, 'bold'), + command=partial(exchange, 8)) + b[8].grid(row=2, column=2, ipady=40) + def create_static_buttons(): - """ Creates scramble and solve buttons """ + """Creates scramble and solve buttons""" + + scramble_btn = Button(root, text='Scramble', font=('Helvetica', 30, 'bold'), width=8, command=partial(init)) + scramble_btn.grid(row=3, column=0, ipady=10) + solve_btn = Button(root, text='Solve', font=('Helvetica', 30, 'bold'), width=8, command=partial(solve_steps)) + solve_btn.grid(row=3, column=2, ipady=10) - scramble_btn = Button(root, text='Scramble', font=('Helvetica', 30, 'bold'), width=8, command=partial(init)) - scramble_btn.grid(row=3, column=0, ipady=10) - solve_btn = Button(root, text='Solve', font=('Helvetica', 30, 'bold'), width=8, command=partial(solve_steps)) - solve_btn.grid(row=3, column=2, ipady=10) def init(): - """ Calls necessary functions """ - - global state - global solution - state = [1, 2, 3, 4, 5, 6, 7, 8, 0] - scramble() - create_buttons() - create_static_buttons() + """Calls necessary functions""" + + global state + global solution + state = [1, 2, 3, 4, 5, 6, 7, 8, 0] + scramble() + create_buttons() + create_static_buttons() + init() root.mainloop() diff --git a/gui/genetic_algorithm_example.py b/gui/genetic_algorithm_example.py index 418da02e9..c987151c8 100644 --- a/gui/genetic_algorithm_example.py +++ b/gui/genetic_algorithm_example.py @@ -1,4 +1,3 @@ -# author: ad71 # A simple program that implements the solution to the phrase generation problem using # genetic algorithms as given in the search.ipynb notebook. # @@ -9,17 +8,13 @@ # Displays a progress bar that indicates the amount of completion of the algorithm # Displays the first few individuals of the current generation -import sys -import time -import random import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) - from tkinter import * from tkinter import ttk import search -from utils import argmax + +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) LARGE_FONT = ('Verdana', 12) EXTRA_LARGE_FONT = ('Consolas', 36, 'bold') @@ -34,20 +29,20 @@ # genetic algorithm variables # feel free to play around with these -target = 'Genetic Algorithm' # the phrase to be generated -max_population = 100 # number of samples in each population -mutation_rate = 0.1 # probability of mutation -f_thres = len(target) # fitness threshold -ngen = 1200 # max number of generations to run the genetic algorithm +target = 'Genetic Algorithm' # the phrase to be generated +max_population = 100 # number of samples in each population +mutation_rate = 0.1 # probability of mutation +f_thres = len(target) # fitness threshold +ngen = 1200 # max number of generations to run the genetic algorithm -generation = 0 # counter to keep track of generation number +generation = 0 # counter to keep track of generation number -u_case = [chr(x) for x in range(65, 91)] # list containing all uppercase characters -l_case = [chr(x) for x in range(97, 123)] # list containing all lowercase characters -punctuations1 = [chr(x) for x in range(33, 48)] # lists containing punctuation symbols +u_case = [chr(x) for x in range(65, 91)] # list containing all uppercase characters +l_case = [chr(x) for x in range(97, 123)] # list containing all lowercase characters +punctuations1 = [chr(x) for x in range(33, 48)] # lists containing punctuation symbols punctuations2 = [chr(x) for x in range(58, 65)] punctuations3 = [chr(x) for x in range(91, 97)] -numerals = [chr(x) for x in range(48, 58)] # list containing numbers +numerals = [chr(x) for x in range(48, 58)] # list containing numbers # extend the gene pool with the required lists and append the space character gene_pool = [] @@ -55,44 +50,51 @@ gene_pool.extend(l_case) gene_pool.append(' ') + # callbacks to update global variables from the slider values def update_max_population(slider_value): - global max_population - max_population = slider_value + global max_population + max_population = slider_value + def update_mutation_rate(slider_value): - global mutation_rate - mutation_rate = slider_value + global mutation_rate + mutation_rate = slider_value + def update_f_thres(slider_value): - global f_thres - f_thres = slider_value + global f_thres + f_thres = slider_value + def update_ngen(slider_value): - global ngen - ngen = slider_value + global ngen + ngen = slider_value + # fitness function def fitness_fn(_list): - fitness = 0 - # create string from list of characters - phrase = ''.join(_list) - # add 1 to fitness value for every matching character - for i in range(len(phrase)): - if target[i] == phrase[i]: - fitness += 1 - return fitness + fitness = 0 + # create string from list of characters + phrase = ''.join(_list) + # add 1 to fitness value for every matching character + for i in range(len(phrase)): + if target[i] == phrase[i]: + fitness += 1 + return fitness + # function to bring a new frame on top def raise_frame(frame, init=False, update_target=False, target_entry=None, f_thres_slider=None): - frame.tkraise() - global target - if update_target and target_entry is not None: - target = target_entry.get() - f_thres_slider.config(to=len(target)) - if init: - population = search.init_population(max_population, gene_pool, len(target)) - genetic_algorithm_stepwise(population) + frame.tkraise() + global target + if update_target and target_entry is not None: + target = target_entry.get() + f_thres_slider.config(to=len(target)) + if init: + population = search.init_population(max_population, gene_pool, len(target)) + genetic_algorithm_stepwise(population) + # defining root and child frames root = Tk() @@ -101,7 +103,7 @@ def raise_frame(frame, init=False, update_target=False, target_entry=None, f_thr # pack frames on top of one another for frame in (f1, f2): - frame.grid(row=0, column=0, sticky='news') + frame.grid(row=0, column=0, sticky='news') # Home Screen (f1) widgets target_entry = Entry(f1, font=('Consolas 46 bold'), exportselection=0, foreground=p_blue, justify=CENTER) @@ -109,64 +111,79 @@ def raise_frame(frame, init=False, update_target=False, target_entry=None, f_thr target_entry.pack(expand=YES, side=TOP, fill=X, padx=50) target_entry.focus_force() -max_population_slider = Scale(f1, from_=3, to=1000, orient=HORIZONTAL, label='Max population', command=lambda value: update_max_population(int(value))) +max_population_slider = Scale(f1, from_=3, to=1000, orient=HORIZONTAL, label='Max population', + command=lambda value: update_max_population(int(value))) max_population_slider.set(max_population) max_population_slider.pack(expand=YES, side=TOP, fill=X, padx=40) -mutation_rate_slider = Scale(f1, from_=0, to=1, orient=HORIZONTAL, label='Mutation rate', resolution=0.0001, command=lambda value: update_mutation_rate(float(value))) +mutation_rate_slider = Scale(f1, from_=0, to=1, orient=HORIZONTAL, label='Mutation rate', resolution=0.0001, + command=lambda value: update_mutation_rate(float(value))) mutation_rate_slider.set(mutation_rate) mutation_rate_slider.pack(expand=YES, side=TOP, fill=X, padx=40) -f_thres_slider = Scale(f1, from_=0, to=len(target), orient=HORIZONTAL, label='Fitness threshold', command=lambda value: update_f_thres(int(value))) +f_thres_slider = Scale(f1, from_=0, to=len(target), orient=HORIZONTAL, label='Fitness threshold', + command=lambda value: update_f_thres(int(value))) f_thres_slider.set(f_thres) f_thres_slider.pack(expand=YES, side=TOP, fill=X, padx=40) -ngen_slider = Scale(f1, from_=1, to=5000, orient=HORIZONTAL, label='Max number of generations', command=lambda value: update_ngen(int(value))) +ngen_slider = Scale(f1, from_=1, to=5000, orient=HORIZONTAL, label='Max number of generations', + command=lambda value: update_ngen(int(value))) ngen_slider.set(ngen) ngen_slider.pack(expand=YES, side=TOP, fill=X, padx=40) -button = ttk.Button(f1, text='RUN', command=lambda: raise_frame(f2, init=True, update_target=True, target_entry=target_entry, f_thres_slider=f_thres_slider)).pack(side=BOTTOM, pady=50) +button = ttk.Button(f1, text='RUN', + command=lambda: raise_frame(f2, init=True, update_target=True, target_entry=target_entry, + f_thres_slider=f_thres_slider)).pack(side=BOTTOM, pady=50) # f2 widgets canvas = Canvas(f2, width=canvas_width, height=canvas_height) canvas.pack(expand=YES, fill=BOTH, padx=20, pady=15) button = ttk.Button(f2, text='EXIT', command=lambda: raise_frame(f1)).pack(side=BOTTOM, pady=15) + # function to run the genetic algorithm and update text on the canvas def genetic_algorithm_stepwise(population): - root.title('Genetic Algorithm') - for generation in range(ngen): - # generating new population after selecting, recombining and mutating the existing population - population = [search.mutate(search.recombine(*search.select(2, population, fitness_fn)), gene_pool, mutation_rate) for i in range(len(population))] - # genome with the highest fitness in the current generation - current_best = ''.join(argmax(population, key=fitness_fn)) - # collecting first few examples from the current population - members = [''.join(x) for x in population][:48] - - # clear the canvas - canvas.delete('all') - # displays current best on top of the screen - canvas.create_text(canvas_width / 2, 40, fill=p_blue, font='Consolas 46 bold', text=current_best) - - # displaying a part of the population on the screen - for i in range(len(members) // 3): - canvas.create_text((canvas_width * .175), (canvas_height * .25 + (25 * i)), fill=lp_blue, font='Consolas 16', text=members[3 * i]) - canvas.create_text((canvas_width * .500), (canvas_height * .25 + (25 * i)), fill=lp_blue, font='Consolas 16', text=members[3 * i + 1]) - canvas.create_text((canvas_width * .825), (canvas_height * .25 + (25 * i)), fill=lp_blue, font='Consolas 16', text=members[3 * i + 2]) - - # displays current generation number - canvas.create_text((canvas_width * .5), (canvas_height * 0.95), fill=p_blue, font='Consolas 18 bold', text=f'Generation {generation}') - - # displays blue bar that indicates current maximum fitness compared to maximum possible fitness - scaling_factor = fitness_fn(current_best) / len(target) - canvas.create_rectangle(canvas_width * 0.1, 90, canvas_width * 0.9, 100, outline=p_blue) - canvas.create_rectangle(canvas_width * 0.1, 90, canvas_width * 0.1 + scaling_factor * canvas_width * 0.8, 100, fill=lp_blue) - canvas.update() - - # checks for completion - fittest_individual = search.fitness_threshold(fitness_fn, f_thres, population) - if fittest_individual: - break + root.title('Genetic Algorithm') + for generation in range(ngen): + # generating new population after selecting, recombining and mutating the existing population + population = [ + search.mutate(search.recombine(*search.select(2, population, fitness_fn)), gene_pool, mutation_rate) for i + in range(len(population))] + # genome with the highest fitness in the current generation + current_best = ''.join(max(population, key=fitness_fn)) + # collecting first few examples from the current population + members = [''.join(x) for x in population][:48] + + # clear the canvas + canvas.delete('all') + # displays current best on top of the screen + canvas.create_text(canvas_width / 2, 40, fill=p_blue, font='Consolas 46 bold', text=current_best) + + # displaying a part of the population on the screen + for i in range(len(members) // 3): + canvas.create_text((canvas_width * .175), (canvas_height * .25 + (25 * i)), fill=lp_blue, + font='Consolas 16', text=members[3 * i]) + canvas.create_text((canvas_width * .500), (canvas_height * .25 + (25 * i)), fill=lp_blue, + font='Consolas 16', text=members[3 * i + 1]) + canvas.create_text((canvas_width * .825), (canvas_height * .25 + (25 * i)), fill=lp_blue, + font='Consolas 16', text=members[3 * i + 2]) + + # displays current generation number + canvas.create_text((canvas_width * .5), (canvas_height * 0.95), fill=p_blue, font='Consolas 18 bold', + text=f'Generation {generation}') + + # displays blue bar that indicates current maximum fitness compared to maximum possible fitness + scaling_factor = fitness_fn(current_best) / len(target) + canvas.create_rectangle(canvas_width * 0.1, 90, canvas_width * 0.9, 100, outline=p_blue) + canvas.create_rectangle(canvas_width * 0.1, 90, canvas_width * 0.1 + scaling_factor * canvas_width * 0.8, 100, + fill=lp_blue) + canvas.update() + + # checks for completion + fittest_individual = search.fitness_threshold(fitness_fn, f_thres, population) + if fittest_individual: + break + raise_frame(f1) -root.mainloop() \ No newline at end of file +root.mainloop() diff --git a/gui/grid_mdp.py b/gui/grid_mdp.py index d975ba5df..e60b49247 100644 --- a/gui/grid_mdp.py +++ b/gui/grid_mdp.py @@ -1,26 +1,22 @@ -# author: ad71 +import os.path +import sys import tkinter as tk import tkinter.messagebox -from tkinter import ttk - from functools import partial - -import sys -import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) - -from mdp import * -import utils -import numpy as np -import time +from tkinter import ttk import matplotlib import matplotlib.animation as animation +from matplotlib import pyplot as plt +from matplotlib import style from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg -from matplotlib.ticker import MaxNLocator from matplotlib.figure import Figure -from matplotlib import style -from matplotlib import pyplot as plt +from matplotlib.ticker import MaxNLocator + +from mdp import * + +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) + matplotlib.use('TkAgg') style.use('ggplot') @@ -41,611 +37,640 @@ green8 = '#008080' green4 = '#004040' +cell_window_mantainer = None + def extents(f): - ''' adjusts axis markers for heatmap ''' + """adjusts axis markers for heatmap""" + + delta = f[1] - f[0] + return [f[0] - delta / 2, f[-1] + delta / 2] - delta = f[1] - f[0] - return [f[0] - delta/2, f[-1] + delta/2] def display(gridmdp, _height, _width): - ''' displays matrix ''' + """displays matrix""" + + dialog = tk.Toplevel() + dialog.wm_title('Values') - dialog = tk.Toplevel() - dialog.wm_title('Values') + container = tk.Frame(dialog) + container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) - container = tk.Frame(dialog) - container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) + for i in range(max(1, _height)): + for j in range(max(1, _width)): + label = ttk.Label(container, text=f'{gridmdp[_height - i - 1][j]:.3f}', font=('Helvetica', 12)) + label.grid(row=i + 1, column=j + 1, padx=3, pady=3) - for i in range(max(1, _height)): - for j in range(max(1, _width)): - label = ttk.Label(container, text=f'{gridmdp[_height - i - 1][j]:.3f}', font=('Helvetica', 12)) - label.grid(row=i + 1, column=j + 1, padx=3, pady=3) + dialog.mainloop() - dialog.mainloop() def display_best_policy(_best_policy, _height, _width): - ''' displays best policy ''' + """displays best policy""" + dialog = tk.Toplevel() + dialog.wm_title('Best Policy') - dialog = tk.Toplevel() - dialog.wm_title('Best Policy') + container = tk.Frame(dialog) + container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) - container = tk.Frame(dialog) - container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) + for i in range(max(1, _height)): + for j in range(max(1, _width)): + label = ttk.Label(container, text=_best_policy[i][j], font=('Helvetica', 12, 'bold')) + label.grid(row=i + 1, column=j + 1, padx=3, pady=3) - for i in range(max(1, _height)): - for j in range(max(1, _width)): - label = ttk.Label(container, text=_best_policy[i][j], font=('Helvetica', 12, 'bold')) - label.grid(row=i + 1, column=j + 1, padx=3, pady=3) + dialog.mainloop() - dialog.mainloop() def initialize_dialogbox(_width, _height, gridmdp, terminals, buttons): - ''' creates dialogbox for initialization ''' - - dialog = tk.Toplevel() - dialog.wm_title('Initialize') - - container = tk.Frame(dialog) - container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) - container.grid_rowconfigure(0, weight=1) - container.grid_columnconfigure(0, weight=1) - - wall = tk.IntVar() - wall.set(0) - term = tk.IntVar() - term.set(0) - reward = tk.DoubleVar() - reward.set(0.0) - - label = ttk.Label(container, text='Initialize', font=('Helvetica', 12), anchor=tk.N) - label.grid(row=0, column=0, columnspan=3, sticky='new', pady=15, padx=5) - label_reward = ttk.Label(container, text='Reward', font=('Helvetica', 10), anchor=tk.N) - label_reward.grid(row=1, column=0, columnspan=3, sticky='new', pady=1, padx=5) - entry_reward = ttk.Entry(container, font=('Helvetica', 10), justify=tk.CENTER, exportselection=0, textvariable=reward) - entry_reward.grid(row=2, column=0, columnspan=3, sticky='new', pady=5, padx=50) - - rbtn_term = ttk.Radiobutton(container, text='Terminal', variable=term, value=TERM_VALUE) - rbtn_term.grid(row=3, column=0, columnspan=3, sticky='nsew', padx=160, pady=5) - rbtn_wall = ttk.Radiobutton(container, text='Wall', variable=wall, value=WALL_VALUE) - rbtn_wall.grid(row=4, column=0, columnspan=3, sticky='nsew', padx=172, pady=5) - - initialize_widget_disability_checks(_width, _height, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, rbtn_term) - - btn_apply = ttk.Button(container, text='Apply', command=partial(initialize_update_table, _width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall)) - btn_apply.grid(row=5, column=0, sticky='nsew', pady=5, padx=5) - btn_reset = ttk.Button(container, text='Reset', command=partial(initialize_reset_all, _width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term)) - btn_reset.grid(row=5, column=1, sticky='nsew', pady=5, padx=5) - btn_ok = ttk.Button(container, text='Ok', command=dialog.destroy) - btn_ok.grid(row=5, column=2, sticky='nsew', pady=5, padx=5) - - dialog.geometry('400x200') - dialog.mainloop() - -def update_table(i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall): - ''' functionality for 'apply' button ''' - - if wall.get() == WALL_VALUE: - buttons[i][j].configure(style='wall.TButton') - buttons[i][j].config(text='Wall') - label_reward.config(foreground='#999') - entry_reward.config(state=tk.DISABLED) - rbtn_term.state(['!focus', '!selected']) - rbtn_term.config(state=tk.DISABLED) - gridmdp[i][j] = WALL_VALUE - - elif wall.get() != WALL_VALUE: - if reward.get() != 0.0: - gridmdp[i][j] = reward.get() - buttons[i][j].configure(style='reward.TButton') - buttons[i][j].config(text=f'R = {reward.get()}') - - if term.get() == TERM_VALUE: - if (i, j) not in terminals: - terminals.append((i, j)) - rbtn_wall.state(['!focus', '!selected']) - rbtn_wall.config(state=tk.DISABLED) - - if gridmdp[i][j] < 0: - buttons[i][j].configure(style='-term.TButton') - - elif gridmdp[i][j] > 0: - buttons[i][j].configure(style='+term.TButton') - - elif gridmdp[i][j] == 0.0: - buttons[i][j].configure(style='=term.TButton') - -def initialize_update_table(_width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall): - ''' runs update_table for all cells ''' - - for i in range(max(1, _height)): - for j in range(max(1, _width)): - update_table(i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall) - -def reset_all(_height, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term): - ''' functionality for reset button ''' - - reward.set(0.0) - term.set(0) - wall.set(0) - gridmdp[i][j] = 0.0 - buttons[i][j].configure(style='TButton') - buttons[i][j].config(text=f'({_height - i - 1}, {j})') - - if (i, j) in terminals: - terminals.remove((i, j)) - - label_reward.config(foreground='#000') - entry_reward.config(state=tk.NORMAL) - rbtn_term.config(state=tk.NORMAL) - rbtn_wall.config(state=tk.NORMAL) - rbtn_wall.state(['!focus', '!selected']) - rbtn_term.state(['!focus', '!selected']) - -def initialize_reset_all(_width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term): - ''' runs reset_all for all cells ''' - - for i in range(max(1, _height)): - for j in range(max(1, _width)): - reset_all(_height, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term) + """creates dialogbox for initialization""" + + dialog = tk.Toplevel() + dialog.wm_title('Initialize') + + container = tk.Frame(dialog) + container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) + container.grid_rowconfigure(0, weight=1) + container.grid_columnconfigure(0, weight=1) + + wall = tk.IntVar() + wall.set(0) + term = tk.IntVar() + term.set(0) + reward = tk.DoubleVar() + reward.set(0.0) + + label = ttk.Label(container, text='Initialize', font=('Helvetica', 12), anchor=tk.N) + label.grid(row=0, column=0, columnspan=3, sticky='new', pady=15, padx=5) + label_reward = ttk.Label(container, text='Reward', font=('Helvetica', 10), anchor=tk.N) + label_reward.grid(row=1, column=0, columnspan=3, sticky='new', pady=1, padx=5) + entry_reward = ttk.Entry(container, font=('Helvetica', 10), justify=tk.CENTER, exportselection=0, + textvariable=reward) + entry_reward.grid(row=2, column=0, columnspan=3, sticky='new', pady=5, padx=50) + + rbtn_term = ttk.Radiobutton(container, text='Terminal', variable=term, value=TERM_VALUE) + rbtn_term.grid(row=3, column=0, columnspan=3, sticky='nsew', padx=160, pady=5) + rbtn_wall = ttk.Radiobutton(container, text='Wall', variable=wall, value=WALL_VALUE) + rbtn_wall.grid(row=4, column=0, columnspan=3, sticky='nsew', padx=172, pady=5) + + initialize_widget_disability_checks(_width, _height, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, + rbtn_term) + + btn_apply = ttk.Button(container, text='Apply', + command=partial(initialize_update_table, _width, _height, gridmdp, terminals, buttons, + reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall)) + btn_apply.grid(row=5, column=0, sticky='nsew', pady=5, padx=5) + btn_reset = ttk.Button(container, text='Reset', + command=partial(initialize_reset_all, _width, _height, gridmdp, terminals, buttons, reward, + term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term)) + btn_reset.grid(row=5, column=1, sticky='nsew', pady=5, padx=5) + btn_ok = ttk.Button(container, text='Ok', command=dialog.destroy) + btn_ok.grid(row=5, column=2, sticky='nsew', pady=5, padx=5) + + dialog.geometry('400x200') + dialog.mainloop() + + +def update_table(i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, + rbtn_wall): + """functionality for 'apply' button""" + if wall.get() == WALL_VALUE: + buttons[i][j].configure(style='wall.TButton') + buttons[i][j].config(text='Wall') + label_reward.config(foreground='#999') + entry_reward.config(state=tk.DISABLED) + rbtn_term.state(['!focus', '!selected']) + rbtn_term.config(state=tk.DISABLED) + gridmdp[i][j] = WALL_VALUE + + elif wall.get() != WALL_VALUE: + if reward.get() != 0.0: + gridmdp[i][j] = reward.get() + buttons[i][j].configure(style='reward.TButton') + buttons[i][j].config(text=f'R = {reward.get()}') + + if term.get() == TERM_VALUE: + if (i, j) not in terminals: + terminals.append((i, j)) + rbtn_wall.state(['!focus', '!selected']) + rbtn_wall.config(state=tk.DISABLED) + + if gridmdp[i][j] < 0: + buttons[i][j].configure(style='-term.TButton') + + elif gridmdp[i][j] > 0: + buttons[i][j].configure(style='+term.TButton') + + elif gridmdp[i][j] == 0.0: + buttons[i][j].configure(style='=term.TButton') + + +def initialize_update_table(_width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, + entry_reward, rbtn_term, rbtn_wall): + """runs update_table for all cells""" + + for i in range(max(1, _height)): + for j in range(max(1, _width)): + update_table(i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, + rbtn_wall) + + +def reset_all(_height, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, + rbtn_term): + """functionality for reset button""" + reward.set(0.0) + term.set(0) + wall.set(0) + gridmdp[i][j] = 0.0 + buttons[i][j].configure(style='TButton') + buttons[i][j].config(text=f'({_height - i - 1}, {j})') + + if (i, j) in terminals: + terminals.remove((i, j)) + + label_reward.config(foreground='#000') + entry_reward.config(state=tk.NORMAL) + rbtn_term.config(state=tk.NORMAL) + rbtn_wall.config(state=tk.NORMAL) + rbtn_wall.state(['!focus', '!selected']) + rbtn_term.state(['!focus', '!selected']) + + +def initialize_reset_all(_width, _height, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, + rbtn_wall, rbtn_term): + """runs reset_all for all cells""" + + for i in range(max(1, _height)): + for j in range(max(1, _width)): + reset_all(_height, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, + rbtn_wall, rbtn_term) + def external_reset(_width, _height, gridmdp, terminals, buttons): - ''' reset from edit menu ''' + """reset from edit menu""" + for i in range(max(1, _height)): + for j in range(max(1, _width)): + gridmdp[i][j] = 0.0 + buttons[i][j].configure(style='TButton') + buttons[i][j].config(text=f'({_height - i - 1}, {j})') - terminals = [] - for i in range(max(1, _height)): - for j in range(max(1, _width)): - gridmdp[i][j] = 0.0 - buttons[i][j].configure(style='TButton') - buttons[i][j].config(text=f'({_height - i - 1}, {j})') def widget_disability_checks(i, j, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, rbtn_term): - ''' checks for required state of widgets in dialogboxes ''' - - if gridmdp[i][j] == WALL_VALUE: - label_reward.config(foreground='#999') - entry_reward.config(state=tk.DISABLED) - rbtn_term.config(state=tk.DISABLED) - rbtn_wall.state(['!focus', 'selected']) - rbtn_term.state(['!focus', '!selected']) - - if (i, j) in terminals: - rbtn_wall.config(state=tk.DISABLED) - rbtn_wall.state(['!focus', '!selected']) - -def flatten_list(_list): - ''' returns a flattened list ''' - - return sum(_list, []) - -def initialize_widget_disability_checks(_width, _height, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, rbtn_term): - ''' checks for required state of widgets when cells are initialized ''' - - bool_walls = [['False']*max(1, _width) for _ in range(max(1, _height))] - bool_terms = [['False']*max(1, _width) for _ in range(max(1, _height))] - - for i in range(max(1, _height)): - for j in range(max(1, _width)): - if gridmdp[i][j] == WALL_VALUE: - bool_walls[i][j] = 'True' - - if (i, j) in terminals: - bool_terms[i][j] = 'True' - - bool_walls_fl = flatten_list(bool_walls) - bool_terms_fl = flatten_list(bool_terms) - - if bool_walls_fl.count('True') == len(bool_walls_fl): - print('`') - label_reward.config(foreground='#999') - entry_reward.config(state=tk.DISABLED) - rbtn_term.config(state=tk.DISABLED) - rbtn_wall.state(['!focus', 'selected']) - rbtn_term.state(['!focus', '!selected']) - - if bool_terms_fl.count('True') == len(bool_terms_fl): - rbtn_wall.config(state=tk.DISABLED) - rbtn_wall.state(['!focus', '!selected']) - rbtn_term.state(['!focus', 'selected']) - -def dialogbox(i, j, gridmdp, terminals, buttons, _height): - ''' creates dialogbox for each cell ''' - - dialog = tk.Toplevel() - dialog.wm_title(f'{_height - i - 1}, {j}') + """checks for required state of widgets in dialog boxes""" - container = tk.Frame(dialog) - container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) - container.grid_rowconfigure(0, weight=1) - container.grid_columnconfigure(0, weight=1) + if gridmdp[i][j] == WALL_VALUE: + label_reward.config(foreground='#999') + entry_reward.config(state=tk.DISABLED) + rbtn_term.config(state=tk.DISABLED) + rbtn_wall.state(['!focus', 'selected']) + rbtn_term.state(['!focus', '!selected']) - wall = tk.IntVar() - wall.set(gridmdp[i][j]) - term = tk.IntVar() - term.set(TERM_VALUE if (i, j) in terminals else 0.0) - reward = tk.DoubleVar() - reward.set(gridmdp[i][j] if gridmdp[i][j] != WALL_VALUE else 0.0) + if (i, j) in terminals: + rbtn_wall.config(state=tk.DISABLED) + rbtn_wall.state(['!focus', '!selected']) - label = ttk.Label(container, text=f'Configure cell {_height - i - 1}, {j}', font=('Helvetica', 12), anchor=tk.N) - label.grid(row=0, column=0, columnspan=3, sticky='new', pady=15, padx=5) - label_reward = ttk.Label(container, text='Reward', font=('Helvetica', 10), anchor=tk.N) - label_reward.grid(row=1, column=0, columnspan=3, sticky='new', pady=1, padx=5) - entry_reward = ttk.Entry(container, font=('Helvetica', 10), justify=tk.CENTER, exportselection=0, textvariable=reward) - entry_reward.grid(row=2, column=0, columnspan=3, sticky='new', pady=5, padx=50) - rbtn_term = ttk.Radiobutton(container, text='Terminal', variable=term, value=TERM_VALUE) - rbtn_term.grid(row=3, column=0, columnspan=3, sticky='nsew', padx=160, pady=5) - rbtn_wall = ttk.Radiobutton(container, text='Wall', variable=wall, value=WALL_VALUE) - rbtn_wall.grid(row=4, column=0, columnspan=3, sticky='nsew', padx=172, pady=5) - - widget_disability_checks(i, j, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, rbtn_term) - - btn_apply = ttk.Button(container, text='Apply', command=partial(update_table, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_term, rbtn_wall)) - btn_apply.grid(row=5, column=0, sticky='nsew', pady=5, padx=5) - btn_reset = ttk.Button(container, text='Reset', command=partial(reset_all, _height, i, j, gridmdp, terminals, buttons, reward, term, wall, label_reward, entry_reward, rbtn_wall, rbtn_term)) - btn_reset.grid(row=5, column=1, sticky='nsew', pady=5, padx=5) - btn_ok = ttk.Button(container, text='Ok', command=dialog.destroy) - btn_ok.grid(row=5, column=2, sticky='nsew', pady=5, padx=5) - - dialog.geometry('400x200') - dialog.mainloop() - - -class MDPapp(tk.Tk): - - def __init__(self, *args, **kwargs): - - tk.Tk.__init__(self, *args, **kwargs) - tk.Tk.wm_title(self, 'Grid MDP') - self.shared_data = { - 'height': tk.IntVar(), - 'width': tk.IntVar() - } - self.shared_data['height'].set(1) - self.shared_data['width'].set(1) - self.container = tk.Frame(self) - self.container.pack(side='top', fill='both', expand=True) - self.container.grid_rowconfigure(0, weight=1) - self.container.grid_columnconfigure(0, weight=1) - - self.frames = {} - - self.menu_bar = tk.Menu(self.container) - self.file_menu = tk.Menu(self.menu_bar, tearoff=0) - self.file_menu.add_command(label='Exit', command=self.exit) - self.menu_bar.add_cascade(label='File', menu=self.file_menu) - - self.edit_menu = tk.Menu(self.menu_bar, tearoff=1) - self.edit_menu.add_command(label='Reset', command=self.master_reset) - self.edit_menu.add_command(label='Initialize', command=self.initialize) - self.edit_menu.add_separator() - self.edit_menu.add_command(label='View matrix', command=self.view_matrix) - self.edit_menu.add_command(label='View terminals', command=self.view_terminals) - self.menu_bar.add_cascade(label='Edit', menu=self.edit_menu) - self.menu_bar.entryconfig('Edit', state=tk.DISABLED) - - self.build_menu = tk.Menu(self.menu_bar, tearoff=1) - self.build_menu.add_command(label='Build and Run', command=self.build) - self.menu_bar.add_cascade(label='Build', menu=self.build_menu) - self.menu_bar.entryconfig('Build', state=tk.DISABLED) - tk.Tk.config(self, menu=self.menu_bar) - - for F in (HomePage, BuildMDP, SolveMDP): - frame = F(self.container, self) - self.frames[F] = frame - frame.grid(row=0, column=0, sticky='nsew') - - self.show_frame(HomePage) - - def placeholder_function(self): - ''' placeholder function ''' - - print('Not supported yet!') - - def exit(self): - ''' function to exit ''' - - if tkinter.messagebox.askokcancel('Exit?', 'All changes will be lost'): - quit() - - def new(self): - ''' function to create new GridMDP ''' - - self.master_reset() - build_page = self.get_page(BuildMDP) - build_page.gridmdp = None - build_page.terminals = None - build_page.buttons = None - self.show_frame(HomePage) - - def get_page(self, page_class): - ''' returns pages from stored frames ''' +def flatten_list(_list): + """returns a flattened list""" + return sum(_list, []) - return self.frames[page_class] - def view_matrix(self): - ''' prints current matrix to console ''' +def initialize_widget_disability_checks(_width, _height, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, + rbtn_term): + """checks for required state of widgets when cells are initialized""" - build_page = self.get_page(BuildMDP) - _height = self.shared_data['height'].get() - _width = self.shared_data['width'].get() - print(build_page.gridmdp) - display(build_page.gridmdp, _height, _width) + bool_walls = [['False'] * max(1, _width) for _ in range(max(1, _height))] + bool_terms = [['False'] * max(1, _width) for _ in range(max(1, _height))] - def view_terminals(self): - ''' prints current terminals to console ''' + for i in range(max(1, _height)): + for j in range(max(1, _width)): + if gridmdp[i][j] == WALL_VALUE: + bool_walls[i][j] = 'True' - build_page = self.get_page(BuildMDP) - print('Terminals', build_page.terminals) + if (i, j) in terminals: + bool_terms[i][j] = 'True' - def initialize(self): - ''' calls initialize from BuildMDP ''' + bool_walls_fl = flatten_list(bool_walls) + bool_terms_fl = flatten_list(bool_terms) - build_page = self.get_page(BuildMDP) - build_page.initialize() + if bool_walls_fl.count('True') == len(bool_walls_fl): + print('`') + label_reward.config(foreground='#999') + entry_reward.config(state=tk.DISABLED) + rbtn_term.config(state=tk.DISABLED) + rbtn_wall.state(['!focus', 'selected']) + rbtn_term.state(['!focus', '!selected']) - def master_reset(self): - ''' calls master_reset from BuildMDP ''' + if bool_terms_fl.count('True') == len(bool_terms_fl): + rbtn_wall.config(state=tk.DISABLED) + rbtn_wall.state(['!focus', '!selected']) + rbtn_term.state(['!focus', 'selected']) - build_page = self.get_page(BuildMDP) - build_page.master_reset() - def build(self): - ''' runs specified mdp solving algorithm ''' +def dialogbox(i, j, gridmdp, terminals, buttons, _height): + """creates dialogbox for each cell""" + global cell_window_mantainer + if (cell_window_mantainer != None): + cell_window_mantainer.destroy() + + dialog = tk.Toplevel() + cell_window_mantainer = dialog + dialog.wm_title(f'{_height - i - 1}, {j}') + + container = tk.Frame(dialog) + container.pack(side=tk.TOP, fill=tk.BOTH, expand=True) + container.grid_rowconfigure(0, weight=1) + container.grid_columnconfigure(0, weight=1) + + wall = tk.IntVar() + wall.set(gridmdp[i][j]) + term = tk.IntVar() + term.set(TERM_VALUE if (i, j) in terminals else 0.0) + reward = tk.DoubleVar() + reward.set(gridmdp[i][j] if gridmdp[i][j] != WALL_VALUE else 0.0) + + label = ttk.Label(container, text=f'Configure cell {_height - i - 1}, {j}', font=('Helvetica', 12), anchor=tk.N) + label.grid(row=0, column=0, columnspan=3, sticky='new', pady=15, padx=5) + label_reward = ttk.Label(container, text='Reward', font=('Helvetica', 10), anchor=tk.N) + label_reward.grid(row=1, column=0, columnspan=3, sticky='new', pady=1, padx=5) + entry_reward = ttk.Entry(container, font=('Helvetica', 10), justify=tk.CENTER, exportselection=0, + textvariable=reward) + entry_reward.grid(row=2, column=0, columnspan=3, sticky='new', pady=5, padx=50) + + rbtn_term = ttk.Radiobutton(container, text='Terminal', variable=term, value=TERM_VALUE) + rbtn_term.grid(row=3, column=0, columnspan=3, sticky='nsew', padx=160, pady=5) + rbtn_wall = ttk.Radiobutton(container, text='Wall', variable=wall, value=WALL_VALUE) + rbtn_wall.grid(row=4, column=0, columnspan=3, sticky='nsew', padx=172, pady=5) + + widget_disability_checks(i, j, gridmdp, terminals, label_reward, entry_reward, rbtn_wall, rbtn_term) + + btn_apply = ttk.Button(container, text='Apply', + command=partial(update_table, i, j, gridmdp, terminals, buttons, reward, term, wall, + label_reward, entry_reward, rbtn_term, rbtn_wall)) + btn_apply.grid(row=5, column=0, sticky='nsew', pady=5, padx=5) + btn_reset = ttk.Button(container, text='Reset', + command=partial(reset_all, _height, i, j, gridmdp, terminals, buttons, reward, term, wall, + label_reward, entry_reward, rbtn_wall, rbtn_term)) + btn_reset.grid(row=5, column=1, sticky='nsew', pady=5, padx=5) + btn_ok = ttk.Button(container, text='Ok', command=dialog.destroy) + btn_ok.grid(row=5, column=2, sticky='nsew', pady=5, padx=5) + + dialog.geometry('400x200') + dialog.mainloop() - frame = SolveMDP(self.container, self) - self.frames[SolveMDP] = frame - frame.grid(row=0, column=0, sticky='nsew') - self.show_frame(SolveMDP) - build_page = self.get_page(BuildMDP) - gridmdp = build_page.gridmdp - terminals = build_page.terminals - solve_page = self.get_page(SolveMDP) - _height = self.shared_data['height'].get() - _width = self.shared_data['width'].get() - solve_page.create_graph(gridmdp, terminals, _height, _width) - def show_frame(self, controller, cb=False): - ''' shows specified frame and optionally runs create_buttons ''' +class MDPapp(tk.Tk): - if cb: - build_page = self.get_page(BuildMDP) - build_page.create_buttons() - frame = self.frames[controller] - frame.tkraise() + def __init__(self, *args, **kwargs): + + tk.Tk.__init__(self, *args, **kwargs) + tk.Tk.wm_title(self, 'Grid MDP') + self.shared_data = { + 'height': tk.IntVar(), + 'width': tk.IntVar()} + self.shared_data['height'].set(1) + self.shared_data['width'].set(1) + self.container = tk.Frame(self) + self.container.pack(side='top', fill='both', expand=True) + self.container.grid_rowconfigure(0, weight=1) + self.container.grid_columnconfigure(0, weight=1) + + self.frames = {} + + self.menu_bar = tk.Menu(self.container) + self.file_menu = tk.Menu(self.menu_bar, tearoff=0) + self.file_menu.add_command(label='Exit', command=self.exit) + self.menu_bar.add_cascade(label='File', menu=self.file_menu) + + self.edit_menu = tk.Menu(self.menu_bar, tearoff=1) + self.edit_menu.add_command(label='Reset', command=self.master_reset) + self.edit_menu.add_command(label='Initialize', command=self.initialize) + self.edit_menu.add_separator() + self.edit_menu.add_command(label='View matrix', command=self.view_matrix) + self.edit_menu.add_command(label='View terminals', command=self.view_terminals) + self.menu_bar.add_cascade(label='Edit', menu=self.edit_menu) + self.menu_bar.entryconfig('Edit', state=tk.DISABLED) + + self.build_menu = tk.Menu(self.menu_bar, tearoff=1) + self.build_menu.add_command(label='Build and Run', command=self.build) + self.menu_bar.add_cascade(label='Build', menu=self.build_menu) + self.menu_bar.entryconfig('Build', state=tk.DISABLED) + tk.Tk.config(self, menu=self.menu_bar) + + for F in (HomePage, BuildMDP, SolveMDP): + frame = F(self.container, self) + self.frames[F] = frame + frame.grid(row=0, column=0, sticky='nsew') + + self.show_frame(HomePage) + + def placeholder_function(self): + """placeholder function""" + + print('Not supported yet!') + + def exit(self): + """function to exit""" + if tkinter.messagebox.askokcancel('Exit?', 'All changes will be lost'): + quit() + + def new(self): + """function to create new GridMDP""" + + self.master_reset() + build_page = self.get_page(BuildMDP) + build_page.gridmdp = None + build_page.terminals = None + build_page.buttons = None + self.show_frame(HomePage) + + def get_page(self, page_class): + """returns pages from stored frames""" + return self.frames[page_class] + + def view_matrix(self): + """prints current matrix to console""" + + build_page = self.get_page(BuildMDP) + _height = self.shared_data['height'].get() + _width = self.shared_data['width'].get() + print(build_page.gridmdp) + display(build_page.gridmdp, _height, _width) + + def view_terminals(self): + """prints current terminals to console""" + build_page = self.get_page(BuildMDP) + print('Terminals', build_page.terminals) + + def initialize(self): + """calls initialize from BuildMDP""" + + build_page = self.get_page(BuildMDP) + build_page.initialize() + + def master_reset(self): + """calls master_reset from BuildMDP""" + build_page = self.get_page(BuildMDP) + build_page.master_reset() + + def build(self): + """runs specified mdp solving algorithm""" + + frame = SolveMDP(self.container, self) + self.frames[SolveMDP] = frame + frame.grid(row=0, column=0, sticky='nsew') + self.show_frame(SolveMDP) + build_page = self.get_page(BuildMDP) + gridmdp = build_page.gridmdp + terminals = build_page.terminals + solve_page = self.get_page(SolveMDP) + _height = self.shared_data['height'].get() + _width = self.shared_data['width'].get() + solve_page.create_graph(gridmdp, terminals, _height, _width) + + def show_frame(self, controller, cb=False): + """shows specified frame and optionally runs create_buttons""" + if cb: + build_page = self.get_page(BuildMDP) + build_page.create_buttons() + frame = self.frames[controller] + frame.tkraise() class HomePage(tk.Frame): - def __init__(self, parent, controller): - ''' HomePage constructor ''' - - tk.Frame.__init__(self, parent) - self.controller = controller - frame1 = tk.Frame(self) - frame1.pack(side=tk.TOP) - frame3 = tk.Frame(self) - frame3.pack(side=tk.TOP) - frame4 = tk.Frame(self) - frame4.pack(side=tk.TOP) - frame2 = tk.Frame(self) - frame2.pack(side=tk.TOP) - - s = ttk.Style() - s.theme_use('clam') - s.configure('TButton', background=grayd, padding=0) - s.configure('wall.TButton', background=gray2, foreground=white) - s.configure('reward.TButton', background=gray9) - s.configure('+term.TButton', background=green8) - s.configure('-term.TButton', background=pblue, foreground=white) - s.configure('=term.TButton', background=green4) - - label = ttk.Label(frame1, text='GridMDP builder', font=('Helvetica', 18, 'bold'), background=grayef) - label.pack(pady=75, padx=50, side=tk.TOP) - - ec_btn = ttk.Button(frame3, text='Empty cells', width=20) - ec_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - ec_btn.configure(style='TButton') - - w_btn = ttk.Button(frame3, text='Walls', width=20) - w_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - w_btn.configure(style='wall.TButton') - - r_btn = ttk.Button(frame3, text='Rewards', width=20) - r_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - r_btn.configure(style='reward.TButton') - - term_p = ttk.Button(frame3, text='Positive terminals', width=20) - term_p.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - term_p.configure(style='+term.TButton') - - term_z = ttk.Button(frame3, text='Neutral terminals', width=20) - term_z.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - term_z.configure(style='=term.TButton') - - term_n = ttk.Button(frame3, text='Negative terminals', width=20) - term_n.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) - term_n.configure(style='-term.TButton') - - label = ttk.Label(frame4, text='Dimensions', font=('Verdana', 14), background=grayef) - label.pack(pady=15, padx=10, side=tk.TOP) - entry_h = tk.Entry(frame2, textvariable=self.controller.shared_data['height'], font=('Verdana', 10), width=3, justify=tk.CENTER) - entry_h.pack(pady=10, padx=10, side=tk.LEFT) - label_x = ttk.Label(frame2, text='X', font=('Verdana', 10), background=grayef) - label_x.pack(pady=10, padx=4, side=tk.LEFT) - entry_w = tk.Entry(frame2, textvariable=self.controller.shared_data['width'], font=('Verdana', 10), width=3, justify=tk.CENTER) - entry_w.pack(pady=10, padx=10, side=tk.LEFT) - button = ttk.Button(self, text='Build a GridMDP', command=lambda: controller.show_frame(BuildMDP, cb=True)) - button.pack(pady=10, padx=10, side=tk.TOP, ipadx=20, ipady=10) - button.configure(style='reward.TButton') + def __init__(self, parent, controller): + """HomePage constructor""" + + tk.Frame.__init__(self, parent) + self.controller = controller + frame1 = tk.Frame(self) + frame1.pack(side=tk.TOP) + frame3 = tk.Frame(self) + frame3.pack(side=tk.TOP) + frame4 = tk.Frame(self) + frame4.pack(side=tk.TOP) + frame2 = tk.Frame(self) + frame2.pack(side=tk.TOP) + + s = ttk.Style() + s.theme_use('clam') + s.configure('TButton', background=grayd, padding=0) + s.configure('wall.TButton', background=gray2, foreground=white) + s.configure('reward.TButton', background=gray9) + s.configure('+term.TButton', background=green8) + s.configure('-term.TButton', background=pblue, foreground=white) + s.configure('=term.TButton', background=green4) + + label = ttk.Label(frame1, text='GridMDP builder', font=('Helvetica', 18, 'bold'), background=grayef) + label.pack(pady=75, padx=50, side=tk.TOP) + + ec_btn = ttk.Button(frame3, text='Empty cells', width=20) + ec_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + ec_btn.configure(style='TButton') + + w_btn = ttk.Button(frame3, text='Walls', width=20) + w_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + w_btn.configure(style='wall.TButton') + + r_btn = ttk.Button(frame3, text='Rewards', width=20) + r_btn.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + r_btn.configure(style='reward.TButton') + + term_p = ttk.Button(frame3, text='Positive terminals', width=20) + term_p.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + term_p.configure(style='+term.TButton') + + term_z = ttk.Button(frame3, text='Neutral terminals', width=20) + term_z.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + term_z.configure(style='=term.TButton') + + term_n = ttk.Button(frame3, text='Negative terminals', width=20) + term_n.pack(pady=0, padx=0, side=tk.LEFT, ipady=10) + term_n.configure(style='-term.TButton') + + label = ttk.Label(frame4, text='Dimensions', font=('Verdana', 14), background=grayef) + label.pack(pady=15, padx=10, side=tk.TOP) + entry_h = tk.Entry(frame2, textvariable=self.controller.shared_data['height'], font=('Verdana', 10), width=3, + justify=tk.CENTER) + entry_h.pack(pady=10, padx=10, side=tk.LEFT) + label_x = ttk.Label(frame2, text='X', font=('Verdana', 10), background=grayef) + label_x.pack(pady=10, padx=4, side=tk.LEFT) + entry_w = tk.Entry(frame2, textvariable=self.controller.shared_data['width'], font=('Verdana', 10), width=3, + justify=tk.CENTER) + entry_w.pack(pady=10, padx=10, side=tk.LEFT) + button = ttk.Button(self, text='Build a GridMDP', command=lambda: controller.show_frame(BuildMDP, cb=True)) + button.pack(pady=10, padx=10, side=tk.TOP, ipadx=20, ipady=10) + button.configure(style='reward.TButton') class BuildMDP(tk.Frame): - def __init__(self, parent, controller): - - tk.Frame.__init__(self, parent) - self.grid_rowconfigure(0, weight=1) - self.grid_columnconfigure(0, weight=1) - self.frame = tk.Frame(self) - self.frame.pack() - self.controller = controller - - def create_buttons(self): - ''' creates interactive cells to build MDP ''' - - _height = self.controller.shared_data['height'].get() - _width = self.controller.shared_data['width'].get() - self.controller.menu_bar.entryconfig('Edit', state=tk.NORMAL) - self.controller.menu_bar.entryconfig('Build', state=tk.NORMAL) - self.gridmdp = [[0.0]*max(1, _width) for _ in range(max(1, _height))] - self.buttons = [[None]*max(1, _width) for _ in range(max(1, _height))] - self.terminals = [] - - s = ttk.Style() - s.theme_use('clam') - s.configure('TButton', background=grayd, padding=0) - s.configure('wall.TButton', background=gray2, foreground=white) - s.configure('reward.TButton', background=gray9) - s.configure('+term.TButton', background=green8) - s.configure('-term.TButton', background=pblue, foreground=white) - s.configure('=term.TButton', background=green4) - - for i in range(max(1, _height)): - for j in range(max(1, _width)): - self.buttons[i][j] = ttk.Button(self.frame, text=f'({_height - i - 1}, {j})', width=int(196/max(1, _width)), command=partial(dialogbox, i, j, self.gridmdp, self.terminals, self.buttons, _height)) - self.buttons[i][j].grid(row=i, column=j, ipady=int(336/max(1, _height)) - 12) - - def initialize(self): - ''' runs initialize_dialogbox ''' - - _height = self.controller.shared_data['height'].get() - _width = self.controller.shared_data['width'].get() - initialize_dialogbox(_width, _height, self.gridmdp, self.terminals, self.buttons) - - def master_reset(self): - ''' runs external reset ''' - - _height = self.controller.shared_data['height'].get() - _width = self.controller.shared_data['width'].get() - if tkinter.messagebox.askokcancel('Reset', 'Are you sure you want to reset all cells?'): - external_reset(_width, _height, self.gridmdp, self.terminals, self.buttons) + def __init__(self, parent, controller): + + tk.Frame.__init__(self, parent) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(0, weight=1) + self.frame = tk.Frame(self) + self.frame.pack() + self.controller = controller + + def create_buttons(self): + """creates interactive cells to build MDP""" + _height = self.controller.shared_data['height'].get() + _width = self.controller.shared_data['width'].get() + self.controller.menu_bar.entryconfig('Edit', state=tk.NORMAL) + self.controller.menu_bar.entryconfig('Build', state=tk.NORMAL) + self.gridmdp = [[0.0] * max(1, _width) for _ in range(max(1, _height))] + self.buttons = [[None] * max(1, _width) for _ in range(max(1, _height))] + self.terminals = [] + + s = ttk.Style() + s.theme_use('clam') + s.configure('TButton', background=grayd, padding=0) + s.configure('wall.TButton', background=gray2, foreground=white) + s.configure('reward.TButton', background=gray9) + s.configure('+term.TButton', background=green8) + s.configure('-term.TButton', background=pblue, foreground=white) + s.configure('=term.TButton', background=green4) + + for i in range(max(1, _height)): + for j in range(max(1, _width)): + self.buttons[i][j] = ttk.Button(self.frame, text=f'({_height - i - 1}, {j})', + width=int(196 / max(1, _width)), + command=partial(dialogbox, i, j, self.gridmdp, self.terminals, + self.buttons, _height)) + self.buttons[i][j].grid(row=i, column=j, ipady=int(336 / max(1, _height)) - 12) + + def initialize(self): + """runs initialize_dialogbox""" + + _height = self.controller.shared_data['height'].get() + _width = self.controller.shared_data['width'].get() + initialize_dialogbox(_width, _height, self.gridmdp, self.terminals, self.buttons) + + def master_reset(self): + """runs external reset""" + _height = self.controller.shared_data['height'].get() + _width = self.controller.shared_data['width'].get() + if tkinter.messagebox.askokcancel('Reset', 'Are you sure you want to reset all cells?'): + external_reset(_width, _height, self.gridmdp, self.terminals, self.buttons) class SolveMDP(tk.Frame): - def __init__(self, parent, controller): - - tk.Frame.__init__(self, parent) - self.grid_rowconfigure(0, weight=1) - self.grid_columnconfigure(0, weight=1) - self.frame = tk.Frame(self) - self.frame.pack() - self.controller = controller - self.terminated = False - self.iterations = 0 - self.epsilon = 0.001 - self.delta = 0 + def __init__(self, parent, controller): - def process_data(self, terminals, _height, _width, gridmdp): - ''' preprocess variables ''' + tk.Frame.__init__(self, parent) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(0, weight=1) + self.frame = tk.Frame(self) + self.frame.pack() + self.controller = controller + self.terminated = False + self.iterations = 0 + self.epsilon = 0.001 + self.delta = 0 - flipped_terminals = [] + def process_data(self, terminals, _height, _width, gridmdp): + """preprocess variables""" - for terminal in terminals: - flipped_terminals.append((terminal[1], _height - terminal[0] - 1)) + flipped_terminals = [] - grid_to_solve = [[0.0]*max(1, _width) for _ in range(max(1, _height))] - grid_to_show = [[0.0]*max(1, _width) for _ in range(max(1, _height))] + for terminal in terminals: + flipped_terminals.append((terminal[1], _height - terminal[0] - 1)) - for i in range(max(1, _height)): - for j in range(max(1, _width)): - if gridmdp[i][j] == WALL_VALUE: - grid_to_show[i][j] = 0.0 - grid_to_solve[i][j] = None + grid_to_solve = [[0.0] * max(1, _width) for _ in range(max(1, _height))] + grid_to_show = [[0.0] * max(1, _width) for _ in range(max(1, _height))] - else: - grid_to_show[i][j] = grid_to_solve[i][j] = gridmdp[i][j] + for i in range(max(1, _height)): + for j in range(max(1, _width)): + if gridmdp[i][j] == WALL_VALUE: + grid_to_show[i][j] = 0.0 + grid_to_solve[i][j] = None - return flipped_terminals, grid_to_solve, np.flipud(grid_to_show) + else: + grid_to_show[i][j] = grid_to_solve[i][j] = gridmdp[i][j] - def create_graph(self, gridmdp, terminals, _height, _width): - ''' creates canvas and initializes value_iteration_paramteres ''' + return flipped_terminals, grid_to_solve, np.flipud(grid_to_show) - self._height = _height - self._width = _width - self.controller.menu_bar.entryconfig('Edit', state=tk.DISABLED) - self.controller.menu_bar.entryconfig('Build', state=tk.DISABLED) + def create_graph(self, gridmdp, terminals, _height, _width): + """creates canvas and initializes value_iteration_parameters""" + self._height = _height + self._width = _width + self.controller.menu_bar.entryconfig('Edit', state=tk.DISABLED) + self.controller.menu_bar.entryconfig('Build', state=tk.DISABLED) - self.terminals, self.gridmdp, self.grid_to_show = self.process_data(terminals, _height, _width, gridmdp) - self.sequential_decision_environment = GridMDP(self.gridmdp, terminals=self.terminals) + self.terminals, self.gridmdp, self.grid_to_show = self.process_data(terminals, _height, _width, gridmdp) + self.sequential_decision_environment = GridMDP(self.gridmdp, terminals=self.terminals) - self.initialize_value_iteration_parameters(self.sequential_decision_environment) + self.initialize_value_iteration_parameters(self.sequential_decision_environment) - self.canvas = FigureCanvasTkAgg(fig, self.frame) - self.canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=True) - self.anim = animation.FuncAnimation(fig, self.animate_graph, interval=50) - self.canvas.show() + self.canvas = FigureCanvasTkAgg(fig, self.frame) + self.canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=True) + self.anim = animation.FuncAnimation(fig, self.animate_graph, interval=50) + self.canvas.show() - def animate_graph(self, i): - ''' performs value iteration and animates graph ''' + def animate_graph(self, i): + """performs value iteration and animates graph""" - # cmaps to use: bone_r, Oranges, inferno, BrBG, copper - self.iterations += 1 - x_interval = max(2, len(self.gridmdp[0])) - y_interval = max(2, len(self.gridmdp)) - x = np.linspace(0, len(self.gridmdp[0]) - 1, x_interval) - y = np.linspace(0, len(self.gridmdp) - 1, y_interval) + # cmaps to use: bone_r, Oranges, inferno, BrBG, copper + self.iterations += 1 + x_interval = max(2, len(self.gridmdp[0])) + y_interval = max(2, len(self.gridmdp)) + x = np.linspace(0, len(self.gridmdp[0]) - 1, x_interval) + y = np.linspace(0, len(self.gridmdp) - 1, y_interval) - sub.clear() - sub.imshow(self.grid_to_show, cmap='BrBG', aspect='auto', interpolation='none', extent=extents(x) + extents(y), origin='lower') - fig.tight_layout() + sub.clear() + sub.imshow(self.grid_to_show, cmap='BrBG', aspect='auto', interpolation='none', extent=extents(x) + extents(y), + origin='lower') + fig.tight_layout() - U = self.U1.copy() + U = self.U1.copy() - for s in self.sequential_decision_environment.states: - self.U1[s] = self.R(s) + self.gamma * max([sum([p * U[s1] for (p, s1) in self.T(s, a)]) for a in self.sequential_decision_environment.actions(s)]) - self.delta = max(self.delta, abs(self.U1[s] - U[s])) + for s in self.sequential_decision_environment.states: + self.U1[s] = self.R(s) + self.gamma * max( + [sum([p * U[s1] for (p, s1) in self.T(s, a)]) for a in self.sequential_decision_environment.actions(s)]) + self.delta = max(self.delta, abs(self.U1[s] - U[s])) - self.grid_to_show = grid_to_show = [[0.0]*max(1, self._width) for _ in range(max(1, self._height))] - for k, v in U.items(): - self.grid_to_show[k[1]][k[0]] = v + self.grid_to_show = grid_to_show = [[0.0] * max(1, self._width) for _ in range(max(1, self._height))] + for k, v in U.items(): + self.grid_to_show[k[1]][k[0]] = v - if (self.delta < self.epsilon * (1 - self.gamma) / self.gamma) or (self.iterations > 60) and self.terminated == False: - self.terminated = True - display(self.grid_to_show, self._height, self._width) + if (self.delta < self.epsilon * (1 - self.gamma) / self.gamma) or ( + self.iterations > 60) and self.terminated is False: + self.terminated = True + display(self.grid_to_show, self._height, self._width) - pi = best_policy(self.sequential_decision_environment, value_iteration(self.sequential_decision_environment, .01)) - display_best_policy(self.sequential_decision_environment.to_arrows(pi), self._height, self._width) - - ax = fig.gca() - ax.xaxis.set_major_locator(MaxNLocator(integer=True)) - ax.yaxis.set_major_locator(MaxNLocator(integer=True)) + pi = best_policy(self.sequential_decision_environment, + value_iteration(self.sequential_decision_environment, .01)) + display_best_policy(self.sequential_decision_environment.to_arrows(pi), self._height, self._width) - def initialize_value_iteration_parameters(self, mdp): - ''' initializes value_iteration parameters ''' + ax = fig.gca() + ax.xaxis.set_major_locator(MaxNLocator(integer=True)) + ax.yaxis.set_major_locator(MaxNLocator(integer=True)) - self.U1 = {s: 0 for s in mdp.states} - self.R, self.T, self.gamma = mdp.R, mdp.T, mdp.gamma + def initialize_value_iteration_parameters(self, mdp): + """initializes value_iteration parameters""" + self.U1 = {s: 0 for s in mdp.states} + self.R, self.T, self.gamma = mdp.R, mdp.T, mdp.gamma - def value_iteration_metastep(self, mdp, iterations=20): - ''' runs value_iteration ''' + def value_iteration_metastep(self, mdp, iterations=20): + """runs value_iteration""" - U_over_time = [] - U1 = {s: 0 for s in mdp.states} - R, T, gamma = mdp.R, mdp.T, mdp.gamma + U_over_time = [] + U1 = {s: 0 for s in mdp.states} + R, T, gamma = mdp.R, mdp.T, mdp.gamma - for _ in range(iterations): - U = U1.copy() + for _ in range(iterations): + U = U1.copy() - for s in mdp.states: - U1[s] = R(s) + gamma * max([sum([p * U[s1] for (p, s1) in T(s, a)]) for a in mdp.actions(s)]) + for s in mdp.states: + U1[s] = R(s) + gamma * max([sum([p * U[s1] for (p, s1) in T(s, a)]) for a in mdp.actions(s)]) - U_over_time.append(U) - return U_over_time + U_over_time.append(U) + return U_over_time if __name__ == '__main__': - app = MDPapp() - app.geometry('1280x720') - app.mainloop() \ No newline at end of file + app = MDPapp() + app.geometry('1280x720') + app.mainloop() diff --git a/gui/romania_problem.py b/gui/romania_problem.py index b1778eef9..9ec94099d 100644 --- a/gui/romania_problem.py +++ b/gui/romania_problem.py @@ -1,14 +1,10 @@ +from copy import deepcopy from tkinter import * -import sys -import os.path -import math -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) + from search import * -from search import breadth_first_tree_search as bfts, depth_first_tree_search as dfts, \ - depth_first_graph_search as dfgs, breadth_first_graph_search as bfs, uniform_cost_search as ucs, \ - astar_search as asts from utils import PriorityQueue -from copy import deepcopy + +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) root = None city_coord = {} @@ -289,7 +285,6 @@ def make_rectangle(map, x0, y0, margin, city_name): def make_legend(map): - rect1 = map.create_rectangle(600, 100, 610, 110, fill="white") text1 = map.create_text(615, 105, anchor=W, text="Un-explored") @@ -325,13 +320,11 @@ def tree_search(problem): display_current(node) if counter % 3 == 1 and counter >= 0: if problem.goal_test(node.state): - return node frontier.extend(node.expand(problem)) display_frontier(frontier) if counter % 3 == 2 and counter >= 0: - display_explored(node) return None @@ -538,9 +531,8 @@ def best_first_graph_search(problem, f): if child.state not in explored and child not in frontier: frontier.append(child) elif child in frontier: - incumbent = frontier[child] - if f(child) < f(incumbent): - del frontier[incumbent] + if f(child) < frontier[child]: + del frontier[child] frontier.append(child) display_frontier(frontier) if counter % 3 == 2 and counter >= 0: @@ -563,7 +555,7 @@ def astar_search(problem, h=None): # TODO: # Remove redundant code. -# Make the interchangbility work between various algorithms at each step. +# Make the interchangeability work between various algorithms at each step. def on_click(): """ This function defines the action of the 'Next' button. @@ -573,7 +565,7 @@ def on_click(): if "Breadth-First Tree Search" == algo.get(): node = breadth_first_tree_search(romania_problem) if node is not None: - final_path = bfts(romania_problem).solution() + final_path = breadth_first_tree_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -581,7 +573,7 @@ def on_click(): elif "Depth-First Tree Search" == algo.get(): node = depth_first_tree_search(romania_problem) if node is not None: - final_path = dfts(romania_problem).solution() + final_path = depth_first_tree_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -589,7 +581,7 @@ def on_click(): elif "Breadth-First Graph Search" == algo.get(): node = breadth_first_graph_search(romania_problem) if node is not None: - final_path = bfs(romania_problem).solution() + final_path = breadth_first_graph_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -597,7 +589,7 @@ def on_click(): elif "Depth-First Graph Search" == algo.get(): node = depth_first_graph_search(romania_problem) if node is not None: - final_path = dfgs(romania_problem).solution() + final_path = depth_first_graph_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -605,7 +597,7 @@ def on_click(): elif "Uniform Cost Search" == algo.get(): node = uniform_cost_search(romania_problem) if node is not None: - final_path = ucs(romania_problem).solution() + final_path = uniform_cost_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -613,7 +605,7 @@ def on_click(): elif "A* - Search" == algo.get(): node = astar_search(romania_problem) if node is not None: - final_path = asts(romania_problem).solution() + final_path = astar_search(romania_problem).solution() final_path.append(start.get()) display_final(final_path) next_button.config(state="disabled") @@ -627,10 +619,9 @@ def reset_map(): city_map.itemconfig(city_coord[city], fill="white") next_button.config(state="normal") -# TODO: Add more search algorithms in the OptionMenu - -def main(): +# TODO: Add more search algorithms in the OptionMenu +if __name__ == "__main__": global algo, start, goal, next_button root = Tk() root.title("Road Map of Romania") @@ -679,7 +670,3 @@ def main(): frame1.pack(side=BOTTOM) create_map(root) root.mainloop() - - -if __name__ == "__main__": - main() diff --git a/gui/tic-tac-toe.py b/gui/tic-tac-toe.py index 5c3bdb497..66d9d6e75 100644 --- a/gui/tic-tac-toe.py +++ b/gui/tic-tac-toe.py @@ -1,11 +1,12 @@ -from tkinter import * -import sys import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) -from games import minimax_decision, alphabeta_player, random_player, TicTacToe +from tkinter import * + +from games import minmax_decision, alpha_beta_player, random_player, TicTacToe # "gen_state" can be used to generate a game state to apply the algorithm from tests.test_games import gen_state +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) + ttt = TicTacToe() root = None buttons = [] @@ -95,9 +96,9 @@ def on_click(button): if "Random" in choice: a, b = random_player(ttt, state) elif "Pro" in choice: - a, b = minimax_decision(state, ttt) + a, b = minmax_decision(state, ttt) else: - a, b = alphabeta_player(ttt, state) + a, b = alpha_beta_player(ttt, state) except (ValueError, IndexError, TypeError) as e: disable_game() result.set("It's a draw :|") @@ -152,8 +153,7 @@ def check_victory(button): return True # check if previous move was on the secondary diagonal and caused a win - if x + y \ - == 2 and buttons[0][2]['text'] == buttons[1][1]['text'] == buttons[2][0]['text'] != " ": + if x + y == 2 and buttons[0][2]['text'] == buttons[1][1]['text'] == buttons[2][0]['text'] != " ": buttons[0][2].config(text="/" + tt + "/") buttons[1][1].config(text="/" + tt + "/") buttons[2][0].config(text="/" + tt + "/") @@ -213,7 +213,7 @@ def exit_game(root): root.destroy() -def main(): +if __name__ == "__main__": global result, choices root = Tk() @@ -230,7 +230,3 @@ def main(): menu = OptionMenu(root, choices, "Vs Random", "Vs Pro", "Vs Legend") menu.pack() root.mainloop() - - -if __name__ == "__main__": - main() diff --git a/gui/tsp.py b/gui/tsp.py index 1830cba23..590fff354 100644 --- a/gui/tsp.py +++ b/gui/tsp.py @@ -1,21 +1,19 @@ from tkinter import * from tkinter import messagebox -import sys -import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) -from search import * + import utils -import numpy as np +from search import * -distances = {} +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) +distances = {} -class TSP_problem(Problem): - """ subclass of Problem to define various functions """ +class TSProblem(Problem): + """subclass of Problem to define various functions""" def two_opt(self, state): - """ Neighbour generating function for Traveling Salesman Problem """ + """Neighbour generating function for Traveling Salesman Problem""" neighbour_state = state[:] left = random.randint(0, len(neighbour_state) - 1) right = random.randint(0, len(neighbour_state) - 1) @@ -25,15 +23,15 @@ def two_opt(self, state): return neighbour_state def actions(self, state): - """ action that can be excuted in given state """ + """action that can be executed in given state""" return [self.two_opt] def result(self, state, action): - """ result after applying the given action on the given state """ + """result after applying the given action on the given state""" return action(state) def path_cost(self, c, state1, action, state2): - """ total distance for the Traveling Salesman to be covered if in state2 """ + """total distance for the Traveling Salesman to be covered if in state2""" cost = 0 for i in range(len(state2) - 1): cost += distances[state2[i]][state2[i + 1]] @@ -41,12 +39,12 @@ def path_cost(self, c, state1, action, state2): return cost def value(self, state): - """ value of path cost given negative for the given state """ + """value of path cost given negative for the given state""" return -1 * self.path_cost(None, None, None, state) -class TSP_Gui(): - """ Class to create gui of Traveling Salesman using simulated annealing where one can +class TSPGui(): + """Class to create gui of Traveling Salesman using simulated annealing where one can select cities, change speed and temperature. Distances between cities are euclidean distances between them. """ @@ -67,7 +65,7 @@ def __init__(self, root, all_cities): Label(self.root, text="Map of Romania", font="Times 13 bold").grid(row=0, columnspan=10) def create_checkboxes(self, side=LEFT, anchor=W): - """ To select cities which are to be a part of Traveling Salesman Problem """ + """To select cities which are to be a part of Traveling Salesman Problem""" row_number = 0 column_number = 0 @@ -85,7 +83,7 @@ def create_checkboxes(self, side=LEFT, anchor=W): row_number += 1 def create_buttons(self): - """ Create start and quit button """ + """Create start and quit button""" Button(self.frame_select_cities, textvariable=self.button_text, command=self.run_traveling_salesman).grid(row=5, column=4, sticky=E + W) @@ -93,7 +91,7 @@ def create_buttons(self): row=5, column=5, sticky=E + W) def create_dropdown_menu(self): - """ Create dropdown menu for algorithm selection """ + """Create dropdown menu for algorithm selection""" choices = {'Simulated Annealing', 'Genetic Algorithm', 'Hill Climbing'} self.algo_var.set('Simulated Annealing') @@ -102,19 +100,19 @@ def create_dropdown_menu(self): dropdown_menu.config(width=19) def run_traveling_salesman(self): - """ Choose selected citites """ + """Choose selected cities""" cities = [] for i in range(len(self.vars)): if self.vars[i].get() == 1: cities.append(self.all_cities[i]) - tsp_problem = TSP_problem(cities) + tsp_problem = TSProblem(cities) self.button_text.set("Reset") self.create_canvas(tsp_problem) def calculate_canvas_size(self): - """ Width and height for canvas """ + """Width and height for canvas""" minx, maxx = sys.maxsize, -1 * sys.maxsize miny, maxy = sys.maxsize, -1 * sys.maxsize @@ -137,7 +135,7 @@ def calculate_canvas_size(self): self.canvas_height = canvas_height def create_canvas(self, problem): - """ creating map with cities """ + """creating map with cities""" map_canvas = Canvas(self.frame_canvas, width=self.canvas_width, height=self.canvas_height) map_canvas.grid(row=3, columnspan=10) @@ -163,18 +161,18 @@ def create_canvas(self, problem): variable=self.speed, label="Speed ----> ", showvalue=0, font="Times 11", relief="sunken", cursor="gumby") speed_scale.grid(row=1, columnspan=5, sticky=N + S + E + W) - + if self.algo_var.get() == 'Simulated Annealing': self.temperature = IntVar() temperature_scale = Scale(self.frame_canvas, from_=100, to=0, orient=HORIZONTAL, - length=200, variable=self.temperature, label="Temperature ---->", - font="Times 11", relief="sunken", showvalue=0, cursor="gumby") + length=200, variable=self.temperature, label="Temperature ---->", + font="Times 11", relief="sunken", showvalue=0, cursor="gumby") temperature_scale.grid(row=1, column=5, columnspan=5, sticky=N + S + E + W) self.simulated_annealing_with_tunable_T(problem, map_canvas) elif self.algo_var.get() == 'Genetic Algorithm': self.mutation_rate = DoubleVar() self.mutation_rate.set(0.05) - mutation_rate_scale = Scale(self.frame_canvas, from_=0, to=1, orient=HORIZONTAL, + mutation_rate_scale = Scale(self.frame_canvas, from_=0, to=1, orient=HORIZONTAL, length=200, variable=self.mutation_rate, label='Mutation Rate ---->', font='Times 11', relief='sunken', showvalue=0, cursor='gumby', resolution=0.001) mutation_rate_scale.grid(row=1, column=5, columnspan=5, sticky='nsew') @@ -182,23 +180,23 @@ def create_canvas(self, problem): elif self.algo_var.get() == 'Hill Climbing': self.no_of_neighbors = IntVar() self.no_of_neighbors.set(100) - no_of_neighbors_scale = Scale(self.frame_canvas, from_=10, to=1000, orient=HORIZONTAL, + no_of_neighbors_scale = Scale(self.frame_canvas, from_=10, to=1000, orient=HORIZONTAL, length=200, variable=self.no_of_neighbors, label='Number of neighbors ---->', - font='Times 11',relief='sunken', showvalue=0, cursor='gumby') + font='Times 11', relief='sunken', showvalue=0, cursor='gumby') no_of_neighbors_scale.grid(row=1, column=5, columnspan=5, sticky='nsew') self.hill_climbing(problem, map_canvas) def exp_schedule(k=100, lam=0.03, limit=1000): - """ One possible schedule function for simulated annealing """ + """One possible schedule function for simulated annealing""" - return lambda t: (k * math.exp(-lam * t) if t < limit else 0) + return lambda t: (k * np.exp(-lam * t) if t < limit else 0) def simulated_annealing_with_tunable_T(self, problem, map_canvas, schedule=exp_schedule()): - """ Simulated annealing where temperature is taken as user input """ + """Simulated annealing where temperature is taken as user input""" current = Node(problem.initial) - while(1): + while True: T = schedule(self.temperature.get()) if T == 0: return current.state @@ -207,7 +205,7 @@ def simulated_annealing_with_tunable_T(self, problem, map_canvas, schedule=exp_s return current.state next = random.choice(neighbors) delta_e = problem.value(next.state) - problem.value(current.state) - if delta_e > 0 or probability(math.exp(delta_e / T)): + if delta_e > 0 or probability(np.exp(delta_e / T)): map_canvas.delete("poly") current = next @@ -221,10 +219,10 @@ def simulated_annealing_with_tunable_T(self, problem, map_canvas, schedule=exp_s map_canvas.after(self.speed.get()) def genetic_algorithm(self, problem, map_canvas): - """ Genetic Algorithm modified for the given problem """ + """Genetic Algorithm modified for the given problem""" def init_population(pop_number, gene_pool, state_length): - """ initialize population """ + """initialize population""" population = [] for i in range(pop_number): @@ -232,7 +230,7 @@ def init_population(pop_number, gene_pool, state_length): return population def recombine(state_a, state_b): - """ recombine two problem states """ + """recombine two problem states""" start = random.randint(0, len(state_a) - 1) end = random.randint(start + 1, len(state_a)) @@ -243,7 +241,7 @@ def recombine(state_a, state_b): return new_state def mutate(state, mutation_rate): - """ mutate problem states """ + """mutate problem states""" if random.uniform(0, 1) < mutation_rate: sample = random.sample(range(len(state)), 2) @@ -251,17 +249,18 @@ def mutate(state, mutation_rate): return state def fitness_fn(state): - """ calculate fitness of a particular state """ - + """calculate fitness of a particular state""" + fitness = problem.value(state) return int((5600 + fitness) ** 2) current = Node(problem.initial) population = init_population(100, current.state, len(current.state)) all_time_best = current.state - while(1): - population = [mutate(recombine(*select(2, population, fitness_fn)), self.mutation_rate.get()) for i in range(len(population))] - current_best = utils.argmax(population, key=fitness_fn) + while True: + population = [mutate(recombine(*select(2, population, fitness_fn)), self.mutation_rate.get()) + for _ in range(len(population))] + current_best = np.argmax(population, key=fitness_fn) if fitness_fn(current_best) > fitness_fn(all_time_best): all_time_best = current_best self.cost.set("Cost = " + str('%0.3f' % (-1 * problem.value(all_time_best)))) @@ -280,10 +279,10 @@ def fitness_fn(state): map_canvas.after(self.speed.get()) def hill_climbing(self, problem, map_canvas): - """ hill climbing where number of neighbors is taken as user input """ + """hill climbing where number of neighbors is taken as user input""" def find_neighbors(state, number_of_neighbors=100): - """ finds neighbors using two_opt method """ + """finds neighbors using two_opt method""" neighbors = [] for i in range(number_of_neighbors): @@ -293,9 +292,9 @@ def find_neighbors(state, number_of_neighbors=100): return neighbors current = Node(problem.initial) - while(1): + while True: neighbors = find_neighbors(current.state, self.no_of_neighbors.get()) - neighbor = utils.argmax_random_tie(neighbors, key=lambda node: problem.value(node.state)) + neighbor = np.argmax_random_tie(neighbors, key=lambda node: problem.value(node.state)) map_canvas.delete('poly') points = [] for city in current.state: @@ -317,7 +316,8 @@ def on_closing(self): if messagebox.askokcancel('Quit', 'Do you want to quit?'): self.root.destroy() -def main(): + +if __name__ == '__main__': all_cities = [] for city in romania_map.locations.keys(): distances[city] = {} @@ -334,13 +334,9 @@ def main(): root = Tk() root.title("Traveling Salesman Problem") - cities_selection_panel = TSP_Gui(root, all_cities) + cities_selection_panel = TSPGui(root, all_cities) cities_selection_panel.create_checkboxes() cities_selection_panel.create_buttons() cities_selection_panel.create_dropdown_menu() root.protocol('WM_DELETE_WINDOW', cities_selection_panel.on_closing) root.mainloop() - - -if __name__ == '__main__': - main() diff --git a/gui/vacuum_agent.py b/gui/vacuum_agent.py index 23292efb3..b07dab282 100644 --- a/gui/vacuum_agent.py +++ b/gui/vacuum_agent.py @@ -1,15 +1,14 @@ -from tkinter import * -import random -import sys import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) +from tkinter import * + from agents import * +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) + loc_A, loc_B = (0, 0), (1, 0) # The two locations for the Vacuum world class Gui(Environment): - """This GUI environment has two locations, A and B. Each can be Dirty or Clean. The agent perceives its location and the location's status.""" @@ -33,7 +32,7 @@ def thing_classes(self): def percept(self, agent): """Returns the agent's location, and the location status (Dirty/Clean).""" - return (agent.location, self.status[agent.location]) + return agent.location, self.status[agent.location] def execute_action(self, agent, action): """Change the location status (Dirty/Clean); track performance. @@ -137,8 +136,7 @@ def move_agent(env, agent, before_step): # TODO: Add more agents to the environment. # TODO: Expand the environment to XYEnvironment. -def main(): - """The main function of the program.""" +if __name__ == "__main__": root = Tk() root.title("Vacuum Environment") root.geometry("420x380") @@ -154,7 +152,3 @@ def main(): create_agent(env, agent) next_button.config(command=lambda: env.update_env(agent)) root.mainloop() - - -if __name__ == "__main__": - main() diff --git a/gui/xy_vacuum_environment.py b/gui/xy_vacuum_environment.py index 4ba4497ea..093abc6c3 100644 --- a/gui/xy_vacuum_environment.py +++ b/gui/xy_vacuum_environment.py @@ -1,10 +1,10 @@ -from tkinter import * -import random -import sys import os.path -sys.path.append(os.path.join(os.path.dirname(__file__), '..')) +from tkinter import * + from agents import * +sys.path.append(os.path.join(os.path.dirname(__file__), '..')) + class Gui(VacuumEnvironment): """This is a two-dimensional GUI environment. Each location may be @@ -13,8 +13,10 @@ class Gui(VacuumEnvironment): xi, yi = (0, 0) perceptible_distance = 1 - def __init__(self, root, width=7, height=7, elements=['D', 'W']): + def __init__(self, root, width=7, height=7, elements=None): super().__init__(width, height) + if elements is None: + elements = ['D', 'W'] self.root = root self.create_frames() self.create_buttons() @@ -71,10 +73,10 @@ def display_element(self, button): def execute_action(self, agent, action): """Determines the action the agent performs.""" - xi, yi = ((self.xi, self.yi)) + xi, yi = (self.xi, self.yi) if action == 'Suck': dirt_list = self.list_things_at(agent.location, Dirt) - if dirt_list != []: + if dirt_list: dirt = dirt_list[0] agent.performance += 100 self.delete_thing(dirt) @@ -166,11 +168,9 @@ def __init__(self, program=None): self.direction = Direction("up") -# TODO: -# Check the coordinate system. -# Give manual choice for agent's location. -def main(): - """The main function.""" +# TODO: Check the coordinate system. +# TODO: Give manual choice for agent's location. +if __name__ == "__main__": root = Tk() root.title("Vacuum Environment") root.geometry("420x440") @@ -189,7 +189,3 @@ def main(): next_button.config(command=env.update_env) reset_button.config(command=lambda: env.reset_env(agt)) root.mainloop() - - -if __name__ == "__main__": - main() diff --git a/images/broxrevised.png b/images/broxrevised.png new file mode 100644 index 000000000..87051a383 Binary files /dev/null and b/images/broxrevised.png differ diff --git a/images/stapler1-test.png b/images/stapler1-test.png new file mode 100644 index 000000000..e550d83f9 Binary files /dev/null and b/images/stapler1-test.png differ diff --git a/improving_sat_algorithms.ipynb b/improving_sat_algorithms.ipynb new file mode 100644 index 000000000..d461e99c4 --- /dev/null +++ b/improving_sat_algorithms.ipynb @@ -0,0 +1,2539 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "# Propositional Logic\n", + "---\n", + "# Improving Boolean Satisfiability Algorithms\n", + "\n", + "## Introduction\n", + "A propositional formula $\\Phi$ in *Conjunctive Normal Form* (CNF) is a conjunction of clauses $\\omega_j$, with $j \\in \\{1,...,m\\}$. Each clause being a disjunction of literals and each literal being either a positive ($x_i$) or a negative ($\\lnot{x_i}$) propositional variable, with $i \\in \\{1,...,n\\}$. By denoting with $[\\lnot]$ the possible presence of $\\lnot$, we can formally define $\\Phi$ as:\n", + "\n", + "$$\\bigwedge_{j = 1,...,m}\\bigg(\\bigvee_{i \\in \\omega_j} [\\lnot] x_i\\bigg)$$\n", + "\n", + "The ***Boolean Satisfiability Problem*** (SAT) consists in determining whether there exists a truth assignment in $\\{0, 1\\}$ (or equivalently in $\\{True,False\\}$) for the variables in $\\Phi$." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from logic import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## DPLL with Branching Heuristics\n", + "The ***Davis-Putnam-Logemann-Loveland*** (DPLL) algorithm is a *complete* (will answer SAT if a solution exists) and *sound* (it will not answer SAT for an unsatisfiable formula) procedue that combines *backtracking search* and *deduction* to decide satisfiability of propositional logic formula in CNF. At each search step a variable and a propositional value are selected for branching purposes. With each branching step, two values can be assigned to a variable, either 0 or 1. Branching corresponds to assigning the chosen value to the chosen variable. Afterwards, the logical consequences of each branching step are evaluated. Each time an unsatisfied clause (ie a *conflict*) is identified, backtracking is executed. Backtracking corresponds to undoing branching steps until an unflipped branch is reached. When both values have been assigned to the selected variable at a branching step, backtracking will undo this branching step. If for the first branching step both values have been considered, and backtracking undoes this first branching step, then the CNF formula can be declared unsatisfiable. This kind of backtracking is called *chronological backtracking*.\n", + "\n", + "Essentially, `DPLL` is a backtracking depth-first search through partial truth assignments which uses a *splitting rule* to replaces the original problem with two smaller subproblems, whereas the original Davis-Putnam procedure uses a variable elimination rule which replaces the original problem with one larger subproblem. Over the years, many heuristics have been proposed in choosing the splitting variable (which variable should be assigned a truth value next).\n", + "\n", + "Search algorithms that are based on a predetermined order of search are called static algorithms, whereas the ones that select them at the runtime are called dynamic. The first SAT search algorithm, the Davis-Putnam procedure is a static algorithm. Static search algorithms are usually very slow in practice and for this reason perform worse than dynamic search algorithms. However, dynamic search algorithms are much harder to design, since they require a heuristic for predetermining the order of search. The fundamental element of a heuristic is a branching strategy for selecting the next branching literal. This must not require a lot of time to compute and yet it must provide a powerful insight into the problem instance.\n", + "\n", + "Two basic heuristics are applied to this algorithm with the potential of cutting the search space in half. These are the *pure literal rule* and the *unit clause rule*.\n", + "- the *pure literal* rule is applied whenever a variable appears with a single polarity in all the unsatisfied clauses. In this case, assigning a truth value to the variable so that all the involved clauses are satisfied is highly effective in the search;\n", + "- if some variable occurs in the current formula in a clause of length 1 then the *unit clause* rule is applied. Here, the literal is selected and a truth value so the respective clause is satisfied is assigned. The iterative application of the unit rule is commonly reffered to as *Boolean Constraint Propagation* (BCP)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdpll_satisfiable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mno_branching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"Check satisfiability of a propositional sentence.\u001b[0m\n", + "\u001b[0;34m This differs from the book code in two ways: (1) it returns a model\u001b[0m\n", + "\u001b[0;34m rather than True when it succeeds; this is more useful. (2) The\u001b[0m\n", + "\u001b[0;34m function find_pure_symbol is passed a list of unknown clauses, rather\u001b[0m\n", + "\u001b[0;34m than a list of all clauses and the model; this is more efficient.\u001b[0m\n", + "\u001b[0;34m >>> dpll_satisfiable(A |'<=>'| B) == {A: True, B: True}\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_cnf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dpll_satisfiable" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mno_branching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"See if the clauses are true in a partial model.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munknown_clauses\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;31m# clauses with an unknown truth value\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mval\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munknown_clauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0munknown_clauses\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfind_pure_symbol\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munknown_clauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfind_unit_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munknown_clauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mor\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdpll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbranching_heuristic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dpll" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each of these branching heuristics was applied only after the *pure literal* and the *unit clause* heuristic failed in selecting a splitting variable." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### MOMs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "MOMs heuristics are simple, efficient and easy to implement. The goal of these heuristics is to prefer the literal having ***Maximum number of Occurences in the Minimum length clauses***. Intuitively, the literals belonging to the minimum length clauses are the most constrained literals in the formula. Branching on them will maximize the effect of BCP and the likelihood of hitting a dead end early in the search tree (for unsatisfiable problems). Conversely, in the case of satisfiable formulas, branching on a highly constrained variable early in the tree will also increase the likelihood of a correct assignment of the remained open literals.\n", + "The MOMs heuristics main disadvatage is that their effectiveness highly depends on the problem instance. It is easy to see that the ideal setting for these heuristics is considering the unsatisfied binary clauses." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mmin_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmin_len\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdefault\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mmin_len\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmin_len\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource min_clauses" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mmoms\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m MOMS (Maximum Occurrence in clauses of Minimum Size) heuristic\u001b[0m\n", + "\u001b[0;34m Returns the literal with the most occurrences in all clauses of minimum size\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmin_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource moms" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Over the years, many types of MOMs heuristics have been proposed.\n", + "\n", + "***MOMSf*** choose the variable $x$ with a maximize the function:\n", + "\n", + "$$[f(x) + f(\\lnot{x})] * 2^k + f(x) * f(\\lnot{x})$$\n", + "\n", + "where $f(x)$ is the number of occurrences of $x$ in the smallest unknown clauses, k is a parameter." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mmomsf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m MOMS alternative heuristic\u001b[0m\n", + "\u001b[0;34m If f(x) the number of occurrences of the variable x in clauses with minimum size,\u001b[0m\n", + "\u001b[0;34m we choose the variable maximizing [f(x) + f(-x)] * 2^k + f(x) * f(-x)\u001b[0m\n", + "\u001b[0;34m Returns x if f(x) >= f(-x) otherwise -x\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmin_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mpow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource momsf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "***Freeman’s POSIT***
    [[1]](#cite-freeman1995improvements) version counts both the number of positive $x$ and negative $\\lnot{x}$ occurrences of a given variable $x$." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mposit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Freeman's POSIT version of MOMs\u001b[0m\n", + "\u001b[0;34m Counts the positive x and negative x for each variable x in clauses with minimum size\u001b[0m\n", + "\u001b[0;34m Returns x if f(x) >= f(-x) otherwise -x\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmin_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource posit" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "***Zabih and McAllester’s*** [[2]](#cite-zabih1988rearrangement) version of the heuristic counts the negative occurrences $\\lnot{x}$ of each given variable $x$." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mzm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Zabih and McAllester's version of MOMs\u001b[0m\n", + "\u001b[0;34m Counts the negative occurrences only of each variable x in clauses with minimum size\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmin_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mop\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'~'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource zm" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### DLIS & DLCS" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Literal count heuristics count the number of unresolved clauses in which a given variable $x$ appears as a positive literal, $C_P$ , and as negative literal, $C_N$. These two numbers an either be onsidered individually or ombined. \n", + "\n", + "***Dynamic Largest Individual Sum*** heuristic considers the values $C_P$ and $C_N$ separately: select the variable with the largest individual value and assign to it value true if $C_P \\geq C_N$, value false otherwise." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdlis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m DLIS (Dynamic Largest Individual Sum) heuristic\u001b[0m\n", + "\u001b[0;34m Choose the variable and value that satisfies the maximum number of unsatisfied clauses\u001b[0m\n", + "\u001b[0;34m Like DLCS but we only consider the literal (thus Cp and Cn are individual)\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dlis" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "***Dynamic Largest Combined Sum*** considers the values $C_P$ and $C_N$ combined: select the variable with the largest sum $C_P + C_N$ and assign to it value true if $C_P \\geq C_N$, value false otherwise." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mdlcs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m DLCS (Dynamic Largest Combined Sum) heuristic\u001b[0m\n", + "\u001b[0;34m Cp the number of clauses containing literal x\u001b[0m\n", + "\u001b[0;34m Cn the number of clauses containing literal -x\u001b[0m\n", + "\u001b[0;34m Here we select the variable maximizing Cp + Cn\u001b[0m\n", + "\u001b[0;34m Returns x if Cp >= Cn otherwise -x\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource dlcs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### JW & JW2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Two branching heuristics were proposed by ***Jeroslow and Wang*** in [[3]](#cite-jeroslow1990solving).\n", + "\n", + "The *one-sided Jeroslow and Wang*’s heuristic compute:\n", + "\n", + "$$J(l) = \\sum_{l \\in \\omega \\land \\omega \\in \\phi} 2^{-|\\omega|}$$\n", + "\n", + "and selects the assignment that satisfies the literal with the largest value $J(l)$." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mjw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Jeroslow-Wang heuristic\u001b[0m\n", + "\u001b[0;34m For each literal compute J(l) = \\sum{l in clause c} 2^{-|c|}\u001b[0m\n", + "\u001b[0;34m Return the literal maximizing J\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mpow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource jw" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The *two-sided Jeroslow and Wang*’s heuristic identifies the variable $x$ with the largest sum $J(x) + J(\\lnot{x})$, and assigns to $x$ value true, if $J(x) \\geq J(\\lnot{x})$, and value false otherwise." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mjw2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m Two Sided Jeroslow-Wang heuristic\u001b[0m\n", + "\u001b[0;34m Compute J(l) also counts the negation of l = J(x) + J(-x)\u001b[0m\n", + "\u001b[0;34m Returns x if J(x) >= J(-x) otherwise -x\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mpow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource jw2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CDCL with 1UIP Learning Scheme, 2WL Lazy Data Structure, VSIDS Branching Heuristic & Restarts\n", + "\n", + "The ***Conflict-Driven Clause Learning*** (CDCL) solver is an evolution of the *DPLL* algorithm that involves a number of additional key techniques:\n", + "\n", + "- non-chronological backtracking or *backjumping*;\n", + "- *learning* new *clauses* from conflicts during search by exploiting its structure;\n", + "- using *lazy data structures* for storing clauses;\n", + "- *branching heuristics* with low computational overhead and which receive feedback from search;\n", + "- periodically *restarting* search.\n", + "\n", + "The first difference between a DPLL solver and a CDCL solver is the introduction of the *non-chronological backtracking* or *backjumping* when a conflict is identified. This requires an iterative implementation of the algorithm because only if the backtrack stack is managed explicitly it is possible to backtrack more than one level." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mcdcl_satisfiable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvsids_decay\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.95\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrestart_strategy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mno_restart\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m >>> cdcl_satisfiable(A |'<=>'| B) == {A: True, B: True}\u001b[0m\n", + "\u001b[0;34m True\u001b[0m\n", + "\u001b[0;34m \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTwoWLClauseDatabase\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_cnf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msymbols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCounter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDiGraph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflicts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrestarts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msum_lbd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue_lbd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munit_propagation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconflict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mdl\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflicts\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlearn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlbd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconflict_analysis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue_lbd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlbd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msum_lbd\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mlbd\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbackjump\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0msymbol\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m*=\u001b[0m \u001b[0mvsids_decay\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mrestart_strategy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflicts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrestarts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue_lbd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msum_lbd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbackjump\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mqueue_lbd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclear\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mrestarts\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdl\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0massign_decision_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource cdcl_satisfiable" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Clause Learning with 1UIP Scheme" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The second important difference between a DPLL solver and a CDCL solver is that the information about a conflict is reused by learning: if a conflicting clause is found, the solver derive a new clause from the conflict and add it to the clauses database.\n", + "\n", + "Whenever a conflict is identified due to unit propagation, a conflict analysis procedure is invoked. As a result, one or more new clauses are learnt, and a backtracking decision level is computed. The conflict analysis procedure analyzes the structure of unit propagation and decides which literals to include in the learnt clause. The decision levels associated with assigned variables define a partial order of the variables. Starting from a given unsatisfied clause (represented in the implication graph with vertex $\\kappa$), the conflict analysis procedure visits variables implied at the most recent decision level (ie the current largest decision level), identifies the antecedents of visited variables, and keeps from the antecedents the literals assigned at decision levels less than the most recent decision level. The clause learning procedure used in the CDCL can be defined by a sequence of selective resolution operations, that at each step yields a new temporary clause. This process is repeated until the most recent decision variable is visited.\n", + "\n", + "The structure of implied assignments induced by unit propagation is a key aspect of the clause learning procedure. Moreover, the idea of exploiting the structure induced by unit propagation was further exploited with ***Unit Implication Points*** (UIPs). A UIP is a *dominator* in the implication graph and represents an alternative decision assignment at the current decision level that results in the same conflict. The main motivation for identifying UIPs is to reduce the size of learnt clauses. Clause learning could potentially stop at any UIP, being quite straightforward to conclude that the set of literals of a clause learnt at the first UIP has clear advantages. Considering the largest decision level of the literals of the clause learnt at each UIP, the clause learnt at the first UIP is guaranteed to contain the smallest one. This guarantees the highest backtrack jump in the search tree." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mconflict_analysis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict_clause\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'K'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'antecedent'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpred\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'K'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnode\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;34m'K'\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'dl'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mdl\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0min_degree\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mfirst_uip\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimmediate_dominators\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'K'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove_node\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'K'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict_side\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdescendants\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfirst_uip\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflict_clause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mintersection\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflict_side\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mantecedent\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'antecedent'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpred\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict_clause\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpl_binary_resolution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflict_clause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mantecedent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the literal block distance is calculated by taking the decision levels from variables of all\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# literals in the clause, and counting how many different decision levels were in this set\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mlbd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'dl'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflict_clause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlbd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcount\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mfirst_uip\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflict_clause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;36m0\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlbd\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mheapq\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnlargest\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlbd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconflict_clause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlbd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource conflict_analysis" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mpl_binary_resolution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mci\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mdi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mci\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mdj\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mdi\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m~\u001b[0m\u001b[0mdj\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m~\u001b[0m\u001b[0mdi\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mdj\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mpl_binary_resolution\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'|'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mci\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'|'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mremove_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0massociate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'|'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munique\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mci\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource pl_binary_resolution" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mbackjump\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mdelete\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mnode\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnode\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'dl'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove_nodes_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdelete\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnode\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdelete\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdel\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msymbols\u001b[0m \u001b[0;34m|=\u001b[0m \u001b[0mdelete\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource backjump" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 2WL Lazy Data Structure" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Implementation issues for SAT solvers include the design of suitable data structures for storing clauses. The implemented data structures dictate the way BCP are implemented and have a significant impact on the run time performance of the SAT solver. Recent state-of-the-art SAT solvers are characterized by using very efficient data structures, intended to reduce the CPU time required per each node in the search tree. Conversely, traditional SAT data structures are accurate, meaning that is possible to know exactly the value of each literal in the clause. Examples of the most recent SAT data structures, which are not accurate and therefore are called lazy, include the watched literals used in Chaff .\n", + "\n", + "The more recent Chaff SAT solver [[4]](#cite-moskewicz2001chaff) proposed a new data structure, the ***2 Watched Literals*** (2WL), in which two references are associated with each clause. There is no order relation between the two references, allowing the references to move in any direction. The lack of order between the two references has the key advantage that no literal references need to be updated when backtracking takes place. In contrast, unit or unsatisfied clauses are identified only after traversing all the clauses’ literals; a clear drawback. The two watched literal pointers are undifferentiated as there is no order relation. Again, each time one literal pointed by one of these pointers is assigned, the pointer has to move inwards. These pointers may move in both directions. This causes the whole clause to be traversed when the clause becomes unit. In addition, no references have to be kept to the just assigned literals, since pointers do not move when backtracking." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0munit_propagation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcheck\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw1\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_neg_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_pos_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw2\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_neg_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_pos_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0munit_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwatching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwatching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_node\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_edges_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcycle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mantecedent\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mp\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mconflict_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_edges_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprop_symbols\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mitertools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcycle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'K'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mantecedent\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcheck\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# we need only visit each clause when one of its two watched literals is assigned to 0 because, until\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# this happens, we can guarantee that there cannot be more than n-2 literals in the clause assigned to 0\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mfirst_watched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msecond_watched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfirst_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munit_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mfirst_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mFalse\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0msecond_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if the only literal with a non-zero value is the other watched literal then\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msecond_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# if it is free, then the clause is a unit clause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munit_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# else (it is False) the clause is a conflict clause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0msecond_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mFalse\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mfirst_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if the only literal with a non-zero value is the other watched literal then\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfirst_watched\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# if it is free, then the clause is a unit clause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0munit_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclauses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbcp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# else (it is False) the clause is a conflict clause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mconflict_clause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mbcp\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource unit_propagation" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mTwoWLClauseDatabase\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mc\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mclauses\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_clauses\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mset_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mset_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_pos_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_neg_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__assign_watching_literals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp1\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw1\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mw2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp2\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdel\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__twl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdiscard\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp1\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdiscard\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mw1\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mw2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdiscard\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp2\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdiscard\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mupdate_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if a non-zero literal different from the other watched literal is found\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mfound\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__find_new_watching_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfound\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# then it will replace the watched literal\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mupdate_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if a non-zero literal different from the other watched literal is found\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mfound\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__find_new_watching_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_second_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfound\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# then it will replace the watched literal\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_first_watched\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_watching\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__watch_list\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__find_new_watching_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mother_watched\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if a non-zero literal different from the other watched literal is found\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0ml\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mother_watched\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# then it is returned\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__assign_watching_literals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclause\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdisjuncts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclause\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mpl_true\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource TwoWLClauseDatabase" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### VSIDS Branching Heuristic" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The early branching heuristics made use of all the information available from the data structures, namely the number of satisfied, unsatisfied and unassigned literals. These heuristics are updated during the search and also take into account the clauses that are learnt. \n", + "\n", + "More recently, a different kind of variable selection heuristic, referred to as ***Variable State Independent Decaying Sum*** (VSIDS), has been proposed by Chaff authors in [[4]](#cite-moskewicz2001chaff). One of the reasons for proposing this new heuristic was the introduction of lazy data structures, where the knowledge of the dynamic size of a clause is not accurate. Hence, the heuristics described above cannot be used. VSIDS selects the literal that appears most frequently over all the clauses, which means that one counter is required for each one of the literals. Initially, all counters are set to zero. During the search, the metrics only have to be updated when a new recorded clause is created. More than to develop an accurate heuristic, the motivation has been to design a fast (but dynamically adapting) heuristic. In fact, one of the key properties of this strategy is the very low overhead, due to being independent of the variable state." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0massign_decision_literal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msymbols\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0msymbol\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0msymbol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mvalue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0msymbols\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_node\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource assign_decision_literal" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Restarts" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Solving NP-complete problems, such as SAT, naturally leads to heavy-tailed run times. To deal with this, SAT solvers frequently restart their search to avoid the runs that take disproportionately longer. What restarting here means is that the solver unsets all variables and starts the search using different variable assignment order.\n", + "\n", + "While at first glance it might seem that restarts should be rare and become rarer as the solving has been going on for longer, so that the SAT solver can actually finish solving the problem, the trend has been towards more aggressive (frequent) restarts.\n", + "\n", + "The reason why frequent restarts help solve problems faster is that while the solver does forget all current variable assignments, it does keep some information, specifically it keeps learnt clauses, effectively sampling the search space, and it keeps the last assigned truth value of each variable, assigning them the same value the next time they are picked to be assigned." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Luby" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this strategy, the number of conflicts between 2 restarts is based on the *Luby* sequence. The *Luby* restart sequence is interesting in that it was proven to be optimal restart strategy for randomized search algorithms where the runs do not share information. While this is not true for SAT solving, as shown in [[5]](cite-haim2014towards) and [[6]](cite-huang2007effect), *Luby* restarts have been quite successful anyway.\n", + "\n", + "The exact description of *Luby* restarts is that the $ith$ restart happens after $u \\cdot Luby(i)$ conflicts, where $u$ is a constant and $Luby(i)$ is defined as:\n", + "\n", + "$$Luby(i) = \\begin{cases} \n", + " 2^{k-1} & i = 2^k - 1 \\\\\n", + " Luby(i - 2^{k-1} + 1) & 2^{k-1} \\leq i < 2^k - 1\n", + " \\end{cases}\n", + "$$\n", + "\n", + "A less exact but more intuitive description of the *Luby* sequence is that all numbers in it are powers of two, and after a number is seen for the second time, the next number is twice as big. The following are the first 16 numbers in the sequence:\n", + "\n", + "$$ (1,1,2,1,1,2,4,1,1,2,1,1,2,4,8,1,...) $$\n", + "\n", + "From the above, we can see that this restart strategy tends towards frequent restarts, but some runs are kept running for much longer, and there is no upper limit on the longest possible time between two restarts." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mluby\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflicts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrestarts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue_lbd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msum_lbd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munit\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m512\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# in the state-of-art tested with unit value 1, 2, 4, 6, 8, 12, 16, 32, 64, 128, 256 and 512\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_luby\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mi\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m \u001b[0;34m<<\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;34m<<\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m \u001b[0;34m<<\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0mi\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m \u001b[0;34m<<\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0m_luby\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m \u001b[0;34m<<\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mk\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0munit\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0m_luby\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrestarts\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue_lbd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource luby" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Glucose" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Glucose restarts were popularized by the *Glucose* solver, and it is an extremely aggressive, dynamic restart strategy. The idea behind it and described in [[7]](cite-audemard2012refining) is that instead of waiting for a fixed amount of conflicts, we restart when the last couple of learnt clauses are, on average, bad.\n", + "\n", + "A bit more precisely, if there were at least $X$ conflicts (and thus $X$ learnt clauses) since the last restart, and the average *Literal Block Distance* (LBD) (a criterion to evaluate the quality of learnt clauses as shown in [[8]](#cite-audemard2009predicting) of the last $X$ learnt clauses was at least $K$ times higher than the average LBD of all learnt clauses, it is time for another restart. Parameters $X$ and $K$ can be tweaked to achieve different restart frequency, and they are usually kept quite small." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mglucose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconflicts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrestarts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mqueue_lbd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msum_lbd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# in the state-of-art tested with (x, k) as (50, 0.8) and (100, 0.7)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# if there were at least x conflicts since the last restart, and then the average LBD of the last\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# x learnt clauses was at least k times higher than the average LBD of all learnt clauses\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue_lbd\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue_lbd\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqueue_lbd\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mk\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0msum_lbd\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mconflicts\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource glucose" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": {} + }, + "source": [ + "## Experimental Results" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "from csp import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Australia" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSP" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "australia_csp = MapColoringCSP(list('RGB'), \"\"\"SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: \"\"\")" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 154 µs, sys: 37 µs, total: 191 µs\n", + "Wall time: 194 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP needs 72 consistency-checks'" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3b(australia_csp, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 263 µs, sys: 0 ns, total: 263 µs\n", + "Wall time: 268 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "{'Q': 'R', 'SA': 'G', 'NSW': 'B', 'NT': 'B', 'V': 'R', 'WA': 'R'}" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time backtracking_search(australia_csp, select_unassigned_variable=mrv, inference=forward_checking)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SAT" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "australia_sat = MapColoringSAT(list('RGB'), \"\"\"SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: \"\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 43.3 ms, sys: 0 ns, total: 43.3 ms\n", + "Wall time: 41.5 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=no_branching_heuristic)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 36.4 ms, sys: 0 ns, total: 36.4 ms\n", + "Wall time: 35.3 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=moms)" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 36.1 ms, sys: 3.9 ms, total: 40 ms\n", + "Wall time: 39.2 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=momsf)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 45.2 ms, sys: 0 ns, total: 45.2 ms\n", + "Wall time: 44.2 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=posit)" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 31.2 ms, sys: 0 ns, total: 31.2 ms\n", + "Wall time: 30.5 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=zm)" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 57 ms, sys: 0 ns, total: 57 ms\n", + "Wall time: 55.9 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=dlis)" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 51.8 ms, sys: 0 ns, total: 51.8 ms\n", + "Wall time: 50.7 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=dlcs)" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 40.6 ms, sys: 0 ns, total: 40.6 ms\n", + "Wall time: 39.3 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=jw)" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 43.2 ms, sys: 1.81 ms, total: 45.1 ms\n", + "Wall time: 43.9 ms\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(australia_sat, branching_heuristic=jw2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 32.9 ms, sys: 16 µs, total: 33 ms\n", + "Wall time: 31.6 ms\n" + ] + } + ], + "source": [ + "%time model = cdcl_satisfiable(australia_sat)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{NSW_B, NT_B, Q_G, SA_R, V_G, WA_G}" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{var for var, val in model.items() if val}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### France" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSP" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "france_csp = MapColoringCSP(list('RGBY'),\n", + " \"\"\"AL: LO FC; AQ: MP LI PC; AU: LI CE BO RA LR MP; BO: CE IF CA FC RA\n", + " AU; BR: NB PL; CA: IF PI LO FC BO; CE: PL NB NH IF BO AU LI PC; FC: BO\n", + " CA LO AL RA; IF: NH PI CA BO CE; LI: PC CE AU MP AQ; LO: CA AL FC; LR:\n", + " MP AU RA PA; MP: AQ LI AU LR; NB: NH CE PL BR; NH: PI IF CE NB; NO:\n", + " PI; PA: LR RA; PC: PL CE LI AQ; PI: NH NO CA IF; PL: BR NB CE PC; RA:\n", + " AU BO FC PA LR\"\"\")" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 599 µs, sys: 112 µs, total: 711 µs\n", + "Wall time: 716 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP needs 516 consistency-checks'" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3b(france_csp, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 560 µs, sys: 0 ns, total: 560 µs\n", + "Wall time: 563 µs\n" + ] + }, + { + "data": { + "text/plain": [ + "{'NH': 'R',\n", + " 'NB': 'G',\n", + " 'CE': 'B',\n", + " 'PL': 'R',\n", + " 'BR': 'B',\n", + " 'IF': 'G',\n", + " 'PI': 'B',\n", + " 'BO': 'R',\n", + " 'CA': 'Y',\n", + " 'FC': 'G',\n", + " 'LO': 'R',\n", + " 'PC': 'G',\n", + " 'AU': 'G',\n", + " 'AL': 'B',\n", + " 'RA': 'B',\n", + " 'LR': 'R',\n", + " 'LI': 'R',\n", + " 'AQ': 'B',\n", + " 'MP': 'Y',\n", + " 'PA': 'G',\n", + " 'NO': 'R'}" + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time backtracking_search(france_csp, select_unassigned_variable=mrv, inference=forward_checking)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SAT" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [], + "source": [ + "france_sat = MapColoringSAT(list('RGBY'),\n", + " \"\"\"AL: LO FC; AQ: MP LI PC; AU: LI CE BO RA LR MP; BO: CE IF CA FC RA\n", + " AU; BR: NB PL; CA: IF PI LO FC BO; CE: PL NB NH IF BO AU LI PC; FC: BO\n", + " CA LO AL RA; IF: NH PI CA BO CE; LI: PC CE AU MP AQ; LO: CA AL FC; LR:\n", + " MP AU RA PA; MP: AQ LI AU LR; NB: NH CE PL BR; NH: PI IF CE NB; NO:\n", + " PI; PA: LR RA; PC: PL CE LI AQ; PI: NH NO CA IF; PL: BR NB CE PC; RA:\n", + " AU BO FC PA LR\"\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.32 s, sys: 0 ns, total: 3.32 s\n", + "Wall time: 3.32 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=no_branching_heuristic)" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.17 s, sys: 390 µs, total: 3.17 s\n", + "Wall time: 3.17 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=moms)" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.49 s, sys: 0 ns, total: 3.49 s\n", + "Wall time: 3.49 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=momsf)" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.5 s, sys: 0 ns, total: 3.5 s\n", + "Wall time: 3.5 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=posit)" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3 s, sys: 2.6 ms, total: 3.01 s\n", + "Wall time: 3.01 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=zm)" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 12.5 s, sys: 11.4 ms, total: 12.5 s\n", + "Wall time: 12.5 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=dlis)" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.41 s, sys: 0 ns, total: 3.41 s\n", + "Wall time: 3.41 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=dlcs)" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 2.92 s, sys: 3.89 ms, total: 2.92 s\n", + "Wall time: 2.92 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=jw)" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 3.71 s, sys: 0 ns, total: 3.71 s\n", + "Wall time: 3.73 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(france_sat, branching_heuristic=jw2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 159 ms, sys: 3.94 ms, total: 163 ms\n", + "Wall time: 162 ms\n" + ] + } + ], + "source": [ + "%time model = cdcl_satisfiable(france_sat)" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{AL_G,\n", + " AQ_G,\n", + " AU_R,\n", + " BO_G,\n", + " BR_Y,\n", + " CA_R,\n", + " CE_B,\n", + " FC_B,\n", + " IF_Y,\n", + " LI_Y,\n", + " LO_Y,\n", + " LR_G,\n", + " MP_B,\n", + " NB_R,\n", + " NH_G,\n", + " NO_Y,\n", + " PA_B,\n", + " PC_R,\n", + " PI_B,\n", + " PL_G,\n", + " RA_Y}" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{var for var, val in model.items() if val}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### USA" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSP" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [], + "source": [ + "usa_csp = MapColoringCSP(list('RGBY'),\n", + " \"\"\"WA: OR ID; OR: ID NV CA; CA: NV AZ; NV: ID UT AZ; ID: MT WY UT;\n", + " UT: WY CO AZ; MT: ND SD WY; WY: SD NE CO; CO: NE KA OK NM; NM: OK TX AZ;\n", + " ND: MN SD; SD: MN IA NE; NE: IA MO KA; KA: MO OK; OK: MO AR TX;\n", + " TX: AR LA; MN: WI IA; IA: WI IL MO; MO: IL KY TN AR; AR: MS TN LA;\n", + " LA: MS; WI: MI IL; IL: IN KY; IN: OH KY; MS: TN AL; AL: TN GA FL;\n", + " MI: OH IN; OH: PA WV KY; KY: WV VA TN; TN: VA NC GA; GA: NC SC FL;\n", + " PA: NY NJ DE MD WV; WV: MD VA; VA: MD DC NC; NC: SC; NY: VT MA CT NJ;\n", + " NJ: DE; DE: MD; MD: DC; VT: NH MA; MA: NH RI CT; CT: RI; ME: NH;\n", + " HI: ; AK: \"\"\")" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.58 ms, sys: 17 µs, total: 1.6 ms\n", + "Wall time: 1.6 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP needs 1284 consistency-checks'" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3b(usa_csp, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 2.15 ms, sys: 0 ns, total: 2.15 ms\n", + "Wall time: 2.15 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "{'NM': 'R',\n", + " 'TX': 'G',\n", + " 'OK': 'B',\n", + " 'AR': 'R',\n", + " 'MO': 'G',\n", + " 'KA': 'R',\n", + " 'LA': 'B',\n", + " 'NE': 'B',\n", + " 'TN': 'B',\n", + " 'MS': 'G',\n", + " 'IA': 'R',\n", + " 'SD': 'G',\n", + " 'IL': 'B',\n", + " 'CO': 'G',\n", + " 'MN': 'B',\n", + " 'KY': 'R',\n", + " 'AL': 'R',\n", + " 'GA': 'G',\n", + " 'FL': 'B',\n", + " 'VA': 'G',\n", + " 'WI': 'G',\n", + " 'IN': 'G',\n", + " 'NC': 'R',\n", + " 'WV': 'B',\n", + " 'OH': 'Y',\n", + " 'PA': 'R',\n", + " 'MD': 'Y',\n", + " 'SC': 'B',\n", + " 'MI': 'R',\n", + " 'DC': 'R',\n", + " 'DE': 'G',\n", + " 'WY': 'R',\n", + " 'ND': 'R',\n", + " 'NJ': 'B',\n", + " 'NY': 'G',\n", + " 'UT': 'B',\n", + " 'AZ': 'G',\n", + " 'ID': 'G',\n", + " 'MT': 'B',\n", + " 'NV': 'R',\n", + " 'CA': 'B',\n", + " 'OR': 'Y',\n", + " 'WA': 'R',\n", + " 'VT': 'R',\n", + " 'MA': 'B',\n", + " 'NH': 'G',\n", + " 'CT': 'R',\n", + " 'RI': 'G',\n", + " 'ME': 'R'}" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time backtracking_search(usa_csp, select_unassigned_variable=mrv, inference=forward_checking)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SAT" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": {}, + "outputs": [], + "source": [ + "usa_sat = MapColoringSAT(list('RGBY'),\n", + " \"\"\"WA: OR ID; OR: ID NV CA; CA: NV AZ; NV: ID UT AZ; ID: MT WY UT;\n", + " UT: WY CO AZ; MT: ND SD WY; WY: SD NE CO; CO: NE KA OK NM; NM: OK TX AZ;\n", + " ND: MN SD; SD: MN IA NE; NE: IA MO KA; KA: MO OK; OK: MO AR TX;\n", + " TX: AR LA; MN: WI IA; IA: WI IL MO; MO: IL KY TN AR; AR: MS TN LA;\n", + " LA: MS; WI: MI IL; IL: IN KY; IN: OH KY; MS: TN AL; AL: TN GA FL;\n", + " MI: OH IN; OH: PA WV KY; KY: WV VA TN; TN: VA NC GA; GA: NC SC FL;\n", + " PA: NY NJ DE MD WV; WV: MD VA; VA: MD DC NC; NC: SC; NY: VT MA CT NJ;\n", + " NJ: DE; DE: MD; MD: DC; VT: NH MA; MA: NH RI CT; CT: RI; ME: NH;\n", + " HI: ; AK: \"\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 46.2 s, sys: 0 ns, total: 46.2 s\n", + "Wall time: 46.2 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=no_branching_heuristic)" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 54.6 s, sys: 0 ns, total: 54.6 s\n", + "Wall time: 54.6 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=moms)" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 44 s, sys: 0 ns, total: 44 s\n", + "Wall time: 44 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=momsf)" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 43.8 s, sys: 0 ns, total: 43.8 s\n", + "Wall time: 43.8 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=posit)" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 52.6 s, sys: 0 ns, total: 52.6 s\n", + "Wall time: 52.6 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=zm)" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 57 s, sys: 0 ns, total: 57 s\n", + "Wall time: 57 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=dlis)" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 43.8 s, sys: 0 ns, total: 43.8 s\n", + "Wall time: 43.8 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=dlcs)" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 53.3 s, sys: 3.82 ms, total: 53.3 s\n", + "Wall time: 53.3 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=jw)" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 44 s, sys: 3.99 ms, total: 44 s\n", + "Wall time: 44 s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(usa_sat, branching_heuristic=jw2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 559 ms, sys: 0 ns, total: 559 ms\n", + "Wall time: 558 ms\n" + ] + } + ], + "source": [ + "%time model = cdcl_satisfiable(usa_sat)" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{AL_B,\n", + " AR_B,\n", + " AZ_R,\n", + " CA_B,\n", + " CO_R,\n", + " CT_Y,\n", + " DC_G,\n", + " DE_Y,\n", + " FL_Y,\n", + " GA_R,\n", + " IA_B,\n", + " ID_Y,\n", + " IL_G,\n", + " IN_R,\n", + " KA_G,\n", + " KY_B,\n", + " LA_G,\n", + " MA_G,\n", + " MD_R,\n", + " ME_G,\n", + " MI_G,\n", + " MN_Y,\n", + " MO_R,\n", + " MS_Y,\n", + " MT_B,\n", + " NC_B,\n", + " ND_G,\n", + " NE_Y,\n", + " NH_Y,\n", + " NJ_G,\n", + " NM_G,\n", + " NV_G,\n", + " NY_R,\n", + " OH_Y,\n", + " OK_Y,\n", + " OR_R,\n", + " PA_B,\n", + " RI_B,\n", + " SC_Y,\n", + " SD_R,\n", + " TN_G,\n", + " TX_R,\n", + " UT_B,\n", + " VA_Y,\n", + " VT_B,\n", + " WA_B,\n", + " WI_R,\n", + " WV_G,\n", + " WY_G}" + ] + }, + "execution_count": 67, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{var for var, val in model.items() if val}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Zebra Puzzle" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### CSP" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": {}, + "outputs": [], + "source": [ + "zebra_csp = Zebra()" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'Milk': 3, 'Norwegian': 1}\n" + ] + } + ], + "source": [ + "zebra_csp.display(zebra_csp.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 2.04 ms, sys: 4 µs, total: 2.05 ms\n", + "Wall time: 2.05 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "'AC3b with DOM J UP needs 737 consistency-checks'" + ] + }, + "execution_count": 78, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time _, checks = AC3b(zebra_csp, arc_heuristic=dom_j_up)\n", + "f'AC3b with DOM J UP needs {checks} consistency-checks'" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'Blue': 2, 'Milk': 3, 'Norwegian': 1}\n" + ] + } + ], + "source": [ + "zebra_csp.display(zebra_csp.infer_assignment())" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 2.13 ms, sys: 0 ns, total: 2.13 ms\n", + "Wall time: 2.14 ms\n" + ] + }, + { + "data": { + "text/plain": [ + "{'Milk': 3,\n", + " 'Blue': 2,\n", + " 'Norwegian': 1,\n", + " 'Coffee': 5,\n", + " 'Green': 5,\n", + " 'Ivory': 4,\n", + " 'Red': 3,\n", + " 'Yellow': 1,\n", + " 'Kools': 1,\n", + " 'Englishman': 3,\n", + " 'Horse': 2,\n", + " 'Tea': 2,\n", + " 'Ukranian': 2,\n", + " 'Spaniard': 4,\n", + " 'Dog': 4,\n", + " 'Japanese': 5,\n", + " 'Parliaments': 5,\n", + " 'LuckyStrike': 4,\n", + " 'OJ': 4,\n", + " 'Water': 1,\n", + " 'Chesterfields': 2,\n", + " 'Winston': 3,\n", + " 'Snails': 3,\n", + " 'Fox': 1,\n", + " 'Zebra': 5}" + ] + }, + "execution_count": 72, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "%time backtracking_search(zebra_csp, select_unassigned_variable=mrv, inference=forward_checking)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### SAT" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "zebra_sat = associate('&', map(to_cnf, map(expr, filter(lambda line: line[0] not in ('c', 'p'), open('aima-data/zebra.cnf').read().splitlines()))))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### DPLL" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 13min 6s, sys: 2.44 ms, total: 13min 6s\n", + "Wall time: 13min 6s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=no_branching_heuristic)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 15min 4s, sys: 22.4 ms, total: 15min 4s\n", + "Wall time: 15min 4s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=moms)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 22min 28s, sys: 40 ms, total: 22min 28s\n", + "Wall time: 22min 28s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=momsf)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 22min 25s, sys: 36 ms, total: 22min 25s\n", + "Wall time: 22min 25s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=posit)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 14min 52s, sys: 32 ms, total: 14min 52s\n", + "Wall time: 14min 52s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=zm)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 2min 31s, sys: 9.87 ms, total: 2min 31s\n", + "Wall time: 2min 32s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=dlis)" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 4min 27s, sys: 12 ms, total: 4min 27s\n", + "Wall time: 4min 27s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=dlcs)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 6min 55s, sys: 39.2 ms, total: 6min 55s\n", + "Wall time: 6min 56s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=jw)" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 8min 57s, sys: 7.94 ms, total: 8min 57s\n", + "Wall time: 8min 57s\n" + ] + } + ], + "source": [ + "%time model = dpll_satisfiable(zebra_sat, branching_heuristic=jw2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### CDCL" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": {} + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 1.64 s, sys: 0 ns, total: 1.64 s\n", + "Wall time: 1.64 s\n" + ] + } + ], + "source": [ + "%time model = cdcl_satisfiable(zebra_sat)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{Englishman_house2,\n", + " Englishman_milk,\n", + " Englishman_oldGold,\n", + " Englishman_redHouse,\n", + " Englishman_snails,\n", + " Japanese_coffee,\n", + " Japanese_greenHouse,\n", + " Japanese_house4,\n", + " Japanese_parliament,\n", + " Japanese_zebra,\n", + " Norwegian_fox,\n", + " Norwegian_house0,\n", + " Norwegian_kool,\n", + " Norwegian_water,\n", + " Norwegian_yellowHouse,\n", + " Spaniard_dog,\n", + " Spaniard_house3,\n", + " Spaniard_ivoryHouse,\n", + " Spaniard_luckyStrike,\n", + " Spaniard_orangeJuice,\n", + " Ukrainian_blueHouse,\n", + " Ukrainian_chesterfield,\n", + " Ukrainian_horse,\n", + " Ukrainian_house1,\n", + " Ukrainian_tea}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{var for var, val in model.items() if val and var.op.startswith(('Englishman', 'Japanese', 'Norwegian', 'Spaniard', 'Ukrainian'))}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## References\n", + "\n", + "[[1]](#ref-1) Freeman, Jon William. 1995. _Improvements to propositional satisfiability search algorithms_.\n", + "\n", + "[[2]](#ref-2) Zabih, Ramin and McAllester, David A. 1988. _A Rearrangement Search Strategy for Determining Propositional Satisfiability_.\n", + "\n", + "[[3]](#ref-3) Jeroslow, Robert G and Wang, Jinchang. 1990. _Solving propositional satisfiability problems_.\n", + "\n", + "[[4]](#ref-4) Moskewicz, Matthew W and Madigan, Conor F and Zhao, Ying and Zhang, Lintao and Malik, Sharad. 2001. _Chaff: Engineering an efficient SAT solver_.\n", + "\n", + "[[5]](#ref-5) Haim, Shai and Heule, Marijn. 2014. _Towards ultra rapid restarts_.\n", + "\n", + "[[6]](#ref-6) Huang, Jinbo and others. 2007. _The Effect of Restarts on the Efficiency of Clause Learning_.\n", + "\n", + "[[7]](#ref-7) Audemard, Gilles and Simon, Laurent. 2012. _Refining restarts strategies for SAT and UNSAT_.\n", + "\n", + "[[8]](#ref-8) Audemard, Gilles and Simon, Laurent. 2009. _Predicting learnt clauses quality in modern SAT solvers_." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/index.ipynb b/index.ipynb index 2ae5742bb..f9da121f2 100644 --- a/index.ipynb +++ b/index.ipynb @@ -18,7 +18,7 @@ "\n", "3. [**Search**](./search.ipynb)\n", "\n", - "4. [**Search - 4th edition**](./search-4e.ipynb)\n", + "4. [**Search - 4th edition**](./search4e.ipynb)\n", "\n", "4. [**Games**](./games.ipynb)\n", "\n", diff --git a/ipyviews.py b/ipyviews.py index fbdc9a580..b304af7bb 100644 --- a/ipyviews.py +++ b/ipyviews.py @@ -6,7 +6,6 @@ import copy import __main__ - # ______________________________________________________________________________ # Continuous environment diff --git a/knowledge.py b/knowledge.py index cf4915b47..8c27c3eb8 100644 --- a/knowledge.py +++ b/knowledge.py @@ -1,25 +1,28 @@ -"""Knowledge in learning, Chapter 19""" +"""Knowledge in learning (Chapter 19)""" -from random import shuffle -from math import log -from utils import powerset from collections import defaultdict +from functools import partial from itertools import combinations, product +from random import shuffle + +import numpy as np + from logic import (FolKB, constant_symbols, predicate_symbols, standardize_variables, variables, is_definite_clause, subst, expr, Expr) -from functools import partial - -# ______________________________________________________________________________ +from utils import power_set def current_best_learning(examples, h, examples_so_far=None): - """ [Figure 19.2] + """ + [Figure 19.2] The hypothesis is a list of dictionaries, with each dictionary representing - a disjunction.""" + a disjunction. + """ + if examples_so_far is None: + examples_so_far = [] if not examples: return h - examples_so_far = examples_so_far or [] e = examples[0] if is_consistent(e, h): return current_best_learning(examples[1:], h, examples_so_far + [e]) @@ -64,7 +67,7 @@ def generalizations(examples_so_far, h): hypotheses = [] # Delete disjunctions - disj_powerset = powerset(range(len(h))) + disj_powerset = power_set(range(len(h))) for disjs in disj_powerset: h2 = h.copy() for d in reversed(list(disjs)): @@ -75,7 +78,7 @@ def generalizations(examples_so_far, h): # Delete AND operations in disjunctions for i, disj in enumerate(h): - a_powerset = powerset(disj.keys()) + a_powerset = power_set(disj.keys()) for attrs in a_powerset: h2 = h[i].copy() for a in attrs: @@ -103,7 +106,7 @@ def add_or(examples_so_far, h): e = examples_so_far[-1] attrs = {k: v for k, v in e.items() if k != 'GOAL'} - a_powerset = powerset(attrs.keys()) + a_powerset = power_set(attrs.keys()) for c in a_powerset: h2 = {} @@ -117,13 +120,16 @@ def add_or(examples_so_far, h): return ors + # ______________________________________________________________________________ def version_space_learning(examples): - """ [Figure 19.3] + """ + [Figure 19.3] The version space is a list of hypotheses, which in turn are a list - of dictionaries/disjunctions.""" + of dictionaries/disjunctions. + """ V = all_hypotheses(examples) for e in examples: if V: @@ -139,7 +145,7 @@ def version_space_update(V, e): def all_hypotheses(examples): """Build a list of all the possible hypotheses""" values = values_table(examples) - h_powerset = powerset(values.keys()) + h_powerset = power_set(values.keys()) hypotheses = [] for s in h_powerset: hypotheses.extend(build_attr_combinations(s, values)) @@ -182,7 +188,7 @@ def build_attr_combinations(s, values): h = [] for i, a in enumerate(s): - rest = build_attr_combinations(s[i+1:], values) + rest = build_attr_combinations(s[i + 1:], values) for v in values[a]: o = {a: v} for r in rest: @@ -198,7 +204,7 @@ def build_h_combinations(hypotheses): """Given a set of hypotheses, builds and returns all the combinations of the hypotheses.""" h = [] - h_powerset = powerset(range(len(hypotheses))) + h_powerset = power_set(range(len(hypotheses))) for s in h_powerset: t = [] @@ -208,6 +214,7 @@ def build_h_combinations(hypotheses): return h + # ______________________________________________________________________________ @@ -233,16 +240,17 @@ def consistent_det(A, E): return True + # ______________________________________________________________________________ -class FOIL_container(FolKB): +class FOILContainer(FolKB): """Hold the kb and other necessary elements required by FOIL.""" def __init__(self, clauses=None): self.const_syms = set() self.pred_syms = set() - FolKB.__init__(self, clauses) + super().__init__(clauses) def tell(self, sentence): if is_definite_clause(sentence): @@ -250,7 +258,7 @@ def tell(self, sentence): self.const_syms.update(constant_symbols(sentence)) self.pred_syms.update(predicate_symbols(sentence)) else: - raise Exception("Not a definite clause: {}".format(sentence)) + raise Exception('Not a definite clause: {}'.format(sentence)) def foil(self, examples, target): """Learn a list of first-order horn clauses @@ -275,7 +283,6 @@ def new_clause(self, examples, target): The horn clause is specified as [consequent, list of antecedents] Return value is the tuple (horn_clause, extended_positive_examples).""" clause = [target, []] - # [positive_examples, negative_examples] extended_examples = examples while extended_examples[1]: l = self.choose_literal(self.new_literals(clause), extended_examples) @@ -283,7 +290,7 @@ def new_clause(self, examples, target): extended_examples = [sum([list(self.extend_example(example, l)) for example in extended_examples[i]], []) for i in range(2)] - return (clause, extended_examples[0]) + return clause, extended_examples[0] def extend_example(self, example, literal): """Generate extended examples which satisfy the literal.""" @@ -294,7 +301,7 @@ def extend_example(self, example, literal): def new_literals(self, clause): """Generate new literals based on known predicate symbols. - Generated literal must share atleast one variable with clause""" + Generated literal must share at least one variable with clause""" share_vars = variables(clause[0]) for l in clause[1]: share_vars.update(variables(l)) @@ -306,14 +313,11 @@ def new_literals(self, clause): if not Expr(pred, args) in clause[1]: yield Expr(pred, *[var for var in args]) - - def choose_literal(self, literals, examples): + def choose_literal(self, literals, examples): """Choose the best literal based on the information gain.""" + return max(literals, key=partial(self.gain, examples=examples)) - return max(literals, key = partial(self.gain , examples = examples)) - - - def gain(self, l ,examples): + def gain(self, l, examples): """ Find the utility of each literal when added to the body of the clause. Utility function is: @@ -331,9 +335,9 @@ def gain(self, l ,examples): """ pre_pos = len(examples[0]) pre_neg = len(examples[1]) - post_pos = sum([list(self.extend_example(example, l)) for example in examples[0]], []) - post_neg = sum([list(self.extend_example(example, l)) for example in examples[1]], []) - if pre_pos + pre_neg ==0 or len(post_pos) + len(post_neg)==0: + post_pos = sum([list(self.extend_example(example, l)) for example in examples[0]], []) + post_neg = sum([list(self.extend_example(example, l)) for example in examples[1]], []) + if pre_pos + pre_neg == 0 or len(post_pos) + len(post_neg) == 0: return -1 # number of positive example that are represented in extended_examples T = 0 @@ -341,10 +345,10 @@ def gain(self, l ,examples): represents = lambda d: all(d[x] == example[x] for x in example) if any(represents(l_) for l_ in post_pos): T += 1 - value = T * (log(len(post_pos) / (len(post_pos) + len(post_neg)) + 1e-12,2) - log(pre_pos / (pre_pos + pre_neg),2)) + value = T * (np.log2(len(post_pos) / (len(post_pos) + len(post_neg)) + 1e-12) - + np.log2(pre_pos / (pre_pos + pre_neg))) return value - def update_examples(self, target, examples, extended_examples): """Add to the kb those examples what are represented in extended_examples List of omitted examples is returned.""" @@ -407,17 +411,12 @@ def guess_value(e, h): def is_consistent(e, h): - return e["GOAL"] == guess_value(e, h) + return e['GOAL'] == guess_value(e, h) def false_positive(e, h): - return guess_value(e, h) and not e["GOAL"] + return guess_value(e, h) and not e['GOAL'] def false_negative(e, h): - return e["GOAL"] and not guess_value(e, h) - - - - - + return e['GOAL'] and not guess_value(e, h) diff --git a/knowledge_FOIL.ipynb b/knowledge_FOIL.ipynb index 63e943416..4cefd7f69 100644 --- a/knowledge_FOIL.ipynb +++ b/knowledge_FOIL.ipynb @@ -18,8 +18,7 @@ "outputs": [], "source": [ "from knowledge import *\n", - "\n", - "from notebook import pseudocode, psource" + "from notebook import psource" ] }, { @@ -624,8 +623,17 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 2 -} +} \ No newline at end of file diff --git a/knowledge_current_best.ipynb b/knowledge_current_best.ipynb index 757062587..5da492cd0 100644 --- a/knowledge_current_best.ipynb +++ b/knowledge_current_best.ipynb @@ -654,7 +654,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.5" + "version": "3.6.7" } }, "nbformat": 4, diff --git a/learning.ipynb b/learning.ipynb index aecd5d2d3..0cadd4e7b 100644 --- a/learning.ipynb +++ b/learning.ipynb @@ -16,6 +16,7 @@ "outputs": [], "source": [ "from learning import *\n", + "from probabilistic_learning import *\n", "from notebook import *" ] }, @@ -2247,8 +2248,17 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 2 -} +} \ No newline at end of file diff --git a/learning.py b/learning.py index 9c58a5d5a..71b6b15e7 100644 --- a/learning.py +++ b/learning.py @@ -1,90 +1,48 @@ -"""Learn to estimate functions from examples. (Chapters 18, 20)""" - -from utils import ( - removeall, unique, product, mode, argmax, argmax_random_tie, isclose, gaussian, - dotproduct, vector_add, scalar_vector_product, weighted_sample_with_replacement, - weighted_sampler, num_or_str, normalize, clip, sigmoid, print_table, - open_data, sigmoid_derivative, probability, norm, matrix_multiplication, relu, relu_derivative, - tanh, tanh_derivative, leaky_relu, leaky_relu_derivative, elu, elu_derivative -) +"""Learning from examples (Chapters 18)""" import copy -import heapq -import math -import random - -from statistics import mean, stdev from collections import defaultdict +from statistics import stdev -# ______________________________________________________________________________ - - -def euclidean_distance(X, Y): - return math.sqrt(sum((x - y)**2 for x, y in zip(X, Y))) - - -def cross_entropy_loss(X,Y): - n=len(X) - return (-1.0/n)*sum(x*math.log(y) + (1-x)*math.log(1-y) for x, y in zip(X, Y)) - - -def rms_error(X, Y): - return math.sqrt(ms_error(X, Y)) - - -def ms_error(X, Y): - return mean((x - y)**2 for x, y in zip(X, Y)) - - -def mean_error(X, Y): - return mean(abs(x - y) for x, y in zip(X, Y)) - +from qpsolvers import solve_qp -def manhattan_distance(X, Y): - return sum(abs(x - y) for x, y in zip(X, Y)) - - -def mean_boolean_error(X, Y): - return mean(int(x != y) for x, y in zip(X, Y)) - - -def hamming_distance(X, Y): - return sum(x != y for x, y in zip(X, Y)) - -# ______________________________________________________________________________ +from probabilistic_learning import NaiveBayesLearner +from utils import * class DataSet: - """A data set for a machine learning problem. It has the following fields: + """ + A data set for a machine learning problem. It has the following fields: d.examples A list of examples. Each one is a list of attribute values. d.attrs A list of integers to index into an example, so example[attr] gives a value. Normally the same as range(len(d.examples[0])). - d.attrnames Optional list of mnemonic names for corresponding attrs. + d.attr_names Optional list of mnemonic names for corresponding attrs. d.target The attribute that a learning algorithm will try to predict. By default the final attribute. d.inputs The list of attrs without the target. d.values A list of lists: each sublist is the set of possible values for the corresponding attribute. If initially None, - it is computed from the known examples by self.setproblem. + it is computed from the known examples by self.set_problem. If not None, an erroneous value raises ValueError. - d.distance A function from a pair of examples to a nonnegative number. + d.distance A function from a pair of examples to a non-negative number. Should be symmetric, etc. Defaults to mean_boolean_error since that can handle any field types. d.name Name of the data set (for output display only). d.source URL or other source where the data came from. d.exclude A list of attribute indexes to exclude from d.inputs. Elements - of this list can either be integers (attrs) or attrnames. + of this list can either be integers (attrs) or attr_names. Normally, you call the constructor and you're done; then you just - access fields like d.examples and d.target and d.inputs.""" + access fields like d.examples and d.target and d.inputs. + """ - def __init__(self, examples=None, attrs=None, attrnames=None, target=-1, - inputs=None, values=None, distance=mean_boolean_error, - name='', source='', exclude=()): - """Accepts any of DataSet's fields. Examples can also be a + def __init__(self, examples=None, attrs=None, attr_names=None, target=-1, inputs=None, + values=None, distance=mean_boolean_error, name='', source='', exclude=()): + """ + Accepts any of DataSet's fields. Examples can also be a string or file from which to parse examples using parse_csv. - Optional parameter: exclude, as documented in .setproblem(). + Optional parameter: exclude, as documented in .set_problem(). >>> DataSet(examples='1, 2, 3') """ @@ -94,7 +52,7 @@ def __init__(self, examples=None, attrs=None, attrnames=None, target=-1, self.distance = distance self.got_values_flag = bool(values) - # Initialize .examples from string or list or data directory + # initialize .examples from string or list or data directory if isinstance(examples, str): self.examples = parse_csv(examples) elif examples is None: @@ -102,39 +60,40 @@ def __init__(self, examples=None, attrs=None, attrnames=None, target=-1, else: self.examples = examples - # Attrs are the indices of examples, unless otherwise stated. + # attrs are the indices of examples, unless otherwise stated. if self.examples is not None and attrs is None: attrs = list(range(len(self.examples[0]))) self.attrs = attrs - # Initialize .attrnames from string, list, or by default - if isinstance(attrnames, str): - self.attrnames = attrnames.split() + # initialize .attr_names from string, list, or by default + if isinstance(attr_names, str): + self.attr_names = attr_names.split() else: - self.attrnames = attrnames or attrs - self.setproblem(target, inputs=inputs, exclude=exclude) + self.attr_names = attr_names or attrs + self.set_problem(target, inputs=inputs, exclude=exclude) - def setproblem(self, target, inputs=None, exclude=()): - """Set (or change) the target and/or inputs. + def set_problem(self, target, inputs=None, exclude=()): + """ + Set (or change) the target and/or inputs. This way, one DataSet can be used multiple ways. inputs, if specified, is a list of attributes, or specify exclude as a list of attributes - to not use in inputs. Attributes can be -n .. n, or an attrname. - Also computes the list of possible values, if that wasn't done yet.""" - self.target = self.attrnum(target) - exclude = list(map(self.attrnum, exclude)) + to not use in inputs. Attributes can be -n .. n, or an attr_name. + Also computes the list of possible values, if that wasn't done yet. + """ + self.target = self.attr_num(target) + exclude = list(map(self.attr_num, exclude)) if inputs: - self.inputs = removeall(self.target, inputs) + self.inputs = remove_all(self.target, inputs) else: - self.inputs = [a for a in self.attrs - if a != self.target and a not in exclude] + self.inputs = [a for a in self.attrs if a != self.target and a not in exclude] if not self.values: self.update_values() self.check_me() def check_me(self): """Check that my fields make sense.""" - assert len(self.attrnames) == len(self.attrs) + assert len(self.attr_names) == len(self.attrs) assert self.target in self.attrs assert self.target not in self.inputs assert set(self.inputs).issubset(set(self.attrs)) @@ -153,12 +112,12 @@ def check_example(self, example): for a in self.attrs: if example[a] not in self.values[a]: raise ValueError('Bad value {} for attribute {} in {}' - .format(example[a], self.attrnames[a], example)) + .format(example[a], self.attr_names[a], example)) - def attrnum(self, attr): + def attr_num(self, attr): """Returns the number used for attr, which can be a name, or -n .. n-1.""" if isinstance(attr, str): - return self.attrnames.index(attr) + return self.attr_names.index(attr) elif attr < 0: return len(self.attrs) + attr else: @@ -169,18 +128,17 @@ def update_values(self): def sanitize(self, example): """Return a copy of example, with non-input attributes replaced by None.""" - return [attr_i if i in self.inputs else None - for i, attr_i in enumerate(example)] + return [attr_i if i in self.inputs else None for i, attr_i in enumerate(example)] def classes_to_numbers(self, classes=None): """Converts class names to numbers.""" if not classes: - # If classes were not given, extract them from values + # if classes were not given, extract them from values classes = sorted(self.values[self.target]) for item in self.examples: item[self.target] = classes.index(item[self.target]) - def remove_examples(self, value=""): + def remove_examples(self, value=''): """Remove examples that contain given value.""" self.examples = [x for x in self.examples if value not in x] self.update_values() @@ -191,17 +149,19 @@ def split_values_by_classes(self): target_names = self.values[self.target] for v in self.examples: - item = [a for a in v if a not in target_names] # Remove target from item - buckets[v[self.target]].append(item) # Add item to bucket of its class + item = [a for a in v if a not in target_names] # remove target from item + buckets[v[self.target]].append(item) # add item to bucket of its class return buckets def find_means_and_deviations(self): - """Finds the means and standard deviations of self.dataset. - means : A dictionary for each class/target. Holds a list of the means + """ + Finds the means and standard deviations of self.dataset. + means : a dictionary for each class/target. Holds a list of the means of the features for the class. - deviations: A dictionary for each class/target. Holds a list of the sample - standard deviations of the features for the class.""" + deviations: a dictionary for each class/target. Holds a list of the sample + standard deviations of the features for the class. + """ target_names = self.values[self.target] feature_numbers = len(self.inputs) @@ -211,13 +171,13 @@ def find_means_and_deviations(self): deviations = defaultdict(lambda: [0] * feature_numbers) for t in target_names: - # Find all the item feature values for item in class t - features = [[] for i in range(feature_numbers)] + # find all the item feature values for item in class t + features = [[] for _ in range(feature_numbers)] for item in item_buckets[t]: for i in range(feature_numbers): features[i].append(item[i]) - # Calculate means and deviations fo the class + # calculate means and deviations fo the class for i in range(feature_numbers): means[t][i] = mean(features[i]) deviations[t][i] = stdev(features[i]) @@ -225,284 +185,194 @@ def find_means_and_deviations(self): return means, deviations def __repr__(self): - return ''.format( - self.name, len(self.examples), len(self.attrs)) - -# ______________________________________________________________________________ + return ''.format(self.name, len(self.examples), len(self.attrs)) def parse_csv(input, delim=','): - r"""Input is a string consisting of lines, each line has comma-delimited - fields. Convert this into a list of lists. Blank lines are skipped. + r""" + Input is a string consisting of lines, each line has comma-delimited + fields. Convert this into a list of lists. Blank lines are skipped. Fields that look like numbers are converted to numbers. The delim defaults to ',' but '\t' and None are also reasonable values. >>> parse_csv('1, 2, 3 \n 0, 2, na') - [[1, 2, 3], [0, 2, 'na']]""" + [[1, 2, 3], [0, 2, 'na']] + """ lines = [line for line in input.splitlines() if line.strip()] return [list(map(num_or_str, line.split(delim))) for line in lines] -# ______________________________________________________________________________ - - -class CountingProbDist: - """A probability distribution formed by observing and counting examples. - If p is an instance of this class and o is an observed value, then - there are 3 main operations: - p.add(o) increments the count for observation o by 1. - p.sample() returns a random element from the distribution. - p[o] returns the probability for o (as in a regular ProbDist).""" - - def __init__(self, observations=None, default=0): - """Create a distribution, and optionally add in some observations. - By default this is an unsmoothed distribution, but saying default=1, - for example, gives you add-one smoothing.""" - if observations is None: - observations = [] - self.dictionary = {} - self.n_obs = 0 - self.default = default - self.sampler = None - - for o in observations: - self.add(o) - - def add(self, o): - """Add an observation o to the distribution.""" - self.smooth_for(o) - self.dictionary[o] += 1 - self.n_obs += 1 - self.sampler = None - - def smooth_for(self, o): - """Include o among the possible observations, whether or not - it's been observed yet.""" - if o not in self.dictionary: - self.dictionary[o] = self.default - self.n_obs += self.default - self.sampler = None - - def __getitem__(self, item): - """Return an estimate of the probability of item.""" - self.smooth_for(item) - return self.dictionary[item] / self.n_obs - - # (top() and sample() are not used in this module, but elsewhere.) - - def top(self, n): - """Return (count, obs) tuples for the n most frequent observations.""" - return heapq.nlargest(n, [(v, k) for (k, v) in self.dictionary.items()]) - - def sample(self): - """Return a random sample from the distribution.""" - if self.sampler is None: - self.sampler = weighted_sampler(list(self.dictionary.keys()), - list(self.dictionary.values())) - return self.sampler() - -# ______________________________________________________________________________ - -def PluralityLearner(dataset): - """A very dumb algorithm: always pick the result that was most popular - in the training data. Makes a baseline for comparison.""" - most_popular = mode([e[dataset.target] for e in dataset.examples]) +def err_ratio(predict, dataset, examples=None): + """ + Return the proportion of the examples that are NOT correctly predicted. + verbose - 0: No output; 1: Output wrong; 2 (or greater): Output correct + """ + examples = examples or dataset.examples + if len(examples) == 0: + return 0.0 + right = 0 + for example in examples: + desired = example[dataset.target] + output = predict(dataset.sanitize(example)) + if output == desired: + right += 1 + return 1 - (right / len(examples)) - def predict(example): - """Always return same result: the most popular from the training set.""" - return most_popular - return predict -# ______________________________________________________________________________ +def grade_learner(predict, tests): + """ + Grades the given learner based on how many tests it passes. + tests is a list with each element in the form: (values, output). + """ + return mean(int(predict(X) == y) for X, y in tests) -def NaiveBayesLearner(dataset, continuous=True, simple=False): - if simple: - return NaiveBayesSimple(dataset) - if continuous: - return NaiveBayesContinuous(dataset) +def train_test_split(dataset, start=None, end=None, test_split=None): + """ + If you are giving 'start' and 'end' as parameters, + then it will return the testing set from index 'start' to 'end' + and the rest for training. + If you give 'test_split' as a parameter then it will return + test_split * 100% as the testing set and the rest as + training set. + """ + examples = dataset.examples + if test_split is None: + train = examples[:start] + examples[end:] + val = examples[start:end] else: - return NaiveBayesDiscrete(dataset) + total_size = len(examples) + val_size = int(total_size * test_split) + train_size = total_size - val_size + train = examples[:train_size] + val = examples[train_size:total_size] + return train, val -def NaiveBayesSimple(distribution): - """A simple naive bayes classifier that takes as input a dictionary of - CountingProbDist objects and classifies items according to these distributions. - The input dictionary is in the following form: - (ClassName, ClassProb): CountingProbDist""" - target_dist = {c_name: prob for c_name, prob in distribution.keys()} - attr_dists = {c_name: count_prob for (c_name, _), count_prob in distribution.items()} - def predict(example): - """Predict the target value for example. Calculate probabilities for each - class and pick the max.""" - def class_probability(targetval): - attr_dist = attr_dists[targetval] - return target_dist[targetval] * product(attr_dist[a] for a in example) +def cross_validation_wrapper(learner, dataset, k=10, trials=1): + """ + [Figure 18.8] + Return the optimal value of size having minimum error on validation set. + errT: a training error array, indexed by size + errV: a validation error array, indexed by size + """ + errs = [] + size = 1 + while True: + errT, errV = cross_validation(learner, dataset, size, k, trials) + # check for convergence provided err_val is not empty + if errT and not np.isclose(errT[-1], errT, rtol=1e-6): + best_size = 0 + min_val = np.inf + i = 0 + while i < size: + if errs[i] < min_val: + min_val = errs[i] + best_size = i + i += 1 + return learner(dataset, best_size) + errs.append(errV) + size += 1 - return argmax(target_dist.keys(), key=class_probability) - return predict +def cross_validation(learner, dataset, size=None, k=10, trials=1): + """ + Do k-fold cross_validate and return their mean. + That is, keep out 1/k of the examples for testing on each of k runs. + Shuffle the examples first; if trials > 1, average over several shuffles. + Returns Training error, Validation error + """ + k = k or len(dataset.examples) + if trials > 1: + trial_errT = 0 + trial_errV = 0 + for t in range(trials): + errT, errV = cross_validation(learner, dataset, size, k, trials) + trial_errT += errT + trial_errV += errV + return trial_errT / trials, trial_errV / trials + else: + fold_errT = 0 + fold_errV = 0 + n = len(dataset.examples) + examples = dataset.examples + random.shuffle(dataset.examples) + for fold in range(k): + train_data, val_data = train_test_split(dataset, fold * (n // k), (fold + 1) * (n // k)) + dataset.examples = train_data + h = learner(dataset, size) + fold_errT += err_ratio(h, dataset, train_data) + fold_errV += err_ratio(h, dataset, val_data) + # reverting back to original once test is completed + dataset.examples = examples + return fold_errT / k, fold_errV / k -def NaiveBayesDiscrete(dataset): - """Just count how many times each value of each input attribute - occurs, conditional on the target value. Count the different - target values too.""" +def leave_one_out(learner, dataset, size=None): + """Leave one out cross-validation over the dataset.""" + return cross_validation(learner, dataset, size, len(dataset.examples)) - target_vals = dataset.values[dataset.target] - target_dist = CountingProbDist(target_vals) - attr_dists = {(gv, attr): CountingProbDist(dataset.values[attr]) - for gv in target_vals - for attr in dataset.inputs} - for example in dataset.examples: - targetval = example[dataset.target] - target_dist.add(targetval) - for attr in dataset.inputs: - attr_dists[targetval, attr].add(example[attr]) - def predict(example): - """Predict the target value for example. Consider each possible value, - and pick the most likely by looking at each attribute independently.""" - def class_probability(targetval): - return (target_dist[targetval] * - product(attr_dists[targetval, attr][example[attr]] - for attr in dataset.inputs)) - return argmax(target_vals, key=class_probability) +def learning_curve(learner, dataset, trials=10, sizes=None): + if sizes is None: + sizes = list(range(2, len(dataset.examples) - trials, 2)) - return predict + def score(learner, size): + random.shuffle(dataset.examples) + return cross_validation(learner, dataset, size, trials) + return [(size, mean([score(learner, size) for _ in range(trials)])) for size in sizes] -def NaiveBayesContinuous(dataset): - """Count how many times each target value occurs. - Also, find the means and deviations of input attribute values for each target value.""" - means, deviations = dataset.find_means_and_deviations() - target_vals = dataset.values[dataset.target] - target_dist = CountingProbDist(target_vals) +def PluralityLearner(dataset): + """ + A very dumb algorithm: always pick the result that was most popular + in the training data. Makes a baseline for comparison. + """ + most_popular = mode([e[dataset.target] for e in dataset.examples]) def predict(example): - """Predict the target value for example. Consider each possible value, - and pick the most likely by looking at each attribute independently.""" - def class_probability(targetval): - prob = target_dist[targetval] - for attr in dataset.inputs: - prob *= gaussian(means[targetval][attr], deviations[targetval][attr], example[attr]) - return prob - - return argmax(target_vals, key=class_probability) - - return predict - -# ______________________________________________________________________________ - + """Always return same result: the most popular from the training set.""" + return most_popular -def NearestNeighborLearner(dataset, k=1): - """k-NearestNeighbor: the k nearest neighbors vote.""" - def predict(example): - """Find the k closest items, and have them vote for the best.""" - best = heapq.nsmallest(k, ((dataset.distance(e, example), e) - for e in dataset.examples)) - return mode(e[dataset.target] for (d, e) in best) return predict -# ______________________________________________________________________________ - - -def truncated_svd(X, num_val=2, max_iter=1000): - """Compute the first component of SVD.""" - - def normalize_vec(X, n=2): - """Normalize two parts (:m and m:) of the vector.""" - X_m = X[:m] - X_n = X[m:] - norm_X_m = norm(X_m, n) - Y_m = [x/norm_X_m for x in X_m] - norm_X_n = norm(X_n, n) - Y_n = [x/norm_X_n for x in X_n] - return Y_m + Y_n - - def remove_component(X): - """Remove components of already obtained eigen vectors from X.""" - X_m = X[:m] - X_n = X[m:] - for eivec in eivec_m: - coeff = dotproduct(X_m, eivec) - X_m = [x1 - coeff*x2 for x1, x2 in zip(X_m, eivec)] - for eivec in eivec_n: - coeff = dotproduct(X_n, eivec) - X_n = [x1 - coeff*x2 for x1, x2 in zip(X_n, eivec)] - return X_m + X_n - - m, n = len(X), len(X[0]) - A = [[0]*(n+m) for _ in range(n+m)] - for i in range(m): - for j in range(n): - A[i][m+j] = A[m+j][i] = X[i][j] - - eivec_m = [] - eivec_n = [] - eivals = [] - - for _ in range(num_val): - X = [random.random() for _ in range(m+n)] - X = remove_component(X) - X = normalize_vec(X) - - for i in range(max_iter): - old_X = X - X = matrix_multiplication(A, [[x] for x in X]) - X = [x[0] for x in X] - X = remove_component(X) - X = normalize_vec(X) - # check for convergence - if norm([x1 - x2 for x1, x2 in zip(old_X, X)]) <= 1e-10: - break - - projected_X = matrix_multiplication(A, [[x] for x in X]) - projected_X = [x[0] for x in projected_X] - eivals.append(norm(projected_X, 1)/norm(X, 1)) - eivec_m.append(X[:m]) - eivec_n.append(X[m:]) - return (eivec_m, eivec_n, eivals) - -# ______________________________________________________________________________ - class DecisionFork: - """A fork of a decision tree holds an attribute to test, and a dict - of branches, one for each of the attribute's values.""" + """ + A fork of a decision tree holds an attribute to test, and a dict + of branches, one for each of the attribute's values. + """ - def __init__(self, attr, attrname=None, default_child=None, branches=None): + def __init__(self, attr, attr_name=None, default_child=None, branches=None): """Initialize by saying what attribute this node tests.""" self.attr = attr - self.attrname = attrname or attr + self.attr_name = attr_name or attr self.default_child = default_child self.branches = branches or {} def __call__(self, example): """Given an example, classify it using the attribute and the branches.""" - attrvalue = example[self.attr] - if attrvalue in self.branches: - return self.branches[attrvalue](example) + attr_val = example[self.attr] + if attr_val in self.branches: + return self.branches[attr_val](example) else: # return default class when attribute is unknown return self.default_child(example) def add(self, val, subtree): - """Add a branch. If self.attr = val, go to the given subtree.""" + """Add a branch. If self.attr = val, go to the given subtree.""" self.branches[val] = subtree def display(self, indent=0): - name = self.attrname + name = self.attr_name print('Test', name) for (val, subtree) in self.branches.items(): print(' ' * 4 * indent, name, '=', val, '==>', end=' ') subtree.display(indent + 1) - print() # newline def __repr__(self): - return ('DecisionFork({0!r}, {1!r}, {2!r})' - .format(self.attr, self.attrname, self.branches)) + return 'DecisionFork({0!r}, {1!r}, {2!r})'.format(self.attr, self.attr_name, self.branches) class DecisionLeaf: @@ -514,14 +384,12 @@ def __init__(self, result): def __call__(self, example): return self.result - def display(self, indent=0): + def display(self): print('RESULT =', self.result) def __repr__(self): return repr(self.result) -# ______________________________________________________________________________ - def DecisionTreeLearner(dataset): """[Figure 18.5]""" @@ -531,24 +399,23 @@ def DecisionTreeLearner(dataset): def decision_tree_learning(examples, attrs, parent_examples=()): if len(examples) == 0: return plurality_value(parent_examples) - elif all_same_class(examples): + if all_same_class(examples): return DecisionLeaf(examples[0][target]) - elif len(attrs) == 0: + if len(attrs) == 0: return plurality_value(examples) - else: - A = choose_attribute(attrs, examples) - tree = DecisionFork(A, dataset.attrnames[A], plurality_value(examples)) - for (v_k, exs) in split_by(A, examples): - subtree = decision_tree_learning( - exs, removeall(A, attrs), examples) - tree.add(v_k, subtree) - return tree + A = choose_attribute(attrs, examples) + tree = DecisionFork(A, dataset.attr_names[A], plurality_value(examples)) + for (v_k, exs) in split_by(A, examples): + subtree = decision_tree_learning(exs, remove_all(A, attrs), examples) + tree.add(v_k, subtree) + return tree def plurality_value(examples): - """Return the most popular target value for this set of examples. - (If target is binary, this is the majority; otherwise plurality.)""" - popular = argmax_random_tie(values[target], - key=lambda v: count(target, v, examples)) + """ + Return the most popular target value for this set of examples. + (If target is binary, this is the majority; otherwise plurality). + """ + popular = argmax_random_tie(values[target], key=lambda v: count(target, v, examples)) return DecisionLeaf(popular) def count(attr, val, examples): @@ -562,67 +429,36 @@ def all_same_class(examples): def choose_attribute(attrs, examples): """Choose the attribute with the highest information gain.""" - return argmax_random_tie(attrs, - key=lambda a: information_gain(a, examples)) + return argmax_random_tie(attrs, key=lambda a: information_gain(a, examples)) def information_gain(attr, examples): """Return the expected reduction in entropy from splitting by attr.""" + def I(examples): - return information_content([count(target, v, examples) - for v in values[target]]) - N = len(examples) - remainder = sum((len(examples_i)/N) * I(examples_i) - for (v, examples_i) in split_by(attr, examples)) + return information_content([count(target, v, examples) for v in values[target]]) + + n = len(examples) + remainder = sum((len(examples_i) / n) * I(examples_i) for (v, examples_i) in split_by(attr, examples)) return I(examples) - remainder def split_by(attr, examples): """Return a list of (val, examples) pairs for each val of attr.""" - return [(v, [e for e in examples if e[attr] == v]) - for v in values[attr]] + return [(v, [e for e in examples if e[attr] == v]) for v in values[attr]] return decision_tree_learning(dataset.examples, dataset.inputs) def information_content(values): """Number of bits to represent the probability distribution in values.""" - probabilities = normalize(removeall(0, values)) - return sum(-p * math.log2(p) for p in probabilities) - -# ______________________________________________________________________________ - - -def RandomForest(dataset, n=5): - """An ensemble of Decision Trees trained using bagging and feature bagging.""" - - def data_bagging(dataset, m=0): - """Sample m examples with replacement""" - n = len(dataset.examples) - return weighted_sample_with_replacement(m or n, dataset.examples, [1]*n) - - def feature_bagging(dataset, p=0.7): - """Feature bagging with probability p to retain an attribute""" - inputs = [i for i in dataset.inputs if probability(p)] - return inputs or dataset.inputs - - def predict(example): - print([predictor(example) for predictor in predictors]) - return mode(predictor(example) for predictor in predictors) - - predictors = [DecisionTreeLearner(DataSet(examples=data_bagging(dataset), - attrs=dataset.attrs, - attrnames=dataset.attrnames, - target=dataset.target, - inputs=feature_bagging(dataset))) for _ in range(n)] - - return predict - -# ______________________________________________________________________________ - -# A decision list is implemented as a list of (test, value) pairs. + probabilities = normalize(remove_all(0, values)) + return sum(-p * np.log2(p) for p in probabilities) def DecisionListLearner(dataset): - """[Figure 18.11]""" + """ + [Figure 18.11] + A decision list implemented as a list of (test, value) pairs. + """ def decision_list_learning(examples): if not examples: @@ -633,8 +469,10 @@ def decision_list_learning(examples): return [(t, o)] + decision_list_learning(examples - examples_t) def find_examples(examples): - """Find a set of examples that all have the same outcome under - some test. Return a tuple of the test, outcome, and examples.""" + """ + Find a set of examples that all have the same outcome under + some test. Return a tuple of the test, outcome, and examples. + """ raise NotImplementedError def passes(example, test): @@ -646,47 +484,141 @@ def predict(example): for test, outcome in predict.decision_list: if passes(example, test): return outcome - + predict.decision_list = decision_list_learning(set(dataset.examples)) return predict -# ______________________________________________________________________________ +def NearestNeighborLearner(dataset, k=1): + """k-NearestNeighbor: the k nearest neighbors vote.""" -def NeuralNetLearner(dataset, hidden_layer_sizes=None, - learning_rate=0.01, epochs=100, activation = sigmoid): - """Layered feed-forward network. + def predict(example): + """Find the k closest items, and have them vote for the best.""" + best = heapq.nsmallest(k, ((dataset.distance(e, example), e) for e in dataset.examples)) + return mode(e[dataset.target] for (d, e) in best) + + return predict + + +def LinearLearner(dataset, learning_rate=0.01, epochs=100): + """ + [Section 18.6.3] + Linear classifier with hard threshold. + """ + idx_i = dataset.inputs + idx_t = dataset.target + examples = dataset.examples + num_examples = len(examples) + + # X transpose + X_col = [dataset.values[i] for i in idx_i] # vertical columns of X + + # add dummy + ones = [1 for _ in range(len(examples))] + X_col = [ones] + X_col + + # initialize random weights + num_weights = len(idx_i) + 1 + w = random_weights(min_value=-0.5, max_value=0.5, num_weights=num_weights) + + for epoch in range(epochs): + err = [] + # pass over all examples + for example in examples: + x = [1] + example + y = np.dot(w, x) + t = example[idx_t] + err.append(t - y) + + # update weights + for i in range(len(w)): + w[i] = w[i] + learning_rate * (np.dot(err, X_col[i]) / num_examples) + + def predict(example): + x = [1] + example + return np.dot(w, x) + + return predict + + +def LogisticLinearLeaner(dataset, learning_rate=0.01, epochs=100): + """ + [Section 18.6.4] + Linear classifier with logistic regression. + """ + idx_i = dataset.inputs + idx_t = dataset.target + examples = dataset.examples + num_examples = len(examples) + + # X transpose + X_col = [dataset.values[i] for i in idx_i] # vertical columns of X + + # add dummy + ones = [1 for _ in range(len(examples))] + X_col = [ones] + X_col + + # initialize random weights + num_weights = len(idx_i) + 1 + w = random_weights(min_value=-0.5, max_value=0.5, num_weights=num_weights) + + for epoch in range(epochs): + err = [] + h = [] + # pass over all examples + for example in examples: + x = [1] + example + y = sigmoid(np.dot(w, x)) + h.append(sigmoid_derivative(y)) + t = example[idx_t] + err.append(t - y) + + # update weights + for i in range(len(w)): + buffer = [x * y for x, y in zip(err, h)] + w[i] = w[i] + learning_rate * (np.dot(buffer, X_col[i]) / num_examples) + + def predict(example): + x = [1] + example + return sigmoid(np.dot(w, x)) + + return predict + + +def NeuralNetLearner(dataset, hidden_layer_sizes=None, learning_rate=0.01, epochs=100, activation=sigmoid): + """ + Layered feed-forward network. hidden_layer_sizes: List of number of hidden units per hidden layer learning_rate: Learning rate of gradient descent epochs: Number of passes over the dataset """ - hidden_layer_sizes = hidden_layer_sizes or [3] # default value + if hidden_layer_sizes is None: + hidden_layer_sizes = [3] i_units = len(dataset.inputs) o_units = len(dataset.values[dataset.target]) # construct a network raw_net = network(i_units, hidden_layer_sizes, o_units, activation) - learned_net = BackPropagationLearner(dataset, raw_net, - learning_rate, epochs, activation) + learned_net = BackPropagationLearner(dataset, raw_net, learning_rate, epochs, activation) def predict(example): - # Input nodes + # input nodes i_nodes = learned_net[0] - # Activate input layer + # activate input layer for v, n in zip(example, i_nodes): n.value = v - # Forward pass + # forward pass for layer in learned_net[1:]: for node in layer: inc = [n.value for n in node.inputs] - in_val = dotproduct(inc, node.weights) + in_val = dot_product(inc, node.weights) node.value = node.activation(in_val) - # Hypothesis + # hypothesis o_nodes = learned_net[-1] prediction = find_max_node(o_nodes) return prediction @@ -694,24 +626,20 @@ def predict(example): return predict -def random_weights(min_value, max_value, num_weights): - return [random.uniform(min_value, max_value) for _ in range(num_weights)] - - def BackPropagationLearner(dataset, net, learning_rate, epochs, activation=sigmoid): - """[Figure 18.23] The back-propagation algorithm for multilayer networks""" - # Initialise weights + """ + [Figure 18.23] + The back-propagation algorithm for multilayer networks. + """ + # initialise weights for layer in net: for node in layer: - node.weights = random_weights(min_value=-0.5, max_value=0.5, - num_weights=len(node.weights)) + node.weights = random_weights(min_value=-0.5, max_value=0.5, num_weights=len(node.weights)) examples = dataset.examples - ''' - As of now dataset.target gives an int instead of list, - Changing dataset class will have effect on all the learners. - Will be taken care of later. - ''' + # As of now dataset.target gives an int instead of list, + # Changing dataset class will have effect on all the learners. + # Will be taken care of later. o_nodes = net[-1] i_nodes = net[0] o_units = len(o_nodes) @@ -722,31 +650,31 @@ def BackPropagationLearner(dataset, net, learning_rate, epochs, activation=sigmo inputs, targets = init_examples(examples, idx_i, idx_t, o_units) for epoch in range(epochs): - # Iterate over each example + # iterate over each example for e in range(len(examples)): i_val = inputs[e] t_val = targets[e] - # Activate input layer + # activate input layer for v, n in zip(i_val, i_nodes): n.value = v - # Forward pass + # forward pass for layer in net[1:]: for node in layer: inc = [n.value for n in node.inputs] - in_val = dotproduct(inc, node.weights) + in_val = dot_product(inc, node.weights) node.value = node.activation(in_val) - # Initialize delta + # initialize delta delta = [[] for _ in range(n_layers)] - # Compute outer layer delta + # compute outer layer delta - # Error for the MSE cost function + # error for the MSE cost function err = [t_val[i] - o_nodes[i].value for i in range(o_units)] - # The activation function used is relu or sigmoid function + # calculate delta at output if node.activation == sigmoid: delta[-1] = [sigmoid_derivative(o_nodes[i].value) * err[i] for i in range(o_units)] elif node.activation == relu: @@ -755,45 +683,47 @@ def BackPropagationLearner(dataset, net, learning_rate, epochs, activation=sigmo delta[-1] = [tanh_derivative(o_nodes[i].value) * err[i] for i in range(o_units)] elif node.activation == elu: delta[-1] = [elu_derivative(o_nodes[i].value) * err[i] for i in range(o_units)] - else: + elif node.activation == leaky_relu: delta[-1] = [leaky_relu_derivative(o_nodes[i].value) * err[i] for i in range(o_units)] + else: + return ValueError("Activation function unknown.") - - # Backward pass + # backward pass h_layers = n_layers - 2 for i in range(h_layers, 0, -1): layer = net[i] h_units = len(layer) - nx_layer = net[i+1] + nx_layer = net[i + 1] # weights from each ith layer node to each i + 1th layer node w = [[node.weights[k] for node in nx_layer] for k in range(h_units)] if activation == sigmoid: - delta[i] = [sigmoid_derivative(layer[j].value) * dotproduct(w[j], delta[i+1]) - for j in range(h_units)] + delta[i] = [sigmoid_derivative(layer[j].value) * dot_product(w[j], delta[i + 1]) + for j in range(h_units)] elif activation == relu: - delta[i] = [relu_derivative(layer[j].value) * dotproduct(w[j], delta[i+1]) - for j in range(h_units)] + delta[i] = [relu_derivative(layer[j].value) * dot_product(w[j], delta[i + 1]) + for j in range(h_units)] elif activation == tanh: - delta[i] = [tanh_derivative(layer[j].value) * dotproduct(w[j], delta[i+1]) - for j in range(h_units)] + delta[i] = [tanh_derivative(layer[j].value) * dot_product(w[j], delta[i + 1]) + for j in range(h_units)] elif activation == elu: - delta[i] = [elu_derivative(layer[j].value) * dotproduct(w[j], delta[i+1]) - for j in range(h_units)] + delta[i] = [elu_derivative(layer[j].value) * dot_product(w[j], delta[i + 1]) + for j in range(h_units)] + elif activation == leaky_relu: + delta[i] = [leaky_relu_derivative(layer[j].value) * dot_product(w[j], delta[i + 1]) + for j in range(h_units)] else: - delta[i] = [leaky_relu_derivative(layer[j].value) * dotproduct(w[j], delta[i+1]) - for j in range(h_units)] + return ValueError("Activation function unknown.") - # Update weights + # update weights for i in range(1, n_layers): layer = net[i] - inc = [node.value for node in net[i-1]] + inc = [node.value for node in net[i - 1]] units = len(layer) for j in range(units): layer[j].weights = vector_add(layer[j].weights, - scalar_vector_product( - learning_rate * delta[i][j], inc)) + scalar_vector_product(learning_rate * delta[i][j], inc)) return net @@ -809,19 +739,20 @@ def PerceptronLearner(dataset, learning_rate=0.01, epochs=100): def predict(example): o_nodes = learned_net[1] - # Forward pass + # forward pass for node in o_nodes: - in_val = dotproduct(example, node.weights) + in_val = dot_product(example, node.weights) node.value = node.activation(in_val) - # Hypothesis + # hypothesis return find_max_node(o_nodes) return predict class NNUnit: - """Single Unit of Multiple Layer Neural Network + """ + Single Unit of Multiple Layer Neural Network inputs: Incoming connections weights: Weights to incoming connections """ @@ -834,141 +765,291 @@ def __init__(self, activation=sigmoid, weights=None, inputs=None): def network(input_units, hidden_layer_sizes, output_units, activation=sigmoid): - """Create Directed Acyclic Network of given number layers. + """ + Create Directed Acyclic Network of given number layers. hidden_layers_sizes : List number of neuron units in each hidden layer excluding input and output layers """ layers_sizes = [input_units] + hidden_layer_sizes + [output_units] - net = [[NNUnit(activation) for n in range(size)] - for size in layers_sizes] + net = [[NNUnit(activation) for _ in range(size)] for size in layers_sizes] n_layers = len(net) - # Make Connection + # make connection for i in range(1, n_layers): for n in net[i]: - for k in net[i-1]: + for k in net[i - 1]: n.inputs.append(k) n.weights.append(0) return net def init_examples(examples, idx_i, idx_t, o_units): - inputs = {} - targets = {} - - for i in range(len(examples)): - e = examples[i] + inputs, targets = {}, {} - # Input values of e + for i, e in enumerate(examples): + # input values of e inputs[i] = [e[i] for i in idx_i] if o_units > 1: - # One-Hot representation of e's target + # one-hot representation of e's target t = [0 for i in range(o_units)] t[e[idx_t]] = 1 targets[i] = t else: - # Target value of e + # target value of e targets[i] = [e[idx_t]] return inputs, targets def find_max_node(nodes): - return nodes.index(argmax(nodes, key=lambda node: node.value)) + return nodes.index(max(nodes, key=lambda node: node.value)) -# ______________________________________________________________________________ +class SVC: -def LinearLearner(dataset, learning_rate=0.01, epochs=100): - """Define with learner = LinearLearner(data); infer with learner(x).""" - idx_i = dataset.inputs - idx_t = dataset.target # As of now, dataset.target gives only one index. - examples = dataset.examples - num_examples = len(examples) + def __init__(self, kernel=linear_kernel, C=1.0, verbose=False): + self.kernel = kernel + self.C = C # hyper-parameter + self.sv_idx, self.sv, self.sv_y = np.zeros(0), np.zeros(0), np.zeros(0) + self.alphas = np.zeros(0) + self.w = None + self.b = 0.0 # intercept + self.verbose = verbose - # X transpose - X_col = [dataset.values[i] for i in idx_i] # vertical columns of X + def fit(self, X, y): + """ + Trains the model by solving a quadratic programming problem. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + # In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations) + self.solve_qp(X, y) + sv = self.alphas > 1e-5 + self.sv_idx = np.arange(len(self.alphas))[sv] + self.sv, self.sv_y, self.alphas = X[sv], y[sv], self.alphas[sv] + + if self.kernel == linear_kernel: + self.w = np.dot(self.alphas * self.sv_y, self.sv) + + for n in range(len(self.alphas)): + self.b += self.sv_y[n] + self.b -= np.sum(self.alphas * self.sv_y * self.K[self.sv_idx[n], sv]) + self.b /= len(self.alphas) + return self + + def solve_qp(self, X, y): + """ + Solves a quadratic programming problem. In QP formulation (dual): + m variables, 2m+1 constraints (1 equation, 2m inequations). + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + m = len(y) # m = n_samples + self.K = self.kernel(X) # gram matrix + P = self.K * np.outer(y, y) + q = -np.ones(m) + lb = np.zeros(m) # lower bounds + ub = np.ones(m) * self.C # upper bounds + A = y.astype(np.float64) # equality matrix + b = np.zeros(1) # equality vector + self.alphas = solve_qp(P, q, A=A, b=b, lb=lb, ub=ub, solver='cvxopt', + sym_proj=True, verbose=self.verbose) + + def predict_score(self, X): + """ + Predicts the score for a given example. + """ + if self.w is None: + return np.dot(self.alphas * self.sv_y, self.kernel(self.sv, X)) + self.b + return np.dot(X, self.w) + self.b - # Add dummy - ones = [1 for _ in range(len(examples))] - X_col = [ones] + X_col + def predict(self, X): + """ + Predicts the class of a given example. + """ + return np.sign(self.predict_score(X)) - # Initialize random weigts - num_weights = len(idx_i) + 1 - w = random_weights(min_value=-0.5, max_value=0.5, num_weights=num_weights) - for epoch in range(epochs): - err = [] - # Pass over all examples - for example in examples: - x = [1] + example - y = dotproduct(w, x) - t = example[idx_t] - err.append(t - y) +class SVR: - # update weights - for i in range(len(w)): - w[i] = w[i] + learning_rate * (dotproduct(err, X_col[i]) / num_examples) + def __init__(self, kernel=linear_kernel, C=1.0, epsilon=0.1, verbose=False): + self.kernel = kernel + self.C = C # hyper-parameter + self.epsilon = epsilon # epsilon insensitive loss value + self.sv_idx, self.sv = np.zeros(0), np.zeros(0) + self.alphas_p, self.alphas_n = np.zeros(0), np.zeros(0) + self.w = None + self.b = 0.0 # intercept + self.verbose = verbose - def predict(example): - x = [1] + example - return dotproduct(w, x) - return predict + def fit(self, X, y): + """ + Trains the model by solving a quadratic programming problem. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + # In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations) + self.solve_qp(X, y) + + sv = np.logical_or(self.alphas_p > 1e-5, self.alphas_n > 1e-5) + self.sv_idx = np.arange(len(self.alphas_p))[sv] + self.sv, sv_y = X[sv], y[sv] + self.alphas_p, self.alphas_n = self.alphas_p[sv], self.alphas_n[sv] -# ______________________________________________________________________________ + if self.kernel == linear_kernel: + self.w = np.dot(self.alphas_p - self.alphas_n, self.sv) + + for n in range(len(self.alphas_p)): + self.b += sv_y[n] + self.b -= np.sum((self.alphas_p - self.alphas_n) * self.K[self.sv_idx[n], sv]) + self.b -= self.epsilon + self.b /= len(self.alphas_p) + + return self + + def solve_qp(self, X, y): + """ + Solves a quadratic programming problem. In QP formulation (dual): + m variables, 2m+1 constraints (1 equation, 2m inequations). + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + # + m = len(y) # m = n_samples + self.K = self.kernel(X) # gram matrix + P = np.vstack((np.hstack((self.K, -self.K)), # alphas_p, alphas_n + np.hstack((-self.K, self.K)))) # alphas_n, alphas_p + q = np.hstack((-y, y)) + self.epsilon + lb = np.zeros(2 * m) # lower bounds + ub = np.ones(2 * m) * self.C # upper bounds + A = np.hstack((np.ones(m), -np.ones(m))) # equality matrix + b = np.zeros(1) # equality vector + alphas = solve_qp(P, q, A=A, b=b, lb=lb, ub=ub, solver='cvxopt', + sym_proj=True, verbose=self.verbose) + self.alphas_p = alphas[:m] + self.alphas_n = alphas[m:] + + def predict(self, X): + if self.kernel != linear_kernel: + return np.dot(self.alphas_p - self.alphas_n, self.kernel(self.sv, X)) + self.b + return np.dot(X, self.w) + self.b + + +class MultiClassLearner: + + def __init__(self, clf, decision_function='ovr'): + self.clf = clf + self.decision_function = decision_function + self.n_class, self.classifiers = 0, [] + + def fit(self, X, y): + """ + Trains n_class or n_class * (n_class - 1) / 2 classifiers + according to the training method, ovr or ovo respectively. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + :return: array of classifiers + """ + labels = np.unique(y) + self.n_class = len(labels) + if self.decision_function == 'ovr': # one-vs-rest method + for label in labels: + y1 = np.array(y) + y1[y1 != label] = -1.0 + y1[y1 == label] = 1.0 + self.clf.fit(X, y1) + self.classifiers.append(copy.deepcopy(self.clf)) + elif self.decision_function == 'ovo': # use one-vs-one method + n_labels = len(labels) + for i in range(n_labels): + for j in range(i + 1, n_labels): + neg_id, pos_id = y == labels[i], y == labels[j] + X1, y1 = np.r_[X[neg_id], X[pos_id]], np.r_[y[neg_id], y[pos_id]] + y1[y1 == labels[i]] = -1.0 + y1[y1 == labels[j]] = 1.0 + self.clf.fit(X1, y1) + self.classifiers.append(copy.deepcopy(self.clf)) + else: + return ValueError("Decision function must be either 'ovr' or 'ovo'.") + return self + + def predict(self, X): + """ + Predicts the class of a given example according to the training method. + """ + n_samples = len(X) + if self.decision_function == 'ovr': # one-vs-rest method + assert len(self.classifiers) == self.n_class + score = np.zeros((n_samples, self.n_class)) + for i in range(self.n_class): + clf = self.classifiers[i] + score[:, i] = clf.predict_score(X) + return np.argmax(score, axis=1) + elif self.decision_function == 'ovo': # use one-vs-one method + assert len(self.classifiers) == self.n_class * (self.n_class - 1) / 2 + vote = np.zeros((n_samples, self.n_class)) + clf_id = 0 + for i in range(self.n_class): + for j in range(i + 1, self.n_class): + res = self.classifiers[clf_id].predict(X) + vote[res < 0, i] += 1.0 # negative sample: class i + vote[res > 0, j] += 1.0 # positive sample: class j + clf_id += 1 + return np.argmax(vote, axis=1) + else: + return ValueError("Decision function must be either 'ovr' or 'ovo'.") def EnsembleLearner(learners): """Given a list of learning algorithms, have them vote.""" + def train(dataset): predictors = [learner(dataset) for learner in learners] def predict(example): return mode(predictor(example) for predictor in predictors) + return predict - return train -# ______________________________________________________________________________ + return train -def AdaBoost(L, K): +def ada_boost(dataset, L, K): """[Figure 18.34]""" - def train(dataset): - examples, target = dataset.examples, dataset.target - N = len(examples) - epsilon = 1/(2*N) - w = [1/N]*N - h, z = [], [] - for k in range(K): - h_k = L(dataset, w) - h.append(h_k) - error = sum(weight for example, weight in zip(examples, w) - if example[target] != h_k(example)) - - # Avoid divide-by-0 from either 0% or 100% error rates: - error = clip(error, epsilon, 1 - epsilon) - for j, example in enumerate(examples): - if example[target] == h_k(example): - w[j] *= error/(1 - error) - w = normalize(w) - z.append(math.log((1 - error)/error)) - return WeightedMajority(h, z) - return train - - -def WeightedMajority(predictors, weights): + examples, target = dataset.examples, dataset.target + n = len(examples) + eps = 1 / (2 * n) + w = [1 / n] * n + h, z = [], [] + for k in range(K): + h_k = L(dataset, w) + h.append(h_k) + error = sum(weight for example, weight in zip(examples, w) if example[target] != h_k(example)) + # avoid divide-by-0 from either 0% or 100% error rates + error = np.clip(error, eps, 1 - eps) + for j, example in enumerate(examples): + if example[target] == h_k(example): + w[j] *= error / (1 - error) + w = normalize(w) + z.append(np.log((1 - error) / error)) + return weighted_majority(h, z) + + +def weighted_majority(predictors, weights): """Return a predictor that takes a weighted vote.""" + def predict(example): - return weighted_mode((predictor(example) for predictor in predictors), - weights) + return weighted_mode((predictor(example) for predictor in predictors), weights) + return predict def weighted_mode(values, weights): - """Return the value with the greatest total weight. + """ + Return the value with the greatest total weight. >>> weighted_mode('abbaa', [1, 2, 3, 1, 2]) 'b' """ @@ -977,15 +1058,41 @@ def weighted_mode(values, weights): totals[v] += w return max(totals, key=totals.__getitem__) -# _____________________________________________________________________________ -# Adapting an unweighted learner for AdaBoost + +def RandomForest(dataset, n=5): + """An ensemble of Decision Trees trained using bagging and feature bagging.""" + + def data_bagging(dataset, m=0): + """Sample m examples with replacement""" + n = len(dataset.examples) + return weighted_sample_with_replacement(m or n, dataset.examples, [1] * n) + + def feature_bagging(dataset, p=0.7): + """Feature bagging with probability p to retain an attribute""" + inputs = [i for i in dataset.inputs if probability(p)] + return inputs or dataset.inputs + + def predict(example): + print([predictor(example) for predictor in predictors]) + return mode(predictor(example) for predictor in predictors) + + predictors = [DecisionTreeLearner(DataSet(examples=data_bagging(dataset), attrs=dataset.attrs, + attr_names=dataset.attr_names, target=dataset.target, + inputs=feature_bagging(dataset))) for _ in range(n)] + + return predict def WeightedLearner(unweighted_learner): - """Given a learner that takes just an unweighted dataset, return - one that takes also a weight for each example. [p. 749 footnote 14]""" + """ + [Page 749 footnote 14] + Given a learner that takes just an unweighted dataset, return + one that takes also a weight for each example. + """ + def train(dataset, weights): return unweighted_learner(replicated_dataset(dataset, weights)) + return train @@ -998,7 +1105,8 @@ def replicated_dataset(dataset, weights, n=None): def weighted_replicate(seq, weights, n): - """Return n selections from seq, with the count of each element of + """ + Return n selections from seq, with the count of each element of seq proportional to the corresponding weight (filling in fractions randomly). >>> weighted_replicate('ABC', [1, 2, 1], 4) @@ -1006,173 +1114,56 @@ def weighted_replicate(seq, weights, n): """ assert len(seq) == len(weights) weights = normalize(weights) - wholes = [int(w*n) for w in weights] - fractions = [(w*n) % 1 for w in weights] - return (flatten([x]*nx for x, nx in zip(seq, wholes)) + + wholes = [int(w * n) for w in weights] + fractions = [(w * n) % 1 for w in weights] + return (flatten([x] * nx for x, nx in zip(seq, wholes)) + weighted_sample_with_replacement(n - sum(wholes), seq, fractions)) -def flatten(seqs): return sum(seqs, []) +# metrics -# _____________________________________________________________________________ -# Functions for testing learners on examples +def accuracy_score(y_pred, y_true): + assert y_pred.shape == y_true.shape + return np.mean(np.equal(y_pred, y_true)) -def err_ratio(predict, dataset, examples=None, verbose=0): - """Return the proportion of the examples that are NOT correctly predicted. - verbose - 0: No output; 1: Output wrong; 2 (or greater): Output correct""" - examples = examples or dataset.examples - if len(examples) == 0: - return 0.0 - right = 0 - for example in examples: - desired = example[dataset.target] - output = predict(dataset.sanitize(example)) - if output == desired: - right += 1 - if verbose >= 2: - print(' OK: got {} for {}'.format(desired, example)) - elif verbose: - print('WRONG: got {}, expected {} for {}'.format( - output, desired, example)) - return 1 - (right/len(examples)) - - -def grade_learner(predict, tests): - """Grades the given learner based on how many tests it passes. - tests is a list with each element in the form: (values, output).""" - return mean(int(predict(X) == y) for X, y in tests) +def r2_score(y_pred, y_true): + assert y_pred.shape == y_true.shape + return 1. - (np.sum(np.square(y_pred - y_true)) / # sum of square of residuals + np.sum(np.square(y_true - np.mean(y_true)))) # total sum of squares -def train_test_split(dataset, start, end): - """Reserve dataset.examples[start:end] for test; train on the remainder.""" - start = int(start) - end = int(end) - examples = dataset.examples - train = examples[:start] + examples[end:] - val = examples[start:end] - return train, val - - -def cross_validation(learner, size, dataset, k=10, trials=1): - """Do k-fold cross_validate and return their mean. - That is, keep out 1/k of the examples for testing on each of k runs. - Shuffle the examples first; if trials>1, average over several shuffles. - Returns Training error, Validataion error""" - k = k or len(dataset.examples) - if trials > 1: - trial_errT = 0 - trial_errV = 0 - for t in range(trials): - errT, errV = cross_validation(learner, size, dataset, - k=10, trials=1) - trial_errT += errT - trial_errV += errV - return trial_errT/trials, trial_errV/trials - else: - fold_errT = 0 - fold_errV = 0 - n = len(dataset.examples) - examples = dataset.examples - random.shuffle(dataset.examples) - for fold in range(k): - train_data, val_data = train_test_split(dataset, fold * (n / k), - (fold + 1) * (n / k)) - dataset.examples = train_data - h = learner(dataset, size) - fold_errT += err_ratio(h, dataset, train_data) - fold_errV += err_ratio(h, dataset, val_data) - - # Reverting back to original once test is completed - dataset.examples = examples - return fold_errT/k, fold_errV/k - -# TODO: The function cross_validation_wrapper needs to be fixed. (The while loop runs forever!) -def cross_validation_wrapper(learner, dataset, k=10, trials=1): - """[Fig 18.8] - Return the optimal value of size having minimum error - on validation set. - err_train: A training error array, indexed by size - err_val: A validation error array, indexed by size - """ - err_val = [] - err_train = [] - size = 1 - - while True: - errT, errV = cross_validation(learner, size, dataset, k) - # Check for convergence provided err_val is not empty - if (err_train and isclose(err_train[-1], errT, rel_tol=1e-6)): - best_size = 0 - min_val = math.inf - - i = 0 - while i < size: - if err_val[i] < min_val: - min_val = err_val[i] - best_size = i - i += 1 - err_val.append(errV) - err_train.append(errT) - print(err_val) - size += 1 - - - -def leave_one_out(learner, dataset, size=None): - """Leave one out cross-validation over the dataset.""" - return cross_validation(learner, size, dataset, k=len(dataset.examples)) - -# TODO learningcurve needs to fixed -def learningcurve(learner, dataset, trials=10, sizes=None): - if sizes is None: - sizes = list(range(2, len(dataset.examples) - 10, 2)) - - def score(learner, size): - random.shuffle(dataset.examples) - return train_test_split(learner, dataset, 0, size) - return [(size, mean([score(learner, size) for t in range(trials)])) - for size in sizes] - -# ______________________________________________________________________________ -# The rest of this file gives datasets for machine learning problems. - - -orings = DataSet(name='orings', target='Distressed', - attrnames="Rings Distressed Temp Pressure Flightnum") +# datasets +orings = DataSet(name='orings', target='Distressed', attr_names='Rings Distressed Temp Pressure Flightnum') zoo = DataSet(name='zoo', target='type', exclude=['name'], - attrnames="name hair feathers eggs milk airborne aquatic " + - "predator toothed backbone breathes venomous fins legs tail " + - "domestic catsize type") + attr_names='name hair feathers eggs milk airborne aquatic predator toothed backbone ' + 'breathes venomous fins legs tail domestic catsize type') - -iris = DataSet(name="iris", target="class", - attrnames="sepal-len sepal-width petal-len petal-width class") - -# ______________________________________________________________________________ -# The Restaurant example from [Figure 18.2] +iris = DataSet(name='iris', target='class', attr_names='sepal-len sepal-width petal-len petal-width class') def RestaurantDataSet(examples=None): - """Build a DataSet of Restaurant waiting examples. [Figure 18.3]""" + """ + [Figure 18.3] + Build a DataSet of Restaurant waiting examples. + """ return DataSet(name='restaurant', target='Wait', examples=examples, - attrnames='Alternate Bar Fri/Sat Hungry Patrons Price ' + - 'Raining Reservation Type WaitEstimate Wait') + attr_names='Alternate Bar Fri/Sat Hungry Patrons Price Raining Reservation Type WaitEstimate Wait') restaurant = RestaurantDataSet() -def T(attrname, branches): - branches = {value: (child if isinstance(child, DecisionFork) - else DecisionLeaf(child)) +def T(attr_name, branches): + branches = {value: (child if isinstance(child, DecisionFork) else DecisionLeaf(child)) for value, child in branches.items()} - return DecisionFork(restaurant.attrnum(attrname), attrname, print, branches) + return DecisionFork(restaurant.attr_num(attr_name), attr_name, print, branches) -""" [Figure 18.2] +""" +[Figure 18.2] A decision tree for deciding whether to wait for a table at a hotel. """ @@ -1185,8 +1176,7 @@ def T(attrname, branches): {'Yes': 'Yes', 'No': T('Bar', {'No': 'No', 'Yes': 'Yes'})}), - 'Yes': T('Fri/Sat', {'No': 'No', 'Yes': 'Yes'})} - ), + 'Yes': T('Fri/Sat', {'No': 'No', 'Yes': 'Yes'})}), '10-30': T('Hungry', {'No': 'Yes', 'Yes': T('Alternate', @@ -1198,33 +1188,36 @@ def T(attrname, branches): def SyntheticRestaurant(n=20): """Generate a DataSet with n examples.""" + def gen(): example = list(map(random.choice, restaurant.values)) example[restaurant.target] = waiting_decision_tree(example) return example - return RestaurantDataSet([gen() for i in range(n)]) -# ______________________________________________________________________________ -# Artificial, generated datasets. + return RestaurantDataSet([gen() for _ in range(n)]) def Majority(k, n): - """Return a DataSet with n k-bit examples of the majority problem: - k random bits followed by a 1 if more than half the bits are 1, else 0.""" + """ + Return a DataSet with n k-bit examples of the majority problem: + k random bits followed by a 1 if more than half the bits are 1, else 0. + """ examples = [] for i in range(n): - bits = [random.choice([0, 1]) for i in range(k)] + bits = [random.choice([0, 1]) for _ in range(k)] bits.append(int(sum(bits) > k / 2)) examples.append(bits) - return DataSet(name="majority", examples=examples) + return DataSet(name='majority', examples=examples) -def Parity(k, n, name="parity"): - """Return a DataSet with n k-bit examples of the parity problem: - k random bits followed by a 1 if an odd number of bits are 1, else 0.""" +def Parity(k, n, name='parity'): + """ + Return a DataSet with n k-bit examples of the parity problem: + k random bits followed by a 1 if an odd number of bits are 1, else 0. + """ examples = [] for i in range(n): - bits = [random.choice([0, 1]) for i in range(k)] + bits = [random.choice([0, 1]) for _ in range(k)] bits.append(sum(bits) % 2) examples.append(bits) return DataSet(name=name, examples=examples) @@ -1232,32 +1225,29 @@ def Parity(k, n, name="parity"): def Xor(n): """Return a DataSet with n examples of 2-input xor.""" - return Parity(2, n, name="xor") + return Parity(2, n, name='xor') def ContinuousXor(n): - "2 inputs are chosen uniformly from (0.0 .. 2.0]; output is xor of ints." + """2 inputs are chosen uniformly from (0.0 .. 2.0]; output is xor of ints.""" examples = [] for i in range(n): - x, y = [random.uniform(0.0, 2.0) for i in '12'] - examples.append([x, y, int(x) != int(y)]) - return DataSet(name="continuous xor", examples=examples) + x, y = [random.uniform(0.0, 2.0) for _ in '12'] + examples.append([x, y, x != y]) + return DataSet(name='continuous xor', examples=examples) -# ______________________________________________________________________________ +def compare(algorithms=None, datasets=None, k=10, trials=1): + """ + Compare various learners on various datasets using cross-validation. + Print results as a table. + """ + # default list of algorithms + algorithms = algorithms or [PluralityLearner, NaiveBayesLearner, NearestNeighborLearner, DecisionTreeLearner] -def compare(algorithms=None, - datasets=None, - k=10, trials=1): - """Compare various learners on various datasets using cross-validation. - Print results as a table.""" - algorithms = algorithms or [PluralityLearner, NaiveBayesLearner, # default list - NearestNeighborLearner, DecisionTreeLearner] # of algorithms - - datasets = datasets or [iris, orings, zoo, restaurant, SyntheticRestaurant(20), # default list - Majority(7, 100), Parity(7, 100), Xor(100)] # of datasets + # default list of datasets + datasets = datasets or [iris, orings, zoo, restaurant, SyntheticRestaurant(20), + Majority(7, 100), Parity(7, 100), Xor(100)] - print_table([[a.__name__.replace('Learner', '')] + - [cross_validation(a, d, k, trials) for d in datasets] - for a in algorithms], - header=[''] + [d.name[0:7] for d in datasets], numfmt='%.2f') + print_table([[a.__name__.replace('Learner', '')] + [cross_validation(a, d, k=k, trials=trials) for d in datasets] + for a in algorithms], header=[''] + [d.name[0:7] for d in datasets], numfmt='%.2f') diff --git a/learning4e.py b/learning4e.py new file mode 100644 index 000000000..12c0defa5 --- /dev/null +++ b/learning4e.py @@ -0,0 +1,1039 @@ +"""Learning from examples (Chapters 18)""" + +import copy +from collections import defaultdict +from statistics import stdev + +from qpsolvers import solve_qp + +from deep_learning4e import Sigmoid +from probabilistic_learning import NaiveBayesLearner +from utils4e import * + + +class DataSet: + """ + A data set for a machine learning problem. It has the following fields: + + d.examples A list of examples. Each one is a list of attribute values. + d.attrs A list of integers to index into an example, so example[attr] + gives a value. Normally the same as range(len(d.examples[0])). + d.attr_names Optional list of mnemonic names for corresponding attrs. + d.target The attribute that a learning algorithm will try to predict. + By default the final attribute. + d.inputs The list of attrs without the target. + d.values A list of lists: each sublist is the set of possible + values for the corresponding attribute. If initially None, + it is computed from the known examples by self.set_problem. + If not None, an erroneous value raises ValueError. + d.distance A function from a pair of examples to a non-negative number. + Should be symmetric, etc. Defaults to mean_boolean_error + since that can handle any field types. + d.name Name of the data set (for output display only). + d.source URL or other source where the data came from. + d.exclude A list of attribute indexes to exclude from d.inputs. Elements + of this list can either be integers (attrs) or attr_names. + + Normally, you call the constructor and you're done; then you just + access fields like d.examples and d.target and d.inputs. + """ + + def __init__(self, examples=None, attrs=None, attr_names=None, target=-1, inputs=None, + values=None, distance=mean_boolean_error, name='', source='', exclude=()): + """ + Accepts any of DataSet's fields. Examples can also be a + string or file from which to parse examples using parse_csv. + Optional parameter: exclude, as documented in .set_problem(). + >>> DataSet(examples='1, 2, 3') + + """ + self.name = name + self.source = source + self.values = values + self.distance = distance + self.got_values_flag = bool(values) + + # initialize .examples from string or list or data directory + if isinstance(examples, str): + self.examples = parse_csv(examples) + elif examples is None: + self.examples = parse_csv(open_data(name + '.csv').read()) + else: + self.examples = examples + + # attrs are the indices of examples, unless otherwise stated. + if self.examples is not None and attrs is None: + attrs = list(range(len(self.examples[0]))) + + self.attrs = attrs + + # initialize .attr_names from string, list, or by default + if isinstance(attr_names, str): + self.attr_names = attr_names.split() + else: + self.attr_names = attr_names or attrs + self.set_problem(target, inputs=inputs, exclude=exclude) + + def set_problem(self, target, inputs=None, exclude=()): + """ + Set (or change) the target and/or inputs. + This way, one DataSet can be used multiple ways. inputs, if specified, + is a list of attributes, or specify exclude as a list of attributes + to not use in inputs. Attributes can be -n .. n, or an attr_name. + Also computes the list of possible values, if that wasn't done yet. + """ + self.target = self.attr_num(target) + exclude = list(map(self.attr_num, exclude)) + if inputs: + self.inputs = remove_all(self.target, inputs) + else: + self.inputs = [a for a in self.attrs if a != self.target and a not in exclude] + if not self.values: + self.update_values() + self.check_me() + + def check_me(self): + """Check that my fields make sense.""" + assert len(self.attr_names) == len(self.attrs) + assert self.target in self.attrs + assert self.target not in self.inputs + assert set(self.inputs).issubset(set(self.attrs)) + if self.got_values_flag: + # only check if values are provided while initializing DataSet + list(map(self.check_example, self.examples)) + + def add_example(self, example): + """Add an example to the list of examples, checking it first.""" + self.check_example(example) + self.examples.append(example) + + def check_example(self, example): + """Raise ValueError if example has any invalid values.""" + if self.values: + for a in self.attrs: + if example[a] not in self.values[a]: + raise ValueError('Bad value {} for attribute {} in {}' + .format(example[a], self.attr_names[a], example)) + + def attr_num(self, attr): + """Returns the number used for attr, which can be a name, or -n .. n-1.""" + if isinstance(attr, str): + return self.attr_names.index(attr) + elif attr < 0: + return len(self.attrs) + attr + else: + return attr + + def update_values(self): + self.values = list(map(unique, zip(*self.examples))) + + def sanitize(self, example): + """Return a copy of example, with non-input attributes replaced by None.""" + return [attr_i if i in self.inputs else None for i, attr_i in enumerate(example)][:-1] + + def classes_to_numbers(self, classes=None): + """Converts class names to numbers.""" + if not classes: + # if classes were not given, extract them from values + classes = sorted(self.values[self.target]) + for item in self.examples: + item[self.target] = classes.index(item[self.target]) + + def remove_examples(self, value=''): + """Remove examples that contain given value.""" + self.examples = [x for x in self.examples if value not in x] + self.update_values() + + def split_values_by_classes(self): + """Split values into buckets according to their class.""" + buckets = defaultdict(lambda: []) + target_names = self.values[self.target] + + for v in self.examples: + item = [a for a in v if a not in target_names] # remove target from item + buckets[v[self.target]].append(item) # add item to bucket of its class + + return buckets + + def find_means_and_deviations(self): + """ + Finds the means and standard deviations of self.dataset. + means : a dictionary for each class/target. Holds a list of the means + of the features for the class. + deviations: a dictionary for each class/target. Holds a list of the sample + standard deviations of the features for the class. + """ + target_names = self.values[self.target] + feature_numbers = len(self.inputs) + + item_buckets = self.split_values_by_classes() + + means = defaultdict(lambda: [0] * feature_numbers) + deviations = defaultdict(lambda: [0] * feature_numbers) + + for t in target_names: + # find all the item feature values for item in class t + features = [[] for _ in range(feature_numbers)] + for item in item_buckets[t]: + for i in range(feature_numbers): + features[i].append(item[i]) + + # calculate means and deviations fo the class + for i in range(feature_numbers): + means[t][i] = mean(features[i]) + deviations[t][i] = stdev(features[i]) + + return means, deviations + + def __repr__(self): + return ''.format(self.name, len(self.examples), len(self.attrs)) + + +def parse_csv(input, delim=','): + r""" + Input is a string consisting of lines, each line has comma-delimited + fields. Convert this into a list of lists. Blank lines are skipped. + Fields that look like numbers are converted to numbers. + The delim defaults to ',' but '\t' and None are also reasonable values. + >>> parse_csv('1, 2, 3 \n 0, 2, na') + [[1, 2, 3], [0, 2, 'na']] + """ + lines = [line for line in input.splitlines() if line.strip()] + return [list(map(num_or_str, line.split(delim))) for line in lines] + + +def err_ratio(learner, dataset, examples=None): + """ + Return the proportion of the examples that are NOT correctly predicted. + verbose - 0: No output; 1: Output wrong; 2 (or greater): Output correct + """ + examples = examples or dataset.examples + if len(examples) == 0: + return 0.0 + right = 0 + for example in examples: + desired = example[dataset.target] + output = learner.predict(dataset.sanitize(example)) + if np.allclose(output, desired): + right += 1 + return 1 - (right / len(examples)) + + +def grade_learner(learner, tests): + """ + Grades the given learner based on how many tests it passes. + tests is a list with each element in the form: (values, output). + """ + return mean(int(learner.predict(X) == y) for X, y in tests) + + +def train_test_split(dataset, start=None, end=None, test_split=None): + """ + If you are giving 'start' and 'end' as parameters, + then it will return the testing set from index 'start' to 'end' + and the rest for training. + If you give 'test_split' as a parameter then it will return + test_split * 100% as the testing set and the rest as + training set. + """ + examples = dataset.examples + if test_split is None: + train = examples[:start] + examples[end:] + val = examples[start:end] + else: + total_size = len(examples) + val_size = int(total_size * test_split) + train_size = total_size - val_size + train = examples[:train_size] + val = examples[train_size:total_size] + + return train, val + + +def model_selection(learner, dataset, k=10, trials=1): + """ + [Figure 18.8] + Return the optimal value of size having minimum error on validation set. + err: a validation error array, indexed by size + """ + errs = [] + size = 1 + while True: + err = cross_validation(learner, dataset, size, k, trials) + # check for convergence provided err_val is not empty + if err and not np.isclose(err[-1], err, rtol=1e-6): + best_size = 0 + min_val = np.inf + i = 0 + while i < size: + if errs[i] < min_val: + min_val = errs[i] + best_size = i + i += 1 + return learner(dataset, best_size) + errs.append(err) + size += 1 + + +def cross_validation(learner, dataset, size=None, k=10, trials=1): + """ + Do k-fold cross_validate and return their mean. + That is, keep out 1/k of the examples for testing on each of k runs. + Shuffle the examples first; if trials > 1, average over several shuffles. + Returns Training error + """ + k = k or len(dataset.examples) + if trials > 1: + trial_errs = 0 + for t in range(trials): + errs = cross_validation(learner, dataset, size, k, trials) + trial_errs += errs + return trial_errs / trials + else: + fold_errs = 0 + n = len(dataset.examples) + examples = dataset.examples + random.shuffle(dataset.examples) + for fold in range(k): + train_data, val_data = train_test_split(dataset, fold * (n // k), (fold + 1) * (n // k)) + dataset.examples = train_data + h = learner(dataset, size) + fold_errs += err_ratio(h, dataset, train_data) + # reverting back to original once test is completed + dataset.examples = examples + return fold_errs / k + + +def leave_one_out(learner, dataset, size=None): + """Leave one out cross-validation over the dataset.""" + return cross_validation(learner, dataset, size, len(dataset.examples)) + + +def learning_curve(learner, dataset, trials=10, sizes=None): + if sizes is None: + sizes = list(range(2, len(dataset.examples) - trials, 2)) + + def score(learner, size): + random.shuffle(dataset.examples) + return cross_validation(learner, dataset, size, trials) + + return [(size, mean([score(learner, size) for _ in range(trials)])) for size in sizes] + + +class PluralityLearner: + """ + A very dumb algorithm: always pick the result that was most popular + in the training data. Makes a baseline for comparison. + """ + + def __init__(self, dataset): + self.most_popular = mode([e[dataset.target] for e in dataset.examples]) + + def predict(self, example): + """Always return same result: the most popular from the training set.""" + return self.most_popular + + +class DecisionFork: + """ + A fork of a decision tree holds an attribute to test, and a dict + of branches, one for each of the attribute's values. + """ + + def __init__(self, attr, attr_name=None, default_child=None, branches=None): + """Initialize by saying what attribute this node tests.""" + self.attr = attr + self.attr_name = attr_name or attr + self.default_child = default_child + self.branches = branches or {} + + def __call__(self, example): + """Given an example, classify it using the attribute and the branches.""" + attr_val = example[self.attr] + if attr_val in self.branches: + return self.branches[attr_val](example) + else: + # return default class when attribute is unknown + return self.default_child(example) + + def add(self, val, subtree): + """Add a branch. If self.attr = val, go to the given subtree.""" + self.branches[val] = subtree + + def display(self, indent=0): + name = self.attr_name + print('Test', name) + for (val, subtree) in self.branches.items(): + print(' ' * 4 * indent, name, '=', val, '==>', end=' ') + subtree.display(indent + 1) + + def __repr__(self): + return 'DecisionFork({0!r}, {1!r}, {2!r})'.format(self.attr, self.attr_name, self.branches) + + +class DecisionLeaf: + """A leaf of a decision tree holds just a result.""" + + def __init__(self, result): + self.result = result + + def __call__(self, example): + return self.result + + def display(self): + print('RESULT =', self.result) + + def __repr__(self): + return repr(self.result) + + +class DecisionTreeLearner: + """[Figure 18.5]""" + + def __init__(self, dataset): + self.dataset = dataset + self.tree = self.decision_tree_learning(dataset.examples, dataset.inputs) + + def decision_tree_learning(self, examples, attrs, parent_examples=()): + if len(examples) == 0: + return self.plurality_value(parent_examples) + if self.all_same_class(examples): + return DecisionLeaf(examples[0][self.dataset.target]) + if len(attrs) == 0: + return self.plurality_value(examples) + A = self.choose_attribute(attrs, examples) + tree = DecisionFork(A, self.dataset.attr_names[A], self.plurality_value(examples)) + for (v_k, exs) in self.split_by(A, examples): + subtree = self.decision_tree_learning(exs, remove_all(A, attrs), examples) + tree.add(v_k, subtree) + return tree + + def plurality_value(self, examples): + """ + Return the most popular target value for this set of examples. + (If target is binary, this is the majority; otherwise plurality). + """ + popular = argmax_random_tie(self.dataset.values[self.dataset.target], + key=lambda v: self.count(self.dataset.target, v, examples)) + return DecisionLeaf(popular) + + def count(self, attr, val, examples): + """Count the number of examples that have example[attr] = val.""" + return sum(e[attr] == val for e in examples) + + def all_same_class(self, examples): + """Are all these examples in the same target class?""" + class0 = examples[0][self.dataset.target] + return all(e[self.dataset.target] == class0 for e in examples) + + def choose_attribute(self, attrs, examples): + """Choose the attribute with the highest information gain.""" + return argmax_random_tie(attrs, key=lambda a: self.information_gain(a, examples)) + + def information_gain(self, attr, examples): + """Return the expected reduction in entropy from splitting by attr.""" + + def I(examples): + return information_content([self.count(self.dataset.target, v, examples) + for v in self.dataset.values[self.dataset.target]]) + + n = len(examples) + remainder = sum((len(examples_i) / n) * I(examples_i) + for (v, examples_i) in self.split_by(attr, examples)) + return I(examples) - remainder + + def split_by(self, attr, examples): + """Return a list of (val, examples) pairs for each val of attr.""" + return [(v, [e for e in examples if e[attr] == v]) for v in self.dataset.values[attr]] + + def predict(self, x): + return self.tree(x) + + +def information_content(values): + """Number of bits to represent the probability distribution in values.""" + probabilities = normalize(remove_all(0, values)) + return sum(-p * np.log2(p) for p in probabilities) + + +class DecisionListLearner: + """ + [Figure 18.11] + A decision list implemented as a list of (test, value) pairs. + """ + + def __init__(self, dataset): + self.predict.decision_list = self.decision_list_learning(set(dataset.examples)) + + def decision_list_learning(self, examples): + if not examples: + return [(True, False)] + t, o, examples_t = self.find_examples(examples) + if not t: + raise Exception + return [(t, o)] + self.decision_list_learning(examples - examples_t) + + def find_examples(self, examples): + """ + Find a set of examples that all have the same outcome under + some test. Return a tuple of the test, outcome, and examples. + """ + raise NotImplementedError + + def passes(self, example, test): + """Does the example pass the test?""" + raise NotImplementedError + + def predict(self, example): + """Predict the outcome for the first passing test.""" + for test, outcome in self.predict.decision_list: + if self.passes(example, test): + return outcome + + +class NearestNeighborLearner: + """k-NearestNeighbor: the k nearest neighbors vote.""" + + def __init__(self, dataset, k=1): + self.dataset = dataset + self.k = k + + def predict(self, example): + """Find the k closest items, and have them vote for the best.""" + best = heapq.nsmallest(self.k, ((self.dataset.distance(e, example), e) for e in self.dataset.examples)) + return mode(e[self.dataset.target] for (d, e) in best) + + +class SVC: + + def __init__(self, kernel=linear_kernel, C=1.0, verbose=False): + self.kernel = kernel + self.C = C # hyper-parameter + self.sv_idx, self.sv, self.sv_y = np.zeros(0), np.zeros(0), np.zeros(0) + self.alphas = np.zeros(0) + self.w = None + self.b = 0.0 # intercept + self.verbose = verbose + + def fit(self, X, y): + """ + Trains the model by solving a quadratic programming problem. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + # In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations) + self.solve_qp(X, y) + sv = self.alphas > 1e-5 + self.sv_idx = np.arange(len(self.alphas))[sv] + self.sv, self.sv_y, self.alphas = X[sv], y[sv], self.alphas[sv] + + if self.kernel == linear_kernel: + self.w = np.dot(self.alphas * self.sv_y, self.sv) + + for n in range(len(self.alphas)): + self.b += self.sv_y[n] + self.b -= np.sum(self.alphas * self.sv_y * self.K[self.sv_idx[n], sv]) + self.b /= len(self.alphas) + return self + + def solve_qp(self, X, y): + """ + Solves a quadratic programming problem. In QP formulation (dual): + m variables, 2m+1 constraints (1 equation, 2m inequations). + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + m = len(y) # m = n_samples + self.K = self.kernel(X) # gram matrix + P = self.K * np.outer(y, y) + q = -np.ones(m) + lb = np.zeros(m) # lower bounds + ub = np.ones(m) * self.C # upper bounds + A = y.astype(np.float64) # equality matrix + b = np.zeros(1) # equality vector + self.alphas = solve_qp(P, q, A=A, b=b, lb=lb, ub=ub, solver='cvxopt', + sym_proj=True, verbose=self.verbose) + + def predict_score(self, X): + """ + Predicts the score for a given example. + """ + if self.w is None: + return np.dot(self.alphas * self.sv_y, self.kernel(self.sv, X)) + self.b + return np.dot(X, self.w) + self.b + + def predict(self, X): + """ + Predicts the class of a given example. + """ + return np.sign(self.predict_score(X)) + + +class SVR: + + def __init__(self, kernel=linear_kernel, C=1.0, epsilon=0.1, verbose=False): + self.kernel = kernel + self.C = C # hyper-parameter + self.epsilon = epsilon # epsilon insensitive loss value + self.sv_idx, self.sv = np.zeros(0), np.zeros(0) + self.alphas_p, self.alphas_n = np.zeros(0), np.zeros(0) + self.w = None + self.b = 0.0 # intercept + self.verbose = verbose + + def fit(self, X, y): + """ + Trains the model by solving a quadratic programming problem. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + # In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations) + self.solve_qp(X, y) + + sv = np.logical_or(self.alphas_p > 1e-5, self.alphas_n > 1e-5) + self.sv_idx = np.arange(len(self.alphas_p))[sv] + self.sv, sv_y = X[sv], y[sv] + self.alphas_p, self.alphas_n = self.alphas_p[sv], self.alphas_n[sv] + + if self.kernel == linear_kernel: + self.w = np.dot(self.alphas_p - self.alphas_n, self.sv) + + for n in range(len(self.alphas_p)): + self.b += sv_y[n] + self.b -= np.sum((self.alphas_p - self.alphas_n) * self.K[self.sv_idx[n], sv]) + self.b -= self.epsilon + self.b /= len(self.alphas_p) + + return self + + def solve_qp(self, X, y): + """ + Solves a quadratic programming problem. In QP formulation (dual): + m variables, 2m+1 constraints (1 equation, 2m inequations). + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + """ + m = len(y) # m = n_samples + self.K = self.kernel(X) # gram matrix + P = np.vstack((np.hstack((self.K, -self.K)), # alphas_p, alphas_n + np.hstack((-self.K, self.K)))) # alphas_n, alphas_p + q = np.hstack((-y, y)) + self.epsilon + lb = np.zeros(2 * m) # lower bounds + ub = np.ones(2 * m) * self.C # upper bounds + A = np.hstack((np.ones(m), -np.ones(m))) # equality matrix + b = np.zeros(1) # equality vector + alphas = solve_qp(P, q, A=A, b=b, lb=lb, ub=ub, solver='cvxopt', + sym_proj=True, verbose=self.verbose) + self.alphas_p = alphas[:m] + self.alphas_n = alphas[m:] + + def predict(self, X): + if self.kernel != linear_kernel: + return np.dot(self.alphas_p - self.alphas_n, self.kernel(self.sv, X)) + self.b + return np.dot(X, self.w) + self.b + + +class MultiClassLearner: + + def __init__(self, clf, decision_function='ovr'): + self.clf = clf + self.decision_function = decision_function + self.n_class, self.classifiers = 0, [] + + def fit(self, X, y): + """ + Trains n_class or n_class * (n_class - 1) / 2 classifiers + according to the training method, ovr or ovo respectively. + :param X: array of size [n_samples, n_features] holding the training samples + :param y: array of size [n_samples] holding the class labels + :return: array of classifiers + """ + labels = np.unique(y) + self.n_class = len(labels) + if self.decision_function == 'ovr': # one-vs-rest method + for label in labels: + y1 = np.array(y) + y1[y1 != label] = -1.0 + y1[y1 == label] = 1.0 + self.clf.fit(X, y1) + self.classifiers.append(copy.deepcopy(self.clf)) + elif self.decision_function == 'ovo': # use one-vs-one method + n_labels = len(labels) + for i in range(n_labels): + for j in range(i + 1, n_labels): + neg_id, pos_id = y == labels[i], y == labels[j] + X1, y1 = np.r_[X[neg_id], X[pos_id]], np.r_[y[neg_id], y[pos_id]] + y1[y1 == labels[i]] = -1.0 + y1[y1 == labels[j]] = 1.0 + self.clf.fit(X1, y1) + self.classifiers.append(copy.deepcopy(self.clf)) + else: + return ValueError("Decision function must be either 'ovr' or 'ovo'.") + return self + + def predict(self, X): + """ + Predicts the class of a given example according to the training method. + """ + n_samples = len(X) + if self.decision_function == 'ovr': # one-vs-rest method + assert len(self.classifiers) == self.n_class + score = np.zeros((n_samples, self.n_class)) + for i in range(self.n_class): + clf = self.classifiers[i] + score[:, i] = clf.predict_score(X) + return np.argmax(score, axis=1) + elif self.decision_function == 'ovo': # use one-vs-one method + assert len(self.classifiers) == self.n_class * (self.n_class - 1) / 2 + vote = np.zeros((n_samples, self.n_class)) + clf_id = 0 + for i in range(self.n_class): + for j in range(i + 1, self.n_class): + res = self.classifiers[clf_id].predict(X) + vote[res < 0, i] += 1.0 # negative sample: class i + vote[res > 0, j] += 1.0 # positive sample: class j + clf_id += 1 + return np.argmax(vote, axis=1) + else: + return ValueError("Decision function must be either 'ovr' or 'ovo'.") + + +def LinearLearner(dataset, learning_rate=0.01, epochs=100): + """ + [Section 18.6.3] + Linear classifier with hard threshold. + """ + idx_i = dataset.inputs + idx_t = dataset.target + examples = dataset.examples + num_examples = len(examples) + + # X transpose + X_col = [dataset.values[i] for i in idx_i] # vertical columns of X + + # add dummy + ones = [1 for _ in range(len(examples))] + X_col = [ones] + X_col + + # initialize random weights + num_weights = len(idx_i) + 1 + w = random_weights(min_value=-0.5, max_value=0.5, num_weights=num_weights) + + for epoch in range(epochs): + err = [] + # pass over all examples + for example in examples: + x = [1] + example + y = np.dot(w, x) + t = example[idx_t] + err.append(t - y) + + # update weights + for i in range(len(w)): + w[i] = w[i] + learning_rate * (np.dot(err, X_col[i]) / num_examples) + + def predict(example): + x = [1] + example + return np.dot(w, x) + + return predict + + +def LogisticLinearLeaner(dataset, learning_rate=0.01, epochs=100): + """ + [Section 18.6.4] + Linear classifier with logistic regression. + """ + idx_i = dataset.inputs + idx_t = dataset.target + examples = dataset.examples + num_examples = len(examples) + + # X transpose + X_col = [dataset.values[i] for i in idx_i] # vertical columns of X + + # add dummy + ones = [1 for _ in range(len(examples))] + X_col = [ones] + X_col + + # initialize random weights + num_weights = len(idx_i) + 1 + w = random_weights(min_value=-0.5, max_value=0.5, num_weights=num_weights) + + for epoch in range(epochs): + err = [] + h = [] + # pass over all examples + for example in examples: + x = [1] + example + y = Sigmoid()(np.dot(w, x)) + h.append(Sigmoid().derivative(y)) + t = example[idx_t] + err.append(t - y) + + # update weights + for i in range(len(w)): + buffer = [x * y for x, y in zip(err, h)] + w[i] = w[i] + learning_rate * (np.dot(buffer, X_col[i]) / num_examples) + + def predict(example): + x = [1] + example + return Sigmoid()(np.dot(w, x)) + + return predict + + +class EnsembleLearner: + """Given a list of learning algorithms, have them vote.""" + + def __init__(self, learners): + self.learners = learners + + def train(self, dataset): + self.predictors = [learner(dataset) for learner in self.learners] + + def predict(self, example): + return mode(predictor.predict(example) for predictor in self.predictors) + + +def ada_boost(dataset, L, K): + """[Figure 18.34]""" + + examples, target = dataset.examples, dataset.target + n = len(examples) + eps = 1 / (2 * n) + w = [1 / n] * n + h, z = [], [] + for k in range(K): + h_k = L(dataset, w) + h.append(h_k) + error = sum(weight for example, weight in zip(examples, w) if example[target] != h_k.predict(example[:-1])) + # avoid divide-by-0 from either 0% or 100% error rates + error = np.clip(error, eps, 1 - eps) + for j, example in enumerate(examples): + if example[target] == h_k.predict(example[:-1]): + w[j] *= error / (1 - error) + w = normalize(w) + z.append(np.log((1 - error) / error)) + return weighted_majority(h, z) + + +class weighted_majority: + """Return a predictor that takes a weighted vote.""" + + def __init__(self, predictors, weights): + self.predictors = predictors + self.weights = weights + + def predict(self, example): + return weighted_mode((predictor.predict(example) for predictor in self.predictors), self.weights) + + +def weighted_mode(values, weights): + """ + Return the value with the greatest total weight. + >>> weighted_mode('abbaa', [1, 2, 3, 1, 2]) + 'b' + """ + totals = defaultdict(int) + for v, w in zip(values, weights): + totals[v] += w + return max(totals, key=totals.__getitem__) + + +class RandomForest: + """An ensemble of Decision Trees trained using bagging and feature bagging.""" + + def __init__(self, dataset, n=5): + self.dataset = dataset + self.n = n + self.predictors = [DecisionTreeLearner(DataSet(examples=self.data_bagging(), attrs=self.dataset.attrs, + attr_names=self.dataset.attr_names, target=self.dataset.target, + inputs=self.feature_bagging())) for _ in range(self.n)] + + def data_bagging(self, m=0): + """Sample m examples with replacement""" + n = len(self.dataset.examples) + return weighted_sample_with_replacement(m or n, self.dataset.examples, [1] * n) + + def feature_bagging(self, p=0.7): + """Feature bagging with probability p to retain an attribute""" + inputs = [i for i in self.dataset.inputs if probability(p)] + return inputs or self.dataset.inputs + + def predict(self, example): + return mode(predictor.predict(example) for predictor in self.predictors) + + +def WeightedLearner(unweighted_learner): + """ + [Page 749 footnote 14] + Given a learner that takes just an unweighted dataset, return + one that takes also a weight for each example. + """ + + def train(dataset, weights): + dataset = replicated_dataset(dataset, weights) + n_samples, n_features = len(dataset.examples), dataset.target + X, y = (np.array([x[:n_features] for x in dataset.examples]), + np.array([x[n_features] for x in dataset.examples])) + return unweighted_learner.fit(X, y) + + return train + + +def replicated_dataset(dataset, weights, n=None): + """Copy dataset, replicating each example in proportion to its weight.""" + n = n or len(dataset.examples) + result = copy.copy(dataset) + result.examples = weighted_replicate(dataset.examples, weights, n) + return result + + +def weighted_replicate(seq, weights, n): + """ + Return n selections from seq, with the count of each element of + seq proportional to the corresponding weight (filling in fractions + randomly). + >>> weighted_replicate('ABC', [1, 2, 1], 4) + ['A', 'B', 'B', 'C'] + """ + assert len(seq) == len(weights) + weights = normalize(weights) + wholes = [int(w * n) for w in weights] + fractions = [(w * n) % 1 for w in weights] + return (flatten([x] * nx for x, nx in zip(seq, wholes)) + + weighted_sample_with_replacement(n - sum(wholes), seq, fractions)) + + +# metrics + +def accuracy_score(y_pred, y_true): + assert y_pred.shape == y_true.shape + return np.mean(np.equal(y_pred, y_true)) + + +def r2_score(y_pred, y_true): + assert y_pred.shape == y_true.shape + return 1. - (np.sum(np.square(y_pred - y_true)) / # sum of square of residuals + np.sum(np.square(y_true - np.mean(y_true)))) # total sum of squares + + +# datasets + +orings = DataSet(name='orings', target='Distressed', attr_names='Rings Distressed Temp Pressure Flightnum') + +zoo = DataSet(name='zoo', target='type', exclude=['name'], + attr_names='name hair feathers eggs milk airborne aquatic predator toothed backbone ' + 'breathes venomous fins legs tail domestic catsize type') + +iris = DataSet(name='iris', target='class', attr_names='sepal-len sepal-width petal-len petal-width class') + + +def RestaurantDataSet(examples=None): + """ + [Figure 18.3] + Build a DataSet of Restaurant waiting examples. + """ + return DataSet(name='restaurant', target='Wait', examples=examples, + attr_names='Alternate Bar Fri/Sat Hungry Patrons Price Raining Reservation Type WaitEstimate Wait') + + +restaurant = RestaurantDataSet() + + +def T(attr_name, branches): + branches = {value: (child if isinstance(child, DecisionFork) else DecisionLeaf(child)) + for value, child in branches.items()} + return DecisionFork(restaurant.attr_num(attr_name), attr_name, print, branches) + + +""" +[Figure 18.2] +A decision tree for deciding whether to wait for a table at a hotel. +""" + +waiting_decision_tree = T('Patrons', + {'None': 'No', 'Some': 'Yes', + 'Full': T('WaitEstimate', + {'>60': 'No', '0-10': 'Yes', + '30-60': T('Alternate', + {'No': T('Reservation', + {'Yes': 'Yes', + 'No': T('Bar', {'No': 'No', + 'Yes': 'Yes'})}), + 'Yes': T('Fri/Sat', {'No': 'No', 'Yes': 'Yes'})}), + '10-30': T('Hungry', + {'No': 'Yes', + 'Yes': T('Alternate', + {'No': 'Yes', + 'Yes': T('Raining', + {'No': 'No', + 'Yes': 'Yes'})})})})}) + + +def SyntheticRestaurant(n=20): + """Generate a DataSet with n examples.""" + + def gen(): + example = list(map(random.choice, restaurant.values)) + example[restaurant.target] = waiting_decision_tree(example) + return example + + return RestaurantDataSet([gen() for _ in range(n)]) + + +def Majority(k, n): + """ + Return a DataSet with n k-bit examples of the majority problem: + k random bits followed by a 1 if more than half the bits are 1, else 0. + """ + examples = [] + for i in range(n): + bits = [random.choice([0, 1]) for _ in range(k)] + bits.append(int(sum(bits) > k / 2)) + examples.append(bits) + return DataSet(name='majority', examples=examples) + + +def Parity(k, n, name='parity'): + """ + Return a DataSet with n k-bit examples of the parity problem: + k random bits followed by a 1 if an odd number of bits are 1, else 0. + """ + examples = [] + for i in range(n): + bits = [random.choice([0, 1]) for _ in range(k)] + bits.append(sum(bits) % 2) + examples.append(bits) + return DataSet(name=name, examples=examples) + + +def Xor(n): + """Return a DataSet with n examples of 2-input xor.""" + return Parity(2, n, name='xor') + + +def ContinuousXor(n): + """2 inputs are chosen uniformly from (0.0 .. 2.0]; output is xor of ints.""" + examples = [] + for i in range(n): + x, y = [random.uniform(0.0, 2.0) for _ in '12'] + examples.append([x, y, x != y]) + return DataSet(name='continuous xor', examples=examples) + + +def compare(algorithms=None, datasets=None, k=10, trials=1): + """ + Compare various learners on various datasets using cross-validation. + Print results as a table. + """ + # default list of algorithms + algorithms = algorithms or [PluralityLearner, NaiveBayesLearner, NearestNeighborLearner, DecisionTreeLearner] + + # default list of datasets + datasets = datasets or [iris, orings, zoo, restaurant, SyntheticRestaurant(20), + Majority(7, 100), Parity(7, 100), Xor(100)] + + print_table([[a.__name__.replace('Learner', '')] + [cross_validation(a, d, k=k, trials=trials) for d in datasets] + for a in algorithms], header=[''] + [d.name[0:7] for d in datasets], numfmt='%.2f') diff --git a/learning_apps.ipynb b/learning_apps.ipynb index 6d5a27a45..dd45b11b5 100644 --- a/learning_apps.ipynb +++ b/learning_apps.ipynb @@ -16,6 +16,7 @@ "outputs": [], "source": [ "from learning import *\n", + "from probabilistic_learning import *\n", "from notebook import *" ] }, @@ -971,8 +972,17 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 2 -} +} \ No newline at end of file diff --git a/logic.py b/logic.py index 6aacc4f95..1624d55a5 100644 --- a/logic.py +++ b/logic.py @@ -1,4 +1,5 @@ -"""Representations and Inference for Logic (Chapters 7-9, 12) +""" +Representations and Inference for Logic. (Chapters 7-9, 12) Covers both Propositional and First-Order Logic. First we have four important data types: @@ -31,25 +32,23 @@ diff, simp Symbolic differentiation and simplification """ -from utils import ( - removeall, unique, first, argmax, probability, - isnumber, issequence, Expr, expr, subexpressions -) -from agents import Agent, Glitter, Bump, Stench, Breeze, Scream -from search import astar_search, PlanRoute - +import heapq import itertools import random -from collections import defaultdict +from collections import defaultdict, Counter -# ______________________________________________________________________________ +import networkx as nx + +from agents import Agent, Glitter, Bump, Stench, Breeze, Scream +from csp import parse_neighbors, UniversalDict +from search import astar_search, PlanRoute +from utils import remove_all, unique, first, probability, isnumber, issequence, Expr, expr, subexpressions, extend class KB: - """A knowledge base to which you can tell and ask sentences. To create a KB, first subclass this class and implement - tell, ask_generator, and retract. Why ask_generator instead of ask? + tell, ask_generator, and retract. Why ask_generator instead of ask? The book is a bit vague on what ask means -- For a Propositional Logic KB, ask(P & Q) returns True or False, but for an FOL KB, something like ask(Brother(x, y)) might return many substitutions @@ -58,7 +57,8 @@ class KB: first one or returns False.""" def __init__(self, sentence=None): - raise NotImplementedError + if sentence: + self.tell(sentence) def tell(self, sentence): """Add the sentence to the KB.""" @@ -81,9 +81,8 @@ class PropKB(KB): """A KB for propositional logic. Inefficient, with no indexing.""" def __init__(self, sentence=None): + super().__init__(sentence) self.clauses = [] - if sentence: - self.tell(sentence) def tell(self, sentence): """Add the sentence's clauses to the KB.""" @@ -106,28 +105,32 @@ def retract(self, sentence): if c in self.clauses: self.clauses.remove(c) + # ______________________________________________________________________________ -def KB_AgentProgram(KB): - """A generic logical knowledge-based agent program. [Figure 7.1]""" +def KBAgentProgram(kb): + """ + [Figure 7.1] + A generic logical knowledge-based agent program. + """ steps = itertools.count() def program(percept): t = next(steps) - KB.tell(make_percept_sentence(percept, t)) - action = KB.ask(make_action_query(t)) - KB.tell(make_action_sentence(action, t)) + kb.tell(make_percept_sentence(percept, t)) + action = kb.ask(make_action_query(t)) + kb.tell(make_action_sentence(action, t)) return action def make_percept_sentence(percept, t): - return Expr("Percept")(percept, t) + return Expr('Percept')(percept, t) def make_action_query(t): - return expr("ShouldDo(action, {})".format(t)) + return expr('ShouldDo(action, {})'.format(t)) def make_action_sentence(action, t): - return Expr("Did")(action[expr('action')], t) + return Expr('Did')(action[expr('action')], t) return program @@ -166,7 +169,7 @@ def variables(s): def is_definite_clause(s): """Returns True for exprs s of the form A & B & ... & C ==> D, - where all literals are positive. In clause form, this is + where all literals are positive. In clause form, this is ~A | ~B | ... | ~C | D, where exactly one clause is positive. >>> is_definite_clause(expr('Farmer(Mac)')) True @@ -175,8 +178,7 @@ def is_definite_clause(s): return True elif s.op == '==>': antecedent, consequent = s.args - return (is_symbol(consequent.op) and - all(is_symbol(arg.op) for arg in conjuncts(antecedent))) + return is_symbol(consequent.op) and all(is_symbol(arg.op) for arg in conjuncts(antecedent)) else: return False @@ -192,16 +194,18 @@ def parse_definite_clause(s): # Useful constant Exprs used in examples and code: -A, B, C, D, E, F, G, P, Q, x, y, z = map(Expr, 'ABCDEFGPQxyz') +A, B, C, D, E, F, G, P, Q, a, x, y, z, u = map(Expr, 'ABCDEFGPQaxyzu') # ______________________________________________________________________________ def tt_entails(kb, alpha): - """Does kb entail the sentence alpha? Use truth tables. For propositional - kb's and sentences. [Figure 7.10]. Note that the 'kb' should be an - Expr which is a conjunction of clauses. + """ + [Figure 7.10] + Does kb entail the sentence alpha? Use truth tables. For propositional + kb's and sentences. Note that the 'kb' should be an Expr which is a + conjunction of clauses. >>> tt_entails(expr('P & Q'), expr('Q')) True """ @@ -317,7 +321,8 @@ def pl_true(exp, model={}): elif op == '^': # xor or 'not equivalent' return pt != qt else: - raise ValueError("illegal operator in logic expression" + str(exp)) + raise ValueError('Illegal operator in logic expression' + str(exp)) + # ______________________________________________________________________________ @@ -325,8 +330,10 @@ def pl_true(exp, model={}): def to_cnf(s): - """Convert a propositional logical sentence to conjunctive normal form. - That is, to the form ((A | ~B | ...) & (B | C | ...) & ...) [p. 253] + """ + [Page 253] + Convert a propositional logical sentence to conjunctive normal form. + That is, to the form ((A | ~B | ...) & (B | C | ...) & ...) >>> to_cnf('~(B | C)') (~B & ~C) """ @@ -368,6 +375,7 @@ def move_not_inwards(s): if s.op == '~': def NOT(b): return move_not_inwards(~b) + a = s.args[0] if a.op == '~': return move_not_inwards(a.args[0]) # ~~A ==> A @@ -445,6 +453,7 @@ def collect(subargs): collect(arg.args) else: result.append(arg) + collect(args) return result @@ -468,20 +477,23 @@ def disjuncts(s): """ return dissociate('|', [s]) + # ______________________________________________________________________________ -def pl_resolution(KB, alpha): - """Propositional-logic resolution: say if alpha follows from KB. [Figure 7.12] +def pl_resolution(kb, alpha): + """ + [Figure 7.12] + Propositional-logic resolution: say if alpha follows from KB. >>> pl_resolution(horn_clauses_KB, A) True """ - clauses = KB.clauses + conjuncts(to_cnf(~alpha)) + clauses = kb.clauses + conjuncts(to_cnf(~alpha)) new = set() while True: n = len(clauses) pairs = [(clauses[i], clauses[j]) - for i in range(n) for j in range(i+1, n)] + for i in range(n) for j in range(i + 1, n)] for (ci, cj) in pairs: resolvents = pl_resolve(ci, cj) if False in resolvents: @@ -500,11 +512,10 @@ def pl_resolve(ci, cj): for di in disjuncts(ci): for dj in disjuncts(cj): if di == ~dj or ~di == dj: - dnew = unique(removeall(di, disjuncts(ci)) + - removeall(dj, disjuncts(cj))) - clauses.append(associate('|', dnew)) + clauses.append(associate('|', unique(remove_all(di, disjuncts(ci)) + remove_all(dj, disjuncts(cj))))) return clauses + # ______________________________________________________________________________ @@ -527,59 +538,176 @@ def retract(self, sentence): def clauses_with_premise(self, p): """Return a list of the clauses in KB that have p in their premise. This could be cached away for O(1) speed, but we'll recompute it.""" - return [c for c in self.clauses - if c.op == '==>' and p in conjuncts(c.args[0])] + return [c for c in self.clauses if c.op == '==>' and p in conjuncts(c.args[0])] -def pl_fc_entails(KB, q): - """Use forward chaining to see if a PropDefiniteKB entails symbol q. +def pl_fc_entails(kb, q): + """ [Figure 7.15] + Use forward chaining to see if a PropDefiniteKB entails symbol q. >>> pl_fc_entails(horn_clauses_KB, expr('Q')) True """ - count = {c: len(conjuncts(c.args[0])) - for c in KB.clauses - if c.op == '==>'} + count = {c: len(conjuncts(c.args[0])) for c in kb.clauses if c.op == '==>'} inferred = defaultdict(bool) - agenda = [s for s in KB.clauses if is_prop_symbol(s.op)] + agenda = [s for s in kb.clauses if is_prop_symbol(s.op)] while agenda: p = agenda.pop() if p == q: return True if not inferred[p]: inferred[p] = True - for c in KB.clauses_with_premise(p): + for c in kb.clauses_with_premise(p): count[c] -= 1 if count[c] == 0: agenda.append(c.args[1]) return False -""" [Figure 7.13] +""" +[Figure 7.13] Simple inference in a wumpus world example """ -wumpus_world_inference = expr("(B11 <=> (P12 | P21)) & ~B11") +wumpus_world_inference = expr('(B11 <=> (P12 | P21)) & ~B11') - -""" [Figure 7.16] +""" +[Figure 7.16] Propositional Logic Forward Chaining example """ horn_clauses_KB = PropDefiniteKB() -for s in "P==>Q; (L&M)==>P; (B&L)==>M; (A&P)==>L; (A&B)==>L; A;B".split(';'): - horn_clauses_KB.tell(expr(s)) +for clause in ['P ==> Q', + '(L & M) ==> P', + '(B & L) ==> M', + '(A & P) ==> L', + '(A & B) ==> L', + 'A', 'B']: + horn_clauses_KB.tell(expr(clause)) """ Definite clauses KB example """ definite_clauses_KB = PropDefiniteKB() -for clause in ['(B & F)==>E', '(A & E & F)==>G', '(B & C)==>F', '(A & B)==>D', '(E & F)==>H', '(H & I)==>J', 'A', 'B', 'C']: +for clause in ['(B & F) ==> E', + '(A & E & F) ==> G', + '(B & C) ==> F', + '(A & B) ==> D', + '(E & F) ==> H', + '(H & I) ==>J', + 'A', 'B', 'C']: definite_clauses_KB.tell(expr(clause)) + +# ______________________________________________________________________________ +# Heuristics for SAT Solvers + + +def no_branching_heuristic(symbols, clauses): + return first(symbols), True + + +def min_clauses(clauses): + min_len = min(map(lambda c: len(c.args), clauses), default=2) + return filter(lambda c: len(c.args) == (min_len if min_len > 1 else 2), clauses) + + +def moms(symbols, clauses): + """ + MOMS (Maximum Occurrence in clauses of Minimum Size) heuristic + Returns the literal with the most occurrences in all clauses of minimum size + """ + scores = Counter(l for c in min_clauses(clauses) for l in prop_symbols(c)) + return max(symbols, key=lambda symbol: scores[symbol]), True + + +def momsf(symbols, clauses, k=0): + """ + MOMS alternative heuristic + If f(x) the number of occurrences of the variable x in clauses with minimum size, + we choose the variable maximizing [f(x) + f(-x)] * 2^k + f(x) * f(-x) + Returns x if f(x) >= f(-x) otherwise -x + """ + scores = Counter(l for c in min_clauses(clauses) for l in disjuncts(c)) + P = max(symbols, + key=lambda symbol: (scores[symbol] + scores[~symbol]) * pow(2, k) + scores[symbol] * scores[~symbol]) + return P, True if scores[P] >= scores[~P] else False + + +def posit(symbols, clauses): + """ + Freeman's POSIT version of MOMs + Counts the positive x and negative x for each variable x in clauses with minimum size + Returns x if f(x) >= f(-x) otherwise -x + """ + scores = Counter(l for c in min_clauses(clauses) for l in disjuncts(c)) + P = max(symbols, key=lambda symbol: scores[symbol] + scores[~symbol]) + return P, True if scores[P] >= scores[~P] else False + + +def zm(symbols, clauses): + """ + Zabih and McAllester's version of MOMs + Counts the negative occurrences only of each variable x in clauses with minimum size + """ + scores = Counter(l for c in min_clauses(clauses) for l in disjuncts(c) if l.op == '~') + return max(symbols, key=lambda symbol: scores[~symbol]), True + + +def dlis(symbols, clauses): + """ + DLIS (Dynamic Largest Individual Sum) heuristic + Choose the variable and value that satisfies the maximum number of unsatisfied clauses + Like DLCS but we only consider the literal (thus Cp and Cn are individual) + """ + scores = Counter(l for c in clauses for l in disjuncts(c)) + P = max(symbols, key=lambda symbol: scores[symbol]) + return P, True if scores[P] >= scores[~P] else False + + +def dlcs(symbols, clauses): + """ + DLCS (Dynamic Largest Combined Sum) heuristic + Cp the number of clauses containing literal x + Cn the number of clauses containing literal -x + Here we select the variable maximizing Cp + Cn + Returns x if Cp >= Cn otherwise -x + """ + scores = Counter(l for c in clauses for l in disjuncts(c)) + P = max(symbols, key=lambda symbol: scores[symbol] + scores[~symbol]) + return P, True if scores[P] >= scores[~P] else False + + +def jw(symbols, clauses): + """ + Jeroslow-Wang heuristic + For each literal compute J(l) = \sum{l in clause c} 2^{-|c|} + Return the literal maximizing J + """ + scores = Counter() + for c in clauses: + for l in prop_symbols(c): + scores[l] += pow(2, -len(c.args)) + return max(symbols, key=lambda symbol: scores[symbol]), True + + +def jw2(symbols, clauses): + """ + Two Sided Jeroslow-Wang heuristic + Compute J(l) also counts the negation of l = J(x) + J(-x) + Returns x if J(x) >= J(-x) otherwise -x + """ + scores = Counter() + for c in clauses: + for l in disjuncts(c): + scores[l] += pow(2, -len(c.args)) + P = max(symbols, key=lambda symbol: scores[symbol] + scores[~symbol]) + return P, True if scores[P] >= scores[~P] else False + + # ______________________________________________________________________________ # DPLL-Satisfiable [Figure 7.17] -def dpll_satisfiable(s): +def dpll_satisfiable(s, branching_heuristic=no_branching_heuristic): """Check satisfiability of a propositional sentence. This differs from the book code in two ways: (1) it returns a model rather than True when it succeeds; this is more useful. (2) The @@ -588,33 +716,29 @@ def dpll_satisfiable(s): >>> dpll_satisfiable(A |'<=>'| B) == {A: True, B: True} True """ - clauses = conjuncts(to_cnf(s)) - symbols = list(prop_symbols(s)) - return dpll(clauses, symbols, {}) + return dpll(conjuncts(to_cnf(s)), prop_symbols(s), {}, branching_heuristic) -def dpll(clauses, symbols, model): +def dpll(clauses, symbols, model, branching_heuristic=no_branching_heuristic): """See if the clauses are true in a partial model.""" unknown_clauses = [] # clauses with an unknown truth value for c in clauses: val = pl_true(c, model) if val is False: return False - if val is not True: + if val is None: unknown_clauses.append(c) if not unknown_clauses: return model P, value = find_pure_symbol(symbols, unknown_clauses) if P: - return dpll(clauses, removeall(P, symbols), extend(model, P, value)) + return dpll(clauses, remove_all(P, symbols), extend(model, P, value), branching_heuristic) P, value = find_unit_clause(clauses, model) if P: - return dpll(clauses, removeall(P, symbols), extend(model, P, value)) - if not symbols: - raise TypeError("Argument should be of the type Expr.") - P, symbols = symbols[0], symbols[1:] - return (dpll(clauses, symbols, extend(model, P, True)) or - dpll(clauses, symbols, extend(model, P, False))) + return dpll(clauses, remove_all(P, symbols), extend(model, P, value), branching_heuristic) + P, value = branching_heuristic(symbols, unknown_clauses) + return (dpll(clauses, remove_all(P, symbols), extend(model, P, value), branching_heuristic) or + dpll(clauses, remove_all(P, symbols), extend(model, P, not value), branching_heuristic)) def find_pure_symbol(symbols, clauses): @@ -665,7 +789,7 @@ def unit_clause_assign(clause, model): if model[sym] == positive: return None, None # clause already True elif P: - return None, None # more than 1 unbound variable + return None, None # more than 1 unbound variable else: P, value = sym, positive return P, value @@ -684,6 +808,274 @@ def inspect_literal(literal): else: return literal, True + +# ______________________________________________________________________________ +# CDCL - Conflict-Driven Clause Learning with 1UIP Learning Scheme, +# 2WL Lazy Data Structure, VSIDS Branching Heuristic & Restarts + + +def no_restart(conflicts, restarts, queue_lbd, sum_lbd): + return False + + +def luby(conflicts, restarts, queue_lbd, sum_lbd, unit=512): + # in the state-of-art tested with unit value 1, 2, 4, 6, 8, 12, 16, 32, 64, 128, 256 and 512 + def _luby(i): + k = 1 + while True: + if i == (1 << k) - 1: + return 1 << (k - 1) + elif (1 << (k - 1)) <= i < (1 << k) - 1: + return _luby(i - (1 << (k - 1)) + 1) + k += 1 + + return unit * _luby(restarts) == len(queue_lbd) + + +def glucose(conflicts, restarts, queue_lbd, sum_lbd, x=100, k=0.7): + # in the state-of-art tested with (x, k) as (50, 0.8) and (100, 0.7) + # if there were at least x conflicts since the last restart, and then the average LBD of the last + # x learnt clauses was at least k times higher than the average LBD of all learnt clauses + return len(queue_lbd) >= x and sum(queue_lbd) / len(queue_lbd) * k > sum_lbd / conflicts + + +def cdcl_satisfiable(s, vsids_decay=0.95, restart_strategy=no_restart): + """ + >>> cdcl_satisfiable(A |'<=>'| B) == {A: True, B: True} + True + """ + clauses = TwoWLClauseDatabase(conjuncts(to_cnf(s))) + symbols = prop_symbols(s) + scores = Counter() + G = nx.DiGraph() + model = {} + dl = 0 + conflicts = 0 + restarts = 1 + sum_lbd = 0 + queue_lbd = [] + while True: + conflict = unit_propagation(clauses, symbols, model, G, dl) + if conflict: + if dl == 0: + return False + conflicts += 1 + dl, learn, lbd = conflict_analysis(G, dl) + queue_lbd.append(lbd) + sum_lbd += lbd + backjump(symbols, model, G, dl) + clauses.add(learn, model) + scores.update(l for l in disjuncts(learn)) + for symbol in scores: + scores[symbol] *= vsids_decay + if restart_strategy(conflicts, restarts, queue_lbd, sum_lbd): + backjump(symbols, model, G) + queue_lbd.clear() + restarts += 1 + else: + if not symbols: + return model + dl += 1 + assign_decision_literal(symbols, model, scores, G, dl) + + +def assign_decision_literal(symbols, model, scores, G, dl): + P = max(symbols, key=lambda symbol: scores[symbol] + scores[~symbol]) + value = True if scores[P] >= scores[~P] else False + symbols.remove(P) + model[P] = value + G.add_node(P, val=value, dl=dl) + + +def unit_propagation(clauses, symbols, model, G, dl): + def check(c): + if not model or clauses.get_first_watched(c) == clauses.get_second_watched(c): + return True + w1, _ = inspect_literal(clauses.get_first_watched(c)) + if w1 in model: + return c in (clauses.get_neg_watched(w1) if model[w1] else clauses.get_pos_watched(w1)) + w2, _ = inspect_literal(clauses.get_second_watched(c)) + if w2 in model: + return c in (clauses.get_neg_watched(w2) if model[w2] else clauses.get_pos_watched(w2)) + + def unit_clause(watching): + w, p = inspect_literal(watching) + G.add_node(w, val=p, dl=dl) + G.add_edges_from(zip(prop_symbols(c) - {w}, itertools.cycle([w])), antecedent=c) + symbols.remove(w) + model[w] = p + + def conflict_clause(c): + G.add_edges_from(zip(prop_symbols(c), itertools.cycle('K')), antecedent=c) + + while True: + bcp = False + for c in filter(check, clauses.get_clauses()): + # we need only visit each clause when one of its two watched literals is assigned to 0 because, until + # this happens, we can guarantee that there cannot be more than n-2 literals in the clause assigned to 0 + first_watched = pl_true(clauses.get_first_watched(c), model) + second_watched = pl_true(clauses.get_second_watched(c), model) + if first_watched is None and clauses.get_first_watched(c) == clauses.get_second_watched(c): + unit_clause(clauses.get_first_watched(c)) + bcp = True + break + elif first_watched is False and second_watched is not True: + if clauses.update_second_watched(c, model): + bcp = True + else: + # if the only literal with a non-zero value is the other watched literal then + if second_watched is None: # if it is free, then the clause is a unit clause + unit_clause(clauses.get_second_watched(c)) + bcp = True + break + else: # else (it is False) the clause is a conflict clause + conflict_clause(c) + return True + elif second_watched is False and first_watched is not True: + if clauses.update_first_watched(c, model): + bcp = True + else: + # if the only literal with a non-zero value is the other watched literal then + if first_watched is None: # if it is free, then the clause is a unit clause + unit_clause(clauses.get_first_watched(c)) + bcp = True + break + else: # else (it is False) the clause is a conflict clause + conflict_clause(c) + return True + if not bcp: + return False + + +def conflict_analysis(G, dl): + conflict_clause = next(G[p]['K']['antecedent'] for p in G.pred['K']) + P = next(node for node in G.nodes() - 'K' if G.nodes[node]['dl'] == dl and G.in_degree(node) == 0) + first_uip = nx.immediate_dominators(G, P)['K'] + G.remove_node('K') + conflict_side = nx.descendants(G, first_uip) + while True: + for l in prop_symbols(conflict_clause).intersection(conflict_side): + antecedent = next(G[p][l]['antecedent'] for p in G.pred[l]) + conflict_clause = pl_binary_resolution(conflict_clause, antecedent) + # the literal block distance is calculated by taking the decision levels from variables of all + # literals in the clause, and counting how many different decision levels were in this set + lbd = [G.nodes[l]['dl'] for l in prop_symbols(conflict_clause)] + if lbd.count(dl) == 1 and first_uip in prop_symbols(conflict_clause): + return 0 if len(lbd) == 1 else heapq.nlargest(2, lbd)[-1], conflict_clause, len(set(lbd)) + + +def pl_binary_resolution(ci, cj): + for di in disjuncts(ci): + for dj in disjuncts(cj): + if di == ~dj or ~di == dj: + return pl_binary_resolution(associate('|', remove_all(di, disjuncts(ci))), + associate('|', remove_all(dj, disjuncts(cj)))) + return associate('|', unique(disjuncts(ci) + disjuncts(cj))) + + +def backjump(symbols, model, G, dl=0): + delete = {node for node in G.nodes() if G.nodes[node]['dl'] > dl} + G.remove_nodes_from(delete) + for node in delete: + del model[node] + symbols |= delete + + +class TwoWLClauseDatabase: + + def __init__(self, clauses): + self.__twl = {} + self.__watch_list = defaultdict(lambda: [set(), set()]) + for c in clauses: + self.add(c, None) + + def get_clauses(self): + return self.__twl.keys() + + def set_first_watched(self, clause, new_watching): + if len(clause.args) > 2: + self.__twl[clause][0] = new_watching + + def set_second_watched(self, clause, new_watching): + if len(clause.args) > 2: + self.__twl[clause][1] = new_watching + + def get_first_watched(self, clause): + if len(clause.args) == 2: + return clause.args[0] + if len(clause.args) > 2: + return self.__twl[clause][0] + return clause + + def get_second_watched(self, clause): + if len(clause.args) == 2: + return clause.args[-1] + if len(clause.args) > 2: + return self.__twl[clause][1] + return clause + + def get_pos_watched(self, l): + return self.__watch_list[l][0] + + def get_neg_watched(self, l): + return self.__watch_list[l][1] + + def add(self, clause, model): + self.__twl[clause] = self.__assign_watching_literals(clause, model) + w1, p1 = inspect_literal(self.get_first_watched(clause)) + w2, p2 = inspect_literal(self.get_second_watched(clause)) + self.__watch_list[w1][0].add(clause) if p1 else self.__watch_list[w1][1].add(clause) + if w1 != w2: + self.__watch_list[w2][0].add(clause) if p2 else self.__watch_list[w2][1].add(clause) + + def remove(self, clause): + w1, p1 = inspect_literal(self.get_first_watched(clause)) + w2, p2 = inspect_literal(self.get_second_watched(clause)) + del self.__twl[clause] + self.__watch_list[w1][0].discard(clause) if p1 else self.__watch_list[w1][1].discard(clause) + if w1 != w2: + self.__watch_list[w2][0].discard(clause) if p2 else self.__watch_list[w2][1].discard(clause) + + def update_first_watched(self, clause, model): + # if a non-zero literal different from the other watched literal is found + found, new_watching = self.__find_new_watching_literal(clause, self.get_first_watched(clause), model) + if found: # then it will replace the watched literal + w, p = inspect_literal(self.get_second_watched(clause)) + self.__watch_list[w][0].remove(clause) if p else self.__watch_list[w][1].remove(clause) + self.set_second_watched(clause, new_watching) + w, p = inspect_literal(new_watching) + self.__watch_list[w][0].add(clause) if p else self.__watch_list[w][1].add(clause) + return True + + def update_second_watched(self, clause, model): + # if a non-zero literal different from the other watched literal is found + found, new_watching = self.__find_new_watching_literal(clause, self.get_second_watched(clause), model) + if found: # then it will replace the watched literal + w, p = inspect_literal(self.get_first_watched(clause)) + self.__watch_list[w][0].remove(clause) if p else self.__watch_list[w][1].remove(clause) + self.set_first_watched(clause, new_watching) + w, p = inspect_literal(new_watching) + self.__watch_list[w][0].add(clause) if p else self.__watch_list[w][1].add(clause) + return True + + def __find_new_watching_literal(self, clause, other_watched, model): + # if a non-zero literal different from the other watched literal is found + if len(clause.args) > 2: + for l in disjuncts(clause): + if l != other_watched and pl_true(l, model) is not False: + # then it is returned + return True, l + return False, None + + def __assign_watching_literals(self, clause, model=None): + if len(clause.args) > 2: + if model is None or not model: + return [clause.args[0], clause.args[-1]] + else: + return [next(l for l in disjuncts(clause) if pl_true(l, model) is None), + next(l for l in disjuncts(clause) if pl_true(l, model) is False)] + + # ______________________________________________________________________________ # Walk-SAT [Figure 7.18] @@ -714,95 +1106,169 @@ def sat_count(sym): count = len([clause for clause in clauses if pl_true(clause, model)]) model[sym] = not model[sym] return count - sym = argmax(prop_symbols(clause), key=sat_count) + + sym = max(prop_symbols(clause), key=sat_count) model[sym] = not model[sym] # If no solution is found within the flip limit, we return failure return None + +# ______________________________________________________________________________ +# Map Coloring SAT Problems + + +def MapColoringSAT(colors, neighbors): + """Make a SAT for the problem of coloring a map with different colors + for any two adjacent regions. Arguments are a list of colors, and a + dict of {region: [neighbor,...]} entries. This dict may also be + specified as a string of the form defined by parse_neighbors.""" + if isinstance(neighbors, str): + neighbors = parse_neighbors(neighbors) + colors = UniversalDict(colors) + clauses = [] + for state in neighbors.keys(): + clause = [expr(state + '_' + c) for c in colors[state]] + clauses.append(clause) + for t in itertools.combinations(clause, 2): + clauses.append([~t[0], ~t[1]]) + visited = set() + adj = set(neighbors[state]) - visited + visited.add(state) + for n_state in adj: + for col in colors[n_state]: + clauses.append([expr('~' + state + '_' + col), expr('~' + n_state + '_' + col)]) + return associate('&', map(lambda c: associate('|', c), clauses)) + + +australia_sat = MapColoringSAT(list('RGB'), """SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: """) + +france_sat = MapColoringSAT(list('RGBY'), + """AL: LO FC; AQ: MP LI PC; AU: LI CE BO RA LR MP; BO: CE IF CA FC RA + AU; BR: NB PL; CA: IF PI LO FC BO; CE: PL NB NH IF BO AU LI PC; FC: BO + CA LO AL RA; IF: NH PI CA BO CE; LI: PC CE AU MP AQ; LO: CA AL FC; LR: + MP AU RA PA; MP: AQ LI AU LR; NB: NH CE PL BR; NH: PI IF CE NB; NO: + PI; PA: LR RA; PC: PL CE LI AQ; PI: NH NO CA IF; PL: BR NB CE PC; RA: + AU BO FC PA LR""") + +usa_sat = MapColoringSAT(list('RGBY'), + """WA: OR ID; OR: ID NV CA; CA: NV AZ; NV: ID UT AZ; ID: MT WY UT; + UT: WY CO AZ; MT: ND SD WY; WY: SD NE CO; CO: NE KA OK NM; NM: OK TX AZ; + ND: MN SD; SD: MN IA NE; NE: IA MO KA; KA: MO OK; OK: MO AR TX; + TX: AR LA; MN: WI IA; IA: WI IL MO; MO: IL KY TN AR; AR: MS TN LA; + LA: MS; WI: MI IL; IL: IN KY; IN: OH KY; MS: TN AL; AL: TN GA FL; + MI: OH IN; OH: PA WV KY; KY: WV VA TN; TN: VA NC GA; GA: NC SC FL; + PA: NY NJ DE MD WV; WV: MD VA; VA: MD DC NC; NC: SC; NY: VT MA CT NJ; + NJ: DE; DE: MD; MD: DC; VT: NH MA; MA: NH RI CT; CT: RI; ME: NH; + HI: ; AK: """) + + # ______________________________________________________________________________ # Expr functions for WumpusKB and HybridWumpusAgent -def facing_east (time): +def facing_east(time): return Expr('FacingEast', time) -def facing_west (time): + +def facing_west(time): return Expr('FacingWest', time) -def facing_north (time): + +def facing_north(time): return Expr('FacingNorth', time) -def facing_south (time): + +def facing_south(time): return Expr('FacingSouth', time) -def wumpus (x, y): + +def wumpus(x, y): return Expr('W', x, y) + def pit(x, y): return Expr('P', x, y) + def breeze(x, y): return Expr('B', x, y) + def stench(x, y): return Expr('S', x, y) + def wumpus_alive(time): return Expr('WumpusAlive', time) + def have_arrow(time): return Expr('HaveArrow', time) + def percept_stench(time): return Expr('Stench', time) + def percept_breeze(time): return Expr('Breeze', time) + def percept_glitter(time): return Expr('Glitter', time) + def percept_bump(time): return Expr('Bump', time) + def percept_scream(time): return Expr('Scream', time) + def move_forward(time): return Expr('Forward', time) + def shoot(time): return Expr('Shoot', time) + def turn_left(time): return Expr('TurnLeft', time) + def turn_right(time): return Expr('TurnRight', time) + def ok_to_move(x, y, time): return Expr('OK', x, y, time) -def location(x, y, time = None): + +def location(x, y, time=None): if time is None: return Expr('L', x, y) else: return Expr('L', x, y, time) + # Symbols def implies(lhs, rhs): return Expr('==>', lhs, rhs) + def equiv(lhs, rhs): return Expr('<=>', lhs, rhs) + # Helper Function def new_disjunction(sentences): t = sentences[0] - for i in range(1,len(sentences)): + for i in range(1, len(sentences)): t |= sentences[i] return t @@ -812,62 +1278,59 @@ def new_disjunction(sentences): class WumpusKB(PropKB): """ - Create a Knowledge Base that contains the atemporal "Wumpus physics" and temporal rules with time zero. + Create a Knowledge Base that contains the a temporal "Wumpus physics" and temporal rules with time zero. """ - def __init__(self,dimrow): + def __init__(self, dimrow): super().__init__() self.dimrow = dimrow - self.tell( ~wumpus(1, 1) ) - self.tell( ~pit(1, 1) ) + self.tell(~wumpus(1, 1)) + self.tell(~pit(1, 1)) - for y in range(1, dimrow+1): - for x in range(1, dimrow+1): + for y in range(1, dimrow + 1): + for x in range(1, dimrow + 1): pits_in = list() wumpus_in = list() - if x > 1: # West room exists + if x > 1: # West room exists pits_in.append(pit(x - 1, y)) wumpus_in.append(wumpus(x - 1, y)) - if y < dimrow: # North room exists + if y < dimrow: # North room exists pits_in.append(pit(x, y + 1)) wumpus_in.append(wumpus(x, y + 1)) - if x < dimrow: # East room exists + if x < dimrow: # East room exists pits_in.append(pit(x + 1, y)) wumpus_in.append(wumpus(x + 1, y)) - if y > 1: # South room exists + if y > 1: # South room exists pits_in.append(pit(x, y - 1)) wumpus_in.append(wumpus(x, y - 1)) self.tell(equiv(breeze(x, y), new_disjunction(pits_in))) self.tell(equiv(stench(x, y), new_disjunction(wumpus_in))) - - ## Rule that describes existence of at least one Wumpus + # Rule that describes existence of at least one Wumpus wumpus_at_least = list() - for x in range(1, dimrow+1): + for x in range(1, dimrow + 1): for y in range(1, dimrow + 1): wumpus_at_least.append(wumpus(x, y)) self.tell(new_disjunction(wumpus_at_least)) - - ## Rule that describes existence of at most one Wumpus - for i in range(1, dimrow+1): - for j in range(1, dimrow+1): - for u in range(1, dimrow+1): - for v in range(1, dimrow+1): - if i!=u or j!=v: + # Rule that describes existence of at most one Wumpus + for i in range(1, dimrow + 1): + for j in range(1, dimrow + 1): + for u in range(1, dimrow + 1): + for v in range(1, dimrow + 1): + if i != u or j != v: self.tell(~wumpus(i, j) | ~wumpus(u, v)) - - ## Temporal rules at time zero + # Temporal rules at time zero self.tell(location(1, 1, 0)) - for i in range(1, dimrow+1): + for i in range(1, dimrow + 1): for j in range(1, dimrow + 1): self.tell(implies(location(i, j, 0), equiv(percept_breeze(0), breeze(i, j)))) self.tell(implies(location(i, j, 0), equiv(percept_stench(0), stench(i, j)))) @@ -881,7 +1344,6 @@ def __init__(self,dimrow): self.tell(~facing_south(0)) self.tell(~facing_west(0)) - def make_action_sentence(self, action, time): actions = [move_forward(time), shoot(time), turn_left(time), turn_right(time)] @@ -895,7 +1357,7 @@ def make_percept_sentence(self, percept, time): # Glitter, Bump, Stench, Breeze, Scream flags = [0, 0, 0, 0, 0] - ## Things perceived + # Things perceived if isinstance(percept, Glitter): flags[0] = 1 self.tell(percept_glitter(time)) @@ -912,7 +1374,7 @@ def make_percept_sentence(self, percept, time): flags[4] = 1 self.tell(percept_scream(time)) - ## Things not perceived + # Things not perceived for i in range(len(flags)): if flags[i] == 0: if i == 0: @@ -926,45 +1388,34 @@ def make_percept_sentence(self, percept, time): elif i == 4: self.tell(~percept_scream(time)) - def add_temporal_sentences(self, time): if time == 0: return t = time - 1 - ## current location rules - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + # current location rules + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): self.tell(implies(location(i, j, time), equiv(percept_breeze(time), breeze(i, j)))) self.tell(implies(location(i, j, time), equiv(percept_stench(time), stench(i, j)))) - s = list() - - s.append( - equiv( - location(i, j, time), location(i, j, time) & ~move_forward(time) | percept_bump(time))) - + s.append(equiv(location(i, j, time), location(i, j, time) & ~move_forward(time) | percept_bump(time))) if i != 1: s.append(location(i - 1, j, t) & facing_east(t) & move_forward(t)) - if i != self.dimrow: s.append(location(i + 1, j, t) & facing_west(t) & move_forward(t)) - if j != 1: s.append(location(i, j - 1, t) & facing_north(t) & move_forward(t)) - if j != self.dimrow: s.append(location(i, j + 1, t) & facing_south(t) & move_forward(t)) - ## add sentence about location i,j + # add sentence about location i,j self.tell(new_disjunction(s)) - ## add sentence about safety of location i,j - self.tell( - equiv(ok_to_move(i, j, time), ~pit(i, j) & ~wumpus(i, j) & wumpus_alive(time)) - ) + # add sentence about safety of location i,j + self.tell(equiv(ok_to_move(i, j, time), ~pit(i, j) & ~wumpus(i, j) & wumpus_alive(time))) - ## Rules about current orientation + # Rules about current orientation a = facing_north(t) & turn_right(t) b = facing_south(t) & turn_left(t) @@ -990,16 +1441,15 @@ def add_temporal_sentences(self, time): s = equiv(facing_south(time), a | b | c) self.tell(s) - ## Rules about last action + # Rules about last action self.tell(equiv(move_forward(t), ~turn_right(t) & ~turn_left(t))) - ##Rule about the arrow + # Rule about the arrow self.tell(equiv(have_arrow(time), have_arrow(t) & ~shoot(t))) - ##Rule about Wumpus (dead or alive) + # Rule about Wumpus (dead or alive) self.tell(equiv(wumpus_alive(time), wumpus_alive(t) & ~percept_scream(time))) - def ask_if_true(self, query): return pl_resolution(self, query) @@ -1007,13 +1457,12 @@ def ask_if_true(self, query): # ______________________________________________________________________________ -class WumpusPosition(): +class WumpusPosition: def __init__(self, x, y, orientation): self.X = x self.Y = y self.orientation = orientation - def get_location(self): return self.X, self.Y @@ -1028,19 +1477,22 @@ def set_orientation(self, orientation): self.orientation = orientation def __eq__(self, other): - if other.get_location() == self.get_location() and \ - other.get_orientation()==self.get_orientation(): + if other.get_location() == self.get_location() and other.get_orientation() == self.get_orientation(): return True else: return False + # ______________________________________________________________________________ class HybridWumpusAgent(Agent): - """An agent for the wumpus world that does logical inference. [Figure 7.20]""" + """ + [Figure 7.20] + An agent for the wumpus world that does logical inference. + """ - def __init__(self,dimentions): + def __init__(self, dimentions): self.dimrow = dimentions self.kb = WumpusKB(self.dimrow) self.t = 0 @@ -1048,15 +1500,14 @@ def __init__(self,dimentions): self.current_position = WumpusPosition(1, 1, 'UP') super().__init__(self.execute) - def execute(self, percept): self.kb.make_percept_sentence(percept, self.t) self.kb.add_temporal_sentences(self.t) temp = list() - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): if self.kb.ask_if_true(location(i, j, self.t)): temp.append(i) temp.append(j) @@ -1071,8 +1522,8 @@ def execute(self, percept): self.current_position = WumpusPosition(temp[0], temp[1], 'RIGHT') safe_points = list() - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): if self.kb.ask_if_true(ok_to_move(i, j, self.t)): safe_points.append([i, j]) @@ -1080,14 +1531,14 @@ def execute(self, percept): goals = list() goals.append([1, 1]) self.plan.append('Grab') - actions = self.plan_route(self.current_position,goals,safe_points) + actions = self.plan_route(self.current_position, goals, safe_points) self.plan.extend(actions) self.plan.append('Climb') if len(self.plan) == 0: unvisited = list() - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): for k in range(self.t): if self.kb.ask_if_true(location(i, j, k)): unvisited.append([i, j]) @@ -1097,13 +1548,13 @@ def execute(self, percept): if u not in unvisited_and_safe and s == u: unvisited_and_safe.append(u) - temp = self.plan_route(self.current_position,unvisited_and_safe,safe_points) + temp = self.plan_route(self.current_position, unvisited_and_safe, safe_points) self.plan.extend(temp) if len(self.plan) == 0 and self.kb.ask_if_true(have_arrow(self.t)): possible_wumpus = list() - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): if not self.kb.ask_if_true(wumpus(i, j)): possible_wumpus.append([i, j]) @@ -1112,8 +1563,8 @@ def execute(self, percept): if len(self.plan) == 0: not_unsafe = list() - for i in range(1, self.dimrow+1): - for j in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): if not self.kb.ask_if_true(ok_to_move(i, j, self.t)): not_unsafe.append([i, j]) temp = self.plan_route(self.current_position, not_unsafe, safe_points) @@ -1133,19 +1584,17 @@ def execute(self, percept): return action - def plan_route(self, current, goals, allowed): problem = PlanRoute(current, goals, allowed, self.dimrow) return astar_search(problem).solution() - def plan_shot(self, current, goals, allowed): shooting_positions = set() for loc in goals: x = loc[0] y = loc[1] - for i in range(1, self.dimrow+1): + for i in range(1, self.dimrow + 1): if i < x: shooting_positions.add(WumpusPosition(i, y, 'EAST')) if i > x: @@ -1157,7 +1606,7 @@ def plan_shot(self, current, goals, allowed): # Can't have a shooting position from any of the rooms the Wumpus could reside orientations = ['EAST', 'WEST', 'NORTH', 'SOUTH'] - for loc in goals: + for loc in goals: for orientation in orientations: shooting_positions.remove(WumpusPosition(loc[0], loc[1], orientation)) @@ -1170,11 +1619,12 @@ def plan_shot(self, current, goals, allowed): # ______________________________________________________________________________ -def SAT_plan(init, transition, goal, t_max, SAT_solver=dpll_satisfiable): - """Converts a planning problem to Satisfaction problem by translating it to a cnf sentence. +def SAT_plan(init, transition, goal, t_max, SAT_solver=cdcl_satisfiable): + """ [Figure 7.22] + Converts a planning problem to Satisfaction problem by translating it to a cnf sentence. >>> transition = {'A': {'Left': 'A', 'Right': 'B'}, 'B': {'Left': 'A', 'Right': 'C'}, 'C': {'Left': 'B', 'Right': 'C'}} - >>> SAT_plan('A', transition, 'C', 2) is None + >>> SAT_plan('A', transition, 'C', 1) is None True """ @@ -1186,14 +1636,16 @@ def translate_to_SAT(init, transition, goal, time): # Symbol claiming state s at time t state_counter = itertools.count() for s in states: - for t in range(time+1): - state_sym[s, t] = Expr("State_{}".format(next(state_counter))) + for t in range(time + 1): + state_sym[s, t] = Expr('S_{}'.format(next(state_counter))) # Add initial state axiom clauses.append(state_sym[init, 0]) # Add goal state axiom - clauses.append(state_sym[goal, time]) + clauses.append(state_sym[first(clause[0] for clause in state_sym + if set(conjuncts(clause[0])).issuperset(conjuncts(goal))), time]) \ + if isinstance(goal, Expr) else clauses.append(state_sym[goal, time]) # All possible transitions transition_counter = itertools.count() @@ -1202,15 +1654,14 @@ def translate_to_SAT(init, transition, goal, time): s_ = transition[s][action] for t in range(time): # Action 'action' taken from state 's' at time 't' to reach 's_' - action_sym[s, action, t] = Expr( - "Transition_{}".format(next(transition_counter))) + action_sym[s, action, t] = Expr('T_{}'.format(next(transition_counter))) # Change the state from s to s_ - clauses.append(action_sym[s, action, t] |'==>'| state_sym[s, t]) - clauses.append(action_sym[s, action, t] |'==>'| state_sym[s_, t + 1]) + clauses.append(action_sym[s, action, t] | '==>' | state_sym[s, t]) + clauses.append(action_sym[s, action, t] | '==>' | state_sym[s_, t + 1]) # Allow only one state at any time - for t in range(time+1): + for t in range(time + 1): # must be a state at any time clauses.append(associate('|', [state_sym[s, t] for s in states])) @@ -1242,7 +1693,7 @@ def extract_solution(model): return [action for s, action, time in true_transitions] # Body of SAT_plan algorithm - for t in range(t_max): + for t in range(t_max + 1): # dictionaries to help extract the solution from model state_sym = {} action_sym = {} @@ -1258,9 +1709,11 @@ def extract_solution(model): def unify(x, y, s={}): - """Unify expressions x,y with substitution s; return a substitution that + """ + [Figure 9.1] + Unify expressions x,y with substitution s; return a substitution that would make x,y equal, or None if x,y can not unify. x and y can be - variables (e.g. Expr('x')), constants, lists, or Exprs. [Figure 9.1] + variables (e.g. Expr('x')), constants, lists, or Exprs. >>> unify(x, 3, {}) {x: 3} """ @@ -1297,7 +1750,9 @@ def unify_var(var, x, s): elif occur_check(var, x, s): return None else: - return extend(s, var, x) + new_s = extend(s, var, x) + cascade_substitution(new_s) + return new_s def occur_check(var, x, s): @@ -1316,16 +1771,6 @@ def occur_check(var, x, s): return False -def extend(s, var, val): - """Copy the substitution s and extend it by setting var to val; return copy. - >>> extend({x: 1}, y, 2) == {x: 1, y: 2} - True - """ - s2 = s.copy() - s2[var] = val - return s2 - - def subst(s, x): """Substitute the substitution s into the expression x. >>> subst({x: 42, y:0}, F(x) + y) @@ -1343,6 +1788,99 @@ def subst(s, x): return Expr(x.op, *[subst(s, arg) for arg in x.args]) +def cascade_substitution(s): + """This method allows to return a correct unifier in normal form + and perform a cascade substitution to s. + For every mapping in s perform a cascade substitution on s.get(x) + and if it is replaced with a function ensure that all the function + terms are correct updates by passing over them again. + >>> s = {x: y, y: G(z)} + >>> cascade_substitution(s) + >>> s == {x: G(z), y: G(z)} + True + """ + + for x in s: + s[x] = subst(s, s.get(x)) + if isinstance(s.get(x), Expr) and not is_variable(s.get(x)): + # Ensure Function Terms are correct updates by passing over them again + s[x] = subst(s, s.get(x)) + + +def unify_mm(x, y, s={}): + """Unify expressions x,y with substitution s using an efficient rule-based + unification algorithm by Martelli & Montanari; return a substitution that + would make x,y equal, or None if x,y can not unify. x and y can be + variables (e.g. Expr('x')), constants, lists, or Exprs. + >>> unify_mm(x, 3, {}) + {x: 3} + """ + + set_eq = extend(s, x, y) + s = set_eq.copy() + while True: + trans = 0 + for x, y in set_eq.items(): + if x == y: + # if x = y this mapping is deleted (rule b) + del s[x] + elif not is_variable(x) and is_variable(y): + # if x is not a variable and y is a variable, rewrite it as y = x in s (rule a) + if s.get(y, None) is None: + s[y] = x + del s[x] + else: + # if a mapping already exist for variable y then apply + # variable elimination (there is a chance to apply rule d) + s[x] = vars_elimination(y, s) + elif not is_variable(x) and not is_variable(y): + # in which case x and y are not variables, if the two root function symbols + # are different, stop with failure, else apply term reduction (rule c) + if x.op is y.op and len(x.args) == len(y.args): + term_reduction(x, y, s) + del s[x] + else: + return None + elif isinstance(y, Expr): + # in which case x is a variable and y is a function or a variable (e.g. F(z) or y), + # if y is a function, we must check if x occurs in y, then stop with failure, else + # try to apply variable elimination to y (rule d) + if occur_check(x, y, s): + return None + s[x] = vars_elimination(y, s) + if y == s.get(x): + trans += 1 + else: + trans += 1 + if trans == len(set_eq): + # if no transformation has been applied, stop with success + return s + set_eq = s.copy() + + +def term_reduction(x, y, s): + """Apply term reduction to x and y if both are functions and the two root function + symbols are equals (e.g. F(x1, x2, ..., xn) and F(x1', x2', ..., xn')) by returning + a new mapping obtained by replacing x: y with {x1: x1', x2: x2', ..., xn: xn'} + """ + for i in range(len(x.args)): + if x.args[i] in s: + s[s.get(x.args[i])] = y.args[i] + else: + s[x.args[i]] = y.args[i] + + +def vars_elimination(x, s): + """Apply variable elimination to x: if x is a variable and occurs in s, return + the term mapped by x, else if x is a function recursively applies variable + elimination to each term of the function.""" + if not isinstance(x, Expr): + return x + if is_variable(x): + return s.get(x, x) + return Expr(x.op, *[vars_elimination(arg, s) for arg in x.args]) + + def standardize_variables(sentence, dic=None): """Replace all the variables in sentence with new variables.""" if dic is None: @@ -1357,12 +1895,25 @@ def standardize_variables(sentence, dic=None): dic[sentence] = v return v else: - return Expr(sentence.op, - *[standardize_variables(a, dic) for a in sentence.args]) + return Expr(sentence.op, *[standardize_variables(a, dic) for a in sentence.args]) standardize_variables.counter = itertools.count() + +# ______________________________________________________________________________ + + +def parse_clauses_from_dimacs(dimacs_cnf): + """Converts a string into CNF clauses according to the DIMACS format used in SAT competitions""" + return map(lambda c: associate('|', c), + map(lambda c: [expr('~X' + str(abs(l))) if l < 0 else expr('X' + str(l)) for l in c], + map(lambda line: map(int, line.split()), + filter(None, ' '.join( + filter(lambda line: line[0] not in ('c', 'p'), + filter(None, dimacs_cnf.strip().replace('\t', ' ').split('\n')))).split(' 0'))))) + + # ______________________________________________________________________________ @@ -1378,17 +1929,18 @@ class FolKB(KB): False """ - def __init__(self, initial_clauses=None): + def __init__(self, clauses=None): + super().__init__() self.clauses = [] # inefficient: no indexing - if initial_clauses: - for clause in initial_clauses: + if clauses: + for clause in clauses: self.tell(clause) def tell(self, sentence): if is_definite_clause(sentence): self.clauses.append(sentence) else: - raise Exception("Not a definite clause: {}".format(sentence)) + raise Exception('Not a definite clause: {}'.format(sentence)) def ask_generator(self, query): return fol_bc_ask(self, query) @@ -1400,10 +1952,14 @@ def fetch_rules_for_goal(self, goal): return self.clauses -def fol_fc_ask(KB, alpha): - """A simple forward-chaining algorithm. [Figure 9.3]""" - # TODO: Improve efficiency - kb_consts = list({c for clause in KB.clauses for c in constant_symbols(clause)}) +def fol_fc_ask(kb, alpha): + """ + [Figure 9.3] + A simple forward-chaining algorithm. + """ + # TODO: improve efficiency + kb_consts = list({c for clause in kb.clauses for c in constant_symbols(clause)}) + def enum_subst(p): query_vars = list({v for clause in p for v in variables(clause)}) for assignment_list in itertools.product(kb_consts, repeat=len(query_vars)): @@ -1411,96 +1967,96 @@ def enum_subst(p): yield theta # check if we can answer without new inferences - for q in KB.clauses: - phi = unify(q, alpha, {}) + for q in kb.clauses: + phi = unify_mm(q, alpha) if phi is not None: yield phi while True: new = [] - for rule in KB.clauses: + for rule in kb.clauses: p, q = parse_definite_clause(rule) for theta in enum_subst(p): - if set(subst(theta, p)).issubset(set(KB.clauses)): + if set(subst(theta, p)).issubset(set(kb.clauses)): q_ = subst(theta, q) - if all([unify(x, q_, {}) is None for x in KB.clauses + new]): + if all([unify_mm(x, q_) is None for x in kb.clauses + new]): new.append(q_) - phi = unify(q_, alpha, {}) + phi = unify_mm(q_, alpha) if phi is not None: yield phi if not new: break for clause in new: - KB.tell(clause) + kb.tell(clause) return None -def fol_bc_ask(KB, query): - """A simple backward-chaining algorithm for first-order logic. [Figure 9.6] - KB should be an instance of FolKB, and query an atomic sentence.""" - return fol_bc_or(KB, query, {}) +def fol_bc_ask(kb, query): + """ + [Figure 9.6] + A simple backward-chaining algorithm for first-order logic. + KB should be an instance of FolKB, and query an atomic sentence. + """ + return fol_bc_or(kb, query, {}) -def fol_bc_or(KB, goal, theta): - for rule in KB.fetch_rules_for_goal(goal): +def fol_bc_or(kb, goal, theta): + for rule in kb.fetch_rules_for_goal(goal): lhs, rhs = parse_definite_clause(standardize_variables(rule)) - for theta1 in fol_bc_and(KB, lhs, unify(rhs, goal, theta)): + for theta1 in fol_bc_and(kb, lhs, unify_mm(rhs, goal, theta)): yield theta1 -def fol_bc_and(KB, goals, theta): +def fol_bc_and(kb, goals, theta): if theta is None: pass elif not goals: yield theta else: first, rest = goals[0], goals[1:] - for theta1 in fol_bc_or(KB, subst(theta, first), theta): - for theta2 in fol_bc_and(KB, rest, theta1): + for theta1 in fol_bc_or(kb, subst(theta, first), theta): + for theta2 in fol_bc_and(kb, rest, theta1): yield theta2 -# A simple KB that defines the relevant conditions of the Wumpus World as in Fig 7.4. +# A simple KB that defines the relevant conditions of the Wumpus World as in Figure 7.4. # See Sec. 7.4.3 wumpus_kb = PropKB() P11, P12, P21, P22, P31, B11, B21 = expr('P11, P12, P21, P22, P31, B11, B21') wumpus_kb.tell(~P11) -wumpus_kb.tell(B11 | '<=>' | ((P12 | P21))) -wumpus_kb.tell(B21 | '<=>' | ((P11 | P22 | P31))) +wumpus_kb.tell(B11 | '<=>' | (P12 | P21)) +wumpus_kb.tell(B21 | '<=>' | (P11 | P22 | P31)) wumpus_kb.tell(~B11) wumpus_kb.tell(B21) -test_kb = FolKB( - map(expr, ['Farmer(Mac)', - 'Rabbit(Pete)', - 'Mother(MrsMac, Mac)', - 'Mother(MrsRabbit, Pete)', - '(Rabbit(r) & Farmer(f)) ==> Hates(f, r)', - '(Mother(m, c)) ==> Loves(m, c)', - '(Mother(m, r) & Rabbit(r)) ==> Rabbit(m)', - '(Farmer(f)) ==> Human(f)', - # Note that this order of conjuncts - # would result in infinite recursion: - # '(Human(h) & Mother(m, h)) ==> Human(m)' - '(Mother(m, h) & Human(h)) ==> Human(m)' - ])) - -crime_kb = FolKB( - map(expr, ['(American(x) & Weapon(y) & Sells(x, y, z) & Hostile(z)) ==> Criminal(x)', - 'Owns(Nono, M1)', - 'Missile(M1)', - '(Missile(x) & Owns(Nono, x)) ==> Sells(West, x, Nono)', - 'Missile(x) ==> Weapon(x)', - 'Enemy(x, America) ==> Hostile(x)', - 'American(West)', - 'Enemy(Nono, America)' - ])) +test_kb = FolKB(map(expr, ['Farmer(Mac)', + 'Rabbit(Pete)', + 'Mother(MrsMac, Mac)', + 'Mother(MrsRabbit, Pete)', + '(Rabbit(r) & Farmer(f)) ==> Hates(f, r)', + '(Mother(m, c)) ==> Loves(m, c)', + '(Mother(m, r) & Rabbit(r)) ==> Rabbit(m)', + '(Farmer(f)) ==> Human(f)', + # Note that this order of conjuncts + # would result in infinite recursion: + # '(Human(h) & Mother(m, h)) ==> Human(m)' + '(Mother(m, h) & Human(h)) ==> Human(m)'])) + +crime_kb = FolKB(map(expr, ['(American(x) & Weapon(y) & Sells(x, y, z) & Hostile(z)) ==> Criminal(x)', + 'Owns(Nono, M1)', + 'Missile(M1)', + '(Missile(x) & Owns(Nono, x)) ==> Sells(West, x, Nono)', + 'Missile(x) ==> Weapon(x)', + 'Enemy(x, America) ==> Hostile(x)', + 'American(West)', + 'Enemy(Nono, America)'])) + # ______________________________________________________________________________ # Example application (not in the book). -# You can use the Expr class to do symbolic differentiation. This used to be +# You can use the Expr class to do symbolic differentiation. This used to be # a part of AI; now it is considered a separate field, Symbolic Algebra. @@ -1527,14 +2083,14 @@ def diff(y, x): elif op == '/': return (v * diff(u, x) - u * diff(v, x)) / (v * v) elif op == '**' and isnumber(x.op): - return (v * u ** (v - 1) * diff(u, x)) + return v * u ** (v - 1) * diff(u, x) elif op == '**': return (v * u ** (v - 1) * diff(u, x) + u ** v * Expr('log')(u) * diff(v, x)) elif op == 'log': return diff(u, x) / u else: - raise ValueError("Unknown op: {} in diff({}, {})".format(op, y, x)) + raise ValueError('Unknown op: {} in diff({}, {})'.format(op, y, x)) def simp(x): @@ -1595,7 +2151,7 @@ def simp(x): if u == 1: return 0 else: - raise ValueError("Unknown op: " + op) + raise ValueError('Unknown op: ' + op) # If we fall through to here, we can not simplify further return Expr(op, *args) diff --git a/logic4e.py b/logic4e.py new file mode 100644 index 000000000..75608ad74 --- /dev/null +++ b/logic4e.py @@ -0,0 +1,1665 @@ +"""Representations and Inference for Logic (Chapters 7-10) + +Covers both Propositional and First-Order Logic. First we have four +important data types: + + KB Abstract class holds a knowledge base of logical expressions + KB_Agent Abstract class subclasses agents.Agent + Expr A logical expression, imported from utils.py + substitution Implemented as a dictionary of var:value pairs, {x:1, y:x} + +Be careful: some functions take an Expr as argument, and some take a KB. + +Logical expressions can be created with Expr or expr, imported from utils, TODO +or with expr, which adds the capability to write a string that uses +the connectives ==>, <==, <=>, or <=/=>. But be careful: these have the +operator precedence of commas; you may need to add parents to make precedence work. +See logic.ipynb for examples. + +Then we implement various functions for doing logical inference: + + pl_true Evaluate a propositional logical sentence in a model + tt_entails Say if a statement is entailed by a KB + pl_resolution Do resolution on propositional sentences + dpll_satisfiable See if a propositional sentence is satisfiable + WalkSAT Try to find a solution for a set of clauses + +And a few other functions: + + to_cnf Convert to conjunctive normal form + unify Do unification of two FOL sentences + diff, simp Symbolic differentiation and simplification +""" +import itertools +import random +from collections import defaultdict + +from agents import Agent, Glitter, Bump, Stench, Breeze, Scream +from search import astar_search, PlanRoute +from utils4e import remove_all, unique, first, probability, isnumber, issequence, Expr, expr, subexpressions + + +# ______________________________________________________________________________ +# Chapter 7 Logical Agents +# 7.1 Knowledge Based Agents + + +class KB: + """ + A knowledge base to which you can tell and ask sentences. + To create a KB, subclass this class and implement tell, ask_generator, and retract. + Ask_generator: + For a Propositional Logic KB, ask(P & Q) returns True or False, but for an + FOL KB, something like ask(Brother(x, y)) might return many substitutions + such as {x: Cain, y: Abel}, {x: Abel, y: Cain}, {x: George, y: Jeb}, etc. + So ask_generator generates these one at a time, and ask either returns the + first one or returns False. + """ + + def __init__(self, sentence=None): + raise NotImplementedError + + def tell(self, sentence): + """Add the sentence to the KB.""" + raise NotImplementedError + + def ask(self, query): + """Return a substitution that makes the query true, or, failing that, return False.""" + return first(self.ask_generator(query), default=False) + + def ask_generator(self, query): + """Yield all the substitutions that make query true.""" + raise NotImplementedError + + def retract(self, sentence): + """Remove sentence from the KB.""" + raise NotImplementedError + + +class PropKB(KB): + """A KB for propositional logic. Inefficient, with no indexing.""" + + def __init__(self, sentence=None): + self.clauses = [] + if sentence: + self.tell(sentence) + + def tell(self, sentence): + """Add the sentence's clauses to the KB.""" + self.clauses.extend(conjuncts(to_cnf(sentence))) + + def ask_generator(self, query): + """Yield the empty substitution {} if KB entails query; else no results.""" + if tt_entails(Expr('&', *self.clauses), query): + yield {} + + def ask_if_true(self, query): + """Return True if the KB entails query, else return False.""" + for _ in self.ask_generator(query): + return True + return False + + def retract(self, sentence): + """Remove the sentence's clauses from the KB.""" + for c in conjuncts(to_cnf(sentence)): + if c in self.clauses: + self.clauses.remove(c) + + +def KB_AgentProgram(KB): + """A generic logical knowledge-based agent program. [Figure 7.1]""" + steps = itertools.count() + + def program(percept): + t = next(steps) + KB.tell(make_percept_sentence(percept, t)) + action = KB.ask(make_action_query(t)) + KB.tell(make_action_sentence(action, t)) + return action + + def make_percept_sentence(percept, t): + return Expr("Percept")(percept, t) + + def make_action_query(t): + return expr("ShouldDo(action, {})".format(t)) + + def make_action_sentence(action, t): + return Expr("Did")(action[expr('action')], t) + + return program + + +# _____________________________________________________________________________ +# 7.2 The Wumpus World + + +# Expr functions for WumpusKB and HybridWumpusAgent + + +def facing_east(time): + return Expr('FacingEast', time) + + +def facing_west(time): + return Expr('FacingWest', time) + + +def facing_north(time): + return Expr('FacingNorth', time) + + +def facing_south(time): + return Expr('FacingSouth', time) + + +def wumpus(x, y): + return Expr('W', x, y) + + +def pit(x, y): + return Expr('P', x, y) + + +def breeze(x, y): + return Expr('B', x, y) + + +def stench(x, y): + return Expr('S', x, y) + + +def wumpus_alive(time): + return Expr('WumpusAlive', time) + + +def have_arrow(time): + return Expr('HaveArrow', time) + + +def percept_stench(time): + return Expr('Stench', time) + + +def percept_breeze(time): + return Expr('Breeze', time) + + +def percept_glitter(time): + return Expr('Glitter', time) + + +def percept_bump(time): + return Expr('Bump', time) + + +def percept_scream(time): + return Expr('Scream', time) + + +def move_forward(time): + return Expr('Forward', time) + + +def shoot(time): + return Expr('Shoot', time) + + +def turn_left(time): + return Expr('TurnLeft', time) + + +def turn_right(time): + return Expr('TurnRight', time) + + +def ok_to_move(x, y, time): + return Expr('OK', x, y, time) + + +def location(x, y, time=None): + if time is None: + return Expr('L', x, y) + else: + return Expr('L', x, y, time) + + +# Symbols + + +def implies(lhs, rhs): + return Expr('==>', lhs, rhs) + + +def equiv(lhs, rhs): + return Expr('<=>', lhs, rhs) + + +# Helper Function + + +def new_disjunction(sentences): + t = sentences[0] + for i in range(1, len(sentences)): + t |= sentences[i] + return t + + +# ______________________________________________________________________________ +# 7.4 Propositional Logic + + +def is_symbol(s): + """A string s is a symbol if it starts with an alphabetic char. + >>> is_symbol('R2D2') + True + """ + return isinstance(s, str) and s[:1].isalpha() + + +def is_var_symbol(s): + """A logic variable symbol is an initial-lowercase string. + >>> is_var_symbol('EXE') + False + """ + return is_symbol(s) and s[0].islower() + + +def is_prop_symbol(s): + """A proposition logic symbol is an initial-uppercase string. + >>> is_prop_symbol('exe') + False + """ + return is_symbol(s) and s[0].isupper() + + +def variables(s): + """Return a set of the variables in expression s. + >>> variables(expr('F(x, x) & G(x, y) & H(y, z) & R(A, z, 2)')) == {x, y, z} + True + """ + return {x for x in subexpressions(s) if is_variable(x)} + + +def is_definite_clause(s): + """ + Returns True for exprs s of the form A & B & ... & C ==> D, + where all literals are positive. In clause form, this is + ~A | ~B | ... | ~C | D, where exactly one clause is positive. + >>> is_definite_clause(expr('Farmer(Mac)')) + True + """ + if is_symbol(s.op): + return True + elif s.op == '==>': + antecedent, consequent = s.args + return (is_symbol(consequent.op) and + all(is_symbol(arg.op) for arg in conjuncts(antecedent))) + else: + return False + + +def parse_definite_clause(s): + """Return the antecedents and the consequent of a definite clause.""" + assert is_definite_clause(s) + if is_symbol(s.op): + return [], s + else: + antecedent, consequent = s.args + return conjuncts(antecedent), consequent + + +# Useful constant Exprs used in examples and code: +A, B, C, D, E, F, G, P, Q, x, y, z = map(Expr, 'ABCDEFGPQxyz') + + +# ______________________________________________________________________________ +# 7.4.4 A simple inference procedure + + +def tt_entails(kb, alpha): + """ + Does kb entail the sentence alpha? Use truth tables. For propositional + kb's and sentences. [Figure 7.10]. Note that the 'kb' should be an + Expr which is a conjunction of clauses. + >>> tt_entails(expr('P & Q'), expr('Q')) + True + """ + assert not variables(alpha) + symbols = list(prop_symbols(kb & alpha)) + return tt_check_all(kb, alpha, symbols, {}) + + +def tt_check_all(kb, alpha, symbols, model): + """Auxiliary routine to implement tt_entails.""" + if not symbols: + if pl_true(kb, model): + result = pl_true(alpha, model) + assert result in (True, False) + return result + else: + return True + else: + P, rest = symbols[0], symbols[1:] + return (tt_check_all(kb, alpha, rest, extend(model, P, True)) and + tt_check_all(kb, alpha, rest, extend(model, P, False))) + + +def prop_symbols(x): + """Return the set of all propositional symbols in x.""" + if not isinstance(x, Expr): + return set() + elif is_prop_symbol(x.op): + return {x} + else: + return {symbol for arg in x.args for symbol in prop_symbols(arg)} + + +def constant_symbols(x): + """Return the set of all constant symbols in x.""" + if not isinstance(x, Expr): + return set() + elif is_prop_symbol(x.op) and not x.args: + return {x} + else: + return {symbol for arg in x.args for symbol in constant_symbols(arg)} + + +def predicate_symbols(x): + """ + Return a set of (symbol_name, arity) in x. + All symbols (even functional) with arity > 0 are considered. + """ + if not isinstance(x, Expr) or not x.args: + return set() + pred_set = {(x.op, len(x.args))} if is_prop_symbol(x.op) else set() + pred_set.update({symbol for arg in x.args for symbol in predicate_symbols(arg)}) + return pred_set + + +def tt_true(s): + """Is a propositional sentence a tautology? + >>> tt_true('P | ~P') + True + """ + s = expr(s) + return tt_entails(True, s) + + +def pl_true(exp, model={}): + """ + Return True if the propositional logic expression is true in the model, + and False if it is false. If the model does not specify the value for + every proposition, this may return None to indicate 'not obvious'; + this may happen even when the expression is tautological. + >>> pl_true(P, {}) is None + True + """ + if exp in (True, False): + return exp + op, args = exp.op, exp.args + if is_prop_symbol(op): + return model.get(exp) + elif op == '~': + p = pl_true(args[0], model) + if p is None: + return None + else: + return not p + elif op == '|': + result = False + for arg in args: + p = pl_true(arg, model) + if p is True: + return True + if p is None: + result = None + return result + elif op == '&': + result = True + for arg in args: + p = pl_true(arg, model) + if p is False: + return False + if p is None: + result = None + return result + p, q = args + if op == '==>': + return pl_true(~p | q, model) + elif op == '<==': + return pl_true(p | ~q, model) + pt = pl_true(p, model) + if pt is None: + return None + qt = pl_true(q, model) + if qt is None: + return None + if op == '<=>': + return pt == qt + elif op == '^': # xor or 'not equivalent' + return pt != qt + else: + raise ValueError("illegal operator in logic expression" + str(exp)) + + +# ______________________________________________________________________________ +# 7.5 Propositional Theorem Proving + + +def to_cnf(s): + """Convert a propositional logical sentence to conjunctive normal form. + That is, to the form ((A | ~B | ...) & (B | C | ...) & ...) [p. 253] + >>> to_cnf('~(B | C)') + (~B & ~C) + """ + s = expr(s) + if isinstance(s, str): + s = expr(s) + s = eliminate_implications(s) # Steps 1, 2 from p. 253 + s = move_not_inwards(s) # Step 3 + return distribute_and_over_or(s) # Step 4 + + +def eliminate_implications(s): + """Change implications into equivalent form with only &, |, and ~ as logical operators.""" + s = expr(s) + if not s.args or is_symbol(s.op): + return s # Atoms are unchanged. + args = list(map(eliminate_implications, s.args)) + a, b = args[0], args[-1] + if s.op == '==>': + return b | ~a + elif s.op == '<==': + return a | ~b + elif s.op == '<=>': + return (a | ~b) & (b | ~a) + elif s.op == '^': + assert len(args) == 2 # TODO: relax this restriction + return (a & ~b) | (~a & b) + else: + assert s.op in ('&', '|', '~') + return Expr(s.op, *args) + + +def move_not_inwards(s): + """Rewrite sentence s by moving negation sign inward. + >>> move_not_inwards(~(A | B)) + (~A & ~B) + """ + s = expr(s) + if s.op == '~': + def NOT(b): + return move_not_inwards(~b) + + a = s.args[0] + if a.op == '~': + return move_not_inwards(a.args[0]) # ~~A ==> A + if a.op == '&': + return associate('|', list(map(NOT, a.args))) + if a.op == '|': + return associate('&', list(map(NOT, a.args))) + return s + elif is_symbol(s.op) or not s.args: + return s + else: + return Expr(s.op, *list(map(move_not_inwards, s.args))) + + +def distribute_and_over_or(s): + """Given a sentence s consisting of conjunctions and disjunctions + of literals, return an equivalent sentence in CNF. + >>> distribute_and_over_or((A & B) | C) + ((A | C) & (B | C)) + """ + s = expr(s) + if s.op == '|': + s = associate('|', s.args) + if s.op != '|': + return distribute_and_over_or(s) + if len(s.args) == 0: + return False + if len(s.args) == 1: + return distribute_and_over_or(s.args[0]) + conj = first(arg for arg in s.args if arg.op == '&') + if not conj: + return s + others = [a for a in s.args if a is not conj] + rest = associate('|', others) + return associate('&', [distribute_and_over_or(c | rest) + for c in conj.args]) + elif s.op == '&': + return associate('&', list(map(distribute_and_over_or, s.args))) + else: + return s + + +def associate(op, args): + """Given an associative op, return an expression with the same + meaning as Expr(op, *args), but flattened -- that is, with nested + instances of the same op promoted to the top level. + >>> associate('&', [(A&B),(B|C),(B&C)]) + (A & B & (B | C) & B & C) + >>> associate('|', [A|(B|(C|(A&B)))]) + (A | B | C | (A & B)) + """ + args = dissociate(op, args) + if len(args) == 0: + return _op_identity[op] + elif len(args) == 1: + return args[0] + else: + return Expr(op, *args) + + +_op_identity = {'&': True, '|': False, '+': 0, '*': 1} + + +def dissociate(op, args): + """Given an associative op, return a flattened list result such + that Expr(op, *result) means the same as Expr(op, *args). + >>> dissociate('&', [A & B]) + [A, B] + """ + result = [] + + def collect(subargs): + for arg in subargs: + if arg.op == op: + collect(arg.args) + else: + result.append(arg) + + collect(args) + return result + + +def conjuncts(s): + """Return a list of the conjuncts in the sentence s. + >>> conjuncts(A & B) + [A, B] + >>> conjuncts(A | B) + [(A | B)] + """ + return dissociate('&', [s]) + + +def disjuncts(s): + """Return a list of the disjuncts in the sentence s. + >>> disjuncts(A | B) + [A, B] + >>> disjuncts(A & B) + [(A & B)] + """ + return dissociate('|', [s]) + + +# ______________________________________________________________________________ + + +def pl_resolution(KB, alpha): + """ + Propositional-logic resolution: say if alpha follows from KB. [Figure 7.12] + >>> pl_resolution(horn_clauses_KB, A) + True + """ + clauses = KB.clauses + conjuncts(to_cnf(~alpha)) + new = set() + while True: + n = len(clauses) + pairs = [(clauses[i], clauses[j]) + for i in range(n) for j in range(i + 1, n)] + for (ci, cj) in pairs: + resolvents = pl_resolve(ci, cj) + if False in resolvents: + return True + new = new.union(set(resolvents)) + if new.issubset(set(clauses)): + return False + for c in new: + if c not in clauses: + clauses.append(c) + + +def pl_resolve(ci, cj): + """Return all clauses that can be obtained by resolving clauses ci and cj.""" + clauses = [] + for di in disjuncts(ci): + for dj in disjuncts(cj): + if di == ~dj or ~di == dj: + dnew = unique(remove_all(di, disjuncts(ci)) + + remove_all(dj, disjuncts(cj))) + clauses.append(associate('|', dnew)) + return clauses + + +# ______________________________________________________________________________ +# 7.5.4 Forward and backward chaining + + +class PropDefiniteKB(PropKB): + """A KB of propositional definite clauses.""" + + def tell(self, sentence): + """Add a definite clause to this KB.""" + assert is_definite_clause(sentence), "Must be definite clause" + self.clauses.append(sentence) + + def ask_generator(self, query): + """Yield the empty substitution if KB implies query; else nothing.""" + if pl_fc_entails(self.clauses, query): + yield {} + + def retract(self, sentence): + self.clauses.remove(sentence) + + def clauses_with_premise(self, p): + """Return a list of the clauses in KB that have p in their premise. + This could be cached away for O(1) speed, but we'll recompute it.""" + return [c for c in self.clauses + if c.op == '==>' and p in conjuncts(c.args[0])] + + +def pl_fc_entails(KB, q): + """Use forward chaining to see if a PropDefiniteKB entails symbol q. + [Figure 7.15] + >>> pl_fc_entails(horn_clauses_KB, expr('Q')) + True + """ + count = {c: len(conjuncts(c.args[0])) + for c in KB.clauses + if c.op == '==>'} + inferred = defaultdict(bool) + agenda = [s for s in KB.clauses if is_prop_symbol(s.op)] + while agenda: + p = agenda.pop() + if p == q: + return True + if not inferred[p]: + inferred[p] = True + for c in KB.clauses_with_premise(p): + count[c] -= 1 + if count[c] == 0: + agenda.append(c.args[1]) + return False + + +""" [Figure 7.13] +Simple inference in a wumpus world example +""" +wumpus_world_inference = expr("(B11 <=> (P12 | P21)) & ~B11") + +""" [Figure 7.16] +Propositional Logic Forward Chaining example +""" +horn_clauses_KB = PropDefiniteKB() +for s in "P==>Q; (L&M)==>P; (B&L)==>M; (A&P)==>L; (A&B)==>L; A;B".split(';'): + horn_clauses_KB.tell(expr(s)) + +""" +Definite clauses KB example +""" +definite_clauses_KB = PropDefiniteKB() +for clause in ['(B & F)==>E', '(A & E & F)==>G', '(B & C)==>F', '(A & B)==>D', '(E & F)==>H', '(H & I)==>J', 'A', 'B', + 'C']: + definite_clauses_KB.tell(expr(clause)) + + +# ______________________________________________________________________________ +# 7.6 Effective Propositional Model Checking +# DPLL-Satisfiable [Figure 7.17] + + +def dpll_satisfiable(s): + """Check satisfiability of a propositional sentence. + This differs from the book code in two ways: (1) it returns a model + rather than True when it succeeds; this is more useful. (2) The + function find_pure_symbol is passed a list of unknown clauses, rather + than a list of all clauses and the model; this is more efficient. + >>> dpll_satisfiable(A |'<=>'| B) == {A: True, B: True} + True + """ + clauses = conjuncts(to_cnf(s)) + symbols = list(prop_symbols(s)) + return dpll(clauses, symbols, {}) + + +def dpll(clauses, symbols, model): + """See if the clauses are true in a partial model.""" + unknown_clauses = [] # clauses with an unknown truth value + for c in clauses: + val = pl_true(c, model) + if val is False: + return False + if val is not True: + unknown_clauses.append(c) + if not unknown_clauses: + return model + P, value = find_pure_symbol(symbols, unknown_clauses) + if P: + return dpll(clauses, remove_all(P, symbols), extend(model, P, value)) + P, value = find_unit_clause(clauses, model) + if P: + return dpll(clauses, remove_all(P, symbols), extend(model, P, value)) + if not symbols: + raise TypeError("Argument should be of the type Expr.") + P, symbols = symbols[0], symbols[1:] + return (dpll(clauses, symbols, extend(model, P, True)) or + dpll(clauses, symbols, extend(model, P, False))) + + +def find_pure_symbol(symbols, clauses): + """ + Find a symbol and its value if it appears only as a positive literal + (or only as a negative) in clauses. + >>> find_pure_symbol([A, B, C], [A|~B,~B|~C,C|A]) + (A, True) + """ + for s in symbols: + found_pos, found_neg = False, False + for c in clauses: + if not found_pos and s in disjuncts(c): + found_pos = True + if not found_neg and ~s in disjuncts(c): + found_neg = True + if found_pos != found_neg: + return s, found_pos + return None, None + + +def find_unit_clause(clauses, model): + """ + Find a forced assignment if possible from a clause with only 1 + variable not bound in the model. + >>> find_unit_clause([A|B|C, B|~C, ~A|~B], {A:True}) + (B, False) + """ + for clause in clauses: + P, value = unit_clause_assign(clause, model) + if P: + return P, value + return None, None + + +def unit_clause_assign(clause, model): + """Return a single variable/value pair that makes clause true in + the model, if possible. + >>> unit_clause_assign(A|B|C, {A:True}) + (None, None) + >>> unit_clause_assign(B|~C, {A:True}) + (None, None) + >>> unit_clause_assign(~A|~B, {A:True}) + (B, False) + """ + P, value = None, None + for literal in disjuncts(clause): + sym, positive = inspect_literal(literal) + if sym in model: + if model[sym] == positive: + return None, None # clause already True + elif P: + return None, None # more than 1 unbound variable + else: + P, value = sym, positive + return P, value + + +def inspect_literal(literal): + """The symbol in this literal, and the value it should take to + make the literal true. + >>> inspect_literal(P) + (P, True) + >>> inspect_literal(~P) + (P, False) + """ + if literal.op == '~': + return literal.args[0], False + else: + return literal, True + + +# ______________________________________________________________________________ +# 7.6.2 Local search algorithms +# Walk-SAT [Figure 7.18] + + +def WalkSAT(clauses, p=0.5, max_flips=10000): + """ + Checks for satisfiability of all clauses by randomly flipping values of variables + >>> WalkSAT([A & ~A], 0.5, 100) is None + True + """ + # Set of all symbols in all clauses + symbols = {sym for clause in clauses for sym in prop_symbols(clause)} + # model is a random assignment of true/false to the symbols in clauses + model = {s: random.choice([True, False]) for s in symbols} + for i in range(max_flips): + satisfied, unsatisfied = [], [] + for clause in clauses: + (satisfied if pl_true(clause, model) else unsatisfied).append(clause) + if not unsatisfied: # if model satisfies all the clauses + return model + clause = random.choice(unsatisfied) + if probability(p): + sym = random.choice(list(prop_symbols(clause))) + else: + # Flip the symbol in clause that maximizes number of sat. clauses + def sat_count(sym): + # Return the the number of clauses satisfied after flipping the symbol. + model[sym] = not model[sym] + count = len([clause for clause in clauses if pl_true(clause, model)]) + model[sym] = not model[sym] + return count + + sym = max(prop_symbols(clause), key=sat_count) + model[sym] = not model[sym] + # If no solution is found within the flip limit, we return failure + return None + + +# ______________________________________________________________________________ +# 7.7 Agents Based on Propositional Logic +# 7.7.1 The current state of the world + + +class WumpusKB(PropKB): + """ + Create a Knowledge Base that contains the atemporal "Wumpus physics" and temporal rules with time zero. + """ + + def __init__(self, dimrow): + super().__init__() + self.dimrow = dimrow + self.tell(~wumpus(1, 1)) + self.tell(~pit(1, 1)) + + for y in range(1, dimrow + 1): + for x in range(1, dimrow + 1): + + pits_in = list() + wumpus_in = list() + + if x > 1: # West room exists + pits_in.append(pit(x - 1, y)) + wumpus_in.append(wumpus(x - 1, y)) + + if y < dimrow: # North room exists + pits_in.append(pit(x, y + 1)) + wumpus_in.append(wumpus(x, y + 1)) + + if x < dimrow: # East room exists + pits_in.append(pit(x + 1, y)) + wumpus_in.append(wumpus(x + 1, y)) + + if y > 1: # South room exists + pits_in.append(pit(x, y - 1)) + wumpus_in.append(wumpus(x, y - 1)) + + self.tell(equiv(breeze(x, y), new_disjunction(pits_in))) + self.tell(equiv(stench(x, y), new_disjunction(wumpus_in))) + + # Rule that describes existence of at least one Wumpus + wumpus_at_least = list() + for x in range(1, dimrow + 1): + for y in range(1, dimrow + 1): + wumpus_at_least.append(wumpus(x, y)) + + self.tell(new_disjunction(wumpus_at_least)) + + # Rule that describes existence of at most one Wumpus + for i in range(1, dimrow + 1): + for j in range(1, dimrow + 1): + for u in range(1, dimrow + 1): + for v in range(1, dimrow + 1): + if i != u or j != v: + self.tell(~wumpus(i, j) | ~wumpus(u, v)) + + # Temporal rules at time zero + self.tell(location(1, 1, 0)) + for i in range(1, dimrow + 1): + for j in range(1, dimrow + 1): + self.tell(implies(location(i, j, 0), equiv(percept_breeze(0), breeze(i, j)))) + self.tell(implies(location(i, j, 0), equiv(percept_stench(0), stench(i, j)))) + if i != 1 or j != 1: + self.tell(~location(i, j, 0)) + + self.tell(wumpus_alive(0)) + self.tell(have_arrow(0)) + self.tell(facing_east(0)) + self.tell(~facing_north(0)) + self.tell(~facing_south(0)) + self.tell(~facing_west(0)) + + def make_action_sentence(self, action, time): + actions = [move_forward(time), shoot(time), turn_left(time), turn_right(time)] + + for a in actions: + if action is a: + self.tell(action) + else: + self.tell(~a) + + def make_percept_sentence(self, percept, time): + # Glitter, Bump, Stench, Breeze, Scream + flags = [0, 0, 0, 0, 0] + + # Things perceived + if isinstance(percept, Glitter): + flags[0] = 1 + self.tell(percept_glitter(time)) + elif isinstance(percept, Bump): + flags[1] = 1 + self.tell(percept_bump(time)) + elif isinstance(percept, Stench): + flags[2] = 1 + self.tell(percept_stench(time)) + elif isinstance(percept, Breeze): + flags[3] = 1 + self.tell(percept_breeze(time)) + elif isinstance(percept, Scream): + flags[4] = 1 + self.tell(percept_scream(time)) + + # Things not perceived + for i in range(len(flags)): + if flags[i] == 0: + if i == 0: + self.tell(~percept_glitter(time)) + elif i == 1: + self.tell(~percept_bump(time)) + elif i == 2: + self.tell(~percept_stench(time)) + elif i == 3: + self.tell(~percept_breeze(time)) + elif i == 4: + self.tell(~percept_scream(time)) + + def add_temporal_sentences(self, time): + if time == 0: + return + t = time - 1 + + # current location rules + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + self.tell(implies(location(i, j, time), equiv(percept_breeze(time), breeze(i, j)))) + self.tell(implies(location(i, j, time), equiv(percept_stench(time), stench(i, j)))) + + s = list() + + s.append( + equiv( + location(i, j, time), location(i, j, time) & ~move_forward(time) | percept_bump(time))) + + if i != 1: + s.append(location(i - 1, j, t) & facing_east(t) & move_forward(t)) + + if i != self.dimrow: + s.append(location(i + 1, j, t) & facing_west(t) & move_forward(t)) + + if j != 1: + s.append(location(i, j - 1, t) & facing_north(t) & move_forward(t)) + + if j != self.dimrow: + s.append(location(i, j + 1, t) & facing_south(t) & move_forward(t)) + + # add sentence about location i,j + self.tell(new_disjunction(s)) + + # add sentence about safety of location i,j + self.tell( + equiv(ok_to_move(i, j, time), ~pit(i, j) & ~wumpus(i, j) & wumpus_alive(time)) + ) + + # Rules about current orientation + + a = facing_north(t) & turn_right(t) + b = facing_south(t) & turn_left(t) + c = facing_east(t) & ~turn_left(t) & ~turn_right(t) + s = equiv(facing_east(time), a | b | c) + self.tell(s) + + a = facing_north(t) & turn_left(t) + b = facing_south(t) & turn_right(t) + c = facing_west(t) & ~turn_left(t) & ~turn_right(t) + s = equiv(facing_west(time), a | b | c) + self.tell(s) + + a = facing_east(t) & turn_left(t) + b = facing_west(t) & turn_right(t) + c = facing_north(t) & ~turn_left(t) & ~turn_right(t) + s = equiv(facing_north(time), a | b | c) + self.tell(s) + + a = facing_west(t) & turn_left(t) + b = facing_east(t) & turn_right(t) + c = facing_south(t) & ~turn_left(t) & ~turn_right(t) + s = equiv(facing_south(time), a | b | c) + self.tell(s) + + # Rules about last action + self.tell(equiv(move_forward(t), ~turn_right(t) & ~turn_left(t))) + + # Rule about the arrow + self.tell(equiv(have_arrow(time), have_arrow(t) & ~shoot(t))) + + # Rule about Wumpus (dead or alive) + self.tell(equiv(wumpus_alive(time), wumpus_alive(t) & ~percept_scream(time))) + + def ask_if_true(self, query): + return pl_resolution(self, query) + + +# ______________________________________________________________________________ + + +class WumpusPosition: + def __init__(self, x, y, orientation): + self.X = x + self.Y = y + self.orientation = orientation + + def get_location(self): + return self.X, self.Y + + def set_location(self, x, y): + self.X = x + self.Y = y + + def get_orientation(self): + return self.orientation + + def set_orientation(self, orientation): + self.orientation = orientation + + def __eq__(self, other): + if (other.get_location() == self.get_location() and + other.get_orientation() == self.get_orientation()): + return True + else: + return False + + +# ______________________________________________________________________________ +# 7.7.2 A hybrid agent + + +class HybridWumpusAgent(Agent): + """An agent for the wumpus world that does logical inference. [Figure 7.20]""" + + def __init__(self, dimentions): + self.dimrow = dimentions + self.kb = WumpusKB(self.dimrow) + self.t = 0 + self.plan = list() + self.current_position = WumpusPosition(1, 1, 'UP') + super().__init__(self.execute) + + def execute(self, percept): + self.kb.make_percept_sentence(percept, self.t) + self.kb.add_temporal_sentences(self.t) + + temp = list() + + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + if self.kb.ask_if_true(location(i, j, self.t)): + temp.append(i) + temp.append(j) + + if self.kb.ask_if_true(facing_north(self.t)): + self.current_position = WumpusPosition(temp[0], temp[1], 'UP') + elif self.kb.ask_if_true(facing_south(self.t)): + self.current_position = WumpusPosition(temp[0], temp[1], 'DOWN') + elif self.kb.ask_if_true(facing_west(self.t)): + self.current_position = WumpusPosition(temp[0], temp[1], 'LEFT') + elif self.kb.ask_if_true(facing_east(self.t)): + self.current_position = WumpusPosition(temp[0], temp[1], 'RIGHT') + + safe_points = list() + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + if self.kb.ask_if_true(ok_to_move(i, j, self.t)): + safe_points.append([i, j]) + + if self.kb.ask_if_true(percept_glitter(self.t)): + goals = list() + goals.append([1, 1]) + self.plan.append('Grab') + actions = self.plan_route(self.current_position, goals, safe_points) + self.plan.extend(actions) + self.plan.append('Climb') + + if len(self.plan) == 0: + unvisited = list() + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + for k in range(self.t): + if self.kb.ask_if_true(location(i, j, k)): + unvisited.append([i, j]) + unvisited_and_safe = list() + for u in unvisited: + for s in safe_points: + if u not in unvisited_and_safe and s == u: + unvisited_and_safe.append(u) + + temp = self.plan_route(self.current_position, unvisited_and_safe, safe_points) + self.plan.extend(temp) + + if len(self.plan) == 0 and self.kb.ask_if_true(have_arrow(self.t)): + possible_wumpus = list() + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + if not self.kb.ask_if_true(wumpus(i, j)): + possible_wumpus.append([i, j]) + + temp = self.plan_shot(self.current_position, possible_wumpus, safe_points) + self.plan.extend(temp) + + if len(self.plan) == 0: + not_unsafe = list() + for i in range(1, self.dimrow + 1): + for j in range(1, self.dimrow + 1): + if not self.kb.ask_if_true(ok_to_move(i, j, self.t)): + not_unsafe.append([i, j]) + temp = self.plan_route(self.current_position, not_unsafe, safe_points) + self.plan.extend(temp) + + if len(self.plan) == 0: + start = list() + start.append([1, 1]) + temp = self.plan_route(self.current_position, start, safe_points) + self.plan.extend(temp) + self.plan.append('Climb') + + action = self.plan[0] + self.plan = self.plan[1:] + self.kb.make_action_sentence(action, self.t) + self.t += 1 + + return action + + def plan_route(self, current, goals, allowed): + problem = PlanRoute(current, goals, allowed, self.dimrow) + return astar_search(problem).solution() + + def plan_shot(self, current, goals, allowed): + shooting_positions = set() + + for loc in goals: + x = loc[0] + y = loc[1] + for i in range(1, self.dimrow + 1): + if i < x: + shooting_positions.add(WumpusPosition(i, y, 'EAST')) + if i > x: + shooting_positions.add(WumpusPosition(i, y, 'WEST')) + if i < y: + shooting_positions.add(WumpusPosition(x, i, 'NORTH')) + if i > y: + shooting_positions.add(WumpusPosition(x, i, 'SOUTH')) + + # Can't have a shooting position from any of the rooms the Wumpus could reside + orientations = ['EAST', 'WEST', 'NORTH', 'SOUTH'] + for loc in goals: + for orientation in orientations: + shooting_positions.remove(WumpusPosition(loc[0], loc[1], orientation)) + + actions = list() + actions.extend(self.plan_route(current, shooting_positions, allowed)) + actions.append('Shoot') + return actions + + +# ______________________________________________________________________________ +# 7.7.4 Making plans by propositional inference + + +def SAT_plan(init, transition, goal, t_max, SAT_solver=dpll_satisfiable): + """Converts a planning problem to Satisfaction problem by translating it to a cnf sentence. + [Figure 7.22] + >>> transition = {'A': {'Left': 'A', 'Right': 'B'}, 'B': {'Left': 'A', 'Right': 'C'}, 'C': {'Left': 'B', 'Right': 'C'}} + >>> SAT_plan('A', transition, 'C', 2) is None + True + """ + + # Functions used by SAT_plan + def translate_to_SAT(init, transition, goal, time): + clauses = [] + states = [state for state in transition] + + # Symbol claiming state s at time t + state_counter = itertools.count() + for s in states: + for t in range(time + 1): + state_sym[s, t] = Expr("State_{}".format(next(state_counter))) + + # Add initial state axiom + clauses.append(state_sym[init, 0]) + + # Add goal state axiom + clauses.append(state_sym[goal, time]) + + # All possible transitions + transition_counter = itertools.count() + for s in states: + for action in transition[s]: + s_ = transition[s][action] + for t in range(time): + # Action 'action' taken from state 's' at time 't' to reach 's_' + action_sym[s, action, t] = Expr( + "Transition_{}".format(next(transition_counter))) + + # Change the state from s to s_ + clauses.append(action_sym[s, action, t] | '==>' | state_sym[s, t]) + clauses.append(action_sym[s, action, t] | '==>' | state_sym[s_, t + 1]) + + # Allow only one state at any time + for t in range(time + 1): + # must be a state at any time + clauses.append(associate('|', [state_sym[s, t] for s in states])) + + for s in states: + for s_ in states[states.index(s) + 1:]: + # for each pair of states s, s_ only one is possible at time t + clauses.append((~state_sym[s, t]) | (~state_sym[s_, t])) + + # Restrict to one transition per timestep + for t in range(time): + # list of possible transitions at time t + transitions_t = [tr for tr in action_sym if tr[2] == t] + + # make sure at least one of the transitions happens + clauses.append(associate('|', [action_sym[tr] for tr in transitions_t])) + + for tr in transitions_t: + for tr_ in transitions_t[transitions_t.index(tr) + 1:]: + # there cannot be two transitions tr and tr_ at time t + clauses.append(~action_sym[tr] | ~action_sym[tr_]) + + # Combine the clauses to form the cnf + return associate('&', clauses) + + def extract_solution(model): + true_transitions = [t for t in action_sym if model[action_sym[t]]] + # Sort transitions based on time, which is the 3rd element of the tuple + true_transitions.sort(key=lambda x: x[2]) + return [action for s, action, time in true_transitions] + + # Body of SAT_plan algorithm + for t in range(t_max): + # dictionaries to help extract the solution from model + state_sym = {} + action_sym = {} + + cnf = translate_to_SAT(init, transition, goal, t) + model = SAT_solver(cnf) + if model is not False: + return extract_solution(model) + return None + + +# ______________________________________________________________________________ +# Chapter 9 Inference in First Order Logic +# 9.2 Unification and First Order Inference +# 9.2.1 Unification + + +def unify(x, y, s={}): + """Unify expressions x,y with substitution s; return a substitution that + would make x,y equal, or None if x,y can not unify. x and y can be + variables (e.g. Expr('x')), constants, lists, or Exprs. [Figure 9.1] + >>> unify(x, 3, {}) + {x: 3} + """ + if s is None: + return None + elif x == y: + return s + elif is_variable(x): + return unify_var(x, y, s) + elif is_variable(y): + return unify_var(y, x, s) + elif isinstance(x, Expr) and isinstance(y, Expr): + return unify(x.args, y.args, unify(x.op, y.op, s)) + elif isinstance(x, str) or isinstance(y, str): + return None + elif issequence(x) and issequence(y) and len(x) == len(y): + if not x: + return s + return unify(x[1:], y[1:], unify(x[0], y[0], s)) + else: + return None + + +def is_variable(x): + """A variable is an Expr with no args and a lowercase symbol as the op.""" + return isinstance(x, Expr) and not x.args and x.op[0].islower() + + +def unify_var(var, x, s): + if var in s: + return unify(s[var], x, s) + elif x in s: + return unify(var, s[x], s) + elif occur_check(var, x, s): + return None + else: + return extend(s, var, x) + + +def occur_check(var, x, s): + """Return true if variable var occurs anywhere in x + (or in subst(s, x), if s has a binding for x).""" + if var == x: + return True + elif is_variable(x) and x in s: + return occur_check(var, s[x], s) + elif isinstance(x, Expr): + return (occur_check(var, x.op, s) or + occur_check(var, x.args, s)) + elif isinstance(x, (list, tuple)): + return first(e for e in x if occur_check(var, e, s)) + else: + return False + + +def extend(s, var, val): + """Copy the substitution s and extend it by setting var to val; return copy. + >>> extend({x: 1}, y, 2) == {x: 1, y: 2} + True + """ + s2 = s.copy() + s2[var] = val + return s2 + + +# 9.2.2 Storage and retrieval + + +class FolKB(KB): + """A knowledge base consisting of first-order definite clauses. + >>> kb0 = FolKB([expr('Farmer(Mac)'), expr('Rabbit(Pete)'), + ... expr('(Rabbit(r) & Farmer(f)) ==> Hates(f, r)')]) + >>> kb0.tell(expr('Rabbit(Flopsie)')) + >>> kb0.retract(expr('Rabbit(Pete)')) + >>> kb0.ask(expr('Hates(Mac, x)'))[x] + Flopsie + >>> kb0.ask(expr('Wife(Pete, x)')) + False + """ + + def __init__(self, initial_clauses=None): + self.clauses = [] # inefficient: no indexing + if initial_clauses: + for clause in initial_clauses: + self.tell(clause) + + def tell(self, sentence): + if is_definite_clause(sentence): + self.clauses.append(sentence) + else: + raise Exception("Not a definite clause: {}".format(sentence)) + + def ask_generator(self, query): + return fol_bc_ask(self, query) + + def retract(self, sentence): + self.clauses.remove(sentence) + + def fetch_rules_for_goal(self, goal): + return self.clauses + + +# ______________________________________________________________________________ +# 9.3 Forward Chaining +# 9.3.2 A simple forward-chaining algorithm + + +def fol_fc_ask(KB, alpha): + """A simple forward-chaining algorithm. [Figure 9.3]""" + kb_consts = list({c for clause in KB.clauses for c in constant_symbols(clause)}) + + def enum_subst(p): + query_vars = list({v for clause in p for v in variables(clause)}) + for assignment_list in itertools.product(kb_consts, repeat=len(query_vars)): + theta = {x: y for x, y in zip(query_vars, assignment_list)} + yield theta + + # check if we can answer without new inferences + for q in KB.clauses: + phi = unify(q, alpha, {}) + if phi is not None: + yield phi + + while True: + new = [] + for rule in KB.clauses: + p, q = parse_definite_clause(rule) + for theta in enum_subst(p): + if set(subst(theta, p)).issubset(set(KB.clauses)): + q_ = subst(theta, q) + if all([unify(x, q_, {}) is None for x in KB.clauses + new]): + new.append(q_) + phi = unify(q_, alpha, {}) + if phi is not None: + yield phi + if not new: + break + for clause in new: + KB.tell(clause) + return None + + +def subst(s, x): + """Substitute the substitution s into the expression x. + >>> subst({x: 42, y:0}, F(x) + y) + (F(42) + 0) + """ + if isinstance(x, list): + return [subst(s, xi) for xi in x] + elif isinstance(x, tuple): + return tuple([subst(s, xi) for xi in x]) + elif not isinstance(x, Expr): + return x + elif is_var_symbol(x.op): + return s.get(x, x) + else: + return Expr(x.op, *[subst(s, arg) for arg in x.args]) + + +def standardize_variables(sentence, dic=None): + """Replace all the variables in sentence with new variables.""" + if dic is None: + dic = {} + if not isinstance(sentence, Expr): + return sentence + elif is_var_symbol(sentence.op): + if sentence in dic: + return dic[sentence] + else: + v = Expr('v_{}'.format(next(standardize_variables.counter))) + dic[sentence] = v + return v + else: + return Expr(sentence.op, + *[standardize_variables(a, dic) for a in sentence.args]) + + +standardize_variables.counter = itertools.count() + + +# __________________________________________________________________ +# 9.4 Backward Chaining + + +def fol_bc_ask(KB, query): + """A simple backward-chaining algorithm for first-order logic. [Figure 9.6] + KB should be an instance of FolKB, and query an atomic sentence.""" + return fol_bc_or(KB, query, {}) + + +def fol_bc_or(KB, goal, theta): + for rule in KB.fetch_rules_for_goal(goal): + lhs, rhs = parse_definite_clause(standardize_variables(rule)) + for theta1 in fol_bc_and(KB, lhs, unify(rhs, goal, theta)): + yield theta1 + + +def fol_bc_and(KB, goals, theta): + if theta is None: + pass + elif not goals: + yield theta + else: + first, rest = goals[0], goals[1:] + for theta1 in fol_bc_or(KB, subst(theta, first), theta): + for theta2 in fol_bc_and(KB, rest, theta1): + yield theta2 + + +# ______________________________________________________________________________ +# A simple KB that defines the relevant conditions of the Wumpus World as in Fig 7.4. +# See Sec. 7.4.3 +wumpus_kb = PropKB() + +P11, P12, P21, P22, P31, B11, B21 = expr('P11, P12, P21, P22, P31, B11, B21') +wumpus_kb.tell(~P11) +wumpus_kb.tell(B11 | '<=>' | (P12 | P21)) +wumpus_kb.tell(B21 | '<=>' | (P11 | P22 | P31)) +wumpus_kb.tell(~B11) +wumpus_kb.tell(B21) + +test_kb = FolKB( + map(expr, ['Farmer(Mac)', + 'Rabbit(Pete)', + 'Mother(MrsMac, Mac)', + 'Mother(MrsRabbit, Pete)', + '(Rabbit(r) & Farmer(f)) ==> Hates(f, r)', + '(Mother(m, c)) ==> Loves(m, c)', + '(Mother(m, r) & Rabbit(r)) ==> Rabbit(m)', + '(Farmer(f)) ==> Human(f)', + # Note that this order of conjuncts + # would result in infinite recursion: + # '(Human(h) & Mother(m, h)) ==> Human(m)' + '(Mother(m, h) & Human(h)) ==> Human(m)'])) + +crime_kb = FolKB( + map(expr, ['(American(x) & Weapon(y) & Sells(x, y, z) & Hostile(z)) ==> Criminal(x)', + 'Owns(Nono, M1)', + 'Missile(M1)', + '(Missile(x) & Owns(Nono, x)) ==> Sells(West, x, Nono)', + 'Missile(x) ==> Weapon(x)', + 'Enemy(x, America) ==> Hostile(x)', + 'American(West)', + 'Enemy(Nono, America)'])) + + +# ______________________________________________________________________________ + +# Example application (not in the book). +# You can use the Expr class to do symbolic differentiation. This used to be +# a part of AI; now it is considered a separate field, Symbolic Algebra. + + +def diff(y, x): + """Return the symbolic derivative, dy/dx, as an Expr. + However, you probably want to simplify the results with simp. + >>> diff(x * x, x) + ((x * 1) + (x * 1)) + """ + if y == x: + return 1 + elif not y.args: + return 0 + else: + u, op, v = y.args[0], y.op, y.args[-1] + if op == '+': + return diff(u, x) + diff(v, x) + elif op == '-' and len(y.args) == 1: + return -diff(u, x) + elif op == '-': + return diff(u, x) - diff(v, x) + elif op == '*': + return u * diff(v, x) + v * diff(u, x) + elif op == '/': + return (v * diff(u, x) - u * diff(v, x)) / (v * v) + elif op == '**' and isnumber(x.op): + return (v * u ** (v - 1) * diff(u, x)) + elif op == '**': + return (v * u ** (v - 1) * diff(u, x) + + u ** v * Expr('log')(u) * diff(v, x)) + elif op == 'log': + return diff(u, x) / u + else: + raise ValueError("Unknown op: {} in diff({}, {})".format(op, y, x)) + + +def simp(x): + """Simplify the expression x.""" + if isnumber(x) or not x.args: + return x + args = list(map(simp, x.args)) + u, op, v = args[0], x.op, args[-1] + if op == '+': + if v == 0: + return u + if u == 0: + return v + if u == v: + return 2 * u + if u == -v or v == -u: + return 0 + elif op == '-' and len(args) == 1: + if u.op == '-' and len(u.args) == 1: + return u.args[0] # --y ==> y + elif op == '-': + if v == 0: + return u + if u == 0: + return -v + if u == v: + return 0 + if u == -v or v == -u: + return 0 + elif op == '*': + if u == 0 or v == 0: + return 0 + if u == 1: + return v + if v == 1: + return u + if u == v: + return u ** 2 + elif op == '/': + if u == 0: + return 0 + if v == 0: + return Expr('Undefined') + if u == v: + return 1 + if u == -v or v == -u: + return 0 + elif op == '**': + if u == 0: + return 0 + if v == 0: + return 1 + if u == 1: + return 1 + if v == 1: + return u + elif op == 'log': + if u == 1: + return 0 + else: + raise ValueError("Unknown op: " + op) + # If we fall through to here, we can not simplify further + return Expr(op, *args) + + +def d(y, x): + """Differentiate and then simplify. + >>> d(x * x - x, x) + ((2 * x) - 1) + """ + return simp(diff(y, x)) diff --git a/making_simple_decision4e.py b/making_simple_decision4e.py new file mode 100644 index 000000000..4a35f94bd --- /dev/null +++ b/making_simple_decision4e.py @@ -0,0 +1,168 @@ +"""Making Simple Decisions (Chapter 15)""" + +import random + +from agents import Agent +from probability import BayesNet +from utils4e import vector_add, weighted_sample_with_replacement + + +class DecisionNetwork(BayesNet): + """An abstract class for a decision network as a wrapper for a BayesNet. + Represents an agent's current state, its possible actions, reachable states + and utilities of those states.""" + + def __init__(self, action, infer): + """action: a single action node + infer: the preferred method to carry out inference on the given BayesNet""" + super().__init__() + self.action = action + self.infer = infer + + def best_action(self): + """Return the best action in the network""" + return self.action + + def get_utility(self, action, state): + """Return the utility for a particular action and state in the network""" + raise NotImplementedError + + def get_expected_utility(self, action, evidence): + """Compute the expected utility given an action and evidence""" + u = 0.0 + prob_dist = self.infer(action, evidence, self).prob + for item, _ in prob_dist.items(): + u += prob_dist[item] * self.get_utility(action, item) + + return u + + +class InformationGatheringAgent(Agent): + """A simple information gathering agent. The agent works by repeatedly selecting + the observation with the highest information value, until the cost of the next + observation is greater than its expected benefit. [Figure 16.9]""" + + def __init__(self, decnet, infer, initial_evidence=None): + """decnet: a decision network + infer: the preferred method to carry out inference on the given decision network + initial_evidence: initial evidence""" + super().__init__() + self.decnet = decnet + self.infer = infer + self.observation = initial_evidence or [] + self.variables = self.decnet.nodes + + def integrate_percept(self, percept): + """Integrate the given percept into the decision network""" + raise NotImplementedError + + def execute(self, percept): + """Execute the information gathering algorithm""" + self.observation = self.integrate_percept(percept) + vpis = self.vpi_cost_ratio(self.variables) + j = max(vpis) + variable = self.variables[j] + + if self.vpi(variable) > self.cost(variable): + return self.request(variable) + + return self.decnet.best_action() + + def request(self, variable): + """Return the value of the given random variable as the next percept""" + raise NotImplementedError + + def cost(self, var): + """Return the cost of obtaining evidence through tests, consultants or questions""" + raise NotImplementedError + + def vpi_cost_ratio(self, variables): + """Return the VPI to cost ratio for the given variables""" + v_by_c = [] + for var in variables: + v_by_c.append(self.vpi(var) / self.cost(var)) + return v_by_c + + def vpi(self, variable): + """Return VPI for a given variable""" + vpi = 0.0 + prob_dist = self.infer(variable, self.observation, self.decnet).prob + for item, _ in prob_dist.items(): + post_prob = prob_dist[item] + new_observation = list(self.observation) + new_observation.append(item) + expected_utility = self.decnet.get_expected_utility(variable, new_observation) + vpi += post_prob * expected_utility + + vpi -= self.decnet.get_expected_utility(variable, self.observation) + return vpi + + +# _________________________________________________________________________ +# chapter 25 Robotics +# TODO: Implement continuous map for MonteCarlo similar to Fig25.10 from the book + + +class MCLmap: + """Map which provides probability distributions and sensor readings. + Consists of discrete cells which are either an obstacle or empty""" + + def __init__(self, m): + self.m = m + self.nrows = len(m) + self.ncols = len(m[0]) + # list of empty spaces in the map + self.empty = [(i, j) for i in range(self.nrows) for j in range(self.ncols) if not m[i][j]] + + def sample(self): + """Returns a random kinematic state possible in the map""" + pos = random.choice(self.empty) + # 0N 1E 2S 3W + orient = random.choice(range(4)) + kin_state = pos + (orient,) + return kin_state + + def ray_cast(self, sensor_num, kin_state): + """Returns distace to nearest obstacle or map boundary in the direction of sensor""" + pos = kin_state[:2] + orient = kin_state[2] + # sensor layout when orientation is 0 (towards North) + # 0 + # 3R1 + # 2 + delta = ((sensor_num % 2 == 0) * (sensor_num - 1), (sensor_num % 2 == 1) * (2 - sensor_num)) + # sensor direction changes based on orientation + for _ in range(orient): + delta = (delta[1], -delta[0]) + range_count = 0 + while (0 <= pos[0] < self.nrows) and (0 <= pos[1] < self.nrows) and (not self.m[pos[0]][pos[1]]): + pos = vector_add(pos, delta) + range_count += 1 + return range_count + + +def monte_carlo_localization(a, z, N, P_motion_sample, P_sensor, m, S=None): + """Monte Carlo localization algorithm from Fig 25.9""" + + def ray_cast(sensor_num, kin_state, m): + return m.ray_cast(sensor_num, kin_state) + + M = len(z) + W = [0] * N + S_ = [0] * N + W_ = [0] * N + v = a['v'] + w = a['w'] + + if S is None: + S = [m.sample() for _ in range(N)] + + for i in range(N): + S_[i] = P_motion_sample(S[i], v, w) + W_[i] = 1 + for j in range(M): + z_ = ray_cast(j, S_[i], m) + W_[i] = W_[i] * P_sensor(z[j], z_) + + S = weighted_sample_with_replacement(N, S_, W_) + return S diff --git a/mdp.py b/mdp.py index 657334d59..1003e26b5 100644 --- a/mdp.py +++ b/mdp.py @@ -1,27 +1,29 @@ -"""Markov Decision Processes (Chapter 17) +""" +Markov Decision Processes (Chapter 17) First we define an MDP, and the special case of a GridMDP, in which states are laid out in a 2-dimensional grid. We also represent a policy as a dictionary of {state: action} pairs, and a Utility function as a dictionary of {state: number} pairs. We then define the value_iteration -and policy_iteration algorithms.""" - -from utils import argmax, vector_add, orientations, turn_right, turn_left +and policy_iteration algorithms. +""" import random -import numpy as np from collections import defaultdict +import numpy as np -class MDP: +from utils import vector_add, orientations, turn_right, turn_left + +class MDP: """A Markov Decision Process, defined by an initial state, transition model, and reward function. We also keep track of a gamma value, for use by algorithms. The transition model is represented somewhat differently from the text. Instead of P(s' | s, a) being a probability number for each state/state/action triplet, we instead have T(s, a) return a list of (p, s') pairs. We also keep track of the possible states, - terminal states, and actions for each state. [page 646]""" + terminal states, and actions for each state. [Page 646]""" def __init__(self, init, actlist, terminals, transitions=None, reward=None, states=None, gamma=0.9): if not (0 < gamma <= 1): @@ -29,9 +31,9 @@ def __init__(self, init, actlist, terminals, transitions=None, reward=None, stat # collect states from transitions table if not passed. self.states = states or self.get_states_from_transitions(transitions) - + self.init = init - + if isinstance(actlist, list): # if actlist is a list, all states have the same actions self.actlist = actlist @@ -39,7 +41,7 @@ def __init__(self, init, actlist, terminals, transitions=None, reward=None, stat elif isinstance(actlist, dict): # if actlist is a dict, different actions for each state self.actlist = actlist - + self.terminals = terminals self.transitions = transitions or {} if not self.transitions: @@ -110,7 +112,6 @@ def check_consistency(self): class MDP2(MDP): - """ Inherits from MDP. Handles terminal states, and transitions to and from terminal states better. """ @@ -126,14 +127,13 @@ def T(self, state, action): class GridMDP(MDP): - """A two-dimensional grid MDP, as in [Figure 17.1]. All you have to do is specify the grid as a list of lists of rewards; use None for an obstacle (unreachable state). Also, you should specify the terminal states. An action is an (x, y) unit vector; e.g. (1, 0) means move east.""" def __init__(self, grid, terminals, init=(0, 0), gamma=.9): - grid.reverse() # because we want row 0 on bottom, not on top + grid.reverse() # because we want row 0 on bottom, not on top reward = {} states = set() self.rows = len(grid) @@ -152,7 +152,7 @@ def __init__(self, grid, terminals, init=(0, 0), gamma=.9): for a in actlist: transitions[s][a] = self.calculate_T(s, a) MDP.__init__(self, init, actlist=actlist, - terminals=terminals, transitions=transitions, + terminals=terminals, transitions=transitions, reward=reward, states=states, gamma=gamma) def calculate_T(self, state, action): @@ -162,10 +162,10 @@ def calculate_T(self, state, action): (0.1, self.go(state, turn_left(action)))] else: return [(0.0, state)] - + def T(self, state, action): return self.transitions[state][action] if action else [(0.0, state)] - + def go(self, state, direction): """Return the state that results from going in this direction.""" @@ -183,6 +183,7 @@ def to_arrows(self, policy): chars = {(1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'} return self.to_grid({s: chars[a] for (s, a) in policy.items()}) + # ______________________________________________________________________________ @@ -195,6 +196,7 @@ def to_arrows(self, policy): [-0.04, -0.04, -0.04, -0.04]], terminals=[(3, 2), (3, 1)]) + # ______________________________________________________________________________ @@ -207,27 +209,28 @@ def value_iteration(mdp, epsilon=0.001): U = U1.copy() delta = 0 for s in mdp.states: - U1[s] = R(s) + gamma * max(sum(p*U[s1] for (p, s1) in T(s, a)) - for a in mdp.actions(s)) + U1[s] = R(s) + gamma * max(sum(p * U[s1] for (p, s1) in T(s, a)) + for a in mdp.actions(s)) delta = max(delta, abs(U1[s] - U[s])) - if delta <= epsilon*(1 - gamma)/gamma: + if delta <= epsilon * (1 - gamma) / gamma: return U def best_policy(mdp, U): """Given an MDP and a utility function U, determine the best policy, - as a mapping from state to action. (Equation 17.4)""" + as a mapping from state to action. [Equation 17.4]""" pi = {} for s in mdp.states: - pi[s] = argmax(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) + pi[s] = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) return pi def expected_utility(a, s, U, mdp): """The expected utility of doing a in state s, according to the MDP and U.""" - return sum(p*U[s1] for (p, s1) in mdp.T(s, a)) + return sum(p * U[s1] for (p, s1) in mdp.T(s, a)) + # ______________________________________________________________________________ @@ -241,7 +244,7 @@ def policy_iteration(mdp): U = policy_evaluation(pi, U, mdp) unchanged = True for s in mdp.states: - a = argmax(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) + a = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) if a != pi[s]: pi[s] = a unchanged = False @@ -256,18 +259,17 @@ def policy_evaluation(pi, U, mdp, k=20): R, T, gamma = mdp.R, mdp.T, mdp.gamma for i in range(k): for s in mdp.states: - U[s] = R(s) + gamma*sum(p*U[s1] for (p, s1) in T(s, pi[s])) + U[s] = R(s) + gamma * sum(p * U[s1] for (p, s1) in T(s, pi[s])) return U class POMDP(MDP): - """A Partially Observable Markov Decision Process, defined by a transition model P(s'|s,a), actions A(s), a reward function R(s), and a sensor model P(e|s). We also keep track of a gamma value, for use by algorithms. The transition and the sensor models are defined as matrices. We also keep track of the possible states - and actions for each state. [page 659].""" + and actions for each state. [Page 659].""" def __init__(self, actions, transitions=None, evidences=None, rewards=None, states=None, gamma=0.95): """Initialize variables of the pomdp""" @@ -282,12 +284,12 @@ def __init__(self, actions, transitions=None, evidences=None, rewards=None, stat self.t_prob = transitions or {} if not self.t_prob: print('Warning: Transition model is undefined') - + # sensor model cannot be undefined self.e_prob = evidences or {} if not self.e_prob: print('Warning: Sensor model is undefined') - + self.gamma = gamma self.rewards = rewards @@ -372,7 +374,7 @@ def max_difference(self, U1, U2): sum2 += sum(element) return abs(sum1 - sum2) - + class Matrix: """Matrix operations class""" @@ -414,19 +416,19 @@ def multiply(A, B): def matmul(A, B): """Inner-product of two matrices""" - return [[sum(ele_a*ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A] + return [[sum(ele_a * ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A] @staticmethod def transpose(A): """Transpose a matrix""" - + return [list(i) for i in zip(*A)] def pomdp_value_iteration(pomdp, epsilon=0.1): """Solving a POMDP by value iteration.""" - U = {'':[[0]* len(pomdp.states)]} + U = {'': [[0] * len(pomdp.states)]} count = 0 while True: count += 1 @@ -440,13 +442,15 @@ def pomdp_value_iteration(pomdp, epsilon=0.1): U1 = defaultdict(list) for action in pomdp.actions: for u in value_matxs: - u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), [[1], [1]]) + u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], + Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), + [[1], [1]]) u1 = Matrix.add(Matrix.scalar_multiply(pomdp.gamma, Matrix.transpose(u1)), [pomdp.rewards[int(action)]]) U1[action].append(u1[0]) U = pomdp.remove_dominated_plans_fast(U1) # replace with U = pomdp.remove_dominated_plans(U1) for accurate calculations - + if count > 10: if pomdp.max_difference(U, prev_U) < epsilon * (1 - pomdp.gamma) / pomdp.gamma: return U @@ -473,16 +477,16 @@ def pomdp_value_iteration(pomdp, epsilon=0.1): """ s = { 'a' : { 'plan1' : [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], - 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], - 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], - }, - 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], - 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], - }, - 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], - }, - } + 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }, + } """ diff --git a/mdp4e.py b/mdp4e.py new file mode 100644 index 000000000..f8871bdc9 --- /dev/null +++ b/mdp4e.py @@ -0,0 +1,516 @@ +""" +Markov Decision Processes (Chapter 16) + +First we define an MDP, and the special case of a GridMDP, in which +states are laid out in a 2-dimensional grid. We also represent a policy +as a dictionary of {state: action} pairs, and a Utility function as a +dictionary of {state: number} pairs. We then define the value_iteration +and policy_iteration algorithms. +""" + +import random +from collections import defaultdict + +import numpy as np + +from utils4e import vector_add, orientations, turn_right, turn_left + + +class MDP: + """A Markov Decision Process, defined by an initial state, transition model, + and reward function. We also keep track of a gamma value, for use by + algorithms. The transition model is represented somewhat differently from + the text. Instead of P(s' | s, a) being a probability number for each + state/state/action triplet, we instead have T(s, a) return a + list of (p, s') pairs. We also keep track of the possible states, + terminal states, and actions for each state. [Page 646]""" + + def __init__(self, init, actlist, terminals, transitions=None, reward=None, states=None, gamma=0.9): + if not (0 < gamma <= 1): + raise ValueError("An MDP must have 0 < gamma <= 1") + + # collect states from transitions table if not passed. + self.states = states or self.get_states_from_transitions(transitions) + + self.init = init + + if isinstance(actlist, list): + # if actlist is a list, all states have the same actions + self.actlist = actlist + + elif isinstance(actlist, dict): + # if actlist is a dict, different actions for each state + self.actlist = actlist + + self.terminals = terminals + self.transitions = transitions or {} + if not self.transitions: + print("Warning: Transition table is empty.") + + self.gamma = gamma + + self.reward = reward or {s: 0 for s in self.states} + + # self.check_consistency() + + def R(self, state): + """Return a numeric reward for this state.""" + + return self.reward[state] + + def T(self, state, action): + """Transition model. From a state and an action, return a list + of (probability, result-state) pairs.""" + + if not self.transitions: + raise ValueError("Transition model is missing") + else: + return self.transitions[state][action] + + def actions(self, state): + """Return a list of actions that can be performed in this state. By default, a + fixed list of actions, except for terminal states. Override this + method if you need to specialize by state.""" + + if state in self.terminals: + return [None] + else: + return self.actlist + + def get_states_from_transitions(self, transitions): + if isinstance(transitions, dict): + s1 = set(transitions.keys()) + s2 = set(tr[1] for actions in transitions.values() + for effects in actions.values() + for tr in effects) + return s1.union(s2) + else: + print('Could not retrieve states from transitions') + return None + + def check_consistency(self): + + # check that all states in transitions are valid + assert set(self.states) == self.get_states_from_transitions(self.transitions) + + # check that init is a valid state + assert self.init in self.states + + # check reward for each state + assert set(self.reward.keys()) == set(self.states) + + # check that all terminals are valid states + assert all(t in self.states for t in self.terminals) + + # check that probability distributions for all actions sum to 1 + for s1, actions in self.transitions.items(): + for a in actions.keys(): + s = 0 + for o in actions[a]: + s += o[0] + assert abs(s - 1) < 0.001 + + +class MDP2(MDP): + """ + Inherits from MDP. Handles terminal states, and transitions to and from terminal states better. + """ + + def __init__(self, init, actlist, terminals, transitions, reward=None, gamma=0.9): + MDP.__init__(self, init, actlist, terminals, transitions, reward, gamma=gamma) + + def T(self, state, action): + if action is None: + return [(0.0, state)] + else: + return self.transitions[state][action] + + +class GridMDP(MDP): + """A two-dimensional grid MDP, as in [Figure 16.1]. All you have to do is + specify the grid as a list of lists of rewards; use None for an obstacle + (unreachable state). Also, you should specify the terminal states. + An action is an (x, y) unit vector; e.g. (1, 0) means move east.""" + + def __init__(self, grid, terminals, init=(0, 0), gamma=.9): + grid.reverse() # because we want row 0 on bottom, not on top + reward = {} + states = set() + self.rows = len(grid) + self.cols = len(grid[0]) + self.grid = grid + for x in range(self.cols): + for y in range(self.rows): + if grid[y][x]: + states.add((x, y)) + reward[(x, y)] = grid[y][x] + self.states = states + actlist = orientations + transitions = {} + for s in states: + transitions[s] = {} + for a in actlist: + transitions[s][a] = self.calculate_T(s, a) + MDP.__init__(self, init, actlist=actlist, + terminals=terminals, transitions=transitions, + reward=reward, states=states, gamma=gamma) + + def calculate_T(self, state, action): + if action: + return [(0.8, self.go(state, action)), + (0.1, self.go(state, turn_right(action))), + (0.1, self.go(state, turn_left(action)))] + else: + return [(0.0, state)] + + def T(self, state, action): + return self.transitions[state][action] if action else [(0.0, state)] + + def go(self, state, direction): + """Return the state that results from going in this direction.""" + + state1 = tuple(vector_add(state, direction)) + return state1 if state1 in self.states else state + + def to_grid(self, mapping): + """Convert a mapping from (x, y) to v into a [[..., v, ...]] grid.""" + + return list(reversed([[mapping.get((x, y), None) + for x in range(self.cols)] + for y in range(self.rows)])) + + def to_arrows(self, policy): + chars = {(1, 0): '>', (0, 1): '^', (-1, 0): '<', (0, -1): 'v', None: '.'} + return self.to_grid({s: chars[a] for (s, a) in policy.items()}) + + +# ______________________________________________________________________________ + + +""" [Figure 16.1] +A 4x3 grid environment that presents the agent with a sequential decision problem. +""" + +sequential_decision_environment = GridMDP([[-0.04, -0.04, -0.04, +1], + [-0.04, None, -0.04, -1], + [-0.04, -0.04, -0.04, -0.04]], + terminals=[(3, 2), (3, 1)]) + + +# ______________________________________________________________________________ +# 16.1.3 The Bellman equation for utilities + + +def q_value(mdp, s, a, U): + if not a: + return mdp.R(s) + res = 0 + for p, s_prime in mdp.T(s, a): + res += p * (mdp.R(s) + mdp.gamma * U[s_prime]) + return res + + +# TODO: DDN in figure 16.4 and 16.5 + +# ______________________________________________________________________________ +# 16.2 Algorithms for MDPs +# 16.2.1 Value Iteration + + +def value_iteration(mdp, epsilon=0.001): + """Solving an MDP by value iteration. [Figure 16.6]""" + + U1 = {s: 0 for s in mdp.states} + R, T, gamma = mdp.R, mdp.T, mdp.gamma + while True: + U = U1.copy() + delta = 0 + for s in mdp.states: + # U1[s] = R(s) + gamma * max(sum(p * U[s1] for (p, s1) in T(s, a)) + # for a in mdp.actions(s)) + U1[s] = max(q_value(mdp, s, a, U) for a in mdp.actions(s)) + delta = max(delta, abs(U1[s] - U[s])) + if delta <= epsilon * (1 - gamma) / gamma: + return U + + +# ______________________________________________________________________________ +# 16.2.2 Policy Iteration + + +def best_policy(mdp, U): + """Given an MDP and a utility function U, determine the best policy, + as a mapping from state to action.""" + + pi = {} + for s in mdp.states: + pi[s] = max(mdp.actions(s), key=lambda a: q_value(mdp, s, a, U)) + return pi + + +def expected_utility(a, s, U, mdp): + """The expected utility of doing a in state s, according to the MDP and U.""" + + return sum(p * U[s1] for (p, s1) in mdp.T(s, a)) + + +def policy_iteration(mdp): + """Solve an MDP by policy iteration [Figure 17.7]""" + + U = {s: 0 for s in mdp.states} + pi = {s: random.choice(mdp.actions(s)) for s in mdp.states} + while True: + U = policy_evaluation(pi, U, mdp) + unchanged = True + for s in mdp.states: + a_star = max(mdp.actions(s), key=lambda a: q_value(mdp, s, a, U)) + # a = max(mdp.actions(s), key=lambda a: expected_utility(a, s, U, mdp)) + if q_value(mdp, s, a_star, U) > q_value(mdp, s, pi[s], U): + pi[s] = a_star + unchanged = False + if unchanged: + return pi + + +def policy_evaluation(pi, U, mdp, k=20): + """Return an updated utility mapping U from each state in the MDP to its + utility, using an approximation (modified policy iteration).""" + + R, T, gamma = mdp.R, mdp.T, mdp.gamma + for i in range(k): + for s in mdp.states: + U[s] = R(s) + gamma * sum(p * U[s1] for (p, s1) in T(s, pi[s])) + return U + + +# ___________________________________________________________________ +# 16.4 Partially Observed MDPs + + +class POMDP(MDP): + """A Partially Observable Markov Decision Process, defined by + a transition model P(s'|s,a), actions A(s), a reward function R(s), + and a sensor model P(e|s). We also keep track of a gamma value, + for use by algorithms. The transition and the sensor models + are defined as matrices. We also keep track of the possible states + and actions for each state. [Page 659].""" + + def __init__(self, actions, transitions=None, evidences=None, rewards=None, states=None, gamma=0.95): + """Initialize variables of the pomdp""" + + if not (0 < gamma <= 1): + raise ValueError('A POMDP must have 0 < gamma <= 1') + + self.states = states + self.actions = actions + + # transition model cannot be undefined + self.t_prob = transitions or {} + if not self.t_prob: + print('Warning: Transition model is undefined') + + # sensor model cannot be undefined + self.e_prob = evidences or {} + if not self.e_prob: + print('Warning: Sensor model is undefined') + + self.gamma = gamma + self.rewards = rewards + + def remove_dominated_plans(self, input_values): + """ + Remove dominated plans. + This method finds all the lines contributing to the + upper surface and removes those which don't. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [values[0]] + y1_max = max(val[1] for val in values) + tgt = values[0] + prev_b = 0 + prev_ix = 0 + while tgt[1] != y1_max: + min_b = 1 + min_ix = 0 + for i in range(prev_ix + 1, len(values)): + if values[i][0] - tgt[0] + tgt[1] - values[i][1] != 0: + trans_b = (values[i][0] - tgt[0]) / (values[i][0] - tgt[0] + tgt[1] - values[i][1]) + if 0 <= trans_b <= 1 and trans_b > prev_b and trans_b < min_b: + min_b = trans_b + min_ix = i + prev_b = min_b + prev_ix = min_ix + tgt = values[min_ix] + best.append(tgt) + + return self.generate_mapping(best, input_values) + + def remove_dominated_plans_fast(self, input_values): + """ + Remove dominated plans using approximations. + Resamples the upper boundary at intervals of 100 and + finds the maximum values at these points. + """ + + values = [val for action in input_values for val in input_values[action]] + values.sort(key=lambda x: x[0], reverse=True) + + best = [] + sr = 100 + for i in range(sr + 1): + x = i / float(sr) + maximum = (values[0][1] - values[0][0]) * x + values[0][0] + tgt = values[0] + for value in values: + val = (value[1] - value[0]) * x + value[0] + if val > maximum: + maximum = val + tgt = value + + if all(any(tgt != v) for v in best): + best.append(np.array(tgt)) + + return self.generate_mapping(best, input_values) + + def generate_mapping(self, best, input_values): + """Generate mappings after removing dominated plans""" + + mapping = defaultdict(list) + for value in best: + for action in input_values: + if any(all(value == v) for v in input_values[action]): + mapping[action].append(value) + + return mapping + + def max_difference(self, U1, U2): + """Find maximum difference between two utility mappings""" + + for k, v in U1.items(): + sum1 = 0 + for element in U1[k]: + sum1 += sum(element) + sum2 = 0 + for element in U2[k]: + sum2 += sum(element) + return abs(sum1 - sum2) + + +class Matrix: + """Matrix operations class""" + + @staticmethod + def add(A, B): + """Add two matrices A and B""" + + res = [] + for i in range(len(A)): + row = [] + for j in range(len(A[0])): + row.append(A[i][j] + B[i][j]) + res.append(row) + return res + + @staticmethod + def scalar_multiply(a, B): + """Multiply scalar a to matrix B""" + + for i in range(len(B)): + for j in range(len(B[0])): + B[i][j] = a * B[i][j] + return B + + @staticmethod + def multiply(A, B): + """Multiply two matrices A and B element-wise""" + + matrix = [] + for i in range(len(B)): + row = [] + for j in range(len(B[0])): + row.append(B[i][j] * A[j][i]) + matrix.append(row) + + return matrix + + @staticmethod + def matmul(A, B): + """Inner-product of two matrices""" + + return [[sum(ele_a * ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A] + + @staticmethod + def transpose(A): + """Transpose a matrix""" + + return [list(i) for i in zip(*A)] + + +def pomdp_value_iteration(pomdp, epsilon=0.1): + """Solving a POMDP by value iteration.""" + + U = {'': [[0] * len(pomdp.states)]} + count = 0 + while True: + count += 1 + prev_U = U + values = [val for action in U for val in U[action]] + value_matxs = [] + for i in values: + for j in values: + value_matxs.append([i, j]) + + U1 = defaultdict(list) + for action in pomdp.actions: + for u in value_matxs: + u1 = Matrix.matmul(Matrix.matmul(pomdp.t_prob[int(action)], + Matrix.multiply(pomdp.e_prob[int(action)], Matrix.transpose(u))), + [[1], [1]]) + u1 = Matrix.add(Matrix.scalar_multiply(pomdp.gamma, Matrix.transpose(u1)), [pomdp.rewards[int(action)]]) + U1[action].append(u1[0]) + + U = pomdp.remove_dominated_plans_fast(U1) + # replace with U = pomdp.remove_dominated_plans(U1) for accurate calculations + + if count > 10: + if pomdp.max_difference(U, prev_U) < epsilon * (1 - pomdp.gamma) / pomdp.gamma: + return U + + +__doc__ += """ +>>> pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .01)) + +>>> sequential_decision_environment.to_arrows(pi) +[['>', '>', '>', '.'], ['^', None, '^', '.'], ['^', '>', '^', '<']] + +>>> from utils import print_table + +>>> print_table(sequential_decision_environment.to_arrows(pi)) +> > > . +^ None ^ . +^ > ^ < + +>>> print_table(sequential_decision_environment.to_arrows(policy_iteration(sequential_decision_environment))) +> > > . +^ None ^ . +^ > ^ < +""" # noqa + +""" +s = { 'a' : { 'plan1' : [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], + 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }, + } +""" diff --git a/neural_nets.ipynb b/neural_nets.ipynb index fe632c27f..1291da547 100644 --- a/neural_nets.ipynb +++ b/neural_nets.ipynb @@ -524,19 +524,17 @@ }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, "source": [ "The output should be 0, which means the item should get classified in the first class, \"setosa\". Note that since the algorithm is non-deterministic (because of the random initial weights) the classification might be wrong. Usually though, it should be correct.\n", "\n", - "To increase accuracy, you can (most of the time) add more layers and nodes. Unfortunately, increasing the number of layers or nodes also increases the computation cost and might result in overfitting." + "To increase accuracy, you can (most of the time) add more layers and nodes. Unfortunately, increasing the number of layers or nodes also increases the computation cost and might result in overfitting.\n", + "\n" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { @@ -556,8 +554,17 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 2 -} +} \ No newline at end of file diff --git a/nlp.py b/nlp.py index f42f9c981..03aabf54b 100644 --- a/nlp.py +++ b/nlp.py @@ -5,6 +5,7 @@ import urllib.request import re + # ______________________________________________________________________________ # Grammars and Lexicons @@ -89,7 +90,7 @@ def ProbRules(**rules): rules[lhs] = [] rhs_separate = [alt.strip().split() for alt in rhs.split('|')] for r in rhs_separate: - prob = float(r[-1][1:-1]) # remove brackets, convert to float + prob = float(r[-1][1:-1]) # remove brackets, convert to float rhs_rule = (r[:-1], prob) rules[lhs].append(rhs_rule) @@ -106,7 +107,7 @@ def ProbLexicon(**rules): rules[lhs] = [] rhs_separate = [word.strip().split() for word in rhs.split('|')] for r in rhs_separate: - prob = float(r[-1][1:-1]) # remove brackets, convert to float + prob = float(r[-1][1:-1]) # remove brackets, convert to float word = r[:-1][0] rhs_rule = (word, prob) rules[lhs].append(rhs_rule) @@ -212,7 +213,7 @@ def __repr__(self): Lexicon(Adj='happy | handsome | hairy', N='man')) -E_Prob = ProbGrammar('E_Prob', # The Probabilistic Grammar from the notebook +E_Prob = ProbGrammar('E_Prob', # The Probabilistic Grammar from the notebook ProbRules( S="NP VP [0.6] | S Conjunction S [0.4]", NP="Pronoun [0.2] | Name [0.05] | Noun [0.2] | Article Noun [0.15] \ @@ -236,52 +237,50 @@ def __repr__(self): Digit="0 [0.35] | 1 [0.35] | 2 [0.3]" )) - - -E_Chomsky = Grammar('E_Prob_Chomsky', # A Grammar in Chomsky Normal Form +E_Chomsky = Grammar('E_Prob_Chomsky', # A Grammar in Chomsky Normal Form Rules( - S='NP VP', - NP='Article Noun | Adjective Noun', - VP='Verb NP | Verb Adjective', + S='NP VP', + NP='Article Noun | Adjective Noun', + VP='Verb NP | Verb Adjective', ), Lexicon( - Article='the | a | an', - Noun='robot | sheep | fence', - Adjective='good | new | sad', - Verb='is | say | are' + Article='the | a | an', + Noun='robot | sheep | fence', + Adjective='good | new | sad', + Verb='is | say | are' )) -E_Prob_Chomsky = ProbGrammar('E_Prob_Chomsky', # A Probabilistic Grammar in CNF +E_Prob_Chomsky = ProbGrammar('E_Prob_Chomsky', # A Probabilistic Grammar in CNF ProbRules( - S='NP VP [1]', - NP='Article Noun [0.6] | Adjective Noun [0.4]', - VP='Verb NP [0.5] | Verb Adjective [0.5]', + S='NP VP [1]', + NP='Article Noun [0.6] | Adjective Noun [0.4]', + VP='Verb NP [0.5] | Verb Adjective [0.5]', ), ProbLexicon( - Article='the [0.5] | a [0.25] | an [0.25]', - Noun='robot [0.4] | sheep [0.4] | fence [0.2]', - Adjective='good [0.5] | new [0.2] | sad [0.3]', - Verb='is [0.5] | say [0.3] | are [0.2]' + Article='the [0.5] | a [0.25] | an [0.25]', + Noun='robot [0.4] | sheep [0.4] | fence [0.2]', + Adjective='good [0.5] | new [0.2] | sad [0.3]', + Verb='is [0.5] | say [0.3] | are [0.2]' )) E_Prob_Chomsky_ = ProbGrammar('E_Prob_Chomsky_', - ProbRules( - S='NP VP [1]', - NP='NP PP [0.4] | Noun Verb [0.6]', - PP='Preposition NP [1]', - VP='Verb NP [0.7] | VP PP [0.3]', - ), - ProbLexicon( - Noun='astronomers [0.18] | eyes [0.32] | stars [0.32] | telescopes [0.18]', - Verb='saw [0.5] | \'\' [0.5]', - Preposition='with [1]' - )) + ProbRules( + S='NP VP [1]', + NP='NP PP [0.4] | Noun Verb [0.6]', + PP='Preposition NP [1]', + VP='Verb NP [0.7] | VP PP [0.3]', + ), + ProbLexicon( + Noun='astronomers [0.18] | eyes [0.32] | stars [0.32] | telescopes [0.18]', + Verb='saw [0.5] | \'\' [0.5]', + Preposition='with [1]' + )) + # ______________________________________________________________________________ # Chart Parsing class Chart: - """Class for parsing sentences using a chart data structure. >>> chart = Chart(E0) >>> len(chart.parses('the stench is in 2 2')) @@ -310,7 +309,7 @@ def parses(self, words, S='S'): def parse(self, words, S='S'): """Parse a list of words; according to the grammar. Leave results in the chart.""" - self.chart = [[] for i in range(len(words)+1)] + self.chart = [[] for i in range(len(words) + 1)] self.add_edge([0, 0, 'S_', [], [S]]) for i in range(len(words)): self.scanner(i, words[i]) @@ -332,7 +331,7 @@ def scanner(self, j, word): """For each edge expecting a word of this category here, extend the edge.""" for (i, j, A, alpha, Bb) in self.chart[j]: if Bb and self.grammar.isa(word, Bb[0]): - self.add_edge([i, j+1, A, alpha + [(Bb[0], word)], Bb[1:]]) + self.add_edge([i, j + 1, A, alpha + [(Bb[0], word)], Bb[1:]]) def predictor(self, edge): """Add to chart any rules for B that could help extend this edge.""" @@ -366,13 +365,13 @@ def CYK_parse(words, grammar): # Combine first and second parts of right-hand sides of rules, # from short to long. - for length in range(2, N+1): - for start in range(N-length+1): + for length in range(2, N + 1): + for start in range(N - length + 1): for len1 in range(1, length): # N.B. the book incorrectly has N instead of length len2 = length - len1 for (X, Y, Z, p) in grammar.cnf_rules(): P[X, start, length] = max(P[X, start, length], - P[Y, start, len1] * P[Z, start+len1, len2] * p) + P[Y, start, len1] * P[Z, start + len1, len2] * p) return P @@ -444,7 +443,7 @@ def onlyWikipediaURLS(urls): """Some example HTML page data is from wikipedia. This function converts relative wikipedia links to full wikipedia URLs""" wikiURLs = [url for url in urls if url.startswith('/wiki/')] - return ["https://en.wikipedia.org"+url for url in wikiURLs] + return ["https://en.wikipedia.org" + url for url in wikiURLs] # ______________________________________________________________________________ @@ -484,17 +483,18 @@ def normalize(pages): """Normalize divides each page's score by the sum of the squares of all pages' scores (separately for both the authority and hub scores). """ - summed_hub = sum(page.hub**2 for _, page in pages.items()) - summed_auth = sum(page.authority**2 for _, page in pages.items()) + summed_hub = sum(page.hub ** 2 for _, page in pages.items()) + summed_auth = sum(page.authority ** 2 for _, page in pages.items()) for _, page in pages.items(): - page.hub /= summed_hub**0.5 - page.authority /= summed_auth**0.5 + page.hub /= summed_hub ** 0.5 + page.authority /= summed_auth ** 0.5 class ConvergenceDetector(object): """If the hub and authority values of the pages are no longer changing, we have reached a convergence and further iterations will have no effect. This detects convergence so that we can stop the HITS algorithm as early as possible.""" + def __init__(self): self.hub_history = None self.auth_history = None @@ -508,10 +508,10 @@ def detect(self): if self.hub_history is None: self.hub_history, self.auth_history = [], [] else: - diffsHub = [abs(x-y) for x, y in zip(curr_hubs, self.hub_history[-1])] - diffsAuth = [abs(x-y) for x, y in zip(curr_auths, self.auth_history[-1])] - aveDeltaHub = sum(diffsHub)/float(len(pagesIndex)) - aveDeltaAuth = sum(diffsAuth)/float(len(pagesIndex)) + diffsHub = [abs(x - y) for x, y in zip(curr_hubs, self.hub_history[-1])] + diffsAuth = [abs(x - y) for x, y in zip(curr_auths, self.auth_history[-1])] + aveDeltaHub = sum(diffsHub) / float(len(pagesIndex)) + aveDeltaAuth = sum(diffsAuth) / float(len(pagesIndex)) if aveDeltaHub < 0.01 and aveDeltaAuth < 0.01: # may need tweaking return True if len(self.hub_history) > 2: # prevent list from getting long @@ -522,13 +522,13 @@ def detect(self): return False -def getInlinks(page): +def getInLinks(page): if not page.inlinks: page.inlinks = determineInlinks(page) return [addr for addr, p in pagesIndex.items() if addr in page.inlinks] -def getOutlinks(page): +def getOutLinks(page): if not page.outlinks: page.outlinks = findOutlinks(page) return [addr for addr, p in pagesIndex.items() if addr in page.outlinks] @@ -538,12 +538,12 @@ def getOutlinks(page): # HITS Algorithm class Page(object): - def __init__(self, address, inlinks=None, outlinks=None, hub=0, authority=0): + def __init__(self, address, inLinks=None, outLinks=None, hub=0, authority=0): self.address = address self.hub = hub self.authority = authority - self.inlinks = inlinks - self.outlinks = outlinks + self.inlinks = inLinks + self.outlinks = outLinks pagesContent = {} # maps Page relative or absolute URL/location to page's HTML content @@ -562,8 +562,8 @@ def HITS(query): hub = {p: pages[p].hub for p in pages} for p in pages: # p.authority ← ∑i Inlinki(p).Hub - pages[p].authority = sum(hub[x] for x in getInlinks(pages[p])) + pages[p].authority = sum(hub[x] for x in getInLinks(pages[p])) # p.hub ← ∑i Outlinki(p).Authority - pages[p].hub = sum(authority[x] for x in getOutlinks(pages[p])) + pages[p].hub = sum(authority[x] for x in getOutLinks(pages[p])) normalize(pages) return pages diff --git a/nlp4e.py b/nlp4e.py new file mode 100644 index 000000000..095f54357 --- /dev/null +++ b/nlp4e.py @@ -0,0 +1,523 @@ +"""Natural Language Processing (Chapter 22)""" + +from collections import defaultdict +from utils4e import weighted_choice +import copy +import operator +import heapq +from search import Problem + + +# ______________________________________________________________________________ +# 22.2 Grammars + + +def Rules(**rules): + """Create a dictionary mapping symbols to alternative sequences. + >>> Rules(A = "B C | D E") + {'A': [['B', 'C'], ['D', 'E']]} + """ + for (lhs, rhs) in rules.items(): + rules[lhs] = [alt.strip().split() for alt in rhs.split('|')] + return rules + + +def Lexicon(**rules): + """Create a dictionary mapping symbols to alternative words. + >>> Lexicon(Article = "the | a | an") + {'Article': ['the', 'a', 'an']} + """ + for (lhs, rhs) in rules.items(): + rules[lhs] = [word.strip() for word in rhs.split('|')] + return rules + + +class Grammar: + + def __init__(self, name, rules, lexicon): + """A grammar has a set of rules and a lexicon.""" + self.name = name + self.rules = rules + self.lexicon = lexicon + self.categories = defaultdict(list) + for lhs in lexicon: + for word in lexicon[lhs]: + self.categories[word].append(lhs) + + def rewrites_for(self, cat): + """Return a sequence of possible rhs's that cat can be rewritten as.""" + return self.rules.get(cat, ()) + + def isa(self, word, cat): + """Return True iff word is of category cat""" + return cat in self.categories[word] + + def cnf_rules(self): + """Returns the tuple (X, Y, Z) for rules in the form: + X -> Y Z""" + cnf = [] + for X, rules in self.rules.items(): + for (Y, Z) in rules: + cnf.append((X, Y, Z)) + + return cnf + + def generate_random(self, S='S'): + """Replace each token in S by a random entry in grammar (recursively).""" + import random + + def rewrite(tokens, into): + for token in tokens: + if token in self.rules: + rewrite(random.choice(self.rules[token]), into) + elif token in self.lexicon: + into.append(random.choice(self.lexicon[token])) + else: + into.append(token) + return into + + return ' '.join(rewrite(S.split(), [])) + + def __repr__(self): + return ''.format(self.name) + + +def ProbRules(**rules): + """Create a dictionary mapping symbols to alternative sequences, + with probabilities. + >>> ProbRules(A = "B C [0.3] | D E [0.7]") + {'A': [(['B', 'C'], 0.3), (['D', 'E'], 0.7)]} + """ + for (lhs, rhs) in rules.items(): + rules[lhs] = [] + rhs_separate = [alt.strip().split() for alt in rhs.split('|')] + for r in rhs_separate: + prob = float(r[-1][1:-1]) # remove brackets, convert to float + rhs_rule = (r[:-1], prob) + rules[lhs].append(rhs_rule) + + return rules + + +def ProbLexicon(**rules): + """Create a dictionary mapping symbols to alternative words, + with probabilities. + >>> ProbLexicon(Article = "the [0.5] | a [0.25] | an [0.25]") + {'Article': [('the', 0.5), ('a', 0.25), ('an', 0.25)]} + """ + for (lhs, rhs) in rules.items(): + rules[lhs] = [] + rhs_separate = [word.strip().split() for word in rhs.split('|')] + for r in rhs_separate: + prob = float(r[-1][1:-1]) # remove brackets, convert to float + word = r[:-1][0] + rhs_rule = (word, prob) + rules[lhs].append(rhs_rule) + + return rules + + +class ProbGrammar: + + def __init__(self, name, rules, lexicon): + """A grammar has a set of rules and a lexicon. + Each rule has a probability.""" + self.name = name + self.rules = rules + self.lexicon = lexicon + self.categories = defaultdict(list) + + for lhs in lexicon: + for word, prob in lexicon[lhs]: + self.categories[word].append((lhs, prob)) + + def rewrites_for(self, cat): + """Return a sequence of possible rhs's that cat can be rewritten as.""" + return self.rules.get(cat, ()) + + def isa(self, word, cat): + """Return True iff word is of category cat""" + return cat in [c for c, _ in self.categories[word]] + + def cnf_rules(self): + """Returns the tuple (X, Y, Z, p) for rules in the form: + X -> Y Z [p]""" + cnf = [] + for X, rules in self.rules.items(): + for (Y, Z), p in rules: + cnf.append((X, Y, Z, p)) + + return cnf + + def generate_random(self, S='S'): + """Replace each token in S by a random entry in grammar (recursively). + Returns a tuple of (sentence, probability).""" + + def rewrite(tokens, into): + for token in tokens: + if token in self.rules: + non_terminal, prob = weighted_choice(self.rules[token]) + into[1] *= prob + rewrite(non_terminal, into) + elif token in self.lexicon: + terminal, prob = weighted_choice(self.lexicon[token]) + into[0].append(terminal) + into[1] *= prob + else: + into[0].append(token) + return into + + rewritten_as, prob = rewrite(S.split(), [[], 1]) + return (' '.join(rewritten_as), prob) + + def __repr__(self): + return ''.format(self.name) + + +E0 = Grammar('E0', + Rules( # Grammar for E_0 [Figure 22.2] + S='NP VP | S Conjunction S', + NP='Pronoun | Name | Noun | Article Noun | Digit Digit | NP PP | NP RelClause', + VP='Verb | VP NP | VP Adjective | VP PP | VP Adverb', + PP='Preposition NP', + RelClause='That VP'), + + Lexicon( # Lexicon for E_0 [Figure 22.3] + Noun="stench | breeze | glitter | nothing | wumpus | pit | pits | gold | east", + Verb="is | see | smell | shoot | fell | stinks | go | grab | carry | kill | turn | feel", # noqa + Adjective="right | left | east | south | back | smelly | dead", + Adverb="here | there | nearby | ahead | right | left | east | south | back", + Pronoun="me | you | I | it", + Name="John | Mary | Boston | Aristotle", + Article="the | a | an", + Preposition="to | in | on | near", + Conjunction="and | or | but", + Digit="0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9", + That="that" + )) + +E_ = Grammar('E_', # Trivial Grammar and lexicon for testing + Rules( + S='NP VP', + NP='Art N | Pronoun', + VP='V NP'), + + Lexicon( + Art='the | a', + N='man | woman | table | shoelace | saw', + Pronoun='I | you | it', + V='saw | liked | feel' + )) + +E_NP_ = Grammar('E_NP_', # Another Trivial Grammar for testing + Rules(NP='Adj NP | N'), + Lexicon(Adj='happy | handsome | hairy', + N='man')) + +E_Prob = ProbGrammar('E_Prob', # The Probabilistic Grammar from the notebook + ProbRules( + S="NP VP [0.6] | S Conjunction S [0.4]", + NP="Pronoun [0.2] | Name [0.05] | Noun [0.2] | Article Noun [0.15] \ + | Article Adjs Noun [0.1] | Digit [0.05] | NP PP [0.15] | NP RelClause [0.1]", + VP="Verb [0.3] | VP NP [0.2] | VP Adjective [0.25] | VP PP [0.15] | VP Adverb [0.1]", + Adjs="Adjective [0.5] | Adjective Adjs [0.5]", + PP="Preposition NP [1]", + RelClause="RelPro VP [1]" + ), + ProbLexicon( + Verb="is [0.5] | say [0.3] | are [0.2]", + Noun="robot [0.4] | sheep [0.4] | fence [0.2]", + Adjective="good [0.5] | new [0.2] | sad [0.3]", + Adverb="here [0.6] | lightly [0.1] | now [0.3]", + Pronoun="me [0.3] | you [0.4] | he [0.3]", + RelPro="that [0.5] | who [0.3] | which [0.2]", + Name="john [0.4] | mary [0.4] | peter [0.2]", + Article="the [0.5] | a [0.25] | an [0.25]", + Preposition="to [0.4] | in [0.3] | at [0.3]", + Conjunction="and [0.5] | or [0.2] | but [0.3]", + Digit="0 [0.35] | 1 [0.35] | 2 [0.3]" + )) + +E_Chomsky = Grammar('E_Prob_Chomsky', # A Grammar in Chomsky Normal Form + Rules( + S='NP VP', + NP='Article Noun | Adjective Noun', + VP='Verb NP | Verb Adjective', + ), + Lexicon( + Article='the | a | an', + Noun='robot | sheep | fence', + Adjective='good | new | sad', + Verb='is | say | are' + )) + +E_Prob_Chomsky = ProbGrammar('E_Prob_Chomsky', # A Probabilistic Grammar in CNF + ProbRules( + S='NP VP [1]', + NP='Article Noun [0.6] | Adjective Noun [0.4]', + VP='Verb NP [0.5] | Verb Adjective [0.5]', + ), + ProbLexicon( + Article='the [0.5] | a [0.25] | an [0.25]', + Noun='robot [0.4] | sheep [0.4] | fence [0.2]', + Adjective='good [0.5] | new [0.2] | sad [0.3]', + Verb='is [0.5] | say [0.3] | are [0.2]' + )) +E_Prob_Chomsky_ = ProbGrammar('E_Prob_Chomsky_', + ProbRules( + S='NP VP [1]', + NP='NP PP [0.4] | Noun Verb [0.6]', + PP='Preposition NP [1]', + VP='Verb NP [0.7] | VP PP [0.3]', + ), + ProbLexicon( + Noun='astronomers [0.18] | eyes [0.32] | stars [0.32] | telescopes [0.18]', + Verb='saw [0.5] | \'\' [0.5]', + Preposition='with [1]' + )) + + +# ______________________________________________________________________________ +# 22.3 Parsing + + +class Chart: + """Class for parsing sentences using a chart data structure. + >>> chart = Chart(E0) + >>> len(chart.parses('the stench is in 2 2')) + 1 + """ + + def __init__(self, grammar, trace=False): + """A datastructure for parsing a string; and methods to do the parse. + self.chart[i] holds the edges that end just before the i'th word. + Edges are 5-element lists of [start, end, lhs, [found], [expects]].""" + self.grammar = grammar + self.trace = trace + + def parses(self, words, S='S'): + """Return a list of parses; words can be a list or string.""" + if isinstance(words, str): + words = words.split() + self.parse(words, S) + # Return all the parses that span the whole input + # 'span the whole input' => begin at 0, end at len(words) + return [[i, j, S, found, []] + for (i, j, lhs, found, expects) in self.chart[len(words)] + # assert j == len(words) + if i == 0 and lhs == S and expects == []] + + def parse(self, words, S='S'): + """Parse a list of words; according to the grammar. + Leave results in the chart.""" + self.chart = [[] for i in range(len(words) + 1)] + self.add_edge([0, 0, 'S_', [], [S]]) + for i in range(len(words)): + self.scanner(i, words[i]) + return self.chart + + def add_edge(self, edge): + """Add edge to chart, and see if it extends or predicts another edge.""" + start, end, lhs, found, expects = edge + if edge not in self.chart[end]: + self.chart[end].append(edge) + if self.trace: + print('Chart: added {}'.format(edge)) + if not expects: + self.extender(edge) + else: + self.predictor(edge) + + def scanner(self, j, word): + """For each edge expecting a word of this category here, extend the edge.""" + for (i, j, A, alpha, Bb) in self.chart[j]: + if Bb and self.grammar.isa(word, Bb[0]): + self.add_edge([i, j + 1, A, alpha + [(Bb[0], word)], Bb[1:]]) + + def predictor(self, edge): + """Add to chart any rules for B that could help extend this edge.""" + (i, j, A, alpha, Bb) = edge + B = Bb[0] + if B in self.grammar.rules: + for rhs in self.grammar.rewrites_for(B): + self.add_edge([j, j, B, [], rhs]) + + def extender(self, edge): + """See what edges can be extended by this edge.""" + (j, k, B, _, _) = edge + for (i, j, A, alpha, B1b) in self.chart[j]: + if B1b and B == B1b[0]: + self.add_edge([i, k, A, alpha + [edge], B1b[1:]]) + + +# ______________________________________________________________________________ +# CYK Parsing + + +class Tree: + def __init__(self, root, *args): + self.root = root + self.leaves = [leaf for leaf in args] + + +def CYK_parse(words, grammar): + """ [Figure 22.6] """ + # We use 0-based indexing instead of the book's 1-based. + P = defaultdict(float) + T = defaultdict(Tree) + + # Insert lexical categories for each word. + for (i, word) in enumerate(words): + for (X, p) in grammar.categories[word]: + P[X, i, i] = p + T[X, i, i] = Tree(X, word) + + # Construct X(i:k) from Y(i:j) and Z(j+1:k), shortest span first + for i, j, k in subspan(len(words)): + for (X, Y, Z, p) in grammar.cnf_rules(): + PYZ = P[Y, i, j] * P[Z, j + 1, k] * p + if PYZ > P[X, i, k]: + P[X, i, k] = PYZ + T[X, i, k] = Tree(X, T[Y, i, j], T[Z, j + 1, k]) + + return T + + +def subspan(N): + """returns all tuple(i, j, k) covering a span (i, k) with i <= j < k""" + for length in range(2, N + 1): + for i in range(1, N + 2 - length): + k = i + length - 1 + for j in range(i, k): + yield (i, j, k) + + +# using search algorithms in the searching part + + +class TextParsingProblem(Problem): + def __init__(self, initial, grammar, goal='S'): + """ + :param initial: the initial state of words in a list. + :param grammar: a grammar object + :param goal: the goal state, usually S + """ + super(TextParsingProblem, self).__init__(initial, goal) + self.grammar = grammar + self.combinations = defaultdict(list) # article combinations + # backward lookup of rules + for rule in grammar.rules: + for comb in grammar.rules[rule]: + self.combinations[' '.join(comb)].append(rule) + + def actions(self, state): + actions = [] + categories = self.grammar.categories + # first change each word to the article of its category + for i in range(len(state)): + word = state[i] + if word in categories: + for X in categories[word]: + state[i] = X + actions.append(copy.copy(state)) + state[i] = word + # if all words are replaced by articles, replace combinations of articles by inferring rules. + if not actions: + for start in range(len(state)): + for end in range(start, len(state) + 1): + # try combinations between (start, end) + articles = ' '.join(state[start:end]) + for c in self.combinations[articles]: + actions.append(state[:start] + [c] + state[end:]) + return actions + + def result(self, state, action): + return action + + def h(self, state): + # heuristic function + return len(state) + + +def astar_search_parsing(words, gramma): + """bottom-up parsing using A* search to find whether a list of words is a sentence""" + # init the problem + problem = TextParsingProblem(words, gramma, 'S') + state = problem.initial + # init the searching frontier + frontier = [(len(state) + problem.h(state), state)] + heapq.heapify(frontier) + + while frontier: + # search the frontier node with lowest cost first + cost, state = heapq.heappop(frontier) + actions = problem.actions(state) + for action in actions: + new_state = problem.result(state, action) + # update the new frontier node to the frontier + if new_state == [problem.goal]: + return problem.goal + if new_state != state: + heapq.heappush(frontier, (len(new_state) + problem.h(new_state), new_state)) + return False + + +def beam_search_parsing(words, gramma, b=3): + """bottom-up text parsing using beam search""" + # init problem + problem = TextParsingProblem(words, gramma, 'S') + # init frontier + frontier = [(len(problem.initial), problem.initial)] + heapq.heapify(frontier) + + # explore the current frontier and keep b new states with lowest cost + def explore(frontier): + new_frontier = [] + for cost, state in frontier: + # expand the possible children states of current state + if not problem.goal_test(' '.join(state)): + actions = problem.actions(state) + for action in actions: + new_state = problem.result(state, action) + if [len(new_state), new_state] not in new_frontier and new_state != state: + new_frontier.append([len(new_state), new_state]) + else: + return problem.goal + heapq.heapify(new_frontier) + # only keep b states + return heapq.nsmallest(b, new_frontier) + + while frontier: + frontier = explore(frontier) + if frontier == problem.goal: + return frontier + return False + + +# ______________________________________________________________________________ +# 22.4 Augmented Grammar + + +g = Grammar("arithmetic_expression", # A Grammar of Arithmetic Expression + rules={ + 'Number_0': 'Digit_0', 'Number_1': 'Digit_1', 'Number_2': 'Digit_2', + 'Number_10': 'Number_1 Digit_0', 'Number_11': 'Number_1 Digit_1', + 'Number_100': 'Number_10 Digit_0', + 'Exp_5': ['Number_5', '( Exp_5 )', 'Exp_1, Operator_+ Exp_4', 'Exp_2, Operator_+ Exp_3', + 'Exp_0, Operator_+ Exp_5', 'Exp_3, Operator_+ Exp_2', 'Exp_4, Operator_+ Exp_1', + 'Exp_5, Operator_+ Exp_0', 'Exp_1, Operator_* Exp_5'], # more possible combinations + 'Operator_+': operator.add, 'Operator_-': operator.sub, 'Operator_*': operator.mul, + 'Operator_/': operator.truediv, + 'Digit_0': 0, 'Digit_1': 1, 'Digit_2': 2, 'Digit_3': 3, 'Digit_4': 4 + }, + lexicon={}) + +g = Grammar("Ali loves Bob", # A example grammer of Ali loves Bob example + rules={ + "S_loves_ali_bob": "NP_ali, VP_x_loves_x_bob", "S_loves_bob_ali": "NP_bob, VP_x_loves_x_ali", + "VP_x_loves_x_bob": "Verb_xy_loves_xy NP_bob", "VP_x_loves_x_ali": "Verb_xy_loves_xy NP_ali", + "NP_bob": "Name_bob", "NP_ali": "Name_ali" + }, + lexicon={ + "Name_ali": "Ali", "Name_bob": "Bob", "Verb_xy_loves_xy": "loves" + }) diff --git a/nlp_apps.ipynb b/nlp_apps.ipynb index 458c55700..2f4796b7a 100644 --- a/nlp_apps.ipynb +++ b/nlp_apps.ipynb @@ -17,7 +17,8 @@ "\n", "* Language Recognition\n", "* Author Recognition\n", - "* The Federalist Papers" + "* The Federalist Papers\n", + "* Text Classification" ] }, { @@ -37,10 +38,8 @@ }, { "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": true - }, + "execution_count": 1, + "metadata": {}, "outputs": [], "source": [ "from utils import open_data\n", @@ -68,10 +67,8 @@ }, { "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, + "execution_count": 2, + "metadata": {}, "outputs": [], "source": [ "from learning import NaiveBayesLearner\n", @@ -92,10 +89,8 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": true - }, + "execution_count": 3, + "metadata": {}, "outputs": [], "source": [ "def recognize(sentence, nBS, n):\n", @@ -122,7 +117,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -138,7 +133,7 @@ "'German'" ] }, - "execution_count": 5, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -149,7 +144,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -165,7 +160,7 @@ "'English'" ] }, - "execution_count": 6, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -176,7 +171,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -192,7 +187,7 @@ "'German'" ] }, - "execution_count": 7, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -203,7 +198,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -219,7 +214,7 @@ "'English'" ] }, - "execution_count": 8, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } @@ -254,10 +249,8 @@ }, { "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": true - }, + "execution_count": 8, + "metadata": {}, "outputs": [], "source": [ "from utils import open_data\n", @@ -285,10 +278,8 @@ }, { "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": true - }, + "execution_count": 9, + "metadata": {}, "outputs": [], "source": [ "from learning import NaiveBayesLearner\n", @@ -307,10 +298,8 @@ }, { "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, + "execution_count": 10, + "metadata": {}, "outputs": [], "source": [ "def recognize(sentence, nBS):\n", @@ -329,7 +318,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 11, "metadata": {}, "outputs": [ { @@ -338,7 +327,7 @@ "'Abbott'" ] }, - "execution_count": 4, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -358,7 +347,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -367,7 +356,7 @@ "'Austen'" ] }, - "execution_count": 5, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -402,10 +391,8 @@ }, { "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": true - }, + "execution_count": 13, + "metadata": {}, "outputs": [], "source": [ "from utils import open_data\n", @@ -423,7 +410,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -432,7 +419,7 @@ "'The Project Gutenberg EBook of The Federalist Papers, by \\nAlexander Hamilton and John Jay and James Madison\\n\\nThis eBook is for the use of anyone anywhere at no cost and with\\nalmost no restrictions whatsoever. You may copy it, give it away or\\nre-use it under the terms of the Project Gutenberg License included\\nwith this eBook or online at www.gutenberg.net\\n\\n\\nTitle: The Federalist Papers\\n\\nAuthor: Alexander Hamilton\\n John Jay\\n James Madison\\n\\nPosting Date: December 12, 2011 [EBook #18]'" ] }, - "execution_count": 2, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -450,10 +437,8 @@ }, { "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, + "execution_count": 15, + "metadata": {}, "outputs": [], "source": [ "wordseq = words(federalist)\n", @@ -469,7 +454,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 16, "metadata": {}, "outputs": [ { @@ -478,7 +463,7 @@ "'federalist no 1 general introduction for the independent journal hamilton to the people of the state of new york after an unequivocal experience of the inefficacy of the subsisting federal government you are called upon to deliberate on a new constitution for the united states of america the subject speaks its own importance comprehending in its consequences nothing less than the existence of the union the safety and welfare of the parts of which it is composed the fate of an empire in many respects the most interesting in the world it has been frequently remarked that it seems to'" ] }, - "execution_count": 4, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -500,10 +485,8 @@ }, { "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": true - }, + "execution_count": 17, + "metadata": {}, "outputs": [], "source": [ "wordseq = [w for w in wordseq if w != 'publius']" @@ -522,7 +505,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 18, "metadata": {}, "outputs": [ { @@ -531,7 +514,7 @@ "(4, 16, 52)" ] }, - "execution_count": 6, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -568,10 +551,8 @@ }, { "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": true - }, + "execution_count": 19, + "metadata": {}, "outputs": [], "source": [ "hamilton = ''.join(hamilton)\n", @@ -603,10 +584,8 @@ }, { "cell_type": "code", - "execution_count": 16, - "metadata": { - "collapsed": true - }, + "execution_count": 20, + "metadata": {}, "outputs": [], "source": [ "import random\n", @@ -687,10 +666,8 @@ }, { "cell_type": "code", - "execution_count": 17, - "metadata": { - "collapsed": true - }, + "execution_count": 21, + "metadata": {}, "outputs": [], "source": [ "dist = {('Madison', 1): P_madison, ('Hamilton', 1): P_hamilton, ('Jay', 1): P_jay}\n", @@ -707,10 +684,8 @@ }, { "cell_type": "code", - "execution_count": 18, - "metadata": { - "collapsed": true - }, + "execution_count": 22, + "metadata": {}, "outputs": [], "source": [ "def recognize(sentence, nBS):\n", @@ -726,7 +701,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -737,7 +712,7 @@ "Straightforward Naive Bayes Learner\n", "\n", "Paper No. 49: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", - "Paper No. 50: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 50: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", "Paper No. 51: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", "Paper No. 52: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", "Paper No. 53: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", @@ -746,8 +721,8 @@ "Paper No. 56: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", "Paper No. 57: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", "Paper No. 58: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", - "Paper No. 18: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", - "Paper No. 19: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 18: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", + "Paper No. 19: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", "Paper No. 20: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", "Paper No. 64: Hamilton: 1.0000 Madison: 0.0000 Jay: 0.0000\n", "\n", @@ -797,13 +772,246 @@ ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "collapsed": true }, + "source": [ + "## Text Classification" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Text Classification** is assigning a category to a document based on the content of the document. Text Classification is one of the most popular and fundamental tasks of Natural Language Processing. Text classification can be applied on a variety of texts like *Short Documents* (like tweets, customer reviews, etc.) and *Long Document* (like emails, media articles, etc.).\n", + "\n", + "We already have seen an example of Text Classification in the above tasks like Language Identification, Author Recognition and Federalist Paper Identification.\n", + "\n", + "### Applications\n", + "Some of the broad applications of Text Classification are:-\n", + "- Language Identification\n", + "- Author Recognition\n", + "- Sentiment Analysis\n", + "- Spam Mail Detection\n", + "- Topic Labelling \n", + "- Word Sense Disambiguation\n", + "\n", + "### Use Cases\n", + "Some of the use cases of Text classification are:-\n", + "- Social Media Monitoring\n", + "- Brand Monitoring\n", + "- Auto-tagging of user queries\n", + "\n", + "For Text Classification, we would be using the Naive Bayes Classifier. The reasons for using Naive Bayes Classifier are:-\n", + "- Being a probabilistic classifier, therefore, will calculate the probability of each category\n", + "- It is fast, reliable and accurate \n", + "- Naive Bayes Classifiers have already been used to solve many Natural Language Processing (NLP) applications.\n", + "\n", + "Here we would here be covering an example of **Word Sense Disambiguation** as an application of Text Classification. It is used to remove the ambiguity of a given word if the word has two different meanings.\n", + "\n", + "As we know that we would be working on determining whether the word *apple* in a sentence refers to `fruit` or to a `company`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 1:- Defining the dataset** \n", + "\n", + "The dataset has been defined here so that everything is clear and can be tested with other things as well." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, "outputs": [], - "source": [] + "source": [ + "train_data = [\n", + " \"Apple targets big business with new iOS 7 features. Finally... A corp iTunes account!\",\n", + " \"apple inc is searching for people to help and try out all their upcoming tablet within our own net page No.\",\n", + " \"Microsoft to bring Xbox and PC games to Apple, Android phones: Report: Microsoft Corp\",\n", + " \"When did green skittles change from lime to green apple?\",\n", + " \"Myra Oltman is the best. I told her I wanted to learn how to make apple pie, so she made me a kit!\",\n", + " \"Surreal Sat in a sewing room, surrounded by crap, listening to beautiful music eating apple pie.\"\n", + "]\n", + "\n", + "train_target = [\n", + " \"company\",\n", + " \"company\",\n", + " \"company\",\n", + " \"fruit\",\n", + " \"fruit\",\n", + " \"fruit\",\n", + "]\n", + "\n", + "class_0 = \"company\"\n", + "class_1 = \"fruit\"\n", + "\n", + "test_data = [\n", + " \"Apple Inc. supplier Foxconn demos its own iPhone-compatible smartwatch\",\n", + " \"I now know how to make a delicious apple pie thanks to the best teachers ever\"\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 2:- Preprocessing the dataset**\n", + "\n", + "In this step, we would be doing some preprocessing on the dataset like breaking the sentence into words and converting to lower case.\n", + "\n", + "We already have a `words(sent)` function defined in `text.py` which does the task of splitting the sentence into words." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "train_data_processed = [words(i) for i in train_data]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 3:- Feature Extraction from the text**\n", + "\n", + "Now we would be extracting features from the text like extracting the set of words used in both the categories i.e. `company` and `fruit`.\n", + "\n", + "The frequency of a word would help in calculating the probability of that word being in a particular class. " + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of words in `company` class: 49\n", + "Number of words in `fruit` class: 49\n" + ] + } + ], + "source": [ + "words_0 = []\n", + "words_1 = []\n", + "\n", + "for sent, tag in zip(train_data_processed, train_target):\n", + " if(tag == class_0):\n", + " words_0 += sent\n", + " elif(tag == class_1):\n", + " words_1 += sent\n", + " \n", + "print(\"Number of words in `{}` class: {}\".format(class_0, len(words_0)))\n", + "print(\"Number of words in `{}` class: {}\".format(class_1, len(words_1)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you might have observed, that our dataset is equally balanced, i.e. we have an equal number of words in both the classes." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 4:- Building the Naive Bayes Model**\n", + "\n", + "Using the Naive Bayes classifier we can calculate the probability of a word in `company` and `fruit` class and then multiplying all of them to get the probability of that sentence belonging each of the given classes. But if a word is not in our dictionary then this leads to the probability of that word belonging to that class becoming zero. For example:- the word *Foxconn* is not in the dictionary of any of the classes. Due to this, the probability of word *Foxconn* being in any of these classes becomes zero, and since all the probabilities are multiplied, this leads to the probability of that sentence belonging to any of the classes becoming zero. \n", + "\n", + "To solve the problem we need to use **smoothing**, i.e. providing a minimum non-zero threshold probability to every word that we come across.\n", + "\n", + "The `UnigramWordModel` class has implemented smoothing by taking an additional argument from the user, i.e. the minimum frequency that we would be giving to every word even if it is new to the dictionary." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "model_words_0 = UnigramWordModel(words_0, 1)\n", + "model_words_1 = UnigramWordModel(words_1, 1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we would be building the Naive Bayes model. For that, we would be making `dist` as we had done earlier in the Authorship Recognition Task." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "from learning import NaiveBayesLearner\n", + "\n", + "dist = {('company', 1): model_words_0, ('fruit', 1): model_words_1}\n", + "\n", + "nBS = NaiveBayesLearner(dist, simple=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 5:- Predict the class of a sentence**\n", + "\n", + "Now we will be writing a function that does pre-process of the sentences which we have taken for testing. And then predicting the class of every sentence in the document." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "def recognize(sentence, nBS):\n", + " sentence_words = words(sentence)\n", + " return nBS(sentence_words)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Apple Inc. supplier Foxconn demos its own iPhone-compatible smartwatch\t-company\n", + "I now know how to make a delicious apple pie thanks to the best teachers ever\t-fruit\n" + ] + } + ], + "source": [ + "# predicting the class of sentences in the test set\n", + "for i in test_data:\n", + " print(i + \"\\t-\" + recognize(i, nBS))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You might have observed that the predictions made by the model are correct and we are able to differentiate between sentences of different classes. You can try more sentences on your own. Unfortunately though, since the datasets are pretty small, chances are the guesses will not always be correct.\n", + "\n", + "As you might have observed, the above method is very much similar to the Author Recognition, which is also a type of Text Classification. Like this most of Text Classification have the same underlying structure and follow a similar procedure." + ] } ], "metadata": { @@ -822,7 +1030,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.1" + "version": "3.6.7" } }, "nbformat": 4, diff --git a/notebook.py b/notebook.py index d60ced855..7f0306335 100644 --- a/notebook.py +++ b/notebook.py @@ -1,22 +1,23 @@ +import time +from collections import defaultdict from inspect import getsource -from utils import argmax, argmin -from games import TicTacToe, alphabeta_player, random_player, Fig52Extended, infinity -from logic import parse_definite_clause, standardize_variables, unify, subst -from learning import DataSet -from IPython.display import HTML, display -from collections import Counter, defaultdict - +import ipywidgets as widgets import matplotlib.pyplot as plt +import networkx as nx import numpy as np +from IPython.display import HTML +from IPython.display import display from PIL import Image +from matplotlib import lines -import os, struct -import array -import time +from games import TicTacToe, alpha_beta_player, random_player, Fig52Extended +from learning import DataSet +from logic import parse_definite_clause, standardize_variables, unify_mm, subst +from search import GraphProblem, romania_map -#______________________________________________________________________________ +# ______________________________________________________________________________ # Magic Words @@ -47,6 +48,7 @@ def psource(*functions): except ImportError: print(source_code) + # ______________________________________________________________________________ # Iris Visualization @@ -55,7 +57,6 @@ def show_iris(i=0, j=1, k=2): """Plots the iris dataset in a 3D plot. The three axes are given by i, j and k, which correspond to three of the four iris features.""" - from mpl_toolkits.mplot3d import Axes3D plt.rcParams.update(plt.rcParamsDefault) @@ -80,7 +81,6 @@ def show_iris(i=0, j=1, k=2): b_versicolor = [v[j] for v in buckets["versicolor"]] c_versicolor = [v[k] for v in buckets["versicolor"]] - for c, m, sl, sw, pl in [('b', 's', a_setosa, b_setosa, c_setosa), ('g', '^', a_virginica, b_virginica, c_virginica), ('r', 'o', a_versicolor, b_versicolor, c_versicolor)]: @@ -92,6 +92,7 @@ def show_iris(i=0, j=1, k=2): plt.show() + # ______________________________________________________________________________ # MNIST @@ -100,7 +101,6 @@ def load_MNIST(path="aima-data/MNIST/Digits", fashion=False): import os, struct import array import numpy as np - from collections import Counter if fashion: path = "aima-data/MNIST/Fashion" @@ -129,22 +129,22 @@ def load_MNIST(path="aima-data/MNIST/Digits", fashion=False): te_lbl = array.array("b", test_lbl_file.read()) test_lbl_file.close() - #print(len(tr_img), len(tr_lbl), tr_size) - #print(len(te_img), len(te_lbl), te_size) + # print(len(tr_img), len(tr_lbl), tr_size) + # print(len(te_img), len(te_lbl), te_size) - train_img = np.zeros((tr_size, tr_rows*tr_cols), dtype=np.int16) + train_img = np.zeros((tr_size, tr_rows * tr_cols), dtype=np.int16) train_lbl = np.zeros((tr_size,), dtype=np.int8) for i in range(tr_size): - train_img[i] = np.array(tr_img[i*tr_rows*tr_cols : (i+1)*tr_rows*tr_cols]).reshape((tr_rows*te_cols)) + train_img[i] = np.array(tr_img[i * tr_rows * tr_cols: (i + 1) * tr_rows * tr_cols]).reshape((tr_rows * te_cols)) train_lbl[i] = tr_lbl[i] - test_img = np.zeros((te_size, te_rows*te_cols), dtype=np.int16) + test_img = np.zeros((te_size, te_rows * te_cols), dtype=np.int16) test_lbl = np.zeros((te_size,), dtype=np.int8) for i in range(te_size): - test_img[i] = np.array(te_img[i*te_rows*te_cols : (i+1)*te_rows*te_cols]).reshape((te_rows*te_cols)) + test_img[i] = np.array(te_img[i * te_rows * te_cols: (i + 1) * te_rows * te_cols]).reshape((te_rows * te_cols)) test_lbl[i] = te_lbl[i] - return(train_img, train_lbl, test_img, test_lbl) + return (train_img, train_lbl, test_img, test_lbl) digit_classes = [str(i) for i in range(10)] @@ -163,7 +163,7 @@ def show_MNIST(labels, images, samples=8, fashion=False): for y, cls in enumerate(classes): idxs = np.nonzero([i == y for i in labels]) idxs = np.random.choice(idxs[0], samples, replace=False) - for i , idx in enumerate(idxs): + for i, idx in enumerate(idxs): plt_idx = i * num_classes + y + 1 plt.subplot(samples, num_classes, plt_idx) plt.imshow(images[idx].reshape((28, 28))) @@ -188,16 +188,17 @@ def show_ave_MNIST(labels, images, fashion=False): idxs = np.nonzero([i == y for i in labels]) print(item_type, y, ":", len(idxs[0]), "images.") - ave_img = np.mean(np.vstack([images[i] for i in idxs[0]]), axis = 0) - #print(ave_img.shape) + ave_img = np.mean(np.vstack([images[i] for i in idxs[0]]), axis=0) + # print(ave_img.shape) - plt.subplot(1, num_classes, y+1) + plt.subplot(1, num_classes, y + 1) plt.imshow(ave_img.reshape((28, 28))) plt.axis("off") plt.title(cls) plt.show() + # ______________________________________________________________________________ # MDP @@ -216,7 +217,7 @@ def plot_grid_step(iteration): for column in range(columns): current_row.append(data[(column, row)]) grid.append(current_row) - grid.reverse() # output like book + grid.reverse() # output like book fig = plt.imshow(grid, cmap=plt.cm.bwr, interpolation='nearest') plt.axis('off') @@ -232,18 +233,20 @@ def plot_grid_step(iteration): return plot_grid_step + def make_visualize(slider): """Takes an input a sliderand returns callback function for timer and animation.""" - def visualize_callback(Visualize, time_step): - if Visualize is True: + def visualize_callback(visualize, time_step): + if visualize is True: for i in range(slider.min, slider.max + 1): slider.value = i time.sleep(float(time_step)) return visualize_callback + # ______________________________________________________________________________ @@ -377,12 +380,13 @@ def display_html(html_string): class Canvas_TicTacToe(Canvas): """Play a 3x3 TicTacToe game on HTML canvas""" + def __init__(self, varname, player_1='human', player_2='random', width=300, height=350, cid=None): - valid_players = ('human', 'random', 'alphabeta') + valid_players = ('human', 'random', 'alpha_beta') if player_1 not in valid_players or player_2 not in valid_players: raise TypeError("Players must be one of {}".format(valid_players)) - Canvas.__init__(self, varname, width, height, cid) + super().__init__(varname, width, height, cid) self.ttt = TicTacToe() self.state = self.ttt.initial self.turn = 0 @@ -394,20 +398,20 @@ def __init__(self, varname, player_1='human', player_2='random', def mouse_click(self, x, y): player = self.players[self.turn] if self.ttt.terminal_test(self.state): - if 0.55 <= x/self.width <= 0.95 and 6/7 <= y/self.height <= 6/7+1/8: + if 0.55 <= x / self.width <= 0.95 and 6 / 7 <= y / self.height <= 6 / 7 + 1 / 8: self.state = self.ttt.initial self.turn = 0 self.draw_board() return if player == 'human': - x, y = int(3*x/self.width) + 1, int(3*y/(self.height*6/7)) + 1 + x, y = int(3 * x / self.width) + 1, int(3 * y / (self.height * 6 / 7)) + 1 if (x, y) not in self.ttt.actions(self.state): # Invalid move return move = (x, y) - elif player == 'alphabeta': - move = alphabeta_player(self.ttt, self.state) + elif player == 'alpha_beta': + move = alpha_beta_player(self.ttt, self.state) else: move = random_player(self.ttt, self.state) self.state = self.ttt.result(self.state, move) @@ -417,11 +421,11 @@ def mouse_click(self, x, y): def draw_board(self): self.clear() self.stroke(0, 0, 0) - offset = 1/20 - self.line_n(0 + offset, (1/3)*6/7, 1 - offset, (1/3)*6/7) - self.line_n(0 + offset, (2/3)*6/7, 1 - offset, (2/3)*6/7) - self.line_n(1/3, (0 + offset)*6/7, 1/3, (1 - offset)*6/7) - self.line_n(2/3, (0 + offset)*6/7, 2/3, (1 - offset)*6/7) + offset = 1 / 20 + self.line_n(0 + offset, (1 / 3) * 6 / 7, 1 - offset, (1 / 3) * 6 / 7) + self.line_n(0 + offset, (2 / 3) * 6 / 7, 1 - offset, (2 / 3) * 6 / 7) + self.line_n(1 / 3, (0 + offset) * 6 / 7, 1 / 3, (1 - offset) * 6 / 7) + self.line_n(2 / 3, (0 + offset) * 6 / 7, 2 / 3, (1 - offset) * 6 / 7) board = self.state.board for mark in board: @@ -433,64 +437,65 @@ def draw_board(self): # End game message utility = self.ttt.utility(self.state, self.ttt.to_move(self.ttt.initial)) if utility == 0: - self.text_n('Game Draw!', offset, 6/7 + offset) + self.text_n('Game Draw!', offset, 6 / 7 + offset) else: - self.text_n('Player {} wins!'.format("XO"[utility < 0]), offset, 6/7 + offset) + self.text_n('Player {} wins!'.format("XO"[utility < 0]), offset, 6 / 7 + offset) # Find the 3 and draw a line self.stroke([255, 0][self.turn], [0, 255][self.turn], 0) for i in range(3): if all([(i + 1, j + 1) in self.state.board for j in range(3)]) and \ - len({self.state.board[(i + 1, j + 1)] for j in range(3)}) == 1: - self.line_n(i/3 + 1/6, offset*6/7, i/3 + 1/6, (1 - offset)*6/7) + len({self.state.board[(i + 1, j + 1)] for j in range(3)}) == 1: + self.line_n(i / 3 + 1 / 6, offset * 6 / 7, i / 3 + 1 / 6, (1 - offset) * 6 / 7) if all([(j + 1, i + 1) in self.state.board for j in range(3)]) and \ - len({self.state.board[(j + 1, i + 1)] for j in range(3)}) == 1: - self.line_n(offset, (i/3 + 1/6)*6/7, 1 - offset, (i/3 + 1/6)*6/7) + len({self.state.board[(j + 1, i + 1)] for j in range(3)}) == 1: + self.line_n(offset, (i / 3 + 1 / 6) * 6 / 7, 1 - offset, (i / 3 + 1 / 6) * 6 / 7) if all([(i + 1, i + 1) in self.state.board for i in range(3)]) and \ - len({self.state.board[(i + 1, i + 1)] for i in range(3)}) == 1: - self.line_n(offset, offset*6/7, 1 - offset, (1 - offset)*6/7) + len({self.state.board[(i + 1, i + 1)] for i in range(3)}) == 1: + self.line_n(offset, offset * 6 / 7, 1 - offset, (1 - offset) * 6 / 7) if all([(i + 1, 3 - i) in self.state.board for i in range(3)]) and \ - len({self.state.board[(i + 1, 3 - i)] for i in range(3)}) == 1: - self.line_n(offset, (1 - offset)*6/7, 1 - offset, offset*6/7) + len({self.state.board[(i + 1, 3 - i)] for i in range(3)}) == 1: + self.line_n(offset, (1 - offset) * 6 / 7, 1 - offset, offset * 6 / 7) # restart button self.fill(0, 0, 255) - self.rect_n(0.5 + offset, 6/7, 0.4, 1/8) + self.rect_n(0.5 + offset, 6 / 7, 0.4, 1 / 8) self.fill(0, 0, 0) - self.text_n('Restart', 0.5 + 2*offset, 13/14) + self.text_n('Restart', 0.5 + 2 * offset, 13 / 14) else: # Print which player's turn it is self.text_n("Player {}'s move({})".format("XO"[self.turn], self.players[self.turn]), - offset, 6/7 + offset) + offset, 6 / 7 + offset) self.update() def draw_x(self, position): self.stroke(0, 255, 0) - x, y = [i-1 for i in position] - offset = 1/15 - self.line_n(x/3 + offset, (y/3 + offset)*6/7, x/3 + 1/3 - offset, (y/3 + 1/3 - offset)*6/7) - self.line_n(x/3 + 1/3 - offset, (y/3 + offset)*6/7, x/3 + offset, (y/3 + 1/3 - offset)*6/7) + x, y = [i - 1 for i in position] + offset = 1 / 15 + self.line_n(x / 3 + offset, (y / 3 + offset) * 6 / 7, x / 3 + 1 / 3 - offset, (y / 3 + 1 / 3 - offset) * 6 / 7) + self.line_n(x / 3 + 1 / 3 - offset, (y / 3 + offset) * 6 / 7, x / 3 + offset, (y / 3 + 1 / 3 - offset) * 6 / 7) def draw_o(self, position): self.stroke(255, 0, 0) - x, y = [i-1 for i in position] - self.arc_n(x/3 + 1/6, (y/3 + 1/6)*6/7, 1/9, 0, 360) + x, y = [i - 1 for i in position] + self.arc_n(x / 3 + 1 / 6, (y / 3 + 1 / 6) * 6 / 7, 1 / 9, 0, 360) + +class Canvas_min_max(Canvas): + """MinMax for Fig52Extended on HTML canvas""" -class Canvas_minimax(Canvas): - """Minimax for Fig52Extended on HTML canvas""" def __init__(self, varname, util_list, width=800, height=600, cid=None): - Canvas.__init__(self, varname, width, height, cid) - self.utils = {node:util for node, util in zip(range(13, 40), util_list)} + super.__init__(varname, width, height, cid) + self.utils = {node: util for node, util in zip(range(13, 40), util_list)} self.game = Fig52Extended() self.game.utils = self.utils self.nodes = list(range(40)) - self.l = 1/40 + self.l = 1 / 40 self.node_pos = {} for i in range(4): base = len(self.node_pos) - row_size = 3**i + row_size = 3 ** i for node in [base + j for j in range(row_size)]: - self.node_pos[node] = ((node - base)/row_size + 1/(2*row_size) - self.l/2, - self.l/2 + (self.l + (1 - 5*self.l)/3)*i) + self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2, + self.l / 2 + (self.l + (1 - 5 * self.l) / 3) * i) self.font("12px Arial") self.node_stack = [] self.explored = {node for node in self.utils} @@ -499,20 +504,21 @@ def __init__(self, varname, util_list, width=800, height=600, cid=None): self.draw_graph() self.stack_manager = self.stack_manager_gen() - def minimax(self, node): + def min_max(self, node): game = self.game player = game.to_move(node) + def max_value(node): if game.terminal_test(node): return game.utility(node, player) self.change_list.append(('a', node)) self.change_list.append(('h',)) - max_a = argmax(game.actions(node), key=lambda x: min_value(game.result(node, x))) + max_a = max(game.actions(node), key=lambda x: min_value(game.result(node, x))) max_node = game.result(node, max_a) self.utils[node] = self.utils[max_node] x1, y1 = self.node_pos[node] x2, y2 = self.node_pos[max_node] - self.change_list.append(('l', (node, max_node - 3*node - 1))) + self.change_list.append(('l', (node, max_node - 3 * node - 1))) self.change_list.append(('e', node)) self.change_list.append(('p',)) self.change_list.append(('h',)) @@ -523,12 +529,12 @@ def min_value(node): return game.utility(node, player) self.change_list.append(('a', node)) self.change_list.append(('h',)) - min_a = argmin(game.actions(node), key=lambda x: max_value(game.result(node, x))) + min_a = min(game.actions(node), key=lambda x: max_value(game.result(node, x))) min_node = game.result(node, min_a) self.utils[node] = self.utils[min_node] x1, y1 = self.node_pos[node] x2, y2 = self.node_pos[min_node] - self.change_list.append(('l', (node, min_node - 3*node - 1))) + self.change_list.append(('l', (node, min_node - 3 * node - 1))) self.change_list.append(('e', node)) self.change_list.append(('p',)) self.change_list.append(('h',)) @@ -537,7 +543,7 @@ def min_value(node): return max_value(node) def stack_manager_gen(self): - self.minimax(0) + self.min_max(0) for change in self.change_list: if change[0] == 'a': self.node_stack.append(change[1]) @@ -566,7 +572,7 @@ def draw_graph(self): for node in self.node_stack: x, y = self.node_pos[node] self.fill(200, 200, 0) - self.rect_n(x - self.l/5, y - self.l/5, self.l*7/5, self.l*7/5) + self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5) for node in self.nodes: x, y = self.node_pos[node] if node in self.explored: @@ -580,12 +586,12 @@ def draw_graph(self): self.line_n(x + self.l, y + self.l, x, y + self.l) self.fill(0, 0, 0) if node in self.explored: - self.text_n(self.utils[node], x + self.l/10, y + self.l*9/10) + self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10) # draw edges for i in range(13): - x1, y1 = self.node_pos[i][0] + self.l/2, self.node_pos[i][1] + self.l + x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l for j in range(3): - x2, y2 = self.node_pos[i*3 + j + 1][0] + self.l/2, self.node_pos[i*3 + j + 1][1] + x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1] if i in [1, 2, 3]: self.stroke(200, 0, 0) else: @@ -598,22 +604,23 @@ def draw_graph(self): self.update() -class Canvas_alphabeta(Canvas): +class Canvas_alpha_beta(Canvas): """Alpha-beta pruning for Fig52Extended on HTML canvas""" + def __init__(self, varname, util_list, width=800, height=600, cid=None): - Canvas.__init__(self, varname, width, height, cid) - self.utils = {node:util for node, util in zip(range(13, 40), util_list)} + super().__init__(varname, width, height, cid) + self.utils = {node: util for node, util in zip(range(13, 40), util_list)} self.game = Fig52Extended() self.game.utils = self.utils self.nodes = list(range(40)) - self.l = 1/40 + self.l = 1 / 40 self.node_pos = {} for i in range(4): base = len(self.node_pos) - row_size = 3**i + row_size = 3 ** i for node in [base + j for j in range(row_size)]: - self.node_pos[node] = ((node - base)/row_size + 1/(2*row_size) - self.l/2, - 3*self.l/2 + (self.l + (1 - 6*self.l)/3)*i) + self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2, + 3 * self.l / 2 + (self.l + (1 - 6 * self.l) / 3) * i) self.font("12px Arial") self.node_stack = [] self.explored = {node for node in self.utils} @@ -624,27 +631,27 @@ def __init__(self, varname, util_list, width=800, height=600, cid=None): self.draw_graph() self.stack_manager = self.stack_manager_gen() - def alphabeta_search(self, node): + def alpha_beta_search(self, node): game = self.game player = game.to_move(node) - # Functions used by alphabeta + # Functions used by alpha_beta def max_value(node, alpha, beta): if game.terminal_test(node): self.change_list.append(('a', node)) self.change_list.append(('h',)) self.change_list.append(('p',)) return game.utility(node, player) - v = -infinity + v = -np.inf self.change_list.append(('a', node)) - self.change_list.append(('ab',node, v, beta)) + self.change_list.append(('ab', node, v, beta)) self.change_list.append(('h',)) for a in game.actions(node): min_val = min_value(game.result(node, a), alpha, beta) if v < min_val: v = min_val max_node = game.result(node, a) - self.change_list.append(('ab',node, v, beta)) + self.change_list.append(('ab', node, v, beta)) if v >= beta: self.change_list.append(('h',)) self.pruned.add(node) @@ -652,8 +659,8 @@ def max_value(node, alpha, beta): alpha = max(alpha, v) self.utils[node] = v if node not in self.pruned: - self.change_list.append(('l', (node, max_node - 3*node - 1))) - self.change_list.append(('e',node)) + self.change_list.append(('l', (node, max_node - 3 * node - 1))) + self.change_list.append(('e', node)) self.change_list.append(('p',)) self.change_list.append(('h',)) return v @@ -664,16 +671,16 @@ def min_value(node, alpha, beta): self.change_list.append(('h',)) self.change_list.append(('p',)) return game.utility(node, player) - v = infinity + v = np.inf self.change_list.append(('a', node)) - self.change_list.append(('ab',node, alpha, v)) + self.change_list.append(('ab', node, alpha, v)) self.change_list.append(('h',)) for a in game.actions(node): max_val = max_value(game.result(node, a), alpha, beta) if v > max_val: v = max_val min_node = game.result(node, a) - self.change_list.append(('ab',node, alpha, v)) + self.change_list.append(('ab', node, alpha, v)) if v <= alpha: self.change_list.append(('h',)) self.pruned.add(node) @@ -681,16 +688,16 @@ def min_value(node, alpha, beta): beta = min(beta, v) self.utils[node] = v if node not in self.pruned: - self.change_list.append(('l', (node, min_node - 3*node - 1))) - self.change_list.append(('e',node)) + self.change_list.append(('l', (node, min_node - 3 * node - 1))) + self.change_list.append(('e', node)) self.change_list.append(('p',)) self.change_list.append(('h',)) return v - return max_value(node, -infinity, infinity) + return max_value(node, -np.inf, np.inf) def stack_manager_gen(self): - self.alphabeta_search(0) + self.alpha_beta_search(0) for change in self.change_list: if change[0] == 'a': self.node_stack.append(change[1]) @@ -725,7 +732,7 @@ def draw_graph(self): self.fill(200, 100, 100) else: self.fill(200, 200, 0) - self.rect_n(x - self.l/5, y - self.l/5, self.l*7/5, self.l*7/5) + self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5) for node in self.nodes: x, y = self.node_pos[node] if node in self.explored: @@ -742,12 +749,12 @@ def draw_graph(self): self.line_n(x + self.l, y + self.l, x, y + self.l) self.fill(0, 0, 0) if node in self.explored and node not in self.pruned: - self.text_n(self.utils[node], x + self.l/10, y + self.l*9/10) + self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10) # draw edges for i in range(13): - x1, y1 = self.node_pos[i][0] + self.l/2, self.node_pos[i][1] + self.l + x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l for j in range(3): - x2, y2 = self.node_pos[i*3 + j + 1][0] + self.l/2, self.node_pos[i*3 + j + 1][1] + x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1] if i in [1, 2, 3]: self.stroke(200, 0, 0) else: @@ -762,21 +769,22 @@ def draw_graph(self): if node not in self.explored: x, y = self.node_pos[node] alpha, beta = self.ab[node] - self.text_n(alpha, x - self.l/2, y - self.l/10) - self.text_n(beta, x + self.l, y - self.l/10) + self.text_n(alpha, x - self.l / 2, y - self.l / 10) + self.text_n(beta, x + self.l, y - self.l / 10) self.update() class Canvas_fol_bc_ask(Canvas): """fol_bc_ask() on HTML canvas""" + def __init__(self, varname, kb, query, width=800, height=600, cid=None): - Canvas.__init__(self, varname, width, height, cid) + super().__init__(varname, width, height, cid) self.kb = kb self.query = query - self.l = 1/20 - self.b = 3*self.l + self.l = 1 / 20 + self.b = 3 * self.l bc_out = list(self.fol_bc_ask()) - if len(bc_out) is 0: + if len(bc_out) == 0: self.valid = False else: self.valid = True @@ -794,10 +802,11 @@ def __init__(self, varname, kb, query, width=800, height=600, cid=None): def fol_bc_ask(self): KB = self.kb query = self.query + def fol_bc_or(KB, goal, theta): for rule in KB.fetch_rules_for_goal(goal): lhs, rhs = parse_definite_clause(standardize_variables(rule)) - for theta1 in fol_bc_and(KB, lhs, unify(rhs, goal, theta)): + for theta1 in fol_bc_and(KB, lhs, unify_mm(rhs, goal, theta)): yield ([(goal, theta1[0])], theta1[1]) def fol_bc_and(KB, goals, theta): @@ -830,22 +839,22 @@ def dfs(node, depth): return (depth, pos) dfs(graph, 0) - y_off = 0.85/len(table) + y_off = 0.85 / len(table) for i, row in enumerate(table): - x_off = 0.95/len(row) + x_off = 0.95 / len(row) for j, node in enumerate(row): - pos[(i, j)] = (0.025 + j*x_off + (x_off - self.b)/2, 0.025 + i*y_off + (y_off - self.l)/2) + pos[(i, j)] = (0.025 + j * x_off + (x_off - self.b) / 2, 0.025 + i * y_off + (y_off - self.l) / 2) for p, c in links: x1, y1 = pos[p] x2, y2 = pos[c] - edges.add((x1 + self.b/2, y1 + self.l, x2 + self.b/2, y2)) + edges.add((x1 + self.b / 2, y1 + self.l, x2 + self.b / 2, y2)) self.table = table self.pos = pos self.edges = edges def mouse_click(self, x, y): - x, y = x/self.width, y/self.height + x, y = x / self.width, y / self.height for node in self.pos: xs, ys = self.pos[node] xe, ye = xs + self.b, ys + self.l @@ -871,7 +880,7 @@ def draw_table(self): self.line_n(x, y + self.l, x + self.b, y + self.l) self.fill(0, 0, 0) self.text_n(self.table[i][j], x + 0.01, y + self.l - 0.01) - #draw edges + # draw edges for x1, y1, x2, y2 in self.edges: self.line_n(x1, y1, x2, y2) else: @@ -894,38 +903,30 @@ def draw_table(self): ##################### Functions to assist plotting in search.ipynb #################### ############################################################################################################ -import networkx as nx -import matplotlib.pyplot as plt -from matplotlib import lines -from ipywidgets import interact -import ipywidgets as widgets -from IPython.display import display -import time -from search import GraphProblem, romania_map -def show_map(graph_data, node_colors = None): +def show_map(graph_data, node_colors=None): G = nx.Graph(graph_data['graph_dict']) node_colors = node_colors or graph_data['node_colors'] node_positions = graph_data['node_positions'] node_label_pos = graph_data['node_label_positions'] - edge_weights= graph_data['edge_weights'] - + edge_weights = graph_data['edge_weights'] + # set the size of the plot - plt.figure(figsize=(18,13)) + plt.figure(figsize=(18, 13)) # draw the graph (both nodes and edges) with locations from romania_locations nx.draw(G, pos={k: node_positions[k] for k in G.nodes()}, node_color=[node_colors[node] for node in G.nodes()], linewidths=0.3, edgecolors='k') # draw labels for nodes node_label_handles = nx.draw_networkx_labels(G, pos=node_label_pos, font_size=14) - + # add a white bounding box behind the node labels [label.set_bbox(dict(facecolor='white', edgecolor='none')) for label in node_label_handles.values()] # add edge lables to the graph nx.draw_networkx_edge_labels(G, pos=node_positions, edge_labels=edge_weights, font_size=14) - + # add a legend white_circle = lines.Line2D([], [], color="white", marker='o', markersize=15, markerfacecolor="white") orange_circle = lines.Line2D([], [], color="orange", marker='o', markersize=15, markerfacecolor="orange") @@ -934,71 +935,74 @@ def show_map(graph_data, node_colors = None): green_circle = lines.Line2D([], [], color="green", marker='o', markersize=15, markerfacecolor="green") plt.legend((white_circle, orange_circle, red_circle, gray_circle, green_circle), ('Un-explored', 'Frontier', 'Currently Exploring', 'Explored', 'Final Solution'), - numpoints=1, prop={'size':16}, loc=(.8,.75)) - + numpoints=1, prop={'size': 16}, loc=(.8, .75)) + # show the plot. No need to use in notebooks. nx.draw will show the graph itself. plt.show() - -## helper functions for visualisations - + + +# helper functions for visualisations + def final_path_colors(initial_node_colors, problem, solution): "Return a node_colors dict of the final path provided the problem and solution." - + # get initial node colors final_colors = dict(initial_node_colors) # color all the nodes in solution and starting node to green final_colors[problem.initial] = "green" for node in solution: - final_colors[node] = "green" + final_colors[node] = "green" return final_colors + def display_visual(graph_data, user_input, algorithm=None, problem=None): initial_node_colors = graph_data['node_colors'] - if user_input == False: + if user_input is False: def slider_callback(iteration): # don't show graph for the first time running the cell calling this function try: show_map(graph_data, node_colors=all_node_colors[iteration]) except: pass - def visualize_callback(Visualize): - if Visualize is True: + + def visualize_callback(visualize): + if visualize is True: button.value = False - + global all_node_colors - + iterations, all_node_colors, node = algorithm(problem) solution = node.solution() all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution)) - + slider.max = len(all_node_colors) - 1 - + for i in range(slider.max + 1): slider.value = i - #time.sleep(.5) - + # time.sleep(.5) + slider = widgets.IntSlider(min=0, max=1, step=1, value=0) slider_visual = widgets.interactive(slider_callback, iteration=slider) display(slider_visual) button = widgets.ToggleButton(value=False) - button_visual = widgets.interactive(visualize_callback, Visualize=button) + button_visual = widgets.interactive(visualize_callback, visualize=button) display(button_visual) - - if user_input == True: + + if user_input is True: node_colors = dict(initial_node_colors) if isinstance(algorithm, dict): assert set(algorithm.keys()).issubset({"Breadth First Tree Search", - "Depth First Tree Search", - "Breadth First Search", - "Depth First Graph Search", - "Best First Graph Search", - "Uniform Cost Search", - "Depth Limited Search", - "Iterative Deepening Search", - "Greedy Best First Search", - "A-star Search", - "Recursive Best First Search"}) + "Depth First Tree Search", + "Breadth First Search", + "Depth First Graph Search", + "Best First Graph Search", + "Uniform Cost Search", + "Depth Limited Search", + "Iterative Deepening Search", + "Greedy Best First Search", + "A-star Search", + "Recursive Best First Search"}) algo_dropdown = widgets.Dropdown(description="Search algorithm: ", options=sorted(list(algorithm.keys())), @@ -1007,33 +1011,33 @@ def visualize_callback(Visualize): elif algorithm is None: print("No algorithm to run.") return 0 - + def slider_callback(iteration): # don't show graph for the first time running the cell calling this function try: show_map(graph_data, node_colors=all_node_colors[iteration]) except: pass - - def visualize_callback(Visualize): - if Visualize is True: + + def visualize_callback(visualize): + if visualize is True: button.value = False - + problem = GraphProblem(start_dropdown.value, end_dropdown.value, romania_map) global all_node_colors - + user_algorithm = algorithm[algo_dropdown.value] - + iterations, all_node_colors, node = user_algorithm(problem) solution = node.solution() all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution)) slider.max = len(all_node_colors) - 1 - + for i in range(slider.max + 1): slider.value = i - #time.sleep(.5) - + # time.sleep(.5) + start_dropdown = widgets.Dropdown(description="Start city: ", options=sorted(list(node_colors.keys())), value="Arad") display(start_dropdown) @@ -1041,11 +1045,11 @@ def visualize_callback(Visualize): end_dropdown = widgets.Dropdown(description="Goal city: ", options=sorted(list(node_colors.keys())), value="Fagaras") display(end_dropdown) - + button = widgets.ToggleButton(value=False) - button_visual = widgets.interactive(visualize_callback, Visualize=button) + button_visual = widgets.interactive(visualize_callback, visualize=button) display(button_visual) - + slider = widgets.IntSlider(min=0, max=1, step=1, value=0) slider_visual = widgets.interactive(slider_callback, iteration=slider) display(slider_visual) @@ -1054,7 +1058,7 @@ def visualize_callback(Visualize): # Function to plot NQueensCSP in csp.py and NQueensProblem in search.py def plot_NQueens(solution): n = len(solution) - board = np.array([2 * int((i + j) % 2) for j in range(n) for i in range(n)]).reshape((n, n)) + board = np.array([2 * int((i + j) % 2) for j in range(n) for i in range(n)]).reshape((n, n)) im = Image.open('images/queen_s.png') height = im.size[1] im = np.array(im).astype(np.float) / 255 @@ -1077,6 +1081,7 @@ def plot_NQueens(solution): fig.tight_layout() plt.show() + # Function to plot a heatmap, given a grid def heatmap(grid, cmap='binary', interpolation='nearest'): fig = plt.figure(figsize=(7, 7)) @@ -1086,13 +1091,15 @@ def heatmap(grid, cmap='binary', interpolation='nearest'): fig.tight_layout() plt.show() + # Generates a gaussian kernel def gaussian_kernel(l=5, sig=1.0): ax = np.arange(-l // 2 + 1., l // 2 + 1.) xx, yy = np.meshgrid(ax, ax) - kernel = np.exp(-(xx**2 + yy**2) / (2. * sig**2)) + kernel = np.exp(-(xx ** 2 + yy ** 2) / (2. * sig ** 2)) return kernel + # Plots utility function for a POMDP def plot_pomdp_utility(utility): save = utility['0'][0] @@ -1109,7 +1116,7 @@ def plot_pomdp_utility(utility): plt.vlines([left, right], -20, 10, linestyles='dashed', colors='c') plt.ylim(-20, 13) plt.xlim(0, 1) - plt.text(left/2 - 0.05, 10, 'Save') - plt.text((right + left)/2 - 0.02, 10, 'Ask') - plt.text((right + 1)/2 - 0.07, 10, 'Delete') + plt.text(left / 2 - 0.05, 10, 'Save') + plt.text((right + left) / 2 - 0.02, 10, 'Ask') + plt.text((right + 1) / 2 - 0.07, 10, 'Delete') plt.show() diff --git a/notebook4e.py b/notebook4e.py new file mode 100644 index 000000000..5b03081c6 --- /dev/null +++ b/notebook4e.py @@ -0,0 +1,1158 @@ +import time +from collections import defaultdict +from inspect import getsource + +import ipywidgets as widgets +import matplotlib.pyplot as plt +import networkx as nx +import numpy as np +from IPython.display import HTML +from IPython.display import display +from PIL import Image +from matplotlib import lines +from matplotlib.colors import ListedColormap + +from games import TicTacToe, alpha_beta_player, random_player, Fig52Extended +from learning import DataSet +from logic import parse_definite_clause, standardize_variables, unify_mm, subst +from search import GraphProblem, romania_map + + +# ______________________________________________________________________________ +# Magic Words + + +def pseudocode(algorithm): + """Print the pseudocode for the given algorithm.""" + from urllib.request import urlopen + from IPython.display import Markdown + + algorithm = algorithm.replace(' ', '-') + url = "https://raw.githubusercontent.com/aimacode/aima-pseudocode/master/md/{}.md".format(algorithm) + f = urlopen(url) + md = f.read().decode('utf-8') + md = md.split('\n', 1)[-1].strip() + md = '#' + md + return Markdown(md) + + +def psource(*functions): + """Print the source code for the given function(s).""" + source_code = '\n\n'.join(getsource(fn) for fn in functions) + try: + from pygments.formatters import HtmlFormatter + from pygments.lexers import PythonLexer + from pygments import highlight + + display(HTML(highlight(source_code, PythonLexer(), HtmlFormatter(full=True)))) + + except ImportError: + print(source_code) + + +def plot_model_boundary(dataset, attr1, attr2, model=None): + # prepare data + examples = np.asarray(dataset.examples) + X = np.asarray([examples[:, attr1], examples[:, attr2]]) + y = examples[:, dataset.target] + h = 0.1 + + # create color maps + cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA', '#00AAFF']) + cmap_bold = ListedColormap(['#FF0000', '#00FF00', '#00AAFF']) + + # calculate min, max and limits + x_min, x_max = X[0].min() - 1, X[0].max() + 1 + y_min, y_max = X[1].min() - 1, X[1].max() + 1 + # mesh the grid + xx, yy = np.meshgrid(np.arange(x_min, x_max, h), + np.arange(y_min, y_max, h)) + Z = [] + for grid in zip(xx.ravel(), yy.ravel()): + # put them back to the example + grid = np.round(grid, decimals=1).tolist() + Z.append(model(grid)) + # Put the result into a color plot + Z = np.asarray(Z) + Z = Z.reshape(xx.shape) + plt.figure() + plt.pcolormesh(xx, yy, Z, cmap=cmap_light) + + # Plot also the training points + plt.scatter(X[0], X[1], c=y, cmap=cmap_bold) + plt.xlim(xx.min(), xx.max()) + plt.ylim(yy.min(), yy.max()) + plt.show() + + +# ______________________________________________________________________________ +# Iris Visualization + + +def show_iris(i=0, j=1, k=2): + """Plots the iris dataset in a 3D plot. + The three axes are given by i, j and k, + which correspond to three of the four iris features.""" + + plt.rcParams.update(plt.rcParamsDefault) + + fig = plt.figure() + ax = fig.add_subplot(111, projection='3d') + + iris = DataSet(name="iris") + buckets = iris.split_values_by_classes() + + features = ["Sepal Length", "Sepal Width", "Petal Length", "Petal Width"] + f1, f2, f3 = features[i], features[j], features[k] + + a_setosa = [v[i] for v in buckets["setosa"]] + b_setosa = [v[j] for v in buckets["setosa"]] + c_setosa = [v[k] for v in buckets["setosa"]] + + a_virginica = [v[i] for v in buckets["virginica"]] + b_virginica = [v[j] for v in buckets["virginica"]] + c_virginica = [v[k] for v in buckets["virginica"]] + + a_versicolor = [v[i] for v in buckets["versicolor"]] + b_versicolor = [v[j] for v in buckets["versicolor"]] + c_versicolor = [v[k] for v in buckets["versicolor"]] + + for c, m, sl, sw, pl in [('b', 's', a_setosa, b_setosa, c_setosa), + ('g', '^', a_virginica, b_virginica, c_virginica), + ('r', 'o', a_versicolor, b_versicolor, c_versicolor)]: + ax.scatter(sl, sw, pl, c=c, marker=m) + + ax.set_xlabel(f1) + ax.set_ylabel(f2) + ax.set_zlabel(f3) + + plt.show() + + +# ______________________________________________________________________________ +# MNIST + + +def load_MNIST(path="aima-data/MNIST/Digits", fashion=False): + import os, struct + import array + import numpy as np + + if fashion: + path = "aima-data/MNIST/Fashion" + + plt.rcParams.update(plt.rcParamsDefault) + plt.rcParams['figure.figsize'] = (10.0, 8.0) + plt.rcParams['image.interpolation'] = 'nearest' + plt.rcParams['image.cmap'] = 'gray' + + train_img_file = open(os.path.join(path, "train-images-idx3-ubyte"), "rb") + train_lbl_file = open(os.path.join(path, "train-labels-idx1-ubyte"), "rb") + test_img_file = open(os.path.join(path, "t10k-images-idx3-ubyte"), "rb") + test_lbl_file = open(os.path.join(path, 't10k-labels-idx1-ubyte'), "rb") + + magic_nr, tr_size, tr_rows, tr_cols = struct.unpack(">IIII", train_img_file.read(16)) + tr_img = array.array("B", train_img_file.read()) + train_img_file.close() + magic_nr, tr_size = struct.unpack(">II", train_lbl_file.read(8)) + tr_lbl = array.array("b", train_lbl_file.read()) + train_lbl_file.close() + + magic_nr, te_size, te_rows, te_cols = struct.unpack(">IIII", test_img_file.read(16)) + te_img = array.array("B", test_img_file.read()) + test_img_file.close() + magic_nr, te_size = struct.unpack(">II", test_lbl_file.read(8)) + te_lbl = array.array("b", test_lbl_file.read()) + test_lbl_file.close() + + # print(len(tr_img), len(tr_lbl), tr_size) + # print(len(te_img), len(te_lbl), te_size) + + train_img = np.zeros((tr_size, tr_rows * tr_cols), dtype=np.int16) + train_lbl = np.zeros((tr_size,), dtype=np.int8) + for i in range(tr_size): + train_img[i] = np.array(tr_img[i * tr_rows * tr_cols: (i + 1) * tr_rows * tr_cols]).reshape((tr_rows * te_cols)) + train_lbl[i] = tr_lbl[i] + + test_img = np.zeros((te_size, te_rows * te_cols), dtype=np.int16) + test_lbl = np.zeros((te_size,), dtype=np.int8) + for i in range(te_size): + test_img[i] = np.array(te_img[i * te_rows * te_cols: (i + 1) * te_rows * te_cols]).reshape((te_rows * te_cols)) + test_lbl[i] = te_lbl[i] + + return (train_img, train_lbl, test_img, test_lbl) + + +digit_classes = [str(i) for i in range(10)] +fashion_classes = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat", + "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"] + + +def show_MNIST(labels, images, samples=8, fashion=False): + if not fashion: + classes = digit_classes + else: + classes = fashion_classes + + num_classes = len(classes) + + for y, cls in enumerate(classes): + idxs = np.nonzero([i == y for i in labels]) + idxs = np.random.choice(idxs[0], samples, replace=False) + for i, idx in enumerate(idxs): + plt_idx = i * num_classes + y + 1 + plt.subplot(samples, num_classes, plt_idx) + plt.imshow(images[idx].reshape((28, 28))) + plt.axis("off") + if i == 0: + plt.title(cls) + + plt.show() + + +def show_ave_MNIST(labels, images, fashion=False): + if not fashion: + item_type = "Digit" + classes = digit_classes + else: + item_type = "Apparel" + classes = fashion_classes + + num_classes = len(classes) + + for y, cls in enumerate(classes): + idxs = np.nonzero([i == y for i in labels]) + print(item_type, y, ":", len(idxs[0]), "images.") + + ave_img = np.mean(np.vstack([images[i] for i in idxs[0]]), axis=0) + # print(ave_img.shape) + + plt.subplot(1, num_classes, y + 1) + plt.imshow(ave_img.reshape((28, 28))) + plt.axis("off") + plt.title(cls) + + plt.show() + + +# ______________________________________________________________________________ +# MDP + + +def make_plot_grid_step_function(columns, rows, U_over_time): + """ipywidgets interactive function supports single parameter as input. + This function creates and return such a function by taking as input + other parameters.""" + + def plot_grid_step(iteration): + data = U_over_time[iteration] + data = defaultdict(lambda: 0, data) + grid = [] + for row in range(rows): + current_row = [] + for column in range(columns): + current_row.append(data[(column, row)]) + grid.append(current_row) + grid.reverse() # output like book + fig = plt.imshow(grid, cmap=plt.cm.bwr, interpolation='nearest') + + plt.axis('off') + fig.axes.get_xaxis().set_visible(False) + fig.axes.get_yaxis().set_visible(False) + + for col in range(len(grid)): + for row in range(len(grid[0])): + magic = grid[col][row] + fig.axes.text(row, col, "{0:.2f}".format(magic), va='center', ha='center') + + plt.show() + + return plot_grid_step + + +def make_visualize(slider): + """Takes an input a sliderand returns callback function + for timer and animation.""" + + def visualize_callback(visualize, time_step): + if visualize is True: + for i in range(slider.min, slider.max + 1): + slider.value = i + time.sleep(float(time_step)) + + return visualize_callback + + +# ______________________________________________________________________________ + + +_canvas = """ + +
    + +
    + + +""" # noqa + + +class Canvas: + """Inherit from this class to manage the HTML canvas element in jupyter notebooks. + To create an object of this class any_name_xyz = Canvas("any_name_xyz") + The first argument given must be the name of the object being created. + IPython must be able to reference the variable name that is being passed.""" + + def __init__(self, varname, width=800, height=600, cid=None): + self.name = varname + self.cid = cid or varname + self.width = width + self.height = height + self.html = _canvas.format(self.cid, self.width, self.height, self.name) + self.exec_list = [] + display_html(self.html) + + def mouse_click(self, x, y): + """Override this method to handle mouse click at position (x, y)""" + raise NotImplementedError + + def mouse_move(self, x, y): + raise NotImplementedError + + def execute(self, exec_str): + """Stores the command to be executed to a list which is used later during update()""" + if not isinstance(exec_str, str): + print("Invalid execution argument:", exec_str) + self.alert("Received invalid execution command format") + prefix = "{0}_canvas_object.".format(self.cid) + self.exec_list.append(prefix + exec_str + ';') + + def fill(self, r, g, b): + """Changes the fill color to a color in rgb format""" + self.execute("fill({0}, {1}, {2})".format(r, g, b)) + + def stroke(self, r, g, b): + """Changes the colors of line/strokes to rgb""" + self.execute("stroke({0}, {1}, {2})".format(r, g, b)) + + def strokeWidth(self, w): + """Changes the width of lines/strokes to 'w' pixels""" + self.execute("strokeWidth({0})".format(w)) + + def rect(self, x, y, w, h): + """Draw a rectangle with 'w' width, 'h' height and (x, y) as the top-left corner""" + self.execute("rect({0}, {1}, {2}, {3})".format(x, y, w, h)) + + def rect_n(self, xn, yn, wn, hn): + """Similar to rect(), but the dimensions are normalized to fall between 0 and 1""" + x = round(xn * self.width) + y = round(yn * self.height) + w = round(wn * self.width) + h = round(hn * self.height) + self.rect(x, y, w, h) + + def line(self, x1, y1, x2, y2): + """Draw a line from (x1, y1) to (x2, y2)""" + self.execute("line({0}, {1}, {2}, {3})".format(x1, y1, x2, y2)) + + def line_n(self, x1n, y1n, x2n, y2n): + """Similar to line(), but the dimensions are normalized to fall between 0 and 1""" + x1 = round(x1n * self.width) + y1 = round(y1n * self.height) + x2 = round(x2n * self.width) + y2 = round(y2n * self.height) + self.line(x1, y1, x2, y2) + + def arc(self, x, y, r, start, stop): + """Draw an arc with (x, y) as centre, 'r' as radius from angles 'start' to 'stop'""" + self.execute("arc({0}, {1}, {2}, {3}, {4})".format(x, y, r, start, stop)) + + def arc_n(self, xn, yn, rn, start, stop): + """Similar to arc(), but the dimensions are normalized to fall between 0 and 1 + The normalizing factor for radius is selected between width and height by + seeing which is smaller.""" + x = round(xn * self.width) + y = round(yn * self.height) + r = round(rn * min(self.width, self.height)) + self.arc(x, y, r, start, stop) + + def clear(self): + """Clear the HTML canvas""" + self.execute("clear()") + + def font(self, font): + """Changes the font of text""" + self.execute('font("{0}")'.format(font)) + + def text(self, txt, x, y, fill=True): + """Display a text at (x, y)""" + if fill: + self.execute('fill_text("{0}", {1}, {2})'.format(txt, x, y)) + else: + self.execute('stroke_text("{0}", {1}, {2})'.format(txt, x, y)) + + def text_n(self, txt, xn, yn, fill=True): + """Similar to text(), but with normalized coordinates""" + x = round(xn * self.width) + y = round(yn * self.height) + self.text(txt, x, y, fill) + + def alert(self, message): + """Immediately display an alert""" + display_html(''.format(message)) + + def update(self): + """Execute the JS code to execute the commands queued by execute()""" + exec_code = "" + self.exec_list = [] + display_html(exec_code) + + +def display_html(html_string): + display(HTML(html_string)) + + +################################################################################ + + +class Canvas_TicTacToe(Canvas): + """Play a 3x3 TicTacToe game on HTML canvas""" + + def __init__(self, varname, player_1='human', player_2='random', + width=300, height=350, cid=None): + valid_players = ('human', 'random', 'alpha_beta') + if player_1 not in valid_players or player_2 not in valid_players: + raise TypeError("Players must be one of {}".format(valid_players)) + super().__init__(varname, width, height, cid) + self.ttt = TicTacToe() + self.state = self.ttt.initial + self.turn = 0 + self.strokeWidth(5) + self.players = (player_1, player_2) + self.font("20px Arial") + self.draw_board() + + def mouse_click(self, x, y): + player = self.players[self.turn] + if self.ttt.terminal_test(self.state): + if 0.55 <= x / self.width <= 0.95 and 6 / 7 <= y / self.height <= 6 / 7 + 1 / 8: + self.state = self.ttt.initial + self.turn = 0 + self.draw_board() + return + + if player == 'human': + x, y = int(3 * x / self.width) + 1, int(3 * y / (self.height * 6 / 7)) + 1 + if (x, y) not in self.ttt.actions(self.state): + # Invalid move + return + move = (x, y) + elif player == 'alpha_beta': + move = alpha_beta_player(self.ttt, self.state) + else: + move = random_player(self.ttt, self.state) + self.state = self.ttt.result(self.state, move) + self.turn ^= 1 + self.draw_board() + + def draw_board(self): + self.clear() + self.stroke(0, 0, 0) + offset = 1 / 20 + self.line_n(0 + offset, (1 / 3) * 6 / 7, 1 - offset, (1 / 3) * 6 / 7) + self.line_n(0 + offset, (2 / 3) * 6 / 7, 1 - offset, (2 / 3) * 6 / 7) + self.line_n(1 / 3, (0 + offset) * 6 / 7, 1 / 3, (1 - offset) * 6 / 7) + self.line_n(2 / 3, (0 + offset) * 6 / 7, 2 / 3, (1 - offset) * 6 / 7) + + board = self.state.board + for mark in board: + if board[mark] == 'X': + self.draw_x(mark) + elif board[mark] == 'O': + self.draw_o(mark) + if self.ttt.terminal_test(self.state): + # End game message + utility = self.ttt.utility(self.state, self.ttt.to_move(self.ttt.initial)) + if utility == 0: + self.text_n('Game Draw!', offset, 6 / 7 + offset) + else: + self.text_n('Player {} wins!'.format("XO"[utility < 0]), offset, 6 / 7 + offset) + # Find the 3 and draw a line + self.stroke([255, 0][self.turn], [0, 255][self.turn], 0) + for i in range(3): + if all([(i + 1, j + 1) in self.state.board for j in range(3)]) and \ + len({self.state.board[(i + 1, j + 1)] for j in range(3)}) == 1: + self.line_n(i / 3 + 1 / 6, offset * 6 / 7, i / 3 + 1 / 6, (1 - offset) * 6 / 7) + if all([(j + 1, i + 1) in self.state.board for j in range(3)]) and \ + len({self.state.board[(j + 1, i + 1)] for j in range(3)}) == 1: + self.line_n(offset, (i / 3 + 1 / 6) * 6 / 7, 1 - offset, (i / 3 + 1 / 6) * 6 / 7) + if all([(i + 1, i + 1) in self.state.board for i in range(3)]) and \ + len({self.state.board[(i + 1, i + 1)] for i in range(3)}) == 1: + self.line_n(offset, offset * 6 / 7, 1 - offset, (1 - offset) * 6 / 7) + if all([(i + 1, 3 - i) in self.state.board for i in range(3)]) and \ + len({self.state.board[(i + 1, 3 - i)] for i in range(3)}) == 1: + self.line_n(offset, (1 - offset) * 6 / 7, 1 - offset, offset * 6 / 7) + # restart button + self.fill(0, 0, 255) + self.rect_n(0.5 + offset, 6 / 7, 0.4, 1 / 8) + self.fill(0, 0, 0) + self.text_n('Restart', 0.5 + 2 * offset, 13 / 14) + else: # Print which player's turn it is + self.text_n("Player {}'s move({})".format("XO"[self.turn], self.players[self.turn]), + offset, 6 / 7 + offset) + + self.update() + + def draw_x(self, position): + self.stroke(0, 255, 0) + x, y = [i - 1 for i in position] + offset = 1 / 15 + self.line_n(x / 3 + offset, (y / 3 + offset) * 6 / 7, x / 3 + 1 / 3 - offset, (y / 3 + 1 / 3 - offset) * 6 / 7) + self.line_n(x / 3 + 1 / 3 - offset, (y / 3 + offset) * 6 / 7, x / 3 + offset, (y / 3 + 1 / 3 - offset) * 6 / 7) + + def draw_o(self, position): + self.stroke(255, 0, 0) + x, y = [i - 1 for i in position] + self.arc_n(x / 3 + 1 / 6, (y / 3 + 1 / 6) * 6 / 7, 1 / 9, 0, 360) + + +class Canvas_min_max(Canvas): + """MinMax for Fig52Extended on HTML canvas""" + + def __init__(self, varname, util_list, width=800, height=600, cid=None): + super().__init__(varname, width, height, cid) + self.utils = {node: util for node, util in zip(range(13, 40), util_list)} + self.game = Fig52Extended() + self.game.utils = self.utils + self.nodes = list(range(40)) + self.l = 1 / 40 + self.node_pos = {} + for i in range(4): + base = len(self.node_pos) + row_size = 3 ** i + for node in [base + j for j in range(row_size)]: + self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2, + self.l / 2 + (self.l + (1 - 5 * self.l) / 3) * i) + self.font("12px Arial") + self.node_stack = [] + self.explored = {node for node in self.utils} + self.thick_lines = set() + self.change_list = [] + self.draw_graph() + self.stack_manager = self.stack_manager_gen() + + def min_max(self, node): + game = self.game + player = game.to_move(node) + + def max_value(node): + if game.terminal_test(node): + return game.utility(node, player) + self.change_list.append(('a', node)) + self.change_list.append(('h',)) + max_a = max(game.actions(node), key=lambda x: min_value(game.result(node, x))) + max_node = game.result(node, max_a) + self.utils[node] = self.utils[max_node] + x1, y1 = self.node_pos[node] + x2, y2 = self.node_pos[max_node] + self.change_list.append(('l', (node, max_node - 3 * node - 1))) + self.change_list.append(('e', node)) + self.change_list.append(('p',)) + self.change_list.append(('h',)) + return self.utils[node] + + def min_value(node): + if game.terminal_test(node): + return game.utility(node, player) + self.change_list.append(('a', node)) + self.change_list.append(('h',)) + min_a = min(game.actions(node), key=lambda x: max_value(game.result(node, x))) + min_node = game.result(node, min_a) + self.utils[node] = self.utils[min_node] + x1, y1 = self.node_pos[node] + x2, y2 = self.node_pos[min_node] + self.change_list.append(('l', (node, min_node - 3 * node - 1))) + self.change_list.append(('e', node)) + self.change_list.append(('p',)) + self.change_list.append(('h',)) + return self.utils[node] + + return max_value(node) + + def stack_manager_gen(self): + self.min_max(0) + for change in self.change_list: + if change[0] == 'a': + self.node_stack.append(change[1]) + elif change[0] == 'e': + self.explored.add(change[1]) + elif change[0] == 'h': + yield + elif change[0] == 'l': + self.thick_lines.add(change[1]) + elif change[0] == 'p': + self.node_stack.pop() + + def mouse_click(self, x, y): + try: + self.stack_manager.send(None) + except StopIteration: + pass + self.draw_graph() + + def draw_graph(self): + self.clear() + # draw nodes + self.stroke(0, 0, 0) + self.strokeWidth(1) + # highlight for nodes in stack + for node in self.node_stack: + x, y = self.node_pos[node] + self.fill(200, 200, 0) + self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5) + for node in self.nodes: + x, y = self.node_pos[node] + if node in self.explored: + self.fill(255, 255, 255) + else: + self.fill(200, 200, 200) + self.rect_n(x, y, self.l, self.l) + self.line_n(x, y, x + self.l, y) + self.line_n(x, y, x, y + self.l) + self.line_n(x + self.l, y + self.l, x + self.l, y) + self.line_n(x + self.l, y + self.l, x, y + self.l) + self.fill(0, 0, 0) + if node in self.explored: + self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10) + # draw edges + for i in range(13): + x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l + for j in range(3): + x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1] + if i in [1, 2, 3]: + self.stroke(200, 0, 0) + else: + self.stroke(0, 200, 0) + if (i, j) in self.thick_lines: + self.strokeWidth(3) + else: + self.strokeWidth(1) + self.line_n(x1, y1, x2, y2) + self.update() + + +class Canvas_alpha_beta(Canvas): + """Alpha-beta pruning for Fig52Extended on HTML canvas""" + + def __init__(self, varname, util_list, width=800, height=600, cid=None): + super().__init__(varname, width, height, cid) + self.utils = {node: util for node, util in zip(range(13, 40), util_list)} + self.game = Fig52Extended() + self.game.utils = self.utils + self.nodes = list(range(40)) + self.l = 1 / 40 + self.node_pos = {} + for i in range(4): + base = len(self.node_pos) + row_size = 3 ** i + for node in [base + j for j in range(row_size)]: + self.node_pos[node] = ((node - base) / row_size + 1 / (2 * row_size) - self.l / 2, + 3 * self.l / 2 + (self.l + (1 - 6 * self.l) / 3) * i) + self.font("12px Arial") + self.node_stack = [] + self.explored = {node for node in self.utils} + self.pruned = set() + self.ab = {} + self.thick_lines = set() + self.change_list = [] + self.draw_graph() + self.stack_manager = self.stack_manager_gen() + + def alpha_beta_search(self, node): + game = self.game + player = game.to_move(node) + + # Functions used by alpha_beta + def max_value(node, alpha, beta): + if game.terminal_test(node): + self.change_list.append(('a', node)) + self.change_list.append(('h',)) + self.change_list.append(('p',)) + return game.utility(node, player) + v = -np.inf + self.change_list.append(('a', node)) + self.change_list.append(('ab', node, v, beta)) + self.change_list.append(('h',)) + for a in game.actions(node): + min_val = min_value(game.result(node, a), alpha, beta) + if v < min_val: + v = min_val + max_node = game.result(node, a) + self.change_list.append(('ab', node, v, beta)) + if v >= beta: + self.change_list.append(('h',)) + self.pruned.add(node) + break + alpha = max(alpha, v) + self.utils[node] = v + if node not in self.pruned: + self.change_list.append(('l', (node, max_node - 3 * node - 1))) + self.change_list.append(('e', node)) + self.change_list.append(('p',)) + self.change_list.append(('h',)) + return v + + def min_value(node, alpha, beta): + if game.terminal_test(node): + self.change_list.append(('a', node)) + self.change_list.append(('h',)) + self.change_list.append(('p',)) + return game.utility(node, player) + v = np.inf + self.change_list.append(('a', node)) + self.change_list.append(('ab', node, alpha, v)) + self.change_list.append(('h',)) + for a in game.actions(node): + max_val = max_value(game.result(node, a), alpha, beta) + if v > max_val: + v = max_val + min_node = game.result(node, a) + self.change_list.append(('ab', node, alpha, v)) + if v <= alpha: + self.change_list.append(('h',)) + self.pruned.add(node) + break + beta = min(beta, v) + self.utils[node] = v + if node not in self.pruned: + self.change_list.append(('l', (node, min_node - 3 * node - 1))) + self.change_list.append(('e', node)) + self.change_list.append(('p',)) + self.change_list.append(('h',)) + return v + + return max_value(node, -np.inf, np.inf) + + def stack_manager_gen(self): + self.alpha_beta_search(0) + for change in self.change_list: + if change[0] == 'a': + self.node_stack.append(change[1]) + elif change[0] == 'ab': + self.ab[change[1]] = change[2:] + elif change[0] == 'e': + self.explored.add(change[1]) + elif change[0] == 'h': + yield + elif change[0] == 'l': + self.thick_lines.add(change[1]) + elif change[0] == 'p': + self.node_stack.pop() + + def mouse_click(self, x, y): + try: + self.stack_manager.send(None) + except StopIteration: + pass + self.draw_graph() + + def draw_graph(self): + self.clear() + # draw nodes + self.stroke(0, 0, 0) + self.strokeWidth(1) + # highlight for nodes in stack + for node in self.node_stack: + x, y = self.node_pos[node] + # alpha > beta + if node not in self.explored and self.ab[node][0] > self.ab[node][1]: + self.fill(200, 100, 100) + else: + self.fill(200, 200, 0) + self.rect_n(x - self.l / 5, y - self.l / 5, self.l * 7 / 5, self.l * 7 / 5) + for node in self.nodes: + x, y = self.node_pos[node] + if node in self.explored: + if node in self.pruned: + self.fill(50, 50, 50) + else: + self.fill(255, 255, 255) + else: + self.fill(200, 200, 200) + self.rect_n(x, y, self.l, self.l) + self.line_n(x, y, x + self.l, y) + self.line_n(x, y, x, y + self.l) + self.line_n(x + self.l, y + self.l, x + self.l, y) + self.line_n(x + self.l, y + self.l, x, y + self.l) + self.fill(0, 0, 0) + if node in self.explored and node not in self.pruned: + self.text_n(self.utils[node], x + self.l / 10, y + self.l * 9 / 10) + # draw edges + for i in range(13): + x1, y1 = self.node_pos[i][0] + self.l / 2, self.node_pos[i][1] + self.l + for j in range(3): + x2, y2 = self.node_pos[i * 3 + j + 1][0] + self.l / 2, self.node_pos[i * 3 + j + 1][1] + if i in [1, 2, 3]: + self.stroke(200, 0, 0) + else: + self.stroke(0, 200, 0) + if (i, j) in self.thick_lines: + self.strokeWidth(3) + else: + self.strokeWidth(1) + self.line_n(x1, y1, x2, y2) + # display alpha and beta + for node in self.node_stack: + if node not in self.explored: + x, y = self.node_pos[node] + alpha, beta = self.ab[node] + self.text_n(alpha, x - self.l / 2, y - self.l / 10) + self.text_n(beta, x + self.l, y - self.l / 10) + self.update() + + +class Canvas_fol_bc_ask(Canvas): + """fol_bc_ask() on HTML canvas""" + + def __init__(self, varname, kb, query, width=800, height=600, cid=None): + super().__init__(varname, width, height, cid) + self.kb = kb + self.query = query + self.l = 1 / 20 + self.b = 3 * self.l + bc_out = list(self.fol_bc_ask()) + if len(bc_out) == 0: + self.valid = False + else: + self.valid = True + graph = bc_out[0][0][0] + s = bc_out[0][1] + while True: + new_graph = subst(s, graph) + if graph == new_graph: + break + graph = new_graph + self.make_table(graph) + self.context = None + self.draw_table() + + def fol_bc_ask(self): + KB = self.kb + query = self.query + + def fol_bc_or(KB, goal, theta): + for rule in KB.fetch_rules_for_goal(goal): + lhs, rhs = parse_definite_clause(standardize_variables(rule)) + for theta1 in fol_bc_and(KB, lhs, unify_mm(rhs, goal, theta)): + yield ([(goal, theta1[0])], theta1[1]) + + def fol_bc_and(KB, goals, theta): + if theta is None: + pass + elif not goals: + yield ([], theta) + else: + first, rest = goals[0], goals[1:] + for theta1 in fol_bc_or(KB, subst(theta, first), theta): + for theta2 in fol_bc_and(KB, rest, theta1[1]): + yield (theta1[0] + theta2[0], theta2[1]) + + return fol_bc_or(KB, query, {}) + + def make_table(self, graph): + table = [] + pos = {} + links = set() + edges = set() + + def dfs(node, depth): + if len(table) <= depth: + table.append([]) + pos = len(table[depth]) + table[depth].append(node[0]) + for child in node[1]: + child_id = dfs(child, depth + 1) + links.add(((depth, pos), child_id)) + return (depth, pos) + + dfs(graph, 0) + y_off = 0.85 / len(table) + for i, row in enumerate(table): + x_off = 0.95 / len(row) + for j, node in enumerate(row): + pos[(i, j)] = (0.025 + j * x_off + (x_off - self.b) / 2, 0.025 + i * y_off + (y_off - self.l) / 2) + for p, c in links: + x1, y1 = pos[p] + x2, y2 = pos[c] + edges.add((x1 + self.b / 2, y1 + self.l, x2 + self.b / 2, y2)) + + self.table = table + self.pos = pos + self.edges = edges + + def mouse_click(self, x, y): + x, y = x / self.width, y / self.height + for node in self.pos: + xs, ys = self.pos[node] + xe, ye = xs + self.b, ys + self.l + if xs <= x <= xe and ys <= y <= ye: + self.context = node + break + self.draw_table() + + def draw_table(self): + self.clear() + self.strokeWidth(3) + self.stroke(0, 0, 0) + self.font("12px Arial") + if self.valid: + # draw nodes + for i, j in self.pos: + x, y = self.pos[(i, j)] + self.fill(200, 200, 200) + self.rect_n(x, y, self.b, self.l) + self.line_n(x, y, x + self.b, y) + self.line_n(x, y, x, y + self.l) + self.line_n(x + self.b, y, x + self.b, y + self.l) + self.line_n(x, y + self.l, x + self.b, y + self.l) + self.fill(0, 0, 0) + self.text_n(self.table[i][j], x + 0.01, y + self.l - 0.01) + # draw edges + for x1, y1, x2, y2 in self.edges: + self.line_n(x1, y1, x2, y2) + else: + self.fill(255, 0, 0) + self.rect_n(0, 0, 1, 1) + # text area + self.fill(255, 255, 255) + self.rect_n(0, 0.9, 1, 0.1) + self.strokeWidth(5) + self.stroke(0, 0, 0) + self.line_n(0, 0.9, 1, 0.9) + self.font("22px Arial") + self.fill(0, 0, 0) + self.text_n(self.table[self.context[0]][self.context[1]] if self.context else "Click for text", 0.025, 0.975) + self.update() + + +############################################################################################################ + +##################### Functions to assist plotting in search.ipynb #################### + +############################################################################################################ + + +def show_map(graph_data, node_colors=None): + G = nx.Graph(graph_data['graph_dict']) + node_colors = node_colors or graph_data['node_colors'] + node_positions = graph_data['node_positions'] + node_label_pos = graph_data['node_label_positions'] + edge_weights = graph_data['edge_weights'] + + # set the size of the plot + plt.figure(figsize=(18, 13)) + # draw the graph (both nodes and edges) with locations from romania_locations + nx.draw(G, pos={k: node_positions[k] for k in G.nodes()}, + node_color=[node_colors[node] for node in G.nodes()], linewidths=0.3, edgecolors='k') + + # draw labels for nodes + node_label_handles = nx.draw_networkx_labels(G, pos=node_label_pos, font_size=14) + + # add a white bounding box behind the node labels + [label.set_bbox(dict(facecolor='white', edgecolor='none')) for label in node_label_handles.values()] + + # add edge lables to the graph + nx.draw_networkx_edge_labels(G, pos=node_positions, edge_labels=edge_weights, font_size=14) + + # add a legend + white_circle = lines.Line2D([], [], color="white", marker='o', markersize=15, markerfacecolor="white") + orange_circle = lines.Line2D([], [], color="orange", marker='o', markersize=15, markerfacecolor="orange") + red_circle = lines.Line2D([], [], color="red", marker='o', markersize=15, markerfacecolor="red") + gray_circle = lines.Line2D([], [], color="gray", marker='o', markersize=15, markerfacecolor="gray") + green_circle = lines.Line2D([], [], color="green", marker='o', markersize=15, markerfacecolor="green") + plt.legend((white_circle, orange_circle, red_circle, gray_circle, green_circle), + ('Un-explored', 'Frontier', 'Currently Exploring', 'Explored', 'Final Solution'), + numpoints=1, prop={'size': 16}, loc=(.8, .75)) + + # show the plot. No need to use in notebooks. nx.draw will show the graph itself. + plt.show() + + +# helper functions for visualisations + +def final_path_colors(initial_node_colors, problem, solution): + """Return a node_colors dict of the final path provided the problem and solution.""" + + # get initial node colors + final_colors = dict(initial_node_colors) + # color all the nodes in solution and starting node to green + final_colors[problem.initial] = "green" + for node in solution: + final_colors[node] = "green" + return final_colors + + +def display_visual(graph_data, user_input, algorithm=None, problem=None): + initial_node_colors = graph_data['node_colors'] + if user_input is False: + def slider_callback(iteration): + # don't show graph for the first time running the cell calling this function + try: + show_map(graph_data, node_colors=all_node_colors[iteration]) + except: + pass + + def visualize_callback(visualize): + if visualize is True: + button.value = False + + global all_node_colors + + iterations, all_node_colors, node = algorithm(problem) + solution = node.solution() + all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution)) + + slider.max = len(all_node_colors) - 1 + + for i in range(slider.max + 1): + slider.value = i + # time.sleep(.5) + + slider = widgets.IntSlider(min=0, max=1, step=1, value=0) + slider_visual = widgets.interactive(slider_callback, iteration=slider) + display(slider_visual) + + button = widgets.ToggleButton(value=False) + button_visual = widgets.interactive(visualize_callback, visualize=button) + display(button_visual) + + if user_input is True: + node_colors = dict(initial_node_colors) + if isinstance(algorithm, dict): + assert set(algorithm.keys()).issubset({"Breadth First Tree Search", + "Depth First Tree Search", + "Breadth First Search", + "Depth First Graph Search", + "Best First Graph Search", + "Uniform Cost Search", + "Depth Limited Search", + "Iterative Deepening Search", + "Greedy Best First Search", + "A-star Search", + "Recursive Best First Search"}) + + algo_dropdown = widgets.Dropdown(description="Search algorithm: ", + options=sorted(list(algorithm.keys())), + value="Breadth First Tree Search") + display(algo_dropdown) + elif algorithm is None: + print("No algorithm to run.") + return 0 + + def slider_callback(iteration): + # don't show graph for the first time running the cell calling this function + try: + show_map(graph_data, node_colors=all_node_colors[iteration]) + except: + pass + + def visualize_callback(visualize): + if visualize is True: + button.value = False + + problem = GraphProblem(start_dropdown.value, end_dropdown.value, romania_map) + global all_node_colors + + user_algorithm = algorithm[algo_dropdown.value] + + iterations, all_node_colors, node = user_algorithm(problem) + solution = node.solution() + all_node_colors.append(final_path_colors(all_node_colors[0], problem, solution)) + + slider.max = len(all_node_colors) - 1 + + for i in range(slider.max + 1): + slider.value = i + # time.sleep(.5) + + start_dropdown = widgets.Dropdown(description="Start city: ", + options=sorted(list(node_colors.keys())), value="Arad") + display(start_dropdown) + + end_dropdown = widgets.Dropdown(description="Goal city: ", + options=sorted(list(node_colors.keys())), value="Fagaras") + display(end_dropdown) + + button = widgets.ToggleButton(value=False) + button_visual = widgets.interactive(visualize_callback, visualize=button) + display(button_visual) + + slider = widgets.IntSlider(min=0, max=1, step=1, value=0) + slider_visual = widgets.interactive(slider_callback, iteration=slider) + display(slider_visual) + + +# Function to plot NQueensCSP in csp.py and NQueensProblem in search.py +def plot_NQueens(solution): + n = len(solution) + board = np.array([2 * int((i + j) % 2) for j in range(n) for i in range(n)]).reshape((n, n)) + im = Image.open('images/queen_s.png') + height = im.size[1] + im = np.array(im).astype(np.float) / 255 + fig = plt.figure(figsize=(7, 7)) + ax = fig.add_subplot(111) + ax.set_title('{} Queens'.format(n)) + plt.imshow(board, cmap='binary', interpolation='nearest') + # NQueensCSP gives a solution as a dictionary + if isinstance(solution, dict): + for (k, v) in solution.items(): + newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1) + newax.imshow(im) + newax.axis('off') + # NQueensProblem gives a solution as a list + elif isinstance(solution, list): + for (k, v) in enumerate(solution): + newax = fig.add_axes([0.064 + (k * 0.112), 0.062 + ((7 - v) * 0.112), 0.1, 0.1], zorder=1) + newax.imshow(im) + newax.axis('off') + fig.tight_layout() + plt.show() + + +# Function to plot a heatmap, given a grid +def heatmap(grid, cmap='binary', interpolation='nearest'): + fig = plt.figure(figsize=(7, 7)) + ax = fig.add_subplot(111) + ax.set_title('Heatmap') + plt.imshow(grid, cmap=cmap, interpolation=interpolation) + fig.tight_layout() + plt.show() + + +# Generates a gaussian kernel +def gaussian_kernel(l=5, sig=1.0): + ax = np.arange(-l // 2 + 1., l // 2 + 1.) + xx, yy = np.meshgrid(ax, ax) + kernel = np.exp(-(xx ** 2 + yy ** 2) / (2. * sig ** 2)) + return kernel + + +# Plots utility function for a POMDP +def plot_pomdp_utility(utility): + save = utility['0'][0] + delete = utility['1'][0] + ask_save = utility['2'][0] + ask_delete = utility['2'][-1] + left = (save[0] - ask_save[0]) / (save[0] - ask_save[0] + ask_save[1] - save[1]) + right = (delete[0] - ask_delete[0]) / (delete[0] - ask_delete[0] + ask_delete[1] - delete[1]) + + colors = ['g', 'b', 'k'] + for action in utility: + for value in utility[action]: + plt.plot(value, color=colors[int(action)]) + plt.vlines([left, right], -20, 10, linestyles='dashed', colors='c') + plt.ylim(-20, 13) + plt.xlim(0, 1) + plt.text(left / 2 - 0.05, 10, 'Save') + plt.text((right + left) / 2 - 0.02, 10, 'Ask') + plt.text((right + 1) / 2 - 0.07, 10, 'Delete') + plt.show() diff --git a/notebooks/chapter19/Learners.ipynb b/notebooks/chapter19/Learners.ipynb new file mode 100644 index 000000000..c6f3d1e4f --- /dev/null +++ b/notebooks/chapter19/Learners.ipynb @@ -0,0 +1,508 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Learners\n", + "\n", + "In this section, we will introduce several pre-defined learners to learning the datasets by updating their weights to minimize the loss function. when using a learner to deal with machine learning problems, there are several standard steps:\n", + "\n", + "- **Learner initialization**: Before training the network, it usually should be initialized first. There are several choices when initializing the weights: random initialization, initializing weights are zeros or use Gaussian distribution to init the weights.\n", + "\n", + "- **Optimizer specification**: Which means specifying the updating rules of learnable parameters of the network. Usually, we can choose Adam optimizer as default.\n", + "\n", + "- **Applying back-propagation**: In neural networks, we commonly use back-propagation to pass and calculate gradient information of each layer. Back-propagation needs to be integrated with the chosen optimizer in order to update the weights of NN properly in each epoch.\n", + "\n", + "- **Iterations**: Iterating over the forward and back-propagation process of given epochs. Sometimes the iterating process will have to be stopped by triggering early access in case of overfitting.\n", + "\n", + "We will introduce several learners with different structures. We will import all necessary packages before that:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from deep_learning4e import *\n", + "from notebook4e import *\n", + "from learning4e import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Perceptron Learner\n", + "\n", + "### Overview\n", + "\n", + "The Perceptron is a linear classifier. It works the same way as a neural network with no hidden layers (just input and output). First, it trains its weights given a dataset and then it can classify a new item by running it through the network.\n", + "\n", + "Its input layer consists of the item features, while the output layer consists of nodes (also called neurons). Each node in the output layer has *n* synapses (for every item feature), each with its own weight. Then, the nodes find the dot product of the item features and the synapse weights. These values then pass through an activation function (usually a sigmoid). Finally, we pick the largest of the values and we return its index.\n", + "\n", + "Note that in classification problems each node represents a class. The final classification is the class/node with the max output value.\n", + "\n", + "Below you can see a single node/neuron in the outer layer. With *f* we denote the item features, with *w* the synapse weights, then inside the node we have the dot product and the activation function, *g*." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![perceptron](images/perceptron.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "Perceptron learner is actually a neural network learner with only one hidden layer which is pre-defined in the algorithm of `perceptron_learner`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "raw_net = [InputLayer(input_size), DenseLayer(input_size, output_size)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Where `input_size` and `output_size` are calculated from dataset examples. In the perceptron learner, the gradient descent optimizer is used to update the weights of the network. we return a function `predict` which we will use in the future to classify a new item. The function computes the (algebraic) dot product of the item with the calculated weights for each node in the outer layer. Then it picks the greatest value and classifies the item in the corresponding class." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Let's try the perceptron learner with the `iris` dataset examples, first let's regulate the dataset classes:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "iris = DataSet(name=\"iris\")\n", + "classes = [\"setosa\", \"versicolor\", \"virginica\"]\n", + "iris.classes_to_numbers(classes)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch:50, total_loss:14.089098023560856\n", + "epoch:100, total_loss:12.439240091345326\n", + "epoch:150, total_loss:11.848151059704785\n", + "epoch:200, total_loss:11.283665595671044\n", + "epoch:250, total_loss:11.153290841913241\n", + "epoch:300, total_loss:11.00747536734494\n", + "epoch:350, total_loss:10.871093050365419\n", + "epoch:400, total_loss:10.838400319844233\n", + "epoch:450, total_loss:10.687417928867456\n", + "epoch:500, total_loss:10.650371951865573\n" + ] + } + ], + "source": [ + "pl = perceptron_learner(iris, epochs=500, learning_rate=0.01, verbose=50)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see from the printed lines that the final total loss is converged to around 10.50. If we check the error ratio of perceptron learner on the dataset after training, we will see it is much higher than randomly guess:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.046666666666666634\n" + ] + } + ], + "source": [ + "print(err_ratio(pl, iris))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we test the trained learner with some test cases:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n" + ] + } + ], + "source": [ + "tests = [([5.0, 3.1, 0.9, 0.1], 0),\n", + " ([5.1, 3.5, 1.0, 0.0], 0),\n", + " ([4.9, 3.3, 1.1, 0.1], 0),\n", + " ([6.0, 3.0, 4.0, 1.1], 1),\n", + " ([6.1, 2.2, 3.5, 1.0], 1),\n", + " ([5.9, 2.5, 3.3, 1.1], 1),\n", + " ([7.5, 4.1, 6.2, 2.3], 2),\n", + " ([7.3, 4.0, 6.1, 2.4], 2),\n", + " ([7.0, 3.3, 6.1, 2.5], 2)]\n", + "print(grade_learner(pl, tests))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It seems the learner is correct on all the test examples.\n", + "\n", + "Now let's try perceptron learner on a more complicated dataset: the MNIST dataset, to see what the result will be. First, we import the dataset to make the examples a `Dataset` object:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "length of training dataset: 60000\n", + "length of test dataset: 10000\n" + ] + } + ], + "source": [ + "train_img, train_lbl, test_img, test_lbl = load_MNIST(path=\"../../aima-data/MNIST/Digits\")\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "train_examples = [np.append(train_img[i], train_lbl[i]) for i in range(len(train_img))]\n", + "test_examples = [np.append(test_img[i], test_lbl[i]) for i in range(len(test_img))]\n", + "print(\"length of training dataset:\", len(train_examples))\n", + "print(\"length of test dataset:\", len(test_examples))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's train the perceptron learner on the first 1000 examples of the dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch:1, total_loss:423.8627535296463\n", + "epoch:2, total_loss:341.31697581698995\n", + "epoch:3, total_loss:328.98647291325443\n", + "epoch:4, total_loss:327.8999700915627\n", + "epoch:5, total_loss:310.081065570072\n", + "epoch:6, total_loss:268.5474616202945\n", + "epoch:7, total_loss:259.0999998773958\n", + "epoch:8, total_loss:259.09999987481393\n", + "epoch:9, total_loss:259.09999987211944\n", + "epoch:10, total_loss:259.0999998693056\n" + ] + } + ], + "source": [ + "mnist = DataSet(examples=train_examples[:1000])\n", + "pl = perceptron_learner(mnist, epochs=10, verbose=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.893\n" + ] + } + ], + "source": [ + "print(err_ratio(pl, mnist))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It looks like we have a near 90% error ratio on training data after the network is trained on it. Then we can investigate the model's performance on the test dataset which it never has seen before:" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.92\n" + ] + } + ], + "source": [ + "test_mnist = DataSet(examples=test_examples[:100])\n", + "print(err_ratio(pl, test_mnist))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It seems a single layer perceptron learner cannot simulate the structure of the MNIST dataset. To improve accuracy, we may not only increase training epochs but also consider changing to a more complicated network structure." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Neural Network Learner\n", + "\n", + "Although there are many different types of neural networks, the dense neural network we implemented can be treated as a stacked perceptron learner. Adding more layers to the perceptron network could add to the non-linearity to the network thus model will be more flexible when fitting complex data-target relations. Whereas it also adds to the risk of overfitting as the side effect of flexibility.\n", + "\n", + "By default we use dense networks with two hidden layers, which has the architecture as the following:\n", + "\n", + "\n", + "\n", + "In our code, we implemented it as:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# initialize the network\n", + "raw_net = [InputLayer(input_size)]\n", + "# add hidden layers\n", + "hidden_input_size = input_size\n", + "for h_size in hidden_layer_sizes:\n", + " raw_net.append(DenseLayer(hidden_input_size, h_size))\n", + " hidden_input_size = h_size\n", + "raw_net.append(DenseLayer(hidden_input_size, output_size))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Where hidden_layer_sizes are the sizes of each hidden layer in a list which can be specified by user. Neural network learner uses gradient descent as default optimizer but user can specify any optimizer when calling `neural_net_learner`. The other special attribute that can be changed in `neural_net_learner` is `batch_size` which controls the number of examples used in each round of update. `neural_net_learner` also returns a `predict` function which calculates prediction by multiplying weight to inputs and applying activation functions.\n", + "\n", + "### Example\n", + "\n", + "Let's also try `neural_net_learner` on the `iris` dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch:10, total_loss:15.931817841643683\n", + "epoch:20, total_loss:8.248422285412149\n", + "epoch:30, total_loss:6.102968668275\n", + "epoch:40, total_loss:5.463915043272969\n", + "epoch:50, total_loss:5.298986288420822\n", + "epoch:60, total_loss:4.032928400456889\n", + "epoch:70, total_loss:3.2628899927346855\n", + "epoch:80, total_loss:6.01336701367312\n", + "epoch:90, total_loss:5.412020420311795\n", + "epoch:100, total_loss:3.1044027319850773\n" + ] + } + ], + "source": [ + "nn = neural_net_learner(iris, epochs=100, learning_rate=0.15, optimizer=gradient_descent, verbose=10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly we check the model's accuracy on both training and test dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "error ration on training set: 0.033333333333333326\n" + ] + } + ], + "source": [ + "print(\"error ration on training set:\",err_ratio(nn, iris))" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "accuracy on test set: 1\n" + ] + } + ], + "source": [ + "tests = [([5.0, 3.1, 0.9, 0.1], 0),\n", + " ([5.1, 3.5, 1.0, 0.0], 0),\n", + " ([4.9, 3.3, 1.1, 0.1], 0),\n", + " ([6.0, 3.0, 4.0, 1.1], 1),\n", + " ([6.1, 2.2, 3.5, 1.0], 1),\n", + " ([5.9, 2.5, 3.3, 1.1], 1),\n", + " ([7.5, 4.1, 6.2, 2.3], 2),\n", + " ([7.3, 4.0, 6.1, 2.4], 2),\n", + " ([7.0, 3.3, 6.1, 2.5], 2)]\n", + "print(\"accuracy on test set:\",grade_learner(nn, tests))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that the error ratio on the training set is smaller than the perceptron learner. As the error ratio is relatively small, let's try the model on the MNIST dataset to see whether there will be a larger difference. " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch:10, total_loss:89.0002153455983\n", + "epoch:20, total_loss:87.29675663038348\n", + "epoch:30, total_loss:86.29591779319225\n", + "epoch:40, total_loss:83.78091780128402\n", + "epoch:50, total_loss:82.17091581738829\n", + "epoch:60, total_loss:83.8434277386084\n", + "epoch:70, total_loss:83.55209905561495\n", + "epoch:80, total_loss:83.106898191118\n", + "epoch:90, total_loss:83.37041170165992\n", + "epoch:100, total_loss:82.57013813500876\n" + ] + } + ], + "source": [ + "nn = neural_net_learner(mnist, epochs=100, verbose=10)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.784\n" + ] + } + ], + "source": [ + "print(err_ratio(nn, mnist))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After the model converging, the model's error ratio on the training set is still high. We will introduce the convolutional network in the following chapters to see how it helps improve accuracy on learning this dataset." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter19/Loss Functions and Layers.ipynb b/notebooks/chapter19/Loss Functions and Layers.ipynb new file mode 100644 index 000000000..25676e899 --- /dev/null +++ b/notebooks/chapter19/Loss Functions and Layers.ipynb @@ -0,0 +1,398 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Loss Function\n", + "\n", + "Loss functions evaluate how well specific algorithm models the given data. Commonly loss functions are used to compare the target data and model's prediction. If predictions deviate too much from actual targets, loss function would output a large value. Usually, loss functions can help other optimization functions to improve the accuracy of the model.\n", + "\n", + "However, there’s no one-size-fits-all loss function to algorithms in machine learning. For each algorithm and machine learning projects, specifying certain loss functions could assist the user in getting better model performance. Here we will demonstrate two loss functions: `mse_loss` and `cross_entropy_loss`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Min Square Error\n", + "\n", + "Min square error(MSE) is the most commonly used loss function in machine learning. The intuition of MSE is straight forward: the distance between two points represents the difference between them. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$MSE = -\\sum_i{(y_i-t_i)^2/n}$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Where $y_i$ is the prediction of the ith example and $t_i$ is the target of the ith example. And n is the total number of examples.\n", + "\n", + "Below is a plot of an MSE function where the true target value is 100, and the predicted values range between -10,000 to 10,000. The MSE loss (Y-axis) reaches its minimum value at prediction (X-axis) = 100." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cross-Entropy\n", + "\n", + "For most deep learning applications, we can get away with just one loss function: cross-entropy loss function. We can think of most deep learning algorithms as learning probability distributions and what we are learning is a distribution of predictions $P(y|x)$ given a series of inputs. \n", + "\n", + "To associate input examples x with output examples y, the parameters that maximize the likelihood of the training set should be:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\\theta^* = argmax_\\theta \\prod_{i=0}^n p(y^{(i)}/x^{(i)})$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Maxmizing the above formula equals to minimizing the negative log form of it:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\\theta^* = argmin_\\theta -\\sum_{i=0}^n logp(y^{(i)}/x^{(i)})$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It can be proven that the above formula equals to minimizing MSE loss.\n", + "\n", + "The majority of deep learning algorithms use cross-entropy in some way. Classifiers that use deep learning calculate the cross-entropy between categorical distributions over the output class. For a given class, its contribution to the loss is dependent on its probability in the following trend:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Examples\n", + "\n", + "First let's import necessary packages." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from deep_learning4e import *\n", + "from notebook4e import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Neural Network Layers\n", + "\n", + "Neural networks may be conveniently described using data structures of computational graphs. A computational graph is a directed graph describing how many variables should be computed, with each variable by computed by applying a specific operation to a set of other variables. \n", + "\n", + "In our code, we provide class `NNUnit` as the basic structure of a neural network. The structure of `NNUnit` is simple, it only stores the following information:\n", + "\n", + "- **val**: the value of the current node.\n", + "- **parent**: parents of the current node.\n", + "- **weights**: weights between parent nodes and current node. It should be in the same size as parents.\n", + "\n", + "There is another class `Layer` inheriting from `NNUnit`. A `Layer` object holds a list of nodes that represents all the nodes in a layer. It also has a method `forward` to pass a value through the current layer. Here we will demonstrate several pre-defined types of layers in a Neural Network." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Output Layers\n", + "\n", + "Neural networks need specialized output layers for each type of data we might ask them to produce. For many problems, we need to model discrete variables that have k distinct values instead of just binary variables. For example, models of natural language may predict a single word from among of vocabulary of tens of thousands or even more choices. To represent these distributions, we use a softmax layer:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$P(y=i|x)=softmax(h(x)^TW+b)_i$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $W$ is matrix of learned weights of output layer $b$ is a vector of learned biases, and the softmax function is:\n", + "\n", + "$$softmax(z_i)=exp(z_i)/\\sum_i exp(z_i)$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is simple to create a output layer and feed an example into it:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.03205860328008499, 0.08714431874203257, 0.23688281808991013, 0.6439142598879722]\n" + ] + } + ], + "source": [ + "layer = OutputLayer(size=4)\n", + "example = [1,2,3,4]\n", + "print(layer.forward(example))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The output can be treated like normalized probability when the input of output layer is calculated by probability." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Input Layers\n", + "\n", + "Input layers can be treated like a mapping layer that maps each element of the input vector to each input layer node. The input layer acts as a storage of input vector information which can be used when doing forward propagation.\n", + "\n", + "In our realization of input layers, the size of the input vector and input layer should match." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[1, 2, 3]\n" + ] + } + ], + "source": [ + "layer = InputLayer(size=3)\n", + "example = [1,2,3]\n", + "print(layer.forward(example))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Hidden Layers\n", + "\n", + "While processing an input vector x of the neural network, it performs several intermediate computations before producing the output y. We can think of these intermediate computations as the state of memory during the execution of a multi-step program. We call the intermediate computations hidden because the data does not specify the values of these variables.\n", + "\n", + "Most neural network hidden layers are based on a linear transformation followed by the application of an elementwise nonlinear function called the activation function g:\n", + "\n", + "$$h=g(W+b)$$\n", + "\n", + "where W is a learned matrix of weights and b is a learned set of bias parameters.\n", + "\n", + "Here we pre-defined several activation functions in `utils.py`: `sigmoid`, `relu`, `elu`, `tanh` and `leaky_relu`. They are all inherited from the `Activation` class. You can get the value of the function or its derivative at a certain point of x:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Sigmoid at 0: 0.5\n", + "Deriavation of sigmoid at 0: 0\n" + ] + } + ], + "source": [ + "s = sigmoid()\n", + "print(\"Sigmoid at 0:\", s.f(0))\n", + "print(\"Deriavation of sigmoid at 0:\", s.derivative(0))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create a hidden layer object, there are several attributes need to be specified:\n", + "\n", + "- **in_size**: the input vector size of each hidden layer node.\n", + "- **out_size**: the size of the output vector of the hidden layer. Thus each node will hide the weight of the size of (in_size). The weights will be initialized randomly.\n", + "- **activation**: the activation function used for this layer.\n", + "\n", + "Now let's demonstrate how a dense hidden layer works briefly:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.21990266877137224, 0.2038864498984756, 0.5543443697256466]\n" + ] + } + ], + "source": [ + "layer = DenseLayer(in_size=4, out_size=3, activation=sigmoid())\n", + "example = [1,2,3,4]\n", + "print(layer.forward(example))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This layer mapped input of size 4 to output of size 3. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Convolutional Layers\n", + "\n", + "The convolutional layer is similar to the hidden layer except they use a different forward strategy. The convolutional layer takes an input of multiple channels and does convolution on each channel with a pre-defined kernel function. Thus the output of the convolutional layer will still be with the same number of channels. If we image each input as an image, then channels represent its color model such as RGB. The output will still have the same color model as the input.\n", + "\n", + "Now let's try the one-dimensional convolution layer:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[array([3.9894228, 3.9894228, 3.9894228]), array([3.9894228, 3.9894228, 3.9894228]), array([3.9894228, 3.9894228, 3.9894228])]\n" + ] + } + ], + "source": [ + "layer = ConvLayer1D(size=3, kernel_size=3)\n", + "example = [[1]*3 for _ in range(3)]\n", + "print(layer.forward(example))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Which can be deemed as a one-dimensional image with three channels." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Pooling Layers\n", + "\n", + "Pooling layers can be treated as a special kind of convolutional layer that uses a special kind of kernel to extract a certain value in the kernel region. Here we use max-pooling to report the maximum value in each group." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[3, 4], [4, 4], [4, 4]]\n" + ] + } + ], + "source": [ + "layer = MaxPoolingLayer1D(size=3, kernel_size=3)\n", + "example = [[1,2,3,4], [2,3,4,1],[3,4,1,2]]\n", + "print(layer.forward(example))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that each time kernel picks up the maximum value in its region." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter19/Optimizer and Backpropagation.ipynb b/notebooks/chapter19/Optimizer and Backpropagation.ipynb new file mode 100644 index 000000000..5194adc7a --- /dev/null +++ b/notebooks/chapter19/Optimizer and Backpropagation.ipynb @@ -0,0 +1,311 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Optimization Algorithms\n", + "\n", + "Training a neural network consists of modifying the network’s parameters to minimize the cost function on the training set. In principle, any kind of optimization algorithm could be used. In practice, modern neural networks are almost always trained with some variant of stochastic gradient descent(SGD). Here we will provide two optimization algorithms: SGD and Adam optimizer.\n", + "\n", + "## Stochastic Gradient Descent\n", + "\n", + "The goal of an optimization algorithm is to find the value of the parameter to make loss function very low. For some types of models, an optimization algorithm might find the global minimum value of loss function, but for neural network, the most efficient way to converge loss function to a local minimum is to minimize loss function according to each example.\n", + "\n", + "Gradient descent uses the following update rule to minimize loss function:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\\theta^{(t+1)} = \\theta^{(t)}-\\alpha\\nabla_\\theta L(\\theta^{(t)})$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where t is the time step of the algorithm and $\\alpha$ is the learning rate. But this rule could be very costly when $L(\\theta)$ is defined as a sum across the entire training set. Using SGD can accelerate the learning process as we can use only a batch of examples to update the parameters. \n", + "\n", + "We implemented the gradient descent algorithm, which can be viewed with the following code:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from deep_learning4e import *\n", + "from notebook4e import *" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

    \n", + "\n", + "
    def gradient_descent(dataset, net, loss, epochs=1000, l_rate=0.01,  batch_size=1):\n",
    +       "    """\n",
    +       "    gradient descent algorithm to update the learnable parameters of a network.\n",
    +       "    :return: the updated network.\n",
    +       "    """\n",
    +       "    # init data\n",
    +       "    examples = dataset.examples\n",
    +       "\n",
    +       "    for e in range(epochs):\n",
    +       "        total_loss = 0\n",
    +       "        random.shuffle(examples)\n",
    +       "        weights = [[node.weights for node in layer.nodes] for layer in net]\n",
    +       "\n",
    +       "        for batch in get_batch(examples, batch_size):\n",
    +       "\n",
    +       "            inputs, targets = init_examples(batch, dataset.inputs, dataset.target, len(net[-1].nodes))\n",
    +       "            # compute gradients of weights\n",
    +       "            gs, batch_loss = BackPropagation(inputs, targets, weights, net, loss)\n",
    +       "            # update weights with gradient descent\n",
    +       "            weights = vector_add(weights, scalar_vector_product(-l_rate, gs))\n",
    +       "            total_loss += batch_loss\n",
    +       "            # update the weights of network each batch\n",
    +       "            for i in range(len(net)):\n",
    +       "                if weights[i]:\n",
    +       "                    for j in range(len(weights[i])):\n",
    +       "                        net[i].nodes[j].weights = weights[i][j]\n",
    +       "\n",
    +       "        if (e+1) % 10 == 0:\n",
    +       "            print("epoch:{}, total_loss:{}".format(e+1,total_loss))\n",
    +       "    return net\n",
    +       "
    \n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(gradient_descent)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There several key elements need to specify when using a `gradient_descent` optimizer:\n", + "\n", + "- **dataset**: A dataset object we used in the previous chapter, such as `iris` and `orings`.\n", + "- **net**: A neural network object which we will cover in the next chapter.\n", + "- **loss**: The loss function used in representing accuracy.\n", + "- **epochs**: How many rounds the training set is used.\n", + "- **l_rate**: learning rate.\n", + "- **batch_size**: The number of examples is used in each update. When very small batch size is used, gradient descent and be treated as SGD." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Adam Optimizer\n", + "\n", + "To mitigate some of the problems caused by the fact that the gradient ignores the second derivatives, some optimization algorithms incorporate the idea of momentum which keeps a running average of the gradients of past mini-batches. Thus Adam optimizer maintains a table saving the previous gradient result.\n", + "\n", + "To view the pseudocode and the implementation, you can use the following codes:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pseudocode(adam_optimizer)\n", + "psource(adam_optimizer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are several attributes to specify when using Adam optimizer that is different from gradient descent: rho and delta. These parameters determine the percentage of the last iteration is memorized. For more details of how this algorithm work, please refer to the article [here](https://arxiv.org/abs/1412.6980).\n", + "\n", + "In the Stanford course on deep learning for computer vision, the Adam algorithm is suggested as the default optimization method for deep learning applications: \n", + ">In practice Adam is currently recommended as the default algorithm to use, and often works slightly better than RMSProp. However, it is often also worth trying SGD+Nesterov Momentum as an alternative." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Backpropagation\n", + "\n", + "The above algorithms are optimization algorithms: they update parameters like $\\theta$ to get smaller loss values. And back-propagation is the method to calculate the gradient for each layer. For complicated models like deep neural networks, the gradients can not be calculated directly as there are enormous array-valued variables.\n", + "\n", + "Fortunately, back-propagation can calculate the gradients briefly which we can interpret as calculating gradients from the last layer to the first which is the inverse process to the forwarding procedure. The derivation of the loss function is passed to previous layers to make them changing toward the direction of minimizing the loss function." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Applying optimizers and back-propagation algorithm together, we can update the weights of a neural network to minimize the loss function with alternatively doing forward and back-propagation process. Here is a figure form [here](https://medium.com/datathings/neural-networks-and-backpropagation-explained-in-a-simple-way-f540a3611f5e) describing how a neural network updates its weights:\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In our implementation, all the steps are integrated into the optimizer objects. The forward-backward process of passing information through the whole neural network is put into the method `BackPropagation`. You can view the code with:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(BackPropagation)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The demonstration of optimizers and back-propagation algorithm will be made together with neural network learners." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter19/RNN.ipynb b/notebooks/chapter19/RNN.ipynb new file mode 100644 index 000000000..b6971b36a --- /dev/null +++ b/notebooks/chapter19/RNN.ipynb @@ -0,0 +1,487 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# RNN\n", + "\n", + "## Overview\n", + "\n", + "When human is thinking, they are thinking based on the understanding of previous time steps but not from scratch. Traditional neural networks can’t do this, and it seems like a major shortcoming. For example, imagine you want to do sentimental analysis of some texts. It will be unclear if the traditional network cannot recognize the short phrase and sentences.\n", + "\n", + "Recurrent neural networks address this issue. They are networks with loops in them, allowing information to persist.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A recurrent neural network can be thought of as multiple copies of the same network, each passing a message to a successor. Consider what happens if we unroll the above loop:\n", + " \n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As demonstrated in the book, recurrent neural networks may be connected in many different ways: sequences in the input, the output, or in the most general case both.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implementation\n", + "\n", + "In our case, we implemented rnn with modules offered by the package of `keras`. To use `keras` and our module, you must have both `tensorflow` and `keras` installed as a prerequisite. `keras` offered very well defined high-level neural networks API which allows for easy and fast prototyping. `keras` supports many different types of networks such as convolutional and recurrent neural networks as well as user-defined networks. About how to get started with `keras`, please read the [tutorial](https://keras.io/).\n", + "\n", + "To view our implementation of a simple rnn, please use the following code:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import warnings\n", + "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n", + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from deep_learning4e import *\n", + "from notebook4e import *" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

    \n", + "\n", + "
    def SimpleRNNLearner(train_data, val_data, epochs=2):\n",
    +       "    """\n",
    +       "    RNN example for text sentimental analysis.\n",
    +       "    :param train_data: a tuple of (training data, targets)\n",
    +       "            Training data: ndarray taking training examples, while each example is coded by embedding\n",
    +       "            Targets: ndarray taking targets of each example. Each target is mapped to an integer.\n",
    +       "    :param val_data: a tuple of (validation data, targets)\n",
    +       "    :param epochs: number of epochs\n",
    +       "    :return: a keras model\n",
    +       "    """\n",
    +       "\n",
    +       "    total_inputs = 5000\n",
    +       "    input_length = 500\n",
    +       "\n",
    +       "    # init data\n",
    +       "    X_train, y_train = train_data\n",
    +       "    X_val, y_val = val_data\n",
    +       "\n",
    +       "    # init a the sequential network (embedding layer, rnn layer, dense layer)\n",
    +       "    model = Sequential()\n",
    +       "    model.add(Embedding(total_inputs, 32, input_length=input_length))\n",
    +       "    model.add(SimpleRNN(units=128))\n",
    +       "    model.add(Dense(1, activation='sigmoid'))\n",
    +       "    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    +       "\n",
    +       "    # train the model\n",
    +       "    model.fit(X_train, y_train, validation_data=(X_val, y_val), epochs=epochs, batch_size=128, verbose=2)\n",
    +       "\n",
    +       "    return model\n",
    +       "
    \n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(SimpleRNNLearner)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`train_data` and `val_data` are needed when creating a simple rnn learner. Both attributes take lists of examples and the targets in a tuple. Please note that we build the network by adding layers to a `Sequential()` model which means data are passed through the network one by one. `SimpleRNN` layer is the key layer of rnn which acts the recursive role. Both `Embedding` and `Dense` layers before and after the rnn layer are used to map inputs and outputs to data in rnn form. And the optimizer used in this case is the Adam optimizer." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "Here is an example of how we train the rnn network made with `keras`. In this case, we used the IMDB dataset which can be viewed [here](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification) in detail. In short, the dataset is consist of movie reviews in text and their labels of sentiment (positive/negative). After loading the dataset we use `keras_dataset_loader` to split it into training, validation and test datasets." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from keras.datasets import imdb\n", + "data = imdb.load_data(num_words=5000)\n", + "train, val, test = keras_dataset_loader(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we build and train the rnn model for 10 epochs:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING: Logging before flag parsing goes to stderr.\n", + "W1018 22:51:23.614058 140557804885824 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", + "Instructions for updating:\n", + "Use tf.where in 2.0, which has the same broadcast rule as np.where\n", + "W1018 22:51:24.267649 140557804885824 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 24990 samples, validate on 25000 samples\n", + "Epoch 1/10\n", + " - 59s - loss: 0.6540 - accuracy: 0.5959 - val_loss: 0.6234 - val_accuracy: 0.6488\n", + "Epoch 2/10\n", + " - 61s - loss: 0.5977 - accuracy: 0.6766 - val_loss: 0.6202 - val_accuracy: 0.6326\n", + "Epoch 3/10\n", + " - 61s - loss: 0.5269 - accuracy: 0.7356 - val_loss: 0.4803 - val_accuracy: 0.7789\n", + "Epoch 4/10\n", + " - 61s - loss: 0.4159 - accuracy: 0.8130 - val_loss: 0.5640 - val_accuracy: 0.7046\n", + "Epoch 5/10\n", + " - 61s - loss: 0.3931 - accuracy: 0.8294 - val_loss: 0.4707 - val_accuracy: 0.8090\n", + "Epoch 6/10\n", + " - 61s - loss: 0.3357 - accuracy: 0.8637 - val_loss: 0.4177 - val_accuracy: 0.8122\n", + "Epoch 7/10\n", + " - 61s - loss: 0.3552 - accuracy: 0.8594 - val_loss: 0.4652 - val_accuracy: 0.7889\n", + "Epoch 8/10\n", + " - 61s - loss: 0.3286 - accuracy: 0.8686 - val_loss: 0.4708 - val_accuracy: 0.7785\n", + "Epoch 9/10\n", + " - 61s - loss: 0.3428 - accuracy: 0.8635 - val_loss: 0.4332 - val_accuracy: 0.8137\n", + "Epoch 10/10\n", + " - 61s - loss: 0.3650 - accuracy: 0.8471 - val_loss: 0.4673 - val_accuracy: 0.7914\n" + ] + } + ], + "source": [ + "model = SimpleRNNLearner(train, val, epochs=10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The accuracy of the training dataset and validation dataset are both over 80% which is very promising. Now let's try on some random examples in the test set:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Autoencoder\n", + "\n", + "Autoencoders are an unsupervised learning technique in which we leverage neural networks for the task of representation learning. It works by compressing the input into a latent-space representation, to do transformations on the data. \n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Autoencoders are learned automatically from data examples. It means that it is easy to train specialized instances of the algorithm that will perform well on a specific type of input and that it does not require any new engineering, only the appropriate training data.\n", + "\n", + "Autoencoders have different architectures for different kinds of data. Here we only provide a simple example of a vanilla encoder, which means they're only one hidden layer in the network:\n", + "\n", + "\n", + "\n", + "You can view the source code by:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "\n", + "\n", + " Codestin Search App\n", + " \n", + " \n", + "\n", + "\n", + "

    \n", + "\n", + "
    def AutoencoderLearner(inputs, encoding_size, epochs=200):\n",
    +       "    """\n",
    +       "    Simple example of linear auto encoder learning producing the input itself.\n",
    +       "    :param inputs: a batch of input data in np.ndarray type\n",
    +       "    :param encoding_size: int, the size of encoding layer\n",
    +       "    :param epochs: number of epochs\n",
    +       "    :return: a keras model\n",
    +       "    """\n",
    +       "\n",
    +       "    # init data\n",
    +       "    input_size = len(inputs[0])\n",
    +       "\n",
    +       "    # init model\n",
    +       "    model = Sequential()\n",
    +       "    model.add(Dense(encoding_size, input_dim=input_size, activation='relu', kernel_initializer='random_uniform',\n",
    +       "                    bias_initializer='ones'))\n",
    +       "    model.add(Dense(input_size, activation='relu', kernel_initializer='random_uniform', bias_initializer='ones'))\n",
    +       "\n",
    +       "    # update model with sgd\n",
    +       "    sgd = optimizers.SGD(lr=0.01)\n",
    +       "    model.compile(loss='mean_squared_error', optimizer=sgd, metrics=['accuracy'])\n",
    +       "\n",
    +       "    # train the model\n",
    +       "    model.fit(inputs, inputs, epochs=epochs, batch_size=10, verbose=2)\n",
    +       "\n",
    +       "    return model\n",
    +       "
    \n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "psource(AutoencoderLearner)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It shows we added two dense layers to the network structures." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter19/images/autoencoder.png b/notebooks/chapter19/images/autoencoder.png new file mode 100644 index 000000000..cd216e9f7 Binary files /dev/null and b/notebooks/chapter19/images/autoencoder.png differ diff --git a/notebooks/chapter19/images/backprop.png b/notebooks/chapter19/images/backprop.png new file mode 100644 index 000000000..8d53530e6 Binary files /dev/null and b/notebooks/chapter19/images/backprop.png differ diff --git a/notebooks/chapter19/images/corss_entropy_plot.png b/notebooks/chapter19/images/corss_entropy_plot.png new file mode 100644 index 000000000..8212405e7 Binary files /dev/null and b/notebooks/chapter19/images/corss_entropy_plot.png differ diff --git a/notebooks/chapter19/images/mse_plot.png b/notebooks/chapter19/images/mse_plot.png new file mode 100644 index 000000000..fd58f9db9 Binary files /dev/null and b/notebooks/chapter19/images/mse_plot.png differ diff --git a/notebooks/chapter19/images/nn.png b/notebooks/chapter19/images/nn.png new file mode 100644 index 000000000..673b9338b Binary files /dev/null and b/notebooks/chapter19/images/nn.png differ diff --git a/notebooks/chapter19/images/nn_steps.png b/notebooks/chapter19/images/nn_steps.png new file mode 100644 index 000000000..4a596133b Binary files /dev/null and b/notebooks/chapter19/images/nn_steps.png differ diff --git a/notebooks/chapter19/images/perceptron.png b/notebooks/chapter19/images/perceptron.png new file mode 100644 index 000000000..68d2a258a Binary files /dev/null and b/notebooks/chapter19/images/perceptron.png differ diff --git a/notebooks/chapter19/images/rnn_connections.png b/notebooks/chapter19/images/rnn_connections.png new file mode 100644 index 000000000..c72d459b8 Binary files /dev/null and b/notebooks/chapter19/images/rnn_connections.png differ diff --git a/notebooks/chapter19/images/rnn_unit.png b/notebooks/chapter19/images/rnn_unit.png new file mode 100644 index 000000000..e4ebabf2b Binary files /dev/null and b/notebooks/chapter19/images/rnn_unit.png differ diff --git a/notebooks/chapter19/images/rnn_units.png b/notebooks/chapter19/images/rnn_units.png new file mode 100644 index 000000000..5724f5d46 Binary files /dev/null and b/notebooks/chapter19/images/rnn_units.png differ diff --git a/notebooks/chapter19/images/vanilla.png b/notebooks/chapter19/images/vanilla.png new file mode 100644 index 000000000..db7a45f9a Binary files /dev/null and b/notebooks/chapter19/images/vanilla.png differ diff --git a/notebooks/chapter21/Active Reinforcement Learning.ipynb b/notebooks/chapter21/Active Reinforcement Learning.ipynb new file mode 100644 index 000000000..1ce3c79e0 --- /dev/null +++ b/notebooks/chapter21/Active Reinforcement Learning.ipynb @@ -0,0 +1,212 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# ACTIVE REINFORCEMENT LEARNING\n", + "\n", + "This notebook mainly focuses on active reinforce learning algorithms. For a general introduction to reinforcement learning and passive algorithms, please refer to the notebook of **[Passive Reinforcement Learning](./Passive%20Reinforcement%20Learning.ipynb)**.\n", + "\n", + "Unlike Passive Reinforcement Learning in Active Reinforcement Learning, we are not bound by a policy pi and we need to select our actions. In other words, the agent needs to learn an optimal policy. The fundamental tradeoff the agent needs to face is that of exploration vs. exploitation. \n", + "\n", + "## QLearning Agent\n", + "\n", + "The QLearningAgent class in the rl module implements the Agent Program described in **Fig 21.8** of the AIMA Book. In Q-Learning the agent learns an action-value function Q which gives the utility of taking a given action in a particular state. Q-Learning does not require a transition model and hence is a model-free method. Let us look into the source before we see some usage examples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource QLearningAgent" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Agent Program can be obtained by creating the instance of the class by passing the appropriate parameters. Because of the __ call __ method the object that is created behaves like a callable and returns an appropriate action as most Agent Programs do. To instantiate the object we need a `mdp` object similar to the `PassiveTDAgent`.\n", + "\n", + " Let us use the same `GridMDP` object we used above. **Figure 17.1 (sequential_decision_environment)** is similar to **Figure 21.1** but has some discounting parameter as **gamma = 0.9**. The enviroment also implements an exploration function **f** which returns fixed **Rplus** until agent has visited state, action **Ne** number of times. The method **actions_in_state** returns actions possible in given state. It is useful when applying max and argmax operations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let us create our object now. We also use the **same alpha** as given in the footnote of the book on **page 769**: $\\alpha(n)=60/(59+n)$ We use **Rplus = 2** and **Ne = 5** as defined in the book. The pseudocode can be referred from **Fig 21.7** in the book." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from rl4e import *\n", + "from mdp import sequential_decision_environment, value_iteration" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, \n", + " alpha=lambda n: 60./(59+n))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now to try out the q_agent we make use of the **run_single_trial** function in rl.py (which was also used above). Let us use **200** iterations." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(200):\n", + " run_single_trial(q_agent,sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let us see the Q Values. The keys are state-action pairs. Where different actions correspond according to:\n", + "\n", + "north = (0, 1) \n", + "south = (0,-1) \n", + "west = (-1, 0) \n", + "east = (1, 0)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "q_agent.Q" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Utility U of each state is related to Q by the following equation.\n", + "\n", + "$$U (s) = max_a Q(s, a)$$\n", + "\n", + "Let us convert the Q Values above into U estimates.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "U = defaultdict(lambda: -1000.) # Very Large Negative Value for Comparison see below.\n", + "for state_action, value in q_agent.Q.items():\n", + " state, action = state_action\n", + " if U[state] < value:\n", + " U[state] = value" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can output the estimated utility values at each state:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "defaultdict(()>,\n", + " {(0, 0): -0.0036556430391564178,\n", + " (1, 0): -0.04862675963288682,\n", + " (2, 0): 0.03384490363100474,\n", + " (3, 0): -0.16618771401113092,\n", + " (3, 1): -0.6015323978614368,\n", + " (0, 1): 0.09161077177913537,\n", + " (0, 2): 0.1834607974581678,\n", + " (1, 2): 0.26393277962204903,\n", + " (2, 2): 0.32369726495311274,\n", + " (3, 2): 0.38898341569576245,\n", + " (2, 1): -0.044858154562400485})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "U" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let us finally compare these estimates to value_iteration results." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{(0, 1): 0.3984432178350045, (1, 2): 0.649585681261095, (3, 2): 1.0, (0, 0): 0.2962883154554812, (3, 0): 0.12987274656746342, (3, 1): -1.0, (2, 1): 0.48644001739269643, (2, 0): 0.3447542300124158, (2, 2): 0.7953620878466678, (1, 0): 0.25386699846479516, (0, 2): 0.5093943765842497}\n" + ] + } + ], + "source": [ + "print(value_iteration(sequential_decision_environment))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter21/Passive Reinforcement Learning.ipynb b/notebooks/chapter21/Passive Reinforcement Learning.ipynb new file mode 100644 index 000000000..cbb5ae9e3 --- /dev/null +++ b/notebooks/chapter21/Passive Reinforcement Learning.ipynb @@ -0,0 +1,424 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Introduction to Reinforcement Learning\n", + "\n", + "This Jupyter notebook and the others in the same folder act as supporting materials for **Chapter 21 Reinforcement Learning** of the book* Artificial Intelligence: A Modern Approach*. The notebooks make use of the implementations in `rl.py` module. We also make use of the implementation of MDPs in the `mdp.py` module to test our agents. It might be helpful if you have already gone through the Jupyter notebook dealing with the Markov decision process. Let us import everything from the `rl` module. It might be helpful to view the source of some of our implementations." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from rl4e import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before we start playing with the actual implementations let us review a couple of things about RL.\n", + "\n", + "1. Reinforcement Learning is concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. \n", + "\n", + "2. Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Further, there is a focus on on-line performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).\n", + "\n", + "-- Source: [Wikipedia](https://en.wikipedia.org/wiki/Reinforcement_learning)\n", + "\n", + "In summary, we have a sequence of state action transitions with rewards associated with some states. Our goal is to find the optimal policy $\\pi$ which tells us what action to take in each state." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Passive Reinforcement Learning\n", + "\n", + "In passive Reinforcement Learning the agent follows a fixed policy $\\pi$. Passive learning attempts to evaluate the given policy $pi$ - without any knowledge of the Reward function $R(s)$ and the Transition model $P(s'\\ |\\ s, a)$.\n", + "\n", + "This is usually done by some method of **utility estimation**. The agent attempts to directly learn the utility of each state that would result from following the policy. Note that at each step, it has to *perceive* the reward and the state - it has no global knowledge of these. Thus, if a certain the entire set of actions offers a very low probability of attaining some state $s_+$ - the agent may never perceive the reward $R(s_+)$.\n", + "\n", + "Consider a situation where an agent is given the policy to follow. Thus, at any point, it knows only its current state and current reward, and the action it must take next. This action may lead it to more than one state, with different probabilities.\n", + "\n", + "For a series of actions given by $\\pi$, the estimated utility $U$:\n", + "$$U^{\\pi}(s) = E(\\sum_{t=0}^\\inf \\gamma^t R^t(s'))$$\n", + "Or the expected value of summed discounted rewards until termination.\n", + "\n", + "Based on this concept, we discuss three methods of estimating utility: direct utility estimation, adaptive dynamic programming, and temporal-difference learning.\n", + "\n", + "### Implementation\n", + "\n", + "Passive agents are implemented in `rl4e.py` as various `Agent-Class`es.\n", + "\n", + "To demonstrate these agents, we make use of the `GridMDP` object from the `MDP` module. `sequential_decision_environment` is similar to that used for the `MDP` notebook but has discounting with $\\gamma = 0.9$.\n", + "\n", + "The `Agent-Program` can be obtained by creating an instance of the relevant `Agent-Class`. The `__call__` method allows the `Agent-Class` to be called as a function. The class needs to be instantiated with a policy ($\\pi$) and an `MDP` whose utility of states will be estimated.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "from mdp import sequential_decision_environment" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `sequential_decision_environment` is a GridMDP object as shown below. The rewards are **+1** and **-1** in the terminal states, and **-0.04** in the rest. Now we define actions and a policy similar to **Fig 21.1** in the book." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# Action Directions\n", + "north = (0, 1)\n", + "south = (0,-1)\n", + "west = (-1, 0)\n", + "east = (1, 0)\n", + "\n", + "policy = {\n", + " (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None,\n", + " (0, 1): north, (2, 1): north, (3, 1): None,\n", + " (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, \n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This enviroment will be extensively used in the following demonstrations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Direct Utility Estimation (DUE)\n", + " \n", + " The first, most naive method of estimating utility comes from the simplest interpretation of the above definition. We construct an agent that follows the policy until it reaches the terminal state. At each step, it logs its current state, reward. Once it reaches the terminal state, it can estimate the utility for each state for *that* iteration, by simply summing the discounted rewards from that state to the terminal one.\n", + "\n", + " It can now run this 'simulation' $n$ times and calculate the average utility of each state. If a state occurs more than once in a simulation, both its utility values are counted separately.\n", + " \n", + " Note that this method may be prohibitively slow for very large state-spaces. Besides, **it pays no attention to the transition probability $P(s'\\ |\\ s, a)$.** It misses out on information that it is capable of collecting (say, by recording the number of times an action from one state led to another state). The next method addresses this issue.\n", + " \n", + "### Examples\n", + "\n", + "The `PassiveDEUAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveDEUAgent` sums over rewards to find the estimated utility for each state. It thus requires the running of several iterations." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource PassiveDUEAgent" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's try the `PassiveDEUAgent` on the newly defined `sequential_decision_environment`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "DUEagent = PassiveDUEAgent(policy, sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can try passing information through the markove model for 200 times in order to get the converged utility value:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(200):\n", + " run_single_trial(DUEagent, sequential_decision_environment)\n", + " DUEagent.estimate_U()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's print our estimated utility for each position:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 1):0.7956939931414414\n", + "(1, 2):0.9162054322837863\n", + "(3, 2):1.0\n", + "(0, 0):0.734717308253083\n", + "(2, 2):0.9595117143816332\n", + "(0, 2):0.8481387156375687\n", + "(1, 0):0.4355860415209706\n", + "(2, 1):-0.550079982553143\n", + "(3, 1):-1.0\n" + ] + } + ], + "source": [ + "print('\\n'.join([str(k)+':'+str(v) for k, v in DUEagent.U.items()]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Adaptive Dynamic Programming (ADP)\n", + " \n", + " This method makes use of knowledge of the past state $s$, the action $a$, and the new perceived state $s'$ to estimate the transition probability $P(s'\\ |\\ s,a)$. It does this by the simple counting of new states resulting from previous states and actions.
    \n", + " The program runs through the policy a number of times, keeping track of:\n", + " - each occurrence of state $s$ and the policy-recommended action $a$ in $N_{sa}$\n", + " - each occurrence of $s'$ resulting from $a$ on $s$ in $N_{s'|sa}$.\n", + " \n", + " It can thus estimate $P(s'\\ |\\ s,a)$ as $N_{s'|sa}/N_{sa}$, which in the limit of infinite trials, will converge to the true value.
    \n", + " Using the transition probabilities thus estimated, it can apply `POLICY-EVALUATION` to estimate the utilities $U(s)$ using properties of convergence of the Bellman functions.\n", + " \n", + "### Examples\n", + "\n", + "The `PassiveADPAgent` class in the `rl` module implements the Agent Program described in **Fig 21.2** of the AIMA Book. `PassiveADPAgent` uses state transition and occurrence counts to estimate $P$, and then $U$. Go through the source below to understand the agent." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We instantiate a `PassiveADPAgent` below with the `GridMDP` shown and train it for 200 steps. The `rl` module has a simple implementation to simulate a single step of the iteration. The function is called `run_single_trial`." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Warning: Transition table is empty.\n" + ] + } + ], + "source": [ + "ADPagent = PassiveADPAgent(policy, sequential_decision_environment)\n", + "for i in range(200):\n", + " run_single_trial(ADPagent, sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The utilities are calculated as :" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 0):0.3014408531958584\n", + "(0, 1):0.40583863351329275\n", + "(1, 2):0.6581480346627065\n", + "(3, 2):1.0\n", + "(3, 0):0.0\n", + "(3, 1):-1.0\n", + "(2, 1):0.5341859348580892\n", + "(2, 0):0.0\n", + "(2, 2):0.810403779650285\n", + "(1, 0):0.23129676787627254\n", + "(0, 2):0.5214746706094832\n" + ] + } + ], + "source": [ + "print('\\n'.join([str(k)+':'+str(v) for k, v in ADPagent.U.items()]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When comparing to the result of `PassiveDUEAgent`, they both have -1.0 for utility at (3,1) and 1.0 at (3,2). Another point to notice is that the spot with the highest utility for both agents is (2,2) beside the terminal states, which is easy to understand when referring to the map." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Temporal-difference learning (TD)\n", + " \n", + " Instead of explicitly building the transition model $P$, the temporal-difference model makes use of the expected closeness between the utilities of two consecutive states $s$ and $s'$.\n", + " For the transition $s$ to $s'$, the update is written as:\n", + "$$U^{\\pi}(s) \\leftarrow U^{\\pi}(s) + \\alpha \\left( R(s) + \\gamma U^{\\pi}(s') - U^{\\pi}(s) \\right)$$\n", + " This model implicitly incorporates the transition probabilities by being weighed for each state by the number of times it is achieved from the current state. Thus, over a number of iterations, it converges similarly to the Bellman equations.\n", + " The advantage of the TD learning model is its relatively simple computation at each step, rather than having to keep track of various counts.\n", + " For $n_s$ states and $n_a$ actions the ADP model would have $n_s \\times n_a$ numbers $N_{sa}$ and $n_s^2 \\times n_a$ numbers $N_{s'|sa}$ to keep track of. The TD model must only keep track of a utility $U(s)$ for each state.\n", + " \n", + "### Examples\n", + "\n", + "`PassiveTDAgent` uses temporal differences to learn utility estimates. We learn the difference between the states and back up the values to previous states. Let us look into the source before we see some usage examples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%psource PassiveTDAgent" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In creating the `TDAgent`, we use the **same learning rate** $\\alpha$ as given in the footnote of the book: $\\alpha(n)=60/(59+n)$" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "TDagent = PassiveTDAgent(policy, sequential_decision_environment, alpha = lambda n: 60./(59+n))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we run **200 trials** for the agent to estimate Utilities." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(200):\n", + " run_single_trial(TDagent,sequential_decision_environment)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The calculated utilities are:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 1):0.36652562797696076\n", + "(1, 2):0.6584162739552614\n", + "(3, 2):1\n", + "(0, 0):0.27775491505339645\n", + "(3, 0):0.0\n", + "(3, 1):-1\n", + "(2, 1):0.6097040420148784\n", + "(2, 0):0.0\n", + "(2, 2):0.7936759402770092\n", + "(1, 0):0.19085842384266813\n", + "(0, 2):0.5258782999305713\n" + ] + } + ], + "source": [ + "print('\\n'.join([str(k)+':'+str(v) for k, v in TDagent.U.items()]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When comparing to previous agents, the result of `PassiveTDAgent` is closer to `PassiveADPAgent`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter21/images/mdp.png b/notebooks/chapter21/images/mdp.png new file mode 100644 index 000000000..e874130ee Binary files /dev/null and b/notebooks/chapter21/images/mdp.png differ diff --git a/notebooks/chapter22/Grammar.ipynb b/notebooks/chapter22/Grammar.ipynb new file mode 100644 index 000000000..3c1a2a005 --- /dev/null +++ b/notebooks/chapter22/Grammar.ipynb @@ -0,0 +1,526 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Grammar\n", + "\n", + "Languages can be represented by a set of grammar rules over a lexicon of words. Different languages can be represented by different types of grammar, but in Natural Language Processing we are mainly interested in context-free grammars.\n", + "\n", + "## Context-Free Grammar\n", + "\n", + "A lot of natural and programming languages can be represented by a **Context-Free Grammar (CFG)**. A CFG is a grammar that has a single non-terminal symbol on the left-hand side. That means a non-terminal can be replaced by the right-hand side of the rule regardless of context. An example of a CFG:\n", + "\n", + "```\n", + "S -> aSb | ε\n", + "```\n", + "\n", + "That means `S` can be replaced by either `aSb` or `ε` (with `ε` we denote the empty string). The lexicon of the language is comprised of the terminals `a` and `b`, while with `S` we denote the non-terminal symbol. In general, non-terminals are capitalized while terminals are not, and we usually name the starting non-terminal `S`. The language generated by the above grammar is the language anbn for n greater or equal than 1." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Probabilistic Context-Free Grammar\n", + "\n", + "While a simple CFG can be very useful, we might want to know the chance of each rule occurring. Above, we do not know if `S` is more likely to be replaced by `aSb` or `ε`. **Probabilistic Context-Free Grammars (PCFG)** are built to fill exactly that need. Each rule has a probability, given in brackets, and the probabilities of a rule sum up to 1:\n", + "\n", + "```\n", + "S -> aSb [0.7] | ε [0.3]\n", + "```\n", + "\n", + "Now we know it is more likely for `S` to be replaced by `aSb` than by `ε`.\n", + "\n", + "An issue with *PCFGs* is how we will assign the various probabilities to the rules. We could use our knowledge as humans to assign the probabilities, but that is laborious and prone to error task. Instead, we can *learn* the probabilities from data. Data is categorized as labeled (with correctly parsed sentences, usually called a **treebank**) or unlabeled (given only lexical and syntactic category names).\n", + "\n", + "With labeled data, we can simply count the occurrences. For the above grammar, if we have 100 `S` rules and 30 of them are of the form `S -> ε`, we assign a probability of 0.3 to the transformation.\n", + "\n", + "With unlabeled data, we have to learn both the grammar rules and the probability of each rule. We can go with many approaches, one of them the **inside-outside** algorithm. It uses a dynamic programming approach, that first finds the probability of a substring being generated by each rule and then estimates the probability of each rule." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Chomsky Normal Form\n", + "\n", + "Grammar is in Chomsky Normal Form (or **CNF**, not to be confused with *Conjunctive Normal Form*) if its rules are one of the three:\n", + "\n", + "* `X -> Y Z`\n", + "* `A -> a`\n", + "* `S -> ε`\n", + "\n", + "Where *X*, *Y*, *Z*, *A* are non-terminals, *a* is a terminal, *ε* is the empty string and *S* is the start symbol (the start symbol should not be appearing on the right-hand side of rules). Note that there can be multiple rules for each left-hand side non-terminal, as long they follow the above. For example, a rule for *X* might be: `X -> Y Z | A B | a | b`.\n", + "\n", + "Of course, we can also have a *CNF* with probabilities.\n", + "\n", + "This type of grammar may seem restrictive, but it can be proven that any context-free grammar can be converted to CNF." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lexicon\n", + "\n", + "The lexicon of a language is defined as a list of allowable words. These words are grouped into the usual classes: `verbs`, `nouns`, `adjectives`, `adverbs`, `pronouns`, `names`, `articles`, `prepositions` and `conjunctions`. For the first five classes, it is impossible to list all words since words are continuously being added in the classes. Recently \"google\" was added to the list of verbs, and words like that will continue to pop up and get added to the lists. For that reason, these first five categories are called **open classes**. The rest of the categories have much fewer words and much less development. While words like \"thou\" were commonly used in the past but have declined almost completely in usage, most changes take many decades or centuries to manifest, so we can safely assume the categories will remain static for the foreseeable future. Thus, these categories are called **closed classes**.\n", + "\n", + "An example lexicon for a PCFG (note that other classes can also be used according to the language, like `digits`, or `RelPro` for relative pronoun):\n", + "\n", + "```\n", + "Verb -> is [0.3] | say [0.1] | are [0.1] | ...\n", + "Noun -> robot [0.1] | sheep [0.05] | fence [0.05] | ...\n", + "Adjective -> good [0.1] | new [0.1] | sad [0.05] | ...\n", + "Adverb -> here [0.1] | lightly [0.05] | now [0.05] | ...\n", + "Pronoun -> me [0.1] | you [0.1] | he [0.05] | ...\n", + "RelPro -> that [0.4] | who [0.2] | which [0.2] | ...\n", + "Name -> john [0.05] | mary [0.05] | peter [0.01] | ...\n", + "Article -> the [0.35] | a [0.25] | an [0.025] | ...\n", + "Preposition -> to [0.25] | in [0.2] | at [0.1] | ...\n", + "Conjunction -> and [0.5] | or [0.2] | but [0.2] | ...\n", + "Digit -> 1 [0.3] | 2 [0.2] | 0 [0.2] | ...\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Grammer Rules\n", + "\n", + "With grammars we combine words from the lexicon into valid phrases. A grammar is comprised of **grammar rules**. Each rule transforms the left-hand side of the rule into the right-hand side. For example, `A -> B` means that `A` transforms into `B`. Let's build a grammar for the language we started building with the lexicon. We will use a PCFG.\n", + "\n", + "```\n", + "S -> NP VP [0.9] | S Conjunction S [0.1]\n", + "\n", + "NP -> Pronoun [0.3] | Name [0.1] | Noun [0.1] | Article Noun [0.25] |\n", + " Article Adjs Noun [0.05] | Digit [0.05] | NP PP [0.1] |\n", + " NP RelClause [0.05]\n", + "\n", + "VP -> Verb [0.4] | VP NP [0.35] | VP Adjective [0.05] | VP PP [0.1]\n", + " VP Adverb [0.1]\n", + "\n", + "Adjs -> Adjective [0.8] | Adjective Adjs [0.2]\n", + "\n", + "PP -> Preposition NP [1.0]\n", + "\n", + "RelClause -> RelPro VP [1.0]\n", + "```\n", + "\n", + "Some valid phrases the grammar produces: \"`mary is sad`\", \"`you are a robot`\" and \"`she likes mary and a good fence`\".\n", + "\n", + "What if we wanted to check if the phrase \"`mary is sad`\" is actually a valid sentence? We can use a **parse tree** to constructively prove that a string of words is a valid phrase in the given language and even calculate the probability of the generation of the sentence.\n", + "\n", + "![parse_tree](images/parse_tree.png)\n", + "\n", + "The probability of the whole tree can be calculated by multiplying the probabilities of each individual rule transormation: `0.9 * 0.1 * 0.05 * 0.05 * 0.4 * 0.05 * 0.3 = 0.00000135`.\n", + "\n", + "To conserve space, we can also write the tree in linear form:\n", + "\n", + "[S [NP [Name **mary**]] [VP [VP [Verb **is**]] [Adjective **sad**]]]\n", + "\n", + "Unfortunately, the current grammar **overgenerates**, that is, it creates sentences that are not grammatically correct (according to the English language), like \"`the fence are john which say`\". It also **undergenerates**, which means there are valid sentences it does not generate, like \"`he believes mary is sad`\"." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implementation\n", + "\n", + "In the module, we have implemented both probabilistic and non-probabilistic grammars. Both of these implementations follow the same format. There are functions for the lexicon and the rules which can be combined to create a grammar object.\n", + "\n", + "### Non-Probabilistic\n", + "\n", + "Execute the cell below to view the implementations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from nlp4e import *\n", + "from notebook4e import psource" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(Lexicon, Rules, Grammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's build a lexicon and a grammar for the above language:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Lexicon {'Verb': ['is', 'say', 'are'], 'Noun': ['robot', 'sheep', 'fence'], 'Adjective': ['good', 'new', 'sad'], 'Adverb': ['here', 'lightly', 'now'], 'Pronoun': ['me', 'you', 'he'], 'RelPro': ['that', 'who', 'which'], 'Name': ['john', 'mary', 'peter'], 'Article': ['the', 'a', 'an'], 'Preposition': ['to', 'in', 'at'], 'Conjunction': ['and', 'or', 'but'], 'Digit': ['1', '2', '0']}\n", + "\n", + "Rules: {'S': [['NP', 'VP'], ['S', 'Conjunction', 'S']], 'NP': [['Pronoun'], ['Name'], ['Noun'], ['Article', 'Noun'], ['Article', 'Adjs', 'Noun'], ['Digit'], ['NP', 'PP'], ['NP', 'RelClause']], 'VP': [['Verb'], ['VP', 'NP'], ['VP', 'Adjective'], ['VP', 'PP'], ['VP', 'Adverb']], 'Adjs': [['Adjective'], ['Adjective', 'Adjs']], 'PP': [['Preposition', 'NP']], 'RelClause': [['RelPro', 'VP']]}\n" + ] + } + ], + "source": [ + "lexicon = Lexicon(\n", + " Verb = \"is | say | are\",\n", + " Noun = \"robot | sheep | fence\",\n", + " Adjective = \"good | new | sad\",\n", + " Adverb = \"here | lightly | now\",\n", + " Pronoun = \"me | you | he\",\n", + " RelPro = \"that | who | which\",\n", + " Name = \"john | mary | peter\",\n", + " Article = \"the | a | an\",\n", + " Preposition = \"to | in | at\",\n", + " Conjunction = \"and | or | but\",\n", + " Digit = \"1 | 2 | 0\"\n", + ")\n", + "\n", + "print(\"Lexicon\", lexicon)\n", + "\n", + "rules = Rules(\n", + " S = \"NP VP | S Conjunction S\",\n", + " NP = \"Pronoun | Name | Noun | Article Noun \\\n", + " | Article Adjs Noun | Digit | NP PP | NP RelClause\",\n", + " VP = \"Verb | VP NP | VP Adjective | VP PP | VP Adverb\",\n", + " Adjs = \"Adjective | Adjective Adjs\",\n", + " PP = \"Preposition NP\",\n", + " RelClause = \"RelPro VP\"\n", + ")\n", + "\n", + "print(\"\\nRules:\", rules)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Both the functions return a dictionary with keys to the left-hand side of the rules. For the lexicon, the values are the terminals for each left-hand side non-terminal, while for the rules the values are the right-hand sides as lists.\n", + "\n", + "We can now use the variables `lexicon` and `rules` to build a grammar. After we've done so, we can find the transformations of a non-terminal (the `Noun`, `Verb` and the other basic classes do **not** count as proper non-terminals in the implementation). We can also check if a word is in a particular class." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "How can we rewrite 'VP'? [['Verb'], ['VP', 'NP'], ['VP', 'Adjective'], ['VP', 'PP'], ['VP', 'Adverb']]\n", + "Is 'the' an article? True\n", + "Is 'here' a noun? False\n" + ] + } + ], + "source": [ + "grammar = Grammar(\"A Simple Grammar\", rules, lexicon)\n", + "\n", + "print(\"How can we rewrite 'VP'?\", grammar.rewrites_for('VP'))\n", + "print(\"Is 'the' an article?\", grammar.isa('the', 'Article'))\n", + "print(\"Is 'here' a noun?\", grammar.isa('here', 'Noun'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Chomsky Normal Form\n", + "If the grammar is in **Chomsky Normal Form**, we can call the class function `cnf_rules` to get all the rules in the form of `(X, Y, Z)` for each `X -> Y Z` rule. Since the above grammar is not in *CNF* though, we have to create a new one." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "E_Chomsky = Grammar(\"E_Prob_Chomsky\", # A Grammar in Chomsky Normal Form\n", + " Rules(\n", + " S = \"NP VP\",\n", + " NP = \"Article Noun | Adjective Noun\",\n", + " VP = \"Verb NP | Verb Adjective\",\n", + " ),\n", + " Lexicon(\n", + " Article = \"the | a | an\",\n", + " Noun = \"robot | sheep | fence\",\n", + " Adjective = \"good | new | sad\",\n", + " Verb = \"is | say | are\"\n", + " ))" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[('S', 'NP', 'VP'), ('NP', 'Article', 'Noun'), ('NP', 'Adjective', 'Noun'), ('VP', 'Verb', 'NP'), ('VP', 'Verb', 'Adjective')]\n" + ] + } + ], + "source": [ + "print(E_Chomsky.cnf_rules())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can generate random phrases using our grammar. Most of them will be complete gibberish, falling under the overgenerated phrases of the grammar. That goes to show that in the grammar the valid phrases are much fewer than the overgenerated ones." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'a fence is 2 at 0 at he at john the fence at a good new sheep in the new sad robot which is who is a good robot which are good sad new now lightly sad at 2 and me are'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "grammar.generate_random('S')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Probabilistic\n", + "\n", + "The probabilistic grammars follow the same approach. They take as input a string, are assembled from grammar and a lexicon and can generate random sentences (giving the probability of the sentence). The main difference is that in the lexicon we have tuples (terminal, probability) instead of strings and for the rules, we have a list of tuples (list of non-terminals, probability) instead of the list of lists of non-terminals.\n", + "\n", + "Execute the cells to read the code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(ProbLexicon, ProbRules, ProbGrammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's build a lexicon and rules for the probabilistic grammar:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Lexicon {'Verb': [('is', 0.5), ('say', 0.3), ('are', 0.2)], 'Noun': [('robot', 0.4), ('sheep', 0.4), ('fence', 0.2)], 'Adjective': [('good', 0.5), ('new', 0.2), ('sad', 0.3)], 'Adverb': [('here', 0.6), ('lightly', 0.1), ('now', 0.3)], 'Pronoun': [('me', 0.3), ('you', 0.4), ('he', 0.3)], 'RelPro': [('that', 0.5), ('who', 0.3), ('which', 0.2)], 'Name': [('john', 0.4), ('mary', 0.4), ('peter', 0.2)], 'Article': [('the', 0.5), ('a', 0.25), ('an', 0.25)], 'Preposition': [('to', 0.4), ('in', 0.3), ('at', 0.3)], 'Conjunction': [('and', 0.5), ('or', 0.2), ('but', 0.3)], 'Digit': [('0', 0.35), ('1', 0.35), ('2', 0.3)]}\n", + "\n", + "Rules: {'S': [(['NP', 'VP'], 0.6), (['S', 'Conjunction', 'S'], 0.4)], 'NP': [(['Pronoun'], 0.2), (['Name'], 0.05), (['Noun'], 0.2), (['Article', 'Noun'], 0.15), (['Article', 'Adjs', 'Noun'], 0.1), (['Digit'], 0.05), (['NP', 'PP'], 0.15), (['NP', 'RelClause'], 0.1)], 'VP': [(['Verb'], 0.3), (['VP', 'NP'], 0.2), (['VP', 'Adjective'], 0.25), (['VP', 'PP'], 0.15), (['VP', 'Adverb'], 0.1)], 'Adjs': [(['Adjective'], 0.5), (['Adjective', 'Adjs'], 0.5)], 'PP': [(['Preposition', 'NP'], 1.0)], 'RelClause': [(['RelPro', 'VP'], 1.0)]}\n" + ] + } + ], + "source": [ + "lexicon = ProbLexicon(\n", + " Verb = \"is [0.5] | say [0.3] | are [0.2]\",\n", + " Noun = \"robot [0.4] | sheep [0.4] | fence [0.2]\",\n", + " Adjective = \"good [0.5] | new [0.2] | sad [0.3]\",\n", + " Adverb = \"here [0.6] | lightly [0.1] | now [0.3]\",\n", + " Pronoun = \"me [0.3] | you [0.4] | he [0.3]\",\n", + " RelPro = \"that [0.5] | who [0.3] | which [0.2]\",\n", + " Name = \"john [0.4] | mary [0.4] | peter [0.2]\",\n", + " Article = \"the [0.5] | a [0.25] | an [0.25]\",\n", + " Preposition = \"to [0.4] | in [0.3] | at [0.3]\",\n", + " Conjunction = \"and [0.5] | or [0.2] | but [0.3]\",\n", + " Digit = \"0 [0.35] | 1 [0.35] | 2 [0.3]\"\n", + ")\n", + "\n", + "print(\"Lexicon\", lexicon)\n", + "\n", + "rules = ProbRules(\n", + " S = \"NP VP [0.6] | S Conjunction S [0.4]\",\n", + " NP = \"Pronoun [0.2] | Name [0.05] | Noun [0.2] | Article Noun [0.15] \\\n", + " | Article Adjs Noun [0.1] | Digit [0.05] | NP PP [0.15] | NP RelClause [0.1]\",\n", + " VP = \"Verb [0.3] | VP NP [0.2] | VP Adjective [0.25] | VP PP [0.15] | VP Adverb [0.1]\",\n", + " Adjs = \"Adjective [0.5] | Adjective Adjs [0.5]\",\n", + " PP = \"Preposition NP [1]\",\n", + " RelClause = \"RelPro VP [1]\"\n", + ")\n", + "\n", + "print(\"\\nRules:\", rules)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's use the above to assemble our probabilistic grammar and run some simple queries:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "How can we rewrite 'VP'? [(['Verb'], 0.3), (['VP', 'NP'], 0.2), (['VP', 'Adjective'], 0.25), (['VP', 'PP'], 0.15), (['VP', 'Adverb'], 0.1)]\n", + "Is 'the' an article? True\n", + "Is 'here' a noun? False\n" + ] + } + ], + "source": [ + "grammar = ProbGrammar(\"A Simple Probabilistic Grammar\", rules, lexicon)\n", + "\n", + "print(\"How can we rewrite 'VP'?\", grammar.rewrites_for('VP'))\n", + "print(\"Is 'the' an article?\", grammar.isa('the', 'Article'))\n", + "print(\"Is 'here' a noun?\", grammar.isa('here', 'Noun'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we have a grammar in *CNF*, we can get a list of all the rules. Let's create a grammar in the form and print the *CNF* rules:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "E_Prob_Chomsky = ProbGrammar(\"E_Prob_Chomsky\", # A Probabilistic Grammar in CNF\n", + " ProbRules(\n", + " S = \"NP VP [1]\",\n", + " NP = \"Article Noun [0.6] | Adjective Noun [0.4]\",\n", + " VP = \"Verb NP [0.5] | Verb Adjective [0.5]\",\n", + " ),\n", + " ProbLexicon(\n", + " Article = \"the [0.5] | a [0.25] | an [0.25]\",\n", + " Noun = \"robot [0.4] | sheep [0.4] | fence [0.2]\",\n", + " Adjective = \"good [0.5] | new [0.2] | sad [0.3]\",\n", + " Verb = \"is [0.5] | say [0.3] | are [0.2]\"\n", + " ))" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[('S', 'NP', 'VP', 1.0), ('NP', 'Article', 'Noun', 0.6), ('NP', 'Adjective', 'Noun', 0.4), ('VP', 'Verb', 'NP', 0.5), ('VP', 'Verb', 'Adjective', 0.5)]\n" + ] + } + ], + "source": [ + "print(E_Prob_Chomsky.cnf_rules())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Lastly, we can generate random sentences from this grammar. The function `prob_generation` returns a tuple (sentence, probability)." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a good good new good sheep that say a good good robot the sad robot to 1 to me you to sheep are\n", + "5.511240000000004e-26\n" + ] + } + ], + "source": [ + "sentence, prob = grammar.generate_random('S')\n", + "print(sentence)\n", + "print(prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As with the non-probabilistic grammars, this one mostly overgenerates. You can also see that the probability is very, very low, which means there are a ton of generate able sentences (in this case infinite, since we have recursion; notice how `VP` can produce another `VP`, for example)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter22/Introduction.ipynb b/notebooks/chapter22/Introduction.ipynb new file mode 100644 index 000000000..0905b91a9 --- /dev/null +++ b/notebooks/chapter22/Introduction.ipynb @@ -0,0 +1,92 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# NATURAL LANGUAGE PROCESSING\n", + "\n", + "The notebooks in this folder cover chapters 23 of the book *Artificial Intelligence: A Modern Approach*, 4th Edition. The implementations of the algorithms can be found in [nlp.py](https://github.com/aimacode/aima-python/blob/master/nlp4e.py).\n", + "\n", + "Run the below cell to import the code from the module and get started!" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from nlp4e import *\n", + "from notebook4e import psource" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## OVERVIEW\n", + "\n", + "**Natural Language Processing (NLP)** is a field of AI concerned with understanding, analyzing and using natural languages. This field is considered a difficult yet intriguing field of study since it is connected to how humans and their languages work.\n", + "\n", + "Applications of the field include translation, speech recognition, topic segmentation, information extraction and retrieval, and a lot more.\n", + "\n", + "Below we take a look at some algorithms in the field. Before we get right into it though, we will take a look at a very useful form of language, **context-free** languages. Even though they are a bit restrictive, they have been used a lot in research in natural language processing.\n", + "\n", + "Below is a summary of the demonstration files in this chapter." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "\n", + "- Introduction: Introduction to the field of nlp and the table of contents.\n", + "- Grammars: Introduction to grammar rules and lexicon of words of a language.\n", + " - Context-free Grammar\n", + " - Probabilistic Context-Free Grammar\n", + " - Chomsky Normal Form\n", + " - Lexicon\n", + " - Grammar Rules\n", + " - Implementation of Different Grammars\n", + "- Parsing: The algorithms parsing sentences according to a certain kind of grammar.\n", + " - Chart Parsing\n", + " - CYK Parsing\n", + " - A-star Parsing\n", + " - Beam Search Parsing\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter22/Parsing.ipynb b/notebooks/chapter22/Parsing.ipynb new file mode 100644 index 000000000..50a4264fb --- /dev/null +++ b/notebooks/chapter22/Parsing.ipynb @@ -0,0 +1,522 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Parsing\n", + "\n", + "## Overview\n", + "\n", + "Syntactic analysis (or **parsing**) of a sentence is the process of uncovering the phrase structure of the sentence according to the rules of grammar. \n", + "\n", + "There are two main approaches to parsing. *Top-down*, start with the starting symbol and build a parse tree with the given words as its leaves, and *bottom-up*, where we start from the given words and build a tree that has the starting symbol as its root. Both approaches involve \"guessing\" ahead, so it may take longer to parse a sentence (the wrong guess mean a lot of backtracking). Thankfully, a lot of effort is spent in analyzing already analyzed substrings, so we can follow a dynamic programming approach to store and reuse these parses instead of recomputing them. \n", + "\n", + "In dynamic programming, we use a data structure known as a chart, thus the algorithms parsing a chart is called **chart parsing**. We will cover several different chart parsing algorithms." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Chart Parsing\n", + "\n", + "### Overview\n", + "\n", + "The chart parsing algorithm is a general form of the following algorithms. Given a non-probabilistic grammar and a sentence, this algorithm builds a parse tree in a top-down manner, with the words of the sentence as the leaves. It works with a dynamic programming approach, building a chart to store parses for substrings so that it doesn't have to analyze them again (just like the CYK algorithm). Each non-terminal, starting from S, gets replaced by its right-hand side rules in the chart until we end up with the correct parses.\n", + "\n", + "### Implementation\n", + "\n", + "A parse is in the form `[start, end, non-terminal, sub-tree, expected-transformation]`, where `sub-tree` is a tree with the corresponding `non-terminal` as its root and `expected-transformation` is a right-hand side rule of the `non-terminal`.\n", + "\n", + "The chart parsing is implemented in a class, `Chart`. It is initialized with grammar and can return the list of all the parses of a sentence with the `parses` function.\n", + "\n", + "The chart is a list of lists. The lists correspond to the lengths of substrings (including the empty string), from start to finish. When we say 'a point in the chart', we refer to a list of a certain length.\n", + "\n", + "A quick rundown of the class functions:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* `parses`: Returns a list of parses for a given sentence. If the sentence can't be parsed, it will return an empty list. Initializes the process by calling `parse` from the starting symbol.\n", + "\n", + "\n", + "* `parse`: Parses the list of words and builds the chart.\n", + "\n", + "\n", + "* `add_edge`: Adds another edge to the chart at a given point. Also, examines whether the edge extends or predicts another edge. If the edge itself is not expecting a transformation, it will extend other edges and it will predict edges otherwise.\n", + "\n", + "\n", + "* `scanner`: Given a word and a point in the chart, it extends edges that were expecting a transformation that can result in the given word. For example, if the word 'the' is an 'Article' and we are examining two edges at a chart's point, with one expecting an 'Article' and the other a 'Verb', the first one will be extended while the second one will not.\n", + "\n", + "\n", + "* `predictor`: If an edge can't extend other edges (because it is expecting a transformation itself), we will add to the chart rules/transformations that can help extend the edge. The new edges come from the right-hand side of the expected transformation's rules. For example, if an edge is expecting the transformation 'Adjective Noun', we will add to the chart an edge for each right-hand side rule of the non-terminal 'Adjective'.\n", + "\n", + "\n", + "* `extender`: Extends edges given an edge (called `E`). If `E`'s non-terminal is the same as the expected transformation of another edge (let's call it `A`), add to the chart a new edge with the non-terminal of `A` and the transformations of `A` minus the non-terminal that matched with `E`'s non-terminal. For example, if an edge `E` has 'Article' as its non-terminal and is expecting no transformation, we need to see what edges it can extend. Let's examine the edge `N`. This expects a transformation of 'Noun Verb'. 'Noun' does not match with 'Article', so we move on. Another edge, `A`, expects a transformation of 'Article Noun' and has a non-terminal of 'NP'. We have a match! A new edge will be added with 'NP' as its non-terminal (the non-terminal of `A`) and 'Noun' as the expected transformation (the rest of the expected transformation of `A`).\n", + "\n", + "You can view the source code by running the cell below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(Chart)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "We will use the grammar `E0` to parse the sentence \"the stench is in 2 2\".\n", + "\n", + "First, we need to build a `Chart` object:" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [], + "source": [ + "chart = Chart(E0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And then we simply call the `parses` function:" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[0, 6, 'S', [[0, 2, 'NP', [('Article', 'the'), ('Noun', 'stench')], []], [2, 6, 'VP', [[2, 3, 'VP', [('Verb', 'is')], []], [3, 6, 'PP', [('Preposition', 'in'), [4, 6, 'NP', [('Digit', '2'), ('Digit', '2')], []]], []]], []]], []]]\n" + ] + } + ], + "source": [ + "print(chart.parses('the stench is in 2 2'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see which edges get added by setting the optional initialization argument `trace` to true." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "chart_trace = Chart(nlp.E0, trace=True)\n", + "chart_trace.parses('the stench is in 2 2')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's try and parse a sentence that is not recognized by the grammar:" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[]\n" + ] + } + ], + "source": [ + "print(chart.parses('the stench 2 2'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An empty list was returned." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CYK Parse\n", + "\n", + "The *CYK Parsing Algorithm* (named after its inventors, Cocke, Younger, and Kasami) utilizes dynamic programming to parse sentences of grammar in *Chomsky Normal Form*.\n", + "\n", + "The CYK algorithm returns an *M x N x N* array (named *P*), where *N* is the number of words in the sentence and *M* the number of non-terminal symbols in the grammar. Each element in this array shows the probability of a substring being transformed from a particular non-terminal. To find the most probable parse of the sentence, a search in the resulting array is required. Search heuristic algorithms work well in this space, and we can derive the heuristics from the properties of the grammar.\n", + "\n", + "The algorithm in short works like this: There is an external loop that determines the length of the substring. Then the algorithm loops through the words in the sentence. For each word, it again loops through all the words to its right up to the first-loop length. The substring will work on in this iteration is the words from the second-loop word with the first-loop length. Finally, it loops through all the rules in the grammar and updates the substring's probability for each right-hand side non-terminal." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "The implementation takes as input a list of words and a probabilistic grammar (from the `ProbGrammar` class detailed above) in CNF and returns the table/dictionary *P*. An item's key in *P* is a tuple in the form `(Non-terminal, the start of a substring, length of substring)`, and the value is a `Tree` object. The `Tree` data structure has two attributes: `root` and `leaves`. `root` stores the value of current tree node and `leaves` is a list of children nodes which may be terminal states(words in the sentence) or a sub tree.\n", + "\n", + "For example, for the sentence \"the monkey is dancing\" and the substring \"the monkey\" an item can be `('NP', 0, 2): `, which means the first two words (the substring from index 0 and length 2) can be parse to a `NP` and the detailed operations are recorded by a `Tree` object.\n", + "\n", + "Before we continue, you can take a look at the source code by running the cell below:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from nlp4e import *\n", + "from notebook4e import psource" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(CYK_parse)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When updating the probability of a substring, we pick the max of its current one and the probability of the substring broken into two parts: one from the second-loop word with third-loop length, and the other from the first part's end to the remainder of the first-loop length.\n", + "\n", + "### Example\n", + "\n", + "Let's build a probabilistic grammar in CNF:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "E_Prob_Chomsky = ProbGrammar(\"E_Prob_Chomsky\", # A Probabilistic Grammar in CNF\n", + " ProbRules(\n", + " S = \"NP VP [1]\",\n", + " NP = \"Article Noun [0.6] | Adjective Noun [0.4]\",\n", + " VP = \"Verb NP [0.5] | Verb Adjective [0.5]\",\n", + " ),\n", + " ProbLexicon(\n", + " Article = \"the [0.5] | a [0.25] | an [0.25]\",\n", + " Noun = \"robot [0.4] | sheep [0.4] | fence [0.2]\",\n", + " Adjective = \"good [0.5] | new [0.2] | sad [0.3]\",\n", + " Verb = \"is [0.5] | say [0.3] | are [0.2]\"\n", + " ))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's see the probabilities table for the sentence \"the robot is good\":" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "defaultdict(, {('Article', 0, 0): , ('Noun', 1, 1): , ('Verb', 2, 2): , ('Adjective', 3, 3): , ('VP', 2, 3): })\n" + ] + } + ], + "source": [ + "words = ['the', 'robot', 'is', 'good']\n", + "grammar = E_Prob_Chomsky\n", + "\n", + "P = CYK_parse(words, grammar)\n", + "print(P)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A `defaultdict` object is returned (`defaultdict` is basically a dictionary but with a default value/type). Keys are tuples in the form mentioned above and the values are the corresponding parse trees which demonstrates how the sentence will be parsed. Let's check the details of each parsing:" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{('Article', 0, 0): ['the'], ('Noun', 1, 1): ['robot'], ('Verb', 2, 2): ['is'], ('Adjective', 3, 3): ['good'], ('VP', 2, 3): [, ]}\n" + ] + } + ], + "source": [ + "parses = {k: p.leaves for k, p in P.items()}\n", + "\n", + "print(parses)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Please note that each item in the returned dict represents a parsing strategy. For instance, `('Article', 0, 0): ['the']` means parsing the article at position 0 from the word `the`. For the key `'VP', 2, 3`, it is mapped to another `Tree` which means this is a nested parsing step. If we print this item in detail: " + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['is']\n", + "['good']\n" + ] + } + ], + "source": [ + "for subtree in P['VP', 2, 3].leaves:\n", + " print(subtree.leaves)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So we can interpret this step as parsing the word at index 2 and 3 together('is' and 'good') as a verh phrase." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A-star Parsing\n", + "\n", + "The CYK algorithm uses space of $O(n^2m)$ for the P and T tables, where n is the number of words in the sentence, and m is the number of nonterminal symbols in the grammar and takes time $O(n^3m)$. This is the best algorithm if we want to find the best parse and works for all possible context-free grammars. But actually, we only want to parse natural languages, not all possible grammars, which allows us to apply more efficient algorithms.\n", + "\n", + "By applying a-start search, we are using the state-space search and we can get $O(n)$ running time. In this situation, each state is a list of items (words or categories), the start state is a list of words, and a goal state is the single item S. \n", + "\n", + "In our code, we implemented a demonstration of `astar_search_parsing` which deals with the text parsing problem. By specifying different `words` and `gramma`, we can use this searching strategy to deal with different text parsing problems. The algorithm returns a boolean telling whether the input words is a sentence under the given grammar.\n", + "\n", + "For detailed implementation, please execute the following block:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(astar_search_parsing)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's try \"the wumpus is dead\" example. First we need to define the grammer and words in the sentence." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [], + "source": [ + "grammar = E0\n", + "words = ['the', 'wumpus', 'is', 'dead']" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'S'" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "astar_search_parsing(words, grammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The algorithm returns a 'S' which means it treats the inputs as a sentence. If we change the order of words to make it unreadable:" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 69, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "words_swaped = [\"the\", \"is\", \"wupus\", \"dead\"]\n", + "astar_search_parsing(words_swaped, grammar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then the algorithm asserts that out words cannot be a sentence." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Beam Search Parsing\n", + "\n", + "In the beam searching algorithm, we still treat the text parsing problem as a state-space searching algorithm. when using beam search, we consider only the b most probable alternative parses. This means we are not guaranteed to find the parse with the highest probability, but (with a careful implementation) the parser can operate in $O(n)$ time and still finds the best parse most of the time. A beam search parser with b = 1 is called a **deterministic parser**.\n", + "\n", + "### Implementation\n", + "\n", + "In the beam search, we maintain a `frontier` which is a priority queue keep tracking of the current frontier of searching. In each step, we explore all the examples in `frontier` and saves the best n examples as the frontier of the exploration of the next step.\n", + "\n", + "For detailed implementation, please view with the following code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(beam_search_parsing)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Let's try both the positive and negative wumpus example on this algorithm:" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'S'" + ] + }, + "execution_count": 70, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beam_search_parsing(words, grammar)" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 71, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beam_search_parsing(words_swaped, grammar)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter22/images/parse_tree.png b/notebooks/chapter22/images/parse_tree.png new file mode 100644 index 000000000..f6ca87b2f Binary files /dev/null and b/notebooks/chapter22/images/parse_tree.png differ diff --git a/notebooks/chapter22/nlp_apps.ipynb b/notebooks/chapter22/nlp_apps.ipynb new file mode 100644 index 000000000..bd38efadf --- /dev/null +++ b/notebooks/chapter22/nlp_apps.ipynb @@ -0,0 +1,1038 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# NATURAL LANGUAGE PROCESSING APPLICATIONS\n", + "\n", + "In this notebook we will take a look at some indicative applications of natural language processing. We will cover content from [`nlp.py`](https://github.com/aimacode/aima-python/blob/master/nlp.py) and [`text.py`](https://github.com/aimacode/aima-python/blob/master/text.py), for chapters 22 and 23 of Stuart Russel's and Peter Norvig's book [*Artificial Intelligence: A Modern Approach*](http://aima.cs.berkeley.edu/)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CONTENTS\n", + "\n", + "* Language Recognition\n", + "* Author Recognition\n", + "* The Federalist Papers\n", + "* Text Classification" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# LANGUAGE RECOGNITION\n", + "\n", + "A very useful application of text models (you can read more on them on the [`text notebook`](https://github.com/aimacode/aima-python/blob/master/text.ipynb)) is categorizing text into a language. In fact, with enough data we can categorize correctly mostly any text. That is because different languages have certain characteristics that set them apart. For example, in German it is very usual for 'c' to be followed by 'h' while in English we see 't' followed by 'h' a lot.\n", + "\n", + "Here we will build an application to categorize sentences in either English or German.\n", + "\n", + "First we need to build our dataset. We will take as input text in English and in German and we will extract n-gram character models (in this case, *bigrams* for n=2). For English, we will use *Flatland* by Edwin Abbott and for German *Faust* by Goethe.\n", + "\n", + "Let's build our text models for each language, which will hold the probability of each bigram occuring in the text." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from utils import open_data\n", + "from text import *\n", + "\n", + "flatland = open_data(\"EN-text/flatland.txt\").read()\n", + "wordseq = words(flatland)\n", + "\n", + "P_flatland = NgramCharModel(2, wordseq)\n", + "\n", + "faust = open_data(\"GE-text/faust.txt\").read()\n", + "wordseq = words(faust)\n", + "\n", + "P_faust = NgramCharModel(2, wordseq)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can use this information to build a *Naive Bayes Classifier* that will be used to categorize sentences (you can read more on Naive Bayes on the [`learning notebook`](https://github.com/aimacode/aima-python/blob/master/learning.ipynb)). The classifier will take as input the probability distribution of bigrams and given a list of bigrams (extracted from the sentence to be classified), it will calculate the probability of the example/sentence coming from each language and pick the maximum.\n", + "\n", + "Let's build our classifier, with the assumption that English is as probable as German (the input is a dictionary with values the text models and keys the tuple `language, probability`):" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from learning import NaiveBayesLearner\n", + "\n", + "dist = {('English', 1): P_flatland, ('German', 1): P_faust}\n", + "\n", + "nBS = NaiveBayesLearner(dist, simple=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we need to write a function that takes as input a sentence, breaks it into a list of bigrams and classifies it with the naive bayes classifier from above.\n", + "\n", + "Once we get the text model for the sentence, we need to unravel it. The text models show the probability of each bigram, but the classifier can't handle that extra data. It requires a simple *list* of bigrams. So, if the text model shows that a bigram appears three times, we need to add it three times in the list. Since the text model stores the n-gram information in a dictionary (with the key being the n-gram and the value the number of times the n-gram appears) we need to iterate through the items of the dictionary and manually add them to the list of n-grams." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "def recognize(sentence, nBS, n):\n", + " sentence = sentence.lower()\n", + " wordseq = words(sentence)\n", + " \n", + " P_sentence = NgramCharModel(n, wordseq)\n", + " \n", + " ngrams = []\n", + " for b, p in P_sentence.dictionary.items():\n", + " ngrams += [b]*p\n", + " \n", + " print(ngrams)\n", + " \n", + " return nBS(ngrams)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can start categorizing sentences." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(' ', 'i'), ('i', 'c'), ('c', 'h'), (' ', 'b'), ('b', 'i'), ('i', 'n'), ('i', 'n'), (' ', 'e'), ('e', 'i'), (' ', 'p'), ('p', 'l'), ('l', 'a'), ('a', 't'), ('t', 'z')]\n" + ] + }, + { + "data": { + "text/plain": [ + "'German'" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"Ich bin ein platz\", nBS, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(' ', 't'), ('t', 'u'), ('u', 'r'), ('r', 't'), ('t', 'l'), ('l', 'e'), ('e', 's'), (' ', 'f'), ('f', 'l'), ('l', 'y'), (' ', 'h'), ('h', 'i'), ('i', 'g'), ('g', 'h')]\n" + ] + }, + { + "data": { + "text/plain": [ + "'English'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"Turtles fly high\", nBS, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(' ', 'd'), ('d', 'e'), ('e', 'r'), ('e', 'r'), (' ', 'p'), ('p', 'e'), ('e', 'l'), ('l', 'i'), ('i', 'k'), ('k', 'a'), ('a', 'n'), (' ', 'i'), ('i', 's'), ('s', 't'), (' ', 'h'), ('h', 'i'), ('i', 'e')]\n" + ] + }, + { + "data": { + "text/plain": [ + "'German'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"Der pelikan ist hier\", nBS, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(' ', 'a'), ('a', 'n'), ('n', 'd'), (' ', 't'), (' ', 't'), ('t', 'h'), ('t', 'h'), ('h', 'u'), ('u', 's'), ('h', 'e'), (' ', 'w'), ('w', 'i'), ('i', 'z'), ('z', 'a'), ('a', 'r'), ('r', 'd'), (' ', 's'), ('s', 'p'), ('p', 'o'), ('o', 'k'), ('k', 'e')]\n" + ] + }, + { + "data": { + "text/plain": [ + "'English'" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"And thus the wizard spoke\", nBS, 2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can add more languages if you want, the algorithm works for as many as you like! Also, you can play around with *n*. Here we used 2, but other numbers work too (even though 2 suffices). The algorithm is not perfect, but it has high accuracy even for small samples like the ones we used. That is because English and German are very different languages. The closer together languages are (for example, Norwegian and Swedish share a lot of common ground) the lower the accuracy of the classifier." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## AUTHOR RECOGNITION\n", + "\n", + "Another similar application to language recognition is recognizing who is more likely to have written a sentence, given text written by them. Here we will try and predict text from Edwin Abbott and Jane Austen. They wrote *Flatland* and *Pride and Prejudice* respectively.\n", + "\n", + "We are optimistic we can determine who wrote what based on the fact that Abbott wrote his novella on much later date than Austen, which means there will be linguistic differences between the two works. Indeed, *Flatland* uses more modern and direct language while *Pride and Prejudice* is written in a more archaic tone containing more sophisticated wording.\n", + "\n", + "Similarly with Language Recognition, we will first import the two datasets. This time though we are not looking for connections between characters, since that wouldn't give that great results. Why? Because both authors use English and English follows a set of patterns, as we show earlier. Trying to determine authorship based on this patterns would not be very efficient.\n", + "\n", + "Instead, we will abstract our querying to a higher level. We will use words instead of characters. That way we can more accurately pick at the differences between their writing style and thus have a better chance at guessing the correct author.\n", + "\n", + "Let's go right ahead and import our data:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "from utils import open_data\n", + "from text import *\n", + "\n", + "flatland = open_data(\"EN-text/flatland.txt\").read()\n", + "wordseq = words(flatland)\n", + "\n", + "P_Abbott = UnigramWordModel(wordseq, 5)\n", + "\n", + "pride = open_data(\"EN-text/pride.txt\").read()\n", + "wordseq = words(pride)\n", + "\n", + "P_Austen = UnigramWordModel(wordseq, 5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This time we set the `default` parameter of the model to 5, instead of 0. If we leave it at 0, then when we get a sentence containing a word we have not seen from that particular author, the chance of that sentence coming from that author is exactly 0 (since to get the probability, we multiply all the separate probabilities; if one is 0 then the result is also 0). To avoid that, we tell the model to add 5 to the count of all the words that appear.\n", + "\n", + "Next we will build the Naive Bayes Classifier:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "from learning import NaiveBayesLearner\n", + "\n", + "dist = {('Abbott', 1): P_Abbott, ('Austen', 1): P_Austen}\n", + "\n", + "nBS = NaiveBayesLearner(dist, simple=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we have build our classifier, we will start classifying. First, we need to convert the given sentence to the format the classifier needs. That is, a list of words." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "def recognize(sentence, nBS):\n", + " sentence = sentence.lower()\n", + " sentence_words = words(sentence)\n", + " \n", + " return nBS(sentence_words)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First we will input a sentence that is something Abbott would write. Note the use of square and the simpler language." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Abbott'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"the square is mad\", nBS)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The classifier correctly guessed Abbott.\n", + "\n", + "Next we will input a more sophisticated sentence, similar to the style of Austen." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Austen'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "recognize(\"a most peculiar acquaintance\", nBS)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The classifier guessed correctly again.\n", + "\n", + "You can try more sentences on your own. Unfortunately though, since the datasets are pretty small, chances are the guesses will not always be correct." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## THE FEDERALIST PAPERS\n", + "\n", + "Let's now take a look at a harder problem, classifying the authors of the [Federalist Papers](https://en.wikipedia.org/wiki/The_Federalist_Papers). The *Federalist Papers* are a series of papers written by Alexander Hamilton, James Madison and John Jay towards establishing the United States Constitution.\n", + "\n", + "What is interesting about these papers is that they were all written under a pseudonym, \"Publius\", to keep the identity of the authors a secret. Only after Hamilton's death, when a list was found written by him detailing the authorship of the papers, did the rest of the world learn what papers each of the authors wrote. After the list was published, Madison chimed in to make a couple of corrections: Hamilton, Madison said, hastily wrote down the list and assigned some papers to the wrong author!\n", + "\n", + "Here we will try and find out who really wrote these mysterious papers.\n", + "\n", + "To solve this we will learn from the undisputed papers to predict the disputed ones. First, let's read the texts from the file:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "from utils import open_data\n", + "from text import *\n", + "\n", + "federalist = open_data(\"EN-text/federalist.txt\").read()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see how the text looks. We will print the first 500 characters:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The Project Gutenberg EBook of The Federalist Papers, by \\nAlexander Hamilton and John Jay and James Madison\\n\\nThis eBook is for the use of anyone anywhere at no cost and with\\nalmost no restrictions whatsoever. You may copy it, give it away or\\nre-use it under the terms of the Project Gutenberg License included\\nwith this eBook or online at www.gutenberg.net\\n\\n\\nTitle: The Federalist Papers\\n\\nAuthor: Alexander Hamilton\\n John Jay\\n James Madison\\n\\nPosting Date: December 12, 2011 [EBook #18]'" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "federalist[:500]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It seems that the text file opens with a license agreement, hardly useful in our case. In fact, the license spans 113 words, while there is also a licensing agreement at the end of the file, which spans 3098 words. We need to remove them. To do so, we will first convert the text into words, to make our lives easier." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "wordseq = words(federalist)\n", + "wordseq = wordseq[114:-3098]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now take a look at the first 100 words:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'federalist no 1 general introduction for the independent journal hamilton to the people of the state of new york after an unequivocal experience of the inefficacy of the subsisting federal government you are called upon to deliberate on a new constitution for the united states of america the subject speaks its own importance comprehending in its consequences nothing less than the existence of the union the safety and welfare of the parts of which it is composed the fate of an empire in many respects the most interesting in the world it has been frequently remarked that it seems to'" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "' '.join(wordseq[:100])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Much better.\n", + "\n", + "As with any Natural Language Processing problem, it is prudent to do some text pre-processing and clean our data before we start building our model. Remember that all the papers are signed as 'Publius', so we can safely remove that word, since it doesn't give us any information as to the real author.\n", + "\n", + "NOTE: Since we are only removing a single word from each paper, this step can be skipped. We add it here to show that processing the data in our hands is something we should always be considering. Oftentimes pre-processing the data in just the right way is the difference between a robust model and a flimsy one." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "wordseq = [w for w in wordseq if w != 'publius']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we have to separate the text from a block of words into papers and assign them to their authors. We can see that each paper starts with the word 'federalist', so we will split the text on that word.\n", + "\n", + "The disputed papers are the papers from 49 to 58, from 18 to 20 and paper 64. We want to leave these papers unassigned. Also, note that there are two versions of paper 70; both from Hamilton.\n", + "\n", + "Finally, to keep the implementation intuitive, we add a `None` object at the start of the `papers` list to make the list index match up with the paper numbering (for example, `papers[5]` now corresponds to paper no. 5 instead of the paper no.6 in the 0-indexed Python)." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4, 16, 52)" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import re\n", + "\n", + "papers = re.split(r'federalist\\s', ' '.join(wordseq))\n", + "papers = [p for p in papers if p not in ['', ' ']]\n", + "papers = [None] + papers\n", + "\n", + "disputed = list(range(49, 58+1)) + [18, 19, 20, 64]\n", + "jay, madison, hamilton = [], [], []\n", + "for i, p in enumerate(papers):\n", + " if i in disputed or i == 0:\n", + " continue\n", + " \n", + " if 'jay' in p:\n", + " jay.append(p)\n", + " elif 'madison' in p:\n", + " madison.append(p)\n", + " else:\n", + " hamilton.append(p)\n", + "\n", + "len(jay), len(madison), len(hamilton)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, from the undisputed papers Jay wrote 4, Madison 17 and Hamilton 51 (+1 duplicate). Let's now build our word models. The Unigram Word Model again will come in handy." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "hamilton = ''.join(hamilton)\n", + "hamilton_words = words(hamilton)\n", + "P_hamilton = UnigramWordModel(hamilton_words, default=1)\n", + "\n", + "madison = ''.join(madison)\n", + "madison_words = words(madison)\n", + "P_madison = UnigramWordModel(madison_words, default=1)\n", + "\n", + "jay = ''.join(jay)\n", + "jay_words = words(jay)\n", + "P_jay = UnigramWordModel(jay_words, default=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now it is time to build our new Naive Bayes Learner. It is very similar to the one found in `learning.py`, but with an important difference: it doesn't classify an example, but instead returns the probability of the example belonging to each class. This will allow us to not only see to whom a paper belongs to, but also the probability of authorship as well. \n", + "We will build two versions of Learners, one will multiply probabilities as is and other will add the logarithms of them.\n", + "\n", + "Finally, since we are dealing with long text and the string of probability multiplications is long, we will end up with the results being rounded to 0 due to floating point underflow. To work around this problem we will use the built-in Python library `decimal`, which allows as to set decimal precision to much larger than normal.\n", + "\n", + "Note that the logarithmic learner will compute a negative likelihood since the logarithm of values less than 1 will be negative.\n", + "Thus, the author with the lesser magnitude of proportion is more likely to have written that paper.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "import decimal\n", + "import math\n", + "from decimal import Decimal\n", + "\n", + "decimal.getcontext().prec = 100\n", + "\n", + "def precise_product(numbers):\n", + " result = 1\n", + " for x in numbers:\n", + " result *= Decimal(x)\n", + " return result\n", + "\n", + "def log_product(numbers):\n", + " result = 0.0\n", + " for x in numbers:\n", + " result += math.log(x)\n", + " return result\n", + "\n", + "def NaiveBayesLearner(dist):\n", + " \"\"\"A simple naive bayes classifier that takes as input a dictionary of\n", + " Counter distributions and can then be used to find the probability\n", + " of a given item belonging to each class.\n", + " The input dictionary is in the following form:\n", + " ClassName: Counter\"\"\"\n", + " attr_dist = {c_name: count_prob for c_name, count_prob in dist.items()}\n", + "\n", + " def predict(example):\n", + " \"\"\"Predict the probabilities for each class.\"\"\"\n", + " def class_prob(target, e):\n", + " attr = attr_dist[target]\n", + " return precise_product([attr[a] for a in e])\n", + "\n", + " pred = {t: class_prob(t, example) for t in dist.keys()}\n", + "\n", + " total = sum(pred.values())\n", + " for k, v in pred.items():\n", + " pred[k] = v / total\n", + "\n", + " return pred\n", + "\n", + " return predict\n", + "\n", + "def NaiveBayesLearnerLog(dist):\n", + " \"\"\"A simple naive bayes classifier that takes as input a dictionary of\n", + " Counter distributions and can then be used to find the probability\n", + " of a given item belonging to each class. It will compute the likelihood by adding the logarithms of probabilities.\n", + " The input dictionary is in the following form:\n", + " ClassName: Counter\"\"\"\n", + " attr_dist = {c_name: count_prob for c_name, count_prob in dist.items()}\n", + "\n", + " def predict(example):\n", + " \"\"\"Predict the probabilities for each class.\"\"\"\n", + " def class_prob(target, e):\n", + " attr = attr_dist[target]\n", + " return log_product([attr[a] for a in e])\n", + "\n", + " pred = {t: class_prob(t, example) for t in dist.keys()}\n", + "\n", + " total = -sum(pred.values())\n", + " for k, v in pred.items():\n", + " pred[k] = v/total\n", + "\n", + " return pred\n", + "\n", + " return predict\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we will build our Learner. Note that even though Hamilton wrote the most papers, that doesn't make it more probable that he wrote the rest, so all the class probabilities will be equal. We can change them if we have some external knowledge, which for this tutorial we do not have." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "dist = {('Madison', 1): P_madison, ('Hamilton', 1): P_hamilton, ('Jay', 1): P_jay}\n", + "nBS = NaiveBayesLearner(dist)\n", + "nBSL = NaiveBayesLearnerLog(dist)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As usual, the `recognize` function will take as input a string and after removing capitalization and splitting it into words, will feed it into the Naive Bayes Classifier." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "def recognize(sentence, nBS):\n", + " return nBS(words(sentence.lower()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can start predicting the disputed papers:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Straightforward Naive Bayes Learner\n", + "\n", + "Paper No. 49: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 50: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", + "Paper No. 51: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 52: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 53: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 54: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 55: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 56: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 57: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 58: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 18: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", + "Paper No. 19: Hamilton: 0.0000 Madison: 0.0000 Jay: 1.0000\n", + "Paper No. 20: Hamilton: 0.0000 Madison: 1.0000 Jay: 0.0000\n", + "Paper No. 64: Hamilton: 1.0000 Madison: 0.0000 Jay: 0.0000\n", + "\n", + "Logarithmic Naive Bayes Learner\n", + "\n", + "Paper No. 49: Hamilton: -0.330591 Madison: -0.327717 Jay: -0.341692\n", + "Paper No. 50: Hamilton: -0.333119 Madison: -0.328454 Jay: -0.338427\n", + "Paper No. 51: Hamilton: -0.330246 Madison: -0.325758 Jay: -0.343996\n", + "Paper No. 52: Hamilton: -0.331094 Madison: -0.327491 Jay: -0.341415\n", + "Paper No. 53: Hamilton: -0.330942 Madison: -0.328364 Jay: -0.340693\n", + "Paper No. 54: Hamilton: -0.329566 Madison: -0.327157 Jay: -0.343277\n", + "Paper No. 55: Hamilton: -0.330821 Madison: -0.328143 Jay: -0.341036\n", + "Paper No. 56: Hamilton: -0.330333 Madison: -0.327496 Jay: -0.342171\n", + "Paper No. 57: Hamilton: -0.330625 Madison: -0.328602 Jay: -0.340772\n", + "Paper No. 58: Hamilton: -0.330271 Madison: -0.327215 Jay: -0.342515\n", + "Paper No. 18: Hamilton: -0.337781 Madison: -0.330932 Jay: -0.331287\n", + "Paper No. 19: Hamilton: -0.335635 Madison: -0.331774 Jay: -0.332590\n", + "Paper No. 20: Hamilton: -0.334911 Madison: -0.331866 Jay: -0.333223\n", + "Paper No. 64: Hamilton: -0.331004 Madison: -0.332968 Jay: -0.336028\n" + ] + } + ], + "source": [ + "print('\\nStraightforward Naive Bayes Learner\\n')\n", + "for d in disputed:\n", + " probs = recognize(papers[d], nBS)\n", + " results = ['{}: {:.4f}'.format(name, probs[(name, 1)]) for name in 'Hamilton Madison Jay'.split()]\n", + " print('Paper No. {}: {}'.format(d, ' '.join(results)))\n", + "\n", + "print('\\nLogarithmic Naive Bayes Learner\\n')\n", + "for d in disputed:\n", + " probs = recognize(papers[d], nBSL)\n", + " results = ['{}: {:.6f}'.format(name, probs[(name, 1)]) for name in 'Hamilton Madison Jay'.split()]\n", + " print('Paper No. {}: {}'.format(d, ' '.join(results)))\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that both learners classify the papers identically. Because of underflow in the straightforward learner, only one author remains with a positive value. The log learner is more accurate with marginal differences between all the authors. \n", + "\n", + "This is a simple approach to the problem and thankfully researchers are fairly certain that papers 49-58 were all written by Madison, while 18-20 were written in collaboration between Hamilton and Madison, with Madison being credited for most of the work. Our classifier is not that far off. It correctly identifies the papers written by Madison, even the ones in collaboration with Hamilton.\n", + "\n", + "Unfortunately, it misses paper 64. Consensus is that the paper was written by John Jay, while our classifier believes it was written by Hamilton. The classifier is wrong there because it does not have much information on Jay's writing; only 4 papers. This is one of the problems with using unbalanced datasets such as this one, where information on some classes is sparser than information on the rest. To avoid this, we can add more writings for Jay and Madison to end up with an equal amount of data for each author." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "## Text Classification" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Text Classification** is assigning a category to a document based on the content of the document. Text Classification is one of the most popular and fundamental tasks of Natural Language Processing. Text classification can be applied on a variety of texts like *Short Documents* (like tweets, customer reviews, etc.) and *Long Document* (like emails, media articles, etc.).\n", + "\n", + "We already have seen an example of Text Classification in the above tasks like Language Identification, Author Recognition and Federalist Paper Identification.\n", + "\n", + "### Applications\n", + "Some of the broad applications of Text Classification are:-\n", + "- Language Identification\n", + "- Author Recognition\n", + "- Sentiment Analysis\n", + "- Spam Mail Detection\n", + "- Topic Labelling \n", + "- Word Sense Disambiguation\n", + "\n", + "### Use Cases\n", + "Some of the use cases of Text classification are:-\n", + "- Social Media Monitoring\n", + "- Brand Monitoring\n", + "- Auto-tagging of user queries\n", + "\n", + "For Text Classification, we would be using the Naive Bayes Classifier. The reasons for using Naive Bayes Classifier are:-\n", + "- Being a probabilistic classifier, therefore, will calculate the probability of each category\n", + "- It is fast, reliable and accurate \n", + "- Naive Bayes Classifiers have already been used to solve many Natural Language Processing (NLP) applications.\n", + "\n", + "Here we would here be covering an example of **Word Sense Disambiguation** as an application of Text Classification. It is used to remove the ambiguity of a given word if the word has two different meanings.\n", + "\n", + "As we know that we would be working on determining whether the word *apple* in a sentence refers to `fruit` or to a `company`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 1:- Defining the dataset** \n", + "\n", + "The dataset has been defined here so that everything is clear and can be tested with other things as well." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [], + "source": [ + "train_data = [\n", + " \"Apple targets big business with new iOS 7 features. Finally... A corp iTunes account!\",\n", + " \"apple inc is searching for people to help and try out all their upcoming tablet within our own net page No.\",\n", + " \"Microsoft to bring Xbox and PC games to Apple, Android phones: Report: Microsoft Corp\",\n", + " \"When did green skittles change from lime to green apple?\",\n", + " \"Myra Oltman is the best. I told her I wanted to learn how to make apple pie, so she made me a kit!\",\n", + " \"Surreal Sat in a sewing room, surrounded by crap, listening to beautiful music eating apple pie.\"\n", + "]\n", + "\n", + "train_target = [\n", + " \"company\",\n", + " \"company\",\n", + " \"company\",\n", + " \"fruit\",\n", + " \"fruit\",\n", + " \"fruit\",\n", + "]\n", + "\n", + "class_0 = \"company\"\n", + "class_1 = \"fruit\"\n", + "\n", + "test_data = [\n", + " \"Apple Inc. supplier Foxconn demos its own iPhone-compatible smartwatch\",\n", + " \"I now know how to make a delicious apple pie thanks to the best teachers ever\"\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 2:- Preprocessing the dataset**\n", + "\n", + "In this step, we would be doing some preprocessing on the dataset like breaking the sentence into words and converting to lower case.\n", + "\n", + "We already have a `words(sent)` function defined in `text.py` which does the task of splitting the sentence into words." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "train_data_processed = [words(i) for i in train_data]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 3:- Feature Extraction from the text**\n", + "\n", + "Now we would be extracting features from the text like extracting the set of words used in both the categories i.e. `company` and `fruit`.\n", + "\n", + "The frequency of a word would help in calculating the probability of that word being in a particular class. " + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of words in `company` class: 49\n", + "Number of words in `fruit` class: 49\n" + ] + } + ], + "source": [ + "words_0 = []\n", + "words_1 = []\n", + "\n", + "for sent, tag in zip(train_data_processed, train_target):\n", + " if(tag == class_0):\n", + " words_0 += sent\n", + " elif(tag == class_1):\n", + " words_1 += sent\n", + " \n", + "print(\"Number of words in `{}` class: {}\".format(class_0, len(words_0)))\n", + "print(\"Number of words in `{}` class: {}\".format(class_1, len(words_1)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you might have observed, that our dataset is equally balanced, i.e. we have an equal number of words in both the classes." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 4:- Building the Naive Bayes Model**\n", + "\n", + "Using the Naive Bayes classifier we can calculate the probability of a word in `company` and `fruit` class and then multiplying all of them to get the probability of that sentence belonging each of the given classes. But if a word is not in our dictionary then this leads to the probability of that word belonging to that class becoming zero. For example:- the word *Foxconn* is not in the dictionary of any of the classes. Due to this, the probability of word *Foxconn* being in any of these classes becomes zero, and since all the probabilities are multiplied, this leads to the probability of that sentence belonging to any of the classes becoming zero. \n", + "\n", + "To solve the problem we need to use **smoothing**, i.e. providing a minimum non-zero threshold probability to every word that we come across.\n", + "\n", + "The `UnigramWordModel` class has implemented smoothing by taking an additional argument from the user, i.e. the minimum frequency that we would be giving to every word even if it is new to the dictionary." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "model_words_0 = UnigramWordModel(words_0, 1)\n", + "model_words_1 = UnigramWordModel(words_1, 1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we would be building the Naive Bayes model. For that, we would be making `dist` as we had done earlier in the Authorship Recognition Task." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "from learning import NaiveBayesLearner\n", + "\n", + "dist = {('company', 1): model_words_0, ('fruit', 1): model_words_1}\n", + "\n", + "nBS = NaiveBayesLearner(dist, simple=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Step 5:- Predict the class of a sentence**\n", + "\n", + "Now we will be writing a function that does pre-process of the sentences which we have taken for testing. And then predicting the class of every sentence in the document." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "def recognize(sentence, nBS):\n", + " sentence_words = words(sentence)\n", + " return nBS(sentence_words)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Apple Inc. supplier Foxconn demos its own iPhone-compatible smartwatch\t-company\n", + "I now know how to make a delicious apple pie thanks to the best teachers ever\t-fruit\n" + ] + } + ], + "source": [ + "# predicting the class of sentences in the test set\n", + "for i in test_data:\n", + " print(i + \"\\t-\" + recognize(i, nBS))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You might have observed that the predictions made by the model are correct and we are able to differentiate between sentences of different classes. You can try more sentences on your own. Unfortunately though, since the datasets are pretty small, chances are the guesses will not always be correct.\n", + "\n", + "As you might have observed, the above method is very much similar to the Author Recognition, which is also a type of Text Classification. Like this most of Text Classification have the same underlying structure and follow a similar procedure." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter24/Image Edge Detection.ipynb b/notebooks/chapter24/Image Edge Detection.ipynb new file mode 100644 index 000000000..6429943a1 --- /dev/null +++ b/notebooks/chapter24/Image Edge Detection.ipynb @@ -0,0 +1,408 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Edge Detection\n", + "\n", + "Edge detection is one of the earliest and popular image processing tasks. Edges are straight lines or curves in the image plane across which there is a “significant” change in image brightness. The goal of edge detection is to abstract away from the messy, multi-megabyte image and towards a more compact, abstract representation.\n", + "\n", + "There are multiple ways to detect an edge in an image but the most may be grouped into two categories, gradient, and Laplacian. Here we will introduce some algorithms among them and their intuitions. First, let's import the necessary packages.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from perception4e import *\n", + "from notebook4e import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Gradient Edge Detection\n", + "\n", + "Because edges correspond to locations in images where the brightness undergoes a sharp change, a naive idea would be to differentiate the image and look for places where the magnitude of the derivative is large. For many simple cases with regular geometry topologies, this simple method could work. \n", + "\n", + "Here we introduce a 2D function $f(x,y)$ to represent the pixel values on a 2D image plane. Thus this method follows the math intuition below:\n", + "\n", + "$$\\frac{\\partial f(x,y)}{\\partial x} = \\lim_{\\epsilon \\rightarrow 0} \\frac{f(x+\\epsilon,y)-\\partial f(x,y)}{\\epsilon}$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Above is exactly the definition of the edges in an image. In real cases, $\\epsilon$ cannot be 0. We can only investigate the pixels in the neighborhood of the current one to get the derivation of a pixel. Thus the previous formula becomes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\\frac{\\partial f(x,y)}{\\partial x} = \\lim_{\\epsilon \\rightarrow 0} \\frac{f(x+1,y)-\\partial f(x,y)}{1}$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To implement the above formula, we can simply apply a filter $[1,-1]$ to extract the differentiated image. For the case of derivation in the y-direction, we can transpose the above filter and apply it to the original image. The relation of partial deviation of the direction of edges are summarized in the following picture:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "We implemented an edge detector using a gradient method as `gradient_edge_detector` in `perceptron.py`. There are two filters defined as $[[1, -1]], [[1], [-1]]$ to extract edges in x and y directions respectively. The filters are applied to an image using `convolve2d` method in `scipy.single` package. The image passed into the function needs to be in the form of `numpy.ndarray` or an iterable object that can be transformed into a `ndarray`.\n", + "\n", + "To view the detailed implementation, please execute the following block" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(gradient_edge_detector)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's try the detector for real case pictures. First, we will show the original picture before edge detection:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "We will use `matplotlib` to read the image as a numpy ndarray:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "image height: 590\n", + "image width: 787\n" + ] + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import matplotlib.image as mpimg\n", + "\n", + "im =mpimg.imread('images/stapler.png')\n", + "print(\"image height:\", len(im))\n", + "print(\"image width:\", len(im[0]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code shows we get an image with a size of $787*590$. `gaussian_derivative_edge_detector` can extract images in both x and y direction and then put them together in a ndarray:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "image height: 590\n", + "image width: 787\n" + ] + } + ], + "source": [ + "edges = gradient_edge_detector(im)\n", + "print(\"image height:\", len(edges))\n", + "print(\"image width:\", len(edges[0]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The edges are in the same shape of the original image. Now we will try print out the image, we implemented a `show_edges` function to do this:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9WW9jSXa1vTlooERNpays6a3uKtgGjL7yb/KP9bUBXzQaNrpt1FyZlalMSRQpkuJ3IT/B5ywdMY3yxQcDGYAgiTwnTsSOPay9YjiDzWZTH8vH8rF8LP8Xy/D/7wZ8LB/Lx/Kx/N7y0YF9LB/Lx/J/tnx0YB/Lx/Kx/J8tHx3Yx/KxfCz/Z8tHB/axfCwfy//ZMt715T//8z9vqqr+/Oc/13j8eOlyuayDg4OazWZVVXVwcFCr1aoODw9rsVjUZrOpyWRSq9Wq5vN5nZyc1PX1dQ2Hw9rf36/NZlODwaDG43Hd3d3V3t5eLZfLGo/HtdlsarVa1d7eXq3X6xqNRrXZbGq9XtfBwUEtFov293q9LmZQ9/b2arVa1Wg0aveNRqO6v7+v1WpVBwcHVVW1Xq9rtVq1vozH41qtVlVVNRqNajAYtOfxma/n8yz0abPZtDrH43Fri+8dDof18PDw5F6e5+dyHW2kHdxXVfXu3bt68eJFHRwc1HK5rIeHh04fqJ86+fvVq1f1/fff15/+9Kc6PDys+Xxel5eXdXh4WFdXV7Ver+vh4aEGg0EtFos6PDxs41NVrc3ZHvrKeO7t7bX2MD6MScpzOBzWarWq/f399hz34eHhoV2DPKiTQpvdPu5nXFwfdQ6Hw3b/w8NDkzX9sm4wTnw/HA7bd9Q/Go1qOBzWcrls+p5yQF8Gg0Etl8uOntM/X7PZbGqxWLS2jUajWiwWTZb8/9z3FF/XtwoBWaVNWHYpd/SL6xjPh4eHJ+OFPjEeyMi2Np1O61/+5V9qvV7XyclJvXr1avCkofUBBHZ1dVWj0ahubm6acQwGg5rNZnV4eNiMdG9vr66vr2t/f78ODg7q6uqqNebu7q6Ojo46ij+fz+vm5qaqqubzeTO+qqr9/f32G+Gu1+u6v7+vo6Oj9szDw8NW32w2a0rFYM9ms44jwUkeHBw0x2gj4RqUx87L19nJoPQInUGgvqqqxWLRcUb8RvF9D3XgPCjj8fiJQdmZLZfLzjPsgK2kKN7d3V3t7+/Xcrlsjp77uBalWi6XzWngAHn+ZrPp1M3vo6OjzljbuDDIxWLRnAeyvL+/b87AZW9vr+nEcDhs8sCIHh4eWh37+/udsbHBHR4eNplbtrShqtp9Lu6b5Uvb3N7hcFjr9bo5RZwXxp16h0yoG51DRxnndAJuv+szEEDuq9WqjZWdkXUK28L5Wsb0gb8p1I1sBoNBs8uqbTA4ODh40g90jyCHbTK+yC4DfpadDuzs7KyWy2UdHx83g6PhODAGb39/vxaLRSdaj8fj5pCm02nN5/OqquZE+E3EQ3lxJDQeoRC9uYa22MD4DoGgVDiK9XrdQSfcT3tz0Hguz+KHAUJZ+gYbhdrb2+sYBu1hYKgTY7QT5fo+w6LwXDtUHPd6vW5KR38JKIyrgwHPwmlYgfjMkZj//bkdoRWX4IKBoPQ434ODg3afn4vSz+fzJnOPC07j4eGh5vN53d/fNyO106GezWZTDw8PrT8gG7fTz0A30NmDg4MOyrMc7LBAOegmtuDgwxgbablvjKGdlh2o9db/M6Z+Fm2mLuRB8OBv6liv100uXG+5GG2tVqteZ8PY4KzcZ+xxtVo1BIotL5fL5nAT7Xfqf/abqvrpp5+aR6RhCPnq6qopOg6DTk+n03bPw8NDzWazhrTm83lTYCDzfD6vwWBQ9/f3tVwuWwqJwdFxkBaKhxBtgFZEp5UJl+2c/MNnOEgrIQPqaEq/5/N5UwieTfF3DLLbAuqqqoZCHDkxSKdC9JNxsUNECa0ooCfS6qotckPBHh4emhIZ1ldVc9ROiWkPyoaztNN0H7mXvxlnI1gK6CzHLI3Af+NgCK6MoQMVzzHCd7DKtthA6SOyzzGk2Hnz98HBQd3f3zcnkobpIGM581zoE/eVwAVdYx3O+9y2PnrB16H/2KnTyaQ7nK6iO+iSi3WCoG79QiamhfARuxbb73RgL168qOFw2NAQ6AYFwRDv7u466OT+/r4zEOPxuI6OjmqxWLTOoQQoMAZmSEwHN5tNSw3gu6gro7+f54i4WCw6XJCFNhgMOtE/Oa39/f0nENrPNiRP9AYS7DNS14FxHR4edmA4dfFjmTLQmZbiZHkWiCFTDdJD+gAyxGkvl8va399v/aeQ9riNyPY5h8MzqqqTqjOW6Aty537/b7nRV4yP+pfLZcdxUQ8ozwZGn6kTHUPnGE/SRI+FUQr3gDbM0/ahBzsOj3Efkkd3yCY8hvDQpO3YX/KUo9GoyQQ9Tw6L59upo9dwks4+PA4eO2im59JybJH7+hAov5HpruxjpwOD16p6VI7JZFKTyaRxYKCn09PTxmnw2cPDQx0dHbW8GPIfARqiuuEMPAYE/zWbzWpvb6/m83ltNpsnRonAgaT85vpMZewIGEzqw2iA1UQnlNXPMq9h5GWU4+hqBenjzFarVUNJTj/4LhXP8BzOiejF9Tg0p74ohw2WgmxJ622IOalhp+/IbqV2/bQTmgE9QuZOpfsmDNAV84QYN06IvlvxHbySk0Rf7u/vGwXisTEazfFwv13IStz3RFXIECRssp3nGPVbj3GSyBSdSV0D7cBLEshN4uNQUreqqoMaPRaMAc80eMmJJE+U8FxPNlCHbZbP+gKYy04Htr+/X/P5vHUUIZGnMtg4AfgHlGO5XDbey8VRhIG1ITunJwI5Qjk6UN9yuWxRiAKySsjsdjgdALkliuLaPkEi6D7Ogf9JpYn8tJGoaDn4GY56OJG9vb32mTkcjxmyTJjPOJnTYhIGuTrioVCMv50QwSadPoHJfTI/41TBqTroj346grs4dcVJ+nnJcyILj0um40bfOX7omXUiddD0AIgF5+I60SE7BJwlz6ra6i1ytw1kv+y0zC86KGCDfWm3ucLk0tAx2wLIl6yoT1agW/pOu0ndLZNEtE6Rq+pJKpplpwNDwVerVUsBE8FUVd3d3TUjHo/HHU7q8PCwzXotFouGpCzo29vbDkTG4CGb3UELzp/t7e01rsikO4KzIVES8eHQiIi0hz45J6cNOB0jLReUNQlOOwz+N1fH99SB0WP4JqXbYP63YrG0xUaGfFGibA+IhkCFQSOTTDeMWJ0KGVFn2pW8klPyvun2nDTpky16gFxsAPSB7CCNyrqcHJbrsbwdVLx8gRlOAr5TWWSVwZM60xHwTF+TjjoDcwY/zyzznQONZWEdSAftZ9AGgjDfeWYX3SUQPjw8NDTqe7kfAEH2wJjs7++3e3eVnevAmCUE1iE0ZgZxXFbkPmRU9ejk/LkHylzC/f19e87d3V1HKBjE/f19axd1elYyI1pCYyMxRzQbprmRRBSkGwieOjuC/e+60snwzDRKf9ZH7mYKRzvsJJAXfbATMjmaXIVTtZwA4D4UEmPCSHONkdfcpbHYMO3gnIbwzAxELn3yoTj1oD0YXl5r4/U4ZxrFd0Yt2R5TEuhnOiyuTdlmfe5D8oiUPgeUOu3vPR7OTFK/MxD36QfBIFNEtx8kxn3wjfzdh3gdNPqQc1/ZicBubm5aWrVarVqqQUe9PseDzKwU3hyI7uhlfgBvy+B6kB0hPaXr73iO4a6d5GKxaG00DM/U0jxMttNRw8XGzuBl3dRDX+GV7Egd6WmzFTeRCNfxPNIEjMkDj0Owg6I9RmE4NpPcnvH1dU7VEq0wrjzXjpT7E2FWVUPmfZySi5EccjG36O+sH5Zj9od2mpg2Kqd99CNTLcuaa5BDOizzP/zP95YL7UfO2JAJcAdIAwGnYxTrIfXYTtwnfqfTt77YaWWQz+tpm/Xejp/nY+dkbfnMLDsdGLNUDw8PbR0XC1pBPHBi9rA4sKrqpB809O7urnUCGJmroT2TZk7EcNhpEtfTJgYDY2FhK8J0OkAx6ey0YReaovSlj6nMCct5VhajWEd9f45zYNCd7tM/Pwe5AdPdVytiprnphHBwJm4tA+qlXZm624BMavOb2WX/ZGrJve6HESd9tZEYbVjmmXo53TMyNAdo/eGe5LbW6y3JTp8sD5AOBDvf2zm5mHIxr+SshvTVAdr9M/plvNxv64z7w8SIgzFysp6iG3Z82C1Lpaqqw+M6XaWe0Wi7VrHPrlx2OrAvvvii5vN5zWaz5swYBKaWR6NRzWazlveS0xOtptNpGyjSv5OTk85gOsofHh7W/f19TafTzroxZkPMsSCIqurMyBiB8RvI7FnJvmhnRXJxvVaujFpuk9OAVMhEDI5gXrZgDsTpWN/AegbuuZkbAoUdoNNl/uY3DpGx3dvb6wQLI66qerLlKPk6pyiptMjTxmSD8no6y4vn2+G6TXZijJHlS0GnPV4OhOkQ7Cxoo/XRjtNZhX8zM2enkw7InJ3XIVr3SAvpN4jc8nNAJSuy04Q/pb8OnkZs5tecFXmBthEtVIMDVlIRVVuwMx6P27q25/S4PXPXl7/88kudnp42Ah7FZ5DxwhDo3uYBEoMQpzMoIaQt6Gg8Htft7W2LIldXV01J+lI6BGEjgMy3t/fAQ8bboBhoBpW29nE0fm5GbXNpoBorjQfaEcfX8L/X99Bvzyr5Htq12WxXkzvAmLyl346UOCaUxuukcFh2EChY31o1xtqLFLnXq9f7Upw+TodCkMtU0OOLLhplWO9s7DYgxtipNcZj3rWqmu7QB/javjbbyTyH6u0MjLxMITDmHlvaAzlOe5yi22Egd8sfQGA0avohsw50ygjQ/Ch9tpPEB5gygS8zWjVgQE7o0K41YFUfcGCffvppcyTHx8e1t7fXUhXQEmV/f7+lkTguowEadnx83EnLbESsVQEqIzwvejR6ykiPIOz0mHxgQOfzeVssas4hyU4/w7Oqvt5ciqfNiUL0pS+dsAEaTdkZJEeR7U15eIMsdfv3aLSdLfMOC773DggKK7LN43gngbkuO1iPY6LU5CCdbtipePwxbgcK/+9ITl95RhqpZcn9zGAzltZZ2uTZadprns3pc5a+oGfDddrqtiILxtiOHwfh+rJYh2iHn4teJidHe422TNX4fhAXumkb9FIf/IODt69zYLRz3lV2OrD379+3LRer1aoN4Hj8OONl4fH7/v6+PRjB403NUSTPgyOzInggud45vwfAvAglOQKuz0F97nr+N6eX16dR2tD7hJ/T388VR6jxeNyJmCiDURkKhgycIvYpvJXQ8nB09304LaJikuDuc59sfR31Wx+qnpK1XqDq51FXH2rDYbrv6VBs7H4G36Uz8Ho3X8fJEbTPqNNBzgg05UX7s41OXX0tz+azvu+zH0Zjz8nCsvSzJ5PJk7Hum6E3LeC/7djskI12TUkQANlhsosOqfqAA5tOp61xNNSzUyZXSf3oGJ9VdZc49K3ExQNTh1GHCdQ0Dl8LN0OEzZTOKW9VtX2cSST3EctJVrqYK/B9nlEz55GkMYNpDs6IxcaY0diBwxuw7Rjov1EHMD63IOFUGBsbZcrEgclt5bdlD7dhxadYRjhSG5mN1Gm4n+dUMfnC5KmyJOLMsfL90CWJ+h280nn3PduBgfqwERyOEVdV9xQK2ma9zGDpdBTkxGeWqduLzLIfyDSvR/eHw2FnLZuvgQOj/r5ZbVAXz/EJHw7kfeWDyygmk0lTdq8FciOGw2GbSXFUd2Qaj8edtAXOoS/PRfDeHoGyOiXAwwPvnZcb4hoNUh8zqQjHgjVH42iSCJK/06n6M1+LItkJmZ8yemLw7Iid5vp/OCsfSeTgwv1+rsl8r8x3muyx8MQMwckIog95sfjX7baToh9G5HbYicRzdTvf8zncXqZDzxWjzhxn82R5T3JARmUYo4OVuSc/t6radjm+R+/NHVNAJEaC1rPxePxkpTv1Gpl6bPtk6oAC0vSOG+qlzw8PD53TLJJysKO1/GibU3/awGL2D5WdDozFqqvV48wi6SG5qhctzmazur29bXslE5Ki8DiryWTSlAtin0MRKScnJ03QPuTNHFdCYxuSicHVatVmNI3KzDVxPcrTR0o6VaPv5lX4zL+NqHzAH/VaSY1qnouOyIR783QExqmqu+aIfvrQPiNSHCEBwaklRkVgcuo1m806zswpTk5eoMxOgb3mzUgMFJspBEaW9VZ1T+egPUZTRjQE3eQSkYsN2ZkAupRjSduMMD07bkRopGq5Ge1Z93D2dhyWDSiOXRTm7tDlPookERXtRQ+ZwMMmjLixIe8m4Zgi2gVYMZeaMvJY0X/PxO8i8ncuc10sFjWZTDpC5uE3Nzct2rBQdL1ed7gTH7VDw82ngOb6TktIJU20kp1nQPtycistdRgd+hmkOim0Pu6gahthMp3I62hTKqHr7UuRq56iODuz8Xh7EJw5BjvPbBOIOuWNsnkPLEqeG41tCE4faB9t8GRMpgJGsfSd1LQP/Tw3Fs9t43JJNIdxowvmF2mrEYv1ij6h29l39y1lZsdlotpB11yZ0//83AHIDi+5NlMticxTJhT3I+XtdjoY+hmWre2MdqczpE2WG4vbd1EAOxEYs4IgJCLRYrGo6XTapkgdUVerVYtsw+GwkYA5Y0TquL+/3440Ho/HHYfpweYZmaZWdVcs02lHTaM0p4aGuZm3JyLgJ5FfDkIihvzf0Sf5NKeuRg5GXpnGcjRNwm2e48BhBJlbhqw8PpPNzzLqSkNwMSJFeftSqExzUlnT8bk9li2y8hh5vDNQ2WAsa7c/xyXHCv6KdrndpNweK/qGHSWnm07K97mtFC9v4XtTLfSTttkmLEOjSn/v1N7BnO8TwduGnMYnerZDsxyYELEjRM6/G4HNZrPa39/vHBHs1Cv5CmbKvCePTgwG243dIADun81mzfmZ33JkrNrO4GWEcZTw1LZJaNIjp3wZVe39jT4wXsP4VDa3xZ+7rqquI3ZxKpB1u2QkM+znefl/8hHIyVyI0YOjuSG/z93qi4xWfitpOiBKLmpNB9d3br7rSTSbz8GobGQ5Ns+No69P9JYcUaI29kJmitiHOuiHg2pfpmGnn3rovroPu/qSmYptzeiTa2wv2SZk4lNMKLlPMreu0V+fQYcTxk+kE3TZicAg5qqqnVtvbiUVnUGzV2dw3r17165LJ5OHtXnrhetK3shOAAXhhzoQlFEK3EwOvlGbHYl5C67PSJeoy/yDI6+NNFOWdKDun1MMGwJ9y8kVBwMHnoz8NkIjK0N9kB4FmdBGnxnntqUR8zy+s94wJmkU1iPzPGxrg49KvgcdyP4YsaE32R6PSaJ1nuNV40aYXinv8UxS3lSA9SvRHO3tc/TJbVrWyd1xf/KK1l1f55lDozyyLssReyYby/od8DwZmKc8bzbdnQAfcl5VH3Bgl5eXHaMfDB6PvGDz9Wq1assfcjq4apuCDofDOj8/r9Vq1RayGhkx6HSMRbMoTA4UMyqOSgjUkwyOlMfHx3V4eFij0eOSjcPDw86xPskn+H9KThTYiVnJrdjmDXE2ri+dsvuTKRPPof/Aa7jIyWTSOwNJvTh1p+Y+GgVUTKACIfWhRZwEium2U9br/ul7HB399ypzoydWxPvdAfQZrsxplh0/MmLBctV2xi+RBA7JsnawsWF7ZtVBgOfzbMvAxm5DzfGxY3XweQ4J+bDJTBEZG4AGfUu9SvTJc5EtZD798Nl2LP716bYEDvcJ30A7Qf99gc9j4xcJPVc+uJB1Mpm0xrI/cTab1XQ6rc1m094W9P79+44hDQaP+x5RIKLk9fV1Jwp7zdHR0VEHrSAAp4hGWiZDrQjmYBwN2WLkKO00kYFHYZN3svK7XSiP0QP3JTIzCnqOIOVZVri+wwtRqGzfaDTqcIn0JaOeEaodFs8yx+M6jFq8LaQvWvZxW/6sD4EyHjzHBy6at6IcHR09WTu1Xq/bxBJt90tCCJrwfSz29AyunTTBOsfZaNwOwfpX9TRjcdrE59Yt6uAZ5uGcdlFSj+zQ0sFyfQZaZOlA7HZxPdfC8zEmTg+9AD4zBsYXe/T5fp5Rzy1HfWUnBzadTtv+RIh2hMDx0ZzA6uNiOSfMKcn+/n5HEfhNYeM2nextrHgEpz728HSe/z0YLiYoE2VWbfmZ5EgywuXgunCfEVXW5771tc/GZBkYQfXVl840r7HTxgkyHpZHn4NxnaYZ+q5xX6zE5nrMddmpPier5HgcWByAkm8hc0g+xwaeXFXKLIObuUjrgumHvn5VbfXCKSjXWl/62uSATjv6OK8cAxyp5UWbq6qD2CzjlIO/T10jC+gLarSLtBMQZL22jPv0yWUnAru9ve2kV/a+eVIi6SKNzGgBwU/Ewzi8Yp30sA+hIbiMhtRpwabHp91WCq7zEc+OnuYMzC/RFhy5I1pGZ6drqUy+LzkDUs4+XiYVlTU2FIzB6JI6gPvZFhSa53kG0u31SvpMnxIZuq15tlnyeEbdOKOsl3sSVVRVJ/1D1qPRqK37o3iNmI+s9ro/1ivyWXKijHO+bgwZWZfM7xkd8r3l69QfQyaFQ098fZ98kKWDkekDxtYUAm332Jg/s6Op6r7iz6jcxRvUE0WanjGVwGoEbJvnPBcUKR88kRWhOPVCEI6+9ugMBo3gEL/7+/uaTCbtdNbFovvW4PF43IHwNiq+J0pagTN60Q6ndnZOTlEw7OeQhnN+95P7kp/y/1zrGVs+x/kaotPHqu4ECnXz28rKb8bI9RoBmIdz34zyCCq5qyEjqTkytw/DoF1MjTvtNGndJ8M+xJyo02lhIl/66edkwZF5HDL981g6LU15+XOjSooJ7HyGeUCPIbb2XCaS/XcxGKCNfcjMIMF1mFczaqadpiCwce+T9RgaBSeKNK8HCjPa9MtWkoN12YnASA19SoQjXJ6dbljrwafROAKnd+Z60jERUe3x8eQWrpGVnZMdXqZ8GR1opyMSqTKRMCMsA+EI5etQWNdrXscDSTuIQH5W34mdNpg8w8m8hNsBZ8ZYpDKRUuOc0jhyD6WdJs93kMOBZoTOiRn6AkVhjvL+/r7tBuF6SsrOTpHn4JTzPvrJ+DCBkc5wvX482txjYELabcjgapSeFIKdUKIg6591hud7TCl27paTbaav2EEZeVGHj0bydxyf5e1+HncKzo1+g3KzrzmetOl/ReJzuCCppBVruVw2sp6CYqKIfMYPpGpGMIoH1x0yJE0nk/daKE5LvGHUTsDRBqHZELgfg37OkIx0HFHtyM3VeIO76zEqMwdStT1NwgSnnbNTdQeWjLAUy95biYxO3VecFw4z992xjcTyt25Q6IeDUHJgOBP2YGbwzPIcmvV9vtbOrS+V5LqqaiS/P3OAsS4k92Mds3O1U2MMvZzI+ml9f+5zZxEeE6dslnPfrKvtjns4ydizkVVbPUK+BIBMKb3ecLPZzr46hec7nD+y+Z8cqfPB90KypIHD7qoeHZvfLmLyLfe+cegbxkFhYgDICu9CneksxuPHAw9tmJREAjZ+Cw+Be+A4Kz95KRQuuQuKX3ji+vI6O3CXVGiMJHk6L/WwU2UtDs+08+B7vjMqsDNi2xcTLKQISX6DUJClF7Q6FfHsJW32LJSdi9GiHR4ONz/fbLYz2Xd3d52DKH04JgXnlJMf3MPYMQFklErBgRJ8jIAXi0XnjTqWrcfLY2R52UmkTjsI0AcvgeHz1DWebTu0jmXfzO1ZTknHZGZUtaVWfIKvA5mvd1oKp02gpG+0Dx0iyDx3lFXrx7PfVLWNnBiEp6FHo1FnxsKeHmWhQSAoKxEbt3Og7cTMZaxWq85O/4xk5rb8WjHanfA00R0CdHTO9Mi/OUU2Oat0VE5ls9iAmPXMaNOXGiW0RkZ8lsS9r6FOf59ps1dU2wCdblgWz/EwvravJPrFiWQ6lJ8n92XHTkkeiutoE/phHg99zUkhczMm9rPQRu8wSOdmGfEMc5aj0ejJ3k4HL9MjTmeNxPx/Ik/bIIEJNIQuuD1eKeCCXVlvrPsGLNbrTJlB9M7Y+I52/W4ERmfhDKjUHACF19D7wTgO8wAI3g7Lb45O4/L7F41kPEDj8bjjDD0IJhG9gdkDyf+gKvMOTg0pyeEkFO5LcVLpURAbkJ1gPtftdjDht4/H8efcw/ONSjE2jkNhbIiUPBOjYBW2nRxcSHI9KbMsdgY2vkTDNmz+x1kxY0hb/VJl6gdFkZI4OHl8q7bn2O/t7dX+/n6bfMLJ2pH0OfTxuLuX1+PB96ArE9TJCaczdh+RsakKZGKeyqjL6bIRK22yPNxW5AgKst0+PDx09nsmJZIBJO0J2/bsK+1Fdn2HKmTZicD8QCrHY97d3bXV7jQYA+R3enUPljtpiOoOemBwVI5cjoqgHEe89Pref5nkqNMKR6CMcP6dXB198v+OdhkN05D42xGpL5J7KYoDi7e3+Nn0weewMZaOsLkeztGSzzIgeDU1n3u8XJCZ0bVl1TdxkKm25ZZGQfGWloODg3r37l2nrbSddjNmtMvkNLppfUzk52DB0UKpj0lxuFjnn0N3GZx5dupKttF/M8bpGIza+1B1Xx/SsRjF+X6j16puIM4TUYyASVE/dCbY/+hEVufbHlSUaLPZnhufjTXqsgEYMZmD4TsvuAMyo2A4A5SKLUImAM1hYaQ+mykNNAeN5/lsoyx9aYGRZtXThYBGaHawqWhZj4u30IDG7Hwo3tJS9RSVVW2NHf6t76ww6vX9ufshHYK5P1MOoIm+lPbLL7+s09PThoRTHkZYlpGDHwaMM/zll1/q1atX7bP5fF7z+fzJIXwY1O3tbXu2qQicoYl2JheOjo7q8PCwJpNJnZyc1MuXL+vLL7+sP/zhD/XZZ5/VF198UZ9++mlnEssOM7MK5JhjxZjndTiTzHBMrVC4FvTExnPuZ7wMVsimMrPwi3zcFusK44HjtPM1wqefyZfaZ/SVnQjsxx9/rG+//bYtLBsMBnV8fFy3t7d1dHTUFgru7e21tV2kkjg6D5Y3gvoFFPBJHJqIAiXByUwNsBa4nmlLojgGhGtJhxI7BQYAACAASURBVMwtmetypMT46Mfd3V1n5X9flB4MBu14oD6eKTmtJFkxDuo3csn0bLFY1CeffNKUk90RJmapm5XoboP/p262i/E557U9d0aaFZn25jlRntzxixvclvv7+zaLys4PorSX8xgtjMfjzktQcbqTyaTNYk0mk87xM5vNpukr4zccDms+n9fR0VFdX193jki3TDwWrn9/f79NMt3e3tbV1VXN5/O21Y46Pv300/rpp58a7cHnfVwl32ETuevBpDeoCvlZ76jfiJ/rGIdcRuL7fe6/Sy7wfi5Agn651hOAi8WibTfkedgN9f/u43Sm02m9ffu2Dfzx8XHzspPJpC2XYAbx6OioIatcZ0N0MO8Dj4Hz4lgdBhJBcI0dEwPHKROguKp+lOhtUNRr0tUFDsgowJMC3J8DaviOcTjlwyFZoRKx0kaCg1EoTteRdjKZPFn0akTHW5n6NjLTb0N5Ip6dc57qauSNPmBg0ApOTZ1241ydlrifRgw4MoyLWUhPzZsnzTQF1GRODdmYHuAenP54PK7j4+OOg7c+MHu7WCzq9va2bm9vW0B2n1ldfnJyUuv1uv7617/W9fV1/fDDDzWdTutPf/pTc3hkHegHssZponOMIzrmoMdnnBHnNNaUi20jl4ckb7lerzsOhuuchjrdQ0ZG833H6SToMIq2o+vLQFx2frtarWo6nTaF8QkUoCauI0Ixc4nwbDAoJ9EQJ4JiJxnObwRC5/zSUKMcnsFzPavj65xmGSkxYNTjNJOSZ1RZwCiEnW/Vdr0Un/l+2um01vyOOT8+x8GYf2HVO8VrtjB6w3hka7iPk2Ua3MrE/Xbijuo4MrjGHPM+bjD5Pfqe6BF0lMbXl9o7pfHxyl6zSHroZyMP80hGMtZ1xhPDZjmQj1LH4cGl3d3d1fn5eV1cXLQJk8Vi0V4cnbyeZZLy4n/rbKZdyQ/iYL0onHt5DmPoWXujaRf/77/7OExfY2dmrjD7TFAEhT9XdjowKgI1GR1ZkebzeZ2ennaWVeDsSBdubm4ap0bjq546AP9N1K3qHmaYkZPBBDnRdtpqktIIyMVpHM8G0tqR2TnzHORBW9wXOwHXzzMTZbmt9CHPR4O/wPkYyrs4FfV19AXei+fc39835OGgkOkAdVsXXGgbz4aCAE0kMW90bmSQqZpf0IKTAwmBCBhvk8pGKqRZlpGNFwfsCQ6na5alJx0Wi0Xd3Ny0e//4xz92TluYTCZ1enpav/zyS5sAI3sAKDg9PTo6qvF4XL/99ltL0a2XTtctw9S/1Pu81g67apveMbZ8brSVjscBLTnfdGBJiaCjbiv2ANL93SnkYDBoZ/5MJpM263V/f99BFazYx8ujfCgEUJqUxXwJL8ZEQQzj6YCjCErTx5exOLaqu//QENw5O3878sI5pJIwOCZ2q6rDD5hgzXTISMXRMdOAXI3thX+MCXX69AiMFydj5zUejzszw4yB03/agzL7eozaM56ejfSSCsbeiMUI2zSAEbdlilNi/GijeSmclBGlN2+jdyB1zvFaLpctLfXY0i/0ijbSHvNFlqmRMgF6OBzW69ev6927d3V4eFiXl5dVVfX69es6PDysv/u7v2sLqDm92OnhfD5vu1+Oj49rOp3WbDbrbN3L1BFn49e+JenOcyxvB1TuAdkTLGwzznD6shZs2X8bLTuoeKyyHVWPmQvyea58EIFl9B0Oh81hkbbk6+dRqpubmzo9PW3KxObt2WzWuJK+QUFwwGwMFeK1qrsA1fyEoxKDZOVD4MnlYDR8lwjLxD7ludlJrjd6ozjfxwjNMVEvzpqB9uQDs192KuaJ7LgNzz3BkXIARXtyw3yl0wgIYKeAKCXBxU4RA8C5+R2IzEoaSYB8U6n9khhm0NC34fDxIExPwcN7cj3tINAhQxv1eLydteYeAoNfWOM1e9TrID0YDOrFixe1Wq2a48IRL5fLTto4n8/r5uamxuNxM9jPPvusBoNBc2TT6bST3r958+aJfjJedjqWqbMeo3sHUuwnZWKdIfit10+XcVA/bcWRJc3znO3wPfV54qmvfPA4HchI3hFpo/Eh/Dibh4eHms1mLdLxGYoCUU9KCvlub2/FITJ5X5QdHkLEUdAO9lxZUOPx4wmd3hfp6MoAcZ1L8hPJSSB481iZDhqZmPszf5GcmGF/VbWZV/gM82Buj98EbX4NOVjJuJ5UhQBkJ7+3t9dSQQwxSfOmVP99vxdfVlXjg9xHp8f8Nj9aVW3pg/fEemsNr3yz0yQFOz4+7siVVNScIgEMXYYvG48fJ5d46xN9ZTkHY8J2I+8BPj09ba8ePD8/r729vbq5ualff/211WPU//XXX9fR0VFdXl7W119/3caWSbT5fF5v376t29vbWi6X9cknn9SLFy86wS11kVTeKbllkcE4dRq5OI0jC6MYIZlW8GsUsX/oBNuGg6ULwb1Pv1x2IjCEa+IyPT7XVW0jLMZjKA4flhuAzZslr+IowbQ6kSyXQZgjOjw8bHwLioUjYFXxev04a4oi+pnU55kr2mdH6ZzdUc5IkL/T2XMf9dt581r14XBYt7e3nXVKpO9EfgwNx418bWDm2aqqEaMYL0rG3+YejTwxCsYjFYv+YTj5Ehb3n9TUaM31ZFrP2IH2efWbjdMBzTpbtSWxjQSsx5YlKJaUlDrtbEFS3OuUHN2nDQQJjlXnxTbX19d1d3dXn376aV1fXzduc39/vy4uLmq5XNZ0Oq2Tk5P65Zdf6vr6uiaTSf3888/14sWLphMnJye1t7dXt7e3nUkU2ydy93HwDu59kwSU+/v7Dp9pWRmZ43QS1SZx73Hy54w9ss0Mqq/sdGAXFxf19u3bJ05js3mcvr+5uenwVTgBiPSTk5PWEAvFXASKVFWdNBMDZfEgjsmDQV2LxaKtt0HprWjpdFF6H+lLfeYWMhpVbZ1ZrhOycdqIXHBuSWBnQKBu1nXRZ9ARhuN1VjyL2WE7dORgR+10AV7z9PS0XYNBuj0YMH0zmetUtmp74oQdfa4XQ6dyv53Hy2SxdxIwZugd35n8x7i8dpA+8wwMMhEKz/YEAwaLbHGK1v3keeGBva6NCYbpdNqyFOqBWrm9vW39QP4vX76s5XLZ0NpsNqv5fN5S1MlkUkdHRzWZTNpaNgcrHKv1AIdmZ2en5CBQ1f8OTuszz3K6TZ995hcHRBiooA9pj7sc2M4U8rfffquzs7N6//593d7ets8PDg7q5uampWlEUVbEk6ZBPFZt824i78nJSVv/glfmWrgopyF4ZZxjOjEmGfjOu9gtAITrejBmOwKnnzy/qnvuvK8z0vCzuBdZMdhGb/xG6b0oE0eFkZNyg+5yDRgol2fA7TmFJY2q2iI13vRTtSVjDfetVIwZz8CwWajclEtOEGOu2gYDnudjbMw/GqE6gJgyIAh6lT46hvL7NFLSQ6fUpIVcm4GJoJGcmwMS4zKZTFqwub+/bzOT9JVTRzzuBwcH7dSXzWZTb9++rXfv3rXFxOgQyG2z2dQPP/zQaJjpdFq//vprnZ2d1du3b+unn36q1WpVL168qPPz8zo6OmryoJ+5di77mycR+3sKji4zFoI/tlLVPU+fFJNxxXZA1IwhiPR3L6O4vLzsvKyD4kWqnjFCmT1r5Qhs5wAhj7J5QI34PJuIQVGHDY57qev09LStHTK5SuEFInZG3t6EwM2Tua0WdBLmJoxzUK0IRq8MlGf67CBoR86Q2SkYbfm5yA8lY5bLywSYxk/kZA6La9kL65cxOOXDAXK9DYEUFGX2rKkXTFpfqKeqWiDd29ur4+Pjuru7q4eHhzo6Oqr7+/tOxoA+8J3lS6oGGgKdOHCZmwE1gCI9RqA8xouZdfruVM1jYzS8Xq/r7Oysk3KBrpjdxNYGg0FDY0ym/f3f/317dSFp6s3NTdMXeMvJZNJZeEuwM4J0m9EdX8tYeybXjsZrBj1+th87QtuQC/z5704hUSo6grBzGwZ8kmc6mI52LoxiApNRUENH7jNpb0IfIRiiUnwPLxaxQpq8tJHwPYPqtiJ8rm2CE7dl58X1dny+PlPRvmN0UCRkZ0NxwOA7HBOR/vj4uHNWFWggOSgcjmf0GA8fBZ3OeL1eP0ktHIC4FmPI6XS33csxMi21knt8OJsO50VfxuNxnZ6etuC2WCya86raHqFN3d5O437TfhbQOoCATNGlXNBp3s/9A63YkTP+zCrTLtKry8vLpuvehQB6Qfdubm5qMBjUzz//XFXbSQVOe8AJMCYsvHX70O/kZ20LHkdSRU+kcK8XpTPmVd3FvX1vIvKEBPbdl9m4/I+O03E6ZqdlLovIYO+KJ3f6g3HZkHivpI0weRtzQSYATSx6+tuCQwikuCh8cmB2UGxBYeDNn1VVm2XxYDvS+r1/o9GoM22Og0OxIM9xznd3d+1vFIdi9IKMGQs4IB8Qh8KYkyMKZvREqXCe5kKs2HZCFHNqOAmnkzynz3k5HfeSHCMdy8POE6dEXwlA6BaGdnh42HnxDHWZo0Wvjo6OOgtHU+e5zmmRObFMo0kbfZSOZWLHxup8c7TMPONUV6tVW15xdXVVt7e39fr1687eTB/rY44RXcUmjHirtjPFBDZS9hxvnBj1YFek9bYVp/xV3Zl000BuX+4YeK7sRGCsDTKpSgFSIgQrHhHk5OSkoTUUNjksKwkD6tNOiU5OwZz+WaDJy2CkpJB9DsSpS9XT7Q1uqz9z1Od/nk1dKBtK7dTKzt1paSpK9p02egYMWTltsaJmnXbipFqkkKRIOGgjCKeCKR+uSYRBagAKol19vIapB1/Db+qkn17UbFLaqJeAOJ/P2yygjcmODmdmhMvyEfQHRNc3yQCiM+Kl3UaK9IFreFl0jikAwbN6o9GoXr582YAFy0QODw/r9PS0s0cYu7NzxeFmcEraI3UO+Rph01ZkkDpBH5hYMh9qh52rAdCTPg4uywcXsjIoNB5kdHJyUrPZ7Enqw+D4RANIUARhISVacgch9hwBUHLzB0m2e4qfuiBI4c2IVIvFojOLZB6NwbFzdtTNwetDAXnaJMaJEtFOnsd9Nl6no1ZkDMMGm2+KypQMoxgMBvX//t//ay8ornpENm/fvm0G79SYdWJVXaQJUsIZgwyQlWfR6D+GRf+437yjuSlkgq4kL2g0l+m4lzDwco6zs7M22121TXns6AnQOBCQlM9iA61gfCx/2Ww2HY7JM3/IC96OAIqcqZ9AD3okaNE224hpkqQJTJAjdxA5z3EWwLjk9h76C7GeqV2CHDtvZwzOaniuddb1kEb+bg6MQUXIKLUdT1X3JE0iOANKjuvUw4pib2yvj6L689Fo1FmNTbEymtdxpKXtfVwUEWo2m3UQHzMhdgR+poWN8cGR2FkZETGojixEGoyWAkz30oBcg5XHlPg318Of2TET4a6vrzvLM/iB+0SRHX2tnBSe4QmfjNg4T/SDgONJE/gtEIv1Cwdh5Dyfz5sTNsq206adpHLv3r1r6aSdJtfliSggUq9HY38jxDhvTmIpEGmmZ+Ft9JlKI+fDw8N2LA/9vrq6aujSOkcQs9OwXVqPWDfHcVAHBwdt3Zjlbx2l/6y/XK/X9f79+6ravg3dugwadtClrRkQrRtca5kz/jn5lmWnA7u/v28zIMw6ofxwJsBgIDGK5XTAg+foUVUduF7V3cCMgjK4KJH5K0fOhKVA/SedjuiEgB11cEJ9qRJ1VG0VEBl46QDRmP6no0si1e2p6jpxvqcddnROCYzqSIlAajxzMHg8140xct9Go8f1SZD4dpB2XEkZIFN2WlhuufLdMoYvMbqyolufrFP007pjLhC58xynYaenp7VeP661AmWNx9tjg0DUnh0DBYFUuNZptvUc2gLnQTaCgXttFsjZa8Zot+2CIIEesaczJwqMXv28vb29hvqur687lAv6wMwmsicIz2azGg6HHR5xMpk050r/zXu5jpyN9VloGajgTz80A1n1AQd2dnbWBA9ywonYcVVtN/K2iqXsjoQoIo13+sU9DA5QHOXmepSD9UiktXSWvZp98BOlRlhOE1E4OwcG2CiIAg8D5+fI7T5zn6MjSpazkObSSEG9Atz/pwPCYTklwrkwFukgGbsc39y3SNs88UAdOB2iJuPpCJsTHXZkTt1znDCsPnSBE8DILH+T5TgdkISdx3C4PQYnTwb2eFRVB10gN/TW/eMwRVALTspOCB3NPoCsTMdwDh9yoO67u7vWV3TDqSr3W1+NNI2eWJ4yHo87a9Lo993dXW02m0YR2Ma4hsk5y9BgJTeaJ4eWCC3XDfaVnQ7sb3/7W33zzTdNsc/OzqqqWm6Kst/d3TXlsWICqYkwnFjBLBKpHwbh0xmtfMBx82I4lSSy/Xw+w3nQdpwVjgzj6CMjq56emMkAcL/vZUC9UJRi9FDVJSpJvZzGoLTmDYwyR6PHY1dIFVESz8baEOwknE67nSZwjbSqqo0fcuR55iD533LN9XUep6QT7PT5znUiX7IA2pWbkqnPbfOSoM1m09AlJDInsqacucdtt8ytR+Z7TbFg8EbL0C7eAsZYoKdGU94ORfCmjTiv5JOMIglyw+Gwrq6u6vj4uM3MozOJfjlZ9uTkpKoeucSqqqurq7bHk0Dq4I9+Ovjapj2mdmIU9pV6Rrev7HRg3377bd3c3NR6va6Li4v2ksuq7pE4k8mkoQmTnsfHx41n4QWlOQvJDAUOiQH3Z0ZmKFMKGoXJNAThUSeO17NQroNUgr9x3j4dAqFTcEImUNOxoTheLJm8iu9LPpA22rBBW7S1b90Rf6fjSgQKf2lE4NTIi1udnvWlfDxn12GRdswu1A9qsbPzc3DeObFiRE7BkHnmcrls/BV6jHNm6w0G6eBlBOgForQ5t0XRf1CzOVGCu6kEE/xepsFvr32kv+gJvJuR/nrdPRnXSzTOz887C3lBhpyKAQrf29uri4uLDsqz/dB/xp3f5mlNEfC/gQX984QHTtVv8e4rO93bL7/8Up9++mmtVqu2NwvBcfqqyWCioo8OYfuD07aqbu7rzts4QFieJLABorCcPMAguxB1+J77bDh2Rj7JAiXkt+/BWRp5wi+R9yMbIqn7RkSt6q74TqO2A3AqCKJ1imhHZ8NMrsj9pYBIHPE9PiDnRG6JlukDqNlLbSjmvrie+00eY7DMErOOijqJ/Ci/F/VafqtVd8ExPOV8Pn+yaZj67aAxKHOVyNuptPfw0nbSrqrtMczm43Aw3t9atSWxCWhGw76/qtrWPdAjYw6gqKqW+r18+bLOz8/r7OysHh4e6u7urk0aDIfDuri4qLOzs86WNQI4Y+EZV2Rtu7SOGWxYf7nGmY3XSuLsLfe+shOBnZyc1Pv371sjnA4Au2mID8+D5KTghHI2zgS+0w3+T9KaLRo4JAbdz7JC2OvbSOyIrYxGUTjNTGsc2b1OxY5pMNhuS/HyEa7ByFAM6jCyQy6WB7Jg5o0Xq7hflqWVxo7HKJHvQSC0P+XkfXjI1Olj3wRDciGURFiemcMReL2cUayDoBcTG2FzDanZYPC4zs3LEAiQIBfGIXlUdOn29rY++eSTDhqDUE99NnJC9pvNpp2D50mfqmrZCXrtpT4U2sZ9Rs84bmR1fHzc2sW2oel0WldXV61eNn8bieIcCWY+ogi7g49jH6YdjdNFj7VlZhRvrjuDHIExeecsOxHY+/fvW4TnhwcicJTMvAHrw5iupYMecKKNEUYfj1G13Xrk86QQBobp9NFOwIZvo0w+xsLlPiKQNxtTjOJ8vzksitEW6A4DMuqkLqNTOx5IUvNQOMOckmYGijqNct0GnlFVbSY2USwpV6YBXsVOO73i3+dj4fyRuVEKKYkdn9NBb4syb8IYIheegzF5CxZby0yQ39zcNPl4I7oXNhtp4JAdyDB+vjcvheOiPcgGFMZPktRs0AZZYoOe4TZiM79pR4feffrppw39XV5e1vHxcduu57PNaK/bhTMzj4UOQOzTX9sCBVslWBB0kjrAWXK9fcPvJvF5m4rTORQo9y5WdY9ENvSnYyamq6rDYaDAHIsD0khSvy8VynTHKMrCIq0xYiNSWhEdnT0w5gvoPwbP5+aWeF4anJ9L+3OdmtEPA08AsXJ4ywmycb0pE+Tsl7JUVWdNj1EYbe1LQ+3MMn3kPu4BlSOzqu45VdQHCqB4GxvpuJergMzNs+E87u7u2okoRnYsDeK3I/1qtaqbm5uWSnIdnCWOl3TN8iYNM8I1GkdOpPYOIjhcO72UEfK1I4Gq8T2evTNnNZ1Oa29vr53Jj5NwtkKbQYHr9eNyk9PT02ZnrCXjhSTMhlKoyyja9uag56VO1N9X13PlgwcaovBegGhCG2WBkDRJDUREQEk4e8EfEcrnjnFEMOuRcp0MCmcPjZKmMTtNQEmsVBYWCsbftJkdABY20dlrvqgrc3630w4AOYM2bPiLxaKur6/r7OzsyWRBErpWEOrjc08S2IEbVeaMj5eW9CkSyu9XwFVtSXrXz/1u92rVnQEEJVHQKeTr9VMYB4YEkiMQ8bO/v98ckB26xwDOxbwqhmydANGwhMHr37gehJVoxkjOBoocaQ9jzN9kMYwNW/OMUJGHJ4noP9whdkR7QW0gVIr7jK1z7l9VF3Ey5hzPTbup31mPnSrPzEkn5MAkhm3oubLz299++61Nn3oRJEpNh40gGACMEuTA/wjCJwFgtF7chrHhNFhbA6zlGV5LRt1wOeZ7Dg4OnixYRGlAPiYXbXAYqq8zXLd8uM/twklTEq2gLP7cZ25dXFx0vnP67FTF8uM5yMRr1QgCyINxNWdDGonhwrvZ+Jl1g1xGTn1KZ/QBgX56eto2rg8Gg7q8vOykY69evaqqarPfPse+qppugoa8kyANBqMyAqAfFOTo4IQxJYVwdHTU1iLStnROjC2OxJyVjdfUxmKx6CBUdJU+TafTjlOynu7t7XXO+SIQIxN0zP87YKxWqzo9PW0BBTl8/vnnHb1FX+zs0EMHUqNvbITimVdPrOAjBoNBa8suBLaTA2PNx3q9bm8WQrlM/nnAvFLYHBEHvdlQUAS8rfksK5WVzqiPXe3Ux9Qvg8T9FAva5KIJYvJ6c1Dci9KZR7NCWvlZHJmIgz7wXJPQoMTBYNDu59o+FMS6OTtar5czn4cDRtFBK6AE/qY+85f0ge0y5jFs2B5fox2npnyGI1qtVm1Ly+3tbTvDar1e18uXL+vly5c1nU5rOp226yeTSZ2cnNR0Om3ohOeAWngObccxVm0PfTT1wBjY4ZgyMFda9cjvTSaT5lTRH6PtDDYeS+zGzt7cMjI16sexoN+2FezRyzIoPB9UV7V9Byjy8ub9m5ubhqyRkXUvAzB9ZFLHDph2Uw/PR3dM89jhTSaTThr+XNmJwByxmIVgMOmsG0jDHMm5F8Nh5TOGRUrD/31kOZ4ZpSIVNDFrRGRH4nvtDIlGvt+KwACY/3G76LdTRwbD98PPOMXgf/cNfuvNmze1WCzq/Py84yCA/8ia/zFaFMMLdau6fBh8Jf0BwRkh+yUj/t7podNUO6aUmbkwzw7j9Pkuz2GDfzFKGQ6HdXJyUuPx9qyvX375pZMu+oBG9ImgZZRCffzmOqc9Od5uC/1izRTtN5pgrJJ7NE/qtXrJHVKsK6ZLvLaP9BT9zfVitJlTXKoe0/Pr6+v2HC8KNzBYr9d1c3PTdLJqu9TIARSny0LenKHmmvF43OHNs/3mC+24nysfJPFfv35dVd1pTaakIRDt7Z1mIXgjF6dTfdtWjDoypTKhn167qrsUwtHBPJfvtUIRhVar7fsEvRzE5CsRyegKpcy/GXBD6/yePr5+/bopkNucvA/yxZiRi9GloyVyof+WqRdqJv/Bs4zcTCjzmVee0z+UEMSe13C/jdhr2pKYtmNzekJ5eHhoa514NwKfJ2fj5RhOl3HyRog+ngZ0m6jJCzut23bUdvSgdf42yqJYtxxgGRscgJd7MMGDEyPY3d3d1fHxcb17966Wy2VbTsHaMS9SxhHZoR4cHLQz9pG/f6dOmJ/jf/SUsUxg5GuRCTL63SnkDz/8UOfn520Zg5XZAjdpR27t6OU0BXgOGWli28rtNJS6EG4qsGdEva4q00T+9iC5DqKpkVVG6PybduAQcv0bvxN++x2Xg8GgbYo9Pz9v63hMGlOMboxG7TyqtukT8vPSABP8iSp9Ppn3XHpLEbKjbvePa7Iddk7oCYhvNps1ngu0hFOjnTyPyOzUjNTEJ5C+ffu2MwteteVuNpvH/YoEYMbUskIWfG7nlVmCdW25XLZXjeV+3dSLqi53aueNTZkjZfwZw0xd7+/v6+7urqX6oCDQKU75+Pi4vvrqq3b8EUdxw5d5HOkzbWIxMTZtHUlZ2raxdUCC0SOBCufvNWkfKjsd2DfffNMiHxCTAfQhchCPHFWL0FnRbMXAkCBe+wYIZWAKGMdiJ2DCD0XmxRRGFWmkDKjPYDJaRHBOmxw9vT7G7ed+Rx8PIBwDysaufhSeiQuQlB2AZ2jsnP3SB+TA6bhuC9czfl7nBSXQd/pGX/rsfhsZ4EiyMDY8i+uM5uDViMbZZ8vRqb8jNEY6GAzauVXz+bzpBDtJIOUJeui1U0rkAyGO46PNHiujKuqy3meajoxtnJli4RzMo/Ec+EEHT/SKvoIIvS0IPYFbImg4sDvgoacHBwfNbuFBCTA+Ftr0CRwd7TT/SiDiPsba/KEDclIUT/Tr2W/qcSErhpY8h72o3+3HYOPYgISOXBx1YxIc2ItioQB2RjaQ5Lccke28INQd3fySDxylD6uzw/PhdbQVhTK5i+A928hn6/X2nCTSGdZ2QZY7RTAfYWiPXDK18jQ2/eY3bbLD8XIM2kV6kGuq+O36ab+fxXXmymwcHr90ZBj8mzdv2kuUOZttsVjU559/3sbJYw4yMNey2Ww3aROkvA8Vg2EcQW/QCOg3dERSOAAAIABJREFU6IKlD4xNLocwN4VM/HpAuCefk5cv0sDwzQUBEoxyzXviBGlTVdW//uu/1j/90z91HDoTJDhTHKOpEWyHvxlXxna5XLbtgVXVQV3IxbaXY4NOmA+0HvSdBEMw8nP6yk4Exrsf6SicEFyQIZ8b6KM1lstle8lsVT0xJtCGV21TktcxscwgG7au1+sG2U0K82wGKhfYGuajjOb1fGwIEcRpMs/yZIIdPPJisGmn7+E7il855lSYaEa6SttJG/gxV0ihDqek5tU4t4q25SQHcs6jk5CBi9Gtr3GkNkFvRAKPxfVXV1cNLbCq3vJ3tMZhMW6k2lwP4Q/qM88D+gKRmj4wlUF9XM81BCf2qCYFwt5L0nOQCnqVSBo54mQ9keZ24JRx3BwKmejZM6Ec0kDfnGbygw7gXAeDQdvXyRvLbWNul/tEP5yO9uk81zHOjN2ushOBXV1d1cXFRTsAzmgBBaPAgXF2UFW13f2kjfa8/O89YEQKIxzQCB01mZ9H7RqhEBFwgFYQb9Y2Z0UhUtHWRBl+lu9x31DwRDrIjDPoHR3twHH0TltQOK+hwwF7gsOK6v4hT0N698HIoo+zSSftIODrqcMGRPtctwMAJySYK2Mf7s3NTW02m3rz5k3d3t7WyclJ3d/ft7PhkY3RuutnsWxfuofDQk4sGEanTDVY3z3zi+MwwmYCwFwOOoATQLfQU9A+smVJkNdRpT56Iu3s7KzW68dJtvPz84a+mH1ETzJYABRMObAbgVX2oGT6zjh6wSzjnvU7m0guL4Ns8nr+rq98cBYSaD+dThvxTL5dtfWyXkFs5bDCQ9QBr+mAHRQDmySx0wSM1ttoDOlRAi+uMy9nB2VDN+eTMyCOYqlEtCs5kb5noJh+hu93HzyAdnJGgThSo2Onbk4n+c5Q3sZn+aJIPMfjaOdZ1V064eNbfFIHY0Ib/DkFtEnKwn5A6ry5uamTk5PmbF69elXr9bo+++yz1jYcR6Iyj48dkdMWdPbNmzc1Ho9rOp029M0kAYG1art3FBm4TusMcnJARdagJZwd+ko6yec4CsYajtDk//n5eX311Vc1GAzq3bt3HXLdNumFtUZJRlzomNeL+XvQH0EZ6iiDInUlxeExsM5uNpuGfpFBZmYuOx3YcrlsnteOgKlbk9dc6xy+arsdhQaRfjjFsCKnAJLI6zNCFxu8o5udnF9OQb2elOB+X2MlTD6Q//2byEk0B005ImVdNjrXZ3lWPXWWRqyWEcVK6tQLbq9q63CNFBPe9/U7n039fcGH69LY3V6+T66KZzML5kWmfis2HI/T/iweS39GkGO9E/yp02U7cYKGDzVAdsnz8LcdEK8ZPDh4PKcfBGbKwHKCPLeNWTaj0eNSHMs0ETXLK1iIC93jYOQglJSLwQZOkHSXDMxIns8tb6fiJvCpG/0wzfJc2enAjo6OGurCKyIEHu4IjRf2oIHImB1J53VwcNA4MqIhRmzkljyMZ0ZzZXiiIwbCvJQRHryFB7HPeewSJCUVCrLep4HaiBMVIB9/nugLhXQfGK9MiY3Wqrobr5mp47nJXfjaLJadZWO5OdhUdXceWEYgN/ps9Oo1cMyEUTdnot3e3tZyuWwryNHP8/PzJ0Ei++Ctbpa7Ax/6ikNlVg5nQpBiEihTPOqiX/SJTeHMHHM21+XlZedlvFXbBcp9XBOcGtu6KDyPjMNIGOfDxAYoajabtR0Oq9WqXr9+XdfX13V8fFzT6bQzNixbMVqvenoENJ970a63qjnzQB9t88+NX9UHHJgdFp7WCpkCJKUzdKWRQGHy8KrtiaImDau2bxByypmEpA3d6JAfKxG8D8oFeqROzzQZnrsd1McmXvqDseTAEpn39/fbK+Or6olzdwCgID/QKYQv93ON/0aWVd2junPfKv1KBEta4JXQ/i6fiVz53kHJXE7O6jpNNzJ1yty3s8PEL4HSlMDBwUE7WhqDvLm56ThATp9wO5AdY+zJjzQuO4F37941fgknxt69qu0Bgh5zEJTfls2qdztdOxWvSTOpbQQP2nfqavRK1vHw8NCQ6dXVVZ2enraN7l7H+ebNm/rkk0+aLn/77bftfqNR+sAMK89EZ5EV/WZW2fwhsqYNZEqMEc99ruyk+N+9e1fHx8dtILx/Dj7HEcVKjULgPFBGZhSNQOgI11hB6WjfOiUrI+0y75J80Gaz6RwiSBriwXfEcx1V1TE26uM0WJyjz/gyqjC6o02OlvANPIf+c6/r8KvekB2KbL6jj69aLpedo3M8XZ3jlxE1eTWcijkU2uGAY5TmtU600SlK8lVGarQDDof+uo7JZFLT6bQuLi7q/Py8Hfw4Ho/r9evX9d1333XQ3WAw6JyPlmkyz0Y2GFRmCp6gWa/XLTUE2XpSg9lEXkr75s2bTv9Ho8fjsk9PT5uecj/yPTw8bEteTE0Y4ZozcwpqfWNl/vv372u5fDz37fz8vC4vL9u91m2XzebxtFkDGQdxns//m82mrR2zz4Drsgwpu/ivqg8gsOPj44ZCrHA+jZWGez2RZx2dLzuCOy2q2jowH6OBYjqKo/wm091JDNj8gJ3GeDxuJ2PiiEFZCBBlyHY7Gpvnof2Qp35xCU6zL611n1y/5eJtFvTHb4ry/TgjoxU7G/63kpioh5/08+2ISbXdJsbMY2MEnKjN51f1pURE8eR7LD87xVR89GU4fHwnA+9sZEzW68d3G2JAe3t7dXx83HHilqtTTOgQ6wV9XK1W9f79+9b38/Pzlg5NJpMO5YIT96ZtApTTTGQLLeAjyK13XmPowEYfcz8ryO/k5KRubm5qMpl0bNpvIbq4uGjo0EEJpGQ/gHwANubhQH+0tY/3ts9Yr7fnuj1HY1R9wIG9evWq/vjHP7aji+EgvBGTxtiofD4UgvZ79Wg0SsxsJakonXFdTB6gpGw5yfQGxehzOkTOTBOqqgm36ikH5pQop+oNjauqGYOdnBFH5vPpUMwJ8Zvrkrj37/ysj6h2Ks53tI9xtDLRdyOLqqfLVRhPO/eUYZ8jywKPRFpY1T1H3mgMw3AKQorF8dcggPH48ZhlCH4iPvr266+/1tHRUTs7zC9wtsNy3ywHIyt0/ubmptENLDlij+tisWipI21ChrTPzzo5OWmnQ3iirGq7W4C6jArdNnQRRwdq4ogk2m/C30EkJ+fQJ5/T51NAsp20DWSIbXqJCDwnIInx/90O7JNPPqnFYtFeweTZEZTIC81QFiuvTyRFUE4RiEYIxAZBfdTjaIW3NhIyr2Xy22kUA8RAUogSfaSvDTIhrdfA9BH/dhiWG39bIZzC8HnOSDklpS+Wq59TVb2fZbtANLQnl7nkBEJVdbg1TyogZ7eVa/y/73dxGp1kvmet3DfGBVl5fFnMav3DSZDqsX5qPB43Up11UMlZpgxdQI6k8AQzZjJ5mSz6RFDG+fs5Rrm0jYDul46kTmba6+LJCNphPfCyDduTHZGzmqpqkxm5Y4V7sI8cO4K5J25wVg5GXnbSV3ZyYEBHc0EgGGAiZTTaHsELvMzOApGpj6iE0bDuzB7X34/H45ZLAzEp5ohcUDLSB88c5WCj2C5e7+aoiLGnEqMYmV7wtxGU6/Lbe9JBJ9flsXBqQL3J/SGXRET8JC/X51iTd0NhUTAU1TJPB2WZoANWTgyfa5yG2Jnlmij6nQiXdpBSwoWxLebo6Kg5LF6g+9tvv9Xbt2/r7u6uXr9+Xb/++mvd3t62WW+PBfLweJsDBJmw3xU95n7TJcxswm2BwvyiFfYx8izWzRlROfVHVrQLztc6xfifnZ21lBoHybNpo8eMDAj+cLFYdLIYp/i5mt7ywd6cAntc/1cIbDwet/e0Oa1xtLXyOLXk/uR/QF10ns2h4/G4reBnIAaDQfsMx5b5vAsKayRHm5l5JKoQgXmWuTPqQpDpsJ3qZrGD4X9+J3ozP+fPfH0GANrGZ7mLwWjUxuZxeq495gC5jzqZvbKTcv/Na2Sa7Wtcb6aV1h8/x+Sw68l0h/4lYuT61E9kkg6SWeaTk5POzNvh4WG9f/++EeAOJC4OUlyDfAhUqb/wYdYHp6Wk15YN9WQmRB/6EDpB2o6tqtopEcgc2wWBPReMaItTxpSHx4lnZsrqQGS79AqBvrLTgc3n83acsR9ixGIexWuKaNjx8XGL0nYU8DF4YL/UAgUB4ZAP8+ovBOw0xJxXoh8vzvSKagaLSGiew8/xAOLUiWZEVD+zautEQTmZIiTSs7NBDsnpMaiJ/pzC5d8OIo6mTmNZ2sLKcsYrnUw6pTSMPofuscni++x03ffk07zWkPY5mDz3LAcBozUvHeF76/pyuWwH/719+7ZNAKzX6/riiy8aanUaaOoiqY5Mxzab7RIGZhR9dLZRJw7YRDgzh+xh9dYzByn2Z/7nf/5nffnll/XZZ5+1NpCqJW+22Ww675scjUZt5wCcN3K1jqR+IweehTxA2Z688TKT5If7ygcRmCEgAkS5aIAHJReqYhSGgaAhOmWjNVoYDocdDs1O0mmoIyHRCGeJ0B2tDKPNJaGwqWiObrPZrHMUUBptpm3mtZIrw6HCSTgNTgRio8yIVtVdfOrPkmgH7qOgVlqnjk5Ds247SnNbfY7D/U5Uyt+JNJNn8/W5Jshj4CAG8nBJxARn42f5Hk86YXiff/75kyUAjCEpnftgWTnoIlMcclInXrPlQFVVLSuiYBPejE1xSrder9vx3NiUUaudOO1ika4PDjDCfPXqVb18+bITFHIxbdod424ujnSb/nmd566y+5Uf/y0UEAreneNovNIXQpKIAPxECCAVQ33PtNERBpPBtvIDi71Mw4Qj9/roG0hPG5gjfVWXs3EOnlPEq9WqTk5OOoruSIuSpdCdbtkxEX2tFFa+JLqNItIppGz5Pvd4cr+5LBsI/IcREHUbRVOPzxZjPIzw7Ez60n6KnajlWVUdh4s+ofBJ9NNW62amu9Tp9V+JShkb0kn+Jp1er9dtZpCf1WrVVqwzu86MqN9VYLnSLztxnyZB1lFVnaOtTO7Td9eJA7y6uqrRaFRnZ2d1dXXV+D7AAEDBtshSFAcP84nOVkD3OG/qc3ZmX8HeW9rptN6B0k5yp948+81/N5YTHlnLUrX1+JREC0ZJ6/W6DXi+/dg5N6keHj+jJwpg+AmRSEHpMWAiJSjSBtjHnzjXtvOgLW4/deX/CXttkH1t5fn53JwA6eNN/Ix0aGm4NlQrCumpp7ApHp8+xIfM+ji8vvY9VxyZ85hr2u66Gdu+AEAxT5Z8DTRGpvU8w8/ebDaNh0WvE6lNp9M6OzvrtIEV9ayqx7gz2PWhWnSGrTrm8tB76oDOIFB7cmg8HrcN8YPB4x7P8XjcTsNAtl6lj+3ZBm0X3sVR9XjgA/phfXMW46UwqV/038GnqjrHDO0qOx0Y77+DyPQ6GsM+GsgAsJXGsJGG0EFy+uSO8mAzSqYG1GkvT3GU4PlOe30Yn08D8MI5nPXV1VWdn593CNY+h+HpXkco8392Ih7w55ybFb3vPHbX5Vml1WrV4YVQZjtpO+3kGxhjxsI7ClwyWrqPljtpViIxo8VMoekLQSKX0fga6wffO1U3uexnMU4OJsnlgUT8PeNhNIW8uObnn39uevr27du6uLjoLH8gZbMDRX52rE6PV6tVO5wT+6K+8/PzzrsUWIjqt0y5HqeJ5tb4306OuvpOGjGaoy2gLD93tVo9Qco+WNTIGsCR29D6yk4Hdnt722A2a1lsLObHHAlBRpmCJCFrA3UEohhNUAf/0+FMm7xEIlORTPu8MNBw3Ijq7Oysc48Now8l+P6qehLp87f7lX9TnH5QcobRiu/n2+mbcE/jNh+WiNGOzX3PtlKHv/fSA6dJiSj7xp0+54tHfE8iO7cvHZadQkb45KyyP04xfT26A2IhUBD0j46O6vz8vIbDYUsF2eCfAc0608d9bjbbrTiTyaSNNXYFT4UTYukLQTv5w77i/uNgHWQdJHICggKoQT481+lu3+Jk82FV1QElz5Wd68AgwL3AkxQvCxwBnUNJxuNxW1s0Ho87kdRIxINvGIxAvTrbe7KIiBRvT/EMk1NHojpRCKFXVXvlEyfR8ow+/ii5Lk9IZHridLWqe6gj9xlhGVE5kjn9TY6CH9rp39zrYoXy/0nu813f6nkX+pxpmdMfxoji/XJ+ViIUy4z7CJpeB5VUAPXRF/NNcKp2mKwuz3u9fsuycrAD7ezv79fR0VFNJpOmq4vFot68eVO//vpr/fDDD/Xdd9/Vf/3XfzV+KJ1tHmnDs1JWOE7Q2Xr9+Bo07Or4+LjtMqAvJvo5R5+3b3MIIvIydw1KRM5VXeDC2NuuMjCRWgI+/CJenCHX8PO7OTDenkx66C0YVd1D+4B7OAtzLJnn53orUJBTr5zZQtGcAiYKcBoK95WQ3AIkfaraTk2/f//+yVuW3Y7kvFzsnPK0jCx9HFofSuWZfXXgWB0xKZbbczwVzzaxmpEUpcxAkTNK7hMliXMrPaUvQvchPuo3cvTizXTmqTtup/vnZ/iIZF9vnUQeTpNpu3Wa34nK9vf36/z8vF68eNF0jtXsAIREXbTTdZvHdNsGg0E77Xe9Xjekh6657+Px9i1cfWOGI0K+ffrlJUnou+sDSDBm9iP8z1gSzLgn17b1lZ1XcF6UPTLKbC/rGUSvf/IgjEbbM4DoBAabyzBQIkdzE5eu3zNS6/W6rYVxusB3Rl5AbpTzt99+q+vr687RxxQbMYPIYOHsTNR6gImmJr35PtO8qqdrxHiOI1+mGp0BjbV61OHrIHuRDw4/jdN9deqLkqVS5/+kPDn54bTVsk0Zp0PLgOHn9qXmFDsbj1Efr4cO+7l2DhnM8jQPAgL2YTTJqn9Om2Wc37x5U2/evKn5fF7v3r2r7777rn7++ed69epVB7knpUIbDBz8nYMkM47IlHTWW664x7J02u+CzXsyxfw2v5Ghl0zQJiNy18HfyHZX6rsTgS0Wi0Y+JnnvyGlOKzkFE3EWRBpJn2HbiTm1JFJRUJREJDynqnt2laMmDm4ymdTx8XEbsOcQVnIlPoCtD+raISSnYdK2DYgCRR9ysKL2IdA+7oT+O8hQzJGNRt03c7skmnFJXtMcF+uj8l4jbMvGKXoabCI69zsdvZ9jWaecLGfvAUVmuePChmzEbJrAJ/7ybE944VB4tjOC/f39Ojk5qapqKSCnqNJu+rJabRdcr1ZPDxrYhYYTBTuVw077dNVjDTdu8t926VUAyMtj2RfEccaeNNxVdjowKnEHqrYOw5HUaVsu1rOS3N/ft5NDaayPjnX6lamEFc4pw2w2a051Mpl0PLw3JVNARff39/XTTz/Vixcveo8qqdo6YJOXWfpSCX+WBkN77ChQDn+e5DOKZSWhbhTCiNPbO5zmYagofiosPKADQgYF8yjmaLw5n3FwdKZYn4xoLGfk6F0ZaaROudx3GwrI0KmdEa/H2XXbofl0VnOFGYBc0G1PjHiGzTN71lWWX1RVe+EshzO+fPmy7u/v6/T0tINOMpil8zG/6rQw0Vz2x85qtVq1F6xcX1/XyclJO9PsuWKkj+yQhe3bOk0m9Rz90pHxri/ZYsJiOhtSoioIUQ9uQlpKrgvzEgEfMQ1Cg6icTqd1c3PT2oTD5IW6Jh2tDPBsVY+DdHR0VD/++GPt7++3dw7yHcoOKnCk7DsoMHkgpzs4WQzDEd2IxVwh8uxDYlYESqZBVpJcd7O3t9cQI+nPbDZrTr+qi2ZYGpDOkesYM/qYDibT2+Rt/HleQ18zEFD3cySxHavl1OdYzVP6oMXsb/JMlrvb6nag/zkLncgTA8dgIbc5yQKSHa7q7u6u3rx50+HOjKD8DIKVZYITRc/y+CoHAL7/7bffmtN7eHioo6Oj+uSTT+rs7KyzOh85mfdGh0ChiYRTh00NfYjAr/qAA5tMJm2tFgNBQ5imtTHTaX9mRXDKyDn5Nuo8OsN/HxwctCNOMD5HawvNhpaobX9/v/7617/WF1988Wy0cbqUaUcaMnLxy32rttEsrzPPZFRpY+DZdormg4yW2MOYfCFBwQiCcaQ+9oXSJxQNhff2Gm8AdhT1KQ2MIdc49TcfkilnOgGnK5Y5xYsioTeSwHfJIIkemhOyvhj9oiM2cK5zfZZ7VXeLEd8RcHk+aIw6vSfTDpeXlTAeX331Vc3n87q6umop/8XFRTvby/0Zj8cNUftz+pCOF+d+dXVVDw8P9dtvv7XlIC9evGh1Xlxc1GQyqTdv3nRk4smIqu3C16pt2urZxdFo1PEnOFg+N2fZV3Y6sPv7+xYJrBR4fwSaq2zNe0HeJ0fkiG8S19HOHJBTIuq0grUOyXD6VmnPZrP6+uuvnzij3JPYR5imQ/Nz801HfXyX+0JdWU+mJWm8OZHRl1aZ3Mf4cvkKsrCjSOMxgezx7ZN/piPcx/OSz+tLyZ0K+j6nqFXdZR99a4lyFsxIKmWdbXCwM9HsurKd/v859OlnE8gwZMveBwUY/RuNMxHAIlLqu7y8rPfv39fd3V17Gzi8Jg4C20y9I9tiKyBB7NNPP33St6pH3elzLH3cmlfkJyHPuOQBiBS2NT1Xdjqw2WzWTqMAxhIxMFT/BhXZ65po9Hsaqx6VxW/tdkQgsj9HtlZ1N46jcJ4ZRWmOj4/r+++/r5OTk7q4uGgnZfZxW30DbAWj+CjqLCgUbU/HkZMdJnvz3PGq7sD3GbwRg1N3o7z8Hvn1oUqjEHNgTjGyX0beRi/ez8qWNCN5rnd/ciIojTjb6pSIunNcjPCN3u0gfTY/8nFW4HTWekIG4N0LLslToj+JUhlrb/B2+1kMy+vYjABns1n98MMPLe2bzWb1/v37Oj09rfv7+7aY9u7urubzef34448NsXG8dNUj2ttsNu2I6aOjo9psNp2XKVOY2aSPBA9znunQkgbwotaqbgDPseorOx3Yixcv6t27d7VYLNpRuxCQ5rB4UQQoxDN+XIsjg3fJdM/G7WlsrvO79xBYRiZ/j2JOp9P67rvv6sWLF21vJ8K3QpoL4Nn+HIGaH8AoMp1I55W8kPtqdJvLIkza4zSSL8joDyLmt2eMbRB2XImQ/IxEjkZQjJ37hNMySkOOTrPdFvN+fGdHmzyOnbKn7OlPbkEx1+Mxzz7YUVm3kt/kXqe49NcO0fVnUOjTO/OJmcrSP2TidZNsOKdMJpM6PDysi4uLzvHVP/zwQzuy+49//GNbMoSzHI/H7SUkq9Xj5AtIyzKwjvtcPY8nbXQhJcxVDAYetgc4t1082E4H9uOPP9Y333xT0+m0bm9v6/T0tNbrxxkCzvlGSTNiG5F5KtqvNMvNw5CO8DpWdtJWpztWbKcOjiCvXr1qb4hJHiAjpme0ntv/1wQ37h6qaCdlJR6Pt/vETNq6z8gLZewjrZMUtSPJv73q3byDnUI6qef66pTVaMTOr6+4D88tRXAabT0wggetols+ssYksBct98k20z1kmusWLXMjVZ+Rxr2JhjO7wAkaZWIr6Gxyl66jqjve3JNrH13QS3OgOAcm4gwe/KpENnWzbCPH37xc8nc4G2zVGY5tlUKQdWC2c+YVeF6w3Fd2OrCXL1+2WavpdNpWzpK/I1g2b+OY/CYhhOiUI2fZcgBZFsFgcqYYAkmkwnYM6p5MJvX69evWbpZMOOVB0LTD62mIpii2N2RXPd0aZCI6ERvOlgFKZ5OcDI6GOpL89qxg8mOJFF0vEy58x3256fs5jsSFqOu0wakUbUE+fWmxURtyIU1Kx22URGDJXSCO1Clb6uS5Hjdfz3OS3Hc6lxMuRk8pzz5yv298+AxHmQGeceWzTNUeHrZv/GFd1nA4bM8kqE2n0yYjiH3kitPy2FpfkzayrtEHp753d3edNN4BPZ1fonGe3TfTnGXnSnzPaODIaGRVd6YFIQHh6QiLQ1NZ/DIDhEhjmRLGSMn7UzEZIHJ0IPN//ud/1ieffNK8+Gr1OFvFe/qqtq9Yt1GQciJUnuXobCEbURGRWU9G8fd2dIb8VpZEEibHPbg2MOrM929SqBeZ2vnjpI2cUwesfFZgO3LDfXNltM/pD22yY+ZZ5rD6lgawewJHz1EyJt1zJwbtY4yMHjn9FCO2QZpvI2Vcr9dNzpY97fbpvk7T3X9zecnH8Vo77MMoEmK/akuBIAOQFnJYr9d1fX1dt7e37Tx+eGhOm+D4HOQ7HA7bsggHhL7JMPptYECw98GnBN2qLeeVOrhYLDqy9wTH/yqFdJqUM3vuBMVbVBCKnaANE5I+I23frFKmURYOQtnb26urq6tarVb18uXLJ/cYGWVxP0x826n2RU1klGjH19gxOJL7mUYzWaib5ydvYMcE0jDUp1gJkyTv6xclxyD7acSRiypzgsR9Teefz0+DT56M+x4eHpoeQQEgFwwBJGxDwzn4sxxLdDbRItcjByM/oxbrja9L5J3F12dQoY1ul7OG5C6TXPf49TmG5KdAnKmHOX583vc6tVydnzSIT4QZjUZt3en/ZB3Y/wiBkbbRQW/cJsUbjbZvpnH0cfplFMZg53YBrw9xLk97iIY+N/vy8rK9yv3s7KzDAfG85GAwclIA+tbnRGgrhueZI3NKfSXXtjm9sBFUbVGcn5WO32NjZOUUljSANvK8jHz+jlQ/HZ/bg+w99kac6ZwHg0EHYYAi0JE+p0R7/PdzyNcTNnxf1eVnQNkOHvztmeTkbqu6KJlnsM+R/jgggGbNYzkdS1TICSsGCk4nq7Y6ac6NvtnB+XBP6sSGvFEaxMnx088FE/OC6UTRe/eJseVv6IC+GXQjd8bLY8NnBKDfnULSqNVq+1Yf8m08/mr1OA27XC4bdES5x+PHWY1cB2ZuAiXnefnqKl4egNF4hT1n0//5z39uDhPl9GGJcAvA4/V63U6YRTFMBHsQLFTa4XSEwe9bq2JF9sFuThkT5VppiLR9xL9RR9U7/LoVAAAgAElEQVR2c66vt9LDn6QD4ToMOF8KbP4J2duhum2gQvpgnsOkrR1nppI2AqdWFGb67LwxFAdEo4A+1LvZbA/7y7FgjJhIsjNiBt0OA9qDgO1tchmEIMOrti8arurO2HF2lw0YZ5zXIkPGHf32b/PDfsuYHVEWDnFAv7ER+uCgwm9nJZa1MwM700RmyVt+CBxUfcCBIQDWhvBABsuRzE6LQoftoOigowQDicElCcp1RBqM8f379/W3v/2tXr58WV988UVnYMxdMINJ5EFxyd9tSI6YRhzJPVFodx+y8TXUhVxRZPMhyNQpDvKxTIxy/Qxklam9Uw2ucVs90WDERb/MB9oZpPInsWty36g4U1jrhQME1/Sla3aUDjAgCz63LN1n5GcjxYHSj4ODgzZBRfu8oj8zh6pqeua01frO/2yARrbuA1kN8uG9FC59KS31OTWmbi9h8DimDK132AjOy4GOMci03NQBqagdn/nyvJd7vDRm1wxk1QccGFOskKRWOKePLFkgNQMa574/GwTRyhHZRp1RgRy5qtqWhvl8Xt9880176zHPyRMw7ED424cyJq9hxUaR3R4bQ6Y16YSt5FZSO9nkK8ypVFVDpTmYfYPreincn7yHUXGfUbrYIFer7bHBbosVd5fiZdrrtjgYoBsei3REnlQyyiPKm7+iGPFWVdND2uQMwDPV5tGqtkttHDjddp6Fo/epHIkq08lmkFksFh2nzeceVyMqHyJqWzAa41qj3RwX9xUU5X5R2EHgwE8fcFzmSOHLDFJAZaYsPF595YMnhq3X687bUPgMJ8CeRjoOvAYleXAZBENdjB3nZYiK86BuXkrwH//xH/X+/fv24gS/eaeq+4YWhENJSFzVTRftUEibrLjpXDMlsyI71eQ5lqsjoe/B6HDaduqOfukkzIM5peDePOUz+bk8TibJbiMxyHO4T9pq/pKxtiG5zcwyZvSnDX63IUqdiyMHg0HHqPL75BmRHfIZjUaN1zLCwFkYtUIV0G70ximjHaAdYXJI6AqcZQZa15EImT75BGRskPt4FZqfZd1N5GyUaNBhx2ruCllYf7yX0WOBjJE9bcdOSW25HmD0HCftstOBvX37tr00E6HgAOxkckD70sDBYNCWQ+DIII2NwniRCI4TI7u8vKzvv/++vv/++/r888/r/Pz8CdfC3jA2xSbiAfUl8eqoz6vdzY9UbZdsOEWhTit6Ctyy4XkUO1kG3VE22+j/zS35+nRKVU/3U9oh5UGR7I90SuJ++1kodJ9TQW6sFzTiNVJzP5Gn67fMbDzeu+rjts0TYSB2/L7XAfnh4aHm83nnFWAZ+Ql6XiaQuwS85tEkOWPA58vlsnPCC/20EzFHOxqNnrw4xIH34OCgczSP092qp8S50TaO1CjTOmH0aF6XOgkIzDDSL+sFDqmqOkui0B/sy3IfDodPkH6Wncsovvjii/ZWYhrvbUUIGUUhVenjZjBknCBeFqTGoLhevPLR0VE7P/wf/uEfmrCJ+Lky2UrjOs250R87HsNwK6AHy2edOZrZQThFIcImn8b3Jj4xOga/Ly0jwvEWJae3yUHu4q9oZx70SJpAnRxdZCdBv7zQsmpLvLJEhjYhUy9MzCUgOCI7MaNk82tGusjFxg7Sc0RPAhw5g0zhSa27tMvjiY71cVLmdCjsWjFKq9oebYQd0BbGKlNfjzGySZrEdIcdhAv3ONvxJERe35fKsWh2NBq11BFHhFzRzRx77qc4pfWYowt9S4JcdiKwH374oc7Ozprz4Y25d3d3rVL4MYSCUZBajMfjOjk5aRGOqE6kx3nZ+dCJzWZTFxcX9Ze//KWqqr766qu6vr7uzGS4c34BAYNotGSEYmhdtVUQT61bYV2fSUkE30cor1bdY39MvPo6Dy7XohR2Zk6ncCDJFWb66rbRV1IWR2AiNuNzdHT0JG2xItvweKaNlOd7M3dV95ROE/vmmZI3tCH6e/TM7zBknB3t7dDs2JAzbWTm2lxRn/E4taVPdl5GdsgCR+HFpmQguXyFlAt07R/qRTaWn5e0pCNCVk6THaC9fMnZkx0pNJDPlaOYHgKUmEJBXs4yCEDJjz1Hu/SVnQ7s9PS0k4sSdaw0OStRtXVGKOv19XWHXF6tHpderFarzlncHgDe7v3dd9/Vl19+2V6HjuF4epmOInD+pi3m0WyopLUogNPiJFq5p2prwIkWKJ7kSELzuX1sEMYYvZcFMNj035yhi2XOwENQJ0djZIFMcd4mtbkmUyrrAxHW9TkNs5Oo2u4YgDfzBE06Kwp7II2+PDvotuJIHd2dGtpYuI8N0ckzeoxwxEarXEc7nIHgDLIOj01ftgKFYk6UoOXASXCjTYyhx8ht4tnJM2dxnciUoAfCg99mrI2uB4NBZ98xSDOPeYKWMILGt5B97SLwqz7gwFhrZXTEw/0/3p/lFkRA5+nmNgaDQdsrxT5FnseRKw8PjxuxOY/I6Slw18riqMFEgslRRx2ucxRxW83xOe1iUC0HowAKjsLKYZLb13kFMs/KlCfb7QjvYscFwsJBIz8HCSt28jg8C6Scqa/RoKOkZ4AZV7gMBwCeY8V1MZnPPZDGGAmnLCAHxtMHLDqNBoEhawKJ+Tye7dQU2eb6JTvxlGVVdYK80asPl7RDJIUlWHuRd2YrfdkDSzhoD0HOAddjn46Yuh1QkgMjMDjg8mzLCwTpJVToP7oN5+jP0rE5U+grOx3Y3t5eez8is42Q4IbE8/m8zQayCM/5PMSp0RIC5Tu8+uXlZf37v/97vXnzpv7whz80gbF3kS0G1E8kQDmMAlJp+MwLbqu6U/gZnRzl7TDg8py20jcrCcbgpQXUQ9/Me1R1lyEkLwSCHY/HT84jt5OlTbQn0Q919ykIjgmk7M9zgsFoF76RtpMa0WZk4XQJI3AbnGpzL/1Dn3iWxyQdkbkW2ml058g/Ho87KSTXeMkHi7i9tCLXQDr9dEaSgcv8qQ0VOxmPx212FPLeM43OGPjMn/MMnD5tsM7jkA0uaAPjgWPGESGnvmzFz0BeRsKegMCWcY7myrwY+UNLcnY6MBwEA4VnNCpBqY0U3BFzOhRv9ubzk5OTms/n9W//9m/17bfftgPYIFg9WEQvIpxPUGAgM6JzMJsFn2mg0aKjGsWpnJ1dokv+znOpHNkSOboYgSFfQ3Tuz7dLV3WNluvM/3kpRZKzfp5LpkCWj8lb2m6eCUXEEDBQc1lcw/1eAc8GfHNA6CD9oh700+mxv0vU6KN5uMZcH7K1k7AxcpiBKRP304ia+91uZJDOLNF30hAgfqf6DlBG/w4Ath87B/TLz0+HS5sZF+9ltC3Qz+QEh8Ptm8mxDcvGky+ACfPhz5WdDgzegBkHKjVxzcOcIgCrcyuG4Sh5MgPzl7/8pfb39+sf//Efm+cm4hnu+zVmKFzyOYbXjuZJlCaHxK547nuOVLaBe9CsTI5ITdhyQjmLg2PFsNxOp55WvuT/LHuUyMjSwcgBhrbwk/2gYMAZEY2AmT22TJk1pQ8oM5HeqCVJY3M/ONdMneys7HwoOY7oLZwXesazGAeW0ngMc0IlJwYYH56LPNDLqu1xROiSU+VEb+i901Y4KvOqfmdFVXcvow88dH9MBYA+c8IJWzLy9+fppJGhwQG8Lm3LpTVOO42+PsR/VX3AgSF8vLONwp7VcDQRCQNZVZ39iVzPG4JOTk6acmIk1GukAOzE2SQ5aT4JJ2WHg1Ml9bOBMvNWVW2dDoNjwjXJ5UynnosYhsN20o6yJs0pKJmVmh/uTz4MwpX22vBJu0ER9M0kMdfZ8M1V5LOQk9uHk/TyBHQJBEmEzZlBFxuJg4KXDFQ9nUlOJGaSn+eAdr1Jm3ZBn2Co6FUaljMFOzT6RdpFXVVbgh95oEMO2PTJDs5O27pkWWGnjDvpJ5/5TLk8np0glsfb4Ig8juboqJ+6klNErsjR40IdyNlLiey8+8pOB3Z1ddU5IsdpkQ0OPsKCQsj8dkNAIFdXV/W3v/2t/vCHP9Tl5WWLyAyiIz6K4KgNP2ClsgP1rAk/8DJc53SYKGxFMGzGcTsNw7k493fqnOlYOiYjVyMLBtLK/fDw8GShqnksE7NWCPrqae5Eqm6rx5fr7JSNMJAVnzvg5Vq2qm4QI0D5jUep3K7fER+ZWwf53IHHxYQ8442OobfmibyI0pwO3znAsozABT0BgZr3hdN1ACJlNvdmboxJg+Fw2GYqk1flHj/fdog+mH6wU/Vz3Q9P7KBPnkyj0DefpsFY5RY/p/WbzaZDF9mm+oJmG9Nnv6mqr7/+umazWVuMB/SzAQA/q7abXemISWki8Xq9rrOzs3r9+nWNx+P65ptvWlp4e3vbGo5CGHaidAwq0cTkpmeDzNFRTAKbMzO359LHByV34u/sGJ1WPlefDTejD87bUcpGWNVdNEt0yxXYyAZDyvaY+OU5VliTuO4Dz3NUdpDCEJ2Wp4wTPZizYezMPWXQQReo2wdlut7/r70725Eruc42HDWITbLVkjXBMmDAPvXd+OoNwzpow4AB2QIkWU0Wh67hPyDezCcXI7ParZPfAAMoVFXm3jGsWMO3vogde7YlSri/Pz2R14DluOrX999/v+7u7g4Ui9zpnMP0zG0gId+cvHKuLnmwfud4Hh8/vcBZPZmpc/VN/mz3uJ4yc1U0ZD71fepDwEI57xYOtI+Ayg7VSn1Me5nlWQQWYunkhlDN3MPiM4Mq6VdffXVwPt9///365ptv1r/8y7+s+/v7w7OMefJy+aJVEDglMq8XfU1jlixMcAl0ClZFb2NmdQmxbdu2/NxJlreyaOzdE8FuRJ0LEfJVGrlbFeqvyEnFCsnKc9XH3f8ig9r22jnmeItdFG/syVzi32dp5cSaA+s36Kx1yuPIH9qvuTdKHVprHV4nlgPM4UxSPV2OF/Z9i/W7TENSf63jCS7JpR/5HgNvTjteSJSnvuhwHaOZRAg3tCv48GmK6igIyrPplJorx5oTza7u7u4O2Y+cbnNdPyZA0dntqJFZLjqwn//85+vt27cnHECCNA1wYMLVJq2l9Zubm/W73/1u/cM//MPhFenCRfkRByZ0Vwlm1DEtMgJ1n2eEN44Ub3r6HQqoPg1zpoqT/7Ou+ti4ur6+abgZaYsd9XGOWfJ7rU883uTLehTIlKW+yz8YMCZaEmmLGrq2uZvp9ST9M4ScYU7D7/2tTEXkRnZlNUtpl2nmHGsOJLokh9Bc60z9PCSSM2psyaZ07+np6XCsc30wvW6Ocp7pmwgtOUUj+ELimZK7QKOTzI60i9BVAEPH0RjTPbMZ+Tcd3/QVM/C/fv365F51I6B0c3Nz8jzlJTL/WQTWSzTn84RN2FxxmJP09PTpDUH/8z//sz5+/Lj+8R//8fCizbU+P7E0RyeR2edyLh4KJwJqMlxpskjyF53qu+eQNyYfzpUYtz4nPCWakSNllYtqLI1NRTONKArvkE0GIOmp4q91+g7DiSJDLkbltU73QNXHqWw6yhxj94j8TG+tvx/3c3lqgYbpeA1QjqXfjqUnPkTWfZ/uqEtrHfcr+TiMTjQnVn19Vt8zurKUjx8/Hl4Wm741nzpsF6JyRF6jDpa5dK+66LWeeefiT/NWxjNPljD9tQgsmpuZestRyx2KvlskmEhu+pLJoc7y7HlgEfR1PqMWuSTwnEJe/+XLl+ubb75Z//3f/72++eabQ/rTpERgq/QZck4uYYsGREHzrC5T3Lk62HXC64mo5t9B4Ev5uFGpCZrpn5GqezLGFL5Jlxeb6VZ1KOucR+lOK2qSsslVzs9Fkvpfu5NfcYWt1EkuKn2xHlFsiDdZmJIqP2Xn/fJSom/nb/J+NzfHU1VFXTrz9EX0bpruPOrs1JPQqVsFDFAvXrw40BNlI81f+xNrU6JfPUw+2ZZHqte3UsTmVzQ20/H0uY256kJzPMeobc9gONH6dDrt26weeS/tNB1ID/4qDsyJzuCurq4Ox+tMb1wnnp4+neL65z//+bAxNWVJsKaFDSiPG5wXeczzjuqTKV4EqQI1r55CnemYBtMkKWih9+4+jU9eYHJHOayp7F0vKqv+nELEsQhJrmKuhIWS23flezmn4U9nJ//T36781YeMx7F5XQaSceog1zruU6quGdQmP9M1/dYx+1Ppfon2UrwcnHqjrjZv0hqV9K75Clm9ePHiME/pT+mpAaonW0JmEtciu+Q7tyGJXpJ9+jOfF/ZQ0mTr4oh6oM4JBPpMuVfPzumajZmhhLDqk1TEHM9fxYF1DMgk1HQc04t++PBh/c3f/M16//79+u6779Y//dM/Hd6GncDn2VIO1OsUbnyC3ltEpVHqiIT7Mx+fKY6rNSmtKzCT9PdzI1CfPT4+Ho6j8fPpqLo/BdUxZGizzRz9VCZTBx9z8YzzomyIKrmI7qaim/q2ipaDSiYerWJa7SbWtY7Kbdqbk5tE/ORtciRzIUIn5XzskExoMNogx9E1/e0pFx3qea7sHK/kuwFjopCrq6v19u3bgwx3KLQxNmYd5DT05kpEmzzmkdumn2udpoaPj48HCsXUvvkr41Bmyr7PClrNnTTOzF4mx1j958rF88C+++679Ytf/OKE+5qEakaYsfzsZz9b33777frlL3+5fvvb3x6gsxOYUnX8tNHNlDQhTeLZenQ43e//tbsj10sNZ0pZmYJrvPbDFR+LDsUyuZO5jUD+0DHqDBpT3+UATD/kakzBu+bx8fGE2Pdzjd5o2O/r6+vDqp3p+dwa0KqdYzQVnRxaBpXcq8+50AnsaAHnX13x/2SnTlbXbv7nql7fzbmVn7Lk3OxjCMlgNus6l+EkI1HsLAYor9llEF7n766Xb/T+ApkAJLnOYK/tNL5JNSTTgmP686M5sHLkYJ9vS9kR2j//+c/XX/7yl/Wzn/3sMGiXV40KCbnBO+iiSmhBMlFo3/Up57mBGs2mUC06SSfEOvrfiKMj7RodqLl8KMJVJpViGuI5518fJ+FrP2Z0TZbyV66G9YiRiMm3Ue3S9wzNgKasdFL1QXnM7+z3ROHO7yTx+z7Es9Y6Iaado6lLs6508ebmeNx0OtZ1bgRtLkTDBiN1Vhn4DsUQiW+vUhaVHFe2IVLf6X/yO0cx1LYBqnZcHKo4jvms7+4N7OlHMu36DoBIPzyBwvsuoa+1nnFgnbwaBC6Nk8O5v78/nN31u9/97sC15Fhawm/ydwLMGc5BlgpNIaQwXTcdRk6jCfVxCQU+Uz+5tOqtTolJf5eKzFMeqq+VrJzW69evD8ee+FSA3I4pdOOvmBK4gtp9BRn7KXpxhbMxyZ108oGKL+Jb6/ToZtN1g1VjCeXV965tT9Pk0CKQZzv1xXlK5r4cwl3j9S1COj3o99PT6QLV5GJ8RVp1TzI8uebkQvU+JhXiqk/qbSvfV1efzrUv1fcVZLOkj+q5QX4XHAIiczGovuSszLRM352jyuQZW/EUfRUAkulc6c9WTNfngsGlctGB/fu///t6/fr1ZydQlj5EUH/33Xfr/v7TG7HXWiePhtze3p4QpN47BTkjsBxTzkBl8pjjrps8QxxSQp1chg+Hi+bkQeSIZr7faQk6Qo/B1RnriHPuplM5gdqKg6h/ySTnM5FZ/QlR2O8MUidUv5Jn/RUtd7+8ylTc6ik91MBML0xP5iNUj4+Ph5XnFNx9VJWbm5uDk24FL9k1zklH5ISSoYGjlVMNa4f4c0JyV2YC6l2/rWOH8poTg/fcQKrMZ0AycE8+S1nf3Nwc0vGZCk5nVZ3qbSv9ZgtzAcUgqdOLWsp5Zhs6NdPiifwb57lykQP77W9/e1CI9oNFat/d3a1f/epX6+7ubn333Xfr17/+9aGzOhR5iniXJktHlhDa32L6MfNwI6D8QSsmppazGBnkKnJcRuHycElYHYBpocbob8ld0YRcTelZk+8zeDmTFF3OprmRB0z55vYSX80mem1+5LJM7bsmQ5gkrI/FJDfh/26u04WKz8DphDLAZDB5qMlv1i/RaP2ZD0TLhTp/OsLa6u/0v2vev39/+KxrTR2rL8PUFhxHaNTsRnSabkx7eXh4OElDLa5aNza5UBFvc61+G5hqWx2eepZz7zN52/ow6QDnLV8hBzzBxK48eyKrjwK9efPmkDL97d/+7fr222/Xn/70p/XrX//6M4JbnmStY47vgXcaSDuLc14Zm8u2Oozqa6+aZLdKInKxP3d3dydGGpxNCUNmObm5ymI6aiTy2opp1Ty3f63j8TIZcYikOluaD7VdX1+fbP6c3J7KMPmqc31MbpLoGo2BSSfhufzTyctpaNQ5pubWc/Obk5k+uTKXzOqLeiR6kI8T+db/+iUhL+coQqifIjLvrw3bypgr3jsfvSttVm45l7lwEqLKqSiLify9rmv6PLu+vr4+BDhTzMnbZuOOYYKJdHwHHpKtCznd0zxKKTRPlxzYRQT24sWL9e7du8N2gLzsy5cv17fffnt4T+Ps7O6zJtpjO/zthFXH5MRER3ln4fskbOXNIqjz/J7kKp/Q/3MM9d/PUvy5QjbH3W9J9toTmquQLsNr/DteTAfVNY13Kpgks87eeWheRMGixOqIVyo9cF7milL9mgsLKug5hDZX0vzecccbzbE4X7Zt0fh389hYzAKmDtcHg62Oea3T48Z1otPBh9p12pM0T87SEPJf1dN8O4dytjkUUaLt2Nb8vBLyrI6JYtO1HKX3zYWcZJMuXSoXEVgQfK11eJHs/f39+sMf/rB+9atffbb6Yc7f/92fkExjimoKfA5orU+OVI5CRyYXsVNQjWytdXjno6Sh11ZqZxLmftY9RUTL7NPcj7PWafrbNd0jkSkaSd5GaTlDFVflr21TvCJdixHKI7RXvaKWjHn2e85/DrN5N2XXccmXiuBsd3JI6kGlelL8UiaRRLKbn691Su7bho656w0Gpsp9pnw8QNJX+jkvfR4iMigaQHYOZKasyii5iNQKpgZVA6qydC52n810vj70qFBBUN2Ygb8ij+cTD5ec2LMO7N27d+vrr79ev/nNb9a333677u+PLx2YHclz+wIA30BTXtv//RZ5ROTW6Y4PKbJJDCcor005Jw9XHzsIUbi/1ufnIhUxamc66XntRIlOuBNvVIpDqh8phIbSffbXh9Krez7LNoOA3NxEhSFjSf/Sets2cLx///6z/hqcvK65nRRDv3PGorLKDEYimB3yr5/1y/4r84kWGrOkc/KavJhHRtVOiH5mAckiCmBuNWilU2eZbPteY/ZluJPWUAbusRLdy8f95Cc/ObyIZ2YcM5WUa0vXTLFLiQ06Dw8PB9SYXHvkqu8Lwtqb6PW5cjGFvLr69OaXt2/frj/84Q/r7//+70/2EKUc5fo+2K0xNcAeHcj7m8rlaX0HZcJqIpy0STbmgHypan3TIIqSGkP37lJfU9uEvktLKpNQzqnOKGgkzXHLf8mZidJSgpREx66RJWP5qsYgKkx29lteJ/TZhuScZn1SSePq4k6dJ8euQ5lpbN/LjyiDEErXy9vopEWBGreIyejum4Ls40y5J4orGKr/of7+7jtTMHm2mZ7N9GuX1qUHcaei5SkndS8uTd5SPew+A+Pd3d2J49tROv0fmowqsn7rNu3d7XVrl4FHIe3KRQf26tWr9ebNm/X09LR+85vffKYolTkQU5QclYZu2lNd07BdRfHxh12U9r4mW6d4LqLLZ8zvKnP1UAQpZ1epvSanFbq5RcQVtlDoTPkyDBHAfHrANCwFmatXO/nbF9OGKYMdMrR+l9bXOh6J7H05qbiy6p0o0XlLPvGvtuFYditwfi9yEE3rvKQhZsBThiJGg2P/64w06p2Drm77O/ukXGZprueufw9ZNOMxyMyAbtDzwIW11snD51M2k/+T+vDadFtKofbO8cZlK9N+Z7mYQv7xj39cL168WN98881BaFauwekpbbitEw2wjhtF1zoaj0odchH5FHm8d/ci1tlmRe5B/k6UsuNbdKymh/XRFFMn0p6lHU/kcd0T8nuuu+2r5KYr9qV6puI7XvsjwmyzZp+7SKJhWsdcZJDjNCWbc9T9zrn6MLmreU06t3NiyWPnCM6NR75G7jODlD+cPFB2If9rMJ3o3tTVel69enXilLQ5KYh0UK52fnZ9fdxvJ63j/IkIp6zVcZGd+md//J2emsbvFgF6f2T1GZzr4yUO7CIC01n0f56/HdumY3XCiVM4KnECEtGYDigsCcb7+/sTbqb09ebm+NjTWqf7X2rbYtq1i5AuPkhWNhGuElbf/N56KhrRJM37fKLNZKXTMbqaJjq+adgqUYo+UUT1ORc7xGz0nYiooGOavtbplgPrnP2cemDKYdSefM+5vs7PLepADiqZTPTUmE0/SyOlNiY3rAxM3Saf2Zt6OsOs9FM98P9zjru0bDogEf2Um3pqgDHoze+d+5lqe51lBsBseDcWg9i58ux5YObiEvJ1MIKvYuTXk+uAditlIrUZFeQwaq/o9fOf//zAA6x1FOokWkNw5uWSkEXEndD9TOflGCsRnPP00ooTqALIL3WfCEteqD7PVLB6+i5HMgOJAcD27ZfOZZfiTkThXM9V0elo/H9GcfVg1tf183Vc/a0hXFL+iQLmG6oMIPKjjvnp6enA0ezojIl+5ZCaa51in/l0ydPT0+HpCfnAHf87Uy1lIzpqdbD57GeWyb3V39n2HOO0feufWc319fUhQxHlrXX6NM+lcvGKN2/eHCqOi5pCeXh4OEE+5/iJBuaxHJMQLwqa5rnqlAA8D/zt27frzZs3n62uuQF1viLKFZMUa/ZlGpufG92m8uYYHOdM7SrTYWXA3ZNy10e3mUyElbOYiGgS8msdV2tVmAnbpxxmGjdlMtPC/p4cYu3bbrLaLS7MhQ9RrUgi2dj2JI3rvyuBGnPy2unkw8PDyWpg8pv62ne+jFe6ZZLS2lHXllWIxutDnxuY5FwzcEYAACAASURBVN9mKqecRE7Jsp/dHJn+Tqc836k6U3zb6jMdoMFCWUY/tPtg1j3LRQcWAVr0nw5pPqipUUjYTkMVfT09HV+saj0KQS+egXa9E6NiT2/ug7yS9kYfU90fIjyRm5855tvb2xNDdoWnPtT3xr1rezqV2W5KGHLLeEVqjU2+pnq8xofRNRRRoWnoHNfO6Yu6NBgXQvqsbTSTt9FAk3F17NoWESivuVAklbFDU2YY9WM6ua61jUlu79LVFnl0dA8PDyeOzleydc3koSe/tnsNWqhnrXUyppubm8+eaGneI/99h0F9mrZRn0yn1/qco56627UF8JkVXCLxL3JgdtxIWMl763zkPSQ9J0zNsK6urj5bSanu+pCwhK9zRcQiJ3Rzc3wId63TEylmGuQpCDNqV3QktT+JUfuvkfj9DAgZRRPY7z7LEU4e0bpqI9nqNEUYjXv2v3t08Mmzvu94t5nmTQcx08sMqXsa20z7JgLr/x1a2M1Fc9Q8ipSUzUzBlW888DkOTYQb2jdd9BpXKSdpfi6dN8U27TSwzHvMJiaHVzH1rb4pw8YjF65d1ofuDTnO1UPHbXCeKe9c1JkLI7vy7Imsu1RJ2OzSaw32HKEK6llidXa+AktEJVnqG7gnT7GDpcHwPqu/cQoq3IT/osbGsFPy6aj8zt/yV441WYTUWnVMKUSDO+MxmPhqtZyTfFf9FOlI7BoQRLvOvc68/nb0i/Mq+s1JzJRgpiwqtfvmKpMTTS7KqD6LLEojm38Rjn20jr4zpek6DUoUJpc6jU3HIMr2x9XIrnORJNlMDq7/60tzPIN9/ZhZiXa8O99s9kU5FRBEWz0X6ziT/3wms36nNzt+9LkVyLV+AAfmSoRGmMCurz+9JbgIY94u1DW/rYNd028fP1hrnazQdM0kWCd09noFXpRxQoX5EzHNdHKma7PNynTKKfXkDCryN3Jec5z1q7YnWZ3SiYJFPI1ZFCe570KHjs40VKfuK8Dsl0o9U8yKS+cWUYoObi4C1J7ZQboYWlrrk5G6wCNHqHOafKyIojmUi7TU377XuHUoax15pSnn5qBxybs5d9Y1F5bq5/zOlXw/L/jsitdN+8hmZmaVE/cag2M/OvDqvcSbTq53losOzNXDSsJwhbJJE+3c3Hz+VLqdK0VyY6aEevdO3qN2XI2rD0UZ0wUV077OVbXGKWq0GKn7fvbNSGK0d8Kmght5Jgmt/KcTSBlEU1MhVQLH2qM0cV0zlZ3O+uHheHZZ16x1POBvtpnjnGggGc000OA4ebsZnXXmIs/6qhz6W+er/KrzHMJrPB7IudPFZLHWMTWb6XfB13mY6Ezd1KHJO4qSJ/qvXed2Iqf+76Fqg6bj0SkaINXTMhqvt83kZ9Euq9uMZq3TRSFteVcucmCWmVpJWBox+tzvD40BIeucT9gb9TxVc3JV1TWFIoSf11dSgmns83qh+twaYD2T0/K3crGouD4KZARtUg0Moicjv46wIp+1cxbniGtTDY3EHe3Oo47VxRpTL+tXNqby1XcOFey+271Sz5L86oPzOHkwU0YdkO0atGZKJrm+49HUJz+b+xWrt7E1/zMo74rB5zn9lsM6l82ItKZdzZT23Lx5Vlz3xZWZxc3Alo1MOc/y7Jn4VarXtTMJVodVhJ9potEjpZpQdMJvJ9doUJs6lF1qZ0paCb1NvmqOy+i/W45f66jUrqBZX9zfXGlLphnNXAAxgvvmahGNCLU65mJBgaB+75xGRWQzOZScZvP7+Ph4eGyoMcz0tr9V7mQlzWD7u8WNyR9W0qe5c93o3blXtavsdg5tOq+JSEWzIiW/n8R5uiw9EHJJf1zlm+lsv135m4jKa+WpdD6WEJ9vl7K+9puJ/rS/q6urwzOPU07a5U6WX3/99Wf98VnOgshfzYF55vg0gP5uILe3t4fnsCZXMPPnh4eHz96Ic2mlQbLUU18znt0kmbvrxa+urg57dGZ+P8eV8fl/dUwkZpozHent7e2JE1rrlOCWp5rk6Awg0+mIcDp80pJT26VRc9Gke3vVmOmYqLVNhhmdBL2lduWbZmowKQZl6fX+bZquY21MIsXpPHUI8WXNgRytbe6cVO3vXmRRMcDMtNzAUF+qc5cyFbB3trRr1342dn93bzLZLZIJLtJJbW0n38nD9b87DibAmVyfOr+bE8uzHFgpnhFWZPX09HRyvM6Mvg0qhRVh5BhNi3IWCWymqvPFHJUm3snXyVSExlNRXCRonLtIMvkqZeL/yU8UYrqSjGr7/v7+cFihfU+OOoFQpIhr5zhFdvJq3WMQmGWiyfog0poLHDogHYn1R+I3BymqAWonSxV+p9hzJcsDDuOy5KGURU4sfdxRFPOz+jupArlh73EXfW3OolPfoe0QW/OW3k/K5txiwxyTczDttDLrNqsxCH748OHAvZlqy1v20h9tsrGIuLKBSUHMctGBeYzKWqcpRt7aCOVEp2DB1BxT54jf3h6PbXZin56eTo77EKIGievXWkfvbU6dkia8eR66SqGBmpOvdSRwcwBOXs7DlND0ts98w0+fpXiumPW5m3oncpvoReRaPaGw+BW5Sd/h14pvz9/lQKq/uubblppvV08tRvEchAsOax1TeJ1uXN1Ml5Op6KxAuVudzFnJAZriPz4+HvTtxYsXJ6R79ySTEKaLESJyA6oLAZ46Og9JtKhX1e3Bhs55f8/XvOk4227U/CrD5kDbcM6qP0ekg2qsAomCwdTJ9Ky2RPHX19eH16lNrlh5N0+XHFflogN79+7dyQpFA9Kj6niMHBp/Ci/Br0CrN+EI0TMy8/uMTaWSd1BZVDijTgKdUVOeqz6LKoXdog3htimJkz8dmYGh/6+vrw/jDbm5ClRbjUEH+fT0tN6+fXtwHFPRRVAdsKdcdgsmzrcvtLi/P758ZCqxjsZl/F0aXKCT80hf5JDSAxcNGs/OOa21DoFSfUnOcWfJtu0WIr24wAzVN2q5Sij34747aY5+ZlpXIM6ZGSjV+fpvgJgLA83XjsIRtRfQknWyrS0XR+qf16aLHiaqLvm+hsq5R4PU+5ni78Y3y0UHNldPJJvlgybZPvP0Otp9KVQeu6g7oert7elpGA0q8njnoZtgo0qTpTHXJ9OOuTJnhJCEb+xO0kyr+rv25n0+//f4+HgS7afjlBeYRQOO31P2yclUtO+9X3Q00+GUU4OyDhF4Tiz0Y4pSEaHO1Fd0N+U7U9OdTGor5z/R0VpHI89JFqD7/5whnVtMqIhW4lnr+y51zGmdQ2eWaWPWVT1+P9Neg67/ZxcCiElF2PbkwGamJGem3tjutL9kL02SDKYznuVZDizIasVGqZub49HDKuqE9b3jz1zX9HMX9bt3F4kSRgNd64jYipIJy35NUlaeZubrfe+BilMOIqMdLzP3SmlMItvdxkLJ6oopqk5B5yhx3v86F2Vkn6dj8nVfa62T44x3fOFcMNEp6oBEG/XHdqajnI5DVDWL8u6aiTanQ1FupUfdL+ro944Drb+iIx2fXOeOdqlM3Z6r6z2xYVB2TvrteyBevnx5chCDMuj3uYfMu37qhsi4fmmPU5fV2eahLRY5su6Zdn0JgT17pHSPjNS43zlh8ghrfX4ShTxEn8VjOGnxNHpxvfmMjk6G0UIi1zpndBFh+l3tTCdlMU9XSZNVqEcHdu49fjocCcz6EqoREagks31XR7tHVBna9Q1Hzp/Bpvrqk8Y8OTxlKPkuag8dTaTXnIpm/W2gsqgTU8cm/THToK4/54BNp+b3s32pA+fEeawu9X7yXP6ee93SWU/qnSml3yevmSHNVFUHo3ynne8cd59PPrkxS2Hk3LTp7FYqxfZ/dArZ4WqleCKNySs1iNlgkdqXTuQUQm5CUl9yYc5e+zMNnATkhJtXV1fr7u7upD8Zlf30vuvr6xMy2RRSEl9eSSRnKRWaxlg7E9GpWCISU+/d/XOlTq6wdpWZc2FK56qWzsf+tLKavHN8Mw3LOLpmpqnTkU+nYsrn2Ezn5fuUi/3MgF340XHnvHaGIpmunNJzU7fQW2PvLeOV6lA2OZN0Shplku21sdb5Y3m8rjrnG7OSqci78czFg9BcRXufDtnFBPXQQOmug0nFqK/ya5fKsynk4+PjCdFYx4vENl6UTtlzPu/fvz+8fWXHbQiXPdTQlYzKjCYJaEJxFac3r0z+zlVAI7MIIYF7RHUIbyKBxmN9/b3jYWon+c1Jm0ZVPaKePpP49ycjmlyDXIXznUHWP+c+PjJCdvIlte/KnpyJDnnnvJPtfMOS47m+vj5ZVfa7OffN76tXr9Za60BjOOc6zFaAp2GLwnWWrYZLT3hYQK/ws6SD6kmf9WYo0zTf+h0V4/+iy1mm/ShTF8YEAZXJ06nrBTjrjgKR3+3+HH5ZRHXNI3oKMs1Zcp+2bbnowH76058enE6DnpOYADSqfuYKY3yaQumAtjrphDm5lSbZaOqqh5xXv4XMORadmMJ2goTkIq1+JOhDCabFOYlWw5TFXLAoQmUwTbppnNyMfQ7hzlUbOcB4joy6Oma6NtGkiwI68Zn6G5j6vEjsXHW/jrv7XWRwzirWa6o2FzxmeisNUTvNSXMrApkI0AUotzlMhJB+W9yG1BwazJKjdEmfzflzLP3d/3K+zZ3XKn+Lz0zu6qmu6tFGZirab1eR5TtFd09Px0NHAwn5g1Z/s4cfnUL2OqUic84mCC5CmMStqU3l/v7+s3OF2ruSo5qcxfT+XWskcqXFF9aqZPPN3Gt9Ijc9+C2FauIV/nTMk78TIYi+hOWW+lrffTpAJzJXiJr4lDpnkNJMB5nMG4cP4Zsat7teo82J184MVLtFj7XWCeJpjvztvLVIUj0ilh2qMFg05mRn4LD+r7/+er148eLwjkrl01MHtplD0bgbY/u+JPRFfOri5KJqt82cbkNpDiPDfZnIDKQ6JgO0gcGgsSP662NjaUNuNu7+tXjkm5ubg32VKlf3T37yk5OzwLQzUbxO2cMEQuqNpf7sFsYsz6aQOigdQgPXKdQpjVruyNUx0xWJd51TcNSIU9sNVGchYpirJimm0U3klnJMtJBiTXI4xTGKCNVnyphz63eIqf77Snr5pxlJQ4+mKi0MPD4+niDY+hPXOJ2rvFmKbtqcfHUa1W2Ri1rrlANJLrVR0djcuHzunYUZtfWYhoeukmmGdn19vf785z+vP/3pT+unP/3pSWA1ta8N21bXk/3kXE1xdGa7hZrms5M9Zpai/kp5zHZFayIzDX0XCOTU3Gya43Lc2lyo0cWgAsBcGIsCmKug1dvLgPQNkzOvzdq/VC46MJ1JnZvEpw3XYHDbo3CNivIia31+8oSku9eafho9hNF3d3cn3IxePwWznkl+a5xyfY7bFHUS5/W5x2XkSzL0jKxxZPAZb/U6yaXv1TF3LWfcRjZfgOv/yeX+/v7kOdeM6/r6+LKFjhqeK4i7qOi8J6u1jsZU/3IwyUwuzoAj8tThVZdviM6ZXV1dHdLDAtXV1acd7MnZdNU6TUO7vzrNOJqr+tZTDSE869UhXlp99cUiOtbZx/n/TCmTademX9bdd/6vA5w8pI5chF+/d5lPtubnfdd87NJO/Uc/cyHDctGB9ahFTsKKyvdT/Enoeo+rN3Vo8iF69IQmknHiMsra8H4VX35IQTrpprwapXxUk7lDEqIqI+oknq17OmBX+1Is29A5aawiqRCIaWApikrT36LR6u45xMfHx8MjSb1NvaJDr62+Twkz1lJkz4zSQTcXolcXcSJ0uz70OVNqjS80m3zUAee7eRCdNMciuuYup9hY0w/5zRYJSqfUY3V758Amqq3fLho1nkm+73grsxoX3xqrv/vbwO5LPbThybFpc7e3tyeOaTpCaRrtrgWM7CC7kBY4Vy46sL/85S+frZisdfS6VqxzUbgpglCzwc63vEx+a6Kv6p9p31dfffXZAW5GClODHexd6/QYj5CTixdF4BRDtCQy6neKnzFXl3ILmahIU+mc9LWOjlHkqQPoO9FDDjF5GlFVUtMO4b/z2LhCr24c1TFJzicXI7xFKkKdagOqejA5HLnCqWONMSQgZyfq1PAzJkt986j0/q/+h4eHk82czfs5NKasRGe1V72S67tS/53XybnpUHb2IwcWmura7DIg06KElJJBMjtpfuqDaae0jMFH9G1wuoTALm5k1RnIa/S234TStaYWRos4G8nwrvFvuSXTPduZijoHvtbpyqKkrM7Y8UkWCl9d9p9vpjGS7wQ8EcB08LuUqHHlTEMxOUxX2JLnJJJFl6LS7ptcibI8t8Ug+Xld/VFmfadcko3yqr6irY5NhKfyRhyn+M5Xvz19IsOd+pPBRTiXVqdzBrvamVxr8+pJLela91Vfbab7c6FkrXXyKJ3yUUdEYvaxOT+nh9MB6EyqMw7MPhusG3MyEN05d+/fvz/U07XxXnd3dwd05uEBzafy0Ef8VST+zc2nLQCtRlp5T+xXhNMqS0S5D9bWMdMBI5npZ+mH5+UnoBQ3YVZev359+LvUUUPQce2E4wpXRacmmqh/pnH9NBZfxCrRbNt97u8mX76usYQSciYquvyjjiJEZhpgwFGBdD7JSj5NZ6rcNP7msWNWkmeGX4o4FTan64KQfbUeHetME5OJNMDj4+PJo0UFY/ckGWSsT0dt/51Dx57sG69Ox5Rvt+FV2eeks7lzxPZOlzsZQ2eyy0AECNmIqC7ZdO38+/7+/vBSD6//8OHDSfAJLDg/Ux4ucj1XftBxOj4P2SRN6J1yTC/eMnUTngIFmU0BJPOtdw6kCJoRJRjTBceQ8e3SR4/20AA6nWAaqJtf1zo98bT0s2tzwBpCCEHiWm5kh2BcRNBxJkPhfvf78lq3XUyUMJGf6CYkaLuOY/IulfpR/aHq7otXM02f/UoHcsTygvJV6Vj9VQbOp5uEp9Opr9VR8JrOun4ml9l/nwnO+UodeAyUiyE5z12qONHSrpg6znQsFD5pIO81yBqkdGLWYT1zDtb6ZFMhXDMpeUFlkCxrt6A80fOu/KATWVMWU5sEq/D0rDkNCWUjzuy0ApL4E5EVNbtWnqCBWmd9Kcefu5r7W+egcKuzSdbJzYlITtMg5O66tnpU7sbaNSpIshZB9YCu5LcBoXs+fvx4QBkT/icfuaAM9+HhdPl8OuZdMT1xjDpoU2uVdyp5RmUwcq7td7zMTMudH+chvdEZ+WBxbeSANP7Qktf1nUhfVNy81Y/GXR3v378/OILkITJ2n5u8lTJXl8uc0ou4OTkwnYOLbSJ5A/+7d+8ObajX876bm5uTM+3sV+2qDwWB7NuVe5H5uXLRga31KR3L+6fIEnJGJxXj9vZ4FE6TI2qrczvYG2dmvf5dpHPVRIfn/wm5a1XylEln2wRNwaV0KXl1yQ1OhU1eM30w4iUHHamnX8xn3mqnSTc1ru7GJioTpYYypsOcPKNF7kaH1v0qnkS5gUnn1RMVoaOQWfeXdhgolcnV1fF44uqegWCmlhpJK7Iugsw03dQ9vYoKkcvRWL3fIDH5XukIOUr5th3XWzBKL+Y8Nc6JwMyKduh/IkznoWDW/fFdcqP21WzMfqy1DhlBbalftXt3d3eY+92KreXZjaxv3749UfqMs87MDgnnFcpa6+Q87LU+OcfdW7mvrq7W69evD2mlfEh5dX1oQqeTM8rUVyerqDTPMfK+lDNhZoB9rpwapxPW567uZQxTNl7vsco5a2WzS9tmGuEqabJpeV/yWMXx+TgdViWHU/31xxM25PAcl6nqzc3Nevv27cmYHh8/bd3wZRc5tEoBsjn7+uuvT9DguWgdreAWjWRWX+V9crrNRzvs5UYzavngbCTUbno5s5XpeLIr7SEHMAl7nfluv9Zapxu5+5mLMSInrysw5/xMKysBG9vS4eXgXKCwveiY9CnbyqFr4z+axO/5vAYgwepWAT16AyiFyJkldIURvzbhZhM1rxWhNDBX6pqAhDkXGeQIdpFLPq7/4+4q1ZkDlBfM+V1fXx8chcpj2jDRie3rtES4RqPJaRg5J/qr7pyaxLnc3URLykk47yqhpK8p7iT856rS7uW2GphGnvMNpagTBibpDYNMfXIxRe5Q/aivZRrNk0g5Z+XKev3NLtJjF5p2qd7kGLMvnZ+6airY/+m1pHvzWH0Gvh3Xppyau2Si81X/svWJeKUkpG2yj8fHxwONdHV1dcJzem99ulQuOrCf/exnB8U3osp1yInsnMMlGJjCxU+l3BqGBhpRGsFXlDZXdyHAFZI+s1+mYPJfFSNyY4hwlVeqf8pgpi8S8k2kaVvyWOv0BR1GcudAp9Mc9FnOsvFlVK0EunBR0Qh1HBOR9lsjnal1shTNJAMfo5m64SqmjildSEecR1Nv53Ot0xViHUaymCnvWsfFjzkHk5tKF/tuOhSdSg7esTYeOcv6Khqv7NLhyVmJzJKfNpl8pvym3Trn2sBMLWdqOudA52Zw9QkKF1qmH/H1fufKsw9zv379+qD4RqMdDE0gooMmRJ4l4X748OFkR7+5vfXZTnA+x5UQhdo5mSD9dDLznXwJVodU/6tnGnvX2l/J/hTbz/09l87X+vxRkNrqPpU6Q5ltiyJSiBxG8zBTUB1k81Dgurm5OTg85WFU1qE1/66yNeY4nMZiqpl+OR/qwOPj8YUfOv/qMJo3FklikZuBYKZp9bM+1f/mSz7IYFERac25nenfLrCb8Uy9cK5DgBp99lMb6cLUFecpG5pyaO7mm5T6bq11wkHOur3H9rPb5GyG0Pgl89XjXbnowF6+fLnevXt34nndhGY+LWLSQFSkJjyhttRaJx8eHg5K2u83b96stY5H6Ta5kp4qZ5OiIskTKOApZNtpvKa3Xdcyb/VVVGyJzOoWeZRCzXRjrdOHodc6TqoGP9FS9+/4MZ2qxqmjdaUsJ5eTmajANLTvpA50SqIvuQ6dt8hTJFL9EvWNs/mfafxMKZW96aHOJI5ThDZlIgrXeYuAXAyYhqeOuqdRB5huS37PosPpHm3KOtR55306s35bR2Pqc1FZ8prHpdcfuTPT1z4LYGQnBkLRV/Pxozmw9+/fr5/85Ccny+k6D1dLEsIkLDOeSbBOdGC6d3Nzc+C82hwX2sl4clJzmX2ig+rq+7liMqObRPNMI6ojI9BhyK9YNJjqr56MYh5kuLteZ6qcMxwPizRomAaaYpdGVTy+pTkRYWjg9bu6q0uFn0UuVAqivtRe10ruplteI09TmZF/l45N45efnM6hOiaCaV6tu9/Zinag3cyNpPKD8ok76iUdeHh4OByWWFtPT0+fnbw6edI+25H3875dJtLczHtyepMuMEiUPdW2Y579zE7bT3apXHRgGUYvdxDm3d8fl7mFnyKwHdKZPM5ax5QvUraJ7z55hwTR6mSObJKPMx00glxdXZ2c02QE65o56dU30WXXCYsjKdf6/Fy0DL77W4Bwp3oOwzSriY7DEgnWr9rXqcq3Tbif0k8C1XnKkc+d0Y3dCL1LOZS5xHAotHm3zmQtEa3DF42a2qaDptw629JAFyZyTqbH9VMkJXrcoVz7Im+UgzK4No+iL1GSSFYbmMauzWWTkwPrmt3znY43mUwHO23QOWmRzvnvJ73UrvIhUlLWL2rt+ea5cXyWiw7MaKpxhcxMSXIgU3h9NyOsiwApRntsJFibnKKLfNf9/f3J7t5zzsdXXN3f3x8ewo2LMtVMAXWa3XcO1q+1Tu4pBalfOw6t72vbQwe9VvK4FL4FBB1VRuqTD7Wf4+oRGpXx48ePJw7EucyImss+qy0J++SnstY3T6Oo3R6QV8bJsWvm0cRzDgyqkvPqRz+R89NwNA45TZ3adFoFqbaPGJCUuwhQ6kNnUXra/GszymM6zXRCRL5bSAh16/h2nFrjjqMyNXbc6qLHaU8eWqemHOub78NoDrN/D0ZMrufKxYe5dwPQoFReicCEMh92NWUwcjowHZ85tg8zGyFqy/RQI1jrE6/11VdfnTzv1hvCjURNkCjJNHlOfH2XOK8UYVNkjW2t02cFG/9uoUNCM4XUmRkBq1cZi6RUytCZe9REvrXlfqAUUr6psTdHIaScXUhLhKfSu4BgKpnD0iDUxRCqVET9DC0a3bu2z6yruUpX5aukDFwQuL6+PslMkpW63HhF+LWto1K+ORuph5nWptsS8c1x9esA5XUrs91zjlKn2BjSt/r69HQ8qUPf0JwZMM0IJjhyH2K2eYn/WusHPMydojRYeYYmzsghVJ0EpcR1hlkaZGopN6FQ7+/vP1ullLAvrdQp1bbpRdHKNlTY7tER7pxX350I9Pr0GTodm8qgQhipWpXNCCciqy+mCypWjtE0dvJ69aHnPUtl5cxEOyl5DknnPOXcb9uZJHjXz4UQnZHUgE7ZdiSrr6+PL/uYjs5iXc1xaU312KeJJqUS+hFN979/1568j3pqEDR4Ordd53cziE9nlvMqHSuT0SlMR++4dWrqtPZU6XrHojNf6zQzkqpIP0xlpwzOlWc3strpPjOvlQexs0aH6om7qcMJIgcparN4JveEwwqjCclZTkg+YXLXWxTWOac1eS1Lxmckl/9a65iSzCjU4y0i2Rx6xt7/IjJl4Oc+kC6aq70WSrreXeDt0hd5zu00zeHcujCdhKg2JyjfY9pmEb1OB3Dud7SAvOAMTqLd+jUPQZzkv8b4+Hg86DEDLAXse4NfdaozM0WbSHIubFSXMhJVW2fXpiv+nb26OTv7DZlPCkiH1+duaZnzWQpYX1ts8B4pE7MK+UYXks6VZ0+jqCPlrSml0bCHaSUQVXqNSnJa8j2Ho3ORELU/tfPy5ctDnxTY/G2krs/97AjZ7vHaSgrqNXOVq+t0XvM8sXlf450krA6n/ppihpR2PIEGv+PgMlyd2OSYvCfFcnGhfuskJcn7viKqNi1MzhLvylfD3a3O9btV84xWi5SRVAAAIABJREFUxJRDLBiKwGp7Oq7qVV9De1PupdDVIxrWuYq+01sDmhzqRNKVmfpJr4giZ0BuLGZKBlYXWqSFqts+5tSnXuasAjkF36en061TrponX4N66PtHI7A3b96ccBaeqW4auSNYp4FpHAksgtlXtwlFNQh5sdoupfz48ePhtff9SBAbeZqcXVpoWmTpTKPalCcyXer7tU73PRVx5GLiT5Kh8qjfGohk6Jx858h+iPx0jCmGnE59dBOwZRLWa53ucE+Zpwzra2WnjKJ1x6Zxhv5EYnIr3aMOumv9/v7+QJpPpy8yNPA0Z/XF7ED9LgCb8ubopuwz/LXWyeJShLh8ovLRiU75z7RXdFk/ldVap5u5nQPL7n/ft5BcG2NjmauT6Ygbbxur96n76eVzPNhFB/aLX/zi0Emf7VOQrrAoDGFlEz3z77xtRlNkqO6cUM+cff/994cVxLU+PwRtrj5l9N1vJMnhWeTOfDvR7e3tyZEmOotpkPM7lWo6jJzqXJHMeCby07BsL9kYlXU4L1++PKADebLuT1bym8lupvhyFpPTapyzeG3lHIqqjsmd+GbnHJl8nG0VHHIYE9HZnroi6ljrSCSbhjWH8YbprimaMiirEF26cFH9PhtqNtE4pjOxnbmoUrvKw7FPHnjOYU5vykkqo+9Mz0VRBqC1jsFEvnutY9rvfDr/c1P3LBcd2Nu3bw9LmyKYhDa9fIPqM7kyO7tbQaoOl9w7WVV47TYBlW3m1L6TcL6fsEmSwOyz2jZqTmEbVTNwlU+jyugnX9GkTQ4jY5M3kqh2HHI5IsEQY8YehzgVp7Y9SdPfk98y2k+jr5QiVNIHV5QnWvQFEgYkUwj3oeVATM+q10ChA6x+nXD9FBUp1zkv9dVUyvRwps6uWNZOemGaGYI0kBj4znGxU3d3mUCgwHRPp2OmUz2Cj+oITblZtvtnXwQPytVFFp2tKL2/Zxp5rjx7nE6IZ8LnJqcOlQJMUt98ujo1+CZ4Rpmiro6xFEvuSgEVIVKuhKDjrO4JoR2HdRqBd8IXPZkS5nByQqYcTYibXVPanI98RGWS4zmFZNhqmrKpLk9ldd50rPKb00k6Ry5QOCYXMNY6HuqnwTcOZdTP5Dgdw45bEyntFlZccHBVt2vmW+Ez5kn6Z9CSzD3TaarjVgKDj0S91Is6MPXY4vgs6obUS2OrPDw8HLIQOUfR0kzT6mtbMPreM9uaLxGdTjJ78r50dqa+Bp61Tncw/GgObK2jcXX0TRXWYdOu0E4w071XM80QGhZhjcCR+iIChZIx7RyOR/dahLm7RxRctdHYdyS07RlFHaOpWtcow84r11m3wU8Ep4PLKI36rR6tdTxNwXvjtnL+a51uvG2e5ziSrylbRquRq4wzbdDJ7pCZMmvOXHEVfYq0RDM50MnDzvoNRH1v0FGmE0X7v1xgOplckpX8p4sKpqG3t8fz8+rLNG51Sf2bKZ5o03HapsWFMf+fXOl8s/isZx4W2f8iM8n7eMVkICXioosZxY/mwOSijOJrne7+rjw9PZ28Xt2XgZrfZnQSpwnp3bt3J0StAtXoZiSxf99///3J0cQ62tLRkI6kfPvIRAwTNk/i8Ry8nwpon/u+saTkKn2G2PEz9VUDjesx8hZVMwaVR7I8Jer/SUC3nF7quYPxGotpskEoGYm4qqv/5VAdS85AZVam6ZTUgbKv/uZ4FnmW5sH+plciAw0zZ9O1bYNxT11ZQ/NRgMp4DXLN7+Q4u1an5ftau85UTznVj+q/ubk5CRYCBx8FSzdEbnP+07ns01XtUFp9a9zutH98fDw5bl47aB4ucWAXd+I3Of3doFLwSkY+lSQFaYAZh6mER/sa/eS5dqmobdvWWkfiMoNyVcQU1LRGpyG3U/vWteOtTFEbm/1T2VLAHJdONKRR1I6MV77Vk+wqpkwRx/PxKCH8VH63AMwVUr9LRn3vfCmDjltJiT3FVW5wysjVUx2retl4dK739/cnJx2oq6a8c9w6kFmms3H+RKbJrLlqA2mGaNDzWmWo05O/dB6mvs50t/7VJx/ZqWhHLuJYpAfm4oc23WfZbXan05po7+7u7qD781ip/i4bmSh6lh+UQqa4EtdF5hTOhhWCRtdg+mzyajlMV2EUQMKTb0t407GZhzsJ87P64l6VHYnf9VPJu+5cnq4xVkwlVczJDfp9TlVFs84MormYRrlbbbJ/54rG69g1yGmk861Q9cn0qHodg1yeZcq8+de5rnXc6lGf5AcNwHNOJlGuDu/Qp6jEa+W9cgw5JhFH+iUfOudnZit+t0P9orMdpzXt0wxiOihlNduuCAji2N69e3eQy66P2bW6r8437nQ9Wf1oDqyla1dg5IXM1VPS6RjODUShNLGetCrHYKqSIxVFyYXZt7WOEUayUchcmSuUOiZfItvPbhHB/yehH4qT9JVL6l45v+4THU4k0vL7Ln3q/why0yWN1986UFGk8pHjUfkkfa0rfZjOZBrvWqfnQF1SXOuv7MYy2+6z7s2JzCBZMR1XnsnUh8S9xsCbUU/UZZky2aEObdFMJqeZ4cvjzvsM5DvdFaT027TZ1U0XNSoz8E9y3yyhuc4mQm35gLkiuSsXHZgIJ35Iw9VJyRfU8TqlIGd+7sQblYTGveBjrU8vE5DDSlgTOc0xODESr0abCbWre24NKPKYtk5OrElJueT5RCKOvf06RrEdWtRw+87TDIT5KmJOM3nIr/QT0qoPOn35KMeZYrahuCIn6dyn/HJgztVMW6ZRJG/l6RycG0sGkWxy/AYZU+nSGOe576tbZ15fnZ84zPo/na78nv2uXEKjycW2TYvTaeU2aZKKXGj9enp6Onn8R6fZPFVElbYVOgtRJY8ZSEVccr1yxbvy7EbWtlFMIwwNCXONdpXp5UUyCn9OmoK+u7s7GJLbH4wmsxgBzME1ksZgSttY4srcf6QDn9B2hzS9Zipb5xxF2LpCKNk6Edda6zOHM+Wqo5skdqtKbSicBpNRejTxWscFgxRNRNO4vvrqq8PCzVrHbS0GvfTFvWqVUM2M6qbZyci9Y3JrXi93Oh2HNISOP3k0Rh3VnINdmee4mybXH1FIOiKFkNEaNLQj0WNFqqPfBujKDLRmLZZ0PbAwU33/ru/q2e3tcfO3uxEKCqLgrpmLGqbc58rVJe/2z//8z09fffXV+o//+I/PuCije5M0SdIJxxVin3uvxmCUa8BGFutyNWRX5qTtihD2hxQj3DkHmiE+lwpZdnzNJRJzyls0MPuzQ8JTnumDfdZ465NOxvbm3Pb9lMNON+TGpmPbfT6/n3VP7nYGyRyFqEX5KQsdVuMxwEz00f2O2f8v6cW5eVSez5WZRu445R9y/3P6vSvTV+QgDXjpz7mxvHr1av3rv/7rYfvW73//+21jFxFYe1SC9MFpXym21jGNqLNt8uu6mUrqpT1QUEeWwH1vo9DSiLNLbediwkQUIhV5Ju+rHiOU6epa62SLxUSbcyLll/p7clfB68bf+Gaa0d9TSSea0ggLOjoYV8RaGZroqvkXfU86QDpBx6YzcJHB6yZfKhqpHn9XREnqlosIu5StuZBgzylblym0lEaowFVJUdpsxzS2Mvc91vdQq46xupLnDv1Nu1CWM9Vba53svdrVY+YgjVAbOzvqeq+9uro6eRdp/XKMk7et7BZQZrnowO7u7k54JVdQNJb+V8El4xTiVOR56qb1eH3tTMexi+o74/KRqIkyrGP394483a3u2eZcpVJeKv8OPU4nJSc0nVSfda0OsZ3mKrV15/xEI7tnz/pOp6uhZ/wz1VH+19fXh1XCGWQmsdwYn1NeH0+azq/P1JnmyNXC9EXawnn0RJbHx0/bftoWoh7pXAxcBYaJKCevWVZT32aqPLnjxjNXFR3rWqdHt/u5KZ2fe8Jxpa0oM6Pob+tPVn7mvriZgmvfO0Q+aZRZLjowVwJEIZ63pJCmYmvUkzyf3te8v7pMDZzUuf9Jx1e/5vaM+tAWkAl/Jb1doNhdt+Me5uqmzvwzocN19L8cmGOeq51GbOdFRciQXCnqmhzEXOyorxpm7c7UWm5iprxyIgUwHZt/TwV1FcoyEWH3miY6F82fiDP9m8hcdFGf+zv+x/lNh9yW0BHrNzfHzdTv378/GZfFhY+ZpqX39UH+bs6LgXTq6URMft79a51/s3f6Egm/c4ITPc4Fs5Bp6LTvJq87M6Zk1PeXUt5nVyElGlNIjwiu4068TqxB6FTMxxtc9ekAvE64LnqZxOHDw8PhuOg+u78/vlrex3dm9Go88nkJV8Nz4jJQI6r37EpBIQdUPUYnUdo5wri+d33nzOfgVaRO5VSGtmEf5pJ7L1GR/yoqJ1u5DfufIWsQ8k0PD8cnHSbqyphFOFMfHh4eDpRFY3P+0hNRmQ6hx85qIz1Z6xOFEo1iMNbZPDwcj1bu713gMZD3v/X4+JTEtaDAN9Jf4qesP1v1c3VHBxgQ0AbWOn2usrmXilAm2d/8Ph3Qec/+ivJ1XDrWWZ5FYHVkGrkkoTuOu0bomnObz0WamhZ5Vbbum29KcTe9z2qlBCKo/s/A1zqekV8bprsiBpeBLxWNsLpS6EvCt0wU1r05aZ32jJRyCD4HGuzX0Zw7niTl0cmEXDIix+jbYuZ8pXyPj48nJ7vWhsioNtdahzZEQNNAHWt1uRA0VwwnJ1PRQAy+GtI07vqafohwJfZF6Mnel9JU0tsQjvd2Rt7kIuvPWp+jp8lnrfWJBjp3QohZwqRE6qfB0LR0d8JL9nt7e3sCICrNr/qsE1bG6e1fhcA8omZGjgav4wp91YGMwfy+Qelk5AmEmCrlXFXa8RWVmZOvdcoxGJmNfLVjpM1B19Y5VDW5sMlJnCvyLxq59+7I4drSiZgS6YT7LoPWmA0mPfojgjD9XuvzR1okmKejzSlrKKIJKYpK8jcNtoRO/NzVcPsWpyf30nWzXlPrnE51yNvO4KTDu7q6Ojj26dx3PE6yk4qRa3XM0wEbYBtzezUtPgfbeHY2MznedMxtPn3eOP2+urWtfht0JkXj32Zz19fXh6d9frQDa7LkCVR4YXJC0bimQIO0Rmjh5PTm1SMX5gtUZ+5uRPXvnJN5vxA5RTJ9EumY9jhOI2YrtFMelRkdU1wdomndLprW3jSGJtwxTHSjIdYfHWNz4yNi1d3Yk2F9r36Vci70XF1dnaSu9qk+VFftTS6wudhxSY5BJGwqKa820aeOuXu9Zp7X7/xPvtOAPOcvJ+T4RNlTH3YrsJPymMFUTkyUaJ9dOLDs6tYhmSJ6nRRKp4/IV6tz7iebXLhtFcB2jnaWiw9zz9xZuKxDmClmHfRFBxr7jjScKdS8NqfTRO7y78rs2/x+ktlOyuTTzvEH08DndpDJT+yQ285J5XgmEVp/i+6S6JPjqcyXi07npVHMvtR/EY99mESsvNNMdeaCikhzbpFw7vq+1cud4akPppDTyft5js9swX56/ySqC467VVIRcHM3sxOdcfqsjUyezf7pRGfQFlxU1FHvq5/q53TQEvc7BKSsSil3+lz7zeOswzHPlVqzinPlIgJLWRO0XlPDSBhNkkZoyjG5JSNG/0+DTFEmyadw1to/puGkCp13xjYdmbxX9ZxbpekarxXJSfArA++ZSGw64JlmOd5QZoaZgxBNzUin49DIRANyGucUbcq7YqpbH3fKmAxDZqJn5dC4vS8Z+zKNxrbj+WbaKXJ0QWiHCtY65c3mAkuy2p0a4vXpgqc22L/unfsW6599Sc7eP+fKAGXKOxcZrCfdya6VTfIqaE07qK3mI90rtXbxqrprr3uzv+fSx7V+wEZWo2mOp+cRPYFSj2xnJmrJIBNiAjeqeT64QlVhdVzVrUDqj/vFJvdj/zwDP0XQGU0y0bP7bWeefySJrJxc8dtxXdNZ6LCc/KK6Cy7KeK2jM6kPcxtH/fAdBGsdH+Z3kaX2+5lpqAFHg1OP2sRoIPS7eZ/OKQczdS6nZUC16KiTm2munJ/RX8SZIXtyScip8/Ez1l1pLPGq1TWDrDqn45r8pVxc36UDO1rF316zQzg52nRL0r46XKHNjs0ezBbSL1Pp6hPIVKSWfjQC+/Of/3xikGsd8/urq+P59CqGKKyO9N3T03Elsk6L5DxeZ66GyOuEFJ6ejk8FVKZj8Axzn4V0MnQuO9K/a43IGXlKkwOc/bcop36/f//+hDtxzNPJNPkZwVyhajw6vX5n1Mm6uaov33///cmr3ZVpjka0nHwcp+m+ZRLabXvIGZnOz7l/eno6KL5UhahAGSSbHKHjmA8mz5Rl8k+VZJvuvnr16jD+fiScnRfLzc3xjCupjeQ4V9tzaL2Axjp1Wt2v09ahVSTnRXpzcUsgoKyvrq4OWySenp4OutupHL5drC0pOTTJ+eo0iNnv+nQuEFiefRby+vp6/ed//udJR3Q6eXINXn7lHBEn11SZ/IppZIP3YdkdvAy2ioIm+moid32Tp/khJGKl6+d9/5v6dtzN7jtX0XZ8xuR33GbgNWutZ697rr/V0W8dg7ycFEN6JHLy95TVOZlOeYma+v8cJaFTnTI4Nwc7DrDPRLvWP+sqEOz4UVMzi5zZlMG5ck5m874fop/ec67eS/ec69Ouz2t9cm4vX75c//Zv/3bYGPxf//Vf//tnISPhQy4zndRL63GbtCZbIrq/Q0gp945fUdlEW0U9lSihuXBQX4u4O1hev6Zjtr+TyO23TlHObEaU+jFTuknsa3jzOyPVLvWyZIi1lWFX96zTYDSjuIi1ceuwRBv9L4LMYfm9r4Drc7+fQTU9qrhaOFNYkXGIwfHPtFfUKzeoM9Xpqgs5RykG06UdOv3w4cPJSunkhAuCIVT5pXTx5ubmM5Q79WkGAGXk5+r77vrukaoRjbfq6L3JW8S1C0rp1qROyu7kR8+Vi6uQbQ1olaEGJp9T2UXUKZSHh+PO5XPIob9T9rkQMI0qotD0QcHZT9PA/rZPE0nN++VNqkNoXn+6v3FK5F9Cfmud3zNk9Pca07HaT17nIt68dq3TE3INNClvbRvJ2x+01uevVAv5TDJ7pjwT9eyciGM3tfE69aI+ekprnytPdWDKZ6ZoyUSnbBrdWKejU87KU+TmNc6fqbo2N23nErKZiG/eMxHYDmHNldLsy/mfgciFpdm/h4eHk6cEtJG1Tt9idA4Rr/UDHiWKD5m7hWelRrY8r52fENjOeriZxphwjLQ5DxHY9fX1evXq1WcpTNdPHkA01vjKueV5GvcsOkFR0DklmG9qafx+phP0usm5eWJp3+fAd8XI7JwUzUU9OoD5hnDl2tw9Pj6evDHbMfeZr45zfCLY2p+Bz7RWWUsc6/ziY6pfuUyi2TpzfrPorKwnpzgNfTqu3XyutU4cZ3unlItjtJ9zsUHnN9uYqGvqW38LLGy/72cQ738DxlwgqGi3jdeFO+c6EFGdjf05SuPZjaxv3rxZj4+PhxXJILAE8FxJk3xz4C7x7tIsBd5vFWWXXniPUSKhiR5n/ZMwffXq1UlfvK5rJtRvTI1xEsdrre2zYVM+poQW04enp0+bg90QuDO8Wb8RMsWpr+fQssGqKOiCxySP+z4HEUqUgJ5ckavKBiTHrswMDo1D5yO62aVKUiA5pebOFVbTo3SkR2p8VEuH0v8GohnMplNpDlqJDCScG4MroPbZuZZHMtDO9uczkum2qWlOpb5MLq6/k1eLLc6DY/GzeSLr7e3t4WUfM8P60Q6sXPb6+tMLbl1ClVfq0DGv7/5Kg8qRuSIo7J7pXo6zfSQZg6toreapKBlWXt0fJ6giD1EdOQz7XKmNflqhSj7TQRs5va7/Ne7p6BqLqVqy0mAyvrlgsdaRrDfS7qKxUc/o3xykiDrcUood6ulRotBM9faz43N0LDvuTp7l+vr6YGS7FHoiQ/++vb09OZm2e5wbA4VpaTLoYfX+99Ep21nr9Awsx6Tjm3roXBo8GovptG9YbxydfOsD52sdkbz11kf1I9Cy1ilFUOYiB+aTHAWIgoAO2EUV03MD5lpHfu0caFnrB+zENxoryAxFwRkZ1zpVNHkbHVkCTcBzI+DHjx8/O62gOp3QhJ7Sud+lyTQ6hQB0AkaTacTm+n7ffc/tXH6uTGQ6nZPOpeIYa7dxFhAeHh4OJyyk8PXda/pu9l2iujloPk3fdinsrq75nXzKWvuVuMmB5SQndzb5xXOrmsp5UhMW9VVUJfo0DU82kycS+Wmk1pUz1UEpJ5H0LoOwHWUx9Wl+P+Xt3+lFOtUOADOQ6J4oAO2ndqc85iqu8p+UxaVV4bWeQWA5ExXcMlfhVIoi3+QKigrdP/P+hDDhrSmIaMKoPtPNwyBR/L73pSCVooD8TJ9NJz7Hf44HOMdNzJUfJ77J1CDP9dU6RS3TCQjLp5HUH52fq7JrnR5103dTsYqWppJ9LqFbHRNxpg9zFdaA5CpncyIfaGpaOxMRpUs6fB10bUeFZEwuJvnbMRUQXGVX7pPUzmALkNIxfW//pC7mnCfDykyP/X7qqd8bTMsestlQbbqh41N2Zks7BDWD1tTv5k20tisXHdjvf//79eLFi8MGwGlUwcXSy5yCXnOinHk6o5MeCjNfdoJ6kDtFLZWaAtqtak0+KyElwBxHfd2lcSpj/W9rh+jNuoXHlRTkkrNTkWtL5dPoZ13yJDsiuHpCAd7b/xrgLCq07YjOpjyMtNfX14dNs31WZE8nDETp0Fw88m03ax0Rky/5EHkXmNwYW/u+QLbr3Gm+MyL1rnkuBSuV9MXFE5UULHQY2sdap0+IOJ+7VHPqz0RdOaSpE7v76ufkkdMLX8qSoxK01GfnVLRrMGizvCXKZi7KzXLRgf3d3/3devv27WFvVUqmV00Z44s0kCJvDjBFTQj93SRqFKYYkoVGyhTrHPSvxFkk4F5lnhHJS01OqT5MYrbrMnYRTtxA/zuuHGVvbLEdHapKNIuKlUw1erdDNLZ2S+s8e3t39zrOc23vSvM4kZFzlUKG3CZ6zfGV7hbBnefpRDo8UKSXwRhARQ/xoqLNtY679A2icwFkBkqdRzSKKVAynKhKPiq5udDUok/9ukRJpAsFm5lymqLqUMx6qkN6IVvO7tW5tdb6+uuvT57SeXx8PFlxVjYGSVF9/WyedMLK7EcjsD/96U/rm2++OUnr8ohN8FrHZ+bmA9BCyj5rb5dIZxLas4gCJEmnZ7+9PW6cc4LkT+pvdZ1L41KoIv+MdO/fvz85akVHNlFExjGj5FrHqJgyzAne7ZlTKZODiFUDSi5zkURit+AzkaQOdZYcsKnEdJprndIJ9sf0IJmEsHskpXmbL4CQT5lL8fXBrSEFqqkblnQi1NR16bs6MNMfZVXx/16M3PwkCwOYDkjj9XnjKVvTfFN/dcEAZ306N9vOkalTyiW9/vDhwwG1yn1NzlEnWD3KLsrGk3C1nUsE/lo/4GHujHTmsxrJXIHzO4VXZPTUzR3H5j1zqXsKQ+OQ5J/GnaAymCY+dCb/1gQ0uRmAxjr70rjP/Z0B6PQ+fPhwwgeaxoYSVIRkMJ2BvFMp7Q497dLF6lZBTZN1nLbZPfVXOSTPDL92Uvh2xxsM58JM85l+yUv6nfXmoGdKGaLJSaUnIc7Zf1GBf/fdHPPMJpKPvGhotzpc/NHx7gw2/ZkpuRmDyNK5cTzagbSE+jMdR+NpN8DLly9PHHryrt3r6+sThz2Div3f8bST1rmEvtZ6xoF5TEkGPdOMGpaf8bpzRO8kH/u7dnQ4CnUikQzAtHWS+Sprf3sSa44qJVCoRrbJgVl2PFVjmumSf5tO6Kib+Jm6zPRERdBBhahmP6dDnPM4of7uuo6mnnOevOU4TCcKYCp37eZUlE9z55zPVUfHUv+n0utIvK8+TELcotE71omI/K1jnES6lEhO3PRJjrf7al9ubFIHGfzUP+dRdKeOp6P1TXRWW6JyAcLcKtVvAUh1zaDeuER/1pOvuITCnl2FTLESnNBeIRS956GEOipJ1oQiATg5IRHOTOWqs/sT0PT2rpwVOeRNJG3X2j/eoeJUv6uufj4dVuOYJKXjkn/ouxysZ6mLUEs7d5FbxyN3mHzmgsKuX93vfiGNpzHpFAoos4jeZ+pn++qW98p1Wk//zzI/k49zbBLpIpbpsOaCRfJ0y0864mEDIZqJnERgydM0fKInN7imn7vAdHNz3FM3OdtdCqcsZv/kq/1OGWtHyUz0W3+qR/AS+kxu0ij1NXByiQN89kDDGQE6CmWHsExvJsJq0kVV7W7++PHjye79JtDOTyLUlM9oP43Cvq21Di+ZKH3p+klm7tJB+akgtX26uTm+Nr3/J1oyfTX65IwnWpAYntG+yGa/krEOsQ2IboA15VReGo5zZX0q9UxVJ+cpyet7IUXDXdvYdryWhtE1fq5c/OycExGJaFwGor6Pw6yfX3311QmP2+el4cl4OgFRsplGfbq5uTlwi6auE8V1/6Qw7GPXhcSds8Zp2z51UGCK8sgxWc+0u+kY3YLlizwMCC0UxCmvdeQLS1svrUCu9YwDi0xtopo0vXswv53yrfBpCAlaxKRR9ln1NokTvq91JO6noc8J3K0arXVMf+ZrrHI8CW8+fjSNdxLdjfnu7u7knhTZNK/PTUWa7PYDTW7gnLI2TxO9mhrJy9hvjal7fbHpDjnOsWUMGfxE3aIR35ko/yIPZv0Gzvrk/+fQQ/2SE6wu05k+LzDrLNY6Hpw5g8ckvCvJS2epzHVMlqenpwMX1vYEgcNMDU1R7bOntVTPdLJ9bz/v749PskiX1J5z05MH6WMZ2tPT02fHqrtfs7ZzUFEnLYZNZNp3f5UDCyHNKC9J57VNbh7WZXMnQYMx320g07C8toP3GuTuNWHuMZlKJsyeBvr4+HjitGcRodUfnxUrgtZf+z6dz3SQlmkUGYER23QyeeRMZmpzdXV1OHxuvmhll3o5Mj7dAAAELUlEQVS2Gjh5kl06MR8VaW9ZJWMzEr98+fJANTQfBUJTinOUwVzdrvS6PMcYejKyu51ERyzarb2pB85/sqr4CjX77YLMRK89W5w+2HfRUO2EVJJ9Y3jx4sXJiSqvX78+XKP8nHP1SkBQX0wTk7sApSwmfZinFE89aXwuhKSfPj5U/wJEP3ofWM84zry8CZJvKorO975JVkrUTiXtf/fsOIntY1prHQjkBr9bKJhRMGGKKN1d7emRbnwVYfW/xl1/U94dST75ku6bqGDep+ObvNW8zjTDKCvKEPl1nwacce+c92wnXkanpfF7re1KC7gaKFepHLuuOucWHhG6CKw2Q1EajuNRttNpmmLtOCQRlePLyZjK72gJt+qo7wZ4EWV9VE593kkq3ZOTr81zNEBjlM/KCXqOfaW5ag5KvXeLac5dY2zc6VvXF2B9FrP6fvQ+sF/+8pfr3bt3h/9dls7j1pEcQx1UyYwaCaEJqiT4BOtu8r6fPNjMu9c6EsbxGkX53fWTPzFlmasrKmjtNFY5ld2YgujTqZWKnCumOipi8un+maYqdw3COtc6oifnqzFomPGeM7WQL5zp3lxUub29PXlL9xz3bqOq17jNQlLduiatYKrks6rTSShT09X0MSc9+SidtXMvgppZy6QcPGzA/ldn1zRWg5rtFtT6maln13ddTi6H1NMJ2U5ofT4JYDpZOzNF9bOZqZXeVvw+FFYqGsi4ZCMXHdgf//jHk0c+8sbuVl7rFL3IXWXYM11ygnWAT0/HJ/1nGjKFsEMsE5kY2et/pbYsjUlkVrqTgQm/p8NobEa+169fn6RZte3vnGcKqjHMdGSOfzp1ZaKTST72003Bfid6Sy71pWunrOXpRH2eYGBgmCmgXMqUR33wO0v6NN/hqMPwyYf+njxjv51P+bo2W9bePEUj5DL71/86PJ2JtMPNzc0JjzqdoT+h3v5Wr/xflOVpEdfXn07yKC0sczHjmkFBRDZ1r2C3Q2ei2dLi+ikYurr6tDfTx8Em/WK56MC++eabA7TL84ZqgueTG+k691tJOq51XC5vMKUtL168OJw+sSOPVSzThUrEX4hAVJXCTlQ20ZbOV6JUfsg+7NCd0crXY03D2qE55ZnRiIymcZhKzdUqyfSZDqWkM7obeObcyhX5MHz6EcoR5U3eRxnteCxT94lqRVCNybbmQoULIlNPmgvnqvrloWZA8hwrubLukwMWXU2Oa1IUjfHDhw/r1atXJ7qxy1hMxXTyyeTjx48ni0aVNhE3jnbUByIK6tlgn08klh75GFfOMbRcQJqLM/VfpCiVYR3PlYsv9fhSvpQv5Uv5/7lc3qf/pXwpX8qX8v9x+eLAvpQv5Uv5P1u+OLAv5Uv5Uv7Pli8O7Ev5Ur6U/7PliwP7Ur6UL+X/bPniwL6UL+VL+T9b/h95WY89BzyAGgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "show_edges(edges)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that the edges are extracted well. We can use the result of this simple algorithm as a baseline and compare the results of other algorithms to it." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Derivative of Gaussian \n", + "\n", + "When considering the situation when there is strong noise in an image, the ups and downs of the noise will induce strong peaks in the gradient profile. In order to be more noise-robust, an algorithm introduced a Gaussian filter before applying the gradient filer. In another way, convolving a gradient filter after a Gaussian filter equals to convolving a derivative of Gaussian filter directly to the image.\n", + "\n", + "Here is how this intuition is represented in math:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$(I\\bigotimes g)\\bigotimes h = I\\bigotimes (g\\bigotimes h) $$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Where $I$ is the image, $g$ is the gradient filter and $h$ is the Gaussian filter. A two dimensional derivative of Gaussian kernel is dipicted in the following figure:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "In our implementation, we initialize Gaussian filters by applying the 2D Gaussian function on a given size of the grid which is the same as the kernel size. Then the x and y direction image filters are calculated as the convolution of the Gaussian filter and the gradient filter:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "x_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1, -1]]), 'same')\n", + "y_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1], [-1]]), 'same')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then both of the filters are applied to the input image to extract the x and y direction edges. For detailed implementation, please view by:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(gaussian_derivative_edge_detector)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's try again on the stapler image and plot the extracted edges:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOyd224kyXW1d1YVz2w2+zTTI2E0smDA8Av5cf0EvjYsyxYkSJbm0D3dzVMVWaf/gviCXy4GqwXr4oeBDoAgWZUZGbEPa6+9IzJz2G639aV9aV/al/Z/sU3+fw/gS/vSvrQv7X/bvgDYl/alfWn/Z9sXAPvSvrQv7f9s+wJgX9qX9qX9n21fAOxL+9K+tP+zbbbry3/5l3/Znp+f1x/+8IeazWY1mUxqvV7XwcFB3d7e1na7rf39/dput7W3t1er1ao2m00dHR3Ver2uxWJRz549q5ubmxqGofb392uz2dR0Oq3pdFq3t7ftvOl0WlVV6/W6ZrNZrVarms3uh7darerg4KBWq1Wt1+va29ur7Xbbfvb29mqz2dQwDLXdbmsymdR0Oq3lctmOr6rabDbt+lXV5jMMQ/uhz6qqYRhqvV63459aseXa7nM6nY6utdlsuudut9vRuKuqptNprVar9t1qtaphGEbj4Of9+/f18uXLOj4+rtVqVZPJpHutyWTS+qmq+v777+s///M/69tvv629vb1aLBb19u3bevPmTd3d3dVms2nzvb29rYODg9putyN5eQ6eB3PYbDY1m80efY+OU57IezabtTlwLvNCNsMw1GQyeTTfHIc/22w2NZlMRv3xHfPx9axzzudv902fq9WqfT6bzWoYhloulzWbzdqcfG3mg26Q193dXbNh5MGc7u7u2v/YBn3TD2OfTCZ1d3fX/MVyZly2d+yU8U0mkzZXbNry6vnKdrsd2T76tkw3m03t7e3Ver0e2SU4MAxDnZ6e1r/+67/WTz/9VJPJpBaLxYPQbde9D2lXV1c1nU7r5uamlstlU+B8Pq+9vb0mgNlsVvP5vGazWe3v79f19XUbJMaPwiaTSS2Xy1osFlVVtVwua39/v00S0MJhENBqtarDw8P2/f7+flMSfVVVE8Bisajlctmcfrlc1mQyGRnDer0eGeJqtWrAYaUxNpSHwuiHa6JElIShrNfrESBst9sR6HkcNgSOBfA5FmPgfMbtuSAbAyhjRCc2HPqh8TdGxhzsHJaJgYGAgR64Nk4KCBkQkBX9MI9hGJrO+Y28DULIw985qEyn09rb22ug5+/44X/rwr8dQKx7BxQHR65pG9psNo/kbFvAbwBM7BVZMQcasvT/XAdZ83cGfdvG/v5+A1PGzv/0a9khYwc05utAgQ+hO/sz+pvNZiP95Dh2bfXaCWAnJye1Wq3q6OioKYWB7+/v12w2a863t7dXd3d3I0CaTqe1v79fwzC0vtzPwcHByBAQ1mq1qtvb2xHjw7FRhvtKJYHk7t9Rz8p0VLDS7PyOMjgI56AkK5Y+DD42Ovp3v3ZGMwzLq8cW+J7fNojNZjMCR+Z4eHhYd3d3dXBwUFUPwYBm0PC48zODQNUD68KBsIuqGsko+0CvZl6O8Hy2XC5HoGkGZdklKKAjZGHHQi7L5XI0TjuagxRsKuXO/DmGPtGJMwGuaxn6GtYt87fsbDcEfQdmwMB67AVe5IHebHOQE+SCbMyMexmGfY3rWRduDvb8bT+17p9qOwHs/fv3zQEZGIK6urqqYRhaCsh3AA4CWK1WtVgsarFYtNQQ9Ofvu7u7qnpgEkZrGzVMy8rC8Jk8qe16vW4gZzrbJi5QMDByPOfaCFGoGRXysdPwPf1zjo3HinGEXS6Xj9gcqTPOmnOz4mkJzk4xbDhmRT4v2ZnTx6p6NJ5M9ZyW+4exYi/oOFmT2S7NOjQAmhHMZrNmX5naG5js2A5WZnAwFc/XqaLBk76Qq9kY1yM1dFBNlpdyxnbIgAzs1gVpv+0CEE3AS4ZLqmn7xw5hj5a3/Yg5YgfWc0/uyMLM0nI0c/wc+6r6DICdn583FoRRL5fLBlAIdLFYjIwSByF/J+ozQRuBc2sml7UCjJz0ZL1e1+3tbRMQgvbEDw8PmzFm3cQOhmO7TmDnT1aWaUkaAr99PjJIA3Af9Etqzmd2MObo83xspmeOpuiPYx140INTOddfPP6qB1Zix7A8aTZQ5MS1nLaSZmbdC/316mFmo4wHlmWmhQ5sU5xvQMAG6A+bN9CiM7NrzkV+ZltZckjAMoOzn1nnZDro2UyJwD8MQ8t0Us/JXu23LlVkWm8ZYwvOJDITMHPNtNw+A4kxCFsHZo+ex1NtJ4DN5/NWpKcGRQGfKDefz0fp4f7+fkPzw8PD9hlO4RyZiOHUBwETsThmsVi0wj+RMalqGjLnGpBc0zHYIMy7u7umeOgzfSTwudaA4TqKca4VnWmgDZ/zzJjcp5kTzcECowZskC3M1nUJIqwNnjGahSDjTLW5ZoJLb65Z06uq5uD7+/vN+WwXmVagY6cs9G8GjpM5WBpos4Rg4EROaVtmXHl8D6Bt1wYFjjGbMwt2EDXLB1AzLXamYtDlfGcIlFMYF5kFtpCpHw37TaboudhWnM3YNjiP62cKb3u+u7sbEY5dLGwngAEiCKqXAx8dHTUw48JpEEZkGyr/54qUDXFvb+/RggGG6XEBsK7ZcH5GWrPATDOPj49HrM5Ak+yC87gO13RzkTeZg50sz03jMEuw3HpplqMov5mPr+Fg4RUmjnFkzboE7ABjtCxwCOTeK347HUuwTBZr2/DfDlB2fKfo/DaDZC7YYrL+dFJAwONCf54/bHt/f78ODw+bnu2omV4D3g5iMBX0jn5sSwTXXprMGJA1Qdm6t207hU9QYRxeUaThWw7CjBkdGNgcbLh2LhTYNpPR99pnAQwDOTk5qeVy2WpUTj9ub2+bMiaTSSsKY+QwtuVyWbe3t6OIOZvNarFYNAAAANfrdVsMsCHhPDYgxpqrfQYJxmNjT0YI+jvaUbdglcaG7ojr6zBet6wdZDrhdMRzNFD6x5Gc/qw31xVsmMzb47cT7mIIfG9AzbTHDIa+uV7qgbkzP4IJx6cMkk2ZFWZKaHkzfo4zGJAeuTnImLXBcviclXJSchgwtp61smTe/GSth7Fjh84oDHxmfslUPCf6J1iYHeOL9OOSiQMNJMHBGAywnrA3bBq5IWenmNb5MAwjJmiWv6vt3AdGSgfaIzSofxac7XwYNwO8vb0dpTZWlJkJq5gInOPSwblmRgbn1DacpPo2fo/DKSnRr9d/pjemyTaaBLKn6LDTNANbOjpzTibjKGdDpW87r+WRKSTf2eGSQZppmb1gM+7D3yFfM89kXcjILKOX2ni8lpXHjh566Zfn5mv0or5B0TpMZkjtq1fMTvtxMEwWbeaN/rIho96Kpo9P+3RAs+3TTEyyP1/LqaIZNscnk3Ogc7B04E2byPn12k4GxsohBfi9vb1GTZ3D2imcbvAd7Mroa8bkwr+pLQpwfcffcR6ACd11f1UP4GnFpcBt0E55cYB0ZloaoNmLQRS5eI9P0v8EFYMpyrcTGbTS2a34HAfAbBlwHXTAdwCSGabH6jmYOfBdpnecbz07WPTAqsdmLZcMPACEZcGcXMYweFlGBgenqS5Z2OYNAq5JeqUWO6uqxupprrEmM8/A6yzDzNnzSPn6ewcRrsl3Ls94vBlEHQBovfIK55Faey7WuwET2wd7evofzenJb6pGkfLo6KhthyA62kA8qbu7u2YALP86QlOIR+AUnrOm5Z25CAKlYiDug+twHkKB0hsk6SPrPTir04Ze/p/Nykx2xFjp29HYaSDjSDZmMPB4DVR2Ngw7nRq9IV/YWjqTC8DIiWuaFSewwsoMLICJdYhMkgV7/D1wMijZDpJNJWOy7DwfBxaag1nWIJPt5uKHUx8Dnut8wzC0Egx/u17ma9APY/WKtsGKcfc2AufYE3ATHA1sjMnbUlwPY/yWretptivGVvVQV3N5wW0ymdTh4WGXIWbbCWAvX75sdSscGWMkzRuGYcRwNpvNaMf80dHRSAjr9bpOTk5aP6yQmIJTkAd4QGXAkXNdxHZNwAyMMbLCQ57tqIzQXHRMoWGQGSWTSfg2j56SHOU5BwNM9sDxTgHzPDcb9q6Uy/TeUa5Xr3JJAAbpeojrclU10qXH7/SW5hQKmRjg3C9BygsGjNGA66BhmXqfHraYzoEzWgZmbpny9JgbY8JWPAbOTTZ4e3vb5ulgb3tkDhlgLAtnNS7yW94AsxedHCDJEHxdAgv9ekMxWzmQjQMu9XKu5+ysZxNmg57nrrYTwD5+/FgnJyftliAiDxM1InszK7+Xy+Wjovh8Ph8VMCeTSbsNaT6f13K5rL29vbq6uqqqcZqQ6WEWsXHIxWIxoqcsFU8mk9HKSVWNdmCn0TE/rzbZOVCcKbkZlh2Q62DMNtjsy7VFO0QaG/NmPLnPK6Nuj7liQOjLjoZMnOojU3bxcz1YBccxLp/rORkkDMxZEzHz8H40g4FTNTNpA2SuBNop7aiwSLMXdOC9XcPwsF8ui/DILsGf8ZCC2lbQP9ciozBztkywKcvRzNIAnWzZZZFk1AZN+wKlIPaAunzhQJJZkLfvpIwchPkuS0nuv9d2AtizZ8/q6uqqttttY0QocbVatXpY1UPEQZBJZ5nY8fHxqH5mug5bY/US4RtkiDxZXHcqS4SdTqej9BTj3N/fb/Nw/QwFomz681hcS/P8iGjMic+h3t7RbZBgXD7XNbBMkZyyeRc9c8iUvmrs7L4fENBh7mYwjN8rTTg97MvRFsM3++Ma9Js1FQMMc3df6DuvlyCFzLJOxzV69RkvOjFPM3hkn8CTfTE3gN5gacfzjnhk4OszB+ZpEEGOTr2YY+5R7DWzXjIZrosfGGQoMxg4uQbZGOfYD13isU0hb/sqLf2AHQzGiF1tJ4Dd3Ny0AWJIZjleJkbYRPGq8b1hHrjzaf73KkXevoNQnPaZCfJ/KotzAEgMzcDH+PnMqRsA19uMSP/+2wqxY7o5WveiEi1Tjh7r9F4fX9OO1gsUgJ9TIssNoLejePw4eM7TAczXzBTK10+ZZUqXjNX9OqjY5pAxn2eNxd/5xnOPIdmv58VxXom3nWJHZnYO9paXZUq/qTeDv5mlATvtyPPwthZvm+A4fA1Z+treymT59gDXtogN2qd61/VxPs+481Q5pOozAHZ8fDwSooGAWlXVw2M7UDTRgtt9vL8HRmSQQsjU0kwjzQoccfmevr03zcyolzIx5p6DpKPzOzfg2fmsIMbuFR0DQc8Ze0ZU9VCjMrsxC3SKwbE5NgMe/QDsBiEfS2riFJPzM5iY6XCOU9DVajV6kkmmBJ4X87DOneI5YFpmGH2mre4rm0E/WZB1ZTlhB07/bWtcv3edXqM/Mhb68FMZvEjjsfknZZqpY4/FOnBlQM55uObpzw1c3mmQQcgLRWZVBndfP1cs/9cp5Hw+b7vbGbipuj87ODgYDY4o54n0Il5GUJQKE4PyOg1JI3HBMBlR7jJm7CgVA0zB2jAc9WwUHJsRyuDisTDOHghxjNmTUww7stmiQZm5UeezA3Mdr2J5Xh47f2PMrkHxGSvJHAvAGFSdyjMHAAZgyBIAnydLR5fM18BO/+jRunTtKFtPp4yJIrLZpPXglMvf4Yypk7xGBljOQY5O4xw0HZCtP+vJDJMfkwH7GkV7jsn6H3rknkzGwW+uf3t7O5qrdYKNMN9kzLnhlz7/bgA7ODhoE8IpMEoKes6NF4tFu1fSAEKB2BM/PDxskZ5CP8+oonFbjydpCktfBiAziaT1vv8LQTqycA4G5J3Hdsxe2mRQQwGZxhFd/F3WVczcemluT9Gu5xiYq8YbEx2Jmb+B2mwDfXG85VE1LmjP5/MRezQop3G6tojzuPjLsSwg9Awa/ZmJeY7YgR3JTsw4uH724wK4g4htgU28vqYZO2MA0MySXcvlOC8g8Zn1y+8ERsvEK4TJmlxbxC5sh4wfvfoWIsDUwIlteA6QE/pg/PSB/J1iuznwcOcL43qq7QQw5/cMHMF564QLwTAMJuJVDhRhJuInVtrYN5tN20+GEMxGnG44sqQBcwyfOdoZfCxA03j6yRw/f9NnRjiPN5kZ83JqgKGl8yVgJyA8peg0Eo8JWdhJbIjM34ZvtohOuYXMY6T1NjB6HvRjoHDNtFf/yDk50PR+qsY1GYOQ5WKGyJiSjVSNGWemcAa7ZJjM0U9bQMbWhefpwNmTFde2fvIzp9EpY9tMruZyvOtmPj51Zr9MpuqyAr6cpCZLMdbdU+2zDAyGZENar9eNQTF4Iip7uJggT2N1rQTh4iywtun04a55JuUagI2J761QG7wVkTTaoMSYLExfwwzASnGUpQ+ajc4Rvqq6Rp/G6d8JigYvB4MExtQXfTm19PgdQPy4YcveK1YGPJqB23KzUyGD3vEGx95xlpNl42BlOZjt2fESiJ7SKc3f+TOz5dShQSNTTo/TZZgcp8flQIe+eiw0yw62W8smgyv2b7tKfViHll9mDMMwjGTDZ+m/XL/qgeyYiTKeXauRn2Vgm83DY3GM7CBpsrPJZDJ6QCHpJGDlpXHS0uvr6/a3H0joCef1rHSPwcZjZ7OQU5BQfQSGgzu9wYBMx6v6t1kkyDq9yaKwlWsnzLTJRuQ5VY1XlgAaU/RMr/hsl8PmvGDajDFZie3DINtLsWm5k91zy3m50b9ZS9aVnE4/Fcn9vR2Ghh7cB5/b0R0wbCMOQgn01r0L7AYy25CZIWNwn8zFnydrtmw9XsDY4Jf67QVS26jBK7/rgabT9aqHnfq5+t1j7247b+Z2Cgfi8zuL5k41XEcBja+ururZs2cj1OZ7lph5/phT0owyKJx+MbLZ7OH5YbAJanSTyWT0FAwbq4VkxmeFAIim2F4QsGHaKOnbdSEDcjIQA61B2IZkMOfc5XLZnhZSNX5Olft2FHb65BTXCzRERdcMPV6nBwYNG3sv4ia4pmzNajNYGNx9voNoBgb0ju1m0d+AZBn12KU3aOZCDHq3jAzoBhfrw1sGMig4WFi+2JUBxM8T8zFe8IEouH7FOWZbfp+E5+MV6iQK+AWpc6aYrmNbBtY3W1N6jLTXdjKw09PTkSFst9v26BuiBhf043BobEydTCb17Nmzlnoa6GBsTq3YNNuL9jyuJLdQIFBviEMAXPfg4KBFG54xZnruiGmgRcn+DDDnOyvCxxronDKgyKT7VY+jXzIixs54kb036HIs4yFNpi8vJiST9QKHHdkOttlsGtM2qCT79I5yp03IxP04YK3X67bXMBdUiNzWG2NC7r4jBHZCkduOSp/exkBfBiBkgA3lijZ9OX1KxuH+sGHLi7G7pOH+nSojL4/TKSJ/uyadWQf9JWNmfMwz2TPH+j5nxsRGZ9u4wct2aR9PZlj18DjtXe2zT6NAYavV/cs9eL7X8fFxE/T+/n59+vSpTdpgBpAwGK9YQRkxBvp0zQ2hg9qkoUQurmOluRaAgNfrdXuaBQ6Uhd2qx9sYrDAzJy/3c60sxDpqojgzBs8ro00aVNYUemzEwOAXdRik7SyOsE4RDQLIlWMMaN4Z3mMrnJMszHO0cyN77/Zm7gYkxsB1WOLHWWAGzhCcqt3e3tZ8Pm8voWFV3UwV2Tv1ytU1ZGO24NqibYInUDhg5aqe60EJMJY7/RiMnS7yO4Ojx2oWZf15O5TPR1/oAv1jS6ljMzGDEnaGHpgvzfdwGvSfajtTyOPj4/Z6MrY42NG8E9k3XvPbObejS1J5HI5I7OVtKz2jnp3b9JyIu8t5bBj87wjm61iAKD7TZUd/xsDfXqHx5x6XHdLjszHxf9J/j4PmiEd/7t+1PAMF35lpMT6DJ306wic78G/PO1fx0CsOzfV8fs7FvzMAJROkP/dpXTvdJ0D2mvs2o+6xlKxReRwp0x74JKu0bCwf24fHYLDytQDSlBfNd9K4+Zg8xxkGNpoppAmFAyHkxtkWx9ufnmp/061ETof427R/s9mMbu7lczsXEyL1QgDe6+PCfjK0qvF9XF7lwVkxPCvb4zZrqRrXfuxgBhPfuFr1+OkTdkrmkEaT4JHKdgqHQh19bQhZdyOiue+qx3UNg76Pca0xAdfXsEx8fA/UaHyWaZvtgf/v7u5aKQEwy9QG0E2wsAN5tc7OSF85H659e3tbt7e37R2oBG76z7cC5eJAysggh2Oa9fIdP5a5mZlZjnWcgdUsKetqyKIX7L1PkH4MrAn8HoOzHV/L4IVf0K/7cyZFWYdzfI0Mzm47AcxMyPtnEASAksjvG1tRHK9Pg2kBTp44fWZxkOZl7xSkv8/akql8AkOCh43B5/dYUwrWffv7NBLkyd8JFkQwp78GjwQNpzT528aSsvOc/T+LKJaxj3Uhm/PsVHbiHvBnaoAevHnSunXEtu6sdzuPZekxcbyd3amft9g4VXKKmXp2CaFnE7bh1LOL6y5p8J39zT+ue6VdmBlyzQwqtqGnUr9k9bmVwd/nmBiP02n7JHqy/xG4bDOWx1NtJ4AxINiVEXQ6nY523Bt4zHiS7mdx2ALNKOPr8T+CSlCxcgFRb7vopXw9YTqae3wumrulQbh+wfFOfVGGC5iWDwzVDmuwM3MYhqGx1pRzKj2dyOM26FFHyvcJZv/0lX2wQm255aZJsy+DCKzn7u6uPTzz5uamrq+vGwPy+N2n++F/O3Ev8AA+2B02Q6MgfXNzM1pQoJbam3um2rYj7DMZTjJFmu0wdZsA5jQ2QSTLG2kXBuLUjQHRPseY/ex/PyAgQZkGm82sh+9o9tveuNv4n/ymqhmyX0oLm2Flz06N89n4fTyO7HTOtZxkFEzcj/RAmMkk7Ez8dvTyUyucZmStwNHc5+d+oarH+1ro30BqcE5K7z64hmsFmWoBWAbZvIMh6zcJUDZOjNvRGACyXpPhIUszlGQkCc5PFXLT4bEHg4sZmY+1syeTwHkdCBIQEtjMeq0bGFjKNcfhwPIUO/GxfEfdzY9rpw9sIJl96jrnkpmLx1L1+Pl0zMOgMQxDW1DLQr2ZauqMsTMGj8UlKc+J822jri8+1XYC2O3tbXu6qot/+Swts68sHrMA4KgAcvMo3aqHVzSZViZb4AWoZiUGtAQvFwxRkGmthW4jtyJwoFwtZE7JrJL5GIwYrw0ABboG4LnDrjJVMdBmFPU4ONcOx/mkit7EaKBxBHa09EoRgG3AMIO0A2ZKkkwa8ExnZAV7sVjU7e1tXV9f183NTYvmTj8s9wSXrJHaln1eApfrU67H5vYOs8DUr4M2srUM7Dt+/A3n+H2leb4bx7hkYobo8dg26Td9wYBsG0pbcW3LQOssAqzwmDabzQgzfHO5a6FPtZ0A5khp5aKkXMbt1ahgZKA4A+JG8adSGjsejmBnTppsttADiaSijmIowyzLCwN2KubGuwFs0PSbMnRa0ktdzfDSIA1odqKntmykAXuOBiP0wfcGQa6RkdmO3XNMfjjHAQld8dt6ppkh+XOXEKbT6aMnI2Q66vFm/4zZ9+LaQbwY5eK9WUGvlOCA6dSV5mt43nZyZIW9eX5sR7IN2E7TXyw7Wm8RweUTxmN7NKCkjmxT/G/f9bUyi/B4XW/NBanUYbadAIaiqSnRqUHLF2bvUQKHBZSrI5vNZvTYHBSAQCwEO4cnxTh9nhEcYTlFyKLiZDJprKpnDKbrpMqM30JOA7IyDKLIxQrLVVaO8++k8mZhZmhJzc1qDfB+6oTn7OfPI0OzXubo1eiM1DmWBFunqskisw7D3wZkHuGyWCxqPp/XfD4f1dGoD/JQPvZ9ZU2R5k3Zh4eHdXx83MASZmm7ZKxOpf1OAOZpUPULMhycnXnYxvEJUrnMTlIu6RsOzJznOiVj8TkOZMgtMwMIRdqu7d12kKup+BDpc/pMLwD12k4Aq3rY5pCT86uh+O3H1SDIFGCme1yDSTsSuAaAsjP1wPncp4HQgGGDcVTl+1z54/OMILkiY7DIVCnHz/EJaBTPiUhOfdKJvbWklyIYAPjcoNUzrAwqnmeCuVlDPsguU1vPwf9jvHZWM2Cn3J6L5WHG70UWs7W9vb26vr6u29vb9oiWHmPnehyzXq/bRkvm53sNPS6XEQw0jNfjtpzp13LNYEjzYoj1mNmIwZI+vOHYNWAznCQlvXQ/s55svTkyp9zqlAyS67GDIUsAT7WdG1mpUSUi+34lWARRbjp9ePZPsikoO5teeVIFBoMh+J4pKymdgAlCr71QYOWQxqbDoNC8A97Xgd310p2MXHaKXmTzd+6PvhyBzcQyLfUbXtJozN6Yd9V41zvzctoCECGDvJE6jYkn61p/HmvWQtlG4+tnMPn2229rf3+/bm5u6ubmZsTmqh7uzzOIGeicMWCTf/rTn+ri4qLOz89bf7zbwfLAFpbLZT179mwEqsjz8PBwBLBHR0ftntuqB/Db29trt9GZvX78+LGur6/buEnP8KWsVRqkDJLWuedgXdAgG2aRtg3s/ykGh48yHo4hkOYz1czUM3txOSUXm5hXZhz5d7adAPbhw4f69ttvR9Ge90NikAglHxVNgRijNvoDgvyPYXv/mA2VSTg1BACdezsVQhimvgYV7zHJ1NS/mSPGxTyTRRmsMEqc1ODMXPI6NDOZZJoGZQzg7u6uzs7O2nVxzKoapeXJ7Dx3rsvfPN6b6/QeF4wxWn8OMhg61/TN43nfrI15sVhU1f1dIFdXV6NVUZ47lozWDspnfpY7AWo+n7d5oEevMGLXnz59ansWOX82m9XNzc2I6QNe5+fn7ZFQ9L1YLBrzOz09bYD3zTff1I8//ljD8LDC59u+DFjI6qnAbAbNOLOUgv56YMcxmU5WjVM9lxeyJaPCNpxZmCTYfowT9t9kuL2STJvzk9/U/c3Yl5eXzQB5UsQw3N9wfX19PRK8H2BISuTGwLwszFMiZrP7F4VQc7BgzPYAJfqmyOqVG7MZBMGenixSZ8rLeS40oiinZlmvoLmeAYglqHoFyPUQs5GnQAGjROnoJNMWp5UYX6ZnXpzIVV07iFNWp2+cy9xw6KyPIO8em92VavEYJ4ya+xczzQK82JuEUzDOw8PDEVtk1QZIOCsAACAASURBVJJx2Xb4jHsCM5Bhc9PptO1T495g2wD9PHv2rE5PT2uxWNQf//jHev78ef3www+1v79f//zP/9zmdX193V456PtemZMzCezCAcmgjq171z12gC3yeT7i3XZA3/v7+yMCQfNeRmzBPuSgaHaVcrJNZmbRS1XddgIYBulIihEsFouRUfOZ0zXTzaS5Rtqk6gYFRwanfPTt2lHm/tm350Dzo259XdN1MwQbDQKnmalxLIr28b5GppL8cFymlY6ymSrk+PncBpH1McvbIJtMwHO0EZtFcnzOyUvoNAexnD868hK6V8WclmXKk0HHgcINObh+ljVB92tb8Dai6XTamKGd0Cn1xcVFffz4sU5PT+vs7KwFDhgbm2QZR6/1nD9rZ/YBvmM++XYh+0nvGgarTOXRj/WF7DOg9/q2rJ6yGY7LxYFsOwHMVI4VHRogNAxDi0CkSmY1rLrwghAEYIBJp+PapCdEdKd/CS4GK85Pp8yaVwrYAG2W5GsZJK0AK8VzSQVZuRg5feNkOQeDvSMrzMlpsMdmJmLZ0Y+jL+w5AaCqRk5tQOEzgwNzNJNwqskxmb7SB/LH4YZhaIExgYLVa1gK+k1WlyybJ86SPvHcu2EY6vLysumMjdoEB2pdDsIA7Wq1Gr3o5Lvvvmv6mE6n9fXXX9fZ2Vn99NNPdX19XbPZ/YucKcUcHh6OHgHFtX7++eeWVkMkshziQGDGw1hdGkhfcGBzP7Yp94eP2Ab9Xa85aGem4RTSY3fp4O8CMJSEQOnc6SFFeYNMOuzBwcGovuVCK8boR+WQOg7Dw6Nh8lVu7q/qfuMtWyHs6Bxj1oKR8j1CxnFcEO0p1umEI0cyQC/ZJ+NwxOVavmOAeWR0d/QyODEGzjezoHlxJRmH2Z73F/UKrpyT16RfA7MjqYu1ZnNZ3/E1Dw8P26qgU3hWMuljsViMnIXarIGdrAIdWzdO/WFGMHdsh3kwB+xqOp3W0dFRu9YPP/xQV1dXdXx8XK9fv67Dw8P69OlTHR0d1T/+4z+2wE6pABkxrvl83lLQ58+f19XVVWNqdmjXsghEDurWF3+bfCTo8Bn6h/EDntafGaDPRS70vYs99hgYzfXcp9pOAOMCRnPyZiI4wONCIBNZLBZ1fHw8AhMizmw2a899qnpMSzEoPyyR5+ZnPu5H+vhzM7XN5uH5SVYQ4GdnI6ryeVJoxsuYHSX43M5tkE0qDSMwi3GqlBHIqUbS/F6Nw8ZhluexbLfbVrT3rUoOYG4GU+ZoALTxOg11pPVxdkrm5OgLSABIGf1hZjwogEyAv81syRhydQ17gS2aSXIeNk8fyBn7AxiZB8X7v/zlL3V8fFxHR0d1dHTUxoaM7u7u6uLioqrunwAzDEN98803o1LN8+fP68WLF411vn///lHQ8RzzrUkGCKeYWULxgkmWZXo2YJBBZjBvs2HrluOMLW6MD59+itlV/Q2PlMaAbm9v6/z8fFT7MhWn9uVCIw8/5HgKlhSeWXZ2VHNkREAYiCdsgzfTQnneuWy094KBGSDNtD9TQoOG+zXoZsPoM331fB0JzWCZfxacncoa+D0eF4LNMCwHj5fnvXkuZjLIxiUCF4bRL79xbIOB93khBz8D3foz0JJ6Yhs4f26fwUa8wIOu0ZcDGrLAaQFby8IBjSe6unZGult1zwAZA2wMVr3ZbOr9+/e12Wzqq6++aosLOOkvf/nLury8rJOTk+a0q9Wq3r17V8+ePRsF8+l0Wm/fvq1hGOrDhw/NT+3oDiLI1Wze99Bid7lSbrt3YHZQghU7LSS7guz4abdm49ZJBkmz9dyS4/ZZBuZn1OM0GJJ3ozNY0k2ilPNZHnnMAKvG+4Wy6OooyDI1q0NO/7Ke5gctonjXdegTI0OgMJuMWB4rArcSnRIhIy8c4KgYTa+gjJzW6/Wo/sNmSq6HQ/A3BuV6AmPKCGrGasaGvgzeyA8nSCbDylTOw4aeWwD47bqc65H5Gdf3HjU7BLUkv5/UwM6+LYMScvIWDqedfI/deAwG5M1m09gSDKaq2s5/p1JV9+API7u8vKxhGOrdu3e1XC7r1atXzbb39/fr4OCgjo+Pa7lc1tHRUZ2entaPP/5Ynz59qoODg7q4uGhpKds4ZrNZSzP9OOosmwBsXiBBDta37Z80FzvLYGDZmYVZvujPmUr6AZ/hLw6oT7XPbqO4uLioo6OjZtQM4vDwsBUtnSeT12+32/aIaNNqp3eueVU95NuODtyhDzCaVXE90h4iWjoEwjFT8RNkze44Pumuo9l2u22pjMHBzuclbLMrMyMzMCvaWyiyoG5ANigwbvYymT3glIyBMRlMXMNkTPTJeLJew3iqHr+ooapGoE0/rkWalSJDflu/NIIM84eNoScHIDsNxXjXw5yiw5KTkbgvGJeBAXvE/vARbBuwMPDm+ewvY7vS6elpnZyctD1rk8mkAdfbt29rtVrV2dlZK82wOXZ/f79OTk7q+Pi4gdzV1VVdX1+PanWwWfpmjGZeZkS+A2C73T5izLZ9znXtmPNMRhywDXo+z0FuV9sJYFdXV/UP//APLdIdHx+3aOQagCO5gYbVE0coJnFyclLX19cjtAYE/axs00/vITIjcGRxsdVFRxpGRj8GLCvQ9aSsl9mxnFbSn/N7FO6UDZBwAZr+vCXEaSbzc2rrAiggCZuyQTEPpz++KwGdmLWiV9f47JCurxn4MkLDZrxJmbH61jMbu42eBoAfHR21uTFGakoHBwcNOBgLTjubzdp+MN+g7eCCrj1Ws28vslhvyJAAhd2TUuamWp+DbCeTSdv8enFx0VYlT09Pm+xdGvnhhx/aOyo2m039+OOP9erVqwZoe3t79Ytf/KI+fvxY8/m8Pn782IKJ972ZgQIc+GSv9uvA6e/Tvw08BqZeuok8sRXOcbB/qu0EsLOzs7q5uXn05mX+dormHJ3Cb7IaK5HobIpLnxgaRmNWYzqcubWdgdeMZb7PHFC8HcWFYzsw83D/jj5mDK73MDb6Npuwo6fCPQY7t2WdBVI7Ps2prdnnarWq+XzeAgf1GBu1U1YDCfP1TvVkbQkgzJ/xeYUUufF4JesEmRnMr6+v27gODw9bUONFy1dXV3V0dDR6DA6rmNhCProYMM97TGlZq7E8HLCcxnuzrIPiZrMZZRV2/JOTk3bOer2uT58+1dXVVZ2entbx8fHodqw3b960LSSHh4f1m9/8pj0hhfTr48eP7ZYsxrG3t1eLxeLR28Hc7JNOBV1X47fLKlwjg7t91pkEzf26+YECT7WdAEY9BsUzANgXEWm1WrXVlaTM1NAANaM7BmomAUglwMHwkolZmE7rKC7a6Lhm0liamUQvtWGMBrMERv62A8NSkI2VbFaEAfTqWTYCF0ETQNlTlICIoRmADZ5ebTRbY96ZUuXiR9YzDGLJGJ2K5XfJ/FyDw25gOF7c4djT09PGuhaLRZ2eno507LphgqzZE5/5YZ4OHPTheiJ2z5jNYqjrVD2sFGKDvkWKhZq7u7tWM+M5aK4rAmAGm++//7751DAMo9qmg7RrdC6RbLfbR3rFLjjOoGf7NntifmZqtnXvLOBz+xCZixftnmo7AQzFoYScgGnt7e1tGxQC4jjf9+i6FxTRiwSkkKbdAKXpKRGXyNa7WTujJd8jeM738ZxvAbtWRX+9p3FkDYw+AVQAyY5gWVY93KfX23Fv6u0aG30mS3VaZOBz7cf1DRd4TfUNJq5XmP5nAPKYXOfwuOnbgMb5jviWE3rM53hhr3zOlgsclgBsNoLtOT2GiWJn2KGL+Ni4a3ouOzjtyRLGer1uK/n2E8bv+4uxadJL2vX1dWPNV1dXNQxDe9DjwcFBq61hs17UMljZ5mkAtu3KunKzbbhea7k6A+L6vrc2a8UGMca1C8Q+eyuRC6M0O6DrII6g6/X9ex57Gwl7Cvb+JTt/1jQwMpRg9OYzRyGzRMbBdXA0mBfnJ1U2MPGZ01aub3lwHM5oJ2bezI8GQPZSTR+DHBPc0gAMoBxjZ5tMJm2D8unp6ei+SoAgQcwvZOnNx6Bm5zAL5n+a5eSW/7sUsd1u28ZVv4EdUGBM2+223a94dnbW9OsaFsd5QQVg5x5H33NrVuLyhDMDBwfbjOugvPWLHfmulwGk9ImNzmazOj8/r9VqVVdXV00mrv9VVSMBNJ6c4ccFMWbGYx155dElDNsl+ve8TDKG4eEOD9uAgd51PeyN/nspe7bPbqNw6lL1YJiHh4etSMnAKbxhAN4X42Vr+sMQLRi+M3MhCqWQMjLjBCwm+DqMn1uTZrNZMxbXyugH40nWCEAnqBr8AFUvLgCYzCNTWAMwyua301Yr2ADFMWaKPpaxoYvJZFK/+tWv6uXLl218BwcHdXV11V5xZr15E2o25siKmufcW+ww8PMdc/ViBee6+O95uaxgQ7escBQWjTabTT1//nwEMPSfhehkpGy7AUyoOTF2s0/qc16hd8pf9XB3is+Ddd3c3FTVPdPjurYH21+PqZrlbDabBpLYNduReDFPvi/SaaYDie+ocfBGL7Y5rm9987fl4NVfB0LfyP5U++ytRDaOpKGuvfC3ayeudbULxsqgWYSdjyiG0vjMqZUF4H1qNhYzPIzUQES/FDdd65tMHpbbMRpapnXMzWwU5VY9fvVX0mPP0SkW5zo6wpQ8PrMhswmcx3ufvArnms9isWjghXPDhh0IvIpGS+Zr/VgWXuVzTdFOmYsn6BXgIM0irTs7O2tPZ3VKY0eGnQ3DUB8/fqyjo6NRMDFbzBU41wSdBjkYcH3qV2xncTptndsuHDxJB52WU3P2vaD04aBicLctcB2eXLu3t9d2EphY2L5dOsIv1ut1XV5eVlXV8+fP27alTBfN2O3b9hczuyxZMG4v4DzVPptCYgB3d3ejGtJ2u211AO8cdiRysdI1BxzQxfIERFa5fEtEbki1AaJIfnuFygrKFJXfXvFywdgKtsNaqJkGMwYUaKUmi/DxbmaVCQQJ4GZejBP5OwV0X0RSpzzMhVW7TAGQfRoqf/s2MR+PfRCAevpzemHDxwaxFRu4WZMdiOZb3qx39ideXV21epFBM9Nxsy7mZfYDkDFOAAK5ARbICPB33RKA5BjbDVtEzIZIMx1IzIwAOAOnt3CQfuKj+PJyuRzdjpWB9ujoqOHB2dlZXV1djVaUuab9lPPtP7lKb+bmsscu8Kr6DICdnJy0aDeZPNyL5tUkFOwVRQaCIfaOrarR8rYZDiyjqlp9A+MHnBA0Bs9k7XR2cm/uzDTWmzmTPQAGGHCmPhiuFzF6YOSUESW5FkFzam3GyXiyuI8jEBC43QXnxjidrjKvTGNxRBu6n4NlVpgG53l+7m+uVfV4g67naSd1NAbwXf9E78yNvjl+Nps1HRkoq6ptBKUe2GPHvTGbTWBDw/CwyZtAzbnYEPbthQjbG77GNUjnAMHlclnHx8f16dOn5nMwTM5BVhlo+Q57YaHDfkudjLa/v99ux7q+vm7bnKhBMj8/tJK+bM8GdTNIl4ws65R/r+0EsO+//77evHnTnPv09LSqHvY0sbKGsiys7Xb7iK6u1+vRSxJgb0wkC48WtFka1Jec3lE5ncasA+PxKpady6mRQcQ3idvgYDkGJafPXNPG6VQyI7znQP8ev+cDqPHoYgyMVMNGVPXgdI7YZmzJ5ojcTlHNcAx6lr9Xlawbmtkn51sGWQ9Dlqkr25prdrAZF+cZr5+wipyXy2ULktyLaF2aKWRpwgtSBl6yExwYwGGMaau9bAKbdzoMC6Pwb5BKX8gyDjLF7j9+/NjqYIeHh83+DDjL5bLevXtXt7e3dXJyUsMw1KtXr2o2m7W7A/JuGuzb2Q8Bw3sBrVPbFw286NmQ204Ae/PmTauRnJ6eNmpp2r9arUYbT512sYLDsnGmAoARrMJMAeMiGniyXuYFRABFpyN2AuoIRA4Q3quVAAyRBCaSt+44V7dynFrY8M18cAzYDTLJ9LPXsn6wWq1arcXsFyNxVHN90SySMTJOy4yW6bb79Vj5HsfKupGvb6flXObo9NCyM7DgeDTX5rhe1pw8B4ISjsuGUFgOqSaOhkw9T2yEMTv44MSTyaSOj4/bvY5mzHb+lIMZv4OxbQRGw4/3V2G7DphXV1c1n8/r4OCgnj171uyG+azX67q6umpBkA3B3GAOO3P24OcAVj2khvZ1Bznm2JsLcjEL3QVeVX/DrUS/+c1varVajZ4hXlU1n8/b0izH5uOEnUPbgZKJZCERhmR2ZgBhoplmYIiO0hiI06JkR3YS0kX647amnmNjKIwP4/OmPsDBgIZBulbkWkFGUoM6RkG9ItN2Agjz5d63dEQDBnLxahZj4Xg/+SGDBNc28Di1ckGY79OouY7H52BAKpzjRX4Jyg4wLlG4hpuBKsdPcAAwE8joxw6HA/o63gOIw2KzXnEkEGLz7t92QmmBMVDvBfS4HrYzm93f6D2ZTOr169fNJnkNHT43nU7rxYsXI/tjTs+ePWtyxZ8yuFSNbw+rGr//wAtiBGDbke+19dgzNXX77FuJQGQuArCQurgQicBhT46WueLEpMyETHFdTIWdkQK4DjeZjG9Q9ebYXj4N2/GtJhnhHDkcxcxkcA7XOBiPU72sD5nBslDh9M51LWRkA+FpA7y3EIA1czMrsKE7LbWBIGPvSaNP5gaDzXQpa3j0zXnehmFGgVxh1NiCU0/rDRnaVvJ9hU5BAT7qpb6vFpChDmXnZ45szubm7PV6XS9evBiBrhkZjBpQNeukT1gYsuF80jNqgOyddMD03A3wZrXMnUxmGO4ft3N0dNQeqEhN7ezsrLFM+yV9mBSgN1Lh1WrVbjF0EPZDCGwPZsS5rSTl7ubyzFPt6W+q2ltYLCAUYGQ03Z9MJm0ZGNBBMM6VneIZvYnaGCHCRBiO5ozJhUyDAH35M8bumoZZFA5FfzghhoxhAJT0x2cGTTc/VdardgAAn6Uxut4C60I+Lu4boK0T5OjVK+RFsOBaflhg9olBZz3Oesi0z9tnDOTog7EAXtSNSFW4jtmbzyWAUBPFSQABxt6L7tS8HFi4np8+nDW+ZIlmY/7fdUSA3gyyVwvE7ghOs9ms1ZDRkeWXDNDBydd++fJlO//Vq1cNtOjH10/bQy+M2aDNM/Vgq7Yx692lGT++241+0JcXBRPY3D77OB2UxSQ98clk/MAyp0YI1I/c8VLwMAztyQJ2gru7u5ZX85YiG62fBmBHw4AcyTkvU1evrDn/p7FsTXTGQG3spIlmDzi0jQyj9THZH9c0AHgVdbVaNSbsTbqAr2+poiVzcYBhk7GPz9uwnM44fadl9DfbdmBzwdy3ySB3p0w0O5a35tiJzPy9agbTmUzu97VdXV2NUsWqavbl24YMNMgeIOMmazbCbjabOjk5afMw+yXYAHyuBSJTb5PwPEkHXScDHDJbgCQ4UzH7QZ4soE0mkzo9Pa2jo6O6urpq5RnGYIDMmqV92DXEy8vL9vSLZKYG/txGQ7/pe1laQK4Jdm6f3Ynv1INO7TR+LRqD8U3R7Dfix4KA1fihiQDien3/wEFABLA0KLi2RCNyp0OQTrh4iFGZwnq+rjVkbcrOZGCyAszSckxZ3ESheXP33d1dffr0qT1WxWluj8VmKst1zcxcU0i25TF6C0cvapqdug+nyujHmx3Nfp89e9YWIy4uLkZMLvcdegneKQkrVjc3N6MUnRqTswDXvnKnt+unaVfeX7bdbtt2gqyfMT+zI2zEvw3+yZ6ZK1kMQWC73bY7HSwbN5dlbB/DMDT2z90LWV6w/fL/s2fPWjmINgxDe8BoVTXmnI8pAlgBW+zZxOIpGaMPlyF6bSeAffr0qV3EgjDLQeBmNdB5106YOFEkGQ97zKrGaRbXzpdt0I/fWef9ZVnv8SZLol/V+DYS03+ie0ZTfp4SKv35cT1mjsir1wyObIu4u7trb4pGhgm0djrXTsx2YUYcC8N0ncnpkJ0bxsYGUK5F0GHeTgezJXNaLBbtnr7pdNpefmGb+umnn5peN5v7bQRkBVXVtvXwyGVSSQCKgGlgQLa91gtifM4Y+I4nfrCd4ejoaGSzZg+2+wQX24TtDV34/8lkUmdnZ6Ogjw5Ikb2Axrj9FnUH4awzbjabOj4+blnKdnt/3/GvfvWrVg8007dfZNDkeweIZFMORHxP8KPOe3d314Cy13YCGFHfkQxFsnPX6ZkNFOFUPdScvFGRBiuyAFz3MEtCKEzIr2mremBfXN/Cqxo/Vsbf29j8lnDO8fkujjvquk7jdMlposGEc2im21XVdjj3CrkoH2B0TYjoZzZXNWZAMGiPDTn6NiWnLRg1x1l2Pp9rpNHaHugfpkGK5JrLbDart2/fVlXVxcVFu81puVy2RyjzzDfmYsZMgOFzs2TAx8HUTmimy7wc9OiX7OL29vbRhl/bi3WYrMlyMpv3MSlP7Mf2jrNn+lb1sErrVWvAkuwJ9juZTFpKCGvzwxCsZ48bGXtrkO3LrJR5gxnYpJm+meeuthPA6DjrEQgnaSAKszNxHIKlNuF8lwhB6omzZX2FCWM4LsxyXRAcQ8w+cqk7o6Gf9e40tbfiZgUY3FyrMBAY0BxtnDJ9+vSprq+vW/DYbDajyJdGQb+WddY1MBhvaWBsLAr0UnJ041Q4mV8PhB3EcNh8AYdTKvROkZdnzRsAeDop9rNcLuvDhw+tD4M9fWJ37BhHNpa9deNFImRDc42TOWHHBGE/GywfGOjgkjUn+s/g6M8Zb5Yc/HQJZw22PY5nX+Zkcn9j+ocPH0Y1RmRxcnLSbGmz2bS3hrO6yebftCPmjlwT8FyD7dVJuaZr2Jz/VNsJYMfHx/Xx48cR2oL2fv4WF+U4BrTZPDwOx6kAxoxzABrUvnxLDsbOVgqAyuzMTuoHIRqkTFMdlR0BMHRTecAui/AoJCOjvzcTMgA4zTBr+emnn0abEZlrGgSNiOc0OlkRzfKnIWsDOWNkHsjSzuEU1XP2/AyyVY/rTVW1k+3SvxmyX3BiYKiqthoJeLgMQb8UrtmQ3dsr5zkmg8WmPE6AC/tnbDzx1rLCLpwmZxrl+fp6GaiwSYLVdrttwMO+MH54o9h8Pq+ff/65rq+vq+p+kY7Ntdg6jNJ2ykIA13dmhQzwJZcwHLB9143rv16sMo64frir7dxG8f79+5YTOzIjMP4m2lFwB9SIaAzE6ZtfUupm2o7SuDb9u/5W9ZAeUaB0+oBzEU0QmlMEjHpvb68plHOsKIzJzMlGBgC7ltQEPRlvSnRhcxiG+vTpU00mk3r27Nlob06OyUAGs3EkdNQyUPo5Wa5tGdCqHnaAI/cEL+SC7JKVmr1a7rlSyPeLxaJubm7q6uqq2ZD7dd3RbNm6oE8zu0+fPjW74DhAbRiGtsuc8oiDHk8yJfhyrhk+4zPrBjBYTLi9vR1tL3gquNjRXSvMAMwckYOZFzLzEyZYZXXhntrSN998M7qB++rqqo0fMNlut+1uGNcDzR7zMTx8n7Uu5O5tULY3xmnmjR1m2cltJwP7+uuvW0emmC44ujietSuci4FTgL29vW2Fdr9k1EohQnpVxStvXMPpYjK8qvEGShtHvoSUYwG3qvHO9oyk/j5rXfmZa2t8xxL9fD4fsQGYqKk18k7Hyccd04dv4aHZGFx8R79+Nprn52LwU5+jt6rHgJZFYvQL89ls7gvh19fXTfbs+vaN/DSDsJuDCc+h415cxn58fFzDMLSVT2TgzarOIkipncoSsAmOZqDoz4X0YRhaScAszPNxiuUCvtkec7O8sw7FPaHc/rNYLNqr2tDZ2dlZe3mI33zk0o3rebPZ/ctQuF3Q76A0sHl/on3V9sln9O10PIEappf2lG0ngLGR1U7L4OxMTiWJdigfw/I9f9QMiH5EO4yCibu+xaTMWvidk+Z419xc3CUFNsg5/cT4mJtBLOVho8PIs+blVA2DIVImmJox0J/l0hSnPTnclGsgTrBwHcW/XYesekg/PE8zMKcb1oNlVlWP2AGf2Y4IirPZrL37cDabtYcq4vTffPPNCBgSvFwewKYoOWBnwzDUzc3NyFaQWwYzxp3bB5AX8uEz7/BH/k7teZoDsj45OXnkvIyb8SJz5Gq/Ys7oA7n827/9W/3TP/1TA4nDw8O2SuqXtuAT+HKyZmzQhXXS06oalWmwS78NCxtJv80FCj5HhsjX51i+vbYTwPb39+v6+nqUe/u2DDMdWMLd3V2dnJyMHNSDdjHcKxVMNiOua09M0A+LM1IbaJzmcG2YDhEXg3B+jgGnoaDgTF0BboONU7eqxy/JgD2RrthJmL9l588xWG8ehgUzV9dr3JI9M1+Dpu/9pD/6RK7YgAEr62C9yNk7PlcmOR8g4/P9/f168eLFiJE6rWSM9FX18EZvR3wa7A+Gwooecs3n1bsOmgEhU3HYtlNu+t5sNnV0dDTa2gCbym0wXNd2Y//IVMt74th862Bj1n15edmCB3NwiouuATlAnnsq0f98Pm/jN2C5D9ufa2XMwXOyHHfVdGk7Aezy8rLevn1bJycnLSIwcLMoBLler1u0qarR27Sz1gKK44xMzKkXE7YzskseRyZPz7qOC7agOUbCd7AuCwgDclQ3gGdERvCu7xBBHV1JB9nV7F3gnJ+ggczNopiDr5spIaDOeJJJ2aHSWTw3GtcC2M1S8lgfjyzNQNOg0el0Oq2XL1+OygUXFxc1nU7r5uam7u7u6uLiom5vb+vFixdVVfXVV181XTvo0LiWX9eGDLmXl20YyPTs7Gxk48iXoGO5Y1/efO0MxYHVQHtxcVFVD2/xJsA5C7FN+IZofATHtt+cn5/Xdnu/J+38/Lx+/vnnka8ul8vG/vAZmBM2BCDO5/NRWcbsj89dj0s2if7xkUyPe/bplHMXaLl9dhWSPSI83DBzcddDyMO93EokYsIAXNYJXB8iYtC3wx/sPgAAIABJREFUi34ImKjhp2SgSK7ZA0AXaxEk486UizFgPPyfTMxprBXn+lLVfUrOErSBg3nxO+sgXMPXxii8H80sxikp83a0Zi6kW2a9Xvm1o2RKaBnwvfdK8XeuVrvG6LqIAebg4KABGq/su7m5qb29vXYPHmzgF7/4RavJkDo6PbYNcZ1kjDjjYrFoL4E9OztrNusaG0GPVU/XAHtObb35uuwfI/Dn00WcVtoG6Nulgqqq169f11dffVXT6bQBkPWHHQ/D0Jj26elps12yAvu2bSjLRrbdDHI0ZJ7g5Tk5czG79lyfajsBzHm6FYGwUT5AYhrKAJ2ukL75LSTOlxGw9x25CMj17FCepA2U8QKkBjLAyNTVTIfWAyunFBauWYdZJDm8nx4L+JjBuf/PpV92xty3hQFknz7OYOtalp0ha25PNcbhVNclgl1zs2xtpKRartdVPTCWDCY8wwtATqDP4rkBnObVXRYRtttte1orYOBg6vqoAcoO7Xnxues8h4eHdXJyUpeXl49KMsjVaZmb7Ri9OzU0yDF+MgBeblv1cHuPSwUGSViqSw3ol/SScXhuzMUBy7Zi/zGzNBb0mLXbZ98LCTA5MsCsXOvxs7syb4d1QbfNNnyzKT8AGgpyWgEIGJQcCbN+RXMqYHAxy7DwGJ/pNkpwSyCzsWAwPESOcdgJ3I/BxzKyLPNvgxO3B2XfTwERQYDrOkgAYMkEc8y0jNbu/6lImgyO47wvzKUC30Y0mUzaC18vLy9rtbp/EzXX29/fr9evX7cyRQYa/nZt0gGNtI5jPE5W5ViUqnr8VFj6o09AwnUibBi9LRaLdouVA7xtvxc0XRohJeRzg7gZKSklm4JhZLe3t3V8fNzsireD84SMtJlk67aLp4J7b+WWz13M90rrU+2zbyVyhPELark4QuJx0CgFQfE9kdMbYK0YszCzFfp0aufI4l3HRnwr0TeL079TX9e5nJr53XgYOWDJ9anlJViy/2c2m7UaYtX4wYhmIwa97Aulul5ApMvietWYxRLFbDwOQGl0rhFm3c9BizE7VeZY5EFfrj851QQgsg7naGzWY+Dn2icnJ+3FHAAaJYs///nPNZlMmuOdnZ2NQBI7SdafjC1ZNY69Xq8bqHKnAOBwfX09qgnlPYowNZ5qQTD3M8o2m01jSVlWsM0aABhn1mAJbNj0zz//XJPJ/b5D7jfcbO5fdPLhw4c6Pz9vvvDmzZsR68xm+8RffIcHY/bnztSsc2clPRab7bPbKH75y1+2wbFTHeNksEQ95+wYCUDBwDDWrBu5FkIzMHkflNPDXNkgssF4XLdBkQcHB60gm07qAjfzw5gYk8dMgdiG7mf2M1c7f6bXNObRAwm+x9DNCjIt8x6mBGbAA8PJGhhj7rEmxuVxZ23Jhul9fJmiGBSsc4OHdZNpdxbGCXY8JZj0j5dWzOfzev/+fS2Xy/r222/ba9XMmHoprceG3M1UYVbe/DoMQz1//rzJAaaDvAGYk5OT2t/fr6urq3KDNVY9vCnbLAsde6UwmVkGxLTB1er+Ju+PHz+O7gAhKJyfn9fl5WUjALngg/xZVPPnpPJOJTnXsvYeTa/ouwa2C7yq/oYnshq0GLDvqWIiCBwm5giRq5Wca0PBIZi4GZb3xyCIdAbadvuwwY4xAAoIhzECxFXjfUuOAgkOBgDXZ5DPxcVF7e/vjyKVjd4tr80409ByHNB9A5sBg3E6lc/CucfgucI2DaLImAjaq73YWayn1I9XQb2wwNh6tRD68/9mF4yd6yN7xsuGTa797t27qqpWf4KZ0SxX686LFAZegpa3spydnbX/WQyDDXsLhQEbOVo/7JZn/6LlRGbiDMAB0AHdtuC5X15e1mQyqefPn7fzDg4OGjDzFFoHT2dWtjOvpFpH/HZ5x/qzfg20ucu/13YC2MePH+vXv/5129zI6o53lbuOhTH70bq+fw2hVj0sCcMAAA2noAAmDkoxlVTUk2M8Zh5WqPNuDNCrp+7PtZCqGjEWPybHwufxN0TiXG17quW4mAcGj8NkmurG58nczL5gkl4QMcvJAIMsk21ZJh4f4O/UzPMxS+b8bNSUcsOq9Wxwo2/GznUtdz+zCwC5vLxsY/n48WO9e/eujo6O6vT0tM7Pz0cZBeyuZ0OWg8F7tVrV5eVlC5S+R3EY7l+s6zKK2Sg+BCAMw1Cnp6eNxeUKneXIeM1QnbJantgTK+KwUZc43BcN5k86atDNe5g9Lgc1jnEQ4H/Ih/1vVx1sJ4Cdn5/Xcrmsm5ub9vRUF48RNsYFHbdiXNDMepELmQjHf7OaUlUj5LfSkwlZYVZC1fgFHDZSjkNYKTDA1g7uOfZqOVzfaQnXMeNIUPU1uU5vVbIHNslYMpp7ngZv5u3ree9Rz4FzVdNzzHTF9TuO8/luZq4GBeZkhtFL67N+BUvxSqEBjfsduQZ2x+1u1kmuiGXg4frMgYB/eXnZ9kSakZDGuaEzQNFZAUSBVC9tcpejex6wG4LuMAxtqwrAyr26uQveYLter1s2lj5sJu2x9eyN4/BPszWPu9d2ApiXTjOiGjxQHpM324IRoQADEDeWshfm9PS0sSsmeXBw0GoEZmCmrozNqRjnIxDqF67Z9Bwgd1XzWGvGDVObTCbNABK0nQI7LfJYMSSu7xSW1gM4H+valltGTbME953HMm5qZU4t/UwojA0H9LYYF9rt7AZZxs21ub7vW6QBNLY7O4XZcNZ9sA+YBeNxwd4lhsViUT/88ENVPTywkTRqb2+vXr58OUpVASGDuwOEa5X0ZwLARtlcQAAcLOfJZNI2cBPUWd3uyTL9FZmTQTE3jj89Pa2bm5t207cDBqDpYOIaMfNDtmlXCazWTQY3ZJm18qfaTgCjBoBge3uyuECvHoSwTHMBIADNO+X95EUmyWde2YLyOu/HaBCmazDU0hgzqY9ZUkYz+nBUsIP4WWRuWb+xPJ6KJsko8nNa1rMYp+fi6/mY/Nws1udlfYP5eLUzZUxD1pnuJigbuHusM2WQRsxnCWgGBx+T9pEO47SI883+ARoveLgm5/E5fTKwmUUhKzNL+1naKmNHH1ljMgO276Xd2Y69VaXq4UWy+CZBKc/ls2RQDuI0B4dkYAlcybaRRfaZbSeAed+W2YMNhlSF2lfVAxMjyqB057k+FgGC6BgPxkXx0imoDdjCsmKz/uKVN88h01rn4ln3GIah3SBrg8400ds7HAW5JqzOlDkN0EHD6RNj6tV8kElvuwIyM/vimjZ4O5xBJpleL13POgefZYrnvw2Wnof7thG7RuqAmKlr2qod26tiDkS5YsotTJvNpt6/f992/282m/rVr35VVdVeNWZ/sD2YnfHbqSOMazK5f9Agj6c2owVcvKiFH3HPsu8RZiGJ8znv97//fZ2dndWbN29GQYzH6vgJE1X3LzRxHZF3WF5fX7cVVtsGAOzSUbKx9AeaiRG29bn22X1gRncP1rUKT9ibBmFQftV91fjhcAiXqOctC0Q9hOItDdQBXMsxqJlJeNUOI8u6FoZusE3mtdls2qY+A0xGCSvHjpkgZjabdNtMMGXcc+pevcCg5Ov7N3J2H4AzASdB3P/nQoWZUrICy9vXT5bluWfK6AjNb4NOr0TQk4evBQha3tgWKRsZwsuXL5veb25u2tu8/Y5EMxeDN+PrpZz8tr4B2kzL3X9mNMx7Pp/XarWqk5OTBtCsZPJYIet0Nps1P7WcsVM/624YHl4Q8v3337d7U92X9W39ucRjxmWfs1/tSh+r/gYAA0RQ5Hr9cGe6nR+jhzERLUj3sugLa4MlICiE41Ujb/QjuqBcogBC8yooxVhv8LPwcFwrxkvlNCLU3d1di0Y29HSIFLojkp3ThmjlZ8qTcuuBQB5DH960amfBOHxrDgbn+p3ZAeP3NgjPA3309rN57MlW+W0mZRlShsgUuDd398nf1qnl4udk+Q4Rro/tsxGZW4xIt9hH5frW6elpvXjxor0ABfm7vpeg5lRrs3l48iysixSWG9AZf2YGjB+d3t7e1ocPH2oymdTLly/bFp9nz561BQHGgI8DYs5AzBi96jwMD4+gyjTcgd+pr8sdxoVe6aXnS9k+WwPDcSnumTZmHgzI8T2KQsEAH+cBHGZYXDONyQZgGu60wEbhIr+/cxTwWDjW6SbXZjzeO2bnzpTNYG0DfSqXz9rbUy1TeOspV22YpwHCaSHG6e0jmT64/pGUP9P0bAbAHJ/7MSNm/rmQ4sIw55htZ582en/n4OWHaGaKY11ttw8buJPdAfaZhs7n83Y7jj+3nzgQmKVYRgRh5AeQu2zBHLyCz3n7+/tt68Yw3G+upS82cnuPnwOZ2bHtncfnQBJOTk5GK6Zpm87grLve35aVwf7veh6YU4qkwtBXFy1hQb7Z26mSVxdwnmEY2goLINAThGsTZneTyeRRBLAh2lB87Gz28ILXVCCK/fDhQ3ttF9/ZgFE0CjUt9tzzvOxrV3Mq0asx0QykT606WnecT+DxfqbccNprvZqYa3ZmtlkH87hzO0I6syN3sjDr2kHKdShHcss7UzGaZezr0tdms2llBHSMU2+39yt0f/rTn5o9XV1d1fn5eZ2entbJyUm3TgaT9XiRG2PihSc4tbMhXlqLPOfzectahmEYPaHDCxFOQZEFAAfIshHWfg4j5m4CgC33cfqcZOTeoG798/9Tdue284j5fF5v3rxpdSTfdc5vhODc1dsvMufP6ObfRmR/nswDo00khx1lfSFzbM5lkYJjHP1QLGkDLWtdNBu9j4FN9Obwt7ZcZXRzrcWFzwwABlzLgEbh3vr0d09dl0bfmY5mem1HyfpfXs91TI+blsDlebuGk6l3Ljx4DDlejsMpDX7IjIDhm/6rqm1xOD09rdls1p7D7/1crvNxrhmo7ZjvqmoESIyLp0ZUPawSMyavKDP+nizxE66bT/ewfyDjHKMJQK9k0/P3TDcJDH93CslNnt664DqHPwOVMzLl/XW+SXo6nTb0ZlXFBm6Ki6LI8bPew0qLx8hvLw5k/75ZfDqd1uXlZYuyZlPIxOllAoVXY/IGXhSVY8DJHJVoyMy3org5lWXe/tyAkUCRqVNGS2+bcb3wqQZz4zj+Rz/o0g7K3JgDdpaAYuDIsWeKaJswG8OZM5Wm0V/eLlM1XpEnZXQKulqt2mokdTMam0avrq5ajYtd9W/fvm1FcIAAeWetyH7FHNjbOAwPj8rhlp7pdNpeRedVa2TAKissi4wLP8hSUcrKpMD2mFlBMn5k6LkZ1LAHfOKp0kvVZwCMZwb16hy5+sdFXcB1+ja66OzhXkIUTp+eQEZegCSXoN1vsjAf4/TEKD8MQzNansvUYwqM0dH8qeaicI9tmZ36s5SX0/SsISVj7bHDZBw+Dh14pYvruH/+f2ol0tfrXYeWaRjzzbTWYJXzs9MQPA0kHreZeY8lZEaQLMtsxZu0rQfXWmGfrtkwBq/ETyb3T4HAD66vr0d7zjLVcu3XY0uQ9TwYG31n6uwHgTpAebHJAc39e3zekpR69vz9nTHCtoBseoHoqbYTwNhoaidgMAwcoSJI351udAbRcWyvDNqwvHsZNCeqIOAe7Xd0xdgAQ54EYAXDZlDi+/fva7O5X2lK0OzRbo6h8O36UYIbwMr1rLSeszhlzN805mlnsVxcc0pH9QZis1RH9QwqfmyzndVAZqbIXOjn5uZmtLzeG6fH4jE5WPYagcmARXN2kOla2h4y8qqn007maLBGV2Z61rHtiPsOOQ825lfArdfrtlVnf3+/vvrqq9ELMwygMBSDucdj3dA/1/VDSS3vDBicn8HE6aPHxnkcbz3aJ61768Zjxtd3gdhnbyXKWws8oYwABiachsG72ZkzemCQbmY8RnRfz7Ulpxw4k2809dhcjyD6ub+cn8e42WxGNNvn+Tgc38ZPfwl2VnquLNIHn/fYGt9l3cnGmKs6dmbvg7NezGZ3Mc+sR5lBJCNlDk5Pc5Eij88g4vEzh16dNaO8dWKgn81mo5JH3nHgmk4CiZlqpr1mjqRusGuApapa+uniPc2+5OzFdTrbRA98XHLw+K0Dl4UykFvGk8mk3TOKTJ9iW+mfDphuXN+3p+1qn30iKwMlEgMamaey/A1bwvEdHWBhvHzWqZ4nw6Tdp3NrM77tdlvz+byNkeVxb+5zbYjGU2J//PHHOj8/b/l/KtxsqOc82Qw8WY/iexSVss7UiutkZDPoO3pmkLG8cBBWsygge878nfvccr7IgXEBBDikAZP6Rq4ocb2cqxt69v2ZuXJlsHVtje8Zi69jR3O0dzZh5lb1+KZ2sw4HLjuca4gOIGY6fvsRY+DJE+v1uv7617/Wer1ujwR6+/ZtTSaTevXq1ajPtKdM77PkYxsxQegFVOa0WCzqw4cPtVrdv4/g9PS0ZSyZrWQ67+zH+ncq7PH6Mde72k4Aw1CWy2UdHR2NDNa3p6AsHCgjuJXrgiKg4aX36XTa9t0wYcCO55Pl7TV+6YJTzaoaASXC2t/fr++//76GYRjtIkYBGBO1AtNzBOxIZoO0M9IXQMEKrZWSTDH7d72gx+LQh89zUMD4clWK4NB7a5TB1rUjg6NTIWTngIPtZAT1PNwvLQE4A4HTVf+2ffVKDHZggxNytj3Z2ZyOOcgafJOxoy9SaAOdGZz/94ZOv+ZtsVjUfD5vcv348WN7zPPJyUlbJUSHZqDYHvJm3A7I3uJkooKd3dzc1Lt37xprZDP3wcFB217k+aVMssSQ+k89JpBlRpPts/vAvPoB8DBxorgHb7bjQSUroA7jnfMsA7smgsD39vbam1YAEzNC1zdSiI4us9ms/vjHP9arV68e1T/cXOOhoVxTeQzUVN4OQz9emMgo12Nl/t6pldMdjJ29NxwDkCSDwKFcy7KD9WTIWKhfpcOiQ4yN6zwFtp5bD4gBGLOgZJ2WAbrIiJ+69MLRMAyj+3Wd3iFjBwP+T2ekv8wY0HcCFMf5mk5nU7akUqzOw8xev35dd3d37b2Z+/v79fLly3r9+vVIZ4AbPsY4GL+vhS5YIb28vKy7u7t69+5dTaf3z+0/Oztrcnjx4kU9f/68rq+vH9X6HJwMkDTruJdFID8D3FNtJ4AZvalXoUweg4MjGLjSmZ2OOJdmoM7JOccFSkf7jFh8l0VGj9XHXV1d1ddffz1Kbbzp0w7g/pICJwh5I22yMfr13wazpP9ZgO/VhQAswMVsp7cI4A2Z9MdtVpne2hFzzMzdOvXnPQBJWfX+99ws654+HOB8vu2G5npLMqCsXXls6MVycF8ca/Cz3hKcbfNmSiYDfIdj+zYw9Owf31qErUMyeL7XZDJpG2CPj49HvmkZXFxc1Gw2a7cbTafTtjkW/7esuFXJIOhm37XO/J1LDi70u7/ckJ3ts0+jYEWRfSI0pzcoARrrDX2kfpvNpqWGfmmEd8I7TfMLQ3F0JgtL860vGJwNtOregU9OTuoPf/hDe+EnfaPskUBm4yeK0lcKEfDuveTADpBRm/ZUbSnvmdzVPEf67KV/Zhd2pASIZBfu0wzEdSYbdY7ZtbTpdNrSVacpHk+vdueGXhL06StXeN1g+GZMABrH2n48/l4mYdaJIzrltM1guw5EsOZhGEaB2fVOB3wY3dHR0aNtPlX3m87/8pe/tPoRe8940cj5+XlNJvcv1b2+vq7/+Z//abXo/f39Bm5kVQBX790SjI+FiAwCZpkGK2SRaTrnuC/rNB/66LYTwE5PT+v6+rqBEMDkpdjtdttWImA1VpQLsNwIDXLbSOy43veCIOjb0Y9+HQVt/NPptI6Pj+vPf/5zPX/+vD0V08Knue6UqY9rBHZ+oqTPN6tyrclOmkqj5SqSr++tCVlv4Vz+7hU+XXswK07ATKBOgLUMHP1Tn06fqx6ncT2Q8sqbdY3MnmJKBmAzU7Nqbjr2Nc2okukZnJ9i1O4n02fLnDkzJ8DIpQazPfdrMHb5hv6QJfdekgkBXrDs+XxeP/zwQ9si8+rVq0ZIWI1nBdYyyWDDXFwSMWjbLnpBuFcD9BzN6Ho11GyffSb+d9991x7yD5rzxInt9uHJjKw4eXUShuRnZxPRQP4cPEyOOwBsMH4qpAvSdiqnuOv1un766ae2RcJ7bUz5ASIDi0EnW47XOT9j9aJG7xjO99g5z6m7QYnvd6Wm2SyXqvHSuo/pscycsx8XzPhzDrRkxhksGIMXV3wdZGXQdIDpAYXTO8bXA3h/7xoufWYaaGaVTIIxJtiZuTpdpF+CpOtAyRCr6lGd2XPl9iXfXsR4sHcyHL73ajSMK+eYhXxIi1Ndy8BAxLVYsOIzM07Xv6xP7NCZyOdAbCeAPX/+vD1o0DdboyTYB6le3hLCLTpQXgZpup1F0Kr7TY+uvfnFHq6JYTQuglZVHR0d1bt371otACNwOuMovV4/3n7h4mMWIi2DZCEc68jqiGbjxpjdzG7o0+e62Jn1sTRCX8cvT8GoPN9mELPxzdi9hr4N+BhnLn48VSOxXDz+Xj3MaXCmIBi7ncJycx9mdWZZfObgi/15FZJ+bWtZmLc8zVqyTmkwol9vbSFgu1SDndqhvZEaUIMh5YMRqW1VVUsf6bd3TbNlg61Bq8dgTS7Mqpm3bd26MaPspeK99jdvoyAfzqKic2MjMMBkpZhCs0PehsR5AIijEEDmdNHKWa1W9eLFi/rpp5/qr3/9a3399dftODs647cz+6mzR0dHDTBN8e1IZlPJXvL5WnZGf5YMz6mwjSeVnsVlO5JrWJlSWkdOd/m+qh7tC+N4SgW+8b0HkvTTAz8DJ//3AoOPd+rMmL1S6qDhmpTZEfLynM0kfPuZ2YCZGzqm5WZfZJIOnYDn1Ar/cH3MzMM2YL1CFFxT84tJrD+eKGF50hf3TXqOXqE2+CTjtg34lW3Wj4HZtUvkZJt2JsV4LNddbecuMRyXwSeLAOAYUKKsFZkU3EaEAHpMBaUYkf2Z09XLy8taLpf16tWrUaTlepnqZAQm3XXkpI8810DQS4/4u5c2Mh7PNaOMU6hkWj3QQIYunDvtyjHYAT2OHE8v/XHL1Ugz4t547Qwpm6oasYHePNImq8YFdcZqFpZz8HiyxmOHSabhaztzsL37/3R4y5/xGLQtY8/d35vx5UqzQdqssKc7Ew/82LLygobnYDkaoDmPUpHHbBtN+aM/A55LPbvYV9VnAMwO6FzZq4NmRgwMSktkcwpAv5vNpr0k1UCXTIw6GUqlcO807fz8vC4uLmq9vn/NkyOZBe15OTVIw8tm5Zq5GPyear0UDUfMdIl5Ox2CbRkUPH7OxeCdbrtv1xkYu69pZpzfp6O4qE7dhfNTXi72ehx23Nyy0QM2Pk/2lfLLFNLB02NwbTLZQaY9zMlAxhw5lzoR9tFbqEnGCyAaGC23HLP/TmD2fs3ZbNbqXdS58FvuseQHnRuIPBeDbQKZg4Zlb9nl+T6P1gv0+Ege+8gmnvymHnJU6kcIjC0EoDhpk9M+p115v6CXohG4B42QWE3x9b0xk9fD/+53vxvdAO5aQNXDZlPoLnQZY7TwrKRe3u50xEaVTshY8vus//k8s7lUusFrOn14W5KBnj78OZ+t1+u2sZF+POdeas24kVmuEtpgnV7bqb1x0oyafpwm8pnnn4bt2if9GtQyGBogzCoAXAcMxsw8sBfryLafgAiQ+AEIZqZ85mCTgQYboV+PN4Osx8z1ez+9NN2BLFlj1cMjqy0f+7aDSaaVPfaUNTWfn8Et+9iVRn72kYekZ45qfjor2yKM2Ln5zAVMGwvGCguz01nQNgQEe3h4WJeXl/Xu3bv2tEueOoHRmYa75uQFiarHK0aZ8jBnR4NMueyE2ZyiMTevNjqiG+hms4fHCZu5caxB1tsiDN5mqnYMN3/v6M64PWeul2kt/dqxvZDRA+M0TKdFPcaYKQ2fp6M4yidbcTNT5H9sGMAfhoe3w9sOM31kfLaT1BH90pfLKsmI/ahv1+kMSj2ZMJdkl5ZVjt8poetx9uWebxhwptNpw4FM5bGdBEMzWNdTMwjuap+tgQEaNk4EOJvNaj6fj+4c9ws2sjhXVW1VcrO5f5QIaWHVw+qW6yCus3Hcy5cvq6rq06dP9c0339Th4WG7zQgl9sDJOX9vKwLnm2VhyK45JStItoPsdgkfwzajsOFkxEtHTeUyBvrKFK03ltRLApKbV8Kc1u5qKZOqMfC7DpOpOb+TiWZqYjkYTJBnpq/JYKrGixdmabnqbXtErk6nPL9MqwAHvrPNOYD3ZOcUMRmPA6/lRfDLDbeZFibQeJw0+ktG7bKQMzSzRbN72wBytc/62Ax8T7WdDIxB8l5Gp4MMlPu0cBqAq+rhTnsbCIZhwWOUpt5MlJ3bd3d37d6r3/72t3VyclIvXrwY1cRgLUmLjez0m58hMDslYGuQcT3F/dC8NL2rGZzS+HuUP5ujoaOunyaaNN6slPvjzEzZg+fnRBGQ3Bfz84qZmUEvDcb4LS/r2d/zWW7LMRD5d7KoHtCb4ThAZCnAKWQGEWwT2TrIcYwDJ2Ox7Nw/tuLN3pnO2SZwegDAdwrkxmyIAQ86SBactpR/G5jzPli+s/48xiwFAaaM1QzSMvQCBRvOd6WPVZ9hYFdXV21TadXD7Q8eFH8zEde+GAwT9K0KVdUKiEZybvKG+cDiXr9+Xd9//3396U9/qjdv3jx6Vv16vW77XrwUjCFktKIxXsbIC0FRvGtB/E7Q6aUntLzVyPWUHgu0QxmAzVDMfly3yXpS1fiJA4yR4/jtz9m5bT1aV66DGSjdchsFizrogmslkFkO7iNrkXYe+jKL8NzNvg2ynMc1qf/ZSZOt9dJa13WG4aH47dIH57kMALi4f47v1ZjIeAwCnosXYNCfHxFlG3AAZg53d3et1uysyAyQ/9NnMgA4jeTzxWLR5opPWKYZzLiW59BrOxnYixcv6ubmZmS7DVtOAAAgAElEQVQYd3d3dXx8PHphLdGI2ph3DrMbl0H7diSE5pUskBfn2mw2dXZ2Vn/84x9rsVjUd9991wAuqXFuosTIXMNgvK5ncFwP6GzkGELSYgsdxaAA79vhWMaRhfik2Vk7cXT0LVjJfGgG8mS2zQDEwMwMOJenXWTdArZnXXMdmB1jdyTNmk2Oxey1x4CQgRmQA4p1iYPmewZS3wYW5megyj7NSuyAGVSwF993a8DBLtktb12ZDHgsBgrLCYZlQuB0M1NM5Mc8epkLfee1sbteZgMQ2waSUDhops0yX/Ai6+nZdjKwd+/e1enp6cjx/QTT7XbbdupXjXfmm+Wcnp5WVbWbMieTSXsIodNMFx4RxtnZWf3ud7+r1WpVX3/9dXuwG07vPLpXAHbURsg9dsj3sEyDiqO3QSBXELM2xjF2QsbZc2D/z7EeK/OC0TzlmLlS5+v6f59DvQT5s8KbEZs2DEPTPbL2yhW6ZCXPzmQGadZBuujmAOU5VNUoPXIzGOSmT+br/pFXriwms+DHq5f0lwza9uIgZVmyEn5wcDDaguKyCsf7fDNm6zgZnRvs2bJmHp6fFysc+Gx7XhDq1cBMSiwT6ylZs/3TY/+7UsiTk5O2slD1YDDURnKVyQjN4Pb29urq6mqUbqzX61Fti0ECZpvNpo6Pj+vm5qb+8Ic/1OvXr9ubjn1tRwDXGzIH9zPyJ5OHp2EMwzB6fDAKXa/XjTnaAOyEGIENN483M0iDBBQts6ypOEo5qmJkudmxV1dD3mYY6cBEzCzE8v1ThuQ0287O+NyH64u+jovjyDFZr23KgOP3ilrGvbdW9yK5GTwA3gt8yTCQq+3OYOXVxQySDoY5Tv72Oxn5zfEOwE7Tttv7nff5whsHYffpWt5TbNNzx/5dNmLOZqL87S1L1MjzxclZ10Re9r9d7KvqMwC2Wq3aTaEusGb6BvpT7DdLsYCrHtCYnBjnqbo3vNvb2zo6Oqrlclnv3r1rT0z1Xi4bWtJ2HACGgvECwj7Oz4a34QJ6diYAM9MyQM+1J4wGOZm6o6w0Lq8YGcTox7UnO5eZYoIojLnqIT3z3Q8GkDQm+vZbqbNG4zkxVrMry8qs3GmU5+V+kIebt+oMw8OjXmjYEU7mWoud3nbCGD0PsyanYAQ5O6zfyoOuzXjRnf0FX0Im2B71nqOjo5HdcH2XWsxo0Pnh4eFIvwS5XokC3fcWZ+wLyCIXpzKgwSYdTGCrtkeXQYwjZmLJAnexsM++FxKgIWe30yB8P1IGuu6aEc2rUAlqPLb68PCw/uu//qtOT0/b/YywvMlk0p4ygXMCHnY++jQzJCdHaBzDb6O9Adi1sGQEXCNTMhsXBkBztHNNB2NNMDLDrRo/oNC1tTy+x2IYF8aVxXY3zvPLjB2RPUenzvnOT2QEuBFMckOvr5nytz4ZS9Xjl5NgU9ifbc+ypi8DGce7dmfZpT1jd1kcN9vKtDcZsnXN92ZzTltd83OGwXm2VevLdUXL2Trjf5iT/Zdzfb8w5MDX8fjN9O3rKSMA1umj+2FO/+saGFEio7QN07mwU0oGmIqpqkfspqrq7Oys7u7u6j/+4z/qm2++qefPn7drWaH0gRPgGEbyBMiqao//MdOgbwsuU0M/PC4djWtaIe47ATPPRcb+n7l5Dr1jzPDcchXSaZyfIFH19Bu3rRfkwndmkz4ugdXnJGB7zE6nOd8FaZ5TZXsxc7OenbbbkfnOc3Q6V/WgL4/dAGdWynwzDbMj91Jb25cB2UwqGQfysw/Y96wTA13K32Bo5m8ikmzROnImZR832JiJ2rbQpV9gYv0hawNpyvCp9tkUEhBj4nYAK7hX3M48FtCB6pvR/fa3v631el2//vWvm1IzfXI+jWHzuQVBlCAacIwpbqZ9jMeOSFR29Koapzt2jKxxJEg1oSsV8ZYSlJiMEhmmnLPo7JS2Z0TUHX3NqmqPWcnaleeRTM5zye+8749aiHeok1rkudYl13QNLx3fQFP1kLL1ADg/d1qJXfGb/gheyVacQpo106/BA3k4uAGSjMF3umTQyjTXwdIsLbcj2fm9xcI+6w3O9jP7ja/rcWBvLlEY3M26yLC8SOf0NQlFL0A+1T57BPuCmIQHkaiOoDK1QLDenIZRHx8f11/+8pf23C4bGoJ2LYXHU7OLP+mnz7MwaYCqb5GgsfJWNX5aBlSXYw0MXuECJHsposfGd6437GJkVY9vvvZxCVYc41oVDKzqYR8OKRMFfAAcY/cqZ6bRPSbpOqNTSD9jirlQerB8Mgj5d9XjRxCl/visV1Jww3Fcg+rdJ8qKN2N3Xcb9ulaU13QtyjUkznGAotbkQEtQMrt7qiSBDA046B27d4DcbDYjm3CgzOCYTIvPmA8LEmaZLjcwpiy32PYNfA7C/+sU8ubmpm0mRZDcuM1g6TydwANMtjMMQ3v8ze9///v66quv6sWLF23wvX1EWWzkc0ciFOgo5adZMBYM0QK1MunH/fn/LC47ZTYN30WBHXUMtsyPvtxf1fhZVMmIOderk5mG0wfX9QZTf0/QsSP5Gh6vX0hBfx47hs7YnIrAHjIN8XWe0pdBw2PyyvJTMmelizmSDZgFJfAYPAA8O3KyLwf+6XQ62tTcC14c488ydYXp8MTiDCyepzMnMzPmj84zW+G6PQbn62VAsizM5qxD1xC9AJLptetpBuxsOwHs5cuXDbxgPr077e343s+CI2C0UOWzs7P66aef6vb2tr7++utmSDc3N23TJGmmC9i+lqMMzC7ZyK6UwsLFQDLC2+lpdtJsrgG6kOmWrMHA68hlA/O1k43k+TCLHlOCXZlpuK7ptMHskxqGQZnfzDdTdOvOAE9/Tl2QtW+pyfl6DozTQY2+Waq3jA16/ozz1+uH+3O94OAMo6pGgWE+n492xafz2fFgNFwvF7rMbgxW2LhBDJkeHByMrpF26zQc2XJtBw+XFjx+fI1xZzAzeDkj8Fgzq8BuGTO4YHtyEPhb0sjPMjA69kZTdmfbULwL38LDMDj++Pi4fvvb39bt7W29fPmyGQvCcz3GzAfHtID439GBsfh/11WSodkJqK9xrgHThWP3nZ/TnhK+Uwhfy3vbaO6Xv0nrcCY/RsisgnPsgMzJRmKG63kZfDjed164pVytNzulAZ7+mTs2xhgcXBgnc8p02tfpLUwwB9uO2SuPV55Op+2pw2amOK/TLLIRA7CZQ6Z6uTjg8XCOa30Oqswr39Dl420rTivpg+t5QyupM4GN4zjH7N1j5Bqea6b/XjByqQemawKEjpxCem672k4A460mZiLr9cNTSx11vdEwFc6tEtvttv77v/+73rx5017Oaafi78zrmZCNPtPSqnoEfBbAZrMZUfhkUgk2Xl3NNIjz/dvyqeo/Q4zjHOX4nbv6uTbnw3Zy64DlNplMGlN2HcNplcfoaxisGI8jq1NTy9vR3fUz5mFmh7ycThG0+Nty8RhTFlzfcu4FDDMH99EDqFwBTV0ydrNkB4lk1iwCkFF4Xukr6Mgy53hs3mQgFwI4hv89zrQ1+xIBb7VajepYZoVOG82UmIf1i805zXRAOjg4GLFNgznBbTKZtLsT/i4Au7i4aDdgc/uQc1sE4Ojh+gkAcnJyUh8/fqybm5v65ptvRkzBu6k9IYTgyTkvNv23sdrQqvqrhAjV9Syzxen0YROlnbNqXLOycyN4A9RI0JPxdgFHyJyHHYVrZkRPZ8UoCCQGWMsrjZF+Pe4MJHkXgseUdRA7km2F6zq9xz7M5p3iZh2F5tqa5cH/1in72MwQGI9ZrMsWfG5g5beDpFMx5mu26wUjPyE15WVGjEx6DA2deh+d7cLszemqQcjzH4ahbX5lVd9gnWPz52aTLtukzrFH10bpx+eayWF3n7sPsuozAIbyYWFOC1ECv22kDPTg4KCOjo7qz3/+cx0cHIzeHWljfUqRjpJG8YzC2fzIazuV6Srn9upZGf1Ji9NAaU4HewyMMfZSQqcLBsCqxw+C5Lwe+NowSBMMLpa703bPBYPKdCudghJCFv298x1ZMHaAyoVt64RxOIgl2FnmfOZ587dTUZhPpijIEjZoAId95a1a2bfZTaY7mcJxHlkHYHZ0dNR0TbDJYIU8bWcZYG0TDkgGVuuWseWN1ZyT90karK2jtGdapsu2ZdtwytIA6jT/qbYTwFC4GdBkMhndq4Vhc1GM/fj4uC4uLurf//3f69tvv20TMGtJ50XAVrxzZwuGY/hNPxiKUzo7cm9+NBtsT9BZYHY/WYuwM+UeHVoaP06LzDNqZb9WctYBTfEpUpsBeauBUwsf43naiRwImHPPuB1lLRsHg8lkvKePzzwG+jfIW8cG+QT8vJ4ZNoHODMFzsfNk6m5ZMC5k7e0nyK2nn9vb2xZozX6SKdkefL7tEV2RxnuFlYyHY5J4pK75m2saVFIu7jNBt3ctf+6HO/A7x/B3pZA83M7CclQ2K0KBt7e39ezZs5rP5/X+/fv67rvvaj6fj6KgjYTJ0TCCjLhp4E6xUtiMs5eKZZ9ch/4diYnCdsAeezCA0LJAnDJDDgYg16Ac8cy03L/rB5ma+jaXYXh4rBEG6Foh1/DtIq7bGdxwdhe0uTa2Yp0afNySOTmgJVhgN+jBiwHYBKyTYrQBIXWCQ/dYo8/5f+2dS28kx7FGk90zQ3IsSzIM////5YW98MKAF5auLGhIDh93oXuap05HNQV7cw0wAYLd9chHZMQXX0RmVVsGMKUWOziPvY7RfW1q4HA4nN6558UUX8u8NYqgnQl8DLDoi7d/lHnTd+qFjVoHDWZ2SMaBpljsnGmD8ZgR166cCtkrF5+F9K/62oPRIGh/dfXrw9kfP35c33333frLX/6yvvvuu/WnP/3pFDbYe7hzsCwbbQGp8bbrcCg2TWgZk72HGZcNdyoObwxGvq9GagAwy3Q/3Uf33Z8LNrSF/MweWpdzJVY6lNO0viBoam+Zk98xq3ZfHOZ44cM5nSq5c2cNXajf/SqI2bHYWDwXZgEOiV1qZFzrdIZlbWMzU3e9lSP2UB0qW2lOiHMUO83Ok9mwj0+LF2Vivp6+2VaYo69fv24eNXQbnlfLwfpWG6V+Lw7Vplre3InvjYdsUjUtRrhXV1frm2++WT/88MO6vb09S7RPVLX5FXukKtCE8A5PEFZj87W2+aYKywLq9z3hGZiYCIeJbsfA9vz8fGJGsArv8DfT8XinsNVMy4ZTQOc4dcFwzDKRHZTeitMnFuzRJwa8t8Gz8+M5dh02Vo+7gFewtuz9SmzrSxmDWYCLN/p64crg2wfFKXY2ZjNmzHz3j+OYRRroHfVMYWV1vrpPP8oS/bkgx7E6E39H580svf+w7SNT9NvbQJCHo7IJN6ZyEcD4lWoeRYBtGbm/fv26bm5u1t3d3frzn/98mnQbqHdqW1ksOCYTY0G5mv/xCpA91hQ6UTBMK8xa6wzhq3yuE6AsG7m/v9/kGKiX+jz2tX4NKdnrxsqUJxDAKgA6z+LrXTfzsQd4djweC8DqJy0skwIJymemM4V1V1dXmxfbtZ/WJa9eeTO0i6+Z8laVuRkP88h/zvWpBcvn4eHh7PnYRgFrbUMonEOB3xuzDfxOMXz9+vXUnlfn22ZX8ih19naCJiK+tiGrnQtt7AFJgdsryT7ulEV/Y8LO187UC3yXykUA+8c//nEyNjqM0a7166R8/Phx/fTTT+vLly/rj3/840kZ7eH80PZeSGOgApwQkB86xqCdd7Ki2qMjIJLEJLM9Ad0IataAoRlkPJGAkGW01mvins/UzzgBQ7fNmC03K4PHhVI1TMILIl8UB4M0EJu5WsYoejc/mgVYXgZrpwIAGPpZ1tUHiWkbp0cfWmB6z8+vPzbjvJhZkkHFLMSbOf3cp0GsOnw4HDavCuIadGxvocjOhrlHNs/Pz6cXhiIjFlua9GecBRgDEv8bGiJvz3mjoJ7DFnyv27NDtR5h440QAGE/tmSmyb2Mw4sHl0DsIoB9++23J2/FD10g3Lu7u/Xtt9+ul5eX9dNPP60//OEPGwEYYR0KOnHuvAvCcnLbjGytreexEdMOYOGkN39d0auXLrtCyfml74afVo561UkGnSB7Jd8HKDiUxGhRpg8fPpweJSkbwzic34IFA2YGW+cqzH4NPpwz0E45L8bl/I3ns16ctpxIJwStvJGN60OPnBP0mMwkvbXEBsf8deEHQOF6swfvG+N+7x2bmIXZjeVD/82mfA19RLbI36kbg0jrMGiWXDQUtR40ldLIaa11Gr9tD71w6Nm8rOef/9UPh8+XwsiLSXwYC5PmV9r+8Y9/XH/961/X4XBY33///SbXYw9YGs+D2hWY98H4uIVoI2/IZQOz4jtmd66E35FkjF6kuLp6fXeWV9aoz8yhk+K+2vPUs9j4r66uTr/+NOWWYFOM7dOnTycW3HbXOn+cg3s9H82JmL67YMw2ZCutQ07mz2003Pfc1gEA3NRj4Kcvrsf64r5SGhIZ4Gz4ZisO/8zKDWqc2wuvLKfJWUACIAc4oQJN5d0UjEFrDxBhlnaIEAm3zw+0VD9sT9xHKUOu7dTGrV+MpRvZGY/7cKlcvOLTp0+nPV+m2Dc3N+uvf/3r+uabb9Y333xzlhNoOOHOkYdqvskMwIbguqzIa73uzanX8TUomNnY09Ovm2w/fvx4CkNsZL5n8hoO3zr2lhqu2YND4OaQDM4TReezx8p1AC9joY6GyJx3ewDKFKo4LHdux/J3/Yxx8qDul8fVUGWt+bU5bs9sp+crP+TkUN7XA1AtTgtQx2RoXa10P+2Auxo49bFEwKzajsE2VibGdU3VNJrgnoK7AcfOpv21QzTTq3ymhZXJxvbmvOUigPGrM09PT+tf//rX+vz583p4eFh///vf17fffnsWVjgO57+Ny5PC/064QwgKe5QcLnLcbdi4PCmOp3kioIqN0Cx0JsH5No73mb96I+euDORmhOQD19qucKEoZSCuzzuwyzoKSg4zPWfuD69DogBENggzSq/OWQ78L7shlPW1VVyHwp0T55DoT4HGdQK2DpktF4c5Tl9M/TE7p57qO063n9eaf4qv/W44ZWdsuVe3XQo67p9ZbwHQTqv315HQR9qz87A8/Syx+2d5tbi9zsVeuQhgDw8P68uXL+vz58/r+++/X3/729/W169fTz+T1klH6R1SQEv9x7U2WgTZX8eBSUwbal0/15QhOObnGkJhC9dgaoV3LO5zfryI7z7vuh0WOk/Hjmn6YkaK/BiLldcLEU5GN29BHYyF/2U5h8PhlLuxwdrjd75ZffVSOteYVTnt4NykQyKvbpfZFixoz3PSOh2CmbGY1Tp0M2PAqVgHygiIIqy7HCvw0Ac/+tN+m/HCnie2RqhXtsNxrrG+23nWGR0Oh9PjggU56xfycbrHoArDwxYdLvrtKXZCDnvt5CxT17VXLubADofDur29XT///PP6xz/+cXp3VxWbCXLsa+/OQLx07pASASKwtbZgRB1G/HoZBHh3d7dhQ1NYAViRB3CeZ0oIO7RzQtJ5m4YxZlLOQ5WRuu/8d1/7Hvvj8XhaDeM89XmBhL7YoPZydlamJlwBF/JzBUnaBSTIpZQJNbShbsvBzsey874sh2722AZEz6HD9mmcFPTNKYvqDTI08GK0AI+BkPEcDq8rmICVbYQxeYXOMpmYEfbmX+uybCfmBDh5bH6/nvvH34cPH07v2ysDo+/0Fz13xGNmZ330HE862cWmvXIRwD58+LB+/PHH9fj4uL777rtNoxYIAqBBTxDK4ESgO9yQwArZ857YxtcTA5rQ2wZTmlvmw7HmnNZaG+PqBPhejLQg0jbMemz03e1uwHJbVgobh412yrN0XD6HJ3x6en2+j+MN9Sh2YjA05t+JXYfPnRvaNbvZA5S9fl8a11TK3Pfqdjjr9AcGXOdtgKpOV9ena/fGa7B2qX51jmsXdtpmTFwL8E56NDknz59ttDpf+7U8PJYea7kYQv7444/reDyengNzGGOjpDHK9GMedLoewANqZ/GcZi1OUCMUwhkL5q3XCpvdNVwx4+J7aX8ZYz0mfWm46j01ZiDOlxAaNxRp/+sRnXeZktc4ACsa9+Apva/LckL2ZrcOzyjMcQG+OSyU3HPmduvxJ51xqsL/fZ3rmLx9wYP5Zqz8IS/3m+LFmDpN11m22ZwfIZ03RXsbRENsMyV0lDHzv/k7y8jhq2XpebFOUI/H0NXr5rysH0Q+U92eL4e9/xGAGSzcKA07P2FFIJ/iAZh52cBtvBUewrKQYAMcJ2S8vb3dxPZVbo4XFGyoZoEFZwNVQ1uXCn9SaO53uOaVTY5ZfpORNy83rRBNsjWVL2ibSVR+/l4PSzGrcIjk41ZU19W+2inVMXjck1Na63XHvT1/rzeLsEyaZ6VfdcAG6b0QDtn6mBmcjz08PKzr6+tNisW6uRdW7bFry88yK0P3anHZ2pSGsTyoz/ZkrGiUZjBuntnXu6298uY2Cq/cuEEaajjRifIgzDZ8PwOavMUkfHbWH4/H9f33359+yZh7DofXpOZaW6ZjoPCD0BVWP08Mi/asLChClXMK+cwI8MaTcthT8rlh6WTcBR2zKDuGyYnUY3fcgG6va/rAhtLQznktK7Hfx9+0AnIrmPg4ZW+VtHnOfneyf62to7Axk4dy/XbYnnsbpgGp8u0LDw3Cnp+WttGogTptU9WZgq51og57AhbLcMrBTazROUTnxn9rDuwigH358mWzyuaVLgvUeaA9L2Qq7fMeFAZmwRbQDofXl9Q9P//6uuq7u7sNcL68bF/J4nzL8Xg8bb5sItXeuJPrUsUoA8GjcW6SWxPsZhxO+ttJwGIdykyMqDKnHxil5eG63Meyk8lwzKb2PCXy8L19ENpzYxmVxXZ7iGWIbhWYzFoculO8Sj0ZqQ2+DNlhuPXYc17Zls04r0TkQO7QDr6pg4az1YcyMsuz/UD3GnVQzwTmHUdD/AkUC3CdMzNz5NA6Wt4MIe39PUiMFGNzktaggOCsaKWKXmK+RPcR1MT8XE+Fa6ZlwdBn2tzztHvKgPG5b80fWGmq5O4rSt/rKd2qYCVGwVBEzwfj8Zwyf76f9ixXOxO3zf0dH8eb4DeITPV5Pp6fnzevYi5TLIjwfwpnbXjug4GQ/9UZ2nt+3v6Wgs+1DTsG7jVTRF/sqK1/3j40PQZl/ZpAxzbFVhuP2zk3b+e4uro6vU7dkRBs2KwY+XY7CNcbdMoaDcRTLpXPnsvJMbtcBLAqppW1MSwTWa/B5LUTRt8pgYhQbIgWQnNG7i9gwGfvFq8xN4yjHnvj5lZ6jfNaFMbiSTDQTJPi8ND3mqU45K6D4Lyva8hvj884zD4tR89tPSr1dQ44ZmfAfQU7jmMwzJsfnrdM+e4+rPX6sHbHMcnBMio7tJx9//Q2VtpvrmnarFuWhGxpk747irERoxcN+8pu7HC6lYG2KNOYHKLTRn8xioKtey4Ae8bUaIxrHCHZNjqvv6W8mQNrKAWi29O40y8vL6e3T1gwXplk4FxnJSrlR3goYA3DA29Yy2ezijKZadWL7wYI57p8rgZZwPZ9LgZGxmggKjjZcXg+PCazsYZL9o7ID8NoCGSw53jzS4+PjxvgMKOcHNb0iI1lwbhgMdN1nbuGlHZclj36Wjbk6xymc34K/dtvZLxnfJWn+0qbzfc4XWA9a/rFc2vG5/HVoVHaZlf2qz914NZP+uDtLiYvXs21Hhvk7Fzdj0vh41q/IQeG4OxBERgNwnBMiz1oMxR7I0+uQcFJdgupBu4cxSTktbb5JNdRAVVQza/0HntL+uP7NkI+nO/XoU3X7wktGywDKlAYvA2ME9uz56RvZioTC2yo7Nft2NDKmmirBuo59V83SdPHSXaWkZ2dZehtK44M9sJqy695nRbAcZo3MyeKV4k5NzEy67b1zNfX2XIvbSN3Mzzf493+raORhP+XxVqGXrmdoiXbVNkZfZiii0sg9mYOrBOJ4ImZ3Ygnn9DQnp3PFij3mErSjgXhYiWwoGzkZlxmD845VAmsRM3j2JszgZaLwyDadVjtPIRZjRW/IYblNlFy8hMA/UTzGVsZm8dig6EvHfe0RQOG4mP0c1oi9zzacTgcKZC7717qd5887wWNGmTDKue/7HzcTnOTrq/Odm+jse3CfZkchGXq/jPWPVbo42WxE+B4PhoON53A/OEUCjTth3WX+j2v1U3rh22z89dycSe+B1eAmVaAGJATlVPYZ4Uzi/KA7Tmpr0Kirg7QYMS1ZUjT4y7+7MmdlqS530pfRoXyT32n365nz4NyzPU2zKjX7FaAyqH5P8vU4+48I1+H7VOYYgcw3W/5+76Hh4czVtMUwVTnNEY7yrJQ2vAWChu9HXL1sMddT/tgOVJwOk9PT2ePAFHQGzOaMq0WRwa9pt+tq5ciGDsS6rE8DNjTOCbbeXp63cvp+TFWOFKaohfKRQZGpV4dMYtyxz2BMC+EM4UL9iau0/VyvxXFhlxPu7dC1clrIr3G5kmawiEbrRWNsbsOvwWVMZh5YjQGbi9QHI/HzSqYQa3hQdnIWmvz8K8ZcpXNMjKT86Zh582sF9RnJ9FQ18zMrMYOEabqlSyz1BoJ7RI2Wq5cb7aAXM1+JydZGRcUDHgTmLROL4Ywx330jTnu86G2na6GToyO4+7HBCzI3mSjzLKAaDaG3MviK7+JwfEKK/pRG7ccL4WPa70BYAzCm+tKLe1NrUj2kFVeT75DGbOXqVhoFvolJZpocUMCU1cb+Vrbd471Ouo3IDhEpY49EGr73Ybiayt31+VxVX5N8Pc+Aynj8XzbKfQ3DL1pd8+TO3GNAtsouc86YEAsQ7RTRD8NUp6bPfbLdV7ocR6wrNKsiz4i248fP2767DJtfeG45Y1s19r+SEv77felce8l3S8rX+scIBhXF2nsWDtuz7NxgPr2cOLq6mrzm2CwfwgAACAASURBVA1llJZRdWWvXAQwlLV7O6qANzc3G0rKtf5so2ag/iEP2rMioKRmSJNSesBWPJ9DUA69LBzGZa/q82V3zZFV0EyGf+CE4wWJtbZPOhjcLXvLyfmfOgfaMbvtSiP9hqEZ6Ch9HpN6+96wMtg6Lzs3xuo59pyZ7ft4AYD+W4Zl1HaoAJ11BBC2Xq51/mMvlDry4/F1JzkydD11Gvywh/ve8pZDtkzsLE0ELANK9225Pf57vuoEK4+JIbLZvZGKQYn9aWbW1k/30RiyV94EMJ5Gd2f76lnv/6Iz9pYoLIPkWUm2UZSN2EC8QsS1puBmEA1P6F83zTWc8zgMmPawBgCHNg1vy5b6Opy1tmGiJ9chJcvOBnmDJMyG/tDG9fX1Cfhgq/TXzohxfv369ex1NfTFbyYo6CIvzy3X+e0dXOdnZ53gb6jZbRBNLaBnBc6J1ZlBui7/qC19Nfg8Pv76y1b9RSkzFoMifed+vyLKYDSxiY7XDquFOek8mcU5vN+bKztg98k26jYnZl1nUCD1PBkU/dOMnaduE/kt5SKA+VUaFD9cSseZqKKnabLZlCccz2BP6boBKwvRIcda2zcv1IOZHXV7hxmj8xZepfNPtNOW++G6zPB6z8T4KNxLfYzXeSZkipx83vLiLboAnPtrZfYL+DxPfrcX8kBZ+3P1fc2N81h2dtRtZ2RZ0A/PBfXYk0/bCKawjT7x1tjWxXcnoFlVt5ECdBgq9sC+RHTeuS6Pyey6rIZiVjaFoRz386tNj1gmdSjWLW9zsDPxPNFej9fx21H5PuTW0ldoW2cLkpzfY6mbNi+dbGLTwrAHMJOgU/w3qJWt2TOsdU5Vnadg8M/P5y9va9tVJnvZvXyS+8NnlHPvvj3QbDjtHI1ZJH3HkxsEC5S+3214lQewYazUO/UZGZbheENtnZT1wvPr9j0mh+oOAytHK/S0g5xrplXsgqHlbUO1HtiRNQltQ7Ie+D6f93fPR59l3CsTqHmMfN7bCMz4C35NpZTFTzo6tQ/wTfZmkOlrcezYO+dlbVw3yWkCdJc3VyHpjMMVH/eklZG4837PlDvafArXWBhWqks0s6xure3PaTXXwT1up4qAMjo8dPwPYDJWe+a11ikEbXvUZeMqIFDMfJqDdCnD8PUONQE6wMJjsPctyF1fX2/m1zrh+XBYaM/vflunpvGY2RTQyqgN7jYk+ri3HWTP6fT9V5U7dZYJ2fEQJk/XeN73AMz9mhZSPOaClR0xIW0Bxvc75HS7vt6g48Jxp5U8567LfWVurBt7dV8qb+bAyAd0lYLJdl6gDMGfPUAfa36q7MMoXW/WUBIh2IvzuTkr6rNnMPhUmJOicKyMwsZZFrqXeyi49j4bz941LlaMCWRgenUgZVoGoz5cjUypv84BuXSryMvLax6qelKlrbFbpgUH6mAcDZMcRvt65FsdJzqYDHHqq68xg7MsvD2Gdi+V5k+nPFPlwPnmxihmm/6b2sPG7Oin4jpcig+WkRl62ZjHe5HB7p5Zvxo/Pz9WxSwL8+DKIq6uXld7+I5HtyL0/UpdWeOzw4m9nJTL3d3dRmAUG0EBpb/gXVC1kpSK872Jatrkf/uLnCqvidX6uxWyoNHwHEP1vDmBT8LdjIF6qLu/Y+Ck7NRGHYd/h2BSzuZfqtiWuVmW5QKTYkyHw+vzuM7fOL826UW3oXi+uHdPdl595tqJVTrMNwO0zKs/7u9UDPKTTUws0GBveyg7m+RkIuKFCMbch7f3HJCJxyUHfRrHpZM01LyIcwtuwINnUrmf1S4LyUlFjjuHQz1uo/mYtbZeo4nNtdbpl1c6md506k15DsUYj8MKlHZSIiuOH/GxIgN+lsHV1dUpVDSt93i6D8iyJNyzLLln2luEfBqu853//IAu/QOc6ly4BnbVpHm96sSeuQbwNrvzfPYZWesQMvb8sjLrpP7EFgDCSTYOs7mGH6qZdAqdaW4JMG10Qurj6up1XyWsl754fmobE0gVAP3Zea1pjsokret8t+PmHX1rvaZNPD9uH92cFtXAGr/V+VIoeRHAbm9vTwZA5YBMJ7cxsr2xJ9SejEFMq1Ze/rdgUQqv3vjBVPqGorHMTX8dXlL/xHT63jDH6f6jnw6T8LLIxbvhnT+xMU1bEuh/vVwZyfF43LBk5+rs9f2TdZQafdkkbMvhvsHHc9OQkLbsfS1fh3o4RetP86VmZjagMrkyvpeXl837rmA5ZtNuwwzKuoGjo46yd37hvWzTzJjcYyMJM9g+gO5ramvUwWeztzrsyoT+0I5JR4HS+juFe9UD9xubtB0B3A7HwRYvWFgfp3IRwPg5JQMQHakHdsc6IM7jUcwyzOYQpBXDObG1XnMxZgr1gAYee2P3Za1XhaMe5zwMtGw5sOewsjBpnggnuD0e52W8x84ASB3c65Wy5+fnTSL3cDhsDNpeDmVDXgDSFOI0IW/52UjKiNwmCvfp06eNfBiPGQNAzjyg8P6l9OZpOO6Q1w7CoRCO7unpaX3+/Hnd3Nxs3uTLvbRNMWABJGWHlgPhEfPQcKwbSJ+fn096Z2NFr61jDk2ttwYQyxbHYH33WD0+gz91EDW4Ho8DZzQ5F87brtEzyIb7gHwpJgjItLsSpvJmmt+VoPAOK22kptd0iuNlYw7LMD6MZPrhViaD6znnbQ8os4VVZmGva+ZGX703iL9uozA4TOyhQOV+cN6gyPX02wrpMTLmtc73V9GHbhVYaxtydZOwjdOGYmZoQJ8ofWVsWdnT2/A9Zr8/zszMYTwhm0HQ7WE46Cgs53g8rh9//HH985//XLe3t6dxlE1Tr1lsnQlzYVCzgwHEpmJGZaDyvNaoucft8t3tVr8859YFxs35RgPoAPKnXhyB2ZbtmmLbdv6yDNEyrtPEPtufvXIRwEy96fhESTtgT6xpqY3A9zfR7PDQgnNoaO9hgXh3vwse2cKZkqSTcRZ8uM/9mibKoGxmh5GZIaHYNVozSHtjezOz4+PxeHoRZZP1DqU5byC1ogIkzH/Dj8qXOjtWgIfzXOMwweyvim8n6DZxVg5xaJNcF/2G7TRPiDzr7CzfghjzXYZydXV1prcGxbISM0izN3SjUYeLHU1DSved/tqJNjrxOTsGg6PrsRO3zprceHyNfCbGjy5aZ92vRnMuv2kbRVdO1vo1MQ49RtGKpDU2rjH9ZpIdFnKtwc6DN5szS7IHoJiqe/IclrmP7r/zFo79qcftMCY/fuL43srh1bGCqJWdcUPPCQH7JgfL1izAAGk257wd84FRGtxfXs4fhUJRG7JMyjflNOijQ2b6j6yRAy/K5K8M0IZs1gj42tEwBjtP98HynIDGeUzrCGC51utTKt535XbszMr0mltCRtVBlwK7++eoxjri68qe3E+DZu2J/wZgZO8IyDbEPSYdDc2ZA4esHXPLxfeBsf2goZsT4RQPBiM0k3BohKD8GluM2gZv0LTAPPkevENEg1YN1WOheHJtCABCvRDnDAo2BhuS8yAGT9iNjRTDoyALOwk7AcsDmZiWmxljKPQBGWBsfvYR+fkXuWF4HgPAZ2CxV23Y6D7x2QzR5xxueLx2apZFAbIhfB2fZd18jnWMNg3Ma50/MmMdmxhU7aCs0/PonKb1rcXjsQ5QvFnZulAZUL/TQ3YUsHXnX0tqug2q+mlWankbJ9Y63+t4qbz5QkOzDoPJWtu9JPWmZUP27s6fuSBMK33ZX0EMwdCv5nmaRC1QcX+9V0OIGiHny4Q6FhtW8xbI1OzB8kOZDofDSXFsTPbOTgY7lMWLM4YC95SToJ6mA8yAu31gYpsuNVo+ex4NJpzzPPgheMuWfnoLA8cmD24H52NlJHXKHDcjbI4GR2FwhMFZdz0XLHq4Heu7++Sow23uFeusv6N7yKrpFSIE63v11UBzOBzW/f39ZtXa7Oru7u4U8pvIID+Hw43kLpWLZz98+LBubm7Wly9fNsaFAfVXu6d3IxEj+w0RNlYrbWP/siCK43XXh3DZ90X7fb7On80cqNssA4Vx2NjcFMWTYZBxQrmsoGyCOjiHTL3sj9yYcO8PQwG658lhpdnTSRECRgXmhrs17LZjkGELBrIDPKZnWs06uqG2/WqO1dczN8iXvKCdosf9WzaPMmcY/f39/UYXbIANj5ojcptOb7i0H+jyW6zEBQdOv7xa6GLArK73GjsG5urp6WljdwY4Ox87yebHqo+186lcBLDj8bgeHh42+7esUH2+yiEgwvBDyl2CtbAQbr1yWUOpq/uEInSljTapH0E6X7PW9hd8LHDawUCnnECZKIZm5sR5A0iBwkn2MjJkaplYVnYSTbg6/0BxmGLwNEg3j+E+INcyaUDSgGPdYPuImWKNxQzK/aFN65BXrf3iSL/+xzpRBt/8U0Mo7uHapiMKuDicygzgRu/ok1eJkVGd6ltMhOsoyNL5tbXmHfy1A9vWlNZwPbRjuZM3tY0hn7JB5+rc7t5q7lnfL51sjqRxqT2KhexEr72jPc7UaUon38o9xeb1DBYS/eBJAOq3QgI6HksVaZrIKRdnsDAld+wPQ1rr/Pk6j6vH3CcSxRiT81fur/Nz9MsyNCigYAAk40FO1NtC35pbY8yVoefPQIaSc41/is/Xo0MFRK6pPK2bBi8DTn/YlfvKAroFAt3hz47OfQH4Oo7j8fwxGwPO8XjcsHDniaw/jI3/ZpReRKk8+O46LSfmhxVWxtEUisNa2L/n39c7PWA9tA5Q/iMGttb5b0OatpvyuxMYl5eazUo4Nikb1zKpZgW0wdK/DdN9mMKAThh98YrVtDpJcfxvA61ReYKt4A4rME7LzAppgLVym7XWsMp+aZt+YFBcx/iow8zS/TETNJjVIxt8zJocUrm+T58+na6nbb++m6SxAYI2+0iT5eMcTdmQHS7gZefscAjZOSrAcVim1GWb8AKP5eY8oOVvmzJrLytFv9x2HQr1tM2ySesJc8p3629lzxi9d6+65OKneAzgjTAY68vLy7q/v9+w0slpnsaye+b/BnV3d7cxtjIeGzRMoMbkibThXV9fnz3aw+fb29u11mtYZGBw7sRhYduZjMyCdd6EiXY9tAFbMt113c11ICsfLyNoGGmHgKLa+Ax+ZpmeeI/dLIExmQUbKBlXc5pVSMsXj0wfDYz8Nfz3OO/u7jb5M8CEuT0cDqccEwVdYww3Nzcbh2iQdyF35HlkHvjzKl8XX3wOkGEOOe/cX0Gkq5IGLZc6RIf1nLdsqXuKClxH9cHXOP3iJzq6E74gwhMTHoN1BvZMXqxpAsa6xyD3QLzlzUeJLCB7IJTMgLbW9j1Ips2m13SU/Npa56/xcNK5oGW2Yg/enFXjfRuMAaAeqWyQB5oZH/1jTGYxjBHQclK+rMSTaBBCbmZtFFPwssHTpAaoud8JYO6vo6k39OcqmwGr7U8Mzn1rMtmysbHRBzMl6uTRIOvVFLajw87Beq4Yu+eIPgGYjBX2Y6dH8Zi76GIdQ6ctFzOdhpBmxm6rYbOLdd06yb2OKCiOZqpnk1Nw2NgIB/ZqEK9TYFy0B3aY9V5iX2u9AWCfP3/exPwvLy8nZIYpOG9idG3IMHUEBbMHc7GBMrFOhOIp3G7zdVYGA9Nar4zKntgTYgEzsXhxvlcpmxdrzoD2620twz3wMGC3X5Y3/2tgXcaeZFIgMvugPx07MvTKo8MQAwLMt+2ZHTgMtGJ7U/Wka5ORUTBOA1bDKHTC4OzFDy8WTBtLXYeBvU6eaww8lqW3qFh3Pd/UWQfWPFYZD/d4Li0f2rQTsEN2vSY37SfHTGA6Bjs6zjm8tpPdKxcB7OHhYd3e3m5yGg4XrGCT0FBGo7iRldUKT15XN6pcKLEfEO1kkyMDAEuDnbuoB+mKTb1cmZHDLrNOew8zFMbonBT1OhR3W2YZrtN5BYOiv7s0Ie9+c75eEofhuaH/niOcB/NPv82k0B+DteVvZfVY8eTe5W0jpM/MffWnc8bnKZ9Y3cZQWQR6fp5/TYs+Wx8re4NaWatLF7dqwA77qvveHG5nYHD3Pe6P9RFd8Dx7LLRf4lGnVRCanj/mOrO+5tr3ypv7wL58+XL6jgelkwYjhGGFpZOeWJ+fdlqjHCjiL7/8chamUF/bQhBNEjopbgXrfTCOeh8L0+xtrbVhBM5XTYoHSHllacqFFMg8rq5s2UDdB8+Zw2vXzTEzRzOPhrqWm0MCwLRhrUHMYIOcO0aDgoGmRlJQnEIh5q91u88O2bh+j1VNoRx9c/jaHNkUPptJ2oC5vuzMbZbldqwGJ+dZm/fiOvSlzJvx2c6pz3npCVwMOg5bp9XU9pvjDfH3ykUA+/r16+YNABaWcwAYiemtBVwUrofjOLkJKwbJQrOdhkkUMwQKCxEI1gpWz4TgTfmd8DXD8TgwSIOm7zdAoHywT8ba8Md5INNt/3Vl1MzHygfgkMSnXoMd57oIYVkY0Kmf663ULp5n+ugFE2Ti8Rv40A0/aO6xWd6txw6WYl1GhgZEJ9y5zoBinfVjNTZYjN1bKRyKOTeMrD1uO1zrLH2mn58+fToBGvdVh1z/pI+T7pUdVufN1DjGGC1n6kIWyMvJezNgM7v7+/v18PCwCdun8uY+ML9Pu3GqQ0uooVfxJm9mUHOd1MGgJ1rPRHmZ2spY8DC6l8F11zbFK23uM/fZW3OdvShK20l2WOjPjKUvPbSc+fMKEePxRtnn5+fNw9dQfJSmISeMzYDunAR9gRGbeTBWr2p6DrluUmpkwL2kA1BigwufnW+tPLxQUsDwf+dvPUbGwmfk31yfdcC6YGbn+W2ivX2knzgT+jAlyG3k1gnrqqMjz93EzCkFn+Zyy3B9vfchIjvaM+ADrtRjZuWw0fp1c3OzPn36dLZxvOUigHUPGOXh4eHM0JvbMNIaaBp2Gmx4uZw9EApnquuJswE1H4By16M5Du9zkwZgezVPCqUAOHkirpvA2Nc6H9h2AG4MuS/w88Zgv0qHAqj4wWyucR7D/XeI5hDSDqMG7L1VZl0FNebAK81VUnJe7pPDVO6zc7m6ujr9WC2g5M/tVwHW4apBws6SvqFX1jXPhcdqtmjgY14Zm+fezLvX9jvsGjkaSCbbmRwr7fp3A3yea5xWsLN07tf39Zjnrjhwc3NzCjXdh6ZiXC4+zG2jM7U2sjtnwufmuFxPB+cwwMli7reAS9PdT4cCnRSUl6fp2W/kxx+4zuN0TsxxO2OhLdPyghf1TJ9rtPbOAIEn3PkPG5vH6vpr4L4fIOuPl6BMgCwKjcwbcrs9jLDHfa/ZjUM+OyD6PrG5AhZjNvt3mOz6uM+sv/qHLOsMDeqMg+011If+2tl5FbNzzXnLrVs2anv0hbEaVFwMnhOL8Wp+iYRLWbMjKDND5qo44JC795e5equU5X2pXGRgKI0ZQPMMZQ0OG00vqacsDMNyGFZKabbX0Mz1wLSa85jqKwBQphDEuRjXyxgdqlpuNgKO7YU/tM1TDPawbtthSBOw9dIei5kUfeJXoRzKmn1aKQ1ATgE01GH8E+Ps3JiBGZgMEmWCDekYL2G4AalOrvpiXUS3qcvMYmK6XOMopdHDXgrF8ui82WYKOmaqZeqeD64F4LzlY3pG08U6ZRm4T46CKCUOZfHWdVISBjfnVanP5GevXAQw6LdDQ3sTe4IirZGY3IsH4AH63oldeWe6laIAxzErko0Yg/QO7z0B2QgvleYjyqI8Dht6wYY+G+CYQIdw/HeyGBmgCBz3UxE2VMvDzMwMhOfZzFy6i98LA83/+B6zMivzWuuU3DUomqFXoa3U/o987cn9rCPzUUPqsaYuDKzU8fLycnr+kvvQUWzGTLdMnfqp13Uz9w3b+d4FKutYgaOLPGZ7Zre2Hz9j6tw3xUyR7567tV5TMRwjrPfbanhZpR2S533q21Te3EbhpKFfDWzF6hsXrGgYgUHLSo6C45HYXW3l4npyIrRj4ymIOBdQj+a+18tRyhKneNwMteMpuPp9UFPo6H67LgqyazK0St361toai+W61mvYx1zZ4MwybfRVWMu5SXIDt5nslAeiXTN1xj5tpWBMDTsArj4ZYYMqw+Z7V5OptxEGsrI+2A56Pdf1aQjydE1RUA/X2Plwnn4b+N1vszz6UACyfhhIaM+AWkLQvpqIoO9d2Tbb5lrrpHXT9+6ViwB2d3e3of5O2hVkakgWlnNEVui1Xn81mEGZwjt8MiBSCHNQAAvVG/oQvHfptx/0efqMVzLbahhoZXOh3W4B8WKBx+OwyquGZrIFKAzRytu8S0MzyxIAgXlZRtzrl+61WHkrO/piR1I5o6gOjwvmXrhwyG3m1Tn1WLxianZL/ydHZ+ftbSYNP9d6dTZmxma3Lnz3LyJ5PxmAbWB3HmwK31wvfZ/kSJ3WM/e784Y8GCvjNFsvuNop2ZFbJxvGuz2z7UnfNmO/dPLz588bT2YjomGU1+GdJ89MoejrlSCUzzkFztP+4+Pjenh42EzklBOwUgNy1AMjnN5x7omeXnHi3IfvaRh4Eu5h+2iTx+8QoYzHit9QlnG4Hj8EDWCa8Xk3Pe1abg5tPT8Oh71S2RzOBPwNAwqmVlzrQEM1PrfP1r0yfifTJ3CfwrbJMRQ0qd8A6nzqxMpYjKAfZlrMj3WLes0KK0/Ph8Nz7rXMab+huHNkHjPF1zsU7mJCwcegbn028LWt6kSd9l6UtNZvYGB+75RXBZunacM+ZkUz+JlyMmBi47VeQ0a8w+Hw668s1xsgGCbFynA4bJ9fLG11KYVnLPTf46WNMi5/t2GgLJafJxuP6WRmcxB2ENRhT0Vd3ovGd2+hqLJ7u4IZtUPWysqsx3pBOwYbrllrnSnpWtul+4b3BiGHsa7b6QmHqwYk7kUufKbYIXtsyI++ePWPdjo/U8hMG762Ty8gAwNs62Dcnn+PsZ9h1Q7PTEbqkD2/FOtGWZ774nunaAT7czTg+iewtsOYyps5MH7N2Rvt7B1p2KBg5mAPUfA6dUIMjAHY6zZnYGVtLI7xeuWE/tpw7cE8jk6Sx+d2bUQWPu05BOZ/PbQfpyFp7tCh9Zb5NkHOnFFM5afwn/oxarPAblB1OoB7fL8dA7L/+vXrhv3Wk3osOBo7DDsOy7HADmunHsuO/jjE47wf57JeeqXVBm9HTJ1mG37pn/Wp80gBlKcQzIWtLh6bx2oW19VGZOaFMNuoichUOm/9IR7PjW284T5bTuygGwXUUb4VPq71BoA5BIGNWXj1EighhultDR4YddcjudNWuoYODS9coKv9xZi1tquqrdvjbf7Inq2exjIwkNSrUE8T/ihQV84ccnLcCWHXD1ABFBOIwYxsAO6fV40bunEdhgIoINPKn3o+fvx4yvMAPPS97Pvl5eX0kjyzBwOAZWEQ9f+9BYuJ+bltO9A6jeaRkF+dOP/dzocPH05vezXL5j/gZBCwnVgO1j8DrOfMqRqu92pkS8HS+tuQ1+cdiVAaFVmG1nXPf+fR4LcHqpv+Xjx5eE0wAgoYhEM2OgnSN5a3EjkJaq9KZ/uzTGttPZnDUIqNCRZmD2Sw9QqVDZRzjMlssHktg+20SlLPYS9dwG6dKAXjJN8Fs7TReZWt3oyCQnvu2g4yNmNDPg4vWwyyzKVlX6Cply5rQv715DUi999MtMwLuVZursMyrnEWtD23nLfz4DrbAMBMXZZnGT8A1y0OXGOHxnwa+HBQHqMZlO2WNprMt6wa1k/zX8AuY5vsyW+LcbTiOTRgT+yV8uZGVocGDLCg5UloHGumZcS2J17rlV15MyLC9iAueSjTca/YWcBmOhaMPcTk+U8CO5yvjPJ5mugqq8dmcDR7sJw476V/s6auqNFHxuI8IvWZFft6O4kqXsfesAPw8jWPj4+nt5kwJ14McHvIxh7ax6uHNhzAlrm1jKmzaQT6TN/o9wSU7o/zigY+rvOqs/vteXMqhjY4z2onxYtJzS2amRk0kY/1x8VM3KkCF2xsz3kV1D9+/HjSNeuFox6cDfswyxT9ea1Xnb/ExC4CmL1Mjc2hjge6R6VrZAjBnTMdNr00g6I0D9WJssJ44jleYGsoNLEoC5PvDX33hG3vWCB3m80D1BvaaXQ1FplNjxhNTx0gDxfnDj0PBQ/YkoEFcLTx4pCcMG744/bqMGAbBibLwWNof8xQLM86WeQzsSzkW9bvUNzGx3evhvKZuuir57D64XYKuh6TQXgKKSlu1/dNEYQZW2Xl+5hXnAebe2s71eGu4ttWPf9OB+yVN3fiUwEob4G4YRp04r3epx0zrcfDGIyo217InqYI7lyLldrGX6W2R3OxkL2NxJMw0dtpRceTY1ZRBmPA5j9tVdHsQQ0MDntoE2brxQvanpbRkaGvqTcug+a7XzHjYs/qOpCvdaOAY8O2zrSu6VxlbpnYEdiwzb691cXjBcSdHOde2jTIuI7WZblMDrB6Qvuuv0DYiKD6bR2ZjlU/kGttt86t4G+2b7xgAcBza303Ifi3AYyJhSIaXRtKIdB6hA6gRmADcOhjA7+9vT1de319fQJTG+MEqIyBvqGQNnRKFWcKL1sv/Z48les0QJ8EL0V03qlKVDCzzDlnhuNQbFJs5FiQs3zsJflshZzGwr3ex9e+eN7RFXtgs0UfL2ujzQnUm5cBYMqeXbfnwec4b+fthLhBgu/ThmoDq2VjJu552AO0guvkdL3laALDaasF3wvyLy8vp0WYMiXrhMHJxMXzaUBEnmX4nQfGvBfGrvUGgP3ud79b9/f34yoIAqhXKrX3OXdoz6NaOWjny5cvJ6X3A8ZlUS71ANxjwy8rsPIBCH6cYfJabW9iVfUqa72+PscLD1xr6tzJ5HhDUOTakNxjXWttmNges6J/yKwAOYW4Xixwn8xQuJ76zLyYX89B9cRzaUApmBbkpygAOZfpWod9bUOv6jWy6kZNA+0U8lonCpq0a/nZKdKW27NTn3S1DGsvIkhsWgAAGKRJREFU9UG7Zu2U6ovr8LYK53HtKBwtMDfe8G1deYuBXXydzsPDw/r9739/VgHe04M20yl6Fl2N3vWsFgoo7p/DmrxOVz782V7Syuqx+Np6v0ve0McmhdhjCZyv4dmImFzLyvKhPc6bLSM/hz9W/GmrxlrbFSQUkHN9tY/nDXBzbpR58OqWgal7qLgewCs7rwFP17hdrumik+fSoRZj8TXMT1Mak+zMoB1NuFQX7FjsHLzi6HsMeG53Cvc8/2a37UPnoCkFAxTFc97jlmHnraH4lBqwA2K8U57uNM7dM2udnn/zRBpluxWAxvGipomTd3MIxCALgkZmA4o9vz2zz7n0gVgLjf7vhTpTnQYcvndRwpNW5afebtHAs045jbZvZaCPZUdWUBTZsnC4zkoS11O4pkZQ42POGSO601SD++B69hZsrDN1Tvb4PdZxOFTmfx2g22BurTOAjg3NjtltGYiYL8977/Gc0I639hjwrENTmGUwn5xoc6G9j7rd/6ltsyTGbIdwOLw+CeM59BxZZhTqRY/2ypuPEpnymcabuje08CTXA1ix8I4NQRlckdz9mDaiToKg8EiUBTddN1F6X4vCeTvAWtvd/g4Zuc+yqud3/WYYlC6euH9uFyqO8ninPoZtI5oWDPyDohSU2Abt7ygqIaDl6LH213walpUtOkVhgOF/xzexiQIx8rIRNfy2c0Rn7MSbU0OG1OuIwjkzitka381IcIYlAZPuTKGsy8ReAEbfNwE+xYzL/ZjsqCuH1qnJtqb0heW0l7I5XXPppGmsN1KSvzEj8wS4UYTv5DmT7Guc+zJY1cDXOk+qlyWZCdlg6H/3gK31yg4IYfaKvSD30YfSZHuPvdC19L+TjKIZPAxI1GHDpJ6yM+q0ArvNhoCcm2i8x+U2DKYoYENbA1eBoMzCsrGxOn9S5mlGBADUWVqWBoOuLvupEsvZ7fEDzTZ05/cIvw20Dq3dXnW5Ye8UlvaY62mp02jO0fW6z2VcjMFzYdZVNjWxMzv8yUExl5fKm9so7C35IxFccKoH8ncbwQReKEZDTX9mQPaQZif01b/4jacFdL31oobiejk+0X8rB3WaXtcYprIXErr+PYbg/lghefaQjZQOk2CgHZeNAiOctrFcX19vnJBDHT/szH185/lAhyyACWNyiGbH0+Q/pbrnzZ/NGZrp0187NB59Q+bIb61X1o5umm3Tb5yB+1L98Hx7Pm1bhNTeumTmyfzsAVML7WGrlh3nbZNluXWGlqfBnOL57yuyPFetj2LnR32/ZawXr+gv0ZhVOIxgAv1nRXE8vWlcwEFdDjVtaBUQ9Tks3EPtsojuJUMZvaeK66tULfbua23zGPWeBYz2sbmYtc4TvV097SpaVza5hlU/h1VmjO6fw3gn1b1r2z8L11yRAez6+nojZ4NfQ6CmBszs7RCdmuB/tzkU5Bs+W//o2wR8vs7MznU6UnB4yBh4vIq+lEH7cR76AIOzTD3vAKfHM+nV/f392eNgZf+uty85wLbdNy+s2YaRWR2rSYhtvGTHerDWq51fYmEXAaw/2UVhYkypXerdO8kON82GUI7SSNNTe76pXQvGx3ocYdfIrVj0008g7HmFerm9cKhsjsK19vK+jn57JcoKWOdgBuh6ylDMJrx4gOJyn3N+VmSDlWVAu31/m8Mq60DZzcRUaL8pCNqysXN8Wi1riI9hUY/l4Otoo2M1S/dbP5oj9pw5vPb1zWlNoST3d8uGWTbFOVHLq8WAZfbDvONcDcxrne9zaxvd02fb8Bxzv53C9AzsWb8vnTTqlz3AWLhuSt75R3E517jZwuqydMNIBOwEuZWfe2zkExuCWZohOtyi/zYWAyf3+Pv0BgiX5qscilGs5BOQWc6nCTwcNkrniXffDehWJOrgPudrOMe1ZsMOx71HzAyIvju8qyE1Z+Kxmg26vwVQ6iozsIFYV60vdlrTc4P9rVLkbx3rXFV+Hq9ZE3oJgzeY2d6sM+hddYexmUFNNrvngPdA3uyTMVAc5vdRsTpjs3rq8zW0aec0LUCc9fvNK/6vmD0xkaWNDIrr7LV7nwHCCuk2LEhAwStCbsv1lGpzvnR3rW1S3AKdAKMMw+Oenj+s3KjX/ajyc02T9h5vix+YtVcm/LNhtK9eLJm2Nrh/VS7nLQy0Dg/tZLxS5/9m4mX69HHP8OoErV/U4e8AroG0jNbfy5i9IDGxQDvGyZH14WY+ozMlAvRnYio18LbV8ewl/Kd7/RNne6kTh5Z9w0QJwgS4TZm8xbamchHA3AmHgs35MEhf6wny+cbyZiacN+20wKiH77CG3u97bJATjbdH9XEb+STYaUHCxYnpjpH77V0nb9q223/Ax4lmA5KTy82VUY+VzswMpuDEdJWyffUcuP/Vl47JTMpys6Pp/DE++tzw5pL8kD/9ar1lLb63xsx3nos0i5v0hjH5rSjuE6W5KPpbkC/DK9iXWNhBu/i7galy4Ttybv7ZusS1U8ht8gNDNpO7lPDf9PvSSb+ccK1Xo7u5uVmPj4+nF9A5obnW9hUd9nwNCWEGBbauRFKPJ9OMzhPTjZRdeaNwL/1jtaxsjH5OsrGAHSYVGG309J0691aWCpBmA76GMVxiqF4pNcV3XQ61+64o5N+lfxS1ITznrYT9xR7L1szTcwIrdOqBcwZts/Sycs+3S/XWffAf+ur7ahMYHg6je7xo3yAI4NVRTgBgXaI0PCzL6nGzqYa4ZZO0i7wtH+cu/bN91iuHuLZ1+tYICDCsrAu+U7kIYL/88ssGjBwKXF1dnR7cpQFe3EZC3B21oSHIJoGncMkD5XtXRDxAv9wOJsVkd48TQraA7Sk8Nq5ryIiyYPQFDwNvWSdgUeVxHsgycB2WB3U631SPSt8ZE9cbbMiBNXnKPFnWXalFXu5bx2S2wpw0DWHjYyx9ooNjDscsAy9IWLaWTcM7G07B0jmqp6ens1dYc0/lUObC/DjE9hz5x15tvBx3sV7xvfJGTtNx20/1bIqS6JN/9pDjXQACEJ34r6MwybCzwWadgrpULj4LeXt7e1IeK91a24R7wxIUxGGHlcwC5HwBwiyIgR2Px43BG/ych/HkMhlXV1ebuL7hbcfU+lyXJ5vPDs+sFJZPc4U2dJ8ziNeTG7B8re/1UvXeWLw8bllPMvR8UR+Kh8ycWjDIdWw2bq41i7Is6YNZg+Xg+y1vh6zMWftpJ2FAdz6q47cxcg1MqsV5ub1inenceazkMqektuf0t5TqpsteaFrn4+tsG9YLzwnzYVvsdQVtMKdAfzb+S4P1GxnsBelU3+OEcApORtW9AYHGDTP5/Pz8vO7u7kbhGegKEs6NWEGdNKV/TIQVGkNpKa11uNUwhHotE3/3eAqqLvaMrX+SyVrbH1Fo/507goXVSVnh3b7Zk/NNHbOB2mE+QODj1R+Px2PwiqLbxklRN17cjs6657YdGnmslqMB2iyuaYpPnz6dvSnCKQ+nDcr8Wzdy5lo/G9n53mNkHZeP28583KU2zVwVdAw2nhuniNqnOgnq4HcE/iMA86NCftWzBdbvRt69eNYbX43Cvt6I7mvdRhGb+xpWFiQmcPQvv1TgZTMeW0Ndt+F2mzcpo7AMqbchTmk4bTU34j5VOSuTht/+bCYz9XGt131AhDo4Njsns0WU1nmihjCeSxvZFEb5uMGKsQM4jRw8x3agntemKsyO7IDs/PjeEN9ypj/YFtfZOTSHync/GlVZ7DGxPdZlcJ2283QuWiw3rrWs/dl2xznG3PvWes1Jv7UX7E0GhmFTIcKssKwIDmem0GKtrbetQTOg5sios6EWiG1wtND9IwlmLkZ6rutqyRQeeByeWOdBbLR9ntF97zhap43TcuO+GqhLDc3/ndifru8bK+w4vHrH6pHZmUufevCYzB6q4FwHiyqTaXhJ8aq0E8OMxX1tysJg7/u7UsmWgYaaBnzas0zt5JEHOcc6IkpzstPLNd2H6XtTI77G43ffKAac/nfYbRDcY3l2SHVK2K7TPHt6vRnnxZOHw/ry5cuJoeBdndBEQDTocGyteZtEqebk5TlXGt5BmUk1z2T26OL2zBb4bUZKWVRpMlTfCjx5i/5cGHXRR495otluswnTevAWszaDf0MAt40MHDYZzDz3DZ3RDcbHDxET1hVA11pnCyDuT1MCyMP5L+thDdXG5Pqb+6vOcA06Z0Zqhzhtfm1I3bDUDMUrkdNPAVKwK9tW2XDHwTUFa/o96faeLVKnQWqKMuyIGumY+eKI3c+vX7+ebTJ/q1wEMHeWRySsGAiQhh1Sco0Fa5ZVr8ZxKxkeHmbkNs34Hh8fN8l959y6vcB5O/+fjJ98BnV2cqmTvIbHXE/XCZkmqHKyolvueO6J3gMSBqtSeReuZW7NQnqfd6VbF+qV3X8eJfLcIDMU2ivHZVoUy7566JVy/69jrUxxBh4jc1W2VMeCDOxMPLYyGwMj1/qz27vEOsriywzbT+yiD2B7dR5ZrDVv38H2eKDf/bDsyc9Z3h6b7cWAZnLTBZ+3gOwigJUiUvAcKOz0gxFG2z3DmGiugQAhwIwaXriN5kvKpAAgPCnCN7X1xHGfQ8Iqltus9/ytHsSloSSfJ4Ay/XbfJioPc5iMykZe5V/rlYVZEZtnc998f70338sayloZn+vpeJ30bthXZtF2AU+3a5lUFwy+lpPtw8XOuXV2LJO8Gw66D23H93ZPmcPm3s95g8ha52kMO6s+92gAdD8LTpVVQ0zLxMza8torbzIwwKodZRB0ti88YzAWFMBiT049/m5PQTH9br0+V8rquj0Ge4KWCtBeuddZVv5fFnASeIyXUo9HqUdsXQaKhk1TPfRpCtes7JeW23uuYXAV1Xmh3mfQsbHUqRiUDRAOZeyk2kbZnNmfw1nLz2+JcF1T2A3DwCmSG3Sk4DEwZlhcd95T/OwvsmecEzh7DNbJ2u6Ub1trC0xuv29kcXTl7+5HU0YlILbR4ovTA5fIwEUA++GHH9aHDx9OYVRjeIdnhG1eimeyPXh+O84My0yrg3XnvSXgcHj9JWcLpfF2BeZj/rk2C87t2wDKUK6uXn+gtue7TaMhwyUvzFin45a7++eXNNYYUIYqF2zUYUA9ZYGO9glNPH8UlNHJ/8qlhuliZXZu08zL+tax9S0NZhPPz+e/yk70YAdoffRcdaweF/pAH5w789gBri6QMCZvs4D1VL5eDTX4U5qO4VjZZR2znVvnkfOMhXYnp9Q2pucezboMaJ73CXw3Y9o9s9b6wx/+sO7u7s7Y1WRQ9jpmH8/Pz5uf2sJ7MBAEDUU1wjvUrPezAOoNCg7OF6GYxPRlMBVyWQ3FQNrVJtqcqH2ZhsvEmAquPmdD76twCqjs5au8HQbYCGyoE8hQHIqarU6GYVlVKS2rKvTUD0Dbcza163Fz3O/mYm5w0sjOjrfz0D6vtU4rk1MY2lDPq5jWIcbBbneuvST/PZbt6MastXpMP+yEuR7ZTk78+vr6bO+c89S1y+aI20/3z3Y3OfrNHOyeWWv9/PPP6+bmZgMgRkQ6XO9TIPFAuN5MZ48Gd5C06Xo9KQUgT5AF7Wf9proRKMdMY12Hc34oob0thuofNTibgLDJyTlQ9hhb2V3fgYXMDUoAQL038mIMnnvPhRdtkG1B03NvGfohc8boENY5KrNAriGnV7mYSVHssCgTeDJWr7q6b056T3ra7Tbug/eFNXw06JpdUti+NIV87ovnqfkjy9q5WtsRdmhH5jGYbLy8vGz0nzE2ZHY/qj+eB1JQDrvfAi7KmxtZvWq31jkNnoDIArDwoL4Ngaz8/PkeA5SF7PwXguCYAcvCsMIQRtFe6+Jae8H2ZQobO1EGQY/b+8489peXl3Vzc3MKYVxXFcvgBHg5h+dS45tWfCdF7JjsPBjbxJB9rsrfXJadT9vzkr/nlOI9azYa2kbOzScatKZ5tEPyuKyzLlOuCzk6hHJfYc6Xnvubnus027UulzV5PE2POAfV0NKyctRUnfQc0S870DJop3Ka4kGfjBtvgdibG1ldeRmPz1lxEIAVtF5hCs0mcPB1BiiEV4X2da6ztJQJ8gOna80JT09YWU/ZX+XgYz5OW8ioY8Xg2hfnI5yvch0oc3NDjGWvTJtOp/L09LTx5A4f19rO9cRWOic2sLLQKX+3145DGF/rcKilgDP1d1qEuGRYBsbqSfNibtP/3/psoDGgTvo3/bk07bMX0Tils8f+zeToZx0Yx80Q7Syru/92Dmx6ZUmRncaZlMnACnp4THul5jxQZlPg5oWa85iW0C14My0rggGvINlJMvBNxwtYa52/idRj6KRyzvt3qMP3EnZOSXbLZpKly2SwHr/nc8rrGID3wHECwxqw2VzHYVZtOdmRte0yhIndOZGOjpVFwigqH3bQU7pah7zcN8/5FFY1GuGe5lDdjx6vrKfQzWUKrZugn2yi93J/AcrOwXP58vKy7u/vN7rlfWQGtH87B7bW1viOx+NpZ7W9jDu8x1ScQ0DpoaNsRC3a21uYflqAHuRbFHWt7YPATJANY6LdDdmopxSXCSuTpB6+17O5Dq+00g/LmT5Mhu3xeSwYqVcqYSUFYu7xSmaNwHM7AY7bt+PzIzNOMTQ0cuF82/GxCWDNJHs/xmIZTvOBjkxPcuy9ywsjLGs3mFlv3XfkYpk3OvA90xYd1308HjeLC5Vpx4Q8zFid4K/8KXaUzXvZ3usQWNw7Ho/r/v5+rbVOuxQOh8Pp8cBL5SKAsYTrRx1YuUIQVZAq5+Pj40kBSpfLhhCa9yNNjMaMzpNdj+TvZhRmaggZhfZqme87CSzMzUzi+fn5lHR13VbaspUarBcGXM8U2tI3g5BBoeG86y3boBjk9sIuj22trdw7p/QFoz8ej6f3xnWcBUv0qADgfEsNyQzGjKhbDMgVMufOH5mJlFW1zy7oTcHWsp4MEsZNLgynxZx7UcMyKYBjr2a0zb3Z6VlXuLepi4mImKwY3JpTdfTD5z4hA6YgZ88zj+BdKm+GkEyijZoGjbgGHfIjeKPmcroQwP3el1XvwrGnp9dnGx0S2nNO+TBPCH33mFzfFGrZMOxN3H9/9/1TfdMzdPTZxauhBQb+/AgJTMv3X11dnRS0e+cYN8VjqKG2Xo6hkM7v2Mg4x/zxuhkzUrNy6nOOz/2rIVNYyfIzrfSZfhhILUPrsce9F6JXJhhnV0jXWqf+2PEiX/rs/k59Ri4GU7Ntxo1MsBHPWZ0cYNGcVl9XZMBDz52W8H3OvXGcFBP98d7Jkpb29z96G8X9/f1GIc2ySnMdIpnh2Jv4fjrrhYLD4XD2W3AOObr9wWBR8JrCPoczje2dzDezIBfl+j3xXNs3WTR87UR0pWYqBV7GZYWz7Av8ToY2HzO1gXHvGa0Bwxs2HU5Pr8nxKpsByQ7By/zNhU0hTJmvQx7GwJ9zXU0NWBe4nzKFap4rh0PuX7cqTKEyYA/r6Vs0LHPX0UiC/hDSAqSTHXj+HSY3jEYm08PlE9u2HU12x7VgA2+AMSP15mKz4Ia/LRcB7Lvvvlv39/cbAa61/WFZdwTBmTWs9SsQOjSYkvFMJOBmim/6XA9WwTbPZIOx0nN/QaIgQB2d5Bprww/61K0DJ8En/HGxXKYcBCy34SAKhFGxIXLP+M0efdyyxbt3tdRj8T4nb3loKIKSGoSnv8rIwOnx0UaBxvMHcE3PCboO53UtA0AdZmSHcSmUpJ2G+QYQGCf923vtkhmZ9dX24XbrhP3n/KmT9XbU6Lt/E6P6zxhqoz3X3JtTRx4r43SYWUAcZX3p5P/8z/9sNgFSoR+CXuv1J5is2Cgj1NYC9oT4OHU35JyEUy9vukup12r41TDOXovPZhdNvk4Mir4xtuvr67PEfBmXcwIeE+1PYaUVoOc7Xoq/V85TQthysbzor4tDRerjz4C81jaHyXcDJox9b/VtLy9ZNtQH961/zSk6rVGmbMdgeU/6M+kFdlFj5LhTIsfjcd3d3Z2NDZn5j/Yq1z3W5pCMKIpHBR0alv24jmlczEmZZCMerkNGPsdiA9HKpVX9jQx3z6y1Pn/+fGJbBgwQHq8yLZPTOZDdRg0LQ2jQYbxRH58oPTWjqqfxO4nsTeizBdhJKItDeA3LfG0/Mz766ZCY482zIAMnpxlPx9hiJ1AmUjbj/35Wj3psgO4L/bc3bmIbhmUAN5iVZeEUyixpr2zIfXE/zTZshIyR7/b67lvnnHrM5jxfZlKVAeyT4w4RzVA9rq5kPz4+ruvr682Yp5AefXEUYec6bWhGHz1XhJ5mXfTTqQ+zv0Yi/DexMStrusNzbXa4t+J5qVxdQrf38l7ey3v5/1ze3Af2Xt7Le3kv/1/LO4C9l/fyXv5ryzuAvZf38l7+a8s7gL2X9/Je/mvLO4C9l/fyXv5ryzuAvZf38l7+a8v/As0toE27V8j6AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "e = gaussian_derivative_edge_detector(im)\n", + "show_edges(e)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that the extracted edges are more similar to the original one. The resulting edges are depending on the initial Gaussian kernel size and how it is initialized." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Laplacian Edge Detector\n", + "\n", + "Laplacian is somewhat different from the methods we have discussed so far. Unlike the above kernels which are only using the first-order derivatives of the original image, the Laplacian edge detector uses the second-order derivatives of the image. Using the second derivatives also makes the detector very sensitive to noise. Thus the image is often Gaussian smoothed before applying the Laplacian filter.\n", + "\n", + "Here are how the Laplacian detector looks like:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "There are two commonly used small Laplacian kernels:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In our implementation, we used the first one as the default kernel and convolve it with the original image using packages provided by `scipy`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's use the Laplacian edge detector to extract edges of the staple example:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9SY9cR3a+f3KunGsixaGpHiC50W4b6I3hjT+CF4YX/speeGXYbrRa3S3JIsUia8qxcs78Ler/RD43mEzB8uIPA7wAQbIq896IE2d4z3tOxC3tdrv4dH26Pl2frv+LV/n/7wF8uj5dn65P10+9PjmwT9en69P1f/b65MA+XZ+uT9f/2euTA/t0fbo+Xf9nr08O7NP16fp0/Z+9qsd++Y//+I+7arUab968iXq9HuVyOVarVdRqtZjP57Fer+Pk5CTW63V0Op1Yr9fpZ7vdLsbjcfT7/RiPx1GtVqNarcZ2u41yuRy1Wi1ms1mUy+X0s1KpFJvNJqrVx2Gt1+v9QKvVwv232236Xrm898P+2Xa7TePlfrvdLiqVSmy328J48mu9Xke1Wi18f7vdRkREqVQKV2+5x3K5jJOTk/Qd33u9XqdxeV4RkcbKvD3Xcrkcm80mNptNWgPmUavVYjweR6PRiHq9HpVKJTabTZKNZefx1+v1uLq6ivl8Hp999lk0Go1YLBaxWq3i1atXcXNzE6VS6aAMLEOPn5/7e/mzkYXlw5j43Xa7jUajEavVqiAby9DfqVaraS02m81BmSLv9XodjUaj8LntdhulUilKpVJUq9VYLpeFtcgv5sjfPAvbqFarSYfL5XLM5/P071wOlhPzrVQqsVwuC7KrVqtpbefzedKDXJY8j99vt9ukk54v62VZ5hdjQ/8iImq1WkGPkd1ut0vz99z5DN+vVCrpcycnJ7HZbKJSqcR8Pk/zrtfrERHRaDRiNBrFV199FZ1OJ96+fVs6NM6jCOz6+jpqtVoMBoNYLpdJcR8eHuLk5CQajUZst9vodrtxf38flUolms1m/PDDD0mQDw8PcXp6WjDI1WoV4/E4IiLm83k0m800Se5ZqVSi0WgkQ5nNZtHr9aLdbsd6vY5Wq5UE9/DwkJQUQ1sulzGfz9Ni7Ha7aDQaUavVCkblP6vVKlarVTIi7rdarZLwcbT8GyOKeHQMLOp6vY7tdhuz2SyNLXdeOPXlcvmBwaFA/Ix78/9KpRLr9To2m00sl8v0bxuQgwbPr9VqMZlMUuAplUqxXC7TfXCWzJnPIB8Uularpc9zb/7d7XbTzwh0rDv3m8/nybhQeORgJ7xer9Pcy+VyVCqVqFaryeGs1+tYLBbp/81mMyIiOReeHRHR6XQKwSe/Z+54HdjsMHAw6BTz3mw2ae0jHg0WXbGjsONGR2u1WloDjNjj4RmMC/3CqfB7gvt8Pk/PQO58zvq42+1ivV5Hu91O9+UZPLtSqRQCltd8sVgUAMjJyUkhUDNG1gOHxrw3m03yJScnJ1EqlVIwZizHWr2OIrAnT57EbDaL09PTiNgjj5OTk6jVaknpZrNZdLvdWCwWsV6v4+nTpwXlwOMiUJxMo9GI3W6XIldu+CC5er0e1Wo1FotFzOfzKJVK8fDwkAyLReRCaVEE7ovD47OOtEQyDAwjQnEdRfg39wEN5fdcr9dRq9UKzgclNILEOa7X64KRoyT8O3dGKKDRAs9kLL6QVbfbjXfv3kWn04mIR8MejUZJaXFuyCtfN6Inn0O5+T9oIiIKxsjnHJ35PMrP2tl5YozIiyDBevAzDBcnXq/XE6KKeHQWlidjwykRTEDRNiIcTI5mQRGsD/q63W5ToOf+7Xb7A0TPd1erVRoPF5kCMuNvZxm+H3KuVqsJxVqmeaaC/tbr9ZQN8Xnm4PkZ+TMGbDh3Ng4cyIZ1rtfryZE6MDA/1sy29rHrKAL77rvvCt7XghgOh2lyy+UyISa8Oc5gt9vFdDqN+Xye0FylUolWqxWLxSIJgwnhzNbrdSyXy3TP7XYb0+m0AFF3u11st9s4OTlJivzw8FCA64zDaaeVjUW0oCxcxs3nQBTb7TY5D1AaPzfqqFar8fDwELPZrADfcZA4HAx8s9mk+/p+oCLGRASz8vB/K3Q+H1JF5gJ6c8qNQoHO8shtA3AU5v8eT57qg94i9mjb42VN+Rw6FLFHuJaPEU61Wo1Wq5VQnZESaJRxcC/kb+SF8WDQjDs3NtawVqulgO45oxfoKAiNoG4DzxEKThi95D52ZLVaLTkcnDx6sVgsUhZip+JgQEaF3XgNcIwg7dz5YbdGx8vlMn2HZwB6HFgt19xBczUajQKq/9h11IG9fPkyKSiLO5/PE+eC4BEGURkjQRGq1Wq02+2Yz+fRbrdjs9nEw8NDAR57IigmSlgqlaLRaCR0QHTjMygGisLi8js4OxtRRBQcc71ePwhZcY6G01YgFpSx+A+fr9frScHzSOjnMs92u53GRfpoR+nUsFarRaVSSYaPw7SB8GyUm7FzfzsJO+3VahWNRiNarVbBQDECfwdnbSOzwiInKAiM06k+v8cp5fK30aGPjAe9eXh4SP/P55qn8kbLoBZ0xugIpGMkakfMPOyEDiGfHLUZDZKKogcYOXLnWUa3rHO1Wo1utxvlcjlarVZ6LrIDxfI55m3n0Ol0UuBFb7l3vV5P/ybo4NDRM/42lYMtgeSRMwEZeTj4WV+RPTI6dB11YG/evIlarRaNRiPm83mcnJzE6elp4q3Ozs6SU2Iw9Xo9KXi73Y5Wq5UMkogRsY84GAkOsd/vF1Ko1WoV8/k8xuNxNJvNNHkTkygAThTEAFKxEEgNEDhKaxK0VqulKAQqcTqWKySGBYJkQSMeI6HTUBQaKE10w6A2m00KACcnJwklRUSB7DRxbUObzWZRqVQKhCv3AIFst9vEG5mbstOI2HM4niepzm63K/Afdmo4JZPa5okYX6VSiXa7XYjGBBin+XaK5quQMUZFgDP6Mvo7VBxCrnBpq9UqWq1WGiufh2w2yjH3xIW8yB7yORiBcw/mYiSKDI3oCVjIyk7Na8j81ut10u9Wq5XWHV0llTNf5nkjG/TV9st6OMUGhdkOc+djbhwHZxvgM6SRecEsv446sH6/n6IlOTUOZDabxf39fVIYHmxhzGazuL29TYaJcFC83DBQvHq9nlJHnAzIDgI7oljhwsFaYNzHysx34Tem02laECIYv0cZneJyWdhEu4g96sh5K6c9PM/Q3JUcvo9M4EKYD4jOBoJ8gfSWD5HYn+f/u90uISzIUyNLc1eMC/TR6XSS8yJFIn0wL2dZmtdZLBYF+TuQ5NVLp7ncHxkZseEQUHqQqauoOAPWD1QPXeJn8izSTsuJtAibID1sNBoFcj8n3svlcpIp5LntAX1iXNgY8oFewZ48ZtabFAzemPsyT+sbsgZocF/WgcCwWq0SReM0G5rhUADCuTE3dND24sojz3awPXYddWCOJrVaLabTaYFsxEin02nsdrvo9/vRbDaTE1iv19FsNmM8HqeFnkwmyfjMEYF6zKe4mmVOyQIxnF+tVim6WJheCC4UE3hsZGLilwjTarWSAi2Xy2g2m+m+REvDXhSEyJRHEiMxpwh2ACha3h7B+ECTjBWjAKkyT3gJFArj58JpYtzj8TjdEw6Tf5vEZVyMAYdC8YXPkE7k1S3kz7MZh9MmVwP9Xf7PHO20jQgImMi+UqkkhwA3ChphLcx32WmCJLk3RnhychLNZjNqtVoMh8MkB/TQxs447IAwXObCM0Fevg+2APXAmGzslUolIX8jdKeO5u/MO9br9Q+AAJfHYSTotWUNQMpkFE4nnSI6JZ5Op2lNms1mIe3/2HW0Cvn8+fNk0LVaLd2UqICR4J2n02kBZVQqlRiPx1Gv12OxWCTheNCuFjWbzZhOp0kxRqPRfqCKyE4DicDk+BFR6JeKOJzy8VkWmUhsfsqLRhrJYpnzwAj8HVI4R1X6jZxOECVz5MOF8mFci8UiBQiUr16vp9Qd0tUpNsbEWHHyTq0xMuQC4kah+JnRLkUYvp/zPcjWqClin0bklT8XVJxCHuLXGL8dOM4dp+hnImOnqszLbRX52rB2OGiCq8eDLkZE4qDQE8ZpXg00Zzvxc/i+K/d2Xu5jdGrN/W1fyNV0AvdGT3On6cKdA6bHiA65EGZ9sx9gvVzZNFeW63rEY/bm+X3sOvrbr7/+OiqVSpyfn6cWCS54MC6jLiYKDAZOu+eFiLtcLqPf76fo5ChNtTKvmhmtRERaEPq8csTGovjZKGtuuF78iH3LgFsdWEDP1yllEu7/p7iQ8yifjdAoZbVapchJUyAXcvFzgfTr9brg7Pke83MlitTHBsRnnYbivEAHjJ2/jTCYq9fVMjbHAYLHiHme52ijZcz+mauYJou513a7TdwPOpU33nreNlbmSzAwj4oDt3MFBeWpDhU/c4tc3JvPTafTJB8yE+63XC7THz+byzQEOsRaOFPhZ3Zodmx83+Q+a8VzPS4HRV92/MyPNVgulwl0sNamnJAJVftc5w9dRxEYUWSxWMTp6WlCO0zIEZpWCpfsc04H4nY6nSbB1Ov1GAwGCZ1g9IvFIkFjk9J4dv6NchN5DWtRTtAJ/A3RPye7HcFcngeS55eVHyO0oRC14QyRjxtjIW/Nl2CkTm0N3SP2qKHdbqfqkB1iHkhQJNpYmLOViXt4/Rmv0yIjAyMu903xGTuqiEjIje+ydozBKRoXTh05kZKgIzyLQFOtVgvpdd57ZcRh+UbsnT+Zgg2YMeP8vavD6JB5MC6ej02QkbA2dqBG+tanPE1j3B5f3kJDb5fHxJzcx2cnZsoChN9oNKLZbCaqJ6+Gu6UHh8O/QbmsB9V6LnhFp7oRj4Do7u6uoOuHrqMI7NWrVzEej2M4HBZgIdwFBgOJ6Uja7XZjvV5Hv98vLO5sNovLy8tC2wGLu1qtUhrZ7Xbj5OQkkYYoLdULd3CjcCiAldKob7fbpbE6KrOA8BI0vjoN9f8dZbiHOQXLKe9ByvksKy9GBLIgGBCBHfkPcRTIBCN0asT/jXJw4hih21GYm1NIdAAZMnaiqBXWcuNZcJxWYPMldoJUMRn3ZrOJ4XD4AZmP7DFIZOZSfalUislkkr6DfPkucmo2mx/oD4UO6wCB1QjHa4O+mWw38uZCH9yblyM8HAnoj90P6Brj4ZnYgDlZrvV63xhMqus1IqXE2WALDmLb7TZxopVKJSaTSVpHbMyIFH3xWAlCIE4C2ng8Tj8jo/CWrUPXUQf23//933F2dhYXFxcxHo8LvVmkZ65CQuCaRKRBjgVfLpcJcSHAyWSS4OR4PI5WqxXX19cR8eiJvfju9j7EFcHN4WRwBKSR7rFizCZ0HYFJtXB4Rmi5I8sbHUEBVh6UC+dvzgMZrVarmM1mSZauAjHOQ+nKZrOJ6XRa4IxQPtYLCI9zNQeHornYYaQTsW8uJGqa2+JzGD3Pw/jdbMxcjRQPcTo4VlIfZGCeCkeBYeDASINx2lAcrA9jjYik1yAlHDI8IONuNpsFHgt5kVlERMEWGAc6YyTNWtpxee2MxEgz+T/PYl3cWhKxb1cyl5cjRGcU5oNxbk6l/TlSc9/DgYz/M5bZbFaojKMP5rQte4KcC3jHrh9FYO/fv0+bfnkwyKJW2+9xIkKUSqUUURAI13w+j06nkyAmC4jxUAmibQLvzj28qAjNXdH87SZCBIPQp9NpWmCnZFT8HF3zlKZcLm7r4QLKO+XkMl+DI2U83MdR3cbBvV0UaDabBYKYiNVsNqPT6RQQBmjMZepms5kaY0GUVFShAnJ0ikMyckMZc77KXBqNmFZW/saoeRapx3a7b1YlEM1msxREiMgggohI9IENCF2Ac/HzncrlfXsR++Zi5gHqAdn4qtVqqafOgSgvRLCLxDrD+rjR03sEkT96cSiN9DOcziMHo0B0yQUe7onDsvMEyTKOiEeHxLMjHgEGsuEeBA2cN87ZRQPW2e0U3kvJARDH0FfEjziwq6ur6PV6sVqtYjgcRsS+/DuZTFKPGJG2Wq3GaDSKbrebFoAOYQu60WhEqVRKnt6l7X6/n5wMDbIorI2BezFBoiHC55k5Z+QUx1HPFUX2d0bs00l/lohrzoIFsiKadyHlYgsVDXx59QgngRNG0ani0teDHPkem8YxeBAGToQx8PO8X4/xm2w1OW5DYE5cGKI3pXvNuYzAjIBwzjyLVAUagjUnwBBU7Cgj9nwf36fqzXPthHm2kZkREvfk3zhUowkoDuSEI6TQQZ8cduBqI/KBGwZ9WKagMXQul71Rv50j4+D3rq7jnL1W1mPkh45CAfneRkWmXRiH19+y9NgBD4PB4ING6nK5nA5tOJRx+DrqwM7OzmK5XKYuXrzkfD5PEC/ikUimhwMeodvtpmoCUL7VasXDw0NSBhyQIaj3/TmXtqNCAOZzut1uodoYsc+/Scdyghrj5jmgSiKXU5VckVAYb3MyckI++cJgrERUkAdR2xVRvkMKlCMbp7bNZrOgKNwPZwXSoRXGkTxPe1B8xkkqaSNxmoIhmrh3wALduHjgi/uD7pgz6IQgwucwUOaC8wSF4QztsJzy5z9HD41iuMgskLv7k/w9I3YQk5GGLzt50lQCNM9oNpupIZZA7JYHgq5Pb+HiuS5SUYlG38gYXMVEf3kmFAQ0kNE39ofONZvNQrBzQIQi8tyd7rqBFRsmc3CT8qHrqAO7vb2NXq+X0hUTt4bYTMA8kXuEECRREcOLiFR9dO8JTiTfa8dl/ogxPDw8FDaWOpKan3AEYPHcg0VENRfE54n+vnCSfh4oiN/bmOEEMNLZbFZoXjVkd+pFA7B5NjhFkCcpN0bmJloUkjEZDSETHLFhO+O0wzKR7+/lAYLU0w3RbNEyB8S6kHYwHhwo4/CG/EqlkqpsLpycnJwU+sCcGhphRexbEBykrGOH0k+QMI4jR3VGg274NQrMkbqRFM8ARfM7xkJwZ22RAX/cIO7KNrxlRLFS7sDO2Nx4ut1uo9PpFO5rR8+a+gQJ246dpYslrkz7GCWn5Hm6fug66sCePn0at7e3cXd3F6enp8m4WUAWixSEMit9LZVKJSk3itBsNtPRIiAuDAKkh3DOz88L6ASB2zgMj/Pob16M1IvfY/iMHeUg6jhtRMiu0Pn+jnREXsZkREXaZ47CPFK5XE5FC8vMz7STJ+WcTCbp36wHzySiRuxTsIh9oYHnMgdQD2gMWXpLionrcrmcqtSslcfrbSw4TgczH5eTV+pYY77PxZgwJvcDWrY5d2QU02g04vT0NKEQ7uPP2/mBSkB7i8Ui8Zp5iuO0DuLffZQ4RzuUHPV7LubD8qOeGJOLRH6Osxn0hmDCOFizWq2WdpwQyBqNRkwmk0LKz72928SICX0DSRml73a7FLSRlVsucO74F2zyY1fpWI/F3/3d3+1+9atfxVdffZUcEV3T7gmhx8kGEVEsA0dEGjipGigDKA5SsIJaMUkTECRR1ydh4HQcGa1YVJXyvhOPm7H7b+6VozjPxc/j3z6yJL/foefY2Vmx3D/G2BuNRozH42QAbr3wnBz1UbK3b9/GfD6Pp0+fpnXd7Xbx6tWrmEwmafuX19AEuhsx83l7zSOKp4163sjTPVs4Gm+C597mvaw7PNdpov/NHPIepFxOEXtOx8bEhf4RpI3c8zk5+BGgzb+iu/4sPzOdgA4x9hzl+rJTY944WVdTbSO2Fd8DPfMaIycKKA7AjIUgzty8n5g+sIh9YHWrj2XRbDZjNpvF73//+6jX63F9ff0/P5H15cuX6SwvJtbtdmM8Hsf5+XkatM912m636TwwEAVVR2Cpy7SNRiO63W6aWK/XO+gMDYutpDiQXBHsGIzgHImt4DlB7fSI7zulPQTHDeFtjHaiJk/9PcbG/ZCPx+6UBEWaz+cpjbDjN8/Fv/m5e7FMFkfsj4bBKTNHEJObWvm+52A+xoR2/rOIfarr9Ip7REShBSf/rlE265HvNfUeQzcmmzR2ZS9PY/l5nmISLPOtPozLvI7Xlmf7Zx6HUTH6mbcdcE8+7+Dgi++6kdyf8/3gnshQttttYZO9HR/3IBjaYZrQBz05w7DDo6GZ9ev1egX+jXsfAhqFeX70N/F4nA7leYjAh4eHaDabiW/B+EkJIx65M5rSKIG7cjSdTtO+ydlsFtfX1wkCTyaTAhJjsXOPbuUwx3ToMp9EOsnP+Z1PTTV/Qfmb6IGBMn5HsYj9QXKOdN5Dap6OORBlGZ8V0/1ujJVeJZTC24IwIKNRrxPjyJWCaqbnyfj4A9ImHcidMM4AFGinYiVmrJTKufLeI86u8lpzufxOOu0WkxxRmLvBmWF8XnePx3ykMwPk53Sce2CYtBUQ3Nl2Z2cAlXGo/YP0Lnf+DkqMgYCL7iBzV8FNBeRpuVNnbA75glzdAUCQRkdbrVZqNeGopRwpem7ICjS7Wq3i7u4uHbPj9o//1XE6FowjA44HRWBwCMHGMZvNCqkOzsJw2CTsZrNJHA5H8LIAh1Ijvgu6w0jNvZTL5VQW5jK/YRK+XC6e404vCwuxXC7ThnO+B2rB2LzRfblcpv2Khu05SZwf3cPl7moUl61Y5t6YI45mtdqf3U50NDfh0wp4Jm0aXnNkCdoldYCHjNgjcOblVNGVs4Li6f/MCzlZ2W0o3pfJGiIPHLRPv/X7CFarVepBdCc+DsEFEhsY46NlhTXmmUbzoKLcsbEmNl4T9w6ufN/onc+5pcc8pAn0vKrvrMTP8t5KttnZ0fNd92vyPQco5IVMaZEy/0zAQx8d+Bkzgc8nYThIfOw66sBevHjxQaUJ7otWCAbJxUQh+kFdFxcXsVgsotfrJWNwAx9GWCqVotvtRrVaLRy65+qn822jAPqsqtVqgcPYbh/bLHCc/J5/s5+Qz7rRMwlKykYqBcphAZwGEcHyVM1bPMyfeC6uoqF8riDmp2aw6N1ut4B6chQGb0NliOjmdAOHCMpzNEYxMUQqrUYglpsVP0+bqRhGFDu4jdhwaKQidkhumcERY/heD9Zqs9nEZDJJzt3ku52g5Zf38WEHpjO8duiC02/vfPDJH+6U52c4Q7cZ5ZxVHghyfcEBkYa5nSRHpcgnR53YgFG1K5UugBnZVSqPp884rcVeDvGNlqH1i/GAJn/yVqL3798nQ8f5tFqtGA6HyVGMx+N48uRJDAaDFMloZF0sFunFEexFu7u7SxNzeX82m6UUwOgI4ZkYBLk4DWBhUEoEb96ESgqIhn+7wuKKFPeMKPa2ROybL3kOn7GC+Myn9XpdKISgLB/L77mHv+v52aEjQ9BTRBT2oDJGOwhvB2ItkLvPuWKcyJTiAdGTcThtMArBSdp5OQ1ibd1ywZhBNjhbxs18WA8qVkbj/Dyi2Obh3jXG1Gw208GCvEwGrhAZErBcJAEx0QNlJ8E6s9ZG0Pzf63+o55F75W0WrIm3A5kQ9/ec6qPXTpX5LLICPNhx2pFa332AgflpZIrtgfa5r7Mk+t5Ae+7zRB+OXUcd2JMnT2I8HsdkMknlVDz7arWKXq8XZ2dnMRwOC2nTYDBIUR5n4z2TOERXZdrtdmFTK0J1pYazrPIKl4/1ySEnSp0TwlY0R3BvZbDD4js5Me7UCUNlsd03xZxNUPJ7nmv04gZGNryCaI0IjGSQS6lUSq0sVjwqvXzPTYl83ymbU2fm7GKGnV6pVEr8Xu6UXTCJiIKxs56W3SG6IJc338WIMBan21z0btkRcoESQU5U7XIU7tM4WE9ONXEV1fqAwyeDcFrKv12g8Vr5fvBEvtAd1hwExOXKpfUA1MS87aDL5ce9y+gDDskFLmTG2KAjcidHvyM6bCDgI45wxOg0fgRf8b9yYN999116ywt5csQe+vFSgfV6nTw3F8pMGogBWrl2u12hAZaufy6iHRM2sY3RViqVlGrQsmDejOjgrSEoDf08eHpzDkRb0jMQBpHckBpnBR/hi14gjB6FthMGzRGVjHh4Bq0iOQkL8cnlaGguMGLfF8SYzXmYJ/FeQKcbGAjyQF5GpcwZdA0Ssbzc9W7+lLExB+5PdCalcDU2Yp/2YKzoKbwrDb/cx6Q1cuEgAaMa0xWsX6PRKFTsqtV98zFcGf9G73BCzInLwS03chwa93NGYCTHGXkO3NvtNlX/8855gwj01cEFO3ABwYcc8F3G6pQf9EwaSMuVCwuWAQi2VqvF+fl52k2DLTHeYzzYUQf2/Pnz5AyI3obiwMN2u51SyNVqlU58gDitVB73hZEm5uc5OR93kxuL5P1ikH7mMSqVxy0yKKtTI8bB521ApFE4P0d9lIbPk+bAq6EAKGLOX/liTkYFH2uApDwPEc34nULjmJGf00icvFPFnNvz9hwrIP/21h7k4DFSUeblxMwL1MfnKcJEFDlAN/NS4WUtarXaBzJk3fljXs1pJ5cRu1GTix2W/6H0j7/ReRdT2u12Wg8HYGSAPFk7ZMaYCcjmrCi6MA7rrAOmv4fN5PJijibB7QQO8ZHWfXPPeVrL511ZR6d4BrZk1I6d2UmztvCTVKWRB/7lJ7dRDIfD1KfFTUzymvNhkQ2ZI4q5uSMVykjPEZHFQsKgcwjN4rkkfWiTcsR+yw0I0txSjoIQHsppgTuyMAanvKSg7j1i4cntMSqewYKDYlzdMQ/hcr7TPBQEdEmFzemr1wLFSItf3p8AAZLISWjmZkTm3yEz1ipHfUY6EfsDDanqIbeISDsK/Fo+7uc0iH97fcwPMrfJZFKoCLpVxqf3ohd+OzXtDbPZLI3LqTS6x3l1jCXfD8ozWH+CkeVj3sd2gH7ahpzBuCqac7b8zfocao1hParV/RvNQT2Mn5/nhQ3aiMicnPKax+SeBIn1ep129OSOKadfkPFPJvF/8YtfxGg0infv3iXnNBqN4uTkJKbTaUJaGDYLsVgs0tnt9FHNZrPodDqFtAhuwo7BJCCL5BYIw+d88uYciLgIlZMBnKY4OufKwxEzVjxzBrQWsKDIJ99ywZhwKKSA+XnyJkHz6hYOyumiORsiN3vo+HfEHn1ygWAJEJ4zxsd62MFZrjiqfD/gZDIptJ0gL1IF5or83LhpBGj5eYO60wobGn9Il9AP3rbkVI7nMEH0x8IAACAASURBVCacWUQkZ8SpE6yZCx44WHq8QBCgIDeDkvYaMTE3DN68Dxv2HUSxJ+uZiwhGs8jWAdCkvR2tq+EPDw+F3kgQYrlcjvv7+5SSoleeC/f2JnwuxknHAbp86PRi9NvBFLv/yQ7sD3/4Qzx58iQuLy/j5uYmRXhXA4Hp5Pnu4KXS0mq1otvtFjis0WiUoLjRBEpGQ6hT2LyvhwVwCmiEYGLdkcqeHijsqIRygypweE5tT05OUs7OfSP2jo97wVEYLRy6OE0iV8bcSYBYnEqxJvCQLhQ4pbZMWL9qdd/Pxb0dIPg+vBaOAgRq9N3pdFIXNvfFWeQkt9M+jNJprdE16GE6naajjiaTScH50UwdUXx5qlG/m4VBlHwWROisws6Nz4MMhsNhSqHd5IncvZfPa2RESHBzzx3zdioJgnfwNvLPHSMcXU78W8/RNTIU7IY+LvY0evuVgx2Iyk6Hz7HWOVJkPtgc9u2DAYza8qLRoevomfhPnz6N+/v7mEwm6bA8Jo3xLpfLgnPi/C826XK+FoPCGTQajQRD7VQcVVgYJkW5FcE4AvBZnAvjcZrIZZ7GfW0myFlYKlT5IoLeUFh6ooyQyuVygQ9EBtzTvWS8vclONB+/USZOhM/BH/I7H5OCXByFiWw4GO6LIiKPpCjVasF5OUqbp/D68PP8d/474pEj7Ha7KVKzRtaJiD1CrNX2B2my/lQP+ZmLPNvttnAoogOFdYJUyPdjPUFJUA+eU0Rxl4T5WRPWnjNBwSmldYvf49wZs/dj+rnMgUDDxXhAdegla4tMyEKM3HFijInnsb7QP0b1XhP3edpGjbx3u/2hhjkPhz38ZARGdCAy8MDcK65Wq7TFKKJYXrfH5j6bzabwIgdKuiwAjsSoB9jripvzfO6H54YMxuma/GfMJscj9mfRRxSNJWL/9maehVPL08WIfUQ2YnIlzOmA50oJGwW2E0FxfPZYRPFt3yg9MuUzfIc1pLLaarXSvDAMHBMtH1Zey9iVXFJZOxvWCx3I/1gH8oKBAxprZEfAM0jv1+t1ejep0zcMvlwupyIPa+/n0SXuog7n7/O86XSajNSVPVdqQTAEN+TPZ43UfPy30y8jTheYxuNxQoDMAb0wR2rngPPG8aLnbNwHLSMH7NppKJwVc2BOtM3gD5iDm6yRDYHXXBlOmbP5GTdzceZz7DqKwFBwFM6TheAH7YC0XNHg9NGcaDyEihCwOQ9zQUROUidKtCymT6mw0QHJ7YidGuRRHv4DJ5IrIj8jQueIwfwScyCCeG7ufEdZD8mFexv1ReydpY8CppxOVQ8E5Fe1gQDNtSAHHLPTZyJxXqWlN41Ii6OlYAKHxPq4Cst88qjNFi0HLZfmuS/I3pwkz8I4XcwAPXFfPxvUys8JMqS/BNzVapV6IZ1S8xkCGu97wPhxFDR2cpkrcm8Y8sOhclHtRIfRKfaLsrY4DAIwMjOl4QBoFGyqwXaBQ7XugTbZ9occXMBhvPgG0nh8CTrnKjEHoaJvh/jugm0c+yULaQPOy6MIE4PGQDAc9uCxYOPx+IP8PPe08/k8hsPhB+dP2RGaNAUBMF7GdXKyf1ckXc9ETaMKQ3sWm3O5SFuJCChKXpnL0VFejUSeyAYSnvk74oI46CHjuzgB3gJtB+FeIHqd3FPnOUbsESaku+dnfooWEiPuPN0GpfAMnAxjcnAArTuIoFcvX74sBA22ePHMnCJAJ0BcXKVSKR02sF6v09FBjMH7+Rg/usA42JaFsZ2fn0e9Xk9b0pBZtfrY0DqdTgt9kRGRTmXp9/up1Wg0GqV1BOnwXDsMV6lBQ+iLtyRRMEM2plIYu9eGdUavKEA4LTSX7DYR5out+kBJH2IA+kUO+bYo+Ow8ja9WqymgoF8GJoeuowjs22+/jb/927+NarUanU4n8V23t7fx5MmTmM1miaQfj8epOgVJF/F4LDWDRVibzSYdL00krNfrhR4xFozIbgLfipxDaldsgM7uqgZ15N7dqZ7huVFbuVxO80SRGDvKlH8uj3yH9nuh0EZKoAQ7ioji0TLMl+1ajor39/cFxTRKJQDl5D6pUS4D0AORFqfN85gbUZRxuoKE495sNum47bzKhMGyZe3169cFfqXf7xd4MmToQGe+1HKERPf2KH+ewNtqteL29jZGo1GqXOOkJpNJcua1Wi21V6zX6+h0OtFsNpPzIbDizOv1elxeXkaz2Yz/+I//iHa7nZxDzsmRSlWr1fQ+xkqlknjWiD1Zb76O7AQ0hE7WarV0D2yAJl/zfKBmdBpHb8oE++N5dBjAg7M+Rnk5T8cf7JDvOoMCkOQZUH4ddWCvXr2KN2/epJ6YanW/mRtldnm83W7HcDj8oM+FQRF12fDJ7yqVSnKCCDlif7IozyaFZFMyikjUR0gIjDy90WjEw8NDAT7zh1TIaRIGDAGdoz9DYBCqnZ17f2h9wJnB+6AETrlAEigiz8GJEDAoRTt1RyEwWuQxn8/Ty1FIKZz2uSGRCweBIqP8OD8UNGKfIoN6ut1uImD5vCuikOnoBGk48oCPssNdrR63rdGSA6pwRZxxc7/lchnNZjMd9UIgtMMkXUSWNrLT09M0//V6nVpEvH8Xw2u321Gv1+P29jYiHvsnQbGTySR+/etfR6fTie+//z5RHycnJ/HFF1/EfD6P77//vpARcMjker1OzZ3oqwOvz8xyRXqxWMR0Oo1ms5n0Bd31/1kT5AICcs8YrzijWoge5NVCnB2VS2dqXNYz855+0xGBhN/9r7YSXV9fx7Nnz6LVaiXoaycBETwejxMyi3jcC0kEmkwmqQzM4PJ3vhEh+FnO9xjJoNigAIjAHKWgkNw7T7m43E2O4wGl2YEArZ3W8B0jQlCjDYtIjjF7UTEgb0ECNTBenFe1Wk2kJ87O/TWG9+YfQCgoqZWOn6OURlEYDsZIMOD/kK9ES+bBXF3IAA3k3IsdPlQAa4TykmaBBFB4HCRzBhW4OMQLUk9PT5M+VKuPJ5C0Wq2o1+vRbrfT/12FNYpwjxw8oVPBSuVxtwkpI5uUeVEz68e4eAkODq3X6yW6hZdg+Ngp7IE1x0EYBGy325TttNvtVOEjHUfm1n8CBJkBTprPuJcTWcAPEhQjiu/WNKeVUw/WC+ucdcK652ccuo4isPPz84SWUMS8fFyrPb4aCacEbxTx2FJAf81kMol+v1+oqmGQdlbmRvJSMc80AnLVwtGJzxNxQDn5ZvC8yoHzg1RnIeFdTAZjQCiI2w5AUlSl+LwrOW4PsOHn82CLDhcHTILSjFLzUyuQo0vzGAGpJ4Y4GAwKXIubMEEuFAbykreRDQbmvizmkcvEFVDGyXehE0BCOQcEUuP3bMGyIwYh+3w2AhRGaB0zj+S/PV/WdjQaJdTNoZzL5TJ6vV58+eWXMRgMUoC7u7uL8/PzuLu7S47zhx9+iO12G4PBIKWYIKhutxvL5TLevXuXUlMcq/sm0SscoavAIK6ISFRP3iyKw3L6hr3sdrsULLgv623U61SeMebkO9yXdxTwc3ogObGGViBA0rEU8igCu7+/T0fjdDqdRBjCF1Wr1RgOh3F6epo4H++AJ3+OeKyW3N/fp9QMjsIkL07DZVtXVyDkIcCJQCAfw3OjN/M1XgjG6Gqne3xQPhPJLK5TMsZJdCF9xRmiYLkhgB5BlcjKB+kxJjfqsrgEDZSI73nPJjIF1oN4vNma+8KlgJYYQ8S+r80RlLnyHBxJxL5nDyVEX0y8R0RCQyg18qZHkM/MZrPEgzI2qq088/7+Pn2ehsnFYpEQOI4ApMSzOGUFZ4V8803yrJErqjhN0Ck0x9dffx3ffPNNrFarePbsWUrFT09Po9frRafTibOzs+h2u/Hs2bP42c9+Fs1mM8kJ1Pby5csolUrx8PAQV1dXqf2CtfGZbeiK1w26wWmkC1NUbU2roKNG2uiKewVtU9wPOwJtud2IIAeSRo6r1arQmIw92uF97DrqwD777LPCK80ZCE2sVLpIVYhMCMpKhCGdnJyk43YgEM0/wAvZ8LgfztNQFcXHgdowiNAWlFPgiEjpKNyO+8TYs+XPg0AwYKIFSuN+OSua95NiwPwew0H5KZYQjZwWumfIzYgR++qgeSXv1yQq8x33A7EPkiBlAha5UMFDJ0zYo+BGhKwbnwXx+dyyiH1PFCkaz3WJnfckmmLAwdDj5XPoIJcj9rwQ/BvGgr50Op0PAgi8LGPASEkzsQlvXwLhwEnxVq0//elPMZ/P4+LiIqWwcLnI5u7uLu7v7xPvBbUBQf7ixYv45S9/mdDr1dVVQT/dRwXyIrAi1/V6nSrBXt98ozZzc0B0hkPQc1EkpwQI4nzWqAqdMcrN74Ef8O8PXUcd2J/+9KdoNBpxfn4eV1dXKafnLC/OvbfxYDCQmygZlcfhcJjeuusTCKzsfB+hI3gT1VQ8mCi5PguMghL9jR5MspvXwynimHEgRnBEFQzUhshlp+O+GIzbVRqUyfwEjiAi0g4IFtgn2jr9zU9UII3Hubg4AvKzQ6VaRoTlQpGpqLGFimeCDAlwbmFhDZEbuoCMXcAwyjXRXK0+vu2dAg9OeLfbpXFAPJvjM5dn8hhEgj65bQN+EtkgF2TorS/cDwTofarVajU+++yzpFu0s3z33Xfx5z//OZ4/fx69Xi/a7XbSzX6/H+VyOZ3Ycn5+Hqenp3F2dlY4RaRer8fZ2Vk8e/Ys8WWkqoyHebp6CNAwIIgobhR3xQ8ddlDmMsobjUZJR9DL9Xqd3mvJOkDwd7vdlOoSFOyocGpuHcnTUV9HOTCIyIeHh7RT3z0lTCJi76Dy84ccoTAAtyv4/B8jCnguDMGtGodK9+YH3PflyotRzyHOAGWlodLbUyL2XA8wnOhm5Afv4ndosjcNefj9l3zPvBBparVaTYURFG0ymSSZuJ0ARGZ+z86Fz8G54ExRLNbL1ULu4QDjSihj5zk+ugfOxTsuWB+cE3yVuUTGYUTmfjYCJ6kISu42G4wYrpCKJD1GPA99IyAxTtYOHoiCCuvIBnp+h4NzgOSMOusJhxl88803SW/n83n86le/SuN4eHiIVquVOMD7+/vodDoxGAxiPB6nqvHp6WlqMSElpppJQHfRCrSE7iAvOycjafO53JM02r2MyNqVZa+hbZN72TZdgEHH0A2KCz95K9GrV6/i+vo6SqVSPHv2LB4eHgpbIICF5gyIggzQJVsW2WQrqRKGZB4L74yzc45NlQUH+vDwkBwjxup/YxAYs6syroCZTwDpWQkYj3tyzB/hFPr9fgG9gWxw1iwWY/Bx15DD5XI5vYKOBYbfgnjFEBgfr8Fzp3jEPqVl0zPfYWwoFfdljKVSKZ1OOx6Pk3wcLc1zuEfMBLIJYebNQY9OM5gj60wEXq1WKTU0SrUzRm8c+SM+PN7YztFozikq62oSmUCB/lD5dfc+jo2UlmKBx8PFeG9ubuLdu3fpjVzsE6YZly552iKQKQ6TSv+zZ8/il7/8ZXzxxRcR8dgNAF2DPfKSWuac0zRcBCW/lTvnBbkANtgJaMxNqHQiYMetVismk0lhf6n1inVBjz92HXVgf/nLX+Lp06exXq/jzZs3hQV1o55PUmy1WslrOirifGgI5D1ww+EwOUCcGhGCkwcM272AQM/dbpeiNIpK71q+dYIKCaVh0hyclaMGwjcKwwjghfxcmhj5P8bntARnnCNY/m3Hi7xxEKBSnCGOx7xFq9VK6QIOAH6L75fL5XRqBs7B3JYrzibqXfklvTcCRW7Myc7AaQJpM6mfq9z+PakrrRsQwf1+P92PN73jXNvtdjo11esVEWkNkA3OZT6fx/39feofM7rACK2foLc8YHqrE0gOvs0bxT2OPHWeTCbptYS1Wi0uLi6SfpbL5Tg7O4uzs7PUqFyr1eLJkyfxhz/8IVqtVgwGg3j//n10u9348ssv4/z8PI2fv1lL7Apdw+k4DacyiC66QMb3zJl5XymOHVvyXuPxeJx8BffjJBt0hwzpGAd2NIX8/PPP4927dykqoDQ4I4z14eGhQNAeKjsjKFIgogaK6pSKnprtdpuUnMXiMximu9oh/8rlcpyfn8fDw0NSHDe/RkTiH3yZ8EcpQWDmtWyg/O1UGkW2gtpp8R2qV6QsTqW4cD6kck5r4ESYMykLVbeI+MBIIh4jOs2pOMxOp5POw3JVlNTDiMttKTyDy602oDPSDuuCtxExL3SIy1tTGM9kMomLi4vo9XoxGo0SomAP3fv376PRaMTp6Wm6DyQ+c0HWdo6VyuMROaRPLiqxJqenp2k9WCO3beDcORnE+o8srEfICnSMzqOz4/E4rq+v4+XLl0kX6f5/8eJFAW3/4he/+AB5LpfLODs7S3oHNUJLEwiLLMrca8SeVsH5sb8WJ4jDhzrwqTOsv52mT0zBgTqoGzR4/Y8hsKMObDqdRr/fLwwI2G6oDqlvQ3IuzP/peMcAQAQsJsq2Wq0+6H1yhYrqGsbhNIwICenLuIyMgKYR+96ViH1q42gUUTz5kufzHb6PcUdEYUw5wnLEYtFNmqPkdpw8E8dlUhrjAr3BodiAUH5HfVehXDBBDgQi1s99Xiiz5+x5uvpkxc6bg/m/+VJHbPr20A/kNJvN4unTp/H111+nVI6Uw/waOyDcCI08/Gyus7OzRHzTrwaaMhrnkALu6abOHKFE7PcXon84VLf+mH/CuHG2t7e3hb455k9qig59//33CWyAKt2fyNjM/QIs0A1vLCeYYT/MgfUBdWGPLo5Zx7gfOgk/il0BLtxH6NTzJ6eQ/X4/tTpE7F/2QMqIknY6nQRxqR74pEj6TuAF8jyaCWGI7sqn0uXTIyDKUSjGkUd1RxP3i/F/kIbHyxj47KH+J/gBowrzAOXyfmM33yedpkSPE3KLBFwNe+5QGC8m0Rlewn1XzJt0CoMg3XZBxfeiSmTS18oH5+Z5mi9hjN4mheOxo3WA82UnCMdnfs1Erp0UDs1pPQgFJ0TK1ul0EifltJKiBJuRaZ9gDLRdODvgtGGcME4Dgt2kNXpD9Q0jb7fbcXp6mjIL1t+pP06MajrIstVqRbPZjOfPn0ez2YyLi4vkyOkxwym6j8pFjjxDgicjiHDlvC22w/1o2WF9yuVyOhOQ/0NX8D2KWAYPH7t+jMQ/+u23b9/GF198kY7V8WZjJokXJfeNeDSm8XicHJsb09jB7q0iQEt+TuQhpYvYp6aQ0CApkBifx0BQJLY5GcGhhPBe/X6/EOHyCorRC9+HXMybG0nxqNj4mBkT2xg5/XNcOEtkQ4Tld3BBNkAXRXBuIF2QDJcJXc6Y4pnIrd1uJ+QREYn0Z/z5SZ0gAfQAx23EdogL89YVnKNTCf547lSo3QcIsqBfiXHhDO/u7mI+n8eTJ0/SfJMBKHBgUPC57BVFB1arx21w0+k0/Z+2IsbDvafTaeJHKWCYT9xuH7vz0X3SMKM9k/buw+r1ejEej1NQpDrP/E1PeEM/usu8jbgj9hvusQOcDjLhO06FQblOq+FUmevd3V3h9GLvlZzP56kdBr/C/L0/+GPXUQf2/PnzuLq6Sk14lPJRhsFgkBwYhkiqCO+B4gNd3RfFRUpnhISg+D5K6S51LwzOjr9dLYootikA21kYoi/PYSFBWkYbNng7B4Sebx1CLu6E32w2qYhhBzebzQr9XZZdXqkjurrlBCeLEwD1kDIgX8b37NmzAtSHP8v5GxyoSfe8UrnZbFLgQo4OBibpTRK7VG6E5vUGQbqyxRojL7eU2CEz9svLyxiPx/Hw8BAvXrxIqMrtOzZ2UBaOCg4YHokgAtp1+mi+DAeHLjEHCkAODASrfr+fCG0jM2/Dy3eboKek/wQSUxysGbZFr5jbJNBb97+xbnCsFFoc0LFB7IAxlUqllB0c4p2RNTK0TTtwf+z60bcSUYWcz+cxGAxSJzqog4fgSR2F2F5BRANddbvdtCmaaI3C23i4r4UFf8DlxTP0dRoSUaxwGE1F7B0NlUvuRWsH9+NewF//7ZQjj3xuMgWV4Mj5OTyDq20YSn7WO8hit9ul37nkzD0pTRsJubs9LxqAFECR5jyMqkgrcnjvk0iQBw7A6SqRFwIe/SI4Wq44W2gGjNBv0KaaBalNDxY9dzjKarUa5+fn8fbt25SugtpB9xizW2Xc1+U1oOjAGqDn2IqzFR88ib553UDP/X4/ta2wxYnOfKr+OHM3HeOMuZdTefSPai0tGdhnv99Pm8AZK+Ol/5NtT6A+0yQAAFeuWU9zbW5LwjH7pdTmoI1Ej13HE9DY72uaTCZxenpa8KBwRUSx4XCYBoIy8H/ug4I4ZXAf0SG4Cyx2pzPKYu6GyEx0MwLk9yy0o+Zq9bgtxY2YRHenncwnIj5wUJTB2byenyfPwnDiAPIwJ2QH5KjnBXWaxZyMzkBmIBFkwLPdmGmob2UBGbiJFcfBeJ12kCr1er24u7srNLniSDFkUAaICQSLPD0v+BXmRKsDDoLxUbBhnUC7jCNHr/1+PyIibm9vPyD4WWe+h76CHnCYHFfjNY6IVF3v9Xqp1YImbHTZLThGfAQQqAc+a94PRAT/iU6aAmBt2G+IrAlobsjdbB6Py768vExyd1sNbTqj0ShRD6BPtko5aFr2EfsTjqmWk7EBFmh7oc8PvTUqO3YddWCnp6fJaXS73QJXQQREGV2Zci7N/jo+G7E//wdYjmfOjYUuY9IkHCGKA79m/sAEKM8gKtmYjEKIuDgSnBzCM8TOlX2xWKTGQ58LjwGg9CZ2USj3wNhx+mfuyuf/jAek4gIHgcPIAsdg5xBRrMbB5+UnYeZI1tU4k9oeP87HaMytA8gOJM/caKaN2O9GYL52bqSMrCtr7UDmwACRDF/Dz9rtdpRKpRiNRkm2zIN7GBUQnAgg8LDIkrF5JwBO2L1soGpQj5tJ0QvWFPmASEFSTv9MjiPvQ4bPWuN0/BJeTkB2es/9SIcrlUq8fv06Go1G2hbo/ZLz+bxwaIH11UGTsbADwqk3c2QXxY85saMp5O9///tk3IPBIDqdTkI29KlUq/vjXowUgPAY+Hw+j+l0muArBCVOC4V1KkQkxGAhlvl5qVRK8N3wGTQUsTcYFC3fqJxvfeLnGI2NKs/9WYzFYlHo7I8ovmqLy8Q34zQ6yNOBiOIZ5ia/GUOn00lbVLjHyclJqnb6tAZ4JkdkyweZIl87F3NaGC9BBefuRk+itR1JtfrhiSBGsmw1w2nYmeUokX+TurgdgN+5p49udu/7Qy69Xq9ARXjrEmhks9kkVEL6dKigACriZ7wCjm1CDorVajUFDQwaPcZhkabhUBqNRtoPyxyRG5V0Aq2LBg5IfosY+gLKQ3dII8vlclxdXcVyuUyct7dH+eQTgqA7EEyVsO5uqMaufXQ96TM9d8euow7sd7/7XeqI//nPfx6j0ajw2jCqU5xISVMmUabZbEa73U6enhaK2WwWw+EwarVa3NzcRLlcTt4c48BQjZosEEdfFAmjoYiAMEx84wC5nO6wEMB0oDCGaqN05Q0DsWMAXTmCM37G4P4cE7T+vJ0Xc8Wg+TkG4gqT+2qQD6SqDdtOyOS87+emXKcpoGt3iuPky+VygYhmLZxG8n/Pl2rgdrtNnBeKz32QHTwOAYd141l5BRjZwtcyTrdljMfjGAwG6f84CPg+kBH35twu1setNfRG8co4B06f2c/lVgT00pV60nqvM/Lg5bTOELBNnAuZC2f8cVF0AFQ8PDzE/f19cr69Xi9lQ9iHe8rgwbAH1gZ9yHvu+L9tza05+Xr/5D6wr776Kl68eBHlcjlev36dlAGviSBrtVoiHBE+6ROdtygPyuQo50qJq2bASAzGFTMbmNMCUAACMxGMcIyEXDFECU18EwU4QZPnEJGdpvGdvBCBEhF1QZ5GaMzLc4yIFJkwaKdm7uViDhiqq1AYEimPHQZKRMCBz8DRcYGaXWHk4l7cG6ft/iGiP+kcMjFKAOlw+U3Z6IDXlZMbzs7O4vT09AOi3ByZdYztbKQ8Rrn8DXnudUE/eT4pMJ8xHWJnQvXWLR7wjFT24cAcbOycTUl4FwQB3rKmMLLb7WI4HEa5XI5er1c4L+3zzz9P7Qvz+Txubm5iPB6nVgzWutPpxGg0SojTugHix9G7qGf+G3txFoH+OAOCJwTlUTg6VLnkOsqBffbZZ/H+/fukwCiT+7QwTFIBohV5b97B7WohToqfOxUgpUSxKWVThbTX94mg9trmZHCK7kHJ+Rp3HJO24hy5XI10ZAFVsDAoGYaGMbmogCyNAPk9z3CXMs51MpkUXrN1aFykgO4Bw2j4ec5Z5V3PjAVnHVE8nSJHVvz+UKS1U86JZx9JTXrmdJF+QtoD8oMXMYi8CAOSIzVCt5iLCxnoLTKFo2JNJ5NJPH/+PI0dwzIvasRt5J2vd7VaLbTb8PYiuGKfS29USwbgthyn90YufGYwGESv1ys0yaIDu92u8LJo7BnC3dVXPkPhgSDpdQKs5HypsyB0kzlxhhvj9lgoYORUjK+jCOz29jYd98pN4LP8MgiUnj1Vo9Eo9QUxIRyDS7Amo3GO8AL+rHkFFMWOwPm1kYUnjqK76sZCWhn5vR0X0RxlBq3YkPPxuMJEhOLfGBfPcCMsXIbJd28fArnhdFwBwtn7u3zHvVTwTKRxKJc3HiNjSHo33NpJ24iIwowN1JpHXRAFDoW2AZCH71+tVtO4GD9r7UIG8jENALLBASFjOtXhXM25USSiOdUcj3XCe35zB03QYj3gZL3FjJTXHNt8Pk/naDFW1ttVX1L2iD23y/O9GR09vri4SOk42Q8bzafTaTrscb1eJ4qHcaHXeYsDetBoNFIxpNlspsMh86zk5OQkpblGk74Xa4Ze88dOMr9+9Ex8JsXmWHJwvLX5KefG5XI5HXCIMP3qJcrL3Ac0tVqtCmjvEArI+7hwchgKC2l0w8UzWRCf+CK1QAAAIABJREFUZ0S0ZmFQMJ7p3+enfaI0TjGstF54FtTEr8lOowKnQwQIxsM8XIny88xJuFufyOqCgpXFzp+UHnQRsUdc3ivq6qJ/T/CqVCqJAHfhxqmSiw1U7vg5yu41hVci5bEewlPiFKh40TuHfvB5EAupTLVajV6vFxGR+r04En06ncbp6WkhJTLqtP64JYJn+dDNnLoA9VNlxDaYH+uEc3f1PC8SuS+Nl5rAg3ltTOg7cwKJ3t3dJbtnVw62fHV1lYAAMjffFREJNRJkkBFtIzyTOThIucP/0HUUgUHgkbYYBWA8OKhms5lOlXQ0MHdluG94CTJy86bf+IsQXLmI2PND5iFyHoTLSMDwm/u55A/iQxkqlUriBpwSsNmYhcPIURLG4BSHBckdBbwCFRlvlgbOYwROh5ENLSlGtP4dhlOtVlNKy7hBKZYZyu6igjlAy5qtNSieFTmieEKHyXtO9KWplfeEMkfzPKSm6BSGzKZg0g0CC6/Ri3hsA+j3+2k/pJ2lUS48lbknWh3Qj2azGZeXlwnREmy5l3uZWFvGjswdfPmeq7+LxeNr0bzRn8DvXQOgR+vQfD5PdACZAfbrObNOPNPZiP+/Wq1iOBwWzttjPag6ViqVdMIsxT32ebrP7+TkJDWwH6JUsLN6vZ4oElMgh67SsX1Gv/vd73ZffvllfPPNN4XqF97TA8BoUFw8Nj9HaZzXRxT5CAoDjvQYKhHIqMM/57MR8QFSwFExVhCLf5YjSS8086C65zYGX8gGp0gkyrkSE5x2sii50ZzlZs7LvIrTUjcb2lgckZEhRs9cjSQi9nwX6Ml7SnN+BwX0kS65DmAAoNPtdn9G1Hq9Tigf4+Q0Wg7KRGZEf29fw+n4jc/WM+RL0OUiTaaqnXO2Rt3m2fjubrdLPYB+obP5PyPGQ3yOkS/66J4tMpBKpVIIXgACy501JZhy1FBExGg0il6vl5yh+U70A9oHB82Z/PV6PVFK2A/PMf3ji7nnPYB5yn1oqx3yvbq6ioiI77///mAp8igCoyeGFMkDdC+JB+e/baBeXCMEdtE7L3a7gvm1iCJEB+nwDBAIkzfEx4Hg+KzkEVGI3BiLK352SigZEQJSlnnVarXCiypAXczf97FDx2HkzbdwhsyfOa/Xjz09OAxvM+HzyMnOhjQAx4+82CjPvHxYY6VSSS9gcZQ3onZFF/lhxEaM5iFBZyAxaIf1el04nBHkSMEobwzl7DeeTbe5X8qLjJ2WR+x5JNaGih5oDJRPKsvz4EFdUDL348AHyuPCAfA34wb9u60GXYFXnM/nMRqNYrFYpP5Kt1kwBjeYcy9fVGCRfX6xdxQ0yIXO2qZAWBQGcj2I2PO/Bi9G6rl/Qa+PtVEc5cDI1YGhnjgKiRd2Vzl/s9j2ruy+j9gjnLwDGMMyEY2yulJkwtoErvmInDB0v5ORFotN64YdnA2Iy07O90HRiCTusSJSYSDMn7lyoiaHPlp2GBCRNSfEcXqMFR7HCARymj452gl4FijYRQNSPyNITgMxie3LyDViX0rPCwLcm/vAUTF2dAl5vXz5MsrlcuG4ZTtvDNV0A9xho9FIXFjEfgsTnKirlnzO72iwHpLGkvJSSXe3POmysxMjaMZhhGz0zgUIwBatU/yeOURE4dSH7XbfS0fLCRdtFbme8wxz1hH7tyNZF0Fy6CMpKNuhXPU3yjS/Z94OeVj/8DEfu44iMN5GZPQwn8/j8vIyRRx3k0dEqlp6iwHKUCqVUl8K96RREMeD8HE6LBItFeaVQCZOAXBU/NukYUTx3YNETBSZnQNEZzgFFidXQJo4WRyjVDgVFh8HYKKaZ282m6R4KAUR206JxSXqU9lhrkaPpBNEzuVymTZqg1Qmk0mqKjsQMSZHbyM1/k+VzqmnK582NJAbn4E/Ya1AfTgfk9EYA++GZL1YPwIVaxWxL4ZQZcPhIgM+S0WP0ykwGDeLUrGFQ2R8rIMDKa9/Q7bQAUbbRtFOR7nYK+ktZzhH1oc+tkrl8QW7cMbYiZHbbDaLTqeT9Gs+n8ft7W2akzlet+BY1rvdLumX38tgPpeAYa4ZcMK4mCvzYDxGgbbxY84r4kc4sL/5m7/Z/eY3v4lvvvkmdrtdAc478ubKDWfA58zXADGZGAvsASNESH2TzEzQCpBHLhbWaZmjPYICSeB8THzaieWLlPM/jM1IMueqzJ2BIthjVq3uX+RqRJJHJuTB/HAe7oInQjNHkIB5HAzf6IyxWtn8vRzNfQx92SitG14z/w3aWa8fD8b0epkPzcdgXi1ij2jQp9lslhyOnVGpVEpv2UJ26F/eiGskna8zso0opsR5JRp7QGbojWXJd8xh8TMjHJ7JuE1vOB027YAuoxfD4TC22238/Oc/T+koDgt+1Pcdj8fx+eefx/X1dUREOsqbixM/zCtbZsgTJEebR94/h04CaghYP/zwQ1QqlZ/Ggf31X/91ROwP4EcJgPUm42x8LJIrJJCuZ2dnaSEcPfMIzkT8soa8goPTMp9C4yf38P39M8hQLm9IdroJimGu5o1YIJ7vPXt2YtvtvuKGgjEfnpE7K6exyIWFxcB4MYnl5c3f3kCPvNyywQWHkc8NhT7kOJya5xHYl1Eu34/Yb+YGsbdarTg7O4vLy8vo9/vpLUEuNrA2HosrYzgo9NOcGEGQtR0Oh8l58sJb2lT8wg6f5eVCCXrFXKrVaiLAT05O0vixG79ExcjK+gYysV46nSSosNa2FXTNXCntD+Xy4ympUAegMbYFOhgTPJCdd19EPBYCeJuUgy3yIw2FenLRhIBg+3QPH880EiMt/dj1oyeyPnny5ANEwMMcge2BEYT7rRAy5+kbvQFj3RjplgEm75/zXYwOpYDP4bNU0Nw06h4slJ1+Lj6Hw0NRnDq6Gx0+AOdmhTtUvSKqsuhUd0EOLFx+MKKdRMT+mG6+Txk+Yo8MvNWD75P2oUw8E6UxJwGisNNibVH+vPPaVUd+ZgRrpAInutk8nijy/v37hJDYRrRcLuPZs2fpuRHFtzrbKTA+1hIOh1QIJ+Y007JkKxV6BbK1TOBImZsLMuaEqExiN/Blq9UqHc1Dip4jeuiJnHzP1wZbIRD+27/9W3zxxRdJxryxCVtAns1mM66urhJ9w8Vz0HmeNx6P037n8Xhc6HtEX/kbZ4TdI0OvlTMZ98c5iINGke/HrqMI7PLyMu7u7pJh5Nt7mEB+xDBbP2jI9FEzDJiUzZUtK6e9stM2UEFE8WWYwF4TmFTLnDJ5R0DEvonUAs7TEyA9bRs0/3kcRGZ3MHO5uTdij3gYmxsJ+QxVTKIh43L/nc9rAuHB18HtGBHhONhTyditYL6nf0egYfzIm/vmHA4/N2LKHTJjRG5s3anX62lz8enpaWrshPfKX8eGDjB/Ir6LQTzXPCStAhjyer2O4XAYk8kkIQqcU76TBLTgwgB0A9kGfCXpPnpPW0NEsWXHXCvfh1IB7Vp3I/aVRKNnZwDIiQZeUmo7aL7j55h3Yz7lcjm9Zg3ODMftIG906nQaJ2XbQf4GFM4+jqGviB9BYK9fv44XL14UzsIH4fgt3KAKFIL9U6Aw0jVHTgwIaJvzAwg234vIiZJMPmJ/GByfN1/kLt80aUFiBMnf5hJQIIQJucrYiHxEHC5XZ7mfUZGdsaOv/83v8ooflTzuiYL5wD3PDyfh1ArC18flGCHlUZLLuyVcWTYKY12MPJ3meW1xxlTIQPsYy+3tbUIAm80m3r59G5PJJDqdTgyHw/jiiy8KG5stz4h9MAOJ8XPmWak8vkoNI18sFnFxcZGiPimsN8MjFyqzfI5g6yo3a0pKCqq5vr5O67TZbNKbg5AVBuxTQNw/54ZU70x4+vRpcvCXl5cpCFH15cRc1t72wIGe6IjfTk4ApU8vD7qMjfnmbSkRe4SFXaGfoGPzlMwfp3aMpz/qwF68eBGlUilubm6i1+slz2sOxZ4eItpbO9zZu91u4/z8PB2l420HNmAbBm0HOFCMgdeHOXUlimAcpCE4Qcbqz/Nzn1ZgYyQCOuVDKfLobgMhShviR0Q6NsgpIgvshY7Yvxcv76nLnQU/g6v0WHCYENrMF7n5WGU/v1wuvh4L+XuLlrdh5f1nPMfjzYl485BweoyNNMZ9WdVqNXGojUYjyef8/Dxms1nae4kDZ11xKKwXOuWeQzv9H374IT0LpBgRqYWCizPGcHLoCHM1ikAHmZt5oHK5nMhxHCvByfsyI6LgOJ1Ogoh4+xG6TwBjxwJzZj8kF9umWCvvZ0R/jaBZ84j9wQVw1k4FvcMB+3UmZ1oEWeDofBz4x66jDuz9+/fx8uXLZPAoMacxwh1wDliv10uDQTg0wjIJzq5ylGTyGIgdjVEU9wDxmKTnuSgQDhDlQqjeKcCzuC+oEdRlWGzUh/FbAc1huJrlZkJHTyMbIzpkjBIzF6difB9EiKKYqzT3ZLLdcwb+82yiL8aeb+MwAZunKSAuBwMQmJGeoyr3IECAQFwZ4/7MHcdASglZ3e1245e//GWiDTjTC341Ys9XGtWDthyk2AO82WzSUermA0EnyA9HihMEzXwM/SM/O/u7u7s4Pz9PaA2ZOQ1Gh+z0GTstEaPRKBHsXi/kzBltVME5EJRtQ25rQq/Ql1arFaPRqAA4VqtV6g/j8FKca+7QHVSsczzfNkgV0tTLoeuoA/vyyy/j9evXUSqV0ivK6a1hQyxGZ6NAMSIiwW/gLxtWWZjz8/P0codarZYE5iNq80oW6MYEP87BEyYCrdfrQhOnS80+jwhDj9hDWhxe7hiMWnLFxKGDRNkE7MoRabOb/SKKaZAJ6nK5nEhNO0tkuV7vX4fGZcfutNrPclRE6fJtJiiUUStIBjLXL4JgvZwG2+hAtQ463mYGOmKsvi9jbLfbMZlM0nsgp9NpKvUPh8N49epVasHhcqAsl8vJoH36CRfHhBMAWYt2u51eIkInOYEMRIYD4MBP1teoCWdEI2pExM3NTdze3sZvfvObwvl6Efu9kG6jYW2gFUjTkSUVTbgrjtWmn4t+OGdOb9++jU6nExcXF6mgQtsE53VxdTqdQpEnotjOwr7evJLNGhK0sBk7NN7MZKR36DrqwOCbIDcRNKQ9h7ExWAbGYEE65g3YyY4iAZtRbpwD0Ys3oWA0hwydpsiI/d42k8zmffr9fmGbRo5++D/K7XQk4hFZktZsNpvCSZQsEL0xKArO3sjFnJYdkheYn9Mdn5fuTYZGREK6DhTePeC0HKMlYCBHp5A2HmTkyxVbpxf5rgD/zhybkUVEcXeDZeFWFv5mbUgpaTZlnQeDQaGxdbfbRafTScaA4YBeXBHl+XnPFagQvaYYAyI8OztL60qlLk+XaZkpl8vJhswvv3r1KkajUUJa/J7ARUB3HyX6bAdveyJgcu12u5Ry9/v9tEZujGXM3j+7Xq8L54HhoIyeADOsjwOf+VN+lm9RsiPLqZ5D19Eq5M3NTZyfnycF8w5/zi3CUWEEzp1ZLOe7EVGYBBHMkR1UErFHQnn64WhpB0K0h5vgvhgHMNfVRjtQjMGNr+7MzlGeXw9Xqz2eTkBhgrTETad5lPccOLaFcYESvEWmXC6nNgH3gDFHHCYOzuvA/DEmHAyKAupB3sw5L2ObmHUKhR743tyL+RpRgeLY08ezcofJvXAefvZms4nxeBzT6TTt8iBN5I3cFxcXcXZ2Fuv1Om5ubuLZs2dxeXmZDhLE2bvJ0nv00FUIcrIN9sGSAYAqXOzAVuic3+126fwtEOL9/X2h4hnxiEDOzs5iPp8nJIfe7Xa76Ha7qe2Do7WRi1Eu3KcRNY6I0zkoclSr1XTP8/Pz1EtHMcPbs1jv4XBYCL4uWiGfarVa0GHW2eDAfX7WuWPOK+JHOvH/6Z/+abdcLuPNmzdp4yoIajqdpmobaAjU4pSBCGEHxs+t6KQuRiR5CoRSE+Xdge7J2qH53qQqGCqKhxHnbQP8fejfng8OtVarpb2F/A4HY+7LcvAYrQieE8gD4/K/LROn5sz3UPrNPJwq4mxRMAcV942Zk2EcOcQ/NEee7bX0yar8n21XH9sryH18f+aB46HRkhQKIh+Hx+ZkgiLFKp5pVE4g87rn8onYB2U+xysISfvp7SKgO81zMHWK661ooDzzoawhgTpvgjXi3m63cXl5GYPBIJrNZrx79y45SKfrpJvn5+cRsacoIvYnyqJb3Afum3PLGJuLDW4GxgHmes6FHaxWq/juu+8iIuLt27cH+ymOppDfffdd/NVf/VUBNpKOucGMVISoQvc8EBQBmU8gB3fbgwl5DIN3/hkdbLf7ipWdVkSk1M2CsRIRMV3qjti/iRlojFM2mnNPWo4OOMcIJchTJByAfxex32CNwjEWHDQKkXfh54GHiOjoS4rCOrmKi2xAjj4LCqWlxO/vI2s7T5Q7Yr91ybsycA5eK6fqXPSyEfGdbpvUzp0I80Um3m8bsSfvoUNwdqTYy+Uyzs7Okn7d3d2lceWEP3pDUAINe08oesO5a/CgzOX+/r5Q9eN+ODccFaiMVprJZBL9fj/JlDFhTybPcfagR/cS8r1KpRJPnjxJOo8OkRJCP3AvV50JZm6PcXM3js6FLfQAZ0bGhv0DhuxQnYYfuo46sOfPn8d2u43RaJR4KyMFhAYyQ2FcvUJQ6/U68UAImdzaKQmLah4CRQKx8ZxDfArlY+7DhRDgiYyK/BkL1YiITuq885zNu4PB4IPvsJguXxMI7DgZP1xXRPGVbZ7feDwuOHI+m1dHc/4qn6dPFDB3sV6vE7HN+uROO2LfBMozmCNrZe4I5eSZXoccfVI1xEjzijCG7R0e3gmB/Bgv0Z45o4sYES0FjLXdbsevf/3rdJTMcDhMxmTnb0To56HXzIXvPXv2LIbDYQGpwf86iPM7AjftSfV6PZ1WwjHOrE2eslsHGSPjIK3zqw2RBWm5q53ujwSRkvY5oCIPc+ERxXPRcurHCNdy5D6MO+defR11YPf39+lkRZPDDJIBMiGXUFFsQ1M8Msd6DIfD9Frz1WqVohKLR8sGhCNVPRBLvqkWodhhsqBsq0CYfAZOCcHlVZOHh4dE/oIgSWFbrVbc3t6m88+53LtGJMd58X/zVjh5O2iiEHs7CRJOI7if0QxOFRmRllClckoGSmE94VOMYBkr97SToiSfb/dCKY2ukLOdAL9H9rxImTm44ZY5umDjFgmcOI4Aw3L6D5o0+jGivbm5iaurq9RLBx3w7Nmz5LCperoB2sEJWfjtWJDydM1fXFzEdvu4oRoOMHc+Xh8HectiMpnExcVFcnSsGUicrIO2CveS0WaCbHjlIYjS6ItMwOPCqfKH442wdxfyGJ95bBw96+wMjGq9q+8fu446sIuLi3Q+FS9PxduaZ8J7e48gzouFZGDb7f5NMvSIoYygAVcraHtgYblfzr0YibllAQMgonNvtmpwT5eoMQSMDkcJGkCB6HUzr8EiMiYQg6umThmoZPpAxIh9lRGFABkyv4h9scHODZSGvLyhOecRI6Kwnjh4NyjiIHDSTnPdksH4vFvDvJFTa1Alc0SGlicHEYIIGL85Hp8CAQJ1tdZ8HwR8jsqNGtwLViqV4vLyMsm5VCrFxcVF4WDD3HmZ67OseE65XE6FGgIGa+b5ErRASKwp98a+IiIFdsbOvZ3FOO2mzcP6TFUyX1OcV552I08COY7e+sS/I4oHg3IdSnXh8Jwt5cg/v446sDdv3sRvfvOb1NmLsM3/sFA+MM6keN6xvlwuU9UDR8N3MR7uR4WLvWOr1SptuwCZIQg+b2IUPsZ7+PKKjCM3iI+Ig3KQekTsN/6ywRXHnfM6RCg4MJTHShcR6R6gFxTT6bX7lnIkReS1Apl/4sXC/j3K7uoewYb0AW7skCLaIaKsrLWVzRyl03qnvqx5nv5iLEZ0DpDm+XBk+TNzrjLnJo3m0FnOdWdcs9ks/vjHP6Yeqbu7u6RDv/3tb9M7J51uORixrk6F+DcoiHVEd16+fJkaSwm4pmlAw/T9GcWwzjg/9GQwGMQXX3wRg8EgFotF4RVxBGBsYLVaxeXlZXKAzWYzneLrvbIEPusWVAi8uVN5924yNxfO+Bw2cYz7Sjp27JfPnz9PaYIdF7wGxDKVR3iUiP32EvgP4KCPbTECitifPolxwcP4eBPGg+CN2FBK7gvcpqPXXeGka9zXCA+FgAuL2Kd6OE3mwSKiDAjeaTTwGbRkhGYS11EbGeX9RDkf5XUBhXGxPi5IVCqVGI/HhX4bywXZ5309OBgXb7icTnExTlfmmKc/gxyMoLhwvO7ZYi1MBtsJ2VDcaGvnh5wh2o0ieQ76gY6wVYi3EUHMl8vlePLkSWy321QQwPmwBiBO/9vz4GKNSflYIxqhaWY1zUCWYroB1N/v91MzKgCE4OWX3HJRsbR9oAc4Lz6Hfd3d3cWLFy8KhS9sgwDEGrMm9JGyLqAudAS+2YH9Y9dRBzYajeJXv/pV6pdhIAgITgPh0XTK+feUkev1elpcR1kTtzgalItSOAKFuDVJmvMcGCDGZiP0a7dQFrgxBIYBgMDMSbXb7bSFxA7FFTlHXSMDUJp5N6dWdkiu6vhkDRcs8gXl+66CYoigF8N4qq15lZLvwAH53Z3c1zLEII26KaWzlq6M5QUO0w7u2bMcIiLtezWy8lE1NnwjXrhbUnSeWa1W075RI2fWDc4FZMvLVwnc8/k8IZ7NZhM3NzeJj2o0GvH8+fMYDAapP819dtAS6Gh+bNJyuYzhcJjGwwt1d7td9Pv9FNj8smQ7c7KW4XAY19fXqWfs8vIyZTmVSiVt9iZo0J8GYl+tVslpE/QI1gThJ0+exHfffZf0w3tnvZbMdb1ep0MgrMeuqAJWOLYod/L5dbQP7J//+Z93y+Uyvv7663j69GlCICaiXVkzOmBw/J/+MfcR8TepBK0IPqqHCXhyefXKqMECMwHIQvN894J5nO5bi4i022AymRTISMbv8fAceIOI+MCB2Vm5ggvK4HPcFxn7uzkKysfu59pRIiPPkTHkRL05i7wtIqJY4bVz8r+Rjx1gfg/zVyC0vDrtOWJwXv+PodNDTvPQZV7S3fggBQcZ9BEDL5fLaRsWRYVarRYXFxdRrT6+7ANnZp1ATy0b5mC95vO0Y6AHFL6oXKM3Lj5RyMrTvOVymU6/dZDzDg1sw1xypVJJQQE06HcqOKgzJ2dE6LLTxnwrn+WwWq3i22+/jUqlEm/evPmf94GRGxOZGbRJdkM8FItesBypmNMBTkbsjwGml8yLDJw1YQo6MYLD4ODQrMCr1Sr1riEsxk2UcSmbMTcajbi+vk5Vq4gotAPYUbOQeaWPNIax20BNmPvzXnTPy04HA+CZtFeYz2BO+SGOKAgpnreLsI52Fu4FswOy43DRg6oayklPEIoP2iLQUKjBQLjQOcZvzsaB91B6aj2ws+ce9AvSA2dkDSLkPszVbSe0IVBwYA242JeJblxcXMSrV69iPB7H/f19IsWtqyBIIw/bAfsSOVEV51kqlRJicXDy71lr+GxXbNkUT7XdPZKVSiW9R4BMymvUarWi2Wymo48IwqxHu91OSBWUx4Xjw3Hl6M10yMeuow7M79gzf2MjN8fhwZlvcApkb48yL5fLgpE4ghvZWZm85YWIGREJvTkl5DIScoXUXcI4zJOTk/QOPY8JBMQ4IvYENgvImBinkQCfsQN3BMsJUVduzd1g/N6Ia8UwJ8cY+AwFBsuCNc6LEcyP8RhJGqk5mjJHNz6iJ+ZZmIvpCa81+sT4vKZu1+H/9DnhHI2Sc4dEgGX+Rm6MJaL4Dk+qtKAyno3uQ5/QMtHpdKJer8erV68SLdHr9aJer8fd3V2SjZ0YOsD4nbqXSqV0MgbVTLIi7s8mcE4+5ufo+ng8Luw4cDbk9hR4yYh9s7XRPJQMJ7w6E2PsDrp2iOa/HNywd+yQ1oxj11EHhvEbhm82m0LKgyBM4jFBFhQlgnPKS/IgE6cGTJRnUwVBoESJQ7CVZ5hjMVIk4mKsRCNHjLu7u0LjLZGJQgLVSVfeIFo5bsikc17a96kRNN+SfviARuTLz+3IkJ0bET2n7XZ/IKKDjquijMVkuZGTN4ZHRGHcKB3zY6y5TuTpNHtF+RnOjxTO8/Pl+0A18H/vD835NNIh5uZmTusRc6Pb3UgbhI2TNFIC9bDf8vT0NM2lVCrF7e1tbDabdDBjuVyOu7u7+NnPfhaXl5dRqezfLLRerwtv2jIXaceOLroIgUyXy2Wcn59Hu92OdrtdOK2Xfi4cB/dFB0ejUaqiQtrDUTtjMhcKuEHHDUAc1OHY8At2uk7RkW1EFALVoesoB/YP//APuxcvXsS///u/R6vVSpHQBsnVbDYLXeJOF0njaIEgogDVIfh5fZMhLGiGaqQ5pYgPkZ4rhR5fzi3kQkGYQGY7JTsfOz9+h6BJvRgXjugQZ2fOiDnmMs3HaAdoBeHf/jvnpHJuyjJzOpkTpoe+x3dzPsqRNn9GRKSGRwh5c0A+wSIngbmfe65M7jot9fz5v/+dFy28O6DRaKT2GPdOISN3oec8FZ9ztoETYqyj0ajwchyOqLq8vEzBZjAYpDO34LhAUi4WeR0YG4jSSJLUMKJ43Lqr9qyjU82I4rsrHZhy1MbxUZY/c8xtje84ELL+PJ/PPDw8xLfffhuNRiNev379P38r0dOnT+Pdu3epJAv/BSLhMDSgqRcUY2QRyZ8RBI6O7zlNgB/A85N2WDAQ+wiAJkjux2KjsAjY3Nlq9bi/jDldXV2lqMNCUA3jO7SLcNHvwjhYZCItlb78915o95nlDgNl4/M2LCOIiOIOf8+B6MvPcCCgXhwAMmXdQHcEBozF7wV0as3a+Go0GomfcTrrC94uovhaMAKYT6rAeTmtNVrLZR2xRwjr9fqDky94BuOv1+sxnU4LTsFO0fMF0fB95MHpqP5/tVrvRPBoAAAgAElEQVSNp0+fRqfTiUajEd1uN0ajUQwGg/jP//zP+Oqrr+KPf/xjfP/993Fzc5MQ0atXrwpVWuTvrT0UyJCPdWu1WsVwOIy3b98mSohewuVymQAFKCiiaIMOkrkTozPADdXWAWcGpMe0mDhboHDCz6j0UnDyfPLrKAL7+7//+92rV6/iz3/+c5TL5cJxxU7zMFg3kHpRnXoxYHfXeyuFy7ou8xuyOt+m8uLobWFzT2Bxzr9EPL68BGI1v9xbZY7I9/fnjBQcjQ2lDyHAfJGcophwBarnVUi+g5zylBUn4ec69eP7vheyQ96sPfP3fPLnRETa7Fwul9OxK0ZTEcXXhuXyMIdlZ5lXNDE4CiOmN3B26KR1Alnmuxmcqhplcn/km1fXcjSObdFK5P2ndsAcggCF0O/3E5fbarXSW7QXi0Xq54qI1LKyWCwSnXFIN4wkXcFGNpZljlhNu1iu6Ah7pHmjVF45PoTIsG3bkVsr8DWj0Si+/vrrqNfr8cMPP/zPq5AvX75MR48AfR1pMES3VxiFuYuc/BzPTveySXQW342eefXRx+igRBy8CP8Euc0GWAQUUXzX4cnJSdzf38f19fUHgmeu5oWc0uYNivACTmmdbtoR2pmASpgXP4dbwkjseEA/PNOtCkYa8I8R+6NVPJb1el0wNkd49xlZEZkXYzYCItC4oABCzNsw3GBrZ+n58n92RyBfEAiOzHoIme/eIgzL/Vbok2WJ3JH1oeCFLJGzix7MxWmcv+fnuQJnQr5er8fNzU1yfnd3d3FychK3t7fR6/Xi888/T2+sQt7OEEy2H1o3ZwugQ9bD9hWxR8MREWdnZ0mHarVa3NzcxHq9jtvb28QpGulz4STzYlZEFDoDkCPywXbyAxfy62gK+fXXX8fZ2VlUq/t9ajgwHgpnhCPyYPzSAH7ulggmbW/c7XYLWyBQQKodLtEzjlarFd1uNyk6OwPYp0XJmef5cDscHgpmI65UKoW9hBybbAU1l2NyE+V0ShZR3IBuxQPe04fjjfGMi/vAFbBXjMuoD8Xg2GO2X3l8vP6OdWQNIP1dDUPOvrxh3FVlnpEXWhhPxOESuauMRkHIBuSQO6mI4lvh3buEHDxWj4OUlN+bH/M+VBue50ibilNefu7KL2tuOSAXAuRms0l63uv1CocNcnLqN998E3/4wx+iUqnEz3/+84S68oIbaAsnhH461d9ut4WtQVASoLJWqxX9fj851sViEdfX1/H69esYDAYFrtlpP7qIs+NneTaC88z1m/X3CTcfu46mkP/yL/+yu7m5ie+//z6Vgpko21HcbMmgvPeOBXe3OxfRGrQE0vKeKVcoybnzEjqKjCK4WdOIECHf3NzE6elpQaCG3nYgFrCNylFus9kUnANzy9stcA4YgBUKFOt03EjA5DHRFD4r30JkhIVDYLsJ88NJe8z5z/P7gDpZZ3roKM6AmniWEVdOePvi/hiQlZw1QB4Yi1tf3CXPDhGjh8lkEmdnZyn9cqWSz7gFKOLjzbs5YjRyN40AZ8i9nZqB0vL3IoB43bxdLpcL7wjNnSkvHeHoK95pabRjJ27d8lz4XLfbTRlSo9GIv/zlLwmI0L3PeqIDvnIZeccOl6vzDiAuEuIwv/3229jtdh9NIX+0Cvmzn/0s/uu//isuLi7Slgx3qmPYwMjpdJqUJZ8Ihm/uxEQ0nAwNshYKf+NcXIb35SZJ7oMT5OhdFgQHSvqUL6h3D3BZAfMufu6Z9zUduuykrWyuqPnezJ3P59VePouc7Ehy8ty8oru4CQ55Vc/7CCOKp+zaoHIEytzd4uDKHCjHvJ3XnPtZBl4HuC8bVb5O/M7pML83P5Y/02tEYORQRDuInBPyOlg2eXHGKR72Y53xesEbsaYcDVWtVlMRqVKpxG9/+9t0vNNut0vvvEQfvLWNijvB//T0NDqdTtzd3UWpVEqHOhLIfEYYMu/1eikbsqNnLRxATRfxfX+GC3/U6/ViOBzGn//856hUfmInPohgPp+nvWP5YqzX63SWULn8+H67wWBQUAoqFY5EEfuKD4oLaU965AiIMhIdjFDMq/io4G63W9jDNRgMUiMgz/H3I4rGWa1WE0GaH7rGq7MMg2307sgHLWEMvo8dn/k/HEtuAHZeGLGdr4lYlNeIxKkTY0Qp3eeTp0BGjhgZqZz5PnOEoEo6v73LgO+BrkFPlomLQSAQjBvny2d3u10yFLcwIEvGXiqVCkETxx0Rhf4q1itvT0BGRnDMk0IVvGlu1CBKuvapYDJWZxak8cyvVCqllgWqmNgnsvnXf/3XNA9SW7r/5/N5PHv2LNEGNzc3UalU0okYdNKzlhQdWq1W2jvrTfO0PvF/xoj8qVQbdaPDbk2yHbhQxdxcdDh0/eiLba+urmK9XqdUxdEBTsj9PBHFA+4YsI1hMpkUNtzSRkBEg/NwXwr3tnE5ArKQJgXZ9kATIuX8iP0Jq0Qmxm1UFREHnRdvYgFxMgejBL+VxQ4HOeCMbLD0umGIyBZ5M3bPOSe9zWU5tYbz8rlrOGnkaMVlzjzLXB4RHcNzusXPKMC42OC00vdn/nkzLM/mPu4jRLYm65EDDo5obrRp9I6jRwfszHiuq4sRxWNvcP55ddJIMc9YuBeycHUP/o5jqSDWQS3m6uBicSy1Wi0dW+22FV72u1wu4+rqKqrVx5anfr+fqA/0g7mxA4Vxudo/n88LlXEQmAMGyA6/4UDi7/JZ/Ad66AILAe5j11GG7I9//GP0+/3UZkC3trkH99UYEVGaLpVKySFxIiVd7Jx9/v9I+7ceu5Ijv/9O1oFk8dAkW2q1NJbGsA3DV77whTEYwHPhAXxhwG/BL9qABXtgW+PWwfKoW+qmWMVTnf4XfD5rf3dws/pxTwJEFXfttVZmZGTEL34RmasT683VRWw8UXmRbvjtW4b6/GfPnu15Dd+h+D1GZq21ZxgdFT0XGoXHs1UhS17fv39/exGKcLpvITJBHRNCHWlq8RdpzBdhzN36JUT7b621vX+zqKDXlHfs3zREs4W71tqoA/LTWsDb7TLGWrSsJklDKkM8LWOoUtdI894Mpzku6jk6Ovpo/DgjiNW9DiHMmXVsyImD80yGGl1BXuU86xAYAiee0Nn+7idjw0CjTB4+/HByhmOoIP6HDx+uH//4x5vjVDgLjJydna3nz5+vtdbeC1GMVSTA2PqbdUwfSgGphQMUoFvypTdNzOD4Li4utvIbyPpT7U4D9q/+1b/alMbplASmHR8f750ywepb3LIqrLmOQRuMEasMtvsc2qlXhkh4KcrfCt8f/ehH6+XLl+vNmzfb1hCe6PT0dDsux6J9+vTpBmMvLy/X69evN8WXyWMAPa9/t2h6iikoT1llbWYWdsrVApqLWGamBLGFcnt7u/e6Mn11zxK6FswsEK7TOTo6+mjDOkW1HUU46DuT67IAGs4yWBwIQ4EUpvj6a4FZtM2EMjhz+5GFYnGYYwR5WxdkE0NFiX2G+9B3zqalFRpnROcaUpb28LtEFgfse/SHvtIbztfrDlUEcDLdHlbEfX19ve3TJH9Z/bnhmpytt/KR5pje0YfHjx9vRwJBmnTg5ORkkxdkx5iRt/Vtbn4wAvv666+37NrLly/3BF0ICEJ28y1l6ndxSrJBFj5ysgRmF7r7TeUgCPzJ06dP15dffrn+9Kc/bSEcD2krFGVGdLoXNPf555/vLcCGSuXutB6a5/41LmRUI3B8fLyNycKvQZvkJqWzINwbGrG4GdamrtsP4zlUid5MbEMgRnqtD29iJk+LrYikBqPHpPDOe4o3khJC/u4smLV5t7e32zEx7rvWBxQFJVkQFmfR+dxfenR0tPFQr1+/3jN8/k6ORZ9QHGfX+5bnND/VlzomKKZGjf44hRgKkWwoF2mhP378eFvsa+0yqp5vLNUDxrSo8vj4eK8ivw2POHnnoi8hprdCcaCoJ8bSeWp06fXr13uRkPHZmH6XAfveM/G9Xdrhb8fHx3vp6JLCjuOFwBoWMSJrrb2tAs3Ute5mZm0YFI3yW0hOAHj37t3eQsNzKHZ1PwLt89da29iEsCaP4ashYmzds8anylKSvSStvjEojGER2DZRuQ6aKYKrk0CaV9HW2lWIN5xrfycxrk2jUmfCUcxFAg2XAJ+y59G1SXo3JJ/z0JDxyZMn2wIQcsiY1zGUMysNwjj0hS6eKyJoQqkhaTOV+lSk0nKTtXa7H6bxns0Z+WTWjDqniofGl/ZdCJMzrg5B8j4rf0VWU/dmZtpY9E9ipGvWHLf0yXcdhAnBCrdLDX2f8VrrexCYKtvyFJeXl3unm3aPo6xiyfXGuLgwoUs3hddDEurMQiiWFIpCXp9//vn6F//iX2wGp0S29HMX9L1797asqdjehOj3hNENBed3D/Vf86aXTvRau7f0aBOVFT30nYMMAq83DZj6nfJ5ntfvtd9k8OjRo70+8coMnPk3xxCEe/nZM+TojdC74e2s6etzy2n6DNqxyBiRjolBkvXE0ay1T/RblPrBEKA8hGDl0nyPEZ9hJoRcpIzANxed37Ozs+3N7q2NAhgmEqphpLMQtbpMkUlrqnxWbvfx48fbi3rq1MimyLwcrTVF1r43gYCz0sxHW9dWd4BM4+v6HxxCOsL27du3e2/OoaAU5NmzZ3ulFsK6o6Oj7QhZHVprP7NHCLwnLyit31olfAgh3Lv34c0xv/nNb9b/+B//Y621y4A0TS6D6jMozcQ1jc2TUta1duFGa5x8bmFWabugcH1CFen2KidOyD3dg0drZpQBxudwMNCtTcRFhxTMwjPOQnahfjNm9YRClW6D2ZQoaLZckmubXaSsWhMGs0Ec5fHevXu3bcBn5LrQen99oWvl+ziM9okcEOl9E5D/I+Xt63Vd10ar2ckFyiATnGh3ADAU/t7wvPVw1a/yd8h8rWU4HCzqpPs0i5rJDQ/NyTDmdRzVITpL3j1LkPzYDc65yY4ias1YfjCJ/8c//nGdnZ2tzz//fM+L1avz3iy5B1t06qW2B6bzvILspL9beK3DITzGdK0PVv7Xv/71un///vr5z3++N/girpubD++9UwPUEKFnGbmesWr21CRLQrj/LJPoYvV70adFeHR0tE3oWvuV6v5GMcmw6KPJhLV2W2NKzLt3+YW5G4LSU8pyFxyB+eume2FRDZljaBoy0guZKX1oKCJRVMPueo5TvxDTx8fH2ykPa328lcjYoJW5jQf32vmGMFAZjx492itcVaXOIPm8svYd4+KEGk0gvmeoTv7Hx8db9T0E7hQUTncCALJVf1d5io4axpHRzIYaB8fecFVSzbw3aQa1di78bFKBo4d2q1/ty8y+f6rd+ddf/OIX6+XLl+sPf/jD3qbWtT4oJoWVJbK4zs7ONiK35x+ZVB1zfHShaBVprf2aMkL5yU9+sl68eLH+8Ic/rBcvXmxvLPb8HgBXY+SZNRad5HorBqQcB6/pO2utPRSH6KUAJgl/N/kx4VX5PIrcMFwhYQ1PjZJrkKRk7/pyNPXqrqvnngak/YHE3r59ux4/frwng6Ojow0VQx91KIjo/v2QY6inFibjgywcaKIZvI6rIYgxF5lNJDgzsb7THQJv3rzZQxXVTToL3R/iC8nN75Nzqs53POYUKpxZOf0spdCXmEDwjRLcU1+urq72nGH7N/mzFvJWl52e2sQSSqSRCRlB9Naf9VEj79k/OIR0gx/96EebsGolGYO+BslE4XuagSopN6F+Pb/n9O8s+C9+8Yv11Vdfrd/85jfr2bNnmzC61QGpT8BQ48z0eZ5QhQK4zlaiOcFVYvdF+JJT66Rc08Xh2Yc2+FI2RqdKX6QxSW/IdL5o1L/uPlCf1qY2b1OOcC1k3O82za2vZNKkS3mSzn/3DV5fX+9lsfFr5ZIa4jTkbihZLpIBWGufsyx353meI+SnNxyWk3qdhb/WzvH2ZJWW3Mi+6U8XIud1fn6+d4ZZax8ZpmYc19rftO9erUWjuzKKDflqKDkMWUyfqX80pzX09Bqy7JhaFMuwQoKojhZeG5v1Rcfv37+/dzjqXe1OA/brX/96vXjxYr18+XK9fft2e5ecB/udgaDkIK7OlNyuAdEa/xP+DFdM4C9/+cv1+eefr5/+9Kd7fX3/fncGOY7EAmHQ/NSEIUIl/I6jeJWG4JrWWhsJ6pkN08oPFv77/8ykTq+31k6p1tonnfV3rY9LUyxUi6nlB57T71Gulju4r+vxOjUa7Zt5JDt9t/CL6JrVqnGZIWiNIJlULvTE4plckUVQI2Zeyvkg+H1uAeF0D2Xu1lrbWCGxm5ubvVftrbUrIeJwheLNkuLTlN10DbkXoyqU0pfSM3SkRoqsTk9P15MnTzaZlq6ovNfaGSZjMnfWUPlYYXt3yXRdK6YtYEBruE6VAJmWp6yxn4b7ULvTgP3zf/7P1x/+8Ietg1988cVa64OlbSp/rR0/wmCYZMfZEKyfBgAtgI8WPQX/7rvv1meffbYeP368fv/7369/8k/+yVpr7WU+1/p4/57JbpwtO0NpC81b//T27YeXeJYngOyEEp5dbqYKb/Kc/21SLJpJsttcWzKzNUaUpRtjeXuKd3p6uleD18wXZa2R1YeGKmutrTbHIYSHPD/qoNyIeWs6v4uy894tOfpqcUyFpWscwDSQRXxFiRIInBRjUBR6ff2hMPfevXtb7ZVFZQ58HzXBCJFfIwjzqR+ymkVJa63trdr0rWU11kWNd+dYUz/JIdV51UA3oulRVdYL2dXI1di1H4wxWTaspluMPIPlvpzl69evt8ipDkIzThX8d/Fg33se2Oeff749xKui1IYd2rBpG1C5L8dRK0J05M5aa/MSJmStXSr8/fv368svv1z/8A//sL766qv1s5/9bMsMzT1wJo2gfaccmglca5ehoThNOuC0KtByFoxGXzTSZ1hsrYGzkD3bdwq3uz2GtysKa/kE79QMZdPbwlPefK1dSt+ug1lnBXlReN4daukJIMfHx5uBE/55wUS3ihWl6ptrZMMsohLrXczlDNtXaNgWKdeRoYUgCSIi6DFLJycn26Gd9rW2lRfl9CanwyjUYPg7WUw0Zf6g3CaFJJu6E6LhcB1KSXlHvK+1v/ugPyXO1LbJjsvyCyFL+7TmkyzRRjWY5W4/++yzPZ65ZRo18OQ60WHlfle78zid//Af/sPt/+9A/b3snQng1XAzPGDJ2ZK1freYKWwLEQkNfP0//+f/rOfPn6/Xr1/vQeK52djP7uOCZuYWj55P1LICi1qI0UxIiV9GWEjK2DCGPX7G4mz6vB7VZPX/7Zc+tP+f+v4Mz8i7xYGzgNg8ddfDPAds1tG138bYxsjrd1FV5WixqgecbZbbNCs15VyZzfPRuuD1q/PvPg2j5gJqISkHNsfVvvnM8/vdRirN2EJo5UD9dN8mVCZ6MYddGwppW3NZx1DjYQyTI57r0s/ajvLLRaMihm5Ud+8i/xYYm5fLy8v193//9+vk5OST54HdicDOz8+3424nhwFtNBz67LPPNq6nStK0rgkDd8/OzjY0VuLzxz/+8fruu++28gcKaXLB0daYdQErIJ2V7T3CpLUuPZGCIrXkoOQnL+L782C3Lp7yBSatymXcPJ9ryLdHEDVkr5GiBC36paz1xMfHx3vZw/IdJWP7shae1jWzeJGTgFq7UDkuBl4/KfhMl8+F1HozZRg1FD1Nd60dArf9DWLT34aDE5W2HouR6iLu6Sk1LGRVZAy5kk+TJ3ahkIs1dHl5uSHWx48fb7xcnw8hFR1pUFhrGk9OTrZxVTdcC1W1uNo9GdieXgK5+V4dEDSsv9bZBA/00rPtwWy/oFJz/YPrwF68eLGdfS0bR2HL/4DR3U/mzScmjWcv7HS/etEvvvhi/eEPf1j/83/+z7XW+mhwJebXWh+l8y02SkwIILN3K7b+ZG4faSglfq9SrrW2UJYRbJxuwsmimc3WabXosN/vRM+9ckKuk5OT9fTp071F/+7duz0erWEqmXWfXAnWwvqS5baQeb656ni7cFoTCLXLrjFglJuTK7ldnkw/3bP9omM+g5iLnLszg9HRNyi+G4jpp7HIQK61tt/71qurq6ttVwJnOWVjnsu7MX76UXTUuZGUOjs723gj48IXNltpHOrXOEmGBx+41s7hM5qzcTA1hvQRb2xuzaMQvC/CrbH3Xkx6zBm0vMQ8l076wRzYt99+u372s5+t29vbjcxv1fZMV+uArKHwQ6vyi8WbPXn+/Pn6+7//+/Xll1+uL774YtufNrfonJycbAQyL3GIM+nAnUjRvx2qnK8XMWEWVb1Ji22LroQox8fH21uaNUYKPOZZi8rIpN6y4yiXN4tGPaP3k1mDQKCYLpjyPodI3MqjnEWfI6EwQzx9cJ0izdbMNXGx1j6BLSvcViOw1u7kVEmMomXPmAmCIhyEsZ0ePrdIG6rhfTlE/WgZAI6NQS5ynsa5W2msn4ZT9vc2qlHqwXkyNGutbaM02ZQsL+9pLoV4PmspBhlOvmoeRd69zRzMPFjg6Ohoe08FxzHXa9chu1Bq5FC704AZ3Onp6ZahIRSCZmWbBrW4Fbc1PLi5udkObHv79sM77c7OztY333yzzs/P109+8pO11n5ZQuP+VpyXS9Aar3cRQlld8H3RA6OgNqqclWdMfsGEd7Genp5u5HQ3AbdvjFshuQmVwamiGx85FInMGqOpbPfu3dvbxkLxW3ndKvzJUdVQVVb9f8PWWXDao470E79ZBZ6hdY1GS1PqRMopkVMXgQXQQtM6MCU+To29urraKxgVWrVBI6UFWlIwT6MoaierGl/lKozG5IfMRyMRxrW1WzhL9++cQoieNwn10hHl5UoFMLbTwNLvlvYwZrPA1vyoDMAfM3KtQWzZy13t7jr9tdvEXIPAu/AKFUjLGkrw46W6OZzgGToks1DQMyChVvVWGBMdlF+YFc74tqaTKYF33K31weDZvsHQWATlnlrfUk9GHh8JPN6mCk5BLIYaZTyDZ1Iwxqxopg167CZqxqOJF166By+SOThfRe8Y6ySEhZo5Rj+QCcM7ZWCum7Toc8pVlc/rPEK5+C/y7JlcPi/HaRFxrKenp+v8/HyvJmytfbRqbuh9w/uG9lNuvt+QnZNpwqt/a7lJDdxa+7V1DHadDyAAsUHiRcezJrC0ylo7rrkke20CPsuato4b6ZBr+WrAgENumU1171Pte0n8KbR2dmYPoAf1KQ0ZG9r4/vn5+To/P1+np6fryy+/XGvtCiTLi6k3aXaRIjAyRTldVK0ONlG+x8gSXreBFFlAmC1vKMIxqSWNa7g1fWjTnxaENhz2O4TnFI21dkXEUJ/vV9ksFmixfaXcrrWgelZaZdRWw1jEVCUU+nRLTve2utb1RTdTRuVT5tzw9sYoBJ+7OcjEnDXxMmvjGNRZh0QO5fG8N0G/m6V0TSMGz4ZORCXQSI32zPZBrn/+85+3PjbJwjBPKkFkxPl1b64+lDhvnd7jx4/3Eib0SakUg4lCgcAk2yrXJt3U31nPdu7Qbett6t6eHn7yL2utf/kv/+WGPIpOLAyT1ZqsGq2SnWut9fnnn6/z8/P17NmzrSj2yZMnG09EEdZaWxjFI9/e3m58QBWBZ5lhqt+LHqqATafXg030U0Pq7xcXF3thnZ81vDME1aps5UbW2lXYUwTf12/wfq0dCcsYlKBu9TIj4/qZFKBYrqeIDe8OVfLrg79bTGvt71tlmI+Pj/cyaa6f4T9UxvAxRp1fPBRn0d0D9o2WoyHPzgOjbwE5NcW/yc/QY/d+9erV3pg5r+5cKCKDMuhkeU6GhK6Tw+3t7d47EhoJqGZHV0wivtwc9M5wmveChZaQGCfSXdSiza1uZCVJ1qQdHTFOOq7uzub86kL7d6i8pu3OOrC//uu/vv2n//Sfrv/6X//rdvyMzjp8TBjVhUHZDE5R47t379aLFy/W7373u3V0dLSdfrrWfhGoRsFM4NyfNY1SFa6I0e8Up9fXMPDauJbyJg39GA6Q3eS3yHEarSIkk88zq27mPacS+ryp9jYwvXvPJgowFw1T+7yGG7P/+gKRdI6LphpGt2+994MHD7bwQqPYNfYl3Ofva+2QnvsaK97RXDS75dnmuH2SZezWmfKZjTQmDzTHWuJ56kTHN//fYuZeZwylRhgH4+gzO3cTaDRT+ubNm72ayM5H10tl7ffK7xBKmocgzKhhorMiLn19+/bt+uqrr9bJyQ+sA3M8s3CwNSGtdRJmFI3YviHOfvXq1frFL36xvv322/XkyZP15ZdfbpxY+Q+TVaK28byJLClZBXKNSS5vNav2ZeiEGp1I17Uo9+3b3RtZ2jfPLvKoIS4JbQyF/FLK+tGwobxP69Xabm5utlDHYZOebxE3a+v5DaGur6/3kKwMcJ/RGqC1djVu5QCLXBhnxrTGQLiCk2MIEcWd02bRtKJ995566HoIk6Oa82MsXmhBV3B1DH0RW9GV5whnfd+cXl1dba8hc61r/M7JG6f7NwNvrTBeJf79XpTaZt5lSUUykjqMtuu7JivT0glNmnDy0BkEPeUsGQidl8czvpbglKY61O5EYP/23/7b25///Ofr17/+9WbAyid5sHKBhj31mmvtQqCeG1TlaTxPIIcWK0G1apmlJ4xD22xcO5GGzJz7UkpoowimfEOf5xmfQoX6OInsepw52RDNXDj9e/tAJpSySRb9rzdufU3lMcdS5LbWLjyc2TJOh1ExNuPUyJHyCwfLudQQTjlb1HVGdIHR8RMy7RHbHSODNtGEAu4uQnOuxsnC6q6GLv6GU2ofIfuiWLsNqiNTr2Yj185nxzCR9NHR0YZwiyYb0lUv/N513LAWiiv66q6J9sO13c1RfSxCLvo6Pf3wjs1/VCX+F198sS4vd2/o8eBHjx5tFr/ZHCRkrf+XX365vv766z0UQJBi3xLG3eA6J7Zx5AwAACAASURBVK2hqUlo6Ooe3QrUcEHrwmvBXcNCE7HWjkfzzFlcV0PQ++sXJWlMX5QhPOYM6vFaDAtNKEhsGMK7qouDBPSrxs/RNMZRrpFSVvkorgUvjIUYORr1Wp4rLKakjo6hS94ZWONFNshgrcRwM2Dlobq3s3yT7/fv5hNqVR/39u3brQi6YdxaH1CNMptmy6CihnqVI+TMeEF7Jycn2za6+/fvb0aJcZ3Io84Cb9WdAfTD/92LzOaa6oGE1pT7eOO3xtnQEcivRcI14I1mimjds2sBci7Fw7h2nRxqdxqwV69ebWd2l1P505/+tFdl30P5bm5uNtLv6Oho/fKXv1x/+Zd/+dFZTjxwLX2teENU8brnlQugMAQ2t07I7hCkVt7EM6rgNVIETUkPFY/O+5e8bW2W0KDZl8pWHwqfmwnDH3V/G2Vtur2GTWHxWvtbsfAmZFayt7+vtUs+mDvyauZQyOXfRA89mBF6XmtXZFp04L7GLdNoriZ6nP/ayLYoiKPt6QlCQMatBcdql4peioSKhlrC0cr3Ih1yf/PmzVZAy6Ca05YS0Qe67z7Vb2E+Y6WPCHn30I+iuPavjl242tC7un+ofMcYej8AociQ8WzW2PP079D995511x9/8YtfrO+++24rddBm2hpkfvTo0Xr9+vX64osv1o9//ON1cXGxfvrTn26LrfCfcs7QqDxYLbBtM4wHZSzRXa+71uFNtJNH4R2ExxCC4kdClMGrh+0CpKjQpO9aJJSnyqA0QyammbNm7njDLk7It16act67d297VyDE1pM6mtI/Pj7eO85ord2bnrQmUciVvMyfejRz2oXSBdakjTF3e5PvHdIH82IcDVUsbP8Y35amFKn1oMIWfZKHa8l31pxNbqf1iebH2HCtNc7lytZa2xl2ypPKVWp1eo1iyhm7fxMNRUvV1TodjrnOCxUjBG1pirXV/lnbjXoa3pa0lyCpAb293X8nBr79B3Ng//pf/+vbf/Nv/s36u7/7u21x44Z4UNkdaMeCBJunJ19rx3NcXV19dGDZDNXqmYuaLIYWZRLEWh9nJicP1v/7fiek6Gzei1ErBO5Cash6SOHJg1LMDGzHcohDkGlD/Epy1ChBEOVbhJfGfyjrqfavXh+f454zwzn5n87DlIFFamwNi9oqy87VlO/knZo9PsQRWlwQaMsgWurgOQzS5NBmlrqc6USAkDTk5VkWcLPgmpooci+HSb6HqJH+rCyLxubn5tt663yVkytCZpCca2Z8QEHpjUZZ5sqWrbXW9qIbf8eBXV5err/7u7/74RzYz3/+8/WnP/1p7zwlXv3o6GjjPB4+fLi++eab9eLFi/XTn/50e1GlhVml5hlMHgMKzlpUJXJngSmDgVM4RHRS9k5wkwL6NUlpSEJoTElnNTEjTRnnHkMLg1fp4pvZvHpaPye0LxdVQ+X/5TsY+yofpbi62r2dvNty1lp7J9l2QTF2lV0XU+WqL/o6DVPnVV99p6E5D1wDVARGZ/QPEnn16tV69erVxm3VCBYZ4EfNy8XFxXa/i4uLzfH2hNVuSVIHxjDa1N0w1XOPj483brNGX8jEaXfbGb0WVh7aO9iiV7I+tBbW2tXO1WhX12aG1tz00IZyZGvtXuICTfWt6V2vBRh0jPw4SPwae2A8E+nOdudfZVxAYTyLSaTgFxcX6/nz5+u3v/3t+uabbzZjd3Nzs1fc19Cu9S6e1Ti5cN/EMYizXIGV91lj/Mba5VEoPW6IYpu0hmh4JMIujG/SgUx6vA4jCw4j2I+OdqUm5VDmGHoAo7E3VPJ9MvCcGsjKbK39F0n0eba3dH56n3J4/VdZk+NcAJ13z4ZI6tjKG5VHY4CLGj2zr79zpHHHx5AxFg1RSmwzaM6/f/v27Z6DFPaVg9U35LnTeluicnR0tJ1eAnm5FolPT+o824pEGQQGvpk788tIW0c9IbVOpDpSA8UBKeydhLq1rU/6z9kaXzdv39zcbIXrM9zHLzNi08h+qt1pwH75y1+up0+frqdPn27KpKM2oF5eXq4vvvhivX//4Ux6ghPi3Lu3e+W9UKipW8pHqQi4Ropna7jZs+lNZhe1vlJqoRMvQcDdQlJD6Jkmr7yb+9dDzxqzEq0zLHaPTjiFgaJ8No+q9uyixxoCUBxStNjIqE6gsL7ndVWxZip9ekSoqaivW5naTwvDwnKNJJBTKIRN5N70PudwdXW1JQV6f4uqRZ2vXr3aDBW9gDS9Bk0o1ORC0cjJycmm96435z05t6i64VhpE4sfkd8z6Mi39ETnndygpupYnVTnEoLjcIvayrtNh1Yd13zeGr9Jk1xdXW36p4SFUeco8b7Vl0Zn8+W3n2p3cmD/8T/+x9vj4+P1v//3/17Pnj3brCer/OzZs7XWh6OmvQS3YdvkleYgTXDj71kBPbmlTqY4vJti11oHDU2fz2tDVYX7BNYtQQjfyfWY2ENIsKUYxtViw8l3USZyqZdca+0R8vpcDsZY19qvgu48tJV/mp9PPqbzxGDUyJYXo9juLzSpISo/Qj79/5RJedTJU87+VxZ1Pne1zl/He3t7+1H20331qxuXEeWcFGQxTwhp6cNaa9ONzlGRkrVQB1DDWAc8yzva9M28Q251EH3e5FPpmznwwubKmu5VH6eO6iM06rntC8P6q1/9ap2env4wDuzy8nJ99tlna60PYaKNxG/evFl/+Zd/uS4vL9fXX3+9nj9/vnlMi4+XKuyFDDqgepSrq483cfMgRVbudXp6umWHSoQ2/p6Lw08nLzSMsChbR1ZPsNYu5V+vaXwgMKWqkRV6CDHW2hmLk5MP9UCtg/F3Dfq8uLjYtr4wJkVvNzc3Gz/XOq/JJ3TBus7/6/XNGcUuahOG4w2baSP7lqMUMTREtSuiNWMNd3w+5VrOqcavnKtxQqAW0NzBIXSjh1BlOVD3rVEsd0QP/by8vPzoLDP0ATBAZxw6YOyeoV/6osBWOF2HQB5NEMg6lj89OjraajmNyWnK+n56errnmITSPb31yZMnezoqMdKMdxsZQmIc3KHymUZTPzgL+Z/+03+6XWut//W//tf62c9+tiGRZ8+erTdv3mxvFCpRSnEOoQzbXOpJi6JY/pYstO6m9zuU2SgRz5j5TjNQlK0IqoiqC9t92yZ/YJEfyqjyJG1FPhMJzedRBinwTxG19ex3NYu8YXwN0uyjBTRrdKp0TVhAoTVQGm4DF+U5Fh8agcFYa5/0rxM6hL54/1bQk40xfEo+RRZF/d2fCPXqd7mcOmqGltG1cFsjttbaCzv77KKoylk0UFl1/uqkXOv/rT9shFGKZtZMWkPWJf571jbWmBtr951Wt4rE6OJcN/oEgd2/f3/97ne/+39HYP/wD/+wzs7O1meffbZ+85vfrJ/+9Kfryy+/XP/3//7fDS3pRFHSDEEI0ELuNVW0FurhwWTL8B0VGMHPwlLPKB/UhTGTChBVwz79gLjsIsC7QU4Qziy87aLTvwcPHmyQW+xfo9OwAgJsjRZuSJ/rpSba8VkVQj8awljs5+fney+6vbraHe7X8MLfZllBnw2FMn6UE+KYfSz61sciseqMRT75OGjQc2oIOs7WMrWkAm8jtHJvOtSsuho7qLSV9kJ3SKUV6JzAWrtjnvSvyZqi+NIO5YWLSrquWg9ZB9JwdK3d28T18fXr15uOkzuH5FlFcfqDW4a+6DXj5Tn6MndvlDLo7/3sB3Ngf/M3f3P79OnT9fvf/359+eWX66uvvtogbLMEk59qPUg/o9Syb5TN4oe+6iHKcRXlGdzksBTCVbm7aMpzTQ82Y/V65UPeYnI202Bo7XO9e/tWZV5rfRROal3wZGCxkt2hLNO8N0PRcdbQ9V6T7+sYex1dKgIuUp6G2nX4yDm/s99F7BPdMWCHrp/0wRynzxm4SVrPcHGir7X2nXbn36J/9OjRVt3ee1r4a62PkFojjTneidzRHWTQ0Nn3qgsy7TWOk//s/Xv9rB+kq5zu7G8N5XRMHFHXFN14//79+u1vf7uOj49/+FuJHj9+vF68eLF++9vfrp/97Gfb4WayVmutzVOXO9A5/549e7YXKjZsq9JRaB7R9pgS1v1H2fx/clbbQIPaWpNTIU94S8l75r5+Tx5vKowJafVx0Zh7TOOKq2kWTf/JsidkqFXrW3XW2oUMMsKtEyrHRSbtN46Ct0VUF2lP42URdN8nHkt/5kKixBcXF9u9e6Kv+8xyh+rXXGD9d3OzK3OBvsmfzOb3Jo1Qwyf8FYJzlk46Lb+LKmGMutuAY6C/rdGbYSY9hcboqkJQL83tS1MsfugFh6jPEBMZkH+TA+bGu0/La6k+MJ4afOPofOhT9aLzMFH5Wrv3y94FsNb6HgT2n//zf769uLhY/+W//Jf1i1/8YiuJ8GACVpFbfmB6oZLBFL8Zq3IABOiEi55N7jsQXQdOyLIhnYhDXrS8xfQaRQtVYp93Mvx/rR3SrNLNSmvGhDK5r6r6QwiuC7+oZn63JRUTFRvrId6QjCh3jYP+ty/1wH2JxURRHTeUZvHiY8qb9Rr3syiKcKD4iUQPUQkdXx1njRbj4fnVAYaBHA5luefP6txaO5TimhqkOpTytoeim0P3L3dmXfk/w/n06dOtzqxJjKJw+l/H6tnk1fKXynQ6/3m6Rr83w8a2SaP86le/Wg8fPvxhHNh/+2//bd2/f3/9xV/8xbq4uNg2vEIIhHB8fLxt4Pb3GWocMmhVbIawoZW3mLhvBVZe6fb2dp2fn++dM/748eO9M6c0Mbjr53lglJoBbDaowvV5CexDMT4FwmsYK26PHPzkPTmDps/JhRwsgG4yp1xeUuEaC6PFofXQzcLh3SibBWGu3ctP7VBYY2ytr6ox7TNc05KCm5ubvfOr1lp7NYju1+ynBQm9+7/FZWE9fPjwo3Pt2icnRtANaKkFnHRKX429n9V4lZejA/TtwYMH28mwniFbjoOD8qbxpj+tBri9vd043IuLi63P9+7d28qhIH46xHj5na4zuJzvNErTecjSNztOV2ssHQnf66eu/2AO7N//+39/e3Z2tr799tsN3kuVHx8f720xmts0evJlBdHB40gmh2OiqyTd80dwkwCfHq+ebXqrcmXlGtwXypwLQ2vY1cV9dbUrpbDoymN8Cp2YsBpb32HEWq9TQ+H/nlekYkH02ZeXu9er1XtORZpK5e/QQY2uMRvrrGGaaKb9nKiiMur/6drJyclHxyDNRIM+T7Q6ERnDVJlVvg0Jm7mrLCCTGveZJadLZNcjiBhiyHc6/Pa3Y2sf11p7KLFocXJ97tlop+toytH1h0K9+fzagYk6q0vk/PDhwy37OdfrP/pEVidAXF1dbSQd5WTQjo6Otu/xNlMxIQ9CcE+Leq1dzZfvmWRepAv39PR0+8zvU7DN8kxDOj0INAl9FJIfWszlCmqALAYLbBojW0607usrr9TPLH6LRH9rjCgEI6KP5DI99pMnT/bCBPPgHlUw81XuomFCCd0ar85rES/UrI/mu0iv8+nZrr93797ecU2dg4kcGio1s2jcrsMBTQTpGRbiDB1PTk42J14k2kQKOVYOnW99kB1vlAIZ1eA5/qfkt37hO8l1ht7v37/fSqEO6VvHXmPt8Ab9te60rjOgoM7feuoRP9AvWone+S70+33tTgT2V3/1V7cOJDRoE1RSlufQ2QkFO5GsfRW9fFW9QS365GFKHK+1j2YmSc2wHCLq28+1dpNojFWCKdDyOQT/+PHjjWeAfA5xQ2vtsjlHRx9S832JL4Ws4en/J6cwxzUXUT2l8Gl68f597mH0bP+f3NlEJfri/+0Pg1C96BwUdVbf3E8Rp0XS0LmITBhZfZvconloCQCDOUsYypPSsU9xlhC+BJQC074/onyUeWuo2/sVoU5DSyY9AXYiTd/tzoDq/Kzxk5DBb1tbxtJ6sBr5ysJn5mNeU93yOcdKT/7+7/9+HR0drd///vf/7wjs+fPnm9W2KCzmKkoVsfH7bJSkROX19fV6/vz5dnDiWrvsUonGTnK9RDNLk1PQh4cPH+69D/IQXKYYmjHyOF1MUBBP2BeTNrtVg1yU1EXh86dPn+4ZQyEhuelTq6mbMYPE9Hse7SKsqdcltyLNIqTKybMZ5KI3Y70rDGyb58MxGkWVlRnPbO5bmU8P9BMaaijGQJljhqoRQDlO6AxXxGAWDVuMNdztE77YfEBTRbAPHz7c6huNWx+bvdYn97FuGEifc350suNhGBmvGbU4q77RCCK+NV1kXZ6ULHqsPKdxc3OzbWQ/Pj7eeyuVz1xjnou26e6n2p0GDDy+vd293mmikMk92Djts6nsvKVO37t3b52fn+8ZFvCZsjQMMMl9YYfJd+3R0dHe+yIvLi72yP3p5dzHpDRkImTXWBxS31X2tXY8Bm+EG+kCPZSNoYAgflFmvSPCvaESchlhi8i1EHhlcuTdzJ0arKJaygeNeX7R8vS4Dc2gS0ru99vb3ZvbGVz98pyWYhjzRJoNxRDQ7d9a+yFma8fmUeZ1ChOdlA6wsCfC7/O6LW1uV+ox3/rb+Z7hZR0JfW0yoKiyDtz66ckpNRrv37/fnKx2c7PLBDOSULojtjseutOC67U+PtiRTD2rR29D1jPULC0xE0UfyfyTf1kfzrOf1ctXV1fb21sKiSGAy8vLraK7YSOBlVBtlmGmjlU34xkYKILpizxq7XmEGjTKdH29e8ddQ+ca5iK3tXbkK+/LKBXtNDSxANyfskzOo97VxFog+l1Z1OjNUBsJWgKX4VJ4rO/u3RDOC0eL0GYIZ261t2/frj//+c97+lID2Fo/zqbPL6qFZKCU29vbzXHWyJNJD0SEQiz+vnfBc9X9cQTqtsikoVYzpXSyKKdb0nyP0aaHk5sjl/KAjTZOTk72SoUa0XCuwkS6o7/mQqusjH2t/Qwr42hXiHtwtL0PZ6cvfQ5nY80xYo8ePdrsxdu3bzfHSufIvTrQ9VNj+4+qA/vrv/7r2xcvXqw//vGPG79jcfR44XIejF3htHZ9vTsrqwQko2grh7Cm/ALlK3Lpgmlt1+QGwO2pANq8jnfAnR1K+5cnsFDB7hoBHm2t9RFiadaS8rcuysszJqc0+85g4BAaGmvlo/Ay3RVRb9x784bz2cZKZhSvi2n2uxzZIY5urbWhx24MJ7PJo3aeG250rBxhn0UGzcyVEpick8Xp+1MO+kg3auT839x2TeDK2o9D8+u7h+an2faiqKurD2VIfXdCG73C1TVryWi21tL4etKJ9TETVpMT7HjNS0l+cmzfrflf//rXP5wDcxCcwbYGRlZyKimLX9RVI+m19QZycXGxLeSiDSc2Ekh5IBZcX+r11tr3Wj3Z0v1qiPS92bfWa/l+s43gOUUoPzNDrRLWlM0it7exaKyLm4KTQUs66kRqhCiecbVejyI6Rsii1NdZg2XcrRnqQponntYIQl76wRgy+LNsYa3doYWM10QBM6xq6Kivwubqm/msHray3Pc7Fy3WnBX30GX730MTy3NpM6T07PK5xlGjO9FhQ2m62jkh16dPn26nxLq+Drzn/vsblGl9FGmutb+tj+O8uLjY+26pCHOED2yIXkqq+gapNuv5qZB9rbXuzFO+efNmff7559vvNjzXW7W+pVzDJHYPdeTmZndCo0lWqgGGvnr16qPtRmvtQjUCa6sC1euvtf+arUOkc4VvK8P0gsLaWe/TZ661C9XmIq1x9DeTTRn0sbIpilAmUkWuXDtPzb4WgTQcrrKQTRFl72/c3d5FDs1Wdnx0oGOgwM2s4gJL7NYAUXwbp2fIP8slWnIyEV1RYMP89rt6UrTYmjeIVptIqoagCJ2D7byJGBiYnrNVpFXd9RnE1PGQe59xe3u7d3hjD45sqFr5Tae21gcaxysXe0xUdag7Htzz+vp6vXjxYiu2Nhc11sCNOf1UuxOBvXjxYp2fn297rnilTrKw8O3bt9tpp1VOHSqRWytOcDV2JteroHy3ZRd+Tq5gG1iQQbN9a63tunrITnA9uOfXSPVUiI7Z3/osp4xCUF3QFLkGvwgFOSq5MPtZ4zf7TVZ2JFCC7h31PePqvOpDFVc4/ebNm72w59Ci50V59c6NsQpP6+xkJbtIkdXuX7TJyM5xa7Jx7Z/vtUyixmkiG1lI90OWi0KKlNzH/JSPs6BnqF2j0c9OTnZ1iRBcxzANZbnAzkt3JDC+zoyT/RY+0oW524axIQuRxKNHj/aywRB7x9HyiAcPHqzPPvtsj7sUFZQXVHv2fe1OA/b1119vD3RgXTNyBHxxcbEePny4/vznP28CqpL5iXdhECG2Q0Vrc3IatvJOPdP+kAJ0Yk18Mya+U46m+wCVRFAGC60nUvA8PB/CUnMGfl+NttbaXrfl+SarxaBkOLmI1kTVWbQId60dkbvWx+hjrbXnSOoFHQ0+5XhxcbEV407jqwnnm9JvcqLohUGcyKLPriybpeu8NUtaY9x5n/domF4Z6ENpBlHG5ASvrq7WkydP1lofZ+M1Mm3E4Hs1nocOIZh9M39r7ZcD1WB3Phry15n3/61Ra6Kl4WbLk9ru37+/GWYOr/WdpXvoS59xdLQ7ZrpJOX2kNz8YgSmJADObGiXUy8vL9fnnn29EfIU0M32qjddaewezEfYUFHRTb9gjfjtJTQDMZ/MwVZqZ7XBtjVtDoVmmQeD6Jtwsr3B8fLzV+eDioCETWu7v7du3m+dpGURr0ZDPeAWes8XFJr73rwHRGj5UqdwHvzFDJvKBMKCp2Shw0+gMIEPCm3s2x9R556lbcGps/b2Gmcw7ZkaNUW2YCvWZ9775fKIqzgTlsdYuQSNEOznZnXpibmp0qkt1OmRrIRfNNJnw+PHjdXZ2tl6/fr0XsjXErcOcut5yDDrUOfR/eiBkJw/66LP+jdw8q0YdWJH15tzL2zYROMn+2b6XxG8Y56eyhpLR7QBDQQCFv1CcI3OEn50kijDDmtZrSeF7VtFHDaHfm50qQqm3nyEoQ9Kx9zBEHsT/CbpGGIlcXgFJ62/Co25eJzdoqEXEDS2hNEYO4dwQs2R4kRtFKi9GgW5uPmSdzYV7QyStKSM/rYjWM5oQcSpCjaZ6qBZY1pO3Nqi0g+cX8RV1dQuOe9Il3t3Y6Sh0zBF4q9Tz58+3e5Aljsd4W6oh4lhrh4D10xirKw4CbH+L3CQ5ZAovLy+3Gi3zLMLpAYlFevPVZ+az89fSn7V2JUtKI25ubvbus9baxnpzc7O9cAeyKwf38uXL7bV3DLvi9xq6XnNXu9OAvXr1ag8y88CyXhZfM0qNW8X7JWV5azxSi1513O9OivDmlq3TR0frs88+24O9tfgd/Hy9lZ8mtAt68kr6U4LRJPHkRRNrrY9CBTLqGPXB/1teMavM8X0tCi13YOItRg7nU9zU5LuEcDi0orY6IH2RIZyV/tP4NwxnAJu4qMwmei8KY7hxLjVqWrOWPYL76dOnm974WSTJMK71Ae0XVci4M0gPHjxY33333YZyaiSh2Ovr670DPxm4IuKizbXWtpiPjo6201TMceVZ51EHbN40b/sxdw2T1WiW47q8vNycntIdReCuIeP5ex1QebSXL1/uncJCPxpK+6z8d8dtPlv0fqh972butfZfDVYyWlpcRmOt/QXfg9BaS3V7e7tBycvLy41HaB3OWrv0eZV4rV318iR3K6iGFCab8AmrnMuEvvra9PwkuT3Hd5pe7/UTJbhXM3Ldb9aSjrX2ObzWXpnwGZ7N8K4h4tym01CK/BoSWjSV5aHTWcmxWUB9Mtd1YmRaHs/v5qaIUxa8PKBF1OfVmEMiNbLCk6K3WeRpoTZcmoZ6cksQknEpq5jZtV7DaNUBk+c0dJ23hsTq/8pn6reQUz8YmDqU8rz37t3baIz2x/5J/TT/UyeAjUOcZ8dW2XcsHIrv15h9qt1pwN68ebOePHmyx2F0wROmTMbJyclm/T0cWnvy5Mm2MCifNG4hOK6sFrl1Xq1p0uoBxfeMiX2KFvY8gqfhlXuV4/A3Fd6b4I72RVf01MnviRBrHTZia30gyLtb/xCKpNiMRMNN11HUclvl/4QhXtDQvpGNaviimbV27+LsXE/k2/op98fX1dPimKacNQultWkMmtM6y5H6WQRs3tUm1qO3BIKxI6+GsS0UbT/LXR3qvxo1z6nROjraPxO+P30HyoeYuj7W2kdEDef1p1vwyl9Vv/qvp0L4u/uXAhGBlIdea+2dPzez7QUM5k5/OdLj4+OP6BhjvQuB3VmJ/7d/+7e3Jycn6+uvv14nJyfr6dOne/F2j7mpkEt8dgANTXimGsObm5u9e7Z5ngEWQa21e5lsiWbtEBKrABk9NWht5ZRc07Ea+ySVZz1UZUQmhzKDDcfVerXWrminnnpWoE9jd4hTbL9wVTKDLW8g86bijUHrQm0mqtyVZ3k+B9WQzn272Pvsyo4eFcnSgeoQNCJUV1fV8VlADP9ETeXMWv5RiqD3cO1EiL03NDj1ot+FarQmXmbYPpv7Novd+ZsZalz1fF8qzm3qUWVqXbQgl21R+A0R1pFAgtUB/Xjz5s361a9+tY6OfmAl/ps3b9bz58+38+xL1h8yXjyTzwjl6OjDe+imh+1RJ67vJtOrq6vNsvPwjIkJ0codUHzGcW5vqQctKvTz+Ph4ex/mrCvSd3u81tovnAWvm2qnRIcUrh5PSG48ri0BXAPESJr0ohl/ayX/NLLti35Q6h7SV55iZrfcuyHn/L1h37179zaurcWwxlelFsoIg8ydzGydr/HQq+Pj4w154cjqfJsEIOvqlH6X84ReW/pRnVcrRxe6U6BjbKRAt0UMDFRR7KHWnQS+12vKJ7YQ3NzN2jbrBD3TsM9prnUqnfO26+tddT6dJgu62tqvJv86jz0a6K72vVnI169fr2+++WYLOyhOjZcHNxxrDdPbt2/Xd999t5HAXczCAouncXP9JwAAIABJREFUZCWSvM8pYbk3kBiZm5uPic2SjoyCvsiMdVHbhtExlYS1B6wGuAu7nldGqsaghHYntai1nplxN6mUnoE/VO1MSSzqtfbroHokimeaO55S7VeznSWPi+Q8+/z8fI/TqgGYHIdnMwqMRJFbn0n2a60NKaAiXG87DB3wctkaX/rg37179/Yya7NkiI4KYcnIYqUfynn0E49chM3hTZ5TzaBwthSHdWEeGfUZRtawdM1O/q3vgOzxT56L92ufXMuId7y2m631gTvv9jQURstbRHQt18DBHR0drWfPnn1yT3XbnQbM26IpU1O7c5+URtGk3eu9CRTS4B002S3Cb9apSnFysjvj++rqas/7UKYiskLbcgmNu016UQ0jxBAbXxMSjNyhze2eAX1WecHvetCixumRyMoz9bdH0EyURx7GgKCdc2UsjDEkdHZ2tkfY19iVp2xhrT7TFwR8jdAMd2vs2rciEfJroqTHYvte+093yLahkjmpzJrR7nUMML6WUy5qM9edu7V2WcY5DqjcOijnw2AWWR/aqmSd+Tt5CsEmzVH91ya6Zbi7XnBgXYe93ne0nuCK0y4Y4AwnsqPjylbW2j8v7FC704AVDay1tleJy1xUMNrkYYpq1IqYQKEY5a0HIJxyDGvtKpAtrAcPHmzfmV6tC4JA+mwL1ATOmhnKiRdyvxlGMErTOLZ8ZPJAiFMhHsNco9l71ZAqQel3J4fSe5+fn29Gjjcln5Z4NDQw3mlUmp5vWFny3vYSzoqhmYaZg2NMyHpybOa14TI5mI8a9YaHFpdseZGEPvRZFvHp6eneW6OM2wLjsFpiVJ6WvNyj/8ijvB+DRU8fPXq0t+ODHPzU3zrorp3u8ihHVYfWjdvWFcPV/bPKmBhF/++9Hz58uJ3v3/KUm5ubdX5+vierGtmuW98/PT3dIot/VBlFvRGh1MhQSp3QqWYPes4QpZhGAmRmiHpOUV+CSxlNNMLRtScnu3OVTNI81bTI0T5FY72+vl6vXr3aM4SQi0Xv2RS66MLi0kcKarGVXFcAeXFxse12IJNuZ5ohAFgNalsQDe/WWttRKjc3H7YGQRqMNHmR8fSE5pYM5nYdMvMTh7bW2ja6WzT6Xw7PIrm9vd08MSRR423ePZd8mlnWR2FtS1/o4Pn5+Xrx4sWevuKc+gwLjezwdJzRoajDNcJXMvR514oiZp81Iyd8Jhvh6exf0WEdcmkddW30w3Xm+vHjx3tFtEXaDUMvLy/3jroie88vshJhPHjwYDsTkHEzr3am9DktvSE7TmpSFLPdacDevXu3vQDCzXE1wqDp+WdYZQFeXe1eHd8TDhCfhHV8fLzBbp918TbEE6Z24TmGhzGYxlZGhHEo+a2Qr+ipntmiL9leQ9BK7sLn6+vrDb2Sx7t37zYlqqKutQuNJE7Ioa2GgOIYC4NWRawna7mEtxaRkUWAt8MLWczlliZS4hzwPmQvIdKsaw12s2rm7sGDBxsZT851qCrS1Zh1K8vcqsTwvXr1ai9DaFzu30avyrHNZJPrObQiPEbY/fGFE1H5HsTkOy0zmte0RIIuVH9mJrD86HwBckP/kv4FAmq78NI1wnPngXUI8UJ2paGOjj4UorfMo8mLrv3vI/K/dy/ky5cvN8PlphY/5FBexyQ0XKjhY3QsLjD59evXH3FCU7hIW0pvQroVQSWy58qw8AAWYFHdWvsV2pSiY/GsyZvUY/TUDvf2t6KV3o8MGKw3b95s/Eo5sWb2ZrU2R9CQkie05cT36nR4xiIbyJIH50kPwfjW+0A/UCnlfvDgwbq4uNjjEMnXM3t/Sv3mzZvtOBXjp2fm4Pb2w8kYNQynp6dbEkHjqMy9edaHop8aEM+uUdZX8+HN2J6z1i6rxwD2OZ/KatchVRbmuhzUPLjSjoHJQ+GqOX764f/0SX+sndPT0w0IFEBIshnrzc0uOaTWzPP0c+45di0uETDggCTXXDsLrWe704D96U9/2tCAyfdwgtcYsk+FF+C1392HUB89erTtQUNWtxFK62agAvsC2whaWtj9cE3gtVZFs1jKFxgP78c7WEytdZF5KUotLCaXp0+ffmTYKEIRQnkDfS4fVFlVCSyWhrhVJt69sllrv1gZAhMSGxflKnfIKBSZtSix3FRLVtbaz442fBYuMxqSFaUhyO/+/fubk2khr9emdVEYM8RNdkVzdVLd6wfZQMhQxqH5qGNBgZQ+0ffyQJOHKjKezXh6Yqzwt/fs3OOppl7T1devX2/rRF/ovQNIi5Z7HwT8IT7OfFv3bErr/OiQOac/n2p3HmjYt+1U2UpUNgshNKToTfeDmiUim2GoF7i52Z0YWc/kOpPqXgTo/ofO85oJgnqeiRjbH7/j4hhNqMd4jbNhUiFxjdhaHxSvh8D1ma2f8bzyQA2RzcdMqjR0Lo+IFzKvxj0XFSWfIZw+VMZdiA3d5pimXHtPz2qJjs98r9tgijTdC6IVkhStkhmnwtnMAzgPtRpk/2Zf9aHzVbQ09aMy6RYpetL56rNLfHeeOw8Nw3y3xqtozPfnOXENn805GeGdm2Q4OjrajtVqWI1vU5bEcTFULWMqd16Z/mAE9u7du/Xo0aO9bQItdCwnAQaWO2C4hI1d7MIP3+dpK+xOIKvOCzoJElIqD9U3EtWTNsyZyqExTnOCeSxjmNxFkVrrpnxORj6Hfibxy8CUkIds26/WLNXrrbW2WqWinXpJSuEn48UD+tkFN1FeF7FFW2RbjoXzKvq4vr7ezo8zXobp+Ph4yxAfSut7Zo/SljkkR5/ra7O8xloqgn7SI32d4U9Pt5CsqI5wbHSSEYMmqmc+76GIdaxduNaImq2J0P1efWbwjIeMhMptTW65f8td8MOa3837+/fvtxdc0xd1pFO//X0ayTo/Cbp/VCHrj370o3V+fr6RpTIIJd7rMUDt7k1kTEyUrAal7eDm6aaua4P0Hj16tKXoC4fxAfrm7xRqxv49kE3af61dOOX5ru3OAN8r/J1GU1+aMrdQeLCe7lGe5FAq3wJs+YnvtpwAX4En4u0a4pm38jXuN9FCEV/74zNjK69RT0tvhBBF2Wvtv01In/GDc89ldzsIxT2bDJoRxrFqFiudgd7meV3TeFUvlGaYD+FW/7/WflW82kbyYgydh1V07Rnmif6Wc9RKQ9CdeVxREdEhZ4SDpp9+WhulTWqMi7Ks5YuLi/XkyZNtA/7V1dVG8zQZZp2UehCZ2Xbk759qdxqwb7/9diPm3r9/v16/fr23qVmas2GjRUDhhTZFYI7paZapxGRLLeq1bm5uNmKTAjZlb7LqTTy/2zraatRMZscwhWcS+n3JhRqbGZa1tqre130on+u6S2Ct3VE2nt26HF7eC1PWWlsB6bt37zZvOTNXJeFPTk4+OourSKXIZCIEi2vSBOrnKKdnlsD33Pfv32+H9Al7Efnmoouf83nw4MF69erVJuOWP2i4W/2X0DEmhtJcuNda+yePXF5ebkkWKNt3uhg5a/LQGjY1YqA/HEr5XI6Ow53Z8Y4NH3Z8fLz3ouhZEF7Kx//JTtKnCPThw4dbeVG3IJnncoqePddNv8so+n/BB96137mr3bmZ+9/9u393+xd/8Rfr17/+9fawtXa7xSkr62whmoCGa53IIpS1Puab1E2plTLphK7GqVD50P0OhYjzOyarPEjrvMpNyIhQAGFyjXdDABPSwkfPLrFLPjUm/l6jNb/fBTPHXYdQZSiSmuUsrSfjJetpyzmVgyp/1f51vt2Dx24yqI3Hhkw9X7+hBQW5zRo35CGLGpq5IbvhS0N+aIksy8OUpGfk19pxcJ1H925pEXlNvWxSpv3v7w1/9f1QmDvXw/xsfn/yf/1/11BDXC+9blg+G0dsTiFkcjBfHT85Pn36dL169Wp99dVX6/T0dP3ud7/7f9/MXU+ihmMOVCbm6Oho4y0ak9fbzKyX2qhJWh8fH2/xNMTjZAYDNAmySH0mb9mXhZqEho5r7TyCfjJUhNr6qoaiFsUk9CGneun5NnATWGNi8TBaTVMbtzHObUMUpRwaUrTPazEq2RrXWruyDuFhQ40an4auHBnZzAVVpXd95U+Ga+3eSrXWh5CyR1rf3t6u8/PzbeGstTZPjz8rX7PWrg6Kd4cezNFaa89AcmRNLhTpQRozg9qdKY0YoKrez1x3Hnx3rV3WXL/Nq7610n1GK5V9ERqqpnPUvpSfqo5O9N2I4dmzZ3uhZu9XGme+FlCtnHloZrfrsI7zLpB1ZxayyGOt3cIp/O2m0G6YXmtXZ9QCyoZFPCmB18PMheDs77nob25uNoPWibEdobUxM6zr52utPaNAmBRqQnbGvPCZwTF2hqVEqjEW7RUFIODrpcu1UIQaHWEG5MjLFVVI+3csM9tTlNVQxWKarUgashHCz7R/qYaZQKgzKA9UJNrQxeI1N+X1itQmaijCZbCKmMoz1cBAgp0D/Z0lGVBEjX15QPSBqGUio4ZY5r4GZWaPK+deV93p3/u9iZqm8TP+ltCU2nHQoWv0r5zWHMfJyckWhjfJwXFbQ+7xjyLxf/KTn6z379+v8/PzPcvMu8+K4CoBw1diuYugKWebjKeHr0E8Pj7ekgkW/cnJyXr58uUeydnTLCzm8kjliKo4PFUNqL81LIQEkffG1SORm8FrYsCibYg3YfRau1KHGQq3hkzfus/QpAuViup8t9xH0/HQivsUcVV59ePkZJfBIzPGFJ0wEU836NdAH5oTMoLQr66uNo5vZj79n46SY6vA3R+iqDEtgYx4VmLRkA+irxGq3mt1aIcMpyQL1FNOqRniGjf60MLQItnylV1jU39xgGTeJFiNfrcfWcPQoPXFgXe9WEP607lpGdbR0dEeqV8DTW4Kou9qdxqwb775Zr148WI9f/58G0AJb62Gqe/hm6c3ts7Dwm/WCMQ0If1pwdnWgFR0YmyLCRGZNWRr7b+Qg5CKLlvz0nDWeMufUEQv5tBH32lldGG6Ce82IZPdDNY0bhbcRE+tiSpC6zMbRkF4NdDmrydVyDRZJEVhFmCNkIwwWZH98fHxll1+8+bNtvWnMm4Yql9Q5MOHD7fs99OnT9e9e/e2/8tods6cZ2++Ss6Ti3H4vZ9VZ9SKQUsW8HwxRou9J9qrQe7WGDKqgSTb8sszsjDnqB1G3lx5j6v+zKiIbPFS1lOTG0dHu8RUk1+ds+vr63V+fr53fNBauyytz7v29bnhNj2jz5wKubILn2p3GrBnz56t7777btuuQVg8ueyRhzsQjSL34SaupGo9ncl++vTppohrrfXdd99t925TI8Qg1DNNaN4Qw2SVAykUbwlGUc1a+2d36TOU0c219+59OAe+L7mdkFoWca39dDneo+Ufhe360JKM3oNxgA5n1qh1X35n0HxGkci2h/TpZ8l7CwGKa2u9GyPIIJAF5DXDTv3lJHx/bsAuHdC/9b76Tz4WpVKV8oNFdmRQZGM9NJHT5zdcbKP7xqFvddbdT9sxVvbVm6JA5U0cNRRcTsl9PLeHcs5tO2o7y1G658OHD9eTJ08+iUJFKOTKwbln5eH3Jj1alHwXB3anAXv58uU6OzvbXkJbgUq79mwwkLCevCii4YJQYCIm1+KvPvvss00ZGZsuHjzIWmtLAfNIjEbT7xR5LpZmszpZ0/vNFLwFAXGpvaryd3O1PrVSfJ6YUTm0f0pIPNMYhFdgOIPR0P34+PgjhW+yohkli68hvf41/Cp532OJqied88pQaFkD2jC1mT6yrD55Xs9h615azxXW1qFxNsbbGrZGEJ5dakATMuuPvkOH5TEruyJ9rWit/S2vVwNs/EJy89G5neS8vjBk1lL3yhr3dFYdB8RUR1802XCxe3Bvb2+3WrMiQvraOXc/6PCudqcBe/DgwfrjH/+4FbN2MlqwenV1tVeZb8FOklAni1TW2oVHLcWYmaV6LwOrMIo09LG/Nxxca0ey6zPBzYQExbdwW+zXui5jo/QND+tpGAbP84xu96lxrEExViijfFcVTt/61uMqauVTDqLcy+THis58VkpBP2eleDkpTf8ZIZu2m2jQJ4jfQm0NYMPnjrfEf7PI5nTyNkJ6DpXxq0MpCpmvAiSPhoaNIqoPE2l2rnuSRRc2HaxTaORQzrOclnm5vb396Gyxoh33rCOaznsamYlY9UVmUQhJRyEvBdZkOo3ZfHYrDg61Ow3Y9fX13stWLQ5v6uUxeB4PFwI1dq6gyguwzhRjGopmXXhcAlOta8GVmK1Q6pltHsWZQUIVdjNw2uQShDf62DCTN7LI6oko+iFvDv73OeXEKG4TCJyJZxaRaJSp9TiU/OpqV6XesLQZoBbMmjPfabNghR0UlLcun1jPzohyFJ1P97TAGYIiRH1sn8jUHM9N9r7jJx1gGMioXCmZdj71zYLs6RrGT15aryt32bmfHHAjAjoNvdZQ1zi655w/rWjXHHEmXRfGQ/cYeXrVzDcZkzN5apB3eW99dbBl5/wQB9h2pwFzWkLrvxqDF3n4WRLZxFDMub3BgNyPYPqc7oZ33YSVBFXCsQiCwjnf2/X+Tyk80+IhPM+sJ/a3EuJFHkV7DFbraGr0anQsuoYAjFOzqhAr5FCuB3qYBqaEa+fVK+8qtyJo8isy6O/k19e/FU26D8Wtk+oCmckQfaZX3WFhkZQ0N/8We+Xd+rYuiC62GtWJkiuPogOOSt9cp8915MbZZEJLZG5ubrbz+62Z6sFsRZYTXTdRovaqIXGTJEVT5c+KylA2khprfajDM36y757VtXbJmIbDs5n3Ho3UhNBd7c46MGGimqr79++vs7OzLb1JEL57797uxapNUVMkh+dNq4o/sgh7TeNrELVbLRqvT+G1JqeL0HaoZiO1GfrW61kAPOshj9aJmiQv3qgnrl5dXe2dcqlMpJ6qCt4QfPJCvuOaGgyLRV3a9fX19nbzclEWXuE9JcRFWYAWY4noQ2HAWvsvnCUr47JtqDKZqLZbr9x/OsoWUZPLbP7eZ621W7juRabTQTUSKFqC2KtLoovyQ/reOrgaKkhxJsCmblXvJs/c7zqeqki9/S5CagjaNVZkr0+9bqJFme4+CxJmrBj8OqfOUZ3YD85C8n4eeHJysveSgsbdFlw7YFGWmK/wqiyutYCbgQNdGUithZnTUqucbraQl5ivbtsTSBSw9+znk9+poOutjWH+ffJ3/uazchhkZRwloOtAGJYWy5b36CLR7CNlgJo4kP3Rdyi0RqPKOPnM1o71iHCL0Xiurq72soxTN4q0yLooxjyVkOd4a1R7f8aZ4Z7lFZVh65Jcd3Nzs7c5fK1dQXHnrOuEkWm00fEwcKIU+tNQvn0s5aEZTyMK4290g1Os7Kw1OgC56WvHwii6V/VzrbX33gX/FyJeX19vIIjeyE4y9PScvt6VhbwTgfHUDJHjpWvhj4+PN5KfEmk3N/sZOANpaMZgmXSv8aIUzVBCEATeDc0z1jaBLQisp6wnbSv0bWu/24o+/F7laajZNyBR4NYBUeQmSSxWMhd2kd/kl2afa7gYgaLMhvz61WwgLqhjqWctcmaAycBYq8zGXWRjjEVoLblp6QEZ1xjP5Mda+85N5u/169d7CMVCn0i9C7oylGVu1o7M8F+uFV6Wc1tr7R3B03KYhpJzDotIioobzhk/edUwae7juiJCsvK9tdYevTKRPF0kC1y0/9uvPBHto0ePtoLkyq1z0tCV8/lUuxOBOUO8k1KLCAZTsHIs5b1svxEqdVJwV+7ZbQSu92zWugbCDnpJhKI2Fr2K36zZNEb+X/5krX1vgEtwj7mYJsrhdVv42TBym4ij3am3NQYlZSlHwzz3E9oyiu/evdtC5RrfenVGBNIynpYGdF4oovH57tHR0UHOwr07hoYDDf3pRdP9NWjd0VAjyYCUq6KbXYBC5/J1natZx8c5VH5HR0d7ZTv+/+DBg624FV80DbD5KeF+dna2ya2GrYbAc2u8psFpCGfuZjTUM/fX2u1S6Ekl1f8a+K4F31f3Z23MQ04nart3b/eaPc/zs06/z/7/B4HdacC++OKL7UYtOyhpX6PRFyt0EizYKuFauyLHWT1tAN2awoh5g43JqLGgbMIDBgGv0O87p2xPGEEdDXdbfHdoC0nv0zoiE9Q4v8aVl9ZKYlvAzdzx6H0xAw5pcnI84MOHH153Ve8+620auvm8WThzRZE+xRN+6v+Qn9+70LT+vWGd56kLbEgONRVFFCEVgUPe+lbutk6YLG9vd697q1zU+dEzYZGEyOvXr/dCb/1oSN0sejPwLW0oEvXsNske89f7rrWP6KB58sVRlTaYdZGauSLTWZ/Y+k062uytvpOj9yPUGJNBQ0jfsVY+1b73PLBHjx5taIC11G5ubjbUBVL6O8JuhitzsbhPSUyDf/78+SYUn/X43SouYycml81pGrkG4dCEeUYriKcA8QwlgQ8hP5PQEGFyLkVvNdiTzK7HnQuhxZrau3fv9hQNQd6wlpE/Pj7eDBwZQGrmicxaU9Xwr9635HZD0VkdXkcmi9Zwn6FiePq255avlMtScA2VrbXjHX3H2P1DWJdU7rxNntdBi3WiZMpBlWtr63X6xbm13KYotxFL7znDLQ7b/5sR1i9/b8HpIQSK51prl8BpmN7wuqGfNg/b1HoYZI1bHSqdNtYZgs9253lgf/u3f3u71gdD9uTJk+0h80FdPMKfcgsEbkAlmycP4rszg+Zn0eCM2U9PT/cOUGwRHKNjUhvLV0CynJ3g+Z05rrYu3oZ1DJfdCya4W6BAf+OetU6T1G1oDYaTc8c6ZVhZM9LmZGZ8zHPnqTVb1Ylm1tZaG9cynVXlUmNeHWmbWdup2GTVULT9b3hfQ16ZVSZzzFq5sGajZ1jc8dUQVgYannf2q3KofpTjajuEUg7p9aFsZPV7rikyI1+60lCysmxyos7Z9YyjdVX06/y/Gshf/epX6+Tk5IedB/bq1avtDcGIehbcthbelCI8evRo8ySPHj3aTm+ooIRoXUStcF9r7b1pqLE0QTVM6UTjY/quxWmEILnZSsI3W9SJLiHdyTau6TllF9dae4W3t7e7F7MKc/XLIoUGeGT8SfmCen1hCYUrqoWMQPyJFGS9unDrHXlNY5le+NCis3G7zm6tHcJoqEEn1to/faNOrw5zhoXCOCelIMihzGkkyLDOxRzq36ec+xzLWjvqwPowh6ICz6sRPT093YBB71laYvJS5qgIrGuhxpu+om9KV7Twt2urzz45OdnLKHJSc46rMxNl6a/QvpvvhdA+KxduvUzdme17N3O/e/dufffdd1utzlr7r4riEbTyOs79MSBCYMFnKLDWh7dls8g8bSeykzP/zxvgfj7VKLtxCC9Mqnt1Itfar2xuKNL7HurfrDejMCanYe7V1dXGX3lmq+aLWkv06vdc6MYFxTRMmUWW6sCMrQZ9Gra19mkCC5KM6gSqhDWczZr5f40hVAjZVZFrdFwzs26cmUTP1JfyfOWtyNg9fe47+qt/mjmEdjyjB0UWqXhWx0cfGaYZbTCwHJW/tWC0Dt440D0cZd9AT66+4zPzBLm1XV9ff8R7dd30zUX0jx431L+5udl7CU8Lxju/n2p3GjC1Gi0itOAoUGF1lWNCVEphEnlIi8aEP3nyZK+MYsb89RJzsspP9U0vDTcsDBwZJNSXSqhc7iLsT0inY6R0a+0fAVSv63km+Pp6d4yz/iI6TSq5Nhs8F7p/5Mao2Hfms9vb221rWA3HLAC2EOactuSkDqWhfps9ceSoT0WRlJ8u1EBLHHiuRVJj0LDSnFQvmyFnXLVmBGe5x6eMDP3qrpKrq6vNSHEUL1++3OZ78mRd9HX+9+7d24wFnXnz5s1ets+ctLQE0qF/6gwZQHNU7qs62jIZTclI1+1cD70OV14ZOT7H+O/fv78ePny4Efnmex5IenPzoeyovOmn2p0G7MmTJ9vxrzc3u9dY8TAgOiF0awhh83w8DONFCEUTk6tox01UN3zjWAp7OwEUufc2cfZzep53NDKms5JYSGo8JrAvVNDPhniTi+I9+94ABqBIpIvVJHbBNGS1CFsjxUApF2jmuB63oaP+l9sqB9JMZHk6/ZpFjS2H8Z2O5/j4eM9ouG+97tHR7mC7GqyGgPrRN77PUMi1Laep4dO/WW5Tfo9Ra1aP7N3Pz9Y3qm73LIagCGhyf+YARXNIDybSqjEju7V2Z3FVvn2Bi/DSOD27hdgQn8+sPfd9/fr1BnhcT6cbqnqe8JbOGH+zj2zL5GXb7jRgf/rTn9bjx4+3VyZ5QwxYPGGqkLGTeagGyD1Ya4rRavx+v0peYpgA2wdK0spe8LV9cU49hSV0nq3n6V9eXu59H4lfg02htHq8Tl6NCYNeZMrYtvL+7Ozsozoez20dTwsnIQYKSUkqr2kUKVENxAznGjZNgtwcnpyc7CGdtXbhdV8hxxh6HhRdhNm5nOMvMV49EYa1n9XBEvYcdO/ZLGTDMvNqoZ2cfCibcI3FNpFrHZM5aYlK55NjmsmIzrfQuCiqzh1wKPpu8WkdVNdfkZsIZO5kacG4NeD9Dz4TxeD/jo6O9rYL1tC3XAZPZm3r1w82YCU5z87O9rZnFApTEPVSUAyE0kVe/qNe3z0p31q7Km4G9Obmw4GH0u71RDzDTNErMWj2pt69b+/xt5l1q6dByncyZ0jruvJJRUz1UIX/PVPMdcLN7vMrurAA/YQOZIssFCF/n1sFp+ST/2n4W7kyWP0uspjiU+KSwK27Mg7XNyw3Pn1ba3+zf/vg3vrpGv3r7oUaJ4a2iL/cn4Xk/vrgp1CniFZNozGWmO615rEcY2vaZqLF94WqdR76hu4pH2XufW8i8uotZ0E2+ClNeOu57t8z6qBOpyS/e/duq+Gr7L2puw6i3F+3zP1gEv/nP//5Oj8//4jbWGtXnV6inWA/5VkIpgiOsE0gJawntr2UaDJ9AAAgAElEQVSoZRJzC0Wf1ed5z6AFS0iMT5EfwyH0Oj093cui1uvN5EWVZK39UgvX1tsUencLDmPEg07+gVczXvIv/zBT9+UxhK7dGVAjoH89dVR/eU1zMKvjhU2da4roH6dYB6Df9+7d2+M9OJyiQwulVfINw/WlIRdZQSZ+kkEztEVYa+2Okamu1aiutX/eXPdINrSvQa5j6Ib8GnWL3bhqnH2/i3/SC+Q+F/+kWzjQ1pDVKckcStLQBfrUqKZ2gjH3prKGwtXhtXZHXDNypQo6l4fanXVgf/M3f3N7dna2Xr58uVlaiiCj4PNDB85NHqtektIV+QhRKFNruqCLuYfLQCGl/q01KZ5/qJlAKEe9y6wJ6318p4u1z2DAmm2ZZ32ttUsi9FRWiiwMwp+0oJfy9gC7bg3qAihvORFi54ATaW0OZSry5IRqvLSGxEV9LceAHlxnjBb2LDae92xYXurAPSqfmUBhLJvl6h7djt21Dc3nWCujUgPkUsOlf0Ui1eOpS1O2dKOoqtfVMHH284DD2e+5TttfBv5QXWfXVvdkWov+Zp0IDYsyJzLVj+vr6/Xo0aP17t279d//+39fZ2dn67e//e3/ex3Y06dPN2EfHx9vx+XKIvAm0IJwAVJ4/fr1XjbPAmZlCyE7EJ5e1gJcV/4AQa212zNX48X7EKr71RNOTqxK7Bklbik0j89bQXjd4KsxQELVNpNloiyCtXZvEbcAz87ONu/VeizyqyK3NMAYa3whZ3NaL25vnvlpogFq0qfj4+M9T1zerGR/ZbfWjhYolWAeoNKGU80gux4/VALc9y8vL7cawxpDfTEXFpgdEr0/eZojRqPzO9GNvljYntN71HjhWBuilverfPwjyxoqfWkSwjwX9U4U3PdPdkx99wQ9u7ra7UTQui2uRrXn7TUK0jd1aXSwtoD8UR9dn59qd55G8e23364XL15sXkjWwMBN4oMHDzYyU8PFIOZm2QHF4SE0fy+vMGEtOFsFnXVmnRQK07CxGUQe1rPL79Uj6YeTJcob1PhSFIrcEK7jrLE1eUVNmgmfz6nxqow0NVBkAfWST1GP67thv5yZn0080IVmmrSpfPPvh8bQ+dU3RqdIXbhcA14DWTnN6vwagLmhfqL2ojMyfPz48bboGT4hbQ0Qx1tH5LknJyd7We/OX/WuIS4ZaRO5zXWAj2yJQsGI7WMz1H78+PGWafW8s7Oz9erVq02uRWGHSnCguu5JNe4i4iZv5r09u5UKh9qdCOz58+cb2moHeV4PA1VL2Lfux2Qj+Zs+L0paa2e43KvZGkJRs6XNbCckWOH4XSg1YfMkTRvWTo8HgZV785boQ2FODZr+TQQqNNNXSMNmYeRwQyTzgTMzHtuVKEQXermTytu4yi0yErM+y+8NrcjMfOA5JofTgtHKAP/FMFah67yKoA/RFOX5ytm1JKDzyiGZMwbPta9fv976dHR0tGXaa7x6GCMZytjXMHtGw05jMc6WN5Qf01q7N8PwtsePH299bIKrRPzJyckWQWilhuiUU2kawupDkZ817vw31IgoiY2gQ+USZ6KmJ1f84Czkq1evtvKBFqj1PXj1WgbWUyjW2iltiX9NR+sxS+BNiwzRQQktWNWUPdQjdqtOoatWHqjPK1ehTYVfa3+XfhMTXdw1kjUIxoWj8azyDr0XPmqt/SppxuzJkydbiGDyuzWrJRYzI9Z5a6mBhEKNaCmCGpImB2QdZ4jTZ5uT6hM5z1ILTrGouaUBDeebBCDPSQyfnJxsBdPmho49efJkOyZnhm/k5fnG2JNda6QmAuzcNznVl7nSUX2jU4xyUf1EKSKgFpwWxWp98/yMEBi6uavFuA8hwMvLy/Xq1auP0Oft7e26uLjY47usw6kXU59+cBby+fPn2wDViHShetBM8RMAoZhghq/eq4ZwcmLCS9+rB+NFKVi9kVBstnI3DICFCS10/yXD19Mr24rumqHsFhqtyjxDqbXWHiqlqIpHmyWivMbajdgWlcyxhW7ReulpeQbPmuEA+RXy9x2BvGq9aENMc9nXrUFm7ltnVk/cMKgGowulWeAmRyZZfXR0tM3x7IPv1PhZmE4tLRpiEA/NpcXYI6U4FDxm+bcai5YM6D/HbF6bqdVX66kUwWwXFxcfnZCKoNf/hsOyqAxxuWVr8e3bt+uzzz7b/l45inBEDnXIa60tdC1FZD47Z8Yzs/2z3WnAfve7321v5Z7nHE3rX+82i1R1kNAeP368KbvrWkpQT97SjGbGTPqh+Lh97O523tbkXF5eblxE0+oT6dSotYShiwu6bHj1KQ/Zz423YQVlUZzKWPtOU9BNvVN0hlgGqImELtgSzM28Qll1JjUMJc0pWMsOmq2EbswFlHJ6erolJ9zPfJe3I18yZazLnc2QvtlRTou+kWsN3uT7Li93x6DXURwf77bX+Dd3eZyfn2/z4LOiQjLr+MwPg7/WrqTHWPWvIeTcY/vw4cP14MGDPV5ZDaWo4fp693Z76xNFcXm5Oxi0BxBAU/Tx/v372+m26rkYQM+xroo4cXLVrUYPngdRP3r0aG9uDrU7DdgXX3yxXr58ua6urrYMZLOMQhEKbHEITxgq7fT0dG9iWufiegu/Rw53wt2zpyLIbNRQEYyJmW81Kt9RRZoCna01QUdHu2LBmdyYB78VXfk3M5P1SgxhwzEFqhNdNmy5d2/3YhUK5PvdUCv0a2shpHC6yFbFNUNYQ8EQCRvna/XMeVFGCyHJv98vioFqGMOGGnWEsmUTlVXHPKuonhFgJFsiYN45Fn2kh/pTRNuwVR+mE5u60bXSUhqNwxF+FdnoC6csYqrh7JqtfpvHSde0oJysPLfyL3Fv3dY5ciIXFxd75ULkSc/olAgD/XBXu9OA/fnPf/5oT6CaL8rqX8lfg+7LPHljpF7DpBLTsiZTaSl534ikdeMnwXeCJQ+01rbUExEoYVbB+6/Iz0S1wLbhtN+bJeX1q9RFmpOPIgfoaPIwDXGFneTPK/uu5+pbF83c3VADL2QkP8rMWWnQTRXVGegN/7X+3nolzm2tte1NPcRdyrY1NCWDot/yMT2U8BDqZyCgDuOY3G55OeEeFMlY1yi7hiw9c75Nqc3irlObc9UIonN66CQYAGTyau7jGRcXFwezn5rici/P8QrGGmz9UEZhN4/mO7O2Dfqi6z+YA3ODvqnEDQtvCdrPQl5WvOUFthjg1Eq0Wqi+SzkYpSqM71OKCsFk1YP2bzM0mkfLNDlgsXZLTE9aqOGDkGqUKY3v9gQK1xYmVwmnJ6+sGvboS5XDM7tHrjJrPVX/3zBT+AHR+F4XfMPuetMiw6KvQ6F3k0T0RbhZeoJMje/29nbjaZvJqpGDFvp+04az5TqrQxwsudd5dA6MFbda+mQW+3YnQJGMMfZsrBaI0v/yrMJ1+gmBet7x8fFe9Tz91U86MbPJa32oAeXERDgNv4WDQlJzMMNqYzZ3OEHPppuAElvz5z//eY8r+1S704D13B4LEYF3fX2999IPyEkadSp7vUhflInkbSshPluJQsJaa6e0BK3Gx/1xLPVgBH5zc7Nevny554GULRR613sUFdzc3GzcgQMgKYrsH+VqdbMM4oTuNVI+L6GrXxCJEJZhFwKReevRalQod+v19NtPcitH1xC2pPgkrlvQ6jML3aK1kNpHDvD29nbbaiXkrvHyPQuh+2DJpUi399dfBmVu5/KTIWBc6sRub3fbjN6/f78hjdZeFZUwOGgM95zoCyclBCTv8sjV4+vr623PIkDQdWeOGdomSYqm6bqIgkGhw6WK1lobrSRB13Pg/Ox89ngsKJdOVVe6FsqvHmp3GrCvv/56Iy6b7gTBnz17timmgROe1ord6+v9CuS+eso9OkH1yFU+hsOEtbnftOglSGcI1nu2Cr/kZPclUhwCvrq62g6I6zswjYtM/K2GoqllC6snYXQiy1VAqL5DlhRD6+IomV1+SF8gLfs/8UEtXq2ym5MaiyY9GKaGVLztNIyer0+uWWt39FF3NiDaeXb3aDrffTxXJhDCNBZj5WhKAzCOJycf3mDO2HE+9HuendXnrLULIdVITaPVkIvsRAGeVaPa0iXGoTpxqLynTms6q667AgHfgar8/v79++3FN48ePVoXFxebznuWMHRy3M24F1UaK8dZvvlT7XvPxL+9vd24MJXdFXSNSjmjLhTw1LNYZgqPO/Dd1iY1kwElWHStjyo57L6e0exlDRllU4oAXejTFGArtyd3U68zD2gzoRR8osvG/pWB8ddwUljyMuHdBM1wNpwrX8IoldtkrLpH1fctlo6pxrMhkJR9kzCdP4c1MrjmpuPv86Hh8jmlD2oMqg+4uTpKC77zWMNXB1F+r/NDn5rdLVpuYqvJpxo2+uaen2oN02dFuiik+4PbZv2XOeEE6AADX32H9kuPzP62b4wrWujo6Gidn5/vHYRwiBYh5yb01lp7+22/+uqrdXR09MPOxP/222+33eTn5+d7e/Z69Gw5hk7S5ILAYh3kDRra1Xi5f2HmIf6qC1SoRmCIRtlLk16P0n2QDRdAfWNqHRSl8B2hQMfc2jd9c68ap8pI6+Kc3J5nzIwP2SiLaPhLyfRrhgOI7y4oxohB972SvtOgHx8fb4rLqPZNUhS9RLnrGO9ZI9R6rOvr670yEnNm3DNz5k3yDZt6PVm3GLb9IS+0CBla0E1ukM/9+/fXxcXFXiLBHOGfKruGmW3lXY21pDlnVeM4IwR/M34hJL21dmbNW+9XAFAHeXS02wvcc9U4nfZ7AgxgSClNHfT9+/fXs2fPtvFPiqntexHYycmH43EL70FjN693r4fuYuEpSuIyVo8ePdo7VsaEm4gS3DPk0ywKtT6M0vTmWsfNC2k4pZub/aOjFSpaeDVAUNBETBSp0J3xaAZ0ZvOKKqvoHQ8595rva0VMlenbt2+34sOJ/iC+osh68In0yLaorWOprFoz2HDZz3JXjNs09tULujHr0zjJ8mTGouK/4279UXWS41KuMREGpNM1cWiu3HfqYZFt57WOit5XXw4hdeNuGA3x9Dtrra1wd61dhDQ5QmtsolgGjY4U2UG3pTY6j3hEv3dM79+/X7///e/X7e3tD0NgFxcX68WLF5vnMFBkvkmYsNLbaHAQJfYoSKHsmzdv9rgG93J9kYTnNUxa68OuAcLXL5M5a7B43Hv37m0nbhwfH++9zPby8nLzolUMC4En9qwarypdEUlDkSJNSHEqbw0W1IofmFnL6cWLyCp7fYdGNYuvi5aRbRmLe1M05G0RCgRYDtEzbm5utu1prSubHn6Oba1dNlGTlq+DefLkyRb2tNxnrbXtVy2CFC00077W2vSy/WZonz59upeh1qdZouC6GhXjMZ8I8nKEHGcTDsZPb2cxrPtrE0w0otAnOtzwr06jVEa5tGkkOQz6ZPz0g74bbyOOhuHe/Vm5TgAz250G7PPPP1+/+93vtt31R0dH6+XLl5uBmm/uMQndz1WE0a0PFkTDmLkoixbqqWS4CLPX1TMgGrtQy5fwMGt9MEzexOy587VuNTpaXw/vmV20NzcfMpwNbQr9Pd8G5sqzXpXB7mvX6limd3eNUL73sjB6PE5bw6TyeXP7FUdEjpIPveeTJ0+2sMW8UXBZrDdv3uzxM4eQ5HyW+jJhRjkuhskcdyN2naH7+U51i6HVnNLw8OHDLSytrPSRAa8DORSh0C2GVgFw0V9DQvxvX348w3drhDOTkJi6wYF3jR1KIFn3zTSXb6MPNryfn59v97dmIXpOiUMhe+uCXDmk7jf+wQZM1hA6YWVbBiAt/ujRo72KaoapxKiaD167BbBVzoYvtkL0Ba4l/XhZwmsoA/1QyhoXnM8MjVrSwGBALY6zNoGqnt++fbspWHf7UxByqrJ5TvnDkq79bmE4xdE/E07xyxs0SVIkAiVQtC6IIooin/fv36+Li4tNduXUzFnrfVzf19fpLwNhzvEo9AbyrmEt91LOqjpJ5uYVUmkdXEsScHyTpK7jtBChMY7N83Cf5NnMqOshlHK6E1np66zKbyjYUJzRIVu61+jg6dOnW9aQ4zMvrdVimNfavWi3xu3i4mLb3M6RM+jsA3vAAEJdHLPx+1fDPkl9DrKntH6q3XkemMVDuPUK79+/32Dl48eP9wxILW7rs3Ta/xm1mcKu0uIbTL57t+hyeqxZrV1v1Xt7KWczbw0leg0k1/7VoOLw6jGm55vPr+dW82MMJfxN5CE+rgbDcyUqKLti1FZrHx8fbwkM/ZnZJ/fs/tGGxv5PWVvMXA7JYu8R1BAe4110MrmdhszCmUNeuQanzrMIWt/ItmHkpzKOzZDVwNC/Gf7OjF3De+UgEIbfFYZ27G0twi36K43Sou179+7tZVftc4Tg8JDWoetaEWCd6yukVRLf39+8ebPOzs42Q69/dKd6CvSU0unco5QOcb6zfW8lfssRTIRFeigcab0KwwANrfWBsPeig+kVTXC9Uzm0KuBa+1X/NRaNwfUdOuiue96yXAXFbcUw71K43XKOJihaM1cDdej3FkU2VGMUypl0xwIuAZLqZJuDmZpuoWersDVo2JxyOM0sFZmVg4EEfLfyopQztOYAy6cYS41TkSR5F020HR8fb1k6iMU8Vi/pyL179/Zqw8i/hp8MGf9mY7WeRjyJ/UmzuL8Gzamh6vFIa+3e6eB4KIZAyMuBFjDQi74TkiMij6I1TrmlF2Tv757rXrjL29vbLbNaDnuOuWF1eXP6M7ONtTl3AYE7DdhXX321Hj58uJ1C2Th2rd15TTgMC83kNdYWVuEhGjr6jlNTG5YwLNBOicfugetich2lLXLDa1QZupfQQm+IdCidbZE29COXbqL9VKNE/f9HkxMPZVF5tj2lvCsov9baC2v0r4Wh2ty/qpFrZVQZ86Cz1dj1EEFzVwTeHQrTYXEOENHp6emmNzVudSJr7bY1+Yz+NYPu56H3ZZYLurq62o5qIqsmABrmM3ZQbUNLaLjjLJdaPYEuyzO9evVqG6t+kS/ZTPRKh3tsFKfuOJsaJOMpx8opGpt1Qw4SeYeykfpSIw/59b50WalWI5DpbD/V7jRg/+yf/bONgMZZ6UQRi2MvSmgSyuvXr7drdbwnN7TjhZ9NwfPChwj/eiyLtAKcz6No9+/f3yalZx/VCMxzwLpYKEZr20rMu5eJLkHvurV2IXcTCkU7rejvhnYIFp/UTfIWXjNDLUJuGOF5rcXTrzoDn2nlCt2/Bt+ROQ3JGMbJ/5TXRCPw9L1f+9frJnruM+7du7fxt3SH8YSarq6u9k4RRVv0jP5puMwnowpJ2dRM7nShiLRo8vT0dG/3wlofsv+e8/jx4z3HM9dYObfq5tu3bzdjhaMyX+oZGS1Gp8ABipMx9hzh3bNnz7aDAtb6sDboaus7yY7utqqggMff9PHBgwdbP38wAvvmm2/W559/vil7QzmxPA5DzF1isPxNs2iEQcEPbfGZJB9BNPykIA09QNVa/8vLy726Hd5rnjtVEv358+d7mbOpgC9fvlyvX7/ejM48f4shgzbWWnsGo89Ffh7KGOEtZmuxYeWx1m4vHhlyEhYl5RFGIHH1C9qc5Sf9vcWYFJ8+1OuXB1prZ1QtHE4H0QzRm2dOp+OjDzWiSOImjeiB7LJMeIlx15MZY+Zv+tTTNXB+7Uv5p/7fZzM5VKM0SeqW5Lx8+XKveLfGWfRiDoTp7s8hNCyE6HpdnVWpAPwfIGDerq+vtzny3W4NamPQCgIKMBqtSACY7373U+1OA/ajH/1oy/JZwA3xeEBWWhhZ6Co97O8lhNfahW2TY5nbPdbaWeaJFkrQCl0bY5eMbSi61u7U1z6XUH2XcbKIjcEG1xpaCjFLKSzQGrHXr1/vpdCNCQIw8c0QMljlg8iI520msgiqmSp9WGu32M/OzvZI26LZyhu6pGQMl6Zv+tmEwtHR0fYCGEkU6Mw1EAmyvoilaBu3wnBxtDUY19fXWxZNWNvxqGucYV3Ddk6WoWOk3INu0X+ois6Yl/Ky+j1PPSk6LcFdB2FezT2AgO5hiDof/glzOWeymJyUnxyfM/afPXu2RSfuVb6tIazxtphb3wGOrj2h9tu3b7f3kt5VQrHW91Ti/9Vf/dXt559/vr799tttV3o9aif46GhXu9F9YpMrWGttIaVJ7ELDVcmOzGyX32e4Y8FNEpJC8vQN13jSSXivtTN6nqe/NXgl3MkEMuhzKRSjPvtuwt2D0vN2vSeFaLU0+deL9u+HnukaY+hnPe1jZoE4ln6/WefO+bzvDAGnty5f2b4K9eZ3O67pqT8VdkBlUz+F6JUlg8jxNOt7SJ79XVlNEUa5xNm/SWx3TDi1WSx9SO79rM8qumy5C13pnsoZ/Qj9qyvTQZY+WGuHKumL+W6ENiv+G0LiPX/729+utdYPq8R//vz5XgU+D8NYWLCMVy1z0dpa+3H/zABCFSwzYyJMJZBJjHtu0/UlcGtMZZT0uWHAXEi8irHypicnH3bY8zwlwY1F/8nI+Jo5q/L6rEoOkZEJFFLiHPlZctf1zZTxaA0xagwZCL/rt3s4IrlOivL3TeZFMWvtG5TOsWdZaC0CZrAtVNf67kTWE2H2sy4GfZ3yIYciSH+HVtQhNhqAkNt3ffMWr7V2vGNlU8NNv+oEO8f6BjnTK4t/opOjow/8Z5HORHQtEzLXa+3zT+0D5Fnjg0M2T3UmDJH1UD7WXLofOuP4+EO9pgL5ho9z18lsdxqwP/7xj9smaDwJpe2GXJ3volhrf1f7mzdv9lK3a+1exlAuqTCboAhgGgzCrtdpuNWJKelOmBMtgdnKKIp43F8Y/PTp03VxcfER4ugrwXiStXZZGK18EwM5DYt/lT8Falq/ix167dhdh7xWkLrWbsG4x6wpmhufez8LqkZBOz4+3sJBYy2qM6YaYIaupQd1SuVEGrqSURHlIV6Q3E9PT7dxFT1cXv5/7d1bbhs5EIVhZgDlwVEuC8j+N5Yl2EDg2AnmYfC1flUUzWsMsIAgsmyp2WRdTx2yX66OsPFeMTtwirJ0ZlJt6mhKCbLNkHq/t+bW+panN0vSmX3PexV0Z3aqHO31y8BX0tMlgaVYHN1tGbrWxTGy9VsHBNC5QjN8gGDx/v37gzh7T+4SWSmPCRFdgMGzjJRqilIMrcCj7tJal6fOMPLWz67f0sgiFvDlBFsK9/VMXUs9KH7y6dOn4/l+6vNpADVm3ZgC/HCtlop/evyV1LzgeXGeifGJ5uawkVx54zNVbNeiRJOw2e5eywmGPcfR/wsDFDPyORiH7+LEkR77HVVyc6+0aPZOz2YJ47rdjqQEBCRXT9roKP9pUkE4sYnbzq7orDRKrG7553XXiKEXWpklbvV5QgikSUAPSqzuteFV4mpLZI7WZzsOHfDSo9pUcF8lNtNBVVQ7ro+Pj1c+AVVFMLlVblfuZmB1PD19VDbVL9aGbslSIK9YWbtqOj6UrYowjafcFpNk0ijMxMy6EL7DolvIlji3ukjmAlPafQKbLaR7LkZXY690Efu7mTlwmpzwxElankrh63g40wKpzWQ7TqWj02UnhjWPselOBOvXErNYUukN5RqVflFH6G85s2ajNRTjV9aL6uYJVUbjoIC6OZtgdhsqUwfpXOekOo6T18+oXDi+4lXtVNOf6m35UGutI+s3d+ZAF5tefvny5bc165jMT3cWmGP8zpaVxlRox+4O4ylvbq1L2djrnU6ng9Ffx27OYMwqkgbmW3LXgYk8ohFFEY1EKBfF3Xh9fT2ejDIjPQUuZiDFplxTaXSselKjyawy+F9WUGyje//mViOKwRBbVswsseXXWut4vJRxz6hnUaTOFH9un/H3kyxZbliFgvt9FeRY3H8uHLPpeH2ngGOcLX163WYkrssJdMNvQdkaNoJmx8oZTFwVQdI93GJxtwRTKchO/N3r6+txmqsdIBXzXJih1ICW8w1Eri87m9kzqoK/t47deQE68Hu24nvQkjzhuxia+Wwi0GeF6sR6v2stEJav1Yqnc9yDBa2byquMAtl2pT9bO+vjdJY6p8fHx6NTbF4bvP4kdx3Y169fj6cvW7SfPy/PlTufz1el4vPz87Gx+XS6nEw6Faf1usylXJO11pUB+FlG1zT/VtuZABpFD4aCXzRb6i2HvJbRzQd+yjREnVuM4ToIDxj1fjtexu5fDWhiDBShStHsdirgFMqx1uU4ms+fP19lmGtdIvOPHz8OHlWNoDwr+IimAydJ2UV1D2oo49q/W91mv1vr+uRS9wUzEa0LbPvshw8fjuZSsTvS7mY77cbZUk22KGNyb5NsKavgpIxZ6Vs9QpxlU+6z5E/jlpF67T45Jfrfknhua2NvBdEFVutaUi9MsHuPbZcri6D67poyqep4qRddV0nKXJPy0m7JXQf27du39fDwsM7n85FhUc46GHvATqfTkX6SpoETMKQka13jHiazxluQeCpL32+DgAF1LI3oLc8YooXivIppMNouyFrXWRBFkll8/Pjxt9RaZlgntNY1A7ugaTOyGvVa66qEmRmDwNKNsy2lv3//fnS5ZplWjIfzZrxVym79KYF1rUvTgMOxZWRmVda75b6svtiJzLkNgkodZiEHY+HsbzkynzVfXZMaXrdszSdZtXSuUddOXNdr9tHxmG/jnlibNTLnhVTa2Zx0Jusxn+LknDSZM5IqG7Fesmx8Os6+2XG5kYJlEyCwSwN+78c9dy7vUb3+l8jqkD9ZCQXWPaSQf3IkQGzYRd+D3yhbHh4ejpttK306CotRRfD+09PTVaQooCtFLjdrRm6L1izMvdjA2jFRhCoykWY/Pz9f7cG0xaN438vLy/EUdMfsKiPm9TrP82EOa13O6xKJi7ettQ6nVWOGSRBOquuw1mWfJYPvxnqf5zjMi5KBg5DFtQtWPpPrNNtop7mvZ0NC4AJPwPVkEsUkhCUAAACcSURBVNayTmHumGBgpXS4N8bbkpqT4PS7D7I0iH737OS2IYMKMaXZlLUrp8sc/Pr1XzOpeB2HVmjn3bt3h7NCYtYcO5/PR0LSTeASlzbvev+lQ1m/VhtK3Y5pYpFz29Y9uUtk3bJly5a/We5mYFu2bNnyN8t2YFu2bHmzsh3Yli1b3qxsB7Zly5Y3K9uBbdmy5c3KdmBbtmx5s/IvSjj13Upl9AQAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "e = laplacian_edge_detector(im)\n", + "show_edges(e)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The edges are more subtle but meanwhile showing small zigzag structures that may be affected by noise. However, the overall performance of edge extracting is still promising." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter24/Image Segmentation.ipynb b/notebooks/chapter24/Image Segmentation.ipynb new file mode 100644 index 000000000..d0a8b36af --- /dev/null +++ b/notebooks/chapter24/Image Segmentation.ipynb @@ -0,0 +1,480 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Segmentation\n", + "\n", + "Image segmentation is another early as well as an important image processing task. Segmentation is the process of breaking an image into groups, based on similarities of the pixels. Pixels can be similar to each other in multiple ways like brightness, color, or texture. The segmentation algorithms are to find a partition of the image into sets of similar pixels which usually indicating objects or certain scenes in an image.\n", + "\n", + "The segmentations in this chapter can be categorized into two complementary ways: one focussing on detecting the boundaries of these groups, and the other on detecting the groups themselves, typically called regions. We will introduce some principles of some algorithms in this notebook to present the basic ideas in segmentation." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Probability Boundary Detection\n", + "\n", + "A boundary curve passing through a pixel $(x,y)$ in an image will have an orientation $\\theta$, so we can formulize boundary detection problem as a classification problem. Based on features from a local neighborhood, we want to compute the probability $P_b(x,y,\\theta)$ that indeed there is a boundary curve at that pixel along that orientation. \n", + "\n", + "One of the sampling ways to calculate $P_b(x,y,\\theta)$ is to generate a series sub-divided into two half disks by a diameter oriented at θ. If there is a boundary at (x, y, θ) the two half disks might be expected to differ significantly in their brightness, color, and texture. For detailed proof of this algorithm, please refer to this [article](https://people.eecs.berkeley.edu/~malik/papers/MFM-boundaries.pdf)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "We implemented a simple demonstration of probability boundary detector as `probability_contour_detection` in `perception.py`. This method takes three inputs:\n", + "\n", + "- image: an image already transformed into the type of numpy ndarray.\n", + "- discs: a list of sub-divided discs.\n", + "- threshold: the standard to tell whether the difference between intensities of two discs implying there is a boundary passing the current pixel.\n", + "\n", + "we also provide a helper function `gen_discs` to gen a list of discs. It takes `scales` as the number of sizes of discs will be generated which is default 1. Please note that for each scale size, there will be 8 sub discs generated which are in the horizontal, verticle and two diagnose directions. Another `init_scale` indicates the starting scale size. For instance, if we use `init_scale` of 10 and `scales` of 2, then scales of sizes of 10 and 20 will be generated and thus we will have 16 sub-divided scales." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's demonstrate the inner mechanism with our navie implementation of the algorithm. First, let's generate some very simple test images. We already generated a grayscale image with only three steps of gray scales in `perceptron.py`:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from perception4e import *\n", + "from notebook4e import *\n", + "import matplotlib.pyplot as plt" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's take a look at it:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAC7UlEQVR4nO3YMYrEMBAAwdViPVsvd6DLD68vOdYdVIWaZDA0Ax577xfQ8356AeCaOCFKnBAlTogSJ0Qdd8Mxhl+5N9ZaT6+Q5xv9bc45rt5dTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBB13A3XWt/aA/jF5YQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixt774/A8z89D4F/MOcfVu8sJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosbe++kdgAsuJ0SJE6LECVHihChxQpQ4IeoHudIUPgvLLmwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.imshow(gray_scale_image, cmap='gray', vmin=0, vmax=255)\n", + "plt.axis('off')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also generate your own grayscale images by calling `gen_gray_scale_picture` and pass the image size and grayscale levels needed:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADB0lEQVR4nO3YMYrEMBAAQesw+Ff6/5c20uWL11xweNtQFWqSSZoBjbXWBvT8fHsB4Jw4IUqcECVOiBInRO1XwzHGo75y55zfXuHPnrTrtj1r3yftum3bdhzHOHt3OSFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6I2q+Gc8679gDeuJwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiNqvhnPOu/YA3ricECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0TtV8M55117AG9cTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEjbXWx+Hr9fo8BP7FcRzj7N3lhChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFqrLW+vQNwwuWEKHFClDghSpwQJU6IEidE/QKgERT5nIvuXQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "gray_img = gen_gray_scale_picture(100, 5)\n", + "plt.imshow(gray_img, cmap='gray', vmin=0, vmax=255)\n", + "plt.axis('off')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's generate the discs we are going to use as sampling masks to tell the intensity difference between two half of the care area of an image. We can generate the discs of size 100 pixels and show them:" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABJCAYAAAA5f/zBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAGTElEQVR4nO3c3XLbOgyF0ehM3v+V3YtTTz0JKVsiSOwNfuu2nRjCD4m6do7H4/EFAABQ2X/ZAQAAAMzGwgMAAMpj4QEAAOWx8AAAgPJYeAAAQHnfZ394HMdWX+F6PB7Hp3+X3JwjP33kpo/c9JGbc+Snj9z873ThwXqjvybgOC6dEaV8krvK+dn9+XEPZ8458lMHC4+AyN+F9PqzGLTfquWH36OFOzhzzpGfmlh4Es2+rJ4/nyFrc84Piw7u4Mw5R35qY+FJsPqyYsjOOeWHRQd3cOacIz97YOFZKPuyYsjOKecnu3fgKbtvlGfq64v87IavpS+SPVivlGJRpJYftXjgQalvlGJ5UopJKZbKWHgWUGxmxZiUqORHJQ54UewbpZiUYnlSjKkaFp7JlJtYOTYF2fnJfn14Uu4bhdgUYuhRjq0CFp6JHJrXIcZMWfmhLn3kps8hN5kxkh9vo7lh4ZnEqWmdYs2Q9Q0O/PbMDTlqc/nwa0b9XHrGpYarRcw+C88ELoP1yjHmlVblhzr0/cwNuWpzuTBX1s+lV1xqt1rU7LPwBHMZrBbn2FdY9UvJ8FsvN+SszeXiXFE/lx5xqdlqkbPPwhPIZbDOVHiGmWblh7z3vcsNuWtzuUBn1s+lN1xqtVr07LPwALDncrGtxkWqjxq1zZhpFp4glQ7cSs8yQ3R+yHffldyQxzaHC3VG7Rz6waE2GWbNPQsPgDIcLrkMXKx6qEnbzBlm4QlQ8ZCt+EyRovJDnvvu5oactqlfsJF1U+8B9VpkmT3zLDwAylG/8LJw0eajBm0rZpaFZ1Dlg7Xys0UYzQ/57YvIDfltU75wq9ddOfeZVtWdhQdAWcqXXyYu3vXIedvKGWXhAVAaS08bF/A65Lpt9Wyy8AzY4SDd4RlH8MHaeLt+RTmD4kU8UivFOivmWEHGnLPwANiC4mWogAt5HnLbljWLLDwAtsHS08bFHI+ctmXOIAsPgK2w9LRxQcchl23Zs8fCA2A72QevKi7qceSwTWHmWHgAbEnhAFbEhX0fuWtTmTUWHgDbUjmI1XBxX0fO2pRm7PvdX1AKdiaaFdjT4/Fg/huO49jm/B9F/7Sp9Q/v8ADYntrBrIKL/D1y1KY4U2/f4UEfjQ7UwTs9bbzT00e/tKn2C+/wAMBfqgd1Ni7238hJm/IMsfAAwAvlAzsTF/w/5KJNfXZYeADgB/WDOwsXPTnocZgZFh4AaHA4wDPsfOHv/OxnXGaFhQcAOlwO8tV2vPh3fOZPOM0ICw8AnHA60FfaaQHY6VmvcJuNtwvPDoXe4Rkxx93eoef6FHPjdrCvMqNWIz9TLZ7KFGfiXa14hwcAPqB4wCuovBBUfrYRrrPAwgMAH3I96GeruBhUfKYIzjPw0cJTufCVnw1zjfYOvdennBvnA3+miJpV+hkVKff+JzXjHR4AuEj54M9UYVGo8AwzVOj5jxeeik1Q8ZmwRlTv0IN96rmpcAHMoPBBfoUYKlHv9U/rxjs8AHCT+kWQxXFxcIx5hUo9fmnhqdQQlZ4Fa0X3Dr3Y55CbShdCpCu1y/46uUOfZXDo7Su14x0eABjkcDFkcFgkHGLMULGnLy88FZqjwjMgx6zeoSf7XHJT8YKI8K5+M+ub+drOXHr5av1uvcPj3CTOsSPX7N6hN/tccuNyUazWq9+Kuma+tiOXHr5Tv9v/peXYLI4xQ8Oq3qFH+1xy43JhrPazfivrmfnaTlx69279hj7D49Q0TrFCy+reoVf7XHLjcnGs9qxfRh0zX9uBS8+O1G/4Q8sOzeMQIzRl9Q492+eQG4cYs2Tmhrr0OeRmNMaQb2kpJ0o5NmjL7p3s11emnBvl2IAzyr0bEVvY19IVE6UYEzyo9I5KHIoUc6MYE3CFYg9HxRT6e3iUEqUUC7yo9Y5aPEqUcqMUCzBCqZcjY/kO+0l/PYPL+gCUUqHgRbl3sudKWXZulPsGuKviXIUvPE+rk8Whg7uceif7EFLGmQPEqzRX0xaep9nJ4tDBXc69w+LTx5kDxKswV9MXnqfXhxlNGAcO7qrWO5FzVQ1nDhDPea6WLTyvODwww+59tfvznyE3QDy3uTr4VyEAAKgu9GvpAAAAilh4AABAeSw8AACgPBYeAABQHgsPAAAoj4UHAACU9wfXBOrVVvZtlAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "discs = gen_discs(100, 1)\n", + "fig=plt.figure(figsize=(10, 10))\n", + "for i in range(8):\n", + " img = discs[0][i]\n", + " fig.add_subplot(1, 8, i+1)\n", + " plt.axis('off')\n", + " plt.imshow(img, cmap='gray', vmin=0, vmax=255)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The white part of disc images is of value 1 while dark places are of value 0. Thus convolving the half-disc image with the corresponding area of an image will yield only half of its content. Of course, discs of size 100 is too large for an image of the same size. We will use discs of size 10 and pass them to the detector." + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [], + "source": [ + "discs = gen_discs(10, 1)\n", + "contours = probability_contour_detection(gray_img, discs[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADOElEQVR4nO3dwUrDUBRF0V7x/3/5ORaKKL2SHV1rqBDewN1TJG3mnPMAet6uPgDwnDghSpwQJU6IEidEvX/1y5nxr1z4ZeecefZzywlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihKgvb98reuXD4TNP75L6VXc676sfvL/Tea/4W/gpywlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlRa9/4/uq3hQOfWU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVFrN76XzczVR4Afs5wQtbac1gl2WU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlRaw8yOudsXQp4WE7IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVFrN76zZ2auPgIBlhOi1pbTqz3sspwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidErT0r5ZyzdSngYTkhS5wQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0St3fjO/zQzVx/hz7KcECVOiFp7W+vtDeyynBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0StPSvlnLN1KeBhOSFLnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRK3d+A53MDNXH+HbLCdErS3nnV6R4A4sJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTouacc/UZgCcsJ0SJE6LECVHihChxQpQ4IeoDHL0j76PZjhkAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "show_edges(contours)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we are using discs of size 10 and some boundary conditions are not dealt with in our naive algorithm, the extracted contour has a bold edge with missings near the image border. But the main structures of contours are extracted correctly which shows the ability of this algorithm." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Group Contour Detection\n", + "\n", + "The alternative approach is based on trying to “cluster” the pixels into regions based on their brightness, color and texture properties. There are multiple grouping algorithms and the simplest and the most popular one is k-means clustering. Basically, the k-means algorithm starts with k randomly selected centroids, which are used as the beginning points for every cluster, and then performs iterative calculations to optimize the positions of the centroids. For a detailed description, please refer to the chapter of unsupervised learning." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "Here we will use the module of `cv2` to perform K-means clustering and show the image. To use it you need to have `opencv-python` pre-installed. Using `cv2.kmeans` is quite simple, you only need to specify the input image and the characters of cluster initialization. Here we use modules provide by `cv2` to initialize the clusters. `cv2.KMEANS_RANDOM_CENTERS` can randomly generate centers of clusters and the cluster number is defined by the user.\n", + "\n", + "`kmeans` method will return the centers and labels of clusters, which can be used to classify pixels of an image. Let's try this algorithm again on the small grayscale image we imported:" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [], + "source": [ + "contours = group_contour_detection(gray_scale_image, 3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's show the extracted contours:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAC7UlEQVR4nO3YMYrEMBAAwdViPVsvd6DLD68vOdYdVIWaZDA0Ax577xfQ8356AeCaOCFKnBAlTogSJ0Qdd8Mxhl+5N9ZaT6+Q5xv9bc45rt5dTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBB13A3XWt/aA/jF5YQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixt774/A8z89D4F/MOcfVu8sJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosbe++kdgAsuJ0SJE6LECVHihChxQpQ4IeoHudIUPgvLLmwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "show_edges(contours)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is not obvious as our generated image already has very clear boundaries. Let's apply the algorithm on the stapler example to see whether it will be more obvious:" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.image as mpimg\n", + "\n", + "stapler_img = mpimg.imread('images/stapler.png', format=\"gray\")" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAKD0lEQVR4nO3dzY7jxBoG4M/OT0ehadEtMSA23AA3wJ7bYM91cFlsEVtYwwKxGjFCYgNNupPYPotR+VQ8TrpnOEec7+h5pNE4dlXZLjuvWnLK1QzDEAAZtf/0AQC8KwEGpCXAgLQEGJCWAAPSWl7a+Pnnnw9fffVVvHjxIlarVWw2m4iIWK/X0TRNrFaraJomFotFtG07LjdNM/4rnyMiFotFRMRYNiKiaZpo23ZcrtcXpV5Rl6mfotZ1Lq17rrkntMMwnLQ5/fy2bZdzOPc0uO/7N+r1fT/21S+//BK73S5evHgRt7e3sdlsYrlcjteo9G3XdXE8Hsf6P//8c3z77bfxww8/RNu28eeff8bNzU18+eWX8emnn8ZmsxmvU9d18fDwMLYVESfL5VhKX9T9Ua+f1qn7oOu6eHx8HLd3XRd935/0S9/30XXd2F9d10Xbtid9VO+jOBwOF67Em+dSznt6HtPl+l5v2/akXr2tvn/Lvuo+Kev6vh/Pp/6OnDO9d0p/1f/6vj/px3P3WdlXqVP3SX1c0zam16c+l6n6+Obq1nVub2/ju+++i6+//jq+//77GIZhtjP8BQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSGt5ceNyGXd3d/Hxxx/HarWKq6uraJomFotFtG0bi8UiIiLa9nUONk0zbq8Nw3Cy/TmGYYhhGKJpmjgej2P9uTbrz3P6vo++799ou17X9/24vuu6N8rO7aO0UR/LtO5c+9N26zbK+vo46rL1uoiI/X4fx+Mx/vjjj/G6nOvjuu7hcIiPPvoovvjii3H/TdPEy5cv49WrV2M7TdNE27bR9/14nacuXdO5a1ZbLBaz16OuV9835d7qui72+300TRPr9TqWy+V4rFNzfV/W1+au83S51KmvU7lWZT/1/TbdR8Tp92WxWMRisRjPr23bcXv5v2wv5cvytN8fHh6i67pYLBaxWq1isVhE3/exWq3i/fffj4eHh9hut7HdbuNwOMT19XUsl8uT4yz7LN+53W4X9/f38fj4GPf39zEMQyyXy/G69X0fx+NxLD/Xh03TnP0Oneujm5ubePny5ey2k768uBXgf5gAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQak9eS8kGWuwdVqNc5ZV8/7WLaV+ehqZT64sr7MoVfmnuv7fnZex+mcj6Wdeq7IS/NB1nMAlv2U9WVuxDJ3X2m767qT+QnrOf7qef8Oh0P0fT+2czgcTuYLLO1M56Ks+2t6bufOY3pOdfl6fdu2MQzDODffuXJ1P5R5HpfL5Wy/lnJN05zMCTk3B+Rz5/qcO6bj8Xgyb+N0PsXp3It1ufK57vO55bLves7Mslzu53p7mSuxzNlY1tffgdJ35f5frVax2WzGuRtL2Xqe1HI8x+NxnJvzvffei+12e3JPlfOur9V0XsVy3qVO13Wx2+0i4vU9eTwex7kyy7q7u7v49ddf48cff4yffvop1ut1XF1dRd/3cXV1FcvlMtbrdVxfX8eHH344HvdqtYrlchm3t7cn16Rpmnj16lXs9/uTeTHr4yzXspQ/t32q7/v466+/nvxe+AsMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaV0cSnQ8Hk+G0JQhKxFxMhzocDhExL+HntRDhyJiHO5QhqrMDYuZW54OO5gbQjQ3FGc6JKYcZxkmUobf1EMYyrnVw41Ku3PDH8rnevhE0zSzw0HK/qdtleXpEIzpcdRl63UREfv9Po7H4xvDXubUdQ+Hw1i3HuqxWq1O+rH0TRlO9NTQjqnp0KSpeihTfT2m90P5XA/L2e/30TRNrNfrcdhMuS9rc31f1tfmrvN0uQwRK/uvr1U9/Gw6JKpWD8sq163u67K9/F+2l/JleXqdHx4exiF95Tr2fT8Odfr999/jgw8+iE8++SQ+++yzuL6+HodNleOsv9cREbvdLu7v7+Px8THu7+9jGIZxiOEwDLHdbmO9Xo/D2Ob68NwQovr6TN3c3MR2u31ymJq/wIC0BBiQlgAD0hJgQFoCDEhLgAFpPflG1vI2xvpR9dT0Meh+vz/ZVh6xTt9yWuqVN1TOvXmz/hwRJ4/cp2+nnL6JdfpWy7p+WX7uG0an6+qfFlyqd2n925ap1W9QLY/M534CUj/CLusiXj+a32w24/rpY/Tp8ZTH7X/n2OfeQjt9Y2p9nPVPR+Z+ZlKO+fHx8WQ/l34GUcw9up97xP9UnXdRt1Nfr3Nl5s55emwRpz9XKP1Z3/fl+3vuOzTXZt12+b/8nGO5XI5vpZ3+fGV639Xmrkl9nBGv78PdbueNrMD/LwEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWstLG3/77bf45ptv4u7uLtbrdazX64iIWCwW0TRNtG0bTdOMn5umiYg42Vavb9vTvKzL1Nvrdub+n7YzV2ZuW73fqbl1T7V3rs65en+n3LvUu9Qnz3XpHN/2eP6bhmF4q/J93/9H23tunafKPKeNc8f+3P03TXO27KU2St25ctNjqsvW5af/l3rT9YfDIbque/J8/AUGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpCXAgLQEGJCWAAPSEmBAWgIMSEuAAWkJMCAtAQakJcCAtAQYkJYAA9ISYEBaAgxIS4ABaQkwIC0BBqQlwIC0BBiQlgAD0hJgQFoCDEhLgAFpCTAgLQEGpNUMw/BPHwPAO/EXGJCWAAPSEmBAWgIMSEuAAWkJMCCtfwGmiMjvMosFAAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "contours = group_contour_detection(stapler_img, 5)\n", + "plt.axis('off')\n", + "plt.imshow(contours, cmap=\"gray\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The segmentation is very rough when using only 5 clusters. Adding to the cluster number will increase the degree of subtle of each group thus the whole picture will be more alike the original one:" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAADnCAYAAACZtwrQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAS7UlEQVR4nO3d247jVLPA8fI56QOd6R4BI7gYiSfgSbjg3XgqDlfDJRIaNAyoxTTNdBInsb0v2GVVqpfdGb79iV3S/ye1OvFh2V5xSnF5reVsGAYBgIjyf3sHAOCfIoABCIsABiAsAhiAsAhgAMIq52Z++eWXw9dffy03NzdSFIUsl0vJskzKspQsy6QoCsnzXIqikCzLJM//jodVVYmIjNN0ep7nkmXZ0Z/y83R9O28YhqN1dJp97/myLF9eav7cOlPrz5U5VbY/Dvveb7Pv+3G5169fy+FwkOvra/noo4+kaRrJ81yqqjqq+/1+L33fS9/3MgyD/Pzzz/Ldd9/Jjz/+KHmey3q9lsvLS/nqq6/kxYsXslgsxnrvuk52u91Ylj3GVP1O1UnqM9R5fd/LZrNJ1snUep7u39S+2PL89lPH5T/nuXMtda74c3nqM5+aN3XO2u2n3s+dW6lpdjv23Ertoy9Ll58q35/Htgw7T89LnfbRRx/J999/L9988428evVKhmFIfvj8AgMQFgEMQFgEMABhEcAAhEUAAxAWAQxAWAQwAGERwACERQADEBYBDEBYBDAAYRHAAIRFAAMQFgEMQFgEMABhEcAAhEUAAxAWAQxAWAQwAGERwACERQADEBYBDEBYBDAAYRHAAIRFAAMQFgEMQFgEMABhEcAAhEUAAxAWAQxAWAQwAGERwACEVc7NzPNcVquVrFYrKctSmqYZp+d5LmVZSpZlUhTF0fRhGCTLsvHPzhMRybJMhmEY39vXOt++HoZhLNNOs8tN0eX6vh/X82Xb5ex8/1q3Z1+nyk1to+/7o7rRZfzx6XQtN7Uftn77vpe2bWWz2UjTNJJlmZRl+ajOD4eD9H0vfd+PZfiy+76XruvG/3Zf+r4fX2vZ9r0qiuLR8el/+7ml6lL3+3A4yN3dndzd3ckwDHI4HGQYBtnv9+P+Hw6Ho/qw28jzXLqukyzLpG1bub+/H8vX+snzXIqikLIspaoqqetaiqKQpmlksVhInudS17VUVTUea57nR+fc2dmZFEUhy+VSzs7O5OLiQoqikKIopK5rybJMuq6TpmlkvV7LdruVoijk4eFB2raVuq5luVxK13Wy3+/Hcne73fgZdF03fnb2fPFsPfh69cv7744917quO1rWfkftZ63bs9/b1LZ02txy+pnZcjWm+P3x+AUGICwCGICwCGAAwiKAAQhrNomvNDFrE/KaCLXJejtfRB4lcjVZmOf5mBjWZKq+1uU04a1l2CRgKgHtt6V84t+uo4lqPQ5N/PptpJKhPunpE/SppL8moL3UfJu8TK3j61WT7/ra3jzQZXwC39500WmHw2FMHiubULbrpm6sKHsO+HrwNxHsTR29KfTxxx/L8+fPj5bdbDby+++/y3a7Pdqu3U9N8Nu6qOt6TJTbetC68fVry9TzM89zqapqTPyXZSmLxWJM2Ot7XU6X0RsEVVXJcrmUuq5lsViMNwuqqpKzs7OjRLYm/quqkr7vZbfbSVEUstlspO972e/3MgyDlGU5HlPXdUfzd7vd+Bna+vfHqvVRVdV4vPa7aL+rU+eflmNv1PjlfJ3r/uh5pPNtGRpn5vALDEBYBDAAYRHAAIRFAAMQ1pNJfE26aQtjERkT+CKPE54ij5OidlnbKl7Lty2+bctxTWymEvCp1uyppGJqPZtwTLUanyojlaDUean5NmHpW9bb5L9d1rcsT5VtE+maYNU/baGuyVHfklrLsy277bZsS3x7A2Cz2Rwta5P4epPH150v354H9iaB76GRqjMRkaZp5PPPP5ftdivv3r2TN2/eyPn5+ZjIVpo8L8tSVquVvHjxYmzJr8dub0xst9uxxfvhcJDdbjfWhU4XEWnb9umW4f+b9Ndkv7bur6pq/L9cLseW/rqsXadpmjHZX1WVXFxcjNM0+Z9lmdR1Pd602O/3cnNzc3Rs2+1W/vzzT7m7u5P379+P54fevLHfAT1G/73032l/08ef/6nzNNVjZmoZW5btSTJZ37NzAeD/MQIYgLAIYADCenI0Cm28p435/Hw7AoCdLvJ3LkJzCZobSOWKbM92e+2t+QpdvigK6bruaMQDmydKNay01/RTjU39dix7bW8b2/l999fwvtGeXd7unzYOtev7ffE5B7tfNv+lORubP7D5RR3xQKedn5/Lp59+KsMwyHq9luVyKW3byh9//DHmOXV/drvduD3df9tw2TditvkVne/PEZ8/tftr56caJ+d5Lre3t5Jlmdzc3Bzl77S+1uv1eO6JyHguawNUuz1bzzafaPOB79+/P5qmuTKbW9P3dnSJ1GcvIkd5L82Rae5LG7/qaBdaP7ZRrX6/7H7b89XSc0rzZpq71PPir7/+elTHuo+6vI7MYRsc6zZ9HlTPa22Ias/TVKN0v92575DFLzAAYRHAAIR1Ul9I/1Nef0baywX7s1CbW9gmGPa9bcZgt+F/ztu+k/Yycu6nsr/U0tf+kizVhGKqv5e9LLGXHfYnrh3oz/a38/vmt5HqMyjyuC+klqd93HRa27bS973c39/Ler1OXnb6bevl1nK5lC+++GK8JBIReXh4kIeHh6MmM3affZOH1GW3zvfs5Zz/7Kf+7LI2ZdF1nXz22WdjMwJtomAve/2x6zFut9tHl8Kp81Ev2eq6FhGRq6uro/1Qtg+mvZT1l6Laz1SbLNhLz67rxqYO9nyz56f93umlpH5Oeilq52szjuVyKWVZyv39vdR1ffTd9XVl0ytlWcqvv/4qv/32mzRNM9aFDnAqIvLy5ctHl4X2c/LNM0TkaJpPn9jUyin4BQYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMKabQem7Z50bGr/AFvb3kP5ITB8txi7vG/L5dsU2THCbXce325natgO26bFH1eqC4ntdjO1vH9t2+zYYWLscDm2bY+fr226bDcYbRdku7XYNly+PkVk7Lriu+D47lR2v225+tlqGfY5CNrmJ9W2a2p6qs51G7YOm6YZu9/sdrtH5fh9KYpi7GJzfX09TrPPaPDnn/+cfFur1Bj59lzQ19p16Kkx9W2d2PrTYXD0vX0Iri6batdo2xra88h/jre3t7Jer6UoCjk7O5M8z2Wz2chms5EXL15I0zSyWq3k5uZG9vu9fPLJJ+N4/LbNpbXb7aRtW7m7u5NffvlF9vu9ZFkmi8VC2raVV69eHZ2Pvv3dKedFqv3l+fm5vH79Otkly+IXGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGIKzZztz2uX6247Z2yrWdfXVg/1RHY9tJNfVcONuZVKfpevpcOv+AA7u+f/1UB27bkVdp52rf8deWqQ9t0H3VZ+rZB0bYZWxnaV+vnn+oh11WO/2mlk11mPXb0M/EPpDEdty1DyTR/1MP1vBlpjrnTx2j3X+l9ec7A6c+B9up2XaA1//60Iyph2Ho67Isj87jpmmOHoShD76o63r8y/Nclsvl+DCNqQfb2A7X9rzQZ2tqx3U9dy4vL+Xq6mp82Id9EIivh9Tnbev/+vr60TTddlVV8vz5c3n37p389NNP8sMPP8hisZDz8/Pxc9NjWS6Xslqtxmdf6jmhdSfy93leVZUsl8vxO+0fxDH13bTTUt9TERmfjTl3HonwCwxAYAQwAGERwACERQADEBYBDEBYBDAAYc02oxCR5Djjdlz21Ljrerte17PL6O1Y32xgalz8w+Ewjolub0/7phT2Vr4u45sJ+Nva9ta9bfLgx1E/HA5Hf13XjWODp8bo1/2wfFMG22RhalzwKalmC36b/nOx9ZB67oBl6zfVfMKv65tT2O2fsv96a74sy0dNIGwzCv/Z+WYtqe2m9sGPh9+27dEY89qEYb/fj2P16/Z0G3meS9M0kmWZLJdLqapKrq6upCxLubi4kLOzM1ksFtI0jdR1PY5Tb+tAz1H9X9e1lGU5ju9vm7P4seb1PGzbVvq+l/V6LXd3dzIMg7x//17atpWmaaSqqrGJyLNnz+Tt27fy5s0bOT8/l6IoxmNaLpdSlqVcXl7K+fm5rFYrOT8/l4uLC1ksFlJVlYjIuN3tdiuHw0Fev34tm81G2rZ9VLe2/lPNJVLfYZ1mn+0wh19gAMIigAEIiwAGIKwnc2BK8zZT3Ux8twA7X3MXU/kUm0fy3YU0R+BzbpoT0PX0Wl7ZfNd+vx9zHLqcdj/R9bXbxOFwOHoWnXb78Dkln8uy9TRVf/a/z+fM5ZpS023OwNaZ/Sx8GTZnZbdv5/nc11SuQted6tKk7/V8mepS5bsiTeVMtCybu0zlX1P16PfT5mDtf7tOqpuLzZHa/dVz7fb29ihnquecyPEzIDWfVFXVmAMsy1KqqpK6rsd8lObELi4ujvLQ9hj1GZlXV1fy7NmzZD1q96e6ruXly5fjPqae4WiPXbs/bbfbR/lnXeby8lKyLJP1ej12J/Kfoa1f/1mkcpZT9Z/CLzAAYRHAAIRFAAMQFgEMQFgEMABhnXwXMtXa3LZ0t3fAUncKbTk6veu6o7tU/g6kvWOhr/1Ab7otvato73bYafau23a7PbrraO8q+Ttu9s6d3yd7rKk7RKn3tmx7x0/vjKXubOoyuo6/e+TvBqfuIPq7hXN3Ue1AfbYHRepukd1uap6/I5aqj6k7TqneDEVRHA0a6O8uTpXh98/++X2auiNpe6XM3Tn257HdBz0X7Tmvd/l0UMP9fj+2sNdpuv26rqVpGrm6uhrvWjZNI4vFQlar1aPW97q/2+12HJQxVQ9d10nXdbLZbGS320nbtnI4HOTh4UE2m40cDgdp21YeHh7G/b2+vpabmxsZhmHsTZGqD/ve98xJ3QmduxPt8QsMQFgEMABhEcAAhDWbA9NraJvf0ryVyHEr7q7rxock2F77Nhem7LW3lmOn2Ycb2DyWbfVs81o63edGbJk2P6br6D7q+6kRGOw0+9+3yp9b76lpOj2Vx/HrPbW9U/I6Nm9l5/nt2/e27FR+46nRA07JNT3F5uhSeTlfbur93LJ+ms+z6b7OHWuqbmy92ZEWpsqa2i+fT9PRMu7v7+Xt27fJnh36vdCHk9iRV1J1Zb/zdp8136Z1b3Nsc9+dqenaS8Dug+aYbR52Dr/AAIRFAAMQFgEMQFgEMABhEcAAhEUAAxAWAQxAWAQwAGERwACERQADEBYBDEBYBDAAYRHAAIRFAAMQFgEMQFgEMABhEcAAhDU7ImvXdUdPSPEjgeqojjpKqh0pVUeG9COo2hFS7dOJRB6PkqrsSKT63z/pyK/ny/AjP6aWOXXeU8uf+mSif7rNqdFYfTn+SVFTZdn1pp42NDfS5lP7k9q31PbnRlJ9alTVuRFdp55MNLWtU6b7UYY/ZL7d36mRcT9kP/36H3IeTZk6h/0IrPo0Io0P9vWpdWT33z756ZTj4BcYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiCs2QEN7+/v5dtvv5XVaiVlWUpd1+OgY/qng44VRTEOoOf/RB4PsuenaRmeDpI2N2Bgqqy5eXPLnDr91Pn/dNkPWd4OKJgapO6pcuYGJHyq3u30Dz2+U9eZOzY1NyDhKeufMv8/WfeUsv9pGU8d+4cc19w6/jyxy9gBQ/U7m2IHOZwaYFIHPD1lv/kFBiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGIKxybuZ2u5Xb21vZ7XZSFIU0TSPDMEhRFONflmWSZZkURSF5nkuWZZLnf8fFPM/Haf5PRMb/RVGM7/18/zrLMhmGYZyu7Htfvu7PKeyyfht2P1L7MLeOemrf/xv+Sfkfuo4uP1Uv/6ZhGGbn6ef5f1Xmf2tdW8enTJ/blp+u7/30vu8n98fOs+ulXtv/U699mdvt9sm64hcYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgLAIYgLAIYADCIoABCIsABiAsAhiAsAhgAMIigAEIiwAGICwCGICwCGAAwiKAAQiLAAYgrOw/eUAnAPyb+AUGICwCGICwCGAAwiKAAQiLAAYgLAIYgLD+Bz86e+p63iwyAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "contours = group_contour_detection(stapler_img, 15)\n", + "plt.axis('off')\n", + "plt.imshow(contours, cmap=\"gray\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Minimum Cut Segmentation\n", + "\n", + "Another way to do clustering is by applying the minimum cut algorithm in graph theory. Roughly speaking, the criterion for partitioning the graph is to minimize the sum of weights of connections across the groups and maximize the sum of weights of connections within the groups.\n", + "\n", + "### Implementation\n", + "\n", + "There are several kinds of representations of a graph such as a matrix or an adjacent list. Here we are using a util function `image_to_graph` to convert an image in ndarray type to an adjacent list. It is integrated into the class of `Graph`. `Graph` takes an image as input and offer the following implementations of some graph theory algorithms:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- bfs: performing bread searches from a source vertex to a terminal vertex. Return `True` if there is a path between the two nodes else return `False`.\n", + "\n", + "- min_cut: performing minimum cut on a graph from a source vertex to sink vertex. The method will return the edges to be cut.\n", + "\n", + "Now let's try the minimum cut method on a simple generated grayscale image of size 10:" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAC3ElEQVR4nO3YMQrDMBAAwSjY//+vGqUPwpAmXsNMqWuuWQ401lovoOd99wLAnjghSpwQJU6IEidEHVfDMYav3IeZc969Aj86z3Ps3l1OiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlRx9VwzvmvPYAvLidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihKix1rp7B2DD5YQocUKUOCFKnBAlTogSJ0R9AF+CDrluZqs6AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "image = gen_gray_scale_picture(size=10, level=2)\n", + "show_edges(image)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[((0, 4), (0, 5)),\n", + " ((1, 4), (1, 5)),\n", + " ((2, 4), (2, 5)),\n", + " ((3, 4), (3, 5)),\n", + " ((4, 0), (5, 0)),\n", + " ((4, 1), (5, 1)),\n", + " ((4, 2), (5, 2)),\n", + " ((4, 3), (5, 3)),\n", + " ((4, 4), (5, 4)),\n", + " ((4, 4), (4, 5))]" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "graph = Graph(image)\n", + "graph.min_cut((0,0), (9,9))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are ten edges to be cut. By cutting the ten edges, we can separate the pictures into two parts by the pixel intensities." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter24/Objects in Images.ipynb b/notebooks/chapter24/Objects in Images.ipynb new file mode 100644 index 000000000..03fc92235 --- /dev/null +++ b/notebooks/chapter24/Objects in Images.ipynb @@ -0,0 +1,454 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Objects in Images\n", + "\n", + "There are two key problems shaping all thinking about objects in images: image classification and object detection. They are much more complicated than the problems like boundary detection. Thus more complicated models are needed to deal with the problems even challenging to human's eyes. For the image classification problem, we use a convolutional neural network to extract patterns of an image. For the case of object detection, we use Recursive CNN, which can assist to find the locations of objects of a set of classes in the image. These two models will be detailly introduced in the following sections." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Image Classification\n", + "\n", + "Image classification is a task where we decide what class an image of a fixed size belongs to. Traditional ways convert grayscale or RGB images into a list of numbers representing the intensity of that pixel and then do classification job on top of this procedure. Currently One of the most popular techniques used in improving the accuracy of traditional image classification ways is Convolutional Neural Networks which is more similar to the principle of human seeing things.\n", + "\n", + "CNN is different from other neural networks in that it has a convolution layer at the beginning. Instead of converting the image to an array of numbers, the image is broken up into some sections by the convolutional kernel, the machine then tries to predict what each section is. Finally, the computer tries to predict what’s in the picture based on the votes of all sections. \n", + "\n", + "A classic CNN would has the following architecture:\n", + "\n", + "$$Input ->Convolution ->ReLU ->Convolution ->ReLU ->Pooling -> ... -> Fully Connected$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "CNNs have an input layer, an output layer, as well as hidden layers. The hidden layers usually consist of convolutional layers, ReLU layers, pooling layers, and fully connected layers. Their functionality can be briefly described as :\n", + "\n", + "- Convolutional layers apply a convolution operation to the input. This layer extracted the features of an image that are used for further processing or classification.\n", + "- Pooling layers combines the outputs of clusters of neurons into a single neuron in the next layer.\n", + "- Fully connected layers connect every neuron in one layer to every neuron in the next layer.\n", + "- RELU layer will apply an elementwise activation function, such as the max(0,x) thresholding at zero.\n", + "\n", + "For a more detailed guidance, please refer to the [course note](http://cs231n.github.io/convolutional-networks/) of Stanford." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Implementation\n", + "\n", + "We implemented a simple CNN with a package of keras which is an advanced level API of TensorFlow. For a more detailed guide, please refer to our previous notebooks or the [official guide](https://keras.io/). The source code can be viewed by importing the necessary packages and executing the following block:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import os, sys\n", + "sys.path = [os.path.abspath(\"../../\")] + sys.path\n", + "from perception4e import *\n", + "from notebook4e import *" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psource(simple_convnet)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `simple_convnet` function takes two inputs and returns a Keras `Sequential` model. The input attributes are the number of hidden layers and the number of output classes. One hidden layer is defined as a pair of convolutional layer and max-pooling layer:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model.add(Conv2D(32, (2, 2), padding='same', kernel_initializer='random_uniform'))\n", + "model.add(MaxPooling2D(padding='same'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The convolution kernel size we used is of size 2x2 and it is initialized by applying random uniform distribution. We also implemented a helper demonstration function `train_model` to show how the convolutional net performs on a certain dataset. This function only takes a CNN model as input and feeds an MNIST dataset into it. The MNIST dataset is split into the training set, validation set and test set by the number of 1000, 100 and 100." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example\n", + "\n", + "Now let's try the simple CNN on the MNIST dataset. For the MNIST dataset, there are totally 10 classes: 0-9. Thus we will build a CNN with 10 prediction classes:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING: Logging before flag parsing goes to stderr.\n", + "W0820 17:50:16.660604 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n", + "\n", + "W0820 17:50:16.847119 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n", + "\n", + "W0820 17:50:16.932054 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n", + "\n", + "W0820 17:50:17.006165 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.\n", + "\n", + "W0820 17:50:17.120162 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", + "\n", + "W0820 17:50:17.130156 4604204480 deprecation_wrapper.py:119] From /Users/tianqiyang/anaconda3/envs/3point6/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead.\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "_________________________________________________________________\n", + "Layer (type) Output Shape Param # \n", + "=================================================================\n", + "conv2d_1 (Conv2D) (None, 1, 28, 32) 3616 \n", + "_________________________________________________________________\n", + "max_pooling2d_1 (MaxPooling2 (None, 1, 14, 32) 0 \n", + "_________________________________________________________________\n", + "conv2d_2 (Conv2D) (None, 1, 14, 32) 4128 \n", + "_________________________________________________________________\n", + "max_pooling2d_2 (MaxPooling2 (None, 1, 7, 32) 0 \n", + "_________________________________________________________________\n", + "conv2d_3 (Conv2D) (None, 1, 7, 32) 4128 \n", + "_________________________________________________________________\n", + "max_pooling2d_3 (MaxPooling2 (None, 1, 4, 32) 0 \n", + "_________________________________________________________________\n", + "flatten_1 (Flatten) (None, 128) 0 \n", + "_________________________________________________________________\n", + "dense_1 (Dense) (None, 10) 1290 \n", + "_________________________________________________________________\n", + "activation_1 (Activation) (None, 10) 0 \n", + "=================================================================\n", + "Total params: 13,162\n", + "Trainable params: 13,162\n", + "Non-trainable params: 0\n", + "_________________________________________________________________\n", + "None\n" + ] + } + ], + "source": [ + "cnn_model = simple_convnet(size=3, num_classes=10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The brief description of the CNN architecture is described as above. Please note that each layer has the number of parameters needs to be trained. More parameters meaning longer to train the network on a dataset. We have 3 convolutional layers and 3 max-pooling layers in total and more than 10000 parameters to train.\n", + "\n", + "Now lets train the model for 5 epochs with the pre-defined training parameters: `epochs=5` and `batch_size=32`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 1000 samples, validate on 100 samples\n", + "Epoch 1/5\n", + " - 0s - loss: 1.9887 - acc: 0.3560 - val_loss: 1.9666 - val_acc: 0.3900\n", + "Epoch 2/5\n", + " - 0s - loss: 1.9144 - acc: 0.3670 - val_loss: 1.8953 - val_acc: 0.4200\n", + "Epoch 3/5\n", + " - 0s - loss: 1.8376 - acc: 0.3920 - val_loss: 1.8257 - val_acc: 0.4200\n", + "Epoch 4/5\n", + " - 0s - loss: 1.7612 - acc: 0.4000 - val_loss: 1.7614 - val_acc: 0.4400\n", + "Epoch 5/5\n", + " - 0s - loss: 1.6921 - acc: 0.4220 - val_loss: 1.7038 - val_acc: 0.4600\n", + "100/100 [==============================] - 0s 36us/step\n", + "[8.314567489624023, 0.47]\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_model(cnn_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Within 5 epochs of training, the model accuracy on the training set improves from 35% to 42% while validation accuracy is improved to 46%. This is still relatively low but much higher than the 10% probability of random guess. To improve the accuracy further, you can try both adding more examples to a dataset such as using 20000 training examples and meanwhile training for more rounds." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Object Detection\n", + "\n", + "An object detection program must mark the locations of each object from a known set of classes in test images. Object detection is hard in many aspects: objects can be in various shapes and sometimes maybe deformed or vague. Objects can appear in an image in any position and they are often mixed up with noisy objects or scenes.\n", + "\n", + "Many object detectors are built out of image classifiers.On top of the classifier, there is an additional task needed for detecting an object: select objects to be classified with windows and report their precise locations. We usually call windows as bounding boxes and there are multiple ways to build it. The very simplest procedure for choosing windows is to use all windows on some grid. Here we will introduce two main procedures of finding a bounding box." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Selective Search\n", + "\n", + "The simplest procedure for building boxes is to slide a window over the image. It produces a large number of boxes, and the boxes themselves ignore important image evidence but it is designed to be fast. \n", + "\n", + "Selective Search starts by over-segmenting the image based on the intensity of the pixels using a graph-based segmentation method. Selective Search algorithm takes these segments as initial input and then add all bounding boxes corresponding to segmented parts to the list of regional proposals. Then the algorithm group adjacent segments based on similarity and continue then go repeat the previous steps.\n", + "\n", + "\n", + "#### Implementation\n", + "\n", + "Here we use the selective search method provided by the `opencv-python` package. To use it, please make sure the additional `opencv-contrib-python` version is also installed. You can create a selective search with the following line of code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ss = cv2.ximgproc.segmentation.createSelectiveSearchSegmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then what to do is to set the input image and selective search mode. Then the model is ready to train:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ss.setBaseImage(im)\n", + "ss.switchToSelectiveSearchQuality()\n", + "rects = ss.process()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The returned `rects` will be the coordinates of the bounding box corners.\n", + "\n", + "#### Example\n", + "\n", + "Here we provided the `selective_search` method to demonstrate the result of the selective search. The method takes a path to the image as input. To execute the demo, please use the following line of code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "image_path = \"./images/stapler.png\"\n", + "selective_search(image_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The bounding boxes are drawn on the original picture showed in the following:\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Some of the bounding boxes do have the stapler or at least most of it in the box, which can assist the classification process." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### R-CNN and Faster R-CNN\n", + "\n", + "[Ross Girshick et al.](https://arxiv.org/pdf/1311.2524.pdf) proposed a method where they use selective search to extract just 2000 regions from the image. Then the regions in bounding boxes are feed into a convolutional neural network to perform classification. The brief architecture can be shown as:\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The problem with R-CNN is that one must pass each box independently through an image classifier thus it takes a huge amount of time to train the network. And meanwhile, the selective search is not that stable and sometimes may generate bad examples.\n", + "\n", + "Faster R-CNN solved the drawbacks of R-CNN by applying a faster object detection algorithm. Instead of feeding the region proposals to the CNN, we feed the input image to the CNN to generate a convolutional feature map. Then we identify the region of interests on the feature map and then reshape them into a fixed size with an ROI pooling layer so it can be put into another classifier. \n", + "\n", + "This algorithm is faster than R-CNN as the image is not frequently fed into the CNN to extract feature maps." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Implementation\n", + "\n", + "For an ROI pooling layer, we implemented a simple demo of it as `pool_rois`. We can fake a simple feature map with `numpy`:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "feature_maps_shape = (200, 100, 1)\n", + "feature_map = np.ones(feature_maps_shape, dtype='float32')\n", + "feature_map[200 - 1, 100 - 3, 0] = 50" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that the fake feature map is all 1 except for one spot with a value of 50. Now let's generate some regio of interests:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "roiss = np.asarray([[0.5, 0.2, 0.7, 0.4], [0.0, 0.0, 1.0, 1.0]])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we only made up two regions of interest. The first only crops some part of the image where all pixels are '1' which ranges from 0.5-0.7 of the length of the horizontal edge and 0.2-0.4 of verticle edge. The range of the second region is the whole image. Now let's pool a 3x7 area out of each region of interest." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[array([[1., 1., 1., 1., 1., 1., 1.],\n", + " [1., 1., 1., 1., 1., 1., 1.],\n", + " [1., 1., 1., 1., 1., 1., 1.]], dtype=float32),\n", + " array([[ 1., 1., 1., 1., 1., 1., 1.],\n", + " [ 1., 1., 1., 1., 1., 1., 1.],\n", + " [ 1., 1., 1., 1., 1., 1., 50.]], dtype=float32)]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pool_rois(feature_map, roiss, 3, 7)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "What we are expecting is that the second pooled region is different from the first one as there is an artificial feature-the '50' in its input. The printed result is exactly the same as we expected." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to try the whole algorithm of the Faster R-CNN, you can refer to [this GitHub repository](https://github.com/endernewton/tf-faster-rcnn) for more detailed guidance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/chapter24/images/RCNN.png b/notebooks/chapter24/images/RCNN.png new file mode 100644 index 000000000..273021fbe Binary files /dev/null and b/notebooks/chapter24/images/RCNN.png differ diff --git a/notebooks/chapter24/images/derivative_of_gaussian.png b/notebooks/chapter24/images/derivative_of_gaussian.png new file mode 100644 index 000000000..0be575529 Binary files /dev/null and b/notebooks/chapter24/images/derivative_of_gaussian.png differ diff --git a/notebooks/chapter24/images/gradients.png b/notebooks/chapter24/images/gradients.png new file mode 100644 index 000000000..ae57bdf3b Binary files /dev/null and b/notebooks/chapter24/images/gradients.png differ diff --git a/notebooks/chapter24/images/laplacian.png b/notebooks/chapter24/images/laplacian.png new file mode 100644 index 000000000..6d7e6916a Binary files /dev/null and b/notebooks/chapter24/images/laplacian.png differ diff --git a/notebooks/chapter24/images/laplacian_kernels.png b/notebooks/chapter24/images/laplacian_kernels.png new file mode 100644 index 000000000..faca3321c Binary files /dev/null and b/notebooks/chapter24/images/laplacian_kernels.png differ diff --git a/notebooks/chapter24/images/stapler.png b/notebooks/chapter24/images/stapler.png new file mode 100644 index 000000000..e550d83f9 Binary files /dev/null and b/notebooks/chapter24/images/stapler.png differ diff --git a/notebooks/chapter24/images/stapler_bbox.png b/notebooks/chapter24/images/stapler_bbox.png new file mode 100644 index 000000000..c5a7c7af0 Binary files /dev/null and b/notebooks/chapter24/images/stapler_bbox.png differ diff --git a/obsolete-search-4e.ipynb b/obsolete_search4e.ipynb similarity index 100% rename from obsolete-search-4e.ipynb rename to obsolete_search4e.ipynb diff --git a/perception4e.py b/perception4e.py new file mode 100644 index 000000000..edd556607 --- /dev/null +++ b/perception4e.py @@ -0,0 +1,467 @@ +"""Perception (Chapter 24)""" + +import cv2 +import keras +import matplotlib.pyplot as plt +import numpy as np +import scipy.signal +from keras.datasets import mnist +from keras.layers import Dense, Activation, Flatten, InputLayer, Conv2D, MaxPooling2D +from keras.models import Sequential + +from utils4e import gaussian_kernel_2D + + +# ____________________________________________________ +# 24.3 Early Image Processing Operators +# 24.3.1 Edge Detection + + +def array_normalization(array, range_min, range_max): + """Normalize an array in the range of (range_min, range_max)""" + if not isinstance(array, np.ndarray): + array = np.asarray(array) + array = array - np.min(array) + array = array * (range_max - range_min) / np.max(array) + range_min + return array + + +def gradient_edge_detector(image): + """ + Image edge detection by calculating gradients in the image + :param image: numpy ndarray or an iterable object + :return: numpy ndarray, representing a gray scale image + """ + if not isinstance(image, np.ndarray): + image = np.asarray(image) + # gradient filters of x and y direction edges + x_filter, y_filter = np.array([[1, -1]]), np.array([[1], [-1]]) + # convolution between filter and image to get edges + y_edges = scipy.signal.convolve2d(image, x_filter, 'same') + x_edges = scipy.signal.convolve2d(image, y_filter, 'same') + edges = array_normalization(x_edges + y_edges, 0, 255) + return edges + + +def gaussian_derivative_edge_detector(image): + """Image edge detector using derivative of gaussian kernels""" + if not isinstance(image, np.ndarray): + image = np.asarray(image) + gaussian_filter = gaussian_kernel_2D() + # init derivative of gaussian filters + x_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1, -1]]), 'same') + y_filter = scipy.signal.convolve2d(gaussian_filter, np.asarray([[1], [-1]]), 'same') + # extract edges using convolution + y_edges = scipy.signal.convolve2d(image, x_filter, 'same') + x_edges = scipy.signal.convolve2d(image, y_filter, 'same') + edges = array_normalization(x_edges + y_edges, 0, 255) + return edges + + +def laplacian_edge_detector(image): + """Extract image edge with laplacian filter""" + if not isinstance(image, np.ndarray): + image = np.asarray(image) + # init laplacian filter + laplacian_kernel = np.asarray([[0, -1, 0], [-1, 4, -1], [0, -1, 0]]) + # extract edges with convolution + edges = scipy.signal.convolve2d(image, laplacian_kernel, 'same') + edges = array_normalization(edges, 0, 255) + return edges + + +def show_edges(edges): + """ helper function to show edges picture""" + plt.imshow(edges, cmap='gray', vmin=0, vmax=255) + plt.axis('off') + plt.show() + + +# __________________________________________________ +# 24.3.3 Optical flow + + +def sum_squared_difference(pic1, pic2): + """SSD of two frames""" + pic1 = np.asarray(pic1) + pic2 = np.asarray(pic2) + assert pic1.shape == pic2.shape + min_ssd = np.inf + min_dxy = (np.inf, np.inf) + + # consider picture shift from -30 to 30 + for Dx in range(-30, 31): + for Dy in range(-30, 31): + # shift the image + shifted_pic = np.roll(pic2, Dx, axis=0) + shifted_pic = np.roll(shifted_pic, Dy, axis=1) + # calculate the difference + diff = np.sum((pic1 - shifted_pic) ** 2) + if diff < min_ssd: + min_dxy = (Dx, Dy) + min_ssd = diff + return min_dxy, min_ssd + + +# ____________________________________________________ +# segmentation + +def gen_gray_scale_picture(size, level=3): + """ + Generate a picture with different gray scale levels + :param size: size of generated picture + :param level: the number of level of gray scales in the picture, + range (0, 255) are equally divided by number of levels + :return image in numpy ndarray type + """ + assert level > 0 + # init an empty image + image = np.zeros((size, size)) + if level == 1: + return image + # draw a square on the left upper corner of the image + for x in range(size): + for y in range(size): + image[x, y] += (250 // (level - 1)) * (max(x, y) * level // size) + return image + + +gray_scale_image = gen_gray_scale_picture(3) + + +def probability_contour_detection(image, discs, threshold=0): + """ + Detect edges/contours by applying a set of discs to an image + :param image: an image in type of numpy ndarray + :param discs: a set of discs/filters to apply to pixels of image + :param threshold: threshold to tell whether the pixel at (x, y) is on an edge + :return image showing edges in numpy ndarray type + """ + # init an empty output image + res = np.zeros(image.shape) + step = discs[0].shape[0] + for x_i in range(0, image.shape[0] - step + 1, 1): + for y_i in range(0, image.shape[1] - step + 1, 1): + diff = [] + # apply each pair of discs and calculate the difference + for d in range(0, len(discs), 2): + disc1, disc2 = discs[d], discs[d + 1] + # crop the region of interest + region = image[x_i: x_i + step, y_i: y_i + step] + diff.append(np.sum(np.multiply(region, disc1)) - np.sum(np.multiply(region, disc2))) + if max(diff) > threshold: + # change color of the center of region + res[x_i + step // 2, y_i + step // 2] = 255 + return res + + +def group_contour_detection(image, cluster_num=2): + """ + Detecting contours in an image with k-means clustering + :param image: an image in numpy ndarray type + :param cluster_num: number of clusters in k-means + """ + img = image + Z = np.float32(img) + criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0) + K = cluster_num + # use kmeans in opencv-python + ret, label, center = cv2.kmeans(Z, K, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS) + center = np.uint8(center) + res = center[label.flatten()] + res2 = res.reshape(img.shape) + # show the image + # cv2.imshow('res2', res2) + # cv2.waitKey(0) + # cv2.destroyAllWindows() + return res2 + + +def image_to_graph(image): + """ + Convert an image to an graph in adjacent matrix form + """ + graph_dict = {} + for x in range(image.shape[0]): + for y in range(image.shape[1]): + graph_dict[(x, y)] = [(x + 1, y) if x + 1 < image.shape[0] else None, + (x, y + 1) if y + 1 < image.shape[1] else None] + return graph_dict + + +def generate_edge_weight(image, v1, v2): + """ + Find edge weight between two vertices in an image + :param image: image in numpy ndarray type + :param v1, v2: verticles in the image in form of (x index, y index) + """ + diff = abs(image[v1[0], v1[1]] - image[v2[0], v2[1]]) + return 255 - diff + + +class Graph: + """Graph in adjacent matrix to represent an image""" + + def __init__(self, image): + """image: ndarray""" + self.graph = image_to_graph(image) + # number of columns and rows + self.ROW = len(self.graph) + self.COL = 2 + self.image = image + # dictionary to save the maximum flow of each edge + self.flow = {} + # initialize the flow + for s in self.graph: + self.flow[s] = {} + for t in self.graph[s]: + if t: + self.flow[s][t] = generate_edge_weight(image, s, t) + + def bfs(self, s, t, parent): + """Breadth first search to tell whether there is an edge between source and sink + parent: a list to save the path between s and t""" + # queue to save the current searching frontier + queue = [s] + visited = [] + + while queue: + u = queue.pop(0) + for node in self.graph[u]: + # only select edge with positive flow + if node not in visited and node and self.flow[u][node] > 0: + queue.append(node) + visited.append(node) + parent.append((u, node)) + return True if t in visited else False + + def min_cut(self, source, sink): + """Find the minimum cut of the graph between source and sink""" + parent = [] + max_flow = 0 + + while self.bfs(source, sink, parent): + path_flow = np.inf + # find the minimum flow of s-t path + for s, t in parent: + path_flow = min(path_flow, self.flow[s][t]) + + max_flow += path_flow + + # update all edges between source and sink + for s in self.flow: + for t in self.flow[s]: + if t[0] <= sink[0] and t[1] <= sink[1]: + self.flow[s][t] -= path_flow + parent = [] + res = [] + for i in self.flow: + for j in self.flow[i]: + if self.flow[i][j] == 0 and generate_edge_weight(self.image, i, j) > 0: + res.append((i, j)) + return res + + +def gen_discs(init_scale, scales=1): + """ + Generate a collection of disc pairs by splitting an round discs with different angles + :param init_scale: the initial size of each half discs + :param scales: scale number of each type of half discs, the scale size will be doubled each time + :return: the collection of generated discs: [discs of scale1, discs of scale2...] + """ + discs = [] + for m in range(scales): + scale = init_scale * (m + 1) + disc = [] + # make the full empty dist + white = np.zeros((scale, scale)) + center = (scale - 1) / 2 + for i in range(scale): + for j in range(scale): + if (i - center) ** 2 + (j - center) ** 2 <= (center ** 2): + white[i, j] = 255 + # generate lower half and upper half + lower_half = np.copy(white) + lower_half[:(scale - 1) // 2, :] = 0 + upper_half = lower_half[::-1, ::-1] + # generate left half and right half + disc += [lower_half, upper_half, np.transpose(lower_half), np.transpose(upper_half)] + # generate upper-left, lower-right, upper-right, lower-left half discs + disc += [np.tril(white, 0), np.triu(white, 0), np.flip(np.tril(white, 0), axis=0), + np.flip(np.triu(white, 0), axis=0)] + discs.append(disc) + return discs + + +# __________________________________________________ +# 24.4 Classifying Images + + +def load_MINST(train_size, val_size, test_size): + """Load MINST dataset from keras""" + (x_train, y_train), (x_test, y_test) = mnist.load_data() + total_size = len(x_train) + if train_size + val_size > total_size: + train_size = total_size - val_size + x_train = x_train.reshape(x_train.shape[0], 1, 28, 28) + x_test = x_test.reshape(x_test.shape[0], 1, 28, 28) + x_train = x_train.astype('float32') + x_train /= 255 + test_x = x_test.astype('float32') + test_x /= 255 + y_train = keras.utils.to_categorical(y_train, 10) + y_test = keras.utils.to_categorical(y_test, 10) + return ((x_train[:train_size], y_train[:train_size]), + (x_train[train_size:train_size + val_size], y_train[train_size:train_size + val_size]), + (x_test[:test_size], y_test[:test_size])) + + +def simple_convnet(size=3, num_classes=10): + """ + Simple convolutional network for digit recognition + :param size: number of convolution layers + :param num_classes: number of output classes + :return a convolution network in keras model type + """ + model = Sequential() + # add input layer for images of size (28, 28) + model.add(InputLayer(input_shape=(1, 28, 28))) + # add convolution layers and max pooling layers + for _ in range(size): + model.add(Conv2D(32, (2, 2), padding='same', kernel_initializer='random_uniform')) + model.add(MaxPooling2D(padding='same')) + + # add flatten layer and output layers + model.add(Flatten()) + model.add(Dense(num_classes)) + model.add(Activation('softmax')) + + # compile model + model.compile(loss='categorical_crossentropy', + metrics=['accuracy']) + print(model.summary()) + return model + + +def train_model(model): + """Train the simple convolution network""" + # load dataset + (train_x, train_y), (val_x, val_y), (test_x, test_y) = load_MINST(1000, 100, 100) + model.fit(train_x, train_y, validation_data=(val_x, val_y), epochs=5, verbose=2, batch_size=32) + scores = model.evaluate(test_x, test_y, verbose=1) + print(scores) + return model + + +# _____________________________________________________ +# 24.5 DETECTING OBJECTS + + +def selective_search(image): + """ + Selective search for object detection + :param image: str, the path of image or image in ndarray type with 3 channels + :return list of bounding boxes, each element is in form of [x_min, y_min, x_max, y_max] + """ + if not image: + im = cv2.imread("./images/stapler1-test.png") + elif isinstance(image, str): + im = cv2.imread(image) + else: + im = np.stack(image * 3, axis=-1) + + # use opencv python to extract bounding box with selective search + ss = cv2.ximgproc.segmentation.createSelectiveSearchSegmentation() + ss.setBaseImage(im) + ss.switchToSelectiveSearchQuality() + rects = ss.process() + + # show bounding boxes with the input image + image_out = im.copy() + for rect in rects[:100]: + print(rect) + x, y, w, h = rect + cv2.rectangle(image_out, (x, y), (x + w, y + h), (0, 255, 0), 1, cv2.LINE_AA) + cv2.imshow("Output", image_out) + cv2.waitKey(0) + return rects + + +# faster RCNN +def pool_rois(feature_map, rois, pooled_height, pooled_width): + """ + Applies ROI pooling for a single image and various ROIs + :param feature_map: ndarray, in shape of (width, height, channel) + :param rois: list of roi + :param pooled_height: height of pooled area + :param pooled_width: width of pooled area + :return list of pooled features + """ + + def curried_pool_roi(roi): + return pool_roi(feature_map, roi, pooled_height, pooled_width) + + pooled_areas = list(map(curried_pool_roi, rois)) + return pooled_areas + + +def pool_roi(feature_map, roi, pooled_height, pooled_width): + """ + Applies a single ROI pooling to a single image + :param feature_map: ndarray, in shape of (width, height, channel) + :param roi: region of interest, in form of [x_min_ratio, y_min_ratio, x_max_ratio, y_max_ratio] + :return feature of pooling output, in shape of (pooled_width, pooled_height) + """ + + # Compute the region of interest + feature_map_height = int(feature_map.shape[0]) + feature_map_width = int(feature_map.shape[1]) + + h_start = int(feature_map_height * roi[0]) + w_start = int(feature_map_width * roi[1]) + h_end = int(feature_map_height * roi[2]) + w_end = int(feature_map_width * roi[3]) + + region = feature_map[h_start:h_end, w_start:w_end, :] + + # Divide the region into non overlapping areas + region_height = h_end - h_start + region_width = w_end - w_start + h_step = region_height // pooled_height + w_step = region_width // pooled_width + + areas = [[( + i * h_step, + j * w_step, + (i + 1) * h_step if i + 1 < pooled_height else region_height, + (j + 1) * w_step if j + 1 < pooled_width else region_width) + for j in range(pooled_width)] + for i in range(pooled_height)] + + # take the maximum of each area and stack the result + def pool_area(x): + return np.max(region[x[0]:x[2], x[1]:x[3], :]) + + pooled_features = np.stack([[pool_area(x) for x in row] for row in areas]) + return pooled_features + +# faster rcnn demo can be installed and shown in jupyter notebook +# def faster_rcnn_demo(directory): +# """ +# show the demo of rcnn, the model is from +# @inproceedings{renNIPS15fasterrcnn, +# Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun}, +# Title = {Faster {R-CNN}: Towards Real-Time Object Detection +# with Region Proposal Networks}, +# Booktitle = {Advances in Neural Information Processing Systems ({NIPS})}, +# Year = {2015}} +# :param directory: the directory where the faster rcnn model is installed +# """ +# os.chdir(directory + '/lib') +# # make file +# os.system("make clean") +# os.system("make") +# # run demo +# os.chdir(directory) +# os.system("./tools/demo.py") +# return 0 diff --git a/planning.py b/planning.py index 1ad91eaf3..1e4a19209 100644 --- a/planning.py +++ b/planning.py @@ -1,13 +1,17 @@ -"""Planning (Chapters 10-11) -""" +"""Planning (Chapters 10-11)""" import copy import itertools +from collections import deque, defaultdict +from functools import reduce as _reduce + +import numpy as np + +import search +from csp import sat_up, NaryCSP, Constraint, ac_search_solver, is_constraint +from logic import FolKB, conjuncts, unify_mm, associate, SAT_plan, cdcl_satisfiable from search import Node from utils import Expr, expr, first -from logic import FolKB, conjuncts, unify -from collections import deque -from functools import reduce as _reduce class PlanningProblem: @@ -17,10 +21,11 @@ class PlanningProblem: The conjunction of these logical statements completely defines a state. """ - def __init__(self, init, goals, actions): - self.init = self.convert(init) + def __init__(self, initial, goals, actions, domain=None): + self.initial = self.convert(initial) if domain is None else self.convert(initial) + self.convert(domain) self.goals = self.convert(goals) self.actions = actions + self.domain = domain def convert(self, clauses): """Converts strings into exprs""" @@ -42,23 +47,122 @@ def convert(self, clauses): new_clauses.append(clause) return new_clauses + def expand_fluents(self, name=None): + + kb = None + if self.domain: + kb = FolKB(self.convert(self.domain)) + for action in self.actions: + if action.precond: + for fests in set(action.precond).union(action.effect).difference(self.convert(action.domain)): + if fests.op[:3] != 'Not': + kb.tell(expr(str(action.domain) + ' ==> ' + str(fests))) + + objects = set(arg for clause in set(self.initial + self.goals) for arg in clause.args) + fluent_list = [] + if name is not None: + for fluent in self.initial + self.goals: + if str(fluent) == name: + fluent_list.append(fluent) + break + else: + fluent_list = list(map(lambda fluent: Expr(fluent[0], *fluent[1]), + {fluent.op: fluent.args for fluent in self.initial + self.goals + + [clause for action in self.actions for clause in action.effect if + clause.op[:3] != 'Not']}.items())) + + expansions = [] + for fluent in fluent_list: + for permutation in itertools.permutations(objects, len(fluent.args)): + new_fluent = Expr(fluent.op, *permutation) + if (self.domain and kb.ask(new_fluent) is not False) or not self.domain: + expansions.append(new_fluent) + + return expansions + + def expand_actions(self, name=None): + """Generate all possible actions with variable bindings for precondition selection heuristic""" + + has_domains = all(action.domain for action in self.actions if action.precond) + kb = None + if has_domains: + kb = FolKB(self.initial) + for action in self.actions: + if action.precond: + kb.tell(expr(str(action.domain) + ' ==> ' + str(action))) + + objects = set(arg for clause in self.initial for arg in clause.args) + expansions = [] + action_list = [] + if name is not None: + for action in self.actions: + if str(action.name) == name: + action_list.append(action) + break + else: + action_list = self.actions + + for action in action_list: + for permutation in itertools.permutations(objects, len(action.args)): + bindings = unify_mm(Expr(action.name, *action.args), Expr(action.name, *permutation)) + if bindings is not None: + new_args = [] + for arg in action.args: + if arg in bindings: + new_args.append(bindings[arg]) + else: + new_args.append(arg) + new_expr = Expr(str(action.name), *new_args) + if (has_domains and kb.ask(new_expr) is not False) or ( + has_domains and not action.precond) or not has_domains: + new_preconds = [] + for precond in action.precond: + new_precond_args = [] + for arg in precond.args: + if arg in bindings: + new_precond_args.append(bindings[arg]) + else: + new_precond_args.append(arg) + new_precond = Expr(str(precond.op), *new_precond_args) + new_preconds.append(new_precond) + new_effects = [] + for effect in action.effect: + new_effect_args = [] + for arg in effect.args: + if arg in bindings: + new_effect_args.append(bindings[arg]) + else: + new_effect_args.append(arg) + new_effect = Expr(str(effect.op), *new_effect_args) + new_effects.append(new_effect) + expansions.append(Action(new_expr, new_preconds, new_effects)) + + return expansions + + def is_strips(self): + """ + Returns True if the problem does not contain negative literals in preconditions and goals + """ + return (all(clause.op[:3] != 'Not' for clause in self.goals) and + all(clause.op[:3] != 'Not' for action in self.actions for clause in action.precond)) + def goal_test(self): """Checks if the goals have been reached""" - return all(goal in self.init for goal in self.goals) + return all(goal in self.initial for goal in self.goals) def act(self, action): """ Performs the action given as argument. Note that action is an Expr like expr('Remove(Glass, Table)') or expr('Eat(Sandwich)') - """ + """ action_name = action.op args = action.args list_action = first(a for a in self.actions if a.name == action_name) if list_action is None: raise Exception("Action '{}' not found".format(action_name)) - if not list_action.check_precond(self.init, args): + if not list_action.check_precond(self.initial, args): raise Exception("Action '{}' pre-conditions not satisfied".format(action)) - self.init = list_action(self.init, args).clauses + self.initial = list_action(self.initial, args).clauses class Action: @@ -74,19 +178,20 @@ class Action: eat = Action(expr("Eat(person, food)"), precond, effect) """ - def __init__(self, action, precond, effect): + def __init__(self, action, precond, effect, domain=None): if isinstance(action, str): action = expr(action) self.name = action.op self.args = action.args - self.precond = self.convert(precond) + self.precond = self.convert(precond) if domain is None else self.convert(precond) + self.convert(domain) self.effect = self.convert(effect) + self.domain = domain def __call__(self, kb, args): return self.act(kb, args) def __repr__(self): - return '{}({})'.format(self.__class__.__name__, Expr(self.name, *self.args)) + return '{}'.format(Expr(self.name, *self.args)) def convert(self, clauses): """Converts strings into Exprs""" @@ -108,6 +213,13 @@ def convert(self, clauses): return clauses + def relaxed(self): + """ + Removes delete list from the action by removing all negative literals from action's effect + """ + return Action(Expr(self.name, *self.args), self.precond, + list(filter(lambda effect: effect.op[:3] != 'Not', self.effect))) + def substitute(self, e, args): """Replaces variables in expression with their respective Propositional symbol""" @@ -146,7 +258,7 @@ def act(self, kb, args): else: new_clause = Expr('Not' + clause.op, *clause.args) - if kb.ask(self.substitute(new_clause, args)) is not False: + if kb.ask(self.substitute(new_clause, args)) is not False: kb.retract(self.substitute(new_clause, args)) return kb @@ -187,21 +299,26 @@ def air_cargo(): >>> """ - return PlanningProblem(init='At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK) & Cargo(C1) & Cargo(C2) & Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)', - goals='At(C1, JFK) & At(C2, SFO)', - actions=[Action('Load(c, p, a)', - precond='At(c, a) & At(p, a) & Cargo(c) & Plane(p) & Airport(a)', - effect='In(c, p) & ~At(c, a)'), - Action('Unload(c, p, a)', - precond='In(c, p) & At(p, a) & Cargo(c) & Plane(p) & Airport(a)', - effect='At(c, a) & ~In(c, p)'), - Action('Fly(p, f, to)', - precond='At(p, f) & Plane(p) & Airport(f) & Airport(to)', - effect='At(p, to) & ~At(p, f)')]) + return PlanningProblem(initial='At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK)', + goals='At(C1, JFK) & At(C2, SFO)', + actions=[Action('Load(c, p, a)', + precond='At(c, a) & At(p, a)', + effect='In(c, p) & ~At(c, a)', + domain='Cargo(c) & Plane(p) & Airport(a)'), + Action('Unload(c, p, a)', + precond='In(c, p) & At(p, a)', + effect='At(c, a) & ~In(c, p)', + domain='Cargo(c) & Plane(p) & Airport(a)'), + Action('Fly(p, f, to)', + precond='At(p, f)', + effect='At(p, to) & ~At(p, f)', + domain='Plane(p) & Airport(f) & Airport(to)')], + domain='Cargo(C1) & Cargo(C2) & Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)') def spare_tire(): - """[Figure 10.2] SPARE-TIRE-PROBLEM + """ + [Figure 10.2] SPARE-TIRE-PROBLEM A problem involving changing the flat tire of a car with a spare tire from the trunk. @@ -221,18 +338,21 @@ def spare_tire(): >>> """ - return PlanningProblem(init='Tire(Flat) & Tire(Spare) & At(Flat, Axle) & At(Spare, Trunk)', - goals='At(Spare, Axle) & At(Flat, Ground)', - actions=[Action('Remove(obj, loc)', - precond='At(obj, loc)', - effect='At(obj, Ground) & ~At(obj, loc)'), - Action('PutOn(t, Axle)', - precond='Tire(t) & At(t, Ground) & ~At(Flat, Axle)', - effect='At(t, Axle) & ~At(t, Ground)'), - Action('LeaveOvernight', - precond='', - effect='~At(Spare, Ground) & ~At(Spare, Axle) & ~At(Spare, Trunk) & \ - ~At(Flat, Ground) & ~At(Flat, Axle) & ~At(Flat, Trunk)')]) + return PlanningProblem(initial='At(Flat, Axle) & At(Spare, Trunk)', + goals='At(Spare, Axle) & At(Flat, Ground)', + actions=[Action('Remove(obj, loc)', + precond='At(obj, loc)', + effect='At(obj, Ground) & ~At(obj, loc)', + domain='Tire(obj)'), + Action('PutOn(t, Axle)', + precond='At(t, Ground) & ~At(Flat, Axle)', + effect='At(t, Axle) & ~At(t, Ground)', + domain='Tire(t)'), + Action('LeaveOvernight', + precond='', + effect='~At(Spare, Ground) & ~At(Spare, Axle) & ~At(Spare, Trunk) & \ + ~At(Flat, Ground) & ~At(Flat, Axle) & ~At(Flat, Trunk)')], + domain='Tire(Flat) & Tire(Spare)') def three_block_tower(): @@ -256,15 +376,17 @@ def three_block_tower(): True >>> """ - - return PlanningProblem(init='On(A, Table) & On(B, Table) & On(C, A) & Block(A) & Block(B) & Block(C) & Clear(B) & Clear(C)', - goals='On(A, B) & On(B, C)', - actions=[Action('Move(b, x, y)', - precond='On(b, x) & Clear(b) & Clear(y) & Block(b) & Block(y)', - effect='On(b, y) & Clear(x) & ~On(b, x) & ~Clear(y)'), - Action('MoveToTable(b, x)', - precond='On(b, x) & Clear(b) & Block(b)', - effect='On(b, Table) & Clear(x) & ~On(b, x)')]) + return PlanningProblem(initial='On(A, Table) & On(B, Table) & On(C, A) & Clear(B) & Clear(C)', + goals='On(A, B) & On(B, C)', + actions=[Action('Move(b, x, y)', + precond='On(b, x) & Clear(b) & Clear(y)', + effect='On(b, y) & Clear(x) & ~On(b, x) & ~Clear(y)', + domain='Block(b) & Block(y)'), + Action('MoveToTable(b, x)', + precond='On(b, x) & Clear(b)', + effect='On(b, Table) & Clear(x) & ~On(b, x)', + domain='Block(b) & Block(x)')], + domain='Block(A) & Block(B) & Block(C)') def simple_blocks_world(): @@ -288,21 +410,21 @@ def simple_blocks_world(): >>> """ - return PlanningProblem(init='On(A, B) & Clear(A) & OnTable(B) & OnTable(C) & Clear(C)', - goals='On(B, A) & On(C, B)', - actions=[Action('ToTable(x, y)', - precond='On(x, y) & Clear(x)', - effect='~On(x, y) & Clear(y) & OnTable(x)'), - Action('FromTable(y, x)', - precond='OnTable(y) & Clear(y) & Clear(x)', - effect='~OnTable(y) & ~Clear(x) & On(y, x)')]) + return PlanningProblem(initial='On(A, B) & Clear(A) & OnTable(B) & OnTable(C) & Clear(C)', + goals='On(B, A) & On(C, B)', + actions=[Action('ToTable(x, y)', + precond='On(x, y) & Clear(x)', + effect='~On(x, y) & Clear(y) & OnTable(x)'), + Action('FromTable(y, x)', + precond='OnTable(y) & Clear(y) & Clear(x)', + effect='~OnTable(y) & ~Clear(x) & On(y, x)')]) def have_cake_and_eat_cake_too(): """ [Figure 10.7] CAKE-PROBLEM - A problem where we begin with a cake and want to + A problem where we begin with a cake and want to reach the state of having a cake and having eaten a cake. The possible actions include baking a cake and eating a cake. @@ -320,14 +442,14 @@ def have_cake_and_eat_cake_too(): >>> """ - return PlanningProblem(init='Have(Cake)', - goals='Have(Cake) & Eaten(Cake)', - actions=[Action('Eat(Cake)', - precond='Have(Cake)', - effect='Eaten(Cake) & ~Have(Cake)'), - Action('Bake(Cake)', - precond='~Have(Cake)', - effect='Have(Cake)')]) + return PlanningProblem(initial='Have(Cake)', + goals='Have(Cake) & Eaten(Cake)', + actions=[Action('Eat(Cake)', + precond='Have(Cake)', + effect='Eaten(Cake) & ~Have(Cake)'), + Action('Bake(Cake)', + precond='~Have(Cake)', + effect='Have(Cake)')]) def shopping_problem(): @@ -353,14 +475,18 @@ def shopping_problem(): >>> """ - return PlanningProblem(init='At(Home) & Sells(SM, Milk) & Sells(SM, Banana) & Sells(HW, Drill)', - goals='Have(Milk) & Have(Banana) & Have(Drill)', - actions=[Action('Buy(x, store)', - precond='At(store) & Sells(store, x)', - effect='Have(x)'), - Action('Go(x, y)', - precond='At(x)', - effect='At(y) & ~At(x)')]) + return PlanningProblem(initial='At(Home) & Sells(SM, Milk) & Sells(SM, Banana) & Sells(HW, Drill)', + goals='Have(Milk) & Have(Banana) & Have(Drill)', + actions=[Action('Buy(x, store)', + precond='At(store) & Sells(store, x)', + effect='Have(x)', + domain='Store(store) & Item(x)'), + Action('Go(x, y)', + precond='At(x)', + effect='At(y) & ~At(x)', + domain='Place(x) & Place(y)')], + domain='Place(Home) & Place(SM) & Place(HW) & Store(SM) & Store(HW) & ' + 'Item(Milk) & Item(Banana) & Item(Drill)') def socks_and_shoes(): @@ -385,20 +511,20 @@ def socks_and_shoes(): >>> """ - return PlanningProblem(init='', - goals='RightShoeOn & LeftShoeOn', - actions=[Action('RightShoe', - precond='RightSockOn', - effect='RightShoeOn'), - Action('RightSock', - precond='', - effect='RightSockOn'), - Action('LeftShoe', - precond='LeftSockOn', - effect='LeftShoeOn'), - Action('LeftSock', - precond='', - effect='LeftSockOn')]) + return PlanningProblem(initial='', + goals='RightShoeOn & LeftShoeOn', + actions=[Action('RightShoe', + precond='RightSockOn', + effect='RightShoeOn'), + Action('RightSock', + precond='', + effect='RightSockOn'), + Action('LeftShoe', + precond='LeftSockOn', + effect='LeftShoeOn'), + Action('LeftSock', + precond='', + effect='LeftSockOn')]) def double_tennis_problem(): @@ -411,26 +537,216 @@ def double_tennis_problem(): Example: >>> from planning import * >>> dtp = double_tennis_problem() - >>> goal_test(dtp.goals, dtp.init) + >>> goal_test(dtp.goals, dtp.initial) False >>> dtp.act(expr('Go(A, RightBaseLine, LeftBaseLine)')) >>> dtp.act(expr('Hit(A, Ball, RightBaseLine)')) - >>> goal_test(dtp.goals, dtp.init) + >>> goal_test(dtp.goals, dtp.initial) False >>> dtp.act(expr('Go(A, LeftNet, RightBaseLine)')) - >>> goal_test(dtp.goals, dtp.init) + >>> goal_test(dtp.goals, dtp.initial) True >>> """ - return PlanningProblem(init='At(A, LeftBaseLine) & At(B, RightNet) & Approaching(Ball, RightBaseLine) & Partner(A, B) & Partner(B, A)', - goals='Returned(Ball) & At(a, LeftNet) & At(a, RightNet)', - actions=[Action('Hit(actor, Ball, loc)', - precond='Approaching(Ball, loc) & At(actor, loc)', - effect='Returned(Ball)'), - Action('Go(actor, to, loc)', - precond='At(actor, loc)', - effect='At(actor, to) & ~At(actor, loc)')]) + return PlanningProblem( + initial='At(A, LeftBaseLine) & At(B, RightNet) & Approaching(Ball, RightBaseLine) & Partner(A, B) & Partner(B, A)', + goals='Returned(Ball) & At(a, LeftNet) & At(a, RightNet)', + actions=[Action('Hit(actor, Ball, loc)', + precond='Approaching(Ball, loc) & At(actor, loc)', + effect='Returned(Ball)'), + Action('Go(actor, to, loc)', + precond='At(actor, loc)', + effect='At(actor, to) & ~At(actor, loc)')]) + + +class ForwardPlan(search.Problem): + """ + [Section 10.2.1] + Forward state-space search + """ + + def __init__(self, planning_problem): + super().__init__(associate('&', planning_problem.initial), associate('&', planning_problem.goals)) + self.planning_problem = planning_problem + self.expanded_actions = self.planning_problem.expand_actions() + + def actions(self, state): + return [action for action in self.expanded_actions if all(pre in conjuncts(state) for pre in action.precond)] + + def result(self, state, action): + return associate('&', action(conjuncts(state), action.args).clauses) + + def goal_test(self, state): + return all(goal in conjuncts(state) for goal in self.planning_problem.goals) + + def h(self, state): + """ + Computes ignore delete lists heuristic by creating a relaxed version of the original problem (we can do that + by removing the delete lists from all actions, i.e. removing all negative literals from effects) that will be + easier to solve through GraphPlan and where the length of the solution will serve as a good heuristic. + """ + relaxed_planning_problem = PlanningProblem(initial=state.state, + goals=self.goal, + actions=[action.relaxed() for action in + self.planning_problem.actions]) + try: + return len(linearize(GraphPlan(relaxed_planning_problem).execute())) + except: + return np.inf + + +class BackwardPlan(search.Problem): + """ + [Section 10.2.2] + Backward relevant-states search + """ + + def __init__(self, planning_problem): + super().__init__(associate('&', planning_problem.goals), associate('&', planning_problem.initial)) + self.planning_problem = planning_problem + self.expanded_actions = self.planning_problem.expand_actions() + + def actions(self, subgoal): + """ + Returns True if the action is relevant to the subgoal, i.e.: + - the action achieves an element of the effects + - the action doesn't delete something that needs to be achieved + - the preconditions are consistent with other subgoals that need to be achieved + """ + + def negate_clause(clause): + return Expr(clause.op.replace('Not', ''), *clause.args) if clause.op[:3] == 'Not' else Expr( + 'Not' + clause.op, *clause.args) + + subgoal = conjuncts(subgoal) + return [action for action in self.expanded_actions if + (any(prop in action.effect for prop in subgoal) and + not any(negate_clause(prop) in subgoal for prop in action.effect) and + not any(negate_clause(prop) in subgoal and negate_clause(prop) not in action.effect + for prop in action.precond))] + + def result(self, subgoal, action): + # g' = (g - effects(a)) + preconds(a) + return associate('&', set(set(conjuncts(subgoal)).difference(action.effect)).union(action.precond)) + + def goal_test(self, subgoal): + return all(goal in conjuncts(self.goal) for goal in conjuncts(subgoal)) + + def h(self, subgoal): + """ + Computes ignore delete lists heuristic by creating a relaxed version of the original problem (we can do that + by removing the delete lists from all actions, i.e. removing all negative literals from effects) that will be + easier to solve through GraphPlan and where the length of the solution will serve as a good heuristic. + """ + relaxed_planning_problem = PlanningProblem(initial=self.goal, + goals=subgoal.state, + actions=[action.relaxed() for action in + self.planning_problem.actions]) + try: + return len(linearize(GraphPlan(relaxed_planning_problem).execute())) + except: + return np.inf + + +def CSPlan(planning_problem, solution_length, CSP_solver=ac_search_solver, arc_heuristic=sat_up): + """ + [Section 10.4.3] + Planning as Constraint Satisfaction Problem + """ + + def st(var, stage): + """Returns a string for the var-stage pair that can be used as a variable""" + return str(var) + "_" + str(stage) + + def if_(v1, v2): + """If the second argument is v2, the first argument must be v1""" + + def if_fun(x1, x2): + return x1 == v1 if x2 == v2 else True + + if_fun.__name__ = "if the second argument is " + str(v2) + " then the first argument is " + str(v1) + " " + return if_fun + + def eq_if_not_in_(actset): + """First and third arguments are equal if action is not in actset""" + + def eq_if_not_in(x1, a, x2): + return x1 == x2 if a not in actset else True + + eq_if_not_in.__name__ = "first and third arguments are equal if action is not in " + str(actset) + " " + return eq_if_not_in + + expanded_actions = planning_problem.expand_actions() + fluent_values = planning_problem.expand_fluents() + for horizon in range(solution_length): + act_vars = [st('action', stage) for stage in range(horizon + 1)] + domains = {av: list(map(lambda action: expr(str(action)), expanded_actions)) for av in act_vars} + domains.update({st(var, stage): {True, False} for var in fluent_values for stage in range(horizon + 2)}) + # initial state constraints + constraints = [Constraint((st(var, 0),), is_constraint(val)) + for (var, val) in {expr(str(fluent).replace('Not', '')): + True if fluent.op[:3] != 'Not' else False + for fluent in planning_problem.initial}.items()] + constraints += [Constraint((st(var, 0),), is_constraint(False)) + for var in {expr(str(fluent).replace('Not', '')) + for fluent in fluent_values if fluent not in planning_problem.initial}] + # goal state constraints + constraints += [Constraint((st(var, horizon + 1),), is_constraint(val)) + for (var, val) in {expr(str(fluent).replace('Not', '')): + True if fluent.op[:3] != 'Not' else False + for fluent in planning_problem.goals}.items()] + # precondition constraints + constraints += [Constraint((st(var, stage), st('action', stage)), if_(val, act)) + # st(var, stage) == val if st('action', stage) == act + for act, strps in {expr(str(action)): action for action in expanded_actions}.items() + for var, val in {expr(str(fluent).replace('Not', '')): + True if fluent.op[:3] != 'Not' else False + for fluent in strps.precond}.items() + for stage in range(horizon + 1)] + # effect constraints + constraints += [Constraint((st(var, stage + 1), st('action', stage)), if_(val, act)) + # st(var, stage + 1) == val if st('action', stage) == act + for act, strps in {expr(str(action)): action for action in expanded_actions}.items() + for var, val in {expr(str(fluent).replace('Not', '')): True if fluent.op[:3] != 'Not' else False + for fluent in strps.effect}.items() + for stage in range(horizon + 1)] + # frame constraints + constraints += [Constraint((st(var, stage), st('action', stage), st(var, stage + 1)), + eq_if_not_in_(set(map(lambda action: expr(str(action)), + {act for act in expanded_actions if var in act.effect + or Expr('Not' + var.op, *var.args) in act.effect})))) + for var in fluent_values for stage in range(horizon + 1)] + csp = NaryCSP(domains, constraints) + sol = CSP_solver(csp, arc_heuristic=arc_heuristic) + if sol: + return [sol[a] for a in act_vars] + + +def SATPlan(planning_problem, solution_length, SAT_solver=cdcl_satisfiable): + """ + [Section 10.4.1] + Planning as Boolean satisfiability + """ + + def expand_transitions(state, actions): + state = sorted(conjuncts(state)) + for action in filter(lambda act: act.check_precond(state, act.args), actions): + transition[associate('&', state)].update( + {Expr(action.name, *action.args): + associate('&', sorted(set(filter(lambda clause: clause.op[:3] != 'Not', + action(state, action.args).clauses)))) + if planning_problem.is_strips() + else associate('&', sorted(set(action(state, action.args).clauses)))}) + for state in transition[associate('&', state)].values(): + if state not in transition: + expand_transitions(expr(state), actions) + + transition = defaultdict(dict) + expand_transitions(associate('&', planning_problem.initial), planning_problem.expand_actions()) + + return SAT_plan(associate('&', sorted(planning_problem.initial)), transition, + associate('&', sorted(planning_problem.goals)), solution_length, SAT_solver=SAT_solver) class Level: @@ -492,12 +808,12 @@ def find_mutex(self): pos_csl, neg_csl = self.separate(self.current_state_links) # Competing needs - for posprecond in pos_csl: - for negprecond in neg_csl: - new_negprecond = Expr(negprecond.op[3:], *negprecond.args) - if new_negprecond == posprecond: - for a in self.current_state_links[posprecond]: - for b in self.current_state_links[negprecond]: + for pos_precond in pos_csl: + for neg_precond in neg_csl: + new_neg_precond = Expr(neg_precond.op[3:], *neg_precond.args) + if new_neg_precond == pos_precond: + for a in self.current_state_links[pos_precond]: + for b in self.current_state_links[neg_precond]: if {a, b} not in self.mutex: self.mutex.append({a, b}) @@ -511,7 +827,7 @@ def find_mutex(self): next_state_1 = self.next_action_links[list(pair)[0]] if (len(next_state_0) == 1) and (len(next_state_1) == 1): state_mutex.append({next_state_0[0], next_state_1[0]}) - + self.mutex = self.mutex + state_mutex def build(self, actions, objects): @@ -546,7 +862,7 @@ def build(self, actions, objects): self.current_state_links[new_clause].append(new_action) else: self.current_state_links[new_clause] = [new_action] - + self.next_action_links[new_action] = [] for clause in a.effect: new_clause = a.substitute(clause, arg) @@ -570,9 +886,9 @@ class Graph: Used in graph planning algorithm to extract a solution """ - def __init__(self, planningproblem): - self.planningproblem = planningproblem - self.kb = FolKB(planningproblem.init) + def __init__(self, planning_problem): + self.planning_problem = planning_problem + self.kb = FolKB(planning_problem.initial) self.levels = [Level(self.kb)] self.objects = set(arg for clause in self.kb.clauses for arg in clause.args) @@ -583,7 +899,7 @@ def expand_graph(self): """Expands the graph by a level""" last_level = self.levels[-1] - last_level(self.planningproblem.actions, self.objects) + last_level(self.planning_problem.actions, self.objects) self.levels.append(last_level.perform_actions()) def non_mutex_goals(self, goals, index): @@ -603,9 +919,9 @@ class GraphPlan: Returns solution for the planning problem """ - def __init__(self, planningproblem): - self.graph = Graph(planningproblem) - self.nogoods = [] + def __init__(self, planning_problem): + self.graph = Graph(planning_problem) + self.no_goods = [] self.solution = [] def check_leveloff(self): @@ -619,44 +935,43 @@ def check_leveloff(self): def extract_solution(self, goals, index): """Extracts the solution""" - level = self.graph.levels[index] + level = self.graph.levels[index] if not self.graph.non_mutex_goals(goals, index): - self.nogoods.append((level, goals)) + self.no_goods.append((level, goals)) return - level = self.graph.levels[index - 1] + level = self.graph.levels[index - 1] - # Create all combinations of actions that satisfy the goal + # Create all combinations of actions that satisfy the goal actions = [] for goal in goals: - actions.append(level.next_state_links[goal]) + actions.append(level.next_state_links[goal]) - all_actions = list(itertools.product(*actions)) + all_actions = list(itertools.product(*actions)) # Filter out non-mutex actions - non_mutex_actions = [] + non_mutex_actions = [] for action_tuple in all_actions: - action_pairs = itertools.combinations(list(set(action_tuple)), 2) - non_mutex_actions.append(list(set(action_tuple))) - for pair in action_pairs: + action_pairs = itertools.combinations(list(set(action_tuple)), 2) + non_mutex_actions.append(list(set(action_tuple))) + for pair in action_pairs: if set(pair) in level.mutex: non_mutex_actions.pop(-1) break - # Recursion - for action_list in non_mutex_actions: + for action_list in non_mutex_actions: if [action_list, index] not in self.solution: self.solution.append([action_list, index]) new_goals = [] - for act in set(action_list): + for act in set(action_list): if act in level.current_action_links: new_goals = new_goals + level.current_action_links[act] if abs(index) + 1 == len(self.graph.levels): return - elif (level, new_goals) in self.nogoods: + elif (level, new_goals) in self.no_goods: return else: self.extract_solution(new_goals, index - 1) @@ -677,26 +992,27 @@ def extract_solution(self, goals, index): return solution def goal_test(self, kb): - return all(kb.ask(q) is not False for q in self.graph.planningproblem.goals) + return all(kb.ask(q) is not False for q in self.graph.planning_problem.goals) def execute(self): """Executes the GraphPlan algorithm for the given problem""" while True: self.graph.expand_graph() - if (self.goal_test(self.graph.levels[-1].kb) and self.graph.non_mutex_goals(self.graph.planningproblem.goals, -1)): - solution = self.extract_solution(self.graph.planningproblem.goals, -1) + if (self.goal_test(self.graph.levels[-1].kb) and self.graph.non_mutex_goals( + self.graph.planning_problem.goals, -1)): + solution = self.extract_solution(self.graph.planning_problem.goals, -1) if solution: return solution - + if len(self.graph.levels) >= 2 and self.check_leveloff(): return None class Linearize: - def __init__(self, planningproblem): - self.planningproblem = planningproblem + def __init__(self, planning_problem): + self.planning_problem = planning_problem def filter(self, solution): """Filter out persistence actions from a solution""" @@ -710,11 +1026,11 @@ def filter(self, solution): new_solution.append(new_section) return new_solution - def orderlevel(self, level, planningproblem): + def orderlevel(self, level, planning_problem): """Return valid linear order of actions for a given level""" for permutation in itertools.permutations(level): - temp = copy.deepcopy(planningproblem) + temp = copy.deepcopy(planning_problem) count = 0 for action in permutation: try: @@ -722,7 +1038,7 @@ def orderlevel(self, level, planningproblem): count += 1 except: count = 0 - temp = copy.deepcopy(planningproblem) + temp = copy.deepcopy(planning_problem) break if count == len(permutation): return list(permutation), temp @@ -731,12 +1047,12 @@ def orderlevel(self, level, planningproblem): def execute(self): """Finds total-order solution for a planning graph""" - graphplan_solution = GraphPlan(self.planningproblem).execute() - filtered_solution = self.filter(graphplan_solution) + graphPlan_solution = GraphPlan(self.planning_problem).execute() + filtered_solution = self.filter(graphPlan_solution) ordered_solution = [] - planningproblem = self.planningproblem + planning_problem = self.planning_problem for level in filtered_solution: - level_solution, planningproblem = self.orderlevel(level, planningproblem) + level_solution, planning_problem = self.orderlevel(level, planning_problem) for element in level_solution: ordered_solution.append(element) @@ -755,39 +1071,35 @@ def linearize(solution): return linear_solution -''' -[Section 10.13] PARTIAL-ORDER-PLANNER - -Partially ordered plans are created by a search through the space of plans -rather than a search through the state space. It views planning as a refinement of partially ordered plans. -A partially ordered plan is defined by a set of actions and a set of constraints of the form A < B, -which denotes that action A has to be performed before action B. -To summarize the working of a partial order planner, -1. An open precondition is selected (a sub-goal that we want to achieve). -2. An action that fulfils the open precondition is chosen. -3. Temporal constraints are updated. -4. Existing causal links are protected. Protection is a method that checks if the causal links conflict - and if they do, temporal constraints are added to fix the threats. -5. The set of open preconditions is updated. -6. Temporal constraints of the selected action and the next action are established. -7. A new causal link is added between the selected action and the owner of the open precondition. -8. The set of new causal links is checked for threats and if found, the threat is removed by either promotion or demotion. - If promotion or demotion is unable to solve the problem, the planning problem cannot be solved with the current sequence of actions - or it may not be solvable at all. -9. These steps are repeated until the set of open preconditions is empty. -''' - class PartialOrderPlanner: + """ + [Section 10.13] PARTIAL-ORDER-PLANNER + + Partially ordered plans are created by a search through the space of plans + rather than a search through the state space. It views planning as a refinement of partially ordered plans. + A partially ordered plan is defined by a set of actions and a set of constraints of the form A < B, + which denotes that action A has to be performed before action B. + To summarize the working of a partial order planner, + 1. An open precondition is selected (a sub-goal that we want to achieve). + 2. An action that fulfils the open precondition is chosen. + 3. Temporal constraints are updated. + 4. Existing causal links are protected. Protection is a method that checks if the causal links conflict + and if they do, temporal constraints are added to fix the threats. + 5. The set of open preconditions is updated. + 6. Temporal constraints of the selected action and the next action are established. + 7. A new causal link is added between the selected action and the owner of the open precondition. + 8. The set of new causal links is checked for threats and if found, the threat is removed by either promotion or + demotion. If promotion or demotion is unable to solve the problem, the planning problem cannot be solved with + the current sequence of actions or it may not be solvable at all. + 9. These steps are repeated until the set of open preconditions is empty. + """ - def __init__(self, planningproblem): - self.planningproblem = planningproblem - self.initialize() - - def initialize(self): - """Initialize all variables""" + def __init__(self, planning_problem): + self.tries = 1 + self.planning_problem = planning_problem self.causal_links = [] - self.start = Action('Start', [], self.planningproblem.init) - self.finish = Action('Finish', self.planningproblem.goals, []) + self.start = Action('Start', [], self.planning_problem.initial) + self.finish = Action('Finish', self.planning_problem.goals, []) self.actions = set() self.actions.add(self.start) self.actions.add(self.finish) @@ -796,55 +1108,7 @@ def initialize(self): self.agenda = set() for precond in self.finish.precond: self.agenda.add((precond, self.finish)) - self.expanded_actions = self.expand_actions() - - def expand_actions(self, name=None): - """Generate all possible actions with variable bindings for precondition selection heuristic""" - - objects = set(arg for clause in self.planningproblem.init for arg in clause.args) - expansions = [] - action_list = [] - if name is not None: - for action in self.planningproblem.actions: - if str(action.name) == name: - action_list.append(action) - else: - action_list = self.planningproblem.actions - - for action in action_list: - for permutation in itertools.permutations(objects, len(action.args)): - bindings = unify(Expr(action.name, *action.args), Expr(action.name, *permutation)) - if bindings is not None: - new_args = [] - for arg in action.args: - if arg in bindings: - new_args.append(bindings[arg]) - else: - new_args.append(arg) - new_expr = Expr(str(action.name), *new_args) - new_preconds = [] - for precond in action.precond: - new_precond_args = [] - for arg in precond.args: - if arg in bindings: - new_precond_args.append(bindings[arg]) - else: - new_precond_args.append(arg) - new_precond = Expr(str(precond.op), *new_precond_args) - new_preconds.append(new_precond) - new_effects = [] - for effect in action.effect: - new_effect_args = [] - for arg in effect.args: - if arg in bindings: - new_effect_args.append(bindings[arg]) - else: - new_effect_args.append(arg) - new_effect = Expr(str(effect.op), *new_effect_args) - new_effects.append(new_effect) - expansions.append(Action(new_expr, new_preconds, new_effects)) - - return expansions + self.expanded_actions = planning_problem.expand_actions() def find_open_precondition(self): """Find open precondition with the least number of possible actions""" @@ -865,7 +1129,7 @@ def find_open_precondition(self): actions_for_precondition[open_precondition] = [action] number = sorted(number_of_ways, key=number_of_ways.__getitem__) - + for k, v in number_of_ways.items(): if v == 0: return None, None, None @@ -893,10 +1157,10 @@ def find_action_for_precondition(self, oprec): # or # choose act0 E Actions such that act0 achieves G - for action in self.planningproblem.actions: + for action in self.planning_problem.actions: for effect in action.effect: if effect.op == oprec.op: - bindings = unify(effect, oprec) + bindings = unify_mm(effect, oprec) if bindings is None: break return action, bindings @@ -915,9 +1179,9 @@ def generate_expr(self, clause, bindings): return Expr(str(clause.name), *new_args) except: return Expr(str(clause.op), *new_args) - + def generate_action_object(self, action, bindings): - """Generate action object given a generic action andvariable bindings""" + """Generate action object given a generic action and variable bindings""" # if bindings is 0, it means the action already exists in self.actions if bindings == 0: @@ -1032,13 +1296,15 @@ def toposort(self, graph): extra_elements_in_dependencies = _reduce(set.union, graph.values()) - set(graph.keys()) - graph.update({element:set() for element in extra_elements_in_dependencies}) + graph.update({element: set() for element in extra_elements_in_dependencies}) while True: ordered = set(element for element, dependency in graph.items() if len(dependency) == 0) if not ordered: break yield ordered - graph = {element: (dependency - ordered) for element, dependency in graph.items() if element not in ordered} + graph = {element: (dependency - ordered) + for element, dependency in graph.items() + if element not in ordered} if len(graph) != 0: raise ValueError('The graph is not acyclic and cannot be linearly ordered') @@ -1060,7 +1326,6 @@ def execute(self, display=True): """Execute the algorithm""" step = 1 - self.tries = 1 while len(self.agenda) > 0: step += 1 # select from Agenda @@ -1106,45 +1371,50 @@ def execute(self, display=True): self.constraints = self.protect((act0, G, act1), action, self.constraints) if step > 200: - print('Couldn\'t find a solution') + print("Couldn't find a solution") return None, None if display: self.display_plan() else: - return self.constraints, self.causal_links + return self.constraints, self.causal_links -def spare_tire_graphplan(): +def spare_tire_graphPlan(): """Solves the spare tire problem using GraphPlan""" return GraphPlan(spare_tire()).execute() -def three_block_tower_graphplan(): + +def three_block_tower_graphPlan(): """Solves the Sussman Anomaly problem using GraphPlan""" return GraphPlan(three_block_tower()).execute() -def air_cargo_graphplan(): + +def air_cargo_graphPlan(): """Solves the air cargo problem using GraphPlan""" return GraphPlan(air_cargo()).execute() -def have_cake_and_eat_cake_too_graphplan(): + +def have_cake_and_eat_cake_too_graphPlan(): """Solves the cake problem using GraphPlan""" return [GraphPlan(have_cake_and_eat_cake_too()).execute()[1]] -def shopping_graphplan(): + +def shopping_graphPlan(): """Solves the shopping problem using GraphPlan""" return GraphPlan(shopping_problem()).execute() -def socks_and_shoes_graphplan(): - """Solves the socks and shoes problem using GraphpPlan""" + +def socks_and_shoes_graphPlan(): + """Solves the socks and shoes problem using GraphPlan""" return GraphPlan(socks_and_shoes()).execute() -def simple_blocks_world_graphplan(): + +def simple_blocks_world_graphPlan(): """Solves the simple blocks world problem""" return GraphPlan(simple_blocks_world()).execute() - class HLA(Action): """ Define Actions for the real-world (that may be refined further), and satisfy resource @@ -1152,8 +1422,7 @@ class HLA(Action): """ unique_group = 1 - def __init__(self, action, precond=None, effect=None, duration=0, - consume=None, use=None): + def __init__(self, action, precond=None, effect=None, duration=0, consume=None, use=None): """ As opposed to actions, to define HLA, we have added constraints. duration holds the amount of time required to execute the task @@ -1175,7 +1444,6 @@ def do_action(self, job_order, available_resources, kb, args): An HLA based version of act - along with knowledge base updation, it handles resource checks, and ensures the actions are executed in the correct order. """ - # print(self.name) if not self.has_usable_resource(available_resources): raise Exception('Not enough usable resources to execute {}'.format(self.name)) if not self.has_consumable_resource(available_resources): @@ -1226,16 +1494,17 @@ def inorder(self, job_order): return True -class Problem(PlanningProblem): +class RealWorldPlanningProblem(PlanningProblem): """ Define real-world problems by aggregating resources as numerical quantities instead of named entities. - This class is identical to PDLL, except that it overloads the act function to handle + This class is identical to PDDL, except that it overloads the act function to handle resource and ordering conditions imposed by HLA as opposed to Action. """ - def __init__(self, init, goals, actions, jobs=None, resources=None): - super().__init__(init, goals, actions) + + def __init__(self, initial, goals, actions, jobs=None, resources=None): + super().__init__(initial, goals, actions) self.jobs = jobs self.resources = resources or {} @@ -1252,12 +1521,12 @@ def act(self, action): list_action = first(a for a in self.actions if a.name == action.name) if list_action is None: raise Exception("Action '{}' not found".format(action.name)) - self.init = list_action.do_action(self.jobs, self.resources, self.init, args).clauses + self.initial = list_action.do_action(self.jobs, self.resources, self.initial, args).clauses - def refinements(hla, state, library): # refinements may be (multiple) HLA themselves ... + def refinements(self, library): # refinements may be (multiple) HLA themselves ... """ - state is a Problem, containing the current state kb - library is a dictionary containing details for every possible refinement. eg: + State is a Problem, containing the current state kb library is a + dictionary containing details for every possible refinement. e.g.: { 'HLA': [ 'Go(Home, SFO)', @@ -1287,147 +1556,148 @@ def refinements(hla, state, library): # refinements may be (multiple) HLA thems ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(SFOLongTermParking)'], ['At(SFO) & ~At(Home)'] - ] - } + ]} """ - e = Expr(hla.name, hla.args) - indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name] + indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == self.name] for i in indices: actions = [] for j in range(len(library['steps'][i])): - # find the index of the step [j] of the HLA - index_step = [k for k,x in enumerate(library['HLA']) if x == library['steps'][i][j]][0] - precond = library['precond'][index_step][0] # preconditions of step [j] - effect = library['effect'][index_step][0] # effect of step [j] + # find the index of the step [j] of the HLA + index_step = [k for k, x in enumerate(library['HLA']) if x == library['steps'][i][j]][0] + precond = library['precond'][index_step][0] # preconditions of step [j] + effect = library['effect'][index_step][0] # effect of step [j] actions.append(HLA(library['steps'][i][j], precond, effect)) yield actions - def hierarchical_search(problem, hierarchy): + def hierarchical_search(self, hierarchy): """ - [Figure 11.5] 'Hierarchical Search, a Breadth First Search implementation of Hierarchical + [Figure 11.5] + 'Hierarchical Search, a Breadth First Search implementation of Hierarchical Forward Planning Search' The problem is a real-world problem defined by the problem class, and the hierarchy is a dictionary of HLA - refinements (see refinements generator for details) """ - act = Node(problem.init, None, [problem.actions[0]]) + act = Node(self.initial, None, [self.actions[0]]) frontier = deque() frontier.append(act) while True: if not frontier: return None plan = frontier.popleft() - (hla, index) = Problem.find_hla(plan, hierarchy) # finds the first non primitive hla in plan actions + # finds the first non primitive hla in plan actions + (hla, index) = RealWorldPlanningProblem.find_hla(plan, hierarchy) prefix = plan.action[:index] - outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions ) - suffix = plan.action[index+1:] - if not hla: # hla is None and plan is primitive + outcome = RealWorldPlanningProblem( + RealWorldPlanningProblem.result(self.initial, prefix), self.goals, self.actions) + suffix = plan.action[index + 1:] + if not hla: # hla is None and plan is primitive if outcome.goal_test(): return plan.action else: - for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements - frontier.append(Node(outcome.init, plan, prefix + sequence+ suffix)) + for sequence in RealWorldPlanningProblem.refinements(hla, hierarchy): # find refinements + frontier.append(Node(outcome.initial, plan, prefix + sequence + suffix)) def result(state, actions): """The outcome of applying an action to the current problem""" - for a in actions: + for a in actions: if a.check_precond(state, a.args): state = a(state, a.args).clauses return state - - def angelic_search(problem, hierarchy, initialPlan): + def angelic_search(self, hierarchy, initial_plan): """ - [Figure 11.8] A hierarchical planning algorithm that uses angelic semantics to identify and - commit to high-level plans that work while avoiding high-level plans that don’t. - The predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression - of refinements. - At top level, call ANGELIC -SEARCH with [Act ] as the initialPlan . + [Figure 11.8] + A hierarchical planning algorithm that uses angelic semantics to identify and + commit to high-level plans that work while avoiding high-level plans that don’t. + The predicate MAKING-PROGRESS checks to make sure that we aren’t stuck in an infinite regression + of refinements. + At top level, call ANGELIC-SEARCH with [Act] as the initialPlan. - initialPlan contains a sequence of HLA's with angelic semantics + InitialPlan contains a sequence of HLA's with angelic semantics - The possible effects of an angelic HLA in initialPlan are : + The possible effects of an angelic HLA in initialPlan are: ~ : effect remove $+: effect possibly add $-: effect possibly remove $$: possibly add or remove - """ - frontier = deque(initialPlan) - while True: + """ + frontier = deque(initial_plan) + while True: if not frontier: return None - plan = frontier.popleft() # sequence of HLA/Angelic HLA's - opt_reachable_set = Problem.reach_opt(problem.init, plan) - pes_reachable_set = Problem.reach_pes(problem.init, plan) - if problem.intersects_goal(opt_reachable_set): - if Problem.is_primitive( plan, hierarchy ): - return ([x for x in plan.action]) - guaranteed = problem.intersects_goal(pes_reachable_set) - if guaranteed and Problem.making_progress(plan, initialPlan): - final_state = guaranteed[0] # any element of guaranteed - return Problem.decompose(hierarchy, problem, plan, final_state, pes_reachable_set) - hla, index = Problem.find_hla(plan, hierarchy) # there should be at least one HLA/Angelic_HLA, otherwise plan would be primitive. + plan = frontier.popleft() # sequence of HLA/Angelic HLA's + opt_reachable_set = RealWorldPlanningProblem.reach_opt(self.initial, plan) + pes_reachable_set = RealWorldPlanningProblem.reach_pes(self.initial, plan) + if self.intersects_goal(opt_reachable_set): + if RealWorldPlanningProblem.is_primitive(plan, hierarchy): + return [x for x in plan.action] + guaranteed = self.intersects_goal(pes_reachable_set) + if guaranteed and RealWorldPlanningProblem.making_progress(plan, initial_plan): + final_state = guaranteed[0] # any element of guaranteed + return RealWorldPlanningProblem.decompose(hierarchy, final_state, pes_reachable_set) + # there should be at least one HLA/AngelicHLA, otherwise plan would be primitive + hla, index = RealWorldPlanningProblem.find_hla(plan, hierarchy) prefix = plan.action[:index] - suffix = plan.action[index+1:] - outcome = Problem(Problem.result(problem.init, prefix), problem.goals , problem.actions ) - for sequence in Problem.refinements(hla, outcome, hierarchy): # find refinements - frontier.append(Angelic_Node(outcome.init, plan, prefix + sequence+ suffix, prefix+sequence+suffix)) - - - def intersects_goal(problem, reachable_set): + suffix = plan.action[index + 1:] + outcome = RealWorldPlanningProblem( + RealWorldPlanningProblem.result(self.initial, prefix), self.goals, self.actions) + for sequence in RealWorldPlanningProblem.refinements(hla, hierarchy): # find refinements + frontier.append( + AngelicNode(outcome.initial, plan, prefix + sequence + suffix, prefix + sequence + suffix)) + + def intersects_goal(self, reachable_set): """ Find the intersection of the reachable states and the goal """ - return [y for x in list(reachable_set.keys()) for y in reachable_set[x] if all(goal in y for goal in problem.goals)] + return [y for x in list(reachable_set.keys()) + for y in reachable_set[x] + if all(goal in y for goal in self.goals)] - - def is_primitive(plan, library): + def is_primitive(plan, library): """ - checks if the hla is primitive action + checks if the hla is primitive action """ - for hla in plan.action: + for hla in plan.action: indices = [i for i, x in enumerate(library['HLA']) if expr(x).op == hla.name] for i in indices: - if library["steps"][i]: + if library["steps"][i]: return False return True - - - def reach_opt(init, plan): + def reach_opt(init, plan): """ - Finds the optimistic reachable set of the sequence of actions in plan + Finds the optimistic reachable set of the sequence of actions in plan """ reachable_set = {0: [init]} - optimistic_description = plan.action #list of angelic actions with optimistic description - return Problem.find_reachable_set(reachable_set, optimistic_description) - + optimistic_description = plan.action # list of angelic actions with optimistic description + return RealWorldPlanningProblem.find_reachable_set(reachable_set, optimistic_description) - def reach_pes(init, plan): - """ + def reach_pes(init, plan): + """ Finds the pessimistic reachable set of the sequence of actions in plan """ reachable_set = {0: [init]} - pessimistic_description = plan.action_pes # list of angelic actions with pessimistic description - return Problem.find_reachable_set(reachable_set, pessimistic_description) + pessimistic_description = plan.action_pes # list of angelic actions with pessimistic description + return RealWorldPlanningProblem.find_reachable_set(reachable_set, pessimistic_description) def find_reachable_set(reachable_set, action_description): """ - Finds the reachable states of the action_description when applied in each state of reachable set. - """ + Finds the reachable states of the action_description when applied in each state of reachable set. + """ for i in range(len(action_description)): - reachable_set[i+1]=[] - if type(action_description[i]) is Angelic_HLA: + reachable_set[i + 1] = [] + if type(action_description[i]) is AngelicHLA: possible_actions = action_description[i].angelic_action() - else: + else: possible_actions = action_description for action in possible_actions: for state in reachable_set[i]: - if action.check_precond(state , action.args) : - if action.effect[0] : + if action.check_precond(state, action.args): + if action.effect[0]: new_state = action(state, action.args).clauses - reachable_set[i+1].append(new_state) - else: - reachable_set[i+1].append(state) + reachable_set[i + 1].append(new_state) + else: + reachable_set[i + 1].append(state) return reachable_set def find_hla(plan, hierarchy): @@ -1437,54 +1707,56 @@ def find_hla(plan, hierarchy): """ hla = None index = len(plan.action) - for i in range(len(plan.action)): # find the first HLA in plan, that is not primitive - if not Problem.is_primitive(Node(plan.state, plan.parent, [plan.action[i]]), hierarchy): - hla = plan.action[i] + for i in range(len(plan.action)): # find the first HLA in plan, that is not primitive + if not RealWorldPlanningProblem.is_primitive(Node(plan.state, plan.parent, [plan.action[i]]), hierarchy): + hla = plan.action[i] index = i break return hla, index - def making_progress(plan, initialPlan): - """ - Prevents from infinite regression of refinements + def making_progress(plan, initial_plan): + """ + Prevents from infinite regression of refinements - (infinite regression of refinements happens when the algorithm finds a plan that - its pessimistic reachable set intersects the goal inside a call to decompose on the same plan, in the same circumstances) + (infinite regression of refinements happens when the algorithm finds a plan that + its pessimistic reachable set intersects the goal inside a call to decompose on + the same plan, in the same circumstances) """ - for i in range(len(initialPlan)): - if (plan == initialPlan[i]): + for i in range(len(initial_plan)): + if plan == initial_plan[i]: return False - return True + return True - def decompose(hierarchy, s_0, plan, s_f, reachable_set): - solution = [] + def decompose(hierarchy, plan, s_f, reachable_set): + solution = [] i = max(reachable_set.keys()) - while plan.action_pes: + while plan.action_pes: action = plan.action_pes.pop() - if (i==0): + if i == 0: return solution - s_i = Problem.find_previous_state(s_f, reachable_set,i, action) - problem = Problem(s_i, s_f , plan.action) - angelic_call = Problem.angelic_search(problem, hierarchy, [Angelic_Node(s_i, Node(None), [action],[action])]) + s_i = RealWorldPlanningProblem.find_previous_state(s_f, reachable_set, i, action) + problem = RealWorldPlanningProblem(s_i, s_f, plan.action) + angelic_call = RealWorldPlanningProblem.angelic_search(problem, hierarchy, + [AngelicNode(s_i, Node(None), [action], [action])]) if angelic_call: - for x in angelic_call: - solution.insert(0,x) - else: + for x in angelic_call: + solution.insert(0, x) + else: return None s_f = s_i - i-=1 + i -= 1 return solution - def find_previous_state(s_f, reachable_set, i, action): """ - Given a final state s_f and an action finds a state s_i in reachable_set - such that when action is applied to state s_i returns s_f. + Given a final state s_f and an action finds a state s_i in reachable_set + such that when action is applied to state s_i returns s_f. """ - s_i = reachable_set[i-1][0] - for state in reachable_set[i-1]: - if s_f in [x for x in Problem.reach_pes(state, Angelic_Node(state, None, [action],[action]))[1]]: - s_i =state + s_i = reachable_set[i - 1][0] + for state in reachable_set[i - 1]: + if s_f in [x for x in RealWorldPlanningProblem.reach_pes( + state, AngelicNode(state, None, [action], [action]))[1]]: + s_i = state break return s_i @@ -1517,8 +1789,10 @@ def job_shop_problem(): add_engine1 = HLA('AddEngine1', precond='~Has(C1, E1)', effect='Has(C1, E1)', duration=30, use={'EngineHoists': 1}) add_engine2 = HLA('AddEngine2', precond='~Has(C2, E2)', effect='Has(C2, E2)', duration=60, use={'EngineHoists': 1}) - add_wheels1 = HLA('AddWheels1', precond='~Has(C1, W1)', effect='Has(C1, W1)', duration=30, use={'WheelStations': 1}, consume={'LugNuts': 20}) - add_wheels2 = HLA('AddWheels2', precond='~Has(C2, W2)', effect='Has(C2, W2)', duration=15, use={'WheelStations': 1}, consume={'LugNuts': 20}) + add_wheels1 = HLA('AddWheels1', precond='~Has(C1, W1)', effect='Has(C1, W1)', duration=30, use={'WheelStations': 1}, + consume={'LugNuts': 20}) + add_wheels2 = HLA('AddWheels2', precond='~Has(C2, W2)', effect='Has(C2, W2)', duration=15, use={'WheelStations': 1}, + consume={'LugNuts': 20}) inspect1 = HLA('Inspect1', precond='~Inspected(C1)', effect='Inspected(C1)', duration=10, use={'Inspectors': 1}) inspect2 = HLA('Inspect2', precond='~Inspected(C2)', effect='Inspected(C2)', duration=10, use={'Inspectors': 1}) @@ -1527,11 +1801,13 @@ def job_shop_problem(): job_group1 = [add_engine1, add_wheels1, inspect1] job_group2 = [add_engine2, add_wheels2, inspect2] - return Problem(init='Car(C1) & Car(C2) & Wheels(W1) & Wheels(W2) & Engine(E2) & Engine(E2) & ~Has(C1, E1) & ~Has(C2, E2) & ~Has(C1, W1) & ~Has(C2, W2) & ~Inspected(C1) & ~Inspected(C2)', - goals='Has(C1, W1) & Has(C1, E1) & Inspected(C1) & Has(C2, W2) & Has(C2, E2) & Inspected(C2)', - actions=actions, - jobs=[job_group1, job_group2], - resources=resources) + return RealWorldPlanningProblem( + initial='Car(C1) & Car(C2) & Wheels(W1) & Wheels(W2) & Engine(E2) & Engine(E2) & ~Has(C1, E1) & ~Has(C2, ' + 'E2) & ~Has(C1, W1) & ~Has(C2, W2) & ~Inspected(C1) & ~Inspected(C2)', + goals='Has(C1, W1) & Has(C1, E1) & Inspected(C1) & Has(C2, W2) & Has(C2, E2) & Inspected(C2)', + actions=actions, + jobs=[job_group1, job_group2], + resources=resources) def go_to_sfo(): @@ -1539,8 +1815,10 @@ def go_to_sfo(): go_home_sfo1 = HLA('Go(Home, SFO)', precond='At(Home) & Have(Car)', effect='At(SFO) & ~At(Home)') go_home_sfo2 = HLA('Go(Home, SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)') - drive_home_sfoltp = HLA('Drive(Home, SFOLongTermParking)', precond='At(Home) & Have(Car)', effect='At(SFOLongTermParking) & ~At(Home)') - shuttle_sfoltp_sfo = HLA('Shuttle(SFOLongTermParking, SFO)', precond='At(SFOLongTermParking)', effect='At(SFO) & ~At(SFOLongTermParking)') + drive_home_sfoltp = HLA('Drive(Home, SFOLongTermParking)', precond='At(Home) & Have(Car)', + effect='At(SFOLongTermParking) & ~At(Home)') + shuttle_sfoltp_sfo = HLA('Shuttle(SFOLongTermParking, SFO)', precond='At(SFOLongTermParking)', + effect='At(SFO) & ~At(SFOLongTermParking)') taxi_home_sfo = HLA('Taxi(Home, SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)') actions = [go_home_sfo1, go_home_sfo2, drive_home_sfoltp, shuttle_sfoltp_sfo, taxi_home_sfo] @@ -1572,44 +1850,41 @@ def go_to_sfo(): ['At(SFO) & ~At(Home)'], ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(SFOLongTermParking)'], - ['At(SFO) & ~At(Home)'] - ] - } + ['At(SFO) & ~At(Home)']]} - return Problem(init='At(Home)', goals='At(SFO)', actions=actions), library + return RealWorldPlanningProblem(initial='At(Home)', goals='At(SFO)', actions=actions), library -class Angelic_HLA(HLA): +class AngelicHLA(HLA): """ Define Actions for the real-world (that may be refined further), under angelic semantics """ - - def __init__(self, action, precond , effect, duration =0, consume = None, use = None): - super().__init__(action, precond, effect, duration, consume, use) + def __init__(self, action, precond, effect, duration=0, consume=None, use=None): + super().__init__(action, precond, effect, duration, consume, use) def convert(self, clauses): """ Converts strings into Exprs - An HLA with angelic semantics can achieve the effects of simple HLA's (add / remove a variable ) - and furthermore can have following effects on the variables: + An HLA with angelic semantics can achieve the effects of simple HLA's (add / remove a variable) + and furthermore can have following effects on the variables: Possibly add variable ( $+ ) Possibly remove variable ( $- ) Possibly add or remove a variable ( $$ ) Overrides HLA.convert function - """ - lib = {'~': 'Not', - '$+': 'PosYes', + """ + lib = {'~': 'Not', + '$+': 'PosYes', '$-': 'PosNot', - '$$' : 'PosYesNot'} + '$$': 'PosYesNot'} if isinstance(clauses, Expr): clauses = conjuncts(clauses) for i in range(len(clauses)): for ch in lib.keys(): if clauses[i].op == ch: - clauses[i] = expr( lib[ch] + str(clauses[i].args[0])) + clauses[i] = expr(lib[ch] + str(clauses[i].args[0])) elif isinstance(clauses, str): for ch in lib.keys(): @@ -1624,81 +1899,81 @@ def convert(self, clauses): return clauses - - - def angelic_action(self): """ - Converts a high level action (HLA) with angelic semantics into all of its corresponding high level actions (HLA). - An HLA with angelic semantics can achieve the effects of simple HLA's (add / remove a variable) - and furthermore can have following effects for each variable: + Converts a high level action (HLA) with angelic semantics into all of its corresponding high level actions (HLA). + An HLA with angelic semantics can achieve the effects of simple HLA's (add / remove a variable) + and furthermore can have following effects for each variable: - Possibly add variable ( $+: 'PosYes' ) --> corresponds to two HLAs: - HLA_1: add variable + Possibly add variable ( $+: 'PosYes' ) --> corresponds to two HLAs: + HLA_1: add variable HLA_2: leave variable unchanged Possibly remove variable ( $-: 'PosNot' ) --> corresponds to two HLAs: HLA_1: remove variable HLA_2: leave variable unchanged - Possibly add / remove a variable ( $$: 'PosYesNot' ) --> corresponds to three HLAs: + Possibly add / remove a variable ( $$: 'PosYesNot' ) --> corresponds to three HLAs: HLA_1: add variable HLA_2: remove variable - HLA_3: leave variable unchanged + HLA_3: leave variable unchanged + + + example: the angelic action with effects possibly add A and possibly add or remove B corresponds to the + following 6 effects of HLAs: - example: the angelic action with effects possibly add A and possibly add or remove B corresponds to the following 6 effects of HLAs: - - '$+A & $$B': HLA_1: 'A & B' (add A and add B) HLA_2: 'A & ~B' (add A and remove B) HLA_3: 'A' (add A) HLA_4: 'B' (add B) HLA_5: '~B' (remove B) - HLA_6: ' ' (no effect) + HLA_6: ' ' (no effect) """ - effects=[[]] + effects = [[]] for clause in self.effect: - (n,w) = Angelic_HLA.compute_parameters(clause, effects) - effects = effects*n # create n copies of effects - it=range(1) - if len(effects)!=0: - # split effects into n sublists (seperate n copies created in compute_parameters) - it = range(len(effects)//n) + (n, w) = AngelicHLA.compute_parameters(clause) + effects = effects * n # create n copies of effects + it = range(1) + if len(effects) != 0: + # split effects into n sublists (separate n copies created in compute_parameters) + it = range(len(effects) // n) for i in it: if effects[i]: - if clause.args: - effects[i] = expr(str(effects[i]) + '&' + str(Expr(clause.op[w:],clause.args[0]))) # make changes in the ith part of effects - if n==3: - effects[i+len(effects)//3]= expr(str(effects[i+len(effects)//3]) + '&' + str(Expr(clause.op[6:],clause.args[0]))) - else: - effects[i] = expr(str(effects[i]) + '&' + str(expr(clause.op[w:]))) # make changes in the ith part of effects - if n==3: - effects[i+len(effects)//3] = expr(str(effects[i+len(effects)//3]) + '&' + str(expr(clause.op[6:]))) - - else: - if clause.args: - effects[i] = Expr(clause.op[w:], clause.args[0]) # make changes in the ith part of effects - if n==3: - effects[i+len(effects)//3] = Expr(clause.op[6:], clause.args[0]) - - else: + if clause.args: + effects[i] = expr(str(effects[i]) + '&' + str( + Expr(clause.op[w:], clause.args[0]))) # make changes in the ith part of effects + if n == 3: + effects[i + len(effects) // 3] = expr( + str(effects[i + len(effects) // 3]) + '&' + str(Expr(clause.op[6:], clause.args[0]))) + else: + effects[i] = expr( + str(effects[i]) + '&' + str(expr(clause.op[w:]))) # make changes in the ith part of effects + if n == 3: + effects[i + len(effects) // 3] = expr( + str(effects[i + len(effects) // 3]) + '&' + str(expr(clause.op[6:]))) + + else: + if clause.args: + effects[i] = Expr(clause.op[w:], clause.args[0]) # make changes in the ith part of effects + if n == 3: + effects[i + len(effects) // 3] = Expr(clause.op[6:], clause.args[0]) + + else: effects[i] = expr(clause.op[w:]) # make changes in the ith part of effects - if n==3: - effects[i+len(effects)//3] = expr(clause.op[6:]) - #print('effects', effects) + if n == 3: + effects[i + len(effects) // 3] = expr(clause.op[6:]) - return [ HLA(Expr(self.name, self.args), self.precond, effects[i] ) for i in range(len(effects)) ] + return [HLA(Expr(self.name, self.args), self.precond, effects[i]) for i in range(len(effects))] + def compute_parameters(clause): + """ + computes n,w - def compute_parameters(clause, effects): - """ - computes n,w - - n = number of HLA effects that the anelic HLA corresponds to - w = length of representation of angelic HLA effect + n = number of HLA effects that the angelic HLA corresponds to + w = length of representation of angelic HLA effect n = 1, if effect is add n = 1, if effect is remove @@ -1708,30 +1983,28 @@ def compute_parameters(clause, effects): """ if clause.op[:9] == 'PosYesNot': - # possibly add/remove variable: three possible effects for the variable - n=3 - w=9 - elif clause.op[:6] == 'PosYes': # possibly add variable: two possible effects for the variable - n=2 - w=6 - elif clause.op[:6] == 'PosNot': # possibly remove variable: two possible effects for the variable - n=2 - w=3 # We want to keep 'Not' from 'PosNot' when adding action - else: # variable or ~variable - n=1 - w=0 - return (n,w) - - -class Angelic_Node(Node): - """ - Extends the class Node. + # possibly add/remove variable: three possible effects for the variable + n = 3 + w = 9 + elif clause.op[:6] == 'PosYes': # possibly add variable: two possible effects for the variable + n = 2 + w = 6 + elif clause.op[:6] == 'PosNot': # possibly remove variable: two possible effects for the variable + n = 2 + w = 3 # We want to keep 'Not' from 'PosNot' when adding action + else: # variable or ~variable + n = 1 + w = 0 + return n, w + + +class AngelicNode(Node): + """ + Extends the class Node. self.action: contains the optimistic description of an angelic HLA self.action_pes: contains the pessimistic description of an angelic HLA """ - def __init__(self, state, parent=None, action_opt=None, action_pes=None, path_cost=0): - super().__init__(state, parent, action_opt , path_cost) - self.action_pes = action_pes - - + def __init__(self, state, parent=None, action_opt=None, action_pes=None, path_cost=0): + super().__init__(state, parent, action_opt, path_cost) + self.action_pes = action_pes diff --git a/probabilistic_learning.py b/probabilistic_learning.py new file mode 100644 index 000000000..1138e702d --- /dev/null +++ b/probabilistic_learning.py @@ -0,0 +1,154 @@ +"""Learning probabilistic models. (Chapters 20)""" + +import heapq + +from utils import weighted_sampler, product, gaussian + + +class CountingProbDist: + """ + A probability distribution formed by observing and counting examples. + If p is an instance of this class and o is an observed value, then + there are 3 main operations: + p.add(o) increments the count for observation o by 1. + p.sample() returns a random element from the distribution. + p[o] returns the probability for o (as in a regular ProbDist). + """ + + def __init__(self, observations=None, default=0): + """ + Create a distribution, and optionally add in some observations. + By default this is an unsmoothed distribution, but saying default=1, + for example, gives you add-one smoothing. + """ + if observations is None: + observations = [] + self.dictionary = {} + self.n_obs = 0 + self.default = default + self.sampler = None + + for o in observations: + self.add(o) + + def add(self, o): + """Add an observation o to the distribution.""" + self.smooth_for(o) + self.dictionary[o] += 1 + self.n_obs += 1 + self.sampler = None + + def smooth_for(self, o): + """ + Include o among the possible observations, whether or not + it's been observed yet. + """ + if o not in self.dictionary: + self.dictionary[o] = self.default + self.n_obs += self.default + self.sampler = None + + def __getitem__(self, item): + """Return an estimate of the probability of item.""" + self.smooth_for(item) + return self.dictionary[item] / self.n_obs + + # (top() and sample() are not used in this module, but elsewhere.) + + def top(self, n): + """Return (count, obs) tuples for the n most frequent observations.""" + return heapq.nlargest(n, [(v, k) for (k, v) in self.dictionary.items()]) + + def sample(self): + """Return a random sample from the distribution.""" + if self.sampler is None: + self.sampler = weighted_sampler(list(self.dictionary.keys()), list(self.dictionary.values())) + return self.sampler() + + +def NaiveBayesLearner(dataset, continuous=True, simple=False): + if simple: + return NaiveBayesSimple(dataset) + if continuous: + return NaiveBayesContinuous(dataset) + else: + return NaiveBayesDiscrete(dataset) + + +def NaiveBayesSimple(distribution): + """ + A simple naive bayes classifier that takes as input a dictionary of + CountingProbDist objects and classifies items according to these distributions. + The input dictionary is in the following form: + (ClassName, ClassProb): CountingProbDist + """ + target_dist = {c_name: prob for c_name, prob in distribution.keys()} + attr_dists = {c_name: count_prob for (c_name, _), count_prob in distribution.items()} + + def predict(example): + """Predict the target value for example. Calculate probabilities for each + class and pick the max.""" + + def class_probability(target_val): + attr_dist = attr_dists[target_val] + return target_dist[target_val] * product(attr_dist[a] for a in example) + + return max(target_dist.keys(), key=class_probability) + + return predict + + +def NaiveBayesDiscrete(dataset): + """ + Just count how many times each value of each input attribute + occurs, conditional on the target value. Count the different + target values too. + """ + + target_vals = dataset.values[dataset.target] + target_dist = CountingProbDist(target_vals) + attr_dists = {(gv, attr): CountingProbDist(dataset.values[attr]) for gv in target_vals for attr in dataset.inputs} + for example in dataset.examples: + target_val = example[dataset.target] + target_dist.add(target_val) + for attr in dataset.inputs: + attr_dists[target_val, attr].add(example[attr]) + + def predict(example): + """ + Predict the target value for example. Consider each possible value, + and pick the most likely by looking at each attribute independently. + """ + + def class_probability(target_val): + return (target_dist[target_val] * product(attr_dists[target_val, attr][example[attr]] + for attr in dataset.inputs)) + + return max(target_vals, key=class_probability) + + return predict + + +def NaiveBayesContinuous(dataset): + """ + Count how many times each target value occurs. + Also, find the means and deviations of input attribute values for each target value. + """ + means, deviations = dataset.find_means_and_deviations() + + target_vals = dataset.values[dataset.target] + target_dist = CountingProbDist(target_vals) + + def predict(example): + """Predict the target value for example. Consider each possible value, + and pick the most likely by looking at each attribute independently.""" + + def class_probability(target_val): + prob = target_dist[target_val] + for attr in dataset.inputs: + prob *= gaussian(means[target_val][attr], deviations[target_val][attr], example[attr]) + return prob + + return max(target_vals, key=class_probability) + + return predict diff --git a/probability.ipynb b/probability.ipynb index ba06860fa..fe9643a83 100644 --- a/probability.ipynb +++ b/probability.ipynb @@ -12,9 +12,7 @@ { "cell_type": "code", "execution_count": 1, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "from probability import *\n", @@ -74,7 +72,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -453,7 +450,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -697,9 +693,7 @@ { "cell_type": "code", "execution_count": 15, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "full_joint = JointProbDist(['Cavity', 'Toothache', 'Catch'])\n", @@ -730,7 +724,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -944,7 +937,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -1118,7 +1110,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -1305,9 +1296,7 @@ { "cell_type": "code", "execution_count": 23, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "alarm_node = BayesNode('Alarm', ['Burglary', 'Earthquake'], \n", @@ -1324,9 +1313,7 @@ { "cell_type": "code", "execution_count": 24, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "john_node = BayesNode('JohnCalls', ['Alarm'], {True: 0.90, False: 0.05})\n", @@ -1344,9 +1331,7 @@ { "cell_type": "code", "execution_count": 25, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "burglary_node = BayesNode('Burglary', '', 0.001)\n", @@ -1397,7 +1382,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -1609,10 +1593,10 @@ { "data": { "text/plain": [ - "{(False, False): 0.001,\n", - " (False, True): 0.29,\n", + "{(True, True): 0.95,\n", " (True, False): 0.94,\n", - " (True, True): 0.95}" + " (False, True): 0.29,\n", + " (False, False): 0.001}" ] }, "execution_count": 30, @@ -1649,7 +1633,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -1786,7 +1769,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -1953,7 +1935,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2083,7 +2064,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2204,9 +2184,7 @@ { "cell_type": "code", "execution_count": 36, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "f5 = make_factor('MaryCalls', {'JohnCalls': True, 'MaryCalls': True}, burglary)" @@ -2220,7 +2198,7 @@ { "data": { "text/plain": [ - "" + "" ] }, "execution_count": 37, @@ -2240,7 +2218,7 @@ { "data": { "text/plain": [ - "{(False,): 0.01, (True,): 0.7}" + "{(True,): 0.7, (False,): 0.01}" ] }, "execution_count": 38, @@ -2282,9 +2260,7 @@ { "cell_type": "code", "execution_count": 40, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "new_factor = make_factor('MaryCalls', {'Alarm': True}, burglary)" @@ -2298,7 +2274,7 @@ { "data": { "text/plain": [ - "{(False,): 0.30000000000000004, (True,): 0.7}" + "{(True,): 0.7, (False,): 0.30000000000000004}" ] }, "execution_count": 41, @@ -2331,7 +2307,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2454,7 +2429,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2573,7 +2547,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2697,7 +2670,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2834,7 +2806,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -2966,6 +2937,33 @@ "elimination_ask('Burglary', dict(JohnCalls=True, MaryCalls=True), burglary).show_approx()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Elimination Ask Optimizations\n", + "\n", + "`elimination_ask` has some critical point to consider and some optimizations could be performed:\n", + "\n", + "- **Operation on factors**:\n", + "\n", + " `sum_out` and `pointwise_product` function used in `elimination_ask` is where space and time complexity arise in the variable elimination algorithm (AIMA3e pg. 526).\n", + "\n", + ">The only trick is to notice that any factor that does not depend on the variable to be summed out can be moved outside the summation.\n", + "\n", + "- **Variable ordering**:\n", + "\n", + " Elimination ordering is important, every choice of ordering yields a valid algorithm, but different orderings cause different intermediate factors to be generated during the calculation (AIMA3e pg. 527). In this case the algorithm applies a reversed order.\n", + "\n", + "> In general, the time and space requirements of variable elimination are dominated by the size of the largest factor constructed during the operation of the algorithm. This in turn is determined by the order of elimination of variables and by the structure of the network. It turns out to be intractable to determine the optimal ordering, but several good heuristics are available. One fairly effective method is a greedy one: eliminate whichever variable minimizes the size of the next factor to be constructed. \n", + "\n", + "- **Variable relevance**\n", + " \n", + " Some variables could be irrelevant to resolve a query (i.e. sums to 1). A variable elimination algorithm can therefore remove all these variables before evaluating the query (AIMA3e pg. 528).\n", + "\n", + "> An optimization is to remove 'every variable that is not an ancestor of a query variable or evidence variable is irrelevant to the query'." + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -2984,7 +2982,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "367 µs ± 126 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + "105 µs ± 11.9 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n" ] } ], @@ -3002,7 +3000,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "241 µs ± 64.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" + "262 µs ± 54.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n" ] } ], @@ -3015,10 +3013,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We observe that variable elimination was faster than enumeration as we had expected but the gain in speed is not a lot, in fact it is just about 30% faster.\n", + "In this test case we observe that variable elimination is slower than what we expected. It has something to do with number of threads, how Python tries to optimize things and this happens because the network is very small, with just 5 nodes. The `elimination_ask` has some critical point and some optimizations must be perfomed as seen above.\n", "
    \n", - "This happened because the bayesian network in question is pretty small, with just 5 nodes, some of which aren't even required in the inference process.\n", - "For more complicated networks, variable elimination will be significantly faster and runtime will reduce not just by a constant factor, but by a polynomial factor proportional to the number of nodes, due to the reduction in repeated calculations." + "Of course, for more complicated networks, variable elimination will be significantly faster and runtime will drop not just by a constant factor, but by a polynomial factor proportional to the number of nodes, due to the reduction in repeated calculations." ] }, { @@ -3040,7 +3037,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -3159,7 +3155,7 @@ }, { "cell_type": "code", - "execution_count": 52, + "execution_count": 51, "metadata": {}, "outputs": [ { @@ -3167,7 +3163,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -3300,9 +3295,7 @@ { "cell_type": "code", "execution_count": 52, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "N = 1000\n", @@ -3319,9 +3312,7 @@ { "cell_type": "code", "execution_count": 53, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "rain_true = [observation for observation in all_observations if observation['Rain'] == True]" @@ -3343,7 +3334,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "0.496\n" + "0.503\n" ] } ], @@ -3368,7 +3359,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "0.503\n" + "0.519\n" ] } ], @@ -3396,7 +3387,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "0.8091451292246521\n" + "0.8265895953757225\n" ] } ], @@ -3449,7 +3440,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -3533,7 +3523,7 @@ "\n", "

    \n", "\n", - "
    def rejection_sampling(X, e, bn, N):\n",
    +       "
    def rejection_sampling(X, e, bn, N=10000):\n",
            "    """Estimate the probability distribution of variable X given\n",
            "    evidence e in BayesNet bn, using N samples.  [Figure 14.14]\n",
            "    Raises a ZeroDivisionError if all the N samples are rejected,\n",
    @@ -3584,7 +3574,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -3703,7 +3692,7 @@
         {
          "data": {
           "text/plain": [
    -       "0.7660377358490567"
    +       "0.8035019455252919"
           ]
          },
          "execution_count": 59,
    @@ -3738,7 +3727,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -3869,7 +3857,7 @@
         {
          "data": {
           "text/plain": [
    -       "({'Cloudy': True, 'Rain': True, 'Sprinkler': False, 'WetGrass': True}, 0.8)"
    +       "({'Rain': True, 'Cloudy': False, 'Sprinkler': True, 'WetGrass': True}, 0.2)"
           ]
          },
          "execution_count": 61,
    @@ -3891,7 +3879,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -3975,7 +3962,7 @@
            "\n",
            "

    \n", "\n", - "
    def likelihood_weighting(X, e, bn, N):\n",
    +       "
    def likelihood_weighting(X, e, bn, N=10000):\n",
            "    """Estimate the probability distribution of variable X given\n",
            "    evidence e in BayesNet bn.  [Figure 14.15]\n",
            "    >>> random.seed(1017)\n",
    @@ -4019,7 +4006,7 @@
         {
          "data": {
           "text/plain": [
    -       "'False: 0.194, True: 0.806'"
    +       "'False: 0.2, True: 0.8'"
           ]
          },
          "execution_count": 63,
    @@ -4052,7 +4039,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -4136,7 +4122,7 @@
            "\n",
            "

    \n", "\n", - "
    def gibbs_ask(X, e, bn, N):\n",
    +       "
    def gibbs_ask(X, e, bn, N=1000):\n",
            "    """[Figure 14.16]"""\n",
            "    assert X not in e, "Query variable must be distinct from evidence"\n",
            "    counts = {x: 0 for x in bn.variable_values(X)}  # bold N in [Figure 14.16]\n",
    @@ -4180,7 +4166,7 @@
         {
          "data": {
           "text/plain": [
    -       "'False: 0.175, True: 0.825'"
    +       "'False: 0.215, True: 0.785'"
           ]
          },
          "execution_count": 65,
    @@ -4209,7 +4195,7 @@
          "name": "stdout",
          "output_type": "stream",
          "text": [
    -      "11.4 ms ± 4.1 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
    +      "13.2 ms ± 3.45 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
          ]
         }
        ],
    @@ -4229,7 +4215,7 @@
          "name": "stdout",
          "output_type": "stream",
          "text": [
    -      "8.63 ms ± 272 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
    +      "11 ms ± 687 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
          ]
         }
        ],
    @@ -4247,7 +4233,7 @@
          "name": "stdout",
          "output_type": "stream",
          "text": [
    -      "1.96 ms ± 696 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
    +      "2.12 ms ± 554 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
          ]
         }
        ],
    @@ -4265,7 +4251,7 @@
          "name": "stdout",
          "output_type": "stream",
          "text": [
    -      "7.03 ms ± 117 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
    +      "14.4 ms ± 2.16 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
          ]
         }
        ],
    @@ -4350,7 +4336,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -4473,9 +4458,7 @@
       {
        "cell_type": "code",
        "execution_count": 71,
    -   "metadata": {
    -    "collapsed": true
    -   },
    +   "metadata": {},
        "outputs": [],
        "source": [
         "umbrella_transition_model = [[0.7, 0.3], [0.3, 0.7]]\n",
    @@ -4565,7 +4548,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -4737,7 +4719,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -4904,7 +4885,7 @@
            ""
           ]
          },
    -     "execution_count": 79,
    +     "execution_count": 78,
          "metadata": {},
          "output_type": "execute_result"
         }
    @@ -4915,7 +4896,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 80,
    +   "execution_count": 79,
        "metadata": {},
        "outputs": [
         {
    @@ -4989,7 +4970,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 81,
    +   "execution_count": 80,
        "metadata": {},
        "outputs": [
         {
    @@ -4997,7 +4978,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -5145,10 +5125,8 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 82,
    -   "metadata": {
    -    "collapsed": true
    -   },
    +   "execution_count": 81,
    +   "metadata": {},
        "outputs": [],
        "source": [
         "umbrella_transition_model = [[0.7, 0.3], [0.3, 0.7]]\n",
    @@ -5167,7 +5145,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 83,
    +   "execution_count": 82,
        "metadata": {},
        "outputs": [
         {
    @@ -5176,7 +5154,7 @@
            "[0.1111111111111111, 0.8888888888888888]"
           ]
          },
    -     "execution_count": 83,
    +     "execution_count": 82,
          "metadata": {},
          "output_type": "execute_result"
         }
    @@ -5189,7 +5167,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 84,
    +   "execution_count": 83,
        "metadata": {},
        "outputs": [
         {
    @@ -5198,7 +5176,7 @@
            "[0.9938650306748466, 0.006134969325153394]"
           ]
          },
    -     "execution_count": 84,
    +     "execution_count": 83,
          "metadata": {},
          "output_type": "execute_result"
         }
    @@ -5218,10 +5196,8 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 85,
    -   "metadata": {
    -    "collapsed": true
    -   },
    +   "execution_count": 84,
    +   "metadata": {},
        "outputs": [],
        "source": [
         "fixed_lag_smoothing(e_t, hmm, d=5, ev=evidence, t=4)"
    @@ -5291,7 +5267,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 86,
    +   "execution_count": 85,
        "metadata": {},
        "outputs": [
         {
    @@ -5299,7 +5275,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -5454,10 +5429,8 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 87,
    -   "metadata": {
    -    "collapsed": true
    -   },
    +   "execution_count": 86,
    +   "metadata": {},
        "outputs": [],
        "source": [
         "umbrella_transition_model = [[0.7, 0.3], [0.3, 0.7]]\n",
    @@ -5467,7 +5440,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 88,
    +   "execution_count": 87,
        "metadata": {
         "scrolled": false
        },
    @@ -5475,10 +5448,10 @@
         {
          "data": {
           "text/plain": [
    -       "['A', 'A', 'A', 'A', 'B', 'A', 'B', 'B', 'B', 'B']"
    +       "['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']"
           ]
          },
    -     "execution_count": 88,
    +     "execution_count": 87,
          "metadata": {},
          "output_type": "execute_result"
         }
    @@ -5496,16 +5469,16 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 89,
    +   "execution_count": 88,
        "metadata": {},
        "outputs": [
         {
          "data": {
           "text/plain": [
    -       "['A', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'A', 'B']"
    +       "['A', 'B', 'A', 'B', 'B', 'B', 'B', 'B', 'B', 'B']"
           ]
          },
    -     "execution_count": 89,
    +     "execution_count": 88,
          "metadata": {},
          "output_type": "execute_result"
         }
    @@ -5573,7 +5546,7 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 90,
    +   "execution_count": 89,
        "metadata": {},
        "outputs": [
         {
    @@ -5581,7 +5554,6 @@
           "text/html": [
            "\n",
    -       "\n",
            "\n",
            "\n",
            "  Codestin Search App\n",
    @@ -5738,19 +5710,21 @@
       },
       {
        "cell_type": "code",
    -   "execution_count": 91,
    +   "execution_count": 90,
        "metadata": {
         "scrolled": true
        },
        "outputs": [
         {
          "data": {
    -      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFYCAYAAACs465lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEfZJREFUeJzt3XuMpXddx/HP1x0aKAWp6QL2oqVaUCRy6UhAIiqFWC5SjEZBIUUxTUShEBAKJmBiYoga1ESDWQu2iQ2gpQpeuFQE0QQrswWEsiANLe1CpVMJF5FYCl//mLMwDjs72znPzpnf8HolmzmXZ87zfWZn5j3Pc848U90dAGAs37boAQCAu07AAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAGHXaiqbqqqx2+47dlV9S8TPHZX1ffO+zjAYgk4AAxIwGFAVXV6Vb2pqlar6saqev66+x5ZVe+tqs9V1a1V9UdVddLsvvfMFvtgVf13Vf1cVf1YVR2uqpdU1W2z93laVT2pqv6jqj5bVS8/nsef3d9V9fyq+kRV3V5Vv1tVvtfAxHxRwWBmMfybJB9MckaS85O8oKp+YrbIV5O8MMlpSR49u/+5SdLdj50t89DuPqW73zi7fv8kd5893iuS/GmSZyY5L8mPJHlFVZ2z1eOv81NJlpM8IsmFSX5pim0HvqGcCx12n6q6KWuBvHPdzScluS7Ji5L8ZXd/17rlX5bkgd39i0d5rBck+dHu/qnZ9U5ybnffMLv+Y0nemuSU7v5qVd0ryReSPKq7r50tczDJb3X3Xx/n4z+xu982u/7cJD/d3efP8SEBNlha9ADApp7W3f9w5EpVPTvJLyf57iSnV9Xn1i27L8k/z5Z7YJJXZ20P+OSsfZ0f3GJd/9XdX51d/vLs7WfW3f/lJKfchce/Zd3lTyY5fYv1A3eRQ+gwnluS3Njd91n3717d/aTZ/a9J8tGs7WXfO8nLk9SE6z+exz9r3eXvSvLpCdcPRMBhRP+W5AtV9dKqukdV7auqh1TVD83uP3II/L+r6vuS/MqG9/9MknOyfVs9fpL8elWdWlVnJbkkyRuPsgwwBwGHwcwOdf9kkocluTHJ7UkuS/Lts0VenOTnk3wxay9G2xjP30xyxexV5D+7jRG2evwkeXPWDqt/IMnfJXntNtYDHIMXsQGT2vgiOeDEsAcOAAMScAAYkEPoADAge+AAMCABB4AB7eiZ2E477bQ+++yzd3KVwB5w8OBWJ5JjK+edd96iRzghdvJzY6c+hjfddFNuv/32LU++tKPPgS8vL/fKysqOrQ/YG6qmPJHct6a9+nqnnfzc2KmP4fLyclZWVrbcMIfQAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMaK6AV9UFVfWxqrqhqi6daigA4Ni2HfCq2pfkj5M8McmDkzyjqh481WAAwObm2QN/ZJIbuvsT3X1HkjckuXCasQCAY5kn4GckuWXd9cOz2/6fqrq4qlaqamV1dXWO1QEAR8wT8KOdaP2bzvTe3Qe6e7m7l/fv3z/H6gCAI+YJ+OEkZ627fmaST883DgBwPOYJ+PuSnFtVD6iqk5I8PclbphkLADiWpe2+Y3ffWVW/luTtSfYleV13Xz/ZZADAprYd8CTp7r9P8vcTzQIAHCdnYgOAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGNNfvgQPARlVH+1MZTM0eOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEBLO7mygwcPpqp2cpXwLaO7Fz0CsIPsgQPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABrTtgFfVWVX1rqo6VFXXV9UlUw4GAGxunnOh35nkRd19XVXdK8nBqrqmuz8y0WwAwCa2vQfe3bd293Wzy19McijJGVMNBgBsbpK/RlZVZyd5eJJrj3LfxUkunmI9AMCauQNeVackeVOSF3T3Fzbe390HkhyYLevvHQLABOZ6FXpV3S1r8b6yu6+eZiQAYCvzvAq9krw2yaHufvV0IwEAW5lnD/wxSZ6V5HFV9YHZvydNNBcAcAzbfg68u/8lSU04CwBwnJyJDQAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABjQJH/M5Hidd955WVlZ2clVAsCeZA8cAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwoKVFD3CiVNWiRwCAE8YeOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQHMHvKr2VdX7q+pvpxgIANjaFHvglyQ5NMHjAADHaa6AV9WZSZ6c5LJpxgEAjse8e+B/kOQlSb622QJVdXFVrVTVyurq6pyrAwCSOQJeVU9Jclt3HzzWct19oLuXu3t5//79210dALDOPHvgj0ny1Kq6Kckbkjyuqv58kqkAgGPadsC7+2XdfWZ3n53k6Un+sbufOdlkAMCm/B44AAxoaYoH6e53J3n3FI8FAGzNHjgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwoEl+D3w36u5FjwBMpKoWPQLsOvbAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABjRXwKvqPlV1VVV9tKoOVdWjpxoMANjc0pzv/4dJ3tbdP1NVJyU5eYKZAIAtbDvgVXXvJI9N8uwk6e47ktwxzVgAwLHMcwj9nCSrSf6sqt5fVZdV1T03LlRVF1fVSlWtrK6uzrE6AOCIeQK+lOQRSV7T3Q9P8qUkl25cqLsPdPdydy/v379/jtUBAEfME/DDSQ5397Wz61dlLegAwAm27YB3938muaWqHjS76fwkH5lkKgDgmOZ9Ffrzklw5ewX6J5L84vwjAQBbmSvg3f2BJMsTzQIAHCdnYgOAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKB5z8RGkqpa9AjsUt296BGAPcoeOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEBLix4AYCvdvegRuAt28v+rqnZsXbuNPXAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0FwBr6oXVtX1VfXhqnp9Vd19qsEAgM1tO+BVdUaS5ydZ7u6HJNmX5OlTDQYAbG7eQ+hLSe5RVUtJTk7y6flHAgC2su2Ad/enkvxekpuT3Jrk8939jo3LVdXFVbVSVSurq6vbnxQA+Lp5DqGfmuTCJA9IcnqSe1bVMzcu190Hunu5u5f379+//UkBgK+b5xD645Pc2N2r3f2VJFcn+eFpxgIAjmWegN+c5FFVdXKt/UHW85McmmYsAOBY5nkO/NokVyW5LsmHZo91YKK5AIBjWJrnnbv7lUleOdEsAMBxciY2ABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADmutELqzp7kWPALBrrJ1dmxPNHjgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAS4seYC+oqkWPwC7V3YseYU/wNTa/nfxc3Ml1fSt/btgDB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAFtGfCqel1V3VZVH15323dU1TVV9fHZ21NP7JgAwHrHswd+eZILNtx2aZJ3dve5Sd45uw4A7JAtA97d70ny2Q03X5jkitnlK5I8beK5AIBj2O5z4Pfr7luTZPb2vpstWFUXV9VKVa2srq5uc3UAwHon/EVs3X2gu5e7e3n//v0nenUA8C1huwH/TFV9Z5LM3t423UgAwFa2G/C3JLlodvmiJG+eZhwA4Hgcz6+RvT7Je5M8qKoOV9VzkrwqyROq6uNJnjC7DgDskKWtFujuZ2xy1/kTzwIAHCdnYgOAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKDq7p1bWdVqkk/exXc7LcntJ2CcRbNdY7FdY9mr25Xs3W2zXd/w3d295V//2tGAb0dVrXT38qLnmJrtGovtGste3a5k726b7brrHEIHgAEJOAAMaISAH1j0ACeI7RqL7RrLXt2uZO9um+26i3b9c+AAwDcbYQ8cANhgVwe8qi6oqo9V1Q1Vdemi55lCVZ1VVe+qqkNVdX1VXbLomaZUVfuq6v1V9beLnmUqVXWfqrqqqj46+3979KJnmkJVvXD2Ofjhqnp9Vd190TNtR1W9rqpuq6oPr7vtO6rqmqr6+OztqYuccTs22a7fnX0e/ntV/VVV3WeRM27H0bZr3X0vrqquqtMWMds8NtuuqnrerGPXV9XvTLnOXRvwqtqX5I+TPDHJg5M8o6oevNipJnFnkhd19/cneVSSX90j23XEJUkOLXqIif1hkrd19/cleWj2wPZV1RlJnp9kubsfkmRfkqcvdqptuzzJBRtuuzTJO7v73CTvnF0fzeX55u26JslDuvsHk/xHkpft9FATuDzfvF2pqrOSPCHJzTs90EQuz4btqqofT3Jhkh/s7h9I8ntTrnDXBjzJI5Pc0N2f6O47krwhax+IoXX3rd193ezyF7MWgzMWO9U0qurMJE9OctmiZ5lKVd07yWOTvDZJuvuO7v7cYqeazFKSe1TVUpKTk3x6wfNsS3e/J8lnN9x8YZIrZpevSPK0HR1qAkfbru5+R3ffObv6r0nO3PHB5rTJ/1eS/H6SlyQZ8oVZm2zXryR5VXf/72yZ26Zc524O+BlJbll3/XD2SOiOqKqzkzw8ybWLnWQyf5C1L8CvLXqQCZ2TZDXJn82eGrisqu656KHm1d2fytrewM1Jbk3y+e5+x2KnmtT9uvvWZO2H5iT3XfA8J8IvJXnrooeYQlU9NcmnuvuDi55lYg9M8iNVdW1V/VNV/dCUD76bA15HuW3In8yOpqpOSfKmJC/o7i8sep55VdVTktzW3QcXPcvElpI8IslruvvhSb6UMQ/H/j+z54QvTPKAJKcnuWdVPXOxU3G8quo3svZ03JWLnmVeVXVykt9I8opFz3ICLCU5NWtPl/56kr+oqqO1bVt2c8APJzlr3fUzM+ghvo2q6m5Zi/eV3X31oueZyGOSPLWqbsra0x2Pq6o/X+xIkzic5HB3HzlKclXWgj66xye5sbtXu/srSa5O8sMLnmlKn6mq70yS2dtJD10uUlVdlOQpSX6h98bvAX9P1n6Q/ODs+8eZSa6rqvsvdKppHE5yda/5t6wdnZzsBXq7OeDvS3JuVT2gqk7K2gts3rLgmeY2++nrtUkOdferFz3PVLr7Zd19ZnefnbX/q3/s7uH36Lr7P5PcUlUPmt10fpKPLHCkqdyc5FFVdfLsc/L87IEX563zliQXzS5flOTNC5xlMlV1QZKXJnlqd//PoueZQnd/qLvv291nz75/HE7yiNnX3uj+OsnjkqSqHpjkpEz4B1t2bcBnL9T4tSRvz9o3lr/o7usXO9UkHpPkWVnbQ/3A7N+TFj0Ux/S8JFdW1b8neViS317wPHObHVG4Ksl1ST6Ute8FQ54Jq6pen+S9SR5UVYer6jlJXpXkCVX18ay9svlVi5xxOzbZrj9Kcq8k18y+d/zJQofchk22a3ibbNfrkpwz+9WyNyS5aMqjJs7EBgAD2rV74ADA5gQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGND/Adcj4cKAmSYuAAAAAElFTkSuQmCC\n",
    +      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFaCAYAAADhKw9uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASOUlEQVR4nO3df4ztd13n8dd779hAKSwlvaj9oaVaUJao0JGARFYpxIJIMbvZBcUUf6SJP6AQFIsmaGI0ZDWoiQZTC7aJDailArqKdPEHmrDVuQWEclEaiu2FSoclCLrGWnz7x5yScXrnzvSc750zn9PHI7mZ8+M75/v+3Dszz/s958w51d0BAMbyn5Y9AADw4Ak4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOBwiFXVx6vq2Tsue2lV/cUEt91V9dWL3g6wHAIOAAMScBhYVZ1bVW+tqs2quqOqXr7tuqdW1Xur6rNVdXdV/UpVnTG77j2zzT5QVf9YVf+zqr6lqk5U1aur6p7Z57ywqp5XVX9bVZ+pqp/Yz+3Pru+qenlVfayqPl1VP19VfubARHwzwaBmMfy9JB9Icl6SS5O8oqq+bbbJF5K8Msk5SZ4+u/6HkqS7nznb5uu7+6zu/q3Z+S9L8rDZ7b02ya8neUmSS5J8c5LXVtVFe93+Nt+ZZD3JU5JcnuT7plg7kJTXQofDq6o+nq1A3rft4jOS3JrkVUl+p7u/Ytv2r0ny+O7+3pPc1iuS/Nfu/s7Z+U5ycXffPjv/LUn+MMlZ3f2Fqnpkks8leVp33zLb5liSn+nut+3z9p/b3e+cnf+hJP+tuy9d4K8EmFlb9gDAnl7Y3f/n/jNV9dIkP5DkK5OcW1Wf3bbtkSR/Ptvu8Ulen60j4DOz9f1+bI99/b/u/sLs9D/PPn5q2/X/nOSsB3H7d207/XdJzt1j/8A+uQsdxnVXkju6+9Hb/jyyu583u/4NST6SraPsRyX5iSQ14f73c/sXbDv9FUk+OeH+4SFNwGFcf5nkc1X141X18Ko6UlVPqqpvnF1//13g/1hVX5PkB3d8/qeSXJT57XX7SfJjVXV2VV2Q5Kokv3WSbYA5CDgManZX93ck+YYkdyT5dJJrk/zn2SY/muS7knw+W09G2xnPn05y/exZ5P9jjhH2uv0keXu27lZ/f5L/neSNc+wHOAlPYgNOi51PkgOm5QgcAAYk4AAwIHehA8CAHIEDwIAO9IVczjnnnL7wwgsPcpfAijh2bK/XoGE/LrnkkmWPcFoc9NfHQf49Hjt27NPdfXTn5Qd6F/r6+npvbGwc2P6A1VE15WvQPHSt6sOmB/31cZB/j1V1rLvXd17uLnQAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0EIBr6rLqupvqur2qrp6qqEAgFObO+BVdSTJryZ5bpInJnlxVT1xqsEAgN0tcgT+1CS3d/fHuvveJG9Jcvk0YwEAp7JIwM9Lcte28ydml/0HVXVlVW1U1cbm5uYCuwMA7rdIwE/21i8PeHuW7r6mu9e7e/3o0Qe8GxoAMIdFAn4iyQXbzp+f5JOLjQMA7MciAf+rJBdX1eOq6owkL0ryjmnGAgBOZW3eT+zu+6rqR5L8UZIjSd7U3bdNNhkAsKu5A54k3f0HSf5golkAgH3ySmwAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgBb6PXAA2E3Vyd4yg6k4AgeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAyouvvgdlZ1cDuDh6iD/J4+SFW17BFWwgH/zD+wfR20A/57PNbd6zsvdwQOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABjQ3AGvqguq6k+q6nhV3VZVV005GACwu7UFPve+JK/q7lur6pFJjlXVzd394YlmAwB2MfcReHff3d23zk5/PsnxJOdNNRgAsLtFjsC/qKouTPLkJLec5Lork1w5xX4AgC0Lv51oVZ2V5M+S/Gx337THtqv5PodwiHg7UU7F24lOY/i3E62qL0ny1iQ37BVvAGA6izwLvZK8Mcnx7n79dCMBAHtZ5Aj8GUm+J8mzqur9sz/Pm2guAOAU5n4SW3f/RZLVfYADAA4xr8QGAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAk7wb2X5dcskl2djYOMhdAsBKcgQOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0NqyBzhdqmrZIwDAaeMIHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwoIUDXlVHqup9VfX7UwwEAOxtiiPwq5Icn+B2AIB9WijgVXV+km9Pcu004wAA+7HoEfgvJXl1kn/bbYOqurKqNqpqY3Nzc8HdAQDJAgGvqucnuae7j51qu+6+prvXu3v96NGj8+4OANhmkSPwZyR5QVV9PMlbkjyrqn5zkqkAgFOaO+Dd/ZruPr+7L0zyoiR/3N0vmWwyAGBXfg8cAAa0NsWNdPefJvnTKW4LANibI3AAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQJP8Hvhh1N3LHgGYUFUtewQ4VByBA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGihgFfVo6vqxqr6SFUdr6qnTzUYALC7tQU//5eTvLO7/3tVnZHkzAlmAgD2MHfAq+pRSZ6Z5KVJ0t33Jrl3mrEAgFNZ5C70i5JsJvmNqnpfVV1bVY/YuVFVXVlVG1W1sbm5ucDuAID7LRLwtSRPSfKG7n5ykn9KcvXOjbr7mu5e7+71o0ePLrA7AOB+iwT8RJIT3X3L7PyN2Qo6AHCazR3w7v77JHdV1RNmF12a5MOTTAUAnNKiz0J/WZIbZs9A/1iS7118JABgLwsFvLvfn2R9olkAgH3ySmwAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAa06CuxkaSqlj0Ch1x3L3sEYMU4AgeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxobdkDAOxHdy97BB6kg/w3q6oD29dh4QgcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADCghQJeVa+sqtuq6kNV9eaqethUgwEAu5s74FV1XpKXJ1nv7iclOZLkRVMNBgDsbtG70NeSPLyq1pKcmeSTi48EAOxl7oB39yeS/EKSO5PcneQfuvtdO7erqiuraqOqNjY3N+efFAD4okXuQj87yeVJHpfk3CSPqKqX7Nyuu6/p7vXuXj969Oj8kwIAX7TIXejPTnJHd292978muSnJN00zFgBwKosE/M4kT6uqM2vrndQvTXJ8mrEAgFNZ5DHwW5LcmOTWJB+c3dY1E80FAJzC2iKf3N0/leSnJpoFANgnr8QGAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGih3wNnS3cvewSAQ2frRTo5XRyBA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADCgtWUPsAqqatkjcMh197JHGJ7vs2kc5NfiQe7rofj14QgcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABrRnwKvqTVV1T1V9aNtlj6mqm6vqo7OPZ5/eMQGA7fZzBH5dkst2XHZ1knd398VJ3j07DwAckD0D3t3vSfKZHRdfnuT62enrk7xw4rkAgFOY9zHwL+3uu5Nk9vGxu21YVVdW1UZVbWxubs65OwBgu9P+JLbuvqa717t7/ejRo6d7dwDwkDBvwD9VVV+eJLOP90w3EgCwl3kD/o4kV8xOX5Hk7dOMAwDsx35+jezNSd6b5AlVdaKqvj/J65I8p6o+muQ5s/MAwAFZ22uD7n7xLlddOvEsAMA+eSU2ABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADqu4+uJ1VbSb5uwf5aeck+fRpGOcwWNW1req6ktVd26quK1ndta3qupLVXdu86/rK7n7Au4EdaMDnUVUb3b2+7DlOh1Vd26quK1ndta3qupLVXduqritZ3bVNvS53oQPAgAQcAAY0QsCvWfYAp9Gqrm1V15Ws7tpWdV3J6q5tVdeVrO7aJl3XoX8MHAB4oBGOwAGAHQQcAAZ0qANeVZdV1d9U1e1VdfWy55lCVV1QVX9SVcer6raqumrZM02tqo5U1fuq6veXPctUqurRVXVjVX1k9m/39GXPNJWqeuXsa/FDVfXmqnrYsmeaR1W9qaruqaoPbbvsMVV1c1V9dPbx7GXOOK9d1vbzs6/Hv66q362qRy9zxnmcbF3brvvRquqqOmcZsy1qt7VV1ctmXbutqv7XIvs4tAGvqiNJfjXJc5M8McmLq+qJy51qEvcleVV3f22SpyX54RVZ13ZXJTm+7CEm9stJ3tndX5Pk67Mi66uq85K8PMl6dz8pyZEkL1ruVHO7LsllOy67Osm7u/viJO+enR/RdXng2m5O8qTu/rokf5vkNQc91ASuywPXlaq6IMlzktx50ANN6LrsWFtVfWuSy5N8XXf/lyS/sMgODm3Akzw1ye3d/bHuvjfJW7K18KF1993dfevs9OezFYLzljvVdKrq/CTfnuTaZc8ylap6VJJnJnljknT3vd392eVONam1JA+vqrUkZyb55JLnmUt3vyfJZ3ZcfHmS62enr0/ywgMdaiInW1t3v6u775ud/b9Jzj/wwRa0y79ZkvxiklcnGfZZ1rus7QeTvK67/2W2zT2L7OMwB/y8JHdtO38iKxS6JKmqC5M8Ockty51kUr+UrW+8f1v2IBO6KMlmkt+YPTRwbVU9YtlDTaG7P5Gto4A7k9yd5B+6+13LnWpSX9rddydb/3lO8tglz3O6fF+SP1z2EFOoqhck+UR3f2DZs5wGj0/yzVV1S1X9WVV94yI3dpgDXie5bNj/je1UVWcleWuSV3T355Y9zxSq6vlJ7unuY8ueZWJrSZ6S5A3d/eQk/5Rx74r9D2aPCV+e5HFJzk3yiKp6yXKn4sGoqp/M1kNzNyx7lkVV1ZlJfjLJa5c9y2myluTsbD18+mNJfruqTta6fTnMAT+R5IJt58/PoHft7VRVX5KteN/Q3Tcte54JPSPJC6rq49l6yONZVfWbyx1pEieSnOju++8puTFbQV8Fz05yR3dvdve/JrkpyTcteaYpfaqqvjxJZh8XusvysKmqK5I8P8l392q8qMdXZes/kx+Y/Rw5P8mtVfVlS51qOieS3NRb/jJb91TO/SS9wxzwv0pycVU9rqrOyNYTa96x5JkWNvvf1huTHO/u1y97nil192u6+/zuvjBb/15/3N3DH811998nuauqnjC76NIkH17iSFO6M8nTqurM2dfmpVmRJ+jNvCPJFbPTVyR5+xJnmVRVXZbkx5O8oLv//7LnmUJ3f7C7H9vdF85+jpxI8pTZ9+AqeFuSZyVJVT0+yRlZ4F3XDm3AZ0/O+JEkf5StHyi/3d23LXeqSTwjyfdk6+j0/bM/z1v2UOzpZUluqKq/TvINSX5uyfNMYnavwo1Jbk3ywWz9TBjyZSyr6s1J3pvkCVV1oqq+P8nrkjynqj6arWc1v26ZM85rl7X9SpJHJrl59nPk15Y65Bx2WddK2GVtb0py0exXy96S5IpF7jnxUqoAMKBDewQOAOxOwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKB/B24h+wUcnnY9AAAAAElFTkSuQmCC\n",
           "text/plain": [
    -       ""
    +       "
    " ] }, - "metadata": {}, + "metadata": { + "needs_background": "light" + }, "output_type": "display_data" } ], @@ -5779,10 +5753,8 @@ }, { "cell_type": "code", - "execution_count": 92, - "metadata": { - "collapsed": true - }, + "execution_count": 91, + "metadata": {}, "outputs": [], "source": [ "def P_motion_sample(kin_state, v, w):\n", @@ -5808,10 +5780,8 @@ }, { "cell_type": "code", - "execution_count": 93, - "metadata": { - "collapsed": true - }, + "execution_count": 92, + "metadata": {}, "outputs": [], "source": [ "def P_sensor(x, y):\n", @@ -5834,10 +5804,8 @@ }, { "cell_type": "code", - "execution_count": 94, - "metadata": { - "collapsed": true - }, + "execution_count": 93, + "metadata": {}, "outputs": [], "source": [ "a = {'v': (0, 0), 'w': 0}\n", @@ -5853,10 +5821,8 @@ }, { "cell_type": "code", - "execution_count": 95, - "metadata": { - "collapsed": true - }, + "execution_count": 94, + "metadata": {}, "outputs": [], "source": [ "S = monte_carlo_localization(a, z, 1000, P_motion_sample, P_sensor, m)" @@ -5871,7 +5837,7 @@ }, { "cell_type": "code", - "execution_count": 96, + "execution_count": 95, "metadata": {}, "outputs": [ { @@ -5879,27 +5845,29 @@ "output_type": "stream", "text": [ "GRID:\n", - " 0 0 9 41 123 12 1 0 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 0 2 107 56 4 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 5 4 9 2 0 0 0 0 0 0 0 0 0 0\n", - " 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - " 0 0 10 260 135 5 0 0 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 0 5 34 50 0 0 0 0 0 0 0 0 0 0\n", - "79 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "26 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 3 2 10 0 0 0 0 0 0 0 0 0 0 0\n", - " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n" + " 0 0 12 0 143 14 0 0 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 0 17 52 201 6 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 3 5 19 9 3 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + " 0 0 6 166 0 21 0 0 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 1 11 75 0 0 0 0 0 0 0 0 0 0 0\n", + " 73 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0\n", + "124 0 0 0 0 0 0 1 0 3 0 0 0 0 0 0 0\n", + " 0 0 0 14 4 15 1 0 0 0 0 0 0 0 0 0 0\n", + " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n" ] }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFYCAYAAACs465lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEqpJREFUeJzt3X+w5Xdd3/HXe3eT5heYmA1okoWQNoCUUUkvlB+VWgLTgEhg2mmhDRPQTma0QGBQDNpBO850mOpQndHBiQGTGTOgDSngLySiljJDo5sAQliUDInJQiS7ixhEbFjy7h/3rF6XvXt37/nuOfu5eTxmdu758b3n8/7u/fG833POPbe6OwDAWLYtewAA4PgJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgcBKqqnuq6vmHXfaqqvrIBLfdVfVP5r0dYLkEHAAGJOAwoKo6v6reU1X7quruqnrdmuueUVUfraovV9X9VfULVXXq7LoPzzb7RFX9dVX9+6r63qraW1VvqqoHZu/z0qp6UVX9WVV9qap+/Fhuf3Z9V9XrqupzVbW/qn6mqnyvgYn5ooLBzGL4G0k+keSCJJcleX1V/evZJt9I8oYkO5M8a3b9DydJdz93ts13dfdZ3f1rs/PfluS02e29JckvJ7kyyT9L8j1J3lJVF290+2u8LMlKkkuTXJHkB6bYd+DvlddCh5NPVd2T1UAeXHPxqUnuSPLGJP+zux+3Zvs3J3lid7/6CLf1+iT/srtfNjvfSS7p7rtm5783ye8kOau7v1FVj0ryYJJndvdts21uT/LT3f3eY7z9F3b3B2bnfzjJv+nuy+b4LwEOs2PZAwDreml3/96hM1X1qiT/Kcnjk5xfVV9es+32JP9ntt0Tk7wtq0fAZ2T16/z2DdY60N3fmJ3+2uztF9dc/7UkZx3H7d+35vSfJzl/g/WB4+QudBjPfUnu7u6z1/x7VHe/aHb925N8JqtH2Y9O8uNJasL1j+X2d605/bgkX5hwfSACDiP6oyQPVtWPVdXpVbW9qp5aVU+fXX/oLvC/rqonJ/mhw97/i0kuzuZtdPtJ8qNVdU5V7UpyTZJfO8I2wBwEHAYzu6v7+5N8d5K7k+xPcn2Sb5lt8iNJ/kOSr2T1yWiHx/Onktw4exb5v9vECBvdfpK8L6t3q388yW8leccm1gGOwpPYgEkd/iQ54MRwBA4AAxJwABiQu9ABYECOwAFgQAIOAANa6Cux7dx5bl/0uF0bbziafnhxa33jocWt9eUFvvbGuRctbi1/VwM4id1z733Zv//Ahi++tNCAX/S4Xdn9kd/beMPB9MG/Xdxaf3n3wtbKb/z0wpaqK395cWudeubC1gI4Xiv/4vnHtJ1DEQAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AA5or4FV1eVX9aVXdVVXXTjUUAHB0mw54VW1P8otJXpjkKUleUVVPmWowAGB98xyBPyPJXd39ue5+KMm7k1wxzVgAwNHME/ALkty35vze2WX/QFVdXVW7q2r3vv0H5lgOADhknoAf6S+l9Ddd0H1dd69098p5O8+dYzkA4JB5Ar43ydq/DXphkgX+/UkAeOSaJ+B/nOSSqnpCVZ2a5OVJ3j/NWADA0Wz674F398Gqek2S302yPck7u/vOySYDANa16YAnSXf/dpLfnmgWAOAYeSU2ABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEBz/R44q2rHaYtb67zvWNha/eqbFrfWe9+4uLUufdnC1qpdz17cWtu2L2wtYPkcgQPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAa0Y9kDcPKqqsWt9bK3LWwtgK3AETgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYECbDnhV7aqqP6iqPVV1Z1VdM+VgAMD65nkt9INJ3tjdd1TVo5LcXlW3dvenJ5oNAFjHpo/Au/v+7r5jdvorSfYkuWCqwQCA9U3yGHhVXZTkaUluO8J1V1fV7qravW//gSmWA4BHvLkDXlVnJXlPktd394OHX9/d13X3SnevnLfz3HmXAwAyZ8Cr6pSsxvum7r5lmpEAgI3M8yz0SvKOJHu6+23TjQQAbGSeI/DnJHllkudV1cdn/1400VwAwFFs+tfIuvsjSWrCWQCAY+SV2ABgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABzfPnRFmCfvjgAldb4K/5P/z1xa21/R8tbKnVFywEmJ4jcAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIB2LHsAjk9t26Ifsm3blz0BwFAcgQPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABjR3wKtqe1V9rKp+c4qBAICNTXEEfk2SPRPcDgBwjOYKeFVdmOT7klw/zTgAwLGY9wj855K8KcnD621QVVdX1e6q2r1v/4E5lwMAkjkCXlUvTvJAd99+tO26+7ruXunulfN2nrvZ5QCANeY5An9OkpdU1T1J3p3keVX1q5NMBQAc1aYD3t1v7u4Lu/uiJC9P8vvdfeVkkwEA6/J74AAwoB1T3Eh3/2GSP5zitgCAjTkCB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAY0ye+BP9L117+2sLX+6z+/eGFr/ZdXXbqwtba/5n0LW6u2+bQHxucIHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKAdyx5gK6hTTl/YWj91x/0LW6sfPri4tR78/OLW+uoDC1tr2wVPX9hawCOLI3AAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIDmCnhVnV1VN1fVZ6pqT1U9a6rBAID1zfta6D+f5APd/W+r6tQkZ0wwEwCwgU0HvKoeneS5SV6VJN39UJKHphkLADiaee5CvzjJviS/UlUfq6rrq+rMwzeqqqurandV7d63/8AcywEAh8wT8B1JLk3y9u5+WpKvJrn28I26+7ruXunulfN2njvHcgDAIfMEfG+Svd192+z8zVkNOgBwgm064N39F0nuq6onzS66LMmnJ5kKADiqeZ+F/tokN82egf65JK+efyQAYCNzBby7P55kZaJZAIBj5JXYAGBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMaN5XYjs+f3MgD+++YSFL1aWvXMg6SVLbti9srUWqbYv79KizH7+wtbLItQBOEEfgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAe1Y6GqnPTr15BcuZKnatn0h6wDAMjgCB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAHNFfCqekNV3VlVn6qqd1XVaVMNBgCsb9MBr6oLkrwuyUp3PzXJ9iQvn2owAGB9896FviPJ6VW1I8kZSb4w/0gAwEY2HfDu/nySn01yb5L7k/xVd3/w8O2q6uqq2l1Vu/cd+MvNTwoA/J157kI/J8kVSZ6Q5PwkZ1bVlYdv193XdfdKd6+cd+45m58UAPg789yF/vwkd3f3vu7+epJbkjx7mrEAgKOZJ+D3JnlmVZ1RVZXksiR7phkLADiaeR4Dvy3JzUnuSPLJ2W1dN9FcAMBR7Jjnnbv7J5P85ESzAADHyCuxAcCABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAY0Fwv5HLctp2SOuuxC11yq+l+eIGr1eKWOvi3C1uqTjl9YWsBnCiOwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAPasewBOD5VW/RnrlNOX/YEAEPZojUAgK1NwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGjDgFfVO6vqgar61JrLvrWqbq2qz87ennNixwQA1jqWI/Abklx+2GXXJvlQd1+S5EOz8wDAgmwY8O7+cJIvHXbxFUlunJ2+MclLJ54LADiKzT4G/tjuvj9JZm8fs96GVXV1Ve2uqt379h/Y5HIAwFon/Els3X1dd69098p5O8890csBwCPCZgP+xar69iSZvX1gupEAgI1sNuDvT3LV7PRVSd43zTgAwLE4ll8je1eSjyZ5UlXtraofTPLWJC+oqs8mecHsPACwIDs22qC7X7HOVZdNPAsAcIy8EhsADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAFVdy9usap9Sf78ON9tZ5L9J2CcZbNfY7FfY9mq+5Vs3X2zX3/v8d193kYbLTTgm1FVu7t7ZdlzTM1+jcV+jWWr7leydffNfh0/d6EDwIAEHAAGNELAr1v2ACeI/RqL/RrLVt2vZOvum/06Tif9Y+AAwDcb4QgcADjMSR3wqrq8qv60qu6qqmuXPc8UqmpXVf1BVe2pqjur6pplzzSlqtpeVR+rqt9c9ixTqaqzq+rmqvrM7OP2rGXPNIWqesPsc/BTVfWuqjpt2TNtRlW9s6oeqKpPrbnsW6vq1qr67OztOcuccTPW2a+fmX0e/klV/a+qOnuZM27GkfZrzXU/UlVdVTuXMds81tuvqnrtrGN3VtV/n3LNkzbgVbU9yS8meWGSpyR5RVU9ZblTTeJgkjd293ckeWaS/7xF9uuQa5LsWfYQE/v5JB/o7icn+a5sgf2rqguSvC7JSnc/Ncn2JC9f7lSbdkOSyw+77NokH+ruS5J8aHZ+NDfkm/fr1iRP7e7vTPJnSd686KEmcEO+eb9SVbuSvCDJvYseaCI35LD9qqp/leSKJN/Z3f80yc9OueBJG/Akz0hyV3d/rrsfSvLurP5HDK277+/uO2anv5LVGFyw3KmmUVUXJvm+JNcve5apVNWjkzw3yTuSpLsf6u4vL3eqyexIcnpV7UhyRpIvLHmeTenuDyf50mEXX5HkxtnpG5O8dKFDTeBI+9XdH+zug7Oz/zfJhQsfbE7rfLyS5H8keVOSIZ+Ytc5+/VCSt3b3/5tt88CUa57MAb8gyX1rzu/NFgndIVV1UZKnJbltuZNM5uey+gX48LIHmdDFSfYl+ZXZQwPXV9WZyx5qXt39+aweDdyb5P4kf9XdH1zuVJN6bHffn6z+0JzkMUue50T4gSS/s+whplBVL0ny+e7+xLJnmdgTk3xPVd1WVf+7qp4+5Y2fzAGvI1w25E9mR1JVZyV5T5LXd/eDy55nXlX14iQPdPfty55lYjuSXJrk7d39tCRfzZh3x/4Ds8eEr0jyhCTnJzmzqq5c7lQcq6r6iaw+HHfTsmeZV1WdkeQnkrxl2bOcADuSnJPVh0t/NMmvV9WR2rYpJ3PA9ybZteb8hRn0Lr7DVdUpWY33Td19y7Lnmchzkrykqu7J6sMdz6uqX13uSJPYm2Rvdx+6l+TmrAZ9dM9Pcnd37+vurye5JcmzlzzTlL5YVd+eJLO3k951uUxVdVWSFyf5j701fg/4H2f1B8lPzL5/XJjkjqr6tqVONY29SW7pVX+U1XsnJ3uC3skc8D9OcklVPaGqTs3qE2zev+SZ5jb76esdSfZ099uWPc9UuvvN3X1hd1+U1Y/V73f38Ed03f0XSe6rqifNLrosyaeXONJU7k3yzKo6Y/Y5eVm2wJPz1nh/kqtmp69K8r4lzjKZqro8yY8leUl3/82y55lCd3+yux/T3RfNvn/sTXLp7GtvdO9N8rwkqaonJjk1E/7BlpM24LMnarwmye9m9RvLr3f3ncudahLPSfLKrB6hfnz270XLHoqjem2Sm6rqT5J8d5L/tuR55ja7R+HmJHck+WRWvxcM+UpYVfWuJB9N8qSq2ltVP5jkrUleUFWfzeozm9+6zBk3Y539+oUkj0py6+x7xy8tdchNWGe/hrfOfr0zycWzXy17d5KrprzXxCuxAcCATtojcABgfQIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADOj/A0dU7lEBXyEDAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFaCAYAAADhKw9uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAATEElEQVR4nO3df6zldX3n8debuSK/F2Swlt+yi7pq2upOjdbU7Qqs+KNis5td7dJg2w1Ju1U0thZtIt1s0pi2cdukjV0WLSQl2i7S6nZbFW271qyLHVBUxFYiCKMIA4aCXSsF3vvHPSS317lzh3u+c858Lo9HMrn3nPO95/P+zNy5z/mee+6Z6u4AAGM5bNkDAACPn4ADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg6HsKq6varOXXfd66vqkxPcd1fVP5v3foDlEHAAGJCAw8Cq6uSq+kBV7a2q26rqjWtue0FVfaqq7q+qu6rqt6rq8Nltn5gddlNVfauq/n1V/UhV7amqt1bVPbOPeU1VvaKq/qaqvllVbz+Q+5/d3lX1xqr6SlXdW1W/VlW+5sBE/GWCQc1i+D+T3JTklCTnJHlTVb1sdsgjSd6cZGeSF81u/9kk6e6XzI75/u4+prt/f3b5aUmOmN3fO5L89yQXJvkXSX44yTuq6qzN7n+NH0uyK8nzk1yQ5Kem2DuQlNdCh0NXVd2e1UA+vObqw5PcmOQtSf5Hd5++5vi3JXlGd//kPu7rTUn+ZXf/2OxyJzm7u2+dXf6RJH+a5JjufqSqjk3yQJIXdvf1s2NuSPJfuvuPDvD+X97dH55d/tkk/6a7z5njtwSYWVn2AMCmXtPdH3vsQlW9Psl/THJGkpOr6v41x+5I8pez456R5F1ZPQM+Kqt/32/YZK37uvuR2fvfnr29e83t305yzOO4/zvXvP/VJCdvsj5wgDyEDuO6M8lt3X38ml/HdvcrZre/O8mXsnqWfVyStyepCdc/kPs/bc37pyf5+oTrwxOagMO4Pp3kgar6xao6sqp2VNVzq+oHZ7c/9hD4t6rqWUl+Zt3H353krGzdZvefJL9QVSdU1WlJLkny+/s4BtgCAYdBzR7q/tEkP5DktiT3JrkiyT+ZHfLzSX48yYNZfTLa+nj+cpKrZs8i/3dbGGGz+0+SD2b1YfXPJvlfSd6zhXWAffAkNuCgWP8kOWBazsABYEACDgAD8hA6AAzIGTgADGihL+Syc+eJfebpp21+4GgefWTzY6Z02I6FLfXQbZ9b2FqHn/Gcha21yN9DgHnc8Jmb7u3uk9Zfv9CAn3n6adn9yY9tfuBg+u8fWOh6dcRxC1vr9gtPWdhaZ1zxhwtbq444fmFrAcyjjj7pq/u63kPoADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKC5Al5V51fVX1fVrVV16VRDAQD7t+WAV9WOJL+d5OVJnp3kdVX17KkGAwA2Ns8Z+AuS3NrdX+nuh5K8P8kF04wFAOzPPAE/Jcmday7vmV33j1TVxVW1u6p27733vjmWAwAeM0/Aax/X9Xdd0X15d+/q7l0n7TxxjuUAgMfME/A9Sdb+596nJvn6fOMAAAdinoD/VZKzq+rpVXV4ktcm+dA0YwEA+7Oy1Q/s7oer6ueSfCTJjiTv7e6bJ5sMANjQlgOeJN39J0n+ZKJZAIAD5JXYAGBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAHN9XPgrKojjlv2CAfNGZfftLC1+va/XNha//nHL17YWkly2advX9hatfLkha0FLI8zcAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIBWlj0Ah7Y6aufi1nrWjy5srV++8a6FrQVwMDgDB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMaMsBr6rTqurPq+qWqrq5qi6ZcjAAYGPzvBb6w0ne0t03VtWxSW6oquu6+4sTzQYAbGDLZ+DdfVd33zh7/8EktyQ5ZarBAICNTfI98Ko6M8nzkly/j9surqrdVbV77733TbEcADzhzR3wqjomyQeSvKm7H1h/e3df3t27unvXSTtPnHc5ACBzBryqnpTVeF/d3ddOMxIAsJl5noVeSd6T5Jbuftd0IwEAm5nnDPzFSX4iyUur6rOzX6+YaC4AYD+2/GNk3f3JJDXhLADAAfJKbAAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABjTP/wfOkvSjjyxwsQWu9e37F7fW4Ucvbq0kWTliYUvVYTsWthawPM7AAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AA1pZ9gA8fnXYjgWutsC1jnnq4tYCGJwzcAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgOYOeFXtqKrPVNUfTzEQALC5Kc7AL0lyywT3AwAcoLkCXlWnJnllkiumGQcAOBDznoH/RpK3Jnl0owOq6uKq2l1Vu/fee9+cywEAyRwBr6pXJbmnu2/Y33HdfXl37+ruXSftPHGrywEAa8xzBv7iJK+uqtuTvD/JS6vq9yaZCgDYry0HvLvf1t2ndveZSV6b5M+6+8LJJgMANuTnwAFgQCtT3El3/0WSv5jivgCAzTkDB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAY0yc+BP9H1w99Z6HrXvfL0ha31rz9y98LW6ge/sbC16tinLWwtgIPBGTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AA1pZ9gDbQa08eaHrnffhbyxsrf7Og4tb6//8t4WtVS+7bGFrARwMzsABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQHMFvKqOr6prqupLVXVLVb1oqsEAgI3N+1Kqv5nkw939b6vq8CRHTTATALCJLQe8qo5L8pIkr0+S7n4oyUPTjAUA7M88D6GflWRvkt+tqs9U1RVVdfT6g6rq4qraXVW799573xzLAQCPmSfgK0men+Td3f28JH+X5NL1B3X35d29q7t3nbTzxDmWAwAeM0/A9yTZ093Xzy5fk9WgAwAH2ZYD3t3fSHJnVT1zdtU5Sb44yVQAwH7N+yz0NyS5evYM9K8k+cn5RwIANjNXwLv7s0l2TTQLAHCAvBIbAAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABzftKbI/PA3fl0Y/9ykKWOuzcty9knWWoqsUt9uRjF7ZUveyyha3FNLp7YWst9PMeBuAMHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMKCVRS72yAPfzLc+evVC1jru3LcvZB04EN29sLWqamFrpR9d3Fq1Y3FrwQCcgQPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABjRXwKvqzVV1c1V9oareV1VHTDUYALCxLQe8qk5J8sYku7r7uUl2JHntVIMBABub9yH0lSRHVtVKkqOSfH3+kQCAzWw54N39tSS/nuSOJHcl+dvu/uj646rq4qraXVW77/v2Av/nIgDYxuZ5CP2EJBckeXqSk5McXVUXrj+uuy/v7l3dvevEIz1nDgCmME9Rz01yW3fv7e5/SHJtkh+aZiwAYH/mCfgdSV5YVUdVVSU5J8kt04wFAOzPPN8Dvz7JNUluTPL52X1dPtFcAMB+rMzzwd19WZLLJpoFADhAnlUGAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADGiunwN/vHac+pwc96sfW+SS21L//f2LW+xJRy9urYceXNxaR5ywuLWSrL5Y4fZTh+1Y9gjwhOUMHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwIAEHgAEJOAAMSMABYEACDgADEnAAGJCAA8CABBwABiTgADAgAQeAAQk4AAxIwAFgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABrSx7AB6/OuL4ZY9wcBz5lGVPADAMZ+AAMCABB4ABCTgADEjAAWBAAg4AAxJwABiQgAPAgAQcAAYk4AAwoE0DXlXvrap7quoLa657SlVdV1Vfnr094eCOCQCsdSBn4FcmOX/ddZcm+Xh3n53k47PLAMCCbBrw7v5Ekm+uu/qCJFfN3r8qyWsmngsA2I+tfg/8e7r7riSZvX3qRgdW1cVVtbuqdu+9974tLgcArHXQn8TW3Zd3967u3nXSzhMP9nIA8ISw1YDfXVXfmySzt/dMNxIAsJmtBvxDSS6avX9Rkg9OMw4AcCAO5MfI3pfkU0meWVV7quqnk7wzyXlV9eUk580uAwALsrLZAd39ug1uOmfiWQCAA+SV2ABgQAIOAAMScAAYkIADwIAEHAAGJOAAMCABB4ABCTgADKi6e3GLVe1N8tXH+WE7k9x7EMY5FGzXvW3XfSXbd2/bdV/J9t3bdt1Xsn33ttV9ndHdJ62/cqEB34qq2t3du5Y9x8GwXfe2XfeVbN+9bdd9Jdt3b9t1X8n23dvU+/IQOgAMSMABYEAjBPzyZQ9wEG3XvW3XfSXbd2/bdV/J9t3bdt1Xsn33Num+DvnvgQMA322EM3AAYB0BB4ABHdIBr6rzq+qvq+rWqrp02fNMoapOq6o/r6pbqurmqrpk2TNNrap2VNVnquqPlz3LVKrq+Kq6pqq+NPuze9GyZ5pKVb159rn4hap6X1UdseyZtqKq3ltV91TVF9Zc95Squq6qvjx7e8IyZ9yqDfb2a7PPx89V1R9W1fHLnHEr9rWvNbf9fFV1Ve1cxmzz2mhvVfWGWddurqpfnWeNQzbgVbUjyW8neXmSZyd5XVU9e7lTTeLhJG/p7n+e5IVJ/tM22ddalyS5ZdlDTOw3k3y4u5+V5PuzTfZXVackeWOSXd393CQ7krx2uVNt2ZVJzl933aVJPt7dZyf5+OzyiK7Md+/tuiTP7e7vS/I3Sd626KEmcGW+e1+pqtOSnJfkjkUPNKErs25vVfWvklyQ5Pu6+zlJfn2eBQ7ZgCd5QZJbu/sr3f1QkvdndeND6+67uvvG2fsPZjUEpyx3qulU1alJXpnkimXPMpWqOi7JS5K8J0m6+6Huvn+5U01qJcmRVbWS5KgkX1/yPFvS3Z9I8s11V1+Q5KrZ+1clec1Ch5rIvvbW3R/t7odnF/9vklMXPticNvgzS5L/muStSYZ9lvUGe/uZJO/s7u/MjrlnnjUO5YCfkuTONZf3ZBuFLkmq6swkz0ty/XInmdRvZPUv3qPLHmRCZyXZm+R3Z98auKKqjl72UFPo7q9l9SzgjiR3Jfnb7v7ocqea1Pd0913J6j+ekzx1yfMcLD+V5E+XPcQUqurVSb7W3Tcte5aD4BlJfriqrq+q/11VPzjPnR3KAa99XDfsv8bWq6pjknwgyZu6+4FlzzOFqnpVknu6+4ZlzzKxlSTPT/Lu7n5ekr/LuA/F/iOz7wlfkOTpSU5OcnRVXbjcqXg8quqXsvqtuauXPcu8quqoJL+U5B3LnuUgWUlyQla/ffoLSf6gqvbVugNyKAd8T5LT1lw+NYM+tLdeVT0pq/G+uruvXfY8E3pxkldX1e1Z/ZbHS6vq95Y70iT2JNnT3Y89UnJNVoO+HZyb5Lbu3tvd/5Dk2iQ/tOSZpnR3VX1vkszezvWQ5aGmqi5K8qok/6G3x4t6/NOs/mPyptnXkVOT3FhVT1vqVNPZk+TaXvXprD5SueUn6R3KAf+rJGdX1dOr6vCsPrHmQ0ueaW6zf229J8kt3f2uZc8zpe5+W3ef2t1nZvXP68+6e/izue7+RpI7q+qZs6vOSfLFJY40pTuSvLCqjpp9bp6TbfIEvZkPJblo9v5FST64xFkmVVXnJ/nFJK/u7v+37Hmm0N2f7+6ndveZs68je5I8f/Z3cDv4oyQvTZKqekaSwzPH/7p2yAZ89uSMn0vykax+QfmD7r55uVNN4sVJfiKrZ6efnf16xbKHYlNvSHJ1VX0uyQ8k+ZUlzzOJ2aMK1yS5Mcnns/o1YciXsayq9yX5VJJnVtWeqvrpJO9Mcl5VfTmrz2p+5zJn3KoN9vZbSY5Nct3s68jvLHXILdhgX9vCBnt7b5KzZj9a9v4kF83zyImXUgWAAR2yZ+AAwMYEHAAGJOAAMCABB4ABCTgADEjAAWBAAg4AA/r/85kBLqIO9qEAAAAASUVORK5CYII=\n", "text/plain": [ - "" + "
    " ] }, - "metadata": {}, + "metadata": { + "needs_background": "light" + }, "output_type": "display_data" } ], @@ -5929,7 +5897,7 @@ }, { "cell_type": "code", - "execution_count": 97, + "execution_count": 96, "metadata": {}, "outputs": [ { @@ -5937,27 +5905,29 @@ "output_type": "stream", "text": [ "GRID:\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 999 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", - "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n" + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 1000 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", + "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n" ] }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFYCAYAAACs465lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEW5JREFUeJzt3X+s7wdd3/HXe702UAqj9halP7B0FhwjKt2VgEzmKGQFGcVs2WDDFHVpohMKQbFogiRLFjIN00TD0hVsExtQSyfMKVJRx0hY9baAUIpCaG0vVHpvCYLODMH3/jjf6vHSc8/t+X56v/d9eTySk/P98Tmfz/tz7znneT6f7/d8T3V3AIBZ/t6mBwAAHjoBB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnA4CVXVXVX13KNue3lVvX+BdXdVffO66wE2S8ABYCABh4Gq6tyqekdVHa6qO6vqldvue3pVfaCqPl9V91bVz1fV6av73rda7MNV9edV9W+q6rur6lBVvbaq7lt9zIur6gVV9cdV9bmq+onjWf/q/q6qV1bVp6rqSFX9dFX5XgML80UFw6xi+D+SfDjJeUkuTfKqqvrnq0W+kuTVSfYneebq/h9Oku5+9mqZb+vuM7v7l1fXvzHJI1bre32S/5bkZUn+cZLvSvL6qrpot/Vv871JDiS5JMnlSX5giX0H/lZ5LXQ4+VTVXdkK5Je33Xx6ktuSvCbJr3b3E7Yt/7okT+ru73+Qdb0qyT/t7u9dXe8kF3f3J1fXvzvJbyY5s7u/UlWPTvKFJM/o7ltWy9ya5D92968d5/qf393vXl3/4ST/srsvXeOfBDjKvk0PAOzoxd392w9cqaqXJ/n3Sb4pyblV9flty56W5H+vlntSkjdl6wj4jGx9nd+6y7bu7+6vrC7/5er9Z7fd/5dJznwI679n2+U/SXLuLtsHHiKn0GGee5Lc2d2P3fb26O5+wer+Nyf5eLaOsh+T5CeS1ILbP571X7Dt8hOSfGbB7QMRcJjo95N8oap+vKoeWVWnVdVTq+o7Vvc/cAr8z6vqW5L80FEf/9kkF2Xvdlt/kvxYVZ1VVRckuSrJLz/IMsAaBByGWZ3q/hdJvj3JnUmOJLk2yd9fLfKjSf5tki9m68loR8fzDUmuXz2L/F/vYYTd1p8k78zWafUPJfmfSd6yh+0Ax+BJbMCijn6SHPDwcAQOAAMJOAAM5BQ6AAzkCBwABhJwABjohL4S2/79Z/eFT7hg9wUB4GvUXXffkyNH7t/1xZdOaMAvfMIFOfj+3959QQD4GnXgnzz3uJZzCh0ABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgdYKeFVdVlV/VFWfrKqrlxoKADi2PQe8qk5L8gtJnp/kKUleWlVPWWowAGBn6xyBPz3JJ7v7U939pSRvT3L5MmMBAMeyTsDPS3LPtuuHVrf9HVV1ZVUdrKqDh4/cv8bmAIAHrBPwB/tLKf1VN3Rf090HuvvAOfvPXmNzAMAD1gn4oSTb/zbo+Uk+s944AMDxWCfgf5Dk4qp6YlWdnuQlSd61zFgAwLHs+e+Bd/eXq+pHkvxWktOSvLW7b19sMgBgR3sOeJJ0928k+Y2FZgEAjpNXYgOAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIH2HPCquqCqfreq7qiq26vqqiUHAwB2tm+Nj/1yktd0921V9egkt1bVzd39sYVmAwB2sOcj8O6+t7tvW13+YpI7kpy31GAAwM4WeQy8qi5M8rQktzzIfVdW1cGqOnj4yP1LbA4AvuatHfCqOjPJO5K8qru/cPT93X1Ndx/o7gPn7D973c0BAFkz4FX1ddmK9w3dfdMyIwEAu1nnWeiV5C1J7ujuNy03EgCwm3WOwJ+V5PuSPKeqPrR6e8FCcwEAx7DnXyPr7vcnqQVnAQCOk1diA4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgdYOeFWdVlUfrKpfX2IgAGB3SxyBX5XkjgXWAwAcp7UCXlXnJ/meJNcuMw4AcDzWPQL/2SSvTfLXOy1QVVdW1cGqOnj4yP1rbg4ASNYIeFW9MMl93X3rsZbr7mu6+0B3Hzhn/9l73RwAsM06R+DPSvKiqroryduTPKeqfmmRqQCAY9pzwLv7dd19fndfmOQlSX6nu1+22GQAwI78HjgADLRviZV09+8l+b0l1gUA7M4ROAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADDQvk0PAKeyN1zy+BO3rdvuPWHbAjbPETgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAw0FoBr6rHVtWNVfXxqrqjqp651GAAwM7WfS30n0vy7u7+V1V1epIzFpgJANjFngNeVY9J8uwkL0+S7v5Ski8tMxYAcCzrnEK/KMnhJL9YVR+sqmur6lFHL1RVV1bVwao6ePjI/WtsDgB4wDoB35fkkiRv7u6nJfmLJFcfvVB3X9PdB7r7wDn7z15jcwDAA9YJ+KEkh7r7ltX1G7MVdADgYbbngHf3nya5p6qevLrp0iQfW2QqAOCY1n0W+iuS3LB6Bvqnknz/+iMBALtZK+Dd/aEkBxaaBQA4Tl6JDQAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CB1n0lNuAY3nDbvZseAThFOQIHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgoLUCXlWvrqrbq+qjVfW2qnrEUoMBADvbc8Cr6rwkr0xyoLufmuS0JC9ZajAAYGfrnkLfl+SRVbUvyRlJPrP+SADAbvYc8O7+dJKfSXJ3knuT/Fl3v+fo5arqyqo6WFUHDx+5f++TAgB/Y51T6GcluTzJE5Ocm+RRVfWyo5fr7mu6+0B3Hzhn/9l7nxQA+BvrnEJ/bpI7u/twd/9VkpuSfOcyYwEAx7JOwO9O8oyqOqOqKsmlSe5YZiwA4FjWeQz8liQ3JrktyUdW67pmobkAgGPYt84Hd/dPJfmphWYBAI6TV2IDgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBdg14Vb21qu6rqo9uu+3rq+rmqvrE6v1ZD++YAMB2x3MEfl2Sy4667eok7+3ui5O8d3UdADhBdg14d78vyeeOuvnyJNevLl+f5MULzwUAHMNeHwP/hu6+N0lW7x+304JVdWVVHayqg4eP3L/HzQEA2z3sT2Lr7mu6+0B3Hzhn/9kP9+YA4GvCXgP+2ap6fJKs3t+33EgAwG72GvB3JblidfmKJO9cZhwA4Hgcz6+RvS3JB5I8uaoOVdUPJnljkudV1SeSPG91HQA4QfbttkB3v3SHuy5deBYA4Dh5JTYAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABqruPnEbqzqc5E8e4oftT3LkYRhn0+zXLPZrllN1v5JTd9/s19/6pu4+Z7eFTmjA96KqDnb3gU3PsTT7NYv9muVU3a/k1N03+/XQOYUOAAMJOAAMNCHg12x6gIeJ/ZrFfs1yqu5Xcurum/16iE76x8ABgK824QgcADjKSR3wqrqsqv6oqj5ZVVdvep4lVNUFVfW7VXVHVd1eVVdteqYlVdVpVfXBqvr1Tc+ylKp6bFXdWFUfX/2/PXPTMy2hql69+hz8aFW9raoesemZ9qKq3lpV91XVR7fd9vVVdXNVfWL1/qxNzrgXO+zXT68+D/+wqv57VT12kzPuxYPt17b7frSquqr2b2K2dey0X1X1ilXHbq+q/7zkNk/agFfVaUl+IcnzkzwlyUur6imbnWoRX07ymu7+h0mekeQ/nCL79YCrktyx6SEW9nNJ3t3d35Lk23IK7F9VnZfklUkOdPdTk5yW5CWbnWrPrkty2VG3XZ3kvd19cZL3rq5Pc12+er9uTvLU7v7WJH+c5HUneqgFXJev3q9U1QVJnpfk7hM90EKuy1H7VVX/LMnlSb61u/9Rkp9ZcoMnbcCTPD3JJ7v7U939pSRvz9Y/xGjdfW9337a6/MVsxeC8zU61jKo6P8n3JLl207Mspaoek+TZSd6SJN39pe7+/GanWsy+JI+sqn1JzkjymQ3Psyfd/b4knzvq5suTXL+6fH2SF5/QoRbwYPvV3e/p7i+vrv6fJOef8MHWtMP/V5L8lySvTTLyiVk77NcPJXljd/+/1TL3LbnNkzng5yW5Z9v1QzlFQveAqrowydOS3LLZSRbzs9n6AvzrTQ+yoIuSHE7yi6uHBq6tqkdteqh1dfens3U0cHeSe5P8WXe/Z7NTLeobuvveZOuH5iSP2/A8D4cfSPKbmx5iCVX1oiSf7u4Pb3qWhT0pyXdV1S1V9b+q6juWXPnJHPB6kNtG/mT2YKrqzCTvSPKq7v7CpudZV1W9MMl93X3rpmdZ2L4klyR5c3c/LclfZObp2L9j9Zjw5UmemOTcJI+qqpdtdiqOV1X9ZLYejrth07Osq6rOSPKTSV6/6VkeBvuSnJWth0t/LMmvVNWDtW1PTuaAH0pywbbr52foKb6jVdXXZSveN3T3TZueZyHPSvKiqrorWw93PKeqfmmzIy3iUJJD3f3AWZIbsxX06Z6b5M7uPtzdf5XkpiTfueGZlvTZqnp8kqzeL3rqcpOq6ookL0zy7/rU+D3gf5CtHyQ/vPr+cX6S26rqGzc61TIOJbmpt/x+ts5OLvYEvZM54H+Q5OKqemJVnZ6tJ9i8a8MzrW3109dbktzR3W/a9DxL6e7Xdff53X1htv6vfqe7xx/RdfefJrmnqp68uunSJB/b4EhLuTvJM6rqjNXn5KU5BZ6ct827klyxunxFknducJbFVNVlSX48yYu6+/9uep4ldPdHuvtx3X3h6vvHoSSXrL72pvu1JM9Jkqp6UpLTs+AfbDlpA756osaPJPmtbH1j+ZXuvn2zUy3iWUm+L1tHqB9avb1g00NxTK9IckNV/WGSb0/ynzY8z9pWZxRuTHJbko9k63vByFfCqqq3JflAkidX1aGq+sEkb0zyvKr6RLae2fzGTc64Fzvs188neXSSm1ffO/7rRofcgx32a7wd9uutSS5a/WrZ25NcseRZE6/EBgADnbRH4ADAzgQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgoP8PmFm83a4TWvMAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFaCAYAAADhKw9uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARl0lEQVR4nO3df6zld13n8dd7OzbQFpbaKUp/YOluwWWJSnckIJF1KWQLshSzmxV2MUXdNNEVCkGxaIIkm2zIalhNNJhuwTaxAd1SBV1FKv5gSdjqtFChFKWh0A5UOlOCoGu2Ft/7xz01l8vcucM9Z+bM+/J4JJN7fnzv+b4/nbn3eb/fc+5pdXcAgFn+0boHAAC+dgIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4HAKq6pPVdXzttz2iqr6wAoeu6vqny77OMB6CDgADCTgMFhVnVdV76yqw1V1T1W9atN9z6iqD1bVF6rq/qr6xao6fXHf+xeb3VFVf11V319V31NVh6rqdVX1wOJzXlJVL6yqv6iqz1fVTx3P4y/u76p6VVV9sqqOVNXPVpXvObAivphgqEUMfyvJHUnOT3JZkldX1b9ebPLlJK9Jsj/Jsxb3/2iSdPdzFtt8e3ef1d2/trj+zUketXi8NyT5H0lenuRfJPnuJG+oqot3evxNvi/JgSSXJrkiyQ+tYu1AUt4LHU5dVfWpbATy4U03n57k9iSvTfI/u/uJm7Z/fZInd/cPHuWxXp3kX3b39y2ud5JLuvvuxfXvSfK7Sc7q7i9X1WOSfDHJM7v71sU2tyX5L939m8f5+C/o7vcsrv9okn/b3Zct8Z8EWNi37gGAHb2ku3//kStV9Yok/ynJtyQ5r6q+sGnb05L878V2T07y5mwcAZ+Rja/323bY14Pd/eXF5b9dfPzcpvv/NslZX8Pj37fp8qeTnLfD/oHj5BQ6zHVfknu6+3Gb/jymu1+4uP8tST6ejaPsxyb5qSS1wv0fz+NfuOnyE5N8doX7h69rAg5z/UmSL1bVT1bVo6vqtKp6WlV95+L+R06B/3VVfWuSH9ny+Z9LcnF2b6fHT5KfqKqzq+rCJFcn+bWjbAPsgoDDUItT3f8myXckuSfJkSTXJfnHi01+PMl/SPKlbLwYbWs835jkhsWryP/9LkbY6fGT5F3ZOK3+4ST/K8lbd7Ef4Ci8iA04Iba+SA5YLUfgADCQgAPAQE6hA8BAjsABYKCT+kYu+/ef0xc98cKdNwQAkiS3feiOI9197tbbT2rAL3rihTn4gd/feUMAIElSZ5776aPd7hQ6AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMtFTAq+ryqvrzqrq7qq5Z1VAAwLHtOuBVdVqSX0rygiRPTfKyqnrqqgYDALa3zBH4M5Lc3d2f7O6HkrwjyRWrGQsAOJZlAn5+kvs2XT+0uO0rVNVVVXWwqg4ePvLgErsDAB6xTMDrKLf1V93QfW13H+juA+fuP2eJ3QEAj1gm4IeSbP6fe1+Q5LPLjQMAHI9lAv6nSS6pqidV1elJXprk3asZCwA4ln27/cTufriqfizJ7yU5LcnbuvvOlU0GAGxr1wFPku7+nSS/s6JZAIDj5J3YAGAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABlrq98CBU88bL33CydvX7feftH0BX8kROAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMNC+dQ8ArNYbb79/3SMAJ4EjcAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWCgXQe8qi6sqj+sqruq6s6qunqVgwEA21vmvdAfTvLa7r69qh6T5LaquqW7P7ai2QCAbez6CLy77+/u2xeXv5TkriTnr2owAGB7K3kOvKouSvL0JLce5b6rqupgVR08fOTBVewOAL7uLR3wqjoryTuTvLq7v7j1/u6+trsPdPeBc/efs+zuAIAsGfCq+oZsxPvG7r55NSMBADtZ5lXoleStSe7q7jevbiQAYCfLHIE/O8kPJHluVX148eeFK5oLADiGXf8aWXd/IEmtcBYA4Dh5JzYAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgoKUDXlWnVdWHquq3VzEQALCzVRyBX53krhU8DgBwnJYKeFVdkOR7k1y3mnEAgOOx7BH4zyd5XZK/326Dqrqqqg5W1cHDRx5ccncAQLJEwKvqRUke6O7bjrVdd1/b3Qe6+8C5+8/Z7e4AgE2WOQJ/dpIXV9WnkrwjyXOr6ldXMhUAcEy7Dnh3v767L+jui5K8NMkfdPfLVzYZALAtvwcOAAPtW8WDdPcfJfmjVTwWALAzR+AAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAwk4AAwkIADwEACDgADCTgADCTgADDQUgGvqsdV1U1V9fGququqnrWqwQCA7e1b8vN/Icl7uvvfVdXpSc5YwUwAwA52HfCqemyS5yR5RZJ090NJHlrNWADAsSxzCv3iJIeT/EpVfaiqrquqM7duVFVXVdXBqjp4+MiDS+wOAHjEMgHfl+TSJG/p7qcn+Zsk12zdqLuv7e4D3X3g3P3nLLE7AOARywT8UJJD3X3r4vpN2Qg6AHCC7Trg3f2XSe6rqqcsbrosycdWMhUAcEzLvgr9lUluXLwC/ZNJfnD5kQCAnSwV8O7+cJIDK5oFADhO3okNAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGGipgFfVa6rqzqr6aFW9vaoetarBAIDt7TrgVXV+klclOdDdT0tyWpKXrmowAGB7y55C35fk0VW1L8kZST67/EgAwE52HfDu/kySn0tyb5L7k/xVd79363ZVdVVVHayqg4ePPLj7SQGAf7DMKfSzk1yR5ElJzktyZlW9fOt23X1tdx/o7gPn7j9n95MCAP9gmVPoz0tyT3cf7u6/S3Jzku9azVgAwLEsE/B7kzyzqs6oqkpyWZK7VjMWAHAsyzwHfmuSm5LcnuQji8e6dkVzAQDHsG+ZT+7un0nyMyuaBQA4Tt6JDQAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABhJwABhIwAFgIAEHgIEEHAAGEnAAGEjAAWAgAQeAgQQcAAYScAAYSMABYCABB4CBBBwABtox4FX1tqp6oKo+uum2b6yqW6rqE4uPZ5/YMQGAzY7nCPz6JJdvue2aJO/r7kuSvG9xHQA4SXYMeHe/P8nnt9x8RZIbFpdvSPKSFc8FABzDbp8D/6buvj9JFh8fv92GVXVVVR2sqoOHjzy4y90BAJud8Bexdfe13X2guw+cu/+cE707APi6sNuAf66qnpAki48PrG4kAGAnuw34u5Ncubh8ZZJ3rWYcAOB4HM+vkb09yQeTPKWqDlXVDyd5U5LnV9Unkjx/cR0AOEn27bRBd79sm7suW/EsAMBx8k5sADCQgAPAQAIOAAMJOAAMJOAAMJCAA8BAAg4AAwk4AAxU3X3ydlZ1OMmnv8ZP25/kyAkY51SwV9e2V9eV7N217dV1JXt3bXt1XcneXdtu1/Ut3X3u1htPasB3o6oOdveBdc9xIuzVte3VdSV7d217dV3J3l3bXl1XsnfXtup1OYUOAAMJOAAMNCHg1657gBNor65tr64r2btr26vrSvbu2vbqupK9u7aVruuUfw4cAPhqE47AAYAtBBwABjqlA15Vl1fVn1fV3VV1zbrnWYWqurCq/rCq7qqqO6vq6nXPtGpVdVpVfaiqfnvds6xKVT2uqm6qqo8v/u6ete6ZVqWqXrP4t/jRqnp7VT1q3TPtRlW9raoeqKqPbrrtG6vqlqr6xOLj2euccbe2WdvPLv49/llV/UZVPW6dM+7G0da16b4fr6quqv3rmG1Z262tql656NqdVfXfltnHKRvwqjotyS8leUGSpyZ5WVU9db1TrcTDSV7b3f8syTOT/Oc9sq7Nrk5y17qHWLFfSPKe7v7WJN+ePbK+qjo/yauSHOjupyU5LclL1zvVrl2f5PItt12T5H3dfUmS9y2uT3R9vnpttyR5Wnd/W5K/SPL6kz3UClyfr15XqurCJM9Pcu/JHmiFrs+WtVXVv0pyRZJv6+5/nuTnltnBKRvwJM9Icnd3f7K7H0ryjmwsfLTuvr+7b19c/lI2QnD+eqdanaq6IMn3Jrlu3bOsSlU9Nslzkrw1Sbr7oe7+wnqnWql9SR5dVfuSnJHks2ueZ1e6+/1JPr/l5iuS3LC4fEOSl5zUoVbkaGvr7vd298OLq/8nyQUnfbAlbfN3liT/Pcnrkox9lfU2a/uRJG/q7v+32OaBZfZxKgf8/CT3bbp+KHsodElSVRcleXqSW9c7yUr9fDa+8P5+3YOs0MVJDif5lcVTA9dV1ZnrHmoVuvsz2TgKuDfJ/Un+qrvfu96pVuqbuvv+ZOOH5ySPX/M8J8oPJfnddQ+xClX14iSf6e471j3LCfDkJN9dVbdW1R9X1Xcu82CncsDrKLeN/Wlsq6o6K8k7k7y6u7+47nlWoapelOSB7r5t3bOs2L4klyZ5S3c/PcnfZO6p2K+weE74iiRPSnJekjOr6uXrnYqvRVX9dDaemrtx3bMsq6rOSPLTSd6w7llOkH1Jzs7G06c/keTXq+porTsup3LADyW5cNP1CzL01N5WVfUN2Yj3jd1987rnWaFnJ3lxVX0qG095PLeqfnW9I63EoSSHuvuRMyU3ZSPoe8HzktzT3Ye7+++S3Jzku9Y80yp9rqqekCSLj0udsjzVVNWVSV6U5D/23nhTj3+SjR8m71h8H7kgye1V9c1rnWp1DiW5uTf8STbOVO76RXqncsD/NMklVfWkqjo9Gy+sefeaZ1ra4qettya5q7vfvO55Vqm7X9/dF3T3Rdn4+/qD7h5/NNfdf5nkvqp6yuKmy5J8bI0jrdK9SZ5ZVWcs/m1elj3yAr2Fdye5cnH5yiTvWuMsK1VVlyf5ySQv7u7/u+55VqG7P9Ldj+/uixbfRw4luXTxNbgX/GaS5yZJVT05yelZ4v+6dsoGfPHijB9L8nvZ+Iby691953qnWolnJ/mBbBydfnjx54XrHoodvTLJjVX1Z0m+I8l/XfM8K7E4q3BTktuTfCQb3xNGvo1lVb09yQeTPKWqDlXVDyd5U5LnV9UnsvGq5jetc8bd2mZtv5jkMUluWXwf+eW1DrkL26xrT9hmbW9LcvHiV8vekeTKZc6ceCtVABjolD0CBwC2J+AAMJCAA8BAAg4AAwk4AAwk4AAwkIADwED/H3ZBvi8oWJldAAAAAElFTkSuQmCC\n", "text/plain": [ - "" + "
    " ] }, - "metadata": {}, + "metadata": { + "needs_background": "light" + }, "output_type": "display_data" } ], @@ -6010,7 +5980,7 @@ }, { "cell_type": "code", - "execution_count": 98, + "execution_count": 97, "metadata": {}, "outputs": [ { @@ -6018,7 +5988,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -6160,7 +6129,7 @@ }, { "cell_type": "code", - "execution_count": 99, + "execution_count": 98, "metadata": {}, "outputs": [ { @@ -6168,7 +6137,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -6349,7 +6317,7 @@ }, { "cell_type": "code", - "execution_count": 100, + "execution_count": 99, "metadata": {}, "outputs": [ { @@ -6357,7 +6325,6 @@ "text/html": [ "\n", - "\n", "\n", "\n", " Codestin Search App\n", @@ -6561,7 +6528,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.1" + "version": "3.6.9" } }, "nbformat": 4, diff --git a/probability.py b/probability.py index 458273b92..e1e77d224 100644 --- a/probability.py +++ b/probability.py @@ -1,31 +1,27 @@ -"""Probability models. (Chapter 13-15) -""" - -from utils import ( - product, argmax, element_wise_product, matrix_multiplication, - vector_to_diagonal, vector_add, scalar_vector_product, inverse_matrix, - weighted_sample_with_replacement, isclose, probability, normalize -) -from logic import extend -from agents import Agent +"""Probability models (Chapter 13-15)""" -import random from collections import defaultdict from functools import reduce -# ______________________________________________________________________________ +from agents import Agent +from utils import * def DTAgentProgram(belief_state): - """A decision-theoretic agent. [Figure 13.1]""" + """ + [Figure 13.1] + A decision-theoretic agent. + """ + def program(percept): belief_state.observe(program.action, percept) - program.action = argmax(belief_state.actions(), - key=belief_state.expected_outcome_utility) + program.action = max(belief_state.actions(), key=belief_state.expected_outcome_utility) return program.action + program.action = None return program + # ______________________________________________________________________________ @@ -39,14 +35,14 @@ class ProbDist: (0.125, 0.375, 0.5) """ - def __init__(self, varname='?', freqs=None): - """If freqs is given, it is a dictionary of values - frequency pairs, + def __init__(self, var_name='?', freq=None): + """If freq is given, it is a dictionary of values - frequency pairs, then ProbDist is normalized.""" self.prob = {} - self.varname = varname + self.var_name = var_name self.values = [] - if freqs: - for (v, p) in freqs.items(): + if freq: + for (v, p) in freq.items(): self[v] = p self.normalize() @@ -68,7 +64,7 @@ def normalize(self): Returns the normalized distribution. Raises a ZeroDivisionError if the sum of the values is 0.""" total = sum(self.prob.values()) - if not isclose(total, 1.0): + if not np.isclose(total, 1.0): for val in self.prob: self.prob[val] /= total return self @@ -76,11 +72,10 @@ def normalize(self): def show_approx(self, numfmt='{:.3g}'): """Show the probabilities rounded and sorted by key, for the sake of portable doctests.""" - return ', '.join([('{}: ' + numfmt).format(v, p) - for (v, p) in sorted(self.prob.items())]) + return ', '.join([('{}: ' + numfmt).format(v, p) for (v, p) in sorted(self.prob.items())]) def __repr__(self): - return "P({})".format(self.varname) + return "P({})".format(self.var_name) class JointProbDist(ProbDist): @@ -103,7 +98,7 @@ def __getitem__(self, values): return ProbDist.__getitem__(self, values) def __setitem__(self, values, p): - """Set P(values) = p. Values can be a tuple or a dict; it must + """Set P(values) = p. Values can be a tuple or a dict; it must have a value for each of the variables in the joint. Also keep track of the values we have seen so far for each variable.""" values = event_values(values, self.variables) @@ -132,12 +127,15 @@ def event_values(event, variables): else: return tuple([event[var] for var in variables]) + # ______________________________________________________________________________ def enumerate_joint_ask(X, e, P): - """Return a probability distribution over the values of the variable X, - given the {var:val} observations e, in the JointProbDist P. [Section 13.3] + """ + [Section 13.3] + Return a probability distribution over the values of the variable X, + given the {var:val} observations e, in the JointProbDist P. >>> P = JointProbDist(['X', 'Y']) >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[2,1] = 0.125 >>> enumerate_joint_ask('X', dict(Y=1), P).show_approx() @@ -157,8 +155,8 @@ def enumerate_joint(variables, e, P): if not variables: return P[e] Y, rest = variables[0], variables[1:] - return sum([enumerate_joint(rest, extend(e, Y, y), P) - for y in P.values(Y)]) + return sum([enumerate_joint(rest, extend(e, Y, y), P) for y in P.values(Y)]) + # ______________________________________________________________________________ @@ -233,9 +231,11 @@ def get_expected_utility(self, action, evidence): class InformationGatheringAgent(Agent): - """A simple information gathering agent. The agent works by repeatedly selecting + """ + [Figure 16.9] + A simple information gathering agent. The agent works by repeatedly selecting the observation with the highest information value, until the cost of the next - observation is greater than its expected benefit. [Figure 16.9]""" + observation is greater than its expected benefit.""" def __init__(self, decnet, infer, initial_evidence=None): """decnet: a decision network @@ -254,7 +254,7 @@ def execute(self, percept): """Execute the information gathering algorithm""" self.observation = self.integrate_percept(percept) vpis = self.vpi_cost_ratio(self.variables) - j = argmax(vpis) + j = max(vpis) variable = self.variables[j] if self.vpi(variable) > self.cost(variable): @@ -298,7 +298,7 @@ class BayesNode: def __init__(self, X, parents, cpt): """X is a variable name, and parents a sequence of variable - names or a space-separated string. cpt, the conditional + names or a space-separated string. cpt, the conditional probability table, takes one of these forms: * A number, the unconditional probability P(X=true). You can @@ -369,21 +369,22 @@ def __repr__(self): T, F = True, False -burglary = BayesNet([ - ('Burglary', '', 0.001), - ('Earthquake', '', 0.002), - ('Alarm', 'Burglary Earthquake', - {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}), - ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}), - ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01}) -]) +burglary = BayesNet([('Burglary', '', 0.001), + ('Earthquake', '', 0.002), + ('Alarm', 'Burglary Earthquake', + {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}), + ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}), + ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01})]) + # ______________________________________________________________________________ def enumeration_ask(X, e, bn): - """Return the conditional probability distribution of variable X - given evidence e, from BayesNet bn. [Figure 14.9] + """ + [Figure 14.9] + Return the conditional probability distribution of variable X + given evidence e, from BayesNet bn. >>> enumeration_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary ... ).show_approx() 'False: 0.716, True: 0.284'""" @@ -409,11 +410,14 @@ def enumerate_all(variables, e, bn): return sum(Ynode.p(y, e) * enumerate_all(rest, extend(e, Y, y), bn) for y in bn.variable_values(Y)) + # ______________________________________________________________________________ def elimination_ask(X, e, bn): - """Compute bn's P(X|e) by variable elimination. [Figure 14.11] + """ + [Figure 14.11] + Compute bn's P(X|e) by variable elimination. >>> elimination_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary ... ).show_approx() 'False: 0.716, True: 0.284'""" @@ -465,23 +469,20 @@ def __init__(self, variables, cpt): def pointwise_product(self, other, bn): """Multiply two factors, combining their variables.""" variables = list(set(self.variables) | set(other.variables)) - cpt = {event_values(e, variables): self.p(e) * other.p(e) - for e in all_events(variables, bn, {})} + cpt = {event_values(e, variables): self.p(e) * other.p(e) for e in all_events(variables, bn, {})} return Factor(variables, cpt) def sum_out(self, var, bn): """Make a factor eliminating var by summing over its values.""" variables = [X for X in self.variables if X != var] - cpt = {event_values(e, variables): sum(self.p(extend(e, var, val)) - for val in bn.variable_values(var)) + cpt = {event_values(e, variables): sum(self.p(extend(e, var, val)) for val in bn.variable_values(var)) for e in all_events(variables, bn, {})} return Factor(variables, cpt) def normalize(self): """Return my probabilities; must be down to one variable.""" assert len(self.variables) == 1 - return ProbDist(self.variables[0], - {k: v for ((k,), v) in self.cpt.items()}) + return ProbDist(self.variables[0], {k: v for ((k,), v) in self.cpt.items()}) def p(self, e): """Look up my value tabulated for e.""" @@ -498,35 +499,42 @@ def all_events(variables, bn, e): for x in bn.variable_values(X): yield extend(e1, X, x) + # ______________________________________________________________________________ # [Figure 14.12a]: sprinkler network -sprinkler = BayesNet([ - ('Cloudy', '', 0.5), - ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}), - ('Rain', 'Cloudy', {T: 0.80, F: 0.20}), - ('WetGrass', 'Sprinkler Rain', - {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})]) +sprinkler = BayesNet([('Cloudy', '', 0.5), + ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}), + ('Rain', 'Cloudy', {T: 0.80, F: 0.20}), + ('WetGrass', 'Sprinkler Rain', + {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})]) + # ______________________________________________________________________________ def prior_sample(bn): - """Randomly sample from bn's full joint distribution. The result - is a {variable: value} dict. [Figure 14.13]""" + """ + [Figure 14.13] + Randomly sample from bn's full joint distribution. + The result is a {variable: value} dict. + """ event = {} for node in bn.nodes: event[node.variable] = node.sample(event) return event + # _________________________________________________________________________ def rejection_sampling(X, e, bn, N=10000): - """Estimate the probability distribution of variable X given - evidence e in BayesNet bn, using N samples. [Figure 14.14] + """ + [Figure 14.14] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn, using N samples. Raises a ZeroDivisionError if all the N samples are rejected, i.e., inconsistent with e. >>> random.seed(47) @@ -544,15 +552,17 @@ def rejection_sampling(X, e, bn, N=10000): def consistent_with(event, evidence): """Is event consistent with the given evidence?""" - return all(evidence.get(k, v) == v - for k, v in event.items()) + return all(evidence.get(k, v) == v for k, v in event.items()) + # _________________________________________________________________________ def likelihood_weighting(X, e, bn, N=10000): - """Estimate the probability distribution of variable X given - evidence e in BayesNet bn. [Figure 14.15] + """ + [Figure 14.15] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn. >>> random.seed(1017) >>> likelihood_weighting('Burglary', dict(JohnCalls=T, MaryCalls=T), ... burglary, 10000).show_approx() @@ -566,9 +576,11 @@ def likelihood_weighting(X, e, bn, N=10000): def weighted_sample(bn, e): - """Sample an event from bn that's consistent with the evidence e; + """ + Sample an event from bn that's consistent with the evidence e; return the event and its weight, the likelihood that the event - accords to the evidence.""" + accords to the evidence. + """ w = 1 event = dict(e) # boldface x in [Figure 14.15] for node in bn.nodes: @@ -579,6 +591,7 @@ def weighted_sample(bn, e): event[Xi] = node.sample(event) return event, w + # _________________________________________________________________________ @@ -606,12 +619,12 @@ def markov_blanket_sample(X, e, bn): Q = ProbDist(X) for xi in bn.variable_values(X): ei = extend(e, X, xi) - # [Equation 14.12:] - Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei) - for Yj in Xnode.children) + # [Equation 14.12] + Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei) for Yj in Xnode.children) # (assuming a Boolean variable here) return probability(Q.normalize()[True]) + # _________________________________________________________________________ @@ -646,52 +659,95 @@ def backward(HMM, b, ev): scalar_vector_product(prediction[1], HMM.transition_model[1]))) -def forward_backward(HMM, ev, prior): - """[Figure 15.4] +def forward_backward(HMM, ev): + """ + [Figure 15.4] Forward-Backward algorithm for smoothing. Computes posterior probabilities - of a sequence of states given a sequence of observations.""" + of a sequence of states given a sequence of observations. + """ t = len(ev) ev.insert(0, None) # to make the code look similar to pseudo code fv = [[0.0, 0.0] for _ in range(len(ev))] b = [1.0, 1.0] - bv = [b] # we don't need bv; but we will have a list of all backward messages here sv = [[0, 0] for _ in range(len(ev))] - fv[0] = prior + fv[0] = HMM.prior for i in range(1, t + 1): fv[i] = forward(HMM, fv[i - 1], ev[i]) for i in range(t, -1, -1): sv[i - 1] = normalize(element_wise_product(fv[i], b)) b = backward(HMM, b, ev[i]) - bv.append(b) sv = sv[::-1] return sv + +def viterbi(HMM, ev): + """ + [Equation 15.11] + Viterbi algorithm to find the most likely sequence. Computes the best path and the + corresponding probabilities, given an HMM model and a sequence of observations. + """ + t = len(ev) + ev = ev.copy() + ev.insert(0, None) + + m = [[0.0, 0.0] for _ in range(len(ev) - 1)] + + # the recursion is initialized with m1 = forward(P(X0), e1) + m[0] = forward(HMM, HMM.prior, ev[1]) + # keep track of maximizing predecessors + backtracking_graph = [] + + for i in range(1, t): + m[i] = element_wise_product(HMM.sensor_dist(ev[i + 1]), + [max(element_wise_product(HMM.transition_model[0], m[i - 1])), + max(element_wise_product(HMM.transition_model[1], m[i - 1]))]) + backtracking_graph.append([np.argmax(element_wise_product(HMM.transition_model[0], m[i - 1])), + np.argmax(element_wise_product(HMM.transition_model[1], m[i - 1]))]) + + # computed probabilities + ml_probabilities = [0.0] * (len(ev) - 1) + # most likely sequence + ml_path = [True] * (len(ev) - 1) + + # the construction of the most likely sequence starts in the final state with the largest probability, and + # runs backwards; the algorithm needs to store for each xt its predecessor xt-1 maximizing its probability + i_max = np.argmax(m[-1]) + + for i in range(t - 1, -1, -1): + ml_probabilities[i] = m[i][i_max] + ml_path[i] = True if i_max == 0 else False + if i > 0: + i_max = backtracking_graph[i - 1][i_max] + + return ml_path, ml_probabilities + + # _________________________________________________________________________ def fixed_lag_smoothing(e_t, HMM, d, ev, t): - """[Figure 15.6] + """ + [Figure 15.6] Smoothing algorithm with a fixed time lag of 'd' steps. Online algorithm that outputs the new smoothed estimate if observation - for new time step is given.""" + for new time step is given. + """ ev.insert(0, None) T_model = HMM.transition_model f = HMM.prior B = [[1, 0], [0, 1]] - evidence = [] - evidence.append(e_t) - O_t = vector_to_diagonal(HMM.sensor_dist(e_t)) + O_t = np.diag(HMM.sensor_dist(e_t)) if t > d: f = forward(HMM, f, e_t) - O_tmd = vector_to_diagonal(HMM.sensor_dist(ev[t - d])) - B = matrix_multiplication(inverse_matrix(O_tmd), inverse_matrix(T_model), B, T_model, O_t) + O_tmd = np.diag(HMM.sensor_dist(ev[t - d])) + B = matrix_multiplication(np.linalg.inv(O_tmd), np.linalg.inv(T_model), B, T_model, O_t) else: B = matrix_multiplication(B, T_model, O_t) t += 1 @@ -702,6 +758,7 @@ def fixed_lag_smoothing(e_t, HMM, d, ev, t): else: return None + # _________________________________________________________________________ @@ -737,18 +794,19 @@ def particle_filtering(e, N, HMM): w[i] = float("{0:.4f}".format(w[i])) # STEP 2 - s = weighted_sample_with_replacement(N, s, w) return s + # _________________________________________________________________________ -## TODO: Implement continuous map for MonteCarlo similar to Fig25.10 from the book +# TODO: Implement continuous map for MonteCarlo similar to Fig25.10 from the book class MCLmap: """Map which provides probability distributions and sensor readings. Consists of discrete cells which are either an obstacle or empty""" + def __init__(self, m): self.m = m self.nrows = len(m) @@ -765,34 +823,36 @@ def sample(self): return kin_state def ray_cast(self, sensor_num, kin_state): - """Returns distace to nearest obstacle or map boundary in the direction of sensor""" + """Returns distance to nearest obstacle or map boundary in the direction of sensor""" pos = kin_state[:2] orient = kin_state[2] # sensor layout when orientation is 0 (towards North) # 0 # 3R1 # 2 - delta = ((sensor_num % 2 == 0)*(sensor_num - 1), (sensor_num % 2 == 1)*(2 - sensor_num)) + delta = ((sensor_num % 2 == 0) * (sensor_num - 1), (sensor_num % 2 == 1) * (2 - sensor_num)) # sensor direction changes based on orientation for _ in range(orient): delta = (delta[1], -delta[0]) range_count = 0 - while (0 <= pos[0] < self.nrows) and (0 <= pos[1] < self.nrows) and (not self.m[pos[0]][pos[1]]): + while 0 <= pos[0] < self.nrows and 0 <= pos[1] < self.nrows and not self.m[pos[0]][pos[1]]: pos = vector_add(pos, delta) range_count += 1 return range_count def monte_carlo_localization(a, z, N, P_motion_sample, P_sensor, m, S=None): - """Monte Carlo localization algorithm from Fig 25.9""" + """ + [Figure 25.9] + Monte Carlo localization algorithm + """ def ray_cast(sensor_num, kin_state, m): return m.ray_cast(sensor_num, kin_state) M = len(z) - W = [0]*N - S_ = [0]*N - W_ = [0]*N + S_ = [0] * N + W_ = [0] * N v = a['v'] w = a['w'] diff --git a/probability-4e.ipynb b/probability4e.ipynb similarity index 100% rename from probability-4e.ipynb rename to probability4e.ipynb diff --git a/probability4e.py b/probability4e.py new file mode 100644 index 000000000..d413a55ae --- /dev/null +++ b/probability4e.py @@ -0,0 +1,776 @@ +"""Probability models (Chapter 12-13)""" + +import copy +import random +from collections import defaultdict +from functools import reduce + +import numpy as np + +from utils4e import product, probability, extend + + +# ______________________________________________________________________________ +# Chapter 12 Qualifying Uncertainty +# 12.1 Acting Under Uncertainty + + +def DTAgentProgram(belief_state): + """A decision-theoretic agent. [Figure 12.1]""" + + def program(percept): + belief_state.observe(program.action, percept) + program.action = max(belief_state.actions(), key=belief_state.expected_outcome_utility) + return program.action + + program.action = None + return program + + +# ______________________________________________________________________________ +# 12.2 Basic Probability Notation + + +class ProbDist: + """A discrete probability distribution. You name the random variable + in the constructor, then assign and query probability of values. + >>> P = ProbDist('Flip'); P['H'], P['T'] = 0.25, 0.75; P['H'] + 0.25 + >>> P = ProbDist('X', {'lo': 125, 'med': 375, 'hi': 500}) + >>> P['lo'], P['med'], P['hi'] + (0.125, 0.375, 0.5) + """ + + def __init__(self, varname='?', freqs=None): + """If freqs is given, it is a dictionary of values - frequency pairs, + then ProbDist is normalized.""" + self.prob = {} + self.varname = varname + self.values = [] + if freqs: + for (v, p) in freqs.items(): + self[v] = p + self.normalize() + + def __getitem__(self, val): + """Given a value, return P(value).""" + try: + return self.prob[val] + except KeyError: + return 0 + + def __setitem__(self, val, p): + """Set P(val) = p.""" + if val not in self.values: + self.values.append(val) + self.prob[val] = p + + def normalize(self): + """Make sure the probabilities of all values sum to 1. + Returns the normalized distribution. + Raises a ZeroDivisionError if the sum of the values is 0.""" + total = sum(self.prob.values()) + if not np.isclose(total, 1.0): + for val in self.prob: + self.prob[val] /= total + return self + + def show_approx(self, numfmt='{:.3g}'): + """Show the probabilities rounded and sorted by key, for the + sake of portable doctests.""" + return ', '.join([('{}: ' + numfmt).format(v, p) + for (v, p) in sorted(self.prob.items())]) + + def __repr__(self): + return "P({})".format(self.varname) + + +# ______________________________________________________________________________ +# 12.3 Inference Using Full Joint Distributions + + +class JointProbDist(ProbDist): + """A discrete probability distribute over a set of variables. + >>> P = JointProbDist(['X', 'Y']); P[1, 1] = 0.25 + >>> P[1, 1] + 0.25 + >>> P[dict(X=0, Y=1)] = 0.5 + >>> P[dict(X=0, Y=1)] + 0.5""" + + def __init__(self, variables): + self.prob = {} + self.variables = variables + self.vals = defaultdict(list) + + def __getitem__(self, values): + """Given a tuple or dict of values, return P(values).""" + values = event_values(values, self.variables) + return ProbDist.__getitem__(self, values) + + def __setitem__(self, values, p): + """Set P(values) = p. Values can be a tuple or a dict; it must + have a value for each of the variables in the joint. Also keep track + of the values we have seen so far for each variable.""" + values = event_values(values, self.variables) + self.prob[values] = p + for var, val in zip(self.variables, values): + if val not in self.vals[var]: + self.vals[var].append(val) + + def values(self, var): + """Return the set of possible values for a variable.""" + return self.vals[var] + + def __repr__(self): + return "P({})".format(self.variables) + + +def event_values(event, variables): + """Return a tuple of the values of variables in event. + >>> event_values ({'A': 10, 'B': 9, 'C': 8}, ['C', 'A']) + (8, 10) + >>> event_values ((1, 2), ['C', 'A']) + (1, 2) + """ + if isinstance(event, tuple) and len(event) == len(variables): + return event + else: + return tuple([event[var] for var in variables]) + + +def enumerate_joint_ask(X, e, P): + """Return a probability distribution over the values of the variable X, + given the {var:val} observations e, in the JointProbDist P. [Section 12.3] + >>> P = JointProbDist(['X', 'Y']) + >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[2,1] = 0.125 + >>> enumerate_joint_ask('X', dict(Y=1), P).show_approx() + '0: 0.667, 1: 0.167, 2: 0.167' + """ + assert X not in e, "Query variable must be distinct from evidence" + Q = ProbDist(X) # probability distribution for X, initially empty + Y = [v for v in P.variables if v != X and v not in e] # hidden variables. + for xi in P.values(X): + Q[xi] = enumerate_joint(Y, extend(e, X, xi), P) + return Q.normalize() + + +def enumerate_joint(variables, e, P): + """Return the sum of those entries in P consistent with e, + provided variables is P's remaining variables (the ones not in e).""" + if not variables: + return P[e] + Y, rest = variables[0], variables[1:] + return sum([enumerate_joint(rest, extend(e, Y, y), P) + for y in P.values(Y)]) + + +# ______________________________________________________________________________ +# 12.4 Independence + + +def is_independent(variables, P): + """ + Return whether a list of variables are independent given their distribution P + P is an instance of JoinProbDist + >>> P = JointProbDist(['X', 'Y']) + >>> P[0,0] = 0.25; P[0,1] = 0.5; P[1,1] = P[1,0] = 0.125 + >>> is_independent(['X', 'Y'], P) + False + """ + for var in variables: + event_vars = variables[:] + event_vars.remove(var) + event = {} + distribution = enumerate_joint_ask(var, event, P) + events = gen_possible_events(event_vars, P) + for e in events: + conditional_distr = enumerate_joint_ask(var, e, P) + if conditional_distr.prob != distribution.prob: + return False + return True + + +def gen_possible_events(vars, P): + """Generate all possible events of a collection of vars according to distribution of P""" + events = [] + + def backtrack(vars, P, temp): + if not vars: + events.append(temp) + return + var = vars[0] + for val in P.values(var): + temp[var] = val + backtrack([v for v in vars if v != var], P, copy.copy(temp)) + + backtrack(vars, P, {}) + return events + + +# ______________________________________________________________________________ +# Chapter 13 Probabilistic Reasoning +# 13.1 Representing Knowledge in an Uncertain Domain + + +class BayesNet: + """Bayesian network containing only boolean-variable nodes.""" + + def __init__(self, node_specs=None): + """ + Nodes must be ordered with parents before children. + :param node_specs: an nested iterable object, each element contains (variable name, parents name, cpt) + for each node + """ + + self.nodes = [] + self.variables = [] + node_specs = node_specs or [] + for node_spec in node_specs: + self.add(node_spec) + + def add(self, node_spec): + """ + Add a node to the net. Its parents must already be in the + net, and its variable must not. + Initialize Bayes nodes by detecting the length of input node specs + """ + if len(node_spec) >= 5: + node = ContinuousBayesNode(*node_spec) + else: + node = BayesNode(*node_spec) + assert node.variable not in self.variables + assert all((parent in self.variables) for parent in node.parents) + self.nodes.append(node) + self.variables.append(node.variable) + for parent in node.parents: + self.variable_node(parent).children.append(node) + + def variable_node(self, var): + """ + Return the node for the variable named var. + >>> burglary.variable_node('Burglary').variable + 'Burglary' + """ + for n in self.nodes: + if n.variable == var: + return n + raise Exception("No such variable: {}".format(var)) + + def variable_values(self, var): + """Return the domain of var.""" + return [True, False] + + def __repr__(self): + return 'BayesNet({0!r})'.format(self.nodes) + + +class BayesNode: + """ + A conditional probability distribution for a boolean variable, + P(X | parents). Part of a BayesNet. + """ + + def __init__(self, X, parents, cpt): + """ + :param X: variable name, + :param parents: a sequence of variable names or a space-separated string. Representing the names of parent nodes + :param cpt: the conditional probability table, takes one of these forms: + + * A number, the unconditional probability P(X=true). You can + use this form when there are no parents. + + * A dict {v: p, ...}, the conditional probability distribution + P(X=true | parent=v) = p. When there's just one parent. + + * A dict {(v1, v2, ...): p, ...}, the distribution P(X=true | + parent1=v1, parent2=v2, ...) = p. Each key must have as many + values as there are parents. You can use this form always; + the first two are just conveniences. + + In all cases the probability of X being false is left implicit, + since it follows from P(X=true). + + >>> X = BayesNode('X', '', 0.2) + >>> Y = BayesNode('Y', 'P', {T: 0.2, F: 0.7}) + >>> Z = BayesNode('Z', 'P Q', + ... {(T, T): 0.2, (T, F): 0.3, (F, T): 0.5, (F, F): 0.7}) + """ + if isinstance(parents, str): + parents = parents.split() + + # We store the table always in the third form above. + if isinstance(cpt, (float, int)): # no parents, 0-tuple + cpt = {(): cpt} + elif isinstance(cpt, dict): + # one parent, 1-tuple + if cpt and isinstance(list(cpt.keys())[0], bool): + cpt = {(v,): p for v, p in cpt.items()} + + assert isinstance(cpt, dict) + for vs, p in cpt.items(): + assert isinstance(vs, tuple) and len(vs) == len(parents) + assert all(isinstance(v, bool) for v in vs) + assert 0 <= p <= 1 + + self.variable = X + self.parents = parents + self.cpt = cpt + self.children = [] + + def p(self, value, event): + """ + Return the conditional probability + P(X=value | parents=parent_values), where parent_values + are the values of parents in event. (event must assign each + parent a value.) + >>> bn = BayesNode('X', 'Burglary', {T: 0.2, F: 0.625}) + >>> bn.p(False, {'Burglary': False, 'Earthquake': True}) + 0.375 + """ + assert isinstance(value, bool) + ptrue = self.cpt[event_values(event, self.parents)] + return ptrue if value else 1 - ptrue + + def sample(self, event): + """ + Sample from the distribution for this variable conditioned + on event's values for parent_variables. That is, return True/False + at random according with the conditional probability given the + parents. + """ + return probability(self.p(True, event)) + + def __repr__(self): + return repr((self.variable, ' '.join(self.parents))) + + +# Burglary example [Figure 13 .2] + + +T, F = True, False + +burglary = BayesNet([ + ('Burglary', '', 0.001), + ('Earthquake', '', 0.002), + ('Alarm', 'Burglary Earthquake', + {(T, T): 0.95, (T, F): 0.94, (F, T): 0.29, (F, F): 0.001}), + ('JohnCalls', 'Alarm', {T: 0.90, F: 0.05}), + ('MaryCalls', 'Alarm', {T: 0.70, F: 0.01}) +]) + + +# ______________________________________________________________________________ +# Section 13.2. The Semantics of Bayesian Networks +# Bayesian nets with continuous variables + + +def gaussian_probability(param, event, value): + """ + Gaussian probability of a continuous Bayesian network node on condition of + certain event and the parameters determined by the event + :param param: parameters determined by discrete parent events of current node + :param event: a dict, continuous event of current node, the values are used + as parameters in calculating distribution + :param value: float, the value of current continuous node + :return: float, the calculated probability + >>> param = {'sigma':0.5, 'b':1, 'a':{'h1':0.5, 'h2': 1.5}} + >>> event = {'h1':0.6, 'h2': 0.3} + >>> gaussian_probability(param, event, 1) + 0.2590351913317835 + """ + + assert isinstance(event, dict) + assert isinstance(param, dict) + buff = 0 + for k, v in event.items(): + # buffer varianle to calculate h1*a_h1 + h2*a_h2 + buff += param['a'][k] * v + res = 1 / (param['sigma'] * np.sqrt(2 * np.pi)) * np.exp(-0.5 * ((value - buff - param['b']) / param['sigma']) ** 2) + return res + + +def logistic_probability(param, event, value): + """ + Logistic probability of a discrete node in Bayesian network with continuous parents, + :param param: a dict, parameters determined by discrete parents of current node + :param event: a dict, names and values of continuous parent variables of current node + :param value: boolean, True or False + :return: int, probability + """ + + buff = 1 + for _, v in event.items(): + # buffer variable to calculate (value-mu)/sigma + + buff *= (v - param['mu']) / param['sigma'] + p = 1 - 1 / (1 + np.exp(-4 / np.sqrt(2 * np.pi) * buff)) + return p if value else 1 - p + + +class ContinuousBayesNode: + """ A Bayesian network node with continuous distribution or with continuous distributed parents """ + + def __init__(self, name, d_parents, c_parents, parameters, type): + """ + A continuous Bayesian node has two types of parents: discrete and continuous. + :param d_parents: str, name of discrete parents, value of which determines distribution parameters + :param c_parents: str, name of continuous parents, value of which is used to calculate distribution + :param parameters: a dict, parameters for distribution of current node, keys corresponds to discrete parents + :param type: str, type of current node's value, either 'd' (discrete) or 'c'(continuous) + """ + + self.parameters = parameters + self.type = type + self.d_parents = d_parents.split() + self.c_parents = c_parents.split() + self.parents = self.d_parents + self.c_parents + self.variable = name + self.children = [] + + def continuous_p(self, value, c_event, d_event): + """ + Probability given the value of current node and its parents + :param c_event: event of continuous nodes + :param d_event: event of discrete nodes + """ + assert isinstance(c_event, dict) + assert isinstance(d_event, dict) + + d_event_vals = event_values(d_event, self.d_parents) + if len(d_event_vals) == 1: + d_event_vals = d_event_vals[0] + param = self.parameters[d_event_vals] + if self.type == "c": + p = gaussian_probability(param, c_event, value) + if self.type == "d": + p = logistic_probability(param, c_event, value) + return p + + +# harvest-buy example. Figure 13.5 + + +harvest_buy = BayesNet([ + ('Subsidy', '', 0.001), + ('Harvest', '', 0.002), + ('Cost', 'Subsidy', 'Harvest', + {True: {'sigma': 0.5, 'b': 1, 'a': {'Harvest': 0.5}}, + False: {'sigma': 0.6, 'b': 1, 'a': {'Harvest': 0.5}}}, 'c'), + ('Buys', '', 'Cost', {T: {'mu': 0.5, 'sigma': 0.5}, F: {'mu': 0.6, 'sigma': 0.6}}, 'd')]) + + +# ______________________________________________________________________________ +# 13.3 Exact Inference in Bayesian Networks +# 13.3.1 Inference by enumeration + + +def enumeration_ask(X, e, bn): + """ + Return the conditional probability distribution of variable X + given evidence e, from BayesNet bn. [Figure 13.10] + >>> enumeration_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary + ... ).show_approx() + 'False: 0.716, True: 0.284' + """ + + assert X not in e, "Query variable must be distinct from evidence" + Q = ProbDist(X) + for xi in bn.variable_values(X): + Q[xi] = enumerate_all(bn.variables, extend(e, X, xi), bn) + return Q.normalize() + + +def enumerate_all(variables, e, bn): + """ + Return the sum of those entries in P(variables | e{others}) + consistent with e, where P is the joint distribution represented + by bn, and e{others} means e restricted to bn's other variables + (the ones other than variables). Parents must precede children in variables. + """ + + if not variables: + return 1.0 + Y, rest = variables[0], variables[1:] + Ynode = bn.variable_node(Y) + if Y in e: + return Ynode.p(e[Y], e) * enumerate_all(rest, e, bn) + else: + return sum(Ynode.p(y, e) * enumerate_all(rest, extend(e, Y, y), bn) + for y in bn.variable_values(Y)) + + +# ______________________________________________________________________________ +# 13.3.2 The variable elimination algorithm + + +def elimination_ask(X, e, bn): + """ + Compute bn's P(X|e) by variable elimination. [Figure 13.12] + >>> elimination_ask('Burglary', dict(JohnCalls=T, MaryCalls=T), burglary + ... ).show_approx() + 'False: 0.716, True: 0.284' + """ + assert X not in e, "Query variable must be distinct from evidence" + factors = [] + for var in reversed(bn.variables): + factors.append(make_factor(var, e, bn)) + if is_hidden(var, X, e): + factors = sum_out(var, factors, bn) + return pointwise_product(factors, bn).normalize() + + +def is_hidden(var, X, e): + """Is var a hidden variable when querying P(X|e)?""" + return var != X and var not in e + + +def make_factor(var, e, bn): + """ + Return the factor for var in bn's joint distribution given e. + That is, bn's full joint distribution, projected to accord with e, + is the pointwise product of these factors for bn's variables. + """ + node = bn.variable_node(var) + variables = [X for X in [var] + node.parents if X not in e] + cpt = {event_values(e1, variables): node.p(e1[var], e1) + for e1 in all_events(variables, bn, e)} + return Factor(variables, cpt) + + +def pointwise_product(factors, bn): + return reduce(lambda f, g: f.pointwise_product(g, bn), factors) + + +def sum_out(var, factors, bn): + """Eliminate var from all factors by summing over its values.""" + result, var_factors = [], [] + for f in factors: + (var_factors if var in f.variables else result).append(f) + result.append(pointwise_product(var_factors, bn).sum_out(var, bn)) + return result + + +class Factor: + """A factor in a joint distribution.""" + + def __init__(self, variables, cpt): + self.variables = variables + self.cpt = cpt + + def pointwise_product(self, other, bn): + """Multiply two factors, combining their variables.""" + variables = list(set(self.variables) | set(other.variables)) + cpt = {event_values(e, variables): self.p(e) * other.p(e) + for e in all_events(variables, bn, {})} + return Factor(variables, cpt) + + def sum_out(self, var, bn): + """Make a factor eliminating var by summing over its values.""" + variables = [X for X in self.variables if X != var] + cpt = {event_values(e, variables): sum(self.p(extend(e, var, val)) + for val in bn.variable_values(var)) + for e in all_events(variables, bn, {})} + return Factor(variables, cpt) + + def normalize(self): + """Return my probabilities; must be down to one variable.""" + assert len(self.variables) == 1 + return ProbDist(self.variables[0], + {k: v for ((k,), v) in self.cpt.items()}) + + def p(self, e): + """Look up my value tabulated for e.""" + return self.cpt[event_values(e, self.variables)] + + +def all_events(variables, bn, e): + """Yield every way of extending e with values for all variables.""" + if not variables: + yield e + else: + X, rest = variables[0], variables[1:] + for e1 in all_events(rest, bn, e): + for x in bn.variable_values(X): + yield extend(e1, X, x) + + +# ______________________________________________________________________________ +# 13.3.4 Clustering algorithms +# [Figure 13.14a]: sprinkler network + + +sprinkler = BayesNet([ + ('Cloudy', '', 0.5), + ('Sprinkler', 'Cloudy', {T: 0.10, F: 0.50}), + ('Rain', 'Cloudy', {T: 0.80, F: 0.20}), + ('WetGrass', 'Sprinkler Rain', + {(T, T): 0.99, (T, F): 0.90, (F, T): 0.90, (F, F): 0.00})]) + + +# ______________________________________________________________________________ +# 13.4 Approximate Inference for Bayesian Networks +# 13.4.1 Direct sampling methods + + +def prior_sample(bn): + """ + Randomly sample from bn's full joint distribution. The result + is a {variable: value} dict. [Figure 13.15] + """ + event = {} + for node in bn.nodes: + event[node.variable] = node.sample(event) + return event + + +# _________________________________________________________________________ + + +def rejection_sampling(X, e, bn, N=10000): + """ + [Figure 13.16] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn, using N samples. + Raises a ZeroDivisionError if all the N samples are rejected, + i.e., inconsistent with e. + >>> random.seed(47) + >>> rejection_sampling('Burglary', dict(JohnCalls=T, MaryCalls=T), + ... burglary, 10000).show_approx() + 'False: 0.7, True: 0.3' + """ + counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 13.16] + for j in range(N): + sample = prior_sample(bn) # boldface x in [Figure 13.16] + if consistent_with(sample, e): + counts[sample[X]] += 1 + return ProbDist(X, counts) + + +def consistent_with(event, evidence): + """Is event consistent with the given evidence?""" + return all(evidence.get(k, v) == v + for k, v in event.items()) + + +# _________________________________________________________________________ + + +def likelihood_weighting(X, e, bn, N=10000): + """ + [Figure 13.17] + Estimate the probability distribution of variable X given + evidence e in BayesNet bn. + >>> random.seed(1017) + >>> likelihood_weighting('Burglary', dict(JohnCalls=T, MaryCalls=T), + ... burglary, 10000).show_approx() + 'False: 0.702, True: 0.298' + """ + + W = {x: 0 for x in bn.variable_values(X)} + for j in range(N): + sample, weight = weighted_sample(bn, e) # boldface x, w in [Figure 14.15] + W[sample[X]] += weight + return ProbDist(X, W) + + +def weighted_sample(bn, e): + """ + Sample an event from bn that's consistent with the evidence e; + return the event and its weight, the likelihood that the event + accords to the evidence. + """ + + w = 1 + event = dict(e) # boldface x in [Figure 13.17] + for node in bn.nodes: + Xi = node.variable + if Xi in e: + w *= node.p(e[Xi], event) + else: + event[Xi] = node.sample(event) + return event, w + + +# _________________________________________________________________________ +# 13.4.2 Inference by Markov chain simulation + + +def gibbs_ask(X, e, bn, N=1000): + """[Figure 13.19]""" + assert X not in e, "Query variable must be distinct from evidence" + counts = {x: 0 for x in bn.variable_values(X)} # bold N in [Figure 14.16] + Z = [var for var in bn.variables if var not in e] + state = dict(e) # boldface x in [Figure 14.16] + for Zi in Z: + state[Zi] = random.choice(bn.variable_values(Zi)) + for j in range(N): + for Zi in Z: + state[Zi] = markov_blanket_sample(Zi, state, bn) + counts[state[X]] += 1 + return ProbDist(X, counts) + + +def markov_blanket_sample(X, e, bn): + """ + Return a sample from P(X | mb) where mb denotes that the + variables in the Markov blanket of X take their values from event + e (which must assign a value to each). The Markov blanket of X is + X's parents, children, and children's parents. + """ + Xnode = bn.variable_node(X) + Q = ProbDist(X) + for xi in bn.variable_values(X): + ei = extend(e, X, xi) + # [Equation 13.12:] + Q[xi] = Xnode.p(xi, e) * product(Yj.p(ei[Yj.variable], ei) + for Yj in Xnode.children) + # (assuming a Boolean variable here) + return probability(Q.normalize()[True]) + + +# _________________________________________________________________________ +# 13.4.3 Compiling approximate inference + + +class complied_burglary: + """compiled version of burglary network""" + + def Burglary(self, sample): + if sample['Alarm']: + if sample['Earthquake']: + return probability(0.00327) + else: + return probability(0.485) + else: + if sample['Earthquake']: + return probability(7.05e-05) + else: + return probability(6.01e-05) + + def Earthquake(self, sample): + if sample['Alarm']: + if sample['Burglary']: + return probability(0.0020212) + else: + return probability(0.36755) + else: + if sample['Burglary']: + return probability(0.0016672) + else: + return probability(0.0014222) + + def MaryCalls(self, sample): + if sample['Alarm']: + return probability(0.7) + else: + return probability(0.01) + + def JongCalls(self, sample): + if sample['Alarm']: + return probability(0.9) + else: + return probability(0.05) + + def Alarm(self, sample): + raise NotImplementedError diff --git a/pytest.ini b/pytest.ini index 7d983c3fc..1561b6fe6 100644 --- a/pytest.ini +++ b/pytest.ini @@ -1,3 +1,5 @@ [pytest] filterwarnings = - ignore::ResourceWarning + ignore::DeprecationWarning + ignore::UserWarning + ignore::RuntimeWarning diff --git a/rl.ipynb b/reinforcement_learning.ipynb similarity index 99% rename from rl.ipynb rename to reinforcement_learning.ipynb index a8f6adc2c..ee3b6a5eb 100644 --- a/rl.ipynb +++ b/reinforcement_learning.ipynb @@ -17,7 +17,7 @@ }, "outputs": [], "source": [ - "from rl import *" + "from reinforcement_learning import *" ] }, { @@ -628,8 +628,17 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "source": [], + "metadata": { + "collapsed": false + } + } } }, "nbformat": 4, "nbformat_minor": 1 -} +} \ No newline at end of file diff --git a/rl.py b/reinforcement_learning.py similarity index 82% rename from rl.py rename to reinforcement_learning.py index 4fc52abef..4cb91af0f 100644 --- a/rl.py +++ b/reinforcement_learning.py @@ -1,15 +1,14 @@ """Reinforcement Learning (Chapter 21)""" +import random from collections import defaultdict -from utils import argmax -from mdp import MDP, policy_evaluation -import random +from mdp import MDP, policy_evaluation class PassiveDUEAgent: - - """Passive (non-learning) agent that uses direct utility estimation + """ + Passive (non-learning) agent that uses direct utility estimation on a given MDP and policy. import sys @@ -18,15 +17,16 @@ class PassiveDUEAgent: south = (0,-1) west = (-1, 0) east = (1, 0) - policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} agent = PassiveDUEAgent(policy, sequential_decision_environment) for i in range(200): run_single_trial(agent,sequential_decision_environment) agent.estimate_U() agent.U[(0, 0)] > 0.2 True - """ + def __init__(self, pi, mdp): self.pi = pi self.mdp = mdp @@ -36,7 +36,7 @@ def __init__(self, pi, mdp): self.s_history = [] self.r_history = [] self.init = mdp.init - + def __call__(self, percept): s1, r1 = percept self.s_history.append(s1) @@ -48,40 +48,40 @@ def __call__(self, percept): else: self.s, self.a = s1, self.pi[s1] return self.a - + def estimate_U(self): # this function can be called only if the MDP has reached a terminal state # it will also reset the mdp history assert self.a is None, 'MDP is not in terminal state' assert len(self.s_history) == len(self.r_history) # calculating the utilities based on the current iteration - U2 = {s : [] for s in set(self.s_history)} + U2 = {s: [] for s in set(self.s_history)} for i in range(len(self.s_history)): s = self.s_history[i] U2[s] += [sum(self.r_history[i:])] - U2 = {k : sum(v)/max(len(v), 1) for k, v in U2.items()} + U2 = {k: sum(v) / max(len(v), 1) for k, v in U2.items()} # resetting history self.s_history, self.r_history = [], [] # setting the new utilities to the average of the previous # iteration and this one for k in U2.keys(): if k in self.U.keys(): - self.U[k] = (self.U[k] + U2[k]) /2 + self.U[k] = (self.U[k] + U2[k]) / 2 else: self.U[k] = U2[k] return self.U def update_state(self, percept): - '''To be overridden in most cases. The default case - assumes the percept to be of type (state, reward)''' + """To be overridden in most cases. The default case + assumes the percept to be of type (state, reward)""" return percept - class PassiveADPAgent: - - """Passive (non-learning) agent that uses adaptive dynamic programming - on a given MDP and policy. [Figure 21.2] + """ + [Figure 21.2] + Passive (non-learning) agent that uses adaptive dynamic programming + on a given MDP and policy. import sys from mdp import sequential_decision_environment @@ -89,7 +89,8 @@ class PassiveADPAgent: south = (0,-1) west = (-1, 0) east = (1, 0) - policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} agent = PassiveADPAgent(policy, sequential_decision_environment) for i in range(100): run_single_trial(agent,sequential_decision_environment) @@ -101,8 +102,9 @@ class PassiveADPAgent: """ class ModelMDP(MDP): - """ Class for implementing modified Version of input MDP with - an editable transition model P and a custom function T. """ + """Class for implementing modified Version of input MDP with + an editable transition model P and a custom function T.""" + def __init__(self, init, actlist, terminals, gamma, states): super().__init__(init, actlist, terminals, states=states, gamma=gamma) nested_dict = lambda: defaultdict(nested_dict) @@ -123,7 +125,7 @@ def __init__(self, pi, mdp): self.Ns1_sa = defaultdict(int) self.s = None self.a = None - self.visited = set() # keeping track of visited states + self.visited = set() # keeping track of visited states def __call__(self, percept): s1, r1 = percept @@ -159,10 +161,12 @@ def update_state(self, percept): class PassiveTDAgent: - """The abstract class for a Passive (non-learning) agent that uses + """ + [Figure 21.4] + The abstract class for a Passive (non-learning) agent that uses temporal differences to learn utility estimates. Override update_state method to convert percept to state and reward. The mdp being provided - should be an instance of a subclass of the MDP Class. [Figure 21.4] + should be an instance of a subclass of the MDP Class. import sys from mdp import sequential_decision_environment @@ -170,7 +174,8 @@ class PassiveTDAgent: south = (0,-1) west = (-1, 0) east = (1, 0) - policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n)) for i in range(200): run_single_trial(agent,sequential_decision_environment) @@ -195,7 +200,7 @@ def __init__(self, pi, mdp, alpha=None): if alpha: self.alpha = alpha else: - self.alpha = lambda n: 1/(1+n) # udacity video + self.alpha = lambda n: 1 / (1 + n) # udacity video def __call__(self, percept): s1, r1 = self.update_state(percept) @@ -219,9 +224,11 @@ def update_state(self, percept): class QLearningAgent: - """ An exploratory Q-learning agent. It avoids having to learn the transition - model because the Q-value of a state can be related directly to those of - its neighbors. [Figure 21.8] + """ + [Figure 21.8] + An exploratory Q-learning agent. It avoids having to learn the transition + model because the Q-value of a state can be related directly to those of + its neighbors. import sys from mdp import sequential_decision_environment @@ -229,7 +236,8 @@ class QLearningAgent: south = (0,-1) west = (-1, 0) east = (1, 0) - policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60./(59+n)) for i in range(200): run_single_trial(q_agent,sequential_decision_environment) @@ -239,6 +247,7 @@ class QLearningAgent: q_agent.Q[((1, 0), (0, -1))] <= 0.5 True """ + def __init__(self, mdp, Ne, Rplus, alpha=None): self.gamma = mdp.gamma @@ -255,10 +264,10 @@ def __init__(self, mdp, Ne, Rplus, alpha=None): if alpha: self.alpha = alpha else: - self.alpha = lambda n: 1./(1+n) # udacity video + self.alpha = lambda n: 1. / (1 + n) # udacity video def f(self, u, n): - """ Exploration function. Returns fixed Rplus until + """Exploration function. Returns fixed Rplus until agent has visited state, action a Ne number of times. Same as ADP agent in book.""" if n < self.Ne: @@ -267,8 +276,8 @@ def f(self, u, n): return u def actions_in_state(self, state): - """ Return actions possible in given state. - Useful for max and argmax. """ + """Return actions possible in given state. + Useful for max and argmax.""" if state in self.terminals: return [None] else: @@ -285,12 +294,12 @@ def __call__(self, percept): if s is not None: Nsa[s, a] += 1 Q[s, a] += alpha(Nsa[s, a]) * (r + gamma * max(Q[s1, a1] - for a1 in actions_in_state(s1)) - Q[s, a]) + for a1 in actions_in_state(s1)) - Q[s, a]) if s in terminals: self.s = self.a = self.r = None else: self.s, self.r = s1, r1 - self.a = argmax(actions_in_state(s1), key=lambda a1: self.f(Q[s1, a1], Nsa[s1, a1])) + self.a = max(actions_in_state(s1), key=lambda a1: self.f(Q[s1, a1], Nsa[s1, a1])) return self.a def update_state(self, percept): diff --git a/reinforcement_learning4e.py b/reinforcement_learning4e.py new file mode 100644 index 000000000..eaaba3e5a --- /dev/null +++ b/reinforcement_learning4e.py @@ -0,0 +1,353 @@ +"""Reinforcement Learning (Chapter 21)""" + +import random +from collections import defaultdict + +from mdp4e import MDP, policy_evaluation + + +# _________________________________________ +# 21.2 Passive Reinforcement Learning +# 21.2.1 Direct utility estimation + + +class PassiveDUEAgent: + """ + Passive (non-learning) agent that uses direct utility estimation + on a given MDP and policy. + + import sys + from mdp import sequential_decision_environment + north = (0, 1) + south = (0,-1) + west = (-1, 0) + east = (1, 0) + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + agent = PassiveDUEAgent(policy, sequential_decision_environment) + for i in range(200): + run_single_trial(agent,sequential_decision_environment) + agent.estimate_U() + agent.U[(0, 0)] > 0.2 + True + """ + + def __init__(self, pi, mdp): + self.pi = pi + self.mdp = mdp + self.U = {} + self.s = None + self.a = None + self.s_history = [] + self.r_history = [] + self.init = mdp.init + + def __call__(self, percept): + s1, r1 = percept + self.s_history.append(s1) + self.r_history.append(r1) + ## + ## + if s1 in self.mdp.terminals: + self.s = self.a = None + else: + self.s, self.a = s1, self.pi[s1] + return self.a + + def estimate_U(self): + # this function can be called only if the MDP has reached a terminal state + # it will also reset the mdp history + assert self.a is None, 'MDP is not in terminal state' + assert len(self.s_history) == len(self.r_history) + # calculating the utilities based on the current iteration + U2 = {s: [] for s in set(self.s_history)} + for i in range(len(self.s_history)): + s = self.s_history[i] + U2[s] += [sum(self.r_history[i:])] + U2 = {k: sum(v) / max(len(v), 1) for k, v in U2.items()} + # resetting history + self.s_history, self.r_history = [], [] + # setting the new utilities to the average of the previous + # iteration and this one + for k in U2.keys(): + if k in self.U.keys(): + self.U[k] = (self.U[k] + U2[k]) / 2 + else: + self.U[k] = U2[k] + return self.U + + def update_state(self, percept): + """To be overridden in most cases. The default case + assumes the percept to be of type (state, reward)""" + return percept + + +# 21.2.2 Adaptive dynamic programming + + +class PassiveADPAgent: + """ + [Figure 21.2] + Passive (non-learning) agent that uses adaptive dynamic programming + on a given MDP and policy. + + import sys + from mdp import sequential_decision_environment + north = (0, 1) + south = (0,-1) + west = (-1, 0) + east = (1, 0) + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + agent = PassiveADPAgent(policy, sequential_decision_environment) + for i in range(100): + run_single_trial(agent,sequential_decision_environment) + + agent.U[(0, 0)] > 0.2 + True + agent.U[(0, 1)] > 0.2 + True + """ + + class ModelMDP(MDP): + """Class for implementing modified Version of input MDP with + an editable transition model P and a custom function T.""" + + def __init__(self, init, actlist, terminals, gamma, states): + super().__init__(init, actlist, terminals, states=states, gamma=gamma) + nested_dict = lambda: defaultdict(nested_dict) + # StackOverflow:whats-the-best-way-to-initialize-a-dict-of-dicts-in-python + self.P = nested_dict() + + def T(self, s, a): + """Return a list of tuples with probabilities for states + based on the learnt model P.""" + return [(prob, res) for (res, prob) in self.P[(s, a)].items()] + + def __init__(self, pi, mdp): + self.pi = pi + self.mdp = PassiveADPAgent.ModelMDP(mdp.init, mdp.actlist, + mdp.terminals, mdp.gamma, mdp.states) + self.U = {} + self.Nsa = defaultdict(int) + self.Ns1_sa = defaultdict(int) + self.s = None + self.a = None + self.visited = set() # keeping track of visited states + + def __call__(self, percept): + s1, r1 = percept + mdp = self.mdp + R, P, terminals, pi = mdp.reward, mdp.P, mdp.terminals, self.pi + s, a, Nsa, Ns1_sa, U = self.s, self.a, self.Nsa, self.Ns1_sa, self.U + + if s1 not in self.visited: # Reward is only known for visited state. + U[s1] = R[s1] = r1 + self.visited.add(s1) + if s is not None: + Nsa[(s, a)] += 1 + Ns1_sa[(s1, s, a)] += 1 + # for each t such that Ns′|sa [t, s, a] is nonzero + for t in [res for (res, state, act), freq in Ns1_sa.items() + if (state, act) == (s, a) and freq != 0]: + P[(s, a)][t] = Ns1_sa[(t, s, a)] / Nsa[(s, a)] + + self.U = policy_evaluation(pi, U, mdp) + ## + ## + self.Nsa, self.Ns1_sa = Nsa, Ns1_sa + if s1 in terminals: + self.s = self.a = None + else: + self.s, self.a = s1, self.pi[s1] + return self.a + + def update_state(self, percept): + """To be overridden in most cases. The default case + assumes the percept to be of type (state, reward).""" + return percept + + +# 21.2.3 Temporal-difference learning + + +class PassiveTDAgent: + """ + [Figure 21.4] + The abstract class for a Passive (non-learning) agent that uses + temporal differences to learn utility estimates. Override update_state + method to convert percept to state and reward. The mdp being provided + should be an instance of a subclass of the MDP Class. + + import sys + from mdp import sequential_decision_environment + north = (0, 1) + south = (0,-1) + west = (-1, 0) + east = (1, 0) + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n)) + for i in range(200): + run_single_trial(agent,sequential_decision_environment) + + agent.U[(0, 0)] > 0.2 + True + agent.U[(0, 1)] > 0.2 + True + """ + + def __init__(self, pi, mdp, alpha=None): + + self.pi = pi + self.U = {s: 0. for s in mdp.states} + self.Ns = {s: 0 for s in mdp.states} + self.s = None + self.a = None + self.r = None + self.gamma = mdp.gamma + self.terminals = mdp.terminals + + if alpha: + self.alpha = alpha + else: + self.alpha = lambda n: 1 / (1 + n) # udacity video + + def __call__(self, percept): + s1, r1 = self.update_state(percept) + pi, U, Ns, s, r = self.pi, self.U, self.Ns, self.s, self.r + alpha, gamma, terminals = self.alpha, self.gamma, self.terminals + if not Ns[s1]: + U[s1] = r1 + if s is not None: + Ns[s] += 1 + U[s] += alpha(Ns[s]) * (r + gamma * U[s1] - U[s]) + if s1 in terminals: + self.s = self.a = self.r = None + else: + self.s, self.a, self.r = s1, pi[s1], r1 + return self.a + + def update_state(self, percept): + """To be overridden in most cases. The default case + assumes the percept to be of type (state, reward).""" + return percept + + +# __________________________________________ +# 21.3. Active Reinforcement Learning +# 21.3.2 Learning an action-utility function + + +class QLearningAgent: + """ + [Figure 21.8] + An exploratory Q-learning agent. It avoids having to learn the transition + model because the Q-value of a state can be related directly to those of + its neighbors. + + import sys + from mdp import sequential_decision_environment + north = (0, 1) + south = (0,-1) + west = (-1, 0) + east = (1, 0) + policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north, + (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,} + q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60./(59+n)) + for i in range(200): + run_single_trial(q_agent,sequential_decision_environment) + + q_agent.Q[((0, 1), (0, 1))] >= -0.5 + True + q_agent.Q[((1, 0), (0, -1))] <= 0.5 + True + """ + + def __init__(self, mdp, Ne, Rplus, alpha=None): + + self.gamma = mdp.gamma + self.terminals = mdp.terminals + self.all_act = mdp.actlist + self.Ne = Ne # iteration limit in exploration function + self.Rplus = Rplus # large value to assign before iteration limit + self.Q = defaultdict(float) + self.Nsa = defaultdict(float) + self.s = None + self.a = None + self.r = None + + if alpha: + self.alpha = alpha + else: + self.alpha = lambda n: 1. / (1 + n) # udacity video + + def f(self, u, n): + """Exploration function. Returns fixed Rplus until + agent has visited state, action a Ne number of times. + Same as ADP agent in book.""" + if n < self.Ne: + return self.Rplus + else: + return u + + def actions_in_state(self, state): + """Return actions possible in given state. + Useful for max and argmax.""" + if state in self.terminals: + return [None] + else: + return self.all_act + + def __call__(self, percept): + s1, r1 = self.update_state(percept) + Q, Nsa, s, a, r = self.Q, self.Nsa, self.s, self.a, self.r + alpha, gamma, terminals = self.alpha, self.gamma, self.terminals, + actions_in_state = self.actions_in_state + + if s in terminals: + Q[s, None] = r1 + if s is not None: + Nsa[s, a] += 1 + Q[s, a] += alpha(Nsa[s, a]) * (r + gamma * max(Q[s1, a1] + for a1 in actions_in_state(s1)) - Q[s, a]) + if s in terminals: + self.s = self.a = self.r = None + else: + self.s, self.r = s1, r1 + self.a = max(actions_in_state(s1), key=lambda a1: self.f(Q[s1, a1], Nsa[s1, a1])) + return self.a + + def update_state(self, percept): + """To be overridden in most cases. The default case + assumes the percept to be of type (state, reward).""" + return percept + + +def run_single_trial(agent_program, mdp): + """Execute trial for given agent_program + and mdp. mdp should be an instance of subclass + of mdp.MDP """ + + def take_single_action(mdp, s, a): + """ + Select outcome of taking action a + in state s. Weighted Sampling. + """ + x = random.uniform(0, 1) + cumulative_probability = 0.0 + for probability_state in mdp.T(s, a): + probability, state = probability_state + cumulative_probability += probability + if x < cumulative_probability: + break + return state + + current_state = mdp.init + while True: + current_reward = mdp.R(current_state) + percept = (current_state, current_reward) + next_action = agent_program(percept) + if next_action is None: + break + current_state = take_single_action(mdp, current_state, next_action) diff --git a/requirements.txt b/requirements.txt index 505ba03b5..dd6b1be8a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,18 @@ -networkx==1.11 +cvxopt +image +ipython +ipythonblocks +ipywidgets jupyter -pandas +keras matplotlib +networkx +numpy +opencv-python +pandas pillow -Image +pytest-cov +qpsolvers +scipy +sortedcontainers +tensorflow \ No newline at end of file diff --git a/search.ipynb b/search.ipynb index aeb035902..caf231dcc 100644 --- a/search.ipynb +++ b/search.ipynb @@ -808,7 +808,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAABTsAAAPKCAYAAABbVI7QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xdc1eX///HnQYaCg8SFmCPcouLG1BQX5Uj9OHKVfBLtY0qOzJELREXNcFbmKC0zS1Nz5RZHoqklOTBH7r1yJvP8/uALv06gggJvODzut9u5+Tnv93Vd7+f7KPThxXVdb5PZbDYLAAAAAAAAALI4G6MDAAAAAAAAAEBaoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCrYGh0AyGiRkZHavn27Hj16lHisRo0acnNzMzAVAAAAAAAAnpfJbDabjQ4BZISzZ89q3759cnBwUOPGjeXk5CRJMpvN2rNnjy5duqTChQurXr16MplMBqcFAAAAAABAalHsRLawefNm5cqVSy+//PITC5lXrlzR+vXr1aVLFzk4OGRgQgAAAAAAADwvip2wehs3btRLL72k0qVLp6h9dHS0vv76a7311luytWWnBwAAAAAAgKyCYiesWnh4uMxmszw9PVPV7++//9aaNWvUsWPHdEoGAAAAAACAtMbT2GHVTpw4kepCpyTlypVLefPm1b1799IhFQAAAAAAANIDxU5YrevXr6tgwYLP3L9x48baunVrGiYCAAAAAABAeqLYCav1888/q0GDBs/c387OTrGxsWmYCAAAAAAAAOmJYiesVo4cOWRj83z/xO3s7NIoDQAAAAAAANIbxU5YrbR49hbP7wIAAAAAAMg6KHbCaplMpkwxBgAAAAAAADIGxU5YLVtbWz18+PC5xoiKikqjNAAAAAAAAEhvFDthtRo3bqwtW7Y8c//bt2/L2dk5DRMBAAAAAAAgPVHshNVycHBQZGTkM++7uX37djVq1ChtQwEAAAAAACDdUOyEVatXr55++umnVPc7e/ascufOrRw5cqRDKgAAAAAAAKQHip2waq6uripevLi2bt2a4j4XLlzQgQMH1LRp03RMBgAAAAAAgLRmMj/rGl8gCzl+/Lj27NmjJk2ayM3NLdk20dHRWrhwoV544QW1b98+gxMCAAAAAADgedkaHQDICGXLltWCBQu0fv16tW/fXs7OzipSpIjs7e1169YtXbhwQba2ttqxY4dcXFwodgIAAAAAAGRBzOxEtnDjxg2VK1dOv/76q0qUKKG7d+/q+vXrioqK0gsvvKDChQvLZDLp2rVrqlSpkkJDQ1WpUiWjYwMAAAAAACAVKHYiWxg6dKju3bunTz/99KltZ8yYoR9//FGbN2+WyWTKgHQAAAAAAABICxQ7YfWuXr2qihUrKjw8XMWKFXtq+5iYGFWrVk1jxoxRhw4dMiAhAAAAAAAA0gLFTli9QYMGKTY2VtOnT09xn9DQUPXo0UMRERFydHRMx3QAAAAAAABIKxQ7YdUuXbokDw8PHTlyRK6urqnq+8Ybb6hcuXIaO3ZsOqUDAAAAAABAWqLYCavm7+8ve3t7ffzxx6nue/78eXl6emrfvn166aWX0iEdAAAAAAAA0hLFTlit8+fPq2rVqjp27JgKFSr0TGOMHz9e+/bt08qVK9M4HQAAAAAAANIaxU5YrT59+ihfvnyaOHHiM4/x6NEjeXh46JNPPpGPj08apgMAAAAAAEBao9gJq3TmzBnVqFFDf/zxhwoUKPBcY61evVqDBw/WoUOHZG9vn0YJAQAAAAAAkNYodgIAAAAAAACwCjZGBwAAAAAAAACAtECxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2Av+ybNkymUwmo2MAAAAAAAAglSh2ItO7ePGievfurWLFisne3l5ubm7q1auXLly4YHQ0AAAAAAAAZCIUO5GpnT59WjVr1tThw4e1cOFCnTx5UosWLdKRI0dUq1YtnTlzJtl+UVFRGRsUAAAAAAAAhqPYiUytb9++srGx0ebNm9WkSRMVL15c3t7e2rx5s2xsbNS3b19JUqNGjdSnTx8NHjxYBQsWVL169SRJISEhqlKlipycnOTm5iY/Pz/99ddfFtf46quvVKJECTk6OqpVq1a6evVqkhyrV69WjRo1lDNnTpUqVUojRoywKKguWrRItWrVUp48eVSoUCF17NhRFy9eTMdPBgAAAAAAAP9GsROZ1q1bt7R+/Xr17dtXjo6OFuccHR317rvv6qefftLt27clxRcczWazdu7cqa+++kqSZGNjo2nTpunIkSNavHixfvnlF/n7+yeOs3fvXvn6+qp37946ePCgWrdurdGjR1tca8OGDerWrZv69eunI0eO6IsvvtCyZcv04YcfJraJiopSYGCgwsPDtWbNGt24cUNdunRJr48GAAAAAAAAyTCZzWaz0SGA5Ozdu1deXl5avny52rVrl+T8ihUr9J///Ed79+7VkCFDdOvWLf3+++9PHHP9+vVq06aN/v77b9nY2Khr1666fv26Nm3alNjGz89P8+fPV8KXxiuvvKJmzZpp1KhRiW1Wrlyp7t276969e8k+zOjYsWOqUKGCzp8/r2LFij3rRwAAAAAAAIBUYGYnMr3HPRk9oRiZcL5GjRpJ2mzdulXNmjVTsWLFlCdPHv3nP/9RVFSUrly5IkmKiIhQ3bp1Lfr8+/2BAwc0fvx45c6dO/HVtWtXPXjwIHGcX3/9VW3atFGJEiWUJ08e1axZU5J07ty557hzAAAAAAAApAbFTmRaZcqUkclk0pEjR5I9HxERIZPJJHd3d0mSk5OTxfmzZ8+qZcuWqlChgpYuXaoDBw7oiy++kPT/H2CUkonNcXFxGjNmjA4ePJj4+v3333XixAkVLFhQDx48kI+PjxwdHfX1119r3759Wr9+vcV1AAAAAAAAkP5sjQ4APE7+/Pnl4+OjTz/9VAMHDrTYt/Phw4f65JNP9Nprryl//vzJ9t+/f7+ioqI0depU5ciRQ5K0Zs0aizYVK1bUnj17LI79+3316tV17NgxlS5dOtnrhIeH68aNG5owYYJKlSolSVq+fHnqbhYAAAAAAADPjZmdyNRmzZqlmJgYNW3aVFu3btX58+cVGhqqZs2ayWw2a9asWY/tW6ZMGcXFxWnatGk6ffq0vv32W02bNs2izXvvvafNmzcrODhYJ06c0Ny5c7VixQqLNqNHj9bixYs1evRoHT58WMeOHdOyZcs0ZMgQSVLx4sXl4OCgWbNm6c8//9TatWst9vcEAAAAAABAxqDYiUzN3d1d+/fvV6VKlfTmm2/qpZdeUteuXVWhQgXt27cvcSZlcqpUqaLp06crJCREFStW1Lx58zRlyhSLNl5eXpo/f74+++wzValSRcuXL1dAQIBFGx8fH61du1bbtm1T7dq1Vbt2bU2cOFHFixeXJBUsWFALFy7UypUrVbFiRQUGBiokJCTNPwsAAAAAAAA8GU9jBwAAAAAAAGAVmNkJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOZAlms1k1atTQ8uXLjY6SImazWc2aNdO0adOMjgIAAAAAAJBtUOxElrBq1SrFxcWpbdu2RkdJEZPJpBkzZmjcuHG6evWq0XEAAAAAAACyBZPZbDYbHQJ4kri4OFWrVk1BQUF6/fXXjY6TKu+//75u376tL774wugoAAAAAAAAVo+Zncj0li9fLnt7e7Vu3droKKk2ZswYrV+/Xnv37jU6CgAAAAAAgNWj2IlMzWw26/r16xo7dqxMJpPRcVItb968Cg4Olr+/v+Li4oyOAwAAAAAAYNVYxo5ML+GfaFYsdkrxy/Dr1asnPz8/9ezZ0+g4AAAAAAAAVotiJ5ABDhw4oJYtW+rYsWNydnY2Og4AAAAAAIBVotgJZJDevXsrV65cmj59utFRAAAAAAAArBLFTiCDXL9+XRUrVtS2bdvk4eFhdBwAAAAAAACrwwOKgAxSsGBBjRkzRv7+/uJ3DAAAAAAAAGmPYieQgf73v//p5s2bWrp0qdFRAAAAAAAArA7L2IEMtn37dr355puKiIiQk5OT0XEAAAAAAACsBjM7Yahbt24ZHSHDNWzYUPXq1VNwcLDRUQAAAAAAAKwKMzthmHnz5mnXrl3y9fWVp6ennJ2dE8+ZzWaZTKbHvs/qLly4oKpVq+qXX36Ru7u70XEAAAAAAACsAsVOGCI2Nlb58+dXVFSUnJ2d1a5dO3Xu3FlVq1ZVvnz5Ets9ePBAdnZ2sre3NzBt+ggODlZYWJhWrVpldBQAAAAAAACrwDJ2GGLZsmWqVKmSfvvtNwUGBmrdunXq2LGjRo0apZ07d+revXuSpGnTplntcu9BgwYpIiJCP/30k9FRAAAAAAAArAIzO2GItWvXasuWLRoyZIiKFCkiSZo1a5YmTZqkmJgYdenSRbVr11bXrl21adMmNWnSxODE6WPt2rUaOHCgDh06JAcHB6PjAAAAAAAAZGkUO5Hh7t+/r9y5c+vPP//USy+9pJiYGNna2iaenz59uqZOnapz586pQYMG2r59u4Fp01+rVq3UoEEDDR061OgoAAAAAAAAWRrFTmSoR48eqVWrVpo4caJq1qxp8eChfxY9jx07pooVK2rPnj2qXbu2kZHT3cmTJ+Xl5aXw8HC5ubkZHQcAAAAAACDLYs9OZKiRI0dq69atGj58uO7evWvxhPWEQmdsbKwmTJigMmXKWH2hU5JKly6t3r17a8iQIUZHAQAAAAAAyNIodiLD3LlzR9OnT9e8efN0+fJlde3aVZcvX5YUX+BMYDab1aBBAy1dutSoqBnuww8/1I4dO7Rz506jowAAAAAAAGRZLGNHhvHz89Off/6prVu3atGiRRowYIC6dOmimTNnJmkbGxurHDlyGJDSOEuWLNHEiRN14MCBbHfvAAAAAAAAaYFiJzLEzZs3VaRIEe3evVu1atWSFF/c8/f315tvvqnx48crV65ciouLk41N9pxwbDab5e3trU6dOundd981Og4AAAAAAECWQ7ETGaJPnz76448/tHXrVsXGxsrGxkYxMTGaMGGCpk2bpo8++kh+fn5GxzTc77//rqZNm+ro0aMqUKCA0XEAAAAAAACyFIqdyBBRUVG6d++eXFxckpwbMWKEZs6cqSlTpqh3794GpMtc/P39FR0drdmzZxsdBQAAAAAAIEuh2AnDJCxZv3nzpvz9/bVhwwZt2bJFnp6eRkcz1O3bt1WhQgWtW7dO1atXNzoOAAAAAABAlpE9N0dEppCwN6eLi4vmz58vT09POTo6GpzKeC+88IKCgoLk7+8vfhcBAAAAAACQcszshOESZnjevXtXefPmNTpOphAbGysvLy+99957evPNN42OAwAAAAAAkCVQ7ESGSng4kSSZTCaD02Rue/fu1X/+8x9FRERQBAYAAAAAAEgBlrEjQw0ePFiLFi2i0JkCderUUfPmzRUUFGR0FAAAAAAAgCyBmZ3IMJcuXZKHh4eOHj2qIkWKGB0nS7h69ao8PDy0c+dOlS9f3ug4AAAAAAAAmRrFTmQYf39/OTg4aMqUKUZHyVKmTp2q9evXa/369cyIBQAAAAAAeAKKncgQ58+fl6enpyIiIlSoUCGj42Qp0dHR8vT01Pjx49W2bVuj4wAAAABAhrt7966uXbum6Ohoo6MAWZqdnZ0KFSpk1c8GodiJDPG///1Pzs7OmjhxotFRsqQtW7aoV69eOnLkiHLlymV0HAAAAADIMHfv3tXVq1fl5uamXLlyseINeEZms1l///23Ll68qMKFC1ttwZNiJ9LdmTNnVKNGDR0/flwuLi5Gx8myOnTooCpVqmj06NFGRwEAAACADHPy5EkVLVpUjo6ORkcBrMLDhw916dIllS5d2ugo6YKnsSPdjRs3Tu+++y6Fzuf08ccfa8aMGTp79qzRUQAAAAAgw0RHR7PCDUhDuXLlsuotISh2Il2dOnVKK1eu1KBBg4yOkuWVKFFC7733nt5//32jowAAAABAhmLpOpB2rP3riWIn0tXYsWPl7++vF154wegoVuGDDz7Qr7/+qi1bthgdBQAAAAAAINOxNToArNcff/yhdevW6eTJk0ZHsRq5cuVSSEiI/P39FR4eLjs7O6MjAQAAAAAAZBrM7ES6GTt2rAYOHKh8+fIZHcWqtGnTRi+++KJmzZpldBQAAAAAwDPw9fVVsWLFkj0XGhoqk8mkzZs3Z3CqtJNwD6GhoUZHSeTr66uSJUsaHQMZgGIn0sXRo0e1efNm+fv7Gx3F6phMJk2fPl0TJkzQ1atXjY4DAAAAAACQaVDsRLoICAjQ+++/rzx58hgdxSqVL19evr6+GjZsmNFRAAAAAABIN7GxsYqJiTE6BrIQip1Ic7///rt27typvn37Gh3Fqo0aNUobN27Unj17jI4CAAAAAEgnJUuWVPfu3bVkyRJVqFBBTk5Oqlmzpnbt2pXiMebOnauqVasqZ86cKlCggHr27Klbt24lnp83b55MJpNWrlyZeCw2NlavvPKK3N3dde/ePUnxE5tMJpMOHTokb29vOTo6ytXVVaNHj1ZcXNwTM5jNZk2dOlXlypWTvb29XF1d1a9fP929e9einclk0ogRIzRx4kSVKlVK9vb2OnTokCTpxo0b6tOnj9zc3OTg4KDy5ctrzpw5Sa61ZcsWVa9eXTlz5pS7u7s+//zzFH9WyPp4QBHSXEBAgIYMGSInJyejo1i1vHnzauLEifL399fevXtlY8PvLgAAAADAGu3cuVN//PGHgoKClDNnTo0aNUqtWrXSmTNn5Ozs/MS+w4YN08cff6z33ntPH330kS5evKiRI0fq8OHD2r17t3LkyCE/Pz9t3LhRfn5+qlWrltzc3BQUFKSwsDDt2rUryarNtm3b6u2339bw4cO1YcMGBQUFycbGRgEBAY/NMWLECAUHB6tv375q3bq1jh49qlGjRik8PFzbt2+3+Jl2wYIFeumllzRlyhQ5OTmpaNGiunv3rurVq6e///5bAQEBKlWqlDZs2KA+ffooMjIycRu9iIgItWjRQjVr1tSSJUsUGRmpgIAA3b9/Xzly5Hj2vwRkGRQ7kaZ+/fVX7d27V998843RUbKF7t27a/bs2friiy/k5+dndBwAAAAAQDq4e/euDh48qBdeeEGSVKRIEdWqVUvr1q1T165dH9vvzJkz+uijjzRmzBiNHj068XjZsmVVv359rV69Wm3btpUkzZkzR1WrVlX37t0VEBCgcePGKSgoSHXq1Ekybq9evRK3VWvevLnu3r2rjz/+WAMGDEi2+Hrr1i2FhISoR48eiQ/b9fHxUcGCBfXmm29qzZo1ev311xPbm81mbdy4Ubly5Uo8FhQUpLNnz+rQoUMqU6aMJKlp06b666+/FBgYqD59+sjW1lbjxo1Tnjx5tHHjxsRJWC+//LLc3d1VtGjRlH3gyNKYCoY0NWbMGA0bNsziGxLSj8lk0syZMzVy5Ejdvn3b6DgAAAAAgHRQt27dxEKnJFWuXFmSdO7cOUnxxcGYmJjEV2xsrCRp06ZNiouLU7du3SzO16lTR3nz5tWOHTsSx3R2dtbixYu1c+dO+fj4qEGDBho6dGiyeTp16mTxvnPnzrp//74OHz6cbPs9e/YoMjJS3bt3T9LP1tZW27dvtzj+6quvJqkrrF+/XnXq1FGpUqUs7sXHx0c3b97U0aNHJUlhYWFq0aKFxWrTF198UfXq1Us2G6wPxU6kmV9++UUHDx5Ur169jI6SrVSvXl1t27bVmDFjjI4CAAAAAEgBW1vbxILkvyUct7X9/4tx8+fPb9HGwcFBkvTo0SNJ0sKFC2VnZ5f4cnd3lyRdu3ZNklS6dGmL83Z2drp7965u3rxpMa6Xl5fKlSunyMhI9e/f/7HbpRUuXDjZ9xcvXky2fcL+oK6urhbHbW1t5eLiYrF/aHLtEu5lx44dSe6jY8eOkpR4L5cvX06SL7nMsF4sY0eaGTNmjEaMGKGcOXMaHSXbGT9+vCpUqCA/Pz9VqVLF6DgAAABIQ7GxsTpw4ICuX78us9msF154QbVq1ZK9vb3R0QA8o0KFCunGjRuKiopK8rV86dIlSakrzrVu3Vr79u1LfJ9QDHVxcZEkbdy40WJmaIKE8wkCAwN14sQJValSRQMHDpS3t7fy5cuXpN/Vq1f10ksvWbyXJDc3t2TzJRRrr1y5okqVKiUej4mJ0c2bN5PkMJlMyWYtVKiQpk+fnuw1ypUrJym+UJqQ59+ZkT1Q7ESa2L17tyIiIvTjjz8aHSVbcnFxUUBAgPz9/RUaGprsfxgAAACQtVy/fl07d+6UyWRSnTp1VL16dZlMJt2+fVvr169XVFSU6tSpoxdffNHoqABSydvbW8HBwVq1apU6dOhgce6HH36Qq6trYvEuJVxcXJIUDCWpWbNmsrGx0blz59SsWbMnjrFz505NmDBBwcHBeuONN1S1alX16dNHixcvTtL2+++/T9yzU5KWLFmi3Llzy8PDI9mxvby85ODgoCVLlqhJkyaJx7/77jvFxMSoYcOGT73HV199VTNnzlTx4sVVqFChx7arW7eu1q1bpwcPHiQuZT9//rx+/vln9uzMJih2Ik2MHj1aI0eO5LfLBnrnnXc0Z84cfffdd+rcubPRcQAAAPActmzZIrPZrLZt2yZZRlqgQAG9/vrrMpvN2rNnjw4cOJD4gBEAWUPTpk3VrFkz+fr66tixY6pTp47u3bunJUuW6Mcff9SXX3752CXkqeHu7q6hQ4eqX79++uOPP9SwYUPlzJlT58+f16ZNm+Tn5ydvb2/dvn1b3bp1k7e3twYPHiyTyaQ5c+aoU6dO8vHxUY8ePSzGnTt3ruLi4lSrVi1t2LBB8+bNU0BAwGOfDJ8/f34NGjRIwcHBcnJyUosWLRQREaGRI0eqfv36atmy5VPvZeDAgfruu+/UoEEDDRw4UOXKldODBw907Ngx7dy5M3Hy1ciRI7V06VI1b95cH3zwgaKiojRmzBiWsWcjFDvx3LZv367Tp08n+eaHjJUjRw7NnDlTXbt2VatWrZQ7d26jIwEAAOAZrF+/XqVLl1bp0qWf2M5kMqlu3bq6cuWKli5dmrhvHYDMz2QyadWqVRo3bpy++uorBQUFyd7eXp6enlq5cqXatGmTZteaMGGCKlSooE8++USffPKJTCaTXnzxRTVp0iTxqea9e/fW33//ra+++ipxpWDHjh3Vs2dP9evXT/Xq1bP4nvTjjz/K399fQUFBypcvn0aOHKlRo0Y9Mcf48eNVsGBBzZ49W59++qlcXFz01ltvKTg4OEWF3Xz58mn37t0aO3asJk2apIsXL8rZ2VnlypVT+/btE9tVqFBB69at0wcffKA33nhDbm5uGjp0qMLCwhQaGvoMnyCyGpPZbDYbHQJZl9lsVqNGjfT2229T7MwkunXrphIlSmjChAlGRwEAAEAq7d+/Xzlz5nzsUtDHOXfunE6ePKnGjRunUzLAOBEREapQoYLRMSApICBAgYGBio6OtniAErIea/664mnseC7btm3T5cuX1a1bN6Oj4P9MnjxZc+bM0cmTJ42OAgAAgFQ6c+ZMqgudklS8eHHdvn1bzGUBAGR3FDvxzMxms0aNGqUxY8bwG51MxM3NTR988IEGDBhgdBQAAACkwqlTp+Tu7v7M/b28vLRnz540TAQAQNZDsRPPbOPGjbp9+zYPw8mEBgwYoOPHj2vt2rVGRwEAAEAKhYeHq1q1as/c383NTZcuXUrDRABgKSAgQGazmQlPyNQoduKZmM1mjR49WgEBAcqRI4fRcfAvDg4Omj59ugYMGKDIyEij4wAAACAF7OzsnnsMe3v7NEgCAEDWRbETz2TdunV6+PChOnToYHQUPMZrr72mChUqKCQkxOgoAAAASIG02G+TPTsBANkdxU6kWsKszsDAQNnY8E8oM5s6daqmTJmiCxcuGB0FAAAAT2EymTLFGAAAZGVUqpBqP/74o8xms9q1a2d0FDyFu7u7+vTpow8++MDoKAAAAHiK6Ojo556ZGRUVlUZpAADImih2IlXi4uI0ZswYBQYG8lvjLGL48OH6+eeftX37dqOjAAAA4Alq1Kih/fv3P3P/M2fOqFixYmmYCACArIdiJ1Jl+fLlsre3V6tWrYyOghRycnLSlClT5O/vr5iYGKPjAAAA4DFKlCihs2fPPnP/Tz/9VJMnT1ZEREQapgKsjNksXd8tHZsmHQqK//P67vjjAKwCxU6kWGxsrMaMGaOxY8cyqzOL6dixowoUKKDZs2cbHQUAAABP4O7uroMHD6a6359//qmmTZuqTp06atiwoXx9fXX69Ol0SAhkUXHR0onZ0ip3aVtz6eBQ6dCY+D+3NY8/fmJ2fDsAWRrFTqTY999/r3z58unVV181OgpSyWQyacaMGQoMDNT169eNjgMAAIDHqFatmq5fv65jx46luM+FCxcUHh6u5s2ba8iQITpx4oRKlCihmjV6rEGuAAAgAElEQVRrql+/frp8+XI6JgaygOj70pbG0q/vSw9OSzEPpLgoSeb4P2MexB//9X1pS5P49ulswYIFMplMyb42b96c7tf/p+XLl2vatGlJjm/evFkmk0m7du3K0DzA86LYiRSJiYlRQEAAszqzMA8PD3Xt2lUjRowwOgoAAACeoFmzZrp69arWrVv3xG2I4uLiFBoaqvDwcIuHh+bLl0+BgYE6duyYHBwcVKlSJQ0dOlQ3b97MiPhA5hIXLYW+Jt3cJ8U+fHLb2IfSzV+k0BYZNsNz6dKlCgsLs3jVrl07Q66d4HHFztq1ayssLExVq1bN0DzA87I1OgAyl0uXLum3335TbGysTCaTihcvrqpVq+rbb79V4cKF1aRJE6Mj4jkEBgaqfPny6t27t2rWrGl0HAAAADxGw4YNdefOHa1evVqxsbHy9PRU4cKFZWNjoxs3bujAgQMym81q0KCBChUqlOwYBQsW1Mcff6yBAwcqKChI5cqVU//+/TVgwADlyZMng+8IMMip+dKtX6W4yJS1j4uUbh2QTn0hlXknfbNJ8vT0VOnSpVPUNjIyUg4ODumc6P/LmzevvLy80mQss9ms6Oho2dvbp8l4wJMwsxMym83atWuXfvjhB509e1Y+Pj56/fXX1apVK+XOnVtLly7V7Nmz9eGHHzKrM4tzdnbW+PHj5e/vr7i4OKPjAAAA4Any5cundu3aqX379nr06JH279+vsLAw3bp1S23atFH79u0fW+j8p2LFiunzzz/Xnj179Mcff6h06dKaOnWqHj16lAF3ARjIbJaOTn76jM5/i30Y38/AhxYlLCFfuXKl3n77bRUoUEBubm6J59etW6c6deooV65ccnZ2Vrt27XTixAmLMerXr69GjRpp48aNqlatmhwdHeXh4aFVq1Yltunevbu++eYbnT17NnEZfULx9XHL2JctW6Y6derI0dFRzs7O6tSpky5cuGDRplixYvL19dXcuXNVrlw52dvba8OGDWn9MQHJotiZzd27d08LFixQ6dKl1b59e9WtW1e2tvETfk0mk9zd3dWxY0dt2bJF9+/f19GjRw1OjOf13//+V7Gxsfr666+NjgIAAIAUMJlM8vDwkLe3t5o2bapq1aopR44cqR6ndOnSWrRokTZv3qzt27erTJkymjt3rqKjeSALrNSNMCny2rP1jbwa3z+dxcbGKiYmJvEVGxtrcb5v376ytbXVN998o/nz50uS1qxZo1atWumFF17Q999/r08++UTh4eGqX7++rly5YtH/+PHjGjRokAYPHqzly5ercOHCat++feIDzAIDA+Xj46MiRYokLqNftmzZY/POmjVLnTp1UuXKlfXDDz9o9uzZCg8PV6NGjXT/vuVep5s2bUp8dsT69etVqVKltPjIgKdiGXs29uDBAy1fvlw9evSQjc2T6945c+ZUhw4dFBoaqri4OHl4eGRQSqQ1GxsbzZw5U+3atVPbtm2VL18+oyMBAAAgA1WuXFkrV67U3r17NWLECE2aNEljx45V586dn/pzAZBpHBgg3T745DYPL0gxqZzVmSDmoRT2luRY7PFtXvCUaiTd6zI1ypcvb/G+Xr16FjMpX375Zc2ZM8eizciRI1W2bFmtXbs28RcfderUUfny5RUSEqLJkycntr1x44Z27dqll156SZJUtWpVFS1aVEuXLtWQIUPk7u6uAgUKyMHB4alL1u/evavhw4fLz8/PIlOtWrVUvnx5LViwQP369Us8fufOHf32228pmoEOpCX+S5aNrVixQt27d0/V/6Fp1KiRTp06pb/++isdkyG91alTR6+++qrGjh1rdBQAAAAYpE6dOtq8ebPmzJmjGTNmyNPTU6tWrZLZwKW7QJoyx0p61n/P5v/rn75WrFihffv2Jb4SZm8m+OfDx6T4gmN4eLg6d+5sMcO7dOnS8vLy0vbt2y3aly9fPrHQKUmurq4qUKCAzp07l+qsP//8s+7fv69u3bpZzEYtUaKEypQpox07dli0f/nllyl0whDM7MymTpw4ocqVKz/T8pdWrVppzZo1atOmTTokQ0YJDg6Wh4eH/Pz8VKFCBaPjAAAAwCCNGzdWWFiY1qxZoxEjRmjChAmaMGGCGjdubHQ04PFSMqPy2DTp4FApLir149s4SOUGSOX7p75vKnh4eDzxAUWurq4W72/dupXscUkqUqSIwsPDLY7lz58/STsHB4dn2rP32rX4LQEaNWqUoqzJZQQyAsXObOr3339X+/btn6lvjhw5FBsbK7PZzAOLsrDChQtrxIgReu+997Rx40b+LgEAALIxk8mk1q1bq2XLlvruu+/0zjvvqESJEho/frzq1KljdDzg2bjUlmzsnrHYaSu51Er7TKn075/TEoqX/96bM+GYi4tLumVJGPvrr79OsvxekvLkyWPxnp8xYRSWsWdD0dHRsre3f64x6tWrp927d6dRIhilb9++unTpklasWGF0FAAAAGQCNjY26tKli44ePao33nhDHTp0UJs2bXTo0CGjowGpV6Cu5PCMy6hzFo7vn8nkzZtXnp6e+v777xUXF5d4/M8//9SePXvUsGHDVI/p4OCgv//++6nt6tevLycnJ506dUo1a9ZM8ipXrlyqrw2kB4qd2dD169efezp54cKFE6fPI+uys7PTzJkzNWjQID18+IwbdwMAAMDq2NnZqVevXjpx4oS8vb3VrFkzdevWTSdPnjQ6GpByJpNUcYiUwzF1/XI4ShWGxPfPhIKCghQREaHWrVtrzZo1Wrx4sZo3by4XFxcNHDgw1eNVrFhR165d05w5c7Rv3z4dPnw42XbOzs6aNGmSxo0bpz59+mjVqlUKDQ3VN998Iz8/P3333XfPe2tAmqDYmQ3dv39fTk5Ozz0OG5dbh8aNG6tWrVoWT+wDAAAAJClnzpwaMGCATpw4oQoVKsjLy0vvvPOOLly4YHQ0IGXce0r5q8fvwZkSNg5S/hqS+9vpm+s5tGrVSqtXr9aNGzfUoUMH9enTR5UrV9auXbtUpEiRVI/Xu3dvderUSUOHDlXt2rXVtm3bx7bt27evVqxYoYiICHXr1k0tWrRQQECAzGazqlat+jy3BaQZk5mKVbZz5coVnTt3TrVr136ucVavXq3WrVunUSoY6dy5c6pWrZoOHDigkiVLGh0HAAAAmdStW7c0efJkzZ07Vz169NDw4cNVsGBBo2PBykVERDzfQ1Wj70uhLaRbB6TYJ6xoy+EYX+hstE6yy/3s1wOygOf+usrEmNmZDRUoUECXL19+rjHOnDmjokWLplEiGK148eIaOHCgBg0aZHQUAAAAZGL58+fXxIkTdfjwYUVFRal8+fIaPXq07ty5Y3Q04PHscktNtkjVQySnlyRbp/+b6WmK/9PWScr9Uvz5JlsodAJZHMXObMjW1lbR0dHPtQz9wIEDql69ehqmgtEGDx6s8PBwbdq0yegoAAAAyORcXV01a9YsHThwQOfPn1eZMmU0efJk9oFH5mVjJ5V5R3r9pOS9UfKcJFUZG/+n9yap9cn48zZ2RicF8JwodmZTXl5e2rNnzzP1jYyMlL29vUyZdLNmPJucOXNq6tSpeu+99xQVFWV0HAAAAGQBJUuW1Jdffqnt27dr3759Kl26tD755BP+/yQyL5NJKviyVL6/5DEy/s+CdTPtw4gApB7FzmyqWLFiOn36tB49epTqvitXrlSTJk3SIRWM1rp1a5UsWVIzZ840OgoAAACykAoVKmjp0qVavXq11qxZo3LlymnhwoWKjY01OhoAIJuh2JmNdezYUYsXL1ZkZGSK+6xevVpeXl5ydHRMx2Qwislk0vTp0xUcHPzc+7oCAAAg+6lRo4Z++uknLVy4UPPmzVPlypX1ww8/PNcWWgAApAbFzmzMzs5Ob775ppYtW6bff//9iW2vXr2qRYsWydPTUyVKlMighDBC2bJl1bNnTw0bNszoKAAAAFmWr6+vTCaTxo0bZ3E8NDRUJpNJN27cMChZvAULFih37vR7CMsrr7yiHTt2KCQkROPHj1etWrW0YcMGip4AgHRHsTObs7OzU7du3RQbG6sWLVpo1apVOn36tG7duqULFy5o586d+uGHH3T8+HF169ZNL774otGRkQFGjhypLVu2aPfu3UZHAQAAyLJy5sypyZMn6/r160ZHMYTJZNKrr76q/fv3a9iwYRowYIAaNWqkXbt2GR0NAGDFKHZCkvTbb7/Jzs5OTZs21f3793XkyBFdu3ZN5cuXV/v27dWgQQMeSJSN5MmTR5MmTZK/vz/7LAEAADwjb29vlSxZUkFBQY9tc/ToUbVs2VJ58uRRoUKF1KVLF125ciXx/L59+9S8eXMVKFBAefPmVf369RUWFmYxhslk0meffaY2bdrI0dFRZcuW1bZt23ThwgX5+PjIyclJnp6e+vXXXyXFzy7973//qwcPHshkMslkMikgICBdPgNJsrGxUYcOHXTo0CH997//Vffu3dWiRYvEPAAApCWKnZAkzZ8/Xz179pSjo6MqV66sBg0aqHr16ipYsKDR0WCQrl27ytHRUfPnzzc6CgAAQJZkY2OjiRMnavbs2Tp16lSS85cvX9Yrr7wiDw8P/fLLL9q8ebPu37+v119/XXFxcZKke/fu6c0339TOnTv1yy+/yNPTUy1atEiyDH7cuHHq3LmzwsPDVbNmTXXp0kU9e/bUu+++q99++01FixaVr6+vJOnll1/WtGnT5OjoqMuXL+vy5csaPHhwun8etra28vX11R9//KGWLVuqVatW6tSpk44dO5bu1wYSmc3S7t3StGlSUFD8n7t3xx8HYBVMZjZNyfYiIiLUuHFjnTt3TnZ2dkbHQSZy8OBB+fj4KCIiQvnz5zc6DgAAQJbh6+urGzduaM2aNfL29lbhwoW1ZMkShYaGytvbW9evX9eMGTP0888/a8uWLYn9bt++rfz582vv3r2qXbt2knHNZrOKFi2qjz76SN27d5cUP7Nz2LBhCg4OliQdPnxYlStX1scff6xBgwZJksV1CxQooAULFqhfv366f/9+BnwayXvw4IFmzZqlKVOmqHXr1hozZgzPB0CyIiIiVKFChecbJDpamj9fmjxZunYt/n10tGRnF/8qVEgaMkTq2TP+PWDl0uTrKpNiZif05Zdf6q233qLQiSQ8PT3Vvn17jR492ugoAAAAWdbkyZO1dOlS7d+/3+L4gQMHtGPHDuXOnTvxlbBHfsJM0GvXrumdd95R2bJllS9fPuXJk0fXrl3TuXPnLMaqUqVK4v8uXLiwJKly5cpJjl27di3tb/AZOTk5aejQoTpx4oTc3NxUvXp1+fv7WyzjB9LE/ftS48bS++9Lp09LDx5IUVHxszmjouLfnz4df75Jk/j2GSAsLEydOnVS0aJFZW9vLxcXFzVr1kwLFy7MstuJrVy5UiEhIUmOJzycLTQ0NE2uk7AFR3KvlStXpsk1/i2t7yG9xgTFzmwvOjpaX331ld5++22joyCTCgoK0tKlSxUeHm50FAAAgCypVq1aat++vYYOHWpxPC4uTi1bttTBgwctXidOnFCrVq0kST169NC+ffs0depU7d69WwcPHlSxYsUUFRVlMdY/Jy4k7LWf3LGE5fGZibOzs4KCghQRESE7OztVqlRJw4cP161bt4yOBmsQHS299pq0b5/08OGT2z58KP3yi9SiRXy/dDRt2jTVq1dPt27d0qRJk7R582Z98cUXKlu2rPr06aM1a9ak6/XTy+OKnenB19dXYWFhSV4NGzbMkOunherVqyssLEzVq1c3OopVsTU6AIy1du1alSlTRuXKlTM6CjIpFxcXBQYGyt/fX9u3b+dBVQAAAM9gwoQJqlixotavX594rHr16vr+++9VokSJx66y2rVrl2bMmKGWLVtKkq5evarLly8/dx57e/tMN3OsUKFCCgkJ0cCBAxUUFKSyZctq4MCB6t+/v3Lnzm10PGRV8+dLv/4qRUamrH1kpHTggPTFF9I776RLpB07dmjQoEHq16+fZsyYYXGuTZs2GjRokB48ePDc14mOjpatrW2yP8NFRkbKwcHhua9hJDc3N3l5eRkd45nExsbKbDYrb968WfYeMjNmdmZz8+fPZ1YnnqpXr166f/++lixZYnQUAACALKl06dLq3bu3pk+fnnisb9++unPnjt544w3t3btXf/75pzZv3qzevXvr3r17kqSyZctq0aJFOnr0qPbt26fOnTvL3t7+ufOULFlSjx490qZNm3Tjxg09fNqMtwz04osvas6cOQoLC9ORI0dUunRpTZ8+XY8ePTI6GrIaszl+j87U/vt++DC+Xzo94mTixInKnz+/Jk+enOx5d3f3xK0pAgICki1W+vr6qmTJkonvz5w5I5PJpE8//VRDhgxR0aJF5eDgoL/++ksLFiyQyWTSjh071LFjRzk7O6tOnTqJfbdv364mTZooT548cnJyko+Pjw4fPmxxvUaNGql+/fravHmzqlevLkdHR3l4eFgsGff19dXChQt18eLFxCXl/8z4T/369VPhwoUV/a8ZtPfv31eePHk0fPjwJ36GKTFv3rwky9pjY2P1yiuvyN3dPfH7bMJnfOjQIXl7e8vR0VGurq4aPXr0U2fDm81mTZ06VeXKlZO9vb1cXV3Vr18/3b1716KdyWTSiBEjNHHiRJUqVUr29vY6dOhQssvYU/JZJ/j2229Vvnx55cyZU5UrV9aqVavUqFEjNWrU6Nk/OCtAsTMbu3Tpknbt2qWOHTsaHQWZXI4cOTRz5kx98MEHhm5iDwAAkJWNHj1atrb/f3Fd0aJF9fPPP8vGxkavvvqqKlWqpL59+8rBwSFxxtUXX3yh+/fvq0aNGurcubPefvvtxxYPUuPll1/W//73P3Xp0kUFCxZ8bNHFSGXKlNHixYu1YcMGbdmyRWXLltW8efMUExNjdDRkFWFh8Q8jehZXr8b3T2OxsbEKDQ1V8+bNlTNnzjQff/z48Tp+/LjmzJmjFStWWFyjW7duKlWqlJYtW6aJEydKil/t2aRJE+XOnVuLFi3S4sWLde/ePTVo0EDnz5+3GPvUqVPq37+/Bg0apOXLl8vV1VUdOnTQyZMnJUmjRo1SixYtVLBgwcQl5StWrEg257vvvqtr164lOf/NN9/owYMH6tWr11Pv1Ww2KyYmJskrgZ+fnzp27Cg/Pz9dvHhRUvw2bWFhYVq8eLHy5MljMV7btm3VtGlTrVy5Ul27dlVQUJDGjh37xAwjRozQoEGD1KxZM61evVpDhgzRggUL1LJlyySF0gULFmjt2rWaMmWK1q5dq6JFiz523Kd91pK0adMmdevWTeXLl9cPP/ygwYMHa8CAATp+/PhTPzurZ0a2FRwcbPbz8zM6BrKQ7t27m4cNG2Z0DAAAAGRDYWFhZm9vb3OZMmXM3377rTk2NtboSMggR48eTXqwf3+zuWHDJ7/c3c1mk8lsjp+jmbqXyRTf/0nj9++f6nu5cuWKWVKKf64aM2aMObnSTY8ePcwlSpRIfH/69GmzJHO1atXMcXFxFm2//PJLsyTzgAEDkozj7u5ubty4scWxO3fumF1cXMz9/3F/DRs2NNva2pqPHz+eeOzq1atmGxsb8/jx4y1yubm5JbnOtm3bzJLM27Ztsxjz39euVq2a2cfHJ0n/f5P02Nf169cT292+fdtcvHhxc6NGjcyhoaHmHDlymCdMmGAxVsJnHBwcbHHcz8/PnDt3bvPt27eTvYebN2+aHRwczD169LDo9/XXX5slmX/88UeLvK6uruaHDx+m6HNJyWddt25dc6VKlSz+vg8cOGCWZG7YsOFTP8Nkv66sBDM7s7Fhw4Zp7ty5RsdAFjJ58mTNnTtXJ06cMDoKAAAAshkvLy9t3bpVn332maZOnapq1appzZo1MqfTUmNYgdjYZ1+KbjbH989i2rZt+9jnLLRr187i/YkTJ3Tq1Cl169bNYmako6Oj6tatqx07dli0L1OmjMqUKZP4vlChQipUqJDOnTv3TFnfffddbdu2LfHny3379um3337TOyncK/Xtt9/Wvn37krycnZ0T2zg7O2vx4sXauXOnfHx81KBBgyQPi0vQqVMni/edO3fW/fv3kyzpT7Bnzx5FRkaqe/fuSfrZ2tpq+/btFsdfffVV5cqVK0X39rTPOjY2Vvv371f79u0t/r6rV6+uUqVKpega1owHFAFIMVdXVw0dOlQDBgzQ2rVrjY4DAACAbKhJkybas2ePVq1apeHDh2v8+PGaMGGCvL29U9Q/Li5ONjbM+8nypk1LWZuhQ6WoqNSP7+AgDRgg9e+f+r5P4OLioly5cuns2bNpOm4CV1fXFJ+79n9L/Hv27KmePXsmaV+8eHGL9/nz50/SxsHB4Zn3023Xrp2KFCmizz//XFOmTNHs2bNVtGhRtW7dOkX9XV1dVbNmzae28/LyUrly5XT06FH179//sV//hQsXTvZ9whL4f7t161Zijn+ytbWVi4tL4vl/5k2pp33WN27cUHR0tAoVKpSk3b/vIzviOzyAVOnfv79OnTqlNWvWGB0FAAAA2ZTJZFKbNm108OBB9evXT35+furSpcsTZ3leuXJFU6dOla+vr0aPHp3kwSiwQrVrS3Z2z9bX1laqVStt8yi+ENaoUSNt2rRJkSl4QnzCnptR/yrY3rx5M9n2j5vVmdw5FxcXSVJwcHCyMyRXr1791HzPw87OTn5+flqwYIGuXbumJUuWqGfPnhZ7G6eFwMBAnThxQlWqVNHAgQN1586dZNtdvXo12fdubm7Jtk8oSF65csXieExMjG7evJn4+SZ40t9NahUoUEB2dnaJBet/+vd9ZEcUOwGkir29vaZPn64BAwbwREwAAAAYKkeOHOrWrZuOHTumkJCQx7aLi4vTu+++q2nTpqlIkSLaunWr3NzctHTpUkliKby1qltXSmbmW4oULhzfPx0MGzZMN2/e1AcffJDs+dOnT+v333+XJJUoUUKSLJZS//XXX9q9e/dz5yhXrpxKliypI0eOqGbNmkleCU+ETw0HBwf9/fffKW7/zjvv6M6dO+rYsaMiIyNT9GCi1Ni5c6cmTJig8ePHa/Xq1frrr7/Up0+fZNt+//33Fu+XLFmi3Llzy8PDI9n2Xl5ecnBw0JIlSyyOf/fdd4qJiVHDhg3T5iaSkSNHDtWsWVM//PCDxfevAwcO6PTp0+l23ayCZewAUs3Hx0ceHh4KCQnRhx9+aHQcAAAAZHN2dnZPXCJ66dIlHT16VCNHjkwspkyaNEmzZs1Sy5Yt5ejomFFRkZFMJmnIEOn996WHD1Pez9Exvl8azsT7p1deeUUhISEaNGiQIiIi5Ovrq+LFi+v27dvasmWL5s2bp8WLF6tKlSp67bXXlC9fPvXq1UuBgYGKjIzU5MmTlTt37ufOYTKZ9Mknn6hNmzaKiopSp06dVKBAAV29elW7d+9W8eLFNWjQoFSNWbFiRd26dUufffaZatasqZw5c6py5cqPbe/m5qbWrVtrxYoVat26tV588cUUX+vixYvas2dPkuMlSpSQq6urbt++rW7dusnb21uDBw+WyWTSnDlz1KlTJ/n4+KhHjx4W/ebOnau4uDjVqlVLGzZs0Lx58xQQEGCxB+g/5c+fX4MGDVJwcLCcnJzUokULRUREaOTIkapfv75atmyZ4nt5FoGBgWrevLnatWun3r1768aNGwoICFCRIkWy/VYd2fvu8VS+vr5q1arVc4/j4eGhgICA5w+ETCMkJEQhISE6f/680VEAAACAJ0rY2++fRYvixYvr1KlTCg8PlxS/9HT+/PlGRUR66dlTql49fg/OlHBwkGrUkN5+O11jDRgwQLt27ZKzs7MGDx6sxo0by9fXVxEREfr8888T9610dnbWmjVrZGNjo06dOmn48OHy9/dP8R61T9OiRQvt2LFDDx48kJ+fn3x8fDRkyBBduXJFdZ9hZqufn586d+6sDz/8ULVr107R/psdO3aUpBQ/mCjBggULVLdu3SSvb775RpLUu3dv/f333/rqq68Sl5B37NhRPXv2VL9+/XTy5EmL8X788Udt2rRJr7/+uhYtWqSRI0dq1KhRT8wwfvx4hYSE6KefflKrVq00ceJEvfXWW1q7dm26FxybNWumb775RhEREWrXrp0mTZqkjz/+WEWKFFG+fPnS9dqZncnMfP0sLTQ09Inf5Bo1aqRt27Y98/h37tyR2Wx+7G8yUsrD4/+xd99RUV3v18D30JsNsSAIRpAiiNhFbGAhNqyUBAtqopGIGlRUYhQLqFHsmq9KswPW2INgB4wNOwYlNkZEiQ0QYRjm/cOf84bYEbgMsz9rzVLunHvvHpYIPPOcc2wxaNAgFjwrmRkzZiA1NfWttn0iIiIioorizz//xNKlS5Gamork5GSMHTsW7u7umDp1KlRUVLBu3TpYWloiOTkZrVu3Rr169RAUFPTWDssknJSUFFhbW5f8Ajk5QM+ewPnzH+7w1NF5Xeg8cAAohc5J+jReXl5ISEjA33//LUhHYmBgIGbNmgWJRFLq64WWt/T0dJibm+Pnn3/+aKH2i7+uKjB2diq4du3aISMj463HmjVrIBKJ4OPjU6LrFhYWQiaToVq1al9c6KTKa+rUqUhKSsKxY8eEjkJERERE9Ja8vDw4OzujXr16WLp0Kfbs2YM//vgDkyZNQteuXTFv3jxYWloCAJo1awaJRILJkyfDz88PZmZmOHDggMCvgEqFnh4QHw8sXgw0bAjo6r7u4BSJXv+pq/v6+OLFr8ex0FkuTp8+jf/973+Ijo6Gn5+f0k+9/lx5eXkYM2YMduzYgePHjyMiIgLdunWDjo4OvvvuO6HjCYr/khSchoYG6tatW+zx9OlTTJ48GQEBAfJ2cLFYDE9PT9SoUQM1atRAr169cPPmTfl1AgMDYWtri8jISJiZmUFTUxO5ublvTWPv3C9L3UQAACAASURBVLkzfHx8EBAQAAMDA9SuXRuTJk1CUVGRfMyjR4/Qt29faGtrw9TUFOHh4eX3CaFypaOjg5CQEPj6+qKwsFDoOERERERExWzduhW2trYICAhAhw4d0Lt3b6xatQoPHjzA6NGj4ejoCOD1BkVvHmPHjkV6ejr69OmD3r1746effsLLz1nvkSomdXVg9Gjg1i0gNhZYsACYPfv1n4cPvz4+enTJd2+nz+bg4IDJkydj2LBhJW7UUmaqqqp4+PAhxo4di27dusHPzw+NGjXCiRMnPriGsTJgsbOSefbsGfr164dOnTphzpw5AICXL1/CyckJWlpaOH78OJKSkmBoaIiuXbsW+6Z9+/ZtbNmyBdu2bcOlS5egpaX1znts3rwZampqSExMxMqVK7F06VJER0fLn/f29satW7cQFxeH3bt3Y8OGDbhz506Zvm4SzsCBA1G7dm2sXr1a6ChERERERMVIJBJkZGTgxYsX8mNGRkaoXr06zp8/Lz8mEokgEonkuxrHx8fj1q1bsLS0hJOTEzcwqkxEIqBdO2D8eGD69Nd/OjiU2WZE9H4ymQzZ2dkICwsTdPp4YGAgZDKZwk1h19DQwK5du5CRkYGCggI8ffoUe/bsee/u8cqExc5KpKioCN9++y1UVVWxadMm+QK8UVFRkMlkiIiIgJ2dHaysrLBmzRrk5ORg37598vMLCgqwceNGNG/eHLa2tu/9Qm/cuDFmz54NCwsLuLu7w8nJCfHx8QCA1NRUHDx4EGvXroWjoyOaNWuG9evXIy8vr+w/ASQIkUiE5cuXY86cOXj06JHQcYiIiIiI5Dp16oS6deti4cKFEIvFuHr1KrZu3Yr09HQ0atQIwOuCy5uZalKpFCdPnsTQoUPx/Plz7NixA66urkK+BCIi+kyKVbamDwoICEBSUhLOnDmDqlWryo+fP38et2/fRpUqVYqNf/nyJdLS0uQfGxsbo06dOh+9j52dXbGP69WrJy9ypaSkQEVFBa1bt5Y/b2pqinr16pXoNZFisLGxweDBgxEQEIDQ0FCh4xARERERAQCsrKwQERGBMWPGoGXLlqhZsyZevXoFf39/WFpaoqioCCoqKvJGkSVLlmDFihXo2LEjlixZAhMTE8hkMvnzRERU8bHYWUlER0dj0aJF2L9/v/wdyjeKiopgb2//zh2z9fX15X/X1dX9pHup/2cNE5FIJH8n9M20D1I+gYGBsLKywtmzZ9GqVSuh4xARERERAXj9xvyJEydw8eJF3Lt3Dy1atEDt2rUBvN6YVUNDA0+ePEFERARmz54Nb29vLFy4ENra2gDAQicRkYJhsbMSuHjxIkaMGIH58+fDxcXlreebN2+OrVu3wsDAoMx3Vre2tkZRURHOnj2Ldu3aAQDu3buHBw8elOl9SXjVqlVDcHAwxo4di6SkJO6kR0REREQVir29Pezt7QFA3qyhoaEBAJgwYQL279+P6dOnY9y4cdDW1pZ3fRIRkWLh/9wKLisrC/369UPnzp0xePBgPHz48K2Hl5cX6tSpg759++L48eO4ffs2Tpw4gYkTJxbbkb00WFpa4uuvv8bo0aORlJSEixcvwtvbW/6uKFVuw4YNg0gkwoULF4SOQkRERET0Xm+KmHfv3kXHjh2xa9cuzJ49G1OnTpVvRvTfQidnsRERKQZ2diq4/fv34+7du7h79y4MDQ3fOUYmk+HEiROYOnUq3Nzc8Pz5c9SrVw9OTk6oUaNGqWeKjIzE999/D2dnZxgYGGDmzJncuEZJqKio4OTJkwq3ix0RERERKSdTU1OMGTMGJiYmcHR0BIAPdnT6+vpi7NixsLS0LM+YVIpkMhnS09MhFouRn58PTU1NGBkZwdjYmEsWEFUSIhnfniIiIiIiIiL6oMLCQixcuBCLFy+Gq6srZsyYAVNTU6FjKYWUlBRYW1t/0TWkUimSk5ORkJCA3NxcFBUVQSqVQlVVFSoqKtDV1YWjoyOaNWsGVVXVUkpOVHGVxtdVRcVp7EQkmPz8fKEjEBERERF9EjU1NUybNg03b96EoaEhmjdvjvHjxyMzM1PoaPQRBQUF2LBhA2JjY/Hs2TNIJBJIpVIAr4ugEokEz549Q2xsLDZs2ICCgoIyzxQZGQmRSPTOR1ntteHt7Y0GDRqUybVLSiQSITAwUOgYVMmw2ElE5a6oqAjx8fFYvnw5Hj58KHQcIiIiIqJPVr16dcydOxfXr1+HSCRC48aN8fPPP+Pp06dCR6N3kEql2Lx5M8RiMSQSyQfHSiQSiMVibN68WV4MLWvbtm1DUlJSsUdcXFy53JuosmKxk4jKnYqKCl6+fIljx45hwoQJQschIiIiIvpsderUwdKlS5GcnIzMzExYWFhg3rx5yM3NFToa/UtycjIyMjI+uXgplUqRkZGB5OTkMk72mr29Pdq2bVvs0bJly3K595fgLD2qyFjsJKJy9WZKSJ8+fTBw4EDExMTg8OHDAqciIiIiIioZExMThIaG4tSpU7h06RLMzc2xfPlyFoMqAJlMhoSEhI92dP6XRCJBQkIChNzipKioCJ07d0aDBg3w/Plz+fErV65AW1sbkydPlh9r0KABBg8ejHXr1sHc3BxaWlpo3rw5jh49+tH7ZGRkYOjQoTAwMICmpibs7OywadOmYmPeTLk/ceIE3NzcUL16dbRp00b+/PHjx9GlSxdUqVIFurq6cHFxwdWrV4tdQyqVYvr06TA0NISOjg46d+6Ma9eulfTTQ/RBLHYSUbkoLCwEAGhoaKCwsBATJ06En58fHB0dP/uHDyIiIiKiisbS0hJRUVE4ePAgDh8+DAsLC4SHh8t/Dqbyl56eXuJO29zcXKSnp5dyordJpVIUFhYWexQVFUFFRQWbNm1CdnY2Ro8eDQDIy8uDp6cnbGxsEBQUVOw6x48fx+LFixEUFISoqChoamqiR48e+Ouvv95779zcXHTq1AkHDx5EcHAwdu/ejSZNmmDIkCFYu3btW+O9vLzw1VdfYfv27Zg/fz4AYP/+/ejSpQv09PSwadMmbNmyBdnZ2ejQoQPu378vPzcwMBDBwcHw8vLC7t270b17d7i6upbGp5DoLWpCB6CyER0djXXr1nGtDxJUWloaioqK0KhRI6ipvf7vZv369QgICICWlhZ++eUXuLq6wszMTOCkRERERESlw97eHnv37kViYiICAgKwYMECzJkzB4MGDYKKCvuNSsuhQ4c+uv7/ixcvStxYIZFIsGvXLlStWvW9Y+rWrYuvv/66RNd/w8rK6q1jvXr1wr59+2BsbIzQ0FAMGDAALi4uSEpKwt27d3HhwgVoaGgUOyczMxMJCQkwMTEBAHTp0gWmpqaYO3cuNm7c+M57R0RE4ObNmzh69Cg6d+4MAOjRowcyMzMxffp0jBw5stjO9IMGDcKvv/5a7Brjx49Hp06d8Pvvv8uPOTk5oWHDhggJCcHSpUvx9OlTLFmyBKNGjcKiRYsAAN27d4eqqiqmTp36+Z80oo9gsbOSCgsLw8iRI4WOQUpu8+bN2Lp1K1JSUpCcnAxfX19cvXoV3377LYYNG4amTZtCS0tL6JhERERERKWuXbt2OHr0KOLi4hAQEIDg4GAEBQWhZ8+eEIlEQsdTCkVFRYKe/yl27doFY2PjYsf+vRt7//79MXr0aIwZMwb5+fkIDw+HhYXFW9dp27atvNAJAFWqVEGvXr2QlJT03nufOHECRkZG8kLnG4MHD8bw4cNx/fp1NGnSpFiWf7t58ybS0tIQEBBQrINZR0cHDg4OOHHiBIDXU+9zc3Ph7u5e7HxPT08WO6lMsNhZCb18+RIFBQXo16+f0FFIyU2bNg0hISFo0aIFbt68iXbt2mHDhg1o37499PX1i4199uwZLl26hE6dOgmUloiIiIiodIlEInTr1g1du3bF7t27MWXKFAQHByM4OJg/936hT+moPH36NOLi4kq0s7qqqqp8w6CyZGtrC3Nz8w+OGTZsGNasWYPatWvj22+/feeYOnXqvPOYWCx+73WfPHkCQ0PDt47XrVtX/vy//Xfso0ePAAAjR458Z7PVm+JrRkbGOzO+KzNRaWAPfSWkra2No0ePQltbW+gopOTU1dWxevVqJCcnY8qUKVizZg1cXV3fKnQeOnQIP/30EwYMGID4+HiB0hIRERERlQ2RSIT+/fvj0qVLGDNmDIYPHw4XFxecO3dO6GiVmpGRUYmXDlBRUYGRkVEpJ/p8L1++xIgRI2Bra4vnz5+/txMyMzPzncc+9Br09fXfuRTAm2M1a9Ysdvy/Hclvnp83bx7Onj371mPv3r0A/n+R9L8Z35WZqDSw2FkJiUQiTougCsPLywuNGzdGamoqTE1NAUC+q+HDhw8xe/Zs/Pzzz/jnn39ga2uLoUOHChmXiIiIiKjMqKqqYvDgwbhx4wb69++Pvn37YuDAgbh+/brQ0SolY2Nj6OrqluhcPT29t6aXC2H8+PEQi8X4/fff8euvv2LZsmU4dOjQW+NOnz5dbEOg7Oxs7N+/Hw4ODu+9dqdOnZCeno6EhIRix7ds2YLatWvD2tr6g9ksLS3RoEEDXLt2DS1btnzrYWdnBwCws7ODrq4uYmJiip0fFRX10ddPVBKcxk5EZS48PByjR4+GWCyGkZGRvBhfVFQEqVSK1NRUREZGokmTJrC0tERgYCACAwOFDU1EREREVEY0NDTwww8/YNiwYVi1ahWcnJzg4uKCwMBANGzYUOh4lYZIJIKjoyNiY2M/a6MidXV1tGvXrlyaiC5evIisrKy3jrds2RK///47QkNDsXHjRjRs2BDjxo1DbGwsvL29cfnyZdSuXVs+vk6dOujevTsCAwOhqamJBQsWIDc3F7/88st77+3t7Y1ly5ZhwIABCAoKgrGxMTZv3ozDhw9jzZo1xTYneheRSIRVq1ahb9++KCgogLu7OwwMDJCZmYnExESYmJjAz88P1atXx08//YSgoCBUqVIF3bt3x9mzZxEWFlbyTxzRB7Czk4jKXOvWrbF9+3ZUrVpVvkg1ANSrVw9jx45Fq1atEB0dDQBYtGgRgoKC8PTpU6HiEhERERGVC21tbUyaNAk3b96EmZkZWrVqBR8fHzx48EDoaJVGs2bNYGho+NHC3RuqqqowNDREs2bNyjjZa25ubnBwcHjrkZGRge+//x5eXl4YPHiwfHxERAREIhG8vb3lM+aA112aEydOREBAADw8PPDq1SscPHjwnZsZvaGrq4vjx4+je/fumDp1Kvr27YtLly5h48aNGDVq1Cfl79mzJ06cOIHc3Fx89913cHFxgb+/Px4+fFisqzQwMBABAQHYuHEjXF1dERsbK5/mTlTaRLJ/f3UQEZURmUyG7777DlKpFKGhoVBVVZW/UxoVFYWQkBAcOHAAtWrVgp+fH3r27ImuXbsKnJqIiIiIqPxkZWVhwYIFCA8Px8iRIzFlypS31k1URikpKR+dUv0hBQUF2Lx5MzIyMj7Y4amurg5DQ0N4eXlBQ0OjxPcrbw0aNED79u2xadMmoaOQAvnSr6uKjJ2dCkomk4F1alIkIpEILVu2xJkzZ1BYWAiRSCTfFfHRo0eQyWTQ09MDAISEhLDQSURERERKx8DAAAsXLsTly5eRnZ0NS0tLzJo1Cy9evBA6mkLT0NDA0KFD0b17d1SvXh3q6uryTk9VVVWoq6ujRo0a6N69O4YOHapQhU4iehs7OysJmUwGkUgk/5OoojI3N8eQIUPg6+sLfX19iMVi9OnTB/r6+jh06BDU1LiUMBERERERAKSlpSEwMBCxsbHw9/eHj48PtLW1hY5V7kqzA00mkyE9PR1isRgFBQXQ0NCAkZERjI2NFfZ3aXZ2UklU5s5OFjsV0Lx58/Ds2TMsWLBA6ChEny0hIQFjxoyBrq4u6tevj9OnT8PIyAiRkZGwtLSUj5NKpUhMTESdOnU+uM4MEREREVFld/XqVcyYMQNnzpzBL7/8ghEjRkBdXV3oWOWmMhdliIRSmb+uOI1dAa1cuRLm5ubyj/fv34/ffvsNS5YswdGjR1FYWChgOqIPc3R0RGhoKBwcHPD48WOMGDECixcvhoWFRbGlGW7fvo3Nmzdj6tSpKCgoEDAxEREREZGwbG1tsXPnTuzatQs7duyAtbU1Nm3aJF8WioiI/j92diqYpKQkdOnSBU+ePIGamhomTZqEDRs2QFtbGwYGBlBTU8PMmTPh6uoqdFSiT1JUVAQVlXe/73Ls2DH4+fmhZcuWWLt2bTknIyIiIiKqmI4ePYqff/4ZL168wNy5c9G3b1+FnYL9KSpzBxqRUCrz1xU7OxXMwoUL4enpCS0tLcTExODo0aNYtWoVxGIxNm/ejEaNGsHLywsPHz4UOirRBxUVFQGAvND53/ddpFIpHj58iNu3b2Pv3r1clJ2IiIiI6P84OTkhISEBCxYsQGBgINq2bYu4uDhuYktEBBY7FU5iYiIuXbqEPXv2YMWKFRg6dCi++eYbAK+nNsyfPx9fffUVLly4IHBSog97U+TMzMwEgGLvRJ8/fx59+vSBl5cXPDw8cO7cOVStWlWQnEREREREFZFIJEKvXr1w4cIF+Pn5wcfHB126dEFSUpLQ0YiIBMVipwLJycmBn58fLC0t4e/vj1u3bsHe3l7+vFQqRd26daGiosJ1O0kh3LlzBz4+Prh58yYAQCwWY+LEiXB0dMTz589x6tQp/O9//4ORkZHASYmIiIiIKiYVFRV4eHjg+vXr8mYBV1dXXL58WehoRESC4JqdCuT69eto3LgxxGIxzpw5gzt37qBbt26wtbWVjzlx4gR69uyJnJwcAZMSfbrWrVvDwMAAgwYNQmBgICQSCebOnYuRI0cKHY2IiIiISOG8evUKa9euRXBwMJycnDBr1ixYWFgIHeuLlObagjKZDEnpSTgjPoPs/GxU0ayC1kat4WDsUKnXPSX6r8q8ZieLnQri/v37aNWqFVasWAE3NzcAgEQiAQCoq6sDAC5evIjAwEBUr14dkZGRQkUl+ixpaWnyndj9/Pwwffp0VK9eXehYREREREQKLScnB8uXL8eSJUvQr18/zJgxA/Xr1xc6VomURlFGIpUgLDkMvyb8ike5jyApkkAilUBdVR3qKuqorVsb/o7+GNlsJNRV1UspOVHFVZmLnZzGriAWLlyIR48ewdvbG3PmzEF2djbU1dWL7WJ948YNiEQiTJs2TcCkRJ/HzMwM06ZNg4mJCYKDg1noJCIiIiIqBXp6eggICEBqaipq1aoFe3t7/PTTT3j06JHQ0cpdTkEOnDc4Y2LsRNx+dhu5klwUSAsggwwF0gLkSnJx+9ltTIydiC4buiCnoGxnSkZGRkIkEr3zERcXBwCIi4uDSCTCqVOnyizH4MGDYW5u/tFxDx8+hK+vLywsLKCtrQ0DAwO0aNEC48ePlzdhfapbt25BJBJh06ZNn533yJEjCAwMLNVrUuXEYqeCiIiIQHx8PAIDA7Fu3Tps2LABAKCqqiof4+npiR07dsDS0lKomEQlMnfuXKSnp8v/XRMRERERUemoUaMGgoODce3aNUilUlhbW+OXX37Bs2fPhI5WLiRSCXps7oGz4rN4KXn5wbEvJS9xRnwGPTf3hET6eUW8kti2bRuSkpKKPVq3bg3g9XJfSUlJaNq0aZnn+JBnz56hdevWOHjwIPz8/HDgwAGsWbMGPXr0wJ49e5Cfn19uWY4cOYJZs2a9dbx+/fpISkrC119/XW5ZqGJTEzoAfdzOnTuhq6sLJycnNG3aFJmZmRg3bhwuX76MOXPmoHbt2igsLIRIJCpW/CRSJMeOHUN+fj5kMhnXyiEiIiIiKmV169bF8uXLMXHiRMyePRsWFhbw8/ODr68vdHV1hY5XZsKSw3Ah4wLypZ9WlMuX5uN8xnmEJ4djdMvRZZrN3t7+vZ2VVatWRdu2bcv0/p8iJiYG9+/fx9WrV2FjYyM/PnDgQMyZM6dC/O6mqalZIT5XVHGws1MBLF68GN7e3gAAfX19LFq0CKtXr8Yff/yBhQsXAgDU1NRY6CSF1r59e3Tp0qVCfLMkIiIiIqqsTE1NERYWhhMnTiA5ORmNGjXCypUry7VDr7zIZDL8mvDrRzs6/+ul5CV+TfgVQm5x8q5p7O3bt0fnzp0RGxuLZs2aQUdHB7a2ttizZ0+xc1NTUzF48GA0aNAA2traMDMzw48//liibt4nT54AeF0s/6///u5WUFCAgIAAmJqaQkNDAw0aNMCMGTM+OtW9ffv26Nq161vHjY2N8d133wEApk+fjqCgIPl9RSIR1NRe9++9bxr7+vXrYWdnB01NTdSqVQvDhg1DZmbmW/fw9vbG5s2bYWVlBV1dXbRq1QqJiYkfzEwVG4udFdyLFy+QlJSEUaNGAQCkUikAYOTIkfD398eqVavQp08f3LlzR8CUREREREREpEisrKwQHR2N/fv34+DBg7C0tERkZCQKCws/+RovXrzA7t27sWfPHvlj586dSEtLK8Pkny4pPQmPcku2RmlmbiaS0pNKOVFxUqkUhYWF8seb3/c/JDU1FX5+fpg0aRJ27tyJOnXqYODAgbh9+7Z8jFgshqmpKZYtW4Y//vgDP//8M/744w/07t37szO+mVbv7u6O2NhY5Obmvnfs4MGDsXDhQgwfPhz79u3D0KFDERwcjJEjR372ff/rhx9+kDeBvZnyn5CQ8N7xq1evhre3N5o0aYLdu3cjKCgI+/fvR+fOnfHyZfHi99GjR7F8+XIEBQUhKioKBQUF6N27N168ePHFuUkYnMZewVWtWhWPHz+Gvr4+gP+/Rqeamhp8fHxQq1Yt+Pv7Y9y4cYiKioKOjo6QcYlKzZt3UdnpSURERERUdpo1a4b9+/cjISEBAQEBWLBgAWbPno2BAwcW2xD33+7cuYNz586hSpUq6NWrF9TVi+9efuHCBWzfvh1GRkZwcHAok9wTDk3AxYcXPzgm/UX6Z3d1vvFS8hJDdw2FcVXj946xr2uPpV8vLdH1gdcF539zdHT86IZEWVlZOHXqFBo2bAgAaNq0KerVq4dt27bB398fAODk5AQnJyf5Oe3atUPDhg3h5OSEK1euoEmTJp+c0dnZGTNmzEBwcDCOHDkCVVVVNGvWDH369MGECRNQtWpVAMClS5ewbds2zJkzB9OnTwcAdO/eHSoqKpg1axamTp2Kxo0bf/J9/8vY2BhGRkYA8NEp64WFhZg5cya6dOmCzZs3y49bWFjAyckJkZGR8PHxkR/PyclBbGwsqlWrBgCoVasWHBwccOjQIbi7u5c4MwmHnZ0K4E2h813c3NywePFiZGVlsdBJlUpRURFatWqFI0eOCB2FiIiIiKjSc3R0xLFjx7Bs2TIsWLAALVu2xMGDB9+ayn3hwgWkpaVh0KBBcHFxeavQCQDNmzfHoEGDYGBggF27dpXXS3iLtEgKGUo2FV0GGaRFH++0/BK7du3C2bNn5Y+wsLCPnmNlZSUvdAKAoaEhDAwMcO/ePfmx/Px8zJ07F1ZWVtDW1oa6urq8+PnXX399ds5Zs2bh7t27WLduHQYPHozHjx9j5syZsLW1xePHjwEAx48fB/C6u/Pf3nz85vnycP36dWRlZb2VpXPnzjAyMnori6Ojo7zQCUBeDP7355QUCzs7K4H+/fujc+fOQscgKlWqqqoICAjAuHHjkJyc/M4fooiIiIiIqPSIRCJ0794d3bp1w65duzBx4kQEBwcjODgYHTp0wLVr15Cbm4suXbp80vUaNWoEXV1d7N27F3369CnVrJ/SUbn09FJMiZuCAmnBZ19fU1UTE9pOwPi240sS75PY2tq+d4Oi93lXM5SmpiZevXol/9jf3x+//fYbAgMD0bZtW1SpUgV3796Fm5tbsXGfo169evjuu+/ka2guW7YMEyZMQEhICObPny9f29PQ0LDYeW/W+nzzfHl4X5Y3ef6b5b+fU01NTQAo8eeKhMfOzkqiRo0aQkcgKnX9+/eHoaEhVq9eLXQUIiIiIiKlIRKJMGDAAFy5cgXff/89hg4diq+//hqnT59Ghw4dPuta9erVg7GxMVJSUsoo7fu1NmoNdZWSNU2oqaihlVGrUk5UPqKiojBixAgEBATA2dkZrVq1Kta5WBrGjx+PqlWr4vr16wD+f8Hw4cOHxca9+bhmzZrvvZaWlhYKCooXpGUyGZ4+fVqibO/L8ubYh7JQ5cBip4IRcjc4ovImEomwfPlyzJ07F48elWxhcSIiIiIiKhlVVVUMHToUf/31F5o3b46ePXuW6DrNmjWTF8XKk4OxA2rr1i7RuXX06sDBuGzWGy1reXl5b82Mi4iIKNG1MjIy3rlxUnp6OrKzs+Xdk506dQLwutD6b2/WzOzYseN772Fqaoq//vqr2OZYR48efWsjoTcdl3l5eR/M3LhxYxgYGLyV5fjx4xCLxfKsVHmx2KlAbt68iZCQEBY8SalYW1tj6NChmDZtmtBRiIiIiIiUkoaGBlq0aPHOacGfSldXFzk5OaWY6uNEIhH8Hf2ho/55+1voqOvAv52/wm6W6uLigvDwcPz222+IjY3F999/jzNnzpToWuvXr0fDhg0xa9YsHDx4EMeOHcPatWvh7OwMLS0t+UY/TZs2hZubG3755RfMmTMHhw8fRmBgIObOnYshQ4Z8cHMiT09PPHr0CCNGjEBcXBzWrFmDH3/8EVWqVCk27s01Fi1ahD///BPnz59/5/XU1NQwa9YsHDp0CMOGDcOhQ4cQGhoKNzc3WFlZYdiwYSX6XJDiYLFTgYSHhyMjI0Nh/8MlKqmZM2fi4MGDJf4GTUREREREJZebmyvfdbuknJ2dceLEiVJK9OlGNhuJ5obNoamq+UnjNVU10cKwBUY0G1HGycrO6tWr0atXL0ybNg0eHh549epVsV3JP0efPn3Qv39/7Nq1C15eXujWrRsCAwNhb2+PxMRENG3aVD522Ua2lgAAIABJREFU06ZNmDRpEkJDQ9GzZ09ERkZi2rRpH914qVu3bli1ahUSExPRp08fbNy4EVu2bHnr31zfvn0xevRoLF++HA4ODmjTps17r+nj44PIyEgkJyejb9++mDp1Knr06IFjx45xc2clIJKxTVAhFBYWwsTEBHFxcR98R4Soslq/fj1WrVqF06dPQ0WF79MQEREREZWXu3fv4vnz57Czs/ui65R0o6KUlBRYW1uX+L45BTnoubknzmecx0vJy/eO01HXQQvDFjjgdQB6Gnolvh+RIvjSr6uKjBUDBXHo0CGYmpqy0ElKa8iQIVBVVUVkZKTQUYiIiIiIlEphYSFUVVW/+DpC9Vrpaeghfmg8FndfjIbVG0JXXReaqpoQQQRNVU3oquuiYY2GWNx9MeKHxrPQSaTg1IQOQJ8mLCwMI0eOFDoGkWBUVFSwcuVK9O7dGwMGDED16tWFjkREREREpBT09fVx5cqVL7qG0JNK1VXVMbrlaIxqMQpJ6Uk4Kz6L7IJsVNGogtZGrdHWuC2XjCOqJDiNXQFkZmbC0tIS9+7d++J1UogU3ahRo6Cjo4OlS5cKHYWIiIiISGns2LEDAwcOLPH5iYmJaNCgAerVq/fZ51bm6bZEQqnMX1ecxq4ANm7ciP79+7PQSQQgKCgIW7ZswdWrV4WOQkRERESkNLS0tJCXl1fi8x88eFCiQicR0edisbOCk8lknMJO9C+1atXCjBkzMG7cOMGnwhARERERKYsuXbogLi6uROeKxWIYGhqWciIiondjsbOCS0pKQlFRERwdHYWOQlRh/PDDD8jKysL27duFjkJEREREpBS0tLSgp6eH1NTUzzrv1atXiIuLQ7t27b7o/mx0ICo9lf3ricXOCi4sLAwjRozgQslE/6KmpoYVK1Zg4sSJyM3NFToOEREREZFScHJyQlpaGlJSUj5pfHZ2NrZu3Ypvv/32i36nVVdX/6Ip9ERUXF5eHtTV1YWOUWa4QVEFlpOTg/r16yMlJQV169YVOg5RhfPNN9/AzMwMc+fOFToKEREREZHSSExMhFgsRps2bWBiYvLW87m5uVi9ejWMjIzg6ekJFZUv67N68eIFMjMzYWRkBG1tbTYDEZWQTCZDXl4exGIx6tSpU2n3hlETOgC9X0xMDDp27MhCJ9F7LFy4EE2bNsXw4cNhZmYmdBwiIiIiIqXQrl07yGQynD17FmfOnIGGhob8ucLCQmhra+PGjRt4+vTpFxc6AcgLMg8ePIBEIvni6xEpM3V19Upd6ATY2VmhOTo6YsqUKXB1dRU6ClGFNW/ePCQlJWHPnj1CRyEiIiIiov9z7949NGvWDCkpKahdu7bQcYhIibDYWUGlpKTA2dkZ9+7dq9TrKBB9qfz8fNja2mL58uXo0aOH0HGIiIiIiOj/+Pr6QkNDAyEhIUJHISIlwmJnBeXv7w+RSIQFCxYIHYWowtu/fz9++uknXLlyBZqamkLHISIiIiIiABkZGbCxscHVq1dRr149oeMQkZJgsbMCkkgkqF+/Po4fPw5LS0uh4xAphN69e6NDhw6YMmWK0FGIiIiIiOj/TJo0Ca9evcLKlSuFjkJESoLFzgpo9+7dCAkJwcmTJ4WOQqQwbt26hbZt2+LSpUswMjISOg4REREREQF4/PgxrKyscOHCBZiamgodh4iUwJdvi0alLiwsDCNGjBA6BpFCMTc3x6hRo+Dv7y90FCIiIiIi+j+1atXCDz/8gLlz5wodhYiUBDs7K5gHDx7AxsYG9+/fh56entBxiBRKTk4OrK2tsWXLFnTo0EHoOEREREREBODJkyewsLDA6dOnYW5uLnQcIqrk2NlZwWzYsAGDBg1ioZOoBPT09LBw4UL4+vpCKpUKHYeIiIiIiADo6+tj3LhxmD17ttBRiEgJsLOzApHJZLC0tMSGDRvQtm1boeMQKSSZTAYnJye4u7vDx8dH6DhEREREREREVI7Y2VmBnDx5EmpqamjTpo3QUYgUlkgkwvLlyxEYGIisrCyh4xARERERERFROWKxswIJDw/HyJEjIRKJhI5CpNDs7Ozg4eGB6dOnCx2FiIiIiIiIiMoRp7FXEC9evICJiQlSU1NRu3ZtoeMQKbynT5/C2toaBw4cQPPmzYWOQ0RERERERETlgJ2dFURUVBS6dOnCQidRKalRowbmzJkDX19f8D0dIiIiIiIiIuXAYmcFER4ejhEjRggdg6hSGTFiBPLz87Fp0yahoxARERERKb3AwEDY2toKHYOIKjlOY68Arl27hu7du+Pu3btQU1MTOg5RpXL69GkMHDgQKSkpqFq1qtBxiIiIiIgUire3N7KysrBv374vvlZOTg7y8/NRs2bNUkhGRPRu7OysAMLCwuDt7c1CJ1EZaNu2Lbp164Y5c+YIHYWIiIiISKnp6emx0ElEZY7FToEVFBRg06ZNGD58uNBRiCqt+fPnIyIiAjdu3BA6ChERERGRwjp79iy6d+8OAwMDVK1aFe3bt0dSUlKxMWvWrIGFhQW0tLRQq1YtuLi4oLCwEACnsRNR+WCxU2B79+5F48aNYW5uLnQUokqrbt26CAgIwPjx47lZERERERFRCWVnZ2PIkCE4efIkzpw5A3t7e/Ts2RNZWVkAgHPnzuHHH3/EzJkz8ddffyEuLg5ff/21wKmJSNmw2CmwsLAwjBw5UugYRJWer68v7t+/j99//13oKERERERECsnZ2RlDhgyBtbU1rKyssGLFCmhpaeHQoUMAgHv37kFXVxeurq4wNTVF06ZN8dNPP3HJNiIqVyx2Cig9PV2+eQoRlS11dXUsX74cfn5+yMvLEzoOEREREZHCefToEUaPHg0LCwtUq1YNVapUwaNHj3Dv3j0AQLdu3WBqaoqvvvoKXl5eWL9+PbKzswVOTUTKhsVOAUVGRsLd3R06OjpCRyFSCl27dkXz5s2xcOFCoaMQERERESmcYcOG4ezZs1iyZAkSExNx8eJFGBsbo6CgAABQpUoVXLhwATExMTAxMcG8efNgZWWFBw8eCJyciJQJi53lRCKR4NGjR3jw4AHy8vJQVFSEiIgITmEnKmchISFYvnw57t69K3QUIiIiIiKFcurUKfj6+qJXr16wsbFBlSpVkJGRUWyMmpoanJ2dMW/ePFy+fBm5ubnYt2/fJ12/qKioLGITkZLhwhllSCaT4fTp0xCLxdDW1kbNmjWhpqaGq1ev4vbt26hbty7s7OyEjkmkVExNTTFu3DhMnDgR27dvFzoOEREREZHCsLCwwKZNm9CmTRvk5ubC398fGhoa8uf37duHtLQ0dOzYEfr6+jh69Ciys7NhbW39Sdfftm0bPDw8yio+ESkJFjvLyM2bN3Hu3Dm0b98eDg4O7xzz7bff4uDBg9DX10fHjh3LOSGR8po8eTJsbGwQHx+PLl26CB2HiIiIiEghhIeHY9SoUWjRogXq1auHwMBAPH78WP589erVsXv3bsyePRsvX76EmZkZQkND0aFDh0+6/syZMzFw4EBuaEREX0Qkk8lkQoeobK5evYrMzMxPLqLcuHED9+7dQ/fu3cs4GRG9sXv3bgQEBODSpUtQV1cXOg4RERERkdLr2LEjvvvuOwwdOlToKESkwLhmZykTi8W4f//+Z3WLWVlZwcjICElJSWWYjIj+rW/fvqhfvz5WrlwpdBQiIiIiIgIwd+5cBAYGQiKRCB2FiBQYi52l7PTp0+jRo8dnn2djY4MHDx6AjbZE5UMkEmHZsmUIDg5GZmam0HGIiIiIiJRex44dYWZmhoiICKGjEJECY7GzFOXm5kJbW7vE57ds2RJnz54txURE9CFWVlbw9vbG1KlThY5CREREREQA5syZg7lz5+LVq1dCRyEiBcViZyk6cuTIF212Ympqirt375ZiIiL6mF9++QWxsbE4ffq00FGIiIiIiJRe27ZtYWdnh3Xr1gkdhYgUFIudpUgmk0FTU/OLrqGlpVVKaYjoU1StWhXz58+Hr68vioqKhI5DRERERKT0Zs+ejXnz5uHly5dCRyEiBcRiZwXDNTuJyt/gwYOhoaGB8PBwoaMQERERESm95s2bw8HBAatXrxY6ChEpIBY7S5FIJKoQ1yCizyMSibBixQpMnz4dT58+FToOEREREZHSmzVrFhYuXIjs7GyhoxCRgmGxsxQVFhZ+8TW4CDORMJo3b45+/fph5syZQkchIiIiIlJ6tra26NKlC5YvXy50FCJSMCIZ502XmrS0NLx48QLNmjUr0fmvXr1CmzZtYGNjA09PT7i4uHzxGqBE9On++ecfWFtbIz4+Hk2aNBE6DhERERGRUktNTYWjoyNu3ryJ6tWrCx2HiBQEOztLkZmZGdLS0kp8fnx8PPbs2YMOHTogJCQEhoaG8Pb2xqFDhyCRSEoxKRG9S82aNREYGAhfX1+un0tEREREJDALCwv07t0bixcvFjoKESkQFjtLmaGhYYkKnnl5ecjLy4OpqSnGjBmD48eP48qVK2jWrBlmzZqFevXqYdSoUYiPj4dUKi2D5EQEAKNHj8azZ88QExMjdBQiIiIiIqU3Y8YMrFq1CllZWUJHISIFwWnsZWDHjh1o37496tSp80njJRIJNm3ahCFDhkBNTe2dY+7evYuYmBhER0cjPT0dgwYNgoeHBxwdHaGiwpo1UWk6efIkvLy8kJKSAl1dXaHjEBEREREptTFjxqBq1apYsGCB0FGISAGw2FkGZDIZfv/9dzRq1Ag2NjYfHJuVlYW9e/fim2++gZaW1idd/9atW4iOjkZ0dDSePHkCd3d3eHh4oHXr1tzNnaiUeHl5oUGDBggKChI6ChERERGRUktPT0fTpk1x7do11K1bV+g4RFTBsdhZhi5fvozU1FRUr14dnTt3Lta1ef78edy5cwf6+vro1KlTibszr1+/Li985ufnw8PDAx4eHrC3t2fhk+gLiMViNG3aFKdPn4a5ubnQcYiIiIiIlNqECRMAAEuXLhU4CRFVdCx2loNnz57h5MmTyM7ORmhoKCZMmIAmTZrgq6++KrV7yGQyXL58GVFRUYiOjoaamho8PT3h4eHx0e5SInq3BQsW4NSpU9i7d6/QUYiIiIiIlNrDhw9hY2ODS5cuwdjYWOg4RFSBsdhZjp4/fw4TExM8f/68TO8jk8lw7tw5REVFISYmBtWqVZN3fFpYWJTpvYkqk/z8fDRp0gRLly5Fz549hY5DRERERKTUpkyZghcvXuC3334TOgoRVWAsdpaj/Px8VK1aFfn5+eV2z6KiIiQlJSE6Ohrbtm2DoaGhvPDZoEGDcstBpKgOHjyIcePG4erVq9DU1BQ6DhERERGR0srKyoKlpSXOnTtXqjMliahyYbGzHMlkMqiqqkIikUBVVbXc7y+VSnHixAlER0djx44dMDMzg4eHB9zc3DgNgOgDXF1d0a5dO0ydOlXoKERERERESm3GjBlIT09HeHi40FGIqIJisbOcaWtr459//oGOjo6gOSQSCY4cOYLo6Gjs3r0btra28PDwwKBBg1CnTh1BsxFVNGlpaWjTpg0uXboEIyMjoeMQERERESmtZ8+eoVGjRkhISOAybUT0Tix2ljN9fX3cunUL+vr6QkeRy8/PR2xsLKKjo7Fv3z60bNkSHh4eGDBgAGrWrCl0PKIKYfr06fj777+xZcsWoaMQERERESm1oKAgXL9+HZs3bxY6ChFVQCx2lrN69erh7NmzFbY7LC8vDwcOHEB0dDT++OMPtGvXDp6enujXrx+qVasmdDwiweTm5sLa2hqbNm1Cx44dhY5DRERERKS0srOzYW5ujvj4eNja2godh4gqGBWhAygbLS0tvHr1SugY76WtrY2BAwciJiYGYrEYw4YNw65du2BiYoK+ffti69atyMnJETomUbnT1dXFokWL4Ovri8LCQqHjEBEREREprSpVqmDy5MkIDAwUOgoRVUAsdpYzbW3tCl3s/Dc9PT14enpi9+7duHfvHgYOHIiNGzfCyMgIbm5u2L59O/Ly8oSOSVRu3NzcULNmTaxZs0boKERERERESs3HxweJiYlITk4WOgoRVTCcxk6f7Z9//sGuXbsQFRWFc+fOoVevXvDw8ICLiws0NTWFjkdUpq5evQpnZ2dcv34dBgYGQschIiIiIlJaK1asQGxsLPbu3St0FCKqQFjspC+SmZmJHTt2IDo6GleuXEHfvn3h4eGBLl26QF1dXeh4RGVi/PjxePXqFTs8iYiIiIgElJ+fj0aNGiEmJgZt27YVOg4RVRAsdlKpEYvF2LZtG6Kjo3Hr1i0MGDAAHh4e6NSpE1RVVYWOR1Rqnj17BisrK+zbtw8tW7YUOg4RERERkdJau3Yttm/fjtjYWKGjEFEFwWInlYk7d+4gJiYG0dHREIvFcHNzg4eHB9q1awcVFS4VS4ovLCwMoaGhSEhI4L9pIiIiIiKBSCQSWFlZISIiAh07dhQ6DhFVACx2Upm7efMmoqOjER0djWfPnsHNzQ2enp5o1aoVRCKR0PGISqSoqAht27bFjz/+iGHDhgkdh4iIiIhIaa1fvx5hYWE4fvw4f8ckIhY7FUHv3r1hYGCAyMhIoaN8sWvXrskLnxKJBO7u7vDw8IC9vT2/KZHC+fPPP9G/f3+kpKSgWrVqQschIiIiIlJKhYWFsLW1xYoVK9CtWzeh4xCRwDj38gskJydDVVUVjo6OQkdRGDY2Npg9ezZu3LiBnTt3AgAGDBgAKysrzJgxA9evXxc4IdGna9OmDb7++mvMnj1b6ChEREREREpLTU0NgYGB+OWXX8B+LiJisfMLrFu3Dj4+Prh69SpSUlI+OFYikZRTKsUgEolgb2+P+fPn4++//8bGjRuRm5uL7t27o0mTJpg7dy5u3rwpdEyij5o3bx42bNjw0f8DiIiIiIio7Li7uyM3Nxf79+8XOgoRCYzFzhLKy8vDli1b8P3332PQoEEICwuTP3fnzh2IRCJs3boVzs7O0NbWxpo1a/DPP//gm2++gbGxMbS1tWFjY4OIiIhi13358iW8vb2hp6eHOnXqIDg4uLxfWrkTiURo3bo1QkJCcO/ePfz222/IzMxEhw4d0KJFC/z666+4c+eO0DGJ3qlOnTr4+eefMW7cOL6LTEREREQkEBUVFcyePRszZsxAUVGR0HGISEAsdpbQ9u3bYWpqCjs7OwwZMgQbNmx4q3tz2rRp8PHxwfXr19GvXz+8evUKzZs3x759+3Dt2jWMHz8eo0ePRnx8vPycSZMm4fDhw9ixYwfi4+ORnJyMEydOlPfLE4yKigrat2+PFStWQCwWY+HChUhLS0OrVq3Qtm1bLF26FGKxWOiYRMX8+OOPePDgAXbt2iV0FCIiIiIipdWvXz+IRCL+XE6k5LhBUQl16tQJffr0waRJkyCTyfDVV18hJCQEAwcOxJ07d/DVV19h0aJFmDhx4gev4+npCT09PYSGhiInJwc1a9ZEeHg4vLy8AAA5OTkwNjZGv379KsUGRSUlkUhw5MgRREVF4ffff4etrS08PDwwaNAg1KlTR+h4RDhy5AhGjBiB69evQ0dHR+g4RERERERK6cCBA5g8eTIuX74MVVVVoeMQkQDY2VkCt27dQkJCAr799lsAr6dhe3l5ITQ0tNi4li1bFvtYKpUiKCgIdnZ2qFmzJvT09LBz507cu3cPAJCWloaCggI4ODjIz9HT00OTJk3K+BVVfOrq6nBxcUFERAQyMjIwadIkJCYmwtLSEl27dkVoaCiePHkidExSYs7OzmjVqhV+/fVXoaMQERERESmtHj16oFq1aoiOjhY6ChEJRE3oAIooNDQUUqkUJiYm8mNvGmTv378vP6arq1vsvEWLFiEkJATLli1DkyZNoKenh4CAADx69KjYNejDNDU14erqCldXV+Tl5eHAgQOIiorCxIkT4ejoCA8PD/Tr1w/VqlUTOiopmZCQEDRr1gze3t5o0KCB0HGIiIiIiJSOSCTCnDlzMGbMGLi7u0NNjWUPImXDzs7PVFhYiPXr12PevHm4ePGi/HHp0iXY2dm9teHQv506dQp9+vTBkCFDYG9vDzMzM6SmpsqfNzc3h7q6Ok6fPi0/lpubi6tXr5bpa1Jk2traGDhwILZt2waxWIwhQ4Zg165dMDExQb9+/bB161bk5OQIHZOUhImJCSZMmAA/Pz+hoxARERERKS1nZ2cYGRlh48aNQkchIgGw2PmZ9u/fj6ysLHz//fewtbUt9vD09ER4ePh7d36zsLBAfHw8Tp06hRs3bmDs2LG4ffu2/Hk9PT2MHDkSU6ZMweHDh3Ht2jWMGDECUqm0vF6eQtPT08M333yD3bt34+7du+jfvz82btwIIyMjuLu7Y8eOHcjLyxM6JlVykydPxsWLF3H48GGhoxARERERKaU33Z2zZ89GQUGB0HGIqJyx2PmZwsLC4OTkhJo1a771nJubG+7evYu4uLh3njt9+nS0bt0aPXr0QMeOHaGrqyvfiOiNRYsWwcnJCf3794eTkxNsbW3RsWPHMnktlVn16tUxbNgwHDhwAH///Te6deuG3377DYaGhhg8eDD27t2L/Px8oWNSJaSlpYUlS5Zg3Lhx/MGKiIiIiEgg7du3h6WlJcLDw4WOQkTljLuxk1LJzMzE9u3bER0djatXr6Jv377w9PSEs7Mz1NXVhY5HlYRMJkOPHj3QrVs3TJw4Ueg4RERERERK6ezZs+jfvz9u3boFLS0toeMQUTlhsZOUVnp6OrZt24bo6GikpaVhwIAB8PT0RMeOHaGqqip0PFJwf/31FxwdHXHlyhUYGhoKHYeIiIiISCn17dsXzs7OGD9+vNBRiKicsNhJBODOnTuIiYlBVFQUMjIyMGjQIHh6esLBwQEqKlztgUrG398fmZmZWL9+vdBRiIiIiIiU0qVLl3D+/HkMHz4cIpFI6DhEVA5Y7CT6j9TUVHnh8/nz53B3d4eHhwdatWrFb470WbKzs2FtbY2YmBi0a9dO6DhEREREREpJJpPxdzkiJcJiJ9EHXLt2DdHR0YiKikJhYSE8PDzg4eGBpk2b8pslfZLNmzdj8eLFOHPmDJdHICIiIiIiIipjLHYSfQKZTIaLFy8iOjoa0dHR0NDQgKenJzw8PNC4cWOh41EFJpPJ0LFjRwwZMgSjRo0SOg4RERERERFRpcZiZznLzMxEkyZN8OjRI6GjUAnJZDKcOXMG0dHRiImJQY0aNeSFT3Nzc6HjUQV08eJFuLi4ICUlBfr6+kLHISIiIiIiIqq0WOwsZ8+fP0f9+vXx4sULoaNQKSgqKkJCQgKio6Oxfft2GBkZwdPTE+7u7jA1NS3R9SQSCTQ1NcsgLQnJx8cHKioqWLlypdBRiIiIiIjoX86fPw8tLS3Y2NgIHYWISgGLneWsoKAAenp6KCgoEDoKlTKpVIrjx48jKioKO3fuRKNGjeDh4QE3NzcYGRl90jVSU1OxbNkyPHz4EM7Ozhg+fDh0dHTKODmVh3/++QeNGzdGbGwsmjZtKnQcIiIiIiKll5iYiJEjR+LevXuoW7cunJ2dMX/+fNSsWVPoaET0BVSEDqBs1NXVUVhYCKlUKnQUKmWqqqpwdnbG2rVrkZGRgZkzZ+LixYto0qQJOnXqhNWrVyM/P/+D13j69Cn09fVhZGQEX19fLF26FBKJpJxeAZWlmjVrYtasWfD19QXfYyIiIiIiEtbz58/xww8/wMLCAn/++SfmzJmDzMxMjBs3TuhoRPSF2NkpAB0dHTx+/Bi6urpCR6FykJ+fjz/++ANRUVHYsGED1NTUPnrO/v37MWLECGzduhXOzs7lkJLKg1QqRatWrTB58mR88803QschIiIiIlIqL1++hIaGBtTU1HDkyBH571wODg4AgGvXrsHBwQHXrl1D/fr1BU5LRCXFzk4BaGtr49WrV0LHoHKiqakJV1dXbNmyBaqqqh8c+2Z5g61bt6Jx48awtLR857hnz55h8eLF2LlzJ7sEFYiqqipWrFiByZMnIycnR+g4RERERERK4+HDh9i4cSNSU1MBAKampkhPT4e9vb18jK6uLuzs7PD06VOhYhJRKWCxUwBaWlosdiopkUj0wec1NDQAAIcOHYKLiwtq164N4PXGRUVFRQCAuLg4zJw5E5MmTYKPjw8SEhLKNjSVKkdHRzg5OSEoKEjoKERERERESkNdXR2LFi3CgwcPAABmZmZo06YNfH19kZ+fj5ycHAQFBeHevXvs6iRScCx2/j/27jsqqrN7G/A9BRiqgnTBjr1GFBsqYgkajEoUG/beTTCvHQsSe2yJvhqFiAUUeRU0BjWKgp3YOxAbiqiggiB15vsjP/kklqACzwxzX2u5hMM5Z+5jlgb27Gc/AigUCrx69Up0DFIzr+e47tu3D0qlEi1atICOjg4AQCqVQiqVYuXKlRg+fDjc3NzQpEkTdOvWDVWqVClwn8ePH+PPP/8s8fxUeIsXL8aGDRsQGxsrOgoRERERkVYoV64cGjdujLVr1+Y3H+3Zswfx8fFwdnZG48aNERMTg40bN8LU1FRwWiL6HCx2CsDOTvoQf39/ODo6olq1avnHzp07h+HDh2Pr1q3Yt28fmjZtivv376NevXqwtbXNP+/nn39Gly5d0LNnTxgaGmLKlClIT08X8Rj0ATY2NvjPf/6DSZMmiY5CRERERKQ1fvzxR1y6dAk9e/bE//73P+zZswc1a9ZEfHw8VCoVRo4cidatW2Pfvn1YtGgRkpKSREcmok/AYqcAnNlJ/6RSqfLneR4+fBhffvklzM3NAQBRUVHw8vJCo0aNcPz4cdSuXRubNm1C2bJlUb9+/fx7HDhwAFOmTEHjxo1x5MgR7Ny5E2FhYTh8+LCQZ6IPmzhxIuLj47F3717RUYiIiIiItIKNjQ02bdoEOzs7jBw5EsuWLcO1a9cwZMgQREVFYdSoUdDT08O9e/cQERGB77//XnRkIvoo+0jcAAAgAElEQVQE/74tNBU5LmOnN+Xk5GDRokUwMjKCXC6Hnp4eWrZsCV1dXeTm5uLSpUu4desWNm/eDJlMhpEjR+LAgQNwdnZGnTp1AACJiYmYO3cuunTpgnXr1gH4e+D21q1bsWTJEri7u4t8RHoHXV1drFy5EmPHjkX79u2hUChERyIiIiIiKvWcnZ3h7OyMZcuW4fnz59DV1c1vNMnNzYVcLseoUaPQsmVLODs74/Tp03BychKcmog+Bjs7BeAydnqTVCqFsbExFixYgAkTJiApKQn79+9HYmIiZDIZhg8fjlOnTsHZ2RnLly+Hjo4Ojh07hszMTJQpUwbA38vcT58+jalTpwL4u4AK/L2boK6ubv48UFIvnTp1Qt26dbF8+XLRUYiIiIiItIqBgQEUCsVbhc68vDxIJBLUr18fXl5eWLNmjeCkRPSxWOwUgMvY6U0ymQwTJ07EkydPcPfuXcyaNQv//e9/MXjwYCQnJ0NXVxeNGzfGkiVLcPPmTYwcORJlypRBWFgYxo8fDwA4duwYbG1t8cUXX0ClUuVvbHTnzh1UqVKFncRqbPny5Vi+fDnu378vOgoRERERkVbIy8uDq6srGjZsiClTpuCPP/7I/5np9XgxAEhLS4OBgQGbR4g0DIudArCzk97H3t4ec+fORWJiIjZv3pz/LuObLl26hG7duuHy5ctYtGgRACA6OhqdOnUCAGRnZwMALl68iJSUFFSoUAFGRkYl9xD0UapUqYIxY8ZgypQpoqMQEREREWkFmUwGR0dHJCQkIDk5GX369EGTJk0wYsQIhISE4OzZswgPD0doaCiqVq1aoABKROqPxU4BOLOTCsPS0vKtY7dv30ZMTAzq1KkDOzs7GBsbAwCSkpJQo0YNAIBc/vco3j179kAul6N58+YA/t4EidTT1KlTcfLkSURGRoqOQkRERESkFebOnQu5XI6xY8ciISEBU6dORU5ODqZOnYru3bvDw8MDAwYM4CZFRBpIomIFpMQNHz48/10josJSqVSQSCSIjY2FQqGAvb09VCoVcnJyMGbMGFy9ehXR0dGQyWRIT0+Hg4MD+vbtCx8fn/yi6Ov7xMTEwNTUFNWqVRP4RPSmkJAQzJs3D+fOncsvWBMRERERUfGZPHkyoqOjcfbs2QLHY2Ji4ODgkL9HwuufxYhIM7CzUwDO7KRP8fp/rg4ODrC3t88/pquri+HDh+P58+cYPnw4/Pz84OTkBBMTE3z77bcFCp2v7dq1Cy1btoSjoyOWLFmCu3fvluiz0Ns8PDxgYWGBtWvXio5CRERERKQVli5divPnzyM8PBzA35sUAYCjo2N+oRMAC51EGobFTgG4jJ2KkkqlgpOTE/z9/ZGamorw8HAMHDgQe/bsga2tLZRKZYHzJRIJFi5ciAcPHmDRokW4desWGjdujBYtWmDlypV4+PChoCfRbhKJBKtWrcK8efPw5MkT0XGIiIiIiEo9mUyG6dOnY//+/QDAFVZEpQSXsQswe/ZsyGQy+Pj4iI5CBADIycnBoUOHEBwcjD179qBBgwbw9PSEh4fHO2eHUvGZPHkyXr58iQ0bNoiOQkRERESkFW7cuIEaNWqwg5OolGBnpwBcxk7qRkdHB25ubggICEBiYiImT56MqKgoVK9eHR06dMDGjRuRkpIiOqZWmDNnDvbu3YuYmBjRUYiIiIiItELNmjXfKnSyL4xIc7HYKYBCoWCxk9SWQqHA119/jW3btuHhw4cYMWIE9u/fj8qVK6NLly4IDAxEamqq6JilVpkyZeDn54dx48a9NYKAiIiIiIiKl0qlgkqlwrNnz0RHIaJPxGKnAJzZSZrCwMAAPXv2REhICBISEtC3b1/s3LkT9vb26N69O4KDg5Geni46ZqkzcOBAAMDmzZsFJyEiIiIi0i4SiQS//fYbOnXqxO5OIg3FYqcAXMZOmsjY2Bj9+vVDWFgY7ty5g65du8Lf3x+2trbw9PREaGgoi/hFRCqVYvXq1Zg+fTpevHghOg4RERERkVZxc3NDTk4OwsLCREchok/AYqcAXMZOms7U1BSDBw/G77//jvj4eLi6umLNmjWwtbWFl5cX9u7di+zsbNExNVqTJk3QuXNnzJ07V3QUIiIiIiKtIpVKMW/ePMyePZujpYg0EIudAnAZO5Um5ubmGDFiBA4fPozr16/DyckJCxcuhI2NDYYOHYoDBw4gNzdXdEyN5Ofnh8DAQFy7dk10FCIiIiIireLu7g49PT2EhISIjkJEH4nFTgHY2UmllbW1NcaNG4fo6GhcuHABderUwcyZM2Fra4vRo0cjMjISeXl5omNqDEtLS8yaNQsTJkzgvCAiIiIiohIkkUgwf/58+Pj48GcYIg3DYqcAnNlJ2sDe3h7ffvstzpw5g1OnTqFixYqYPHky7O3tMXHiRJw4cYJLQgphzJgxSEpKQmhoqOgoRERERERapWPHjjA3N8e2bdtERyGijyBRsV2oxJ0+fRoTJkzA6dOnRUchKnE3b95EcHAwgoKC8PLlS/Tq1Qu9e/dG48aNIZFIRMdTS5GRkRg0aBCuXbsGAwMD0XGIiIiIiLRGZGQkhg0bhuvXr0NHR0d0HCIqBHZ2CsCZnaTNatSogdmzZ+Pq1avYt28fFAoF+vTpg2rVqmH69Om4ePEil2z/Q9u2beHk5IRFixaJjkJEREREpFXatm2LSpUq4ddffxUdhYgKiZ2dAty6dQtfffUVbt26JToKkVpQqVQ4f/48goKCsGPHDujr68PT0xOenp6oVauW6Hhq4f79+2jUqBHOnj2LypUri45DRERERKQ1Tp48id69e+PWrVvQ09MTHYeI/gU7OwXgBkVEBUkkEnzxxRdYvHgxbt++DX9/fzx//hzt27dHgwYN4Ofnh/j4eNExhbK3t8fkyZPx7bffio5CRERERKRVmjdvjrp16+KXX34RHYWICoGdnQI8fvwYderUwZMnT0RHIVJrSqUS0dHRCAoKwq5du1ChQgV4enqiV69eqFChguh4JS4zMxN169bFTz/9hE6dOomOQ0RERESkNf7880907doVcXFx0NfXFx2HiD6AxU4BUlNTUb58eaSlpYmOQqQxcnNzERkZieDgYISGhqJGjRro3bs3evbsCRsbG9HxSkx4eDi8vb1x+fJl6Orqio5DRERERKQ1evTogVatWnG1FZGaY7FTgJycHBgYGCAnJ0d0FCKNlJ2djUOHDiE4OBhhYWFo0KABevfuDQ8PD1hYWIiOV6xUKhW6dOkCFxcXTJkyRXQcIiIiIiKtcfnyZXTo0AFxcXEwMjISHYeI3oPFTgFUKhXkcjmysrIgl8tFxyHSaJmZmfj9998RHByM/fv3o2nTpvD09ET37t1hZmYmOl6xuHXrFlq0aIFLly7B1tZWdBwiIiIiIq3Rp08f1K9fH9OmTRMdhYjeg8VOQQwNDZGUlMR3g4iKUEZGBvbt24egoCAcOnQIzs7O8PT0xNdffw0TExPR8YrU1KlT8eDBAwQGBoqOQkRERESkNW7evIlWrVohLi4OZcqUER2HiN6BxU5BzM3NcePGDZibm4uOQlQqpaamIiwsDMHBwTh27BhcXV3h6emJr776CoaGhqLjfbaXL1+iZs2aCA4ORsuWLUXHISIiIiLSGoMGDUKlSpUwZ84c0VGI6B1Y7BTEzs4Op06dgp2dnegoRKXes2fPsHv3bgQFBeHUqVNwc3ODp6cn3NzcoFAoRMf7ZNu2bcOSJUsQExMDmUwmOg4RERERkVb466+/0LRpU9y8eRPlypUTHYeI/kEqOoC2UigUePXqlegYRFrB1NQUgwcPRkREBOLi4uDi4oLVq1fDxsYGAwYMwL59+5CdnS065kfr06cPjI2NsWHDBtFRiIiIiIi0RpUqVeDh4YGlS5eKjkJE78DOTkHq1q2L7du3o169eqKjEGmtxMREhISEIDg4GNevX0e3bt3Qu3dvuLi4aMzmYRcvXkSHDh1w/fp1vqtMRERERFRC7t+/j4YNG+LatWuwsrISHYeI3sDOTkH09fWRmZkpOgaRVrOxscH48eMRHR2N8+fPo3bt2pgxYwZsbW0xevRoREZGIi8vT3TMD2rQoAF69uyJWbNmiY5CRERERKQ17O3t0a9fPyxatEh0FCL6B3Z2CuLs7IwFCxagdevWoqMQ0T/Ex8djx44dCA4OxuPHj9GzZ0/07t0bzZo1g0QiER3vLSkpKahVqxYiIiLQsGFD0XGIiIiIiLRCYmIi6tSpg8uXL6N8+fKi4xDR/2FnpyAKhYKdnURqqmrVqpg2bRouXLiAw4cPw8zMDEOHDkWlSpUwZcoUxMTEQJ3eJzIzM8O8efMwfvx4tcpFRERERFSa2djYYOjQofDz8xMdhYjewGKnIFzGTqQZatasCR8fH1y9ehV79+6Fnp4eevfuDQcHB8yYMQOXLl1SiwLjsGHDkJGRgW3btomOQkRERESkNb7//nsEBQXh7t27oqMQ0f9hsVMQdnYSaRaJRIJ69erB19cXsbGxCA4ORk5ODtzd3VG7dm3MnTsXN27cEJZPJpNh9erV+P7775GWliYsBxERERGRNrGwsMDo0aMxf/580VGI6P+w2CmIQqHAq1evRMcgok8gkUjQuHFjLF68GLdv38amTZvw7NkztGvXDg0aNICfnx/i4+NLPFeLFi3g6uoKX1/fEn9tIiIiIiJt9d1332H37t2Ii4sTHYWIwGKnMOzsJCodpFIpmjdvjhUrVuD+/ftYtWoVEhIS0Lx5czRp0gTLli3D/fv3SyzPokWLsHHjRty8ebPEXpOIiIiISJuZmppi0qRJmDt3rugoRAQWO4XhzE6i0kcmk6FNmzb4+eef8fDhQ/j5+eH69eto2LAhWrZsiVWrViExMbFYM9jY2GDatGmYNGmSWswSJSIiIiLSBhMnTsSBAwdw7do10VGItB6LnYJwGTtR6SaXy9GhQwf88ssvSExMxPTp0xETE4PatWvDxcUF69atw5MnT4rltcePH487d+4gPDy8WO5PREREREQFGRsbw9vbG3PmzBEdhUjrsdgpCJexE2kPXV1ddOnSBZs3b0ZiYiImTpyIyMhIVKtWDZ06dcqf+VmUr7dq1SpMnjyZ/84QEREREZWQsWPHIjo6GhcuXBAdhUirsdgpCJexE2knhUKBbt26ISgoCA8fPsTQoUOxd+9eVKxYEe7u7tiyZQtSU1M/+3U6dOiABg0aYOnSpfnH0tLSEBcXhytXruD+/fvIy8v77NchIiIiIqK/GRgYYOrUqZg9e7boKERaTaLiUDchVqxYgTt37mDFihWioxCRGkhNTUVYWBiCgoIQFRUFV1dX9O7dG126dIGhoeEn3fPOnTto3Lgx/P39kZ2dDRMTE9jZ2UGhUOD58+e4c+cOVCoVWrduDQsLiyJ+IiIiIiIi7ZOZmQkHBwfs2rULTZs2FR2HSCux2CnIunXrcP78efz3v/8VHYWI1MyzZ8/wv//9D8HBwTh16hTc3NzQu3dvfPnll1AoFIW+T0JCAvz9/dGvXz9UqVLlnecolUpERUXhyZMn8PDwgEQiKarHICIiIiLSSv/9738RGhqKiIgI0VGItBKXsQvCmZ1E9D6mpqYYMmQIIiIiEBcXh7Zt22LlypWwsbHBgAED8NtvvyE7O/uD97h9+zbOnz+PWbNmvbfQCQBSqRRt2rSBq6srtm7dyh3ciYiIiIg+0+DBg3Hr1i1ERUWJjkKklVjsFIQzO4moMCwsLDBq1CgcOXIEV69ehaOjIxYsWAAbGxsMGzYMBw8eRG5uboFrUlNTERMTA3d390K/jqmpKTp37ow9e/YU9SMQEREREWkVXV1d+Pj4YNasWWwmIBKAxU5BFAoFXr16JToGEWkQW1tbTJgwAcePH8f58+dRs2ZNTJ8+HeXLl8eYMWNw9OhR5OXl4fDhw+jevftH39/MzAz6+vpIS0srhvRERERERNqjf//+SExMxOHDh0VHIdI6LHYKwmXsRPQ5KlSoAG9vb5w9exYnTpyAnZ0dJkyYADs7O8THx0Mul3/Sfdu1a8dvyIiIiIiIPpNcLsecOXMwc+ZMdncSlTAWOwXhMnYiKipVq1bF9OnTcfHiRaxYsQJ9+vT55Hvp6Oi8tSyeiIiIiIg+nqenJ9LS0rB//37RUYi0CoudgtSuXRs+Pj6iYxBRKWNgYABbW9vPuoehoSFycnKKKBERERERkXaSSqWYN28eZ3cSlTAWOwUpV64c2rVrJzoGEZUyRfFNlJGRER49elQEaYiIiIiItFv37t2hUqmwe/du0VGItManDXWjzyaRSERHIKJSqCj+bUlISEC7du2gr68Pa2trWFtbw8rK6q2PX/9uaWkJXV3dIkhPRERERFS6SCQSzJ8/H1OnTsXXX38NqZQ9Z0TFjcVOIqJSREdHBxkZGTAwMPjke+jp6SErKwvPnz/Ho0ePkJSUhEePHuV/HBsbW+DYkydPYGJi8t6i6JsfW1hYQCaTFeETExERERGpt86dO8PX1xc7duxA7969RcchKvUkKg6OICIqNbKysnDgwAG4u7t/0vUqlQqhoaHw8PAo9DVKpRLJyclvFUX/+XFSUhJSUlJgZmb2zg7Rf35sZmbGd76JiIiIqFQ4dOgQxo4di6tXr0IuZ98ZUXHi3zAiolLkdVemSqX6pCXtZ86cgZOT00ddI5VKYWFhAQsLC9StW/eD5+bm5uLJkycFCqCPHj1CQkIC/vzzzwIF0tTUVFhaWn5wCf3rj8uWLcvxIERERESktlxdXWFjY4OtW7di4MCBouMQlWrs7FRTOTk5kEqlXO5JRB/t3r17+Ouvv9C2bduPui4vLw9BQUHo169f8QT7SNnZ2Xj8+PE7O0T/eSwrKwtWVlb/2i1qZWUFIyMjFkaJiIiIqMRFRUVh4MCBuHHjBmfeExUjFjsFiYiIQLNmzVCmTJn8Y6//U0gkEvzyyy9QKpUYMWKEqIhEpMFOnDgBfX19NGrUqFDnK5VKBAYGomfPnp8171OUV69efbAY+uYxAIXqFrW2toa+vr7gJyu8DRs24OjRo9DX14eLiwv69OnDoi4RERGRmunUqRN69OiBkSNHio5CVGqx2CmIVCrF8ePH0bx583d+ff369diwYQOio6Ohp6dXwumIqDQ4efIkUlNT0aFDhw/OvkxOTkZYWBg8PDxgYmJSggnFePnyZaG6RZOSkqCnp/fBYuibv4t6dz49PR0TJ07EiRMn0LVrVzx69AixsbHo3bs3xo8fDwC4fv065s2bh1OnTkEmk2HAgAGYPXu2kLxERERE2uzMmTPw8PBAbGwsFAqF6DhEpRKLnYIYGhpi+/btaN68OTIyMpCZmYnMzEy8evUKmZmZOH36NKZNm4aUlBSULVtWdFwi0lCPHz9GVFQUJBIJXFxcYGpqmv+1P//8E4cPH8aRI0cQHh7OsRn/oFKp8OLFi0J1iz558gRGRkaF6ha1sLAo0qH0J0+eRMeOHeHv749vvvkGALBu3TrMmjUL8fHxSEpKQrt27eDo6Ahvb2/ExsZiw4YNaNu2LRYsWFBkOYiIiIiocLp27Yr27dtjwoQJoqMQlUosdgpiY2ODpKSk/CWSEokkf0anTCaDoaEhVCoVLl68WKA4QUT0KfLy8nDs2DGkpaXlH6tbty5sbW1RtWpV7N27t9BL3ultSqUSKSkphdqRPjk5Gaampv/aLWptbY1y5cr96470gYGB+M9//oP4+Hjo6upCJpPh7t27cHd3x7hx46Cjo4NZs2bhxo0bMDIyAgBs2rQJc+fOxfnz52FmZlYSf0RERERE9H8uXLiAzp07Iy4uTiNHSBGpO+7GLkheXh6+++47tGvXDnK5HHK5HDo6Ovm/y2QyKJVKGBsbi45KRKWATCaDi4vLO7/m7e0NX19f7Nq1q4RTlR5SqRTm5uYwNzdHnTp1Pnhubm4unj59+laH6MOHD3H+/PkCBdIXL17AwsICly9fRrly5d55P2NjY2RlZSEsLAyenp4AgP379+P69etITU2Fjo4OTE1NYWRkhKysLOjp6aFmzZrIyspCVFQUvv766yL/8yAiIiKi92vYsCFatmyJn376CVOmTBEdh6jUYbFTELlcjsaNG8PNzU10FCLSciNHjsSiRYtw+fJl1KtXT3ScUk8ul+d3bjZo0OCD52ZnZ+PJkycfHGfy5ZdfYsiQIZgwYQI2bdoES0tLJCQkIC8vDxYWFihfvjwSEhKwbds29O3bFy9fvsTq1avx5MkTpKenF/XjEREREVEhzJkzB+3atcOoUaPY5ERUxGRz5syZIzqENkpJSYGTkxPs7Oze+ppKpeIOukRUYnR0dKBUKrFjx478mY+kHmQyGUxMTD64lF0ul6Np06Zo1KgRsrOzYWNjgypVquDFixdo2rQpevTogfT0dEydOhW+vr4IDw/P7/Ds1KkTateunX8vlUqFhw8f4urVq8jJyYGenh50dHRK4lGJiIiItIqlpSUuXryI+Ph4tG7dWnQcolKFMzvV1LNnz5CTkwNzc/N/nddGRPS50tLSULVqVRw7dgw1a9YUHYc+0/z58xEWFob169fnz2J98eIFrl27Bmtra2zatAl//PEHFi9ejFatWuVfp1KpEB4eDj8/v/yl9Do6OoXekV5PT0/UIxMRERFpnNjYWLRo0QK3bt3iXh1ERYjFTkF27tyJqlWr4osvvihwXKlUQiqVIiQkBDExMRg3btw7uz+JiIraggULcPPmTWzevFl0FPoI58+fR15eHho1agSVSoX//e9/GD16NLy9vTFlypT8lQJvvnHWpk0b2NnZYfXq1R/coEilUiE1NbVQO9I/fvwYhoaGhd6Rnh2jnycjIwNHjhyBUqnMXxGiUCjg4uICuZxTioiIiDTF0KFDYWtri/nz54uOQlRqsNgpSOPGjeHu7o73TRE4efIkxo8fj2XLlqFNmzYlG46ItNKLFy9QtWpVnDp1CtWqVRMdhwrp999/x6xZs5CWlgZLS0ukpKTA1dUVfn5+MDQ0xK5duyCTydC0aVNkZGRg2rRpiIqKwu7du9GsWbMiy6FUKvHs2bNC7Uj/9OlTlC1bttA70stksiLLqen++usvnD9/HgYGBmjXrl2BbtoXL17gyJEjyM3NRevWrWFpaSkwKRERERXGnTt34OjoiBs3bsDc3Fx0HKJSgcVOQdq1a4eqVavC29sbL1++xKtXr5CZmYmMjAxkZWXh4cOH+O677xAYGIg+ffqIjktEWsLHxwcJCQnYuHGj6ChUSFlZWbh58yZu3bqFp0+folq1amjfvn3+14ODg+Hj44Pbt2/DwsICjRo1wpQpU4TOhsrLy3vnjvTv+vj58+cwNzd/Z1H0nwVSMzOzUj3z+vjx41AqlXB2dv7geSqVCvv27UPlypVRp06dEkpHREREn2rMmDEwMjLC4sWLRUchKhVY7BTEy8sLW7duha6uLpRKJWQyGeRyOeRyOXR0dGBkZIScnBwEBATA1dVVdFwi0hIpKSlwcHDAn3/+iUqVKomOQ5/oXRvdZWRkIDk5GQYGBihXrpygZB8vJycHT548+eAS+tcfp6enw8rK6oNL6F9/bGJiolGF0VOnTkGhUKBhw4aFvuaPP/6Avb09qlevXozJiIiI6HM9ePAA9evXx9WrV2FtbS06DpHGY7FTkF69eiEjIwNLliyBTCYrUOyUy+WQSqXIy8uDqakpN3wgIiIqhMzMTDx+/LhQM0Zzc3ML1S1qbW0NQ0NDoc+VnJyMM2fOwM3N7aOv3bZtGzw9PTkKgIiISM1NnjwZSqUSK1euFB2FSOOx2CnIgAEDIJVKERAQIDoKERGR1klPT3+rCPq+5fRyubzQO9IrFIoizxoaGoqvv/76kwqWycnJuHTpElxcXIo8FxERERWdpKQk1K5dGxcuXIC9vb3oOEQajdt1CtK3b19kZ2fnf/56yaFKpcr/JZVKNWqJHRERkaYwNDRElSpVUKVKlQ+ep1KpkJaW9s5i6JkzZ97akV5fX79QO9JbWloWakf617utf2pnZrly5ZCSkvJJ1xIREVHJsbKywvDhw7FgwQKsW7dOdBwijcbOTiIiIqIioFKpCr0j/ZMnT1CmTJl/7Ra9e/cumjVr9lk7qx8/fhwODg7cnZ2IiEjNJScno0aNGjh79iwqV64sOg6RxmKxU6C8vDxcv34dcXFxqFSpEho2bIjMzEycO3cOr169Qt26dWFlZSU6JhERERWxvLw8JCcn/+sSeolEgkuXLn3Wa929exfPnz9HgwYNiig9ERERFRcfHx/cu3cP/v7+oqMQaSwuYxdo0aJFmDlzJnR1dWFhYYH58+dDIpFg4sSJkEgk6NatGxYuXMiCJxF9tLZt26Ju3bpYs2YNAKBSpUoYN24cvL2933tNYc4hoqIhk8lgaWkJS0tL1KtX773nhYWFffZr6enpISsr67PvQ0RERMVv8uTJcHBwwM2bN1GjRg3RcYg0klR0AG119OhRbN26FQsXLkRmZiZ+/PFHLF26FBs2bMDPP/+MgIAAXL16FevXrxcdlYjU0JMnTzBmzBhUqlQJenp6sLKygqurKw4ePAjg7w1Nfvjhh4+659mzZzFmzJjiiEtEn0gikUCpVH7WPZ4/f46yZcsWUSIiIiIqTmXLlsXkyZMxd+5c0VGINBY7OwW5f/8+ypQpg++++w4A8M033+D48eO4dOkS+vbtCwC4evUqTpw4ITImEakpDw8PZGRkYOPGjahWrRoeP36Mo0ePIjk5GQBgZmb20fe0sLAo6phE9JmaNm2K6OhotG7d+pPvcePGDXz11VdFmIqIiIiK04QJE1CtWjVcuXIFdevWFR2HSOOws1MQHR0dZGRkFNhdVUdHB+np6fmfZ2VlITc3V0Q8IlJjz58/R1RUFBYuXAhXV1dUrFgRTZo0gbe3N3r37g3g72Xs48aNK3Ddy5cv0b9/f6fUT6oAACAASURBVBgZGcHa2hpLly4t8PVKlSoVOCaRSBASEvLBc4ioeFlZWeHx48effL1KpUJeXh7kcr6/TUREpCmMjIzw/fffw8fHR3QUIo3EYqcg9vb2UKlU2Lp1KwDg1KlTOH36NCQSCX755ReEhIQgIiICbdq0EZyUiNSNkZERjIyMEBYWhszMzEJft3z5ctSqVQvnzp3D3LlzMX36dISGhhZjUiIqCnZ2dkhISPika48fP46WLVsWcSIiIiIqbqNHj8apU6dw7tw50VGINA7f5hekYcOG6Ny5MwYPHoxff/0Vt2/fRqNGjTBs2DD06dMHCoUCTZs2xfDhw0VHJSI1I5fLERAQgOHDh2P9+vVo1KgRWrZsiZ49e8LJyem91zk5OWHGjBkAgOrVq+Ps2bNYvnw5evToUVLRiegTODk54ddff0W/fv2go6NT6OtSUlKQmJiIVq1aFWM6IiIiKg76+vqYPn06Zs+ejb179yIuLg7Xrl2DRCIBABgbG8PZ2bnAalEi+hs7OwUxMDDAvHnzsGPHDtSoUQOTJk3Ctm3b0LFjR1y4cAFbtmzB9u3bYW5uLjoqEakhDw8PPHz4EOHh4XBzc8OJEyfQrFkz+Pn5vfea5s2bv/X5tWvXijsqEX0miUSC3r17Y8uWLYXu5n78+DF+++03fPPNN8WcjoiIiIrLoEGDcP/+ffzyyy9IT09H165d4e7uDnd3dzRo0ABhYWHYtWvXZ428ISqN2NkpkI6ODrp164Zu3boVOG5vbw97e3tBqYhIUygUCnTo0AEdOnTA7NmzMWzYMMyZMwfe3t5Fcn+JRAKVSlXgWE5OTpHcm4g+jkKhQP/+/REaGgpzc3O0bdv2nZ0cmZmZ2LdvH5YvX47g4OD87g8iIiLSLM+fP8fu3bsRGRkJU1PTt75uamqK7t27Q6lU4uDBgyhTpgyaNWsmICmR+mGxUw28Lia8+QOJSqXiDyhE9FFq166N3Nzc93Z+nTp16q3Pa9Wq9d77WVhYIDExMf/zpKSkAp8TUcnS0dGBp6cnUlJSEBYWBpVKBR0dHejp6SEzMxM5OTnQ09ND586dceXKFQwbNgz79+/n9xNEREQa5uXLlwgLC8PAgQP/9f/jUqkUnTp1wrlz53Dy5Mm3VnMRaSMWO9XAu/7x4g8mRPQ+ycnJ6NmzJ4YMGYL69evD2NgYMTExWLx4MVxdXWFiYvLO606dOoUffvgB33zzDSIjI7F58+b8TdLepV27dvjpp5/QokULyGQyTJ8+HQqForgei4gKyczMDN27dwfw95ujWVlZ0NPTK/C9w/Tp09GiRQusW7cOo0ePFhWViIiIPsHu3bvRv3//j6oLfPHFFzh8+DDu37/PlaKk9VjsJCLSMEZGRmjWrBlWrlyJuLg4ZGVloXz58ujbty9mzpz53uu+/fZbXLp0CQsWLIChoSHmzZv3wXl+y5Ytw9ChQ9G2bVtYWVlh8eLFuH79enE8EhF9IolE8s43IXR0dBAYGIhWrVqhffv2cHBwEJCOiIiIPtbt27dRs2ZNSKUfv8WKi4sLdu3axWInaT2J6p8D2YiIiIioVFi1ahW2b9+OqKgoyOV8j5uIiEjdhYSEwMPD45NXe+7Zswdubm7Q1dUt4mREmoO7sQukVCoRGxsrOgYRERGVUuPGjYOhoSEWL14sOgoRERH9C5VKBZlM9llj7VxdXXHkyJEiTEWkeVjsFEipVKJmzZpv7XZMREREVBSkUin8/f2xYsUKnD9/XnQcIiIi+oC0tLR37rz+MYyMjJCdnV1EiYg0E4udAsnlckilUuTm5oqOQkRERKWUvb09li1bBi8vL2RmZoqOQ0RERO+RkZEBAwODz74PG6pI27HYKZhCocCrV69ExyAiIqJSrH///qhZsyZmzZolOgoRERG9h4mJCVJTU0XHINJ4LHYKplAo2GVBRERExUoikWDdunXYunUrjh49KjoOERERvYO+vj5evHjxWfdISEiApaVlESUi0kwsdgqmr6/PYicRaaw2bdogMDBQdAwiKgRzc3M8fPgQbdq0ER2FiIiI3kEikUAmk33WqLvTp0/DycmpCFMRaR4WOwVjZycRabJZs2ZhwYIFyMvLEx2FiIiIiEjjubi4fPJu6jk5OZDL5Z+1mztRacBip2Cc2UlEmszV1RWmpqYICQkRHYWIiIiISOOVKVMGaWlpSElJ+ehrd+3aBVdX12JIRaRZWOwUjMvYiUiTSSQSzJ49G/Pnz4dSqRQdh4iIiIhI43Xv3h179+7Fs2fPCn3N7t270aJFCxgZGRVjMiLNwGKnYFzGTkSa7ssvv4S+vj52794tOgoRERERkcaTSCTw8vLCH3/8gX379n2wqeDOnTsIDAxE06ZNUaFChRJMSaS+5KIDaDsuYyciTSeRSDBz5kzMnTsX3bt354wgIiIiIqLPJJFI4O7ujipVqmDatGkoX7487O3tUbZsWbx69QqJiYlIS0tDxYoV0b9/f34PTvQGdnYKxs5OIioNunbtCqVSiX379omOQqQ2Bg0aBIlE8tavCxcuiI5GREREGmDjxo1o1KgRxo0bh6+//hq2trbIzs6GkZERWrZsCQ8PDzg6OrLQSfQP7OwUjDM7iag0eN3dOW/ePHTp0oXfcBH9n/bt2yMwMLDAMXNzc0FpgOzsbOjq6gp7fSIiIiqcrKws/PDDDwgNDQUASKVS2NrawtbWVnAyIvXHzk7B2NlJRKVFjx49kJ6ejgMHDoiOQqQ29PT0YG1tXeCXXC7Hb7/9hlatWqFs2bIwMzODm5sbbt68WeDaEydOoGHDhlAoFPjiiy+wd+9eSCQSREdHAwBycnIwZMgQVK5cGfr6+qhevTqWLl0KlUqVf4/+/fujW7du8PPzQ/ny5VGxYkUAwK+//gpHR0cYGxvDysoKnp6eSExMzL8uOzsb48aNg42NDfT09GBvb48ZM2aUwJ8YERERAX93ddavXx9NmjQRHYVI47CzUzDO7CSi0kIqleZ3d3bs2JHdnUQfkJ6ejm+//Rb16tVDRkYG5s2bB3d3d1y9ehU6OjpITU2Fu7s7OnfujG3btuH+/fuYNGlSgXvk5eWhQoUK2LFjBywsLHDq1CmMGDECFhYWGDhwYP55f/zxB0xMTHDgwIH8QmhOTg7mz5+PGjVq4MmTJ/j+++/Rt29fHDlyBADw448/Ijw8HDt27ECFChWQkJCA2NjYkvsDIiIi0mJZWVlYuHAhQkJCREch0kgS1Ztv/1OJmzx5MipUqIDJkyeLjkJE9Nny8vJQu3ZtrF27Fu3atRMdh0ioQYMGYcuWLVAoFPnHnJ2dsX///rfOTU1NRdmyZXHixAk0a9YMP/30E3x8fJCQkJB//ebNmzFw4EBERUWhVatW73xNb29vXLlyBb///juAvzs7Dx06hHv37n1w+fqVK1dQr149JCYmwtraGmPGjEFcXBwiIiL4xgUREVEJW7t2Lfbu3ct5+ESfiMvYBeMydiIqTWQyGaZPn4758+eLjkKkFlq3bo0LFy7k//rll18AALGxsejTpw+qVKkCExMT2NraQqVS4d69ewCAGzduoH79+gUKpU5OTm/d/6effoKjoyMsLCxgZGSE1atX59/jtXr16r1V6IyJiUHXrl1RsWJFGBsb59/79bWDBw9GTEwMatSogfHjx2P//v1QKpVF9wdDRERE7/R6VqePj4/oKEQai8VOwbiMnYhKm759++LevXuIiooSHYVIOAMDA1SrVi3/V/ny5QEAXbp0QUpKCjZs2IDTp0/jzz//hFQqRXZ2NgBApVL9a0fl1q1b4e3tjSFDhiAiIgIXLlzAyJEj8+/xmqGhYYHP09LS0KlTJxgbG2PLli04e/YsfvvtNwDIv7ZJkya4c+cOfH19kZOTg/79+8PNzQ1cEERERFS8/P39UbduXTRt2lR0FCKNxZmdgikUCiQnJ4uOQURUZHR0dDBt2jTMnz+fmxURvUNSUhJiY2OxceNGODs7AwDOnDlToHOyVq1aCA4ORlZWFvT09PLPeVN0dDRatGiBMWPG5B+Li4v719e/du0aUlJSsHDhQtjb2wMALl269NZ5JiYm6NWrF3r16gUvLy+0atUKt2/fRpUqVT7+oYmIiOhfZWVlwc/PDzt37hQdhUijsbNTMH19fS5jJ6JSZ8CAAXjw4AGePn0qOgqR2jE3N4eZmRnWr1+PuLg4REZGYuzYsZBK//+3ZV5eXlAqlRgxYgSuX7+OgwcPYuHChQCQ3/FZvXp1xMTEICIiArGxsZgzZw6OHz/+r69fqVIl6OrqYvXq1bh9+zb27t371lK5pUuXIigoCDdu3EBsbCy2b9+OMmXKwNbWtgj/JIiIiOhNr7s63zW6hogKj8VOwbiMnYhKI11dXVy5cgXlypUTHYVI7chkMgQHB+PcuXOoW7cuxo8fjx9++AE6Ojr555iYmCA8PBwXLlxAw4YN8Z///Adz584FgPw5nmPGjEGPHj3g6emJpk2b4sGDB2/t2P4uVlZWCAgIQEhICGrVqgVfX18sX768wDlGRkZYtGgRHB0d4ejomL/p0ZszRImIiKhojRo1Kn+0DBF9Ou7GLtjmzZtx8OBBBAYGio5CREREamzXrl3o1asXnj59ClNTU9FxiIiIiIjUEmd2CsZl7ERERPQu/v7+cHBwgJ2dHS5fvoxvv/0W3bp1Y6GTiIiIiOgDWOwUTKFQsNhJRFpJqVQWmFFIRAU9evQIc+bMwaNHj2BjYwN3d/f8uZ1ERERERPRuXMYu2MGDB7Fo0SIcOnRIdBQiohKhVCoRFhaG7du3o1q1aujatSuHsBMREREREVGRYEuNYOzsJCJtkZOTAwC4cOECvvvuOyiVSkRFRWHo0KFITU0VnI6IiIiISDPl5uZCIpFg9+7dxXoNkaZgsVMwzuwkotIuIyMDU6ZMQf369dG1a1eEhISgRYsW2L59OyIjI2FtbY3p06eLjklEREREVOTc3d3Rvn37d37t+vXrkEgkOHjwYAmnAuRyORITE+Hm5lbir01U3FjsFEyhUODVq1eiYxARFQuVSoU+ffrgxIkT8PX1Rb169RAeHo6cnBzI5XJIpVJMnDgRR48eRXZ2tui4RERERERFatiwYTh8+DDu3Lnz1tc2btyIihUrwtXVteSDAbC2toaenp6Q1yYqTix2CsZl7ERUmt28eRO3bt2Cl5cXPDw8sGDBAixfvhwhISF48OABMjMz8dtvv8Hc3Bzp6emi4xLRv1i+fDmcnZ2Rl5cnOgoREZFG6NKlC6ysrODv71/geE5ODgIDAzFkyBBIpVJ4e3ujevXq0NfXR+XKlTF16lRkZWXln3/37l107doVZmZmMDAwQK1atbBz5853vmZcXBwkEgkuXLiQf+yfy9a5jJ1KMxY7BeMydiIqzYyMjPDq1Su0bt06/5iTkxOqVKmCQYMGoWnTpjh+/Djc3NxgamoqMCkRFcakSZMgk8mwfPly0VGIiIg0glwux8CBAxEQEAClUpl/PDw8HE+fPsXgwYMBACYmJggICMD169exZs0abNmyBQsXLsw/f9SoUcjOzkZkZCSuXr2K5cuXo0yZMiX+PESagMVOwdjZSUSlmZ2dHWrWrIkVK1bkf3MXHh6O9PR0+Pr6YsSIERg4cCAGDRoEAAW+ASQi9SOVShEQEIDFixfj0qVLouMQERFphKFDh+LevXs4dOhQ/rGNGzeiY8eOsLe3BwDMnj0bLVq0QKVKldClSxdMnToV27dvzz//7t27cHZ2Rv369VG5cmW4ubmhY8eOJf4sRJpALjqAtuPMTiIq7ZYsWYJevXrB1dUVjRo1QlRUFLp27QonJyc4OTnln5ednQ1dXV2BSYmoMCpVqoTFixfDy8sLZ86c4awvIiKif+Hg4IDWrVtj06ZN6NixIx4+fIiIiAgEBwfnnxMcHIxVq1YhPj4eL1++RG5uLqTS/9+fNnHiRIwbNw779u2Dq6srevTogUaNGol4HCK1x85OwV53dqpUKtFRiIiKRb169bB69WrUqFED586dQ7169TBnzhwAQHJyMn7//Xf0798fI0eOxM8//4zY2FixgYnoXw0aNAiVKlXK/7tMREREHzZs2DDs3r0bKSkpCAgIgJmZGbp27QoAiI6ORr9+/dC5c2eEh4fj/PnzmDdvXoENPEeOHIm//voLAwcOxI0bN9CsWTP4+vq+87VeF0nfrDPk5OQU49MRqRcWOwWTyWSQy+X8h4eISrX27dtj3bp12Lt3LzZt2gQrKysEBASgTZs2+Oqrr/DgwQOkpKRgzZo16Nu3r+i4RPQvJBIJNmzYgICAABw/flx0HCIiIrX3zTffQKFQYMuWLdi0aRMGDBgAHR0dAMDx48dRsWJFzJgxA02aNIGDg8M7d2+3t7fHyJEjsXPnTsyePRvr169/52tZWloCABITE/OPvblZEVFpx2KnGuBSdiLSBnl5eTAyMsKDBw/QoUMHDB8+HM2bN8f169dx4MABhIaG4vTp08jOzsaiRYtExyWif2FpaYm1a9di4MCBePnypeg4REREak1fXx99+/bFnDlzEB8fj6FDh+Z/rXr16rh37x62b9+O+Ph4rFmzBjt27Chw/fjx4xEREYG//voL58+fR0REBGrXrv3O1zIyMoKjoyMWLlyIa9euITo6Gt9//32xPh+ROmGxUw1wkyIi0gYymQwAsHz5cjx9+hR//PEHNmzYAAcHB0ilUshkMhgbG6NJkya4fPmy4LREVBjdunWDs7MzvL29RUchIiJSe8OGDcOzZ8/QokUL1KpVK/949+7dMXnyZEyYMAENGzZEZGQk5s6dW+DavLw8jB07FrVr10anTp1Qvnx5+Pv7v/e1AgICkJubC0dHR4wZM+a9S96JSiOJisMihatYsSKOHTuGihUrio5CRFSsEhIS0K5dOwwcOBAzZszI33399Vyhly9fombNmpg5cyZGjRolMioRFdKLFy/QoEEDrF27Fm5ubqLjEBEREZGWY2enGmBnJxFpi4yMDGRmZqJfv34A/i5ySqVSZGZmYteuXXBxcYG5uTm6d+8uOCkRFVaZMmXg7++PYcOGITk5WXQcIiIiItJyLHaqAc7sJCJtUb16dZiZmcHPzw93795FdnY2tm3bhgkTJmDJkiUoX7481qxZAysrK9FRiegjuLi4wNPTE6NHjwYXDRERERGRSCx2qgF2dhKRNlm7di2uX7+ORo0aoVy5cli6dClu3bqFTp06YcWKFWjVqpXoiET0CRYsWIArV64gKChIdBQiIiIi0mJy0QHo713ZWOwkIm3RvHlz7N+/HxEREdDT0wMANGzYEHZ2doKTEdHn0NfXR2BgINzc3ODs7My/00REREQkBIudaoDL2IlI2xgZGcHDw0N0DCIqYo0bN8b48eMxZMgQREREQCKRiI5ERERERFqGy9jVAJexExERUWkxbdo0vHjxAj///LPoKERERELl5OSgSpUqiIqKEh2FSKuw2KkGuIydiAhQqVTc2ISoFJDL5di8eTN8fHxw69Yt0XGIiIiE2bJlCypXrgxnZ2fRUYi0CoudaoCdnUREQGhoKJYtWyY6BhEVgRo1amDOnDkYMGAAcnNzRcchIiIqcTk5OfD19YWPj4/oKERah8VONcCZnUREgIODA5YtW8Z/D4lKiTFjxsDExAQLFy4UHYWIiKjEbdmyBZUqVULr1q1FRyHSOix2qgF2dhIRAfXr10ezZs2wYcMG0VGIqAhIpVJs2rQJq1atwrlz50THISIiKjHs6iQSi8VONcCZnUREf5s5cyYWL17MfxOJSgk7Ozv8+OOP8PLy4t9rIiLSGlu3bkXFihXZ1UkkCIudaoDL2ImI/ta4cWM0aNAA/v7+oqMQURHp27cv6tSpgxkzZoiOQkREVOxyc3PZ1UkkGIudaoDL2ImI/r9Zs2Zh4cKFyM7OFh2FiIqARCLB2rVrERQUhMjISNFxiIiIitWWLVtQoUIFtGnTRnQUIq3FYqca4DJ2IqL/r1mzZqhRowY2b94sOgoRFZFy5cphw4YNGDRoEFJTU0XHISIiKhbs6iRSDyx2qgF2dhIRFTRr1iz88MMPyM3NFR2FiIpI586d0alTJ0yaNEl0FCIiomKxdetW2Nvbs6uTSDAWO9UAZ3YSERXk7OyMChUqYNu2baKjEFERWrZsGY4ePYo9e/aIjkJERFSkcnNzMX/+fHZ1EqkBFjvVADs7iYjeNmvWLCxYsAB5eXmioxBRETEyMsLmzZsxatQoPH78WHQcIiKiIrN161bY2dmhbdu2oqMQaT0WO9UAZ3YSEb3NxcUF5ubm2LFjh+goRFSEWrZsiYEDB2LEiBFQqVSi4xAREX2217M658yZIzoKEYHFTrXAZexERG+TSCSYPXs2fH19oVQqRcchoiI0d+5c3L59G7/++qvoKERERJ9t27ZtKF++PLs6idQEi51qgMvYiYjerWPHjjA0NERoaKjoKERUhPT09BAYGIgpU6bg7t27ouMQERF9stezOtnVSaQ+WOxUA1zGTkT0bhKJBLNmzYKvry+XuxKVMvXr14e3tzcGDRrE7m0iItJY27Ztg62tLbs6idQIi51qgJ2dRETv99VXX0EikSA8PFx0FCIqYt7e3sjJycHKlStFRyEiIvponNVJpJ5Y7FQDnNlJRPR+r7s758+fz+5OolJGJpPh119/hZ+fH65duyY6DhER0UfZvn07bGxs2NVJpGZY7FQD7OwkIvqwbt26ITMzE7///rvoKERUxKpWrQo/Pz94eXkhOztbdBwiIqJCeXNWp0QiER2HiN7AYqca4MxOIqIPk0qlmDFjBrs7iUqpYcOGwdraGr6+vqKjEBERFUpQUBCsra3Z1UmkhiQq/tQoXEZGBsqVK8el7EREH5CXl4c6dergp59+gqurq+g4RFTEEhMT0ahRI+zZswdOTk6i4xAREb1Xbm4u6tSpg7Vr16Jdu3ai4xDRP7CzUw0oFApkZWWxW4mI6ANkMhlmzJiBefPmiY5CRMXAxsYGa9asgZeXFzIyMkTHISIieq+goCBYWVnBxcVFdBQiegd2dqoJPT09pKamQk9PT3QUIiK1lZubi5o1a2LTpk1o3bq16DhEVAz69+8PU1NTrF69WnQUIiKit+Tl5aF27dr4+eefudqISE2xs1NNcJMiIqJ/J5fLMX36dMyfP190FCIqJmvWrMGePXtw8OBB0VGIiIjeEhQUBEtLSy5fJ1JjLHaqCYVCwZmdRESF4OXlhdjYWJw8eVJ0FCIqBmXLlsXGjRsxZMgQPHv2THQcIiKifHl5eZg3bx53YCdScyx2qgl2dhIRFY6Ojg6mTp3K7k6iUqxDhw7o1q0bxo0bJzoKERFRPnZ1EmkGFjvVhL6+PoudRESFNHjwYFy+fBkxMTGioxBRMVm0aBFiYmKwY8cO0VGIiIiQl5eH+fPnw8fHh12dRGqOxU41wWXsRESFp6enh++//57dnUSlmIGBAQIDAzF+/HgkJiaKjkNERFouODgY5ubm3JSISAOw2KkmuIydiOjjDBs2DGfPnsXFixdFRyGiYtK0aVOMGjUKQ4cOhUqlEh2HiIi0FGd1EmkWFjvVBJexExF9HH19fXh7e8PX11d0FCIqRjNnzkRSUhI2bNggOgoREWkpdnUSaRYWO9UEOzuJiD7eyJEjcezYMVy9elV0FCIqJjo6OggMDMSMGTMQHx8vOg4REWkZzuok0jwsdqoJzuwkIvp4hoaGmDx5MhYsWCA6ChEVo9q1a2PGjBkYMGAA8vLyRMchIiItsmPHDpiZmaF9+/aioxBRIbHYqSbY2UlE9GnGjh2LQ4cO4ebNm6KjEFExmjBhAvT09LB06VLRUYiISEtwVieRZmKxU01wZicR0acxNjbG+PHj4efnJzoKERUjqVSKgIAALF26lBuTERFRidixYwdMTU3Z1UmkYVjsVBNcxk5E9OnGjx+Pffv24a+//hIdhYiKUYUKFbB06VJ4eXkhKytLdBwiIirFXs/qZFcnkeZhsVNNcBk7EdGnK1u2LMaMGYMffvhBdBQiKmYDBgxA1apVMXv2bNFRiIioFNu5cyfKli2LDh06iI5CRB+JxU41wWXsRESfZ9KkSQgNDcXdu3dFRyGiYiSRSLB+/Xps3rwZ0dHRouMQEVEpxFmdRJqNxU41wc5OIqLPY2ZmhuHDh2PR/2PvzsNjPN+3gZ+TPbKpkqpYs5GV2GltCUVKrW2CihBLKVIUEWQj9lJKayux1f5NbSVtI7GTEImQVVARam+EkG2e94++yU9qS5jMPTM5P8fhODozz/PMOWk7Mtdc933Nny86ChFVsBo1amDVqlUYMmQIcnJyRMchIiINs3PnTpiZmbGrk0hNsdipIrhnJxHRu5s4cSK2bduGrKws0VGIqIJ99tln6NixIyZNmiQ6ChERaRDu1Umk/ljsVBHs7CQienfm5uYYOnQoFi5cKDoKESnBkiVL8Mcff+DAgQOioxARkYbYtWsXTE1N8cknn4iOQkRvicVOFcE9O4mIFOPbb7/Fxo0b8ffff4uOQkQVzNTUFGFhYRg5ciTu3bsnOg4REak5uVzOvTqJNACLnSqCy9iJiBTjww8/xKBBg/Ddd9+JjkJEStChQwcMGDAAX331FSRJEh2HiIjU2K5du2BiYsKuTiI1x2KniuAydiIixZk6dSp+/vln3L17V3QUIlKC2bNnIzk5Gb/88ovoKEREpKbkcjmCg4PZ1UmkAVjsVBFcxk5EpDi1a9fGF198gSVLloiOQkRKYGBggM2bN2PChAnIzMwUHYeIiNRQcVdn165dRUchonfEYqeKYGcnEZFi+fn5YdWqVXjw4IHoKESkBC4uLvD19cXQoUMhl8tFxyEiIjVSvFdnYGAguzqJNACLnSqCe3YSESlW/fr10bt3byxbtkx0FCJSkqlTp+LJkydYsWKF6ChERKRGdu/eDSMjPSuiUAAAIABJREFUI3Tr1k10FCJSAJnEndxVQlxcHIYPH464uDjRUYiINMbly5fRunVrZGRkwMzMTHQcIlKC9PR0tGnTBsePH0ejRo1ExyEiIhUnl8vh7OyMhQsXonv37qLjEJECsLNTBdy9exeJiYnQ1tbG77//jsuXL4uORESkEaytrdG9e3csX74cAJCamoqIiAjs27cPUVFRXOJOpIFsbGwQEhICLy8vFBYWio5DREQqjl2dRJqHnZ2CSJKEmJgYZGVloXr16mjatCmMjIyQl5eH9PR0pKenw8jICK6urtDV1RUdl4hIbV24cAGDBw+Gv78/nJycYGVlBT09PTx+/Bhnz57FgwcPUL9+fTRr1kx0VCJSEEmS0K1bN3z00UcICAgQHYeIiFRUcVfnggUL4O7uLjoOESkIi50CPHnyBLt27YKrqyvq1KnzyuMeP36M/fv3o0WLFrCyslJiQiIizZCSkoLExER8+umnqFKlyiuPu3r1Ko4ePQoPDw8YGBgoMSERVZSsrCy4uLjgt99+Q/PmzUXHISIiFbRr1y4sWLAAZ86c4WAiIg3CYqeS5ebmYseOHRg8eDC0tbXLdE5ERAQsLS1hY2NTwemIiDTHpUuXcOfOHXTq1KlMxxcUFGDz5s0YOHAg9PX1KzgdESnD1q1bERISgri4OBgaGoqOQ0REKkQul6Nx48aYP38+uzqJNAz37FSy//3vf+UqdAJA165dkZCQgCdPnlRgMiIizfHgwQNkZGSUudAJALq6uhg0aBB2795dgcmISJkGDBiAxo0bw9/fX3QUIiJSMf/73/9gaGjIoUREGojFTiVKS0uDs7NzuQqdxT777DNERkZWQCoiIs1z5MgRfPrpp+U+T09PDw0aNMCNGzcqIBURibBixQrs3LkTUVFRoqMQEZGKkMvlCAkJQWBgIJevE2kgFjuVKDExEc7Ozm91rp6eHvLy8sBdB4iIXk8ul0OSpLf6YgkAWrdujdOnTys4FRGJ8v7772PNmjXw9vZGdna26DhERKQCwsPDoa+vz+XrRBqKxU4lycvLe+c94Fq1aoXY2FgFJSIi0kzHjx9H+/bt3/p8mUwGbW1tyOVyBaYiIpG6d+8Od3d3+Pr6io5CRESCyeVyBAcHIygoiF2dRBqKxU4luX379msnr5dF3bp1cfv2bQUlIiLSTNnZ2ahevfo7XaN69ersACPSMAsXLsTx48cRHh4uOgoREQnErk4izcdip5Lk5OTA2Nj4na/DZexERK+niPdJExMT5OTkKCANEakKY2NjbNy4EaNHj+aXx0RElRT36iSqHFjsVBJFfXDmGzIR0esp4n0yJycHpqamCkhDRKqkbdu2GDZsGEaMGMEvkImIKqFff/0Vurq6bzXIkojUB4udSlKzZk1kZma+0zWuXr2KWrVqKSgREZFmeu+99965a+vu3bssdhJpqKCgIFy/fh3r168XHYWIiJSIe3USVR4sdiqJnp4e8vPz3+ka0dHRaNq0qYISERFppo8++ggnTpx46/MlSYIkSdDS4l+RRJpIT08PmzZtwtSpU3H16lXRcYiISEnY1UlUefCTnBI1adIEcXFxb3Xus2fP8NNPP6Fnz56IiYlRcDIiIs0hk8kgk8lQWFj4Vufv2bMHO3bswPXr1xWcjIhUhZOTE6ZMmQJvb28UFRWJjkNERBWMe3USVS4sdiqRlZUVkpKSUFBQUO5z9+zZg0OHDsHd3R39+/dH9+7dcerUqQpISUSk/lxdXbF3795yn/fs2TNkZ2fDxsYGLi4umDJlCh4+fFgBCYlItIkTJ0KSJHz//feioxARUQXbs2cPtLW10aNHD9FRiEgJWOxUsv79+2Pz5s3l6jg6cOAAWrZsiWrVqmHMmDFIT09H7969MWDAAHTp0gXHjx+vwMREROrHzMwMDg4O+P3338t8Tl5eHrZu3YqBAwdi9uzZuHDhAh4+fIiGDRti8eLFyMvLq8DERKRs2traCAsLw7x583Dx4kXRcYiIqIJwr06iyofFTiUzMDCAp6cnfvnlF6Snp7/22AcPHmDLli1wdHREgwYNSu7X19fHqFGjkJaWBk9PT3h5ecHV1RXR0dEVnJ6ISH00bNgQlpaW2Lp1K7Kzs197bHJyMnbs2IFBgwZBV1cXAGBhYYE1a9YgOjoa0dHRaNSoEbZs2QK5XK6M+ESkBJaWlpg7dy4GDx78znurExGRatq7dy+7OokqGZkkSZLoEJVVQkICMjIyYGpqCmdnZ5iZmeHJkye4fPkyMjMzUa1aNbRv3x7a2tqvvU5BQQG2bNmC0NBQ1KpVCwEBAXB1deW3VkREAAoLCxEdHY3s7GzUr18flpaWMDQ0RHZ2Ns6fP48nT57Azs4O9vb2r73OkSNHMHnyZBQWFmLBggXo3Lmzkl4BEVUkSZLw2WefoXHjxpg9e7boOEREpECSJKFp06YIDg7GZ599JjoOESkJi50qIDs7GykpKcjOzoaRkRHq1auH2rVrl/s6hYWF2LZtG2bPno33338fgYGB6NKlC4ueRET/3/Xr13H9+nXk5ubiq6++wq+//gpnZ+cyny9JEnbt2oVp06bB2toa8+fPR+PGjSswMREpw99//40mTZogPDwcbdq0ER2HiIgU5Ndff0VISAjOnTvHz8VElQiLnRqoqKgIO3bswKxZs2BqaoqAgAB0796db+5ERM/p3Lkzvv32W3Tr1q3c5+bn52PVqlUIDQ1F165dMWvWLNStW7cCUhKRsuzevRt+fn6Ij4+HkZGR6DhERPSOirs6g4KC0KtXL9FxiEiJuGenBtLW1saAAQOQmJiIiRMnYurUqWjZsiX27dsH1raJiP5la2v7xr2TX0VPTw/jxo1DWloa6tSpAxcXF0ydOhX//POPglMSkbL069cPbdq0wZQpU0RHISIiBdi7dy8AcPk6USXEYqcG09bWxhdffIGEhAT4+flhxowZaNasGcLDwzlgg4gqPRsbm7cudhYzNTUtmdz+4MED2NracnI7kRpbtmwZ9u3bh4iICNFRiIjoHUiShKCgIE5gJ6qkWOysBLS0tNCvXz+cP38egYGBmD17NlxcXLBr1y4WPYmo0lJEsbNY8eT2qKgoREVFcXI7kZqqWrUq1q9fDx8fHzx48EB0HCIiekvs6iSq3LhnZyUkSRIOHDiAkJAQ5ObmYubMmejfv/8bp74TEWmS1NRUfPrpp7h8+bLCr/385PaFCxfCzc1N4c9BRBXH19cXd+7cwdatW0VHISKicpIkCc2aNUNAQAB69+4tOg4RCcBiZyUmSRIiIiIQHByM7OxszJgxAx4eHix6ElGlkJ+fD1NTU+Tk5EBXV1fh139+cruNjQ3mz59frsnvRCTO06dP0bRpUwQGBsLT01N0HCIiKoe9e/ciMDAQcXFxXMJOVElxGXslJpPJ0K1bN5w8eRJLly7Fjz/+CHt7e2zcuBGFhYWi4xERVSg9PT1YWFjg6tWrFXJ9mUyGzz//HElJSXB3d0eXLl3g7e2N69evV8jzEZHiGBoaYuPGjfD19cXNmzdFxyEiojIq3qszMDCQhU6iSozFToJMJkOXLl1w7Ngx/PTTT1i3bh0aNWqE9evXo6CgQHQ8IqIKY2Njg7S0tAp9juLJ7enp6ahduzYntxOpiRYtWmD06NEYNmwYuBCKiEg97Nu3D5IkoVevXqKjEJFAXMZOZZKfnw89PT3RMYiINIa5uTn8/Pzw9ddfQ19fX3QcInqJgoICtG3bFj4+Pvjqq69ExyEioteQJAnNmzfHjBkz0KdPH9FxiEggdnZSmdjY2GDlypXIy8sTHYWISCM8P7n9l19+4eR2IhWkq6uLTZs2YebMmUhPTxcdh4iIXmP//v0oKipiVycRsdhJZbN9+3bs3bsX1tbWWL58OZ49eyY6EhGRWnNwcMC+ffsQFhaG77//Hi1atEBkZKToWET0H40aNcLMmTMxZMgQ7mlORKSiJEnCnDlzEBgYCC0tljmIKjsuY6dyiY2NxaxZs3Du3DlMmTIFI0eOhKGhoehYRERqTZIk7Ny5E9OmTYOtrS0ntxOpGLlcji5duqBz586YNm2a6DhERPQfkiRBLpdDJpOx2ElE7Oyk8mnRogX27t2Lffv2ITo6GlZWVli8eDGePHkiOhoRkdqSyWT44osvkJycXGpye2ZmpuhoRARAS0sL69evx5IlSxAfHy86DhER/YdMJoO2tjYLnUQEgMXOcpHJZNi1a9c7XSMsLAzGxsYKSiRO06ZNER4ejt9++w0nT56ElZUVFixYgMePH4uORkQarH79+li0aFGFP4+o9+r/Tm5v0qQJJ7cTqYi6deviu+++w+DBg7mdDxEREZEKY7ET/xYxX/fH29sbAHDr1i307NnznZ7Lw8MDV65cUUBq1dCkSRPs2rULf/75J+Li4mBlZYW5c+fi0aNHoqMRkZrx9vYued/V0dFB3bp1MXr0aDx8+LDkmNjYWIwZM6bCs4h+rzY1NcXs2bNx4cIF3L9/H7a2tliyZAmHxBEJ9uWXX8LW1hYzZ84UHYWIiIiIXoF7dgL4+++/S/55//79GDFiBG7dulVyn6GhIczMzEREqxD5+fnQ09OrkGsnJSUhNDQUv//+O3x9fTFu3DiN+tkRUcXx9vZGVlYWNm3ahMLCQiQlJWHYsGFo164dtm7dKjqeUJcuXYKfnx8uXryI0NBQeHp6cpkWkSB3795F48aNsW3bNrRv3150HCIiIiL6D35SAlCzZs2SP1WrVn3hvuJi3fPL2K9duwaZTIZt27ahQ4cOMDQ0hIuLCy5cuICLFy+ibdu2MDIywscff4yrV6+WPNd/l0ZmZmaiV69eqFatGqpUqYJGjRph27ZtJY8nJiaic+fOMDQ0RLVq1eDt7Y3s7OySx2NjY/HJJ5+gevXqMDU1xccff4xTp06Ven0ymQwrVqxA3759YWRkBH9/fxQVFcHHxwcNGjSAoaEhbGxssGDBAsjl8nf6Wdrb22PLli04fvw40tPTYW1tjeDg4FKdWUREr6Kvr4+aNWuidu3a+OSTT+Dh4YHff/+95PH/LmOXyWT46aef0KtXL1SpUgW2traIiorCjRs30LVrVxgZGaFJkyaIi4srOaf4fTgyMhKOjo4wMjJCp06dXvteDQAHDhxAq1atYGhoiPfffx89e/YsWcr6suX1HTt2xNixYxXyc+HkdiLVUaNGDaxatQre3t7IyckRHYeIqNJhvxYRvQmLne8oMDAQU6dOxfnz51G1alUMHDgQ48aNQ2hoKGJiYvDs2TOMHz/+leePGTMGubm5iIqKwqVLl/D999+XFFxzc3PRrVs3GBsbIyYmBuHh4Th58iSGDRtWcn5OTg4GDx6MY8eOISYmBk2aNIG7uzvu3btX6nmCg4Ph7u6OxMREfP3115DL5bCwsMCOHTuQnJyM0NBQzJkzB+vXr1fIz6Vhw4bYsGEDTp06hb/++gs2NjaYOXMm7t+/r5DrE5Hmu3LlCg4dOgRdXd3XHjd79mx4enoiISEBzZs3x4ABA+Dj44MxY8bg/PnzqFWrVsl2JMXy8vIwd+5crFu3DqdOncI///yDr7766pXPcejQIfTq1QtdunTBuXPnEBUVhQ4dOrzzF0Tl1aFDB5w5cwZTp07FyJEj0b17d1y4cEGpGYgI6NmzJ1xdXTFhwgTRUYiIKoXnC5wymQwAlP57GBGpEYlK2blzp/SqHwsAaefOnZIkSdLVq1clANLKlStLHt+3b58EQNq9e3fJfevXr5eMjIxeedvJyUkKCgp66fOtXr1aMjU1lR49elRyX1RUlARASk9Pf+k5crlcqlmzprRp06ZSuceOHfu6ly1JkiRNnTpVcnNze+NxbyMjI0MaPny4VK1aNWnatGnS3bt3K+R5iEh9DRkyRNLW1paMjIwkAwMDCYAEQFq8eHHJMfXq1ZMWLlxYchuA5OfnV3I7MTFRAiB99913JfcVv28Wv++sX79eAiClpKSUHLN582ZJV1dXKioqKjnm+ffqtm3bSh4eHq/M/t9ckiRJHTp0kL7++uvy/hjKLC8vT1q2bJlkbm4ueXt7S9evX6+w5yKiFz169Ehq0KCBtHfvXtFRiIg03rNnz6Tjx49LI0aMkGbOnCnl5uaKjkREKoydne/I2dm55J8/+OADAICTk1Op+548eYLc3NyXnu/r64vZs2ejTZs2mDFjBs6dO1fyWHJyMpydnWFiYlJyX9u2baGlpYWkpCQAwJ07dzBq1CjY2trCzMwMJiYmuHPnDq5fv17qeZo3b/7Cc69cuRLNmzdHjRo1YGxsjCVLlrxwnqJYWlpizZo1iIuLw4MHD2Bra4spU6bgzp07FfJ8RKSe2rdvj/j4eMTExGDcuHFwd3d/bXc8ULb3YQCl3m/09fXRsGHDktu1atVCQUHBK6eenz9/Hm5ubuV/QRWoeHJ7WloaatWqhSZNmsDPz4+T24mUxMTEBBs2bMCoUaNw9+5d0XGIiDRaaGgoRo8ejQsXLmDLli1o2LBhqc/ORETPY7HzHT2/vLK4nf5l972qxd7HxwdXr17F0KFDkZaWhrZt2yIoKAjAv636xef/V/H9Q4YMQWxsLJYsWYKTJ08iPj4etWvXRn5+fqnjjYyMSt3evn07vvnmG3h7eyMiIgLx8fEYM2bMC+cpWr169bBy5UokJCQgNzcXjRo1wqRJk0oNiSKiyqtKlSqwtraGk5MTli1bhtzcXMyaNeu157zN+7COjk6pa7zrcigtLa0X9o8qKCh4q2uVl5mZGUJDQ3HhwgXcu3ePk9uJlKhdu3b48ssvMWrUKO4hR0RUQW7duoXFixdjyZIliIiIwMmTJ1GnTp2SAZaFhYUAuJcnEf0fFjtVQO3atTFy5Ejs2LEDISEhWL16NYB/h/0kJCSU2vz+5MmTkMvlsLOzAwAcP34c48aNw6effgoHBweYmJiUmiT/KsePH0erVq0wduxYNG3aFNbW1sjIyKiYF/gSderUwfLly5GYmIjCwkLY29vjm2++wc2bN5WWgYhUX2BgIObPny/8vcHFxeW1A4Fq1KhR6r332bNnSElJUUa0EhYWFli7di2ioqJw+PBhNGrUCL/88gv3syKqYCEhIUhPT8fmzZtFRyEi0khLliyBm5sb3NzcYGZmhg8++ACTJ0/Grl27kJOTU/Il9qpVq7iXOREBYLFTOF9fXxw6dAhXrlxBfHw8Dh06BHt7ewDAoEGDYGRkBC8vLyQmJuLo0aMYNWoU+vbtC2trawCAra0tNm/ejKSkJMTGxsLT0xN6enpvfF5bW1vExcXh4MGDSE9Px6xZs3DkyJEKfa0vY2FhgaVLl+LSpUvQ1taGo6Mjxo4dixs3big9CxGpno4dO8LBwQGzZ88WmmP69OnYuXMnZsyYgaSkJFy6dAlLliwp2aLE1dUVW7ZsQXR0NC5duoRhw4YprbPzv4ont69fv75kcvvhw4eFZCGqDAwMDLBp0yZMmjSpwrYDIiKqrPLz85GVlQUbGxsUFRUBAIqKiuDq6gp9fX2Eh4cDANLT0zFmzJhSW8ARUeXFYqdgcrkc48aNg729Pbp06YIPPvgAGzZsAPDvcs6IiAg8evQILVu2RK9evdCmTRusW7eu5Px169bh8ePHaNasGTw9PTFs2DDUr1//jc87atQofPHFFxg4cCBatGiBa9euYdKkSRX1Mt/oww8/xHfffYeUlBRUqVIFzs7OGD16NP766y9hmYhINUycOBE///yz0PcDd3d3hIeH4+DBg3BxcUGHDh0QFRUFLa1//xqdNm0aXF1d0atXL3zyySf4+OOP0bRpU2F5gX8LxcWT20eMGMHJ7UQVqEmTJpgwYQKGDh3KbmoiIgXS09ODp6cnrK2toa2tDQDQ1taGqakpPvroI+zbtw8A4O/vj88++wwNGjQQGZeIVIRM4sYWpILu3r2LxYsXY/Xq1ejbty/8/f3L9BdXUVERkpKSULduXZiZmSkhKRGR6svPz8eqVaswe/ZsuLu7IyQkBHXq1BEdi0ijFBYWon379vDw8ICvr6/oOEREGqN4tYyurm6puRZRUVEYNWoUdu7ciWbNmiE1NRVWVlYioxKRimBnJ6mkGjVqYO7cuUhLS0PNmjXRvHlzDBs2DA8fPnzteUlJSVi4cCHatWuHESNGvPF4IqLKgJPbiSqejo4ONm7ciFmzZiE5OVl0HCIitVf8e4quru4Lhc78/Hy0adMG1apVQ8uWLdG3b18WOomoBIudpNLef/99zJo1C5cvX0bdunVhbGz82uNr164NT09PfP311/j555+xZMkSPHv2TElpiYhUGye3E1Usa2trzJ49G15eXsL27SUi0gQPHjzA6NGjsXHjRly7dg0ASgqdwL9f5BoYGMDBwQEFBQVYuHChoKREpIpY7CS18N577yEoKKhk0t7rjnN3d8eDBw9gZWWFbt26wcDAoORxfvAgIvq/ye2HDx9GZGQk7OzsOLmdSEFGjRqF6tWrIzQ0VHQUIiK1tX79emzfvh3ff/89Jk+ejC1btiAzMxPAv1PXi4cVzZ07F3v37kW9evVExiUiFcM9O0ljPL+s4cMPP8TgwYMREBBQ0g16/fp17Ny5E7m5uRg8eHCZBjkREVUG0dHRmDJlCoqKirBw4UK4urqKjkSk1m7evAkXFxfs378fLVq0EB2HiEjtnDx5Er6+vvDy8sKePXuQkpICNzc3aGtrY/fu3bhx4wYnrxPRK7GzkzRG8bd7CxcuhLa2Nvr06VNq2fuDBw9w584dnDp1CpaWlli8eDG7mIiI8OLkdnd3dyQmJoqORaS2atWqhWXLlmHw4MHIzc0VHYeISO20bdsWrVu3xtOnT/Hnn39i6dKluH79OjZv3gxLS0scPHgQGRkZomMSkYpisZM0RvES9++//x4eHh5wdHQs9XiTJk0QGhqKoKAgAICpqamyIxKRClu3bh28vLxExxBGJpPhiy++QHJyMrp164bOnTtj6NChJUvGiKh8PDw80LRpU0ybNk10FCIitTRx4kQcOnQImZmZ6NevH7y9vWFiYoIqVapgwoQJmDRpEr9QIqKXYrGTNEJxh+aSJUsgSRL69u37wrKGoqIi6OjoYM2aNXB2dkavXr2gpVX6f4GnT58qLTMRqRZbW1ukp6eLjiGcnp4exo8fz8ntRAqwfPly7N69G5GRkaKjEBGplaKiIjRo0AAffvghAgMDAQDTpk3DnDlzcOLECSxevBitW7dGlSpVBCclIlXEPTtJrUmShMjISBgZGaFNmzaoV68e+vTpg1mzZsHExKTUPp7Av/t2WltbY+XKlRg2bFjJNWQyGa5evYqff/4Z+fn58PLyeqEzlIg02+3bt+Hg4IB79+6JjqJSsrKyEBgYiL1792LatGkYM2YM9PX1RcciUhsREREYMWIELly4gKpVq4qOQ0Sk8p7/DJeamoqJEyeiVq1a2L9/PxISEmBubi44IRGpOnZ2klorLnZ+9NFHsLKywqNHj9CvX7+Srs7ivySLOz9DQ0Nha2uLHj16lFyj+JgHDx5AJpMhOTkZzs7OnKJKVMmYm5sjPz8fDx8+FB1FpbxscvvWrVu55zFRGXXt2hU9e/bE+PHjRUchIlJpxavsnv8M17BhQ7Ru3RphYWHw9/cvKXTy9xAieh0WO0mtaWlpYe7cuUhLS0PHjh2RnZ2NadOm4fz586X+AtTS0kJWVhbCwsLg6+v70m8DmzVrhoCAAPj6+gIAHBwclPY6iEg8mUwGGxsbLmV/BUdHR+zfvx/r1q3D4sWL0bJlSxw+fFh0LCK1sGDBApw+fRq7d+8WHYWISCVlZ2cjODgY0dHRyM7OBoCSLcd8fHywdu3akr3VJUl6YTsyIqLncRk7aZRr165hypQpMDIywpo1a/DkyRNUqVIFurq6GDNmDKKiohAVFYWaNWuWOu/5pRJffvklUlNTERsbK+IlEJFAnp6e6NmzJwYNGiQ6ikqTy+XYuXMn/P390bBhQ8yfPx9OTk6iYxGptNOnT6N3796Ij49/4fcQIqLKbvTo0Vi1ahXq1q2Lnj174osvvoCzszPMzMxKHZeXl8ftdIjojfh1CGmU+vXrY8eOHfjpp5+gra2N0NBQdOrUCdu3b8emTZswceLEl37AKC50njt3Djt27IC/v7+yoxORCrCxsUFaWproGCpPS0sLHh4enNxOVA6tW7fG8OHDMWLECLDXgIjo/+Tk5OD06dNYuXIlJk2ahD179uDzzz/HjBkzcOTIkZIthi5evIiRI0fiyZMnghMTkapjsZM0koGBAWQyGb799lvUqFEDX375JZ48eQJDQ0MUFRW99By5XI6lS5fCwcEBffr0UXJiIlIFXMZePi+b3D5t2jRObid6hYCAANy7dw+3b98WHYWISGVkZmaiadOmqFmzJsaNG4fr169j5syZ2Lt3L7744gsEBATg6NGj8PX1xcOHD2FkZCQ6MhGpOC5jp0rh/v37mD59OlavXo2xY8ciJCTkhYmo8fHxaNWqFbZs2YL+/fsLSkpEIp0+fRrjxo3jNhZv6caNGwgMDMS+ffvg7++P0aNHc6kZ0X/I5XLIZLKSVSVERJWdXC5Heno6Pvjggxc+o61YsQKLFi3CP//8g+zsbKSmpsLGxkZQUiJSFyx2UqVy7949xMTEoGvXrtDW1sbNmzdhbm4OHR0dDB06FOfOnUNCQgI/gBBVUvfv34eVlRUePnzI94F3cPHiRfj5+SEpKQmhoaHw8PDgIAEiIiIqs8LCQujo6JTcLp7KvmHDBoGpiEhdsNhJlVZ2djYmT56Ms2fPYtCgQQgKCsL69evZ1UlUyVWrVg2pqamoUaOG6ChqLzo6GpMnT4YkSViwYAFcXV1FRyJSefn5+Vi6dCksLS3Rr18/0XGIiISSy+WIjY1FmzZtkJycjIYNG4qORERqgG0WVGmZmZlh8eLFaNq0KQICAvDkyRMUFBSBTD5bAAAgAElEQVTg6dOnrzxHkiTI5XIlpiQiZeO+nYrTsWNHnDlzBpMnT8aIESPg7u6OxMTEMp3L72KpssrMzER6ejpmzpyJAwcOiI5DRCSUlpYWHj9+jKlTp7LQSURlxmInVWrGxsZYu3Yt7t27h8mTJ2PQoEGYNm0aHj9+/MKxkiThzJkzcHJywtatW1856IiI1BuLnYr1ssntw4YNe+Mk1YKCAjx8+BAxMTFKSkokniRJsLKywtKlS+Ht7Y0RI0YgLy9PdCwiogonSdIrv+h0dXVFaGiokhMRkTpjsZMIgKGhIebPn4/c3FwMGjQIhoaGLxwjk8nQqlUrLF68GD/88AMcHBywefNmFBYWCkhMRBXFxsYGaWlpomNonOcnt1taWr70ffZ5Y8aMQbt27TBq1CjUr18f69evV1JSIuWTJKnU7xMGBgaYPHkyLC0t8dNPPwlMRkSkHFFRUfjtt99eWvCUyWTc+5uIyoXvGETPMTAwQIsWLaCtrf3Sx2UyGbp27YoTJ05gxYoVWL16Nezt7bFhwwYWPYk0BDs7K5aZmRlmzJjx2gFQP/74I7Zu3YoxY8Zgx44dCAgIQGhoKA4ePAiAS9xJM8jlcty8eRNFRUWQyWTQ0dEp+f+ieFp7bm4uTExMBCclIqpYkiQhICAA//zzDwdEEpFC6Lz5ECL6L5lMBjc3N7i5uSE6OhohISEICQmBv78/vLy8oKurKzoiEb0lW1tbFjuV4HUfZlauXInhw4djzJgxAP4tQJ89exZr1qxBt27dIJPJkJqayr27SG0VFBSgXr16uH37Ntq1awcjIyM0b94cLi4usLCwQLVq1bBp0ybEx8fDwsJCdFwiogp1+PBh3L17F56enqKjEJGGYGcn0Tvq2LEjDh8+jLCwMGzbtg22trZYvXo18vPzRUcjordgY2ODy5cvs3tQkPz8fFhZWZXs6Vn870GSpJLOt8TERNjZ2aFHjx7IzMwUGZforejq6mLixImQJAnjxo2Do6Mjjh49ilmzZqFHjx5o2bIl1q5dix9++AHdunUTHZeIqMJIkoSgoCAEBAS8cnUdEVF5sdhJpCDt2rXDH3/8gS1btiA8PBzW1tb48ccfOViASM2YmZnB0NAQf//9t+golZKenh46dOiAXbt2Yffu3ZDJZDhw4ABOnDgBMzMzFBUVwcnJCRkZGTA1NUW9evXg4+ODp0+fio5OVC7ffvstHB0dERkZifnz5+Pw4cM4d+4cUlNT8eeffyIjIwOjRo0qOT4rKwtZWVkCExMRKd7hw4dx584ddnUSkUKx2EmkYG3btsXBgwexc+dO/Pbbb7CyssIPP/yAZ8+eiY5GRGXEfTvFKO7i/OabbzBv3jyMGjUKrVq1gq+vLy5evAhXV1doa2ujsLAQDRo0wC+//IKzZ88iPT0dVatWxaZNmwS/AqLy2bt3L37++Wfs2bMHMpkMRUVFqFq1KlxcXKCvrw8dnX93nLp37x42bNgAPz8/FjyJSGMUd3XOnDmTXZ1EpFAsdhJVkFatWmH//v3Ys2cP/vzzT1hZWeH7779Hbm6u6GhE9AYsdipfYWEhIiMjcevWLQDAV199hXv37mH06NFwdHREmzZtMGDAAAAoKXgCwIcffgg3NzcUFBQgMTGR3fSkVurXr485c+bA29sbjx8/fuWH/erVq6NFixbIzc2Fh4eHklMSEVWMqKgodnUSUYVgsZOogjVr1gx79uzB/v37cezYMVhZWWHRokUl+9ERkephsVP57t+/j61btyIkJASPHj1CdnY2ioqKEB4ejszMTEydOhXAv3t6Fk+ufvDgAfr27Yt169Zh3bp1WLBgAfT19QW/EqLymTRpEiZMmICUlJSXPl5UVAQA6Ny5M4yNjXHy5ElERkYqMyIRkcI939VZ3MVORKQoLHYSKYmLiwt2796NiIgIxMTEwNLSEvPnz0dOTo7oaET0HzY2NkhLSxMdo1L54IMPMHr0aJw4cQL29vbo3bs3atWqhStXriAgIACfffYZAJR8INqzZw+6d++O+/fvY9WqVfD29haYnujdzJgxA82bNy91X/G2Dtra2oiPj0fTpk0RERGBlStXwsXFRURMIiKFiYqKwu3bt9nVSUQVQiZx3CyREJcuXUJoaCj+/PNPfPPNNxg7dixMTU1FxyIiAOfPn4eXlxcSExNFR6mUDhw4gIyMDNjZ2aFZs2aoVq1ayWP5+fmIiIiAj48PnJycsGrVKlhbWwP4tzgkk8lExSZ6Z+np6TAzM4O5uXnJffPnz8fMmTPh5uaGuXPnwtnZGVpa7FcgIvUlSRI6duyI4cOHY/DgwaLjEJEGYrGTSLCUlBSEhobi0KFDGD9+PMaNG4eqVauKjkVUqT1+/Bjm5uZ4/PgxiwqCyeXyUv8OZsyYgVWrVqFHjx4ICgpCvXr1XjiGSF0tW7YMO3bswPHjx3Ht2jV4eXkhLi4OgYGB8PHxKVX453/3RKSuoqKiMGrUKCQlJXEJOxFVCBY7iVREeno6QkNDsX//fnz99dfw9fUt9aGGiJSrVq1aOHPmDOrUqSM6CgHIzMzEhAkTEBERgZEjR+K7774THYlI4QoLC1G1alW0adMGsbGxcHR0xIIFC9CqVatXDi96+vQpDA0NlZyUiOjtsKuTiJSBXwcTqQgbGxuEhYXhzJkzyMrKgq2tLWbMmIH79++LjkZUKXFIkWoxNzdHzZo1sXbtWsybNw/A/w1u+S9Jkl75GJEq09HRwb59+xAZGYmePXvi119/Rdu2bV9a6Hz8+DF++uknLF26VEBSIqK3Ex0djZs3b2LAgAGioxCRBmOxk0jFWFlZYe3atYiNjcXdu3dha2sLPz8/3L17V3Q0okqFxU7Voq+vj+XLl8PDwwO6uroA8MpONwDo2LEjli5diry8PGVFJFKITp06YeTIkTh27Nhrl3caGxtDX18f+/btw/jx45WYkIjo7QUHB3MCOxFVOBY7iVRUgwYNsGrVKpw/fx6PHj1Cw4YNMXnyZNy+fVt0NKJKgcVO9SWTyfDjjz/i999/h52dHbZt2wa5XC46FlGZrVy5EhYWFoiOjn7tcQMGDEDPnj2xfPnyNx5LRCRadHQ0srKyMHDgQNFRiEjDsdhJpOLq1q2LH3/8ERcuXEBeXh7s7OwwYcIE3Lp1S3Q0Io1mY2ODtLQ00THoLTk5OeHAgQP4+eefsWjRIrRq1QpRUVGiYxGVWfES9lfJzs7G0qVLERoaii5dusDKykqJ6YiIyi8oKIhdnUSkFCx2EqmJ2rVrY9myZbh06RIAwMHBAePHj0dWVpbgZESaiZ2dmqFTp06IiYnBpEmT4OPjg08//RQXL14UHYvojWrUqAFzc3Pk5ubi2bNnpR5LSEhA7969ERISgtmzZyMiIoLD1IhIpbGrk4iUicVOIjXz4YcfYsmSJUhKSoKenh6cnJzw9ddf4/r166KjEWkUa2trXLt2jYNuNICWlhY8PT2RnJyMTz75BG5ubhg2bBhu3LghOhrRG23atAmzZ8+GJEl49uwZli9fjvbt2yMvLw8xMTHw9fUVHZGI6I2Cg4MxY8YMdnUSkVKw2EmkpmrWrIlFixYhJSUFJiYmcHFxwahRo3Dt2jXR0Yg0gqGhIWrUqMEvEjSIvr4+fH19kZaWhpo1a6Jx48bw9/dHdna26GhEr9SpUyfMmTMHixYtwqBBgzBhwgRMnDgRx44dg6Ojo+h4RERvFB0djczMTAwaNEh0FCKqJFjsJFJz5ubmmDdvHlJTU1G9enU0a9YMw4cPx5UrV0RHI1J7XMqumczMzDBnzhwkJCTg77//hq2tLZYuXYr8/HzR0YheYGtri0WLFmHq1KlISkrC8ePHERgYCG1tbdHRiIjKhBPYiUjZWOwk0hDVq1dHaGgo0tPTYWFhgZYtW2Lo0KEs1BC9AxY7NVvt2rWxbt06/PnnnyWT27dv387J7aRyJk6ciM6dO6Nu3bpo1aqV6DhERGV25MgRdnUSkdKx2EmkYapVq4bg4GBcvnwZDRo0QNu2beHl5YXU1FTR0YjUDoudlUPx5Pa1a9di4cKFnNxOKmn9+vWIjIzEgQMHREchIioz7tVJRCKw2EmkoapWrYqAgABkZGSgUaNGaNeuHQYOHIikpCTR0YjUho2NDdLS0kTHICXh5HZSZRYWFjh16hTq1asnOgoRUZkcOXIE169fx5dffik6ChFVMix2Emk4U1NT+Pv7IyMjA40bN0anTp3g4eGBxMRE0dGIVB47Oyuf5ye3d+nSBa6urvDx8eHkdlIJLVq0eOlQIkmSBKQhInq94OBgTJ8+nV2dRKR0LHYSVRImJiaYOnUqMjIy0KJFC3Tp0gX9+vVDfHy86GhEKsvS0hKZmZkoKCgQHYWUTF9fH9988w3S0tJgbm7Oye2ksiRJwpEjR/DXX3+JjkJEVOLo0aP466+/2NVJREKw2ElUyRgbG+Pbb7/FlStX8PHHH8Pd3R29e/fGuXPnREcjUjn6+vqoVasWrl27JjoKCVK1alXMnTuXk9tJZclkMpw5cwbe3t4crkVEKqN4r05dXV3RUYioEpJJXPdCVKk9ffoUa9euxfz58+Hi4oKZM2eiZcuW5bpGYmIiMjIyoK2tXbKUTltbG25ubjAwMKiI2ERK07VrV/j6+sLd3V10FFIBiYmJ8PPzQ0pKCubMmYPPP/8cWlr87pjEKioqQocOHdC/f3988803ouMQUSV39OhRDB06FCkpKSx2EpEQLHYSEQDg2bNnWLduHebNmwcHBwcEBASgTZs2rz0nMjIS//zzDxwdHdGwYcNSjz19+hSHDx/G06dP0b59e5ibm1dkfKIKM3bsWNjY2MDX11d0FFIhhw8fxpQpUyCTybBw4UJ07NhRdCSq5DIyMtC6dWscOXIE9vb2ouMQUSXm5uaGQYMGYdiwYaKjEFElxWInEZWSl5eHDRs2YM6cObC1tUVAQAA+/vjjUsfI5XJs3boVbm5uqFmz5muvJ0kS9uzZAwcHB9jY2FRkdKIKsXTpUqSnp2P58uWio5CKkcvl2L59O6ZPnw57e3vMmzfvpcNjiJRl9erVWLVqFU6fPs1uKiIS4tixYxgyZAhSU1P5PkREwnDdFRGVoq+vj5EjRyItLQ0eHh7w8vKCq6srjhw5UnLMtm3b8Nlnn72x0An8u5dY7969kZaWxmnGpJY4kZ1eRUtLCwMGDEBycjI6d+4MNzc3Tm4noUaMGIGaNWti1qxZoqMQUSXFvTqJSBWw2ElEL6WnpwcfHx+kpqbCy8sLw4cPR4cOHbBixQq0a9cOJiYm5brep59+imPHjlVQWqKKY2Njg7S0NNExSIUVT25PTU3l5HYSSiaTYe3atVi1ahXOnDkjOg4RVTLHjx/HlStXMHjwYNFRiKiSY7GTiF5LV1cX3t7eSE5OxogRI5CYmIg6deq81bUcHByQmpqq4IREFat+/fq4efMm8vLyREchFVc8uT0+Pr5kcvuyZcs4uZ2U6sMPP8Ty5cvh5eWF3Nxc0XGIqBIJDg7G9OnT2dVJRMKx2ElEZaKjo4OPP/74nTYad3Z2RmJiogJTEVU8XV1d1KtXD1euXBEdhdREnTp1sG7dOvzxxx84dOgQ7OzssH37dnCbdFKWzz//HC1atMDUqVNFRyGiSuL48eO4fPkyvLy8REchImKxk4jKLj4+Hi1atHina+jo6CgoDZHycN9OehvOzs747bffsGbNGixcuBCtWrVCdHS06FhUSfzwww/49ddf8ccff4iOQkSVAPfqJCJVwmInEZWZtrY2ZDLZO11DR0cHcrlcQYmIlIPFTnoXrq6uiImJwYQJEzBs2DD06NEDFy9eFB2LNNx7772HdevWwcfHBw8fPhQdh4g02IkTJ9jVSUQqhcVOIiozRSzB1NLSYrGT1A6LnfSu/ju53dXVFT4+PsjKyhIdjTRYly5d0KtXL4wbN050FCLSYNyrk4hUDYudRKRUBQUFXMpOaofFTlKU4sntaWlpMDc3h7OzM6ZPn87J7VRh5s+fj9jYWOzcuVN0FCLSQCdOnEB6ejq7OolIpbDYSURlVrt27Xce0lJQUKCgNETKY2Njg7S0NNExSIM8P7n91q1bnNxOFaZKlSrYtGkTxo0bh1u3bomOQ0QaprirU09PT3QUIqISLHYSUZk1bdoUcXFxb31+VlYWLCwsFJiISDnq1q2Lu3fvIjc3V3QU0jCc3E7K0LJlS4wcORLDhw/nf1tEpDAnT55EWloauzqJSOWw2ElE5WJgYPDWBZ9Tp06hdevWCk5EVPG0tbVhaWmJjIwM0VFIQz0/uX3BggWc3E4KN3PmTPz9999Ys2aN6ChEpCHY1UlEqorFTiIql65du2L79u3lHjIUGxuLBg0avPM0dyJRuG8nKYOrqytiY2MxYcIEDB06FD169MClS5dExyINoKuri02bNsHf359f3BDROzt58iRSU1MxZMgQ0VGIiF7AYicRlYuuri769euHjRs3lnn/zZiYGOTl5aFZs2YVnI6o4rDYScpSPLk9JSUFnTt3RqdOnTi5nRTC3t4e06dPx5AhQ1BUVCQ6DhGpMXZ1EpEqY7GTiMrN1NQUAwYMQHh4OA4ePPjKgRrJycnYtWsX9PT08PHHHys5JZFisdhJyvb85PYaNWpwcjsphK+vL3R1dbFo0SLRUYhITZ06dYpdnUSk0mQSdyknonfw+PFjHD58GEVFRdDW1sbVq1dhZmYGY2NjNGrUCI6OjqIjEinE4cOHERwcjCNHjoiOQpVUZmYmAgIC8Ntvv2H69On46quv2FFDb+Wvv/5C8+bNERkZCWdnZ9FxiEjNdOvWDX379sXIkSNFRyEieikWO4lIoQYMGICePXti4MCBoqMQKVRmZiZatmyJW7duiY5CldyFCxfg5+eH1NRUzJ07F59//jn3Q6ZyCwsLw+LFixEbGwt9fX3RcYhITZw6dQqenp5IT0/nF25EpLK4jJ2IFOq9997Dw4cPRccgUjgLCwtkZ2cjJydHdBSq5J6f3D5//nxObqe3MmTIEFhZWSEwMFB0FCJSI8HBwfD392ehk4hUGoudRKRQLHaSptLS0oK1tTUuX74sOgoRAE5up3cjk8mwatUqbNiwAcePHxcdh4jUwOnTp5GcnIyhQ4eKjkJE9FosdhKRQrHYSZqMQ4pI1Tw/ud3NzQ2dOnXC8OHDObmdysTc3BwrV67EkCFD2LVORG/Erk4iUhcsdhKRQrHYSZqMxU5SVfr6+pgwYQLS0tJQvXp1Tm6nMuvVqxc6dOiAb7/9VnQUIlJhp0+fRlJSErs6iUgtsNhJRArFYidpMhY7SdVVrVoV8+bNQ3x8PG7evAlbW1ssW7YM+fn5oqORCvv+++/x+++/48CBA6KjEJGKCg4OxrRp09jVSURqgcVOIlIoFjtJk7HYSeqiTp06WL9+Pf744w8cOnQIdnZ22LFjByRJEh2NVJCpqSnCwsIwcuRI3Lt3T3QcIlIxZ86cwaVLl9jVSURqg8VOIlIoFjtJk7HYSeqmeHL76tWrSya3HzlyRHQsUkEdOnSAp6cnRo8ezaI4EZVSvFenvr6+6ChERGUik/jbDBERUZlIkgRTU1NkZmaiatWqouMQlYtcLsf27dvh7+8PR0dHzJs3Dw4ODqJjkQp59uwZmjVrBn9/fwwaNEh0HCJSATExMejfvz/S09NZ7CQitcHOTiIiojKSyWTs7iS19fzkdldXV05upxcYGBhg06ZNmDBhAm7cuCE6DhGpgOK9OlnoJCJ1wmInERFRObDYSeqOk9vpdZo2bYrx48dj6NChkMvlouMQkUAxMTFITEzEsGHDREchIioXFjuJiIjKgcVO0hQvm9z+ww8/cHI7wc/PDzk5Ofjxxx9FRyEigdjVSUTqisVOIiKicmCxkzTN85PbDx48CHt7e05ur+R0dHSwceNGBAUFITU1VXQcIhIgJiYGFy5cYFcnEaklDigiIpUSFBSEXbt24eLFi6KjEL3UyZMnMWHCBJw5c0Z0FKIKERkZiSlTpkBHRwcLFixAhw4dynxuXFwcrl+/Di2tf79Pl8vlaNSoERo1alRRcakCrVixAhs3bsSJEyego6MjOg4RKVGPHj3g7u6OMWPGiI5CRFRuLHYSUQlvb2/cu3cP+/fvF5bh8ePHyMvLw/vvvy8sA9Hr3L17F7a2tnjw4AFkMpnoOEQVQi6XY9u2bZg+ffobJ7cXFhbi0KFDyMvLg4uLCywtLUs9fvHiRaSkpMDU1BRdunTh/zdqRJIkdO3aFe3atcPMmTNFxyEiJYmNjUXfvn1x+fJlLmEnIrXEZexEpFKMjY1Z6CSVVr16dUiShPv374uOQlRhtLS0MHDgwDdObn/8+DE2bdoEV1dX9OvX74VCJwA4Ojqif//+aNasGTZu3IiCggJlvQx6RzKZDOvXr8cPP/yAc+fOiY5DRErCvTqJSN2x2ElEZSKTybBr165S99WvXx+LFi0quZ2WloYOHTrAwMAADRs2xG+//QZjY2OEhYWVHJOYmIjOnTvD0NAQ1apVg7e3d6kJwEFBQXB0dKzw10P0tmQyGfftpErjZZPbZ8yYgUePHiE/Px87d+7EkCFDUKVKlTde6/3334eHhwd++eUX7geqRiwsLLB06VIMHjwYT58+FR2HiCpYbGwsEhIS4OPjIzoKEdFbY7GTiBRCLpejT58+0NHRwenTpxEWFobg4GDk5eWVHJObm4tu3brB2NgYMTExCA8Px8mTJ7nxOakdW1tbFjupUime3H7+/HncuHEDtra2CA4OxsCBA0v25ywLAwMD9OrVCwcPHqzAtKRonp6ecHJywvTp00VHIaIKFhISAj8/P3Z1EpFa407jRKQQf/zxB1JTU/H777/DwsICALBkyRJ89NFHJcds2bKlZMmjiYkJAGD16tXo1KkTLl++DGtrayHZicqLnZ1UWdWtWxdhYWE4e/YsYmNj3+rDcNWqVfH06VNIksT9O9WETCbDjz/+CGdnZ/Ts2ROdOnUSHYmIKsDZs2dx/vx57Ny5U3QUIqJ3ws5OIlKIlJQU1KpVq6TQCQAtWrQo1fGTnJwMZ2fnkkInALRt2xZaWlpISkpSal6id8FiJ1V2d+/exZAhQ976/NatW+PMmTMKTEQV7f3338fatWtf2H6GiDRH8V6dBgYGoqMQEb0TFjuJqExkMtkLe6w9P2SiLB06rzuG3T2kTljspMouLy+vTPt0voqFhQX+/vtvBSYiZejevTu6d+8OX19f0VGISMHOnTuH8+fPc69OItIILHYSUZnUqFEDt27dKrl9+/btUrft7OyQlZWFmzdvltx39uxZyOXyktv29vZISEhATk5OyX0nT56EXC6HnZ1dBb8CIsUpLnZyyApVVjo6774Tkra2tgKSkLItWrQIx48fR3h4uOgoRKRAwcHB8PPzY1cnEWkEFjuJqJRHjx4hPj6+1J9r167B1dUVK1asKNnLx9vbu9QvQ126dEHDhg0xZMgQJCQk4PTp05g4cSJ0dHRKujYHDRoEIyMjeHl5ITExEUePHsWoUaPQt29f7tdJauW9996Dnp4ebt++LToKkRCKKPTzywL1ZGxsjA0bNmDMmDG4c+eO6DhEpADnzp1DXFwchg8fLjoKEZFCsNhJRKUcO3YMLi4upf58++23+O6772BpaYmOHTuif//+GD58OMzNzUvO09LSQnh4OPLy8tCyZUsMGTIE06dPh0wmKymKVqlSBREREXj06BFatmyJXr16oU2bNli3bp2ol0v01riUnYgqq48++gje3t4YMWIEi9ZEGiA4OBhTp05lVycRaQxOYyeiEmFhYQgLC3vl4wcPHix1u1+/fqVu29ra4ujRoyW3ExISUFBQUKpr08nJCZGRka98jry8PBgbG5czOZHy2draIj09He3atRMdhUjp8vLy3mmaekFBAYtkai44OBgtW7ZEWFgYhg4dKjoOEb2luLg4nDt3Djt27BAdhYhIYVjsJCKFCQ8Ph5GREWxsbHDt2jVMnDgRjRs3RtOmTd94riRJuHLlCiIjI+Hs7KyEtETvhp2dVJk1b94c586dQ/Pmzd/q/D/++AOurq4KTkXKpKenh02bNsHV1RWdOnVC/fr1RUciorfAvTqJSBNxGTsRKUxOTg7Gjh0Le3t7DBo0CHZ2doiIiChT5092djbs7e2hp6eHmTNnKiEt0bthsZMqs/r16+PatWtvff6aNWuwceNGFBYWKi4UKZ2TkxOmTJmCIUOGlBpISETqIS4uDmfPnsWIESNERyEiUiiZxDVERERE5RYXF4ehQ4ciISFBdBQiIVJSUnDnzh20b9++XOft27cPRkZGmD17Nu7evYulS5eyy1ONFRUVoWPHjujTpw8mTpwoOg4RlUOvXr3g5uaG8ePHi45CRKRQLHYSERG9hZycHNSsWROPHz9+630LidRdbGwsHjx4gK5du5bp+EOHDqFevXqws7ODJEn49ddfMWnSJDRp0gSLFi2CpaVlBSeminDlyhW0atUK0dHRcHBwEB2HiMrg/Pnz6NGjBy5fvgxDQ0PRcYiIFIrL2ImIiN6CiYkJTExMcPPmTdFRiISpWrUqRo4ciZ9//hkZGRmvPC4xMRHbtm2DnZ0d7OzsAAAymQx9+vRBUlISmjdvjpYtW2L69Ol4/PixsuKTglhaWmLu3LkYPHgw8vPzRcchojIonsDOQicRaSJ2dhJRhfDw8ECfPn3g6ekpOgpRhWnXrh1CQkLQqVMn0VGIlO7Zs2do06YNhg8fjq+//hrnz59HRkYGdHR0oK2tDUmSIJfLUVhYCCcnJzRs2PC118vKysK0adNw+PBhzJ07F4MG/T/27jssqmt9G/AzQy82MEKiiKggorGXoEiJvYVERQREQewNlWLDaFQ02BCNorGAYsVekRg02LCggAIiKIIlGktQpEnb3x/+5DscTY5lZvYAz31dc504uz3jwZ3uekcAACAASURBVGHm3Wu9ywVSKe/LVxSCIOC7775Dy5YtsXDhQrHjENG/4KhOIqrsWOwkIrkYO3YsWrZsiXHjxokdhUhuPDw80LFjR4wePVrsKEQKN2nSJPz555/Yu3fvO60c3n68/JQWDzExMfD09ISKigqCgoLQoUMHmeQl+Xv8+DFatWqFgwcP4ptvvhE7DhH9gx9++AG2trbw9PQUOwoRkVzwdjkRyUWtWrWQlZUldgwiueKK7FRVHThwAEePHsWmTZveW9CUSCSf3MvW0tISFy9exNixY/H999/Dzc0Njx49+tzIpACGhoZYs2YNhg0bhtzcXLHjENF7xMXF4dKlS7xRS0SVGoudRCQXLHZSVcBiJ1VFGRkZGDNmDHbt2oWaNWvK5RpSqRTDhw/HrVu3YGhoiK+//hoBAQF4/fq1XK5HsjNw4EB07NgRvr6+YkchoveYP38+e3USUaXHaexEJBefM4WRqKK4fv06nJyckJSUJHYUIoUoKipCly5dMGjQIHh7eyvsurdv34a3tzcSExOxfPlyfPfdd/z9osRevHiBFi1aYMOGDejZs6fYcYjo/8THx6NPnz64c+cOi51EVKmx2ElERPSJ8vLyoK+vj9zcXC6kQlWCr68vkpKScOTIEVF+5k+ePIkpU6agbt26CAwMRLNmzRSegT5MVFQU3NzckJCQAD09PbHjEBGAAQMGwNraGlOmTBE7ChGRXPGbGRER0SfS1taGvr4+7t+/L3YUIrmLiIjAzp07sWXLFtGK+927d0d8fDz69+8POzs7TJ48GX///bcoWejfde3aFQMGDMDEiRPFjkJEeDOq8+LFixgzZozYUYiI5I7FTiIios9gamqK1NRUsWMQydXDhw/h7u6O7du3o3bt2qJmUVNTw6RJk5CcnIzi4mI0bdoUwcHBKC4uFjUXvWvx4sW4du0adu/eLXYUoipv/vz58PX15fR1IqoSWOwkIiL6DFykiCq74uJiODs7Y8KECbC2thY7TpnatWtj7dq1OHnyJMLDw9GmTRucPn1a7Fj0H7S1tREWFobJkyfjzz//FDsOUZWVkJCAmJgYjuokoiqDPTuJiIg+w7Jly/Dw4UMEBgaKHYWoyhIEAQcOHICXlxfatGmDZcuWwcTEROxY9H/mzZuHS5cu4fjx41xYikgEAwcOhJWVFaZOnSp2FCIiheDITiISRUFBAVauXCl2DKLPxpGdROKTSCQYMGAAkpOT0aZNG7Rv3x5+fn7IyckROxoBmD17Np49e4b169eLHYWoyklISMCFCxc4qpOIqhQWO4lIIf57EHlRURGmTZuGV69eiZSISDZY7CRSHlpaWpg9ezYSEhKQkZEBc3NzbNu27Z3fQaRYampq2Lp1K/z8/HD79m2x4xBVKW97dWpra4sdhYhIYTiNnYjkYv/+/WjWrBkMDAxQs2bNsudLSkoAvCl+VqtWDWlpaahXr55YMYk+W0FBAWrWrImcnByoqqqKHYeI/sOFCxfg6ekJNTU1BAUFoX379mJHqtKCgoKwe/dunD17FioqKmLHIar0rl+/jp49e+LOnTssdhJRlcKRnUQkF7Nnz0br1q0xbNgwBAcH49y5c8jKyoKKigpUVFSgqqoKDQ0NPH/+XOyoRJ9FU1MThoaGyMzMFDsKEf2XTp064dKlSxg9ejTs7e3h7u6Ox48fix2rypo0aRK0tLSwZMkSsaMQVQnz58+Hj48PC51EVOWw2ElEchEdHY3Vq1cjLy8Pc+fOhaurK4YMGQI/Pz8cP34cAKCnp4cnT56InJTo85mamiI1NVXsGERyk5GRAYlEgtjY2Ap3balUCjc3N6SkpKBOnTpo3rw5lixZgtevX8s4Kf0vUqkUISEhWLFiBeLj48WOQ1SpXb9+HefPn8fYsWPFjkJEpHAsdhKRXNSpUwceHh74/fffkZCQAF9fX9SoUQOHDh3CqFGjYGVlhYyMDOTn54sdleizsW8nVQZubm6QSCSQSCRQU1NDw4YN4e3tjdzcXBgZGeHRo0do1aoVAOCPP/6ARCLBs2fPZJrB1tYWEydOLPfcf1/7U1WvXh0BAQGIiYnB+fPn0axZMxw+fJj9PBWsfv36WL58OVxdXVFQUCB2HKJKa/78+fD29uaoTiKqkljsJCK5Ki4uxpdffolx48YhPDwc+/btg7+/P9q2bYu6deuiuLhY7IhEn83MzIzFTqoUunXrhkePHiE9PR0LFy7E2rVr4e3tDRUVFRgaGorSl1bW1zY1NcWhQ4ewZs0azJgxA7169UJycrJMzk0fxtXVFWZmZvjxxx/FjkJUKd24cQPnzp3jqE4iqrJY7CQiufrvL6dmZmZwc3NDUFAQoqKiYGtrK04wIhniyE6qLDQ0NGBoaAgjIyM4OzvDxcUFBw8eLDeVPCMjA3Z2dgCAL774AhKJBG5ubgDeLD63ZMkSNGrUCFpaWvj666+xbdu2cteYP38+jI2Ny641bNgwAG9GlkZHR2PNmjVlI0wzMjLkNoW+Z8+eSEhIQN++fWFjYwNPT09kZWXJ9Br0fhKJBOvWrcO2bdtw9uxZseMQVTpve3Xq6OiIHYWISBRcNpaI5OrZs2e4ceMGkpKScO/ePbx69QpqamqwsbHBwIEDAbz5ciyRSEROSvTpWOykykpLSwtFRUXlnjMyMsK+ffswcOBAJCUlQU9PD1paWgAAPz8/7N27F2vWrEGTJk0QExODUaNGoVatWujbty/27duHZcuWYefOnfj666/x5MkTXLx4EcCblbpTU1Nhbm6ORYsWAXhTTL1//77cXp+amhomT54MJycn/PjjjzA3N8dPP/2EUaNGcbVwOfviiy+wfv16DB8+HAkJCahWrZrYkYgqhRs3buDs2bMIDQ0VOwoRkWhY7CQiublx4wbmzp2LmJgYaGhooE6dOtDU1ERpaSmOHj2K8PBwrFy5El9++aXYUYk+i4mJCR4+fIjCwkKoq6uLHYdIJi5fvowdO3aga9eu5Z5XUVGBnp4egDf9mWvXrg0AyM3NxYoVK/Dbb7+hS5cuAN7827h8+TLWrFmDvn37IjMzE19++SV69OgBNTU11K9fH+3atQMA1KhRA+rq6tDW1oahoaECX+mbwltwcDDGjh0LT09PBAcHIygoiLMP5Kx///44dOgQpk2bhg0bNogdh6hSeNurk6M6iagq4zR2IpKLhw8fwsvLC7dv38aWLVtw8eJFREdH48SJE9i/fz/8/f1x//59rFy5UuyoRJ9NTU0N9erVw927d8WOQvRZTpw4AV1dXWhqasLS0hLW1tZYvXr1Bx2bnJyMgoIC9OrVC7q6umWP4OBg3LlzBwDg4OCAgoICmJiYwMPDA3v27FGqVdFbtmyJ06dPY86cOXBzc4ODgwMyMjLEjlWprVixAlFRUThy5IjYUYgqvMTERJw9exbjxo0TOwoRkahY7CQiubh58ybu3LmDyMhI9OjRA4aGhtDS0oK2tjbq1KkDJycnDB06FL/99pvYUYlkglPZqTKwtrZGfHw8bt26hYKCAuzfvx916tT5oGNLS0sBAEeOHEF8fHzZIykpqey93sjICLdu3cL69etRvXp1eHl5oW3btsjNzZXba/pYEokEgwYNws2bN9GyZUu0a9cOc+bMUaqMlUn16tURGhqKMWPG4OnTp2LHIarQOKqTiOgNFjuJSC50dHSQk5MDbW3tf9zn9u3b7NFFlYapqSlSU1PFjkH0WbS1tdG4cWMYGxtDTU3tH/d7266hpKSk7DkLCwtoaGggMzMTjRs3LvcwNjYu209TUxN9+/ZFYGAgrly5gqSkJJw/f77svP95TjFpaWnBz88P8fHxSE9Ph7m5OXbs2AFBEMSOVulYW1vDxcUFY8eO5d8v0SdKTEzEmTNnOKqTiAjs2UlEcmJiYgJjY2N4enpi+vTpUFFRgVQqRV5eHu7fv4+9e/fiyJEjCAsLEzsqkUyYmZkhKSlJ7BhECmFsbAyJRIJjx46hf//+0NLSQrVq1eDt7Q1vb28IggBra2vk5OTg4sWLkEqlGD16NEJDQ1FcXIyOHTtCV1cXu3fvhpqaGkxNTQEADRo0wOXLl5GRkQFdXd2y3qBiqlevHrZv347z58/D09MTa9asQVBQUFmvUZKNBQsWoH379ti2bRtcXV3FjkNU4SxYsABeXl4c1UlEBBY7iUhODA0NERgYCBcXF0RHR6NRo0YoLi5GQUEBCgsLoauri8DAQPTs2VPsqEQyYWpqioMHD4odg0gh6tati59++gmzZ8/GyJEjMWzYMISGhmLBggUwMDDAsmXLMG7cOFSvXh2tWrWCr68vAKBmzZoICAiAt7c3ioqKYGFhgf3798PExAQA4O3tjeHDh8PCwgL5+flK1Qe3c+fOuHz5MkJDQ9G/f3/07t0bixYtUvhiSpWVpqYmwsLC0L17d9ja2sLIyEjsSEQVRmJiIqKjo7F582axoxARKQWJwLkiRCRHhYWF2LNnD5KSklBcXIyaNWuiYcOGaNOmDczMzMSORyQz6enpsLOzQ2ZmpthRiEjOsrOzsXDhQmzevBnTp0/H5MmToaGhIXasSmHRokWIiorCyZMnIZWy4xbRh3B0dES7du3g4+MjdhQiIqXAYicREZEMFBcXQ1dXFy9evICmpqbYcYje69atW2jSpInYMSqNtLQ0TJs2DSkpKVixYgX69esHiUQidqwKrbi4GNbW1hgyZAgmT54sdhwipZeUlIRvv/0W6enpnMJORPR/WOwkIrl7+zbz9n8lEgm/DFKlZG5ujgMHDqBp06ZiRyF6R0FBAb755hvEx8eLHaXSOXHiBKZOnQpjY2MEBgbyPeAzpaWlwdLSEufOnYO5ubnYcYiU2pAhQ9CmTZuydiFERMTV2IlIAd4WN6VSKaRSKQudVGklJyfzizkpLS8vL7YPkZNevXrh+vXr6N27N6ytrTFlyhRkZWWJHavCMjU1xYIFC+Dq6oqioiKx4xApraSkJJw+fRrjx48XOwoRkVJhsZOIiEhGWMwnZbV3715ERERgw4YNYkeptNTU1ODp6Ynk5GQUFBSgadOmWL9+PUpKSsSOViGNHTsW+vr6WLRokdhRiJTW2xXYdXV1xY5CRKRUOI2diOTqP6euExGR4t29excdO3bEsWPH0L59e7HjVBnx8fHw9PTEy5cvERQUBBsbG7EjVTh//vknWrdujaNHj/Jnl+i/JCcnw87ODnfu3GGxk4jov3BkJxHJ1ZYtW3D8+HGxYxARVUmFhYUYMmQIZs6cyWKRgrVq1Qp//PEHZs+ejeHDh2Pw4MHIzMwUO1aF8tVXX2HVqlVwdXVFfn6+2HGIlMqCBQswbdo0FjqJiN6DxU4ikqvk5GQkJiaKHYOIqEqaNWsW6tSpgylTpogdpUqSSCRwcHDAzZs38fXXX6Nt27b48ccfkZubK3a0CsPR0RGtW7fGzJkzxY5CpDSSk5Nx6tQpTJgwQewoRERKicVOIpKrWrVqcZEGov9TUFCAvLw8sWNQFXH06FGEh4cjNDSUrUREpqWlhTlz5iAuLg63b99G06ZNsXPnTrCb1IdZs2YN9u7di6ioKLGjECkFjuokIvp37NlJRHK1bt06xMXFYf369WJHIRLd2rVr8ezZM8yePRsqKipix6FK7MGDB2jbti327dsHKysrsePQfzl37hw8PT2hpaWFoKAgtG3bVuxISi8yMhKjRo3C9evXUbNmTbHjEMmVIAiIiYnBkydPIJX+//FJqqqqqFu3Lnr06MFenVRlxMXFITMzEyoqKuVuEnbt2hU6OjoiJiNlpip2ACKq3Diyk6qSTZs2wcrKCqampigtLYVEIilX1DQyMkJwcDCcnJxgamoqYlKqzIqLi+Hs7AxPT08WOpWUlZUVLl++jNDQUPTr1w99+/aFv78/DAwMxI6mtHr27Il+/fph8uTJ2Lp1q9hxiOSitLQUx44dQ2FhISwtLdGpU6dy23Nzc7F161a4ubmhuLhYpJRE8icIAk6ePIns7Gy0bt0a33//fbntr1+/xqlTp5CTkwMrKyt8+eWXIiUlZcVp7EQkVyx2UlUyY8YMnD59GlKpFKqqqmWFzlevXiE5ORn37t1DUlISEhISRE5KldlPP/0EDQ0NzJgxQ+wo9C9UVFTg4eGBlJQU1KpVC82aNcOyZctQWFgodjSltXTpUsTExGDfvn1iRyGSuYKCAmzZsgW2trYYOHAgvvrqq3f20dHRwbhx4/Dzzz/jt99+w71790RISiRfJSUl2L59O1q1aoVBgwahUaNG7+yjoaGB3r17w8HBAVevXsXNmzdFSErKjNPYiUiurly5gnHjxiE2NlbsKERyZ29vj5ycHNjZ2eH69etIS0vDn3/+iZycHEilUtSpUwfa2tr4+eef0bdvX7HjUiX0+++/Y9iwYbh27RoMDQ3FjkMfITU1FdOmTUNqaioCAwPRp08f9lp9j5iYGPzwww+Ij4/nzzhVGqWlpdiyZQuGDh0KNTW1Dz5u7969sLOzg76+vhzTESnW9u3bYW9v/1FtGiIjI2Fubg5jY2M5JqOKhCM7iUiuOLKTqpJOnTrh9OnTOHToEPLz82FlZQVfX1+EhITgyJEjOHToEA4dOgRra2uxo1Il9Ndff2H48OHYunUri0AVkJmZGY4ePYqgoCB4eXmhT58+SElJETuW0rG0tISHhwdGjRrFBZ6o0oiIiMCgQYM+qtAJAAMHDsTJkyfllKpqevXqFaZMmQJjY2NoaWmhU6dOuHLlStn2nJwcTJo0CfXq1YOWlhaaNGmCwMBAERNXLtHR0bCzs/vofrQ9e/bEhQsX5JSKKiL27CQiuWKxk6qS+vXro1atWtixYwf09PSgoaEBLS0tLkZEcldaWoqhQ4dixIgR6Natm9hx6DP07t0b3bp1wy+//IIuXbpg6NChmDt37gctylNcXAxV1cr/8X7u3Lno2LEjNm/eDA8PD7HjEH0WQRCQn5+PatWqffSxEokEX331FZ48eYI6derIIV3VM3LkSFy/fh1btmxBvXr1sG3bNnTr1g3JycmoW7cupk2bht9//x1hYWEwMTHBmTNnMGrUKNSuXRuurq5ix6/wnj59Chsbm086tmXLlkhKSkKzZs1knIoqIo7sJCK5qlmzJrKzs1FaWip2FCK5a968OTQ1NfHVV19BX18furq6ZYVOQRDKHkSy9vPPP+P169eYO3eu2FFIBtTU1DB16lQkJSUhLy8P5ubmiIyM/Nf3D0EQcOLECYwfPx67du1SYFrFU1dXR1hYGGbMmIH09HSx4xB9ltjYWLRv3/6Tj7eyssK5c+dkmKjqys/Px759+/Dzzz/D1tYWjRs3xrx589C4cWMEBwcDAC5cuABXV1fY2dmhQYMGGDZsGL755htcunRJ5PQVX0ZGBho0aPDJx1tYWLB3J5VhsZOI5EpFRQU6OjrIzs4WOwqR3DVt2hSzZs1CSUkJcnJysHfvXiQlJQF4M/ri7YNIls6dO4dVq1Zhx44dVWJUX1VSp04drF+/HhEREf+z/UVxcTGys7OhoqKCMWPGwNbWFs+ePVNQUsVr3rw5ZsyYATc3N5SUlIgdh+iTPXz48LP6DEqlUkil/FovC8XFxSgpKYGmpma557W0tMoKylZWVjhy5Aju378P4E3xMz4+Hr169VJ43somISEBbdu2/axz8HMQvcV3RSKSO05lp6pCVVUVEyZMQPXq1ZGfn48FCxbAysoK48aNw40bN8r240hnkpXnz5/D2dkZmzZtQr169cSOQ3LSunVraGpq/uvNEjU1NTg7O2P16tVo0KAB1NXV8fLlSwWmVLwpU6ZAIpGwXx5VaLJodcN2ObJRrVo1WFpaYuHChXj48CFKSkqwbds2xMTE4NGjRwCAVatWoVWrVqhfvz7U1NRgY2ODgIAA9OvXT+T0FZ9UKv3sQQFqamq8AUYAWOwkIgVgsZOqkreFTF1dXWRlZWHJkiUwMzPDgAEDMH36dFy8eJEjMEgmBEGAm5sbHBwc0LdvX7HjkJz9ry+AhYWFAN6sYpuZmYnJkyejUaNGACrvDRYVFRWEhoYiICCg3A0loopEFu1tEhMTy80g4ePfH//2nhgWFgapVIp69epBQ0MDq1atgpOTU1lBefXq1Th//jwOHz6Mq1evIjAwEN7e3jhx4sQ75yotLYWXl5for7eiPFavXv3Z/xZUVFRY7CQALHYSkQKw2ElVydsP0RoaGjAyMsKzZ88wdepUnD9/HiUlJfjll1+waNEipKamih2VKriVK1fir7/+wuLFi8WOQiITBAHq6uoAgBkzZsDJyQmWlpZl2wsLC5GWlobt27cjMjJSrJhyYWJigoCAALi6upYVfIkqElkUOy0sLMr1Bufj3x//dtO5UaNGiI6ORk5ODu7fv4/Lly+jqKgIJiYmyM/Px8yZM7FkyRL0798fLVq0wMSJEzFkyBAsW7bsnXNJpVIsX75c9NdbUR4TJkz47H8Lr1+/Lvt9SFUbi51EJHcsdlJVIpFIyvpntW3bFomJiQCAkpISjBkzBnXq1IGfnx8WLFggclKqyK5cuYLFixdj9+7d/FBPZaNYZsyYARUVFQwbNgz6+vpl26dOnYpvv/0WixcvxvDhw9G5c+eyfnOVgbu7O+rXr4+ffvpJ7ChEH6169eqf3V+3uLhYRmnoLR0dHXz55ZfIyspCZGQk7O3tUVRUhKKionfaBqioqFTaEfSKZGJi8tmDAYqKimSUhio6dm8lIrljsZOqkuzsbOzbtw+PHj3C+fPnkZqaiqZNmyI7OxuCIMDAwAB2dnaoU6eO2FGpgnr58iUcHR2xdu1amJiYiB2HRFZaWgpVVVXcu3cPa9aswaxZs9CyZcuy7YsWLUJYWBhWrlyJfv36QU1NDd9//z3CwsIwa9YsEZPLjkQiwYYNG9CyZUv07dsXnTp1EjsS0Qd5+fIlLl68iLNnz+LHH3/8pHPExcWhVatWMk5WdUVGRqK0tBTm5ua4ffs2fHx80KRJE7i7u5f16JwxYwZ0dXVhbGyM6OhobN26FUuWLBE7eoXXokUL7Nu3D2ZmZp90/IMHD1C3bl0Zp6KKisVOIpI7FjupKsnKysKMGTNgZmYGdXV1lJaWYtSoUahevToMDAxQu3Zt1KhRA1988YXYUakCEgQBI0eORK9evTBo0CCx45DIbty4AQ0NDZiZmcHT0xPNmjXD999/D21tbQDApUuXsHDhQixevBgjR44sO+7bb7/F1q1b4ePjAzU1NbHiy5SBgQGCg4MxbNgwxMfHQ1dXV+xIRP/o0aNHWLlyJTZu3IjevXujc+fOKCkp+aSFhm7fvg0HBwc5pKyaXr58iZkzZ+LBgwfQ09PDwIED4e/vX/ZeuWvXLsycORMuLi74+++/YWxsjAULFmDixIkiJ68ctLS0kJOT80nv4TExMfxsRGUkgiB8fpMQIqJ/sWjRIrx69Yp95ajKOH/+PPT19fHo0SP06NEDubm5nGpMMrFu3ToEBwfj0qVL0NTUFDsOiai0tBQzZszAsmXL4OzsjMOHD2P9+vVwdHQs60c3aNAgZGZm4sqVKwDeFMslEglGjBiBjIwMnDp1CgCQm5uL8PBwtGjRAm3bthXtNcnC8OHDoa2tjeDgYLGjEL3j1q1bWLp0Kfbv3w9XV1dMnToVDRo0QF5eHvbv3w8XFxdIJB++GvWpU6dQv359NG7cWI6piRSnuLgYYWFhGDZs2EcV/y9fvgw1NTW0bt1ajumoImHPTiKSO47spKqmc+fOMDc3h7W1NRITE99b6GRvJ/pY169fx5w5cxAeHs5CJ0EqlWLJkiXYuXMnrly5gpycHDx58qSsUJKZmYmDBw+WTY0tKSmBRCJBSkoKMjIy0Lp167I+f9HR0Th+/DicnZ3RvXv3Ct3Pc9WqVTh+/DgiIiLEjkJU5tKlSxgwYAC6dOkCIyMjpKamIigoCA0aNAAAaGtro2fPntixY8cHfz6IioqCnp4eC51UqaiqqmLw4MHYunUrXr9+/UHHXLx4EcXFxSx0Ujmcxk5EcsdiJ1U1paWlkEqlUFFRQZMmTZCamoqMjAzk5eWhsLAQ7du3Z69F+ig5OTkYPHgwAgMD0aRJE7HjkBJxdHSEo6Mj5s+fDx8fH/z1119YtGgRIiIiYGZmhjZt2gBA2QiZvXv34sWLF7C2toaq6puvAn369EHDhg0REREBLy8vnDhxAqNGjRLtNX2OGjVqICQkBMOGDcP169ehp6cndiSqogRBQEREBJYsWYKMjAx4eXkhLCwMOjo6793/iy++gL29Pfbs2YNatWrBzs7unTYTgiAgNjYWmZmZaNWqFQudVCnp6OjAxcUFhw8fhqamJrp27QotLa139ouJiUFmZiYsLCzQokULEZKSMuM0diKSu8jISCxfvhy//fab2FGIFCY/Px9r167FunXrcP/+fRQWFgIAzMzMYGBgAAcHB/Z3og82fPhwSKVShISEiB2FlNiLFy+QkJAAGxsbHDp0CG5uboiNjUWjRo0AABEREfj555/RuHFjbNq0CcCbKYOqqqrIycmBh4cHEhMTkZSUJObLkImpU6fi0aNH2LVrl9hRqIopKirC7t27sWTJEkgkEvj6+mLw4MEf1R83Ozsbp0+fhiAIUFFRwduv7G9vmBobG8srPpFSyc/PR1RUFIqKispNay8sLMS2bdtga2uLKVOmiJiQlBVHdhKR3HFkJ1VFv/76K4KCgtCnTx+Ympri1KlTKCoqwpQpU3Dnzh3s2LED6urqGD16tNhRSclt2bIFly9fRmxsrNhRSMnVrFkTNjY2AABzc3MYGxsjIiICgwYNQnp6OiZNmoTmzZtj8uTJAP5/obO0tBSRkZHYs2dP2Y3Jt9sqqkWLFqFNmzbYtWsXhgwZInYcqgJyc3OxadMmrFixAiYmJliyZAl69uz5UT0436pevTrs7e3lkJKoYtHS0kK/fv3eu61e8kej9wAAIABJREFUvXpwdnbGpEmTPmlxL6rcOLKTiOQuLS0NvXv3xu3bt8WOQqQQaWlpcHJywsCBAzF16lRoamoiLy8PK1aswIULF3D8+HEEBQVh48aNuHHjhthxSYmlpKSgS5cuOHXqFL7++mux41AFs3v3bkyYMAE1atRAXl4e2rZti4CAADRr1gzA/1+w6N69e3BwcICenh4iIiLKnq/oYmNj0adPH8TFxaFu3bpix6FK6tmzZ1i9ejWCg4PRpUsXTJ8+HR06dBA7FlGV0LFjR8yaNYs3B+gdXKCIiOSOIzupqpFKpUhPT4enp2fZQjLa2tpo164dkpOTAQBdu3bFvXv3xIxJSi4/Px+DBw+Gv78/C530SRwdHcsKMefPn8fhw4fLCp2lpaWQSCQoLCzEvn37EBsbi19//bVsW2XQrl07TJw4ESNGjADHd5CsZWRkYNKkSTAzM8OjR49w9uxZ7Nu3j4VOIgXy9PREUFCQ2DFICbHYSURyV7NmTbx8+bLSfHki+l9MTEwglUoRExNT7vn9+/fD0tISJSUlyMnJQY0aNfDixQuRUpKymzp1KiwsLCrsQjGkPN4uQPRWXl4eXr16BQC4desWli1bBk9PTxgZGaGkpKRSTQecOXMmsrKysG7dOrGjUCWRkJAAFxcXtG3bFjo6OkhKSsKvv/7KxeOIRDBo0CDcunUL169fFzsKKZmK24iHiCoMVVVVaGtr49WrV6hRo4bYcYjkTiqVwtPTEx4eHrCyskL9+vURFxeH06dP48iRI1BRUYGBgQG2bt363tUlicLDw/H777/j2rVrlWI6MSkHqfTNOIdDhw5h2bJlGDp0KNLT01FUVIQVK1YAQKX7eVNTU0NYWBisrKzQrVs3mJqaih2JKiBBEPDHH38gICAA169fx5QpU7B27Vp+riUSmbq6OsaPH4+goKCyhfeIAPbsJCIFMTY2RnR0NBo0aCB2FCKFKC4uRnBwMKKjo/H06VMYGBhg6tSpsLS0FDsaKbk7d+7A0tISERERaNu2rdhxqJJaunQp5s2bh/z8fHh5eWHp0qWVblTnf1q9ejV27NiBs2fPVuiFl0ixSkpKcPDgQQQEBCA7Oxs+Pj4YOnQoNDQ0xI5GRP/n6dOnMDMzQ2pqKr744gux45CSYLGTiBSiVatWCAkJQevWrcWOQqRQL168QFFREWrXrl3pRkyR7BUWFqJz584YOnQoPD09xY5Dldzr168xc+ZMrFy5EkOGDMH69etRrVq1d/YTBAFFRUVQV1cXIaVslJaWokePHrCzs8Ps2bPFjkNKrqCgAGFhYVi6dCn09PQwffp02Nvbl42OJiLl4uHhgYYNG/L9ncrw3ZqIFIKLFFFVVbNmTXzxxRcsdNIHmTFjBr766itMnjxZ7ChUBWhoaGDFihW4du0azMzMUFhY+M4+giBg3759aNGiBSIiIkRIKRtSqRQhISEICgpCXFyc2HFISb148QI///wzGjZsiIMHD2Ljxo2IiYnBDz/8wEInkRLz9PTE2rVr3/t7jKomzuEgIoVgsZOI6N8dPnwY+/btQ1xcHIvjpFCtWrVCq1at3rtNIpFg0KBB0NbWxpQpU/DLL78gMDAQZmZmCk75+YyMjLBixQq4uroiNjYWmpqaYkciJfHnn39i5cqV2LRpE/r06YPIyEh8/fXXYsciog/UokULPHz4UOwYpER4e4qIFILFTiKif3bv3j2MGjUKO3fuhJ6enthxiN7Rp08f3LhxA127dkXnzp3h7e2Nly9fih3ro7m4uKBp06bw8/MTOwopgZSUFHh4eKB58+Z4/fo1rl27hrCwMBY6iYgqOBY7iUghWOwkInq/4uJiODs7Y+rUqejUqZPYcYj+kbq6OqZNm4bExES8fPkS5ubm2LhxI0pKSsSO9sEkEgmCg4OxY8cOREdHix2HRHLx4kX88MMPsLGxgbGxMdLS0hAUFARjY2OxoxERkQyw2ElECsFiJ1VVxcXFyM/PFzsGKbG5c+dCR0cHvr6+Ykch+iAGBgbYsGEDjh07hi1btqBDhw44d+6c2LE+WO3atbFhwwa4ubkhOztb7DikIIIg4NixY7CxsYGTkxO6du2Ku3fv4scff4S+vr7Y8YiISIZY7CQihWCxk6qqJUuWYN68eWLHICX122+/ITQ0FGFhYVz8giqcNm3a4MyZM/Dx8YGzszOcnJxw//59sWN9kL59+6J79+6YOnWq2FFIzoqKihAWFoYWLVpg9uzZGDNmDNLS0jBx4kRoa2uLHY+IiOSAn6qJSK6Ki4tx8uRJ5OXlQUtLC0eOHMGBAwfw4MEDsaMRKYSpqSnS0tLEjkFK6NGjRxg+fDjCwsJQp04dseMQfRKJRIIhQ4YgJSUFTZo0QevWrTF//nzk5eWJHe1/Wr58Of744w8cPnxY7CgkBzk5OQgKCkLjxo0REhKCZcuWIS4uDs7OzlBVVd51ekNDQ6Grq6vQa/7xxx+QSCR49uyZQq9LVU9GRgYkEgliY2PFjkKVnEQQBEHsEERU+WRlZeHUqVNQUVGBnZ0datSoUbZNEARcvHgRDx8+hJGRETp27ChiUiL5io+Px9ChQ5GYmCh2FFIiJSUl6NGjB6ysrPDTTz+JHYdIZjIzM+Hr64uLFy9i6dKlcHBwgEQiETvWPzp37hwGDx6MhIQEfPHFF2LHIRl4+vQpVq9ejeDgYNja2sLX1xft27eX+XVsbW3RvHlz/PLLL+WeDw0NxcSJE5GTk/NJ583Pz8erV68UehOssLAQf//9NwwMDJT63yspNzc3Nzx79gxHjx4t93xsbCzat2+Pu3fvwsjICE+fPkXt2rWV+qYDVXwc2UlEMpeeno6oqCgMGDAA33//fblCJ/BmFIilpSUGDRoEPT09HDhwQKSkRPLXuHFjpKeno7S0VOwopEQWL16MkpIS/Pjjj2JHIZIpY2Nj7N69G2FhYVi8eDFsbW0RHx8vdqx/ZGVlBVdXV4wZMwYcA6J8Pub/k7t372LixIlo0qQJ/vrrL1y4cAF79uyRS6HzUxUWFv7PfbS0tBQ+2l9dXR2GhoYsdJLcqaiowNDQ8F8LnUVFRQpMRJUVi51EJFN//vknEhMTMWjQoA/6wGRqagpLS0scOnRIAemIFE9XVxe1atVi6wYqc+bMGfzyyy/Yvn07VFRUxI5DJBfW1taIjY2Fi4sLevXqhTFjxuDp06dix3qv+fPn4/bt29i6davYUeg/vHjx4oM+S8bHx8PZ2Rnt27dHtWrVkJycjPXr18PU1FQBKf+dm5sb+vXrh4CAANSrVw/16tVDaGgoJBLJOw83NzcA75/GfuzYMXTs2BFaWlrQ19dH//79UVBQAOBNAXX69OmoV68edHR00L59e0RGRpYd+3aKelRUFDp27AhtbW20a9cO165de2cfTmMnefvvaexvf/aOHz+ODh06QF1dHZGRkbh//z7s7e2hp6cHbW1tmJubY9euXWXnuXHjBrp16wYtLS3o6enBzc0NL1++BABERkZCXV0dz58/L3ftWbNmoWXLlgCA58+fw8nJCfXq1YOWlhaaNWuGkJAQBf0tkCKw2ElEMnX69Gl89913H3WMoaEhTE1Ny33oIqpM2LeT3nr27BlcXFwQEhKCunXrih2HSK5UVFQwevRopKSkQEdHBxYWFli5cqXSjdrR0NBAWFgYvL29kZmZKXacKi8xMRF9+/ZF06ZNkZSU9I/7CYKAoKAg9O3bF61bt0Z6ejoWL14MQ0NDBab936Kjo3H9+nWcOHECUVFRcHR0xKNHj8oebwszNjY27z3+xIkTsLe3R/fu3XH16lWcPn0aNjY2ZTNG3N3dER0djR07duDGjRsYPnw4+vfvj4SEhHLnmTlzJn7++Wdcu3YN+vr6cHFx4WhmUhrTp0/HwoULkZKSgo4dO2L8+PHIy8vD6dOnkZSUhJUrV6JmzZoAgLy8PPTq1Qu6urq4fPkyDhw4gAsXLmDEiBEAgG7dukFfXx979uwpO78gCNi5cyeGDh0KACgoKECbNm1w9OhRJCUlwdPTE2PGjEFUVJTiXzzJh0BEJCNJSUlCUlLSJx+/Z88eGaYhUh4jR44UgoODxY5BIispKRH69u0r+Pj4iB2FSBQ3b94UevXqJZibmwsRERFix3nH4sWLBTs7O6GkpETsKFVSbGys0KlTJ0FDQ0NwcHAQbt269a/7l5aWCvn5+UJBQYGCEpZnY2MjTJgw4Z3nQ0JCBB0dHUEQBGH48OFC7dq1/zHjkydPBGNjY8HT0/O9xwuCIHTq1ElwdHR87/G3b98WJBKJkJmZWe55e3t7Ydy4cYIgCMLp06cFAMKJEyfKtp87d04AINy/f7/cPk+fPv2Ql070XsOHDxdUVFQEHR2dcg8tLS0BgHD37l3h7t27AgDhypUrgiD8/5+9vXv3ljvX119/LcybN++91/n111+F6tWrC9nZ2WXPvT1PWlqaIAiCMGXKFMHKyqps+9mzZwWpVCo8ePDgH/M7OjoKHh4en/z6SblwZCcRyczNmzdhYWHxycfr6em9M92AqDLgyE4CgMDAQDx//hz+/v5iRyEShbm5OY4fP45ly5Zh8uTJ6NevH1JTU8WOVcbHxwevX7/GqlWrxI5S5aSnp8Pd3R2ZmZl4/PgxwsPDYWZm9q/HSCQSaGpqQkNDQ0EpP03z5s3fm7GwsBA//PADmjZtiuXLl//j8XFxcejatet7t127dg2CIMDCwgK6urplj2PHjuHOnTvl9m3RokXZf3/11VcAgCdPnnzKSyL6R9bW1oiPjy/32LFjx/88rl27duX+7OnpiYULF8LS0hJ+fn64evVq2babN2+iRYsWqFatWtlznTp1glQqRXJyMgBg6NChOH/+fNlo/e3bt8PW1rZsVk1JSQn8/f3RokUL6OvrQ1dXF/v378e9e/c++++AlAOLnUQkE4IgfHbvORsbG5w/f15GiYiUB4uddOnSJQQEBGDnzp1QU1MTOw6RaCQSCfr27YvExETY2dmhc+fO8PHxKeu1JiYVFRVs3boVCxcuLPvCTPLz119/lf13w4YNy6auP378GL///jvc3d0xZ86ccn36lEn16tXf+3P74sWLcotz6ujovPf4sWPHIisrC7t37/7kz9ClpaWQSCS4cuVKueLSzZs3sXnz5nL7/ufvnre9ULl4IsmatrY2GjduXO5Rr169/3ncf/878fDwwN27d+Hu7o7U1FR06tQJ8+bNA/Dme+c/9fN9+3zbtm1hbm6OHTt2oKioCHv27Cmbwg4Ay5Ytw/Lly+Hj44OoqCjEx8fj+++//6BFxKhiYLGTiGQiPz//nWbqH0tFRYWrQFKlZGpqqlSjl0ixXrx4gSFDhmDdunVo0KCB2HGIlIK6ujq8vLyQmJiIrKwsmJubY9OmTaIXXxo1agR/f38MGzZM6XqLVgalpaVYuHAhmjVrBgcHB0yfPr2sL2evXr3w4sULfPPNNxg/fjy0tbURHR0NZ2dnLFiwQCkK4v+pSZMmZSMr/9O1a9fQpEmTfz122bJlOHLkCI4ePYrq1av/676tW7f+xz6CrVu3hiAIePz48TsFJvaFpoquXr16GD16NMLDwzF//nz8+uuvAAALCwskJCTg1atXZfteuHABpaWlaNq0adlzLi4u2L59O06cOIHc3FwMHDiwbNu5c+fQv39/uLq6olWrVmjUqBE/q1cyLHYSkUwUFRXJZLTSf39gJKoMGjVqhIyMDBQXF4sdhRRMEASMHDkS/fr1w4ABA8SOQ6R0DAwMsHHjRhw9ehQhISHo0KGD6LM8Ro8ejTp16mDhwoWi5qhsMjIy0K1bNxw6dAh+fn7o1asXIiIisGbNGgBvZvj06NEDEydORFRUFNasWYMzZ84gMDAQoaGhOHPmjMivoLxx48YhPT0dkyZNQkJCAm7duoXAwEDs3LkT3t7e/3jc77//jlmzZmHt2rXQ0tLC48eP8fjx438s5s6ePRt79uyBn58fkpOTkZSUhMDAQOTl5cHMzAwuLi5wc3PD3r17kZ6ejtjYWCxbtgz79++X10snkjtPT0+cOHEC6enpiI+Px4kTJ8rapbm4uEBHRwfDhg3DjRs3cObMGYwZMwYDBgxA48aNy84xdOhQJCcnY86cOfjuu+/K3VgwMzNDVFQUzp07h5SUFEycOBF3795V+Osk+WGxk4hkolq1asjOzhY7BpFS0tLSgoGBAfsAVUHBwcFIT0/H0qVLxY5CpNTatm2Ls2fPwsvLC0OGDIGzszMePHggShaJRIJNmzZh3bp1uHz5sigZKqOzZ88iMzMTx44dg5OTE2bNmoWGDRuiuLgYr1+/BgCMHDkSEydOhJGRUdlxnp6eyMvLw61bt8SK/l4NGzbEmTNnkJaWhh49eqBDhw7YtWsX9uzZgz59+vzjcefOnUNRUREGDx6ML7/8suzh6en53v379OmDAwcOICIiAq1bt4aNjQ1Onz4NqfTNV/mQkBC4u7vD19cX5ubm6NevH86cOQNjY2O5vG4iRSgtLcWkSZNgYWGB7t27w8DAAFu2bAHwZqp8ZGQksrOz0aFDB9jb28PS0vKd1g3GxsawsrJCQkJCuSnsAODn54cOHTqgd+/esLa2ho6ODlxcXBT2+kj+JAKHURGRjOzbt6/c9ICPlZaWhry8PLRs2VKGqYiUQ7du3eDj44OePXuKHYUUJD4+Ht27d8eFCxdgamoqdhyiCiM3NxdLlizBmjVr4OnpCW9vb2hpaSk8x549ezBnzhxcu3YN2traCr9+ZTN//nxERUVhy5YtaNCgAQRBgL29Pdzd3fHDDz+8s78gCBAEAa9fv4aJiQk8PDy4wBsREX0QjuwkIpn5p0btH+r69essdFKlxUWKqpZXr17B0dERQUFBLHQSfSQdHR389NNPiI2NxY0bN9C0aVPs2bNH4a1uHBwc0LZtW8yYMUOh162sBg8ejBcvXmDkyJEYOXIkqlWrhsuXL8PLywtjx45953ekRCKBVCpFSEgIvvrqK4wcOVKk5EREVNGw2ElEMmNnZ4dTp0590rF5eXmijNogUhQWO6sOQRAwbtw4dOnSBc7OzmLHIaqwGjRogPDwcGzZsgX+/v6ws7NDQkKCQjP88ssvOHDgAE6ePKnQ61ZG5ubmOHDgQNk0682bNyMlJQULFixAamoqvLy8ALz5TLh+/Xps2LABVlZWWLBgAUaOHAljY2P2diciog/CYicRyYyqqir09fWRkpLyUccJgoDw8HB069ZNTsmIxMdiZ9URGhqKuLg4rFq1SuwoRJWCjY0Nrl69CicnJ/Ts2RNjx47F06dPFXLtWrVqYfPmzRgxYgSysrIUcs3KrGHDhkhOTkbnzp0xePBg1KxZEy4uLujduzcyMzPx9OlTaGtr4/79+1i5ciW6dOmCtLQ0jB8/HlKpFBKJROyXQEREFQCLnUQkU9bW1sjIyEBycvIH7V9cXIywsDD88MMPUFdXl3M6IvGYmpoiNTVV7BgkZ8nJyfDx8UF4eDh7/BHJkIqKCsaMGYObN29CS0sLzZo1Q1BQEIqKiuR+7e7du8Pe3h6TJ0+W+7Uqk6KiondGYgqCgGvXrsHS0rLc85cvX0b9+vVRrVo1AMD06dORlJSExYsXQ1dXV2GZiYiocmCxk4hkrlevXvj777+xb98+/PXXX+/dp6SkBKdOncKePXswaNAg1KhRQ8EpiRSrYcOGuH//vkK+mJM48vLy4OjoiICAADRr1kzsOESVUq1atRAYGIjo6GgcP34cLVq0QGRkpNyvu2TJEly+fBl79+6V+7Uquri4ODg5OcHJyemdbRKJBG5ubli3bh1WrVqFO3fuwM/PDzdu3ICLiws0NTUBoKzoSURE9Cm4GjsRyY0gCDh37hz++usv5Ofno6CgAIaGhmXFHhsbG+jr64uckkhxGjVqhIiICJiZmYkdheRg9OjRyM3NxbZt2zjVkkgBBEHAsWPHMHXqVDRt2hTLly+X64Jgly5dwnfffYf4+Hh8+eWXcrtORSQIAk6dOoWAgAAkJydj6tSpGDVqFKpXr/7OvkVFRXByckJiYiIKCwuhr68Pf39/9OjRQ4TkRFSVXL9+Hb1790ZGRgbU1NTEjkNyxGInESnExo0bERMTg02bNokdhUg0vXr1wqRJk9C3b1+xo5CM7dq1C3PmzMG1a9c4IolIwV6/fo1Vq1YhICAAI0aMgJ+f33uLbLLw9t/50aNHeVMDb2bq7N+/HwEBAcjNzYWvry9cXFw+qDXRrVu3oKKigsaNGysgKRHRG3Z2dhg9evR7R59T5cFp7ESkEFlZWahVq5bYMYhExUWKKqfbt29j0qRJ2L17NwudRCLQ0NCAj48PEhMT8fz5c5ibmyMkJASlpaUyv9acOXPw+PFjbNy4Uebnrkjy8/Oxbt06NGnSBIGBgZgzZw6SkpLg7u7+wT3YmzRpwkInESnclClTsHLlSrFjkJyx2ElECsFiJxGLnZXR69ev4ejoiLlz56JNmzZixyGq0gwNDbFp0yYcPnwYGzduRIcOHXDhwgWZXkNdXR1hYWGYNWsW0tPTZXruiiArKwuLFi1Cw4YNcezYMYSGhuLChQuwt7eHVMqvlkSk/Pr164enT5/i4sWLYkchOeJvJCJSCBY7iVjsrIx8fX1hbGyMCRMmiB2FiP5Pu3btcO7cOUybNg2Ojo5wcXHBgwcPZHZ+CwsLzJo1C8OGDUNJSYnMzqvMHjx4AG9vbzRu3Bi3bt3CyZMnceTIEVhZWYkdjYjoo6ioqGDSpEkICgoSOwrJEYudRKQQLHYSsdhZ2Rw8eBCHDh3Cpk2b2LuPSMlIJBI4OzsjJSUFDRs2RKtWrbBw4ULk5+fL5Pyenp5QVVXF8uXLZXI+ZXXz5k24u7ujRYsWKCkpQVxcHLZs2YLmzZuLHY2I6JONGDECkZGRMr0RRsqFxU4iUggWO4mABg0a4NGjRygoKBA7Cn2mzMxMjBkzBrt27eJ7G5ES09HRwYIFCxAbG4uEhARYWFhg3759+Nw1WqVSKbZs2YKlS5fi+vXrMkqrPN5OTbe1tUWjRo1w+/ZtBAYGon79+mJHIyL6bDVq1MDQoUOxdu1asaOQnLDYSUQKwWInEaCqqgpjY+Mq2eetMikqKoKTkxO8vb3xzTffiB2HiD5AgwYNsGfPHoSEhGD+/Pn49ttvP7tIaWxsjKVLl8LV1RWvX7+WUVLxlJaWlk1NHzp0KHr27ImMjAz4+flBT09P7HhERDI1adIkbNy4UWYj/km5sNhJRArBYifRG5zKXvHdvXsXenp68PLyEjsKEX0kW1tbXL16FY6OjujevTvGjRuHZ8+effL5hg8fDhMTE8ybN092IRWssLAQW7ZsQYsWLTB37lxMnDgRqampGD9+PLS0tMSOR0QkF6ampujQoQO2b98udhSSAxY7iUgh0tLSYGZmJnYMItGx2FnxmZqa4vDhw1x5mKiCUlVVxdixY5GSkgINDQ1YWFhg1apVKCoq+uhzSSQS/PrrrwgNDcX58+flkFZ+cnJyEBgYiMaNGyMsLAyBgYG4evUqhgwZAlVVVbHjERHJnaenJ1auXPnZrU1I+fBTOhERkQKx2FnxSSQSFjqJKoFatWph5cqV+OOPP3D06FG0bNkSv/3220efp06dOli3bh2GDRuGnJwcOSSVrSdPnsDPzw8mJiaIiYnBgQMH8Pvvv6N79+5cbI2IqpRu3bpBEAScOnVK7CgkY/ykTkREpEAsdhIRKRcLCwtERkYiICAAEyZMgL29PW7fvv1R57C3t4e1tbVSt7e4c+cOxo8fD3Nzczx//hwxMTEIDw9H27ZtxY5GRCQKiUQCT09PBAUFiR2FZIzFTiIiIgVisZOISPlIJBL0798fiYmJ6Ny5M7755htMnz4dr169+uBzBAUFITIyEsePH5dj0o937do1ODo6omPHjqhVqxZu3ryJ4OBgNG7cWOxoRESiGzp0KGJiYj76JhcpNxY7iYiIFMjIyAjPnj1DXl6e2FHoPW7evIm9e/fizJkzePTokdhxiEjBNDQ04Ovri8TERDx9+hRNmjRBaGgoSktL/+ex1atXR2hoKEaNGoXnz58rIO0/EwShbGq6vb09OnbsiLt378Lf3x8GBgaiZiMiUiba2toYOXIkVq9eLXYUkiEWO4lIZiQSCfbu3Svz8y5btgwNGjQo+/O8efPQvHlzmV+HSBFUVFRgYmLCu8dK6ODBgxg8eDDGjx8PBwcHbNmypdx2Nq8nqjoMDQ2xefNmHDp0COvXr0fHjh0RExPzP4+ztbXFkCFDMG7cOFHeM0pKShAeHo527dph8uTJcHFxwZ07dzBt2jRUq1ZN4XmIiCqC8ePHIywsDNnZ2WJHIRlhsZOoCnNzc4NEIsHIkSPf2ebr6wuJRIJ+/fqJkOzfeXt7Izo6WuwYRJ/MzMyMU9mVzJMnT+Du7o6RI0ciLS0NPj4++PXXX5GdnQ1BEFBQUMCFO4iqoPbt2+PChQuYMmUKHBwc4OrqiocPH/7rMf7+/khKSsLOnTsVlBLIz89HcHAwzMzMEBQUhLlz5yIxMRFubm5QV1dXWA4ioorIyMgI3bt3R0hIiNhRSEZY7CSq4oyMjLB7927k5uaWPVdcXIywsDDUr19fxGT/TFdXF/r6+mLHIPpk7NupfJYsWQJbW1t4enqiRo0a8PDwQJ06dTBixAh88803GDduHK5evSp2TCISgUQigYuLC1JSUmBsbIyWLVvC398fBQUF791fU1MTYWFhmDJlCh48eCDXbFlZWfD394eJiQkiIiKwdetWnD9/Ht999x2kUn7VIyL6UJ6enli1ahVKSkrEjkIywN+ARFVcixYtYGpqivDw8LLnjh07Bk1NTdja2pbbNyQkBBYWFtDU1ISZmRkCAwPf6WH1999/w8HBATo6OmjYsCG2bdtWbvuMGTPQpEkTaGlpoUGDBvD19X3ny8KSJUtoTZfdAAAgAElEQVRgaGgIXV1dDBs2DDk5OeW2//c09itXrqBHjx6oXbs2qlevDisrqw+aakYkFhY7lY+Wlhby8/ORlZUFAPDz80NGRgasra3Rq1cv3L59Gxs3bkRhYaHISYlILLq6uli4cCGuXLmCuLg4WFhYYP/+/e+drt6mTRtMnjwZ7u7uKC0thSAIOHv2LA4dOoQjR47g8OHDOHToEKKioj7pi/X9+/fh5eWFRo0aIS0tDVFRUTh8+DA6d+4si5dKRFTlWFpaQl9fH8eOHRM7CskAi51EBA8PD2zevLnsz5s3b4a7u3u5KZsbNmzArFmzMH/+fNy8eRPLly9HQEAA1q5dW+5c8+fPh729PRISEuDo6IgRI0YgMzOzbLuOjg42b96MmzdvYu3atdi1axf8/f3LtoeHh8PPzw8//fQTrl27hiZNmmDFihX/mv/Vq1dwdXXF2bNncfnyZbRq1Qp9+vTBs2fPPvevhkgu/h979x3W1NmwAfwOGxFBtoCKksSBq7j3tra4aRU3gqN1oRarfbV1t1ZtFbW2LkRRaxW0zmrrqgP3qgNlCagoU5G9cr4//MxbXhyMwEnI/bsurjY5Izf8EXPuPOd5WHaqHxsbG4SEhGDGjBnw9vbG+vXrcejQIUydOhULFiyAu7s7duzYwUWLiAh16tRBUFAQNm3ahPnz56N79+74559/iuw3e/ZspKamYs6cOdi7dy/kcjn69++Pvn37ol+/fujfvz9cXV1x4MABBAcHIysr672vfe/ePXh6eqJp06YAgFu3biEgIAAuLi4q/z2JiLSJRCKBj48P/Pz8xI5CqiAQkdYaPXq04ObmJqSkpAhGRkZCWFiY8PTpU8HAwECIiYlRbhcEQahZs6awbdu2QsevXLlSaNCggfIxAGH27NnKx3l5eYKxsbEQGBj41gw///yz4OzsrHzctm1bYezYsYX26d69u1C7dm3l43nz5gkuLi5vPadCoRDs7Oze+bpEYnr06JFgZ2cndgz6H8uWLRMGDx4sfPfdd4Krq6sQHx8v5OfnC4IgCJcuXRJcXV2F0NBQkVMSkTrJy8sT1q1bJ9jY2AgTJ04UkpKSlNvS0tKE1atXC5mZmcU6z9atW4XExMQ3bj937pzQt29fwdbWVli8eLGQkpKist+BiIheycnJEWrUqCH8888/YkehMuLITiJC9erVMXDgQPj7+2Pr1q3o0qVLofk6ExMT8ejRI0yYMAFVq1ZV/syePRuRkZGFztWkSRPl/+vp6cHa2hoJCQnK54KCgtChQwflberTp09HbGyscntoaCjatm1b6Jz/+/h/JSQkYMKECZDL5TAzM4OpqSkSEhIKnZdIndjb2+Ply5dc8VFkeXl5SE5OVj6eOXMmdu3ahcGDByMvLw95eXnQ1dWFIAj44YcfYGVlhfr164uYmIjUjZ6eHj7//HOEhoZCV1cXDRo0wJo1a5CZmYk9e/Zg4sSJMDY2LtZ5Ro4ciaNHjyrnUVcoFMpb00eNGoWPPvoIDx8+xJw5c1C9evXy/tWIiLSOgYEBJk6cyNGdlYCe2AGISD14eXlh9OjRqFq1KhYuXFho2+t5OX/55Re0a9funefR19cv9FgikSiPv3jxIjw8PDBv3jysXLkS5ubmOHDgAHx9fcuUffTo0YiPj8fKlSvh5OQEQ0NDdO/enXPrkdrS0dGBs7MzIiIi4OrqKnYcrRQQEIDDhw/j2LFjGDp0KFatWgVjY2NIJBLUqlUL1apVQ/PmzdG3b1/ExcUhNDQU169fFzs2EakpCwsLrF69GhMmTMC0adNw6NAh7N+/H7q6usU+h0QiwdChQ7Fnzx5kZ2dj+fLlMDIywqxZs+Du7l6icxERUem8HkSzdOlSWFlZiR2HSokjO4kIANC9e3cYGBggKSkJAwYMKLTN1tYWDg4OiIyMhFQqLfJTXOfPn4eDgwO+/vprtGzZEjKZrNB8ngDQoEEDXLx4sdBz//v4f507dw5TpkyBm5sbXFxcYGpqynn1SO3J5XLO2ymS48eP44svvkD9+vWxfPlybNy4sdC8xXp6ejhy5AiGDRuG69evo1mzZti7dy/Mzc1FTE1EmsDFxQV//PEHPDw8YGRkVOLjdXV18eLFC2zbtg1+fn64evUqBg8ezKKTiKiCWFtbY+DAgdiwYYPYUagMOLKTiAC8Gk3wzz//QBAEGBoaFtk+f/58TJkyBebm5vj444+Rl5eH69ev48mTJ/jqq6+K9RpyuRxPnjzBjh070LZtWxw7dgy//vproX18fHwwatQotGzZEl26dEFQUBAuXboECwuLd553+/btaN26NTIyMvDll1/CwMCgZH8AogrGRYrEkZWVBW9vb8ydOxfTp08HAERHRyM9PR0LFy6ElZUVZDIZevbsiR9//BHZ2dmlKiyISHudPXsW/fr1K/XxY8aMgYODA3r06KHCVEREVFw+Pj5wc3PDzJkzi9y5SJqBZScRKZmamr5129ixY2FiYoLly5fjq6++grGxMVxcXDB58uRin79v376YOXMmpk2bhqysLPTq1QsLFy7ExIkTlfsMGTIEUVFRmDNnDjIzM9GvXz/MmDEDAQEBbz2vv78/xo8fj+bNm8Pe3h7z589HYmJisXMRiUEmk+Hvv/8WO4bW+eWXX+Dq6govLy/lc3/99RdevHiBmjVr4smTJ7CysoKjoyMaNGjwxi9/iIjeJTU1FZaWlqU+3tDQEAUFBSpMREREJdG0aVPIZDIEBQVh6NChYsehUpAIgiCIHYKIiEjbnD17FrNmzUJISIjYUbTKxYsXERMTA3d3d+jp6WHp0qVYtmwZzpw5g0aNGiElJQXOzs74/PPP8e2334odl4g00MGDB9G3b1/Rz0FERKX3+++/Y+nSpe+dUo3UE+fsJCIiEgFvYxdHmzZtMGjQIOjp6SEvLw/16tXDX3/9hUaNGkGhUMDCwgK9evVC1apVxY5KRBqKY0mIiDRf3759kZCQwLJTQ7HsJCIiEoGtrS2ys7Px/PlzsaNohZcvXyr/X0/v1Sw++vr66N+/P5o3bw4A0NHRQVpaGqKiolC9enVRchIRASxMiYjEpquriylTpsDPz0/sKFQKLDuJiIhEIJFIOLqzgkyfPh3ff/89YmJiALz6278uEnR0/vtRSKFQYMaMGcjPz8fnn38uSlYi0nw6OjrIzs4u9fEKhQJ5eXkqTERERKXh5eWFY8eOIT4+XuwoVEIsO4mIiEQil8tZdpazzZs3w8/PD35+fvjyyy9x6dIl5OfnQyKRFNrv1q1b8PLywp9//on9+/eLlJaIKoPu3bvjxIkTpT7+3Llz6NixowoTERFRaZiZmSE6Oho2NjZiR6ESYtlJREQkEo7sLF8pKSkICgrC0qVLsX//fly+fBne3t4IDg7GixcvCu1bp04dtGrVClu2bEGtWrVESkxElYGxsTGysrJKfSt6QkICL6yJiNSEqalpkS/JSf2x7CQiIhIJy87ypaOjg169esHFxQXdu3dHaGgoZDIZJkyYgB9//BFRUVEAgLS0NAQFBWHMmDHo1q2byKmJqDLo1q0bgoODS3zckSNH0Lp163JIREREpcGiUzNJBM5+TUTl6IcffsDjx4+xcuVKsaMQqZ0LFy7Ax8cHly9fFjtKpZWVlQVjY+NCz61cuRJff/01evTogS+++AJr165FdHQ0Ll26JFJKIqqMYmJicPXqVQwaNKhYF8t//PEHnJyc0KBBgwpIR0REVHnpiR2AiCq358+fc1Vjord4PbJTEAR+a1xO/l10FhQUQFdXF9OnT0enTp0wcuRI9OnTB5mZmbh9+7aIKYmoMqpduzZMTEywe/duVKtWDR9++GGhRdGAV6uuX7x4EY8fP0br1q05jQYRkQbJyMjAhQsXUL16ddSvXx8mJiZiR6L/x7KTiMrV8+fPUb9+fbFjEKklS0tLAEBycjKsrKxETlP56erqQhAECIKA5s2bY+vWrWjdujV27NjB9ykiKhdWVlYYMmQIOnTogBs3bqBhw4aF3ovy8/PRunVrtG3bVuyoRERUAsnJyfDw8EBiYiLi4+Ph5uaGTZs2iR2L/h9vYyeicvX6LYaj1ojerFWrVli1ahXatWsndhStkpKSgjZt2qBevXo4ePCg2HGIqBKLiIhA+/bt8ejRIxgYGIgdh4iISkGhUODIkSPYsGEDWrVqBalUioULF2LVqlUwMjLCuHHj8NVXX8HT01PsqAQuUERE5UwikbDoJHoHLlJUvt72na4gCBg2bBiLTiIqd/7+/hgxYgSLTiIiDebp6YkvvvgCzZs3x5kzZ/DNN9+gV69e6NWrFzp16oTx48djzZo1Ysek/8eyk4iISERyuZxlZzlJTExEbm7uGwtPS0tLzJs3T4RURKRN8vPzERAQAG9vb7GjEBFRKT148ACXLl3CuHHjMG/ePBw7dgwTJ07E7t27lfvUqFEDhoaGSExMFDEpvcayk4iISEQc2Vk+8vPz8cknn2DlypVvHV3OUedEVN5er7DesGFDsaMQEVEp5ebmQqFQwMPDA8Crz5AeHh5ITk6Gj48PlixZgmXLlsHFxQXW1tZvvbOIKg7LTiIiIhGx7CwfixYtgr6+PmbOnCl2FCLSYps3b+aoTiIiDde4cWMIgoBDhw4pnztz5gxkMhlsbGxw+PBh2NvbY/To0QD4hbo64AJFREREInrx4gVq1qyJly9f8oORipw8eRIjRozA9evXYWdnJ3YcItJSz549Q4MGDRAbGwtTU1Ox4xARURls3LgRa9euRffu3dGiRQvs3LkTdnZ22LRpE548eYJq1arxvV6N6IkdgIiISJuZm5vDyMgI8fHxLOZUID4+HiNHjsTWrVv59yQiUW3duhXu7u68+CUiqgTGjRuHtLQ0bN++Hfv374elpSXmz58PAHBwcADwar54a2trEVPSaxzZSUREJLJ27dph6dKl6NSpk9hRNJpCocBHH32EFi1aYMmSJWLHISItJggC6tevj4CAALRt21bsOEREpCLx8fFITU2FXC4HAKSmpmL//v346aefYGhoCGtrawwaNAj9+vXjl10i4pydRKQyBQUFhR7zuxSi4uG8naqxbNkyZGRkYMGCBWJHISItJ5FI8ODBAxadRESVjI2NDeRyOXJzc7F48WLIZDJ4enoiMTER7u7uqFOnDrZs2YKxY8eKHVWr8TZ2IlIZXV3dQo8lEgkSExORnZ0Nc3NzfrNF9BZyuZxlZxmdP38eK1euxNWrV6Gnx483RERERKR6EokECoUCCxcuxJYtW9ChQweYm5sjOTkZZ8+eRVBQEMLCwtChQwccPXoUvXv3FjuyVuLITiJSiezsbIwfPx55eXkAgNzcXKxbtw7e3t4YN24cpk2bhps3b4qckkg9cWRn2aSkpGDYsGHYtGkTatasKXYcIiIiIqrErl69ih9++AG+vr5Yv349/P39sW7dOsTExGDFihWQy+Xw8PDAjz/+KHZUrcWyk4hUIj4+Hps2bYK+vj5yc3Oxdu1aTJs2DSYmJpDJZLh48SJ69OiBmJgYsaMSqR2WnaUnCALGjBkDd3d39O3bV+w4RERERFTJXbp0Cd26dYOPj49yQSIHBwd069YN9+7dAwD07t0bDRs2RHZ2tphRtRbv8yIilUhJSYGZmRkA4OHDh9i4cSNWrVqFiRMnAng18rN///74/vvvsW7dOjGjEqkdqVSKyMhIKBQK6Ojwe8iSWL16NeLi4rBnzx6xoxARERGRFrC0tERoaCjy8/NhYGAAAAgLC8O2bdvg6+sLAGjTpg3atWsHIyMjMaNqLV5REZFKJCQkoHr16gCgfNMfNWoUFAoFCgoKYGRkhE8//RS3bt0SOSmR+jE1NUW1atUQFxcndhSNcvXqVSxevBi//fab8oMmEZHY5s+fj0aNGokdg4iIysmwYcOgq6uL2bNnw9/fH/7+/pg7dy5kMhkGDRoEALCwsIC5ubnISbUXy04iUonU1FRER0fDz88PS5YsAQDk5ORAR0dHuXBRWlpakRXbiegV3speMqmpqfDw8MBPP/2EunXrih2HiDSEp6cnJBKJ8sfKygp9+vTB/fv3xY5WIU6fPg2JRIKkpCSxoxARabSAgADExcVhwYIFWLVqFZKSkjB79mzUqVNH7GgE3sZORCpiZWWFZs2a4eDBg0hOToZcLsfTp09haWkJ4FXRGRoaCrlcLnJSIvUkk8kQFhaGrl27ih1F7QmCgPHjx6Nnz54YPHiw2HGISMP06NEDgYGBAIC4uDjMnDkTAwcORGhoqMjJ3i03N5ej2ImI1ET79u3RunVrPHv2DM+fP0fjxo3FjkT/wpGdRKQSXbp0wV9//YV169Zh/fr1mDlzJmxtbZXbw8PDkZ6ejt69e4uYkkh9yeVyjuwspo0bN+L+/ftc4ZKISsXQ0BB2dnaws7ODq6srpk+fjvv37yMrKwvR0dGQSCS4evVqoWMkEgmCgoKUj+Pi4jB8+HBYWlqiSpUqaNasGU6dOlXomF27dsHZ2RmmpqYYMGBAodGUV65cQa9evWBlZYVq1aqhQ4cOuHDhQpHX/OmnnzBo0CCYmJjgP//5DwDg3r17cHNzg6mpKWxsbDB06FA8e/ZMedzt27fRvXt3VKtWDaampmjatClOnTqF6Oho5Rdq1tbWkEgk8PT0VMnflIhIG+np6cHR0ZFFpxriyE4iUokTJ04gLS1NOUfJa4IgQCKRwNXVFTt37hQpHZH6k8lkCAkJETuG2rt9+zbmzJmDs2fPwtjYWOw4RKTh0tLS8Ntvv6Fx48bFfk/JyMhA586dYWNjg3379sHBwaHInOTR0dH47bffsG/fPmRkZMDDwwNz5szB+vXrla87cuRI+Pn5QSKRYO3atfj4448RHh4OKysr5XkWLFiAb7/9FitWrIBEIsHTp0/RqVMneHt7Y8WKFcjLy8OcOXPQr18/XLx4ETo6Ohg2bBiaNm2Ky5cvQ09PD7dv34aRkRFq1qyJ4OBguLu74+7du7CwsOD7KBERVUosO4lIJfbu3Yv169ejd+/eGDJkCPr27QsLCwtIJBIAr0pPAMrHRFQY5+x8v4yMDAwePBg//PAD6tevL3YcItJQR48eRdWqVQG8el+pWbMmjhw5Uuzjd+7ciWfPnuHChQvKYtLZ2bnQPvn5+QgICICZmRkAYPz48diyZYtye7du3Qrtv2bNGgQHB+Po0aMYMWKE8vkhQ4Zg7NixysfffPMNmjZtiu+//1753LZt22BhYYGrV6+iVatWiImJga+vr/J9UiqVKve1sLAAANjY2BQqVYmIqGxeX+8CvOZVB7yNnYhU4t69e/jwww9hYmKCuXPnYvTo0dixY4dydenXCwEQ0Zs5Ozvj4cOHXMTrHSZPnozWrVtj1KhRYkchIg3WqVMn3Lx5Ezdv3sSlS5fQrVs39OrVC48ePSrW8Tdu3ECTJk3eWRbWrl1bWXQCgL29PRISEpSPExISMGHCBMjlcpiZmcHU1BQJCQmIjY0tdJ4WLVoUenzt2jWcOXMGVatWVf7UrFkTABAZGQkAmDFjBsaOHYtu3bphyZIlWrP4EhGRmCQSCZYsWQJ/f3+xoxBYdhKRisTHx8PLywuBgYFYsmQJcnNzMWvWLHh6emL37t2FPuATUVFVqlSBlZVVsS+2tU1gYCAuXLiAtWvXih2FiDRclSpVIJVKIZVK0apVK2zevBkvX77Ehg0boKPz6vLo3yN08vLyCh3/721vo6+vX+ixRCKBQqFQPh49ejSuXLmClStXIiQkBDdv3oSjoyNyc3MLHWdiYlLosUKhgJubm7Ksff0THh6OPn36AADmz5+Pe/fuYcCAAQgJCUGTJk148U1EVAFatWoFPz+/Yv07QeWLZScRqURaWhqMjIxgZGSEUaNG4ciRI1i1ahUkEgnGjBmDfv36ISAgoMiHeCL6L97K/mYPHjzAjBkzsHv3buWtp0REqiKRSKCjo4PMzExYW1sDAJ4+farcfvPmzUL7u7q64p9//im04FBJnTt3DlOmTIGbmxtcXFxgampa6DXfxtXVFXfv3kXt2rWVhe3rH1NTU+V+MpkMU6dOxeHDh+Ht7Y1NmzYBgHI1d95FQESkej179kR+fn6RBeuo4rHsJCKVyMjIUF4g5OfnQ1dXF5988gmOHTuGP/74A/b29vDy8lLe1k5ERclkMoSFhYkdQ61kZWVh8ODBWLx4MZo0aSJ2HCKqBHJycvDs2TM8e/YMoaGhmDJlCtLT09G3b18YGxujTZs2+P7773H37l2EhITA19e30PHDhg2DjY0NBgwYgLNnz+Lhw4c4cOBAiS5u5XI5tm/fjnv37uHKlSvw8PBQFpHvMmnSJKSmpmLIkCG4dOkSoqKicPz4cYwfPx5paWnIysrCpEmTcPr0aURHR+PSpUs4d+4cGjZsCODV7fUSiQSHDx9GYmIi0tPTS/bHIyKit5JIJPDx8YGfn5/YUbQey04iUonMzEzl3FR6eq/WPlMoFBAEAZ06dcLevXtx69YtODo6ihmTSK1xZGdRX3zxBerXr4/x48eLHYWIKonjx4+jRo0aqFGjBlq3bo0rV65gz5496NKlCwAob/lu2bIlJkyYgMWLFxc63sTEBH///TccHBzQt29fuLi4YN68eSWam9zf3x/p6elo3rw5PDw84OXlBScnp/ceZ29vj/Pnz0NHRwe9e/eGi4sLJk2aBENDQxgaGkJXVxfPnz/H6NGjUa9ePQwcOBBt27bFjz/+CABwcHDAggULMGfOHNja2mLy5MnFzkxERO83cuRIhISEKOdRJnFIBE4mQEQqkJKSAnNzc+VcV/8mCAIEQXjjNiL6rwMHDmD9+vU4fPiw2FHUQlBQEGbNmoXr168XWuiDiIiIiEhdzZo1Czk5OVi1apXYUbQWy04iIiI1ERoaiv79+/NWdgBRUVFo06YNDh8+jJYtW4odh4iIiIioWGJjY9GsWTNER0ejWrVqYsfRShxmRUTl4vVoTiIqvrp16yI2Nhb5+fliRxFVbm4uPDw88J///IdFJxERERFplFq1aqFHjx4ICAgQO4rWYtlJROXiwoULOHfunNgxiDSKoaEhatSogejoaLGjiOqrr76CnZ0dfHx8xI5CRERERFRiPj4+WL16NRQKhdhRtBLLTiIqF8eOHcOJEyfEjkGkcbR9kaJDhw5hz5492LJlS4kW+yAiIiIiUhft2rVD9erVORe/SFh2ElG5eP78OapXry52DCKNI5PJtHbOzsePH2Ps2LHYuXMnLC0txY5DRERERFQqEokEPj4+8PPzEzuKVmLZSUTlgmUnUelo68jO/Px8DB06FD4+PujQoYPYcYiI3qlt27Y4dOiQ2DGIiEiNDR48GPfu3cOdO3fEjqJ1WHYSUblg2UlUOnK5XCvLzvnz58PY2BizZs0SOwoR0TvdvXsXsbGx6N27t9hRiIhIjRkYGOCzzz7j6E4RsOwkonLBspOodLRxZOfx48exZcsWBAYGQkeHH02ISL1t3rwZnp6e0NPTEzsKERGpuc8++wxBQUFISkoSO4pW4RUFEZULlp1EpePk5IS4uDjk5uaKHaVCPHv2DKNGjcK2bdtga2srdhwionfKycnB9u3b4eXlJXYUIiLSADY2NhgwYAA2btwodhStwrKTiMoFy06i0tHX10fNmjURFRUldpRyp1AoMHLkSIwdOxbdu3cXOw4R0XsdOHAAjRo1grOzs9hRiIhIQ/j4+OCnn35CXl6e2FG0BstOIioXLDuJSk9bbmVfunQpcnJy8M0334gdhYioWDZv3gxvb2+xYxARkQZp1qwZpFIpgoODxY6iNVh2EpHKZWVlAQCMjY1FTkKkmbSh7Dx79ixWr16NnTt3ct47ItIIsbGxuHLlCgYNGiR2FCIi0jA+Pj5cqKgCsewkIpXjqE6ispHJZAgLCxM7RrlJSkrC8OHDsXnzZjg6Ooodh4ioWLZs2YKhQ4fyy1wiIiqxfv364dmzZ7h8+bLYUbQCy04iUjmWnURlI5fLK+3ITkEQMGbMGAwePBhubm5ixyEiKhaFQoEtW7bwFnYiIioVXV1dTJ48maM7KwjLTiJSOZadRGVTmW9jX7VqFRISEvDtt9+KHYWIqNhOnDgBCwsLfPDBB2JHISIiDeXt7Y0//vgDT548ETtKpceyk4hUjmUnUdnUqlULiYmJyvlvK4vLly/ju+++w65du2BgYCB2HCKiYtu0aRPGjh0rdgwiItJg5ubmGDZsGH7++Wexo1R6LDuJSOVYdhKVja6uLpycnBAZGSl2FJVJTU2Fh4cHfv75Z9SpU0fsOERExZaUlIRjx45h2LBhYkchIiINN2XKFGzYsKHSDWpQNyw7iUjlWHYSlV1lupVdEASMHTsWH330Edzd3cWOQ0RUItu3b0efPn1gbm4udhQiItJw9erVQ8uWLbFz506xo1RqLDuJSOVYdhKVXWUqO9evX4/w8HD88MMPYkchIioRQRCwefNm3sJOREQq4+PjAz8/PwiCIHaUSotlJxGpHMtOorKTyWQICwsTO0aZ3bp1C19//TV2794NIyMjseMQEZXIlStXkJWVhc6dO4sdhYiIKomePXsiPz8fp0+fFjtKpcWyk4hUjmUnUdlVhpGd6enpGDx4MFauXAm5XC52HCKiEtu0aRO8vLwgkUjEjkJERJWERCLB1KlT4efnJ3aUSotlJxGpHMtOorKTy+UaX3ZOmjQJ7du3x4gRI8SOQkRUYhkZGQgKCoKnp6fYUYiIqJIZOXIkzp07V6kWJFUnLDuJSOVYdhKVnYODA168eIH09HSxo5TK1q1bceXKFaxZs0bsKEREpbJnzx60b98e9vb2YkchIqJKxsTEBN7e3li7dq3YUSollp1EpHIsO4nKTkdHB87OzoiIiBA7SomFhobC19cXu3fvhomJidhxiIhKZdOmTVyYiIiIys2kSZOwbds2vHz5UuwolTlX4AAAACAASURBVA7LTiJSOZadRKqhifN2ZmVlYciQIfj222/RqFEjseMQEZXK/fv3ERkZiY8//ljsKEREVEnVqlUL3bp1Q0BAgNhRKh2WnUSkciw7iVRDE8vO6dOnw8XFhaOhiEij+fv7Y9SoUdDX1xc7ChERVWLTpk3DmjVroFAoxI5SqbDsJCKVys7OhkKhgLGxsdhRiDSeTCZDWFiY2DGK7bfffsPx48exfv16rlxMRBorLy8P27Ztg7e3t9hRiIiokmvXrh3MzMxw5MgRsaNUKiw7iUilXo/qZNFBVHaaNLIzMjISU6ZMwe7du1GtWjWx4xARldqhQ4cgl8shl8vFjkJERJWcRCKBj48P/Pz8xI5SqbDsJCKV4i3sRKojl8s1ouzMycnBkCFDMHfuXLi6uoodh4ioTDZv3sxRnUREVGEGDx6MO3fu4M6dO2JHqTRYdhKRSrHsJFIdOzs7ZGVlITU1Vewo7zR79mw4OjpiypQpYkchIiqTJ0+eICQkBJ988onYUYiISEsYGhri888/x+rVq8WOUmmw7CQilWLZSaQ6EokEUqlUrUd3HjhwAPv27YO/vz+nryAijRcQEIDBgwfDxMRE7ChERKRFJkyYgD179iA5OVnsKJUCy04iUimWnUSqpc7zdsbGxmLcuHHYuXMnLCwsxI5DRFQmCoWCt7ATEZEobG1t0b9/f2zYsEHsKJUCy04iUimWnUSqpa5lZ15eHoYOHYoZM2agXbt2YschIiqz06dPw9TUFC1atBA7ChERaSEfHx+sW7cOeXl5YkfReCw7iUilWHYSqZa6lp3z5s2DqakpZs6cKXYUIiKVCA4Ohre3N6fkICIiUXzwwQeoW7cu9u7dK3YUjceyk4hUimUnkWrJZDKEhYWJHaOQP//8E9u2bcO2bdugo8OPEkSk+QRBwNq1azFp0iSxoxARkRbz8fGBn5+f2DE0Hq9QiEilWHYSqZZcLlerkZ1Pnz6Fp6cnAgMDYWNjI3YcIiKVkEgkkEgk0NXVFTsKERFpsf79++Pp06e4fPmy2FE0GstOIiqz5ORk7N+/HwcOHICBgQESExNx6dIlCIIgdjQijWdlZQWFQqEWKzMWFBRgxIgRGD9+PLp27Sp2HCIiIiKiSkVXVxeTJ0/m6M4ykghsI4iolG7cuIGoqChYWFigU6dOhUZDxMbG4vLly9DX10evXr1gbGwsYlIizdayZUusWbMGbdq0ETXHokWLcPLkSRw/fpyjn4iIiIiIysGLFy9Qt25d3LlzB/b29mLH0UgsO4moVA4ePIi6devCxcXlnfvl5ubit99+Q+/evWFtbV1B6Ygql2HDhuGjjz7CyJEjRcvw999/Y8iQIbh+/To/dBERERERlaNJkybBwsICixYtEjuKRuJt7ERUYgcPHsQHH3zw3qITAAwMDDBixAj89ddfSE1NrYB0RJWP2CuyJyYmYsSIEdiyZQuLTiIiIiKicjZ16lRs2LAB2dnZYkfRSCw7iahErl+/DmdnZzg6Ohb7GIlEAg8PDxw+fLgckxFVXmKWnQqFAqNHj1aOLiUi0lSJiYnYtGkTfvnlF/z88884f/682JGIiIjeqF69emjevDl27twpdhSNpCd2ACLSLA8fPoS7u3uJj9PR0UHdunXx+PHjEhWlRPSq7AwLCxPltX/88Uc8f/4cixcvFuX1iYhUYf/+/Vi+fDnu3r0LExMTODg4ID8/H7Vr18ann36Kfv36wcTEROyYRERESj4+Pvjyyy8xZswYSCQSseNoFI7sJKJiS0xMhJWVVamPb926NS5duqTCRETa4fXIzoqeZvvSpUtYtmwZdu3aBX19/Qp9bSIiVZo1axZat26NqKgoPH78GCtWrMDgwYORn5+PZcuWYfPmzWJHJCIiKqRXr17Iy8vD6dOnxY6icVh2ElGxhYSEoGPHjqU+XiKRQEeHbztEJWVhYQEDAwMkJCRU2Gs+f/4cHh4eWL9+PWrXrl1hr0tEpGpRUVF48eIFZsyYgerVqwMAOnbsiFmzZmHdunUYMGAApk2bhl9//VXkpERERP8lkUgwdepU+Pn5iR1F47B1IKJi09HRKXNZqaenV+Gj04gqg4qct1MQBIwdOxZ9+/bFwIEDK+Q1iYjKi0QigaWlJdavXw/g1XtcQUEBBEGAo6Mj5s2bB09PTxw/fhx5eXkipyUiIvqvkSNH4ty5c4iKihI7ikZh2UlExaaKklIikfBCgqgUKrLsXLduHaKjo7F8+fIKeT0iovJUp04dfPrpp9i1axd27doFANDV1S00/1ndunVx7949TtlBRERqxcTEBF5eXli7dq3YUTQKFygiogoVGRkJKysrSKVSyGQySKXSQj92dnacfJnoDSqq7Lx58ybmz5+PkJAQGBoalvvrERGVJ0EQIJFIMGnSJCQmJmLkyJFYuHAhPvvsM3z44YeQSCS4ceMGduzYgYkTJ4odl4iIqIjJkyfjgw8+wIIFC2Bqaip2HI0gEXg/KREV09mzZyGXy2Fra1vqcwQFBaF79+6IiIgo8hMeHo7MzMwiBejrH3t7e875SVpr165dCA4Oxp49e8rtNdLS0tC8eXMsWLAAQ4cOLbfXISKqSKmpqUhLS4MgCEhOTkZQUBB27tyJmJgY1KlTB6mpqfDw8MCqVaugq6srdlwiIqIiPv30U3Tq1AlTpkwRO4pGYNlJRMUmCAL27t0Ld3f3Uh3//PlzXL9+Hd27d3/rPqmpqYiMjHxjEZqamgpnZ+c3FqE1a9ZkEUqV2rVr1+Dl5YVbt26Vy/kFQcDIkSNhbGyMjRs3lstrEBFVpNTUVPj7+2PhwoWoUaMGCgoKYGtrix49emDAgAHQ19fHjRs38MEHH6BBgwZixyUiInqrc+fOYcyYMXjw4AGve4uBt7ETUbG9Xk09Pz8fenolf/s4ffo0+vXr9859zMzM4OrqCldX1yLb0tPTCxWhV69exa+//oqIiAgkJyejTp06RUpQmUyGmjVrliovkTqRyWSIiIhQ3pKpagEBAbh58yYuX76s8nMTEYlhyZIlOHfuHH755RdYWFhg7dq1OHjwILKysnDy5EmsWLECw4YNEzsmERHRe7Vv3x7VqlXDkSNH0KdPH7HjqD2O7CSiEklPT8eBAwdKfHEQFhaGuLg4dOnSpVxyZWZmIioqqtBI0Nf/Hx8fj9q1axcpQaVSKWrXrs3FCEhj2NnZ4dq1a3BwcFDpee/du4fOnTvj9OnTcHFxUem5iYjE4uDggA0bNsDNzQ0AkJiYiBEjRqBz5844fvw4Hj9+jMOHD0Mmk4mclIiI6P0CAwOxbds2/PXXX2JHUXssO4moxJ48eYKQkBB88sknxRphFhYWhvDwcOXFRkXLzs7Gw4cPi5SgERERiIuLg6OjY5ESVCqVok6dOjAwMBAlM9GbdOzYEYsWLVLplwaZmZlo1aoVZsyYAS8vL5Wdl4hITBEREfj000+xevVqdOzYUfm8jY0Nrly5gtq1a6N+/fr47LPPMG3atHIbNU9ERKQqOTk5cHJywvHjxzlA4T1YdhJRqSQnJ+Po0aNo0KDBG285B4AXL17g1KlTMDc3R9euXSs4YfHk5uYiOjq6SAkaERGBR48eoUaNGm9cOb5u3bowMjISOz5pGS8vL7Rt2xbjxo1T2TnHjRuHrKwsBAYG8kKfiCoFQRBQUFCAQYMGwczMDBs3bkRmZiYCAwPx7bffIj4+HgDg6+uL6Oho7Nq1i9PdEBGRRliwYAHi4uKwfv16saOoNf6rTkSlYmlpieHDhyMyMhJBQUHQ1dWFoaEhDA0NkZ6ejry8PJiZmaFv375qfQFhYGAAuVwOuVxeZFteXh5iY2MLFaEnT55EREQEoqOjYWNjU6QElUqlcHZ2RpUqVUT4baiyk8lkCA8PV9n5fv31V/z999+4du0ai04iqjQkEgn09PTwySef4PPPP0dISAhMTEyQmpqKZcuWFdo3NzdXrT+nEBER/dtnn32G+vXrY/r06bh//36hxYpMTU3RuXNnLmAEjuwkIhXKy8tDbm4uqlSpUumLk4KCAsTGxhYZDRoREYGoqChYWlq+cdV4qVSKqlWrVkjGrKws7NmzB7du3YKpqSk+/PBDtGzZkhd1GiwoKAg7duzAvn37ynyu8PBwtGvXDn/++Sc++OADFaQjIlI/iYmJ8Pf3R0JCAkaPHo0mTZoAAO7fv4/OnTtj48aN7108kYiISF1cv34dO3fuRNeuXfHRRx8VKjaTkpJw5swZCIKAHj16wMzMTMSk4mLZSUSkYgUFBXjy5EmREjQ8PByRkZEwMzN7axGqyn+QHj16hKVLlyI9PR2BgYHo3bs3AgICYGNjAwC4cuUKjh8/jqysLMjlcrRp0wbOzs6FimrOYaZebt26heHDh+POnTtlOk9OTg7atWsHLy8vTJo0SUXpiIg0Q1paGn777TecPHkSO3fuFDsOERFRsRw8eBDOzs5o2LDhO/dTKBTYs2cP2rRpg9q1a1dQOvXCspOIqAIpFAo8ffq0SAn6+v+rVKlSpAB9fat89erVS/RaBQUFiIuLQ82aNdG8eXN07twZixcvVt5i7+npiaSkJBgYGODx48fIzs7G4sWLlSNcFAoFdHR08OLFCzx79gx2dnYwNzdX+d+Eii8jIwNWVlbIyMgo0+0pPj4+ePToEYKDg1lmE5FWio+PhyAIsLOzEzsKERHRex06dAjNmjWDo6NjsY/Zt28f2rVrB1tb23JMpp5YdhIRqQlBEBAfH//GEjQ8PBz6+vpFStBevXrB2tr6vYWVnZ0dZs6cienTpytLsgcPHsDExASOjo5QKBTw9fXF1q1bce3aNTg5OQF4dZvfggULEBISgvj4eLRo0QIBAQGQSqXl/eegt3B0dMT58+dL/S3t77//junTp+P69eslLtCJiIiIiKhi/fPPPwCgnIqluARBwK+//ophw4aVRyy1xrKTiEgDCIKApKSkIiXoV199hUaNGr2z7MzIyICNjQ38/f0xZMiQt+6XkpICGxsbXLhwAS1btgQAtG/fHpmZmfjll1/g6OgIb29v5OXl4dChQzA2Nlb570nv17VrV8yZMwc9evQo8bExMTFo2bIlDhw4gDZt2pRDOiIi9fP6cocj2YmISBMFBwfD3d29VMfeuXMH+vr6qFevnopTqTeuUkFEpAEkEgmsra1hbW2Ntm3bFuuY1/NtPnz4EBKJRDlX57+3vz43AOzfvx/6+vqQyWQAgJCQEFy4cAE3b95Ufou4cuVKuLi44OHDh++dK4bKx+sV2Utadubl5cHDwwNffvkli04i0ipTp07F119/XeTfQSIiInX34sWLMk0l1qhRI+zdu1fryk6uR09EVEkpFAoAQGhoKKpVqwYLC4tC2/+9+ND27dsxb948TJ8+Hebm5sjJycGxY8fg6OiIJk2aID8/HwBgZmYGOzs73L59u2J/GVJ6XXaW1Ndff43q1atjxowZ5ZCKiEg9RUVFYdeuXVq9Ii0REWmus2fPokuXLmU6R1nm+tdUHNlJRFTJ3bt3DzY2Nsr5GQVBgEKhgK6uLjIyMjB//nwEBwdj4sSJmD17NoBXq3WHhoZCLpcD+G9xGh8fD2tra6SmpirPxdsCK5ZMJsOZM2dKdMzRo0exY8cOXL9+XSs/7BCR9tqyZQuGDx8OQ0NDsaMQERGViq6ubpmOr1q1KrKysrRqGjKWnURElZAgCHjx4gUsLS0RFhYGJycn5aiW10XnrVu34OPjgxcvXmDdunXo3bt3ofIyPj5eeav661veY2NjoaurW2SU6Ot94uPjYWVlBT09/vNSXko6sjMuLg5jxozBrl27YG1tXY7JiIjUS0FBAbZs2YI//vhD7ChERESloopldgwNDZGdnc2yk4iINNuTJ0/Qq1cvZGdnIzo6GnXq1MH69evRuXNntG7dGoGBgfjhhx/Qvn17fPfdd6hWrRqAV/N3CoKAatWqITMzE1WrVgXw328Tb926BWNjY+Vq7f87qrN37964f/8+atWqVWTleKlUCicnJ+jr61fcH6IScnZ2RnR0NPLz899bKhcUFGD48OGYOHEiOnfuXEEJiYjUw7Fjx+Dg4IDGjRuLHYWIiEg0qampWjedC8tOIqJKyMHBAbt27cKNGzcQFxeHa9eu4eeff8alS5ewevVqTJ8+HSkpKbC3t8eKFStQr149yGQyNG7cGIaGhpBIJKhXrx4uXryIuLg42NvbA3i1iJGrq6vy9vZ/k0gkuHnzJnJycvDw4UPlivEPHjzA4cOHERERgSdPnsDBwaFICSqVSlGnTh3eZlgMRkZGsLW1RUxMDJydnd+57+LFi6Gjo4P//Oc/FZSOiEh9bN68Gd7e3mLHICIiKrVatWohMjLyvZ/73yU3N1frprKSCKoYE0tERBrl/v37CA8Px99//43bt28jKioKMTEx8PPzw4QJE6Cjo4MbN25g2LBhcHNzw8cff4xffvkFx48fx6lTp9C0adNSvW5ubi5iYmIQERGB8PBwZSEaERGB2NhY2NnZvbEIrVu3rlbddvE+PXv2xBdffIHevXu/dZ9Tp05h2LBhuH79OmrUqFGB6YiIxBcfH4969eohNjZWefcCERGRJgoODoa7u3upjk1LS8OFCxfQq1cvFadSbyw7iYhISaFQFPrWb9++fVi2bBmioqLQsmVLzJ8/Hy1atCiX187Pz0dsbGyREjQiIgIPHz6EtbV1kRJUKpXC2dkZJiYm5ZJJXU2cOBENGjTAlClT3rg9ISEBrq6u8Pf317oPNkREALBixQrcvXsXW7ZsETsKERFRmRw+fBjdunUr1eCPAwcO4KOPPtK6qcRYdhJRmXl6eiIpKQmHDh0SOwqVIzFXXi8oKMCjR4+KlKARERGIioqCubl5kRL09Y+pqakomctLfn4+Zs+ejZcvX6JPnz6QSCRwcnJSzkmnUCjg5uaGZs2a4bvvvhM5LRFRxRMEAQ0bNsTGjRvRoUMHseMQERGVSW5uLn799VeMGjWqRNdj4eHhePToEbp161aO6dQTy04iLeDp6YmtW7cCAPT09FC9enW4uLjgk08+wfjx48v8LY8qys7Xi+hcuXKl3EYOUuWkUCjw5MmTIiVoeHg4IiMjYWpq+sYSVCqVwtzcXOz4xRYfH4/z589DR0cHnTt3RvXq1ZXbHjx4gDt37sDY2Bg3b97E4cOHcfr0aa37BpeICADOnz8Pb29vhIaGivYlHRERkSqlpKTg8OHDGD58eLHm3wwPD0dYWBjc3NwqIJ364QJFRFqiR48eCAwMREFBARITE3Hy5EnMmzcPgYGBOHHixBtvA87NzYWBgYEIaYmKT0dHBzVr1kTNmjXRtWvXQtsEQcDTp08LlaB79+5V3ipvZGT0xhJUJpPBwsJCpN+oqMuXL+PFixcYOHDgGy/c69Wrh3r16iEjIwOHDh3C6tWrWXQSkdZ6vTARi04iIqosLCwsMHDgQOzatQu1atVC+/bt3/jvXEpKCk6fPg0LCwutLToBjuwk0gpvG3l5584duLq64quvvsKCBQvg5OQET09PxMbGYu/evejZsyf27NmD27dvY/r06Th//jyMjY3Rr18/+Pn5wczMrND527RpgzVr1iAjIwOffvop1q1bp5xXRBAELF++HOvXr0dcXBykUilmzZqFESNGAECRN+rOnTvj9OnTuHLlCubMmYPr168jNzcXTZo0wfLly9G2bdsK+MtRZSYIAhISEoqMBn39X11d3TeWoFKpFFZWVhV2EX358mXo6OgUe8SzIAjYvXs3evToAUtLy3JOR0SkXl6+fInatWvj/v37sLW1FTsOERGRyj179gznz5+HRCKBnp4edHR0oFAokJOTA0tLS3Tu3Bm6urpixxQVy04iLfCu28z79euHqKgo3LlzB05OTkhJScHcuXMxaNAgCIIABwcHyGQytGzZEosWLUJKSgrGjRuHxo0bIzg4WHn+4OBg9O7dG/PmzcOTJ0/g5eUFd3d3rF69GgAwZ84cBAUFwc/PD/Xq1cOFCxcwbtw47N69G25ubrhy5QpatWqFo0ePomnTpjAwMICFhQVOnjyJJ0+eoEWLFpBIJFi7di127NiB8PBwWFlZVejfkbSHIAhITk4uUoK+/snPz39jCSqVSmFra6uyIjQ+Ph43b97Ehx9+WOL8O3bsUH6ZQESkLTZu3IgjR45g3759YkchIiIqd4IgQKFQaH25+b9YdhJpgXeVnbNnz8bq1auRmZmpXOTk4MGDyu0bN26Er68vHj9+rFzo5fTp0+jatSvCw8MhlUrh6emJ33//HY8fP0bVqlUBANu3b4e3tzdSUlIAAFZWVvjzzz/RsWNH5bmnTZuGsLAwHDlypNhzdgqCAHt7eyxfvpxFDokmJSUFkZGRb1w5PjMz840lqFQqRY0aNYo1x85re/fufeut6+9z//595Ofno1GjRiU+lohIU7Vp0wZff/21Vt+6R0REpO04ZyeRlvvfFbb/t2gMDQ1FkyZNCq1o3a5dO+jo6ODevXuQSqUAgCZNmiiLTgBo27YtcnNzERkZiZycHGRnZ6N3796FXisvLw9OTk7vzJeQkICvv/4ap06dQnx8PAoKCpCVlYXY2Niy/NpEZWJhYQELCwu0bNmyyLbU1NRCRei5c+cQEBCAiIgIpKamwtnZ+Y0rxzs6OhYqQgsKCiCRSEo9SrR+/foICgpi2UlEWuPOnTt49OhRiUfDExERUeXCspNIy927dw9169ZVPv7fhYr+twz9t+KWMAqFAgBw8OBB1KpVq9C29y2iMnr0aMTHx2PlypVwcnKCoaEhunfvjtzc3GK9NlFFMzMzg6urK1xdXYtsS0tLQ2RkpHIU6OXLl7Fz505EREQgOTkZdevWVZafhoaGmDlzZpmyGBkZIScnB4aGhmU6DxGRJti8eTM8PT2hp8dLHCIiIm3GTwJEWuzOnTs4evQo5s6d+9Z9GjZsCH9/f6SlpSlHd4aEhEChUKBBgwbK/W7fvo2MjAxlWXrx4kUYGBjA2dkZCoUChoaGiImJQbdu3d74Oq9XfS8oKCj0/Llz57B69Wrl7Wjx8fF4+vRp6X9pIhGZmpqiWbNmaNasWZFtGRkZiIqKUhah9+/fR/Xq1cv0enZ2dkhOToa9vX2ZzkNEpO5ycnKwfft2XLx4UewoREREJDKWnURaIicnB8+ePYNCoUBiYiJOnDiBb7/9Fs2bN4evr+9bjxs+fDjmzZuHUaNGYeHChXj+/DkmTJiAQYMGKW9hB4D8/Hx4eXnhm2++QVxcHGbPno1x48Ypy09fX1/4+vpCEAR06tQJ6enpuHjxInR0dDB+/HjY2NjA2NgYx44dg5OTE4yMjGBmZga5XI7t27ejdevWyMjIwJdffqksRokqExMTEzRu3BiNGzcGABw4cKDM56xSpQoyMjLKfB4iInW3f/9+NG7cGM7OzmJHISIiIpEVf5UEItJox48fR40aNVCrVi10794dBw4cwLx583DmzJkit67/W5UqVXDs2DG8fPkSrVq1Qv/+/dG2bVv4+/sX2q9z585wcXFB165dMXDgQHTr1g3Lli1Tbl+0aBHmz5+PFStWwMXFBT179kRwcDDq1KkDANDT08Pq1auxadMm2Nvbo3///gAAf39/pKeno3nz5vDw8ICXl9d75/kkqgxUsaJ7amoqzM3NVZCGiEi9bd68GWPHjhU7BhEREakBrsZORESkhm7fvg0DAwPUq1ev1OfYu3cvBgwYUKIV4ImINE1MTAyaN2+OR48ewdjYWOw4REREJDJe/RAREamhxo0b486dO6U+/vXCYCw6iaiy27JlCzw8PFh0EhEREQDO2UlERKS2jI2NCy38VRJnzpxBp06dyiEVEZH6KCgowJYtW7B//36xoxAREZGa4HAPIiIiNdW9e3fs3bsXJZ1xJjU1FUlJSbCysiqnZERE6uHEiROwsrJCs2bNxI5CREREaoJlJxERkZoyNDTEhx9+iF27dhW78ExNTcXvv/8Od3f3ck5HRCS+TZs2wdvbW+wYREREpEa4QBEREZGaS0lJweHDh9GiRQs0aNDgjfsoFAr8/fffSE5Ohru7u0pWcyciUmdJSUmQSqWIjo6Gubm52HGIiIhITbDsJCIi0hB37tzBgwcPYGRkBFtbW1SpUgWpqal4+vQpAKBTp068dZ2ItMaqVatw7do1BAYGih2FiIhIpZ49e4ZRo0bh/PnzyMzMLPG0Vv/m6emJpKQkHDp0SIUJ1RvLTiIiIg2Tm5uLpKQkZGZmwszMDJaWllx1nYi0iiAIaNy4MdauXYsuXbqIHYeIiKhEPD09sXXr1iLPt27dGhcvXoSvry+OHj2Kffv2wdTUFHZ2dqV+rdTUVAiCoFV3QXA1diIiIg1jYGAAe3t7sWMQEYnm8uXLyMnJQefOncWOQkREVCo9evQocneCgYEBACAiIgLNmzeHTCYr9fnz8/Ohq6sLMzOzMuXURBwGQkREREREGmXTpk3w8vLi/MRERKSxDA0NYWdnV+jHwsICTk5O2L9/P7Zt2waJRAJPT08AQGxsLAYOHAhTU1OYmppi0KBBePz4sfJ88+fPR6NGjRAQEABnZ2cYGhoiIyMDnp6e6NOnj3I/QRCwbNkyODs7w9jYGI0bN8b27dsr+tcvVxzZSUREREREGiM9PR1BQUG4e/eu2FGIiIhU7sqVKxg2bBgsLCzg5+cHY2NjCIKAAQMGwMjICCdPnoREIsHkyZMxYMAAXLlyRfnl38OHD7Fz507s2bMHBgYGMDIyKnL+uXPnIigoCD/99BPq1auHCxcuYNy4cahevTrc3Nwq+tctFyw7iYiIiIhIY+zZswcdO3bkdB5ERKTRjh49iqpVqxZ6btKkSfj+++9haGgIY2Nj5Vydf/31F27duoXIyEg4OTkBAHbu3AmpVIoTJ06gR48eAF7N7R8YGAhbW9s3vmZGRgZ+/PFH/PnnEHzM9wAAELRJREFUn+jYsSMAoE6dOrh8+TJ++uknlp1EREREREQVbdOmTfjyyy/FjkFERFQmnTp1woYNGwo997ZFhEJDQ2Fvb68sOgGgbt26sLe3x71795Rlp6Oj41uLTgC4d+8esrOz0bt370JTweTl5RU6t6Zj2UlERERERBohNDQUUVFR+Pjjj8WOQkREVCZVqlSBVCot1r6CILx1nup/P29iYvLO8ygUCgDAwYMHUatWrULb9PX1i5VFE7DsJCIiIiIijeDv74/Ro0dXqgsyIiKi92nYsCGePHmC6Oho5QjMqKgoxMXFoWHDhiU6j6GhIWJiYtCtW7dySis+lp1ERERERKT2cnNzsW3bNpw9e1bsKERERGWWk5ODZ8+eFXpOV1cX1tbWRfbt0aMHmjZtiuHDh2P16tUQBAFTpkyBq6triUpLU1NT+Pr6wtfXF4IgoFOnTkhPT8fFixeho6OD8ePHl/n3UgcsO4mIiIiISO0dOnQI9evXh1wuFzsKERFRmR0/fhw1atQo9JyDgwMeP35cZF+JRILff/8dU6dORZcuXQC8KkDXrFnz1tvb32bRokWwtbXFihUr8Pnnn6NatWpo1qxZpZoPWyIIgiB2CCIiIiIiondxc3PDkCFDMGrUKLGjEBERkRpj2UlERERERGrt8ePHaNKkCR4/fowqVaqIHYeIiIjUmI7YAYiIiIiIiN4lICAAQ4YMYdFJRERE78WRnUREREREpLYUCgWkUil2796NFi1aiB2HiIiI1BxHdhIREWmY+fPno1GjRmLHICKqEKdOnYKpqSmaN28udhQiIiLSACw7/6+9+4/Vuqz/B/68ETkczoFNzrAfgMQRISg4SSAWzjlxobDmPFGK0YaDTQJmbZoZmzSiWBlqLsBsUpow1MCs4a9Vp0z/MGQHiMLDDx2K6CjAgiO/jp3780f7su8JEPCc0+HcPB5/8b7u68frvv86e3Jd7wsA2smuXbvyta99LRdeeGHKysrSt2/fXHPNNXn66adbNe9tt92W559/vo2qBDizLV26NNOnTz/t22YBgLOTY+wA0A62b9+esWPHpmfPnvnOd76TmpqaNDc35/e//33uuuuuvPHGG8eMOXLkSLp169YB1QKcmfbu3Zvq6uq89tpr6d27d0eXAwB0AnZ2AkA7mDlzZorFYtauXZsvfelLGTJkSIYOHZrZs2dnw4YNSZJCoZDFixentrY2FRUVmTNnTv79739n2rRpGThwYMrLy3PRRRflrrvuSnNz89G5//sYe3Nzc+bPn5/+/funrKwsw4cPz69//eujn3/mM5/Jrbfe2qK+ffv2pby8PL/61a+SJMuWLcvo0aPTs2fPnH/++fniF7+YnTt3tudPBHBSy5cvzzXXXCPoBABOmbATANrY3r178+yzz2b27NmprKw85vPzzjvv6L/nzZuXCRMmZOPGjZk1a1aam5vTt2/fPP7443nllVfyve99LwsWLMjPf/7zE65333335Yc//GF+8IMfZOPGjbnuuutSW1ub9evXJ0mmTJmSRx99tEVgumrVqpSXl2fixIlJ/rOrdN68edmwYUNWr16d3bt3Z/LkyW31kwCctmKxmAcffDDTp0/v6FIAgE7EMXYAaGNr1qzJmDFj8sQTT+S66647Yb9CoZDZs2fnxz/+8fvOd8cdd2Tt2rX53e9+l+Q/OztXrlyZv/71r0mSvn375uabb87cuXOPjrniiivSr1+/LFu2LHv27MlHPvKRPPPMMxk3blyS5KqrrsqFF16YBx544LhrNjQ0ZOjQodmxY0f69et3Wt8foC38v53x27ZtS5cu9mgAAKfGXw0A0MZO5/8RR40adUzbT37yk4waNSp9+vRJZWVl7r333uO+4zP5z3H0t956K2PHjm3Rftlll2XTpk1JkqqqqowfPz7Lly9Pkrz99tv5wx/+kClTphztX19fn2uvvTYDBgxIz549j9Z1onUB2tvSpUtz0003CToBgNPiLwcAaGMXXXRRCoVCXnnllZP2raioaPH82GOP5etf/3qmTp2a5557LuvXr8/MmTNz5MiR953neLcU//9tU6ZMyapVq3Lo0KGsWLEi/fv3z2WXXZYkeffddzN+/Pj06NEjjzzySF5++eU8++yzSXLSdQHaw4EDB/LYY49l6tSpHV0KANDJCDsBoI317t0748ePz6JFi9LY2HjM5//85z9POPbFF1/MmDFjMnv27IwcOTKDBg3Kq6++esL+vXr1ykc/+tG8+OKLx8wzbNiwo8/XXnttkmT16tVZvnx5vvzlLx8NQxsaGrJ79+4sWLAgl19+eT7+8Y/n73//+2l9Z4C2tHLlylx66aXp379/R5cCAHQywk4AaAdLlixJsVjMqFGj8stf/jKbN29OQ0ND7r///owYMeKE4wYPHpz6+vo888wz2bp1a+bPn5/nn3/+fdf6xje+kYULF2bFihXZsmVL5s6dmxdeeKHFDezdu3dPbW1tvvvd76a+vr7FEfYLLrggZWVlWbRoUV577bU89dRTufPOO1v/IwB8QEuXLs20adM6ugwAoBPq2tEFAEApGjhwYOrr67NgwYJ885vfzM6dO1NVVZWampoTXgqUJDfffHPWr1+fG2+8McViMV/4whdy66235mc/+9kJx9xyyy3Zv39/br/99uzatStDhgzJqlWr8qlPfapFv6985St56KGHMnLkyAwdOvRoe58+ffLwww9nzpw5Wbx4cUaMGJF77rknV199det/CIDTtGXLljQ0NOTzn/98R5cCAHRCbmMHAADOGHfccUfee++9LFy4sKNLAQA6IWEnAABwRnjvvffSv3//1NXVtdiBDgBwqryzEwAAOCM8/fTTqa6uFnQCAB+YsBMAADgjPPjggy4mAgBaxTF2AACgw7311lv5xCc+kR07dqSysrKjywEAOik7OwEAgA738MMPZ9KkSYJOAKBV7OwEAAA6VLFYzODBg/PII4/k0ksv7ehyAIBOzM5OAACgQ/3pT39KWVlZxowZ09GlAACdXNeOLgAAADg7HD58OHV1dWlqajrads4552TZsmWZNm1aCoVCB1YHAJQCYScAANCu3nzzzbz00kspKyvLuHHj0qNHj6OfHTx4MFu3bk1VVVVef/31DBgwoAMrBQA6O+/sBAAA2k19fX327NmTq6666qQ7N+vq6tKzZ8+MHj36f1QdAFBqhJ0AAEC7+Mtf/pLGxsZ89rOfPeUxa9asSdeuXTNy5Mh2rAwAKFUuKAIAANrcoUOHsnnz5tMKOpPkkksuyeuvv5533323nSoDAEqZsBMAAGhzdXV1mThx4gcaO2HChNTV1bVxRQDA2UDYCQAAtLmDBw+2uIjodJSVleXw4cPxxi0A4HQJOwEAgDa1bdu2DB48uFVz1NTU5G9/+1sbVQQAnC2EnQAAQJt68803M2DAgFbNccEFF2Tnzp1tVBEAcLYQdgIAAG3q8OHDKSsra9Uc5557bpqamtqoIgDgbCHsBAAA2tR5552Xd955p1Vz7Nu3L7169WqjigCAs4WwEwAAaFPDhw9PfX19q+b485//nIsvvriNKgIAzhbCTgAAoE2Vl5fn4MGDrZqjsbExPXv2bKOKAICzhbATAABoczU1NVm3bt0HGrtp06YMHTq0jSsCAM4Gwk4AAKDNDRo0KA0NDWlsbDytcQcOHEh9fX2GDRvWTpUBAKVM2AkAALSL66+/PitXrsy//vWvU+q/f//+PP7447nhhhvauTIAoFQVisVisaOLAAAASlNzc3OefPLJlJeXZ9y4cenWrdsxfZqamlJXV5f9+/entrY2XbrYkwEAfDDCTgAAoN01Njamrq4uTU1NOffcc9OtW7ccOXIkTU1N6dq1a6688koXEgEArSbsBAAA/qeKxeLR0LNQKHR0OQBACRF2AgAAAAAlwctwAAAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAAACgJAg7AQAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAAACgJAg7AQAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAoFU+9rGPZeHChf+Ttf74xz+mUChk9+7d/5P1AIDOpVAsFosdXQQAAHBm2rVrV77//e9n9erV2bFjR3r16pVBgwZl8uTJuemmm1JZWZl//OMfqaioSI8ePdq9niNHjmTv3r350Ic+lEKh0O7rAQCdS9eOLgAAADgzbd++PWPHjk2vXr0yf/78jBgxIs3NzdmyZUt+8YtfpKqqKjfeeGP69OnT6rWOHDmSbt26nbRft27d8uEPf7jV6wEApckxdgAA4Li++tWvpkuXLlm7dm1uuOGGDBs2LJ/85CdTW1ubJ598MpMnT05y7DH2QqGQlStXtpjreH0WL16c2traVFRUZM6cOUmSp556KkOGDEn37t1z+eWX59FHH02hUMj27duTHHuM/aGHHkplZWWLtRx1B4Czl7ATAAA4xt69e/Pcc89l1qxZqaioOG6f1h4jnzdvXiZMmJCNGzdm1qxZeeONN1JbW5uJEydmw4YNueWWW3L77be3ag0A4Owi7AQAAI6xdevWFIvFDBkypEV7v379UllZmcrKysyYMaNVa1x//fWZPn16qqurM3DgwNx///2prq7O3XffnSFDhmTSpEmtXgMAOLsIOwEAgFP2wgsvZP369bnkkkty6NChVs01atSoFs8NDQ0ZPXp0ix2jY8aMadUaAMDZxQVFAADAMQYNGpRCoZCGhoYW7QMHDkyS9715vVAopFgstmhramo6pt9/H48vFounfTS+S5cup7QWAHB2sLMTAAA4RlVVVT73uc9l0aJFaWxsPK2xffr0ydtvv330edeuXS2eT2To0KF5+eWXW7StWbPmpGsdOHAg+/btO9q2fv3606oXACgdwk4AAOC4lixZkubm5nz605/OihUrsmnTpmzZsiUrVqzIhg0bcs455xx33JVXXpnFixdn7dq1WbduXaZOnZru3bufdL0ZM2bk1VdfzW233ZbNmzfniSeeyAMPPJDkxJchjRkzJhUVFfnWt76Vbdu2ZdWqVVmyZMkH/9IAQKcm7AQAAI6ruro669aty9VXX50777wzF198cUaOHJl77rknM2fOzI9+9KPjjrv77rtTXV2dK664IpMmTcr06dNz/vnnn3S9AQMGZNWqVfnNb36Tmpqa3Hvvvfn2t7+dJCcMS3v37p3ly5fnt7/9bYYPH56f/vSnmT9//gf/0gBAp1Yo/vcLbgAAAM4Q9913X+bOnZt33nknXbrYqwEAvD8XFAEAAGeMxYsXZ/To0enTp09eeumlzJ8/P1OnThV0AgCnRNgJAACcMbZt25YFCxZkz5496devX2bMmJG5c+d2dFkAQCfhGDsAAAAAUBKcBQEAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCf8HebVl/k0i9zQAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAABTsAAAPKCAYAAABbVI7QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xdc1eX///HnQYaCg8SFmCPcouLG1BQX5Uj9OHKVfBLtY0qOzJELREXNcFbmKC0zS1Nz5RZHoqklOTBH7r1yJvP8/uALv06gggJvODzut9u5+Tnv93Vd7+f7KPThxXVdb5PZbDYLAAAAAAAAALI4G6MDAAAAAAAAAEBaoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOAAAAAAAAAFaBYicAAAAAAAAAq0CxEwAAAAAAAIBVoNgJAAAAAAAAwCrYGh0AyGiRkZHavn27Hj16lHisRo0acnNzMzAVAAAAAAAAnpfJbDabjQ4BZISzZ89q3759cnBwUOPGjeXk5CRJMpvN2rNnjy5duqTChQurXr16MplMBqcFAAAAAABAalHsRLawefNm5cqVSy+//PITC5lXrlzR+vXr1aVLFzk4OGRgQgAAAAAAADwvip2wehs3btRLL72k0qVLp6h9dHS0vv76a7311luytWWnBwAAAAAAgKyCYiesWnh4uMxmszw9PVPV7++//9aaNWvUsWPHdEoGAAAAAACAtMbT2GHVTpw4kepCpyTlypVLefPm1b1799IhFQAAAAAAANIDxU5YrevXr6tgwYLP3L9x48baunVrGiYCAAAAAABAeqLYCav1888/q0GDBs/c387OTrGxsWmYCAAAAAAAAOmJYiesVo4cOWRj83z/xO3s7NIoDQAAAAAAANIbxU5YrbR49hbP7wIAAAAAAMg6KHbCaplMpkwxBgAAAAAAADIGxU5YLVtbWz18+PC5xoiKikqjNAAAAAAAAEhvFDthtRo3bqwtW7Y8c//bt2/L2dk5DRMBAAAAAAAgPVHshNVycHBQZGTkM++7uX37djVq1ChtQwEAAAAAACDdUOyEVatXr55++umnVPc7e/ascufOrRw5cqRDKgAAAAAAAKQHip2waq6uripevLi2bt2a4j4XLlzQgQMH1LRp03RMBgAAAAAAgLRmMj/rGl8gCzl+/Lj27NmjJk2ayM3NLdk20dHRWrhwoV544QW1b98+gxMCAAAAAADgedkaHQDICGXLltWCBQu0fv16tW/fXs7OzipSpIjs7e1169YtXbhwQba2ttqxY4dcXFwodgIAAAAAAGRBzOxEtnDjxg2VK1dOv/76q0qUKKG7d+/q+vXrioqK0gsvvKDChQvLZDLp2rVrqlSpkkJDQ1WpUiWjYwMAAAAAACAVKHYiWxg6dKju3bunTz/99KltZ8yYoR9//FGbN2+WyWTKgHQAAAAAAABICxQ7YfWuXr2qihUrKjw8XMWKFXtq+5iYGFWrVk1jxoxRhw4dMiAhAAAAAAAA0gLFTli9QYMGKTY2VtOnT09xn9DQUPXo0UMRERFydHRMx3QAAAAAAABIKxQ7YdUuXbokDw8PHTlyRK6urqnq+8Ybb6hcuXIaO3ZsOqUDAAAAAABAWqLYCavm7+8ve3t7ffzxx6nue/78eXl6emrfvn166aWX0iEdAAAAAAAA0hLFTlit8+fPq2rVqjp27JgKFSr0TGOMHz9e+/bt08qVK9M4HQAAAAAAANIaxU5YrT59+ihfvnyaOHHiM4/x6NEjeXh46JNPPpGPj08apgMAAAAAAEBao9gJq3TmzBnVqFFDf/zxhwoUKPBcY61evVqDBw/WoUOHZG9vn0YJAQAAAAAAkNYodgIAAAAAAACwCjZGBwAAAAAAAACAtECxEwAAAAAAAIBVoNgJAAAAAAAAwCpQ7AQAAAAAAABgFSh2Av+ybNkymUwmo2MAAAAAAAAglSh2ItO7ePGievfurWLFisne3l5ubm7q1auXLly4YHQ0AAAAAAAAZCIUO5GpnT59WjVr1tThw4e1cOFCnTx5UosWLdKRI0dUq1YtnTlzJtl+UVFRGRsUAAAAAAAAhqPYiUytb9++srGx0ebNm9WkSRMVL15c3t7e2rx5s2xsbNS3b19JUqNGjdSnTx8NHjxYBQsWVL169SRJISEhqlKlipycnOTm5iY/Pz/99ddfFtf46quvVKJECTk6OqpVq1a6evVqkhyrV69WjRo1lDNnTpUqVUojRoywKKguWrRItWrVUp48eVSoUCF17NhRFy9eTMdPBgAAAAAAAP9GsROZ1q1bt7R+/Xr17dtXjo6OFuccHR317rvv6qefftLt27clxRcczWazdu7cqa+++kqSZGNjo2nTpunIkSNavHixfvnlF/n7+yeOs3fvXvn6+qp37946ePCgWrdurdGjR1tca8OGDerWrZv69eunI0eO6IsvvtCyZcv04YcfJraJiopSYGCgwsPDtWbNGt24cUNdunRJr48GAAAAAAAAyTCZzWaz0SGA5Ozdu1deXl5avny52rVrl+T8ihUr9J///Ed79+7VkCFDdOvWLf3+++9PHHP9+vVq06aN/v77b9nY2Khr1666fv26Nm3alNjGz89P8+fPV8KXxiuvvKJmzZpp1KhRiW1Wrlyp7t276969e8k+zOjYsWOqUKGCzp8/r2LFij3rRwAAAAAAAIBUYGYnMr3HPRk9oRiZcL5GjRpJ2mzdulXNmjVTsWLFlCdPHv3nP/9RVFSUrly5IkmKiIhQ3bp1Lfr8+/2BAwc0fvx45c6dO/HVtWtXPXjwIHGcX3/9VW3atFGJEiWUJ08e1axZU5J07ty557hzAAAAAAAApAbFTmRaZcqUkclk0pEjR5I9HxERIZPJJHd3d0mSk5OTxfmzZ8+qZcuWqlChgpYuXaoDBw7oiy++kPT/H2CUkonNcXFxGjNmjA4ePJj4+v3333XixAkVLFhQDx48kI+PjxwdHfX1119r3759Wr9+vcV1AAAAAAAAkP5sjQ4APE7+/Pnl4+OjTz/9VAMHDrTYt/Phw4f65JNP9Nprryl//vzJ9t+/f7+ioqI0depU5ciRQ5K0Zs0aizYVK1bUnj17LI79+3316tV17NgxlS5dOtnrhIeH68aNG5owYYJKlSolSVq+fHnqbhYAAAAAAADPjZmdyNRmzZqlmJgYNW3aVFu3btX58+cVGhqqZs2ayWw2a9asWY/tW6ZMGcXFxWnatGk6ffq0vv32W02bNs2izXvvvafNmzcrODhYJ06c0Ny5c7VixQqLNqNHj9bixYs1evRoHT58WMeOHdOyZcs0ZMgQSVLx4sXl4OCgWbNm6c8//9TatWst9vcEAAAAAABAxqDYiUzN3d1d+/fvV6VKlfTmm2/qpZdeUteuXVWhQgXt27cvcSZlcqpUqaLp06crJCREFStW1Lx58zRlyhSLNl5eXpo/f74+++wzValSRcuXL1dAQIBFGx8fH61du1bbtm1T7dq1Vbt2bU2cOFHFixeXJBUsWFALFy7UypUrVbFiRQUGBiokJCTNPwsAAAAAAAA8GU9jBwAAAAAAAGAVmNkJAAAAAAAAwCpQ7AQAAAAAAABgFSh2AgAAAAAAALAKFDsBAAAAAAAAWAWKnQAAAAAAAACsAsVOZAlms1k1atTQ8uXLjY6SImazWc2aNdO0adOMjgIAAAAAAJBtUOxElrBq1SrFxcWpbdu2RkdJEZPJpBkzZmjcuHG6evWq0XEAAAAAAACyBZPZbDYbHQJ4kri4OFWrVk1BQUF6/fXXjY6TKu+//75u376tL774wugoAAAAAAAAVo+Zncj0li9fLnt7e7Vu3droKKk2ZswYrV+/Xnv37jU6CgAAAAAAgNWj2IlMzWw26/r16xo7dqxMJpPRcVItb968Cg4Olr+/v+Li4oyOAwAAAAAAYNVYxo5ML+GfaFYsdkrxy/Dr1asnPz8/9ezZ0+g4AAAAAAAAVotiJ5ABDhw4oJYtW+rYsWNydnY2Og4AAAAAAIBVotgJZJDevXsrV65cmj59utFRAAAAAAAArBLFTiCDXL9+XRUrVtS2bdvk4eFhdBwAAAAAAACrwwOKgAxSsGBBjRkzRv7+/uJ3DAAAAAAAAGmPYieQgf73v//p5s2bWrp0qdFRAAAAAAAArA7L2IEMtn37dr355puKiIiQk5OT0XEAAAAAAACsBjM7Yahbt24ZHSHDNWzYUPXq1VNwcLDRUQAAAAAAAKwKMzthmHnz5mnXrl3y9fWVp6ennJ2dE8+ZzWaZTKbHvs/qLly4oKpVq+qXX36Ru7u70XEAAAAAAACsAsVOGCI2Nlb58+dXVFSUnJ2d1a5dO3Xu3FlVq1ZVvnz5Ets9ePBAdnZ2sre3NzBt+ggODlZYWJhWrVpldBQAAAAAAACrwDJ2GGLZsmWqVKmSfvvtNwUGBmrdunXq2LGjRo0apZ07d+revXuSpGnTplntcu9BgwYpIiJCP/30k9FRAAAAAAAArAIzO2GItWvXasuWLRoyZIiKFCkiSZo1a5YmTZqkmJgYdenSRbVr11bXrl21adMmNWnSxODE6WPt2rUaOHCgDh06JAcHB6PjAAAAAAAAZGkUO5Hh7t+/r9y5c+vPP//USy+9pJiYGNna2iaenz59uqZOnapz586pQYMG2r59u4Fp01+rVq3UoEEDDR061OgoAAAAAAAAWRrFTmSoR48eqVWrVpo4caJq1qxp8eChfxY9jx07pooVK2rPnj2qXbu2kZHT3cmTJ+Xl5aXw8HC5ubkZHQcAAAAAACDLYs9OZKiRI0dq69atGj58uO7evWvxhPWEQmdsbKwmTJigMmXKWH2hU5JKly6t3r17a8iQIUZHAQAAAAAAyNIodiLD3LlzR9OnT9e8efN0+fJlde3aVZcvX5YUX+BMYDab1aBBAy1dutSoqBnuww8/1I4dO7Rz506jowAAAAAAAGRZLGNHhvHz89Off/6prVu3atGiRRowYIC6dOmimTNnJmkbGxurHDlyGJDSOEuWLNHEiRN14MCBbHfvAAAAAAAAaYFiJzLEzZs3VaRIEe3evVu1atWSFF/c8/f315tvvqnx48crV65ciouLk41N9pxwbDab5e3trU6dOundd981Og4AAAAAAECWQ7ETGaJPnz76448/tHXrVsXGxsrGxkYxMTGaMGGCpk2bpo8++kh+fn5GxzTc77//rqZNm+ro0aMqUKCA0XEAAAAAAACyFIqdyBBRUVG6d++eXFxckpwbMWKEZs6cqSlTpqh3794GpMtc/P39FR0drdmzZxsdBQAAAAAAIEuh2AnDJCxZv3nzpvz9/bVhwwZt2bJFnp6eRkcz1O3bt1WhQgWtW7dO1atXNzoOAAAAAABAlpE9N0dEppCwN6eLi4vmz58vT09POTo6GpzKeC+88IKCgoLk7+8vfhcBAAAAAACQcszshOESZnjevXtXefPmNTpOphAbGysvLy+99957evPNN42OAwAAAAAAkCVQ7ESGSng4kSSZTCaD02Rue/fu1X/+8x9FRERQBAYAAAAAAEgBlrEjQw0ePFiLFi2i0JkCderUUfPmzRUUFGR0FAAAAAAAgCyBmZ3IMJcuXZKHh4eOHj2qIkWKGB0nS7h69ao8PDy0c+dOlS9f3ug4AAAAAAAAmRrFTmQYf39/OTg4aMqUKUZHyVKmTp2q9evXa/369cyIBQAAAAAAeAKKncgQ58+fl6enpyIiIlSoUCGj42Qp0dHR8vT01Pjx49W2bVuj4wAAAABAhrt7966uXbum6Ohoo6MAWZqdnZ0KFSpk1c8GodiJDPG///1Pzs7OmjhxotFRsqQtW7aoV69eOnLkiHLlymV0HAAAAADIMHfv3tXVq1fl5uamXLlyseINeEZms1l///23Ll68qMKFC1ttwZNiJ9LdmTNnVKNGDR0/flwuLi5Gx8myOnTooCpVqmj06NFGRwEAAACADHPy5EkVLVpUjo6ORkcBrMLDhw916dIllS5d2ugo6YKnsSPdjRs3Tu+++y6Fzuf08ccfa8aMGTp79qzRUQAAAAAgw0RHR7PCDUhDuXLlsuotISh2Il2dOnVKK1eu1KBBg4yOkuWVKFFC7733nt5//32jowAAAABAhmLpOpB2rP3riWIn0tXYsWPl7++vF154wegoVuGDDz7Qr7/+qi1bthgdBQAAAAAAINOxNToArNcff/yhdevW6eTJk0ZHsRq5cuVSSEiI/P39FR4eLjs7O6MjAQAAAAAAZBrM7ES6GTt2rAYOHKh8+fIZHcWqtGnTRi+++KJmzZpldBQAAAAAwDPw9fVVsWLFkj0XGhoqk8mkzZs3Z3CqtJNwD6GhoUZHSeTr66uSJUsaHQMZgGIn0sXRo0e1efNm+fv7Gx3F6phMJk2fPl0TJkzQ1atXjY4DAAAAAACQaVDsRLoICAjQ+++/rzx58hgdxSqVL19evr6+GjZsmNFRAAAAAABIN7GxsYqJiTE6BrIQip1Ic7///rt27typvn37Gh3Fqo0aNUobN27Unj17jI4CAAAAAEgnJUuWVPfu3bVkyRJVqFBBTk5Oqlmzpnbt2pXiMebOnauqVasqZ86cKlCggHr27Klbt24lnp83b55MJpNWrlyZeCw2NlavvPKK3N3dde/ePUnxE5tMJpMOHTokb29vOTo6ytXVVaNHj1ZcXNwTM5jNZk2dOlXlypWTvb29XF1d1a9fP929e9einclk0ogRIzRx4kSVKlVK9vb2OnTokCTpxo0b6tOnj9zc3OTg4KDy5ctrzpw5Sa61ZcsWVa9eXTlz5pS7u7s+//zzFH9WyPp4QBHSXEBAgIYMGSInJyejo1i1vHnzauLEifL399fevXtlY8PvLgAAAADAGu3cuVN//PGHgoKClDNnTo0aNUqtWrXSmTNn5Ozs/MS+w4YN08cff6z33ntPH330kS5evKiRI0fq8OHD2r17t3LkyCE/Pz9t3LhRfn5+qlWrltzc3BQUFKSwsDDt2rUryarNtm3b6u2339bw4cO1YcMGBQUFycbGRgEBAY/NMWLECAUHB6tv375q3bq1jh49qlGjRik8PFzbt2+3+Jl2wYIFeumllzRlyhQ5OTmpaNGiunv3rurVq6e///5bAQEBKlWqlDZs2KA+ffooMjIycRu9iIgItWjRQjVr1tSSJUsUGRmpgIAA3b9/Xzly5Hj2vwRkGRQ7kaZ+/fVX7d27V998843RUbKF7t27a/bs2friiy/k5+dndBwAAAAAQDq4e/euDh48qBdeeEGSVKRIEdWqVUvr1q1T165dH9vvzJkz+uijjzRmzBiNHj068XjZsmVVv359rV69Wm3btpUkzZkzR1WrVlX37t0VEBCgcePGKSgoSHXq1Ekybq9evRK3VWvevLnu3r2rjz/+WAMGDEi2+Hrr1i2FhISoR48eiQ/b9fHxUcGCBfXmm29qzZo1ev311xPbm81mbdy4Ubly5Uo8FhQUpLNnz+rQoUMqU6aMJKlp06b666+/FBgYqD59+sjW1lbjxo1Tnjx5tHHjxsRJWC+//LLc3d1VtGjRlH3gyNKYCoY0NWbMGA0bNsziGxLSj8lk0syZMzVy5Ejdvn3b6DgAAAAAgHRQt27dxEKnJFWuXFmSdO7cOUnxxcGYmJjEV2xsrCRp06ZNiouLU7du3SzO16lTR3nz5tWOHTsSx3R2dtbixYu1c+dO+fj4qEGDBho6dGiyeTp16mTxvnPnzrp//74OHz6cbPs9e/YoMjJS3bt3T9LP1tZW27dvtzj+6quvJqkrrF+/XnXq1FGpUqUs7sXHx0c3b97U0aNHJUlhYWFq0aKFxWrTF198UfXq1Us2G6wPxU6kmV9++UUHDx5Ur169jI6SrVSvXl1t27bVmDFjjI4CAAAAAEgBW1vbxILkvyUct7X9/4tx8+fPb9HGwcFBkvTo0SNJ0sKFC2VnZ5f4cnd3lyRdu3ZNklS6dGmL83Z2drp7965u3rxpMa6Xl5fKlSunyMhI9e/f/7HbpRUuXDjZ9xcvXky2fcL+oK6urhbHbW1t5eLiYrF/aHLtEu5lx44dSe6jY8eOkpR4L5cvX06SL7nMsF4sY0eaGTNmjEaMGKGcOXMaHSXbGT9+vCpUqCA/Pz9VqVLF6DgAAABIQ7GxsTpw4ICuX78us9msF154QbVq1ZK9vb3R0QA8o0KFCunGjRuKiopK8rV86dIlSakrzrVu3Vr79u1LfJ9QDHVxcZEkbdy40WJmaIKE8wkCAwN14sQJValSRQMHDpS3t7fy5cuXpN/Vq1f10ksvWbyXJDc3t2TzJRRrr1y5okqVKiUej4mJ0c2bN5PkMJlMyWYtVKiQpk+fnuw1ypUrJym+UJqQ59+ZkT1Q7ESa2L17tyIiIvTjjz8aHSVbcnFxUUBAgPz9/RUaGprsfxgAAACQtVy/fl07d+6UyWRSnTp1VL16dZlMJt2+fVvr169XVFSU6tSpoxdffNHoqABSydvbW8HBwVq1apU6dOhgce6HH36Qq6trYvEuJVxcXJIUDCWpWbNmsrGx0blz59SsWbMnjrFz505NmDBBwcHBeuONN1S1alX16dNHixcvTtL2+++/T9yzU5KWLFmi3Llzy8PDI9mxvby85ODgoCVLlqhJkyaJx7/77jvFxMSoYcOGT73HV199VTNnzlTx4sVVqFChx7arW7eu1q1bpwcPHiQuZT9//rx+/vln9uzMJih2Ik2MHj1aI0eO5LfLBnrnnXc0Z84cfffdd+rcubPRcQAAAPActmzZIrPZrLZt2yZZRlqgQAG9/vrrMpvN2rNnjw4cOJD4gBEAWUPTpk3VrFkz+fr66tixY6pTp47u3bunJUuW6Mcff9SXX3752CXkqeHu7q6hQ4eqX79++uOPP9SwYUPlzJlT58+f16ZNm+Tn5ydvb2/dvn1b3bp1k7e3twYPHiyTyaQ5c+aoU6dO8vHxUY8ePSzGnTt3ruLi4lSrVi1t2LBB8+bNU0BAwGOfDJ8/f34NGjRIwcHBcnJyUosWLRQREaGRI0eqfv36atmy5VPvZeDAgfruu+/UoEEDDRw4UOXKldODBw907Ngx7dy5M3Hy1ciRI7V06VI1b95cH3zwgaKiojRmzBiWsWcjFDvx3LZv367Tp08n+eaHjJUjRw7NnDlTXbt2VatWrZQ7d26jIwEAAOAZrF+/XqVLl1bp0qWf2M5kMqlu3bq6cuWKli5dmrhvHYDMz2QyadWqVRo3bpy++uorBQUFyd7eXp6enlq5cqXatGmTZteaMGGCKlSooE8++USffPKJTCaTXnzxRTVp0iTxqea9e/fW33//ra+++ipxpWDHjh3Vs2dP9evXT/Xq1bP4nvTjjz/K399fQUFBypcvn0aOHKlRo0Y9Mcf48eNVsGBBzZ49W59++qlcXFz01ltvKTg4OEWF3Xz58mn37t0aO3asJk2apIsXL8rZ2VnlypVT+/btE9tVqFBB69at0wcffKA33nhDbm5uGjp0qMLCwhQaGvoMnyCyGpPZbDYbHQJZl9lsVqNGjfT2229T7MwkunXrphIlSmjChAlGRwEAAEAq7d+/Xzlz5nzsUtDHOXfunE6ePKnGjRunUzLAOBEREapQoYLRMSApICBAgYGBio6OtniAErIea/664mnseC7btm3T5cuX1a1bN6Oj4P9MnjxZc+bM0cmTJ42OAgAAgFQ6c+ZMqgudklS8eHHdvn1bzGUBAGR3FDvxzMxms0aNGqUxY8bwG51MxM3NTR988IEGDBhgdBQAAACkwqlTp+Tu7v7M/b28vLRnz540TAQAQNZDsRPPbOPGjbp9+zYPw8mEBgwYoOPHj2vt2rVGRwEAAEAKhYeHq1q1as/c383NTZcuXUrDRABgKSAgQGazmQlPyNQoduKZmM1mjR49WgEBAcqRI4fRcfAvDg4Omj59ugYMGKDIyEij4wAAACAF7OzsnnsMe3v7NEgCAEDWRbETz2TdunV6+PChOnToYHQUPMZrr72mChUqKCQkxOgoAAAASIG02G+TPTsBANkdxU6kWsKszsDAQNnY8E8oM5s6daqmTJmiCxcuGB0FAAAAT2EymTLFGAAAZGVUqpBqP/74o8xms9q1a2d0FDyFu7u7+vTpow8++MDoKAAAAHiK6Ojo556ZGRUVlUZpAADImih2IlXi4uI0ZswYBQYG8lvjLGL48OH6+eeftX37dqOjAAAA4Alq1Kih/fv3P3P/M2fOqFixYmmYCACArIdiJ1Jl+fLlsre3V6tWrYyOghRycnLSlClT5O/vr5iYGKPjAAAA4DFKlCihs2fPPnP/Tz/9VJMnT1ZEREQapgKsjNksXd8tHZsmHQqK//P67vjjAKwCxU6kWGxsrMaMGaOxY8cyqzOL6dixowoUKKDZs2cbHQUAAABP4O7uroMHD6a6359//qmmTZuqTp06atiwoXx9fXX69Ol0SAhkUXHR0onZ0ip3aVtz6eBQ6dCY+D+3NY8/fmJ2fDsAWRrFTqTY999/r3z58unVV181OgpSyWQyacaMGQoMDNT169eNjgMAAIDHqFatmq5fv65jx46luM+FCxcUHh6u5s2ba8iQITpx4oRKlCihmjV6rEGuAAAgAElEQVRrql+/frp8+XI6JgaygOj70pbG0q/vSw9OSzEPpLgoSeb4P2MexB//9X1pS5P49ulswYIFMplMyb42b96c7tf/p+XLl2vatGlJjm/evFkmk0m7du3K0DzA86LYiRSJiYlRQEAAszqzMA8PD3Xt2lUjRowwOgoAAACeoFmzZrp69arWrVv3xG2I4uLiFBoaqvDwcIuHh+bLl0+BgYE6duyYHBwcVKlSJQ0dOlQ3b97MiPhA5hIXLYW+Jt3cJ8U+fHLb2IfSzV+k0BYZNsNz6dKlCgsLs3jVrl07Q66d4HHFztq1ayssLExVq1bN0DzA87I1OgAyl0uXLum3335TbGysTCaTihcvrqpVq+rbb79V4cKF1aRJE6Mj4jkEBgaqfPny6t27t2rWrGl0HAAAADxGw4YNdefOHa1evVqxsbHy9PRU4cKFZWNjoxs3bujAgQMym81q0KCBChUqlOwYBQsW1Mcff6yBAwcqKChI5cqVU//+/TVgwADlyZMng+8IMMip+dKtX6W4yJS1j4uUbh2QTn0hlXknfbNJ8vT0VOnSpVPUNjIyUg4ODumc6P/LmzevvLy80mQss9ms6Oho2dvbp8l4wJMwsxMym83atWuXfvjhB509e1Y+Pj56/fXX1apVK+XOnVtLly7V7Nmz9eGHHzKrM4tzdnbW+PHj5e/vr7i4OKPjAAAA4Any5cundu3aqX379nr06JH279+vsLAw3bp1S23atFH79u0fW+j8p2LFiunzzz/Xnj179Mcff6h06dKaOnWqHj16lAF3ARjIbJaOTn76jM5/i30Y38/AhxYlLCFfuXKl3n77bRUoUEBubm6J59etW6c6deooV65ccnZ2Vrt27XTixAmLMerXr69GjRpp48aNqlatmhwdHeXh4aFVq1Yltunevbu++eYbnT17NnEZfULx9XHL2JctW6Y6derI0dFRzs7O6tSpky5cuGDRplixYvL19dXcuXNVrlw52dvba8OGDWn9MQHJotiZzd27d08LFixQ6dKl1b59e9WtW1e2tvETfk0mk9zd3dWxY0dt2bJF9+/f19GjRw1OjOf13//+V7Gxsfr666+NjgIAAIAUMJlM8vDwkLe3t5o2bapq1aopR44cqR6ndOnSWrRokTZv3qzt27erTJkymjt3rqKjeSALrNSNMCny2rP1jbwa3z+dxcbGKiYmJvEVGxtrcb5v376ytbXVN998o/nz50uS1qxZo1atWumFF17Q999/r08++UTh4eGqX7++rly5YtH/+PHjGjRokAYPHqzly5ercOHCat++feIDzAIDA+Xj46MiRYokLqNftmzZY/POmjVLnTp1UuXKlfXDDz9o9uzZCg8PV6NGjXT/vuVep5s2bUp8dsT69etVqVKltPjIgKdiGXs29uDBAy1fvlw9evSQjc2T6945c+ZUhw4dFBoaqri4OHl4eGRQSqQ1GxsbzZw5U+3atVPbtm2VL18+oyMBAAAgA1WuXFkrV67U3r17NWLECE2aNEljx45V586dn/pzAZBpHBgg3T745DYPL0gxqZzVmSDmoRT2luRY7PFtXvCUaiTd6zI1ypcvb/G+Xr16FjMpX375Zc2ZM8eizciRI1W2bFmtXbs28RcfderUUfny5RUSEqLJkycntr1x44Z27dqll156SZJUtWpVFS1aVEuXLtWQIUPk7u6uAgUKyMHB4alL1u/evavhw4fLz8/PIlOtWrVUvnx5LViwQP369Us8fufOHf32228pmoEOpCX+S5aNrVixQt27d0/V/6Fp1KiRTp06pb/++isdkyG91alTR6+++qrGjh1rdBQAAAAYpE6dOtq8ebPmzJmjGTNmyNPTU6tWrZLZwKW7QJoyx0p61n/P5v/rn75WrFihffv2Jb4SZm8m+OfDx6T4gmN4eLg6d+5sMcO7dOnS8vLy0vbt2y3aly9fPrHQKUmurq4qUKCAzp07l+qsP//8s+7fv69u3bpZzEYtUaKEypQpox07dli0f/nllyl0whDM7MymTpw4ocqVKz/T8pdWrVppzZo1atOmTTokQ0YJDg6Wh4eH/Pz8VKFCBaPjAAAAwCCNGzdWWFiY1qxZoxEjRmjChAmaMGGCGjdubHQ04PFSMqPy2DTp4FApLir149s4SOUGSOX7p75vKnh4eDzxAUWurq4W72/dupXscUkqUqSIwsPDLY7lz58/STsHB4dn2rP32rX4LQEaNWqUoqzJZQQyAsXObOr3339X+/btn6lvjhw5FBsbK7PZzAOLsrDChQtrxIgReu+997Rx40b+LgEAALIxk8mk1q1bq2XLlvruu+/0zjvvqESJEho/frzq1KljdDzg2bjUlmzsnrHYaSu51Er7TKn075/TEoqX/96bM+GYi4tLumVJGPvrr79OsvxekvLkyWPxnp8xYRSWsWdD0dHRsre3f64x6tWrp927d6dRIhilb9++unTpklasWGF0FAAAAGQCNjY26tKli44ePao33nhDHTp0UJs2bXTo0CGjowGpV6Cu5PCMy6hzFo7vn8nkzZtXnp6e+v777xUXF5d4/M8//9SePXvUsGHDVI/p4OCgv//++6nt6tevLycnJ506dUo1a9ZM8ipXrlyqrw2kB4qd2dD169efezp54cKFE6fPI+uys7PTzJkzNWjQID18+IwbdwMAAMDq2NnZqVevXjpx4oS8vb3VrFkzdevWTSdPnjQ6GpByJpNUcYiUwzF1/XI4ShWGxPfPhIKCghQREaHWrVtrzZo1Wrx4sZo3by4XFxcNHDgw1eNVrFhR165d05w5c7Rv3z4dPnw42XbOzs6aNGmSxo0bpz59+mjVqlUKDQ3VN998Iz8/P3333XfPe2tAmqDYmQ3dv39fTk5Ozz0OG5dbh8aNG6tWrVoWT+wDAAAAJClnzpwaMGCATpw4oQoVKsjLy0vvvPOOLly4YHQ0IGXce0r5q8fvwZkSNg5S/hqS+9vpm+s5tGrVSqtXr9aNGzfUoUMH9enTR5UrV9auXbtUpEiRVI/Xu3dvderUSUOHDlXt2rXVtm3bx7bt27evVqxYoYiICHXr1k0tWrRQQECAzGazqlat+jy3BaQZk5mKVbZz5coVnTt3TrVr136ucVavXq3WrVunUSoY6dy5c6pWrZoOHDigkiVLGh0HAAAAmdStW7c0efJkzZ07Vz169NDw4cNVsGBBo2PBykVERDzfQ1Wj70uhLaRbB6TYJ6xoy+EYX+hstE6yy/3s1wOygOf+usrEmNmZDRUoUECXL19+rjHOnDmjokWLplEiGK148eIaOHCgBg0aZHQUAAAAZGL58+fXxIkTdfjwYUVFRal8+fIaPXq07ty5Y3Q04PHscktNtkjVQySnlyRbp/+b6WmK/9PWScr9Uvz5JlsodAJZHMXObMjW1lbR0dHPtQz9wIEDql69ehqmgtEGDx6s8PBwbdq0yegoAAAAyORcXV01a9YsHThwQOfPn1eZMmU0efJk9oFH5mVjJ5V5R3r9pOS9UfKcJFUZG/+n9yap9cn48zZ2RicF8JwodmZTXl5e2rNnzzP1jYyMlL29vUyZdLNmPJucOXNq6tSpeu+99xQVFWV0HAAAAGQBJUuW1Jdffqnt27dr3759Kl26tD755BP+/yQyL5NJKviyVL6/5DEy/s+CdTPtw4gApB7FzmyqWLFiOn36tB49epTqvitXrlSTJk3SIRWM1rp1a5UsWVIzZ840OgoAAACykAoVKmjp0qVavXq11qxZo3LlymnhwoWKjY01OhoAIJuh2JmNdezYUYsXL1ZkZGSK+6xevVpeXl5ydHRMx2Qwislk0vTp0xUcHPzc+7oCAAAg+6lRo4Z++uknLVy4UPPmzVPlypX1ww8/PNcWWgAApAbFzmzMzs5Ob775ppYtW6bff//9iW2vXr2qRYsWydPTUyVKlMighDBC2bJl1bNnTw0bNszoKAAAAFmWr6+vTCaTxo0bZ3E8NDRUJpNJN27cMChZvAULFih37vR7CMsrr7yiHTt2KCQkROPHj1etWrW0YcMGip4AgHRHsTObs7OzU7du3RQbG6sWLVpo1apVOn36tG7duqULFy5o586d+uGHH3T8+HF169ZNL774otGRkQFGjhypLVu2aPfu3UZHAQAAyLJy5sypyZMn6/r160ZHMYTJZNKrr76q/fv3a9iwYRowYIAaNWqkXbt2GR0NAGDFKHZCkvTbb7/Jzs5OTZs21f3793XkyBFdu3ZN5cuXV/v27dWgQQMeSJSN5MmTR5MmTZK/vz/7LAEAADwjb29vlSxZUkFBQY9tc/ToUbVs2VJ58uRRoUKF1KVLF125ciXx/L59+9S8eXMVKFBAefPmVf369RUWFmYxhslk0meffaY2bdrI0dFRZcuW1bZt23ThwgX5+PjIyclJnp6e+vXXXyXFzy7973//qwcPHshkMslkMikgICBdPgNJsrGxUYcOHXTo0CH997//Vffu3dWiRYvEPAAApCWKnZAkzZ8/Xz179pSjo6MqV66sBg0aqHr16ipYsKDR0WCQrl27ytHRUfPnzzc6CgAAQJZkY2OjiRMnavbs2Tp16lSS85cvX9Yrr7wiDw8P/fLLL9q8ebPu37+v119/XXFxcZKke/fu6c0339TOnTv1yy+/yNPTUy1atEiyDH7cuHHq3LmzwsPDVbNmTXXp0kU9e/bUu+++q99++01FixaVr6+vJOnll1/WtGnT5OjoqMuXL+vy5csaPHhwun8etra28vX11R9//KGWLVuqVatW6tSpk44dO5bu1wYSmc3S7t3StGlSUFD8n7t3xx8HYBVMZjZNyfYiIiLUuHFjnTt3TnZ2dkbHQSZy8OBB+fj4KCIiQvnz5zc6DgAAQJbh6+urGzduaM2aNfL29lbhwoW1ZMkShYaGytvbW9evX9eMGTP0888/a8uWLYn9bt++rfz582vv3r2qXbt2knHNZrOKFi2qjz76SN27d5cUP7Nz2LBhCg4OliQdPnxYlStX1scff6xBgwZJksV1CxQooAULFqhfv366f/9+BnwayXvw4IFmzZqlKVOmqHXr1hozZgzPB0CyIiIiVKFChecbJDpamj9fmjxZunYt/n10tGRnF/8qVEgaMkTq2TP+PWDl0uTrKpNiZif05Zdf6q233qLQiSQ8PT3Vvn17jR492ugoAAAAWdbkyZO1dOlS7d+/3+L4gQMHtGPHDuXOnTvxlbBHfsJM0GvXrumdd95R2bJllS9fPuXJk0fXrl3TuXPnLMaqUqVK4v8uXLiwJKly5cpJjl27di3tb/AZOTk5aejQoTpx4oTc3NxUvXp1+fv7WyzjB9LE/ftS48bS++9Lp09LDx5IUVHxszmjouLfnz4df75Jk/j2GSAsLEydOnVS0aJFZW9vLxcXFzVr1kwLFy7MstuJrVy5UiEhIUmOJzycLTQ0NE2uk7AFR3KvlStXpsk1/i2t7yG9xgTFzmwvOjpaX331ld5++22joyCTCgoK0tKlSxUeHm50FAAAgCypVq1aat++vYYOHWpxPC4uTi1bttTBgwctXidOnFCrVq0kST169NC+ffs0depU7d69WwcPHlSxYsUUFRVlMdY/Jy4k7LWf3LGE5fGZibOzs4KCghQRESE7OztVqlRJw4cP161bt4yOBmsQHS299pq0b5/08OGT2z58KP3yi9SiRXy/dDRt2jTVq1dPt27d0qRJk7R582Z98cUXKlu2rPr06aM1a9ak6/XTy+OKnenB19dXYWFhSV4NGzbMkOunherVqyssLEzVq1c3OopVsTU6AIy1du1alSlTRuXKlTM6CjIpFxcXBQYGyt/fX9u3b+dBVQAAAM9gwoQJqlixotavX594rHr16vr+++9VokSJx66y2rVrl2bMmKGWLVtKkq5evarLly8/dx57e/tMN3OsUKFCCgkJ0cCBAxUUFKSyZctq4MCB6t+/v3Lnzm10PGRV8+dLv/4qRUamrH1kpHTggPTFF9I776RLpB07dmjQoEHq16+fZsyYYXGuTZs2GjRokB48ePDc14mOjpatrW2yP8NFRkbKwcHhua9hJDc3N3l5eRkd45nExsbKbDYrb968WfYeMjNmdmZz8+fPZ1YnnqpXr166f/++lixZYnQUAACALKl06dLq3bu3pk+fnnisb9++unPnjt544w3t3btXf/75pzZv3qzevXvr3r17kqSyZctq0aJFOnr0qPbt26fOnTvL3t7+ufOULFlSjx490qZNm3Tjxg09fNqMtwz04osvas6cOQoLC9ORI0dUunRpTZ8+XY8ePTI6GrIaszl+j87U/vt++DC+Xzo94mTixInKnz+/Jk+enOx5d3f3xK0pAgICki1W+vr6qmTJkonvz5w5I5PJpE8//VRDhgxR0aJF5eDgoL/++ksLFiyQyWTSjh071LFjRzk7O6tOnTqJfbdv364mTZooT548cnJyko+Pjw4fPmxxvUaNGql+/fravHmzqlevLkdHR3l4eFgsGff19dXChQt18eLFxCXl/8z4T/369VPhwoUV/a8ZtPfv31eePHk0fPjwJ36GKTFv3rwky9pjY2P1yiuvyN3dPfH7bMJnfOjQIXl7e8vR0VGurq4aPXr0U2fDm81mTZ06VeXKlZO9vb1cXV3Vr18/3b1716KdyWTSiBEjNHHiRJUqVUr29vY6dOhQssvYU/JZJ/j2229Vvnx55cyZU5UrV9aqVavUqFEjNWrU6Nk/OCtAsTMbu3Tpknbt2qWOHTsaHQWZXI4cOTRz5kx98MEHhm5iDwAAkJWNHj1atrb/f3Fd0aJF9fPPP8vGxkavvvqqKlWqpL59+8rBwSFxxtUXX3yh+/fvq0aNGurcubPefvvtxxYPUuPll1/W//73P3Xp0kUFCxZ8bNHFSGXKlNHixYu1YcMGbdmyRWXLltW8efMUExNjdDRkFWFh8Q8jehZXr8b3T2OxsbEKDQ1V8+bNlTNnzjQff/z48Tp+/LjmzJmjFStWWFyjW7duKlWqlJYtW6aJEydKil/t2aRJE+XOnVuLFi3S4sWLde/ePTVo0EDnz5+3GPvUqVPq37+/Bg0apOXLl8vV1VUdOnTQyZMnJUmjRo1SixYtVLBgwcQl5StWrEg257vvvqtr164lOf/NN9/owYMH6tWr11Pv1Ww2KyYmJskrgZ+fnzp27Cg/Pz9dvHhRUvw2bWFhYVq8eLHy5MljMV7btm3VtGlTrVy5Ul27dlVQUJDGjh37xAwjRozQoEGD1KxZM61evVpDhgzRggUL1LJlyySF0gULFmjt2rWaMmWK1q5dq6JFiz523Kd91pK0adMmdevWTeXLl9cPP/ygwYMHa8CAATp+/PhTPzurZ0a2FRwcbPbz8zM6BrKQ7t27m4cNG2Z0DAAAAGRDYWFhZm9vb3OZMmXM3377rTk2NtboSMggR48eTXqwf3+zuWHDJ7/c3c1mk8lsjp+jmbqXyRTf/0nj9++f6nu5cuWKWVKKf64aM2aMObnSTY8ePcwlSpRIfH/69GmzJHO1atXMcXFxFm2//PJLsyTzgAEDkozj7u5ubty4scWxO3fumF1cXMz9/3F/DRs2NNva2pqPHz+eeOzq1atmGxsb8/jx4y1yubm5JbnOtm3bzJLM27Ztsxjz39euVq2a2cfHJ0n/f5P02Nf169cT292+fdtcvHhxc6NGjcyhoaHmHDlymCdMmGAxVsJnHBwcbHHcz8/PnDt3bvPt27eTvYebN2+aHRwczD169LDo9/XXX5slmX/88UeLvK6uruaHDx+m6HNJyWddt25dc6VKlSz+vg8cOGCWZG7YsOFTP8Nkv66sBDM7s7Fhw4Zp7ty5RsdAFjJ58mTNnTtXJ06cMDoKAAAAshkvLy9t3bpVn332maZOnapq1appzZo1MqfTUmNYgdjYZ1+KbjbH989i2rZt+9jnLLRr187i/YkTJ3Tq1Cl169bNYmako6Oj6tatqx07dli0L1OmjMqUKZP4vlChQipUqJDOnTv3TFnfffddbdu2LfHny3379um3337TOyncK/Xtt9/Wvn37krycnZ0T2zg7O2vx4sXauXOnfHx81KBBgyQPi0vQqVMni/edO3fW/fv3kyzpT7Bnzx5FRkaqe/fuSfrZ2tpq+/btFsdfffVV5cqVK0X39rTPOjY2Vvv371f79u0t/r6rV6+uUqVKpega1owHFAFIMVdXVw0dOlQDBgzQ2rVrjY4DAACAbKhJkybas2ePVq1apeHDh2v8+PGaMGGCvL29U9Q/Li5ONjbM+8nypk1LWZuhQ6WoqNSP7+AgDRgg9e+f+r5P4OLioly5cuns2bNpOm4CV1fXFJ+79n9L/Hv27KmePXsmaV+8eHGL9/nz50/SxsHB4Zn3023Xrp2KFCmizz//XFOmTNHs2bNVtGhRtW7dOkX9XV1dVbNmzae28/LyUrly5XT06FH179//sV//hQsXTvZ9whL4f7t161Zijn+ytbWVi4tL4vl/5k2pp33WN27cUHR0tAoVKpSk3b/vIzviOzyAVOnfv79OnTqlNWvWGB0FAAAA2ZTJZFKbNm108OBB9evXT35+furSpcsTZ3leuXJFU6dOla+vr0aPHp3kwSiwQrVrS3Z2z9bX1laqVStt8yi+ENaoUSNt2rRJkSl4QnzCnptR/yrY3rx5M9n2j5vVmdw5FxcXSVJwcHCyMyRXr1791HzPw87OTn5+flqwYIGuXbumJUuWqGfPnhZ7G6eFwMBAnThxQlWqVNHAgQN1586dZNtdvXo12fdubm7Jtk8oSF65csXieExMjG7evJn4+SZ40t9NahUoUEB2dnaJBet/+vd9ZEcUOwGkir29vaZPn64BAwbwREwAAAAYKkeOHOrWrZuOHTumkJCQx7aLi4vTu+++q2nTpqlIkSLaunWr3NzctHTpUkliKby1qltXSmbmW4oULhzfPx0MGzZMN2/e1AcffJDs+dOnT+v333+XJJUoUUKSLJZS//XXX9q9e/dz5yhXrpxKliypI0eOqGbNmkleCU+ETw0HBwf9/fffKW7/zjvv6M6dO+rYsaMiIyNT9GCi1Ni5c6cmTJig8ePHa/Xq1frrr7/Up0+fZNt+//33Fu+XLFmi3Llzy8PDI9n2Xl5ecnBw0JIlSyyOf/fdd4qJiVHDhg3T5iaSkSNHDtWsWVM//PCDxfevAwcO6PTp0+l23ayCZewAUs3Hx0ceHh4KCQnRhx9+aHQcAAAAZHN2dnZPXCJ66dIlHT16VCNHjkwspkyaNEmzZs1Sy5Yt5ejomFFRkZFMJmnIEOn996WHD1Pez9Exvl8azsT7p1deeUUhISEaNGiQIiIi5Ovrq+LFi+v27dvasmWL5s2bp8WLF6tKlSp67bXXlC9fPvXq1UuBgYGKjIzU5MmTlTt37ufOYTKZ9Mknn6hNmzaKiopSp06dVKBAAV29elW7d+9W8eLFNWjQoFSNWbFiRd26dUufffaZatasqZw5c6py5cqPbe/m5qbWrVtrxYoVat26tV588cUUX+vixYvas2dPkuMlSpSQq6urbt++rW7dusnb21uDBw+WyWTSnDlz1KlTJ/n4+KhHjx4W/ebOnau4uDjVqlVLGzZs0Lx58xQQEGCxB+g/5c+fX4MGDVJwcLCcnJzUokULRUREaOTIkapfv75atmyZ4nt5FoGBgWrevLnatWun3r1768aNGwoICFCRIkWy/VYd2fvu8VS+vr5q1arVc4/j4eGhgICA5w+ETCMkJEQhISE6f/680VEAAACAJ0rY2++fRYvixYvr1KlTCg8PlxS/9HT+/PlGRUR66dlTql49fg/OlHBwkGrUkN5+O11jDRgwQLt27ZKzs7MGDx6sxo0by9fXVxEREfr8888T9610dnbWmjVrZGNjo06dOmn48OHy9/dP8R61T9OiRQvt2LFDDx48kJ+fn3x8fDRkyBBduXJFdZ9hZqufn586d+6sDz/8ULVr107R/psdO3aUpBQ/mCjBggULVLdu3SSvb775RpLUu3dv/f333/rqq68Sl5B37NhRPXv2VL9+/XTy5EmL8X788Udt2rRJr7/+uhYtWqSRI0dq1KhRT8wwfvx4hYSE6KefflKrVq00ceJEvfXWW1q7dm26FxybNWumb775RhEREWrXrp0mTZqkjz/+WEWKFFG+fPnS9dqZncnMfP0sLTQ09Inf5Bo1aqRt27Y98/h37tyR2Wx+7G8yUsrD4/+xd99RUV3v18D30JsNsSAIRpAiiNhFbGAhNqyUBAtqopGIGlRUYhQLqFHsmq9KswPW2INgB4wNOwYlNkZEiQ0QYRjm/cOf84bYEbgMsz9rzVLunHvvHpYIPPOcc2wxaNAgFjwrmRkzZiA1NfWttn0iIiIioorizz//xNKlS5Gamork5GSMHTsW7u7umDp1KlRUVLBu3TpYWloiOTkZrVu3Rr169RAUFPTWDssknJSUFFhbW5f8Ajk5QM+ewPnzH+7w1NF5Xeg8cAAohc5J+jReXl5ISEjA33//LUhHYmBgIGbNmgWJRFLq64WWt/T0dJibm+Pnn3/+aKH2i7+uKjB2diq4du3aISMj463HmjVrIBKJ4OPjU6LrFhYWQiaToVq1al9c6KTKa+rUqUhKSsKxY8eEjkJERERE9Ja8vDw4OzujXr16WLp0Kfbs2YM//vgDkyZNQteuXTFv3jxYWloCAJo1awaJRILJkyfDz88PZmZmOHDggMCvgEqFnh4QHw8sXgw0bAjo6r7u4BSJXv+pq/v6+OLFr8ex0FkuTp8+jf/973+Ijo6Gn5+f0k+9/lx5eXkYM2YMduzYgePHjyMiIgLdunWDjo4OvvvuO6HjCYr/khSchoYG6tatW+zx9OlTTJ48GQEBAfJ2cLFYDE9PT9SoUQM1atRAr169cPPmTfl1AgMDYWtri8jISJiZmUFTUxO5ublvTWPv3C9L3UQAACAASURBVLkzfHx8EBAQAAMDA9SuXRuTJk1CUVGRfMyjR4/Qt29faGtrw9TUFOHh4eX3CaFypaOjg5CQEPj6+qKwsFDoOERERERExWzduhW2trYICAhAhw4d0Lt3b6xatQoPHjzA6NGj4ejoCOD1BkVvHmPHjkV6ejr69OmD3r1746effsLLz1nvkSomdXVg9Gjg1i0gNhZYsACYPfv1n4cPvz4+enTJd2+nz+bg4IDJkydj2LBhJW7UUmaqqqp4+PAhxo4di27dusHPzw+NGjXCiRMnPriGsTJgsbOSefbsGfr164dOnTphzpw5AICXL1/CyckJWlpaOH78OJKSkmBoaIiuXbsW+6Z9+/ZtbNmyBdu2bcOlS5egpaX1znts3rwZampqSExMxMqVK7F06VJER0fLn/f29satW7cQFxeH3bt3Y8OGDbhz506Zvm4SzsCBA1G7dm2sXr1a6ChERERERMVIJBJkZGTgxYsX8mNGRkaoXr06zp8/Lz8mEokgEonkuxrHx8fj1q1bsLS0hJOTEzcwqkxEIqBdO2D8eGD69Nd/OjiU2WZE9H4ymQzZ2dkICwsTdPp4YGAgZDKZwk1h19DQwK5du5CRkYGCggI8ffoUe/bsee/u8cqExc5KpKioCN9++y1UVVWxadMm+QK8UVFRkMlkiIiIgJ2dHaysrLBmzRrk5ORg37598vMLCgqwceNGNG/eHLa2tu/9Qm/cuDFmz54NCwsLuLu7w8nJCfHx8QCA1NRUHDx4EGvXroWjoyOaNWuG9evXIy8vr+w/ASQIkUiE5cuXY86cOXj06JHQcYiIiIiI5Dp16oS6deti4cKFEIvFuHr1KrZu3Yr09HQ0atQIwOuCy5uZalKpFCdPnsTQoUPx/Plz7NixA66urkK+BCIi+kyKVbamDwoICEBSUhLOnDmDqlWryo+fP38et2/fRpUqVYqNf/nyJdLS0uQfGxsbo06dOh+9j52dXbGP69WrJy9ypaSkQEVFBa1bt5Y/b2pqinr16pXoNZFisLGxweDBgxEQEIDQ0FCh4xARERERAQCsrKwQERGBMWPGoGXLlqhZsyZevXoFf39/WFpaoqioCCoqKvJGkSVLlmDFihXo2LEjlixZAhMTE8hkMvnzRERU8bHYWUlER0dj0aJF2L9/v/wdyjeKiopgb2//zh2z9fX15X/X1dX9pHup/2cNE5FIJH8n9M20D1I+gYGBsLKywtmzZ9GqVSuh4xARERERAXj9xvyJEydw8eJF3Lt3Dy1atEDt2rUBvN6YVUNDA0+ePEFERARmz54Nb29vLFy4ENra2gDAQicRkYJhsbMSuHjxIkaMGIH58+fDxcXlreebN2+OrVu3wsDAoMx3Vre2tkZRURHOnj2Ldu3aAQDu3buHBw8elOl9SXjVqlVDcHAwxo4di6SkJO6kR0REREQVir29Pezt7QFA3qyhoaEBAJgwYQL279+P6dOnY9y4cdDW1pZ3fRIRkWLh/9wKLisrC/369UPnzp0xePBgPHz48K2Hl5cX6tSpg759++L48eO4ffs2Tpw4gYkTJxbbkb00WFpa4uuvv8bo0aORlJSEixcvwtvbW/6uKFVuw4YNg0gkwoULF4SOQkRERET0Xm+KmHfv3kXHjh2xa9cuzJ49G1OnTpVvRvTfQidnsRERKQZ2diq4/fv34+7du7h79y4MDQ3fOUYmk+HEiROYOnUq3Nzc8Pz5c9SrVw9OTk6oUaNGqWeKjIzE999/D2dnZxgYGGDmzJncuEZJqKio4OTJkwq3ix0RERERKSdTU1OMGTMGJiYmcHR0BIAPdnT6+vpi7NixsLS0LM+YVIpkMhnS09MhFouRn58PTU1NGBkZwdjYmEsWEFUSIhnfniIiIiIiIiL6oMLCQixcuBCLFy+Gq6srZsyYAVNTU6FjKYWUlBRYW1t/0TWkUimSk5ORkJCA3NxcFBUVQSqVQlVVFSoqKtDV1YWjoyOaNWsGVVXVUkpOVHGVxtdVRcVp7EQkmPz8fKEjEBERERF9EjU1NUybNg03b96EoaEhmjdvjvHjxyMzM1PoaPQRBQUF2LBhA2JjY/Hs2TNIJBJIpVIAr4ugEokEz549Q2xsLDZs2ICCgoIyzxQZGQmRSPTOR1ntteHt7Y0GDRqUybVLSiQSITAwUOgYVMmw2ElE5a6oqAjx8fFYvnw5Hj58KHQcIiIiIqJPVr16dcydOxfXr1+HSCRC48aN8fPPP+Pp06dCR6N3kEql2Lx5M8RiMSQSyQfHSiQSiMVibN68WV4MLWvbtm1DUlJSsUdcXFy53JuosmKxk4jKnYqKCl6+fIljx45hwoQJQschIiIiIvpsderUwdKlS5GcnIzMzExYWFhg3rx5yM3NFToa/UtycjIyMjI+uXgplUqRkZGB5OTkMk72mr29Pdq2bVvs0bJly3K595fgLD2qyFjsJKJy9WZKSJ8+fTBw4EDExMTg8OHDAqciIiIiIioZExMThIaG4tSpU7h06RLMzc2xfPlyFoMqAJlMhoSEhI92dP6XRCJBQkIChNzipKioCJ07d0aDBg3w/Plz+fErV65AW1sbkydPlh9r0KABBg8ejHXr1sHc3BxaWlpo3rw5jh49+tH7ZGRkYOjQoTAwMICmpibs7OywadOmYmPeTLk/ceIE3NzcUL16dbRp00b+/PHjx9GlSxdUqVIFurq6cHFxwdWrV4tdQyqVYvr06TA0NISOjg46d+6Ma9eulfTTQ/RBLHYSUbkoLCwEAGhoaKCwsBATJ06En58fHB0dP/uHDyIiIiKiisbS0hJRUVE4ePAgDh8+DAsLC4SHh8t/Dqbyl56eXuJO29zcXKSnp5dyordJpVIUFhYWexQVFUFFRQWbNm1CdnY2Ro8eDQDIy8uDp6cnbGxsEBQUVOw6x48fx+LFixEUFISoqChoamqiR48e+Ouvv95779zcXHTq1AkHDx5EcHAwdu/ejSZNmmDIkCFYu3btW+O9vLzw1VdfYfv27Zg/fz4AYP/+/ejSpQv09PSwadMmbNmyBdnZ2ejQoQPu378vPzcwMBDBwcHw8vLC7t270b17d7i6upbGp5DoLWpCB6CyER0djXXr1nGtDxJUWloaioqK0KhRI6ipvf7vZv369QgICICWlhZ++eUXuLq6wszMTOCkRERERESlw97eHnv37kViYiICAgKwYMECzJkzB4MGDYKKCvuNSsuhQ4c+uv7/ixcvStxYIZFIsGvXLlStWvW9Y+rWrYuvv/66RNd/w8rK6q1jvXr1wr59+2BsbIzQ0FAMGDAALi4uSEpKwt27d3HhwgVoaGgUOyczMxMJCQkwMTEBAHTp0gWmpqaYO3cuNm7c+M57R0RE4ObNmzh69Cg6d+4MAOjRowcyMzMxffp0jBw5stjO9IMGDcKvv/5a7Brjx49Hp06d8Pvvv8uPOTk5oWHDhggJCcHSpUvx9OlTLFmyBKNGjcKiRYsAAN27d4eqqiqmTp36+Z80oo9gsbOSCgsLw8iRI4WOQUpu8+bN2Lp1K1JSUpCcnAxfX19cvXoV3377LYYNG4amTZtCS0tL6JhERERERKWuXbt2OHr0KOLi4hAQEIDg4GAEBQWhZ8+eEIlEQsdTCkVFRYKe/yl27doFY2PjYsf+vRt7//79MXr0aIwZMwb5+fkIDw+HhYXFW9dp27atvNAJAFWqVEGvXr2QlJT03nufOHECRkZG8kLnG4MHD8bw4cNx/fp1NGnSpFiWf7t58ybS0tIQEBBQrINZR0cHDg4OOHHiBIDXU+9zc3Ph7u5e7HxPT08WO6lMsNhZCb18+RIFBQXo16+f0FFIyU2bNg0hISFo0aIFbt68iXbt2mHDhg1o37499PX1i4199uwZLl26hE6dOgmUloiIiIiodIlEInTr1g1du3bF7t27MWXKFAQHByM4OJg/936hT+moPH36NOLi4kq0s7qqqqp8w6CyZGtrC3Nz8w+OGTZsGNasWYPatWvj22+/feeYOnXqvPOYWCx+73WfPHkCQ0PDt47XrVtX/vy//Xfso0ePAAAjR458Z7PVm+JrRkbGOzO+KzNRaWAPfSWkra2No0ePQltbW+gopOTU1dWxevVqJCcnY8qUKVizZg1cXV3fKnQeOnQIP/30EwYMGID4+HiB0hIRERERlQ2RSIT+/fvj0qVLGDNmDIYPHw4XFxecO3dO6GiVmpGRUYmXDlBRUYGRkVEpJ/p8L1++xIgRI2Bra4vnz5+/txMyMzPzncc+9Br09fXfuRTAm2M1a9Ysdvy/Hclvnp83bx7Onj371mPv3r0A/n+R9L8Z35WZqDSw2FkJiUQiTougCsPLywuNGzdGamoqTE1NAUC+q+HDhw8xe/Zs/Pzzz/jnn39ga2uLoUOHChmXiIiIiKjMqKqqYvDgwbhx4wb69++Pvn37YuDAgbh+/brQ0SolY2Nj6OrqluhcPT29t6aXC2H8+PEQi8X4/fff8euvv2LZsmU4dOjQW+NOnz5dbEOg7Oxs7N+/Hw4ODu+9dqdOnZCeno6EhIRix7ds2YLatWvD2tr6g9ksLS3RoEEDXLt2DS1btnzrYWdnBwCws7ODrq4uYmJiip0fFRX10ddPVBKcxk5EZS48PByjR4+GWCyGkZGRvBhfVFQEqVSK1NRUREZGokmTJrC0tERgYCACAwOFDU1EREREVEY0NDTwww8/YNiwYVi1ahWcnJzg4uKCwMBANGzYUOh4lYZIJIKjoyNiY2M/a6MidXV1tGvXrlyaiC5evIisrKy3jrds2RK///47QkNDsXHjRjRs2BDjxo1DbGwsvL29cfnyZdSuXVs+vk6dOujevTsCAwOhqamJBQsWIDc3F7/88st77+3t7Y1ly5ZhwIABCAoKgrGxMTZv3ozDhw9jzZo1xTYneheRSIRVq1ahb9++KCgogLu7OwwMDJCZmYnExESYmJjAz88P1atXx08//YSgoCBUqVIF3bt3x9mzZxEWFlbyTxzRB7Czk4jKXOvWrbF9+3ZUrVpVvkg1ANSrVw9jx45Fq1atEB0dDQBYtGgRgoKC8PTpU6HiEhERERGVC21tbUyaNAk3b96EmZkZWrVqBR8fHzx48EDoaJVGs2bNYGho+NHC3RuqqqowNDREs2bNyjjZa25ubnBwcHjrkZGRge+//x5eXl4YPHiwfHxERAREIhG8vb3lM+aA112aEydOREBAADw8PPDq1SscPHjwnZsZvaGrq4vjx4+je/fumDp1Kvr27YtLly5h48aNGDVq1Cfl79mzJ06cOIHc3Fx89913cHFxgb+/Px4+fFisqzQwMBABAQHYuHEjXF1dERsbK5/mTlTaRLJ/f3UQEZURmUyG7777DlKpFKGhoVBVVZW/UxoVFYWQkBAcOHAAtWrVgp+fH3r27ImuXbsKnJqIiIiIqPxkZWVhwYIFCA8Px8iRIzFlypS31k1URikpKR+dUv0hBQUF2Lx5MzIyMj7Y4amurg5DQ0N4eXlBQ0OjxPcrbw0aNED79u2xadMmoaOQAvnSr6uKjJ2dCkomk4F1alIkIpEILVu2xJkzZ1BYWAiRSCTfFfHRo0eQyWTQ09MDAISEhLDQSURERERKx8DAAAsXLsTly5eRnZ0NS0tLzJo1Cy9evBA6mkLT0NDA0KFD0b17d1SvXh3q6uryTk9VVVWoq6ujRo0a6N69O4YOHapQhU4iehs7OysJmUwGkUgk/5OoojI3N8eQIUPg6+sLfX19iMVi9OnTB/r6+jh06BDU1LiUMBERERERAKSlpSEwMBCxsbHw9/eHj48PtLW1hY5V7kqzA00mkyE9PR1isRgFBQXQ0NCAkZERjI2NFfZ3aXZ2UklU5s5OFjsV0Lx58/Ds2TMsWLBA6ChEny0hIQFjxoyBrq4u6tevj9OnT8PIyAiRkZGwtLSUj5NKpUhMTESdOnU+uM4MEREREVFld/XqVcyYMQNnzpzBL7/8ghEjRkBdXV3oWOWmMhdliIRSmb+uOI1dAa1cuRLm5ubyj/fv34/ffvsNS5YswdGjR1FYWChgOqIPc3R0RGhoKBwcHPD48WOMGDECixcvhoWFRbGlGW7fvo3Nmzdj6tSpKCgoEDAxEREREZGwbG1tsXPnTuzatQs7duyAtbU1Nm3aJF8WioiI/j92diqYpKQkdOnSBU+ePIGamhomTZqEDRs2QFtbGwYGBlBTU8PMmTPh6uoqdFSiT1JUVAQVlXe/73Ls2DH4+fmhZcuWWLt2bTknIyIiIiKqmI4ePYqff/4ZL168wNy5c9G3b1+FnYL9KSpzBxqRUCrz1xU7OxXMwoUL4enpCS0tLcTExODo0aNYtWoVxGIxNm/ejEaNGsHLywsPHz4UOirRBxUVFQGAvND53/ddpFIpHj58iNu3b2Pv3r1clJ2IiIiI6P84OTkhISEBCxYsQGBgINq2bYu4uDhuYktEBBY7FU5iYiIuXbqEPXv2YMWKFRg6dCi++eYbAK+nNsyfPx9fffUVLly4IHBSog97U+TMzMwEgGLvRJ8/fx59+vSBl5cXPDw8cO7cOVStWlWQnEREREREFZFIJEKvXr1w4cIF+Pn5wcfHB126dEFSUpLQ0YiIBMVipwLJycmBn58fLC0t4e/vj1u3bsHe3l7+vFQqRd26daGiosJ1O0kh3LlzBz4+Prh58yYAQCwWY+LEiXB0dMTz589x6tQp/O9//4ORkZHASYmIiIiIKiYVFRV4eHjg+vXr8mYBV1dXXL58WehoRESC4JqdCuT69eto3LgxxGIxzpw5gzt37qBbt26wtbWVjzlx4gR69uyJnJwcAZMSfbrWrVvDwMAAgwYNQmBgICQSCebOnYuRI0cKHY2IiIiISOG8evUKa9euRXBwMJycnDBr1ixYWFgIHeuLlObagjKZDEnpSTgjPoPs/GxU0ayC1kat4WDsUKnXPSX6r8q8ZieLnQri/v37aNWqFVasWAE3NzcAgEQiAQCoq6sDAC5evIjAwEBUr14dkZGRQkUl+ixpaWnyndj9/Pwwffp0VK9eXehYREREREQKLScnB8uXL8eSJUvQr18/zJgxA/Xr1xc6VomURlFGIpUgLDkMvyb8ike5jyApkkAilUBdVR3qKuqorVsb/o7+GNlsJNRV1UspOVHFVZmLnZzGriAWLlyIR48ewdvbG3PmzEF2djbU1dWL7WJ948YNiEQiTJs2TcCkRJ/HzMwM06ZNg4mJCYKDg1noJCIiIiIqBXp6eggICEBqaipq1aoFe3t7/PTTT3j06JHQ0cpdTkEOnDc4Y2LsRNx+dhu5klwUSAsggwwF0gLkSnJx+9ltTIydiC4buiCnoGxnSkZGRkIkEr3zERcXBwCIi4uDSCTCqVOnyizH4MGDYW5u/tFxDx8+hK+vLywsLKCtrQ0DAwO0aNEC48ePlzdhfapbt25BJBJh06ZNn533yJEjCAwMLNVrUuXEYqeCiIiIQHx8PAIDA7Fu3Tps2LABAKCqqiof4+npiR07dsDS0lKomEQlMnfuXKSnp8v/XRMRERERUemoUaMGgoODce3aNUilUlhbW+OXX37Bs2fPhI5WLiRSCXps7oGz4rN4KXn5wbEvJS9xRnwGPTf3hET6eUW8kti2bRuSkpKKPVq3bg3g9XJfSUlJaNq0aZnn+JBnz56hdevWOHjwIPz8/HDgwAGsWbMGPXr0wJ49e5Cfn19uWY4cOYJZs2a9dbx+/fpISkrC119/XW5ZqGJTEzoAfdzOnTuhq6sLJycnNG3aFJmZmRg3bhwuX76MOXPmoHbt2igsLIRIJCpW/CRSJMeOHUN+fj5kMhnXyiEiIiIiKmV169bF8uXLMXHiRMyePRsWFhbw8/ODr68vdHV1hY5XZsKSw3Ah4wLypZ9WlMuX5uN8xnmEJ4djdMvRZZrN3t7+vZ2VVatWRdu2bcv0/p8iJiYG9+/fx9WrV2FjYyM/PnDgQMyZM6dC/O6mqalZIT5XVHGws1MBLF68GN7e3gAAfX19LFq0CKtXr8Yff/yBhQsXAgDU1NRY6CSF1r59e3Tp0qVCfLMkIiIiIqqsTE1NERYWhhMnTiA5ORmNGjXCypUry7VDr7zIZDL8mvDrRzs6/+ul5CV+TfgVQm5x8q5p7O3bt0fnzp0RGxuLZs2aQUdHB7a2ttizZ0+xc1NTUzF48GA0aNAA2traMDMzw48//liibt4nT54AeF0s/6///u5WUFCAgIAAmJqaQkNDAw0aNMCMGTM+OtW9ffv26Nq161vHjY2N8d133wEApk+fjqCgIPl9RSIR1NRe9++9bxr7+vXrYWdnB01NTdSqVQvDhg1DZmbmW/fw9vbG5s2bYWVlBV1dXbRq1QqJiYkfzEwVG4udFdyLFy+QlJSEUaNGAQCkUikAYOTIkfD398eqVavQp08f3LlzR8CUREREREREpEisrKwQHR2N/fv34+DBg7C0tERkZCQKCws/+RovXrzA7t27sWfPHvlj586dSEtLK8Pkny4pPQmPcku2RmlmbiaS0pNKOVFxUqkUhYWF8seb3/c/JDU1FX5+fpg0aRJ27tyJOnXqYODAgbh9+7Z8jFgshqmpKZYtW4Y//vgDP//8M/744w/07t37szO+mVbv7u6O2NhY5Obmvnfs4MGDsXDhQgwfPhz79u3D0KFDERwcjJEjR372ff/rhx9+kDeBvZnyn5CQ8N7xq1evhre3N5o0aYLdu3cjKCgI+/fvR+fOnfHyZfHi99GjR7F8+XIEBQUhKioKBQUF6N27N168ePHFuUkYnMZewVWtWhWPHz+Gvr4+gP+/Rqeamhp8fHxQq1Yt+Pv7Y9y4cYiKioKOjo6QcYlKzZt3UdnpSURERERUdpo1a4b9+/cjISEBAQEBWLBgAWbPno2BAwcW2xD33+7cuYNz586hSpUq6NWrF9TVi+9efuHCBWzfvh1GRkZwcHAok9wTDk3AxYcXPzgm/UX6Z3d1vvFS8hJDdw2FcVXj946xr2uPpV8vLdH1gdcF539zdHT86IZEWVlZOHXqFBo2bAgAaNq0KerVq4dt27bB398fAODk5AQnJyf5Oe3atUPDhg3h5OSEK1euoEmTJp+c0dnZGTNmzEBwcDCOHDkCVVVVNGvWDH369MGECRNQtWpVAMClS5ewbds2zJkzB9OnTwcAdO/eHSoqKpg1axamTp2Kxo0bf/J9/8vY2BhGRkYA8NEp64WFhZg5cya6dOmCzZs3y49bWFjAyckJkZGR8PHxkR/PyclBbGwsqlWrBgCoVasWHBwccOjQIbi7u5c4MwmHnZ0K4E2h813c3NywePFiZGVlsdBJlUpRURFatWqFI0eOCB2FiIiIiKjSc3R0xLFjx7Bs2TIsWLAALVu2xMGDB9+ayn3hwgWkpaVh0KBBcHFxeavQCQDNmzfHoEGDYGBggF27dpXXS3iLtEgKGUo2FV0GGaRFH++0/BK7du3C2bNn5Y+wsLCPnmNlZSUvdAKAoaEhDAwMcO/ePfmx/Px8zJ07F1ZWVtDW1oa6urq8+PnXX399ds5Zs2bh7t27WLduHQYPHozHjx9j5syZsLW1xePHjwEAx48fB/C6u/Pf3nz85vnycP36dWRlZb2VpXPnzjAyMnori6Ojo7zQCUBeDP7355QUCzs7K4H+/fujc+fOQscgKlWqqqoICAjAuHHjkJyc/M4fooiIiIiIqPSIRCJ0794d3bp1w65duzBx4kQEBwcjODgYHTp0wLVr15Cbm4suXbp80vUaNWoEXV1d7N27F3369CnVrJ/SUbn09FJMiZuCAmnBZ19fU1UTE9pOwPi240sS75PY2tq+d4Oi93lXM5SmpiZevXol/9jf3x+//fYbAgMD0bZtW1SpUgV3796Fm5tbsXGfo169evjuu+/ka2guW7YMEyZMQEhICObPny9f29PQ0LDYeW/W+nzzfHl4X5Y3ef6b5b+fU01NTQAo8eeKhMfOzkqiRo0aQkcgKnX9+/eHoaEhVq9eLXQUIiIiIiKlIRKJMGDAAFy5cgXff/89hg4diq+//hqnT59Ghw4dPuta9erVg7GxMVJSUsoo7fu1NmoNdZWSNU2oqaihlVGrUk5UPqKiojBixAgEBATA2dkZrVq1Kta5WBrGjx+PqlWr4vr16wD+f8Hw4cOHxca9+bhmzZrvvZaWlhYKCooXpGUyGZ4+fVqibO/L8ubYh7JQ5cBip4IRcjc4ovImEomwfPlyzJ07F48elWxhcSIiIiIiKhlVVVUMHToUf/31F5o3b46ePXuW6DrNmjWTF8XKk4OxA2rr1i7RuXX06sDBuGzWGy1reXl5b82Mi4iIKNG1MjIy3rlxUnp6OrKzs+Xdk506dQLwutD6b2/WzOzYseN772Fqaoq//vqr2OZYR48efWsjoTcdl3l5eR/M3LhxYxgYGLyV5fjx4xCLxfKsVHmx2KlAbt68iZCQEBY8SalYW1tj6NChmDZtmtBRiIiIiIiUkoaGBlq0aPHOacGfSldXFzk5OaWY6uNEIhH8Hf2ho/55+1voqOvAv52/wm6W6uLigvDwcPz222+IjY3F999/jzNnzpToWuvXr0fDhg0xa9YsHDx4EMeOHcPatWvh7OwMLS0t+UY/TZs2hZubG3755RfMmTMHhw8fRmBgIObOnYshQ4Z8cHMiT09PPHr0CCNGjEBcXBzWrFmDH3/8EVWqVCk27s01Fi1ahD///BPnz59/5/XU1NQwa9YsHDp0CMOGDcOhQ4cQGhoKNzc3WFlZYdiwYSX6XJDiYLFTgYSHhyMjI0Nh/8MlKqmZM2fi4MGDJf4GTUREREREJZebmyvfdbuknJ2dceLEiVJK9OlGNhuJ5obNoamq+UnjNVU10cKwBUY0G1HGycrO6tWr0atXL0ybNg0eHh549epVsV3JP0efPn3Qv39/7Nq1C15eXujWrRsCAwNhb2+PxMRENG3aVD522Ua2lgAAIABJREFU06ZNmDRpEkJDQ9GzZ09ERkZi2rRpH914qVu3bli1ahUSExPRp08fbNy4EVu2bHnr31zfvn0xevRoLF++HA4ODmjTps17r+nj44PIyEgkJyejb9++mDp1Knr06IFjx45xc2clIJKxTVAhFBYWwsTEBHFxcR98R4Soslq/fj1WrVqF06dPQ0WF79MQEREREZWXu3fv4vnz57Czs/ui65R0o6KUlBRYW1uX+L45BTnoubknzmecx0vJy/eO01HXQQvDFjjgdQB6Gnolvh+RIvjSr6uKjBUDBXHo0CGYmpqy0ElKa8iQIVBVVUVkZKTQUYiIiIiIlEphYSFUVVW/+DpC9Vrpaeghfmg8FndfjIbVG0JXXReaqpoQQQRNVU3oquuiYY2GWNx9MeKHxrPQSaTg1IQOQJ8mLCwMI0eOFDoGkWBUVFSwcuVK9O7dGwMGDED16tWFjkREREREpBT09fVx5cqVL7qG0JNK1VXVMbrlaIxqMQpJ6Uk4Kz6L7IJsVNGogtZGrdHWuC2XjCOqJDiNXQFkZmbC0tIS9+7d++J1UogU3ahRo6Cjo4OlS5cKHYWIiIiISGns2LEDAwcOLPH5iYmJaNCgAerVq/fZ51bm6bZEQqnMX1ecxq4ANm7ciP79+7PQSQQgKCgIW7ZswdWrV4WOQkRERESkNLS0tJCXl1fi8x88eFCiQicR0edisbOCk8lknMJO9C+1atXCjBkzMG7cOMGnwhARERERKYsuXbogLi6uROeKxWIYGhqWciIiondjsbOCS0pKQlFRERwdHYWOQlRh/PDDD8jKysL27duFjkJEREREpBS0tLSgp6eH1NTUzzrv1atXiIuLQ7t27b7o/mx0ICo9lf3ricXOCi4sLAwjRozgQslE/6KmpoYVK1Zg4sSJyM3NFToOEREREZFScHJyQlpaGlJSUj5pfHZ2NrZu3Ypvv/32i36nVVdX/6Ip9ERUXF5eHtTV1YWOUWa4QVEFlpOTg/r16yMlJQV169YVOg5RhfPNN9/AzMwMc+fOFToKEREREZHSSExMhFgsRps2bWBiYvLW87m5uVi9ejWMjIzg6ekJFZUv67N68eIFMjMzYWRkBG1tbTYDEZWQTCZDXl4exGIx6tSpU2n3hlETOgC9X0xMDDp27MhCJ9F7LFy4EE2bNsXw4cNhZmYmdBwiIiIiIqXQrl07yGQynD17FmfOnIGGhob8ucLCQmhra+PGjRt4+vTpFxc6AcgLMg8ePIBEIvni6xEpM3V19Upd6ATY2VmhOTo6YsqUKXB1dRU6ClGFNW/ePCQlJWHPnj1CRyEiIiIiov9z7949NGvWDCkpKahdu7bQcYhIibDYWUGlpKTA2dkZ9+7dq9TrKBB9qfz8fNja2mL58uXo0aOH0HGIiIiIiOj/+Pr6QkNDAyEhIUJHISIlwmJnBeXv7w+RSIQFCxYIHYWowtu/fz9++uknXLlyBZqamkLHISIiIiIiABkZGbCxscHVq1dRr149oeMQkZJgsbMCkkgkqF+/Po4fPw5LS0uh4xAphN69e6NDhw6YMmWK0FGIiIiIiOj/TJo0Ca9evcLKlSuFjkJESoLFzgpo9+7dCAkJwcmTJ4WOQqQwbt26hbZt2+LSpUswMjISOg4REREREQF4/PgxrKyscOHCBZiamgodh4iUwJdvi0alLiwsDCNGjBA6BpFCMTc3x6hRo+Dv7y90FCIiIiIi+j+1atXCDz/8gLlz5wodhYiUBDs7K5gHDx7AxsYG9+/fh56entBxiBRKTk4OrK2tsWXLFnTo0EHoOEREREREBODJkyewsLDA6dOnYW5uLnQcIqrk2NlZwWzYsAGDBg1ioZOoBPT09LBw4UL4+vpCKpUKHYeIiIiIiADo6+tj3LhxmD17ttBRiEgJsLOzApHJZLC0tMSGDRvQtm1boeMQKSSZTAYnJye4u7vDx8dH6DhEREREREREVI7Y2VmBnDx5EmpqamjTpo3QUYgUlkgkwvLlyxEYGIisrCyh4xARERERERFROWKxswIJDw/HyJEjIRKJhI5CpNDs7Ozg4eGB6dOnCx2FiIiIiIiIiMoRp7FXEC9evICJiQlSU1NRu3ZtoeMQKbynT5/C2toaBw4cQPPmzYWOQ0RERERERETlgJ2dFURUVBS6dOnCQidRKalRowbmzJkDX19f8D0dIiIiIiIiIuXAYmcFER4ejhEjRggdg6hSGTFiBPLz87Fp0yahoxARERERKb3AwEDY2toKHYOIKjlOY68Arl27hu7du+Pu3btQU1MTOg5RpXL69GkMHDgQKSkpqFq1qtBxiIiIiIgUire3N7KysrBv374vvlZOTg7y8/NRs2bNUkhGRPRu7OysAMLCwuDt7c1CJ1EZaNu2Lbp164Y5c+YIHYWIiIiISKnp6emx0ElEZY7FToEVFBRg06ZNGD58uNBRiCqt+fPnIyIiAjdu3BA6ChERERGRwjp79iy6d+8OAwMDVK1aFe3bt0dSUlKxMWvWrIGFhQW0tLRQq1YtuLi4oLCwEACnsRNR+WCxU2B79+5F48aNYW5uLnQUokqrbt26CAgIwPjx47lZERERERFRCWVnZ2PIkCE4efIkzpw5A3t7e/Ts2RNZWVkAgHPnzuHHH3/EzJkz8ddffyEuLg5ff/21wKmJSNmw2CmwsLAwjBw5UugYRJWer68v7t+/j99//13oKERERERECsnZ2RlDhgyBtbU1rKyssGLFCmhpaeHQoUMAgHv37kFXVxeurq4wNTVF06ZN8dNPP3HJNiIqVyx2Cig9PV2+eQoRlS11dXUsX74cfn5+yMvLEzoOEREREZHCefToEUaPHg0LCwtUq1YNVapUwaNHj3Dv3j0AQLdu3WBqaoqvvvoKXl5eWL9+PbKzswVOTUTKhsVOAUVGRsLd3R06OjpCRyFSCl27dkXz5s2xcOFCoaMQERERESmcYcOG4ezZs1iyZAkSExNx8eJFGBsbo6CgAABQpUoVXLhwATExMTAxMcG8efNgZWWFBw8eCJyciJQJi53lRCKR4NGjR3jw4AHy8vJQVFSEiIgITmEnKmchISFYvnw57t69K3QUIiIiIiKFcurUKfj6+qJXr16wsbFBlSpVkJGRUWyMmpoanJ2dMW/ePFy+fBm5ubnYt2/fJ12/qKioLGITkZLhwhllSCaT4fTp0xCLxdDW1kbNmjWhpqaGq1ev4vbt26hbty7s7OyEjkmkVExNTTFu3DhMnDgR27dvFzoOEREREZHCsLCwwKZNm9CmTRvk5ubC398fGhoa8uf37duHtLQ0dOzYEfr6+jh69Ciys7NhbW39Sdfftm0bPDw8yio+ESkJFjvLyM2bN3Hu3Dm0b98eDg4O7xzz7bff4uDBg9DX10fHjh3LOSGR8po8eTJsbGwQHx+PLl26CB2HiIiIiEghhIeHY9SoUWjRogXq1auHwMBAPH78WP589erVsXv3bsyePRsvX76EmZkZQkND0aFDh0+6/syZMzFw4EBuaEREX0Qkk8lkQoeobK5evYrMzMxPLqLcuHED9+7dQ/fu3cs4GRG9sXv3bgQEBODSpUtQV1cXOg4RERERkdLr2LEjvvvuOwwdOlToKESkwLhmZykTi8W4f//+Z3WLWVlZwcjICElJSWWYjIj+rW/fvqhfvz5WrlwpdBQiIiIiIgIwd+5cBAYGQiKRCB2FiBQYi52l7PTp0+jRo8dnn2djY4MHDx6AjbZE5UMkEmHZsmUIDg5GZmam0HGIiIiIiJRex44dYWZmhoiICKGjEJECY7GzFOXm5kJbW7vE57ds2RJnz54txURE9CFWVlbw9vbG1KlThY5CREREREQA5syZg7lz5+LVq1dCRyEiBcViZyk6cuTIF212Ympqirt375ZiIiL6mF9++QWxsbE4ffq00FGIiIiIiJRe27ZtYWdnh3Xr1gkdhYgUFIudpUgmk0FTU/OLrqGlpVVKaYjoU1StWhXz58+Hr68vioqKhI5DRERERKT0Zs+ejXnz5uHly5dCRyEiBcRiZwXDNTuJyt/gwYOhoaGB8PBwoaMQERERESm95s2bw8HBAatXrxY6ChEpIBY7S5FIJKoQ1yCizyMSibBixQpMnz4dT58+FToOEREREZHSmzVrFhYuXIjs7GyhoxCRgmGxsxQVFhZ+8TW4CDORMJo3b45+/fph5syZQkchIiIiIlJ6tra26NKlC5YvXy50FCJSMCIZ502XmrS0NLx48QLNmjUr0fmvXr1CmzZtYGNjA09PT7i4uHzxGqBE9On++ecfWFtbIz4+Hk2aNBE6DhERERGRUktNTYWjoyNu3ryJ6tWrCx2HiBQEOztLkZmZGdLS0kp8fnx8PPbs2YMOHTogJCQEhoaG8Pb2xqFDhyCRSEoxKRG9S82aNREYGAhfX1+un0tEREREJDALCwv07t0bixcvFjoKESkQFjtLmaGhYYkKnnl5ecjLy4OpqSnGjBmD48eP48qVK2jWrBlmzZqFevXqYdSoUYiPj4dUKi2D5EQEAKNHj8azZ88QExMjdBQiIiIiIqU3Y8YMrFq1CllZWUJHISIFwWnsZWDHjh1o37496tSp80njJRIJNm3ahCFDhkBNTe2dY+7evYuYmBhER0cjPT0dgwYNgoeHBxwdHaGiwpo1UWk6efIkvLy8kJKSAl1dXaHjEBEREREptTFjxqBq1apYsGCB0FGISAGw2FkGZDIZfv/9dzRq1Ag2NjYfHJuVlYW9e/fim2++gZaW1idd/9atW4iOjkZ0dDSePHkCd3d3eHh4oHXr1tzNnaiUeHl5oUGDBggKChI6ChERERGRUktPT0fTpk1x7do11K1bV+g4RFTBsdhZhi5fvozU1FRUr14dnTt3Lta1ef78edy5cwf6+vro1KlTibszr1+/Li985ufnw8PDAx4eHrC3t2fhk+gLiMViNG3aFKdPn4a5ubnQcYiIiIiIlNqECRMAAEuXLhU4CRFVdCx2loNnz57h5MmTyM7ORmhoKCZMmIAmTZrgq6++KrV7yGQyXL58GVFRUYiOjoaamho8PT3h4eHx0e5SInq3BQsW4NSpU9i7d6/QUYiIiIiIlNrDhw9hY2ODS5cuwdjYWOg4RFSBsdhZjp4/fw4TExM8f/68TO8jk8lw7tw5REVFISYmBtWqVZN3fFpYWJTpvYkqk/z8fDRp0gRLly5Fz549hY5DRERERKTUpkyZghcvXuC3334TOgoRVWAsdpaj/Px8VK1aFfn5+eV2z6KiIiQlJSE6Ohrbtm2DoaGhvPDZoEGDcstBpKgOHjyIcePG4erVq9DU1BQ6DhERERGR0srKyoKlpSXOnTtXqjMliahyYbGzHMlkMqiqqkIikUBVVbXc7y+VSnHixAlER0djx44dMDMzg4eHB9zc3DgNgOgDXF1d0a5dO0ydOlXoKERERERESm3GjBlIT09HeHi40FGIqIJisbOcaWtr459//oGOjo6gOSQSCY4cOYLo6Gjs3r0btra28PDwwKBBg1CnTh1BsxFVNGlpaWjTpg0uXboEIyMjoeMQERERESmtZ8+eoVGjRkhISOAybUT0Tix2ljN9fX3cunUL+vr6QkeRy8/PR2xsLKKjo7Fv3z60bNkSHh4eGDBgAGrWrCl0PKIKYfr06fj777+xZcsWoaMQERERESm1oKAgXL9+HZs3bxY6ChFVQCx2lrN69erh7NmzFbY7LC8vDwcOHEB0dDT++OMPtGvXDp6enujXrx+qVasmdDwiweTm5sLa2hqbNm1Cx44dhY5DRERERKS0srOzYW5ujvj4eNja2godh4gqGBWhAygbLS0tvHr1SugY76WtrY2BAwciJiYGYrEYw4YNw65du2BiYoK+ffti69atyMnJETomUbnT1dXFokWL4Ovri8LCQqHjEBEREREprSpVqmDy5MkIDAwUOgoRVUAsdpYzbW3tCl3s/Dc9PT14enpi9+7duHfvHgYOHIiNGzfCyMgIbm5u2L59O/Ly8oSOSVRu3NzcULNmTaxZs0boKERERERESs3HxweJiYlITk4WOgoRVTCcxk6f7Z9//sGuXbsQFRWFc+fOoVevXvDw8ICLiws0NTWFjkdUpq5evQpnZ2dcv34dBgYGQschIiIiIlJaK1asQGxsLPbu3St0FCKqQFjspC+SmZmJHTt2IDo6GleuXEHfvn3h4eGBLl26QF1dXeh4RGVi/PjxePXqFTs8iYiIiIgElJ+fj0aNGiEmJgZt27YVOg4RVRAsdlKpEYvF2LZtG6Kjo3Hr1i0MGDAAHh4e6NSpE1RVVYWOR1Rqnj17BisrK+zbtw8tW7YUOg4RERERkdJau3Yttm/fjtjYWKGjEFEFwWInlYk7d+4gJiYG0dHREIvFcHNzg4eHB9q1awcVFS4VS4ovLCwMoaGhSEhI4L9pIiIiIiKBSCQSWFlZISIiAh07dhQ6DhFVACx2Upm7efMmoqOjER0djWfPnsHNzQ2enp5o1aoVRCKR0PGISqSoqAht27bFjz/+iGHDhgkdh4iIiIhIaa1fvx5hYWE4fvw4f8ckIhY7FUHv3r1hYGCAyMhIoaN8sWvXrskLnxKJBO7u7vDw8IC9vT2/KZHC+fPPP9G/f3+kpKSgWrVqQschIiIiIlJKhYWFsLW1xYoVK9CtWzeh4xCRwDj38gskJydDVVUVjo6OQkdRGDY2Npg9ezZu3LiBnTt3AgAGDBgAKysrzJgxA9evXxc4IdGna9OmDb7++mvMnj1b6ChEREREREpLTU0NgYGB+OWXX8B+LiJisfMLrFu3Dj4+Prh69SpSUlI+OFYikZRTKsUgEolgb2+P+fPn4++//8bGjRuRm5uL7t27o0mTJpg7dy5u3rwpdEyij5o3bx42bNjw0f8DiIiIiIio7Li7uyM3Nxf79+8XOgoRCYzFzhLKy8vDli1b8P3332PQoEEICwuTP3fnzh2IRCJs3boVzs7O0NbWxpo1a/DPP//gm2++gbGxMbS1tWFjY4OIiIhi13358iW8vb2hp6eHOnXqIDg4uLxfWrkTiURo3bo1QkJCcO/ePfz222/IzMxEhw4d0KJFC/z666+4c+eO0DGJ3qlOnTr4+eefMW7cOL6LTEREREQkEBUVFcyePRszZsxAUVGR0HGISEAsdpbQ9u3bYWpqCjs7OwwZMgQbNmx4q3tz2rRp8PHxwfXr19GvXz+8evUKzZs3x759+3Dt2jWMHz8eo0ePRnx8vPycSZMm4fDhw9ixYwfi4+ORnJyMEydOlPfLE4yKigrat2+PFStWQCwWY+HChUhLS0OrVq3Qtm1bLF26FGKxWOiYRMX8+OOPePDgAXbt2iV0FCIiIiIipdWvXz+IRCL+XE6k5LhBUQl16tQJffr0waRJkyCTyfDVV18hJCQEAwcOxJ07d/DVV19h0aJFmDhx4gev4+npCT09PYSGhiInJwc1a9ZEeHg4vLy8AAA5OTkwNjZGv379KsUGRSUlkUhw5MgRREVF4ffff4etrS08PDwwaNAg1KlTR+h4RDhy5AhGjBiB69evQ0dHR+g4RERERERK6cCBA5g8eTIuX74MVVVVoeMQkQDY2VkCt27dQkJCAr799lsAr6dhe3l5ITQ0tNi4li1bFvtYKpUiKCgIdnZ2qFmzJvT09LBz507cu3cPAJCWloaCggI4ODjIz9HT00OTJk3K+BVVfOrq6nBxcUFERAQyMjIwadIkJCYmwtLSEl27dkVoaCiePHkidExSYs7OzmjVqhV+/fVXoaMQERERESmtHj16oFq1aoiOjhY6ChEJRE3oAIooNDQUUqkUJiYm8mNvGmTv378vP6arq1vsvEWLFiEkJATLli1DkyZNoKenh4CAADx69KjYNejDNDU14erqCldXV+Tl5eHAgQOIiorCxIkT4ejoCA8PD/Tr1w/VqlUTOiopmZCQEDRr1gze3t5o0KCB0HGIiIiIiJSOSCTCnDlzMGbMGLi7u0NNjWUPImXDzs7PVFhYiPXr12PevHm4ePGi/HHp0iXY2dm9teHQv506dQp9+vTBkCFDYG9vDzMzM6SmpsqfNzc3h7q6Ok6fPi0/lpubi6tXr5bpa1Jk2traGDhwILZt2waxWIwhQ4Zg165dMDExQb9+/bB161bk5OQIHZOUhImJCSZMmAA/Pz+hoxARERERKS1nZ2cYGRlh48aNQkchIgGw2PmZ9u/fj6ysLHz//fewtbUt9vD09ER4ePh7d36zsLBAfHw8Tp06hRs3bmDs2LG4ffu2/Hk9PT2MHDkSU6ZMweHDh3Ht2jWMGDECUqm0vF6eQtPT08M333yD3bt34+7du+jfvz82btwIIyMjuLu7Y8eOHcjLyxM6JlVykydPxsWLF3H48GGhoxARERERKaU33Z2zZ89GQUGB0HGIqJyx2PmZwsLC4OTkhJo1a771nJubG+7evYu4uLh3njt9+nS0bt0aPXr0QMeOHaGrqyvfiOiNRYsWwcnJCf3794eTkxNsbW3RsWPHMnktlVn16tUxbNgwHDhwAH///Te6deuG3377DYaGhhg8eDD27t2L/Px8oWNSJaSlpYUlS5Zg3Lhx/MGKiIiIiEgg7du3h6WlJcLDw4WOQkTljLuxk1LJzMzE9u3bER0djatXr6Jv377w9PSEs7Mz1NXVhY5HlYRMJkOPHj3QrVs3TJw4Ueg4RERERERK6ezZs+jfvz9u3boFLS0toeMQUTlhsZOUVnp6OrZt24bo6GikpaVhwIAB8PT0RMeOHaGqqip0PFJwf/31FxwdHXHlyhUYGhoKHYeIiIiISCn17dsXzs7OGD9+vNBRiKicsNhJBODOnTuIiYlBVFQUMjIyMGjQIHh6esLBwQEqKlztgUrG398fmZmZWL9+vdBRiIiIiIiU0qVLl3D+/HkMHz4cIpFI6DhEVA5Y7CT6j9TUVHnh8/nz53B3d4eHhwdatWrFb470WbKzs2FtbY2YmBi0a9dO6DhEREREREpJJpPxdzkiJcJiJ9EHXLt2DdHR0YiKikJhYSE8PDzg4eGBpk2b8pslfZLNmzdj8eLFOHPmDJdHICIiIiIiIipjLHYSfQKZTIaLFy8iOjoa0dHR0NDQgKenJzw8PNC4cWOh41EFJpPJ0LFjRwwZMgSjRo0SOg4RERERERFRpcZiZznLzMxEkyZN8OjRI6GjUAnJZDKcOXMG0dHRiImJQY0aNeSFT3Nzc6HjUQV08eJFuLi4ICUlBfr6+kLHISIiIiIiIqq0WOwsZ8+fP0f9+vXx4sULoaNQKSgqKkJCQgKio6Oxfft2GBkZwdPTE+7u7jA1NS3R9SQSCTQ1NcsgLQnJx8cHKioqWLlypdBRiIiIiIjoX86fPw8tLS3Y2NgIHYWISgGLneWsoKAAenp6KCgoEDoKlTKpVIrjx48jKioKO3fuRKNGjeDh4QE3NzcYGRl90jVSU1OxbNkyPHz4EM7Ozhg+fDh0dHTKODmVh3/++QeNGzdGbGwsmjZtKnQcIiIiIiKll5iYiJEjR+LevXuoW7cunJ2dMX/+fNSsWVPoaET0BVSEDqBs1NXVUVhYCKlUKnQUKmWqqqpwdnbG2rVrkZGRgZkzZ+LixYto0qQJOnXqhNWrVyM/P/+D13j69Cn09fVhZGQEX19fLF26FBKJpJxeAZWlmjVrYtasWfD19QXfYyIiIiIiEtbz58/xww8/wMLCAn/++SfmzJmDzMxMjBs3TuhoRPSF2NkpAB0dHTx+/Bi6urpCR6FykJ+fjz/++ANRUVHYsGED1NTUPnrO/v37MWLECGzduhXOzs7lkJLKg1QqRatWrTB58mR88803QschIiIiIlIqL1++hIaGBtTU1HDkyBH571wODg4AgGvXrsHBwQHXrl1D/fr1BU5LRCXFzk4BaGtr49WrV0LHoHKiqakJV1dXbNmyBaqqqh8c+2Z5g61bt6Jx48awtLR857hnz55h8eLF2LlzJ7sEFYiqqipWrFiByZMnIycnR+g4RERERERK4+HDh9i4cSNSU1MBAKampkhPT4e9vb18jK6uLuzs7PD06VOhYhJRKWCxUwBaWlosdiopkUj0wec1NDQAAIcOHYKLiwtq164N4PXGRUVFRQCAuLg4zJw5E5MmTYKPjw8SEhLKNjSVKkdHRzg5OSEoKEjoKERERERESkNdXR2LFi3CgwcPAABmZmZo06YNfH19kZ+fj5ycHAQFBeHevXvs6iRScCx2/j/27jsqqrN7G/A9BRiqgnTBjr1GFBsqYgkajEoUG/beTTCvHQsSe2yJvhqFiAUUeRU0BjWKgp3YOxAbiqiggiB15vsjP/kklqACzwxzX2u5hMM5Z+5jlgb27Gc/AigUCrx69Up0DFIzr+e47tu3D0qlEi1atICOjg4AQCqVQiqVYuXKlRg+fDjc3NzQpEkTdOvWDVWqVClwn8ePH+PPP/8s8fxUeIsXL8aGDRsQGxsrOgoRERERkVYoV64cGjdujLVr1+Y3H+3Zswfx8fFwdnZG48aNERMTg40bN8LU1FRwWiL6HCx2CsDOTvoQf39/ODo6olq1avnHzp07h+HDh2Pr1q3Yt28fmjZtivv376NevXqwtbXNP+/nn39Gly5d0LNnTxgaGmLKlClIT08X8Rj0ATY2NvjPf/6DSZMmiY5CRERERKQ1fvzxR1y6dAk9e/bE//73P+zZswc1a9ZEfHw8VCoVRo4cidatW2Pfvn1YtGgRkpKSREcmok/AYqcAnNlJ/6RSqfLneR4+fBhffvklzM3NAQBRUVHw8vJCo0aNcPz4cdSuXRubNm1C2bJlUb9+/fx7HDhwAFOmTEHjxo1x5MgR7Ny5E2FhYTh8+LCQZ6IPmzhxIuLj47F3717RUYiIiIiItIKNjQ02bdoEOzs7jBw5EsuWLcO1a9cwZMgQREVFYdSoUdDT08O9e/cQERGB77//XnRkIvoo+0jcAAAgAElEQVQE/74tNBU5LmOnN+Xk5GDRokUwMjKCXC6Hnp4eWrZsCV1dXeTm5uLSpUu4desWNm/eDJlMhpEjR+LAgQNwdnZGnTp1AACJiYmYO3cuunTpgnXr1gH4e+D21q1bsWTJEri7u4t8RHoHXV1drFy5EmPHjkX79u2hUChERyIiIiIiKvWcnZ3h7OyMZcuW4fnz59DV1c1vNMnNzYVcLseoUaPQsmVLODs74/Tp03BychKcmog+Bjs7BeAydnqTVCqFsbExFixYgAkTJiApKQn79+9HYmIiZDIZhg8fjlOnTsHZ2RnLly+Hjo4Ojh07hszMTJQpUwbA38vcT58+jalTpwL4u4AK/L2boK6ubv48UFIvnTp1Qt26dbF8+XLRUYiIiIiItIqBgQEUCsVbhc68vDxIJBLUr18fXl5eWLNmjeCkRPSxWOwUgMvY6U0ymQwTJ07EkydPcPfuXcyaNQv//e9/MXjwYCQnJ0NXVxeNGzfGkiVLcPPmTYwcORJlypRBWFgYxo8fDwA4duwYbG1t8cUXX0ClUuVvbHTnzh1UqVKFncRqbPny5Vi+fDnu378vOgoRERERkVbIy8uDq6srGjZsiClTpuCPP/7I/5np9XgxAEhLS4OBgQGbR4g0DIudArCzk97H3t4ec+fORWJiIjZv3pz/LuObLl26hG7duuHy5ctYtGgRACA6OhqdOnUCAGRnZwMALl68iJSUFFSoUAFGRkYl9xD0UapUqYIxY8ZgypQpoqMQEREREWkFmUwGR0dHJCQkIDk5GX369EGTJk0wYsQIhISE4OzZswgPD0doaCiqVq1aoABKROqPxU4BOLOTCsPS0vKtY7dv30ZMTAzq1KkDOzs7GBsbAwCSkpJQo0YNAIBc/vco3j179kAul6N58+YA/t4EidTT1KlTcfLkSURGRoqOQkRERESkFebOnQu5XI6xY8ciISEBU6dORU5ODqZOnYru3bvDw8MDAwYM4CZFRBpIomIFpMQNHz48/10josJSqVSQSCSIjY2FQqGAvb09VCoVcnJyMGbMGFy9ehXR0dGQyWRIT0+Hg4MD+vbtCx8fn/yi6Ov7xMTEwNTUFNWqVRP4RPSmkJAQzJs3D+fOncsvWBMRERERUfGZPHkyoqOjcfbs2QLHY2Ji4ODgkL9HwuufxYhIM7CzUwDO7KRP8fp/rg4ODrC3t88/pquri+HDh+P58+cYPnw4/Pz84OTkBBMTE3z77bcFCp2v7dq1Cy1btoSjoyOWLFmCu3fvluiz0Ns8PDxgYWGBtWvXio5CRERERKQVli5divPnzyM8PBzA35sUAYCjo2N+oRMAC51EGobFTgG4jJ2KkkqlgpOTE/z9/ZGamorw8HAMHDgQe/bsga2tLZRKZYHzJRIJFi5ciAcPHmDRokW4desWGjdujBYtWmDlypV4+PChoCfRbhKJBKtWrcK8efPw5MkT0XGIiIiIiEo9mUyG6dOnY//+/QDAFVZEpQSXsQswe/ZsyGQy+Pj4iI5CBADIycnBoUOHEBwcjD179qBBgwbw9PSEh4fHO2eHUvGZPHkyXr58iQ0bNoiOQkRERESkFW7cuIEaNWqwg5OolGBnpwBcxk7qRkdHB25ubggICEBiYiImT56MqKgoVK9eHR06dMDGjRuRkpIiOqZWmDNnDvbu3YuYmBjRUYiIiIiItELNmjXfKnSyL4xIc7HYKYBCoWCxk9SWQqHA119/jW3btuHhw4cYMWIE9u/fj8qVK6NLly4IDAxEamqq6JilVpkyZeDn54dx48a9NYKAiIiIiIiKl0qlgkqlwrNnz0RHIaJPxGKnAJzZSZrCwMAAPXv2REhICBISEtC3b1/s3LkT9vb26N69O4KDg5Geni46ZqkzcOBAAMDmzZsFJyEiIiIi0i4SiQS//fYbOnXqxO5OIg3FYqcAXMZOmsjY2Bj9+vVDWFgY7ty5g65du8Lf3x+2trbw9PREaGgoi/hFRCqVYvXq1Zg+fTpevHghOg4RERERkVZxc3NDTk4OwsLCREchok/AYqcAXMZOms7U1BSDBw/G77//jvj4eLi6umLNmjWwtbWFl5cX9u7di+zsbNExNVqTJk3QuXNnzJ07V3QUIiIiIiKtIpVKMW/ePMyePZujpYg0EIudAnAZO5Um5ubmGDFiBA4fPozr16/DyckJCxcuhI2NDYYOHYoDBw4gNzdXdEyN5Ofnh8DAQFy7dk10FCIiIiIireLu7g49PT2EhISIjkJEH4nFTgHY2UmllbW1NcaNG4fo6GhcuHABderUwcyZM2Fra4vRo0cjMjISeXl5omNqDEtLS8yaNQsTJkzgvCAiIiIiohIkkUgwf/58+Pj48GcYIg3DYqcAnNlJ2sDe3h7ffvstzpw5g1OnTqFixYqYPHky7O3tMXHiRJw4cYJLQgphzJgxSEpKQmhoqOgoRERERERapWPHjjA3N8e2bdtERyGijyBRsV2oxJ0+fRoTJkzA6dOnRUchKnE3b95EcHAwgoKC8PLlS/Tq1Qu9e/dG48aNIZFIRMdTS5GRkRg0aBCuXbsGAwMD0XGIiIiIiLRGZGQkhg0bhuvXr0NHR0d0HCIqBHZ2CsCZnaTNatSogdmzZ+Pq1avYt28fFAoF+vTpg2rVqmH69Om4ePEil2z/Q9u2beHk5IRFixaJjkJEREREpFXatm2LSpUq4ddffxUdhYgKiZ2dAty6dQtfffUVbt26JToKkVpQqVQ4f/48goKCsGPHDujr68PT0xOenp6oVauW6Hhq4f79+2jUqBHOnj2LypUri45DRERERKQ1Tp48id69e+PWrVvQ09MTHYeI/gU7OwXgBkVEBUkkEnzxxRdYvHgxbt++DX9/fzx//hzt27dHgwYN4Ofnh/j4eNExhbK3t8fkyZPx7bffio5CRERERKRVmjdvjrp16+KXX34RHYWICoGdnQI8fvwYderUwZMnT0RHIVJrSqUS0dHRCAoKwq5du1ChQgV4enqiV69eqFChguh4JS4zMxN169bFTz/9hE6dOomOQ0RERESkNf7880907doVcXFx0NfXFx2HiD6AxU4BUlNTUb58eaSlpYmOQqQxcnNzERkZieDgYISGhqJGjRro3bs3evbsCRsbG9HxSkx4eDi8vb1x+fJl6Orqio5DRERERKQ1evTogVatWnG1FZGaY7FTgJycHBgYGCAnJ0d0FCKNlJ2djUOHDiE4OBhhYWFo0KABevfuDQ8PD1hYWIiOV6xUKhW6dOkCFxcXTJkyRXQcIiIiIiKtcfnyZXTo0AFxcXEwMjISHYeI3oPFTgFUKhXkcjmysrIgl8tFxyHSaJmZmfj9998RHByM/fv3o2nTpvD09ET37t1hZmYmOl6xuHXrFlq0aIFLly7B1tZWdBwiIiIiIq3Rp08f1K9fH9OmTRMdhYjeg8VOQQwNDZGUlMR3g4iKUEZGBvbt24egoCAcOnQIzs7O8PT0xNdffw0TExPR8YrU1KlT8eDBAwQGBoqOQkRERESkNW7evIlWrVohLi4OZcqUER2HiN6BxU5BzM3NcePGDZibm4uOQlQqpaamIiwsDMHBwTh27BhcXV3h6emJr776CoaGhqLjfbaXL1+iZs2aCA4ORsuWLUXHISIiIiLSGoMGDUKlSpUwZ84c0VGI6B1Y7BTEzs4Op06dgp2dnegoRKXes2fPsHv3bgQFBeHUqVNwc3ODp6cn3NzcoFAoRMf7ZNu2bcOSJUsQExMDmUwmOg4RERERkVb466+/0LRpU9y8eRPlypUTHYeI/kEqOoC2UigUePXqlegYRFrB1NQUgwcPRkREBOLi4uDi4oLVq1fDxsYGAwYMwL59+5CdnS065kfr06cPjI2NsWHDBtFRiIiIiIi0RpUqVeDh4YGlS5eKjkJE78DOTkHq1q2L7du3o169eqKjEGmtxMREhISEIDg4GNevX0e3bt3Qu3dvuLi4aMzmYRcvXkSHDh1w/fp1vqtMRERERFRC7t+/j4YNG+LatWuwsrISHYeI3sDOTkH09fWRmZkpOgaRVrOxscH48eMRHR2N8+fPo3bt2pgxYwZsbW0xevRoREZGIi8vT3TMD2rQoAF69uyJWbNmiY5CRERERKQ17O3t0a9fPyxatEh0FCL6B3Z2CuLs7IwFCxagdevWoqMQ0T/Ex8djx44dCA4OxuPHj9GzZ0/07t0bzZo1g0QiER3vLSkpKahVqxYiIiLQsGFD0XGIiIiIiLRCYmIi6tSpg8uXL6N8+fKi4xDR/2FnpyAKhYKdnURqqmrVqpg2bRouXLiAw4cPw8zMDEOHDkWlSpUwZcoUxMTEQJ3eJzIzM8O8efMwfvx4tcpFRERERFSa2djYYOjQofDz8xMdhYjewGKnIFzGTqQZatasCR8fH1y9ehV79+6Fnp4eevfuDQcHB8yYMQOXLl1SiwLjsGHDkJGRgW3btomOQkRERESkNb7//nsEBQXh7t27oqMQ0f9hsVMQdnYSaRaJRIJ69erB19cXsbGxCA4ORk5ODtzd3VG7dm3MnTsXN27cEJZPJpNh9erV+P7775GWliYsBxERERGRNrGwsMDo0aMxf/580VGI6P+w2CmIQqHAq1evRMcgok8gkUjQuHFjLF68GLdv38amTZvw7NkztGvXDg0aNICfnx/i4+NLPFeLFi3g6uoKX1/fEn9tIiIiIiJt9d1332H37t2Ii4sTHYWIwGKnMOzsJCodpFIpmjdvjhUrVuD+/ftYtWoVEhIS0Lx5czRp0gTLli3D/fv3SyzPokWLsHHjRty8ebPEXpOIiIiISJuZmppi0qRJmDt3rugoRAQWO4XhzE6i0kcmk6FNmzb4+eef8fDhQ/j5+eH69eto2LAhWrZsiVWrViExMbFYM9jY2GDatGmYNGmSWswSJSIiIiLSBhMnTsSBAwdw7do10VGItB6LnYJwGTtR6SaXy9GhQwf88ssvSExMxPTp0xETE4PatWvDxcUF69atw5MnT4rltcePH487d+4gPDy8WO5PREREREQFGRsbw9vbG3PmzBEdhUjrsdgpCJexE2kPXV1ddOnSBZs3b0ZiYiImTpyIyMhIVKtWDZ06dcqf+VmUr7dq1SpMnjyZ/84QEREREZWQsWPHIjo6GhcuXBAdhUirsdgpCJexE2knhUKBbt26ISgoCA8fPsTQoUOxd+9eVKxYEe7u7tiyZQtSU1M/+3U6dOiABg0aYOnSpfnH0tLSEBcXhytXruD+/fvIy8v77NchIiIiIqK/GRgYYOrUqZg9e7boKERaTaLiUDchVqxYgTt37mDFihWioxCRGkhNTUVYWBiCgoIQFRUFV1dX9O7dG126dIGhoeEn3fPOnTto3Lgx/P39kZ2dDRMTE9jZ2UGhUOD58+e4c+cOVCoVWrduDQsLiyJ+IiIiIiIi7ZOZmQkHBwfs2rULTZs2FR2HSCux2CnIunXrcP78efz3v/8VHYWI1MyzZ8/wv//9D8HBwTh16hTc3NzQu3dvfPnll1AoFIW+T0JCAvz9/dGvXz9UqVLlnecolUpERUXhyZMn8PDwgEQiKarHICIiIiLSSv/9738RGhqKiIgI0VGItBKXsQvCmZ1E9D6mpqYYMmQIIiIiEBcXh7Zt22LlypWwsbHBgAED8NtvvyE7O/uD97h9+zbOnz+PWbNmvbfQCQBSqRRt2rSBq6srtm7dyh3ciYiIiIg+0+DBg3Hr1i1ERUWJjkKklVjsFIQzO4moMCwsLDBq1CgcOXIEV69ehaOjIxYsWAAbGxsMGzYMBw8eRG5uboFrUlNTERMTA3d390K/jqmpKTp37ow9e/YU9SMQEREREWkVXV1d+Pj4YNasWWwmIBKAxU5BFAoFXr16JToGEWkQW1tbTJgwAcePH8f58+dRs2ZNTJ8+HeXLl8eYMWNw9OhR5OXl4fDhw+jevftH39/MzAz6+vpIS0srhvRERERERNqjf//+SExMxOHDh0VHIdI6LHYKwmXsRPQ5KlSoAG9vb5w9exYnTpyAnZ0dJkyYADs7O8THx0Mul3/Sfdu1a8dvyIiIiIiIPpNcLsecOXMwc+ZMdncSlTAWOwXhMnYiKipVq1bF9OnTcfHiRaxYsQJ9+vT55Hvp6Oi8tSyeiIiIiIg+nqenJ9LS0rB//37RUYi0CoudgtSuXRs+Pj6iYxBRKWNgYABbW9vPuoehoSFycnKKKBERERERkXaSSqWYN28eZ3cSlTAWOwUpV64c2rVrJzoGEZUyRfFNlJGRER49elQEaYiIiIiItFv37t2hUqmwe/du0VGItManDXWjzyaRSERHIKJSqCj+bUlISEC7du2gr68Pa2trWFtbw8rK6q2PX/9uaWkJXV3dIkhPRERERFS6SCQSzJ8/H1OnTsXXX38NqZQ9Z0TFjcVOIqJSREdHBxkZGTAwMPjke+jp6SErKwvPnz/Ho0ePkJSUhEePHuV/HBsbW+DYkydPYGJi8t6i6JsfW1hYQCaTFeETExERERGpt86dO8PX1xc7duxA7969RcchKvUkKg6OICIqNbKysnDgwAG4u7t/0vUqlQqhoaHw8PAo9DVKpRLJyclvFUX/+XFSUhJSUlJgZmb2zg7Rf35sZmbGd76JiIiIqFQ4dOgQxo4di6tXr0IuZ98ZUXHi3zAiolLkdVemSqX6pCXtZ86cgZOT00ddI5VKYWFhAQsLC9StW/eD5+bm5uLJkycFCqCPHj1CQkIC/vzzzwIF0tTUVFhaWn5wCf3rj8uWLcvxIERERESktlxdXWFjY4OtW7di4MCBouMQlWrs7FRTOTk5kEqlXO5JRB/t3r17+Ouvv9C2bduPui4vLw9BQUHo169f8QT7SNnZ2Xj8+PE7O0T/eSwrKwtWVlb/2i1qZWUFIyMjFkaJiIiIqMRFRUVh4MCBuHHjBmfeExUjFjsFiYiIQLNmzVCmTJn8Y6//U0gkEvzyyy9QKpUYMWKEqIhEpMFOnDgBfX19NGrUqFDnK5VKBAYGomfPnp8171OUV69efbAY+uYxAIXqFrW2toa+vr7gJyu8DRs24OjRo9DX14eLiwv69OnDoi4RERGRmunUqRN69OiBkSNHio5CVGqx2CmIVCrF8ePH0bx583d+ff369diwYQOio6Ohp6dXwumIqDQ4efIkUlNT0aFDhw/OvkxOTkZYWBg8PDxgYmJSggnFePnyZaG6RZOSkqCnp/fBYuibv4t6dz49PR0TJ07EiRMn0LVrVzx69AixsbHo3bs3xo8fDwC4fv065s2bh1OnTkEmk2HAgAGYPXu2kLxERERE2uzMmTPw8PBAbGwsFAqF6DhEpRKLnYIYGhpi+/btaN68OTIyMpCZmYnMzEy8evUKmZmZOH36NKZNm4aUlBSULVtWdFwi0lCPHz9GVFQUJBIJXFxcYGpqmv+1P//8E4cPH8aRI0cQHh7OsRn/oFKp8OLFi0J1iz558gRGRkaF6ha1sLAo0qH0J0+eRMeOHeHv749vvvkGALBu3TrMmjUL8fHxSEpKQrt27eDo6Ahvb2/ExsZiw4YNaNu2LRYsWFBkOYiIiIiocLp27Yr27dtjwoQJoqMQlUosdgpiY2ODpKSk/CWSEokkf0anTCaDoaEhVCoVLl68WKA4QUT0KfLy8nDs2DGkpaXlH6tbty5sbW1RtWpV7N27t9BL3ultSqUSKSkphdqRPjk5Gaampv/aLWptbY1y5cr96470gYGB+M9//oP4+Hjo6upCJpPh7t27cHd3x7hx46Cjo4NZs2bhxo0bMDIyAgBs2rQJc+fOxfnz52FmZlYSf0RERERE9H8uXLiAzp07Iy4uTiNHSBGpO+7GLkheXh6+++47tGvXDnK5HHK5HDo6Ovm/y2QyKJVKGBsbi45KRKWATCaDi4vLO7/m7e0NX19f7Nq1q4RTlR5SqRTm5uYwNzdHnTp1Pnhubm4unj59+laH6MOHD3H+/PkCBdIXL17AwsICly9fRrly5d55P2NjY2RlZSEsLAyenp4AgP379+P69etITU2Fjo4OTE1NYWRkhKysLOjp6aFmzZrIyspCVFQUvv766yL/8yAiIiKi92vYsCFatmyJn376CVOmTBEdh6jUYbFTELlcjsaNG8PNzU10FCLSciNHjsSiRYtw+fJl1KtXT3ScUk8ul+d3bjZo0OCD52ZnZ+PJkycfHGfy5ZdfYsiQIZgwYQI2bdoES0tLJCQkIC8vDxYWFihfvjwSEhKwbds29O3bFy9fvsTq1avx5MkTpKenF/XjEREREVEhzJkzB+3atcOoUaPY5ERUxGRz5syZIzqENkpJSYGTkxPs7Oze+ppKpeIOukRUYnR0dKBUKrFjx478mY+kHmQyGUxMTD64lF0ul6Np06Zo1KgRsrOzYWNjgypVquDFixdo2rQpevTogfT0dEydOhW+vr4IDw/P7/Ds1KkTateunX8vlUqFhw8f4urVq8jJyYGenh50dHRK4lGJiIiItIqlpSUuXryI+Ph4tG7dWnQcolKFMzvV1LNnz5CTkwNzc/N/nddGRPS50tLSULVqVRw7dgw1a9YUHYc+0/z58xEWFob169fnz2J98eIFrl27Bmtra2zatAl//PEHFi9ejFatWuVfp1KpEB4eDj8/v/yl9Do6OoXekV5PT0/UIxMRERFpnNjYWLRo0QK3bt3iXh1ERYjFTkF27tyJqlWr4osvvihwXKlUQiqVIiQkBDExMRg3btw7uz+JiIraggULcPPmTWzevFl0FPoI58+fR15eHho1agSVSoX//e9/GD16NLy9vTFlypT8lQJvvnHWpk0b2NnZYfXq1R/coEilUiE1NbVQO9I/fvwYhoaGhd6Rnh2jnycjIwNHjhyBUqnMXxGiUCjg4uICuZxTioiIiDTF0KFDYWtri/nz54uOQlRqsNgpSOPGjeHu7o73TRE4efIkxo8fj2XLlqFNmzYlG46ItNKLFy9QtWpVnDp1CtWqVRMdhwrp999/x6xZs5CWlgZLS0ukpKTA1dUVfn5+MDQ0xK5duyCTydC0aVNkZGRg2rRpiIqKwu7du9GsWbMiy6FUKvHs2bNC7Uj/9OlTlC1bttA70stksiLLqen++usvnD9/HgYGBmjXrl2BbtoXL17gyJEjyM3NRevWrWFpaSkwKRERERXGnTt34OjoiBs3bsDc3Fx0HKJSgcVOQdq1a4eqVavC29sbL1++xKtXr5CZmYmMjAxkZWXh4cOH+O677xAYGIg+ffqIjktEWsLHxwcJCQnYuHGj6ChUSFlZWbh58yZu3bqFp0+folq1amjfvn3+14ODg+Hj44Pbt2/DwsICjRo1wpQpU4TOhsrLy3vnjvTv+vj58+cwNzd/Z1H0nwVSMzOzUj3z+vjx41AqlXB2dv7geSqVCvv27UPlypVRp06dEkpHREREn2rMmDEwMjLC4sWLRUchKhVY7BTEy8sLW7duha6uLpRKJWQyGeRyOeRyOXR0dGBkZIScnBwEBATA1dVVdFwi0hIpKSlwcHDAn3/+iUqVKomOQ5/oXRvdZWRkIDk5GQYGBihXrpygZB8vJycHT548+eAS+tcfp6enw8rK6oNL6F9/bGJiolGF0VOnTkGhUKBhw4aFvuaPP/6Avb09qlevXozJiIiI6HM9ePAA9evXx9WrV2FtbS06DpHGY7FTkF69eiEjIwNLliyBTCYrUOyUy+WQSqXIy8uDqakpN3wgIiIqhMzMTDx+/LhQM0Zzc3ML1S1qbW0NQ0NDoc+VnJyMM2fOwM3N7aOv3bZtGzw9PTkKgIiISM1NnjwZSqUSK1euFB2FSOOx2CnIgAEDIJVKERAQIDoKERGR1klPT3+rCPq+5fRyubzQO9IrFIoizxoaGoqvv/76kwqWycnJuHTpElxcXIo8FxERERWdpKQk1K5dGxcuXIC9vb3oOEQajdt1CtK3b19kZ2fnf/56yaFKpcr/JZVKNWqJHRERkaYwNDRElSpVUKVKlQ+ep1KpkJaW9s5i6JkzZ97akV5fX79QO9JbWloWakf617utf2pnZrly5ZCSkvJJ1xIREVHJsbKywvDhw7FgwQKsW7dOdBwijcbOTiIiIqIioFKpCr0j/ZMnT1CmTJl/7Ra9e/cumjVr9lk7qx8/fhwODg7cnZ2IiEjNJScno0aNGjh79iwqV64sOg6RxmKxU6C8vDxcv34dcXFxqFSpEho2bIjMzEycO3cOr169Qt26dWFlZSU6JhERERWxvLw8JCcn/+sSeolEgkuXLn3Wa929exfPnz9HgwYNiig9ERERFRcfHx/cu3cP/v7+oqMQaSwuYxdo0aJFmDlzJnR1dWFhYYH58+dDIpFg4sSJkEgk6NatGxYuXMiCJxF9tLZt26Ju3bpYs2YNAKBSpUoYN24cvL2933tNYc4hoqIhk8lgaWkJS0tL1KtX773nhYWFffZr6enpISsr67PvQ0RERMVv8uTJcHBwwM2bN1GjRg3RcYg0klR0AG119OhRbN26FQsXLkRmZiZ+/PFHLF26FBs2bMDPP/+MgIAAXL16FevXrxcdlYjU0JMnTzBmzBhUqlQJenp6sLKygqurKw4ePAjg7w1Nfvjhh4+659mzZzFmzJjiiEtEn0gikUCpVH7WPZ4/f46yZcsWUSIiIiIqTmXLlsXkyZMxd+5c0VGINBY7OwW5f/8+ypQpg++++w4A8M033+D48eO4dOkS+vbtCwC4evUqTpw4ITImEakpDw8PZGRkYOPGjahWrRoeP36Mo0ePIjk5GQBgZmb20fe0sLAo6phE9JmaNm2K6OhotG7d+pPvcePGDXz11VdFmIqIiIiK04QJE1CtWjVcuXIFdevWFR2HSOOws1MQHR0dZGRkFNhdVUdHB+np6fmfZ2VlITc3V0Q8IlJjz58/R1RUFBYuXAhXV1dUrFgRTZo0gbe3N3r37g3g72Xs48aNK3Ddy5cv0b9/f6fUT6oAACAASURBVBgZGcHa2hpLly4t8PVKlSoVOCaRSBASEvLBc4ioeFlZWeHx48effL1KpUJeXh7kcr6/TUREpCmMjIzw/fffw8fHR3QUIo3EYqcg9vb2UKlU2Lp1KwDg1KlTOH36NCQSCX755ReEhIQgIiICbdq0EZyUiNSNkZERjIyMEBYWhszMzEJft3z5ctSqVQvnzp3D3LlzMX36dISGhhZjUiIqCnZ2dkhISPika48fP46WLVsWcSIiIiIqbqNHj8apU6dw7tw50VGINA7f5hekYcOG6Ny5MwYPHoxff/0Vt2/fRqNGjTBs2DD06dMHCoUCTZs2xfDhw0VHJSI1I5fLERAQgOHDh2P9+vVo1KgRWrZsiZ49e8LJyem91zk5OWHGjBkAgOrVq+Ps2bNYvnw5evToUVLRiegTODk54ddff0W/fv2go6NT6OtSUlKQmJiIVq1aFWM6IiIiKg76+vqYPn06Zs+ejb179yIuLg7Xrl2DRCIBABgbG8PZ2bnAalEi+hs7OwUxMDDAvHnzsGPHDtSoUQOTJk3Ctm3b0LFjR1y4cAFbtmzB9u3bYW5uLjoqEakhDw8PPHz4EOHh4XBzc8OJEyfQrFkz+Pn5vfea5s2bv/X5tWvXijsqEX0miUSC3r17Y8uWLYXu5n78+DF+++03fPPNN8WcjoiIiIrLoEGDcP/+ffzyyy9IT09H165d4e7uDnd3dzRo0ABhYWHYtWvXZ428ISqN2NkpkI6ODrp164Zu3boVOG5vbw97e3tBqYhIUygUCnTo0AEdOnTA7NmzMWzYMMyZMwfe3t5Fcn+JRAKVSlXgWE5OTpHcm4g+jkKhQP/+/REaGgpzc3O0bdv2nZ0cmZmZ2LdvH5YvX47g4OD87g8iIiLSLM+fP8fu3bsRGRkJU1PTt75uamqK7t27Q6lU4uDBgyhTpgyaNWsmICmR+mGxUw28Lia8+QOJSqXiDyhE9FFq166N3Nzc93Z+nTp16q3Pa9Wq9d77WVhYIDExMf/zpKSkAp8TUcnS0dGBp6cnUlJSEBYWBpVKBR0dHejp6SEzMxM5OTnQ09ND586dceXKFQwbNgz79+/n9xNEREQa5uXLlwgLC8PAgQP/9f/jUqkUnTp1wrlz53Dy5Mm3VnMRaSMWO9XAu/7x4g8mRPQ+ycnJ6NmzJ4YMGYL69evD2NgYMTExWLx4MVxdXWFiYvLO606dOoUffvgB33zzDSIjI7F58+b8TdLepV27dvjpp5/QokULyGQyTJ8+HQqForgei4gKyczMDN27dwfw95ujWVlZ0NPTK/C9w/Tp09GiRQusW7cOo0ePFhWViIiIPsHu3bvRv3//j6oLfPHFFzh8+DDu37/PlaKk9VjsJCLSMEZGRmjWrBlWrlyJuLg4ZGVloXz58ujbty9mzpz53uu+/fZbXLp0CQsWLIChoSHmzZv3wXl+y5Ytw9ChQ9G2bVtYWVlh8eLFuH79enE8EhF9IolE8s43IXR0dBAYGIhWrVqhffv2cHBwEJCOiIiIPtbt27dRs2ZNSKUfv8WKi4sLdu3axWInaT2J6p8D2YiIiIioVFi1ahW2b9+OqKgoyOV8j5uIiEjdhYSEwMPD45NXe+7Zswdubm7Q1dUt4mREmoO7sQukVCoRGxsrOgYRERGVUuPGjYOhoSEWL14sOgoRERH9C5VKBZlM9llj7VxdXXHkyJEiTEWkeVjsFEipVKJmzZpv7XZMREREVBSkUin8/f2xYsUKnD9/XnQcIiIi+oC0tLR37rz+MYyMjJCdnV1EiYg0E4udAsnlckilUuTm5oqOQkRERKWUvb09li1bBi8vL2RmZoqOQ0RERO+RkZEBAwODz74PG6pI27HYKZhCocCrV69ExyAiIqJSrH///qhZsyZmzZolOgoRERG9h4mJCVJTU0XHINJ4LHYKplAo2GVBRERExUoikWDdunXYunUrjh49KjoOERERvYO+vj5evHjxWfdISEiApaVlESUi0kwsdgqmr6/PYicRaaw2bdogMDBQdAwiKgRzc3M8fPgQbdq0ER2FiIiI3kEikUAmk33WqLvTp0/DycmpCFMRaR4WOwVjZycRabJZs2ZhwYIFyMvLEx2FiIiIiEjjubi4fPJu6jk5OZDL5Z+1mztRacBip2Cc2UlEmszV1RWmpqYICQkRHYWIiIiISOOVKVMGaWlpSElJ+ehrd+3aBVdX12JIRaRZWOwUjMvYiUiTSSQSzJ49G/Pnz4dSqRQdh4iIiIhI43Xv3h179+7Fs2fPCn3N7t270aJFCxgZGRVjMiLNwGKnYFzGTkSa7ssvv4S+vj52794tOgoRERERkcaTSCTw8vLCH3/8gX379n2wqeDOnTsIDAxE06ZNUaFChRJMSaS+5KIDaDsuYyciTSeRSDBz5kzMnTsX3bt354wgIiIiIqLPJJFI4O7ujipVqmDatGkoX7487O3tUbZsWbx69QqJiYlIS0tDxYoV0b9/f34PTvQGdnYKxs5OIioNunbtCqVSiX379omOQqQ2Bg0aBIlE8tavCxcuiI5GREREGmDjxo1o1KgRxo0bh6+//hq2trbIzs6GkZERWrZsCQ8PDzg6OrLQSfQP7OwUjDM7iag0eN3dOW/ePHTp0oXfcBH9n/bt2yMwMLDAMXNzc0FpgOzsbOjq6gp7fSIiIiqcrKws/PDDDwgNDQUASKVS2NrawtbWVnAyIvXHzk7B2NlJRKVFjx49kJ6ejgMHDoiOQqQ29PT0YG1tXeCXXC7Hb7/9hlatWqFs2bIwMzODm5sbbt68WeDaEydOoGHDhlAoFPjiiy+wd+9eSCQSREdHAwBycnIwZMgQVK5cGfr6+qhevTqWLl0KlUqVf4/+/fujW7du8PPzQ/ny5VGxYkUAwK+//gpHR0cYGxvDysoKnp6eSExMzL8uOzsb48aNg42NDfT09GBvb48ZM2aUwJ8YERERAX93ddavXx9NmjQRHYVI47CzUzDO7CSi0kIqleZ3d3bs2JHdnUQfkJ6ejm+//Rb16tVDRkYG5s2bB3d3d1y9ehU6OjpITU2Fu7s7OnfujG3btuH+/fuYNGlSgXvk5eWhQoUK2LFjBywsLHDq1CmMGDECFhYWGDhwYP55f/zxB0xMTHDgwIH8QmhOTg7mz5+PGjVq4MmTJ/j+++/Rt29fHDlyBADw448/Ijw8HDt27ECFChWQkJCA2NjYkvsDIiIi0mJZWVlYuHAhQkJCREch0kgS1Ztv/1OJmzx5MipUqIDJkyeLjkJE9Nny8vJQu3ZtrF27Fu3atRMdh0ioQYMGYcuWLVAoFPnHnJ2dsX///rfOTU1NRdmyZXHixAk0a9YMP/30E3x8fJCQkJB//ebNmzFw4EBERUWhVatW73xNb29vXLlyBb///juAvzs7Dx06hHv37n1w+fqVK1dQr149JCYmwtraGmPGjEFcXBwiIiL4xgUREVEJW7t2Lfbu3ct5+ESfiMvYBeMydiIqTWQyGaZPn4758+eLjkKkFlq3bo0LFy7k//rll18AALGxsejTpw+qVKkCExMT2NraQqVS4d69ewCAGzduoH79+gUKpU5OTm/d/6effoKjoyMsLCxgZGSE1atX59/jtXr16r1V6IyJiUHXrl1RsWJFGBsb59/79bWDBw9GTEwMatSogfHjx2P//v1QKpVF9wdDRERE7/R6VqePj4/oKEQai8VOwbiMnYhKm759++LevXuIiooSHYVIOAMDA1SrVi3/V/ny5QEAXbp0QUpKCjZs2IDTp0/jzz//hFQqRXZ2NgBApVL9a0fl1q1b4e3tjSFDhiAiIgIXLlzAyJEj8+/xmqGhYYHP09LS0KlTJxgbG2PLli04e/YsfvvtNwDIv7ZJkya4c+cOfH19kZOTg/79+8PNzQ1cEERERFS8/P39UbduXTRt2lR0FCKNxZmdgikUCiQnJ4uOQURUZHR0dDBt2jTMnz+fmxURvUNSUhJiY2OxceNGODs7AwDOnDlToHOyVq1aCA4ORlZWFvT09PLPeVN0dDRatGiBMWPG5B+Li4v719e/du0aUlJSsHDhQtjb2wMALl269NZ5JiYm6NWrF3r16gUvLy+0atUKt2/fRpUqVT7+oYmIiOhfZWVlwc/PDzt37hQdhUijsbNTMH19fS5jJ6JSZ8CAAXjw4AGePn0qOgqR2jE3N4eZmRnWr1+PuLg4REZGYuzYsZBK//+3ZV5eXlAqlRgxYgSuX7+OgwcPYuHChQCQ3/FZvXp1xMTEICIiArGxsZgzZw6OHz/+r69fqVIl6OrqYvXq1bh9+zb27t371lK5pUuXIigoCDdu3EBsbCy2b9+OMmXKwNbWtgj/JIiIiOhNr7s63zW6hogKj8VOwbiMnYhKI11dXVy5cgXlypUTHYVI7chkMgQHB+PcuXOoW7cuxo8fjx9++AE6Ojr555iYmCA8PBwXLlxAw4YN8Z///Adz584FgPw5nmPGjEGPHj3g6emJpk2b4sGDB2/t2P4uVlZWCAgIQEhICGrVqgVfX18sX768wDlGRkZYtGgRHB0d4ejomL/p0ZszRImIiKhojRo1Kn+0DBF9Ou7GLtjmzZtx8OBBBAYGio5CREREamzXrl3o1asXnj59ClNTU9FxiIiIiIjUEmd2CsZl7ERERPQu/v7+cHBwgJ2dHS5fvoxvv/0W3bp1Y6GTiIiIiOgDWOwUTKFQsNhJRFpJqVQWmFFIRAU9evQIc+bMwaNHj2BjYwN3d/f8uZ1ERERERPRuXMYu2MGDB7Fo0SIcOnRIdBQiohKhVCoRFhaG7du3o1q1aujatSuHsBMREREREVGRYEuNYOzsJCJtkZOTAwC4cOECvvvuOyiVSkRFRWHo0KFITU0VnI6IiIiISDPl5uZCIpFg9+7dxXoNkaZgsVMwzuwkotIuIyMDU6ZMQf369dG1a1eEhISgRYsW2L59OyIjI2FtbY3p06eLjklEREREVOTc3d3Rvn37d37t+vXrkEgkOHjwYAmnAuRyORITE+Hm5lbir01U3FjsFEyhUODVq1eiYxARFQuVSoU+ffrgxIkT8PX1Rb169RAeHo6cnBzI5XJIpVJMnDgRR48eRXZ2tui4RERERERFatiwYTh8+DDu3Lnz1tc2btyIihUrwtXVteSDAbC2toaenp6Q1yYqTix2CsZl7ERUmt28eRO3bt2Cl5cXPDw8sGDBAixfvhwhISF48OABMjMz8dtvv8Hc3Bzp6emi4xLRv1i+fDmcnZ2Rl5cnOgoREZFG6NKlC6ysrODv71/geE5ODgIDAzFkyBBIpVJ4e3ujevXq0NfXR+XKlTF16lRkZWXln3/37l107doVZmZmMDAwQK1atbBz5853vmZcXBwkEgkuXLiQf+yfy9a5jJ1KMxY7BeMydiIqzYyMjPDq1Su0bt06/5iTkxOqVKmCQYMGoWnTpjh+/Djc3NxgamoqMCkRFcakSZMgk8mwfPly0VGIiIg0glwux8CBAxEQEAClUpl/PDw8HE+fPsXgwYMBACYmJggICMD169exZs0abNmyBQsXLsw/f9SoUcjOzkZkZCSuXr2K5cuXo0yZMiX+PESagMVOwdjZSUSlmZ2dHWrWrIkVK1bkf3MXHh6O9PR0+Pr6YsSIERg4cCAGDRoEAAW+ASQi9SOVShEQEIDFixfj0qVLouMQERFphKFDh+LevXs4dOhQ/rGNGzeiY8eOsLe3BwDMnj0bLVq0QKVKldClSxdMnToV27dvzz//7t27cHZ2Rv369VG5cmW4ubmhY8eOJf4sRJpALjqAtuPMTiIq7ZYsWYJevXrB1dUVjRo1QlRUFLp27QonJyc4OTnln5ednQ1dXV2BSYmoMCpVqoTFixfDy8sLZ86c4awvIiKif+Hg4IDWrVtj06ZN6NixIx4+fIiIiAgEBwfnnxMcHIxVq1YhPj4eL1++RG5uLqTS/9+fNnHiRIwbNw779u2Dq6srevTogUaNGol4HCK1x85OwV53dqpUKtFRiIiKRb169bB69WrUqFED586dQ7169TBnzhwAQHJyMn7//Xf0798fI0eOxM8//4zY2FixgYnoXw0aNAiVKlXK/7tMREREHzZs2DDs3r0bKSkpCAgIgJmZGbp27QoAiI6ORr9+/dC5c2eEh4fj/PnzmDdvXoENPEeOHIm//voLAwcOxI0bN9CsWTP4+vq+87VeF0nfrDPk5OQU49MRqRcWOwWTyWSQy+X8h4eISrX27dtj3bp12Lt3LzZt2gQrKysEBASgTZs2+Oqrr/DgwQOkpKRgzZo16Nu3r+i4RPQvJBIJNmzYgICAABw/flx0HCIiIrX3zTffQKFQYMuWLdi0aRMGDBgAHR0dAMDx48dRsWJFzJgxA02aNIGDg8M7d2+3t7fHyJEjsXPnTsyePRvr169/52tZWloCABITE/OPvblZEVFpx2KnGuBSdiLSBnl5eTAyMsKDBw/QoUMHDB8+HM2bN8f169dx4MABhIaG4vTp08jOzsaiRYtExyWif2FpaYm1a9di4MCBePnypeg4REREak1fXx99+/bFnDlzEB8fj6FDh+Z/rXr16rh37x62b9+O+Ph4rFmzBjt27Chw/fjx4xEREYG//voL58+fR0REBGrXrv3O1zIyMoKjoyMWLlyIa9euITo6Gt9//32xPh+ROmGxUw1wkyIi0gYymQwAsHz5cjx9+hR//PEHNmzYAAcHB0ilUshkMhgbG6NJkya4fPmy4LREVBjdunWDs7MzvL29RUchIiJSe8OGDcOzZ8/QokUL1KpVK/949+7dMXnyZEyYMAENGzZEZGQk5s6dW+DavLw8jB07FrVr10anTp1Qvnx5+Pv7v/e1AgICkJubC0dHR4wZM+a9S96JSiOJisMihatYsSKOHTuGihUrio5CRFSsEhIS0K5dOwwcOBAzZszI33399Vyhly9fombNmpg5cyZGjRolMioRFdKLFy/QoEEDrF27Fm5ubqLjEBEREZGWY2enGmBnJxFpi4yMDGRmZqJfv34A/i5ySqVSZGZmYteuXXBxcYG5uTm6d+8uOCkRFVaZMmXg7++PYcOGITk5WXQcIiIiItJyLHaqAc7sJCJtUb16dZiZmcHPzw93795FdnY2tm3bhgkTJmDJkiUoX7481qxZAysrK9FRiegjuLi4wNPTE6NHjwYXDRERERGRSCx2qgF2dhKRNlm7di2uX7+ORo0aoVy5cli6dClu3bqFTp06YcWKFWjVqpXoiET0CRYsWIArV64gKChIdBQiIiIi0mJy0QHo713ZWOwkIm3RvHlz7N+/HxEREdDT0wMANGzYEHZ2doKTEdHn0NfXR2BgINzc3ODs7My/00REREQkBIudaoDL2IlI2xgZGcHDw0N0DCIqYo0bN8b48eMxZMgQREREQCKRiI5ERERERFqGy9jVAJexExERUWkxbdo0vHjxAj///LPoKERERELl5OSgSpUqiIqKEh2FSKuw2KkGuIydiAhQqVTc2ISoFJDL5di8eTN8fHxw69Yt0XGIiIiE2bJlCypXrgxnZ2fRUYi0CoudaoCdnUREQGhoKJYtWyY6BhEVgRo1amDOnDkYMGAAcnNzRcchIiIqcTk5OfD19YWPj4/oKERah8VONcCZnUREgIODA5YtW8Z/D4lKiTFjxsDExAQLFy4UHYWIiKjEbdmyBZUqVULr1q1FRyHSOix2qgF2dhIRAfXr10ezZs2wYcMG0VGIqAhIpVJs2rQJq1atwrlz50THISIiKjHs6iQSi8VONcCZnUREf5s5cyYWL17MfxOJSgk7Ozv8+OOP8PLy4t9rIiLSGlu3bkXFihXZ1UkkCIudaoDL2ImI/ta4cWM0aNAA/v7+oqMQURHp27cv6tSpgxkzZoiOQkREVOxyc3PZ1UkkGIudaoDL2ImI/r9Zs2Zh4cKFyM7OFh2FiIqARCLB2rVrERQUhMjISNFxiIiIitWWLVtQoUIFtGnTRnQUIq3FYqca4DJ2IqL/r1mzZqhRowY2b94sOgoRFZFy5cphw4YNGDRoEFJTU0XHISIiKhbs6iRSDyx2qgF2dhIRFTRr1iz88MMPyM3NFR2FiIpI586d0alTJ0yaNEl0FCIiomKxdetW2Nvbs6uTSDAWO9UAZ3YSERXk7OyMChUqYNu2baKjEFERWrZsGY4ePYo9e/aIjkJERFSkcnNzMX/+fHZ1EqkBFjvVADs7iYjeNmvWLCxYsAB5eXmioxBRETEyMsLmzZsxatQoPH78WHQcIiKiIrN161bY2dmhbdu2oqMQaT0WO9UAZ3YSEb3NxcUF5ubm2LFjh+goRFSEWrZsiYEDB2LEiBFQqVSi4xAREX2217M658yZIzoKEYHFTrXAZexERG+TSCSYPXs2fH19oVQqRcchoiI0d+5c3L59G7/++qvoKERERJ9t27ZtKF++PLs6idQEi51qgMvYiYjerWPHjjA0NERoaKjoKERUhPT09BAYGIgpU6bg7t27ouMQERF9stezOtnVSaQ+WOxUA1zGTkT0bhKJBLNmzYKvry+XuxKVMvXr14e3tzcGDRrE7m0iItJY27Ztg62tLbs6idQIi51qgJ2dRETv99VXX0EikSA8PFx0FCIqYt7e3sjJycHKlStFRyEiIvponNVJpJ5Y7FQDnNlJRPR+r7s758+fz+5OolJGJpPh119/hZ+fH65duyY6DhER0UfZvn07bGxs2NVJpGZY7FQD7OwkIvqwbt26ITMzE7///rvoKERUxKpWrQo/Pz94eXkhOztbdBwiIqJCeXNWp0QiER2HiN7AYqca4MxOIqIPk0qlmDFjBrs7iUqpYcOGwdraGr6+vqKjEBERFUpQUBCsra3Z1UmkhiQq/tQoXEZGBsqVK8el7EREH5CXl4c6dergp59+gqurq+g4RFTEEhMT0ahRI+zZswdOTk6i4xAREb1Xbm4u6tSpg7Vr16Jdu3ai4xDRP7CzUw0oFApkZWWxW4mI6ANkMhlmzJiBefPmiY5CRMXAxsYGa9asgZeXFzIyMkTHISIieq+goCBYWVnBxcVFdBQiegd2dqoJPT09pKamQk9PT3QUIiK1lZubi5o1a2LTpk1o3bq16DhEVAz69+8PU1NTrF69WnQUIiKit+Tl5aF27dr4+eefudqISE2xs1NNcJMiIqJ/J5fLMX36dMyfP190FCIqJmvWrMGePXtw8OBB0VGIiIjeEhQUBEtLSy5fJ1JjLHaqCYVCwZmdRESF4OXlhdjYWJw8eVJ0FCIqBmXLlsXGjRsxZMgQPHv2THQcIiKifHl5eZg3bx53YCdScyx2qgl2dhIRFY6Ojg6mTp3K7k6iUqxDhw7o1q0bxo0bJzoKERFRPnZ1EmkGFjvVhL6+PoudRESFNHjwYFy+fBkxMTGioxBRMVm0aBFiYmKwY8cO0VGIiIiQl5eH+fPnw8fHh12dRGqOxU41wWXsRESFp6enh++//57dnUSlmIGBAQIDAzF+/HgkJiaKjkNERFouODgY5ubm3JSISAOw2KkmuIydiOjjDBs2DGfPnsXFixdFRyGiYtK0aVOMGjUKQ4cOhUqlEh2HiIi0FGd1EmkWFjvVBJexExF9HH19fXh7e8PX11d0FCIqRjNnzkRSUhI2bNggOgoREWkpdnUSaRYWO9UEOzuJiD7eyJEjcezYMVy9elV0FCIqJjo6OggMDMSMGTMQHx8vOg4REWkZzuok0jwsdqoJzuwkIvp4hoaGmDx5MhYsWCA6ChEVo9q1a2PGjBkYMGAA8vLyRMchIiItsmPHDpiZmaF9+/aioxBRIbHYqSbY2UlE9GnGjh2LQ4cO4ebNm6KjEFExmjBhAvT09LB06VLRUYiISEtwVieRZmKxU01wZicR0acxNjbG+PHj4efnJzoKERUjqVSKgIAALF26lBuTERFRidixYwdMTU3Z1UmkYVjsVBNcxk5E9OnGjx+Pffv24a+//hIdhYiKUYUKFbB06VJ4eXkhKytLdBwiIirFXs/qZFcnkeZhsVNNcBk7EdGnK1u2LMaMGYMffvhBdBQiKmYDBgxA1apVMXv2bNFRiIioFNu5cyfKli2LDh06iI5CRB+JxU41wWXsRESfZ9KkSQgNDcXdu3dFRyGiYiSRSLB+/Xps3rwZ0dHRouMQEVEpxFmdRJqNxU41wc5OIqLPY2ZmhuHDh2PR/2PvzsNjPN+3gZ+TPbKpkqpYs5GV2GltCUVKrW2CihBLKVIUEWQj9lJKayux1f5NbSVtI7GTEImQVVARam+EkG2e94++yU9qS5jMPTM5P8fhODozz/PMOWk7Mtdc933Nny86ChFVsBo1amDVqlUYMmQIcnJyRMchIiINs3PnTpiZmbGrk0hNsdipIrhnJxHRu5s4cSK2bduGrKws0VGIqIJ99tln6NixIyZNmiQ6ChERaRDu1Umk/ljsVBHs7CQienfm5uYYOnQoFi5cKDoKESnBkiVL8Mcff+DAgQOioxARkYbYtWsXTE1N8cknn4iOQkRvicVOFcE9O4mIFOPbb7/Fxo0b8ffff4uOQkQVzNTUFGFhYRg5ciTu3bsnOg4REak5uVzOvTqJNACLnSqCy9iJiBTjww8/xKBBg/Ddd9+JjkJEStChQwcMGDAAX331FSRJEh2HiIjU2K5du2BiYsKuTiI1x2KniuAydiIixZk6dSp+/vln3L17V3QUIlKC2bNnIzk5Gb/88ovoKEREpKbkcjmCg4PZ1UmkAVjsVBFcxk5EpDi1a9fGF198gSVLloiOQkRKYGBggM2bN2PChAnIzMwUHYeIiNRQcVdn165dRUchonfEYqeKYGcnEZFi+fn5YdWqVXjw4IHoKESkBC4uLvD19cXQoUMhl8tFxyEiIjVSvFdnYGAguzqJNACLnSqCe3YSESlW/fr10bt3byxbtkx0FCJSkqlTp+LJkydYsWKF6ChERKRGdu/eDSMjPSuiUAAAIABJREFUI3Tr1k10FCJSAJnEndxVQlxcHIYPH464uDjRUYiINMbly5fRunVrZGRkwMzMTHQcIlKC9PR0tGnTBsePH0ejRo1ExyEiIhUnl8vh7OyMhQsXonv37qLjEJECsLNTBdy9exeJiYnQ1tbG77//jsuXL4uORESkEaytrdG9e3csX74cAJCamoqIiAjs27cPUVFRXOJOpIFsbGwQEhICLy8vFBYWio5DREQqjl2dRJqHnZ2CSJKEmJgYZGVloXr16mjatCmMjIyQl5eH9PR0pKenw8jICK6urtDV1RUdl4hIbV24cAGDBw+Gv78/nJycYGVlBT09PTx+/Bhnz57FgwcPUL9+fTRr1kx0VCJSEEmS0K1bN3z00UcICAgQHYeIiFRUcVfnggUL4O7uLjoOESkIi50CPHnyBLt27YKrqyvq1KnzyuMeP36M/fv3o0WLFrCyslJiQiIizZCSkoLExER8+umnqFKlyiuPu3r1Ko4ePQoPDw8YGBgoMSERVZSsrCy4uLjgt99+Q/PmzUXHISIiFbRr1y4sWLAAZ86c4WAiIg3CYqeS5ebmYseOHRg8eDC0tbXLdE5ERAQsLS1hY2NTwemIiDTHpUuXcOfOHXTq1KlMxxcUFGDz5s0YOHAg9PX1KzgdESnD1q1bERISgri4OBgaGoqOQ0REKkQul6Nx48aYP38+uzqJNAz37FSy//3vf+UqdAJA165dkZCQgCdPnlRgMiIizfHgwQNkZGSUudAJALq6uhg0aBB2795dgcmISJkGDBiAxo0bw9/fX3QUIiJSMf/73/9gaGjIoUREGojFTiVKS0uDs7NzuQqdxT777DNERkZWQCoiIs1z5MgRfPrpp+U+T09PDw0aNMCNGzcqIBURibBixQrs3LkTUVFRoqMQEZGKkMvlCAkJQWBgIJevE2kgFjuVKDExEc7Ozm91rp6eHvLy8sBdB4iIXk8ul0OSpLf6YgkAWrdujdOnTys4FRGJ8v7772PNmjXw9vZGdna26DhERKQCwsPDoa+vz+XrRBqKxU4lycvLe+c94Fq1aoXY2FgFJSIi0kzHjx9H+/bt3/p8mUwGbW1tyOVyBaYiIpG6d+8Od3d3+Pr6io5CRESCyeVyBAcHIygoiF2dRBqKxU4luX379msnr5dF3bp1cfv2bQUlIiLSTNnZ2ahevfo7XaN69ersACPSMAsXLsTx48cRHh4uOgoREQnErk4izcdip5Lk5OTA2Nj4na/DZexERK+niPdJExMT5OTkKCANEakKY2NjbNy4EaNHj+aXx0RElRT36iSqHFjsVBJFfXDmGzIR0esp4n0yJycHpqamCkhDRKqkbdu2GDZsGEaMGMEvkImIKqFff/0Vurq6bzXIkojUB4udSlKzZk1kZma+0zWuXr2KWrVqKSgREZFmeu+99965a+vu3bssdhJpqKCgIFy/fh3r168XHYWIiJSIe3USVR4sdiqJnp4e8vPz3+ka0dHRaNq0qYISERFppo8++ggnTpx46/MlSYIkSdDS4l+RRJpIT08PmzZtwtSpU3H16lXRcYiISEnY1UlUefCTnBI1adIEcXFxb3Xus2fP8NNPP6Fnz56IiYlRcDIiIs0hk8kgk8lQWFj4Vufv2bMHO3bswPXr1xWcjIhUhZOTE6ZMmQJvb28UFRWJjkNERBWMe3USVS4sdiqRlZUVkpKSUFBQUO5z9+zZg0OHDsHd3R39+/dH9+7dcerUqQpISUSk/lxdXbF3795yn/fs2TNkZ2fDxsYGLi4umDJlCh4+fFgBCYlItIkTJ0KSJHz//feioxARUQXbs2cPtLW10aNHD9FRiEgJWOxUsv79+2Pz5s3l6jg6cOAAWrZsiWrVqmHMmDFIT09H7969MWDAAHTp0gXHjx+vwMREROrHzMwMDg4O+P3338t8Tl5eHrZu3YqBAwdi9uzZuHDhAh4+fIiGDRti8eLFyMvLq8DERKRs2traCAsLw7x583Dx4kXRcYiIqIJwr06iyofFTiUzMDCAp6cnfvnlF6Snp7/22AcPHmDLli1wdHREgwYNSu7X19fHqFGjkJaWBk9PT3h5ecHV1RXR0dEVnJ6ISH00bNgQlpaW2Lp1K7Kzs197bHJyMnbs2IFBgwZBV1cXAGBhYYE1a9YgOjoa0dHRaNSoEbZs2QK5XK6M+ESkBJaWlpg7dy4GDx78znurExGRatq7dy+7OokqGZkkSZLoEJVVQkICMjIyYGpqCmdnZ5iZmeHJkye4fPkyMjMzUa1aNbRv3x7a2tqvvU5BQQG2bNmC0NBQ1KpVCwEBAXB1deW3VkREAAoLCxEdHY3s7GzUr18flpaWMDQ0RHZ2Ns6fP48nT57Azs4O9vb2r73OkSNHMHnyZBQWFmLBggXo3Lmzkl4BEVUkSZLw2WefoXHjxpg9e7boOEREpECSJKFp06YIDg7GZ599JjoOESkJi50qIDs7GykpKcjOzoaRkRHq1auH2rVrl/s6hYWF2LZtG2bPno33338fgYGB6NKlC4ueRET/3/Xr13H9+nXk5ubiq6++wq+//gpnZ+cyny9JEnbt2oVp06bB2toa8+fPR+PGjSswMREpw99//40mTZogPDwcbdq0ER2HiIgU5Ndff0VISAjOnTvHz8VElQiLnRqoqKgIO3bswKxZs2BqaoqAgAB0796db+5ERM/p3Lkzvv32W3Tr1q3c5+bn52PVqlUIDQ1F165dMWvWLNStW7cCUhKRsuzevRt+fn6Ij4+HkZGR6DhERPSOirs6g4KC0KtXL9FxiEiJuGenBtLW1saAAQOQmJiIiRMnYurUqWjZsiX27dsH1raJiP5la2v7xr2TX0VPTw/jxo1DWloa6tSpAxcXF0ydOhX//POPglMSkbL069cPbdq0wZQpU0RHISIiBdi7dy8AcPk6USXEYqcG09bWxhdffIGEhAT4+flhxowZaNasGcLDwzlgg4gqPRsbm7cudhYzNTUtmdz+4MED2NracnI7kRpbtmwZ9u3bh4iICNFRiIjoHUiShKCgIE5gJ6qkWOysBLS0tNCvXz+cP38egYGBmD17NlxcXLBr1y4WPYmo0lJEsbNY8eT2qKgoREVFcXI7kZqqWrUq1q9fDx8fHzx48EB0HCIiekvs6iSq3LhnZyUkSRIOHDiAkJAQ5ObmYubMmejfv/8bp74TEWmS1NRUfPrpp7h8+bLCr/385PaFCxfCzc1N4c9BRBXH19cXd+7cwdatW0VHISKicpIkCc2aNUNAQAB69+4tOg4RCcBiZyUmSRIiIiIQHByM7OxszJgxAx4eHix6ElGlkJ+fD1NTU+Tk5EBXV1fh139+cruNjQ3mz59frsnvRCTO06dP0bRpUwQGBsLT01N0HCIiKoe9e/ciMDAQcXFxXMJOVElxGXslJpPJ0K1bN5w8eRJLly7Fjz/+CHt7e2zcuBGFhYWi4xERVSg9PT1YWFjg6tWrFXJ9mUyGzz//HElJSXB3d0eXLl3g7e2N69evV8jzEZHiGBoaYuPGjfD19cXNmzdFxyEiojIq3qszMDCQhU6iSozFToJMJkOXLl1w7Ngx/PTTT1i3bh0aNWqE9evXo6CgQHQ8IqIKY2Njg7S0tAp9juLJ7enp6ahduzYntxOpiRYtWmD06NEYNmwYuBCKiEg97Nu3D5IkoVevXqKjEJFAXMZOZZKfnw89PT3RMYiINIa5uTn8/Pzw9ddfQ19fX3QcInqJgoICtG3bFj4+Pvjqq69ExyEioteQJAnNmzfHjBkz0KdPH9FxiEggdnZSmdjY2GDlypXIy8sTHYWISCM8P7n9l19+4eR2IhWkq6uLTZs2YebMmUhPTxcdh4iIXmP//v0oKipiVycRsdhJZbN9+3bs3bsX1tbWWL58OZ49eyY6EhGRWnNwcMC+ffsQFhaG77//Hi1atEBkZKToWET0H40aNcLMmTMxZMgQ7mlORKSiJEnCnDlzEBgYCC0tljmIKjsuY6dyiY2NxaxZs3Du3DlMmTIFI0eOhKGhoehYRERqTZIk7Ny5E9OmTYOtrS0ntxOpGLlcji5duqBz586YNm2a6DhERPQfkiRBLpdDJpOx2ElE7Oyk8mnRogX27t2Lffv2ITo6GlZWVli8eDGePHkiOhoRkdqSyWT44osvkJycXGpye2ZmpuhoRARAS0sL69evx5IlSxAfHy86DhER/YdMJoO2tjYLnUQEgMXOcpHJZNi1a9c7XSMsLAzGxsYKSiRO06ZNER4ejt9++w0nT56ElZUVFixYgMePH4uORkQarH79+li0aFGFP4+o9+r/Tm5v0qQJJ7cTqYi6deviu+++w+DBg7mdDxEREZEKY7ET/xYxX/fH29sbAHDr1i307NnznZ7Lw8MDV65cUUBq1dCkSRPs2rULf/75J+Li4mBlZYW5c+fi0aNHoqMRkZrx9vYued/V0dFB3bp1MXr0aDx8+LDkmNjYWIwZM6bCs4h+rzY1NcXs2bNx4cIF3L9/H7a2tliyZAmHxBEJ9uWXX8LW1hYzZ84UHYWIiIiIXoF7dgL4+++/S/55//79GDFiBG7dulVyn6GhIczMzEREqxD5+fnQ09OrkGsnJSUhNDQUv//+O3x9fTFu3DiN+tkRUcXx9vZGVlYWNm3ahMLCQiQlJWHYsGFo164dtm7dKjqeUJcuXYKfnx8uXryI0NBQeHp6cpkWkSB3795F48aNsW3bNrRv3150HCIiIiL6D35SAlCzZs2SP1WrVn3hvuJi3fPL2K9duwaZTIZt27ahQ4cOMDQ0hIuLCy5cuICLFy+ibdu2MDIywscff4yrV6+WPNd/l0ZmZmaiV69eqFatGqpUqYJGjRph27ZtJY8nJiaic+fOMDQ0RLVq1eDt7Y3s7OySx2NjY/HJJ5+gevXqMDU1xccff4xTp06Ven0ymQwrVqxA3759YWRkBH9/fxQVFcHHxwcNGjSAoaEhbGxssGDBAsjl8nf6Wdrb22PLli04fvw40tPTYW1tjeDg4FKdWUREr6Kvr4+aNWuidu3a+OSTT+Dh4YHff/+95PH/LmOXyWT46aef0KtXL1SpUgW2traIiorCjRs30LVrVxgZGaFJkyaIi4srOaf4fTgyMhKOjo4wMjJCp06dXvteDQAHDhxAq1atYGhoiPfffx89e/YsWcr6suX1HTt2xNixYxXyc+HkdiLVUaNGDaxatQre3t7IyckRHYeIqNJhvxYRvQmLne8oMDAQU6dOxfnz51G1alUMHDgQ48aNQ2hoKGJiYvDs2TOMHz/+leePGTMGubm5iIqKwqVLl/D999+XFFxzc3PRrVs3GBsbIyYmBuHh4Th58iSGDRtWcn5OTg4GDx6MY8eOISYmBk2aNIG7uzvu3btX6nmCg4Ph7u6OxMREfP3115DL5bCwsMCOHTuQnJyM0NBQzJkzB+vXr1fIz6Vhw4bYsGEDTp06hb/++gs2NjaYOXMm7t+/r5DrE5Hmu3LlCg4dOgRdXd3XHjd79mx4enoiISEBzZs3x4ABA+Dj44MxY8bg/PnzqFWrVsl2JMXy8vIwd+5crFu3DqdOncI///yDr7766pXPcejQIfTq1QtdunTBuXPnEBUVhQ4dOrzzF0Tl1aFDB5w5cwZTp07FyJEj0b17d1y4cEGpGYgI6NmzJ1xdXTFhwgTRUYiIKoXnC5wymQwAlP57GBGpEYlK2blzp/SqHwsAaefOnZIkSdLVq1clANLKlStLHt+3b58EQNq9e3fJfevXr5eMjIxeedvJyUkKCgp66fOtXr1aMjU1lR49elRyX1RUlARASk9Pf+k5crlcqlmzprRp06ZSuceOHfu6ly1JkiRNnTpVcnNze+NxbyMjI0MaPny4VK1aNWnatGnS3bt3K+R5iEh9DRkyRNLW1paMjIwkAwMDCYAEQFq8eHHJMfXq1ZMWLlxYchuA5OfnV3I7MTFRAiB99913JfcVv28Wv++sX79eAiClpKSUHLN582ZJV1dXKioqKjnm+ffqtm3bSh4eHq/M/t9ckiRJHTp0kL7++uvy/hjKLC8vT1q2bJlkbm4ueXt7S9evX6+w5yKiFz169Ehq0KCBtHfvXtFRiIg03rNnz6Tjx49LI0aMkGbOnCnl5uaKjkREKoydne/I2dm55J8/+OADAICTk1Op+548eYLc3NyXnu/r64vZs2ejTZs2mDFjBs6dO1fyWHJyMpydnWFiYlJyX9u2baGlpYWkpCQAwJ07dzBq1CjY2trCzMwMJiYmuHPnDq5fv17qeZo3b/7Cc69cuRLNmzdHjRo1YGxsjCVLlrxwnqJYWlpizZo1iIuLw4MHD2Bra4spU6bgzp07FfJ8RKSe2rdvj/j4eMTExGDcuHFwd3d/bXc8ULb3YQCl3m/09fXRsGHDktu1atVCQUHBK6eenz9/Hm5ubuV/QRWoeHJ7WloaatWqhSZNmsDPz4+T24mUxMTEBBs2bMCoUaNw9+5d0XGIiDRaaGgoRo8ejQsXLmDLli1o2LBhqc/ORETPY7HzHT2/vLK4nf5l972qxd7HxwdXr17F0KFDkZaWhrZt2yIoKAjAv636xef/V/H9Q4YMQWxsLJYsWYKTJ08iPj4etWvXRn5+fqnjjYyMSt3evn07vvnmG3h7eyMiIgLx8fEYM2bMC+cpWr169bBy5UokJCQgNzcXjRo1wqRJk0oNiSKiyqtKlSqwtraGk5MTli1bhtzcXMyaNeu157zN+7COjk6pa7zrcigtLa0X9o8qKCh4q2uVl5mZGUJDQ3HhwgXcu3ePk9uJlKhdu3b48ssvMWrUKO4hR0RUQW7duoXFixdjyZIliIiIwMmTJ1GnTp2SAZaFhYUAuJcnEf0fFjtVQO3atTFy5Ejs2LEDISEhWL16NYB/h/0kJCSU2vz+5MmTkMvlsLOzAwAcP34c48aNw6effgoHBweYmJiUmiT/KsePH0erVq0wduxYNG3aFNbW1sjIyKiYF/gSderUwfLly5GYmIjCwkLY29vjm2++wc2bN5WWgYhUX2BgIObPny/8vcHFxeW1A4Fq1KhR6r332bNnSElJUUa0EhYWFli7di2ioqJw+PBhNGrUCL/88gv3syKqYCEhIUhPT8fmzZtFRyEi0khLliyBm5sb3NzcYGZmhg8++ACTJ0/Grl27kJOTU/Il9qpVq7iXOREBYLFTOF9fXxw6dAhXrlxBfHw8Dh06BHt7ewDAoEGDYGRkBC8vLyQmJuLo0aMYNWoU+vbtC2trawCAra0tNm/ejKSkJMTGxsLT0xN6enpvfF5bW1vExcXh4MGDSE9Px6xZs3DkyJEKfa0vY2FhgaVLl+LSpUvQ1taGo6Mjxo4dixs3big9CxGpno4dO8LBwQGzZ88WmmP69OnYuXMnZsyYgaSkJFy6dAlLliwp2aLE1dUVW7ZsQXR0NC5duoRhw4YprbPzv4ont69fv75kcvvhw4eFZCGqDAwMDLBp0yZMmjSpwrYDIiKqrPLz85GVlQUbGxsUFRUBAIqKiuDq6gp9fX2Eh4cDANLT0zFmzJhSW8ARUeXFYqdgcrkc48aNg729Pbp06YIPPvgAGzZsAPDvcs6IiAg8evQILVu2RK9evdCmTRusW7eu5Px169bh8ePHaNasGTw9PTFs2DDUr1//jc87atQofPHFFxg4cCBatGiBa9euYdKkSRX1Mt/oww8/xHfffYeUlBRUqVIFzs7OGD16NP766y9hmYhINUycOBE///yz0PcDd3d3hIeH4+DBg3BxcUGHDh0QFRUFLa1//xqdNm0aXF1d0atXL3zyySf4+OOP0bRpU2F5gX8LxcWT20eMGMHJ7UQVqEmTJpgwYQKGDh3KbmoiIgXS09ODp6cnrK2toa2tDQDQ1taGqakpPvroI+zbtw8A4O/vj88++wwNGjQQGZeIVIRM4sYWpILu3r2LxYsXY/Xq1ejbty/8/f3L9BdXUVERkpKSULduXZiZmSkhKRGR6svPz8eqVaswe/ZsuLu7IyQkBHXq1BEdi0ijFBYWon379vDw8ICvr6/oOEREGqN4tYyurm6puRZRUVEYNWoUdu7ciWbNmiE1NRVWVlYioxKRimBnJ6mkGjVqYO7cuUhLS0PNmjXRvHlzDBs2DA8fPnzteUlJSVi4cCHatWuHESNGvPF4IqLKgJPbiSqejo4ONm7ciFmzZiE5OVl0HCIitVf8e4quru4Lhc78/Hy0adMG1apVQ8uWLdG3b18WOomoBIudpNLef/99zJo1C5cvX0bdunVhbGz82uNr164NT09PfP311/j555+xZMkSPHv2TElpiYhUGye3E1Usa2trzJ49G15eXsL27SUi0gQPHjzA6NGjsXHjRly7dg0ASgqdwL9f5BoYGMDBwQEFBQVYuHChoKREpIpY7CS18N577yEoKKhk0t7rjnN3d8eDBw9gZWWFbt26wcDAoORxfvAgIvq/ye2HDx9GZGQk7OzsOLmdSEFGjRqF6tWrIzQ0VHQUIiK1tX79emzfvh3ff/89Jk+ejC1btiAzMxPAv1PXi4cVzZ07F3v37kW9evVExiUiFcM9O0ljPL+s4cMPP8TgwYMREBBQ0g16/fp17Ny5E7m5uRg8eHCZBjkREVUG0dHRmDJlCoqKirBw4UK4urqKjkSk1m7evAkXFxfs378fLVq0EB2HiEjtnDx5Er6+vvDy8sKePXuQkpICNzc3aGtrY/fu3bhx4wYnrxPRK7GzkzRG8bd7CxcuhLa2Nvr06VNq2fuDBw9w584dnDp1CpaWlli8eDG7mIiI8OLkdnd3dyQmJoqORaS2atWqhWXLlmHw4MHIzc0VHYeISO20bdsWrVu3xtOnT/Hnn39i6dKluH79OjZv3gxLS0scPHgQGRkZomMSkYpisZM0RvES9++//x4eHh5wdHQs9XiTJk0QGhqKoKAgAICpqamyIxKRClu3bh28vLxExxBGJpPhiy++QHJyMrp164bOnTtj6NChJUvGiKh8PDw80LRpU0ybNk10FCIitTRx4kQcOnQImZmZ6NevH7y9vWFiYoIqVapgwoQJmDRpEr9QIqKXYrGTNEJxh+aSJUsgSRL69u37wrKGoqIi6OjoYM2aNXB2dkavXr2gpVX6f4GnT58qLTMRqRZbW1ukp6eLjiGcnp4exo8fz8ntRAqwfPly7N69G5GRkaKjEBGplaKiIjRo0AAffvghAgMDAQDTpk3DnDlzcOLECSxevBitW7dGlSpVBCclIlXEPTtJrUmShMjISBgZGaFNmzaoV68e+vTpg1mzZsHExKTUPp7Av/t2WltbY+XKlRg2bFjJNWQyGa5evYqff/4Z+fn58PLyeqEzlIg02+3bt+Hg4IB79+6JjqJSsrKyEBgYiL1792LatGkYM2YM9PX1RcciUhsREREYMWIELly4gKpVq4qOQ0Sk8p7/DJeamoqJEyeiVq1a2L9/PxISEmBubi44IRGpOnZ2klorLnZ+9NFHsLKywqNHj9CvX7+Srs7ivySLOz9DQ0Nha2uLHj16lFyj+JgHDx5AJpMhOTkZzs7OnKJKVMmYm5sjPz8fDx8+FB1FpbxscvvWrVu55zFRGXXt2hU9e/bE+PHjRUchIlJpxavsnv8M17BhQ7Ru3RphYWHw9/cvKXTy9xAieh0WO0mtaWlpYe7cuUhLS0PHjh2RnZ2NadOm4fz586X+AtTS0kJWVhbCwsLg6+v70m8DmzVrhoCAAPj6+gIAHBwclPY6iEg8mUwGGxsbLmV/BUdHR+zfvx/r1q3D4sWL0bJlSxw+fFh0LCK1sGDBApw+fRq7d+8WHYWISCVlZ2cjODgY0dHRyM7OBoCSLcd8fHywdu3akr3VJUl6YTsyIqLncRk7aZRr165hypQpMDIywpo1a/DkyRNUqVIFurq6GDNmDKKiohAVFYWaNWuWOu/5pRJffvklUlNTERsbK+IlEJFAnp6e6NmzJwYNGiQ6ikqTy+XYuXMn/P390bBhQ8yfPx9OTk6iYxGptNOnT6N3796Ij49/4fcQIqLKbvTo0Vi1ahXq1q2Lnj174osvvoCzszPMzMxKHZeXl8ftdIjojfh1CGmU+vXrY8eOHfjpp5+gra2N0NBQdOrUCdu3b8emTZswceLEl37AKC50njt3Djt27IC/v7+yoxORCrCxsUFaWproGCpPS0sLHh4enNxOVA6tW7fG8OHDMWLECLDXgIjo/+Tk5OD06dNYuXIlJk2ahD179uDzzz/HjBkzcOTIkZIthi5evIiRI0fiyZMnghMTkapjsZM0koGBAWQyGb799lvUqFEDX375JZ48eQJDQ0MUFRW99By5XI6lS5fCwcEBffr0UXJiIlIFXMZePi+b3D5t2jRObid6hYCAANy7dw+3b98WHYWISGVkZmaiadOmqFmzJsaNG4fr169j5syZ2Lt3L7744gsEBATg6NGj8PX1xcOHD2FkZCQ6MhGpOC5jp0rh/v37mD59OlavXo2xY8ciJCTkhYmo8fHxaNWqFbZs2YL+/fsLSkpEIp0+fRrjxo3jNhZv6caNGwgMDMS+ffvg7++P0aNHc6kZ0X/I5XLIZLKSVSVERJWdXC5Heno6Pvjggxc+o61YsQKLFi3CP//8g+zsbKSmpsLGxkZQUiJSFyx2UqVy7949xMTEoGvXrtDW1sbNmzdhbm4OHR0dDB06FOfOnUNCQgI/gBBVUvfv34eVlRUePnzI94F3cPHiRfj5+SEpKQmhoaHw8PDgIAEiIiIqs8LCQujo6JTcLp7KvmHDBoGpiEhdsNhJlVZ2djYmT56Ms2fPYtCgQQgKCsL69evZ1UlUyVWrVg2pqamoUaOG6ChqLzo6GpMnT4YkSViwYAFcXV1FRyJSefn5+Vi6dCksLS3Rr18/0XGIiISSy+WIjY1FmzZtkJycjIYNG4qORERqgG0WVGmZmZlh8eLFaNq0KQICAvDkyRMUFBSBTD5bAAAgAElEQVTg6dOnrzxHkiTI5XIlpiQiZeO+nYrTsWNHnDlzBpMnT8aIESPg7u6OxMTEMp3L72KpssrMzER6ejpmzpyJAwcOiI5DRCSUlpYWHj9+jKlTp7LQSURlxmInVWrGxsZYu3Yt7t27h8mTJ2PQoEGYNm0aHj9+/MKxkiThzJkzcHJywtatW1856IiI1BuLnYr1ssntw4YNe+Mk1YKCAjx8+BAxMTFKSkokniRJsLKywtKlS+Ht7Y0RI0YgLy9PdCwiogonSdIrv+h0dXVFaGiokhMRkTpjsZMIgKGhIebPn4/c3FwMGjQIhoaGLxwjk8nQqlUrLF68GD/88AMcHBywefNmFBYWCkhMRBXFxsYGaWlpomNonOcnt1taWr70ffZ5Y8aMQbt27TBq1CjUr18f69evV1JSIuWTJKnU7xMGBgaYPHkyLC0t8dNPPwlMRkSkHFFRUfjtt99eWvCUyWTc+5uIyoXvGETPMTAwQIsWLaCtrf3Sx2UyGbp27YoTJ05gxYoVWL16Nezt7bFhwwYWPYk0BDs7K5aZmRlmzJjx2gFQP/74I7Zu3YoxY8Zgx44dCAgIQGhoKA4ePAiAS9xJM8jlcty8eRNFRUWQyWTQ0dEp+f+ieFp7bm4uTExMBCclIqpYkiQhICAA//zzDwdEEpFC6Lz5ECL6L5lMBjc3N7i5uSE6OhohISEICQmBv78/vLy8oKurKzoiEb0lW1tbFjuV4HUfZlauXInhw4djzJgxAP4tQJ89exZr1qxBt27dIJPJkJqayr27SG0VFBSgXr16uH37Ntq1awcjIyM0b94cLi4usLCwQLVq1bBp0ybEx8fDwsJCdFwiogp1+PBh3L17F56enqKjEJGGYGcn0Tvq2LEjDh8+jLCwMGzbtg22trZYvXo18vPzRUcjordgY2ODy5cvs3tQkPz8fFhZWZXs6Vn870GSpJLOt8TERNjZ2aFHjx7IzMwUGZforejq6mLixImQJAnjxo2Do6Mjjh49ilmzZqFHjx5o2bIl1q5dix9++AHdunUTHZeIqMJIkoSgoCAEBAS8cnUdEVF5sdhJpCDt2rXDH3/8gS1btiA8PBzW1tb48ccfOViASM2YmZnB0NAQf//9t+golZKenh46dOiAXbt2Yffu3ZDJZDhw4ABOnDgBMzMzFBUVwcnJCRkZGTA1NUW9evXg4+ODp0+fio5OVC7ffvstHB0dERkZifnz5+Pw4cM4d+4cUlNT8eeffyIjIwOjRo0qOT4rKwtZWVkCExMRKd7hw4dx584ddnUSkUKx2EmkYG3btsXBgwexc+dO/Pbbb7CyssIPP/yAZ8+eiY5GRGXEfTvFKO7i/OabbzBv3jyMGjUKrVq1gq+vLy5evAhXV1doa2ujsLAQDRo0wC+//IKzZ88iPT0dVatWxaZNmwS/AqLy2bt3L37++Wfs2bMHMpkMRUVFqFq1KlxcXKCvrw8dnX93nLp37x42bNgAPz8/FjyJSGMUd3XOnDmTXZ1EpFAsdhJVkFatWmH//v3Ys2cP/vzzT1hZWeH7779Hbm6u6GhE9AYsdipfYWEhIiMjcevWLQDAV199hXv37mH06NFwdHREmzZtMGDAAAAoKXgCwIcffgg3NzcUFBQgMTGR3fSkVurXr485c+bA29sbjx8/fuWH/erVq6NFixbIzc2Fh4eHklMSEVWMqKgodnUSUYVgsZOogjVr1gx79uzB/v37cezYMVhZWWHRokUl+9ERkephsVP57t+/j61btyIkJASPHj1CdnY2ioqKEB4ejszMTEydOhXAv3t6Fk+ufvDgAfr27Yt169Zh3bp1WLBgAfT19QW/EqLymTRpEiZMmICUlJSXPl5UVAQA6Ny5M4yNjXHy5ElERkYqMyIRkcI939VZ3MVORKQoLHYSKYmLiwt2796NiIgIxMTEwNLSEvPnz0dOTo7oaET0HzY2NkhLSxMdo1L54IMPMHr0aJw4cQL29vbo3bs3atWqhStXriAgIACfffYZAJR8INqzZw+6d++O+/fvY9WqVfD29haYnujdzJgxA82bNy91X/G2Dtra2oiPj0fTpk0RERGBlStXwsXFRURMIiKFiYqKwu3bt9nVSUQVQiZx3CyREJcuXUJoaCj+/PNPfPPNNxg7dixMTU1FxyIiAOfPn4eXlxcSExNFR6mUDhw4gIyMDNjZ2aFZs2aoVq1ayWP5+fmIiIiAj48PnJycsGrVKlhbWwP4tzgkk8lExSZ6Z+np6TAzM4O5uXnJffPnz8fMmTPh5uaGuXPnwtnZGVpa7FcgIvUlSRI6duyI4cOHY/DgwaLjEJEGYrGTSLCUlBSEhobi0KFDGD9+PMaNG4eqVauKjkVUqT1+/Bjm5uZ4/PgxiwqCyeXyUv8OZsyYgVWrVqFHjx4ICgpCvXr1XjiGSF0tW7YMO3bswPHjx3Ht2jV4eXkhLi4OgYGB8PHxKVX453/3RKSuoqKiMGrUKCQlJXEJOxFVCBY7iVREeno6QkNDsX//fnz99dfw9fUt9aGGiJSrVq1aOHPmDOrUqSM6CgHIzMzEhAkTEBERgZEjR+K7774THYlI4QoLC1G1alW0adMGsbGxcHR0xIIFC9CqVatXDi96+vQpDA0NlZyUiOjtsKuTiJSBXwcTqQgbGxuEhYXhzJkzyMrKgq2tLWbMmIH79++LjkZUKXFIkWoxNzdHzZo1sXbtWsybNw/A/w1u+S9Jkl75GJEq09HRwb59+xAZGYmePXvi119/Rdu2bV9a6Hz8+DF++uknLF26VEBSIqK3Ex0djZs3b2LAgAGioxCRBmOxk0jFWFlZYe3atYiNjcXdu3dha2sLPz8/3L17V3Q0okqFxU7Voq+vj+XLl8PDwwO6uroA8MpONwDo2LEjli5diry8PGVFJFKITp06YeTIkTh27Nhrl3caGxtDX18f+/btw/jx45WYkIjo7QUHB3MCOxFVOBY7iVRUgwYNsGrVKpw/fx6PHj1Cw4YNMXnyZNy+fVt0NKJKgcVO9SWTyfDjjz/i999/h52dHbZt2wa5XC46FlGZrVy5EhYWFoiOjn7tcQMGDEDPnj2xfPnyNx5LRCRadHQ0srKyMHDgQNFRiEjDsdhJpOLq1q2LH3/8ERcuXEBeXh7s7OwwYcIE3Lp1S3Q0Io1mY2ODtLQ00THoLTk5OeHAgQP4+eefsWjRIrRq1QpRUVGiYxGVWfES9lfJzs7G0qVLERoaii5dusDKykqJ6YiIyi8oKIhdnUSkFCx2EqmJ2rVrY9myZbh06RIAwMHBAePHj0dWVpbgZESaiZ2dmqFTp06IiYnBpEmT4OPjg08//RQXL14UHYvojWrUqAFzc3Pk5ubi2bNnpR5LSEhA7969ERISgtmzZyMiIoLD1IhIpbGrk4iUicVOIjXz4YcfYsmSJUhKSoKenh6cnJzw9ddf4/r166KjEWkUa2trXLt2jYNuNICWlhY8PT2RnJyMTz75BG5ubhg2bBhu3LghOhrRG23atAmzZ8+GJEl49uwZli9fjvbt2yMvLw8xMTHw9fUVHZGI6I2Cg4MxY8YMdnUSkVKw2EmkpmrWrIlFixYhJSUFJiYmcHFxwahRo3Dt2jXR0Yg0gqGhIWrUqMEvEjSIvr4+fH19kZaWhpo1a6Jx48bw9/dHdna26GhEr9SpUyfMmTMHixYtwqBBgzBhwgRMnDgRx44dg6Ojo+h4RERvFB0djczMTAwaNEh0FCKqJFjsJFJz5ubmmDdvHlJTU1G9enU0a9YMw4cPx5UrV0RHI1J7XMqumczMzDBnzhwkJCTg77//hq2tLZYuXYr8/HzR0YheYGtri0WLFmHq1KlISkrC8ePHERgYCG1tbdHRiIjKhBPYiUjZWOwk0hDVq1dHaGgo0tPTYWFhgZYtW2Lo0KEs1BC9AxY7NVvt2rWxbt06/PnnnyWT27dv387J7aRyJk6ciM6dO6Nu3bpo1aqV6DhERGV25MgRdnUSkdKx2EmkYapVq4bg4GBcvnwZDRo0QNu2beHl5YXU1FTR0YjUDoudlUPx5Pa1a9di4cKFnNxOKmn9+vWIjIzEgQMHREchIioz7tVJRCKw2EmkoapWrYqAgABkZGSgUaNGaNeuHQYOHIikpCTR0YjUho2NDdLS0kTHICXh5HZSZRYWFjh16hTq1asnOgoRUZkcOXIE169fx5dffik6ChFVMix2Emk4U1NT+Pv7IyMjA40bN0anTp3g4eGBxMRE0dGIVB47Oyuf5ye3d+nSBa6urvDx8eHkdlIJLVq0eOlQIkmSBKQhInq94OBgTJ8+nV2dRKR0LHYSVRImJiaYOnUqMjIy0KJFC3Tp0gX9+vVDfHy86GhEKsvS0hKZmZkoKCgQHYWUTF9fH9988w3S0tJgbm7Oye2ksiRJwpEjR/DXX3+JjkJEVOLo0aP466+/2NVJREKw2ElUyRgbG+Pbb7/FlStX8PHHH8Pd3R29e/fGuXPnREcjUjn6+vqoVasWrl27JjoKCVK1alXMnTuXk9tJZclkMpw5cwbe3t4crkVEKqN4r05dXV3RUYioEpJJXPdCVKk9ffoUa9euxfz58+Hi4oKZM2eiZcuW5bpGYmIiMjIyoK2tXbKUTltbG25ubjAwMKiI2ERK07VrV/j6+sLd3V10FFIBiYmJ8PPzQ0pKCubMmYPPP/8cWlr87pjEKioqQocOHdC/f3988803ouMQUSV39OhRDB06FCkpKSx2EpEQLHYSEQDg2bNnWLduHebNmwcHBwcEBASgTZs2rz0nMjIS//zzDxwdHdGwYcNSjz19+hSHDx/G06dP0b59e5ibm1dkfKIKM3bsWNjY2MDX11d0FFIhhw8fxpQpUyCTybBw4UJ07NhRdCSq5DIyMtC6dWscOXIE9vb2ouMQUSXm5uaGQYMGYdiwYaKjEFElxWInEZWSl5eHDRs2YM6cObC1tUVAQAA+/vjjUsfI5XJs3boVbm5uqFmz5muvJ0kS9uzZAwcHB9jY2FRkdKIKsXTpUqSnp2P58uWio5CKkcvl2L59O6ZPnw57e3vMmzfvpcNjiJRl9erVWLVqFU6fPs1uKiIS4tixYxgyZAhSU1P5PkREwnDdFRGVoq+vj5EjRyItLQ0eHh7w8vKCq6srjhw5UnLMtm3b8Nlnn72x0An8u5dY7969kZaWxmnGpJY4kZ1eRUtLCwMGDEBycjI6d+4MNzc3Tm4noUaMGIGaNWti1qxZoqMQUSXFvTqJSBWw2ElEL6WnpwcfHx+kpqbCy8sLw4cPR4cOHbBixQq0a9cOJiYm5brep59+imPHjlVQWqKKY2Njg7S0NNExSIUVT25PTU3l5HYSSiaTYe3atVi1ahXOnDkjOg4RVTLHjx/HlStXMHjwYNFRiKiSY7GTiF5LV1cX3t7eSE5OxogRI5CYmIg6deq81bUcHByQmpqq4IREFat+/fq4efMm8vLyREchFVc8uT0+Pr5kcvuyZcs4uZ2U6sMPP8Ty5cvh5eWF3Nxc0XGIqBIJDg7G9OnT2dVJRMKx2ElEZaKjo4OPP/74nTYad3Z2RmJiogJTEVU8XV1d1KtXD1euXBEdhdREnTp1sG7dOvzxxx84dOgQ7OzssH37dnCbdFKWzz//HC1atMDUqVNFRyGiSuL48eO4fPkyvLy8REchImKxk4jKLj4+Hi1atHina+jo6CgoDZHycN9OehvOzs747bffsGbNGixcuBCtWrVCdHS06FhUSfzwww/49ddf8ccff4iOQkSVAPfqJCJVwmInEZWZtrY2ZDLZO11DR0cHcrlcQYmIlIPFTnoXrq6uiImJwYQJEzBs2DD06NEDFy9eFB2LNNx7772HdevWwcfHBw8fPhQdh4g02IkTJ9jVSUQqhcVOIiozRSzB1NLSYrGT1A6LnfSu/ju53dXVFT4+PsjKyhIdjTRYly5d0KtXL4wbN050FCLSYNyrk4hUDYudRKRUBQUFXMpOaofFTlKU4sntaWlpMDc3h7OzM6ZPn87J7VRh5s+fj9jYWOzcuVN0FCLSQCdOnEB6ejq7OolIpbDYSURlVrt27Xce0lJQUKCgNETKY2Njg7S0NNExSIM8P7n91q1bnNxOFaZKlSrYtGkTxo0bh1u3bomOQ0QaprirU09PT3QUIqISLHYSUZk1bdoUcXFxb31+VlYWLCwsFJiISDnq1q2Lu3fvIjc3V3QU0jCc3E7K0LJlS4wcORLDhw/nf1tEpDAnT55EWloauzqJSOWw2ElE5WJgYPDWBZ9Tp06hdevWCk5EVPG0tbVhaWmJjIwM0VFIQz0/uX3BggWc3E4KN3PmTPz9999Ys2aN6ChEpCHY1UlEqorFTiIql65du2L79u3lHjIUGxuLBg0avPM0dyJRuG8nKYOrqytiY2MxYcIEDB06FD169MClS5dExyINoKuri02bNsHf359f3BDROzt58iRSU1MxZMgQ0VGIiF7AYicRlYuuri769euHjRs3lnn/zZiYGOTl5aFZs2YVnI6o4rDYScpSPLk9JSUFnTt3RqdOnTi5nRTC3t4e06dPx5AhQ1BUVCQ6DhGpMXZ1EpEqY7GTiMrN1NQUAwYMQHh4OA4ePPjKgRrJycnYtWsX9PT08PHHHys5JZFisdhJyvb85PYaNWpwcjsphK+vL3R1dbFo0SLRUYhITZ06dYpdnUSk0mQSdyknonfw+PFjHD58GEVFRdDW1sbVq1dhZmYGY2NjNGrUCI6OjqIjEinE4cOHERwcjCNHjoiOQpVUZmYmAgIC8Ntvv2H69On46quv2FFDb+Wvv/5C8+bNERkZCWdnZ9FxiEjNdOvWDX379sXIkSNFRyEieikWO4lIoQYMGICePXti4MCBoqMQKVRmZiZatmyJW7duiY5CldyFCxfg5+eH1NRUzJ07F59//jn3Q6ZyCwsLw+LFixEbGwt9fX3RcYhITZw6dQqenp5IT0/nF25EpLK4jJ2IFOq9997Dw4cPRccgUjgLCwtkZ2cjJydHdBSq5J6f3D5//nxObqe3MmTIEFhZWSEwMFB0FCJSI8HBwfD392ehk4hUGoudRKRQLHaSptLS0oK1tTUuX74sOgoRAE5up3cjk8mwatUqbNiwAcePHxcdh4jUwOnTp5GcnIyhQ4eKjkJE9FosdhKRQrHYSZqMQ4pI1Tw/ud3NzQ2dOnXC8OHDObmdysTc3BwrV67EkCFD2LVORG/Erk4iUhcsdhKRQrHYSZqMxU5SVfr6+pgwYQLS0tJQvXp1Tm6nMuvVqxc6dOiAb7/9VnQUIlJhp0+fRlJSErs6iUgtsNhJRArFYidpMhY7SdVVrVoV8+bNQ3x8PG7evAlbW1ssW7YM+fn5oqORCvv+++/x+++/48CBA6KjEJGKCg4OxrRp09jVSURqgcVOIlIoFjtJk7HYSeqiTp06WL9+Pf744w8cOnQIdnZ22LFjByRJEh2NVJCpqSnCwsIwcuRI3Lt3T3QcIlIxZ86cwaVLl9jVSURqg8VOIlIoFjtJk7HYSeqmeHL76tWrSya3HzlyRHQsUkEdOnSAp6cnRo8ezaI4EZVSvFenvr6+6ChERGUik/jbDBERUZlIkgRTU1NkZmaiatWqouMQlYtcLsf27dvh7+8PR0dHzJs3Dw4ODqJjkQp59uwZmjVrBn9/fwwaNEh0HCJSATExMejfvz/S09NZ7CQitcHOTiIiojKSyWTs7iS19fzkdldXV05upxcYGBhg06ZNmDBhAm7cuCE6DhGpgOK9OlnoJCJ1wmInERFRObDYSeqOk9vpdZo2bYrx48dj6NChkMvlouMQkUAxMTFITEzEsGHDREchIioXFjuJiIjKgcVO0hQvm9z+ww8/cHI7wc/PDzk5Ofjxxx9FRyEigdjVSUTqisVOIiKicmCxkzTN85PbDx48CHt7e05ur+R0dHSwceNGBAUFITU1VXQcIhIgJiYGFy5cYFcnEaklDigiIpUSFBSEXbt24eLFi6KjEL3UyZMnMWHCBJw5c0Z0FKIKERkZiSlTpkBHRwcLFixAhw4dynxuXFwcrl+/Di2tf79Pl8vlaNSoERo1alRRcakCrVixAhs3bsSJEyego6MjOg4RKVGPHj3g7u6OMWPGiI5CRFRuLHYSUQlvb2/cu3cP+/fvF5bh8ePHyMvLw/vvvy8sA9Hr3L17F7a2tnjw4AFkMpnoOEQVQi6XY9u2bZg+ffobJ7cXFhbi0KFDyMvLg4uLCywtLUs9fvHiRaSkpMDU1BRdunTh/zdqRJIkdO3aFe3atcPMmTNFxyEiJYmNjUXfvn1x+fJlLmEnIrXEZexEpFKMjY1Z6CSVVr16dUiShPv374uOQlRhtLS0MHDgwDdObn/8+DE2bdoEV1dX9OvX74VCJwA4Ojqif//+aNasGTZu3IiCggJlvQx6RzKZDOvXr8cPP/yAc+fOiY5DRErCvTqJSN2x2ElEZSKTybBr165S99WvXx+LFi0quZ2WloYOHTrAwMAADRs2xG+//QZjY2OEhYWVHJOYmIjOnTvD0NAQ1apVg7e3d6kJwEFBQXB0dKzw10P0tmQyGfftpErjZZPbZ8yYgUePHiE/Px87d+7EkCFDUKVKlTde6/3334eHhwd++eUX7geqRiwsLLB06VIMHjwYT58+FR2HiCpYbGwsEhIS4OPjIzoKEdFbY7GTiBRCLpejT58+0NHRwenTpxEWFobg4GDk5eWVHJObm4tu3brB2NgYMTExCA8Px8mTJ7nxOakdW1tbFjupUime3H7+/HncuHEDtra2CA4OxsCBA0v25ywLAwMD9OrVCwcPHqzAtKRonp6ecHJywvTp00VHIaIKFhISAj8/P3Z1EpFa407jRKQQf/zxB1JTU/H777/DwsICALBkyRJ89NFHJcds2bKlZMmjiYkJAGD16tXo1KkTLl++DGtrayHZicqLnZ1UWdWtWxdhYWE4e/YsYmNj3+rDcNWqVfH06VNIksT9O9WETCbDjz/+CGdnZ/Ts2ROdOnUSHYmIKsDZs2dx/vx57Ny5U3QUIqJ3ws5OIlKIlJQU1KpVq6TQCQAtWrQo1fGTnJwMZ2fnkkInALRt2xZaWlpISkpSal6id8FiJ1V2d+/exZAhQ976/NatW+PMmTMKTEQV7f3338fatWtf2H6GiDRH8V6dBgYGoqMQEb0TFjuJqExkMtkLe6w9P2SiLB06rzuG3T2kTljspMouLy+vTPt0voqFhQX+/vtvBSYiZejevTu6d+8OX19f0VGISMHOnTuH8+fPc69OItIILHYSUZnUqFEDt27dKrl9+/btUrft7OyQlZWFmzdvltx39uxZyOXyktv29vZISEhATk5OyX0nT56EXC6HnZ1dBb8CIsUpLnZyyApVVjo6774Tkra2tgKSkLItWrQIx48fR3h4uOgoRKRAwcHB8PPzY1cnEWkEFjuJqJRHjx4hPj6+1J9r167B1dUVK1asKNnLx9vbu9QvQ126dEHDhg0xZMgQJCQk4PTp05g4cSJ0dHRKujYHDRoEIyMjeHl5ITExEUePHsWoUaPQt29f7tdJauW9996Dnp4ebt++LToKkRCKKPTzywL1ZGxsjA0bNmDMmDG4c+eO6DhEpADnzp1DXFwchg8fLjoKEZFCsNhJRKUcO3YMLi4upf58++23+O6772BpaYmOHTuif//+GD58OMzNzUvO09LSQnh4OPLy8tCyZUsMGTIE06dPh0wmKymKVqlSBREREXj06BFatmyJXr16oU2bNli3bp2ol0v01riUnYgqq48++gje3t4YMWIEi9ZEGiA4OBhTp05lVycRaQxOYyeiEmFhYQgLC3vl4wcPHix1u1+/fqVu29ra4ujRoyW3ExISUFBQUKpr08nJCZGRka98jry8PBgbG5czOZHy2draIj09He3atRMdhUjp8vLy3mmaekFBAYtkai44OBgtW7ZEWFgYhg4dKjoOEb2luLg4nDt3Djt27BAdhYhIYVjsJCKFCQ8Ph5GREWxsbHDt2jVMnDgRjRs3RtOmTd94riRJuHLlCiIjI+Hs7KyEtETvhp2dVJk1b94c586dQ/Pmzd/q/D/++AOurq4KTkXKpKenh02bNsHV1RWdOnVC/fr1RUciorfAvTqJSBNxGTsRKUxOTg7Gjh0Le3t7DBo0CHZ2doiIiChT5092djbs7e2hp6eHmTNnKiEt0bthsZMqs/r16+PatWtvff6aNWuwceNGFBYWKi4UKZ2TkxOmTJmCIUOGlBpISETqIS4uDmfPnsWIESNERyEiUiiZxDVERERE5RYXF4ehQ4ciISFBdBQiIVJSUnDnzh20b9++XOft27cPRkZGmD17Nu7evYulS5eyy1ONFRUVoWPHjujTpw8mTpwoOg4RlUOvXr3g5uaG8ePHi45CRKRQLHYSERG9hZycHNSsWROPHz9+630LidRdbGwsHjx4gK5du5bp+EOHDqFevXqws7ODJEn49ddfMWnSJDRp0gSLFi2CpaVlBSeminDlyhW0atUK0dHRcHBwEB2HiMrg/Pnz6NGjBy5fvgxDQ0PRcYiIFIrL2ImIiN6CiYkJTExMcPPmTdFRiISpWrUqRo4ciZ9//hkZGRmvPC4xMRHbtm2DnZ0d7OzsAAAymQx9+vRBUlISmjdvjpYtW2L69Ol4/PixsuKTglhaWmLu3LkYPHgw8vPzRcchojIonsDOQicRaSJ2dhJRhfDw8ECfPn3g6ekpOgpRhWnXrh1CQkLQqVMn0VGIlO7Zs2do06YNhg8fjq+//hrnz59HRkYGdHR0oK2tDUmSIJfLUVhYCCcnJzRs2PC118vKysK0adNw+PBhzJ07F4MG/T/27jssqmt9G/AzQy82MEKiiKggorGXoEiJvYVERQREQewNlWLDaFQ02BCNorGAYsVekRg02LCggAIiKIIlGktQpEnb3x/+5DscTY5lZvYAz31dc504uz3jwZ3uekcAACAASURBVGHm3Wu9ywVSKe/LVxSCIOC7775Dy5YtsXDhQrHjENG/4KhOIqrsWOwkIrkYO3YsWrZsiXHjxokdhUhuPDw80LFjR4wePVrsKEQKN2nSJPz555/Yu3fvO60c3n68/JQWDzExMfD09ISKigqCgoLQoUMHmeQl+Xv8+DFatWqFgwcP4ptvvhE7DhH9gx9++AG2trbw9PQUOwoRkVzwdjkRyUWtWrWQlZUldgwiueKK7FRVHThwAEePHsWmTZveW9CUSCSf3MvW0tISFy9exNixY/H999/Dzc0Njx49+tzIpACGhoZYs2YNhg0bhtzcXLHjENF7xMXF4dKlS7xRS0SVGoudRCQXLHZSVcBiJ1VFGRkZGDNmDHbt2oWaNWvK5RpSqRTDhw/HrVu3YGhoiK+//hoBAQF4/fq1XK5HsjNw4EB07NgRvr6+YkchoveYP38+e3USUaXHaexEJBefM4WRqKK4fv06nJyckJSUJHYUIoUoKipCly5dMGjQIHh7eyvsurdv34a3tzcSExOxfPlyfPfdd/z9osRevHiBFi1aYMOGDejZs6fYcYjo/8THx6NPnz64c+cOi51EVKmx2ElERPSJ8vLyoK+vj9zcXC6kQlWCr68vkpKScOTIEVF+5k+ePIkpU6agbt26CAwMRLNmzRSegT5MVFQU3NzckJCQAD09PbHjEBGAAQMGwNraGlOmTBE7ChGRXPGbGRER0SfS1taGvr4+7t+/L3YUIrmLiIjAzp07sWXLFtGK+927d0d8fDz69+8POzs7TJ48GX///bcoWejfde3aFQMGDMDEiRPFjkJEeDOq8+LFixgzZozYUYiI5I7FTiIios9gamqK1NRUsWMQydXDhw/h7u6O7du3o3bt2qJmUVNTw6RJk5CcnIzi4mI0bdoUwcHBKC4uFjUXvWvx4sW4du0adu/eLXYUoipv/vz58PX15fR1IqoSWOwkIiL6DFykiCq74uJiODs7Y8KECbC2thY7TpnatWtj7dq1OHnyJMLDw9GmTRucPn1a7Fj0H7S1tREWFobJkyfjzz//FDsOUZWVkJCAmJgYjuokoiqDPTuJiIg+w7Jly/Dw4UMEBgaKHYWoyhIEAQcOHICXlxfatGmDZcuWwcTEROxY9H/mzZuHS5cu4fjx41xYikgEAwcOhJWVFaZOnSp2FCIiheDITiISRUFBAVauXCl2DKLPxpGdROKTSCQYMGAAkpOT0aZNG7Rv3x5+fn7IyckROxoBmD17Np49e4b169eLHYWoyklISMCFCxc4qpOIqhQWO4lIIf57EHlRURGmTZuGV69eiZSISDZY7CRSHlpaWpg9ezYSEhKQkZEBc3NzbNu27Z3fQaRYampq2Lp1K/z8/HD79m2x4xBVKW97dWpra4sdhYhIYTiNnYjkYv/+/WjWrBkMDAxQs2bNsudLSkoAvCl+VqtWDWlpaahXr55YMYk+W0FBAWrWrImcnByoqqqKHYeI/sOFCxfg6ekJNTU1BAUFoX379mJHqtKCgoKwe/dunD17FioqKmLHIar0rl+/jp49e+LOnTssdhJRlcKRnUQkF7Nnz0br1q0xbNgwBAcH49y5c8jKyoKKigpUVFSgqqoKDQ0NPH/+XOyoRJ9FU1MThoaGyMzMFDsKEf2XTp064dKlSxg9ejTs7e3h7u6Ox48fix2rypo0aRK0tLSwZMkSsaMQVQnz58+Hj48PC51EVOWw2ElEchEdHY3Vq1cjLy8Pc+fOhaurK4YMGQI/Pz8cP34cAKCnp4cnT56InJTo85mamiI1NVXsGERyk5GRAYlEgtjY2Ap3balUCjc3N6SkpKBOnTpo3rw5lixZgtevX8s4Kf0vUqkUISEhWLFiBeLj48WOQ1SpXb9+HefPn8fYsWPFjkJEpHAsdhKRXNSpUwceHh74/fffkZCQAF9fX9SoUQOHDh3CqFGjYGVlhYyMDOTn54sdleizsW8nVQZubm6QSCSQSCRQU1NDw4YN4e3tjdzcXBgZGeHRo0do1aoVAOCPP/6ARCLBs2fPZJrB1tYWEydOLPfcf1/7U1WvXh0BAQGIiYnB+fPn0axZMxw+fJj9PBWsfv36WL58OVxdXVFQUCB2HKJKa/78+fD29uaoTiKqkljsJCK5Ki4uxpdffolx48YhPDwc+/btg7+/P9q2bYu6deuiuLhY7IhEn83MzIzFTqoUunXrhkePHiE9PR0LFy7E2rVr4e3tDRUVFRgaGorSl1bW1zY1NcWhQ4ewZs0azJgxA7169UJycrJMzk0fxtXVFWZmZvjxxx/FjkJUKd24cQPnzp3jqE4iqrJY7CQiufrvL6dmZmZwc3NDUFAQoqKiYGtrK04wIhniyE6qLDQ0NGBoaAgjIyM4OzvDxcUFBw8eLDeVPCMjA3Z2dgCAL774AhKJBG5ubgDeLD63ZMkSNGrUCFpaWvj666+xbdu2cteYP38+jI2Ny641bNgwAG9GlkZHR2PNmjVlI0wzMjLkNoW+Z8+eSEhIQN++fWFjYwNPT09kZWXJ9Br0fhKJBOvWrcO2bdtw9uxZseMQVTpve3Xq6OiIHYWISBRcNpaI5OrZs2e4ceMGkpKScO/ePbx69QpqamqwsbHBwIEDAbz5ciyRSEROSvTpWOykykpLSwtFRUXlnjMyMsK+ffswcOBAJCUlQU9PD1paWgAAPz8/7N27F2vWrEGTJk0QExODUaNGoVatWujbty/27duHZcuWYefOnfj666/x5MkTXLx4EcCblbpTU1Nhbm6ORYsWAXhTTL1//77cXp+amhomT54MJycn/PjjjzA3N8dPP/2EUaNGcbVwOfviiy+wfv16DB8+HAkJCahWrZrYkYgqhRs3buDs2bMIDQ0VOwoRkWhY7CQiublx4wbmzp2LmJgYaGhooE6dOtDU1ERpaSmOHj2K8PBwrFy5El9++aXYUYk+i4mJCR4+fIjCwkKoq6uLHYdIJi5fvowdO3aga9eu5Z5XUVGBnp4egDf9mWvXrg0AyM3NxYoVK/Dbb7+hS5cuAN7827h8+TLWrFmDvn37IjMzE19++SV69OgBNTU11K9fH+3atQMA1KhRA+rq6tDW1oahoaECX+mbwltwcDDGjh0LT09PBAcHIygoiLMP5Kx///44dOgQpk2bhg0bNogdh6hSeNurk6M6iagq4zR2IpKLhw8fwsvLC7dv38aWLVtw8eJFREdH48SJE9i/fz/8/f1x//59rFy5UuyoRJ9NTU0N9erVw927d8WOQvRZTpw4AV1dXWhqasLS0hLW1tZYvXr1Bx2bnJyMgoIC9OrVC7q6umWP4OBg3LlzBwDg4OCAgoICmJiYwMPDA3v27FGqVdFbtmyJ06dPY86cOXBzc4ODgwMyMjLEjlWprVixAlFRUThy5IjYUYgqvMTERJw9exbjxo0TOwoRkahY7CQiubh58ybu3LmDyMhI9OjRA4aGhtDS0oK2tjbq1KkDJycnDB06FL/99pvYUYlkglPZqTKwtrZGfHw8bt26hYKCAuzfvx916tT5oGNLS0sBAEeOHEF8fHzZIykpqey93sjICLdu3cL69etRvXp1eHl5oW3btsjNzZXba/pYEokEgwYNws2bN9GyZUu0a9cOc+bMUaqMlUn16tURGhqKMWPG4OnTp2LHIarQOKqTiOgNFjuJSC50dHSQk5MDbW3tf9zn9u3b7NFFlYapqSlSU1PFjkH0WbS1tdG4cWMYGxtDTU3tH/d7266hpKSk7DkLCwtoaGggMzMTjRs3LvcwNjYu209TUxN9+/ZFYGAgrly5gqSkJJw/f77svP95TjFpaWnBz88P8fHxSE9Ph7m5OXbs2AFBEMSOVulYW1vDxcUFY8eO5d8v0SdKTEzEmTNnOKqTiAjs2UlEcmJiYgJjY2N4enpi+vTpUFFRgVQqRV5eHu7fv4+9e/fiyJEjCAsLEzsqkUyYmZkhKSlJ7BhECmFsbAyJRIJjx46hf//+0NLSQrVq1eDt7Q1vb28IggBra2vk5OTg4sWLkEqlGD16NEJDQ1FcXIyOHTtCV1cXu3fvhpqaGkxNTQEADRo0wOXLl5GRkQFdXd2y3qBiqlevHrZv347z58/D09MTa9asQVBQUFmvUZKNBQsWoH379ti2bRtcXV3FjkNU4SxYsABeXl4c1UlEBBY7iUhODA0NERgYCBcXF0RHR6NRo0YoLi5GQUEBCgsLoauri8DAQPTs2VPsqEQyYWpqioMHD4odg0gh6tati59++gmzZ8/GyJEjMWzYMISGhmLBggUwMDDAsmXLMG7cOFSvXh2tWrWCr68vAKBmzZoICAiAt7c3ioqKYGFhgf3798PExAQA4O3tjeHDh8PCwgL5+flK1Qe3c+fOuHz5MkJDQ9G/f3/07t0bixYtUvhiSpWVpqYmwsLC0L17d9ja2sLIyEjsSEQVRmJiIqKjo7F582axoxARKQWJwLkiRCRHhYWF2LNnD5KSklBcXIyaNWuiYcOGaNOmDczMzMSORyQz6enpsLOzQ2ZmpthRiEjOsrOzsXDhQmzevBnTp0/H5MmToaGhIXasSmHRokWIiorCyZMnIZWy4xbRh3B0dES7du3g4+MjdhQiIqXAYicREZEMFBcXQ1dXFy9evICmpqbYcYje69atW2jSpInYMSqNtLQ0TJs2DSkpKVixYgX69esHiUQidqwKrbi4GNbW1hgyZAgmT54sdhwipZeUlIRvv/0W6enpnMJORPR/WOwkIrl7+zbz9n8lEgm/DFKlZG5ujgMHDqBp06ZiRyF6R0FBAb755hvEx8eLHaXSOXHiBKZOnQpjY2MEBgbyPeAzpaWlwdLSEufOnYO5ubnYcYiU2pAhQ9CmTZuydiFERMTV2IlIAd4WN6VSKaRSKQudVGklJyfzizkpLS8vL7YPkZNevXrh+vXr6N27N6ytrTFlyhRkZWWJHavCMjU1xYIFC+Dq6oqioiKx4xApraSkJJw+fRrjx48XOwoRkVJhsZOIiEhGWMwnZbV3715ERERgw4YNYkeptNTU1ODp6Ynk5GQUFBSgadOmWL9+PUpKSsSOViGNHTsW+vr6WLRokdhRiJTW2xXYdXV1xY5CRKRUOI2diOTqP6euExGR4t29excdO3bEsWPH0L59e7HjVBnx8fHw9PTEy5cvERQUBBsbG7EjVTh//vknWrdujaNHj/Jnl+i/JCcnw87ODnfu3GGxk4jov3BkJxHJ1ZYtW3D8+HGxYxARVUmFhYUYMmQIZs6cyWKRgrVq1Qp//PEHZs+ejeHDh2Pw4MHIzMwUO1aF8tVXX2HVqlVwdXVFfn6+2HGIlMqCBQswbdo0FjqJiN6DxU4ikqvk5GQkJiaKHYOIqEqaNWsW6tSpgylTpogdpUqSSCRwcHDAzZs38fXXX6Nt27b48ccfkZubK3a0CsPR0RGtW7fGzJkzxY5CpDSSk5Nx6tQpTJgwQewoRERKicVOIpKrWrVqcZEGov9TUFCAvLw8sWNQFXH06FGEh4cjNDSUrUREpqWlhTlz5iAuLg63b99G06ZNsXPnTrCb1IdZs2YN9u7di6ioKLGjECkFjuokIvp37NlJRHK1bt06xMXFYf369WJHIRLd2rVr8ezZM8yePRsqKipix6FK7MGDB2jbti327dsHKysrsePQfzl37hw8PT2hpaWFoKAgtG3bVuxISi8yMhKjRo3C9evXUbNmTbHjEMmVIAiIiYnBkydPIJX+//FJqqqqqFu3Lnr06MFenVRlxMXFITMzEyoqKuVuEnbt2hU6OjoiJiNlpip2ACKq3Diyk6qSTZs2wcrKCqampigtLYVEIilX1DQyMkJwcDCcnJxgamoqYlKqzIqLi+Hs7AxPT08WOpWUlZUVLl++jNDQUPTr1w99+/aFv78/DAwMxI6mtHr27Il+/fph8uTJ2Lp1q9hxiOSitLQUx44dQ2FhISwtLdGpU6dy23Nzc7F161a4ubmhuLhYpJRE8icIAk6ePIns7Gy0bt0a33//fbntr1+/xqlTp5CTkwMrKyt8+eWXIiUlZcVp7EQkVyx2UlUyY8YMnD59GlKpFKqqqmWFzlevXiE5ORn37t1DUlISEhISRE5KldlPP/0EDQ0NzJgxQ+wo9C9UVFTg4eGBlJQU1KpVC82aNcOyZctQWFgodjSltXTpUsTExGDfvn1iRyGSuYKCAmzZsgW2trYYOHAgvvrqq3f20dHRwbhx4/Dzzz/jt99+w71790RISiRfJSUl2L59O1q1aoVBgwahUaNG7+yjoaGB3r17w8HBAVevXsXNmzdFSErKjNPYiUiurly5gnHjxiE2NlbsKERyZ29vj5ycHNjZ2eH69etIS0vDn3/+iZycHEilUtSpUwfa2tr4+eef0bdvX7HjUiX0+++/Y9iwYbh27RoMDQ3FjkMfITU1FdOmTUNqaioCAwPRp08f9lp9j5iYGPzwww+Ij4/nzzhVGqWlpdiyZQuGDh0KNTW1Dz5u7969sLOzg76+vhzTESnW9u3bYW9v/1FtGiIjI2Fubg5jY2M5JqOKhCM7iUiuOLKTqpJOnTrh9OnTOHToEPLz82FlZQVfX1+EhITgyJEjOHToEA4dOgRra2uxo1Il9Ndff2H48OHYunUri0AVkJmZGY4ePYqgoCB4eXmhT58+SElJETuW0rG0tISHhwdGjRrFBZ6o0oiIiMCgQYM+qtAJAAMHDsTJkyfllKpqevXqFaZMmQJjY2NoaWmhU6dOuHLlStn2nJwcTJo0CfXq1YOWlhaaNGmCwMBAERNXLtHR0bCzs/vofrQ9e/bEhQsX5JSKKiL27CQiuWKxk6qS+vXro1atWtixYwf09PSgoaEBLS0tLkZEcldaWoqhQ4dixIgR6Natm9hx6DP07t0b3bp1wy+//IIuXbpg6NChmDt37gctylNcXAxV1cr/8X7u3Lno2LEjNm/eDA8PD7HjEH0WQRCQn5+PatWqffSxEokEX331FZ48eYI6derIIV3VM3LkSFy/fh1btmxBvXr1sG3bNnTr1g3JycmoW7cupk2bht9//x1hYWEwMTHBmTNnMGrUKNSuXRuurq5ix6/wnj59Chsbm086tmXLlkhKSkKzZs1knIoqIo7sJCK5qlmzJrKzs1FaWip2FCK5a968OTQ1NfHVV19BX18furq6ZYVOQRDKHkSy9vPPP+P169eYO3eu2FFIBtTU1DB16lQkJSUhLy8P5ubmiIyM/Nf3D0EQcOLECYwfPx67du1SYFrFU1dXR1hYGGbMmIH09HSx4xB9ltjYWLRv3/6Tj7eyssK5c+dkmKjqys/Px759+/Dzzz/D1tYWjRs3xrx589C4cWMEBwcDAC5cuABXV1fY2dmhQYMGGDZsGL755htcunRJ5PQVX0ZGBho0aPDJx1tYWLB3J5VhsZOI5EpFRQU6OjrIzs4WOwqR3DVt2hSzZs1CSUkJcnJysHfvXiQlJQF4M/ri7YNIls6dO4dVq1Zhx44dVWJUX1VSp04drF+/HhEREf+z/UVxcTGys7OhoqKCMWPGwNbWFs+ePVNQUsVr3rw5ZsyYATc3N5SUlIgdh+iTPXz48LP6DEqlUkil/FovC8XFxSgpKYGmpma557W0tMoKylZWVjhy5Aju378P4E3xMz4+Hr169VJ43somISEBbdu2/axz8HMQvcV3RSKSO05lp6pCVVUVEyZMQPXq1ZGfn48FCxbAysoK48aNw40bN8r240hnkpXnz5/D2dkZmzZtQr169cSOQ3LSunVraGpq/uvNEjU1NTg7O2P16tVo0KAB1NXV8fLlSwWmVLwpU6ZAIpGwXx5VaLJodcN2ObJRrVo1WFpaYuHChXj48CFKSkqwbds2xMTE4NGjRwCAVatWoVWrVqhfvz7U1NRgY2ODgIAA9OvXT+T0FZ9UKv3sQQFqamq8AUYAWOwkIgVgsZOqkreFTF1dXWRlZWHJkiUwMzPDgAEDMH36dFy8eJEjMEgmBEGAm5sbHBwc0LdvX7HjkJz9ry+AhYWFAN6sYpuZmYnJkyejUaNGACrvDRYVFRWEhoYiICCg3A0loopEFu1tEhMTy80g4ePfH//2nhgWFgapVIp69epBQ0MDq1atgpOTU1lBefXq1Th//jwOHz6Mq1evIjAwEN7e3jhx4sQ75yotLYWXl5for7eiPFavXv3Z/xZUVFRY7CQALHYSkQKw2ElVydsP0RoaGjAyMsKzZ88wdepUnD9/HiUlJfjll1+waNEipKamih2VKriVK1fir7/+wuLFi8WOQiITBAHq6uoAgBkzZsDJyQmWlpZl2wsLC5GWlobt27cjMjJSrJhyYWJigoCAALi6upYVfIkqElkUOy0sLMr1Bufj3x//dtO5UaNGiI6ORk5ODu7fv4/Lly+jqKgIJiYmyM/Px8yZM7FkyRL0798fLVq0wMSJEzFkyBAsW7bsnXNJpVIsX75c9NdbUR4TJkz47H8Lr1+/Lvt9SFUbi51EJHcsdlJVIpFIyvpntW3bFomJiQCAkpISjBkzBnXq1IGfnx8WLFggclKqyK5cuYLFixdj9+7d/FBPZaNYZsyYARUVFQwbNgz6+vpl26dOnYpvv/0WixcvxvDhw9G5c+eyfnOVgbu7O+rXr4+ffvpJ7ChEH6169eqf3V+3uLhYRmnoLR0dHXz55ZfIyspCZGQk7O3tUVRUhKKionfaBqioqFTaEfSKZGJi8tmDAYqKimSUhio6dm8lIrljsZOqkuzsbOzbtw+PHj3C+fPnkZqaiqZNmyI7OxuCIMDAwAB2dnaoU6eO2FGpgnr58iUcHR2xdu1amJiYiB2HRFZaWgpVVVXcu3cPa9aswaxZs9CyZcuy7YsWLUJYWBhWrlyJfv36QU1NDd9//z3CwsIwa9YsEZPLjkQiwYYNG9CyZUv07dsXnTp1EjsS0Qd5+fIlLl68iLNnz+LHH3/8pHPExcWhVatWMk5WdUVGRqK0tBTm5ua4ffs2fHx80KRJE7i7u5f16JwxYwZ0dXVhbGyM6OhobN26FUuWLBE7eoXXokUL7Nu3D2ZmZp90/IMHD1C3bl0Zp6KKisVOIpI7FjupKsnKysKMGTNgZmYGdXV1lJaWYtSoUahevToMDAxQu3Zt1KhRA1988YXYUakCEgQBI0eORK9evTBo0CCx45DIbty4AQ0NDZiZmcHT0xPNmjXD999/D21tbQDApUuXsHDhQixevBgjR44sO+7bb7/F1q1b4ePjAzU1NbHiy5SBgQGCg4MxbNgwxMfHQ1dXV+xIRP/o0aNHWLlyJTZu3IjevXujc+fOKCkp+aSFhm7fvg0HBwc5pKyaXr58iZkzZ+LBgwfQ09PDwIED4e/vX/ZeuWvXLsycORMuLi74+++/YWxsjAULFmDixIkiJ68ctLS0kJOT80nv4TExMfxsRGUkgiB8fpMQIqJ/sWjRIrx69Yp95ajKOH/+PPT19fHo0SP06NEDubm5nGpMMrFu3ToEBwfj0qVL0NTUFDsOiai0tBQzZszAsmXL4OzsjMOHD2P9+vVwdHQs60c3aNAgZGZm4sqVKwDeFMslEglGjBiBjIwMnDp1CgCQm5uL8PBwtGjRAm3bthXtNcnC8OHDoa2tjeDgYLGjEL3j1q1bWLp0Kfbv3w9XV1dMnToVDRo0QF5eHvbv3w8XFxdIJB++GvWpU6dQv359NG7cWI6piRSnuLgYYWFhGDZs2EcV/y9fvgw1NTW0bt1ajumoImHPTiKSO47spKqmc+fOMDc3h7W1NRITE99b6GRvJ/pY169fx5w5cxAeHs5CJ0EqlWLJkiXYuXMnrly5gpycHDx58qSsUJKZmYmDBw+WTY0tKSmBRCJBSkoKMjIy0Lp167I+f9HR0Th+/DicnZ3RvXv3Ct3Pc9WqVTh+/DgiIiLEjkJU5tKlSxgwYAC6dOkCIyMjpKamIigoCA0aNAAAaGtro2fPntixY8cHfz6IioqCnp4eC51UqaiqqmLw4MHYunUrXr9+/UHHXLx4EcXFxSx0Ujmcxk5EcsdiJ1U1paWlkEqlUFFRQZMmTZCamoqMjAzk5eWhsLAQ7du3Z69F+ig5OTkYPHgwAgMD0aRJE7HjkBJxdHSEo6Mj5s+fDx8fH/z1119YtGgRIiIiYGZmhjZt2gBA2QiZvXv34sWLF7C2toaq6puvAn369EHDhg0REREBLy8vnDhxAqNGjRLtNX2OGjVqICQkBMOGDcP169ehp6cndiSqogRBQEREBJYsWYKMjAx4eXkhLCwMOjo6793/iy++gL29Pfbs2YNatWrBzs7unTYTgiAgNjYWmZmZaNWqFQudVCnp6OjAxcUFhw8fhqamJrp27QotLa139ouJiUFmZiYsLCzQokULEZKSMuM0diKSu8jISCxfvhy//fab2FGIFCY/Px9r167FunXrcP/+fRQWFgIAzMzMYGBgAAcHB/Z3og82fPhwSKVShISEiB2FlNiLFy+QkJAAGxsbHDp0CG5uboiNjUWjRo0AABEREfj555/RuHFjbNq0CcCbKYOqqqrIycmBh4cHEhMTkZSUJObLkImpU6fi0aNH2LVrl9hRqIopKirC7t27sWTJEkgkEvj6+mLw4MEf1R83Ozsbp0+fhiAIUFFRwduv7G9vmBobG8srPpFSyc/PR1RUFIqKispNay8sLMS2bdtga2uLKVOmiJiQlBVHdhKR3HFkJ1VFv/76K4KCgtCnTx+Ympri1KlTKCoqwpQpU3Dnzh3s2LED6urqGD16tNhRSclt2bIFly9fRmxsrNhRSMnVrFkTNjY2AABzc3MYGxsjIiICgwYNQnp6OiZNmoTmzZtj8uTJAP5/obO0tBSRkZHYs2dP2Y3Jt9sqqkWLFqFNmzbYtWsXhgwZInYcqgJyc3OxadMmrFixAiYmJliyZAl69uz5UT0436pevTrs7e3lkJKoYtHS0kK/fv3eu61e8kej9wAAIABJREFUvXpwdnbGpEmTPmlxL6rcOLKTiOQuLS0NvXv3xu3bt8WOQqQQaWlpcHJywsCBAzF16lRoamoiLy8PK1aswIULF3D8+HEEBQVh48aNuHHjhthxSYmlpKSgS5cuOHXqFL7++mux41AFs3v3bkyYMAE1atRAXl4e2rZti4CAADRr1gzA/1+w6N69e3BwcICenh4iIiLKnq/oYmNj0adPH8TFxaFu3bpix6FK6tmzZ1i9ejWCg4PRpUsXTJ8+HR06dBA7FlGV0LFjR8yaNYs3B+gdXKCIiOSOIzupqpFKpUhPT4enp2fZQjLa2tpo164dkpOTAQBdu3bFvXv3xIxJSi4/Px+DBw+Gv78/C530SRwdHcsKMefPn8fhw4fLCp2lpaWQSCQoLCzEvn37EBsbi19//bVsW2XQrl07TJw4ESNGjADHd5CsZWRkYNKkSTAzM8OjR49w9uxZ7Nu3j4VOIgXy9PREUFCQ2DFICbHYSURyV7NmTbx8+bLSfHki+l9MTEwglUoRExNT7vn9+/fD0tISJSUlyMnJQY0aNfDixQuRUpKymzp1KiwsLCrsQjGkPN4uQPRWXl4eXr16BQC4desWli1bBk9PTxgZGaGkpKRSTQecOXMmsrKysG7dOrGjUCWRkJAAFxcXtG3bFjo6OkhKSsKvv/7KxeOIRDBo0CDcunUL169fFzsKKZmK24iHiCoMVVVVaGtr49WrV6hRo4bYcYjkTiqVwtPTEx4eHrCyskL9+vURFxeH06dP48iRI1BRUYGBgQG2bt363tUlicLDw/H777/j2rVrlWI6MSkHqfTNOIdDhw5h2bJlGDp0KNLT01FUVIQVK1YAQKX7eVNTU0NYWBisrKzQrVs3mJqaih2JKiBBEPDHH38gICAA169fx5QpU7B27Vp+riUSmbq6OsaPH4+goKCyhfeIAPbsJCIFMTY2RnR0NBo0aCB2FCKFKC4uRnBwMKKjo/H06VMYGBhg6tSpsLS0FDsaKbk7d+7A0tISERERaNu2rdhxqJJaunQp5s2bh/z8fHh5eWHp0qWVblTnf1q9ejV27NiBs2fPVuiFl0ixSkpKcPDgQQQEBCA7Oxs+Pj4YOnQoNDQ0xI5GRP/n6dOnMDMzQ2pqKr744gux45CSYLGTiBSiVatWCAkJQevWrcWOQqRQL168QFFREWrXrl3pRkyR7BUWFqJz584YOnQoPD09xY5Dldzr168xc+ZMrFy5EkOGDMH69etRrVq1d/YTBAFFRUVQV1cXIaVslJaWokePHrCzs8Ps2bPFjkNKrqCgAGFhYVi6dCn09PQwffp02Nvbl42OJiLl4uHhgYYNG/L9ncrw3ZqIFIKLFFFVVbNmTXzxxRcsdNIHmTFjBr766itMnjxZ7ChUBWhoaGDFihW4du0azMzMUFhY+M4+giBg3759aNGiBSIiIkRIKRtSqRQhISEICgpCXFyc2HFISb148QI///wzGjZsiIMHD2Ljxo2IiYnBDz/8wEInkRLz9PTE2rVr3/t7jKomzuEgIoVgsZOI6N8dPnwY+/btQ1xcHIvjpFCtWrVCq1at3rtNIpFg0KBB0NbWxpQpU/DLL78gMDAQZmZmCk75+YyMjLBixQq4uroiNjYWmpqaYkciJfHnn39i5cqV2LRpE/r06YPIyEh8/fXXYsciog/UokULPHz4UOwYpER4e4qIFILFTiKif3bv3j2MGjUKO3fuhJ6enthxiN7Rp08f3LhxA127dkXnzp3h7e2Nly9fih3ro7m4uKBp06bw8/MTOwopgZSUFHh4eKB58+Z4/fo1rl27hrCwMBY6iYgqOBY7iUghWOwkInq/4uJiODs7Y+rUqejUqZPYcYj+kbq6OqZNm4bExES8fPkS5ubm2LhxI0pKSsSO9sEkEgmCg4OxY8cOREdHix2HRHLx4kX88MMPsLGxgbGxMdLS0hAUFARjY2OxoxERkQyw2ElECsFiJ1VVxcXFyM/PFzsGKbG5c+dCR0cHvr6+Ykch+iAGBgbYsGEDjh07hi1btqBDhw44d+6c2LE+WO3atbFhwwa4ubkhOztb7DikIIIg4NixY7CxsYGTkxO6du2Ku3fv4scff4S+vr7Y8YiISIZY7CQihWCxk6qqJUuWYN68eWLHICX122+/ITQ0FGFhYVz8giqcNm3a4MyZM/Dx8YGzszOcnJxw//59sWN9kL59+6J79+6YOnWq2FFIzoqKihAWFoYWLVpg9uzZGDNmDNLS0jBx4kRoa2uLHY+IiOSAn6qJSK6Ki4tx8uRJ5OXlQUtLC0eOHMGBAwfw4MEDsaMRKYSpqSnS0tLEjkFK6NGjRxg+fDjCwsJQp04dseMQfRKJRIIhQ4YgJSUFTZo0QevWrTF//nzk5eWJHe1/Wr58Of744w8cPnxY7CgkBzk5OQgKCkLjxo0REhKCZcuWIS4uDs7OzlBVVd51ekNDQ6Grq6vQa/7xxx+QSCR49uyZQq9LVU9GRgYkEgliY2PFjkKVnEQQBEHsEERU+WRlZeHUqVNQUVGBnZ0datSoUbZNEARcvHgRDx8+hJGRETp27ChiUiL5io+Px9ChQ5GYmCh2FFIiJSUl6NGjB6ysrPDTTz+JHYdIZjIzM+Hr64uLFy9i6dKlcHBwgEQiETvWPzp37hwGDx6MhIQEfPHFF2LHIRl4+vQpVq9ejeDgYNja2sLX1xft27eX+XVsbW3RvHlz/PLLL+WeDw0NxcSJE5GTk/NJ583Pz8erV68UehOssLAQf//9NwwMDJT63yspNzc3Nzx79gxHjx4t93xsbCzat2+Pu3fvwsjICE+fPkXt2rWV+qYDVXwc2UlEMpeeno6oqCgMGDAA33//fblCJ/BmFIilpSUGDRoEPT09HDhwQKSkRPLXuHFjpKeno7S0VOwopEQWL16MkpIS/Pjjj2JHIZIpY2Nj7N69G2FhYVi8eDFsbW0RHx8vdqx/ZGVlBVdXV4wZMwYcA6J8Pub/k7t372LixIlo0qQJ/vrrL1y4cAF79uyRS6HzUxUWFv7PfbS0tBQ+2l9dXR2GhoYsdJLcqaiowNDQ8F8LnUVFRQpMRJUVi51EJFN//vknEhMTMWjQoA/6wGRqagpLS0scOnRIAemIFE9XVxe1atVi6wYqc+bMGfzyyy/Yvn07VFRUxI5DJBfW1taIjY2Fi4sLevXqhTFjxuDp06dix3qv+fPn4/bt29i6davYUeg/vHjx4oM+S8bHx8PZ2Rnt27dHtWrVkJycjPXr18PU1FQBKf+dm5sb+vXrh4CAANSrVw/16tVDaGgoJBLJOw83NzcA75/GfuzYMXTs2BFaWlrQ19dH//79UVBQAOBNAXX69OmoV68edHR00L59e0RGRpYd+3aKelRUFDp27AhtbW20a9cO165de2cfTmMnefvvaexvf/aOHz+ODh06QF1dHZGRkbh//z7s7e2hp6cHbW1tmJubY9euXWXnuXHjBrp16wYtLS3o6enBzc0NL1++BABERkZCXV0dz58/L3ftWbNmoWXLlgCA58+fw8nJCfXq1YOWlhaaNWuGkJAQBf0tkCKw2ElEMnX69Gl89913H3WMoaEhTE1Ny33oIqpM2LeT3nr27BlcXFwQEhKCunXrih2HSK5UVFQwevRopKSkQEdHBxYWFli5cqXSjdrR0NBAWFgYvL29kZmZKXacKi8xMRF9+/ZF06ZNkZSU9I/7CYKAoKAg9O3bF61bt0Z6ejoWL14MQ0NDBab936Kjo3H9+nWcOHECUVFRcHR0xKNHj8oebwszNjY27z3+xIkTsLe3R/fu3XH16lWcPn0aNjY2ZTNG3N3dER0djR07duDGjRsYPnw4+vfvj4SEhHLnmTlzJn7++Wdcu3YN+vr6cHFx4WhmUhrTp0/HwoULkZKSgo4dO2L8+PHIy8vD6dOnkZSUhJUrV6JmzZoAgLy8PPTq1Qu6urq4fPkyDhw4gAsXLmDEiBEAgG7dukFfXx979uwpO78gCNi5cyeGDh0KACgoKECbNm1w9OhRJCUlwdPTE2PGjEFUVJTiXzzJh0BEJCNJSUlCUlLSJx+/Z88eGaYhUh4jR44UgoODxY5BIispKRH69u0r+Pj4iB2FSBQ3b94UevXqJZibmwsRERFix3nH4sWLBTs7O6GkpETsKFVSbGys0KlTJ0FDQ0NwcHAQbt269a/7l5aWCvn5+UJBQYGCEpZnY2MjTJgw4Z3nQ0JCBB0dHUEQBGH48OFC7dq1/zHjkydPBGNjY8HT0/O9xwuCIHTq1ElwdHR87/G3b98WJBKJkJmZWe55e3t7Ydy4cYIgCMLp06cFAMKJEyfKtp87d04AINy/f7/cPk+fPv2Ql070XsOHDxdUVFQEHR2dcg8tLS0BgHD37l3h7t27AgDhypUrgiD8/5+9vXv3ljvX119/LcybN++91/n111+F6tWrC9nZ2WXPvT1PWlqaIAiCMGXKFMHKyqps+9mzZwWpVCo8ePDgH/M7OjoKHh4en/z6SblwZCcRyczNmzdhYWHxycfr6em9M92AqDLgyE4CgMDAQDx//hz+/v5iRyEShbm5OY4fP45ly5Zh8uTJ6NevH1JTU8WOVcbHxwevX7/GqlWrxI5S5aSnp8Pd3R2ZmZl4/PgxwsPDYWZm9q/HSCQSaGpqQkNDQ0EpP03z5s3fm7GwsBA//PADmjZtiuXLl//j8XFxcejatet7t127dg2CIMDCwgK6urplj2PHjuHOnTvl9m3RokXZf3/11VcAgCdPnnzKSyL6R9bW1oiPjy/32LFjx/88rl27duX+7OnpiYULF8LS0hJ+fn64evVq2babN2+iRYsWqFatWtlznTp1glQqRXJyMgBg6NChOH/+fNlo/e3bt8PW1rZsVk1JSQn8/f3RokUL6OvrQ1dXF/v378e9e/c++++AlAOLnUQkE4IgfHbvORsbG5w/f15GiYiUB4uddOnSJQQEBGDnzp1QU1MTOw6RaCQSCfr27YvExETY2dmhc+fO8PHxKeu1JiYVFRVs3boVCxcuLPvCTPLz119/lf13w4YNy6auP378GL///jvc3d0xZ86ccn36lEn16tXf+3P74sWLcotz6ujovPf4sWPHIisrC7t37/7kz9ClpaWQSCS4cuVKueLSzZs3sXnz5nL7/ufvnre9ULl4IsmatrY2GjduXO5Rr169/3ncf/878fDwwN27d+Hu7o7U1FR06tQJ8+bNA/Dme+c/9fN9+3zbtm1hbm6OHTt2oKioCHv27Cmbwg4Ay5Ytw/Lly+Hj44OoqCjEx8fj+++//6BFxKhiYLGTiGQiPz//nWbqH0tFRYWrQFKlZGpqqlSjl0ixXrx4gSFDhmDdunVo0KCB2HGIlIK6ujq8vLyQmJiIrKwsmJubY9OmTaIXXxo1agR/f38MGzZM6XqLVgalpaVYuHAhmjVrBgcHB0yfPr2sL2evXr3w4sULfPPNNxg/fjy0tbURHR0NZ2dnLFiwQCkK4v+pSZMmZSMr/9O1a9fQpEmTfz122bJlOHLkCI4ePYrq1av/676tW7f+xz6CrVu3hiAIePz48TsFJvaFpoquXr16GD16NMLDwzF//nz8+uuvAAALCwskJCTg1atXZfteuHABpaWlaNq0adlzLi4u2L59O06cOIHc3FwMHDiwbNu5c+fQv39/uLq6olWrVmjUqBE/q1cyLHYSkUwUFRXJZLTSf39gJKoMGjVqhIyMDBQXF4sdhRRMEASMHDkS/fr1w4ABA8SOQ6R0DAwMsHHjRhw9ehQhISHo0KGD6LM8Ro8ejTp16mDhwoWi5qhsMjIy0K1bNxw6dAh+fn7o1asXIiIisGbNGgBvZvj06NEDEydORFRUFNasWYMzZ84gMDAQoaGhOHPmjMivoLxx48YhPT0dkyZNQkJCAm7duoXAwEDs3LkT3t7e/3jc77//jlmzZmHt2rXQ0tLC48eP8fjx438s5s6ePRt79uyBn58fkpOTkZSUhMDAQOTl5cHMzAwuLi5wc3PD3r17kZ6ejtjYWCxbtgz79++X10snkjtPT0+cOHEC6enpiI+Px4kTJ8rapbm4uEBHRwfDhg3DjRs3cObMGYwZMwYDBgxA48aNy84xdOhQJCcnY86cOfjuu+/K3VgwMzNDVFQUzp07h5SUFEycOBF3795V+Osk+WGxk4hkolq1asjOzhY7BpFS0tLSgoGBAfsAVUHBwcFIT0/H0qVLxY5CpNTatm2Ls2fPwsvLC0OGDIGzszMePHggShaJRIJNmzZh3bp1uHz5sigZKqOzZ88iMzMTx44dg5OTE2bNmoWGDRuiuLgYr1+/BgCMHDkSEydOhJGRUdlxnp6eyMvLw61bt8SK/l4NGzbEmTNnkJaWhh49eqBDhw7YtWsX9uzZgz59+vzjcefOnUNRUREGDx6ML7/8suzh6en53v379OmDAwcOICIiAq1bt4aNjQ1Onz4NqfTNV/mQkBC4u7vD19cX5ubm6NevH86cOQNjY2O5vG4iRSgtLcWkSZNgYWGB7t27w8DAAFu2bAHwZqp8ZGQksrOz0aFDB9jb28PS0vKd1g3GxsawsrJCQkJCuSnsAODn54cOHTqgd+/esLa2ho6ODlxcXBT2+kj+JAKHURGRjOzbt6/c9ICPlZaWhry8PLRs2VKGqYiUQ7du3eDj44OePXuKHYUUJD4+Ht27d8eFCxdgamoqdhyiCiM3NxdLlizBmjVr4OnpCW9vb2hpaSk8x549ezBnzhxcu3YN2traCr9+ZTN//nxERUVhy5YtaNCgAQRBgL29Pdzd3fHDDz+8s78gCBAEAa9fv4aJiQk8PDy4wBsREX0QjuwkIpn5p0btH+r69essdFKlxUWKqpZXr17B0dERQUFBLHQSfSQdHR389NNPiI2NxY0bN9C0aVPs2bNH4a1uHBwc0LZtW8yYMUOh162sBg8ejBcvXmDkyJEYOXIkqlWrhsuXL8PLywtjx45953ekRCKBVCpFSEgIvvrqK4wcOVKk5EREVNGw2ElEMmNnZ4dTp0590rF5eXmijNogUhQWO6sOQRAwbtw4dOnSBc7OzmLHIaqwGjRogPDwcGzZsgX+/v6ws7NDQkKCQjP88ssvOHDgAE6ePKnQ61ZG5ubmOHDgQNk0682bNyMlJQULFixAamoqvLy8ALz5TLh+/Xps2LABVlZWWLBgAUaOHAljY2P2diciog/CYicRyYyqqir09fWRkpLyUccJgoDw8HB069ZNTsmIxMdiZ9URGhqKuLg4rFq1SuwoRJWCjY0Nrl69CicnJ/Ts2RNjx47F06dPFXLtWrVqYfPmzRgxYgSysrIUcs3KrGHDhkhOTkbnzp0xePBg1KxZEy4uLujduzcyMzPx9OlTaGtr4/79+1i5ciW6dOmCtLQ0jB8/HlKpFBKJROyXQEREFQCLnUQkU9bW1sjIyEBycvIH7V9cXIywsDD88MMPUFdXl3M6IvGYmpoiNTVV7BgkZ8nJyfDx8UF4eDh7/BHJkIqKCsaMGYObN29CS0sLzZo1Q1BQEIqKiuR+7e7du8Pe3h6TJ0+W+7Uqk6KiondGYgqCgGvXrsHS0rLc85cvX0b9+vVRrVo1AMD06dORlJSExYsXQ1dXV2GZiYiocmCxk4hkrlevXvj777+xb98+/PXXX+/dp6SkBKdOncKePXswaNAg1KhRQ8EpiRSrYcOGuH//vkK+mJM48vLy4OjoiICAADRr1kzsOESVUq1atRAYGIjo6GgcP34cLVq0QGRkpNyvu2TJEly+fBl79+6V+7Uquri4ODg5OcHJyemdbRKJBG5ubli3bh1WrVqFO3fuwM/PDzdu3ICLiws0NTUBoKzoSURE9Cm4GjsRyY0gCDh37hz++usv5Ofno6CgAIaGhmXFHhsbG+jr64uckkhxGjVqhIiICJiZmYkdheRg9OjRyM3NxbZt2zjVkkgBBEHAsWPHMHXqVDRt2hTLly+X64Jgly5dwnfffYf4+Hh8+eWXcrtORSQIAk6dOoWAgAAkJydj6tSpGDVqFKpXr/7OvkVFRXByckJiYiIKCwuhr68Pf39/9OjRQ4TkRFSVXL9+Hb1790ZGRgbU1NTEjkNyxGInESnExo0bERMTg02bNokdhUg0vXr1wqRJk9C3b1+xo5CM7dq1C3PmzMG1a9c4IolIwV6/fo1Vq1YhICAAI0aMgJ+f33uLbLLw9t/50aNHeVMDb2bq7N+/HwEBAcjNzYWvry9cXFw+qDXRrVu3oKKigsaNGysgKRHRG3Z2dhg9evR7R59T5cFp7ESkEFlZWahVq5bYMYhExUWKKqfbt29j0qRJ2L17NwudRCLQ0NCAj48PEhMT8fz5c5ibmyMkJASlpaUyv9acOXPw+PFjbNy4Uebnrkjy8/Oxbt06NGnSBIGBgZgzZw6SkpLg7u7+wT3YmzRpwkInESnclClTsHLlSrFjkJyx2ElECsFiJxGLnZXR69ev4ejoiLlz56JNmzZixyGq0gwNDbFp0yYcPnwYGzduRIcOHXDhwgWZXkNdXR1hYWGYNWsW0tPTZXruiiArKwuLFi1Cw4YNcezYMYSGhuLChQuwt7eHVMqvlkSk/Pr164enT5/i4sWLYkchOeJvJCJSCBY7iVjsrIx8fX1hbGyMCRMmiB2FiP5Pu3btcO7cOUybNg2Ojo5wcXHBgwcPZHZ+CwsLzJo1C8OGDUNJSYnMzqvMHjx4AG9vbzRu3Bi3bt3CyZMnceTIEVhZWYkdjYjoo6ioqGDSpEkICgoSOwrJEYudRKQQLHYSsdhZ2Rw8eBCHDh3Cpk2b2LuPSMlIJBI4OzsjJSUFDRs2RKtWrbBw4ULk5+fL5Pyenp5QVVXF8uXLZXI+ZXXz5k24u7ujRYsWKCkpQVxcHLZs2YLmzZuLHY2I6JONGDECkZGRMr0RRsqFxU4iUggWO4mABg0a4NGjRygoKBA7Cn2mzMxMjBkzBrt27eJ7G5ES09HRwYIFCxAbG4uEhARYWFhg3759+Nw1WqVSKbZs2YKlS5fi+vXrMkqrPN5OTbe1tUWjRo1w+/ZtBAYGon79+mJHIyL6bDVq1MDQoUOxdu1asaOQnLDYSUQKwWInEaCqqgpjY+Mq2eetMikqKoKTkxO8vb3xzTffiB2HiD5AgwYNsGfPHoSEhGD+/Pn49ttvP7tIaWxsjKVLl8LV1RWvX7+WUVLxlJaWlk1NHzp0KHr27ImMjAz4+flBT09P7HhERDI1adIkbNy4UWYj/km5sNhJRArBYifRG5zKXvHdvXsXenp68PLyEjsKEX0kW1tbXL16FY6OjujevTvGjRuHZ8+effL5hg8fDhMTE8ybN092IRWssLAQW7ZsQYsWLTB37lxMnDgRqampGD9+PLS0tMSOR0QkF6ampujQoQO2b98udhSSAxY7iUgh0tLSYGZmJnYMItGx2FnxmZqa4vDhw1x5mKiCUlVVxdixY5GSkgINDQ1YWFhg1apVKCoq+uhzSSQS/PrrrwgNDcX58+flkFZ+cnJyEBgYiMaNGyMsLAyBgYG4evUqhgwZAlVVVbHjERHJnaenJ1auXPnZrU1I+fBTOhERkQKx2FnxSSQSFjqJKoFatWph5cqV+OOPP3D06FG0bNkSv/3220efp06dOli3bh2GDRuGnJwcOSSVrSdPnsDPzw8mJiaIiYnBgQMH8Pvvv6N79+5cbI2IqpRu3bpBEAScOnVK7CgkY/ykTkREpEAsdhIRKRcLCwtERkYiICAAEyZMgL29PW7fvv1R57C3t4e1tbVSt7e4c+cOxo8fD3Nzczx//hwxMTEIDw9H27ZtxY5GRCQKiUQCT09PBAUFiR2FZIzFTiIiIgVisZOISPlIJBL0798fiYmJ6Ny5M7755htMnz4dr169+uBzBAUFITIyEsePH5dj0o937do1ODo6omPHjqhVqxZu3ryJ4OBgNG7cWOxoRESiGzp0KGJiYj76JhcpNxY7iYiIFMjIyAjPnj1DXl6e2FHoPW7evIm9e/fizJkzePTokdhxiEjBNDQ04Ovri8TERDx9+hRNmjRBaGgoSktL/+ex1atXR2hoKEaNGoXnz58rIO0/EwShbGq6vb09OnbsiLt378Lf3x8GBgaiZiMiUiba2toYOXIkVq9eLXYUkiEWO4lIZiQSCfbu3Svz8y5btgwNGjQo+/O8efPQvHlzmV+HSBFUVFRgYmLCu8dK6ODBgxg8eDDGjx8PBwcHbNmypdx2Nq8nqjoMDQ2xefNmHDp0COvXr0fHjh0RExPzP4+ztbXFkCFDMG7cOFHeM0pKShAeHo527dph8uTJcHFxwZ07dzBt2jRUq1ZN4XmIiCqC8ePHIywsDNnZ2WJHIRlhsZOoCnNzc4NEIsHIkSPf2ebr6wuJRIJ+/fqJkOzfeXt7Izo6WuwYRJ/MzMyMU9mVzJMnT+Du7o6RI0ciLS0NPj4++PXXX5GdnQ1BEFBQUMCFO4iqoPbt2+PChQuYMmUKHBwc4OrqiocPH/7rMf7+/khKSsLOnTsVlBLIz89HcHAwzMzMEBQUhLlz5yIxMRFubm5QV1dXWA4ioorIyMgI3bt3R0hIiNhRSEZY7CSq4oyMjLB7927k5uaWPVdcXIywsDDUr19fxGT/TFdXF/r6+mLHIPpk7NupfJYsWQJbW1t4enqiRo0a8PDwQJ06dTBixAh88803GDduHK5evSp2TCISgUQigYuLC1JSUmBsbIyWLVvC398fBQUF791fU1MTYWFhmDJlCh48eCDXbFlZWfD394eJiQkiIiKwdetWnD9/Ht999x2kUn7VIyL6UJ6enli1ahVKSkrEjkIywN+ARFVcixYtYGpqivDw8LLnjh07Bk1NTdja2pbbNyQkBBYWFtDU1ISZmRkCAwPf6WH1999/w8HBATo6OmjYsCG2bdtWbvuMGTPQpEkTaGlpoUGDBvD19X3ny8KSJUtoTZfdAAAgAElEQVRgaGgIXV1dDBs2DDk5OeW2//c09itXrqBHjx6oXbs2qlevDisrqw+aakYkFhY7lY+Wlhby8/ORlZUFAPDz80NGRgasra3Rq1cv3L59Gxs3bkRhYaHISYlILLq6uli4cCGuXLmCuLg4WFhYYP/+/e+drt6mTRtMnjwZ7u7uKC0thSAIOHv2LA4dOoQjR47g8OHDOHToEKKioj7pi/X9+/fh5eWFRo0aIS0tDVFRUTh8+DA6d+4si5dKRFTlWFpaQl9fH8eOHRM7CskAi51EBA8PD2zevLnsz5s3b4a7u3u5KZsbNmzArFmzMH/+fNy8eRPLly9HQEAA1q5dW+5c8+fPh729PRISEuDo6IgRI0YgMzOzbLuOjg42b96MmzdvYu3atdi1axf8/f3LtoeHh8PPzw8//fQTrl27hiZNmmDFihX/mv/Vq1dwdXXF2bNncfnyZbRq1Qp9+vTBs2fPPvevhkgu/h979x3W1NmwAfwOGxFBtoCKksSBq7j3tra4aRU3gqN1oRarfbV1t1ZtFbW2LkRRaxW0zmrrqgP3qgNlCagoU5G9cr4//MxbXhyMwEnI/bsurjY5Izf8EXPuPOd5WHaqHxsbG4SEhGDGjBnw9vbG+vXrcejQIUydOhULFiyAu7s7duzYwUWLiAh16tRBUFAQNm3ahPnz56N79+74559/iuw3e/ZspKamYs6cOdi7dy/kcjn69++Pvn37ol+/fujfvz9cXV1x4MABBAcHIysr672vfe/ePXh6eqJp06YAgFu3biEgIAAuLi4q/z2JiLSJRCKBj48P/Pz8xI5CqiAQkdYaPXq04ObmJqSkpAhGRkZCWFiY8PTpU8HAwECIiYlRbhcEQahZs6awbdu2QsevXLlSaNCggfIxAGH27NnKx3l5eYKxsbEQGBj41gw///yz4OzsrHzctm1bYezYsYX26d69u1C7dm3l43nz5gkuLi5vPadCoRDs7Oze+bpEYnr06JFgZ2cndgz6H8uWLRMGDx4sfPfdd4Krq6sQHx8v5OfnC4IgCJcuXRJcXV2F0NBQkVMSkTrJy8sT1q1bJ9jY2AgTJ04UkpKSlNvS0tKE1atXC5mZmcU6z9atW4XExMQ3bj937pzQt29fwdbWVli8eLGQkpKist+BiIheycnJEWrUqCH8888/YkehMuLITiJC9erVMXDgQPj7+2Pr1q3o0qVLofk6ExMT8ejRI0yYMAFVq1ZV/syePRuRkZGFztWkSRPl/+vp6cHa2hoJCQnK54KCgtChQwflberTp09HbGyscntoaCjatm1b6Jz/+/h/JSQkYMKECZDL5TAzM4OpqSkSEhIKnZdIndjb2+Ply5dc8VFkeXl5SE5OVj6eOXMmdu3ahcGDByMvLw95eXnQ1dWFIAj44YcfYGVlhfr164uYmIjUjZ6eHj7//HOEhoZCV1cXDRo0wJo1a5CZmYk9e/Zg4sSJMDY2LtZ5Ro4ciaNHjyrnUVcoFMpb00eNGoWPPvoIDx8+xJw5c1C9evXy/tWIiLSOgYEBJk6cyNGdlYCe2AGISD14eXlh9OjRqFq1KhYuXFho2+t5OX/55Re0a9funefR19cv9FgikSiPv3jxIjw8PDBv3jysXLkS5ubmOHDgAHx9fcuUffTo0YiPj8fKlSvh5OQEQ0NDdO/enXPrkdrS0dGBs7MzIiIi4OrqKnYcrRQQEIDDhw/j2LFjGDp0KFatWgVjY2NIJBLUqlUL1apVQ/PmzdG3b1/ExcUhNDQU169fFzs2EakpCwsLrF69GhMmTMC0adNw6NAh7N+/H7q6usU+h0QiwdChQ7Fnzx5kZ2dj+fLlMDIywqxZs+Du7l6icxERUem8HkSzdOlSWFlZiR2HSokjO4kIANC9e3cYGBggKSkJAwYMKLTN1tYWDg4OiIyMhFQqLfJTXOfPn4eDgwO+/vprtGzZEjKZrNB8ngDQoEEDXLx4sdBz//v4f507dw5TpkyBm5sbXFxcYGpqynn1SO3J5XLO2ymS48eP44svvkD9+vWxfPlybNy4sdC8xXp6ejhy5AiGDRuG69evo1mzZti7dy/Mzc1FTE1EmsDFxQV//PEHPDw8YGRkVOLjdXV18eLFC2zbtg1+fn64evUqBg8ezKKTiKiCWFtbY+DAgdiwYYPYUagMOLKTiAC8Gk3wzz//QBAEGBoaFtk+f/58TJkyBebm5vj444+Rl5eH69ev48mTJ/jqq6+K9RpyuRxPnjzBjh070LZtWxw7dgy//vproX18fHwwatQotGzZEl26dEFQUBAuXboECwuLd553+/btaN26NTIyMvDll1/CwMCgZH8AogrGRYrEkZWVBW9vb8ydOxfTp08HAERHRyM9PR0LFy6ElZUVZDIZevbsiR9//BHZ2dmlKiyISHudPXsW/fr1K/XxY8aMgYODA3r06KHCVEREVFw+Pj5wc3PDzJkzi9y5SJqBZScRKZmamr5129ixY2FiYoLly5fjq6++grGxMVxcXDB58uRin79v376YOXMmpk2bhqysLPTq1QsLFy7ExIkTlfsMGTIEUVFRmDNnDjIzM9GvXz/MmDEDAQEBbz2vv78/xo8fj+bNm8Pe3h7z589HYmJisXMRiUEmk+Hvv/8WO4bW+eWXX+Dq6govLy/lc3/99RdevHiBmjVr4smTJ7CysoKjoyMaNGjwxi9/iIjeJTU1FZaWlqU+3tDQEAUFBSpMREREJdG0aVPIZDIEBQVh6NChYsehUpAIgiCIHYKIiEjbnD17FrNmzUJISIjYUbTKxYsXERMTA3d3d+jp6WHp0qVYtmwZzpw5g0aNGiElJQXOzs74/PPP8e2334odl4g00MGDB9G3b1/Rz0FERKX3+++/Y+nSpe+dUo3UE+fsJCIiEgFvYxdHmzZtMGjQIOjp6SEvLw/16tXDX3/9hUaNGkGhUMDCwgK9evVC1apVxY5KRBqKY0mIiDRf3759kZCQwLJTQ7HsJCIiEoGtrS2ys7Px/PlzsaNohZcvXyr/X0/v1Sw++vr66N+/P5o3bw4A0NHRQVpaGqKiolC9enVRchIRASxMiYjEpquriylTpsDPz0/sKFQKLDuJiIhEIJFIOLqzgkyfPh3ff/89YmJiALz6278uEnR0/vtRSKFQYMaMGcjPz8fnn38uSlYi0nw6OjrIzs4u9fEKhQJ5eXkqTERERKXh5eWFY8eOIT4+XuwoVEIsO4mIiEQil8tZdpazzZs3w8/PD35+fvjyyy9x6dIl5OfnQyKRFNrv1q1b8PLywp9//on9+/eLlJaIKoPu3bvjxIkTpT7+3Llz6NixowoTERFRaZiZmSE6Oho2NjZiR6ESYtlJREQkEo7sLF8pKSkICgrC0qVLsX//fly+fBne3t4IDg7GixcvCu1bp04dtGrVClu2bEGtWrVESkxElYGxsTGysrJKfSt6QkICL6yJiNSEqalpkS/JSf2x7CQiIhIJy87ypaOjg169esHFxQXdu3dHaGgoZDIZJkyYgB9//BFRUVEAgLS0NAQFBWHMmDHo1q2byKmJqDLo1q0bgoODS3zckSNH0Lp163JIREREpcGiUzNJBM5+TUTl6IcffsDjx4+xcuVKsaMQqZ0LFy7Ax8cHly9fFjtKpZWVlQVjY+NCz61cuRJff/01evTogS+++AJr165FdHQ0Ll26JFJKIqqMYmJicPXqVQwaNKhYF8t//PEHnJyc0KBBgwpIR0REVHnpiR2AiCq358+fc1Vjord4PbJTEAR+a1xO/l10FhQUQFdXF9OnT0enTp0wcuRI9OnTB5mZmbh9+7aIKYmoMqpduzZMTEywe/duVKtWDR9++GGhRdGAV6uuX7x4EY8fP0br1q05jQYRkQbJyMjAhQsXUL16ddSvXx8mJiZiR6L/x7KTiMrV8+fPUb9+fbFjEKklS0tLAEBycjKsrKxETlP56erqQhAECIKA5s2bY+vWrWjdujV27NjB9ykiKhdWVlYYMmQIOnTogBs3bqBhw4aF3ovy8/PRunVrtG3bVuyoRERUAsnJyfDw8EBiYiLi4+Ph5uaGTZs2iR2L/h9vYyeicvX6LYaj1ojerFWrVli1ahXatWsndhStkpKSgjZt2qBevXo4ePCg2HGIqBKLiIhA+/bt8ejRIxgYGIgdh4iISkGhUODIkSPYsGEDWrVqBalUioULF2LVqlUwMjLCuHHj8NVXX8HT01PsqAQuUERE5UwikbDoJHoHLlJUvt72na4gCBg2bBiLTiIqd/7+/hgxYgSLTiIiDebp6YkvvvgCzZs3x5kzZ/DNN9+gV69e6NWrFzp16oTx48djzZo1Ysek/8eyk4iISERyuZxlZzlJTExEbm7uGwtPS0tLzJs3T4RURKRN8vPzERAQAG9vb7GjEBFRKT148ACXLl3CuHHjMG/ePBw7dgwTJ07E7t27lfvUqFEDhoaGSExMFDEpvcayk4iISEQc2Vk+8vPz8cknn2DlypVvHV3OUedEVN5er7DesGFDsaMQEVEp5ebmQqFQwMPDA8Crz5AeHh5ITk6Gj48PlixZgmXLlsHFxQXW1tZvvbOIKg7LTiIiIhGx7CwfixYtgr6+PmbOnCl2FCLSYps3b+aoTiIiDde4cWMIgoBDhw4pnztz5gxkMhlsbGxw+PBh2NvbY/To0QD4hbo64AJFREREInrx4gVq1qyJly9f8oORipw8eRIjRozA9evXYWdnJ3YcItJSz549Q4MGDRAbGwtTU1Ox4xARURls3LgRa9euRffu3dGiRQvs3LkTdnZ22LRpE548eYJq1arxvV6N6IkdgIiISJuZm5vDyMgI8fHxLOZUID4+HiNHjsTWrVv59yQiUW3duhXu7u68+CUiqgTGjRuHtLQ0bN++Hfv374elpSXmz58PAHBwcADwar54a2trEVPSaxzZSUREJLJ27dph6dKl6NSpk9hRNJpCocBHH32EFi1aYMmSJWLHISItJggC6tevj4CAALRt21bsOEREpCLx8fFITU2FXC4HAKSmpmL//v346aefYGhoCGtrawwaNAj9+vXjl10i4pydRKQyBQUFhR7zuxSi4uG8naqxbNkyZGRkYMGCBWJHISItJ5FI8ODBAxadRESVjI2NDeRyOXJzc7F48WLIZDJ4enoiMTER7u7uqFOnDrZs2YKxY8eKHVWr8TZ2IlIZXV3dQo8lEgkSExORnZ0Nc3NzfrNF9BZyuZxlZxmdP38eK1euxNWrV6Gnx483RERERKR6EokECoUCCxcuxJYtW9ChQweYm5sjOTkZZ8+eRVBQEMLCwtChQwccPXoUvXv3FjuyVuLITiJSiezsbIwfPx55eXkAgNzcXKxbtw7e3t4YN24cpk2bhps3b4qckkg9cWRn2aSkpGDYsGHYtGkTatasKXYcIiIiIqrErl69ih9++AG+vr5Yv349/P39sW7dOsTExGDFihWQy+Xw8PDAjz/+KHZUrcWyk4hUIj4+Hps2bYK+vj5yc3Oxdu1aTJs2DSYmJpDJZLh48SJ69OiBmJgYsaMSqR2WnaUnCALGjBkDd3d39O3bV+w4RERERFTJXbp0Cd26dYOPj49yQSIHBwd069YN9+7dAwD07t0bDRs2RHZ2tphRtRbv8yIilUhJSYGZmRkA4OHDh9i4cSNWrVqFiRMnAng18rN///74/vvvsW7dOjGjEqkdqVSKyMhIKBQK6Ojwe8iSWL16NeLi4rBnzx6xoxARERGRFrC0tERoaCjy8/NhYGAAAAgLC8O2bdvg6+sLAGjTpg3atWsHIyMjMaNqLV5REZFKJCQkoHr16gCgfNMfNWoUFAoFCgoKYGRkhE8//RS3bt0SOSmR+jE1NUW1atUQFxcndhSNcvXqVSxevBi//fab8oMmEZHY5s+fj0aNGokdg4iIysmwYcOgq6uL2bNnw9/fH/7+/pg7dy5kMhkGDRoEALCwsIC5ubnISbUXy04iUonU1FRER0fDz88PS5YsAQDk5ORAR0dHuXBRWlpakRXbiegV3speMqmpqfDw8MBPP/2EunXrih2HiDSEp6cnJBKJ8sfKygp9+vTB/fv3xY5WIU6fPg2JRIKkpCSxoxARabSAgADExcVhwYIFWLVqFZKSkjB79mzUqVNH7GgE3sZORCpiZWWFZs2a4eDBg0hOToZcLsfTp09haWkJ4FXRGRoaCrlcLnJSIvUkk8kQFhaGrl27ih1F7QmCgPHjx6Nnz54YPHiw2HGISMP06NEDgYGBAIC4uDjMnDkTAwcORGhoqMjJ3i03N5ej2ImI1ET79u3RunVrPHv2DM+fP0fjxo3FjkT/wpGdRKQSXbp0wV9//YV169Zh/fr1mDlzJmxtbZXbw8PDkZ6ejt69e4uYkkh9yeVyjuwspo0bN+L+/ftc4ZKISsXQ0BB2dnaws7ODq6srpk+fjvv37yMrKwvR0dGQSCS4evVqoWMkEgmCgoKUj+Pi4jB8+HBYWlqiSpUqaNasGU6dOlXomF27dsHZ2RmmpqYYMGBAodGUV65cQa9evWBlZYVq1aqhQ4cOuHDhQpHX/OmnnzBo0CCYmJjgP//5DwDg3r17cHNzg6mpKWxsbDB06FA8e/ZMedzt27fRvXt3VKtWDaampmjatClOnTqF6Oho5Rdq1tbWkEgk8PT0VMnflIhIG+np6cHR0ZFFpxriyE4iUokTJ04gLS1NOUfJa4IgQCKRwNXVFTt37hQpHZH6k8lkCAkJETuG2rt9+zbmzJmDs2fPwtjYWOw4RKTh0tLS8Ntvv6Fx48bFfk/JyMhA586dYWNjg3379sHBwaHInOTR0dH47bffsG/fPmRkZMDDwwNz5szB+vXrla87cuRI+Pn5QSKRYO3atfj4448RHh4OKysr5XkWLFiAb7/9FitWrIBEIsHTp0/RqVMneHt7Y8WKFcjLy8OcOXPQr18/XLx4ETo6Ohg2bBiaNm2Ky5cvQ09PD7dv34aRkRFq1qyJ4OBguLu74+7du7CwsOD7KBERVUosO4lIJfbu3Yv169ejd+/eGDJkCPr27QsLCwtIJBIAr0pPAMrHRFQY5+x8v4yMDAwePBg//PAD6tevL3YcItJQR48eRdWqVQG8el+pWbMmjhw5Uuzjd+7ciWfPnuHChQvKYtLZ2bnQPvn5+QgICICZmRkAYPz48diyZYtye7du3Qrtv2bNGgQHB+Po0aMYMWKE8vkhQ4Zg7NixysfffPMNmjZtiu+//1753LZt22BhYYGrV6+iVatWiImJga+vr/J9UiqVKve1sLAAANjY2BQqVYmIqGxeX+8CvOZVB7yNnYhU4t69e/jwww9hYmKCuXPnYvTo0dixY4dydenXCwEQ0Zs5Ozvj4cOHXMTrHSZPnozWrVtj1KhRYkchIg3WqVMn3Lx5Ezdv3sSlS5fQrVs39OrVC48ePSrW8Tdu3ECTJk3eWRbWrl1bWXQCgL29PRISEpSPExISMGHCBMjlcpiZmcHU1BQJCQmIjY0tdJ4WLVoUenzt2jWcOXMGVatWVf7UrFkTABAZGQkAmDFjBsaOHYtu3bphyZIlWrP4EhGRmCQSCZYsWQJ/f3+xoxBYdhKRisTHx8PLywuBgYFYsmQJcnNzMWvWLHh6emL37t2FPuATUVFVqlSBlZVVsS+2tU1gYCAuXLiAtWvXih2FiDRclSpVIJVKIZVK0apVK2zevBkvX77Ehg0boKPz6vLo3yN08vLyCh3/721vo6+vX+ixRCKBQqFQPh49ejSuXLmClStXIiQkBDdv3oSjoyNyc3MLHWdiYlLosUKhgJubm7Ksff0THh6OPn36AADmz5+Pe/fuYcCAAQgJCUGTJk148U1EVAFatWoFPz+/Yv07QeWLZScRqURaWhqMjIxgZGSEUaNG4ciRI1i1ahUkEgnGjBmDfv36ISAgoMiHeCL6L97K/mYPHjzAjBkzsHv3buWtp0REqiKRSKCjo4PMzExYW1sDAJ4+farcfvPmzUL7u7q64p9//im04FBJnTt3DlOmTIGbmxtcXFxgampa6DXfxtXVFXfv3kXt2rWVhe3rH1NTU+V+MpkMU6dOxeHDh+Ht7Y1NmzYBgHI1d95FQESkej179kR+fn6RBeuo4rHsJCKVyMjIUF4g5OfnQ1dXF5988gmOHTuGP/74A/b29vDy8lLe1k5ERclkMoSFhYkdQ61kZWVh8ODBWLx4MZo0aSJ2HCKqBHJycvDs2TM8e/YMoaGhmDJlCtLT09G3b18YGxujTZs2+P7773H37l2EhITA19e30PHDhg2DjY0NBgwYgLNnz+Lhw4c4cOBAiS5u5XI5tm/fjnv37uHKlSvw8PBQFpHvMmnSJKSmpmLIkCG4dOkSoqKicPz4cYwfPx5paWnIysrCpEmTcPr0aURHR+PSpUs4d+4cGjZsCODV7fUSiQSHDx9GYmIi0tPTS/bHIyKit5JIJPDx8YGfn5/YUbQey04iUonMzEzl3FR6eq/WPlMoFBAEAZ06dcLevXtx69YtODo6ihmTSK1xZGdRX3zxBerXr4/x48eLHYWIKonjx4+jRo0aqFGjBlq3bo0rV65gz5496NKlCwAob/lu2bIlJkyYgMWLFxc63sTEBH///TccHBzQt29fuLi4YN68eSWam9zf3x/p6elo3rw5PDw84OXlBScnp/ceZ29vj/Pnz0NHRwe9e/eGi4sLJk2aBENDQxgaGkJXVxfPnz/H6NGjUa9ePQwcOBBt27bFjz/+CABwcHDAggULMGfOHNja2mLy5MnFzkxERO83cuRIhISEKOdRJnFIBE4mQEQqkJKSAnNzc+VcV/8mCAIEQXjjNiL6rwMHDmD9+vU4fPiw2FHUQlBQEGbNmoXr168XWuiDiIiIiEhdzZo1Czk5OVi1apXYUbQWy04iIiI1ERoaiv79+/NWdgBRUVFo06YNDh8+jJYtW4odh4iIiIioWGJjY9GsWTNER0ejWrVqYsfRShxmRUTl4vVoTiIqvrp16yI2Nhb5+fliRxFVbm4uPDw88J///IdFJxERERFplFq1aqFHjx4ICAgQO4rWYtlJROXiwoULOHfunNgxiDSKoaEhatSogejoaLGjiOqrr76CnZ0dfHx8xI5CRERERFRiPj4+WL16NRQKhdhRtBLLTiIqF8eOHcOJEyfEjkGkcbR9kaJDhw5hz5492LJlS4kW+yAiIiIiUhft2rVD9erVORe/SFh2ElG5eP78OapXry52DCKNI5PJtHbOzsePH2Ps2LHYuXMnLC0txY5DRERERFQqEokEPj4+8PPzEzuKVmLZSUTlgmUnUelo68jO/Px8DB06FD4+PujQoYPYcYiI3qlt27Y4dOiQ2DGIiEiNDR48GPfu3cOdO3fEjqJ1WHYSUblg2UlUOnK5XCvLzvnz58PY2BizZs0SOwoR0TvdvXsXsbGx6N27t9hRiIhIjRkYGOCzzz7j6E4RsOwkonLBspOodLRxZOfx48exZcsWBAYGQkeHH02ISL1t3rwZnp6e0NPTEzsKERGpuc8++wxBQUFISkoSO4pW4RUFEZULlp1EpePk5IS4uDjk5uaKHaVCPHv2DKNGjcK2bdtga2srdhwionfKycnB9u3b4eXlJXYUIiLSADY2NhgwYAA2btwodhStwrKTiMoFy06i0tHX10fNmjURFRUldpRyp1AoMHLkSIwdOxbdu3cXOw4R0XsdOHAAjRo1grOzs9hRiIhIQ/j4+OCnn35CXl6e2FG0BstOIioXLDuJSk9bbmVfunQpcnJy8M0334gdhYioWDZv3gxvb2+xYxARkQZp1qwZpFIpgoODxY6iNVh2EpHKZWVlAQCMjY1FTkKkmbSh7Dx79ixWr16NnTt3ct47ItIIsbGxuHLlCgYNGiR2FCIi0jA+Pj5cqKgCsewkIpXjqE6ispHJZAgLCxM7RrlJSkrC8OHDsXnzZjg6Ooodh4ioWLZs2YKhQ4fyy1wiIiqxfv364dmzZ7h8+bLYUbQCy04iUjmWnURlI5fLK+3ITkEQMGbMGAwePBhubm5ixyEiKhaFQoEtW7bwFnYiIioVXV1dTJ48maM7KwjLTiJSOZadRGVTmW9jX7VqFRISEvDtt9+KHYWIqNhOnDgBCwsLfPDBB2JHISIiDeXt7Y0//vgDT548ETtKpceyk4hUjmUnUdnUqlULiYmJyvlvK4vLly/ju+++w65du2BgYCB2HCKiYtu0aRPGjh0rdgwiItJg5ubmGDZsGH7++Wexo1R6LDuJSOVYdhKVja6uLpycnBAZGSl2FJVJTU2Fh4cHfv75Z9SpU0fsOERExZaUlIRjx45h2LBhYkchIiINN2XKFGzYsKHSDWpQNyw7iUjlWHYSlV1lupVdEASMHTsWH330Edzd3cWOQ0RUItu3b0efPn1gbm4udhQiItJw9erVQ8uWLbFz506xo1RqLDuJSOVYdhKVXWUqO9evX4/w8HD88MMPYkchIioRQRCwefNm3sJOREQq4+PjAz8/PwiCIHaUSotlJxGpHMtOorKTyWQICwsTO0aZ3bp1C19//TV2794NIyMjseMQEZXIlStXkJWVhc6dO4sdhYiIKomePXsiPz8fp0+fFjtKpcWyk4hUjmUnUdlVhpGd6enpGDx4MFauXAm5XC52HCKiEtu0aRO8vLwgkUjEjkJERJWERCLB1KlT4efnJ3aUSotlJxGpHMtOorKTy+UaX3ZOmjQJ7du3x4gRI8SOQkRUYhkZGQgKCoKnp6fYUYiIqJIZOXIkzp07V6kWJFUnLDuJSOVYdhKVnYODA168eIH09HSxo5TK1q1bceXKFaxZs0bsKEREpbJnzx60b98e9vb2YkchIqJKxsTEBN7e3li7dq3YUSollp1EpHIsO4nKTkdHB87OzoiIiBA7SomFhobC19cXu3fvhomJidhxiIhKZdOmTVyYiIiIys2kSZOwbds2vHz5UuwolTlX4AAAACAASURBVA7LTiJSOZadRKqhifN2ZmVlYciQIfj222/RqFEjseMQEZXK/fv3ERkZiY8//ljsKEREVEnVqlUL3bp1Q0BAgNhRKh2WnUSkciw7iVRDE8vO6dOnw8XFhaOhiEij+fv7Y9SoUdDX1xc7ChERVWLTpk3DmjVroFAoxI5SqbDsJCKVys7OhkKhgLGxsdhRiDSeTCZDWFiY2DGK7bfffsPx48exfv16rlxMRBorLy8P27Ztg7e3t9hRiIiokmvXrh3MzMxw5MgRsaNUKiw7iUilXo/qZNFBVHaaNLIzMjISU6ZMwe7du1GtWjWx4xARldqhQ4cgl8shl8vFjkJERJWcRCKBj48P/Pz8xI5SqbDsJCKV4i3sRKojl8s1ouzMycnBkCFDMHfuXLi6uoodh4ioTDZv3sxRnUREVGEGDx6MO3fu4M6dO2JHqTRYdhKRSrHsJFIdOzs7ZGVlITU1Vewo7zR79mw4OjpiypQpYkchIiqTJ0+eICQkBJ988onYUYiISEsYGhri888/x+rVq8WOUmmw7CQilWLZSaQ6EokEUqlUrUd3HjhwAPv27YO/vz+nryAijRcQEIDBgwfDxMRE7ChERKRFJkyYgD179iA5OVnsKJUCy04iUimWnUSqpc7zdsbGxmLcuHHYuXMnLCwsxI5DRFQmCoWCt7ATEZEobG1t0b9/f2zYsEHsKJUCy04iUimWnUSqpa5lZ15eHoYOHYoZM2agXbt2YschIiqz06dPw9TUFC1atBA7ChERaSEfHx+sW7cOeXl5YkfReCw7iUilWHYSqZa6lp3z5s2DqakpZs6cKXYUIiKVCA4Ohre3N6fkICIiUXzwwQeoW7cu9u7dK3YUjceyk4hUimUnkWrJZDKEhYWJHaOQP//8E9u2bcO2bdugo8OPEkSk+QRBwNq1azFp0iSxoxARkRbz8fGBn5+f2DE0Hq9QiEilWHYSqZZcLlerkZ1Pnz6Fp6cnAgMDYWNjI3YcIiKVkEgkkEgk0NXVFTsKERFpsf79++Pp06e4fPmy2FE0GstOIiqz5ORk7N+/HwcOHICBgQESExNx6dIlCIIgdjQijWdlZQWFQqEWKzMWFBRgxIgRGD9+PLp27Sp2HCIiIiKiSkVXVxeTJ0/m6M4ykghsI4iolG7cuIGoqChYWFigU6dOhUZDxMbG4vLly9DX10evXr1gbGwsYlIizdayZUusWbMGbdq0ETXHokWLcPLkSRw/fpyjn4iIiIiIysGLFy9Qt25d3LlzB/b29mLH0UgsO4moVA4ePIi6devCxcXlnfvl5ubit99+Q+/evWFtbV1B6Ygql2HDhuGjjz7CyJEjRcvw999/Y8iQIbh+/To/dBERERERlaNJkybBwsICixYtEjuKRuJt7ERUYgcPHsQHH3zw3qITAAwMDDBixAj89ddfSE1NrYB0RJWP2CuyJyYmYsSIEdiyZQuLTiIiIiKicjZ16lRs2LAB2dnZYkfRSCw7iahErl+/DmdnZzg6Ohb7GIlEAg8PDxw+fLgckxFVXmKWnQqFAqNHj1aOLiUi0lSJiYnYtGkTfvnlF/z88884f/682JGIiIjeqF69emjevDl27twpdhSNpCd2ACLSLA8fPoS7u3uJj9PR0UHdunXx+PHjEhWlRPSq7AwLCxPltX/88Uc8f/4cixcvFuX1iYhUYf/+/Vi+fDnu3r0LExMTODg4ID8/H7Vr18ann36Kfv36wcTEROyYRERESj4+Pvjyyy8xZswYSCQSseNoFI7sJKJiS0xMhJWVVamPb926NS5duqTCRETa4fXIzoqeZvvSpUtYtmwZdu3aBX19/Qp9bSIiVZo1axZat26NqKgoPH78GCtWrMDgwYORn5+PZcuWYfPmzWJHJCIiKqRXr17Iy8vD6dOnxY6icVh2ElGxhYSEoGPHjqU+XiKRQEeHbztEJWVhYQEDAwMkJCRU2Gs+f/4cHh4eWL9+PWrXrl1hr0tEpGpRUVF48eIFZsyYgerVqwMAOnbsiFmzZmHdunUYMGAApk2bhl9//VXkpERERP8lkUgwdepU+Pn5iR1F47B1IKJi09HRKXNZqaenV+Gj04gqg4qct1MQBIwdOxZ9+/bFwIEDK+Q1iYjKi0QigaWlJdavXw/g1XtcQUEBBEGAo6Mj5s2bB09PTxw/fhx5eXkipyUiIvqvkSNH4ty5c4iKihI7ikZh2UlExaaKklIikfBCgqgUKrLsXLduHaKjo7F8+fIKeT0iovJUp04dfPrpp9i1axd27doFANDV1S00/1ndunVx7949TtlBRERqxcTEBF5eXli7dq3YUTQKFygiogoVGRkJKysrSKVSyGQySKXSQj92dnacfJnoDSqq7Lx58ybmz5+PkJAQGBoalvvrERGVJ0EQIJFIMGnSJCQmJmLkyJFYuHAhPvvsM3z44YeQSCS4ceMGduzYgYkTJ4odl4iIqIjJkyfjgw8+wIIFC2Bqaip2HI0gEXg/KREV09mzZyGXy2Fra1vqcwQFBaF79+6IiIgo8hMeHo7MzMwiBejrH3t7e875SVpr165dCA4Oxp49e8rtNdLS0tC8eXMsWLAAQ4cOLbfXISKqSKmpqUhLS4MgCEhOTkZQUBB27tyJmJgY1KlTB6mpqfDw8MCqVaugq6srdlwiIqIiPv30U3Tq1AlTpkwRO4pGYNlJRMUmCAL27t0Ld3f3Uh3//PlzXL9+Hd27d3/rPqmpqYiMjHxjEZqamgpnZ+c3FqE1a9ZkEUqV2rVr1+Dl5YVbt26Vy/kFQcDIkSNhbGyMjRs3lstrEBFVpNTUVPj7+2PhwoWoUaMGCgoKYGtrix49emDAgAHQ19fHjRs38MEHH6BBgwZixyUiInqrc+fOYcyYMXjw4AGve4uBt7ETUbG9Xk09Pz8fenolf/s4ffo0+vXr9859zMzM4OrqCldX1yLb0tPTCxWhV69exa+//oqIiAgkJyejTp06RUpQmUyGmjVrliovkTqRyWSIiIhQ3pKpagEBAbh58yYuX76s8nMTEYlhyZIlOHfuHH755RdYWFhg7dq1OHjwILKysnDy5EmsWLECw4YNEzsmERHRe7Vv3x7VqlXDkSNH0KdPH7HjqD2O7CSiEklPT8eBAwdKfHEQFhaGuLg4dOnSpVxyZWZmIioqqtBI0Nf/Hx8fj9q1axcpQaVSKWrXrs3FCEhj2NnZ4dq1a3BwcFDpee/du4fOnTvj9OnTcHFxUem5iYjE4uDggA0bNsDNzQ0AkJiYiBEjRqBz5844fvw4Hj9+jMOHD0Mmk4mclIiI6P0CAwOxbds2/PXXX2JHUXssO4moxJ48eYKQkBB88sknxRphFhYWhvDwcOXFRkXLzs7Gw4cPi5SgERERiIuLg6OjY5ESVCqVok6dOjAwMBAlM9GbdOzYEYsWLVLplwaZmZlo1aoVZsyYAS8vL5Wdl4hITBEREfj000+xevVqdOzYUfm8jY0Nrly5gtq1a6N+/fr47LPPMG3atHIbNU9ERKQqOTk5cHJywvHjxzlA4T1YdhJRqSQnJ+Po0aNo0KDBG285B4AXL17g1KlTMDc3R9euXSs4YfHk5uYiOjq6SAkaERGBR48eoUaNGm9cOb5u3bowMjISOz5pGS8vL7Rt2xbjxo1T2TnHjRuHrKwsBAYG8kKfiCoFQRBQUFCAQYMGwczMDBs3bkRmZiYCAwPx7bffIj4+HgDg6+uL6Oho7Nq1i9PdEBGRRliwYAHi4uKwfv16saOoNf6rTkSlYmlpieHDhyMyMhJBQUHQ1dWFoaEhDA0NkZ6ejry8PJiZmaFv375qfQFhYGAAuVwOuVxeZFteXh5iY2MLFaEnT55EREQEoqOjYWNjU6QElUqlcHZ2RpUqVUT4baiyk8lkCA8PV9n5fv31V/z999+4du0ai04iqjQkEgn09PTwySef4PPPP0dISAhMTEyQmpqKZcuWFdo3NzdXrT+nEBER/dtnn32G+vXrY/r06bh//36hxYpMTU3RuXNnLmAEjuwkIhXKy8tDbm4uqlSpUumLk4KCAsTGxhYZDRoREYGoqChYWlq+cdV4qVSKqlWrVkjGrKws7NmzB7du3YKpqSk+/PBDtGzZkhd1GiwoKAg7duzAvn37ynyu8PBwtGvXDn/++Sc++OADFaQjIlI/iYmJ8Pf3R0JCAkaPHo0mTZoAAO7fv4/OnTtj48aN7108kYiISF1cv34dO3fuRNeuXfHRRx8VKjaTkpJw5swZCIKAHj16wMzMTMSk4mLZSUSkYgUFBXjy5EmREjQ8PByRkZEwMzN7axGqyn+QHj16hKVLlyI9PR2BgYHo3bs3AgICYGNjAwC4cuUKjh8/jqysLMjlcrRp0wbOzs6FimrOYaZebt26heHDh+POnTtlOk9OTg7atWsHLy8vTJo0SUXpiIg0Q1paGn777TecPHkSO3fuFDsOERFRsRw8eBDOzs5o2LDhO/dTKBTYs2cP2rRpg9q1a1dQOvXCspOIqAIpFAo8ffq0SAn6+v+rVKlSpAB9fat89erVS/RaBQUFiIuLQ82aNdG8eXN07twZixcvVt5i7+npiaSkJBgYGODx48fIzs7G4sWLlSNcFAoFdHR08OLFCzx79gx2dnYwNzdX+d+Eii8jIwNWVlbIyMgo0+0pPj4+ePToEYKDg1lmE5FWio+PhyAIsLOzEzsKERHRex06dAjNmjWDo6NjsY/Zt28f2rVrB1tb23JMpp5YdhIRqQlBEBAfH//GEjQ8PBz6+vpFStBevXrB2tr6vYWVnZ0dZs6cienTpytLsgcPHsDExASOjo5QKBTw9fXF1q1bce3aNTg5OQF4dZvfggULEBISgvj4eLRo0QIBAQGQSqXl/eegt3B0dMT58+dL/S3t77//junTp+P69eslLtCJiIiIiKhi/fPPPwCgnIqluARBwK+//ophw4aVRyy1xrKTiEgDCIKApKSkIiXoV199hUaNGr2z7MzIyICNjQ38/f0xZMiQt+6XkpICGxsbXLhwAS1btgQAtG/fHpmZmfjll1/g6OgIb29v5OXl4dChQzA2Nlb570nv17VrV8yZMwc9evQo8bExMTFo2bIlDhw4gDZt2pRDOiIi9fP6cocj2YmISBMFBwfD3d29VMfeuXMH+vr6qFevnopTqTeuUkFEpAEkEgmsra1hbW2Ntm3bFuuY1/NtPnz4EBKJRDlX57+3vz43AOzfvx/6+vqQyWQAgJCQEFy4cAE3b95Ufou4cuVKuLi44OHDh++dK4bKx+sV2Utadubl5cHDwwNffvkli04i0ipTp07F119/XeTfQSIiInX34sWLMk0l1qhRI+zdu1fryk6uR09EVEkpFAoAQGhoKKpVqwYLC4tC2/+9+ND27dsxb948TJ8+Hebm5sjJycGxY8fg6OiIJk2aID8/HwBgZmYGOzs73L59u2J/GVJ6XXaW1Ndff43q1atjxowZ5ZCKiEg9RUVFYdeuXVq9Ii0REWmus2fPokuXLmU6R1nm+tdUHNlJRFTJ3bt3DzY2Nsr5GQVBgEKhgK6uLjIyMjB//nwEBwdj4sSJmD17NoBXq3WHhoZCLpcD+G9xGh8fD2tra6SmpirPxdsCK5ZMJsOZM2dKdMzRo0exY8cOXL9+XSs/7BCR9tqyZQuGDx8OQ0NDsaMQERGViq6ubpmOr1q1KrKysrRqGjKWnURElZAgCHjx4gUsLS0RFhYGJycn5aiW10XnrVu34OPjgxcvXmDdunXo3bt3ofIyPj5eeav661veY2NjoaurW2SU6Ot94uPjYWVlBT09/vNSXko6sjMuLg5jxozBrl27YG1tXY7JiIjUS0FBAbZs2YI//vhD7ChERESloopldgwNDZGdnc2yk4iINNuTJ0/Qq1cvZGdnIzo6GnXq1MH69evRuXNntG7dGoGBgfjhhx/Qvn17fPfdd6hWrRqAV/N3CoKAatWqITMzE1WrVgXw328Tb926BWNjY+Vq7f87qrN37964f/8+atWqVWTleKlUCicnJ+jr61fcH6IScnZ2RnR0NPLz899bKhcUFGD48OGYOHEiOnfuXEEJiYjUw7Fjx+Dg4IDGjRuLHYWIiEg0qampWjedC8tOIqJKyMHBAbt27cKNGzcQFxeHa9eu4eeff8alS5ewevVqTJ8+HSkpKbC3t8eKFStQr149yGQyNG7cGIaGhpBIJKhXrx4uXryIuLg42NvbA3i1iJGrq6vy9vZ/k0gkuHnzJnJycvDw4UPlivEPHjzA4cOHERERgSdPnsDBwaFICSqVSlGnTh3eZlgMRkZGsLW1RUxMDJydnd+57+LFi6Gjo4P//Oc/FZSOiEh9bN68Gd7e3mLHICIiKrVatWohMjLyvZ/73yU3N1frprKSCKoYE0tERBrl/v37CA8Px99//43bt28jKioKMTEx8PPzw4QJE6Cjo4MbN25g2LBhcHNzw8cff4xffvkFx48fx6lTp9C0adNSvW5ubi5iYmIQERGB8PBwZSEaERGB2NhY2NnZvbEIrVu3rlbddvE+PXv2xBdffIHevXu/dZ9Tp05h2LBhuH79OmrUqFGB6YiIxBcfH4969eohNjZWefcCERGRJgoODoa7u3upjk1LS8OFCxfQq1cvFadSbyw7iYhISaFQFPrWb9++fVi2bBmioqLQsmVLzJ8/Hy1atCiX187Pz0dsbGyREjQiIgIPHz6EtbV1kRJUKpXC2dkZJiYm5ZJJXU2cOBENGjTAlClT3rg9ISEBrq6u8Pf317oPNkREALBixQrcvXsXW7ZsETsKERFRmRw+fBjdunUr1eCPAwcO4KOPPtK6qcRYdhJRmXl6eiIpKQmHDh0SOwqVIzFXXi8oKMCjR4+KlKARERGIioqCubl5kRL09Y+pqakomctLfn4+Zs+ejZcvX6JPnz6QSCRwcnJSzkmnUCjg5uaGZs2a4bvvvhM5LRFRxRMEAQ0bNsTGjRvRoUMHseMQERGVSW5uLn799VeMGjWqRNdj4eHhePToEbp161aO6dQTy04iLeDp6YmtW7cCAPT09FC9enW4uLjgk08+wfjx48v8LY8qys7Xi+hcuXKl3EYOUuWkUCjw5MmTIiVoeHg4IiMjYWpq+sYSVCqVwtzcXOz4xRYfH4/z589DR0cHnTt3RvXq1ZXbHjx4gDt37sDY2Bg3b97E4cOHcfr0aa37BpeICADOnz8Pb29vhIaGivYlHRERkSqlpKTg8OHDGD58eLHm3wwPD0dYWBjc3NwqIJ364QJFRFqiR48eCAwMREFBARITE3Hy5EnMmzcPgYGBOHHixBtvA87NzYWBgYEIaYmKT0dHBzVr1kTNmjXRtWvXQtsEQcDTp08LlaB79+5V3ipvZGT0xhJUJpPBwsJCpN+oqMuXL+PFixcYOHDgGy/c69Wrh3r16iEjIwOHDh3C6tWrWXQSkdZ6vTARi04iIqosLCwsMHDgQOzatQu1atVC+/bt3/jvXEpKCk6fPg0LCwutLToBjuwk0gpvG3l5584duLq64quvvsKCBQvg5OQET09PxMbGYu/evejZsyf27NmD27dvY/r06Th//jyMjY3Rr18/+Pn5wczMrND527RpgzVr1iAjIwOffvop1q1bp5xXRBAELF++HOvXr0dcXBykUilmzZqFESNGAECRN+rOnTvj9OnTuHLlCubMmYPr168jNzcXTZo0wfLly9G2bdsK+MtRZSYIAhISEoqMBn39X11d3TeWoFKpFFZWVhV2EX358mXo6OgUe8SzIAjYvXs3evToAUtLy3JOR0SkXl6+fInatWvj/v37sLW1FTsOERGRyj179gznz5+HRCKBnp4edHR0oFAokJOTA0tLS3Tu3Bm6urpixxQVy04iLfCu28z79euHqKgo3LlzB05OTkhJScHcuXMxaNAgCIIABwcHyGQytGzZEosWLUJKSgrGjRuHxo0bIzg4WHn+4OBg9O7dG/PmzcOTJ0/g5eUFd3d3rF69GgAwZ84cBAUFwc/PD/Xq1cOFCxcwbtw47N69G25ubrhy5QpatWqFo0ePomnTpjAwMICFhQVOnjyJJ0+eoEWLFpBIJFi7di127NiB8PBwWFlZVejfkbSHIAhITk4uUoK+/snPz39jCSqVSmFra6uyIjQ+Ph43b97Ehx9+WOL8O3bsUH6ZQESkLTZu3IgjR45g3759YkchIiIqd4IgQKFQaH25+b9YdhJpgXeVnbNnz8bq1auRmZmpXOTk4MGDyu0bN26Er68vHj9+rFzo5fTp0+jatSvCw8MhlUrh6emJ33//HY8fP0bVqlUBANu3b4e3tzdSUlIAAFZWVvjzzz/RsWNH5bmnTZuGsLAwHDlypNhzdgqCAHt7eyxfvpxFDokmJSUFkZGRb1w5PjMz840lqFQqRY0aNYo1x85re/fufeut6+9z//595Ofno1GjRiU+lohIU7Vp0wZff/21Vt+6R0REpO04ZyeRlvvfFbb/t2gMDQ1FkyZNCq1o3a5dO+jo6ODevXuQSqUAgCZNmiiLTgBo27YtcnNzERkZiZycHGRnZ6N3796FXisvLw9OTk7vzJeQkICvv/4ap06dQnx8PAoKCpCVlYXY2Niy/NpEZWJhYQELCwu0bNmyyLbU1NRCRei5c+cQEBCAiIgIpKamwtnZ+Y0rxzs6OhYqQgsKCiCRSEo9SrR+/foICgpi2UlEWuPOnTt49OhRiUfDExERUeXCspNIy927dw9169ZVPv7fhYr+twz9t+KWMAqFAgBw8OBB1KpVq9C29y2iMnr0aMTHx2PlypVwcnKCoaEhunfvjtzc3GK9NlFFMzMzg6urK1xdXYtsS0tLQ2RkpHIU6OXLl7Fz505EREQgOTkZdevWVZafhoaGmDlzZpmyGBkZIScnB4aGhmU6DxGRJti8eTM8PT2hp8dLHCIiIm3GTwJEWuzOnTs4evQo5s6d+9Z9GjZsCH9/f6SlpSlHd4aEhEChUKBBgwbK/W7fvo2MjAxlWXrx4kUYGBjA2dkZCoUChoaGiImJQbdu3d74Oq9XfS8oKCj0/Llz57B69Wrl7Wjx8fF4+vRp6X9pIhGZmpqiWbNmaNasWZFtGRkZiIqKUhah9+/fR/Xq1cv0enZ2dkhOToa9vX2ZzkNEpO5ycnKwfft2XLx4UewoREREJDKWnURaIicnB8+ePYNCoUBiYiJOnDiBb7/9Fs2bN4evr+9bjxs+fDjmzZuHUaNGYeHChXj+/DkmTJiAQYMGKW9hB4D8/Hx4eXnhm2++QVxcHGbPno1x48Ypy09fX1/4+vpCEAR06tQJ6enpuHjxInR0dDB+/HjY2NjA2NgYx44dg5OTE4yMjGBmZga5XI7t27ejdevWyMjIwJdffqksRokqExMTEzRu3BiNGzcGABw4cKDM56xSpQoyMjLKfB4iInW3f/9+NG7cGM7OzmJHISIiIpEVf5UEItJox48fR40aNVCrVi10794dBw4cwLx583DmzJkit67/W5UqVXDs2DG8fPkSrVq1Qv/+/dG2bVv4+/sX2q9z585wcXFB165dMXDgQHTr1g3Lli1Tbl+0aBHmz5+PFStWwMXFBT179kRwcDDq1KkDANDT08Pq1auxadMm2Nvbo3///gAAf39/pKeno3nz5vDw8ICXl9d75/kkqgxUsaJ7amoqzM3NVZCGiEi9bd68GWPHjhU7BhEREakBrsZORESkhm7fvg0DAwPUq1ev1OfYu3cvBgwYUKIV4ImINE1MTAyaN2+OR48ewdjYWOw4REREJDJe/RAREamhxo0b486dO6U+/vXCYCw6iaiy27JlCzw8PFh0EhEREQDO2UlERKS2jI2NCy38VRJnzpxBp06dyiEVEZH6KCgowJYtW7B//36xoxAREZGa4HAPIiIiNdW9e3fs3bsXJZ1xJjU1FUlJSbCysiqnZERE6uHEiROwsrJCs2bNxI5CREREaoJlJxERkZoyNDTEhx9+iF27dhW78ExNTcXvv/8Od3f3ck5HRCS+TZs2wdvbW+wYREREpEa4QBEREZGaS0lJweHDh9GiRQs0aNDgjfsoFAr8/fffSE5Ohru7u0pWcyciUmdJSUmQSqWIjo6Gubm52HGIiIhITbDsJCIi0hB37tzBgwcPYGRkBFtbW1SpUgWpqal4+vQpAKBTp068dZ2ItMaqVatw7do1BAYGih2FiIhIpZ49e4ZRo0bh/PnzyMzMLPG0Vv/m6emJpKQkHDp0SIUJ1RvLTiIiIg2Tm5uLpKQkZGZmwszMDJaWllx1nYi0iiAIaNy4MdauXYsuXbqIHYeIiKhEPD09sXXr1iLPt27dGhcvXoSvry+OHj2Kffv2wdTUFHZ2dqV+rdTUVAiCoFV3QXA1diIiIg1jYGAAe3t7sWMQEYnm8uXLyMnJQefOncWOQkREVCo9evQocneCgYEBACAiIgLNmzeHTCYr9fnz8/Ohq6sLMzOzMuXURBwGQkREREREGmXTpk3w8vLi/MRERKSxDA0NYWdnV+jHwsICTk5O2L9/P7Zt2waJRAJPT08AQGxsLAYOHAhTU1OYmppi0KBBePz4sfJ88+fPR6NGjRAQEABnZ2cYGhoiIyMDnp6e6NOnj3I/QRCwbNkyODs7w9jYGI0bN8b27dsr+tcvVxzZSUREREREGiM9PR1BQUG4e/eu2FGIiIhU7sqVKxg2bBgsLCzg5+cHY2NjCIKAAQMGwMjICCdPnoREIsHkyZMxYMAAXLlyRfnl38OHD7Fz507s2bMHBgYGMDIyKnL+uXPnIigoCD/99BPq1auHCxcuYNy4cahevTrc3Nwq+tctFyw7iYiIiIhIY+zZswcdO3bkdB5ERKTRjh49iqpVqxZ6btKkSfj+++9haGgIY2Nj5Vydf/31F27duoXIyEg4OTkBAHbu3AmpVIoTJ06gR48eAF7N7R8YGAhbW9s3vmZGRgZ+/PFH/PnnEHzM9wAAELRJREFUn+jYsSMAoE6dOrh8+TJ++uknlp1EREREREQVbdOmTfjyyy/FjkFERFQmnTp1woYNGwo997ZFhEJDQ2Fvb68sOgGgbt26sLe3x71795Rlp6Oj41uLTgC4d+8esrOz0bt370JTweTl5RU6t6Zj2UlERERERBohNDQUUVFR+Pjjj8WOQkREVCZVqlSBVCot1r6CILx1nup/P29iYvLO8ygUCgDAwYMHUatWrULb9PX1i5VFE7DsJCIiIiIijeDv74/Ro0dXqgsyIiKi92nYsCGePHmC6Oho5QjMqKgoxMXFoWHDhiU6j6GhIWJiYtCtW7dySis+lp1ERERERKT2cnNzsW3bNpw9e1bsKERERGWWk5ODZ8+eFXpOV1cX1tbWRfbt0aMHmjZtiuHDh2P16tUQBAFTpkyBq6triUpLU1NT+Pr6wtfXF4IgoFOnTkhPT8fFixeho6OD8ePHl/n3UgcsO4mIiIiISO0dOnQI9evXh1wuFzsKERFRmR0/fhw1atQo9JyDgwMeP35cZF+JRILff/8dU6dORZcuXQC8KkDXrFnz1tvb32bRokWwtbXFihUr8Pnnn6NatWpo1qxZpZoPWyIIgiB2CCIiIiIiondxc3PDkCFDMGrUKLGjEBERkRpj2UlERERERGrt8ePHaNKkCR4/fowqVaqIHYeIiIjUmI7YAYiIiIiIiN4lICAAQ4YMYdFJRERE78WRnUREREREpLYUCgWkUil2796NFi1aiB2HiIiI1BxHdhIREWmY+fPno1GjRmLHICKqEKdOnYKpqSmaN28udhQiIiLSACw7/6+9+4/Vuqz/B/68ETkczoFNzrAfgMQRISg4SSAWzjlxobDmPFGK0YaDTQJmbZoZmzSiWBlqLsBsUpow1MCs4a9Vp0z/MGQHiMLDDx2K6CjAgiO/jp3780f7su8JEPCc0+HcPB5/8b7u68frvv86e3Jd7wsA2smuXbvyta99LRdeeGHKysrSt2/fXHPNNXn66adbNe9tt92W559/vo2qBDizLV26NNOnTz/t22YBgLOTY+wA0A62b9+esWPHpmfPnvnOd76TmpqaNDc35/e//33uuuuuvPHGG8eMOXLkSLp169YB1QKcmfbu3Zvq6uq89tpr6d27d0eXAwB0AnZ2AkA7mDlzZorFYtauXZsvfelLGTJkSIYOHZrZs2dnw4YNSZJCoZDFixentrY2FRUVmTNnTv79739n2rRpGThwYMrLy3PRRRflrrvuSnNz89G5//sYe3Nzc+bPn5/+/funrKwsw4cPz69//eujn3/mM5/Jrbfe2qK+ffv2pby8PL/61a+SJMuWLcvo0aPTs2fPnH/++fniF7+YnTt3tudPBHBSy5cvzzXXXCPoBABOmbATANrY3r178+yzz2b27NmprKw85vPzzjvv6L/nzZuXCRMmZOPGjZk1a1aam5vTt2/fPP7443nllVfyve99LwsWLMjPf/7zE65333335Yc//GF+8IMfZOPGjbnuuutSW1ub9evXJ0mmTJmSRx99tEVgumrVqpSXl2fixIlJ/rOrdN68edmwYUNWr16d3bt3Z/LkyW31kwCctmKxmAcffDDTp0/v6FIAgE7EMXYAaGNr1qzJmDFj8sQTT+S66647Yb9CoZDZs2fnxz/+8fvOd8cdd2Tt2rX53e9+l+Q/OztXrlyZv/71r0mSvn375uabb87cuXOPjrniiivSr1+/LFu2LHv27MlHPvKRPPPMMxk3blyS5KqrrsqFF16YBx544LhrNjQ0ZOjQodmxY0f69et3Wt8foC38v53x27ZtS5cu9mgAAKfGXw0A0MZO5/8RR40adUzbT37yk4waNSp9+vRJZWVl7r333uO+4zP5z3H0t956K2PHjm3Rftlll2XTpk1JkqqqqowfPz7Lly9Pkrz99tv5wx/+kClTphztX19fn2uvvTYDBgxIz549j9Z1onUB2tvSpUtz0003CToBgNPiLwcAaGMXXXRRCoVCXnnllZP2raioaPH82GOP5etf/3qmTp2a5557LuvXr8/MmTNz5MiR953neLcU//9tU6ZMyapVq3Lo0KGsWLEi/fv3z2WXXZYkeffddzN+/Pj06NEjjzzySF5++eU8++yzSXLSdQHaw4EDB/LYY49l6tSpHV0KANDJCDsBoI317t0748ePz6JFi9LY2HjM5//85z9POPbFF1/MmDFjMnv27IwcOTKDBg3Kq6++esL+vXr1ykc/+tG8+OKLx8wzbNiwo8/XXnttkmT16tVZvnx5vvzlLx8NQxsaGrJ79+4sWLAgl19+eT7+8Y/n73//+2l9Z4C2tHLlylx66aXp379/R5cCAHQywk4AaAdLlixJsVjMqFGj8stf/jKbN29OQ0ND7r///owYMeKE4wYPHpz6+vo888wz2bp1a+bPn5/nn3/+fdf6xje+kYULF2bFihXZsmVL5s6dmxdeeKHFDezdu3dPbW1tvvvd76a+vr7FEfYLLrggZWVlWbRoUV577bU89dRTufPOO1v/IwB8QEuXLs20adM6ugwAoBPq2tEFAEApGjhwYOrr67NgwYJ885vfzM6dO1NVVZWampoTXgqUJDfffHPWr1+fG2+8McViMV/4whdy66235mc/+9kJx9xyyy3Zv39/br/99uzatStDhgzJqlWr8qlPfapFv6985St56KGHMnLkyAwdOvRoe58+ffLwww9nzpw5Wbx4cUaMGJF77rknV199det/CIDTtGXLljQ0NOTzn/98R5cCAHRCbmMHAADOGHfccUfee++9LFy4sKNLAQA6IWEnAABwRnjvvffSv3//1NXVtdiBDgBwqryzEwAAOCM8/fTTqa6uFnQCAB+YsBMAADgjPPjggy4mAgBaxTF2AACgw7311lv5xCc+kR07dqSysrKjywEAOik7OwEAgA738MMPZ9KkSYJOAKBV7OwEAAA6VLFYzODBg/PII4/k0ksv7ehyAIBOzM5OAACgQ/3pT39KWVlZxowZ09GlAACdXNeOLgAAADg7HD58OHV1dWlqajrads4552TZsmWZNm1aCoVCB1YHAJQCYScAANCu3nzzzbz00kspKyvLuHHj0qNHj6OfHTx4MFu3bk1VVVVef/31DBgwoAMrBQA6O+/sBAAA2k19fX327NmTq6666qQ7N+vq6tKzZ8+MHj36f1QdAFBqhJ0AAEC7+Mtf/pLGxsZ89rOfPeUxa9asSdeuXTNy5Mh2rAwAKFUuKAIAANrcoUOHsnnz5tMKOpPkkksuyeuvv5533323nSoDAEqZsBMAAGhzdXV1mThx4gcaO2HChNTV1bVxRQDA2UDYCQAAtLmDBw+2uIjodJSVleXw4cPxxi0A4HQJOwEAgDa1bdu2DB48uFVz1NTU5G9/+1sbVQQAnC2EnQAAQJt68803M2DAgFbNccEFF2Tnzp1tVBEAcLYQdgIAAG3q8OHDKSsra9Uc5557bpqamtqoIgDgbCHsBAAA2tR5552Xd955p1Vz7Nu3L7169WqjigCAs4WwEwAAaFPDhw9PfX19q+b485//nIsvvriNKgIAzhbCTgAAoE2Vl5fn4MGDrZqjsbExPXv2bKOKAICzhbATAABoczU1NVm3bt0HGrtp06YMHTq0jSsCAM4Gwk4AAKDNDRo0KA0NDWlsbDytcQcOHEh9fX2GDRvWTpUBAKVM2AkAALSL66+/PitXrsy//vWvU+q/f//+PP7447nhhhvauTIAoFQVisVisaOLAAAASlNzc3OefPLJlJeXZ9y4cenWrdsxfZqamlJXV5f9+/entrY2XbrYkwEAfDDCTgAAoN01Njamrq4uTU1NOffcc9OtW7ccOXIkTU1N6dq1a6688koXEgEArSbsBAAA/qeKxeLR0LNQKHR0OQBACRF2AgAAAAAlwctwAAAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAAACgJAg7AQAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAAACgJAg7AQAAAICSIOwEAAAAAEqCsBMAAAAAKAnCTgAAoFU+9rGPZeHChf+Ttf74xz+mUChk9+7d/5P1AIDOpVAsFosdXQQAAHBm2rVrV77//e9n9erV2bFjR3r16pVBgwZl8uTJuemmm1JZWZl//OMfqaioSI8ePdq9niNHjmTv3r350Ic+lEKh0O7rAQCdS9eOLgAAADgzbd++PWPHjk2vXr0yf/78jBgxIs3NzdmyZUt+8YtfpKqqKjfeeGP69OnT6rWOHDmSbt26nbRft27d8uEPf7jV6wEApckxdgAA4Li++tWvpkuXLlm7dm1uuOGGDBs2LJ/85CdTW1ubJ598MpMnT05y7DH2QqGQlStXtpjreH0WL16c2traVFRUZM6cOUmSp556KkOGDEn37t1z+eWX59FHH02hUMj27duTHHuM/aGHHkplZWWLtRx1B4Czl7ATAAA4xt69e/Pcc89l1qxZqaioOG6f1h4jnzdvXiZMmJCNGzdm1qxZeeONN1JbW5uJEydmw4YNueWWW3L77be3ag0A4Owi7AQAAI6xdevWFIvFDBkypEV7v379UllZmcrKysyYMaNVa1x//fWZPn16qqurM3DgwNx///2prq7O3XffnSFDhmTSpEmtXgMAOLsIOwEAgFP2wgsvZP369bnkkkty6NChVs01atSoFs8NDQ0ZPXp0ix2jY8aMadUaAMDZxQVFAADAMQYNGpRCoZCGhoYW7QMHDkyS9715vVAopFgstmhramo6pt9/H48vFounfTS+S5cup7QWAHB2sLMTAAA4RlVVVT73uc9l0aJFaWxsPK2xffr0ydtvv330edeuXS2eT2To0KF5+eWXW7StWbPmpGsdOHAg+/btO9q2fv3606oXACgdwk4AAOC4lixZkubm5nz605/OihUrsmnTpmzZsiUrVqzIhg0bcs455xx33JVXXpnFixdn7dq1WbduXaZOnZru3bufdL0ZM2bk1VdfzW233ZbNmzfniSeeyAMPPJDkxJchjRkzJhUVFfnWt76Vbdu2ZdWqVVmyZMkH/9IAQKcm7AQAAI6ruro669aty9VXX50777wzF198cUaOHJl77rknM2fOzI9+9KPjjrv77rtTXV2dK664IpMmTcr06dNz/vnnn3S9AQMGZNWqVfnNb36Tmpqa3Hvvvfn2t7+dJCcMS3v37p3ly5fnt7/9bYYPH56f/vSnmT9//gf/0gBAp1Yo/vcLbgAAAM4Q9913X+bOnZt33nknXbrYqwEAvD8XFAEAAGeMxYsXZ/To0enTp09eeumlzJ8/P1OnThV0AgCnRNgJAACcMbZt25YFCxZkz5496devX2bMmJG5c+d2dFkAQCfhGDsAAAAAUBKcBQEAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCcJOAAAAAKAkCDsBAAAAgJIg7AQAAAAASoKwEwAAAAAoCf8HebVl/k0i9zQAAAAASUVORK5CYII=", "text/plain": [ "" ] @@ -1520,8 +1520,8 @@ " all_node_colors.append(dict(node_colors))\n", " elif child in frontier:\n", " incumbent = frontier[child]\n", - " if f(child) < f(incumbent):\n", - " del frontier[incumbent]\n", + " if f(child) < incumbent:\n", + " del frontier[child]\n", " frontier.append(child)\n", " node_colors[child.state] = \"orange\"\n", " iterations += 1\n", @@ -1623,7 +1623,7 @@ " elif limit >= 0:\n", " cutoff_occurred = True\n", " limit += 1\n", - " all_node_color.pop()\n", + " all_node_colors.pop()\n", " iterations -= 1\n", " node_colors[node.state] = \"gray\"\n", "\n", @@ -2162,6 +2162,8 @@ "outputs": [], "source": [ "# Heuristics for 8 Puzzle Problem\n", + "import math\n", + "\n", "def linear(node):\n", " return sum([1 if node.state[i] != goal[i] else 0 for i in range(8)])\n", "\n", @@ -2853,6 +2855,7 @@ " neighbor = argmax_random_tie(neighbors,\n", " key=lambda node: problem.value(node.state))\n", " if problem.value(neighbor.state) <= problem.value(current.state):\n", + " \"\"\"Note that it is based on negative path cost method\"\"\"\n", " current.state = neighbor.state\n", " iterations -= 1\n", " \n", @@ -3341,7 +3344,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeAAAAHwCAYAAAB+ArwOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsvW3ofW2b13Wse+//dd3JqFP4ImYaNUmxEHqazIhqyKIaCpUgSwos4oYsdMIeyBcW9CYIhMAI7hypIErC6IGoQAhMCPMBfWETIY4x04hmMYyS13Vfe/93L/Y+9z7Wsb7H0/mw1vr9fuuA//+31vlwnOd+Wp/1Pc6HNd1uNzrssMMOO+yww9a1b23dgcMOO+ywww77iHYA+LDDDjvssMM2sAPAhx122GGHHbaBHQA+7LDDDjvssA3sAPBhhx122GGHbWAHgA877LDDDjtsAzsAfNhhhx122GEb2AHgww5byaZp+rPTNP0DIu03T9P0hzr4vk3T9De0+jnssMPWswPAhx122GGHHbaBHQA+7LCd2DRNPzBN0++fpun/nqbpJ6dp+q0s71dP0/S/TNP0s9M0/blpmn73NE1fPPL+4KPYn5ym6S9P0/Qbp2n6kWmafnqapn9tmqa/8Kjz66dp+tFpmv6PaZr+32mafkfE/yP/Nk3Tb52m6c9M0/QXp2n6d6dpOq4fhx3WYMcP6LDDdmAPmP23RPQniegHiejXEtGPTdP0Dz2KXInoXyaiX0REf9cj/7cQEd1ut7/3UeZvvt1u33e73X7f4/yvJaJvP/z9TiL6D4nonyaiv52I/h4i+p3TNP0yzz+z30BEP0xEfxsR/Toi+ud6vPbDDvuoNh17QR922Do2TdOfpTvgLiz5CyL640T024nov7jdbr+Ylf83iOhX3G63fxb4+jEi+vtut9tveJzfiOiX3263P/04/xEi+u+J6Ptut9t1mqafT0Q/R0S/5na7/eFHmT9GRP/27Xb7r4L+/5Hb7fY/PM5/CxH947fb7dc2vCWHHfah7bx1Bw477IPZr7/dbn+gnEzT9JuJ6J8nol9CRD8wTdPPsrInIvqfH+V+BRH9Lror0J9H99/uH3Pa+n9ut9v1cfxXHn//PMv/K0T0fQn/P8WO/08i+gGn/cMOO8ywIwR92GH7sJ8iop+83W7fz/79/Nvt9qOP/P+AiP53uqvcX0BEv4OIpo7tR/z/EDv+xUT0Mx3bP+ywD2cHgA87bB/2vxLRz03T9K9P0/RXTdN0mqbpV03T9Hc88ksI+S9P0/QriehfEPX/PBH9Mqo3zz8R0b86TdNfPU3TDxHRbyOi3wfKHHbYYUE7AHzYYTuwR6j4HyOiv4WIfpKI/iIR/R4i+oWPIv8KEf0mIvpLdJ9MJeH3bxHRf/yYxfxPVHTB809E9F/TPSz9J4jovyOiH69o57DDDnvYMQnrsMMOc01O8jrssMPa7VDAhx122GGHHbaBHQA+7LDDDjvssA3sCEEfdthhhx122AZ2KODDDjvssMMO28CGbMQxTT/vRvT9I1wfNrPoMtCjXFu5FhvVRovfkVGv0RG1jP9I2ai/3uWiZbdq97A2+1m63f4/90c6aCes7yei74xxfRizT8Fy0Y854i/zlXkv/cv6rW1jTZ/fdPDRy+/FL1LVRtRvxF9PXxmfvV9rxudh9fbdUKkjBH3YB7URYBzdxqeOPnv6arW3sCNuz5vEjPW+iT1sT3YA+N3be7lwbAmLmotbz/6OhGVv37W+RgDkgNJh+7YDwG/W9qJe9mQjYL4lfNdUqZ+oX3uj+/1ebhbfws3nYSPtAPBh9D4uQHuAbw/w9ARhax96+MnY3ucX7N0+0mt9H3YA+E3anseFtlInvX29h/eu1T4ahHv62vNv9LC92PHpf3jb6kIR8dezb6OVb63tDbrSZP9qZtAWH9G6Z4rP6v2U8Lumr97+evftsD3YoYDfnH2Uu/63AN/aUO0ewsy11tLvTN0z9R+i6PVd2eJmlGi7oZjDRtkB4A9tW4R3e14Ee4M8c/HKvndvGbrIWl5P9oZojzdsW/Qrakdg863Y8Um9KdviIhOxPfoaceEbDRvPRvxcs5tgIOOvMRomHRGWjvqM+or07SP4OmyUHQB+d9Y7JNvD1x7hOxK8Pd7ftX6aWju1YM7COFO+9LUHiCO+PgI4DwhvaQeA34ytPTbl+dqjGt9yacqeNqDoYahfWSjXwrgniFvVcG9V3dNXDz+HbWnHGPCbsL0q1h5+1la9UV/R97x2HDQz5rwXa+nziJuZPX6/IvYWf4OHjbC3dgU4TLW1LiJrXhz3eGHMlKvpQ9ayfekRbqxVxxlF3FMN9wxJ9/BTfO1NnR+h6LXtAPDubU938B+1L5lymbZ7ttfTb82FmL/enjCOlusJ0LXCv2v6idgB4bXtAPCubS3IrOWjh581wTtq/XBtG2uZ1qfMjGVu0bFRr421lOzaqnoNCEfHgw8Ir2kHgHdre4HvXvrRw0dvtbvmjlkZy6rRqKH+R2f2couAzfIdKbM3EO9BmfcMjx/Www4A79L2EqLdAzR7+OipdkevG65tq6ePDLTla8wCuReMPYhabfUC8V6U+TE7+q3YAeDd2VozJN8CfNfow5ZLl2r8r2EtS5CyS496qbJeSvStKNke65db+nBYD9vTr/6wXSjfPYSsR/dhi5nTWb97sxooZ9RxRqlavlpV5Bph6V6KemT9iI/DWu2tXg3eob0FcO25/lrQXWNiVrad3lY72YooHlK22ukBYy/fa2MPID4g/N7tAPAubGv4fmTw9oTuXidlZc3qV3RdLrcIRDXfPcZwW0AYBfkoEO5F0R8QHmEHgDe30fDdqu6Wbe8hjJ7117O9GqsZ4+UWDS9bbfVSrVr9VphaMIvWHdF2qb9lSPuwGjsAvKntWTlupZhHqd29LLeq9TvavD5kx3uJYiFmzbenjnvBeBSIvfojwtIHhN+a7eGX/0Ftr/DdAtqj1O4eZllnfPVqK2s1a3m5tYSYpe8aBdkyntsC0x6h7RqYjgK4126k/mEZOwC8ibVeSLcIOe+tzb2GzjN+Wnz3tJqwMree63pbx3xrodcKxJa6veuNrBupf1jUDgCvbnsM374H8G451h3xkfUXsZqfb8smG9yi4WWtzR4TsGpVsVVvlNKuvTHYQkV7dSP1D4vYAeBV7S3Bd+0w9Qi1u0WoPeOn1m+LZduIhpeLeSDV/GbGfL36Wt0R4em9g3gUwEv9A8ItdgB4NdvbmG1NvTXbWrPvLe1F6md99WgnYiPGfqOTsTJART5qFO5IVbwmUNcMZ7eOKR9m2QHgVWxP8N0DeHuHmNe8YfDqRn3U+OxtmTajypbIhyny10PhWvV61BnhzwPxHuAdqau1eZhlB4CH24gL/VpAXAt6W/e5pZ5XN+OnxXcPy+xixc2DKfLdEnK26mogrAF4RBVnAFmjpPcC71L3CEn3tAPAQ61F1dTUXQOkWyverW8WInWjPrL+RlvNUiOiPrObPR814KiFWqaNEYDspWxrQ9KW1dY7DNkefvXv1EaEndcA6Rrg3ePrs+p49by6UR+1fntaZnZzMUudIr+eQo4qXKteiyquCSe3KOLsuPIa4NbqWPWs/h2G7ADwEBsxE3h0nTVAvdVNwl5memf9jPDhXRw9f7Vjwd44cA1YZb01YayV7xlm3hLcVp1IPdS/w6QdAO5uIy7qI4EzGmZ7gu6an02kbkvZWmsFdstYcGYcWKvbc/x3NBBHg3VLcJd6x5hwix0A7mq9L/B7gtRI8O7pRqK2jlUvmp9tb4RFZjBz09QfKtMr9Nxbufb2vQWIe4wN9wxje20dRnQAuKPtEb5rg20vr2Xr8eJIvue/1bdnNWO9xbJjvlb4uWa2c0Qd1wIzC1cExWx4ugbEa0O7to5V77ADwM229XjvHsqOAv3eAe3lWT6zfnpbSyi6x5hvFq6yjgfNPZTl5SOqOAPiHmDtPZas1bHqfWw7ADzUat7eNYHaA1p77Ve0bG/oRj7zDPzW/Ilm1gG37m4V3W2qx7hujXptVZ2ZceJI/UzZTL80v5bVwvSAsLQDwE22Rth5axi1Kt61+jRyvHjUmPDeQtCZ8LPVh8iYr6WQR43r1ijdNULZa6rhHmW18qXOEY6O2gHgKusddt5a9baAd8v+rKnqNR9Wea9exkfUvFBgbXsWVItF4JqB8jdKGvJjQbYGxtlytWVbQDxCpfeazEVKHavex7MQgKdp+oeJ6N8johMR/Z7b7fbvDO3Vrm1v8N0TeHsDei049wpNt0zaivjIWA8/kclWRO2KNwLTUj4C5B4wbinHy44E8dbA1vx6dbx6H8fcX+k0TSci+veJ6B8kop8moj8yTdN/c7vd/rfRndufvYWQc225NcA7ur9bh9Ct8l49aVOw3Ai7Pf56ffXC0d8oPrw1vRl1uyZkewF2KxCvGb7mdQ4Iaxa5GvxqIvrTt9vtzxARTdP0nxPRryOiDwbg2hDhiIv9aODtBby9yrT0wSprlbfqEG0LWM+8vnmA7gFYrfxoGFvA2grsGjhbVG7vkPQB4RqLAPgHiein2PlPE9HfKQtN0/QdIvrO/ewXdujaW7IeYee9qMi1Q8Mj4dzSh0wbVvliewZu1tBrubHjFsBq5WthHFGFPRVmDdiivmrrjYAwsgPCWYsA2Pu13RNut+8S0XeJiKbpBxb5b9tqLrp7CiX3Ur2jYVnTXgS6a4xXF3tPoM2YdZkYpXgj5SL1sr6zk7ZqoV6rhrdSzFpZr45X7/1aBMA/TUQ/xM7/OiL6mTHd2ZvVjueNuLiPAtZIEI4KM4+GbrTcR4Vt1DQoZ4CMynrlamCcAeiIEHYWqDVquFUxo35Kf1pZy7fVl/dtEQD/ESL65dM0/fVE9H8R0T9JRL9paK92YbWhxjXguxa01oRzzWscOWa8E+BGfqE9zVu91GzyPURALh2pBW0NjCNw7FWGl/P8WCDupYazcG1VwweEi7k/79vtdpmm6V8iov+R7suQfu/tdvtTw3u2qe0FvnsG16gwc1axj1Trg4EbhetaEL4E2uoO6IhKbgFtBIge/DKqOBtS9mAYUa1bqOEDwj1sut36D9fex4C/093vOjYaviNDziNUb482e/e7VwRAlhkAXA9oe4BwBqpW2SHqGV2fZEPyYo064pWR+dnyNf0a4WPEexH1o5XTylrlrTpvwb5Lt9vPuBeUtQNc79S8sWLL9qIca3z2UMW9FW9NnzqDV/tV9YJxbT3tWmfVl3W04Vsvr9r4ZyMndWXDwBl1GlXFnprtoYgz+V75Gp9RP1o5yz62Ej4APLMa9avVGTVJqQbOa6vetcPM2fyOwEUf4R4BHAkv9zStvSYoSxhL0Fpjxt5s514gtvqU9VGbXxPGHg1hC6YfF8IHgJ/WC749w857BtlotbtD6PaA7QgIR+pkhckIQ1Cu7pP8PL2xXg+MXn62PO9TC2hHquEeCh35yZSzykfqvW07ALyLMd/RIec1wdvzJmFj6EaBu6UK3vIXbKnc2n51AbLVeAbGvVVxTf5oNRyBZ01Y2ypHoKxW3vL/9u2DA3iP8PX89FS9ewFvz340QFe62juAa2C9lQrW1K/3Gqr6q4WqM8q3RRX3BHEvtWxBtMdYNCqjldPKWuWtOm/XPjCA14bv2iHnUXBtAfqodgZCtyeUrXQvr6UsqtcDxFsAPd2epow95ZsBNVEM3DUg7gnp3iFu9GU6IJyxDwrgtw7fXmHcEXle/zaGbitwR6jiaH60TMSi8FwTslIRR85TFoWx7FQG1DVQj6jabJ70q6nhVtUu+xItg/rC7WNA+IMCuMZaws5emZqwdKTs1nkj2hgA3hrI7kUJR8uja1k0/IvKaXnRUHMNbLXz6jC1taypVnFaoNOUopdX276EtBWSbrXIndoWIZN92wcEcI36XXPMtzak20NZ9gDmiDBzB+j2Vr61QLbSvbyacqh89Bq41+ulNXac6m/NWHGtIq4db65Rw9Gx3p5jzLVltHKoT7IO79/btA8G4PcC3x6gXBO8NX0iqgJvFLpbAbgVvj1+sU3KkfnYI5ylhfsYCU97YWMrLwJtD7ZRNZyBZ2bctzYcfcyO1uwDAViDbya8q5VvhW8GzCMB2wrejdRuDXRbABsBa2/4jvilRkBslUHXxWjaGlYdpi7fQUsVy4YyalaDag3oe8A7q3Zb81EZq6xV3qqzf/sgAI6My0bqRODrgVeWqR1T7QnYKBwj4K250egI3R4AXlv9tqriVouCOArhNS2ylAnVCVmNKs4o3YiC9VRvzbh1NCR9QHi0fQAArxl2zqheL78Gvj0BOwq8ndRuDWjXAnArkL08yyQnaupq9aKwjZSTZXqAfBUYt6pimY7UJy+XAXR0bNhTwzWTvGryURmrrFXeqrNfe+cA7gXflnJae3uCb2u4uaYfSfi2AHbPELbSvTzPamGk1es1w1mmyXYzeZk6XnrINBAT5dWpB+IsVDPpVl60nMyryUdlPo69YwD3hK8HT69MLXx7hpZHq94B4B0N3VoA18B2tAK2ymdmPVvl0XUykuZdX0defzVVbKWHLKOItdB0RIG2wFZri4w6e4Lw+1fB7xTAa4adZblM2NnKy4Z6W9JHg7ez2l0bwHtRw5kyXvm1J15lICvV6lbiKNSup4iRQ5SuqWFNMUfSS1u1Y8l7gTAFy3p19mfvEMAfBb490rU+bADeHiB9CzDW0qx0L6/GuL/smG8Uwl77vceAW80LaZuW3WWLj/FqMK0Bce9QdVQxy7yafFTGK0tK+bdh7wzAW8LXy4sq2pGQzaher552s9FB7W4N4FEwzqRF8nqZBWPtGofSPajWQjdzI5A1b+JWGsREfnhaLlWSaR6ILYBG4ZxJ19oq6aTkyXqRfFSm1vYfjn5HAF4Tvl6ZteHbo6wHWa+NSvD2BOwIAG+hhL28SL5lNeFnK70XdHneSOj2sKbwNAJqRO16ijkD55p0DcKZvEg+KqOV08p6dfZh7wTAveBbW64Vvi3jtBmlisrW1OFpg8A7AsCjlHAmz0qz0r28jFmKV5bpMd5rnfcCasYPUr1a+Nmr3wXEBNI0oHoKtxbO0fQDwj3tHQA4CtKIWfDMlEF50bfaK9dTDa8I317A3KNKrjnPpGXyM5YBTcZP5rz3caSs1q9W/65JEKOJUtbkKQ+EHkDXgDC37N1VFJTvB8JvHMAefLWXlx3P1cpYoemoKm4N/fL0DJCzdRrB2wu2o6EdSbeOa861tEheiyH4oPyI4uXl0HlEAWePkWWv+aWf2RuRZhDXqmEUkiaR70GYgmWt9EiZTB4ZZShQzvLp1dnO3jCAR8PXG/ftCd8MZHl6TVqLn5XBuyWUa457nHvp2TLIasLOMl1Law01ZwBaA9uoRRWzlu/2ywpLR8PL2gSt7Ljw3iGcKaeV9epsY28UwDVjvlq9teAr69TAtwXILeAlCsG3Fwh7ALgVujXAzeRl0iJ5GeN+0HUqC+IaCEfgXANwrX6t9VDHpiEQF7NmS2tqOApuEn7eAoSRvW0Iv0EA1475rg1fq04LUL36KM3y7fkYDN63BGWtnlU+cq6lWenZcp7qReU0UGegK89bgKz5j9TpabXq2DQ0PmzBEkE1qoZbVbNMj0KYWwTC0rQybxfCbwzAbyXsHOnL2vC1yjeEm0cAckSel5bJjx57eZm0SJ5lqF6L+o2cR+bmeODN5O/B0NhwWhFbajgzNoxg2JJGRnpWRcu8SD4q49m+IfyGANwTvi3lvPZknjb+itK8q3EWpgPhuxZUe/pqhW0rgCPnWlomf5TVhGO9ctqxVdZLs/ojy1u+vDxKlJdpphUI8waisCQjzyrfA8KeP5mO6tdCuBam20L4jQC4N3wjL1uWsUAZGfcdqXIj8I2EoRvBuzZIe/qO1o0eZ/KsNCu91bjf6HgvSouo4Oyx5l+DWDRNWgiIScvcDKimhaSlAwnLrBL2Qs8ZCEdDzt6b3hvCXnvbQfgNALgWvtHykfD03uBr+YuAtlH17gWuvfx5aZn8bB4619Iy+Z5pwJV5HnS9857gRTYCoBmzVH9EgZc01crvkhfWIEqgjDVmHAWuBmFkUdBmfEbLWOX2CeGdAzgbFvbqRl5uBNAobwR8o3Vrxnt3Dt6asiP65aVZx5k8K81KrzUNuDzPAnHkvOa4B3gjUO4N7giIPSCbxseGJUQJpCEIk1K/FsKobSIdwla4mStxmUesTGTiVguEeR/G244BHIGv1v0ofGU5D74aZDPwRf6yoEXteaCtUL29gZnxs1adXmnWcSYvku7leWapX56PID0KvLI/Wlo2T1oNeC2AamVr/blqGIWkEXRlGQqUpYo0Av60fJku86RtAWGrbn/bKYBbws57h2+ryl0ZvqNhl60zCtAt5aPHNedeeo15F30JVZS2NniRZYBbA17NeoFYlg31s2dIOqqkyUkj4E/Ll+nSMoDOlMmWXwfCOwNwi+rV6kfg6+VrkNXqZMLOkXIWUKPwrQQvShsJ3DXbyJbJ5FvHkXMtLZLnWasCjo73lnoehFFZrdxIpTvCMuoZ1VXNCklr0PQgTIo/AmkWTLMQlh+WB2HUJvrALYjuA8I7AnDLeG9r/dq3wavn5SNVK+taZTrDVzY9AqQ9fY3oQzav5jhyrqVF8jxbC041KtDKa1WV8oYik59pw/pLlXWgRSAsLQJhWRYZasMDs9YX5K8Gwl4bNTYWwjsBcBSe2e5a0NLK9Aw9W/VqJkzxtMHK9z2Atkefo+VrjiPnXrpWNqIQi0VD0Fb4uZxr5RCAon0OAylRp8YntxpFK+sOhzBqmMiHMAIpArWVhtqR+dJa7wy1+i3jwaU+KT7abGMAZ1Sr1dU9j/v2hm8mRN0AXw1AHqDWArBsZ0S/onmZ/Oi5lmalZ8pIuKI8C7TauQwpy/RMmgXqLLCy13atfAt4W03tvxwXLqaNC0fC0b0gTMKHzLeUbkQFk1MGtc8t88Xor4Y3+iplw8VZ+EbK9YCvVj8CX1Q3Cl8N3MnJViMBOervKN+1Zbw06zhy7qVH8jUgaWVqwRs5roVwFqBZq/XvhbJluWj9ISHp7JhwDwi/l0lZvD9EvUC8MoBrxmmz8EXlewIf+Y1OsEJpmbCzln6mZb0OIWcJji1A27PtDGwjZb206HHk3Eu3TKujqWAJ2pJmgdg6zkIY9b8GUKOhbZkGZA/UUd+qbQVhEmW0tD1PyiJQR7M+IB4M4NaJVVn41vq1ABoJPVv5EqwozTsvx5qvBvjuBY577lc2L3ocOdfSMvnFUDgX5aNrkQbYkteieknJqwWoBuJWv6PMg7QFc2gjISwtAmqeJo+RLwJteuHfHhDW6lgmOZAD8iAAT7QNfFGdbOhZy9Pga4WUkWmwtSzyeivgq6X1BmNPnyP91/Tfq28d15xLG/QLftMWBRqq4ylXq73av8VHsWhd07IQLhaFtQdc+WI063WXlbH+Y7lz30TRFSc7/fnWdCsCXy9fg6xWp2XSleZX1omo5Ar47gGMvdsZcXNg/fXSao6zaRlD9a0wtDwvZSIh6IwKjlxLtbKRa36Lf2Qt4eOMr2j4ulkJE81DqhJQGWWMfGvtbRWK1sp6dfrbDgHsdaln6LlHHs+Pwjcaeu4E3zUBNwrAW/QL/Y3mRfK9PCKi8w0kEtG54QJxYQ3JNi/TPN2DbclDcOXlrbQatedB1mJDr+trBsQ1ihqdpxVxBMLFMZEPxgjMSdTR0nqEokdCmJR6/WxnAK6FL6q3VugZXjWNNnrANzHmOwJqa8B1jb7JOpqPiB8vzTwWkEVwPV+XacK+Bcp8vpx8X7zM87pzFucAzNZxjQKu/Tt7bSLNglMvEGcsEv6uzauGMNH8AySljGaWX288WPqw/G9hY9veCYAj3RgJXw2yHnyRzwiQe8C3WAf4toJ0DwDuDeXM31Qag60ErQAjAioR0SkA40jZawEvK/NZpj3PHwU4mDmUowrYu8ZqZTN1a8ug8q2WUboteU0QJsIh54zqtSBMogy3SCg6k4fytba1stLGQXgHAB4N30z7Gnw1/xKQqFxP+Mp2g/DdEn6j2xl5Y5D5q6Y9gGvAVoJWQvNkhJvPCRhfhBrmfq8PsPK2r5fTEs4czBzKz+NJB7KnYiPXbA/UqIxn2fKt1kMFe2nQNAgTza9hPK8m9FzMKpMdD5a2BYRJqV9vGwN4jeZlGxHISkOARXUtIGfMg3TSDT8OgwP8HQ3AaPk9gRjmAZWrADcCWw2yGSVcyl9RSPrRhgS0ZZ/VnPImBPcd51YzploLM698tK7mIwNLqz8E/KK0aggTqCDB6YWjM+PBUcAiy94hZfxHy/a9S9sIwC3gs3xkZz1reWclPTOhCqV551Zb/DwZdt4qrbff3tCWeZGyBMoT0ULpBoDLYSshi+AaAe7pxBTsdQlU6YMDueSVtNKnAubS3+vlPC97vs5D10gdc2Xc+peSedz6Xj/zNwCWj5obgGYIE+WWJ5EoQ4Qb2kMoGplVZn0IrwzgXs1F4auBNNqX6Lgvslr4oqu7Nb4Mio+A2kgAr1Un0ldZRq2jq1wEXA22SwUszk8AxMEf/+mkl7s+Xoj0f72eFmFoBGYO5TCQa2BcLAtYVB6Vi+T3sCiUs+DVfIchTGQr2HLtsZYUIR+eX62tLUPRXlvSBxl+YrYCgGubyIz7trabCUuj/ChstfKonuZDUb9bQK4XBFvr9L45kHnPczCeK6BrAfekKWEBQQTYM+nq92TkcbvSS+1Kf5dHXoG2BDQHs4QyV8omkLMwJnGMzrW0GusNXc9aFK92nIawBCE/L06KRceDs4DTIKyVyX5QoyBc/BTLf3kGAbh8sLWWHUNda9Yzyvdgq/XDm3SF8gLw3QJma4K+Z1krbfaXgTegcjPA5bCVUERgjcIWGapboMzzrnR69gWBWUL5+lTD1yeQkUJOwZgoBmYL1jItWkeztQAdBWumbBWECeRlIRxRxppF3/CaULRXLjs+XfwVe7M7YfUe97V8aHVbQs+yjAVoq0wQvsXO4HgkfEcBeA0wh9LwmO63zlc4jiuhi4CrwfakHMs63CxF7NmFKWHu/6l6H76vdIJg5lDWgCwVslTHJowpqIo1kLaJA0fHAAAgAElEQVSCtqdlYFrjK9OG+folMIl0mGrhZ8+ndYeUDUVLXzWhaM9q6uRsZwDOwjdSLgpoD+4Skigte26V4cfOpKsWQPWA4xoAXg3CNwhdIl/pPv8awNVgK0EbUcORvGLXGXSvMI+3eaHTs08czBzKUil7QEYwfr1gPm5MvirmlgFtpGwtrDMw9er3OA5DGC1PIsLwtCCIymiWCUVHIexZNhSt1elnOwJwVsmiOt7L4fmR0HNPy9wYBPuAWB05PgfSM6CLtLMmfKsArIeZa8GLVK4GXQ+4GmTDE7Lo8gRp1rjqTVlFleXypvMLwpplFGWkbCS9Bn7SF7GypNTLpmvAPYO0mVnLkyxYamUy48O1dzzSRoSix9oOAByBDepmJgys+cjWbVG/Eb9nUD857rvnYy1fKxNJ7wFeI8xcC10LuC3h55bQM6rLlS4RDkOX84XiZQqZq2OuqKUyluPGcv3xIjx9dzRXxM800o+1a+p219q7RVRy9MZAgz0p6e5rRzOjuWmw1T4MzSJgl8fcb0YFy3wvXI5MzgTvZxsDuCd8Zbmowjwr6Vb7WdjKfG+C1w7g2wK/iJ+sz57QJgqBtxW6NUpYKy+tZTKWFo7mgCXCkC3lZBi6lEXlZmU4jM8nPzwtx4kpCGJue4DxCLXd6geanJTFzdtIIzohi4w6yK9n2bo1EK7pl28bArgWvjWmQVZLtyZeRfqEYIpmPVsADy43KseR9Bq4tQKxZ3vNNxXLiVUozIzAWwvdTOjZVsPWGLBPER5+1saAkeot51L5onHhxSQtBmRY5kQLVTyfVT2ftPWZKA/iKHwjdbLWAkt5Hj0mevU/mg4NvQlInfaYFR1tMwraSCh6HxDeAMAtk6K0+rJs79CzVU7CE8FUmgXoM1U911ee7xGAI9t3fcwVLwcvDzNbarcWupnQs6WMpY+MaeFn2Y5UsLxPEqLFBwJtKY/UsReiJqIZiJ/vRwuI0flezFK4WQjzc/4+aOnueDCRvj5YmxXtWUYZy+PoHVXteHDE+oWkVwRwZnJTL/ha9c9KupWvwTUKY01lB9V1BkI1dboCb4N2O4EXqd1W6CLI1k7I0tKipq395W2jsWEEWARkTR3z13p9vKuWKibS1hYnQSzPe+VlrQa0UR8SttE6LoSJsGpF8JRp2rllvIwH9lY1WquCeX1q6sMKAB41qzhikZfnhZ5b/Uvfko7SnNCz5wJBCNXx8nrDN3pzMAK+gTFerngRPCPgtZYd1Yai0bmsmzU0I1pCssWqZ03POxS258zpy/n+mdOUA1wmj5Sy2THbSF5pC7Xr5XHTbiJCnEGgrZkVnfGPrGZCVosKzpStB/EgAE9UD16tS6PUr9eHXupXS0uGnrsCarAfLU+mZyAeamOperPgRYo2onYtpZsLRc9//FroOaOEta0oZShaC0NboedSVhvv5dBfhKBpOaZMRG5omk/WMtXwvbHtQtMtynct1eyGoi2CW/CUZaw6GaXcyyJ9z/gq9iZ3wsrAN+NLqx+deOX518yb9VwRepbFI3k94VsL4F4+zfJ2uDkL3qwSxvC1Q9W8DC8n/WjnyCQ8tbqRbSj9WdD2WDAKQZc2EahL/dPjfy00vXjNPCz9tDPNdtUqSdY1tTW/xqIg9fJbVLMLYSKsgnnFaOjZy0dlZPuoXZku81A+KoP6McZ2BOBsV2T51sldVn5Ji6pdBH+LkkRm6NmDGT+38r2yI+Hcs10EXiJC4eZW8HpAfuXlgdsyAzoSgtbKyLW+9zQ84aocZ2dBIxifHm1cgfItNp9FvZyw9QTxbAkTGB+Wa4it39caYqtF6Xq+SJSvCam7EEZ3MZFZ0dxqlwh54W9pNXdR/ZcYRWwnALa6URN6RgC0/GbVr+VPwlhLSyw5KscRqGn1miGXrD+67WcaHueVE6ws8GqhZCvEXBuiLobBLMHbNwyNtpwk0idclTo1s6C18DPvM1K+C+CSA2KmiOWmHrPNPOj8uO4qYWl0rqVploFo1hfR3J81Hu3V18qGxoOJYhOyivValoR8e/myD55Zk7II9K/dNgaw13wEvpl8lOeBUqZp58i8mc5GXQQgeR7Ji5RvAmCwfPbcLTNXvWicV85sjipeD7wxMGMlPC/TZ0KWNG3bSS8Ebc2EjsyCjsC4+JKhaVnfAzF7Ufc/YtmSOT4sw9K1lvWRBbQFVa080bwNeZ7Jm1mPULRn0fJbLUvq8aVZetzIauCbLVe77Eim1U7E0vx+YunOgxb4scZuD1qyfgR0Xp3ewI208TzXVa8MN8t1vBK82vguTvfC0Hb4uS0MHVPDkfzM+l8ESF4+MvEKjwHrYWye/0qfg7gEtYlek7WevhcwFuPDZ7LD0rNyRn5Plav5Q2WIpXkhaFkmOjasvm5U0ApFo7oZQCOwa5aFLMr32ugL4Y0AXNusVy/rtzX0rMFV+glOtkJuZRULylr5SJ0o0GtvAqw2IjcBQfhq63mz8EXw9MLMHnhrn46EYBoNO9eYNit5TZvDXj4W8Yxf/6Ob/HGI9/Pz8/wz0UMJEz2XLOmdeJQTabWws8rLOrxeNAQd8eOVN/kSXRvMLQssBOVe0GsNRRfrB+ENABxpcm31i0zrp6d+NR/lOKF+PXjJ8xTIOqVly6DyIT/+0iI01hsBbyTU7M2E9qBrh6Lbws/ZMWAUipYTsrQtJXn5SOjZGuvlylZCf6ZwFzcERlg6MDYMQ9IjLRtKtupRwFfNTQHKD6ngYp4Kjo4FRy0yIzpitTDtA+EVARxtSgOkVz/7UryxX1lO+kftWepXHgdmPctmMqDV6si03vDtCeBn+kv1oqVF3lhvNtwcAW/LTOjarSl5Xcv4GK5V13vyEU+vmXSFQsvu2l8S4WQjT274MYOzMzY8X7JkQLjPdbbeasLSkXoZhTyzlh2ySDkvZkG5FtSyrV4quPguVvclGQjgGtcZ+K459ivry3zUhkVRxTSQ1oA2CtRM2R6QRnXUdDzRCs1wRuFmK6yM07wwtB2evr+EnBKOhKB5PVRWM60MmmzF28zOgi51IjDWxno11ftKfeWV1zY/B3A+nQhNRuOWHheutZrxXs8fAZ88rSacbfl4mnxiUmSHLJRvzYCOWFQFexD2bggiVgfjQQAeHNIxlx3V1NfSMv49nwH1W5qyeO2BtzYtW3Y4gHHI2VK9NeHmjOL11G4Wur1C0NIHMu1pSFr4mcieBS3Vsh5iluHs+bpgNL5sjTujELTmh1WyTS5XImU/6VaLKtdImubTSuM+PNUcet2S2BEV7EE3q4JHh6Jr1gfHb+QGAbjGakPPlp+zkm75Lmk91W9gEpYFWguIWh2ZVgNTKz0K10iZRbo/0Uob60XQ1MB7LzduXLjk4b+ZMLSthiOG1v5K39o4b6kf234y8PQjwuO9HOIlzxrvlSFo/n4t1h2fLnQ9nRc7aS2WKz08p0RERrlG61sg1tKjZa1ylmqeWc2yJGQRFVwDQWnSf8Znj/ax7QTALaHn0S8h4j86nkyUUr+RMhHAeumaHyu9pi+oD4s8f7zXCjlr478y7V5Xh29kGVIuBF03KYuXleU0sxRkpkym3J5MbixS0ojo+b15prPx4W+dr3MI8ycrzZ0t0yMK00tHv7GMHw/QJV1yTutzekKWWZDlZ9cFI/NC31Z6TXtjbAcAziz7qfXlLQmKbrJh5XsK23irLSXrgRGl1aTXADyqesMAnsNXm+WcmWiF1XE8PM3L4zwrBL1Uy1r+6y2OTcaS9TJ51uSrcm7Ngn5tKYl3u9JC0NHdrlCeG2Z+vm5cdqamhZsyS7pZCSOzwBqFs5UXha7lxxs7Tk3I0o5b1wVHx4szELX6FCnbbhsDODvumlG/0ZcWuQHITvhKjP1KMKE8EmVqoBhWoAPrqL5yk63ioJXquC48jfP8EHXJl3la/isvH362VLGEkTX5qvjiY8NoXNga643OgLZAbMFWhqpf9ZYhaVT2UeH+R8ySLlY9OSszuSoL1do8L6ws02RdVGdmHFrasSy7hQrOwNlrr49tCODWSU9RfxoMtTZb1a82Dpzc8QoBS9ZB6dE6EZCvAmB7shURQfh6IWcExvhELRu8LbtkafnF0NiwLIPOLZNlpeItadbkq1JGwjYz1iuBbO12VepoM6P5+yXTrPRikVnS8MlKtWaNE2cnWbXkZSZolTSTWR9JBWvl620jANeEnVvVb2ac1mpXq++1abgpxxrzZRkPrNE8lL8qmGOTrazxXh/G8XFepJBf+flxYe6Lp8n8UneZ50M3si64WO1TkFAo2oJxJPwsQ87c0GQsPaycG5+GYFZcLB9xGIRwBn4Ri4SmCZTxxnqtdJmm1Z1V2FoFa3VqVLDXJint5mxlAEeAVwGy1MznaJuofnSpkSRYctmRBUAvHeVFoRn1YeWHARyfbOWt442q3kxoWrYz94nD0K3h59YwNK+LwMTrZWZBIyBbm21EVPEcxDjPfo34gRPW63cczgzunvV4JautFUb53CwgZ/IiE7q6qWBkEsgRQKNO1QK9VtW2q+GVANwKwtpyVl0Lpp4U1ep7/g2XGkRRvQgwtTZb4ez50MrM0nOTrVonWnmqNzqhS/pd5uX2itbKvd7GiALOjQlr4efSXmT7SSsMLZcUWVD1FHEkLI3KeptvqBYdF26BcBaypbmsr1ZljMp3UcG1m29YUI6AsHY9r9fHNjW8AoB7znKutZ7jzRr9KnxHoFsL3IjKjfqrVc0wj435MrNmOiPTw9B2uHpe1wc6T5d+X3l2CFrL08rJPP28ZUJJf0MPSUBLguL+Tgt/KK2kF2t6X04E1woXCy1TIopPluLWA6bRNi21G0lbGFfB3LyxYE31RlQwsmi4uZcK5vUp7WMQgCfKg1frijf22xp+luWlgkWK9pORJo8DjzyzgIfSRingaFlPNas+4mO+0c010HjvvWm7nOZP1r2X08PgvHxJ575fZedgtSZlST/zj6VtFrQVhs5uxLFUxb4iRttOIkOzl9UZzcQVu+03ZE71hRKOqtpeV1sL7qgMKqvla8O1XtqikRFPSiqW3ewj65dbto/l2v+mdsKKwrfGrMlXvdWvdgyqZJWulh6Gn1E2Wj6jjDvCNzvZKluOaAnje5qukLlfXhaly3xZZl4Oh6A9NWwZUpE8PRuCtmZAyzI8TbbxyrOXIekznQv8O0BXWm8IE9WFn0sTPep4NwLWWHBEdatjwZ55qtcKW3uAtCAdhWvtjULMs2nTNP1eIvpHiegv3G63X7VBF4yyGfVrtZNRv6iNRvUbUboRhYvK9VC5NXUQfB/WCt/IZKusOp77yoMXLz3SJ2XN/+J8eYzOX+n2BQLtBY1AzI8zM6CLH00583yerm07yftaC1fu88o+9YQD06ogXGMRxZup40G7WQUjp5ElSahui0Kuscis6n4W+Tj/IyL63UT0n3RvPQTFvZk3+Uoeg2KekI6Wj0IzUqcFuCaA+yrfzGSrCFAz4WZP8ebC0GNnQkvzQs+8PQ5RBGQMY/3pR6V9bdcrNKPZW9/rqV5rlnTKaiBcTpF5Ys2zyExoq05ETWvADqtgTcmidAk2DXSZdcLyWCtjtYesP4Tdj+92u/3BaZp+addWXYso2Ij6jcx0js5mRm1Yfeqofr08y3cGmrV11b7WLTXKQjWqZjPlXnl99oq+/11CNxOG5mW8NGRS8Za00oamdksdpHw56JBa9UCMNtrQHj2Y2Y6yVjUbTlVbZYmSZq0TulCdyCzpkh6aEU2UB2ZmtrRXJgvO7PKneusWMJmm6TtE9J372V+zZtOd2vKArOUlNgixVG4mveRlFbBlrb4WdefKl6gOvkRostQLdMgs+EbKvfLqniOM0u9vkR6mRuV4WXks67xFk7OmtRnTaAMR2y+eKV1r6IlLRMbsaK6EpUUUaMZqQKvVzUzMiraxcKI5XPN73H9LyVrrRsHb7fZdIvouEdE0/RI0Hz3YbFb9aukRWHpmTdaSsjFoGtiiyne0AkZ9SfsbF3ZGCvQL+noB2Yyv+0usUcL60iSezuvI/NJ2MRyKrp8JzZVgzQxoro4zM6B5fmmHp5fcyAMXrNnP87rz8d7ulg1Hr2E1oNfqojpe+Bmys3ZjjuiSJHmOfMhjC/Iobx0VvKYMHdRkhEZamjf5Clnl5CsPrFqeZq2QRX3L3AB48H3YKPhaZYiQarZ93etg//e3wQY3r4/T4xOuEJhlWXSOTIOyFYaWY8MSyN4M6PIa0JIjOTbL4Y3SLJiiusOtF4S9ZUKt1ntGdVQdP20SBbTjHutvazb2iIwFexCmZJu6lxUs0lSL+q3xXVtfU7/Oj84CYUQZZ2BqtYF81QDdgi94sMIa8O01cetern7Lylc6VsPzvzElzMvO03wQc4h5k6/KsT3pSt8D+vW4Qn+8dw5vrHTnY8NtKvdEF/oefankXenrSJ0WCNdM0tLUp+VHs5pJYJHJWSaHIhOiNJVLwXzNb1QF11obiN2Pb5qm/4yIfoSIftE0TT9NRP/m7Xb78Y5NPCy7dEjme+FnWSe69EhrgwLpzIXkdET9bgrT7L91ws6jyhDh2dF+er8JWfwvLyPTrRnQUtlqdS60VMEyHD0H8kshRx9FWPrjhZp5efR66kA7B/r36AuW+/UT6BK4X9L36EpX+noB6a8fNwZflAZMqwpHa8uYapYUaVargLV8l2mygLWJRlaFWm3VbE/Z0n4diF063m63fyrlkYhe4YfeFlGx1sYbWUP1NZ+BH1pW/fK8zD+truWzti0B3zVmO39JX0M4fknfmwEuD+j8dpWyLG+b55X0+9tsK2ENtks1bCtflM/ByNPmIegCV+1xgvFHEfogRuuI50pXMw59r94Xj9cpYa4B90v6GkCY6CTqWyDedHa0Zi0KGJUzVbDcnlKqWM20GdHR8la+d8fQ4yaA6I3thNVT/fawqL/A26cpXK1sVAF7baK6XtudbLTyfbYDYHhP7w9fpHrRTcD9rdSVcEmTeTwdjQ/L89rZz3IM1i+PFShKr1WrPe0F/ItIf91IyDRefq7MUdpyIhuy8oCRxd7RlsJFYOPpGQVqpWsWHTeO9m1m0XW5suHaMHTUonAdM3N6BwCOwi4LWY1A3Jd27rUvj523EYEvCtqmsPAa/5ZPNloj7PwFfY+IXrD9gr4HIYpnSLdv3CHT7x8VBnU5n/+9mMCVqvj5dbgKKF/iy22u55dKLXY5LUPP93bLOC4f650rYx5OLmVK36Uibnm27zy8bavj+/cAKd2vF3VK2nxM+Gt1jFg1LxxdnqykgUxTlFYaz8umR0y7GUhbdjJWLVQz+0P3AHcf2xjAkV2lMvk9lh7JNiKTr4hCIYca0PYy7wbgjcD3i0eYuc+YsD/W66nj+1uJAc/Ll/SS5oWhieaw5aA9XV66itsJXFOuZ57/maV/a+b3emZh6FNk4tUcxq8yOAStg9h+6EJ5Z8fa1zSf1DWH8JnQmLAw0MXr5fGanmvhH+faU5Q0aGYtC2XNh7SIeg9PxmpdksQ7FYGppWCju2P1V8EbAjgDxuzmGMii6hcp3cjSI6VJCdIsaHspVa0/pJzvAL5c5X75BG9/+Fpl7m/HspxML/3Uws8R6BbgarDlgD0HhS8qdzlJGL/Or+dv0elyfQJZKmQLxnK8WIK0vGdIvaKynnHgS9V6oquYeJW1OYS1MWHR6NOulxPeqONy7gdZIswKjR81ws9TwiHwysZ7qODWdbpemXUgvBGAI7OUs/maUq0xVF9Tv8G2Ii8rCmate1HYR+Bs+loHvuXC3wO+rRO07vW0dJxWjs1Z0NcrBK4G20leEzIX1Mfn+elR5yYEWgEzB3IZy+RARsq4vLYCYp5eXq8GYk/lzlU2Vs18ZnPJLxOv+Gzn+/sfBfMcwqFx996Tsrhq1cZeR17Fa5ZFuZOxohbZeIOcfE9po3LRfrXbBgDOwrdF/Xpju7J+1J+0wMYbJS2qfL06lh9P5UahLH2tDN8sJMt4LxEpPtpUbyt4kdJFCrcAcQZbfqypX28SDa93Ev7PdzDfzjqQX+r4ughTz8eC5VjxC7ASxK/u4BnQvR41WGY7FzCjJUjYyvKju49QnR4Q1sRixDRoS39Rk+1GZkWr5oWhI3DzgMzLtJjlow+EVwRwzVhsYp/lUBkNyBb4tT4kJl95ajYCQlk+Wj+saqM+5HKjbeD75eNSeFLr9Fs3fH+rcLnShxO99NwrjYFahJdPl886cDXYakD2zLsIn1/tTOxiz4EsYfxSxnMYS4VL9IKu/uAFezerF+DnY7ElBO2GhhULA5XZmbS7H2FZCBdoZmBrqWPNT41ironULkw+JYk7RkBFX9Lasd7oZCxUzoMwKb5jtoEC1qylKz2XIWnU02zw+r6IMs368OqqEJ6HkU5igLHA93kOYWWDcF5Xn7Usy2wBX6l6LcWbAu9VnMtjXmawlW/38usCJoINnCvFwS3Ta5ZkFVXrAbWo+uwDIR6F574u59fypIvILNd5C5jWjZRUoSlV6ph1rdDUcLjN1jXB2dnTI5YT1ftcAcC91+hafs8g/yzOo/kyrWLylZaegaHmP+q3RnnLdgp8ndDzveh8rFPCKjNea4GyTNCSy4wyS5dk2DoD6OInAl4+rhuGrkwnka6de2aoXzrRUomJ/n46I1W8DE+jfaDvTeghaLTV5Kv+coz3Sme27CinhPESpKWV5VihSVjYARHNJ2VdLyeaz4xWbuKzqlibmNV6lfcUcLTss4I1g4woF4auKWMp7awK5vUI1PV7OMAmyoEXdcMKP7dAPfKSI8ujAn4ykE2HghW/KD/iwypD5MK3qF8UlkWqNaZCW5citc+evvd5WQ+Fm2WoWapdCF2kcj0FjK4FnjArvLNClbJ9CeQHrKfH8SxE/QhPE9EMxHc3LxDLhy5EQtBR42t941D21v2+xoBrlPaj4ixadEWbdKA9oyPKMwpoWaaXKrbmHZiTsbSlPwiOPVRuq/LNtE30hnbCisDXspa1v1L9Wgoa+Q5OvuJ5WZWq+dD8oXoppYv+3X80kUlXRGi2sL2pRQaukXwiem5X2Tcs3QBeBFdP/cpjbTw4YkgRSdCWPADdBYwfaQXGZ5qDuIwTcxDfm7o+3rH4rlmljpw4dSL88ARuMaV7h7DnLzz+K+xKr9CzvjzpUwymCKRIhWoqODPGbBn6/p2d/FlBT3VmZixbgK5RymN2vdJa3tCizUfKabQiioWbCeRp5Z3+RFWvVseCrNaFDNxNpSv/zR+ykJ90hScsaROqauH7pZq3hDPyd3/pywldvM883KyFmk3wcsBq0EWh515jwREVjGBb8ng+0QzQCxCfiApeZGh63iW+dGkeXkbqmM9mvtLZeHiCNBuy5WETFmS/YP9H7X7jcLEnZV1O99+ZtlVlsawq9q4/rSq4tKX5C4lGLQwj82smSNUuSYr0tY9tCGCt6cxSoB47X0W2oYzkEwZeDYy1JmsUdBTsMG857ktEqRnPRP6EqpJnzWZugW8EzhHV64F3Nr4rwauBmGgJXaSEPfUbGafTVLAE6sXI4wqYpz/K8/C0DE3fm+VjxKewCvZMKl0M2q+f0F4+Bele34Ps4mEMwK702lVsVh7uluWMB8tohJZHIJ+XQ9bj6p8KQRPZYWhy0pFj7zxqURXcF8IbK+CIjeqi5VeDuBN+jjSZgaJMy/qW+VSR9zAeeob5Bsx4GVSe59nLfawtInW48vxX2/XwReHmxRivpXij4I2qYO96YF285TGHrlY32Mb8mWhi1vSAGdP8EYscgvP8C8wraXw/7JJeVPq927pCRnthL8qzSVlEr7HhxUMb2Onis4rkkShHwbKtFvYfAZ4VWvZmR1ttRepY1g/CGwG4h/r1/JW0lcPPpUhW9UbLegrYg3k4b/mIQTv0nB1jxflafWu28zJPKud+ytcNN0vQaoo3MhZsHWscQCpXlpehaKmiAmO/C2UmL/KP84l0NTwXh/OxYf68Xr5m2Bqj5eHoEy23orzn32dUo60ridBEq7tqvg9f6MoYQV0avwGQS/g+FyVsbVVp3Qx5eUS2WrbgbEEV+ZH5i3raYwrVCg/LAjM7thxVwahsnW0A4F5U2jL8rEy+0u4DNNWpAdRTyl4bnj8vT1lyZIWeiSxlGQemVh+FnYnQMqQ4fOUTlMq5BV4iWqpeTfHycwldMtL4XyI7/GxdA7QLpoS0NetZgzFPI5FOy3pFDb9APJ+kpT0i8QVNfi73jZagfYWT9f2g5xAu21Zam3NY6jcSmi7fs+tJqO/Lmb51vj4gXN44sEEHf4s0SMo6Mo9EOV7fgrPmJ6K6TfN2xtKcjwhDe/3rbysC2GtqhPqNnmvtJNb+RlxbII0o5oj6zShjqw5YckREqUlXGpCzwJzPaNbHdL01wV88Lq36mHBM9aoTrCLglXUI/PWUMOKAd70pnyuHbamnQNOEMVLF3AfwySdq3Y0HXjmIcxOdylpgBE4Lwlf4Rr5AKc2CrObrlc/3z77M1T+aFU1nek7KQvC1YBgBNoH8MDQVH6gvbmXvLnFEGNoCfQbeNW8Y9jDYaprJqt9IOWvzjsjDHIKvI6NekdsW9WvlWRB+pj/UL73Gp9BmG7XwnYeZ52pWA2akLTtEbW9VyduZ3QhYY70ctlHwjhwH5ibHcWV5qXa05UiyX/LCLsFrgZj1bbrcN/Tgarg8HpFO9NyrWQtJa6apVw3C2uYaWrjZgqy1RvgJ3WUlul5Oi3kVn4n0ULQFXw/KWl7J9xQzMut7aN4AWGFoy6KKtPjroWA9H/WKeyCAM6577JYVDSdH29T8GeHnFgUbBXVW/Ubbe/59hZ6JCIaeNbgS0exvZJyVT7Lyws6WSq6BbyTknAo3W+BFateDrheOlnleOlLBUjWV48j4r6aAlaVKEhLqJC0jmivHeZfh5+89x41liPl79AWdaD47WoMwCjdbkLXGfzXlzNcHc4OhaCJb1TOhcfUAACAASURBVGpiDKlcBNpaBaxZikk1D2iQgI002KKCIxDO2yAAZ2YIe1tKyjIRpcrLSYVbo6wTNwg93tEaBSzb9wAN28EbbhAR2GzjoZIVmGbgu/S1DDtnQtTasiUUgq6Gbw14tfCzVLkIuJ76jSiRjArmalcb/9XK8dcl2XOdl0dquIwNf33iS4r0cV6pjLXJVmWcV8ISgRVBsway95eshbQvi/fnOR5cEvgGHfJzlNeEiArWQJtRwAj8VhnV+AMaUCfQ3YTneMQuWDX2JnbCGrVPdNQknHmadRxwGwGoBsSMb5mOui19qtB+0IaWG24U42qXX5DkxhXz8kulrPmSQJa+UYgbtXNPW7bLbyC6w1dTxTyNRDpSx5nZ0MXsIcjdm1TD1/MrbIugyU3L58uS7uXOi+8gqivr8bq8XhnTJSLYzitv/jrkgyBKOaLXePDzYQ18gw4JSE3RRrmFgB4Bp6bCvfI9lHXKUjI8YP1BvjGAkdV0KRt+rrXE7GdkFpA1QMt6XvsI5iFoz9Uv0WsMOLLkSB/3zS5HyreDlLEcE5Yq2YLvF199joGXjHwy0jQQkyjH//IyBPIsQxdkKwxtqV8CaRII8nvG3wPjt8Ih/MVX34BZ0rEZzcXkmHBRzXO1uqyLYC5VclG1WqhaAvuefnqA++XrQic6abOiWY8Wu2TJz6mkES3f44gyRuWyYOaWhXQqDJ2ZfIU6VhuGzrYV681GVjORKhp+9tpCPjvMfs7AWNbV/GjqNaKALfU7K4O3m7wfeypUbnBhLU+yASuhiZYeefBFMLbg+wV93a56W8GrQTc6ASuqfr31v2jcV8JYu/Aj6HKfnj3eH2tc+BVuRiCOQRiVlSFitBsWClsjyEbV+KIseGoSEVfCM2c2ZC1lHFHMBPJRe7JfKD9kslPRMLTML+e1kFw/VL0RgKNLimpD1JoijsR3veOASUhqsEQqFXXHUs2e0vXSiEiu+dWecmSFfGvOvYlUlh80czoDXz7TWYVvuXJzmH5FGKwczEQ2nBGIPSXM82S6laYpGu4vMu7L/3oKODD2O6srypdx4bstJ2fdoXqfUDWHLoYwAiKHbmQMWM6M1kLimaVNvOyVzvffGn9qEtqmUgOtl0ZOujQPoB68w+Vbt6aMNCyhHJX041XwBgBuhSrRvNu14edI+cCTjzLNRspbCjiifrU0FcKvpQBl4tXzGEy8siZOZWEslTFWu7kQtQXf1+YdDny/ppzqbQWvBt3IBCxP/fJ8pIDLuQVdFIaOKGAJ2qR9IppNzqJv0wzCV0K7Wi0fuoCWFfHZz0glI5ByBYsmXcnw8jx9qX552aeifgD3cnntlrWYkEVkw5dbVBlLa1W2mmIOW/RhCprqjUKydolRHwivDGALvr3Vrzw/g3wv/Bw0DYzZep4CjqR7cJ5BWN9ukmg5iQlNaOqvjPExV7eRMV8d6NcZfMt4LxHRxNVtFr5ybNgLTxNIL2lkpPM0ctKKoQttASuRvv5Xpkv1aoFXCztLMPPXy88fJseFOYRftoTwlc4z1amN1RbLzoz2AM0Nl+W+5qFouDa4PDFJquBiSNmSSOuhgKPXtzRw5Z0BOs76qbEMWNshvCKA157x3PLSEO2S1TW4RkGNFLCW7ilgDcwP48uOiF7ql5u8W+djwnd318UF6ATSZFltLTA/1vJkGxrQeT4RzZQvEXuIQg/4SpUrwawp4RYVrKW9BZPXS3D95BAu24FKCF9ZJf5QhGJSsXLoSYWK8uSDHpaTsvB48L3ty+y89O0K+ilV8NPKPtFEmDFI2ZJIyyrgWgZaBn1lw9C1HdJ8tIC0DcIrAdiDr9cNL/ycNeTDUuBK+LmlWZRuwbklzKzVYROviHLq1wofZ0LPlmq12pB5OGTtj/l+KuFmBNivHm9MOeehaQ/aSPUiOJMoR0a6PPbCz6hc5kEMUmldRX2prs4gTapdqV6/EvW+Ivj9f0L4q8/0vW+L10JziKFnBEsVOh9/9fMsNS2BXAC+GONd3KDOVfA97QJV8GxzDvTc4F4K2Kor8y3TrnVmfQlWD27eJhyZTTpQPa1fXvm4DQZwBJCoC1mwFh/e5K7a/aYNt5rSzdTlXdR8a/WIsB8XzMtNN4jm6tcLL9/TtDFiHKaOTbSqHfedb7pRDd+vlPSIIpaQRTDOKGCZz9O0c2TaRRdBlf9F48IyzYJBi4n3REL4dLnS1w/GlolZ3ObjvPMdr/i5VLMy2nP/e3p0yQbp/Zzt+Qx83l/SS1HD9cqzseALXc+n5RaVGlRbFLAGY5RvmfWdTAvYqFqtBW62D1rbRFkQDwLwRNtushEBbu3NQbCa98+rG0lHIJZ/NXWtbLpBNIfqvSoOA0sFi9b7toz7anlylys5Przs24rw9RQyKfkE/lpKmOdHTKpgTQFLuBI71hRwOUeqWRpSxt9m598GdfgM6eL+q/ujDc/n62J29LyqDsFyvlwv/Do/0zz8jCdZ2Up3NsbL/KLjZxtMBc8mZEkVjKBqgVlTwNIP0fJz7M20mWX3hi75WfU5Igwt/RO9kZ2wIup3ZPgZpQXekp7vmgXmWgWMzmdll5tuEMXUrwQsz5P1ahStVs5+0hEOOz/rsQlXRAC+MrSsQVWbHa2FmyPgjUK3lwIu5SMKWEuTdVBomavjqziXxsPOSgiamzUxKwO6SJ5Ut7Kcp3TvZVCdMp683GHr2fZZjFtrzwyOgFZTxVYdaZay9sxT1jPzNuXIWK8wNFX48G1jAPew6EtAwI5svqHcyXhK1jJPCUfUrkxLQdlXvxrk7mVsiN6b1MLQc8BHx4cj64l52PmZx5Qv3GBDg28Uykgdk1GW5xFI539lunXsmVQ2UQWspREtweuBtvQDqdykaRCWm2jw89MDfEXdcvChMDLREp5auWIWoK80n3Rl5ckdsooKXowFE3sjLNAiZWspYATn+QvFbVppIdMg56lVqYgtWLasN+4L4Q0BHGk683CEmocvWPnBt0YqWE2tWtC1ICq7orUj8zXfypaTRf1as5C1sV2Z502Ykmo1Mj6sgdnK+/L69RO+X37dGb58kw4PzATOLWVMohz/y8sQyNOMfye0UHJUASMQo4s/Ur9neqlc/jqscLqE9qMdbXb0EpLlfL48yVpWVPKWs535uO1F+EDjwxHI6yq4gLlMyIJjwfcOYbDKPJmPIIzKWMpYmsco1VftIwpbrAbUfVvfkWXWCY9oR7ZROfs5o44RZLV0C95RH0QkZz4Xs0JpUv3yckgZyzzuh0Oclzsp56htBPNXOjt/jPkSEQajBswIVGvgK9N4v0ikkShPIk07rzWkhGrNWgcs25PHPJ+rb+BjerRVlpRdTsslRVxV3ovP1a0MI5cxXzl2W8qVdG1p0pW9+OKHt8v7YbVNRE8VfH0sSZqp4Au7jmnqVeahsp7glOmk5HW1SBhaql7PegG9343BRgDeivud2u3hxlLDEdDycuhYwreoXz7bWTzrl0hfVxtVv5nlSly18rIWVPFGHPLhCnzSFb1CzxKSaMKVN87L84mlkVHHA68HXQU+pmqUVspqy5Ak7CSMkZpCypf3k+ejsny8V/PD+1pMwP2Foc/3CVRsTw45oWru5vX94qaN/75mQsfHdLn/kofWH8snJPFZ10T0jFY9H9TAN+YgwjdOmsLVPk8iXF+7OZIm62o2DN7aG5B5RGHLgx7ytgEJtSatyVda+lmkRX3LtIaHL3hpPB2Fj4mda/4QaFFdBGnpV+x6RUTmlpP36hEF6oeU8XKlJWQ1oOuTuvByo9lTjSRYvdnOCNZovDeqeqPg1ZQw0Ry4GRWshR+Lz5btJ2UfeN2MWVeigK/zs/7n2czoe3UM1AxorZnQ1r7SHMh39fw6jyjwMiOaiJ4PaljsES1nRL8cLT8vCVTELAL5EeWrgTxtEVkeWY7U2l7vOtjLilbbXLaeBeTIE5US7SGoWrBFdWWTFmhRmxZ4n8dL9VuOi0nlev+rAVFfD8zL8TwJ0rm/ZUhZg7tZR5t05alaoiWUeXoUvqgcGedI7co0YmnyNx9RwVL9Fj/y4mzB2ANxFLr8Pcia/J4zP9NXROfHOLHcLYvfQHLTFe0StNqMaRl61lS2Fs6W/Srl4POH5ZIk/qQkBEkLvhLCZNTlFgGxZyaoa5cjbWHtba8IYKupqPod1T7K7/DwBaupCPs1uKPyWjmRbu35zI+XqnWpTl/pesjanjSllbOXLan1xVrf2VONOBwRUBGULajKXbK0chHwSrhqKthSwFpaRP2WPKSAuVkgRn2R8JY+L5SbEf0VK1/C1+IpSmciul4+38eEHzOjX6DFypWDloORA1MLMXP1bG13qU3WKsdynHgxO/p0ne2ONVPB6CEN5ZhIhy+CqaaIUTlp3TmYWY7Ucxw4E4Zug/BKAB7RTNRnZverRPg5Y5Yi1vK0crKOLIvSwfN+73+vs6UOUv2ipUX3fH3SFII2P8agX8LdB/Ny3Fc+2WgGQQ2UCLQWfDnMiTDMyWiPRBqBOjKNn8tjdK7lWQq2tIWgK0PRsp5My4SgC1S9mwfehlbmdL9t/pKILqeXEr6K+Q3fEztgZUArFe18VvQrpMx9yZ2uuI+MCi7rgmdLkoho8ZAGSwHz900CmkA9EumeGOXtaRbildXBSD3vaUlEeEKXZmMgvAKAvSYyajez/KhXmwlDEPVC0JH0iAL2yj2MLz16FcWQndUTFwoOSJ4uw33cN68392WDWbZb/srQMxHhSVdES0BKVWpBUqpYouXvTVO6FpxRPzwVjNqOhHTlTGJkIqxr1kf+5ASviHG4y7YtAJR6/Bp9Zcd0nxl9Pc8VbVk+JCEnZ0ij9bmyzlX4KuU1mL5eht6uHP+9ElPYTAU/J2MR0fMhDfx90t4/nq7BmkC+51uW6a6GkfPaCVIjOljncyCAa11LONb6GVxPKtcMeLVmPD+R8LN6rE++smCnjdsWCy0FYup1WQdvuIF8ayHqqnFfTaGW8LIWTo48K5iMfDLKk5Im8wmkeyYvtFb4GSlZpJrReUb9yv7JcDR6cINcD3xZli2haCJS1wdL0MrxWRQ6jq0NfvnRJm6hMHcBrYTzHOb3PaJnk7GI6LkkyQo9y3Si+edNII+fEyiHfGdMrSvHgSOOegO1JsydZ85KIWjNem4nGSmvTcCS/jo9/QjVt8LQ2rls3wo/L9p7Tb4iosXkKzRxSk6okiC11K8MWUNgsmO09tgCM9rYA477WqDlEEVAbYGvp3o1EBMri6DMz+UxiXLctC0i5bEWauZ5sg46r7UzLQGLTFPdIq1s0nG9PB9n/wxFE73Ggy3Qag9M0CZTSRXMzzlYZZhbW0+srUGGk7HQs4IlYC1Ao7JI/WoK2cqTloJ2ZivKlq0qS8eGSXdoGwPYM969EWHj5PgvUrvROkTLehHQyr8ucFG962LnK6IlWMvxK01Xv0iVzutpY7rzdiww42M79AxBak26siDbA74ytK2dE0gnkUcij0CeNKRkLKCSkS7VraV2+Wv/tki3IIsu+tF88b2fiOh0IqJ7oJZOp1fl1/dyORMaPYJQwlXuhCWXF83LvX5vVvia9wuFqMv2lDwMfe/co35RwREFnIGwBXFu0RuzMOMswhdwZp6O1PLgBiut3jYEsLc2N+MDKdxebQRNwtkDtRaClsea0rWAO6snFC8LPxPNJ39okOXHHKqyDgemthtWDLKR0LOy5AiB1go3a3UivjLw1cLTxMoQ6eBFKpiIbtGL2YVo8i6Q8kKspWVNPmAhonR5nyI3GPyYpZX1wdcz3W/WlKVJWiTmlTYHtdwJC02s4u1o48Q8fF3a1MaWnyqahaGLPVUwf1awBc+oekUQttJXMa/RSKc26TjsxRuzSJdRmcj2k4m34yz+ec1bKtZStlYXXQX82veZSA8/S2De//pgRuqX10Hq14O0DGujtFmeDD1H4WmFqMtxWWqEHl0o94KWO2RpfmUeAqwBXQ7cy+tteNo34prySSpfIjoz9TtZcC3pUuleQNrVKM+ttGNBWNY9if6dlOPLMo0vTXp29cRDvwboSF8eNB+b1ZYbLVXwPX0ebkZwXmxJKcD8CkNflk9JqoGnpZJlPeSHQPmIqXX5OPCaD0mI+OmngjcC8MBn8YbM8h17jmNVc9l7hwjYk/5R+PmZB1SBBmY0uxnV8SCLQs859Uvz0HMxCTEJQQ3OBI5lfelDgtCCumxbtofS6AVeDl0JW2QIyMXH+fTyC7/1/AJ7BemyLDdwc2CqLl5GtkWi3pWWF3/U10eaFopGG3TItcEItDzcrIGSH8uJVk+IGqCV7czGi1kY+nJR7nSs0DLKl+nyGOVrbUbKFdtehFLfXbVytiMFbEG5JWzc4SV64eSa5j3lnAlBm39f6lcLP2tjvMUQmF95y0cRyjq2mr7AOrJfi7+P0DPRQ/0S2eoXwbYmBG2p5qKCySiH+knsL0uT0OUwvYALFwLyJ/Edu1xeYdlvLq/8UmwG4qJkpUK2FPPVyCd6qVNNIaNJY1Z9+f7x8gwG5+s9DE1EdL5eoQq+u10uUbq7wsuDpPLV1g+jCV0WaGU7i349wtBEtHxAgwZfTeV6EAbvpwvXVmWsWnRDjtLJUXs79/GzAYB7jcFGn2gUqZ/c/zkCVa+cVk/Lt4618mi7SRB+fuYZKldTr/M0bUOOum0lVVV8veY23LDSJGRRuNk7lpO8yKij5dMrj4NXg66ELQpH83T+bHcO3kXa9VH2IsaMLSuvoWb5UcQ3N/6d52uREdhZWosKnpfT94mW478ItMWXB1oZukY3AOoDGi7sNkoLNyMIk1JG+tL8aia/RykgR2R1bYjaCsOsI813pIC5ZbpVym40AQtBGXXfgyU61/wiBbw4no//yq0n738xTLMTpDxf1o5aqE4ps2z7aj/lKApf7R8KN19Jf2oSKW0Q6W1K1aso3gJYBF0J28h9+DdX/O1HMC52JhCajqrhiEVuUDVFbIFZSUMPbEDbVEq1eU+bh4vljlj3MvMnGCHQ8jaK33ud5Q2ABm0ehiYiPA5svUcWODMhaA3e3cxaD7weJONPTsrbigDuEWKu7W7nCVileLRKLXwjPlA+CD8XQ+Hn+zl+AtK8DA5PSzBLv7JcVP0u23/40jbcsP4RLSHIYYng7ClfIv/xhQi2om0LvAi6/Cdf+/NHQEYwnoWme8K31PfSpX+kdPm1mNcVac8JWef7EEZ5dvB8VvJ8i0lLBfNjTQVHxpIl0LkKRuk8DE1E860p+XOCvRCz97eUle9rLbwtc8v3nIjVc3lRG4R3qoC59Vay8iVzXwMnYFnp0RC0lz7z8fjhs7W/WvhZG+PlFwAN2Fp9pGqttmUbnvp92lX8lUqWp0WUcTRfg7qmelFZusOXj/Fq4C0/cflTz1zjVF5KRc1AXI5nIO5lkXHgiNI9sTSpyARUeChaquC7q9f3MzM7WZsFrc2olqBdrPcFftVxaL41JdqUIwNhme+do3T+3hPpnyE384tsgXXcgxLi7dRDeCUAj5pg5Rl/eZXtoBBzpEmrTgS2lg8Ugp61uww/F9OgG1sGZEOzlLHGfmUblvotKrn4Mreb1OCnQdWqq5XTQs0efDmwH+ea6uXgRUr3AtK4WeJRs7KdwbN/qAwDcTcIR6I5GgS0i77MA2klFM0f1oAnO+lhYTlp6t6Es3RI1EGg1aAt2y5haCJaPif4uUEHLSEr3x/5vloqNgphad1D1NkdseTErEi9LLzrIPytdI20ZcHXck8w+H7CgjGCoszX8rS2vPqWL+PJR1qY+OXWV7nzGco6mHkaalv6smZHE9FL/WpKl5Q0LY93T1O2Mh/5LH6t36wC38vlBd/LdQ7fbx7/eLMlTXZJNo3yviHdJ/+rjUNfro/+ixsK9+YDvefoBoXYMc8nUQ59DpE8es2aP7G1wfObPfQd9I95ffuG9rIoz/uBTPZptnsWeMCKadHhLHTeenkedolea1vjvja49VZ1q+3dLNOyzxPuqLojXyovhKyde/WD+fLJR0Q2NLXj7NIha+MOqYpln+ZjxK+Zz0Rg2ZEGAE/VklMueyz9gnIo5IxUr1S8SAHz9B7GRRIRqWqYqFIJy1AxcoqOS+fQsfzuozwJ4TM9lyWdLtfFFpVoj+ZybE2acpcOAaWsqeOIaj7RhehEr4cyPF/345Msu2KVD1Z7z/kHfwHpsj7y1UPlhkRnz7By1iLKm5wycxsE4Ik2fSxgug3lbTAVppPv+bBCzV4Y2wxBvx6+IO+IrfCzHL9F6bKuDeZ5+ciSJrSz1rOMtukGUrpWqBmpL6SyWkBMOK/AV4ac+TivB94hY8CWsa+QnKRVDsMgtkKgMnogy/DJVwgY8m8px8uzPDkWfD2dH125En9QgzejGY3lWqCVvpZhaw5wLTQ+3yeaiJbjwOV9lRAlWr4f3KJ8iwC9CyetHbFaGhm5wcYniv4qttXfVdYL2iu89NYxYJSugVrkoUcPzt05oV6QjsLHHJZyuZEVykMTrlAbafXLL+QS0giYFkS1NK0eb8uAL59oJVWvB96RCti0K9H5hDf7CKlhtGa3GLpgy7LlfdWUrgSNV15RwdcnEM+z7+W9K7FJU6UuAu2rDVsdRzcCIaLlIwrRwxmQgtVg6aXzc/neyvooX1qzekZjtz0AiwD/rh/GMLo70v+AGdBeuDkKXwu0ZvsXQuO/xbRxWqx6/WVFWLUud8dCx3K50TL/pX6JCG+6QbSEY+YfCZ+a/wi0FUhL+MqQM4frSAXcbCAkXba2dCHMv8PcDwczL8thgcoiWGt5SDU/9pc+XWj2oAa+LhhPjsKTpu7d42p1DnBvTbFUxy8/WDWXtNIH/ojClwIWm3Ig82CphaaRD68N6bNY6EssnURB6N1FyLSxW09K27ECrlW68glJLb4oF2r2QBlN99qzQtDC+OYbSPUiNaulexOrMEyX64OhytWO2eMGJ+0CK4FJ7Fj+lWmWyuXHXphaAXMEvhyuWyrgc9SfADHcyMO75vE8ra524Y6qOQ0uj5sA/qCG61nOXL6KEPALlpHtJYmWM5rl5h2aOi7taaqZ94eI8CMK5Xss34dsCBm9r6ieZrxbq941FlsXrFFbYRZ0D4sScJRvekEPATkCTC9NK2OBdpH32jVGmxGZgSjKlxOrMuZtUKAeP5YeEdESnsU0eEbzyjEKV6N8AumVypeUYwu+1kxo9E+rx9NKWU2Ry8sXf12z2dHyPUI3KTzP+yy9zyyaZ6SfLtfXzZ74HqLZykTzm9SMyaEZ6UdLl+1qv9Vvna+vMHRy6KrJhsk5S8VvoSH7zV3aSAH3eAGl67XPEO5krfCNhKqTlh3/9cLMxTLrgNFYsPTphr3lxhvowowu0OXYChkjRasde6FnWZbalK+nguVLlXkt9gn4hqawxwxFF5Ukw84lLI3gWX4HV3EeAUrxK/2zCVrT5T4ZqzyoYT42q81otidH3bu/DDF7m3rc/cXS0a5Ys3HgFsuoYK2Mp4i1dilST3OeVblrPuZQ974T21FXIhYJTWdfkhei5n/Vi464mz4vIUeUU8JnWM9+hjAqg7e3vKjliQhPvrLCyQiu/K9Mk/WkLwLHKPTM6ykWhW9UAZNxzg1tSyC7idJ4SFoNTwMQQwjzcdirku5d3OXMZl4W5V0UX+X384Dz+UrP5wXL7Sk1oKIwtBdiluFjOd4r61rp87d2/ozgp6GnI2nviwY/FL5Hx7JOLYQ/mL0x6rWoXUt2AmuNaEeUrQfcdNsMZmf8zUdhYxlORmFmLRRnqdmXD3l+WdSFs6f55CuimGq14KupX2LHUlVbbSMlTFj9WvCNhHyRCo5MyAqP61ZY+XXJh0OU5ww/ISyhOysMztFnI9VssRPI49Dgvk7imIHiHmV5LUl6gRMv/7HGe19dw+oYbU3JZ1VH0/nNAbdy0w2fjuRZRP1qdbJ5VZZRrVpZr1ORNvqMKW8wBryX9cHclC9o79sTDb6ams6OAT9M7owjoSvDxjI9cpzZvYqXR8Dmdefh58/z8LMGzStIR2FjS8ly0KJ8WU7261HWg28xDt+LckziXP5DL4toCW5ZhvcFtUOBY94naWU3r0Un5PtJIp1AHoF66HPWbsi0dtjxxPydLvOZy0T6mK01X2JxMzn7bSyPz+ImOJI+a/MxzHRW5n6Y140ekTrLhyZGovVD1sqMzAvoZ29kEpa0nQp3T/22+HDLz/d/Lsa3n/TGc3n6y33sAgLHcEFdlI7Kn9gVfJIXXCJdyaKLMa8ny5Y8DdISzhda+gzCV4LWC0Uj8PJuEciXW1hm6pZ83l95TKzuLF+8biIxKUu+l/z9LCZvcNDnq0UkeBq6mdL+PsqcrzR7yMf8+3xRgWqBVttwBrXBLZuuTgazJmJZFrmOtUL7LP5VOcnWW1vY+bYTAPe6+0BLkLyynawGstF61hiwVkW5E/bGc2X6/Ny+IHhLl6yLD5ywdTHCz+hv5AKs5ck0pOBI5Mt0IrhRxSyfYvAtx/wvgjJPt/6RqKelE9n90tIQhBf7RhdDNzqWCtY+b5RmfTf4X+Sf6DHj3gadHsVZftf1mc7L30wkfZ4m1tGz9f/qvtDq3BHSrylZcFv5XbXT1kBtb39lAI96w6xPtbFN9KXJKNxaMGdMmYD1yvZ/zDLdK5MJv3lrjVH4eWbogmuBlJ9HL9paOQvELJ3v8aypXxSuzcCXn1vdQaFhWTaytInAsexn2CwVzBvRGs7eTGk++Tnrx3SZ/3y0YRtpFmi1OtpcCqRkZbqmdtUlgcpckLBFoJopX20DHhW7A1sRwCPgOzLuD4pGYOzlZSZhRceAhWk7YGkTsOZN+3fk3mxmlI7AjNo3Zz8TOObn3oXWCmnKNA3EUv2y+nybSQTfiziOwFcqVQJpKJSM8rS6KK2URa+BRH5IBUtDNz3oc4p+ZrI8grN2oyWYxoc/lkvkLix9fuPIYYyjPRcIZQ32LK1rSQAAIABJREFU1k2AekPQOgbshauj0bsWEKfr8gq9JupKP1qn2hi0kxD0WrbyU5DW9OU8gGHZJL7I3OvEwmBeiBrV0yZszZQDmv1c/krVql28LWVlqSmZpoQqeTpXv0R2GFobT+XHHHpEPih599A/6Vu+HKSyNSXsjQc/09jNyLMyunGx3mfl/VbLeX+LGQAv48DnK54zcT/2fyvFtAlb0TkYyFRgi/X+z/kgbIOeaquZZFVzTes2HrzTeULCdgBg+UZpkBwVvu4c2rAmMGTUr9eG8yX3dsKSx/dz/OP3drC6d2OpErJhbDn+OzNDsbgKOHLRtvzIkClQYtbEKy0EjZSx7IoHRw2yGpyRD55mKWHeBlLEzzz5vnAVbIX0NeVqhZq1PPTXAb33nGAiHai8vDzWFa6MUAUiQ4sb5+Vv9ny+4mWIgevGonwmfc2yzdazsXo2rQTgrR+W/DbuhmaWCQs9TM6ALmaBNgJd6SO7XKmcZ8d/J0kO7WJNtLywygu+Fmouf7U0TQE/8qX69cxSkURLCJJy7kFXS9eUL0+zIGwp4gurI5/8NHshFnhlOS0Nhaa1Gyfthg3kRcaBI8uUIBiNdNROOcZha3wDvHTufDmt4S6tbDbvMNV2oIB7mzflPPBNaZ4MlfDXY6xECTHxkJQ203lRJ3FXzn1b6XFIP8ozBfI0pH7kuaeM+YU+o4BlGZEfVb/oXeIgjsK3nEfC0KSU56rbm3jlhZs94888RjcxixdA4q8F2ojSJZEn64K+nC46FOfpy+84qufNqJbp8thqY1YmMgacNW8uS039JvukHHtl92crAHjfb4BqI+/oeoR2FDudL/pifK2OA1o8O/OSSpdlPDNfgqZ0Sh6q6120IxdscPHOqF8iHWBIDWvwlQbuCcw8bTa2dezdQCBFL98bGIbWOinLRULTqHPoWNYReXI9cOQmFOVHbnJrrAbMwsG2Zd+sWu6/2uYNKuAVP70eX76WiQjeLOhZHfwjjO52hcpm/EWWaWjLLGbn/KptqRmkngica3XQxVu7KEtAP8rJ2b2e+tXGgC0FGhkD5qaFo5Eilm1KBS77gPrupS/C0FqHiPThAi/ULOtbSleeizy+6YtcDicnYlnL7bhFlg5pM6ctH94cDlUNv5zO/2ppqI527rWVrZcyC4L7pP5gAHt3Bft8U1azHi8/OQkr7jYO7poyZqjbEsjWeF7kgq7V1epZ/okWuz95ZoWjiTD4EAit8LKVznmmgdwKRWs3EtIWfXmEoRcbc3jDBlYjqLyndK0Xysd/xTBIZC3usy7p+6ZnQVoFZvnbLxOyel9us/60iandN+iwbGRENu/7DSpgzTYMdbcq5ZovcqJO7TiTnDgVWdKE0qNjx4sJWNwiapdoCc1o/eSFXJt8JWHnzRi2ZhrzblgTsKRpQLRmWaN2tb4RSEevE4WhnxYBo/U5os9MKyvbc/IyG3JoE7GkRTf2iJZx+9V7LDirdIcNs60p2sYzZSCA9zb2mxm472S9x0+cMt4a4Naxp3lX8OxMLV2WQcdEQnlYCrRGSXmgrbmQlyJXH1iaRdSnNG+pkTzn9aR/S2kjQGdfn9ygZNEBdI6iENGIhqeeg30owyGRzTOQxcrEx3Mj/qRpKyNCKyve/ZhusZ48yPkaBOD3uW3Y06K7wWh1IuUybQS2m2tRui0TS/j4WPTisrhmWBdMK89L1wypLMVPJvxMlJuEVY4tQCrdWuRrqlmDMDq2YCzzYfmHw5v1nmbvETUoe1GQTN7DNFhGZ0K/zpeNReZfaP5meafsG/jsQAzKkfw3bT1e3CeKMnBnIejM3UPkwQsDlO4bWJrcOv6baisYclvm+1CfWebi6YUm5UX7AtKcNso610hXLOhG1W8Gvmg8WJb1FKylgi3TQulmZ7RIh5WG5H3xpbWD2lXy5DyE7EzomrW71uM8m5Vxj92wotayE9YHs50BeOfWvF1kl16Y/rwlSNaPtTVE3RqWW8yA1szqpgfdqBnl4d7G5CtAzVpgJ/14vrywdm3bZtnoe5+PsMbrWSCubbfCeg4DfTx7f5HVA8CeWbP03sAdXu2dc+YOvNaafUYunJnxwgi8A/C1QJa51kdnQVttWn3R1hdbb4mmrj0r0YJZ1CDy8fPohBfZaA1nP8xaiiQtsy64powVmnYVtTY0NXoi1mFhOwA8ykZNYNjBj6D2UW1Vpl2wMxNteJkateQVT17otXW60eaj8NXys8OjmTJam0SkrwdG59HGIj4b7/NiW7fmGslA1wpN78Z2cF1ax/q+UBfA0zT90DRN/9M0TT8xTdOfmqbpt3XtwWFdTJ3tuEMzVfmI0GCv0CZadtTp3qK2TE2Y20r3dsTiJse3vXuiZ/oaX9XonIHO4efsZhyt1gTkdwVN/mL2tgJHt4gCvhDRb7/dbn8jEf0aIvoXp2n6m8Z2aw0b9O3byZe6ejaksJoLR+aisEoYeoD/FvBGzAJjD0uN4SbKWpuMmNb6de31efA1wGhP8rdorTfnW1zTdnIdHW0ugG+325+73W5//HH8l4joJ4joB0d3bPd3NGvvkdrxC2k/9oznza9qkT1oh4TIooolshSFp5e/V5AW6RYo6y3VscpHIJuZyWy1lWmnxbQ2zaVIxdCQQ/brlR1zNmz5DG37XRqlertbzXXnTawR3j/FU2PA0zT9UiL6W4noD4O870zT9EenafqjRD/Xp3dP633/T9R8iRkhIaJhs0a70gkeL8vZX2BU1/JXbdE10bJp7+JR/p5AWqRboKy8Vfz0cFn+euXdNhvq1rbTYlofp8hnegJlsl8v7rvxq3kRDrzfhyy/W6u57vS4pg23TRsPWRjA0zR9HxH9fiL6sdvttiDs7Xb77u12++Hb7fbDRL+gZx/flu3kM79e6378FkBHXFCagb3/m9ywWXu19YAt9zHyaXJVH8leWMX6cT3rl8c3A1ciokvzncdhgywE4GmaPtEdvv/p7Xb7L8d2qdjxqWfsc+uPTLGaC40HVVN1jwBqy7ILRUF9Os//1rpvLRMFs1auBexSjRelj/KH2ypP2/HtbYF56w4Ylu5by+DMdhaZBT0R0Y8T0U/cbrffNb5L78RGfbl3+KPRgNpF3daOQaGmow+xWOHiLcGX2nnU8VVjxaemwi2AR0PrJWR/RqFl7dyzyPp8a2gi0J4Xao7YMnwd/23w9mW9IUM+NbbD69JbsIgC/ruJ6J8hor9/mqY/8fj3o4P7tS9rHQep8b2C1V5YsvW08umLR+OFFNatHYvkSYmXIVWiZXKzVaueBUgL9si0tyTT90U/shED/vmcQJpVp9FuzM/lZI/7lu9wy02oLMNhbSlqqx4R0fWivCEHLBus75vnfmVvt9sfove4B1jULnR/l8r73vIjL756GfB3uZxme0Ff6TSbmXylszo7k5dd1ju5sz4t36H2z6fX0o8z6bNWT6T/Ds4ir5S16iTbmEDfCpguNIdUpMlP9Aqa8WPLysfO/Zd2vxHnqB7PjwCem6Z6NVNvUKKT5twGAv4tpb1iuHo3irXGNgf3ivtZr2Q72wkr9ZCzx99eu9sGbfMvoW/XQePBqT4krmqhi1JmqURNeNrLQzOekRoG52h8FJW3mJCBJFK90qfnp+XhnaG61uzkyIz06HBCxpg/ORchunKg5EdWCGRmVb8pcFcs6fuoNgjA7+9ORbXa6fjZ/QbNsv6VyBqDWo4r4TCbVdYLwXljYLzNxf2DpWQiKsdTSDzMKesrVCyqDi1FipiEZGbsVU50ahnWlmCP3hCUOvIGg7/loSVIsgHvBsj7LL1hCg38DTCP/gYsy4wRW78ddfXDxQli8m1aD3A22jcUZeBABby3mWi1z5wBNnINnLW3bdKfBVKvrGfa2FRkzMoa45ot/bCuW5mJNjxNXsgRZSxFDfr06TwPs1rvJArfetBDcETjusivTOfg11S1dTOQDT8/+3Jevk+zTkjjQJVwPoFy0qd19xAMTV/POhSvdA4oYfziaidV1czZcFdH9FgD/OaBbT3mZKztLATdYhsCv3VTjuhDApzyNaFnTxnPLxYY6JqPkq4rA/2CckMwjISaTxSDqmeBi7aECYebpQ5lHe+Yn2sw5V3TVDFaIqQtG4qoX81mdU/KxCu0+Un2s/LqRcLcII//jCRotTDyazKWD92oZWCcDlFvtIXr24J11XO/UqUHA9jrzJv6NPrbwFnUHMbe2FPWes745CZnnM4sAmJPFWvlkTIu9ZV2Sxj10zkXhpZQBq5ncDzTEpayOW3Ml+dF4fvJOJY3EegG45Moz+15w4Lg591wRWHr3TgFlbHchCM6M1la5LfSMgQEfV9OdEE349kbfVm3Ns3zm6qzZ2bkgf2OFPAA67HdZEsYOjMuE1S/kbt5eczLRseq5PKMK51ndZdKm51HlIx14bbqRMKZ3jiiEYbmoNLgNKtHc7BqkEUQ1sLLCLokyntA9tR4Rs+V8DORswUlCg2jmyNPnsvPyPpuCJ8tS5C4Gka/JZkeUbHexCzL36sDzqeVeTpUS/hZttP9qVSRJ3Hva2h0BQBv+RyXnTbTAmunzPVynt39Ru7WFwAMKGYNqDIdt2eMgT1k0qXAUrso87+yjFVH86OlaRO1zneYZNYAE+khaQS8Us4bs9VCzjxPawuBHR0j6Fo3Gp9oGX6e+OclPw/rJgulnUGad+PEzQA0vwHUfhs1ytZK92AcuTle+NRuyiM39TXzTqLjyXsWsab1X3HzDhVwz+nFlU2v9QVLtGMpUL1OVO3iyz+aHZoJvc0sAjrtAhyVaxE1DXzJMHRkGdCsfqCMzPfGfxGUvVC0JyxLWU3NWwbDz9Z57WfmfXZRZSzM++7Xbr/afXjIg26N9Qg1d7WOE2o3tncI4GINM9t6f7lGP+XoscRAznjUliREJlZp51qaTNdC1s8uizv6eQj6/rU0J2JFL7wy/OyFrjNh6PM8pIpmQyNgIRVshaI1JczrWUBE4NXgi2Y9a9CNqF8efl5UlmpWi0JEP2tNYVufpfi8y0/oej65N5Za+v37vAQzT28d7gmDOzMxc5R22Q2s92crAbhmgw2i+Sc36o3svGbZUsGRMAyqL9MCExesSVhaegTM0ZnQqC2siOf+yrjbVbsYE+njwLJcFs5am1oYuiQz2DzPKTYT2oOwBkikctEYsFae9w/5tiaIpceAT48bFQTa7A2QB1mZFlXGzPgELD7Oah0j82dGz3+j2s1uNmp0uZzwNpRoTknkehOxHnNgVgV1z8bq2bQDBbzmu94R4jWz/ay81rtLZaJFZKLHQoEqP3JtiZKXLlWAbKdczJ7ncmA1Ejb0LsiemtKUMYIFgsnDvL2ONRhrECblXIMrerkI0sgfV9u8r7Lf2mtx1a92IxO9AYpCVr5o+R1RbtyiE7CsSE92bDi6sQ26OV4o5NpNOLIWVcq99kFoK7hrWxHAIxRsq8/GD7EmZJNVwZE7VWZX5e5XrunVLhhaaFgLmWnjyi//rwsKB27kQnM5zS+KswsnumCXMhpc0QXYKy/9gwt4mYzFoaOpYO5mAa5HHoIrUqy8rDWOrAGbCMO4/JXw1fqO4Pssy9Wv7Jx8Mdpn4d1hyPLcP/oM+bm4GbieXzeAy5n8OJyMZvyj3w1Pt6JNWnoI7izypW7CkbymmOleXo2l/aFIadRJNOI6BvgrK+BRYeT+s9NmrtE/mV/TtZayqJ4cA774Y0laenR50Ssdg1a2j28C5heyy+k0X4dpXWgllEmUsy78SDFZ6kvrx3k+I9qDMFKR6HiLEDS6GZDHkRA0VL/8PUWAJZq/59rNl7yRQtELSxkDWN/OtNiAg2h5o+oNzWgRHm7o5lcLaVtLmub9UD6RcjNeG7Gzlh2ha9+wiah73d64jS87CEETvZdwQthWmMwAF+ILi48Naz96HbTZtlEbV3FRNFUMT7MULBl5llqy4M6sNQTNjz+Jv0gh8+5p3UaQ9vzxvsp+o9eCQs8L9Ys6J9PkMbrpQXWsMkThGfJy/JebF2ZGpoWLpU8vXVu6BNMv574PZOl5ef5gl3rPdgJgyzJz4DPhh85qvFYFR+4gUZlFGAmP88hxIak6y3E2HR3P2gXhueV42fwCMlMPbBz4hpRoMaScZDqvZ4U5Sxovp7UNFFkkFM27h0LQSIGSSEeKNaqMCfiQ/sk4ln0PWUT9Wu8xgTre0IEGcEUZX06viX+XEx4uKef3v8swM/qua+mRMeHs0qXQul9rEpanXr1Q9ah5LmoD2Wt47whsu7+dAnjtqeOdwhstX+BOExs+X07PH6KcCV3749cmnXgXGQxv3Id5CPtxfP7WfFcsBEmiOQyRYrLUlKaevboapIlCs6KLedDjUCZagtgCciQELctkQ9Bh9SvfLy1ddtb6TMjI02Ar653m8wz4jZ8MJ0fCzGiIRZq3RCmajmwW+Wp5EEMkP1pWExDasF6VRXbBqrVxsn0DALfAtdcdT+ANrf1SRMZOvO5Elxst7mKxJtEU62UGzVz6vBs+aGeznNkFRZ0VzWahXtCFVIOtVEhWqFlTZujCj+qAevJRezIkrUHsLNI5qC0QI8CifxqcZRkS5dGx9GfCV0YsinmRiOhnROJvRBkDKN8nX90vh3KylK9m52BGN7vFT3ZM2UqftcMiXdfL6TUBq0TGojf/0bHfqHVnV4QBpdEML7Id7SMS8RV7E7vQ2O509h9xh8rwNJmv+Szp/C+JNGHXy5nO5ytdLyc6na50oROd6PrKpxOd6PL4m08vF4KTSC8XmFLmTNdn3ZefV3+ujxL3Opd5+vmRfvlMtzPRVG5ITuw9KBfVK3hvruwv0fw3dmV/z+DY+2zQZ/GwiZSvxhUlvuwbVo8f9zJtprQ10Use87QwfOWxBCaJPFmOQB3Nf8Sv8DMXjThShG4gSzovO4f1HMyvdD10jdK1tmQ7zSFora4GXw/Wux3v9Tq2XgR2IwB/Q/PLwZq+OrYNLr6wTA2Es/cLz3on+kxEp/OVLpcTnc4SonP4nRkVoulLOMdAS1TAvIR2KXNi/i+nE50uV7qeic4SphyqCIQc1Cdavuc8j7+HRPi9l+dXpdzDVAjzvj8sAtsL4W+tlo5MtmGBt5zL4zB8eaNa1ACF8PmxBVj5otBf6YvY8aNcCT/fhzuWYeaiMqMqt5xrcLWWLmnpWIELhfwcdmLp3kMYpNWAsyd8w3V6kX37XbJ2pIAt8yimlan1rRTjlmm6tvtI6SJ4P/9Oz7zr5USn8wOG1xOdThyMc4VbftSR9Bcsy4XgGgZtUbtIHfMLzyy9PJzh8pk+IXBKpcvVLPpr5aELPH+f+bkzvEaEIfzN5Q4qTQ17QM0AF5k221k7R8ch5csdIvgiRYvKa2URnLVhB+SXpZe1v89d2AT8LKWqLVeSflAZXSEv09H4rxzCKfb5cnrJegTEjPrVzlGdHlyEPqw5OrzCCKCiDvVrZ6eTsIj6vpkNvqJjwZExkpoveqM974wJP6XIW6PL071ZmdbFR5vIhfo2S2djwXBvaCJd5WTywIX5afyiTqTDhOcLQztDaROeJPhkWe5ejgUji8yC9uDLfcnNNqBJGEtDihYpVk/hZvKAfxm1Rd/lebr+fY6CWWsPlUFjzYsy13KTGrgr9Cxyjaq9PtXOq3nHtqEC7hmGRn4jsrNzH6yxWUvBat3T6lrKuISdztfFOHAxTX1y5SvT+RgyV8koPH3vGgslM9XMj5HilmFoors6uZ4/z8PQ8gJrhaat950rUZ7ufXWsPKaQiwq+XJfQulzxt08LSdeqX+krqoJlnlS+RMlxX3ku1SrKJ9IVLpF/o8XriRd9O9Nz8tV97HcJTQumaGcrC8xy6VJkTFlLX9wggJUPiwlY3hhwZHxXM6scAm8axrJwVFSVetYS1e3uCjYEMLJIOHgUuG90v1x2tMjLyY4BmyHoV95nMf5brGYcmMNSghaFp+d+5qFqPulKg/bLN0s/f4sul890psdkLPn6UWga/eXHMpSMwtSyDs9HEAdmjgcH6hf7RPlYDvqlSEXL07SQMxHBPZ6r4IsiD0gFS3hKNRv5x+sJIPMnHxXjY74aTGVYOgLmSDg5O17Mbw5mdjnRYgesDFyjoWfNTyZUfWH/qm3NaGnfMPfOADzCNGBbUtROdvO8cpk0IhvIXvMPEF+vJ6ITV6b+TGdtRvNcDWM1fc+7PutzX1IRL9NfKpiI9MlYmgqW75l875BqLuWIlmDmdYiV0coCK7d26kcYhHDWUHtRBczBS7Qc8yWqhC8CpVSr2rGV5pVnUOaTr4r6tWBaYKdBVoKZCKtWXJ//ZnS1jdKJlAlYGfMAa+VpZYaEmyP7M0vF22LjlfEgAN/o9QZYatVTszzfopb0M0olg+aLWe+kFwb1QtGpEPRDb5XQ8/ny+KuHjFuWDhWf93KvTkvQzn3NJ2ZxFfxSx/fJWMVvWZIUUsHlvZGAvijnF8JA56Yp4iQ4I2pYKl3+kVtpSCHXKuASbiZiavcsQs6lYg18I1DWYEriWFO6Mo3llZ2v+IMXMHTtvc01YHMVzeHN/Zbypb4bZkZtifHf2QMYrDCz9lfWRedR5dzFRu0BPWoGdPEb6/cKCjgLw0qpl6pbCWgLuJ5a9SAszyOhaSMETZcTyXFgIqLTCS//wWO2d2fa0qFXngbt5brhe17xu/TF275wyCMVTOCvVME8LTreK6GKVC5aWxy0AuHziejy8HM+E124rw6KGP0SvLXAMtxc+kakLDXqCV8Ues7CWWuf9ZmrXyILuMvvLAerDFXbYWlf5c7Lz8FshrTl+O/lTHD8l0hPs8r2gq+nikO/Ja1QLVAzP+D+4ekdz4LubSvMhC5l+d9oeXQeaRfe5d5/fOVuODI70r5g4DEpfoxmNC/HsPT6qO35hez8WpLEX06F+nnmyWNUDuVznxkFyHzzHbOkwiSi2faVRHOlemZpcga0YA0sh3yU4/NpCd9P5wr4ctPK8U7LdOSDlyVRTn7WKI19rmjfZyIMx8hTiqK/m9olTZotws+eAkZWC9XotW5IODpj0WcErLlL493QTfIAsxRnJgwdrVMsMtjKfYGJWK2CnAJdQKFRmX8Wf5HfgKExWzm7OTaj+d64NaOZq1srjC2VhhoCP5VQ+n1G9POTQhdtmYaUMXoPtXIov9EmuoPt5l0XHm1ak7D410KmS/sEjlG4uZzD8d7iXLvpKPlfOuUy6drNTlT9nl+PHUQbb8zHav3w8avscrZ0VuUu83wwywetLLafRBYNPVvDq9FyGngvRh40WTCqRFuo31MZ67YSgInaIDzKWuha0YwVYpZpWtfCIegzyXFgIpptSxkB7b0Ze+kQ8kUkZjFTCTPrm3rIMen5mPIjTY4F3wvqNyfWNpS8HDnlUH4nk+PCn873DTvguQHjyC9oppbZ9dsFL5Gp6BfgI4rBVwOnpZq9crwPLO0FX3puvPEKJesTsfgxgiyaZCXTZZ0eS5qKwQiXBVov1Bwpi0zzYVkaxtL4ryDqpAaYY8aMVwQwUZty1dJLWgTwVpkkjOVnjRQrcp+FMCn1ZDoCkBgHJqLnrlhR0BLNoalBu9QvhsaLiebjv/dzfTLX6dk+3p6SiF67Y/GLM38vT0r6q0Nz08oV+zYRfQXSI8Y/H3E+PY4tEMu0M9FzDDnUvLhOS+iWNBW85RidW9C04GtB2gOsBn6ZR+wvzcd+ieYwjKhfOQ4sfcn6xfezfQDmYpklTVf05LPZVpTgbxao2bJdjU9kisyAbrFtoLwygHuat7woE9pOKHArrEwsz+I5gimJNAT0SGj66eO+LWVZD1zukk9nfRtIBFoiGRa+wguGVef+0pZriuUNgLU95b2NuQqehaK9LSf5+2ZB1gtBW+maFThcRBqAJwIxEYaxlY7KcZPQJQqCt/Rdwq4Gvt6x5Uemke0non6JEACdCVCEJm8tjz0wyzqyHSv8vBj/RaDVIKqV80LQvJ2s2k1ZhP69J2BlQ971tgGANdiNClF3CjNr4K2tkw1BZyBc2hRhaCJaPB1Jm518d61fALJwrlXB/Ph5YeKPKkTLkkgcZyAr1fGZdMVrAZorcxQtcS5EBcRojFhCdzGDWjYH+imh+2yz9E9GDhCMJZB7wffboG4EzkbaC77Ldb+ZpUNS/aKx3yhkNTCjdqDfy2m5/IiP/1oKWMsnpwzyQ0reRfzrbgiMMk2OB/eC6Zt+HGEEqJ3AmfbdeSIWr6/BU2tLAhxBVvP/THuFoYtdzyei0wt83tIhbalSyUNjZvP6Mn2ucC048+OyLOlK5ycMrufPRMQUHn8/UVp5b14Nzk1+Pqi+rAeG4FRDyrekKWAu30YOY76EqeQR4ZC1NBW6pS/8HEG2pMu0LHwlaLUQs0zPKOPzfMvJMvFqCdU4ZIkwTMtxMVT/nr4EM68j20Hql8N3EX72FHAkXStjgTRTrhrK2S0oW32OfWLSG1yGFHljM29+9FvTaJkfQuRHgcqVtKfP+XKkiMWWGi3zssdyuVJsGRTIe4znzR7UII95WkkvLhBYJHzkhZ1oCV2tHQ9UvIz0c1rWnc4vaJYlQ+Uf0WvJUPmHyhUfk+xzsA+L90K+pqjytd4j9J7L90z2h2jZJ6LZlpPoiUdEL/VajnmZcszB/ErH9bXfkbXDlQVmzWD4uVg0wmqVjZapvXSGQByB4MBr98L6QRnd269kLcuLZPki91onYiVM+7ylikHvcEbBEjifhZlFmwufZyK2EQd/OENkB6u7S/xowWJySdLL17y+TM8ug7q/rIAK5u8RP7ZC0ShfszIRi1/4uZXPA/mT349IXkln6nkCZd0nFP3/7Z1NqC1detf/9e7d770BCUHMIKZDIihiCNgBCYFMJGTQJiFOFeJI6IlCRCXoSBw4cCKZZNKoKESU4AdIQCRgmiBotKMxJLRCEMUYoRUJGqTv+57zloPaa++nnno+11r1cc6pP1xO1Vpx+L3lAAAgAElEQVTP+tj7nFu//X/Wqtqlr6KLUK7BuBzzcg7IKHzpuRbjuWNeJqSupS9ckNxvkbQOq7lfbe1Xuv+dp5h5f8vx52CW0s9Fi/SzlWqOpqAtE0BjVksvR7+CsFaZTyfraUcAZ1UDzhU2YllpyVJvgVOr0+IBv70L5wtwv33n9h/8loaWb0Oag3F57+/jP7/2HGgtxQw8Ut92uvlRNt/odZm3ucC+N5geS+81Fd/hXKAn6T1rL0AypBLPP0RpfdG4MrfoeJJrpz9LDC/XwMvPuTP14KuloKVyLdaAvXTPr7apKrMO6639Tn1fFudFkpsOg/n5Mnv4hph+LpJS0Va5FufBmNdpfaRhHbHzkht9YnW1UI2sMbdpZwB70JMIVSPPIfNxlHVgVE5HgiIf2gM1by/9hNAH2Q1dxF2wBFruWul6MYfzVP9wwfzcc9h0E5b2cI6p7RLEAPR7g+8v+N7RdPwNVg4hFlhCtkUFDk/KOR1fGpPCmMdobWi9Vma5YAm0dO40hoOQtnnH2r8T2mnH79iYViwr03Y9c/drQ1Zzu/LjKCXIe+5XcuMSmPnmqwd8Wfo564C9Oi3GquvmjD8NHFtlXv26a7yeVgRwLTy32qCl9dGhb6+LaAoaTlzYAV+B6/PslqTyDUnlnmBAd61zaOouGJjDlJ5f8HTr48LK5Z3TUsqaflED1X1d75aKvoLsip4akGDyXrXc0xtV+T1YwNQ+SGllwNIdR/9kJeDSYw5mCbTauQfHKHB5uQRZrY/bTz/1vExFL4/lh2PwOFpeju0Utdw3ILvpUg5g8ehJMf38aOQ7Xc/hWu1rU9Dd09e9PiV7/fRfi14RwEDMNmZcsLVuzF2uRcEO68Ce45AAKvXB66S3zHPH/CLNy6/AYzf0/BuS9LXc+fcBTz/nLvgxhYczlcBd+pYA/8ziJcCXcaU16/s5SUUDwPU9MBS48t9Vr8dNXiEDnLpbz+GWc8nhWulpOvfo/3kJuPxYSkF76WepLAPf97e2HL7S7UjcPdMYAl8p9Uz/fjhktdQzUKCog1lqr23KKudT2yW0yzHtm6afAcxvPdK++9dywBJUJSBacZaaeRhd/13zCxiyfde96JUBXNQrlVzT50oP5MhIgySv06DLzzXYan3e0tBF9BuSLpfY9wFP3S4f0jG1mz9AQ9vEVcTT3HQT1lRfuQZ9v7/mM/l7g6l7lN63UnaBLB7PH/DRImtcr110DjyOA5aWcajSMsn18vOM8/XG4POhfWB+LD1wA5hvrvJSvcv08HXRB48r4tmZzLjlnPbNyxdPviqKOF2pXKrLlPF6zymHOeVNMNIu07aH+81rIwADOUe6Bgg7pZaLsl3Vulh+ztuVei89XVJWwCINXf6vL8E4Txt7KWpg6XSXrth2waWPKXbpgvmxtCsawHw9+NFw7lqzKWhp41UNNGl7K+WsxXK3DKUNWEyRl4bmQJXKaLkG1Ah8i6sF7I1WXhr69tN+4IbseLU1YWDuQjXHrLlfCeDerupyzsv55qupg5v7lb56MJJ+1uoiKWhpPKufKmnQyzyAg5/3cNBc9S90QwAD9RDMtos+VcuiorARK6pI5t2K8UALck7jpdTmvfz2yH/lCxqA4oBlmM4dKE9dy06X1j3Ol31E1pOl9bB5/x/j46mzqVxbD+bvbe3/AArB0g9PJUOo11LJVhoakFPRIOfRDwLaa5fcLi+3XCoHJLAE6HtW7gH3vVBupKk/fQfxgRscrD487c1agLTZSt6gxevo+TLNvYS2++SrKSjmgDW48j6ktrzcArOnKjhvuVnKg3lR8yeMrQEM6DCNut7IOrA3ltdnoF3EcUhApPVSP1pbaVqSewKLXVywL6BPxlq6YC1tvHTBPI6nzfi3IEmbsPixDtqre0xvTcJ74ONvTP9x0vfHRso9FWjwC58GYsnhepCm8/OuBREXLEGX/9TAW86jrteLi8CX/ORPu6K7nj/gYwZj+9aiDJi1uOlXYvcPyHAWy7Vbj6R7fyMOuMa9RsCZgasYx9d/tU8M1i1G26zf9tAOAI6q525oCdA1z6RuEDfbgDx16SW1OODZ8fyWJOqCASy+JeleDvrFCA8HOw39SDXzzVEcoFoKOwba5YcBfjxz25f5pqzFb5S7VG3jVSnPpJr570Iq5yDmMZYrlqBb44DpuQXdUs/BW44liAJ94au56ety09WHy5TT9kD6Ae9EOE5t48975sAtY2sOl/av9Tkr1zZfFUVcbdQBa2URiFtttX5E1QJRs/CZ24+2c7/AbgCuccFR2Ea08v3AkRgepwGal1sOGAj0cTtgLhiYb6aamkr39C4d7FR3Yec8Ze2noqfyCGgfY38A8A6f3F4yuwVK+takVmkQ5Y6X6qK0k/qgztdyx6UthPG0efM5SXUaZKUyC7Zg5xp835E+JIdL14cpfInzfboAn7xfOl+eQo6s+3oOV3OwUh+0nh4v2xrp7Nvar/i1g0+D7GQt0HqxWhuprRXXrMj9vxZgJfWabL8XfWAH3Ko1vns4KQuK0lASYDUIa2CWxpw5svnXFAKYrQXrtxTJD+Ao57SunNN2VNrTs3jKWj623Pl8HPFbk+4FkF2jVp6NiarnTmpLfL4SdOkxd7xSmeV0wc41J8vnJjlprW/gvulqOn64R+DhQgHZgZYYno3hQORxVh+eMy7ttGUbLQM0PXiDvGgqD7b02IOn5VJ78mu/rC+TNZH11513BHAEcJE0cemn9zqwI+v3JgGWlnspaK8PD8L8fOGmruDPhwZwXwvWbhuajrnTlV1qEU9LT9O5iHVaKvoTvMPkdfVNXVJ/dGf0bFPWYkYVmn2oMeqpYwU7j7hfXl7aQTjPzF069hxv+Sm53NLec720X+k+30g6+h2/3egjfHj3WOf9MG3Hm51TqGqp5+iasLW2S8dtccZ05zN1v7MHb2hOlqd7tRQylHip3HLRXmxY2fVfSS3Q3P5LH3Z2wBIcrVuSWlyptQ7M+6VjNuyG9qSliWkdSAxIHHe2HAhS3eznZXZLUhF3wYDsZq26xxSWqWcL5loqmtblPgCQndFkU1YIwldMtyhdyHFUEpx5GloCsQZdWg7WDsJY2pz4fKQ6ywFrQLYcsFbvuWMpHX075vC1djxb6eWa1PPyeAlRWqY54wjgAZDHThL3y7920Eo9S1COpJ8zKWirD+mfKR5Qm14u52veftSulQBsPcmklyLrxdHbkaS2hrwQ6Y9McjC8Pw5QbUzNAUM5X4B5fktSEd0RbbtgfU13PuV5qpjugp7a6OvIGmjpmi8/53Wzfm4Qvjw9A98Q7hGWfp/0vt9yzN9LClYOWQhl9Ly0t9Z9aT0g/86z6fCMA74KdZLj5eeSS+4B3/dL+H64PBwtd7uRHcvFFQPyuqx1a1Lknl/JGdNzsY7c9wso7rfIcr+AfD2KuNcIrLuJMiOyQYoDtkYWlK0vefD0KaIMXNEB0xeQ/cKF2p3IHlQzT8XS5iZ0zSU1kS6cYGWaA9agKkHdcsGLPh4uGJDXgou024amrmynO7W5Ltpmxqit+4CPH1C+DffJeywhnN3tzFPLvK6899rvQ3LAHoyldkDsupB1wBHwlnIO4lrXK9WRx1RKzpfCVwLsJ7d0czkGINZx18r7BGA62KltvTMu6uJ+tbJaByyZS6tv7+8xxLEa2ke+/ehIj69cFcBUrd965KWhA6BU4yPrzJ2/HUnrx3LItFyCrwfnoAsGcP+qwiK68YlvgpqmYzvdqfyyiLXWgK0xrLoPeIcLnphLDkKY6gr7O3+BOTQ0B0xBLMVY6WZeDuGYvKaQ+N+WBF8NulIZd8sWnGvgeyu31nw5UDNrtpk67dahFmcsbfqSvnJQdL+W27UgqzldLVZrx+ulMaJQFjvz0scecGthm21X78I3AjCwzv21PW9bsq5yybcp+uFN61aCabbchC89vtzulL0VESBfLlNguTBEnS5fD5aepkX7pW2lY6+ObtIq4uno2VgChHElX95QZH3nL39fqaIOOAJdWs77oG+D9Tcn/Z3RMskN8w8evcAr1b9jfbEvXKDw/eT952aPmLTg+wk+nh23g9laO65zxvT43u628YruzRDdL5XmSHm9B1Srv1ap/dSmnzMDW314AF3DUU/aEMBADpi9gF2zDlyprCv23K8EU16uxWufH2ZjPlzwvZrenkQachdc9Hwvm8OatpPWh/ktR7St90hL657jRxn/xiaWL75V3XdHb3U7UK0k15xpy6VBl9bzcskFS4C1nK1UDyFecb4ABNjx9LH+wIsSJ4FQq5McbBEfT5qf1hc/BnB/6AYguN8pcH7sgVKDnuZkI7D1AF8Fa8nG8+OjqX0z1w6XnB7ru5IrLWU168DaNyI5c+V/G7XvpuZmtfEkp6uVS07NccHPT+TqflkCMXq7kRbL09hWKlrqh6/zTrud55uvuCuWXDJuac6Pv/Epnq/A5dL4cayAg0+PumHJ+ba4XwrlyPy08wsrk6BbfmpO2AKx5YqVutia79Ltlnp6u5HkWqlr/gD56VmWu+Wg5h8ItPuBdSd8gzx/5rP0pQuWk80AkgMcRl0W8N3ZGbn9qHVQPobWX5+d1Dt95o/uQI7AuhboFsS14+DtSJm/Ac+pWrHaxVgq81wwsLgtqeyKpg+0kG43ugiOVYudynV3C8iwnD/96h3esXoJ9BKEn2+XwdmacNkdXb5L+ErS0RSEXLSOf9CR0tCai9WgK2Uzyjz4RiyQGD5HLsn90mMNxi3gBTsPwveT95PrldZ8Nfg+7gN+gNG691eD5nINOJZ6foy9bFvqxHGeL7ONV7O9Gdp9v15KOQpIr99apZyx9sQrK9YDpGf/a0Da7zamnQAM1IEzek8wv79Xu983Mo+KB3Rk3lXNqUYdMC/nsVLfvOwKSA/neC5fWXjrWEsnLx1pKZffCO8xllJbDt2n24XLugUJUJwv5I1Zk24Qfn/7FiXtfdMuKBy6vIw7YM35cvBrtyKV8yLNCUu/iogDlqAcBbHmeoHlQzhIf/xbjbQ1XwAifDn4JPhG1oR5P/4tTXbquZyLaXLtoRuRnc/ZsqgD5mVSP9q/sEZhEH7csv5bFIFm1P32044A1lTjgrW2mjJp6MCQVNbwEiij/dJ23AFDOOfxvJy74OsI7eEcAICLDFqeTp7XyeXS7UfPuMzSyFJb2u4TvLt9IFg+F3rqC7NYDcJXPONjAM+XKy6Xsns6+NQsyQHzD0meA7bKFr8nEgfoLtiT54AtJ6yVWeAFUq7Xgq98n68O3wg0I+nrTFo6k7a+g5lsvBK/79dyqGBlPBZKucU9CeAZpdtEHS8fhNZJwNbaRMaMzqNeOwN4i41WlrhDpmXacSANrf3haUDU4iL9Si5YAzTvZxbzSEXz25IALO4N1m83kp/zTKWt+drrvLKj5s74AwBpE1eBMHXND32MsrHsw7vJ/V+envH89BkuF0wpaQ5aaa0XpD7qgK24UgYs/xRpHS2T5LnfiBOOOGDucjloJRgH1nsB3OCqg02D7/xe36meAzd6S5EEagmo2gM3eJqapp6LxNuOpmDf0Wpu1ivPgjbjdKtcsZeKXuPLFyLut/8TtA7ogAHbBa+Rhj6YLNBKsfxCfFXKeV8StDH/usIi+oQsYHm7Ed3JLKWqH8CWn7DFH3PpbcR6gD7zweD2AUMk5yx4+vH02f1B/8GPXnJf3o7lSDbDirfaaH83EnRpeQa85WflWrDmfIH5/oD8ZigGOgWKdBzrnt3HPORxeL9W6vk+Lkk9m7cdlZ+Wo/XKIcRokiBvOe8q0FpPi5I6ytrwouM8epJrJRLRvL43RO/7g6OA7bgbugxbFHWwEjij/UuOiNelyx8umH5d4WwalyUYAX29F7DhJ8Hwk/IM58UY83TzFc+L2JKalh5ZWS590iYuYdLLzVkQHl/JXe0Tae/9DqiT5nXWRqzS/+ONefTvvSY+f+lYgi59TbzOAm+JrUw5081WGnytdHEkLS255PpzeR7Tr2kJdgCLjVfiFy5kHK1WbvVVDVFBqfbSJweutdPPvdwvbbf7oyiLIiCWAJd1qrU7prlD5mPzeShPxeJTjbgRz/XQMs3l8r5r4DvT4+sKueiGrPk05V3QRdbjJ7X1Y6k/DZwasLWNWaWvC55wXwPGFc+YP0ELl+kDh5qS1n7HGoglsHrQLeUgdUXRz7jSHMlrXJRz91vKOFB5eQS8t/IW+GrApPUAFvAtu6Ol9rU7nrNwBogjZ0+8AiBvvCo/OWAlaHqwlfrSFIG11zYE9E9J0B7pZ0/rfUvSBgAu8oCa2Y2cvSeYx2ThXsZErF1N914/2u/Xqvfq1PLbxYAW00dWMjZqEJXqOcAz9w9bY1kQniArg3gSWQMGyKXz9h3HsyGZG6aQoZC1QMw/WD0JPymMAflvQXLBkqS3LOJ+pZ9S+jmxCctON8ubrYAlTCVgWrcaae21dV2tLyk+sgOaul9ATj0/83t+uZuVYCuVU2ltpHrPJUuqdtB89zPvtMi6nahmZ7OnLEjbQL8hgIH6j+ySvBRypg96dZNAHyCq5EatmIg0lxsdV3PBVt0TcN+QdX3GZ7dbkfiu6MuFruna7w2t52At4HzGdbGZKrOj2epr0se3y+E0YzcFzUWm/XzFww0/K2lpfk6hyR2w5nzp76qHC+YxWkpac8KeA6bgZecFvEB7yrkXfC0XXet0p1/RA7bmvcLR1LPmcIH531oEohlQhmHaosijJ7mkTxG0jfZJROq7Zve1No+8NgawJ+/biKIP8MiModVxF83HMrbkeG41Ksm4W/1a9dx5eXVXTFdM4csagOm2HSCwoQl8I420u/mRmpuXz79zuMja0fzJDbTeOi9NQT+2hj3OpxJ6+bzg+fKMy+UJ1+fyGj67P0FrlpaWHDB9f6U4DXYcxJYLjkpzvxEn7K0Dl3ICXkB2vRxcUfiWFDLgO+Oa25asFLf3QYCmmgFl45jwwA3znl8Lwlodj5Nkud8IvKN9L8TXRzlANfX4koReQO3zyWQnAGeAmYmV2tWkoSu/ISkypYwiDtjqtyk9PX9CFlV5Qpb0tYWSqOu1djdLD9+YdlfLj5qcQOuv82r11WJrwzQtjQLiW5zodIE5eLm7pR+IJOcr/elaf1vS340EW15+Feq89DMBsZRuBtDN9bbAWd+cVeeMedn0KxHqvAduSN/1W35KYPTqeH3U/WZU1TcPsr75SOqQg7QGrLXA7/fm7eiANQBm0shrpaElBT4IULfTQ1kHnJnPwu1CgPLyMZUAu094dvh8a365HwMPByulq8vtR8WDzuumeGmTV4m37jem/a8itjb8fAUuT25+JHZrUlZSdoPWSXOQ6q20s1bPUtKW6wUkVxhL63rw5eus5VjvU4enNt6j36vaP4BFPID7/b6LB25wSa42UsfrIzCMuGutvJlDPdPP2nlUPT+RxLVzCjrqbmlcSxqau2JJmTS0MxSXx3cpLgL1GgfM69X0tP69wXddMINkBnr27UvFySzTyY/bkPQ1YaDseH2s+T4jtwZcEtLLlPTtw8jtCVrUDd/T0s9soxZ3uNbab9T90t+Z95bzt5qfZ6HLyrR1XkB2vdNL6+N8y4aqSB/UKevx+ngfiMuOjveM6+z/j5h61p54lUkLS9Jio+yy2KSNq85HSz9bAwDrp5+jsX1BvTOANbXeG6w9hIPLqm9IQ2t8jqYJJRh7v/ea8aL1V4B/Y9JiKOX+YE80/axBFsD9gi2JQ1YSXfPl55eZB1+uC0t6wmUGYghpaQpiXNkasbURC5hfiGkZ3/lc8z/Yc7/02NuABRm803FurXd6STropDRzDKaPtDMAN74Fzur5wv0q8I24Tgt4GZcadboRpdrwACv9LKn23l+pD2s+0b7qtRKAYzchT4q6YKrK5zZXzYVfAa2FOKMLyOFmO6ut5YS4tOlb8TP4LteDL9en+bcmVf4p0TVcD7KW0536mpxyca1efETFMfMNWgXMDxA/blkSQfzE1oipy5WgKwGZx2l/Z1zSr0YDrwRnxe0CuKeZAd3xAmDwksrrXG/UGXvxUThXnbNbjtRnPWtApL/fqDP24jWIS6qBuyjKBe3hFy2bpGrdckTZF7v7gzjom+ABUqJPRdrXBKjnij2od/yCBul3qTlSDZhWPL9APwl1UhsTzvJTsqz7g6Mqu50j6eQIhAscizOmG7EoUOnmLOpouRuW5zwH8aPs1pqlpsutS4DgiimMI19D6GVLNGkpZ17PgXsro9AFluDla7zT8Ry8pYwDax5bD98CW0B+4IYGX2tuveEbuuVIAinYuQdiCG2sPqxyTVqc2s6apBZrPQkrKu3Wo4j7jYxVB/2NUtAReGWdcC9gSlaynGsbtCp3Q0ecsNamKOKCsmlrCgFTy6dkSd+alBWFpuekabq5QFB6AMdyDMxTxrOy+aYxLupyOZgfpU/3Pp5Jf3cQPz/j+XqZwfhyA6sK46kz2wW33obEz4X0MiBDd/opb66apvg49tLNFiSn2HlamkLb/65fH75R4EcfxOHCl95yRH+fnmvV+JVpYzldrTzigKucsbdW6+12jt77a43Xqvo+N1wDLpOs2aFclIW0119NLL8KOvOJOhTtj9ZLZ3vOuGZM3v/s88kjFf2R8rzoKBQk6D07jUubAsVIm0NInOJ8Vf1KDgbvIhb6wKS00+ZF6iLgneplZ/s4lx0uj/du6aH9lnMAszJtJ7IFV6k/b0e2JWn3M4Xv/U3lqWfAZ0jEHduTW55XQbNG2fRz2FIztW6+4vLGbQP6hgAusiAoAS26Y9nqQ3O5kc1alV/QYDlUSdpvQmN8xAFb4/GxNbAvLvT6/cEzBZhI13+ni5u/Zvt4tOTkoFa5z/emR5pZ/qDAN2nRtHQ5LrvDZ6lpPM2fMV3gVlLUzBkDxB0DtvulU7V+BwJsgQdwgTh0y08O3sexD16tvDYtnb1vuCbVPf0qloCmG64W9/ve32gn9WxBVqvP9KXBPuJ0tTauNPtOlXnucqub5XOIfCmDFlunHQAM5CEcqY/cPhSVB3HpOPg9wRZoJWXXhjOyUpum+kGYPhkrssb70Pz5zROUC8wnyBVQz8d7gJOCm7aRdkA/EtXLFLSelr5CSk3ff97S0wAWKWr6VYgFyFMcFn9DM7es/H2NQjlfPdCAO8XK67tAgZC1/qs5ZN+d9oRv+ZKGVvhqr2H2WrVNVwDcp11FQHr/xQjnnmvWxrYUnYt7/dDSy5Zz9YCrfZqwxs7UtcTa2gnAQJ90cMu4miuWYitdcOm2yHKmETB7zjazPqz1EZYM4Su/X9iBML/HNwdhro/vl8t52RKSwHxdWHK5wBzMVjl3xLN7hQmIS+pchDFuzvgGYwAzh0yhPJ2Xg/g79Hzl5x+x84fLLXMvr4/+fGLnGjgf740NXl4eTR979+jSslJufX1hdGzVzbubrkjquSgCxyycPZcrKet+EagHsPziBc8J116YMrczdXlhzdoRwED9QzVofXQzVsQ5e/GaCwZCX1NoORXLAUfa0/ro+nCt7v0vn5S12JQFhCBM3SaF8DMuoU1WrXqkj+ewpmCe75R+wFVyvzyellswBoDnyxRXgAxgAWVgCdOp7vFxiMN19noFd1teD31P6E8JuuU8C95Sv7brXS9GeA2RTVdS6vn+iwier/VPU9T5NrtfqYy7X88NR8bO1LXE+toZwEDc3WZdsAVQ6xakmnGdDwyWI4VSJ8WsJQnMEsRFsA/3cvVJWUB6t+7cCU8uljvjZ6HME3WqWrqZanGfL+lHA3GJLmnoefkzaftYKy6QkoBcHPJ0/LR46MnjyyHmYJ29DuFBKXyzEAduef3zOmvtV9401Qpe3seW8P1A0tYlhj8IpAm+rW5WKqu9ZmTAHB7HevKV5FSzk7eA3Nv99t9BfQAAA/HNVFp9r7VeDdB8DOqSYY/d6oJrfkMe8CPlVhmfm/D9waIqIEyhJz3xSr7Xl67nLl2tJ+qGl9CdA1sDMa2j5XPX+wDs9FMHcomjcLz3fYn/kdD2UpkG3HnZHIylbOmUdfDSNlJZBJBlrB7w5aAtZXzu9N7i+2tcA75RQHvQXsMBS3WmntD3uc/ZW48sRaG6xu1LqwG45Pz34HvjQzRmkoCsva5S7nxNYXa3s9WXJisFrbVtSk/PU9EfXZX11OfLAsIUVo+pLL/IgZbRW084ICXxNPJrEQV8pg2VdGvPdHwVyuQ0c/mpQVmCJW2T3QlN597L+fL58vdFur2oCr6PycfAZpVb/dWoxf2mxvRg5t37m5V1exOXVJ8d/wkHeBJWmUh0mIgLzt4XLNVzqGrn3hwbb0nS4KjVc3kvW4rV+uyyNhzcGQ3Mv9xeeCHS1wjSZzfP14TLxivL5T5iNGD5qeZlWwrA5Xqvn4KWXTF3vI/Us+R+az9QSLDldZKj9eo5SEt9DXil2Ihj1m4RagG0eF7jfAH5/xp3tLxOA6JX3uqAvTpT0c1Xtelnq30Emj1dbd3cN7KoGRBLbbMA7+GCI0CW5mc8GYv+jjzgWpusPEVTzzXzMlUD4Y9VMNIvY5hLbxMVv+1I6oumkx9lc9BT2AIPENPUNG2j7oImoAX01DNNaVNpa9j0tSzLLuL53P350H28L8uYyG1JWjrZSlfHHO1URp9epfWlQbsLfIs4ZCUwr/3PUtSRS87bVObWI9oxjdHOM+Ot4X7bnItLtmEY3gP4RQDvbvH/cBzHv1I3nEeCCCR7bMbSYqKbsbT46IcF1gRKs55Q9uSlrSWJY9c54a1EwSjVSTug+ZryI56v986dLR2v1E9lSxgD9DuO531I68Elns49+vqpuIP2Us9SjARRXp8Br1QmxdQ42tJOuhWpvDZaxu8dDsN3/iavB1lpHOuakIFzrTMGUL/5qufu5ky77eELxGjxAcAPjuP4O8MwfA7AvxyG4Z+N4/iv64e1QJXdkEXrsrckeRWupIcAACAASURBVLdBSS5YKiu/COP50FI4lDJep9XzGE+9YO1qXwg/PC2F6eNcUoGh3Jd+K9Iz65eOxEHL5yeBVkpFA8t0dImXVMay0tNa2pkeR5ywBl6vXAOj52Y9YLaknAGIz3qmryPsfKdG27pWrbxnX1KbhUZS2cP98jbhiSTqs+rTn3v5HsdxBPA7t9PP3f5lvm9QUYVbvOsILlgrM8bS3K7WRAK1VBdV2s0G+lPbrQ9hnkbO7HSegzS621leHy7HEnAlR0shO738+Zpv6aPEln4eMfM3jLv5SLpZKreAS48lR8whyvvQ3C1t02t3dNRFR/qq3u08TaAPaHtCu9VNR2G+aCQd87LsbUPerUfRnddZ99sP5iECDsNwAfDLAH4/gJ8ex/GXhJgvAfjSdPa7G6eV3ZCluWBLWRfszY+XB74lSQKyBV2tjdV3VJH14Wi7e5vY7mhAhoYnKY08De8TXdp9XcRTvVrZS5I0dy39TOOjqWgvBS251lLnrc9q5bRP6bVl4au/dw585Qm0OdC1QOuN36Pcr7wp8nCM3s61VX3nE7rqjeP4DOALwzB8C4B/MgzD94zj+Gss5ssAvgwAw/CdQYfc4oIjWuOWJK1MO1a6AmTwZjZnWX8LvdaLa8A8S6sfb01Ycri0TnoOtPbFC8DDTdNyPwW9TGvz1DM/ntrO09Clfe716+lnei65Y2tTVmZt2AJvGac25RyN81LdobQzsHS/HjB7QdbqKxNv1UvlprRvPco+9zlTLzlobfNVi/vt/2EgRb9xHH97GIavAPgigF9zwoPSYNVrLTjS1lsbltaWI7chJR5PGXXBnkPmMVQ9U9Bm+pnHrAdhDlO+UWoZv9zZDDwgyNsu4Sk/8YrG8tQ0rQe0tPL8liTedgliOw0tvU9emQVcWh9xwp7jpeVaypjGe7Gaa86knelrCK/5Aj58qXq6WatdTaxWl4lfNNKOrbKI1nkwxpbwBWK7oL8VwKc3+H4TgB8C8Nf7TiMD4Uy9FlN7i1HEBSPWNwevBlbNAWfdb7ZNr7+3FSGsbZySVGDmgWoeL9/n68XyMXk9hfgybr7ZyoI2HysqHiut/fI4C7rlpxSTAS/tpzU1TfuNwFeEdit8H29w3OnWuOMMnDPzlMpNZdxv5rnPljwHm3W/0TH6KOKAvw3A372tA38E4GfHcfw5u8mIdTZKZeJ6uWCpjeeCAw/miHAdiXIphiviWmms1Vc0Zjbu9uloKX38qFveXiTDUd6kRZ2rVs4d7fTSlulk7Zakx7G/A1r6MLLnTujyM7P2a7nhls1Wkbj0hivAhm8UoHDiLXltWhywNUdzQt6xVSYpcqtSD0fcy1V/im5PwhrH8VcBfG/9RIC2rxLMuOAaJ50FuueCpXLnwRyWC651wBaErblk+srM6a4+EKbpWanO2hEtuWHL4Ur98jno9/7On4ZF+5peon7vb3YHdGQjm5eK9txwJg2dcbzl2Esj0zY1KWctjrpeAO23Gnl1cGJq+rL6tcaT6qS+TK3lfrO7kaMP3uB1rannOnivuQOKKAriKAxbxB1urbzd08FpWM08rkf7puWaeqw1WyCegXwJ4QvZIR35PmFrVzJ3mJG2NN2rSQJgRNTBAnO4bi1p/pkUdCQVrUG55fakFvjy12+tAQMB+D7euHbQZWFq1UXirP4jZaasAVrV2kcNINeFL7AZgIsi0JPo4d1r2/uWJO9cmxuNByl3bkmiU+7hgjXXmlnnTbnaGslO+PnpisuVDVq1MUt2s1pKWnLUtIw7ZKmOlvMUtJRS5qnnZUz/HdCP1+ann/lx7U7oKHhpnbUuXPqKxHpp7K7O1yrX6no4ZknR9lIdL7POF6rZzZy5yEjtswCMut/oXOq1MYCBPmlpIO+Wa29JstaSI+lvY57l7+DKjr3mViwCdVJMDWgt0If6q/0CB3nTU604fCM7nS+sbtlGT0Fr9XTDlQ3i3A5oqY1WbgGX1tekoKX66EatKFA90NI58M1WANoesiGVa7ERsHrtI9Dn9VzeBwUI9QtpX7oQvfWIn0vtI4puvorIu4D1WS/eAcBF2XXZ2rVgzaFqpNNirPpGFywNVSTBzHO4NVCs6dNrF5o3gfDTBTAe1oHLfK2VywKz5Hx5meWcaT3wADEv15xtqZtehv8NSHyzFU+bzz8w5Fwwj1+maXXg0vio2y0/LcdLyzXw0njrfuHIzmltpzOA7eBbC2ar3msX6YuXSceiJChJIJQ6yjya0oOrNVFel22vtanTjgAG2iEsxfScQ7Y9hzRYefAZ0dbasBZD5aWupTqrXaTO/c9p6QFhQHbDz08XXK7P01qxwJvM/b3ypqh5e83dPursbz/S0tC0veZ8I+nn6C5oLZb2SV+zdBzZCa253fIzsi7MYb3G/cKRnc4A7O/zbQEwnD68OqtPXscVjffOZ8q6XwumkXoqz2Fb5bXq29/OAAbaAaj1RckVvSWJAzR6bo0BoVyQBGJgCTdrrZgPG4G0VZeBbcapS3oabuPm1oW13cuaG+bpZl5mwVZPJVs7oO1nQWvumcY9XvJxdkHTWG1HM4+xUs28fc2GLKnMcr2AkXIGlvDNwFaKscq3aiPNE6Scn5v/f72dz09KPVcEalHwWRPmdVn32//hHwcAMJDbUdzqgrNrwZH+O7lgDZyWA44CzoJhdi131Q1a83Xhz54u6jOkn3HB5RL7Hl/gAWvaXrutiMdrII6knzUY1977K93OlJXkirPAleIst8v7yIKXxoYcbsD1AkbKeRpgGwDXlEtjRsqtOG0MVV5gb/fLY/ixNnZE28IXOAyALWUBm2lr1XOAZndES+XGeFHQWulqqR+rnP69WWvQvEyT5c61DxdiHElJ3+A7uzXppsv1edq9GmQQv9VIcpQvQdY6eKaP+bm8GctKP9NjbX23/PTccuae4RBoHfje5y8536IM3CKAjvTJpf2as9CV5uBpE/cbGawH/CLud83xZR0IwJlUtOWCPQhqMQFIitLIdjXqnQ1ZgA+uzKYtrdxbK/ZA3LL2677N/uassi4MwIUwT0lbtxXxcynFTI8jO6AtVzxNX38EJU9Dl7GptI1jUqxVl30QB20jrf9G1oY1YLfeLxy5xQgQ1nunQWJuNFJW266mHEI5L9NiurlfyaVq7tbbCa2Nx+N5fS/3ux58gdUAzBfmo8O0pKK1Og/OERfMXS8/p7HaLuikC9ZAXMpqXWYUzlq5NHabGXto8dYMwNPngOv0aTtyq5K0mar2e32XdfpaL63XYWunnqUPA9bmq5pUtJd65n1Jx5H7gbX6iFO2XHJkN3Q65QwgtNmqpizaJts/L+dlMOIioJX6uSvy1CurjMu6gNT2afXP2/eAL++j06Mo+0gig6baTVkWSKPjtLpgrU8I9cHbkjQQZ1yq166U9wJpdy3vF565X17GNmhZ0twtreP3/vKvGSztcs537nYz9/5yONdIArYG3uwuaKk+sjacSU+7O6QV11vKXNfLzzV4ZWOkNh4Ys2W1oDWhy6V1Wo6P6n57O9q2i+bGKegoiKNOdU0XzKHqnUtjc5JaaWknLAtiD7AtKWXPBdfA3P3Mo6ekyy7p65W5WrZBiwJ2mYJegpg7U+uJV/OyvPNdK/1MX3uk3EtB0+MIdLW4KHijbWZlbK03dIvR1HE7TNdoky2Dc+6B2QQxzW5q7vdToUwaiJ5HnW7PjVVWfatzj2mnNWD3alvZLttvdke0NR4HcsPDOShcLfBaQG0BcwSmuzjmZUpa2iXN14atW4v4ubfeCzxATEGrueIW50sByDeQteyEju6A5scRJxyFbqnznHIqBa3cXlTO3VuMAPs8C9jW9tl++Nje3LS5qpJSzxrFeZnmdiPxUr9rul8vrt+Fb8dNWB4soyDccy04Mget38CHBc53YMl7GPUQYnq1KXFeHy0fGCRdMV04S4Pr83RhNXS5zGH2GlS7E1p6HzjAo9ClxxFXHL1v2HtKltSf5nqBTvCFE8PrvfZef1Ybb1yrjRUvHS8kBda6X97GG28r99u7ja6dr0oBCKXbZPusXQu2IN7JBbc64BDMjPNIm2w9lQR371d3j7lB+OkKXJ/UDVr8CVryE6/8FLO31jtvk0k9Lx0vT0OXvou0VHRGUlst/Uzj+U/aLrM+HAFz6OlZ0bVeAKldzjxGapOJt2Ij8ZHxW88laN+lbbyiHfF6CdBSvdSHpb3cb1/4ArsDGMhDLhJH+1xrLVirl+YkxdDNWQ6EyzHYuZdmrklLZ6G8Syr6JuHpWTQlzZ+gRTdpWWlpafOWt9ZLU9DTjJagBqKpZz0NXebH5W02q3kcJT/nQKXts9DlMdEU9L1cAO90Hry9iB73gGambY/z2v6ltmDHqqQGkQdiSPHauVemzSdTp9VvC1/gEAAG8hC24mvG8mCpydtsxfuW5lCOA+vB/DxSF63fG7rWZx8rBoDkhvna8NPT5b5JS9stPcFSd77AA8R8rXeqk9eSCxwzm64oiKXzUlaUSbFL7rdHGjqzKSu7Nqztbgagb7ICYN5eRI9bz6N1Pcesga1XJ6rmoRsSoCPy4rV6a804ctvR9vAFDgNgoB3Cazycg/flpaattWKpLLHhS4KtBVpvTTUD2T1dbpH3mYu6YZaWppuyliC+3NeIrXt/+a1IUopZcrK5+3/lb0EqcyptilpS0Fr7ml3QNDYKXamfjOMFUJduts55XS1kawGcmWstbKU4XjfTSAI+VY4BGYCe243edhRNeUs6LnyB1QA8ou5+3t4QrhmH1lsAjZ5DKaPxQDoVXZQBreV0Jci2tG2V9FaF6sv7d0tLK0/RoiC+i60R81TzNJT9kI3lGnLdLujH+TwFXZN+LvPWFH0Qx9obsqS6MHgBhNLN9HgtyNLz3v31nBePW4jCF8axtfEKrM66UHi3HW1za1B7X5/iIA/ioG9Yza09rZIgyOU50podzlJ7aROWRJHgrUne2nAUlkdzuhFZcL6fs1uWni747PqMj67PoiO+0HICYmtjlnY7EhD5zl87FV3GK/H0nJZN5bn/L5EUNAd2BLilnfYISw5w0yWzW4qAAHgBqPf1Qjleo27N/loBK7U3xSHLj3kZj9cGiThoKZ7Xr5V6zlwIs2n2hzZMQZdJtjjUVhfce0MW75fXW4DWdkg7vxINPhqgM23QWJc53qIOwH19+PoEPF0Wu6Uv16f7Bp6F2jK81eI7teVNYcuvVYz1Lb8oDbi8TWbtl8fwPsUU9PMcuuYGKwBp10uPt4QvyDmSdV4chHOpjQZoUZKDkxp6rpWeZ6Cm9X00tc1xhzXgXmniqLL9RF1wdD2YHnup6OCtSfRYgmMUWN0g55RHFP3QEK2blRM3rKwP0x3TjzLdEUduOdJuV8o8epKuDxdZu6Gj0lyzBlt6Hln7pXGhFLSSap7KjA1WAEKbrLTjaF20Ta/jNdpIr8UUhawEXJ561j4NcGXWj7V6Xq6NYc2lxf32+XCwA4CBnBuW2u7hgrMbsHgfHMJam3LspKKLNDi2ulXpXFJLTM1nrDR86TlZH3664rPrE8qDPPitSyU1Te8jfsZ8w1YNjIHYNx9ZaWfJxVpu2Nqs5aWjM6loDlVaJ6agBbcLYJFqBpSdzdOgj5+9odkC0bXm1LO9Ku9xk9p/Zikmsys6Aj4+F6nty4AvsBuAizw37MHOitVAm52X1Ta7Icvb1AWEUtEahKS/nYhbjawDZ91tr/Xk6K9Oi5PeK2XHdAGx9UUP958VMJ5GtHdBT+XyYye1NWBe5ymShs5uzErdnuS43VI3SzMDNnjp8dYglI6z7daYM5xyUdHHTVobryL/8TVA8z69teHoeBF5/fRPie8MYKAvhC3RfiIuuCbtrI3HZfXD6wLfG8yPNffbmkqOgLqHIjDNltPXfy8TQEx2TfP0NHfFGRgDgJeKfsSsm34u6vFYShpjpqizbhfAYnPV1Hkb6LY6XrO/KGSfhBjebibtliNAhqLnbrMbs7jWSD3XOt911qMPAGDAT0lHIczjrPreG7L4OZ8Dr7dS0YH14DIFIJdetlLWGRhrMWuCmcqDrgRfrX0BMds1TdPTLTAGHs7XSkVP01o65RIL5NPPRdk0dOZ+YBo/izWgO50/1nYBY313GqAP7LT6KOx6ATjTV+8PAqKsW44sl6rFRODrud+IrLbRC9E+8AUOA+CItCvumrJSxlKcB2WpDy2G1hkQpmH02IMwnPJMf/wYRn1tX9GYaKzannzRw03Sc6bN3dP3oGWR9h3F3q5m67uNpYd0SPVW3/P4XPqZtuHQpcdSmhkION7yMwuZVmAdDb5IHEfrxWBr3TcCox4xNbcdtY7Zq01cBwNwNq0steGgbnXBtanoyGuxwMzrEl/a0Ao6SWu7XO0DQDYmGkvfIy01zXdOs/Q0ANcV32Pw+H5iyR1P5ctd0eW8tKPi34YUTUtHdkDzc9URPy9BS+Ebgi4A9T5e/rMVvF59z357jkXLWuYlStp0JTXwUs+8TaRPCa41kF3D/a4LX2A1AJdfaE33FriifXoQjrTL7oqO7oLm55bzDa4594DwHmnkHpLeNg2+FnR5f/efenq6HGspagAmkIH5VyV6j6GcYrZ7FOUUw9yw4XCBJXBpvQndafDHzyyErbq9YNur715zEmVtuuqZeqaqTT1HwWx9ePDiMvOxxj/Ek7DoC8wMlYVwdN1Wqss4Va/fllR06RtOG2M9uBbCMOoz4Jb66aUMaKNteJwWcz9n9xMDM2cs7aLm7ngqo/UylIH45qvej6IE5s6WzpMfP83KFZcLxKDr/TwiBLcYu0cbUZFNVxJINbhGnK4FZG0cq1zrM1PfGl9/sdswBZ2FcQ8IW/UahLXybCo6CuHSD4SxJFA7m7Jq3G0vYB7BOXsg5mUWfDUQ33dPA1qausCHu2MAC4cM8JQ2cbxXefNVgfRUl/tvzOF6L3/SoQvIsOVxC5cLtEGX/2yB35ptesxvrTmLkjZdFWnwze4gjkBcirf6tNpk2nlzbukjrg0BTCVdESW1pqN5jNVf7a5oz9VK7SywW+DG7bgRwmB1Wtke6WkJgBYUvTKrP+tPyAP07P0Q1owBE8gAFlAGoIK5xBdxONZK6odvLtNgCwSBC7RBl/+kf38RYFl1RwRwz7mJ4vD9lB3zOqvcO5dEJ+fBnddJbaU6LUaL8+YQaZ/XTgAG5lcwS5mNWZFUdDSdLPXrtbXgKb0GXh9JWQPNEO4N2T2cbxS6NXXRn2IfujsGCLAUKANzME/nczhTLb7RyZC1a5uO9yhTYAvIwAWwcLn02PtZjlug3Bt82ba1Y6/RhygLvhDq6M8a+GoxPE4aX+srqmPDF9gVwEURCHoAi8RGFIGzlYqW5EE5CmGpz0YI87je8TwGLFZSD/eerVtV869HlCTd5kQlPZnrUfd4hrXdh/1/THLCn/EyD7qADtXIT15WA+a1QFwbL/3s0adXJkqCL5W3PkvPI/+Jov/Rog5Xa9N7Pmu1X+oAAAbWhzCPsUBbm4rmZZE1Yw/CtB2UNgkIF0UBuLcifxZSvPVr0WKi5VY/5oeS4VEGgO6qpilrYO6SiygMP7ouH5OZ1QKuRQvosjdSgm3kuAXMLbDbA7JrzClTJ0qDr3cOVs615q7nyPiaMv17setdEA8CYCB/td1qbA/wEnA1wFr9av0Ayyt8JYRr09KtjrIV6hGgWu2yv15eHoEwhLG0+vuxAGVgCeYiDmhJFM5RMEvu+In9PWmA1eqyZT1+rgnlzBj8dW45Z1G18O2ZeqbK7nq22vN2WlvvA4Sm9eALHArAwBI4XK0umPYd3RWt9VkLYcndSm2AJYgt91wB4aItYdqiDEgj0I7+tMaLAJfXQ4gBPR/m5/cx2d/kVbjX0Ek1TzHOo02tMg3E3nELfMvxWnBeC5iZNj3mIGor+FJFgRyBrwfXlwtf4HAALrKufD3Xg6MQ1sqj68HR1LM0P5BYfkzPAxAuigBXBIMSG20T6cP61UdiNRB7cK2BslXG6yEca/Pl9WBl91jj8aRRRaDLyyJQ7lHWAjt+3guqR52bqC3hmwXy23a+RSsBeETbZihgPQjzGGueGQhL9RYhMlC2nDcQhjBtUuSlmy3HHD3vLQ2e2nmmLyumBcLSMSC/dwiUQaj3FPnw48VrIPaOI6C16tb4uRY0a+dRO4aoveEb3axl1dfWeVoTvqXvQzwJi77QFhhrfffuE8hdvSNtJaAWZZ2xVp5IR/NpanCxfgI6SDQYe+UZV17zoeGtK/J+7AVgqcyLiZavDegeP1vbLtQbvpa8NV6pTOs3k3qOjCm9SWvAN7uh7KENU9A1MOZXe6nPyM7krAum9WumoqV5axCOwjoBYUtZkHrte4i/bS2flaz++U8rxioD5u+D9wFGOqdlEOqk+qi0dhnwWnUWaKT6HkBqAS0/P+JP93e9Bny9tpmymtQzV+QPvuY/RbZNPXiLNgQwFd9k5Mm60q4FYa2tB2Gp3irLbNxaAcJbu8ke/Wkgbv0pjZEBrgZYLwVtnYOVS3VSjBcbibNAy89rj1ugq9Vp9UcBbev4qlrhK6kVvlRR+EbrMjFSXKRNtp+cdgJwUSaNbEE4Gp+BcLTO+wAQhTCEsloIU10RhjDXlu52b2Vg7JV5xwic0zKrnM8zKivWg3EvEGcgbNUd9edafYrq5Xwjbam8siyssheWLeHbD7xFOwMYyLthrY/opqxMX1kIe/VeGZ+3BWFLUlvA3SHdax33CJLm5bnfTB0vQ+IYgXNaZpX3kNZfbxDX1O8BuSOANgxeYHv4Zsv4sRbDX6wH1ygQW+HbH7xFBwBwUcQNW0CNumnJBcPo1xpTqrMAzsta0tFWe+28IiVdoy1BbEEz29aK0eJrjhE4p2W8nNdl5f1ueoGXnnvwlWJbAFfTZosxatuI8sArldWs+VIdGb7SG3Zc+AKHAjCwDoQ9SEb6kaBptbFAqs1JiuN9WzFcnSAslUV+orJNVi19Zsf34o6UAbBkzVGrq4Fu5LgVwFbdlnBeM1aU91xnWqbBV1ItkLWymk1XNdDL/sfbH77A4QAMtEM4Gr/Vpiyvned6IZTxdDR/OAeU2ASEqVrgtpesPxEN2FKdV8br4cRo8+NtaRkv53WtikJXKmsBcY+ynnBdu002VlU05SyVZVxyaxmE+h51rannY8AXOCSAgTYIa2vKEWhbEI7WWSlirV0NmDmEodTRc6oynw63Klmw7eFyLVkwreknU1ZzDOOcllnlvC4r7/eQBS8/73EcgVZt3VZtatubOjJ8uTSIZVLPkT4zkD4OfIHDAhjQQUqVvdpaEI3EtEIYqAOuVMbB6u2Q5mOXGMcN06GjsK3VVpCW6rQyqW1v8Eagu6YDtvrzwMvLIsf0/Egg7hXbUqfKSjlrMKZla7lcaywrhpd77aR6KUaL02KjbdfRgQFc5LlhDcJauwiErb4s6HsQ9vrskaKuTUkDVWlpSUdJP1twjcJYq+8JXi8VTculuhpZv59aGG8J32hZb1ivVaeKPtIwAloJklvCl+ulwHdb8Ba9AAAD/SHsxWWuxNG1Yq9tLYSlMi8lLSnhhnkzCjWpziuTprIlwCWA1sDWAu2W67+175/UJlK2Noil2FoQb1UXjTfFXS9Ql3KOxktlEYBKsLdiuDz4SsrA19M+8AVeDICBvhCWYj0IW/UchhDqJChKbS248vZemZWu1sAcWBumL5OrJ4i98VtWH3hZpj5yHKlDoIyWS3WSou+rFafV9YavdlxTvzWcW+JN1bpeWpaNj5ZZ5ZEY/gas7Xy1+Ei79fWCAAzEHW1Ea0I4O34Uwt78rdckjWeNFXDDNJz+RGMZOtVlyjL1XjtrzEz9FspCOANgft7juEfZ3vGmesIXQp1UpsFSUgt899De/8FsvTAAAzboJABZbbR4KybjhL314AiEgdjDOqS4UmatGVt9BNaGyzSoekExoujnEK+t10/kWKpD4hxOOa2jiv4vjryvWkwrjI8I4payaDw/VmWBN1PW2+W2OF8uXue1e53rvlQvEMBAPYRhtLP69iCste8BYVpuwZLHAXPQRtPP/I+yAcRHkgfbSGwNhKPnRdF1YD7fFmWumZGynmB+SVC2jk1F1nq9shp4rg3f2jpen43TYiPtttULBTBQB2EtFvABa0E4WtcTwrysvAbPNfM2YPGADudEWro04cNstUa8hVohDCz/TL1yKPVZee+tVp+BKz9vATE9XqO+F2xTf7MWeGl5i+v1+mmJjaadLfhKWhO+xwBv0QsGMLBuOvroEAbstWLLNXspa62sjAk0g7hGNf14n8U0iEaOM3XlHE6MFSvV95TVp1TXC8Za3JaA7tmnq5Z0My3fwuGuDd8InF8nfIEXD2Dg9UEYiIGVlmug9ByytzYstSljBtPS9GVRvQSHS9ULwlpMkeV81/jf6v0OIuCVynrA+EgAjsS6ksAL1K3RrpFyjpb3WPON1Gv9vw74Aq8CwEBfCNfGaeNlIayNnXHImT4iQLcceALEdAjujjnApHpelqnXFP0gYPWZGc+Ksdqu+YElcw21yqOQra1bA7w9+gop6nppuQfADDi3gK8FuV7wfV16JQAG+kFYiuVxnlPWxuN10XQ0ELt/mJZn3LTloKkst13hiHvBsUXWryrahn+AgFGvxVjltE6rzyryvvaCr3ceOabnkfKeMLbGdhVd59XKs2nfKNxryqPwXcv5arFWvNVmf70iAHvaE8LROg22NE4CpARnXu5BNvKwEGvsUt4I4qOmprOgtiDt9WmBmNZTeevcGWXAK5XXwteqW/O45sOAKS/VXFPe28me8D2CXhmAvXTx3hAGfJdL++FX4ozr1cqjIK5p1wjiiNYGNHezLWu9Vt+0DEpbD8RSbIusPiKuVyp7aTCmx9VulzdeA3xrut7oHDJ1kXopxoq14q02x9ErAzCwhBnXnhDm9VZdy/ovB2TNJq8sKxPMhwAAEv9JREFUcDuAmE/Rc8V7p6g9CGddb8Tx9v4f671/GTfslb1EGIe0N3jXKs/EZeEr6W3BF3iVAC468ppwDwgDNgRpm0i55Ya9GKmci77Gzq64VRH32rNvC8Jw5tIK4+j7+tLdMD2PlDdBl3dwBPD2nIcV540r1UdjpDgv3mpzPL1iAAMvC8LAHLDaurAWF3XJUl+9Qez1W+mKqfZaK+7lejPO2poLl7TZK6st4OvF9ARwpo0rze0C6wAvCsIad7sHfM+0M9UrB3BPrQ1hXh/98JCFcE0by5UDtsu22iRBTLvlrKfDPbFy6zhTV3OeKYvUSbFUXru1HLBU/lIgHFY21WzVZV1vdMzWvrgyfwg1aWdNrx++wJsAcC8XrMVvBeEawGXcs9dGcsNenFQu1TU44r2ccKt6OOAS21s94CuV9TzfxO0CMejW1q2VPu4xn0wfkXopJhOnxXptjq03AGDg5UMY0B2oFpcFdG2bmrVgK21d1JCeLl1tAWb+q5T+PLQ/MQ/CMOrX0lrwlcq2BnBKreC1ANUK3tpxrf5O+O6hNwJgYAkOKgvCUptaCNO+WiDd6mx5/zWuNQri2v7pBXBFGK8B6l4QLvVwYnrIew+yYD4KgFOqWd/N1K0J3pa66Lxq6qWYTJwW67V5GXpDAPZkXQklN1wDYR5jgUqrp/3XuuFSFwWlVRcBsRSbqatcK34p8iAcjWkZvzamFr5SWU8ghxXdzdxS1wPeawC7Fa418NV+UW8PvsCbBHBNOlpr1wPCUkx0XZifR93wGnXczUbcc7auwhXTaZZujgjnPSAcfR/WcL9SWet5Smu73V79tIx/wvfoeoMABuohHI3fAsLA3HVGgZ2po2N4dTD6bYF9GYfXVbhiaQgNzl69FcPLpLY8XmqTiemhPdLQUtlq4LXcLrAOeNcaZ6v51dRLMZreLnyBNwtgoA7CWps9IMzra4FpOU7e1qrz+q1xvbzeSk8DTSnqvZzx3unn0n9LXO/0dFenC2yTZm7pi9e3OFur7Vau9lzzjeoNAxjYB8JAPWQj9dk1Zq2vCIhLfcQRg8RE+uL1EVBXwphOWdPeqes1IXw0AFt9hhVNMXv1e7rJI82zpo9MX1as1+bl6o0DGKiHMIR2EQhLcRHI0vE8sPYCbbbeAyRvX+O+o2M1wBjwAbA3kHsp8xqyKWitvLvDLap1utl6z+FFHGCt462p791fNEaKy8Z6bV62TgADqF8TjsA1GueBT+qntxves74FtivDGGiDxJFhfQQAV8tb0wX6gm7ttdO9wVszZrQfK9aKt9q8fJ0AvusIEJbiIm4ZyLlhK36t+hLjwbMVthvAGJCvFzWgPQKcW1PQVl3315bZSKVNwIIyr+8Bqd7p3y3mXBujxWmxVrzV5nXoBPBMR4YwoENP6kdyw7TNViBu6SOaoub1Jcaqp/0UNQAZQvdliB4QWhPUa7vgJnHgSgOtsX65hXtco4+11nJP+K6hE8AL1UI4Gt8Ca66sW65p0+rAtT5oTKQPKZ1utYnOi8Z0csdUGpSj9VbcmoqMt+qcjgremj4jMT366AXWSMwJ3x46ASyqBsJamzWdcCTGg1mkjec8M320jJMZt8RYrliKKXEcAJ2ADGE4PvTeKemiTecRAS6wHiB7p4l79pMFb6RNbUwmTou14q02r08ngFXtBWHAd7Geo4zE9HCqkZge0LTmL7Up7WpioMStCGSqVujxP7OjwHyhWuBG49aAbm3MmmNtvZZ7wrenwgAehuEC4KsA/vs4jj+63pSOJA/CgAxWCO0i8KKxEcBK/WVjom3o+JkYGtcjRgJmDeS9vr15bgTkrA4JXAm2QPwCL8XWAmNvt9grZg9He8K3tzIO+CcAfA3AN680l4Mqshabadd7XbinG6ZlvUAsxbUC04rJuGmpbxobSVeXWA0yBwHzZsrAFshdqPeGrhS3pmvX+tra9faIteKtNq9bIQAPw/B5AD8C4K8B+POrzuiQqnHCVru1U9JSXBSWLY6YxkUcpxSXjSlxEixr2vWIpW00IAEvE87W6wHqLrKtF/0IpKS2PUHZu79e4F0jLhurxUfavW59FIz7KQA/CeAzLWAYhi8Nw/DVYRi+Cvy/LpM7lrb4I4mO8akQu7Vj2CLuSYhreU2R/mtjtXlp7YAJZvTfERWZo/c6vfcnEq/9LdT+nqOAjv4/q53va4GvpRO+mlwHPAzDjwL4+jiOvzwMwx/V4sZx/DKAL09tfu9RryaN8pyw5oIhtLPWkGtja1PCUlxtWprG0disI65NYXtxkbSy1F6LteK1+Wnta/7bZFx0y3/LyEW3xgH1cMV7fGDcInXe2rb3nLXYmnirzdtRJAX9AwB+bBiGHwbwHsA3D8PwM+M4/vi6UzuqaiBstYumozOxWhywBFukv9qUM43tuU7cGkdjvfZWH6WNlYK2LjIROEt9UrV+1s26mchFc+00tFa+lyvcK4W+RpwW2zPeavO2NIxj/D/wzQH/RW8X9OSAv9Q4taPL25ilgVhrl4lvjW2J69G+95zWeD+scu9za+3fRk1fPZW5KG7tiLdynNF5rbVZac8PAj1ivTZeu9eiL2Mcf8tNT533AVdLSy0XWSnpqLvV4lvT19m0NI9tbR+NbXG8tX3S2Izj1cblffI5SPL62kIZZ+zNseZi3MOFvSRwrbEuu0fK2WrjtXt7SgF4HMevAPjKKjN5laqBMIQ21jpy79jMHKLrvzSWxnuxrevKPWL5PLQ6Kw0ttePKAHoPtaaevT7WTHseOXarlHa235p4q43X7m3qdMDNstaEa9tZa8nR2B7OOdpvBvqZufX64JHptyji5Hmd1yfv1+pH629NZS+QrWnoninqI8C01fFm+zjh+9J1AriLPJgCOSdc2rVAxovPuGHet9ZvqyOm8a3uOTIPC57ZjVjS+JF+ubTfyx7qmYbufeHeGtC94o8+v5p4q43X7m3rBHA3eU7YcoNQ2u61jizNJ+Mkax3mWqlsq2/apgbIUjve1roAeY45I+l33zOdHb2QrrEmbLVby0326rtnP3uA2mrT0u7UCeCuqoWw1bYHKHvHWy5NA08G6Dy+FsY0PgpXC6yek41unorCWWujqRa2tRfJtdeFezqxPSDda1wtfgsHWwter+0p4ATwCrIcLVAHYatdjRuW5pcBsQYsq/8ecI3Ms1f/VNH0s7cRSxrb6i/aZi31XhN+ic64Jr6nw9zrtVltvHZe21NFJ4BXUw1Ma9tloWqN0xvcUputPgRE21hOl7fz2kbaS/1Y/UUl/T57Xwh7rgtvmfLcYh30iODt3cZr57U9RXUCeFW1QBhK2z3T2L3b1K4V8zatMLbG8ebnjZnpR1Lmv2jrhW/NFPYe7niLVOzeHxR6t2lp57U9xXUCeHXVQthq23Nnde9xeoLY6q/nB4FoO6+t1l7qR+vL6vsIylxgX0pKurbNVs7yKJumTvj21gngTbQGhK22NQ66JwStNla77KYtq02NK+btvLaR9lo/Ul+StrwPGKi7iEY/JLQA12u/VQp2S7Ad4fVG2kban5J0AngzeSAF+q4LW+1aXGItvKXxrHbZ9DRtw9ttnW723geujGveUz3XfyP9Hckhb+0oj+J6vbaR9qc0nQDeVBZIgbbNWVDabrWeTMeyxpPaRjZsSXU9XG4Ph1u7Bmz16Y3RU61p7ugFODLOWinQLd3uGnM5Wjuv7amITgBvrlYIw2i/tRsuWmO9NeOKo31KbaOvQ2vP+9D6kfqy+vTG2FrZi+1LTUuv1WdL2z0AesJ3C50A3kUtEPbar7kJaY8xpba9gJpNN/P2Uh9SP1pfWp+a1loP3mr3dI+0tNfPGtBt6bel7R5jem0j7U9FdQJ4N60JYa997zXlaFso7VvWwFv71dpG2tM+rH5oX1TZ/35HufD1XguO9rmWYzuii2zJeOzV9lRWJ4B31ZEhDKVtK7TWcMTRfmv7bkkxZ6HsjbGFWi7CWwE30ofX/qiblo7qfCN9nMroBPDuikAU2N6VemNvAeKasTMwXXvNV+pP69Mb42h6iWvCrf3vCV6v/dpjR/o4ldUJ4EPIgyiwHkij7fccW2sfdcVavQfjSB+8H68/3ifX1vf9Wqq94G59u1Kknz3h9dLbR/o4VaMTwIdRK4Qjfay1uau0RaC91sfaIG+dH+2jKJNejv5Xi1zoekC65wU169aPAN3IGC+9/RZzONWiE8CH0tEhHG2Phj56gVjro8bRrgFkr29Le14Ua1LjW64LR2L2BudrmcOpVp0APpy2gjCMPnruDl4LpBnH6rliKyY6Fu/P61Pqm2uv/55bbMLKjtXqdiNjHQF6R5nHCd8tdAL4xaoVwhG1uuGj9BFx5ZGYMhac8XifkX6tsSS1/tftvclrDfBG+3wpwHop8D21lU4AH1JReK7thHv2AaOfHm62xzxojBcXdcVSv5H+Pe15Ea1xR703Zr0maPaay1av51QvnQA+rLaCcLQPOP1EnWxrP703U/WGsTWu1X9knK3VeiFe44Edrw2aL20up3rqBPCh1SONHO2nVyoYG/UThXmPfjJxdNyizH8z7wLYE9C9L7Zr7IbOxL1G2B2tn1M9dQL48OoJTzh99YBw6QdOXz0ButX7Q+MisXR8qpb/dke6SK65IzoTezRIvdZ+TvXWCeAXoV6QifTVC3rROW2ZIi/q8WznbGzNXI6mrXZHv2bw9uzrhO9L10v633/KVRTCPZTZNbwFhKP9RPvK9FdikYinc6E60n/JHhu99gJvtL/XDN9TR9eR/refMtVrPTjaV09I9UyRw+mr9+1EWbCucevRFv9Ne+6qzsIhE98TTq8dvifIj64TwC9KRwRnpi8E+juyG0awz5Y2kiIXUu81rH3bUs2FvDd4M32+VPhGdd7r+xJ0AvhV6qgQjva39caqmodr1IA42y6jPS64te7pNYA32le0v6P2dWpNfbT3BE5l1fvisHVf0f72uNA8JfusdX21bY+glvln270V+Eb1Uv9mTmk6AfzmdeT1pCiEe88t+wCJFif4UmDc43VGlf0gFO2zl97S3E6tqRPAL1J7/MeO6rU4iLUeKmG1P9pFsccHhGzbNZ6edeT07gnMt6wTwKewj3PtrTUuZFtDuPSx98W21xzWfB2v4W+2t06YvzSdAD6V0B5rwcC+bqIGwj1BvOXFsud4Nf2skbE58rrvCcy3rhPAL1bnf/LttPYjFyN9rfX7WQP0a8P3yEswp07FdQL41Eo6+geEl/iB4wigXKPPE5Sn3qZOAJ9K6rxYxrWma21t/xI/gAD7zfvoHxTP/5cvUSeAT70yrXUhOtoF7ogAPeKcPL3EOZ96LRrGcezf6TD8TwD/tXvH6+r3APhfe0/ilet8j7fR+T5vo/N93kYv8X3+znEcv9ULWgXAL1HDMHx1HMc/svc8XrPO93gbne/zNjrf5230mt/nMwV96tSpU6dO7aATwKdOnTp16tQOOgH80Jf3nsAb0Pkeb6Pzfd5G5/u8jV7t+3yuAZ86derUqVM76HTAp06dOnXq1A46AXzq1KlTp07toDcP4GEYvjgMw38ahuE3hmH4S3vP5zVqGIa/PQzD14dh+LW95/KaNQzDdwzD8AvDMHxtGIZfH4bhJ/ae02vUMAzvh2H4N8Mw/Ifb+/xX957Ta9UwDJdhGP79MAw/t/dc1tCbBvAwDBcAPw3gjwH4bgB/chiG7953Vq9SfwfAF/eexBvQE4C/MI7jHwLw/QD+zPn3vIo+APjBcRz/MIAvAPjiMAzfv/OcXqt+AsDX9p7EWnrTAAbwfQB+YxzH/zyO4ycA/gGAP77znF6dxnH8RQD/e+95vHaN4/g/xnH8d7fj/4vpwvXt+87q9Wmc9Du308/d/p27WTtrGIbPA/gRAH9z77mspbcO4G8H8N/I+W/ivGCdegUahuG7AHwvgF/adyavU7fU6K8A+DqAnx/H8Xyf++unAPwkgM/2nshaeusAHoSy85PsqRetYRh+F4B/BODPjeP4f/aez2vUOI7P4zh+AcDnAXzfMAzfs/ecXpOGYfhRAF8fx/GX957LmnrrAP5NAN9Bzj8P4Ld2msupU80ahuFzmOD798Zx/Md7z+e1axzH3wbwFZx7HHrrBwD82DAM/wXT0uAPDsPwM/tOqb/eOoD/LYA/MAzD7xuG4WMAfwLAP915TqdOVWkYhgHA3wLwtXEc/8be83mtGobhW4dh+Jbb8TcB+CEA/3HfWb0ujeP4l8dx/Pw4jt+F6br8L8Zx/PGdp9VdbxrA4zg+AfizAP45pg0rPzuO46/vO6vXp2EY/j6AfwXgDw7D8JvDMPzpvef0SvUDAP4UJrfwK7d/P7z3pF6hvg3ALwzD8KuYPsT//DiOr/I2mVPr6nwU5alTp06dOrWD3rQDPnXq1KlTp/bSCeBTp06dOnVqB50APnXq1KlTp3bQCeBTp06dOnVqB50APnXq1KlTp3bQCeBTp06dOnVqB50APnXq1KlTp3bQ/we5egeI3ld27AAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeAAAAHwCAYAAAB+ArwOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsvW3ofW2b13Wse+//dd3JqFP4ImYaNUmxEHqazIhqyKIaCpUgSwos4oYsdMIeyBcW9CYIhMAI7hypIErC6IGoQAhMCPMBfWETIY4x04hmMYyS13Vfe/93L/Y+9z7Wsb7H0/mw1vr9fuuA//+31vlwnOd+Wp/1Pc6HNd1uNzrssMMOO+yww9a1b23dgcMOO+ywww77iHYA+LDDDjvssMM2sAPAhx122GGHHbaBHQA+7LDDDjvssA3sAPBhhx122GGHbWAHgA877LDDDjtsAzsAfNhhhx122GEb2AHgww5byaZp+rPTNP0DIu03T9P0hzr4vk3T9De0+jnssMPWswPAhx122GGHHbaBHQA+7LCd2DRNPzBN0++fpun/nqbpJ6dp+q0s71dP0/S/TNP0s9M0/blpmn73NE1fPPL+4KPYn5ym6S9P0/Qbp2n6kWmafnqapn9tmqa/8Kjz66dp+tFpmv6PaZr+32mafkfE/yP/Nk3Tb52m6c9M0/QXp2n6d6dpOq4fhx3WYMcP6LDDdmAPmP23RPQniegHiejXEtGPTdP0Dz2KXInoXyaiX0REf9cj/7cQEd1ut7/3UeZvvt1u33e73X7f4/yvJaJvP/z9TiL6D4nonyaiv52I/h4i+p3TNP0yzz+z30BEP0xEfxsR/Toi+ud6vPbDDvuoNh17QR922Do2TdOfpTvgLiz5CyL640T024nov7jdbr+Ylf83iOhX3G63fxb4+jEi+vtut9tveJzfiOiX3263P/04/xEi+u+J6Ptut9t1mqafT0Q/R0S/5na7/eFHmT9GRP/27Xb7r4L+/5Hb7fY/PM5/CxH947fb7dc2vCWHHfah7bx1Bw477IPZr7/dbn+gnEzT9JuJ6J8nol9CRD8wTdPPsrInIvqfH+V+BRH9Lror0J9H99/uH3Pa+n9ut9v1cfxXHn//PMv/K0T0fQn/P8WO/08i+gGn/cMOO8ywIwR92GH7sJ8iop+83W7fz/79/Nvt9qOP/P+AiP53uqvcX0BEv4OIpo7tR/z/EDv+xUT0Mx3bP+ywD2cHgA87bB/2vxLRz03T9K9P0/RXTdN0mqbpV03T9Hc88ksI+S9P0/QriehfEPX/PBH9Mqo3zz8R0b86TdNfPU3TDxHRbyOi3wfKHHbYYUE7AHzYYTuwR6j4HyOiv4WIfpKI/iIR/R4i+oWPIv8KEf0mIvpLdJ9MJeH3bxHRf/yYxfxPVHTB809E9F/TPSz9J4jovyOiH69o57DDDnvYMQnrsMMOc01O8jrssMPa7VDAhx122GGHHbaBHQA+7LDDDjvssA3sCEEfdthhhx122AZ2KODDDjvssMMO28CGbMQxTT/vRvT9I1wfNrPoMtCjXFu5FhvVRovfkVGv0RG1jP9I2ai/3uWiZbdq97A2+1m63f4/90c6aCes7yei74xxfRizT8Fy0Y854i/zlXkv/cv6rW1jTZ/fdPDRy+/FL1LVRtRvxF9PXxmfvV9rxudh9fbdUKkjBH3YB7URYBzdxqeOPnv6arW3sCNuz5vEjPW+iT1sT3YA+N3be7lwbAmLmotbz/6OhGVv37W+RgDkgNJh+7YDwG/W9qJe9mQjYL4lfNdUqZ+oX3uj+/1ebhbfws3nYSPtAPBh9D4uQHuAbw/w9ARhax96+MnY3ucX7N0+0mt9H3YA+E3anseFtlInvX29h/eu1T4ahHv62vNv9LC92PHpf3jb6kIR8dezb6OVb63tDbrSZP9qZtAWH9G6Z4rP6v2U8Lumr97+evftsD3YoYDfnH2Uu/63AN/aUO0ewsy11tLvTN0z9R+i6PVd2eJmlGi7oZjDRtkB4A9tW4R3e14Ee4M8c/HKvndvGbrIWl5P9oZojzdsW/Qrakdg863Y8Um9KdviIhOxPfoaceEbDRvPRvxcs5tgIOOvMRomHRGWjvqM+or07SP4OmyUHQB+d9Y7JNvD1x7hOxK8Pd7ftX6aWju1YM7COFO+9LUHiCO+PgI4DwhvaQeA34ytPTbl+dqjGt9yacqeNqDoYahfWSjXwrgniFvVcG9V3dNXDz+HbWnHGPCbsL0q1h5+1la9UV/R97x2HDQz5rwXa+nziJuZPX6/IvYWf4OHjbC3dgU4TLW1LiJrXhz3eGHMlKvpQ9ayfekRbqxVxxlF3FMN9wxJ9/BTfO1NnR+h6LXtAPDubU938B+1L5lymbZ7ttfTb82FmL/enjCOlusJ0LXCv2v6idgB4bXtAPCubS3IrOWjh581wTtq/XBtG2uZ1qfMjGVu0bFRr421lOzaqnoNCEfHgw8Ir2kHgHdre4HvXvrRw0dvtbvmjlkZy6rRqKH+R2f2couAzfIdKbM3EO9BmfcMjx/Www4A79L2EqLdAzR7+OipdkevG65tq6ePDLTla8wCuReMPYhabfUC8V6U+TE7+q3YAeDd2VozJN8CfNfow5ZLl2r8r2EtS5CyS496qbJeSvStKNke65db+nBYD9vTr/6wXSjfPYSsR/dhi5nTWb97sxooZ9RxRqlavlpV5Bph6V6KemT9iI/DWu2tXg3eob0FcO25/lrQXWNiVrad3lY72YooHlK22ukBYy/fa2MPID4g/N7tAPAubGv4fmTw9oTuXidlZc3qV3RdLrcIRDXfPcZwW0AYBfkoEO5F0R8QHmEHgDe30fDdqu6Wbe8hjJ7117O9GqsZ4+UWDS9bbfVSrVr9VphaMIvWHdF2qb9lSPuwGjsAvKntWTlupZhHqd29LLeq9TvavD5kx3uJYiFmzbenjnvBeBSIvfojwtIHhN+a7eGX/0Ftr/DdAtqj1O4eZllnfPVqK2s1a3m5tYSYpe8aBdkyntsC0x6h7RqYjgK4126k/mEZOwC8ibVeSLcIOe+tzb2GzjN+Wnz3tJqwMree63pbx3xrodcKxJa6veuNrBupf1jUDgCvbnsM374H8G451h3xkfUXsZqfb8smG9yi4WWtzR4TsGpVsVVvlNKuvTHYQkV7dSP1D4vYAeBV7S3Bd+0w9Qi1u0WoPeOn1m+LZduIhpeLeSDV/GbGfL36Wt0R4em9g3gUwEv9A8ItdgB4NdvbmG1NvTXbWrPvLe1F6md99WgnYiPGfqOTsTJART5qFO5IVbwmUNcMZ7eOKR9m2QHgVWxP8N0DeHuHmNe8YfDqRn3U+OxtmTajypbIhyny10PhWvV61BnhzwPxHuAdqau1eZhlB4CH24gL/VpAXAt6W/e5pZ5XN+OnxXcPy+xixc2DKfLdEnK26mogrAF4RBVnAFmjpPcC71L3CEn3tAPAQ61F1dTUXQOkWyverW8WInWjPrL+RlvNUiOiPrObPR814KiFWqaNEYDspWxrQ9KW1dY7DNkefvXv1EaEndcA6Rrg3ePrs+p49by6UR+1fntaZnZzMUudIr+eQo4qXKteiyquCSe3KOLsuPIa4NbqWPWs/h2G7ADwEBsxE3h0nTVAvdVNwl5memf9jPDhXRw9f7Vjwd44cA1YZb01YayV7xlm3hLcVp1IPdS/w6QdAO5uIy7qI4EzGmZ7gu6an02kbkvZWmsFdstYcGYcWKvbc/x3NBBHg3VLcJd6x5hwix0A7mq9L/B7gtRI8O7pRqK2jlUvmp9tb4RFZjBz09QfKtMr9Nxbufb2vQWIe4wN9wxje20dRnQAuKPtEb5rg20vr2Xr8eJIvue/1bdnNWO9xbJjvlb4uWa2c0Qd1wIzC1cExWx4ugbEa0O7to5V77ADwM229XjvHsqOAv3eAe3lWT6zfnpbSyi6x5hvFq6yjgfNPZTl5SOqOAPiHmDtPZas1bHqfWw7ADzUat7eNYHaA1p77Ve0bG/oRj7zDPzW/Ilm1gG37m4V3W2qx7hujXptVZ2ZceJI/UzZTL80v5bVwvSAsLQDwE22Rth5axi1Kt61+jRyvHjUmPDeQtCZ8LPVh8iYr6WQR43r1ijdNULZa6rhHmW18qXOEY6O2gHgKusddt5a9baAd8v+rKnqNR9Wea9exkfUvFBgbXsWVItF4JqB8jdKGvJjQbYGxtlytWVbQDxCpfeazEVKHavex7MQgKdp+oeJ6N8johMR/Z7b7fbvDO3Vrm1v8N0TeHsDei049wpNt0zaivjIWA8/kclWRO2KNwLTUj4C5B4wbinHy44E8dbA1vx6dbx6H8fcX+k0TSci+veJ6B8kop8moj8yTdN/c7vd/rfRndufvYWQc225NcA7ur9bh9Ct8l49aVOw3Ai7Pf56ffXC0d8oPrw1vRl1uyZkewF2KxCvGb7mdQ4Iaxa5GvxqIvrTt9vtzxARTdP0nxPRryOiDwbg2hDhiIv9aODtBby9yrT0wSprlbfqEG0LWM+8vnmA7gFYrfxoGFvA2grsGjhbVG7vkPQB4RqLAPgHiein2PlPE9HfKQtN0/QdIvrO/ewXdujaW7IeYee9qMi1Q8Mj4dzSh0wbVvliewZu1tBrubHjFsBq5WthHFGFPRVmDdiivmrrjYAwsgPCWYsA2Pu13RNut+8S0XeJiKbpBxb5b9tqLrp7CiX3Ur2jYVnTXgS6a4xXF3tPoM2YdZkYpXgj5SL1sr6zk7ZqoV6rhrdSzFpZr45X7/1aBMA/TUQ/xM7/OiL6mTHd2ZvVjueNuLiPAtZIEI4KM4+GbrTcR4Vt1DQoZ4CMynrlamCcAeiIEHYWqDVquFUxo35Kf1pZy7fVl/dtEQD/ESL65dM0/fVE9H8R0T9JRL9paK92YbWhxjXguxa01oRzzWscOWa8E+BGfqE9zVu91GzyPURALh2pBW0NjCNw7FWGl/P8WCDupYazcG1VwweEi7k/79vtdpmm6V8iov+R7suQfu/tdvtTw3u2qe0FvnsG16gwc1axj1Trg4EbhetaEL4E2uoO6IhKbgFtBIge/DKqOBtS9mAYUa1bqOEDwj1sut36D9fex4C/093vOjYaviNDziNUb482e/e7VwRAlhkAXA9oe4BwBqpW2SHqGV2fZEPyYo064pWR+dnyNf0a4WPEexH1o5XTylrlrTpvwb5Lt9vPuBeUtQNc79S8sWLL9qIca3z2UMW9FW9NnzqDV/tV9YJxbT3tWmfVl3W04Vsvr9r4ZyMndWXDwBl1GlXFnprtoYgz+V75Gp9RP1o5yz62Ej4APLMa9avVGTVJqQbOa6vetcPM2fyOwEUf4R4BHAkv9zStvSYoSxhL0Fpjxt5s514gtvqU9VGbXxPGHg1hC6YfF8IHgJ/WC749w857BtlotbtD6PaA7QgIR+pkhckIQ1Cu7pP8PL2xXg+MXn62PO9TC2hHquEeCh35yZSzykfqvW07ALyLMd/RIec1wdvzJmFj6EaBu6UK3vIXbKnc2n51AbLVeAbGvVVxTf5oNRyBZ01Y2ypHoKxW3vL/9u2DA3iP8PX89FS9ewFvz340QFe62juAa2C9lQrW1K/3Gqr6q4WqM8q3RRX3BHEvtWxBtMdYNCqjldPKWuWtOm/XPjCA14bv2iHnUXBtAfqodgZCtyeUrXQvr6UsqtcDxFsAPd2epow95ZsBNVEM3DUg7gnp3iFu9GU6IJyxDwrgtw7fXmHcEXle/zaGbitwR6jiaH60TMSi8FwTslIRR85TFoWx7FQG1DVQj6jabJ70q6nhVtUu+xItg/rC7WNA+IMCuMZaws5emZqwdKTs1nkj2hgA3hrI7kUJR8uja1k0/IvKaXnRUHMNbLXz6jC1taypVnFaoNOUopdX276EtBWSbrXIndoWIZN92wcEcI36XXPMtzak20NZ9gDmiDBzB+j2Vr61QLbSvbyacqh89Bq41+ulNXac6m/NWHGtIq4db65Rw9Gx3p5jzLVltHKoT7IO79/btA8G4PcC3x6gXBO8NX0iqgJvFLpbAbgVvj1+sU3KkfnYI5ylhfsYCU97YWMrLwJtD7ZRNZyBZ2bctzYcfcyO1uwDAViDbya8q5VvhW8GzCMB2wrejdRuDXRbABsBa2/4jvilRkBslUHXxWjaGlYdpi7fQUsVy4YyalaDag3oe8A7q3Zb81EZq6xV3qqzf/sgAI6My0bqRODrgVeWqR1T7QnYKBwj4K250egI3R4AXlv9tqriVouCOArhNS2ylAnVCVmNKs4o3YiC9VRvzbh1NCR9QHi0fQAArxl2zqheL78Gvj0BOwq8ndRuDWjXAnArkL08yyQnaupq9aKwjZSTZXqAfBUYt6pimY7UJy+XAXR0bNhTwzWTvGryURmrrFXeqrNfe+cA7gXflnJae3uCb2u4uaYfSfi2AHbPELbSvTzPamGk1es1w1mmyXYzeZk6XnrINBAT5dWpB+IsVDPpVl60nMyryUdlPo69YwD3hK8HT69MLXx7hpZHq94B4B0N3VoA18B2tAK2ymdmPVvl0XUykuZdX0defzVVbKWHLKOItdB0RIG2wFZri4w6e4Lw+1fB7xTAa4adZblM2NnKy4Z6W9JHg7ez2l0bwHtRw5kyXvm1J15lICvV6lbiKNSup4iRQ5SuqWFNMUfSS1u1Y8l7gTAFy3p19mfvEMAfBb490rU+bADeHiB9CzDW0qx0L6/GuL/smG8Uwl77vceAW80LaZuW3WWLj/FqMK0Bce9QdVQxy7yafFTGK0tK+bdh7wzAW8LXy4sq2pGQzaher552s9FB7W4N4FEwzqRF8nqZBWPtGofSPajWQjdzI5A1b+JWGsREfnhaLlWSaR6ILYBG4ZxJ19oq6aTkyXqRfFSm1vYfjn5HAF4Tvl6ZteHbo6wHWa+NSvD2BOwIAG+hhL28SL5lNeFnK70XdHneSOj2sKbwNAJqRO16ijkD55p0DcKZvEg+KqOV08p6dfZh7wTAveBbW64Vvi3jtBmlisrW1OFpg8A7AsCjlHAmz0qz0r28jFmKV5bpMd5rnfcCasYPUr1a+Nmr3wXEBNI0oHoKtxbO0fQDwj3tHQA4CtKIWfDMlEF50bfaK9dTDa8I317A3KNKrjnPpGXyM5YBTcZP5rz3caSs1q9W/65JEKOJUtbkKQ+EHkDXgDC37N1VFJTvB8JvHMAefLWXlx3P1cpYoemoKm4N/fL0DJCzdRrB2wu2o6EdSbeOa861tEheiyH4oPyI4uXl0HlEAWePkWWv+aWf2RuRZhDXqmEUkiaR70GYgmWt9EiZTB4ZZShQzvLp1dnO3jCAR8PXG/ftCd8MZHl6TVqLn5XBuyWUa457nHvp2TLIasLOMl1Law01ZwBaA9uoRRWzlu/2ywpLR8PL2gSt7Ljw3iGcKaeV9epsY28UwDVjvlq9teAr69TAtwXILeAlCsG3Fwh7ALgVujXAzeRl0iJ5GeN+0HUqC+IaCEfgXANwrX6t9VDHpiEQF7NmS2tqOApuEn7eAoSRvW0Iv0EA1475rg1fq04LUL36KM3y7fkYDN63BGWtnlU+cq6lWenZcp7qReU0UGegK89bgKz5j9TpabXq2DQ0PmzBEkE1qoZbVbNMj0KYWwTC0rQybxfCbwzAbyXsHOnL2vC1yjeEm0cAckSel5bJjx57eZm0SJ5lqF6L+o2cR+bmeODN5O/B0NhwWhFbajgzNoxg2JJGRnpWRcu8SD4q49m+IfyGANwTvi3lvPZknjb+itK8q3EWpgPhuxZUe/pqhW0rgCPnWlomf5TVhGO9ctqxVdZLs/ojy1u+vDxKlJdpphUI8waisCQjzyrfA8KeP5mO6tdCuBam20L4jQC4N3wjL1uWsUAZGfcdqXIj8I2EoRvBuzZIe/qO1o0eZ/KsNCu91bjf6HgvSouo4Oyx5l+DWDRNWgiIScvcDKimhaSlAwnLrBL2Qs8ZCEdDzt6b3hvCXnvbQfgNALgWvtHykfD03uBr+YuAtlH17gWuvfx5aZn8bB4619Iy+Z5pwJV5HnS9857gRTYCoBmzVH9EgZc01crvkhfWIEqgjDVmHAWuBmFkUdBmfEbLWOX2CeGdAzgbFvbqRl5uBNAobwR8o3Vrxnt3Dt6asiP65aVZx5k8K81KrzUNuDzPAnHkvOa4B3gjUO4N7giIPSCbxseGJUQJpCEIk1K/FsKobSIdwla4mStxmUesTGTiVguEeR/G244BHIGv1v0ofGU5D74aZDPwRf6yoEXteaCtUL29gZnxs1adXmnWcSYvku7leWapX56PID0KvLI/Wlo2T1oNeC2AamVr/blqGIWkEXRlGQqUpYo0Av60fJku86RtAWGrbn/bKYBbws57h2+ryl0ZvqNhl60zCtAt5aPHNedeeo15F30JVZS2NniRZYBbA17NeoFYlg31s2dIOqqkyUkj4E/Ll+nSMoDOlMmWXwfCOwNwi+rV6kfg6+VrkNXqZMLOkXIWUKPwrQQvShsJ3DXbyJbJ5FvHkXMtLZLnWasCjo73lnoehFFZrdxIpTvCMuoZ1VXNCklr0PQgTIo/AmkWTLMQlh+WB2HUJvrALYjuA8I7AnDLeG9r/dq3wavn5SNVK+taZTrDVzY9AqQ9fY3oQzav5jhyrqVF8jxbC041KtDKa1WV8oYik59pw/pLlXWgRSAsLQJhWRYZasMDs9YX5K8Gwl4bNTYWwjsBcBSe2e5a0NLK9Aw9W/VqJkzxtMHK9z2Atkefo+VrjiPnXrpWNqIQi0VD0Fb4uZxr5RCAon0OAylRp8YntxpFK+sOhzBqmMiHMAIpArWVhtqR+dJa7wy1+i3jwaU+KT7abGMAZ1Sr1dU9j/v2hm8mRN0AXw1AHqDWArBsZ0S/onmZ/Oi5lmalZ8pIuKI8C7TauQwpy/RMmgXqLLCy13atfAt4W03tvxwXLqaNC0fC0b0gTMKHzLeUbkQFk1MGtc8t88Xor4Y3+iplw8VZ+EbK9YCvVj8CX1Q3Cl8N3MnJViMBOervKN+1Zbw06zhy7qVH8jUgaWVqwRs5roVwFqBZq/XvhbJluWj9ISHp7JhwDwi/l0lZvD9EvUC8MoBrxmmz8EXlewIf+Y1OsEJpmbCzln6mZb0OIWcJji1A27PtDGwjZb206HHk3Eu3TKujqWAJ2pJmgdg6zkIY9b8GUKOhbZkGZA/UUd+qbQVhEmW0tD1PyiJQR7M+IB4M4NaJVVn41vq1ABoJPVv5EqwozTsvx5qvBvjuBY577lc2L3ocOdfSMvnFUDgX5aNrkQbYkteieknJqwWoBuJWv6PMg7QFc2gjISwtAmqeJo+RLwJteuHfHhDW6lgmOZAD8iAAT7QNfFGdbOhZy9Pga4WUkWmwtSzyeivgq6X1BmNPnyP91/Tfq28d15xLG/QLftMWBRqq4ylXq73av8VHsWhd07IQLhaFtQdc+WI063WXlbH+Y7lz30TRFSc7/fnWdCsCXy9fg6xWp2XSleZX1omo5Ar47gGMvdsZcXNg/fXSao6zaRlD9a0wtDwvZSIh6IwKjlxLtbKRa36Lf2Qt4eOMr2j4ulkJE81DqhJQGWWMfGvtbRWK1sp6dfrbDgHsdaln6LlHHs+Pwjcaeu4E3zUBNwrAW/QL/Y3mRfK9PCKi8w0kEtG54QJxYQ3JNi/TPN2DbclDcOXlrbQatedB1mJDr+trBsQ1ihqdpxVxBMLFMZEPxgjMSdTR0nqEokdCmJR6/WxnAK6FL6q3VugZXjWNNnrANzHmOwJqa8B1jb7JOpqPiB8vzTwWkEVwPV+XacK+Bcp8vpx8X7zM87pzFucAzNZxjQKu/Tt7bSLNglMvEGcsEv6uzauGMNH8AySljGaWX288WPqw/G9hY9veCYAj3RgJXw2yHnyRzwiQe8C3WAf4toJ0DwDuDeXM31Qag60ErQAjAioR0SkA40jZawEvK/NZpj3PHwU4mDmUowrYu8ZqZTN1a8ug8q2WUboteU0QJsIh54zqtSBMogy3SCg6k4fytba1stLGQXgHAB4N30z7Gnw1/xKQqFxP+Mp2g/DdEn6j2xl5Y5D5q6Y9gGvAVoJWQvNkhJvPCRhfhBrmfq8PsPK2r5fTEs4czBzKz+NJB7KnYiPXbA/UqIxn2fKt1kMFe2nQNAgTza9hPK8m9FzMKpMdD5a2BYRJqV9vGwN4jeZlGxHISkOARXUtIGfMg3TSDT8OgwP8HQ3AaPk9gRjmAZWrADcCWw2yGSVcyl9RSPrRhgS0ZZ/VnPImBPcd51YzploLM698tK7mIwNLqz8E/KK0aggTqCDB6YWjM+PBUcAiy94hZfxHy/a9S9sIwC3gs3xkZz1reWclPTOhCqV551Zb/DwZdt4qrbff3tCWeZGyBMoT0ULpBoDLYSshi+AaAe7pxBTsdQlU6YMDueSVtNKnAubS3+vlPC97vs5D10gdc2Xc+peSedz6Xj/zNwCWj5obgGYIE+WWJ5EoQ4Qb2kMoGplVZn0IrwzgXs1F4auBNNqX6Lgvslr4oqu7Nb4Mio+A2kgAr1Un0ldZRq2jq1wEXA22SwUszk8AxMEf/+mkl7s+Xoj0f72eFmFoBGYO5TCQa2BcLAtYVB6Vi+T3sCiUs+DVfIchTGQr2HLtsZYUIR+eX62tLUPRXlvSBxl+YrYCgGubyIz7trabCUuj/ChstfKonuZDUb9bQK4XBFvr9L45kHnPczCeK6BrAfekKWEBQQTYM+nq92TkcbvSS+1Kf5dHXoG2BDQHs4QyV8omkLMwJnGMzrW0GusNXc9aFK92nIawBCE/L06KRceDs4DTIKyVyX5QoyBc/BTLf3kGAbh8sLWWHUNda9Yzyvdgq/XDm3SF8gLw3QJma4K+Z1krbfaXgTegcjPA5bCVUERgjcIWGapboMzzrnR69gWBWUL5+lTD1yeQkUJOwZgoBmYL1jItWkeztQAdBWumbBWECeRlIRxRxppF3/CaULRXLjs+XfwVe7M7YfUe97V8aHVbQs+yjAVoq0wQvsXO4HgkfEcBeA0wh9LwmO63zlc4jiuhi4CrwfakHMs63CxF7NmFKWHu/6l6H76vdIJg5lDWgCwVslTHJowpqIo1kLaJA0fHAAAgAElEQVSCtqdlYFrjK9OG+folMIl0mGrhZ8+ndYeUDUVLXzWhaM9q6uRsZwDOwjdSLgpoD+4Skigte26V4cfOpKsWQPWA4xoAXg3CNwhdIl/pPv8awNVgK0EbUcORvGLXGXSvMI+3eaHTs08czBzKUil7QEYwfr1gPm5MvirmlgFtpGwtrDMw9er3OA5DGC1PIsLwtCCIymiWCUVHIexZNhSt1elnOwJwVsmiOt7L4fmR0HNPy9wYBPuAWB05PgfSM6CLtLMmfKsArIeZa8GLVK4GXQ+4GmTDE7Lo8gRp1rjqTVlFleXypvMLwpplFGWkbCS9Bn7SF7GypNTLpmvAPYO0mVnLkyxYamUy48O1dzzSRoSix9oOAByBDepmJgys+cjWbVG/Eb9nUD857rvnYy1fKxNJ7wFeI8xcC10LuC3h55bQM6rLlS4RDkOX84XiZQqZq2OuqKUyluPGcv3xIjx9dzRXxM800o+1a+p219q7RVRy9MZAgz0p6e5rRzOjuWmw1T4MzSJgl8fcb0YFy3wvXI5MzgTvZxsDuCd8Zbmowjwr6Vb7WdjKfG+C1w7g2wK/iJ+sz57QJgqBtxW6NUpYKy+tZTKWFo7mgCXCkC3lZBi6lEXlZmU4jM8nPzwtx4kpCGJue4DxCLXd6geanJTFzdtIIzohi4w6yK9n2bo1EK7pl28bArgWvjWmQVZLtyZeRfqEYIpmPVsADy43KseR9Bq4tQKxZ3vNNxXLiVUozIzAWwvdTOjZVsPWGLBPER5+1saAkeot51L5onHhxSQtBmRY5kQLVTyfVT2ftPWZKA/iKHwjdbLWAkt5Hj0mevU/mg4NvQlInfaYFR1tMwraSCh6HxDeAMAtk6K0+rJs79CzVU7CE8FUmgXoM1U911ee7xGAI9t3fcwVLwcvDzNbarcWupnQs6WMpY+MaeFn2Y5UsLxPEqLFBwJtKY/UsReiJqIZiJ/vRwuI0flezFK4WQjzc/4+aOnueDCRvj5YmxXtWUYZy+PoHVXteHDE+oWkVwRwZnJTL/ha9c9KupWvwTUKY01lB9V1BkI1dboCb4N2O4EXqd1W6CLI1k7I0tKipq395W2jsWEEWARkTR3z13p9vKuWKibS1hYnQSzPe+VlrQa0UR8SttE6LoSJsGpF8JRp2rllvIwH9lY1WquCeX1q6sMKAB41qzhikZfnhZ5b/Uvfko7SnNCz5wJBCNXx8nrDN3pzMAK+gTFerngRPCPgtZYd1Yai0bmsmzU0I1pCssWqZ03POxS258zpy/n+mdOUA1wmj5Sy2THbSF5pC7Xr5XHTbiJCnEGgrZkVnfGPrGZCVosKzpStB/EgAE9UD16tS6PUr9eHXupXS0uGnrsCarAfLU+mZyAeamOperPgRYo2onYtpZsLRc9//FroOaOEta0oZShaC0NboedSVhvv5dBfhKBpOaZMRG5omk/WMtXwvbHtQtMtynct1eyGoi2CW/CUZaw6GaXcyyJ9z/gq9iZ3wsrAN+NLqx+deOX518yb9VwRepbFI3k94VsL4F4+zfJ2uDkL3qwSxvC1Q9W8DC8n/WjnyCQ8tbqRbSj9WdD2WDAKQZc2EahL/dPjfy00vXjNPCz9tDPNdtUqSdY1tTW/xqIg9fJbVLMLYSKsgnnFaOjZy0dlZPuoXZku81A+KoP6McZ2BOBsV2T51sldVn5Ji6pdBH+LkkRm6NmDGT+38r2yI+Hcs10EXiJC4eZW8HpAfuXlgdsyAzoSgtbKyLW+9zQ84aocZ2dBIxifHm1cgfItNp9FvZyw9QTxbAkTGB+Wa4it39caYqtF6Xq+SJSvCam7EEZ3MZFZ0dxqlwh54W9pNXdR/ZcYRWwnALa6URN6RgC0/GbVr+VPwlhLSyw5KscRqGn1miGXrD+67WcaHueVE6ws8GqhZCvEXBuiLobBLMHbNwyNtpwk0idclTo1s6C18DPvM1K+C+CSA2KmiOWmHrPNPOj8uO4qYWl0rqVploFo1hfR3J81Hu3V18qGxoOJYhOyivValoR8e/myD55Zk7II9K/dNgaw13wEvpl8lOeBUqZp58i8mc5GXQQgeR7Ji5RvAmCwfPbcLTNXvWicV85sjipeD7wxMGMlPC/TZ0KWNG3bSS8Ebc2EjsyCjsC4+JKhaVnfAzF7Ufc/YtmSOT4sw9K1lvWRBbQFVa080bwNeZ7Jm1mPULRn0fJbLUvq8aVZetzIauCbLVe77Eim1U7E0vx+YunOgxb4scZuD1qyfgR0Xp3ewI208TzXVa8MN8t1vBK82vguTvfC0Hb4uS0MHVPDkfzM+l8ESF4+MvEKjwHrYWye/0qfg7gEtYlek7WevhcwFuPDZ7LD0rNyRn5Plav5Q2WIpXkhaFkmOjasvm5U0ApFo7oZQCOwa5aFLMr32ugL4Y0AXNusVy/rtzX0rMFV+glOtkJuZRULylr5SJ0o0GtvAqw2IjcBQfhq63mz8EXw9MLMHnhrn46EYBoNO9eYNit5TZvDXj4W8Yxf/6Ob/HGI9/Pz8/wz0UMJEz2XLOmdeJQTabWws8rLOrxeNAQd8eOVN/kSXRvMLQssBOVe0GsNRRfrB+ENABxpcm31i0zrp6d+NR/lOKF+PXjJ8xTIOqVly6DyIT/+0iI01hsBbyTU7M2E9qBrh6Lbws/ZMWAUipYTsrQtJXn5SOjZGuvlylZCf6ZwFzcERlg6MDYMQ9IjLRtKtupRwFfNTQHKD6ngYp4Kjo4FRy0yIzpitTDtA+EVARxtSgOkVz/7UryxX1lO+kftWepXHgdmPctmMqDV6si03vDtCeBn+kv1oqVF3lhvNtwcAW/LTOjarSl5Xcv4GK5V13vyEU+vmXSFQsvu2l8S4WQjT274MYOzMzY8X7JkQLjPdbbeasLSkXoZhTyzlh2ySDkvZkG5FtSyrV4quPguVvclGQjgGtcZ+K459ivry3zUhkVRxTSQ1oA2CtRM2R6QRnXUdDzRCs1wRuFmK6yM07wwtB2evr+EnBKOhKB5PVRWM60MmmzF28zOgi51IjDWxno11ftKfeWV1zY/B3A+nQhNRuOWHheutZrxXs8fAZ88rSacbfl4mnxiUmSHLJRvzYCOWFQFexD2bggiVgfjQQAeHNIxlx3V1NfSMv49nwH1W5qyeO2BtzYtW3Y4gHHI2VK9NeHmjOL11G4Wur1C0NIHMu1pSFr4mcieBS3Vsh5iluHs+bpgNL5sjTujELTmh1WyTS5XImU/6VaLKtdImubTSuM+PNUcet2S2BEV7EE3q4JHh6Jr1gfHb+QGAbjGakPPlp+zkm75Lmk91W9gEpYFWguIWh2ZVgNTKz0K10iZRbo/0Uob60XQ1MB7LzduXLjk4b+ZMLSthiOG1v5K39o4b6kf234y8PQjwuO9HOIlzxrvlSFo/n4t1h2fLnQ9nRc7aS2WKz08p0RERrlG61sg1tKjZa1ylmqeWc2yJGQRFVwDQWnSf8Znj/ax7QTALaHn0S8h4j86nkyUUr+RMhHAeumaHyu9pi+oD4s8f7zXCjlr478y7V5Xh29kGVIuBF03KYuXleU0sxRkpkym3J5MbixS0ojo+b15prPx4W+dr3MI8ycrzZ0t0yMK00tHv7GMHw/QJV1yTutzekKWWZDlZ9cFI/NC31Z6TXtjbAcAziz7qfXlLQmKbrJh5XsK23irLSXrgRGl1aTXADyqesMAnsNXm+WcmWiF1XE8PM3L4zwrBL1Uy1r+6y2OTcaS9TJ51uSrcm7Ngn5tKYl3u9JC0NHdrlCeG2Z+vm5cdqamhZsyS7pZCSOzwBqFs5UXha7lxxs7Tk3I0o5b1wVHx4szELX6FCnbbhsDODvumlG/0ZcWuQHITvhKjP1KMKE8EmVqoBhWoAPrqL5yk63ioJXquC48jfP8EHXJl3la/isvH362VLGEkTX5qvjiY8NoXNga643OgLZAbMFWhqpf9ZYhaVT2UeH+R8ySLlY9OSszuSoL1do8L6ws02RdVGdmHFrasSy7hQrOwNlrr49tCODWSU9RfxoMtTZb1a82Dpzc8QoBS9ZB6dE6EZCvAmB7shURQfh6IWcExvhELRu8LbtkafnF0NiwLIPOLZNlpeItadbkq1JGwjYz1iuBbO12VepoM6P5+yXTrPRikVnS8MlKtWaNE2cnWbXkZSZolTSTWR9JBWvl620jANeEnVvVb2ac1mpXq++1abgpxxrzZRkPrNE8lL8qmGOTrazxXh/G8XFepJBf+flxYe6Lp8n8UneZ50M3si64WO1TkFAo2oJxJPwsQ87c0GQsPaycG5+GYFZcLB9xGIRwBn4Ri4SmCZTxxnqtdJmm1Z1V2FoFa3VqVLDXJint5mxlAEeAVwGy1MznaJuofnSpkSRYctmRBUAvHeVFoRn1YeWHARyfbOWt442q3kxoWrYz94nD0K3h59YwNK+LwMTrZWZBIyBbm21EVPEcxDjPfo34gRPW63cczgzunvV4JautFUb53CwgZ/IiE7q6qWBkEsgRQKNO1QK9VtW2q+GVANwKwtpyVl0Lpp4U1ep7/g2XGkRRvQgwtTZb4ez50MrM0nOTrVonWnmqNzqhS/pd5uX2itbKvd7GiALOjQlr4efSXmT7SSsMLZcUWVD1FHEkLI3KeptvqBYdF26BcBaypbmsr1ZljMp3UcG1m29YUI6AsHY9r9fHNjW8AoB7znKutZ7jzRr9KnxHoFsL3IjKjfqrVc0wj435MrNmOiPTw9B2uHpe1wc6T5d+X3l2CFrL08rJPP28ZUJJf0MPSUBLguL+Tgt/KK2kF2t6X04E1woXCy1TIopPluLWA6bRNi21G0lbGFfB3LyxYE31RlQwsmi4uZcK5vUp7WMQgCfKg1frijf22xp+luWlgkWK9pORJo8DjzyzgIfSRingaFlPNas+4mO+0c010HjvvWm7nOZP1r2X08PgvHxJ575fZedgtSZlST/zj6VtFrQVhs5uxLFUxb4iRttOIkOzl9UZzcQVu+03ZE71hRKOqtpeV1sL7qgMKqvla8O1XtqikRFPSiqW3ewj65dbto/l2v+mdsKKwrfGrMlXvdWvdgyqZJWulh6Gn1E2Wj6jjDvCNzvZKluOaAnje5qukLlfXhaly3xZZl4Oh6A9NWwZUpE8PRuCtmZAyzI8TbbxyrOXIekznQv8O0BXWm8IE9WFn0sTPep4NwLWWHBEdatjwZ55qtcKW3uAtCAdhWvtjULMs2nTNP1eIvpHiegv3G63X7VBF4yyGfVrtZNRv6iNRvUbUboRhYvK9VC5NXUQfB/WCt/IZKusOp77yoMXLz3SJ2XN/+J8eYzOX+n2BQLtBY1AzI8zM6CLH00583yerm07yftaC1fu88o+9YQD06ogXGMRxZup40G7WQUjp5ElSahui0Kuscis6n4W+Tj/IyL63UT0n3RvPQTFvZk3+Uoeg2KekI6Wj0IzUqcFuCaA+yrfzGSrCFAz4WZP8ebC0GNnQkvzQs+8PQ5RBGQMY/3pR6V9bdcrNKPZW9/rqV5rlnTKaiBcTpF5Ys2zyExoq05ETWvADqtgTcmidAk2DXSZdcLyWCtjtYesP4Tdj+92u/3BaZp+addWXYso2Ij6jcx0js5mRm1Yfeqofr08y3cGmrV11b7WLTXKQjWqZjPlXnl99oq+/11CNxOG5mW8NGRS8Za00oamdksdpHw56JBa9UCMNtrQHj2Y2Y6yVjUbTlVbZYmSZq0TulCdyCzpkh6aEU2UB2ZmtrRXJgvO7PKneusWMJmm6TtE9J372V+zZtOd2vKArOUlNgixVG4mveRlFbBlrb4WdefKl6gOvkRostQLdMgs+EbKvfLqniOM0u9vkR6mRuV4WXks67xFk7OmtRnTaAMR2y+eKV1r6IlLRMbsaK6EpUUUaMZqQKvVzUzMiraxcKI5XPN73H9LyVrrRsHb7fZdIvouEdE0/RI0Hz3YbFb9aukRWHpmTdaSsjFoGtiiyne0AkZ9SfsbF3ZGCvQL+noB2Yyv+0usUcL60iSezuvI/NJ2MRyKrp8JzZVgzQxoro4zM6B5fmmHp5fcyAMXrNnP87rz8d7ulg1Hr2E1oNfqojpe+Bmys3ZjjuiSJHmOfMhjC/Iobx0VvKYMHdRkhEZamjf5Clnl5CsPrFqeZq2QRX3L3AB48H3YKPhaZYiQarZ93etg//e3wQY3r4/T4xOuEJhlWXSOTIOyFYaWY8MSyN4M6PIa0JIjOTbL4Y3SLJiiusOtF4S9ZUKt1ntGdVQdP20SBbTjHutvazb2iIwFexCmZJu6lxUs0lSL+q3xXVtfU7/Oj84CYUQZZ2BqtYF81QDdgi94sMIa8O01cetern7Lylc6VsPzvzElzMvO03wQc4h5k6/KsT3pSt8D+vW4Qn+8dw5vrHTnY8NtKvdEF/oefankXenrSJ0WCNdM0tLUp+VHs5pJYJHJWSaHIhOiNJVLwXzNb1QF11obiN2Pb5qm/4yIfoSIftE0TT9NRP/m7Xb78Y5NPCy7dEjme+FnWSe69EhrgwLpzIXkdET9bgrT7L91ws6jyhDh2dF+er8JWfwvLyPTrRnQUtlqdS60VMEyHD0H8kshRx9FWPrjhZp5efR66kA7B/r36AuW+/UT6BK4X9L36EpX+noB6a8fNwZflAZMqwpHa8uYapYUaVargLV8l2mygLWJRlaFWm3VbE/Z0n4diF063m63fyrlkYhe4YfeFlGx1sYbWUP1NZ+BH1pW/fK8zD+truWzti0B3zVmO39JX0M4fknfmwEuD+j8dpWyLG+b55X0+9tsK2ENtks1bCtflM/ByNPmIegCV+1xgvFHEfogRuuI50pXMw59r94Xj9cpYa4B90v6GkCY6CTqWyDedHa0Zi0KGJUzVbDcnlKqWM20GdHR8la+d8fQ4yaA6I3thNVT/fawqL/A26cpXK1sVAF7baK6XtudbLTyfbYDYHhP7w9fpHrRTcD9rdSVcEmTeTwdjQ/L89rZz3IM1i+PFShKr1WrPe0F/ItIf91IyDRefq7MUdpyIhuy8oCRxd7RlsJFYOPpGQVqpWsWHTeO9m1m0XW5suHaMHTUonAdM3N6BwCOwi4LWY1A3Jd27rUvj523EYEvCtqmsPAa/5ZPNloj7PwFfY+IXrD9gr4HIYpnSLdv3CHT7x8VBnU5n/+9mMCVqvj5dbgKKF/iy22u55dKLXY5LUPP93bLOC4f650rYx5OLmVK36Uibnm27zy8bavj+/cAKd2vF3VK2nxM+Gt1jFg1LxxdnqykgUxTlFYaz8umR0y7GUhbdjJWLVQz+0P3AHcf2xjAkV2lMvk9lh7JNiKTr4hCIYca0PYy7wbgjcD3i0eYuc+YsD/W66nj+1uJAc/Ll/SS5oWhieaw5aA9XV66itsJXFOuZ57/maV/a+b3emZh6FNk4tUcxq8yOAStg9h+6EJ5Z8fa1zSf1DWH8JnQmLAw0MXr5fGanmvhH+faU5Q0aGYtC2XNh7SIeg9PxmpdksQ7FYGppWCju2P1V8EbAjgDxuzmGMii6hcp3cjSI6VJCdIsaHspVa0/pJzvAL5c5X75BG9/+Fpl7m/HspxML/3Uws8R6BbgarDlgD0HhS8qdzlJGL/Or+dv0elyfQJZKmQLxnK8WIK0vGdIvaKynnHgS9V6oquYeJW1OYS1MWHR6NOulxPeqONy7gdZIswKjR81ws9TwiHwysZ7qODWdbpemXUgvBGAI7OUs/maUq0xVF9Tv8G2Ii8rCmate1HYR+Bs+loHvuXC3wO+rRO07vW0dJxWjs1Z0NcrBK4G20leEzIX1Mfn+elR5yYEWgEzB3IZy+RARsq4vLYCYp5eXq8GYk/lzlU2Vs18ZnPJLxOv+Gzn+/sfBfMcwqFx996Tsrhq1cZeR17Fa5ZFuZOxohbZeIOcfE9po3LRfrXbBgDOwrdF/Xpju7J+1J+0wMYbJS2qfL06lh9P5UahLH2tDN8sJMt4LxEpPtpUbyt4kdJFCrcAcQZbfqypX28SDa93Ev7PdzDfzjqQX+r4ughTz8eC5VjxC7ASxK/u4BnQvR41WGY7FzCjJUjYyvKju49QnR4Q1sRixDRoS39Rk+1GZkWr5oWhI3DzgMzLtJjlow+EVwRwzVhsYp/lUBkNyBb4tT4kJl95ajYCQlk+Wj+saqM+5HKjbeD75eNSeFLr9Fs3fH+rcLnShxO99NwrjYFahJdPl886cDXYakD2zLsIn1/tTOxiz4EsYfxSxnMYS4VL9IKu/uAFezerF+DnY7ElBO2GhhULA5XZmbS7H2FZCBdoZmBrqWPNT41ironULkw+JYk7RkBFX9Lasd7oZCxUzoMwKb5jtoEC1qylKz2XIWnU02zw+r6IMs368OqqEJ6HkU5igLHA93kOYWWDcF5Xn7Usy2wBX6l6LcWbAu9VnMtjXmawlW/38usCJoINnCvFwS3Ta5ZkFVXrAbWo+uwDIR6F574u59fypIvILNd5C5jWjZRUoSlV6ph1rdDUcLjN1jXB2dnTI5YT1ftcAcC91+hafs8g/yzOo/kyrWLylZaegaHmP+q3RnnLdgp8ndDzveh8rFPCKjNea4GyTNCSy4wyS5dk2DoD6OInAl4+rhuGrkwnka6de2aoXzrRUomJ/n46I1W8DE+jfaDvTeghaLTV5Kv+coz3Sme27CinhPESpKWV5VihSVjYARHNJ2VdLyeaz4xWbuKzqlibmNV6lfcUcLTss4I1g4woF4auKWMp7awK5vUI1PV7OMAmyoEXdcMKP7dAPfKSI8ujAn4ykE2HghW/KD/iwypD5MK3qF8UlkWqNaZCW5citc+evvd5WQ+Fm2WoWapdCF2kcj0FjK4FnjArvLNClbJ9CeQHrKfH8SxE/QhPE9EMxHc3LxDLhy5EQtBR42t941D21v2+xoBrlPaj4ixadEWbdKA9oyPKMwpoWaaXKrbmHZiTsbSlPwiOPVRuq/LNtE30hnbCisDXspa1v1L9Wgoa+Q5OvuJ5WZWq+dD8oXoppYv+3X80kUlXRGi2sL2pRQaukXwiem5X2Tcs3QBeBFdP/cpjbTw4YkgRSdCWPADdBYwfaQXGZ5qDuIwTcxDfm7o+3rH4rlmljpw4dSL88ARuMaV7h7DnLzz+K+xKr9CzvjzpUwymCKRIhWoqODPGbBn6/p2d/FlBT3VmZixbgK5RymN2vdJa3tCizUfKabQiioWbCeRp5Z3+RFWvVseCrNaFDNxNpSv/zR+ykJ90hScsaROqauH7pZq3hDPyd3/pywldvM883KyFmk3wcsBq0EWh515jwREVjGBb8ng+0QzQCxCfiApeZGh63iW+dGkeXkbqmM9mvtLZeHiCNBuy5WETFmS/YP9H7X7jcLEnZV1O99+ZtlVlsawq9q4/rSq4tKX5C4lGLQwj82smSNUuSYr0tY9tCGCt6cxSoB47X0W2oYzkEwZeDYy1JmsUdBTsMG857ktEqRnPRP6EqpJnzWZugW8EzhHV64F3Nr4rwauBmGgJXaSEPfUbGafTVLAE6sXI4wqYpz/K8/C0DE3fm+VjxKewCvZMKl0M2q+f0F4+Bele34Ps4mEMwK702lVsVh7uluWMB8tohJZHIJ+XQ9bj6p8KQRPZYWhy0pFj7zxqURXcF8IbK+CIjeqi5VeDuBN+jjSZgaJMy/qW+VSR9zAeeob5Bsx4GVSe59nLfawtInW48vxX2/XwReHmxRivpXij4I2qYO96YF285TGHrlY32Mb8mWhi1vSAGdP8EYscgvP8C8wraXw/7JJeVPq927pCRnthL8qzSVlEr7HhxUMb2Onis4rkkShHwbKtFvYfAZ4VWvZmR1ttRepY1g/CGwG4h/r1/JW0lcPPpUhW9UbLegrYg3k4b/mIQTv0nB1jxflafWu28zJPKud+ytcNN0vQaoo3MhZsHWscQCpXlpehaKmiAmO/C2UmL/KP84l0NTwXh/OxYf68Xr5m2Bqj5eHoEy23orzn32dUo60ridBEq7tqvg9f6MoYQV0avwGQS/g+FyVsbVVp3Qx5eUS2WrbgbEEV+ZH5i3raYwrVCg/LAjM7thxVwahsnW0A4F5U2jL8rEy+0u4DNNWpAdRTyl4bnj8vT1lyZIWeiSxlGQemVh+FnYnQMqQ4fOUTlMq5BV4iWqpeTfHycwldMtL4XyI7/GxdA7QLpoS0NetZgzFPI5FOy3pFDb9APJ+kpT0i8QVNfi73jZagfYWT9f2g5xAu21Zam3NY6jcSmi7fs+tJqO/Lmb51vj4gXN44sEEHf4s0SMo6Mo9EOV7fgrPmJ6K6TfN2xtKcjwhDe/3rbysC2GtqhPqNnmvtJNb+RlxbII0o5oj6zShjqw5YckREqUlXGpCzwJzPaNbHdL01wV88Lq36mHBM9aoTrCLglXUI/PWUMOKAd70pnyuHbamnQNOEMVLF3AfwySdq3Y0HXjmIcxOdylpgBE4Lwlf4Rr5AKc2CrObrlc/3z77M1T+aFU1nek7KQvC1YBgBNoH8MDQVH6gvbmXvLnFEGNoCfQbeNW8Y9jDYaprJqt9IOWvzjsjDHIKvI6NekdsW9WvlWRB+pj/UL73Gp9BmG7XwnYeZ52pWA2akLTtEbW9VyduZ3QhYY70ctlHwjhwH5ibHcWV5qXa05UiyX/LCLsFrgZj1bbrcN/Tgarg8HpFO9NyrWQtJa6apVw3C2uYaWrjZgqy1RvgJ3WUlul5Oi3kVn4n0ULQFXw/KWl7J9xQzMut7aN4AWGFoy6KKtPjroWA9H/WKeyCAM6577JYVDSdH29T8GeHnFgUbBXVW/Ubbe/59hZ6JCIaeNbgS0exvZJyVT7Lyws6WSq6BbyTknAo3W+BFateDrheOlnleOlLBUjWV48j4r6aAlaVKEhLqJC0jmivHeZfh5+89x41liPl79AWdaD47WoMwCjdbkLXGfzXlzNcHc4OhaCJb1TOhcfUAACAASURBVGpiDKlcBNpaBaxZikk1D2iQgI002KKCIxDO2yAAZ2YIe1tKyjIRpcrLSYVbo6wTNwg93tEaBSzb9wAN28EbbhAR2GzjoZIVmGbgu/S1DDtnQtTasiUUgq6Gbw14tfCzVLkIuJ76jSiRjArmalcb/9XK8dcl2XOdl0dquIwNf33iS4r0cV6pjLXJVmWcV8ISgRVBsway95eshbQvi/fnOR5cEvgGHfJzlNeEiArWQJtRwAj8VhnV+AMaUCfQ3YTneMQuWDX2JnbCGrVPdNQknHmadRxwGwGoBsSMb5mOui19qtB+0IaWG24U42qXX5DkxhXz8kulrPmSQJa+UYgbtXNPW7bLbyC6w1dTxTyNRDpSx5nZ0MXsIcjdm1TD1/MrbIugyU3L58uS7uXOi+8gqivr8bq8XhnTJSLYzitv/jrkgyBKOaLXePDzYQ18gw4JSE3RRrmFgB4Bp6bCvfI9lHXKUjI8YP1BvjGAkdV0KRt+rrXE7GdkFpA1QMt6XvsI5iFoz9Uv0WsMOLLkSB/3zS5HyreDlLEcE5Yq2YLvF199joGXjHwy0jQQkyjH//IyBPIsQxdkKwxtqV8CaRII8nvG3wPjt8Ih/MVX34BZ0rEZzcXkmHBRzXO1uqyLYC5VclG1WqhaAvuefnqA++XrQic6abOiWY8Wu2TJz6mkES3f44gyRuWyYOaWhXQqDJ2ZfIU6VhuGzrYV681GVjORKhp+9tpCPjvMfs7AWNbV/GjqNaKALfU7K4O3m7wfeypUbnBhLU+yASuhiZYeefBFMLbg+wV93a56W8GrQTc6ASuqfr31v2jcV8JYu/Aj6HKfnj3eH2tc+BVuRiCOQRiVlSFitBsWClsjyEbV+KIseGoSEVfCM2c2ZC1lHFHMBPJRe7JfKD9kslPRMLTML+e1kFw/VL0RgKNLimpD1JoijsR3veOASUhqsEQqFXXHUs2e0vXSiEiu+dWecmSFfGvOvYlUlh80czoDXz7TWYVvuXJzmH5FGKwczEQ2nBGIPSXM82S6laYpGu4vMu7L/3oKODD2O6srypdx4bstJ2fdoXqfUDWHLoYwAiKHbmQMWM6M1kLimaVNvOyVzvffGn9qEtqmUgOtl0ZOujQPoB68w+Vbt6aMNCyhHJX041XwBgBuhSrRvNu14edI+cCTjzLNRspbCjiifrU0FcKvpQBl4tXzGEy8siZOZWEslTFWu7kQtQXf1+YdDny/ppzqbQWvBt3IBCxP/fJ8pIDLuQVdFIaOKGAJ2qR9IppNzqJv0wzCV0K7Wi0fuoCWFfHZz0glI5ByBYsmXcnw8jx9qX552aeifgD3cnntlrWYkEVkw5dbVBlLa1W2mmIOW/RhCprqjUKydolRHwivDGALvr3Vrzw/g3wv/Bw0DYzZep4CjqR7cJ5BWN9ukmg5iQlNaOqvjPExV7eRMV8d6NcZfMt4LxHRxNVtFr5ybNgLTxNIL2lkpPM0ctKKoQttASuRvv5Xpkv1aoFXCztLMPPXy88fJseFOYRftoTwlc4z1amN1RbLzoz2AM0Nl+W+5qFouDa4PDFJquBiSNmSSOuhgKPXtzRw5Z0BOs76qbEMWNshvCKA157x3PLSEO2S1TW4RkGNFLCW7ilgDcwP48uOiF7ql5u8W+djwnd318UF6ATSZFltLTA/1vJkGxrQeT4RzZQvEXuIQg/4SpUrwawp4RYVrKW9BZPXS3D95BAu24FKCF9ZJf5QhGJSsXLoSYWK8uSDHpaTsvB48L3ty+y89O0K+ilV8NPKPtFEmDFI2ZJIyyrgWgZaBn1lw9C1HdJ8tIC0DcIrAdiDr9cNL/ycNeTDUuBK+LmlWZRuwbklzKzVYROviHLq1wofZ0LPlmq12pB5OGTtj/l+KuFmBNivHm9MOeehaQ/aSPUiOJMoR0a6PPbCz6hc5kEMUmldRX2prs4gTapdqV6/EvW+Ivj9f0L4q8/0vW+L10JziKFnBEsVOh9/9fMsNS2BXAC+GONd3KDOVfA97QJV8GxzDvTc4F4K2Kor8y3TrnVmfQlWD27eJhyZTTpQPa1fXvm4DQZwBJCoC1mwFh/e5K7a/aYNt5rSzdTlXdR8a/WIsB8XzMtNN4jm6tcLL9/TtDFiHKaOTbSqHfedb7pRDd+vlPSIIpaQRTDOKGCZz9O0c2TaRRdBlf9F48IyzYJBi4n3REL4dLnS1w/GlolZ3ObjvPMdr/i5VLMy2nP/e3p0yQbp/Zzt+Qx83l/SS1HD9cqzseALXc+n5RaVGlRbFLAGY5RvmfWdTAvYqFqtBW62D1rbRFkQDwLwRNtushEBbu3NQbCa98+rG0lHIJZ/NXWtbLpBNIfqvSoOA0sFi9b7toz7anlylys5Przs24rw9RQyKfkE/lpKmOdHTKpgTQFLuBI71hRwOUeqWRpSxt9m598GdfgM6eL+q/ujDc/n62J29LyqDsFyvlwv/Do/0zz8jCdZ2Up3NsbL/KLjZxtMBc8mZEkVjKBqgVlTwNIP0fJz7M20mWX3hi75WfU5Igwt/RO9kZ2wIup3ZPgZpQXekp7vmgXmWgWMzmdll5tuEMXUrwQsz5P1ahStVs5+0hEOOz/rsQlXRAC+MrSsQVWbHa2FmyPgjUK3lwIu5SMKWEuTdVBomavjqziXxsPOSgiamzUxKwO6SJ5Ut7Kcp3TvZVCdMp683GHr2fZZjFtrzwyOgFZTxVYdaZay9sxT1jPzNuXIWK8wNFX48G1jAPew6EtAwI5svqHcyXhK1jJPCUfUrkxLQdlXvxrk7mVsiN6b1MLQc8BHx4cj64l52PmZx5Qv3GBDg28Uykgdk1GW5xFI539lunXsmVQ2UQWspREtweuBtvQDqdykaRCWm2jw89MDfEXdcvChMDLREp5auWIWoK80n3Rl5ckdsooKXowFE3sjLNAiZWspYATn+QvFbVppIdMg56lVqYgtWLasN+4L4Q0BHGk683CEmocvWPnBt0YqWE2tWtC1ICq7orUj8zXfypaTRf1as5C1sV2Z502Ykmo1Mj6sgdnK+/L69RO+X37dGb58kw4PzATOLWVMohz/y8sQyNOMfye0UHJUASMQo4s/Ur9neqlc/jqscLqE9qMdbXb0EpLlfL48yVpWVPKWs535uO1F+EDjwxHI6yq4gLlMyIJjwfcOYbDKPJmPIIzKWMpYmsco1VftIwpbrAbUfVvfkWXWCY9oR7ZROfs5o44RZLV0C95RH0QkZz4Xs0JpUv3yckgZyzzuh0Oclzsp56htBPNXOjt/jPkSEQajBswIVGvgK9N4v0ikkShPIk07rzWkhGrNWgcs25PHPJ+rb+BjerRVlpRdTsslRVxV3ovP1a0MI5cxXzl2W8qVdG1p0pW9+OKHt8v7YbVNRE8VfH0sSZqp4Au7jmnqVeahsp7glOmk5HW1SBhaql7PegG9343BRgDeivud2u3hxlLDEdDycuhYwreoXz7bWTzrl0hfVxtVv5nlSly18rIWVPFGHPLhCnzSFb1CzxKSaMKVN87L84mlkVHHA68HXQU+pmqUVspqy5Ak7CSMkZpCypf3k+ejsny8V/PD+1pMwP2Foc/3CVRsTw45oWru5vX94qaN/75mQsfHdLn/kofWH8snJPFZ10T0jFY9H9TAN+YgwjdOmsLVPk8iXF+7OZIm62o2DN7aG5B5RGHLgx7ytgEJtSatyVda+lmkRX3LtIaHL3hpPB2Fj4mda/4QaFFdBGnpV+x6RUTmlpP36hEF6oeU8XKlJWQ1oOuTuvByo9lTjSRYvdnOCNZovDeqeqPg1ZQw0Ry4GRWshR+Lz5btJ2UfeN2MWVeigK/zs/7n2czoe3UM1AxorZnQ1r7SHMh39fw6jyjwMiOaiJ4PaljsES1nRL8cLT8vCVTELAL5EeWrgTxtEVkeWY7U2l7vOtjLilbbXLaeBeTIE5US7SGoWrBFdWWTFmhRmxZ4n8dL9VuOi0nlev+rAVFfD8zL8TwJ0rm/ZUhZg7tZR5t05alaoiWUeXoUvqgcGedI7co0YmnyNx9RwVL9Fj/y4mzB2ANxFLr8Pcia/J4zP9NXROfHOLHcLYvfQHLTFe0StNqMaRl61lS2Fs6W/Srl4POH5ZIk/qQkBEkLvhLCZNTlFgGxZyaoa5cjbWHtba8IYKupqPod1T7K7/DwBaupCPs1uKPyWjmRbu35zI+XqnWpTl/pesjanjSllbOXLan1xVrf2VONOBwRUBGULajKXbK0chHwSrhqKthSwFpaRP2WPKSAuVkgRn2R8JY+L5SbEf0VK1/C1+IpSmciul4+38eEHzOjX6DFypWDloORA1MLMXP1bG13qU3WKsdynHgxO/p0ne2ONVPB6CEN5ZhIhy+CqaaIUTlp3TmYWY7Ucxw4E4Zug/BKAB7RTNRnZverRPg5Y5Yi1vK0crKOLIvSwfN+73+vs6UOUv2ipUX3fH3SFII2P8agX8LdB/Ny3Fc+2WgGQQ2UCLQWfDnMiTDMyWiPRBqBOjKNn8tjdK7lWQq2tIWgK0PRsp5My4SgC1S9mwfehlbmdL9t/pKILqeXEr6K+Q3fEztgZUArFe18VvQrpMx9yZ2uuI+MCi7rgmdLkoho8ZAGSwHz900CmkA9EumeGOXtaRbildXBSD3vaUlEeEKXZmMgvAKAvSYyajez/KhXmwlDEPVC0JH0iAL2yj2MLz16FcWQndUTFwoOSJ4uw33cN68392WDWbZb/srQMxHhSVdES0BKVWpBUqpYouXvTVO6FpxRPzwVjNqOhHTlTGJkIqxr1kf+5ASviHG4y7YtAJR6/Bp9Zcd0nxl9Pc8VbVk+JCEnZ0ij9bmyzlX4KuU1mL5eht6uHP+9ElPYTAU/J2MR0fMhDfx90t4/nq7BmkC+51uW6a6GkfPaCVIjOljncyCAa11LONb6GVxPKtcMeLVmPD+R8LN6rE++smCnjdsWCy0FYup1WQdvuIF8ayHqqnFfTaGW8LIWTo48K5iMfDLKk5Im8wmkeyYvtFb4GSlZpJrReUb9yv7JcDR6cINcD3xZli2haCJS1wdL0MrxWRQ6jq0NfvnRJm6hMHcBrYTzHOb3PaJnk7GI6LkkyQo9y3Si+edNII+fEyiHfGdMrSvHgSOOegO1JsydZ85KIWjNem4nGSmvTcCS/jo9/QjVt8LQ2rls3wo/L9p7Tb4iosXkKzRxSk6okiC11K8MWUNgsmO09tgCM9rYA477WqDlEEVAbYGvp3o1EBMri6DMz+UxiXLctC0i5bEWauZ5sg46r7UzLQGLTFPdIq1s0nG9PB9n/wxFE73Ggy3Qag9M0CZTSRXMzzlYZZhbW0+srUGGk7HQs4IlYC1Ao7JI/WoK2cqTloJ2ZivKlq0qS8eGSXdoGwPYM969EWHj5PgvUrvROkTLehHQyr8ucFG962LnK6IlWMvxK01Xv0iVzutpY7rzdiww42M79AxBak26siDbA74ytK2dE0gnkUcij0CeNKRkLKCSkS7VraV2+Wv/tki3IIsu+tF88b2fiOh0IqJ7oJZOp1fl1/dyORMaPYJQwlXuhCWXF83LvX5vVvia9wuFqMv2lDwMfe/co35RwREFnIGwBXFu0RuzMOMswhdwZp6O1PLgBiut3jYEsLc2N+MDKdxebQRNwtkDtRaClsea0rWAO6snFC8LPxPNJ39okOXHHKqyDgemthtWDLKR0LOy5AiB1go3a3UivjLw1cLTxMoQ6eBFKpiIbtGL2YVo8i6Q8kKspWVNPmAhonR5nyI3GPyYpZX1wdcz3W/WlKVJWiTmlTYHtdwJC02s4u1o48Q8fF3a1MaWnyqahaGLPVUwf1awBc+oekUQttJXMa/RSKc26TjsxRuzSJdRmcj2k4m34yz+ec1bKtZStlYXXQX82veZSA8/S2De//pgRuqX10Hq14O0DGujtFmeDD1H4WmFqMtxWWqEHl0o94KWO2RpfmUeAqwBXQ7cy+tteNo34prySSpfIjoz9TtZcC3pUuleQNrVKM+ttGNBWNY9if6dlOPLMo0vTXp29cRDvwboSF8eNB+b1ZYbLVXwPX0ebkZwXmxJKcD8CkNflk9JqoGnpZJlPeSHQPmIqXX5OPCaD0mI+OmngjcC8MBn8YbM8h17jmNVc9l7hwjYk/5R+PmZB1SBBmY0uxnV8SCLQs859Uvz0HMxCTEJQQ3OBI5lfelDgtCCumxbtofS6AVeDl0JW2QIyMXH+fTyC7/1/AJ7BemyLDdwc2CqLl5GtkWi3pWWF3/U10eaFopGG3TItcEItDzcrIGSH8uJVk+IGqCV7czGi1kY+nJR7nSs0DLKl+nyGOVrbUbKFdtehFLfXbVytiMFbEG5JWzc4SV64eSa5j3lnAlBm39f6lcLP2tjvMUQmF95y0cRyjq2mr7AOrJfi7+P0DPRQ/0S2eoXwbYmBG2p5qKCySiH+knsL0uT0OUwvYALFwLyJ/Edu1xeYdlvLq/8UmwG4qJkpUK2FPPVyCd6qVNNIaNJY1Z9+f7x8gwG5+s9DE1EdL5eoQq+u10uUbq7wsuDpPLV1g+jCV0WaGU7i349wtBEtHxAgwZfTeV6EAbvpwvXVmWsWnRDjtLJUXs79/GzAYB7jcFGn2gUqZ/c/zkCVa+cVk/Lt4618mi7SRB+fuYZKldTr/M0bUOOum0lVVV8veY23LDSJGRRuNk7lpO8yKij5dMrj4NXg66ELQpH83T+bHcO3kXa9VH2IsaMLSuvoWb5UcQ3N/6d52uREdhZWosKnpfT94mW478ItMWXB1oZukY3AOoDGi7sNkoLNyMIk1JG+tL8aia/RykgR2R1bYjaCsOsI813pIC5ZbpVym40AQtBGXXfgyU61/wiBbw4no//yq0n738xTLMTpDxf1o5aqE4ps2z7aj/lKApf7R8KN19Jf2oSKW0Q6W1K1aso3gJYBF0J28h9+DdX/O1HMC52JhCajqrhiEVuUDVFbIFZSUMPbEDbVEq1eU+bh4vljlj3MvMnGCHQ8jaK33ud5Q2ABm0ehiYiPA5svUcWODMhaA3e3cxaD7weJONPTsrbigDuEWKu7W7nCVileLRKLXwjPlA+CD8XQ+Hn+zl+AtK8DA5PSzBLv7JcVP0u23/40jbcsP4RLSHIYYng7ClfIv/xhQi2om0LvAi6/Cdf+/NHQEYwnoWme8K31PfSpX+kdPm1mNcVac8JWef7EEZ5dvB8VvJ8i0lLBfNjTQVHxpIl0LkKRuk8DE1E860p+XOCvRCz97eUle9rLbwtc8v3nIjVc3lRG4R3qoC59Vay8iVzXwMnYFnp0RC0lz7z8fjhs7W/WvhZG+PlFwAN2Fp9pGqttmUbnvp92lX8lUqWp0WUcTRfg7qmelFZusOXj/Fq4C0/cflTz1zjVF5KRc1AXI5nIO5lkXHgiNI9sTSpyARUeChaquC7q9f3MzM7WZsFrc2olqBdrPcFftVxaL41JdqUIwNhme+do3T+3hPpnyE384tsgXXcgxLi7dRDeCUAj5pg5Rl/eZXtoBBzpEmrTgS2lg8Ugp61uww/F9OgG1sGZEOzlLHGfmUblvotKrn4Mreb1OCnQdWqq5XTQs0efDmwH+ea6uXgRUr3AtK4WeJRs7KdwbN/qAwDcTcIR6I5GgS0i77MA2klFM0f1oAnO+lhYTlp6t6Es3RI1EGg1aAt2y5haCJaPif4uUEHLSEr3x/5vloqNgphad1D1NkdseTErEi9LLzrIPytdI20ZcHXck8w+H7CgjGCoszX8rS2vPqWL+PJR1qY+OXWV7nzGco6mHkaalv6smZHE9FL/WpKl5Q0LY93T1O2Mh/5LH6t36wC38vlBd/LdQ7fbx7/eLMlTXZJNo3yviHdJ/+rjUNfro/+ixsK9+YDvefoBoXYMc8nUQ59DpE8es2aP7G1wfObPfQd9I95ffuG9rIoz/uBTPZptnsWeMCKadHhLHTeenkedolea1vjvja49VZ1q+3dLNOyzxPuqLojXyovhKyde/WD+fLJR0Q2NLXj7NIha+MOqYpln+ZjxK+Zz0Rg2ZEGAE/VklMueyz9gnIo5IxUr1S8SAHz9B7GRRIRqWqYqFIJy1AxcoqOS+fQsfzuozwJ4TM9lyWdLtfFFpVoj+ZybE2acpcOAaWsqeOIaj7RhehEr4cyPF/345Msu2KVD1Z7z/kHfwHpsj7y1UPlhkRnz7By1iLKm5wycxsE4Ik2fSxgug3lbTAVppPv+bBCzV4Y2wxBvx6+IO+IrfCzHL9F6bKuDeZ5+ciSJrSz1rOMtukGUrpWqBmpL6SyWkBMOK/AV4ac+TivB94hY8CWsa+QnKRVDsMgtkKgMnogy/DJVwgY8m8px8uzPDkWfD2dH125En9QgzejGY3lWqCVvpZhaw5wLTQ+3yeaiJbjwOV9lRAlWr4f3KJ8iwC9CyetHbFaGhm5wcYniv4qttXfVdYL2iu89NYxYJSugVrkoUcPzt05oV6QjsLHHJZyuZEVykMTrlAbafXLL+QS0giYFkS1NK0eb8uAL59oJVWvB96RCti0K9H5hDf7CKlhtGa3GLpgy7LlfdWUrgSNV15RwdcnEM+z7+W9K7FJU6UuAu2rDVsdRzcCIaLlIwrRwxmQgtVg6aXzc/neyvooX1qzekZjtz0AiwD/rh/GMLo70v+AGdBeuDkKXwu0ZvsXQuO/xbRxWqx6/WVFWLUud8dCx3K50TL/pX6JCG+6QbSEY+YfCZ+a/wi0FUhL+MqQM4frSAXcbCAkXba2dCHMv8PcDwczL8thgcoiWGt5SDU/9pc+XWj2oAa+LhhPjsKTpu7d42p1DnBvTbFUxy8/WDWXtNIH/ojClwIWm3Ig82CphaaRD68N6bNY6EssnURB6N1FyLSxW09K27ECrlW68glJLb4oF2r2QBlN99qzQtDC+OYbSPUiNaulexOrMEyX64OhytWO2eMGJ+0CK4FJ7Fj+lWmWyuXHXphaAXMEvhyuWyrgc9SfADHcyMO75vE8ra524Y6qOQ0uj5sA/qCG61nOXL6KEPALlpHtJYmWM5rl5h2aOi7taaqZ94eI8CMK5Xss34dsCBm9r6ieZrxbq941FlsXrFFbYRZ0D4sScJRvekEPATkCTC9NK2OBdpH32jVGmxGZgSjKlxOrMuZtUKAeP5YeEdESnsU0eEbzyjEKV6N8AumVypeUYwu+1kxo9E+rx9NKWU2Ry8sXf12z2dHyPUI3KTzP+yy9zyyaZ6SfLtfXzZ74HqLZykTzm9SMyaEZ6UdLl+1qv9Vvna+vMHRy6KrJhsk5S8VvoSH7zV3aSAH3eAGl67XPEO5krfCNhKqTlh3/9cLMxTLrgNFYsPTphr3lxhvowowu0OXYChkjRasde6FnWZbalK+nguVLlXkt9gn4hqawxwxFF5Ukw84lLI3gWX4HV3EeAUrxK/2zCVrT5T4ZqzyoYT42q81otidH3bu/DDF7m3rc/cXS0a5Ys3HgFsuoYK2Mp4i1dilST3OeVblrPuZQ974T21FXIhYJTWdfkhei5n/Vi464mz4vIUeUU8JnWM9+hjAqg7e3vKjliQhPvrLCyQiu/K9Mk/WkLwLHKPTM6ykWhW9UAZNxzg1tSyC7idJ4SFoNTwMQQwjzcdirku5d3OXMZl4W5V0UX+X384Dz+UrP5wXL7Sk1oKIwtBdiluFjOd4r61rp87d2/ozgp6GnI2nviwY/FL5Hx7JOLYQ/mL0x6rWoXUt2AmuNaEeUrQfcdNsMZmf8zUdhYxlORmFmLRRnqdmXD3l+WdSFs6f55CuimGq14KupX2LHUlVbbSMlTFj9WvCNhHyRCo5MyAqP61ZY+XXJh0OU5ww/ISyhOysMztFnI9VssRPI49Dgvk7imIHiHmV5LUl6gRMv/7HGe19dw+oYbU3JZ1VH0/nNAbdy0w2fjuRZRP1qdbJ5VZZRrVpZr1ORNvqMKW8wBryX9cHclC9o79sTDb6ams6OAT9M7owjoSvDxjI9cpzZvYqXR8Dmdefh58/z8LMGzStIR2FjS8ly0KJ8WU7261HWg28xDt+LckziXP5DL4toCW5ZhvcFtUOBY94naWU3r0Un5PtJIp1AHoF66HPWbsi0dtjxxPydLvOZy0T6mK01X2JxMzn7bSyPz+ImOJI+a/MxzHRW5n6Y140ekTrLhyZGovVD1sqMzAvoZ29kEpa0nQp3T/22+HDLz/d/Lsa3n/TGc3n6y33sAgLHcEFdlI7Kn9gVfJIXXCJdyaKLMa8ny5Y8DdISzhda+gzCV4LWC0Uj8PJuEciXW1hm6pZ83l95TKzuLF+8biIxKUu+l/z9LCZvcNDnq0UkeBq6mdL+PsqcrzR7yMf8+3xRgWqBVttwBrXBLZuuTgazJmJZFrmOtUL7LP5VOcnWW1vY+bYTAPe6+0BLkLyynawGstF61hiwVkW5E/bGc2X6/Ny+IHhLl6yLD5ywdTHCz+hv5AKs5ck0pOBI5Mt0IrhRxSyfYvAtx/wvgjJPt/6RqKelE9n90tIQhBf7RhdDNzqWCtY+b5RmfTf4X+Sf6DHj3gadHsVZftf1mc7L30wkfZ4m1tGz9f/qvtDq3BHSrylZcFv5XbXT1kBtb39lAI96w6xPtbFN9KXJKNxaMGdMmYD1yvZ/zDLdK5MJv3lrjVH4eWbogmuBlJ9HL9paOQvELJ3v8aypXxSuzcCXn1vdQaFhWTaytInAsexn2CwVzBvRGs7eTGk++Tnrx3SZ/3y0YRtpFmi1OtpcCqRkZbqmdtUlgcpckLBFoJopX20DHhW7A1sRwCPgOzLuD4pGYOzlZSZhRceAhWk7YGkTsOZN+3fk3mxmlI7AjNo3Zz8TOObn3oXWCmnKNA3EUv2y+nybSQTfiziOwFcqVQJpKJSM8rS6KK2URa+BRH5IBUtDNz3oc4p+ZrI8grN2oyWYxoc/lkvkLix9fuPIYYyjPRcIZQ32LK1rSQAAIABJREFU1k2AekPQOgbshauj0bsWEKfr8gq9JupKP1qn2hi0kxD0WrbyU5DW9OU8gGHZJL7I3OvEwmBeiBrV0yZszZQDmv1c/krVql28LWVlqSmZpoQqeTpXv0R2GFobT+XHHHpEPih599A/6Vu+HKSyNSXsjQc/09jNyLMyunGx3mfl/VbLeX+LGQAv48DnK54zcT/2fyvFtAlb0TkYyFRgi/X+z/kgbIOeaquZZFVzTes2HrzTeULCdgBg+UZpkBwVvu4c2rAmMGTUr9eG8yX3dsKSx/dz/OP3drC6d2OpErJhbDn+OzNDsbgKOHLRtvzIkClQYtbEKy0EjZSx7IoHRw2yGpyRD55mKWHeBlLEzzz5vnAVbIX0NeVqhZq1PPTXAb33nGAiHai8vDzWFa6MUAUiQ4sb5+Vv9ny+4mWIgevGonwmfc2yzdazsXo2rQTgrR+W/DbuhmaWCQs9TM6ALmaBNgJd6SO7XKmcZ8d/J0kO7WJNtLywygu+Fmouf7U0TQE/8qX69cxSkURLCJJy7kFXS9eUL0+zIGwp4gurI5/8NHshFnhlOS0Nhaa1Gyfthg3kRcaBI8uUIBiNdNROOcZha3wDvHTufDmt4S6tbDbvMNV2oIB7mzflPPBNaZ4MlfDXY6xECTHxkJQ203lRJ3FXzn1b6XFIP8ozBfI0pH7kuaeM+YU+o4BlGZEfVb/oXeIgjsK3nEfC0KSU56rbm3jlhZs94888RjcxixdA4q8F2ojSJZEn64K+nC46FOfpy+84qufNqJbp8thqY1YmMgacNW8uS039JvukHHtl92crAHjfb4BqI+/oeoR2FDudL/pifK2OA1o8O/OSSpdlPDNfgqZ0Sh6q6120IxdscPHOqF8iHWBIDWvwlQbuCcw8bTa2dezdQCBFL98bGIbWOinLRULTqHPoWNYReXI9cOQmFOVHbnJrrAbMwsG2Zd+sWu6/2uYNKuAVP70eX76WiQjeLOhZHfwjjO52hcpm/EWWaWjLLGbn/KptqRmkngica3XQxVu7KEtAP8rJ2b2e+tXGgC0FGhkD5qaFo5Eilm1KBS77gPrupS/C0FqHiPThAi/ULOtbSleeizy+6YtcDicnYlnL7bhFlg5pM6ctH94cDlUNv5zO/2ppqI527rWVrZcyC4L7pP5gAHt3Bft8U1azHi8/OQkr7jYO7poyZqjbEsjWeF7kgq7V1epZ/okWuz95ZoWjiTD4EAit8LKVznmmgdwKRWs3EtIWfXmEoRcbc3jDBlYjqLyndK0Xysd/xTBIZC3usy7p+6ZnQVoFZvnbLxOyel9us/60iandN+iwbGRENu/7DSpgzTYMdbcq5ZovcqJO7TiTnDgVWdKE0qNjx4sJWNwiapdoCc1o/eSFXJt8JWHnzRi2ZhrzblgTsKRpQLRmWaN2tb4RSEevE4WhnxYBo/U5os9MKyvbc/IyG3JoE7GkRTf2iJZx+9V7LDirdIcNs60p2sYzZSCA9zb2mxm472S9x0+cMt4a4Naxp3lX8OxMLV2WQcdEQnlYCrRGSXmgrbmQlyJXH1iaRdSnNG+pkTzn9aR/S2kjQGdfn9ygZNEBdI6iENGIhqeeg30owyGRzTOQxcrEx3Mj/qRpKyNCKyve/ZhusZ48yPkaBOD3uW3Y06K7wWh1IuUybQS2m2tRui0TS/j4WPTisrhmWBdMK89L1wypLMVPJvxMlJuEVY4tQCrdWuRrqlmDMDq2YCzzYfmHw5v1nmbvETUoe1GQTN7DNFhGZ0K/zpeNReZfaP5meafsG/jsQAzKkfw3bT1e3CeKMnBnIejM3UPkwQsDlO4bWJrcOv6baisYclvm+1CfWebi6YUm5UX7AtKcNso610hXLOhG1W8Gvmg8WJb1FKylgi3TQulmZ7RIh5WG5H3xpbWD2lXy5DyE7EzomrW71uM8m5Vxj92wotayE9YHs50BeOfWvF1kl16Y/rwlSNaPtTVE3RqWW8yA1szqpgfdqBnl4d7G5CtAzVpgJ/14vrywdm3bZtnoe5+PsMbrWSCubbfCeg4DfTx7f5HVA8CeWbP03sAdXu2dc+YOvNaafUYunJnxwgi8A/C1QJa51kdnQVttWn3R1hdbb4mmrj0r0YJZ1CDy8fPohBfZaA1nP8xaiiQtsy64powVmnYVtTY0NXoi1mFhOwA8ykZNYNjBj6D2UW1Vpl2wMxNteJkateQVT17otXW60eaj8NXys8OjmTJam0SkrwdG59HGIj4b7/NiW7fmGslA1wpN78Z2cF1ax/q+UBfA0zT90DRN/9M0TT8xTdOfmqbpt3XtwWFdTJ3tuEMzVfmI0GCv0CZadtTp3qK2TE2Y20r3dsTiJse3vXuiZ/oaX9XonIHO4efsZhyt1gTkdwVN/mL2tgJHt4gCvhDRb7/dbn8jEf0aIvoXp2n6m8Z2aw0b9O3byZe6ejaksJoLR+aisEoYeoD/FvBGzAJjD0uN4SbKWpuMmNb6de31efA1wGhP8rdorTfnW1zTdnIdHW0ugG+325+73W5//HH8l4joJ4joB0d3bPd3NGvvkdrxC2k/9oznza9qkT1oh4TIooolshSFp5e/V5AW6RYo6y3VscpHIJuZyWy1lWmnxbQ2zaVIxdCQQ/brlR1zNmz5DG37XRqlertbzXXnTawR3j/FU2PA0zT9UiL6W4noD4O870zT9EenafqjRD/Xp3dP633/T9R8iRkhIaJhs0a70gkeL8vZX2BU1/JXbdE10bJp7+JR/p5AWqRboKy8Vfz0cFn+euXdNhvq1rbTYlofp8hnegJlsl8v7rvxq3kRDrzfhyy/W6u57vS4pg23TRsPWRjA0zR9HxH9fiL6sdvttiDs7Xb77u12++Hb7fbDRL+gZx/flu3kM79e6378FkBHXFCagb3/m9ywWXu19YAt9zHyaXJVH8leWMX6cT3rl8c3A1ciokvzncdhgywE4GmaPtEdvv/p7Xb7L8d2qdjxqWfsc+uPTLGaC40HVVN1jwBqy7ILRUF9Os//1rpvLRMFs1auBexSjRelj/KH2ypP2/HtbYF56w4Ylu5by+DMdhaZBT0R0Y8T0U/cbrffNb5L78RGfbl3+KPRgNpF3daOQaGmow+xWOHiLcGX2nnU8VVjxaemwi2AR0PrJWR/RqFl7dyzyPp8a2gi0J4Xao7YMnwd/23w9mW9IUM+NbbD69JbsIgC/ruJ6J8hor9/mqY/8fj3o4P7tS9rHQep8b2C1V5YsvW08umLR+OFFNatHYvkSYmXIVWiZXKzVaueBUgL9si0tyTT90U/shED/vmcQJpVp9FuzM/lZI/7lu9wy02oLMNhbSlqqx4R0fWivCEHLBus75vnfmVvt9sfove4B1jULnR/l8r73vIjL756GfB3uZxme0Ff6TSbmXylszo7k5dd1ju5sz4t36H2z6fX0o8z6bNWT6T/Ds4ir5S16iTbmEDfCpguNIdUpMlP9Aqa8WPLysfO/Zd2vxHnqB7PjwCem6Z6NVNvUKKT5twGAv4tpb1iuHo3irXGNgf3ivtZr2Q72wkr9ZCzx99eu9sGbfMvoW/XQePBqT4krmqhi1JmqURNeNrLQzOekRoG52h8FJW3mJCBJFK90qfnp+XhnaG61uzkyIz06HBCxpg/ORchunKg5EdWCGRmVb8pcFcs6fuoNgjA7+9ORbXa6fjZ/QbNsv6VyBqDWo4r4TCbVdYLwXljYLzNxf2DpWQiKsdTSDzMKesrVCyqDi1FipiEZGbsVU50ahnWlmCP3hCUOvIGg7/loSVIsgHvBsj7LL1hCg38DTCP/gYsy4wRW78ddfXDxQli8m1aD3A22jcUZeBABby3mWi1z5wBNnINnLW3bdKfBVKvrGfa2FRkzMoa45ot/bCuW5mJNjxNXsgRZSxFDfr06TwPs1rvJArfetBDcETjusivTOfg11S1dTOQDT8/+3Jevk+zTkjjQJVwPoFy0qd19xAMTV/POhSvdA4oYfziaidV1czZcFdH9FgD/OaBbT3mZKztLATdYhsCv3VTjuhDApzyNaFnTxnPLxYY6JqPkq4rA/2CckMwjISaTxSDqmeBi7aECYebpQ5lHe+Yn2sw5V3TVDFaIqQtG4qoX81mdU/KxCu0+Un2s/LqRcLcII//jCRotTDyazKWD92oZWCcDlFvtIXr24J11XO/UqUHA9jrzJv6NPrbwFnUHMbe2FPWes745CZnnM4sAmJPFWvlkTIu9ZV2Sxj10zkXhpZQBq5ncDzTEpayOW3Ml+dF4fvJOJY3EegG45Moz+15w4Lg591wRWHr3TgFlbHchCM6M1la5LfSMgQEfV9OdEE349kbfVm3Ns3zm6qzZ2bkgf2OFPAA67HdZEsYOjMuE1S/kbt5eczLRseq5PKMK51ndZdKm51HlIx14bbqRMKZ3jiiEYbmoNLgNKtHc7BqkEUQ1sLLCLokyntA9tR4Rs+V8DORswUlCg2jmyNPnsvPyPpuCJ8tS5C4Gka/JZkeUbHexCzL36sDzqeVeTpUS/hZttP9qVSRJ3Hva2h0BQBv+RyXnTbTAmunzPVynt39Ru7WFwAMKGYNqDIdt2eMgT1k0qXAUrso87+yjFVH86OlaRO1zneYZNYAE+khaQS8Us4bs9VCzjxPawuBHR0j6Fo3Gp9oGX6e+OclPw/rJgulnUGad+PEzQA0vwHUfhs1ytZK92AcuTle+NRuyiM39TXzTqLjyXsWsab1X3HzDhVwz+nFlU2v9QVLtGMpUL1OVO3iyz+aHZoJvc0sAjrtAhyVaxE1DXzJMHRkGdCsfqCMzPfGfxGUvVC0JyxLWU3NWwbDz9Z57WfmfXZRZSzM++7Xbr/afXjIg26N9Qg1d7WOE2o3tncI4GINM9t6f7lGP+XoscRAznjUliREJlZp51qaTNdC1s8uizv6eQj6/rU0J2JFL7wy/OyFrjNh6PM8pIpmQyNgIRVshaI1JczrWUBE4NXgi2Y9a9CNqF8efl5UlmpWi0JEP2tNYVufpfi8y0/oej65N5Za+v37vAQzT28d7gmDOzMxc5R22Q2s92crAbhmgw2i+Sc36o3svGbZUsGRMAyqL9MCExesSVhaegTM0ZnQqC2siOf+yrjbVbsYE+njwLJcFs5am1oYuiQz2DzPKTYT2oOwBkikctEYsFae9w/5tiaIpceAT48bFQTa7A2QB1mZFlXGzPgELD7Oah0j82dGz3+j2s1uNmp0uZzwNpRoTknkehOxHnNgVgV1z8bq2bQDBbzmu94R4jWz/ay81rtLZaJFZKLHQoEqP3JtiZKXLlWAbKdczJ7ncmA1Ejb0LsiemtKUMYIFgsnDvL2ONRhrECblXIMrerkI0sgfV9u8r7Lf2mtx1a92IxO9AYpCVr5o+R1RbtyiE7CsSE92bDi6sQ26OV4o5NpNOLIWVcq99kFoK7hrWxHAIxRsq8/GD7EmZJNVwZE7VWZX5e5XrunVLhhaaFgLmWnjyi//rwsKB27kQnM5zS+KswsnumCXMhpc0QXYKy/9gwt4mYzFoaOpYO5mAa5HHoIrUqy8rDWOrAGbCMO4/JXw1fqO4Pssy9Wv7Jx8Mdpn4d1hyPLcP/oM+bm4GbieXzeAy5n8OJyMZvyj3w1Pt6JNWnoI7izypW7CkbymmOleXo2l/aFIadRJNOI6BvgrK+BRYeT+s9NmrtE/mV/TtZayqJ4cA774Y0laenR50Ssdg1a2j28C5heyy+k0X4dpXWgllEmUsy78SDFZ6kvrx3k+I9qDMFKR6HiLEDS6GZDHkRA0VL/8PUWAJZq/59rNl7yRQtELSxkDWN/OtNiAg2h5o+oNzWgRHm7o5lcLaVtLmub9UD6RcjNeG7Gzlh2ha9+wiah73d64jS87CEETvZdwQthWmMwAF+ILi48Naz96HbTZtlEbV3FRNFUMT7MULBl5llqy4M6sNQTNjz+Jv0gh8+5p3UaQ9vzxvsp+o9eCQs8L9Ys6J9PkMbrpQXWsMkThGfJy/JebF2ZGpoWLpU8vXVu6BNMv574PZOl5ef5gl3rPdgJgyzJz4DPhh85qvFYFR+4gUZlFGAmP88hxIak6y3E2HR3P2gXhueV42fwCMlMPbBz4hpRoMaScZDqvZ4U5Sxovp7UNFFkkFM27h0LQSIGSSEeKNaqMCfiQ/sk4ln0PWUT9Wu8xgTre0IEGcEUZX06viX+XEx4uKef3v8swM/qua+mRMeHs0qXQul9rEpanXr1Q9ah5LmoD2Wt47whsu7+dAnjtqeOdwhstX+BOExs+X07PH6KcCV3749cmnXgXGQxv3Id5CPtxfP7WfFcsBEmiOQyRYrLUlKaevboapIlCs6KLedDjUCZagtgCciQELctkQ9Bh9SvfLy1ddtb6TMjI02Ar653m8wz4jZ8MJ0fCzGiIRZq3RCmajmwW+Wp5EEMkP1pWExDasF6VRXbBqrVxsn0DALfAtdcdT+ANrf1SRMZOvO5Elxst7mKxJtEU62UGzVz6vBs+aGeznNkFRZ0VzWahXtCFVIOtVEhWqFlTZujCj+qAevJRezIkrUHsLNI5qC0QI8CifxqcZRkS5dGx9GfCV0YsinmRiOhnROJvRBkDKN8nX90vh3KylK9m52BGN7vFT3ZM2UqftcMiXdfL6TUBq0TGojf/0bHfqHVnV4QBpdEML7Id7SMS8RV7E7vQ2O509h9xh8rwNJmv+Szp/C+JNGHXy5nO5ytdLyc6na50oROd6PrKpxOd6PL4m08vF4KTSC8XmFLmTNdn3ZefV3+ujxL3Opd5+vmRfvlMtzPRVG5ITuw9KBfVK3hvruwv0fw3dmV/z+DY+2zQZ/GwiZSvxhUlvuwbVo8f9zJtprQ10Use87QwfOWxBCaJPFmOQB3Nf8Sv8DMXjThShG4gSzovO4f1HMyvdD10jdK1tmQ7zSFora4GXw/Wux3v9Tq2XgR2IwB/Q/PLwZq+OrYNLr6wTA2Es/cLz3on+kxEp/OVLpcTnc4SonP4nRkVoulLOMdAS1TAvIR2KXNi/i+nE50uV7qeic4SphyqCIQc1Cdavuc8j7+HRPi9l+dXpdzDVAjzvj8sAtsL4W+tlo5MtmGBt5zL4zB8eaNa1ACF8PmxBVj5otBf6YvY8aNcCT/fhzuWYeaiMqMqt5xrcLWWLmnpWIELhfwcdmLp3kMYpNWAsyd8w3V6kX37XbJ2pIAt8yimlan1rRTjlmm6tvtI6SJ4P/9Oz7zr5USn8wOG1xOdThyMc4VbftSR9Bcsy4XgGgZtUbtIHfMLzyy9PJzh8pk+IXBKpcvVLPpr5aELPH+f+bkzvEaEIfzN5Q4qTQ17QM0AF5k221k7R8ch5csdIvgiRYvKa2URnLVhB+SXpZe1v89d2AT8LKWqLVeSflAZXSEv09H4rxzCKfb5cnrJegTEjPrVzlGdHlyEPqw5OrzCCKCiDvVrZ6eTsIj6vpkNvqJjwZExkpoveqM974wJP6XIW6PL071ZmdbFR5vIhfo2S2djwXBvaCJd5WTywIX5afyiTqTDhOcLQztDaROeJPhkWe5ejgUji8yC9uDLfcnNNqBJGEtDihYpVk/hZvKAfxm1Rd/lebr+fY6CWWsPlUFjzYsy13KTGrgr9Cxyjaq9PtXOq3nHtqEC7hmGRn4jsrNzH6yxWUvBat3T6lrKuISdztfFOHAxTX1y5SvT+RgyV8koPH3vGgslM9XMj5HilmFoors6uZ4/z8PQ8gJrhaat950rUZ7ufXWsPKaQiwq+XJfQulzxt08LSdeqX+krqoJlnlS+RMlxX3ku1SrKJ9IVLpF/o8XriRd9O9Nz8tV97HcJTQumaGcrC8xy6VJkTFlLX9wggJUPiwlY3hhwZHxXM6scAm8axrJwVFSVetYS1e3uCjYEMLJIOHgUuG90v1x2tMjLyY4BmyHoV95nMf5brGYcmMNSghaFp+d+5qFqPulKg/bLN0s/f4sul890psdkLPn6UWga/eXHMpSMwtSyDs9HEAdmjgcH6hf7RPlYDvqlSEXL07SQMxHBPZ6r4IsiD0gFS3hKNRv5x+sJIPMnHxXjY74aTGVYOgLmSDg5O17Mbw5mdjnRYgesDFyjoWfNTyZUfWH/qm3NaGnfMPfOADzCNGBbUtROdvO8cpk0IhvIXvMPEF+vJ6ITV6b+TGdtRvNcDWM1fc+7PutzX1IRL9NfKpiI9MlYmgqW75l875BqLuWIlmDmdYiV0coCK7d26kcYhHDWUHtRBczBS7Qc8yWqhC8CpVSr2rGV5pVnUOaTr4r6tWBaYKdBVoKZCKtWXJ//ZnS1jdKJlAlYGfMAa+VpZYaEmyP7M0vF22LjlfEgAN/o9QZYatVTszzfopb0M0olg+aLWe+kFwb1QtGpEPRDb5XQ8/ny+KuHjFuWDhWf93KvTkvQzn3NJ2ZxFfxSx/fJWMVvWZIUUsHlvZGAvijnF8JA56Yp4iQ4I2pYKl3+kVtpSCHXKuASbiZiavcsQs6lYg18I1DWYEriWFO6Mo3llZ2v+IMXMHTtvc01YHMVzeHN/Zbypb4bZkZtifHf2QMYrDCz9lfWRedR5dzFRu0BPWoGdPEb6/cKCjgLw0qpl6pbCWgLuJ5a9SAszyOhaSMETZcTyXFgIqLTCS//wWO2d2fa0qFXngbt5brhe17xu/TF275wyCMVTOCvVME8LTreK6GKVC5aWxy0AuHziejy8HM+E124rw6KGP0SvLXAMtxc+kakLDXqCV8Ues7CWWuf9ZmrXyILuMvvLAerDFXbYWlf5c7Lz8FshrTl+O/lTHD8l0hPs8r2gq+nikO/Ja1QLVAzP+D+4ekdz4LubSvMhC5l+d9oeXQeaRfe5d5/fOVuODI70r5g4DEpfoxmNC/HsPT6qO35hez8WpLEX06F+nnmyWNUDuVznxkFyHzzHbOkwiSi2faVRHOlemZpcga0YA0sh3yU4/NpCd9P5wr4ctPK8U7LdOSDlyVRTn7WKI19rmjfZyIMx8hTiqK/m9olTZotws+eAkZWC9XotW5IODpj0WcErLlL493QTfIAsxRnJgwdrVMsMtjKfYGJWK2CnAJdQKFRmX8Wf5HfgKExWzm7OTaj+d64NaOZq1srjC2VhhoCP5VQ+n1G9POTQhdtmYaUMXoPtXIov9EmuoPt5l0XHm1ak7D410KmS/sEjlG4uZzD8d7iXLvpKPlfOuUy6drNTlT9nl+PHUQbb8zHav3w8avscrZ0VuUu83wwywetLLafRBYNPVvDq9FyGngvRh40WTCqRFuo31MZ67YSgInaIDzKWuha0YwVYpZpWtfCIegzyXFgIpptSxkB7b0Ze+kQ8kUkZjFTCTPrm3rIMen5mPIjTY4F3wvqNyfWNpS8HDnlUH4nk+PCn873DTvguQHjyC9oppbZ9dsFL5Gp6BfgI4rBVwOnpZq9crwPLO0FX3puvPEKJesTsfgxgiyaZCXTZZ0eS5qKwQiXBVov1Bwpi0zzYVkaxtL4ryDqpAaYY8aMVwQwUZty1dJLWgTwVpkkjOVnjRQrcp+FMCn1ZDoCkBgHJqLnrlhR0BLNoalBu9QvhsaLiebjv/dzfTLX6dk+3p6SiF67Y/GLM38vT0r6q0Nz08oV+zYRfQXSI8Y/H3E+PY4tEMu0M9FzDDnUvLhOS+iWNBW85RidW9C04GtB2gOsBn6ZR+wvzcd+ieYwjKhfOQ4sfcn6xfezfQDmYpklTVf05LPZVpTgbxao2bJdjU9kisyAbrFtoLwygHuat7woE9pOKHArrEwsz+I5gimJNAT0SGj66eO+LWVZD1zukk9nfRtIBFoiGRa+wguGVef+0pZriuUNgLU95b2NuQqehaK9LSf5+2ZB1gtBW+maFThcRBqAJwIxEYaxlY7KcZPQJQqCt/Rdwq4Gvt6x5Uemke0non6JEACdCVCEJm8tjz0wyzqyHSv8vBj/RaDVIKqV80LQvJ2s2k1ZhP69J2BlQ971tgGANdiNClF3CjNr4K2tkw1BZyBc2hRhaCJaPB1Jm518d61fALJwrlXB/Ph5YeKPKkTLkkgcZyAr1fGZdMVrAZorcxQtcS5EBcRojFhCdzGDWjYH+imh+2yz9E9GDhCMJZB7wffboG4EzkbaC77Ldb+ZpUNS/aKx3yhkNTCjdqDfy2m5/IiP/1oKWMsnpwzyQ0reRfzrbgiMMk2OB/eC6Zt+HGEEqJ3AmfbdeSIWr6/BU2tLAhxBVvP/THuFoYtdzyei0wt83tIhbalSyUNjZvP6Mn2ucC048+OyLOlK5ycMrufPRMQUHn8/UVp5b14Nzk1+Pqi+rAeG4FRDyrekKWAu30YOY76EqeQR4ZC1NBW6pS/8HEG2pMu0LHwlaLUQs0zPKOPzfMvJMvFqCdU4ZIkwTMtxMVT/nr4EM68j20Hql8N3EX72FHAkXStjgTRTrhrK2S0oW32OfWLSG1yGFHljM29+9FvTaJkfQuRHgcqVtKfP+XKkiMWWGi3zssdyuVJsGRTIe4znzR7UII95WkkvLhBYJHzkhZ1oCV2tHQ9UvIz0c1rWnc4vaJYlQ+Uf0WvJUPmHyhUfk+xzsA+L90K+pqjytd4j9J7L90z2h2jZJ6LZlpPoiUdEL/VajnmZcszB/ErH9bXfkbXDlQVmzWD4uVg0wmqVjZapvXSGQByB4MBr98L6QRnd269kLcuLZPki91onYiVM+7ylikHvcEbBEjifhZlFmwufZyK2EQd/OENkB6u7S/xowWJySdLL17y+TM8ug7q/rIAK5u8RP7ZC0ShfszIRi1/4uZXPA/mT349IXkln6nkCZd0nFP3/7Z1NqC1detf/9e7d770BCUHMIKZDIihiCNgBCYFMJGTQJiFOFeJI6IlCRCXoSBw4cCKZZNKoKESU4AdIQCRgmiBotKMxJLRCEMUYoRUJGqTv+57zloPaa++nnno+11r1cc6pP1xO1Vpx+L3lAAAgAElEQVTP+tj7nFu//X/Wqtqlr6KLUK7BuBzzcg7IKHzpuRbjuWNeJqSupS9ckNxvkbQOq7lfbe1Xuv+dp5h5f8vx52CW0s9Fi/SzlWqOpqAtE0BjVksvR7+CsFaZTyfraUcAZ1UDzhU2YllpyVJvgVOr0+IBv70L5wtwv33n9h/8loaWb0Oag3F57+/jP7/2HGgtxQw8Ut92uvlRNt/odZm3ucC+N5geS+81Fd/hXKAn6T1rL0AypBLPP0RpfdG4MrfoeJJrpz9LDC/XwMvPuTP14KuloKVyLdaAvXTPr7apKrMO6639Tn1fFudFkpsOg/n5Mnv4hph+LpJS0Va5FufBmNdpfaRhHbHzkht9YnW1UI2sMbdpZwB70JMIVSPPIfNxlHVgVE5HgiIf2gM1by/9hNAH2Q1dxF2wBFruWul6MYfzVP9wwfzcc9h0E5b2cI6p7RLEAPR7g+8v+N7RdPwNVg4hFlhCtkUFDk/KOR1fGpPCmMdobWi9Vma5YAm0dO40hoOQtnnH2r8T2mnH79iYViwr03Y9c/drQ1Zzu/LjKCXIe+5XcuMSmPnmqwd8Wfo564C9Oi3GquvmjD8NHFtlXv26a7yeVgRwLTy32qCl9dGhb6+LaAoaTlzYAV+B6/PslqTyDUnlnmBAd61zaOouGJjDlJ5f8HTr48LK5Z3TUsqaflED1X1d75aKvoLsip4akGDyXrXc0xtV+T1YwNQ+SGllwNIdR/9kJeDSYw5mCbTauQfHKHB5uQRZrY/bTz/1vExFL4/lh2PwOFpeju0Utdw3ILvpUg5g8ehJMf38aOQ7Xc/hWu1rU9Dd09e9PiV7/fRfi14RwEDMNmZcsLVuzF2uRcEO68Ce45AAKvXB66S3zHPH/CLNy6/AYzf0/BuS9LXc+fcBTz/nLvgxhYczlcBd+pYA/8ziJcCXcaU16/s5SUUDwPU9MBS48t9Vr8dNXiEDnLpbz+GWc8nhWulpOvfo/3kJuPxYSkF76WepLAPf97e2HL7S7UjcPdMYAl8p9Uz/fjhktdQzUKCog1lqr23KKudT2yW0yzHtm6afAcxvPdK++9dywBJUJSBacZaaeRhd/13zCxiyfde96JUBXNQrlVzT50oP5MhIgySv06DLzzXYan3e0tBF9BuSLpfY9wFP3S4f0jG1mz9AQ9vEVcTT3HQT1lRfuQZ9v7/mM/l7g6l7lN63UnaBLB7PH/DRImtcr110DjyOA5aWcajSMsn18vOM8/XG4POhfWB+LD1wA5hvrvJSvcv08HXRB48r4tmZzLjlnPbNyxdPviqKOF2pXKrLlPF6zymHOeVNMNIu07aH+81rIwADOUe6Bgg7pZaLsl3Vulh+ztuVei89XVJWwCINXf6vL8E4Txt7KWpg6XSXrth2waWPKXbpgvmxtCsawHw9+NFw7lqzKWhp41UNNGl7K+WsxXK3DKUNWEyRl4bmQJXKaLkG1Ah8i6sF7I1WXhr69tN+4IbseLU1YWDuQjXHrLlfCeDerupyzsv55qupg5v7lb56MJJ+1uoiKWhpPKufKmnQyzyAg5/3cNBc9S90QwAD9RDMtos+VcuiorARK6pI5t2K8UALck7jpdTmvfz2yH/lCxqA4oBlmM4dKE9dy06X1j3Ol31E1pOl9bB5/x/j46mzqVxbD+bvbe3/AArB0g9PJUOo11LJVhoakFPRIOfRDwLaa5fcLi+3XCoHJLAE6HtW7gH3vVBupKk/fQfxgRscrD487c1agLTZSt6gxevo+TLNvYS2++SrKSjmgDW48j6ktrzcArOnKjhvuVnKg3lR8yeMrQEM6DCNut7IOrA3ltdnoF3EcUhApPVSP1pbaVqSewKLXVywL6BPxlq6YC1tvHTBPI6nzfi3IEmbsPixDtqre0xvTcJ74ONvTP9x0vfHRso9FWjwC58GYsnhepCm8/OuBREXLEGX/9TAW86jrteLi8CX/ORPu6K7nj/gYwZj+9aiDJi1uOlXYvcPyHAWy7Vbj6R7fyMOuMa9RsCZgasYx9d/tU8M1i1G26zf9tAOAI6q525oCdA1z6RuEDfbgDx16SW1OODZ8fyWJOqCASy+JeleDvrFCA8HOw39SDXzzVEcoFoKOwba5YcBfjxz25f5pqzFb5S7VG3jVSnPpJr570Iq5yDmMZYrlqBb44DpuQXdUs/BW44liAJ94au56ety09WHy5TT9kD6Ae9EOE5t48975sAtY2sOl/av9Tkr1zZfFUVcbdQBa2URiFtttX5E1QJRs/CZ24+2c7/AbgCuccFR2Ea08v3AkRgepwGal1sOGAj0cTtgLhiYb6aamkr39C4d7FR3Yec8Ze2noqfyCGgfY38A8A6f3F4yuwVK+takVmkQ5Y6X6qK0k/qgztdyx6UthPG0efM5SXUaZKUyC7Zg5xp835E+JIdL14cpfInzfboAn7xfOl+eQo6s+3oOV3OwUh+0nh4v2xrp7Nvar/i1g0+D7GQt0HqxWhuprRXXrMj9vxZgJfWabL8XfWAH3Ko1vns4KQuK0lASYDUIa2CWxpw5svnXFAKYrQXrtxTJD+Ao57SunNN2VNrTs3jKWj623Pl8HPFbk+4FkF2jVp6NiarnTmpLfL4SdOkxd7xSmeV0wc41J8vnJjlprW/gvulqOn64R+DhQgHZgZYYno3hQORxVh+eMy7ttGUbLQM0PXiDvGgqD7b02IOn5VJ78mu/rC+TNZH11513BHAEcJE0cemn9zqwI+v3JgGWlnspaK8PD8L8fOGmruDPhwZwXwvWbhuajrnTlV1qEU9LT9O5iHVaKvoTvMPkdfVNXVJ/dGf0bFPWYkYVmn2oMeqpYwU7j7hfXl7aQTjPzF069hxv+Sm53NLec720X+k+30g6+h2/3egjfHj3WOf9MG3Hm51TqGqp5+iasLW2S8dtccZ05zN1v7MHb2hOlqd7tRQylHip3HLRXmxY2fVfSS3Q3P5LH3Z2wBIcrVuSWlyptQ7M+6VjNuyG9qSliWkdSAxIHHe2HAhS3eznZXZLUhF3wYDsZq26xxSWqWcL5loqmtblPgCQndFkU1YIwldMtyhdyHFUEpx5GloCsQZdWg7WDsJY2pz4fKQ6ywFrQLYcsFbvuWMpHX075vC1djxb6eWa1PPyeAlRWqY54wjgAZDHThL3y7920Eo9S1COpJ8zKWirD+mfKR5Qm14u52veftSulQBsPcmklyLrxdHbkaS2hrwQ6Y9McjC8Pw5QbUzNAUM5X4B5fktSEd0RbbtgfU13PuV5qpjugp7a6OvIGmjpmi8/53Wzfm4Qvjw9A98Q7hGWfp/0vt9yzN9LClYOWQhl9Ly0t9Z9aT0g/86z6fCMA74KdZLj5eeSS+4B3/dL+H64PBwtd7uRHcvFFQPyuqx1a1Lknl/JGdNzsY7c9wso7rfIcr+AfD2KuNcIrLuJMiOyQYoDtkYWlK0vefD0KaIMXNEB0xeQ/cKF2p3IHlQzT8XS5iZ0zSU1kS6cYGWaA9agKkHdcsGLPh4uGJDXgou024amrmynO7W5Ltpmxqit+4CPH1C+DffJeywhnN3tzFPLvK6899rvQ3LAHoyldkDsupB1wBHwlnIO4lrXK9WRx1RKzpfCVwLsJ7d0czkGINZx18r7BGA62KltvTMu6uJ+tbJaByyZS6tv7+8xxLEa2ke+/ehIj69cFcBUrd965KWhA6BU4yPrzJ2/HUnrx3LItFyCrwfnoAsGcP+qwiK68YlvgpqmYzvdqfyyiLXWgK0xrLoPeIcLnphLDkKY6gr7O3+BOTQ0B0xBLMVY6WZeDuGYvKaQ+N+WBF8NulIZd8sWnGvgeyu31nw5UDNrtpk67dahFmcsbfqSvnJQdL+W27UgqzldLVZrx+ulMaJQFjvz0scecGthm21X78I3AjCwzv21PW9bsq5yybcp+uFN61aCabbchC89vtzulL0VESBfLlNguTBEnS5fD5aepkX7pW2lY6+ObtIq4uno2VgChHElX95QZH3nL39fqaIOOAJdWs77oG+D9Tcn/Z3RMskN8w8evcAr1b9jfbEvXKDw/eT952aPmLTg+wk+nh23g9laO65zxvT43u628YruzRDdL5XmSHm9B1Srv1ap/dSmnzMDW314AF3DUU/aEMBADpi9gF2zDlyprCv23K8EU16uxWufH2ZjPlzwvZrenkQachdc9Hwvm8OatpPWh/ktR7St90hL657jRxn/xiaWL75V3XdHb3U7UK0k15xpy6VBl9bzcskFS4C1nK1UDyFecb4ABNjx9LH+wIsSJ4FQq5McbBEfT5qf1hc/BnB/6AYguN8pcH7sgVKDnuZkI7D1AF8Fa8nG8+OjqX0z1w6XnB7ru5IrLWU168DaNyI5c+V/G7XvpuZmtfEkp6uVS07NccHPT+TqflkCMXq7kRbL09hWKlrqh6/zTrud55uvuCuWXDJuac6Pv/Epnq/A5dL4cayAg0+PumHJ+ba4XwrlyPy08wsrk6BbfmpO2AKx5YqVutia79Ltlnp6u5HkWqlr/gD56VmWu+Wg5h8ItPuBdSd8gzx/5rP0pQuWk80AkgMcRl0W8N3ZGbn9qHVQPobWX5+d1Dt95o/uQI7AuhboFsS14+DtSJm/Ac+pWrHaxVgq81wwsLgtqeyKpg+0kG43ugiOVYudynV3C8iwnD/96h3esXoJ9BKEn2+XwdmacNkdXb5L+ErS0RSEXLSOf9CR0tCai9WgK2Uzyjz4RiyQGD5HLsn90mMNxi3gBTsPwveT95PrldZ8Nfg+7gN+gNG691eD5nINOJZ6foy9bFvqxHGeL7ONV7O9Gdp9v15KOQpIr99apZyx9sQrK9YDpGf/a0Da7zamnQAM1IEzek8wv79Xu983Mo+KB3Rk3lXNqUYdMC/nsVLfvOwKSA/neC5fWXjrWEsnLx1pKZffCO8xllJbDt2n24XLugUJUJwv5I1Zk24Qfn/7FiXtfdMuKBy6vIw7YM35cvBrtyKV8yLNCUu/iogDlqAcBbHmeoHlQzhIf/xbjbQ1XwAifDn4JPhG1oR5P/4tTXbquZyLaXLtoRuRnc/ZsqgD5mVSP9q/sEZhEH7csv5bFIFm1P32044A1lTjgrW2mjJp6MCQVNbwEiij/dJ23AFDOOfxvJy74OsI7eEcAICLDFqeTp7XyeXS7UfPuMzSyFJb2u4TvLt9IFg+F3rqC7NYDcJXPONjAM+XKy6Xsns6+NQsyQHzD0meA7bKFr8nEgfoLtiT54AtJ6yVWeAFUq7Xgq98n68O3wg0I+nrTFo6k7a+g5lsvBK/79dyqGBlPBZKucU9CeAZpdtEHS8fhNZJwNbaRMaMzqNeOwN4i41WlrhDpmXacSANrf3haUDU4iL9Si5YAzTvZxbzSEXz25IALO4N1m83kp/zTKWt+drrvLKj5s74AwBpE1eBMHXND32MsrHsw7vJ/V+envH89BkuF0wpaQ5aaa0XpD7qgK24UgYs/xRpHS2T5LnfiBOOOGDucjloJRgH1nsB3OCqg02D7/xe36meAzd6S5EEagmo2gM3eJqapp6LxNuOpmDf0Wpu1ivPgjbjdKtcsZeKXuPLFyLut/8TtA7ogAHbBa+Rhj6YLNBKsfxCfFXKeV8StDH/usIi+oQsYHm7Ed3JLKWqH8CWn7DFH3PpbcR6gD7zweD2AUMk5yx4+vH02f1B/8GPXnJf3o7lSDbDirfaaH83EnRpeQa85WflWrDmfIH5/oD8ZigGOgWKdBzrnt3HPORxeL9W6vk+Lkk9m7cdlZ+Wo/XKIcRokiBvOe8q0FpPi5I6ytrwouM8epJrJRLRvL43RO/7g6OA7bgbugxbFHWwEjij/UuOiNelyx8umH5d4WwalyUYAX29F7DhJ8Hwk/IM58UY83TzFc+L2JKalh5ZWS590iYuYdLLzVkQHl/JXe0Tae/9DqiT5nXWRqzS/+ONefTvvSY+f+lYgi59TbzOAm+JrUw5081WGnytdHEkLS255PpzeR7Tr2kJdgCLjVfiFy5kHK1WbvVVDVFBqfbSJweutdPPvdwvbbf7oyiLIiCWAJd1qrU7prlD5mPzeShPxeJTjbgRz/XQMs3l8r5r4DvT4+sKueiGrPk05V3QRdbjJ7X1Y6k/DZwasLWNWaWvC55wXwPGFc+YP0ELl+kDh5qS1n7HGoglsHrQLeUgdUXRz7jSHMlrXJRz91vKOFB5eQS8t/IW+GrApPUAFvAtu6Ol9rU7nrNwBogjZ0+8AiBvvCo/OWAlaHqwlfrSFIG11zYE9E9J0B7pZ0/rfUvSBgAu8oCa2Y2cvSeYx2ThXsZErF1N914/2u/Xqvfq1PLbxYAW00dWMjZqEJXqOcAz9w9bY1kQniArg3gSWQMGyKXz9h3HsyGZG6aQoZC1QMw/WD0JPymMAflvQXLBkqS3LOJ+pZ9S+jmxCctON8ubrYAlTCVgWrcaae21dV2tLyk+sgOaul9ATj0/83t+uZuVYCuVU2ltpHrPJUuqdtB89zPvtMi6nahmZ7OnLEjbQL8hgIH6j+ySvBRypg96dZNAHyCq5EatmIg0lxsdV3PBVt0TcN+QdX3GZ7dbkfiu6MuFruna7w2t52At4HzGdbGZKrOj2epr0se3y+E0YzcFzUWm/XzFww0/K2lpfk6hyR2w5nzp76qHC+YxWkpac8KeA6bgZecFvEB7yrkXfC0XXet0p1/RA7bmvcLR1LPmcIH531oEohlQhmHaosijJ7mkTxG0jfZJROq7Zve1No+8NgawJ+/biKIP8MiModVxF83HMrbkeG41Ksm4W/1a9dx5eXVXTFdM4csagOm2HSCwoQl8I420u/mRmpuXz79zuMja0fzJDbTeOi9NQT+2hj3OpxJ6+bzg+fKMy+UJ1+fyGj67P0FrlpaWHDB9f6U4DXYcxJYLjkpzvxEn7K0Dl3ICXkB2vRxcUfiWFDLgO+Oa25asFLf3QYCmmgFl45jwwA3znl8Lwlodj5Nkud8IvKN9L8TXRzlANfX4koReQO3zyWQnAGeAmYmV2tWkoSu/ISkypYwiDtjqtyk9PX9CFlV5Qpb0tYWSqOu1djdLD9+YdlfLj5qcQOuv82r11WJrwzQtjQLiW5zodIE5eLm7pR+IJOcr/elaf1vS340EW15+Feq89DMBsZRuBtDN9bbAWd+cVeeMedn0KxHqvAduSN/1W35KYPTqeH3U/WZU1TcPsr75SOqQg7QGrLXA7/fm7eiANQBm0shrpaElBT4IULfTQ1kHnJnPwu1CgPLyMZUAu094dvh8a365HwMPByulq8vtR8WDzuumeGmTV4m37jem/a8itjb8fAUuT25+JHZrUlZSdoPWSXOQ6q20s1bPUtKW6wUkVxhL63rw5eus5VjvU4enNt6j36vaP4BFPID7/b6LB25wSa42UsfrIzCMuGutvJlDPdPP2nlUPT+RxLVzCjrqbmlcSxqau2JJmTS0MxSXx3cpLgL1GgfM69X0tP69wXddMINkBnr27UvFySzTyY/bkPQ1YaDseH2s+T4jtwZcEtLLlPTtw8jtCVrUDd/T0s9soxZ3uNbab9T90t+Z95bzt5qfZ6HLyrR1XkB2vdNL6+N8y4aqSB/UKevx+ngfiMuOjveM6+z/j5h61p54lUkLS9Jio+yy2KSNq85HSz9bAwDrp5+jsX1BvTOANbXeG6w9hIPLqm9IQ2t8jqYJJRh7v/ea8aL1V4B/Y9JiKOX+YE80/axBFsD9gi2JQ1YSXfPl55eZB1+uC0t6wmUGYghpaQpiXNkasbURC5hfiGkZ3/lc8z/Yc7/02NuABRm803FurXd6STropDRzDKaPtDMAN74Fzur5wv0q8I24Tgt4GZcadboRpdrwACv9LKn23l+pD2s+0b7qtRKAYzchT4q6YKrK5zZXzYVfAa2FOKMLyOFmO6ut5YS4tOlb8TP4LteDL9en+bcmVf4p0TVcD7KW0536mpxyca1efETFMfMNWgXMDxA/blkSQfzE1oipy5WgKwGZx2l/Z1zSr0YDrwRnxe0CuKeZAd3xAmDwksrrXG/UGXvxUThXnbNbjtRnPWtApL/fqDP24jWIS6qBuyjKBe3hFy2bpGrdckTZF7v7gzjom+ABUqJPRdrXBKjnij2od/yCBul3qTlSDZhWPL9APwl1UhsTzvJTsqz7g6Mqu50j6eQIhAscizOmG7EoUOnmLOpouRuW5zwH8aPs1pqlpsutS4DgiimMI19D6GVLNGkpZ17PgXsro9AFluDla7zT8Ry8pYwDax5bD98CW0B+4IYGX2tuveEbuuVIAinYuQdiCG2sPqxyTVqc2s6apBZrPQkrKu3Wo4j7jYxVB/2NUtAReGWdcC9gSlaynGsbtCp3Q0ecsNamKOKCsmlrCgFTy6dkSd+alBWFpuekabq5QFB6AMdyDMxTxrOy+aYxLupyOZgfpU/3Pp5Jf3cQPz/j+XqZwfhyA6sK46kz2wW33obEz4X0MiBDd/opb66apvg49tLNFiSn2HlamkLb/65fH75R4EcfxOHCl95yRH+fnmvV+JVpYzldrTzigKucsbdW6+12jt77a43Xqvo+N1wDLpOs2aFclIW0119NLL8KOvOJOhTtj9ZLZ3vOuGZM3v/s88kjFf2R8rzoKBQk6D07jUubAsVIm0NInOJ8Vf1KDgbvIhb6wKS00+ZF6iLgneplZ/s4lx0uj/du6aH9lnMAszJtJ7IFV6k/b0e2JWn3M4Xv/U3lqWfAZ0jEHduTW55XQbNG2fRz2FIztW6+4vLGbQP6hgAusiAoAS26Y9nqQ3O5kc1alV/QYDlUSdpvQmN8xAFb4/GxNbAvLvT6/cEzBZhI13+ni5u/Zvt4tOTkoFa5z/emR5pZ/qDAN2nRtHQ5LrvDZ6lpPM2fMV3gVlLUzBkDxB0DtvulU7V+BwJsgQdwgTh0y08O3sexD16tvDYtnb1vuCbVPf0qloCmG64W9/ve32gn9WxBVqvP9KXBPuJ0tTauNPtOlXnucqub5XOIfCmDFlunHQAM5CEcqY/cPhSVB3HpOPg9wRZoJWXXhjOyUpum+kGYPhkrssb70Pz5zROUC8wnyBVQz8d7gJOCm7aRdkA/EtXLFLSelr5CSk3ff97S0wAWKWr6VYgFyFMcFn9DM7es/H2NQjlfPdCAO8XK67tAgZC1/qs5ZN+d9oRv+ZKGVvhqr2H2WrVNVwDcp11FQHr/xQjnnmvWxrYUnYt7/dDSy5Zz9YCrfZqwxs7UtcTa2gnAQJ90cMu4miuWYitdcOm2yHKmETB7zjazPqz1EZYM4Su/X9iBML/HNwdhro/vl8t52RKSwHxdWHK5wBzMVjl3xLN7hQmIS+pchDFuzvgGYwAzh0yhPJ2Xg/g79Hzl5x+x84fLLXMvr4/+fGLnGjgf740NXl4eTR979+jSslJufX1hdGzVzbubrkjquSgCxyycPZcrKet+EagHsPziBc8J116YMrczdXlhzdoRwED9QzVofXQzVsQ5e/GaCwZCX1NoORXLAUfa0/ro+nCt7v0vn5S12JQFhCBM3SaF8DMuoU1WrXqkj+ewpmCe75R+wFVyvzyellswBoDnyxRXgAxgAWVgCdOp7vFxiMN19noFd1teD31P6E8JuuU8C95Sv7brXS9GeA2RTVdS6vn+iwier/VPU9T5NrtfqYy7X88NR8bO1LXE+toZwEDc3WZdsAVQ6xakmnGdDwyWI4VSJ8WsJQnMEsRFsA/3cvVJWUB6t+7cCU8uljvjZ6HME3WqWrqZanGfL+lHA3GJLmnoefkzaftYKy6QkoBcHPJ0/LR46MnjyyHmYJ29DuFBKXyzEAduef3zOmvtV9401Qpe3seW8P1A0tYlhj8IpAm+rW5WKqu9ZmTAHB7HevKV5FSzk7eA3Nv99t9BfQAAA/HNVFp9r7VeDdB8DOqSYY/d6oJrfkMe8CPlVhmfm/D9waIqIEyhJz3xSr7Xl67nLl2tJ+qGl9CdA1sDMa2j5XPX+wDs9FMHcomjcLz3fYn/kdD2UpkG3HnZHIylbOmUdfDSNlJZBJBlrB7w5aAtZXzu9N7i+2tcA75RQHvQXsMBS3WmntD3uc/ZW48sRaG6xu1LqwG45Pz34HvjQzRmkoCsva5S7nxNYXa3s9WXJisFrbVtSk/PU9EfXZX11OfLAsIUVo+pLL/IgZbRW084ICXxNPJrEQV8pg2VdGvPdHwVyuQ0c/mpQVmCJW2T3QlN597L+fL58vdFur2oCr6PycfAZpVb/dWoxf2mxvRg5t37m5V1exOXVJ8d/wkHeBJWmUh0mIgLzt4XLNVzqGrn3hwbb0nS4KjVc3kvW4rV+uyyNhzcGQ3Mv9xeeCHS1wjSZzfP14TLxivL5T5iNGD5qeZlWwrA5Xqvn4KWXTF3vI/Us+R+az9QSLDldZKj9eo5SEt9DXil2Ihj1m4RagG0eF7jfAH5/xp3tLxOA6JX3uqAvTpT0c1Xtelnq30Emj1dbd3cN7KoGRBLbbMA7+GCI0CW5mc8GYv+jjzgWpusPEVTzzXzMlUD4Y9VMNIvY5hLbxMVv+1I6oumkx9lc9BT2AIPENPUNG2j7oImoAX01DNNaVNpa9j0tSzLLuL53P350H28L8uYyG1JWjrZSlfHHO1URp9epfWlQbsLfIs4ZCUwr/3PUtSRS87bVObWI9oxjdHOM+Ot4X7bnItLtmEY3gP4RQDvbvH/cBzHv1I3nEeCCCR7bMbSYqKbsbT46IcF1gRKs55Q9uSlrSWJY9c54a1EwSjVSTug+ZryI56v986dLR2v1E9lSxgD9DuO531I68Elns49+vqpuIP2Us9SjARRXp8Br1QmxdQ42tJOuhWpvDZaxu8dDsN3/iavB1lpHOuakIFzrTMGUL/5qufu5ky77eELxGjxAcAPjuP4O8MwfA7AvxyG4Z+N4/iv64e1QJXdkEXrsrckeRWupIcAACAASURBVLdBSS5YKiu/COP50FI4lDJep9XzGE+9YO1qXwg/PC2F6eNcUoGh3Jd+K9Iz65eOxEHL5yeBVkpFA8t0dImXVMay0tNa2pkeR5ywBl6vXAOj52Y9YLaknAGIz3qmryPsfKdG27pWrbxnX1KbhUZS2cP98jbhiSTqs+rTn3v5HsdxBPA7t9PP3f5lvm9QUYVbvOsILlgrM8bS3K7WRAK1VBdV2s0G+lPbrQ9hnkbO7HSegzS621leHy7HEnAlR0shO738+Zpv6aPEln4eMfM3jLv5SLpZKreAS48lR8whyvvQ3C1t02t3dNRFR/qq3u08TaAPaHtCu9VNR2G+aCQd87LsbUPerUfRnddZ99sP5iECDsNwAfDLAH4/gJ8ex/GXhJgvAfjSdPa7G6eV3ZCluWBLWRfszY+XB74lSQKyBV2tjdV3VJH14Wi7e5vY7mhAhoYnKY08De8TXdp9XcRTvVrZS5I0dy39TOOjqWgvBS251lLnrc9q5bRP6bVl4au/dw585Qm0OdC1QOuN36Pcr7wp8nCM3s61VX3nE7rqjeP4DOALwzB8C4B/MgzD94zj+Gss5ssAvgwAw/CdQYfc4oIjWuOWJK1MO1a6AmTwZjZnWX8LvdaLa8A8S6sfb01Ycri0TnoOtPbFC8DDTdNyPwW9TGvz1DM/ntrO09Clfe716+lnei65Y2tTVmZt2AJvGac25RyN81LdobQzsHS/HjB7QdbqKxNv1UvlprRvPco+9zlTLzlobfNVi/vt/2EgRb9xHH97GIavAPgigF9zwoPSYNVrLTjS1lsbltaWI7chJR5PGXXBnkPmMVQ9U9Bm+pnHrAdhDlO+UWoZv9zZDDwgyNsu4Sk/8YrG8tQ0rQe0tPL8liTedgliOw0tvU9emQVcWh9xwp7jpeVaypjGe7Gaa86knelrCK/5Aj58qXq6WatdTaxWl4lfNNKOrbKI1nkwxpbwBWK7oL8VwKc3+H4TgB8C8Nf7TiMD4Uy9FlN7i1HEBSPWNwevBlbNAWfdb7ZNr7+3FSGsbZySVGDmgWoeL9/n68XyMXk9hfgybr7ZyoI2HysqHiut/fI4C7rlpxSTAS/tpzU1TfuNwFeEdit8H29w3OnWuOMMnDPzlMpNZdxv5rnPljwHm3W/0TH6KOKAvw3A372tA38E4GfHcfw5u8mIdTZKZeJ6uWCpjeeCAw/miHAdiXIphiviWmms1Vc0Zjbu9uloKX38qFveXiTDUd6kRZ2rVs4d7fTSlulk7Zakx7G/A1r6MLLnTujyM7P2a7nhls1Wkbj0hivAhm8UoHDiLXltWhywNUdzQt6xVSYpcqtSD0fcy1V/im5PwhrH8VcBfG/9RIC2rxLMuOAaJ50FuueCpXLnwRyWC651wBaErblk+srM6a4+EKbpWanO2hEtuWHL4Ur98jno9/7On4ZF+5peon7vb3YHdGQjm5eK9txwJg2dcbzl2Esj0zY1KWctjrpeAO23Gnl1cGJq+rL6tcaT6qS+TK3lfrO7kaMP3uB1rannOnivuQOKKAriKAxbxB1urbzd08FpWM08rkf7puWaeqw1WyCegXwJ4QvZIR35PmFrVzJ3mJG2NN2rSQJgRNTBAnO4bi1p/pkUdCQVrUG55fakFvjy12+tAQMB+D7euHbQZWFq1UXirP4jZaasAVrV2kcNINeFL7AZgIsi0JPo4d1r2/uWJO9cmxuNByl3bkmiU+7hgjXXmlnnTbnaGslO+PnpisuVDVq1MUt2s1pKWnLUtIw7ZKmOlvMUtJRS5qnnZUz/HdCP1+ann/lx7U7oKHhpnbUuXPqKxHpp7K7O1yrX6no4ZknR9lIdL7POF6rZzZy5yEjtswCMut/oXOq1MYCBPmlpIO+Wa29JstaSI+lvY57l7+DKjr3mViwCdVJMDWgt0If6q/0CB3nTU604fCM7nS+sbtlGT0Fr9XTDlQ3i3A5oqY1WbgGX1tekoKX66EatKFA90NI58M1WANoesiGVa7ERsHrtI9Dn9VzeBwUI9QtpX7oQvfWIn0vtI4puvorIu4D1WS/eAcBF2XXZ2rVgzaFqpNNirPpGFywNVSTBzHO4NVCs6dNrF5o3gfDTBTAe1oHLfK2VywKz5Hx5meWcaT3wADEv15xtqZtehv8NSHyzFU+bzz8w5Fwwj1+maXXg0vio2y0/LcdLyzXw0njrfuHIzmltpzOA7eBbC2ar3msX6YuXSceiJChJIJQ6yjya0oOrNVFel22vtanTjgAG2iEsxfScQ7Y9hzRYefAZ0dbasBZD5aWupTqrXaTO/c9p6QFhQHbDz08XXK7P01qxwJvM/b3ypqh5e83dPursbz/S0tC0veZ8I+nn6C5oLZb2SV+zdBzZCa253fIzsi7MYb3G/cKRnc4A7O/zbQEwnD68OqtPXscVjffOZ8q6XwumkXoqz2Fb5bXq29/OAAbaAaj1RckVvSWJAzR6bo0BoVyQBGJgCTdrrZgPG4G0VZeBbcapS3oabuPm1oW13cuaG+bpZl5mwVZPJVs7oO1nQWvumcY9XvJxdkHTWG1HM4+xUs28fc2GLKnMcr2AkXIGlvDNwFaKscq3aiPNE6Scn5v/f72dz09KPVcEalHwWRPmdVn32//hHwcAMJDbUdzqgrNrwZH+O7lgDZyWA44CzoJhdi131Q1a83Xhz54u6jOkn3HB5RL7Hl/gAWvaXrutiMdrII6knzUY1977K93OlJXkirPAleIst8v7yIKXxoYcbsD1AkbKeRpgGwDXlEtjRsqtOG0MVV5gb/fLY/ixNnZE28IXOAyALWUBm2lr1XOAZndES+XGeFHQWulqqR+rnP69WWvQvEyT5c61DxdiHElJ3+A7uzXppsv1edq9GmQQv9VIcpQvQdY6eKaP+bm8GctKP9NjbX23/PTccuae4RBoHfje5y8536IM3CKAjvTJpf2as9CV5uBpE/cbGawH/CLud83xZR0IwJlUtOWCPQhqMQFIitLIdjXqnQ1ZgA+uzKYtrdxbK/ZA3LL2677N/uassi4MwIUwT0lbtxXxcynFTI8jO6AtVzxNX38EJU9Dl7GptI1jUqxVl30QB20jrf9G1oY1YLfeLxy5xQgQ1nunQWJuNFJW266mHEI5L9NiurlfyaVq7tbbCa2Nx+N5fS/3ux58gdUAzBfmo8O0pKK1Og/OERfMXS8/p7HaLuikC9ZAXMpqXWYUzlq5NHabGXto8dYMwNPngOv0aTtyq5K0mar2e32XdfpaL63XYWunnqUPA9bmq5pUtJd65n1Jx5H7gbX6iFO2XHJkN3Q65QwgtNmqpizaJts/L+dlMOIioJX6uSvy1CurjMu6gNT2afXP2/eAL++j06Mo+0gig6baTVkWSKPjtLpgrU8I9cHbkjQQZ1yq166U9wJpdy3vF565X17GNmhZ0twtreP3/vKvGSztcs537nYz9/5yONdIArYG3uwuaKk+sjacSU+7O6QV11vKXNfLzzV4ZWOkNh4Ys2W1oDWhy6V1Wo6P6n57O9q2i+bGKegoiKNOdU0XzKHqnUtjc5JaaWknLAtiD7AtKWXPBdfA3P3Mo6ekyy7p65W5WrZBiwJ2mYJegpg7U+uJV/OyvPNdK/1MX3uk3EtB0+MIdLW4KHijbWZlbK03dIvR1HE7TNdoky2Dc+6B2QQxzW5q7vdToUwaiJ5HnW7PjVVWfatzj2mnNWD3alvZLttvdke0NR4HcsPDOShcLfBaQG0BcwSmuzjmZUpa2iXN14atW4v4ubfeCzxATEGrueIW50sByDeQteyEju6A5scRJxyFbqnznHIqBa3cXlTO3VuMAPs8C9jW9tl++Nje3LS5qpJSzxrFeZnmdiPxUr9rul8vrt+Fb8dNWB4soyDccy04Mget38CHBc53YMl7GPUQYnq1KXFeHy0fGCRdMV04S4Pr83RhNXS5zGH2GlS7E1p6HzjAo9ClxxFXHL1v2HtKltSf5nqBTvCFE8PrvfZef1Ybb1yrjRUvHS8kBda6X97GG28r99u7ja6dr0oBCKXbZPusXQu2IN7JBbc64BDMjPNIm2w9lQR371d3j7lB+OkKXJ/UDVr8CVryE6/8FLO31jtvk0k9Lx0vT0OXvou0VHRGUlst/Uzj+U/aLrM+HAFz6OlZ0bVeAKldzjxGapOJt2Ij8ZHxW88laN+lbbyiHfF6CdBSvdSHpb3cb1/4ArsDGMhDLhJH+1xrLVirl+YkxdDNWQ6EyzHYuZdmrklLZ6G8Syr6JuHpWTQlzZ+gRTdpWWlpafOWt9ZLU9DTjJagBqKpZz0NXebH5W02q3kcJT/nQKXts9DlMdEU9L1cAO90Hry9iB73gGambY/z2v6ltmDHqqQGkQdiSPHauVemzSdTp9VvC1/gEAAG8hC24mvG8mCpydtsxfuW5lCOA+vB/DxSF63fG7rWZx8rBoDkhvna8NPT5b5JS9stPcFSd77AA8R8rXeqk9eSCxwzm64oiKXzUlaUSbFL7rdHGjqzKSu7Nqztbgagb7ICYN5eRI9bz6N1Pcesga1XJ6rmoRsSoCPy4rV6a804ctvR9vAFDgNgoB3Cazycg/flpaattWKpLLHhS4KtBVpvTTUD2T1dbpH3mYu6YZaWppuyliC+3NeIrXt/+a1IUopZcrK5+3/lb0EqcyptilpS0Fr7ml3QNDYKXamfjOMFUJduts55XS1kawGcmWstbKU4XjfTSAI+VY4BGYCe243edhRNeUs6LnyB1QA8ou5+3t4QrhmH1lsAjZ5DKaPxQDoVXZQBreV0Jci2tG2V9FaF6sv7d0tLK0/RoiC+i60R81TzNJT9kI3lGnLdLujH+TwFXZN+LvPWFH0Qx9obsqS6MHgBhNLN9HgtyNLz3v31nBePW4jCF8axtfEKrM66UHi3HW1za1B7X5/iIA/ioG9Yza09rZIgyOU50podzlJ7aROWRJHgrUne2nAUlkdzuhFZcL6fs1uWni747PqMj67PoiO+0HICYmtjlnY7EhD5zl87FV3GK/H0nJZN5bn/L5EUNAd2BLilnfYISw5w0yWzW4qAAHgBqPf1Qjleo27N/loBK7U3xSHLj3kZj9cGiThoKZ7Xr5V6zlwIs2n2hzZMQZdJtjjUVhfce0MW75fXW4DWdkg7vxINPhqgM23QWJc53qIOwH19+PoEPF0Wu6Uv16f7Bp6F2jK81eI7teVNYcuvVYz1Lb8oDbi8TWbtl8fwPsUU9PMcuuYGKwBp10uPt4QvyDmSdV4chHOpjQZoUZKDkxp6rpWeZ6Cm9X00tc1xhzXgXmniqLL9RF1wdD2YHnup6OCtSfRYgmMUWN0g55RHFP3QEK2blRM3rKwP0x3TjzLdEUduOdJuV8o8epKuDxdZu6Gj0lyzBlt6Hln7pXGhFLSSap7KjA1WAEKbrLTjaF20Ta/jNdpIr8UUhawEXJ561j4NcGXWj7V6Xq6NYc2lxf32+XCwA4CBnBuW2u7hgrMbsHgfHMJam3LspKKLNDi2ulXpXFJLTM1nrDR86TlZH3664rPrE8qDPPitSyU1Te8jfsZ8w1YNjIHYNx9ZaWfJxVpu2Nqs5aWjM6loDlVaJ6agBbcLYJFqBpSdzdOgj5+9odkC0bXm1LO9Ku9xk9p/Zikmsys6Aj4+F6nty4AvsBuAizw37MHOitVAm52X1Ta7Icvb1AWEUtEahKS/nYhbjawDZ91tr/Xk6K9Oi5PeK2XHdAGx9UUP958VMJ5GtHdBT+XyYye1NWBe5ymShs5uzErdnuS43VI3SzMDNnjp8dYglI6z7daYM5xyUdHHTVobryL/8TVA8z69teHoeBF5/fRPie8MYKAvhC3RfiIuuCbtrI3HZfXD6wLfG8yPNffbmkqOgLqHIjDNltPXfy8TQEx2TfP0NHfFGRgDgJeKfsSsm34u6vFYShpjpqizbhfAYnPV1Hkb6LY6XrO/KGSfhBjebibtliNAhqLnbrMbs7jWSD3XOt911qMPAGDAT0lHIczjrPreG7L4OZ8Dr7dS0YH14DIFIJdetlLWGRhrMWuCmcqDrgRfrX0BMds1TdPTLTAGHs7XSkVP01o65RIL5NPPRdk0dOZ+YBo/izWgO50/1nYBY313GqAP7LT6KOx6ATjTV+8PAqKsW44sl6rFRODrud+IrLbRC9E+8AUOA+CItCvumrJSxlKcB2WpDy2G1hkQpmH02IMwnPJMf/wYRn1tX9GYaKzannzRw03Sc6bN3dP3oGWR9h3F3q5m67uNpYd0SPVW3/P4XPqZtuHQpcdSmhkION7yMwuZVmAdDb5IHEfrxWBr3TcCox4xNbcdtY7Zq01cBwNwNq0steGgbnXBtanoyGuxwMzrEl/a0Ao6SWu7XO0DQDYmGkvfIy01zXdOs/Q0ANcV32Pw+H5iyR1P5ctd0eW8tKPi34YUTUtHdkDzc9URPy9BS+Ebgi4A9T5e/rMVvF59z357jkXLWuYlStp0JTXwUs+8TaRPCa41kF3D/a4LX2A1AJdfaE33FriifXoQjrTL7oqO7oLm55bzDa4594DwHmnkHpLeNg2+FnR5f/efenq6HGspagAmkIH5VyV6j6GcYrZ7FOUUw9yw4XCBJXBpvQndafDHzyyErbq9YNur715zEmVtuuqZeqaqTT1HwWx9ePDiMvOxxj/Ek7DoC8wMlYVwdN1Wqss4Va/fllR06RtOG2M9uBbCMOoz4Jb66aUMaKNteJwWcz9n9xMDM2cs7aLm7ngqo/UylIH45qvej6IE5s6WzpMfP83KFZcLxKDr/TwiBLcYu0cbUZFNVxJINbhGnK4FZG0cq1zrM1PfGl9/sdswBZ2FcQ8IW/UahLXybCo6CuHSD4SxJFA7m7Jq3G0vYB7BOXsg5mUWfDUQ33dPA1qausCHu2MAC4cM8JQ2cbxXefNVgfRUl/tvzOF6L3/SoQvIsOVxC5cLtEGX/2yB35ptesxvrTmLkjZdFWnwze4gjkBcirf6tNpk2nlzbukjrg0BTCVdESW1pqN5jNVf7a5oz9VK7SywW+DG7bgRwmB1Wtke6WkJgBYUvTKrP+tPyAP07P0Q1owBE8gAFlAGoIK5xBdxONZK6odvLtNgCwSBC7RBl/+kf38RYFl1RwRwz7mJ4vD9lB3zOqvcO5dEJ+fBnddJbaU6LUaL8+YQaZ/XTgAG5lcwS5mNWZFUdDSdLPXrtbXgKb0GXh9JWQPNEO4N2T2cbxS6NXXRn2IfujsGCLAUKANzME/nczhTLb7RyZC1a5uO9yhTYAvIwAWwcLn02PtZjlug3Bt82ba1Y6/RhygLvhDq6M8a+GoxPE4aX+srqmPDF9gVwEURCHoAi8RGFIGzlYqW5EE5CmGpz0YI87je8TwGLFZSD/eerVtV869HlCTd5kQlPZnrUfd4hrXdh/1/THLCn/EyD7qADtXIT15WA+a1QFwbL/3s0adXJkqCL5W3PkvPI/+Jov/Rog5Xa9N7Pmu1X+oAAAbWhzCPsUBbm4rmZZE1Yw/CtB2UNgkIF0UBuLcifxZSvPVr0WKi5VY/5oeS4VEGgO6qpilrYO6SiygMP7ouH5OZ1QKuRQvosjdSgm3kuAXMLbDbA7JrzClTJ0qDr3cOVs615q7nyPiaMv17setdEA8CYCB/td1qbA/wEnA1wFr9av0Ayyt8JYRr09KtjrIV6hGgWu2yv15eHoEwhLG0+vuxAGVgCeYiDmhJFM5RMEvu+In9PWmA1eqyZT1+rgnlzBj8dW45Z1G18O2ZeqbK7nq22vN2WlvvA4Sm9eALHArAwBI4XK0umPYd3RWt9VkLYcndSm2AJYgt91wB4aItYdqiDEgj0I7+tMaLAJfXQ4gBPR/m5/cx2d/kVbjX0Ek1TzHOo02tMg3E3nELfMvxWnBeC5iZNj3mIGor+FJFgRyBrwfXlwtf4HAALrKufD3Xg6MQ1sqj68HR1LM0P5BYfkzPAxAuigBXBIMSG20T6cP61UdiNRB7cK2BslXG6yEca/Pl9WBl91jj8aRRRaDLyyJQ7lHWAjt+3guqR52bqC3hmwXy23a+RSsBeETbZihgPQjzGGueGQhL9RYhMlC2nDcQhjBtUuSlmy3HHD3vLQ2e2nmmLyumBcLSMSC/dwiUQaj3FPnw48VrIPaOI6C16tb4uRY0a+dRO4aoveEb3axl1dfWeVoTvqXvQzwJi77QFhhrfffuE8hdvSNtJaAWZZ2xVp5IR/NpanCxfgI6SDQYe+UZV17zoeGtK/J+7AVgqcyLiZavDegeP1vbLtQbvpa8NV6pTOs3k3qOjCm9SWvAN7uh7KENU9A1MOZXe6nPyM7krAum9WumoqV5axCOwjoBYUtZkHrte4i/bS2flaz++U8rxioD5u+D9wFGOqdlEOqk+qi0dhnwWnUWaKT6HkBqAS0/P+JP93e9Bny9tpmymtQzV+QPvuY/RbZNPXiLNgQwFd9k5Mm60q4FYa2tB2Gp3irLbNxaAcJbu8ke/Wkgbv0pjZEBrgZYLwVtnYOVS3VSjBcbibNAy89rj1ugq9Vp9UcBbev4qlrhK6kVvlRR+EbrMjFSXKRNtp+cdgJwUSaNbEE4Gp+BcLTO+wAQhTCEsloIU10RhjDXlu52b2Vg7JV5xwic0zKrnM8zKivWg3EvEGcgbNUd9edafYrq5Xwjbam8siyssheWLeHbD7xFOwMYyLthrY/opqxMX1kIe/VeGZ+3BWFLUlvA3SHdax33CJLm5bnfTB0vQ+IYgXNaZpX3kNZfbxDX1O8BuSOANgxeYHv4Zsv4sRbDX6wH1ygQW+HbH7xFBwBwUcQNW0CNumnJBcPo1xpTqrMAzsta0tFWe+28IiVdoy1BbEEz29aK0eJrjhE4p2W8nNdl5f1ueoGXnnvwlWJbAFfTZosxatuI8sArldWs+VIdGb7SG3Zc+AKHAjCwDoQ9SEb6kaBptbFAqs1JiuN9WzFcnSAslUV+orJNVi19Zsf34o6UAbBkzVGrq4Fu5LgVwFbdlnBeM1aU91xnWqbBV1ItkLWymk1XNdDL/sfbH77A4QAMtEM4Gr/Vpiyvned6IZTxdDR/OAeU2ASEqVrgtpesPxEN2FKdV8br4cRo8+NtaRkv53WtikJXKmsBcY+ynnBdu002VlU05SyVZVxyaxmE+h51rannY8AXOCSAgTYIa2vKEWhbEI7WWSlirV0NmDmEodTRc6oynw63Klmw7eFyLVkwreknU1ZzDOOcllnlvC4r7/eQBS8/73EcgVZt3VZtatubOjJ8uTSIZVLPkT4zkD4OfIHDAhjQQUqVvdpaEI3EtEIYqAOuVMbB6u2Q5mOXGMcN06GjsK3VVpCW6rQyqW1v8Eagu6YDtvrzwMvLIsf0/Egg7hXbUqfKSjlrMKZla7lcaywrhpd77aR6KUaL02KjbdfRgQFc5LlhDcJauwiErb4s6HsQ9vrskaKuTUkDVWlpSUdJP1twjcJYq+8JXi8VTculuhpZv59aGG8J32hZb1ivVaeKPtIwAloJklvCl+ulwHdb8Ba9AAAD/SHsxWWuxNG1Yq9tLYSlMi8lLSnhhnkzCjWpziuTprIlwCWA1sDWAu2W67+175/UJlK2Noil2FoQb1UXjTfFXS9Ql3KOxktlEYBKsLdiuDz4SsrA19M+8AVeDICBvhCWYj0IW/UchhDqJChKbS248vZemZWu1sAcWBumL5OrJ4i98VtWH3hZpj5yHKlDoIyWS3WSou+rFafV9YavdlxTvzWcW+JN1bpeWpaNj5ZZ5ZEY/gas7Xy1+Ei79fWCAAzEHW1Ea0I4O34Uwt78rdckjWeNFXDDNJz+RGMZOtVlyjL1XjtrzEz9FspCOANgft7juEfZ3vGmesIXQp1UpsFSUgt899De/8FsvTAAAzboJABZbbR4KybjhL314AiEgdjDOqS4UmatGVt9BNaGyzSoekExoujnEK+t10/kWKpD4hxOOa2jiv4vjryvWkwrjI8I4payaDw/VmWBN1PW2+W2OF8uXue1e53rvlQvEMBAPYRhtLP69iCste8BYVpuwZLHAXPQRtPP/I+yAcRHkgfbSGwNhKPnRdF1YD7fFmWumZGynmB+SVC2jk1F1nq9shp4rg3f2jpen43TYiPtttULBTBQB2EtFvABa0E4WtcTwrysvAbPNfM2YPGADudEWro04cNstUa8hVohDCz/TL1yKPVZee+tVp+BKz9vATE9XqO+F2xTf7MWeGl5i+v1+mmJjaadLfhKWhO+xwBv0QsGMLBuOvroEAbstWLLNXspa62sjAk0g7hGNf14n8U0iEaOM3XlHE6MFSvV95TVp1TXC8Za3JaA7tmnq5Z0My3fwuGuDd8InF8nfIEXD2Dg9UEYiIGVlmug9ByytzYstSljBtPS9GVRvQSHS9ULwlpMkeV81/jf6v0OIuCVynrA+EgAjsS6ksAL1K3RrpFyjpb3WPON1Gv9vw74Aq8CwEBfCNfGaeNlIayNnXHImT4iQLcceALEdAjujjnApHpelqnXFP0gYPWZGc+Ksdqu+YElcw21yqOQra1bA7w9+gop6nppuQfADDi3gK8FuV7wfV16JQAG+kFYiuVxnlPWxuN10XQ0ELt/mJZn3LTloKkst13hiHvBsUXWryrahn+AgFGvxVjltE6rzyryvvaCr3ceOabnkfKeMLbGdhVd59XKs2nfKNxryqPwXcv5arFWvNVmf70iAHvaE8LROg22NE4CpARnXu5BNvKwEGvsUt4I4qOmprOgtiDt9WmBmNZTeevcGWXAK5XXwteqW/O45sOAKS/VXFPe28me8D2CXhmAvXTx3hAGfJdL++FX4ozr1cqjIK5p1wjiiNYGNHezLWu9Vt+0DEpbD8RSbIusPiKuVyp7aTCmx9VulzdeA3xrut7oHDJ1kXopxoq14q02x9ErAzCwhBnXnhDm9VZdy/ovB2TNJq8sKxPMhwAAEv9JREFUcDuAmE/Rc8V7p6g9CGddb8Tx9v4f671/GTfslb1EGIe0N3jXKs/EZeEr6W3BF3iVAC468ppwDwgDNgRpm0i55Ya9GKmci77Gzq64VRH32rNvC8Jw5tIK4+j7+tLdMD2PlDdBl3dwBPD2nIcV540r1UdjpDgv3mpzPL1iAAMvC8LAHLDaurAWF3XJUl+9Qez1W+mKqfZaK+7lejPO2poLl7TZK6st4OvF9ARwpo0rze0C6wAvCsIad7sHfM+0M9UrB3BPrQ1hXh/98JCFcE0by5UDtsu22iRBTLvlrKfDPbFy6zhTV3OeKYvUSbFUXru1HLBU/lIgHFY21WzVZV1vdMzWvrgyfwg1aWdNrx++wJsAcC8XrMVvBeEawGXcs9dGcsNenFQu1TU44r2ccKt6OOAS21s94CuV9TzfxO0CMejW1q2VPu4xn0wfkXopJhOnxXptjq03AGDg5UMY0B2oFpcFdG2bmrVgK21d1JCeLl1tAWb+q5T+PLQ/MQ/CMOrX0lrwlcq2BnBKreC1ANUK3tpxrf5O+O6hNwJgYAkOKgvCUptaCNO+WiDd6mx5/zWuNQri2v7pBXBFGK8B6l4QLvVwYnrIew+yYD4KgFOqWd/N1K0J3pa66Lxq6qWYTJwW67V5GXpDAPZkXQklN1wDYR5jgUqrp/3XuuFSFwWlVRcBsRSbqatcK34p8iAcjWkZvzamFr5SWU8ghxXdzdxS1wPeawC7Fa418NV+UW8PvsCbBHBNOlpr1wPCUkx0XZifR93wGnXczUbcc7auwhXTaZZujgjnPSAcfR/WcL9SWet5Smu73V79tIx/wvfoeoMABuohHI3fAsLA3HVGgZ2po2N4dTD6bYF9GYfXVbhiaQgNzl69FcPLpLY8XmqTiemhPdLQUtlq4LXcLrAOeNcaZ6v51dRLMZreLnyBNwtgoA7CWps9IMzra4FpOU7e1qrz+q1xvbzeSk8DTSnqvZzx3unn0n9LXO/0dFenC2yTZm7pi9e3OFur7Vau9lzzjeoNAxjYB8JAPWQj9dk1Zq2vCIhLfcQRg8RE+uL1EVBXwphOWdPeqes1IXw0AFt9hhVNMXv1e7rJI82zpo9MX1as1+bl6o0DGKiHMIR2EQhLcRHI0vE8sPYCbbbeAyRvX+O+o2M1wBjwAbA3kHsp8xqyKWitvLvDLap1utl6z+FFHGCt462p791fNEaKy8Z6bV62TgADqF8TjsA1GueBT+qntxves74FtivDGGiDxJFhfQQAV8tb0wX6gm7ttdO9wVszZrQfK9aKt9q8fJ0AvusIEJbiIm4ZyLlhK36t+hLjwbMVthvAGJCvFzWgPQKcW1PQVl3315bZSKVNwIIyr+8Bqd7p3y3mXBujxWmxVrzV5nXoBPBMR4YwoENP6kdyw7TNViBu6SOaoub1Jcaqp/0UNQAZQvdliB4QWhPUa7vgJnHgSgOtsX65hXtco4+11nJP+K6hE8AL1UI4Gt8Ca66sW65p0+rAtT5oTKQPKZ1utYnOi8Z0csdUGpSj9VbcmoqMt+qcjgremj4jMT366AXWSMwJ3x46ASyqBsJamzWdcCTGg1mkjec8M320jJMZt8RYrliKKXEcAJ2ADGE4PvTeKemiTecRAS6wHiB7p4l79pMFb6RNbUwmTou14q02r08ngFXtBWHAd7Geo4zE9HCqkZge0LTmL7Up7WpioMStCGSqVujxP7OjwHyhWuBG49aAbm3MmmNtvZZ7wrenwgAehuEC4KsA/vs4jj+63pSOJA/CgAxWCO0i8KKxEcBK/WVjom3o+JkYGtcjRgJmDeS9vr15bgTkrA4JXAm2QPwCL8XWAmNvt9grZg9He8K3tzIO+CcAfA3AN680l4Mqshabadd7XbinG6ZlvUAsxbUC04rJuGmpbxobSVeXWA0yBwHzZsrAFshdqPeGrhS3pmvX+tra9faIteKtNq9bIQAPw/B5AD8C4K8B+POrzuiQqnHCVru1U9JSXBSWLY6YxkUcpxSXjSlxEixr2vWIpW00IAEvE87W6wHqLrKtF/0IpKS2PUHZu79e4F0jLhurxUfavW59FIz7KQA/CeAzLWAYhi8Nw/DVYRi+Cvy/LpM7lrb4I4mO8akQu7Vj2CLuSYhreU2R/mtjtXlp7YAJZvTfERWZo/c6vfcnEq/9LdT+nqOAjv4/q53va4GvpRO+mlwHPAzDjwL4+jiOvzwMwx/V4sZx/DKAL09tfu9RryaN8pyw5oIhtLPWkGtja1PCUlxtWprG0disI65NYXtxkbSy1F6LteK1+Wnta/7bZFx0y3/LyEW3xgH1cMV7fGDcInXe2rb3nLXYmnirzdtRJAX9AwB+bBiGHwbwHsA3D8PwM+M4/vi6UzuqaiBstYumozOxWhywBFukv9qUM43tuU7cGkdjvfZWH6WNlYK2LjIROEt9UrV+1s26mchFc+00tFa+lyvcK4W+RpwW2zPeavO2NIxj/D/wzQH/RW8X9OSAv9Q4taPL25ilgVhrl4lvjW2J69G+95zWeD+scu9za+3fRk1fPZW5KG7tiLdynNF5rbVZac8PAj1ivTZeu9eiL2Mcf8tNT533AVdLSy0XWSnpqLvV4lvT19m0NI9tbR+NbXG8tX3S2Izj1cblffI5SPL62kIZZ+zNseZi3MOFvSRwrbEuu0fK2WrjtXt7SgF4HMevAPjKKjN5laqBMIQ21jpy79jMHKLrvzSWxnuxrevKPWL5PLQ6Kw0ttePKAHoPtaaevT7WTHseOXarlHa235p4q43X7m3qdMDNstaEa9tZa8nR2B7OOdpvBvqZufX64JHptyji5Hmd1yfv1+pH629NZS+QrWnoninqI8C01fFm+zjh+9J1AriLPJgCOSdc2rVAxovPuGHet9ZvqyOm8a3uOTIPC57ZjVjS+JF+ubTfyx7qmYbufeHeGtC94o8+v5p4q43X7m3rBHA3eU7YcoNQ2u61jizNJ+Mkax3mWqlsq2/apgbIUjve1roAeY45I+l33zOdHb2QrrEmbLVby0326rtnP3uA2mrT0u7UCeCuqoWw1bYHKHvHWy5NA08G6Dy+FsY0PgpXC6yek41unorCWWujqRa2tRfJtdeFezqxPSDda1wtfgsHWwter+0p4ATwCrIcLVAHYatdjRuW5pcBsQYsq/8ecI3Ms1f/VNH0s7cRSxrb6i/aZi31XhN+ic64Jr6nw9zrtVltvHZe21NFJ4BXUw1Ma9tloWqN0xvcUputPgRE21hOl7fz2kbaS/1Y/UUl/T57Xwh7rgtvmfLcYh30iODt3cZr57U9RXUCeFW1QBhK2z3T2L3b1K4V8zatMLbG8ebnjZnpR1Lmv2jrhW/NFPYe7niLVOzeHxR6t2lp57U9xXUCeHXVQthq23Nnde9xeoLY6q/nB4FoO6+t1l7qR+vL6vsIylxgX0pKurbNVs7yKJumTvj21gngTbQGhK22NQ66JwStNla77KYtq02NK+btvLaR9lo/Ul+StrwPGKi7iEY/JLQA12u/VQp2S7Ad4fVG2kban5J0AngzeSAF+q4LW+1aXGItvKXxrHbZ9DRtw9ttnW723geujGveUz3XfyP9Hckhb+0oj+J6vbaR9qc0nQDeVBZIgbbNWVDabrWeTMeyxpPaRjZsSXU9XG4Ph1u7Bmz16Y3RU61p7ugFODLOWinQLd3uGnM5Wjuv7amITgBvrlYIw2i/tRsuWmO9NeOKo31KbaOvQ2vP+9D6kfqy+vTG2FrZi+1LTUuv1WdL2z0AesJ3C50A3kUtEPbar7kJaY8xpba9gJpNN/P2Uh9SP1pfWp+a1loP3mr3dI+0tNfPGtBt6bel7R5jem0j7U9FdQJ4N60JYa997zXlaFso7VvWwFv71dpG2tM+rH5oX1TZ/35HufD1XguO9rmWYzuii2zJeOzV9lRWJ4B31ZEhDKVtK7TWcMTRfmv7bkkxZ6HsjbGFWi7CWwE30ofX/qiblo7qfCN9nMroBPDuikAU2N6VemNvAeKasTMwXXvNV+pP69Mb42h6iWvCrf3vCV6v/dpjR/o4ldUJ4EPIgyiwHkij7fccW2sfdcVavQfjSB+8H68/3ifX1vf9Wqq94G59u1Kknz3h9dLbR/o4VaMTwIdRK4Qjfay1uau0RaC91sfaIG+dH+2jKJNejv5Xi1zoekC65wU169aPAN3IGC+9/RZzONWiE8CH0tEhHG2Phj56gVjro8bRrgFkr29Le14Ua1LjW64LR2L2BudrmcOpVp0APpy2gjCMPnruDl4LpBnH6rliKyY6Fu/P61Pqm2uv/55bbMLKjtXqdiNjHQF6R5nHCd8tdAL4xaoVwhG1uuGj9BFx5ZGYMhac8XifkX6tsSS1/tftvclrDfBG+3wpwHop8D21lU4AH1JReK7thHv2AaOfHm62xzxojBcXdcVSv5H+Pe15Ea1xR703Zr0maPaay1av51QvnQA+rLaCcLQPOP1EnWxrP703U/WGsTWu1X9knK3VeiFe44Edrw2aL20up3rqBPCh1SONHO2nVyoYG/UThXmPfjJxdNyizH8z7wLYE9C9L7Zr7IbOxL1G2B2tn1M9dQL48OoJTzh99YBw6QdOXz0ButX7Q+MisXR8qpb/dke6SK65IzoTezRIvdZ+TvXWCeAXoV6QifTVC3rROW2ZIi/q8WznbGzNXI6mrXZHv2bw9uzrhO9L10v633/KVRTCPZTZNbwFhKP9RPvK9FdikYinc6E60n/JHhu99gJvtL/XDN9TR9eR/refMtVrPTjaV09I9UyRw+mr9+1EWbCucevRFv9Ne+6qzsIhE98TTq8dvifIj64TwC9KRwRnpi8E+juyG0awz5Y2kiIXUu81rH3bUs2FvDd4M32+VPhGdd7r+xJ0AvhV6qgQjva39caqmodr1IA42y6jPS64te7pNYA32le0v6P2dWpNfbT3BE5l1fvisHVf0f72uNA8JfusdX21bY+glvln270V+Eb1Uv9mTmk6AfzmdeT1pCiEe88t+wCJFif4UmDc43VGlf0gFO2zl97S3E6tqRPAL1J7/MeO6rU4iLUeKmG1P9pFsccHhGzbNZ6edeT07gnMt6wTwKewj3PtrTUuZFtDuPSx98W21xzWfB2v4W+2t06YvzSdAD6V0B5rwcC+bqIGwj1BvOXFsud4Nf2skbE58rrvCcy3rhPAL1bnf/LttPYjFyN9rfX7WQP0a8P3yEswp07FdQL41Eo6+geEl/iB4wigXKPPE5Sn3qZOAJ9K6rxYxrWma21t/xI/gAD7zfvoHxTP/5cvUSeAT70yrXUhOtoF7ogAPeKcPL3EOZ96LRrGcezf6TD8TwD/tXvH6+r3APhfe0/ilet8j7fR+T5vo/N93kYv8X3+znEcv9ULWgXAL1HDMHx1HMc/svc8XrPO93gbne/zNjrf5230mt/nMwV96tSpU6dO7aATwKdOnTp16tQOOgH80Jf3nsAb0Pkeb6Pzfd5G5/u8jV7t+3yuAZ86derUqVM76HTAp06dOnXq1A46AXzq1KlTp07toDcP4GEYvjgMw38ahuE3hmH4S3vP5zVqGIa/PQzD14dh+LW95/KaNQzDdwzD8AvDMHxtGIZfH4bhJ/ae02vUMAzvh2H4N8Mw/Ifb+/xX957Ta9UwDJdhGP79MAw/t/dc1tCbBvAwDBcAPw3gjwH4bgB/chiG7953Vq9SfwfAF/eexBvQE4C/MI7jHwLw/QD+zPn3vIo+APjBcRz/MIAvAPjiMAzfv/OcXqt+AsDX9p7EWnrTAAbwfQB+YxzH/zyO4ycA/gGAP77znF6dxnH8RQD/e+95vHaN4/g/xnH8d7fj/4vpwvXt+87q9Wmc9Du308/d/p27WTtrGIbPA/gRAH9z77mspbcO4G8H8N/I+W/ivGCdegUahuG7AHwvgF/adyavU7fU6K8A+DqAnx/H8Xyf++unAPwkgM/2nshaeusAHoSy85PsqRetYRh+F4B/BODPjeP4f/aez2vUOI7P4zh+AcDnAXzfMAzfs/ecXpOGYfhRAF8fx/GX957LmnrrAP5NAN9Bzj8P4Ld2msupU80ahuFzmOD798Zx/Md7z+e1axzH3wbwFZx7HHrrBwD82DAM/wXT0uAPDsPwM/tOqb/eOoD/LYA/MAzD7xuG4WMAfwLAP915TqdOVWkYhgHA3wLwtXEc/8be83mtGobhW4dh+Jbb8TcB+CEA/3HfWb0ujeP4l8dx/Pw4jt+F6br8L8Zx/PGdp9VdbxrA4zg+AfizAP45pg0rPzuO46/vO6vXp2EY/j6AfwXgDw7D8JvDMPzpvef0SvUDAP4UJrfwK7d/P7z3pF6hvg3ALwzD8KuYPsT//DiOr/I2mVPr6nwU5alTp06dOrWD3rQDPnXq1KlTp/bSCeBTp06dOnVqB50APnXq1KlTp3bQCeBTp06dOnVqB50APnXq1KlTp3bQCeBTp06dOnVqB50APnXq1KlTp3bQ/we5egeI3ld27AAAAABJRU5ErkJggg==", "text/plain": [ "" ] @@ -3531,7 +3534,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeAAAAHwCAYAAAB+ArwOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztvX/MdV1a13et93ned4AMZZpIUueHjka0NSRCO1Ia0taMNB2RiqZJiwYTf2WSWuPQ0FLxD9umfzVNiH+UNHkLRBONaIttLdUaGiGUhCIzCAYcNRMYwxTiSA2BSWfeee9ndv84977vfda5fq9rrb32Ptc3eXKfvda11tr3eZ7nfM73Wj92WZYFUqlUKpVKjdVre99AKpVKpVL3qARwKpVKpVI7KAGcSqVSqdQOSgCnUqlUKrWDEsCpVCqVSu2gBHAqlUqlUjsoAZxKpVKp1A5KAKdSg1RK+WQp5eursj9SSvnRgL6XUspvae0nlUqNUwI4lUqlUqkdlABOpSZRKeXdpZTvL6X8s1LKz5dS/vSm7mtKKT9WSvmVUsovlVL+u1LKG491P/IY9tOllM+UUv7DUsrvKqV8qpTy7aWUTz+2+f2llG8opfzjUso/L6X8WU3/j/VLKeVPl1J+rpTyy6WU/7aUkp8fqVSD8j9QKjWBHmH2vwHATwPAewDgdwPAt5ZS/t3HkFcA8J8AwK8DgH/jsf5PAgAsy/JvPcb8jmVZ3rksy199vP6XAOCLHvv7cwDwPwDAtwDAvwYA/yYA/LlSym+W+t/oDwDABwDgXwWAbwKAPxbxu6dS96qSZ0GnUmNUSvkkXAD3sCl+AwB+EgC+DQD+x2VZfsMm/jsA4Lcuy/JHkb6+FQD+7WVZ/sDj9QIAX7Esyycer38XAPwtAHjnsiyvSilfCgC/CgBfuyzLjz/GfAwA/utlWf4XZf+/Z1mW/+Px+k8CwL+/LMvvbnhLUqm71su9byCVujP9/mVZ/s/1opTyRwDgTwDAbwSAd5dSfmUT+wIA/q/HuN8KAN8JFwf6JXD5v/sxYaz/d1mWV4+vP/v4859u6j8LAO809P8Lm9f/BADeLYyfSqUYZQo6lZpDvwAAP78sy7s2f750WZZveKz/7wHgH8LF5f4LAPBnAaAEjq/p/32b178BAH4xcPxU6u6UAE6l5tDfBYBfLaX856WULy6lvCilfGUp5Xc+1q8p5M+UUv5lAPiPqvb/FAB+M/gl9Q8A8J+VUv7FUsr7AOAjAPBXkZhUKqVUAjiVmkCPqeJ/DwC+CgB+HgB+GQC+GwC+7DHkPwWAPwQAvwaXxVQ1/P5LAPiLj6uY/wPHLUj9AwD8r3BJS/8UAPzvAPA9jnFSqdSjchFWKpUSVS/ySqVS7UoHnEqlUqnUDkoAp1KpVCq1gzIFnUqlUqnUDkoHnEqlUqnUDupyEEcpX7IAvKtH16lUKnUA1VuoLdfaOq4N1r61X889cvfDxWvLvGNGxVL6JCzLL4sddToJ610A8OE+XadSqdT0er26rj9qufrXiXKuTtPGM6am3FKH1dfXWBsqjovXtrX2pdHvlEMgU9CpVCp1YlnAM4usAIyE71glgFOpVGpX7X0k/7yAGq+xfxcJ4FQqlbo77QV9L+zP+SUhAZxKpVKhOics+qj3ezV67temBHAqlUodQtxCqyhxi6k0bSx1qQRwKpVKdZW0CjjlVxTg9/mikABOpVKpaeRxuTM549YvF5btR1Grn/dz6QngVCqVCpMVQD0//F8nXu+lGe6h1r4p8gRwKpVKdVNU+jkSXp55Xk1fljrt2L3d777KGfJUKpUKUeuH/t4p01Yw94S5RaPfR//RlQngVCqV6iLL8ZNSW0qaoyIjwTjSwXvc7wj4RpwVfVGmoFOpVGp3jUqZas+PtvZlleXLSLTmgC9AOuBUKpUKUE+Aah+I0Hs8i0akyWtp3g/PfcVCd6sEcCqVSoXL+/QjqU4zHpV+bv24124/sj79yPrkI2l8Stbfvx94VyWAU6lUqknWx+95Fel+90w/e9UypqVtf/CuSgCnUqmUWx4QjnK/HrUeBCLVed1vS+pZ+3uMA++qBHAqlUq5pAFF1GIjD2Q1c8fRp221ztFa4Htc8K7KVdCpVCplltWladSaCp7d/daawf/tB1+AOd6BVCqVOpA8DhSAd8eWOuvYVHvP+c49n3wU5X7nd76rEsCpVCqllvZwiJaFWR4Yag/esK4WjrhP7FrzfvSC7/7gXZUATqVSKZV6wVfrKj2uWNM+YrEXVzcLfOcB76oEcCqVSrGyfOi3Pg2pdR6WGr/nimoLRjSxVvgey/VulQBOpVIpVFa35VkV7anzuF+q3DO/25J6luKtccdzvVslgFOpVOpGI+DrSelyfVpTzK1zzVJdROrZm3KeG7yrchtSKpVKXSlihW2Ped9aniMnW1dEa/ryxkb1dwz4AqQDTqVSqUd59622LrqKTgNr+m4d03reM3df1D140s7HgS9AAjiVSt29Wg6MiIQv17f2HkadeEW1wa6lLw4Rx1EeC7yrEsCpVOpOFQleLD7qIA5u7EjIRhwMYj16837hC5AATqVSdyXrQRTa9r3gG7HoqnUvsLZNxP5nC8CPC95VCeBUKnUH6gVerN2IU7Ba9/VGrMDeqkcqnhrv+OBdlQBOpVInVit4pT72OAVLGxd92hUo6xK+WiWAU6nUyRS1jaZ1L3CPfcD1dS+HG3E/Uj9YPRaz6lzwBUgAp1Kp02gv8GJtZoFvrehTsEbA93zgXZUATqVSB1bkoRE9wCvF9IZvrzaWPjz1qw4C3/r2H3zNUqlUanJpoQvQ5na59qO3Id0bfCcDbydSJoBTqdTksgAXoN3tcn2M3obUc/529JzvQVLOA6mYAE6lUhNqD+hy/XjBK8UkfJ+1E3h3pGACOJVKTaQe4NX02+PwjTrG4oqtMMMU3caTuqbGngC+UfTD+nk19hZSqVTKIStwAcZAl2sfvQWpro8A3V5zvhPP93pp15GSCeBUKjVQHuACxEFX6qt17y8W1yPFOyrt7F3BrakfAF4r4QYTUfU84FLKh0op/6iU8olSyp/pfVOpVOoser36Y9HLzR/tON6+qPZUOyy+jqtjsHouth4Pq9P272nTsn1qR/ha/tm8BFt8sMQhSykvAOC7AODfAYBPAcBPlFL+xrIs/6D3zaVSqaPJ63BXRTpdqT+r26XatKakW0DXq03EPdV1AN3BGxET0e7zcd1+DQB8YlmWnwMAKKV8HwB8EwAkgFOpFLRB1/rJ1ppilvoYuf1Iit8rHd1rvrcTfKW/7onT0Jqh3gMAv7C5/hQA/Ot1UCnlwwDw4cvVlwXcWiqVmlejnK5lrD3Bi8Va4BsNWK7uJPCNAu+OK6E0Q2Pv3HJTsCxvAsCbAAClvPumPpVKHVUjYWsZrwW4XHsLdLH4HuD11mnbHCTlHAHdVuB6qens6lMA8L7N9XsB4Bd13adSqePpqMCV+vLMB7fMBXvBW9f3hLIlbkfXy/3VRaegvW06DfMTAPAVpZTfBAD/DwB8MwD8oa53lUqlBqoVuADHc7lc20joYmXRKWWuLiLdbBkbIAy8I6C7Y/pZNfyyLA+llD8FAH8bAF4AwPcuy/Kz3e8slUp1VLpcXTvvqucW9xjhhrl2k6ebqb+KPdPP1naBKWhYluVvAsDfNN5CKpWaRqOBaxnzqNDFYkeA11vXAlcO/B3B2+KCtTGe2CDtbMBTqVRfndXpSn1Ywduy2pkqiwJcRN3ErtfqeCPnfGdPQadSqaMpoatrExHfulCp98Ksg4HX63hnSj/P0XUqlRqrFvCeaREV167V6WLlmoVYo93wxOlmC3i9Llgb44kNVAI4lTq8vOBN6OpjJWhpYnpB8aCOd4/UcwTxNH1ELsJKpVKzaZTbPSJ0e6SWtWNZY6LqJgWv1u16oNsDuIOJmABOpQ6lEW53ZuiOnM+l2nv2A/eY+43sZyLwtkBX+8+8B/m2faYDTqXOpFnAe1ToalPLlvYt0JXqpbYR88RYfTB4ezpgTb02JqLNvMOkUimfZgGvps/ovbpndLvW+hOlmkelnidPO08ydCqVojULeI8I3T3cribmZI43Os3she6ItLO1baagU6kj6gjg9aaZZ4XurG44atV0XT+R251h8VU64FTq3nV08PZ2u9pY7VizQre+nsTtYt3NAt2RC6+0faQDTqWOoBnA2yPN3Au6Z3S61nhL3Q6LqiKgO9IBe+KDlABOpXbRiH28ezjeiLSxNrYndD0xPaFb1x/Y7Vrnf6X6GR3wPt2lUilZR3a9MzjekVDVxCR4xRht24g6S0xEmwYlgFOpYTqy6+3leM8K3Tqm5x7hidPMoxZe7b3oyvlXkgBOpborweuP2wuonhjp/qPnkQO3EHmg2wrX3sBNB5xK3bNGgFczztHAO2ofriam1elKfe4IXaxsr7neVuCOnAfW9JcOOJXaU7O7XusCq2jw9nK7e6xgrssmcrpYd73mea1w7QVjbUxLfJASwKlUqGYHL9d+VvDuAVWp3vpl4Y6gGwncSNhG0U7TTzrgVGqk7iXdHDUHq2k3W711DvjOodsDuCdyvzsPnUqdRQlfW0zvudmZtg1p6ncG797zvKPmf62xLX2mA06lRuhs8G1Zsbx32jYS2gndZrgexQF72wQoAZxKudQCXoB953tbXG8P8PbcI9vidhO6zWVceUudpt4bO6KfPt2lUvegdL14zMjVwZFQ7pnO7rhtKAqoe6egpbromJb4YCWAUymT7mGVsxUyWEwUIKOgfFDoRgC1N4StsS11lhhLXHT7nANOpSJ1NteLxUe43hGu1Vt3Euj2grC2naU/Tz+9YlriO2mS20ilZlbrfO9s42jHOit8I1PdO4J3hhS0p7ylzhJjiYvuIx1wKhWhe3S+1rT0aLh6U83ePiaG7mgIR5ZLddExLfGdNMltpFIz6mzwjZjvjXK9PcEbPS7A1NCNdLR7gVhTr42xxEW3XZUOOJVqUcJ3zHxqJJSjoe6ErhWIo+Z4ZwJuBIwtcdbYFr2EBHAq5VeudO4D31ZnawWvB8hBTneP1LIXsBaA7gXbns53RwomgFOpK50dvhJ46xjvQRWRgI0Cb2e3e8ZU80jXGxnjiY3sJx1wKmXV7PDl2kbAV6rvlTo+GHhnh+4RgHvHrnerSW4jldpbs8PX2kazx5eTxflq+pgBvpOCd2S6WdtXZHlUvTXOGjuinzHdplJH0hHgG5l2rmM41xo5p6spnwy8FnBGxWLX3pgeZVLd6FRzOuBU6qiaHb6te3ylmJ7wjSifALxakJ4h3Tw61dzDzbZSLYKKOQecSkm6B/hqnClW3wO+1HgW1zsBePeE8N7A7QXiXnHeeKvq/hPAqRSnUft8vX3NAl8NNK3lra43GLytoG2BqMfl7g3hljpLTI+4qHZBSgCn7lCj4CuNE/XfT7PPl6vrAV8vRLV9DQLvDC7YG2Mp85RLdZp6bUzv2Og+0gGnUnuqBb69D9nQ1mnisHLt4qy6TNvXAPhGuN+90s0WkI52vzPBdgL6TXALqdRIzTDvuyd8OSBa5lcj5oFbUs6B4J3Z/Y5IN/eAbY/U8VHcL0A64FTqVglfvM664CoS1No2ja43GrwjoTsKuHu73jtwvLUmvKVUqofuDb7WeizOA1RLe63DbXC9reCNSEVb+tG2by3zlLfUWWI8sZ741nYBSgCnUqyOCl/PXt/Z4Iv14Ug3t4I3Iv0c4XJngXBEvTXOEx9NN0t/mYJOpVZ53e+I/x7W7UaauB7wpcaMgC8X3+B6PZCNTD/PlnqOLNfWa2N6x0a066CJbiWVOqr23G5kiWmBr7RCmWvfMt+7KgC+HuBSUNwbvHtBdy/na20TSTZPX+mAUymA/u53ptSzZcWzJE86OhK+RucbBVmvE7bUea61Mdq2UnlLnSXGEzu6TUdNdjup1L1oJHy5/lrTya3w5cZ3ut4e7rcVyBHXrWWecqlOU2+NGxUf3X6rdMCp1MzuVxvvWfHM9eFNMU8IXw6aLRBuAe+RIdxSZ4mxxLW260k4ru8EcCrl0Z6pZ400kKbAGrHiWRNnga9zpbMGoNY6bz0XH3HtjfGWS3Waem1MS7y3TUu7YE1yG6lUtFog17Pv1tSzFKOBL9Ve+lTHnGsdxzn2TvBthbFUpqm31HmutTHathF1mnprXGu7KKINImMCOJV6Uu//vXvO+0pxEmgtW4cGOt8oCLc64vp1j+vWMq68pc4S0xI/epwBmvjWUimv9nS/vT4drP1GzPtyZVh/VJ3GOU8GXwuQufiIa2+MVN5Sp6m3xrW2m8n95hxw6j6156EbXB+W+2pNPVOi4Mt9mmv36dbul4Nvw4KrmSCsfR1xrY2JLNfWa2Na4lvaTUy5iW8tlRol7X8DL9xHp541c79YW+scb13GtafGEOCrgaYHtFrgWsBsqfNcW8o85VKdpl4bE9GmpV1rW0npgFOpker1X0mCb63RqWcOvpIb7gjf3u53NghbY1vqLDGe2Na2kf8VvX0lgFP3J49Dndn9Wj8FvaddtZxSRSkIvhT8ejljT1n92lKnuW4t48pb6jxx3viIthPSbsJbSqWOpohPn+h5X8tpV1x/HHy1874Hge9MEPbGeMulOkuMJW7vdlHta6UDTqUk7eV+NbL+1/QAXCrTAJ7bXhQkDHxY+SwO2ALbSPD2gG4klL3xEe0nJd2kt5VKWdVz6xGnI7lfa1ndnwWyQe43+qc31lJvrdNca2M85VKdJcYSt3e7qPbHHDqVOoIi3W/UJ1hdbz3xSlOmgW3n1POon9Y67WtLnebaUsaVt9RZYlriI9r3pJvUd6agU/ejXouvog/d8IwhxVgPyeDKWlY9HxS+VuC2QliK1bS3lHHlUp2m3hrX2mbvtsGa6FZSqSOJ+6+jdb/Rx01aFl55z3XWyNhub+j2cMKWOs21NsZTHlXvjY1sH0W01n7SAadSlCLcb3S7vceKdL91O+bTiGqSENbHaNtq6zT12piW+Kg+JqbcxLeWSmm0x+KriE+eVvdb1/VyvxJ8pXYOK3AUCHtee661MZ5yqU5Tb42bqW1Ee0zpgFOp0dprJXarWu67oS0HOWubnvDlxudeW+o015YyrrylzhPnjY/sY1LSTXpbqVQvaf7JR4LU+2kWOffL9YuNQbncDu63BZDePlqd72gH3Apirs7ypScqbqa2Ee0xpQNOpWr1/Are8rQj6315j5zEylpWYTdstWoBqqfN3hC21LXEeMu19dqYlvjZxvX0nQBOnV89Ur4zpJFncr/WbUerFAuvNIDiYGht0wJfL4QtdZprSxlXLtVp6q1xUe0i2k9CvkluI5U6gqzuF4u3ut+6vof7jbYoyq//2yYRLtfTZpQD7gHeSPc7q/P19BFJtc6ETACnDqoIEI1W66EbXGyL+61jWt0vIyvwtLEREPbGaMeX6jTXljJPubZeGxPZrrWPGT4CKk14S6nUXtpj8ZVFmiMnJfX+L+9wv1wdBWUuVtO/d0yvA7bUSWN7yrhybb02xhMb2U/0P3FPfzkHnEpFypp+9vTpXZxFuV/OoWrTz9R+Y0ca2+N+ufYW59r6UyrTvrbUYdeWMq5cqtPUW+Na20S0n4x4k91OKtVDo/+ZY+O1uusId869D1JaWfMeDnC/HpfaA74zgDcy7Tyj+90zzdzaTzrgVMoi60MNWtViPzTbglpgK2mA+21xu9y4e8B3Rgcc4X73cqF7gjlYk95WKsUpeqtQ70cORi++ksbz9luDtRXURPdeJ+Z1zVR7y/1o4euF7ewOeKT7PRLYKaUDTqV6auTc74yLrwynXlkAh5VFO2MJUB5XzL221HmuqTJPubZeGxPRprWPiSk38a2lUhGa6WuxR9rUeHT6uWGxFSYrzDR1dYxlXE8qmrun1lS059pSxpVLdZYYS1xk+9n+m6cDTqVGy5t+7qWW9LNFhsVX2JDeOi+wLWNZxuReS2NGwJgq75163iNlvJf7DtaEt5RKjdTei68ijp2U2rfAlbsPxfujdbqaeGudFOuN04zbKx2tjfGUS3Wa+tb4vceL6mO+oVKpCM14VvPe47Tej/aTujH9zNV5nSNX1wJlb6pc+1qqk+K1baTyljpPXGubvcbrJPGWSinfCwDfCACfXpblK/vfUioVpZH/4zTp55H3Y3nwQt2GKsf6aFx8pS3rAWWLtI47XbA/NqL96PEoKWdlXlPE/AUA+FDDraRSk2p0+lka35N+ltLSvdLPRmndI1fG9eVJG1vcr0Yj4PtSEcPFYeVSnaa+jpFiuTbesTRfHLzjdZI45LIsP1JKeX//W0mljqDZ0s+9FbT6WVunjfeAlpMF+NJ97OWCLWVcuVSnqffGHmksTqNXQZdSPgwAH75cfVlUt6nUQTULQL3awQ7sqQc41q9s/fLS2rdnHO/9eNqNahOssFtYluVNAHgTAKCUdy9R/aZS86vVYo1UnZKe5b5SXWR1vy2uWBvTEj96HG/fuQ84lTqLrNuPWtz3QZz76lgx58rV9b6fvRSRep49Jd3beUe1n3OoVCr1rD1XSE+svUGmkeUeZ/t9IqGsqbfGzdZ3Z4mroEspfwUAfgwAflsp5VOllD/e/7ZSKUpv730DEynSrXZyvg99ujWp9R4eqp/e8fZ+L0bDt3U19F59W/vC/kSloJdl+YO+O02lUj5FwXCSr/lR4tzkXq50Noe7qtU9jkhH93KuIxd/NUqzDziVSgHAYeZHj6LRjrAej7q2Ot570l7wtbhia5+trrhBM353S6VS9y6vs/QswIpysdt+qD6pmJmcdC8H2WOh1ojFXx37muWvPJWaSK3/LaxOmRsvXfeNLJCNhis2tjSG5x6igWxZJW3pQ9NPNEx7r4YeSMVMQadSpxIG7AkgzqV1vXV1jGV8aaGUd9yWtPUMKe8jLNSK6q+O3yEVnQ44lWpSz1UhKZNanGaPFLAm3awdd6YU9apZFmr1dsSevgIfxpBKTabcijSFq5Xkda2j3K5WXH89F2zN4IgB7OlrqS7a7WpivIu+qD9BSgCnUsPlhWdPCyR9qelEg6jUdAuYpZ9acYD2vNaMpb32KmqeWNtXBHg9ae2d0tAJ4FQqtVGH7EIUDDyQjZa2/9kc8Qg33ZqSjgKvRjtuPdoqAZxKHV49PkmoT+wa0I7nrkSnmC2x3kVYWocuLczSuF2PI45Qr7nUqJR0Sx8tDvflYv+Tc8CpVKpdlCN2kMGSJvXEemBlWe2sgTFVZoF0Sz97ybNKunX+NjIVDUDDtKMSwKkTa6ZPqBQpC9h6zvW2zCtTZZZUdetccK954K0s7rHXQq2oVPRA0JK3sMuoqVSz3oY5VwLPeE87qt4+o9lOo2lTbx/yjGMRNx5XhvWhfa1pO4OioNxar4WuVy8N32iKbpx0wKnUVDopwD2rdiPbRLpgrWO3rIrGFOGCuf56Ombvth9PvdjW6HJfPuB/OigBnErdtXZeWtuSYg68DdV4VhhjcZr6yMVZe8zCWOeDe7peD3QHKgGcSqUErQuxHqrrVR3mz1rmY3vMAXudcc/FWTOodeV0D9erdbxR0H356vZProJOpVIX7XxymDa92xOyUnttPRfrKZP6bn3N9bu3WhZpoeWdoIsBdvunQQng1IGVR1L6pXnvBn1aW+dtqXaWMTT1Hmes7VPqj3stjaFt20PWYyu1fazlHHzZPg3QDQSspARwKpWqZPliY/x07wHTnqlnaQyqThOP9U/VU7Ga8paxeilqnlhyvRrwDgRurQRwKrWrZtpP0pJREByIFhpR0LSmmKPngaV70brfKHfsgbMk7wlWVJkHvuS4AnidwH3t5SvVH+26iARwKnVX8loeaSFWgCLcWIs77OWIPWXYWFQ9FTujwtLRBOC04FUIB2usEsCpVGqMvO511E9OFhhr4y1lrWCObEvJm8yJcr7B4B2hBHAq1VUzpZg94j6da1e8w3F+kuONgLsnPS3dS09HbE1V91LLQxai4Suop8tlxx06WiqVmlDYh1enFea9XKvkZjVzuZ75X01MzzIgylpS8b3UvGcYgS+XchZcrxe6L16+Ev+UPIoydR/KrUix4t7P2vEO0p6pZo2jbYFwXRdVpqm3fFnpIcuWJQq+aKwOvBphcI1UAjiVUmnmM5q9dkf75SU4DT0CmD1SzVrQUjHWPrG+qDKtS6a0pzOWZIUvIS14e8EWvafuI6RSp9XR53e32vkTeKSbbUk1a2OjXLK2LzCU7fFXXf9XUW9PaoevBrwR0H3x8uHpTz4NKZW6S41KyXOf4g4X3HILFjhHttH2YxlDqx7paW8iJUId4cvJC90tbJ+g61ACOJW6C23BrP2wwGDOlTV+Uvd0rL3aRI1prQOkzlJmAbOnj5HJIQd8LeCNgi2mBHAqlaqEQdZqr4JdcC+Xq4FwFGCl2Mg6qWwmtbhfBKJcylkL3h6wxZQATqWGarbFXJ6UNdfG+YEV6VyjY6UYi6NudcK1IvjQG9Ca+d+r+jb4UpLAOwq6WyWAUyfXjF/5ZxX2XkmAXttgcQYXjJW1uF2uzupO9wC+5r48ddrxNPWj1AhfyfVaofvy5SvxT8nnAadSKb+0aWhMxk9ub4o4Eoxcf5r2lrGi7ivSGY+CseSGa/cbAF9KWvDWcI1UAjiVSgmSFmNhLngtUx5PGeXwvDDnYnqAFZMlVtPG8p5IZbMoAL4a8PYCbq0EcCp1SLVuN/J+yg78dG51eFFOVtO3B/hRoLXICuM91TAXy8FXHHbgedAJ4FTq1Np+4FDQ9mxRovoLdMFS3V7p5Ii0cWQK2ut0veqWjhb+nSjdrwe+HsebZ0GnUilEUYdxvE28xiR9KjekoqPT01ydBnhSe21MT2fscfFcjLdeK83cbyUtfLmUswW6Pc6ETgCnUmGabYuRRRZoS67a66iZYeqyFtDu4YA1/Xp+euSFcqSLtqoCngW+ZJcTnAudAE6l7lbUhxMFWIsLdm5LqrvRwBEr88ZLgLWAUXq910+rwlxuTDet8JVcbz6MIZUK05kemBChqPQ0BVvJOg2GcKQ7toCOGkeCYXS51YGPEvffsgan4jzn2zIavlw/vnOhkUcW5j7gVCoVIwra0qd2YypIM4rRAAAgAElEQVRaYrk1XRoZT8VIwNOOEVVe188obvGVIvVcywtfrSLnghPAqdSV7tkxa4BpXUndkIredmVN52rLPC5Zajc6lWyFq/bLgOaLx86qAWiFr/5s6D5p6QRwKuXSHqD2fiL2/CS1WC5HKnrbzALhnmAeCd/6/rgyb5+WfwK9wbsFqOB+tTDk4MtpxFxwAjiVujtp54EpJ6txwVT5IAhjZRQ4rQ54JHwfkGtrHxHy9tfpe6p23tcD31ELsAASwKlU6kqeRVoUqDXlnSDc0wFT8S0/6365Oo07tt6nFK+RB9LS4RuPkuZ+tfDl4Gp6RvCLV+yfovx3nQBO3amOvGd3L1m2JK3SQNggDpDUa48D5oA1g9OtX0uxknqnljVi0s9baeZ9KfhS/anmgTeAjVICOJUSdURYew/WsNRZFl5xMrjgeiisTFsvxVniW1LDHpBaIDtberrz8gkrfCVFQ3erBHAq1U1HXVHNAZSrG5SK3jaVnLCmnoqLdsLbsVtAbfkiwKk3mIO0TT9L7jcSvj3BuyoBnEoNU7STbnG5ddsIF8zVdYTw9rUmHa1J63pgLf1sSTNb6ywx2jY9AU2kn7m5X82TjTzw9YD3BTxc/dH+W04Ap1IpRByg6zrt3O5ACFuATLXxuGLuJ3bfUfDl+sbUAlMPkAckg2r3i6+UZhZhKcFbw/ZFw5uZAE6lUg55UtFcXTCEt68lIEeUaYGqAbi2DpAybTxVNzINrVwBverFVSpaTj1z7a/KFeCNgC2mBHAqFaIzLNSypqm5VdGTQXj7utXtetPP2P22wNTqdDVjTiTNsZOUbueKafiy/XSA7lYJ4FTqSUddNKVV70/ZCSDckoZuKYtOKUuQjEhL90pRc8L+iym2H1ncbwR8W8D7El5pn8WQAE6dWb2AenZQc5JcMifPgi0jhCmNhDA3fi9wtrha6/gTy3KQBlln+GVfwqubPxYlgFN3qFnSxb1B3nqqlacPS9paW7eA+cQsyglLALSknLkyTapaWxb15QGEsonEbT3aSpr7RRdiEfDVuN4W2GJKAKcOrt4wnQXWvaQBrHWuuAeEAboszrLWa2HNtbP0ifUl9UcpwjnvDO5t+tmTesbgK4E3Eri1EsCp1LTqAX/NJ6jnU7YVwg/Kus4rpKMcsPQzGrTS70nF7iVsBbThEA0pzgJfSr2gu1UCOJXqorPNE2NAjXTCdT1XN8HiLE06m/vpGZN7jSna9WN9t8IcOUyDWv3Mud/ruBj4WvQCXl39yYcxpFKHU6vj9X4aYmlorC/PnHJPCA84tMPigDXQle4Fq+faePs6kTRHSt6UEW+KxvXWsH3R4JITwKmTyupAsXgtEDVxZ51LloDqaaOFsENeyNVl1G1409XSPVjgq9FsgBbSz173a4Uvp1bYYkoApw4sD9RaQXiG1LLW3XrjpFS0pk0N4aAV0nXzCCdsSTdHpqDBWO5JWWvG2FHaOeKneOSXkFxvD/CuSgCnUofRkeAfDeG6vq7baYV0K3wl8FP3TPXLlWP9TAZV6fSrl8z2JMn9UvCl1BO8qxLAqdShwNZTkS4Yi6udLNauZfX0TmdIa+Z6LcCV4N1SvqcM/824k6+keIA2+GrBi80FP7fNRVipU2uP9HOkRkHfs3CqVnTKOhLCdX3HFdIaCEt1dUwdZ4Ev16c0lqcfr6h/6itYHWc+S48TvLo2wpcdN2Dh1VYJ4NQJFQG3FlgfwVFHfMqOhvAO88KtZaPmdFthOgrGAKanIGkWX1nngTH4clCNhu5WCeBUSq2WldJHEeWYqU/iaAhzkMX67jwvjLli6/ywta0lNV2/xq4xWcE6IIWtffpRi/ul4Ev2l3PAqVRPjXCrZ4O0Rl4IY9oJwly9NPcquVOre/XAdwSYOwgDrMb9RsLX6ni3zwt+AQ/5NKTUmcUBjQLqKAh6gT7TnHaEC6biPeloTUyHxVnaOWHPXC9Xr32tGctap5E2vvG7rbT4ypJ6tsJXUg1crxLAqdSNZnCsnnvYc+659bGEnqMuLfWd54Rb5ou519Q9adu0yNsn98+wnv99hGhr+plzv1r4Sq43Ari1EsCplEqR87+zLNLyfJBE2SVtOvpAEJbKpL6xWCnlrZHXGfeUYmvRU6jxIQ0SICn4cv1FQnerBHDqYPKkn6Pi70mco41IRVPxXgg/VPXcCmmDJLhSsdr23GstfGeBKiflf7UVrFj6WeN+pXlfC3yt4N0+tjAfxpBKPcniVKO3H93bOdEjIexdnBUI4WhnzJVjdVHOeFJZ3e9VWwG+VMpZA94tbFseW5gATqVcGg3NPdy6xwVz7aIhzI0ljR+wOpors6SgqZhWHQCwreLc71WcEpBa8EYpAZw6kPZKP8+Yqp7BNc8IYSwdTbVvWB2tKa/LNCllb50US5VRioI3uyDregGWNf2MPenoedgtmLWLsPBfutXlckoAp06uGUC1lQbmI++59ZN2BIRbtykFQrjuoiUFTcVY67BrSQdIVVvSz9Kq5+vY2xXTnqck8co54NSpFOl+PWNIcWed/209SzpqT3HrNqUgCGtdZuRcrwRADSBHQFT6b7huQTKsgAaQ3S8HX2ze9/qadr3ifbEPY9ApAZw6sSi47ZV+3iOVHQF4CcLSB2ovCGN9tDxXWKnWFDTXB9WPN1Zq71HgP2Mq/dyy+Oop3gFf7bOB82EMqTvSXu53lrnfWe6Dk/dTfTSEubaOdLRnrrV3qjmijVbkk44UMQp53a8Xvug95MMYUimrItyvtt/IQzqOfrgHpch9xa3nR3PtlRDWQC5yDpgaQ7qvCIUmgR5v0OpmMRAHwZdyvR7orm3yLOjUSTRijnSvedjRkG6RBnreVDTXVjsHbSUP9wCHhiGj5oBbYiPaeRUIa+vKZ7QP5Zyv1I6K8c79rkoApyaWBJojbT3y9je7s7UqCsLRx1YGbE+K1GypZo+u0tD0e6qZ/71po3S/VJu1nefxhDkHnEqxYOp18lXv9HO09rgPzSf8TBDW1ima9FqENaP7DZjjtT6AQXK/XOpZ+2jCqMcTaiUCuJTyvlLKD5VSPl5K+dlSykfC7yKVulF0enY294vd51nnf61qhbAUo5kPHnhmdMRK5mjYtvyTErcjYSucr93vi+rnVaxiz68HvlQ/nucCa8+C1rzNDwDwbcuy/GQp5UsB4GOllB9cluUfqO8qlQrVLNt5ervflv61cZb38m1lvw+KfrV9aVT3pRmfUkvbgG41cZ1ukZUIVUW54QELN3GI+6VSzxb4ep8JHPV0JNEBL8vyS8uy/OTj618DgI8DwHtCRk+lUHm3HVnd70xbj2ZJW0ep9QMqelHWABeMDSvVj14kRcnzz17bZjv/q3j+r8b9auB7cxvKJyNJT0fa7XnApZT3A8BXA8CPI3UfLqV8tJTyUYD/L+buUneoGUHU4n5nniO2ynLfZ0uLM/LsBU51V69n+EZKDeBSyjsB4PsB4FuXZfnVun5ZljeXZfnAsiwfAPiSyHtM3Y1aVj1HuN+9tBfg7wiSKbvqfx6WNLThn5a0+tm69QjTyOcCW6QCcCnldbjA9y8vy/LXu9xJKsUqatUzFx8NvZndrxW+6X6nkTr129B2gOxHSyJzyMq5X7lv+SEPWl1gH7QIq5RSAOB7AODjy7J8p/luUilRvaB01EcO9gZ8T/hGaaa/j46K+jV7vV0tLviq3OcgpcVXGlkf0PBcrrvnlscUahzw1wHAHwaAD5ZSfurxzze4R0ylrtQKjJ4Lr0a439Fw6w1fbf+zZAICZHGaM6xeHvV8EcM4mu1HtTzuV7ulSIJv1DOCxbdoWZYfBVAfbZlKGTQDfK19zyKP+50Fvi2K+ns5uMNuBWtL+8a3TnsAx1beOVjrs4E1Y7VCd6s8CSs1sSLnfT1jaGJb9uWOnDeeCTjRXxQ0fcz+hepRPf6aev/Va9PUzAEc+qH88Is4ySoSvpf+Uqnh6uV8uXYj3W/kf6s99ip7fv89P0pa/74C4TzybTiCC2bOgI4SlX6Odr/R8L30mUoN1Uzw7eF+W+Z+W/47Rj0zuTd8e2w166XAmbegudLw8TUQ1tyf8Xfg5ng1Zz97HhMotafg2wO8qzIFnRqoHit0ve16LLyKVgSkR+cfOY0CaM+DjANkXUkc9Vce/baoXfAz2LD533oBVi39IwT9e3X3gO+l/1Squ6IOhvA42b1Tz3u63yM5X6k/qr2m3zpm0Mde5C64nrcctVLaeRCHvnv94RncIwq17tkC37pPbe4kAZzqrN7w9bQZlXrec4wzwdeinfY4a+BjAdSM25Wa/on7FmBJh2+gQwnP89WMoYFvzKKuVKqLIo9DHDHv2yP1PGLfbwuoRyy2ioBvpPvV3scEOy81AOy939gK4asvGfoFWPX+X83xky0nX7W3i0lN5xxwqoMiIRO1uIiLH7XqeaanL3m0h7tsgS+m4Pd7tJPtsfK5RY7+rEdQXobxAc8DSs/qaa+O9L8/Nb2izwvey/lS8aPmk6NP4NK0k9QLvpGrnq00fJ2pc6j1+6AWrpayHq7Z+aWDW4ClVWT62Zp6jgTv83ipVLOiPyi9W1VmPpxjxLGWPeDr+YjoDd/eK9WD0889tiC1grX1k187n+08AxpAf6QkFRcJzB7wBUgAp5o1C3w9bUadCz3DKUwzuV5P354xuPE6Hb7xEinjbsNaL8W2QtgLa+NfJ7cAK+Lxgzd9NrjfXvC9jJlKmdXrw7zHIQ0RbrnH3O1I93sU1yv11Wu6wLn4ygtLKfXcuvrYCmFsvJYFYDdxG7eKrobm9wGvYLSkn63QjIDvNrZEPY4wlXpWTxc1A3xHrXpumWyz/h30hu+oef+Wv7OO7hcbgnLBIyFsVYTLNa6Ats7/rrKufpbcbyt8WxxyAjil1D3Cl1LkfGNE+6g58Vnha9EIt+9v1jRGj7lcy/gNsj4BaYWkZ/UzB0T9qVq5Dzg1hXpu92id7+0N39bU84iFV5axKc0O3sgsAjeWY/GVtBipdQGWF8LWFdDe8VmXr1+AZdn/+9RG8eCFllXT9Li5DSnVXb33WfZwvVy70XPEvVfqRvTZc65X2791tbOlLwm+A+Z+63hLWtoyvgW4rXAWIYynn5/neh9uyiS9UM4D8320u9/oBVkJ4FSlEQccHBW+PQDaMs5I+PZw1F74en5vCc4OYXO9mvlfqh8prm7TA7gtZZic87yXIXSpZGrxlcf9joTv5R5SqS7zj95xRsHXOoblQ7/3Xl5LXGu7HnOqnpQz13crXAPcrwVSUiwG79YlCty4mvGs97zRa1eroB/ndpGV0Wv6uXaqEeDTLLyixsltSKlOOgp4pT488LUAdRR8KWljPe9R65iW/jV99pgqCEo9Y11YwGRxuh4ItkjTt+aen177D+CgJIHQ4n61fUf1QykBfJcaBV7tWDPANyK2ddFWr9Rz9JclT99e18uNoXlvAuFrAZCnPy0EvWXUGJzMEObnf6Wymxjh6UW6pxbx7rcFvq3uOAF8d7oH+HrGaz0Vq8fBHhHzy73gG9nvHvBtUEt6VlzA5LyH1jKLtE7eMP9bp581YPXM/WpXPec2pFSgeqy0bR2vBbxc+97zxK2QGuWSLf3u5XqlmIgvS1R9gPv1zvtaIdw679zqmqWxb/p5dpza+V9JXthJK59xh51PQ0qFaEbwavr1ut7eK6T3nvfdG757g5dqZ3W+DfCl5n69Md5rTlZga8a2uHyFPI8kBMCB6nW/VvjmKuiUQbOlmrX9nhW+lEa5ZM0YLf1Z+uz5dzUAvt56S91aX/c/cq63w/wvdvwklX7m9v5a535b4JuroFNKjZ4ztIzZK+UstY2Ar2Xc1rRpD5fs7T96/jhyWkHz3jXCl5LV/WqdrzYF3JJStqSZ6/vDrsl2FfCQVdFWF2xxv9pDN7C2mvIoJYBPo6OmmzV9eeAb2Ua76CpyzlKK7QnfmcFLxXeAbwtYLbGY06X6kcpHwJm9V3z+VxK199frfrn424cztD6M4Rb2+TSku9LIdLNlvJ4pZ8t9aNq0wtc6nja21SVb40emm7nxJoQvVu8Fryb9zF17pE1Va8ZmIYwvsnopPHbw0hUFwxj3G/mEI6vLxpQAPrTS9ca0i4DvrPO+0a63N3i5tjuknSUXzA1tLdc44x5zwJr5XhHCOsdXy/LwhcuQ/njtvK+8CjrukJEE8CF1ZPBq+hs132ttEwHwkfO+EdvBLH1p4nqBF4sLPOVK64IpSG/LOahqU8C908yeexO2H11e37pgy+Kr5zay+5VgbV8FHX+6VwL4cJpxdbO275YPZ6l9b/hG9GGBb4tLPhp4ufY7wVeq5xyiJYaqk5wv1gcVaymTJEGZSC9zq58laQ/nsKSetfPAUr+tSgAfRkd2va1AmGFr0lEWXUXBd0bwUvEd4Kt1txqX3DsFrZXm/rRjMOnn14S5Xqyccr/efb9c6lm/CEsP3vrLQS7COpVmdL0j0s1S+6h0ptSmF3xboOr5rxvpekd/qdK+B53hi7XZOwVtSTVbvgxYwEw8fEF69q8WctLTizSPJ9Rca+/LOhdN95OaWGd2vS3g5dr3dr3WfkbCt3WB2IzgpdoEud66K9HtCT/rMq1bjkhBv0RiW+FscdzC6udt+Tb9TG0T4tyvZ95XA99R4H3uL3VSWf5qjzTXq70HbZue8KXU+sXK0ucR0s1c253ga3GP2vrW/jz3tI2L6k+Rfq6lcb+adLJFLY8njLoHud/UpPJ+SM/sejX9zDrf6+mnh6PV9sn1q2lridkLvADd4atJM2vipdR0HYfVczEWp2rtT8wKVCB9mgMm0tLI4qtbd3q78Mrifq0PVKCcby/wPvefmlAj4Hsm15vw1fcrtbPGjPp7CXS9dXceF9ySgpbSxFoAasDpTWfXZeh4G/erWP3MLb56ijGcSuWZ97WknXXnTbc669Rk8sD3zK5Xat97sRXXV/SeXE+/1ri9HS/XfoDrrbtscb6aWCyeKueAisGU6lsDbK+TvrnW7/0FuH3wwuW17H6pWKneA1/P/mGvEsBT6UjwnRW8XDuuzR7wbXHJVkjv7Xi5tpbfJSjlXF974Outt7rdWl6Ycn1Z7uklkO5XWnx13Y3N/VoXXnHwtbreXg9lSABPoZlSziNc7yxzilJdLzdL9R2519cSv9fcveW9CXS99bUFpJo4rcO21GmcrhamFjhz9xDofuuVz/W+4K00qWcrfKPAu8bnPuDTa1b49nS9Uvs94Wv5++jx3877ZWGmdDMVPxi+UqymP+0YotNkrrE2lDzpZW3MozTuV3uq1U3fwlORtCueo+Gbc8CHV++081ng22N1dG/49lhIFeWS91hgxbXrAN6628gUdE8IS/+NeqSgpb6urunFV5L7xeRxv1r4ck9J8oA3OhWdAN5VM8B3dvBK7T0f9r3TpSNje8DXC16urfXLyyD4asqpWGkMrp5LQdf1VNo4MgVtdd8vH57g+9rLVy73a32EoHRIRit8W58L7FECeDclfMeDl6uz9jcDqK3wne2L0ADXW197X2vdcA1N7rVUV/cttbe00/bFud9KGvcrPfFIcyY0DnD9nG/L1idKt18Ocg74ZNL+VY1a5dyzfY+2vV2vNX4kfO8EvFjXo+Bbg9QyDlanAaPkbDXtNOMFul9MlscRtsBX63rzecCnl9X9RsK3t+ud7cPe2+c9wHcG8AJ0gW8LiLX1GkCvr6Vy6p6tDpXrx+x26zb0sZOt7pdzpdp9wRb4zvAsYIAE8AEU7Xxb+2n5J5Pwtd1Ly4EcVHttPwdyvXX3rS6Yq8dAKsVqyjEnTckLZqovSzvieEkAm/ul5nU1W4Z0q5Nj4OsFr/ZfdgJ4uCygPIvzbfmwbxk3ar6Xa7O386Vie3yZmdz11tfRLtjqbuv6+l64txODstaxcu2ofiRnXq183j7z1+N+r4ePm/e1zvfm4wjvSrPCd8a0cUvbURAZuThL22+C9+Za40StEObqLUDWALO+d6y9Fs5cO/H6ee4XwLbv9/KaBqwmfdwC39ZHEebTkA6vM8J3Rsc8m+u1xrfM+Ub+7t52ncGLDRHlerevW4G8LZOcM9a2h5P1tjPM/UrP+6X2/Nb1dVscuDjora434klIdZ95EtbpNTN87831RsUfAb4Tgbcua4FvC5AxWHJtJAcMVX19rYEs1sbV7hEkiPtd4fuyWgm9PXKyJfUcDd8I8OZBHIdUtPvtDd8ebXumuaPv5+zwPZnrra81r7fXWvh6HTD1WqqTwKxpY4Uz2ub6zOeI1HNv+Fpd70joXo+bmkizw3d0O6ntyPsZPT/cAt/oL0EHAW997X3d6oCxsggH3ORkkXHEsfCFVwCX1PPLzUKsp/IXr9jjIr0PYvDC1wve1nOgMwU9jaK2B2nV0/l62iV842I1GuH4uTYHhy/X/7YMA3PLeFp3zMVIYMYkjVVtO8IeK7iWY+c96/fzao6i7A/fkedAX+4hNYki3O9ZnK8XvFy/e6ape8RGOd8JXS82jBZeXGwP51v/pNpJ5dK1xrVyfXjc7wpfwf1KTzvCDtywH0WpHyMSvB7oWtokgLtK637vCb735Hqt8Vhsr7RzFKwHgle6toK4Bcga+FLu0upePVBtgfVVDL7war1+ek3s+W2Z942Gb9QZ0JFOOAG8u44I35nAK/V9lG1J0fDt/aVjMHjrsijXS732QJgro0BN1Wmu63G4NhZYr+6XOe/5xdYBC6uePfO+rfCNOwmr3wIsgARwR42a+z06fEeDl2vXG1qtsa3wjfoCcWDXu329lwP21PV0wFf98Ht+rauet9D0wtd2KIf+MA5L2XW9fDxlLsI6hCLcr7ftCAfoHadXn7O5Xio+Gr4nBG99PaMDll5b6rbX0bC++R0u7pc7bnJ74AYH2bX8qe0g+FrBu8eDGAASwJNrpoMyPP9UEr72+ISv+ZpzxNr4CAdsuTfp95F+JyuIqTGv2mwWXj2K2vMLgM/7AvBbjq6HHw/fqLOg63FqpQPeVb336krtZ04731PK2Ro/I3w7gxcr04KKi/W4XazMCmGqHeeGLe7YDFVFPy8BLHt+63nfp7hq0dWl62tXbHW+ONjtc715FvTdaMTc75lAJNVxfUptR71PR4HvSV0vV6d1qBEOmCvD+rPUuaAK16JgrThu8vkaXxRFLZaithtty6LhawFvyznQEQu0EsC7qMX93hN8vb+rt21vp2+J9UL1AK63F4j3dsB1mfTaUre99sKaBfbtcZPcWc9ah7uWX35yW5H4uWEshhqnfo1f51GUJ1SE++3hoD1/zb3hO5Pr7Q1eKt7y3tTtD+B6sSF6uFyubk8HrC2n6iwOV9OmHgcAsAM3uLOeuXlfDJ4R8G3bB0yvkK77ofqw1gPkHPDEannLjwQXLn6WRVbediPhq23bCt87c73b1xEO2ApiCpQRoFW3ked9uf2+1IpnDKyt8PU8H/j5euxJWBYlgIdKervP4uyoeO/v19J2RIqdatP63kTDdwfwYmU9XHAUcLVlGuDW9VjMaBd8c41vOeL2+0rpZaocg69mvtd2AAfteFsO5MD645UOeLBGHbwRMe4Mzk4zbmvb6Dlxzzia2JZ/OwnfrvD11FkcsFRX9yO1V7e5Pu0K4Bq06zW23xcAX/EslWM/a/kXZPUBb889wAAJ4IHqBRlrmyPBd5Z2o7MCJ3C+kddWKHuA3OKApf5ncMBXZbenXT2nmh/QRVfPQ8grnqlyW2ral3JuOYyjbl/LshVJ+78rATyFjgQMa9+R6fHR7Xq/973hS93LwEcGSjFncMCUg9W8puq8oKVinsqutxxh8F0lzftqVkK3wNfrem0HcfTZhpSLsIaqZUVv9JgJ35h2vTMIkfDdwfVaQWu9bgFxtAOWYinIamK0YMbirTEIfFdJh21wK54t8NUutuK3IdnAq4EuBdxchHV6zXYgBBXfCzBSPzO1S/ii3fe8jn7tdcCan1SZBGWsfQucWWeMH7ahge86v2uFLwdXTcq5F3itW5BsME4HPEi93G/vvxorNLSxkYd0eAHqbdvbKbd+sZHg2xm82BBRcPXWeRxwDwhHu2APnFlAX5/zvD1so4bvqtHw5ff/0qCmYuv4uk0dqynX1ucc8CG05wrm1r4t/Y6E78h2rfC1jKdxvrUmhi/Xd0/4Yv1GOGBuDO6epN+bi1e34Q/bAADypCuAa1heutTvAb68bn8kIRZ7W0873lmfCZwAnlJRfy0RQNXGzuB8Z3S9VLm2D+37X8ftuNCqhwuOeK0BLVfXwwFTdRKo1a7YftgGt7KZ2+srzfl64RvxVKQ6DruWyuv+OeUirCHqkQb1jGf5a9wbvrOA19uuF3wj44LgK4HWeu2NjXTAvSC8vqbgzMFaE1vXm4BNH7YhwZdatWxNO9tWQceBV14FTUG47XGEOQd8WFk/4C3xCV+5XZRTbn1fNHFYzCD4WuHcCtvtdZTrxcq8UKbiuPvzulx1m+fDNqSTrrC9vq3wxQDrfRyhNB9MxWqusb7qPjXa9ptzwN012v1aZF2BrInV3nNvsO3RrpfrtcRK8D3IKucernf7ugeE19eacovT5eq8cDbAd7viOdr5ahdbtYI38jGEe2xFSgBPpQh4RS/2oeIitipZ47k2PdqNdr1Y7I4p50iXy8Va46LqPU5Y44Kp19iYFhijoMXK8GMmsWf7RsK35WlImm1Iaz/bmNt63aKsug+qjbauVs4Bd9XILTAjF1KNgu9IiHLtItuMeu93gm/LtQfE3tfWsigYe143u1ysnX67kXTEpLTVyLLYCk9H6xZjcTHc6227bdvrehuE6z5pJYBPLMtf255/xQlfXWzCtxt8sbG5f0peByy95hwxV68tewmg3W6E7fWtYUo5zFtQ43VyOlq7Ejr2TOg6jiur+9FoHSvngHfTCPfbIzba/UakqLk2Pdp52hwYvhykPNd7gngv57v+1Dhdiwvm6g3MUpMAACAASURBVNG42+1G0gMWJPhiKWQKol74jjgPuo7Dr2MewmBVAtis3guoJLW637PDd5RTHrUqHYsZAF9rfStsqbojQFh6TdVpXLEKyPIZz9ijBTXw9aSdW07Fouq3P29f38Kai9+22SpyEVbYHHAp5YsA4EcA4B2P8f/Tsiz/hfmO7kIzud/Z4XvPrlcbOwi+kde9nS71OhrC2nItiDWwxa7VMfTTjVrgSzlYySljbdb6y69gT0djsdt4Kvb52r8Iq9dKaI2degsAPrgsy2dKKa8DwI+WUv7Wsiz/d5c7mlozul/tPZ0ZvmdzvVRcI3wlV2u9jnS61Os9IOz5aYWvxuVKMFY8WpCC76oe8OVWOUedBd3rHGh+FbR+PjjMAS/LsgDAZx4vX3/8o+v9rhQJjh77eL0aBd9oiEaPtSd8B7jeumwkiC2Qldp5wdwLvsDUUfVYTA3sRvjWoJXdMA1Z76Eccp2clqbint9GzhFjAI6ZCw7dhlRKeQEAHwOA3wIA37Usy48jMR8GgA9frr5MeZupNo10pj37PxN8oxdlnTDlvL2m/jqOAt9tn1I5d611vVtVe30BAOpTrgBk+D7FbWC6jXu+boevfvWzdTGW9yhK/Vww1UeLVABeluUVAHxVKeVdAPA/l1K+clmWn6li3gSANwEASnn3CR3ynqcrRTpaqj/LwRJSTAR8jwZeKr4ldjL4RoO41e1i9Xs5YK0jphytKeb6oI16ry92vjPlcuWyW+fLwVWa65UXYN3GrOXXP+d+HnCXgziWZfmVUsoPA8CHAOBnhPATadTc76iFV5oY7xeBFofMxUvtotvsfQBK8Hzv0UDc4oAtdVIbKX5H+FIPV6AWXGlgqpnvleaHL7eqB69tLljniLexdTweG7P4ah2zwBdU8eInainlywHg7Uf4fjEAfD0A/Demuzq1ot2YJjYy9eyFb48tSVT8yDYHX2iFddsLtlxs9OuIsh4/LfDVwJZtcw1fas53PeGqhiq2pcgz36txyWvdtvw2/rYe63OVdv7Xsh2JLuMXXElp6siDOH49APzFx3ng1wDgry3L8gPK/k8gr/u1ttsr9Rx1H63w3dv1Wvu6M9dbX2tee9pY4YrV94axFcQt8H0qv4XverZzDV89aHGYavcDa+Z5ow7iwCAqwZlq93zdby9w5Crovw8AX20a/TSSIOoBpOXDHFPkgQ5Ri65aoLM3eLk6S18tsQdyvVGA1by2Aper88LYGtMCW6yMWO3MPdXIstKZc728E8ahvMbj5Zo54Fsob1/bUtD4Iq26PdaWKsOEgTwfxrCrIuaMW/5qerr2yIVZnM4IXyzuQPDlxu3xmqvn4jwxLW3X19L7iMVLZQA38N0Kc74ANvg+t7mO3ZZR/W7L6njt/PDabq27Ltc74tvXEXPAur2/df8J4GYd1f16nWdU33s4X+7voudcLxW/03zvWVxwRJnWAUswtThf6rXVCT+V6+d8JedLOVxubrgue+7bcoa0DF4rdCPngK3bkKypaEkJYFQt8N174ZVGHvhiioZv5Pt64hXOWLezwnZ73RPCHhh7fmpATIFVE8PAl5vzlVPM+GIry0Ir7dzwtp9tHVa+7evyuu04Si1sex5D+QJexa2CTlkUCYmo8TWQlNpg7TyA5sa3vke9wUvFt8Z2hu8eYI54HQnhKBh7QKyFLdamEb60y+WBKjlkrLwuq/u9/Ho+8GqgG3Uwh6UM65tSPo7QrR6pZ4u8zsrrYj1zuh5AjziIY5Z0MxY74Nm9vcBsHYOK8QA5CsJUmygwd4Yvt+BKcqhSGplOQ9PpaWnrUn0P2z7WOiy+/sm5XCtwpXlfCqyWIyhXpQN2aWTqudc+Wk4R874e+FKKAKDU5g5cb30dFeuBsgeylnoPhFt+auta4SusduZOt2p1s1pIY22x2G3Z5de1zAnbFmVhMfxrec6Xd786GKcDnlotqWcvpD2p517umIrjxrTGn2SRFdZ1D9hysVEgbgEuVtYLwtY6C2yxstX1ApDw5U63srhZ7aIsy57h27Fo8GJAtUJ35Epo/yrodMBGHWXhlbeviL49/1wSvk2SoKmN7V3XCl9sjL0csKWuLudgXJdtnS8ACd9VUtp5Kw6qT/0xK6Iv9T74tmxPWsuuf/pWQ2/bcPF1m/p9lISBOx2wSdGLoaxjaf8aNEDQxESknkdsSZoJvFR8J/ha3Gp9HRXb87W2vgW0UozXDXMuWOuEt/Bl0s5W5yutdLY4ZMu88GXM2O1Ja911OT3/q4GtdCBHHa8px5T7gNXyrgqW2re+tVFfCnqlnq19pOtVqwdALbF7OOBoCK+vPS7YA2QOtliZEr7cuc6eeVzLPmDPyul63Ou28opnaVFW3X4VB+Y6to7Druv2t3U8jBPAYfLA1xLf8oHvWfwUsZ3Iu+hKuheuP+v2ol7gpWI7wHcEiKMAq3kdAWEPjHtBd/uauq7LjPCV4EhBFUADbQq0Nkg/t5PBq0lP4z/1Z0NrD+LAAIuB1bcKOgGsUK8tR1aAeORZIBWxqKpXahqLk+JPssIZ6zrCvVrqtH1Gwnc0hDEoSrEcdKnXXJkRvtK2IgmqFjdribv8apq0NJeGjlmUhcXcvn5Ay7dtqXpt3VYJYFGt8N3zxCtPTK95X0k94Huifb1Y1xHQ9NZp2mjKI4HL1UltPIBuAXEwfD3p5MjUtG5fMe/A6/aXt+YW9GubtX6tu/5pe0rStp47kMOaggYAePmKhnECuKsiUs97K+Kv3gpw671Y3reR8A2UBFht7BHhq+kfq6PeI+09RsEXqvJG+FLywBfvIzo1LT+iUOeEr/tay65/6t0wBl0JuOj+YAawLx6EOeAlAcxoZOrZEjvS/Upjt7rjFud7hynnumwkbLVx3tdWSEc4YC90ubrOzhfAttrZ4ny51LFvNTTvrJ+v9eDFoFq7Vg641lXQNWAxqL540O3nfVHxu+ia3SOAR6eeo7cdSTERq56j4Gq5B6mvXk5254VW0rU3djSIPfVRdRbY1tcW6G7LpL6N8PWudo6MsY5/iZcd83WcvCDr8jbiQN5em1ZBb2C7BS0G2Bqmz33g5ZiKzgDfG4BbU8E9U88el+rptxWe0j8ZDeQi4HtC11tfRzvdPVxvSxkVI7XRwJark8q41zWgAVTw9a52HhEDQK+OvtTZU9PPfV7Hb8vX+LWsjtn+3MZqYFtDtoZr4aZ/NYdjJYBr9drvK7XTxHpdcoQTbe3TWm9xoXfqeqXrVsB6+tvbAff4GQ3iGr7MgxU0q53fAZ9nIPcK3gFv3bR5Ca/gDfj8pjwC0PqFXJe3g4f0tv02vi6XoFsDV4KtCFkMrJzr5UCcKeitWuHraaeFYetpUZa+JbiOrsdiuHLLe90aO2iVs3TdE8RU3BkhbK3zvH76077aeYUvBrU34K2bNu+At1DobfvZguwN+DwB1pgFWnw5nX7moKsB7tbZrsC9Am0NzRqwGFR1x0Ff95UOeFUEfKNWMXvf7oiFV0eBb6859hO5Xq4uEsqt8PVCuBXK3pgWEG/hq3iwAvcsXws0+6Wl7XuHKfDWZXV/6/Xl7SRccgXdGrhbd4sC9xVSRl1TrlcLYoAEsF4tb0EPKGju5wx/ba1bjA6ecra2p+q0wG1p73G72HhaByy11wBbM04kfG/GtW01Arhd9ftc/gylOv42rl453DflrJkTrsu2/UWB9wm627eEgi4VU9dpyqnYBDBAjHPde89vxIlXs7nf1rRzK3w7ghfrfhbna+2Lio92w1EOOLJOA9yb1/KcL0D7VqOtEwaAx+t459vijp/LcRi/gFdQu+G1TgNd1uVqYMulpSnYGlZBJ4C7p54tb11P9+vZ82vpr1YP+J5kvpdzYJprD2y1cdGvR0O45ae2TgPlBvh6tho9L6i6hW9k2vmNxwVd2oVeXL/Y77iWYeDdul0VdCngSrDlQIvBtcUN3/cirL3g29v9ev66rIDl2h8FvpY+JzhYQ7rW9DMSypHA5epaYFtfc1DlylSv9audL0308N22aYHvukgLAFu01edgj3WsLXhv3DHidl9c3koAMECXc78aF4xd1/G1OBDfrwOeDb5eQMyYeo7smyprBWqmnE39asq1cMXqLaDVxGjdrKZOUy/G4/ClpHWgeJpZB8p3bLYhbVdMx6WptWlpHXhX6AKA7HQtwI2aA5YcL1Z/nw54phXPmrEscaP/qjxfCCglfNHrvUHsfR1Rpo3hHK32Z4jLrV4LW40AgF3xXF9jaVkrBOutSNpU9xvVfmNsFTY1z7y2qfslfycEvKjbfQWyy7W4YA60rU4Ya3t/Djg6/YvJ8sFPKerEq97ul1OLc6bGOhh8sa6tMPbU7f26B3yl9tg1VhfhfKXXV9cbelSywHe7FWcLtrVs+1rqY9vGts2oTidf168x8sEb1673CtKPc7zb+d0bx6txu9s6qn5bvi3jXtdtqBhN/X0BWAuPFvdrfat6ut/Rc82RcI6Grzb139H1YmVaN2uJnRm4WP1eDri7C35OOwOAaa+vZ7UzAGxcqexkrc43avU0l24mU81bWFIgXn+2zAPXryPngrF29wPgEfCNahflfqU2rQ7V2/cM8O3oerHuI697g5jqs6cD9jjhlp/R8L26vk47A/SDrxWEEkytMF+vb+emr+eUAa5XN9fpZtLx1uClQMxBl1vpzDlkqh6Ieq6M0vkB3HqQg6Uvqn2P/ahSjPVeW+Ac6YwTvs2A3V5HwFTz+ggQ1tZZQczA97Wred54+N4unrpts24R0sI3ymlz6WZunpcEL5Z2plywdh4Yq6Pq69d13FZaCJ97EdYM8LXEe6GiGTtycZTld7XAOfr333m+FxvKC1sudtTrnkDuBWFrHVWmKScWXAFAJ+dbQw2HLwdvALhZTFXHS/Xciu31HgFuF1mR4K1BW19TLpgq06aksbq6vq7DrqXyrc7rgCPh29J+xKIvacw9U8+WcT3xB4WvF7ZcXTRkqfII4GJlURD2xjS9Hg9faTWzBF/pUA9pDzBfL6ebb1Y2b1c1U+ClFl9h0PXMAXtPwfI64fM54NYTnqx9euCrcXXauOi/mhY4WwDb+iWBupeDp5zr6x4g7gltC0S5Om2bKDC3whd5sAKADF8A+5OFtoCLhK8uRa378nC5h+pJTZLrrV8DU+ZJR9flgMRzr7kFWVQZ1narczngHm5zDwdLacS2I04tgLV8UYn853YS+HLjWl9H9ucpo+q4WM0YUr91mQbSYvktfFdt9/pSeoF8Mr/clHHAe46Rtithe3Jv66i+rNfXzvd2rvcdbwnzvBhUI+eBe5yG5XXA5wHwaOcr9bHHwqtWRcKZkzWjEJkdOCh8R7jWiD68btgb63XA2r6MzhfgdsHVU5lrr6+8+tgKSGpOt80Z61POrOv1LMDSzAN7FmVR9XUMdr0V9x3s+AD2ONSIXycKvlrN7H6lfl8ydVw/WLxmPKqvSeE7GsSRsdp6D4QjfnrKuNdP1zr4tj7ZyFtPPRO4Fb7U/DB2L+946/O3e3o/B/SWIu08cN1mew1CmWbhlXcFtOR2sfrjzgF7U8PaX2VU6nkv99sC5yhnLPWjcbSTL7aqr71gjgJxbzjPBGELaKV6Ab7UnC9A22MF/fUyfFeY1vCl5o8pqK/zvU99eV2vZgGWNA/csgraOvdrnQfG+jueA24BYxR8e7pfLWSkmMh5Va4tB26uLmJl9uTwtcCWa9sC397AlepHQdgacwD4Pm/fsW0P4oFKp5WpPcUm17yB781c71ugn+eVwCuloS1zwBhsKdBq5n41EF51DABHuNEZ4Rvl4jUnSXH1Un+963rBd9KUs3RtBbGmfLTbxcp6Qdha1xm+q8bs9eXT0tHwJV3zq8fDPj73dpvrresAqQfiWjsHrElD168tC7LqeE5zAzgqDTwCvla1nmFskeW+W7Ydaeui0tva8RplhWlLf9q+qDisvJfz1ZRp/om0/pTqsPvUwvcpXr/auV5wRelF9clNAfq57rr+cpu30Nz2tb7exq5j0yujOcg/IOM+PD2n9/HtiYEvNefbejAH54Bb54GxGEoPMNMccK8511Hw7bHwStOf9X2LPHQjok4aX/OlBIsZlHbu5Xy5uh7O1VKPxba64b0csOq1f7UzAL8dKGLLjzxXK6eOsaMl6362J1ttnS8731svumoFb+14tfPA2lXQEozrGOyaKsO0rwMu0Hex0yjjHpF69oImoo2n7Qj3q/mycVD4jgBxD/h6QcvVzQ7fp/uPg699xbPlmk8tS/DF0s7YSmcSvutcrwTYtx7fVwnMFHip+V4OxBHzwFR8LQ2E505Be2W93R7zvlHxLX1JAJvJ/SZ8TXWj4DuzAx4FX+Z4yVVb+D6VGeBbp469cK6BW/eznRNeH9Jgge+TK6YWW9Xg/BxRrnHE2pXQEogx6HLApcBrnQeu22CaJwUdpVnga7mPXu7XMqZl7663Ttrzq6njYgLgi3XbC7aW2FHO1VPfWtcTvm4Q4081AriGb33E5KU5BVYavtyiKmq7EbYtSNpOxKWvw+C7dcAUYNfUNFR1WLmUmt5eg1C+lgFSbnXB2n3AnBM+lwOOcnSe/jTtZnK/o6V1yd4vIyeCLxWnKef6sbTxOlpr/CjHi5UZ4btNOz9f8/CtF2FJK54BtIuh8GstYLm08/Y1Bd83PveF6/neNZ1cwxfbeqRJN2vAS7ldbsUzBmmqvq6ryzGwco4Xiz+HA97j9nqu0Pb03XI/PeZ+o7IDmt8rcLvRVi3wtfTtebulsSQgauM0X0qwuh7OVxoHuz/LlxQEvqs4+D43vwbpU9vKDT+X4+74+le57u9FNQbW9nYO2l631l/Bf+N8ARDnC4DDMwK+1i1IGvC2zgOnAwbw3Vqr+/XARQtIz9GLkedgc2N5XbXX/Q7c69sK22i3a30d7WhbynrAd4QDRuC7zvu+ePn8Kbo9aAPgeksPVkalli9D49uCuNSzdtEVtopZV3e92nl7utWN86VWOnPOdxsPQKesAenDuyBLgm6UA+Zgi7nj4wLYe0s9U89Wacbx3IsFlD2cfIT7PQl8I+oSvreKhu9TvzR8pVOuAPBFVJdheqWeuUM7dHVu+GKQxVZAf+7xvcXiPe4YiDLOFXNlgJRvy7jXGFQ5CNcxx0tBt9xKBHyj3C8WG3HkpKQe7pfrs2XuF5j6hK+7/ojw5Rywpkx8vaDwfepKccQkAP1owFXauVztvC+3L3gL0eex6ZXSbvhSK525cit4gagHJgaQOAAaupj7xcBMxWwlrX42amcARww/E3y16u1+e8i77Yjro9M/v3uGbzS494Kv1+0q4as533kVBV/t4RtYH5cyDXCv221XQFNbmqg9wutxkyh8a4eLQRaLAei7IIuCMQbXiFXQyvnfRXLDczvgqGFng+8o92vZW9vb/XJxHJixdh1WPB8NvlIfUllL2zPC91GW852fyhCQrore76t1yZQrpsB8DePrBVcq+NaQpVLU27S0dzEWIPXbMoBbEFNlGhdMpKVruD7UUH7U2wyEvzDHHHCv7qPcX6SL1P6uEe6XUw9nzD3tKGrshO/VawucNW1bQcvVeX9SdRb4Alauf7iC5qCN63J+v+/lVm5XMtvT0rb0NQbim/lgbJ+vBF/NIizqhCwqdQ1MHFbPXQPyU5uShmvYbkGLwfWBAW4d/4V9HXDp13XToRQWWd2vN7YVmFFwtszbamRxv532+lrqLfHauj1ccJRrpupGQFgqq+uvyq+PmNxqu+L5cv3KvN1IuyWpTj1jsdsxsTrMQdf3WW83em537ZAB4Op4yavHCW4hCJvXlGOlnKvG+daulhoDmPjtNcDtfddl22t4hi4H3Bq2lNul3LHSAPd2wNGKhG/UftZoRT5SUEuKiCcVafsYBF8LkHs4YQmsmngtnK1uWBungaclXut2sX5VDljeblTP+wLotxvVwgCIbUnCYRmReqaOqqzPd36Ad7z1+auznUmXW28dqh3sKyIOK9fOCUtgBriFKwZYAroccLegvYFwBde3QdYaozTARwLwKPh62mHjefb99lTE7xwx9ztAUUC1xHohq+nH6mCtfXjgq/l9te01YObaXJXL8F1l2W50GYpOPWOuVZNSvrRp35pErXh+nvfdPFhhC1Jq7vatzU8AGb7c3G+dugakvQW8igVYNXRr4G5hewVmuBYGXSYTbdZBADwq7expr43XxFndrxaC3jrPgRyWcSeY942ItYLY0o+1zFvHxffsRypT1+u2GwHcPt1Igq98zjPubum0MQ5PyjFvX79RHSeJpZvred83Pvf29VONapBqXDAGX8rtYtDWOGIg6qAqA6QNXKArAZeC7fZ1DVjK+XIgPokDjtwLq+mTaz/b3K+lv4iFU545Yq7NBPD1wlgLXwm6s8BXWwdIXB1rccCa+5DaAAC24nmVZtEVwG06+bmchi8d87Dp8xaQmpQyB2ntwq3nRVeI86VAyK12luDLPaxhWwdAg5lbkAVVDFw7XQm6GHAfkLK6HKvfCos9AYCPAF+sjfZwDqnd2dzvVpPBtyWWksUhW+rrMk2dBPKWdhI8rf1KDvimXF7xvIpbdMWtgq4XXNVx122eHe+lXD6j2TPve/vEJATy9XYjKsXMAVMDX8xBAxLLwZZKNTPg3bpdCbo1cCnYeiFc6+CLsHpvpTmboue0I/qPXlXd0J3F+VrGsrpiC8hb4cn1KZVJfUs/sb6tcLfe60sckgC6M54v3V9D9KoPBpScG962rftaX2NjUwCn2nDzvmULQCR1S8IQc8sYTKEqr6+18KXuB56vKcdrBa8Gut7537fhsA44YjVudPvoQzewuFHHTnr7pOJ2cr+1IgGrdcY9X3tdsBd4mrrWn9a6ugx9jR+2IS262sqTeqaAWgPS4n7reWhN6plMSVepZxSqq2vd/sTiuMcQWh/WULtjLq76osA5Xg66rc7XM/8LcDgH3PtYxxHzvp7+tW16PiIRU+t4A92v5ToqttWBa/tsdbpR0NY6XW18XYfFU/eHwbdKPQMAmnrGjpC8hSQN2lo8YLUPUKAXb1GLq9gY7KQrzNFyZdRWJC98LQ9rgOtrDLyY29VAFwOrNv3MpZ6xugM44FFw65F6nsn9Riy+0n4JodyvVo3utwW+1r6pOo8rtsS2uERrDHZvmlisXANhz++Glt/Cd1XEs33xgzP4hyfUfdcLseoxqPljzZwwt98XPWyD+sPt2aXqNfDFjq/UwNcA3hq68HhNQdcD4d5bkACGA3iPox+9fVhSz57+tW2itmB53K8nfU0Be2f49nC7FqBa+vC6YatjtjhXyf1q+8FiOVjflOMnXWmf7buV7mQrft4XBy73IAbu4A5r6pnf73sFRmoRFVVXQ7FOWT8A/jCGliclAVzBVwNei/PlgHuybUgF+i568sDE0of13nsuZurRnydF3Op+d1QUfK3A1bpcbCypzANWbXttf5rfwxKL1aPtVsLw+30Bbh+ygEHtKfYGiLdzxlthgL3c6isUktt2FGTX/p7jArYcaRdY1S4XA3I9h1xDtu7b8KQkK3gxsGoh7FkFLa2A3sYfIAXtVQR8e4+tje259ajjPCw5bicwt7rd6DptWlbbVgtnL3SlGIuLjYKwywG37fel5nQpID8Pz7nhGqz1eDRM1/J6DGqh17af+ktC87yvpo5LKXNPUKqBDNf9UenmFbwAz8DlwKuFbl2+LavL6zoqptbBFmFpFXW7ke5XqyO4Rk/6mdL2PQ5c+Sz9ExgBY02cFrTe/rd1FuhqxpLaSIDF4rwO+On1der5tat53uuPyK0DvnTDO91a3HzwNgbrn5oX5l5fQ5sCM/7Epcvb8djPA/KQhfon5Xg1dVwf29fA9A3VawB0SxHneqm5Xyt4W5wvl34GOCWA93S+3PhRB29YdSfpZwtwRzjjltfavr0OVxMbkYKuy7l+re6eff2cegYAVep5KyqlzLna53bY3O6tw63jsXHrOO7Eq9vFWrz7JeFJpYIpmGJ11FYl7VhVOZZy1rjeGrbYa20qelvGldd1terYk6WgI29zVvd7pvRzkPttAaqlL22dNQ0tvda6QQ9srRCPSD17xqnL0Nf4fl8A32lXVEp5K2nVM+VKNc7YcuIVNw564IbWzVpSz9KCKmkRF5K65uDbCl4JuhYI13VUTK2TOGDr7R3F/bYq4mxnrp3nC8NgjXa7lnux9ql1qNbUsKYNN2Y0jLF40QHrnu9rPe0KEwVbPPahase7X6x8vZ/t/W7r+BXQxIEblJuVAFvP4WKglRZiOeD72c/xrpeCcA1R76KsbRlX3ksTA7gHfPdwv5qxWtyvd1zKvdbSxHVw3L3crhSrifM6ZEs9VacBcg+n622P9SHd0/p6+3zfR2lXPW+FzedqQXuJxVPU3JwuNl/r2XZ0fb/XK6rRhVcA/J5b7o8mhS3t7RXg+/bjIqzPfs7mejHHq90HrNkDPAK2mCYF8EzwbXW/vcEeQZetImHqTD9b3a3lLeDA4AWrJ56rj3CtUW3qtpoY7ZcFqv6q/PEjEjlwYyvPgRu18FQz/zQkzP3W/VErmuttR2sdte3o6h65s545KFKQtYAag2u9P7gac1md7oOcctakoKX0NBDXox2upAkBPOEtTaXeD144uCJSzxFjcbER7lc7ngeyLS65BcJX94LPonHuF0A+cONqCMTVWoRBddvv+horr1PRVIqactwvHvftPP36mPuF6vWrqryOrfvZxtVldR/1WEh9BHw1h3BY5n731kS0897KLO5XK2v6uaVvSpHOODj93NP9euqiXnPjaAEmxbS4YCm+pR9LClrpfrmFVwDynl9M2hOv+JiHq/625VvYUwdqrPGXt+M2VX15O57dLwDI7reOkeaL1z9bJ0sdS4k54nWv72bO97Nv2eFbQ1i7EItLSc+mCQDccgszLRbqlX723r+2XS9n3OGpIel29gAAFONJREFUR7V6wNgCUK0kF6gZL9LRemJb+9CmoAHActzkVprTrZ6Hu00rU5LcMeZSMZhu72lbh9/XA12+PfEKAyKAnI7GoKuJwR7UwKS41wVXEnw/+3jbGISlvcAa9zurdgTwBOwf7n5b5V2c5TkqU7I+HRTpdj3A5RThfq1lkWljT5ueKWjq9VPsA2iOm8ROvAKgockdrmHZdiTPBVPzt5TLvYau5H4BqrlfgGsIAshOdxujATMG4hq+G9e8wpdbcPVZwCG6hS9XD8zrI2gHCkYM2TPV6pHW/fZc/dzzi4F19fMO7jcitsX99gBxXWcBs7YPawraC2HJAT+V8cdNbkWdeIWtRqYXY92ukNaoXji1HZ8qx8Bc3+v6egtqyv2anS0QMRo3LI1FON+3Gee7ha/W9VLp5aOBd9UAQs3gdDHtPfe7l7a/2yz3BMd3v56xrA5Sau9xw9pxLW2pdtrUNAC0nHgFAGQZB1qL+73c8u2iqnqcOt28ff0Sheyrq7ZXC7pq90stgOKgu42VQKyFb/WH2ue7QhdAhi8FXA7CR1QnOpZ+XU/nfveSljKR6Wcqfqud3S8Hjkgwex2v95+lFqYtaWuq3nIPaqeLtbGdeHXdnTTPe+1YMbBaVC/Oql+v19c/6ScePbe5TVev5Vcrn7ffLziHS0G1XqSFtQEgIUv1WZ9wVTvfbbca+GpTz0fVa3vfgE1Rjs37mEJtfMTiMCk9bR3PqwHzwi3uN2pMTVzkfWjhpe1HC1HrPVHlmr8j75eRlzQMa/eLxiDu9zKcZjGVbu53jcf6xg/hoMH8PDY1n/zq9oELANcQBLgGJ2xioKp/qOrqfjCIA/ITAzrAzeMEsT28ALd19wZfgNPaxB6/Vu90bVT/Fnt4QPVIN1tTzJpyD5Couoi/NmtaWhuvSSdTcVdtrvf9ag/dsB45KaWVPdK42fqe6vQzdt83bnl94AIAvc1o+5pKFwNTL80XMylrasUz5Xa3aWguDoiyM+hAn8hRqedI96vtPxre0fd5oLlgS9vWOE0bayJDKusx5+tJS2tiqWuqP/L15shJ5dwvJmw7kSXNTLnf535wh7zWX34tvLwGM97/A9Judb6vbvf9wuYaSyXXIIUqhuurLpcWeQGQi64ouGrgu97CGeELYEhBl1JelFL+XinlB3reEK69AUGN3zqhJ/WvrY9sp5k/Dpz/bQFshNmPyqq3uN+odLQkq6sd5X4raVc+U4uqNAutLJJS19iRktfl13PEWDnWzxPwscVXAPSpV1gMVk+5X+oaNj+3b8mj++Xmfan5Xs1hHGeFL4BtDvgjAPDxXjdCywKRlk+rmZIBlnsZtSit0/yvZ0isbhSMLaC1ppw5tbpfj7O13AcXw76+feACwK37vap7Uad6uYVWOKS5eV2qreSQsXvh4CqV36TVKZiu1/UcMAZSyv1SKW0ulf1YVqeeuf25np9nhS+AEsCllPcCwO8FgO/uezu1op3vHvO4I9PP2k/3vTMKj2pJN1v6peqivq/1SkP3+G7TA+Ct7ldx5GT9uMFLF7j75eZ+OVFOV2qLwbTeosSlpbfbkOrxto8cvFp8haWUMZhSgKaASjloDOKPP+vUM8Ctc/XCd7sn+IzSOuA/DwDfDgBfoAJKKR8upXy0lPJRgM8E3JoVEr2cWHT6eTZNAuNaUQ7X4357mX2vI9bCcNQ/yRb32yjszGdTe2Qe13wPW0CK88o01LF6bBHXlarU781cbZ0qrl+vfTxU11vVfTA/l8fXD48/t7AF4vXZoWqRCOBSyjcCwKeXZfkYF7csy5vLsnxgWZYPALwz7Abj1HvxlVbW+d+RGjz/61VvGHvbtECn5d68c7jS2B4XbClrPPP5+brP4iv5Gb3btPV1Cnn7unbGz28F7pCpZ/4+qYYuwK27XX9iKWUsBa1xzOs4jz+3e34Bnt3v1sEC8rqOkVZAn1kaB/x1APD7SimfBIDvA4APllL+Ute7umv3GwXoyKcUdbBcPeZwveMbFwmFpZ97pKE9jlpzX9o+NGVY+hmuF18BAHvwBn5LtKtt3Wp02x8PXCzNfH0vt+X19XbvLwDgbhdzsxRMAW5Ty1g/2lT1GrKZ+926W2wRFoAewvcgEcDLsnzHsizvXZbl/QDwzQDwd5Zl+ZZ+t9TDEe7lfo+yd3hntcwFj5jv1bSXxmr9gmGdo40CuaYf1xww7X634o6d9EBV42yp/uutR5Sb3cZvf8pnRSOOmlv9DNVPzBnXLrcuoxZura9rMG9+YnO/9RA1bKX9vfcEX4DpTsI6MlBG3HuETYza/7vTHPio+d6Itj1A3KqWxVZUH1IdmUlA5jjhduvR7RB8SnpbZoG0tPqZbnftZjFI12CWti4BwPWDF1bV8Fx/Ui54206CLQX5bf3jn+22IwB8WxFUr6VDNgDuC74Axv/yy7L8MAD8cJc7cWt0+rlF1nsd+YVk4PxvD4fbMgbWxgtpT0qWqmt1ra3yThNwEL7p8/KJL6WfNYuvsO1I13U82LWqH0O4/Sk99xdzwNghHGv6+UlY+rku4+aBOUDXq6glMMOz492+rud8LRDetr8nTeSAj+J+I4Hf8vhBS78tGux0e8/39uizpf+W9LPUhzVNTfVnTTtLqo6dXMWlnwFu3aJnTpd75i96T0zK2zunzH0RuOp/m35ehQ2JwRnglmgYtLEYaqzHuu3K5+15z3VzDLzYrd9j6nnVJAD2AkPzv3+W1c/ROslToSwuloPe6PSzJ7anY49oZ/kyoK0zLr7SPPP3qp5ID3uecERBWeoLP8GKdsZ1ObcV6WXlPAEAX1RFlWPzwnU5l36uob0CdzP3yzleawr63jQBgI/ifDmNOIDjJBo5z+ltM1P6uefiKU8/XCxVZhwbO/nq6hqFlvTwBX4e13R/V4C9TTNv4+oYbBvTWne7/agamHO6NYjrcmx1NADuchkwb0+9WiUttMJWQN+z691qZwDPCqkZ5n8t7VsXYFkt0bY8cP9vj/neqFRxa59WEFOxrYAdtQiLbGtf/QzAp23rOM1Z0NRDFbi+6nlk6jCNerw6pj71qtZ2/rdQaWMszawpX0VtNZIWYcH1vt968ZQmBU3NFd+jdgRwK+T2SD/3nP+11ke3k9p3+FLSG7jasTXfM6IWjkXE76WIRVgA3dPPlzq726XmhLk5Ynwrkm4hGHV0JQDcHr4BQLtXbDEWVc7BHCurwPx01OTDdTgAn3bevqbmg+9ROwF4BHyPJsvvNGvm4ASKnFedLf2sTXRo2kmQxcoUi69mTz/X/T6XXaeit6+p+d/69drPi6slxkCvYoaGcixtLSzQoo6dpB6akCloWTsA+GzwmHn+d/vpN8E9RS9CsvQ5Kv3ckpKWYrWrlq39WfoJex+ZtK3i1CvcYcrpZ6vqOVv8yUr1lqTtNqVbMG/LsfYAcDv/C0ADEksXa8uxhVvMIqw6/Uy5XyrdXJfduwYDeCQE9krhplTqmdK19Lf3orCW8Ufee4dFWPXe363q5/5aJM3Jco8V3I55nVJud9DYnDJ3z6UmFhDXEeWUc96GIOnnbZMaqNS8MHdL96ZBudwD70sNUc8FWB5FLcDqpN7zv5r2Fkj3cL1e9XD6lhjxvcI/erHTr7bpZ2qrUaQsJ2fRz/W9hSp33vPNSVivXj3t/wUAfLvRes0truLmfzVzwNvXr/D08zaFTC2sytOueHX+SJjVTe69AEvq2/K+jVgYFnACVk9wjurT0n/0/K3Uv3UeN2IVNFWGzg/Lq5/r+V8A/iAMqmxb3mv+F0D3BQADc90WO5YSTQ7U87ZSOcBtGpoqr1dBI2Cun3r0VL5psl5j6WmAnPut1SkFXWBe+PaW5vfu9d7c2Xs+2nnu6YSj5389as4wyO635/yv72CO23ld6n6kgzuohzFsX98swNKsUpbKt3WYk66FgLlOP1OQpcx0rnzGNcFBHBZpPwHOPP97hHsU1GP+d4btR1JfVN2ssyrRq6A30s7/XrrQp58l0HLP/q1jtn3S/fFjYTHYAqyres0CLNhcc+XaxVxUynobtoEvNhy1wrkeKt3vsw4G4Nl0Ahi6FXgAx1ajU9Wtfc84/6tNL3vS0C73rjv7+apO8fCFaFlcMrW1iFowxp1j/eSc6wM4MHHlmjS0Zv63ilk219iDF7BrQOoTvLdKAJ9Wk21BotQbfD00q2PVqueqcucY2PyvJK8zbdF2xbRusZbuiwQKbm4B1iqtK9b2R/S7XflMpZupLrC9wamLTghgCTazLcA6kga/DxFwHr3lqGWcqAVYkbKmycV58gpIzPyv5ulH1KlTa/v66UdWaZ5+hD2akLrX+v5uYrYroAH4NLMEVGoBFtcfsg0J236EQXV7q9wWpNSzDgTgo0Jwti1Imr6Dx5zxry56LljrAvdcgOUZk6uzvCfE8ZPybcjQwuKsklZXSwuwuHvSrIC+qteuXN6Kmu+t64R53qfyR2Hbj1ZRJ1xB9TrTz7QOBODemjhNS8rzCWzdahT8vsy+5Wi0Sx6RDraUR2cdNE2ZhzLgQ/FbjzRttQuw1jJsLOyaArXmOMoX9f4eaZEVpofqj6U/wa7W8791KHf0ZP06dVECOFQeWLXsAY5oN5EiFv+MdNs9XW+rtAuv6nJNn6b70D39aKt6AZZV3vlf6wIs3xwwvgJ6XYB1swJ6uy93Kw60WPta2KrnepzH7Uea+V9u6pkrv3cdBMAz5jDvGJRRurcFWD2des/MQss2pACNWAGtFeeGr5/7K7tmqsy8AhqABu3azroAqz5sg4Aw52qpldGpZx0EwFr1WIA1k+4I3hF/VXsswPLKunWIuj6QIhZgkX3D7SP+XPco7Nm19qHuR7OlqBYH2rV++1MoX6prKp3MHTuZzpfXyQDcQwf+hEvJ6rUAK1p7p9Y1dWQb38ewbi5W7tt7JKVmBbS2PR9H/A6at80CWq7dA1EOzwuwuC6lc57TBeNKAKcc6nQIxyhFLP5ugfJe3+mk3ztySsBxApZuKP2pWBZx+3W5/rEtSLr+K1f88Op2CxIAf4SkBrSeum26eV0Fjfxa0vGS6YJlHQDA9+xAIz4RT562PlKauVbv+7UuxOL6sI6pDRcWZuldpN3dYmlqbdrae1+qLUiUWkBrXVHN9CdlsKlV0KlbHQDAI9QLUkchgmYLUsDv4nWFR3kbvSugo36/XlubIvpsdL5bcS5TAiMXoz2oQ3PsJAZX9UEgEY6W29+rXGlNPfVIuo2UXglgl07uKiN1FHhqdKbfRVLEFwflGdDba+0WJM1DEHoIByu/B5iKr19fqQXC3EIsqg/CIT880CdgbRdbcSdfpQumNTmAj3jCwlE06BSsaE1+e6T2OrVq1HhHngroLAnM24cwXEkDUOm7hmfrEjHfi71OtWlyAFu0tyvde3yNJrrH3innMwPhoL+PdgGWa9vOAFmdtfVhEehjCAFkyEpvj3deWanchuTXiQDcQzN/0s18bwfX3quUPWctR99DZ71wbk0aLcupV5zEuWlsBTSA7ulGVL22H0T1HmBD05RBCeBdlRDdVSHznBE3otTewB24B7iWdQ9w9CEcdMztKVgtIk/BApCdrlfEHuC3mXvJ/b4xSgBPpYlSxCN1dIj1HiMCkr2+SHR6PyP29gLQ242sfUSp9XAPVpIjZg7b4BzydjW05QzohLGsBPBQ9QJsgntoX0dKXPRaazfJQyfiQG2bd9ZvWXJkAFrmbLXDKeM8LjilVwI4lRqpUZA6wJcEz2MIe4g7BStSve7/Rq0rpJFyyx7gBLNeEwN41CfInbrH1H3K4/R3gnmvU7Ci2nJA7XoOtOWWB66cyj3Adk0M4JRPB7A+PXSnv3bKL+5QDU5SzDCnCzDF0uSErF8J4FQqQp3PTD60Bh1DaetnP3LtOfaNqIO45th6fXolgFNjNNkq29T9ioK117lKT0LqIgvDg3ifTjdeCeBUqpfOtr3qBNrjVC33mNy2odQplABOpVLxIh7EkLrVi9Z87w4Z7YmS6IdWAvgwytXaqVQqdSaVZYn/plpK+WcA8E/CO+6rXwcAv7z3TZxc+R6PUb7PY5Tv8xgd8X3+jcuyfLkU1AXAR1Qp5aPLsnxg7/s4s/I9HqN8n8co3+cxOvP7nCnoVCqVSqV2UAI4lUqlUqkdlAB+1pt738AdKN/jMcr3eYzyfR6j077POQecSqVSqdQOSgecSqVSqdQOSgCnUqlUKrWD7h7ApZQPlVL+USnlE6WUP7P3/ZxRpZTvLaV8upTyM3vfy5lVSnlfKeWHSikfL6X8bCnlI3vf0xlVSvmiUsrfLaX89OP7/F/tfU9nVSnlRSnl75VSfmDve+mhuwZwKeUFAHwXAPweAPjtAPAHSym/fd+7OqX+AgB8aO+buAM9AMC3LcvyrwDA1wLAf5z/nrvoLQD44LIsvwMAvgoAPlRK+dqd7+ms+ggAfHzvm+iluwYwAHwNAHxiWZafW5bl8wDwfQDwTTvf0+m0LMuPAMA/3/s+zq5lWX5pWZaffHz9a3D54HrPvnd1Pi0Xfebx8vXHP7maNVillPcCwO8FgO/e+1566d4B/B4A+IXN9acgP7BSJ1Ap5f0A8NUA8OP73sk59Zga/SkA+DQA/OCyLPk+x+vPA8C3A8AX9r6RXrp3ABekLL/Jpg6tUso7AeD7AeBbl2X51b3v54xaluXVsixfBQDvBYCvKaV85d73dCaVUr4RAD69LMvH9r6Xnrp3AH8KAN63uX4vAPziTveSSjWrlPI6XOD7l5dl+et738/ZtSzLrwDAD0OucYjW1wHA7yulfBIuU4MfLKX8pX1vKV73DuCfAICvKKX8plLKGwDwzQDwN3a+p1TKpVJKAYDvAYCPL8vynXvfz1lVSvnyUsq7Hl9/MQB8PQD8w33v6lxaluU7lmV577Is74fL5/LfWZblW3a+rXDdNYCXZXkAgD8FAH8bLgtW/tqyLD+7712dT6WUvwIAPwYAv62U8qlSyh/f+55Oqq8DgD8MF7fwU49/vmHvmzqhfj0A/FAp5e/D5Uv8Dy7LcsptMqm+yqMoU6lUKpXaQXftgFOpVCqV2ksJ4FQqlUqldlACOJVKpVKpHZQATqVSqVRqByWAU6lUKpXaQQngVCqVSqV2UAI4lUqlUqkd9P8DnGSSkMm/7/MAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeAAAAHwCAYAAAB+ArwOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztvX/MdV1a13et93ned4AMZZpIUueHjka0NSRCO1Ia0taMNB2RiqZJiwYTf2WSWuPQ0FLxD9umfzVNiH+UNHkLRBONaIttLdUaGiGUhCIzCAYcNRMYwxTiSA2BSWfeee9ndv84977vfda5fq9rrb32Ptc3eXKfvda11tr3eZ7nfM73Wj92WZYFUqlUKpVKjdVre99AKpVKpVL3qARwKpVKpVI7KAGcSqVSqdQOSgCnUqlUKrWDEsCpVCqVSu2gBHAqlUqlUjsoAZxKpVKp1A5KAKdSg1RK+WQp5eursj9SSvnRgL6XUspvae0nlUqNUwI4lUqlUqkdlABOpSZRKeXdpZTvL6X8s1LKz5dS/vSm7mtKKT9WSvmVUsovlVL+u1LKG491P/IY9tOllM+UUv7DUsrvKqV8qpTy7aWUTz+2+f2llG8opfzjUso/L6X8WU3/j/VLKeVPl1J+rpTyy6WU/7aUkp8fqVSD8j9QKjWBHmH2vwHATwPAewDgdwPAt5ZS/t3HkFcA8J8AwK8DgH/jsf5PAgAsy/JvPcb8jmVZ3rksy199vP6XAOCLHvv7cwDwPwDAtwDAvwYA/yYA/LlSym+W+t/oDwDABwDgXwWAbwKAPxbxu6dS96qSZ0GnUmNUSvkkXAD3sCl+AwB+EgC+DQD+x2VZfsMm/jsA4Lcuy/JHkb6+FQD+7WVZ/sDj9QIAX7Esyycer38XAPwtAHjnsiyvSilfCgC/CgBfuyzLjz/GfAwA/utlWf4XZf+/Z1mW/+Px+k8CwL+/LMvvbnhLUqm71su9byCVujP9/mVZ/s/1opTyRwDgTwDAbwSAd5dSfmUT+wIA/q/HuN8KAN8JFwf6JXD5v/sxYaz/d1mWV4+vP/v4859u6j8LAO809P8Lm9f/BADeLYyfSqUYZQo6lZpDvwAAP78sy7s2f750WZZveKz/7wHgH8LF5f4LAPBnAaAEjq/p/32b178BAH4xcPxU6u6UAE6l5tDfBYBfLaX856WULy6lvCilfGUp5Xc+1q8p5M+UUv5lAPiPqvb/FAB+M/gl9Q8A8J+VUv7FUsr7AOAjAPBXkZhUKqVUAjiVmkCPqeJ/DwC+CgB+HgB+GQC+GwC+7DHkPwWAPwQAvwaXxVQ1/P5LAPiLj6uY/wPHLUj9AwD8r3BJS/8UAPzvAPA9jnFSqdSjchFWKpUSVS/ySqVS7UoHnEqlUqnUDkoAp1KpVCq1gzIFnUqlUqnUDkoHnEqlUqnUDupyEEcpX7IAvKtH16lUKnUA1VuoLdfaOq4N1r61X889cvfDxWvLvGNGxVL6JCzLL4sddToJ610A8OE+XadSqdT0er26rj9qufrXiXKuTtPGM6am3FKH1dfXWBsqjovXtrX2pdHvlEMgU9CpVCp1YlnAM4usAIyE71glgFOpVGpX7X0k/7yAGq+xfxcJ4FQqlbo77QV9L+zP+SUhAZxKpVKhOics+qj3ezV67temBHAqlUodQtxCqyhxi6k0bSx1qQRwKpVKdZW0CjjlVxTg9/mikABOpVKpaeRxuTM549YvF5btR1Grn/dz6QngVCqVCpMVQD0//F8nXu+lGe6h1r4p8gRwKpVKdVNU+jkSXp55Xk1fljrt2L3d777KGfJUKpUKUeuH/t4p01Yw94S5RaPfR//RlQngVCqV6iLL8ZNSW0qaoyIjwTjSwXvc7wj4RpwVfVGmoFOpVGp3jUqZas+PtvZlleXLSLTmgC9AOuBUKpUKUE+Aah+I0Hs8i0akyWtp3g/PfcVCd6sEcCqVSoXL+/QjqU4zHpV+bv24124/sj79yPrkI2l8Stbfvx94VyWAU6lUqknWx+95Fel+90w/e9UypqVtf/CuSgCnUqmUWx4QjnK/HrUeBCLVed1vS+pZ+3uMA++qBHAqlUq5pAFF1GIjD2Q1c8fRp221ztFa4Htc8K7KVdCpVCplltWladSaCp7d/daawf/tB1+AOd6BVCqVOpA8DhSAd8eWOuvYVHvP+c49n3wU5X7nd76rEsCpVCqllvZwiJaFWR4Yag/esK4WjrhP7FrzfvSC7/7gXZUATqVSKZV6wVfrKj2uWNM+YrEXVzcLfOcB76oEcCqVSrGyfOi3Pg2pdR6WGr/nimoLRjSxVvgey/VulQBOpVIpVFa35VkV7anzuF+q3DO/25J6luKtccdzvVslgFOpVOpGI+DrSelyfVpTzK1zzVJdROrZm3KeG7yrchtSKpVKXSlihW2Ped9aniMnW1dEa/ryxkb1dwz4AqQDTqVSqUd59622LrqKTgNr+m4d03reM3df1D140s7HgS9AAjiVSt29Wg6MiIQv17f2HkadeEW1wa6lLw4Rx1EeC7yrEsCpVOpOFQleLD7qIA5u7EjIRhwMYj16837hC5AATqVSdyXrQRTa9r3gG7HoqnUvsLZNxP5nC8CPC95VCeBUKnUH6gVerN2IU7Ba9/VGrMDeqkcqnhrv+OBdlQBOpVInVit4pT72OAVLGxd92hUo6xK+WiWAU6nUyRS1jaZ1L3CPfcD1dS+HG3E/Uj9YPRaz6lzwBUgAp1Kp02gv8GJtZoFvrehTsEbA93zgXZUATqVSB1bkoRE9wCvF9IZvrzaWPjz1qw4C3/r2H3zNUqlUanJpoQvQ5na59qO3Id0bfCcDbydSJoBTqdTksgAXoN3tcn2M3obUc/529JzvQVLOA6mYAE6lUhNqD+hy/XjBK8UkfJ+1E3h3pGACOJVKTaQe4NX02+PwjTrG4oqtMMMU3caTuqbGngC+UfTD+nk19hZSqVTKIStwAcZAl2sfvQWpro8A3V5zvhPP93pp15GSCeBUKjVQHuACxEFX6qt17y8W1yPFOyrt7F3BrakfAF4r4QYTUfU84FLKh0op/6iU8olSyp/pfVOpVOoser36Y9HLzR/tON6+qPZUOyy+jqtjsHouth4Pq9P272nTsn1qR/ha/tm8BFt8sMQhSykvAOC7AODfAYBPAcBPlFL+xrIs/6D3zaVSqaPJ63BXRTpdqT+r26XatKakW0DXq03EPdV1AN3BGxET0e7zcd1+DQB8YlmWnwMAKKV8HwB8EwAkgFOpFLRB1/rJ1ppilvoYuf1Iit8rHd1rvrcTfKW/7onT0Jqh3gMAv7C5/hQA/Ot1UCnlwwDw4cvVlwXcWiqVmlejnK5lrD3Bi8Va4BsNWK7uJPCNAu+OK6E0Q2Pv3HJTsCxvAsCbAAClvPumPpVKHVUjYWsZrwW4XHsLdLH4HuD11mnbHCTlHAHdVuB6qens6lMA8L7N9XsB4Bd13adSqePpqMCV+vLMB7fMBXvBW9f3hLIlbkfXy/3VRaegvW06DfMTAPAVpZTfBAD/DwB8MwD8oa53lUqlBqoVuADHc7lc20joYmXRKWWuLiLdbBkbIAy8I6C7Y/pZNfyyLA+llD8FAH8bAF4AwPcuy/Kz3e8slUp1VLpcXTvvqucW9xjhhrl2k6ebqb+KPdPP1naBKWhYluVvAsDfNN5CKpWaRqOBaxnzqNDFYkeA11vXAlcO/B3B2+KCtTGe2CDtbMBTqVRfndXpSn1Ywduy2pkqiwJcRN3ErtfqeCPnfGdPQadSqaMpoatrExHfulCp98Ksg4HX63hnSj/P0XUqlRqrFvCeaREV167V6WLlmoVYo93wxOlmC3i9Llgb44kNVAI4lTq8vOBN6OpjJWhpYnpB8aCOd4/UcwTxNH1ELsJKpVKzaZTbPSJ0e6SWtWNZY6LqJgWv1u16oNsDuIOJmABOpQ6lEW53ZuiOnM+l2nv2A/eY+43sZyLwtkBX+8+8B/m2faYDTqXOpFnAe1ToalPLlvYt0JXqpbYR88RYfTB4ezpgTb02JqLNvMOkUimfZgGvps/ovbpndLvW+hOlmkelnidPO08ydCqVojULeI8I3T3cribmZI43Os3she6ItLO1baagU6kj6gjg9aaZZ4XurG44atV0XT+R251h8VU64FTq3nV08PZ2u9pY7VizQre+nsTtYt3NAt2RC6+0faQDTqWOoBnA2yPN3Au6Z3S61nhL3Q6LqiKgO9IBe+KDlABOpXbRiH28ezjeiLSxNrYndD0xPaFb1x/Y7Vrnf6X6GR3wPt2lUilZR3a9MzjekVDVxCR4xRht24g6S0xEmwYlgFOpYTqy6+3leM8K3Tqm5x7hidPMoxZe7b3oyvlXkgBOpborweuP2wuonhjp/qPnkQO3EHmg2wrX3sBNB5xK3bNGgFczztHAO2ofriam1elKfe4IXaxsr7neVuCOnAfW9JcOOJXaU7O7XusCq2jw9nK7e6xgrssmcrpYd73mea1w7QVjbUxLfJASwKlUqGYHL9d+VvDuAVWp3vpl4Y6gGwncSNhG0U7TTzrgVGqk7iXdHDUHq2k3W711DvjOodsDuCdyvzsPnUqdRQlfW0zvudmZtg1p6ncG797zvKPmf62xLX2mA06lRuhs8G1Zsbx32jYS2gndZrgexQF72wQoAZxKudQCXoB953tbXG8P8PbcI9vidhO6zWVceUudpt4bO6KfPt2lUvegdL14zMjVwZFQ7pnO7rhtKAqoe6egpbromJb4YCWAUymT7mGVsxUyWEwUIKOgfFDoRgC1N4StsS11lhhLXHT7nANOpSJ1NteLxUe43hGu1Vt3Euj2grC2naU/Tz+9YlriO2mS20ilZlbrfO9s42jHOit8I1PdO4J3hhS0p7ylzhJjiYvuIx1wKhWhe3S+1rT0aLh6U83ePiaG7mgIR5ZLddExLfGdNMltpFIz6mzwjZjvjXK9PcEbPS7A1NCNdLR7gVhTr42xxEW3XZUOOJVqUcJ3zHxqJJSjoe6ErhWIo+Z4ZwJuBIwtcdbYFr2EBHAq5VeudO4D31ZnawWvB8hBTneP1LIXsBaA7gXbns53RwomgFOpK50dvhJ46xjvQRWRgI0Cb2e3e8ZU80jXGxnjiY3sJx1wKmXV7PDl2kbAV6rvlTo+GHhnh+4RgHvHrnerSW4jldpbs8PX2kazx5eTxflq+pgBvpOCd2S6WdtXZHlUvTXOGjuinzHdplJH0hHgG5l2rmM41xo5p6spnwy8FnBGxWLX3pgeZVLd6FRzOuBU6qiaHb6te3ylmJ7wjSifALxakJ4h3Tw61dzDzbZSLYKKOQecSkm6B/hqnClW3wO+1HgW1zsBePeE8N7A7QXiXnHeeKvq/hPAqRSnUft8vX3NAl8NNK3lra43GLytoG2BqMfl7g3hljpLTI+4qHZBSgCn7lCj4CuNE/XfT7PPl6vrAV8vRLV9DQLvDC7YG2Mp85RLdZp6bUzv2Og+0gGnUnuqBb69D9nQ1mnisHLt4qy6TNvXAPhGuN+90s0WkI52vzPBdgL6TXALqdRIzTDvuyd8OSBa5lcj5oFbUs6B4J3Z/Y5IN/eAbY/U8VHcL0A64FTqVglfvM664CoS1No2ja43GrwjoTsKuHu73jtwvLUmvKVUqofuDb7WeizOA1RLe63DbXC9reCNSEVb+tG2by3zlLfUWWI8sZ741nYBSgCnUqyOCl/PXt/Z4Iv14Ug3t4I3Iv0c4XJngXBEvTXOEx9NN0t/mYJOpVZ53e+I/x7W7UaauB7wpcaMgC8X3+B6PZCNTD/PlnqOLNfWa2N6x0a066CJbiWVOqr23G5kiWmBr7RCmWvfMt+7KgC+HuBSUNwbvHtBdy/na20TSTZPX+mAUymA/u53ptSzZcWzJE86OhK+RucbBVmvE7bUea61Mdq2UnlLnSXGEzu6TUdNdjup1L1oJHy5/lrTya3w5cZ3ut4e7rcVyBHXrWWecqlOU2+NGxUf3X6rdMCp1MzuVxvvWfHM9eFNMU8IXw6aLRBuAe+RIdxSZ4mxxLW260k4ru8EcCrl0Z6pZ400kKbAGrHiWRNnga9zpbMGoNY6bz0XH3HtjfGWS3Waem1MS7y3TUu7YE1yG6lUtFog17Pv1tSzFKOBL9Ve+lTHnGsdxzn2TvBthbFUpqm31HmutTHathF1mnprXGu7KKINImMCOJV6Uu//vXvO+0pxEmgtW4cGOt8oCLc64vp1j+vWMq68pc4S0xI/epwBmvjWUimv9nS/vT4drP1GzPtyZVh/VJ3GOU8GXwuQufiIa2+MVN5Sp6m3xrW2m8n95hxw6j6156EbXB+W+2pNPVOi4Mt9mmv36dbul4Nvw4KrmSCsfR1xrY2JLNfWa2Na4lvaTUy5iW8tlRol7X8DL9xHp541c79YW+scb13GtafGEOCrgaYHtFrgWsBsqfNcW8o85VKdpl4bE9GmpV1rW0npgFOpker1X0mCb63RqWcOvpIb7gjf3u53NghbY1vqLDGe2Na2kf8VvX0lgFP3J49Dndn9Wj8FvaddtZxSRSkIvhT8ejljT1n92lKnuW4t48pb6jxx3viIthPSbsJbSqWOpohPn+h5X8tpV1x/HHy1874Hge9MEPbGeMulOkuMJW7vdlHta6UDTqUk7eV+NbL+1/QAXCrTAJ7bXhQkDHxY+SwO2ALbSPD2gG4klL3xEe0nJd2kt5VKWdVz6xGnI7lfa1ndnwWyQe43+qc31lJvrdNca2M85VKdJcYSt3e7qPbHHDqVOoIi3W/UJ1hdbz3xSlOmgW3n1POon9Y67WtLnebaUsaVt9RZYlriI9r3pJvUd6agU/ejXouvog/d8IwhxVgPyeDKWlY9HxS+VuC2QliK1bS3lHHlUp2m3hrX2mbvtsGa6FZSqSOJ+6+jdb/Rx01aFl55z3XWyNhub+j2cMKWOs21NsZTHlXvjY1sH0W01n7SAadSlCLcb3S7vceKdL91O+bTiGqSENbHaNtq6zT12piW+Kg+JqbcxLeWSmm0x+KriE+eVvdb1/VyvxJ8pXYOK3AUCHtee661MZ5yqU5Tb42bqW1Ee0zpgFOp0dprJXarWu67oS0HOWubnvDlxudeW+o015YyrrylzhPnjY/sY1LSTXpbqVQvaf7JR4LU+2kWOffL9YuNQbncDu63BZDePlqd72gH3Apirs7ypScqbqa2Ee0xpQNOpWr1/Are8rQj6315j5zEylpWYTdstWoBqqfN3hC21LXEeMu19dqYlvjZxvX0nQBOnV89Ur4zpJFncr/WbUerFAuvNIDiYGht0wJfL4QtdZprSxlXLtVp6q1xUe0i2k9CvkluI5U6gqzuF4u3ut+6vof7jbYoyq//2yYRLtfTZpQD7gHeSPc7q/P19BFJtc6ETACnDqoIEI1W66EbXGyL+61jWt0vIyvwtLEREPbGaMeX6jTXljJPubZeGxPZrrWPGT4CKk14S6nUXtpj8ZVFmiMnJfX+L+9wv1wdBWUuVtO/d0yvA7bUSWN7yrhybb02xhMb2U/0P3FPfzkHnEpFypp+9vTpXZxFuV/OoWrTz9R+Y0ca2+N+ufYW59r6UyrTvrbUYdeWMq5cqtPUW+Na20S0n4x4k91OKtVDo/+ZY+O1uusId869D1JaWfMeDnC/HpfaA74zgDcy7Tyj+90zzdzaTzrgVMoi60MNWtViPzTbglpgK2mA+21xu9y4e8B3Rgcc4X73cqF7gjlYk95WKsUpeqtQ70cORi++ksbz9luDtRXURPdeJ+Z1zVR7y/1o4euF7ewOeKT7PRLYKaUDTqV6auTc74yLrwynXlkAh5VFO2MJUB5XzL221HmuqTJPubZeGxPRprWPiSk38a2lUhGa6WuxR9rUeHT6uWGxFSYrzDR1dYxlXE8qmrun1lS059pSxpVLdZYYS1xk+9n+m6cDTqVGy5t+7qWW9LNFhsVX2JDeOi+wLWNZxuReS2NGwJgq75163iNlvJf7DtaEt5RKjdTei68ijp2U2rfAlbsPxfujdbqaeGudFOuN04zbKx2tjfGUS3Wa+tb4vceL6mO+oVKpCM14VvPe47Tej/aTujH9zNV5nSNX1wJlb6pc+1qqk+K1baTyljpPXGubvcbrJPGWSinfCwDfCACfXpblK/vfUioVpZH/4zTp55H3Y3nwQt2GKsf6aFx8pS3rAWWLtI47XbA/NqL96PEoKWdlXlPE/AUA+FDDraRSk2p0+lka35N+ltLSvdLPRmndI1fG9eVJG1vcr0Yj4PtSEcPFYeVSnaa+jpFiuTbesTRfHLzjdZI45LIsP1JKeX//W0mljqDZ0s+9FbT6WVunjfeAlpMF+NJ97OWCLWVcuVSnqffGHmksTqNXQZdSPgwAH75cfVlUt6nUQTULQL3awQ7sqQc41q9s/fLS2rdnHO/9eNqNahOssFtYluVNAHgTAKCUdy9R/aZS86vVYo1UnZKe5b5SXWR1vy2uWBvTEj96HG/fuQ84lTqLrNuPWtz3QZz76lgx58rV9b6fvRSRep49Jd3beUe1n3OoVCr1rD1XSE+svUGmkeUeZ/t9IqGsqbfGzdZ3Z4mroEspfwUAfgwAflsp5VOllD/e/7ZSKUpv730DEynSrXZyvg99ujWp9R4eqp/e8fZ+L0bDt3U19F59W/vC/kSloJdl+YO+O02lUj5FwXCSr/lR4tzkXq50Noe7qtU9jkhH93KuIxd/NUqzDziVSgHAYeZHj6LRjrAej7q2Ot570l7wtbhia5+trrhBM353S6VS9y6vs/QswIpysdt+qD6pmJmcdC8H2WOh1ojFXx37muWvPJWaSK3/LaxOmRsvXfeNLJCNhis2tjSG5x6igWxZJW3pQ9NPNEx7r4YeSMVMQadSpxIG7AkgzqV1vXV1jGV8aaGUd9yWtPUMKe8jLNSK6q+O3yEVnQ44lWpSz1UhKZNanGaPFLAm3awdd6YU9apZFmr1dsSevgIfxpBKTabcijSFq5Xkda2j3K5WXH89F2zN4IgB7OlrqS7a7WpivIu+qD9BSgCnUsPlhWdPCyR9qelEg6jUdAuYpZ9acYD2vNaMpb32KmqeWNtXBHg9ae2d0tAJ4FQqtVGH7EIUDDyQjZa2/9kc8Qg33ZqSjgKvRjtuPdoqAZxKHV49PkmoT+wa0I7nrkSnmC2x3kVYWocuLczSuF2PI45Qr7nUqJR0Sx8tDvflYv+Tc8CpVKpdlCN2kMGSJvXEemBlWe2sgTFVZoF0Sz97ybNKunX+NjIVDUDDtKMSwKkTa6ZPqBQpC9h6zvW2zCtTZZZUdetccK954K0s7rHXQq2oVPRA0JK3sMuoqVSz3oY5VwLPeE87qt4+o9lOo2lTbx/yjGMRNx5XhvWhfa1pO4OioNxar4WuVy8N32iKbpx0wKnUVDopwD2rdiPbRLpgrWO3rIrGFOGCuf56Ombvth9PvdjW6HJfPuB/OigBnErdtXZeWtuSYg68DdV4VhhjcZr6yMVZe8zCWOeDe7peD3QHKgGcSqUErQuxHqrrVR3mz1rmY3vMAXudcc/FWTOodeV0D9erdbxR0H356vZProJOpVIX7XxymDa92xOyUnttPRfrKZP6bn3N9bu3WhZpoeWdoIsBdvunQQng1IGVR1L6pXnvBn1aW+dtqXaWMTT1Hmes7VPqj3stjaFt20PWYyu1fazlHHzZPg3QDQSspARwKpWqZPliY/x07wHTnqlnaQyqThOP9U/VU7Ga8paxeilqnlhyvRrwDgRurQRwKrWrZtpP0pJREByIFhpR0LSmmKPngaV70brfKHfsgbMk7wlWVJkHvuS4AnidwH3t5SvVH+26iARwKnVX8loeaSFWgCLcWIs77OWIPWXYWFQ9FTujwtLRBOC04FUIB2usEsCpVGqMvO511E9OFhhr4y1lrWCObEvJm8yJcr7B4B2hBHAq1VUzpZg94j6da1e8w3F+kuONgLsnPS3dS09HbE1V91LLQxai4Suop8tlxx06WiqVmlDYh1enFea9XKvkZjVzuZ75X01MzzIgylpS8b3UvGcYgS+XchZcrxe6L16+Ev+UPIoydR/KrUix4t7P2vEO0p6pZo2jbYFwXRdVpqm3fFnpIcuWJQq+aKwOvBphcI1UAjiVUmnmM5q9dkf75SU4DT0CmD1SzVrQUjHWPrG+qDKtS6a0pzOWZIUvIS14e8EWvafuI6RSp9XR53e32vkTeKSbbUk1a2OjXLK2LzCU7fFXXf9XUW9PaoevBrwR0H3x8uHpTz4NKZW6S41KyXOf4g4X3HILFjhHttH2YxlDqx7paW8iJUId4cvJC90tbJ+g61ACOJW6C23BrP2wwGDOlTV+Uvd0rL3aRI1prQOkzlJmAbOnj5HJIQd8LeCNgi2mBHAqlaqEQdZqr4JdcC+Xq4FwFGCl2Mg6qWwmtbhfBKJcylkL3h6wxZQATqWGarbFXJ6UNdfG+YEV6VyjY6UYi6NudcK1IvjQG9Ca+d+r+jb4UpLAOwq6WyWAUyfXjF/5ZxX2XkmAXttgcQYXjJW1uF2uzupO9wC+5r48ddrxNPWj1AhfyfVaofvy5SvxT8nnAadSKb+0aWhMxk9ub4o4Eoxcf5r2lrGi7ivSGY+CseSGa/cbAF9KWvDWcI1UAjiVSgmSFmNhLngtUx5PGeXwvDDnYnqAFZMlVtPG8p5IZbMoAL4a8PYCbq0EcCp1SLVuN/J+yg78dG51eFFOVtO3B/hRoLXICuM91TAXy8FXHHbgedAJ4FTq1Np+4FDQ9mxRovoLdMFS3V7p5Ii0cWQK2ut0veqWjhb+nSjdrwe+HsebZ0GnUilEUYdxvE28xiR9KjekoqPT01ydBnhSe21MT2fscfFcjLdeK83cbyUtfLmUswW6Pc6ETgCnUmGabYuRRRZoS67a66iZYeqyFtDu4YA1/Xp+euSFcqSLtqoCngW+ZJcTnAudAE6l7lbUhxMFWIsLdm5LqrvRwBEr88ZLgLWAUXq910+rwlxuTDet8JVcbz6MIZUK05kemBChqPQ0BVvJOg2GcKQ7toCOGkeCYXS51YGPEvffsgan4jzn2zIavlw/vnOhkUcW5j7gVCoVIwra0qd2YypIM4rRAAAgAElEQVRaYrk1XRoZT8VIwNOOEVVe188obvGVIvVcywtfrSLnghPAqdSV7tkxa4BpXUndkIredmVN52rLPC5Zajc6lWyFq/bLgOaLx86qAWiFr/5s6D5p6QRwKuXSHqD2fiL2/CS1WC5HKnrbzALhnmAeCd/6/rgyb5+WfwK9wbsFqOB+tTDk4MtpxFxwAjiVujtp54EpJ6txwVT5IAhjZRQ4rQ54JHwfkGtrHxHy9tfpe6p23tcD31ELsAASwKlU6kqeRVoUqDXlnSDc0wFT8S0/6365Oo07tt6nFK+RB9LS4RuPkuZ+tfDl4Gp6RvCLV+yfovx3nQBO3amOvGd3L1m2JK3SQNggDpDUa48D5oA1g9OtX0uxknqnljVi0s9baeZ9KfhS/anmgTeAjVICOJUSdURYew/WsNRZFl5xMrjgeiisTFsvxVniW1LDHpBaIDtberrz8gkrfCVFQ3erBHAq1U1HXVHNAZSrG5SK3jaVnLCmnoqLdsLbsVtAbfkiwKk3mIO0TT9L7jcSvj3BuyoBnEoNU7STbnG5ddsIF8zVdYTw9rUmHa1J63pgLf1sSTNb6ywx2jY9AU2kn7m5X82TjTzw9YD3BTxc/dH+W04Ap1IpRByg6zrt3O5ACFuATLXxuGLuJ3bfUfDl+sbUAlMPkAckg2r3i6+UZhZhKcFbw/ZFw5uZAE6lUg55UtFcXTCEt68lIEeUaYGqAbi2DpAybTxVNzINrVwBverFVSpaTj1z7a/KFeCNgC2mBHAqFaIzLNSypqm5VdGTQXj7utXtetPP2P22wNTqdDVjTiTNsZOUbueKafiy/XSA7lYJ4FTqSUddNKVV70/ZCSDckoZuKYtOKUuQjEhL90pRc8L+iym2H1ncbwR8W8D7El5pn8WQAE6dWb2AenZQc5JcMifPgi0jhCmNhDA3fi9wtrha6/gTy3KQBlln+GVfwqubPxYlgFN3qFnSxb1B3nqqlacPS9paW7eA+cQsyglLALSknLkyTapaWxb15QGEsonEbT3aSpr7RRdiEfDVuN4W2GJKAKcOrt4wnQXWvaQBrHWuuAeEAboszrLWa2HNtbP0ifUl9UcpwjnvDO5t+tmTesbgK4E3Eri1EsCp1LTqAX/NJ6jnU7YVwg/Kus4rpKMcsPQzGrTS70nF7iVsBbThEA0pzgJfSr2gu1UCOJXqorPNE2NAjXTCdT1XN8HiLE06m/vpGZN7jSna9WN9t8IcOUyDWv3Mud/ruBj4WvQCXl39yYcxpFKHU6vj9X4aYmlorC/PnHJPCA84tMPigDXQle4Fq+faePs6kTRHSt6UEW+KxvXWsH3R4JITwKmTyupAsXgtEDVxZ51LloDqaaOFsENeyNVl1G1409XSPVjgq9FsgBbSz173a4Uvp1bYYkoApw4sD9RaQXiG1LLW3XrjpFS0pk0N4aAV0nXzCCdsSTdHpqDBWO5JWWvG2FHaOeKneOSXkFxvD/CuSgCnUofRkeAfDeG6vq7baYV0K3wl8FP3TPXLlWP9TAZV6fSrl8z2JMn9UvCl1BO8qxLAqdShwNZTkS4Yi6udLNauZfX0TmdIa+Z6LcCV4N1SvqcM/824k6+keIA2+GrBi80FP7fNRVipU2uP9HOkRkHfs3CqVnTKOhLCdX3HFdIaCEt1dUwdZ4Ev16c0lqcfr6h/6itYHWc+S48TvLo2wpcdN2Dh1VYJ4NQJFQG3FlgfwVFHfMqOhvAO88KtZaPmdFthOgrGAKanIGkWX1nngTH4clCNhu5WCeBUSq2WldJHEeWYqU/iaAhzkMX67jwvjLli6/ywta0lNV2/xq4xWcE6IIWtffpRi/ul4Ev2l3PAqVRPjXCrZ4O0Rl4IY9oJwly9NPcquVOre/XAdwSYOwgDrMb9RsLX6ni3zwt+AQ/5NKTUmcUBjQLqKAh6gT7TnHaEC6biPeloTUyHxVnaOWHPXC9Xr32tGctap5E2vvG7rbT4ypJ6tsJXUg1crxLAqdSNZnCsnnvYc+659bGEnqMuLfWd54Rb5ou519Q9adu0yNsn98+wnv99hGhr+plzv1r4Sq43Ari1EsCplEqR87+zLNLyfJBE2SVtOvpAEJbKpL6xWCnlrZHXGfeUYmvRU6jxIQ0SICn4cv1FQnerBHDqYPKkn6Pi70mco41IRVPxXgg/VPXcCmmDJLhSsdr23GstfGeBKiflf7UVrFj6WeN+pXlfC3yt4N0+tjAfxpBKPcniVKO3H93bOdEjIexdnBUI4WhnzJVjdVHOeFJZ3e9VWwG+VMpZA94tbFseW5gATqVcGg3NPdy6xwVz7aIhzI0ljR+wOpors6SgqZhWHQCwreLc71WcEpBa8EYpAZw6kPZKP8+Yqp7BNc8IYSwdTbVvWB2tKa/LNCllb50US5VRioI3uyDregGWNf2MPenoedgtmLWLsPBfutXlckoAp06uGUC1lQbmI++59ZN2BIRbtykFQrjuoiUFTcVY67BrSQdIVVvSz9Kq5+vY2xXTnqck8co54NSpFOl+PWNIcWed/209SzpqT3HrNqUgCGtdZuRcrwRADSBHQFT6b7huQTKsgAaQ3S8HX2ze9/qadr3ifbEPY9ApAZw6sSi47ZV+3iOVHQF4CcLSB2ovCGN9tDxXWKnWFDTXB9WPN1Zq71HgP2Mq/dyy+Oop3gFf7bOB82EMqTvSXu53lrnfWe6Dk/dTfTSEubaOdLRnrrV3qjmijVbkk44UMQp53a8Xvug95MMYUimrItyvtt/IQzqOfrgHpch9xa3nR3PtlRDWQC5yDpgaQ7qvCIUmgR5v0OpmMRAHwZdyvR7orm3yLOjUSTRijnSvedjRkG6RBnreVDTXVjsHbSUP9wCHhiGj5oBbYiPaeRUIa+vKZ7QP5Zyv1I6K8c79rkoApyaWBJojbT3y9je7s7UqCsLRx1YGbE+K1GypZo+u0tD0e6qZ/71po3S/VJu1nefxhDkHnEqxYOp18lXv9HO09rgPzSf8TBDW1ima9FqENaP7DZjjtT6AQXK/XOpZ+2jCqMcTaiUCuJTyvlLKD5VSPl5K+dlSykfC7yKVulF0enY294vd51nnf61qhbAUo5kPHnhmdMRK5mjYtvyTErcjYSucr93vi+rnVaxiz68HvlQ/nucCa8+C1rzNDwDwbcuy/GQp5UsB4GOllB9cluUfqO8qlQrVLNt5ervflv61cZb38m1lvw+KfrV9aVT3pRmfUkvbgG41cZ1ukZUIVUW54QELN3GI+6VSzxb4ep8JHPV0JNEBL8vyS8uy/OTj618DgI8DwHtCRk+lUHm3HVnd70xbj2ZJW0ep9QMqelHWABeMDSvVj14kRcnzz17bZjv/q3j+r8b9auB7cxvKJyNJT0fa7XnApZT3A8BXA8CPI3UfLqV8tJTyUYD/L+buUneoGUHU4n5nniO2ynLfZ0uLM/LsBU51V69n+EZKDeBSyjsB4PsB4FuXZfnVun5ZljeXZfnAsiwfAPiSyHtM3Y1aVj1HuN+9tBfg7wiSKbvqfx6WNLThn5a0+tm69QjTyOcCW6QCcCnldbjA9y8vy/LXu9xJKsUqatUzFx8NvZndrxW+6X6nkTr129B2gOxHSyJzyMq5X7lv+SEPWl1gH7QIq5RSAOB7AODjy7J8p/luUilRvaB01EcO9gZ8T/hGaaa/j46K+jV7vV0tLviq3OcgpcVXGlkf0PBcrrvnlscUahzw1wHAHwaAD5ZSfurxzze4R0ylrtQKjJ4Lr0a439Fw6w1fbf+zZAICZHGaM6xeHvV8EcM4mu1HtTzuV7ulSIJv1DOCxbdoWZYfBVAfbZlKGTQDfK19zyKP+50Fvi2K+ns5uMNuBWtL+8a3TnsAx1beOVjrs4E1Y7VCd6s8CSs1sSLnfT1jaGJb9uWOnDeeCTjRXxQ0fcz+hepRPf6aev/Va9PUzAEc+qH88Is4ySoSvpf+Uqnh6uV8uXYj3W/kf6s99ip7fv89P0pa/74C4TzybTiCC2bOgI4SlX6Odr/R8L30mUoN1Uzw7eF+W+Z+W/47Rj0zuTd8e2w166XAmbegudLw8TUQ1tyf8Xfg5ng1Zz97HhMotafg2wO8qzIFnRqoHit0ve16LLyKVgSkR+cfOY0CaM+DjANkXUkc9Vce/baoXfAz2LD533oBVi39IwT9e3X3gO+l/1Squ6IOhvA42b1Tz3u63yM5X6k/qr2m3zpm0Mde5C64nrcctVLaeRCHvnv94RncIwq17tkC37pPbe4kAZzqrN7w9bQZlXrec4wzwdeinfY4a+BjAdSM25Wa/on7FmBJh2+gQwnP89WMoYFvzKKuVKqLIo9DHDHv2yP1PGLfbwuoRyy2ioBvpPvV3scEOy81AOy939gK4asvGfoFWPX+X83xky0nX7W3i0lN5xxwqoMiIRO1uIiLH7XqeaanL3m0h7tsgS+m4Pd7tJPtsfK5RY7+rEdQXobxAc8DSs/qaa+O9L8/Nb2izwvey/lS8aPmk6NP4NK0k9QLvpGrnq00fJ2pc6j1+6AWrpayHq7Z+aWDW4ClVWT62Zp6jgTv83ipVLOiPyi9W1VmPpxjxLGWPeDr+YjoDd/eK9WD0889tiC1grX1k187n+08AxpAf6QkFRcJzB7wBUgAp5o1C3w9bUadCz3DKUwzuV5P354xuPE6Hb7xEinjbsNaL8W2QtgLa+NfJ7cAK+Lxgzd9NrjfXvC9jJlKmdXrw7zHIQ0RbrnH3O1I93sU1yv11Wu6wLn4ygtLKfXcuvrYCmFsvJYFYDdxG7eKrobm9wGvYLSkn63QjIDvNrZEPY4wlXpWTxc1A3xHrXpumWyz/h30hu+oef+Wv7OO7hcbgnLBIyFsVYTLNa6Ats7/rrKufpbcbyt8WxxyAjil1D3Cl1LkfGNE+6g58Vnha9EIt+9v1jRGj7lcy/gNsj4BaYWkZ/UzB0T9qVq5Dzg1hXpu92id7+0N39bU84iFV5axKc0O3sgsAjeWY/GVtBipdQGWF8LWFdDe8VmXr1+AZdn/+9RG8eCFllXT9Li5DSnVXb33WfZwvVy70XPEvVfqRvTZc65X2791tbOlLwm+A+Z+63hLWtoyvgW4rXAWIYynn5/neh9uyiS9UM4D8320u9/oBVkJ4FSlEQccHBW+PQDaMs5I+PZw1F74en5vCc4OYXO9mvlfqh8prm7TA7gtZZic87yXIXSpZGrxlcf9joTv5R5SqS7zj95xRsHXOoblQ7/3Xl5LXGu7HnOqnpQz13crXAPcrwVSUiwG79YlCty4mvGs97zRa1eroB/ndpGV0Wv6uXaqEeDTLLyixsltSKlOOgp4pT488LUAdRR8KWljPe9R65iW/jV99pgqCEo9Y11YwGRxuh4ItkjTt+aen177D+CgJIHQ4n61fUf1QykBfJcaBV7tWDPANyK2ddFWr9Rz9JclT99e18uNoXlvAuFrAZCnPy0EvWXUGJzMEObnf6Wymxjh6UW6pxbx7rcFvq3uOAF8d7oH+HrGaz0Vq8fBHhHzy73gG9nvHvBtUEt6VlzA5LyH1jKLtE7eMP9bp581YPXM/WpXPec2pFSgeqy0bR2vBbxc+97zxK2QGuWSLf3u5XqlmIgvS1R9gPv1zvtaIdw679zqmqWxb/p5dpza+V9JXthJK59xh51PQ0qFaEbwavr1ut7eK6T3nvfdG757g5dqZ3W+DfCl5n69Md5rTlZga8a2uHyFPI8kBMCB6nW/VvjmKuiUQbOlmrX9nhW+lEa5ZM0YLf1Z+uz5dzUAvt56S91aX/c/cq63w/wvdvwklX7m9v5a535b4JuroFNKjZ4ztIzZK+UstY2Ar2Xc1rRpD5fs7T96/jhyWkHz3jXCl5LV/WqdrzYF3JJStqSZ6/vDrsl2FfCQVdFWF2xxv9pDN7C2mvIoJYBPo6OmmzV9eeAb2Ua76CpyzlKK7QnfmcFLxXeAbwtYLbGY06X6kcpHwJm9V3z+VxK199frfrn424cztD6M4Rb2+TSku9LIdLNlvJ4pZ8t9aNq0wtc6nja21SVb40emm7nxJoQvVu8Fryb9zF17pE1Va8ZmIYwvsnopPHbw0hUFwxj3G/mEI6vLxpQAPrTS9ca0i4DvrPO+0a63N3i5tjuknSUXzA1tLdc44x5zwJr5XhHCOsdXy/LwhcuQ/njtvK+8CjrukJEE8CF1ZPBq+hs132ttEwHwkfO+EdvBLH1p4nqBF4sLPOVK64IpSG/LOahqU8C908yeexO2H11e37pgy+Kr5zay+5VgbV8FHX+6VwL4cJpxdbO275YPZ6l9b/hG9GGBb4tLPhp4ufY7wVeq5xyiJYaqk5wv1gcVaymTJEGZSC9zq58laQ/nsKSetfPAUr+tSgAfRkd2va1AmGFr0lEWXUXBd0bwUvEd4Kt1txqX3DsFrZXm/rRjMOnn14S5Xqyccr/efb9c6lm/CEsP3vrLQS7COpVmdL0j0s1S+6h0ptSmF3xboOr5rxvpekd/qdK+B53hi7XZOwVtSTVbvgxYwEw8fEF69q8WctLTizSPJ9Rca+/LOhdN95OaWGd2vS3g5dr3dr3WfkbCt3WB2IzgpdoEud66K9HtCT/rMq1bjkhBv0RiW+FscdzC6udt+Tb9TG0T4tyvZ95XA99R4H3uL3VSWf5qjzTXq70HbZue8KXU+sXK0ucR0s1c253ga3GP2vrW/jz3tI2L6k+Rfq6lcb+adLJFLY8njLoHud/UpPJ+SM/sejX9zDrf6+mnh6PV9sn1q2lridkLvADd4atJM2vipdR0HYfVczEWp2rtT8wKVCB9mgMm0tLI4qtbd3q78Mrifq0PVKCcby/wPvefmlAj4Hsm15vw1fcrtbPGjPp7CXS9dXceF9ySgpbSxFoAasDpTWfXZeh4G/erWP3MLb56ijGcSuWZ97WknXXnTbc669Rk8sD3zK5Xat97sRXXV/SeXE+/1ri9HS/XfoDrrbtscb6aWCyeKueAisGU6lsDbK+TvrnW7/0FuH3wwuW17H6pWKneA1/P/mGvEsBT6UjwnRW8XDuuzR7wbXHJVkjv7Xi5tpbfJSjlXF974Outt7rdWl6Ycn1Z7uklkO5XWnx13Y3N/VoXXnHwtbreXg9lSABPoZlSziNc7yxzilJdLzdL9R2519cSv9fcveW9CXS99bUFpJo4rcO21GmcrhamFjhz9xDofuuVz/W+4K00qWcrfKPAu8bnPuDTa1b49nS9Uvs94Wv5++jx3877ZWGmdDMVPxi+UqymP+0YotNkrrE2lDzpZW3MozTuV3uq1U3fwlORtCueo+Gbc8CHV++081ng22N1dG/49lhIFeWS91hgxbXrAN6628gUdE8IS/+NeqSgpb6urunFV5L7xeRxv1r4ck9J8oA3OhWdAN5VM8B3dvBK7T0f9r3TpSNje8DXC16urfXLyyD4asqpWGkMrp5LQdf1VNo4MgVtdd8vH57g+9rLVy73a32EoHRIRit8W58L7FECeDclfMeDl6uz9jcDqK3wne2L0ADXW197X2vdcA1N7rVUV/cttbe00/bFud9KGvcrPfFIcyY0DnD9nG/L1idKt18Ocg74ZNL+VY1a5dyzfY+2vV2vNX4kfO8EvFjXo+Bbg9QyDlanAaPkbDXtNOMFul9MlscRtsBX63rzecCnl9X9RsK3t+ud7cPe2+c9wHcG8AJ0gW8LiLX1GkCvr6Vy6p6tDpXrx+x26zb0sZOt7pdzpdp9wRb4zvAsYIAE8AEU7Xxb+2n5J5Pwtd1Ly4EcVHttPwdyvXX3rS6Yq8dAKsVqyjEnTckLZqovSzvieEkAm/ul5nU1W4Z0q5Nj4OsFr/ZfdgJ4uCygPIvzbfmwbxk3ar6Xa7O386Vie3yZmdz11tfRLtjqbuv6+l64txODstaxcu2ofiRnXq183j7z1+N+r4ePm/e1zvfm4wjvSrPCd8a0cUvbURAZuThL22+C9+Za40StEObqLUDWALO+d6y9Fs5cO/H6ee4XwLbv9/KaBqwmfdwC39ZHEebTkA6vM8J3Rsc8m+u1xrfM+Ub+7t52ncGLDRHlerevW4G8LZOcM9a2h5P1tjPM/UrP+6X2/Nb1dVscuDjora434klIdZ95EtbpNTN87831RsUfAb4Tgbcua4FvC5AxWHJtJAcMVX19rYEs1sbV7hEkiPtd4fuyWgm9PXKyJfUcDd8I8OZBHIdUtPvtDd8ebXumuaPv5+zwPZnrra81r7fXWvh6HTD1WqqTwKxpY4Uz2ub6zOeI1HNv+Fpd70joXo+bmkizw3d0O6ntyPsZPT/cAt/oL0EHAW997X3d6oCxsggH3ORkkXHEsfCFVwCX1PPLzUKsp/IXr9jjIr0PYvDC1wve1nOgMwU9jaK2B2nV0/l62iV842I1GuH4uTYHhy/X/7YMA3PLeFp3zMVIYMYkjVVtO8IeK7iWY+c96/fzao6i7A/fkedAX+4hNYki3O9ZnK8XvFy/e6ape8RGOd8JXS82jBZeXGwP51v/pNpJ5dK1xrVyfXjc7wpfwf1KTzvCDtywH0WpHyMSvB7oWtokgLtK637vCb735Hqt8Vhsr7RzFKwHgle6toK4Bcga+FLu0upePVBtgfVVDL7war1+ek3s+W2Z942Gb9QZ0JFOOAG8u44I35nAK/V9lG1J0fDt/aVjMHjrsijXS732QJgro0BN1Wmu63G4NhZYr+6XOe/5xdYBC6uePfO+rfCNOwmr3wIsgARwR42a+z06fEeDl2vXG1qtsa3wjfoCcWDXu329lwP21PV0wFf98Ht+rauet9D0wtd2KIf+MA5L2XW9fDxlLsI6hCLcr7ftCAfoHadXn7O5Xio+Gr4nBG99PaMDll5b6rbX0bC++R0u7pc7bnJ74AYH2bX8qe0g+FrBu8eDGAASwJNrpoMyPP9UEr72+ISv+ZpzxNr4CAdsuTfp95F+JyuIqTGv2mwWXj2K2vMLgM/7AvBbjq6HHw/fqLOg63FqpQPeVb336krtZ04731PK2Ro/I3w7gxcr04KKi/W4XazMCmGqHeeGLe7YDFVFPy8BLHt+63nfp7hq0dWl62tXbHW+ONjtc715FvTdaMTc75lAJNVxfUptR71PR4HvSV0vV6d1qBEOmCvD+rPUuaAK16JgrThu8vkaXxRFLZaithtty6LhawFvyznQEQu0EsC7qMX93hN8vb+rt21vp2+J9UL1AK63F4j3dsB1mfTaUre99sKaBfbtcZPcWc9ah7uWX35yW5H4uWEshhqnfo1f51GUJ1SE++3hoD1/zb3hO5Pr7Q1eKt7y3tTtD+B6sSF6uFyubk8HrC2n6iwOV9OmHgcAsAM3uLOeuXlfDJ4R8G3bB0yvkK77ofqw1gPkHPDEannLjwQXLn6WRVbediPhq23bCt87c73b1xEO2ApiCpQRoFW3ked9uf2+1IpnDKyt8PU8H/j5euxJWBYlgIdKervP4uyoeO/v19J2RIqdatP63kTDdwfwYmU9XHAUcLVlGuDW9VjMaBd8c41vOeL2+0rpZaocg69mvtd2AAfteFsO5MD645UOeLBGHbwRMe4Mzk4zbmvb6Dlxzzia2JZ/OwnfrvD11FkcsFRX9yO1V7e5Pu0K4Bq06zW23xcAX/EslWM/a/kXZPUBb889wAAJ4IHqBRlrmyPBd5Z2o7MCJ3C+kddWKHuA3OKApf5ncMBXZbenXT2nmh/QRVfPQ8grnqlyW2ral3JuOYyjbl/LshVJ+78rATyFjgQMa9+R6fHR7Xq/973hS93LwEcGSjFncMCUg9W8puq8oKVinsqutxxh8F0lzftqVkK3wNfrem0HcfTZhpSLsIaqZUVv9JgJ35h2vTMIkfDdwfVaQWu9bgFxtAOWYinIamK0YMbirTEIfFdJh21wK54t8NUutuK3IdnAq4EuBdxchHV6zXYgBBXfCzBSPzO1S/ii3fe8jn7tdcCan1SZBGWsfQucWWeMH7ahge86v2uFLwdXTcq5F3itW5BsME4HPEi93G/vvxorNLSxkYd0eAHqbdvbKbd+sZHg2xm82BBRcPXWeRxwDwhHu2APnFlAX5/zvD1so4bvqtHw5ff/0qCmYuv4uk0dqynX1ucc8CG05wrm1r4t/Y6E78h2rfC1jKdxvrUmhi/Xd0/4Yv1GOGBuDO6epN+bi1e34Q/bAADypCuAa1heutTvAb68bn8kIRZ7W0873lmfCZwAnlJRfy0RQNXGzuB8Z3S9VLm2D+37X8ftuNCqhwuOeK0BLVfXwwFTdRKo1a7YftgGt7KZ2+srzfl64RvxVKQ6DruWyuv+OeUirCHqkQb1jGf5a9wbvrOA19uuF3wj44LgK4HWeu2NjXTAvSC8vqbgzMFaE1vXm4BNH7YhwZdatWxNO9tWQceBV14FTUG47XGEOQd8WFk/4C3xCV+5XZRTbn1fNHFYzCD4WuHcCtvtdZTrxcq8UKbiuPvzulx1m+fDNqSTrrC9vq3wxQDrfRyhNB9MxWqusb7qPjXa9ptzwN012v1aZF2BrInV3nNvsO3RrpfrtcRK8D3IKucernf7ugeE19eacovT5eq8cDbAd7viOdr5ahdbtYI38jGEe2xFSgBPpQh4RS/2oeIitipZ47k2PdqNdr1Y7I4p50iXy8Va46LqPU5Y44Kp19iYFhijoMXK8GMmsWf7RsK35WlImm1Iaz/bmNt63aKsug+qjbauVs4Bd9XILTAjF1KNgu9IiHLtItuMeu93gm/LtQfE3tfWsigYe143u1ysnX67kXTEpLTVyLLYCk9H6xZjcTHc6227bdvrehuE6z5pJYBPLMtf255/xQlfXWzCtxt8sbG5f0peByy95hwxV68tewmg3W6E7fWtYUo5zFtQ43VyOlq7Ejr2TOg6jiur+9FoHSvngHfTCPfbIzba/UakqLk2Pdp52hwYvhykPNd7gngv57v+1Dhdiwvm6g3MUpMAACAASURBVNG42+1G0gMWJPhiKWQKol74jjgPuo7Dr2MewmBVAtis3guoJLW637PDd5RTHrUqHYsZAF9rfStsqbojQFh6TdVpXLEKyPIZz9ijBTXw9aSdW07Fouq3P29f38Kai9+22SpyEVbYHHAp5YsA4EcA4B2P8f/Tsiz/hfmO7kIzud/Z4XvPrlcbOwi+kde9nS71OhrC2nItiDWwxa7VMfTTjVrgSzlYySljbdb6y69gT0djsdt4Kvb52r8Iq9dKaI2degsAPrgsy2dKKa8DwI+WUv7Wsiz/d5c7mlozul/tPZ0ZvmdzvVRcI3wlV2u9jnS61Os9IOz5aYWvxuVKMFY8WpCC76oe8OVWOUedBd3rHGh+FbR+PjjMAS/LsgDAZx4vX3/8o+v9rhQJjh77eL0aBd9oiEaPtSd8B7jeumwkiC2Qldp5wdwLvsDUUfVYTA3sRvjWoJXdMA1Z76Eccp2clqbint9GzhFjAI6ZCw7dhlRKeQEAHwOA3wIA37Usy48jMR8GgA9frr5MeZupNo10pj37PxN8oxdlnTDlvL2m/jqOAt9tn1I5d611vVtVe30BAOpTrgBk+D7FbWC6jXu+boevfvWzdTGW9yhK/Vww1UeLVABeluUVAHxVKeVdAPA/l1K+clmWn6li3gSANwEASnn3CR3ynqcrRTpaqj/LwRJSTAR8jwZeKr4ldjL4RoO41e1i9Xs5YK0jphytKeb6oI16ry92vjPlcuWyW+fLwVWa65UXYN3GrOXXP+d+HnCXgziWZfmVUsoPA8CHAOBnhPATadTc76iFV5oY7xeBFofMxUvtotvsfQBK8Hzv0UDc4oAtdVIbKX5H+FIPV6AWXGlgqpnvleaHL7eqB69tLljniLexdTweG7P4ah2zwBdU8eInainlywHg7Uf4fjEAfD0A/Demuzq1ot2YJjYy9eyFb48tSVT8yDYHX2iFddsLtlxs9OuIsh4/LfDVwJZtcw1fas53PeGqhiq2pcgz36txyWvdtvw2/rYe63OVdv7Xsh2JLuMXXElp6siDOH49APzFx3ng1wDgry3L8gPK/k8gr/u1ttsr9Rx1H63w3dv1Wvu6M9dbX2tee9pY4YrV94axFcQt8H0qv4XverZzDV89aHGYavcDa+Z5ow7iwCAqwZlq93zdby9w5Crovw8AX20a/TSSIOoBpOXDHFPkgQ5Ri65aoLM3eLk6S18tsQdyvVGA1by2Aper88LYGtMCW6yMWO3MPdXIstKZc728E8ahvMbj5Zo54Fsob1/bUtD4Iq26PdaWKsOEgTwfxrCrIuaMW/5qerr2yIVZnM4IXyzuQPDlxu3xmqvn4jwxLW3X19L7iMVLZQA38N0Kc74ANvg+t7mO3ZZR/W7L6njt/PDabq27Ltc74tvXEXPAur2/df8J4GYd1f16nWdU33s4X+7voudcLxW/03zvWVxwRJnWAUswtThf6rXVCT+V6+d8JedLOVxubrgue+7bcoa0DF4rdCPngK3bkKypaEkJYFQt8N174ZVGHvhiioZv5Pt64hXOWLezwnZ73RPCHhh7fmpATIFVE8PAl5vzlVPM+GIry0Ir7dzwtp9tHVa+7evyuu04Si1sex5D+QJexa2CTlkUCYmo8TWQlNpg7TyA5sa3vke9wUvFt8Z2hu8eYI54HQnhKBh7QKyFLdamEb60y+WBKjlkrLwuq/u9/Ho+8GqgG3Uwh6UM65tSPo7QrR6pZ4u8zsrrYj1zuh5AjziIY5Z0MxY74Nm9vcBsHYOK8QA5CsJUmygwd4Yvt+BKcqhSGplOQ9PpaWnrUn0P2z7WOiy+/sm5XCtwpXlfCqyWIyhXpQN2aWTqudc+Wk4R874e+FKKAKDU5g5cb30dFeuBsgeylnoPhFt+auta4SusduZOt2p1s1pIY22x2G3Z5de1zAnbFmVhMfxrec6Xd786GKcDnlotqWcvpD2p517umIrjxrTGn2SRFdZ1D9hysVEgbgEuVtYLwtY6C2yxstX1ApDw5U63srhZ7aIsy57h27Fo8GJAtUJ35Epo/yrodMBGHWXhlbeviL49/1wSvk2SoKmN7V3XCl9sjL0csKWuLudgXJdtnS8ACd9VUtp5Kw6qT/0xK6Iv9T74tmxPWsuuf/pWQ2/bcPF1m/p9lISBOx2wSdGLoaxjaf8aNEDQxESknkdsSZoJvFR8J/ha3Gp9HRXb87W2vgW0UozXDXMuWOuEt/Bl0s5W5yutdLY4ZMu88GXM2O1Ja911OT3/q4GtdCBHHa8px5T7gNXyrgqW2re+tVFfCnqlnq19pOtVqwdALbF7OOBoCK+vPS7YA2QOtliZEr7cuc6eeVzLPmDPyul63Ou28opnaVFW3X4VB+Y6to7Druv2t3U8jBPAYfLA1xLf8oHvWfwUsZ3Iu+hKuheuP+v2ol7gpWI7wHcEiKMAq3kdAWEPjHtBd/uauq7LjPCV4EhBFUADbQq0Nkg/t5PBq0lP4z/1Z0NrD+LAAIuB1bcKOgGsUK8tR1aAeORZIBWxqKpXahqLk+JPssIZ6zrCvVrqtH1Gwnc0hDEoSrEcdKnXXJkRvtK2IgmqFjdribv8apq0NJeGjlmUhcXcvn5Ay7dtqXpt3VYJYFGt8N3zxCtPTK95X0k94Huifb1Y1xHQ9NZp2mjKI4HL1UltPIBuAXEwfD3p5MjUtG5fMe/A6/aXt+YW9GubtX6tu/5pe0rStp47kMOaggYAePmKhnECuKsiUs97K+Kv3gpw671Y3reR8A2UBFht7BHhq+kfq6PeI+09RsEXqvJG+FLywBfvIzo1LT+iUOeEr/tay65/6t0wBl0JuOj+YAawLx6EOeAlAcxoZOrZEjvS/Upjt7rjFud7hynnumwkbLVx3tdWSEc4YC90ubrOzhfAttrZ4ny51LFvNTTvrJ+v9eDFoFq7Vg641lXQNWAxqL540O3nfVHxu+ia3SOAR6eeo7cdSTERq56j4Gq5B6mvXk5254VW0rU3djSIPfVRdRbY1tcW6G7LpL6N8PWudo6MsY5/iZcd83WcvCDr8jbiQN5em1ZBb2C7BS0G2Bqmz33g5ZiKzgDfG4BbU8E9U88el+rptxWe0j8ZDeQi4HtC11tfRzvdPVxvSxkVI7XRwJark8q41zWgAVTw9a52HhEDQK+OvtTZU9PPfV7Hb8vX+LWsjtn+3MZqYFtDtoZr4aZ/NYdjJYBr9drvK7XTxHpdcoQTbe3TWm9xoXfqeqXrVsB6+tvbAff4GQ3iGr7MgxU0q53fAZ9nIPcK3gFv3bR5Ca/gDfj8pjwC0PqFXJe3g4f0tv02vi6XoFsDV4KtCFkMrJzr5UCcKeitWuHraaeFYetpUZa+JbiOrsdiuHLLe90aO2iVs3TdE8RU3BkhbK3zvH76077aeYUvBrU34K2bNu+At1DobfvZguwN+DwB1pgFWnw5nX7moKsB7tbZrsC9Am0NzRqwGFR1x0Ff95UOeFUEfKNWMXvf7oiFV0eBb6859hO5Xq4uEsqt8PVCuBXK3pgWEG/hq3iwAvcsXws0+6Wl7XuHKfDWZXV/6/Xl7SRccgXdGrhbd4sC9xVSRl1TrlcLYoAEsF4tb0EPKGju5wx/ba1bjA6ecra2p+q0wG1p73G72HhaByy11wBbM04kfG/GtW01Arhd9ftc/gylOv42rl453DflrJkTrsu2/UWB9wm627eEgi4VU9dpyqnYBDBAjHPde89vxIlXs7nf1rRzK3w7ghfrfhbna+2Lio92w1EOOLJOA9yb1/KcL0D7VqOtEwaAx+t459vijp/LcRi/gFdQu+G1TgNd1uVqYMulpSnYGlZBJ4C7p54tb11P9+vZ82vpr1YP+J5kvpdzYJprD2y1cdGvR0O45ae2TgPlBvh6tho9L6i6hW9k2vmNxwVd2oVeXL/Y77iWYeDdul0VdCngSrDlQIvBtcUN3/cirL3g29v9ev66rIDl2h8FvpY+JzhYQ7rW9DMSypHA5epaYFtfc1DlylSv9audL0308N22aYHvukgLAFu01edgj3WsLXhv3DHidl9c3koAMECXc78aF4xd1/G1OBDfrwOeDb5eQMyYeo7smyprBWqmnE39asq1cMXqLaDVxGjdrKZOUy/G4/ClpHWgeJpZB8p3bLYhbVdMx6WptWlpHXhX6AKA7HQtwI2aA5YcL1Z/nw54phXPmrEscaP/qjxfCCglfNHrvUHsfR1Rpo3hHK32Z4jLrV4LW40AgF3xXF9jaVkrBOutSNpU9xvVfmNsFTY1z7y2qfslfycEvKjbfQWyy7W4YA60rU4Ya3t/Djg6/YvJ8sFPKerEq97ul1OLc6bGOhh8sa6tMPbU7f26B3yl9tg1VhfhfKXXV9cbelSywHe7FWcLtrVs+1rqY9vGts2oTidf168x8sEb1673CtKPc7zb+d0bx6txu9s6qn5bvi3jXtdtqBhN/X0BWAuPFvdrfat6ut/Rc82RcI6Grzb139H1YmVaN2uJnRm4WP1eDri7C35OOwOAaa+vZ7UzAGxcqexkrc43avU0l24mU81bWFIgXn+2zAPXryPngrF29wPgEfCNahflfqU2rQ7V2/cM8O3oerHuI697g5jqs6cD9jjhlp/R8L26vk47A/SDrxWEEkytMF+vb+emr+eUAa5XN9fpZtLx1uClQMxBl1vpzDlkqh6Ieq6M0vkB3HqQg6Uvqn2P/ahSjPVeW+Ac6YwTvs2A3V5HwFTz+ggQ1tZZQczA97Wred54+N4unrpts24R0sI3ymlz6WZunpcEL5Z2plywdh4Yq6Pq69d13FZaCJ97EdYM8LXEe6GiGTtycZTld7XAOfr333m+FxvKC1sudtTrnkDuBWFrHVWmKScWXAFAJ+dbQw2HLwdvALhZTFXHS/Xciu31HgFuF1mR4K1BW19TLpgq06aksbq6vq7DrqXyrc7rgCPh29J+xKIvacw9U8+WcT3xB4WvF7ZcXTRkqfII4GJlURD2xjS9Hg9faTWzBF/pUA9pDzBfL6ebb1Y2b1c1U+ClFl9h0PXMAXtPwfI64fM54NYTnqx9euCrcXXauOi/mhY4WwDb+iWBupeDp5zr6x4g7gltC0S5Om2bKDC3whd5sAKADF8A+5OFtoCLhK8uRa378nC5h+pJTZLrrV8DU+ZJR9flgMRzr7kFWVQZ1narczngHm5zDwdLacS2I04tgLV8UYn853YS+HLjWl9H9ucpo+q4WM0YUr91mQbSYvktfFdt9/pSeoF8Mr/clHHAe46Rtithe3Jv66i+rNfXzvd2rvcdbwnzvBhUI+eBe5yG5XXA5wHwaOcr9bHHwqtWRcKZkzWjEJkdOCh8R7jWiD68btgb63XA2r6MzhfgdsHVU5lrr6+8+tgKSGpOt80Z61POrOv1LMDSzAN7FmVR9XUMdr0V9x3s+AD2ONSIXycKvlrN7H6lfl8ydVw/WLxmPKqvSeE7GsSRsdp6D4QjfnrKuNdP1zr4tj7ZyFtPPRO4Fb7U/DB2L+946/O3e3o/B/SWIu08cN1mew1CmWbhlXcFtOR2sfrjzgF7U8PaX2VU6nkv99sC5yhnLPWjcbSTL7aqr71gjgJxbzjPBGELaKV6Ab7UnC9A22MF/fUyfFeY1vCl5o8pqK/zvU99eV2vZgGWNA/csgraOvdrnQfG+jueA24BYxR8e7pfLWSkmMh5Va4tB26uLmJl9uTwtcCWa9sC397AlepHQdgacwD4Pm/fsW0P4oFKp5WpPcUm17yB781c71ugn+eVwCuloS1zwBhsKdBq5n41EF51DABHuNEZ4Rvl4jUnSXH1Un+963rBd9KUs3RtBbGmfLTbxcp6Qdha1xm+q8bs9eXT0tHwJV3zq8fDPj73dpvrresAqQfiWjsHrElD168tC7LqeE5zAzgqDTwCvla1nmFskeW+W7Ydaeui0tva8RplhWlLf9q+qDisvJfz1ZRp/om0/pTqsPvUwvcpXr/auV5wRelF9clNAfq57rr+cpu30Nz2tb7exq5j0yujOcg/IOM+PD2n9/HtiYEvNefbejAH54Bb54GxGEoPMNMccK8511Hw7bHwStOf9X2LPHQjok4aX/OlBIsZlHbu5Xy5uh7O1VKPxba64b0csOq1f7UzAL8dKGLLjzxXK6eOsaMl6362J1ttnS8731svumoFb+14tfPA2lXQEozrGOyaKsO0rwMu0Hex0yjjHpF69oImoo2n7Qj3q/mycVD4jgBxD/h6QcvVzQ7fp/uPg699xbPlmk8tS/DF0s7YSmcSvutcrwTYtx7fVwnMFHip+V4OxBHzwFR8LQ2E505Be2W93R7zvlHxLX1JAJvJ/SZ8TXWj4DuzAx4FX+Z4yVVb+D6VGeBbp469cK6BW/eznRNeH9Jgge+TK6YWW9Xg/BxRrnHE2pXQEogx6HLApcBrnQeu22CaJwUdpVnga7mPXu7XMqZl7663Ttrzq6njYgLgi3XbC7aW2FHO1VPfWtcTvm4Q4081AriGb33E5KU5BVYavtyiKmq7EbYtSNpOxKWvw+C7dcAUYNfUNFR1WLmUmt5eg1C+lgFSbnXB2n3AnBM+lwOOcnSe/jTtZnK/o6V1yd4vIyeCLxWnKef6sbTxOlpr/CjHi5UZ4btNOz9f8/CtF2FJK54BtIuh8GstYLm08/Y1Bd83PveF6/neNZ1cwxfbeqRJN2vAS7ldbsUzBmmqvq6ryzGwco4Xiz+HA97j9nqu0Pb03XI/PeZ+o7IDmt8rcLvRVi3wtfTtebulsSQgauM0X0qwuh7OVxoHuz/LlxQEvqs4+D43vwbpU9vKDT+X4+74+le57u9FNQbW9nYO2l631l/Bf+N8ARDnC4DDMwK+1i1IGvC2zgOnAwbw3Vqr+/XARQtIz9GLkedgc2N5XbXX/Q7c69sK22i3a30d7WhbynrAd4QDRuC7zvu+ePn8Kbo9aAPgeksPVkalli9D49uCuNSzdtEVtopZV3e92nl7utWN86VWOnPOdxsPQKesAenDuyBLgm6UA+Zgi7nj4wLYe0s9U89Wacbx3IsFlD2cfIT7PQl8I+oSvreKhu9TvzR8pVOuAPBFVJdheqWeuUM7dHVu+GKQxVZAf+7xvcXiPe4YiDLOFXNlgJRvy7jXGFQ5CNcxx0tBt9xKBHyj3C8WG3HkpKQe7pfrs2XuF5j6hK+7/ojw5Rywpkx8vaDwfepKccQkAP1owFXauVztvC+3L3gL0eex6ZXSbvhSK525cit4gagHJgaQOAAaupj7xcBMxWwlrX42amcARww/E3y16u1+e8i77Yjro9M/v3uGbzS494Kv1+0q4as533kVBV/t4RtYH5cyDXCv221XQFNbmqg9wutxkyh8a4eLQRaLAei7IIuCMQbXiFXQyvnfRXLDczvgqGFng+8o92vZW9vb/XJxHJixdh1WPB8NvlIfUllL2zPC91GW852fyhCQrore76t1yZQrpsB8DePrBVcq+NaQpVLU27S0dzEWIPXbMoBbEFNlGhdMpKVruD7UUH7U2wyEvzDHHHCv7qPcX6SL1P6uEe6XUw9nzD3tKGrshO/VawucNW1bQcvVeX9SdRb4Alauf7iC5qCN63J+v+/lVm5XMtvT0rb0NQbim/lgbJ+vBF/NIizqhCwqdQ1MHFbPXQPyU5uShmvYbkGLwfWBAW4d/4V9HXDp13XToRQWWd2vN7YVmFFwtszbamRxv532+lrqLfHauj1ccJRrpupGQFgqq+uvyq+PmNxqu+L5cv3KvN1IuyWpTj1jsdsxsTrMQdf3WW83em537ZAB4Op4yavHCW4hCJvXlGOlnKvG+daulhoDmPjtNcDtfddl22t4hi4H3Bq2lNul3LHSAPd2wNGKhG/UftZoRT5SUEuKiCcVafsYBF8LkHs4YQmsmngtnK1uWBungaclXut2sX5VDljeblTP+wLotxvVwgCIbUnCYRmReqaOqqzPd36Ad7z1+auznUmXW28dqh3sKyIOK9fOCUtgBriFKwZYAroccLegvYFwBde3QdYaozTARwLwKPh62mHjefb99lTE7xwx9ztAUUC1xHohq+nH6mCtfXjgq/l9te01YObaXJXL8F1l2W50GYpOPWOuVZNSvrRp35pErXh+nvfdPFhhC1Jq7vatzU8AGb7c3G+dugakvQW8igVYNXRr4G5hewVmuBYGXSYTbdZBADwq7expr43XxFndrxaC3jrPgRyWcSeY942ItYLY0o+1zFvHxffsRypT1+u2GwHcPt1Igq98zjPubum0MQ5PyjFvX79RHSeJpZvred83Pvf29VONapBqXDAGX8rtYtDWOGIg6qAqA6QNXKArAZeC7fZ1DVjK+XIgPokDjtwLq+mTaz/b3K+lv4iFU545Yq7NBPD1wlgLXwm6s8BXWwdIXB1rccCa+5DaAAC24nmVZtEVwG06+bmchi8d87Dp8xaQmpQyB2ntwq3nRVeI86VAyK12luDLPaxhWwdAg5lbkAVVDFw7XQm6GHAfkLK6HKvfCos9AYCPAF+sjfZwDqnd2dzvVpPBtyWWksUhW+rrMk2dBPKWdhI8rf1KDvimXF7xvIpbdMWtgq4XXNVx122eHe+lXD6j2TPve/vEJATy9XYjKsXMAVMDX8xBAxLLwZZKNTPg3bpdCbo1cCnYeiFc6+CLsHpvpTmboue0I/qPXlXd0J3F+VrGsrpiC8hb4cn1KZVJfUs/sb6tcLfe60sckgC6M54v3V9D9KoPBpScG962rftaX2NjUwCn2nDzvmULQCR1S8IQc8sYTKEqr6+18KXuB56vKcdrBa8Gut7537fhsA44YjVudPvoQzewuFHHTnr7pOJ2cr+1IgGrdcY9X3tdsBd4mrrWn9a6ugx9jR+2IS262sqTeqaAWgPS4n7reWhN6plMSVepZxSqq2vd/sTiuMcQWh/WULtjLq76osA5Xg66rc7XM/8LcDgH3PtYxxHzvp7+tW16PiIRU+t4A92v5ToqttWBa/tsdbpR0NY6XW18XYfFU/eHwbdKPQMAmnrGjpC8hSQN2lo8YLUPUKAXb1GLq9gY7KQrzNFyZdRWJC98LQ9rgOtrDLyY29VAFwOrNv3MpZ6xugM44FFw65F6nsn9Riy+0n4JodyvVo3utwW+1r6pOo8rtsS2uERrDHZvmlisXANhz++Glt/Cd1XEs33xgzP4hyfUfdcLseoxqPljzZwwt98XPWyD+sPt2aXqNfDFjq/UwNcA3hq68HhNQdcD4d5bkACGA3iPox+9fVhSz57+tW2itmB53K8nfU0Be2f49nC7FqBa+vC6YatjtjhXyf1q+8FiOVjflOMnXWmf7buV7mQrft4XBy73IAbu4A5r6pnf73sFRmoRFVVXQ7FOWT8A/jCGliclAVzBVwNei/PlgHuybUgF+i568sDE0of13nsuZurRnydF3Op+d1QUfK3A1bpcbCypzANWbXttf5rfwxKL1aPtVsLw+30Bbh+ygEHtKfYGiLdzxlthgL3c6isUktt2FGTX/p7jArYcaRdY1S4XA3I9h1xDtu7b8KQkK3gxsGoh7FkFLa2A3sYfIAXtVQR8e4+tje259ajjPCw5bicwt7rd6DptWlbbVgtnL3SlGIuLjYKwywG37fel5nQpID8Pz7nhGqz1eDRM1/J6DGqh17af+ktC87yvpo5LKXNPUKqBDNf9UenmFbwAz8DlwKuFbl2+LavL6zoqptbBFmFpFXW7ke5XqyO4Rk/6mdL2PQ5c+Sz9ExgBY02cFrTe/rd1FuhqxpLaSIDF4rwO+On1der5tat53uuPyK0DvnTDO91a3HzwNgbrn5oX5l5fQ5sCM/7Epcvb8djPA/KQhfon5Xg1dVwf29fA9A3VawB0SxHneqm5Xyt4W5wvl34GOCWA93S+3PhRB29YdSfpZwtwRzjjltfavr0OVxMbkYKuy7l+re6eff2cegYAVep5KyqlzLna53bY3O6tw63jsXHrOO7Eq9vFWrz7JeFJpYIpmGJ11FYl7VhVOZZy1rjeGrbYa20qelvGldd1terYk6WgI29zVvd7pvRzkPttAaqlL22dNQ0tvda6QQ9srRCPSD17xqnL0Nf4fl8A32lXVEp5K2nVM+VKNc7YcuIVNw564IbWzVpSz9KCKmkRF5K65uDbCl4JuhYI13VUTK2TOGDr7R3F/bYq4mxnrp3nC8NgjXa7lnux9ql1qNbUsKYNN2Y0jLF40QHrnu9rPe0KEwVbPPahase7X6x8vZ/t/W7r+BXQxIEblJuVAFvP4WKglRZiOeD72c/xrpeCcA1R76KsbRlX3ksTA7gHfPdwv5qxWtyvd1zKvdbSxHVw3L3crhSrifM6ZEs9VacBcg+n622P9SHd0/p6+3zfR2lXPW+FzedqQXuJxVPU3JwuNl/r2XZ0fb/XK6rRhVcA/J5b7o8mhS3t7RXg+/bjIqzPfs7mejHHq90HrNkDPAK2mCYF8EzwbXW/vcEeQZetImHqTD9b3a3lLeDA4AWrJ56rj3CtUW3qtpoY7ZcFqv6q/PEjEjlwYyvPgRu18FQz/zQkzP3W/VErmuttR2sdte3o6h65s545KFKQtYAag2u9P7gac1md7oOcctakoKX0NBDXox2upAkBPOEtTaXeD144uCJSzxFjcbER7lc7ngeyLS65BcJX94LPonHuF0A+cONqCMTVWoRBddvv+horr1PRVIqactwvHvftPP36mPuF6vWrqryOrfvZxtVldR/1WEh9BHw1h3BY5n731kS0897KLO5XK2v6uaVvSpHOODj93NP9euqiXnPjaAEmxbS4YCm+pR9LClrpfrmFVwDynl9M2hOv+JiHq/625VvYUwdqrPGXt+M2VX15O57dLwDI7reOkeaL1z9bJ0sdS4k54nWv72bO97Nv2eFbQ1i7EItLSc+mCQDccgszLRbqlX723r+2XS9n3OGpIel29gAAFONJREFUR7V6wNgCUK0kF6gZL9LRemJb+9CmoAHActzkVprTrZ6Hu00rU5LcMeZSMZhu72lbh9/XA12+PfEKAyKAnI7GoKuJwR7UwKS41wVXEnw/+3jbGISlvcAa9zurdgTwBOwf7n5b5V2c5TkqU7I+HRTpdj3A5RThfq1lkWljT5ueKWjq9VPsA2iOm8ROvAKgockdrmHZdiTPBVPzt5TLvYau5H4BqrlfgGsIAshOdxujATMG4hq+G9e8wpdbcPVZwCG6hS9XD8zrI2gHCkYM2TPV6pHW/fZc/dzzi4F19fMO7jcitsX99gBxXWcBs7YPawraC2HJAT+V8cdNbkWdeIWtRqYXY92ukNaoXji1HZ8qx8Bc3+v6egtqyv2anS0QMRo3LI1FON+3Gee7ha/W9VLp5aOBd9UAQs3gdDHtPfe7l7a/2yz3BMd3v56xrA5Sau9xw9pxLW2pdtrUNAC0nHgFAGQZB1qL+73c8u2iqnqcOt28ff0Sheyrq7ZXC7pq90stgOKgu42VQKyFb/WH2ue7QhdAhi8FXA7CR1QnOpZ+XU/nfveSljKR6Wcqfqud3S8Hjkgwex2v95+lFqYtaWuq3nIPaqeLtbGdeHXdnTTPe+1YMbBaVC/Oql+v19c/6ScePbe5TVev5Vcrn7ffLziHS0G1XqSFtQEgIUv1WZ9wVTvfbbca+GpTz0fVa3vfgE1Rjs37mEJtfMTiMCk9bR3PqwHzwi3uN2pMTVzkfWjhpe1HC1HrPVHlmr8j75eRlzQMa/eLxiDu9zKcZjGVbu53jcf6xg/hoMH8PDY1n/zq9oELANcQBLgGJ2xioKp/qOrqfjCIA/ITAzrAzeMEsT28ALd19wZfgNPaxB6/Vu90bVT/Fnt4QPVIN1tTzJpyD5Couoi/NmtaWhuvSSdTcVdtrvf9ag/dsB45KaWVPdK42fqe6vQzdt83bnl94AIAvc1o+5pKFwNTL80XMylrasUz5Xa3aWguDoiyM+hAn8hRqedI96vtPxre0fd5oLlgS9vWOE0bayJDKusx5+tJS2tiqWuqP/L15shJ5dwvJmw7kSXNTLnf535wh7zWX34tvLwGM97/A9Judb6vbvf9wuYaSyXXIIUqhuurLpcWeQGQi64ouGrgu97CGeELYEhBl1JelFL+XinlB3reEK69AUGN3zqhJ/WvrY9sp5k/Dpz/bQFshNmPyqq3uN+odLQkq6sd5X4raVc+U4uqNAutLJJS19iRktfl13PEWDnWzxPwscVXAPSpV1gMVk+5X+oaNj+3b8mj++Xmfan5Xs1hHGeFL4BtDvgjAPDxXjdCywKRlk+rmZIBlnsZtSit0/yvZ0isbhSMLaC1ppw5tbpfj7O13AcXw76+feACwK37vap7Uad6uYVWOKS5eV2qreSQsXvh4CqV36TVKZiu1/UcMAZSyv1SKW0ulf1YVqeeuf25np9nhS+AEsCllPcCwO8FgO/uezu1op3vHvO4I9PP2k/3vTMKj2pJN1v6peqivq/1SkP3+G7TA+Ct7ldx5GT9uMFLF7j75eZ+OVFOV2qLwbTeosSlpbfbkOrxto8cvFp8haWUMZhSgKaASjloDOKPP+vUM8Ctc/XCd7sn+IzSOuA/DwDfDgBfoAJKKR8upXy0lPJRgM8E3JoVEr2cWHT6eTZNAuNaUQ7X4357mX2vI9bCcNQ/yRb32yjszGdTe2Qe13wPW0CK88o01LF6bBHXlarU781cbZ0qrl+vfTxU11vVfTA/l8fXD48/t7AF4vXZoWqRCOBSyjcCwKeXZfkYF7csy5vLsnxgWZYPALwz7Abj1HvxlVbW+d+RGjz/61VvGHvbtECn5d68c7jS2B4XbClrPPP5+brP4iv5Gb3btPV1Cnn7unbGz28F7pCpZ/4+qYYuwK27XX9iKWUsBa1xzOs4jz+3e34Bnt3v1sEC8rqOkVZAn1kaB/x1APD7SimfBIDvA4APllL+Ute7umv3GwXoyKcUdbBcPeZwveMbFwmFpZ97pKE9jlpzX9o+NGVY+hmuF18BAHvwBn5LtKtt3Wp02x8PXCzNfH0vt+X19XbvLwDgbhdzsxRMAW5Ty1g/2lT1GrKZ+926W2wRFoAewvcgEcDLsnzHsizvXZbl/QDwzQDwd5Zl+ZZ+t9TDEe7lfo+yd3hntcwFj5jv1bSXxmr9gmGdo40CuaYf1xww7X634o6d9EBV42yp/uutR5Sb3cZvf8pnRSOOmlv9DNVPzBnXLrcuoxZura9rMG9+YnO/9RA1bKX9vfcEX4DpTsI6MlBG3HuETYza/7vTHPio+d6Itj1A3KqWxVZUH1IdmUlA5jjhduvR7RB8SnpbZoG0tPqZbnftZjFI12CWti4BwPWDF1bV8Fx/Ui54206CLQX5bf3jn+22IwB8WxFUr6VDNgDuC74Axv/yy7L8MAD8cJc7cWt0+rlF1nsd+YVk4PxvD4fbMgbWxgtpT0qWqmt1ra3yThNwEL7p8/KJL6WfNYuvsO1I13U82LWqH0O4/Sk99xdzwNghHGv6+UlY+rku4+aBOUDXq6glMMOz492+rud8LRDetr8nTeSAj+J+I4Hf8vhBS78tGux0e8/39uizpf+W9LPUhzVNTfVnTTtLqo6dXMWlnwFu3aJnTpd75i96T0zK2zunzH0RuOp/m35ehQ2JwRnglmgYtLEYaqzHuu3K5+15z3VzDLzYrd9j6nnVJAD2AkPzv3+W1c/ROslToSwuloPe6PSzJ7anY49oZ/kyoK0zLr7SPPP3qp5ID3uecERBWeoLP8GKdsZ1ObcV6WXlPAEAX1RFlWPzwnU5l36uob0CdzP3yzleawr63jQBgI/ifDmNOIDjJBo5z+ltM1P6uefiKU8/XCxVZhwbO/nq6hqFlvTwBX4e13R/V4C9TTNv4+oYbBvTWne7/agamHO6NYjrcmx1NADuchkwb0+9WiUttMJWQN+z691qZwDPCqkZ5n8t7VsXYFkt0bY8cP9vj/neqFRxa59WEFOxrYAdtQiLbGtf/QzAp23rOM1Z0NRDFbi+6nlk6jCNerw6pj71qtZ2/rdQaWMszawpX0VtNZIWYcH1vt968ZQmBU3NFd+jdgRwK+T2SD/3nP+11ke3k9p3+FLSG7jasTXfM6IWjkXE76WIRVgA3dPPlzq726XmhLk5Ynwrkm4hGHV0JQDcHr4BQLtXbDEWVc7BHCurwPx01OTDdTgAn3bevqbmg+9ROwF4BHyPJsvvNGvm4ASKnFedLf2sTXRo2kmQxcoUi69mTz/X/T6XXaeit6+p+d/69drPi6slxkCvYoaGcixtLSzQoo6dpB6akCloWTsA+GzwmHn+d/vpN8E9RS9CsvQ5Kv3ckpKWYrWrlq39WfoJex+ZtK3i1CvcYcrpZ6vqOVv8yUr1lqTtNqVbMG/LsfYAcDv/C0ADEksXa8uxhVvMIqw6/Uy5XyrdXJfduwYDeCQE9krhplTqmdK19Lf3orCW8Ufee4dFWPXe363q5/5aJM3Jco8V3I55nVJud9DYnDJ3z6UmFhDXEeWUc96GIOnnbZMaqNS8MHdL96ZBudwD70sNUc8FWB5FLcDqpN7zv5r2Fkj3cL1e9XD6lhjxvcI/erHTr7bpZ2qrUaQsJ2fRz/W9hSp33vPNSVivXj3t/wUAfLvRes0truLmfzVzwNvXr/D08zaFTC2sytOueHX+SJjVTe69AEvq2/K+jVgYFnACVk9wjurT0n/0/K3Uv3UeN2IVNFWGzg/Lq5/r+V8A/iAMqmxb3mv+F0D3BQADc90WO5YSTQ7U87ZSOcBtGpoqr1dBI2Cun3r0VL5psl5j6WmAnPut1SkFXWBe+PaW5vfu9d7c2Xs+2nnu6YSj5389as4wyO635/yv72CO23ld6n6kgzuohzFsX98swNKsUpbKt3WYk66FgLlOP1OQpcx0rnzGNcFBHBZpPwHOPP97hHsU1GP+d4btR1JfVN2ssyrRq6A30s7/XrrQp58l0HLP/q1jtn3S/fFjYTHYAqyres0CLNhcc+XaxVxUynobtoEvNhy1wrkeKt3vsw4G4Nl0Ahi6FXgAx1ajU9Wtfc84/6tNL3vS0C73rjv7+apO8fCFaFlcMrW1iFowxp1j/eSc6wM4MHHlmjS0Zv63ilk219iDF7BrQOoTvLdKAJ9Wk21BotQbfD00q2PVqueqcucY2PyvJK8zbdF2xbRusZbuiwQKbm4B1iqtK9b2R/S7XflMpZupLrC9wamLTghgCTazLcA6kga/DxFwHr3lqGWcqAVYkbKmycV58gpIzPyv5ulH1KlTa/v66UdWaZ5+hD2akLrX+v5uYrYroAH4NLMEVGoBFtcfsg0J236EQXV7q9wWpNSzDgTgo0Jwti1Imr6Dx5zxry56LljrAvdcgOUZk6uzvCfE8ZPybcjQwuKsklZXSwuwuHvSrIC+qteuXN6Kmu+t64R53qfyR2Hbj1ZRJ1xB9TrTz7QOBODemjhNS8rzCWzdahT8vsy+5Wi0Sx6RDraUR2cdNE2ZhzLgQ/FbjzRttQuw1jJsLOyaArXmOMoX9f4eaZEVpofqj6U/wa7W8791KHf0ZP06dVECOFQeWLXsAY5oN5EiFv+MdNs9XW+rtAuv6nJNn6b70D39aKt6AZZV3vlf6wIs3xwwvgJ6XYB1swJ6uy93Kw60WPta2KrnepzH7Uea+V9u6pkrv3cdBMAz5jDvGJRRurcFWD2des/MQss2pACNWAGtFeeGr5/7K7tmqsy8AhqABu3azroAqz5sg4Aw52qpldGpZx0EwFr1WIA1k+4I3hF/VXsswPLKunWIuj6QIhZgkX3D7SP+XPco7Nm19qHuR7OlqBYH2rV++1MoX6prKp3MHTuZzpfXyQDcQwf+hEvJ6rUAK1p7p9Y1dWQb38ewbi5W7tt7JKVmBbS2PR9H/A6at80CWq7dA1EOzwuwuC6lc57TBeNKAKcc6nQIxyhFLP5ugfJe3+mk3ztySsBxApZuKP2pWBZx+3W5/rEtSLr+K1f88Op2CxIAf4SkBrSeum26eV0Fjfxa0vGS6YJlHQDA9+xAIz4RT562PlKauVbv+7UuxOL6sI6pDRcWZuldpN3dYmlqbdrae1+qLUiUWkBrXVHN9CdlsKlV0KlbHQDAI9QLUkchgmYLUsDv4nWFR3kbvSugo36/XlubIvpsdL5bcS5TAiMXoz2oQ3PsJAZX9UEgEY6W29+rXGlNPfVIuo2UXglgl07uKiN1FHhqdKbfRVLEFwflGdDba+0WJM1DEHoIByu/B5iKr19fqQXC3EIsqg/CIT880CdgbRdbcSdfpQumNTmAj3jCwlE06BSsaE1+e6T2OrVq1HhHngroLAnM24cwXEkDUOm7hmfrEjHfi71OtWlyAFu0tyvde3yNJrrH3innMwPhoL+PdgGWa9vOAFmdtfVhEehjCAFkyEpvj3deWanchuTXiQDcQzN/0s18bwfX3quUPWctR99DZ71wbk0aLcupV5zEuWlsBTSA7ulGVL22H0T1HmBD05RBCeBdlRDdVSHznBE3otTewB24B7iWdQ9w9CEcdMztKVgtIk/BApCdrlfEHuC3mXvJ/b4xSgBPpYlSxCN1dIj1HiMCkr2+SHR6PyP29gLQ242sfUSp9XAPVpIjZg7b4BzydjW05QzohLGsBPBQ9QJsgntoX0dKXPRaazfJQyfiQG2bd9ZvWXJkAFrmbLXDKeM8LjilVwI4lRqpUZA6wJcEz2MIe4g7BStSve7/Rq0rpJFyyx7gBLNeEwN41CfInbrH1H3K4/R3gnmvU7Ci2nJA7XoOtOWWB66cyj3Adk0M4JRPB7A+PXSnv3bKL+5QDU5SzDCnCzDF0uSErF8J4FQqQp3PTD60Bh1DaetnP3LtOfaNqIO45th6fXolgFNjNNkq29T9ioK117lKT0LqIgvDg3ifTjdeCeBUqpfOtr3qBNrjVC33mNy2odQplABOpVLxIh7EkLrVi9Z87w4Z7YmS6IdWAvgwytXaqVQqdSaVZYn/plpK+WcA8E/CO+6rXwcAv7z3TZxc+R6PUb7PY5Tv8xgd8X3+jcuyfLkU1AXAR1Qp5aPLsnxg7/s4s/I9HqN8n8co3+cxOvP7nCnoVCqVSqV2UAI4lUqlUqkdlAB+1pt738AdKN/jMcr3eYzyfR6j077POQecSqVSqdQOSgecSqVSqdQOSgCnUqlUKrWD7h7ApZQPlVL+USnlE6WUP7P3/ZxRpZTvLaV8upTyM3vfy5lVSnlfKeWHSikfL6X8bCnlI3vf0xlVSvmiUsrfLaX89OP7/F/tfU9nVSnlRSnl75VSfmDve+mhuwZwKeUFAHwXAPweAPjtAPAHSym/fd+7OqX+AgB8aO+buAM9AMC3LcvyrwDA1wLAf5z/nrvoLQD44LIsvwMAvgoAPlRK+dqd7+ms+ggAfHzvm+iluwYwAHwNAHxiWZafW5bl8wDwfQDwTTvf0+m0LMuPAMA/3/s+zq5lWX5pWZaffHz9a3D54HrPvnd1Pi0Xfebx8vXHP7maNVillPcCwO8FgO/e+1566d4B/B4A+IXN9acgP7BSJ1Ap5f0A8NUA8OP73sk59Zga/SkA+DQA/OCyLPk+x+vPA8C3A8AX9r6RXrp3ABekLL/Jpg6tUso7AeD7AeBbl2X51b3v54xaluXVsixfBQDvBYCvKaV85d73dCaVUr4RAD69LMvH9r6Xnrp3AH8KAN63uX4vAPziTveSSjWrlPI6XOD7l5dl+et738/ZtSzLrwDAD0OucYjW1wHA7yulfBIuU4MfLKX8pX1vKV73DuCfAICvKKX8plLKGwDwzQDwN3a+p1TKpVJKAYDvAYCPL8vynXvfz1lVSvnyUsq7Hl9/MQB8PQD8w33v6lxaluU7lmV577Is74fL5/LfWZblW3a+rXDdNYCXZXkAgD8FAH8bLgtW/tqyLD+7712dT6WUvwIAPwYAv62U8qlSyh/f+55Oqq8DgD8MF7fwU49/vmHvmzqhfj0A/FAp5e/D5Uv8Dy7LcsptMqm+yqMoU6lUKpXaQXftgFOpVCqV2ksJ4FQqlUqldlACOJVKpVKpHZQATqVSqVRqByWAU6lUKpXaQQngVCqVSqV2UAI4lUqlUqkd9P8DnGSSkMm/7/MAAAAASUVORK5CYII=", "text/plain": [ "" ] @@ -3676,7 +3679,7 @@ "\n", " * Random chance to mutate individuals.\n", "\n", - "5) Repeat from step 2) until an individual is fit enough or the maximum number of iterations was reached." + "5) Repeat from step 2) until an individual is fit enough or the maximum number of iterations is reached." ] }, { @@ -4156,7 +4159,7 @@ "source": [ "We pick a gene in `x` to mutate and a gene from the gene pool to replace it with.\n", "\n", - "To help initializing the population we have the helper function `init_population`\":" + "To help initializing the population we have the helper function `init_population`:" ] }, { @@ -5318,7 +5321,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FdWd7/nP93IOIIZfBw6YAGOg\nkyczHQO2nBa7iQwxpA0IRmd6umGMXs1kuJO5hiDY6Zbn6Scmz41mVCB07OncXGnw3jagaduI2lGi\nEQwYtQ+00jHpnseAiYj8OMIJ6DERuGv+qLM9e+9TVbvO3lW7dlW9X8+zn7131aq11t6Lw3evVatW\nmXNOAACgtf27tCsAAABqI2ADAJABBGwAADKAgA0AQAYQsAEAyAACNgAAGUDABgAgAwjYAABkAAEb\naDFm9kEz+0czO2Fmh83sbjNrC0k/zsz+pj9tn5n9i5n9+2bWGUDyCNhA6/l/JR2V9H5JF0r6nyX9\n334JzWy4pCclnS/pDySNlfRnku4wsxVNqS2ApiBgA61nuqQHnHO/cc4dlvS4pI8GpL1W0v8g6X9z\nzh1wzp12zj0uaYWk/2RmoyXJzJyZfah0kJltNrP/VPZ+sZm9aGa9Zvasmc0s2/cBM3vQzI6Z2YHy\nHwJmdquZPWBm/9XMTpnZy2bWVbb/z83s9f59/2Zmn4znKwKKh4ANtJ4Nkpaa2SgzmyJpobyg7edT\nkn7gnHu7avuDkkZJuqRWYWZ2kaS/lfQfJE2Q9J8lbTOzEWb27yQ9IuklSVMkfVLSSjO7vCyLKyVt\nlTRO0jZJd/fn+xFJN0r6fefcaEmXS3q1Vn0A+CNgA61np7we9UlJByV1S/p+QNqJkt6o3uicOyOp\nR1JnhPL+T0n/2Tn3vHPurHPuXkm/lRfsf19Sp3Pua865d51z+yX9F0lLy47f5Zz7R+fcWUn/TdKs\n/u1nJY2Q9Ltm1u6ce9U594sI9QHgg4ANtJD+Hu0Tkv5B0rnyAvJ4Sf9PwCE98s51V+fT1n/ssQjF\nni9pdf9weK+Z9UqaJukD/fs+ULVvjaTJZccfLnvdJ2mkmbU5516RtFLSrZKOmtlWM/tAhPoA8EHA\nBlpLh7xgebdz7rfOuTclbZK0KCD9k5IWmtm5Vdv/V0mnJb3Q/75P3hB5yXllr1+T9HXn3Liyxyjn\n3Jb+fQeq9o12zgXVp4Jz7rvOuY/LC/xOwT88ANRAwAZaiHOuR9IBSV8wszYzGyfp38s7h+znv8kb\nNv9e/+Vg7f3nl/9K0h3OuV/3p3tR0v9uZsPM7NPyZp6X/BdJ/5eZzTHPuWZ2Rf+EtRckneyfPHZO\n//EXmNnv1/osZvYRM7vMzEZI+o2kd+QNkwOoAwEbaD3/i6RPyxvOfkXSGUk3+SV0zv1W0gJ5PeHn\n5QXFxyV9U9JXy5J+SdISSb2SrlHZOXHnXLe889h3SzrRX+b1/fvO9h93obwfEj2S7pF3+VgtIyR9\no/+Yw5ImyRtOB1AHc86lXQcAMTGzdkk/kPS6pOsdf+BAbtDDBnLEOXda3vnrX0j6SMrVARAjetgA\nAGQAPWwAADIg8IYCzTJx4kT3wQ9+MO1qJGbPnj1pVyFRs2fPTrsKiaMNs432y768t6GkHudczUWO\nUh8S7+rqct3d3anWIUlmlnYVEhXrv589MXxXs+P/90wbZhvtl315b0NJe5xzXbUSMSSOdB250wvU\ncQRraSCvI2vjyQ8AWgQBG+k4/aYXWA9+OZn8D97s5X/6SDL5A0CTpX4OGwUUV286in39K3AmMFQO\nAM1EDxvN1cxg3QrlAkBMCNhojr0j0g+ae0w6vjXdOgBAnQjYSN4ek9y7DWdz4x0x1OXAsvR/OABA\nHTiHjWTtHdlwFlZ2scNfP+A9u0avBNw7Qrrotw1mAgDNQw8byXK1g2LnAum+H/jvs4ArE4O2RxZD\njx8AmomAjeTUGHq2Lu/R0yt99i8bD8Kl/EqPC/6ksfoBQCshYCMZNYLht+73315v0PY77uX9EQ4k\naAPICAI24nfmaM0kK+5sQj0U8QfAmZ7E6wEAjSJgI34vTY4tq6DJZQ1POiv3Us019wEgdcwSR7ze\nGLj2yq93Wwq0rjv68Lfrlk71SWPmSSefkUaPil6dTV8ZeB1WHx1eL513U/SMAaDJ6GEjXof+XFJw\nMD5YNlo+d9bg/UE951KQDgrWQcddv8R7/tVh//3v1fP1Vf4JAKBFELDRVNMWDbzetbEy0IYNc3/4\nau95wmXBaarzKn9//uKh1RMAWg0BG/FpcMb16yFz1V55zXs+fjI4Tdi+SJgxDqCFEbDRVIvmBu+b\nuih4XxRhve/FlzaWNwCkjYCNRPTt9t/+2Ibm1qPkkfX+2995trn1AIB6EbARj9OVs7rOGeGdQz5n\nxMC2KJdibX6kvuIf3lk7TXn5o0Z670cOr0p0+lh9FQCAhBGwEY997/fd3LdbOv289zrKZVw3fHXw\ntjNnK9/39A5Oc9Xq2nmXyu/dIb29KyDRvkm1MwKAFBCwkbi2YY0dP/ySyvedCxrLb+z7GjseANJA\nwEZTRellL11T+d658PSf+1o85QJAKyNgo+Xcv31o6TdtS6YeANBKEgnYZvZpM/s3M3vFzP4iiTLQ\nWlati5622b3doZQ3lM8BAM0Ue8A2s2GS/lrSQkm/K2mZmf1u3OWgtayLeWXPL9weLV3cd/2K+3MA\nQFyS6GFfLOkV59x+59y7krZK+kwC5SDDFq8M3//tB73nnXv99297xnsOuq92SfXs8euuqF03AGhF\nSQTsKZJeK3t/sH/be8xsuZl1m1n3sWNc91oE0z9Q+f6xoMuqqsxf7r/9MxF7wtXXZ9/rc9kYAGRB\nEgHbb0Hminm+zrnvOOe6nHNdnZ3ci7gIfnzP4G0LV4Qf0xGy1Kgkjf9E+P6Va8P3A0CWJBGwD0qa\nVvZ+qqRDCZSDVjIrfKRkis96JI/XWBb0RI2befSeCt+/YUv4fl8ze+o4CACSl0TA/idJHzaz6WY2\nXNJSSVx4k3dtE+s6LKkZ41ffXOeB7RNirQcAxKUt7gydc2fM7EZJT0gaJulvnXMvx10OEOb7O9Ku\nAQDEK/aALUnOuX+U9I9J5I3smtwhHTmeXvlzLkivbABoFCudIT6zw9cQPTzEFczKfexD0oKLpd+Z\nWn8ez22ukaBG/QEgTYn0sIEgrjv4vPWiuY3dL/vyG6XtzwWXCwBZRsBGvKbeJR0Mn/HVu0MaN997\nfWS7NKmjcv/1t0r3Phq9yLmzpF0bpSfuHth24JA040rvdaSe/bS/il4gAKSAIXHEa3LtG1OXbm/p\nur1gvXW71+suPYYSrCVp90uVx295wluopdSrntwRfrwkadIXh1YoADSZuVr3LkxYV1eX6+7O73il\nmd86Mvnh++/n9DFpn8+F11WiXtK1ZJ50wxJp/mzpxCnpJ/uk2zZJP9sfoX5R/mnN7Am9nKuQbZgj\ntF/25b0NJe1xztX8H5EhccSvvf7V67at8wJ0kPFjpBlTpGsWVm7f9aJ06efrLJRrrwFkAAEbyZjt\npD3hv4pLE9Da26R3qyaLDWVBFdctffzCgd50+xzpzNmIvWtmhgPICAI2khMhaEsDwbreVc/Kjzv7\ngnT6+Yh5EawBZAiTzpCs6bUX9C5NFvNz63LpxNNeb7n06Nvtbfcz7OKIwXr69yIkAoDWwaSzhOV9\nskSkfz8BvezqwHrVfOmhu+qvy7I13ozzcoHD4kPoXdOG2Ub7ZV/e21BMOkPLmO2kvaMk986gXT1P\nSRPGVm4bPU96qy969h1jpDd/JG25zXtI0jc2S7fc7ZN4+hapY2n0zAGgRRCw0RwX9Ufgqt522zBp\n+pXSqw3cgPX4ycre+i8fHdzTlsQ5awCZxjlsNFdZ0HTd0sM7GwvWfs5f7F23XTEcTrAGkHH0sNF8\ns510+ri0b4Kuu0K67ooEy5p5tKHrwgGgVdDDRjraO7zAPW19MvlP2+DlT7AGkBP0sJGuSSu9hxTp\nmu2aGPoGkFP0sNE6ZruBx6wTg3av9uuMz3yj8jgAyCl62GhNbeMGBeC1f5dSXQCgBdDDBgAgAwjY\nAABkAAEbAIAMIGADAJABqd/8w8xyPbU37e83aQVYlJ82zDjaL/sK0Ibc/AMAEnP2hPRiR8Wm1eul\ntTdVpZt5SGp/f/Pqhdyih52wtL/fpPHrPvvy3oaxtl8LLu6T9/aTCvE3GKmHzTlsAAhz5E4vUMcR\nrKWBvI6sjSc/FAY97ISl/f0mjV/32Zf3Nqy7/U6/Ke2bGG9l/Mw8LLVPrvvwvLefVIi/Qc5hA0Bd\n4upNR7HvPO+ZpXVRA0PiAFCumcG6FcpFZhCwAUCS9o5IP2juMen41nTrgJZFwAaAPSa5dxvO5sY7\nYqjLgWXp/3BAS2LSWcLS/n6TxoSX7Mt7G9Zsv70jJffbhsown+lCrruhLCUbLl1Uu155bz+pEH+D\nXNYFADVFCNadC6T7fuC/zy9Yh22PLIYeP/KFHnbC0v5+k8av++zLexuGtl+NoecoPeewwFwr7Udn\nSD99ILQKNWeP5739pEL8DdLDBoBANYL1t+73315vz9nvuJf3RziQ89noR8AGUDxnjtZMsuLOJtRD\nEX8AnOlJvB5ofQRsAMXzUv0ri1ULmlzW8KSzci91xpgZsoqVzgAUyxsD116FnaN23dGHv123dKpP\nGjNPOvmMNHpU9Ops+srA69Bz5ofXS+dV3woMRUIPG0CxHPpzScHB+GDZaPncWYP3B/WcS0E6KFgH\nHXf9Eu/5V4f9979Xz9dX+SdAYRCwAaDMtEUDr3dtrAy0YcPcH77ae55wWXCa6rzK35+/eGj1RPEQ\nsAEUR4Mzrl8Pmav2ymve8/GTwWnC9kXCjPFCI2ADQJlFc4P3TV0UvC+KsN734ksbyxv5R8AGUEh9\nu/23P7ahufUoeWS9//Z3nm1uPdC6CNgAiuF05ayuc0Z455DPGTGwLcqlWJsfqa/4h3fWTlNe/qiR\n3vuRw6sSnT5WXwWQeSxNmrC0v9+ksSxi9uW9Dd9rv5Dzv2fOSu1z+tP7BO3qGeXVacqPl6RjT0oT\nxw0tj/I0vTukse8LrG7FcqV5bz+pEH+DLE0KAFG0DWvs+OGXVL7vXNBYfqHBGoVFwAaAMlEWS1m6\npvJ9rQ7g574WT7kottgDtpn9rZkdNbOfxp03ALSC+7cPLf2mbcnUA8WSRA97s6RPJ5AvANRt1bro\naZvd2x1KeUP5HMiX2AO2c+4ZScfjzhcAGrEu5pU9v3B7tHRx3/Ur7s+B7OAcNgD4WLwyfP+3H/Se\nd+7137/tGe856L7aJVetrnx/3RW164ZiSiVgm9lyM+s2szhvQAcAdZv+gcr3j+2Kdtz85f7bPxOx\nJ1x9ffa9X412HIonlYDtnPuOc64rynVnANAMP75n8LaFK8KP6QhZalSSxn8ifP/KteH7gXIMiQMo\nhlnhK4RNmTR42+M1lgU9UeNmHr2nwvdv2BK+39fMnjoOQh4kcVnXFkk/kfQRMztoZv9H3GUAwJC1\nTazrsKRmjF99c50Htk+ItR7Ijra4M3TOLYs7TwDIm+/vSLsGyBqGxAGg3+SOdMufc0G65aO1cfOP\nhKX9/SaNGw9kX97bcFD7hdwERKp/CPxjH/IC/oFD0i8O1pdHzbuFzR78bzHv7ScV4m8w0s0/Yh8S\nB4Asc93BQXvR3Mbul335jdL254LLBcIQsAEUy9S7pIPhM756d0jj5nuvj2yXJlUNlV9/q3Tvo9GL\nnDtL2rVReuLugW0HDkkzrvReH46yNvm0v4peIHKJIfGEpf39Jo3huOzLexv6tl+NYXHJ62WXer1b\nt0vL1oSnH4rvfl1advngckL5DIdL+W8/qRB/g5GGxAnYCUv7+00a/1lkX97b0Lf9Th+T9vlceF0l\n6vnsJfOkG5ZI82dLJ05JP9kn3bZJ+tn+CPWLEqxn9gRezpX39pMK8TfIOWwA8NXeWfeh29Z5ATrI\n+DHSjCnSNQsrt+96Ubr083UWyrXXED3sxKX9/SaNX/fZl/c2DG2/iEPj7W3Su88N3h65DlW96PY5\n0pmzjQ2Fv1ePnLefVIi/QXrYABBqtosUtEvBut5LvsqPO/uCdPr5iHnVCNYoFhZOAVBs02sv6G1d\nwQH21uXSiae93nLp0bfb2+5n2MURg/X070VIhCJhSDxhaX+/SWM4Lvvy3oaR2i+gl10dWK+aLz10\nV/11WbbGm3FeLnBYPGLvOu/tJxXib5BZ4q0g7e83afxnkX15b8PI7bd3lOTeqdhkXVLPU9KEsZVJ\nR8+T3uqLXoeOMdKbP6rc9o3N0i13+wTs6VukjqWR8857+0mF+BvkHDYARHZRfwSu6m23DZOmXym9\neqj+rI+frOyt//LRwT1tSZyzRijOYQNAubKg6bqlh3c2Fqz9nL/Yu267ondNsEYNDIknLO3vN2kM\nx2Vf3tuw7vY7fVza14Trn2cebei68Ly3n1SIv8FIQ+L0sAHAT3uH1+udtj6Z/Kdt8PJvIFijWOhh\nJyzt7zdp/LrPvry3YaztF+Ga7ZpiHvrOe/tJhfgbpIcNALGa7QYes04M2r3arzM+843K44A60cNO\nWNrfb9L4dZ99eW9D2i/7CtCG9LABAMgLAjYAABlAwAYAIANSX+ls9uzZ6u6Oco+5bMr7+aW8n1uS\naMOso/2yL+9tGBU9bAAAMiD1HjZQFIF3ZRqCeu/HDCD76GEDCbr52oF7JMehlNeqa+LJD0B2ELCB\nBHSM8QLrnV9KJv+1N3n5T+pIJn8ArYchcSBmcfWmozjSf4tGhsqB/KOHDcSomcG6FcoF0DwEbCAG\nv3k2/aDpuqU//VS6dQCQHAI20CDXLY0Y3ng+N97ReB5bb0//hwOAZHAOG2jAO7sbz6P8/PNfP+A9\nNxp0f/OsNPIPG8sDQGuhhw00YOSI2mk6F0j3/cB/X9BksUYnkcXR4wfQWgjYQJ1q9YKty3v09Eqf\n/cvGg3Apv9Ljgj9prH4AsoWADdShVjD81v3+2+sN2n7Hvby/9nEEbSA/CNjAEHVGWKxkxZ3J10OK\n9gNgwtjk6wEgeQRsYIiObo8vr6AecJw9456n4ssLQHqYJQ4MwZ9dO/Dar3dbCrSuO/rwt+uWTvVJ\nY+ZJJ5+RRo+KXp9NX4lWn5XLpG9uiZ4vgNZDDxsYgjv61wYPCsYHjw68njtr8P6gnnMpSAcF66Dj\nrl/iPf/qsP/+Uj3Xr/bfDyA7CNhAjKYtGni9a2NloA0b5v7w1d7zhMuC01TnVf7+/MVDqyeA7CFg\nAxE1el759aPB+155zXs+fjI4Tdi+KJgxDmQbARuI0aK5wfumLgreF0VY73vxpY3lDaD1EbCBOvQF\nLEn62Ibm1qPkkfX+2995trn1AJAcAjYQweQJle/PGeENMZ9TtjRplCHnzY/UV/7DO2unKS9/1Ejv\n/ciqJUonjquvfADpI2ADERx+wn97327p9PPe6yiXcd3w1cHbzpytfN/TOzjNVRFmeZfK790hvb3L\nP82xJ2vnA6A1EbCBBrUNa+z44ZdUvu9c0Fh+Y9/X2PEAWhMBG4hRlF720jWV750LT/+5r8VTLoBs\nI2ADTXb/EJc23bQtmXoAyJbYA7aZTTOzp83s52b2spl9Ke4ygGZbtS562mb3dodS3lA+B4DWkkQP\n+4yk1c65/0nSJZL+o5n9bgLlAE2zblW8+X3h9mjp4r7rV9yfA0DzxB6wnXNvOOf29r8+JennkqbE\nXQ7QyhavDN//7Qe95517/fdve8Z7Drqvdkn17PHrrqhdNwDZlOg5bDP7oKTfk/R81fblZtZtZt3H\njh1LsgpAU0z/QOX7xwIuq6o2f7n/9s9E7AlXX599r89lYwDyIbGAbWbvk/SgpJXOuYpVkJ1z33HO\ndTnnujo7O5OqAtA0P75n8LaFK8KP6QhZalSSxn8ifP/KteH7AeRLIgHbzNrlBev7nHP/kEQZQDNN\n/GT4/imTBm97vMayoCdq3Myj91T4/g113N86bD1yAK0tiVniJmmjpJ8755iTilx489f1HZfUjPGr\nb67vuEbv+AUgPUn0sOdKulbSZWb2Yv+jwfsUASj3/R1p1wBAs7XFnaFzbpckiztfoNVN7pCOHE+v\n/DkXpFc2gOSx0hkQUa3h7cNDXMGs3Mc+JC24WPqdqfXn8dzm8P0sXwpkW+w9bKDIXHdwYFw0t7H7\nZV9+o7T9ueByAeQbARsYgtXrpbU3hafp3SGNm++9PrJdmtRRuf/6W6V7H41e5txZ0q6N0hN3D2w7\ncEiacaX3OkrP/osxr5gGoPnM1bpVUMK6urpcd3d+uwfepPn8SvvfTzNUt2GU3qx1DaTbul1atiY8\n/VB89+vSsssHl1OrPkHy3ob8DWZf3ttQ0h7nXM2TVgTshOX9H1ra/36aoboNJ46Tjj0Z4biI54yX\nzJNuWCLNny2dOCX9ZJ902ybpZ/trHxslWE+4LPxyrry3IX+D2Zf3NlTEgM2QODBEPb31H7ttnReg\ng4wfI82YIl2zsHL7rhelSz9fX5lcew3kAwEbqEOUoejSBLT2NundqsliQ5mx7bqlj184UF77HOnM\n2caHwgFkCwEbqFPU88elYF1v8Cw/7uwL0unno+VFsAbyheuwgQYsvaV2GusKDp63LpdOPO0F/tKj\nb7e33c+wi6MF4j/+cu00ALKFSWcJy/tkibT//TRDrTYM6mVXB9ar5ksP3VV/PZat8Wac11N2mLy3\nIX+D2Zf3NhSTzoDmsC7p7V3SqJGD9/U8JU0YW7lt9Dzprb7o+XeMkd78kbTlNu8hSd/YLN1y9+C0\nS2+R7v9h9LwBZAcBG4jBuR/3nqt7vG3DpOlXSq8eqj/v4ycre8y/fHRwT1vinDWQd5zDBmJUHjRd\nt/TwzsaCtZ/zF3vXbZf/OCBYA/lHDxuImXVJ40dLx5+WrrvCeySlc0Fj14UDyA562EACTpzyAvfK\ntcnkv+JOL3+CNVAc9LCBBG3Y4j2keO6oxdA3UFz0sIEmKV2PbV0Dd/Mqt3r94G3nXV55HIDioocN\npODXb/kH4HX3Nb8uALKBHjYAABlAwAYAIAMI2AAAZAABGwCADEj95h9mluuV69P+fpNWgEX5acOM\no/2yrwBtyM0/cu3sCenFjopNq9dLa2+qSjfzkNT+/ubVCwCQCHrYCYv1+90Twy/p2fF+3fy6z768\ntyHtl30FaMNIPWzOYbe6I3d6gTqOYC0N5HUkoTUzAQCJoIedsLq/39NvSvsmxlsZPzMPS+2T6z6c\nX/fZl/c2pP2yrwBtyDnszIqrNx3FvvO855iHygEA8WJIvNU0M1i3QrkAgEgI2K1i74j0g+Yek45v\nTbcOAABfBOxWsMck927D2dx4Rwx1ObAs/R8OAIBBmHSWsJrf796RkvttQ2X43fWp4Xsv23Dpotr1\nYsJL9uW9DWm/7CtAG3JZVyZECNadC6T7fuC/L+geyQ3fOzmGHj8AID70sBMW+v3WGHqO0nMOC8y1\n0n50hvTTB0KrUHP2OL/usy/vbUj7ZV8B2pAedkurEay/db//9np7zn7Hvbw/woGczwaAlkDATsOZ\nozWTrLizCfVQxB8AZ3oSrwcAIBwBOw0v1b+yWLWgyWUNTzor91JnjJkBAOrBSmfN9sbAtVdh56hd\nd/Thb9ctneqTxsyTTj4jjR4VvTqbvjLwOvSc+eH10nnVtwIDADQLPexmO/TnkoKD8cGy0fK5swbv\nD+o5l4J0ULAOOu76Jd7zrw7773+vnq+v8k8AAGgKAnaLmbZo4PWujZWBNmyY+8NXe88TLgtOU51X\n+fvzFw+tngCA5iJgN1ODM65fD5mr9spr3vPxk8FpwvZFwoxxAEgNAbvFLJobvG/qouB9UYT1vhdf\n2ljeAIBkEbBT0rfbf/tjG5pbj5JH1vtvf+fZ5tYDAOCPgN0spytndZ0zwjuHfM6IgW1RLsXa/Eh9\nxT+8s3aa8vJHjfTejxxelej0sfoqAABoCEuTJuy97zfk/O+Zs1L7nP70PkG7ekZ5dZry4yXp2JPS\nxHFDy6M8Te8Oaez7AqtbsVwpyyJmX97bkPbLvgK0IUuTZkXbsMaOH35J5fvOBY3lFxqsAQCpIGC3\nmCiLpSxdU/m+1o/Pz30tnnIBAOmJPWCb2Ugze8HMXjKzl83sq3GXUXT3bx9a+k3bkqkHAKB5kuhh\n/1bSZc65WZIulPRpM7ukxjG5t2pd9LTN7u0OpbyhfA4AQHxiD9jO81b/2/b+R75nDESwLuaVPb9w\ne7R0cd/1K+7PAQCIJpFz2GY2zMxelHRU0g+dc89X7V9uZt1mFuc9pXJl8crw/d9+0Hveudd//7Zn\nvOeg+2qXXLW68v11V9SuGwCg+RK9rMvMxkl6SNIXnXM/DUiT6953lMu6JGnGldKBQ1XH9v+cCRqy\nrnVHr7D9QXlHui0nl3XlSt7bkPbLvgK0YfqXdTnneiXtkPTpJMvJgx/fM3jbwhXhx3SELDUqSeM/\nEb5/5drw/QCA1pHELPHO/p61zOwcSQsk/Wvc5WTOrPAVwqZMGrzt8RrLgp6ocTOP3lPh+zdsCd/v\na2ZPHQcBABrVlkCe75d0r5kNk/eD4AHn3KMJlJMtbRPrOiypGeNX31znge0TYq0HACCa2AO2c26f\npN+LO1/E6/s70q4BAGAoWOmshUzuSLf8ORekWz4AIBg3/0jYoO+3xmzxeofAP/YhL+AfOCT94mB9\nedScIT57cFMxQzX78t6GtF/2FaANI80ST+IcNhoQdinWormN3S/78hul7c8FlwsAaF0E7Gabepd0\nMHzGV+8Oadx87/WR7dKkqqHy62+V7h3CNL65s6RdG6Un7h7YduCQd+23JB2Osjb5tL+KXiAAIHYM\niSfM9/utMSwueb3sUq9363Zp2Zrw9EPx3a9Lyy4fXE4on+FwieG4PMh7G9J+2VeANow0JE7ATpjv\n93v6mLTP58LrKlHPZy+ZJ92wRJo/WzpxSvrJPum2TdLP9keoX5RgPbMn8HIu/rPIvry3Ie2XfQVo\nQ85ht6z2zroP3bbOC9BBxo+RZkyRrllYuX3Xi9Kln6+zUK69BoDU0cNOWOj3G3FovL1Neve5wdsj\n16GqF90+RzpztrGh8Pfqwa//wb/SAAAgAElEQVT7zMt7G9J+2VeANqSH3fJmu0hBuxSs673kq/y4\nsy9Ip5+PmFeNYA0AaB4WTknb9NoLeltXcIC9dbl04mmvt1x69O32tvsZdnHEYD39exESAQCahSHx\nhEX6fgN62dWB9ar50kN31V+XZWu8GeflAofFI/auGY7Lvry3Ie2XfQVoQ2aJt4LI3+/eUZJ7p2KT\ndUk9T0kTxlYmHT1Peqsveh06xkhv/qhy2zc2S7fc7ROwp2+ROpZGzpv/LLIv721I+2VfAdqQc9iZ\nclF/BK7qbbcNk6ZfKb16qP6sj5+s7K3/8tHBPW1JnLMGgBbGOexWUxY0Xbf08M7GgrWf8xd7121X\n9K4J1gDQ0hgST1jd3+/p49K+Jlz/PPNoQ9eFMxyXfXlvQ9ov+wrQhpGGxOlht6r2Dq/XO219MvlP\n2+Dl30CwBgA0Dz3shMX6/Ua4ZrummIe++XWffXlvQ9ov+wrQhvSwc2e2G3jMOjFo92q/zvjMNyqP\nAwBkEj3shKX9/SaNX/fZl/c2pP2yrwBtSA8bAIC8IGADAJABBGwAADIg9ZXOZs+ere7uKPd5zKa8\nn1/K+7kliTbMOtov+/LehlHRwwYAIANS72EDANAsgXcoHIJItyhOAD1sAECu3XytF6jjCNbSQF6r\nroknv6gI2ACAXOoY4wXWO7+UTP5rb/Lyn9SRTP7VGBIHAOROXL3pKI7036446aFyetgAgFxpZrBu\nZrkEbABALvzm2fSCdYnrlv70U8nkTcAGAGSe65ZGDG88nxvvaDyPrbcn88OBc9gAgEx7Z3fjeZSf\nf/7rB7znRoPub56VRv5hY3mUo4cNAMi0kSNqp+lcIN33A/99QZPFGp1EFkePvxwBGwCQWbV6wdbl\nPXp6pc/+ZeNBuJRf6XHBnzRWv6EgYAMAMqlWMPzW/f7b6w3afse9vL/2cXEFbQI2ACBzOiMsVrLi\nzuTrIUX7ATBhbOPlELABAJlzdHt8eQX1gOMczu55qvE8mCUOAMiUP7t24LVf77YUaF139OFv1y2d\n6pPGzJNOPiONHhW9Ppu+Eq0+K5dJ39wSPd9q9LABAJlyR//a4EHB+ODRgddzZw3eH9RzLgXpoGAd\ndNz1S7znXx3231+q5/rV/vujImADAHJl2qKB17s2VgbasGHuD1/tPU+4LDhNdV7l789fPLR6DhUB\nGwCQGY2eV379aPC+V17zno+fDE4Tti+KRupPwAYA5MqiucH7pi4K3hdFWO978aWN5V0LARsAkEl9\nAUuSPrahufUoeWS9//Z3no0nfwI2ACATJk+ofH/OCG+I+ZyypUmjDDlvfqS+8h/eWTtNefmjRnrv\nR1YtUTpxXH3lE7ABAJlw+An/7X27pdPPe6+jXMZ1w1cHbztztvJ9T+/gNFdFmOVdKr93h/T2Lv80\nx56snY8fAjYAIPPahjV2/PBLKt93Lmgsv7Hva+x4PwRsAECuROllL11T+d658PSf+1o85TYikYBt\nZsPM7J/N7NEk8gcAoBH3D3Fp003bkqnHUCTVw/6SpJ8nlDcAoIBWrYueNunebiPlDeVzlIs9YJvZ\nVElXSLon7rwBAMW1blW8+X3h9mjp4r7rV72fI4ke9jclfVnSfw9KYGbLzazbzLqPHTuWQBUAAEW3\neGX4/m8/6D3v3Ou/f9sz3nPQfbVLqmePX3dF7brVI9aAbWaLJR11zu0JS+ec+45zrss519XZ2Rln\nFQAABTX9A5XvHwu4rKra/OX+2z8TsSdcfX32vT6XjcUh7h72XElXmtmrkrZKuszM/i7mMgAAGOTH\nPidiF64IP6YjZKlRSRr/ifD9K9eG749TrAHbOXeLc26qc+6DkpZK+pFz7rNxlgEAKKaJnwzfP2XS\n4G2P11gW9ESNm3n0ngrfv6GO+1uHrUcehuuwAQCZ8Oav6zsuqRnjV99c33H13vGrrb7DanPO7ZC0\nI6n8AQBI0/d3NLc8etgAgNyY3JFu+XMuSC5vAjYAIDNqDW8fHuIKZuU+9iFpwcXS70ytP4/nNofv\nb2R4PrEhcQAA0uC6gwPjormN3S/78hul7c8Fl5skAjYAIFNWr5fW3hSepneHNG6+9/rIdmlS1VD5\n9bdK9w7hbhdzZ0m7NkpP3D2w7cAhacaV3usoPfsvNrhimrlatyhJWFdXl+vuTvhnSYrMLO0qJCrt\nfz/NQBtmG+2XfX5tGKU3a10D6bZul5atCU8/FN/9urTs8sHl1KpPgD3OuZqD5QTshPGfRfbRhtlG\n+2WfXxtOHCcdezLCsRHPGS+ZJ92wRJo/WzpxSvrJPum2TdLP9tc+NkqwnnBZ6OVckQI2Q+IAgMzp\n6a3/2G3rvAAdZPwYacYU6ZqFldt3vShd+vn6yqz32utyBGwAQCZFGYouTUBrb5PerZosNpQZ265b\n+viFA+W1z5HOnG14KHxICNgAgMyKev64FKzrDZ7lx519QTr9fLS84lxljeuwAQCZtvSW2mmsKzh4\n3rpcOvG0F/hLj77d3nY/wy6OFoj/+Mu10wwFk84SxoSX7KMNs432y74obRjUy64OrFfNlx66q/66\nLFvjzTivp+wQTDoDABSDdUlv75JGjRy8r+cpacLYym2j50lv9UXPv2OM9OaPpC23eQ9J+sZm6Za7\nB6ddeot0/w+j5x0VARsAkAvnftx7ru7xtg2Tpl8pvXqo/ryPn6zsMf/y0cE9bSm5O4NJnMMGAORM\nedB03dLDOxsL1n7OX+xdt13+4yDJYC3RwwYA5JB1SeNHS8eflq67wnskpXNBY9eFR0UPGwCQSydO\neYF75dpk8l9xp5d/M4K1RA8bAJBzG7Z4DymeO2olPfQdhB42AKAwStdjW9fA3bzKrV4/eNt5l1ce\nlxZ62ACAQvr1W/4BeN19za9LFPSwAQDIAAI2AAAZQMAGACADUl9L3MxyvRBu2t9v0vK+TrNEG2Yd\n7Zd9BWjDSGuJ08MGACADmCUOIDZZvsYVaHX0sAE05OZrB+4hHIdSXquuiSc/IC84h52wtL/fpHH+\nLPvqbcPS7QaTNvmPpKPH6z+e9su+ArQh98MGkIy4etNRHOm/hSFD5Sg6hsQBDEkzg3UrlAu0CgI2\ngEh+82z6QdN1S3/6qXTrAKSFgA2gJtctjRjeeD433tF4HltvT/+HA5AGJp0lLO3vN2lMeMm+Wm34\nzm5p5IgGy/A5/9xo0P3tu9LIP6ydrujtlwcFaEMWTgHQuCjBunOBdN8P/PcFTRZrdBJZHD1+IEvo\nYScs7e83afy6z76wNqzVC47Scw4LzLXSfnSG9NMHhl6HijIK3H55UYA2pIcNoH61gvW37vffXm/P\n2e+4l/fXPo7z2SgKAjaAQTo7aqdZcWfy9ZCi/QCYMDb5egBpI2ADGOTo9vjyCuoBx9kz7nkqvryA\nVsVKZwAq/Nm1A6/DzlG77ujD365bOtUnjZknnXxGGj0qen02fSVafVYuk765JXq+QNbQwwZQ4Y4v\nec9Bwfjg0YHXc2cN3h/Ucy4F6aBgHXTc9Uu8518d9t9fquf61f77gbwgYAMYkmmLBl7v2lgZaMOG\nuT98tfc84bLgNNV5lb8/f/HQ6gnkDQEbwHsaPa/8+tHgfa+85j0fPxmcJmxfFMwYR54RsAEMyaK5\nwfumLgreF0VY73vxpY3lDWQdARuAr77d/tsf29DcepQ8st5/+zvPNrceQFoI2AAkSZMnVL4/Z4Q3\nxHxO2dKkUYacNz9SX/kP76ydprz8USO99yOrliidOK6+8oFWx9KkCUv7+00ayyJmX6kNw4LxmbNS\n+xwFpqueUV6dpvx4STr25ODAWiuP8jS9O6Sx7wuub3leRWm/PCtAG7I0KYB4tA1r7Pjhl1S+71zQ\nWH5hwRrIKwI2gCGJsljK0jWV72t1kD73tXjKBfIskYBtZq+a2b+Y2YtmxoUWQMHcP8SlTTdtS6Ye\nQJ4k2cP+hHPuwijj8gDSt2pd9LTN7u0OpbyhfA4gSxgSByBJWrcq3vy+cHu0dHHf9SvuzwG0iqQC\ntpO03cz2mNny6p1mttzMuhkuB7Jr8crw/d9+0Hveudd//7ZnvOeg+2qXXFW1Rvh1V9SuG5BHiVzW\nZWYfcM4dMrNJkn4o6YvOuWcC0uZ6vn4BLkdIuwqJK0ob1rrGesaV0oFDldtKxwQNWde6o1fY/qC8\no1wLzmVd+VKANkzvsi7n3KH+56OSHpJ0cRLlAGieH98zeNvCFeHHdIQsNSpJ4z8Rvn/l2vD9QJHE\nHrDN7FwzG116LemPJP007nIAxGviJ8P3T5k0eNvjNZYFPVHjZh69p8L3b6jj/tZh65EDWdaWQJ6T\nJT3UP0zTJum7zrnHEygHQIze/HV9xyU1Y/zqm+s7rtE7fgGtKvaA7ZzbL8nntvYAEN33d6RdA6C1\ncFkXgMgmd6Rb/pwL0i0fSBM3/0hY2t9v0pihmn3VbVhrFna9Q+Af+5AX8A8ckn5xsL486qlb0dov\njwrQhpFmiSdxDhtAjoVdirVobmP3y778Rmn7c8HlAkVGwAZQYfV6ae1N4Wl6d0jj5nuvj2yXJlUN\nlV9/q3Tvo9HLnDtL2rVReuLugW0HDnnXfkvS4Qhrk38x5hXTgFbDkHjC0v5+k8ZwXPb5tWHUxUlK\n6bZul5atCU8/FN/9urTs8sHl1KqPnyK2X94UoA0jDYkTsBOW9vebNP6zyD6/Npw4Tjr2ZIRjI57P\nXjJPumGJNH+2dOKU9JN90m2bpJ/tr31slGA94bLgy7mK2H55U4A25Bw2gPr09NZ/7LZ1XoAOMn6M\nNGOKdM3Cyu27XpQu/Xx9ZXLtNYqAHnbC0v5+k8av++wLa8OoQ9HtbdK7zw3eHlV1Oe1zpDNnGxsK\nfy/vArdfXhSgDelhA2hM1PPHpWBd7yVf5cedfUE6/Xy0vJp9X24gTSycAiDU0ltqp7Gu4OB563Lp\nxNNe4C89+nZ72/0MuzhaIP7jL9dOA+QJQ+IJS/v7TRrDcdkXpQ2DetnVgfWq+dJDd9Vfl2VrvBnn\n9ZQdhPbLvgK0IbPEW0Ha32/S+M8i+6K24du7pFEjq47tknqekiaMrdw+ep70Vl/0OnSMkd78UeW2\nb2yWbrl7cMBeeot0/w+j5037ZV8B2pBz2ADic+7HvefqANo2TJp+pfTqofrzPn6yssf8y0cH97Ql\nzlmj2DiHDWBIyoOm65Ye3tlYsPZz/mLvuu3yHwcEaxQdQ+IJS/v7TRrDcdlXbxuOHy0dfzrmyvjo\nXNDYdeG0X/YVoA0jDYnTwwZQlxOnvF7vyrXJ5L/izv5z5A0EayBP6GEnLO3vN2n8us++ONswjjtq\nxT30TftlXwHakB42gOYqXY9tXQN38yq3ev3gbeddXnkcAH/0sBOW9vebNH7dZ1/e25D2y74CtCE9\nbAAA8oKADQBABhCwAQDIgNRXOps9e7a6u2OYWtqi8n5+Ke/nliTaMOtov+zLextGRQ8bAIAMIGAD\nAJABqQ+JAwBayJ4Yhp9n53+YPg30sAGg6I7c6QXqOIK1NJDXkYTWrS0oAjYAFNXpN73AevDLyeR/\n8GYv/9NHksm/YBgSB4Aiiqs3HcW+87xnhsobQg8bAIqmmcG6FcrNCQI2ABTF3hHpB809Jh3fmm4d\nMoqADQBFsMck927D2dx4Rwx1ObAs/R8OGcQ5bADIu70jG86i/Nanf/2A99zw/c/3jpAu+m2DmRQH\nPWwAyDtXOyh2LpDu+4H/vqD7lDd8//IYevxFQsAGgDyrMfRsXd6jp1f67F82HoRL+ZUeF/xJY/XD\nAAI2AORVjWD4rfv9t9cbtP2Oe3l/hAMJ2pEQsAEgj84crZlkxZ1NqIci/gA405N4PbKOgA0AefTS\n5NiyCppc1vCks3IvdcaYWT4xSxwA8uaNgWuv/Hq3pUDruqMPf7tu6VSfNGaedPIZafSo6NXZ9JWB\n12H10eH10nk3Rc+4YOhhA0DeHPpzScHB+GDZaPncWYP3B/WcS0E6KFgHHXf9Eu/5V4f9979Xz9dX\n+SeAJAI2ABTOtEUDr3dtrAy0YcPcH77ae55wWXCa6rzK35+/eGj1RCUCNgDkSYMzrl8Pmav2ymve\n8/GTwWnC9kXCjPFABGwAKJhFc4P3TV0UvC+KsN734ksby7voCNgAkFN9u/23P7ahufUoeWS9//Z3\nnm1uPbKKgA0AeXG6clbXOSO8c8jnjBjYFuVSrM2P1Ff8wztrpykvf9RI7/3I4VWJTh+rrwI5R8AG\ngLzY937fzX27pdPPe6+jXMZ1w1cHbztztvJ9T+/gNFetrp13qfzeHdLbuwIS7ZtUO6MCImADQAG0\nDWvs+OGXVL7vXNBYfmPf19jxRZRIwDazcWb292b2r2b2czP7gyTKAQAMXZRe9tI1le+dC0//ua/F\nUy6CJdXD3iDpcefc/yhplqSfJ1QOACAB928fWvpN25KpBwbEHrDNbIykeZI2SpJz7l3nnM/ZDgBA\nnFati5622b3doZQ3lM9RJEn0sGdIOiZpk5n9s5ndY2bnJlAOAKDMuphX9vzC7dHSxX3Xr7g/R14k\nEbDbJF0k6W+cc78n6W1Jf1GewMyWm1m3mXUfO8b0fQBIw+KV4fu//aD3vHOv//5tz3jPQffVLqme\nPX7dFbXrhsGSCNgHJR10zvVfRKC/lxfA3+Oc+45zrss519XZyS3VAKAZpn+g8v1jQZdVVZm/3H/7\nZyL2hKuvz77X57Ix1BZ7wHbOHZb0mpl9pH/TJyX9LO5yAABD8+N7Bm9buCL8mI6QpUYlafwnwvev\nXBu+H9EldT/sL0q6z8yGS9ov6YaEygEAlMw6Jr0UPGo5xWc9ksdrLAt6osbNPHpPhe/fsCV8v6+Z\nPXUclH+JBGzn3IuSuOIOAJqpbWJdhyU1Y/zqm+s8sH1CrPXIC1Y6AwAk4vs70q5BvhCwAaBAJnek\nW/6cC9ItP8sI2ACQJ7PD1xA9PMQVzMp97EPSgoul35lafx7Pba6RoEb9iyypSWcAgBbluoPPWy+a\n29j9si+/Udr+XHC5qB8BGwDyZupd0sHwGV+9O6Rx873XR7ZLk6qGyq+/Vbr30ehFzp0l7dooPXH3\nwLYDh6QZV3qvI/Xsp/1V9AILiCFxAMibybVvTF26vaXr9oL11u1er7v0GEqwlqTdL1Uev+UJb6GW\nUq860rnzSV8cWqEFY67WPdMS1tXV5bq78ztOYmZpVyFRaf/7aQbaMNsK236nj0n7fC68rhL1kq4l\n86QblkjzZ0snTkk/2Sfdtkn62f4IdYzyX/zMnsDLufLehpL2OOdqtgRD4gCQR+31L/u8bZ0XoIOM\nHyPNmCJds7By+64XpUs/X2ehXHtdEwEbAPJqtpP2hPdOSxPQ2tukd6smiw1lQRXXLX38woHedPsc\n6czZiL1rZoZHQsAGgDyLELSlgWBd76pn5cedfUE6/XzEvAjWkTHpDADybnrtBb1Lk8X83LpcOvG0\n11suPfp2e9v9DLs4YrCe/r0IiVDCpLOE5X2yRNr/fpqBNsw22q9fQC+7OrBeNV966K7667NsjTfj\nvFzgsHjE3nXe21BMOgMAvGe2k/aOktw7g3b1PCVNGFu5bfQ86a2+6Nl3jJHe/JG05TbvIUnf2Czd\ncrdP4ulbpI6l0TOHJAI2ABTHRf0RuKq33TZMmn6l9Oqh+rM+frKyt/7LRwf3tCVxzroBnMMGgKIp\nC5quW3p4Z2PB2s/5i73rtiuGwwnWDaGHDQBFNNtJp49L+ybouiuk665IsKyZRxu6LhweetgAUFTt\nHV7gnrY+mfynbfDyJ1jHgh42ABTdpJXeQ4p0zXZNDH0ngh42AGDAbDfwmHVi0O7Vfp3xmW9UHodE\n0MMGAPhrGzcoAK/9u5TqAnrYAABkAQEbAIAMIGADAJABqa8lbma5nqGQ9vebtAKs8UsbZhztl30F\naMNIa4nTwwYAIANyM0s80k3Sa6j3PrAAACQt0z3sm68duDdrHEp5rbomnvwAAIhLJs9hl27jlrTJ\nfyQdPd5YHml/v0nj/Fn25b0Nab/sK0Ab5vN+2HH1pqM40n9rOIbKAQBpy9SQeDODdSuUCwBASSYC\n9m+eTT9oum7pTz+Vbh0AAMXV8gHbdUsjhjeez413NJ7H1tvT/+EAACimlp509s5uaeSIBvP3Of/c\naND97bvSyD+Mljbt7zdpTHjJvry3Ie2XfQVow+wvnBIlWHcukO77gf++oMlijU4ii6PHDwDAULRs\nD7tWLzhKzzksMNdK+9EZ0k8fGHodBpWT/1+GaVchcbRhttF+2VeANsxuD7tWsP7W/f7b6+05+x33\n8v7ax3E+GwDQLC0XsDs7aqdZcWfy9ZCi/QCYMDb5egAA0HIB++j2+PIK6gHH2TPueSq+vAAACNJS\nK5392bUDr8POUbvu6MPfrls61SeNmSedfEYaPSp6fTZ9JVp9Vi6Tvrkler4AAAxVS/Ww7/iS9xwU\njA8eHXg9d9bg/UE951KQDgrWQcddv8R7/tVh//2leq5f7b8fAIC4tFTArmXaooHXuzZWBtqwYe4P\nX+09T7gsOE11XuXvz188tHoCABC3lgnYjZ5Xfv1o8L5XXvOej58MThO2LwpmjAMAktQyATuKRXOD\n901dFLwvirDe9+JLG8sbAIBGtWTA7tvtv/2xDc2tR8kj6/23v/Nsc+sBACiulgjYkydUvj9nhDfE\nfE7Z0qRRhpw3P1Jf+Q/vrJ2mvPxRI733I6uWKJ04rr7yAQCopSWWJg0LxmfOSu1zvNd+6apnlFen\nKT9eko49OTiw1sqjPE3vDmns+4LrOyiv/C+pl3YVEkcbZhvtl30FaMPsLk1arm1YY8cPv6TyfeeC\nxvILC9YAACSl5QN2uSiLpSxdU/m+1g+zz30tnnIBAEhS7AHbzD5iZi+WPU6a2cq4ywly/xCXNt20\nLZl6AAAQp9gDtnPu35xzFzrnLpQ0W1KfpIfCjlm1Lnr+ze7tDqW8oXwOAACGIukh8U9K+oVz7pdh\nidatirfQL9weLV3cd/2K+3MAAFCSdMBeKmnQbTHMbLmZdZtZXeuDLa4xwP7tB73nnXv99297xnsO\nuq92yVVVa4Rfd0XtugEAkITELusys+GSDkn6qHPuSEi60Mu6JGnGldKBQ5XbSscEDVnXuqNX2P6g\nvKNcC85lXflDG2Yb7Zd9BWjD1C/rWihpb1iwjurH9/hkviL8mI6QpUYlafwnwvevXBu+HwCAZkoy\nYC+Tz3C4n4mfDN8/ZdLgbY/XWBb0RI2befSeCt+/oY77W4etRw4AQCMSCdhmNkrSpyT9Q5T0b/66\nznISmjF+9c31HdfoHb8AAAjSlkSmzrk+SRNqJmxR39+Rdg0AAKiUmZXOJnekW/6cC9ItHwBQbC1x\n84/S61qzsOsdAv/Yh7yAf+CQ9IuD9eVRb93S/n6TxgzV7Mt7G9J+2VeANow0SzyRIfGkhF2KtWhu\nY/fLvvxGaftzweUCAJCmlgrYq9dLa28KT9O7Qxo333t9ZLs0qWqo/PpbpXsfjV7m3FnSro3SE3cP\nbDtwyLv2W5IOR1ib/Isxr5gGAEC1lhoSl6IvTlJKt3W7tGxNePqh+O7XpWWXDy6nVn2CpP39Jo3h\nuOzLexvSftlXgDaMNCTecgF74jjp2JMRjot4PnvJPOmGJdL82dKJU9JP9km3bZJ+tr/2sVGC9YTL\nwi/nSvv7TRr/WWRf3tuQ9su+ArRhNs9h9/TWf+y2dV6ADjJ+jDRjinTNwsrtu16ULv18fWVy7TUA\noBlaroddEnUour1Neve5wdujqi6nfY505mzjQ+Hv5Z//X4ZpVyFxtGG20X7ZV4A2zGYPuyTq+eNS\nsK73kq/y486+IJ1+Plpezb4vNwCg2Fp64ZSlt9ROY13BwfPW5dKJp73AX3r07fa2+xl2cbRA/Mdf\nrp0GAIA4teyQeElQL7s6sF41X3rorvrrsWyNN+O8nrLDpP39Jo3huOzLexvSftlXgDbM5ixxP2/v\nkkaNrDquS+p5SpowtnL76HnSW33Ry+8YI735o8pt39gs3XL34IC99Bbp/h9Gz1sqxD+0tKuQONow\n22i/7CtAG2b7HHa5cz/uPVcH0LZh0vQrpVcP1Z/38ZOVPeZfPjq4py1xzhoAkK6WPoddrTxoum7p\n4Z2NBWs/5y/2rtsu/3FAsAYApC0TQ+LVxo+Wjj+dRG0qdS5o7LpwqRBDOWlXIXG0YbbRftlXgDaM\nNCSeqR52yYlTXq935dpk8l9xZ/858gaDNQAAcclkD9tPHHfUSmLoO+3vN2n8us++vLch7Zd9BWjD\n/Paw/ZSux7augbt5lVu9fvC28y6vPA4AgFaVmx52q0r7+00av+6zL+9tSPtlXwHasFg9bAAA8oyA\nDQBABhCwAQDIgFZY6axH0i+bWN7E/jKbIqXzS039jCnIexvSfjGi/WLX9M9XgDY8P0qi1CedNZuZ\ndUc5uZ9lef+MfL5s4/NlW94/n9S6n5EhcQAAMoCADQBABhQxYH8n7Qo0Qd4/I58v2/h82Zb3zye1\n6Gcs3DlsAACyqIg9bAAAMoeADQBABhQqYJvZp83s38zsFTP7i7TrEycz+1szO2pmP027Lkkws2lm\n9rSZ/dzMXjazL6Vdp7iZ2Ugze8HMXur/jF9Nu05xM7NhZvbPZvZo2nVJgpm9amb/YmYvmlkM9xBs\nLWY2zsz+3sz+tf9v8Q/SrlNczOwj/e1Wepw0s5Vp16tcYc5hm9kwSf+fpE9JOijpnyQtc879LNWK\nxcTM5kl6S9J/dc5dkHZ94mZm75f0fufcXjMbLWmPpKvy0n6SZN7qEOc6594ys3ZJuyR9yTn3XMpV\ni42ZrZLUJWmMc25x2vWJm5m9KqnLOZfLhVPM7F5JP3bO3WNmwyWNcs71pl2vuPXHi9clzXHONXNh\nr1BF6mFfLOkV59x+59y7krZK+kzKdYqNc+4ZScfTrkdSnHNvOOf29r8+JennkqakW6t4Oc9b/W/b\n+x+5+UVtZlMlXSHpnn58C9IAAAJTSURBVLTrgqEzszGS5knaKEnOuXfzGKz7fVLSL1opWEvFCthT\nJL1W9v6gcvYfflGY2Qcl/Z6k59OtSfz6h4xflHRU0g+dc3n6jN+U9GVJ/z3tiiTISdpuZnvMbHna\nlYnZDEnHJG3qP61xj5mdm3alErJU0pa0K1GtSAHbbzHa3PReisLM3ifpQUkrnXMn065P3JxzZ51z\nF0qaKuliM8vF6Q0zWyzpqHNuT9p1Sdhc59xFkhZK+o/9p6ryok3SRZL+xjn3e5LelpSruUCS1D/U\nf6Wk76Vdl2pFCtgHJU0rez9V0qGU6oI69J/XfVDSfc65f0i7PknqH2rcIenTKVclLnMlXdl/jner\npMvM7O/SrVL8nHOH+p+PSnpI3qm4vDgo6WDZqM/fywvgebNQ0l7n3JG0K1KtSAH7nyR92Mym9/+C\nWippW8p1QkT9E7I2Svq5c25d2vVJgpl1mtm4/tfnSFog6V/TrVU8nHO3OOemOuc+KO9v70fOuc+m\nXK1Ymdm5/RMi1T9U/EeScnPVhnPusKTXzOwj/Zs+KSk3kz7LLFMLDodLrXF7zaZwzp0xsxslPSFp\nmKS/dc69nHK1YmNmWyTNlzTRzA5K+opzbmO6tYrVXEnXSvqX/nO8krTGOfePKdYpbu+XdG//DNV/\nJ+kB51wuL3/KqcmSHuq/FWSbpO865x5Pt0qx+6Kk+/o7Pfsl3ZByfWJlZqPkXUn0H9Kui5/CXNYF\nAECWFWlIHACAzCJgAwCQAQRsAAAygIANAEAGELABAMgAAjYAABlAwAYAIAP+fzFY3dTllVswAAAA\nAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FdWd7/nP93IOIIZfBw6YAGOgkyczHQO2nBa7iQwxpA0IRmd6umGMXs1kuJO5hiDY6Zbn6Scmz41mVCB07OncXGnw3jagaduI2lGiEQwYtQ+00jHpnseAiYj8OMIJ6DERuGv+qLM9e+9TVbvO3lW7dlW9X8+zn7131aq11t6Lw3evVatWmXNOAACgtf27tCsAAABqI2ADAJABBGwAADKAgA0AQAYQsAEAyAACNgAAGUDABgAgAwjYAABkAAEbaDFm9kEz+0czO2Fmh83sbjNrC0k/zsz+pj9tn5n9i5n9+2bWGUDyCNhA6/l/JR2V9H5JF0r6nyX9334JzWy4pCclnS/pDySNlfRnku4wsxVNqS2ApiBgA61nuqQHnHO/cc4dlvS4pI8GpL1W0v8g6X9zzh1wzp12zj0uaYWk/2RmoyXJzJyZfah0kJltNrP/VPZ+sZm9aGa9Zvasmc0s2/cBM3vQzI6Z2YHyHwJmdquZPWBm/9XMTpnZy2bWVbb/z83s9f59/2Zmn4znKwKKh4ANtJ4Nkpaa2SgzmyJpobyg7edTkn7gnHu7avuDkkZJuqRWYWZ2kaS/lfQfJE2Q9J8lbTOzEWb27yQ9IuklSVMkfVLSSjO7vCyLKyVtlTRO0jZJd/fn+xFJN0r6fefcaEmXS3q1Vn0A+CNgA61np7we9UlJByV1S/p+QNqJkt6o3uicOyOpR1JnhPL+T0n/2Tn3vHPurHPuXkm/lRfsf19Sp3Pua865d51z+yX9F0lLy47f5Zz7R+fcWUn/TdKs/u1nJY2Q9Ltm1u6ce9U594sI9QHgg4ANtJD+Hu0Tkv5B0rnyAvJ4Sf9PwCE98s51V+fT1n/ssQjFni9pdf9weK+Z9UqaJukD/fs+ULVvjaTJZccfLnvdJ2mkmbU5516RtFLSrZKOmtlWM/tAhPoA8EHABlpLh7xgebdz7rfOuTclbZK0KCD9k5IWmtm5Vdv/V0mnJb3Q/75P3hB5yXllr1+T9HXn3Liyxyjn3Jb+fQeq9o12zgXVp4Jz7rvOuY/LC/xOwT88ANRAwAZaiHOuR9IBSV8wszYzGyfp38s7h+znv8kbNv9e/+Vg7f3nl/9K0h3OuV/3p3tR0v9uZsPM7NPyZp6X/BdJ/5eZzTHPuWZ2Rf+EtRckneyfPHZO//EXmNnv1/osZvYRM7vMzEZI+o2kd+QNkwOoAwEbaD3/i6RPyxvOfkXSGUk3+SV0zv1W0gJ5PeHn5QXFxyV9U9JXy5J+SdISSb2SrlHZOXHnXLe889h3SzrRX+b1/fvO9h93obwfEj2S7pF3+VgtIyR9o/+Yw5ImyRtOB1AHc86lXQcAMTGzdkk/kPS6pOsdf+BAbtDDBnLEOXda3vnrX0j6SMrVARAjetgAAGQAPWwAADIg8IYCzTJx4kT3wQ9+MO1qJGbPnj1pVyFRs2fPTrsKiaMNs432y768t6GkHudczUWOUh8S7+rqct3d3anWIUlmlnYVEhXrv589MXxXs+P/90wbZhvtl315b0NJe5xzXbUSMSSOdB250wvUcQRraSCvI2vjyQ8AWgQBG+k4/aYXWA9+OZn8D97s5X/6SDL5A0CTpX4OGwUUV286in39K3AmMFQOAM1EDxvN1cxg3QrlAkBMCNhojr0j0g+ae0w6vjXdOgBAnQjYSN4ek9y7DWdz4x0x1OXAsvR/OABAHTiHjWTtHdlwFlZ2scNfP+A9u0avBNw7Qrrotw1mAgDNQw8byXK1g2LnAum+H/jvs4ArE4O2RxZDjx8AmomAjeTUGHq2Lu/R0yt99i8bD8Kl/EqPC/6ksfoBQCshYCMZNYLht+73315v0PY77uX9EQ4kaAPICAI24nfmaM0kK+5sQj0U8QfAmZ7E6wEAjSJgI34vTY4tq6DJZQ1POiv3Us019wEgdcwSR7zeGLj2yq93Wwq0rjv68Lfrlk71SWPmSSefkUaPil6dTV8ZeB1WHx1eL513U/SMAaDJ6GEjXof+XFJwMD5YNlo+d9bg/UE951KQDgrWQcddv8R7/tVh//3v1fP1Vf4JAKBFELDRVNMWDbzetbEy0IYNc3/4au95wmXBaarzKn9//uKh1RMAWg0BG/FpcMb16yFz1V55zXs+fjI4Tdi+SJgxDqCFEbDRVIvmBu+buih4XxRhve/FlzaWNwCkjYCNRPTt9t/+2Ibm1qPkkfX+2995trn1AIB6EbARj9OVs7rOGeGdQz5nxMC2KJdibX6kvuIf3lk7TXn5o0Z670cOr0p0+lh9FQCAhBGwEY997/fd3LdbOv289zrKZVw3fHXwtjNnK9/39A5Oc9Xq2nmXyu/dIb29KyDRvkm1MwKAFBCwkbi2YY0dP/ySyvedCxrLb+z7GjseANJAwEZTRellL11T+d658PSf+1o85QJAKyNgo+Xcv31o6TdtS6YeANBKEgnYZvZpM/s3M3vFzP4iiTLQWlati5622b3doZQ3lM8BAM0Ue8A2s2GS/lrSQkm/K2mZmf1u3OWgtayLeWXPL9weLV3cd/2K+3MAQFyS6GFfLOkV59x+59y7krZK+kwC5SDDFq8M3//tB73nnXv99297xnsOuq92SfXs8euuqF03AGhFSQTsKZJeK3t/sH/be8xsuZl1m1n3sWNc91oE0z9Q+f6xoMuqqsxf7r/9MxF7wtXXZ9/rc9kYAGRBEgHbb0Hminm+zrnvOOe6nHNdnZ3ci7gIfnzP4G0LV4Qf0xGy1Kgkjf9E+P6Va8P3A0CWJBGwD0qaVvZ+qqRDCZSDVjIrfKRkis96JI/XWBb0RI2befSeCt+/YUv4fl8ze+o4CACSl0TA/idJHzaz6WY2XNJSSVx4k3dtE+s6LKkZ41ffXOeB7RNirQcAxKUt7gydc2fM7EZJT0gaJulvnXMvx10OEOb7O9KuAQDEK/aALUnOuX+U9I9J5I3smtwhHTmeXvlzLkivbABoFCudIT6zw9cQPTzEFczKfexD0oKLpd+ZWn8ez22ukaBG/QEgTYn0sIEgrjv4vPWiuY3dL/vyG6XtzwWXCwBZRsBGvKbeJR0Mn/HVu0MaN997fWS7NKmjcv/1t0r3Phq9yLmzpF0bpSfuHth24JA040rvdaSe/bS/il4gAKSAIXHEa3LtG1OXbm/pur1gvXW71+suPYYSrCVp90uVx295wluopdSrntwRfrwkadIXh1YoADSZuVr3LkxYV1eX6+7O73ilmd86Mvnh++/n9DFpn8+F11WiXtK1ZJ50wxJp/mzpxCnpJ/uk2zZJP9sfoX5R/mnN7Am9nKuQbZgjtF/25b0NJe1xztX8H5EhccSvvf7V67at8wJ0kPFjpBlTpGsWVm7f9aJ06efrLJRrrwFkAAEbyZjtpD3hv4pLE9Da26R3qyaLDWVBFdctffzCgd50+xzpzNmIvWtmhgPICAI2khMhaEsDwbreVc/Kjzv7gnT6+Yh5EawBZAiTzpCs6bUX9C5NFvNz63LpxNNeb7n06Nvtbfcz7OKIwXr69yIkAoDWwaSzhOV9skSkfz8BvezqwHrVfOmhu+qvy7I13ozzcoHD4kPoXdOG2Ub7ZV/e21BMOkPLmO2kvaMk986gXT1PSRPGVm4bPU96qy969h1jpDd/JG25zXtI0jc2S7fc7ZN4+hapY2n0zAGgRRCw0RwX9Ufgqt522zBp+pXSqw3cgPX4ycre+i8fHdzTlsQ5awCZxjlsNFdZ0HTd0sM7GwvWfs5f7F23XTEcTrAGkHH0sNF8s510+ri0b4Kuu0K67ooEy5p5tKHrwgGgVdDDRjraO7zAPW19MvlP2+DlT7AGkBP0sJGuSSu9hxTpmu2aGPoGkFP0sNE6ZruBx6wTg3av9uuMz3yj8jgAyCl62GhNbeMGBeC1f5dSXQCgBdDDBgAgAwjYAABkAAEbAIAMIGADAJABqd/8w8xyPbU37e83aQVYlJ82zDjaL/sK0Ibc/AMAEnP2hPRiR8Wm1eultTdVpZt5SGp/f/Pqhdyih52wtL/fpPHrPvvy3oaxtl8LLu6T9/aTCvE3GKmHzTlsAAhz5E4vUMcRrKWBvI6sjSc/FAY97ISl/f0mjV/32Zf3Nqy7/U6/Ke2bGG9l/Mw8LLVPrvvwvLefVIi/Qc5hA0Bd4upNR7HvPO+ZpXVRA0PiAFCumcG6FcpFZhCwAUCS9o5IP2juMen41nTrgJZFwAaAPSa5dxvO5sY7YqjLgWXp/3BAS2LSWcLS/n6TxoSX7Mt7G9Zsv70jJffbhsown+lCrruhLCUbLl1Uu155bz+pEH+DXNYFADVFCNadC6T7fuC/zy9Yh22PLIYeP/KFHnbC0v5+k8av++zLexuGtl+NoecoPeewwFwr7UdnSD99ILQKNWeP5739pEL8DdLDBoBANYL1t+73315vz9nvuJf3RziQ89noR8AGUDxnjtZMsuLOJtRDEX8AnOlJvB5ofQRsAMXzUv0ri1ULmlzW8KSzci91xpgZsoqVzgAUyxsD116FnaN23dGHv123dKpPGjNPOvmMNHpU9Ops+srA69Bz5ofXS+dV3woMRUIPG0CxHPpzScHB+GDZaPncWYP3B/WcS0E6KFgHHXf9Eu/5V4f9979Xz9dX+SdAYRCwAaDMtEUDr3dtrAy0YcPcH77ae55wWXCa6rzK35+/eGj1RPEQsAEUR4Mzrl8Pmav2ymve8/GTwWnC9kXCjPFCI2ADQJlFc4P3TV0UvC+KsN734ksbyxv5R8AGUEh9u/23P7ahufUoeWS9//Z3nm1uPdC6CNgAiuF05ayuc0Z455DPGTGwLcqlWJsfqa/4h3fWTlNe/qiR3vuRw6sSnT5WXwWQeSxNmrC0v9+ksSxi9uW9Dd9rv5Dzv2fOSu1z+tP7BO3qGeXVacqPl6RjT0oTxw0tj/I0vTukse8LrG7FcqV5bz+pEH+DLE0KAFG0DWvs+OGXVL7vXNBYfqHBGoVFwAaAMlEWS1m6pvJ9rQ7g574WT7kottgDtpn9rZkdNbOfxp03ALSC+7cPLf2mbcnUA8WSRA97s6RPJ5AvANRt1broaZvd2x1KeUP5HMiX2AO2c+4ZScfjzhcAGrEu5pU9v3B7tHRx3/Ur7s+B7OAcNgD4WLwyfP+3H/Sed+7137/tGe856L7aJVetrnx/3RW164ZiSiVgm9lyM+s2szhvQAcAdZv+gcr3j+2Kdtz85f7bPxOxJ1x9ffa9X412HIonlYDtnPuOc64rynVnANAMP75n8LaFK8KP6QhZalSSxn8ifP/KteH7gXIMiQMohlnhK4RNmTR42+M1lgU9UeNmHr2nwvdv2BK+39fMnjoOQh4kcVnXFkk/kfQRMztoZv9H3GUAwJC1TazrsKRmjF99c50Htk+ItR7Ijra4M3TOLYs7TwDIm+/vSLsGyBqGxAGg3+SOdMufc0G65aO1cfOPhKX9/SaNGw9kX97bcFD7hdwERKp/CPxjH/IC/oFD0i8O1pdHzbuFzR78bzHv7ScV4m8w0s0/Yh8SB4Asc93BQXvR3Mbul335jdL254LLBcIQsAEUy9S7pIPhM756d0jj5nuvj2yXJlUNlV9/q3Tvo9GLnDtL2rVReuLugW0HDkkzrvReH46yNvm0v4peIHKJIfGEpf39Jo3huOzLexv6tl+NYXHJ62WXer1bt0vL1oSnH4rvfl1advngckL5DIdL+W8/qRB/g5GGxAnYCUv7+00a/1lkX97b0Lf9Th+T9vlceF0l6vnsJfOkG5ZI82dLJ05JP9kn3bZJ+tn+CPWLEqxn9gRezpX39pMK8TfIOWwA8NXeWfeh29Z5ATrI+DHSjCnSNQsrt+96Ubr083UWyrXXED3sxKX9/SaNX/fZl/c2DG2/iEPj7W3Su88N3h65DlW96PY50pmzjQ2Fv1ePnLefVIi/QXrYABBqtosUtEvBut5LvsqPO/uCdPr5iHnVCNYoFhZOAVBs02sv6G1dwQH21uXSiae93nLp0bfb2+5n2MURg/X070VIhCJhSDxhaX+/SWM4Lvvy3oaR2i+gl10dWK+aLz10V/11WbbGm3FeLnBYPGLvOu/tJxXib5BZ4q0g7e83afxnkX15b8PI7bd3lOTeqdhkXVLPU9KEsZVJR8+T3uqLXoeOMdKbP6rc9o3N0i13+wTs6VukjqWR8857+0mF+BvkHDYARHZRfwSu6m23DZOmXym9eqj+rI+frOyt//LRwT1tSZyzRijOYQNAubKg6bqlh3c2Fqz9nL/Yu267ondNsEYNDIknLO3vN2kMx2Vf3tuw7vY7fVza14Trn2cebei68Ly3n1SIv8FIQ+L0sAHAT3uH1+udtj6Z/Kdt8PJvIFijWOhhJyzt7zdp/LrPvry3YaztF+Ga7ZpiHvrOe/tJhfgbpIcNALGa7QYes04M2r3arzM+843K44A60cNOWNrfb9L4dZ99eW9D2i/7CtCG9LABAMgLAjYAABlAwAYAIANSX+ls9uzZ6u6Oco+5bMr7+aW8n1uSaMOso/2yL+9tGBU9bAAAMiD1HjZQFIF3ZRqCeu/HDCD76GEDCbr52oF7JMehlNeqa+LJD0B2ELCBBHSM8QLrnV9KJv+1N3n5T+pIJn8ArYchcSBmcfWmozjSf4tGhsqB/KOHDcSomcG6FcoF0DwEbCAGv3k2/aDpuqU//VS6dQCQHAI20CDXLY0Y3ng+N97ReB5bb0//hwOAZHAOG2jAO7sbz6P8/PNfP+A9Nxp0f/OsNPIPG8sDQGuhhw00YOSI2mk6F0j3/cB/X9BksUYnkcXR4wfQWgjYQJ1q9YKty3v09Eqf/cvGg3Apv9Ljgj9prH4AsoWADdShVjD81v3+2+sN2n7Hvby/9nEEbSA/CNjAEHVGWKxkxZ3J10OK9gNgwtjk6wEgeQRsYIiObo8vr6AecJw9456n4ssLQHqYJQ4MwZ9dO/Dar3dbCrSuO/rwt+uWTvVJY+ZJJ5+RRo+KXp9NX4lWn5XLpG9uiZ4vgNZDDxsYgjv61wYPCsYHjw68njtr8P6gnnMpSAcF66Djrl/iPf/qsP/+Uj3Xr/bfDyA7CNhAjKYtGni9a2NloA0b5v7w1d7zhMuC01TnVf7+/MVDqyeA7CFgAxE1el759aPB+155zXs+fjI4Tdi+KJgxDmQbARuI0aK5wfumLgreF0VY73vxpY3lDaD1EbCBOvQFLEn62Ibm1qPkkfX+2995trn1AJAcAjYQweQJle/PGeENMZ9TtjRplCHnzY/UV/7DO2unKS9/1Ejv/ciqJUonjquvfADpI2ADERx+wn97327p9PPe6yiXcd3w1cHbzpytfN/TOzjNVRFmeZfK790hvb3LP82xJ2vnA6A1EbCBBrUNa+z44ZdUvu9c0Fh+Y9/X2PEAWhMBG4hRlF720jWV750LT/+5r8VTLoBsI2ADTXb/EJc23bQtmXoAyJbYA7aZTTOzp83s52b2spl9Ke4ygGZbtS562mb3dodS3lA+B4DWkkQP+4yk1c65/0nSJZL+o5n9bgLlAE2zblW8+X3h9mjp4r7rV9yfA0DzxB6wnXNvOOf29r8+JennkqbEXQ7QyhavDN//7Qe95517/fdve8Z7Drqvdkn17PHrrqhdNwDZlOg5bDP7oKTfk/R81fblZtZtZt3Hjh1LsgpAU0z/QOX7xwIuq6o2f7n/9s9E7AlXX599r89lYwDyIbGAbWbvk/SgpJXOuYpVkJ1z33HOdTnnujo7O5OqAtA0P75n8LaFK8KP6QhZalSSxn8ifP/KteH7AeRLIgHbzNrlBev7nHP/kEQZQDNN/GT4/imTBm97vMayoCdq3Myj91T4/g113N86bD1yAK0tiVniJmmjpJ8755iTilx489f1HZfUjPGrb67vuEbv+AUgPUn0sOdKulbSZWb2Yv+jwfsUASj3/R1p1wBAs7XFnaFzbpckiztfoNVN7pCOHE+v/DkXpFc2gOSx0hkQUa3h7cNDXMGs3Mc+JC24WPqdqfXn8dzm8P0sXwpkW+w9bKDIXHdwYFw0t7H7ZV9+o7T9ueByAeQbARsYgtXrpbU3hafp3SGNm++9PrJdmtRRuf/6W6V7H41e5txZ0q6N0hN3D2w7cEiacaX3OkrP/osxr5gGoPnM1bpVUMK6urpcd3d+uwfepPn8SvvfTzNUt2GU3qx1DaTbul1atiY8/VB89+vSsssHl1OrPkHy3ob8DWZf3ttQ0h7nXM2TVgTshOX9H1ra/36aoboNJ46Tjj0Z4biI54yXzJNuWCLNny2dOCX9ZJ902ybpZ/trHxslWE+4LPxyrry3IX+D2Zf3NlTEgM2QODBEPb31H7ttnRegg4wfI82YIl2zsHL7rhelSz9fX5lcew3kAwEbqEOUoejSBLT2NundqsliQ5mx7bqlj184UF77HOnM2caHwgFkCwEbqFPU88elYF1v8Cw/7uwL0unno+VFsAbyheuwgQYsvaV2GusKDp63LpdOPO0F/tKjb7e33c+wi6MF4j/+cu00ALKFSWcJy/tkibT//TRDrTYM6mVXB9ar5ksP3VV/PZat8Wac11N2mLy3IX+D2Zf3NhSTzoDmsC7p7V3SqJGD9/U8JU0YW7lt9Dzprb7o+XeMkd78kbTlNu8hSd/YLN1y9+C0S2+R7v9h9LwBZAcBG4jBuR/3nqt7vG3DpOlXSq8eqj/v4ycre8y/fHRwT1vinDWQd5zDBmJUHjRdt/TwzsaCtZ/zF3vXbZf/OCBYA/lHDxuImXVJ40dLx5+WrrvCeySlc0Fj14UDyA562EACTpzyAvfKtcnkv+JOL3+CNVAc9LCBBG3Y4j2keO6oxdA3UFz0sIEmKV2PbV0Dd/Mqt3r94G3nXV55HIDioocNpODXb/kH4HX3Nb8uALKBHjYAABlAwAYAIAMI2AAAZAABGwCADEj95h9mluuV69P+fpNWgEX5acOMo/2yrwBtyM0/cu3sCenFjopNq9dLa2+qSjfzkNT+/ubVCwCQCHrYCYv1+90Twy/p2fF+3fy6z768tyHtl30FaMNIPWzOYbe6I3d6gTqOYC0N5HUkoTUzAQCJoIedsLq/39NvSvsmxlsZPzMPS+2T6z6cX/fZl/c2pP2yrwBtyDnszIqrNx3FvvO855iHygEA8WJIvNU0M1i3QrkAgEgI2K1i74j0g+Yek45vTbcOAABfBOxWsMck927D2dx4Rwx1ObAs/R8OAIBBmHSWsJrf796RkvttQ2X43fWp4Xsv23Dpotr1YsJL9uW9DWm/7CtAG3JZVyZECNadC6T7fuC/L+geyQ3fOzmGHj8AID70sBMW+v3WGHqO0nMOC8y10n50hvTTB0KrUHP2OL/usy/vbUj7ZV8B2pAedkurEay/db//9np7zn7Hvbw/woGczwaAlkDATsOZozWTrLizCfVQxB8AZ3oSrwcAIBwBOw0v1b+yWLWgyWUNTzor91JnjJkBAOrBSmfN9sbAtVdh56hdd/Thb9ctneqTxsyTTj4jjR4VvTqbvjLwOvSc+eH10nnVtwIDADQLPexmO/TnkoKD8cGy0fK5swbvD+o5l4J0ULAOOu76Jd7zrw7773+vnq+v8k8AAGgKAnaLmbZo4PWujZWBNmyY+8NXe88TLgtOU51X+fvzFw+tngCA5iJgN1ODM65fD5mr9spr3vPxk8FpwvZFwoxxAEgNAbvFLJobvG/qouB9UYT1vhdf2ljeAIBkEbBT0rfbf/tjG5pbj5JH1vtvf+fZ5tYDAOCPgN0spytndZ0zwjuHfM6IgW1RLsXa/Eh9xT+8s3aa8vJHjfTejxxelej0sfoqAABoCEuTJuy97zfk/O+Zs1L7nP70PkG7ekZ5dZry4yXp2JPSxHFDy6M8Te8Oaez7AqtbsVwpyyJmX97bkPbLvgK0IUuTZkXbsMaOH35J5fvOBY3lFxqsAQCpIGC3mCiLpSxdU/m+1o/Pz30tnnIBAOmJPWCb2Ugze8HMXjKzl83sq3GXUXT3bx9a+k3bkqkHAKB5kuhh/1bSZc65WZIulPRpM7ukxjG5t2pd9LTN7u0OpbyhfA4AQHxiD9jO81b/2/b+R75nDESwLuaVPb9we7R0cd/1K+7PAQCIJpFz2GY2zMxelHRU0g+dc89X7V9uZt1mFuc9pXJl8crw/d9+0Hveudd//7ZnvOeg+2qXXLW68v11V9SuGwCg+RK9rMvMxkl6SNIXnXM/DUiT6953lMu6JGnGldKBQ1XH9v+cCRqyrnVHr7D9QXlHui0nl3XlSt7bkPbLvgK0YfqXdTnneiXtkPTpJMvJgx/fM3jbwhXhx3SELDUqSeM/Eb5/5drw/QCA1pHELPHO/p61zOwcSQsk/Wvc5WTOrPAVwqZMGrzt8RrLgp6ocTOP3lPh+zdsCd/va2ZPHQcBABrVlkCe75d0r5kNk/eD4AHn3KMJlJMtbRPrOiypGeNX31znge0TYq0HACCa2AO2c26fpN+LO1/E6/s70q4BAGAoWOmshUzuSLf8ORekWz4AIBg3/0jYoO+3xmzxeofAP/YhL+AfOCT94mB9edScIT57cFMxQzX78t6GtF/2FaANI80ST+IcNhoQdinWormN3S/78hul7c8FlwsAaF0E7Gabepd0MHzGV+8Oadx87/WR7dKkqqHy62+V7h3CNL65s6RdG6Un7h7YduCQd+23JB2Osjb5tL+KXiAAIHYMiSfM9/utMSwueb3sUq9363Zp2Zrw9EPx3a9Lyy4fXE4on+FwieG4PMh7G9J+2VeANow0JE7ATpjv93v6mLTP58LrKlHPZy+ZJ92wRJo/WzpxSvrJPum2TdLP9keoX5RgPbMn8HIu/rPIvry3Ie2XfQVoQ85ht6z2zroP3bbOC9BBxo+RZkyRrllYuX3Xi9Kln6+zUK69BoDU0cNOWOj3G3FovL1Neve5wdsj16GqF90+RzpztrGh8Pfqwa//wb/SAAAgAElEQVT7zMt7G9J+2VeANqSH3fJmu0hBuxSs673kq/y4sy9Ip5+PmFeNYA0AaB4WTknb9NoLeltXcIC9dbl04mmvt1x69O32tvsZdnHEYD39exESAQCahSHxhEX6fgN62dWB9ar50kN31V+XZWu8GeflAofFI/auGY7Lvry3Ie2XfQVoQ2aJt4LI3+/eUZJ7p2KTdUk9T0kTxlYmHT1Peqsveh06xkhv/qhy2zc2S7fc7ROwp2+ROpZGzpv/LLIv721I+2VfAdqQc9iZclF/BK7qbbcNk6ZfKb16qP6sj5+s7K3/8tHBPW1JnLMGgBbGOexWUxY0Xbf08M7GgrWf8xd7121X9K4J1gDQ0hgST1jd3+/p49K+Jlz/PPNoQ9eFMxyXfXlvQ9ov+wrQhpGGxOlht6r2Dq/XO219MvlP2+Dl30CwBgA0Dz3shMX6/Ua4ZrummIe++XWffXlvQ9ov+wrQhvSwc2e2G3jMOjFo92q/zvjMNyqPAwBkEj3shKX9/SaNX/fZl/c2pP2yrwBtSA8bAIC8IGADAJABBGwAADIg9ZXOZs+ere7uKPd5zKa8n1/K+7kliTbMOtov+/LehlHRwwYAIANS72EDANAsgXcoHIJItyhOAD1sAECu3XytF6jjCNbSQF6rroknv6gI2ACAXOoY4wXWO7+UTP5rb/Lyn9SRTP7VGBIHAOROXL3pKI7036446aFyetgAgFxpZrBuZrkEbABALvzm2fSCdYnrlv70U8nkTcAGAGSe65ZGDG88nxvvaDyPrbcn88OBc9gAgEx7Z3fjeZSff/7rB7znRoPub56VRv5hY3mUo4cNAMi0kSNqp+lcIN33A/99QZPFGp1EFkePvxwBGwCQWbV6wdblPXp6pc/+ZeNBuJRf6XHBnzRWv6EgYAMAMqlWMPzW/f7b6w3afse9vL/2cXEFbQI2ACBzOiMsVrLizuTrIUX7ATBhbOPlELABAJlzdHt8eQX1gOMczu55qvE8mCUOAMiUP7t24LVf77YUaF139OFv1y2d6pPGzJNOPiONHhW9Ppu+Eq0+K5dJ39wSPd9q9LABAJlyR//a4EHB+ODRgddzZw3eH9RzLgXpoGAddNz1S7znXx3231+q5/rV/vujImADAHJl2qKB17s2VgbasGHuD1/tPU+4LDhNdV7l789fPLR6DhUBGwCQGY2eV379aPC+V17zno+fDE4Tti+KRupPwAYA5MqiucH7pi4K3hdFWO978aWN5V0LARsAkEl9AUuSPrahufUoeWS9//Z3no0nfwI2ACATJk+ofH/OCG+I+ZyypUmjDDlvfqS+8h/eWTtNefmjRnrvR1YtUTpxXH3lE7ABAJlw+An/7X27pdPPe6+jXMZ1w1cHbztztvJ9T+/gNFdFmOVdKr93h/T2Lv80x56snY8fAjYAIPPahjV2/PBLKt93Lmgsv7Hva+x4PwRsAECuROllL11T+d658PSf+1o85TYikYBtZsPM7J/N7NEk8gcAoBH3D3Fp003bkqnHUCTVw/6SpJ8nlDcAoIBWrYueNunebiPlDeVzlIs9YJvZVElXSLon7rwBAMW1blW8+X3h9mjp4r7rV72fI4ke9jclfVnSfw9KYGbLzazbzLqPHTuWQBUAAEW3eGX4/m8/6D3v3Ou/f9sz3nPQfbVLqmePX3dF7brVI9aAbWaLJR11zu0JS+ec+45zrss519XZ2RlnFQAABTX9A5XvHwu4rKra/OX+2z8TsSdcfX32vT6XjcUh7h72XElXmtmrkrZKuszM/i7mMgAAGOTHPidiF64IP6YjZKlRSRr/ifD9K9eG749TrAHbOXeLc26qc+6DkpZK+pFz7rNxlgEAKKaJnwzfP2XS4G2P11gW9ESNm3n0ngrfv6GO+1uHrUcehuuwAQCZ8Oav6zsuqRnjV99c33H13vGrrb7DanPO7ZC0I6n8AQBI0/d3NLc8etgAgNyY3JFu+XMuSC5vAjYAIDNqDW8fHuIKZuU+9iFpwcXS70ytP4/nNofvb2R4PrEhcQAA0uC6gwPjormN3S/78hul7c8Fl5skAjYAIFNWr5fW3hSepneHNG6+9/rIdmlS1VD59bdK9w7hbhdzZ0m7NkpP3D2w7cAhacaV3usoPfsvNrhimrlatyhJWFdXl+vuTvhnSYrMLO0qJCrtfz/NQBtmG+2XfX5tGKU3a10D6bZul5atCU8/FN/9urTs8sHl1KpPgD3OuZqD5QTshPGfRfbRhtlG+2WfXxtOHCcdezLCsRHPGS+ZJ92wRJo/WzpxSvrJPum2TdLP9tc+NkqwnnBZ6OVckQI2Q+IAgMzp6a3/2G3rvAAdZPwYacYU6ZqFldt3vShd+vn6yqz32utyBGwAQCZFGYouTUBrb5PerZosNpQZ265b+viFA+W1z5HOnG14KHxICNgAgMyKev64FKzrDZ7lx519QTr9fLS84lxljeuwAQCZtvSW2mmsKzh43rpcOvG0F/hLj77d3nY/wy6OFoj/+Mu10wwFk84SxoSX7KMNs432y74obRjUy64OrFfNlx66q/66LFvjzTivp+wQTDoDABSDdUlv75JGjRy8r+cpacLYym2j50lv9UXPv2OM9OaPpC23eQ9J+sZm6Za7B6ddeot0/w+j5x0VARsAkAvnftx7ru7xtg2Tpl8pvXqo/ryPn6zsMf/y0cE9bSm5O4NJnMMGAORMedB03dLDOxsL1n7OX+xdt13+4yDJYC3RwwYA5JB1SeNHS8eflq67wnskpXNBY9eFR0UPGwCQSydOeYF75dpk8l9xp5d/M4K1RA8bAJBzG7Z4DymeO2olPfQdhB42AKAwStdjW9fA3bzKrV4/eNt5l1celxZ62ACAQvr1W/4BeN19za9LFPSwAQDIAAI2AAAZQMAGACADUl9L3MxyvRBu2t9v0vK+TrNEG2Yd7Zd9BWjDSGuJ08MGACADmCUOIDZZvsYVaHX0sAE05OZrB+4hHIdSXquuiSc/IC84h52wtL/fpHH+LPvqbcPS7QaTNvmPpKPH6z+e9su+ArQh98MGkIy4etNRHOm/hSFD5Sg6hsQBDEkzg3UrlAu0CgI2gEh+82z6QdN1S3/6qXTrAKSFgA2gJtctjRjeeD433tF4HltvT/+HA5AGJp0lLO3vN2lMeMm+Wm34zm5p5IgGy/A5/9xo0P3tu9LIP6ydrujtlwcFaEMWTgHQuCjBunOBdN8P/PcFTRZrdBJZHD1+IEvoYScs7e83afy6z76wNqzVC47Scw4LzLXSfnSG9NMHhl6HijIK3H55UYA2pIcNoH61gvW37vffXm/P2e+4l/fXPo7z2SgKAjaAQTo7aqdZcWfy9ZCi/QCYMDb5egBpI2ADGOTo9vjyCuoBx9kz7nkqvryAVsVKZwAq/Nm1A6/DzlG77ujD365bOtUnjZknnXxGGj0qen02fSVafVYuk765JXq+QNbQwwZQ4Y4vec9Bwfjg0YHXc2cN3h/Ucy4F6aBgHXTc9Uu8518d9t9fquf61f77gbwgYAMYkmmLBl7v2lgZaMOGuT98tfc84bLgNNV5lb8/f/HQ6gnkDQEbwHsaPa/8+tHgfa+85j0fPxmcJmxfFMwYR54RsAEMyaK5wfumLgreF0VY73vxpY3lDWQdARuAr77d/tsf29DcepQ8st5/+zvPNrceQFoI2AAkSZMnVL4/Z4Q3xHxO2dKkUYacNz9SX/kP76ydprz8USO99yOrliidOK6+8oFWx9KkCUv7+00ayyJmX6kNw4LxmbNS+xwFpqueUV6dpvx4STr25ODAWiuP8jS9O6Sx7wuub3leRWm/PCtAG7I0KYB4tA1r7Pjhl1S+71zQWH5hwRrIKwI2gCGJsljK0jWV72t1kD73tXjKBfIskYBtZq+a2b+Y2YtmxoUWQMHcP8SlTTdtS6YeQJ4k2cP+hHPuwijj8gDSt2pd9LTN7u0OpbyhfA4gSxgSByBJWrcq3vy+cHu0dHHf9SvuzwG0iqQCtpO03cz2mNny6p1mttzMuhkuB7Jr8crw/d9+0Hveudd//7ZnvOeg+2qXXFW1Rvh1V9SuG5BHiVzWZWYfcM4dMrNJkn4o6YvOuWcC0uZ6vn4BLkdIuwqJK0ob1rrGesaV0oFDldtKxwQNWde6o1fY/qC8o1wLzmVd+VKANkzvsi7n3KH+56OSHpJ0cRLlAGieH98zeNvCFeHHdIQsNSpJ4z8Rvn/l2vD9QJHEHrDN7FwzG116LemPJP007nIAxGviJ8P3T5k0eNvjNZYFPVHjZh69p8L3b6jj/tZh65EDWdaWQJ6TJT3UP0zTJum7zrnHEygHQIze/HV9xyU1Y/zqm+s7rtE7fgGtKvaA7ZzbL8nntvYAEN33d6RdA6C1cFkXgMgmd6Rb/pwL0i0fSBM3/0hY2t9v0pihmn3VbVhrFna9Q+Af+5AX8A8ckn5xsL486qlb0dovjwrQhpFmiSdxDhtAjoVdirVobmP3y778Rmn7c8HlAkVGwAZQYfV6ae1N4Wl6d0jj5nuvj2yXJlUNlV9/q3Tvo9HLnDtL2rVReuLugW0HDnnXfkvS4Qhrk38x5hXTgFbDkHjC0v5+k8ZwXPb5tWHUxUlK6bZul5atCU8/FN/9urTs8sHl1KqPnyK2X94UoA0jDYkTsBOW9vebNP6zyD6/Npw4Tjr2ZIRjI57PXjJPumGJNH+2dOKU9JN90m2bpJ/tr31slGA94bLgy7mK2H55U4A25Bw2gPr09NZ/7LZ1XoAOMn6MNGOKdM3Cyu27XpQu/Xx9ZXLtNYqAHnbC0v5+k8av++wLa8OoQ9HtbdK7zw3eHlV1Oe1zpDNnGxsKfy/vArdfXhSgDelhA2hM1PPHpWBd7yVf5cedfUE6/Xy0vJp9X24gTSycAiDU0ltqp7Gu4OB563LpxNNe4C89+nZ72/0MuzhaIP7jL9dOA+QJQ+IJS/v7TRrDcdkXpQ2DetnVgfWq+dJDd9Vfl2VrvBnn9ZQdhPbLvgK0IbPEW0Ha32/S+M8i+6K24du7pFEjq47tknqekiaMrdw+ep70Vl/0OnSMkd78UeW2b2yWbrl7cMBeeot0/w+j5037ZV8B2pBz2ADic+7HvefqANo2TJp+pfTqofrzPn6yssf8y0cH97Qlzlmj2DiHDWBIyoOm65Ye3tlYsPZz/mLvuu3yHwcEaxQdQ+IJS/v7TRrDcdlXbxuOHy0dfzrmyvjoXNDYdeG0X/YVoA0jDYnTwwZQlxOnvF7vyrXJ5L/izv5z5A0EayBP6GEnLO3vN2n8us++ONswjjtqxT30TftlXwHakB42gOYqXY9tXQN38yq3ev3gbeddXnkcAH/0sBOW9vebNH7dZ1/e25D2y74CtCE9bAAA8oKADQBABhCwAQDIgNRXOps9e7a6u2OYWtqi8n5+Ke/nliTaMOtov+zLextGRQ8bAIAMIGADAJABqQ+JAwBayJ4Yhp9n53+YPg30sAGg6I7c6QXqOIK1NJDXkYTWrS0oAjYAFNXpN73AevDLyeR/8GYv/9NHksm/YBgSB4Aiiqs3HcW+87xnhsobQg8bAIqmmcG6FcrNCQI2ABTF3hHpB809Jh3fmm4dMoqADQBFsMck927D2dx4Rwx1ObAs/R8OGcQ5bADIu70jG86i/Nanf/2A99zw/c/3jpAu+m2DmRQHPWwAyDtXOyh2LpDu+4H/vqD7lDd8//IYevxFQsAGgDyrMfRsXd6jp1f67F82HoRL+ZUeF/xJY/XDAAI2AORVjWD4rfv9t9cbtP2Oe3l/hAMJ2pEQsAEgj84crZlkxZ1NqIci/gA405N4PbKOgA0AefTS5NiyCppc1vCks3IvdcaYWT4xSxwA8uaNgWuv/Hq3pUDruqMPf7tu6VSfNGaedPIZafSo6NXZ9JWB12H10eH10nk3Rc+4YOhhA0DeHPpzScHB+GDZaPncWYP3B/WcS0E6KFgHHXf9Eu/5V4f9979Xz9dX+SeAJAI2ABTOtEUDr3dtrAy0YcPcH77ae55wWXCa6rzK35+/eGj1RCUCNgDkSYMzrl8Pmav2ymve8/GTwWnC9kXCjPFABGwAKJhFc4P3TV0UvC+KsN734ksby7voCNgAkFN9u/23P7ahufUoeWS9//Z3nm1uPbKKgA0AeXG6clbXOSO8c8jnjBjYFuVSrM2P1Ff8wztrpykvf9RI7/3I4VWJTh+rrwI5R8AGgLzY937fzX27pdPPe6+jXMZ1w1cHbztztvJ9T+/gNFetrp13qfzeHdLbuwIS7ZtUO6MCImADQAG0DWvs+OGXVL7vXNBYfmPf19jxRZRIwDazcWb292b2r2b2czP7gyTKAQAMXZRe9tI1le+dC0//ua/FUy6CJdXD3iDpcefc/yhplqSfJ1QOACAB928fWvpN25KpBwbEHrDNbIykeZI2SpJz7l3nnM/ZDgBAnFati5622b3doZQ3lM9RJEn0sGdIOiZpk5n9s5ndY2bnJlAOAKDMuphX9vzC7dHSxX3Xr7g/R14kEbDbJF0k6W+cc78n6W1Jf1GewMyWm1m3mXUfO8b0fQBIw+KV4fu//aD3vHOv//5tz3jPQffVLqmePX7dFbXrhsGSCNgHJR10zvVfRKC/lxfA3+Oc+45zrss519XZyS3VAKAZpn+g8v1jQZdVVZm/3H/7ZyL2hKuvz77X57Ix1BZ7wHbOHZb0mpl9pH/TJyX9LO5yAABD8+N7Bm9buCL8mI6QpUYlafwnwvevXBu+H9EldT/sL0q6z8yGS9ov6YaEygEAlMw6Jr0UPGo5xWc9ksdrLAt6osbNPHpPhe/fsCV8v6+ZPXUclH+JBGzn3IuSuOIOAJqpbWJdhyU1Y/zqm+s8sH1CrPXIC1Y6AwAk4vs70q5BvhCwAaBAJnekW/6cC9ItP8sI2ACQJ7PD1xA9PMQVzMp97EPSgoul35lafx7Pba6RoEb9iyypSWcAgBbluoPPWy+a29j9si+/Udr+XHC5qB8BGwDyZupd0sHwGV+9O6Rx873XR7ZLk6qGyq+/Vbr30ehFzp0l7dooPXH3wLYDh6QZV3qvI/Xsp/1V9AILiCFxAMibybVvTF26vaXr9oL11u1er7v0GEqwlqTdL1Uev+UJb6GWUq860rnzSV8cWqEFY67WPdMS1tXV5bq78ztOYmZpVyFRaf/7aQbaMNsK236nj0n7fC68rhL1kq4l86QblkjzZ0snTkk/2Sfdtkn62f4IdYzyX/zMnsDLufLehpL2OOdqtgRD4gCQR+31L/u8bZ0XoIOMHyPNmCJds7By+64XpUs/X2ehXHtdEwEbAPJqtpP2hPdOSxPQ2tukd6smiw1lQRXXLX38woHedPsc6czZiL1rZoZHQsAGgDyLELSlgWBd76pn5cedfUE6/XzEvAjWkTHpDADybnrtBb1Lk8X83LpcOvG011suPfp2e9v9DLs4YrCe/r0IiVDCpLOE5X2yRNr/fpqBNsw22q9fQC+7OrBeNV966K7667NsjTfjvFzgsHjE3nXe21BMOgMAvGe2k/aOktw7g3b1PCVNGFu5bfQ86a2+6Nl3jJHe/JG05TbvIUnf2CzdcrdP4ulbpI6l0TOHJAI2ABTHRf0RuKq33TZMmn6l9Oqh+rM+frKyt/7LRwf3tCVxzroBnMMGgKIpC5quW3p4Z2PB2s/5i73rtiuGwwnWDaGHDQBFNNtJp49L+ybouiuk665IsKyZRxu6LhweetgAUFTtHV7gnrY+mfynbfDyJ1jHgh42ABTdpJXeQ4p0zXZNDH0ngh42AGDAbDfwmHVi0O7Vfp3xmW9UHodE0MMGAPhrGzcoAK/9u5TqAnrYAABkAQEbAIAMIGADAJABqa8lbma5nqGQ9vebtAKs8UsbZhztl30FaMNIa4nTwwYAIANyM0s80k3Sa6j3PrAAACQt0z3sm68duDdrHEp5rbomnvwAAIhLJs9hl27jlrTJfyQdPd5YHml/v0nj/Fn25b0Nab/sK0Ab5vN+2HH1pqM40n9rOIbKAQBpy9SQeDODdSuUCwBASSYC9m+eTT9oum7pTz+Vbh0AAMXV8gHbdUsjhjeez413NJ7H1tvT/+EAACimlp509s5uaeSIBvP3Of/caND97bvSyD+Mljbt7zdpTHjJvry3Ie2XfQVow+wvnBIlWHcukO77gf++oMlijU4ii6PHDwDAULRsD7tWLzhKzzksMNdK+9EZ0k8fGHodBpWT/1+GaVchcbRhttF+2VeANsxuD7tWsP7W/f7b6+05+x338v7ax3E+GwDQLC0XsDs7aqdZcWfy9ZCi/QCYMDb5egAA0HIB++j2+PIK6gHH2TPueSq+vAAACNJSK5392bUDr8POUbvu6MPfrls61SeNmSedfEYaPSp6fTZ9JVp9Vi6Tvrkler4AAAxVS/Ww7/iS9xwUjA8eHXg9d9bg/UE951KQDgrWQcddv8R7/tVh//2leq5f7b8fAIC4tFTArmXaooHXuzZWBtqwYe4PX+09T7gsOE11XuXvz188tHoCABC3lgnYjZ5Xfv1o8L5XXvOej58MThO2LwpmjAMAktQyATuKRXOD901dFLwvirDe9+JLG8sbAIBGtWTA7tvtv/2xDc2tR8kj6/23v/Nsc+sBACiulgjYkydUvj9nhDfEfE7Z0qRRhpw3P1Jf+Q/vrJ2mvPxRI733I6uWKJ04rr7yAQCopSWWJg0LxmfOSu1zvNd+6apnlFenKT9eko49OTiw1sqjPE3vDmns+4LrOyiv/C+pl3YVEkcbZhvtl30FaMPsLk1arm1YY8cPv6TyfeeCxvILC9YAACSl5QN2uSiLpSxdU/m+1g+zz30tnnIBAEhS7AHbzD5iZi+WPU6a2cq4ywly/xCXNt20LZl6AAAQp9gDtnPu35xzFzrnLpQ0W1KfpIfCjlm1Lnr+ze7tDqW8oXwOAACGIukh8U9K+oVz7pdhidatirfQL9weLV3cd/2K+3MAAFCSdMBeKmnQbTHMbLmZdZtZXeuDLa4xwP7tB73nnXv99297xnsOuq92yVVVa4Rfd0XtugEAkITELusys+GSDkn6qHPuSEi60Mu6JGnGldKBQ5XbSscEDVnXuqNX2P6gvKNcC85lXflDG2Yb7Zd9BWjD1C/rWihpb1iwjurH9/hkviL8mI6QpUYlafwnwvevXBu+HwCAZkoyYC+Tz3C4n4mfDN8/ZdLgbY/XWBb0RI2befSeCt+/oY77W4etRw4AQCMSCdhmNkrSpyT9Q5T0b/66znISmjF+9c31HdfoHb8AAAjSlkSmzrk+SRNqJmxR39+Rdg0AAKiUmZXOJnekW/6cC9ItHwBQbC1x84/S61qzsOsdAv/Yh7yAf+CQ9IuD9eVRb93S/n6TxgzV7Mt7G9J+2VeANow0SzyRIfGkhF2KtWhuY/fLvvxGaftzweUCAJCmlgrYq9dLa28KT9O7Qxo333t9ZLs0qWqo/PpbpXsfjV7m3FnSro3SE3cPbDtwyLv2W5IOR1ib/Isxr5gGAEC1lhoSl6IvTlJKt3W7tGxNePqh+O7XpWWXDy6nVn2CpP39Jo3huOzLexvSftlXgDaMNCTecgF74jjp2JMRjot4PnvJPOmGJdL82dKJU9JP9km3bZJ+tr/2sVGC9YTLwi/nSvv7TRr/WWRf3tuQ9su+ArRhNs9h9/TWf+y2dV6ADjJ+jDRjinTNwsrtu16ULv18fWVy7TUAoBlaroddEnUour1Neve5wdujqi6nfY505mzjQ+Hv5Z//X4ZpVyFxtGG20X7ZV4A2zGYPuyTq+eNSsK73kq/y486+IJ1+Plpezb4vNwCg2Fp64ZSlt9ROY13BwfPW5dKJp73AX3r07fa2+xl2cbRA/Mdfrp0GAIA4teyQeElQL7s6sF41X3rorvrrsWyNN+O8nrLDpP39Jo3huOzLexvSftlXgDbM5ixxP2/vkkaNrDquS+p5SpowtnL76HnSW33Ry+8YI735o8pt39gs3XL34IC99Bbp/h9Gz1sqxD+0tKuQONow22i/7CtAG2b7HHa5cz/uPVcH0LZh0vQrpVcP1Z/38ZOVPeZfPjq4py1xzhoAkK6WPoddrTxoum7p4Z2NBWs/5y/2rtsu/3FAsAYApC0TQ+LVxo+Wjj+dRG0qdS5o7LpwqRBDOWlXIXG0YbbRftlXgDaMNCSeqR52yYlTXq935dpk8l9xZ/858gaDNQAAcclkD9tPHHfUSmLoO+3vN2n8us++vLch7Zd9BWjD/Paw/ZSux7augbt5lVu9fvC28y6vPA4AgFaVmx52q0r7+00av+6zL+9tSPtlXwHasFg9bAAA8oyADQBABhCwAQDIgFZY6axH0i+bWN7E/jKbIqXzS039jCnIexvSfjGi/WLX9M9XgDY8P0qi1CedNZuZdUc5uZ9lef+MfL5s4/NlW94/n9S6n5EhcQAAMoCADQBABhQxYH8n7Qo0Qd4/I58v2/h82Zb3zye16Gcs3DlsAACyqIg9bAAAMoeADQBABhQqYJvZp83s38zsFTP7i7TrEycz+1szO2pmP027Lkkws2lm9rSZ/dzMXjazL6Vdp7iZ2Ugze8HMXur/jF9Nu05xM7NhZvbPZvZo2nVJgpm9amb/YmYvmlkM9xBsLWY2zsz+3sz+tf9v8Q/SrlNczOwj/e1Wepw0s5Vp16tcYc5hm9kwSf+fpE9JOijpnyQtc879LNWKxcTM5kl6S9J/dc5dkHZ94mZm75f0fufcXjMbLWmPpKvy0n6SZN7qEOc6594ys3ZJuyR9yTn3XMpVi42ZrZLUJWmMc25x2vWJm5m9KqnLOZfLhVPM7F5JP3bO3WNmwyWNcs71pl2vuPXHi9clzXHONXNhr1BF6mFfLOkV59x+59y7krZK+kzKdYqNc+4ZScfTrkdSnHNvOOf29r8+JennkqakW6t4Oc9b/W/b+x+5+UVtZlMlXSHpnn58C9IAAAJTSURBVLTrgqEzszGS5knaKEnOuXfzGKz7fVLSL1opWEvFCthTJL1W9v6gcvYfflGY2Qcl/Z6k59OtSfz6h4xflHRU0g+dc3n6jN+U9GVJ/z3tiiTISdpuZnvMbHnalYnZDEnHJG3qP61xj5mdm3alErJU0pa0K1GtSAHbbzHa3PReisLM3ifpQUkrnXMn065P3JxzZ51zF0qaKuliM8vF6Q0zWyzpqHNuT9p1Sdhc59xFkhZK+o/9p6ryok3SRZL+xjn3e5LelpSruUCS1D/Uf6Wk76Vdl2pFCtgHJU0rez9V0qGU6oI69J/XfVDSfc65f0i7PknqH2rcIenTKVclLnMlXdl/jnerpMvM7O/SrVL8nHOH+p+PSnpI3qm4vDgo6WDZqM/fywvgebNQ0l7n3JG0K1KtSAH7nyR92Mym9/+CWippW8p1QkT9E7I2Svq5c25d2vVJgpl1mtm4/tfnSFog6V/TrVU8nHO3OOemOuc+KO9v70fOuc+mXK1Ymdm5/RMi1T9U/EeScnPVhnPusKTXzOwj/Zs+KSk3kz7LLFMLDodLrXF7zaZwzp0xsxslPSFpmKS/dc69nHK1YmNmWyTNlzTRzA5K+opzbmO6tYrVXEnXSvqX/nO8krTGOfePKdYpbu+XdG//DNV/J+kB51wuL3/KqcmSHuq/FWSbpO865x5Pt0qx+6Kk+/o7Pfsl3ZByfWJlZqPkXUn0H9Kui5/CXNYFAECWFWlIHACAzCJgAwCQAQRsAAAygIANAEAGELABAMgAAjYAABlAwAYAIAP+fzFY3dTllVswAAAAAElFTkSuQmCC", "text/plain": [ "" ] @@ -5374,7 +5377,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FNWd7/vPd9gbEMOvDRtMgGtg\nkifnTow4skecIXKJIWNAMHru3Bm4Ro/m5nJu7jEEwcmMPM88MXlONFcFQuLcycmRAc8ZA5pxjKgT\nJf4AA0adDaNMTGbuY8BERH5sgYBiInDW/aN2u7t7V1VXd1d1dVW9X8/TT3dXrVprdS82316rVq0y\n55wAAEB7+520KwAAAGojYAMAkAEEbAAAMoCADQBABhCwAQDIAAI2AAAZQMAGACADCNgAAGQAARto\nM2b2QTP7RzM7amYHzOwuM+sIST/GzP6mP+1JM/sXM/sPrawzgOQRsIH28/9KOiTp/ZIukPS/SPq/\n/RKa2VBJT0g6V9IfShot6c8l3W5mS1tSWwAtQcAG2s9USfc7537jnDsg6TFJHw1Ie42k/0nS/+ac\n2+ucO+Wce0zSUkn/2cxGSpKZOTP7UOkgM9tgZv+57P0CM3vRzI6Z2bNmdn7Zvg+Y2QNmdtjM9pb/\nEDCzW8zsfjP7b2Z2wsxeNrOesv1/YWav9+/7NzP7ZDxfEVA8BGyg/ayVtMjMRpjZJEnz5AVtP5+S\n9EPn3NtV2x+QNELSxbUKM7MLJf2tpP8oaZyk/yJps5kNM7PfkfSwpJckTZL0SUnLzOyysiyukLRJ\n0hhJmyXd1Z/vRyTdIOkPnHMjJV0m6dVa9QHgj4ANtJ9t8nrUxyXtk9Qr6QcBacdLeqN6o3PutKQ+\nSd0Ryvs/Jf0X59zzzrkzzrl7JP1WXrD/A0ndzrmvOefedc7tkfRfJS0qO367c+4fnXNnJP13SdP7\nt5+RNEzS75lZp3PuVefcLyLUB4APAjbQRvp7tI9L+gdJZ8sLyGMl/T8Bh/TJO9ddnU9H/7GHIxR7\nrqQV/cPhx8zsmKQpkj7Qv+8DVftWSppYdvyBstcnJQ03sw7n3CuSlkm6RdIhM9tkZh+IUB8APgjY\nQHvpkhcs73LO/dY596ak9ZLmB6R/QtI8Mzu7avv/KumUpBf635+UN0Reck7Z69ckfd05N6bsMcI5\nt7F/396qfSOdc0H1qeCc+55z7uPyAr9T8A8PADUQsIE24pzrk7RX0hfMrMPMxkj6D/LOIfv57/KG\nzb/ffzlYZ//55W9Jut059+v+dC9K+t/NbIiZfVrezPOS/yrp/zKzmeY528wu75+w9oKk4/2Tx87q\nP/48M/uDWp/FzD5iZpea2TBJv5H0jrxhcgANIGAD7effS/q0vOHsVySdlnSjX0Ln3G8lzZXXE35e\nXlB8TNI3JX21LOmXJC2UdEzS1So7J+6c65V3HvsuSUf7y7yuf9+Z/uMukPdDok/S3fIuH6tlmKRv\n9B9zQNIEecPpABpgzrm06wAgJmbWKemHkl6XdJ3jDxzIDXrYQI44507JO3/9C0kfSbk6AGJEDxsA\ngAyghw0AQAYE3lCgVcaPH+8++MEPpl2NxOzcuTPtKiRqxowZaVchcbRhttF+2Zf3NpTU55yruchR\n6kPiPT09rre3t/mMdlrzecyI/7swi6FebSztfz+tQBtmG+2XfXlvQ0k7nXM9tRJle0j84B1eoI4j\nWEsDeR1cFU9+AADEJJsB+9SbXmDd9+Vk8t93k5f/qYPJ5A8AQJ1SP4ddt7h601Hs7l+9MYGhcgAA\n6pGtHnYrg3U7lAsAQL9sBOxdw9IPmjtNOrIp3ToAAAqr/QP2TpPcu01nc8PtMdRl7+L0fzgAAAqp\nvc9h7xredBZWNlH+r+/3nl2zV5HtGiZd+NsmMwEAILr27mG72kGxe6507w/991nAVW1B2yOLoccP\nAEA92jdg1xh6th7v0XdM+uxfNR+ES/mVHuf9aXP1AwAgTu0ZsGsEw2/f57+90aDtd9zLeyIcSNAG\nALRI+wXs04dqJll6RwvqoYg/AE73JV4PAADaL2C/NDG2rIImlzU96azcSzXXawcAoGntNUv8jYFr\nr/x6t6VA63qjD3+7XunESWnUbOn4M9LIEdGrs/4rA6/D6qMDa6RzboyeMQAAdWqvHvb+v5AUHIz3\nlY2Wz5o+eH9Qz7kUpIOCddBx1y30nn91wH//e/V8fbl/AgAAYtJeAbuGKfMHXm9fVxlow4a5P3yV\n9zzu0uA01XmVvz93QX31BAAgbu0TsJuccf16yFy1V17zno8cD04Tti8SZowDABLUPgE7gvmzgvdN\nnh+8L4qw3veCS5rLGwCAZrVlwD65w3/7o2tbW4+Sh9f4b3/n2dbWAwBQXO0RsE9Vzuo6a5h3Dvms\nYQPbolyKteHhxop/aFvtNOXljxjuvR8+tCrRqcONVQAAgBraI2Dvfr/v5pM7pFPPe6+jXMZ1/VcH\nbzt9pvJ937HBaa5cUTvvUvnHtkpvbw9ItHtC7YwAAGhAewTsEB1Dmjt+6MWV77vnNpff6Pc1dzwA\nAI1o+4BdLkove9HKyvfOhaf/3NfiKRcAgCRlKmBHcd+W+tKv35xMPQAAiFMiAdvMPm1m/2Zmr5jZ\nX9ZKv3x1HXm3uLdbT3n1fA4AAOoRe8A2syGS/lrSPEm/J2mxmf1e2DGrY17Z8wu3RUsX912/4v4c\nAACUJNHDvkjSK865Pc65dyVtkvSZOAtYsCx8/3ce8J637fLfv/kZ7znovtol1bPHr728dt0AAEhC\nEgF7kqTXyt7v69/2HjNbYma9ZtZ7+HDta5enfqDy/aNBl1VVmbPEf/tnIvaEq6/PvsfnsjEAAFoh\niYDtt6h2xVxt59x3nXM9zrme7u7a95P+8d2Dt81bGn5MV8hSo5I09hPh+5etCt8PAEArJRGw90ma\nUvZ+sqT9oUdMD+9lT/JZj+SxGsuCHq1xM49jJ8L3r90Yvt/X+X0NHAQAQG1JBOx/kvRhM5tqZkMl\nLZIUfvFUx/iGCkpqxvhVNzV4YOe4WOsBAEBJR9wZOudOm9kNkh6XNETS3zrnXo67nCT9YGvaNQAA\noFLsAVuSnHP/KOkf48xzYpd08EicOdZn5nnplQ0AQPusdDYjfA3RA3WuYFbuYx+S5l4k/e7kxvN4\nbkONBDXqDwBAMxLpYSfF9Qaft54/q7n7ZV92g7TlueByAQBIU3sF7Ml3SvvCZ3wd2yqNmeO9PrhF\nmtBVuf+6W6R7Hole5Kzp0vZ10uN3DWzbu1+adoX3OlLPfsq3ohcIAEAD2mdIXJIm1r4xden2lq7X\nC9abtni97tKjnmAtSTteqjx+4+PeQi2lXvXErvDjJUkTvlhfoQAA1MlcrftPJqynp8f19paNOZ86\nLO32ufC6StRLuhbOlq5fKM2ZIR09If1kt3Treulne2ofG2ko/Py+0Mu5zPzWkcmPtP/9tAJtmG20\nX/blvQ0l7XTO1Yxq7TUkLkmdtVc+C7J5tRegg4wdJU2bJF09r3L79helSz7fYKFcew0AaIH2C9iS\nN+N6Z/gvqtIEtM4O6d2qyWL1LKjieqWPXzDQm+6cKZ0+E7F3zcxwAECLtGfAliIFbWkgWDe66ln5\ncWdekE49HzEvgjUAoIXaa9JZtam1F/QuTRbzc8sS6ejTXm+59Di5w9vuZ8hFEYP11O9HSAQAQHza\nb9JZtYBednVgvXKO9OCdjddj8Upvxnm5wGHxOnrXeZ8skfa/n1agDbON9su+vLehMjvprNoMJ+0a\nIbl3Bu3qe1IaN7py28jZ0lsno2ffNUp68ylp463eQ5K+sUG6+S6fxFM3Sl2LomcOAEBM2j9gS9KF\n/RG4qrfdMUSaeoX0avjNO0MdOV7ZW//lI4N72pI4Zw0ASFV7n8OuVhY0Xa/00LbmgrWfcxd4121X\nDIcTrAEAKctGD7vcDCedOiLtHqdrL5euvTzBss4/1NR14QAAxCVbPeySzi4vcE9Zk0z+U9Z6+ROs\nAQBtIns97HITlnkPKdI12zUx9A0AaFPZ7GH7meEGHtOPDtq9wq8zfv4blccBANCmst3DDtIxZlAA\nXvV3KdUFAIAY5KeHDQBAjhGwAQDIAAI2AAAZQMAGACADUr/5h5nlenp22t9v0gqwKD9tmHG0X/YV\noA0j3fyDHjYAwNeYkZW3J3a90vKrB287Z1zaNS0GetgJS/v7TRq/7rMv721I+9Un8LbCdai+/XGz\nCtCG9LABALXddM1AbzkO5b1xxIcedsLS/n6TlvfemUQbZh3tF6xrlPTmUzFWJsDEP5YOHWn8+AK0\nYaQedj5XOgMAhIqrNx3FwS3ec9xD5UXDkDgAFEwrg3U7lJsXBGwAKIjfPJt+0HS90p99Kt06ZBUB\nGwAKwPVKw4Y2n88Ntzefx6bb0v/hkEVMOktY2t9v0vI+YUmiDbOO9pPe2SENH9ZkOT7nn5sNur99\nVxr+R7XTFaANuawLABAtWHfPle79of++oMlizU4ii6PHXyT0sBOW9vebtLz3ziTaMOuK3n61esFR\nes5hgblW2o9Ok356f/11qCgj/21IDxsAiqxWsP72ff7bG+05+x338p7ax3E+OxoCNgDkUHdX7TRL\n70i+HlK0HwDjRidfj6wjYANADh3aEl9eQT3gOHvGfU/Gl1desdIZAOTMn18z8DrsHLXrjT787Xql\nEyelUbOl489II0dEr8/6r0Srz7LF0jc3Rs+3aOhhA0DO3P4l7zkoGO87NPB61vTB+4N6zqUgHRSs\ng467bqH3/KsD/vtL9Vyzwn8/PARsACiYKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31RCUC\nNgDkSLPnlV8/FLzvlde85yPHg9OE7YuCGePBCNgAUDDzZwXvmzw/eF8UYb3vBZc0l3fREbABIKdO\n7vDf/uja1taj5OE1/tvfeba19cgqAjYA5MTEcZXvzxrmDTGfVbY0aZQh5w0PN1b+Q9tqpykvf8Rw\n7/3wqiVKx49prPy8Y2nShKX9/SYt78taSrRh1hWp/cKC8ekzUufM4HTVM8qr05QfL0mHnxgcWGvl\nUZ7m2FZp9PuC61ueVwHakKVJAQCejiHNHT/04sr33XObyy8sWMMfARsACibKYimLVla+r9XJ/dzX\n4ikXwWIP2Gb2t2Z2yMx+GnfeAIDWuK/OpU3Xb06mHhiQRA97g6RPJ5AvACDE8tXR07a6t1tPefV8\njiKJPWA7556RdCTufAEA4VYvjze/L9wWLV3cd/2K+3PkBeewAaCgFiwL3/+dB7znbbv8929+xnsO\nuq92yZVVa4Rfe3ntumGwVAK2mS0xs14zYxE6AGiRqR+ofP/o9mjHzVniv/0zEXvC1ddn3/PVaMeh\nUioB2zn3XedcT5TrzgAA8fjx3YO3zVsafkxXyFKjkjT2E+H7l60K34/oGBIHgJwY/8nw/ZMmDN72\nWI1lQY/WuJnHsRPh+9c2cH/rsPXIiyyJy7o2SvqJpI+Y2T4z+z/iLgMAMNibv27suKRmjF91U2PH\nNXvHr7zqiDtD59ziuPMEAGTPD7amXYN8YUgcAApkYle65c88L93ys4ybfyQs7e83aXm/cYREG2Zd\nEduv1h25Gh0C/9iHvIC/d7/0i32N5dFI3QrQhpFu/hH7kDgAoL253uCgPX9Wc/fLvuwGactzweWi\ncQRsAMiZFWukVTeGpzm2VRozx3t9cIs0oWqo/LpbpHseiV7mrOnS9nXS43cNbNu7X5p2hff6QIS1\nyb8Y84ppecOQeMLS/n6TlvfhVIk2zLqitl+U3qz1DKTbtEVavDI8fT2+93Vp8WWDy6lVHz8FaMNI\nQ+IE7ISl/f0mLe//2Uu0YdYVtf3Gj5EOPxHh+IjnsxfOlq5fKM2ZIR09If1kt3Treulne2ofGyVY\nj7s0+HKuArQh57ABoKj6jjV+7ObVXoAOMnaUNG2SdPW8yu3bX5Qu+XxjZXLtdW30sBOW9vebtLz3\nziTaMOuK3n5Rh6I7O6R3nxu8ParqcjpnSqfPNDcU/l7e+W9DetgAUHRRzx+XgnWjl3yVH3fmBenU\n89HyavV9ubOMhVMAIOcW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00GMCQeMLS/n6T\nlvfhVIk2zDrazxPUy64OrFfOkR68s/H6LF7pzThvpOwgBWhDZom3g7S/36Tl/T97iTbMOtpvwNvb\npRHDq47vkfqelMaNrtw+crb01sno9egaJb35VOW2b2yQbr5rcMBedLN034+i512ANuQcNgBgwNkf\n956rA2jHEGnqFdKr+xvP+8jxyh7zLx8Z3NOWOGfdDM5hA0DBlAdN1ys9tK25YO3n3AXeddvlPw4I\n1s1hSDxhaX+/Scv7cKpEG2Yd7Rds7EjpyNMxViZA99zmrgsvQBtGGhKnhw0ABXX0hNfrXbYqmfyX\n3tF/jryJYI0B9LATlvb3m7S8984k2jDraL/6xHFHrbiHvgvQhvSwAQD1KV2PbT0Dd/Mqt2LN4G3n\nXFZ5HJJBDzthaX+/Sct770yiDbOO9su+ArQhPWwAAPKCgA0AQAYQsAEAyIDUVzqbMWOGentjmJbY\npvJ+finv55Yk2jDraL/sy3sbRkUPGwCADEi9hw0gR3bG0BOakf8eI9AIetgAmnPwDi9QxxGspYG8\nDia0/BaQUQRsAI059aYXWPd9OZn8993k5X/qYDL5AxnDkDiA+sXVm45i9zneM0PlKDh62ADq08pg\n3Q7lAm2CgA0gml3D0g+aO006sindOgApIWADqG2nSe7dprO54fYY6rJ3cfo/HIAUcA4bQLhdw5vO\novwOTn99v/fc9G0cdw2TLvxtk5kA2UEPG0A4Vzsods+V7v2h/76g2y02fRvGGHr8QJYQsAEEqzH0\nXLr/cd8x6bN/1XwQLr+nsvVI5/1pc/UD8oSADcBfjWD47fv8tzcatP2Oe3lPhAMJ2igIAjaAwU4f\nqplk6R0tqIci/gA43Zd4PYC0EbABDPbSxNiyCppc1vSks3IvdceYGdCemCUOoNIbA9de+fVuS4HW\n9UYf/na90omT0qjZ0vFnpJEjoldn/VcGXofVRwfWSOfcGD1jIGPoYQOotP8vJAUH431lo+Wzpg/e\nH9RzLgXpoGAddNx1C73nXx3w3/9ePV9f7p8AyAkCNoC6TJk/8Hr7uspAGzbM/eGrvOdxlwanqc6r\n/P25C+qrJ5A3BGwAA5qccf16yFy1V17zno8cD04Tti8SZowjxwjYAOoyf1bwvsnzg/dFEdb7XnBJ\nc3kDWUfABuDr5A7/7Y+ubW09Sh5e47/9nWdbWw8gLQRsAJ5TlbO6zhrmnUM+a9jAtiiXYm14uLHi\nH9pWO015+SOGe++HD61KdOpwYxUA2hwBG4Bn9/t9N5/cIZ163nsd5TKu6786eNvpM5Xv+44NTnPl\nitp5l8o/tlV6e3tAot0TamcEZBABG0BNHUOaO37oxZXvu+c2l9/o9zV3PJBFBGwAdYnSy160svK9\nc+HpP/e1eMoF8oyADSB2922pL/36zcnUA8iT2AO2mU0xs6fN7Odm9rKZfSnuMgDEb/nq6Glb3dut\np7x6PgeQJUn0sE9LWuGc+58lXSzpP5nZ7yVQDoAYrY55Zc8v3BYtXdx3/Yr7cwDtIvaA7Zx7wzm3\nq//1CUk/lzQp7nIApGvBsvD933nAe962y3//5me856D7apdUzx6/9vLadQPyKNFz2Gb2QUm/L+n5\nqu1LzKzXzHoPH+aaSSALpn6g8v2jQZdVVZmzxH/7ZyL2hKuvz77H57IxoAgSC9hm9j5JD0ha5pyr\nWCHYOfdd51yPc66nu5v72AJZ8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+HyiSRAK2mXXKC9b3Ouf+\nIYkyAMRsevho1ySf9Ugeq7Es6NEaN/M4diJ8/9qN4ft9nd/XwEFA+0tilrhJWifp58455msCWdEx\nvqHDkpoxftVNDR7YOS7WegDtIoke9ixJ10i61Mxe7H80eQ8fAEXzg61p1wBoLx1xZ+ic2y6Jm9IC\nOTSxSzp4JL3yZ56XXtlA2ljpDMCAGeFriB6ocwWzch/7kDT3Iul3Jzeex3MbaiSoUX8gy2LvYQPI\nN9cbfN56/qzm7pd92Q3SlueCywWKjIANoNLkO6V94TO+jm2VxszxXh/cIk3oqtx/3S3SPY9EL3LW\ndGn7Ounxuwa27d0vTbvCex2pZz/lW9ELBDKIIXEAlSbWvjF16faWrtcL1pu2eL3u0qOeYC1JO16q\nPH7j495CLaVe9cSu8OMlSRO+WF+hQMaYq3Xfu4T19PS43t78jnV5V7nlV9r/flqhkG146rC02+fC\n6ypRL+laOFu6fqE0Z4Z09IT0k93Sreuln+2JUL8o/z2c3xd4OVch2y9n8t6GknY652r+NTEkDmCw\nzsZXINy82gvQQcaOkqZNkq6eV7l9+4vSJZ9vsFCuvUYBELAB+JvhpJ3hPZvSBLTODundqsli9Syo\n4nqlj18w0JvunCmdPhOxd83McBQEARtAsAhBWxoI1o2uelZ+3JkXpFPPR8yLYI0CYdIZgHBTay/o\nXZos5ueWJdLRp73eculxcoe33c+QiyIG66nfj5AIyA8mnSUs75Ml0v730wq0oQJ72dWB9co50oN3\nNl6XxSu9GeflAofFI/auab/sy3sbiklnAGIzw0m7RkjunUG7+p6Uxo2u3DZytvTWyejZd42S3nxK\n2nir95Ckb2yQbr7LJ/HUjVLXouiZAzlBwAYQzYX9Ebiqt90xRJp6hfTq/sazPnK8srf+y0cG97Ql\ncc4ahcY5bAD1KQuarld6aFtzwdrPuQu867YrhsMJ1ig4etgA6jfDSaeOSLvH6drLpWsvT7Cs8w81\ndV04kBf0sAE0prPLC9xT1iST/5S1Xv4Ea0ASPWwAzZqwzHtIka7Zromhb8AXPWwA8ZnhBh7Tjw7a\nvcKvM37+G5XHAfBFDxtAMjrGDArAq/4upboAOUAPGwCADCBgAwCQAQRsAAAygIANAEAGpH7zDzPL\n9bTQtL/fpBVgUX7aMONov+wrQBty8w8AAAKdOSq92FWxacUaadWNVenO3y91vr919QpADzthaX+/\nSePXffblvQ1pv+yLtQ3bcHGfqD1szmEDAPLt4B1eoI4jWEsDeR1cFU9+EdHDTlja32/S+HWffXlv\nQ9ov+xpuw1NvSrvHx1sZP+cfkDonNnw457ABAMUVV286it3neM8JL63LkDgAIF9aGaxbWC4BGwCQ\nD7uGpResS3aadGRTIlkTsAEA2bfTJPdu09nccHsMddm7OJEfDkw6S1ja32/SmPCSfXlvQ9ov+2q2\n4a7hkvttU2WYz5Qv19tUlpINlS6sXS8u6wIAFEOEYN09V7r3h/77/IJ12PbIYujxl6OHnbC0v9+k\n8es++/LehrRf9oW2YY2h5yg957DAXCvtR6dJP70/tAo1Z4/TwwYA5FuNYP3t+/y3N9pz9jvu5T0R\nDozpfDYBGwCQPacP1Uyy9I4W1EMRfwCc7mu6HAI2ACB7Xmp8ZbFqQZPLmp50Vu6l7qazYKUzAEC2\nvDFw7VXYOWrXG3342/VKJ05Ko2ZLx5+RRo6IXp31Xxl4HXrO/MAa6ZzqW4FFRw8bAJAt+/9CUnAw\n3lc2Wj5r+uD9QT3nUpAOCtZBx1230Hv+1QH//e/V8/Xl/gkiImADAHJlyvyB19vXVQbasGHuD1/l\nPY+7NDhNdV7l789dUF8960XABgBkR5Mzrl8Pmav2ymve85HjwWnC9kXSRP0J2ACAXJk/K3jf5PnB\n+6II630vuKS5vGshYAMAMunkDv/tj65tbT1KHl7jv/2dZ+PJn4ANAMiGU5Wzus4a5p1DPmvYwLYo\nl2JteLix4h/aVjtNefkjhnvvhw+tSnTqcEPlszRpwtL+fpNW+GURcyDvbUj7Zd97bRhy/vf0Galz\nZn96n6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdWtWK6UpUkBAIXRMaS544deXPm+e25z+YUG\n6wYRsAEAuRJlsZRFKyvf1xqI+dzX4im3GbEHbDMbbmYvmNlLZvaymX017jIAAGjGfVvqS79+czL1\nqEcSPezfSrrUOTdd0gWSPm1mF9c4BgCAUMtXR0+bdG+3mfLq+RzlYg/YzvNW/9vO/ke+Z30AABK3\nurmVPQf5wm3R0sV9169GP0ci57DNbIiZvSjpkKQfOeeer9q/xMx6zSzOe6EAAPCeBcvC93/nAe95\n2y7//Zuf8Z6D7qtdcuWKyvfXXl67bo1I9LIuMxsj6UFJX3TO/TQgTa5731xSkn20YbbRftkX5bIu\nSZp2hbR3f9Wx/d3CoCHrWnf0CtsflHek23K222VdzrljkrZK+nSS5QAA8OO7B2+btzT8mK6QpUYl\naewnwvcvWxW+P05JzBLv7u9Zy8zOkjRX0r/GXQ4AoGCmh68QNmnC4G2P1VgW9GiNm3kcOxG+f+3G\n8P2+zu9r4CCpo6Gjwr1f0j1mNkTeD4L7nXOPJFAOAKBIOsY3dFhSM8avuqnBAzvHNXRY7AHbObdb\n0u/HnS8AAO3kB1tbWx4rnQEAcmNiV7rlzzwvuby5+UfC0v5+k1aoGao5lfc2pP2yb1Ab1pgt3ugQ\n+Mc+5AX8vfulX+xrLI+aM8RnDP73GHWWeBLnsAEASE3YpVjzZzV3v+zLbpC2PBdcbpII2ACAbJl8\np7QvfMbXsa3SmDne64NbpAlVQ+XX3SLdU8d06FnTpe3rpMfvGti2d7937bckHYiyNvmUb0Uv0AdD\n4glL+/tNWiGH43Im721I+2WfbxvWGBaXvF52qde7aYu0eGV4+np87+vS4ssGlxPKZzhcij4kTsBO\nWNrfb9IK+59FjuS9DWm/7PNtw1OHpd0+F15XiXo+e+Fs6fqF0pwZ0tET0k92S7eul362J0L9ogTr\n8/sCL+fiHDYAIL86uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQhu89rocPeyEpf39Jq2wv+5z\nJO9tSPtlX2gbRhwa7+yQ3n1u8PbIdajqRXfOlE6faW4o/L160MMGAOTeDBcpaJeCdaOXfJUfd+YF\n6dTzEfOqEazrwcIpAIBsm1pjBNUvAAAgAElEQVR7QW/rCQ6wtyyRjj7t9ZZLj5M7vO1+hlwUMVhP\n/X6ERNExJJ6wtL/fpBV+OC4H8t6GtF/2RWrDgF52dWC9co704J2N12XxSm/GebnAYfGIvWtmibeJ\ntL/fpPGfRfblvQ1pv+yL3Ia7RkjunYpN1iP1PSmNG12ZdORs6a2T0evQNUp686nKbd/YIN18l0/A\nnrpR6loUOW/OYQMAiuXC/ghc1dvuGCJNvUJ6dX/jWR85Xtlb/+Ujg3vakmI9Z12Nc9gAgHwpC5qu\nV3poW3PB2s+5C7zrtit61wkGa4kh8cSl/f0mjeG47Mt7G9J+2ddwG546Iu1u/vrnms4/1NR14VGH\nxOlhAwDyqbPL6/VOWZNM/lPWevk3EazrQQ87YWl/v0nj13325b0Nab/si7UNI1yzXVPMQ9/0sAEA\nqDbDDTymHx20e4VfZ/z8NyqPSwk97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAy\nIPWVzmbMmKHe3ij3J8umvJ9fyvu5JYk2zDraL/vy3oZR0cMGACADUu9hI7pIN0qvodF7wQIA0kUP\nu83ddM3A/VnjUMpr+dXx5AcAaA0CdpvqGuUF1ju+lEz+q2708p/QlUz+AIB4MSTehuLqTUdxsP/2\ncAyVA0B7o4fdZloZrNuhXABANATsNvGbZ9MPmq5X+rNPpVsHAIA/AnYbcL3SsKHN53PD7c3nsem2\n9H84AAAG4xx2yt7Z0Xwe5eef//p+77nZoPubZ6Xhf9RcHgCA+NDDTtnwYbXTdM+V7v2h/76gyWLN\nTiKLo8cPAIgPATtFtXrB1uM9+o5Jn/2r5oNwKb/S47w/ba5+AIDWIWCnpFYw/PZ9/tsbDdp+x728\np/ZxBG0AaA8E7BR0R1isZOkdyddDivYDYNzo5OsBAAhHwE7BoS3x5RXUA46zZ9z3ZHx5AQAawyzx\nFvvzawZe+/VuS4HW9UYf/na90omT0qjZ0vFnpJEjotdn/Vei1WfZYumbG6PnCwCIFz3sFru9f23w\noGC879DA61nTB+8P6jmXgnRQsA467rqF3vOvDvjvL9VzzQr//QCA1iBgt5kp8wdeb19XGWjDhrk/\nfJX3PO7S4DTVeZW/P3dBffUEALQWAbuFmj2v/Pqh4H2vvOY9HzkenCZsXxTMGAeA9BCw28z8WcH7\nJs8P3hdFWO97wSXN5Q0ASBYBOyUnA5YkfXRta+tR8vAa/+3vPNvaegAA/BGwW2TiuMr3Zw3zhpjP\nKluaNMqQ84aHGyv/oW2105SXP2K493541RKl48c0Vj4AoDkE7BY58Lj/9pM7pFPPe6+jXMZ1/VcH\nbzt9pvJ937HBaa6MMMu7VP6xrdLb2/3THH6idj4AgPgRsNtAx5Dmjh96ceX77rnN5Tf6fc0dDwCI\nHwG7zUTpZS9aWfneufD0n/taPOUCANKTSMA2syFm9s9m9kgS+RfdfXUubbp+czL1AAC0TlI97C9J\n+nlCeWfS8tXR07a6t1tPefV8DgBAfGIP2GY2WdLlku6OO+8sW7083vy+cFu0dHHf9SvuzwEAiCaJ\nHvY3JX1Z0v8ISmBmS8ys18x6Dx8+nEAVsm/BsvD933nAe962y3//5me856D7apdUzx6/9vLadQMA\ntF6sAdvMFkg65JzbGZbOOfdd51yPc66nu7s7zipk1tQPVL5/NOCyqmpzlvhv/0zEnnD19dn3+Fw2\nBgBIX9w97FmSrjCzVyVtknSpmf1dzGXk0o99TiDMWxp+TFfIUqOSNPYT4fuXrQrfDwBoH7EGbOfc\nzc65yc65D0paJOkp59xn4ywjq8Z/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXIAQHK4DrtF\n3vx1Y8clNWP8qpsaO67ZO34BABrTkVTGzrmtkrYmlT+a84OtadcAAFAPethtZGJXuuXPPC/d8gEA\nwQjYLVRrePtAnSuYlfvYh6S5F0m/O7nxPJ7bEL6f5UsBID2JDYmjMa43ODDOn9Xc/bIvu0Ha8lxw\nuQCA9kXAbrEVa6RVN4anObZVGjPHe31wizShaqj8uluke+pYpX3WdGn7Ounxuwa27d0vTbvCex2l\nZ//FmFdMAwDUx1ytWz0lrKenx/X25rd7Z2aDtkXpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP2\nv59W8GvDPMl7G9J+2Zf3NpS00zlX86QjATthfv/Qxo+RDj8R4diI54wXzpauXyjNmSEdPSH9ZLd0\n63rpZ3tqHxslWI+7NPhyrrT//bRC3v+zyHsb0n7Zl/c2VMSAzZB4CvqONX7s5tVegA4ydpQ0bZJ0\n9bzK7dtflC75fGNlcu01AKSPgJ2SKEPRpQlonR3Su1WTxeqZse16pY9fMFBe50zp9JnmhsIBAK1F\nwE5R1PPHpWDdaPAsP+7MC9Kp56PlRbAGgPbBddgpW3Rz7TTWExw8b1kiHX3aC/ylx8kd3nY/Qy6K\nFoj/5Mu10wAAWodJZwmLMlkiqJddHVivnCM9eGfjdVm80ptx3kjZQdL+99MKeZ/wkvc2pP2yL+9t\nKCadZYf1SG9vl0YMH7yv70lp3OjKbSNnS2+djJ5/1yjpzaekjbd6D0n6xgbp5rsGp110s3Tfj6Ln\nDQBoDQJ2mzj7495zdY+3Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBoB2xjnsNlMeNF2v9NC25oK1\nn3MXeNdtl/84IFgDQHujh92GrEcaO1I68rR07eXeIyndc5u7LhwA0Br0sNvU0RNe4F62Kpn8l97h\n5U+wBoBsoIfd5tZu9B5SPHfUYugbALKJHnaGlK7Htp6Bu3mVW7Fm8LZzLqs8DgCQTfSwM+rXb/kH\n4NX3tr4uAIDk0cMGACADCNgAAGQAARsAgAxIfS1xM8v1Qrhpf79JK8Aav7RhxtF+2VeANoy0ljg9\nbAAAMoBZ4kCr7IyhJzQj3z0NAMHoYQNJOniHF6jjCNbSQF4HE1oCD0Db4hx2wtL+fpPG+bMAp96U\ndo+PvzLVzj8gdU5sKou8tyF/g9lXgDbkfthAKuLqTUex+xzvmaFyIPcYEgfi1Mpg3Q7lAmgZAjYQ\nh13D0g+aO006sindOgBIDAEbaNZOk9y7TWdzw+0x1GXv4vR/OABIBJPOEpb295u0wk942TVccr9t\nKn+/m7g0fStVGypdGK1eeW9D/gazrwBtyMIpQOIiBOvuudK9P/TfF3TL06ZvhRpDjx9Ae6GHnbC0\nv9+kFfrXfY2h5yg957DAXCvtR6dJP70/tAqRZo/nvQ35G8y+ArQhPWwgMTWC9bfv89/eaM/Z77iX\n90Q4kPPZQG4QsIF6nT5UM8nSO1pQD0X8AXC6L/F6AEgeARuo10vNrSxWLmhyWdOTzsq91B1jZgDS\nwkpnQD3eGLj2KuwcteuNPvzteqUTJ6VRs6Xjz0gjR0SvzvqvDLwOPWd+YI10zo3RMwbQduhhA/XY\n/xeSgoPxvrLR8lnTB+8P6jmXgnRQsA467rqF3vOvDvjvf6+ery/3TwAgMwjYQIymzB94vX1dZaAN\nG+b+8FXe87hLg9NU51X+/twF9dUTQPYQsIGompxx/XrIXLVXXvOejxwPThO2LxJmjAOZRsAGYjR/\nVvC+yfOD90UR1vtecElzeQNofwRsoAEnd/hvf3Rta+tR8vAa/+3vPNvaegBIDgEbiOJU5ayus4Z5\n55DPGjawLcqlWBsebqz4h7bVTlNe/ojh3vvhQ6sSnTrcWAUApI6lSROW9vebtMIsixhy/vf0Galz\nZn9an6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdUdtFxp3tuQv8HsK0AbsjQp0AodQ5o7fujF\nle+75zaXX2iwBpBZBGwgRlEWS1m0svJ9rc7D574WT7kAsi2RgG1mr5rZv5jZi2YW5yKLQObdt6W+\n9Os3J1MPANmSZA/7E865C6KMywPtbvnq6Glb3dutp7x6PgeA9sKQOBDB6phX9vzCbdHSxX3Xr7g/\nB4DWSSpgO0lbzGynmS2p3mlmS8ysl+Fy5NWCZeH7v/OA97xtl//+zc94z0H31S65ckXl+2svr103\nANmUyGVdZvYB59x+M5sg6UeSvuiceyYgba7n6xfgcoS0q5C4Wpd1SdK0K6S9+6uO6/85GjRkXeuO\nXmH7g/KOdFtOLuvKlby3n1SINkzvsi7n3P7+50OSHpR0URLlAO3ix3cP3jZvafgxXSFLjUrS2E+E\n71+2Knw/gHyJPWCb2dlmNrL0WtIfS/pp3OUALTU9fIWwSRMGb3usxrKgR2vczOPYifD9azeG7/d1\nfl8DBwFoBx0J5DlR0oP9wzQdkr7nnHssgXKA1ukY39BhSc0Yv+qmBg/sHBdrPQC0TuwB2zm3R9L0\nuPMFMOAHW9OuAYBW47IuICYTu9Itf+Z56ZYPIFnc/CNhaX+/SSvcDNUas8UbHQL/2Ie8gL93v/SL\nfY3lUXOG+Az/f4t5b0P+BrOvAG0YaZZ4EuewgcIKuxRr/qzm7pd92Q3SlueCywWQbwRsoB6T75T2\nhc/4OrZVGjPHe31wizShaqj8ulukex6JXuSs6dL2ddLjdw1s27vfu/Zbkg5EWZt8yreiFwigLTEk\nnrC0v9+kFXI4rsawuOT1sku93k1bpMUrw9PX43tflxZfNricUAHD4VL+25C/wewrQBtGGhInYCcs\n7e83aYX8z+LUYWm3z4XXVaKez144W7p+oTRnhnT0hPST3dKt66Wf7YlQtyjB+vy+0Mu58t6G/A1m\nXwHakHPYQCI6uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQrn2GsgFetgJS/v7TVqhf91HHBrv\n7JDefW7w9sjlV/WiO2dKp880PxT+Xl1y3ob8DWZfAdqQHjaQqBm1bwoiDQTrRi/5Kj/uzAvSqecj\n5hUhWAPIDhZOAZoxtfaC3tYTHGBvWSIdfdrrLZceJ3d42/0MuShisJ76/QiJAGQJQ+IJS/v7TRrD\ncQrsZVcH1ivnSA/e2Xg9Fq/0ZpxX1C1oWLyO3nXe25C/wewrQBsyS7wdpP39Jo3/LPrtGiG5dyo2\nWY/U96Q0bnRl0pGzpbdORi+/a5T05lOV276xQbr5Lp+APXWj1LUoeubKfxvyN5h9BWhDzmEDLXNh\nfwSu6m13DJGmXiG9ur/xrI8cr+yt//KRwT1tSZyzBnKOc9hAnMqCpuuVHtrWXLD2c+4C77rtit41\nwRrIPYbEE5b295s0huMCnDoi7W7B9c/nH2rqunAp/23I32D2FaANIw2J08MGktDZ5fV6p6xJJv8p\na738mwzWALKDHnbC0v5+k8av+zpEuGa7pgSGvvPehvwNZl8B2pAeNtBWZriBx/Sjg3av8OuMn/9G\n5XEACosedsLS/n6Txq/77Mt7G9J+2VeANqSHDQBAXhCwAQDIAAI2AAAZkPpKZzNmzFBvb5T7BGZT\n3s8v5f3ckkQbZh3tl315b8Oo6GEDAJABBGwAADIg9SFxAMiKwNuZ1iHS/cwBH/SwASDETdd4gTqO\nYC0N5LX86njyQ3EQsAHAR9coL7De8aVk8l91o5f/hK5k8kf+MCQOAFXi6k1HcbD/3uYMlaMWetgA\nUKaVwbodykV2ELABQNJvnk0/aLpe6c8+lW4d0L4I2AAKz/VKw4Y2n88Ntzefx6bb0v/hgPbEOWwA\nhfbOjubzKD///Nf3e8/NBt3fPCsN/6Pm8kC+0MMGUGjDh9VO0z1XuveH/vuCJos1O4ksjh4/8oWA\nDaCwavWCrcd79B2TPvtXzQfhUn6lx3l/2lz9UCwEbACFVCsYfvs+/+2NBm2/417eU/s4gjZKCNgA\nCqc7wmIlS+9Ivh5StB8A40YnXw+0PwI2gMI5tCW+vIJ6wHH2jPuejC8vZBezxAEUyp9fM/Dar3db\nCrSuN/rwt+uVTpyURs2Wjj8jjRwRvT7rvxKtPssWS9/cGD1f5A89bACFcnv/2uBBwXjfoYHXs6YP\n3h/Ucy4F6aBgHXTcdQu9518d8N9fqueaFf77URwEbAAoM2X+wOvt6yoDbdgw94ev8p7HXRqcpjqv\n8vfnLqivnigeAjaAwmj2vPLrh4L3vfKa93zkeHCasH1RMGO82AjYAFBm/qzgfZPnB++LIqz3veCS\n5vJG/hGwARTSyYAlSR9d29p6lDy8xn/7O8+2th5oXwRsAIUwcVzl+7OGeUPMZ5UtTRplyHnDw42V\n/9C22mnKyx8x3Hs/vGqJ0vFjGisf2UfABlAIBx73335yh3Tqee91lMu4rv/q4G2nz1S+7zs2OM2V\nEWZ5l8o/tlV6e7t/msNP1M4H+UTABlB4HUOaO37oxZXvu+c2l9/o9zV3PPIpkYBtZmPM7O/N7F/N\n7Odm9odJlAMAcYvSy160svK9c+HpP/e1eMpFsSXVw14r6THn3L+TNF3SzxMqBwBa7r46lzZdvzmZ\neqBYYg/YZjZK0mxJ6yTJOfeuc87njA4AtM7y1dHTtrq3W0959XwO5EsSPexpkg5LWm9m/2xmd5vZ\n2QmUAwCRrV4eb35fuC1aurjv+hX350B2JBGwOyRdKOlvnHO/L+ltSX9ZnsDMlphZr5n1Hj58OIEq\nAEBzFiwL3/+dB7znbbv8929+xnsOuq92SfXs8Wsvr103FFMSAXufpH3Ouf4LJfT38gL4e5xz33XO\n9Tjnerq7uxOoAgDUZ+oHKt8/GnBZVbU5S/y3fyZiT7j6+ux7fC4bA6QEArZz7oCk18zsI/2bPinp\nZ3GXAwBx+vHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfD5RLapb4FyXda2a7JV0g6daEygGASMZ/Mnz/\npAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXLkW0cSmTrnXpTEVYUA2sabv27suKRmjF91U2PHNXvH\nL2QXK50BQAp+sDXtGiBrCNgA0G9iV7rlzzwv3fLR3gjYAAqj1vD2gTpXMCv3sQ9Jcy+Sfndy43k8\ntyF8P8uXFlsi57ABIKtcb3BgnD+ruftlX3aDtOW54HKBMARsAIWyYo206sbwNMe2SmPmeK8PbpEm\nVA2VX3eLdM8j0cucNV3avk56/K6BbXv3S9Ou8F5H6dl/MeYV05A95mrdZiZhPT09rrc3vz8tzSzt\nKiQq7X8/rUAbZptf+0XpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP39pPy/zcoaadzruYJDwJ2\nwvL+Dy3tfz+tQBtmm1/7jR8jHX4iwrERzxkvnC1dv1CaM0M6ekL6yW7p1vXSz/bUPjZKsB53afDl\nXHlvPyn/f4OKGLAZEgdQOH1N3D9w82ovQAcZO0qaNkm6el7l9u0vSpd8vrEyufYaEgEbQEFFGYou\nTUDr7JDerZosVs+MbdcrffyCgfI6Z0qnzzQ3FI7iIWADKKyo549LwbrR4Fl+3JkXpFPPR8uLYI1y\nXIcNoNAW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00KBYmnSUs75Ml0v730wq0YbZF\nab+gXnZ1YL1yjvTgnY3XZfFKb8Z5I2UHyXv7Sfn/GxSTzgAgGuuR3t4ujRg+eF/fk9K40ZXbRs6W\n3joZPf+uUdKbT0kbb/UekvSNDdLNdw1Ou+hm6b4fRc8bxUHABgBJZ3/ce67u8XYMkaZeIb26v/G8\njxyv7DH/8pHBPW2Jc9YIxzlsAChTHjRdr/TQtuaCtZ9zF3jXbZf/OCBYoxZ62ABQxXqksSOlI09L\n117uPZLSPbe568JRHPSwAcDH0RNe4F62Kpn8l97h5U+wRlT0sAEgxNqN3kOK545aDH2jUfSwASCi\n0vXY1jNwN69yK9YM3nbOZZXHAY2ihw0ADfj1W/4BePW9ra8LioEeNgAAGUDABgAgAwjYAABkQOpr\niZtZrhfCTfv7TVoB1vilDTOO9su+ArRhpLXE6WEDAJABzBJH2+AaVwAIRg8bqbrpmoF7CMehlNfy\nq+PJDwDaBeewE5b295u0Rs+flW43mLSJfywdOtJcHrRhttF+2VeANuR+2GhPcfWmozjYfwtDhsoB\nZB1D4mipVgbrdigXAOJCwEZL/ObZ9IOm65X+7FPp1gEAGkXARuJcrzRsaPP53HB783lsui39Hw4A\n0AgmnSUs7e83abUmvLyzQxo+rMkyfM4/Nxt0f/uuNPyPoqUtehtmHe2XfQVoQxZOQfqiBOvuudK9\nP/TfFzRZrNlJZHH0+AGglehhJyzt7zdpYb/ua/WCo/ScwwJzrbQfnSb99P766zConAK3YR7QftlX\ngDakh4301ArW377Pf3ujPWe/417eU/s4zmcDyAoCNmLX3VU7zdI7kq+HFO0HwLjRydcDAJpFwEbs\nDm2JL6+gHnCcPeO+J+PLCwCSwkpniNWfXzPwOuwcteuNPvzteqUTJ6VRs6Xjz0gjR0Svz/qvRKvP\nssXSNzdGzxcAWo0eNmJ1+5e856BgvO/QwOtZ0wfvD+o5l4J0ULAOOu66hd7zrw747y/Vc80K//0A\n0C4I2GipKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BIB2Q8BGbJo9r/z6oeB9r7zmPR85\nHpwmbF8UzBgH0M4I2Gip+bOC902eH7wvirDe94JLmssbANJGwEYiTu7w3/7o2tbWo+ThNf7b33m2\ntfUAgEYRsBGLieMq3581zBtiPqtsadIoQ84bHm6s/Ie21U5TXv6I4d774VVLlI4f01j5AJA0liZN\nWNrfb9JKyyKGBePTZ6TOmQpMVz2jvDpN+fGSdPiJwYG1Vh7laY5tlUa/L7i+g/IqSBvmFe2XfQVo\nQ5YmRXvoGNLc8UMvrnzfPbe5/MKCNQC0KwI2WirKYimLVla+r/Xj+nNfi6dcAGhnsQdsM/uImb1Y\n9jhuZsviLgf5dV+dS5uu35xMPQCgncQesJ1z/+acu8A5d4GkGZJOSnow7nLQXpavjp621b3desqr\n53MAQCslPST+SUm/cM79MuFykLLVy+PN7wu3RUsX912/4v4cABCXpAP2IkmDbqlgZkvMrNfMWFuq\noBbUOEnynQe85227/PdvfsZ7DrqvdsmVVWuEX3t57boBQDtK7LIuMxsqab+kjzrnDoaky/V8/QJc\njiCp9jXW066Q9u6v3FY6JmjIutYdvcL2B+Ud5VpwLuvKF9ov+wrQhqlf1jVP0q6wYI3i+PHdg7fN\nWxp+TFfIUqOSNPYT4fuXrQrfDwBZkmTAXiyf4XDk0/hPhu+fNGHwtsdqLAt6tMbNPI6dCN+/toF/\nfWHrkQNAmhIJ2GY2QtKnJP1DEvmj/bz568aOS2rG+FU3NXZcs3f8AoCkdCSRqXPupKRxNRMCCfnB\n1rRrAADxYqUztMzErnTLn3leuuUDQDO4+UfC0v5+k1Y9Q7XWLOxGh8A/9iEv4O/dL/1iX2N5NFq3\norVh3tB+2VeANow0SzyRIXEgSNilWPNnNXe/7MtukLY8F1wuAGQZARuxWrFGWnVjeJpjW6Uxc7zX\nB7dIE6qGyq+7RbrnkehlzpoubV8nPX7XwLa9+71rvyXpQIS1yb8Y84ppABA3hsQTlvb3mzS/4bio\ni5OU0m3aIi1eGZ6+Ht/7urT4ssHl1KpPkCK2YZ7QftlXgDaMNCROwE5Y2t9v0vz+sxg/Rjr8RIRj\nI57PXjhbun6hNGeGdPSE9JPd0q3rpZ/tqX1slGA97tLwy7mK2IZ5QvtlXwHakHPYSEffscaP3bza\nC9BBxo6Spk2Srp5XuX37i9Iln2+sTK69BpAF9LATlvb3m7SwX/dRh6I7O6R3nxu8ParqcjpnSqfP\nND8U/l7+BW7DPKD9sq8AbUgPG+mKev64FKwbveSr/LgzL0inno+WV6vvyw0AzWDhFCRq0c2101hP\ncPC8ZYl09Gkv8JceJ3d42/0MuShaIP6TL9dOAwDthCHxhKX9/SYtynBcUC+7OrBeOUd68M7G67J4\npTfjvJGyw9CG2Ub7ZV8B2pBZ4u0g7e83aVH/s3h7uzRieNWxPVLfk9K40ZXbR86W3joZvQ5do6Q3\nn6rc9o0N0s13DQ7Yi26W7vtR9Lwl2jDraL/sK0Abcg4b7ePsj3vP1QG0Y4g09Qrp1f2N533keGWP\n+ZePDO5pS5yzBpBtnMNGS5UHTdcrPbStuWDt59wF3nXb5T8OCNYAso4h8YSl/f0mrdHhuLEjpSNP\nx1wZH91zm7suXKINs472y74CtGGkIXF62EjF0RNer3fZqmTyX3pH/znyJoM1ALQLetgJS/v7TVqc\nv+7juKNWEkPftGG20X7ZV4A2pIeNbCldj209A3fzKrdizeBt51xWeRwA5BU97ISl/f0mjV/32Zf3\nNqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyoB1WOuuT9MsWlje+v8yWSOn8Uks/Ywry3oa0X4xov9i1\n/PMVoA3PjZIo9UlnrWZmvVFO7mdZ3j8jny/b+HzZlvfPJ7XvZ2RIHACADCBgAwCQAUUM2N9NuwIt\nkPfPyOfLNj5ftuX980lt+hkLdw4bAIAsKmIPGwCAzCFgAwCQAYUK2Gb2aTP7NzN7xcz+Mu36xMnM\n/tbMDpnZT9OuSxLMbIqZPW1mPzezl83sS2nXKW5mNtzMXjCzl/o/41fTrlPczGyImf2zmT2Sdl2S\nYGavmtm/mNmLZhbD/efai5mNMbO/N7N/7f9b/MO06xQXM/tIf7uVHsfNbFna9SpXmHPYZjZE0v8n\n6VOS9kn6J0mLnXM/S7ViMTGz2ZLekvTfnHPnpV2fuJnZ+yW93zm3y8xGStop6cq8tJ8kmbc6xNnO\nubfMrFPSdklfcs49l3LVYmNmyyX1SBrlnFuQdn3iZmavSupxzuVy4RQzu0fSj51zd5vZUEkjnHO5\nu+t8f7x4XdJM51wrF/YKVaQe9kWSXnHO7XHOvStpk6TPpFyn2DjnnpF0JO16JMU594Zzblf/6xOS\nfi5pUrq1ipfzvNX/tl2ok/MAAAJgSURBVLP/kZtf1GY2WdLlku5Ouy6on5mNkjRb0jpJcs69m8dg\n3e+Tkn7RTsFaKlbAniTptbL3+5Sz//CLwsw+KOn3JT2fbk3i1z9k/KKkQ5J+5JzL02f8pqQvS/of\naVckQU7SFjPbaWZL0q5MzKZJOixpff9pjbvN7Oy0K5WQRZI2pl2JakUK2H6L0eam91IUZvY+SQ9I\nWuacO552feLmnDvjnLtA0mRJF5lZLk5vmNkCSYecczvTrkvCZjnnLpQ0T9J/6j9VlRcdki6U9DfO\nud+X9LakXM0FkqT+of4rJH0/7bpUK1LA3idpStn7yZL2p1QXNKD/vO4Dku51zv1D2vVJUv9Q41ZJ\nn065KnGZJemK/nO8myRdamZ/l26V4uec29//fEjSg/JOxeXFPkn7ykZ9/l5eAM+beZJ2OecOpl2R\nakUK2P8k6cNmNrX/F9QiSZtTrhMi6p+QtU7Sz51zq9OuTxLMrNvMxvS/PkvSXEn/mm6t4uGcu9k5\nN9k590F5f3tPOec+m3K1YmVmZ/dPiFT/UPEfS8rNVRvOuQOSXjOzj/Rv+qSk3Ez6LLNYbTgcLrXH\n7TVbwjl32sxukPS4pCGS/tY593LK1YqNmW2UNEfSeDPbJ+krzrl16dYqVrMkXSPpX/rP8UrSSufc\nP6ZYp7i9X9I9/TNUf0fS/c65XF7+lFMTJT3YfyvIDknfc849lm6VYvdFSff2d3r2SLo+5frEysxG\nyLuS6D+mXRc/hbmsCwCALCvSkDgAAJlFwAYAIAMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAG\n/P+uMuaa/akHvAAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FNWd7/vPd9gbEMOvDRtMgGtgkifnTow4skecIXKJIWNAMHru3Bm4Ro/m5nJu7jEEwcmMPM88MXlONFcFQuLcycmRAc8ZA5pxjKgTJf4AA0adDaNMTGbuY8BERH5sgYBiInDW/aN2u7t7V1VXd1d1dVW9X8/TT3dXrVprdS82316rVq0y55wAAEB7+520KwAAAGojYAMAkAEEbAAAMoCADQBABhCwAQDIAAI2AAAZQMAGACADCNgAAGQAARtoM2b2QTP7RzM7amYHzOwuM+sIST/GzP6mP+1JM/sXM/sPrawzgOQRsIH28/9KOiTp/ZIukPS/SPq//RKa2VBJT0g6V9IfShot6c8l3W5mS1tSWwAtQcAG2s9USfc7537jnDsg6TFJHw1Ie42k/0nS/+ac2+ucO+Wce0zSUkn/2cxGSpKZOTP7UOkgM9tgZv+57P0CM3vRzI6Z2bNmdn7Zvg+Y2QNmdtjM9pb/EDCzW8zsfjP7b2Z2wsxeNrOesv1/YWav9+/7NzP7ZDxfEVA8BGyg/ayVtMjMRpjZJEnz5AVtP5+S9EPn3NtV2x+QNELSxbUKM7MLJf2tpP8oaZyk/yJps5kNM7PfkfSwpJckTZL0SUnLzOyysiyukLRJ0hhJmyXd1Z/vRyTdIOkPnHMjJV0m6dVa9QHgj4ANtJ9t8nrUxyXtk9Qr6QcBacdLeqN6o3PutKQ+Sd0Ryvs/Jf0X59zzzrkzzrl7JP1WXrD/A0ndzrmvOefedc7tkfRfJS0qO367c+4fnXNnJP13SdP7t5+RNEzS75lZp3PuVefcLyLUB4APAjbQRvp7tI9L+gdJZ8sLyGMl/T8Bh/TJO9ddnU9H/7GHIxR7rqQV/cPhx8zsmKQpkj7Qv+8DVftWSppYdvyBstcnJQ03sw7n3CuSlkm6RdIhM9tkZh+IUB8APgjYQHvpkhcs73LO/dY596ak9ZLmB6R/QtI8Mzu7avv/KumUpBf635+UN0Reck7Z69ckfd05N6bsMcI5t7F/396qfSOdc0H1qeCc+55z7uPyAr9T8A8PADUQsIE24pzrk7RX0hfMrMPMxkj6D/LOIfv57/KGzb/ffzlYZ//55W9Jut059+v+dC9K+t/NbIiZfVrezPOS/yrp/zKzmeY528wu75+w9oKk4/2Tx87qP/48M/uDWp/FzD5iZpea2TBJv5H0jrxhcgANIGAD7effS/q0vOHsVySdlnSjX0Ln3G8lzZXXE35eXlB8TNI3JX21LOmXJC2UdEzS1So7J+6c65V3HvsuSUf7y7yuf9+Z/uMukPdDok/S3fIuH6tlmKRv9B9zQNIEecPpABpgzrm06wAgJmbWKemHkl6XdJ3jDxzIDXrYQI44507JO3/9C0kfSbk6AGJEDxsAgAyghw0AQAYE3lCgVcaPH+8++MEPpl2NxOzcuTPtKiRqxowZaVchcbRhttF+2Zf3NpTU55yruchR6kPiPT09rre3t/mMdlrzecyI/7swi6FebSztfz+tQBtmG+2XfXlvQ0k7nXM9tRJle0j84B1eoI4jWEsDeR1cFU9+AADEJJsB+9SbXmDd9+Vk8t93k5f/qYPJ5A8AQJ1SP4ddt7h601Hs7l+9MYGhcgAA6pGtHnYrg3U7lAsAQL9sBOxdw9IPmjtNOrIp3ToAAAqr/QP2TpPcu01nc8PtMdRl7+L0fzgAAAqpvc9h7xredBZWNlH+r+/3nl2zV5HtGiZd+NsmMwEAILr27mG72kGxe6507w/991nAVW1B2yOLoccPAEA92jdg1xh6th7v0XdM+uxfNR+ES/mVHuf9aXP1AwAgTu0ZsGsEw2/f57+90aDtd9zLeyIcSNAGALRI+wXs04dqJll6RwvqoYg/AE73JV4PAADaL2C/NDG2rIImlzU96azcSzXXawcAoGntNUv8jYFrr/x6t6VA63qjD3+7XunESWnUbOn4M9LIEdGrs/4rA6/D6qMDa6RzboyeMQAAdWqvHvb+v5AUHIz3lY2Wz5o+eH9Qz7kUpIOCddBx1y30nn91wH//e/V8fbl/AgAAYtJeAbuGKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BAAgbu0TsJuccf16yFy1V17zno8cD04Tti8SZowDABLUPgE7gvmzgvdNnh+8L4qw3veCS5rLGwCAZrVlwD65w3/7o2tbW4+Sh9f4b3/n2dbWAwBQXO0RsE9Vzuo6a5h3DvmsYQPbolyKteHhxop/aFvtNOXljxjuvR8+tCrRqcONVQAAgBraI2Dvfr/v5pM7pFPPe6+jXMZ1/VcHbzt9pvJ937HBaa5cUTvvUvnHtkpvbw9ItHtC7YwAAGhAewTsEB1Dmjt+6MWV77vnNpff6Pc1dzwAAI1o+4BdLkove9HKyvfOhaf/3NfiKRcAgCRlKmBHcd+W+tKv35xMPQAAiFMiAdvMPm1m/2Zmr5jZX9ZKv3x1HXm3uLdbT3n1fA4AAOoRe8A2syGS/lrSPEm/J2mxmf1e2DGrY17Z8wu3RUsX912/4v4cAACUJNHDvkjSK865Pc65dyVtkvSZOAtYsCx8/3ce8J637fLfv/kZ7znovtol1bPHr728dt0AAEhCEgF7kqTXyt7v69/2HjNbYma9ZtZ7+HDta5enfqDy/aNBl1VVmbPEf/tnIvaEq6/PvsfnsjEAAFohiYDtt6h2xVxt59x3nXM9zrme7u7a95P+8d2Dt81bGn5MV8hSo5I09hPh+5etCt8PAEArJRGw90maUvZ+sqT9oUdMD+9lT/JZj+SxGsuCHq1xM49jJ8L3r90Yvt/X+X0NHAQAQG1JBOx/kvRhM5tqZkMlLZIUfvFUx/iGCkpqxvhVNzV4YOe4WOsBAEBJR9wZOudOm9kNkh6XNETS3zrnXo67nCT9YGvaNQAAoFLsAVuSnHP/KOkf48xzYpd08EicOdZn5nnplQ0AQPusdDYjfA3RA3WuYFbuYx+S5l4k/e7kxvN4bkONBDXqDwBAMxLpYSfF9Qaft54/q7n7ZV92g7TlueByAQBIU3sF7Ml3SvvCZ3wd2yqNmeO9PrhFmtBVuf+6W6R7Hole5Kzp0vZ10uN3DWzbu1+adoX3OlLPfsq3ohcIAEAD2mdIXJIm1r4xden2lq7XC9abtni97tKjnmAtSTteqjx+4+PeQi2lXvXErvDjJUkTvlhfoQAA1MlcrftPJqynp8f19paNOZ86LO32ufC6StRLuhbOlq5fKM2ZIR09If1kt3Treulne2ofG2ko/Py+0Mu5zPzWkcmPtP/9tAJtmG20X/blvQ0l7XTO1Yxq7TUkLkmdtVc+C7J5tRegg4wdJU2bJF09r3L79helSz7fYKFcew0AaIH2C9iSN+N6Z/gvqtIEtM4O6d2qyWL1LKjieqWPXzDQm+6cKZ0+E7F3zcxwAECLtGfAliIFbWkgWDe66ln5cWdekE49HzEvgjUAoIXaa9JZtam1F/QuTRbzc8sS6ejTXm+59Di5w9vuZ8hFEYP11O9HSAQAQHzab9JZtYBednVgvXKO9OCdjddj8Upvxnm5wGHxOnrXeZ8skfa/n1agDbON9su+vLehMjvprNoMJ+0aIbl3Bu3qe1IaN7py28jZ0lsno2ffNUp68ylp463eQ5K+sUG6+S6fxFM3Sl2LomcOAEBM2j9gS9KF/RG4qrfdMUSaeoX0avjNO0MdOV7ZW//lI4N72pI4Zw0ASFV7n8OuVhY0Xa/00LbmgrWfcxd4121XDIcTrAEAKctGD7vcDCedOiLtHqdrL5euvTzBss4/1NR14QAAxCVbPeySzi4vcE9Zk0z+U9Z6+ROsAQBtIns97HITlnkPKdI12zUx9A0AaFPZ7GH7meEGHtOPDtq9wq8zfv4blccBANCmst3DDtIxZlAAXvV3KdUFAIAY5KeHDQBAjhGwAQDIAAI2AAAZQMAGACADUr/5h5nlenp22t9v0gqwKD9tmHG0X/YVoA0j3fyDHjYAwNeYkZW3J3a90vKrB287Z1zaNS0GetgJS/v7TRq/7rMv721I+9Un8LbCdai+/XGzCtCG9LABALXddM1AbzkO5b1xxIcedsLS/n6TlvfemUQbZh3tF6xrlPTmUzFWJsDEP5YOHWn8+AK0YaQedj5XOgMAhIqrNx3FwS3ec9xD5UXDkDgAFEwrg3U7lJsXBGwAKIjfPJt+0HS90p99Kt06ZBUBGwAKwPVKw4Y2n88Ntzefx6bb0v/hkEVMOktY2t9v0vI+YUmiDbOO9pPe2SENH9ZkOT7nn5sNur99Vxr+R7XTFaANuawLABAtWHfPle79of++oMlizU4ii6PHXyT0sBOW9vebtLz3ziTaMOuK3n61esFRes5hgblW2o9Ok356f/11qCgj/21IDxsAiqxWsP72ff7bG+05+x338p7ax3E+OxoCNgDkUHdX7TRL70i+HlK0HwDjRidfj6wjYANADh3aEl9eQT3gOHvGfU/Gl1desdIZAOTMn18z8DrsHLXrjT787XqlEyelUbOl489II0dEr8/6r0Srz7LF0jc3Rs+3aOhhA0DO3P4l7zkoGO87NPB61vTB+4N6zqUgHRSsg467bqH3/KsD/vtL9Vyzwn8/PARsACiYKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31RCUCNgDkSLPnlV8/FLzvlde85yPHg9OE7YuCGePBCNgAUDDzZwXvmzw/eF8UYb3vBZc0l3fREbABIKdO7vDf/uja1taj5OE1/tvfeba19cgqAjYA5MTEcZXvzxrmDTGfVbY0aZQh5w0PN1b+Q9tqpykvf8Rw7/3wqiVKx49prPy8Y2nShKX9/SYt78taSrRh1hWp/cKC8ekzUufM4HTVM8qr05QfL0mHnxgcWGvlUZ7m2FZp9PuC61ueVwHakKVJAQCejiHNHT/04sr33XObyy8sWMMfARsACibKYimLVla+r9XJ/dzX4ikXwWIP2Gb2t2Z2yMx+GnfeAIDWuK/OpU3Xb06mHhiQRA97g6RPJ5AvACDE8tXR07a6t1tPefV8jiKJPWA7556RdCTufAEA4VYvjze/L9wWLV3cd/2K+3PkBeewAaCgFiwL3/+dB7znbbv8929+xnsOuq92yZVVa4Rfe3ntumGwVAK2mS0xs14zYxE6AGiRqR+ofP/o9mjHzVniv/0zEXvC1ddn3/PVaMehUioB2zn3XedcT5TrzgAA8fjx3YO3zVsafkxXyFKjkjT2E+H7l60K34/oGBIHgJwY/8nw/ZMmDN72WI1lQY/WuJnHsRPh+9c2cH/rsPXIiyyJy7o2SvqJpI+Y2T4z+z/iLgMAMNibv27suKRmjF91U2PHNXvHr7zqiDtD59ziuPMEAGTPD7amXYN8YUgcAApkYle65c88L93ys4ybfyQs7e83aXm/cYREG2ZdEduv1h25Gh0C/9iHvIC/d7/0i32N5dFI3QrQhpFu/hH7kDgAoL253uCgPX9Wc/fLvuwGactzweWicQRsAMiZFWukVTeGpzm2VRozx3t9cIs0oWqo/LpbpHseiV7mrOnS9nXS43cNbNu7X5p2hff6QIS1yb8Y84ppecOQeMLS/n6TlvfhVIk2zLqitl+U3qz1DKTbtEVavDI8fT2+93Vp8WWDy6lVHz8FaMNIQ+IE7ISl/f0mLe//2Uu0YdYVtf3Gj5EOPxHh+IjnsxfOlq5fKM2ZIR09If1kt3Treulne2ofGyVYj7s0+HKuArQh57ABoKj6jjV+7ObVXoAOMnaUNG2SdPW8yu3bX5Qu+XxjZXLtdW30sBOW9vebtLz3ziTaMOuK3n5Rh6I7O6R3nxu8ParqcjpnSqfPNDcU/l7e+W9DetgAUHRRzx+XgnWjl3yVH3fmBenU89HyavV9ubOMhVMAIOcW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00GMCQeMLS/n6TlvfhVIk2zDrazxPUy64OrFfOkR68s/H6LF7pzThvpOwgBWhDZom3g7S/36Tl/T97iTbMOtpvwNvbpRHDq47vkfqelMaNrtw+crb01sno9egaJb35VOW2b2yQbr5rcMBedLN034+i512ANuQcNgBgwNkf956rA2jHEGnqFdKr+xvP+8jxyh7zLx8Z3NOWOGfdDM5hA0DBlAdN1ys9tK25YO3n3AXeddvlPw4I1s1hSDxhaX+/Scv7cKpEG2Yd7Rds7EjpyNMxViZA99zmrgsvQBtGGhKnhw0ABXX0hNfrXbYqmfyX3tF/jryJYI0B9LATlvb3m7S8984k2jDraL/6xHFHrbiHvgvQhvSwAQD1KV2PbT0Dd/Mqt2LN4G3nXFZ5HJJBDzthaX+/Sct770yiDbOO9su+ArQhPWwAAPKCgA0AQAYQsAEAyIDUVzqbMWOGentjmJbYpvJ+finv55Yk2jDraL/sy3sbRkUPGwCADEi9hw0gR3bG0BOakf8eI9AIetgAmnPwDi9QxxGspYG8Dia0/BaQUQRsAI059aYXWPd9OZn8993k5X/qYDL5AxnDkDiA+sXVm45i9zneM0PlKDh62ADq08pg3Q7lAm2CgA0gml3D0g+aO006sindOgApIWADqG2nSe7dprO54fYY6rJ3cfo/HIAUcA4bQLhdw5vOovwOTn99v/fc9G0cdw2TLvxtk5kA2UEPG0A4Vzsods+V7v2h/76g2y02fRvGGHr8QJYQsAEEqzH0XLr/cd8x6bN/1XwQLr+nsvVI5/1pc/UD8oSADcBfjWD47fv8tzcatP2Oe3lPhAMJ2igIAjaAwU4fqplk6R0tqIci/gA43Zd4PYC0EbABDPbSxNiyCppc1vSks3IvdceYGdCemCUOoNIbA9de+fVuS4HW9UYf/na90omT0qjZ0vFnpJEjoldn/VcGXofVRwfWSOfcGD1jIGPoYQOotP8vJAUH431lo+Wzpg/eH9RzLgXpoGAddNx1C73nXx3w3/9ePV9f7p8AyAkCNoC6TJk/8Hr7uspAGzbM/eGrvOdxlwanqc6r/P25C+qrJ5A3BGwAA5qccf16yFy1V17zno8cD04Tti8SZowjxwjYAOoyf1bwvsnzg/dFEdb7XnBJc3kDWUfABuDr5A7/7Y+ubW09Sh5e47/9nWdbWw8gLQRsAJ5TlbO6zhrmnUM+a9jAtiiXYm14uLHiH9pWO015+SOGe++HD61KdOpwYxUA2hwBG4Bn9/t9N5/cIZ163nsd5TKu6786eNvpM5Xv+44NTnPlitp5l8o/tlV6e3tAot0TamcEZBABG0BNHUOaO37oxZXvu+c2l9/o9zV3PJBFBGwAdYnSy160svK9c+HpP/e1eMoF8oyADSB2922pL/36zcnUA8iT2AO2mU0xs6fN7Odm9rKZfSnuMgDEb/nq6Glb3dutp7x6PgeQJUn0sE9LWuGc+58lXSzpP5nZ7yVQDoAYrY55Zc8v3BYtXdx3/Yr7cwDtIvaA7Zx7wzm3q//1CUk/lzQp7nIApGvBsvD933nAe962y3//5me856D7apdUzx6/9vLadQPyKNFz2Gb2QUm/L+n5qu1LzKzXzHoPH+aaSSALpn6g8v2jQZdVVZmzxH/7ZyL2hKuvz77H57IxoAgSC9hm9j5JD0ha5pyrWCHYOfdd51yPc66nu5v72AJZ8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+HyiSRAK2mXXKC9b3Ouf+IYkyAMRsevho1ySf9Ugeq7Es6NEaN/M4diJ8/9qN4ft9nd/XwEFA+0tilrhJWifp58455msCWdExvqHDkpoxftVNDR7YOS7WegDtIoke9ixJ10i61Mxe7H80eQ8fAEXzg61p1wBoLx1xZ+ic2y6Jm9ICOTSxSzp4JL3yZ56XXtlA2ljpDMCAGeFriB6ocwWzch/7kDT3Iul3Jzeex3MbaiSoUX8gy2LvYQPIN9cbfN56/qzm7pd92Q3SlueCywWKjIANoNLkO6V94TO+jm2VxszxXh/cIk3oqtx/3S3SPY9EL3LWdGn7Ounxuwa27d0vTbvCex2pZz/lW9ELBDKIIXEAlSbWvjF16faWrtcL1pu2eL3u0qOeYC1JO16qPH7j495CLaVe9cSu8OMlSRO+WF+hQMaYq3Xfu4T19PS43t78jnV5V7nlV9r/flqhkG146rC02+fC6ypRL+laOFu6fqE0Z4Z09IT0k93Sreuln+2JUL8o/z2c3xd4OVch2y9n8t6GknY652r+NTEkDmCwzsZXINy82gvQQcaOkqZNkq6eV7l9+4vSJZ9vsFCuvUYBELAB+JvhpJ3hPZvSBLTODundqsli9Syo4nqlj18w0JvunCmdPhOxd83McBQEARtAsAhBWxoI1o2uelZ+3JkXpFPPR8yLYI0CYdIZgHBTay/oXZos5ueWJdLRp73eculxcoe33c+QiyIG66nfj5AIyA8mnSUs75Ml0v730wq0oQJ72dWB9co50oN3Nl6XxSu9GeflAofFI/auab/sy3sbiklnAGIzw0m7RkjunUG7+p6Uxo2u3DZytvTWyejZd42S3nxK2nir95Ckb2yQbr7LJ/HUjVLXouiZAzlBwAYQzYX9Ebiqt90xRJp6hfTq/sazPnK8srf+y0cG97Qlcc4ahcY5bAD1KQuarld6aFtzwdrPuQu867YrhsMJ1ig4etgA6jfDSaeOSLvH6drLpWsvT7Cs8w81dV04kBf0sAE0prPLC9xT1iST/5S1Xv4Ea0ASPWwAzZqwzHtIka7Zromhb8AXPWwA8ZnhBh7Tjw7avcKvM37+G5XHAfBFDxtAMjrGDArAq/4upboAOUAPGwCADCBgAwCQAQRsAAAygIANAEAGpH7zDzPL9bTQtL/fpBVgUX7aMONov+wrQBty8w8AAAKdOSq92FWxacUaadWNVenO3y91vr919QpADzthaX+/SePXffblvQ1pv+yLtQ3bcHGfqD1szmEDAPLt4B1eoI4jWEsDeR1cFU9+EdHDTlja32/S+HWffXlvQ9ov+xpuw1NvSrvHx1sZP+cfkDonNnw457ABAMUVV286it3neM8JL63LkDgAIF9aGaxbWC4BGwCQD7uGpResS3aadGRTIlkTsAEA2bfTJPdu09nccHsMddm7OJEfDkw6S1ja32/SmPCSfXlvQ9ov+2q24a7hkvttU2WYz5Qv19tUlpINlS6sXS8u6wIAFEOEYN09V7r3h/77/IJ12PbIYujxl6OHnbC0v9+k8es++/LehrRf9oW2YY2h5yg957DAXCvtR6dJP70/tAo1Z4/TwwYA5FuNYP3t+/y3N9pz9jvu5T0RDozpfDYBGwCQPacP1Uyy9I4W1EMRfwCc7mu6HAI2ACB7Xmp8ZbFqQZPLmp50Vu6l7qazYKUzAEC2vDFw7VXYOWrXG3342/VKJ05Ko2ZLx5+RRo6IXp31Xxl4HXrO/MAa6ZzqW4FFRw8bAJAt+/9CUnAw3lc2Wj5r+uD9QT3nUpAOCtZBx1230Hv+1QH//e/V8/Xl/gkiImADAHJlyvyB19vXVQbasGHuD1/lPY+7NDhNdV7l789dUF8960XABgBkR5Mzrl8Pmav2ymve85HjwWnC9kXSRP0J2ACAXJk/K3jf5PnB+6II630vuKS5vGshYAMAMunkDv/tj65tbT1KHl7jv/2dZ+PJn4ANAMiGU5Wzus4a5p1DPmvYwLYol2JteLix4h/aVjtNefkjhnvvhw+tSnTqcEPlszRpwtL+fpNW+GURcyDvbUj7Zd97bRhy/vf0GalzZn96n6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdWtWK6UpUkBAIXRMaS544deXPm+e25z+YUG6wYRsAEAuRJlsZRFKyvf1xqI+dzX4im3GbEHbDMbbmYvmNlLZvaymX017jIAAGjGfVvqS79+czL1qEcSPezfSrrUOTdd0gWSPm1mF9c4BgCAUMtXR0+bdG+3mfLq+RzlYg/YzvNW/9vO/ke+Z30AABK3urmVPQf5wm3R0sV9169GP0ci57DNbIiZvSjpkKQfOeeer9q/xMx6zSzOe6EAAPCeBcvC93/nAe952y7//Zuf8Z6D7qtdcuWKyvfXXl67bo1I9LIuMxsj6UFJX3TO/TQgTa5731xSkn20YbbRftkX5bIuSZp2hbR3f9Wx/d3CoCHrWnf0CtsflHek23K222VdzrljkrZK+nSS5QAA8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+P05JzBLv7u9Zy8zOkjRX0r/GXQ4AoGCmh68QNmnC4G2P1VgW9GiNm3kcOxG+f+3G8P2+zu9r4CCpo6Gjwr1f0j1mNkTeD4L7nXOPJFAOAKBIOsY3dFhSM8avuqnBAzvHNXRY7AHbObdb0u/HnS8AAO3kB1tbWx4rnQEAcmNiV7rlzzwvuby5+UfC0v5+k1aoGao5lfc2pP2yb1Ab1pgt3ugQ+Mc+5AX8vfulX+xrLI+aM8RnDP73GHWWeBLnsAEASE3YpVjzZzV3v+zLbpC2PBdcbpII2ACAbJl8p7QvfMbXsa3SmDne64NbpAlVQ+XX3SLdU8d06FnTpe3rpMfvGti2d7937bckHYiyNvmUb0Uv0AdD4glL+/tNWiGH43Im721I+2WfbxvWGBaXvF52qde7aYu0eGV4+np87+vS4ssGlxPKZzhcij4kTsBOWNrfb9IK+59FjuS9DWm/7PNtw1OHpd0+F15XiXo+e+Fs6fqF0pwZ0tET0k92S7eul362J0L9ogTr8/sCL+fiHDYAIL86uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQhu89rocPeyEpf39Jq2wv+5zJO9tSPtlX2gbRhwa7+yQ3n1u8PbIdajqRXfOlE6faW4o/L160MMGAOTeDBcpaJeCdaOXfJUfd+YF6dTzEfOqEazrwcIpAIBsm1pjBNUvAAAgAElEQVR7QW/rCQ6wtyyRjj7t9ZZLj5M7vO1+hlwUMVhP/X6ERNExJJ6wtL/fpBV+OC4H8t6GtF/2RWrDgF52dWC9co704J2N12XxSm/GebnAYfGIvWtmibeJtL/fpPGfRfblvQ1pv+yL3Ia7RkjunYpN1iP1PSmNG12ZdORs6a2T0evQNUp686nKbd/YIN18l0/AnrpR6loUOW/OYQMAiuXC/ghc1dvuGCJNvUJ6dX/jWR85Xtlb/+Ujg3vakmI9Z12Nc9gAgHwpC5quV3poW3PB2s+5C7zrtit61wkGa4kh8cSl/f0mjeG47Mt7G9J+2ddwG546Iu1u/vrnms4/1NR14VGHxOlhAwDyqbPL6/VOWZNM/lPWevk3EazrQQ87YWl/v0nj13325b0Nab/si7UNI1yzXVPMQ9/0sAEAqDbDDTymHx20e4VfZ/z8NyqPSwk97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyIPWVzmbMmKHe3ij3J8umvJ9fyvu5JYk2zDraL/vy3oZR0cMGACADUu9hI7pIN0qvodF7wQIA0kUPu83ddM3A/VnjUMpr+dXx5AcAaA0CdpvqGuUF1ju+lEz+q2708p/QlUz+AIB4MSTehuLqTUdxsP/2cAyVA0B7o4fdZloZrNuhXABANATsNvGbZ9MPmq5X+rNPpVsHAIA/AnYbcL3SsKHN53PD7c3nsem29H84AAAG4xx2yt7Z0Xwe5eef//p+77nZoPubZ6Xhf9RcHgCA+NDDTtnwYbXTdM+V7v2h/76gyWLNTiKLo8cPAIgPATtFtXrB1uM9+o5Jn/2r5oNwKb/S47w/ba5+AIDWIWCnpFYw/PZ9/tsbDdp+x728p/ZxBG0AaA8E7BR0R1isZOkdyddDivYDYNzo5OsBAAhHwE7BoS3x5RXUA46zZ9z3ZHx5AQAawyzxFvvzawZe+/VuS4HW9UYf/na90omT0qjZ0vFnpJEjotdn/Vei1WfZYumbG6PnCwCIFz3sFru9f23woGC879DA61nTB+8P6jmXgnRQsA467rqF3vOvDvjvL9VzzQr//QCA1iBgt5kp8wdeb19XGWjDhrk/fJX3PO7S4DTVeZW/P3dBffUEALQWAbuFmj2v/Pqh4H2vvOY9HzkenCZsXxTMGAeA9BCw28z8WcH7Js8P3hdFWO97wSXN5Q0ASBYBOyUnA5YkfXRta+tR8vAa/+3vPNvaegAA/BGwW2TiuMr3Zw3zhpjPKluaNMqQ84aHGyv/oW2105SXP2K493541RKl48c0Vj4AoDkE7BY58Lj/9pM7pFPPe6+jXMZ1/VcHbzt9pvJ937HBaa6MMMu7VP6xrdLb2/3THH6idj4AgPgRsNtAx5Dmjh96ceX77rnN5Tf6fc0dDwCIHwG7zUTpZS9aWfneufD0n/taPOUCANKTSMA2syFm9s9m9kgS+RfdfXUubbp+czL1AAC0TlI97C9J+nlCeWfS8tXR07a6t1tPefV8DgBAfGIP2GY2WdLlku6OO+8sW7083vy+cFu0dHHf9SvuzwEAiCaJHvY3JX1Z0v8ISmBmS8ys18x6Dx8+nEAVsm/BsvD933nAe962y3//5me856D7apdUzx6/9vLadQMAtF6sAdvMFkg65JzbGZbOOfdd51yPc66nu7s7zipk1tQPVL5/NOCyqmpzlvhv/0zEnnD19dn3+Fw2BgBIX9w97FmSrjCzVyVtknSpmf1dzGXk0o99TiDMWxp+TFfIUqOSNPYT4fuXrQrfDwBoH7EGbOfczc65yc65D0paJOkp59xn4ywjq8Z/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXIAQHK4DrtF3vx1Y8clNWP8qpsaO67ZO34BABrTkVTGzrmtkrYmlT+a84OtadcAAFAPethtZGJXuuXPPC/d8gEAwQjYLVRrePtAnSuYlfvYh6S5F0m/O7nxPJ7bEL6f5UsBID2JDYmjMa43ODDOn9Xc/bIvu0Ha8lxwuQCA9kXAbrEVa6RVN4anObZVGjPHe31wizShaqj8uluke+pYpX3WdGn7Ounxuwa27d0vTbvCex2lZ//FmFdMAwDUx1ytWz0lrKenx/X25rd7Z2aDtkXpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP2v59W8GvDPMl7G9J+2Zf3NpS00zlX86QjATthfv/Qxo+RDj8R4diI54wXzpauXyjNmSEdPSH9ZLd063rpZ3tqHxslWI+7NPhyrrT//bRC3v+zyHsb0n7Zl/c2VMSAzZB4CvqONX7s5tVegA4ydpQ0bZJ09bzK7dtflC75fGNlcu01AKSPgJ2SKEPRpQlonR3Su1WTxeqZse16pY9fMFBe50zp9JnmhsIBAK1FwE5R1PPHpWDdaPAsP+7MC9Kp56PlRbAGgPbBddgpW3Rz7TTWExw8b1kiHX3aC/ylx8kd3nY/Qy6KFoj/5Mu10wAAWodJZwmLMlkiqJddHVivnCM9eGfjdVm80ptx3kjZQdL+99MKeZ/wkvc2pP2yL+9tKCadZYf1SG9vl0YMH7yv70lp3OjKbSNnS2+djJ5/1yjpzaekjbd6D0n6xgbp5rsGp110s3Tfj6LnDQBoDQJ2mzj7495zdY+3Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBoB2xjnsNlMeNF2v9NC25oK1n3MXeNdtl/84IFgDQHujh92GrEcaO1I68rR07eXeIyndc5u7LhwA0Br0sNvU0RNe4F62Kpn8l97h5U+wBoBsoIfd5tZu9B5SPHfUYugbALKJHnaGlK7Htp6Bu3mVW7Fm8LZzLqs8DgCQTfSwM+rXb/kH4NX3tr4uAIDk0cMGACADCNgAAGQAARsAgAxIfS1xM8v1Qrhpf79JK8Aav7RhxtF+2VeANoy0ljg9bAAAMoBZ4kCr7IyhJzQj3z0NAMHoYQNJOniHF6jjCNbSQF4HE1oCD0Db4hx2wtL+fpPG+bMAp96Udo+PvzLVzj8gdU5sKou8tyF/g9lXgDbkfthAKuLqTUex+xzvmaFyIPcYEgfi1Mpg3Q7lAmgZAjYQh13D0g+aO006sindOgBIDAEbaNZOk9y7TWdzw+0x1GXv4vR/OABIBJPOEpb295u0wk942TVccr9tKn+/m7g0fStVGypdGK1eeW9D/gazrwBtyMIpQOIiBOvuudK9P/TfF3TL06ZvhRpDjx9Ae6GHnbC0v9+kFfrXfY2h5yg957DAXCvtR6dJP70/tAqRZo/nvQ35G8y+ArQhPWwgMTWC9bfv89/eaM/Z77iX90Q4kPPZQG4QsIF6nT5UM8nSO1pQD0X8AXC6L/F6AEgeARuo10vNrSxWLmhyWdOTzsq91B1jZgDSwkpnQD3eGLj2KuwcteuNPvzteqUTJ6VRs6Xjz0gjR0SvzvqvDLwOPWd+YI10zo3RMwbQduhhA/XY/xeSgoPxvrLR8lnTB+8P6jmXgnRQsA467rqF3vOvDvjvf6+ery/3TwAgMwjYQIymzB94vX1dZaANG+b+8FXe87hLg9NU51X+/twF9dUTQPYQsIGompxx/XrIXLVXXvOejxwPThO2LxJmjAOZRsAGYjR/VvC+yfOD90UR1vtecElzeQNofwRsoAEnd/hvf3Rta+tR8vAa/+3vPNvaegBIDgEbiOJU5ayus4Z555DPGjawLcqlWBsebqz4h7bVTlNe/ojh3vvhQ6sSnTrcWAUApI6lSROW9vebtMIsixhy/vf0GalzZn9an6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdUdtFxp3tuQv8HsK0AbsjQp0AodQ5o7fujFle+75zaXX2iwBpBZBGwgRlEWS1m0svJ9rc7D574WT7kAsi2RgG1mr5rZv5jZi2YW5yKLQObdt6W+9Os3J1MPANmSZA/7E865C6KMywPtbvnq6Glb3dutp7x6PgeA9sKQOBDB6phX9vzCbdHSxX3Xr7g/B4DWSSpgO0lbzGynmS2p3mlmS8ysl+Fy5NWCZeH7v/OA97xtl//+zc94z0H31S65ckXl+2svr103ANmUyGVdZvYB59x+M5sg6UeSvuiceyYgba7n6xfgcoS0q5C4Wpd1SdK0K6S9+6uO6/85GjRkXeuOXmH7g/KOdFtOLuvKlby3n1SINkzvsi7n3P7+50OSHpR0URLlAO3ix3cP3jZvafgxXSFLjUrS2E+E71+2Knw/gHyJPWCb2dlmNrL0WtIfS/pp3OUALTU9fIWwSRMGb3usxrKgR2vczOPYifD9azeG7/d1fl8DBwFoBx0J5DlR0oP9wzQdkr7nnHssgXKA1ukY39BhSc0Yv+qmBg/sHBdrPQC0TuwB2zm3R9L0uPMFMOAHW9OuAYBW47IuICYTu9Itf+Z56ZYPIFnc/CNhaX+/SSvcDNUas8UbHQL/2Ie8gL93v/SLfY3lUXOG+Az/f4t5b0P+BrOvAG0YaZZ4EuewgcIKuxRr/qzm7pd92Q3SlueCywWQbwRsoB6T75T2hc/4OrZVGjPHe31wizShaqj8ulukex6JXuSs6dL2ddLjdw1s27vfu/Zbkg5EWZt8yreiFwigLTEknrC0v9+kFXI4rsawuOT1sku93k1bpMUrw9PX43tflxZfNricUAHD4VL+25C/wewrQBtGGhInYCcs7e83aYX8z+LUYWm3z4XXVaKez144W7p+oTRnhnT0hPST3dKt66Wf7YlQtyjB+vy+0Mu58t6G/A1mXwHakHPYQCI6uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQrn2GsgFetgJS/v7TVqhf91HHBrv7JDefW7w9sjlV/WiO2dKp880PxT+Xl1y3ob8DWZfAdqQHjaQqBm1bwoiDQTrRi/5Kj/uzAvSqecj5hUhWAPIDhZOAZoxtfaC3tYTHGBvWSIdfdrrLZceJ3d42/0MuShisJ76/QiJAGQJQ+IJS/v7TRrDcQrsZVcH1ivnSA/e2Xg9Fq/0ZpxX1C1oWLyO3nXe25C/wewrQBsyS7wdpP39Jo3/LPrtGiG5dyo2WY/U96Q0bnRl0pGzpbdORi+/a5T05lOV276xQbr5Lp+APXWj1LUoeubKfxvyN5h9BWhDzmEDLXNhfwSu6m13DJGmXiG9ur/xrI8cr+yt//KRwT1tSZyzBnKOc9hAnMqCpuuVHtrWXLD2c+4C77rtit41wRrIPYbEE5b295s0huMCnDoi7W7B9c/nH2rqunAp/23I32D2FaANIw2J08MGktDZ5fV6p6xJJv8pa738mwzWALKDHnbC0v5+k8av+zpEuGa7pgSGvvPehvwNZl8B2pAeNtBWZriBx/Sjg3av8OuMn/9G5XEACosedsLS/n6Txq/77Mt7G9J+2VeANqSHDQBAXhCwAQDIAAI2AAAZkPpKZzNmzFBvb5T7BGZT3s8v5f3ckkQbZh3tl315b8Oo6GEDAJABBGwAADIg9SFxAMiKwNuZ1iHS/cwBH/SwASDETdd4gTqOYC0N5LX86njyQ3EQsAHAR9coL7De8aVk8l91o5f/hK5k8kf+MCQOAFXi6k1HcbD/3uYMlaMWetgAUKaVwbodykV2ELABQNJvnk0/aLpe6c8+lW4d0L4I2AAKz/VKw4Y2n88Ntzefx6bb0v/hgPbEOWwAhfbOjubzKD///Nf3e8/NBt3fPCsN/6Pm8kC+0MMGUGjDh9VO0z1XuveH/vuCJos1O4ksjh4/8oWADaCwavWCrcd79B2TPvtXzQfhUn6lx3l/2lz9UCwEbACFVCsYfvs+/+2NBm2/417eU/s4gjZKCNgACqc7wmIlS+9Ivh5StB8A40YnXw+0PwI2gMI5tCW+vIJ6wHH2jPuejC8vZBezxAEUyp9fM/Dar3dbCrSuN/rwt+uVTpyURs2Wjj8jjRwRvT7rvxKtPssWS9/cGD1f5A89bACFcnv/2uBBwXjfoYHXs6YP3h/Ucy4F6aBgHXTcdQu9518d8N9fqueaFf77URwEbAAoM2X+wOvt6yoDbdgw94ev8p7HXRqcpjqv8vfnLqivnigeAjaAwmj2vPLrh4L3vfKa93zkeHCasH1RMGO82AjYAFBm/qzgfZPnB++LIqz3veCS5vJG/hGwARTSyYAlSR9d29p6lDy8xn/7O8+2th5oXwRsAIUwcVzl+7OGeUPMZ5UtTRplyHnDw42V/9C22mnKyx8x3Hs/vGqJ0vFjGisf2UfABlAIBx73335yh3Tqee91lMu4rv/q4G2nz1S+7zs2OM2VEWZ5l8o/tlV6e7t/msNP1M4H+UTABlB4HUOaO37oxZXvu+c2l9/o9zV3PPIpkYBtZmPM7O/N7F/N7Odm9odJlAMAcYvSy160svK9c+HpP/e1eMpFsSXVw14r6THn3L+TNF3SzxMqBwBa7r46lzZdvzmZeqBYYg/YZjZK0mxJ6yTJOfeuc87njA4AtM7y1dHTtrq3W0959XwO5EsSPexpkg5LWm9m/2xmd5vZ2QmUAwCRrV4eb35fuC1aurjv+hX350B2JBGwOyRdKOlvnHO/L+ltSX9ZnsDMlphZr5n1Hj58OIEqAEBzFiwL3/+dB7znbbv8929+xnsOuq92SfXs8Wsvr103FFMSAXufpH3Ouf4LJfT38gL4e5xz33XO9Tjnerq7uxOoAgDUZ+oHKt8/GnBZVbU5S/y3fyZiT7j6+ux7fC4bA6QEArZz7oCk18zsI/2bPinpZ3GXAwBx+vHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfD5RLapb4FyXda2a7JV0g6daEygGASMZ/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXLkW0cSmTrnXpTEVYUA2sabv27suKRmjF91U2PHNXvHL2QXK50BQAp+sDXtGiBrCNgA0G9iV7rlzzwv3fLR3gjYAAqj1vD2gTpXMCv3sQ9Jcy+Sfndy43k8tyF8P8uXFlsi57ABIKtcb3BgnD+ruftlX3aDtOW54HKBMARsAIWyYo206sbwNMe2SmPmeK8PbpEmVA2VX3eLdM8j0cucNV3avk56/K6BbXv3S9Ou8F5H6dl/MeYV05A95mrdZiZhPT09rrc3vz8tzSztKiQq7X8/rUAbZptf+0XpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP39pPy/zcoaadzruYJDwJ2wvL+Dy3tfz+tQBtmm1/7jR8jHX4iwrERzxkvnC1dv1CaM0M6ekL6yW7p1vXSz/bUPjZKsB53afDlXHlvPyn/f4OKGLAZEgdQOH1N3D9w82ovQAcZO0qaNkm6el7l9u0vSpd8vrEyufYaEgEbQEFFGYouTUDr7JDerZosVs+MbdcrffyCgfI6Z0qnzzQ3FI7iIWADKKyo549LwbrR4Fl+3JkXpFPPR8uLYI1yXIcNoNAW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00KBYmnSUs75Ml0v730wq0YbZFab+gXnZ1YL1yjvTgnY3XZfFKb8Z5I2UHyXv7Sfn/GxSTzgAgGuuR3t4ujRg+eF/fk9K40ZXbRs6W3joZPf+uUdKbT0kbb/UekvSNDdLNdw1Ou+hm6b4fRc8bxUHABgBJZ3/ce67u8XYMkaZeIb26v/G8jxyv7DH/8pHBPW2Jc9YIxzlsAChTHjRdr/TQtuaCtZ9zF3jXbZf/OCBYoxZ62ABQxXqksSOlI09L117uPZLSPbe568JRHPSwAcDH0RNe4F62Kpn8l97h5U+wRlT0sAEgxNqN3kOK545aDH2jUfSwASCi0vXY1jNwN69yK9YM3nbOZZXHAY2ihw0ADfj1W/4BePW9ra8LioEeNgAAGUDABgAgAwjYAABkQOpriZtZrhfCTfv7TVoB1vilDTOO9su+ArRhpLXE6WEDAJABzBJH2+AaVwAIRg8bqbrpmoF7CMehlNfyq+PJDwDaBeewE5b295u0Rs+flW43mLSJfywdOtJcHrRhttF+2VeANuR+2GhPcfWmozjYfwtDhsoBZB1D4mipVgbrdigXAOJCwEZL/ObZ9IOm65X+7FPp1gEAGkXARuJcrzRsaPP53HB783lsui39Hw4A0AgmnSUs7e83abUmvLyzQxo+rMkyfM4/Nxt0f/uuNPyPoqUtehtmHe2XfQVoQxZOQfqiBOvuudK9P/TfFzRZrNlJZHH0+AGglehhJyzt7zdpYb/ua/WCo/ScwwJzrbQfnSb99P766zConAK3YR7QftlXgDakh4301ArW377Pf3ujPWe/417eU/s4zmcDyAoCNmLX3VU7zdI7kq+HFO0HwLjRydcDAJpFwEbsDm2JL6+gHnCcPeO+J+PLCwCSwkpniNWfXzPwOuwcteuNPvzteqUTJ6VRs6Xjz0gjR0Svz/qvRKvPssXSNzdGzxcAWo0eNmJ1+5e856BgvO/QwOtZ0wfvD+o5l4J0ULAOOu66hd7zrw747y/Vc80K//0A0C4I2GipKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BIB2Q8BGbJo9r/z6oeB9r7zmPR85HpwmbF8UzBgH0M4I2Gip+bOC902eH7wvirDe94JLmssbANJGwEYiTu7w3/7o2tbWo+ThNf7b33m2tfUAgEYRsBGLieMq3581zBtiPqtsadIoQ84bHm6s/Ie21U5TXv6I4d774VVLlI4f01j5AJA0liZNWNrfb9JKyyKGBePTZ6TOmQpMVz2jvDpN+fGSdPiJwYG1Vh7laY5tlUa/L7i+g/IqSBvmFe2XfQVoQ5YmRXvoGNLc8UMvrnzfPbe5/MKCNQC0KwI2WirKYimLVla+r/Xj+nNfi6dcAGhnsQdsM/uImb1Y9jhuZsviLgf5dV+dS5uu35xMPQCgncQesJ1z/+acu8A5d4GkGZJOSnow7nLQXpavjp621b3desqr53MAQCslPST+SUm/cM79MuFykLLVy+PN7wu3RUsX912/4v4cABCXpAP2IkmDbqlgZkvMrNfMWFuqoBbUOEnynQe85227/PdvfsZ7DrqvdsmVVWuEX3t57boBQDtK7LIuMxsqab+kjzrnDoaky/V8/QJcjiCp9jXW066Q9u6v3FY6JmjIutYdvcL2B+Ud5VpwLuvKF9ov+wrQhqlf1jVP0q6wYI3i+PHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfDwBZkmTAXiyf4XDk0/hPhu+fNGHwtsdqLAt6tMbNPI6dCN+/toF/fWHrkQNAmhIJ2GY2QtKnJP1DEvmj/bz568aOS2rG+FU3NXZcs3f8AoCkdCSRqXPupKRxNRMCCfnB1rRrAADxYqUztMzErnTLn3leuuUDQDO4+UfC0v5+k1Y9Q7XWLOxGh8A/9iEv4O/dL/1iX2N5NFq3orVh3tB+2VeANow0SzyRIXEgSNilWPNnNXe/7MtukLY8F1wuAGQZARuxWrFGWnVjeJpjW6Uxc7zXB7dIE6qGyq+7RbrnkehlzpoubV8nPX7XwLa9+71rvyXpQIS1yb8Y84ppABA3hsQTlvb3mzS/4bioi5OU0m3aIi1eGZ6+Ht/7urT4ssHl1KpPkCK2YZ7QftlXgDaMNCROwE5Y2t9v0vz+sxg/Rjr8RIRjI57PXjhbun6hNGeGdPSE9JPd0q3rpZ/tqX1slGA97tLwy7mK2IZ5QvtlXwHakHPYSEffscaP3bzaC9BBxo6Spk2Srp5XuX37i9Iln2+sTK69BpAF9LATlvb3m7SwX/dRh6I7O6R3nxu8ParqcjpnSqfPND8U/l7+BW7DPKD9sq8AbUgPG+mKev64FKwbveSr/LgzL0inno+WV6vvyw0AzWDhFCRq0c2101hPcPC8ZYl09Gkv8JceJ3d42/0MuShaIP6TL9dOAwDthCHxhKX9/SYtynBcUC+7OrBeOUd68M7G67J4pTfjvJGyw9CG2Ub7ZV8B2pBZ4u0g7e83aVH/s3h7uzRieNWxPVLfk9K40ZXbR86W3joZvQ5do6Q3n6rc9o0N0s13DQ7Yi26W7vtR9Lwl2jDraL/sK0Abcg4b7ePsj3vP1QG0Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBpBtnMNGS5UHTdcrPbStuWDt59wF3nXb5T8OCNYAso4h8YSl/f0mrdHhuLEjpSNPx1wZH91zm7suXKINs472y74CtGGkIXF62EjF0RNer3fZqmTyX3pH/znyJoM1ALQLetgJS/v7TVqcv+7juKNWEkPftGG20X7ZV4A2pIeNbCldj209A3fzKrdizeBt51xWeRwA5BU97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyoB1WOuuT9MsWlje+v8yWSOn8Uks/Ywry3oa0X4xov9i1/PMVoA3PjZIo9UlnrWZmvVFO7mdZ3j8jny/b+HzZlvfPJ7XvZ2RIHACADCBgAwCQAUUM2N9NuwItkPfPyOfLNj5ftuX980lt+hkLdw4bAIAsKmIPGwCAzCFgAwCQAYUK2Gb2aTP7NzN7xcz+Mu36xMnM/tbMDpnZT9OuSxLMbIqZPW1mPzezl83sS2nXKW5mNtzMXjCzl/o/41fTrlPczGyImf2zmT2Sdl2SYGavmtm/mNmLZhbD/efai5mNMbO/N7N/7f9b/MO06xQXM/tIf7uVHsfNbFna9SpXmHPYZjZE0v8n6VOS9kn6J0mLnXM/S7ViMTGz2ZLekvTfnHPnpV2fuJnZ+yW93zm3y8xGStop6cq8tJ8kmbc6xNnOubfMrFPSdklfcs49l3LVYmNmyyX1SBrlnFuQdn3iZmavSupxzuVy4RQzu0fSj51zd5vZUEkjnHO5u+t8f7x4XdJM51wrF/YKVaQe9kWSXnHO7XHOvStpk6TPpFyn2DjnnpF0JO16JMU594Zzblf/6xOSfi5pUrq1ipfzvNX/tl2ok/MAAAJgSURBVLP/kZtf1GY2WdLlku5Ouy6on5mNkjRb0jpJcs69m8dg3e+Tkn7RTsFaKlbAniTptbL3+5Sz//CLwsw+KOn3JT2fbk3i1z9k/KKkQ5J+5JzL02f8pqQvS/ofaVckQU7SFjPbaWZL0q5MzKZJOixpff9pjbvN7Oy0K5WQRZI2pl2JakUK2H6L0eam91IUZvY+SQ9IWuacO552feLmnDvjnLtA0mRJF5lZLk5vmNkCSYecczvTrkvCZjnnLpQ0T9J/6j9VlRcdki6U9DfOud+X9LakXM0FkqT+of4rJH0/7bpUK1LA3idpStn7yZL2p1QXNKD/vO4Dku51zv1D2vVJUv9Q41ZJn065KnGZJemK/nO8myRdamZ/l26V4uec29//fEjSg/JOxeXFPkn7ykZ9/l5eAM+beZJ2OecOpl2RakUK2P8k6cNmNrX/F9QiSZtTrhMi6p+QtU7Sz51zq9OuTxLMrNvMxvS/PkvSXEn/mm6t4uGcu9k5N9k590F5f3tPOec+m3K1YmVmZ/dPiFT/UPEfS8rNVRvOuQOSXjOzj/Rv+qSk3Ez6LLNYbTgcLrXH7TVbwjl32sxukPS4pCGS/tY593LK1YqNmW2UNEfSeDPbJ+krzrl16dYqVrMkXSPpX/rP8UrSSufcP6ZYp7i9X9I9/TNUf0fS/c65XF7+lFMTJT3YfyvIDknfc849lm6VYvdFSff2d3r2SLo+5frEysxGyLuS6D+mXRc/hbmsCwCALCvSkDgAAJlFwAYAIAMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAG/P+uMuaa/akHvAAAAABJRU5ErkJggg==", "text/plain": [ "" ] @@ -5430,7 +5433,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FNWd7/vPd9gbEMOvDRtMgGtg\nkifnTow4skecIXKJIWNAMHru3Bm4Ro/m5nJu7jEEwcmMPM88MXlONFcFQuLcycmRAc8ZA5pxjKgT\nJf4AA0adDaNMTGbuY8BERH5sgYBiInDW/aN2u7t7V1VXd1d1dVW9X8/TT3dXrVprdS82316rVq0y\n55wAAEB7+520KwAAAGojYAMAkAEEbAAAMoCADQBABhCwAQDIAAI2AAAZQMAGACADCNgAAGQAARto\nM2b2QTP7RzM7amYHzOwuM+sIST/GzP6mP+1JM/sXM/sPrawzgOQRsIH28/9KOiTp/ZIukPS/SPq/\n/RKa2VBJT0g6V9IfShot6c8l3W5mS1tSWwAtQcAG2s9USfc7537jnDsg6TFJHw1Ie42k/0nS/+ac\n2+ucO+Wce0zSUkn/2cxGSpKZOTP7UOkgM9tgZv+57P0CM3vRzI6Z2bNmdn7Zvg+Y2QNmdtjM9pb/\nEDCzW8zsfjP7b2Z2wsxeNrOesv1/YWav9+/7NzP7ZDxfEVA8BGyg/ayVtMjMRpjZJEnz5AVtP5+S\n9EPn3NtV2x+QNELSxbUKM7MLJf2tpP8oaZyk/yJps5kNM7PfkfSwpJckTZL0SUnLzOyysiyukLRJ\n0hhJmyXd1Z/vRyTdIOkPnHMjJV0m6dVa9QHgj4ANtJ9t8nrUxyXtk9Qr6QcBacdLeqN6o3PutKQ+\nSd0Ryvs/Jf0X59zzzrkzzrl7JP1WXrD/A0ndzrmvOefedc7tkfRfJS0qO367c+4fnXNnJP13SdP7\nt5+RNEzS75lZp3PuVefcLyLUB4APAjbQRvp7tI9L+gdJZ8sLyGMl/T8Bh/TJO9ddnU9H/7GHIxR7\nrqQV/cPhx8zsmKQpkj7Qv+8DVftWSppYdvyBstcnJQ03sw7n3CuSlkm6RdIhM9tkZh+IUB8APgjY\nQHvpkhcs73LO/dY596ak9ZLmB6R/QtI8Mzu7avv/KumUpBf635+UN0Reck7Z69ckfd05N6bsMcI5\nt7F/396qfSOdc0H1qeCc+55z7uPyAr9T8A8PADUQsIE24pzrk7RX0hfMrMPMxkj6D/LOIfv57/KG\nzb/ffzlYZ//55W9Jut059+v+dC9K+t/NbIiZfVrezPOS/yrp/zKzmeY528wu75+w9oKk4/2Tx87q\nP/48M/uDWp/FzD5iZpea2TBJv5H0jrxhcgANIGAD7effS/q0vOHsVySdlnSjX0Ln3G8lzZXXE35e\nXlB8TNI3JX21LOmXJC2UdEzS1So7J+6c65V3HvsuSUf7y7yuf9+Z/uMukPdDok/S3fIuH6tlmKRv\n9B9zQNIEecPpABpgzrm06wAgJmbWKemHkl6XdJ3jDxzIDXrYQI44507JO3/9C0kfSbk6AGJEDxsA\ngAyghw0AQAYE3lCgVcaPH+8++MEPpl2NxOzcuTPtKiRqxowZaVchcbRhttF+2Zf3NpTU55yruchR\n6kPiPT09rre3t/mMdlrzecyI/7swi6FebSztfz+tQBtmG+2XfXlvQ0k7nXM9tRJle0j84B1eoI4j\nWEsDeR1cFU9+AADEJJsB+9SbXmDd9+Vk8t93k5f/qYPJ5A8AQJ1SP4ddt7h601Hs7l+9MYGhcgAA\n6pGtHnYrg3U7lAsAQL9sBOxdw9IPmjtNOrIp3ToAAAqr/QP2TpPcu01nc8PtMdRl7+L0fzgAAAqp\nvc9h7xredBZWNlH+r+/3nl2zV5HtGiZd+NsmMwEAILr27mG72kGxe6507w/991nAVW1B2yOLoccP\nAEA92jdg1xh6th7v0XdM+uxfNR+ES/mVHuf9aXP1AwAgTu0ZsGsEw2/f57+90aDtd9zLeyIcSNAG\nALRI+wXs04dqJll6RwvqoYg/AE73JV4PAADaL2C/NDG2rIImlzU96azcSzXXawcAoGntNUv8jYFr\nr/x6t6VA63qjD3+7XunESWnUbOn4M9LIEdGrs/4rA6/D6qMDa6RzboyeMQAAdWqvHvb+v5AUHIz3\nlY2Wz5o+eH9Qz7kUpIOCddBx1y30nn91wH//e/V8fbl/AgAAYtJeAbuGKfMHXm9fVxlow4a5P3yV\n9zzu0uA01XmVvz93QX31BAAgbu0TsJuccf16yFy1V17zno8cD04Tti8SZowDABLUPgE7gvmzgvdN\nnh+8L4qw3veCS5rLGwCAZrVlwD65w3/7o2tbW4+Sh9f4b3/n2dbWAwBQXO0RsE9Vzuo6a5h3Dvms\nYQPbolyKteHhxop/aFvtNOXljxjuvR8+tCrRqcONVQAAgBraI2Dvfr/v5pM7pFPPe6+jXMZ1/VcH\nbzt9pvJ937HBaa5cUTvvUvnHtkpvbw9ItHtC7YwAAGhAewTsEB1Dmjt+6MWV77vnNpff6Pc1dzwA\nAI1o+4BdLkove9HKyvfOhaf/3NfiKRcAgCRlKmBHcd+W+tKv35xMPQAAiFMiAdvMPm1m/2Zmr5jZ\nX9ZKv3x1HXm3uLdbT3n1fA4AAOoRe8A2syGS/lrSPEm/J2mxmf1e2DGrY17Z8wu3RUsX912/4v4c\nAACUJNHDvkjSK865Pc65dyVtkvSZOAtYsCx8/3ce8J637fLfv/kZ7znovtol1bPHr728dt0AAEhC\nEgF7kqTXyt7v69/2HjNbYma9ZtZ7+HDta5enfqDy/aNBl1VVmbPEf/tnIvaEq6/PvsfnsjEAAFoh\niYDtt6h2xVxt59x3nXM9zrme7u7a95P+8d2Dt81bGn5MV8hSo5I09hPh+5etCt8PAEArJRGw90ma\nUvZ+sqT9oUdMD+9lT/JZj+SxGsuCHq1xM49jJ8L3r90Yvt/X+X0NHAQAQG1JBOx/kvRhM5tqZkMl\nLZIUfvFUx/iGCkpqxvhVNzV4YOe4WOsBAEBJR9wZOudOm9kNkh6XNETS3zrnXo67nCT9YGvaNQAA\noFLsAVuSnHP/KOkf48xzYpd08EicOdZn5nnplQ0AQPusdDYjfA3RA3WuYFbuYx+S5l4k/e7kxvN4\nbkONBDXqDwBAMxLpYSfF9Qaft54/q7n7ZV92g7TlueByAQBIU3sF7Ml3SvvCZ3wd2yqNmeO9PrhF\nmtBVuf+6W6R7Hole5Kzp0vZ10uN3DWzbu1+adoX3OlLPfsq3ohcIAEAD2mdIXJIm1r4xden2lq7X\nC9abtni97tKjnmAtSTteqjx+4+PeQi2lXvXErvDjJUkTvlhfoQAA1MlcrftPJqynp8f19paNOZ86\nLO32ufC6StRLuhbOlq5fKM2ZIR09If1kt3Treulne2ofG2ko/Py+0Mu5zPzWkcmPtP/9tAJtmG20\nX/blvQ0l7XTO1Yxq7TUkLkmdtVc+C7J5tRegg4wdJU2bJF09r3L79helSz7fYKFcew0AaIH2C9iS\nN+N6Z/gvqtIEtM4O6d2qyWL1LKjieqWPXzDQm+6cKZ0+E7F3zcxwAECLtGfAliIFbWkgWDe66ln5\ncWdekE49HzEvgjUAoIXaa9JZtam1F/QuTRbzc8sS6ejTXm+59Di5w9vuZ8hFEYP11O9HSAQAQHza\nb9JZtYBednVgvXKO9OCdjddj8Upvxnm5wGHxOnrXeZ8skfa/n1agDbON9su+vLehMjvprNoMJ+0a\nIbl3Bu3qe1IaN7py28jZ0lsno2ffNUp68ylp463eQ5K+sUG6+S6fxFM3Sl2LomcOAEBM2j9gS9KF\n/RG4qrfdMUSaeoX0avjNO0MdOV7ZW//lI4N72pI4Zw0ASFV7n8OuVhY0Xa/00LbmgrWfcxd4121X\nDIcTrAEAKctGD7vcDCedOiLtHqdrL5euvTzBss4/1NR14QAAxCVbPeySzi4vcE9Zk0z+U9Z6+ROs\nAQBtIns97HITlnkPKdI12zUx9A0AaFPZ7GH7meEGHtOPDtq9wq8zfv4blccBANCmst3DDtIxZlAA\nXvV3KdUFAIAY5KeHDQBAjhGwAQDIAAI2AAAZQMAGACADUr/5h5nlenp22t9v0gqwKD9tmHG0X/YV\noA0j3fyDHjYAwNeYkZW3J3a90vKrB287Z1zaNS0GetgJS/v7TRq/7rMv721I+9Un8LbCdai+/XGz\nCtCG9LABALXddM1AbzkO5b1xxIcedsLS/n6TlvfemUQbZh3tF6xrlPTmUzFWJsDEP5YOHWn8+AK0\nYaQedj5XOgMAhIqrNx3FwS3ec9xD5UXDkDgAFEwrg3U7lJsXBGwAKIjfPJt+0HS90p99Kt06ZBUB\nGwAKwPVKw4Y2n88Ntzefx6bb0v/hkEVMOktY2t9v0vI+YUmiDbOO9pPe2SENH9ZkOT7nn5sNur99\nVxr+R7XTFaANuawLABAtWHfPle79of++oMlizU4ii6PHXyT0sBOW9vebtLz3ziTaMOuK3n61esFR\nes5hgblW2o9Ok356f/11qCgj/21IDxsAiqxWsP72ff7bG+05+x338p7ax3E+OxoCNgDkUHdX7TRL\n70i+HlK0HwDjRidfj6wjYANADh3aEl9eQT3gOHvGfU/Gl1desdIZAOTMn18z8DrsHLXrjT787Xql\nEyelUbOl489II0dEr8/6r0Srz7LF0jc3Rs+3aOhhA0DO3P4l7zkoGO87NPB61vTB+4N6zqUgHRSs\ng467bqH3/KsD/vtL9Vyzwn8/PARsACiYKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31RCUC\nNgDkSLPnlV8/FLzvlde85yPHg9OE7YuCGePBCNgAUDDzZwXvmzw/eF8UYb3vBZc0l3fREbABIKdO\n7vDf/uja1taj5OE1/tvfeba19cgqAjYA5MTEcZXvzxrmDTGfVbY0aZQh5w0PN1b+Q9tqpykvf8Rw\n7/3wqiVKx49prPy8Y2nShKX9/SYt78taSrRh1hWp/cKC8ekzUufM4HTVM8qr05QfL0mHnxgcWGvl\nUZ7m2FZp9PuC61ueVwHakKVJAQCejiHNHT/04sr33XObyy8sWMMfARsACibKYimLVla+r9XJ/dzX\n4ikXwWIP2Gb2t2Z2yMx+GnfeAIDWuK/OpU3Xb06mHhiQRA97g6RPJ5AvACDE8tXR07a6t1tPefV8\njiKJPWA7556RdCTufAEA4VYvjze/L9wWLV3cd/2K+3PkBeewAaCgFiwL3/+dB7znbbv8929+xnsO\nuq92yZVVa4Rfe3ntumGwVAK2mS0xs14zYxE6AGiRqR+ofP/o9mjHzVniv/0zEXvC1ddn3/PVaMeh\nUioB2zn3XedcT5TrzgAA8fjx3YO3zVsafkxXyFKjkjT2E+H7l60K34/oGBIHgJwY/8nw/ZMmDN72\nWI1lQY/WuJnHsRPh+9c2cH/rsPXIiyyJy7o2SvqJpI+Y2T4z+z/iLgMAMNibv27suKRmjF91U2PH\nNXvHr7zqiDtD59ziuPMEAGTPD7amXYN8YUgcAApkYle65c88L93ys4ybfyQs7e83aXm/cYREG2Zd\nEduv1h25Gh0C/9iHvIC/d7/0i32N5dFI3QrQhpFu/hH7kDgAoL253uCgPX9Wc/fLvuwGactzweWi\ncQRsAMiZFWukVTeGpzm2VRozx3t9cIs0oWqo/LpbpHseiV7mrOnS9nXS43cNbNu7X5p2hff6QIS1\nyb8Y84ppecOQeMLS/n6TlvfhVIk2zLqitl+U3qz1DKTbtEVavDI8fT2+93Vp8WWDy6lVHz8FaMNI\nQ+IE7ISl/f0mLe//2Uu0YdYVtf3Gj5EOPxHh+IjnsxfOlq5fKM2ZIR09If1kt3Treulne2ofGyVY\nj7s0+HKuArQh57ABoKj6jjV+7ObVXoAOMnaUNG2SdPW8yu3bX5Qu+XxjZXLtdW30sBOW9vebtLz3\nziTaMOuK3n5Rh6I7O6R3nxu8ParqcjpnSqfPNDcU/l7e+W9DetgAUHRRzx+XgnWjl3yVH3fmBenU\n89HyavV9ubOMhVMAIOcW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00GMCQeMLS/n6T\nlvfhVIk2zDrazxPUy64OrFfOkR68s/H6LF7pzThvpOwgBWhDZom3g7S/36Tl/T97iTbMOtpvwNvb\npRHDq47vkfqelMaNrtw+crb01sno9egaJb35VOW2b2yQbr5rcMBedLN034+i512ANuQcNgBgwNkf\n956rA2jHEGnqFdKr+xvP+8jxyh7zLx8Z3NOWOGfdDM5hA0DBlAdN1ys9tK25YO3n3AXeddvlPw4I\n1s1hSDxhaX+/Scv7cKpEG2Yd7Rds7EjpyNMxViZA99zmrgsvQBtGGhKnhw0ABXX0hNfrXbYqmfyX\n3tF/jryJYI0B9LATlvb3m7S8984k2jDraL/6xHFHrbiHvgvQhvSwAQD1KV2PbT0Dd/Mqt2LN4G3n\nXFZ5HJJBDzthaX+/Sct770yiDbOO9su+ArQhPWwAAPKCgA0AQAYQsAEAyIDUVzqbMWOGentjmJbY\npvJ+finv55Yk2jDraL/sy3sbRkUPGwCADEi9hw0gR3bG0BOakf8eI9AIetgAmnPwDi9QxxGspYG8\nDia0/BaQUQRsAI059aYXWPd9OZn8993k5X/qYDL5AxnDkDiA+sXVm45i9zneM0PlKDh62ADq08pg\n3Q7lAm2CgA0gml3D0g+aO006sindOgApIWADqG2nSe7dprO54fYY6rJ3cfo/HIAUcA4bQLhdw5vO\novwOTn99v/fc9G0cdw2TLvxtk5kA2UEPG0A4Vzsods+V7v2h/76g2y02fRvGGHr8QJYQsAEEqzH0\nXLr/cd8x6bN/1XwQLr+nsvVI5/1pc/UD8oSADcBfjWD47fv8tzcatP2Oe3lPhAMJ2igIAjaAwU4f\nqplk6R0tqIci/gA43Zd4PYC0EbABDPbSxNiyCppc1vSks3IvdceYGdCemCUOoNIbA9de+fVuS4HW\n9UYf/na90omT0qjZ0vFnpJEjoldn/VcGXofVRwfWSOfcGD1jIGPoYQOotP8vJAUH431lo+Wzpg/e\nH9RzLgXpoGAddNx1C73nXx3w3/9ePV9f7p8AyAkCNoC6TJk/8Hr7uspAGzbM/eGrvOdxlwanqc6r\n/P25C+qrJ5A3BGwAA5qccf16yFy1V17zno8cD04Tti8SZowjxwjYAOoyf1bwvsnzg/dFEdb7XnBJ\nc3kDWUfABuDr5A7/7Y+ubW09Sh5e47/9nWdbWw8gLQRsAJ5TlbO6zhrmnUM+a9jAtiiXYm14uLHi\nH9pWO015+SOGe++HD61KdOpwYxUA2hwBG4Bn9/t9N5/cIZ163nsd5TKu6786eNvpM5Xv+44NTnPl\nitp5l8o/tlV6e3tAot0TamcEZBABG0BNHUOaO37oxZXvu+c2l9/o9zV3PJBFBGwAdYnSy160svK9\nc+HpP/e1eMoF8oyADSB2922pL/36zcnUA8iT2AO2mU0xs6fN7Odm9rKZfSnuMgDEb/nq6Glb3dut\np7x6PgeQJUn0sE9LWuGc+58lXSzpP5nZ7yVQDoAYrY55Zc8v3BYtXdx3/Yr7cwDtIvaA7Zx7wzm3\nq//1CUk/lzQp7nIApGvBsvD933nAe962y3//5me856D7apdUzx6/9vLadQPyKNFz2Gb2QUm/L+n5\nqu1LzKzXzHoPH+aaSSALpn6g8v2jQZdVVZmzxH/7ZyL2hKuvz77H57IxoAgSC9hm9j5JD0ha5pyr\nWCHYOfdd51yPc66nu5v72AJZ8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+HyiSRAK2mXXKC9b3Ouf+\nIYkyAMRsevho1ySf9Ugeq7Es6NEaN/M4diJ8/9qN4ft9nd/XwEFA+0tilrhJWifp58455msCWdEx\nvqHDkpoxftVNDR7YOS7WegDtIoke9ixJ10i61Mxe7H80eQ8fAEXzg61p1wBoLx1xZ+ic2y6Jm9IC\nOTSxSzp4JL3yZ56XXtlA2ljpDMCAGeFriB6ocwWzch/7kDT3Iul3Jzeex3MbaiSoUX8gy2LvYQPI\nN9cbfN56/qzm7pd92Q3SlueCywWKjIANoNLkO6V94TO+jm2VxszxXh/cIk3oqtx/3S3SPY9EL3LW\ndGn7Ounxuwa27d0vTbvCex2pZz/lW9ELBDKIIXEAlSbWvjF16faWrtcL1pu2eL3u0qOeYC1JO16q\nPH7j495CLaVe9cSu8OMlSRO+WF+hQMaYq3Xfu4T19PS43t78jnV5V7nlV9r/flqhkG146rC02+fC\n6ypRL+laOFu6fqE0Z4Z09IT0k93Sreuln+2JUL8o/z2c3xd4OVch2y9n8t6GknY652r+NTEkDmCw\nzsZXINy82gvQQcaOkqZNkq6eV7l9+4vSJZ9vsFCuvUYBELAB+JvhpJ3hPZvSBLTODundqsli9Syo\n4nqlj18w0JvunCmdPhOxd83McBQEARtAsAhBWxoI1o2uelZ+3JkXpFPPR8yLYI0CYdIZgHBTay/o\nXZos5ueWJdLRp73eculxcoe33c+QiyIG66nfj5AIyA8mnSUs75Ml0v730wq0oQJ72dWB9co50oN3\nNl6XxSu9GeflAofFI/auab/sy3sbiklnAGIzw0m7RkjunUG7+p6Uxo2u3DZytvTWyejZd42S3nxK\n2nir95Ckb2yQbr7LJ/HUjVLXouiZAzlBwAYQzYX9Ebiqt90xRJp6hfTq/sazPnK8srf+y0cG97Ql\ncc4ahcY5bAD1KQuarld6aFtzwdrPuQu867YrhsMJ1ig4etgA6jfDSaeOSLvH6drLpWsvT7Cs8w81\ndV04kBf0sAE0prPLC9xT1iST/5S1Xv4Ea0ASPWwAzZqwzHtIka7Zromhb8AXPWwA8ZnhBh7Tjw7a\nvcKvM37+G5XHAfBFDxtAMjrGDArAq/4upboAOUAPGwCADCBgAwCQAQRsAAAygIANAEAGpH7zDzPL\n9bTQtL/fpBVgUX7aMONov+wrQBty8w8AAAKdOSq92FWxacUaadWNVenO3y91vr919QpADzthaX+/\nSePXffblvQ1pv+yLtQ3bcHGfqD1szmEDAPLt4B1eoI4jWEsDeR1cFU9+EdHDTlja32/S+HWffXlv\nQ9ov+xpuw1NvSrvHx1sZP+cfkDonNnw457ABAMUVV286it3neM8JL63LkDgAIF9aGaxbWC4BGwCQ\nD7uGpResS3aadGRTIlkTsAEA2bfTJPdu09nccHsMddm7OJEfDkw6S1ja32/SmPCSfXlvQ9ov+2q2\n4a7hkvttU2WYz5Qv19tUlpINlS6sXS8u6wIAFEOEYN09V7r3h/77/IJ12PbIYujxl6OHnbC0v9+k\n8es++/LehrRf9oW2YY2h5yg957DAXCvtR6dJP70/tAo1Z4/TwwYA5FuNYP3t+/y3N9pz9jvu5T0R\nDozpfDYBGwCQPacP1Uyy9I4W1EMRfwCc7mu6HAI2ACB7Xmp8ZbFqQZPLmp50Vu6l7qazYKUzAEC2\nvDFw7VXYOWrXG3342/VKJ05Ko2ZLx5+RRo6IXp31Xxl4HXrO/MAa6ZzqW4FFRw8bAJAt+/9CUnAw\n3lc2Wj5r+uD9QT3nUpAOCtZBx1230Hv+1QH//e/V8/Xl/gkiImADAHJlyvyB19vXVQbasGHuD1/l\nPY+7NDhNdV7l789dUF8960XABgBkR5Mzrl8Pmav2ymve85HjwWnC9kXSRP0J2ACAXJk/K3jf5PnB\n+6II630vuKS5vGshYAMAMunkDv/tj65tbT1KHl7jv/2dZ+PJn4ANAMiGU5Wzus4a5p1DPmvYwLYo\nl2JteLix4h/aVjtNefkjhnvvhw+tSnTqcEPlszRpwtL+fpNW+GURcyDvbUj7Zd97bRhy/vf0Galz\nZn96n6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdWtWK6UpUkBAIXRMaS544deXPm+e25z+YUG\n6wYRsAEAuRJlsZRFKyvf1xqI+dzX4im3GbEHbDMbbmYvmNlLZvaymX017jIAAGjGfVvqS79+czL1\nqEcSPezfSrrUOTdd0gWSPm1mF9c4BgCAUMtXR0+bdG+3mfLq+RzlYg/YzvNW/9vO/ke+Z30AABK3\nurmVPQf5wm3R0sV9169GP0ci57DNbIiZvSjpkKQfOeeer9q/xMx6zSzOe6EAAPCeBcvC93/nAe95\n2y7//Zuf8Z6D7qtdcuWKyvfXXl67bo1I9LIuMxsj6UFJX3TO/TQgTa5731xSkn20YbbRftkX5bIu\nSZp2hbR3f9Wx/d3CoCHrWnf0CtsflHek23K222VdzrljkrZK+nSS5QAA8OO7B2+btzT8mK6QpUYl\naewnwvcvWxW+P05JzBLv7u9Zy8zOkjRX0r/GXQ4AoGCmh68QNmnC4G2P1VgW9GiNm3kcOxG+f+3G\n8P2+zu9r4CCpo6Gjwr1f0j1mNkTeD4L7nXOPJFAOAKBIOsY3dFhSM8avuqnBAzvHNXRY7AHbObdb\n0u/HnS8AAO3kB1tbWx4rnQEAcmNiV7rlzzwvuby5+UfC0v5+k1aoGao5lfc2pP2yb1Ab1pgt3ugQ\n+Mc+5AX8vfulX+xrLI+aM8RnDP73GHWWeBLnsAEASE3YpVjzZzV3v+zLbpC2PBdcbpII2ACAbJl8\np7QvfMbXsa3SmDne64NbpAlVQ+XX3SLdU8d06FnTpe3rpMfvGti2d7937bckHYiyNvmUb0Uv0AdD\n4glL+/tNWiGH43Im721I+2WfbxvWGBaXvF52qde7aYu0eGV4+np87+vS4ssGlxPKZzhcij4kTsBO\nWNrfb9IK+59FjuS9DWm/7PNtw1OHpd0+F15XiXo+e+Fs6fqF0pwZ0tET0k92S7eul362J0L9ogTr\n8/sCL+fiHDYAIL86uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQhu89rocPeyEpf39Jq2wv+5z\nJO9tSPtlX2gbRhwa7+yQ3n1u8PbIdajqRXfOlE6faW4o/L160MMGAOTeDBcpaJeCdaOXfJUfd+YF\n6dTzEfOqEazrwcIpAIBsm1pjBNUvAAAgAElEQVR7QW/rCQ6wtyyRjj7t9ZZLj5M7vO1+hlwUMVhP\n/X6ERNExJJ6wtL/fpBV+OC4H8t6GtF/2RWrDgF52dWC9co704J2N12XxSm/GebnAYfGIvWtmibeJ\ntL/fpPGfRfblvQ1pv+yL3Ia7RkjunYpN1iP1PSmNG12ZdORs6a2T0evQNUp686nKbd/YIN18l0/A\nnrpR6loUOW/OYQMAiuXC/ghc1dvuGCJNvUJ6dX/jWR85Xtlb/+Ujg3vakmI9Z12Nc9gAgHwpC5qu\nV3poW3PB2s+5C7zrtit61wkGa4kh8cSl/f0mjeG47Mt7G9J+2ddwG546Iu1u/vrnms4/1NR14VGH\nxOlhAwDyqbPL6/VOWZNM/lPWevk3EazrQQ87YWl/v0nj13325b0Nab/si7UNI1yzXVPMQ9/0sAEA\nqDbDDTymHx20e4VfZ/z8NyqPSwk97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAy\nIPWVzmbMmKHe3ij3J8umvJ9fyvu5JYk2zDraL/vy3oZR0cMGACADUu9hI7pIN0qvodF7wQIA0kUP\nu83ddM3A/VnjUMpr+dXx5AcAaA0CdpvqGuUF1ju+lEz+q2708p/QlUz+AIB4MSTehuLqTUdxsP/2\ncAyVA0B7o4fdZloZrNuhXABANATsNvGbZ9MPmq5X+rNPpVsHAIA/AnYbcL3SsKHN53PD7c3nsem2\n9H84AAAG4xx2yt7Z0Xwe5eef//p+77nZoPubZ6Xhf9RcHgCA+NDDTtnwYbXTdM+V7v2h/76gyWLN\nTiKLo8cPAIgPATtFtXrB1uM9+o5Jn/2r5oNwKb/S47w/ba5+AIDWIWCnpFYw/PZ9/tsbDdp+x728\np/ZxBG0AaA8E7BR0R1isZOkdyddDivYDYNzo5OsBAAhHwE7BoS3x5RXUA46zZ9z3ZHx5AQAawyzx\nFvvzawZe+/VuS4HW9UYf/na90omT0qjZ0vFnpJEjotdn/Vei1WfZYumbG6PnCwCIFz3sFru9f23w\noGC879DA61nTB+8P6jmXgnRQsA467rqF3vOvDvjvL9VzzQr//QCA1iBgt5kp8wdeb19XGWjDhrk/\nfJX3PO7S4DTVeZW/P3dBffUEALQWAbuFmj2v/Pqh4H2vvOY9HzkenCZsXxTMGAeA9BCw28z8WcH7\nJs8P3hdFWO97wSXN5Q0ASBYBOyUnA5YkfXRta+tR8vAa/+3vPNvaegAA/BGwW2TiuMr3Zw3zhpjP\nKluaNMqQ84aHGyv/oW2105SXP2K493541RKl48c0Vj4AoDkE7BY58Lj/9pM7pFPPe6+jXMZ1/VcH\nbzt9pvJ937HBaa6MMMu7VP6xrdLb2/3THH6idj4AgPgRsNtAx5Dmjh96ceX77rnN5Tf6fc0dDwCI\nHwG7zUTpZS9aWfneufD0n/taPOUCANKTSMA2syFm9s9m9kgS+RfdfXUubbp+czL1AAC0TlI97C9J\n+nlCeWfS8tXR07a6t1tPefV8DgBAfGIP2GY2WdLlku6OO+8sW7083vy+cFu0dHHf9SvuzwEAiCaJ\nHvY3JX1Z0v8ISmBmS8ys18x6Dx8+nEAVsm/BsvD933nAe962y3//5me856D7apdUzx6/9vLadQMA\ntF6sAdvMFkg65JzbGZbOOfdd51yPc66nu7s7zipk1tQPVL5/NOCyqmpzlvhv/0zEnnD19dn3+Fw2\nBgBIX9w97FmSrjCzVyVtknSpmf1dzGXk0o99TiDMWxp+TFfIUqOSNPYT4fuXrQrfDwBoH7EGbOfc\nzc65yc65D0paJOkp59xn4ywjq8Z/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXIAQHK4DrtF\n3vx1Y8clNWP8qpsaO67ZO34BABrTkVTGzrmtkrYmlT+a84OtadcAAFAPethtZGJXuuXPPC/d8gEA\nwQjYLVRrePtAnSuYlfvYh6S5F0m/O7nxPJ7bEL6f5UsBID2JDYmjMa43ODDOn9Xc/bIvu0Ha8lxw\nuQCA9kXAbrEVa6RVN4anObZVGjPHe31wizShaqj8uluke+pYpX3WdGn7Ounxuwa27d0vTbvCex2l\nZ//FmFdMAwDUx1ytWz0lrKenx/X25rd7Z2aDtkXpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP2\nv59W8GvDPMl7G9J+2Zf3NpS00zlX86QjATthfv/Qxo+RDj8R4diI54wXzpauXyjNmSEdPSH9ZLd0\n63rpZ3tqHxslWI+7NPhyrrT//bRC3v+zyHsb0n7Zl/c2VMSAzZB4CvqONX7s5tVegA4ydpQ0bZJ0\n9bzK7dtflC75fGNlcu01AKSPgJ2SKEPRpQlonR3Su1WTxeqZse16pY9fMFBe50zp9JnmhsIBAK1F\nwE5R1PPHpWDdaPAsP+7MC9Kp56PlRbAGgPbBddgpW3Rz7TTWExw8b1kiHX3aC/ylx8kd3nY/Qy6K\nFoj/5Mu10wAAWodJZwmLMlkiqJddHVivnCM9eGfjdVm80ptx3kjZQdL+99MKeZ/wkvc2pP2yL+9t\nKCadZYf1SG9vl0YMH7yv70lp3OjKbSNnS2+djJ5/1yjpzaekjbd6D0n6xgbp5rsGp110s3Tfj6Ln\nDQBoDQJ2mzj7495zdY+3Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBoB2xjnsNlMeNF2v9NC25oK1\nn3MXeNdtl/84IFgDQHujh92GrEcaO1I68rR07eXeIyndc5u7LhwA0Br0sNvU0RNe4F62Kpn8l97h\n5U+wBoBsoIfd5tZu9B5SPHfUYugbALKJHnaGlK7Htp6Bu3mVW7Fm8LZzLqs8DgCQTfSwM+rXb/kH\n4NX3tr4uAIDk0cMGACADCNgAAGQAARsAgAxIfS1xM8v1Qrhpf79JK8Aav7RhxtF+2VeANoy0ljg9\nbAAAMoBZ4kCr7IyhJzQj3z0NAMHoYQNJOniHF6jjCNbSQF4HE1oCD0Db4hx2wtL+fpPG+bMAp96U\ndo+PvzLVzj8gdU5sKou8tyF/g9lXgDbkfthAKuLqTUex+xzvmaFyIPcYEgfi1Mpg3Q7lAmgZAjYQ\nh13D0g+aO006sindOgBIDAEbaNZOk9y7TWdzw+0x1GXv4vR/OABIBJPOEpb295u0wk942TVccr9t\nKn+/m7g0fStVGypdGK1eeW9D/gazrwBtyMIpQOIiBOvuudK9P/TfF3TL06ZvhRpDjx9Ae6GHnbC0\nv9+kFfrXfY2h5yg957DAXCvtR6dJP70/tAqRZo/nvQ35G8y+ArQhPWwgMTWC9bfv89/eaM/Z77iX\n90Q4kPPZQG4QsIF6nT5UM8nSO1pQD0X8AXC6L/F6AEgeARuo10vNrSxWLmhyWdOTzsq91B1jZgDS\nwkpnQD3eGLj2KuwcteuNPvzteqUTJ6VRs6Xjz0gjR0SvzvqvDLwOPWd+YI10zo3RMwbQduhhA/XY\n/xeSgoPxvrLR8lnTB+8P6jmXgnRQsA467rqF3vOvDvjvf6+ery/3TwAgMwjYQIymzB94vX1dZaAN\nG+b+8FXe87hLg9NU51X+/twF9dUTQPYQsIGompxx/XrIXLVXXvOejxwPThO2LxJmjAOZRsAGYjR/\nVvC+yfOD90UR1vtecElzeQNofwRsoAEnd/hvf3Rta+tR8vAa/+3vPNvaegBIDgEbiOJU5ayus4Z5\n55DPGjawLcqlWBsebqz4h7bVTlNe/ojh3vvhQ6sSnTrcWAUApI6lSROW9vebtMIsixhy/vf0Galz\nZn9an6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdUdtFxp3tuQv8HsK0AbsjQp0AodQ5o7fujF\nle+75zaXX2iwBpBZBGwgRlEWS1m0svJ9rc7D574WT7kAsi2RgG1mr5rZv5jZi2YW5yKLQObdt6W+\n9Os3J1MPANmSZA/7E865C6KMywPtbvnq6Glb3dutp7x6PgeA9sKQOBDB6phX9vzCbdHSxX3Xr7g/\nB4DWSSpgO0lbzGynmS2p3mlmS8ysl+Fy5NWCZeH7v/OA97xtl//+zc94z0H31S65ckXl+2svr103\nANmUyGVdZvYB59x+M5sg6UeSvuiceyYgba7n6xfgcoS0q5C4Wpd1SdK0K6S9+6uO6/85GjRkXeuO\nXmH7g/KOdFtOLuvKlby3n1SINkzvsi7n3P7+50OSHpR0URLlAO3ix3cP3jZvafgxXSFLjUrS2E+E\n71+2Knw/gHyJPWCb2dlmNrL0WtIfS/pp3OUALTU9fIWwSRMGb3usxrKgR2vczOPYifD9azeG7/d1\nfl8DBwFoBx0J5DlR0oP9wzQdkr7nnHssgXKA1ukY39BhSc0Yv+qmBg/sHBdrPQC0TuwB2zm3R9L0\nuPMFMOAHW9OuAYBW47IuICYTu9Itf+Z56ZYPIFnc/CNhaX+/SSvcDNUas8UbHQL/2Ie8gL93v/SL\nfY3lUXOG+Az/f4t5b0P+BrOvAG0YaZZ4EuewgcIKuxRr/qzm7pd92Q3SlueCywWQbwRsoB6T75T2\nhc/4OrZVGjPHe31wizShaqj8ulukex6JXuSs6dL2ddLjdw1s27vfu/Zbkg5EWZt8yreiFwigLTEk\nnrC0v9+kFXI4rsawuOT1sku93k1bpMUrw9PX43tflxZfNricUAHD4VL+25C/wewrQBtGGhInYCcs\n7e83aYX8z+LUYWm3z4XXVaKez144W7p+oTRnhnT0hPST3dKt66Wf7YlQtyjB+vy+0Mu58t6G/A1m\nXwHakHPYQCI6uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQrn2GsgFetgJS/v7TVqhf91HHBrv\n7JDefW7w9sjlV/WiO2dKp880PxT+Xl1y3ob8DWZfAdqQHjaQqBm1bwoiDQTrRi/5Kj/uzAvSqecj\n5hUhWAPIDhZOAZoxtfaC3tYTHGBvWSIdfdrrLZceJ3d42/0MuShisJ76/QiJAGQJQ+IJS/v7TRrD\ncQrsZVcH1ivnSA/e2Xg9Fq/0ZpxX1C1oWLyO3nXe25C/wewrQBsyS7wdpP39Jo3/LPrtGiG5dyo2\nWY/U96Q0bnRl0pGzpbdORi+/a5T05lOV276xQbr5Lp+APXWj1LUoeubKfxvyN5h9BWhDzmEDLXNh\nfwSu6m13DJGmXiG9ur/xrI8cr+yt//KRwT1tSZyzBnKOc9hAnMqCpuuVHtrWXLD2c+4C77rtit41\nwRrIPYbEE5b295s0huMCnDoi7W7B9c/nH2rqunAp/23I32D2FaANIw2J08MGktDZ5fV6p6xJJv8p\na738mwzWALKDHnbC0v5+k8av+zpEuGa7pgSGvvPehvwNZl8B2pAeNtBWZriBx/Sjg3av8OuMn/9G\n5XEACosedsLS/n6Txq/77Mt7G9J+2VeANqSHDQBAXhCwAQDIAAI2AAAZkPpKZzNmzFBvb5T7BGZT\n3s8v5f3ckkQbZh3tl315b8Oo6GEDAJABBGwAADIg9SFxAMiKwNuZ1iHS/cwBH/SwASDETdd4gTqO\nYC0N5LX86njyQ3EQsAHAR9coL7De8aVk8l91o5f/hK5k8kf+MCQOAFXi6k1HcbD/3uYMlaMWetgA\nUKaVwbodykV2ELABQNJvnk0/aLpe6c8+lW4d0L4I2AAKz/VKw4Y2n88Ntzefx6bb0v/hgPbEOWwA\nhfbOjubzKD///Nf3e8/NBt3fPCsN/6Pm8kC+0MMGUGjDh9VO0z1XuveH/vuCJos1O4ksjh4/8oWA\nDaCwavWCrcd79B2TPvtXzQfhUn6lx3l/2lz9UCwEbACFVCsYfvs+/+2NBm2/417eU/s4gjZKCNgA\nCqc7wmIlS+9Ivh5StB8A40YnXw+0PwI2gMI5tCW+vIJ6wHH2jPuejC8vZBezxAEUyp9fM/Dar3db\nCrSuN/rwt+uVTpyURs2Wjj8jjRwRvT7rvxKtPssWS9/cGD1f5A89bACFcnv/2uBBwXjfoYHXs6YP\n3h/Ucy4F6aBgHXTcdQu9518d8N9fqueaFf77URwEbAAoM2X+wOvt6yoDbdgw94ev8p7HXRqcpjqv\n8vfnLqivnigeAjaAwmj2vPLrh4L3vfKa93zkeHCasH1RMGO82AjYAFBm/qzgfZPnB++LIqz3veCS\n5vJG/hGwARTSyYAlSR9d29p6lDy8xn/7O8+2th5oXwRsAIUwcVzl+7OGeUPMZ5UtTRplyHnDw42V\n/9C22mnKyx8x3Hs/vGqJ0vFjGisf2UfABlAIBx73335yh3Tqee91lMu4rv/q4G2nz1S+7zs2OM2V\nEWZ5l8o/tlV6e7t/msNP1M4H+UTABlB4HUOaO37oxZXvu+c2l9/o9zV3PPIpkYBtZmPM7O/N7F/N\n7Odm9odJlAMAcYvSy160svK9c+HpP/e1eMpFsSXVw14r6THn3L+TNF3SzxMqBwBa7r46lzZdvzmZ\neqBYYg/YZjZK0mxJ6yTJOfeuc87njA4AtM7y1dHTtrq3W0959XwO5EsSPexpkg5LWm9m/2xmd5vZ\n2QmUAwCRrV4eb35fuC1aurjv+hX350B2JBGwOyRdKOlvnHO/L+ltSX9ZnsDMlphZr5n1Hj58OIEq\nAEBzFiwL3/+dB7znbbv8929+xnsOuq92SfXs8Wsvr103FFMSAXufpH3Ouf4LJfT38gL4e5xz33XO\n9Tjnerq7uxOoAgDUZ+oHKt8/GnBZVbU5S/y3fyZiT7j6+ux7fC4bA6QEArZz7oCk18zsI/2bPinp\nZ3GXAwBx+vHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfD5RLapb4FyXda2a7JV0g6daEygGASMZ/Mnz/\npAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXLkW0cSmTrnXpTEVYUA2sabv27suKRmjF91U2PHNXvH\nL2QXK50BQAp+sDXtGiBrCNgA0G9iV7rlzzwv3fLR3gjYAAqj1vD2gTpXMCv3sQ9Jcy+Sfndy43k8\ntyF8P8uXFlsi57ABIKtcb3BgnD+ruftlX3aDtOW54HKBMARsAIWyYo206sbwNMe2SmPmeK8PbpEm\nVA2VX3eLdM8j0cucNV3avk56/K6BbXv3S9Ou8F5H6dl/MeYV05A95mrdZiZhPT09rrc3vz8tzSzt\nKiQq7X8/rUAbZptf+0XpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP39pPy/zcoaadzruYJDwJ2\nwvL+Dy3tfz+tQBtmm1/7jR8jHX4iwrERzxkvnC1dv1CaM0M6ekL6yW7p1vXSz/bUPjZKsB53afDl\nXHlvPyn/f4OKGLAZEgdQOH1N3D9w82ovQAcZO0qaNkm6el7l9u0vSpd8vrEyufYaEgEbQEFFGYou\nTUDr7JDerZosVs+MbdcrffyCgfI6Z0qnzzQ3FI7iIWADKKyo549LwbrR4Fl+3JkXpFPPR8uLYI1y\nXIcNoNAW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00KBYmnSUs75Ml0v730wq0YbZF\nab+gXnZ1YL1yjvTgnY3XZfFKb8Z5I2UHyXv7Sfn/GxSTzgAgGuuR3t4ujRg+eF/fk9K40ZXbRs6W\n3joZPf+uUdKbT0kbb/UekvSNDdLNdw1Ou+hm6b4fRc8bxUHABgBJZ3/ce67u8XYMkaZeIb26v/G8\njxyv7DH/8pHBPW2Jc9YIxzlsAChTHjRdr/TQtuaCtZ9zF3jXbZf/OCBYoxZ62ABQxXqksSOlI09L\n117uPZLSPbe568JRHPSwAcDH0RNe4F62Kpn8l97h5U+wRlT0sAEgxNqN3kOK545aDH2jUfSwASCi\n0vXY1jNwN69yK9YM3nbOZZXHAY2ihw0ADfj1W/4BePW9ra8LioEeNgAAGUDABgAgAwjYAABkQOpr\niZtZrhfCTfv7TVoB1vilDTOO9su+ArRhpLXE6WEDAJABzBJH2+AaVwAIRg8bqbrpmoF7CMehlNfy\nq+PJDwDaBeewE5b295u0Rs+flW43mLSJfywdOtJcHrRhttF+2VeANuR+2GhPcfWmozjYfwtDhsoB\nZB1D4mipVgbrdigXAOJCwEZL/ObZ9IOm65X+7FPp1gEAGkXARuJcrzRsaPP53HB783lsui39Hw4A\n0AgmnSUs7e83abUmvLyzQxo+rMkyfM4/Nxt0f/uuNPyPoqUtehtmHe2XfQVoQxZOQfqiBOvuudK9\nP/TfFzRZrNlJZHH0+AGglehhJyzt7zdpYb/ua/WCo/ScwwJzrbQfnSb99P766zConAK3YR7QftlX\ngDakh4301ArW377Pf3ujPWe/417eU/s4zmcDyAoCNmLX3VU7zdI7kq+HFO0HwLjRydcDAJpFwEbs\nDm2JL6+gHnCcPeO+J+PLCwCSwkpniNWfXzPwOuwcteuNPvzteqUTJ6VRs6Xjz0gjR0Svz/qvRKvP\nssXSNzdGzxcAWo0eNmJ1+5e856BgvO/QwOtZ0wfvD+o5l4J0ULAOOu66hd7zrw747y/Vc80K//0A\n0C4I2GipKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BIB2Q8BGbJo9r/z6oeB9r7zmPR85\nHpwmbF8UzBgH0M4I2Gip+bOC902eH7wvirDe94JLmssbANJGwEYiTu7w3/7o2tbWo+ThNf7b33m2\ntfUAgEYRsBGLieMq3581zBtiPqtsadIoQ84bHm6s/Ie21U5TXv6I4d774VVLlI4f01j5AJA0liZN\nWNrfb9JKyyKGBePTZ6TOmQpMVz2jvDpN+fGSdPiJwYG1Vh7laY5tlUa/L7i+g/IqSBvmFe2XfQVo\nQ5YmRXvoGNLc8UMvrnzfPbe5/MKCNQC0KwI2WirKYimLVla+r/Xj+nNfi6dcAGhnsQdsM/uImb1Y\n9jhuZsviLgf5dV+dS5uu35xMPQCgncQesJ1z/+acu8A5d4GkGZJOSnow7nLQXpavjp621b3desqr\n53MAQCslPST+SUm/cM79MuFykLLVy+PN7wu3RUsX912/4v4cABCXpAP2IkmDbqlgZkvMrNfMWFuq\noBbUOEnynQe85227/PdvfsZ7DrqvdsmVVWuEX3t57boBQDtK7LIuMxsqab+kjzrnDoaky/V8/QJc\njiCp9jXW066Q9u6v3FY6JmjIutYdvcL2B+Ud5VpwLuvKF9ov+wrQhqlf1jVP0q6wYI3i+PHdg7fN\nWxp+TFfIUqOSNPYT4fuXrQrfDwBZkmTAXiyf4XDk0/hPhu+fNGHwtsdqLAt6tMbNPI6dCN+/toF/\nfWHrkQNAmhIJ2GY2QtKnJP1DEvmj/bz568aOS2rG+FU3NXZcs3f8AoCkdCSRqXPupKRxNRMCCfnB\n1rRrAADxYqUztMzErnTLn3leuuUDQDO4+UfC0v5+k1Y9Q7XWLOxGh8A/9iEv4O/dL/1iX2N5NFq3\norVh3tB+2VeANow0SzyRIXEgSNilWPNnNXe/7MtukLY8F1wuAGQZARuxWrFGWnVjeJpjW6Uxc7zX\nB7dIE6qGyq+7RbrnkehlzpoubV8nPX7XwLa9+71rvyXpQIS1yb8Y84ppABA3hsQTlvb3mzS/4bio\ni5OU0m3aIi1eGZ6+Ht/7urT4ssHl1KpPkCK2YZ7QftlXgDaMNCROwE5Y2t9v0vz+sxg/Rjr8RIRj\nI57PXjhbun6hNGeGdPSE9JPd0q3rpZ/tqX1slGA97tLwy7mK2IZ5QvtlXwHakHPYSEffscaP3bza\nC9BBxo6Spk2Srp5XuX37i9Iln2+sTK69BpAF9LATlvb3m7SwX/dRh6I7O6R3nxu8ParqcjpnSqfP\nND8U/l7+BW7DPKD9sq8AbUgPG+mKev64FKwbveSr/LgzL0inno+WV6vvyw0AzWDhFCRq0c2101hP\ncPC8ZYl09Gkv8JceJ3d42/0MuShaIP6TL9dOAwDthCHxhKX9/SYtynBcUC+7OrBeOUd68M7G67J4\npTfjvJGyw9CG2Ub7ZV8B2pBZ4u0g7e83aVH/s3h7uzRieNWxPVLfk9K40ZXbR86W3joZvQ5do6Q3\nn6rc9o0N0s13DQ7Yi26W7vtR9Lwl2jDraL/sK0Abcg4b7ePsj3vP1QG0Y4g09Qrp1f2N533keGWP\n+ZePDO5pS5yzBpBtnMNGS5UHTdcrPbStuWDt59wF3nXb5T8OCNYAso4h8YSl/f0mrdHhuLEjpSNP\nx1wZH91zm7suXKINs472y74CtGGkIXF62EjF0RNer3fZqmTyX3pH/znyJoM1ALQLetgJS/v7TVqc\nv+7juKNWEkPftGG20X7ZV4A2pIeNbCldj209A3fzKrdizeBt51xWeRwA5BU97ISl/f0mjV/32Zf3\nNqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyoB1WOuuT9MsWlje+v8yWSOn8Uks/Ywry3oa0X4xov9i1\n/PMVoA3PjZIo9UlnrWZmvVFO7mdZ3j8jny/b+HzZlvfPJ7XvZ2RIHACADCBgAwCQAUUM2N9NuwIt\nkPfPyOfLNj5ftuX980lt+hkLdw4bAIAsKmIPGwCAzCFgAwCQAYUK2Gb2aTP7NzN7xcz+Mu36xMnM\n/tbMDpnZT9OuSxLMbIqZPW1mPzezl83sS2nXKW5mNtzMXjCzl/o/41fTrlPczGyImf2zmT2Sdl2S\nYGavmtm/mNmLZhbD/efai5mNMbO/N7N/7f9b/MO06xQXM/tIf7uVHsfNbFna9SpXmHPYZjZE0v8n\n6VOS9kn6J0mLnXM/S7ViMTGz2ZLekvTfnHPnpV2fuJnZ+yW93zm3y8xGStop6cq8tJ8kmbc6xNnO\nubfMrFPSdklfcs49l3LVYmNmyyX1SBrlnFuQdn3iZmavSupxzuVy4RQzu0fSj51zd5vZUEkjnHO5\nu+t8f7x4XdJM51wrF/YKVaQe9kWSXnHO7XHOvStpk6TPpFyn2DjnnpF0JO16JMU594Zzblf/6xOS\nfi5pUrq1ipfzvNX/tl2ok/MAAAJgSURBVLP/kZtf1GY2WdLlku5Ouy6on5mNkjRb0jpJcs69m8dg\n3e+Tkn7RTsFaKlbAniTptbL3+5Sz//CLwsw+KOn3JT2fbk3i1z9k/KKkQ5J+5JzL02f8pqQvS/of\naVckQU7SFjPbaWZL0q5MzKZJOixpff9pjbvN7Oy0K5WQRZI2pl2JakUK2H6L0eam91IUZvY+SQ9I\nWuacO552feLmnDvjnLtA0mRJF5lZLk5vmNkCSYecczvTrkvCZjnnLpQ0T9J/6j9VlRcdki6U9DfO\nud+X9LakXM0FkqT+of4rJH0/7bpUK1LA3idpStn7yZL2p1QXNKD/vO4Dku51zv1D2vVJUv9Q41ZJ\nn065KnGZJemK/nO8myRdamZ/l26V4uec29//fEjSg/JOxeXFPkn7ykZ9/l5eAM+beZJ2OecOpl2R\nakUK2P8k6cNmNrX/F9QiSZtTrhMi6p+QtU7Sz51zq9OuTxLMrNvMxvS/PkvSXEn/mm6t4uGcu9k5\nN9k590F5f3tPOec+m3K1YmVmZ/dPiFT/UPEfS8rNVRvOuQOSXjOzj/Rv+qSk3Ez6LLNYbTgcLrXH\n7TVbwjl32sxukPS4pCGS/tY593LK1YqNmW2UNEfSeDPbJ+krzrl16dYqVrMkXSPpX/rP8UrSSufc\nP6ZYp7i9X9I9/TNUf0fS/c65XF7+lFMTJT3YfyvIDknfc849lm6VYvdFSff2d3r2SLo+5frEysxG\nyLuS6D+mXRc/hbmsCwCALCvSkDgAAJlFwAYAIAMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAG\n/P+uMuaa/akHvAAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FNWd7/vPd9gbEMOvDRtMgGtgkifnTow4skecIXKJIWNAMHru3Bm4Ro/m5nJu7jEEwcmMPM88MXlONFcFQuLcycmRAc8ZA5pxjKgTJf4AA0adDaNMTGbuY8BERH5sgYBiInDW/aN2u7t7V1VXd1d1dVW9X8/TT3dXrVprdS82316rVq0y55wAAEB7+520KwAAAGojYAMAkAEEbAAAMoCADQBABhCwAQDIAAI2AAAZQMAGACADCNgAAGQAARtoM2b2QTP7RzM7amYHzOwuM+sIST/GzP6mP+1JM/sXM/sPrawzgOQRsIH28/9KOiTp/ZIukPS/SPq//RKa2VBJT0g6V9IfShot6c8l3W5mS1tSWwAtQcAG2s9USfc7537jnDsg6TFJHw1Ie42k/0nS/+ac2+ucO+Wce0zSUkn/2cxGSpKZOTP7UOkgM9tgZv+57P0CM3vRzI6Z2bNmdn7Zvg+Y2QNmdtjM9pb/EDCzW8zsfjP7b2Z2wsxeNrOesv1/YWav9+/7NzP7ZDxfEVA8BGyg/ayVtMjMRpjZJEnz5AVtP5+S9EPn3NtV2x+QNELSxbUKM7MLJf2tpP8oaZyk/yJps5kNM7PfkfSwpJckTZL0SUnLzOyysiyukLRJ0hhJmyXd1Z/vRyTdIOkPnHMjJV0m6dVa9QHgj4ANtJ9t8nrUxyXtk9Qr6QcBacdLeqN6o3PutKQ+Sd0Ryvs/Jf0X59zzzrkzzrl7JP1WXrD/A0ndzrmvOefedc7tkfRfJS0qO367c+4fnXNnJP13SdP7t5+RNEzS75lZp3PuVefcLyLUB4APAjbQRvp7tI9L+gdJZ8sLyGMl/T8Bh/TJO9ddnU9H/7GHIxR7rqQV/cPhx8zsmKQpkj7Qv+8DVftWSppYdvyBstcnJQ03sw7n3CuSlkm6RdIhM9tkZh+IUB8APgjYQHvpkhcs73LO/dY596ak9ZLmB6R/QtI8Mzu7avv/KumUpBf635+UN0Reck7Z69ckfd05N6bsMcI5t7F/396qfSOdc0H1qeCc+55z7uPyAr9T8A8PADUQsIE24pzrk7RX0hfMrMPMxkj6D/LOIfv57/KGzb/ffzlYZ//55W9Jut059+v+dC9K+t/NbIiZfVrezPOS/yrp/zKzmeY528wu75+w9oKk4/2Tx87qP/48M/uDWp/FzD5iZpea2TBJv5H0jrxhcgANIGAD7effS/q0vOHsVySdlnSjX0Ln3G8lzZXXE35eXlB8TNI3JX21LOmXJC2UdEzS1So7J+6c65V3HvsuSUf7y7yuf9+Z/uMukPdDok/S3fIuH6tlmKRv9B9zQNIEecPpABpgzrm06wAgJmbWKemHkl6XdJ3jDxzIDXrYQI44507JO3/9C0kfSbk6AGJEDxsAgAyghw0AQAYE3lCgVcaPH+8++MEPpl2NxOzcuTPtKiRqxowZaVchcbRhttF+2Zf3NpTU55yruchR6kPiPT09rre3t/mMdlrzecyI/7swi6FebSztfz+tQBtmG+2XfXlvQ0k7nXM9tRJle0j84B1eoI4jWEsDeR1cFU9+AADEJJsB+9SbXmDd9+Vk8t93k5f/qYPJ5A8AQJ1SP4ddt7h601Hs7l+9MYGhcgAA6pGtHnYrg3U7lAsAQL9sBOxdw9IPmjtNOrIp3ToAAAqr/QP2TpPcu01nc8PtMdRl7+L0fzgAAAqpvc9h7xredBZWNlH+r+/3nl2zV5HtGiZd+NsmMwEAILr27mG72kGxe6507w/991nAVW1B2yOLoccPAEA92jdg1xh6th7v0XdM+uxfNR+ES/mVHuf9aXP1AwAgTu0ZsGsEw2/f57+90aDtd9zLeyIcSNAGALRI+wXs04dqJll6RwvqoYg/AE73JV4PAADaL2C/NDG2rIImlzU96azcSzXXawcAoGntNUv8jYFrr/x6t6VA63qjD3+7XunESWnUbOn4M9LIEdGrs/4rA6/D6qMDa6RzboyeMQAAdWqvHvb+v5AUHIz3lY2Wz5o+eH9Qz7kUpIOCddBx1y30nn91wH//e/V8fbl/AgAAYtJeAbuGKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BAAgbu0TsJuccf16yFy1V17zno8cD04Tti8SZowDABLUPgE7gvmzgvdNnh+8L4qw3veCS5rLGwCAZrVlwD65w3/7o2tbW4+Sh9f4b3/n2dbWAwBQXO0RsE9Vzuo6a5h3DvmsYQPbolyKteHhxop/aFvtNOXljxjuvR8+tCrRqcONVQAAgBraI2Dvfr/v5pM7pFPPe6+jXMZ1/VcHbzt9pvJ937HBaa5cUTvvUvnHtkpvbw9ItHtC7YwAAGhAewTsEB1Dmjt+6MWV77vnNpff6Pc1dzwAAI1o+4BdLkove9HKyvfOhaf/3NfiKRcAgCRlKmBHcd+W+tKv35xMPQAAiFMiAdvMPm1m/2Zmr5jZX9ZKv3x1HXm3uLdbT3n1fA4AAOoRe8A2syGS/lrSPEm/J2mxmf1e2DGrY17Z8wu3RUsX912/4v4cAACUJNHDvkjSK865Pc65dyVtkvSZOAtYsCx8/3ce8J637fLfv/kZ7znovtol1bPHr728dt0AAEhCEgF7kqTXyt7v69/2HjNbYma9ZtZ7+HDta5enfqDy/aNBl1VVmbPEf/tnIvaEq6/PvsfnsjEAAFohiYDtt6h2xVxt59x3nXM9zrme7u7a95P+8d2Dt81bGn5MV8hSo5I09hPh+5etCt8PAEArJRGw90maUvZ+sqT9oUdMD+9lT/JZj+SxGsuCHq1xM49jJ8L3r90Yvt/X+X0NHAQAQG1JBOx/kvRhM5tqZkMlLZIUfvFUx/iGCkpqxvhVNzV4YOe4WOsBAEBJR9wZOudOm9kNkh6XNETS3zrnXo67nCT9YGvaNQAAoFLsAVuSnHP/KOkf48xzYpd08EicOdZn5nnplQ0AQPusdDYjfA3RA3WuYFbuYx+S5l4k/e7kxvN4bkONBDXqDwBAMxLpYSfF9Qaft54/q7n7ZV92g7TlueByAQBIU3sF7Ml3SvvCZ3wd2yqNmeO9PrhFmtBVuf+6W6R7Hole5Kzp0vZ10uN3DWzbu1+adoX3OlLPfsq3ohcIAEAD2mdIXJIm1r4xden2lq7XC9abtni97tKjnmAtSTteqjx+4+PeQi2lXvXErvDjJUkTvlhfoQAA1MlcrftPJqynp8f19paNOZ86LO32ufC6StRLuhbOlq5fKM2ZIR09If1kt3Treulne2ofG2ko/Py+0Mu5zPzWkcmPtP/9tAJtmG20X/blvQ0l7XTO1Yxq7TUkLkmdtVc+C7J5tRegg4wdJU2bJF09r3L79helSz7fYKFcew0AaIH2C9iSN+N6Z/gvqtIEtM4O6d2qyWL1LKjieqWPXzDQm+6cKZ0+E7F3zcxwAECLtGfAliIFbWkgWDe66ln5cWdekE49HzEvgjUAoIXaa9JZtam1F/QuTRbzc8sS6ejTXm+59Di5w9vuZ8hFEYP11O9HSAQAQHzab9JZtYBednVgvXKO9OCdjddj8Upvxnm5wGHxOnrXeZ8skfa/n1agDbON9su+vLehMjvprNoMJ+0aIbl3Bu3qe1IaN7py28jZ0lsno2ffNUp68ylp463eQ5K+sUG6+S6fxFM3Sl2LomcOAEBM2j9gS9KF/RG4qrfdMUSaeoX0avjNO0MdOV7ZW//lI4N72pI4Zw0ASFV7n8OuVhY0Xa/00LbmgrWfcxd4121XDIcTrAEAKctGD7vcDCedOiLtHqdrL5euvTzBss4/1NR14QAAxCVbPeySzi4vcE9Zk0z+U9Z6+ROsAQBtIns97HITlnkPKdI12zUx9A0AaFPZ7GH7meEGHtOPDtq9wq8zfv4blccBANCmst3DDtIxZlAAXvV3KdUFAIAY5KeHDQBAjhGwAQDIAAI2AAAZQMAGACADUr/5h5nlenp22t9v0gqwKD9tmHG0X/YVoA0j3fyDHjYAwNeYkZW3J3a90vKrB287Z1zaNS0GetgJS/v7TRq/7rMv721I+9Un8LbCdai+/XGzCtCG9LABALXddM1AbzkO5b1xxIcedsLS/n6TlvfemUQbZh3tF6xrlPTmUzFWJsDEP5YOHWn8+AK0YaQedj5XOgMAhIqrNx3FwS3ec9xD5UXDkDgAFEwrg3U7lJsXBGwAKIjfPJt+0HS90p99Kt06ZBUBGwAKwPVKw4Y2n88Ntzefx6bb0v/hkEVMOktY2t9v0vI+YUmiDbOO9pPe2SENH9ZkOT7nn5sNur99Vxr+R7XTFaANuawLABAtWHfPle79of++oMlizU4ii6PHXyT0sBOW9vebtLz3ziTaMOuK3n61esFRes5hgblW2o9Ok356f/11qCgj/21IDxsAiqxWsP72ff7bG+05+x338p7ax3E+OxoCNgDkUHdX7TRL70i+HlK0HwDjRidfj6wjYANADh3aEl9eQT3gOHvGfU/Gl1desdIZAOTMn18z8DrsHLXrjT787XqlEyelUbOl489II0dEr8/6r0Srz7LF0jc3Rs+3aOhhA0DO3P4l7zkoGO87NPB61vTB+4N6zqUgHRSsg467bqH3/KsD/vtL9Vyzwn8/PARsACiYKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31RCUCNgDkSLPnlV8/FLzvlde85yPHg9OE7YuCGePBCNgAUDDzZwXvmzw/eF8UYb3vBZc0l3fREbABIKdO7vDf/uja1taj5OE1/tvfeba19cgqAjYA5MTEcZXvzxrmDTGfVbY0aZQh5w0PN1b+Q9tqpykvf8Rw7/3wqiVKx49prPy8Y2nShKX9/SYt78taSrRh1hWp/cKC8ekzUufM4HTVM8qr05QfL0mHnxgcWGvlUZ7m2FZp9PuC61ueVwHakKVJAQCejiHNHT/04sr33XObyy8sWMMfARsACibKYimLVla+r9XJ/dzX4ikXwWIP2Gb2t2Z2yMx+GnfeAIDWuK/OpU3Xb06mHhiQRA97g6RPJ5AvACDE8tXR07a6t1tPefV8jiKJPWA7556RdCTufAEA4VYvjze/L9wWLV3cd/2K+3PkBeewAaCgFiwL3/+dB7znbbv8929+xnsOuq92yZVVa4Rfe3ntumGwVAK2mS0xs14zYxE6AGiRqR+ofP/o9mjHzVniv/0zEXvC1ddn3/PVaMehUioB2zn3XedcT5TrzgAA8fjx3YO3zVsafkxXyFKjkjT2E+H7l60K34/oGBIHgJwY/8nw/ZMmDN72WI1lQY/WuJnHsRPh+9c2cH/rsPXIiyyJy7o2SvqJpI+Y2T4z+z/iLgMAMNibv27suKRmjF91U2PHNXvHr7zqiDtD59ziuPMEAGTPD7amXYN8YUgcAApkYle65c88L93ys4ybfyQs7e83aXm/cYREG2ZdEduv1h25Gh0C/9iHvIC/d7/0i32N5dFI3QrQhpFu/hH7kDgAoL253uCgPX9Wc/fLvuwGactzweWicQRsAMiZFWukVTeGpzm2VRozx3t9cIs0oWqo/LpbpHseiV7mrOnS9nXS43cNbNu7X5p2hff6QIS1yb8Y84ppecOQeMLS/n6TlvfhVIk2zLqitl+U3qz1DKTbtEVavDI8fT2+93Vp8WWDy6lVHz8FaMNIQ+IE7ISl/f0mLe//2Uu0YdYVtf3Gj5EOPxHh+IjnsxfOlq5fKM2ZIR09If1kt3Treulne2ofGyVYj7s0+HKuArQh57ABoKj6jjV+7ObVXoAOMnaUNG2SdPW8yu3bX5Qu+XxjZXLtdW30sBOW9vebtLz3ziTaMOuK3n5Rh6I7O6R3nxu8ParqcjpnSqfPNDcU/l7e+W9DetgAUHRRzx+XgnWjl3yVH3fmBenU89HyavV9ubOMhVMAIOcW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00GMCQeMLS/n6TlvfhVIk2zDrazxPUy64OrFfOkR68s/H6LF7pzThvpOwgBWhDZom3g7S/36Tl/T97iTbMOtpvwNvbpRHDq47vkfqelMaNrtw+crb01sno9egaJb35VOW2b2yQbr5rcMBedLN034+i512ANuQcNgBgwNkf956rA2jHEGnqFdKr+xvP+8jxyh7zLx8Z3NOWOGfdDM5hA0DBlAdN1ys9tK25YO3n3AXeddvlPw4I1s1hSDxhaX+/Scv7cKpEG2Yd7Rds7EjpyNMxViZA99zmrgsvQBtGGhKnhw0ABXX0hNfrXbYqmfyX3tF/jryJYI0B9LATlvb3m7S8984k2jDraL/6xHFHrbiHvgvQhvSwAQD1KV2PbT0Dd/Mqt2LN4G3nXFZ5HJJBDzthaX+/Sct770yiDbOO9su+ArQhPWwAAPKCgA0AQAYQsAEAyIDUVzqbMWOGentjmJbYpvJ+finv55Yk2jDraL/sy3sbRkUPGwCADEi9hw0gR3bG0BOakf8eI9AIetgAmnPwDi9QxxGspYG8Dia0/BaQUQRsAI059aYXWPd9OZn8993k5X/qYDL5AxnDkDiA+sXVm45i9zneM0PlKDh62ADq08pg3Q7lAm2CgA0gml3D0g+aO006sindOgApIWADqG2nSe7dprO54fYY6rJ3cfo/HIAUcA4bQLhdw5vOovwOTn99v/fc9G0cdw2TLvxtk5kA2UEPG0A4Vzsods+V7v2h/76g2y02fRvGGHr8QJYQsAEEqzH0XLr/cd8x6bN/1XwQLr+nsvVI5/1pc/UD8oSADcBfjWD47fv8tzcatP2Oe3lPhAMJ2igIAjaAwU4fqplk6R0tqIci/gA43Zd4PYC0EbABDPbSxNiyCppc1vSks3IvdceYGdCemCUOoNIbA9de+fVuS4HW9UYf/na90omT0qjZ0vFnpJEjoldn/VcGXofVRwfWSOfcGD1jIGPoYQOotP8vJAUH431lo+Wzpg/eH9RzLgXpoGAddNx1C73nXx3w3/9ePV9f7p8AyAkCNoC6TJk/8Hr7uspAGzbM/eGrvOdxlwanqc6r/P25C+qrJ5A3BGwAA5qccf16yFy1V17zno8cD04Tti8SZowjxwjYAOoyf1bwvsnzg/dFEdb7XnBJc3kDWUfABuDr5A7/7Y+ubW09Sh5e47/9nWdbWw8gLQRsAJ5TlbO6zhrmnUM+a9jAtiiXYm14uLHiH9pWO015+SOGe++HD61KdOpwYxUA2hwBG4Bn9/t9N5/cIZ163nsd5TKu6786eNvpM5Xv+44NTnPlitp5l8o/tlV6e3tAot0TamcEZBABG0BNHUOaO37oxZXvu+c2l9/o9zV3PJBFBGwAdYnSy160svK9c+HpP/e1eMoF8oyADSB2922pL/36zcnUA8iT2AO2mU0xs6fN7Odm9rKZfSnuMgDEb/nq6Glb3dutp7x6PgeQJUn0sE9LWuGc+58lXSzpP5nZ7yVQDoAYrY55Zc8v3BYtXdx3/Yr7cwDtIvaA7Zx7wzm3q//1CUk/lzQp7nIApGvBsvD933nAe962y3//5me856D7apdUzx6/9vLadQPyKNFz2Gb2QUm/L+n5qu1LzKzXzHoPH+aaSSALpn6g8v2jQZdVVZmzxH/7ZyL2hKuvz77H57IxoAgSC9hm9j5JD0ha5pyrWCHYOfdd51yPc66nu5v72AJZ8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+HyiSRAK2mXXKC9b3Ouf+IYkyAMRsevho1ySf9Ugeq7Es6NEaN/M4diJ8/9qN4ft9nd/XwEFA+0tilrhJWifp58455msCWdExvqHDkpoxftVNDR7YOS7WegDtIoke9ixJ10i61Mxe7H80eQ8fAEXzg61p1wBoLx1xZ+ic2y6Jm9ICOTSxSzp4JL3yZ56XXtlA2ljpDMCAGeFriB6ocwWzch/7kDT3Iul3Jzeex3MbaiSoUX8gy2LvYQPIN9cbfN56/qzm7pd92Q3SlueCywWKjIANoNLkO6V94TO+jm2VxszxXh/cIk3oqtx/3S3SPY9EL3LWdGn7Ounxuwa27d0vTbvCex2pZz/lW9ELBDKIIXEAlSbWvjF16faWrtcL1pu2eL3u0qOeYC1JO16qPH7j495CLaVe9cSu8OMlSRO+WF+hQMaYq3Xfu4T19PS43t78jnV5V7nlV9r/flqhkG146rC02+fC6ypRL+laOFu6fqE0Z4Z09IT0k93Sreuln+2JUL8o/z2c3xd4OVch2y9n8t6GknY652r+NTEkDmCwzsZXINy82gvQQcaOkqZNkq6eV7l9+4vSJZ9vsFCuvUYBELAB+JvhpJ3hPZvSBLTODundqsli9Syo4nqlj18w0JvunCmdPhOxd83McBQEARtAsAhBWxoI1o2uelZ+3JkXpFPPR8yLYI0CYdIZgHBTay/oXZos5ueWJdLRp73eculxcoe33c+QiyIG66nfj5AIyA8mnSUs75Ml0v730wq0oQJ72dWB9co50oN3Nl6XxSu9GeflAofFI/auab/sy3sbiklnAGIzw0m7RkjunUG7+p6Uxo2u3DZytvTWyejZd42S3nxK2nir95Ckb2yQbr7LJ/HUjVLXouiZAzlBwAYQzYX9Ebiqt90xRJp6hfTq/sazPnK8srf+y0cG97Qlcc4ahcY5bAD1KQuarld6aFtzwdrPuQu867YrhsMJ1ig4etgA6jfDSaeOSLvH6drLpWsvT7Cs8w81dV04kBf0sAE0prPLC9xT1iST/5S1Xv4Ea0ASPWwAzZqwzHtIka7Zromhb8AXPWwA8ZnhBh7Tjw7avcKvM37+G5XHAfBFDxtAMjrGDArAq/4upboAOUAPGwCADCBgAwCQAQRsAAAygIANAEAGpH7zDzPL9bTQtL/fpBVgUX7aMONov+wrQBty8w8AAAKdOSq92FWxacUaadWNVenO3y91vr919QpADzthaX+/SePXffblvQ1pv+yLtQ3bcHGfqD1szmEDAPLt4B1eoI4jWEsDeR1cFU9+EdHDTlja32/S+HWffXlvQ9ov+xpuw1NvSrvHx1sZP+cfkDonNnw457ABAMUVV286it3neM8JL63LkDgAIF9aGaxbWC4BGwCQD7uGpResS3aadGRTIlkTsAEA2bfTJPdu09nccHsMddm7OJEfDkw6S1ja32/SmPCSfXlvQ9ov+2q24a7hkvttU2WYz5Qv19tUlpINlS6sXS8u6wIAFEOEYN09V7r3h/77/IJ12PbIYujxl6OHnbC0v9+k8es++/LehrRf9oW2YY2h5yg957DAXCvtR6dJP70/tAo1Z4/TwwYA5FuNYP3t+/y3N9pz9jvu5T0RDozpfDYBGwCQPacP1Uyy9I4W1EMRfwCc7mu6HAI2ACB7Xmp8ZbFqQZPLmp50Vu6l7qazYKUzAEC2vDFw7VXYOWrXG3342/VKJ05Ko2ZLx5+RRo6IXp31Xxl4HXrO/MAa6ZzqW4FFRw8bAJAt+/9CUnAw3lc2Wj5r+uD9QT3nUpAOCtZBx1230Hv+1QH//e/V8/Xl/gkiImADAHJlyvyB19vXVQbasGHuD1/lPY+7NDhNdV7l789dUF8960XABgBkR5Mzrl8Pmav2ymve85HjwWnC9kXSRP0J2ACAXJk/K3jf5PnB+6II630vuKS5vGshYAMAMunkDv/tj65tbT1KHl7jv/2dZ+PJn4ANAMiGU5Wzus4a5p1DPmvYwLYol2JteLix4h/aVjtNefkjhnvvhw+tSnTqcEPlszRpwtL+fpNW+GURcyDvbUj7Zd97bRhy/vf0GalzZn96n6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdWtWK6UpUkBAIXRMaS544deXPm+e25z+YUG6wYRsAEAuRJlsZRFKyvf1xqI+dzX4im3GbEHbDMbbmYvmNlLZvaymX017jIAAGjGfVvqS79+czL1qEcSPezfSrrUOTdd0gWSPm1mF9c4BgCAUMtXR0+bdG+3mfLq+RzlYg/YzvNW/9vO/ke+Z30AABK3urmVPQf5wm3R0sV9169GP0ci57DNbIiZvSjpkKQfOeeer9q/xMx6zSzOe6EAAPCeBcvC93/nAe952y7//Zuf8Z6D7qtdcuWKyvfXXl67bo1I9LIuMxsj6UFJX3TO/TQgTa5731xSkn20YbbRftkX5bIuSZp2hbR3f9Wx/d3CoCHrWnf0CtsflHek23K222VdzrljkrZK+nSS5QAA8OO7B2+btzT8mK6QpUYlaewnwvcvWxW+P05JzBLv7u9Zy8zOkjRX0r/GXQ4AoGCmh68QNmnC4G2P1VgW9GiNm3kcOxG+f+3G8P2+zu9r4CCpo6Gjwr1f0j1mNkTeD4L7nXOPJFAOAKBIOsY3dFhSM8avuqnBAzvHNXRY7AHbObdb0u/HnS8AAO3kB1tbWx4rnQEAcmNiV7rlzzwvuby5+UfC0v5+k1aoGao5lfc2pP2yb1Ab1pgt3ugQ+Mc+5AX8vfulX+xrLI+aM8RnDP73GHWWeBLnsAEASE3YpVjzZzV3v+zLbpC2PBdcbpII2ACAbJl8p7QvfMbXsa3SmDne64NbpAlVQ+XX3SLdU8d06FnTpe3rpMfvGti2d7937bckHYiyNvmUb0Uv0AdD4glL+/tNWiGH43Im721I+2WfbxvWGBaXvF52qde7aYu0eGV4+np87+vS4ssGlxPKZzhcij4kTsBOWNrfb9IK+59FjuS9DWm/7PNtw1OHpd0+F15XiXo+e+Fs6fqF0pwZ0tET0k92S7eul362J0L9ogTr8/sCL+fiHDYAIL86uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQhu89rocPeyEpf39Jq2wv+5zJO9tSPtlX2gbRhwa7+yQ3n1u8PbIdajqRXfOlE6faW4o/L160MMGAOTeDBcpaJeCdaOXfJUfd+YF6dTzEfOqEazrwcIpAIBsm1pjBNUvAAAgAElEQVR7QW/rCQ6wtyyRjj7t9ZZLj5M7vO1+hlwUMVhP/X6ERNExJJ6wtL/fpBV+OC4H8t6GtF/2RWrDgF52dWC9co704J2N12XxSm/GebnAYfGIvWtmibeJtL/fpPGfRfblvQ1pv+yL3Ia7RkjunYpN1iP1PSmNG12ZdORs6a2T0evQNUp686nKbd/YIN18l0/AnrpR6loUOW/OYQMAiuXC/ghc1dvuGCJNvUJ6dX/jWR85Xtlb/+Ujg3vakmI9Z12Nc9gAgHwpC5quV3poW3PB2s+5C7zrtit61wkGa4kh8cSl/f0mjeG47Mt7G9J+2ddwG546Iu1u/vrnms4/1NR14VGHxOlhAwDyqbPL6/VOWZNM/lPWevk3EazrQQ87YWl/v0nj13325b0Nab/si7UNI1yzXVPMQ9/0sAEAqDbDDTymHx20e4VfZ/z8NyqPSwk97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyIPWVzmbMmKHe3ij3J8umvJ9fyvu5JYk2zDraL/vy3oZR0cMGACADUu9hI7pIN0qvodF7wQIA0kUPu83ddM3A/VnjUMpr+dXx5AcAaA0CdpvqGuUF1ju+lEz+q2708p/QlUz+AIB4MSTehuLqTUdxsP/2cAyVA0B7o4fdZloZrNuhXABANATsNvGbZ9MPmq5X+rNPpVsHAIA/AnYbcL3SsKHN53PD7c3nsem29H84AAAG4xx2yt7Z0Xwe5eef//p+77nZoPubZ6Xhf9RcHgCA+NDDTtnwYbXTdM+V7v2h/76gyWLNTiKLo8cPAIgPATtFtXrB1uM9+o5Jn/2r5oNwKb/S47w/ba5+AIDWIWCnpFYw/PZ9/tsbDdp+x728p/ZxBG0AaA8E7BR0R1isZOkdyddDivYDYNzo5OsBAAhHwE7BoS3x5RXUA46zZ9z3ZHx5AQAawyzxFvvzawZe+/VuS4HW9UYf/na90omT0qjZ0vFnpJEjotdn/Vei1WfZYumbG6PnCwCIFz3sFru9f23woGC879DA61nTB+8P6jmXgnRQsA467rqF3vOvDvjvL9VzzQr//QCA1iBgt5kp8wdeb19XGWjDhrk/fJX3PO7S4DTVeZW/P3dBffUEALQWAbuFmj2v/Pqh4H2vvOY9HzkenCZsXxTMGAeA9BCw28z8WcH7Js8P3hdFWO97wSXN5Q0ASBYBOyUnA5YkfXRta+tR8vAa/+3vPNvaegAA/BGwW2TiuMr3Zw3zhpjPKluaNMqQ84aHGyv/oW2105SXP2K493541RKl48c0Vj4AoDkE7BY58Lj/9pM7pFPPe6+jXMZ1/VcHbzt9pvJ937HBaa6MMMu7VP6xrdLb2/3THH6idj4AgPgRsNtAx5Dmjh96ceX77rnN5Tf6fc0dDwCIHwG7zUTpZS9aWfneufD0n/taPOUCANKTSMA2syFm9s9m9kgS+RfdfXUubbp+czL1AAC0TlI97C9J+nlCeWfS8tXR07a6t1tPefV8DgBAfGIP2GY2WdLlku6OO+8sW7083vy+cFu0dHHf9SvuzwEAiCaJHvY3JX1Z0v8ISmBmS8ys18x6Dx8+nEAVsm/BsvD933nAe962y3//5me856D7apdUzx6/9vLadQMAtF6sAdvMFkg65JzbGZbOOfdd51yPc66nu7s7zipk1tQPVL5/NOCyqmpzlvhv/0zEnnD19dn3+Fw2BgBIX9w97FmSrjCzVyVtknSpmf1dzGXk0o99TiDMWxp+TFfIUqOSNPYT4fuXrQrfDwBoH7EGbOfczc65yc65D0paJOkp59xn4ywjq8Z/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXIAQHK4DrtF3vx1Y8clNWP8qpsaO67ZO34BABrTkVTGzrmtkrYmlT+a84OtadcAAFAPethtZGJXuuXPPC/d8gEAwQjYLVRrePtAnSuYlfvYh6S5F0m/O7nxPJ7bEL6f5UsBID2JDYmjMa43ODDOn9Xc/bIvu0Ha8lxwuQCA9kXAbrEVa6RVN4anObZVGjPHe31wizShaqj8uluke+pYpX3WdGn7Ounxuwa27d0vTbvCex2lZ//FmFdMAwDUx1ytWz0lrKenx/X25rd7Z2aDtkXpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP2v59W8GvDPMl7G9J+2Zf3NpS00zlX86QjATthfv/Qxo+RDj8R4diI54wXzpauXyjNmSEdPSH9ZLd063rpZ3tqHxslWI+7NPhyrrT//bRC3v+zyHsb0n7Zl/c2VMSAzZB4CvqONX7s5tVegA4ydpQ0bZJ09bzK7dtflC75fGNlcu01AKSPgJ2SKEPRpQlonR3Su1WTxeqZse16pY9fMFBe50zp9JnmhsIBAK1FwE5R1PPHpWDdaPAsP+7MC9Kp56PlRbAGgPbBddgpW3Rz7TTWExw8b1kiHX3aC/ylx8kd3nY/Qy6KFoj/5Mu10wAAWodJZwmLMlkiqJddHVivnCM9eGfjdVm80ptx3kjZQdL+99MKeZ/wkvc2pP2yL+9tKCadZYf1SG9vl0YMH7yv70lp3OjKbSNnS2+djJ5/1yjpzaekjbd6D0n6xgbp5rsGp110s3Tfj6LnDQBoDQJ2mzj7495zdY+3Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBoB2xjnsNlMeNF2v9NC25oK1n3MXeNdtl/84IFgDQHujh92GrEcaO1I68rR07eXeIyndc5u7LhwA0Br0sNvU0RNe4F62Kpn8l97h5U+wBoBsoIfd5tZu9B5SPHfUYugbALKJHnaGlK7Htp6Bu3mVW7Fm8LZzLqs8DgCQTfSwM+rXb/kH4NX3tr4uAIDk0cMGACADCNgAAGQAARsAgAxIfS1xM8v1Qrhpf79JK8Aav7RhxtF+2VeANoy0ljg9bAAAMoBZ4kCr7IyhJzQj3z0NAMHoYQNJOniHF6jjCNbSQF4HE1oCD0Db4hx2wtL+fpPG+bMAp96Udo+PvzLVzj8gdU5sKou8tyF/g9lXgDbkfthAKuLqTUex+xzvmaFyIPcYEgfi1Mpg3Q7lAmgZAjYQh13D0g+aO006sindOgBIDAEbaNZOk9y7TWdzw+0x1GXv4vR/OABIBJPOEpb295u0wk942TVccr9tKn+/m7g0fStVGypdGK1eeW9D/gazrwBtyMIpQOIiBOvuudK9P/TfF3TL06ZvhRpDjx9Ae6GHnbC0v9+kFfrXfY2h5yg957DAXCvtR6dJP70/tAqRZo/nvQ35G8y+ArQhPWwgMTWC9bfv89/eaM/Z77iX90Q4kPPZQG4QsIF6nT5UM8nSO1pQD0X8AXC6L/F6AEgeARuo10vNrSxWLmhyWdOTzsq91B1jZgDSwkpnQD3eGLj2KuwcteuNPvzteqUTJ6VRs6Xjz0gjR0SvzvqvDLwOPWd+YI10zo3RMwbQduhhA/XY/xeSgoPxvrLR8lnTB+8P6jmXgnRQsA467rqF3vOvDvjvf6+ery/3TwAgMwjYQIymzB94vX1dZaANG+b+8FXe87hLg9NU51X+/twF9dUTQPYQsIGompxx/XrIXLVXXvOejxwPThO2LxJmjAOZRsAGYjR/VvC+yfOD90UR1vtecElzeQNofwRsoAEnd/hvf3Rta+tR8vAa/+3vPNvaegBIDgEbiOJU5ayus4Z555DPGjawLcqlWBsebqz4h7bVTlNe/ojh3vvhQ6sSnTrcWAUApI6lSROW9vebtMIsixhy/vf0GalzZn9an6BdPaO8Ok358ZJ0+Alp/Jj68ihPc2yrNPp9gdUdtFxp3tuQv8HsK0AbsjQp0AodQ5o7fujFle+75zaXX2iwBpBZBGwgRlEWS1m0svJ9rc7D574WT7kAsi2RgG1mr5rZv5jZi2YW5yKLQObdt6W+9Os3J1MPANmSZA/7E865C6KMywPtbvnq6Glb3dutp7x6PgeA9sKQOBDB6phX9vzCbdHSxX3Xr7g/B4DWSSpgO0lbzGynmS2p3mlmS8ysl+Fy5NWCZeH7v/OA97xtl//+zc94z0H31S65ckXl+2svr103ANmUyGVdZvYB59x+M5sg6UeSvuiceyYgba7n6xfgcoS0q5C4Wpd1SdK0K6S9+6uO6/85GjRkXeuOXmH7g/KOdFtOLuvKlby3n1SINkzvsi7n3P7+50OSHpR0URLlAO3ix3cP3jZvafgxXSFLjUrS2E+E71+2Knw/gHyJPWCb2dlmNrL0WtIfS/pp3OUALTU9fIWwSRMGb3usxrKgR2vczOPYifD9azeG7/d1fl8DBwFoBx0J5DlR0oP9wzQdkr7nnHssgXKA1ukY39BhSc0Yv+qmBg/sHBdrPQC0TuwB2zm3R9L0uPMFMOAHW9OuAYBW47IuICYTu9Itf+Z56ZYPIFnc/CNhaX+/SSvcDNUas8UbHQL/2Ie8gL93v/SLfY3lUXOG+Az/f4t5b0P+BrOvAG0YaZZ4EuewgcIKuxRr/qzm7pd92Q3SlueCywWQbwRsoB6T75T2hc/4OrZVGjPHe31wizShaqj8ulukex6JXuSs6dL2ddLjdw1s27vfu/Zbkg5EWZt8yreiFwigLTEknrC0v9+kFXI4rsawuOT1sku93k1bpMUrw9PX43tflxZfNricUAHD4VL+25C/wewrQBtGGhInYCcs7e83aYX8z+LUYWm3z4XXVaKez144W7p+oTRnhnT0hPST3dKt66Wf7YlQtyjB+vy+0Mu58t6G/A1mXwHakHPYQCI6uxs+dPNqL0AHGTtKmjZJunpe5fbtL0qXfL7BQrn2GsgFetgJS/v7TVqhf91HHBrv7JDefW7w9sjlV/WiO2dKp880PxT+Xl1y3ob8DWZfAdqQHjaQqBm1bwoiDQTrRi/5Kj/uzAvSqecj5hUhWAPIDhZOAZoxtfaC3tYTHGBvWSIdfdrrLZceJ3d42/0MuShisJ76/QiJAGQJQ+IJS/v7TRrDcQrsZVcH1ivnSA/e2Xg9Fq/0ZpxX1C1oWLyO3nXe25C/wewrQBsyS7wdpP39Jo3/LPrtGiG5dyo2WY/U96Q0bnRl0pGzpbdORi+/a5T05lOV276xQbr5Lp+APXWj1LUoeubKfxvyN5h9BWhDzmEDLXNhfwSu6m13DJGmXiG9ur/xrI8cr+yt//KRwT1tSZyzBnKOc9hAnMqCpuuVHtrWXLD2c+4C77rtit41wRrIPYbEE5b295s0huMCnDoi7W7B9c/nH2rqunAp/23I32D2FaANIw2J08MGktDZ5fV6p6xJJv8pa738mwzWALKDHnbC0v5+k8av+zpEuGa7pgSGvvPehvwNZl8B2pAeNtBWZriBx/Sjg3av8OuMn/9G5XEACosedsLS/n6Txq/77Mt7G9J+2VeANqSHDQBAXhCwAQDIAAI2AAAZkPpKZzNmzFBvb5T7BGZT3s8v5f3ckkQbZh3tl315b8Oo6GEDAJABBGwAADIg9SFxAMiKwNuZ1iHS/cwBH/SwASDETdd4gTqOYC0N5LX86njyQ3EQsAHAR9coL7De8aVk8l91o5f/hK5k8kf+MCQOAFXi6k1HcbD/3uYMlaMWetgAUKaVwbodykV2ELABQNJvnk0/aLpe6c8+lW4d0L4I2AAKz/VKw4Y2n88Ntzefx6bb0v/hgPbEOWwAhfbOjubzKD///Nf3e8/NBt3fPCsN/6Pm8kC+0MMGUGjDh9VO0z1XuveH/vuCJos1O4ksjh4/8oWADaCwavWCrcd79B2TPvtXzQfhUn6lx3l/2lz9UCwEbACFVCsYfvs+/+2NBm2/417eU/s4gjZKCNgACqc7wmIlS+9Ivh5StB8A40YnXw+0PwI2gMI5tCW+vIJ6wHH2jPuejC8vZBezxAEUyp9fM/Dar3dbCrSuN/rwt+uVTpyURs2Wjj8jjRwRvT7rvxKtPssWS9/cGD1f5A89bACFcnv/2uBBwXjfoYHXs6YP3h/Ucy4F6aBgHXTcdQu9518d8N9fqueaFf77URwEbAAoM2X+wOvt6yoDbdgw94ev8p7HXRqcpjqv8vfnLqivnigeAjaAwmj2vPLrh4L3vfKa93zkeHCasH1RMGO82AjYAFBm/qzgfZPnB++LIqz3veCS5vJG/hGwARTSyYAlSR9d29p6lDy8xn/7O8+2th5oXwRsAIUwcVzl+7OGeUPMZ5UtTRplyHnDw42V/9C22mnKyx8x3Hs/vGqJ0vFjGisf2UfABlAIBx73335yh3Tqee91lMu4rv/q4G2nz1S+7zs2OM2VEWZ5l8o/tlV6e7t/msNP1M4H+UTABlB4HUOaO37oxZXvu+c2l9/o9zV3PPIpkYBtZmPM7O/N7F/N7Odm9odJlAMAcYvSy160svK9c+HpP/e1eMpFsSXVw14r6THn3L+TNF3SzxMqBwBa7r46lzZdvzmZeqBYYg/YZjZK0mxJ6yTJOfeuc87njA4AtM7y1dHTtrq3W0959XwO5EsSPexpkg5LWm9m/2xmd5vZ2QmUAwCRrV4eb35fuC1aurjv+hX350B2JBGwOyRdKOlvnHO/L+ltSX9ZnsDMlphZr5n1Hj58OIEqAEBzFiwL3/+dB7znbbv8929+xnsOuq92SfXs8Wsvr103FFMSAXufpH3Ouf4LJfT38gL4e5xz33XO9Tjnerq7uxOoAgDUZ+oHKt8/GnBZVbU5S/y3fyZiT7j6+ux7fC4bA6QEArZz7oCk18zsI/2bPinpZ3GXAwBx+vHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfD5RLapb4FyXda2a7JV0g6daEygGASMZ/Mnz/pAmDtz1WY1nQozVu5nHsRPj+tQ3c3zpsPXLkW0cSmTrnXpTEVYUA2sabv27suKRmjF91U2PHNXvHL2QXK50BQAp+sDXtGiBrCNgA0G9iV7rlzzwv3fLR3gjYAAqj1vD2gTpXMCv3sQ9Jcy+Sfndy43k8tyF8P8uXFlsi57ABIKtcb3BgnD+ruftlX3aDtOW54HKBMARsAIWyYo206sbwNMe2SmPmeK8PbpEmVA2VX3eLdM8j0cucNV3avk56/K6BbXv3S9Ou8F5H6dl/MeYV05A95mrdZiZhPT09rrc3vz8tzSztKiQq7X8/rUAbZptf+0XpzVrPQLpNW6TFK8PT1+N7X5cWXza4nFr18ZP39pPy/zcoaadzruYJDwJ2wvL+Dy3tfz+tQBtmm1/7jR8jHX4iwrERzxkvnC1dv1CaM0M6ekL6yW7p1vXSz/bUPjZKsB53afDlXHlvPyn/f4OKGLAZEgdQOH1N3D9w82ovQAcZO0qaNkm6el7l9u0vSpd8vrEyufYaEgEbQEFFGYouTUDr7JDerZosVs+MbdcrffyCgfI6Z0qnzzQ3FI7iIWADKKyo549LwbrR4Fl+3JkXpFPPR8uLYI1yXIcNoNAW3Vw7jfUEB89blkhHn/YCf+lxcoe33c+Qi6IF4j/5cu00KBYmnSUs75Ml0v730wq0YbZFab+gXnZ1YL1yjvTgnY3XZfFKb8Z5I2UHyXv7Sfn/GxSTzgAgGuuR3t4ujRg+eF/fk9K40ZXbRs6W3joZPf+uUdKbT0kbb/UekvSNDdLNdw1Ou+hm6b4fRc8bxUHABgBJZ3/ce67u8XYMkaZeIb26v/G8jxyv7DH/8pHBPW2Jc9YIxzlsAChTHjRdr/TQtuaCtZ9zF3jXbZf/OCBYoxZ62ABQxXqksSOlI09L117uPZLSPbe568JRHPSwAcDH0RNe4F62Kpn8l97h5U+wRlT0sAEgxNqN3kOK545aDH2jUfSwASCi0vXY1jNwN69yK9YM3nbOZZXHAY2ihw0ADfj1W/4BePW9ra8LioEeNgAAGUDABgAgAwjYAABkQOpriZtZrhfCTfv7TVoB1vilDTOO9su+ArRhpLXE6WEDAJABzBJH2+AaVwAIRg8bqbrpmoF7CMehlNfyq+PJDwDaBeewE5b295u0Rs+flW43mLSJfywdOtJcHrRhttF+2VeANuR+2GhPcfWmozjYfwtDhsoBZB1D4mipVgbrdigXAOJCwEZL/ObZ9IOm65X+7FPp1gEAGkXARuJcrzRsaPP53HB783lsui39Hw4A0AgmnSUs7e83abUmvLyzQxo+rMkyfM4/Nxt0f/uuNPyPoqUtehtmHe2XfQVoQxZOQfqiBOvuudK9P/TfFzRZrNlJZHH0+AGglehhJyzt7zdpYb/ua/WCo/ScwwJzrbQfnSb99P766zConAK3YR7QftlXgDakh4301ArW377Pf3ujPWe/417eU/s4zmcDyAoCNmLX3VU7zdI7kq+HFO0HwLjRydcDAJpFwEbsDm2JL6+gHnCcPeO+J+PLCwCSwkpniNWfXzPwOuwcteuNPvzteqUTJ6VRs6Xjz0gjR0Svz/qvRKvPssXSNzdGzxcAWo0eNmJ1+5e856BgvO/QwOtZ0wfvD+o5l4J0ULAOOu66hd7zrw747y/Vc80K//0A0C4I2GipKfMHXm9fVxlow4a5P3yV9zzu0uA01XmVvz93QX31BIB2Q8BGbJo9r/z6oeB9r7zmPR85HpwmbF8UzBgH0M4I2Gip+bOC902eH7wvirDe94JLmssbANJGwEYiTu7w3/7o2tbWo+ThNf7b33m2tfUAgEYRsBGLieMq3581zBtiPqtsadIoQ84bHm6s/Ie21U5TXv6I4d774VVLlI4f01j5AJA0liZNWNrfb9JKyyKGBePTZ6TOmQpMVz2jvDpN+fGSdPiJwYG1Vh7laY5tlUa/L7i+g/IqSBvmFe2XfQVoQ5YmRXvoGNLc8UMvrnzfPbe5/MKCNQC0KwI2WirKYimLVla+r/Xj+nNfi6dcAGhnsQdsM/uImb1Y9jhuZsviLgf5dV+dS5uu35xMPQCgncQesJ1z/+acu8A5d4GkGZJOSnow7nLQXpavjp621b3desqr53MAQCslPST+SUm/cM79MuFykLLVy+PN7wu3RUsX912/4v4cABCXpAP2IkmDbqlgZkvMrNfMWFuqoBbUOEnynQe85227/PdvfsZ7DrqvdsmVVWuEX3t57boBQDtK7LIuMxsqab+kjzrnDoaky/V8/QJcjiCp9jXW066Q9u6v3FY6JmjIutYdvcL2B+Ud5VpwLuvKF9ov+wrQhqlf1jVP0q6wYI3i+PHdg7fNWxp+TFfIUqOSNPYT4fuXrQrfDwBZkmTAXiyf4XDk0/hPhu+fNGHwtsdqLAt6tMbNPI6dCN+/toF/fWHrkQNAmhIJ2GY2QtKnJP1DEvmj/bz568aOS2rG+FU3NXZcs3f8AoCkdCSRqXPupKRxNRMCCfnB1rRrAADxYqUztMzErnTLn3leuuUDQDO4+UfC0v5+k1Y9Q7XWLOxGh8A/9iEv4O/dL/1iX2N5NFq3orVh3tB+2VeANow0SzyRIXEgSNilWPNnNXe/7MtukLY8F1wuAGQZARuxWrFGWnVjeJpjW6Uxc7zXB7dIE6qGyq+7RbrnkehlzpoubV8nPX7XwLa9+71rvyXpQIS1yb8Y84ppABA3hsQTlvb3mzS/4bioi5OU0m3aIi1eGZ6+Ht/7urT4ssHl1KpPkCK2YZ7QftlXgDaMNCROwE5Y2t9v0vz+sxg/Rjr8RIRjI57PXjhbun6hNGeGdPSE9JPd0q3rpZ/tqX1slGA97tLwy7mK2IZ5QvtlXwHakHPYSEffscaP3bzaC9BBxo6Spk2Srp5XuX37i9Iln2+sTK69BpAF9LATlvb3m7SwX/dRh6I7O6R3nxu8ParqcjpnSqfPND8U/l7+BW7DPKD9sq8AbUgPG+mKev64FKwbveSr/LgzL0inno+WV6vvyw0AzWDhFCRq0c2101hPcPC8ZYl09Gkv8JceJ3d42/0MuShaIP6TL9dOAwDthCHxhKX9/SYtynBcUC+7OrBeOUd68M7G67J4pTfjvJGyw9CG2Ub7ZV8B2pBZ4u0g7e83aVH/s3h7uzRieNWxPVLfk9K40ZXbR86W3joZvQ5do6Q3n6rc9o0N0s13DQ7Yi26W7vtR9Lwl2jDraL/sK0Abcg4b7ePsj3vP1QG0Y4g09Qrp1f2N533keGWP+ZePDO5pS5yzBpBtnMNGS5UHTdcrPbStuWDt59wF3nXb5T8OCNYAso4h8YSl/f0mrdHhuLEjpSNPx1wZH91zm7suXKINs472y74CtGGkIXF62EjF0RNer3fZqmTyX3pH/znyJoM1ALQLetgJS/v7TVqcv+7juKNWEkPftGG20X7ZV4A2pIeNbCldj209A3fzKrdizeBt51xWeRwA5BU97ISl/f0mjV/32Zf3NqT9sq8AbUgPGwCAvCBgAwCQAQRsAAAyoB1WOuuT9MsWlje+v8yWSOn8Uks/Ywry3oa0X4xov9i1/PMVoA3PjZIo9UlnrWZmvVFO7mdZ3j8jny/b+HzZlvfPJ7XvZ2RIHACADCBgAwCQAUUM2N9NuwItkPfPyOfLNj5ftuX980lt+hkLdw4bAIAsKmIPGwCAzCFgAwCQAYUK2Gb2aTP7NzN7xcz+Mu36xMnM/tbMDpnZT9OuSxLMbIqZPW1mPzezl83sS2nXKW5mNtzMXjCzl/o/41fTrlPczGyImf2zmT2Sdl2SYGavmtm/mNmLZhbD/efai5mNMbO/N7N/7f9b/MO06xQXM/tIf7uVHsfNbFna9SpXmHPYZjZE0v8n6VOS9kn6J0mLnXM/S7ViMTGz2ZLekvTfnHPnpV2fuJnZ+yW93zm3y8xGStop6cq8tJ8kmbc6xNnOubfMrFPSdklfcs49l3LVYmNmyyX1SBrlnFuQdn3iZmavSupxzuVy4RQzu0fSj51zd5vZUEkjnHO5u+t8f7x4XdJM51wrF/YKVaQe9kWSXnHO7XHOvStpk6TPpFyn2DjnnpF0JO16JMU594Zzblf/6xOSfi5pUrq1ipfzvNX/tl2ok/MAAAJgSURBVLP/kZtf1GY2WdLlku5Ouy6on5mNkjRb0jpJcs69m8dg3e+Tkn7RTsFaKlbAniTptbL3+5Sz//CLwsw+KOn3JT2fbk3i1z9k/KKkQ5J+5JzL02f8pqQvS/ofaVckQU7SFjPbaWZL0q5MzKZJOixpff9pjbvN7Oy0K5WQRZI2pl2JakUK2H6L0eam91IUZvY+SQ9IWuacO552feLmnDvjnLtA0mRJF5lZLk5vmNkCSYecczvTrkvCZjnnLpQ0T9J/6j9VlRcdki6U9DfOud+X9LakXM0FkqT+of4rJH0/7bpUK1LA3idpStn7yZL2p1QXNKD/vO4Dku51zv1D2vVJUv9Q41ZJn065KnGZJemK/nO8myRdamZ/l26V4uec29//fEjSg/JOxeXFPkn7ykZ9/l5eAM+beZJ2OecOpl2RakUK2P8k6cNmNrX/F9QiSZtTrhMi6p+QtU7Sz51zq9OuTxLMrNvMxvS/PkvSXEn/mm6t4uGcu9k5N9k590F5f3tPOec+m3K1YmVmZ/dPiFT/UPEfS8rNVRvOuQOSXjOzj/Rv+qSk3Ez6LLNYbTgcLrXH7TVbwjl32sxukPS4pCGS/tY593LK1YqNmW2UNEfSeDPbJ+krzrl16dYqVrMkXSPpX/rP8UrSSufcP6ZYp7i9X9I9/TNUf0fS/c65XF7+lFMTJT3YfyvIDknfc849lm6VYvdFSff2d3r2SLo+5frEysxGyLuS6D+mXRc/hbmsCwCALCvSkDgAAJlFwAYAIAMI2AAAZAABGwCADCBgAwCQAQRsAAAygIANAEAG/P+uMuaa/akHvAAAAABJRU5ErkJggg==", "text/plain": [ "" ] @@ -5623,7 +5626,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FdWd7/nP93IOIIZfBw6YAGOg\nkyczHSO2nBa7iQwxpA0IRmd6umGMXs1kuJO5hqDY6Zbn6Scmz41mVCB07OncXGnw3jagaduI2lGi\nEQwYtQ+00jHpnseAiYj8OMIJKCYCd80fdbZn/6iqXWfvql27qt6v59nP3rtq1Vpr73Xgu9eqVavM\nOScAANDe/l3aFQAAAPURsAEAyAACNgAAGUDABgAgAwjYAABkAAEbAIAMIGADAJABBGwAADKAgA20\nGTP7oJn9o5kdM7ODZna3mXWEpB9nZn8zkPakmf2Lmf37VtYZQPII2ED7+X8lHZb0fkkXSPqfJf3f\nfgnNbLikJyWdK+kPJI2V9GeS7jCz5S2pLYCWIGAD7We6pAecc79xzh2U9LikjwakvUbS/yDpf3PO\n7XPOnXLOPS5puaT/ZGajJcnMnJl9qHSQmW00s/9U9n6Rmb1oZv1m9qyZnV+27wNm9qCZHTGzfeU/\nBMzsVjN7wMz+q5mdMLOXzaynbP+fm9nrA/v+zcw+Gc9XBBQPARtoP+skLTGzUWY2RdICeUHbz6ck\n/cA593bV9gcljZJ0cb3CzOxCSX8r6T9ImiDpP0vaYmYjzOzfSXpE0kuSpkj6pKQVZnZZWRZXSNos\naZykLZLuHsj3I5JukPT7zrnRki6T9Gq9+gDwR8AG2s92eT3q45L2S+qV9P2AtBMlvVG90Tl3WlKf\npO4I5f2fkv6zc+5559wZ59y9kn4rL9j/vqRu59zXnHPvOuf2SvovkpaUHb/DOfePzrkzkv6bpJkD\n289IGiHpd82s0zn3qnPuFxHqA8AHARtoIwM92ick/YOks+UF5PGS/p+AQ/rkneuuzqdj4NgjEYo9\nV9LKgeHwfjPrlzRN0gcG9n2gat8qSZPLjj9Y9vqkpJFm1uGce0XSCkm3SjpsZpvN7AMR6gPABwEb\naC9d8oLl3c653zrn3pS0QdLCgPRPSlpgZmdXbf9fJZ2S9MLA+5PyhshLzil7/ZqkrzvnxpU9Rjnn\nNg3s21e1b7RzLqg+FZxz33XOfVxe4HcK/uEBoA4CNtBGnHN9kvZJ+oKZdZjZOEn/Xt45ZD//Td6w\n+fcGLgfrHDi//FeS7nDO/Xog3YuS/nczG2Zmn5Y387zkv0j6v8xstnnONrPLByasvSDp+MDksbMG\njj/PzH6/3mcxs4+Y2aVmNkLSbyS9I2+YHEADCNhA+/lfJH1a3nD2K5JOS7rRL6Fz7reS5svrCT8v\nLyg+Lumbkr5alvRLkhZL6pd0tcrOiTvneuWdx75b0rGBMq8b2Hdm4LgL5P2Q6JN0j7zLx+oZIekb\nA8cclDRJ3nA6gAaYcy7tOgCIiZl1SvqBpNclXef4Bw7kBj1sIEecc6fknb/+haSPpFwdADGihw0A\nQAbQwwYAIAMCbyjQKhMnTnQf/OAH065GYnbt2pV2FRI1a9astKuQONow22i/7Mt7G0rqc87VXeQo\n9SHxnp4e19vbm2odkmRmaVchUWn//bRCXG3oYvgzH1ylOz55b0P+DWZf3ttQ0i7nXN1/3QyJAwm6\n+RovUMcRrKXBvG66Op78AGQHARtIQNcYL7De+aVk8l99o5f/pK5k8gfQflI/hw3kTVy96SgObfWe\nkxgqB9Be6GEDMWplsG6HcgG0DgEbiMFvnk0/aLpe6U8/lW4dACSHgA00yfVKI4Y3n88NdzSfx+bb\n0//hACAZnMMGmvDOzubzKD///NcPeM/NBt3fPCuN/MPm8gDQXuhhA00YOaJ+mu750n0/8N8XNFms\n2UlkcfT4AbQXAjbQoHq9YOvxHn390mf/svkgXMqv9DjvT5qrH4BsIWADDagXDL91v//2RoO233Ev\n761/HEEbyA8CNjBE3REWK1l+Z/L1kKL9AJgwNvl6AEgeARsYosNb48srqAccZ8+476n48gKQHmaJ\nA0PwZ9cMvvbr3ZYCreuNPvzteqUTJ6Uxc6Xjz0ijR0Wvz4avRKvPiqXSNzdFzxdA+6GHDQzBHQNr\ngwcF4/2HB1/PmVm7P6jnXArSQcE66LjrFnvPvzrov79Uz7Ur/fcDyA4CNhCjaQsHX+9YXxlow4a5\nP3yV9zzh0uA01XmVvz930dDqCSB7CNhARM2eV379cPC+V17zno8eD04Tti8KZowD2UbABmK0cE7w\nvqkLg/dFEdb7XnRJc3kDaH8EbKABJwOWJH1sXWvrUfLIWv/t7zzb2noASA4BG4hg8oTK92eN8IaY\nzypbmjTKkPPGRxor/+Ht9dOUlz9qpPd+ZNUSpRPHNVY+gPQRsIEIDj7hv/3kTunU897rKJdxXf/V\n2m2nz1S+7+uvTXNlhFnepfL7t0lv7/BPc+TJ+vkAaE8EbKBJHcOaO374xZXvu+c3l9/Y9zV3PID2\nRMAGYhSll71kVeV758LTf+5r8ZQLINsI2ECL3T/EpU03bEmmHgCyJZGAbWafNrN/M7NXzOwvkigD\naKWb1kRP2+re7lDKG8rnANBeYg/YZjZM0l9LWiDpdyUtNbPfjbscoJXW3BRvfl+4PVq6uO/6Fffn\nANA6SfSwL5L0inNur3PuXUmbJX0mgXKAtrVoRfj+bz/oPW/f7b9/yzPec9B9tUuqZ49fe3n9ugHI\npiQC9hRJr5W93z+w7T1mtszMes2s98iRIwlUAWit6R+ofP9YwGVV1eYt89/+mYg94errs+/1uWwM\nQD4kEbDNZ1vFPFjn3Heccz3OuZ7u7u4EqgC01o/vqd22YHn4MV0hS41K0vhPhO9fsTp8P4B8SSJg\n75c0rez9VEkHEigHaJmJnwzfP2VS7bbH6ywLeqzOzTz6T4TvX9fA/a3D1iMH0N6SCNj/JOnDZjbd\nzIZLWiKJC1OQaW/+urHjkpoxftXNjR3X7B2/AKSnI+4MnXOnzewGSU9IGibpb51zL8ddDlBk39+W\ndg0AtFrsAVuSnHP/KOkfk8gbaFeTu6RDR9Mrf/Z56ZUNIHmsdAZEVG94++AQVzAr97EPSfMvkn5n\nauN5PLcxfD/LlwLZlkgPGygq1xscGBfOae5+2ZfdIG19LrhcAPlGwAaGYOVaafWN4Wn6t0nj5nmv\nD22VJnVV7r/uVuneR6OXOWemtGO99MTdg9v2HZBmXOG9jtKz/2LMK6YBaD1z9W4VlLCenh7X25vf\n7oGZ32Xp+ZH2308rVLdhlN6s9Qym27xVWroqPP1QfPfr0tLLasupV58geW9D/g1mX97bUNIu51zd\nk1YE7ITl/Q8t7b+fVqhuw4njpCNPRjgu4jnjxXOl6xdL82ZJx05IP9kj3bZB+tne+sdGCdYTLg2/\nnCvvbci/wezLexsqYsBmSBwYor7+xo/dssYL0EHGj5FmTJGuXlC5fceL0iWfb6xMrr0G8oGADTQg\nylB0aQJaZ4f0btVksaHM2Ha90scvGCyvc7Z0+kzzQ+EAsoWADTQo6vnjUrBuNHiWH3fmBenU89Hy\nIlgD+cJ12EATltxSP431BAfPW5dJx572An/pcXKnt93PsIuiBeI//nL9NACyhUlnCcv7ZIm0/35a\noV4bBvWyqwPrlfOkh+5qvB5LV3kzzhspO0ze25B/g9mX9zYUk86A1rAe6e0d0qiRtfv6npImjK3c\nNnqu9NbJ6Pl3jZHe/JG06TbvIUnf2Cjdcndt2iW3SPf/MHreALKDgA3E4OyPe8/VPd6OYdL0K6RX\nm7jB7NHjlT3mXz5a29OWOGcN5B3nsIEYlQdN1ys9vL25YO3n3EXeddvlPw4I1kD+0cMGYmY90vjR\n0tGnpWsv9x5J6Z7f3HXhALKDHjaQgGMnvMC9YnUy+S+/08ufYA0UBz1sIEHrNnkPKZ47ajH0DRQX\nPWygRUrXY1vP4N28yq1cW7vtnMsqjwNQXPSwgRT8+i3/ALzmvtbXBUA20MMGACADCNgAAGQAARsA\ngAwgYAMAkAGp3/zDzHK9cn3a32/SCrAoP22YcbRf9hWgDbn5R66dOSa92FWxaeVaafWNVenOPyB1\nvr919QIAJIIedsJi/X53xfBLela8Xze/7rMv721I+2VfAdowUg+bc9jt7tCdXqCOI1hLg3kdSmjN\nTABAIuhhJ6zh7/fUm9KeifFWxs/5B6XOyQ0fzq/77Mt7G9J+2VeANuQcdmbF1ZuOYs853nPMQ+UA\ngHgxJN5uWhms26FcAEAkBOx2sXtE+kFzl0lHN6dbBwCALwJ2O9hlknu36WxuuCOGuuxbmv4PBwBA\nDSadJazu97t7pOR+21QZfnd9avreyzZcurB+vZjwkn15b0PaL/sK0IZc1pUJEYJ193zpvh/47wu6\nR3LT906OoccPAIgPPeyEhX6/dYaeo/ScwwJzvbQfnSH99IHQKtSdPc6v++zLexvSftlXgDakh93W\n6gTrb93vv73RnrPfcS/vjXAg57MBoC0QsNNw+nDdJMvvbEE9FPEHwOm+xOsBAAhHwE7DS42vLFYt\naHJZ05POyr3UHWNmAIBGsNJZq70xeO1V2Dlq1xt9+Nv1SidOSmPmSsefkUaPil6dDV8ZfB16zvzg\nWumc6luBAQBahR52qx34c0nBwXh/2Wj5nJm1+4N6zqUgHRSsg467brH3/KuD/vvfq+frN/knAAC0\nBAG7zUxbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuGVk8AQGsRsFupyRnXr4fMVXvlNe/56PHg\nNGH7ImHGOACkhoDdZhbOCd43dWHwvijCet+LLmkubwBAsgjYKTm503/7Y+taW4+SR9b6b3/n2dbW\nAwDgj4DdKqcqZ3WdNcI7h3zWiMFtUS7F2vhIY8U/vL1+mvLyR4303o8cXpXo1JHGKgAAaApLkybs\nve835Pzv6TNS5+yB9D5Bu3pGeXWa8uMl6ciT0sRxQ8ujPE3/Nmns+wKrW7FcKcsiZl/e25D2y74C\ntCFLk2ZFx7Dmjh9+ceX77vnN5RcarAEAqSBgt5koi6UsWVX5vt6Pz899LZ5yAQDpiT1gm9nfmtlh\nM/tp3HnDc//WoaXfsCWZegAAWieJHvZGSZ9OIN9Mu2lN9LSt7u0OpbyhfA4AQHxiD9jOuWckHY07\n36xbE/PKnl+4PVq6uO/6FffnAABEwznsNrVoRfj+bz/oPW/f7b9/yzPec9B9tUuuXFn5/trL69cN\nANB6qQRsM1tmZr1mFudNIDNt+gcq3z+2I9px85b5b/9MxJ5w9fXZ93412nEAgNZKJWA7577jnOuJ\nct1ZUfz4ntptC5aHH9MVstSoJI3/RPj+FavD9wMA2gdD4q0yM3yFsCmTarc9XmdZ0GN1bubRfyJ8\n/7pN4ft9nd/XwEEAgGYlcVnXJkk/kfQRM9tvZv9H3GVkUsfEhg5Lasb4VTc3eGDnhFjrAQCIpiPu\nDJ1zS+POE/H7/ra0awAAGAqGxNvI5K50y599XrrlAwCCcfOPhNV8vyE3AZEaHwL/2Ie8gL/vgPSL\n/Y3lUfduYbNqm4obD2Rf3tuQ9su+ArRhpJt/xD4kjua43uCgvXBOc/fLvuwGaetzweUCANoXAbvV\npt4l7Q+f8dW/TRo3z3t9aKs0qWqo/LpbpXsfjV7knJnSjvXSE3cPbtt3QJpxhff6YJS1yaf9VfQC\nAQCxY0g8Yb7fb51hccnrZZd6vZu3SktXhacfiu9+XVp6WW05oXyGwyWG4/Ig721I+2VfAdow0pA4\nATthvt/vqSPSHp8Lr6tEPZ+9eK50/WJp3izp2AnpJ3uk2zZIP9sboX5RgvX5fYGXc/GfRfblvQ1p\nv+wrQBtyDrttdXY3fOiWNV6ADjJ+jDRjinT1gsrtO16ULvl8g4Vy7TUApI4edsJCv9+IQ+OdHdK7\nz9Vuj1yHql5052zp9JnmhsLfqwe/7jMv721I+2VfAdqQHnbbm+UiBe1SsG70kq/y4868IJ16PmJe\ndYI1AKB1WDglbdPrL+htPcEB9tZl0rGnvd5y6XFyp7fdz7CLIgbr6d+LkAgA0CoMiScs0vcb0Muu\nDqxXzpMeuqvxuixd5c04Lxc4LB6xd81wXPblvQ1pv+wrQBsyS7wdRP5+d4+S3DsVm6xH6ntKmjC2\nMunoudJbJ6PXoWuM9OaPKrd9Y6N0y90+AXv6JqlrSeS8+c8i+/LehrRf9hWgDTmHnSkXDkTgqt52\nxzBp+hXSqwcaz/ro8cre+i8fre1pS+KcNQC0Mc5ht5uyoOl6pYe3Nxes/Zy7yLtuu6J3TbAGgLbG\nkHjCGv5+Tx2V9rTg+ufzDzd1XTjDcdmX9zak/bKvAG0YaUicHna76uzyer3T1iaT/7R1Xv5NBGsA\nQOvQw05YrN9vhGu264p56Jtf99mX9zak/bKvAG1IDzt3ZrnBx8xjNbtX+nXGz3+j8jgAQCbRw05Y\n2t9v0vh1n315b0PaL/sK0Ib0sAEAyAsCNgAAGUDABgAgA1Jf6WzWrFnq7Y1yn8dsyvv5pbyfW5Jo\nw6yj/bIv720YFT1sAAAyIPUeNgCgjbTheg/w0MMGgKI7dKcXqOMI1tJgXodWx5MfJBGwAaC4Tr3p\nBdb9X04m//03e/mfOpRM/gXDkDgAFFFcveko9pzjPTNU3hR62ABQNK0M1u1Qbk4QsAGgKHaPSD9o\n7jLp6OZ065BRBGwAKIJdJrl3m87mhjtiqMu+pen/cMggzmEDQN7tHtl0FlZ2a4q/fsB7ds2uebV7\nhHThb5vMpDjoYQNA3rn6QbF7vnTfD/z3WcB9pIK2RxZDj79ICNgAkGd1hp6tx3v09Uuf/cvmg3Ap\nv9LjvD9prn4YRMAGgLyqEwy/db//9kaDtt9xL++NcCBBOxICNgDk0enDdZMsv7MF9VDEHwCn+xKv\nR9YRsAEgj16aHFtWQZPLmp50Vu6l7hgzyydmiQNA3rwxeO2VX++2FGhdb/Thb9crnTgpjZkrHX9G\nGj0qenU2fGXwdVh9dHCtdM6N0TMuGHrYAJA3B/5cUnAw3l82Wj5nZu3+oJ5zKUgHBeug465b7D3/\n6qD//vfq+fpN/gkgiYANAIUzbeHg6x3rKwNt2DD3h6/ynidcGpymOq/y9+cuGlo9UYmADQB50uSM\n69dD5qq98pr3fPR4cJqwfZEwYzwQARsACmbhnOB9UxcG74sirPe96JLm8i46AjYA5NTJnf7bH1vX\n2nqUPLLWf/s7z7a2HllFwAaAvDhVOavrrBHeOeSzRgxui3Ip1sZHGiv+4e3105SXP2qk937k8KpE\np440VoGcI2ADQF7seb/v5pM7pVPPe6+jXMZ1/Vdrt50+U/m+r782zZUr6+ddKr9/m/T2joBEeybV\nz6iACNgAUAAdw5o7fvjFle+75zeX39j3NXd8ERGwAaBgovSyl6yqfO9cePrPfS2echGMgA0AqHH/\n1qGl37AlmXpgUOwB28ymmdnTZvZzM3vZzL4UdxkAgFo3rYmettW93aGUN5TPUSRJ9LBPS1rpnPuf\nJF0s6T+a2e8mUA4AoMyamFf2/MLt0dLFfdevuD9HXsQesJ1zbzjndg+8PiHp55KmxF0OAKA5i1aE\n7//2g97z9t3++7c84z0H3Ve7pHr2+LWX168baiV6DtvMPijp9yQ9X7V9mZn1mlnvkSNcbwcArTD9\nA5XvHwu6rKrKvGX+2z8TsSdcfX32vT6XjaG+xAK2mb1P0oOSVjjnKlaXdc59xznX45zr6e7mHqgA\n0Ao/vqd224Ll4cd0hSw1KknjPxG+f8Xq8P2ILpGAbWad8oL1fc65f0iiDABAlZnhI5ZTfNYjebzO\nsqDH6tzMo/9E+P51m8L3+zq/r4GD8i+JWeImab2knzvnmOsHAK3SMbGhw5KaMX7VzQ0e2Dkh1nrk\nRRI97DmSrpF0qZm9OPBo8v4vAICs+f62tGuQLx1xZ+ic2yGJG5oCQBua3CUdOppe+bPPS6/srGOl\nMwDIk1nha4geHOIKZuU+9iFp/kXS70xtPI/nNtZJUKf+RRZ7DxsA0N5cb/B564Vzmrtf9mU3SFuf\nCy4XjSNgA0DeTL1L2h8+46t/mzRunvf60FZpUlfl/utule59NHqRc2ZKO9ZLT9w9uG3fAWnGFd7r\nSD37aX8VvcACYkgcAPJmcv0bU5dub+l6vWC9eavX6y49hhKsJWnnS5XHb3rCW6il1Kue3BV+vCRp\n0heHVmjBmKt3z7SE9fT0uN7e/I6TeFe55Vfafz+tQBtmW2Hb79QRaY/PhddVol7StXiudP1iad4s\n6dgJ6Sd7pNs2SD/bG6GOUf6LP78v8HKuvLehpF3OubotwZA4AORRZ+OrSG5Z4wXoIOPHSDOmSFcv\nqNy+40Xpks83WCjXXtdFwAaAvJrlpF3hvdPSBLTODundqsliQ1lQxfVKH79gsDfdOVs6fSZi75qZ\n4ZEQsAEgzyIEbWkwWDe66ln5cWdekE49HzEvgnVkTDoDgLybXn9B79JkMT+3LpOOPe31lkuPkzu9\n7X6GXRQxWE//XoREKGHSWcLyPlki7b+fVqANs432GxDQy64OrFfOkx66q/H6LF3lzTgvFzgsHrF3\nnfc2FJPOAADvmeWk3aMk907Nrr6npAljK7eNniu9dTJ69l1jpDd/JG26zXtI0jc2Srfc7ZN4+iap\na0n0zCGJgA0AxXHhQASu6m13DJOmXyG9eqDxrI8er+yt//LR2p62JM5ZN4Fz2ABQNGVB0/VKD29v\nLlj7OXeRd912xXA4wbop9LABoIhmOenUUWnPBF17uXTt5QmWdf7hpq4Lh4ceNgAUVWeXF7inrU0m\n/2nrvPwJ1rGghw0ARTdphfeQIl2zXRdD34mghw0AGDTLDT5mHqvZvdKvM37+G5XHIRH0sAEA/jrG\n1QTg1X+XUl1ADxsAgCwgYAMAkAEEbAAAMoCADQBABqR+8w8zy/WUwrS/36QVYFF+2jDjaL/sK0Ab\nRrr5Bz1stKVxoytv5ed6pZuurt12zoS0awoArUEPO2Fpf79Ji/PXfeAt+IYg0j14h4g2zDbaL/sK\n0Ib0sNH+br5msLcch/LeOADkCT3shKX9/Sat0V/3pXvnJm3yH0mHjzaXB22YbbRf9hWgDSP1sFnp\nDC0XV286ikMD9+NNYqgcAFqJIXG0VCuDdTuUCwBxIWCjJX7zbPpB0/VKf/qpdOsAAI0iYCNxrlca\nMbz5fG64o/k8Nt+e/g8HAGgEk84Slvb3m7R6E17e2SmNHNFkGT7nn5sNur99Vxr5h9HSFr0Ns472\ny74CtCGXdSF9UYJ193zpvh/47wuaLNbsJLI4evwA0Er0sBOW9vebtLBf9/V6wVF6zmGBuV7aj86Q\nfvrA0OtQU06B2zAPaL/sK0Ab0sNGeuoF62/d77+90Z6z33Ev761/HOezAWQFARux6+6qn2b5ncnX\nQ4r2A2DC2OTrAQDNImAjdoe3xpdXUA84zp5x31Px5QUASWGlM8Tqz64ZfB12jtr1Rh/+dr3SiZPS\nmLnS8Wek0aOi12fDV6LVZ8VS6ZuboucLAK1GDxuxuuNL3nNQMN5/ePD1nJm1+4N6zqUgHRSsg467\nbrH3/KuD/vtL9Vy70n8/ALQLAjZaatrCwdc71lcG2rBh7g9f5T1PuDQ4TXVe5e/PXTS0egJAuyFg\nIzbNnld+/XDwvlde856PHg9OE7YvCmaMA2hnBGy01MI5wfumLgzeF0VY73vRJc3lDQBpI2AjESd3\n+m9/bF1r61HyyFr/7e8829p6AECjCNiIxeQJle/PGuENMZ9VtjRplCHnjY80Vv7D2+unKS9/1Ejv\n/ciqJUonjmusfABIGkuTJizt7zdppWURw4Lx6TNS52wFpqueUV6dpvx4STryZG1grZdHeZr+bdLY\n9wXXtyavgrRhXtF+2VeANmRpUrSHjmHNHT/84sr33fObyy8sWANAuyJgo6WiLJayZFXl+3o/rj/3\ntXjKBYB2FnvANrORZvaCmb1kZi+b2VfjLgP5dv8QlzbdsCWZegBAO0mih/1bSZc652ZKukDSp83s\n4jrHIONuWhM9bat7u0MpbyifAwBaKfaA7TxvDbztHHjke8YAtOamePP7wu3R0sV916+4PwcAxCWR\nc9hmNszMXpR0WNIPnXPPV+1fZma9ZsbaUgW1aEX4/m8/6D1v3+2/f8sz3nPQfbVLrqxaI/zay+vX\nDQDaUaKXdZnZOEkPSfqic+6nAWly3fsuwOUIkupfYz3jCmnfgcptpWOChqzr3dErbH9Q3lGuBeey\nrnyh/bKvAG2Y/mVdzrl+SdskfTrJctD+fnxP7bYFy8OP6QpZalSSxn8ifP+K1eH7ASBLkpgl3j3Q\ns5aZnSVpvqR/jbsctJeJnwzfP2VS7bbH6ywLeqzOzTz6T4TvX9fA/a3D1iMHgDR1JJDn+yXda2bD\n5P0geMA592gC5aCNvPnrxo5Lasb4VTc3dlyzd/wCgKTEHrCdc3sk/V7c+QJD8f1tadcAAOLFSmdo\nmcld6ZY/+7x0yweAZnDzj4Sl/f0mrXqGar1Z2I0OgX/sQ17A33dA+sX+xvJotG5Fa8O8of2yrwBt\nGGmWeBLnsIFAYZdiLZzT3P2yL7tB2vpccLkAkGUEbMRq5Vpp9Y3hafq3SePmea8PbZUmVQ2VX3er\ndO8QpinOmSntWC89cffgtn0HvGu/JelghLXJvxjzimkAEDeGxBOW9vebNL/huKiLk5TSbd4qLV0V\nnn4ovvt1aellteXUq0+QIrZhntB+2VeANow0JE7ATlja32/S/P6zmDhOOvJkhGMjns9ePFe6frE0\nb5Z07IT0kz3SbRukn+2tf2yUYD3h0vDLuYrYhnlC+2VfAdqQc9hIR19/48duWeMF6CDjx0gzpkhX\nL6jcvuNF6ZLPN1Ym114DyAJ62AlL+/tNWtiv+6hD0Z0d0rvP1W6PqrqcztnS6TPND4W/l3+B2zAP\naL/sK0Ab0sNGuqKePy4F60Yv+So/7swL0qnno+XV6vtyA0AzWDgFiVpyS/001hMcPG9dJh172gv8\npcfJnd52P8MuihaI//jL9dOjhkQyAAAgAElEQVQAQDthSDxhaX+/SYsyHBfUy64OrFfOkx66q/G6\nLF3lzThvpOwwtGG20X7ZV4A2ZJZ4O0j7+01a1P8s3t4hjRpZdWyP1PeUNGFs5fbRc6W3TkavQ9cY\n6c0fVW77xkbplrtrA/aSW6T7fxg9b4k2zDraL/sK0Iacw0b7OPvj3nN1AO0YJk2/Qnr1QON5Hz1e\n2WP+5aO1PW2Jc9YAso1z2Gip8qDpeqWHtzcXrP2cu8i7brv8xwHBGkDWMSSesLS/36Q1Ohw3frR0\n9OmYK+Oje35z14VLtGHW0X7ZV4A2jDQkTg8bqTh2wuv1rlidTP7L7xw4R95ksAaAdkEPO2Fpf79J\ni/PXfRx31Epi6Js2zDbaL/sK0Ib0sJEtpeuxrWfwbl7lVq6t3XbOZZXHAUBe0cNOWNrfb9L4dZ99\neW9D2i/7CtCG9LABAMgLAjYAABlAwAYAIANSX+ls1qxZ6u2NYXpwm8r7+aW8n1uSaMOso/2yL+9t\nGBU9bAAAMiD1HjYAZEW7rhWAYqCHDQAhbr5m8F7scSjlddPV8eSH4iBgA4CPrjFeYL3zS8nkv/pG\nL/9JXcnkj/xhSBwAqsTVm47i0MCtYBkqRz30sAGgTCuDdTuUi+wgYAOApN88m37QdL3Sn34q3Tqg\nfRGwARSe65VGDG8+nxvuaD6Pzben/8MB7Ylz2AAK7Z2dzedRfv75rx/wnpsNur95Vhr5h83lgXyh\nhw2g0EaOqJ+me7503w/89wVNFmt2ElkcPX7kCwEbQGHV6wWX7rPe1y999i+bD8Ll9263Hum8P2mu\nfigWAjaAQqoXDL91v//2RoO233Ev761/HEEbJQRsAIXTHWGxkuV3Jl8PKdoPgAljk68H2h8BG0Dh\nHN4aX15BPeA4e8Z9T8WXF7KLWeIACuXPrhl87de7LQVa1xt9+Nv1SidOSmPmSsefkUaPil6fDV+J\nVp8VS6VvboqeL/KHHjaAQrljYG3woGC8//Dg6zkza/cH9ZxLQTooWAcdd91i7/lXB/33l+q5dqX/\nfhQHARsAykxbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuGVk8UDwEbQGE0e1759cPB+155zXs+\nejw4Tdi+KJgxXmwEbAAos3BO8L6pC4P3RRHW+150SXN5I/8I2AAK6WTAkqSPrWttPUoeWeu//Z1n\nW1sPtC8CNoBCmDyh8v1ZI7wh5rPKliaNMuS88ZHGyn94e/005eWPGum9H1m1ROnEcY2Vj+wjYAMo\nhINP+G8/uVM69bz3OsplXNd/tXbb6TOV7/v6a9NcGWGWd6n8/m3S2zv80xx5sn4+yCcCNoDC6xjW\n3PHDL6583z2/ufzGvq+545FPBGwAKBOll71kVeV758LTf+5r8ZSLYkskYJvZMDP7ZzN7NIn8ASBN\n9w9xadMNW5KpB4olqR72lyT9PKG8AWDIbloTPW2re7tDKW8onwP5EnvANrOpki6XdE/ceQNAo9bc\nFG9+X7g9Wrq47/oV9+dAdiTRw/6mpC9L+u9BCcxsmZn1mlnvkSNHEqgCADRn0Yrw/d9+0Hvevtt/\n/5ZnvOeg+2qXVM8ev/by+nVDMcUasM1skaTDzrldYemcc99xzvU453q6u7vjrAIANGT6ByrfPxZw\nWVW1ecv8t38mYk+4+vrse30uGwOk+HvYcyRdYWavStos6VIz+7uYywCA2P3Y5yTeguXhx3SFLDUq\nSeM/Eb5/xerw/UC5WAO2c+4W59xU59wHJS2R9CPn3GfjLAMAGjHxk+H7p0yq3fZ4nWVBj9W5mUf/\nifD96xq4v3XYeuTIN67DBlAIb/66seOSmjF+1c2NHdfsHb+QXR1JZeyc2yZpW1L5A0CWfX9b2jVA\n1tDDBoABk7vSLX/2eemWj/ZGwAZQGPWGtw8OcQWzch/7kDT/Iul3pjaex3Mbw/ezfGmxJTYkDgBZ\n5HqDA+PCOc3dL/uyG6StzwWXC4QhYAMolJVrpdU3hqfp3yaNm+e9PrRVmlQ1VH7drdK9Q7hTwpyZ\n0o710hN3D27bd0CacYX3OkrP/osxr5iG7DFX7zYzCevp6XG9vfn9aWlmaVchUWn//bQCbZhtfu0X\npTdrPYPpNm+Vlq4KTz8U3/26tPSy2nLq1cdP3ttPyv+/QUm7nHN1T3gQsBOW9z+0tP9+WoE2zDa/\n9ps4TjryZIRjI54zXjxXun6xNG+WdOyE9JM90m0bpJ/trX9slGA94dLgy7ny3n5S/v8NKmLAZkgc\nQOH09Td+7JY1XoAOMn6MNGOKdPWCyu07XpQu+XxjZXLtNSQCNoCCijIUXZqA1tkhvVs1WWwoM7Zd\nr/TxCwbL65wtnT7T3FA4ioeADaCwop4/LgXrRoNn+XFnXpBOPR8tL4I1ynEdNoBCW3JL/TTWExw8\nb10mHXvaC/ylx8md3nY/wy6KFoj/+Mv106BYmHSWsLxPlkj776cVaMNsi9J+Qb3s6sB65Tzpobsa\nr8vSVd6M80bKDpL39pPy/29QTDoDgGisR3p7hzRqZO2+vqekCWMrt42eK711Mnr+XWOkN38kbbrN\ne0jSNzZKt9xdm3bJLdL9P4yeN4qDgA0Aks7+uPdc3ePtGCZNv0J69UDjeR89Xtlj/uWjtT1tiXPW\nCMc5bAAoUx40Xa/08PbmgrWfcxd5122X/zggWKMeetgAUMV6pPGjpaNPS9de7j2S0j2/uevCURz0\nsAHAx7ETXuBesTqZ/Jff6eVPsEZU9LABIMS6Td5DiueOWgx9o1H0sAEgotL12NYzeDevcivX1m47\n57LK44BG0cMGgAb8+i3/ALzmvtbXBcVADxsAgAwgYAMAkAEEbAAAMiD1tcTNLNcL4ab9/SatAGv8\n0oYZR/tlXwHaMNJa4vSwAQDIAGaJAwCKY1cMIxKz0unx08MGAOTboTu9QB1HsJYG8zqU0DJ4ATiH\nnbC0v9+kcf4s+/LehrRf9jXchqfelPZMjLcyfs4/KHVObvjwqOewGRIHAORPXL3pKPac4z0nPFTO\nkDgAIF9aGaxbWC4BGwCQD7tHpBesS3aZdHRzIlkTsAEA2bfLJPdu09nccEcMddm3NJEfDkw6S1ja\n32/SmPCSfXlvQ9ov++q24e6RkvttU2X43cil6dup2nDpwvr1YuEUAEAxRAjW3fOl+37gvy/otqdN\n3w41hh5/OXrYCUv7+00av+6zL+9tSPtlX2gb1hl6jtJzDgvM9dJ+dIb00wdCq1B39jg9bABAvtUJ\n1t+63397oz1nv+Ne3hvhwJjOZxOwAQDZc/pw3STL72xBPRTxB8DpvqbLIWADALLnpcZXFqsWNLms\n6Uln5V7qbjoLVjoDAGTLG4PXXoWdo3a90Ye/Xa904qQ0Zq50/Blp9Kjo1dnwlcHXoefMD66Vzrkx\nesZV6GEDALLlwJ9LCg7G+8tGy+fMrN0f1HMuBemgYB103HWLvedfHfTf/149X7/JP0FEBGwAQK5M\nWzj4esf6ykAbNsz94au85wmXBqepzqv8/bmLhlbPoSJgAwCyo8kZ16+HzFV75TXv+ejx4DRh+yJp\nov4EbABAriycE7xv6sLgfVGE9b4XXdJc3vUQsAEAmXRyp//2x9a1th4lj6z13/7Os/HkT8AGAGTD\nqcpZXWeN8M4hnzVicFuUS7E2PtJY8Q9vr5+mvPxRI733I4dXJTp1pKHyWZo0YWl/v0kr/LKIOZD3\nNqT9su+9Ngw5/3v6jNQ5eyC9T9CunlFenab8eEk68qQ0cdzQ8ihP079NGvu+wOpWLFfK0qQAgMLo\nGNbc8cMvrnzfPb+5/EKDdYMI2ACAXImyWMqSVZXv6w3EfO5r8ZTbjEQCtpm9amb/YmYvmlmci7sB\nANC0+7cOLf2GLcnUYyiS7GF/wjl3QZRxeQAA6rlpTfS0Sfd2mylvKJ+jHEPiAIBMWNPcyp41vnB7\ntHRx3/Wr0c+RVMB2kraa2S4zW1a908yWmVkvw+UAgKQsWhG+/9sPes/bd/vv3/KM9xx0X+2SK1dW\nvr/28vp1a0Qil3WZ2QeccwfMbJKkH0r6onPumYC0ub7mgktKso82zDbaL/uiXNYlSTOukPYdqDp2\noFsYNGRd745eYfuD8o50W852uazLOXdg4PmwpIckXZREOQAAlPz4ntptC5aHH9MVstSoJI3/RPj+\nFavD98cp9oBtZmeb2ejSa0l/JOmncZcDACiYmeErhE2ZVLvt8TrLgh6rczOP/hPh+9dtCt/v6/y+\nBg6SOho6KtxkSQ8NDNN0SPquc+7xBMoBABRJx8SGDktqxvhVNzd4YOeEhg6LPWA75/ZK8rllOAAA\n+fH9ba0tj8u6AAC5Mbkr3fJnn5dc3tz8I2Fpf79JK9QM1ZzKexvSftlX04Z1Zos3OgT+sQ95AX/f\nAekX+xvLo+4M8Vm1f49RZ4kncQ4bAIDUhF2KtXBOc/fLvuwGaetzweUmiYANAMiWqXdJ+8NnfPVv\nk8bN814f2ipNqhoqv+5W6d5Hoxc5Z6a0Y730xN2D2/Yd8K79lqSDUdYmn/ZX0Qv0wZB4wtL+fpNW\nyOG4nMl7G9J+2efbhnWGxSWvl13q9W7eKi1dFZ5+KL77dWnpZbXlhPIZDpeiD4kTsBOW9vebtML+\nZ5EjeW9D2i/7fNvw1BFpj8+F11Wins9ePFe6frE0b5Z07IT0kz3SbRukn+2NUL8owfr8vsDLuTiH\nDQDIr87uhg/dssYL0EHGj5FmTJGuXlC5fceL0iWfb7DQBq+9LkcPO2Fpf79JK+yv+xzJexvSftkX\n2oYRh8Y7O6R3n6vdHrkOVb3oztnS6TPNDYW/Vw962ACA3JvlIgXtUrBu9JKv8uPOvCCdej5iXnWC\n9VCwcAoAINum11/Q23qCA+yty6RjT3u95dLj5E5vu59hF0UM1tO/FyFRdAyJJyzt7zdphR+Oy4G8\ntyHtl32R2jCgl10dWK+cJz10V+N1WbrKm3FeLnBYPGLvmlnibSLt7zdp/GeRfXlvQ9ov+yK34e5R\nknunYpP1SH1PSRPGViYdPVd662T0OnSNkd78UeW2b2yUbrnbJ2BP3yR1LYmcN+ewAQDFcuFABK7q\nbXcMk6ZfIb16oPGsjx6v7K3/8tHanrakWM9ZV+McNgAgX8qCpuuVHt7eXLD2c+4i77rtit51gsFa\nYkg8cWl/v0ljOC778t6GtF/2NdyGp45Ke5q//rmu8w83dV141CFxetgAgHzq7PJ6vdPWJpP/tHVe\n/k0E66Ggh52wtL/fpPHrPvvy3oa0X/bF2oYRrtmuK+ahb3rYAABUm+UGHzOP1exe6dcZP/+NyuNS\nQg87YWl/v0nj13325b0Nab/sK0Ab0sMGACAvCNgAAGQAARsAgAxIfaWzWbNmqbc3yv3Jsinv55fy\nfm5Jog2zjvbLvry3YVT0sAEAyAACNgAAGZD6kDiAHGnDRSmAvKCHDaA5h+70AnUcwVoazOvQ6njy\nA3KCgA2gMafe9ALr/i8nk//+m738Tx1KJn8gYxgSBzB0cfWmo9hzjvfMUDkKjh42gKFpZbBuh3KB\nNkHABhDN7hHpB81dJh3dnG4dgJQQsAHUt8sk927T2dxwRwx12bc0/R8OQAo4hw0g3O6RTWdhZfch\n+usHvGfX7AKHu0dIF/62yUyA7KCHDSCcqx8Uu+dL9/3Af58F3DQwaHtkMfT4gSwhYAMIVmfo2Xq8\nR1+/9Nm/bD4Il/IrPc77k+bqB+QJARuAvzrB8Fv3+29vNGj7Hffy3ggHErRREARsALVOH66bZPmd\nLaiHIv4AON2XeD2AtBGwAdR6aXJsWQVNLmt60lm5l7pjzAxoT8wSB1DpjcFrr/x6t6VA63qjD3+7\nXunESWnMXOn4M9LoUdGrs+Erg6/D6qODa6VzboyeMZAx9LABVDrw55KCg/H+stHyOTNr9wf1nEtB\nOihYBx133WLv+VcH/fe/V8/Xb/JPAOQEARvAkExbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuG\nVk8gbwjYAAY1OeP69ZC5aq+85j0fPR6cJmxfJMwYR44RsAEMycI5wfumLgzeF0VY73vRJc3lDWQd\nARuAr5M7/bc/tq619Sh5ZK3/9neebW09gLQQsAF4TlXO6jprhHcO+awRg9uiXIq18ZHGin94e/00\n5eWPGum9Hzm8KtGpI41VAGhzBGwAnj3v9918cqd06nnvdZTLuK7/au2202cq3/f116a5cmX9vEvl\n92+T3t4RkGjPpPoZARlEwAZQV8ew5o4ffnHl++75zeU39n3NHQ9kUSIB28zGmdnfm9m/mtnPzewP\nkigHQOtF6WUvWVX53rnw9J/7WjzlAnmWVA97naTHnXP/o6SZkn6eUDkA2tD9W4eWfsOWZOoB5Ens\nAdvMxkiaK2m9JDnn3nXO+ZyxAtBObloTPW2re7tDKW8onwPIkiR62DMkHZG0wcz+2czuMbOzEygH\nQIzWxLyy5xduj5Yu7rt+xf05gHaRRMDukHShpL9xzv2epLcl/UV5AjNbZma9ZtZ75AiXYABZtGhF\n+P5vP+g9b9/tv3/LM95z0H21S6pnj197ef26AXmURMDeL2m/c27gQhD9vbwA/h7n3Heccz3OuZ7u\nbm6LB2TB9A9Uvn8s6LKqKvOW+W//TMSecPX12ff6XDYGFEHsAds5d1DSa2b2kYFNn5T0s7jLAdBa\nP76ndtuC5eHHdIUsNSpJ4z8Rvn/F6vD9QJEkdT/sL0q6z8yGS9or6fqEygEQl5lHpJeCR7ym+KxH\n8nidZUGP1bmZR/+J8P3rNoXv93V+XwMHAe0vkYDtnHtREldNAlnSMbGhw5KaMX7VzQ0e2Dkh1noA\n7YKVzgC0pe9vS7sGQHshYAOIbHJXuuXPPi/d8oE0EbABDJoVvobowSGuYFbuYx+S5l8k/c7UxvN4\nbmOdBHXqD2RZUpPOAOSU6w0+b71wTnP3y77sBmnrc8HlAkVGwAZQaepd0v7wGV/926Rx87zXh7ZK\nk6qGyq+7Vbr30ehFzpkp7VgvPXH34LZ9B6QZV3ivI/Xsp/1V9AKBDGJIHEClyfVvTF26vaXr9YL1\n5q1er7v0GEqwlqSdL1Uev+kJb6GWUq860rnzSV8cWqFAxpird9+7hPX09Lje3vyOdZlZ2lVIVNp/\nP61QyDY8dUTa43PhdZWol3Qtnitdv1iaN0s6dkL6yR7ptg3Sz/ZGqF+U/x7O7wu8nKuQ7ZczeW9D\nSbucc3X/NTEkDqBWZ+NLBm9Z4wXoIOPHSDOmSFcvqNy+40Xpks83WCjXXqMACNgA/M1y0q7wnk1p\nAlpnh/Ru1WSxoSyo4nqlj18w2JvunC2dPhOxd83McBQEARtAsAhBWxoM1o2uelZ+3JkXpFPPR8yL\nYI0CYdIZgHDT6y/oXZos5ufWZdKxp73eculxcqe33c+wiyIG6+nfi5AIyA8mnSUs75Ml0v77aQXa\nUIG97OrAeuU86aG7Gq/L0lXejPNygcPiEXvXtF/25b0NxaQzALGZ5aTdoyT3Ts2uvqekCWMrt42e\nK711Mnr2XWOkN38kbbrNe0jSNzZKt9ztk3j6JqlrSfTMgZwgYAOI5sKBCFzV2+4YJk2/Qnr1QONZ\nHz1e2Vv/5aO1PW1JnLNGoXEOG8DQlAVN1ys9vL25YO3n3EXeddsVw+EEaxQcPWwAQzfLSaeOSnsm\n6NrLpWsvT7Cs8w83dV04kBf0sAE0prPLC9zT1iaT/7R1Xv4Ea0ASPWwAzZq0wntIka7Zrouhb8AX\nPWwA8ZnlBh8zj9XsXunXGT//jcrjAPiihw0gGR3jagLw6r9LqS5ADtDDBgAgAwjYAABkAAEbAIAM\nSH0tcTPL9SyTtL/fpBVgjV/aMONov+wrQBtGWkucHjYAABmQm1nikW50X0ej9/IFACBpme5h33zN\n4P1141DK66ar48kPAIC4ZPIcdulWfEmb/EfS4aPN5ZH295s0zp9lX97bkPbLvgK0YT7vhx1XbzqK\nQwO392OoHACQtkwNibcyWLdDuQAAlGQiYP/m2fSDpuuV/vRT6dYBAFBcbR+wXa80Ynjz+dxwR/N5\nbL49/R8OAIBiautJZ+/slEaOaDJ/n/PPzQbd374rjfzDaGnT/n6TxoSX7Mt7G9J+2VeANsz+wilR\ngnX3fOm+H/jvC5os1uwksjh6/AAADEXb9rDr9YKj9JzDAnO9tB+dIf30gaHXoaac/P8yTLsKiaMN\ns432y74CtGF2e9j1gvW37vff3mjP2e+4l/fWP47z2QCAVmm7gN3dVT/N8juTr4cU7QfAhLHJ1wMA\ngLYL2Ie3xpdXUA84zp5x31Px5QUAQJC2Wunsz64ZfB12jtr1Rh/+dr3SiZPSmLnS8Wek0aOi12fD\nV6LVZ8VS6ZuboucLAMBQtVUP+44vec9BwXj/4cHXc2bW7g/qOZeCdFCwDjruusXe868O+u8v1XPt\nSv/9AADEpa0Cdj3TFg6+3rG+MtCGDXN/+CrvecKlwWmq8yp/f+6iodUTAIC4tU3Abva88uuHg/e9\n8pr3fPR4cJqwfVEwYxwAkKS2CdhRLJwTvG/qwuB9UYT1vhdd0lzeAAA0qy0D9smd/tsfW9faepQ8\nstZ/+zvPtrYeAIDiaouAPXlC5fuzRnhDzGeVLU0aZch54yONlf/w9vppyssfNdJ7P7JqidKJ4xor\nHwCAetpiadKwYHz6jNQ523vtl656Rnl1mvLjJenIk7WBtV4e5Wn6t0lj3xdc35q88r+kXtpVSBxt\nmG20X/YVoA2zuzRpuY5hzR0//OLK993zm8svLFgDAJCUtg/Y5aIslrJkVeX7ej/MPve1eMoFACBJ\nsQdsM/uImb1Y9jhuZiviLifI/UNc2nTDlmTqAQBAnGIP2M65f3POXeCcu0DSLEknJT0UdsxNa6Ln\n3+re7lDKG8rnAABgKJIeEv+kpF84534ZlmjNTfEW+oXbo6WL+65fcX8OAABKkg7YSyTV3BbDzJaZ\nWa+ZNbQ+2KI6A+zfftB73r7bf/+WZ7znoPtql1xZtUb4tZfXrxsAAElI7LIuMxsu6YCkjzrnDoWk\nC72sS5JmXCHtO1C5rXRM0JB1vTt6he0PyjvKteBc1pU/tGG20X7ZV4A2TP2yrgWSdocF66h+fI9P\n5svDj+kKWWpUksZ/Inz/itXh+wEAaKUkA/ZS+QyH+5n4yfD9UybVbnu8zrKgx+rczKP/RPj+dQ3c\n3zpsPXIAAJqRSMA2s1GSPiXpH6Kkf/PXDZaT0Izxq25u7Lhm7/gFAECQjiQydc6dlDShbsI29f1t\nadcAAIBKmVnpbHJXuuXPPi/d8gEAxdYWN/8ova43C7vRIfCPfcgL+PsOSL/Y31gejdYt7e83acxQ\nzb68tyHtl30FaMNIs8QTGRJPStilWAvnNHe/7MtukLY+F1wuAABpaquAvXKttPrG8DT926Rx87zX\nh7ZKk6qGyq+7Vbr30ehlzpkp7VgvPXH34LZ9B7xrvyXpYIS1yb8Y84ppAABUa6shcSn64iSldJu3\nSktXhacfiu9+XVp6WW059eoTJO3vN2kMx2Vf3tuQ9su+ArRhpCHxtgvYE8dJR56McFzE89mL50rX\nL5bmzZKOnZB+ske6bYP0s731j40SrCdcGn45V9rfb9L4zyL78t6GtF/2FaANs3kOu6+/8WO3rPEC\ndJDxY6QZU6SrF1Ru3/GidMnnGyuTa68BAK3Qdj3skqhD0Z0d0rvP1W6PqrqcztnS6TPND4W/l3/+\nfxmmXYXE0YbZRvtlXwHaMJs97JKo549LwbrRS77KjzvzgnTq+Wh5tfq+3ACAYmvrhVOW3FI/jfUE\nB89bl0nHnvYCf+lxcqe33c+wi6IF4j/+cv00AADEqW2HxEuCetnVgfXKedJDdzVej6WrvBnnjZQd\nJu3vN2kMx2Vf3tuQ9su+ArRhNmeJ+3l7hzRqZNVxPVLfU9KEsZXbR8+V3joZvfyuMdKbP6rc9o2N\n0i131wbsJbdI9/8wet5SIf7Q0q5C4mjDbKP9sq8AbZjtc9jlzv6491wdQDuGSdOvkF490HjeR49X\n9ph/+WhtT1vinDUAIF1tfQ67WnnQdL3Sw9ubC9Z+zl3kXbdd/uOAYA0ASFsmhsSrjR8tHX06idpU\n6p7f3HXhUiGGctKuQuJow2yj/bKvAG0YaUg8Uz3skmMnvF7vitXJ5L/8zoFz5E0GawAA4pLJHraf\nOO6olcTQd9rfb9L4dZ99eW9D2i/7CtCG+e1h+yldj209g3fzKrdybe22cy6rPA4AgHaVmx52u0r7\n+00av+6zL+9tSPtlXwHasFg9bAAA8oyADQBABhCwAQDIgHZY6axP0i9bWN7EgTJbIqXzSy39jCnI\nexvSfjGi/WLX8s9XgDY8N0qi1CedtZqZ9UY5uZ9lef+MfL5s4/NlW94/n9S+n5EhcQAAMoCADQBA\nBhQxYH8n7Qq0QN4/I58v2/h82Zb3zye16Wcs3DlsAACyqIg9bAAAMoeADQBABhQqYJvZp83s38zs\nFTP7i7TrEycz+1szO2xmP027Lkkws2lm9rSZ/dzMXjazL6Vdp7iZ2Ugze8HMXhr4jF9Nu05xM7Nh\nZvbPZvZo2nVJgpm9amb/YmYvmlkM9xBsL2Y2zsz+3sz+deDf4h+kXae4mNlHBtqt9DhuZivSrle5\nwpzDNrNhkv4/SZ+StF/SP0la6pz7WaoVi4mZzZX0lqT/6pw7L+36xM3M3i/p/c653WY2WtIuSVfm\npf0kybzVIc52zr1lZp2Sdkj6knPuuZSrFhszu0lSj6QxzrlFadcnbmb2qqQe51wuF04xs3sl/dg5\nd4+ZDZc0yjnXn3a94jYQL16XNNs518qFvUIVqYd9kaRXnHN7nXPvStos6TMp1yk2zrlnJB1Nux5J\ncc694ZzbPfD6hKSfS5qSbq3i5TxvDbztHHjk5he1mU2VdLmke9KuC4bOzMZImitpvSQ5597NY7Ae\n8ElJv2inYC0VK2BPkRolYLIAAAIzSURBVPRa2fv9ytl/+EVhZh+U9HuSnk+3JvEbGDJ+UdJhST90\nzuXpM35T0pcl/fe0K5IgJ2mrme0ys2VpVyZmMyQdkbRh4LTGPWZ2dtqVSsgSSZvSrkS1IgVsv8Vo\nc9N7KQoze5+kByWtcM4dT7s+cXPOnXHOXSBpqqSLzCwXpzfMbJGkw865XWnXJWFznHMXSlog6T8O\nnKrKiw5JF0r6G+fc70l6W1Ku5gJJ0sBQ/xWSvpd2XaoVKWDvlzSt7P1USQdSqgsaMHBe90FJ9znn\n/iHt+iRpYKhxm6RPp1yVuMyRdMXAOd7Nki41s79Lt0rxc84dGHg+LOkheafi8mK/pP1loz5/Ly+A\n580CSbudc4fSrki1IgXsf5L0YTObPvALaomkLSnXCRENTMhaL+nnzrk1adcnCWbWbWbjBl6fJWm+\npH9Nt1bxcM7d4pyb6pz7oLx/ez9yzn025WrFyszOHpgQqYGh4j+SlJurNpxzByW9ZmYfGdj0SUm5\nmfRZZqnacDhcao/ba7aEc+60md0g6QlJwyT9rXPu5ZSrFRsz2yRpnqSJZrZf0lecc+vTrVWs5ki6\nRtK/DJzjlaRVzrl/TLFOcXu/pHsHZqj+O0kPOOdyeflTTk2W9NDArSA7JH3XOfd4ulWK3Rcl3TfQ\n6dkr6fqU6xMrMxsl70qi/5B2XfwU5rIuAACyrEhD4gAAZBYBGwCADCBgAwCQAQRsAAAygIANAEAG\nELABAMgAAjYAABnw/wPRIOc/pYUmbAAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAHwCAYAAABkPlyAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+4FdWd7/nP93IOIIZfBw6YAGOgkyczHSO2nBa7iQwxpA0IRmd6umGMXs1kuJO5hqDY6Zbn6Scmz41mVCB07OncXGnw3jagaduI2lGiEQwYtQ+00jHpnseAiYj8OMIJKCYCd80fdbZn/6iqXWfvql27qt6v59nP3rtq1Vpr73Xgu9eqVavMOScAANDe/l3aFQAAAPURsAEAyAACNgAAGUDABgAgAwjYAABkAAEbAIAMIGADAJABBGwAADKAgA20GTP7oJn9o5kdM7ODZna3mXWEpB9nZn8zkPakmf2Lmf37VtYZQPII2ED7+X8lHZb0fkkXSPqfJf3ffgnNbLikJyWdK+kPJI2V9GeS7jCz5S2pLYCWIGAD7We6pAecc79xzh2U9LikjwakvUbS/yDpf3PO7XPOnXLOPS5puaT/ZGajJcnMnJl9qHSQmW00s/9U9n6Rmb1oZv1m9qyZnV+27wNm9qCZHTGzfeU/BMzsVjN7wMz+q5mdMLOXzaynbP+fm9nrA/v+zcw+Gc9XBBQPARtoP+skLTGzUWY2RdICeUHbz6ck/cA593bV9gcljZJ0cb3CzOxCSX8r6T9ImiDpP0vaYmYjzOzfSXpE0kuSpkj6pKQVZnZZWRZXSNosaZykLZLuHsj3I5JukPT7zrnRki6T9Gq9+gDwR8AG2s92eT3q45L2S+qV9P2AtBMlvVG90Tl3WlKfpO4I5f2fkv6zc+5559wZ59y9kn4rL9j/vqRu59zXnHPvOuf2SvovkpaUHb/DOfePzrkzkv6bpJkD289IGiHpd82s0zn3qnPuFxHqA8AHARtoIwM92ick/YOks+UF5PGS/p+AQ/rkneuuzqdj4NgjEYo9V9LKgeHwfjPrlzRN0gcG9n2gat8qSZPLjj9Y9vqkpJFm1uGce0XSCkm3SjpsZpvN7AMR6gPABwEbaC9d8oLl3c653zrn3pS0QdLCgPRPSlpgZmdXbf9fJZ2S9MLA+5PyhshLzil7/ZqkrzvnxpU9RjnnNg3s21e1b7RzLqg+FZxz33XOfVxe4HcK/uEBoA4CNtBGnHN9kvZJ+oKZdZjZOEn/Xt45ZD//Td6w+fcGLgfrHDi//FeS7nDO/Xog3YuS/nczG2Zmn5Y387zkv0j6v8xstnnONrPLByasvSDp+MDksbMGjj/PzH6/3mcxs4+Y2aVmNkLSbyS9I2+YHEADCNhA+/lfJH1a3nD2K5JOS7rRL6Fz7reS5svrCT8vLyg+Lumbkr5alvRLkhZL6pd0tcrOiTvneuWdx75b0rGBMq8b2Hdm4LgL5P2Q6JN0j7zLx+oZIekbA8cclDRJ3nA6gAaYcy7tOgCIiZl1SvqBpNclXef4Bw7kBj1sIEecc6fknb/+haSPpFwdADGihw0AQAbQwwYAIAMCbyjQKhMnTnQf/OAH065GYnbt2pV2FRI1a9astKuQONow22i/7Mt7G0rqc87VXeQo9SHxnp4e19vbm2odkmRmaVchUWn//bRCXG3oYvgzH1ylOz55b0P+DWZf3ttQ0i7nXN1/3QyJAwm6+RovUMcRrKXBvG66Op78AGQHARtIQNcYL7De+aVk8l99o5f/pK5k8gfQflI/hw3kTVy96SgObfWekxgqB9Be6GEDMWplsG6HcgG0DgEbiMFvnk0/aLpe6U8/lW4dACSHgA00yfVKI4Y3n88NdzSfx+bb0//hACAZnMMGmvDOzubzKD///NcPeM/NBt3fPCuN/MPm8gDQXuhhA00YOaJ+mu750n0/8N8XNFms2UlkcfT4AbQXAjbQoHq9YOvxHn390mf/svkgXMqv9DjvT5qrH4BsIWADDagXDL91v//2RoO233Ev761/HEEbyA8CNjBE3REWK1l+Z/L1kKL9AJgwNvl6AEgeARsYosNb48srqAccZ8+476n48gKQHmaJA0PwZ9cMvvbr3ZYCreuNPvzteqUTJ6Uxc6Xjz0ijR0Wvz4avRKvPiqXSNzdFzxdA+6GHDQzBHQNrgwcF4/2HB1/PmVm7P6jnXArSQcE66LjrFnvPvzrov79Uz7Ur/fcDyA4CNhCjaQsHX+9YXxlow4a5P3yV9zzh0uA01XmVvz930dDqCSB7CNhARM2eV379cPC+V17zno8eD04Tti8KZowD2UbABmK0cE7wvqkLg/dFEdb7XnRJc3kDaH8EbKABJwOWJH1sXWvrUfLIWv/t7zzb2noASA4BG4hg8oTK92eN8IaYzypbmjTKkPPGRxor/+Ht9dOUlz9qpPd+ZNUSpRPHNVY+gPQRsIEIDj7hv/3kTunU897rKJdxXf/V2m2nz1S+7+uvTXNlhFnepfL7t0lv7/BPc+TJ+vkAaE8EbKBJHcOaO374xZXvu+c3l9/Y9zV3PID2RMAGYhSll71kVeV758LTf+5r8ZQLINsI2ECL3T/EpU03bEmmHgCyJZGAbWafNrN/M7NXzOwvkigDaKWb1kRP2+re7lDKG8rnANBeYg/YZjZM0l9LWiDpdyUtNbPfjbscoJXW3BRvfl+4PVq6uO/6FffnANA6SfSwL5L0inNur3PuXUmbJX0mgXKAtrVoRfj+bz/oPW/f7b9/yzPec9B9tUuqZ49fe3n9ugHIpiQC9hRJr5W93z+w7T1mtszMes2s98iRIwlUAWit6R+ofP9YwGVV1eYt89/+mYg94errs+/1uWwMQD4kEbDNZ1vFPFjn3Heccz3OuZ7u7u4EqgC01o/vqd22YHn4MV0hS41K0vhPhO9fsTp8P4B8SSJg75c0rez9VEkHEigHaJmJnwzfP2VS7bbH6ywLeqzOzTz6T4TvX9fA/a3D1iMH0N6SCNj/JOnDZjbdzIZLWiKJC1OQaW/+urHjkpoxftXNjR3X7B2/AKSnI+4MnXOnzewGSU9IGibpb51zL8ddDlBk39+Wdg0AtFrsAVuSnHP/KOkfk8gbaFeTu6RDR9Mrf/Z56ZUNIHmsdAZEVG94++AQVzAr97EPSfMvkn5nauN5PLcxfD/LlwLZlkgPGygq1xscGBfOae5+2ZfdIG19LrhcAPlGwAaGYOVaafWN4Wn6t0nj5nmvD22VJnVV7r/uVuneR6OXOWemtGO99MTdg9v2HZBmXOG9jtKz/2LMK6YBaD1z9W4VlLCenh7X25vf7oGZ32Xp+ZH2308rVLdhlN6s9Qym27xVWroqPP1QfPfr0tLLasupV58geW9D/g1mX97bUNIu51zdk1YE7ITl/Q8t7b+fVqhuw4njpCNPRjgu4jnjxXOl6xdL82ZJx05IP9kj3bZB+tne+sdGCdYTLg2/nCvvbci/wezLexsqYsBmSBwYor7+xo/dssYL0EHGj5FmTJGuXlC5fceL0iWfb6xMrr0G8oGADTQgylB0aQJaZ4f0btVksaHM2Ha90scvGCyvc7Z0+kzzQ+EAsoWADTQo6vnjUrBuNHiWH3fmBenU89HyIlgD+cJ12EATltxSP431BAfPW5dJx572An/pcXKnt93PsIuiBeI//nL9NACyhUlnCcv7ZIm0/35aoV4bBvWyqwPrlfOkh+5qvB5LV3kzzhspO0ze25B/g9mX9zYUk86A1rAe6e0d0qiRtfv6npImjK3cNnqu9NbJ6Pl3jZHe/JG06TbvIUnf2Cjdcndt2iW3SPf/MHreALKDgA3E4OyPe8/VPd6OYdL0K6RXm7jB7NHjlT3mXz5a29OWOGcN5B3nsIEYlQdN1ys9vL25YO3n3EXeddvlPw4I1kD+0cMGYmY90vjR0tGnpWsv9x5J6Z7f3HXhALKDHjaQgGMnvMC9YnUy+S+/08ufYA0UBz1sIEHrNnkPKZ47ajH0DRQXPWygRUrXY1vP4N28yq1cW7vtnMsqjwNQXPSwgRT8+i3/ALzmvtbXBUA20MMGACADCNgAAGQAARsAgAwgYAMAkAGp3/zDzHK9cn3a32/SCrAoP22YcbRf9hWgDbn5R66dOSa92FWxaeVaafWNVenOPyB1vr919QIAJIIedsJi/X53xfBLela8Xze/7rMv721I+2VfAdowUg+bc9jt7tCdXqCOI1hLg3kdSmjNTABAIuhhJ6zh7/fUm9KeifFWxs/5B6XOyQ0fzq/77Mt7G9J+2VeANuQcdmbF1ZuOYs853nPMQ+UAgHgxJN5uWhms26FcAEAkBOx2sXtE+kFzl0lHN6dbBwCALwJ2O9hlknu36WxuuCOGuuxbmv4PBwBADSadJazu97t7pOR+21QZfnd9avreyzZcurB+vZjwkn15b0PaL/sK0IZc1pUJEYJ193zpvh/47wu6R3LT906OoccPAIgPPeyEhX6/dYaeo/ScwwJzvbQfnSH99IHQKtSdPc6v++zLexvSftlXgDakh93W6gTrb93vv73RnrPfcS/vjXAg57MBoC0QsNNw+nDdJMvvbEE9FPEHwOm+xOsBAAhHwE7DS42vLFYtaHJZ05POyr3UHWNmAIBGsNJZq70xeO1V2Dlq1xt9+Nv1SidOSmPmSsefkUaPil6dDV8ZfB16zvzgWumc6luBAQBahR52qx34c0nBwXh/2Wj5nJm1+4N6zqUgHRSsg467brH3/KuD/vvfq+frN/knAAC0BAG7zUxbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuGVk8AQGsRsFupyRnXr4fMVXvlNe/56PHgNGH7ImHGOACkhoDdZhbOCd43dWHwvijCet+LLmkubwBAsgjYKTm503/7Y+taW4+SR9b6b3/n2dbWAwDgj4DdKqcqZ3WdNcI7h3zWiMFtUS7F2vhIY8U/vL1+mvLyR4303o8cXpXo1JHGKgAAaApLkybsve835Pzv6TNS5+yB9D5Bu3pGeXWa8uMl6ciT0sRxQ8ujPE3/Nmns+wKrW7FcKcsiZl/e25D2y74CtCFLk2ZFx7Dmjh9+ceX77vnN5RcarAEAqSBgt5koi6UsWVX5vt6Pz899LZ5yAQDpiT1gm9nfmtlhM/tp3HnDc//WoaXfsCWZegAAWieJHvZGSZ9OIN9Mu2lN9LSt7u0OpbyhfA4AQHxiD9jOuWckHY0736xbE/PKnl+4PVq6uO/6FffnAABEwznsNrVoRfj+bz/oPW/f7b9/yzPec9B9tUuuXFn5/trL69cNANB6qQRsM1tmZr1mFudNIDNt+gcq3z+2I9px85b5b/9MxJ5w9fXZ93412nEAgNZKJWA7577jnOuJct1ZUfz4ntptC5aHH9MVstSoJI3/RPj+FavD9wMA2gdD4q0yM3yFsCmTarc9XmdZ0GN1bubRfyJ8/7pN4ft9nd/XwEEAgGYlcVnXJkk/kfQRM9tvZv9H3GVkUsfEhg5Lasb4VTc3eGDnhFjrAQCIpiPuDJ1zS+POE/H7/ra0awAAGAqGxNvI5K50y599XrrlAwCCcfOPhNV8vyE3AZEaHwL/2Ie8gL/vgPSL/Y3lUfduYbNqm4obD2Rf3tuQ9su+ArRhpJt/xD4kjua43uCgvXBOc/fLvuwGaetzweUCANoXAbvVpt4l7Q+f8dW/TRo3z3t9aKs0qWqo/LpbpXsfjV7knJnSjvXSE3cPbtt3QJpxhff6YJS1yaf9VfQCAQCxY0g8Yb7fb51hccnrZZd6vZu3SktXhacfiu9+XVp6WW05oXyGwyWG4/Ig721I+2VfAdow0pA4ATthvt/vqSPSHp8Lr6tEPZ+9eK50/WJp3izp2AnpJ3uk2zZIP9sboX5RgvX5fYGXc/GfRfblvQ1pv+wrQBtyDrttdXY3fOiWNV6ADjJ+jDRjinT1gsrtO16ULvl8g4Vy7TUApI4edsJCv9+IQ+OdHdK7z9Vuj1yHql5052zp9JnmhsLfqwe/7jMv721I+2VfAdqQHnbbm+UiBe1SsG70kq/y4868IJ16PmJedYI1AKB1WDglbdPrL+htPcEB9tZl0rGnvd5y6XFyp7fdz7CLIgbr6d+LkAgA0CoMiScs0vcb0MuuDqxXzpMeuqvxuixd5c04Lxc4LB6xd81wXPblvQ1pv+wrQBsyS7wdRP5+d4+S3DsVm6xH6ntKmjC2MunoudJbJ6PXoWuM9OaPKrd9Y6N0y90+AXv6JqlrSeS8+c8i+/LehrRf9hWgDTmHnSkXDkTgqt52xzBp+hXSqwcaz/ro8cre+i8fre1pS+KcNQC0Mc5ht5uyoOl6pYe3Nxes/Zy7yLtuu6J3TbAGgLbGkHjCGv5+Tx2V9rTg+ufzDzd1XTjDcdmX9zak/bKvAG0YaUicHna76uzyer3T1iaT/7R1Xv5NBGsAQOvQw05YrN9vhGu264p56Jtf99mX9zak/bKvAG1IDzt3ZrnBx8xjNbtX+nXGz3+j8jgAQCbRw05Y2t9v0vh1n315b0PaL/sK0Ib0sAEAyAsCNgAAGUDABgAgA1Jf6WzWrFnq7Y1yn8dsyvv5pbyfW5Jow6yj/bIv720YFT1sAAAyIPUeNgCgjbTheg/w0MMGgKI7dKcXqOMI1tJgXodWx5MfJBGwAaC4Tr3pBdb9X04m//03e/mfOpRM/gXDkDgAFFFcveko9pzjPTNU3hR62ABQNK0M1u1Qbk4QsAGgKHaPSD9o7jLp6OZ065BRBGwAKIJdJrl3m87mhjtiqMu+pen/cMggzmEDQN7tHtl0FlZ2a4q/fsB7ds2uebV7hHThb5vMpDjoYQNA3rn6QbF7vnTfD/z3WcB9pIK2RxZDj79ICNgAkGd1hp6tx3v09Uuf/cvmg3Apv9LjvD9prn4YRMAGgLyqEwy/db//9kaDtt9xL++NcCBBOxICNgDk0enDdZMsv7MF9VDEHwCn+xKvR9YRsAEgj16aHFtWQZPLmp50Vu6l7hgzyydmiQNA3rwxeO2VX++2FGhdb/Thb9crnTgpjZkrHX9GGj0qenU2fGXwdVh9dHCtdM6N0TMuGHrYAJA3B/5cUnAw3l82Wj5nZu3+oJ5zKUgHBeug465b7D3/6qD//vfq+fpN/gkgiYANAIUzbeHg6x3rKwNt2DD3h6/ynidcGpymOq/y9+cuGlo9UYmADQB50uSM69dD5qq98pr3fPR4cJqwfZEwYzwQARsACmbhnOB9UxcG74sirPe96JLm8i46AjYA5NTJnf7bH1vX2nqUPLLWf/s7z7a2HllFwAaAvDhVOavrrBHeOeSzRgxui3Ip1sZHGiv+4e3105SXP2qk937k8KpEp440VoGcI2ADQF7seb/v5pM7pVPPe6+jXMZ1/Vdrt50+U/m+r782zZUr6+ddKr9/m/T2joBEeybVz6iACNgAUAAdw5o7fvjFle+75zeX39j3NXd8ERGwAaBgovSyl6yqfO9cePrPfS2echGMgA0AqHH/1qGl37AlmXpgUOwB28ymmdnTZvZzM3vZzL4UdxkAgFo3rYmettW93aGUN5TPUSRJ9LBPS1rpnPufJF0s6T+a2e8mUA4AoMyamFf2/MLt0dLFfdevuD9HXsQesJ1zbzjndg+8PiHp55KmxF0OAKA5i1aE7//2g97z9t3++7c84z0H3Ve7pHr2+LWX168baiV6DtvMPijp9yQ9X7V9mZn1mlnvkSNcbwcArTD9A5XvHwu6rKrKvGX+2z8TsSdcfX32vT6XjaG+xAK2mb1P0oOSVjjnKlaXdc59xznX45zr6e7mHqgA0Ao/vqd224Ll4cd0hSw1KknjPxG+f8Xq8P2ILpGAbWad8oL1fc65f0iiDABAlZnhI5ZTfNYjebzOsqDH6tzMo/9E+P51m8L3+zq/r4GD8i+JWeImab2knzvnmOsHAK3SMbGhw5KaMX7VzQ0e2Dkh1nrkRRI97DmSrpF0qZm9OPBo8v4vAICs+f62tGuQLx1xZ+ic2yGJG5oCQBua3CUdOppe+bPPS6/srGOlMwDIk1nha4geHOIKZuU+9iFp/kXS70xtPI/nNtZJUKf+RRZ7DxsA0N5cb/B564Vzmrtf9mU3SFufCy4XjSNgA0DeTL1L2h8+46t/mzRunvf60FZpUlfl/utule59NHqRc2ZKO9ZLT9w9uG3fAWnGFd7rSD37aX8VvcACYkgcAPJmcv0bU5dub+l6vWC9eavX6y49hhKsJWnnS5XHb3rCW6il1Kue3BV+vCRp0heHVmjBmKt3z7SE9fT0uN7e/I6TeFe55Vfafz+tQBtmW2Hb79QRaY/PhddVol7StXiudP1iad4s6dgJ6Sd7pNs2SD/bG6GOUf6LP78v8HKuvLehpF3OubotwZA4AORRZ+OrSG5Z4wXoIOPHSDOmSFcvqNy+40Xpks83WCjXXtdFwAaAvJrlpF3hvdPSBLTODundqsliQ1lQxfVKH79gsDfdOVs6fSZi75qZ4ZEQsAEgzyIEbWkwWDe66ln5cWdekE49HzEvgnVkTDoDgLybXn9B79JkMT+3LpOOPe31lkuPkzu97X6GXRQxWE//XoREKGHSWcLyPlki7b+fVqANs432GxDQy64OrFfOkx66q/H6LF3lzTgvFzgsHrF3nfc2FJPOAADvmeWk3aMk907Nrr6npAljK7eNniu9dTJ69l1jpDd/JG26zXtI0jc2Srfc7ZN4+iapa0n0zCGJgA0AxXHhQASu6m13DJOmXyG9eqDxrI8er+yt//LR2p62JM5ZN4Fz2ABQNGVB0/VKD29vLlj7OXeRd912xXA4wbop9LABoIhmOenUUWnPBF17uXTt5QmWdf7hpq4Lh4ceNgAUVWeXF7inrU0m/2nrvPwJ1rGghw0ARTdphfeQIl2zXRdD34mghw0AGDTLDT5mHqvZvdKvM37+G5XHIRH0sAEA/jrG1QTg1X+XUl1ADxsAgCwgYAMAkAEEbAAAMoCADQBABqR+8w8zy/WUwrS/36QVYFF+2jDjaL/sK0AbRrr5Bz1stKVxoytv5ed6pZuurt12zoS0awoArUEPO2Fpf79Ji/PXfeAt+IYg0j14h4g2zDbaL/sK0Ib0sNH+br5msLcch/LeOADkCT3shKX9/Sat0V/3pXvnJm3yH0mHjzaXB22YbbRf9hWgDSP1sFnpDC0XV286ikMD9+NNYqgcAFqJIXG0VCuDdTuUCwBxIWCjJX7zbPpB0/VKf/qpdOsAAI0iYCNxrlcaMbz5fG64o/k8Nt+e/g8HAGgEk84Slvb3m7R6E17e2SmNHNFkGT7nn5sNur99Vxr5h9HSFr0Ns472y74CtCGXdSF9UYJ193zpvh/47wuaLNbsJLI4evwA0Er0sBOW9vebtLBf9/V6wVF6zmGBuV7aj86QfvrA0OtQU06B2zAPaL/sK0Ab0sNGeuoF62/d77+90Z6z33Ev761/HOezAWQFARux6+6qn2b5ncnXQ4r2A2DC2OTrAQDNImAjdoe3xpdXUA84zp5x31Px5QUASWGlM8Tqz64ZfB12jtr1Rh/+dr3SiZPSmLnS8Wek0aOi12fDV6LVZ8VS6ZuboucLAK1GDxuxuuNL3nNQMN5/ePD1nJm1+4N6zqUgHRSsg467brH3/KuD/vtL9Vy70n8/ALQLAjZaatrCwdc71lcG2rBh7g9f5T1PuDQ4TXVe5e/PXTS0egJAuyFgIzbNnld+/XDwvlde856PHg9OE7YvCmaMA2hnBGy01MI5wfumLgzeF0VY73vRJc3lDQBpI2AjESd3+m9/bF1r61HyyFr/7e8829p6AECjCNiIxeQJle/PGuENMZ9VtjRplCHnjY80Vv7D2+unKS9/1Ejv/ciqJUonjmusfABIGkuTJizt7zdppWURw4Lx6TNS52wFpqueUV6dpvx4STryZG1grZdHeZr+bdLY9wXXtyavgrRhXtF+2VeANmRpUrSHjmHNHT/84sr33fObyy8sWANAuyJgo6WiLJayZFXl+3o/rj/3tXjKBYB2FnvANrORZvaCmb1kZi+b2VfjLgP5dv8QlzbdsCWZegBAO0mih/1bSZc652ZKukDSp83s4jrHIONuWhM9bat7u0MpbyifAwBaKfaA7TxvDbztHHjke8YAtOamePP7wu3R0sV916+4PwcAxCWRc9hmNszMXpR0WNIPnXPPV+1fZma9ZsbaUgW1aEX4/m8/6D1v3+2/f8sz3nPQfbVLrqxaI/zay+vXDQDaUaKXdZnZOEkPSfqic+6nAWly3fsuwOUIkupfYz3jCmnfgcptpWOChqzr3dErbH9Q3lGuBeeyrnyh/bKvAG2Y/mVdzrl+SdskfTrJctD+fnxP7bYFy8OP6QpZalSSxn8ifP+K1eH7ASBLkpgl3j3Qs5aZnSVpvqR/jbsctJeJnwzfP2VS7bbH6ywLeqzOzTz6T4TvX9fA/a3D1iMHgDR1JJDn+yXda2bD5P0geMA592gC5aCNvPnrxo5Lasb4VTc3dlyzd/wCgKTEHrCdc3sk/V7c+QJD8f1tadcAAOLFSmdomcld6ZY/+7x0yweAZnDzj4Sl/f0mrXqGar1Z2I0OgX/sQ17A33dA+sX+xvJotG5Fa8O8of2yrwBtGGmWeBLnsIFAYZdiLZzT3P2yL7tB2vpccLkAkGUEbMRq5Vpp9Y3hafq3SePmea8PbZUmVQ2VX3erdO8QpinOmSntWC89cffgtn0HvGu/JelghLXJvxjzimkAEDeGxBOW9vebNL/huKiLk5TSbd4qLV0Vnn4ovvt1aellteXUq0+QIrZhntB+2VeANow0JE7ATlja32/S/P6zmDhOOvJkhGMjns9ePFe6frE0b5Z07IT0kz3SbRukn+2tf2yUYD3h0vDLuYrYhnlC+2VfAdqQc9hIR19/48duWeMF6CDjx0gzpkhXL6jcvuNF6ZLPN1Ym114DyAJ62AlL+/tNWtiv+6hD0Z0d0rvP1W6PqrqcztnS6TPND4W/l3+B2zAPaL/sK0Ab0sNGuqKePy4F60Yv+So/7swL0qnno+XV6vtyA0AzWDgFiVpyS/001hMcPG9dJh172gv8pcfJnd52P8MuihaI//jL9dOjhkQyAAAgAElEQVQAQDthSDxhaX+/SYsyHBfUy64OrFfOkx66q/G6LF3lzThvpOwwtGG20X7ZV4A2ZJZ4O0j7+01a1P8s3t4hjRpZdWyP1PeUNGFs5fbRc6W3TkavQ9cY6c0fVW77xkbplrtrA/aSW6T7fxg9b4k2zDraL/sK0Iacw0b7OPvj3nN1AO0YJk2/Qnr1QON5Hz1e2WP+5aO1PW2Jc9YAso1z2Gip8qDpeqWHtzcXrP2cu8i7brv8xwHBGkDWMSSesLS/36Q1Ohw3frR09OmYK+Oje35z14VLtGHW0X7ZV4A2jDQkTg8bqTh2wuv1rlidTP7L7xw4R95ksAaAdkEPO2Fpf79Ji/PXfRx31Epi6Js2zDbaL/sK0Ib0sJEtpeuxrWfwbl7lVq6t3XbOZZXHAUBe0cNOWNrfb9L4dZ99eW9D2i/7CtCG9LABAMgLAjYAABlAwAYAIANSX+ls1qxZ6u2NYXpwm8r7+aW8n1uSaMOso/2yL+9tGBU9bAAAMiD1HjYAZEW7rhWAYqCHDQAhbr5m8F7scSjlddPV8eSH4iBgA4CPrjFeYL3zS8nkv/pGL/9JXcnkj/xhSBwAqsTVm47i0MCtYBkqRz30sAGgTCuDdTuUi+wgYAOApN88m37QdL3Sn34q3TqgfRGwARSe65VGDG8+nxvuaD6Pzben/8MB7Ylz2AAK7Z2dzedRfv75rx/wnpsNur95Vhr5h83lgXyhhw2g0EaOqJ+me7503w/89wVNFmt2ElkcPX7kCwEbQGHV6wWX7rPe1y999i+bD8Ll9263Hum8P2mufigWAjaAQqoXDL91v//2RoO233Ev761/HEEbJQRsAIXTHWGxkuV3Jl8PKdoPgAljk68H2h8BG0DhHN4aX15BPeA4e8Z9T8WXF7KLWeIACuXPrhl87de7LQVa1xt9+Nv1SidOSmPmSsefkUaPil6fDV+JVp8VS6VvboqeL/KHHjaAQrljYG3woGC8//Dg6zkza/cH9ZxLQTooWAcdd91i7/lXB/33l+q5dqX/fhQHARsAykxbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuGVk8UDwEbQGE0e1759cPB+155zXs+ejw4Tdi+KJgxXmwEbAAos3BO8L6pC4P3RRHW+150SXN5I/8I2AAK6WTAkqSPrWttPUoeWeu//Z1nW1sPtC8CNoBCmDyh8v1ZI7wh5rPKliaNMuS88ZHGyn94e/005eWPGum9H1m1ROnEcY2Vj+wjYAMohINP+G8/uVM69bz3OsplXNd/tXbb6TOV7/v6a9NcGWGWd6n8/m3S2zv80xx5sn4+yCcCNoDC6xjW3PHDL6583z2/ufzGvq+545FPBGwAKBOll71kVeV758LTf+5r8ZSLYkskYJvZMDP7ZzN7NIn8ASBN9w9xadMNW5KpB4olqR72lyT9PKG8AWDIbloTPW2re7tDKW8onwP5EnvANrOpki6XdE/ceQNAo9bcFG9+X7g9Wrq47/oV9+dAdiTRw/6mpC9L+u9BCcxsmZn1mlnvkSNHEqgCADRn0Yrw/d9+0Hvevtt//5ZnvOeg+2qXVM8ev/by+nVDMcUasM1skaTDzrldYemcc99xzvU453q6u7vjrAIANGT6ByrfPxZwWVW1ecv8t38mYk+4+vrse30uGwOk+HvYcyRdYWavStos6VIz+7uYywCA2P3Y5yTeguXhx3SFLDUqSeM/Eb5/xerw/UC5WAO2c+4W59xU59wHJS2R9CPn3GfjLAMAGjHxk+H7p0yq3fZ4nWVBj9W5mUf/ifD96xq4v3XYeuTIN67DBlAIb/66seOSmjF+1c2NHdfsHb+QXR1JZeyc2yZpW1L5A0CWfX9b2jVA1tDDBoABk7vSLX/2eemWj/ZGwAZQGPWGtw8OcQWzch/7kDT/Iul3pjaex3Mbw/ezfGmxJTYkDgBZ5HqDA+PCOc3dL/uyG6StzwWXC4QhYAMolJVrpdU3hqfp3yaNm+e9PrRVmlQ1VH7drdK9Q7hTwpyZ0o710hN3D27bd0CacYX3OkrP/osxr5iG7DFX7zYzCevp6XG9vfn9aWlmaVchUWn//bQCbZhtfu0XpTdrPYPpNm+Vlq4KTz8U3/26tPSy2nLq1cdP3ttPyv+/QUm7nHN1T3gQsBOW9z+0tP9+WoE2zDa/9ps4TjryZIRjI54zXjxXun6xNG+WdOyE9JM90m0bpJ/trX9slGA94dLgy7ny3n5S/v8NKmLAZkgcQOH09Td+7JY1XoAOMn6MNGOKdPWCyu07XpQu+XxjZXLtNSQCNoCCijIUXZqA1tkhvVs1WWwoM7Zdr/TxCwbL65wtnT7T3FA4ioeADaCwop4/LgXrRoNn+XFnXpBOPR8tL4I1ynEdNoBCW3JL/TTWExw8b10mHXvaC/ylx8md3nY/wy6KFoj/+Mv106BYmHSWsLxPlkj776cVaMNsi9J+Qb3s6sB65Tzpobsar8vSVd6M80bKDpL39pPy/29QTDoDgGisR3p7hzRqZO2+vqekCWMrt42eK711Mnr+XWOkN38kbbrNe0jSNzZKt9xdm3bJLdL9P4yeN4qDgA0Aks7+uPdc3ePtGCZNv0J69UDjeR89Xtlj/uWjtT1tiXPWCMc5bAAoUx40Xa/08PbmgrWfcxd5122X/zggWKMeetgAUMV6pPGjpaNPS9de7j2S0j2/uevCURz0sAHAx7ETXuBesTqZ/Jff6eVPsEZU9LABIMS6Td5DiueOWgx9o1H0sAEgotL12NYzeDevcivX1m4757LK44BG0cMGgAb8+i3/ALzmvtbXBcVADxsAgAwgYAMAkAEEbAAAMiD1tcTNLNcL4ab9/SatAGv80oYZR/tlXwHaMNJa4vSwAQDIAGaJAwCKY1cMIxKz0unx08MGAOTboTu9QB1HsJYG8zqU0DJ4ATiHnbC0v9+kcf4s+/LehrRf9jXchqfelPZMjLcyfs4/KHVObvjwqOewGRIHAORPXL3pKPac4z0nPFTOkDgAIF9aGaxbWC4BGwCQD7tHpBesS3aZdHRzIlkTsAEA2bfLJPdu09nccEcMddm3NJEfDkw6S1ja32/SmPCSfXlvQ9ov++q24e6RkvttU2X43cil6dup2nDpwvr1YuEUAEAxRAjW3fOl+37gvy/otqdN3w41hh5/OXrYCUv7+00av+6zL+9tSPtlX2gb1hl6jtJzDgvM9dJ+dIb00wdCq1B39jg9bABAvtUJ1t+63397oz1nv+Ne3hvhwJjOZxOwAQDZc/pw3STL72xBPRTxB8DpvqbLIWADALLnpcZXFqsWNLms6Uln5V7qbjoLVjoDAGTLG4PXXoWdo3a90Ye/Xa904qQ0Zq50/Blp9Kjo1dnwlcHXoefMD66VzrkxesZV6GEDALLlwJ9LCg7G+8tGy+fMrN0f1HMuBemgYB103HWLvedfHfTf/149X7/JP0FEBGwAQK5MWzj4esf6ykAbNsz94au85wmXBqepzqv8/bmLhlbPoSJgAwCyo8kZ16+HzFV75TXv+ejx4DRh+yJpov4EbABAriycE7xv6sLgfVGE9b4XXdJc3vUQsAEAmXRyp//2x9a1th4lj6z13/7Os/HkT8AGAGTDqcpZXWeN8M4hnzVicFuUS7E2PtJY8Q9vr5+mvPxRI733I4dXJTp1pKHyWZo0YWl/v0kr/LKIOZD3NqT9su+9Ngw5/3v6jNQ5eyC9T9CunlFenab8eEk68qQ0cdzQ8ihP079NGvu+wOpWLFfK0qQAgMLoGNbc8cMvrnzfPb+5/EKDdYMI2ACAXImyWMqSVZXv6w3EfO5r8ZTbjEQCtpm9amb/YmYvmlmci7sBANC0+7cOLf2GLcnUYyiS7GF/wjl3QZRxeQAA6rlpTfS0Sfd2mylvKJ+jHEPiAIBMWNPcyp41vnB7tHRx3/Wr0c+RVMB2kraa2S4zW1a908yWmVkvw+UAgKQsWhG+/9sPes/bd/vv3/KM9xx0X+2SK1dWvr/28vp1a0Qil3WZ2QeccwfMbJKkH0r6onPumYC0ub7mgktKso82zDbaL/uiXNYlSTOukPYdqDp2oFsYNGRd745eYfuD8o50W852uazLOXdg4PmwpIckXZREOQAAlPz4ntptC5aHH9MVstSoJI3/RPj+FavD98cp9oBtZmeb2ejSa0l/JOmncZcDACiYmeErhE2ZVLvt8TrLgh6rczOP/hPh+9dtCt/v6/y+Bg6SOho6KtxkSQ8NDNN0SPquc+7xBMoBABRJx8SGDktqxvhVNzd4YOeEhg6LPWA75/ZK8rllOAAA+fH9ba0tj8u6AAC5Mbkr3fJnn5dc3tz8I2Fpf79JK9QM1ZzKexvSftlX04Z1Zos3OgT+sQ95AX/fAekX+xvLo+4M8Vm1f49RZ4kncQ4bAIDUhF2KtXBOc/fLvuwGaetzweUmiYANAMiWqXdJ+8NnfPVvk8bN814f2ipNqhoqv+5W6d5Hoxc5Z6a0Y730xN2D2/Yd8K79lqSDUdYmn/ZX0Qv0wZB4wtL+fpNWyOG4nMl7G9J+2efbhnWGxSWvl13q9W7eKi1dFZ5+KL77dWnpZbXlhPIZDpeiD4kTsBOW9vebtML+Z5EjeW9D2i/7fNvw1BFpj8+F11Wins9ePFe6frE0b5Z07IT0kz3SbRukn+2NUL8owfr8vsDLuTiHDQDIr87uhg/dssYL0EHGj5FmTJGuXlC5fceL0iWfb7DQBq+9LkcPO2Fpf79JK+yv+xzJexvSftkX2oYRh8Y7O6R3n6vdHrkOVb3oztnS6TPNDYW/Vw962ACA3JvlIgXtUrBu9JKv8uPOvCCdej5iXnWC9VCwcAoAINum11/Q23qCA+yty6RjT3u95dLj5E5vu59hF0UM1tO/FyFRdAyJJyzt7zdphR+Oy4G8tyHtl32R2jCgl10dWK+cJz10V+N1WbrKm3FeLnBYPGLvmlnibSLt7zdp/GeRfXlvQ9ov+yK34e5RknunYpP1SH1PSRPGViYdPVd662T0OnSNkd78UeW2b2yUbrnbJ2BP3yR1LYmcN+ewAQDFcuFABK7qbXcMk6ZfIb16oPGsjx6v7K3/8tHanrakWM9ZV+McNgAgX8qCpuuVHt7eXLD2c+4i77rtit51gsFaYkg8cWl/v0ljOC778t6GtF/2NdyGp45Ke5q//rmu8w83dV141CFxetgAgHzq7PJ6vdPWJpP/tHVe/k0E66Ggh52wtL/fpPHrPvvy3oa0X/bF2oYRrtmuK+ahb3rYAABUm+UGHzOP1exe6dcZP/+NyuNSQg87YWl/v0nj13325b0Nab/sK0Ab0sMGACAvCNgAAGQAARsAgAxIfaWzWbNmqbc3yv3Jsinv55fyfm5Jog2zjvbLvry3YVT0sAEAyAACNgAAGZD6kDiAHGnDRSmAvKCHDaA5h+70AnUcwVoazOvQ6njyA3KCgA2gMafe9ALr/i8nk//+m738Tx1KJn8gYxgSBzB0cfWmo9hzjvfMUDkKjh42gKFpZbBuh3KBNkHABhDN7hHpB81dJh3dnG4dgJQQsAHUt8sk927T2dxwRwx12bc0/R8OQAo4hw0g3O6RTWdhZfch+usHvGfX7AKHu0dIF/62yUyA7KCHDSCcqx8Uu+dL9/3Af58F3DQwaHtkMfT4gSwhYAMIVmfo2Xq8R1+/9Nm/bD4Il/IrPc77k+bqB+QJARuAvzrB8Fv3+29vNGj7Hffy3ggHErRREARsALVOH66bZPmdLaiHIv4AON2XeD2AtBGwAdR6aXJsWQVNLmt60lm5l7pjzAxoT8wSB1DpjcFrr/x6t6VA63qjD3+7XunESWnMXOn4M9LoUdGrs+Erg6/D6qODa6VzboyeMZAx9LABVDrw55KCg/H+stHyOTNr9wf1nEtBOihYBx133WLv+VcH/fe/V8/Xb/JPAOQEARvAkExbOPh6x/rKQBs2zP3hq7znCZcGp6nOq/z9uYuGVk8gbwjYAAY1OeP69ZC5aq+85j0fPR6cJmxfJMwYR44RsAEMycI5wfumLgzeF0VY73vRJc3lDWQdARuAr5M7/bc/tq619Sh5ZK3/9neebW09gLQQsAF4TlXO6jprhHcO+awRg9uiXIq18ZHGin94e/005eWPGum9Hzm8KtGpI41VAGhzBGwAnj3v9918cqd06nnvdZTLuK7/au2202cq3/f116a5cmX9vEvl92+T3t4RkGjPpPoZARlEwAZQV8ew5o4ffnHl++75zeU39n3NHQ9kUSIB28zGmdnfm9m/mtnPzewPkigHQOtF6WUvWVX53rnw9J/7WjzlAnmWVA97naTHnXP/o6SZkn6eUDkA2tD9W4eWfsOWZOoB5EnsAdvMxkiaK2m9JDnn3nXO+ZyxAtBObloTPW2re7tDKW8onwPIkiR62DMkHZG0wcz+2czuMbOzEygHQIzWxLyy5xduj5Yu7rt+xf05gHaRRMDukHShpL9xzv2epLcl/UV5AjNbZma9ZtZ75AiXYABZtGhF+P5vP+g9b9/tv3/LM95z0H21S6pnj197ef26AXmURMDeL2m/c27gQhD9vbwA/h7n3Heccz3OuZ7ubm6LB2TB9A9Uvn8s6LKqKvOW+W//TMSecPX12ff6XDYGFEHsAds5d1DSa2b2kYFNn5T0s7jLAdBaP76ndtuC5eHHdIUsNSpJ4z8Rvn/F6vD9QJEkdT/sL0q6z8yGS9or6fqEygEQl5lHpJeCR7ym+KxH8nidZUGP1bmZR/+J8P3rNoXv93V+XwMHAe0vkYDtnHtREldNAlnSMbGhw5KaMX7VzQ0e2Dkh1noA7YKVzgC0pe9vS7sGQHshYAOIbHJXuuXPPi/d8oE0EbABDJoVvobowSGuYFbuYx+S5l8k/c7UxvN4bmOdBHXqD2RZUpPOAOSU6w0+b71wTnP3y77sBmnrc8HlAkVGwAZQaepd0v7wGV/926Rx87zXh7ZKk6qGyq+7Vbr30ehFzpkp7VgvPXH34LZ9B6QZV3ivI/Xsp/1V9AKBDGJIHEClyfVvTF26vaXr9YL15q1er7v0GEqwlqSdL1Uev+kJb6GWUq860rnzSV8cWqFAxpird9+7hPX09Lje3vyOdZlZ2lVIVNp/P61QyDY8dUTa43PhdZWol3Qtnitdv1iaN0s6dkL6yR7ptg3Sz/ZGqF+U/x7O7wu8nKuQ7ZczeW9DSbucc3X/NTEkDqBWZ+NLBm9Z4wXoIOPHSDOmSFcvqNy+40Xpks83WCjXXqMACNgA/M1y0q7wnk1pAlpnh/Ru1WSxoSyo4nqlj18w2JvunC2dPhOxd83McBQEARtAsAhBWxoM1o2uelZ+3JkXpFPPR8yLYI0CYdIZgHDT6y/oXZos5ufWZdKxp73eculxcqe33c+wiyIG6+nfi5AIyA8mnSUs75Ml0v77aQXaUIG97OrAeuU86aG7Gq/L0lXejPNygcPiEXvXtF/25b0NxaQzALGZ5aTdoyT3Ts2uvqekCWMrt42eK711Mnr2XWOkN38kbbrNe0jSNzZKt9ztk3j6JqlrSfTMgZwgYAOI5sKBCFzV2+4YJk2/Qnr1QONZHz1e2Vv/5aO1PW1JnLNGoXEOG8DQlAVN1ys9vL25YO3n3EXeddsVw+EEaxQcPWwAQzfLSaeOSnsm6NrLpWsvT7Cs8w83dV04kBf0sAE0prPLC9zT1iaT/7R1Xv4Ea0ASPWwAzZq0wntIka7Zrouhb8AXPWwA8ZnlBh8zj9XsXunXGT//jcrjAPiihw0gGR3jagLw6r9LqS5ADtDDBgAgAwjYAABkAAEbAIAMSH0tcTPL9SyTtL/fpBVgjV/aMONov+wrQBtGWkucHjYAABmQm1nikW50X0ej9/IFACBpme5h33zN4P1141DK66ar48kPAIC4ZPIcdulWfEmb/EfS4aPN5ZH295s0zp9lX97bkPbLvgK0YT7vhx1XbzqKQwO392OoHACQtkwNibcyWLdDuQAAlGQiYP/m2fSDpuuV/vRT6dYBAFBcbR+wXa80Ynjz+dxwR/N5bL49/R8OAIBiautJZ+/slEaOaDJ/n/PPzQbd374rjfzDaGnT/n6TxoSX7Mt7G9J+2VeANsz+wilRgnX3fOm+H/jvC5os1uwksjh6/AAADEXb9rDr9YKj9JzDAnO9tB+dIf30gaHXoaac/P8yTLsKiaMNs432y74CtGF2e9j1gvW37vff3mjP2e+4l/fWP47z2QCAVmm7gN3dVT/N8juTr4cU7QfAhLHJ1wMAgLYL2Ie3xpdXUA84zp5x31Px5QUAQJC2Wunsz64ZfB12jtr1Rh/+dr3SiZPSmLnS8Wek0aOi12fDV6LVZ8VS6ZuboucLAMBQtVUP+44vec9BwXj/4cHXc2bW7g/qOZeCdFCwDjruusXe868O+u8v1XPtSv/9AADEpa0Cdj3TFg6+3rG+MtCGDXN/+CrvecKlwWmq8yp/f+6iodUTAIC4tU3Abva88uuHg/e98pr3fPR4cJqwfVEwYxwAkKS2CdhRLJwTvG/qwuB9UYT1vhdd0lzeAAA0qy0D9smd/tsfW9faepQ8stZ/+zvPtrYeAIDiaouAPXlC5fuzRnhDzGeVLU0aZch54yONlf/w9vppyssfNdJ7P7JqidKJ4xorHwCAetpiadKwYHz6jNQ523vtl656Rnl1mvLjJenIk7WBtV4e5Wn6t0lj3xdc35q88r+kXtpVSBxtmG20X/YVoA2zuzRpuY5hzR0//OLK993zm8svLFgDAJCUtg/Y5aIslrJkVeX7ej/MPve1eMoFACBJsQdsM/uImb1Y9jhuZiviLifI/UNc2nTDlmTqAQBAnGIP2M65f3POXeCcu0DSLEknJT0UdsxNa6Ln3+re7lDKG8rnAABgKJIeEv+kpF84534ZlmjNTfEW+oXbo6WL+65fcX8OAABKkg7YSyTV3BbDzJaZWa+ZNbQ+2KI6A+zfftB73r7bf/+WZ7znoPtql1xZtUb4tZfXrxsAAElI7LIuMxsu6YCkjzrnDoWkC72sS5JmXCHtO1C5rXRM0JB1vTt6he0PyjvKteBc1pU/tGG20X7ZV4A2TP2yrgWSdocF66h+fI9P5svDj+kKWWpUksZ/Inz/itXh+wEAaKUkA/ZS+QyH+5n4yfD9UybVbnu8zrKgx+rczKP/RPj+dQ3c3zpsPXIAAJqRSMA2s1GSPiXpH6Kkf/PXDZaT0Izxq25u7Lhm7/gFAECQjiQydc6dlDShbsI29f1tadcAAIBKmVnpbHJXuuXPPi/d8gEAxdYWN/8ova43C7vRIfCPfcgL+PsOSL/Y31gejdYt7e83acxQzb68tyHtl30FaMNIs8QTGRJPStilWAvnNHe/7MtukLY+F1wuAABpaquAvXKttPrG8DT926Rx87zXh7ZKk6qGyq+7Vbr30ehlzpkp7VgvPXH34LZ9B7xrvyXpYIS1yb8Y84ppAABUa6shcSn64iSldJu3SktXhacfiu9+XVp6WW059eoTJO3vN2kMx2Vf3tuQ9su+ArRhpCHxtgvYE8dJR56McFzE89mL50rXL5bmzZKOnZB+ske6bYP0s731j40SrCdcGn45V9rfb9L4zyL78t6GtF/2FaANs3kOu6+/8WO3rPECdJDxY6QZU6SrF1Ru3/GidMnnGyuTa68BAK3Qdj3skqhD0Z0d0rvP1W6PqrqcztnS6TPND4W/l3/+fxmmXYXE0YbZRvtlXwHaMJs97JKo549LwbrRS77KjzvzgnTq+Wh5tfq+3ACAYmvrhVOW3FI/jfUEB89bl0nHnvYCf+lxcqe33c+wi6IF4j/+cv00AADEqW2HxEuCetnVgfXKedJDdzVej6WrvBnnjZQdJu3vN2kMx2Vf3tuQ9su+ArRhNmeJ+3l7hzRqZNVxPVLfU9KEsZXbR8+V3joZvfyuMdKbP6rc9o2N0i131wbsJbdI9/8wet5SIf7Q0q5C4mjDbKP9sq8AbZjtc9jlzv6491wdQDuGSdOvkF490HjeR49X9ph/+WhtT1vinDUAIF1tfQ67WnnQdL3Sw9ubC9Z+zl3kXbdd/uOAYA0ASFsmhsSrjR8tHX06idpU6p7f3HXhUiGGctKuQuJow2yj/bKvAG0YaUg8Uz3skmMnvF7vitXJ5L/8zoFz5E0GawAA4pLJHrafOO6olcTQd9rfb9L4dZ99eW9D2i/7CtCG+e1h+yldj209g3fzKrdybe22cy6rPA4AgHaVmx52u0r7+00av+6zL+9tSPtlXwHasFg9bAAA8oyADQBABhCwAQDIgHZY6axP0i9bWN7EgTJbIqXzSy39jCnIexvSfjGi/WLX8s9XgDY8N0qi1CedtZqZ9UY5uZ9lef+MfL5s4/NlW94/n9S+n5EhcQAAMoCADQBABhQxYH8n7Qq0QN4/I58v2/h82Zb3zye16Wcs3DlsAACyqIg9bAAAMoeADQBABhQqYJvZp83s38zsFTP7i7TrEycz+1szO2xmP027Lkkws2lm9rSZ/dzMXjazL6Vdp7iZ2Ugze8HMXhr4jF9Nu05xM7NhZvbPZvZo2nVJgpm9amb/YmYvmlkM9xBsL2Y2zsz+3sz+deDf4h+kXae4mNlHBtqt9DhuZivSrle5wpzDNrNhkv4/SZ+StF/SP0la6pz7WaoVi4mZzZX0lqT/6pw7L+36xM3M3i/p/c653WY2WtIuSVfmpf0kybzVIc52zr1lZp2Sdkj6knPuuZSrFhszu0lSj6QxzrlFadcnbmb2qqQe51wuF04xs3sl/dg5d4+ZDZc0yjnXn3a94jYQL16XNNs518qFvUIVqYd9kaRXnHN7nXPvStos6TMp1yk2zrlnJB1Nux5Jcc694ZzbPfD6hKSfS5qSbq3i5TxvDbztHHjk5he1mU2VdLmke9KuC4bOzMZImitpvSQ5597NY7Ae8ElJv2inYC0VK2BPkRolYLIAAAIzSURBVPRa2fv9ytl/+EVhZh+U9HuSnk+3JvEbGDJ+UdJhST90zuXpM35T0pcl/fe0K5IgJ2mrme0ys2VpVyZmMyQdkbRh4LTGPWZ2dtqVSsgSSZvSrkS1IgVsv8Voc9N7KQoze5+kByWtcM4dT7s+cXPOnXHOXSBpqqSLzCwXpzfMbJGkw865XWnXJWFznHMXSlog6T8OnKrKiw5JF0r6G+fc70l6W1Ku5gJJ0sBQ/xWSvpd2XaoVKWDvlzSt7P1USQdSqgsaMHBe90FJ9znn/iHt+iRpYKhxm6RPp1yVuMyRdMXAOd7Nki41s79Lt0rxc84dGHg+LOkheafi8mK/pP1loz5/Ly+A580CSbudc4fSrki1IgXsf5L0YTObPvALaomkLSnXCRENTMhaL+nnzrk1adcnCWbWbWbjBl6fJWm+pH9Nt1bxcM7d4pyb6pz7oLx/ez9yzn025WrFyszOHpgQqYGh4j+SlJurNpxzByW9ZmYfGdj0SUm5mfRZZqnacDhcao/ba7aEc+60md0g6QlJwyT9rXPu5ZSrFRsz2yRpnqSJZrZf0lecc+vTrVWs5ki6RtK/DJzjlaRVzrl/TLFOcXu/pHsHZqj+O0kPOOdyeflTTk2W9NDArSA7JH3XOfd4ulWK3Rcl3TfQ6dkr6fqU6xMrMxsl70qi/5B2XfwU5rIuAACyrEhD4gAAZBYBGwCADCBgAwCQAQRsAAAygIANAEAGELABAMgAAjYAABnw/wPRIOc/pYUmbAAAAABJRU5ErkJggg==", "text/plain": [ "" ] @@ -6526,7 +6529,16 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.4" + "version": "3.7.6" + }, + "pycharm": { + "stem_cell": { + "cell_type": "raw", + "metadata": { + "collapsed": false + }, + "source": [] + } }, "widgets": { "state": { diff --git a/search.py b/search.py index 5b9eb2822..5012c1a18 100644 --- a/search.py +++ b/search.py @@ -1,30 +1,18 @@ -"""Search (Chapters 3-4) +""" +Search (Chapters 3-4) The way to use this code is to subclass Problem to create a class of problems, then create problem instances and solve them with calls to the various search -functions.""" - -from utils import ( - is_in, argmin, argmax, argmax_random_tie, probability, weighted_sampler, - memoize, print_table, open_data, PriorityQueue, name, - distance, vector_add -) +functions. +""" -from collections import defaultdict, deque -import math -import random import sys -import bisect -from operator import itemgetter - - -infinity = float('inf') - -# ______________________________________________________________________________ +from collections import deque +from utils import * -class Problem(object): +class Problem: """The abstract class for a formal problem. You should subclass this and implement the methods actions and result, and possibly __init__, goal_test, and path_cost. Then you will create instances @@ -64,24 +52,25 @@ def path_cost(self, c, state1, action, state2): """Return the cost of a solution path that arrives at state2 from state1 via action, assuming cost c to get up to state1. If the problem is such that the path doesn't matter, this function will only look at - state2. If the path does matter, it will consider c and maybe state1 + state2. If the path does matter, it will consider c and maybe state1 and action. The default method costs 1 for every step in the path.""" return c + 1 def value(self, state): - """For optimization problems, each state has a value. Hill-climbing + """For optimization problems, each state has a value. Hill Climbing and related algorithms try to maximize this value.""" raise NotImplementedError + + # ______________________________________________________________________________ class Node: - """A node in a search tree. Contains a pointer to the parent (the node that this is a successor of) and to the actual state for this node. Note that if a state is arrived at by two paths, then there are two nodes with - the same state. Also includes the action that got us to this state, and - the total path_cost (also known as g) to reach the node. Other functions + the same state. Also includes the action that got us to this state, and + the total path_cost (also known as g) to reach the node. Other functions may add an f and h value; see best_first_graph_search and astar_search for an explanation of how the f and h values are handled. You will not need to subclass this class.""" @@ -110,11 +99,9 @@ def expand(self, problem): def child_node(self, problem, action): """[Figure 3.10]""" next_state = problem.result(self.state, action) - next_node = Node(next_state, self, action, - problem.path_cost(self.path_cost, self.state, - action, next_state)) + next_node = Node(next_state, self, action, problem.path_cost(self.path_cost, self.state, action, next_state)) return next_node - + def solution(self): """Return the sequence of actions to go from the root to this node.""" return [node.action for node in self.path()[1:]] @@ -136,14 +123,21 @@ def __eq__(self, other): return isinstance(other, Node) and self.state == other.state def __hash__(self): + # We use the hash value of the state + # stored in the node instead of the node + # object itself to quickly search a node + # with the same state in a Hash Table return hash(self.state) + # ______________________________________________________________________________ class SimpleProblemSolvingAgentProgram: - - """Abstract framework for a problem-solving agent. [Figure 3.1]""" + """ + [Figure 3.1] + Abstract framework for a problem-solving agent. + """ def __init__(self, initial_state=None): """State is an abstract representation of the state @@ -176,15 +170,19 @@ def formulate_problem(self, state, goal): def search(self, problem): raise NotImplementedError + # ______________________________________________________________________________ # Uninformed Search algorithms def breadth_first_tree_search(problem): - """Search the shallowest nodes in the search tree first. - Search through the successors of a problem to find a goal. - The argument frontier should be an empty queue. - Repeats infinitely in case of loops. [Figure 3.7]""" + """ + [Figure 3.7] + Search the shallowest nodes in the search tree first. + Search through the successors of a problem to find a goal. + The argument frontier should be an empty queue. + Repeats infinitely in case of loops. + """ frontier = deque([Node(problem.initial)]) # FIFO queue @@ -197,10 +195,13 @@ def breadth_first_tree_search(problem): def depth_first_tree_search(problem): - """Search the deepest nodes in the search tree first. - Search through the successors of a problem to find a goal. - The argument frontier should be an empty queue. - Repeats infinitely in case of loops. [Figure 3.7]""" + """ + [Figure 3.7] + Search the deepest nodes in the search tree first. + Search through the successors of a problem to find a goal. + The argument frontier should be an empty queue. + Repeats infinitely in case of loops. + """ frontier = [Node(problem.initial)] # Stack @@ -213,12 +214,16 @@ def depth_first_tree_search(problem): def depth_first_graph_search(problem): - """Search the deepest nodes in the search tree first. - Search through the successors of a problem to find a goal. - The argument frontier should be an empty queue. - Does not get trapped by loops. - If two paths reach a state, only use the first one. [Figure 3.7]""" + """ + [Figure 3.7] + Search the deepest nodes in the search tree first. + Search through the successors of a problem to find a goal. + The argument frontier should be an empty queue. + Does not get trapped by loops. + If two paths reach a state, only use the first one. + """ frontier = [(Node(problem.initial))] # Stack + explored = set() while frontier: node = frontier.pop() @@ -226,8 +231,7 @@ def depth_first_graph_search(problem): return node explored.add(node.state) frontier.extend(child for child in node.expand(problem) - if child.state not in explored and - child not in frontier) + if child.state not in explored and child not in frontier) return None @@ -253,7 +257,7 @@ def breadth_first_graph_search(problem): return None -def best_first_graph_search(problem, f): +def best_first_graph_search(problem, f, display=False): """Search the nodes with the lowest f scores first. You specify the function f(node) that you want to minimize; for example, if f is a heuristic estimate to the goal, then we have greedy best @@ -269,26 +273,28 @@ def best_first_graph_search(problem, f): while frontier: node = frontier.pop() if problem.goal_test(node.state): + if display: + print(len(explored), "paths have been expanded and", len(frontier), "paths remain in the frontier") return node explored.add(node.state) for child in node.expand(problem): if child.state not in explored and child not in frontier: frontier.append(child) elif child in frontier: - incumbent = frontier[child] - if f(child) < f(incumbent): - del frontier[incumbent] + if f(child) < frontier[child]: + del frontier[child] frontier.append(child) return None -def uniform_cost_search(problem): +def uniform_cost_search(problem, display=False): """[Figure 3.14]""" - return best_first_graph_search(problem, lambda node: node.path_cost) + return best_first_graph_search(problem, lambda node: node.path_cost, display) def depth_limited_search(problem, limit=50): """[Figure 3.17]""" + def recursive_dls(node, problem, limit): if problem.goal_test(node.state): return node @@ -315,17 +321,19 @@ def iterative_deepening_search(problem): if result != 'cutoff': return result + # ______________________________________________________________________________ # Bidirectional Search # Pseudocode from https://webdocs.cs.ualberta.ca/%7Eholte/Publications/MM-AAAI2016.pdf def bidirectional_search(problem): - e = problem.find_min_edge() - gF, gB = {problem.initial : 0}, {problem.goal : 0} - openF, openB = [problem.initial], [problem.goal] + e = 0 + if isinstance(problem, GraphProblem): + e = problem.find_min_edge() + gF, gB = {Node(problem.initial): 0}, {Node(problem.goal): 0} + openF, openB = [Node(problem.initial)], [Node(problem.goal)] closedF, closedB = [], [] - U = infinity - + U = np.inf def extend(U, open_dir, open_other, g_dir, g_other, closed_dir): """Extend search in given direction""" @@ -334,14 +342,14 @@ def extend(U, open_dir, open_other, g_dir, g_other, closed_dir): open_dir.remove(n) closed_dir.append(n) - for c in problem.actions(n): + for c in n.expand(problem): if c in open_dir or c in closed_dir: - if g_dir[c] <= problem.path_cost(g_dir[n], n, None, c): + if g_dir[c] <= problem.path_cost(g_dir[n], n.state, None, c.state): continue open_dir.remove(c) - g_dir[c] = problem.path_cost(g_dir[n], n, None, c) + g_dir[c] = problem.path_cost(g_dir[n], n.state, None, c.state) open_dir.append(c) if c in open_other: @@ -349,33 +357,32 @@ def extend(U, open_dir, open_other, g_dir, g_other, closed_dir): return U, open_dir, closed_dir, g_dir - def find_min(open_dir, g): """Finds minimum priority, g and f values in open_dir""" - m, m_f = infinity, infinity + # pr_min_f isn't forward pr_min instead it's the f-value + # of node with priority pr_min. + pr_min, pr_min_f = np.inf, np.inf for n in open_dir: f = g[n] + problem.h(n) - pr = max(f, 2*g[n]) - m = min(m, pr) - m_f = min(m_f, f) - - return m, m_f, min(g.values()) + pr = max(f, 2 * g[n]) + pr_min = min(pr_min, pr) + pr_min_f = min(pr_min_f, f) + return pr_min, pr_min_f, min(g.values()) def find_key(pr_min, open_dir, g): """Finds key in open_dir with value equal to pr_min and minimum g value.""" - m = infinity - state = -1 + m = np.inf + node = Node(-1) for n in open_dir: - pr = max(g[n] + problem.h(n), 2*g[n]) + pr = max(g[n] + problem.h(n), 2 * g[n]) if pr == pr_min: if g[n] < m: m = g[n] - state = n - - return state + node = n + return node while openF and openB: pr_min_f, f_min_f, g_min_f = find_min(openF, gF) @@ -392,49 +399,50 @@ def find_key(pr_min, open_dir, g): # Extend backward U, openB, closedB, gB = extend(U, openB, openF, gB, gF, closedB) - return infinity + return np.inf + # ______________________________________________________________________________ # Informed (Heuristic) Search greedy_best_first_graph_search = best_first_graph_search + + # Greedy best-first search is accomplished by specifying f(n) = h(n). -def astar_search(problem, h=None): +def astar_search(problem, h=None, display=False): """A* search is best-first graph search with f(n) = g(n)+h(n). You need to specify the h function when you call astar_search, or else in your Problem subclass.""" h = memoize(h or problem.h, 'h') - return best_first_graph_search(problem, lambda n: n.path_cost + h(n)) + return best_first_graph_search(problem, lambda n: n.path_cost + h(n), display) + # ______________________________________________________________________________ # A* heuristics class EightPuzzle(Problem): + """ The problem of sliding tiles numbered from 1 to 8 on a 3x3 board, where one of the + squares is a blank. A state is represented as a tuple of length 9, where element at + index i represents the tile number at index i (0 if it's an empty square) """ - """ The problem of sliding tiles numbered from 1 to 8 on a 3x3 board, - where one of the squares is a blank. A state is represented as a tuple of length 9, - where element at index i represents the tile number at index i (0 if it's an empty square) """ - def __init__(self, initial, goal=(1, 2, 3, 4, 5, 6, 7, 8, 0)): """ Define goal state and initialize a problem """ + super().__init__(initial, goal) - self.goal = goal - Problem.__init__(self, initial, goal) - def find_blank_square(self, state): """Return the index of the blank square in a given state""" return state.index(0) - + def actions(self, state): """ Return the actions that can be executed in the given state. The result would be a list, since there are only four possible actions in any given state of the environment """ - - possible_actions = ['UP', 'DOWN', 'LEFT', 'RIGHT'] + + possible_actions = ['UP', 'DOWN', 'LEFT', 'RIGHT'] index_blank_square = self.find_blank_square(state) if index_blank_square % 3 == 0: @@ -456,7 +464,7 @@ def result(self, state, action): blank = self.find_blank_square(state) new_state = list(state) - delta = {'UP':-3, 'DOWN':3, 'LEFT':-1, 'RIGHT':1} + delta = {'UP': -3, 'DOWN': 3, 'LEFT': -1, 'RIGHT': 1} neighbor = blank + delta[action] new_state[blank], new_state[neighbor] = new_state[neighbor], new_state[blank] @@ -472,18 +480,19 @@ def check_solvability(self, state): inversion = 0 for i in range(len(state)): - for j in range(i+1, len(state)): - if (state[i] > state[j]) and state[i] != 0 and state[j]!= 0: + for j in range(i + 1, len(state)): + if (state[i] > state[j]) and state[i] != 0 and state[j] != 0: inversion += 1 - + return inversion % 2 == 0 - + def h(self, node): """ Return the heuristic value for a given state. Default heuristic function used is h(n) = number of misplaced tiles """ return sum(s != g for (s, g) in zip(node.state, self.goal)) + # ______________________________________________________________________________ @@ -492,11 +501,10 @@ class PlanRoute(Problem): def __init__(self, initial, goal, allowed, dimrow): """ Define goal state and initialize a problem """ - + super().__init__(initial, goal) self.dimrow = dimrow self.goal = goal self.allowed = allowed - Problem.__init__(self, initial, goal) def actions(self, state): """ Return the actions that can be executed in the given state. @@ -598,10 +606,10 @@ def recursive_best_first_search(problem, h=None): def RBFS(problem, node, flimit): if problem.goal_test(node.state): - return node, 0 # (The second value is immaterial) + return node, 0 # (The second value is immaterial) successors = node.expand(problem) if len(successors) == 0: - return None, infinity + return None, np.inf for s in successors: s.f = max(s.path_cost + h(s), node.f) while True: @@ -613,27 +621,29 @@ def RBFS(problem, node, flimit): if len(successors) > 1: alternative = successors[1].f else: - alternative = infinity + alternative = np.inf result, best.f = RBFS(problem, best, min(flimit, alternative)) if result is not None: return result, best.f node = Node(problem.initial) node.f = h(node) - result, bestf = RBFS(problem, node, infinity) + result, bestf = RBFS(problem, node, np.inf) return result def hill_climbing(problem): - """From the initial node, keep choosing the neighbor with highest value, - stopping when no neighbor is better. [Figure 4.2]""" + """ + [Figure 4.2] + From the initial node, keep choosing the neighbor with highest value, + stopping when no neighbor is better. + """ current = Node(problem.initial) while True: neighbors = current.expand(problem) if not neighbors: break - neighbor = argmax_random_tie(neighbors, - key=lambda node: problem.value(node.state)) + neighbor = argmax_random_tie(neighbors, key=lambda node: problem.value(node.state)) if problem.value(neighbor.state) <= problem.value(current.state): break current = neighbor @@ -642,7 +652,7 @@ def hill_climbing(problem): def exp_schedule(k=20, lam=0.005, limit=100): """One possible schedule function for simulated annealing""" - return lambda t: (k * math.exp(-lam * t) if t < limit else 0) + return lambda t: (k * np.exp(-lam * t) if t < limit else 0) def simulated_annealing(problem, schedule=exp_schedule()): @@ -658,9 +668,10 @@ def simulated_annealing(problem, schedule=exp_schedule()): return current.state next_choice = random.choice(neighbors) delta_e = problem.value(next_choice.state) - problem.value(current.state) - if delta_e > 0 or probability(math.exp(delta_e / T)): + if delta_e > 0 or probability(np.exp(delta_e / T)): current = next_choice + def simulated_annealing_full(problem, schedule=exp_schedule()): """ This version returns all the states encountered in reaching the goal state.""" @@ -676,9 +687,10 @@ def simulated_annealing_full(problem, schedule=exp_schedule()): return current.state next_choice = random.choice(neighbors) delta_e = problem.value(next_choice.state) - problem.value(current.state) - if delta_e > 0 or probability(math.exp(delta_e / T)): + if delta_e > 0 or probability(np.exp(delta_e / T)): current = next_choice + def and_or_graph_search(problem): """[Figure 4.11]Used when the environment is nondeterministic and completely observable. Contains OR nodes where the agent is free to choose any action. @@ -714,17 +726,19 @@ def and_search(states, problem, path): # body of and or search return or_search(problem.initial, problem, []) + # Pre-defined actions for PeakFindingProblem -directions4 = { 'W':(-1, 0), 'N':(0, 1), 'E':(1, 0), 'S':(0, -1) } -directions8 = dict(directions4) -directions8.update({'NW':(-1, 1), 'NE':(1, 1), 'SE':(1, -1), 'SW':(-1, -1) }) +directions4 = {'W': (-1, 0), 'N': (0, 1), 'E': (1, 0), 'S': (0, -1)} +directions8 = dict(directions4) +directions8.update({'NW': (-1, 1), 'NE': (1, 1), 'SE': (1, -1), 'SW': (-1, -1)}) + class PeakFindingProblem(Problem): """Problem of finding the highest peak in a limited grid""" def __init__(self, initial, grid, defined_actions=directions4): """The grid is a 2 dimensional array/list whose state is specified by tuple of indices""" - Problem.__init__(self, initial) + super().__init__(initial) self.grid = grid self.defined_actions = defined_actions self.n = len(grid) @@ -737,7 +751,7 @@ def actions(self, state): allowed_actions = [] for action in self.defined_actions: next_state = vector_add(state, self.defined_actions[action]) - if next_state[0] >= 0 and next_state[1] >= 0 and next_state[0] <= self.n - 1 and next_state[1] <= self.m - 1: + if 0 <= next_state[0] <= self.n - 1 and 0 <= next_state[1] <= self.m - 1: allowed_actions.append(action) return allowed_actions @@ -755,11 +769,13 @@ def value(self, state): class OnlineDFSAgent: - - """[Figure 4.21] The abstract class for an OnlineDFSAgent. Override + """ + [Figure 4.21] + The abstract class for an OnlineDFSAgent. Override update_state method to convert percept to state. While initializing the subclass a problem needs to be provided which is an instance of - a subclass of the Problem class.""" + a subclass of the Problem class. + """ def __init__(self, problem): self.problem = problem @@ -800,6 +816,7 @@ def update_state(self, percept): assumes the percept to be of type state.""" return percept + # ______________________________________________________________________________ @@ -810,8 +827,7 @@ class OnlineSearchProblem(Problem): Carried in a deterministic and a fully observable environment.""" def __init__(self, initial, goal, graph): - self.initial = initial - self.goal = goal + super().__init__(initial, goal) self.graph = graph def actions(self, state): @@ -838,7 +854,6 @@ def goal_test(self, state): class LRTAStarAgent: - """ [Figure 4.24] Abstract class for LRTA*-Agent. A problem needs to be provided which is an instance of a subclass of Problem Class. @@ -853,7 +868,7 @@ def __init__(self, problem): self.s = None self.a = None - def __call__(self, s1): # as of now s1 is a state rather than a percept + def __call__(self, s1): # as of now s1 is a state rather than a percept if self.problem.goal_test(s1): self.a = None return self.a @@ -865,11 +880,11 @@ def __call__(self, s1): # as of now s1 is a state rather than a percept # minimum cost for action b in problem.actions(s) self.H[self.s] = min(self.LRTA_cost(self.s, b, self.problem.output(self.s, b), - self.H) for b in self.problem.actions(self.s)) + self.H) for b in self.problem.actions(self.s)) # an action b in problem.actions(s1) that minimizes costs - self.a = argmin(self.problem.actions(s1), - key=lambda b: self.LRTA_cost(s1, b, self.problem.output(s1, b), self.H)) + self.a = min(self.problem.actions(s1), + key=lambda b: self.LRTA_cost(s1, b, self.problem.output(s1, b), self.H)) self.s = s1 return self.a @@ -888,11 +903,12 @@ def LRTA_cost(self, s, a, s1, H): except: return self.problem.c(s, a, s1) + self.problem.h(s1) + # ______________________________________________________________________________ # Genetic Algorithm -def genetic_search(problem, fitness_fn, ngen=1000, pmut=0.1, n=20): +def genetic_search(problem, ngen=1000, pmut=0.1, n=20): """Call genetic_algorithm on the appropriate parts of a problem. This requires the problem to have states that can mate and mutate, plus a value method that scores states.""" @@ -916,22 +932,20 @@ def genetic_algorithm(population, fitness_fn, gene_pool=[0, 1], f_thres=None, ng if fittest_individual: return fittest_individual - - return argmax(population, key=fitness_fn) + return max(population, key=fitness_fn) def fitness_threshold(fitness_fn, f_thres, population): if not f_thres: return None - fittest_individual = argmax(population, key=fitness_fn) + fittest_individual = max(population, key=fitness_fn) if fitness_fn(fittest_individual) >= f_thres: return fittest_individual return None - def init_population(pop_number, gene_pool, state_length): """Initializes population for genetic algorithm pop_number : Number of individuals in population @@ -967,7 +981,7 @@ def recombine_uniform(x, y): result[ix] = x[ix] if i < n / 2 else y[ix] return ''.join(str(r) for r in result) - + def mutate(x, gene_pool, pmut): if random.uniform(0, 1) >= pmut: @@ -979,7 +993,8 @@ def mutate(x, gene_pool, pmut): r = random.randrange(0, g) new_gene = gene_pool[r] - return x[:c] + [new_gene] + x[c+1:] + return x[:c] + [new_gene] + x[c + 1:] + # _____________________________________________________________________________ # The remainder of this file implements examples for the search algorithms. @@ -989,18 +1004,17 @@ def mutate(x, gene_pool, pmut): class Graph: - - """A graph connects nodes (vertices) by edges (links). Each edge can also - have a length associated with it. The constructor call is something like: + """A graph connects nodes (vertices) by edges (links). Each edge can also + have a length associated with it. The constructor call is something like: g = Graph({'A': {'B': 1, 'C': 2}) this makes a graph with 3 nodes, A, B, and C, with an edge of length 1 from - A to B, and an edge of length 2 from A to C. You can also do: + A to B, and an edge of length 2 from A to C. You can also do: g = Graph({'A': {'B': 1, 'C': 2}, directed=False) This makes an undirected graph, so inverse links are also added. The graph stays undirected; if you add more links with g.connect('B', 'C', 3), then - inverse link is also added. You can use g.nodes() to get a list of nodes, + inverse link is also added. You can use g.nodes() to get a list of nodes, g.get('A') to get a dict of links out of A, and g.get('A', 'B') to get the - length of the link from A to B. 'Lengths' can actually be any object at + length of the link from A to B. 'Lengths' can actually be any object at all, and nodes can be any hashable object.""" def __init__(self, graph_dict=None, directed=True): @@ -1046,7 +1060,7 @@ def nodes(self): def UndirectedGraph(graph_dict=None): """Build a Graph where every edge (including future ones) goes both ways.""" - return Graph(graph_dict = graph_dict, directed=False) + return Graph(graph_dict=graph_dict, directed=False) def RandomGraph(nodes=list(range(10)), min_links=2, width=400, height=300, @@ -1070,9 +1084,10 @@ def RandomGraph(nodes=list(range(10)), min_links=2, width=400, height=300, def distance_to_node(n): if n is node or g.get(node, n): - return infinity + return np.inf return distance(g.locations[n], here) - neighbor = argmin(nodes, key=distance_to_node) + + neighbor = min(nodes, key=distance_to_node) d = distance(g.locations[neighbor], here) * curvature() g.connect(node, neighbor, int(d)) return g @@ -1127,7 +1142,7 @@ def distance_to_node(n): State_6=dict(Suck=['State_8'], Left=['State_5']), State_7=dict(Suck=['State_7', 'State_3'], Right=['State_8']), State_8=dict(Suck=['State_8', 'State_6'], Left=['State_7']) - )) +)) """ [Figure 4.23] One-dimensional state space Graph @@ -1139,7 +1154,7 @@ def distance_to_node(n): State_4=dict(Right='State_5', Left='State_3'), State_5=dict(Right='State_6', Left='State_4'), State_6=dict(Left='State_5') - )) +)) one_dim_state_space.least_costs = dict( State_1=8, State_2=9, @@ -1162,11 +1177,10 @@ def distance_to_node(n): class GraphProblem(Problem): - """The problem of searching a graph from one node to another.""" def __init__(self, initial, goal, graph): - Problem.__init__(self, initial, goal) + super().__init__(initial, goal) self.graph = graph def actions(self, A): @@ -1178,11 +1192,11 @@ def result(self, state, action): return action def path_cost(self, cost_so_far, A, action, B): - return cost_so_far + (self.graph.get(A, B) or infinity) + return cost_so_far + (self.graph.get(A, B) or np.inf) def find_min_edge(self): """Find minimum value of edges.""" - m = infinity + m = np.inf for d in self.graph.graph_dict.values(): local_min = min(d.values()) m = min(m, local_min) @@ -1198,7 +1212,7 @@ def h(self, node): return int(distance(locs[node.state], locs[self.goal])) else: - return infinity + return np.inf class GraphProblemStochastic(GraphProblem): @@ -1221,24 +1235,22 @@ def path_cost(self): class NQueensProblem(Problem): - """The problem of placing N queens on an NxN board with none attacking - each other. A state is represented as an N-element array, where + each other. A state is represented as an N-element array, where a value of r in the c-th entry means there is a queen at column c, row r, and a value of -1 means that the c-th column has not been - filled in yet. We fill in columns left to right. + filled in yet. We fill in columns left to right. >>> depth_first_tree_search(NQueensProblem(8)) """ def __init__(self, N): + super().__init__(tuple([-1] * N)) self.N = N - self.initial = tuple([-1] * N) - Problem.__init__(self, self.initial) def actions(self, state): """In the leftmost empty column, try all non-conflicting rows.""" - if state[-1] is not -1: + if state[-1] != -1: return [] # All columns filled; no successors else: col = state.index(-1) @@ -1262,11 +1274,11 @@ def conflict(self, row1, col1, row2, col2): return (row1 == row2 or # same row col1 == col2 or # same column row1 - col1 == row2 - col2 or # same \ diagonal - row1 + col1 == row2 + col2) # same / diagonal + row1 + col1 == row2 + col2) # same / diagonal def goal_test(self, state): """Check if all columns filled, no conflicts.""" - if state[-1] is -1: + if state[-1] == -1: return False return not any(self.conflicted(state, state[col], col) for col in range(len(state))) @@ -1281,6 +1293,7 @@ def h(self, node): return num_conflicts + # ______________________________________________________________________________ # Inverse Boggle: Search for a high-scoring Boggle board. A good domain for # iterative-repair and related search techniques, as suggested by Justin Boyan. @@ -1301,6 +1314,7 @@ def random_boggle(n=4): random.shuffle(cubes) return list(map(random.choice, cubes)) + # The best 5x5 board found by Boyan, with our word list this board scores # 2274 words, for a score of 9837 @@ -1335,7 +1349,7 @@ def boggle_neighbors(n2, cache={}): on_top = i < n on_bottom = i >= n2 - n on_left = i % n == 0 - on_right = (i+1) % n == 0 + on_right = (i + 1) % n == 0 if not on_top: neighbors[i].append(i - n) if not on_left: @@ -1358,15 +1372,15 @@ def boggle_neighbors(n2, cache={}): def exact_sqrt(n2): """If n2 is a perfect square, return its square root, else raise error.""" - n = int(math.sqrt(n2)) + n = int(np.sqrt(n2)) assert n * n == n2 return n + # _____________________________________________________________________________ class Wordlist: - """This class holds a list of words. You can use (word in wordlist) to check if a word is in the list, or wordlist.lookup(prefix) to see if prefix starts any of the words in the list.""" @@ -1401,11 +1415,11 @@ def __contains__(self, word): def __len__(self): return len(self.words) + # _____________________________________________________________________________ class BoggleFinder: - """A class that allows you to find all the words in a Boggle board.""" wordlist = None # A class variable, holding a wordlist @@ -1462,6 +1476,7 @@ def __len__(self): """The number of words found.""" return len(self.found) + # _____________________________________________________________________________ @@ -1493,13 +1508,13 @@ def mutate_boggle(board): board[i] = random.choice(random.choice(cubes16)) return i, oldc + # ______________________________________________________________________________ # Code to compare searchers on various problems. class InstrumentedProblem(Problem): - """Delegates to a problem, and keeps statistics.""" def __init__(self, problem): @@ -1547,6 +1562,7 @@ def do(searcher, problem): p = InstrumentedProblem(problem) searcher(p) return p + table = [[name(s)] + [do(s, p) for p in problems] for s in searchers] print_table(table, header) @@ -1558,4 +1574,3 @@ def compare_graph_searchers(): GraphProblem('Q', 'WA', australia_map)], header=['Searcher', 'romania_map(Arad, Bucharest)', 'romania_map(Oradea, Neamt)', 'australia_map']) - diff --git a/search4e.ipynb b/search4e.ipynb index 9667a4a09..7c636f2e7 100644 --- a/search4e.ipynb +++ b/search4e.ipynb @@ -8,36 +8,48 @@ "\n", "Implementation of search algorithms and search problems for AIMA.\n", "\n", - "We start by defining the abstract class for a `Problem`; problem domains will subclass this, and then you can create individual problems with specific initial states and goals. We also ddefine a `Node` in a search tree, and some functions on nodes: `expand` to generate successors, and `path_actions`, `path_states` and `path` to recover aspects of the path from the node. Finally, a `PriorityQueue`, which allows you to keep a collection of items, and continually remove from it the item with minimum `f(item)` score." + "# Problems and Nodes\n", + "\n", + "We start by defining the abstract class for a `Problem`; specific problem domains will subclass this. To make it easier for algorithms that use a heuristic evaluation function, `Problem` has a default `h` function (uniformly zero), and subclasses can define their own default `h` function.\n", + "\n", + "We also define a `Node` in a search tree, and some functions on nodes: `expand` to generate successors; `path_actions` and `path_states` to recover aspects of the path from the node. " ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 93, "metadata": {}, "outputs": [], "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "import random\n", "import heapq\n", "import math\n", "import sys\n", "from collections import defaultdict, deque, Counter\n", + "from itertools import combinations\n", "\n", - "class Problem(object):\n", - " \"\"\"The abstract class for a formal problem. You should subclass this,\n", - " overriding `actions` and `results`, and other methods if necessary.\n", - " Note: a problem can specify a default heuristic if desired. By default, \n", - " the heuristic is 0 for all states, and the step cost is 1 for all actions.\"\"\"\n", "\n", - " def __init__(self, initial=None, goal=None, **other_keywords):\n", - " \"\"\"Specify the initial and goal states.\n", - " Subclasses can use other keywords if they want.\"\"\"\n", - " self.__dict__.update(initial=initial, goal=goal, **other_keywords) \n", + "class Problem(object):\n", + " \"\"\"The abstract class for a formal problem. A new domain subclasses this,\n", + " overriding `actions` and `results`, and perhaps other methods.\n", + " The default heuristic is 0 and the default action cost is 1 for all states.\n", + " When yiou create an instance of a subclass, specify `initial`, and `goal` states \n", + " (or give an `is_goal` method) and perhaps other keyword args for the subclass.\"\"\"\n", "\n", - " def actions(self, state): raise NotImplementedError\n", - " def result(self, state, action): raise NotImplementedError\n", - " def is_goal(self, state): return state == self.goal\n", - " def step_cost(self, s, action, s1): return 1\n", - " def h(self, node): return 0\n", + " def __init__(self, initial=None, goal=None, **kwds): \n", + " self.__dict__.update(initial=initial, goal=goal, **kwds) \n", + " \n", + " def actions(self, state): raise NotImplementedError\n", + " def result(self, state, action): raise NotImplementedError\n", + " def is_goal(self, state): return state == self.goal\n", + " def action_cost(self, s, a, s1): return 1\n", + " def h(self, node): return 0\n", + " \n", + " def __str__(self):\n", + " return '{}({!r}, {!r})'.format(\n", + " type(self).__name__, self.initial, self.goal)\n", " \n", "\n", "class Node:\n", @@ -47,7 +59,11 @@ "\n", " def __repr__(self): return '<{}>'.format(self.state)\n", " def __len__(self): return 0 if self.parent is None else (1 + len(self.parent))\n", - " def __lt__(self, other): return self.state < other.state\n", + " def __lt__(self, other): return self.path_cost < other.path_cost\n", + " \n", + " \n", + "failure = Node('failure', path_cost=math.inf) # Indicates an algorithm couldn't find a solution.\n", + "cutoff = Node('cutoff', path_cost=math.inf) # Indicates iterative deepening search was cut off.\n", " \n", " \n", "def expand(problem, node):\n", @@ -55,30 +71,31 @@ " s = node.state\n", " for action in problem.actions(s):\n", " s1 = problem.result(s, action)\n", - " cost = node.path_cost + problem.step_cost(s, action, s1)\n", + " cost = node.path_cost + problem.action_cost(s, action, s1)\n", " yield Node(s1, node, action, cost)\n", " \n", "\n", "def path_actions(node):\n", " \"The sequence of actions to get to this node.\"\n", - " return [] if node.parent is None else path_actions(node.parent) + [node.action]\n", + " if node.parent is None:\n", + " return [] \n", + " return path_actions(node.parent) + [node.action]\n", "\n", "\n", "def path_states(node):\n", " \"The sequence of states to get to this node.\"\n", - " return ([] if node.parent is None else path_states(node.parent) ) + [node.state]\n", - "\n", - "\n", - "def path(node):\n", - " \"Alternating states and actions to get to this node.\"\n", - " return ([] if node.parent is None else path(node.parent) + [node.action] ) + [node.state]" + " if node in (cutoff, failure, None): \n", + " return []\n", + " return path_states(node.parent) + [node.state]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Queues" + "# Queues\n", + "\n", + "First-in-first-out and Last-in-first-out queues, and a `PriorityQueue`, which allows you to keep a collection of items, and continually remove from it the item with minimum `f(item)` score." ] }, { @@ -87,26 +104,29 @@ "metadata": {}, "outputs": [], "source": [ - "FIFOQueue = list\n", + "FIFOQueue = deque\n", "\n", - "LIFOQueue = deque\n", + "LIFOQueue = list\n", "\n", "class PriorityQueue:\n", " \"\"\"A queue in which the item with minimum f(item) is always popped first.\"\"\"\n", "\n", " def __init__(self, items=(), key=lambda x: x): \n", " self.key = key\n", - " self.items = []\n", + " self.items = [] # a heap of (score, item) pairs\n", " for item in items:\n", " self.add(item)\n", " \n", " def add(self, item):\n", " \"\"\"Add item to the queuez.\"\"\"\n", - " heapq.heappush(self.items, (self.key(item), item))\n", + " pair = (self.key(item), item)\n", + " heapq.heappush(self.items, pair)\n", "\n", " def pop(self):\n", " \"\"\"Pop and return the item with min f(item) value.\"\"\"\n", " return heapq.heappop(self.items)[1]\n", + " \n", + " def top(self): return self.items[0][1]\n", "\n", " def __len__(self): return len(self.items)" ] @@ -115,562 +135,1200 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Search Algorithms\n", + "# Search Algorithms: Best-First\n", "\n", - "Here are the six major state-space search algorithms covered in the book:" + "Best-first search with various *f(n)* functions gives us different search algorithms. Note that A\\*, weighted A\\* and greedy search can be given a heuristic function, `h`, but if `h` is not supplied they use the problem's default `h` function (if the problem does not define one, it is taken as *h(n)* = 0)." ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 356, "metadata": {}, "outputs": [], "source": [ - "def breadth_first_search(problem):\n", - " \"Search shallowest nodes in the search tree first.\"\n", - " frontier = LIFOQueue([Node(problem.initial)])\n", - " reached = set()\n", + "def best_first_search(problem, f):\n", + " \"Search nodes with minimum f(node) value first.\"\n", + " node = Node(problem.initial)\n", + " frontier = PriorityQueue([node], key=f)\n", + " reached = {problem.initial: node}\n", " while frontier:\n", " node = frontier.pop()\n", " if problem.is_goal(node.state):\n", " return node\n", " for child in expand(problem, node):\n", " s = child.state\n", - " if s not in reached:\n", - " reached.add(s)\n", - " frontier.appendleft(child)\n", + " if s not in reached or child.path_cost < reached[s].path_cost:\n", + " reached[s] = child\n", + " frontier.add(child)\n", " return failure\n", "\n", - "def depth_limited_search(problem, limit=5):\n", - " \"Search deepest nodes in the search tree first.\"\n", - " frontier = FIFOQueue([Node(problem.initial)])\n", - " solution = failure\n", - " while frontier:\n", - " node = frontier.pop()\n", - " if len(node) > limit:\n", - " solution = cutoff\n", - " else:\n", - " for child in expand(problem, node):\n", - " if problem.is_goal(child.state):\n", - " return child\n", - " frontier.append(child)\n", - " return solution\n", - "\n", - "def iterative_deepening_search(problem):\n", - " \"Do depth-limited search with increasing depth limits.\"\n", - " for limit in range(1, sys.maxsize):\n", - " result = depth_limited_search(problem, limit)\n", - " if result != cutoff:\n", - " return result\n", - "\n", "\n", - "def best_first_search(problem, f):\n", - " \"Search nodes with minimum f(node) value first.\"\n", + "def best_first_tree_search(problem, f):\n", + " \"A version of best_first_search without the `reached` table.\"\n", " frontier = PriorityQueue([Node(problem.initial)], key=f)\n", - " reached = {}\n", " while frontier:\n", " node = frontier.pop()\n", " if problem.is_goal(node.state):\n", " return node\n", " for child in expand(problem, node):\n", - " s = child.state\n", - " if s not in reached or child.path_cost < reached[s].path_cost:\n", - " reached[s] = child\n", + " if not is_cycle(child):\n", " frontier.add(child)\n", " return failure\n", "\n", "\n", - "def uniform_cost_search(problem):\n", - " \"Search niodes with minimum path cost first.\"\n", - " return best_first_search(problem, lambda node: node.path_cost)\n", + "def g(n): return n.path_cost\n", "\n", "\n", "def astar_search(problem, h=None):\n", - " \"\"\"Search niodes with minimum f(n) = g(n) + h(n).\"\"\"\n", + " \"\"\"Search nodes with minimum f(n) = g(n) + h(n).\"\"\"\n", " h = h or problem.h\n", - " return best_first_search(problem, lambda node: node.path_cost + h(node))\n", + " return best_first_search(problem, f=lambda n: g(n) + h(n))\n", "\n", - "failure = Node('failure', path_cost=math.inf) # Indicates an algorithm couldn't find a solution.\n", - "cutoff = Node('cutoff', path_cost=math.inf) # Indicates iterative deeepening search was cut off." + "\n", + "def astar_tree_search(problem, h=None):\n", + " \"\"\"Search nodes with minimum f(n) = g(n) + h(n), with no `reached` table.\"\"\"\n", + " h = h or problem.h\n", + " return best_first_tree_search(problem, f=lambda n: g(n) + h(n))\n", + "\n", + "\n", + "def weighted_astar_search(problem, h=None, weight=1.4):\n", + " \"\"\"Search nodes with minimum f(n) = g(n) + weight * h(n).\"\"\"\n", + " h = h or problem.h\n", + " return best_first_search(problem, f=lambda n: g(n) + weight * h(n))\n", + "\n", + " \n", + "def greedy_bfs(problem, h=None):\n", + " \"\"\"Search nodes with minimum h(n).\"\"\"\n", + " h = h or problem.h\n", + " return best_first_search(problem, f=h)\n", + "\n", + "\n", + "def uniform_cost_search(problem):\n", + " \"Search nodes with minimum path cost first.\"\n", + " return best_first_search(problem, f=g)\n", + "\n", + "\n", + "def breadth_first_bfs(problem):\n", + " \"Search shallowest nodes in the search tree first; using best-first.\"\n", + " return best_first_search(problem, f=len)\n", + "\n", + "\n", + "def depth_first_bfs(problem):\n", + " \"Search deepest nodes in the search tree first; using best-first.\"\n", + " return best_first_search(problem, f=lambda n: -len(n))\n", + "\n", + "\n", + "def is_cycle(node, k=30):\n", + " \"Does this node form a cycle of length k or less?\"\n", + " def find_cycle(ancestor, k):\n", + " return (ancestor is not None and k > 0 and\n", + " (ancestor.state == node.state or find_cycle(ancestor.parent, k - 1)))\n", + " return find_cycle(node.parent, k)\n", + "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Problem Domains\n", + "# Other Search Algorithms\n", + "\n", + "Here are the other search algorithms:" + ] + }, + { + "cell_type": "code", + "execution_count": 234, + "metadata": {}, + "outputs": [], + "source": [ + "def breadth_first_search(problem):\n", + " \"Search shallowest nodes in the search tree first.\"\n", + " node = Node(problem.initial)\n", + " if problem.is_goal(problem.initial):\n", + " return node\n", + " frontier = FIFOQueue([node])\n", + " reached = {problem.initial}\n", + " while frontier:\n", + " node = frontier.pop()\n", + " for child in expand(problem, node):\n", + " s = child.state\n", + " if problem.is_goal(s):\n", + " return child\n", + " if s not in reached:\n", + " reached.add(s)\n", + " frontier.appendleft(child)\n", + " return failure\n", + "\n", + "\n", + "def iterative_deepening_search(problem):\n", + " \"Do depth-limited search with increasing depth limits.\"\n", + " for limit in range(1, sys.maxsize):\n", + " result = depth_limited_search(problem, limit)\n", + " if result != cutoff:\n", + " return result\n", + " \n", + " \n", + "def depth_limited_search(problem, limit=10):\n", + " \"Search deepest nodes in the search tree first.\"\n", + " frontier = LIFOQueue([Node(problem.initial)])\n", + " result = failure\n", + " while frontier:\n", + " node = frontier.pop()\n", + " if problem.is_goal(node.state):\n", + " return node\n", + " elif len(node) >= limit:\n", + " result = cutoff\n", + " elif not is_cycle(node):\n", + " for child in expand(problem, node):\n", + " frontier.append(child)\n", + " return result\n", "\n", - "Now we turn our attention to defining some problem domains.\n", "\n", - "# Water Pouring Problems" + "def depth_first_recursive_search(problem, node=None):\n", + " if node is None: \n", + " node = Node(problem.initial)\n", + " if problem.is_goal(node.state):\n", + " return node\n", + " elif is_cycle(node):\n", + " return failure\n", + " else:\n", + " for child in expand(problem, node):\n", + " result = depth_first_recursive_search(problem, child)\n", + " if result:\n", + " return result\n", + " return failure" + ] + }, + { + "cell_type": "code", + "execution_count": 236, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['N', 'I', 'V', 'U', 'B', 'F', 'S', 'O', 'Z', 'A', 'T', 'L']" + ] + }, + "execution_count": 236, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "path_states(depth_first_recursive_search(r2))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bidirectional Best-First Search" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 412, "metadata": {}, "outputs": [], "source": [ - "class PourProblem(Problem):\n", - " \"\"\"Problem about pouring water between jugs to achieve some water level.\n", - " Each state is a tuples of water levels. In the initialization, provide a tuple of \n", - " sizes, e.g. PourProblem((2, 4, 3), 7, sizes=(8, 16, 32)), \n", - " which means three jugs of sizes (8, 16, 32), initially filled with (2, 4, 3) units of \n", - " water, respectively, and the goal is to get a level of 7 in any one of the jugs.\"\"\"\n", - " \n", - " def actions(self, state):\n", - " \"\"\"The actions executable in this state.\"\"\"\n", - " jugs = range(len(state))\n", - " return ([('Fill', i) for i in jugs if state[i] < self.sizes[i]] +\n", - " [('Dump', i) for i in jugs if state[i]] +\n", - " [('Pour', i, j) for i in jugs if state[i] for j in jugs if i != j])\n", + "def bidirectional_best_first_search(problem_f, f_f, problem_b, f_b, terminated):\n", + " node_f = Node(problem_f.initial)\n", + " node_b = Node(problem_f.goal)\n", + " frontier_f, reached_f = PriorityQueue([node_f], key=f_f), {node_f.state: node_f}\n", + " frontier_b, reached_b = PriorityQueue([node_b], key=f_b), {node_b.state: node_b}\n", + " solution = failure\n", + " while frontier_f and frontier_b and not terminated(solution, frontier_f, frontier_b):\n", + " def S1(node, f):\n", + " return str(int(f(node))) + ' ' + str(path_states(node))\n", + " print('Bi:', S1(frontier_f.top(), f_f), S1(frontier_b.top(), f_b))\n", + " if f_f(frontier_f.top()) < f_b(frontier_b.top()):\n", + " solution = proceed('f', problem_f, frontier_f, reached_f, reached_b, solution)\n", + " else:\n", + " solution = proceed('b', problem_b, frontier_b, reached_b, reached_f, solution)\n", + " return solution\n", "\n", - " def result(self, state, action):\n", - " \"\"\"The state that results from executing this action in this state.\"\"\"\n", - " result = list(state)\n", - " act, i, *_ = action\n", - " if act == 'Fill': # Fill i to capacity\n", - " result[i] = self.sizes[i]\n", - " elif act == 'Dump': # Empty i\n", - " result[i] = 0\n", - " elif act == 'Pour': # Pour from i into j\n", - " j = action[2]\n", - " amount = min(state[i], self.sizes[j] - state[j])\n", - " result[i] -= amount\n", - " result[j] += amount\n", - " return tuple(result)\n", + "def inverse_problem(problem):\n", + " if isinstance(problem, CountCalls):\n", + " return CountCalls(inverse_problem(problem._object))\n", + " else:\n", + " inv = copy.copy(problem)\n", + " inv.initial, inv.goal = inv.goal, inv.initial\n", + " return inv" + ] + }, + { + "cell_type": "code", + "execution_count": 413, + "metadata": {}, + "outputs": [], + "source": [ + "def bidirectional_uniform_cost_search(problem_f):\n", + " def terminated(solution, frontier_f, frontier_b):\n", + " n_f, n_b = frontier_f.top(), frontier_b.top()\n", + " return g(n_f) + g(n_b) > g(solution)\n", + " return bidirectional_best_first_search(problem_f, g, inverse_problem(problem_f), g, terminated)\n", "\n", - " def is_goal(self, state):\n", - " \"\"\"True if the goal level is in any one of the jugs.\"\"\"\n", - " return self.goal in state\n", - " \n", + "def bidirectional_astar_search(problem_f):\n", + " def terminated(solution, frontier_f, frontier_b):\n", + " nf, nb = frontier_f.top(), frontier_b.top()\n", + " return g(nf) + g(nb) > g(solution)\n", + " problem_f = inverse_problem(problem_f)\n", + " return bidirectional_best_first_search(problem_f, lambda n: g(n) + problem_f.h(n),\n", + " problem_b, lambda n: g(n) + problem_b.h(n), \n", + " terminated)\n", + " \n", + "\n", + "def proceed(direction, problem, frontier, reached, reached2, solution):\n", + " node = frontier.pop()\n", + " for child in expand(problem, node):\n", + " s = child.state\n", + " print('proceed', direction, S(child))\n", + " if s not in reached or child.path_cost < reached[s].path_cost:\n", + " frontier.add(child)\n", + " reached[s] = child\n", + " if s in reached2: # Frontiers collide; solution found\n", + " solution2 = (join_nodes(child, reached2[s]) if direction == 'f' else\n", + " join_nodes(reached2[s], child))\n", + " #print('solution', path_states(solution2), solution2.path_cost, \n", + " # path_states(child), path_states(reached2[s]))\n", + " if solution2.path_cost < solution.path_cost:\n", + " solution = solution2\n", + " return solution\n", + "\n", + "S = path_states\n", + "\n", + "#A-S-R + B-P-R => A-S-R-P + B-P\n", + "def join_nodes(nf, nb):\n", + " \"\"\"Join the reverse of the backward node nb to the forward node nf.\"\"\"\n", + " #print('join', S(nf), S(nb))\n", + " join = nf\n", + " while nb.parent is not None:\n", + " cost = join.path_cost + nb.path_cost - nb.parent.path_cost\n", + " join = Node(nb.parent.state, join, nb.action, cost)\n", + " nb = nb.parent\n", + " #print(' now join', S(join), 'with nb', S(nb), 'parent', S(nb.parent))\n", + " return join\n", " \n", - "class GreenPourProblem(PourProblem): \n", - " \"\"\"A PourProblem in which we count not the number of steps, but the amount of water used.\"\"\"\n", - " def step_cost(self, state, action, result=None):\n", - " \"The cost is the amount of water used in a fill.\"\n", - " act, i, *_ = action\n", - " return self.sizes[i] - state[i] if act == 'Fill' else 0" + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#A , B = uniform_cost_search(r1), uniform_cost_search(r2)\n", + "#path_states(A), path_states(B)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#path_states(append_nodes(A, B))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# TODO: RBFS" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Problem Domains\n", + "\n", + "Now we turn our attention to defining some problem domains as subclasses of `Problem`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Route Finding Problems" + "# Route Finding Problems\n", + "\n", + "![](romania.png)\n", + "\n", + "In a `RouteProblem`, the states are names of \"cities\" (or other locations), like `'A'` for Arad. The actions are also city names; `'Z'` is the action to move to city `'Z'`. The layout of cities is given by a separate data structure, a `Map`, which is a graph where there are vertexes (cities), links between vertexes, distances (costs) of those links (if not specified, the default is 1 for every link), and optionally the 2D (x, y) location of each city can be specified. A `RouteProblem` takes this `Map` as input and allows actions to move between linked cities. The default heuristic is straight-line distance to the goal, or is uniformly zero if locations were not given." ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 398, "metadata": {}, "outputs": [], "source": [ "class RouteProblem(Problem):\n", - " \"\"\"A problem to find a route between places on a map.\n", - " Use RouteProblem('S', 'G', map=Map(...)})\"\"\"\n", + " \"\"\"A problem to find a route between locations on a `Map`.\n", + " Create a problem with RouteProblem(start, goal, map=Map(...)}).\n", + " States are the vertexes in the Map graph; actions are destination states.\"\"\"\n", " \n", " def actions(self, state): \n", - " \"\"\"The places neighboring `state`. (Action names are same as place names.)\"\"\"\n", + " \"\"\"The places neighboring `state`.\"\"\"\n", " return self.map.neighbors[state]\n", " \n", " def result(self, state, action):\n", " \"\"\"Go to the `action` place, if the map says that is possible.\"\"\"\n", " return action if action in self.map.neighbors[state] else state\n", " \n", - " def step_cost(self, s, action, s1):\n", - " \"\"\"The actual distance between s and s1.\"\"\"\n", + " def action_cost(self, s, action, s1):\n", + " \"\"\"The distance (cost) to go from s to s1.\"\"\"\n", " return self.map.distances[s, s1]\n", " \n", " def h(self, node):\n", " \"Straight-line distance between state and the goal.\"\n", " locs = self.map.locations\n", - " s, g = locs[node.state], locs[self.goal]\n", - " return abs(complex(*s) - complex(*g))\n", - "\n", + " return straight_line_distance(locs[node.state], locs[self.goal])\n", + " \n", + " \n", + "def straight_line_distance(A, B):\n", + " \"Straight-line distance between two points.\"\n", + " return sum(abs(a - b)**2 for (a, b) in zip(A, B)) ** 0.5" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ "class Map:\n", - " \"\"\"Builds an undirected graph of {vertex: [neighbors...]}, with two additional annotations:\n", - " neighbors:\n", - " distances: a dict of {(v1, v2): number} giving the distance from v1 to v2;\n", - " locations: a dict of {v: (x, y)} giving the (x, y) location of each vertex.\"\"\"\n", - " def __init__(self, distances, locations=()):\n", - " self.neighbors = undirected_graph(distances)\n", - " self.distances = distances\n", + " \"\"\"A map of places in a 2D world: a graph with vertexes and links between them. \n", + " In `Map(links, locations)`, `links` can be either [(v1, v2)...] pairs, \n", + " or a {(v1, v2): distance...} dict. Optional `locations` can be {v1: (x, y)} \n", + " If `directed=False` then for every (v1, v2) link, we add a (v2, v1) link.\"\"\"\n", + "\n", + " def __init__(self, links, locations=None, directed=False):\n", + " if not hasattr(links, 'items'): # Distances are 1 by default\n", + " links = {link: 1 for link in links}\n", + " if not directed:\n", + " for (v1, v2) in list(links):\n", + " links[v2, v1] = links[v1, v2]\n", + " self.distances = links\n", + " self.neighbors = multimap(links)\n", " self.locations = locations or defaultdict(lambda: (0, 0))\n", - " for (v1, v2) in list(distances):\n", - " distances[v2, v1] = distances[v1, v2]\n", - " \n", - "def undirected_graph(pairs):\n", - " \"Given {(v1, v2)...} pairs, return a graph of {v1: [v2,...], v2:[v1,...]}.\"\n", - " graph = defaultdict(tuple)\n", - " for (v1, v2) in pairs:\n", - " graph[v1] += (v2,)\n", - " graph[v2] += (v1,)\n", - " return dict(graph)\n", "\n", - "romania = Map(distances={\n", - " ('O', 'Z'): 71, ('O', 'S'): 151, ('A', 'Z'): 75, ('A', 'S'): 140, ('A', 'T'): 118, \n", - " ('L', 'T'): 111, ('L', 'M'): 70, ('D', 'M'): 75, ('C', 'D'): 120, ('C', 'R'): 146, \n", - " ('C', 'P'): 138, ('R', 'S'): 80, ('F', 'S'): 99, ('B', 'F'): 211, ('B', 'P'): 101, \n", - " ('B', 'G'): 90, ('B', 'U'): 85, ('H', 'U'): 98, ('E', 'H'): 86, ('U', 'V'): 142, \n", - " ('I', 'V'): 92, ('I', 'N'): 87, ('P', 'R'): 97},\n", - " locations=dict(\n", - " A=(91, 492), B=(400, 327), C=(253, 288), D=(165, 299), E=(562, 293), F=(305, 449),\n", - " G=(375, 270), H=(534, 350), I=(473, 506), L=(165, 379), M=(168, 339), N=(406, 537),\n", - " O=(131, 571), P=(320, 368), R=(233, 410), S=(207, 457), T=(94, 410), U=(456, 350),\n", - " V=(509, 444), Z=(108, 531)))" + " \n", + "def multimap(pairs) -> dict:\n", + " \"Given (key, val) pairs, make a dict of {key: [val,...]}.\"\n", + " result = defaultdict(list)\n", + " for key, val in pairs:\n", + " result[key].append(val)\n", + " return result" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 400, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "('Z', 'S', 'T')" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ - "romania.neighbors['A'] # Neighbors of " + "# Some specific RouteProblems\n", + "\n", + "romania = Map(\n", + " {('O', 'Z'): 71, ('O', 'S'): 151, ('A', 'Z'): 75, ('A', 'S'): 140, ('A', 'T'): 118, \n", + " ('L', 'T'): 111, ('L', 'M'): 70, ('D', 'M'): 75, ('C', 'D'): 120, ('C', 'R'): 146, \n", + " ('C', 'P'): 138, ('R', 'S'): 80, ('F', 'S'): 99, ('B', 'F'): 211, ('B', 'P'): 101, \n", + " ('B', 'G'): 90, ('B', 'U'): 85, ('H', 'U'): 98, ('E', 'H'): 86, ('U', 'V'): 142, \n", + " ('I', 'V'): 92, ('I', 'N'): 87, ('P', 'R'): 97},\n", + " {'A': ( 76, 497), 'B': (400, 327), 'C': (246, 285), 'D': (160, 296), 'E': (558, 294), \n", + " 'F': (285, 460), 'G': (368, 257), 'H': (548, 355), 'I': (488, 535), 'L': (162, 379),\n", + " 'M': (160, 343), 'N': (407, 561), 'O': (117, 580), 'P': (311, 372), 'R': (227, 412),\n", + " 'S': (187, 463), 'T': ( 83, 414), 'U': (471, 363), 'V': (535, 473), 'Z': (92, 539)})\n", + "\n", + "\n", + "r0 = RouteProblem('A', 'A', map=romania)\n", + "r1 = RouteProblem('A', 'B', map=romania)\n", + "r2 = RouteProblem('N', 'L', map=romania)\n", + "r3 = RouteProblem('E', 'T', map=romania)\n", + "r4 = RouteProblem('O', 'M', map=romania)" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 232, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "75" + "['A', 'S', 'R', 'P', 'B']" ] }, - "execution_count": 7, + "execution_count": 232, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "romania.distances['A', 'Z']" + "path_states(uniform_cost_search(r1)) # Lowest-cost path from Arab to Bucharest" ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 233, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "(91, 492)" + "['A', 'S', 'F', 'B']" ] }, - "execution_count": 8, + "execution_count": 233, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "romania.locations['A']" + "path_states(breadth_first_search(r1)) # Breadth-first: fewer steps, higher path cost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# 8 Puzzle Problems\n", - "\n" + "# Grid Problems\n", + "\n", + "A `GridProblem` involves navigating on a 2D grid, with some cells being impassible obstacles. By default you can move to any of the eight neighboring cells that are not obstacles (but in a problem instance you can supply a `directions=` keyword to change that). Again, the default heuristic is straight-line distance to the goal. States are `(x, y)` cell locations, such as `(4, 2)`, and actions are `(dx, dy)` cell movements, such as `(0, -1)`, which means leave the `x` coordinate alone, and decrement the `y` coordinate by 1." ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ - "class EightPuzzle(Problem):\n", - " \"\"\" The problem of sliding tiles numbered from 1 to 8 on a 3x3 board,\n", - " where one of the squares is a blank, trying to reach a goal configuration.\n", - " A board state is represented as a tuple of length 9, where the element at index i \n", - " represents the tile number at index i, or 0 if for the empty square, e.g. the goal:\n", - " 1 2 3\n", - " 4 5 6 ==> (1, 2, 3, 4, 5, 6, 7, 8, 0)\n", - " 7 8 _\n", - " \"\"\"\n", + "class GridProblem(Problem):\n", + " \"\"\"Finding a path on a 2D grid with obstacles. Obstacles are (x, y) cells.\"\"\"\n", + "\n", + " def __init__(self, initial=(15, 30), goal=(130, 30), obstacles=(), **kwds):\n", + " Problem.__init__(self, initial=initial, goal=goal, \n", + " obstacles=set(obstacles) - {initial, goal}, **kwds)\n", + "\n", + " directions = [(-1, -1), (0, -1), (1, -1),\n", + " (-1, 0), (1, 0),\n", + " (-1, +1), (0, +1), (1, +1)]\n", " \n", - " def actions(self, state):\n", - " \"\"\"The numbers of the squares that the blank can move to.\"\"\"\n", - " moves = ((1, 3), (0, 2, 4), (1, 5),\n", - " (0, 4, 6), (1, 3, 5, 7), (2, 4, 8),\n", - " (3, 7), (4, 6, 8), (7, 5))\n", - " blank = state.index(0)\n", - " return moves[blank]\n", + " def action_cost(self, s, action, s1): return straight_line_distance(s, s1)\n", " \n", - " def result(self, state, action):\n", - " \"\"\"Swap the blank with the square numbered `action`.\"\"\"\n", - " s = list(state)\n", - " blank = state.index(0)\n", - " s[action], s[blank] = s[blank], s[action]\n", - " return tuple(s)\n", + " def h(self, node): return straight_line_distance(node.state, self.goal)\n", + " \n", + " def result(self, state, action): \n", + " \"Both states and actions are represented by (x, y) pairs.\"\n", + " return action if action not in self.obstacles else state\n", " \n", - " def h(self, node):\n", + " def actions(self, state):\n", + " \"\"\"You can move one cell in any of `directions` to a non-obstacle cell.\"\"\"\n", + " x, y = state\n", + " return {(x + dx, y + dy) for (dx, dy) in self.directions} - self.obstacles\n", + " \n", + "class ErraticVacuum(Problem):\n", + " def actions(self, state): \n", + " return ['suck', 'forward', 'backward']\n", + " \n", + " def results(self, state, action): return self.table[action][state]\n", + " \n", + " table = dict(suck= {1:{5,7}, 2:{4,8}, 3:{7}, 4:{2,4}, 5:{1,5}, 6:{8}, 7:{3,7}, 8:{6,8}},\n", + " forward= {1:{2}, 2:{2}, 3:{4}, 4:{4}, 5:{6}, 6:{6}, 7:{8}, 8:{8}},\n", + " backward={1:{1}, 2:{1}, 3:{3}, 4:{3}, 5:{5}, 6:{5}, 7:{7}, 8:{7}})" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "# Some grid routing problems\n", + "\n", + "# The following can be used to create obstacles:\n", + " \n", + "def random_lines(X=range(15, 130), Y=range(60), N=150, lengths=range(6, 12)):\n", + " \"\"\"The set of cells in N random lines of the given lengths.\"\"\"\n", + " result = set()\n", + " for _ in range(N):\n", + " x, y = random.choice(X), random.choice(Y)\n", + " dx, dy = random.choice(((0, 1), (1, 0)))\n", + " result |= line(x, y, dx, dy, random.choice(lengths))\n", + " return result\n", + "\n", + "def line(x, y, dx, dy, length):\n", + " \"\"\"A line of `length` cells starting at (x, y) and going in (dx, dy) direction.\"\"\"\n", + " return {(x + i * dx, y + i * dy) for i in range(length)}\n", + "\n", + "random.seed(42) # To make this reproducible\n", + "\n", + "frame = line(-10, 20, 0, 1, 20) | line(150, 20, 0, 1, 20)\n", + "cup = line(102, 44, -1, 0, 15) | line(102, 20, -1, 0, 20) | line(102, 44, 0, -1, 24)\n", + "\n", + "d1 = GridProblem(obstacles=random_lines(N=100) | frame)\n", + "d2 = GridProblem(obstacles=random_lines(N=150) | frame)\n", + "d3 = GridProblem(obstacles=random_lines(N=200) | frame)\n", + "d4 = GridProblem(obstacles=random_lines(N=250) | frame)\n", + "d5 = GridProblem(obstacles=random_lines(N=300) | frame)\n", + "d6 = GridProblem(obstacles=cup | frame)\n", + "d7 = GridProblem(obstacles=cup | frame | line(50, 35, 0, -1, 10) | line(60, 37, 0, -1, 17) | line(70, 31, 0, -1, 19))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 8 Puzzle Problems\n", + "\n", + "![](https://ece.uwaterloo.ca/~dwharder/aads/Algorithms/N_puzzles/images/puz3.png)\n", + "\n", + "A sliding tile puzzle where you can swap the blank with an adjacent piece, trying to reach a goal configuration. The cells are numbered 0 to 8, starting at the top left and going row by row left to right. The pieces are numebred 1 to 8, with 0 representing the blank. An action is the cell index number that is to be swapped with the blank (*not* the actual number to be swapped but the index into the state). So the diagram above left is the state `(5, 2, 7, 8, 4, 0, 1, 3, 6)`, and the action is `8`, because the cell number 8 (the 9th or last cell, the `6` in the bottom right) is swapped with the blank.\n", + "\n", + "There are two disjoint sets of states that cannot be reached from each other. One set has an even number of \"inversions\"; the other has an odd number. An inversion is when a piece in the state is larger than a piece that follows it.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 397, + "metadata": {}, + "outputs": [], + "source": [ + "class EightPuzzle(Problem):\n", + " \"\"\" The problem of sliding tiles numbered from 1 to 8 on a 3x3 board,\n", + " where one of the squares is a blank, trying to reach a goal configuration.\n", + " A board state is represented as a tuple of length 9, where the element at index i \n", + " represents the tile number at index i, or 0 if for the empty square, e.g. the goal:\n", + " 1 2 3\n", + " 4 5 6 ==> (1, 2, 3, 4, 5, 6, 7, 8, 0)\n", + " 7 8 _\n", + " \"\"\"\n", + "\n", + " def __init__(self, initial, goal=(0, 1, 2, 3, 4, 5, 6, 7, 8)):\n", + " assert inversions(initial) % 2 == inversions(goal) % 2 # Parity check\n", + " self.initial, self.goal = initial, goal\n", + " \n", + " def actions(self, state):\n", + " \"\"\"The indexes of the squares that the blank can move to.\"\"\"\n", + " moves = ((1, 3), (0, 2, 4), (1, 5),\n", + " (0, 4, 6), (1, 3, 5, 7), (2, 4, 8),\n", + " (3, 7), (4, 6, 8), (7, 5))\n", + " blank = state.index(0)\n", + " return moves[blank]\n", + " \n", + " def result(self, state, action):\n", + " \"\"\"Swap the blank with the square numbered `action`.\"\"\"\n", + " s = list(state)\n", + " blank = state.index(0)\n", + " s[action], s[blank] = s[blank], s[action]\n", + " return tuple(s)\n", + " \n", + " def h1(self, node):\n", " \"\"\"The misplaced tiles heuristic.\"\"\"\n", - " return sum(s != g for (s, g) in zip(node.state, self.goal))\n", + " return hamming_distance(node.state, self.goal)\n", + " \n", + " def h2(self, node):\n", + " \"\"\"The Manhattan heuristic.\"\"\"\n", + " X = (0, 1, 2, 0, 1, 2, 0, 1, 2)\n", + " Y = (0, 0, 0, 1, 1, 1, 2, 2, 2)\n", + " return sum(abs(X[s] - X[g]) + abs(Y[s] - Y[g])\n", + " for (s, g) in zip(node.state, self.goal) if s != 0)\n", + " \n", + " def h(self, node): return h2(self, node)\n", + " \n", + " \n", + "def hamming_distance(A, B):\n", + " \"Number of positions where vectors A and B are different.\"\n", + " return sum(a != b for a, b in zip(A, B))\n", + " \n", + "\n", + "def inversions(board):\n", + " \"The number of times a piece is a smaller number than a following piece.\"\n", + " return sum((a > b and a != 0 and b != 0) for (a, b) in combinations(board, 2))\n", " \n", " \n", "def board8(board, fmt=(3 * '{} {} {}\\n')):\n", " \"A string representing an 8-puzzle board\"\n", - " return fmt.format(*board).replace('0', '_')" + " return fmt.format(*board).replace('0', '_')\n", + "\n", + "class Board(defaultdict):\n", + " empty = '.'\n", + " off = '#'\n", + " def __init__(self, board=None, width=8, height=8, to_move=None, **kwds):\n", + " if board is not None:\n", + " self.update(board)\n", + " self.width, self.height = (board.width, board.height) \n", + " else:\n", + " self.width, self.height = (width, height)\n", + " self.to_move = to_move\n", + "\n", + " def __missing__(self, key):\n", + " x, y = key\n", + " if x < 0 or x >= self.width or y < 0 or y >= self.height:\n", + " return self.off\n", + " else:\n", + " return self.empty\n", + " \n", + " def __repr__(self):\n", + " def row(y): return ' '.join(self[x, y] for x in range(self.width))\n", + " return '\\n'.join(row(y) for y in range(self.height))\n", + " \n", + " def __hash__(self): \n", + " return hash(tuple(sorted(self.items()))) + hash(self.to_move)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "# Some specific EightPuzzle problems\n", + "\n", + "e1 = EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8))\n", + "e2 = EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0))\n", + "e3 = EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6))\n", + "e4 = EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1))\n", + "e5 = EightPuzzle((8, 6, 7, 2, 5, 4, 3, 0, 1))" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1 4 2\n", + "_ 7 5\n", + "3 6 8\n", + "\n", + "1 4 2\n", + "3 7 5\n", + "_ 6 8\n", + "\n", + "1 4 2\n", + "3 7 5\n", + "6 _ 8\n", + "\n", + "1 4 2\n", + "3 _ 5\n", + "6 7 8\n", + "\n", + "1 _ 2\n", + "3 4 5\n", + "6 7 8\n", + "\n", + "_ 1 2\n", + "3 4 5\n", + "6 7 8\n", + "\n" + ] + } + ], + "source": [ + "# Solve an 8 puzzle problem and print out each state\n", + "\n", + "for s in path_states(astar_search(e1)):\n", + " print(board8(s))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Specific Problems and Solutions\n", + "# Water Pouring Problems\n", "\n", - "Now that we have some domains, we can make specific problems in those domains, and solve them:\n", + "![](http://puzzles.nigelcoldwell.co.uk/images/water22.png)\n", "\n", - "\n" + "In a [water pouring problem](https://en.wikipedia.org/wiki/Water_pouring_puzzle) you are given a collection of jugs, each of which has a size (capacity) in, say, litres, and a current level of water (in litres). The goal is to measure out a certain level of water; it can appear in any of the jugs. For example, in the movie *Die Hard 3*, the heroes were faced with the task of making exactly 4 gallons from jugs of size 5 gallons and 3 gallons.) A state is represented by a tuple of current water levels, and the available actions are:\n", + "- `(Fill, i)`: fill the `i`th jug all the way to the top (from a tap with unlimited water).\n", + "- `(Dump, i)`: dump all the water out of the `i`th jug.\n", + "- `(Pour, i, j)`: pour water from the `i`th jug into the `j`th jug until either the jug `i` is empty, or jug `j` is full, whichever comes first." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "class PourProblem(Problem):\n", + " \"\"\"Problem about pouring water between jugs to achieve some water level.\n", + " Each state is a tuples of water levels. In the initialization, also provide a tuple of \n", + " jug sizes, e.g. PourProblem(initial=(0, 0), goal=4, sizes=(5, 3)), \n", + " which means two jugs of sizes 5 and 3, initially both empty, with the goal\n", + " of getting a level of 4 in either jug.\"\"\"\n", + " \n", + " def actions(self, state):\n", + " \"\"\"The actions executable in this state.\"\"\"\n", + " jugs = range(len(state))\n", + " return ([('Fill', i) for i in jugs if state[i] < self.sizes[i]] +\n", + " [('Dump', i) for i in jugs if state[i]] +\n", + " [('Pour', i, j) for i in jugs if state[i] for j in jugs if i != j])\n", + "\n", + " def result(self, state, action):\n", + " \"\"\"The state that results from executing this action in this state.\"\"\"\n", + " result = list(state)\n", + " act, i, *_ = action\n", + " if act == 'Fill': # Fill i to capacity\n", + " result[i] = self.sizes[i]\n", + " elif act == 'Dump': # Empty i\n", + " result[i] = 0\n", + " elif act == 'Pour': # Pour from i into j\n", + " j = action[2]\n", + " amount = min(state[i], self.sizes[j] - state[j])\n", + " result[i] -= amount\n", + " result[j] += amount\n", + " return tuple(result)\n", + "\n", + " def is_goal(self, state):\n", + " \"\"\"True if the goal level is in any one of the jugs.\"\"\"\n", + " return self.goal in state" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In a `GreenPourProblem`, the states and actions are the same, but instead of all actions costing 1, in these problems the cost of an action is the amount of water that flows from the tap. (There is an issue that non-*Fill* actions have 0 cost, which in general can lead to indefinitely long solutions, but in this problem there is a finite number of states, so we're ok.)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "class GreenPourProblem(PourProblem): \n", + " \"\"\"A PourProblem in which the cost is the amount of water used.\"\"\"\n", + " def action_cost(self, s, action, s1):\n", + " \"The cost is the amount of water used.\"\n", + " act, i, *_ = action\n", + " return self.sizes[i] - s[i] if act == 'Fill' else 0" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 27, "metadata": {}, "outputs": [], "source": [ + "# Some specific PourProblems\n", + "\n", "p1 = PourProblem((1, 1, 1), 13, sizes=(2, 16, 32))\n", "p2 = PourProblem((0, 0, 0), 21, sizes=(8, 11, 31))\n", - "p3 = PourProblem((0, 0), 8, sizes=(7,9))\n", + "p3 = PourProblem((0, 0), 8, sizes=(7,9))\n", "p4 = PourProblem((0, 0, 0), 21, sizes=(8, 11, 31))\n", + "p5 = PourProblem((0, 0), 4, sizes=(3, 5))\n", "\n", "g1 = GreenPourProblem((1, 1, 1), 13, sizes=(2, 16, 32))\n", "g2 = GreenPourProblem((0, 0, 0), 21, sizes=(8, 11, 31))\n", - "g3 = GreenPourProblem((0, 0), 8, sizes=(7,9))\n", + "g3 = GreenPourProblem((0, 0), 8, sizes=(7,9))\n", "g4 = GreenPourProblem((0, 0, 0), 21, sizes=(8, 11, 31))\n", - "\n", - "r1 = RouteProblem('A', 'B', map=romania)\n", - "r2 = RouteProblem('N', 'L', map=romania)\n", - "r3 = RouteProblem('E', 'T', map=romania)\n", - "r4 = RouteProblem('O', 'M', map=romania)\n", - "\n", - "goal = (1, 2, 3, 4, 5, 6, 7, 8, 0)\n", - "e1 = EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6), goal)\n", - "e2 = EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8), goal)\n", - "e3 = EightPuzzle((2, 5, 8, 1, 4, 7, 0, 3, 6), goal)\n", - "e4 = EightPuzzle((0, 1, 2, 3, 4, 5, 6, 7, 8), goal)" + "g5 = GreenPourProblem((0, 0), 4, sizes=(3, 5))" ] }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "['A', 'S', 'R', 'P', 'B']" + "([('Fill', 1), ('Pour', 1, 0), ('Dump', 0), ('Pour', 1, 0)],\n", + " [(1, 1, 1), (1, 16, 1), (2, 15, 1), (0, 15, 1), (2, 13, 1)])" ] }, - "execution_count": 11, + "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# Solve a problem (which gives a node) and recover the sequence of states in that node's path\n", - "path_states(astar_search(r1))" + "# Solve the PourProblem of getting 13 in some jug, and show the actions and states\n", + "soln = breadth_first_search(p1)\n", + "path_actions(soln), path_states(soln)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Pancake Sorting Problems\n", + "\n", + "Given a stack of pancakes of various sizes, can you sort them into a stack of decreasing sizes, largest on bottom to smallest on top? You have a spatula with which you can flip the top `i` pancakes. This is shown below for `i = 3`; on the top the spatula grabs the first three pancakes; on the bottom we see them flipped:\n", + "\n", + "\n", + "![](https://upload.wikimedia.org/wikipedia/commons/0/0f/Pancake_sort_operation.png)\n", + "\n", + "How many flips will it take to get the whole stack sorted? This is an interesting [problem](https://en.wikipedia.org/wiki/Pancake_sorting) that Bill Gates has [written about](https://people.eecs.berkeley.edu/~christos/papers/Bounds%20For%20Sorting%20By%20Prefix%20Reversal.pdf). A reasonable heuristic for this problem is the *gap heuristic*: if we look at neighboring pancakes, if, say, the 2nd smallest is next to the 3rd smallest, that's good; they should stay next to each other. But if the 2nd smallest is next to the 4th smallest, that's bad: we will require at least one move to separate them and insert the 3rd smallest between them. The gap heuristic counts the number of neighbors that have a gap like this. In our specification of the problem, pancakes are ranked by size: the smallest is `1`, the 2nd smallest `2`, and so on, and the representation of a state is a tuple of these rankings, from the top to the bottom pancake. Thus the goal state is always `(1, 2, ..., `*n*`)` and the initial (top) state in the diagram above is `(2, 1, 4, 6, 3, 5)`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "class PancakeProblem(Problem):\n", + " \"\"\"A PancakeProblem the goal is always `tuple(range(1, n+1))`, where the\n", + " initial state is a permutation of `range(1, n+1)`. An act is the index `i` \n", + " of the top `i` pancakes that will be flipped.\"\"\"\n", + " \n", + " def __init__(self, initial): \n", + " self.initial, self.goal = tuple(initial), tuple(sorted(initial))\n", + " \n", + " def actions(self, state): return range(2, len(state) + 1)\n", + "\n", + " def result(self, state, i): return state[:i][::-1] + state[i:]\n", + " \n", + " def h(self, node):\n", + " \"The gap heuristic.\"\n", + " s = node.state\n", + " return sum(abs(s[i] - s[i - 1]) > 1 for i in range(1, len(s)))" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "c0 = PancakeProblem((2, 1, 4, 6, 3, 5))\n", + "c1 = PancakeProblem((4, 6, 2, 5, 1, 3))\n", + "c2 = PancakeProblem((1, 3, 7, 5, 2, 6, 4))\n", + "c3 = PancakeProblem((1, 7, 2, 6, 3, 5, 4))\n", + "c4 = PancakeProblem((1, 3, 5, 7, 9, 2, 4, 6, 8))" ] }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "['A', 'S', 'F', 'B']" + "[(2, 1, 4, 6, 3, 5),\n", + " (6, 4, 1, 2, 3, 5),\n", + " (5, 3, 2, 1, 4, 6),\n", + " (4, 1, 2, 3, 5, 6),\n", + " (3, 2, 1, 4, 5, 6),\n", + " (1, 2, 3, 4, 5, 6)]" ] }, - "execution_count": 12, + "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# Breadth first search finds a solution with fewer steps, but in this case higher path cost\n", - "path_states(breadth_first_search(r1))" + "# Solve a pancake problem\n", + "path_states(astar_search(c0))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Jumping Frogs Puzzle\n", + "\n", + "In this puzzle (which also can be played as a two-player game), the initial state is a line of squares, with N pieces of one kind on the left, then one empty square, then N pieces of another kind on the right. The diagram below uses 2 blue toads and 2 red frogs; we will represent this as the string `'LL.RR'`. The goal is to swap the pieces, arriving at `'RR.LL'`. An `'L'` piece moves left-to-right, either sliding one space ahead to an empty space, or two spaces ahead if that space is empty and if there is an `'R'` in between to hop over. The `'R'` pieces move right-to-left analogously. An action will be an `(i, j)` pair meaning to swap the pieces at those indexes. The set of actions for the N = 2 position below is `{(1, 2), (3, 2)}`, meaning either the blue toad in position 1 or the red frog in position 3 can swap places with the blank in position 2.\n", + "\n", + "![](https://upload.wikimedia.org/wikipedia/commons/2/2f/ToadsAndFrogs.png)" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "class JumpingPuzzle(Problem):\n", + " \"\"\"Try to exchange L and R by moving one ahead or hopping two ahead.\"\"\"\n", + " def __init__(self, N=2):\n", + " self.initial = N*'L' + '.' + N*'R'\n", + " self.goal = self.initial[::-1]\n", + " \n", + " def actions(self, state):\n", + " \"\"\"Find all possible move or hop moves.\"\"\"\n", + " idxs = range(len(state))\n", + " return ({(i, i + 1) for i in idxs if state[i:i+2] == 'L.'} # Slide\n", + " |{(i, i + 2) for i in idxs if state[i:i+3] == 'LR.'} # Hop\n", + " |{(i + 1, i) for i in idxs if state[i:i+2] == '.R'} # Slide\n", + " |{(i + 2, i) for i in idxs if state[i:i+3] == '.LR'}) # Hop\n", + "\n", + " def result(self, state, action):\n", + " \"\"\"An action (i, j) means swap the pieces at positions i and j.\"\"\"\n", + " i, j = action\n", + " result = list(state)\n", + " result[i], result[j] = state[j], state[i]\n", + " return ''.join(result)\n", + " \n", + " def h(self, node): return hamming_distance(node.state, self.goal)" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "[(1, 1, 1),\n", - " ('Fill', 1),\n", - " (1, 16, 1),\n", - " ('Pour', 1, 0),\n", - " (2, 15, 1),\n", - " ('Dump', 0),\n", - " (0, 15, 1),\n", - " ('Pour', 1, 0),\n", - " (2, 13, 1)]" + "{(1, 2), (3, 2)}" ] }, - "execution_count": 13, + "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# Solve a problem and recover the path of alternating states and actions\n", - "path(breadth_first_search(p1))" + "JumpingPuzzle(N=2).actions('LL.RR')" ] }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 34, "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "4 _ 2\n", - "5 1 3\n", - "7 8 6\n", - "\n", - "4 1 2\n", - "5 _ 3\n", - "7 8 6\n", - "\n", - "4 1 2\n", - "_ 5 3\n", - "7 8 6\n", - "\n", - "_ 1 2\n", - "4 5 3\n", - "7 8 6\n", - "\n", - "1 _ 2\n", - "4 5 3\n", - "7 8 6\n", - "\n", - "1 2 _\n", - "4 5 3\n", - "7 8 6\n", - "\n", - "1 2 3\n", - "4 5 _\n", - "7 8 6\n", - "\n", - "1 2 3\n", - "4 5 6\n", - "7 8 _\n", - "\n" - ] + "data": { + "text/plain": [ + "['LLL.RRR',\n", + " 'LLLR.RR',\n", + " 'LL.RLRR',\n", + " 'L.LRLRR',\n", + " 'LRL.LRR',\n", + " 'LRLRL.R',\n", + " 'LRLRLR.',\n", + " 'LRLR.RL',\n", + " 'LR.RLRL',\n", + " '.RLRLRL',\n", + " 'R.LRLRL',\n", + " 'RRL.LRL',\n", + " 'RRLRL.L',\n", + " 'RRLR.LL',\n", + " 'RR.RLLL',\n", + " 'RRR.LLL']" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ - "# Solve an 8 puzzle problem and print out each state\n", - "\n", - "for s in path_states(astar_search(e1)):\n", - " print(board8(s))" + "j3 = JumpingPuzzle(N=3)\n", + "j9 = JumpingPuzzle(N=9)\n", + "path_states(astar_search(j3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Reporting Metrics\n", + "# Reporting Summary Statistics on Search Algorithms\n", "\n", - "Now let's gather some metrics on how well each algorithm does. We'll use `CountCalls` to wrap a `Problem` object in such a way that calls to its methods are delegated, but each call increments a counter. Once we've solved the problem, we print out summary statistics." + "Now let's gather some metrics on how well each algorithm does. We'll use `CountCalls` to wrap a `Problem` object in such a way that calls to its methods are delegated to the original problem, but each call increments a counter. Once we've solved the problem, we print out summary statistics." ] }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "class CountCalls:\n", - " \"\"\"Delegate all attribute accesses to the object, and count them in ._counts\"\"\"\n", + " \"\"\"Delegate all attribute gets to the object, and count them in ._counts\"\"\"\n", " def __init__(self, obj):\n", " self._object = obj\n", " self._counts = Counter()\n", " \n", " def __getattr__(self, attr):\n", + " \"Delegate to the original object, after incrementing a counter.\"\n", " self._counts[attr] += 1\n", " return getattr(self._object, attr)\n", + "\n", " \n", - "def report(searchers, problems):\n", - " \"Show metrics for each searcher on each problem.\"\n", + "def report(searchers, problems, verbose=True):\n", + " \"\"\"Show summary statistics for each searcher (and on each problem unless verbose is false).\"\"\"\n", " for searcher in searchers:\n", " print(searcher.__name__ + ':')\n", " total_counts = Counter()\n", " for p in problems:\n", - " prob = CountCalls(p)\n", - " soln = searcher(prob)\n", - " cts = prob._counts; \n", - " cts.update(len=len(path_actions(soln)), cost=soln.path_cost)\n", - " total_counts += cts\n", - " report_line(cts, type(p).__name__)\n", - " report_line(total_counts, 'TOTAL\\n')\n", + " prob = CountCalls(p)\n", + " soln = searcher(prob)\n", + " counts = prob._counts; \n", + " counts.update(actions=len(soln), cost=soln.path_cost)\n", + " total_counts += counts\n", + " if verbose: report_counts(counts, str(p)[:40])\n", + " report_counts(total_counts, 'TOTAL\\n')\n", " \n", - "def report_line(counts, name):\n", - " \"Print one line of the report.\"\n", - " print('{:7,d} Exp |{:7,d} Gen |{:7,d} Goal |{:5.0f} cost |{:3d} len | {}'\n", - " .format(counts['actions'], counts['result'], counts['is_goal'], \n", - " counts['cost'], counts['len'], name))" + "def report_counts(counts, name):\n", + " \"\"\"Print one line of the counts report.\"\"\"\n", + " print('{:9,d} nodes |{:9,d} goal |{:5.0f} cost |{:8,d} actions | {}'.format(\n", + " counts['result'], counts['is_goal'], counts['cost'], counts['actions'], name))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here's a tiny report for uniform-cost search on the jug pouring problems:" ] }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "astar_search:\n", - " 150 Exp | 1,325 Gen | 151 Goal | 4 cost | 4 len | PourProblem\n", - " 378 Exp | 3,381 Gen | 379 Goal | 9 cost | 9 len | PourProblem\n", - " 528 Exp | 4,706 Gen | 530 Goal | 13 cost | 13 len | TOTAL\n", + "uniform_cost_search:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 52 nodes | 14 goal | 6 cost | 19 actions | PourProblem((0, 0), 4)\n", + " 8,122 nodes | 931 goal | 42 cost | 968 actions | TOTAL\n", "\n" ] } ], "source": [ - "# Here's a tiny report\n", - "report([astar_search], [p1, p2])" + "report([uniform_cost_search], [p1, p2, p3, p4, p5])" ] }, { - "cell_type": "markdown", + "cell_type": "code", + "execution_count": 37, "metadata": {}, - "source": [ - "The last line says that, over the two problems `[p1, p2]`, the `astar_search` algorithm expanded 528 nodes, generating 4,706 nodes and doing 530 goal checks. Together, the two solutions had a path cost of 13 and also a total length of 13 (since step cost was 1 in these problems). \n", - "\n", - "Now let's do a bigger report, concentrating first on the easier problems, then harder ones:" - ] + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "uniform_cost_search:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 3,590 nodes | 719 goal | 7 cost | 725 actions | PancakeProblem((4, 6, 2, 5, 1, 3), (1, 2\n", + " 30,204 nodes | 5,035 goal | 8 cost | 5,042 actions | PancakeProblem((1, 3, 7, 5, 2, 6, 4), (1\n", + " 22,068 nodes | 3,679 goal | 6 cost | 3,684 actions | PancakeProblem((1, 7, 2, 6, 3, 5, 4), (1\n", + " 81,467 nodes | 12,321 goal | 174 cost | 12,435 actions | TOTAL\n", + "\n", + "breadth_first_search:\n", + " 596 nodes | 597 goal | 4 cost | 73 actions | PourProblem((1, 1, 1), 13)\n", + " 596 nodes | 597 goal | 15 cost | 73 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 2,618 nodes | 2,619 goal | 9 cost | 302 actions | PourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 32 cost | 302 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 120 nodes | 121 goal | 14 cost | 42 actions | PourProblem((0, 0), 8)\n", + " 120 nodes | 121 goal | 36 cost | 42 actions | GreenPourProblem((0, 0), 8)\n", + " 2,618 nodes | 2,619 goal | 9 cost | 302 actions | PourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 32 cost | 302 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 9 cost | 302 actions | PourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 32 cost | 302 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 2,951 nodes | 2,952 goal | 7 cost | 598 actions | PancakeProblem((4, 6, 2, 5, 1, 3), (1, 2\n", + " 25,945 nodes | 25,946 goal | 8 cost | 4,333 actions | PancakeProblem((1, 3, 7, 5, 2, 6, 4), (1\n", + " 5,975 nodes | 5,976 goal | 6 cost | 1,002 actions | PancakeProblem((1, 7, 2, 6, 3, 5, 4), (1\n", + " 52,011 nodes | 52,024 goal | 213 cost | 7,975 actions | TOTAL\n", + "\n" + ] + } + ], + "source": [ + "report((uniform_cost_search, breadth_first_search), \n", + " (p1, g1, p2, g2, p3, g3, p4, g4, p4, g4, c1, c2, c3)) " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Comparing heuristics\n", + "\n", + "First, let's look at the eight puzzle problems, and compare three different heuristics the Manhattan heuristic, the less informative misplaced tiles heuristic, and the uninformed (i.e. *h* = 0) breadth-first search:" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "breadth_first_search:\n", + " 81 nodes | 82 goal | 5 cost | 35 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 160,948 nodes | 160,949 goal | 22 cost | 59,960 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 218,263 nodes | 218,264 goal | 23 cost | 81,829 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 418,771 nodes | 418,772 goal | 26 cost | 156,533 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 448,667 nodes | 448,668 goal | 27 cost | 167,799 actions | EightPuzzle((8, 6, 7, 2, 5, 4, 3, 0, 1),\n", + "1,246,730 nodes |1,246,735 goal | 103 cost | 466,156 actions | TOTAL\n", + "\n", + "astar_misplaced_tiles:\n", + " 17 nodes | 7 goal | 5 cost | 11 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 23,407 nodes | 8,726 goal | 22 cost | 8,747 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 38,632 nodes | 14,433 goal | 23 cost | 14,455 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 124,324 nodes | 46,553 goal | 26 cost | 46,578 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 156,111 nodes | 58,475 goal | 27 cost | 58,501 actions | EightPuzzle((8, 6, 7, 2, 5, 4, 3, 0, 1),\n", + " 342,491 nodes | 128,194 goal | 103 cost | 128,292 actions | TOTAL\n", + "\n", + "astar_search:\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 3,614 nodes | 1,349 goal | 22 cost | 1,370 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 5,373 nodes | 2,010 goal | 23 cost | 2,032 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 10,832 nodes | 4,086 goal | 26 cost | 4,111 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 11,669 nodes | 4,417 goal | 27 cost | 4,443 actions | EightPuzzle((8, 6, 7, 2, 5, 4, 3, 0, 1),\n", + " 31,503 nodes | 11,868 goal | 103 cost | 11,966 actions | TOTAL\n", + "\n" + ] + } + ], + "source": [ + "def astar_misplaced_tiles(problem): return astar_search(problem, h=problem.h1)\n", + "\n", + "report([breadth_first_search, astar_misplaced_tiles, astar_search], \n", + " [e1, e2, e3, e4, e5])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We see that all three algorithms get cost-optimal solutions, but the better the heuristic, the fewer nodes explored. \n", + "Compared to the uninformed search, the misplaced tiles heuristic explores about 1/4 the number of nodes, and the Manhattan heuristic needs just 2%.\n", + "\n", + "Next, we can show the value of the gap heuristic for pancake sorting problems:" + ] }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 39, "metadata": {}, "outputs": [ { @@ -678,82 +1336,226 @@ "output_type": "stream", "text": [ "astar_search:\n", - " 150 Exp | 1,325 Gen | 151 Goal | 4 cost | 4 len | PourProblem\n", - " 185 Exp | 1,646 Gen | 186 Goal | 10 cost | 12 len | GreenPourProblem\n", - " 5 Exp | 15 Gen | 6 Goal | 418 cost | 4 len | RouteProblem\n", - " 15 Exp | 35 Gen | 16 Goal | 910 cost | 9 len | RouteProblem\n", - " 14 Exp | 34 Gen | 15 Goal | 805 cost | 8 len | RouteProblem\n", - " 9 Exp | 22 Gen | 10 Goal | 445 cost | 5 len | RouteProblem\n", - " 11 Exp | 29 Gen | 12 Goal | 7 cost | 7 len | EightPuzzle\n", - " 389 Exp | 3,106 Gen | 396 Goal | 2599 cost | 49 len | TOTAL\n", + " 1,285 nodes | 258 goal | 7 cost | 264 actions | PancakeProblem((4, 6, 2, 5, 1, 3), (1, 2\n", + " 3,804 nodes | 635 goal | 8 cost | 642 actions | PancakeProblem((1, 3, 7, 5, 2, 6, 4), (1\n", + " 294 nodes | 50 goal | 6 cost | 55 actions | PancakeProblem((1, 7, 2, 6, 3, 5, 4), (1\n", + " 2,256 nodes | 283 goal | 9 cost | 291 actions | PancakeProblem((1, 3, 5, 7, 9, 2, 4, 6, \n", + " 7,639 nodes | 1,226 goal | 30 cost | 1,252 actions | TOTAL\n", "\n", "uniform_cost_search:\n", - " 150 Exp | 1,325 Gen | 151 Goal | 4 cost | 4 len | PourProblem\n", - " 185 Exp | 1,646 Gen | 186 Goal | 10 cost | 12 len | GreenPourProblem\n", - " 13 Exp | 33 Gen | 14 Goal | 418 cost | 4 len | RouteProblem\n", - " 19 Exp | 43 Gen | 20 Goal | 910 cost | 9 len | RouteProblem\n", - " 20 Exp | 45 Gen | 21 Goal | 805 cost | 8 len | RouteProblem\n", - " 12 Exp | 32 Gen | 13 Goal | 445 cost | 5 len | RouteProblem\n", - " 124 Exp | 335 Gen | 125 Goal | 7 cost | 7 len | EightPuzzle\n", - " 523 Exp | 3,459 Gen | 530 Goal | 2599 cost | 49 len | TOTAL\n", + " 3,590 nodes | 719 goal | 7 cost | 725 actions | PancakeProblem((4, 6, 2, 5, 1, 3), (1, 2\n", + " 30,204 nodes | 5,035 goal | 8 cost | 5,042 actions | PancakeProblem((1, 3, 7, 5, 2, 6, 4), (1\n", + " 22,068 nodes | 3,679 goal | 6 cost | 3,684 actions | PancakeProblem((1, 7, 2, 6, 3, 5, 4), (1\n", + "2,271,792 nodes | 283,975 goal | 9 cost | 283,983 actions | PancakeProblem((1, 3, 5, 7, 9, 2, 4, 6, \n", + "2,327,654 nodes | 293,408 goal | 30 cost | 293,434 actions | TOTAL\n", + "\n" + ] + } + ], + "source": [ + "report([astar_search, uniform_cost_search], [c1, c2, c3, c4])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We need to explore 300 times more nodes without the heuristic.\n", + "\n", + "# Comparing graph search and tree search\n", + "\n", + "Keeping the *reached* table in `best_first_search` allows us to do a graph search, where we notice when we reach a state by two different paths, rather than a tree search, where we have duplicated effort. The *reached* table consumes space and also saves time. How much time? In part it depends on how good the heuristics are at focusing the search. Below we show that on some pancake and eight puzzle problems, the tree search expands roughly twice as many nodes (and thus takes roughly twice as much time):" + ] + }, + { + "cell_type": "code", + "execution_count": 188, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "astar_search:\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 3,614 nodes | 1,349 goal | 22 cost | 1,370 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 5,373 nodes | 2,010 goal | 23 cost | 2,032 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 10,832 nodes | 4,086 goal | 26 cost | 4,111 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 15 nodes | 6 goal | 418 cost | 9 actions | RouteProblem('A', 'B')\n", + " 34 nodes | 15 goal | 910 cost | 23 actions | RouteProblem('N', 'L')\n", + " 33 nodes | 14 goal | 805 cost | 21 actions | RouteProblem('E', 'T')\n", + " 20 nodes | 9 goal | 445 cost | 13 actions | RouteProblem('O', 'M')\n", + " 19,936 nodes | 7,495 goal | 2654 cost | 7,589 actions | TOTAL\n", "\n", - "breadth_first_search:\n", - " 127 Exp | 1,116 Gen | 128 Goal | 4 cost | 4 len | PourProblem\n", - " 127 Exp | 1,116 Gen | 128 Goal | 15 cost | 4 len | GreenPourProblem\n", - " 11 Exp | 29 Gen | 12 Goal | 450 cost | 3 len | RouteProblem\n", - " 20 Exp | 45 Gen | 21 Goal | 1085 cost | 9 len | RouteProblem\n", - " 18 Exp | 41 Gen | 19 Goal | 837 cost | 7 len | RouteProblem\n", - " 15 Exp | 38 Gen | 16 Goal | 445 cost | 5 len | RouteProblem\n", - " 143 Exp | 397 Gen | 144 Goal | 7 cost | 7 len | EightPuzzle\n", - " 461 Exp | 2,782 Gen | 468 Goal | 2843 cost | 39 len | TOTAL\n", + "astar_tree_search:\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 5,384 nodes | 2,000 goal | 22 cost | 2,021 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 9,116 nodes | 3,404 goal | 23 cost | 3,426 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 19,084 nodes | 7,185 goal | 26 cost | 7,210 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 15 nodes | 6 goal | 418 cost | 9 actions | RouteProblem('A', 'B')\n", + " 47 nodes | 19 goal | 910 cost | 27 actions | RouteProblem('N', 'L')\n", + " 46 nodes | 18 goal | 805 cost | 25 actions | RouteProblem('E', 'T')\n", + " 24 nodes | 10 goal | 445 cost | 14 actions | RouteProblem('O', 'M')\n", + " 33,731 nodes | 12,648 goal | 2654 cost | 12,742 actions | TOTAL\n", + "\n" + ] + } + ], + "source": [ + "report([astar_search, astar_tree_search], [e1, e2, e3, e4, r1, r2, r3, r4])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Comparing different weighted search values\n", + "\n", + "Below we report on problems using these four algorithms:\n", + "\n", + "|Algorithm|*f*|Optimality|\n", + "|:---------|---:|:----------:|\n", + "|Greedy best-first search | *f = h*|nonoptimal|\n", + "|Extra weighted A* search | *f = g + 2 × h*|nonoptimal|\n", + "|Weighted A* search | *f = g + 1.4 × h*|nonoptimal|\n", + "|A* search | *f = g + h*|optimal|\n", + "|Uniform-cost search | *f = g*|optimal|\n", + "\n", + "We will see that greedy best-first search (which ranks nodes solely by the heuristic) explores the fewest number of nodes, but has the highest path costs. Weighted A* search explores twice as many nodes (on this problem set) but gets 10% better path costs. A* is optimal, but explores more nodes, and uniform-cost is also optimal, but explores an order of magnitude more nodes." + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "greedy_bfs:\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 29 nodes | 12 goal | 910 cost | 20 actions | RouteProblem('N', 'L')\n", + " 19 nodes | 8 goal | 837 cost | 14 actions | RouteProblem('E', 'T')\n", + " 14 nodes | 6 goal | 572 cost | 10 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 909 nodes | 138 goal | 136 cost | 258 actions | GridProblem((15, 30), (130, 30))\n", + " 974 nodes | 147 goal | 152 cost | 277 actions | GridProblem((15, 30), (130, 30))\n", + " 5,146 nodes | 4,984 goal | 99 cost | 5,082 actions | JumpingPuzzle('LLLLLLLLL.RRRRRRRRR', 'RR\n", + " 1,569 nodes | 568 goal | 58 cost | 625 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 1,424 nodes | 257 goal | 164 cost | 406 actions | GridProblem((15, 30), (130, 30))\n", + " 1,899 nodes | 342 goal | 153 cost | 470 actions | GridProblem((15, 30), (130, 30))\n", + " 18,239 nodes | 2,439 goal | 134 cost | 2,564 actions | GridProblem((15, 30), (130, 30))\n", + " 18,329 nodes | 2,460 goal | 152 cost | 2,594 actions | GridProblem((15, 30), (130, 30))\n", + " 287 nodes | 109 goal | 33 cost | 141 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 1,128 nodes | 408 goal | 46 cost | 453 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 49,990 nodes | 11,889 goal | 3901 cost | 12,930 actions | TOTAL\n", "\n", - "iterative_deepening_search:\n", - " 981 Exp | 7,610 Gen | 7,610 Goal | 4 cost | 4 len | PourProblem\n", - " 981 Exp | 7,610 Gen | 7,610 Goal | 15 cost | 4 len | GreenPourProblem\n", - " 10 Exp | 27 Gen | 27 Goal | 450 cost | 3 len | RouteProblem\n", - " 547 Exp | 1,308 Gen | 1,308 Goal | 910 cost | 9 len | RouteProblem\n", - " 172 Exp | 406 Gen | 406 Goal | 837 cost | 7 len | RouteProblem\n", - " 63 Exp | 175 Gen | 175 Goal | 572 cost | 5 len | RouteProblem\n", - " 742 Exp | 2,108 Gen | 2,108 Goal | 7 cost | 7 len | EightPuzzle\n", - " 3,496 Exp | 19,244 Gen | 19,244 Goal | 2795 cost | 39 len | TOTAL\n", + "extra_weighted_astar_search:\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 29 nodes | 12 goal | 910 cost | 20 actions | RouteProblem('N', 'L')\n", + " 23 nodes | 9 goal | 805 cost | 16 actions | RouteProblem('E', 'T')\n", + " 18 nodes | 8 goal | 445 cost | 12 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 1,575 nodes | 239 goal | 136 cost | 357 actions | GridProblem((15, 30), (130, 30))\n", + " 1,384 nodes | 231 goal | 133 cost | 349 actions | GridProblem((15, 30), (130, 30))\n", + " 10,990 nodes | 10,660 goal | 99 cost | 10,758 actions | JumpingPuzzle('LLLLLLLLL.RRRRRRRRR', 'RR\n", + " 1,720 nodes | 633 goal | 24 cost | 656 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 9,282 nodes | 1,412 goal | 163 cost | 1,551 actions | GridProblem((15, 30), (130, 30))\n", + " 1,354 nodes | 228 goal | 134 cost | 345 actions | GridProblem((15, 30), (130, 30))\n", + " 16,024 nodes | 2,098 goal | 129 cost | 2,214 actions | GridProblem((15, 30), (130, 30))\n", + " 16,950 nodes | 2,237 goal | 140 cost | 2,359 actions | GridProblem((15, 30), (130, 30))\n", + " 1,908 nodes | 709 goal | 25 cost | 733 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 1,312 nodes | 489 goal | 30 cost | 518 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 62,593 nodes | 18,976 goal | 3628 cost | 19,904 actions | TOTAL\n", "\n", - "depth_limited_search:\n", - " 472 Exp | 3,522 Gen | 3,522 Goal | 6 cost | 6 len | PourProblem\n", - " 472 Exp | 3,522 Gen | 3,522 Goal | 16 cost | 6 len | GreenPourProblem\n", - " 29 Exp | 69 Gen | 69 Goal | 686 cost | 5 len | RouteProblem\n", - " 28 Exp | 59 Gen | 59 Goal | inf cost | 0 len | RouteProblem\n", - " 40 Exp | 100 Gen | 100 Goal | inf cost | 0 len | RouteProblem\n", - " 47 Exp | 139 Gen | 139 Goal | 661 cost | 6 len | RouteProblem\n", - " 292 Exp | 803 Gen | 803 Goal | inf cost | 0 len | EightPuzzle\n", - " 1,380 Exp | 8,214 Gen | 8,214 Goal | inf cost | 23 len | TOTAL\n", + "weighted_astar_search:\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 32 nodes | 14 goal | 910 cost | 22 actions | RouteProblem('N', 'L')\n", + " 29 nodes | 12 goal | 805 cost | 19 actions | RouteProblem('E', 'T')\n", + " 18 nodes | 8 goal | 445 cost | 12 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 1,631 nodes | 236 goal | 128 cost | 350 actions | GridProblem((15, 30), (130, 30))\n", + " 1,706 nodes | 275 goal | 131 cost | 389 actions | GridProblem((15, 30), (130, 30))\n", + " 10,990 nodes | 10,660 goal | 99 cost | 10,758 actions | JumpingPuzzle('LLLLLLLLL.RRRRRRRRR', 'RR\n", + " 2,082 nodes | 771 goal | 22 cost | 792 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 8,385 nodes | 1,266 goal | 154 cost | 1,396 actions | GridProblem((15, 30), (130, 30))\n", + " 1,400 nodes | 229 goal | 133 cost | 344 actions | GridProblem((15, 30), (130, 30))\n", + " 12,122 nodes | 1,572 goal | 124 cost | 1,686 actions | GridProblem((15, 30), (130, 30))\n", + " 24,129 nodes | 3,141 goal | 127 cost | 3,255 actions | GridProblem((15, 30), (130, 30))\n", + " 3,960 nodes | 1,475 goal | 25 cost | 1,499 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 1,992 nodes | 748 goal | 26 cost | 773 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 68,500 nodes | 20,418 goal | 3585 cost | 21,311 actions | TOTAL\n", + "\n", + "astar_search:\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 15 nodes | 6 goal | 418 cost | 9 actions | RouteProblem('A', 'B')\n", + " 34 nodes | 15 goal | 910 cost | 23 actions | RouteProblem('N', 'L')\n", + " 33 nodes | 14 goal | 805 cost | 21 actions | RouteProblem('E', 'T')\n", + " 20 nodes | 9 goal | 445 cost | 13 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 26,711 nodes | 3,620 goal | 127 cost | 3,734 actions | GridProblem((15, 30), (130, 30))\n", + " 12,932 nodes | 1,822 goal | 124 cost | 1,936 actions | GridProblem((15, 30), (130, 30))\n", + " 10,991 nodes | 10,661 goal | 99 cost | 10,759 actions | JumpingPuzzle('LLLLLLLLL.RRRRRRRRR', 'RR\n", + " 3,614 nodes | 1,349 goal | 22 cost | 1,370 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 62,509 nodes | 8,729 goal | 154 cost | 8,859 actions | GridProblem((15, 30), (130, 30))\n", + " 15,190 nodes | 2,276 goal | 133 cost | 2,391 actions | GridProblem((15, 30), (130, 30))\n", + " 25,303 nodes | 3,196 goal | 124 cost | 3,310 actions | GridProblem((15, 30), (130, 30))\n", + " 32,572 nodes | 4,149 goal | 127 cost | 4,263 actions | GridProblem((15, 30), (130, 30))\n", + " 5,373 nodes | 2,010 goal | 23 cost | 2,032 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 10,832 nodes | 4,086 goal | 26 cost | 4,111 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + " 206,144 nodes | 41,949 goal | 3543 cost | 42,841 actions | TOTAL\n", + "\n", + "uniform_cost_search:\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 30 nodes | 13 goal | 418 cost | 16 actions | RouteProblem('A', 'B')\n", + " 42 nodes | 19 goal | 910 cost | 27 actions | RouteProblem('N', 'L')\n", + " 44 nodes | 20 goal | 805 cost | 27 actions | RouteProblem('E', 'T')\n", + " 30 nodes | 12 goal | 445 cost | 16 actions | RouteProblem('O', 'M')\n", + " 124 nodes | 46 goal | 5 cost | 50 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 355,452 nodes | 44,984 goal | 127 cost | 45,098 actions | GridProblem((15, 30), (130, 30))\n", + " 326,962 nodes | 41,650 goal | 124 cost | 41,764 actions | GridProblem((15, 30), (130, 30))\n", + " 10,992 nodes | 10,662 goal | 99 cost | 10,760 actions | JumpingPuzzle('LLLLLLLLL.RRRRRRRRR', 'RR\n", + " 214,952 nodes | 79,187 goal | 22 cost | 79,208 actions | EightPuzzle((1, 2, 3, 4, 5, 6, 7, 8, 0),\n", + " 558,084 nodes | 70,738 goal | 154 cost | 70,868 actions | GridProblem((15, 30), (130, 30))\n", + " 370,370 nodes | 47,243 goal | 133 cost | 47,358 actions | GridProblem((15, 30), (130, 30))\n", + " 349,062 nodes | 43,693 goal | 124 cost | 43,807 actions | GridProblem((15, 30), (130, 30))\n", + " 366,996 nodes | 45,970 goal | 127 cost | 46,084 actions | GridProblem((15, 30), (130, 30))\n", + " 300,925 nodes | 112,082 goal | 23 cost | 112,104 actions | EightPuzzle((4, 0, 2, 5, 1, 3, 7, 8, 6),\n", + " 457,766 nodes | 171,571 goal | 26 cost | 171,596 actions | EightPuzzle((7, 2, 4, 5, 0, 6, 8, 3, 1),\n", + "3,311,831 nodes | 667,891 goal | 3543 cost | 668,783 actions | TOTAL\n", "\n" ] } ], "source": [ - "easy = (p1, g1, r1, r2, r3, r4, e1)\n", - "hard = (g2, p2, g3, p3, g4, p4, e2, e3, e4)\n", - "\n", - "report((astar_search, uniform_cost_search, breadth_first_search, \n", - " iterative_deepening_search, depth_limited_search), easy)" + "def extra_weighted_astar_search(problem): return weighted_astar_search(problem, weight=2)\n", + " \n", + "report((greedy_bfs, extra_weighted_astar_search, weighted_astar_search, astar_search, uniform_cost_search), \n", + " (r0, r1, r2, r3, r4, e1, d1, d2, j9, e2, d3, d4, d6, d7, e3, e4))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "One thing to notice: on three of the problems, `depth_limited_search` had a path cost of `inf`, meaning that the search was cut off, so it reported an infinite cost.\n", + "We see that greedy search expands the fewest nodes, but has the highest path costs. In contrast, A\\* gets optimal path costs, but expands 4 or 5 times more nodes. Weighted A* is a good compromise, using half the compute time as A\\*, and achieving path costs within 1% or 2% of optimal. Uniform-cost is optimal, but is an order of magnitude slower than A\\*.\n", "\n", - "If we look at the whole `cost` column, we see that the optimal algorithms, `astar_search` and `uniform_cost_search`, give the best results, while `breadth_first_search` and `iterative_deepening_search` have non-optimal costs on some problems, because they find a solution with the minimal number of steps, but not the minimal path cost. We see that `astar_search` has fewer expansions, generated nodes, and goal tests that `uniform_cost_search`, which means the heuristic helps (if only by 10% or so).\n", + "# Comparing many search algorithms\n", "\n", - "Next I'll try some harder problems; I won't even try the tree search algorithms on these problems; too many redundant paths." + "Finally, we compare a host of algorihms (even the slow ones) on some of the easier problems:" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 42, "metadata": { - "scrolled": true + "scrolled": false }, "outputs": [ { @@ -761,54 +1563,1069 @@ "output_type": "stream", "text": [ "astar_search:\n", - " 451 Exp | 4,048 Gen | 452 Goal | 21 cost | 19 len | GreenPourProblem\n", - " 378 Exp | 3,381 Gen | 379 Goal | 9 cost | 9 len | PourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 35 cost | 16 len | GreenPourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 14 cost | 14 len | PourProblem\n", - " 451 Exp | 4,048 Gen | 452 Goal | 21 cost | 19 len | GreenPourProblem\n", - " 378 Exp | 3,381 Gen | 379 Goal | 9 cost | 9 len | PourProblem\n", - " 10,338 Exp | 27,461 Gen | 10,339 Goal | 23 cost | 23 len | EightPuzzle\n", - " 14,119 Exp | 37,562 Gen | 14,120 Goal | 24 cost | 24 len | EightPuzzle\n", - " 5,989 Exp | 15,951 Gen | 5,990 Goal | 22 cost | 22 len | EightPuzzle\n", - " 32,164 Exp | 96,084 Gen | 32,173 Goal | 178 cost |155 len | TOTAL\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 15 nodes | 6 goal | 418 cost | 9 actions | RouteProblem('A', 'B')\n", + " 34 nodes | 15 goal | 910 cost | 23 actions | RouteProblem('N', 'L')\n", + " 33 nodes | 14 goal | 805 cost | 21 actions | RouteProblem('E', 'T')\n", + " 20 nodes | 9 goal | 445 cost | 13 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 18,151 nodes | 2,096 goal | 2706 cost | 2,200 actions | TOTAL\n", "\n", "uniform_cost_search:\n", - " 451 Exp | 4,048 Gen | 452 Goal | 21 cost | 19 len | GreenPourProblem\n", - " 378 Exp | 3,381 Gen | 379 Goal | 9 cost | 9 len | PourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 35 cost | 16 len | GreenPourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 14 cost | 14 len | PourProblem\n", - " 451 Exp | 4,048 Gen | 452 Goal | 21 cost | 19 len | GreenPourProblem\n", - " 378 Exp | 3,381 Gen | 379 Goal | 9 cost | 9 len | PourProblem\n", - "103,882 Exp |279,376 Gen |103,883 Goal | 23 cost | 23 len | EightPuzzle\n", - "121,025 Exp |325,288 Gen |121,026 Goal | 24 cost | 24 len | EightPuzzle\n", - " 76,710 Exp |206,476 Gen | 76,711 Goal | 22 cost | 22 len | EightPuzzle\n", - "303,335 Exp |826,250 Gen |303,344 Goal | 178 cost |155 len | TOTAL\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 30 nodes | 13 goal | 418 cost | 16 actions | RouteProblem('A', 'B')\n", + " 42 nodes | 19 goal | 910 cost | 27 actions | RouteProblem('N', 'L')\n", + " 44 nodes | 20 goal | 805 cost | 27 actions | RouteProblem('E', 'T')\n", + " 30 nodes | 12 goal | 445 cost | 16 actions | RouteProblem('O', 'M')\n", + " 124 nodes | 46 goal | 5 cost | 50 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 18,304 nodes | 2,156 goal | 2706 cost | 2,260 actions | TOTAL\n", "\n", "breadth_first_search:\n", - " 422 Exp | 3,840 Gen | 423 Goal | 32 cost | 9 len | GreenPourProblem\n", - " 422 Exp | 3,840 Gen | 423 Goal | 9 cost | 9 len | PourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 36 cost | 14 len | GreenPourProblem\n", - " 30 Exp | 126 Gen | 31 Goal | 14 cost | 14 len | PourProblem\n", - " 422 Exp | 3,840 Gen | 423 Goal | 32 cost | 9 len | GreenPourProblem\n", - " 422 Exp | 3,840 Gen | 423 Goal | 9 cost | 9 len | PourProblem\n", - "118,340 Exp |316,026 Gen |118,341 Goal | 23 cost | 23 len | EightPuzzle\n", - "131,021 Exp |350,990 Gen |131,022 Goal | 24 cost | 24 len | EightPuzzle\n", - " 80,968 Exp |218,918 Gen | 80,969 Goal | 22 cost | 22 len | EightPuzzle\n", - "332,077 Exp |901,546 Gen |332,086 Goal | 201 cost |133 len | TOTAL\n", + " 596 nodes | 597 goal | 4 cost | 73 actions | PourProblem((1, 1, 1), 13)\n", + " 596 nodes | 597 goal | 15 cost | 73 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 2,618 nodes | 2,619 goal | 9 cost | 302 actions | PourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 32 cost | 302 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 120 nodes | 121 goal | 14 cost | 42 actions | PourProblem((0, 0), 8)\n", + " 120 nodes | 121 goal | 36 cost | 42 actions | GreenPourProblem((0, 0), 8)\n", + " 2,618 nodes | 2,619 goal | 9 cost | 302 actions | PourProblem((0, 0, 0), 21)\n", + " 2,618 nodes | 2,619 goal | 32 cost | 302 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 18 nodes | 19 goal | 450 cost | 10 actions | RouteProblem('A', 'B')\n", + " 42 nodes | 43 goal | 1085 cost | 27 actions | RouteProblem('N', 'L')\n", + " 36 nodes | 37 goal | 837 cost | 22 actions | RouteProblem('E', 'T')\n", + " 30 nodes | 31 goal | 445 cost | 16 actions | RouteProblem('O', 'M')\n", + " 81 nodes | 82 goal | 5 cost | 35 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 12,111 nodes | 12,125 goal | 2973 cost | 1,548 actions | TOTAL\n", + "\n", + "breadth_first_bfs:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,062 nodes | 124 goal | 15 cost | 127 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 3,757 nodes | 420 goal | 24 cost | 428 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 36 cost | 43 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 3,757 nodes | 420 goal | 24 cost | 428 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 28 nodes | 12 goal | 450 cost | 14 actions | RouteProblem('A', 'B')\n", + " 55 nodes | 24 goal | 910 cost | 32 actions | RouteProblem('N', 'L')\n", + " 51 nodes | 22 goal | 837 cost | 28 actions | RouteProblem('E', 'T')\n", + " 40 nodes | 16 goal | 445 cost | 20 actions | RouteProblem('O', 'M')\n", + " 124 nodes | 46 goal | 5 cost | 50 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 17,068 nodes | 2,032 goal | 2782 cost | 2,119 actions | TOTAL\n", + "\n", + "iterative_deepening_search:\n", + " 6,133 nodes | 6,118 goal | 4 cost | 822 actions | PourProblem((1, 1, 1), 13)\n", + " 6,133 nodes | 6,118 goal | 15 cost | 822 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 288,706 nodes | 288,675 goal | 9 cost | 36,962 actions | PourProblem((0, 0, 0), 21)\n", + " 288,706 nodes | 288,675 goal | 62 cost | 36,962 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 3,840 nodes | 3,824 goal | 14 cost | 949 actions | PourProblem((0, 0), 8)\n", + " 3,840 nodes | 3,824 goal | 36 cost | 949 actions | GreenPourProblem((0, 0), 8)\n", + " 288,706 nodes | 288,675 goal | 9 cost | 36,962 actions | PourProblem((0, 0, 0), 21)\n", + " 288,706 nodes | 288,675 goal | 62 cost | 36,962 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 27 nodes | 25 goal | 450 cost | 13 actions | RouteProblem('A', 'B')\n", + " 167 nodes | 173 goal | 910 cost | 82 actions | RouteProblem('N', 'L')\n", + " 117 nodes | 120 goal | 837 cost | 56 actions | RouteProblem('E', 'T')\n", + " 108 nodes | 109 goal | 572 cost | 44 actions | RouteProblem('O', 'M')\n", + " 116 nodes | 118 goal | 5 cost | 47 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + "1,175,305 nodes |1,175,130 goal | 2985 cost | 151,632 actions | TOTAL\n", + "\n", + "depth_limited_search:\n", + " 4,433 nodes | 4,374 goal | 10 cost | 627 actions | PourProblem((1, 1, 1), 13)\n", + " 4,433 nodes | 4,374 goal | 30 cost | 627 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 37,149 nodes | 37,106 goal | 10 cost | 4,753 actions | PourProblem((0, 0, 0), 21)\n", + " 37,149 nodes | 37,106 goal | 54 cost | 4,753 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 452 nodes | 453 goal | inf cost | 110 actions | PourProblem((0, 0), 8)\n", + " 452 nodes | 453 goal | inf cost | 110 actions | GreenPourProblem((0, 0), 8)\n", + " 37,149 nodes | 37,106 goal | 10 cost | 4,753 actions | PourProblem((0, 0, 0), 21)\n", + " 37,149 nodes | 37,106 goal | 54 cost | 4,753 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 17 nodes | 8 goal | 733 cost | 14 actions | RouteProblem('A', 'B')\n", + " 40 nodes | 38 goal | 910 cost | 26 actions | RouteProblem('N', 'L')\n", + " 29 nodes | 23 goal | 992 cost | 20 actions | RouteProblem('E', 'T')\n", + " 35 nodes | 29 goal | 895 cost | 22 actions | RouteProblem('O', 'M')\n", + " 351 nodes | 349 goal | 5 cost | 138 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 158,838 nodes | 158,526 goal | inf cost | 20,706 actions | TOTAL\n", + "\n", + "greedy_bfs:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 29 nodes | 12 goal | 910 cost | 20 actions | RouteProblem('N', 'L')\n", + " 19 nodes | 8 goal | 837 cost | 14 actions | RouteProblem('E', 'T')\n", + " 14 nodes | 6 goal | 572 cost | 10 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 18,120 nodes | 2,082 goal | 2897 cost | 2,184 actions | TOTAL\n", + "\n", + "weighted_astar_search:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 32 nodes | 14 goal | 910 cost | 22 actions | RouteProblem('N', 'L')\n", + " 29 nodes | 12 goal | 805 cost | 19 actions | RouteProblem('E', 'T')\n", + " 18 nodes | 8 goal | 445 cost | 12 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 18,137 nodes | 2,090 goal | 2738 cost | 2,193 actions | TOTAL\n", + "\n", + "extra_weighted_astar_search:\n", + " 948 nodes | 109 goal | 4 cost | 112 actions | PourProblem((1, 1, 1), 13)\n", + " 1,696 nodes | 190 goal | 10 cost | 204 actions | GreenPourProblem((1, 1, 1), 13)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 124 nodes | 30 goal | 14 cost | 43 actions | PourProblem((0, 0), 8)\n", + " 124 nodes | 30 goal | 35 cost | 45 actions | GreenPourProblem((0, 0), 8)\n", + " 3,499 nodes | 389 goal | 9 cost | 397 actions | PourProblem((0, 0, 0), 21)\n", + " 4,072 nodes | 454 goal | 21 cost | 463 actions | GreenPourProblem((0, 0, 0), 21)\n", + " 0 nodes | 1 goal | 0 cost | 0 actions | RouteProblem('A', 'A')\n", + " 9 nodes | 4 goal | 450 cost | 6 actions | RouteProblem('A', 'B')\n", + " 29 nodes | 12 goal | 910 cost | 20 actions | RouteProblem('N', 'L')\n", + " 23 nodes | 9 goal | 805 cost | 16 actions | RouteProblem('E', 'T')\n", + " 18 nodes | 8 goal | 445 cost | 12 actions | RouteProblem('O', 'M')\n", + " 15 nodes | 6 goal | 5 cost | 10 actions | EightPuzzle((1, 4, 2, 0, 7, 5, 3, 6, 8),\n", + " 18,128 nodes | 2,085 goal | 2738 cost | 2,188 actions | TOTAL\n", "\n" ] } ], "source": [ - "report((astar_search, uniform_cost_search, breadth_first_search), hard)" + "report((astar_search, uniform_cost_search, breadth_first_search, breadth_first_bfs, \n", + " iterative_deepening_search, depth_limited_search, greedy_bfs, \n", + " weighted_astar_search, extra_weighted_astar_search), \n", + " (p1, g1, p2, g2, p3, g3, p4, g4, r0, r1, r2, r3, r4, e1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This time we see that A* is an order of magnitude more efficient than the two uninformed algorithm. Note that again, uniform cost is optimal, but breadth-first is not: it optimized for path length, not path cost." + "This confirms some of the things we already knew: A* and uniform-cost search are optimal, but the others are not. A* explores fewer nodes than uniform-cost. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Visualizing Reached States\n", + "\n", + "I would like to draw a picture of the state space, marking the states that have been reached by the search.\n", + "Unfortunately, the *reached* variable is inaccessible inside `best_first_search`, so I will define a new version of `best_first_search` that is identical except that it declares *reached* to be `global`. I can then define `plot_grid_problem` to plot the obstacles of a `GridProblem`, along with the initial and goal states, the solution path, and the states reached during a search." + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [], + "source": [ + "def best_first_search(problem, f):\n", + " \"Search nodes with minimum f(node) value first.\"\n", + " global reached # <<<<<<<<<<< Only change here\n", + " node = Node(problem.initial)\n", + " frontier = PriorityQueue([node], key=f)\n", + " reached = {problem.initial: node}\n", + " while frontier:\n", + " node = frontier.pop()\n", + " if problem.is_goal(node.state):\n", + " return node\n", + " for child in expand(problem, node):\n", + " s = child.state\n", + " if s not in reached or child.path_cost < reached[s].path_cost:\n", + " reached[s] = child\n", + " frontier.add(child)\n", + " return failure\n", + "\n", + "\n", + "def plot_grid_problem(grid, solution, reached=(), title='Search', show=True):\n", + " \"Use matplotlib to plot the grid, obstacles, solution, and reached.\"\n", + " reached = list(reached)\n", + " plt.figure(figsize=(16, 10))\n", + " plt.axis('off'); plt.axis('equal')\n", + " plt.scatter(*transpose(grid.obstacles), marker='s', color='darkgrey')\n", + " plt.scatter(*transpose(reached), 1**2, marker='.', c='blue')\n", + " plt.scatter(*transpose(path_states(solution)), marker='s', c='blue')\n", + " plt.scatter(*transpose([grid.initial]), 9**2, marker='D', c='green')\n", + " plt.scatter(*transpose([grid.goal]), 9**2, marker='8', c='red')\n", + " if show: plt.show()\n", + " print('{} {} search: {:.1f} path cost, {:,d} states reached'\n", + " .format(' ' * 10, title, solution.path_cost, len(reached)))\n", + " \n", + "def plots(grid, weights=(1.4, 2)): \n", + " \"\"\"Plot the results of 4 heuristic search algorithms for this grid.\"\"\"\n", + " solution = astar_search(grid)\n", + " plot_grid_problem(grid, solution, reached, 'A* search')\n", + " for weight in weights:\n", + " solution = weighted_astar_search(grid, weight=weight)\n", + " plot_grid_problem(grid, solution, reached, '(b) Weighted ({}) A* search'.format(weight))\n", + " solution = greedy_bfs(grid)\n", + " plot_grid_problem(grid, solution, reached, 'Greedy best-first search')\n", + " \n", + "def transpose(matrix): return list(zip(*matrix))" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3b/PJFt6H/bTw11xTGsvocR2RFiZAgrv3dAADRJWaDhYLNATEPQGBmUl+hMM7ipQ4sxKSBAObqBgXoNYCIQyg7DBBQw48V7QViZIkeFQvCMTl7zwtoN55+47Pd39VnVVnfM9pz4foDC7z+3ues6Pqn5PV/XTh9PpVAAAACDFq9YJAAAAwHMWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqgAAAESxUAUAACCKhSoAAABRLFQBAACIYqEKAABAFAtVAAAAolioAgAAEMVCFQAAgCgWqkAzh0M5HA7l88OhHBJjAAC0YaEKtPRQSvmTp38TYwAANHA4nU6tcwB26unq5UMp5cvTqZzSYgAAtGGhCgAAQBS3/gIAABDFQhVYJKn4UcuiS2n5AAD0zEIVWCqp+FHLoktp+QAAdMt3VIFFkooftSy6lJYPAEDPLFQBAACI4tZfAAAAoliows71WjAoKZaWTw/9AABwi4Uq0GvBoKRYWj499AMAwFW+owo712vBoKRYWj499AMAwC0WqgAAAERx6y8AAABRLFRhUAoG1Yul5aMf5ucNAGSxUIVxKRhUL5aWj36YnzcAEMR3VGFQLQvl7C2Wlo9+mJ83AJDFQhUAAIAobv0FAAAgioUqBEsrOJOUT1IsLR/9kBeb+1gA2DsLVciWVnAmKZ+kWFo++iEvNvexALBrvqMKwQ5hBWeS8kmKpeWjH/Jicx8LAHtnoQoAAEAUt/4CAAAQxUIVGkgq8qJQjr7RD/oGAOKcTiebzVZ5K+X0eSmnf13K6fNeYmn5JMXS8tEPebG0fK7laLPZbDZbytY8AZttj1spp8PTH4qHXmJp+STF0vLRD3mxtHyu5Wiz2Ww2W8qmmBIAAABRfEcVAACAKBaqcKekIiiKweibhFhaPkmxtHzm5A0ATbS+99hm63UrQUVQasTS8kmKpeWjH/JiafnMydtms9lsthZb8wRstl63ElQEpUYsLZ+kWFo++iEvlpbPnLxtNpvNZmuxKaYEwCYeHx+/KqV878J/enc8Hj+rnQ8A0A/fUQVgK5cWqbfiAAClFAtV+ERSIZOkWFo+SbG0fNL64ZKkvM2R+W0BgK1ZqMKnHkopf/L0r9jHkvJJiqXlk9YPlyTlbY7MbwsAbMp3VOHM05WDh1LKl6dTOYmVb08SSfkkxdLySemHt28ff1GuePPm+Colb3NkflsAYGsWqsC3FL9hTY+Pj1ffYI7Ho1tJJ3BM8pz5AOyJW3+B5xS/gSyOSZ4zH4DdsFBlt0YveLJFsZSkHJNiafmk9cMlSXn3MEeS+hAAarBQZc9GL3iyRbGUpByTYmn5pPXDJUl59zBHLkk7PwDAanxHld06XCkScim+l9it4jfH4/GQkGNiLC2flH5QTGn5HEnsQ9rxvW9gTyxUgW/t8Y8gxUm20+N8SpsPPfYh2zEfgD1x6y+wd4qT8Jz5AAABLFTZhVoFT0aJLe3HHmNL2pvWlpZ9uHbf9jgftpgj6X0IAGuzUGUvahU8GSV2TVKONdps3syL3YqfS8q7Rs5p+dQ6ZwDAXXxHlV04zCgSMvWxI8b2WExpjWI1KW1pHduib3ucD2vOkV76kDp8RxXYEwtV4Ft7/CNoj22upce+Tcs5LR/aMh+APXHrLwAAAFEsVAEAAIhiocou1KrMOUpsaT/2GFvS3rS2tOzDtfu2x/mwxRwZpQ8BYCoLVfaiVmXOUWLXJOVYo83mzbzYrfi5pLxr5JyWT8vzCAC8SDElduHpU/6Honrr3RVGVf01b+b2Qy8Va2vknJZPrfnAuhRTAvbEQhX41h7/CNpjm2vpsW/Tck7Lh7ZuzYcL3h2Px882SwZgY279BQAYz/daJwCwhIUqw2lZ8GSU2BZ9mx5b0t60trTsw7X7tsf5sMUc6bEPp7YDAC6xUGVELQuejBK7JinHGm02b+bFbsXPJeVdI+e0fJL6FQA+4TuqDOfp0/uHUrHgySgxxZQ+ppiSYkpr5ZyWT8s5wv1mfkfV95iBrlmoAt/aY+GWPba5lh77Ni3ntHxoy0K1nsfHx6/K5e/5KlIFlbj1FwAAPnatGJUiVVCJhSpd26LQR1LhkbSCJ0k51mizebNOPyzp2x7nwxZzZKqkPlzSDgCwUKV3tYrB7C12TVKONdps3syL3YqfS8q7Rs5b5LN2ji3PLQDwEd9RpWtPn9Q/lI2KwbQuPFI7ppjSxxRTUkxprZy3yOfS9w8T+lAxpe34jmo9vh8O7VmoAt/a4xvz0jZPLbhx43Gj+KTASI/zKS3ntHxoy0K1HscetOfWX4BlphbcGHmRWsr47QMAKrJQpRu1Cn0kFR5JK3iSlGONNm/RX6Nb0uZR5sMW55a1JfUrAFxioUpPWhaD2VvsmqQca7R5i/4a3ZI2jzIftji3rC2pXwHgExaq9OTLUsoPn/7dKlZrP+mxa5JyrNHmLfprdEvaPMp82OLcsrakfgWATyimBDs1t7hPo0IyaQWI7i4YNLcISo/O50iPxUjSck7KJ/B4nOOTY7dHoxRTmlqELiCfdEPMa7jGFVXYrx7elNNyXJLPu9WygDbSjsc5es59RFOL0NXS6/zoNW+Y5DutE4BLDg1/46/VvmvH3r7NH5d7ctzatXnz8mPff+p9Xz9c/y3NNGuPX/qc2/rcUvMYSMoFAFxRJVXLQh9JBV3Sipv0kOPW5uQ3cj9cs3be6XOu1rmlhqRcANg5C1VStSz0kVTQJa24SQ85bq1l8akerJ13+pzrtZjSJUm5ALBzbv0l0tNtZz+vHZvy2A9FF57fBvf4+P7foNi7p1tNb7SjzFZ7XO7JcWvX5s3Ux47SD9esnXf6nFvz3LJGPksk5QIArqjCfD0UL+ghxynSChC1yietHy7pIUeW6XmMe86d7fU6P3rNGyZxRZXmEgoLzSl40ktBkRGKKV0qQHSrsNDaP8UwZ94sef49/dD6WJly/NyaY0l51ygA1TKftfgZDEZ1aW7X+GmopJ+fgkSuqJIgrbDQKAVFRiqm1GtxmaT+ann8XJKU99rza+m5pcfzDQCsykKVBGmFhUYpKDJSMaVei8sk9VfL4+eSpLxrFIBqmQ8AdMetvzTXqnDS/cWUJjQqwBbtSBirpOIyaz1/QQGvF4tm1Yx9mvf5f22fY435tWahtltu3TZ4h3du7V3Xh/FsnMOQc+RG38bkCCzniirM10PxgrVz7KHNI7v2x+4oRbN4r+V4mkvrG61Pk9rjnAg74IoqVSUUS1laTKmXwjZLirS8eXN8dV/f9FHsZqqUYkq9FCXqNe8aOS/ddw0t5w0AnHNFldqSiqVsUfCkx9glPfRNDUtzaTVWacdPet41cl667xqS+hqAnbNQpbakYilbFDzpMXZJD31TQ1oxpRr7UExpm5yX7ruGpL4GYOfc+ktVCcVSbsXS8mlVNGZaoZw2BX72WUxp+32sEes17xo5L913DS3nDVwztyjVygWkgIZcUQXupZgFrKtl0TIF09Y3Wp+2ao/3FNgpV1TZTEJhlG2KKY0Ru6dwS0KhHMWUttmHYkrb5jztsWMUalurb3pX62dSbl1BPB6Ph62eC7A1V1TZUlJhlC0KnowSm2Pq89PyvtfSXGq0OW0u9Zh3jZxbtiU9disOwE5ZqLKlpMIoiildj82RVCinBsWU5ufXY95pxZT2FrsVB2CnDqfTEHfYADfMLUbRqxa3qi287a7luLzb+tbEjvvmIrdCru/GOG8+P0fR462/c/abWBxprb6p1f+jv9cwLldUYR+i/uDnWy3Hpca+rxVfmVKUJW3OjlYYJ4WibMstOc7Yh9HfaxiUYkpsJqlYx96LKdUsQMQvpY/L9sfP/cWBWvfNmzfHV8W5pdp5N11ym12t+liNq51AHa6osqWkYh0KntBC+rj0cPy00kPfjBLrwR7bDNCUhSpbSirWoeAJLaSPSw/HTys99M0osR7ssc0ATbn1l8083eL0855iafmsFXt8PG8lNaSPS/Lx02PffChY8vy25ad2vDsej58lnAsSY/eqWSDm0pg+3dpetc2XbNUPE299VSgH2IwrqrAPimpkajku6XOix75RGKguBWLe0w+8pMfzKbiiyjqSCnMoeHIp9mlRm7dvH39xbTwvFZKZ8/xWP2uQ5p5xuVxYaN5YjXH83F+IKbVoWUKOibFepRxTrQuPkc9Vb3rliiprSSrMoeDJvNglc/qG61qNleOnbeyapByTYr1KO6YAhmKhylqSCnMoeDIvdsmcvuG6VmPl+GkbuyYpx6RYr9KOKYChHE6nru+8Ae506xbaKbfuLn3+hde7VhDko2Ida+93qRr5pLV5bTWL4rQywjhtYcncbv01gJQx7bEf5oz73Pb1dN6duo+Ac6SiWTThiirs17UCB60KHyhEs1/GmHsoEPOefhhf63Nk6/2zU4opMVtSEY7sYjDpsXnFas7jey3gMacYTI0iPRlzadnxs5e5lDRWSbH7+7C/glvbHFP390OrongAU7iiyj2SinC0LFyxt9it+J7M6YMtxmDrfbQ8fkaXNFZJsTmS8u7hmHLsAd2yUOUeSUU4FIOpF7sV35OWxXNq7KPl8TO6pLFKis2RlHcPx5RjD+iWW3+Z7en2oZ+PGEvLJyl2Hn98PP+v+3Ctby55flvdU3+9e7pNb9YY3Orr5DkyNbaXuZQ0VkmxORLy/lDY5vkt6x/m8NqxGY9d/dwC0JorqgDLzCkmoiDFZQqy0JPE4zgxp1vSivmla90vrffPTrmiyk1JRSVyClfsM3Ye30MBnCl986Fk/9QCJYoptS0Gs2U/3Mql1hj0GOutv1LPfT21xU+dzKO/2CtXVHlJUlGJkQpX9Bi7FR/VFn2zdAzWfL2Rjp+pWuaSNFZJsWuScuzh3DdSWwAsVHlRUlGJkQpX9Bi7FR/VFn2zdsGTHuZIUjGYlrkkjVVS7JqkHHs4943UFgC3/nJbQuGKmrE1X/NGwY27il60jn3avvP/Op4t+mbtYkPJc6RGbO48bJlL0lglxa6pP5emnbNTz301jh+AmlxRhe1cK27RW9EL2lN4BLY39ZydeNwl5gSwiCuqfCuhSEXr2Jqv2UsBHMWU3qvVN7WLDfV6/PR4nL10TCSNVVIspb+mz6V6x2NOm+uOPUAprqjysaQiFS2LY2zxmueS2ry0b0ZRq2+SxrSH46fH4ywtn/TYNUn5jH5MXdJy7AHK4XTyoRbvJX3CPsIVoVs/VfHmzfFVSpvvv2o472dBXvL4+Hj1ZPT89aY+bq5afdPj2Kdc/Uk/zl46JpLGKiV263hOGr9Wx+3obZ7i1hy5ZMn7wNT99rQP6JmFKmxk9DeguX88hHu35u/UJY39hwIxNfd5p7vGIKyvV8+lo/Gb4pMxHn380vXQZgvVm89veX5Y9X0TLnHrL8A4C4FLemlbL3nWNlK/jNSWUSjU1reWx5Tjmc1ZqO7U4VAOh0P5/OlWHLEN++aSpDYv7ZtR1OqbluOXbpTjrFb7elSrv0aaS1vu53g8fnY8Hg9v3hxfvXlz/P6bN8dXx+PxcDweP0vqf2CfLFT3q1VhiPTYVq95LqnNS/tmFLX6puX4pRvlONsin1HU6q+R5lLS+4BzFVCNhep+fVlK+eHTv2If2+I1zyW1eWnfjKJW37Qcv3SjHGdb5DOKWv010lxKeh9wrgKqsVDdqdOpnE6n8vPnlffEtnvNc0ltXto3o6jVNy3HL90ox1mt9vWoVn+NNJda7Tup/4F9+k7rBAACjFw45F3po+jFyGOwRC/jN4UxZnMXKumOXJ225fnB8czmLFSB1SX8jmoPP7tQw8B/oO2C8YPFRvmg5xPOD4zOrb871aqaX3psq9c8l9TmpX2zdvumvt7aj+uhb9Jjrfed0tdp+aTH0vorLZeksUoaE2B8Fqr7lVThLym21WueS2rz0r65pEZ/rf24HvomPdZ63+eScmmZT3rsmqR89nhMJcWAHTqcTr6/vkdPn1Q+lFK+/FDEQGzdvnn79vEX1/r/zZvjq5Q239s3a7dv6uut/bge+qaXWIt9J/X1rVyOx+MhaaxSYrdu0U8av1bHbdJYtYrdmiNTbfl1lCV8RQVus1CFjYz+BrR2+0b6juroY3/J4+PjV2Xg74KVCQVZ9jjuSyX12RoLojBDFBGyUIX9cusv0JtrlQZVIGxr5EVqKeO3j/GYs/m8n8ENFqqDu1aUIKlIQlJsq9c8l9TmpX2zdvteer3j8fjZ8Xg8vHlzfPXmzfH7b94cXx2Px8PxePysVf/X6pv02OiW9kPSWCXF0vprJEnjbEw+9eH97MLW/ZVwWIOF6vgUZ5gX2+o1zyW1eWnfXJLUX2u/Xsu+SY+Nbmk/JI1VUuyatHx6lDTOxgSYxXdUB/f0ieRDUZyheuGKxMIca/ZNq2JKif3fct8psQG/3/eJl8ZOMaV1501SMaVejXC+Gfk7qsBtFqqwkaQ3uRuFbu4uttGqmFKr1+tl31PsoPDRJl4au/RxT5TUZyN+2DLCvLNQhf1y6y/sw7VFyZLFytpFIBSVqMcidT7zkN6Ys0DXvtM6AdbT4vbW0WJrvubbt8vHKimXT+Pvr8Sul/e6r1ez/1vue+2+GUn9W0dv55Mw9omxlP66NX49XEFLGtNacwQYmyuqY3ko7QoGjRLb6jXPtWzfvbm0zDu9/1vue+2+GUlavyaNfVLsmrR80iWNqTEBFvMd1YG4opp2RTWnoM4auaSNVVL/t9z32n0zkqTjTDGly7Feiim5opo5R6byHVXok4UqzDS1MNHMN9e7ixpNscc3XMWUrhuxaMwlScV4EsY9UVKf1chlwEJmm753laKYEuyZW39hvi0KE430hwv59lBkpUUbFQTjJaOd62u0x/EDO6WY0kDc+lvn1t+phXLmFqxpVSRk1HmjmNKt2NqFsPqcIzX7NSfHvNg1SecMrtt6nD5csfWVBtgfV1TH8lDGLopTIzb3seemPm7qc7do3737rZVjUpuX9k2NfSfF0vJJiqXlkxS7Ji0fLks7foBB+I7qQFz1qHVFdVqhnLmf7m5ZOEQxpW37uuW+k2Jp+STF0vJJie2tmNKI3w+vMU5THrv2+PmOKrRnoQozTX3zmvsHyZZvfFu84U4tKlVD4wIln7S3xz9wAoq8VJ83SwT0Vw1NC+UMWkxpuD+6Us5pa49f2NyMeb+Fmtz6C9xri6JS92q5YBhlsdK6Ha33P1dv+d6jZaGcUQvojNau0dqTKun9FqpRTGkgbkGrdXvRtDEYvZhSUsGghCIoqX3Tax8m9U1qf9Wwdb9OLZQzyphuefUroQ9T35vv6cO553Fgfa6ojuWhKOqxNDb3seemPm7qc7do37373eI1a7SvlvS+6bUP02OjSzvvGtPrkvow7b15qhrv9cBEFqpj+bKU8sOnf2/F5jx2b7G5jz039XFTn7tF++7d7xavWaN9taT3Ta99mB4bXdp515hel9SHae/NU9V4rwcmUkwJZlJMabvXvLCPLgrWnLc3qQjHVAlFXvY2b3qQOl+3sONz2hAFeQYvpjQ5l9A5dq8h5ib3c0UVSNbDm+0oxURat2PN/fcwb3rQek6MKHFuJuaUoNdCXyON50ht4Q6KKQ1EMaX2BRsUU1q3La0L1tz7G7M9FlM6nZYXtUlpS+t5s4Wpv9Hc4+/0puQz4ryZKqH/a7031zwnLm0z7J0rqmPZoijB3mJzH3tuSYGFWu27d79bvOaSvGvooW+SYon5jGLk+ZWYz94k9X+t9+b0GOye76gOpLdPrxNjUx479WrGrce99Nz1rxQsvwKzxWsuybuG5L5JjKXk03rebMEV1bbzZsXvqEb+0ZU+b6Y8dsTj4oOZ31GNnGP32tP35PmUhSrMpJjSdq85Zx81hPfNtYIZuy8+0XrebGHquWXvf9QNVkimmhHmTY/Hhfn6stSxow63/gLJWhat6LVghj968sdurtHasyXzfz7zqx3z9TZzc+cUU+rUCLdZJcamPFYxpfVec8tiFjVvA2vRN4kFm1ruO33erNm+xLFPmTc9F0S6NL9KyRqr5DnS43HR03x1ZZMWXFHt10MZp3BFUmzuY89NfdzU527Rvnv3u8Vrpre5h76psY89nlsuSeubtXMcad70aPRjqtYcuSSpLaPMV9ic76h2KuXT69FiUx6rmNJ6r7ntJ9Utr6i2LTTVqv9r7afHedNDjsnn3TX6Jp0rqnXem1u3pdf56ooqLViowkzpxZTmFmdILhi0RMv8WheaSuj/XvXQrz3k2ErPhbT2PnbXTC0ct+S4aFWcrqf5an7Sglt/YTyKMwB71WvxlV7zrqFG4bhWxel6Gfde8mQwiil1KuU2q9FiUx6bXkxpq/3O2U/CmNbMr8W+E/u/5b7T+7WHHMeYN/cX0kqMpeXTsh8uWeu4aHdMTZuvNX7jFxK5otqvh5JVuGKU2NzHnpv6uKnPXdqWNfc7Zz9pY7p1fq33vfU+9nhuuSStb9bO0bzJi6Xlk3Q8znnsKMcU7IrvqHbKJ7T5BRtaFVPaar9z9pNQuEIxpczjJz2mmJJ5kxRLy6dF7Nb3OKe+5750XKQfU66osleuqHbqdCqn06n8/Pmb2dTY0uePHJv72HNTHzf1uUvbsuZ+5+wnbUy3zq/1vrfexxbHT3ps7X7tIUfzJi+Wlk/Lfrhk7eNiyXNb9gOMykIVAACAKBaqnTocyuFwKJ8/3RoyK7b0+SPH5j723NTHTX3u0rasud85+0kb063za73vrfexxfGTHlu7X3vI0bzJi6Xl07IfLln7uFjy3Jb9AKOyUO3X0i/j1/jSf4+xuY89N/VxU5+7tC1r7nfOftLGdOv8Wu97633s8dxySVrfrJ2jeZMXS8sn6Xic89hRjinYFcWUOvX0CdtDUZxh1diUxyqmNK8fWo2pYkqZx096TDEl8yYplpZPi5hiSoopsV8WqhM8Pj5+VS7/6PO74/H9b2CxH7feNJ+/Ydx63EvPXaLWfqf2Qyst86ux7/T+n+PGOXYUq75XjDT21NPr3zJrvOe+dFzM2Mfo56pSwucDl/V6fL/Erb/TXDspjX6yYpl3rROATox+Lh29ffRh9L9lrr3nrvlePEpf3bKHNo5oyOPbQrVTS7+M36oQQHps7mPPPX/c8Xj87Hg8Ht68Ob568+b4/afbhyY9d622TLG0b2q0Ze1+qDVvau17631scfwsmbO9GnnsW86bkWNbvea5pDbPOTc8f+yl99zj8Xg4Ho+frd03QB0Wqv1a+mX8Gl/67zE297Hnajx3yT6W5tyqLWv3Q615U2vfW++j1rlldCOPvfekbWJbvea5pDbPOTe06hugAgvVfn1ZSvnh079zY0ufP3Js7mPP1Xjukn0szblVW9buh1rzpta+t95HrXPL6EYee+9J28S2es1zSW2ec25o1TdABYopTaBwBc/VKNiwhGJK7ymm1I+5c7ZHa45Jq7EPKCRzV1GQUYuMzNXrOSPpfLqHc1Up2fOBy3o9vl/iiirUVaPYA+0Z53lG75dR2te6KMe9+x+yyAhNjHIs37KHNtKJ77ROgJc9fYH/oRS/y7Z1bMpj375dMlbvP73fsi238rsv52u/Fbr8NVv1w/bzpu04Jx8/rY6L1r9Rum6O1zLMOrdsoaf+Snu/7r0ftsx7et9MO1et/bunc66WjXpljX1yRbUPWxQLWPs1R4nNfey5tLZMsbRvlrxmq34Y6Zi6pIfjJz12SVrfrJ3j2ueWLfTYX+bNNjnX2k+vxwp0z0K1Dz0UZxglNvex59LaMsXSvlnymq36YaRj6pIejp/02CVpfbN2jmufW7bQY3+ZN9vkXGs/vR4r0D3FlCZwGwXPpc8HxZTe20vRi0sS+r9X6fO6lKbFlJofU1vfNjmyhPFbW2pxupavZ77v06jj7ooq7NvIRRNGbhvbUQjrutZ90Hr/AFSkmFIHkoszjBab8tj0ghS38luzGEx6P1wqenGrwMVIko+f/Ng2hZ3WfM2tCqaN0Dc9nrNT500PUufIiH0NLbii2octigXUKH7QY2zuY8+ltWXN/Grtp1XfjKSH42dvsa1ec4qkfmg5by5J6oe0edOD9DkCLOA7qhO0vu87+RPa0WJTHrvVz1es98lwnZ/XSO+HuX0zklb9X2s/PcbWfM25P32R1A9trqj2d66qPW96tdb4rT1H/DwNtY067haqE4w6+NxnZkGKd8fj+9vlaqk1X3s8LkYsJrKy6vO1lsfHx69KKd9rncfWUo+9lno8V23B+a8rH52L1xq75/N9B+fEYd/PLhn1POfWX9jWyG8CPVKM5baR5+vIbQPGUuN8Nfo5cfT27YJiSh1IvpVotNiUx7YrZNK2cEit/WwbyyoGk3gr3qjnlj0XN0no/7a3/uqbl/oh+RZoV4Jhv1xR7cNDyS/OMEps7mOnaNmWNXOptZ8eY2s8P8Xo55Y9Sur/lvPmkqR+qNU3lyS12XELlFJ8R3WS1vd9J39CO1psymPnXgWr/Um1YkqZ86aXK6przpHW45Le11tQTKneObG3vum1H/Z6RfXs+6Sr9MEWr5ms5+9mztV6rbIVC9UJRh12sESyAAAgAElEQVR87jP35F57jiim1I/EPxS2HrsdFPBoyrH3qdHPVTeOqckFeZL7IfE8WYOF6nLJ83ptvR7fL3HrL8ynIA9rSZtLNfKxSIV1XTumHGv9qnEuTnv/Wdvo7dsFxZQ6kHwr0WixaY/9tCDPrVuq6t/ieC2Tdfum5wIlOcfU/cWdEm/jm5b3tazzpdwWvcZtzGMcP23PiSl9M7V9vffDJWtdKVr7alQPV7f29NMt9MsV1T48lPziDKPE1nj+uZZtWTOXWvvpMdZ63+d66Icepc2RJfZ4/FyS1A9bHFMj9QOwMxaqffiylPLDp3/XiG3xmqPE1nj+uZZtWTOXWvvpMdZ63+d66Icepc2RJfZ4/FyS1A9bHFMj9QOwMxaqHTidyul0Kj9/fgvMktgWrzlKbI3nn2vZljVz6aHNvc6bpHGu1Q89SpsjrdrS6/GT3g9bHFMj9QOwP76jCkBN70qfRV6aFOaYWyW5USXPd77vRgsX5vuoc3GV86bzA72xUAWgGn+wzNbDor6HHNmHIefitfNmJz8xM+SYUIdbfztwOJTD4VA+f6qEtzi2xWuOElvj+edatmXNXHpoc6/zJmmce+2H9Ngaz0/Ww7xZO++0eTO1fb32A7A/Fqp96LXCYo+xNZ5/LqlK4hZ9s/Z+eoy13ve5PfZDemyN5yfrYd6snXfavLlkpH4AduZwOvVw10BbrX8P6+lTxYdS+vjNup5j9z4/6Tctt8oluc2tYy32ndj/aeOSFNtinJOk/MZsL8fKmn0ztX299sPc21sTfve01t+Nndz6G/PbsSNrvVbZioXqBKMOPutJmiMJb5COi/vNLZ7TAYU0FujlD9EFNp8fS85VvR+Pz9vX6znbQvW+/SRJnl+j6PX4folbf4F7XauC2qQ66kC6/aP4itHaU9vox1P6/EjPbw7nbFowv7ibqr9hUm4l2mvs3ue/fdt2TGvk8mn8/VWQpPHrbd7MHb9eJY1Vf3Pk/uOsl9uGS8k9J/Z+PI5yzmYdPV9ZY59cUc3zULKKM+wttsbzz7Vsy5q5tGxLemyr1xxF0liNNEdGmkvp58ReJR0Do89hYGW+ozpBzfu+sz/ZHz927/OTilTUKqbUeqySYrXGr1fJhVpqxVrsu5e5dDweD6nnxF768JoRjj3fUb1vP1vvmyyjfkfVQnWCUQef9STNkaRcmK+X4hhzmHdt9DKXtp4fC4spddGH14xw7LVeqFZwd0ExC1U+GPVvP7f+AmQZrfDEaO3pSQ99n55jen57MPoYjFSwC1almFKYUW9B6yV27/P3WUwpb/x6mzeXY30XPEnKJynWZt99zKWt97PsnNi2D6e17/rtyUnjfG/sw9XGkW7JBqZxRTXPQxm7qEd6bI3nn2vZljVzadmW9FhaPvohL5aWT1Ks5n7OJfXD0r65JKktW/QDMDDfUZ1AMaX9xO59vmJK+46l5aMf8mJp+STFtt5P0vm59hXV9Pbd2w+32tzhd1Tv/lvSd1T5YNTvqFqoTjDq4HOfx8fHr8qM75SMUExpbptXdnehCUjQ+PhZYohjb/T38B20r9fjZzILVZYa9Tzg1l+Y4PCTw+Hwk8PvHH5yOJT8N8xrhSeWFKRo2eb0/oaX9DqHe8373BbnROoZZR5eYx7CFYophUm5lWivsUvxw08Oh3Iqf1hK+f1Syh+fTqdyOEz/cKp+W7Yp/tFS0hxxTOmHe/umR2PMmz6KSt1/62/b8dvj8bPkZ42m3o4NuKKa6KFkFWfYW+yj+NMV1D8sp1e/Vw7lUE6vfu+P/58/LjNvmU9q39K+aSWpHxxT12Np+STFemXetI3Nfey5pLbs8fi5ZIw2Hw5/pxwOf+/Z9ndap8R4fEd1AsWU9hP7KP47P/6y/M5P/rCU8rullP/ww3//1cOvlt/69d8qv/+f/P6kK6sjFLNoXWgivQ/T8tEPWbHWx88SNY69LV5zlNiUx45eTCnx+Em5otrsO6qHw2+WUv68lPIrz6L/XynlPy+n0/+5yj6YZdTvqFqoTjDq4HPdt1dS/+bX/mH5W3/16QP+5tfKP/iP/rNJi9UR5kjrPxRG6EP2q/Xxs8Tej70eCsmN/jdK4vGzZKH6/LmhhaJuz7vD4Tf/snz2F98rX310W+YvSinvymfl18tXf99itb5RzwNu/YUz3y5SS/ndi4vUUkr5W39VfvaXPyt33Abcq5bFHhSaoHe9zuFe816TQnLtpc3DKflMLeCVOMbXc3p/e++fny9SS3m/oPhe+aqUUv7cbcCsRTGlMCm3Eu029qFw0unV75VXv/gPbo3VX5/+uvzsL39WSik3r6xGte/O2IdPV1PySYul5aMfsmKOn37nza1CRbUsyTGhD7c4fta+3Xnu77K+nPe0Yz5hfs30H5dSfuXaVa5XpZRTKb/y4/Lj3/4nh/Ivlt7aDK6o5nkoWcUZ9hb77VLK77+0SP3gr09/Xf7s3/1Z+Vd/9a9uPSypfebNNrG0fPRDXiwtn6RYWj7XcmxlSY5JfbjFHLmkxjiPNL9W9Yvy6tVPyw/+h7KjNrMd31GdQDGlHcVmXFEtZVphpRGKWXxoS1I+SbG0fPRDXiwtn6RYWj5Tr7TVcDweD0tyHPX9J/+Kah/z65qrf9seDn+vlPK/lxu3B59Keffj8uP/+p+UP3BFtaJRv6NqoTrBqIPPZR99R/VZtd9zU6v/7mmOhBaGWNOk4iZr20G/ltKob9cUOk7d92tLrQv5THn/2OPfKGu3uUYfhp4fZvnuv//35b/8x/+4fOev/uriLZm/KKV8VX69/N3yb8q/K99+TfXd6VScgzY26nnArb9w5vQHp1Mp5R+VUv55+Ztfu/ygv/m1WT9RsyNdvwlP0Kp9o/drKWO0MbENiTn1RCE51tL9sfjN3/7b5c9+8pPyrnxWzi8Ff1ik/lb52fNFaikDtJt2FFMKs/fbrFrHfhk/PZTf+fE/Kt/7v/9h+fv/vHxU/fdvfq2Uv/jd8vu/+w8mLVKT2rf1vOmwMMRsLY6pPfRrKVnzfaRxSuibft+T2hbCmvLY0YspXWrL2m2u0Yep54e5vvqN3yj/23//35X/4g/+oBx+8cvl6v/79a+W3yo/K/9X+c1PnuPWX+7limqeh6JwRcvYL+P/y48fyp/+USl/8bvl2yurT4vU8qd/NOdKalL7as2bkbU8pkaXNN9HGqekvvGeNC8297HnktqyRT9csvaxO/r5YbavfuM3yr/8Z/+s/M//9J9+u/3d8m8uLlKfDNkPbM93VCdQTGk/sfN4KeUXpZxK+a/+21K+/z+W8n/8N6X86R+VUg7l7dvHSWM6ajGLngpDrOnSeJay9ZW68fu1lP6PldRxSu/XtHySYlMeq5jS8jbX6MPU88Na3rw53vrPF983Wc+o31G1UJ1g1MHnZYfDhxPqqZT/9H8t5d/+dinl/ZBPXajuaY60LjxSQ4vx3EO/ltL/sZI6Tr33K7ft8W+UTosp3X1++NGPflC+/vq7F/7LqXz4myTZ6dRBkp0b9Tzg1l+Y5FDKv/2dcscbwt6KYYze3lbtG71fSxmjjYltSMwJ9ujuY/HyIrWUHhapxTmIBRRTCpNyK9FeY+fxOWNXyji3Wd0/b7YvPJJ4m9v2+2lb0CWnH9Jj18cpJ8e8WFo+SbEpj1VMaXmb6/Th/efxUj4pspts0vswTOGKap6HklWcYW+xW/Epktoy+ry5RN+0jaXlkxRLyycplpZPUmzuY88ltWWLfrhk6X7WfL0t+iHdSG2hMd9RnUAxpf3EzuPlxqeYl76j6orq9rF9XlHtI5aWT1IsLZ+kWFo+SbEpj008J25/RbW/YkpLYsUVVV4w6ndULVQnGHXwednhcP2EemmhemcRh6/K5R/Efnc8vr9ViF8a/Xi8MR96NeQ8dtz2Y8BjarIRzomX9FhMaarDoXQ9X08KJzWRNIfX5NZfaO/aG1K3b1QsMtq4j9aeDxy3/TAm9GTGfI272KRwEquyUA1zOJTD4VA+f7pNYpNYrf30GLsVn2Lpvtd8vdHnTVJ/1Wpfr5LmTat5mJZ3UqzmfvYmaZy3mCNrtzmpDy95+/bxbPufPlzBfFVK+X4p5dXpVA4NY5859liThWqeqV9CXxKrtZ8eY7fiUyzd95qvN/q8uWSkvhlJ0rxpNQ/T8k6K1dzP3iSN8xZz5JKR3lemShorxx6r8R3VCRRT2k/sPF4qFFNKL+KQNm8S+6tW+3qVNI9bzcOUvBNjLcdqdKMce+fxkYsplZl/dxyPx0PSWF0bP7Y16ndULVQnGHXw+dhhZgGDS28YHVq18MsOipZsXijn1vmmVysXGYvnfSHLiMfUyrorAHZpTH/0ox+Ur7/+7qWHvzudys32tfo7b42/O5xvKGXctYpbf+GXJr9ZvH79zZZ51LT2QqDLhcUMNdo3WjGKe9sz+lyintGOqbX1eKx9MqZXFqmlZLdvj393wGTfaZ0AL+vtNqueY9cMcvX0ojXnzdu3tbOvb/tj6v2VjaTjos0t0PeMTgbn3bT3pOxjKuHW5IR+mDdHPh3TcuOW2SXnm41v873l21uOX5ojaWMFa3FFtQ8Ppc/CFT3G9miLeTMyx9T12Fav2RtzZF4sLZ89vv8k9cMW7z81nrv22M+ZI2ljBavwHdUJWt/33d+n133GDofrnwaOfEV1aqGIa/G0KwNbUyin1hXVfueSOeI9aU4sYa6nF12a8thy44pqeXZ1cu4YbNk3U3O+lV9aMSXaaL1W2YqF6gSjDj4fu7VQPff69Tfliy9+umU69KO7QiQ96LkAjvcF5kiY673N2cPMIkQXfFRgacnfeSvkctHp/e+UllIy5kgF3ksXGHWt4tZf+KXJBTduFG1gf5ILdfRMARz2ovVcb73/eyw976553t7iPaDHMVnKeymfUEypA26zqhP78OnqjNtyPpH6+20Jt5bVsOUtWj0Vs0i+PW96rN8COM673pPmxa7P9SnPn3t+T7/Nd0o/3Grf86/qvHlzvPq4j/vw+utNuH13DS/cmrzSXqAzrqj24aEoXFErdis+xdJ9r/l6S9rRq1pzpNW+02Np+bSaD/pmXiwtn6TY3MdOkdS+pf2wxJLjuUYua+8DuuM7qhO0vu/bp9dNP7W9+kn1pQJLrqi21eqKaloxi5Tjp3U+ra6oKqZk3tTsm51eUZ303nzrimqZWKzopf66lcsMdxd7GknP36VsrfVaZSsWqhOMOvi87DCzEvA986HG/NpJIYZNj0fnAZ4zH0gx9/ze2/w8zCxWNGOh+pFrRRKf99fcXKYa+ZcF5uhtbiYZ9T3Jrb8AAP0avfDO5IXh69ff3Pz/t0wskrj6InVOjoMbfR5zB8WUOuA2q8yCDWuN1ZIiDmvs45LkT99ufWq47a2et/NKmscpx0/rfFrNh733jXlTu28+LcY0t9hXeuyGm7fvfrhC+vz23TLzVt2pfxOMflW0xi3jcM4V1T48FIUrasVuxadYuu81X29JO3pVa4602nd6LC2fVvNB38yLpeWTFFvj+eeS2rekHXMeO+c113zuSFqOMzvlO6oTtL7v26fX+QUbPhilmFKvV1QVU8o6flrns+0VVcWU1oql5ZMUu/f5Nd5XasUON2pFlDsKIpUF7+tznzsSV1SztV6rbMVCdYJRB5+X3XqD3FkxpXfH4/vby1prdTw6D7T3+Pi4SSGTtW09H3rph4lizi0jmXq+ajyXPhr7wx2Fik6n8rwtk9q85H197nNH4n0u26h/o7j1F5hilD+K6Zt5+N5I/TBSW3rUsv/P9z03F8V36tHXNKGYUgdSbiXaQ+w8XmOsEosp9WrbWz3b7buXWOsxSLF13/TSD3MkzeOk2L3Pn/q+0nouTX2/PZ3Ki1+vmNrmrXJ86dbYJbdj7+WrJ3DOFdU+PJSs4gwjx27Fp1i67zVfb0k7elVrjrTad3qs5n6S1eqbkSTN46TYGs8/lzSXtjjHLt3Pvc+t8V6/9PnpMfiIhWofviyl/LB8/GnektgWrzlK7FZ8iqX7XvP1lrSjV7XmSKt9p8dq7idZrb4ZSdI8Toqt8fxzSXNpi3Ps0v3c+9wa7/VLn58eg4+49bcDT7dE/Hyt2BavOUrsPH6Y+fXze/b9eKP+wlrtu7WPkWw5R17qw6R5nHL8rB3rZR7f2zcfCts8v4XxQ5tb36K5taR5nBS79/lT31daHVM/+tEPytdff7eUGb9pulabl7yv33rulu/1e3n/gXOuqALQi9ELeuyxsNDoY5quSf8/LVKnMkdgp1xR7UBKcYY9xM7jNcZKMaX1KKaUdfysv5/3P2Wx7TjfLlry4X/f+imAe/tmD8ep35itc0xNf1/Z/pi6FCu3r6TeNUdaF1Pa8r1+xPcfmMIV1T5s8WX1pC/PJ8VuxadYuu81X29JO3pVa4602nd6LC2fVsfK0r4Z2ejzJu2YuiStfVPy2+o1p1i7X9d+7pznJ8XgRRaqfUgrzjBy7FZ8ihpFLxQquK5lsYekeZx2/PQYW2Jp34xs9HmTdkxdkta+Kflt9ZpTrN2vaz93zvOTYvAiC9UOnE7ldDqVnz+/XWJJbIvXHCV2Kz7F0n2v+XpL2tGrWnOk1b7TY2n5tDpWlvbNyEafN2nH1CVp7ZuS31avOcXa/br2c+c8PykGU/iOKtz2rlwpcPLmzfGj///69TfleLz0SIBJrp5vBqEozg4dDuWrMn1eN5wjH6+hPlQmfvNm0uLK3IYNWKjCDadT+ew8djhcftOaWcUQ4CPH4/GT8w0M4Ooi9XQqM38sZksfp3LrPT0rbxiXW387dTiUw+FQPn+qpDYrtvT5I8fmPvbc0n2v+XpT9zG6Wn2YNI97OH7SY0uM3jfmTT99c0nSMbVFP0zdzxIt+2HtfJLmA5yzUO2XypzbxOY+9lx6dcY9qtWHSfO4h+MnPbbE6H1j3tSLrfH8c0nH1Bb9MHU/S7Tsh7XzSZoP8JHD6eR7zS+59Xt5z39Xr6anT6UeSkn8rcN+Y1MeW2b+/ttLr3frdxsv/ebgPe27tY+pWs31c3OPx7XmyEu/r5k0j5OPn/TYGr+j6rdC9zdv0vqmxvvK1FhZ4T1zzTbfyuft28fnz7mRdt33+hHff1hX4lplDRaqE4w6+NzncOU7qle8u/Q91+dqzK9b++jAu+ff3Wt1PPZwHnh8fLxWtOTdS99/vPHcll7Me21Tx3nOfAjt23PV+5rttDpfHeYVTvrou561jpMXFqCT3PMd1SVj0sP7D22NOkfc+gvzzanul/7HaQ/04XTX+mpKHyb2c2JO9+ihHT3kSL458+j8vbTKHHz9+pulL6HCL1RiodqppV9gr/FF+R5jUx57OpXPnj5NfVVK+X554Tias+97n7tkH3u0RR+mzeN780uT1IdLcu5B8nl3r7E1nn+u4fHz7Xvm6fT+vbTFsfLFFz8tb98+frS9YFLeLc/PSXO21/MfmSxU+7X0C+ytvjyfHlvj+edqPHfJPvZoiz5Mm8f35pcmqQ+X5NyDXs+7I8fWeP4556B5euibpDnbw5jSCQvVfn1ZSvnh079zY0ufP3Jsjeefq/HcJfvYoy36MG0e35tfmqQ+XJJzD3o9744cW+P555yD5umhb5LmbA9jSicUU5pg1C8os57DjAJLr19/U7744qdbpjOce4vYrKmH88BWxTpauqdvOyleFCdlHidpPJcmFbg6XClgdO29Zq1xvrbfa14qQNTyHHSrwNI9hZMuUUyJLY06R1xRnebaF+d9oZ4PJs+Fr7/+7pZ5jMhxVkdiP9+bk0XqfInjn6DlXJq674uPq/Bes6Rw0r2P2cSNAkuOC3ox5FrlO60T6EFiyf6nL6k/lOI369aMLXj+Z+excvu345p4/lttc39btedP5M7dM8Zv367/muv/BuiS/I6fzOFezy0vjVWPav/2ZSfn3V3MpZdyXPv1Zvw+6i13zNfr56Apz0/67di1z889vP/4zdS2Etcqa3BFtV9Lv9Se9CX7pNhWr5kiPb9athjPtHncIr9a++nx2FsqqV9Hnzdpc2ntHHs836Tl06q/lj4/qc3wIt9R7dSon163jq35muXGFdUJ5fA3McIV1TW+h3Hfp+HX++t4PB4S5nHrKwot2jy3H3qVcEVo9HmTOJemnFvKzPeatebSrf2Wu66oLpsjrc9/W+bXw/uPK6pswUIVNnKYUWDpki2KLk0tSvTSc1vqtZjSDgr8TCr8UkNqYaglUo6/vWk9l87H/TCzgFErrT6MXWjTc1jlYncx52NYwq2/sJ1FX2DfoBBG11+of6bXggHxf1wulNS+9Lkw12jt6UnLvr+076Tj7KIbhYnSJfft3HmY3BaYTDGlTo16m1Xr2MqvubjA0pa3KyUUCbnHVp8Sb13Motf+nivjeL6/MNRLt9h9+N8rXGG/+vyE2xQ7Pu/GzKUt+uZ8rjz3/CrmrZ9b2cDdXylJs+2tv0v2e3ke3jqPuM2XEbii2q+HMnbhilaxmvuZIimX0S3tL/39XtLx3OtxkdQPI513e4zdiqdIz2+OVueWLc5Vezx3MhjfUe3UuJ9ej/3Jfpn/kzUbXlHts5jSVpZeaVu7v3vUQ1EPV1TzYmn5JMXO42Vi4SRXVO+z7V1M6xd7SjyPwJosVKGiw8ICS1e8Oz3dZnzNDgr5EGCEDzNaF8/ZgKIqnTrMLJzUaqF6OpVJH+D0ILkgX63XhCRu/YW6tijMMeUPGYtUtqbgTybHfr8mj915AaOKBY3Oj/uezwM95w5DUkxpIG6z6uIWtEmFOcrMW4SXFHEg0z23bbX+HcEtXrP+rb9TRqcvCf06+rxpcBvlzdttP/y0Wf3bP7cpPjXlsVNv25/6ejXPLVvMEbf0MgJXVMfyUBSuWBpLzGeKtV+P9taeN3s8fmocez1I6tfR502teZh03KfNkamSxrTWHHFOpCu+ozoQn16P88l+WbnoUu8FLvbIFdVWV1THO1bSizOl5ZMSO9yuaTCpgFH62O/rimrfxZSgBQtVCPTCHyhTfFRgqfcCF3t0TyEMhTWWG/FYMfbZDjOLJpUyvYDRnsa+52NXMSW4zK2/kGlpUYfzP3oUiYBpRjtWRmvPiOYWvDKmYzGecIViSgNxm9VQt6AtLrr08XM/LXBxz++ohvRNXOze598ag/te7/r4JfdDVqxOMZg93K65r3lzf+zaPCjl/ZXT9OM+ZY70Ugit1i25bvNlBK6ojuWhKFyxNJaWz5y8zy157tLX3FtsjeefS3q90Y+fln1zSVLe5k292DXpx33aHEmXNm967EN2wndUB+LT6/19sl9uF11atcCSK6p1r6gqpjRWLG2skmJp+SSdx6ddUR1/Lk15bC+F0NYck62KKUEKC1Xo2GFe0aVFBZYUZljf2oUwFNboxwZjf60gz7vj8fjZhTgNHGYWTjo9K5p0zZK5dGPesJE1z8XO+YzOrb/QtzlFGPwxAuO6dnw77rPMGY8aRXbMj7oUToIZFFMaiNusdnkL2ieFX8rEAkv3FJ4IaXNc7N7nr10EpXVRlbRxSYptPVatx968mXyb7y2f3Ko55TWXjH0vBYjmeOmW19Fulb5mi9eE2lxRHctDUbhiaSwtn6VtOTf1cXOeL7bO888lvd4ej59afXNJj2Nv3syLXbK0b6a+5tTn9mrkY2XO2G3xmlCV76gOxKfXPtkvL1xRLc8+sVdMqf28UUxpP7Gtx6r12Js3656f511RvX/seylANMeerqgqpsToLFRhMIcZBZZev/6mfPHFTyc9duvCDDsp6vFRYRvFlN4LHPvNCxAZ+/EdNiicdMnUsQ88zjYxoYDUMMfKSG2BS9z6C+OZXKzh66+/u/prLjD8H1BlH228R1q/pOVDnxROqk+xIhiIYkoDcZuVW9CeYrMKLF1y6ZahrfMesajHJVPbvHYfJh8/iWO//a2/6+679dg77168zfeWSbdlTtnP1LFvfZwtvRW1xntNwrxZ6zbdLV4TanNFdSwPReGKpbG0fLZo3xQt+2Z0U9u8dh/2cPwkqdU3a+977dfrYd6kxy7Zom+m7qeVtDmydo5J82ur14SqfEd1IHv/9Non+9dj5cYV1bdvHz+JtbmiOl5Rj0ue9+1oBXXufX7i2K/dX+fx0cbeeXfeebesekV12ti3Ps5yrqj2e6ycxxRTYnQWqrADhxkFlpjrVEo5fBK9VqjqrLjJ1XHZU0GdW3nv0Z7GvkeHmUWSLjndWTjpkl6On5R5ONKxMlJb4BK3/sI+KDCxmct/C1wpVGUcLtMv9GRpUaI9zvc9thlYSDGlwe3pNiu3oN2MLS6wxHwv/57f9efed0vbuq9X7/g5fjI/93wbeY9jv6fz7vWevmrTW17nFknq7fbWqf2wh2Nlzm26W7wm1OaK6vgeyv4KVyyJpeVTq81sY8kYrD2mezx+etXj2I80b9aeX7X6pkY+I82RS5LavHTc93juZDC+ozq4ET697vVT2/RYcUW1hheuqI5VUGeL19zjFdUex36EebPhuXPjK6rz5nv6fNj+imq/x8p5TDElRmehCoEeHx8XF+t4yZs3xy1fnsvenZ5uwy5FMaWt9FJc5pqexr7GuWol747H42e3HnBYoUjSJacXCid11IdLvNj/tYx0nhypLXCJW38h0+Z/tLx+/c2MR3f9d3+Srcf1WsGSvRUy2Vt7W+plgTUlzy3aMmUu9tKHS+yhjcDKFFPaqZTbVtJiKfnMLY5xj0s/nbKl3m6pmhM7j5cbtwZOHef78lm3KFGvx0+Nfph7++Cc5699zuj9XLWWCbf0zrXKOa2nPlwi5dyimJJbf+mHK6r79VByCgEkxRLzGUVSv9aaN5cseVx6LC2fVsfoFnNk7Xz2eK5auy177MMl0s4ta+eYdA7a6jWhKt9R3amkTwOTYin59F4M5hJXVL/1bT+MVNTjUj+0zqfXK6o9FVPq6Vz1Uj+UikWSeu3DJYk+aYUAABUfSURBVJYW92l17Cacb67FFFNidBaqEKj3YjALxBTcWOJwmP5m//r1Nxdvw1YII9vSIiajFNLq4Vz1ox/9oHz99XdXf93TC0WSpuqhD9eQck4bqQDRSG2BS9z6C5n2WgxmlIIbk8dviz+goaL4c9UWx9i8YnQviu/DFeyhjcDKFFPaqZTbVtJiOfncXwym99vIMvp/8bz5ZPzKzNsKk9rX3/FT49bfeWO39Plr55Nwrqo1b8qCW3pvF706bt6H977mrbwvXWlLOKbq3fp7rWey2rz0Nt0tXhNqc0V1vx5KTiGApFhaPkvb0qOkPtxi3kyV1L49Hj9LxnNO30x9/tr5JMVq7meKpH5d4/lTJM2HWnPkkqQ2Lx3jGvMGNuU7qjuV9GlgUiwtnz1eUe2tmMWHvJdc1Xn79nE3/TBKTDGlfuZN2eiKao1+vff5rqiOeaycxxRTYnQWqjCY3gtzjFoA4jCjwNIOvDs93R7dK8WU6jocylel0nfYT8+KJPXar73mXcNIfTNSW+ASt/4CSUYuuDFy2+YapWjWEtfmg3lyWa05o/8BQiimxLdSbmVpGUvL577bmq6Pccrv2I0Wm/jYxQWWRpI0fmsfZ9PmyPIiREvySYpNeez11s1yxy3V118spW/m5t2qLSnvza3HdIu+WbstkMIVVZ57KDnFAVrF0vJZ2pZz+mab2BrP35uk8atxnKXlkxSb+9h79divW+R9SdJ8qHX8XJLU5qVj7D2J7vmOKt9K+oRwhE9t213p2aZQREr7EmP3Pr/s+IpquePqVlJsaTGl1vkk9OHMK6prHCvdFam69/mKKeWO6ZoxxZQYnYUqDGaL4gqPj4/VCplM9O54PHZdjKeUUg4KLD3XVYGltCImaflMdWhUJGmqlv1647z74vkvaT4Evn9socl7UtI4wxbc+gtMkfZHRlo+91K45ZdGGVPmUSTpumt909ux0lu+99hDG6E6C1VuOhzK4XAonz/dGjJ8LC2fpW05t/brtZTW1/c8/3Qqnz1d5XlVSvl+KeXV6VQOvcamPLaXMV37OEvLJyk2073z87Me+3XpvpdoPM7dSTt+jBUjsFDlJUkFA2rE0vJZ2pZza79eS2l9nZRPD/1wSVJb1m5HWj5JsTmScmzZNzXO2y3HuUdJc3Or14SqfEeVm54+YXsogUUEtoil5XNPbItCEbe+B9NKUtGLlvtOik15bLldFCdmTNc+ztLySejD06mcDvO/p121LS37dUmhnLnFlC6p0YcjaXGcKabE6CxUoaKdFJWoYstCEQOOU6tCH5/045s3xzkvEVtgKa2ISVo+5w4rFU26pyDSEo2LKd2976T5kPhB5xYc97A+t/5CXSMtflraujDKaOPUqj2f7Pf1628WPZ9urTGWPRZEYh/j1qqN1/a7hz5nB77TOgH6k3LL2Baxrffz9u3S3u9X8u2H57ERx6nF8XOpH7/44qellI/nQ7lxO3DCfJg7R9LOLQn9dT2791dJE3JM7Nd7973GOWy9try/myP5vbnX2Ic7ZbaYY5DAFVXu8VByinCsHau5n71JGuc9jlPL42dqPq3yXnuOpJ1b0vsrLcekfl267yXS+j8pn6TYGs+HSL6jymwpnyRuEdt6P3spKnFJX1dUxxunGv1/Hp9aiKZ0WGBJMaV5sXJjjLOvqCqmtFX75sTS8kmKrfF8SGWhChXtpajEJT0VdhhxnJILfRxuVH49VS6eM1VaEZO0fM71OMalKKYE0JLvqEJd78o+C8T0VthhtHFK7/+r/X1hgRNbCXiOmpWla3/w8qMf/aB8/fV3pz48fW4C0IiFKlRU4ydCfJK+XIufctmzSwvPG1fgRvkAYZR2fOLWIjX56ikAWRRTYhWHQzkcDuXzp+8/dBtLy2dpW87pm+36OimftH64pNU8bnWczXn+yHo9fqa2pcZ+13j+FGn9n5RPUmzuY6EnFqqsJakCXsvqeUmxS/TNNrG0fNL64ZJW87jVcTbn+SPr9fiZ2pYa+13j+VOk9X9SPkmxuY+FbiimxCqePrF7KAEV8JbE0vK5J7ZVlcqU9iXG0vJJ6Yclc7GEVwJeWvV3xMrSH7x5c7z1n6tXRE48n06JqfrbxxxpHZv7WOiJhSoMxndUSbFkLh5uVIm9oHqBpaXHWVJl6ZnFjxZJ/o5qzQJXlFJKeaceAHCLW38BiPP69TdzHm5xsUCtRWrJr/BrHtWlv4GbLFTZTFKxgS2KEqTHLtE32/V1Uj5p/XDJS6/5xRc/LW/fPn60LXm9Lds8JZc5zx/Iq1LK90spr06ncjidymc9HD/U4xy7PHYrDr2zUGVLScUGtihKkB67RN9sE0vLJ60fLlkyj5e8XqvjbM7zR9Hr8UM9vc6RpNitOHTNd1TZzNMnew8loNjA1FhaPvfEFFMyb1L6YclcvPTcuUV6ko6zOc+v7YV+XWLSmFyLJ5072cbU96RrcbHrfQMjsFCFwSim9N6AhVG6KzyyZC5eeu6tBdVLtwbXlFxMSeGk65IKXO3FxGOl5bm8u/MujMStv8CoRlqkljJee/auSWEhhZNu6jHnnk3t75bnPuddaOg7rRNgXEm3xuzpFs63b7cZk5T2rdEPvUro13m3/t7flp7Hb9rx8/4qTe1bkUvg79PW2s/LsU/HpNffUR0t1lJSP7j1l71xRZUtPZScYgNTY2n5LG3LuT32zUiS+nXOHFnSlh61PH7m5DPlcc6717U8fyX1zejn8aR+WHreha74jiqbSfrEsb9P9usVeRm1b0YsjJJ+FeU8XrOYUtJ3VKcUU2p13BdXVCPOp66ortMPNaT39a049M5CFQYzUjGlAQsiLdLh+N09F+cWU6KU16+/KV988dNv///cwkmnzoof1dDyfDrSuXyJ1gvVPfU1pHHrL5DMIvWXdl/o5fXrb1qnEO18UTqzcNLu59cV1/pFf9XTsq+NMzSkmBJVJd0u09staNNvVdum/9PakuR4PB4Sxr517Dy+djGlD1cLZ9zKynUxtzO23PfLsXWLXt3TN5dk9E2d2Iefh0nJJy0GI3NFldoeSk4BgkuxtHyWtuVcr33Tg6SxTzt+Lll7HjNPD/NG7LqkHM2RtjEYlu+oUlXSp5D9fbLftvhHWluSuKJ67YrqusWULj23uKL6kedFpV74Tq8rquExxZTMkal9A6OyUIXBjFSAo3URjal669da1i6mdOm5h4M/1u5xGqBw0p6LrTnnAHvg1l8gWQ+FLHrIcWT6f75R+myXi1SAvVBMiebSbqFJyue+22W36es27eujiEZaPin9sHYxpSvPNUdevgU6+jbRe5/fS7G1LSSNX/IcGS0Ge+OKKgkeSlZRgqR8lrblnL7ZJpaWT1o/XNJqHqf1Tfpx30Pf7FHS+PUwR0aJwa74jirNpX1amZTPfVcZximm1EssLZ+UfqhRTKl1m1PmSNnlFdU+iq1tocdjIPn46SUGe2OhCoMZqZgSfatRTIn3DjeKSp0GKJx0SS/F1rbgGAD2wK2/MJ5rhVJGKaACfGqPx/3IbQPYPcWUiOT2oiWxbQoQ5bQvL5aWT0o/VCqmFNcPjeZIF0Wl1u2bPoqt3Ru7dcU4Jcf8OdJvDHBFlVwPRcGGtFhaPkmxtHzS+uGSOa9573OTYmn5JMXS8kmKXZOUozmyTQx2z3dUieRT27xYWj5JsbR8UvpBMSVzRN9sc0W1x2PAHHFFFeayUAVgE6MUU3p8fPyqlPK9mvuc4N3x+P7WV8aUdAwAtODWXwC4LW2RWkpmTgCwGgtVunE4lMPhUD5/ukVmk1it/fQYS8snKZaWT1o/XDLnNe997hbtS5I09o6f7frmkqQczZHt2gx7Z6FKT2oVL6ixnx5jafkkxdLySeuHS+a85r3P3aJ9SZLG3vGzTeyapBzNkW1isHu+o0o3nj5pfCgKNjQr7JCUT1IsLZ+UfhilmNKtXFoapaBOWj4pMcWU9j1HAAtVYEWhRWfWpIDNDAMVU4p8o1RQZ2xJxwDLdf7+6L2PJtz6C6yp1zfhqUZv39rezYynSsw3MSfWNcrxw3s9v3/0nDsd+07rBGAJtxdl3Xr19u2c0euTeTNnjrz/BP6+223njcG2sfvbYY7om3tjH65gpeRjjiyL7eH9Edbmiiq9eygKNtSKzX3sqMyb67GtXvNcUpudW5bH0vJJiqXlox/WaQswge+o0jWf2mZ9op1adGZNl4qYlJI1VslzZPrVh5xiSml9M1osLZ+kWFo++mGbc1oPfC+aFixUgdWkFp1ZkzfrOhSSAUbS+/uj8y4tuPUXWNPoRT5Gb18ShWSAkfR87uo5d3p2Op1stqG2Uk6HUk6fl3I6zI0tff7IsbR8kmJp+eiHvFhaPkmxtHySYmn56Id12mKz2aZtzROw2dbent4U/nUpp8/nxpY+f+RYWj5JsbR89ENeLC2fpFhaPkmxtHz0wzptsdls07bmCdhsa2/Fp7abxNLySYql5aMf8mJp+STF0vJJiqXlox/WaYvNZpu2HU6nUwEAAIAUiikBAAAQxUKVXTgcyuFwKJ8//abZ1dicx+4tlpZPUiwtH/2QF0vLJymWlk9SLC0f/TA/b2CB1vce22w1tjKjyMHUx+4tlpZPUiwtH/2QF0vLJymWlk9SLC0f/TA/b5vNdv/WPAGbrcZWdliwYe1YWj5JsbR89ENeLC2fpFhaPkmxtHz0w/y8bTbb/ZtiSgAAAETxHVUAAACiWKiyC6MXbFDMQt8kxNLySYql5ZMUS8snKZaWj34Aqmp977HNVmMrMwofTH3s3mJp+STF0vLRD3mxtHySYmn5JMXS8tEPy/8esdls07fmCdhsNbYyeMGGGrG0fJJiafnoh7xYWj5JsbR8kmJp+eiH5X+P2Gy26ZtiSgAAAETxHVUAAACiWKiyWz0UbEiKpeWTFEvLRz/kxdLySYql5ZMUS8tnpH4AOtD63mObrdVWOijYkBRLyycplpaPfsiLpeWTFEvLJymWls9I/WCz2fK35gnYbK220kHBhqRYWj5JsbR89ENeLC2fpFhaPkmxtHxG6gebzZa/KaYEAABAFN9RBQAAIIqFKtwpqdBEr8UsRoml5aMf8mJp+STF0vJJiqXl00M/AANpfe+xzdbrVoIKTdSIpeWTFEvLRz/kxdLySYql5ZMUS8unh36w2WzjbM0TsNl63UpQoYkasbR8kmJp+eiHvFhaPkmxtHySYmn59NAPNpttnE0xJQAAAKL4jioAAABRLFRhY0kFKRT10Df6Qd+kxdLySYql5VOrzQCllNL83mObbfStBBWkWBJLyycplpaPfsiLpeWTFEvLJymWlk+tNttsNtvpdCrNE7DZRt9KUEGKJbG0fJJiafnoh7xYWj5JsbR8kmJp+dRqs81ms51OiikBAAAQxndUAQAAiGKhCgAAQBQLVQiRVHVR9Ul9ox/0jb5pH0vLZ07eAIu1/pKszWZ7v5WgqouXYmn5JMXS8tEPebG0fJJiafkkxdLymZO3zWazLd2aJ2Cz2d5vJajq4qVYWj5JsbR89ENeLC2fpFhaPkmxtHzm5G2z2WxLN1V/AQAAiOI7qgAAAESxUIUBKOrRNpaWj37Ii6XlkxRLyycp1nrfAE21vvfYZrMt34qiHk1jafnoh7xYWj5JsbR8kmKt922z2Wwtt+YJ2Gy25VtR1KNpLC0f/ZAXS8snKZaWT1Ks9b5tNput5aaYEgAAAFF8RxUAAIAoFqqwI70W9UiPpeWjH/JiafkkxdLySYqt8XyAbrW+99hms9XbSqdFPdJjafnoh7xYWj5JsbR8kmJrPN9ms9l63ZonYLPZ6m2l06Ie6bG0fPRDXiwtn6RYWj5JsTWeb7PZbL1uiikBAAAQxXdUAQAAiGKhCkyWVGQkKZaWj37Ii6XlkxRLyycpBrBrre89ttls/WwlqMhIUiwtH/2QF0vLJymWlk9SzGaz2fa8NU/AZrP1s5WgIiNJsbR89ENeLC2fpFhaPkkxm81m2/OmmBIAAABRfEcVAACAKBaqQBVJBUoUg9EP+iYnlpbP0rYAsJLW9x7bbLZ9bCWoQMnasbR89ENeLC2fpFhaPkvbYrPZbLZ1tuYJ2Gy2fWwlqEDJ2rG0fPRDXiwtn6RYWj5L22Kz2Wy2dTbFlAAAAIjiO6oAAABEsVAFoiQVRtljMRj9oG/23jcAhGh977HNZrM930pQYZSpsbR89ENeLC2fpFhiPjabzWZrvzVPwGaz2Z5vJagwytRYWj76IS+Wlk9SLDEfm81ms7XfFFMCAAAgiu+oAgAAEMVCFRjG6MVgkvRQFCcplpZPUmzuYwHYBwtVYCQPpZQ/efq3ZqzmflK07IceY2n5JMXmPhaAHfAdVWAYT1deHkopX55O5VQrVnM/KVr2Q4+xtHySYnMfC8A+WKgCAAAQxa2/AAAARLFQBSBCUoEfxZTWaTMA3MtCFYAUSQV+FFNaHgOAu/mOKgARkgr8KKY0VvEvAPpjoQoAAEAUt/4CAAAQxUIVgO4lFRHqtZgSACSxUAVgBElFhHotpgQAMXxHFYDuJRUR6rWYEgAksVAFAAAgilt/AQAAiGKhCgBn0oopAcDeWKgCwKfSiikBwK74jioAnEkrpgQAe2OhCgAAQBS3/gIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiGKhCgAAQBQLVQAAAKJYqAIAABDFQhUAAIAoFqoAAABEsVAFAAAgioUqAAAAUSxUAQAAiPL/AykaYnoisiJNAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " A* search search: 154.2 path cost, 7,418 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3b+PNMl5H/DqlzxxSfNeWpEdEXBmCBLumNMgQYeODgJmARPCRRSUSP+BQZ4DJc6kRAKh4A0U7AjGwSCc2YQNHaD0DrSVWpFhOJLehemXfOG3Hezue/vOzvR0T1d3PVX9+QCHw9XNj+ruqu55pnq/0/V9nwAAACCKZ6U7AAAAAI8pVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKF8tXQHAGjTfr9/mVJ6/8j/ut3tds/X7g8AUA8rqgAs5ViROtQOAJBSUqgCAAAQjEIVAACAUBSqAAAAhCJMCXhL+A3EYk7ymPEAbIkVVeAx4TcQiznJY8YDsBkKVQAAAEJRqAIAABCKQhUAAIBQhCkBmyachMeMBwCIwYoqsHXCSXjMeACAABSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUL5augMAAMyz3+/7g6bb3W73vEhnADKwogoA0J73S3cAYA6FKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCkfoLAACP7Pf7l+l4IJU0ZViJFVUAAHjXqdRkacqwEoUqAAAAoShUAQAACEWhCgAAQCjClABmGBu4MfC4VggYAQCysaIKMM/YwI2Wi9SU2t8+AGBFClUAAABCUagCAAAQikIVAACAUIQpwUbVEO5Tuo/7/b4/aBIYxGaVno8zmbuBjA2hC9CfU48/vDaUYlzTNCuqsF01fOCM1sc5/bnN1gsoI9p8nKLmvrdobAjdWmodH7X2G0axogqwgjnfegf69h4AYBVWVAEAAAhFoQoAAEAobv2FiSoJFBGwAABAtayownTRi9SU6ujjGNECiEr1J9p+OKaGPjJPzce45r6zvFrHR639hlGsqAJhHVsVHgoW2u123bI9KsPqOBEYh7Sq1LVmi9czmMKKKgAAAKEoVAEAAAjFrb8AwQ0EeAnNasjUoLbMv69rLGUWIXiv1THinAjbYEUVpqshvCB3H2vY5pad+rDbSmgWd0oeT2Mpv9b2aaTtcU6EDbCiChNt4dtaIQ4AAJRkRRUAAIBQFKoAAACE4tZf4CLCLABYWuGQMaAgK6rApYRZQF4lQ8sEpuXX2j4ttT2uKbBRVlQBIAB3IrRlreM5tIJ4LhhvznMBlmZFFQAAgFAUqgAAAITi1l/YgLXDKIRZjDP1uJx4jUv3dejQqxz7hviEssHyCp9PzWUuZkUVtsEH/phKHpc13vtU+MqYUJZoY7a1YJwohLLNN2eesQ2tX2tolBVVABZR87fogmSoRc3zbAm55q47g6A8K6oAAACEolAFAAAgFLf+AkAGgoHWJSDmzlL7YeStr2H2A9AeK6qwDUI1Yip5XKKPiRr3jWCgdQmIuWM/cE6N51OwogpbcOwb76Fvy8eEUcx9/qVaCrgYuxJRal+XZJUGIA/nU2plRRUAAIBQFKoAAACE4tZfIARBNNtVOBQHILQA50jXYYqwogrbdSrgoFTwgSCa7XKMuYSAmDv2Q/tKnyNLvz8bZUUVNsq3o0DNnMPuzNkPWwxqA+phRRUAAIBQFKoAAACE4tZfgIUcua1OIAVULkCwzTHOLUBzrKgCzDMlTCTah9soBLJQk4jzOGKfhkQL84uu9H4p/f5slBVVgBmOrWIMBZTwVCthMI47jGP1dxr7i62yogoAAEAoClUAAABCcesvLGQgcEPoBUAwztkAsVhRheWcCreoLfSC8gSPwPLGnrMjzruIfQKYxYoqQHBWcyAO8xFgHVZUAQAACEWhCgAAQChu/QWyy/17kmNfb8b7NhuWMhAQE02zx2COio7fGI4xZFT4/GA+szgrqgDtFALH1LJttfRzbS3tl5a2pRWC2upWck6ZzyzOiioAwAZZEQMis6IKAABAKApVAAAAQnHrLwAATTgSqif0ByplRRWg7eCQWratln6uraX90tK2UI+WQ39KzinzmcVZUQWy2+123dTnDP20zOPXy/241llJqJvjB5zi/EDrrKgCAAAQikIVAACAUNz6C8Bs+/3+ZQr2t2BDt39fQCDLhmUeS2sxZoGqWVEFanMqwEGwQ1mhitQFtL59tMeYjc/1DAZYUQWqYoUAgBa4nsEwK6oAAACEolAFAAAgFIUqAAAAofgbVdiAgUTWOamQt6deM8jrcULEhN4KGIfUxpgFqqZQhW04VZRcXKzkDoEQKrGqTRSpu92uK90H2mAsAazPrb8AAACEolAFAAAgFIUqAAAAofgbVZgoRzDRfr/vL30uAHVqMMjMtQtYjBVVmC57MNHM58JUW0gDLbGNp95zC/ubcVo716+xPeYPbJQVVYCNsQKyDPsV8ptxpxJQOSuqAAAAhKJQBQAAIBS3/gIXyREqtUJfTj0+5y1iTYSJBAh5qWo/Bthfa6jqmECrIl1vYU1WVIFLLREqdamSBUMrxUrp7Sj9/lPV1t9LlAzKaTVAp7Xtam17oop0vYXVWFEFAIrY2mrQ1rYXYA4rqgAAAISiUAUAACAUt/4CYW0ksIbMjBuiCjo2BfI0JOgYu5SxuXFWVIHIarjYthImUno7cr5/DeOmBqXHRIsijs2IfYqg1qCvlo5nS9vCBayoApyw2+260n1Yi2+tY3s8Fod+XmlLYxaW5JwI5VlRBQAAIBSFKgAAAKG49RegQgOBGcIn2KylgmSGbreGS+UYr8YmLbOiCkRWMrSi1sAM4RPxj91UrW3Pkoz/6YyvcozXYcbmxllRBcKaszIocGa7jBtqZHwRmfFJCVZUAQAACEWhCgAAQChu/YXGLBUmAgBrWyM4TjgdxGRFFdqjSAW2qtbwlVr7vYY1guNKhdPVctxr6SeNsaIKADTB6hc1GTtehbyxVVZUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUOdF3qui592HWpG2oDAACWoVCFpz5IKf37+38PtQEAAAtQqMJTX6SUfvf+30NtAADAAvyO6gj7/f5lOv6jz7d+s609fZ/6lNLnp9r2+xK9gnYNnGNDGfotwzNcKyjOZ5n5Ip6rZpyXTjEeKtTq/LaiOs6pk1KokxXh3JbuAFSi9XNp69tHHVr/LHPqmpvzWtzKvhqyhW1sUZPz24oqHLgPTPogpfTF/UrqO203N+Ne59g3WAt88wkAm1fzqhFwnBVVeEqYEgAAFKRQhaeEKQEAQEFu/YUDwpQAngoQJHNRKEirISMArbOiCutaI+yB8hznaVrfL61sX+lQjkvfv8mQEYpoZS4P2cI2UgkrqmzWsdCkU+2XhCkd49v7bXCcp4m0v4YCz3a7XbdmX4BYxp6rcp9HpryecxgtsaLKlp0KSBKmBAAABSlU2bJTAUnClAAAoCC3/rJZx0KTTrULU2qL37MF+JJzIhCRFVXYtpZDE1reNpYjCOu00vug9PsDsCIrqmzCuYCktcKUSttSkMKx0AurBpwTKdgpGvsGgDVZUWUrpgQkCVMCAICCFKpsxZSAJGFKAABQkFt/2YRzAUlTHjsUpnTk1tJbt8uxthm3ODc7Xvf7/cuU0vul+wHks8afc+R+j5GvN/pcfGn/NnBObPZ6tiVWVGFZLV8EaiSMZVjL47XlbQPassb5qvVzYuvbtwlWVNkEYUqkFC8MRrgTEEHkoD3nSdguK6pshTAlAACohEKVrRCmBAAAlXDrL5uwVpgSMGwDAR6wqoE5JUwGqJoVVZhOIA+5RBtLa/RHkQp5nZpT5lq91jgXR7v+5Nb69m2CFVWaMzY46dIwpb6/+4b63YCl/Zu1to92zFntGAoYiRyMUrMo+1W4DNzJNSdzn09rOD9bbacGVlRp0dgwpLlhSgKWAABgAQpVWjQ2DGlumJKAJQAAWIBbf2nO2OCkS8OUjrUJWAKWMDV8qtBtwUJ7KOLIeDcWBzg/UBsrqgCsqdaAi1L9riEQp4Y+sg2tjsVaz5sptXtMWIEVVZqzfJjS07abm7W2Durmm3WAaU6dNwWr0TorqrRImBIAAFRMoUqLhCkBAEDF3PpLc4QpUbOp4TknXiPS7WCCNDZgxpgLPT5yzEcALmNFFbjUqXCHmkMfImjtQ3Fr27O21udT9PERvX9TOGdTgvHFxayo0hxhSuuIvAoCrZgzz4KtrFNYxHP2lGsz8+12u650H2AKK6q0SJgSAMTn2gqcpFClRcKUACA+11bgJIUqzen71Pd9+vzxLUNj23I8HwA4z7UVGOJvVAFiuU3bCHBheTWMpejjo4Z92LqixyD333ofeb3QyddQkkIVIBAfWMjFWJqvhn3YemjWsWPQ2Db7IgROcOsvVeu61HVd+vA+JXB2W47nAwDvWuLaDLRNoUrt5iT0Sv0FgHUscW0GGqZQpXZzEnql/gLAOpa4NgMN8zeqVO0+FfDzXG1jHrvf71+mlN6/ucmwAZV42OZCby9ogqoVnj9zmHtkc+m1eeo1t7G/X4VNs6IKI3SfdF33Sff97pOuS/E/cJ5K0ZyTrllym6Pvbzin1jFca78PLXFOZD2tjMNTjEM4wYoqVbsPVfggpfTFw2+uzWk71t590nWpT3+WUvpRSumnfd+nroub5WAFBOBLzonrW+LaHMlutxv8EDC0qnvuucCXrKhSu0XDlO5XUP8s9c9+L3WpS/2z3/vp//pp6vuQ104AiGCJazORdN1vpq7754/++c3SXaI9VlSp3XJhSt//yRcppT9LKf0wPXvz9ZRSSs/efP2zf/gspZTSj/7pj0KvrAJAIUtcm4mi6347pfTXKaWvPGr9f6nr/kXq+/9WqFc0SKFK1ZYKU+o+6e6K1F9/4/fTb/zynf//q/5X6T//779JKSlWAVomSO4yS1ybW7T2+BoZNDU87rrut/8hPf/F++nlO7dlvkkp3abnv/hW1/2OYpVc3PoLB97e7pvSDw+L1Ld+45fps3/4LG3oNuCSYQ+CJqhdrWO41n7nJEiuvGjjcEx/xgZ4RTzGp/t0d3vvXx8WqSndFRTvp5cppfTXbgMmFyuqVC13YMPb4KT+2e+9vd33hF/1v0pbuQ241m/1IQLzh5Zlvw4ftD3Mn8ftNzf7N6f6c329ezb1fYZe75Lwo4bn/D9JKX3l1CrXs5RSn9JXfpJ+8r1/26X/UFNAFjFZUaV2uQMbvpdS+tG5IvXBr/pfpZ///c/T3/7yby/oOgBUL/d1+GzQ4YL9YaY36dmzT9NHf5LsazJQqFK73IEN/zWl9NP05tn/HfPmX+u+ln7wj3+Qfusbv3VB1wGgermvw8NBh+dDloQ2FfQsvXnzUfr0j5J9TQZu/aVquQMb+h/3ffdJ9wepe5NSSj9MKf2jU+/9te5r6bvf+m7zt/1OUTh4ZA1Fwk02sF9Tqjg45kHQ41T9fiW27NfhgaDDh/b9Pm9/hl4vl6Dnh6NOhS699xd/kf7VH/5h+uovf3l0petNSull+tb7f5L+6NOUUrr/aHTb9+l52khAFnlZUYUD/Y/7PqX0Bymlv0y//sbxB/36G4rU46q4CM9Qavta368ptbGNEbchYp9qIkiOXKqfi6+/+c30808+SbfpeTr8o977IjV9N32W/j69k6VU/XZTjhVVqpY7xOHL9v6D9P2f/EF6/3/+fvqdv0zvpP/++hsp/eKH6Uc//JeKVICGWY1+19LBSafaDttvbvL2cej1eNfLb387/c2/+zfpBz/+cerefFmu/p9XX0vfTZ+l/55++8lzhClxKSuq1G65wIb/8pMP0s/+PKVf/DC9XVm9L1LTz/5ckQrA1qwRnCRMKbiX3/52+o9/+qfpP/3xH7/955+l/3G0SL1nX3MRK6rUbuHAhi6ln/35Xet3/uJtkZqSIhWAzVkjOEmYUgVef/Ob6fU3v/n2vw9u9z1kX3MRhSpVWzqw4W7R9L5Y/cW/TunvvpcUqQBs0RrBSa2GKY318ccfpVev3hv56D7V8Jnk1DGFc9z6C6N0Kf3d99MFF4SthWG0vr2ltq/1/ZpSG9sYcRsi9gm2aNRcHF+kplRDkZqcg5jBiipVWzqwYWp/drtdFVeNpawRPHIqNv/+/Zvc/wJd6uA40ZJSwUkthSm9+75354dzfUnpSaBubZ6lkccUzrGiSu1KBjYAQKtKBSe1FKYUqS9raX37WJFCldqVDGwAgFaVCk5qKUwpUl/W0vr2sSK3/lK1dcKUlrXf71+m4z+IfetWwu0ZGA+1anIcm7f1aHBOraJUcFJLYUoHnyeejMMWf+VuyjGFc6yoQnmnPkD5YLVNrR331rbngXlbD8eECBYYh+H+3FNwElkpVGlO16Wu69KH93+8P6ltqB0AWjTnurlG21B7zm2ZY4H3eJZS+k5K6dnNzT4d/+evUt+n7vFj+z51Bdue+wxFTgpVWiRMCQDGixScVGuYUu73mPLcSMfKZyiyUajSImFKADBepOCkWsOUcr/HlOdGOlY+Q5GNMCWac2lgw7Ggg6mGfuMzwusdkTX4pXRoSW37i9NKjyVowchz4u39b3yGCE66NEzp448/Sq9evZfSo98hvQ8ruu37NLh9OcKUzgUnzXm9c/2LdKwEJ5GTFVX40uiLytXV6yX7sabchUDrhcUa29daGMWl29P6WGI9rc2p3Gqca0+O6X2RekyJ7Zv7nsYsJCuqNOj+D/g/SCl9cf/N3ui2U25uFsquhwNWbCGv6HNqhTtBzppz3Vyj7Wn73TF93JYeraRO3b6bm7z768zTn43Z5jn9u7Tfl3w+giVZUaVF/uAfAMaLFLwzJUwpd8DSWGv0ZYnPKD4fURWFKi3yB/8AMF6k4J0pYUq5A5bGWqMvS3xG8fmIqnR9b0X/nKHbcna7nd+JakTXjb+95erqdXrx4tMlu0M9BCwtIMLtkJdyXWCKCGO9tjGbIazoIWAppTTvc97Uvtz//ugkEcbIClxLZ2i1VrGiCl8aHV4wENrA9tQYRFIDYSJsRemxXvr9LzH3vJvzvD3ltWrc12txLeUJYUo059KwgIdvV8eGMxxz7Fur8yEJ+5PvketbsI18G7vot4Zb2YclnQtLKR3oMnbewhTnVpFyj8Xr693kMJ/SbYftQ9v3OPzw+np38nFjw4rWCE4CjrOiSouWCGxYoz+wdZHCW8xboigVBBSpbaj9UmsEMTmPwAwKVVq0RGDDGv2BrYsU3mLeEkWpIKBIbUPtl1ojiMl5BGZw6y/Nub+V5vNL2g7buww3kp57772faIWU0ry5u3abectaco/FCPNnSluG4KRT3t4yfX29OxmSePCZYFJfHt+W/XCcHt9m7DwCw6yoAgDUq/WAntGF4dXV68H/HjIyJPHivnBW6+OYC1hRZROWCGyY8z5jQxxKiRxlLtSoXZHCW2qct7Tp/Ph8Gjw2FLAUYf5cEqZ0wtuwomPb/LBC+jhAKk0MSbwkxKlFkT8X0C4rqmzFWoENQlngcpHCW8xbosg9PiPNn7nzbM42j2XeQyEKVbZircAGoSxwuUjhLeYtUeQen5Hmz9x5NmebxzLvoRC3/rIJa4UptRLKcuT22ttzv/UHc52fK/uXKaX3j4WRrN1W0sN+KN2PTJxbzsh9XYkwp77+9Y+e/E3osevtuWvwJds857qeI2ARGM+KKjBGKx+KqZtxeKel/dDSttSoyP4fGVx0jvCd9djXFGFFlU0QpgSxmCvwrjHXpBxzpcY51fepy7XNU9977HMfBzYd79/pgKtzQUVDIYJCjmiZFVW2QpgSxGKuwLumXJNyhynVoNQ25w6pAkZSqLIVwpQgFnMF3jXlmpQ7TKkGpbY5d0gVMJJbf9kEYUoQi7nyVGMhSUw05po0tm18mNK4vn388dPwo7Xl2ualwpS2fv6CJVhRBaAWrQd6bLFIbf2YRjdq/xcuUo0R2CgrqmyWMCUo55K50vd3P2Ny2XPHzsdxgSdD4SacJvjltBJhSmPnVErp5LyYaTCAaMp+KB2m5FoP+VlRZcuEKUE5c+bKGm2wtpJhSqXmxdygQ2FK0DCFKlsmTAnKmTNX1miDtZUMUyo1L+YGHQpTgoa59ZfNmhumdH29e+e/r65eC4iBkS6ZK2u0mY+UUiJM6Vhb16XVQr0uDTo81pY7TOlRgNSo256dWyA/K6owbHSIQ+lERKB6rYfGtL59rVgr1Cv0eJh4TQ+9LVArK6pw4CAQ4UnQRBr4dlWYEoyzZCDSvDClNbb+uN3uLtiG9o0dm1Mee1mY0tHgpCGjwo9yt+Xc5jPbN9bk/eBaD9NZUYWnWgmpgMgiBSeZj6xtiRChNUJ/Ss7HUmFKY1/PuQUyU6jCU62EVEBkkYKTzEfWtkSI0BqhPyXnY6kwpbGv59wCmbn1Fw6MCJoY8va24PvH3d7fPlx9wELB3428dUvkOPv9/lQIytl9OPDcRTy+De5hDhxpu73/nceqw5TW3rcXMs9WlDNE6FjbhGChSWNz2bl3N0+OnRtSOn/OGPL495EPgxAfG/p/j0U5t0DrrKjCdFNCE6J/OK2BfTjeqX01Zh9G3M8R+3SJGrajhj6S35TjvnRg0Cpj8Orq9dyXEJwEK1Gowghdl7quSx92Xer6Pj3v+9Slu/nznXRmHj1+7lDbkn1e6j1gaWPnT+42WNKUcZh7HE8Y72+vcX1/d+1rYe69ePFpurnZv/PPGVn3AzCeQhXGKRVcMYcQB1ogTIkWRQtTmtPH1ude69sHYSlUYZxSwRVzCHGgBcKUaFG0MKU5fWx97rW+fRCWMCUYIWfA0tXV63Rz8+mo950TYOQ322hB7lCWOYEsY+djweAxMlsqCGvM2Pz444/Sq1fvvfO8Y9eaY23X17t0dfU6vXjx5bXm0eud/C3wx5YMTjrWFjVsqPXtg8isqI5z6g/n/UE9D0aPhcMPHpxlnq0j4n6+tE+CgaaLePwjKDaW5l4rDp8/8fVKjIdiY3AgYMm8oBZN1ipWVEcQ2c8x9+EIH6SUvrj/CZp32tLIb63XdH29e5bu+/c4rn+M3W4nDKJhQ+e5g7Hen2qb8tgl21q8m8D8Y2Fvrw2l5u39T1FdfG4ZuqYdmz9z+g3RtFqrWFGFy9UYnBC9f8S0VvCL0BIoI9K8nXtuGcu5BYJTqMLlagxOiN4/Ylor+EVoCZQRad7OPbeM5dwCwSlU4UJ9n/q+T58/vhXoWFsk0ftHTFPG+tjHrtEGjBNp3s49t6yxzcA6/I0qLOc2nQjiuL7enX3yYWJjpv604NR+Db19S6WHTnj/pT9k3Qb6G5mTc69Socd24y4eS8dSe6fpU0rz/jR5zLUmZRxfpc9zp4w8/0U6hx06OQ5PbFvkbYHRFKqwkIeApce6bvw3sq9evSdA5YiKL77hPrxlFmb75oyRoQ+0j+fj2MfNfR/KmjOWrq/nrsB1qe/PV6rHxtJQgTrmNWcIcx64QNi+nxqHA+eRsNsCU7j1FzLqutR1XfrwPiXwZFvu15vTBjmtMWbNC9ZWciy5rgBbpVCFvCKlEkovpIRISaHmBbmUHEuuK8AmKVQhr0iphNILKSFSUqh5QS4lx5LrCrBJXd8LMYO1TPkb1Qluj/097GNRAy5oSwt/W7lC6NTahKqsoOvSIufY2v5Gtfb5s+Q5bIm/S/e37rTOiiqsa4n0zjEfjhSpLE0ybUzm/jqW2M8Xz6mrq9fZX3Okms8DNfcdmiT1FxZ2HzDxQUrpi4eVz4O2/rAtpfRmxns8eb2bm0wbw2ou+TZ86Nv16+vdszQwRnK0LfGaa7eZKxwz4pw95OK5N+axx8bsw0+brTHv323bjbrGLbMf9ievm1YXoU5WVGF5awRSCLjgnLUCVCIFIgmDIZc542atOZX7vSO1TX0s0ACFKixvjUAKARecs1aASqRAJGEw5DJn3Kw1p3K/d6S2qY8FGuDWX1jY/S1Kn09p66bfpPT2lqf75z4ELH2eUkr7/eTXozGXjMOpbWu9z5Jt5grHHJyfJwUnLT2nhsZshDmVo23MY4f2Q+0hT7BVVlQhprmhDocfooREwDitzZXWtieCKcFJ9j/nGCNwghVVKGBEgMTs0KVzARdDwRPHCKOYbk74x9p9Eaa0bhh2ULfIAAAPIUlEQVSM4Je4SgUn5QxTGvvcWtrm7odIzG8Yz4oqlLFG8IvgifIiHYNogSdbaxtqJ5Y1zrtrhQhFmgMl9wNQoa7v3bYPa8v1bXMa/hmbwW/2ragub+2fU/DzNHHbDtutqMa15Hl37Z9lWf/naUquqK53h8ocOef30DnfeYQWKFShYl2Xpkzgh4CllNL0cAkXvfxyf8jwoaUeCxz7UwE/t7vd7vmRdka4IDhp9Xk2ZywNjBsWolCF8dz6C3WbEsLgwwi069T8Nu/naT04yfhYV41jBIoRpgSBXRK6lEYGLNUSPFGz3OEf0UNV1nqfGtumHCvWEyk4qUSYUovj8Nztzm67h3pYUYXY5oZPjHk9lpM7/CP3sS8ZeLK1tqF2ymlpTs3pY0u2uM3QJIUqxPZFSul307vf7I9tG/t6LGfOscr9enPG0pT3XeN9amwbaqeclubUnD62ZIvbDE0SpgSNmRKwdHX1Or148emoxy59S9RGQj3eCbYRpnQn4LFfPIDIsY+nhuCkY8Ye+4DzbBEjAqSamSstbQscY0UV2jM6rOHVq/eyv+YMzX+AStvYxktE2y/R+sM6BCfVr8bjApwgTAkqkztg6ZhjYRQj3/vithZDPY4RplTHsY8WplTjPowmenDSEmFKkSyxwtfK+QY4zooq1Cd3wNLY95j73gIu7uQ+LrUGv0Q/9tHClGrch9FEnz9LhCm1rpXzDXCEQhXqkztgaex7zH1vARd3hClN63cp0cKUatyH0USfP0uEKbWulfMNcIQwJdiAKQFLTNWndCRT5VRQ1UG4iUCdNNzvLdrSsS+l1uCkY2qZP1HGYUtzpaVtgWOsqMI2CJhYzPHPAieCqhyH4+wX1tZ6cFI09iEwmTAlaFTugCWmOxZKtWT4Ua1hSn2/ezI+5wVz7ase2wJi5qsxOGmtMKXWV9rMFWiHFVVol0CJ8kqFt+R+vbXClJboT422uM25RZ8rwpSWY39BIxSq0C6BEuWVCm/J/XprhSkJ5rqzxW3OLfpcEaa0HPsLGiFMCQLa7/eTgj4ucX29W/LlOe724TbslIQpLaWWcJlTajr2a5yr5vr4449O/c34UWsHJ9WwDzO43e12z88/bHktnSdb2hY4xooqxLT4h5arq9cTHl315/5Ilj6upwJLthZksrXtLSl8gTWlSE1lxk74fZjBFrYRyEyYEmzUsZ9OWdK5YKGa2w7b00BQ1bLhR3lDidYLU8rdtvx+GApsGrOSMbQSIkxpVaPOS2M5dgD5WFEF1hIpoGStwJNjagx0WSL4pZW2JUTrT8ty71fHDiAThSqwlkgBJWsFnhxTY6BLrWFKtQY2RetPy3LvV8cOIBO3/gKreHyr5H7/0Pbl/79vu72/bfPzx8+9v10ubNthezd84+fb/XB9vUtXV6+P3oYdafsu3Q8ttz2M4SVE608EUwORxjq/X08GHR0NB3LsAPKxogoxbTUMppXAjdHHb4kP37CiVc5VC82TMX0/dU7Kea7awvl+C9sIZGZFFQKaE+Nf+09zRDIjROhJmE8aCFia896R2qL1Z9kwpSlHc5po/RmS6ydHzm1fmjh/TsganJTLEj/b4mdLgBZYUQU4bYkQodzvHaktWn+EKdVjje1rfR8CNEWhCnDaEiFCud87Ulu0/ghTqsca29f6PgRoilt/AU7IGSI0FLB0fb071vz2Vsdjz43eduaxt/e3R4cLSRrbJkwpr4O5cirAKNt7DLUBEIMVVSCSlgM3Wt62qVoJzZrj1HgwTpYZH/YrQGWsqMKGCNFYzogAnNkBSy2JEIhUMkzpIUAnSn/WMiaE68xLjApEmhIABkBMVlQB8thiAM4ckQKRSh67aP1Z2pQQrrHPb3l/AWyWQhUgjy0G4MwRKRCp5LGL1p+lTQnhGvv8lvcXwGa59Rc4a7/fLxJuMsPtEr89OMclAThDAUsbcBgWVVXAUq7womj9WdrhdkwNTsp9TCMbOO+GO/8NCXj9OGnG75BXdUygFlZUgTGifciI1p9LCXj5UivHlGmmHPetzZdT+6a2uVJbfy+xhW2E1SlUAVbUdanruvRh16Wu79Pzvk9dujsXfyel9KzvU1dr25jHjt03tbRNEa0/S8vQ58Ox9Dz6NgOQj0IVYF2RgoCWCKaZE2ITaVuWCOOJ1p+lze1zjdsMQCYKVYB1RQoCWiKYZk6ITaRtWSKMJ1p/lja3zzVuMwCZCFOCFZUOlZgRFLEpSx6nx793+RCAs0Lbbd/vFg8qOmx/2I+P+3N9vTt8ymPVBCxdEl4UrT9LOwgOezKnzoWJtRCIBMDlrKjCugQu5LF0qEprx6nU9jx536ur17OeT7WmHsutBSe1bAvHstQ2nnrfLexzNsCKKrCK3W4n/KSg+/CZD1JKX9yvSmVvO2x/vJL64MWLT1NKKV1f7549PC49Wkkt0e9L2o5t21CfS/ZnDef6N/Tcvk/dlDFGXfxsy3LsW1pnRRVgG0qGKY3tT6l+5w4vmhs0VWOYkuAkALJSqAJsQ8kwpbH9KdXv3OFFc4OmagxTEpwEQFZu/QXYgDXChg7bhwJ+DoJ2hoQMWBq7baef/zRoak5o1pA1QtQe9+XrX/8ovXr13qTnC04C4JAVVVjXVgMOatvu2vp7TvTtmdK/VgKWWtmOJyYWqdHHJgCFWFGFFa0RfDC0eiLQaJzaAypKhQ0dtk8IHHp+2JaCByzNDVMqHX5U0NsQrXNBUwBsmxVVgPaUDBbKHZI057k1hCltjX0DwCgKVYD2lAwWyh2SNOe5NYQpbY19A8AoXd+7wwZa4tZfopgzFrtu0u2fDwFLq5k7z9YIOBrr44+nhx9dqu9T2HPQQ8BV6X5syG3tf2YBLMuKKgDhXF29nvJwxcUMaxWpKX5wknG0LvsbGKRQBdiorktd16UP74NsZrcNtU997xcvPk03N/t3/llzW6Zs85i+THl+Q56llL6TUnrW96nr+/R8g/sAgAspVAG2q9YwpTmvJ0xpPbmPMQAbolAF2K5aw5TmvJ4wpfXkPsYAbIgwJWiMMKU7DQajVBc8MmcsHnvu9fXu5OPP3Rq8pshhSoKTTosUcLUVI+dKyXN5deddaIkVVaBVLRWpKbW3PVtXJFhIcNKgGvtcs7H7u+S5z3kXCvpq6Q4AENt98M0HKaUv+v7uZ2OOtR2239xc/ppDz41uzL7p+7tVmrH79vz+2r851Z/r692zh8ellE4+Lt19eT27L+fGSFTHVs5K3qHi7hhg66yoAnBODWFKkUzZNyXDncY8bo3+AcATClUAzqkhTCmSKfumZLjTmMet0T8AeEKhCsCgvk9936fPH9++eaxtqH3Oa9Zmyr7J3TalP2Met0b/AOAYf6MKhNVgcu8cmw96ubp6fTIMaCgReG3X1yUKs126unqdXrz49G3LQ8LvyP5sfnydcJuOn4Psr/WcOgZrvTdQiEIViKyKIlWwyToeirDH+7vrrNY9OCzihxJ+a/vpmFL8NEl5jgFsl1t/AahK16Wu69KH96myXODYPrRfAYhEoQpAbaTJzielF4DQFKoA1Eaa7HxSegEIzd+oAlCV+xTZz1NKqXOT6jvGhko93odDbZEJWwNomxVVILIaEhdr6GPL7P/pWtlnilSAhllRBcKS9sgx92E/H6SUvuj79PxIWx+tben3SSm9Gdhlz8b2EQCisKIKQG3GBgFFalvzfQ4JTgKgOgpVAGozNggoUtua73NIcBIA1XHrLwBVGRsEFKlt6fcZCpVqITgJgO2xogrtORWU0kqACvDUFud9y9sGsHlWVKExAohgG+aESrWg9XPdfr9v5lgBXMKKKgDUKXfoEgCEoVAFgDrlDl0CgDDc+gsAA/b7/cuU0vul+3Hgtu93z9OMcCcAiMyKKgAMi1akphSzTwCQjUIVAACAUBSqAAAAhKJQBQAAIBRhSkA2QUNncrpt/bcbAciv8uujax9FWFEFcqr1IjxW69uX2+3E9qgi9jdin8irlfnDnZqvHzX3nYpZUQVgEa18A9/KdlAX4w7YOiuqAAAAhKJQBQAAIBSFKgAAAKEoVIGcWg/5aH37IhEkA7Sk5nNXzX2nYl3f96X7AAAAAG9ZUQUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEMr/B8wj0o1qhF8ZAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (1.4) A* search search: 154.2 path cost, 944 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3b+PNMl5H/DqVzxxRfNeWpEdEXBmCBSOzGmQkENHBwOzgAniIgpMpP/AIM+BEmdSQoJg8AYKdgzjYAjObMEGD1B6B9pKrcgwHInvwvRLvvA7Dnb3vX1np3u6p6u7nqr+fIADeXXzo7q7qnuerZ7vdIfDIQEAAEAUz0p3AAAAAB5TqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEL5UukOANCm/X7/MqX0/on/dLvb7Z6v3R8AoB5WVAFYyqkidagdACClpFAFAAAgGIUqAAAAoShUAQAACEWYEvCW8BuIxZzkMeMB2BIrqsBjwm8gFnOSx4wHYDMUqgAAAISiUAUAACAUhSoAAAChCFMCNk04CY8ZDwAQgxVVYOuEk/CY8QAAAShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoXypdAcAAJhnv98fjppud7vd8yKdAcjAiioAQHveL90BgDkUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIReovAAA8st/vX6bTgVTSlGElVlQBAOBdfanJ0pRhJQpVAAAAQlGoAgAAEIpCFQAAgFCEKQHMMDZwY+BxrRAwAgBkY0UVYJ6xgRstF6kptb99AMCKFKoAAACEolAFAAAgFIUqAAAAoQhTgo2qIdyndB/3+/3hqElgEJtVej7OZO4GMjaELkB/+h5/fG0oxbimaVZUYbtq+MAZrY9z+nObrRdQRrT5OEXNfW/R2BC6tdQ6PmrtN4xiRRVgBXP+6h3or/cAAKuwogoAAEAoClUAAABCcesvTFRJoIiABQAAqmVFFaaLXqSmVEcfx4gWQFSqP9H2wyk19JF5aj7GNfed5dU6PmrtN4xiRRUI69Sq8FCw0G6365btURlWx4nAOKRVpa41W7yewRRWVAEAAAhFoQoAAEAobv0FCG4gwEtoVkOmBrVl/n1dYymzCMF7rY4R50TYBiuqMF0N4QW5+1jDNres78NuK6FZ3Cl5PI2l/Frbp5G2xzkRNsCKKky0hb/WCnEAAKAkK6oAAACEolAFAAAgFLf+AhcRZgHA0gqHjAEFWVEFLiXMAvIqGVomMC2/1vZpqe1xTYGNsqIKAAG4E6Etax3PoRXEc8F4c54LsDQrqgAAAISiUAUAACAUt/7CBqwdRiHMYpypx6XnNS7d16FDr3LsG+ITygbLK3w+NZe5mBVV2AYf+GMqeVzWeO++8JUxoSzRxmxrwThRCGWbb848Yxtav9bQKCuqACyi5r+iC5KhFjXPsyXkmrvuDILyrKgCAAAQikIVAACAUNz6CwAZCAZal4CYO0vth5G3vobZD0B7rKjCNgjViKnkcYk+JmrcN4KB1iUg5o79wDk1nk/Biipswam/eA/9tXxMGMXc51+qpYCLsSsRpfZ1SVZpAPJwPqVWVlQBAAAIRaEKAABAKG79BUIQRLNdhUNxAEILcI50HaYIK6qwXX0BB6WCDwTRbJdjzCUExNyxH9pX+hxZ+v3ZKCuqsFH+OgrUzDnszpz9sMWgNqAeVlQBAAAIRaEKAABAKG79BVjIidvqBFJA5QIE25zi3AI0x4oqwDxTwkSifbiNQiALNYk4jyP2aUi0ML/oSu+X0u/PRllRBZjh1CrGUEAJT7USBuO4wzhWf6exv9gqK6oAAACEolAFAAAgFLf+wkIGAjeEXgAE45wNEIsVVVhOX7hFbaEXlCd4BJY39pwdcd5F7BPALFZUAYKzmgNxmI8A67CiCgAAQCgKVQAAAEJx6y+QXe7fkxz7ejPet9mwlIGAmGiaPQZzVHT8xnCMIaPC5wfzmcVZUQVopxA4pZZtq6Wfa2tpv7S0La0Q1Fa3knPKfGZxVlQBADbIihgQmRVVAAAAQlGoAgAAEIpbfwEAaMKJUD2hP1ApK6oAbQeH1LJttfRzbS3tl5a2hXq0HPpTck6ZzyzOiiqQ3W6366Y+Z+inZR6/Xu7Htc5KQt0cP6CP8wOts6IKAABAKApVAAAAQnHrLwCz7ff7lynYd8GGbv++gECWDcs8ltZizAJVs6IK1KYvwEGwQ1mhitQFtL59tMeYjc/1DAZYUQWqYoUAgBa4nsEwK6oAAACEolAFAAAgFIUqAAAAofiOKmzAQCLrnFTI277XDPJ69IiY0FsB45DaGLNA1RSqsA19RcnFxUruEAihEqvaRJG62+260n2gDcYSwPrc+gsAAEAoClUAAABCUagCAAAQiu+owkQ5gon2+/3h0ucCUKcGg8xcu4DFWFGF6bIHE818Lky1hTTQEtvY955b2N+M09q5fo3tMX9go6yoAmyMFZBl2K+Q34w7lYDKWVEFAAAgFIUqAAAAobj1F7hIjlCpFfrS9/ict4g1ESYSIOSlqv0YYH+toapjAq2KdL2FNVlRBS61RKjUpUoWDK0UK6W3o/T7T1Vbfy9RMiin1QCd1rarte2JKtL1FlZjRRUAKGJrq0Fb216AOayoAgAAEIpCFQAAgFDc+guEtZHAGjIzbogq6NgUyNOQoGPsUsbmxllRBSKr4WLbSphI6e3I+f41jJsalB4TLYo4NiP2KYJag75aOp4tbQsXsKIK0GO323Wl+7AWf7WO7fFYHPp5pS2NWViScyKUZ0UVAACAUBSqAAAAhOLWX4AKDQRmCJ9gs5YKkhm63RoulWO8Gpu0zIoqEFnJ0IpaAzOET8Q/dlO1tj1LMv6nM77KMV6HGZsbZ0UVCGvOyqDAme0ybqiR8UVkxiclWFEFAAAgFIUqAAAAobj1FxqzVJgIAKxtjeA44XQQkxVVaI8iFdiqWsNXau33GtYIjisVTlfLca+lnzTGiioA0ASrX9Rk7HgV8sZWWVEFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQ/I7qCPv9/mU6/aPPt36zDWCegXNsKEO/ZXiGawXF+SwzX8Rz1YzzUh/joUKtzm8rquP0nZRCnawI57Z0B6ASrZ9LW98+6tD6Z5m+a27Oa3Er+2rIFraxRU3ObyuqcKTrUpdS+iCl9PnhkA7HbTc3417n1F+wFvjLJwBsXs2rRsBpVlThqQ9SSv/+/n+H2gAAgAUoVOGpz1NK//L+f4faAACABbj1F47c3+77WV/bfl+iVwBlBQiSuSgUpNWQEYDWWVGFda0R9kB5jvM0re+XVravdCjHpe/fZMgIRbQyl4dsYRuphBVVOJIrTOkUf73fBsd5mkj7ayjwbLfbdWv2BYhl7Lkq93lkyus5h9ESK6rwlDAlAAAoSKEKTwlTAgCAgtz6C0eEKbXP79kCfME5EYjIiipsW8uhCS1vG8sRhNWv9D4o/f4ArMiKKpt1KjSprz1XmFJpWwpSOBV6YdWAcyIFO0Vj38Rx7jrV2rUL2CYrqmxZX0CSMCUAIht7nXLtAqqlUGXL+gKShCkBENnY65RrF1Att/6yWadCk/rax4Ypnbi19Nbtcqxtxi3OzY7X/X7/MqX0ful+QA6Pr0ldl56M7e7uSx63h0N6nhoOAlzj6xy532Pk640+F1/avw2cE5u9nm2JFVVYVssXgRoJYxnW8nhtedvYtr6xbczXa41j1/r4aH37NsGKKpu1xTClrYv211XhTsA5565JY587dO06FbR3SWDTEm3Ok7BdVlTZMmFKAEQ355o059o1J7ApdxuwQQpVtkyYEgDRzbkmzbl2zQlsyt0GbJBbf9msJcKUgGEbCPCArM4FJ53x5uH/XF/v0tXV6/TixSeT37d0G7BNVlRhOoE85BJtLK3RH0UqXG7W/Hn16r1c/WBZa5yLo11/cmt9+zbBiiqbMDbEYdxj7wJ53g2p2L9JMNGccKehgJFTwSjMF2W/CpdZVpQQoVP9Ger3zc0Xt/tcX++q3ea+/XBKrjmZ+3xaw/k5WrggnGJFla2YEtgg8AFgu6KFCK1xrYm0za6tQEpJocp2TAlsEPgAsF3RQoTWuNZE2mbXViCl5NZfNmJKYMMlgQ8CloAlTA2fKnRb8G1LtxGeCy/qTty4uUbbkiIFJ00JUzox3psai7k5P1AbK6oArKnWgItS/a4hfKqGPl6qim27unpdugtRVHG8LlDreTOldo8JK7CiyibkDVN62nZzs+bWQL38ZZ1Izp3bS/btjGcpY5hfpOCkKWFKW9F33hSsRuusqLIVwpQAOFbruT13HyMFJ9Ww/4EVKFTZCmFKAByr9dyeu4+RgpNq2P/ACtz6yyYIU6IWU8Nzel4j0u1ggjQ2YMaYKzo+zgUnRTXndt+e31h9+3r3QU63h0N6noKHKQFts6IKXKov3KHm0IcIqvigPEFr27O21udTpPERqS8p9Xw9c0xwUoZwpWD7goq1fg5jQVZU2QRhSvlZJYPlzZlnwVbWQxobnHRzU9dtMy9efJJSSun6evc2dCk9WjUdQ5hSe3a73co/fATzWFFlK4QpAXCs9fP4nO0TpgQUpVBlK4QpAXCs9fP4nO0TpgQUpVBlEw6HdDgc0mePbyM61TblsX3PB6AOrZ/H52zfnGth7jZgm3xHFSCW29RWkIkgjXJqGEsVjI9m6qVJ46Hrnmz4QxLw2oqO49zf9T7xepLRoYdCFSAQH1jIxVjKpQsdQjO2kBpbZJ4oUB8UKRZPjePGgsKi/zEJinHrL5vQdanruvTN+zTB3rYpj+17PgB1GHsen3NdmHtNmXP9yX09i7YfgLYpVNkKqb8AHBt7Hi+Zdjvn+pP7ehZtPwANU6iyFVJ/ATg29jxeMu12zvUn9/Us2n4AGuY7qmzCfXrgZ+faxjx2v9+/TCm9f3OzWHfDedjmQm8vaIKqFZ4/czQ79z766MP06tV7KaX0Zszjx15DcreNeex+n7ff3fCNtW/31/3jHgKWFtsPU6+5jX1/FTbNiiqM0H3cdd3H3Xe7j7suxf/A2ZeiOSdds+Q2R9/fcE6tY7jWfh97cu67L1Ivfn7jpmzvGmOklXHYZ2vjC0azokpz7oMWPkgpff7wO2xj2061dx93XTqkn6SUfpBS+tnhcEjdmT85l9TqCgjAWO+ex+/OiY/b0vBK6rN04TUkd9uYx+a4u+foPSbtr7X2QyTnUqCHVnUjJ0hDNFZUaVG2wIb7FdSfpMOz76cudenw7Ps/+18/S4dDyGsnAHdqDAxaIkxprOj7i2i67vdT1/3TR//8fuku0R4rqrQoT2DDd3/8eUrpJyml76Vnb34vpZTSsze/9+mvPk0ppfSDf/yD0CurABtWY2DQEmFKY0XfX0TSdd9IKf0ipfQ7j1r/X+q6f5YOh/9WqFc0SKFKc3IEV3Qfd3dF6m+/8sfpd3/9zn//zeE36T//779JKSlWASI6CgeaFGZ1LjzvIbxohbbb3W73fE6Y0ljRA5ZatXbQ2sigqeEQta77xq/S81++n16+c1vmm5TSbXr+y6913R8qVsnFrb9w5O3tvil977hIfet3f50+/dWnaUO3AZcMexA0Qe1qHcO19vvYlELgeJsFyd0pGbAUbRyO6c/YUMNIx/hBf5/ubu/9xXGRmtJdQfF+eplSSr9wGzC5WFGlObNCHB6Ckw7Pvv/2dt8evzn8Jm3lNmABTXA582dZ587vZ54+GJwU4WfI1ghTGvG+xQKWHubPu9u8733v6+vd5DCsode7JPyo4Tn/j1JKv9O3yvUspXRI6Xd+nH78nX/Tpf9QU0AWMVlRpUVzAhu+k1L6wbki9cFvDr9Jf/33f53+9td/O6/HAFxqjSCgkkr1MVLA0lrvzUxv0rNnn6QP/zzZ12SgUKVFcwIb/mtK6WfpzbP/O+aNvtx9Of3RP/yj9Adf+YN5PQbgUmsEAZVUqo+RApbWem9mepbevPkwffKnyb4mA7f+0pw5gQ2HHx0O3cfdD1P3JqWUvpdS+gd97/Pl7svp21/7dvO3/U6xdjBEAcMhEwvZwH5NqdC+zSnocap+v56TKzjpVFuOoKK5SvUxUsDScfvQNl/yGWCN4xz0/HBSX+jSez//efoXf/In6Uu//vXJla43KaWX6Wvv/3n6009SGj72MIYVVThy+NHhkFL6YUrpL9Nvv3L6Qb/9iiL1tCouwjOU2r7W92tKbWxjxG2I2KclzQlOuvQxS4kWInSsZMBSjarfB6+/+tX01x9/nG7T8ydfUL4vUtO306fp79M7WUrVbzflWFGlOXNCHL5oP3yQvvvjH6b3/+cfpz/8y/RO+u9vv5LSL7+XfvC9f65IhQ0pFWozV85gmyXacr7mmV0xOWTncHga5BNt36wUpnSqj6sFLB23D23zJe8Tde5G9PLrX09/82//dfqjH/0odW++ONz/59WX07fTp+m/p288eY4wJS5lRZUW5Qls+C8//iD91U9T+uX30tuV1fsiNf3VTxWpsD21BrLkDrZZKygndyhOpG1eYt/kFmn/r/k+nPHy619P//Ev/iL9pz/7s7f//JP0P04Wqffsay5iRZUWZQxs6FL6q5/etX7r52+L1JQUqbBBtQay5A62WSsoJ3coTqRtXmLf5BZp/6/5Pozw+qtfTa+/+tW3/350u+8x+5qLKFRpzpwwpeP2u0XT+2L1l/8qpb/7TlKkwjadC+25vt71PTMVPm8cB9u8I1Jbjuf3mRW0V6htzGNLhSmdapsYsPSOgbbbwyE9rz1M6ZSPPvowvXr13oxXKH5uGaVvbMM5bv2FUbqU/u676YILQvQwjNxa395S29f6fk2pvm2cEBAS/4PkBtQ2vmq1xH5uJYznyb6ZV6SmVMm5xdzjYlZU2YRLAxumvs9ut6viqrGUNX4Goy82//79m9z/rf+8yINIAT8zQ3so74LgpHhtYx67ZhBQ7oClS9+7ljClc8FcKdO+CWbU3IMxrKiyFUsENgB5RQqxEb5St0hjpNYwpVNKzpUaw5QiHbu1bHGbWYhCla1YIrAByCtSiI3wlbpFGiO1himdUnKu1BimFOnYrWWL28xCusPBCvw5W7zVkDtd13+Lys3N0/SFS8bDfr9/Espy73Yrt3xO0fp8HBgPtbpoHJ8KK4KxDocvvrzX4JwaLcI5ceg6OsfV1ev04sUnT9ovvA4vfl3Zyjnt8dxjPa1+NrKiCuX1Xbiav6BxUmvH/dLtaW0/sJ7j8BZjqaxFwnTmBxGtboFxGG6xSXASWSlU2YSuS13XpW/ef6G/t22oHchnypyM7OZm/+SfdHdt/VZK6dnhkLqHf061a1vkNZ/XOJZqdW4u3/+0zMXHM3d/htrmmPkeg/vh1Hnm7p9/l0rO3RNt5h5ZKVTZCmFKEEvLgRtzQ3G21rbm+5BftGNXaozMeY+5/Ys0n809slGoshXClCCWlgM35obibK1tzfchv2jHrtQYmfMec/sXaT6be2QjTGmEVr+gzLumBh2cClOqUNbApg2EliwecDV0vqnVufNkzSEjuULVWE6Lcyqz0MF9GcKYbh9+37VP7s95U89p525x9jmUc1odI1ZU4QujLypXV6+X7MeachcHVRYbE6yxfa2FUYzZnirHTUPngda1Nqdyiz7/5h6/Ets35T2NT+jxpdIdgDXcf6n/g5TS54fD3V9nT7X1aWT1lApEXtnI6fH8G3qcucdc0edU7Su+Y6+vl7Y9rIYenTPe5OzjzU3ebT7z9GdDzz33eQS2xIoqW+EL/xCL+QdtKBm4lbuPa7yezyMwkkKVrfCFf4jF/IM2lAzcyt3HNV7P5xEYya2/bML9rTSfnWvrc329e+ffr65epxcvPsnWv5Jqv+1sbSf2V+ggkqgez79uIOah5bkHLRh7fZ3Tdtw+dM7o8fZW4fvnPgQsfZZSSvuJ3zA46svU4KRZn0eONXQNdy3lCSuq8IXRgQavXr23ZD+oS/QgkhqYe2xZ6TCd0u9/iUgBS4KT8nAt5QkrqmzCWoEN19e7USEJ74Y47HvfI1ekeEN/cR20ZAT7VvbhWo7mxaS5d8k8y902NG9hirmrSFPPTWv8VEXuuXei/ck549ScPL4jo6+PQ2FKgpOgHCuqbEW0wAbBCWzdGmEka4W8AO9aK0wp9/V67OMEJ8EKFKpsRbTABsEJbN0aYSRrhbwA71orTCn39Xrs4wQnwQrc+ssmrBXYcElYxNQQB2jBuXkxNPfWCG8xb+FyS4cpnWq7YE6+vVX4+nrXG9Q2Jzjp8e3ID/17fJux8wgMs6IKAFAvAT09rq5ej37syKC20UXqlPcmpWQcc4IVVTZrTGBDjtc8H8oyd0vyWyNs41JCjepzybzI/Xr5w5Ry7BmY71QYU8TzZOYwpVFz8mGF9HEAW5oYkjj2vHRz0/byaOTPBbTLiipbJkwJ1pF7XghTgvqUDFNaI3QJyEyhypYJU4J15J4XwpSgPiXDlNYIXQIyc+svmyVMqd+J28Zu5/7WH9t1ybwYmntjA0qWbCtpv99PCnQJzrmloDXH0ok5dXs47J6nFcKUcl3Xpz4XmMeKKjBGKx+KoQUtzceWtqVGJfe/Y18PQUcUYUWVzRKmBOvIHaYEtGHpMKVTz53Tx6HHPQ5sOt2/fW+I07mgoqFwLCFHtMyKKlsmTAnWYV4Ap7QUpuScBpkpVNkyYUqwDvMCOKWlMCXnNMjMrb9sljAlWEfuMKVWNRaSBGeNPTeMeWzpMCXXesjPiioAxLDFIlVIS1kl979jDwyyogpHhClBXrnDlM6FlsxpGxt4MhRuQj/BL7FE/GmgWsOUXOshPyuq8JQwJcgr97yY83rmI8QmTAlIKSlU4RRhSpBX7nkx5/XMR4hNmBKQUnLrLzwxNjjh+nr3zr9fXb0WpgQn5A5TuuT1xraZj1BWlDCljz76ML169V5KKfV+HSBX/4DTrKjCsNFhD/cXNIBLtR4u0/r20ZCJ13RjGxZgRRWOHAUiPD9uSwN/XRWmBE/lDlPKFZwUbT5GDLaBKEqFKfWYHOjmWg/TWVGFp0qFvECrhCkBc5UKU5rTF+cWmEGhCk+VCnmBVglTAuYqFaY0py/OLTCDW3/hyJyQl/TotuD7x93e3z5cfcBCwd+NvHVL5Dj7/f5lSun9E//p7D4ceO5sj295exjv59qOw8oeqy1Macl9m5F5RlHn5sm5c8aQx7+PPHRuGfpvj0U5t0DrrKjCdFNCE6J/OK2BfThe374asw/t5+XUsG9r6CNtW2UMXl29nvsSgpNgJQpVGKHrUtd16Ztdl7rDIT0/HFKX7ubPt9KZefT4uUNtS/Z5qfeAEsbOqTltQJtevPgk3dzs3/nnjLfX+sPh7jOAcwasQ6EK46wR/JKbEAdaJUwJWItzBhSiUIVx1gh+yU2IA60SpgSsxTkDChGmBCPkDFi6unqdbm4+GfW+cwKM/GYbtfjoow/Tq1fvjX78+dCSu1CWS0KchoydjwWDx8iscBCWgKsAxgYnAflZUR2n74vzvlDPg9FjYcoHclJK5tlaiu3niXNiTD8FA01nnp1WcixtbRwXG4MDAUvmBbVoslaxojqCv2hyyn2Iwgcppc/vf4Lmnbb0aCU1iuvr3bN037/Hcf1j7HY7oRENm3qeOxr/h6H2c21peK48G3ruqbYW7yYw/2jd3M9aQ3cynJo/l5yrHp/rIJJWaxUrqnC5GgMWovePevSNpdwBRgKRgCU4t0BwClW4XI0BC9H7Rz36xlLuACOBSMASnFsgOIUqXOhwSIfDIX32+FagU22RRO8f9egbS2PnxdixmPv1AFJyboEa+I4qLOc29YRhXF/vsr3J1dXr9OLFqBThqr9Q/0jffg29fYXTQ9dIoh2VUNp1acp+uPSY9s69SoUe240rOZZGHfeJ55ZFx1Lp81yfkee/yCnLveOwZ9sibwuMplCFhTwELD3Wdfn/Ivvq1XubClqp+OIb7sNbZmO3r/dxh0PKMo7njJGxgSxTg1sufR/KquR80zunCoylms9zYfveNw4HziNhtwWmcOsvZNR1qeu69M37lMDetlLvu0Zf2LY5427OOM7dBgCUpVCFvEolBkovJIpSCb+52wCAghSqkFepxEDphURRKuE3dxsAUJDvqEIuwSwPAAANJ0lEQVRG96mAn/W1dcvdWPjm4f/cv8ft/XdkP0vpi4CLm5vL32CFMJ5q2TdfOBrvk4JVzs2fNdr2+/7+jT3Oc8dD5vEkVAWAKllRhXWtld55XBwIVmBpp8b2lHEn2XYZ5j5rqnke19x3aJIVVVjYfUDLBymlzx+SgI/aDpe2pUcrqUPvO2cllTIuSevMkSQ7Z2wet595q2eXvs+SbeYKXK7k6r0kbWiPFVVYXqngFwExXGLuOKwxOMlcAYBgFKqwvFLBLwJiuMTccVhjcJK5AgDBuPUXFrZk8MuZcKa3twVfX+/S1dXr9OLFJ9M3gE25dGzWGJx0qm0oTAmok8A7qJMVVajb6PCHV6/eW7If0EpwUuS+XaK17YHWmKPQw4oqVOZcOFMaCFi6vt69DbC5udn3Pu4UYRTTnQ/umXYM1u7fJWFKPUIGJ51u22UNPOsLmho69uYatMv8hvGsqEJ9BCzVI3pwz1oBRJFCkkqGM0U69gAQmkIV6iNgqR7Rg3vWCiCKFJJUMpwp0rEHgNDc+guVEbBUj+jBPRcGeD0JTToz5kKEJJVuO27Pfez3+31fmNVtyd+2JLaBcQNQnBVVaI+AJZY09UOtoJB19B0XRQhDjI91OR/CBFZUoQFzApZYzvkwpZK9uywcaOj1DofUxQhEitd23F762EOrzoUVDf1UjaAjiMWKKrQhemjPVkU/LoKT1msbagcAjihUoQ3RQ3u2KvpxEZy0XttQOwBwxK2/0ICZAUshtBjq8fj2zvvwnNv73+msIkzpVHDS3Nc81fZw7E/sr1P7cI22d47TnG3ray997NmeFs+xQNusqAJjrBEAsYUPULVt45T+zhkj0fZLtP5ADlsY18KKoCFWVKFRYwNwTrm+3j17eG5fGEzu0JmthMvM2eY5+zB3cFK6+0NnljES8dgLU4K8BBUBU1lRhXbNCW6ZEgazROhMy3Ifl7Gvl/uYLDFGIhGmBAAFKVShXXOCW6aEwSwROtOy3Mdl7OvlPiZLjJFIhCkBQEHd4dD7c1JAI7oumeiLOaSUnt7RdnX1Or148cmT9se3v835Pb++oKPj9/3oow/Tq1fvDb3URQ6HExt9oaH9sEWX3CLptyG3q5b5E2UctjRXWtoWOMWKKmyDgInFnP4s0FMc5jwOJ4NRjt93iSI15R9Pxie0zRwHJhOmBI06CnR5ftyWUnpTsHubcCqUKlf40bI9f2JwO+YHbu2ejM95wVx7Yxt6WGkDamFFFdpVY4BNa9YIP1rD3MCgUm0AQKUUqtCuGgNsWrNG+NEa5gYGlWoDACrl1l8IaL/fnwzKmeLxbaX7/dO26+vdnJdnnLe3oHZ3N9vd3t+G/VlKXxyXU+5vaf3s/rmTxkPuY/u4L0NtUx67ZNvQfiWvHOeqldzudne3mEez9j4sFL4Udv8DcVlRhZgW/9BydfV6wqOrCJWswaXHtWQhUGMISo19rlUNRWpKsfsZuW+5bGEbgcysqMJGnfrplCWdCxaque24PQ0EVV0SpjS4Yx8FHQ29bxoZiJRzP5Q7LnnDmaYGNo0Jq6nlJ0UAoBQrqsBaIoXsLBHaMyckqdTj1toPLbcBAAtQqAJriRSys0Roz5yQpFKPW2s/tNwGACzArb/AKh7fKnkq3Om+7fb+ts2iYTxT247bu+EbP9/uh+vrXbq6ev3ObdgfffRhevXqvXceN2Ts+5bYDy23CWzahoGgI+FAAAuzogoxbTUMppXAjdHH774o7f33ie/T975bHU8sr5axdWk/+85JOc9VtezDObawjUBmVlQhoDl/qRfSUsZR+M6TMJ80coX0jHOBSIuHCNURprR821AQ1pZYVZxviX04dB0YE/YFEIEVVYA81gjfiRQiJEwJAFiMQhUgjzXCdyKFCAlTAgAW49ZfgAzOhe8MBR1dX++yvEfptmj9EaYEAPWyogpE0nLgxtxta3nfbJHgKwAYYEUVNkSIxrpmBiydC05aJDRLmNI6YUpCiABgmBVVgOXMCeQpFeYjTGlaGwCwAIUqwHLmBPKUCvMRpjStDQBYgFt/gbP2+/3LlPcH7ue6reHWyTkBS1MCjHISpjSuTZgSSxs471Zx/nsQ8PrRa8bvkFd1TKAWVlSBMaJ9yIjWn0sJ1AH69J3najv/1dbfS2xhG2F1ClWAFXVd6roufbPrUnc4pOeHQ+rS3bn4WymlZ4fDXfvjxxXucjrVl77+jX1sK20AwDIUqgDrqjG4R5hSfxsAsACFKsC6agzuEabU3wYALECYEqyodKjEjKCITVnyOD3+/c2HQJ6xbTPMCvq4NEzpYT+e2Jbbw2H3fMxrRm0TpgQAy7KiCusSuJDH0mFDrR2nUtvTShgMcLkthMOV2kaBfDTNiiqwit1uJ4CmIfeBQh+klD6/X2l80v54JXXM82tqG9o24At+tmU59i2ts6IKwCWmhCmNfX6NbQDAAhSqAFxiSpjS2OfX2AYALMCtvwBMNi5Madrza2qbG6a0ZrBaoRC1WQFeAGBFFda11YCD2ra7tv6e09r2tKD1QKnWtw+AhVlRhRWtscIwtHoi0GgcK0HLixCIJEwJAOKyogpACZECkYQpAUAwClUASogUiCRMCQCCcesvAKs7H1Z0Fzb0+Bbb+wCj28Nh93zouWu0zQ1TIp6pAVeFQqpmv3fJfh8RuAUMsqIKQER9BYOQnjrUGOBlbK3L/gYGWVEFYHVzwoqEKS1D2BoAkVhRBaCEOWFFwpQAoHEKVQBKmBNWJEwJABrn1l+gSVODUSrQVPDInLCikaFLack2YB2Fz+VNnXehNlZUgVa1VKSm1N725FLrfokWNhStPyXYB+sau79LzvFazy/QBCuqAFSlhVCj3Ks0Qz85IiRpnFPHpOR+dUyBrbOiCkBthBoBQOMUqgDURqgRADROoQpAVQ6HdDgc0mcPv28KALTHd1SBsBpM7p1D0MtptynWGHGc2tI3vhzn9ZSc444zFKRQBSKLVID0EmxSjp+OYEnGV3mOAWyXW38BAAAIRaEKAABAKApVAAAAQvEdVQCgOsLWANpmRRWIrIbExRr6SPv6xmHL41ORCtAwK6pAWNIeYRxzBYDWWFEFAAAgFIUqAAAAoShUAQAACEWhCu3ZYqgKsD3OaQANE6YEjRGqAmxB6+e6/X5/KN0HgJKsqAIAABCKQhUAAIBQ3PoLAAP2+/3LlNL7pftx5Lb1W18B2DYrqgAwLFqRmlLMPgFANgpVAAAAQlGoAgAAEIpCFQAAgFCEKQHZBA2dyUmADQCTVX59dO2jCCuqQE61XoTHan37crud2B5VxP5G7BN5tTJ/uFPz9aPmvlMxK6oALKKVv8C3sh3UxbgDts6KKgAAAKEoVAEAAAhFoQoAAEAoClUgp9ZDPlrfvkgEyQAtqfncVXPfqVh3OBxK9wEAAADesqIKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACAUhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACCU/w8XqOddMO5L7QAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (2) A* search search: 162.8 path cost, 782 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3c+rLOl5H/C3rzSek4lmFG+SgEEku2AkRlkGJljIy+CFMPRZiKBFkK2N/R+EmcnCm+zsjYPQ4mJEchrCJYTsgpDRQJYZ4cTb7EKW1jlkfDWX3M7innNun+6q6qquH+/zvvX5gBhNTf94q+qtqn7OU/3tzX6/TwAAABDFs9wDAAAAgEMKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQCgKVQAAAEJRqAIAABCKQhUAAIBQFKoAAACEolAFAAAgFIUqAAAAoShUAQAACEWhCgAAQChfzT0AAOq02+1uU0rvN/ynu+12+8HS4wEAyqGjCsBcmorUruUAACklhSoAAADBKFQBAAAIRaEKAABAKMKUgEfCbyAWxySHzAdgTXRUgUPCbyAWxySHzAdgNRSqAAAAhKJQBQAAIBSFKgAAAKEIUwJWTTgJh8wHAIhBRxVYO+EkHDIfACAAhSoAAAChKFQBAAAIRaEKAABAKApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAISiUAUAACCUr+YeAAAA4+x2u/3RorvtdvtBlsEATEBHFQCgPu/nHgDAGApVAAAAQlGoAgAAEIpCFQAAgFAUqgAAAIQi9RcAAA7sdrvb1BxIJU0ZFqKjCgAAT7WlJktThoUoVAEAAAhFoQoAAEAoClUAAABCEaYEMELfwI2Ox9VCwAgAMBkdVYBx+gZu1FykplT/+gEAC1KoAgAAEIpCFQAAgFAUqgAAAIQiTAlWqoRwn9xj3O12+6NFAoNYrdzH40iO3UD6htAFGE/b44+vDbmY11RNRxXWq4QPnNHGOGY8d5ONAvKIdjwOUfLYa9Q3hG4ppc6PUscNveioAixgzF+9A/31HgBgETqqAAAAhKJQBQAAIBS3/sJAhQSKCFgAAKBYOqowXPQiNaUyxthHtACiXOOJth2alDBGxil5H5c8duZX6vwoddzQi44qEFZTV7grWGi73W7mHVEeuuNEYB5Sq1zXmjVez2AIHVUAAABCUagCAAAQilt/AYLrCPASmlWRoUFtE/++rrk0sQjBe7XOEedEWAcdVRiuhPCCqcdYwjrXrO3Dbi2hWbyRc3+aS9OrbZtGWh/nRFgBHVUYaA1/rRXiAABATjqqAAAAhKJQBQAAIBS3/gIXEWYBwNwyh4wBGemoApcSZgHTyhlaJjBterVt01zr45oCK6WjCgABuBOhLkvtz64O4rlgvDHPBZibjioAAAChKFQBAAAIxa2/sAJLh1EIs+hn6H5peY1Lt3Xo0Ksptg3xCWWD+WU+nzqWuZiOKqyDD/wx5dwvS7x3W/hKn1CWaHO2tmCcKISyjTfmOGMdar/WUCkdVQBmUfJf0QXJUIqSj7M5THXsujMI8tNRBQAAIBSFKgAAAKG49RcAJiAYaFkCYt6Yazv0vPU1zHYA6qOjCusgVCOmnPsl+pwocdsIBlqWgJg3bAfOKfF8CjqqsAZNf/Hu+mt5nzCKsc+/VE0BF307Ebm2dU66NADTcD6lVDqqAAAAhKJQBQAAIBS3/gIhCKJZr8yhOAChBThHug6ThY4qrFdbwEGu4ANBNOtlH3MJATFv2A71y32OzP3+rJSOKqyUv44CJXMOe2PMdlhjUBtQDh1VAAAAQlGoAgAAEIpbfwFm0nBbnUAKKFyAYJsmzi1AdXRUAcYZEiYS7cNtFAJZKEnE4zjimLpEC/OLLvd2yf3+rJSOKsAITV2MroASTtUSBmO/Qz+6v8PYXqyVjioAAAChKFQBAAAIxa2/MJOOwA2hFwDBOGcDxKKjCvNpC7coLfSC/ASPwPz6nrMjHncRxwQwio4qQHC6ORCH4xFgGTqqAAAAhKJQBQAAIBS3/gKTm/r3JPu+3oj3rTYspSMgJppq98EYBe2/PuxjmFDm84PjmdnpqALUUwg0KWXdShnn0mraLjWtSy0EtZUt5zHleGZ2OqoAACukIwZEpqMKAABAKApVAAAAQnHrLwAAVWgI1RP6A4XSUQWoOziklHUrZZxLq2m71LQulKPm0J+cx5TjmdnpqAKT2263m6HP6fppmcPXm/pxtdNJKJv9B7RxfqB2OqoAAACEolAFAAAgFLf+AjDabre7TcG+C9Z1+/cFBLKs2MRzaSnmLFA0HVWgNG0BDoId8gpVpM6g9vWjPuZsfK5n0EFHFXrYbNImpfRhSumX+33a91l2c5NrtHXTIQCgBq5n0E1HFfr5MKX0H+//OXQZAAAwgEIV+vllSun37/85dBkAADCAQhV62O/Tfr9Pnz/c4jtkGQAAMIzvqMJAm01qSze92+9TyO+bdCSyjkmFvGt7zSCvR4uICb0FMA8pjTkLFE2hCsO1fcCP/MF/8jFPHQIhVGJRkefqZLbb7Sb3GKiDuQSwPLf+wpHNJm02m/Tt+wTf1mV9nwsAAAyjUIVTY9J8pf4CAMBIClU4NSbNV+ovAACM5DuqcOQ+sffzlDqDk9q8fvg/19fbdHX1Kj1//uLkQbvd7jgVeEyoEQAFqDDIzLULmI2OKnQb9YHi5ct3FnkfGGgNaaA51rHtPdewvemntnP9Euvj+IGV0lGFI/dBSB+mM7fv3tzsHv//9fV25lHBdHRA5mG7wvT6HlcNdyoBhdNRhVMCkQAAICOFKpwSiAQAABm59ReOHIUp0aIjFGTxcI2hASUT3yJWRZhIgJCXorZjgO21hKL2CdQq0vUWlqSjClyq7UN6jg/vOQuGWoqV3OuR+/2HKm28l8gZlFNrgE5t61Xb+kQV6XoLi9FRhSN9w5QAGGdt3aC1rS/AGDqqcEqYEgAAZKRQhVPClAAAICO3/sIRYUpxrCSwhomZN0QVdG4K5KlI0Dl2KXNz5XRUgchKuNjWEiaSez2mfP8S5k0Jcs+JGkWcmxHHFEGpQV817c+a1oUL6KjCEWFKPNhut6vpqfurdWyHc7Hr55XWNGdhTs6JkJ+OKpwSpgQAABkpVOGUMCUAAMjIrb9wRJgSJegIzBA+wWrNFSTTdbs1XGqK+WpuUjMdVSCynKEVpQZmCJ+Iv++Gqm195mT+D2d+5WO+djM3V05HFY4IU4pjTGdQ4Mx6mTeUyPwiMvOTHHRU4ZQwJQAAyEihCqeEKQEAQEZu/YUjpYcpzRUmAgBLWyI4TjgdxKSjCvVRpAJrVWr4SqnjXsISwXG5wulK2e+ljJPK6KjCEWFKAGXS/aIkfeerkDfWSkcVTglTAgCAjBSqcEqYEgAAZOTWXzhSepgSAACUTkcVAACAUBSqcGSzSZvNJn37PlQJAABYmEIVTglTAgCAjBSqcEqYEgAAZCRMqYfdbnebmn/0+c5vttVn6jCl6+vtk3+/unqVnj9/Mf6FoRId59hQun7L8AzXCrLzWWa8iOeqEeelNuZDgWo9vnVU+2k7KYU6WZHP1dWr3o99+fKdGUcCRar9XFr7+lGG2j/L3A1cfolatlWXNaxjjao8vnVU4ch9iNKHacCtvw8d0uvr7bOD576+9H3vu7oAQA8ld42AZjqqcGpMmFKu5wIAQDUUqnBqTJhSrucCAEA13PoLR8aEKfV9blPA0uFzAaIJECRzUShIrSEjALXTUYX59A5wELBUnSVCPWpS+3apZf1yh3Jc+v5VhoyQRS3Hcpc1rCOF0FGFI5eEKTU9d79PHzS8XmvAUlOYkoClMunSDBNpe3X91MN2u53gB6uAUvU9V019Hhnyes5h1ERHFU5NHYjU9/XGPBcAAKqhUIVTUwci9X29Mc8FAIBquPUXjkwVpnTB6z3eFnz/uLv724cFLE2s69YogLVxTgQi0lGFZQ0JKVgi6KPm0ISa1435CMJql3sb5H5/ABakowpHxoQpnXu93AFLawpSaAq90DXgnEjBTtHYNnH0vTYcLru5yTVagMvoqMKpqQOMBCwBMKW+1wbXC6BYClU4NXWAkYAlAKbU99rgegEUy62/cGRMmFLDraV3+/32JBBJwBJzGnGL812tt3fudrvbtMz3vmF2R9eQk7nddL3Y7ZYd4xKW+DrH1O/R8/V6n4svHd8KzonVXs/WREcV5tXnIhAtYKlmwli61Ty/al431q1tbpvz5Vpi39U+P2pfv1XQUYUjU4cpnXuPqQOWBGa0i/bXVeFOwDnnzvl9n9t1bYgctOc8CeulowqnlgifELAEQB9TXy8AiqBQhVNLhE8IWAKgj6mvFwBFcOsvHBkTpnTJe1zwvp0BSzUGZlCPFQR4wKTOBSed8Xi9uL7epqurV+n58xcTjxBgHjqqEJOApXWIFu60xHjMV7jcqOPn5ct3phoH81riXBzt+jO12tdvFXRU4UjfkIrr6+2z9BhSsWsNP7rkfccELAlTKseYcKeugJHIwSgli7JdhcvM61x40ZLLjpd3jfvm5u3tNNfX25FboQxTHZNTn09LOD9HCxeEJjqqcCpXSIXADID8+p6Ll1jWtRygagpVOJUrpEJgBkB+fc/FSyzrWg5QNbf+wpG+oUaHj5siwGiqgCWBGVCPoeFTmW4LvqvpNsJz4UVN5+IllnGqYb5XNRen5vxAaXRUoRy9gwEEZhBYqQEXucZdQvhUCWO8VBHrdnX1KvcQoihif12g1PNmSvXuExagowpH+gZXLBFgNCZgCSLyl3UiORdqlHNsZ8wZ5iesK5i286ZgNWqnowqnIgUYjQlYAqBbqefYJcL8ALJSqMKpSAFGYwKWAOhW6jl2iTA/gKzc+gtHcoUpnXuPoeOjTEPDc1peI9LtYII0VmDEnMs6P84FJ0U15nbf499Yvbp61XitAchNRxW4VFu4Q8mhDxEU8UF5gNrWZ2m1H0+R5keksaTU8lXRPsFJQ8KVhO8xs9rPYcxIRxWORApTOve+y73rKV0ymN+Y4yxYZz2kvufTm5uZbpuZycPPk11fbx9Dl1JH+N65UCkBS3XYbrfuw6IoOqpwKlKYUqT3BahN7efTMdez2rcNEJxCFU5FClOK9L4Atan9fDrmelb7tgGCU6jCkf0+7ff79Pm5W536Pm5qud4XoDa1n0/HXM9q3zZAfL6jClXyuaJgdylcqMsogjTyKWEuFTA/qjmfts6HzeZkJe/2+xQlhyDrPJ76u94NrycZHVooVKFK8hJK5QMLUzGXprIJHULTt5BqKjwbCtQHYf7A0TSPKwsKC7OtIRq3/sKRzSZtNpv07fvEw9GPm9qY9801ZoCIxpzvl1g29LGXjnvq5wJMQaEKp2pO/ZXiCPDW1Km4Uy8b+thLxz31cwFGU6jCqZpTf6U4Arw1dSru1MuGPvbScU/9XIDRfEcVjtwnHH6eUkqbjpubbm52rT+ePqe+47u+3j7596urV0+eO8Rut7tN+b5HI2iComU+fsao/tjrez5tOncusazPY3e7y8fdtc4ppcdr3P3jHgKWBl9Dxhh6/FT2/VVYNR1V6GWf0j/6eQqY/tg7MfPly3fGvE/OD9klfsCHQ6XO4VLHfaztPFlA4vDshmyDXPOhlnnYxjyEFjqqcOQ+JOLD9Hhr0z6l3/vDlP7pT1L67/8qpf/871LOVN3D8T2kOB6NubXTe/Tc/ZBlOY0Z9xLLoo3Hdoi3rFR1zJvtyXmy6XG5t8Ol2+bmpmvkZ9dlsWtITcfPuRTorq5u5ARpiEZHFU4dhEXcF6nf+mlKz16/+efv/WHK3FldIhwjWmBGriCTOQJPal4WbTyRlpVqjfOmSQnHVF+lbgci2Wx+M202/+Tgf7+Ze0jUZ7Pfh/xjVSj+MrYuj3/N/c4nv0zv/+/X6Vs/Tek3vnj7gC/fS+mvvp/+w/d/N23OfMEnpennSI+uQNd3Z591PbdtWe7v/Fxfby8a91LLoo3Hdoi1LPfxM8YSx94cr3nJsjTDuXOJbdOVl9B0/SltO0Q8fqJ0VIdum8nee7P5ZkrpFymlrxws/X8ppX+e9vv/Mcl7MEittYpCtYdadz7tNp9uNimlP09fvvcHT4rUB1++l3737/+z9MN/+MOzxerSc2TT/gPuI+xTSqY6XMbxU7qbm47Eovn0CrOa+jPKPNeQy11dvUrPn7/IPYwnpipUgwatdc+7zeabv0of/NX76fbJbZmvU0p36YP09XT7LcXq8mqtVdz6C0cei9SUvt9YpKaU0m98kT771Wfpx//nxyngH3tmCGYo9hwHATh+SnZ19SrXW2dLWs/0vo1GBgHOoc/26RvgFa1ITalrTG9u7/3FcZGa0puC4v10m1JKv3AbMFMRpgQHNp9uNmmf/jztn/3L9Oz13+l67K/3v06f/eqzlFLq1Vmd05hwDACeeLy9NdfPkB2aOkypx3uEu4Zst9tN39udL7ldfejt0z3GW+vPOv2DlNJX2rpcz1JK+5S+8kn65Hf+zSb9p5ICsohJRxWe+p2U0g/PFakPfr3/dfrZ3/ws/fUXfz3zsM4SSAEwjWjnziXO7yVcQ5YIfGKk1+nZsxfpe3+abGsmoFCFp/4ypfTj9PrZ3/Z58Lubd9N3/95302+/99szD+usX6aUfj89/YmFpmUAdIt27lzi/F7CNaTvePquS7T1q8Kz9Pr199KLP062NRNw6y8c2H+8328+3fwobV6nlNL3U0p/t+2x727eTR99/aPst/2mlNL9rTSfty3LPDyAYhyeO3dZMpSeOnd+n2KM0a8hxwFP19fb1pClc+vStGyJ/Rw0OKlRWzDPOz/5SfoXf/RH6atffNHY6XqdUrpNX3//T9Mfv0jpcd7c3d9O/nnDU6CTjioc2X+836eUfpRS+mn68r3mB335XpgitadQ4RjUyNeO2tk2BTk+V+Y8d0Y6b0caS0opZMjSOUUUqV1efe1r6Weffpru0gcnX1q+L1LTR+mz9DfpSZZS8etNPjqqcOTNl/73H6bvfPKj9J1PUzrqrL67eTd91POnaXI6F46x9O/51aIpqCOleX9LcA3bNaX4v5db6n6Kvl2jjefpsu3oc+f8592uvX+5qa8hly4b+nM5lx27l2yhdbr9xjfSf/u3/zp99+OP0+b121Pe/335bvoofZb+Z/rmyXOEKXEpHVU49eZL/z//5MP00Fl9+M7q62d/W1AndUygRFvwgUCK8dtG0Ee7JbbhGvdTpG2T8/gpcdnQx04p2nZYYtz0cPuNb6T/8md/lv7rn/zJ4//+cfpfjUXqPduaiyhU4dTjl/4fbwPevP6LtE/7tHn9F4UUqSmNC5RoCz4QSDF+2wj6aLfENlzjfoq0bXIePyUuG/rYKUXbDkuMm55efe1r6e63fuvxf0e3+x6zrbmIW3/hyHHowmPAUkr/PqX0l5vN5g+yDW6ASwIlupb1eWyE4JG5Xbptxixbw3ZNad5tuMSyqPspwrbpWhZtPJGW9XnsXPMu2naYc9y5jt0f/OB7I79ru08pxf/D+dh9ynrpqEIP+4/3+/3H+5/fd1iHCBdAMbPa1zfX+tW+XVOqYx0jrkPEMcEanRyL4wOh4hepyTmIEXRU4ciY0IXtdlvEVWMu2+2b4JE5tcXm379/ldt/ie3KePZTeWIENsUMU8plbPBOpDClc8FcKZ2E59agV3gb9KGjCqeELgCsQ6TAoGhhSrnUFKa0tn2X0jrXmZkoVOGU0AWAdYgUGBQtTCmXmsKU1rbvUlrnOjOTzX7wV+7WZ423GtJu6vmw2+1uU/MPYt+5lfBU7cdjx3woVZXz2HFbjgqPqd5qOCd2/Y7qzc3bFKSOYKK7h9+BbbPEdWWzSauYh/t9GV+crU2tn410VCG/tgtX9Rc0GtW232tbnweO23LYJyvQEUwUZf/PMI5wzSbBSUxKmBIcWVtwBQBEMUfwzhLX9XPvcebpnQFENze71tCl7Xa7iRQAJjiJKemowilBAACQxxzX2yWu62PeY+z4IgWA+bzEZBSqcEoQAADkMcf1donr+pj3GDu+SAFgPi8xGbf+wpH7W1U+T+ltCEff24K6vsx+ialfr8GkwS+5Q0tK2160yz2XoAY9z4mhzmuH1+Bzrq+3fV/28dbZzZtYmYeApftr/aAhNjoc99DgpKZ1fvpZZNzzcy+DS+moQrfaPyhPvX6213i1hVFcuj61zyWWU9sxNbXox9oc+2/udR7y+uYntNBRhSOCk8gpUmcDahD9mFrgTpCiHIfxPPy0zFEwUWu4UDoIJup63Jhr/ZzBSQKI4C0dVTglCAAA8mi7Bo8JJhrzuL7PXSo4CVZDoQqnBAEAQB5t1+AxwURjHtf3uUsFJ8FquPUXjgwJMKiB286GadheoYJIAErWFsZzFFbU6vA3R8+ELT153NXVq/T8+YvBY5w6OGmoiq7hrqWc0FGFbkIOOCd6EEmpHHusRe65nvv9L9E45qurV53/3uXly3cuHYvgpGm4lnJCRxU6NP11r+uvl9vttuPvvM2mfr2h71GTqbZXk7Vswyii/2XdfGAqY+f60Lk453lyTkeBQycBS4ed1AcPHdLr622vgKUe7ys4CRakowoAQHRTBxjlel/BSdCTQhUAgOimDjDK9b6Ck6Ant/4CABDauRCirvDDvkFMPZ47KDjp8Hbkh/Ed/mbrGgIbYQwdVQCAcgnoWU7vInVIkBMpJfOYBjqqwInIYRtCbADeGhr6t0YDwo8ufu7NTd3t0cifC6iXjioAADWbOkwJWIBCFQCAmk0dpgQswK2/wImG28buov+uJfXb7XaDgkxqVdl2cG7JKPNcWmzfTxim1Or6evvk36+uXj3+jitwGR1VoI9aPhRTNvPwjZq2Q03rUqKc27/Efd878Ofly3fmHMfSBB2RhY4qAADVmipMab9PHzS83uu2515fb589PO7wp2qOnQsq6grHEnJEzXRUAQCo2dRhSn1fTxATjKBQBQCgZlOHKfV9PUFMMIJbfwEggMpCkiCMCcOUBh2jh8/d1f0zqzALHVUASlF7oMcai9Ta92l0Obd/ift+yDFa4vpBKDqqABRhiZ+y6Bta0vU42gl+iWUtPw00VZjSmYc+Bifdd1KfPPfmZug7AzqqAADUbOowpb6PE6YEIyhUAQCo2dRhSn0fJ0wJRnDrLwAA1bokTOkHP/heevnynZQ6fie17T2alglTguF0VGG4toAEwQnAGLWfQ2pfPypyX6T2ZW7DDHRUYaC1hE8Ay3JugXmMCVNq0RmcJEwJpqGjCgBAzaYONeobnCRMCUZQqAIAULOpQ436BicJU4IR3PoL9JLxdyPv3BLZz263u03NP0h/dht2PDenKvZ90G17rIptTbnmPE5ubnaPgUjX19vWx3X9t0PngpOalglTguF0VIHoon/Aj6RtW/XZhhG3c8QxXaKE9ShhjNRtkTl4dfVq7EsIToKF6KgCALAKz5+/OFl2ppM6ODhJmBJMQ0cVAACajQlOEqYEIyhUAQCg2ZjgJGFKMIJbf2FBQ8MiMgYYQbHmCmXpezw6buuROQhLwFUAlwQnNS0TpgTD6aj20/bFeV+oZyiBJcM5zpYRcTtfOibH2XAR938EOefS2uZxtjnYEbDkuKAUVdYqOqo9+Ismtdhut5uH/z+063P4XOrjPBeb44/ajT0HdV3Tmo6fSwKRHpZBNLVew3VUAQBYG4FIEJxCFQCAtRGIBMG59RcAgFUZE5IELEOhCsu6SxkTJDO979TatmHo9cucHrpEEm2khNKcx9kcQs/tyoU/Zw88t8w6l3Kf59r0PP9FOocda52HLesWeV2gN4UqLMiFY7yCt2G4D28TC7N+Y+ZI30CWocEtl74PeRVyvmk99jLMpTDngQuEHXvbPOw4j4RdFxjCd1QBAAAIRaEKAABAKApVAAAAQvEdVViBKQIuFgjjKZZtsw599/PY+TDxfBKqAkCRdFRhHQQrMDfJtDE59llSyeeBkscOVdJRBQjokrROSbJATjm7985/UB8dVQAAAEJRqAIAABCKW38BAKiWwDsok44qrIOQCOintmOltvWB2jhGoYWOKqxAU8DF0L8wC6OYnr/yx7NUGIzgF1gnxzf0p6MKAABAKApVAAAAQnHrLwAUbrfb3aaU3m/4T3c5f9uS2DrmDUB2OqoAUL62YkMRQhfzY1mCk2AAHVUAAKpwLqxIkBmUQ0cVAACAUBSqAAAAhOLWXyCElYR6CLZpEHDf209UJ+BxBtBJRxXoY4l03D+5AAAHrklEQVQAiDV8gFrDOl4i2naJNh6YwhrmtbAiqIiOKnBCoAQAU3JdAYbSUQUAACAUhSoAAAChuPUXAI50/dYilMI8Bkqmowrr1RY6kSuMYg0hGGtYx0vYLlA3xzgwmI4qrFS0n9+INh6WM/W+10WCdkKNgFLoqAIAABCKQhUAAIBQ3PoLAe12u9tU14+z37m1F+pT0Lkq7Dlo6W2Y6db4sNsfiEtHFWIq4YPfELWtT1TRArJyWdv65lTKsR15nJHHNpU1rCMwMR1VgEroWLyxxHbo6kr1CasR+AQA3XRUAQAACEWhCgAAQChu/QUW0fNWR4EbQBgdQUfOVQAz01GFmNYaBiNwA8pSyrnq0nG2nZOmPFeVsg3HWMM6AhPTUYWAxvylXkgLsBRdxfHm2IZjw74AItBRBQAAIBSFKgAAAKEoVAEAAAhFoQpEsrbAjbb1Xdt2WCP7HgA6CFOCFRGiEYsgmvWy7wGgm44qAAAAoShUAQAACMWtv8BZu93uNk37A/dj3bl1EqhZx3m3qPNfwOtHqxG/Q17UPoFS6KgCfUT7kBFtPABTazvPlXb+K228l1jDOsLiFKoAAACEolAFAAAgFIUqAAAAoQhTggXlDpUYERSxKrn30wyyBH3UEgYDACxPRxWWVVPxk9PdzK9f237KtT61hMEAl5v7fB1BrnVse981bHNWQEcVWMR2u93kHgMAy3L3xHxsW2qnowoAAEAoClUAAABCcesvACxsycCuTCFqArMAGEVHFZa11oCD0ta7tPGeU9v61KD2QKna1w+AmemowoKW6DB0dU8EGvWjEwQAkJeOKgAAAKEoVAEAAAjFrb8AhNMRNiSkh1kMDbjKFFI1+r1zjvuIYxnopKMKQERtBYOQnjKUGOBlbi3L9gY66agCAMLWAAhFRxUAAIBQFKoAAACE4tZfoEpDg1EKIHikQYX7GTiQ+Rh33oWMdFSBWtVWvNS2PlMpdbtECxuKNp4cbINl9d3eOY/xUs8vUAUdVQBY2NRdmq6fHBGS1E/TPsm5Xe1TYO10VAEAAAhFoQoAAEAoClUAAABC8R1VICyJrk8Ieml2l2LNEfupLm3zy35eTs5j3H6GjBSqQGSRCpBWgk3y8dMRzMn8ys8+gPVy6y8AAAChKFQBAAAIRaEKAABAKL6jCgAUR9gaQN10VIHISkhcLGGM1K9tHtY8PxWpABXTUQXCkvYI/ThWAKiNjioAAAChKFQBAAAIRaEKAABAKApVqM8aQ1WA9XFOA6iYMCWojFAVYA1qP9ftdrt97jEA5KSjCgAAQCgKVQAAAEJx6y8AdNjtdrcppfdzj+PIXe23vgKwbjqqANAtWpGaUswxAcBkFKoAAACEolAFAAAgFIUqAAAAoQhTAiYTNHRmSgJsABis8Oujax9Z6KgCUyr1ItxX7es3tbuBy6OKON6IY2JatRw/vFHy9aPksVMwHVUAZlHLX+BrWQ/KYt4Ba6ejCgAAQCgKVQAAAEJRqAIAABCKQhWYUu0hH7WvXySCZICalHzuKnnsFGyz3+9zjwEAAAAe6agCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAhFoQoAAEAoClUAAABCUagCAAAQikIVAACAUBSqAAAAhKJQBQAAIBSFKgAAAKEoVAEAAAjl/wPl7yXh8I1XfgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " Greedy best-first search search: 164.5 path cost, 448 states reached\n" + ] + } + ], + "source": [ + "plots(d3)" ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAJCCAYAAADay3qxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3cGOJMl5H/CoEUmsZOxQOvmum2HIS/psQX4BWyAE1BwEiSfZPOkRRFKPoJNswQdK4GEKMAjZL0DBvnvXkq9+BFPahihChKZ86O5hT09ldWZl5BdfRP5+wGLJ2KrKyMjIrPo6sv51OJ/PBQAAAKK8at0BAAAA9kUhCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChvtK6AwDAfp1Opy9LKZ9e+E93x+PxdXR/AIhhRRQAaOlSEXqtHYABKEQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAI9ZXWHQC2cTqdviylfHrhP90dj8fX0f0BAIBHVkRhXJeK0GvtAAAQQiEKAABAKIUoAAAAoRSiAAAAhBJWBMBm5oZmbRGulT2wK3v/AGBLVkQB2NLc0KwtwrWyB3Zl7x8AbEYhCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQ6iutOwDA9k6n05ellE8v/Ke74/H4Oro/AMC+WREF2IdLRei1dgCAzShEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEJJzQVoTKItALA3VkQB2pNoCwDsikIUAACAUApRAAAAQilEAQAACCWsCABmECq1D44zQAwrogAwj1CpfXCcAQIoRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQn2ldQcAmOd0On1ZSvl0g9c93/jUu+Px+LpqZ2Cntjq/g7gWAItZEQXoR7YPqdn6Az3r+Xzque9AIwpRAAAAQilEAQAACKUQBQAAIJSwImBoVwJAhGt0qkaoy4qApjWvZ84tVPs4reU4A9RjRRQY3VTB0mO4xl3rDjzTqj89HrtS+u03y9x6nLOd30v03HegESuiAJ1Ys8pybSXneDwebn1doA6rqMDeWBEFAAAglEIUAACAUG7NhY0Jy9lG48Aax46bzJ1zDUN6XpzbNc49Ynj/ATKzIgrbaxWWMxUeMUqoRMsPwj6EtzXKHM5oztw2/6/LND9HCmsDBmNFFAblr92MqtXcXhP4lO1nSHq2dbiWYC+AGFZEAQAACKUQBQAAIJRbc4GqBJmwd72fA24jBiCCFVGgtm4/gEMlzoHtZAoCAmAFK6IAwGKCewBYw4ooAAAAoRSiAAAAhHJrLlTUKqTkynbv/J5oG44JwHWNg71ci6ExK6JQV6s31KnttujPHsJE5uxjpmPS0tRY9ThP5u5Lj/sGLbS8Hu7tWgzpWBEFqsr2F+ZrP0UhbGV72ebDGnP3ZaR99lMuAGzFiigAAAChFKIAAACEcmsuEKJxKAWVRR7PidtDBY0Qzq3KLCG0Dq6zIgpEUYSOpfXxbL19YL2WwV4R2xZaB1dYEQUAIJxVQdg3K6IAAACEUogCAAAQyq25wEf2EizUKnhE4AlsS0gMQH5WRIFLhi9Cd2gqmOPWwI6WISMZtk9uQmIAkrMiCrADtVeB1rzetRXh4/F4uPV1YWsZ5qc7KoBRWBEFAAAglEIUAACAUG7NhYYu3GIlSAMgyF6C2WpZEwIlQAp4zooo5JLlA5EgGGAPslxze7EmBEqAFPABK6LAR0b663SrYJwl2+05fORwKIdSymellC/O53Ke0/b2baveAgBZWBEFYI3PSin/9eHfS9sAgJ1SiAKwxhellN95+PfSNgBgp9yaCxBsya24jW7bnR0e8nDr7edL2k6nSr0EALplRRSA54SHwMeEuAFUZEUUgjwNqOk5nIZ9uCWESFgRI9kyzAwAK6IAXLYmhEhYEQBwlUIUgEvWhBAJKwIArnJrLnTmdDp9WXyHj43dEkI0t01YETVlvyZm7x9AK1ZEoT8+0CwzFTAieGSaseFRD+dP9mti9v4BNGFFFBja3J8hifQ8BOVaeFVEYMqWwUTCivqW8fwBYAxWRAGICCYSVgQAvKcQBSAimEhYEQDwnltzAUpsoMiS35Fd8Zuzd3Nvq9w2mOh+XOfejhv0G7svjs1OAmY+GIed7DMASVgRBbg32gfwLPuTpR9PzelTxn7X9nwf97DPLWQKdgJIw4oozDQ3lAV6I5gItlM78CnorgGAzVkRhfkEsDAqwUQAQCiFKMwngIVRCSYCAEIpRGGm87mcz+fy+dNbcC+1QW/mzu01bQAAT/mOKMC9uzJWWEuWgJSU4+p7dveMAxk0mocfpWdfSY6WtA0bUIgClNhAkePxeLj1sb25NK61xiaTW45TL/u2RIv5OuI4EuJS0ThVSErahg24NZfdOxzK4XAo33hI+lzdBi9pOW9qz/fa50qv55Rrxr1McwmA3BSiIB2UeC3nTURC7ppzpddzyjXjXqa5BEBiClGQDkq8lvMmIiF3zbnS6znlmnEv01wCIDHfEWX3HpI9P6/VBi9pOW9qz/db2k6nZf3rQe1x6FWr+bU1QTQA9VkRhbFlSU6FW/Uwh2/tYw/7tsRo+/OUInQ8I89X6IIVUXblIdzis1LKF4+/cVi7raXeE1b3ouW8iTgHXmp7+3Z+/x5TdzP0u37bsn3L0+98bSyXaWy9d8E+WRFlb4RmkIGwomX9y9bvluOVqT+Z2ljO2AJNKUTZG6EZZCCsaFn/svW75Xhl6k+mNpYztkBTbs1lV1qGZsz90fU1P84e8MPud4+3Sm5lD6EgW4etXBnDu4fbQbsLK8oQstS6bc5jH4/909ufH8Y7xbHfqm2JiGtxD7IEQQH7ZUUUWCKiQBy6CA0yNYbGdnyOPXMI6gGasyLKsIRmkJWwomX9y9Lv1m1bjO0obdm8eXN8VTY7f07vlvTleDweRhpbYBxWRBmZ0AyyajlvsgfMCOmZblv62Ocy7cvo191s+zzS2AKDUIgyMqEZZCWsaFn/svW7h7CiSzLty+jX3Wz7PNLYAoM4nM/uvIBaRg+3KGX733vbwxh26sWgqgRBUz30sbYP9vna+TPqbzVmvGbcMtbXQsbmHuMFbgqeWzO/epibW+3fLa83sY3Rrl+XbB6KSB5WRIElIgIuhGjkNOfDT+sPSD30sbbR9mcEt17DIoOmWsybqXFxzZ9vD+f7HvaRB8KKGEIPoRlP//o596+urf/SH/VX6g/H8f4voZmOaauQkWzWBOVE6aGPtQkrmvZSaNC1c3LLwKE1QVPXHve8z9feQ2r38SVWuYDnrIgyCqEZfct0/MyRaT3sXw99rG3u/mU6B6LOqTXPz7QvtR+3RR8BFlGIMgqhGX3LdPzMkWk97F8PfaxNWNG0HsOd5vZlzeO26CPAIm7NZQgPtxB9nqGN5TIdvy3bTqfStR72r4c+1jZ3/zKcA1u1Pfftb3+r/OxnXy2llPe33h7uv2hwdz6X1yXheM3ty9I+L3lsT+fP3HAnIC8ropCbEIex9Hw85/S99f710MfaRtufKh6K0EueFy4CdJbJNF6R4U5Z7GFe7mEfeWBFlO5kCshYEppxi0t/1a0dEd86EKmUXMdv27bpIKaa22kXwLJN0FQPfYy6ttQKvGm9Ly2vu1nnQ42woiVjs+b8oQ0rvYzGiig9yhSQIcShjkzHr+W8iZiLmfa55dj02Lb0sc9l2peW191M/V5z7NY8bos+AiyiEKVHmQIyhDjUken4tZw3rUJPsrdl608P43BJpn1ped3N1O81x27N47boI8Aih/O5+V15MIwavw9a+1baNa8X9Tui1NXjMb0SPNKrJoEpPR77tX75l39+vvKd0DkeA4w2MTdUZ4v3j5Hnwxb7luGrKg0Id6IZK6IAZDBSEVrKePuT1soitJTtj1XtUJ1MgUH0z7WKZhSidOdwKIfDoXzjIUAhXRvXZTpWmdq2es3nMu3z6OdPyzkS0Z9MbVPevj29/6f12Mzd7kuPOx6Pr4/H4+HNm+OrN2+O33zz5vjqeDwejsfj6yVj4zwFWlOI0qNMoRJCHJbLdKwytW31ms9l2ufRzx9hRXFta/UYQrR2bJynQFMKUXqUKVRCiMNymY5VpratXvO5TPs8+vkjrCiuba0eQ4jWjo3zFGjK74jSnYffNvs8axvXZTpWL7U9how8/d2+0/0dfncPv6dXdd7U6/fzV95iG5NjU25pG9EW15YMx75V2+FQPgr9OdS7WfT9b+8+vOZjgFHY+bjVsYt8TYAlrIgCTKsdMjISY3Cd4Jj6Fs25Tz75+dX/X3Nb0DHXKpqxIkpqD+EIn5VSvnj4q2z6Nq7LdKxearu2YrfFvOmp3z2tZr55c3xVEs6vLeZI5JyNbpves1LO53L4cBxO754/5gc/+FEp5cP5UJ6shG41XnOPyVbHju30/hM40JoVUbLLFIYhxKGOTMeqdsjI2m302u/sMs2lqDlySaZ92WLOrRmHuY+LOB9r9wUgJYUo2WUKwxDiUEemY5Up+KXnfmeXaS5FzZFLMu1LVPhO7cdFnI+1+wKQ0uF8ducG1HI6nfZ4Qt0dj8fXrTuxhWvHM/MtWWv6/RhCVL1TDWU4ViOO61Pf/va3ys9+9tVm2z+fP/yNy7nnwOGw6PbVxwCjReb2ZYvrTa/XsDmix6vmdoB7VkSBtYb9cL1Tox3PLEEco43rB1oWoWXdMV7y3KGPIUA0YUWkkSEMQ1jR9jIdq0zBL1n6nTGE6Hg8HjLMh72EO3Xgo/CpUm4+B14/byuVA4yEFQFcZkWUTDKFYQiG2E6mY5Up+CVbv7PJNB9cH9pacv7MfX7EeT/39SK2C9CcQpRMMoVhCIbYTqZjlSn4JVu/s8k0H1wf2lpy/sx9fqvQIGFFwG4JK4KKdhpWlE218KRegz5WhhWlm8OZx3qujON6q9bBRG/fnlY9/6X5VDvAqHZYUUfBV5sG2Y0UVnTlmL44hmueC61ZEQVG08MHtK1NBbDMCWbJEu7zKFt/bjXKfjQtQj/55OcRm8keYNTLNa6XfmYwNVZzxnDNc6EpYUU0kSE8JDqs6M2b40cBG/WCUE6T4RovbXfuc689LpsMoT8tw4rO5+NHASwxz93unMrUn1bjOuexa64Fc9vKlTCftauVEWbsc9UAo9phRYKvgFFYEaWVTOEhUWEkrbYdEaSRTaZxXbKNTPM4U1u2/vQwDpdEXeuy6/EaO9L4A5RSFKK0kyk8JCqMJHuoztzn9iDTuLYMKxqlLVt/ehiHS6Kuddn1eI0dafwBSiluzfUl70Yebl/6fMS2KVtu+3Tlbrhaz732uGwyjOvSti1ec5S2bP2p1XY4lI/efw4Xok8utU21v3lzvPzge+9vL527nalt9+zGY3XN83F9DDCqdo3t6asRT10I//HZCm4war1iRdSXvGGuXsJWeuknDPU+ExQk1Er2AKNeGBu4zZD1yu5XRGkjR3hIbFjRltteE6oz/7kxYSuZ2voNKxqrLVt/arWV/s0KM2vx8zvXfobjxuN3c4CRECKAy6yI0kqmAJCoAI9W284WbpLpWBmbPtqy9SfqmpFdr/vS6vqw5nEAw1GI0kqmAJCoAI9W284WbpLpWBmbPtqy9SfqmpFdr/vS6vqw5nEAw3FrLk1kCArZqm1K72FFNdq2eM2s47q0bet96bktW39qhRD1bm6AzrXbZF+wSQhHxeN8zfuxefPmWD755OflBz/40dW+9BQIt8aK+QAMxoooMBXCIfQHbrNBeESuz+5BwUTZQzhmXyN/9rOvbtkPgC5ZEWVzGUJBhBVde731IUSjhs4IK8rRlq0/tUOI3r7dyVJYEhWP/aIAo5f6cu168+bN8X0wVA/nT68/NwPEsiJKhEyhIC2DR1ptO9vYZOpPpnGN2pce27L1Z+21gLYyHfuI683a5zsHgE0oRImQKRSkZfDIKKE6ewyducTYxLVl68/aawFtZTr2Edebtc93DgCbcGsum8sQHhLZNuXDUIrTl6WUT5/eivUYVHFL2zVPb5G68np3D7foCp2pFFY0N7BmKvBk7mP31patPyOGEI1u46Cqm/uyVTja2ufXvnYCPLIiCnXNDf7JFsKRrT8jMKa8KCj0p1cC0/rl2AEvsiLK5jKEh0S1Pf7UwJoQnFaEztQNK5o/8ozqUgjRx6EzXy2Hw7H78+daOM3xeEy1bpwpqCoiHG3rfb7ctn0I3tJApDljs/S9eckxAD5mRZQImcJDosIZegxyyDY2mdou6fEY097o508PMo1D7etNr/Om5VjXfs1ezwsIpxAlQqbwkGzBKplkG5tMbZf0eIxpb/TzpweZxmH0sKIexrr2a/Z6XkC4w/m877sGTqfT5ABku52IcVybd628NN8fA5aCulPb3eNt01MuhQvBGpduze3tfeXKef/BOZXtvbRVfw6HdbdifvLJz8sPfvCjj9p7mzcVrqd3j7/TOmWLY7z0vbnWcVmzL9nOPbYx6nG2IgptZAtymNOfnou0OX3vef9IZqAQoqnzwvly2apr+89+9tVa/Wht7fwwv2AHhBWxuQxBGpFt8x5bN8jhWmjDx+Eo9YN7etAqXGheYM28Pu61LVt/1pyPvY3D3ACdnq4PG4/hR9f2UsqiUJ3afQ7Y502up1uGyUWHFY3+/gq3siJKhEyhBlHBCZlCGyK20YNM+5dt3mRvy9afVudjD/vSg0xjGNHntc/Pvs9rn7um35nGC7qjECVCplCD0QN5LtljGMklmfYv27zJ3patP63Oxx72pQeZxjCiz2ufn32f1z53Tb8zjRd0R1jRoF/+JY+FoQ0vBjRcEjGPtwhY+va3vzXSd6IuunRrbkMvhjZF6Cj46qbxGul9Ze6+ZNvnleEvswKa5lobYMR7H7w/Cita/1z6MepxtiIK21vygTvzh/PqAUujF6EJA2uyzK8s/XhJL/2krtoBTdnC6XqV7Xx0XGElYUVUlSEopHXb8/aIMawR2hARsHRhbFaHeNSWbAWzugznSk/BHJnOx7XPF1Z0r8Hxm3XtLAmvh2vNDWubOw4tw4pqBf/1fk2EmqyIUlurQIRMbdfa56gddDDS2LBetvmQXabzce3zI/alB9mvp6OLmHNbvCc59lCZQpTaMgWFZAtWmStTOEq2sWG9bPMhu0zn49rnR+xLD7JfT0cXMee2eE9y7KEyhShVnc/lfD6Xz5/e9rK3tmvtW49h7dfLNjasl20+ZJfpfFz7/Ih96UH26+noIubcFu9Jjj3U5zuiUNHChNyp13j+pnRTkm42NcZmawnDhWrLEq5xV5LPhQdZxovlpuZY5mPay3kxy4rr6eQ4fPj+eCyffPLz8oMf/OjW7SzZduZ5A91SiEJdkx8iLoU2PI3cvhLxP8oHk5vHJs5XSynHKq80atR6DRl+Qoax9TjHRviD46M1P/d1aRym3h9rJ6/3OG+gZ27NparDoRwOh/KNhzS4XbatHa+5j6v93B7Gpse2ufuWrd+Z2rL1p9X52MO+ZJPp+O3x/Kk9hnP1MDY9nD+wNYUotWVK5MyULjlliwTAW5/bw9j02HZJtkTU7G3Z+tPqfOxhX7LJdPz2eP5cUvv11mxjbX8ixguGdTif9/0dabfP1fXwl73PysRvZY3edpi+vfbF31Mr139D7urvl719e5p87pa/fRY1NtmO85y2pcckS78ztmXrT+1jn3kc5u7Ltce1fC/NdPz2dP7UHsNy5f1xzftHi7FZOl4vnT8+x+7DqMdZITrogWVbhxuCd1Z8R/SSDwKMWs3jW8bhkjzfEa3HtWW/5h770+mUPsBrI3eZv4vn3F2v9hgufH9cZYMApFUUopQy7nF2ay7cZtGHx5npgUtS+bJ8eF3djx0k1cKULOdxtL3uN7cLS62tHYAETFOIUlWmL/1HtF1yPpdDuT+3vllKefX27Wnyr6tPX/N8Lq+fP/fadub2p9U4XHLr2PTcNnffsvU7U1u2/mxx7OlDpvnVw/lTawyXvj8CfXAiU1umL/23DBaY+9g126m9jS3GYW5/5j6ux7ZLph6Xqd+Z2rL1J+K8JadM86uH8+eSlu8/QCIKUWr7opTyOw//3kPblLmPXbOd2tvYYhzm9mfu43psu2TqcZn6naktW38izltyyjS/ejh/Lmn5/gMkIqxo0C//Us9hYSDPw+1D762ZY4cFAQ1TAQu15vHScbjkUihRQ5sGpqy9tlwJskkd9PLcTgJ5PjgmC8KKdvsGnOX9dQfzs8n1IuKz1ZL3xx5kC0ma0NX7z0hGrVesiMLLlnxIqR2oMPv1AgIWVn1YSxhKlP3D51T/svf7ud76e4tb9zEsgCWZTPs9+vwcef8yzaPVOglJGnk+0cBXWneAsTwEDXxWSvvfTqvV9sIuv/h7kG/frhqv1xf6c+33Rm/ZxupxSLbSOduW82bpcV/7/Kxt1/ZjJLcdu+NH53e23zzM+vugLJPpvK+43UXnz5xzqix8fwXWsSJKbZkCGloGj0SENswlGGLaHudNpvNnJFHnvOPHLTLNm2zXJfMdGlGIUlumgIZsoTNzn187oEEwxLQ9zptM589Ios55x49bZJo32a5L5js0Iqxo0C//cpu1wUSX1J5jGQMaer01t5WZYUVDXJv2HMhzya3HruNQndThJnuYn+bcfBnfX5+7NdTo29/+1sT3UM+lXPgos3UAIsuM8pngOd8RZfcO3z8cSim/VUr5q1LOLYOJlmw3zYeDhCFE5JJqvja25prR6xhm7/fo89OcWyb9fLg11Gj6eZdrmInHDxUQRXsKUarKEI6ypO3w/cOhnMufllL+oJTyZ1N/GXzwUTBRo9CZ1QFGKz0bh6+Ww+F4UzBE7SCUPaxulJLn/JnXtiyQJ0tIT40wnzdvjrOuGUvGgbourZwtvY60Wo3Y+vzpOWhsxdisvl7dGAa46D386bVl7jVo6TaeP991iC34jii1ZQohuNr2sBL6p+X86vfKoRzK+dXvlX/3H0uZvtZuEX7QY6CIYIj2mp8/wW3Z+jPy+U0/os6fHvV6bZlrzTVoll8tPyl/VL7/279R/vd/+6Py/d8uh8Ov3dBPuMqKKLVlCiGYbvu33/uilPKnpZTfLa/e/XIppZRX7365/MYP7x/13/9TubAyukX4QY+BIoIh2st3TrUJFMnUdkmP5zf9iDp/etTrtWWuNdegF/3L8jflf5Z/U75e/u7P/6j88atX5d2fl1L+qRwOv1nO57+5ob9wkUKUqh5u3fg8c9vh+4f7IvQff+U/lK/99MMd+NpPy1Qx+vT1HkMcnt66dLrP67k7Ho+vP3xsmVRr/w6BN4at6fPz9mtjw7Rs59TWbdn6E31+wyVbnz89X597u7bc8B7+/jbbN2+OHwULPQkmWnw77mMR+rr8XTmU8ukvPbzEu1LKXXn9118/HH5DMUotbs1lV97fjlvK735UhD56LEY/vE33+Rf0p8IMWoUcRAUIZA8qyN4/uFWvc7vXftPvseux36v6/DxYaFmg0S++jvSr5Sfvi9DnBcKrUsqn5ctSSvkfbtOlFiuiVJUhPOTFYKLzq997fzvulK/9tJRv/pd/KP/6z/6iHMp3zt89n5eEOGwcVjQrwCjb+C8dm1tk+pmILYOTMh3TFvOmdX8anN/nx7mdKYzpkp5/RqA3258/t8+5yCC6NaFgt49N9bZZY10qBBNe+rm14/F4OBzK4Q/Ln/z218vf/flh4o/qD8XpL5VS/nkp5Sdr+wJWRKkt05f+n7f9VinlD14sQh/dP+4PHp43tY0pa4IERm671s58mY5py3mTqe2SkcaBfMybZX3JNjbZxvWzH5Vv/cm78kptQBiTjdoyfen/edtflVL+rLx79Q+z9uT+cX/28LypbUxpFWaSve1aO/NlOqYt502mtktGGgfyMW+W9SXb2GQb1y++VX70h6/Ku8ifg2PnDufzvn8W6NrtHm4xGs8H3xEt5Z9deejfl1J+WEr5ztt/8fbvyoLvfj6fN+bYtDVj8xgYVb1Tbdw9v614sLH5aP/WuLJ/Vbczsy9Dn9+tflOz5THu5XdEM6t9XmQ7z1rNz8Nh/W95Tt2a+7CBXyul/N93pfzqpZWqd6WUV6X8bSnl18v5/JO1fWG+bOdALVZE2ZXzd8/nUsp3Sik/LP/4K5cfdN/+w1Luvxtacn2g5xdGOi619yXb2ESsPxR/AAAW3UlEQVTtX7b95nYtj3GPYTfEajU/V83NTz75+fUH3BeXv3lXXn/0ZdSH1NxSSvlNRSi1KESp6nAoh8OhfOPhy/Up28r3zp+VH3/3O+VrP/3P5X7l86m/L1/76X8uP/7ud8r3zp89fe7acZj7uJHblozNHtWeN5lEzZuWc3vrvmTbv1Edj8fXx+Px8PSf1n2aa4/zpuU5Vbs/L7Wdz+X1+VwO5f7z+zfLy5/j3z/u7dvTBz/xMtnvcv4/Xy9f/sarUv72XMrdP5VXf38u5e5VKX/79fKln26hKoUotWX6Mv9024+/91l5XBl9/M7o/b9/WEr5zsN/XxMO0CrMJHvbtXbqz5tMouZNpvCQqPOn13AU6tvjvGl5TtXuT0RfXn7sfbH5698r3/v9b5b/9f++V773++X+dlxFKFUpRKkt05f5r7a9v0338O4vyrmcy+HdX5Rf3I67NhygVZhJ9rZr7dSfN5lEzZtM4SFR50+v4SjUt8d50/Kcqt2fiL7Me+z5/JM/Lt/9y78u/+rf/3H57l+6HZctCCsa9Mu/zPcQYPRbpZS/eihCP7Dl70EybUYgz1DHpWbIVcaxqXk9zXTdbtmXhKFUl9wU3rJkXEcZh6gAnE7Ga42mgVbR16DDlQCjh9t4Sym5+sxyox6/r7TuALT2UHz+uHU/WOyujPNhqnY4SraxEf6yjUzHeEpEH0cZh6gAnB7Ga43R9++5qeu96y7pKUSp6uEL9p+VUr44n+//Std729u3W40Wa9T+i/dWf22cO+eet1+bdy+/5v3YZDh/arTVHZu460PLcWCfzJt7t55TGc/7GW2zrvd7OO70x3dEqa3Vl/RHCV1gPGtDM9a85iht19qfy3R9aDkO7JN5c2/tOZXpvPdZhmEpRKktUzBOj6ELjGekQJ6W5172sYnoy5Jts0/mzb09hhX5LEN33JpLVQ+3g3w+Utvp9NFuEuDCrbKbB1BsYe6ce95+bd5lOC8i2563ZxqbyL4s2TbjuhY29PT2y6fzY2+3Zd56Tl1qexzvuWOY6drpekF2VkSBXuwtgAJe0kMYSUQfexiHmlwLYxlv2IgVUarKFHDSKqyo5s9wjC7jz4ysMWogT+u2zGPTMqwoIpTq7dvTu+k9/FiLa9qlOyVqX3fnvt5o17Qpa35Cqvb4z3VrWNHSzwCZrp3CisjOiii1ZfpCvi/4Ey0qiCbTeRF17mUfm4i+RO2L6x8jWntOrdlOpjZIQyFKbZm+kO8L/kQbPZBHWFHbvkTti+sfI1p7Tq3ZTqY2SMOtuVSV4Qv5a9qWhhLAU6MG8rRue96eaWzm9uXSteXxuXPbljy2VhuM4vawovXbydAmrIiMrIjCh4QSAFtwbaGW1uFMc7Y/9ZjWfQcSsSJKVRm+kO/L/LQyaiBP67bMYzO3L64t1NLDz1j10EegPSui1JbpC/m+zE+00QN5Wp6P2cdmbl8AgKIQpb5MX8j3ZX6ijR7II6xofV8AgFLK4XzexU9eTfIbjzxV47fKlvyOKLt0N+e2tQ1+h+/Lcvl7irP608qVfsOjj+Zwxt8Rjf7tzZFUeB+9+Tq3dNszjnO6a9re51cPRr0+WBEFiNXqA8jUdlN9ILoge/9ozxzhJZnmSKa+QFPCiqgqU8CJsCKyahHIkyng55awopHd+tfs2is1NbYBAHNZEaW2TAEnworIqmUgT+3Xcz4CAIspRKktU8CJsCKyahnIU/v1nI8AwGJuzaWqh1vpPu+17XS6tndQx6V5+Lz92lysPbfrnT/3IRxPbwN+2O7d+Xx8fes29qTXUKnWltxC7HbjfYo67he249yFCVZEAWLdte7AhnoNRGrp+XwwhjCW5+fuyO8BsIgVUTaXKfQkW1hRz5HbL5kTyHP5GJzeTb3mmzfHV9eeu+7Yt9nuSGFFW21jiehzatRI/VvV2merloyq1eqoc4qMrIgSIVPoiXCUOEsCeTKF6rTc7ihhRRF9BgA6phAlQqbQE+EocZYE8mQK1Wm53VHCiiL6DAB0zK25bC5DCNHcthphRR2EZoQEJ8wJ5LnUFhGqk2m7Uf2JCSvaZhtLZLr9rHZf1r5eprEZydxxXTP+Gxw7ATpAc1ZEYX+EnvRhKtBC0AWwlveBOK7ZMMGKKJvLEEKUNaxoFLcG8rQK1cm03emxuV+t6Gn/MoQVMb45gUhrQqSsHI/p1iCtpfNhjyFlcCsrokTIFEIkHGUbawN5soUGtdruKPtXexsAwGAUokTIFEIkHGUbawN5soUGtdruKPtXexsAwGDcmsvmMoQQzW2rEVbUg8q3nt093Ea6OJDnUtsew4pOp9OXpZRPn97a+tCf2WObYf8yhBUBjOrxvWKD113zmUDwFTezIgqsJfRivakxNLYQbw/hMr3tY2/93UrG94SMfaITVkTZXIYQosiwouPxePjwNU/v1r9qbsKK6oQVbbntvYQVbRkUUuNOgjVBOc+fGxGissfgnkurO3OPy1YhSXsPwFl6TIA+WBElQqYQoqhwlL0Frggr2ma7UdvJvg0AYDAKUSJkCiGKCkfZW+CKsKJtthu1nezbAAAG49ZcNpchhGhuW62wor0FIC0JnckQqpNpuy/paf+EFcHYtgrLieBWXsjHiiiwlhAJoBTXgj3osggNtPU5kPEcy9gnOmFFlCYyBBMtDVt58+b46hePux5ANPc1ewygWBs6kyFUJ9N2hRXVDStqZW5gzUh6vH5BTdHngJ9JYTRWRGklUzDRmrCVGo/tzdpxzRSqk327UdvJvg0AYDAKUVrJFEy0JmylxmN7s3ZcM4XqZN9u1HaybwMAGIxClCbO53I+n8vnT2+9y9Q2t881HtubteNa+xi0OvYR2+11/2pvAwAYj++IAty7K5eDOFIHMVxJsbzb4feJUh+r53pOIGUTPVyDpvoIsJhCFKB0HQIx9aFw+A+LA4TlDH+MmK+Ha1CmPi4NBbv1erGX8DFowa25NHE4lMPhUL7xkJCZrm1un2s8tjdrx7X2MRilbe0+z33umm20OnYAwHgUorSSKSFXau4yo6fmZppza58vNRcASEkhSiuZEnKl5i4zempupjm39vlScwGAlHxHlCYeEjE/z9Z2Os3r87XHLX3sUq0DTt6+fdqXRW13x+Pxda1jMFrblIg5G33+LG2Dp9Z+Z893/qjlwlzaY0gc3MyK6HQaXaaUOvJpOW96DTiZ22/n5DaMK8C2en1/Jr8h38N3vyLqL1d5PISTfFZK+eLxNwSj256u4l3v3/28ufR6S15zj14+LtNjO3LbreNVY85GnD8AwG1GrVesiJJJ9uCYtYE83Mt0nDO1Tekx3AkA4CqFKJlkD45ZG8jDvUzHOVPblB7DnQAArtr9rbnkkSE4Zm3YymOQ0NxbFPcWmnE6nc4rwo5WBSU93JqaIpioTVjRx3Oz9tjUDuYiVkQQWuuwtS3MvY5Xvt4LxQG6Z0UUPrT2y+BDfcAaiOMyPQbZx2bIgIakIuZC9vnWC+MYx7UGNmJFlNTig2PWB7rs0Zs3x1flhbFpvfqbKZioRVjR/FCjiLCv+duw6gO0dOka1Pr9DEZhRZTsMoXJCGqZ1sPYZJojLcOK1myjVXASADAYhSjZZQqTEdQyrYexyTRHWoYVrdlGq+AkAGAwbs0ltUxhMmtDZ0bWw9hkmiNtwopu30btvixtA/Zjq0Art9NCPlZEoa49hhrM3eeWY7PH4wIZORfrGHkcBTHBTlgRpTuZAmamglWy9CdTm7HJG3KVOayodh+32j/m6TV86tpq2vF4PET2BWAUVkTpUaaAmalglUz9ydSWrT+Z2lqK2JeR5g0AsJJClB5lCpiZClbJ1J9Mbdn6k6mtpR7CihwrABiIW3PpTqaAmalglUz9qRdEcx8g8fT2xIdwmruH31/d7djUaGuph7CiDMfq2v6xvSshNne93fI70r7woQu3cTumMMGKKDDXVICEYAkeTQWojByscolx2MZI16CR9qW20c4TxxQmWBGlOxlCS3oIVokMatn72AgremzbJpBqi9fcMqzI6gfcbs354ydaoC9WROlRptCSHoJVMgXRZNuXTG0tZRqHHuYNALCSQpQeZQot6SFYJVMQTbZ9ydTWUqZx6GHeAAAruTWX7mQILbnWlq0/mYJosuxLxraWMo3DrfOmRpiWsKK6rgTybLGt2rdkCpgB2JgVUQBGsMfwl+yhLj2Pfc9937OM50TGPkEKVkQZQoYgk2zBKsKK+mrLJtvYRMzPWmFFUS6t2AlrYc+WrGJfO1eOx+Nhq+cCv2BFlFFkCzLJ1J+IoBZjs74tm2xjEzE/ez1WANAdhSijyBZkkqk/wor6aMsm29hEzM9ejxUAdMetuQyht2CVHtuuBbW8fXt69/i/rwXEZNmXjG3ZZBubiDAtYUUAEMeKKLAFQR9xpoIwBGTkN/ox6nn/eu47QBesiDKszMEqPbYtDWrZ09i0DCuq8RMTmcZhi7CirEYPG6r98ycCYgDGYkWUkfUQrNJj21x7HJteA3AyjcPaeQMAdEAhysh6CFbpsW2uPY5NrwE4mcZhi7AiACAZt+YyrMzBKj22LQ1qGXVsTqfTl6WUT5/eCnotoKmHsKIM43qtbc5jBQnBxx6vV402f1fr9uwr+1FtG0A8K6IAy0x9qBPQxJ7sLcyn11Cwltelmtt23YUBWRFlV7IEq/TYJqzo5XFoEVZUQ4Zx3WNYUU/2HgZk1Q2gPiui7E22YJUe2+YafWwuEVa0TdvSxwIAySlE2ZtswSo9ts01+thcIqxom7aljwUAknNrLruSJVilxzZhRS+Pw9p5c0nE70q+fXt694vtlVJWBi/VbJsbDiWsaCxLQ3Y6+P3V2aE6HezLbBf2ZVfhQo3DoqLs6phSlxVRAJ7L9MFJSMk+jXZ8R9ufW+1tHPawv3vYRzaiEGX3DodyOBzKNx6CT1a3bfGamdrmGn1s5u5z7XGNkn2slz4WqOd4PB6u/dO6f0AfFKIgrOiW/Ztj9LG5pNewokuyj/XSxwIAiShEQVjRLfs3x+hjc0mvYUWXZB/rpY8FABIRVsTuCSua1yas6OVxWBP6k02GsV7SR2FF+VwJahFusmO1w5i2CHcaKTAKMrMiCrCduSEOd5v2Yrls/WG+qWM355iuee4la4KmRpuDI+3PSPtS2/Ox2cNY7WEf2YgVUZjpIRDls1LKFw8rMRfbljy2p7anP59x63hl2ZfW4/C87XF1KMP+ZWybM7ZLj8uo1qw0ZlqlvNSXa6tUIwXkvLQvexmHHrw03pnOKfOGjKyIwnzCipYZfWzmMm/Wt00RVgQAnVKIwnzCipYZfWzmMm/Wt00RVgQAnXJrLswkrOj5Xl436tjUGIdRx2ZN22OwzdNbbF8aa2FFQEsrQo0EdkGxIgpADnODnVqqHeYD7FMP1zvYnBVR2ECmoJdMIT1Z9qX1OIw6NpHj2oIVDFjvlmAcP6cCY7IiCtvIFPSSKaSnZb8zjcPoYxMxrgBAxxSisI1MQS+ZQnpa9jvTOIw+NhHjCgB0zK25sIEM4S+124QV7Tes6FqQUK026ng8Vq378RK3WgJgRRSAl6QvbHjPseqD4Kt9c5yhWBGFpjKExGQIk8mwf8KK+g4Sgp4IvqrnlvCjUq6vyt/6msAyVkShrUwhMS3DZDLtn7Ci630EAFhNIQptZQqJaRkmk2n/hBVd7yMAwGpuzYWNXQkPuTufj69LwuCYGmEyS8JILmz7prGJaNtnWNHU3gEwpfPwsDu3kLM1K6Kwvak3oVvfnNK/qVWwh33sSQ/BGsJf7u1tfyGznt/Leu47nbAiCh0QHDNtpOCeDMFEl9u2/6v42p/z8Jf7e5nGoXYYjJ98ARiLFVHog+CYaSMF92QKJjLnAIDNKEShD4Jjpo0U3JMpmMicAwA249Zc6IDgmGkjBfdkCCa61gbk5vZloCdWRKE/ewgjybyPmfsGwDwRAWc9v1/03Hc6YUUUOtMqjGTpX9pvCSPpwaXxtwoB0JeI99KRw8OgBiuiAAAAhFKIAgAAEEohCgAAQCjfEQWgitPp9GUp5dPA7d363dy7DN/dWjpevotc7srl8RKqAtAhhSgAtYQVoStl6WeWfnQhwx8PsssSOuOPJsAcbs0FAAAglEIUAACAUApRAAAAQvmOKLxgywCWRt+jSRHUAi0591jiyvuAYwpwIyui8LLRAkVG2x/ykF563fNzz3j1Y+q62eJ6OjVvzCegK1ZEAahizcrQLSuULyWEZk/uvDRe1/qcJRF1ruzj3ysrsMAorIgCAAAQSiEKAABAKLfmAgxmy4AtAIAarIjCy0YLgBhtf/iYIvRetrmerT/Afgi5Ih0rovCCtcEQI4WPQE+EugDccz0kIyuiAAAAhFKIAgAAEMqtuVDR0pCYRr+zd+cWHaA3WUO4XMcBbmNFFOpK9yHpgog+Cj9oaw/jP+o+ChSZ1sP1NcrexsJ5AQOyIgqsJnQplx5XSoR63evx2MHWnBcwJiuiAAAAhFKIAgAAEMqtuQCwsaxBOxcIwYEBuQaRkRVRqKuH4IRb+ygsYpqxWW/0MezhA2Apufs5ylyoIftYjH4+9yjzuf1UL/2kgsP53CJ1HPZDCAvQ6Cc+blLrujTStW+kfWGf9ngNIj8rogAAAIRSiAIAABBKWBEAwEY6ColZQ8AMsJgVUdie0Aagl/O9l372ZPQitJR97GPvejm3e+knFVgRhY35KzHgOgC05BpERlZEAQAACKUQBQAAIJRCFAAAgFAKUQCA7ewhfGUP+whUJqwIAGAjQmIALrMiCgAAQCiFKAAAAKEUogAAAIRSiAIAW5gKsOkx2GakfQFI4XA+n1v3AQAAgB2xIgoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEOr/A27LKJSZmrwOAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " A* search search: 133.0 path cost, 2,196 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAJCCAYAAADay3qxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3c2OLMl5HuCoJoc4lDGH4op77QyLHtFrCfIN2AJhoHpBULOiPVfBmeFV0CK8GBOE0QUIhOwbICHtPbTki+DGJE+Do4EGPOVFd5+prpNZlT+RX0ZEPg9ADJinfiIjs7Lq6y/qrd3xeEwAAAAQ5WbtAQAAALAtClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACPXVtQcAAGzX4XB4lVJ6t+Of7vf7/cvo8QAQQ0cUAFhTVxF6aTsADVCIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEOqraw8AWMbhcHiVUnq345/u9/v9y+jxAADAEx1RaFdXEXppOwAAhFCIAgAAEEohCgAAQCiFKAAAAKGEFQGwmKGhWUuEa5Ue2FX6+ABgSTqiACxpaGjWEuFapQd2lT4+AFiMQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAINRX1x4AAMs7HA6vUkrvdvzT/X6/fxk9HgBg23REAbahqwi9tB0AYDEKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQUnMBVibRFgDYGh1RgPVJtAUANkUhCgAAQCiFKAAAAKEUogAAAIQSVgQAAwiV2gbHGSCGjigADCNUahscZ4AAClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFBfXXsAAAxzOBxepZTeXeBxjxPver/f719mHQxs1FKv7yCuBcBoOqIA9SjtQ2pp44Ga1fx6qnnswEoUogAAAIRSiAIAABBKIQoAAEAoYUVA0y4EgAjXqFSOUJcZAU1zHs85N1Lu4zSX4wyQj44o0Lq+gqXGcI37tQdwZq3x1HjsUqp33Iwz9TiX9voeo+axAyvREQWoxJwuy6VOzn6/3019XCAPXVRga3REAQAACKUQBQAAIJSlubAwYTnLWDmwxrFjkqHn3IohPVfP7RyvPWJ4/wFKpiMKy1srLKcvPKKVUIk1Pwj7EL6uVs7hEg05t53/l5V0frYU1gY0RkcUGuWv3bRqrXN7TuBTaT9DUrOlw7UEewHE0BEFAAAglEIUAACAUJbmAlkJMmHran8NWEYMQAQdUSC3aj+AQyZeA8spKQgIgBl0RAGA0QT3ADCHjigAAAChFKIAAACEsjQXMlorpOTC8977PdF1OCYAl60c7OVaDCvTEYW81npD7XveNcazhTCRIftY0jFZU99c1XieDN2XGvcN1rDm9XBr12Iojo4okFVpf2G+9FMUwlaWV9r5MMfQfWlpn/2UCwBL0REFAAAglEIUAACAUJbmAiFWDqUgs8jj2bM8VNAI4SxVZgyhdXCZjigQRRHalrWP59rPD8y3ZrBXxHMLrYMLdEQBAAinKwjbpiMKAABAKIUoAAAAoSzNBd6ylWChtYJHBJ7AsoTEAJRPRxTo0nwRukF9wRxTAzvWDBkp4fkpm5AYgMLpiAJsQO4u0JzHu9QR3u/3u6mPC0sr4fy0ogJohY4oAAAAoRSiAAAAhLI0F1bUscRKkAZAkK0Es+UyJwRKgBRwTkcUylLKByJBMMAWlHLNrcWcECgBUsAzOqLAW1r66/RawThjnlf4CACwNTqiAAAAhFKIAgAAEMrSXIBgY5birrRsV3gIALAoHVEAzgkPgbcJcQPISEcUgpwG1AinASjbkmFmAOiIAgAAEEwhCgAAQChLc6Eyh8PhVfIdPoCUUvnXxNLHB7AWHVGojw804/QFjAge6WdueFLD66f0a2Lp4wNYhY4o0LQSf4bkPATlUniVwBTWVOLrB4A26IgCAAAQSiEKAABAKEtzAVJsoMiY35Gd8Zuz9yUsqxw7r0G/sXt1bjYSMPNsHjayzwAUQkcU4EFrH8BL2Z9SxnFqyJhKHHdu5/u4hX1eQ0nBTgDF0BEFAFhI7pUJQasGABanIwoAAEAohSgAAAChFKIAAACE8h1RgAf3qa2wllICUoqcV9+ze2AeKMFK5+Fb6dkXkqMlbcMCFKIAKTZQZL/f76betjZd85prbkoy5TjVsm9jrHG+tjiPhOgqGvsKSUnbsABLcwEAAAilEAUAACCUQhQAAIBQviMKAHCBIBqA/HREoW2lJKfCVDWcw1PHWMO+jdHa/pxShLan5fMVqqAjCg2pPWEVzuVOMy5Jy/sGY3jvgm3SEQUAACCUQhQAAIBQluZCkKE/uj7nx9kDftj9funlhEJB5rswh4sfP9bl2F8XcS0G4DodUWCMiAJRETpf3xya2/Y59gwhqAdYnY4oAEBGS4bvjO3UCgICSqUjCgAAQCiFKAAAAKEszQUINmZpXe7AlBmPdzXsJkfQ1Mz9DRljYYQQNWKpoKme15TzpkINXr+6ODc3REcUGCMi4EKIRpmGfPhZ+wNSDWPMrbX9acHUa1hk0NQa503fvLjmD7eF1/sW9pFHOqIQ5DQw4lLXZ+jtIqwRctHyX0LXPp6wNbe3+5uU0nsppV8dj+mYUkq7Xdo9bbu7O7zuu2+NIT/nYy7pmtPytR2YRkcUAGjVeymlv33876VtAARTiAIArfpVSuk/Pf730jYAglmaCwA05f33v5s+//ydlFJ6s/R297Bo9f54TC9TSp+mlNLhsMboyGGpcCcgjo4olE2IQ1tqPp5Dxr72/tUwxtxa258sHovQLueFiwCdcUqar8hwp1Js4bzcwj7ySEcUCtb1V92x4RPXAjdKCrNoXdRf6YeGYeVWQxeihjHypdNgoa6woadt0x/z4XyY85hDxzh33CXw+lmX+ac1OqIAQKmWCBvK/ZhDH09IEsAJhSgAUKolwoZyP+bQxxOSBHDC0lwAVncheKRWAlMyeFzC+umlbSfBRENdDDDKPcanc/vu7st/fwxJco5UJPfXWAr6WozzkNXoiAJQgpaK0JTa259ijSxCuyx9rHKH6pQUGET9XKtYjY4oABBqTsDPmNCfu7svf5/l9nYfNp7Tbaed0Bx0r4BW6IgCANHmBPwsEfqTezyCiQCuUIgCANHmBPwsEfqTezyCiQCusDQXoMeFAJ3Nhzs0GC5EoCEhROfbdrv01jm3y/eruIMCjKaM+3BIAHTQEQXolztkpCXm4DLBMfmNOudevPji4v/P+VxQMdcqVqMjCkDT9vt9vr4ZbywZ8NO17dJYjse0ex4QdHh9fptPPvl5Siml29v9zcljvnW73PuXO6yIcri2wDw6ogDAFBEBP2NCf4beds7thBUBZKIQBQCmiAj4GRP6M/S2c24nrAggE0tzIcjhcLj4e3djb5f7eWfcd/PBPS0RQrSMEuY197XldMnp17/+3fT55+88+/euIKE5264ZGhB0FoB0SZYAI2FFAN10RIG5FC1tae14lhLE0dq8PnNehAabc4zH3LfpYwgQTUcUgKIJBOHMm7Chp3CglIYHBJ2FC70835YyBxgJKwLopiMKANSkLwgodwjRnPsKKwK4QiEKANSkLwgodwjRnPsKKwK4wtJcYLbcISgzCU+CBb3//tvBRJFOfyP0NAho6BLYa+FCuQOMcocV5Qq+CrhuuxYPdOGYXp3DOfeFtemIAq0RKNIfwDIkmKWUcJ8npY1nqlb2Y9Ui9MWLLyKepvQAo1qucbWMswR9czVkDufcF1alIwpBlgxcufSX7WvPO/S+hXU9uWDOX8H9BX0ZT/M6JeymL5CnOxjn0Bu0c3u7fyvkZ8p40oUwn7u7On+rZMkAI2FFAN10RAEgztywm7WCdloP34mYVwBOKEQBIM7csJu1gnZaD9+JmFcATmx+aa4veQMQ5SwU5633n66gnL7wnK7tt7f7S09/HrRz9fGuBPc0Y8kAoxxhRbV+NaJj3D5bwQSt1is6or7kDUPVErZSyzihqfeZoCChtZQeYFQLcwPTNFmvbL4jCgxT81/cYGkTQ39q9yb86FJI0pJBbX1ydBBzBRgJKwLopiMKAPNtMfSnpX3pkjvACIATClEAmG+LoT8t7UuX3AFGAJywNBcAZpoSQlS7S8txT81YJrtqCEeuAKPb23168eKL9MknP19glPWpNXgJyE9HFOgL4RD6A9MsEB5R1mf3oGCi0kM4Bl8jP//8nSXHAVAlHVHYOCFE+UwJrHnaNvf+LW8rbTy5Q4ju7jL8vgch5gQYzbFG4NMcup7AEDqiAPnMDayZc/+Wt5U2ntZDiOjn2ANkohAFyGduYM2c+7e8rbTxtB5CRD/HHiATS3NhBYfD4a0wkwWfa8gSqVVDQVoxJbCmL/Bk6G23tq208bQYQkS/mQFGAJzQEYW8hgb/lBbCUdp4WmBOuSoo9KdWAtPq5dgBV+mIQka6inW5Fjpzdzf9vsuOnBp0hRDd3u5v0rPz5p202+2rD2269FMutQXtjDXndT/0elOD5+fI/q0gp8hzDqiDjiiwZXOCR4SWMEXroU1bNGceWppD5xwwikIU2LI5wSNCS5ii9dCmLZozDy3NoXMOGMXSXGCQyIClBXSGMV0LF7q93V96zDfLwgSUMNS1sJtL2+bef8q2p9f96bLRw8OK4/vH5ZefnmzbpKFhRT3Xk9en//7ixRfpk09+nnmEMXKFtT1uu3/8ndYL52aGQQOr0hGFdZQW5DBkPLUWoSkNG3vN+0dhGgoh6ntdeL10m3Vt//zzd3KNY21zzw/nF2yAjigs4HrwQt4ghy0HhQy1VrjQsMCaYWPc6rbSxjPn9VjbPAwN7Ko9aGeOszl869qeTrqemZ6j+PNm7v4OeY4tn3PQCh1RWIbQhvKUNIe1BtGseV6XNJ4551Kt8zB0X7Yo4trS0nkzVEnXbGABClFYhtCG8pQ0h7UG0ax5Xpc0noiQq9LmYei+bFHEtaWl82aokq7ZwAJ2x+Px+q0adjgceifAkkZy6AptuOApoGGUiPP40nNM9f77323pO1GdupbmrqgztClaRcFXk+arpfeVoftS2j7PGc+F83PS+bDbpW1/0Mrn2fvjEufc2Pe5iPfXAedrUa89ltHqcdYRheWN+cBd8ofz7AFLrRehBQbWlHJ+lTKOa2oZJ3nlDmgqLZyuVqW9Hh1XmElYEcwwJBgix2OWENowthMwcG5mh3jkVlgHkw0oOXRGWNE01wKMegJ+irsezjU0rG3oPKx5ztXcdYJS6YjCPGOCIeY8Zo2hDUvMDbSo1tAZr+V+rVzHl+CcA1JKClGYa0wwxJzHrDG0YYm5gRbVGjrjtdyvlev4EpxzQEpJIQqzHI/peDymT0+Xzl3aPvUxh24ryRJzAy0a8/qec33IfW3xWu7XynV8Cc454InviEJGIxNy+x7j/A12UpJuaXLMzdIKDBfKrZRwjftU+LnwqJT5Yry+c6zkY1rL62KQGdfT3nl4/v64Ty9efJE++eTnU59nzHOXfN5AtRSikFfvh4iu0IbT8IMLEf+tfDCZPDdx3kkp7bM8UqtR6zmU8BMytK3Gc6yFPzg+mfNzX13z0Pf+mDt5vcbzBmpmaS4MtNul3W6X/uwxta9329zHjHruqYaOZYm5qXEbTNF3LpV0Hjvf54s6diWdN3PmYYnHcy2H9ShEYbglEg/nJAWulR4YlQa5VjpoRNooXCM1dxuijl1J502XNR/PtRxWohCF4ZZIPJyTFLhWemBUGuRa6aARaaNwjdTcbYg6diWdN13WfDzXcliJ74jCQI8pfZ+m1B28s5uwgOfsMS958wPfj7d7CjD6NKWUDm9/xXIxU+Zh7tzUuC3ymFCnw+HQG+B1d3d6u8vbl9x2yd3d4fX1W3V+X/Ded/EeRFyXop5nzvXv2uNdeg+5ve38Xv/5e+YzfY+32z2ELN3dfRmA5FoOy9ERhWlGBQgNTA8ck8pXSoDR7HFsIKkW+pTyOo621f1murDU2twBSEA/hSh0mBNecDymXXp4bX0npXRzd3fojZc/fczjMb08v+/YMeaWO8RhzNxEKCmsA6AWEde/Oe+PQB28kKHb3PCCiBCiiDCFNQOaIpQU1gFQi4jrn2ssNE4hCt3mhhdEhBBFhCmsGdAUoaSwDoBaRFz/XGOhccKKoMO1QJ4x978UdDAnoCGdhDHc3u7TixdfZF/mOncezh0Oh+PQIJQ5P4g+0P3xuH8T+PSklLCiC0E2VQW9XArkaUhVx4QvjT0/A65LuS1ybkaEtc18f5ylJwBpUX3v4XPOuQXOV9c6stIRhevGfIjOHagw+PECAhZmFRMFhhKVXhz1ja/0cZ+rbbxTTN3HsACWwpS0362fny3vX0nn0WyVhCS1fD6xAh1RNu8xWOG9lNKvHv/a+mzblbvfnN/3/P5DO4A943nZMZ5BP5nQ83hv7d/QbZee4+6unXz7XPM19rizPTV0Fi51VPb7vVAuQl17fxx7zT7fnka+vwLz6IjCMoFBcwIVcgc0CN8Zx3wBlGmJ67PrNqxEIQrLBAbNCVTIHdAgfGcc8wVQpiWuz67bsBJLc9m8OYE8XWEK59vHhjbMCWjoCVh4s9So675Dt7Woa9nh6ZLap2M3ZRvbsnSITYmhTwP3WbhJpZY65+a8VgZeiwcH0Z1vX/u9b0hI0tRgwvff/27n91CXCDqEoRSibN7u490upfSXKaVfpnRcM5hozPMW84G0wBAiylLU+bqyOdeMWuew9HG3fn4658Yp/nyYGmrUd7+Rj9dUQBTrU4iyKedhBbuPd7t0TD9OKf0gpfSTlI4ppd4/ib4VTDQk/CBHlyx3gNFMZ/PwTtrt9pOCIe7uDr3jnhKEUuFPKzSv9W6YMJ+6dZ2fY68jjnPdcgcgTQwDHPUefnu7f/M+fOl99PR2l55jv9/vxgQ8QS6+I8rWvAkleOyE/jgdb76fdmmXjjffT//hv6TUf61dM/ygpGAcwRAAtCJ3AFJEuN3cQMXrt93tvvnD9PFffTv9n//5w/TxX6Xd7psTxgkX6YiyNQ+hBP/+o1+llH6cUvpeunn99ZRSSjevv56+/bOHW/2v/5o6OqNrhh+UFIwjGAKAVuQOQIoIt5sbqHj5trvdn6aU/v6j9NFXfph+dHOTXv/3lNIf0m73F+l4/KcJ44VOClE25XE57kMR+i9/9J/T1z57foOvfZb6itHnAUQPIQ5dIQn7/f7lnLCivnGniQFGuV0by6Vt59tzzA0ATDXnPS3Xtgnv4W+W2d7e7t8KHDoJJhq05He3+3Ip2L9J/5R+m76RXqbfpZuU0lceH+J1Suk+vfzHb+x231aMkouluWzKm+W4KX3vrSL0yVMx+nyZ7vkX9PvCDNYKOYgKECg9qKD08cFUtZ7btY6beo9djeOeNebzwKGpgUZ/nH6T/iH9+Zsi9NRNSund9CqllP7eMl1y0RFlM94EEx1vvv9mOW6fr32W0nf+2z+nf/eTn6Zd+uD44fE4JoAod1jRtedYK2Bh7Lal56akYBzBSeS01rktuGe75pxzuUO8WgwFm/IenjIEE97dPV+KdPqTMd9Kv0436XVvl+px+1dSSt9KKf1m7lhAR5Qt+cuU0g+uFqFPHm73g8f7pTTnS//LKClgQVgRAAwXEWoERVOIsiW/TCn9JL2++edBt3643U8e75fS1C/9L6ekgAVhRQAwXESoERTN0lw24/jh8bj7ePdB2r1OKaXvpZT+1YWb/z7dvP5ZSumDu39997vD4fDumGWkEYE8JQQsjN12vn3O3DwFRk1/hPV0LDO7z7n0ssC5idq/rM/DehxjSpbj/MwdanS6xHaqX6dvpdfpJr1O3Z2qx+1/SCn9evaTQdIRZWOOHx6PKaUPUko/S//yR903etj+s5QevhuayvpAz5daOi6596W0uYnav9L2m+nWPMY1ht0Qa63zc9a5+eLFFxe3/TZ9M/15+of0Kn3jrS+jPqbmppTSX6Tj0fdDyUIhyqbsdmmXPjq+l37x4Qfpa5/9TUrp92c3+X362md/k37x4Qfpo+N7jyEBk55nt0t/NvX+pejajznbLm0HKMF+v3+53+93p/9be0xs1+l75vGYXh6PaZcePr9/J13/HP/mdnd3h2c/8fLkk09+nk4f8/+mP7354/S7b9+k9NtjSvd/SDe/P6Z0f5PSb7+RXvnpFrJSiLI1D1/6/8VH76WnzujTd0Yf/vuzlNIHj/8+JxyglXABYUUAsJ45AUbTQhYfis0/+Sh99NffSf/7/32UPvrrlNKfKELJTSHK1rz50v+bZbq71z9Nx3RMu9c/TV8ux50bDtBKuICwIgBYz5wAo+khi8fjb36UPvy7f0z/9j/+KH34d5bjsgRhRWzKeRDAmwCjlP5HSumXj0XorECdw+FwHBpsVPpvTZ7ux9M8zNl2vh2YrsBQqsWDuHqec9V5GHgdvzoPUQFNlczXnMdrKtBqToDRmM8yY0IHIReFKJv3WHz+Yu1xMNp9KuxD+Ay5w1FKmxvhL8so6Rj3iRhjK/MQFYBTw3zN0fr+neu73rvuUjyFKJv3GJrzXkrpV49//Xu2TQevTLn/4n3pr/S1hZW01A0A4Lmzzy0vO7b5LEMVfEcU5gUBAABEmhsaCEVQiMK8IAAAgEhzQwOhCJbmsnnXvqA/NqyIPFoPoACYYu2woa0pcb6HBgv5LEPpdESBWhT1QQAKUEMYScQYa5iHnFwLY5lvWIiOKGR2HmzTUghObqX/fA2ULGKFwNjX6BrXtK55yH3dHfp4W7mmXZvDyPkH6qUjCgAAQCiFKAAAAKEszYUTJYYSAPVzbQGA53RE4TkfFIEluLaQy9rhTEOev+82a48dKIiOKABAJWr4GasaxgisT0cUAACAUApRAAAAQlmaCyvy22jjrDVfmZ/3fo1laxfCclYZz1BjQ35Kek2VNJYoEfvc8RxFn8PEK/kcEVwGX9IRBYi11geQvuct/QNR6eNjfc4RrinpHClpLLAqHVEAWNF+v99Nud/Y7uOU59liVxeAGDqiAAAAhFKIAgAAEMrSXACyqDUQqSTmcJoxS4gtN96mqONeclASlEZHFCDW/doDWFCtgUhrOj8fzCG05fy12/J7AIyiIwormhpS0rJLf7Vecr7Wel7yij5Wzpvncu2zriWtWqs76jVFiXREAQAACKUQBQAAIJSluZBZBaEZghNoVknLz3KPZe7jlTQ3LRk6r3Pmf4Fj530AWJ2OKGyP0JM69AVaCLoA5vI+EMc1G3roiAIUSLcCug0JRJoTIqVz3KapQVpjz4cthpTBVDqiAAAAhFKIAgAAEMrSXNigzEvPhF7MdDgcXqXu72yZWwBSShffK+Y+7pzPBN6nmExHFJhL6MV8fXNobiHeFsJlatvH2sa7lBLfE0ocE5XQEYXMzoMKBF9AnCWDQnK8lucE5cy9tkyZmy1ev7q6O0OPy1IhSVsPwBl7TIA66IgCAAAQSiEKAABAKEtzAQC4aqmwnAiW8kJ5dESBuYRIACm5FmxBlUVooKVfAyW+xkocE5XQEYWBhgZSTH1MoF5LXB9K5/rF1kW/BvxMCq3REQUAACCUQhQAAIBQClEAAABC+Y4owIP71B3EUXQQw4UUy/sNfp+o6GN1ruYEUhZRwzWob4wAoylEAVLVIRB9Hwqb/7DYQFhO88eI4Wq4BpU0xrGhYFOvF1sJH4M1WJoLAABAKIUoAAAAoRSiAAAAhPIdUahMxQEnWwzPATKb+5093/kjl45zyfscjKAj2p9GV1JKHeVZ87ypsQhNafi4vSaXYV4BllXr+zPla/I9fPMdUX+5YgrnzXLM7TLMKwDUqdX3cB1RAAAAQilEAQAACLX5pbmQ09ggoa2FZqy4v5sPkLhwbm5+bngQEYRWcdhar6HXtczXP69boHo6ovDc3C+DN/UBqyGOS/8clD43TQY0FCriXCj9fKuFeYzjWgML0RGFE/7CPM1+v99du83Wur/k4TUJrKnrGuT9DPLQEQUAACCUQhQAAIBQluYCAFCEpQKtLKeF8uiIQl5bDDUYus9rzs0WjwuUyGsxj5bnURATbISOKGQkWKWfuQFqvQ5c6qYNCWsD4G06ogAAAIRSiAIAABDK0lxgkAsBEve1LrcD6tHSNailfeG5jmXcjin00BEFhuoLkBAswZO+AJWWg1Wnib2ZAAAIdElEQVS6mIdltHQNamlfcmvtdeKYQg8dUQCy8Ff/B+YBppvz+vETLVAXHVEAAABCKUQBAAAIZWkuANUT/lKeC8dkiefKvSTTeQOwMB1RAFqwxfCX0kNdap77mse+ZSW+JkocExRBRxQAKtTVsRPWwpaN6WJfeq3s9/vdUvcFvqQjCgAAQCiFKAAAAKEszQVm61imJOgDAIBeOqLAEgR9xOkLwhCQUb7Wj1HN+1fz2AGqoCMKUDGd53q1HjaU+9wUEAPQFh1RAAAAQilEAQAACGVpLsAIh8PhVer+DqyAJqAoF65XEbJdE113oU06ogDj9H2oE9DElmwtzKfWULA1r0s5n9t1FxqkIwoAXLT1MCBdN4D8dEQBAAAIpRAFAAAglKW5AAWL+F3JjucoJgBESMk2jQ3ZqeD3VwefrxXsy2AlX1sirBwWFWVTx5S8dEQBOFfSBychJdvU2vFtbX+m2to8bGF/t7CPLERHFACAwa6FV7XU1QWWoyMKAABAKIUoAAAAoSzNBchk68EccErQFF1yL9tdYhmwpcUQQ0cUYDlDQxzuFx3FeKWNh+H6jt2QYzrnvl3mBE21dg62tD8t7Utu53Ozhbnawj6yEB1RgJXpDpHLnHOppPOwayyXulTXwnNqMicIqKV5qMG1+S7pNeW8oUQ6ogAAAIRSiAIAABDK0lwAVnch2AagSDNCjQR2QdIRBaAMNRShucN8gG2q4XoHi9MRBYABdDBgvinBOH5OBdqkIwoAAEAohSgAAAChLM0F4CJBQvWo5VhZagmAjigA1xRf2PCGY1UHwVfb5jhD0hEFAAgl+CqfKeFHKV3uyk99TGAcHVEAAABCKUQBAAAIZWkuLOxCeMj9lOVZGwkjmTQ3AFCKyt+vvQ+zOB1RWF7fm9DUN6fi39Qy2MI+1qSGYA3hLw+2tr9Qsprfy2oeO5XQEQXgooi/is/9OQ9/uX9Q0jzkDoPxky8AbdERBQAAIJRCFAAAgFCW5gIANMDyZaAmOqJQny2EkZS8jyWPDYBhIgLOan6/qHnsVEJHFCqzVhjJ2L+0TwkjqUHX/OtCANQl4r205fAwyEFHFAAAgFAKUQAAAEIpRAEAAAjlO6IAZHE4HF6llN4NfL6p3829L+G7W2Pny3eR033qni+hKgAVUogCkEtYETpTKeMsZRxVKOGPB6UrJXTGH02AISzNBQAAIJRCFAAAgFAKUQAAAEL5jihcsWQAy0rfoykiqAXW5LXHGBfeBxxTgIl0ROG61gJFWtsfyiG99LLz1575qkffdXON62nfeeN8AqqiIwpAFnM6Q1M6lNcSQktP7uyar0tjLiURdajS579WOrBAK3REAQAACKUQBQAAIJSluQCNWTJgCwAgBx1RuK61AIjW9oe3KUIflHaulzYeYDuEXFEcHVG4Ym4wREvhI1AToS4AD1wPKZGOKAAAAKEUogAAAISyNBcyGhsSs9Lv7N1bogPUptQQLtdxgGl0RCGv4j4kdYgYo/CDdW1h/lvdR4Ei/Wq4vkbZ2lx4XUCDdESB2YQulaXGTolQrwc1HjtYmtcFtElHFAAAgFAKUQAAAEJZmgsACys1aKeDEBxokGsQJdIRhbxqCE6YOkZhEf3MzXytz2ENHwBTKnucrZwLOZQ+F62/nmtU8mv7VC3jJIPd8bhG6jhshxAWYKWf+Jgk13WppWtfS/vCNm3xGkT5dEQBAAAIpRAFAAAglLAiAICFVBQSM4eAGWA0HVFYntAGoJbXey3jrEnrRWhK29jH2tXy2q5lnGSgIwoL81diwHUAWJNrECXSEQUAACCUQhQAAIBQClEAAABCKUQBAJazhfCVLewjkJmwIgCAhQiJAeimIwoAAEAohSgAAAChFKIAAACEUogCAEvoC7CpMdimpX0BKMLueDyuPQYAAAA2REcUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACDU/weM6W3qaeGRAwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (1.4) A* search search: 133.0 path cost, 440 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAJCCAYAAADay3qxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3c+OJNl1H+CTTQ7RpDBNcUXAK0Nbiaa5l0C+gC0QBrIWBDULQ/Q8BaeHT0FD8KItEEYlYAxkvwAJaq+hKe+9MsCNKXXB9IADdnpRVT3ZWfkn/tw4cW/E922GzK7KvHHjZmSeOjd/udnv9wEAAABZns09AAAAANZFIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqb489wAAgPXa7XavI+L9E/90t91uX2SPB4AcOqIAwJxOFaGXbgdgARSiAAAApFKIAgAAkEohCgAAQCqFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApPry3AMAprHb7V5HxPsn/uluu92+yB4PAAA80hGF5TpVhF66HQAAUihEAQAASKUQBQAAIJVCFAAAgFTCigCYTNfQrCnCtWoP7Kp9fAAwJR1RAKbUNTRrinCt2gO7ah8fAExGIQoAAEAqhSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkOrLcw8AgOntdrvXEfH+iX+62263L7LHAwCsm44owDqcKkIv3Q4AMBmFKAAAAKkUogAAAKRSiAIAAJBKIQoAAEAqqbkAM5NoCwCsjY4owPwk2gIAq6IQBQAAIJVCFAAAgFQKUQAAAFIJKwKADoRKrYPzDJBDRxQAuhEqtQ7OM0AChSgAAACpFKIAAACkUogCAACQSiEKAABAKoUoAAAAqRSiAAAApFKIAgAAkEohCgAAQKovzz0AALrZ7XavI+L9Ce53P/BX77bb7Yuig4GVmur5ncS1AOhNRxSgHbW9Sa1tPNCylp9PLY8dmIlCFAAAgFQKUQAAAFIpRAEAAEglrAhYtAsBIMI1GlUi1GVEQNOY+7Pmeip9nsZyngHK0REFlu5cwdJiuMbd3AM4Mtd4Wjx3Ee2Om36Gnufant99tDx2YCY6ogCNGNNludTJ2W63m6H3C5ShiwqsjY4oAAAAqRSiAAAApLI1FyYmLGcaMwfWOHcM0nXNzRjSc3Vtl3jukcPrD1AzHVGY3lxhOefCI5YSKjHnG2Fvwue1lDVcoy5r2/q/rKb1uaSwNmBhdERhofy1m6Waa22PCXyq7WtIWjZ1uJZgL4AcOqIAAACkUogCAACQytZcoChBJqxd688B24gByKAjCpTW7BtwKMRzYDo1BQEBMIKOKADQm+AeAMbQEQUAACCVQhQAAIBUtuZCQXOFlFx43DvfJzoP5wTgspmDvVyLYWY6olDWXC+o5x53jvGsIUykyzHWdE7mdG6uWlwnXY+lxWODOcx5PVzbtRiqoyMKFFXbX5gvfRWFsJXp1bYexuh6LEs6Zl/lAsBUdEQBAABIpRAFAAAgla25QIqZQykoLPN8ntkeKmiEdLYq04fQOrhMRxTIoghdlrnP59yPD4w3Z7BXxmMLrYMLdEQBAEinKwjrpiMKAABAKoUoAAAAqWzNBZ5YS7DQXMEjAk9gWkJiAOqnIwqcsvgidIXOBXMMDeyYM2SkhsenbkJiACqnIwqwAqW7QGPu71JHeLvdbobeL0ythvVpRwWwFDqiAAAApFKIAgAAkMrWXJjRiS1WgjQAkqwlmK2UMSFQAqSAYzqiUJda3hAJggHWoJZrbivGhEAJkALeoSMKPLGkv07PFYzT53GFjwAAa6MjCgAAQCqFKAAAAKlszQVI1mcr7kzbdoWHAACT0hEF4JjwEHhKiBtAQTqikOQwoEY4DUDdpgwzA0BHFAAAgGQKUQAAAFLZmguN2e12r8Nn+AAiov5rYu3jA5iLjii0xxuafs4FjAgeOc/c8KiF50/t18TaxwcwCx1RYNFq/BqS4xCUS+FVAlOYU43PHwCWQUcUAACAVApRAAAAUtmaCxC5gSJ9vkd2xHfO3tWwrbLvvCZ9x+7VuVlJwMw787CSYwagEjqiAPeW9ga8luOpZRyHuoypxnGXdnyMazjmOdQU7ARQDR1RAICJlN6ZkLRrAGByOqIAAACkUogCAACQSiEKAABAKp8RBbh3F8sKa6klIKXKefU5u3vmgRrMtA6fpGdfSI6WtA0TUIgCRG6gyHa73Qz92dacmtdSc1OTIeeplWPrY471usR5JMWpovFcISlpGyZgay4AAACpFKIAAACkUogCAACQymdEAQAuEEQDUJ6OKCxbLcmpMFQLa3joGFs4tj6WdjyHFKHLs+T1Ck3QEYUFaT1hFY6VTjOuyZKPDfrw2gXrpCMKAABAKoUoAAAAqWzNhSRdv3R9zJezJ3yx+93U2wmFgox3YQ4nP3/My7m/LuNaDMB1OqJAHxkFoiJ0vHNzaG6Xz7mnC0E9wOx0RAEACpoyfKdvp1YQEFArHVEAAABSKUQBAABIZWsuQLI+W+tKB6aMuL+rYTclgqZGHm/KGCsjhGghpgqaOvOcsm4atMDr1ynW5oroiAJ9ZARcCNGoU5c3P3O/QWphjKUt7XiWYOg1LDNoao51c25eXPO7W8PzfQ3HyAMdUUhyGBhxqevT9ecyzBFyseS/hM59PmFtbm62zyLi2xHxq/0+9hERm01sHm+7vd29Ofe7LYb8HI+5pmvOkq/twDA6ogDAUn07Iv7rw38v3QZAMoUoALBUv4qIf/fw30u3AZDM1lwAYFE++OD78dln70VEvN16u7nftHq338eLiPg0ImK3m2N0lDBVuBOQR0cU6ibEYVlaPp9dxj738bUwxtKWdjxFPBShpxwXLgJ0+qlpvjLDnWqxhnW5hmPkgY4oVOzUX3X7hk9cC9yoKcxi6bL+St81DKu0FroQLYyRyw7Dhh4DiIb//v16GHufa+H5My/zz9LoiAIALRkbNiTACKACClEAoCVjw4YEGAFUwNZcAGZ3IXikVQJTJvKwdfbTx/9/EEzU1cUAo9KE6ixD6Y+xVPSxGOuQ2eiIAlCDJRWhEcs7nmr1LEJPmfpclQ7VqSkwiPa5VjEbHVEAoEqnQoT6BAvd3n7x/Sw3N9vJHieT7hWwFDqiAECtsoKFBBgBJFOIAgC1ygoWEmAEkMzWXIAzhIyct8BwISp0GEy02cSTNbcp9624qQFGAOiIAlxSOmRkSczBZYJjyuu15p4///zi/y/5WNAw1ypmoyMKwKJtt9tyfTPe6hrwU+q2S2PZ72Nz+LO3t7s3xz/z6tUnERFxc7N9dnCfT36u7/GxXq4tMI6OKAAwRNeAn9K39RlP6Z8TYARQiEIUABiia8BP6dv6jKf0zwkwAijE1lxIstvtOm3j6vpzpR93xO+uPrhnSYQQTaOGeS19bbm9/eJ/f/Wr34/PPnvvnX8/FSQ05rZrDoONdrtuP3flcQQYAUxIRxQYS9GyLEs7n7UEcSxtXt9xXIQmG3OO+/zuos8hQDYdUQCqJhCEI2/Dhg4Dg94NKzr/y0eBQy+ObwsBRgApdEQBgJacCwzKCCESYARQiEIUAGjJucCgjBAiAUYAhdiaC4xWOgRlJOFJMKEPPngaTJTp8DtCD0OJLm3HPXQYVnTqttoDjEoFXyVct12LO7pwTq/O4ZjfhbnpiAJLI1DkfABLl2CWWsJ9HtU2nqGWchyzFqHPn3+e8TC1Bxi1co1rZZw1ODdXXeZwzO/CrHREIcmUgSuX/rJ97XG7/m5lXU8uGPNXcH9Bn8bjvHYNuzkXgHPtZw+7hcdubrZPQn6GjCcuhPnc3l743pSKTRlg1LVTC7A2OqIAkKdr2E2fQJ7SQTtrDOQRYASQTCEKAHm6ht30CeQpHbSzxkAeAUYAyVa/NdeHvAHIchSK8+T151RQzrnwnFO339xsLz38cdDO1fu7EtyzGFMGGO0K7FZu9aMRJ8btvRUMsNR6RUfUh7yhq1bCVloZJyzqdSYpSGgutQcYtcLcwDCLrFdW3xEFumn5L24wtYGhP617G350KSRpyqC2c0p0EEsFGAkrAjhNRxQAxltj6M+SjuWU0gFGABxQiALAeGsM/VnSsZxSOsAIgAO25gLASENCiFp3aTvuoRHbZGcN4SgVYHRzs43nzz+PV68+mWCU7Wk1eAkoT0cUOBfCIfQHhpkgPKKu9+5JwUS1h3B0vkZ+9tl7U44DoEk6orByQojKGRJY83jb2N9f8m21jad0CNHtbYHv9yDFmACjMeYIfBpD1xPoQkcUoJyxgTVjfn/Jt9U2nqWHEHGecw9QiEIUoJyxgTVjfn/Jt9U2nqWHEHGecw9QiK25MIPdbvckzGTCx+qyRWrWUJClGBJYcy7wpOvPru222sazxBAizhsZYATAAR1RKKtr8E9tIRy1jWcJzClXJYX+tEpgWrucO+AqHVEoSFexLddCZ25vh//utCOnBadCiG5uts/inXXzXmw22+ZDmy59lUtrQTt9jXned73etODdNbJ9EuR0bW33JRAJ2qcjCqzZmOARoSUMsfTQpjUaMw9LmkPrBuhFIQqs2ZjgEaElDLH00KY1GjMPS5pD6wboxdZcoJPMgKUJnAxjuhYudHOzvXSfb7ciCiihq2thN5duG/v7Q257fN4fbhvd3e84vnvYfvnpwW2r1DWs6Mz15M3hvz9//nm8evVJ4RHm6BjWdvfw/atP1jawPjqiMI/aghy6jKfVIjSi29hbPj4qs6AQonPPC8+X00Zd2z/77L1S45ibdQNcpSMKE7ge0NAvyEFQyHhzhQt1C6zpNsa13lbbeMY8H1ubh66BXa0H7YxxNIdPru1x0PUs9BjVr5sSxwIsn44oTENQSH1qmsNWg2jmXNc1jWfMWmp1HroeyxplXFuWvm6sJVghhShMQ1BIfWqaw1aDaOZc1zWNJyPkqrZ56Hosa5RxbVn6urGWYIU2+/26d0Bc+h4qWxop4VRoQ0+P4Q5nZazjKb6z7YMPvr+kz0SddGpr7oxOhjZlayj4atB8Lel1peux1HbMY8ZzYX0OWg+bja2mV1x9jTtlijXX93Uu4/W1w3qt6rnHNJZ6nnVEYXpj33DX8oa9eMDS0ovQCgNrallLtYzjmlbGSVmlg3ZqC6erTavPM+cVRhJWBCN0CYaY6nGyg0L6dgI6zs3oEI/SKutgsgI1h84IKxrmWoDRmYCf6q6HYx1eTy99HVbptTmFlrtOUCsdURinTzBE6cepPdwha26gdUsKneFe6bleOvMFK6QQhXH6BEOUfpzawx2y5gZat6TQGe6VnuulM1+wQgpRGGG/j/1+H58ef+/ZudtLPk7pxygta26gdX2e311/tvRtXcfNvdJzvXTmC9bJZ0ShoAIJuefu9/hFd1DK4JymmpuSKgwXKq2WcI27qHwtPKhlvujv3Bqr+Zy28rzo5Ph6+vz552cD6oYlC2/j+fPP49WrT4YM75wW1w00SyEKZZ19E3EqBOda+MGFF+cW36wUnZtpvBcR5wM1+lhq1HoJNXyFDMvW4hpr7Y+Ll5y6/j0WjIfXv7FfbVM6eb3FdQMtszUXRthsYrPZxL9+SPIr8vtd73PsYw/VdcxZc1P7bXBNn7VU0zq23sfLOndrWzdTzA1QnkIUxhmb4NdiYmVWuuFc6aAZaaNwSGruemWdu7WtmynmBihMIQrjjE3wazGxMivdcK500Iy0UTgkNXe9ss7d2tbNFHMDFLbZ79cdPOZzXJR06fMuhT8j+sS50IYp1/GmUABRPZ8RLce1Zb26nvvdbld9gNdE7mr+LJ7n7nhd57DPa1yWCQKQRrm25qzXdVjqedYRhbp1TuorHdrQ0eg30StIqoVz1liERqz3uHmqujTamV5LYZUUotBRVqDB4X3u9/Fiv49N3D9XvxPJz9nSx3x8LLe3u1n/8iysCCDXtde4/T42XW6b7QCAYjyRobusQIOaghOyQojmIqwIIJdrLBARClHoIyvQoKbghKwQorkIKwLI5RoLRETEl+ceALRiv499RHwacTqkZ1NoM+bh45x57DTXjrmv3W63v73t/rNjHquDu/1++yIuzPWY23ZP85d6uRBkU3XQy7GVBPI0dU74Qt/1mXBdKq26tVnqGjvla+HNzXa6Oz/jXEjSmDU3wXqtbj3RNh1RGKbXG+uFBPKMKiYqnIPai6Nz46t93MdaG+8QQ4+xuqCWJDUd99LX55KPr6Z1NFojIUlLXk/MQEcUTngIm/l2RPzq4S+w79x26Xf3+9gc/uzt7e5NyfGM+d1Tx9L1tkuPcerrV1pVar66dn5ZrxY6C0v9ygDadHTdfXHitl7X7OPbI2L06zXQnY4onDY2JKF0oMKY+xMM0Y/5AqjTFNdn122YiUIUThsbklA6UGHM/QmG6Md8AdRpiuuz6zbMxNZcOGFsSE/J0JoT4znrTMDC261Gp363621LdGrb4eGW2sdzN+Q21mXqEJsaQ586HrNwk0ZNtebGPFc6Xos7B9Ed3z73a1+XkKRzoUbXfPDB909+DnXo/UEJClFWb/PxZhMR342IX+w/2p96gezzQpwRnnAXFb0hrTCEiLpUtV5nNub60Ooc1j7upa9Pa66f6tfD0FCjc7/X8/4WFRDF/BSirMpxWMHm480m9vHTiPjriPibzcebD/cf7fc9woGeHd7fqccp0SW7FtAQuQELR8f8Xmw220HBEJeCnIYEoTT41QqLt/RumDCftp1an32vI85z20oHIA0MA+z1Gn5zs337OnzpdfTw5y49xna73fQJeIJSfEaUtXkbSvDQCf1p7J/9MDaxif2zH0bETx9u7xpekBV+UFMwjmAIAJaidABSRrjd2Pco1392s/nGj+Pjv/xW/I//9uP4+C9js/nGgHHCRTqirM19KMH3Xv4qIn4aET+IZ2++GhHx8N8fRETE915+GD9/2SW8ICv8oKZgHMEQACxF6QCkjHC7rr87LGRxs/mziPjly3j5pR/HT549izf/OSL+EJvNX8R+/48DxgsnKURZlYftuPdF6O+/9qP4yu+Of+SP4vdf+1G8/79/FLGPiMs7rg63xDyGJGy32xdThhWdui0zYOHaWC7ddnx7ibkBgKHGvKaVum3Aa/jb9x43N9sngUMHwUSdtvxuNl9su/3T+Mf4p/h6vIh/jmcR8aWHu3gTEXfx4tdf32y+pRilFFtzWZW323EjfnCiCL33ld9FfOtnEf/mP0Rc+EjEmZCeuUIOsgIEag8qqH18MFSra7vVcdPuuWtx3KPGfBw4NDTQ6I/jt/H38edvi9BDzyLi/XgdEfFL23QpRUeU1XgbTLR/9sO323HPeSxGIyL++3+Mh85op3CAiPJhRdceY66Ahb63TT03NQXjCE6ipLnWtuCe9Rqz5kqHeC0xFGzIa3gUCCa8vX13K9LhV8Z8M34Tz+LN2S7Vw+1fiohvRsRvx44FdERZk+9GxF9fLUIffeV3Ed/5TxH/8hePtwz/0P80agpYEFYEAN1lhBpB1RSirMkvIuJv4s2z/9fpp3//tYh/+PcR/+u7j7cM+9D/dGoKWBBWBADdZYQaQdVszWU19h/t95uPNx/G5k3EfTruH5394d9/LeLXPzjclhvXtuO+81gJgTw1BCz0ve349jFzs9vtXkflXzx+zoltZnclt15WODdZx1f0cZiPc0zNSqzP0qFGh1tsh/pNfDPexLN4E6c7VQ+3/yEifjP6wSB0RFmZ/Uf7fUR8GBE/i99/7fQPnShCzwQTMa+aCq2xSh9LbXOTdXy1HTfDzXmOWwy7Iddc63PU2jz1Xubwtn+Kb8Sfx9/H6/j6kw+jPqTmRkT8Rez3Ph9KETqirMr9h/73347vvfwwvvdxxNPO6P+Nr/zuZ3H3Lz6M2HQKJjr/ONOGFWVoLawIYKxTHS3hY8zlWqhRXA4wuhqy+OrVJ7HdbjeP9/k/489+9cfxz38aEb/cR3zpTTx79izevHkW8Yevx2vfI0pROqKszf2H/n/+8tvx2Bl9/Mzo/X9/FhEfPvz7mHCApYQLCCsCgPmMCTAaFrJ4X2z+yct4+VffiX/4Py/j5V9FxJ8oQilNIcravP3Q/9ttups3fxv72Mfmzd9GxIcPt48NB1hKuICwIgCYz5gAo+Ehi/v9b38SH/3dr+Nf/dufxEd/ZzsuU7A1l1U5DgJ4G2AU8V8i4hcPReioQJ3dbrfvuuW09u1eh8fxOA9jbju+HRiuwlCqyYO4zjzmrPPQ8Tp+dR6yApoama8x97eoQKsxAUZ93sv0CR2EUhSirN5D8fnzucdBb3dR2ZvwEUqHo9Q2N8JfplHTOT4nY4xLmYesAJwW5muMpR/fsXPXe9ddqqcQBZpU+i/el/5Kv91uL/zNuT5L6gYA8K5rAUanQgPtRqJGPiMKAADtGBsaCFVQiAIAQDvGhgZCFWzNBaq09AAKgCHmDhtamxrnu2uw0JjgRcigIwq0oqo3AlCBFsJIMsbYwjyU5FqYy3zDRHREobDjYJslheCUVvvX10DNMnYI9H2OznFNOzUPpa+7Xe9vLde0a3OYOf9Au3REAQAASKUQBQAAIJWtuXCgxlACoH2uLQDwLh1ReJc3isAUXFsoZe5wpi6Pf+5n5h47UBEdUQCARrTwNVYtjBGYn44oAAAAqRSiAAAApLI1F2bku9H6mWu+Cj/u3Rzb1i6E5cwynq76hvzU9JyqaSxZMo75xGNUvYbJV/MaEVwGX9ARBcg11xuQc49b+xui2sfH/KwRrqlpjdQ0FpiVjigAzGi73W6G/F7f7uOQx1ljVxeAHDqiAAAApFKIAgAAkMrWXACKaDUQqSbmcJg+W4htN16nrPNec1AS1EZHFCDX3dwDmFCrgUhzOl4P5hCW5fi5u+TXAOhFRxRmNDSkZMku/dV6yvma63EpK/tcWTfvKnXMupYs1VzdUc8paqQjCgAAQCqFKAAAAKlszYXCGgjNEJzAYtW0/az0WMbeX01zsyRd53XM/E9w7rwOALPTEYX1EXrShnOBFoIugLG8DuRxzYYzdEQBKqRbAad1CUQaEyKlc7xMQ4O0+q6HNYaUwVA6ogAAAKRSiAIAAJDK1lxYocJbz4RejLTb7V7H6c9smVsAIuLia8XY+x3znsDrFIPpiAJjCb0Y79wcmlvIt4ZwmdaOsbXxTqXG14Qax0QjdEShsOOgAsEXkGfKoJASz+UxQTljry1D5maN169T3Z2u52WqkKS1B+D0PSdAG3REAQAASKUQBQAAIJWtuQAAXDVVWE4GW3mhPjqiwFhCJIAI14I1aLIITTT1c6DG51iNY6IROqLQUddAiqH3CbRriutD7Vy/WLvs54CvSWFpdEQBAABIpRAFAAAglUIUAACAVD4jCnDvLk4HcVQdxHAhxfJuhZ8nqvpcHWs5gZRJtHANOjdGgN4UogDRdAjEuTeFi3+zuICwnMWfI7pr4RpU0xj7hoINvV6sJXwM5mBrLgAAAKkUogAAAKRSiAIAAJDKZ0ShMQ0HnKwxPAcobOxn9nzmj1JOrCWvc9CDjuj5NLqaUuqoz5zrpsUiNKL7uD0np2FeAabV6usz9Vvka/jqO6L+csUQ1s10zO00zCsAtGmpr+E6ogAAAKRSiAIAAJBq9VtzoaS+QUJrC82Y8XhXHyBxYW2ufm64lxGE1nDY2lldr2uFr3+et0DzdEThXWM/DL6oN1gL4rycn4Pa52aRAQ2VylgLta+3VpjHPK41MBEdUTjgL8zDbLfbzbWfWVv3lzI8J4E5nboGeT2DMnREAQAASKUQBQAAIJWtuQAAVGGqQCvbaaE+OqJQ1hpDDboe85xzs8bzAjXyXCxjyfMoiAlWQkcUChKscp65AVq9DlzqpnUJawPgKR1RAAAAUilEAQAASGVrLtDJhQCJu1a32wHtWNI1aEnHwrtObON2TuEMHVGgq3MBEoIleHQuQGXJwSqnmIdpLOkatKRjKW1pzxPnFM7QEQWgCH/1v2ceYLgxzx9f0QJt0REFAAAglUIUAACAVLbmAtA84S/1uXBOpnis0lsyrRuAiemIArAEawx/qT3UpeW5b3nsa1bjc6LGMUEVdEQBoEGnOnbCWlizPl3sS8+V7Xa7mep3gS/oiAIAAJBKIQoAAEAqW3OB0U5sUxL0AQDAWTqiwBQEfeQ5F4QhIKN+Sz9HLR9fy2MHaIKOKEDDdJ7btfSwodJrU0AMwLLoiAIAAJBKIQoAAEAqW3MBetjtdq9CzRnhAAAHpElEQVTj9GdgBTQBVblwvcpQ7JrougvLpCMK0M+5N3UCmliTtYX5tBoKNud1qeRju+7CAumIAgAXrT0MSNcNoDwdUQAAAFIpRAEAAEhlay5AxTK+V/LEY1QTACKkZJ36huw08P2rnddrA8fSWc3Xlgwzh0VlWdU5pSwdUQCO1fTGSUjJOi3t/C7teIZa2zys4XjXcIxMREcUAIDOroVXLamrC0xHRxQAAIBUClEAAABS2ZoLUMjagzngkKApTim9bXeKbcC2FkMOHVGA6XQNcbibdBT91TYeujt37rqc0zG/e8qYoKmlrcElHc+SjqW047lZw1yt4RiZiI4owMx0hyhlzFqqaR2eGsulLtW18JyWjAkCWtI8tODafNf0nLJuqJGOKAAAAKkUogAAAKSyNReA2V0ItgGo0ohQI4FdEDqiANShhSK0dJgPsE4tXO9gcjqiANCBDgaMNyQYx9epwDLpiAIAAJBKIQoAAEAqW3MBuEiQUDtaOVe2WgKgIwrANdUXNrzlXLVB8NW6Oc8QOqIAAKkEX5UzJPwo4nJXfuh9Av3oiAIAAJBKIQoAAEAqW3NhYhfCQ+6GbM9aSRjJoLkBgFo0/nrtdZjJ6YjC9M69CA19car+Ra2ANRxjS1oI1hD+cm9txws1a/m1rOWx0wgdUQAuyvir+Niv8/CX+3s1zUPpMBhf+QKwLDqiAAAApFKIAgAAkMrWXACABbB9GWiJjii0Zw1hJDUfY81jA6CbjICzll8vWh47jdARhcbMFUbS9y/tQ8JIWnBq/nUhANqS8Vq65PAwKEFHFAAAgFQKUQAAAFIpRAEAAEjlM6IAFLHb7V5HxPuJjzf0s7l3NXx2q+98+Sxy3MXp+RKqAtAghSgApaQVoSPVMs5axtGEGv54ULtaQmf80QTowtZcAAAAUilEAQAASKUQBQAAIJXPiMIVUwawzPQ5miqCWmBOnnv0ceF1wDkFGEhHFK5bWqDI0o6Hekgvvez4uWe+2nHuujnH9fTcurGegKboiAJQxJjO0JAO5bWE0NqTO0/N16Ux15KI2lXt898qHVhgKXREAQAASKUQBQAAIJWtuQALM2XAFgBACTqicN3SAiCWdjw8pQi9V9tar208wHoIuaI6OqJwxdhgiCWFj0BLhLoA3HM9pEY6ogAAAKRSiAIAAJDK1lwoqG9IzEzfs3dniw7QmlpDuFzHAYbREYWyqnuTdELGGIUfzGsN87/UYxQocl4L19csa5sLzwtYIB1RYDShS3VpsVMi1Otei+cOpuZ5AcukIwoAAEAqhSgAAACpbM0FgInVGrRzghAcWCDXIGqkIwpltRCcMHSMwiLOMzfjLX0OW3gDGFH3OJeyFkqofS6W/nxuUc3P7UOtjJMCNvv9HKnjsB5CWICZvuJjkFLXpSVd+5Z0LKzTGq9B1E9HFAAAgFQKUQAAAFIJKwIAmEhDITFjCJgBetMRhekJbQBaeb63Ms6WLL0IjVjHMbauled2K+OkAB1RmJi/EgOuA8CcXIOokY4oAAAAqRSiAAAApFKIAgAAkEohCgAwnTWEr6zhGIHChBUBAExESAzAaTqiAAAApFKIAgAAkEohCgAAQCqFKAAwhXMBNi0G2yzpWACqsNnv93OPAQAAgBXREQUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASKUQBQAAIJVCFAAAgFQKUQAAAFIpRAEAAEilEAUAACCVQhQAAIBUClEAAABSKUQBAABIpRAFAAAglUIUAACAVApRAAAAUilEAQAASPX/AeIfP+8eZiEsAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (2) A* search search: 134.2 path cost, 418 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAJCCAYAAADay3qxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3cGOJMl5H/CoJpcYUtileCLgk6GrRNO6SxBfQBYIA9UHgtqDIXqfgpzlU9AQfFgLPHQBBiH7BUhQd+2a8t0nA7yY0jRML7jglA/dPVNTnZmVWRn5ZUTk7wcQK+V0dUVGRkbV1xH1r93xeEwAAAAQ5WbtBgAAALAtClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACPXltRsAAGzX4XB4lVJ6v+Of7vf7/QfR7QEghhVRAGBNXUXo0HEAGqAQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACCUQhQAAIBQClEAAABCKUQBAAAIpRAFAAAglEIUAACAUApRAAAAQilEAQAACKUQBQAAINSX124AsIzD4fAqpfR+xz/d7/f7D6LbAwAAT6yIQru6itCh4wAAEEIhCgAAQCiFKAAAAKEUogAAAIQSVgTAYsaGZi0RrlV6YFfp7QOAJVkRBWBJY0OzlgjXKj2wq/T2AcBiFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKG+vHYDAFje4XB4lVJ6v+Of7vf7/QfR7QEAts2KKMA2dBWhQ8cBABajEAUAACCUQhQAAIBQClEAAABCKUQBAAAIJTUXYGUSbQGArbEiCrA+ibYAwKYoRAEAAAilEAUAACCUQhQAAIBQwooAYAShUtvgOgPEsCIKAOMIldoG1xkggEIUAACAUApRAAAAQilEAQAACKUQBQAAIJRCFAAAgFAKUQAAAEIpRAEAAAilEAUAACDUl9duAADjHA6HVyml9xf4vccrH3q/3+8/yNoY2Kil7u8g5gJgMiuiAPUo7U1qae2BmtV8P9XcdmAlClEAAABCKUQBAAAIpRAFAAAglLAioGkDASDCNSqVI9RlRkDTnN9nzE2U+zrN5ToD5GNFFGhdX8FSY7jG/doNOLNWe2q8dinV226mufY6l3Z/T1Fz24GVWBEFqMScVZahlZz9fr+79vcCeVhFBbbGiigAAAChFKIAAACEsjUXFiYsZxkrB9a4dlxl7JhbMaTn4tjOce8Rw+sPUDIrorC8tcJy+sIjWgmVWPONsDfh62plDJdozNg2/oeVND5bCmsDGmNFFBrlr920aq2xPSfwqbSvIanZ0uFagr0AYlgRBQAAIJRCFAAAgFC25gJZCTJh62q/B2wjBiCCFVEgt2rfgEMm7oHllBQEBMAMVkQBgMkE9wAwhxVRAAAAQilEAQAACGVrLmS0VkjJwPPe+z7RdbgmAMNWDvYyF8PKrIhCXmu9oPY97xrt2UKYyJhzLOmarKmvr2ocJ2PPpcZzgzWsOR9ubS6G4lgRBbIq7S/MQ19FIWxleaWNhznGnktL5+yrXABYihVRAAAAQilEAQAACGVrLhBi5VAKMou8nj3bQwWNEM5WZaYQWgfDrIgCURShbVn7eq79/MB8awZ7RTy30DoYYEUUAIBwVgVh26yIAgAAEEohCgAAQChbc4FnthIstFbwiMATWJaQGIDyWREFujRfhG5QXzDHtYEda4aMlPD8lE1IDEDhrIgCbEDuVaA5v29oRXi/3++u/b2wtBLGpx0VQCusiAIAABBKIQoAAEAoW3NhRR1brARpAATZSjBbLnNCoARIAeesiEJZSnlDJAgG2IJS5txazAmBEiAFvMOKKPBMS3+dXisYZ8rzCh8BALbGiigAAAChFKIAAACEsjUXINiUrbgrbdsVHgIALMqKKADnhIfAc0LcADKyIgpBTgNqhNMAlG3JMDMArIgCAAAQTCEKAABAKFtzoTKHw+FV8hk+gJRS+XNi6e0DWIsVUaiPNzTT9AWMCB7pp294UsP9U/qcWHr7AFZhRRRoWolfQ3IegjIUXiUwhTWVeP8A0AYrogAAAIRSiAIAABDK1lyAFBsoMuV7ZGd85+x9Cdsqp/Zr0HfsXuybjQTMvNMPGzlnAAphRRTgQWtvwEs5n1LacWpMm0psd27n57iFc15DScFOAMWwIgoAsJDcOxOCdg0ALM6KKAAAAKEUogAAAIRSiAIAABDKZ0QBHtyntsJaSglIKbJffc7ugX6gBCuNw2fp2QPJ0ZK2YQEKUYAUGyiy3+931/5sbbr6NVfflOSa61TLuU2xxnhtsR8J0VU09hWSkrZhAbbmAgAAEEohCgAAQCiFKAAAAKF8RhQAYIAgGoD8rIhC20pJToVr1TCGr21jDec2RWvnc0oR2p6WxytUwYooNKT2hFU4lzvNuCQtnxtM4bULtsmKKAAAAKEUogAAAISyNReCjP3S9Tlfzh7wxe73S28nFAoy30AfLn79WJdrf1nEXAzAZVZEgSkiCkRF6Hx9fahv2+faM4agHmB1VkQBADJaMnxn6kqtICCgVFZEAQAACKUQBQAAIJStuQDBpmytyx2YMuP3XQy7yRE0NfN8Q9pYGCFEjVgqaKrnnjJuKtTg/NXF2NwQK6LAFBEBF0I0yjTmzc/ab5BqaGNurZ1PC66dwyKDptYYN339Ys4fbwv3+xbOkUdWRCHIaWDE0KrP2J+LsEbIRct/CV37esLW3N7ub1JK304pfXY8pmNKKe12afd07O7u8LrvsTWG/Jy3uaQ5p+W5HbiOFVEAoFXfTin918f/Dh0DIJhCFABo1WcppX//+N+hYwAEszUXAGjKhx9+N33++XsppfRm6+3uYdPq/fGYPkgpfZpSSofDGq0jh6XCnYA4VkShbEIc2lLz9RzT9rXPr4Y25tba+WTxWIR2OS9cBOhMU1J/RYY7lWIL43IL58gjK6JQsK6/6k4Nn7gUuFFSmEXrov5KPzYMK7caViFqaCPDTsOGngKIrn/8w3iY+zu3wv2zLv1Pa6yIAgA1mRs2JMAIoAAKUQCgJnPDhgQYARTA1lwAVjcQPFIrgSkLedw6++nT/38STDTWYIBRbkJ12pD7YywFfSzGOGQ1VkQBKEFLRWhK7Z1PsSYWoV2Wvla5Q3VKCgyifuYqVmNFFACoxpRgobu7t9/Pcnu7n/Q7Sw0wsnoFtMKKKABQkyWChQQYAQRTiAIANVkiWEiAEUAwW3MBeggZ6ddguBCVOA8ryiQ0wAgAK6IAQ3KHjLREHwwTHFOYFy++mPLjxjdbYa5iNVZEAWjafr/frd2GFo0N+Ik4NsYnn/wspZTS7e3+5unx6WQl9NrzY7vMLTCPFVEA4BpjA34ijs1t99ifE2AEkIlCFAC4xtiAn4hjc9s99ucEGAFkYmsuBDkcDqO2cY39udzPO+Oxmw/uaYkQomWU0K+555a7u7f/91e/+t30+efvvfPvu45NixHHLjkNO7rweAFGAAuyIgrMpWhpS2vXs5Qgjtb69R3nRWgJRoYTTRkfTV9DgGhWRAEomkAQLjke0+40SOju7jA2hOiD82NJgBFACCuiAEALIkKIBBgBZKIQBQBaEBFCJMAIIBNbc4HZcoegzCQ8CRb04YfPg4nWdjgcjqfhSUNOw4q6jpUeYJQr+Cpg3jYXjzRwTS/24ZzHwtqsiAKtESjSH8AyJpillHCfJ6W151qtnEdxRejIUKIpSg8wqmWOq6WdJejrqzF9OOexsCorohBkycCVob9sX3resY8tbNWTAXP+Cu4v6Mt46texYTd9ATiXfnYopOf2dn9z7XOPDfO5uztM7JkyLBlgNHalFmBrrIgCQJyxYTd9ATgRQTtbDOQRYAQQTCEKAHHGht30BeBEBO1sMZBHgBFAsM1vzfUhbwCinIXiPHv96QrK6QvP6Tp+e7sfevrzoJ2Lv+9CcE8zlgwwOmTYrVzrRyM62u29FVyh1XrFiqgPecNYtYSt1NJOaOp1ZoHQoJKUHmBUC30D12myXtn8iigwTs1/cYM1jAj9qd2b8KOhkKQlg9r65FhBzBVgJKwIoJsVUQBYRushNi2dS5fcAUYAnFCIAsAyWg+xaelcuuQOMALghK25ALCAS8FEtRvajntqxjbZVUM4cgUY3d7u04sXX6RPPvnZAq2sT63BS0B+VkSBvhAOoT+Qz8witKz37kHBRKUX7qPnyM8/f2/JdgBUyYoobJwQonwuhdMMHaM9Y4OJ7u4yfL8HIeYEGM2xRuDTHFY9gTGsiALkMzbcRJDJNrjO7XE/A2SiEAXIZ2y4iSCTbXCd2+N+BsjE1lxYweFwCAsuGblFatVQkFZcCqd5DDe5f9zS9+n542nD0/19+v2Rt7f71dpDPjMDjAA4YUUU8hob/FNaCEdp7WlBX5/q6/a5xnkITKuXawdcZEUUMrKqWJdLQUKnK1pTHzv1ecf8zqWOsa7b2/1NGnmd1hojl++V/q9yqS1oZ6qx9/2lxw7NNzV4d4zsnwU55Z6DBCJB/ayIAls2J3gk92On/M7cx1jXlOu01hgxlvrN6YeW+tC4ASZRiAJbNid4JPdjp/zO3MdY15TrtNYYMZb6zemHlvrQuAEmsTUXGCUyYGkBnWFMl8KFLgTMvNmKeEVAycXHdh3PcExQUoEuBeBc87O5jnUFLx0evvb0/nH75acnxzZpbFjR+Xzy4sUXZ329WBNDCGsDprIiCusoLchhTHtqLUJTGtf2ms9vrC2cI3kJ3Zpm9Nz++efvLdmOtRk3wEVWRGEBlwMa+oMcrjEU2tB6UMhYc8KF5ri7e7vMUcJXeAgwKs+UaxIfQjSu3bUH7cxx1ofP5vZ0sgNi6LE19WHusDZzEGyTFVFYhtCG8uj/B/qhPDWEFY1t9xZtsQ/XCnoDGqIQhWUIbSiP/n+gH8pTQ1jR2HZv0Rb7cK2gN6AhtubCAoQ2jPPhh9/t/JzUixdfpE8++VnW57p0TTbkPCgpfBxWFHzVGXKVW9lhRePaXXvQzqmB8Xkx9Kzr2IUwszf34+3tfpG5bwkz59PV5yCgDFZEYXmthDZkD1jqC+tYIMTjvO0hff/ixReD//9bq34kao1xWMvYr6Wd5JV7zp4bYFRauN25ufdJrfdZ6dcFimdFFGYYEygy9fGlhjZMXRka2Te9IR63t/ub88fP6Zux1+Q0XCi3NVc6hoKSahqHLRJW1J45AUa5574lXDOf1j4HCf6D/KyIwjxTAkXGPr6V0IbS+qaVfl1Cy+OwBsKK2tN6mE/u9tRwzkBmClGYZ0qgyNjHtxLaUFrftNKvS2h5HNZAWFF7Wg/zyd2eGs4ZyEwhCjMcj+l4PKZPz7cN9R0f8/ixjy1daX3TSr8uoeVxWIMp/T/2Z3MfG9tuHuTu19L6Ond7ajhnID+FKGS026VXu106nv7vws8fz/73Kqqt0erom3bf7/QHJa3S17WEfGRsZ7tjq1B9166UsdfbjnZeF94d84XNQX1KHzfQFGFFkFdv+t/I0IZa0wPHqKBvdtkCKQ6HQ2/lsUboxb6jWwf+GLBoX0d8JUp55JxEKn2MPQUYnVrrflzOu2P+KaztdP4r7ZxLHzfQGiuiMMNul3a7Xfq3j+l+i/3OJZ7nWmPbt2bfzGlPxHNEiegH5pvS17nHZ+4xQr+15qUpzzPnXHI/1hwE7VOIwjxLpPqVnh6YO3kz6rlzJ1aWfp1Saj+5sxVSc7dhrXlpyvOMFZFgbg6CxilEYZ4lUv1KTw/MnbwZ9dy5EytLv04ptZ/c2Qqpuduw1rw05XnGikgwNwdB43xGFGZ4TPP7NPOvffNF57uHzUf3j58nyv08Vzk958dAiXc+y7Pr2DDVdexK531z8XmmPnfXNb3m2OGQVnepjRf6puhx2JK7u8Obvn4aN3d3b//9dCx1HV/y2Nh2D+n4vPT9Fj+Ll/l+fMfAsfvjMX2Q+7VqQrvnPHZwDlpijj0cDs9e04DlWBGFFQylB3Yo+UVxdtvO+2Ji3+S2tWTEKedb8jikLsZStyXmn9L7es4ctETCben9BU2xIgozPAYmfDul9NmU7zp7Sg+8vd3fPD0+nfz1d8zznB4bu4Ix1aXnnfr7jse0e7fdz1dUpvbNTG+eo+v85hxb6prMddbuD86PpRnjcCvf91frtac8c+7HGc+Tda6b2ZYZc9D+2WO3PC9BjayIwjxzgxNKD21Y4nlLCkJZK+RlTQKM5tMP5BI1lkqa69YMbQIKohCFeeYGJ5Qe2rDE85YUhLJWyMuaBBjNpx/Ipdagt9xhRVGhTUBBbM2FGeYGQJQc2tDRvtkhDofD4XhNEMrt7X7O0/bKFUzUdWzuNRkIzZgV9BIdYNRi+MfpGP7qV7+bPv/8vXf+/fZ2n168+OLNNnPKNXV8dgQvzXIpLCrH3Lfbvbs1dejevSaMbqqA0KamgtXmjLnc4zVtNGiM5VgRhXKUHhwz6znnhBAtEWC0cijSGH39vfS1zz0OmypCz50XoZeOk1IqKxSs6PG54Dy1+L07o+1zx8ecdpc0NktU9P1CfayIwkhLBCLkCm2ICCsa+rm7u2W/q2SplaXdbr9YgEdNgTVLBhjV1A9rOQ3mujS3LBk60339+r+iZb/f5/tiJjrNnfuGVlRzhxClk/C3sV/t09OW2aFN175er7Xat8DKJVTBiiiMFxXcU3qYT0tKCvBYkwCjdU3pw4gx6/ptQ0QIUUltMa6hMApRGC8quKf0MJ+WlBTgsSYBRuua0ocRY9b124aIEKKS2mJcQ2F2x+O2dwMMbYew7Yg+1wT3dG1fvTTGzkMmhvSFo+Qax0NtWXprbmvGXJOS5qYp4zC9DQpJKbW15ezDD58HE+VwPKYx46HW0Keiw01aGp9dlgp667Lk60CG83hnXipNxDjsm7+mBqt5b7yOkt4T5GRFlM3bfbzb7T7efWf38aQ8wElvCCNCG3reIAteIIc5AUbNjMGFAojG9k+NRWhK5be7mfHZJSqUbennyfD7Nz8OMwWrNX2/EE9YEZtyHlaw+3i3S8f0k5TS36SU/nb38e6j44+OxzkhDsdj2o0N+xjRxkmhDWNDT645NtTma8NWcgehtL66ESlfgNH+2WOnjsO5j894D/Se8+lq0IXVm6vvUfLrWq2dOo+UtBpxPm72++fH08TgnxJ2vIxdsZsaztR3PP7Y9fPk0Ovo6WtzGrju+/1+Zx5iDVZE2Zo3YQWPK6A/Sceb76dd2qXjzfdTSj95PD436GCtkIW1Qkvmhq1QnpLG4RK/c63gHoFBLMm826+GuSXitXncz+523/hh+vivvpX+x3/7Yfr4r9Ju940RvwMmsSLK1jyEFXzn5WcppZ+klL6Xbl5/NaWUHv/7vZRSSt95+VH6+cs5QQdrhSysFVoyN2yF8pQ0Dpf4nWsF9wgMYknm3X41zC0Rr82Xf3a3+5OU0i9fppdf+mH68c1Nev1fUkq/T7vdn6fj8Z9G/C4YRSHKpjxux30oQn/3tR+kr/z2/Ef+IP3uaz9I7//vH6R0TCnt0qRPjp48T0rp05RSOkzc1XT62K5jF9rzZutN18/NOTbkUpv7jk/tG+KUPA5z/84c98BYY++VvvsHhoyZd5ca2xUYnJf6jpd+7PZ2/yxw6CSYaNQ27NNAuj9O/5T+OX09fZD+Jd2klL70+Ctep5Tu0we/+vpu9y3FKLnYmsumvNmOm9L3OorQB1/5bUrf+mlKf/kfU5r4kYigYIjSwgJKak9JbWmdvn50ft8PzANz+qzW/q613S0bfU2iwo5yqa29uZwHDl0brPaH6TfpH9KfvSlCT92klN5Pr1JK6Ze26ZKLFVE2400w0fHm+2+24/Z5KkZTSum//6eUer5doTvEYX8WJDCn1Q/mBMcsYFbYSu6+OVXS10S0GJxU2Dhc05t7oCso5GllImd42NPYjg5RmRq2VlJwD29dunf7x8N7abfbZx03uYPoLt1naTvz0ijn71tOA56+mX6dbtLr3lWqx+NfSil9M6X0m0UayKZYEWVL/iKl9DcXi9AnX/ltSn/6n1P617+45rnWCjiJsEToDHUoaRyuaYuhTdSt5XFTUluACRSibMkvUkp/m17f/L9RP/27r6X0j/8hpf/1F9c811oBJxGWCJ2hDiWNwzVtMbSJurU8bkpqCzCBrblsxvFHx+Pu491Hafc6pYd03D/o/eHffS2lX31vcFvu4HNlDuSZGRyT1dywlVx9czgcXqXyv6S8U8c2s/uc24qX6pvTrdRP1+702IXvz2zG2DE8514Zc/9EHFsrUGxgDGe9V7ai5XFT0uvjmnLMv79O30yv0016nbpXqh6P/z6l9OvZTwbJiigbc/zR8ZhS+iil9NP0u691/9DIIrSwUISoQJCSgkeqLEJ75D6XVfqmsHtiKSXdAy3rG8MRY9s1bovr+ahrjj499s/pG+nP0j+kV+nrzz5Y+5iam1JKf56OR58PJQsromzKQ4DB8dvpOy8/St/5OKXnK6P/N33ltz9N9//qo5R2V4V1vH2eZQJ5up5jWvjEesei+oZ1nH59wLk5wT3nx4fuyZwBQWOODY3hpe+ftc+vVV0rrhsIH2tq3Ix9few7PieMKXoOSsNhTIPBaik9zNv7/X739Dv/Z/qTz/4w/csfp5R+eUzpS6/Tzc1Nev36JqXffz298j2iZGVFlK15CDD4+ctvp6eV0afPjD7896cppY8e/31O0EFEUMJa4RPCirhG1LgpKZQl6v5pJXSGWC2Pm6h7as5zrzUHXf7Zh2Lzj16ml3/9p+kf/8/L9PKvU0p/pAglN4UoW/MmwODNNt3d679Lx3RMu9d/l1L66PH43KCDiKCEtcInhBVxjahxU1Ioi7AiStbyuIm6p+Y891pz0LifPR5/8+P0o7//Vfo3/+7H6Ud/bzsuS9gdj83tNplkaLuN70Pbht3Hu116+GqXXzwWoe9ocUtWDS7df61dl/PznTM3ldg3OefTkubtNdtSSWDXVeFCU/q1lX6ICmiqpL/mWDzQqqQ5aLdLvW05Ht8GXZTUZqZr9fr5jCib91h8/nztdjDZfWrnzVTuMI3S+kZYyDJKusZ9ItrYSj9EBTTV0F9ztH5+5/rme/MuxVOIAlXK/Rfvlv7a6Ost8ik5rGjtQBhgHdeEFZovKJHPiAJAv5JCS6Y8N9CuGgOk4BmFKAD0Kym0ZMpzA+2qMUAKnrE1FyhSx1bZxQMo4NzjFrdP+44dDtc/duqxKc9NuzYQNlSUEvv7mnnEfEGJrIgCtSjqjQAUoIYwkog21tAPOZkLY+lvWIgVUcgs59dwtK7ErxmBXJYOKzoe96NCSuYcu7s7vJ5yzmvMaV07JXLPu2N/31bmtDlfIZW7/4F6WREFgGVEhRVFHAOArBSiALCMqLCiiGMAkJWtuXCixFACoE7vBoU8zC2n3+X3FB4y9tiUn811DACWYkUU3qUIBZZgbiGXtcOZxjx/38+s3XagIFZEAWAB74b+rN0aWlHD11jV0EZgfVZEAWAZQn8AoIdCFACWIfQHAHrYmgsr8t1o06zVX5mf936NbWsDQVyrtGesqQFiJd1TU7+DswUR/d/xHEWPYeKVPEaEIsJbVkQBYq31BqTveUt/Q1R6+1ifMcIlJY2RktoCq7IiCgAr2u/3u2seN3X18ZrnKWmFGYC2WBEFAAAglEIUAACAULbmApBFrYFIJdGH15myhdh2422Kuu4lByVBaayIAsS6X7sBC6o1EGlN5+NBH0Jbzu/dll8DYBIrorCia0NKWjb0V+sl+2ut5yWv6Gtl3Lwr1zlbtaRVa62OuqcokRVRAAAAQilEAQAACGVrLmRWQWiG4ASaVdL2s9xtmfv7Suqblozt1zn9v8C18zoArM6KKGyP0JM69AVaCLoA5vI6EMecDT2siAIUyGoFdBsTiDQnRMrKcZuuDdKaOh62GFIG17IiCgAAQCiFKAAAAKFszYUNyrz1TOjFTIfD4VXq/syWvgUgpTT4WjH39855T+B1iqtZEQXmEnoxX18f6luIt4VwmdrOsbb2LqXE14QS20QlrIhCZudBBYIvIM6SQSE57uU5QTlz55Zr+maL81fX6s7Y67JUSNLWA3CmXhOgDlZEAQAACKUQBQAAIJStuQAAXLRUWE4EW3mhPFZEgbmESAApmQu2oMoiNNDS90CJ91iJbaISVkRhpLGBFNf+TqBeS8wPpTN/sXXR94CvSaE1VkQBAAAIpRAFAAAglEIUAACAUD4jCvDgPnUHcRQdxDCQYnm/wc8TFX2tztWcQMoiapiD+toIMJlCFCApCEN+AAAJ/ElEQVRVHQLR96aw+TeLDYTlNH+NGK+GOaikNk4NBbt2vthK+BiswdZcAAAAQilEAQAACKUQBQAAIJTPiEJlKg442WJ4DpDZ3M/s+cwfuXSMJa9zMIEV0f40upJS6ijPmuOmxiI0pfHtdk8uQ78CLKvW12fK1+Rr+OZXRP3limsYN8vRt8vQrwBQp1Zfw62IAgAAEEohCgAAQKjNb82FnKYGCW0tNGPF8918gMTA2Nx83/AgIgit4rC1XmPntczzn/sWqJ4VUXjX3A+DN/UGqyGuS38flN43TQY0FCpiLJQ+3mqhH+OYa2AhVkThhL8wX2e/3+8u/czWVn/Jwz0JrKlrDvJ6BnlYEQUAACCUQhQAAIBQtuYCAFCEpQKtbKeF8lgRhby2GGow9pzX7JstXhcokXsxj5b7URATbIQVUchIsEo/fQPUOg8MraaNCWsD4DkrogAAAIRSiAIAABDK1lxglIEAiftat9sB9WhpDmrpXHhXxzZu1xR6WBEFxuoLkBAswZO+AJWWg1W66IdltDQHtXQuubV2n7im0MOKKABZ+Kv/A/0A15tz//iKFqiLFVEAAABCKUQBAAAIZWsuANUT/lKegWuyxHPl3pJp3AAszIooAC3YYvhL6aEuNfd9zW3fshLviRLbBEWwIgoAFepasRPWwpZNWcUeulf2+/1uqccCb1kRBQAAIJRCFAAAgFC25gKzdWxTEvQBAEAvK6LAEgR9xOkLwhCQUb7Wr1HN51dz2wGqYEUUoGJWnuvVethQ7rEpIAagLVZEAQAACKUQBQAAIJStuQATHA6HV6n7M7ACmoCiDMxXEbLNieZdaJMVUYBp+t7UCWhiS7YW5lNrKNia81LO5zbvQoOsiAIAg7YeBmTVDSA/K6IAAACEUogCAAAQytZcgIJFfK9kx3MUEwAipGSbpobsVPD9q6PHawXnMlrJc0uElcOiomzqmpKXFVEAzpX0xklIyTa1dn1bO59rba0ftnC+WzhHFmJFFACA0S6FV7W0qgssx4ooAAAAoRSiAAAAhLI1FyCTrQdzwClBU3TJvW13iW3AthZDDCuiAMsZG+Jwv2grpiutPYzXd+3GXNM5j+0yJ2iqtTHY0vm0dC65nffNFvpqC+fIQqyIAqzM6hC5zBlLJY3DrrYMrVJdCs+pyZwgoJb6oQaX+ruke8q4oURWRAEAAAilEAUAACCUrbkArG4g2AagSDNCjQR2QbIiCkAZaihCc4f5ANtUw3wHi7MiCgAjWMGA+a4JxvF1KtAmK6IAAACEUogCAAAQytZcAAYJEqpHLdfKVksArIgCcEnxhQ1vuFZ1EHy1ba4zJCuiAAChBF/lc034UUrDq/LX/k5gGiuiAAAAhFKIAgAAEMrWXFjYQHjI/TXbszYSRnJV3wBAKSp/vfY6zOKsiMLy+l6Ern1xKv5FLYMtnGNNagjWEP7yYGvnCyWr+bWs5rZTCSuiAAyK+Kv43K/z8Jf7ByX1Q+4wGF/5AtAWK6IAAACEUogCAAAQytZcAIAG2L4M1MSKKNRnC2EkJZ9jyW0DYJyIgLOaXy9qbjuVsCIKlVkrjGTqX9qvCSOpQVf/W4UAqEvEa2nL4WGQgxVRAAAAQilEAQAACKUQBQAAIJTPiAKQxeFweJVSej/w+a79bO59CZ/dmtpfPouc7lN3fwlVAaiQQhSAXMKK0JlKaWcp7ahCCX88KF0poTP+aAKMYWsuAAAAoRSiAAAAhFKIAgAAEMpnROGCJQNYVvocTRFBLbAm9x5TDLwOuKYAV7IiCpe1FijS2vlQDumlw87vPf1Vj755c435tG/cGE9AVayIApDFnJWha1YoLyWElp7c2dVfQ20uJRF1rNL7v1ZWYIFWWBEFAAAglEIUAACAULbmAjRmyYAtAIAcrIjCZa0FQLR2PjynCH1Q2lgvrT3Adgi5ojhWROGCucEQLYWPQE2EugA8MB9SIiuiAAAAhFKIAgAAEMrWXMhoakjMSt+zd2+LDlCbUkO4zOMA17EiCnkV9yapQ0QbhR+sawv93+o5ChTpV8P8GmVrfeG+gAZZEQVmE7pUlhpXSoR6Pajx2sHS3BfQJiuiAAAAhFKIAgAAEMrWXABYWKlBOx2E4ECDzEGUyIoo5FVDcMK1bRQW0U/fzNd6H9bwBjClstvZyljIofS+aP1+rlHJ9/apWtpJBrvjcY3UcdgOISzASl/xcZVc81JLc19L58I2bXEOonxWRAEAAAilEAUAACCUsCIAgIVUFBIzh4AZYDIrorA8oQ1ALfd7Le2sSetFaErbOMfa1XJv19JOMrAiCgvzV2LAPACsyRxEiayIAgAAEEohCgAAQCiFKAAAAKEUogAAy9lC+MoWzhHITFgRAMBChMQAdLMiCgAAQCiFKAAAAKEUogAAAIRSiAIAS+gLsKkx2KalcwEowu54PK7dBgAAADbEiigAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQCiFKAAAAKEUogAAAIRSiAIAABBKIQoAAEAohSgAAAChFKIAAACEUogCAAAQSiEKAABAKIUoAAAAoRSiAAAAhFKIAgAAEEohCgAAQKj/D3dlYGiWPh2hAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " Greedy best-first search search: 153.0 path cost, 502 states reached\n" + ] + } + ], + "source": [ + "plots(d4)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# The cost of weighted A* search\n", + "\n", + "Now I want to try a much simpler grid problem, `d6`, with only a few obstacles. We see that A* finds the optimal path, skirting below the obstacles. Weighterd A* with a weight of 1.4 finds the same optimal path while exploring only 1/3 the number of states. But weighted A* with weight 2 takes the slightly longer path above the obstacles, because that path allowed it to stay closer to the goal in straight-line distance, which it over-weights. And greedy best-first search has a bad showing, not deviating from its path towards the goal until it is almost inside the cup made by the obstacles." + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3cGOZOd53+GvRorgKOA4WgXIKs42EpraZJVAvoFA8IZeCPLOia5CHN5CVoqDLBxACw4gGEj2gQRlH8iOgWwM30EscyDGUOA5WbDFUD3dzaruqnN+56vnAV4QeDmc89ZXVcP+d59657AsywAAAICKF1sPAAAAAF8kqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAIAAJAiqAKbORzG4XAY7x8O43BqDwCAeQmqwJZuxhg/uf3nqT0AACZ1WJZl6xmAK3X7E9KbMcYvlmUsp/QAAJiXoAoAAECKW38BAABIEVSBs3vOkqQ99mrzOIfHZwQA+gRV4BKesyRpj73aPM7h8RkBgDifUQXO7jlLkvbYq83jHCzhAoC9E1QBAABIcesvAAAAKYIqcLTSUpxSrzaPc+j1avOs9ZgB4KkEVeAUpaU4pV5tHufQ69XmWesxA8DTLMuilFJH1RjLYYzl/TGWg97/79XmcQ69Xm2etR6zUkop9dSyTAkAAIAUt/4CAACQIqjClbMM5vm92jzOoderzVPq1ea5xOMD4Am2vvdYKbVt3X6u7K/GWN5/rHfKr722Xm0e59Dr1eYp9WrzXOLxKaWUOr02H0AptW0Ny2Ce3avN4xx6vdo8pV5tnks8PqWUUqeXZUoAAACk+IwqAAAAKYIqTMrCk/V6tXmcQ69Xm6fUq81ziccHwBNsfe+xUuoyNSw8Wa1Xm8c59Hq1eUq92jyXeHxKKaVOr80HUEpdpoaFJ6v1avM4h16vNk+pV5vnEo9PKaXU6WWZEgAwrdevX38yxnhv6zl25s0HH3zwcushgOvmM6oAwMyE1NM5M2BzgirsjIUnvV5tHufQ69XmKfXWvA4A+yGowv7cjDF+cvvPNXtbXrveq83jHHq92jyl3prXAWAnfEYVdub2JwQ3Y4xfLMtY1uptee16rzaPc+j1avOUepe+zscfv347ONkHH3zgp9HApgRVAGBar1+/9oXOEwiqwNbc+gsAzOzN1gMA8ARb//04SqmHq/Z3AZbmKfVq8ziHXq82T6lXm6fUu/R1Pv744+Wh+uIMSim1RfmJKrRd48KTPfZq8ziHXq82T6lXm6fUW/M6AC1bJ2Wl1MN1Td/Z33OvNo9z6PVq85R6tXlKvUtfx09UlVLlOizLcnK4BQBg3x5bNGWZErA1t/4CAACQIqjCBg6HcTgcxvu3f9ffLnq1eUq92jzOoderzVPq1eYp9da8DkCNoArbKC3rqC312GOvNo9z6PVq85R6tXlKvTWvA9Cy9YdklbrGKi3rqCz12HOvNo9z6PVq85R6tXlKvUtfxzIlpVS5LFMCALhClikBZW79BQAAIEVQhTMqLeHY61KPPfZq8ziHXq82T6lXm6fUW/M6ADWCKpxXaQnHXpd67LFXm8c59Hq1eUq92jyl3prXAWjZ+kOySs1UpSUce1vqsedebR7n0OvV5in1avOUepe+jmVKSqlyWaYEAHCFLFMCytz6CwAAQIqgCkcoLdeYfanHHnu1eZxDr1ebp9SrzVPqrXkdgBpBFY5TWq4x+1KPPfZq8ziHXq82T6lXm6fUW/M6AC1bf0hWqT1UabnGrEs99tyrzeMcer3aPKVebZ5S79LXsUxJKVUuy5SO8Pr160/GGO/d86/efPDBBy/XngcA4LksU4I5zJpV3Pp7nPue+Mf6AAAAa5gyqwiqcEdpkUapV5un1KvN4xx6vdo8pV5tnlJvzesA1Aiq8K7SIo1SrzZPqVebxzn0erV5Sr3aPKXemtcBSPEZ1SP4DMd1uf0u880Y4xfLMha98fnrvzRPqVebxzn0erV5Sr3aPKXepa/z8cev344H+PoG9mPWrCKoHmHWJx8AuF6+voE5zPpedusvAAAAKYIqV2vLxRV77NXmKfVq8ziHXq82T6lXm6fUW/M6ADWCKtfMUo/TerV5Sr3aPM6h16vNU+rV5in11rwOQIrPqB5h1vu+r93td5NvhqUeiaUee+7V5nEOvV5tnlKvNk+pd+nrWKYEc5g1qwiqR5j1yQcArpevb2AOs76X3foLAABAiqDKVagtrthjrzZPqVebxzn0erV5Sr3aPKXemtcBqBFUuRa1xRV77NXmKfVq8ziHXq82T6lXm6fUW/M6ACk+o3qEWe/7via33zm+GZZ6ZJd67LlXm8c59Hq1eUq92jyl3qWvY5kSzGHWrCKoHmHWJx8AuF6+voE5zPpedusvAAAAKYIqAAAAKYIqV6G2YXGPvdo8pV5tHufQ69XmKfVq85R6a14HoEZQ5VrUNizusVebp9SrzeMcer3aPKVebZ5Sb83rAKRYpnSEWT+gfE1uv3N8M2yfzG6f3HOvNo9z6PVq85R6tXlKvUtfx9ZfmMOsWUVQPcKsTz4AcL18fQNzmPW97NZfAAAAUgRVprOHxRV77NXmKfVq8ziHXq82T6lXm6fUW/M6ADWCKjPaw+KKPfZq85R6tXmcQ69Xm6fUq81T6q15HYAUn1E9wqz3fc/q9rvENyO4uGLPvdo8pV5tHufQ69XmKfVq85R6l76OZUowh1mziqB6hFmffADgevn6BuYw63vZrb8AAACkCKpMZw+LK/bYq81T6tXmcQ69Xm2eUq82T6m35nUAagRVZrSHxRV77NXmKfVq8ziHXq82T6lXm6fUW/M6ACk+o3qEWe/7ntXtd4lvRnBxxZ57tXlKvdo8zqHXq81T6tXmKfUufR3LlGAOs2YVQfUIsz75AMD18vUNzGHW97JbfwEAAEgRVNm1vS6u2GOvNk+pV5vHOfR6tXlKvdo8pd6a1wGoEVTZu70urthjrzZPqVebxzn0erV5Sr3aPKXemtcBSPEZ1SPMet/3DG6/I3wzdrK4Ys+92jylXm0e59Dr1eYp9WrzlHqXvo5lSjCHWbOKoHqEWZ98AOB6+foG5jDre9mtvwAAAKQIquzaXhdX7LFXm6fUq83jHHq92jylXm2eUm/N6wDUCKrs3V4XV+yxV5un1KvN4xx6vdo8pV5tnlJvzesApPiM6hFmve97BrffEb4ZO1lcsedebZ5SrzaPc+j1avOUerV5Sr1LX8cyJZjDrFlFUD3CrE8+AHC9fH0Dc5j1vfzVrQeAPTh8dDiMMb4zxvjZ8qHv7gAAwCX5jCq7tsbiisNHh8NYxo/GMv7bWMaPbkNrauHGGr3aPKVebR7n0OvV5in1avOUemteZxal52+m18g1vpbYnqDK3l10ccVtKP3RWF58fxzGYSwvvj/G52G1tHBjrWUbpXlKvdo8zqHXq81T6tXmKfXWvM4sSs/fTK+R3+4dDt/44fjou98af/5ffjg++u44HL4x4Mx8RvUIs973PYPb7+LdjEssrvj9V78Yv//Rj8YY3xtj/KMvXPZXY4wfj59++IPx01dnu3a9V5un1KvN4xx6vdo8pV5tnlLv0teZcZlS6fmb4TVyb28c/sUY4+fLGF95O168eDHevj2M8fdjjH89luV/Pu2Z4zlmzSqC6hFmffJ52Oc/Sf311//t+Nqn7/6CX399jL/43hj/9T+M4Y4XYH1vlmW83HoI9s3XN5zscPjm346Xf/He+OS3bst8O8Z4M16O3x2ffEtYXd+s72W3/sIdn4fUMb53b0gdY4yvfTrGt348xr/5d2MM3+wBVvfe1gMAV+az23t/fjekjvFZoHhvfDLGGD93GzDnIqiya2dfDPCbxUlvX3x//Pbtvu8SVoENnf3Pv416tXlKvTWvs0el52r218jhMA4fjlffWcb4ykPh4bb/lTHGP3ngl8BJBFX27tzLAr4zxvjj8eLtPzzq6l/7dIxv/6cx/tnPTp8c4HnWX6BymV5tnlJvzevsUem5mv01cvNn4w/+/dvxQnZgNT6jeoRZ7/ueweHcywJ+8xPV5cX3jwqrPqsKbOfFiCx0eU6vNk+pd+nr7H2ZUum5mvU18sXeD8dH3301Xv3nw+MfPXgzxviXY1n+12PPHec1a1YRVI8w65PP/X7rM6qP3f4rpAItFixxEl/fXK/DYXwyTvys+z8efzP+evzeeDn+9t5bMt+OMV6M8csxxj8fy/I355iT48z6Xvbje7hj+XBZxhg/GGP8ePz66/f/IiEV6LFgCTjWyX9e/HJ8Y/yr8d/HJ+N3x90fxf9m6+/47K+oEVI5C0GVXbvUUoLxarkZP/3wB+Nrn/7J+OzvTf2iX42vffon480/fTHG4dtjjBfLMg7LMg7js/fUlL3aPKVebR7n0Oud8/ccjygteSkug9lbb83rVHiNnNY7x39/ir8c3/w8rC5jvPn78eJXyxhvXozxS381DecmqLJ3l1tK8NNXN+M3P1l9++L/jDHG7T9/PMb4we2/ryxTmGphww57tXmcQ693qd/zrtJj9rp5fm/N61R4jZzWO8d/f5K/HN8cvzf+erwar/7o2+N//O9X49Ufjc9u9xVSOSufUT3CrPd9z+CwwlKCzxcsjfHHY4z/OA7jB8uHy3Lua9d7tXlKvdo8zqHXO+fvOcY7d9190e4WLNXmKfUufZ3iMiWvkXVeI+PxP0eO9c6fN2xj1qwiqB5h1ief490uWPrOGONnt59hBdjE4XDSF4QWLPEgX9/M5/CEJUlPdfuxBAJmfS9/desBYA9uw+lPt54DYHz21z8c+4WoBUtwXdZ6z79Z6TpcMZ9RZdees0DgoaUC5/49Z+nV5in1avM4h17vnL/nsoyXMy1Yqs1T6q15nS3UzmGPvcf6R3rqkreXpdcScxJU2bvaUoKZe7V5Sr3aPM6h11vzOneVzsHr5rTemtfZQu0c9th7rH+MWV5LTMhnVI8w633fM7j9Lt7N2Mniij33avOUerV5nEOvd+nrjB0vWKrNU+pd+jpbL1OqnMOee3f74/QlSWf984FtzJpVBNUjzPrkAzCHgwVLPIGvb/bhcMEFSRYizWHW97JbfwFg/05ZbGLBEuzLpd6zFiKRJqiya89ZQPDQEoBz/56z9GrzlHq1eZxDr3fp6+x5wVJtnlJvzeuc017Pod470ZMXIm31uoG7BFX27hJLAM79e87Sq81T6tXmcQ693tbXvsvZ7KO35nXOaa/nUO+dojYPnMxnVI8w633fM7j9zt7N2Mniij33avOUerV5nEOvt8W1x04WLNWeq1Lv0te51DKlvZ1DvTdOX5A0xgrvcTpmzSqC6hFmffIBmNfBgiW+hK9vtnWwJIkzmfW97NZfAJiTBUvQZkkSPEJQZTrPXQzwnP9+5l5tnlKvNo9z6PW2uPZeFixtee16b83rPNVM57Dl83ykdxYkWZLEzARVZvTcxQDnXjYwS682T6lXm8c59HrFee5yNr3emtd5qpnOofR+vM9aZwMJPqN6hFnv+57V7XcFb4aFDWft1eYp9WrzOIderzLPCC5YqpxNsXfp65xjmdIM5xB9P97nnffopc6bfZk1qwiqR5j1yQfguhwsWOILfH3zfIcLLkS6y4IkHjLre9mtvwBwPSxYgvNa631iQRJXR1BlOs9dIPCc/37mXm2eUq82j3Po9SrzFBcsVc6m2FvzOndd4zmc+wxP8M6SpHt69y5I2nhuuChBlRk9d4HAuZcSzNKrzVPq1eZxDr1ebZ6HZrzL2WzbW/M6d13jOZz7DI+15dlAls+oHmHW+75ndfudwpthYcNZe7V5Sr3aPM6h16vN88Xe2HjBUuUcir1LX+exZUp/+IcfbLJca2+vkXH6QqT7PPmsz3G27N+sWUVQPcKsTz4AHCxYulq+vnnYwZIkdmTW97JbfwHgulmwBO+yJAk2JqhyFU5ZKvCcpQQz92rzlHq1eZxDr1eb54u9rRcsVc6h2FvzOndd4zkcezYPOGYh0n29e5ckXeJsYG8EVa7FWksJZu7V5in1avM4h16vNs8pc9/lbNbrrXmdu67xHI49m/vs4WxgV3xG9Qiz3vd9TW6/o3gzdrywYetebZ5SrzaPc+j1avN8WW+suGCp8piLvUtfxzKlJ78H7rPJeZ36a5nTrFlFUD3CrE8+ANzn8MiCJYtf5uHrm4c99h64j/cFW5r1vezWXwDgrgcXvBwOY7lTn6w5GDzH4TA+uec1/E6d+NtaiAQX8NWtBwAAWu77K2ge+eLdJmD25KTXq5+Uwnb8RJWrcN/2u/t6p/zaa+vV5in1avM4h16vNs9zH8tdzuZyZ73H53QP53Cs0jmccjYwA0GVa3GJ7XnX1qvNU+rV5nEOvV5tnuc+lruczWV6a17nrtnP4VilczjlbGD3LFM6wqwfUL4mt99lvBnBzYJ76dXmKfVq8ziHXq82z1N640KbgCuPr9i79HWucevv2Mk23+eeDddl1qwiqB5h1icfAI51OG3BzJv7PudKy0xf3xw+W+p19s9L+4wqezDTe/mL3PoLABzjlM2mFiyxtku85mzzhQ0JqlytU5YS6PXmKfVq8ziHXq82z1N6yzJe3v506cUY49vjS76GuKazuVRvzevctddzOMHnr+NlGYe7r+3b3svSOTzz8cLuCKpcs1OWEuj15in1avM4h16vNs8lHt9dzub5vTWvc9dez+FYpcd37ucOpuAzqkeY9b7va3f7HcmbYanHsxY26Dkb53C9ZzPOsGCp8liKvUtfZ6ZlSmOyJUmnnA3MmlUE1SPM+uQDwHMcLFjatT18fXOwJAm+1B7ey0/h1l8A4KksWOLSLEmCKyWowh2lJQmlXm2eUq82j3Po9WrznKt3jgVLlcdS7K15nbtq53CCKZYkneEcYPcEVXhXaUlCqVebp9SrzeMcer3aPGs95ruczWm9Na9zV+0cjlWae8tzgN3zGdUjzHrfN/e7/e7lzQgsSSj1avOUerV5nEOvV5vnkr1x4oKlytzF3qWvs4dlSofTPgc9xiRLkh56PcB9Zs0qguoRZn3yAeDcTgkWv/M7/3f86Z/+2SXH4YnO9fXN4ULLkB5iSRLXaNas4tZfAOCcjl5U83d/9w8uOQcNay7RsiQJJiKowhFKyxS26tXmKfVq8ziHXq82zyV7py5YoukS74GnemAh0n29XS5JOvd5wSz8zwOOU1qmsFWvNk+pV5vHOfR6tXm2PAf2ofbcl17H3iuwAp9RPcKs931zvNvvct6MwDKFrXq1eUq92jzOoderzbN2bzyyYOnjj18/9K/Y0LmWKY3Hl2sdZVnGofA6tjiJqlmziqB6hFmffABYw+H0za2MZQx3go4xLEiCLzNrVnHr73Ee+nC+D+0DwJfz/8uT7fZry3Pz2oEvN2dWWZZFKXWmGmM5jLG8P8ZymK1Xm6fUq83jHHq92jyV3hjLoq6mvH8eeXxKqXdr8wGUmqlu/+fzV2Ms78/Wq81T6tXmcQ69Xm2eSm+MZevwpNYr759HHp9S6t3afAClZqoR+g7tuXu1eUq92jzOoderzVPpjbFsHZ7UeuX988jjU0q9W7f/owAAWJclS9djsRAJOJFlSgDAVva96INjeZ6BkwmqsIHDYRwOh/H+7d+dtotebZ5SrzaPc+j1avNUessyXt7+pO3FGOPbY4wXn93tpbf1tc/ce+n9A5xs63uPlbrGGqElDsf2avOUerV5nEOvV5un1KvNU+rV5nEO53ksSqnjavMBlLrGGqElDsf2avOUerV5nEOvV5un1KvNU+rV5nEO53ksSqnjyjIlAAAAUnxGFQAAgBRBFcJqCyBK85R6tXmcQ69Xm6fUq81T6tXmucZzADa09b3HSqmHa8QWQJTmKfVq8ziHXq82T6lXm6fUq81zjeeglNquNh9AKfVwjdgCiNI8pV5tHufQ69XmKfVq85R6tXmu8RyUUtuVZUoAAACk+IwqAAAAKYIq7Mw1LrOo92rzOIderzZPqVebp9SrzVM7B2ByW997rJQ6rcYVLrOo92rzOIderzZPqVebp9SrzVM7B6XU3LX5AEqp02pc4TKLeq82j3Po9WrzlHq1eUq92jy1c1BKzV2WKQEAAJDiM6oAAACkCKowKUs91uvV5nEOvV5tnlKvNk+pV5wHYDVb33uslLpMDUs9VuvV5nEOvV5tnlKvNk+pV5xHKaXWqs0HUEpdpoalHqv1avM4h16vNk+pV5un1CvOo5RSa5VlSgAAAKT4jCoAAAApgipcub0u9Sj1avM4h16vNk+pV5vnEo8PgCfY+t5jpdS2NXa61KPUq83jHHq92jylXm2eSzw+pZRSp9fmAyiltq2x06UepV5tHufQ69XmKfVq81zi8SmllDq9LFMCAAAgxWdUAQAASBFUgaOVFpSUerV5nEOvV5tnrccMAE8lqAKnuBlj/OT2n3q/rTSPc+j1avOs9ZgB4Gm2/pCsUmo/VVpQUurV5nEOvV5tnrUes1JKKfXUskwJAACAFLf+AgAAkCKoAmdXWuhiUY5zsOAHAPZHUAUuobTQxaKc9Xq1eSz4AYCd8hlV4Oxuf4J1M8b4xbKMZfZebR7n8PiMAECfoAoAAECKW38BAABIEVSBzViKAwDAfQRVYEuW4gAA8A6fUQU2YykOAAD3EVQBAABIcesvAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAADYcZriAAABrElEQVQAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKYIqAAAAKf8PLmrpHsltQLwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " A* search search: 124.1 path cost, 3,305 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAGO9JREFUeJzt3T+PXNd9x+HfHSuCo0BrqTKQKnGTwhEoNq4cyG8gMAIDq0KQXTnRq7Cod6E4SOFChQkEAeI+kGBXaSIlDuAudZpIJiHFUGDeFLtkqOXu8s7u3HO/58zzNIKPaJ3D+UPNR3Pvb6d5ngsAAABS7LY+AAAAADxNqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABBFqAIAABDlha0PAACwlvv37z+oqpe3PkdnHp6enp5sfQjguPlGFQAYmUjdn8cM2JxQBQCGMk01TVO9Pk01bX0WAG5GqAIAo7lTVf9w/lcAOiRUAYDRfFJVPzj/KwAdEqoAwFDmueZ5ro/nueatzwLAzQhVAGBkD7c+AAD78+NpAIAunA9HulNVnzz+tvT5a2c/ZuVm/99t19be5+c/v//oNs8HwJp8owoA9OKyIUkjr7XcByDKNM9u3wAA8iV92zn6N6qnp6d+tA+wKaEKAHCE7t+/f+WHQKEKbM2lvwAAAEQRqgDApqappmmq188vS7XW+LEBSCRUAYCtJQ0wSlpruQ9AFPeoAgCbShpglLS29j6GKQHJhCoAwBEyTAlI5tJfAAAAoghVAODg0gYG9bjWch+ANEIVAFhD2sCgHtda7gMQxT2qAMDBpQwM6nlt7X0MUwKSCVUAgCNkmBKQzKW/AAAARBGqAMBivQ4M6nGt5T4AaYQqALCPXgcG9bjWch+AKO5RBQAW621gUM9ra+9jmBKQTKgucP/+/QdV9fIlf+vh6enpSevzAADclmFKMIZRW8Wlv8tc9sRftw4AANDCkK0iVAGA4QcG9bjWch+ANEIVAKgaf2BQj2st9wGI4h7VBdzDAcDoRh0Y1PPa2vsYpgRjGLVVfKMKANQ81zzP9fHTkdRibcu909da7gOQRqgCAAAQRagCwMCShgMZppT52AAkEqoAMLak4UCGKe231nIfgCiGKS0w6g3KAIwvaTiQYUpZj41hSjCGUVvFN6oAMLCk4UCGKWU+NgCJhCoAAABRhCoAAABRhCoADCJpYm3aZNse11ruA5BGqALAOJIm1qZNtu1xreU+AFFM/V1g1ElaAIwlaWJtymTbntfW3sfUXxjDqK0iVBcY9ckHAI6XzzcwhlHfyy79BQAAIIpQBYAOJQ396WFgUI9rLfcBSCNUAaBPSUN/ehgY1ONay30AorhHdYFRr/sGoF9JQ3+SBwb1vLb2PoYpwRhGbRWhusCoTz4AcLx8voExjPpedukvAAAAUYQqAARJGuYz0sCgHtda7gOQRqgCQJakYT4jDQzqca3lPgBR3KO6wKjXfQOQJ2mYzwgDg3peW3sfw5RgDKO2ilBdYNQnHwA4Xj7fwBhGfS+79BcAAIAoQhUANpI0uGf0gUE9rrXcByCNUAWA7SQN7hl9YFCPay33AYjiHtUFRr3uG4BtJQ3uGXVgUM9ra+9jmBKMYdRWEaoLjPrkAwDHy+cbGMOo72WX/gIAABBFqAJAA0lDepLW0s6TtNZyH4A0QhUA2kga0pO0lnaepLWW+wBEcY/qAqNe9w1AO0lDepLW0s6TtLb2PoYpwRhGbRWhusCoTz4AcLx8voExjPpedukvAAAAUYQqAByQgUH7raWdJ2mt5T4AaYQqAByWgUH7raWdJ2mt5T4AUdyjusCo130DcHgGBmUNDOp5be19DFOCMYzaKkJ1gVGffADgePl8A2MY9b38wtYHgB5M701TVb1RVR/N7/qvOwAAsCb3qMIFFwdNTO9NU831fs31zzXX++fRGjVwY6ShHj2upZ3H45C3lnaepLW08ySttdwHHvNaIoVQhWc9GTRxHqXv17x7u6aaat69XfUkVpMGbow01KPHtbTzeBzy1tLOk7SWdp6ktZb7wGPPf91M06s/qfe+/1r92z/9pN77fk3Tq+2Pyejco7rAqNd9c7nz/zJ4p75375P63nvvV9VbVfVHT/2Sz6vqg/rw3Xfqw3t3KmDgRou1tPMkraWdx+OQt5Z2nqS1tPMkra29j2FKXOa5r6Wavl1Vv5yrvvaodrtdPXo0Vf2+qv6i5vnX2538eI3aKkJ1gVGffK725JvUL1/663rxi2d/wZcvVf37W1W/+NsqV7wA7T2c5zrZ+hD0zecbnmea6kFVvfz4f3+7fl2/qu/WSf32K5dlPqqqh3VS36gHr4nV9kZ9L7v0Fy54EqlVb10aqVVVL35R9doHVX/5N1XlP/YAzb38/F8CcGtP/qx5pT69NFKrzoLi5XpQVfVLlwFzKEIVnvJkcNKj3dv11ct9nyVWgQ21GLLTYi3tPElrLfdhLGu8Rr5Z/1W7enRlPJyvf62qvnnI3wvHS6jCV71RVT+u3aM/XPSrX/yi6u7fV/3JR+ueCuBZSUN/ehgY1ONay30Yi9cI3XOP6gKjXvfNs/7/R9Hs3l4Uq+5VBbazq5ChP8kDg3peW3sfw5TGdajXSJ3dflpVVX9Wv6l/qe/UST28buuHVfWdmuffHP53xVVGbRWhusCoTz6X+8o9qtdd/itSgSwGLLEXn2+Ow3RhINJNvVKf1n/Wn156j2rVWdHuqj6rqm/VPH962/1YbtT3skt/4YL53Xmuqneq6oP68qXLf5FIBfIYsARc5iB/NnxWr9Z361f1oL5RF7+Kfzz1t85+RI1I5SCEKlwwTTXVvflOffjuO/XiFz+ts5+b+rTP68UvfloP/3hXNd2tqt081zTPNdXZe2rItbTzJK2lncfjkLd2yH9mXSNpEFDawKAe11ruw3ZavUZuaVdVd/+j/nz3Sv32tV3VZ3PVw9/X7vO56uGu6jM/moZDE6rwrLMhAh/eu1OPv1l9tPufqqrzv35QVe+c//2UgRsjDfXocS3tPB6HvLW1/pkXJf2evW5uv9ZyH7bT6jVymDOexei37tW9H96tf/3ve3Xvh3V2ua9I5aDco7rAqNd9c7mLgwWeDFiq+nFV/V1N9c787jwnDdwYYahHz2tp5/E45K0d8p9Z9cxVd0/rbsBS2nmS1tbexzClDGu/Rur6PzOWWvRnC9sYtVWE6gKjPvksdz5g6Y2q+uj8HlaATUzTXh8IDVjiSj7f5JkONPjo0M5vQSDUqO/lF7Y+APTgPE4/3PocAHX24x+WfpCN+8ALXCvxPXvtz6OBtbhHFQAOaO2hOPNcJyMNWEo7T9Jay324nU6ek5sOdDvxumELQhUADittKE6LfVsNgzm2tZb7cDs9PCc9nBGecI/qAqNe9w3A4W0xFKc6HrCUdp6ktbX3MUzpcBq9l2/roH8WkGPUVhGqC4z65AMwhsmAJW7A55t1TAYi0dio72WX/gJA//YZdhL3ARoGk/geMxCJ7ghVAGhgzaE4PQ9YSjtP0lrLfYh7XBcNPjIQiZEJVQBoY8uBPClnMUxpv7WW+5D1uLZ6jUAs96guMOp13wC003ogT3UyYGnLvdPX1t7HMKWvavTeW2rRe/S252YMo7aKUF1g1CcfgHFNBizxHD7fXG3aeCCSwUfsY9T3skt/AWBMBizBzW35njD4CEqoAsBm1hyU08uApS33Tl9ruc8owh6bRQORLlkz+AhKqALAlgxY2nbv9LWW+4wi6bFJOgt0xz2qC4x63TcA22o9pKcCByxt8Tj0srb2PiMOU2r0XlnqoO8puMqorSJUFxj1yQfguEwGLPGU0T/fTAYicSRGfS+79BcAjocBSxwTA5GgY0IVAIKsOTwnccBSq316XGu5T4o1HodbMhAJNiJUASDLsQ1YarVPj2st90mxxuNw6POM8lhDNPeoLjDqdd8A5Gk9uKc2HrC0xe+5l7W190kcpnTIx6EMROJIjNoqQnWBUZ98AJiuGbBkGMzYRv98c91reynvAXow6nv5ha0PAABs6mFdMXTmkg/6JgHT1LTt5F4DkWBDQhUAjthl4XnNN1EmAdParV9zvhWFPhmmBAAd2moabK+TbXtca7nPTaWfL/E8wDJCFQD6NMok4Fa/lx7XWu5zU+nnSzwPsIBhSguMeoMyAP1acxpsNZwEvPbvpee1tfc5xNTftR+H6QADkcrkXgY3aqsI1QVGffIB4DJ7xoEBS5069OebadvBR1dyjyqjG7VVXPoLAFy0z7TTuDBhM4mvBZN7oVNCFQAGcaihMfNcJ+ffQu2q6m495/NCDwODelxruc8SWw0buvhanOea9lg7MRAJ+iRUAWAcPQ5YanXuHtda7rPElsOGDESCI+Me1QVGve4bgLH0OGBp7XP3vLb2PvsOU7rla+TG5rmmNR5bGMWorSJUFxj1yQeApQxYGs9tPt+0HJxkGBJcb9RWcekvALCEAUs8rdVzbBgSHCmhCgADSx+wdMgzjrbWcp+Lbjk46aaDjy4dhrTn3sAghCoAjC19wFKrM/a41nKfi1r8f2/7OAADc4/qAqNe9w3A+NIHLK19xp7X1t7numFKb755eu3zVysN19rncQDOjNoqQnWBUZ98ALgNA5b6tvTzzb6Dkww/grZGbRWX/gIAN2XA0nHY57kz/Ag4CKEKAGw2YOmmex/DWst9LrrF4KQTg4+AQxCqAEDVdgN61th7lLWW+1zUYugSwJXco7rAqNd9A8BjWw1YOuTeo62tvc/SYUp1oKFZwDpGbRWhusCoTz4AHNo+A5a+/vX/rZ/97B/XPA57+tGP/qp+97s/WPzrDU6C7Y3aKi79BQAOafEwnX2CiDb2fE4MTgJWI1QBgMUOPWCJrhicBDTjXx4AwD4OPbSHfniegWaEKgCwj0+q6gfnf913jb55noFmXtj6AABAP86nuH68z9p0zUWhb755evAzjmGuCruadulzD3AIvlFd5qphAYYIAMDz+ffl3rIitTyHkGzIVvHjaQCAg1vycz33+VE2NOfnowKb8o0qALAGg3f65vkDNuUbVQDg4Hyj2j3fqAKbEqoAwCaEaq7zn4ULsBmX/gIAW+l60MfAPC/A5oQqANDENNU0TfX6+SWkNc91cv7N3a6q7lbVbp5rsna2tuHeJxefK4DWhCoA0MpVw3iWDu45trXE8wA04R5VAKCJq4bxLBm8dIxriecBaEWoAgAAEMWlvwAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAEQRqgAAAET5Pzxfqg2F7ogAAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (1.4) A* search search: 124.1 path cost, 975 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAGSNJREFUeJzt3TGPHOd9x/H/nAXBUSBaqgykStykcARZjSsH9hsIXBigCkHunOhVWNS7UBykVKED0iR9IMGu0kRKHMBd6jSRzIMUQIE5Kbh3Od7t3s3e7sz8nmc/n0bwY5Iz3FuK/Gp3fxzGcSwAAABIcbb2DQAAAMB1QhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUAAIAoQhUA6Mow1DAM9YNhqKHls8T7AViKUAUAevNmVf3D5p8tnyXeD8AihnEc174HAICj2bwK+GZVfT6ONbZ6lng/AEsRqgBAt4ahnlbVq2vfR2MuxrEerX0TwGkTqgBAt4bBq4EPMY4+mwqsy2dUAYBmGQKah8cVWJtQBQBaZghoHh5XYFXe+gsANOu+IaCqerbi7bXsrAwsASsSqgBAFwwnzcrAErAooQoAdKGv4aSxKuyjoAaWgCW9tPYNAABMMeFtvjt9/PH5/DfYgbfffrzz//P3rQJLMqYEALTCwM+6PP7AYoQqANCKz6vqZ/Xiq6fbzpiHxx9YjLf+AgBN2Ly19LNdZ4NPUM7takF581hfDix9tus7ADyUV1QBAKiqqm9/+3/3+eYWloHZWP0FAOJMHe7Z4+9MnfT3giadzX2djz8+3/l4vf3246vH6xiPK8C+vKIKACSaOtwzdcznkB9vrbMlr3PTsR9XgL14RRUAiOMV1Yc/Dl5RBXogVAGALgzD7igaxzK1dMP5+fnOx+vx48dXj9ddj+sWlwNLAAfx1l8AAO5ysce3NbAEHIVQBQBWNQw1DEP9YPOW0YPPlrjGEmdLXueux2sc69HmFemzqnqr7vnz4yHXBbgkVAGAta01DpR+tuR1blrr+wJUlc+oAgArO9Y4UBlTmmVMaa7HGuAuQhUAaM4w1NPa4/OQxpRumzqmtI2BJWBu3voLALRon9GefcaAmMbAEjAroQoALGKOwaAdrkZ/xvH5GFDSSFILY0rbGFgCliRUAYClzDEYNPU6LZ4teZ0pDCwBi/EZVQBgEcccDKrOhpPSxpS2fUZ1ia8JwCWhCgBEM5w0j0PGlLYxsAQck7f+AgDpDCe1wcAScDRCFQA4ujkGg3boYjiphTGlbQwsAXMRqgDAHOYYDDrkOi2eLXmdhzKwBMzCZ1QBgKM75mBQncBwUgtjStsYWALmIlQBgGh3jfQYTnq4Y48pbeNrBzzUS2vfQAvOz893rQ1ePH782GIdABzJngu/hpPyXdSOr+eWiLUEDA/Qa6sI1Wl2/YZpsQ4Ajmvn761egWvPtvC841VWf66Ch+myVYwpAQBHd+iy7SE/Zi9nS17nmA657qHPEaAfQhUAmMOhy7aH/Ji9nC15nWM69hLwPt8f6IQxpQmWGBsAgJ48dNm2TnTht9XV322OvQS868cEnuu1VXxGFQA4uk1MfHbX2Z7DSZN+zJ7O5r7O+fnNqx3Hfdcd7v5j81XEbr7dxTjWo12PD9Avb/0FANayz9CHhd9+7PO1bHoMBng4oQoAHGSmIZ+zqnqrqs7GsYZxrEdJQ0fGlPZz/bqbV0iHuvY13uf733UG9EOoAgCHmmPIJ2nUyJjS4ZZ6PgCdMKY0Qa8fUAaAY3jouM+w++/TrDqx4aSexpS2OWRgaRxr2Ocxg1PTa6sYUwIADjJhPOfWaNI9gzpRo0bGlA53yMDSzf+gcX1k6eaPCfTDW38BgLntO4hjOOn07Ps1N7IEnROqAMBkxx7tuTmqc6rDSb2PKW1z38DS5n9P+v53nQFtEqoAwD4MJy13tuR11mBgCdjJmNIEvX5AGQD29ZDRnjrSUM6pnc19nSXHlLZZanALetdrqxhTAgAme8hw0qE/5qmezX2dJceUttnncdjhKrQNLEF/vPUXADimfSLVaBL32ec5YmAJOiJUAYCtZhjouTWadOh1ej5b8jopbt7ftpGlfb7/rjMgn1AFAHY59kDPmoNBLZ4teZ0U+zwOU79/+s8Z2MKY0gS9fkAZAO5y7OGk2jJ+89DrnMLZ3NdZe0xpmymPQ+35HDOwRO96bRVjSgDAVsceTlpjMKjls7mvs/aY0jZTHofh7j92G1iCTnjrLwDwUIaTWIOBJTgBQhUAWGQ4ac3BoBbPlrxOuuv3bWAJToNQBQCqlhlOMqa039mS10m31HMRCGFMaYJeP6AMAJeWGE5aYzCo5bO5r5M4prTLUs9FaFGvrWJMCQBYZDjJmNJ+Z3NfJ3FMaZcJz8+7GFiCBnnrLwAwheEkkhlYgs4IVQBgkeEkY0r7P9bGlHZbYmDJYw3rEaoAQNUyYzXGlPY7W/I6LVrrOXuKjzUszpjSBL1+QBkALi0xVmNMKeuxaWlMaZu1nrOHfk3h2HptFWNKAMAiw0nGlPY7m/s6LY0pbXPkgaUXHOHMYBMcyFt/AYCbDCfRgzWfmwab4EBCFQBOzFrDSWsOBrV4tuR1enH957fvwNKc9zLXGfRMqALA6UkaoZnjx+zlbMnr9CLp5+xrBwcwpjRBrx9QBuA0JY3QHPr9ez6b+zqtjyltc+Bz+9hmH2yCqn5bxZgSAJyYtYaTjCntdzb3dVofU9rmwIGlYzvmYJNxJk6Ot/4CwGkznMQpafU5bJyJkyNUAaBjScNJaw4GtXi25HV6dv3nvG1gafPcPtrZEj+P+849R+iBUAWAviUNJxlT2u9syev0bM2v3zF5jnBSjClN0OsHlAHo34HjMouMwSQNGCWdzX2dHseUtln661fzDTbd+vW41M+PbL22ilCdoNcvPgCnZdh/OMnvcR3z55t5DEPTcXc52kRDev21bPUXJhg+GIaq+nFVfTq+77/uAM0ynATzu6h2x49avW865DOqcMPNEYHhg2GosT6ssf65xvpwE61Rgxs9jXq0eJZ2Px6HvLPE+9li9uGkFh6bpLMlr8Nhrj+uxx5sWvPnsu8ZHJNQhduuRgQ2UfphjWfv1lBDjWfvVl3FatLgRk+jHi2epd2PxyHvLPF+bvLY5J0teR0O09PX6f77GYbXf1kf/PSN+rd//GV98NMahteXv0165zOqE/T6vm+22/yXwTfrJ08+r5988GFVvVNVf3ztm3xVVR/VJ++/V588ebMCBjeWOEu7n6SztPvxOOSdpdxPrTyclPzYJJ7NfZ1TGVNawoq/budw978Lavh+Vf16rPrWszo7O6tnz4aqP1TVX9Y4/nbhe6X6bRWhOkGvX3x2u3ol9ZtX/rpe/vr2N/jmlap/f6fqn/62yjtegA4YTjo9/nzThiFonOn79dv6Tf2oHtXvX3hb5rOquqhH9Z16+oZYXV6vv5a99RduuIrUqne2RmpV1ctfV73xUdVf/U1Vzu8fAA9lOAlyRfz6fK2+2BqpVc+D4tV6WlX1a28D5liEKlxzNZz07OzdevHtvreJVaBdqwwn7RpfWeva6WdLXocs179Od40zLTna9N36rzqrZzu/4eb8W1X13QN+6nBFqMKLflxVv6izZ3806Vu//HXVW39f9aefzntXAMe11jjQroGYpPtJOlvyOmRZ8zkCEXxGdYJe3/fNbf//V9GcvTspVn1WFWjTKsNJxpSyHhtjSrnWeI7UPaNNf16/q3+pH9aju9+JfFFVP6xx/N3+P2seqtdWEaoT9PrFZ7sXPqN619t/RSrQKMNJVPnzDS+6b7Tptfqi/rP+bOtnVKueV+5Z1ZdV9b0axy9muUm26vXXsrf+wg3j++NYVe9V1Uf1zSvbv5FIBdoVMcwCxLnz3w1f1uv1o/pNPa3v3Hrp9XL1t57/FTUilaMQqnDDMNRQT8Y365P336uXv/5VPf97U6/7ql7++ld18SdnVcPB4wWtnKXdT9JZ2v14HPLOwu5nteEkY0qZjw2na+po0+XZf9RfnL1Wv3/jrOrLseriD3X21Vh1cVb1pb+ahmMTqnDb82GBT568WZevrD47+5+qqs0/P6qq9zb/f8rgRk+jHi2epd2PxyHvLO1+ks7S7ifpbMnrcJr2f948j9HvPaknP3+r/vW/n9STn9fzt/uKVI7KZ1Qn6PV932x3c2zgamCp6hdV9Xc11Hvj++OYNLjRw6hHy2dp9+NxyDtLu5+ks7T7STqb+zrGlJjjOcvyem0VoTpBr198ptsMLP24qj7dfIYVAJrmzzfQh15/Lb+09g1ACzZx+sna9wEAAKfAZ1QBoGNLjPEcOuSTdD9JZ0teByCNUAWAviWNA605GNTi2ZLXAYjiM6oT9Pq+bwD6lzQOZEwp67ExpgR96LVVvKIKAB0bxxrHsT67Hj9JZ2n3k3S25HUA0ghVAAAAoghVADgxpzgY1OLZktcBSCNUAeD0nOJgUItnS14HIIoxpQl6/YAyAKfplAaDWj6b+zrGlKAPvbaKV1QB4MSc4mBQi2dLXgcgjVAFAAAgilAFAAAgilAFALpftm3xbMnrAKQRqgBAVf/Lti2eLXkdgChWfyfodUkLAC71umzb8tnc17H6C33otVW8ogoAdL9s2+LZktcBSCNUAQAAiCJUAYDuB4NaPFvyOgBphCoAUNX/YFCLZ0teByCKMaUJev2AMgBc6nUwqOWzua9jTAn60GureEUVAOh+MKjFsyWvA5BGqAIAABBFqAIAW/U0GNTi2ZLXAUgjVAGAXXoaDGrxbMnrAEQxpjRBrx9QBoC79DAY1PLZ3NcxpgR96LVVvKIKAGzV02BQi2dLXgcgjVAFAAAgilAFACZrdTCoxbMlrwOQRqgCAPtodTCoxbMlrwMQxZjSBL1+QBkA9tXaYFDLZ3Nfx5gS9KHXVvGKKgAwWauDQS2eLXkdgDRCFQAAgChCFQA4SAuDQS2eLXkdgDRCFQA4VAuDQS2eLXkdgCjGlCbo9QPKAHAMyYNBLZ/NfR1jStCHXlvFK6oAwEFaGAxq8WzJ6wCkEaoAAABEEaoAwNGlDQa1eLbkdQDSCFUAYA5pg0Etni15HYAoxpQm6PUDygAwl5TBoJbP5r6OMSXoQ6+tIlQn6PWLDwBrOj8/f1pVr659H9zmzzfQjl5bxVt/AYC1iFQAthKqAMDRGfIB4BBCFQCYgyEfAB5MqAIAc/i8qn62+eddZwBwy0tr3wAA0J/Nyuxn950BwDZeUZ3mYs9zAOB+fh/N5OsCbemyVfz1NAAAAETxiioAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABRhCoAAABR/g/qIHhYi9KlRgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (2) A* search search: 128.6 path cost, 879 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAF0RJREFUeJzt3T+PHPd9x/HvrATCoSHKqgSkCtw6Aq3egfwEAhcCVoUgN4EVPgqTfBa0jVSBCi6QJukDCnYv0n96VwHUxPIdxAAMdOPilofjcvdul7cz+/nNvl6AQXi0pxnOHe17c3c/1/V9XwAAAJBidugLAAAAgMuEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAAAFGEKgAwKV1XXdfVj7uuupaPJV4PwFiEKgAwNXer6j+Wv7Z8LPF6AEbR9X1/6GsAANib5bOAd6vqWd9X3+qxxOsBGItQBQAmq+vqpKreOfR1NOa07+vOoS8COG5CFQCYrK7zbOCb6HvvTQUOy3tUAYBmGQIahvsKHJpQBQBaZghoGO4rcFBe+gsANOu6IaCqOjvg5bVsVgaWgAMSqgDAJBhOGpSBJWBUQhUAmIRpDSf1VWFvBTWwBIzp7UNfAADAm1p5me9Gjx8vxrmgxn3yyXzjP/PzVoExGVMCAFpm4Gc8BpaA0QhVAKBlz6rq47rmGVX2Yt29dv+BQXjpLwDQrOXLTZ9WVXXeQTm0iwXl5b1+ObD09FAXBEyXZ1QBAKiqqu997/93ebiFZWAwVn8BgGbt8DNTt/q5oEnHhj7P48eLjffrk0/mF/drH/cVYFeeUQUAWrbtmM+2Q0BJx8Y8z6p931eAnXhGFQBolmdUPaMKTJNQBQAmoes2R1Hfl6mlFYvFYuP9ms/nF/frqvu6xsuBJYAb8dJfAACucrrDYw0sAXshVAGAZnVddV1XP16+3HSnx6UfG/M8V92vvq87y2ekZ1X1YV3z/eNNzgvwklAFAFpmTGk/51l1qI8FqCrvUQUAGmZMafgxpXUfu497DXAVoQoATIIxpd1sO6a0joElYGhe+gsAwK4MLAGDEqoAQBP2NQ7UyrExz7MNA0vAmIQqANCKMQZ+ko6NeZ5tGFgCRuM9qgBAE64bBypjSnsbU1r3HtUx7j/AS0IVAJgEY0q7ucmY0jruP7BPXvoLAMA+bBxY6rrqV/5zMuaFAe15+9AXAABA+9b9CJornmW1BAxcyTOqAEATrP4Od543NcbnBDhOQhUAaIXV3+HO86YsAQODMKYEADTB6u9hV3/XsQQMDEWobmGxWJzU+vdSnM7n89fejwEAjM/q7G72vfq7zlWfkzVO173PFbjaVFvFS3+3s+kN/4YAAAA227gEvIbvq+DNTLJVhCoA0ARjSsOdZ58un6Pv687y2exZVX1Y13zvuct9AKZNqAIArTCmNNx59mnfn6ddPh6YCKEKALTiWVV9vPz1qmM3+dikY2OeZ5/2/Xna5eOBiRCqAEAT+r76vq+nl9dg1x27yccmHRvzPPt0w/OeVdVXVXXWddV3XZ2Mdd1AFqEKAMCYDCwB1xKqAEATjCkNd56h3WRgafXjrzoGTIdQBQBaYUxpuPMM7abXkvR7AUYgVAGAVhhTGu48Q7vptST9XoARCFUAoAnGlIY7z9Buci1dV32tGVkysATTJlQBADi0XQaWqowsweQJVQCgCcaUhjvPIVw3sLT871t9/FXHgDYJVQCgFcaUhjvPIRhYAjYSqgBAK4wpDXeeQzCwBGwkVAGAJhhTGu48h7CH6zOwBBMmVAEASLXLyJKBJZgQoQoANMGY0nDnSbF6fetGlnb5+E3HgHxCFQBohTGl4c6TYpf7sO3Hp/+egTWEKgDQCmNKw50nxS73YduPT/89A2sIVQCgCcaUhjtPil3uwwYGlmAihCoAAC0xsARHQKgCAHH2PQ6UNJJkTGl3l6/bwBIcB6EKACTa9zhQ0kiSMaXdjfG5B4J0fe/l+tdZLBYbb9J8Pvc3cQCwZ8tnuu5W1bOX7y287lidvz9xk9mu/75DHxv6PI8fLzber7Tvb8b43EOrptoqnlEFAOLsexwoaSTJmNLubvh7MbAEDRKqAAC0zsASTIxQBQDiGFMa9zwtMrAE0yZUAYBExpTGPU+LDCzBhBlT2sJU36AMAKmMKb35fZjimNI6Bpbg3FRbxTOqAEAcY0rjnqdFBpZg2oQqAABTZGAJGiZUAYA4xpTGPc9UGFiC6RCqAEAiY0rjnmcqDCzBRBhT2sJU36AMAKmMKb35fTiWMaV1DCxxjKbaKm8f+gIAAFYto+DppmNdVye1w/sKr/v3JR4b+jyLxerZ2rfF181VLiJ2+bjTvq87q/8+YBxe+gsAtGiX8ZtdRnWYNgNL0AihCgDEueGwzcV4Tt+fj+okjSQZUxqXgSVok1AFABId23CSMaXhGFiCBhlT2sJU36AMAKmObTjJmNJwDCwxdVNtFWNKAECcYxtOMqY0HANL0CahClvoHnZdVX1UVV/2970MAWBkhpMY0mlt/zVmYAlG4j2qsGJ1JKF72HXV16Pq67+rr0fLaI0a3JjSqEeLx9Kux33IO5Z2PUnHdn3sGpMYTjrk180xunwfhhpYmtLXiK8lDkGowusuRhKWUfqo+tln1VVX/eyzqotYTRrcmNKoR4vH0q7Hfcg7lnY9Scd2feyqpN9Lq183x2gqX3OHOXfXvffLevizD+r3//nLeviz6rr3CvbMmNIWpvoGZdZb/s3g3frpg2f104ePqurTqvr+pYd8W1Vf1JP79+rJg7sVMLgxxrG060k6lnY97kPesbTrSTq2zWPrCIaTDvF1cyxjSutM5WvuIOeu7kdV9du+6q2zms1mdXbWVX1XVf9Uff/H7T8L7MtUW8V7VGFF31ffPeyeVdWjenH787r1fPUh368Xtz+vd/7n86q+qrrq1vxPwJSPpV1P0rG063Ef8o6lXU/SsauOb5I0fmRMqR17Hlh6Rdqfn30e+1H9sb6pd+tO/bVmVfXW8lacVdVp3fnDu133gVhlX7z0F1ZcvNy36tM1kXru1vOqD76o+ud/raqNf4kFwLAMJzEUX1srflB/qd/VTy4i9bJZVb1TJ1VVvy0vA2ZPhCpccjGcdDb7rF59ue/rxCrA2CY7nLTu2Jjn4dV7s+vA0jF4v76uWZ1tvBHL429V1ftjXRPTdvR/6GDFR1X1i5qd/d1Wj771vOrDf6v6hy+HvSoAqrJGbMY4NuZ5cL8gilCFV31ZVb+ps9n/bfXoF7ervvqXqj9/NOxVAVB1PnLz8fLXYzg25nlwvyCKUIVL+vt9X13dq9nZv9f5uu9mL25X/eHTqv/6VZVXUAEMru+r7/t6ennxdMrHxjwP7td1vq7366xmG+eQl8e/q6qvx7ompk2owor+ft9X1b2q+qJe3F7/IJEKMDbjNhzaUX8NflPv1U/qd3VS774Wq8vV36rzH1Hzl/GvjikSqrCi66qrB/3denL/Xt16/ut6/ZnVb+vW81/X6d/PqrrLox6vjC5M7Vja9SQdS7se9yHvWNr1JB3b4bGTHk4yppTpuoGlhv787OXYn+ofZz+ov34wq/qmrzr9rmbf9lWns6pv3q0TP5qGvRKq8Lrz4YQnD+7Wy2dWX75n9fzXL6rq3vKfpwxuTGnUo8VjadfjPuQdS7uepGNp15N0bMzzsF7S10PGn5/zGP3hg3rw8w/rq/99UA9+XlU/FKnsW9f3R/+S+2stFouNN2k+n/sbyYlZ/i3z3ap61vfVX/zImqpfVNVvqqt7/f2+X33cuo+d0rG060k6lnY97kPesbTrSTqWdj1Jx4Y+z+PHi01vN/T9zVLS10MLf344jKm2ilDdwlQ/+Wyve9h1df6ja75cvocVAJrm+xuYhqn+WX770BcALVjG6ZNDXwcAABwD71EFAOKMMSKUfmzM8wCkEaoAQKKkwZpDHRvzPABRhCoAkOhZVX28/PVYj415HoAoQhUAiNP31fd9Pb28Jnpsx8Y8D0AaoQoAAEAUoQoAxEkaNTKmBDA+oQoAJEoaNTKmBDAyoQoAJEoaNTKmBDAyoQoAxEkaNTKmBDA+oQoAAEAUoQoAAEAUoQoAxEla37X6CzA+oQoAJEpa37X6CzAyoQoAJEpa37X6CzAyoQoAxEla37X6CzA+oQoAAEAUoQoAxEkaNTKmBDA+oQoAJEoaNTKmBDAyoQoAJEoaNTKmBDAyoQoAxEkaNTKmBDA+oQoAAEAUoQoANCFp6MiYEsCwhCoA0IqkoSNjSgADEqoAQCuSho6MKQEMSKgCAE1IGjoypgQwLKEKAABAFKEKADQhaejImBLAsIQqANCKpKEjY0oAAxKqAEArkoaOjCkBDEioAgBNSBo6MqYEMCyhCgAAQBShCgA0IWnoyJgSwLCEKgDQiqShI2NKAAMSqgBAK5KGjowpAQxIqAIATUgaOjKmBDAsoQoAAEAUoQoANCFp6MiYEsCwhCoA0IqkoSNjSgADEqoAQCuSho6MKQEMqOt776W/zmKx2HiT5vO5l84AwBtYLBYnVfXOoa+D1/n+Btox1VbxjCoAcCgiFYC1hCoAAABRhCoAAABRhCoAAABRhCoAAABRhOp2Tnc8DgBcz/+PZvJ5gbZMslX8eBoAAACieEYVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKEIVAACAKH8DxgVAk+apKx8AAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " Greedy best-first search search: 133.9 path cost, 758 states reached\n" + ] + } + ], + "source": [ + "plots(d6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the next problem, `d7`, we see a similar story. the optimal path found by A*, and we see that again weighted A* with weight 1.4 does great and with weight 2 ends up erroneously going below the first two barriers, and then makes another mistake by reversing direction back towards the goal and passing above the third barrier. Again, greedy best-first makes bad decisions all around." + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3cGqJOmVH/AT16Jpy6gkrwxemdkacbs3XnnQvIARs8m7EJrd2L3yI6irX2FW8hgvxqBFFYgBe28k7L1pewzeyI9gSV2oDRJT4UWliuqsvFkRNzMi/t8Xvx8cGk7fqjhxMjKrzr0Rp4ZxHAsAAABS3G1dAAAAALzLoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyoAAABRDKoAAABEMagCAAAQxaAKAABAFIMqAAAAUQyqAAAARDGoAgAAEMWgCgAAQBSDKgAAAFEMqgAAAEQxqAIAABDFoAoAAEAUgyrQtWGoYRjqk2GoYakcAAC3ZVAFendfVT8//nepHAAANzSM47h1DQCLOf7k876qvhzHGpfIAQBwWwZVAAAAorj1FwAAgCgGVaBJayxJmrNMKame3vsAAPTPoAq0ao0lSXOWKSXV03sfAIDOeUYVaNIaS5LmLFNKqqf3PgAA/TOoAgAAEMWtvwAAAEQxqAJRkpYDpS0RSs+l1ZOUW/M4ANADgyqQJmk5UNoSofRcWj1JuTWPAwDN84wqEGWNZTy3zqXVow95uTWPAwA9MKgCAAAQxa2/AAAARDGoAqtIWmzT6qKc9FxaPUm5tHqScnO/FoB9MKgCa0labNPqopz0XFo9Sbm0epJyc78WgD0Yx1EIIRaPqnGoGj+pGofecmn16ENeLq2epNzcrxVCCLGPsEwJAACAKG79BQAAIIpBFQAAgCgGVeAqSdtDW9ha2nMurZ6kXFo9Sbm0epY4PwCeYOuHZIUQbcdx2cmvqsZP9ppLq0cf8nJp9STl0upZ4vyEEELMj80LEEK0HRW0PXSrXFo9+pCXS6snKZdWzxLnJ4QQYn7Y+gsAAEAUz6gCAAAQxaAKvGePC08sg9EHvdGbtNxSvydAE7a+91gIkRe1w4Un1+TS6tGHvFxaPUm5tHqSckv9nkII0UJsXoAQIi9qhwtPrsml1aMPebm0epJyafUk5Zb6PYUQooWwTAkAAIAonlEFAAAgikEVdiRtqUcvubR69CEvl1ZPUi6tnqRcWj3XngvALFvfeyyEWC8qbKlHL7m0evQhL5dWT1IurZ6kXFo9156LEELMic0LEEKsFxW21KOXXFo9+pCXS6snKZdWT1IurZ5rz0UIIeaEZUoAAABE8YwqAAAAUQyq0AFLPbbNpdWjD3m5tHqScmn1JOXS6mmhD0BHtr73WAhxfZSlHpvm0urRh7xcWj1JubR6knJp9bTQByFEP7F5AUKI66Ms9dg0l1aPPuTl0upJyqXVk5RLq6eFPggh+gnLlAAAAIjiGVUAAACiGFQhWKvLLPaWS6tHH/JyafUk5dLqScql1bPHPgAb2vreYyHE41GNLrPYWy6tHn3Iy6XVk5RLqycpl1bPHvsghNguNi9ACPF4VKPLLPaWS6tHH/JyafUk5dLqScql1bPHPgghtgvLlACAbg1DfVVV39m6jsa8Gsd6tnURwL4ZVAGAbg1D+YvOE4yj5zSBbVmmBBtIWlLRwjKL9FxaPfqQl0urJym35nGYbo/XiGsJwmx977EQe4wKWlIxNZdWT1IurR59yMul1ZOUW/o4VeMonhS7uUaeUo8QYvnYvAAh9hgVtKRiai6tnqRcWj36kJdLqycpt/RxqsatB75WYzfXyFPqEUIsH55RBQC6MFictCQLloBVGVQBgC4MXS1OGqvCHoscLVgCVvStrQuAnhyXLdxX1Zfj+OYvTL3k0upJyqXVow95ubR6knK3/D3rghcvXl763xw9PBwe/X89XCNL1gjclq2/cFv3VfXz4397y6XVk5RLq0cf8nJp9STllvo9ub2erhHXHIRz6y/cUNJ3d/f4HW290YfUXFo9Sblb/p5V9boe4Seq01z6iWq9+QFH09eIn6hCOwyqAEBzhpmLkwyq03xgUD1lwRKwGLf+AgAtmjykfvzxH5asoysze2XDMrCcrf99HCFaiKR/v82/Y5eXS6tHH/JyafUk5Z7666vG8ULEnF/ydfPixYvxsUjqddrrMqduIcTTY/MChGghjn/4/Kpq/GSvubR6knJp9ehDXi6tnqTcU3991XhpeIo5v+Tr5gODakyv016XOXULIZ4emxcgRAtRQd+h3SqXVk9SLq0efcjLpdWTlHvqr68aLw1PMeeXfN34ierydQshnh7DOI4FAJBq7uKkcaxhwXK68fLly0f/Eng4HN72cBhmbba1YAm4CcuUAIB0c5b2vFqsiv2a01MLloCbMKjCiWGoYRjqk+O/kyanN3qjD3oT0ptH3FXVp1V19+ZOsXqWdH4tXDfnvPt141jPjj+lftvrqb+2pT5s1WvgEVvfeyxEWlTQIoakXFo9Sbm0evQhL5dWT1JuytdWjZeekYw5l9aum6nLlLZ+TdJel2vPRQgxLTYvQIi0qKBFDEm5tHqScmn16ENeLq2epNyUr60aLw1FMefS2nUzdZnS1q9J2uty7bkIIaaFZUoAQIzB4qTVTF2mdM5gwRKwMM+oAgBJLE5qgwVLwKIMquxW7wsbLLPQG33IyaXVk5S7lD+j28VJW14353zo144rLlhaqw9bvs7AGVvfeyzEVlGdL2y4dS6tnqRcWj36kJdLqycpd5qvGi89+xhTdw/XzTXLlNZ87dJelyVeZyHE+7F5AUJsFdX5woZb59LqScql1aMPebm0epJyp/mq8dKwE1N3D9fNNcuU1nzt0l6XJV5nIcT7YZkSALCJweKkTV2zTOmcYd6CpaksYoKd8owqALAVi5P6ssRrZBET7JRBlV2wsOH6XFo9Sbm0evQhL5dWT1Lugl0tTtryujnnKb/fOHPB0hxJr8vG7wvYj63vPRZijagZCw2mfu3ecmn1JOXS6tGHvFxaPSm5qvHSM40RNfZ83dx6mdK53Ade40mR9rqs9doLsffYvAAh1oiysOHqXFo9Sbm0evQhL5dWT0quarw0oETU2PN1c+tlSudyH3iNJ0Xa67LWay/E3sMypQlevnz52LKHV4fDwQP+AHDBMHNpUlXVaHHS4m69TOmcYZkFS4+xeIld6nVW8YzqNI/94eoBfwD4sLl/Xlqc1I81X0t/L2OvupxVDKrsQtriihZzafUk5dLq0Ye8XFo9W/bh1HiyfGfc6eKkLa+bc251jPHMgqXjazwpd6nGJetOzMHeGFTZi/uq+vnxv5dyc752b7m0epJyafXoQ14urZ4t+3BOUo1JuTWPcyqtD1Ml1b1lH6B5nlGdYI1nOFjW8TuS91X15Ti+eV7mXG7O1+4tl1ZPUi6tHn3Iy6XVs3auql7XI8axhoQaE3NLH+fFi5ePvi4PD4e7hD4M859xjah7iRw8ptdZxaA6Qa8vPkBLhics5CGfpUnbaeHvN08YVKewdImutPBefgq3/gLQCkNqfyxN4kOWuEZ8lkADDKp059pFBdf8+p5zafUk5dLq6b0PNO29pUlVWddxUm7N40yxRR+mLmNq4VyWyEHPDKr06NpFBUmLE5JyafUk5dLq6b0PtMv7Z15uzeNM0UIfpkp6nX0ewhmeUZ2g1/u+e3X8TuN9BS6uaDmXVk9SLq2eXvtQFxby0Iz3Ft1UZV3HSbmlj3NpmdK5v98k96Hmfz50sXQJqvqdVQyqE/T64gOkGixO6pLFSVl6+vvNYOkSO9bTe/ldbv0FIJEhtT8WJ7EkS5egMwZVmrbEUoKkJQlJubR6knJp9fTUh3NevHj5XtQHFq1snUurZ6PcM++febk1j/NUKX0Yd7p06VIeWmdQpXVbLmzYWy6tnqRcWj099WGqpD64Rubl0upJyq15nKdqtQ9TJV0Pc3oDzfOM6gS93vfdg+N3D++rkcUVLefS6knKpdXTQx/qwmKU409Qv+Hh4RC9GCWtnqRcWj1JuaWPM3eZ0jmt9aE6W7p0Kc9+9DqrGFQn6PXFB0g1XFiMcm5Q9VkM8+3x7zeXPlvOGS0AowG9vpe/tXUBACxn6Gx77scf/2HrEoC2vaoZn4kTB1vbgWEBBlWAvjU9pL7704xL3zEGmGLqQDnzJ69Nf85CKsuUaMatN+U9tiVvjeO0mEurJymXVs9jNbZo6rkk9brVa0Rv8nJrHueWWu3DNb1p4RqB1hhUacmW2/Pk8upJyqXV81iNLZp6Lkm9bvUa0Zu83JrHuaVW+3BNb1q4RqAplilN0OsDyq05flfwvhrZLNhbLq2epFxaPe/mav6GyzRvN25e2lBq62+7ubR6knJLH+cWW3/Paa0PLW8Hnvu19KnXWcWgOkGvLz7Ql6GzxUlV059R9VkM83lPPW6YuR34ChYxcbVe38tu/QXoR1dDar3ZzgmwhbU+f3r73IabMajSjLSlBHvLpdWTlEus59SLFy/fi3rzZ8CnVXU3jjUcf3r55Ny1v/5M7tnU80vqfwvXSFIurZ6k3JrH2UJaH97NjWM9m/JZtWYflugNJDOo0pK0pQR7y6XVk5RLrGeKVvuwxrm02psWc2n1JOXWPM4W0vqwVQ+3vEYglmdUJ+j1vu/WHL8DeF8hSwn2lkurJymXUk9dWP5x/AnqN9x6AdHS52eZUp+5tHqScksfZ6llSlOl9GGpz90Znvz5de250IdeZxWD6gS9vvhAu4aZi5PODaqtfX75LIbb8p663mDpEgF6fS+79RegTZOH1I8//sOSdQDsmaVLsBCDKs24ZlnAnKUCaxynxVxaPUm5rY99xttFHy9evKy/+Zu/PftFrfZhjXNptTct5tLqScqteZwUrV0jU5cu3WIR01q9gRQGVVpy68UHSywl6DmXVk9Sbutjn7rm61rowxrn0mpvWsyl1ZOUW/M4KfZ4jUy1Vm8ggmdUJ+j1vu/WHL/bd18NL65oOZdWT1Jui2PX5QUebxdzrLmAaOlztkypz1xaPUm5pY+z9TKlc/Z0jdT8RUzvfc4tUSPt6XVWMahO0OuLD7RhmLk46Xh7WVX19fnV07lAAu+pbQ3LLWKyeGlnen0vu/UXIN+cJRprLfYA4DpLfV5bvEQXDKo049bLAh5bILDGcVrMpdWTlFvzOGecLut4NvXX9tSHpOvB+0dvWutNC5Jel1vlpi5iSu0XLM2gSkvWWmiwxnFazKXVk5Rb8zin1vi1PfWh92ukxVxaPUm5NY+TLul12fJ6mCqtHpjNM6oT9Hrfd2uO38W7r4YXV7ScS6snKbf0cWri4qStFxAt3QfLlPrMpdWTlFv6OInLlB6T9Lqsnav5S5eqPvBng6VLfel1VjGoTtDriw/kGa5YnHROT59fPZ0LJPCeasNg6RIf0Ot72a2/AFksTgLgXZYusUsGVSKtsQTgscUAWx07PZdWT1Juqd/zjEmLk6b+fi30YatzabU3LebS6knKrXmcXiS9frfKXVq6dO3iJdcSyQyqpFpjCcBjiwG2OnZ6Lq2epNxSv+eppN9vy2tkjXNptTct5tLqScqteZxeJL1+W14jU7mWiOUZ1Ql6ve872fG7c/fV2eKKlnNp9STlbvl71hWLk7ZeQLR0vy1T6jOXVk9SbunjtLRMaaqk12+La6TmL166+Z8DrK/XWcWgOkGvLz6wreHGi5PO6enzq7Nzeey1f3U4HCw3YRU9vad4Y1hu8dIpi5iC9Ppe/tbWBUALhi+Goap+UFW/HD/33R1uxuKk/XrstbfcBLjGq1rnc8RnFYvzjCqRbv1w/5wlAKf54YthqLF+WmP9lxrrp8ehdbMak3ojd5vePOLJi5OmHiOtD0nnslZvWjyXVq+bFnNrHqdnSa/p0tfIpcVLdcXSpXP2eC2xLoMqqdZYQPDYEoC3+eNQ+tMa735cQw013v246u2wulWNEb2Ru2lvzlmixjWOsVa/lz6XtXrT4rm0et20mFvzOD1Lek3TrpEn+V79un5SX/zw+/U//tNP6osf1jD842t/TzjlGdUJer3vO9nxu3P3teVSgj97/mX92Rc/raofVdU/eqe831XVz+oXn39Wv3i+eo0RvQmoJyn31F9fN16ctPUCoqX73dMypZbPJeX9s4fc0sfpcZnSOUmvaco1UvOXLr31z+vv6r/Vv6zv1m9fva67u7t6/Xqo+vuq+tMax7976u/L0/U6qxhUJ+j1xedxb3+S+vtv/+v66Ov3v+D33676nz+q+s//rsodLyzgKYuTzunp88u5wG25DvdreOLSpT8Oqc/qt9+4LfN1Vb2qZ/Xd+ur7htX19fpedusvnHg7pFb96OyQWlX10ddV3/9Z1b/6N1U2snN7FicBsKTZf858r359dkitejNQfKe+qqr6r24D5lYMqkTaainB28VJr+9+XN+83fd9hlVu56aLk87lzlniGGu8T9c4l7V60+K5tHrdtJhb8zhkvfZLXyNTly69m/u39Vd//t367avHhodj/h9U1T+50GaYzKBKqq2WEvygqv6y7l7/w0lVfvR11af/oeqf/XLSl8Mj1ri21zhuT+eyVm9aPJdWr5sWc2seh6zXPu0auf/b+vO/el13ZgdW4xnVCXq97zvZ8TuA97X2ooI//kR1vPvxpGHVs6rcxmLLcyxTysj1dC5L92brepJySx9nL8uUpkp67VOukXdzP6kvfvi8nv/H4fK/ofqqqv5FjeP/ntN7rtPrrGJQnaDXF5/zvvGM6qXbfw2p3MitFied09Pnl3OB23IdMsubZ0//z+uq7537serrqrqr+k1V/UmN46/XLW7fen0v+/E9nBg/H8eq+qyqfla///b5LzKkcjsWJwGQ783w+aev6tl7/7bNH7f+1pt/osaQyk0YVIm09VKCej7e1y8+/6w++vqv682/m/qu39VHX/91vfqnd1XDpAUEPeTS6knKXfnrb7446bFr+1TKUo/Ec1mrNy2eS6vXTYu5NY/DeUnXQ8T7p8b/9d366vt3Vb8Zq179fd39bqx6dVf1G/80DbdmUCXV9ksJfvH8vv74k9XXd/+vqur4359V1WfH/5+yTKGrhQ0N5tLqeazGU632YY1zWas3LZ5Lq9dNi7k1j8N5SddDxvvnzTD6J8/r+V98Wv/9/z6v539Rb273NaRyU55RnaDX+76THb+Ld18BSwneLliq+suq+vc11Gfj5+O4VY1JvZHL741lShm5ns5lT++frXNLH8cypQ9Luh5aeP+wjV5nFYPqBL2++Ex3XLD0g6r65fEZVmhCT59fzgVuy3UIfej1vfytrQuAFhyH019sXQcAAOyBZ1SJtOpigC2XEjSUS6snKZdWz2M1nmq1D2ucy1q9afFcWr1uWsyteRyANAZVUu1nKUE7ubR6knJp9TxW46lW+7DGuazVmxbPpdXrpsXcmscBiOIZ1Ql6ve872fE7vfe186UESbm0epJyafW8m7NMKSPX07ns6f2zdW7p41imBH3odVYxqE7Q64sP9K+nzy/nArflOoQ+9PpedusvAAAAUQyqNKOnxRUt5tLqScql1fNYjada7cMa57JWb1o8l1avmxZzax4HII1BlZb0tLiixVxaPUm5tHoeq/FUq31Y41zW6k2L59LqddNibs3jAETxjOoEvd733Zrjd3/vq+HFFS3n0upJyqXV827OMqWMXE/nsqf3z9a5pY9jmRL0oddZxaA6Qa8vPtC/nj6/nAvclusQ+tDre9mtvwAAAEQxqNKMnhZXtJhLqycpl1bPYzWearUPa5zLWr1p8VxavW5azK15HIA0BlVa0tPiihZzafUk5dLqeazGU632YY1zWas3LZ5Lq9dNi7k1jwMQxTOqE/R633drjt/9va+GF1e0nEurJymXVs+7OcuUMnI9ncue3j9b55Y+jmVK0IdeZxWD6gS9vvhA/3r6/HIucFuuQ+hDr+9lt/4C9O3VzDwAwOYMqjSjp8UVLebS6knKpdXzbu5wODw7HA7Dw8Ph7uHh8OnDw+HucDgMh8PhWat9OCeh10/pTYvn0up102JuzeMApDGo0pKeFle0mEurJymXVk/vfTgnqQ9zetPiubR63bSYW/M4AFE8ozpBr/d9t+b43d/7anhxRcu5tHqScmn19NqHlhcQ9XQurV03LeeWPo5lStCHXmcVg+oEvb74AC3p6bO4p3OhXa5D6EOv72W3/gIAABDFoEozelpc0WIurZ6kXFo9vffhnKQ+zOlNi+fS6nXTYm7N4wCkMajSkp4WV7SYS6snKZdWT+99OCepD3N60+K5tHrdtJhb8zgAUTyjOkGv93235vjd3/tqeHFFy7m0epJyafX02oeWFxD1dC6tXTct55Y+jmVK0IdeZxWD6gS9vvgALenps7inc6FdrkPoQ6/vZbf+AgAAEMWgCgAAQBSDKk1rdcNii7m0epJyafX03odzkvowpzdTJZ1Lq9dNi7k1jwOQxqBK61rdsNhiLq2epFxaPb334ZykPszpzVRJ59LqddNibs3jAESxTGmCXh9Q7sHxO8L31ciGxZZzafUk5dLq6bUPLW/KnXMu5/5cSTqX1q6blnNLH8fWX+hDr7OKQXWCXl98gJb09Fnc07nQLtch9KHX97JbfwEAAIhiUKU7LSyuaDGXVk9SLq2e3vtwTlIf5vTm1pLOOe26aTG35nEA0hhU6VELiytazKXVk5RLq6f3PpyT1Ic5vbm1pHNOu25azK15HIAonlGdoNf7vnt1/C7xfQUurmg5l1ZPUi6tnl77sOdlSlMlnXPKddNybunjWKYEfeh1VjGoTtDriw/Qkp4+i3s6F9rlOoQ+9PpedusvAAAAUQyq7ELa4ooWc2n1JOXS6um9D+ck9WFOb7bSQm/k1j0OQBqDKnuRtriixVxaPUm5tHp678M5SX2Y05uttNAbuXWPAxDFM6oT9Hrf954cv3N8X5Z6xC71aDmXVk+vfbBM6XaSe7N1PUm5pY+z9XUI3Eavs4pBdYJeX3yAlvT0WdzTuVzj5cuXX1XVd7aug/ft6TqE1vX6Z4pbfwFoxauZefIZUgE4y6DKLqQtrmgxl1ZPUi6tnl77cDgcnh0Oh+Hh4XD38HD49OHhcHc4HIbD4fAsqQ9zerOGpD5s1QMA2mNQZS/SFle0mEurJymXVo8+5OUu5ZeW1IetegBAYzyjOkGv933vyfG79/dlqUfsUo+Wc2n16ENe7jS/5hKbpD6c5i79+cq2/P0G2tHrrGJQnaDXFx+Abfhz5Q2Daq49XYfQul7/THHr7zQWeADA7flzNJPXBdrS56wyjqMQu4yqcagaP6kahw/l5fLqScql1aMPebnT/IsXL8bH4prPsBZzafUk5RLrEUKItcJPVNkzy2Dm5dLqScql1aMPeblL+adKOj/XzTK5xHoA1rH1pCzEVtHCd6+Tcmn1JOXS6tGHvNxp3k9UXTct9kYIIdaMYRzH6VMtAHC1XhdfAMCtuPUXAACAKAZVODEMNQxDfXL89/7k9EZv9GGx3kyVdC6um/32BmBNBlV4X9LiiqRcWj1JubR69CEvdyk/RdK5uG7WyyXWA7COrR+SFSItkhZXJOXS6knKpdWjD3m50/zcZUpJ5+K62W9vhBBizbBMCQBWZpkSAFzm1l8AAACiGFRhgqRlFpZ65OXS6tGHvNyl/Kmkul03evOh6xVgKQZVmCZpmYWlHnm5tHr0IS93KX8qqW7Xzba5xHoA1rH1Q7JCtBBJyyws9cjLpdWjD3m50/ylZUpJdbtu9Oa0HiGEWCssUwKAlVmmBACXufUXAACAKAZVuKGkpRe9L/VIyqXVow95uUv5U0l1u2705kPXK8BSDKpwW0lLL3pf6pGUS6tHH/Jyl/Knkup23WybS6wHYB1bPyQrRE+RtPSi96UeSbm0evTlRM9RAAAIBUlEQVQhL3eat0zJddNib4QQYs2wTAkAVmaZEgBc5tZfAAAAohhUYQNJyzFaXeqRlEurRx/ycpfyp5Lqdt3ozYeuV4ClGFRhG0nLMVpd6pGUS6tHH/Jyl/Knkup23WybS6wHYB1bPyQrxB4jaTlGq0s9knJp9ehDXu40b5mS66bF3gghxJphmRIArMwyJQC4zK2/AAAARDGoQrC0JRpJ9STl0urRh7zcpfyppLpdN/vtDcDWDKqQLW2JRlI9Sbm0evQhL3cpfyqpbtfNtrmtjw2wna0fkhVCPB5pSzSS6knKpdWjD3m507xlSq6b9N4IIcTWYZkSAKzMMiUAuMytvwAAAEQxqEIHel/qkZ5Lq0cf8nKX8qeS6nbd7Lc3AFszqEIfel/qkZ5Lq0cf8nKX8qeS6nbdbJvb+tgA29n6IVkhxPXR+1KP9FxaPfqQlzvNW6bkuknvjRBCbB2WKQHAyixTAoDL3PoLAOt7NTMPALtiUIUdaXWpR3ourR59yMud5g+Hw7PD4TA8PBzuHh4Onz48HO4Oh8NwOByeJdXtutlHbwAibX3vsRBivTg+g/SrqvGTublrf33PubR69CEvl1ZPUi6tnqTcmscRQoi02LwAIcR6UY0u9UjPpdWjD3m5tHqScmn1JOXWPI4QQqSFZUoAAABE8YwqAAAAUQyqwHssPJmXS6tHH/JyafUk5dLqScot9XsCNGHre4+FEHlRFp7MyqXVow95ubR6knJp9STllvo9hRCihdi8ACFEXpSFJ7NyafXoQ14urZ6kXFo9Sbmlfk8hhGghLFMCAAAgimdUAQAAiGJQBa6StHikp4UnLebS6knKpdWTlEurZ4nzA+AJtr73WAjRdlTQ4pGtcmn16ENeLq2epFxaPUucnxBCiPmxeQFCiLajghaPbJVLq0cf8nJp9STl0upZ4vyEEELMD8uUAAAAiOIZVQAAAKIYVAEAAIhiUAVWkbSF09ZSfdCbnNzcrwVgHwyqwFruq+rnx//2lkurRx/ycmn1JOXmfi0Ae7D1NichxD4iaQunraX6oDc5ublfK4QQYh9h6y8AAABR3PoLAABAFIMqECVpyYtFOfrQWm8AoBcGVSBN0pIXi3Lm5dLqScqteRwAaJ5nVIEox58O3VfVl+NYYwu5tHr0IS+35nEAoAcGVQAAAKK49RcAAIAoBlWgSXtclJOeW/M4AEDfDKpAq/a4KCc9t+ZxAICOeUYVaNIeF+Wk59Y8DgDQN4MqAAAAUdz6CwAAQBSDKtA1y30AANpjUAV6Z7kPAEBjPKMKdM1yHwCA9hhUAQAAiOLWXwAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACCKQRUAAIAoBlUAAACiGFQBAACIYlAFAAAgikEVAACAKAZVAAAAohhUAQAAiGJQBQAAIIpBFQAAgCgGVQAAAKIYVAEAAIhiUAUAACDK/wcquXLwuqOEZgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " A* search search: 127.4 path cost, 4,058 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHjZJREFUeJzt3UGPJdlZJuAvklar8cgFXllihdjMgrHavWHFCP4AYmEpa2GZHUP/Crr7X3gYzcJCXlRKCGlmP2oLVrOhegCJv+ANmC65kWy5YhaVlc7Oupl5b0bEifec+zwSavE5s86JuHFv1ls37pvTPM8FAAAAKS723gAAAADcJqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAIYyTTVNU313mmrqeZa4H4BWBFUAYDQfVtXfXP+351nifgCamOZ53nsPAACruX4X8MOq+mKea+51lrgfgFYEVQBgWNNUX1bVN/feR2dezXM923sTwHkTVAGAYU2TdwOfYp59NhXYl8+oAgBdUATUjvMK7E1QBQB6oQioHecV2JVbfwGALjylCKiqXu+03d5dlIIlYEeCKgAwBMVJm1KwBDQlqAIAQxirOGmuCvsoqIIloKX39t4AAHDe1vp9nw+t8eLF1XYHMJDnzy/v/d/8vlWgJWVKAMDelpQkKfhpx/kHmhFUAYC9fVFV36uvvyu6ZMY2nH+gGbf+AgC7ur5l9OUpM8VJu7hpUJ7efFr1bcHSy/u+AeCpvKMKAPTo6JD6wQe/3HIfQznxXPmHAmAzWn8BgEWWlCE99fvr4d+PetTvAE2fbb3OixdX957D588vb87hGuca4FTeUQUAllpafLR2cdLa5Ux7zVquc9fa5xrgJN5RBQAW8Y6qd1SPORaAUwiqAEC0U4uT5rmmDbczjKurq3v/Enh5eXlzDqfppLD5tmAJYBG3/gIA6U4p7Xm12S7O1ynnVMESsApBFQA4aJpqmqb67vWtnJvMTv3aAy6q6qOqupjnmua5nrXYd9q5WbrOXbe/bp7r2fW71Dfn+tjvPXVdgLcEVQDgPmmFQXvtUZlSu+8FqCqfUQUA7pFSGFRnUJyUXKa01WMC8BBBFQCIoTipnWPLlA5RsARsza2/AEASxUl9ULAEbEpQBQDiCoPuMWxxUlqZ0iEKloCWBFUAoCqvMChpj2nnpkWZ0iEKloBmfEYVAIgpDKozLU5KK1M69BnVFqVXAG8JqgDALhQn7WtJmdIhCpaANbn1FwDYi+KksShYAlYjqALAwJLKgU4szzmr4qQeypQOUbAEbEVQBYCxJZUDnVKek7RHZUr3U7AEbMJnVAFgYEnlQHdnj3ym8ayKk3ooUzpEwRKwFUH1CFdXV/eVPby6vLxUBAAADzi1NKlKcVILa5cpHaJgCbY3alZx6+9x7vvhqggAAB536s9LxUnjULAE2xsyqwiqAMRKKrbpoRQnfXbI3fKdcy1O2vO6WdvtNdYoWGq1byCLoApAsqRimx5KcdJn90naY9Ks5TprWrtg6ZTvBwbhM6pHaPEZDgDeNQUV2yyZpe2n9aweKNSZ55oS9pg423qdNcqUDllyPdSBgqX7/kzgjVGzynt7bwAA7nP9F9KXVYcLeaYDP37TZ2n7aXXMD7n9OJt93ZbrXF3dXW0dj637yDVyE2Kvv+7V9e3DB88PMC63/gLQi65LIThIadJ5UrAEPEpQBSDCXsUvNPVOaVJVVoFR0qzlOlu7ve6pBUt3v/+hGTAOQRWAFHsVv9COoqnTZi3X2drSvSQdC9CAMqUjjPoBZYAkCwtY6MPRRTlm25+brcqUDlny/D6lcAvO0ahZRZkSABEeK06if3sUBvU823qdrcqUDllSsDRNXw+ht0uW7v6ZwDjc+gtAIiF1PIqTeMip14fXCBicoAowmJGKXw558eLqnf+rd0t6pqRZ2n52mj3bszCox1nLdfbwWMHS9f9/1PefOgPyCaoA4xmp+OVYSQU4CoNOm6XtJ2nWcp09LN1fj8cMHEmZ0hFG/YAyMKYRil/qgWKV63dQv+b588t3Snr2Og9bn5vRZmn7SZptvU7LMqVDjtnzdOfzqXc8+XkPIxk1qwiqRxj1wQfGNw1YSnQoqHothtP18PebR4LqMd6WLsGwenguP4VbfwHGNlRI/eCDX+69BaCtpSVcQ70GwjkRVAE6dgbFITfFKi9eXNWPfvS3e+9nF0kFP70WBvU4a7lOirv7O1SytPTPXDoD2hBUAfo2enHISMeyRFLBT6+FQT3OWq6T4pTzsOTPHOV8wbB8RvUIo973DfTvseKQeqCUqBM3ZSl7F7/sactCnb1naftJmm29TuJz6pjzUKe/rq1atgZpRs0qguoRRn3wgfFNy4tIdnX79yh6LYZ19fqcOvV17bHfxwq96/W5/Jj39t4AAOuYxmv4XVqiAozpVZ3wWrfgH+w0BsOOBFWAcdz7Fze/0gUYxbHhcYU7Skb6hz/ojjIlgE6s3UipoTRTUutsq8czaT9Js5brpNvr+M7xXEMKQRWgH2s3UmoozZTUOjtSs22Ps5brpNvr+M7xXEMEZUpHGPUDykBfpkcaKeuBJsxDt/4+f365ehPmku9/bJbYULqFLc9h4ixtP0mzrdfp6Tn1lOOrdVrPNQYTb9SsIqgeYdQHH+jXdGJx0gifUfVaDOsa/Tk17dt6roiJZkZ9Lrv1F6BPR4fUDz745Zb7AEi1Z3O4IiZYSFAFCLSwmOOiqj6qqosXL67qRz/62033+hTKSN5IKu7Za5a2n6RZy3VGcfv45rmeXf8O1ZvXxHmu6ZjZ2ntZYwbnRlAFyLSkmKOHAg9lJG8kFfeMXhjU46zlOqNIOjdJe4Hu+IzqEUa97xvINS0rCbkp/0gtS3ns+HovfjnWU87DaLO0/STNtl7Hc+rJr7HHUsREE6NmFe+oAgSa55rnuV7Oc83TVF9Ob0pBXlfVP9Qjf4G6/b1NNvsEh/Z47GwkS87DKLO0/STNWq4zirBzc/OaPU01330df2D25eiPExxDUAXId0opx57lIQCjUMQEOxNUAXa2sEjjbiHIs7QSDoUibySV9CTN0vaTNGu5Dn0WMZ3ytR57eiOoAuxvSZFGDyUcCkXeSCrpSZql7Sdp1nIdss5rq2sEYilTOsKoH1AGMkzLSj0eLOtIKEt57PiOnSUcyxJrnYfRZmn7SZptvU7vz6m1rXVeq2ER09J9M4ZRs4qgeoRRH3wgzzTVl3XC55Oubzm710ivXyMdCyTwnNrGNEWGwFfzXM/23gTbGPW57NZfgCyKkwD6lvjarKCJ7giqAA0tLLg4qjhpz8IMxS8Kg06dpe0nadZyHZa5fV73LmI6Zo9rzGBrgipAW0sKLnoozFD8ojDo1FnafpJmLddhmR4ekx72CDd8RvUIo973DbQ3LSvhOKpc4/asdVnKY/s5h+KXQ8dx39zMuVGmNIa1HpNap4jpPif/DFHE1IdRs4qgeoRRH3xgX9PKxUmHjPT6Ndix3PfYv7q8vFR4QhMjPadGMSli4glGfS6/t/cGoAfTZ9NUVX9UVT+ZP/GvO6xGcdL5uu+xV3gC5+1V5b0OpO2HM+EzqnDH3cKA6bNpqrl+WHP9n5rrh9ehNapwY6RSjx5na3z/AU8uTjphjSfb8xpJsvQaWfJnjjJL20/SrOU67Of2Y3JqEVOrgibXEnsQVOFdN4UB16H0hzVf/KCmmmq++EHVTVhNKtwYqdSjx9ka339XeunFntdIkqXXyJI/c5RZ2n6SZi3XYT+trpH19jhN3/rL+uxPv1P/73/9ZX32pzVN31ppHbjhM6pHGPW+bw67/pfBD+uPP/2i/vizH1bV96vqP936kp9X1Y/r808+rs8//bACCjdazNL2kzR76vfXysVJLctS9rhGEotfnnqNPHQsz59frlp4kj5L20/SbOt1Ep9T52jra6TWKWj69etSTb9fVX83V/3G67q4uKjXr6eqX1XVf615/qcV1uJEo2YVQfUIoz743O/mndRffOO/1ftfvfsFv/hG1T9+v+p///cqd7ywgacUJx0y0uuXY4F1uQ7Pw7RiQdPv1z/V39cf1rP696/dlvm6ql7Vs/qt+vI7wmp7oz6X3foLd9yE1KrvHwypVVXvf1X1nR9X/clfVEUW9NE5xUkArGWVnym/Xf92MKRWvQkU36wvq6r+zm3ArEVQhVtuipNeX/ygvn6777uEVdazanHS2gUXil/u38vSc7P2Oj3O0vaTNGu5DmO5/TifWtBU9xQxfbt+Whf1+t7wcD3/jar69iYHxdkRVOHr/qiq/rwuXv/mUV/9/ldVH/3Pqt/9yba7YnTpZSmKX97YovBk7XV6nKXtJ2nWch3G4hqhez6jeoRR7/vmXb/+VTQXPzgqrPqsKuvYrDxnjbKUY9ddc9+JxS9LjuPuXJmSMqWEc7P3c4rtrHWN1K0ipv9c/1L/t/6gnj18J/GrqvqDmud/Wf+ouM+oWcU7qnDL/Mk811Qf18Xrv6437b73E1JZyTzXPM/18vZfTteetdhfD8eyxNI9H3ssLc5h0ixtP0mzluswli2ukZ/Wt+t1XdxbIXw9/1VV/XTNY+F8Capwx/zJPFfVx1X14/rFNw5/kZDKehQnAZDq5mfUz+pb9Yf19/Vl/dY7YfVt62+9+RU1/9Zyg4xLUIU7pqmm+nT+sD7/5ON6/6u/qnffWf15vf/VX9Wr37momk4qJeh5lrafpNnC71+9OGlJWYril+32fOyxtDiHSbO0/STNWq4Db92+RuY7RUz/XP/l4rfr379zUfWzuerVr+ri53PVq4uqn/nVNKxNUIV3vSkR+PzTD+vtO6uvL/6jqur6vz+uqo+v//eUwo2RSj16nKXtZ0kRRg/noYVW10iLtdNnaftJmrVcB956+Lp5E0Z/79P69M8+qn/410/r0z+rqt8TUlmbMqUjjPoBZQ6b7hQL3BQsVf15Vf2Pmurj+ZN5vvt1h753pFnafpJmafu5PTu1LCX5PLQsftn6GlGm1MfzZ+/Z1usoU+KQpdcs7Y2aVQTVI4z64HO86bNpqje/uuYn159hhS6M9PrlWGBdrkMYw6jP5ff23gD04Dqcfr73PgAA4Bz4jCpAQ3uWsmy9lz2PZYlWe15yLHtdN3ueG7O26wCkEVQB2tqzlGXrvex5LEu02vOSY9nrutnz3Ji1XQcgis+oHmHU+76B9qbGpSwtS3v2PJYlr8UtHpOlx9L6uhmpMKjn2dbrKFOCMYyaVbyjCtDQPNc8z/Xy9l9EW8xa7GXPY1mi1Z6XHMte182e58as7ToAaQRVAAAAogiqADvbqyzlHItf0kpxejyWXq+bHmct1wFII6gC7G+vspRzLH5JK8VZIqn0p4frpsdZy3UAoihTOsKoH1AGMkw7laWcY5nSlvs75dz0fCxbn5u995M023odZUowhlGzindUAXa2V1nKORa/pJXi9HgsvV43Pc5argOQRlAFAAAgiqAKEKhFWcroxS89lOKsLemY066bHmct1wFII6gCZGpRljJ68UsPpThrSzrmtOumx1nLdQCiKFM6wqgfUAZyTQ3KUkYvU9pyL0vPzVYlNknHnHLd9Dzbeh1lSjCGUbOKd1QBArUoSxm9+KWHUpy1JR1z2nXT46zlOgBpBFWAsb06cQ4AsDtBFaATTylLuby8fHZ5eTk9f3558fz55UfPn19eXF5eTpeXl89GKn5JKsA55dy0kHQe9rxuepy1XAcgjaAK0I+kkpe04pek83DKuWkh6Tzsed30OGu5DkAUZUpHGPUDykBfpqCSl0OzrddpWQy19blpWWKTdB72uG56nm29jjIlGMOoWUVQPcKoDz5AT0Z6LR7pWOiX6xDGMOpz2a2/AAAARBFUATqm+CXvPJxybvbSw7kxa7sOQBpBFaBvil/arbvFudlLD+fGrO06AFF8RvUIo973DfRvUvxSVcqUTpV8bvbeT9Js63X2vg6BdYyaVQTVI4z64AP0ZKTX4pGOhX65DmEMoz6X3foLAABAFEEV4AyMXvySVIBzyrlJN9J10+Os5ToAaQRVgPMwevFLUgHOKecm3UjXTY+zlusARPEZ1SOMet83cD6mwYtflCltY4TrpufZ1uv0ch0CDxs1q3hHFeAMzHPN81wvb/8FeO1Zy3X2OL4tzk26ka6bHmct1wFII6gCAAAQRVAFAAAgiqAKcKZGaihNamo95dz0qNfrpsdZy3UA0giqAOdrpIbSpKbWkVp/D+n1uulx1nIdgChaf48wapMWcN7WbhPd4s+8PdP6m6G366bn2dbr9HwdAr82albxjirAmRqpoTSpqfWUc9OjXq+bHmct1wFII6gCAAAQRVAF4EavxS9JBTinnJuRpV03Pc5argOQRlAF4LZei1+SCnBGL1M6Vtp10+Os5ToAUZQpHWHUDygD3NVr8YsypTwp103Ps63XOYfrEM7BqFlFUD3CqA8+QE9Gei0e6ViWuLq6+rKqvrn3PnjXOV2H0LtRf6a49ReAXrw6cU4+IRWAg97bewMA9GefWzgvn22/Rqtbf5ecfQAYn3dUAXgKpTinzR6aAwB3CKoAPMUXVfW96/8+Njd7eA4A3CGoAnCyea55nuvl7Vtb75ubPTwHAN4lqB5HgQcArM/P0UweF+jLkFnFr6cBYDVJBUZJs7tzv78SAB7mHVUA1pRUYJQ0e2gOANzhHVUAVpP0LmbS7O7cO6oA8DBBFQAau7q6uveHr6AKAG79BQAAIIygCkBz01TTNNV3r2+HHX720BwAeJegCsAekoqOlCkBQBifUQWguaSiI2VKAJBHUAWAxpQpAcDD3PoLAABAFEEVgAhJ5UfKlABgX4IqACmSyo+UKQHAjnxGFYAISeVHypQAYF+CKgA0pkwJAB7m1l8AAACiCKoAdCWpJEmZEgBsQ1AFoDdJJUnKlABgAz6jCkBXkkqSlCkBwDYEVQBoTJkSADzMrb8AAABEEVQBGJIyJQDol6AKwKiUKQFAp3xGFYAhKVMCgH55RxWAIc1zzfNcL2+Hxb1mD80BgHcJqgAAAEQRVAE4a8qUACCPoArAuVOmBABhBFUAzt0XVfW96/9uNXtoDgDcIagCcNaUKQFAHkEVAACAKIIqABxBmRIAtCOoAsBxlCkBQCOCKgAcR5kSADQiqALAEZQpAUA7gioAtPfqxDkAnJVpnv3DLgCs5bos6cOq+uL2u6eH5vd9LQCcO++oAsC6TilTUrAEAAd4RxUAVuQdVQBYTlAFAAAgilt/AQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACi/H9tufMPU4+G/wAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (1.4) A* search search: 127.4 path cost, 1,289 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHp1JREFUeJzt3b+PZNlZBuDvtn+wLNoxjpwQIEQG1thCIgIZ/gCE0Eo9gWUcIMz+FewuOfkCIkBogy4JIUFCBFgm9wwQEhAhOTLMCGMNcl+Crhl6u6uqq7rqnvueW88jjcb+tqvPubduV8/bVfX2MI5jAQAAQIqLuTcAAAAAtwmqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAWZRhqGIb62jDU0PMscT8ArQiqAMDSPK2qv1r/3fMscT8ATQzjOM69BwCAk1k/C/i0ql6MY429zhL3A9CKoAoALNYw1Muqem/ufXTm1TjWk7k3AZw3QRUAWKxh8GzgY4yj96YC8/IeVQCgW4qApuG8AnMTVAGAnikCmobzCszKS38BgG49VARUVdczbq9nF6VgCZiRoAoALILipEkpWAKaElQBgEVYVnHSWBX2VlAFS0BLn597AwAA+9jjZb5bXV2tpt/gAjx7drn1v/l9q0BLypQAgF4o+JmX8w80I6gCAL14UVXv12efPd00YxrOP9CMl/4CAF1Yv7T0+bbZ4B2Uk3ro/AOckmdUAYCF89bJfb3zzv9u/W/DUOOdPy8bbg04M1p/AYBF2NX6q7H2vtVqtfV8XV5evj1fziswB8+oAgBxhqGGYaivrVtlD5qd+vPNNWu5zj7na9+P2/e2ALsIqgBAon0bZvdtnT3m8801a7nOXac+rwAH8dJfACDOvr+z887vUb3e8SkvDv18c8+mXufqarX1fD17dvn2fJ3ivAIcSlAFALqzLvJ5b9+P917K+07xHtUNXo1jPTlqYwDlpb8AQJ/2DqlV9WqyXZyHQ87fIfcLwFaCKgAwq1MXAdXNv2++XlUX41jDONaTpJKk3sqUxrGerJ+Rfnte973toesCvCGoAgBzU5zUR5mSgiWgGe9RBQBmpTgpu0xpqvMPsIugCgBEU5w0jX3LlDZRsARMzUt/AYB0ipPyKFgCJiWoAgBNHFsYtMUiipPSypQ2UbAEtCSoAgCtHFsYtO/nXMqs5Tr7ULAENOM9qgBAE48tDKozKE5KK1Pa9B7VFvcTwBuCKgAQQ3FSO8eUKW2iYAk4JS/9BQCSKE7ql4Il4GQEVQDg5I4tDNpiscVJPZQpbaJgCZiKoAoATOHYwqBjPudSZi3XeSwFS8AkvEcVADi5xxYG1ZkWJ/VQprSJgiVgKoLqHlar1bZih1eXl5eKAADgERQnzevUZUqbKFiC6S01q3jp7362fRNVBAAAj6c4afkULMH0FplVBFUA4OROUJRzVsVJvZYpbXLqgqVW+wayCKoAwBSOLcpJKjVSpnSYUxcsHXJ7YCG8R3UPLd7DAQBLsk+5zwPvXzyr4qRey5Q2OXXB0rbPCdxYalb5/NwbAACWZx0mnldtLk0aHvin0+3bn+ts6nVWq7urncZD6z5w378NseuPe7V++fDG8wMsl5f+AgBTO7TQQ3HSsilYAh4kqAIARzmmtOdu0c65FictqUxpk2MKlu7eftcMWA5BFQA4luKkaWYt15laq2sEWAhlSntY6huUAeAUjinPGccakgqMkmZTrzNVmdImra4ROEdLzSrKlADowmq1ulfIs/bq8vLySev98P8eKk465PZmn9VjmdImxxQs3W2Hvl2ydPdzAsvhpb8A9GJb+FG2kuWQ+0NpEm8cei34uoeFE1QBgL0dWWpzrzTpkM95brOW68zhoYKl9f/f6/a7ZkCfBFUA4BDHlNrMWRjU46zlOnNQsARsJagCAId4UVXvr//eNdv3tod8znObtVxnDsfuL/34gCMoUwIA9nZMcdIchUE9z6Zep2WZ0iaHnIct3jYFK1iC5fGMKgDwWIqTmNoh142CJVgQQRUA2OjUxUlzFgb1OGu5Toq7+9tUsnTI7bfNgHyCKgCwzamLk5QpHTZruU6KQ87DvrdPP2Zgg2Ecx4c/6sytVqutJ+ny8tJP5wAa8Fjc3voZqKdV9WL93sHPzOrWewQ3uNh12zezfdY519nU61xdrbbef3N9Te1zHupE1x0sxVK/PypTAliw1Wq1rezm1eXl5ZPW+6Evpy5OUqZ02GzqdeYuU9pkn/Mw7P5nt4IlWAgv/QVYtm3BQukIh1KcRAoFS3AGBFWAjikO4VRaFCcpUzr8XJ9bmdI2t/etYAnOg6AK0DfFIZxKi+IkZUqHzVquk67F9QkEEVQB+vaiqt5f/71rBg855lra97bbPt8xt1/yrOU66Vpcn0AQQRWgY+NY4zjW89tNlptm8JDb180w1MthqLFuimm+X7tbVve+Drddm8fcfsmzluukO/JY3l7Hw3Bzffd6HuCcCKoAwF2Kk+iNgiVYGEEVoGNKQniMuYqTlCllnptetShYcq5hPoIqQN+UhPAYcxUnKVM6bNZynR4lXcdLP9fQ3DCOXpr/kNVqtfUkXV5e+ikZMJv1T+qfVtWLN++1uj27ulptfW9hb49fHotP56Hrpna/J/Vi120PnR17+yXPpl6n98eHpOv4kPsUTm2p3x8/P/cGAHi89T9+nm+brVZz7Ip0t6+RYaiXdcB79h665g6dTfE5lzKbep3eHx8eOr5h9z/P34bYTR93gtmrcawnd/cH7M9LfwHgvClOYqnmvF4VNsGRBFWATijw6FtSSc8B18hJi5PmLAzqcdZynaW4fXyHFixNuZepZrBkgipAPxR49C2ppCepcKbVOj3OWq6zFEnH7L6DIyhT2sNS36AM9GVTMcdDs97LUm7r/bH4Mfdfi1nNXDiTch4SZ1Ovs6THhzeOvN5PrcnXD/T+/XEbZUoAnTjHspQleajAaJim0OXB2S5LKAzqeTb1Okt8fDiyYOnUTlnYpJyJs+OlvwDQXg9FK4qTWKJer+seHjPgpARVgEAKN/p1SClOmMmLk+YsDOpx1nKdJbt9zJsKltbX+8lmLY7joblrhCUQVAEyKdzo1yGlOEmWXhjU46zlOks25/13Sq4RzooypT0s9Q3KQK5TFW4sqSyll8fibYUnt+fVttBlX5MXv+xzbuYuMEqaTb3Okh4fdml9/9V0X9/3vkZbHR/Zevn+eChlSgCBlKX06+59MmwoTkq01MKgnmdTr3Mujw+t779huliws5xp2/zAmdImYgiqsIfh42Goqm9U1XfHD70MAThIfEitfgtmINGr6uPrfpNe980CeY8q3HG3RGD4eBhqrE9qrL+vsT5Zh9aowo0llXr0OEvbz7Y9plvKsRyy56ur1b0/NWHJy45Zk+KkXr9+ln5uON7t83rqwqY5j+XQGZySoAr3vS0RWIfST2q8+FYNNdR48a2qt2E1qXBjSaUePc7S9tNrEcZSjuXYPSddN75+5p21XIfjLOl+eng/w/DlP6yPf/ur9c9/84f18W/XMHy5/TZZOmVKe1jqG5TZbP2Twaf1Gx+9qN/4+JOq+mZV/cytD/nvqvq0/vHDD+ofP3paAYUbLWZp+0mape3n9qynspSlHMvd4xiG7WUk62dQP+PZs8tZSo18/eTNpl6nl6+pHkx5P1X78rXdj0E1/FJVfW+s+tx1XVxc1PX1UPWTqvr1Gsd/bbxXarlZxXtU4Y5xrHH4eHhRVZ/U63e/U1/80d0P+Zl6/e536r3/+E7VWFVDDRseApY8S9tP0mzCdTYWXCypLGUpx7KtFOeY2y95lrafpNnU6/TyNdWDKe+nbd9rJrS1tOmX6l/rP+tL9aT+qy6q6nPrD72uqlf15F++NAxfFVY5FS/9hTvevty36psbQuqNL/6o6qufVv3WH1Rtf7IETknBBcB5iig7+9n6Yf1T/drbkHrbRVW9Vy+rqr7nZcCciqAKt7wtTrq++FZ99uW+9wmrNLb0gosej6XHPVe1Keg55Nwk7Sdp1nIdsty+n3aVM7UsbfpK/aAu6nrrB67nn6uqrxxx6PCWoAqf9Y2q+v26uP7pvT76iz+q+vqfV/38d6fdFdzooYTjGD0eS497rjrPwqAeZy3XIcuc1whEEFThs75bVX9W1xf/s9dHv3636vu/V/Xv35h2V3DjRVW9v/770FkPejyWHvdcddy1dOpZ2n6SZi3XIcuc1whEEFThlvHDcayhPqiL67+sm3bf7V6/W/Uv36z62z+p8goqGhjHGsexnt9uBN131oMej6XHPVcddy2depa2n6RZy3XIMuc1ss0P6it1XRdbK4jX859U1Q/2PU7YRVCFO8YPx7GqPqiqT+v1u5s/SEhlBsNw86tObv15OfeeOIacAETZWdr0n/Xl+rX6p3pZX7oXVtetv1U3v6Lmh1NtkPPi19PABuOH4zh8PHywbv29/3tUv/ijT+tX/uyD8W/+1L80mdyw/fdwagLumh9yATnGm1+B9oBfrhr+65er6nt1U5xUVVUXVT/5Ur30e1Q5Kc+owh1vmvbqo7HqzTOr4/plwDd/f1pVH9RHY6U0Qy6pfbLHWct17uq11XMpDaU97nmbpX/99DhruQ7n6VHXzU0Y/YXX9YVf/bA++t3X9YVfrapfEFI5NUEV7nvbgPf2ZcAvf+7v6nqoevlzf1dVH6znSc2QS2qf7HHWcp27em1xXEojZY973mbpXz89zlquw3l63HUzjj/8qXr9zh/Vh3/8U/X6HS/3ZQrDOHrl4kNWq9XWk3R5eeknkguz/gni06p68aZcYPjNj4b68r/9Xv3wF/98/Iebp1o3ftyCZ2n7SZpNvU7V1u6KqpsfOG697dXVautt53z8esx5SDyWfY5j2P7S7bq6Wt2bJR/LFLM5106fTb1O4tcUbU1xzdLeUrOKoLqHpd75QB92BZ0NXt1+n9GSHr96PZZegirnp9evKeCzlvq17KW/APl2NjHeoWAJAOieoAoQ6HZxxTjWk3GsoW4es79eDzx291CW0mPxS497PlarIp8W6/Q4a7kOQBpBFSDTMSUoPZSl9Fj80uOej7WkwqAeZy3XAYgiqAJkelFV76//3jXb97Zp9j2+pGPpcc/HOuaYDzk3LdbpcdZyHYAogipAoHGscRzr+e0mxU2zLa6r6vtVdf3s2WV9+9u/M+leH2Pf4zvgmCfX456PdcwxH3JuWqzT46zlOgBpBFWAPu1dsPTjH39hyn0AAJycoArQiWMKlubUY/FLj3uewpIKg3qctVwHIE3sP2wAuKfXYpQei1963PMUllQY1OOs5ToAUQRVgH70WozSY/FLj3uewpIKg3qctVwHIIqgCtCJXotReix+6XHPU1hSYVCPs5brAKQRVAEAAIgiqAJ07JhiFMUv2/W451Z6vW56nLVcByCNoArQt2OKURS/bNfjnlvp9brpcdZyHYAogipA344pRlH8sl2Pe26l1+umx1nLdQCifH7uDQDweOtClOdVVcOOF/I9e3a5aXz95n9suu0xswc+9tU41pNa7/uN28eyabZabV5nCg/tZdNsGOplVb13+7/vuk969Zhzs2s2xedcymzqdVp+TQEcyjOqAMvxau4N7Om9hz+kSwcd1zvv/O9U+wCA7gmqAB27XYwyjvVkHGuom8f2r1fwY3xS8UuLYpq798vV1ar+4i/+eoKjmUcPhUE9zlquA5Am9h8xAOyl17KUpOKXVsU0Pdwvj9VDYVCPs5brAEQRVAH61mtZSlLxS6timh7ul8fqoTCox1nLdQCiCKoAHRvHGsexnq8LUrbOAl1X1fer6noYahyGejnXsey77u3ZMNTLYajx9nE8Zp2leMw5PNXtlzxruQ5AGkEVYNkULE3jkP32ch8AQAxBFWBhHipYGscappjt87H77nvX7NQmKKG5ew6eHHj77vRQGNTjrOU6AGkEVYDl6aH45Zh9n9qpS2gOOTdL0UNhUI+zlusARBnG0VsUHrJarbaepMvLSz+RBKKsnyl5WlUv3rwPrcVsn4+t3e/lvNh126ur1dbbHvNY/JhjPvQ4Wh3LXKa+bs51NvU6S7sO4VwtNat4RhVgYXooftlir4KlU3toz4cWJx1ybpaih8KgHmct1wFII6gC0NIhxUIpBUuKkwCgMUEVgEk9VO60721PvZdDZ1vcK046pBRnydIKg3qctVwHII2gCsDUpigmmnovUxRAnVuJTVphUI+zlusARBFUAZjai6p6f/33rtm+t22xl2P2t+22pz6WdMeeG7O26wBEEVQBmNSpCpaePbusb3/7dybdyymKk86xTGmTtMKgHmct1wFII6gCMLe9C4h+/OMvTLmPKsVJABBBUAWguWMKlk617q7ZFnsVJylT2k6Z0mGzlusApBFUAZjDXCUvLYqTlCltp0zpsFnLdQCiCKoAzGGukpcWxUnKlLZTpnTYrOU6AFEEVQCam6vkpUVxkjKl7ZQpHTZruQ5AGkEVgHOlOAkAQgmqAEQ4puRlgiKZRxcnKVM6XFKBUdKs5ToAaQRVAFIcU/Jy6iKZVqU43EgqMEqatVwHIIqgCkCKY0peTl0k06oUhxtJBUZJs5brAEQZxtF76R+yWq22nqTLy0svnQE4sXXJ0WzWv9d1Mr6v3FitVi/rsPcK08g5XYfQu6V+T/GMKgCJ5iwvUpzUjpAKwEaCKgARbpe8jGM9WT+r+bbUaMKlT1qcpEwJAI4nqAKQYq7ilzlLcQCADQRVAFLMVfwyZykOALCBoApAhHGscRzr+Tj+f5HSptlc6x4za3UsALAUgup+thVrKNwAaGeKx1yP4/Ny/jO5X6Avi8wqfj0NALHWxUNPq+rFm2cie5zdnV9dra63HXPPv0oAAE7FM6oAJGtRdKRMCQDCeEYVgFhJz4p6RhUA2hFUAaCx1Wq19ZuvoAoAXvoLAABAGEEVgLMxDDUMQ31t/TLcZrNdcwDgPkEVgHOiTAkAOuA9qgCcDWVKANAHz6gCcDbGscZxrOe3A2SL2a45AHCfoAoAAEAUQRWAs6ZMCQDyCKoAnDtlSgAQRlAF4Ny9qKr3139PNds1BwDuEFQBOGvKlAAgj6AKAABAFEEVAO5QpgQA8xJUAeA+ZUoAMCNBFQDuU6YEADMSVAHgDmVKADAvQRUAAIAogioAAABRBFUA2IPWXwBoR1AFgP1o/QWARgRVANiP1l8AaERQBYA9aP0FgHYEVQAAAKIIqgDwSMqUAGAagioAPJ4yJQCYgKAKAI+nTAkAJiCoAsAjKVMCgGkIqgAAAEQRVAHghJQpAcDxBFUAOC1lSgBwJEEVAE5LmRIAHElQBYATUqYEAMcTVAGgvVcHzgHgrAzj6Ae7AAAA5PCMKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAFEEVAACAKIIqAAAAUQRVAAAAogiqAAAARBFUAQAAiCKoAgAAEEVQBQAAIIqgCgAAQBRBFQAAgCiCKgAAAFEEVQAAAKIIqgAAAEQRVAEAAIgiqAIAABBFUAUAACCKoAoAAEAUQRUAAIAogioAAABRBFUAAACiCKoAAABEEVQBAACIIqgCAAAQRVAFAAAgiqAKAABAlP8DscRvdl+TaAMAAAAASUVORK5CYII=\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " (b) Weighted (2) A* search search: 140.4 path cost, 982 states reached\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAJCCAYAAADJHDpFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAG55JREFUeJzt3c+OZHd9xuHvaYw1MfIYVpayilhFCpbxnghxASyIpZ6FE7xIIL4HFh4vuAcCYmFFjjSNIha5ARDsPRZEyjJbbwLMyI5lizlZdM2kpqeq+9TU+fP+Tj2PZBmOq/ucru5q+zPV9XbX930BAABAirOlLwAAAAC2CVUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAYFW6rrquq292XXUtH0u8HoC5CFUAYG1er6p/3/y95WOJ1wMwi67v+6WvAQBgNJtnAV+vqo/6vvpWjyVeD8BchCoAsFpdVw+q6uWlr6MxD/u+bi99EcBpE6oAwGp1nWcDn0ffe20qsCyvUQUAmmUIaBruV2BpQhUAaJkhoGm4X4FF+dFfAKBZNw0BVdWjBS+vZWdlYAlYkFAFAFbBcNKkDCwBsxKqAMAqrGs4qa8KeymogSVgTi8sfQEAAM/ryo/57nXv3sU8F9S4O3fO9/4zv28VmJMxJQCgZQZ+5mNgCZiNUAUAWvZRVb1ZNzyjyih23dfuf2ASfvQXAGjW5sdN71dVdV5BOant+/q6YwBj8IwqALByXjo51K1bX+z9Z11X/ZW/Hsx4acCJsfoLAKzCdau/FmufdXFxsff+Oj8/f3J/uV+BJXhGFQBoVtdV13X1zc367EG3Sz8253mG3F9Dbzf0bQGuI1QBgJYNXZ0dulibdGzO81w19v0KcBA/+gsANOvK71F9dM1Nzx7f7rrfAZp0bOrz3Lt3sff+unPn/Mn9Ncb9CnAooQoArILXUh5mjNeo7vCw7+v2URcGUH70FwCA6z084LYvT3YVwEkRqgBAs4wpjXOe6+6vvq/bm2ekz6rqjbrhvx8NLAFjEKoAQMuMKY1znquWeluAqvIaVQCgYcaUph9T2vW2Y9zXANcRqgDAKhhTOszQMaVdDCwBU/OjvwAAHMrAEjApoQoANGGscaBWjs15niEMLAFzEqoAQCvmGPhJOjbneYYwsATMxmtUAYAm3DQOVMaURhtT2vUa1Tnuf4DHhCoAsArGlA5zzJjSLgaWgDH50V8AAMZgYAkYjVAFAJpgTGm68zwvA0vAVIQqANAKY0rTned5GVgCJuE1qgBAE4wpLTumtIuBJWAqQnWAi4uLB7X7tRQPz8/PDQEAQABjSocZe0xpFwNLML21toof/R1m3wv+DQEAAOxnYAmmt8pWEaoAxFpqIIZMxpSmO8+Yxh5Ymuu6gSxCFYBkSw3EkMmY0nTnGdPYn6dD3h5YCa9RHWCO13AA8Kxjh2hYF2NKeWNKu4z9edr3PoFLa22VF5a+AADYZ/MfpPerqrqunhmL6C7/9ft4gOX+3NfHvLa/HnYd6675z7Gb3jbx2NTnubi4erZxHPN5qq2I3X5877t/gPXyo78AtGKVYxFwggwsATcSqgBEmGMoh7YZU5ruPFM7ZmDp6ttfdwxYD6EKQIo5hnJomzGl6c4ztWOvJeljAWZgTGmAtb5AGSDJHEM5tM2YUhtjSrsc87nr++qMqMF+a20VY0oARLhpOOkGOwdYyvjKqhhTGvfYVGNKuxzzueu6pyPUYxxOgx/9BSDRsQMqBligLYcMLFV5jMPqCVUAIgwdRrl37+LJX4e+P+MrbTOmNN15lnDTwNLm/w96++uOAW0SqgCkGHsYxfjK+hhTmu48SzCwBOwlVAFI8VFVvbn5+1Tvb+xzMK9jPqdD3zbp2JznWcKx15f+8QFHMKYEQIShwyoHMLC0MsaUxj0255jSLofcD3t4jMOKeUYVgGbduvXFITc3vgLtOWRkyWMcVkSoAhDheUZQ3n//l49HlZ4MsBx6DuMr7TCmNN15Uly9vl0jS4e8/b5jQD6hCkCKY0ZQxh7UIZMxpenOk+KQ+2Ho26d/zMAOQhWAFMeMoIw9qEMmY0rTnSfFIffD0LdP/5iBHYwpAazYxcXFg9r9uq2H5+fnt+e+nuscM6Z0wNsaX2mYMaVxjy09prTLkPvBYxxOg2dUAdZt37jImkdHjK/AunmMwwkQqgBEOGbwZPttja+sw9jjQEkjScaUDucxDqdHqAKQYuwxJeMrbRt7HChpJMmY0uE8xuHECFUAUow9pmR8pW1jjwMljSQZUzqcxzicGGNKAEQYa0zpOd6f8ZVAzzMOZEyp/TGlfTzG4fR4RhWAU2B8BdbNYxxWRqgCNGxNIyFjX7fxlbYZU5r3PC3yGId1E6oAbVvTSMjY1218pW3GlOY9T4s8xmHFhCpA29Y0EjL2dRtfaZsxpXnP0yKPcVgxoQrQsL6vvu/r/mZUZO+xFox93UfeN4+q6sOqetR11XddPWj1fm3V0M/f0M/LMe9vqWNznqdFHuOwbkIVgFNlfAXWzWMcGiZUARq2pkGQOa57yfGVUxy7OYYxpXnPsxYGlmA9hCpA29Y0CDLHdS85vnKKYzfHMKY073nWwsASrIRQBWjbmgZB5rjuJcdXTnHs5hjGlOY9z1oYWIKVEKoADVvTWMoc173k+Mr2bbuuHnRd9ce+zzW76XO16z485v0lHpvzPGthYAnWQ6gCwP+ba3xl39sadBnukPvqkM8r62ZgCRohVAEatqbxj6Wue67xlbFHf9buyPvhyeev7y8/r0kjScaU5mVgCdokVAHatqbxj6Wue67xFYMuhzm14SRjStMxsAQNEqoAbVvT+MdS1z3X+IpBl8Oc2nCSMaXpGFiCBglVgIataSxlqeueenzl0NGfIe/zFJzacJIxpekYWII2CVUAuN6x4yvHDrIYdDGcxLQMLEEgoQpX7BxO+M7drvu7v/+n7jt3Fx/XWPuoR4vH0q5njqGPue6HpWxfyxjjK/vcu3fx5K9D3+dajh162x1WMZy05PeWUzT2Y3ztXyO+lliCUIVnPTWS0L3XdfXGz39Rr/3bz+qNn/+ie6/rdt3uBI6lXU/SsbTrmWPoY677YSljj6/MdT0tHjv0tlclfSytfm85RWv5mku8HhhF1/d+vP4mFxcXe++k8/Nzf4q0Mps/GXy9qj6qyydQf1J9vVVdfaX6+qS6+qCq3qm7fT2+3dZr0bq1Hku7nqRjadezfezevYu9r+U75vvX1PdDXfMaxF3PPo79vfiY66vLPwR+8rZdt/91bNsfy50759dd0lPvc8g1tnJsyG3rgPs77eNL/t4y1feHFqzla26xc3fd1z6vL7/64/rRX/+ofvxfL9YXH1ff/+GQzwHjWWurCNUB1vrJ53qbZ05/UlVvVdVXtv7RJ1WXsdq/6wFEtla/fw2Nu8fm/liuu77NjxAOuu3QUL36Pk/NIfc3w7X6/WEOvuau0XXfqKrfVNWXto7+uar+tvr+98tc1Glb62P5haUvABI9idTPX/phvfjp1X/8lfr8pR/W7976Ydf1VSf+7yviPOz7uj33SbuuHtQMIyO3bn0x9SmGelh7Pt7r/gN329WP5datL+qzz76887ZD3+cJMpzEVI5+jK/R39Tv67f1St2uPz31+sFHVfWwbv/ula57TawyFqEKVzz1TOqzkXrpxU+rXvvg8n//x7+UWCXIUouUk5x3+5mL6/7EeG67/jDgkGdgdn0s77//y6p6+k+/T/k/iHc5+WeymM2hj/FT8NX6Q/22vvVMpFZd/iz0y/Wgquo31XVf92PAjMGYEmzp3uu66usn9ejsH+rpH/d91uNY/e4/V532v7sIM/YS45Jrj0mrksfeD0Nvm/Qxpxn6OVjLsTnPg/vrJq/Wx3VWj/bGw+b4l6rq1bmuiXUTqvC0b1fVD+rs0V8MuvWLn1a98fOqv/r1tFcFhxl7iXHJtcekVclj74eht036mNMkrZvOcWzO8+D+gihCFZ7266r6WT06+99Bt/78paoP/7Hqv7897VXBYT6qqjc3f5/q/Y19jkPOvZRj74eht036mNMM/Rys5dic58H9BVGEKmzp3+376uqdOnv0r3W57rvf5y9V/e4tr1El0aOq+rCqHt25c15vv/29o95Z31ff93V/86tWHmxep/XkHFO+bmv73FOd45hrOeT6ht426WNOM/RzsJZjc54H99dNPq5X61Gd7f29PZvjf66qj+e6JtZNqMIVm185805VfVCfv7T7RiKVhuxbkn1Oc441tbbouu96j/k4WrsPpuS+YGkn/TX4x/pafat+Ww/qlWdidbP6W3X5K2oMKTEKoQpXdF11dbd/vX717jv14qc/rWefWf2kXvz0p/XwL8+qujeq6qzvq9usUZ5V1SqPpV1P0rGU69nx5XywY8ZEJvj4bqcPmWxfX9/X7X0fx9XbjvA+V3Ps2K+HNR+b8zzslv54nPvc/1nfOPtq/em1s6o/9lUP/1xnn/RVD8+q/vhKPfCraRiVUIVnXQ4n/Oru6/X4mdXHr1m9/PsHVfXO5p+nDG6sadSjxWOJ1/O8phgMSvr4xnbs18gx73Mtx9KuJ+nYnOdht6Svh4zHz2WMfv1u3f3+G/Xh/9ytu9+vqq+LVMbW9f3J/8j9ja773X3bv++Oddj8KfPrVfVR31f/5FfWVP2gqn5WXb3Tv9v3V2+3623XdCztepKOpVxP1d6XDtW9exfPHNv1/euYc/R9dVN+fPfuXew991Lfi5/3a+S6j+XOnfOzIe9zLcfSrifp2NTnSXxMpUn6emjh8cMy1toqQnWAtX7yGa57r+vq8lfX/HrzGlaI010zajQ0VK+8vwd1wGtSNz8eNpk1fS9e08dCu3wdwjqs9bH8wtIXAC3YxOmvlr4OmNkhw0knPTICAIzLa1QBGnbMMMqRQytHDQZNcD2szNiDQS0em/M8AGmEKkDbjhlGOWZo5ZBRjzmuh/VZapwm6dic5wGIIlQB2vZRVb25+fsYbzv0/e273VLXw/oM/XpY87E5zwMQRagCNKzvq+/7uv88i4vbb9t19WAzxvSoqj6sa9Z9rzvvWNczxvujbUO/HtZ8bM7zAKQRqgBUGU4CAIIIVYCGjT1etMczw0lTDLIYfmFb0qiRMSWA+QlVgLaNPV409HZTDLIYfmFb0qiRMSWAmQlVgLaNPV409HZTDLIYfmFb0qiRMSWAmb2w9AUA8Pw2gyj3q6q6gT/I9/bb36vPPvty1Q2DSbvOcd2xY910nouLMc9GuqFfd2s+NvV5PKaAZJ5RBTgxm0gdynASADA7oQrQsAmGUWYZTtrF8AvbkkaNjCkBzE+oArRt7GGUJcdXDL+wLWnUyJgSwMyEKkDbxh5GWXJ8xfAL25JGjYwpAczMmBJAw4aOKd25c37w+7vu2BQMv7AtadTImBLA/DyjCrAexw4fGU4CACIIVYCGbQ+j9H3d7vvqamsQ6YY3X2w4aRfDL2xLGjUypgQwP6EK0LZjxlLShlbSrodlJY0aGVMCmJlQBWjbMWMpaUMradfDspJGjYwpAcxMqAI0rO+r7/u6vxlI2Xts7LedQtr1sKyhXw9rPjbneQDSCFWAdds3kGQ4CQCIJVQBVuamgaWE4aRdDL9wk6ShI2NKANMSqgDr0+qoSgvXyLKSho6MKQFMSKgCrE+royotXCPLSho6MqYEMCGhCrAyrY6qtHCNLCtp6MiYEsC0hCoAAABRhCoAEQy/cJOkoSNjSgDTEqoApDD8wk2Sho6MKQFMSKgCkMLwCzdJGjoypgQwIaEKQATDL9wkaejImBLAtIQqAAAAUYQqANCEpKEjY0oA0xKqAEArkoaOjCkBTEioAgCtSBo6MqYEMCGhCgA0IWnoyJgSwLSEKgAAAFGEKgDQhKShI2NKANMSqgBAK5KGjowpAUxIqAIArUgaOjKmBDChru+9lv4mFxcXe++k8/NzPzoDMIM1fS9e08dyjIuLiwdV9fLS18GzTunrEFq31n+neEYVgFY8PPA4+UQqADu9sPQFAMAQ5+fnt5e+BgBgHp5RBQAAIIpQBQAAIIpQBQAAIIpQHcaABwCMz79HM/m8QFtW2Sp+PQ0AzGytv0oAAMbiGVUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAAACiCFUAmN/DA48DwEnp+r5f+hoAAADgCc+oAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEEWoAgAAEOX/AIS6zUzgLYR1AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + " Greedy best-first search search: 151.6 path cost, 826 states reached\n" + ] + } + ], + "source": [ + "plots(d7)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Nondeterministic Actions\n", + "\n", + "To handle problems with nondeterministic problems, we'll replace the `result` method with `results`, which returns a collection of possible result states. We'll represent the solution to a problem not with a `Node`, but with a plan that consist of two types of component: sequences of actions, like `['forward', 'suck']`, and condition actions, like\n", + "`{5: ['forward', 'suck'], 7: []}`, which says that if we end up in state 5, then do `['forward', 'suck']`, but if we end up in state 7, then do the empty sequence of actions." + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [], + "source": [ + "def and_or_search(problem):\n", + " \"Find a plan for a problem that has nondterministic actions.\"\n", + " return or_search(problem, problem.initial, [])\n", + " \n", + "def or_search(problem, state, path):\n", + " \"Find a sequence of actions to reach goal from state, without repeating states on path.\"\n", + " if problem.is_goal(state): return []\n", + " if state in path: return failure # check for loops\n", + " for action in problem.actions(state):\n", + " plan = and_search(problem, problem.results(state, action), [state] + path)\n", + " if plan != failure:\n", + " return [action] + plan\n", + " return failure\n", + "\n", + "def and_search(problem, states, path):\n", + " \"Plan for each of the possible states we might end up in.\"\n", + " if len(states) == 1: \n", + " return or_search(problem, next(iter(states)), path)\n", + " plan = {}\n", + " for s in states:\n", + " plan[s] = or_search(problem, s, path)\n", + " if plan[s] == failure: return failure\n", + " return [plan]" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [], + "source": [ + "class MultiGoalProblem(Problem):\n", + " \"\"\"A version of `Problem` with a colllection of `goals` instead of one `goal`.\"\"\"\n", + " \n", + " def __init__(self, initial=None, goals=(), **kwds): \n", + " self.__dict__.update(initial=initial, goals=goals, **kwds)\n", + " \n", + " def is_goal(self, state): return state in self.goals\n", + " \n", + "class ErraticVacuum(MultiGoalProblem):\n", + " \"\"\"In this 2-location vacuum problem, the suck action in a dirty square will either clean up that square,\n", + " or clean up both squares. A suck action in a clean square will either do nothing, or\n", + " will deposit dirt in that square. Forward and backward actions are deterministic.\"\"\"\n", + " \n", + " def actions(self, state): \n", + " return ['suck', 'forward', 'backward']\n", + " \n", + " def results(self, state, action): return self.table[action][state]\n", + " \n", + " table = {'suck':{1:{5,7}, 2:{4,8}, 3:{7}, 4:{2,4}, 5:{1,5}, 6:{8}, 7:{3,7}, 8:{6,8}},\n", + " 'forward': {1:{2}, 2:{2}, 3:{4}, 4:{4}, 5:{6}, 6:{6}, 7:{8}, 8:{8}},\n", + " 'backward': {1:{1}, 2:{1}, 3:{3}, 4:{3}, 5:{5}, 6:{5}, 7:{7}, 8:{7}}}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's find a plan to get from state 1 to the goal of no dirt (states 7 or 8):" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['suck', {5: ['forward', 'suck'], 7: []}]" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "and_or_search(ErraticVacuum(1, {7, 8}))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This plan says \"First suck, and if we end up in state 5, go forward and suck again; if we end up in state 7, do nothing because that is a goal.\"\n", + "\n", + "Here are the plans to get to a goal state starting from any one of the 8 states:" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{1: ['suck', {5: ['forward', 'suck'], 7: []}],\n", + " 2: ['suck', {8: [], 4: ['backward', 'suck']}],\n", + " 3: ['suck'],\n", + " 4: ['backward', 'suck'],\n", + " 5: ['forward', 'suck'],\n", + " 6: ['suck'],\n", + " 7: [],\n", + " 8: []}" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{s: and_or_search(ErraticVacuum(s, {7,8})) \n", + " for s in range(1, 9)}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Comparing Algorithms on EightPuzzle Problems of Different Lengths" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [], + "source": [ + "from functools import lru_cache\n", + "\n", + "def build_table(table, depth, state, problem):\n", + " if depth > 0 and state not in table:\n", + " problem.initial = state\n", + " table[state] = len(astar_search(problem))\n", + " for a in problem.actions(state):\n", + " build_table(table, depth - 1, problem.result(state, a), problem)\n", + " return table\n", + "\n", + "def invert_table(table):\n", + " result = defaultdict(list)\n", + " for key, val in table.items():\n", + " result[val].append(key)\n", + " return result\n", + "\n", + "goal = (0, 1, 2, 3, 4, 5, 6, 7, 8)\n", + "table8 = invert_table(build_table({}, 25, goal, EightPuzzle(goal)))" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "2.6724" + ] + }, + "execution_count": 78, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def report8(table8, M, Ds=range(2, 25, 2), searchers=(breadth_first_search, astar_misplaced_tiles, astar_search)):\n", + " \"Make a table of average nodes generated and effective branching factor\"\n", + " for d in Ds:\n", + " line = [d]\n", + " N = min(M, len(table8[d]))\n", + " states = random.sample(table8[d], N)\n", + " for searcher in searchers:\n", + " nodes = 0\n", + " for s in states:\n", + " problem = CountCalls(EightPuzzle(s))\n", + " searcher(problem)\n", + " nodes += problem._counts['result']\n", + " nodes = int(round(nodes/N))\n", + " line.append(nodes)\n", + " line.extend([ebf(d, n) for n in line[1:]])\n", + " print('{:2} & {:6} & {:5} & {:5} && {:.2f} & {:.2f} & {:.2f}'\n", + " .format(*line))\n", + "\n", + " \n", + "def ebf(d, N, possible_bs=[b/100 for b in range(100, 300)]):\n", + " \"Effective Branching Factor\"\n", + " return min(possible_bs, key=lambda b: abs(N - sum(b**i for i in range(1, d+1))))\n", + "\n", + "def edepth_reduction(d, N, b=2.67):\n", + " \n", + " \n", + "\n", + "from statistics import mean \n", + "\n", + "def random_state():\n", + " x = list(range(9))\n", + " random.shuffle(x)\n", + " return tuple(x)\n", + "\n", + "meanbf = mean(len(e3.actions(random_state())) for _ in range(10000))\n", + "meanbf" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{0: 1,\n", + " 1: 2,\n", + " 2: 4,\n", + " 3: 8,\n", + " 4: 16,\n", + " 5: 20,\n", + " 6: 36,\n", + " 7: 60,\n", + " 8: 87,\n", + " 9: 123,\n", + " 10: 175,\n", + " 11: 280,\n", + " 12: 397,\n", + " 13: 656,\n", + " 14: 898,\n", + " 15: 1452,\n", + " 16: 1670,\n", + " 17: 2677,\n", + " 18: 2699,\n", + " 19: 4015,\n", + " 20: 3472,\n", + " 21: 4672,\n", + " 22: 3311,\n", + " 23: 3898,\n", + " 24: 1945,\n", + " 25: 1796,\n", + " 26: 621,\n", + " 27: 368,\n", + " 28: 63,\n", + " 29: 19,\n", + " 30: 0}" + ] + }, + "execution_count": 72, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "{n: len(v) for (n, v) in table30.items()}" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 24min 7s, sys: 11.6 s, total: 24min 19s\n", + "Wall time: 24min 44s\n" + ] + } + ], + "source": [ + "%time table30 = invert_table(build_table({}, 30, goal, EightPuzzle(goal)))" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " 2 & 5 & 6 & 6 && 1.79 & 2.00 & 2.00\n", + " 4 & 33 & 12 & 12 && 2.06 & 1.49 & 1.49\n", + " 6 & 128 & 24 & 19 && 2.01 & 1.42 & 1.34\n", + " 8 & 368 & 48 & 31 && 1.91 & 1.40 & 1.30\n", + "10 & 1033 & 116 & 48 && 1.85 & 1.43 & 1.27\n", + "12 & 2672 & 279 & 84 && 1.80 & 1.45 & 1.28\n", + "14 & 6783 & 678 & 174 && 1.77 & 1.47 & 1.31\n", + "16 & 17270 & 1683 & 364 && 1.74 & 1.48 & 1.32\n", + "18 & 41558 & 4102 & 751 && 1.72 & 1.49 & 1.34\n", + "20 & 91493 & 9905 & 1318 && 1.69 & 1.50 & 1.34\n", + "22 & 175921 & 22955 & 2548 && 1.66 & 1.50 & 1.34\n", + "24 & 290082 & 53039 & 5733 && 1.62 & 1.50 & 1.36\n", + "CPU times: user 6min, sys: 3.63 s, total: 6min 4s\n", + "Wall time: 6min 13s\n" + ] + } + ], + "source": [ + "%time report8(table30, 20, range(26, 31, 2))" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "26 & 395355 & 110372 & 10080 && 1.58 & 1.50 & 1.35\n", + "28 & 463234 & 202565 & 22055 && 1.53 & 1.49 & 1.36\n" + ] + }, + { + "ename": "ZeroDivisionError", + "evalue": "division by zero", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n", + "\u001b[0;32m\u001b[0m in \u001b[0;36mreport8\u001b[0;34m(table8, M, Ds, searchers)\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0msearcher\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mproblem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0mnodes\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mproblem\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_counts\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'result'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mnodes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mround\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mN\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 14\u001b[0m \u001b[0mline\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnodes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 15\u001b[0m \u001b[0mline\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mebf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mn\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mline\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mZeroDivisionError\u001b[0m: division by zero" + ] + } + ], + "source": [ + "%time report8(table30, 20, range(26, 31, 2))" + ] + }, + { + "cell_type": "code", + "execution_count": 315, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0 116 116 ['A']\n", + "140 0 140 ['A', 'S']\n", + "0 83 83 ['A']\n", + "118 0 118 ['A', 'T']\n", + "0 45 45 ['A']\n", + "75 0 75 ['A', 'Z']\n", + "0 176 176 ['B']\n", + "101 92 193 ['B', 'P']\n", + "211 0 211 ['B', 'F']\n", + "0 77 77 ['B']\n", + "90 0 90 ['B', 'G']\n", + "0 100 100 ['B']\n", + "101 0 101 ['B', 'P']\n", + "0 80 80 ['B']\n", + "85 0 85 ['B', 'U']\n", + "0 87 87 ['C']\n", + "120 0 120 ['C', 'D']\n", + "0 109 109 ['C']\n", + "138 0 138 ['C', 'P']\n", + "0 128 128 ['C']\n", + "146 0 146 ['C', 'R']\n", + "0 47 47 ['D']\n", + "75 0 75 ['D', 'M']\n", + "0 62 62 ['E']\n", + "86 0 86 ['E', 'H']\n", + "0 98 98 ['F']\n", + "99 0 99 ['F', 'S']\n", + "0 77 77 ['H']\n", + "98 0 98 ['H', 'U']\n", + "0 85 85 ['I']\n", + "87 0 87 ['I', 'N']\n", + "0 78 78 ['I']\n", + "92 0 92 ['I', 'V']\n", + "0 36 36 ['L']\n", + "70 0 70 ['L', 'M']\n", + "0 86 86 ['L']\n", + "111 0 111 ['L', 'T']\n", + "0 136 136 ['O']\n", + "151 0 151 ['O', 'S']\n", + "0 48 48 ['O']\n", + "71 0 71 ['O', 'Z']\n", + "0 93 93 ['P']\n", + "97 0 97 ['P', 'R']\n", + "0 65 65 ['R']\n", + "80 0 80 ['R', 'S']\n", + "0 127 127 ['U']\n", + "142 0 142 ['U', 'V']\n" + ] + }, + { + "data": { + "text/plain": [ + "(1.2698088530709188, 1.2059558858330393)" + ] + }, + "execution_count": 315, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from itertools import combinations\n", + "from statistics import median, mean\n", + "\n", + "# Detour index for Romania\n", + "\n", + "L = romania.locations\n", + "def ratio(a, b): return astar_search(RouteProblem(a, b, map=romania)).path_cost / sld(L[a], L[b])\n", + "nums = [ratio(a, b) for a,b in combinations(L, 2) if b in r1.actions(a)]\n", + "mean(nums), median(nums) # 1.7, 1.6 # 1.26, 1.2 for adjacent cities" + ] + }, + { + "cell_type": "code", + "execution_count": 300, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 300, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sld" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { diff --git a/tests/.pytest_cache/v/cache/lastfailed b/tests/.pytest_cache/v/cache/lastfailed deleted file mode 100644 index e69de29bb..000000000 diff --git a/tests/test_agents.py b/tests/test_agents.py index dd390fc89..d1a669486 100644 --- a/tests/test_agents.py +++ b/tests/test_agents.py @@ -1,13 +1,19 @@ import random -from agents import Direction -from agents import Agent -from agents import ReflexVacuumAgent, ModelBasedVacuumAgent, TrivialVacuumEnvironment, compare_agents,\ - RandomVacuumAgent, TableDrivenVacuumAgent, TableDrivenAgentProgram, RandomAgentProgram, \ - SimpleReflexAgentProgram, ModelBasedReflexAgentProgram, rule_match +import pytest -random.seed("aima-python") +from agents import (ReflexVacuumAgent, ModelBasedVacuumAgent, TrivialVacuumEnvironment, compare_agents, + RandomVacuumAgent, TableDrivenVacuumAgent, TableDrivenAgentProgram, RandomAgentProgram, + SimpleReflexAgentProgram, ModelBasedReflexAgentProgram, Wall, Gold, Explorer, Thing, Bump, Glitter, + WumpusEnvironment, Pit, VacuumEnvironment, Dirt, Direction, Agent) +# random seed may affect the placement +# of things in the environment which may +# lead to failure of tests. Please change +# the seed if the tests are failing with +# current changes in any stochastic method +# function or variable. +random.seed(9) def test_move_forward(): d = Direction("up") @@ -56,12 +62,12 @@ def test_add(): assert l2.direction == Direction.D -def test_RandomAgentProgram() : - #create a list of all the actions a vacuum cleaner can perform +def test_RandomAgentProgram(): + # create a list of all the actions a Vacuum cleaner can perform list = ['Right', 'Left', 'Suck', 'NoOp'] # create a program and then an object of the RandomAgentProgram program = RandomAgentProgram(list) - + agent = Agent(program) # create an object of TrivialVacuumEnvironment environment = TrivialVacuumEnvironment() @@ -70,10 +76,10 @@ def test_RandomAgentProgram() : # run the environment environment.run() # check final status of the environment - assert environment.status == {(1, 0): 'Clean' , (0, 0): 'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} -def test_RandomVacuumAgent() : +def test_RandomVacuumAgent(): # create an object of the RandomVacuumAgent agent = RandomVacuumAgent() # create an object of TrivialVacuumEnvironment @@ -83,10 +89,11 @@ def test_RandomVacuumAgent() : # run the environment environment.run() # check final status of the environment - assert environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} def test_TableDrivenAgent(): + random.seed(10) loc_A, loc_B = (0, 0), (1, 0) # table defining all the possible states of the agent table = {((loc_A, 'Clean'),): 'Right', @@ -98,8 +105,7 @@ def test_TableDrivenAgent(): ((loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck', ((loc_B, 'Dirty'), (loc_B, 'Clean')): 'Left', ((loc_A, 'Dirty'), (loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', - ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck' - } + ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck'} # create an program and then an object of the TableDrivenAgent program = TableDrivenAgentProgram(table) @@ -107,22 +113,22 @@ def test_TableDrivenAgent(): # create an object of TrivialVacuumEnvironment environment = TrivialVacuumEnvironment() # initializing some environment status - environment.status = {loc_A:'Dirty', loc_B:'Dirty'} + environment.status = {loc_A: 'Dirty', loc_B: 'Dirty'} # add agent to the environment environment.add_thing(agent) # run the environment by single step everytime to check how environment evolves using TableDrivenAgentProgram - environment.run(steps = 1) - assert environment.status == {(1,0): 'Clean', (0,0): 'Dirty'} + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Dirty'} - environment.run(steps = 1) - assert environment.status == {(1,0): 'Clean', (0,0): 'Dirty'} + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Dirty'} - environment.run(steps = 1) - assert environment.status == {(1,0): 'Clean', (0,0): 'Clean'} + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} -def test_ReflexVacuumAgent() : +def test_ReflexVacuumAgent(): # create an object of the ReflexVacuumAgent agent = ReflexVacuumAgent() # create an object of TrivialVacuumEnvironment @@ -132,31 +138,31 @@ def test_ReflexVacuumAgent() : # run the environment environment.run() # check final status of the environment - assert environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} def test_SimpleReflexAgentProgram(): class Rule: - + def __init__(self, state, action): self.__state = state self.action = action - + def matches(self, state): return self.__state == state - + loc_A = (0, 0) loc_B = (1, 0) - + # create rules for a two state Vacuum Environment rules = [Rule((loc_A, "Dirty"), "Suck"), Rule((loc_A, "Clean"), "Right"), - Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] - + Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] + def interpret_input(state): return state - + # create a program and then an object of the SimpleReflexAgentProgram - program = SimpleReflexAgentProgram(rules, interpret_input) + program = SimpleReflexAgentProgram(rules, interpret_input) agent = Agent(program) # create an object of TrivialVacuumEnvironment environment = TrivialVacuumEnvironment() @@ -165,7 +171,7 @@ def interpret_input(state): # run the environment environment.run() # check final status of the environment - assert environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} def test_ModelBasedReflexAgentProgram(): @@ -181,9 +187,9 @@ def matches(self, state): loc_A = (0, 0) loc_B = (1, 0) - # create rules for a two-state vacuum environment + # create rules for a two-state Vacuum Environment rules = [Rule((loc_A, "Dirty"), "Suck"), Rule((loc_A, "Clean"), "Right"), - Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] + Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] def update_state(state, action, percept, model): return percept @@ -201,7 +207,7 @@ def update_state(state, action, percept, model): assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} -def test_ModelBasedVacuumAgent() : +def test_ModelBasedVacuumAgent(): # create an object of the ModelBasedVacuumAgent agent = ModelBasedVacuumAgent() # create an object of TrivialVacuumEnvironment @@ -211,10 +217,10 @@ def test_ModelBasedVacuumAgent() : # run the environment environment.run() # check final status of the environment - assert environment.status == {(1,0):'Clean' , (0,0) : 'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} -def test_TableDrivenVacuumAgent() : +def test_TableDrivenVacuumAgent(): # create an object of the TableDrivenVacuumAgent agent = TableDrivenVacuumAgent() # create an object of the TrivialVacuumEnvironment @@ -224,16 +230,16 @@ def test_TableDrivenVacuumAgent() : # run the environment environment.run() # check final status of the environment - assert environment.status == {(1, 0):'Clean', (0, 0):'Clean'} + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} -def test_compare_agents() : +def test_compare_agents(): environment = TrivialVacuumEnvironment agents = [ModelBasedVacuumAgent, ReflexVacuumAgent] result = compare_agents(environment, agents) - performance_ModelBasedVacummAgent = result[0][1] - performance_ReflexVacummAgent = result[1][1] + performance_ModelBasedVacuumAgent = result[0][1] + performance_ReflexVacuumAgent = result[1][1] # The performance of ModelBasedVacuumAgent will be at least as good as that of # ReflexVacuumAgent, since ModelBasedVacuumAgent can identify when it has @@ -241,7 +247,7 @@ def test_compare_agents() : # NoOp leading to 0 performance change, whereas ReflexVacuumAgent cannot # identify the terminal state and thus will keep moving, leading to worse # performance compared to ModelBasedVacuumAgent. - assert performance_ReflexVacummAgent <= performance_ModelBasedVacummAgent + assert performance_ReflexVacuumAgent <= performance_ModelBasedVacuumAgent def test_TableDrivenAgentProgram(): @@ -250,17 +256,132 @@ def test_TableDrivenAgentProgram(): (('bar', 1),): 'action3', (('bar', 2),): 'action1', (('foo', 1), ('foo', 1),): 'action2', - (('foo', 1), ('foo', 2),): 'action3', - } + (('foo', 1), ('foo', 2),): 'action3'} agent_program = TableDrivenAgentProgram(table) assert agent_program(('foo', 1)) == 'action1' assert agent_program(('foo', 2)) == 'action3' - assert agent_program(('invalid percept',)) == None + assert agent_program(('invalid percept',)) is None def test_Agent(): def constant_prog(percept): return percept + agent = Agent(constant_prog) result = agent.program(5) assert result == 5 + + +def test_VacuumEnvironment(): + # initialize Vacuum Environment + v = VacuumEnvironment(6, 6) + # get an agent + agent = ModelBasedVacuumAgent() + agent.direction = Direction(Direction.R) + v.add_thing(agent) + v.add_thing(Dirt(), location=(2, 1)) + + # check if things are added properly + assert len([x for x in v.things if isinstance(x, Wall)]) == 20 + assert len([x for x in v.things if isinstance(x, Dirt)]) == 1 + + # let the action begin! + assert v.percept(agent) == ("Clean", "None") + v.execute_action(agent, "Forward") + assert v.percept(agent) == ("Dirty", "None") + v.execute_action(agent, "TurnLeft") + v.execute_action(agent, "Forward") + assert v.percept(agent) == ("Dirty", "Bump") + v.execute_action(agent, "Suck") + assert v.percept(agent) == ("Clean", "None") + old_performance = agent.performance + v.execute_action(agent, "NoOp") + assert old_performance == agent.performance + + +def test_WumpusEnvironment(): + def constant_prog(percept): + return percept + + # initialize Wumpus Environment + w = WumpusEnvironment(constant_prog) + + # check if things are added properly + assert len([x for x in w.things if isinstance(x, Wall)]) == 20 + assert any(map(lambda x: isinstance(x, Gold), w.things)) + assert any(map(lambda x: isinstance(x, Explorer), w.things)) + assert not any(map(lambda x: not isinstance(x, Thing), w.things)) + + # check that gold and wumpus are not present on (1,1) + assert not any(map(lambda x: isinstance(x, Gold) or isinstance(x, WumpusEnvironment), w.list_things_at((1, 1)))) + + # check if w.get_world() segments objects correctly + assert len(w.get_world()) == 6 + for row in w.get_world(): + assert len(row) == 6 + + # start the game! + agent = [x for x in w.things if isinstance(x, Explorer)][0] + gold = [x for x in w.things if isinstance(x, Gold)][0] + pit = [x for x in w.things if isinstance(x, Pit)][0] + + assert not w.is_done() + + # check Walls + agent.location = (1, 2) + percepts = w.percept(agent) + assert len(percepts) == 5 + assert any(map(lambda x: isinstance(x, Bump), percepts[0])) + + # check Gold + agent.location = gold.location + percepts = w.percept(agent) + assert any(map(lambda x: isinstance(x, Glitter), percepts[4])) + agent.location = (gold.location[0], gold.location[1] + 1) + percepts = w.percept(agent) + assert not any(map(lambda x: isinstance(x, Glitter), percepts[4])) + + # check agent death + agent.location = pit.location + assert w.in_danger(agent) + assert not agent.alive + assert agent.killed_by == Pit.__name__ + assert agent.performance == -1000 + + assert w.is_done() + + +def test_WumpusEnvironmentActions(): + random.seed(9) + def constant_prog(percept): + return percept + + # initialize Wumpus Environment + w = WumpusEnvironment(constant_prog) + + agent = [x for x in w.things if isinstance(x, Explorer)][0] + gold = [x for x in w.things if isinstance(x, Gold)][0] + pit = [x for x in w.things if isinstance(x, Pit)][0] + + agent.location = (1, 1) + assert agent.direction.direction == "right" + w.execute_action(agent, 'TurnRight') + assert agent.direction.direction == "down" + w.execute_action(agent, 'TurnLeft') + assert agent.direction.direction == "right" + w.execute_action(agent, 'Forward') + assert agent.location == (2, 1) + + agent.location = gold.location + w.execute_action(agent, 'Grab') + assert agent.holding == [gold] + + agent.location = (1, 1) + w.execute_action(agent, 'Climb') + assert not any(map(lambda x: isinstance(x, Explorer), w.things)) + + assert w.is_done() + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_agents4e.py b/tests/test_agents4e.py new file mode 100644 index 000000000..295a1ee47 --- /dev/null +++ b/tests/test_agents4e.py @@ -0,0 +1,386 @@ +import random + +import pytest + +from agents4e import (ReflexVacuumAgent, ModelBasedVacuumAgent, TrivialVacuumEnvironment, compare_agents, + RandomVacuumAgent, TableDrivenVacuumAgent, TableDrivenAgentProgram, RandomAgentProgram, + SimpleReflexAgentProgram, ModelBasedReflexAgentProgram, Wall, Gold, Explorer, Thing, Bump, + Glitter, WumpusEnvironment, Pit, VacuumEnvironment, Dirt, Direction, Agent) + +# random seed may affect the placement +# of things in the environment which may +# lead to failure of tests. Please change +# the seed if the tests are failing with +# current changes in any stochastic method +# function or variable. +random.seed(9) + +def test_move_forward(): + d = Direction("up") + l1 = d.move_forward((0, 0)) + assert l1 == (0, -1) + + d = Direction(Direction.R) + l1 = d.move_forward((0, 0)) + assert l1 == (1, 0) + + d = Direction(Direction.D) + l1 = d.move_forward((0, 0)) + assert l1 == (0, 1) + + d = Direction("left") + l1 = d.move_forward((0, 0)) + assert l1 == (-1, 0) + + l2 = d.move_forward((1, 0)) + assert l2 == (0, 0) + + +def test_add(): + d = Direction(Direction.U) + l1 = d + "right" + l2 = d + "left" + assert l1.direction == Direction.R + assert l2.direction == Direction.L + + d = Direction("right") + l1 = d.__add__(Direction.L) + l2 = d.__add__(Direction.R) + assert l1.direction == "up" + assert l2.direction == "down" + + d = Direction("down") + l1 = d.__add__("right") + l2 = d.__add__("left") + assert l1.direction == Direction.L + assert l2.direction == Direction.R + + d = Direction(Direction.L) + l1 = d + Direction.R + l2 = d + Direction.L + assert l1.direction == Direction.U + assert l2.direction == Direction.D + + +def test_RandomAgentProgram(): + # create a list of all the actions a Vacuum cleaner can perform + list = ['Right', 'Left', 'Suck', 'NoOp'] + # create a program and then an object of the RandomAgentProgram + program = RandomAgentProgram(list) + + agent = Agent(program) + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_RandomVacuumAgent(): + # create an object of the RandomVacuumAgent + agent = RandomVacuumAgent() + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_TableDrivenAgent(): + random.seed(10) + loc_A, loc_B = (0, 0), (1, 0) + # table defining all the possible states of the agent + table = {((loc_A, 'Clean'),): 'Right', + ((loc_A, 'Dirty'),): 'Suck', + ((loc_B, 'Clean'),): 'Left', + ((loc_B, 'Dirty'),): 'Suck', + ((loc_A, 'Dirty'), (loc_A, 'Clean')): 'Right', + ((loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', + ((loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck', + ((loc_B, 'Dirty'), (loc_B, 'Clean')): 'Left', + ((loc_A, 'Dirty'), (loc_A, 'Clean'), (loc_B, 'Dirty')): 'Suck', + ((loc_B, 'Dirty'), (loc_B, 'Clean'), (loc_A, 'Dirty')): 'Suck'} + + # create an program and then an object of the TableDrivenAgent + program = TableDrivenAgentProgram(table) + agent = Agent(program) + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # initializing some environment status + environment.status = {loc_A: 'Dirty', loc_B: 'Dirty'} + # add agent to the environment + environment.add_thing(agent, location=(1, 0)) + # run the environment by single step everytime to check how environment evolves using TableDrivenAgentProgram + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Dirty'} + + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Dirty'} + + environment.run(steps=1) + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_ReflexVacuumAgent(): + # create an object of the ReflexVacuumAgent + agent = ReflexVacuumAgent() + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_SimpleReflexAgentProgram(): + class Rule: + + def __init__(self, state, action): + self.__state = state + self.action = action + + def matches(self, state): + return self.__state == state + + loc_A = (0, 0) + loc_B = (1, 0) + + # create rules for a two state Vacuum Environment + rules = [Rule((loc_A, "Dirty"), "Suck"), Rule((loc_A, "Clean"), "Right"), + Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] + + def interpret_input(state): + return state + + # create a program and then an object of the SimpleReflexAgentProgram + program = SimpleReflexAgentProgram(rules, interpret_input) + agent = Agent(program) + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_ModelBasedReflexAgentProgram(): + class Rule: + + def __init__(self, state, action): + self.__state = state + self.action = action + + def matches(self, state): + return self.__state == state + + loc_A = (0, 0) + loc_B = (1, 0) + + # create rules for a two-state Vacuum Environment + rules = [Rule((loc_A, "Dirty"), "Suck"), Rule((loc_A, "Clean"), "Right"), + Rule((loc_B, "Dirty"), "Suck"), Rule((loc_B, "Clean"), "Left")] + + def update_state(state, action, percept, transition_model, sensor_model): + return percept + + # create a program and then an object of the ModelBasedReflexAgentProgram class + program = ModelBasedReflexAgentProgram(rules, update_state, None, None) + agent = Agent(program) + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_ModelBasedVacuumAgent(): + # create an object of the ModelBasedVacuumAgent + agent = ModelBasedVacuumAgent() + # create an object of TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_TableDrivenVacuumAgent(): + # create an object of the TableDrivenVacuumAgent + agent = TableDrivenVacuumAgent() + # create an object of the TrivialVacuumEnvironment + environment = TrivialVacuumEnvironment() + # add agent to the environment + environment.add_thing(agent) + # run the environment + environment.run() + # check final status of the environment + assert environment.status == {(1, 0): 'Clean', (0, 0): 'Clean'} + + +def test_compare_agents(): + environment = TrivialVacuumEnvironment + agents = [ModelBasedVacuumAgent, ReflexVacuumAgent] + + result = compare_agents(environment, agents) + performance_ModelBasedVacuumAgent = result[0][1] + performance_ReflexVacuumAgent = result[1][1] + + # The performance of ModelBasedVacuumAgent will be at least as good as that of + # ReflexVacuumAgent, since ModelBasedVacuumAgent can identify when it has + # reached the terminal state (both locations being clean) and will perform + # NoOp leading to 0 performance change, whereas ReflexVacuumAgent cannot + # identify the terminal state and thus will keep moving, leading to worse + # performance compared to ModelBasedVacuumAgent. + assert performance_ReflexVacuumAgent <= performance_ModelBasedVacuumAgent + + +def test_TableDrivenAgentProgram(): + table = {(('foo', 1),): 'action1', + (('foo', 2),): 'action2', + (('bar', 1),): 'action3', + (('bar', 2),): 'action1', + (('foo', 1), ('foo', 1),): 'action2', + (('foo', 1), ('foo', 2),): 'action3'} + agent_program = TableDrivenAgentProgram(table) + assert agent_program(('foo', 1)) == 'action1' + assert agent_program(('foo', 2)) == 'action3' + assert agent_program(('invalid percept',)) is None + + +def test_Agent(): + def constant_prog(percept): + return percept + + agent = Agent(constant_prog) + result = agent.program(5) + assert result == 5 + + +def test_VacuumEnvironment(): + # initialize Vacuum Environment + v = VacuumEnvironment(6, 6) + # get an agent + agent = ModelBasedVacuumAgent() + agent.direction = Direction(Direction.R) + v.add_thing(agent, location=(1, 1)) + v.add_thing(Dirt(), location=(2, 1)) + + # check if things are added properly + assert len([x for x in v.things if isinstance(x, Wall)]) == 20 + assert len([x for x in v.things if isinstance(x, Dirt)]) == 1 + + # let the action begin! + assert v.percept(agent) == ("Clean", "None") + v.execute_action(agent, "Forward") + assert v.percept(agent) == ("Dirty", "None") + v.execute_action(agent, "TurnLeft") + v.execute_action(agent, "Forward") + assert v.percept(agent) == ("Dirty", "Bump") + v.execute_action(agent, "Suck") + assert v.percept(agent) == ("Clean", "None") + old_performance = agent.performance + v.execute_action(agent, "NoOp") + assert old_performance == agent.performance + + +def test_WumpusEnvironment(): + def constant_prog(percept): + return percept + + # initialize Wumpus Environment + w = WumpusEnvironment(constant_prog) + + # check if things are added properly + assert len([x for x in w.things if isinstance(x, Wall)]) == 20 + assert any(map(lambda x: isinstance(x, Gold), w.things)) + assert any(map(lambda x: isinstance(x, Explorer), w.things)) + assert not any(map(lambda x: not isinstance(x, Thing), w.things)) + + # check that gold and wumpus are not present on (1,1) + assert not any(map(lambda x: isinstance(x, Gold) or isinstance(x, WumpusEnvironment), w.list_things_at((1, 1)))) + + # check if w.get_world() segments objects correctly + assert len(w.get_world()) == 6 + for row in w.get_world(): + assert len(row) == 6 + + # start the game! + agent = [x for x in w.things if isinstance(x, Explorer)][0] + gold = [x for x in w.things if isinstance(x, Gold)][0] + pit = [x for x in w.things if isinstance(x, Pit)][0] + + assert not w.is_done() + + # check Walls + agent.location = (1, 2) + percepts = w.percept(agent) + assert len(percepts) == 5 + assert any(map(lambda x: isinstance(x, Bump), percepts[0])) + + # check Gold + agent.location = gold.location + percepts = w.percept(agent) + assert any(map(lambda x: isinstance(x, Glitter), percepts[4])) + agent.location = (gold.location[0], gold.location[1] + 1) + percepts = w.percept(agent) + assert not any(map(lambda x: isinstance(x, Glitter), percepts[4])) + + # check agent death + agent.location = pit.location + assert w.in_danger(agent) + assert not agent.alive + assert agent.killed_by == Pit.__name__ + assert agent.performance == -1000 + + assert w.is_done() + + +def test_WumpusEnvironmentActions(): + random.seed(9) + def constant_prog(percept): + return percept + + # initialize Wumpus Environment + w = WumpusEnvironment(constant_prog) + + agent = [x for x in w.things if isinstance(x, Explorer)][0] + gold = [x for x in w.things if isinstance(x, Gold)][0] + pit = [x for x in w.things if isinstance(x, Pit)][0] + + agent.location = (1, 1) + assert agent.direction.direction == "right" + w.execute_action(agent, 'TurnRight') + assert agent.direction.direction == "down" + w.execute_action(agent, 'TurnLeft') + assert agent.direction.direction == "right" + w.execute_action(agent, 'Forward') + assert agent.location == (2, 1) + + agent.location = gold.location + w.execute_action(agent, 'Grab') + assert agent.holding == [gold] + + agent.location = (1, 1) + w.execute_action(agent, 'Climb') + assert not any(map(lambda x: isinstance(x, Explorer), w.things)) + + assert w.is_done() + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_csp.py b/tests/test_csp.py index 2bc907b6c..a070cd531 100644 --- a/tests/test_csp.py +++ b/tests/test_csp.py @@ -3,7 +3,6 @@ from csp import * import random - random.seed("aima-python") @@ -11,21 +10,21 @@ def test_csp_assign(): var = 10 val = 5 assignment = {} - australia.assign(var, val, assignment) + australia_csp.assign(var, val, assignment) - assert australia.nassigns == 1 + assert australia_csp.nassigns == 1 assert assignment[var] == val def test_csp_unassign(): var = 10 assignment = {var: 5} - australia.unassign(var, assignment) + australia_csp.unassign(var, assignment) assert var not in assignment -def test_csp_nconflits(): +def test_csp_nconflicts(): map_coloring_test = MapColoringCSP(list('RGB'), 'A: B C; B: C; C: ') assignment = {'A': 'R', 'B': 'G'} var = 'C' @@ -68,17 +67,16 @@ def test_csp_result(): def test_csp_goal_test(): map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') state = (('A', '1'), ('B', '3'), ('C', '2')) - assert map_coloring_test.goal_test(state) is True + assert map_coloring_test.goal_test(state) state = (('A', '1'), ('C', '2')) - assert map_coloring_test.goal_test(state) is False + assert not map_coloring_test.goal_test(state) def test_csp_support_pruning(): map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') map_coloring_test.support_pruning() - assert map_coloring_test.curr_domains == {'A': ['1', '2', '3'], 'B': ['1', '2', '3'], - 'C': ['1', '2', '3']} + assert map_coloring_test.curr_domains == {'A': ['1', '2', '3'], 'B': ['1', '2', '3'], 'C': ['1', '2', '3']} def test_csp_suppose(): @@ -89,8 +87,7 @@ def test_csp_suppose(): removals = map_coloring_test.suppose(var, value) assert removals == [('A', '2'), ('A', '3')] - assert map_coloring_test.curr_domains == {'A': ['1'], 'B': ['1', '2', '3'], - 'C': ['1', '2', '3']} + assert map_coloring_test.curr_domains == {'A': ['1'], 'B': ['1', '2', '3'], 'C': ['1', '2', '3']} def test_csp_prune(): @@ -101,16 +98,14 @@ def test_csp_prune(): map_coloring_test.support_pruning() map_coloring_test.prune(var, value, removals) - assert map_coloring_test.curr_domains == {'A': ['1', '2'], 'B': ['1', '2', '3'], - 'C': ['1', '2', '3']} + assert map_coloring_test.curr_domains == {'A': ['1', '2'], 'B': ['1', '2', '3'], 'C': ['1', '2', '3']} assert removals is None map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') removals = [('A', '2')] map_coloring_test.support_pruning() map_coloring_test.prune(var, value, removals) - assert map_coloring_test.curr_domains == {'A': ['1', '2'], 'B': ['1', '2', '3'], - 'C': ['1', '2', '3']} + assert map_coloring_test.curr_domains == {'A': ['1', '2'], 'B': ['1', '2', '3'], 'C': ['1', '2', '3']} assert removals == [('A', '2'), ('A', '3')] @@ -126,9 +121,9 @@ def test_csp_choices(): assert map_coloring_test.choices(var) == ['1', '2'] -def test_csp_infer_assignement(): +def test_csp_infer_assignment(): map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') - map_coloring_test.infer_assignment() == {} + assert map_coloring_test.infer_assignment() == {} var = 'A' value = '3' @@ -136,7 +131,7 @@ def test_csp_infer_assignement(): value = '1' map_coloring_test.prune(var, value, None) - map_coloring_test.infer_assignment() == {'A': '2'} + assert map_coloring_test.infer_assignment() == {'A': '2'} def test_csp_restore(): @@ -146,8 +141,7 @@ def test_csp_restore(): map_coloring_test.restore(removals) - assert map_coloring_test.curr_domains == {'A': ['2', '3', '1'], 'B': ['1', '2', '3'], - 'C': ['2', '3']} + assert map_coloring_test.curr_domains == {'A': ['2', '3', '1'], 'B': ['1', '2', '3'], 'C': ['2', '3']} def test_csp_conflicted_vars(): @@ -174,7 +168,7 @@ def test_csp_conflicted_vars(): def test_revise(): neighbors = parse_neighbors('A: B; B: ') domains = {'A': [0], 'B': [4]} - constraints = lambda X, x, Y, y: x % 2 == 0 and (x+y) == 4 + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) csp.support_pruning() @@ -182,43 +176,99 @@ def test_revise(): Xj = 'B' removals = [] - assert revise(csp, Xi, Xj, removals) is False + consistency, _ = revise(csp, Xi, Xj, removals) + assert not consistency assert len(removals) == 0 domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) csp.support_pruning() - assert revise(csp, Xi, Xj, removals) is True + assert revise(csp, Xi, Xj, removals) assert removals == [('A', 1), ('A', 3)] def test_AC3(): neighbors = parse_neighbors('A: B; B: ') domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} - constraints = lambda X, x, Y, y: x % 2 == 0 and (x+y) == 4 and y % 2 != 0 + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 and y % 2 != 0 removals = [] csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) - assert AC3(csp, removals=removals) is False + consistency, _ = AC3(csp, removals=removals) + assert not consistency - constraints = lambda X, x, Y, y: (x % 2) == 0 and (x+y) == 4 + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 removals = [] csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) - assert AC3(csp, removals=removals) is True + assert AC3(csp, removals=removals) assert (removals == [('A', 1), ('A', 3), ('B', 1), ('B', 3)] or removals == [('B', 1), ('B', 3), ('A', 1), ('A', 3)]) - - domains = {'A': [ 2, 4], 'B': [ 3, 5]} - constraints = lambda X, x, Y, y: int(x) > int (y) - removals=[] + + domains = {'A': [2, 4], 'B': [3, 5]} + constraints = lambda X, x, Y, y: (X == 'A' and Y == 'B') or (X == 'B' and Y == 'A') and x > y + removals = [] csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) assert AC3(csp, removals=removals) +def test_AC3b(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 and y % 2 != 0 + removals = [] + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + consistency, _ = AC3b(csp, removals=removals) + assert not consistency + + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 + removals = [] + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + assert AC3b(csp, removals=removals) + assert (removals == [('A', 1), ('A', 3), ('B', 1), ('B', 3)] or + removals == [('B', 1), ('B', 3), ('A', 1), ('A', 3)]) + + domains = {'A': [2, 4], 'B': [3, 5]} + constraints = lambda X, x, Y, y: (X == 'A' and Y == 'B') or (X == 'B' and Y == 'A') and x > y + removals = [] + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + assert AC3b(csp, removals=removals) + + +def test_AC4(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 and y % 2 != 0 + removals = [] + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + consistency, _ = AC4(csp, removals=removals) + assert not consistency + + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 + removals = [] + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + assert AC4(csp, removals=removals) + assert (removals == [('A', 1), ('A', 3), ('B', 1), ('B', 3)] or + removals == [('B', 1), ('B', 3), ('A', 1), ('A', 3)]) + + domains = {'A': [2, 4], 'B': [3, 5]} + constraints = lambda X, x, Y, y: (X == 'A' and Y == 'B') or (X == 'B' and Y == 'A') and x > y + removals = [] + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + assert AC4(csp, removals=removals) + + def test_first_unassigned_variable(): map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') assignment = {'A': '1', 'B': '2'} @@ -247,7 +297,7 @@ def test_num_legal_values(): def test_mrv(): neighbors = parse_neighbors('A: B; B: C; C: ') domains = {'A': [0, 1, 2, 3, 4], 'B': [4], 'C': [0, 1, 2, 3, 4]} - constraints = lambda X, x, Y, y: x % 2 == 0 and (x+y) == 4 + constraints = lambda X, x, Y, y: x % 2 == 0 and x + y == 4 csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) assignment = {'A': 0} @@ -269,13 +319,13 @@ def test_mrv(): def test_unordered_domain_values(): map_coloring_test = MapColoringCSP(list('123'), 'A: B C; B: C; C: ') assignment = None - assert unordered_domain_values('A', assignment, map_coloring_test) == ['1', '2', '3'] + assert unordered_domain_values('A', assignment, map_coloring_test) == ['1', '2', '3'] def test_lcv(): neighbors = parse_neighbors('A: B; B: C; C: ') domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4, 5], 'C': [0, 1, 2, 3, 4]} - constraints = lambda X, x, Y, y: x % 2 == 0 and (x+y) == 4 + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) assignment = {'A': 0} @@ -303,51 +353,48 @@ def test_forward_checking(): var = 'B' value = 3 assignment = {'A': 1, 'C': '3'} - assert forward_checking(csp, var, value, assignment, None) == True + assert forward_checking(csp, var, value, assignment, None) assert csp.curr_domains['A'] == A_curr_domains assert csp.curr_domains['C'] == C_curr_domains assignment = {'C': 3} - assert forward_checking(csp, var, value, assignment, None) == True + assert forward_checking(csp, var, value, assignment, None) assert csp.curr_domains['A'] == [1, 3] csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) csp.support_pruning() assignment = {} - assert forward_checking(csp, var, value, assignment, None) == True + assert forward_checking(csp, var, value, assignment, None) assert csp.curr_domains['A'] == [1, 3] assert csp.curr_domains['C'] == [1, 3] csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) - domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4, 7], 'C': [0, 1, 2, 3, 4]} csp.support_pruning() value = 7 assignment = {} - assert forward_checking(csp, var, value, assignment, None) == False + assert not forward_checking(csp, var, value, assignment, None) assert (csp.curr_domains['A'] == [] or csp.curr_domains['C'] == []) def test_backtracking_search(): - assert backtracking_search(australia) - assert backtracking_search(australia, select_unassigned_variable=mrv) - assert backtracking_search(australia, order_domain_values=lcv) - assert backtracking_search(australia, select_unassigned_variable=mrv, - order_domain_values=lcv) - assert backtracking_search(australia, inference=forward_checking) - assert backtracking_search(australia, inference=mac) - assert backtracking_search(usa, select_unassigned_variable=mrv, - order_domain_values=lcv, inference=mac) + assert backtracking_search(australia_csp) + assert backtracking_search(australia_csp, select_unassigned_variable=mrv) + assert backtracking_search(australia_csp, order_domain_values=lcv) + assert backtracking_search(australia_csp, select_unassigned_variable=mrv, order_domain_values=lcv) + assert backtracking_search(australia_csp, inference=forward_checking) + assert backtracking_search(australia_csp, inference=mac) + assert backtracking_search(usa_csp, select_unassigned_variable=mrv, order_domain_values=lcv, inference=mac) def test_min_conflicts(): - assert min_conflicts(australia) - assert min_conflicts(france) + assert min_conflicts(australia_csp) + assert min_conflicts(france_csp) - tests = [(usa, None)] * 3 - assert failure_test(min_conflicts, tests) >= 1/3 + tests = [(usa_csp, None)] * 3 + assert failure_test(min_conflicts, tests) >= 1 / 3 australia_impossible = MapColoringCSP(list('RG'), 'SA: WA NT Q NSW V; NT: WA Q; NSW: Q V; T: ') assert min_conflicts(australia_impossible, 1000) is None @@ -379,7 +426,6 @@ def test_nqueens_csp(): assert 2 not in assignment assert 3 not in assignment - assignment = {} assignment = {0: 0, 1: 1, 2: 4, 3: 1, 4: 6} csp.assign(5, 7, assignment) assert len(assignment) == 6 @@ -419,10 +465,10 @@ def test_parse_neighbours(): def test_topological_sort(): root = 'NT' - Sort, Parents = topological_sort(australia,root) + Sort, Parents = topological_sort(australia_csp, root) - assert Sort == ['NT','SA','Q','NSW','V','WA'] - assert Parents['NT'] == None + assert Sort == ['NT', 'SA', 'Q', 'NSW', 'V', 'WA'] + assert Parents['NT'] is None assert Parents['SA'] == 'NT' assert Parents['Q'] == 'SA' assert Parents['NSW'] == 'Q' @@ -431,11 +477,186 @@ def test_topological_sort(): def test_tree_csp_solver(): - australia_small = MapColoringCSP(list('RB'), - 'NT: WA Q; NSW: Q V') + australia_small = MapColoringCSP(list('RB'), 'NT: WA Q; NSW: Q V') tcs = tree_csp_solver(australia_small) assert (tcs['NT'] == 'R' and tcs['WA'] == 'B' and tcs['Q'] == 'B' and tcs['NSW'] == 'R' and tcs['V'] == 'B') or \ (tcs['NT'] == 'B' and tcs['WA'] == 'R' and tcs['Q'] == 'R' and tcs['NSW'] == 'B' and tcs['V'] == 'R') + +def test_ac_solver(): + assert ac_solver(csp_crossword) == {'one_across': 'has', + 'one_down': 'hold', + 'two_down': 'syntax', + 'three_across': 'land', + 'four_across': 'ant'} or {'one_across': 'bus', + 'one_down': 'buys', + 'two_down': 'search', + 'three_across': 'year', + 'four_across': 'car'} + assert ac_solver(two_two_four) == {'T': 7, 'F': 1, 'W': 6, 'O': 5, 'U': 3, 'R': 0, 'C1': 1, 'C2': 1, 'C3': 1} or \ + {'T': 9, 'F': 1, 'W': 2, 'O': 8, 'U': 5, 'R': 6, 'C1': 1, 'C2': 0, 'C3': 1} + assert ac_solver(send_more_money) == \ + {'S': 9, 'M': 1, 'E': 5, 'N': 6, 'D': 7, 'O': 0, 'R': 8, 'Y': 2, 'C1': 1, 'C2': 1, 'C3': 0, 'C4': 1} + + +def test_ac_search_solver(): + assert ac_search_solver(csp_crossword) == {'one_across': 'has', + 'one_down': 'hold', + 'two_down': 'syntax', + 'three_across': 'land', + 'four_across': 'ant'} or {'one_across': 'bus', + 'one_down': 'buys', + 'two_down': 'search', + 'three_across': 'year', + 'four_across': 'car'} + assert ac_search_solver(two_two_four) == {'T': 7, 'F': 1, 'W': 6, 'O': 5, 'U': 3, 'R': 0, + 'C1': 1, 'C2': 1, 'C3': 1} or \ + {'T': 9, 'F': 1, 'W': 2, 'O': 8, 'U': 5, 'R': 6, 'C1': 1, 'C2': 0, 'C3': 1} + assert ac_search_solver(send_more_money) == {'S': 9, 'M': 1, 'E': 5, 'N': 6, 'D': 7, 'O': 0, 'R': 8, 'Y': 2, + 'C1': 1, 'C2': 1, 'C3': 0, 'C4': 1} + + +def test_different_values_constraint(): + assert different_values_constraint('A', 1, 'B', 2) + assert not different_values_constraint('A', 1, 'B', 1) + + +def test_flatten(): + sequence = [[0, 1, 2], [4, 5]] + assert flatten(sequence) == [0, 1, 2, 4, 5] + + +def test_sudoku(): + h = Sudoku(easy1) + assert backtracking_search(h, select_unassigned_variable=mrv, inference=forward_checking) is not None + g = Sudoku(harder1) + assert backtracking_search(g, select_unassigned_variable=mrv, inference=forward_checking) is not None + + +def test_make_arc_consistent(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0], 'B': [3]} + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + csp.support_pruning() + Xi = 'A' + Xj = 'B' + + assert make_arc_consistent(Xi, Xj, csp) == [] + + domains = {'A': [0], 'B': [4]} + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + csp.support_pruning() + Xi = 'A' + Xj = 'B' + + assert make_arc_consistent(Xi, Xj, csp) == [0] + + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + csp.support_pruning() + + assert make_arc_consistent(Xi, Xj, csp) == [0, 2, 4] + + +def test_assign_value(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 + Xi = 'A' + Xj = 'B' + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + csp.support_pruning() + + assignment = {'A': 1} + assert assign_value(Xi, Xj, csp, assignment) is None + + assignment = {'A': 2} + assert assign_value(Xi, Xj, csp, assignment) == 2 + + constraints = lambda X, x, Y, y: (x + y) == 4 + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + csp.support_pruning() + + assignment = {'A': 1} + assert assign_value(Xi, Xj, csp, assignment) == 3 + + +def test_no_inference(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4, 5]} + constraints = lambda X, x, Y, y: (x + y) < 8 + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + + var = 'B' + value = 3 + assignment = {'A': 1} + assert no_inference(csp, var, value, assignment, None) + + +def test_mac(): + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0], 'B': [0]} + constraints = lambda X, x, Y, y: x % 2 == 0 + var = 'B' + value = 0 + assignment = {'A': 0} + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + assert mac(csp, var, value, assignment, None) + + neighbors = parse_neighbors('A: B; B: ') + domains = {'A': [0, 1, 2, 3, 4], 'B': [0, 1, 2, 3, 4]} + constraints = lambda X, x, Y, y: x % 2 == 0 and (x + y) == 4 and y % 2 != 0 + var = 'B' + value = 3 + assignment = {'A': 1} + + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + consistency, _ = mac(csp, var, value, assignment, None) + assert not consistency + + constraints = lambda X, x, Y, y: x % 2 != 0 and (x + y) == 6 and y % 2 != 0 + csp = CSP(variables=None, domains=domains, neighbors=neighbors, constraints=constraints) + _, consistency = mac(csp, var, value, assignment, None) + assert consistency + + +def test_queen_constraint(): + assert queen_constraint(0, 1, 0, 1) + assert queen_constraint(2, 1, 4, 2) + assert not queen_constraint(2, 1, 3, 2) + + +def test_zebra(): + z = Zebra() + algorithm = min_conflicts + # would take very long + ans = algorithm(z, max_steps=10000) + assert ans is None or ans == {'Red': 3, 'Yellow': 1, 'Blue': 2, 'Green': 5, 'Ivory': 4, 'Dog': 4, 'Fox': 1, + 'Snails': 3, 'Horse': 2, 'Zebra': 5, 'OJ': 4, 'Tea': 2, 'Coffee': 5, 'Milk': 3, + 'Water': 1, 'Englishman': 3, 'Spaniard': 4, 'Norwegian': 1, 'Ukranian': 2, + 'Japanese': 5, 'Kools': 1, 'Chesterfields': 2, 'Winston': 3, 'LuckyStrike': 4, + 'Parliaments': 5} + + # restrict search space + z.domains = {'Red': [3, 4], 'Yellow': [1, 2], 'Blue': [1, 2], 'Green': [4, 5], 'Ivory': [4, 5], 'Dog': [4, 5], + 'Fox': [1, 2], 'Snails': [3], 'Horse': [2], 'Zebra': [5], 'OJ': [1, 2, 3, 4, 5], + 'Tea': [1, 2, 3, 4, 5], 'Coffee': [1, 2, 3, 4, 5], 'Milk': [3], 'Water': [1, 2, 3, 4, 5], + 'Englishman': [1, 2, 3, 4, 5], 'Spaniard': [1, 2, 3, 4, 5], 'Norwegian': [1], + 'Ukranian': [1, 2, 3, 4, 5], 'Japanese': [1, 2, 3, 4, 5], 'Kools': [1, 2, 3, 4, 5], + 'Chesterfields': [1, 2, 3, 4, 5], 'Winston': [1, 2, 3, 4, 5], 'LuckyStrike': [1, 2, 3, 4, 5], + 'Parliaments': [1, 2, 3, 4, 5]} + ans = algorithm(z, max_steps=10000) + assert ans == {'Red': 3, 'Yellow': 1, 'Blue': 2, 'Green': 5, 'Ivory': 4, 'Dog': 4, 'Fox': 1, 'Snails': 3, + 'Horse': 2, 'Zebra': 5, 'OJ': 4, 'Tea': 2, 'Coffee': 5, 'Milk': 3, 'Water': 1, 'Englishman': 3, + 'Spaniard': 4, 'Norwegian': 1, 'Ukranian': 2, 'Japanese': 5, 'Kools': 1, 'Chesterfields': 2, + 'Winston': 3, 'LuckyStrike': 4, 'Parliaments': 5} + + if __name__ == "__main__": pytest.main() diff --git a/tests/test_deep_learning4e.py b/tests/test_deep_learning4e.py new file mode 100644 index 000000000..34676b02b --- /dev/null +++ b/tests/test_deep_learning4e.py @@ -0,0 +1,80 @@ +import pytest +from keras.datasets import imdb + +from deep_learning4e import * +from learning4e import DataSet, grade_learner, err_ratio + +random.seed("aima-python") + +iris_tests = [([5.0, 3.1, 0.9, 0.1], 0), + ([5.1, 3.5, 1.0, 0.0], 0), + ([4.9, 3.3, 1.1, 0.1], 0), + ([6.0, 3.0, 4.0, 1.1], 1), + ([6.1, 2.2, 3.5, 1.0], 1), + ([5.9, 2.5, 3.3, 1.1], 1), + ([7.5, 4.1, 6.2, 2.3], 2), + ([7.3, 4.0, 6.1, 2.4], 2), + ([7.0, 3.3, 6.1, 2.5], 2)] + + +def test_neural_net(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + n_samples, n_features = len(iris.examples), iris.target + + X, y = (np.array([x[:n_features] for x in iris.examples]), + np.array([x[n_features] for x in iris.examples])) + + nnl_gd = NeuralNetworkLearner(iris, [4], l_rate=0.15, epochs=100, optimizer=stochastic_gradient_descent).fit(X, y) + assert grade_learner(nnl_gd, iris_tests) > 0.7 + assert err_ratio(nnl_gd, iris) < 0.15 + + nnl_adam = NeuralNetworkLearner(iris, [4], l_rate=0.001, epochs=200, optimizer=adam).fit(X, y) + assert grade_learner(nnl_adam, iris_tests) > 0.7 + assert err_ratio(nnl_adam, iris) < 0.15 + + +def test_perceptron(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + n_samples, n_features = len(iris.examples), iris.target + + X, y = (np.array([x[:n_features] for x in iris.examples]), + np.array([x[n_features] for x in iris.examples])) + + pl_gd = PerceptronLearner(iris, l_rate=0.01, epochs=100, optimizer=stochastic_gradient_descent).fit(X, y) + assert grade_learner(pl_gd, iris_tests) == 1 + assert err_ratio(pl_gd, iris) < 0.2 + + pl_adam = PerceptronLearner(iris, l_rate=0.01, epochs=100, optimizer=adam).fit(X, y) + assert grade_learner(pl_adam, iris_tests) == 1 + assert err_ratio(pl_adam, iris) < 0.2 + + +def test_rnn(): + data = imdb.load_data(num_words=5000) + + train, val, test = keras_dataset_loader(data) + train = (train[0][:1000], train[1][:1000]) + val = (val[0][:200], val[1][:200]) + + rnn = SimpleRNNLearner(train, val) + score = rnn.evaluate(test[0][:200], test[1][:200], verbose=False) + assert score[1] >= 0.2 + + +def test_autoencoder(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + inputs = np.asarray(iris.examples) + + al = AutoencoderLearner(inputs, 100) + print(inputs[0]) + print(al.predict(inputs[:1])) + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_games.py b/tests/test_games.py index b5c30ee67..b7541ee93 100644 --- a/tests/test_games.py +++ b/tests/test_games.py @@ -1,18 +1,21 @@ +import pytest + from games import * # Creating the game instances f52 = Fig52Game() ttt = TicTacToe() +random.seed("aima-python") + -def gen_state(to_move='X', x_positions=[], o_positions=[], h=3, v=3, k=3): +def gen_state(to_move='X', x_positions=[], o_positions=[], h=3, v=3): """Given whose turn it is to move, the positions of X's on the board, the positions of O's on the board, and, (optionally) number of rows, columns and how many consecutive X's or O's required to win, return the corresponding game state""" - moves = set([(x, y) for x in range(1, h + 1) for y in range(1, v + 1)]) \ - - set(x_positions) - set(o_positions) + moves = set([(x, y) for x in range(1, h + 1) for y in range(1, v + 1)]) - set(x_positions) - set(o_positions) moves = list(moves) board = {} for pos in x_positions: @@ -22,41 +25,45 @@ def gen_state(to_move='X', x_positions=[], o_positions=[], h=3, v=3, k=3): return GameState(to_move=to_move, utility=0, board=board, moves=moves) -def test_minimax_decision(): - assert minimax_decision('A', f52) == 'a1' - assert minimax_decision('B', f52) == 'b1' - assert minimax_decision('C', f52) == 'c1' - assert minimax_decision('D', f52) == 'd3' +def test_minmax_decision(): + assert minmax_decision('A', f52) == 'a1' + assert minmax_decision('B', f52) == 'b1' + assert minmax_decision('C', f52) == 'c1' + assert minmax_decision('D', f52) == 'd3' -def test_alphabeta_search(): - assert alphabeta_search('A', f52) == 'a1' - assert alphabeta_search('B', f52) == 'b1' - assert alphabeta_search('C', f52) == 'c1' - assert alphabeta_search('D', f52) == 'd3' +def test_alpha_beta_search(): + assert alpha_beta_search('A', f52) == 'a1' + assert alpha_beta_search('B', f52) == 'b1' + assert alpha_beta_search('C', f52) == 'c1' + assert alpha_beta_search('D', f52) == 'd3' state = gen_state(to_move='X', x_positions=[(1, 1), (3, 3)], o_positions=[(1, 2), (3, 2)]) - assert alphabeta_search(state, ttt) == (2, 2) + assert alpha_beta_search(state, ttt) == (2, 2) state = gen_state(to_move='O', x_positions=[(1, 1), (3, 1), (3, 3)], o_positions=[(1, 2), (3, 2)]) - assert alphabeta_search(state, ttt) == (2, 2) + assert alpha_beta_search(state, ttt) == (2, 2) state = gen_state(to_move='O', x_positions=[(1, 1)], o_positions=[]) - assert alphabeta_search(state, ttt) == (2, 2) + assert alpha_beta_search(state, ttt) == (2, 2) state = gen_state(to_move='X', x_positions=[(1, 1), (3, 1)], o_positions=[(2, 2), (3, 1)]) - assert alphabeta_search(state, ttt) == (1, 3) + assert alpha_beta_search(state, ttt) == (1, 3) def test_random_tests(): - assert Fig52Game().play_game(alphabeta_player, alphabeta_player) == 3 + assert Fig52Game().play_game(alpha_beta_player, alpha_beta_player) == 3 # The player 'X' (one who plays first) in TicTacToe never loses: - assert ttt.play_game(alphabeta_player, alphabeta_player) >= 0 + assert ttt.play_game(alpha_beta_player, alpha_beta_player) >= 0 # The player 'X' (one who plays first) in TicTacToe never loses: - assert ttt.play_game(alphabeta_player, random_player) >= 0 + assert ttt.play_game(alpha_beta_player, random_player) >= 0 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_games4e.py b/tests/test_games4e.py new file mode 100644 index 000000000..7dfa47f11 --- /dev/null +++ b/tests/test_games4e.py @@ -0,0 +1,96 @@ +import pytest + +from games4e import * + +# Creating the game instances +f52 = Fig52Game() +ttt = TicTacToe() +con4 = ConnectFour() + +random.seed("aima-python") + + +def gen_state(to_move='X', x_positions=[], o_positions=[], h=3, v=3): + """Given whose turn it is to move, the positions of X's on the board, the + positions of O's on the board, and, (optionally) number of rows, columns + and how many consecutive X's or O's required to win, return the corresponding + game state""" + + moves = set([(x, y) for x in range(1, h + 1) for y in range(1, v + 1)]) - set(x_positions) - set(o_positions) + moves = list(moves) + board = {} + for pos in x_positions: + board[pos] = 'X' + for pos in o_positions: + board[pos] = 'O' + return GameState(to_move=to_move, utility=0, board=board, moves=moves) + + +def test_minmax_decision(): + assert minmax_decision('A', f52) == 'a1' + assert minmax_decision('B', f52) == 'b1' + assert minmax_decision('C', f52) == 'c1' + assert minmax_decision('D', f52) == 'd3' + + +def test_alpha_beta_search(): + assert alpha_beta_search('A', f52) == 'a1' + assert alpha_beta_search('B', f52) == 'b1' + assert alpha_beta_search('C', f52) == 'c1' + assert alpha_beta_search('D', f52) == 'd3' + + state = gen_state(to_move='X', x_positions=[(1, 1), (3, 3)], + o_positions=[(1, 2), (3, 2)]) + assert alpha_beta_search(state, ttt) == (2, 2) + + state = gen_state(to_move='O', x_positions=[(1, 1), (3, 1), (3, 3)], + o_positions=[(1, 2), (3, 2)]) + assert alpha_beta_search(state, ttt) == (2, 2) + + state = gen_state(to_move='O', x_positions=[(1, 1)], + o_positions=[]) + assert alpha_beta_search(state, ttt) == (2, 2) + + state = gen_state(to_move='X', x_positions=[(1, 1), (3, 1)], + o_positions=[(2, 2), (3, 1)]) + assert alpha_beta_search(state, ttt) == (1, 3) + + +def test_monte_carlo_tree_search(): + state = gen_state(to_move='X', x_positions=[(1, 1), (3, 3)], + o_positions=[(1, 2), (3, 2)]) + assert monte_carlo_tree_search(state, ttt) == (2, 2) + + state = gen_state(to_move='O', x_positions=[(1, 1), (3, 1), (3, 3)], + o_positions=[(1, 2), (3, 2)]) + assert monte_carlo_tree_search(state, ttt) == (2, 2) + + # uncomment the following when removing the 3rd edition + # state = gen_state(to_move='O', x_positions=[(1, 1)], + # o_positions=[]) + # assert monte_carlo_tree_search(state, ttt) == (2, 2) + + state = gen_state(to_move='X', x_positions=[(1, 1), (3, 1)], + o_positions=[(2, 2), (3, 1)]) + assert monte_carlo_tree_search(state, ttt) == (1, 3) + + # should never lose to a random or alpha_beta player in a ttt game + assert ttt.play_game(mcts_player, random_player) >= 0 + assert ttt.play_game(mcts_player, alpha_beta_player) >= 0 + + # should never lose to a random player in a connect four game + assert con4.play_game(mcts_player, random_player) >= 0 + + +def test_random_tests(): + assert Fig52Game().play_game(alpha_beta_player, alpha_beta_player) == 3 + + # The player 'X' (one who plays first) in TicTacToe never loses: + assert ttt.play_game(alpha_beta_player, alpha_beta_player) >= 0 + + # The player 'X' (one who plays first) in TicTacToe never loses: + assert ttt.play_game(alpha_beta_player, random_player) >= 0 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_knowledge.py b/tests/test_knowledge.py index ab86089ae..d3829de02 100644 --- a/tests/test_knowledge.py +++ b/tests/test_knowledge.py @@ -1,16 +1,15 @@ +import pytest + from knowledge import * from utils import expr import random random.seed("aima-python") - - party = [ {'Pizza': 'Yes', 'Soda': 'No', 'GOAL': True}, {'Pizza': 'Yes', 'Soda': 'Yes', 'GOAL': True}, - {'Pizza': 'No', 'Soda': 'No', 'GOAL': False} -] + {'Pizza': 'No', 'Soda': 'No', 'GOAL': False}] animals_umbrellas = [ {'Species': 'Cat', 'Rain': 'Yes', 'Coat': 'No', 'GOAL': True}, @@ -19,8 +18,7 @@ {'Species': 'Dog', 'Rain': 'Yes', 'Coat': 'No', 'GOAL': False}, {'Species': 'Dog', 'Rain': 'No', 'Coat': 'No', 'GOAL': False}, {'Species': 'Cat', 'Rain': 'No', 'Coat': 'No', 'GOAL': False}, - {'Species': 'Cat', 'Rain': 'No', 'Coat': 'Yes', 'GOAL': True} -] + {'Species': 'Cat', 'Rain': 'No', 'Coat': 'Yes', 'GOAL': True}] conductance = [ {'Sample': 'S1', 'Mass': 12, 'Temp': 26, 'Material': 'Cu', 'Size': 3, 'GOAL': 0.59}, @@ -31,13 +29,13 @@ {'Sample': 'S4', 'Mass': 18, 'Temp': 100, 'Material': 'Pb', 'Size': 3, 'GOAL': 0.04}, {'Sample': 'S4', 'Mass': 18, 'Temp': 100, 'Material': 'Pb', 'Size': 3, 'GOAL': 0.04}, {'Sample': 'S5', 'Mass': 24, 'Temp': 100, 'Material': 'Pb', 'Size': 4, 'GOAL': 0.04}, - {'Sample': 'S6', 'Mass': 36, 'Temp': 26, 'Material': 'Pb', 'Size': 6, 'GOAL': 0.05}, -] + {'Sample': 'S6', 'Mass': 36, 'Temp': 26, 'Material': 'Pb', 'Size': 6, 'GOAL': 0.05}] + def r_example(Alt, Bar, Fri, Hun, Pat, Price, Rain, Res, Type, Est, GOAL): - return {'Alt': Alt, 'Bar': Bar, 'Fri': Fri, 'Hun': Hun, 'Pat': Pat, - 'Price': Price, 'Rain': Rain, 'Res': Res, 'Type': Type, 'Est': Est, - 'GOAL': GOAL} + return {'Alt': Alt, 'Bar': Bar, 'Fri': Fri, 'Hun': Hun, 'Pat': Pat, 'Price': Price, + 'Rain': Rain, 'Res': Res, 'Type': Type, 'Est': Est, 'GOAL': GOAL} + restaurant = [ r_example('Yes', 'No', 'No', 'Yes', 'Some', '$$$', 'No', 'Yes', 'French', '0-10', True), @@ -51,35 +49,28 @@ def r_example(Alt, Bar, Fri, Hun, Pat, Price, Rain, Res, Type, Est, GOAL): r_example('No', 'Yes', 'Yes', 'No', 'Full', '$', 'Yes', 'No', 'Burger', '>60', False), r_example('Yes', 'Yes', 'Yes', 'Yes', 'Full', '$$$', 'No', 'Yes', 'Italian', '10-30', False), r_example('No', 'No', 'No', 'No', 'None', '$', 'No', 'No', 'Thai', '0-10', False), - r_example('Yes', 'Yes', 'Yes', 'Yes', 'Full', '$', 'No', 'No', 'Burger', '30-60', True) -] + r_example('Yes', 'Yes', 'Yes', 'Yes', 'Full', '$', 'No', 'No', 'Burger', '30-60', True)] def test_current_best_learning(): examples = restaurant hypothesis = [{'Alt': 'Yes'}] h = current_best_learning(examples, hypothesis) - values = [] - for e in examples: - values.append(guess_value(e, h)) + values = [guess_value(e, h) for e in examples] assert values == [True, False, True, True, False, True, False, True, False, False, False, True] examples = animals_umbrellas initial_h = [{'Species': 'Cat'}] h = current_best_learning(examples, initial_h) - values = [] - for e in examples: - values.append(guess_value(e, h)) + values = [guess_value(e, h) for e in examples] assert values == [True, True, True, False, False, False, True] examples = party initial_h = [{'Pizza': 'Yes'}] h = current_best_learning(examples, initial_h) - values = [] - for e in examples: - values.append(guess_value(e, h)) + values = [guess_value(e, h) for e in examples] assert values == [True, True, False] @@ -111,65 +102,61 @@ def test_minimal_consistent_det(): A, B, C, D, E, F, G, H, I, x, y, z = map(expr, 'ABCDEFGHIxyz') # knowledge base containing family relations -small_family = FOIL_container([expr("Mother(Anne, Peter)"), - expr("Mother(Anne, Zara)"), - expr("Mother(Sarah, Beatrice)"), - expr("Mother(Sarah, Eugenie)"), - expr("Father(Mark, Peter)"), - expr("Father(Mark, Zara)"), - expr("Father(Andrew, Beatrice)"), - expr("Father(Andrew, Eugenie)"), - expr("Father(Philip, Anne)"), - expr("Father(Philip, Andrew)"), - expr("Mother(Elizabeth, Anne)"), - expr("Mother(Elizabeth, Andrew)"), - expr("Male(Philip)"), - expr("Male(Mark)"), - expr("Male(Andrew)"), - expr("Male(Peter)"), - expr("Female(Elizabeth)"), - expr("Female(Anne)"), - expr("Female(Sarah)"), - expr("Female(Zara)"), - expr("Female(Beatrice)"), - expr("Female(Eugenie)"), -]) - -smaller_family = FOIL_container([expr("Mother(Anne, Peter)"), - expr("Father(Mark, Peter)"), - expr("Father(Philip, Anne)"), - expr("Mother(Elizabeth, Anne)"), - expr("Male(Philip)"), - expr("Male(Mark)"), - expr("Male(Peter)"), - expr("Female(Elizabeth)"), - expr("Female(Anne)") - ]) - +small_family = FOILContainer([expr("Mother(Anne, Peter)"), + expr("Mother(Anne, Zara)"), + expr("Mother(Sarah, Beatrice)"), + expr("Mother(Sarah, Eugenie)"), + expr("Father(Mark, Peter)"), + expr("Father(Mark, Zara)"), + expr("Father(Andrew, Beatrice)"), + expr("Father(Andrew, Eugenie)"), + expr("Father(Philip, Anne)"), + expr("Father(Philip, Andrew)"), + expr("Mother(Elizabeth, Anne)"), + expr("Mother(Elizabeth, Andrew)"), + expr("Male(Philip)"), + expr("Male(Mark)"), + expr("Male(Andrew)"), + expr("Male(Peter)"), + expr("Female(Elizabeth)"), + expr("Female(Anne)"), + expr("Female(Sarah)"), + expr("Female(Zara)"), + expr("Female(Beatrice)"), + expr("Female(Eugenie)")]) + +smaller_family = FOILContainer([expr("Mother(Anne, Peter)"), + expr("Father(Mark, Peter)"), + expr("Father(Philip, Anne)"), + expr("Mother(Elizabeth, Anne)"), + expr("Male(Philip)"), + expr("Male(Mark)"), + expr("Male(Peter)"), + expr("Female(Elizabeth)"), + expr("Female(Anne)")]) # target relation target = expr('Parent(x, y)') -#positive examples of target +# positive examples of target examples_pos = [{x: expr('Elizabeth'), y: expr('Anne')}, - {x: expr('Elizabeth'), y: expr('Andrew')}, - {x: expr('Philip'), y: expr('Anne')}, - {x: expr('Philip'), y: expr('Andrew')}, - {x: expr('Anne'), y: expr('Peter')}, - {x: expr('Anne'), y: expr('Zara')}, - {x: expr('Mark'), y: expr('Peter')}, - {x: expr('Mark'), y: expr('Zara')}, - {x: expr('Andrew'), y: expr('Beatrice')}, - {x: expr('Andrew'), y: expr('Eugenie')}, - {x: expr('Sarah'), y: expr('Beatrice')}, - {x: expr('Sarah'), y: expr('Eugenie')}] + {x: expr('Elizabeth'), y: expr('Andrew')}, + {x: expr('Philip'), y: expr('Anne')}, + {x: expr('Philip'), y: expr('Andrew')}, + {x: expr('Anne'), y: expr('Peter')}, + {x: expr('Anne'), y: expr('Zara')}, + {x: expr('Mark'), y: expr('Peter')}, + {x: expr('Mark'), y: expr('Zara')}, + {x: expr('Andrew'), y: expr('Beatrice')}, + {x: expr('Andrew'), y: expr('Eugenie')}, + {x: expr('Sarah'), y: expr('Beatrice')}, + {x: expr('Sarah'), y: expr('Eugenie')}] # negative examples of target examples_neg = [{x: expr('Anne'), y: expr('Eugenie')}, - {x: expr('Beatrice'), y: expr('Eugenie')}, - {x: expr('Mark'), y: expr('Elizabeth')}, - {x: expr('Beatrice'), y: expr('Philip')}] - + {x: expr('Beatrice'), y: expr('Eugenie')}, + {x: expr('Mark'), y: expr('Elizabeth')}, + {x: expr('Beatrice'), y: expr('Philip')}] def test_tell(): @@ -179,10 +166,11 @@ def test_tell(): smaller_family.tell(expr("Male(George)")) smaller_family.tell(expr("Female(Mum)")) assert smaller_family.ask(expr("Male(George)")) == {} - assert smaller_family.ask(expr("Female(Mum)"))=={} + assert smaller_family.ask(expr("Female(Mum)")) == {} assert not smaller_family.ask(expr("Female(George)")) assert not smaller_family.ask(expr("Male(Mum)")) + def test_extend_example(): """ Create the extended examples of the given clause. @@ -198,12 +186,13 @@ def test_new_literals(): assert len(list(small_family.new_literals([expr('p'), []]))) == 8 assert len(list(small_family.new_literals([expr('p & q'), []]))) == 20 + def test_new_clause(): """ Finds the best clause to add in the set of clauses. """ clause = small_family.new_clause([examples_pos, examples_neg], target)[0][1] - assert len(clause) == 1 and ( clause[0].op in ['Male', 'Female', 'Father', 'Mother' ] ) + assert len(clause) == 1 and (clause[0].op in ['Male', 'Female', 'Father', 'Mother']) def test_choose_literal(): @@ -224,69 +213,73 @@ def test_gain(): """ Calculates the utility of each literal, based on the information gained. """ - gain_father = small_family.gain( expr('Father(x,y)'), [examples_pos, examples_neg] ) - gain_male = small_family.gain(expr('Male(x)'), [examples_pos, examples_neg] ) + gain_father = small_family.gain(expr('Father(x,y)'), [examples_pos, examples_neg]) + gain_male = small_family.gain(expr('Male(x)'), [examples_pos, examples_neg]) assert round(gain_father, 2) == 2.49 - assert round(gain_male, 2) == 1.16 + assert round(gain_male, 2) == 1.16 + def test_update_examples(): """Add to the kb those examples what are represented in extended_examples List of omitted examples is returned. """ - extended_examples = [{x: expr("Mark") , y: expr("Peter")}, - {x: expr("Philip"), y: expr("Anne")} ] - + extended_examples = [{x: expr("Mark"), y: expr("Peter")}, + {x: expr("Philip"), y: expr("Anne")}] + uncovered = smaller_family.update_examples(target, examples_pos, extended_examples) - assert {x: expr("Elizabeth"), y: expr("Anne") } in uncovered + assert {x: expr("Elizabeth"), y: expr("Anne")} in uncovered assert {x: expr("Anne"), y: expr("Peter")} in uncovered - assert {x: expr("Philip"), y: expr("Anne") } not in uncovered + assert {x: expr("Philip"), y: expr("Anne")} not in uncovered assert {x: expr("Mark"), y: expr("Peter")} not in uncovered - def test_foil(): """ Test the FOIL algorithm, when target is Parent(x,y) """ clauses = small_family.foil([examples_pos, examples_neg], target) assert len(clauses) == 2 and \ - ((clauses[0][1][0] == expr('Father(x, y)') and clauses[1][1][0] == expr('Mother(x, y)')) or \ - (clauses[1][1][0] == expr('Father(x, y)') and clauses[0][1][0] == expr('Mother(x, y)'))) + ((clauses[0][1][0] == expr('Father(x, y)') and clauses[1][1][0] == expr('Mother(x, y)')) or + (clauses[1][1][0] == expr('Father(x, y)') and clauses[0][1][0] == expr('Mother(x, y)'))) target_g = expr('Grandparent(x, y)') examples_pos_g = [{x: expr('Elizabeth'), y: expr('Peter')}, - {x: expr('Elizabeth'), y: expr('Zara')}, - {x: expr('Elizabeth'), y: expr('Beatrice')}, - {x: expr('Elizabeth'), y: expr('Eugenie')}, - {x: expr('Philip'), y: expr('Peter')}, - {x: expr('Philip'), y: expr('Zara')}, - {x: expr('Philip'), y: expr('Beatrice')}, - {x: expr('Philip'), y: expr('Eugenie')}] + {x: expr('Elizabeth'), y: expr('Zara')}, + {x: expr('Elizabeth'), y: expr('Beatrice')}, + {x: expr('Elizabeth'), y: expr('Eugenie')}, + {x: expr('Philip'), y: expr('Peter')}, + {x: expr('Philip'), y: expr('Zara')}, + {x: expr('Philip'), y: expr('Beatrice')}, + {x: expr('Philip'), y: expr('Eugenie')}] examples_neg_g = [{x: expr('Anne'), y: expr('Eugenie')}, - {x: expr('Beatrice'), y: expr('Eugenie')}, - {x: expr('Elizabeth'), y: expr('Andrew')}, - {x: expr('Elizabeth'), y: expr('Anne')}, - {x: expr('Elizabeth'), y: expr('Mark')}, - {x: expr('Elizabeth'), y: expr('Sarah')}, - {x: expr('Philip'), y: expr('Anne')}, - {x: expr('Philip'), y: expr('Andrew')}, - {x: expr('Anne'), y: expr('Peter')}, - {x: expr('Anne'), y: expr('Zara')}, - {x: expr('Mark'), y: expr('Peter')}, - {x: expr('Mark'), y: expr('Zara')}, - {x: expr('Andrew'), y: expr('Beatrice')}, - {x: expr('Andrew'), y: expr('Eugenie')}, - {x: expr('Sarah'), y: expr('Beatrice')}, - {x: expr('Mark'), y: expr('Elizabeth')}, - {x: expr('Beatrice'), y: expr('Philip')}, - {x: expr('Peter'), y: expr('Andrew')}, - {x: expr('Zara'), y: expr('Mark')}, - {x: expr('Peter'), y: expr('Anne')}, - {x: expr('Zara'), y: expr('Eugenie')}] + {x: expr('Beatrice'), y: expr('Eugenie')}, + {x: expr('Elizabeth'), y: expr('Andrew')}, + {x: expr('Elizabeth'), y: expr('Anne')}, + {x: expr('Elizabeth'), y: expr('Mark')}, + {x: expr('Elizabeth'), y: expr('Sarah')}, + {x: expr('Philip'), y: expr('Anne')}, + {x: expr('Philip'), y: expr('Andrew')}, + {x: expr('Anne'), y: expr('Peter')}, + {x: expr('Anne'), y: expr('Zara')}, + {x: expr('Mark'), y: expr('Peter')}, + {x: expr('Mark'), y: expr('Zara')}, + {x: expr('Andrew'), y: expr('Beatrice')}, + {x: expr('Andrew'), y: expr('Eugenie')}, + {x: expr('Sarah'), y: expr('Beatrice')}, + {x: expr('Mark'), y: expr('Elizabeth')}, + {x: expr('Beatrice'), y: expr('Philip')}, + {x: expr('Peter'), y: expr('Andrew')}, + {x: expr('Zara'), y: expr('Mark')}, + {x: expr('Peter'), y: expr('Anne')}, + {x: expr('Zara'), y: expr('Eugenie')}] clauses = small_family.foil([examples_pos_g, examples_neg_g], target_g) - assert len(clauses[0]) == 2 - assert clauses[0][1][0].op == 'Parent' - assert clauses[0][1][0].args[0] == x + assert len(clauses[0]) == 2 + assert clauses[0][1][0].op == 'Parent' + assert clauses[0][1][0].args[0] == x assert clauses[0][1][1].op == 'Parent' assert clauses[0][1][1].args[1] == y + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_learning.py b/tests/test_learning.py index ec3a2f188..63a7fd9aa 100644 --- a/tests/test_learning.py +++ b/tests/test_learning.py @@ -1,74 +1,18 @@ import pytest -import math -import random -from utils import open_data -from learning import * +from learning import * random.seed("aima-python") -def test_euclidean(): - distance = euclidean_distance([1, 2], [3, 4]) - assert round(distance, 2) == 2.83 - - distance = euclidean_distance([1, 2, 3], [4, 5, 6]) - assert round(distance, 2) == 5.2 - - distance = euclidean_distance([0, 0, 0], [0, 0, 0]) - assert distance == 0 - -def test_cross_entropy(): - loss = cross_entropy_loss([1,0], [0.9, 0.3]) - assert round(loss,2) == 0.23 - - loss = cross_entropy_loss([1,0,0,1], [0.9,0.3,0.5,0.75]) - assert round(loss,2) == 0.36 - - loss = cross_entropy_loss([1,0,0,1,1,0,1,1], [0.9,0.3,0.5,0.75,0.85,0.14,0.93,0.79]) - assert round(loss,2) == 0.26 - - -def test_rms_error(): - assert rms_error([2, 2], [2, 2]) == 0 - assert rms_error((0, 0), (0, 1)) == math.sqrt(0.5) - assert rms_error((1, 0), (0, 1)) == 1 - assert rms_error((0, 0), (0, -1)) == math.sqrt(0.5) - assert rms_error((0, 0.5), (0, -0.5)) == math.sqrt(0.5) - - -def test_manhattan_distance(): - assert manhattan_distance([2, 2], [2, 2]) == 0 - assert manhattan_distance([0, 0], [0, 1]) == 1 - assert manhattan_distance([1, 0], [0, 1]) == 2 - assert manhattan_distance([0, 0], [0, -1]) == 1 - assert manhattan_distance([0, 0.5], [0, -0.5]) == 1 - - -def test_mean_boolean_error(): - assert mean_boolean_error([1, 1], [0, 0]) == 1 - assert mean_boolean_error([0, 1], [1, 0]) == 1 - assert mean_boolean_error([1, 1], [0, 1]) == 0.5 - assert mean_boolean_error([0, 0], [0, 0]) == 0 - assert mean_boolean_error([1, 1], [1, 1]) == 0 - - -def test_mean_error(): - assert mean_error([2, 2], [2, 2]) == 0 - assert mean_error([0, 0], [0, 1]) == 0.5 - assert mean_error([1, 0], [0, 1]) == 1 - assert mean_error([0, 0], [0, -1]) == 0.5 - assert mean_error([0, 0.5], [0, -0.5]) == 0.5 - - def test_exclude(): iris = DataSet(name='iris', exclude=[3]) assert iris.inputs == [0, 1, 2] def test_parse_csv(): - Iris = open_data('iris.csv').read() - assert parse_csv(Iris)[0] == [5.1, 3.5, 1.4, 0.2, 'setosa'] + iris = open_data('iris.csv').read() + assert parse_csv(iris)[0] == [5.1, 3.5, 1.4, 0.2, 'setosa'] def test_weighted_mode(): @@ -80,99 +24,55 @@ def test_weighted_replicate(): def test_means_and_deviation(): - iris = DataSet(name="iris") - + iris = DataSet(name='iris') means, deviations = iris.find_means_and_deviations() - - assert round(means["setosa"][0], 3) == 5.006 - assert round(means["versicolor"][0], 3) == 5.936 - assert round(means["virginica"][0], 3) == 6.588 - - assert round(deviations["setosa"][0], 3) == 0.352 - assert round(deviations["versicolor"][0], 3) == 0.516 - assert round(deviations["virginica"][0], 3) == 0.636 + assert round(means['setosa'][0], 3) == 5.006 + assert round(means['versicolor'][0], 3) == 5.936 + assert round(means['virginica'][0], 3) == 6.588 + assert round(deviations['setosa'][0], 3) == 0.352 + assert round(deviations['versicolor'][0], 3) == 0.516 + assert round(deviations['virginica'][0], 3) == 0.636 def test_plurality_learner(): - zoo = DataSet(name="zoo") - - pL = PluralityLearner(zoo) - assert pL([1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 4, 1, 0, 1]) == "mammal" - - -def test_naive_bayes(): - iris = DataSet(name="iris") - - # Discrete - nBD = NaiveBayesLearner(iris, continuous=False) - assert nBD([5, 3, 1, 0.1]) == "setosa" - assert nBD([6, 3, 4, 1.1]) == "versicolor" - assert nBD([7.7, 3, 6, 2]) == "virginica" - - # Continuous - nBC = NaiveBayesLearner(iris, continuous=True) - assert nBC([5, 3, 1, 0.1]) == "setosa" - assert nBC([6, 5, 3, 1.5]) == "versicolor" - assert nBC([7, 3, 6.5, 2]) == "virginica" - - # Simple - data1 = 'a'*50 + 'b'*30 + 'c'*15 - dist1 = CountingProbDist(data1) - data2 = 'a'*30 + 'b'*45 + 'c'*20 - dist2 = CountingProbDist(data2) - data3 = 'a'*20 + 'b'*20 + 'c'*35 - dist3 = CountingProbDist(data3) - - dist = {('First', 0.5): dist1, ('Second', 0.3): dist2, ('Third', 0.2): dist3} - nBS = NaiveBayesLearner(dist, simple=True) - assert nBS('aab') == 'First' - assert nBS(['b', 'b']) == 'Second' - assert nBS('ccbcc') == 'Third' + zoo = DataSet(name='zoo') + pl = PluralityLearner(zoo) + assert pl([1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 4, 1, 0, 1]) == 'mammal' def test_k_nearest_neighbors(): - iris = DataSet(name="iris") - kNN = NearestNeighborLearner(iris, k=3) - assert kNN([5, 3, 1, 0.1]) == "setosa" - assert kNN([5, 3, 1, 0.1]) == "setosa" - assert kNN([6, 5, 3, 1.5]) == "versicolor" - assert kNN([7.5, 4, 6, 2]) == "virginica" - - -def test_truncated_svd(): - test_mat = [[17, 0], - [0, 11]] - _, _, eival = truncated_svd(test_mat) - assert isclose(abs(eival[0]), 17) - assert isclose(abs(eival[1]), 11) - - test_mat = [[17, 0], - [0, -34]] - _, _, eival = truncated_svd(test_mat) - assert isclose(abs(eival[0]), 34) - assert isclose(abs(eival[1]), 17) - - test_mat = [[1, 0, 0, 0, 2], - [0, 0, 3, 0, 0], - [0, 0, 0, 0, 0], - [0, 2, 0, 0, 0]] - _, _, eival = truncated_svd(test_mat) - assert isclose(abs(eival[0]), 3) - assert isclose(abs(eival[1]), 5**0.5) - - test_mat = [[3, 2, 2], - [2, 3, -2]] - _, _, eival = truncated_svd(test_mat) - assert isclose(abs(eival[0]), 5) - assert isclose(abs(eival[1]), 3) + iris = DataSet(name='iris') + knn = NearestNeighborLearner(iris, k=3) + assert knn([5, 3, 1, 0.1]) == 'setosa' + assert knn([6, 5, 3, 1.5]) == 'versicolor' + assert knn([7.5, 4, 6, 2]) == 'virginica' def test_decision_tree_learner(): - iris = DataSet(name="iris") - dTL = DecisionTreeLearner(iris) - assert dTL([5, 3, 1, 0.1]) == "setosa" - assert dTL([6, 5, 3, 1.5]) == "versicolor" - assert dTL([7.5, 4, 6, 2]) == "virginica" + iris = DataSet(name='iris') + dtl = DecisionTreeLearner(iris) + assert dtl([5, 3, 1, 0.1]) == 'setosa' + assert dtl([6, 5, 3, 1.5]) == 'versicolor' + assert dtl([7.5, 4, 6, 2]) == 'virginica' + + +def test_svc(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + n_samples, n_features = len(iris.examples), iris.target + X, y = (np.array([x[:n_features] for x in iris.examples]), + np.array([x[n_features] for x in iris.examples])) + svm = MultiClassLearner(SVC()).fit(X, y) + assert svm.predict([[5.0, 3.1, 0.9, 0.1]]) == 0 + assert svm.predict([[5.1, 3.5, 1.0, 0.0]]) == 0 + assert svm.predict([[4.9, 3.3, 1.1, 0.1]]) == 0 + assert svm.predict([[6.0, 3.0, 4.0, 1.1]]) == 1 + assert svm.predict([[6.1, 2.2, 3.5, 1.0]]) == 1 + assert svm.predict([[5.9, 2.5, 3.3, 1.1]]) == 1 + assert svm.predict([[7.5, 4.1, 6.2, 2.3]]) == 2 + assert svm.predict([[7.3, 4.0, 6.1, 2.4]]) == 2 + assert svm.predict([[7.0, 3.3, 6.1, 2.5]]) == 2 def test_information_content(): @@ -185,22 +85,22 @@ def test_information_content(): def test_random_forest(): - iris = DataSet(name="iris") - rF = RandomForest(iris) - tests = [([5.0, 3.0, 1.0, 0.1], "setosa"), - ([5.1, 3.3, 1.1, 0.1], "setosa"), - ([6.0, 5.0, 3.0, 1.0], "versicolor"), - ([6.1, 2.2, 3.5, 1.0], "versicolor"), - ([7.5, 4.1, 6.2, 2.3], "virginica"), - ([7.3, 3.7, 6.1, 2.5], "virginica")] - assert grade_learner(rF, tests) >= 1/3 + iris = DataSet(name='iris') + rf = RandomForest(iris) + tests = [([5.0, 3.0, 1.0, 0.1], 'setosa'), + ([5.1, 3.3, 1.1, 0.1], 'setosa'), + ([6.0, 5.0, 3.0, 1.0], 'versicolor'), + ([6.1, 2.2, 3.5, 1.0], 'versicolor'), + ([7.5, 4.1, 6.2, 2.3], 'virginica'), + ([7.3, 3.7, 6.1, 2.5], 'virginica')] + assert grade_learner(rf, tests) >= 1 / 3 def test_neural_network_learner(): - iris = DataSet(name="iris") - classes = ["setosa", "versicolor", "virginica"] + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] iris.classes_to_numbers(classes) - nNL = NeuralNetLearner(iris, [5], 0.15, 75) + nnl = NeuralNetLearner(iris, [5], 0.15, 75) tests = [([5.0, 3.1, 0.9, 0.1], 0), ([5.1, 3.5, 1.0, 0.0], 0), ([4.9, 3.3, 1.1, 0.1], 0), @@ -210,23 +110,22 @@ def test_neural_network_learner(): ([7.5, 4.1, 6.2, 2.3], 2), ([7.3, 4.0, 6.1, 2.4], 2), ([7.0, 3.3, 6.1, 2.5], 2)] - assert grade_learner(nNL, tests) >= 1/3 - assert err_ratio(nNL, iris) < 0.21 + assert grade_learner(nnl, tests) >= 1 / 3 + assert err_ratio(nnl, iris) < 0.21 def test_perceptron(): - iris = DataSet(name="iris") + iris = DataSet(name='iris') iris.classes_to_numbers() - classes_number = len(iris.values[iris.target]) - perceptron = PerceptronLearner(iris) + pl = PerceptronLearner(iris) tests = [([5, 3, 1, 0.1], 0), ([5, 3.5, 1, 0], 0), ([6, 3, 4, 1.1], 1), ([6, 2, 3.5, 1], 1), ([7.5, 4, 6, 2], 2), ([7, 3, 6, 2.5], 2)] - assert grade_learner(perceptron, tests) > 1/2 - assert err_ratio(perceptron, iris) < 0.4 + assert grade_learner(pl, tests) > 1 / 2 + assert err_ratio(pl, iris) < 0.4 def test_random_weights(): @@ -236,20 +135,23 @@ def test_random_weights(): test_weights = random_weights(min_value, max_value, num_weights) assert len(test_weights) == num_weights for weight in test_weights: - assert weight >= min_value and weight <= max_value + assert min_value <= weight <= max_value -def test_adaboost(): - iris = DataSet(name="iris") +def test_ada_boost(): + iris = DataSet(name='iris') iris.classes_to_numbers() - WeightedPerceptron = WeightedLearner(PerceptronLearner) - AdaboostLearner = AdaBoost(WeightedPerceptron, 5) - adaboost = AdaboostLearner(iris) + wl = WeightedLearner(PerceptronLearner) + ab = ada_boost(iris, wl, 5) tests = [([5, 3, 1, 0.1], 0), ([5, 3.5, 1, 0], 0), ([6, 3, 4, 1.1], 1), ([6, 2, 3.5, 1], 1), ([7.5, 4, 6, 2], 2), ([7, 3, 6, 2.5], 2)] - assert grade_learner(adaboost, tests) > 4/6 - assert err_ratio(adaboost, iris) < 0.25 + assert grade_learner(ab, tests) > 2 / 3 + assert err_ratio(ab, iris) < 0.25 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_learning4e.py b/tests/test_learning4e.py new file mode 100644 index 000000000..b345efad7 --- /dev/null +++ b/tests/test_learning4e.py @@ -0,0 +1,127 @@ +import pytest + +from deep_learning4e import PerceptronLearner +from learning4e import * + +random.seed("aima-python") + + +def test_exclude(): + iris = DataSet(name='iris', exclude=[3]) + assert iris.inputs == [0, 1, 2] + + +def test_parse_csv(): + iris = open_data('iris.csv').read() + assert parse_csv(iris)[0] == [5.1, 3.5, 1.4, 0.2, 'setosa'] + + +def test_weighted_mode(): + assert weighted_mode('abbaa', [1, 2, 3, 1, 2]) == 'b' + + +def test_weighted_replicate(): + assert weighted_replicate('ABC', [1, 2, 1], 4) == ['A', 'B', 'B', 'C'] + + +def test_means_and_deviation(): + iris = DataSet(name='iris') + means, deviations = iris.find_means_and_deviations() + assert round(means['setosa'][0], 3) == 5.006 + assert round(means['versicolor'][0], 3) == 5.936 + assert round(means['virginica'][0], 3) == 6.588 + assert round(deviations['setosa'][0], 3) == 0.352 + assert round(deviations['versicolor'][0], 3) == 0.516 + assert round(deviations['virginica'][0], 3) == 0.636 + + +def test_plurality_learner(): + zoo = DataSet(name='zoo') + pl = PluralityLearner(zoo) + assert pl.predict([1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 4, 1, 0, 1]) == 'mammal' + + +def test_k_nearest_neighbors(): + iris = DataSet(name='iris') + knn = NearestNeighborLearner(iris, k=3) + assert knn.predict([5, 3, 1, 0.1]) == 'setosa' + assert knn.predict([6, 5, 3, 1.5]) == 'versicolor' + assert knn.predict([7.5, 4, 6, 2]) == 'virginica' + + +def test_decision_tree_learner(): + iris = DataSet(name='iris') + dtl = DecisionTreeLearner(iris) + assert dtl.predict([5, 3, 1, 0.1]) == 'setosa' + assert dtl.predict([6, 5, 3, 1.5]) == 'versicolor' + assert dtl.predict([7.5, 4, 6, 2]) == 'virginica' + + +def test_svc(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + n_samples, n_features = len(iris.examples), iris.target + X, y = (np.array([x[:n_features] for x in iris.examples]), + np.array([x[n_features] for x in iris.examples])) + svm = MultiClassLearner(SVC()).fit(X, y) + assert svm.predict([[5.0, 3.1, 0.9, 0.1]]) == 0 + assert svm.predict([[5.1, 3.5, 1.0, 0.0]]) == 0 + assert svm.predict([[4.9, 3.3, 1.1, 0.1]]) == 0 + assert svm.predict([[6.0, 3.0, 4.0, 1.1]]) == 1 + assert svm.predict([[6.1, 2.2, 3.5, 1.0]]) == 1 + assert svm.predict([[5.9, 2.5, 3.3, 1.1]]) == 1 + assert svm.predict([[7.5, 4.1, 6.2, 2.3]]) == 2 + assert svm.predict([[7.3, 4.0, 6.1, 2.4]]) == 2 + assert svm.predict([[7.0, 3.3, 6.1, 2.5]]) == 2 + + +def test_information_content(): + assert information_content([]) == 0 + assert information_content([4]) == 0 + assert information_content([5, 4, 0, 2, 5, 0]) > 1.9 + assert information_content([5, 4, 0, 2, 5, 0]) < 2 + assert information_content([1.5, 2.5]) > 0.9 + assert information_content([1.5, 2.5]) < 1.0 + + +def test_random_forest(): + iris = DataSet(name='iris') + rf = RandomForest(iris) + tests = [([5.0, 3.0, 1.0, 0.1], 'setosa'), + ([5.1, 3.3, 1.1, 0.1], 'setosa'), + ([6.0, 5.0, 3.0, 1.0], 'versicolor'), + ([6.1, 2.2, 3.5, 1.0], 'versicolor'), + ([7.5, 4.1, 6.2, 2.3], 'virginica'), + ([7.3, 3.7, 6.1, 2.5], 'virginica')] + assert grade_learner(rf, tests) >= 1 / 3 + + +def test_random_weights(): + min_value = -0.5 + max_value = 0.5 + num_weights = 10 + test_weights = random_weights(min_value, max_value, num_weights) + assert len(test_weights) == num_weights + for weight in test_weights: + assert min_value <= weight <= max_value + + +def test_ada_boost(): + iris = DataSet(name='iris') + classes = ['setosa', 'versicolor', 'virginica'] + iris.classes_to_numbers(classes) + wl = WeightedLearner(PerceptronLearner(iris)) + ab = ada_boost(iris, wl, 5) + tests = [([5, 3, 1, 0.1], 0), + ([5, 3.5, 1, 0], 0), + ([6, 3, 4, 1.1], 1), + ([6, 2, 3.5, 1], 1), + ([7.5, 4, 6, 2], 2), + ([7, 3, 6, 2.5], 2)] + assert grade_learner(ab, tests) > 2 / 3 + assert err_ratio(ab, iris) < 0.25 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_logic.py b/tests/test_logic.py index 378f1f0fc..2ead21746 100644 --- a/tests/test_logic.py +++ b/tests/test_logic.py @@ -1,10 +1,19 @@ import pytest + from logic import * -from utils import expr_handle_infix_ops, count, Symbol +from utils import expr_handle_infix_ops, count + +random.seed("aima-python") definite_clauses_KB = PropDefiniteKB() -for clause in ['(B & F)==>E', '(A & E & F)==>G', '(B & C)==>F', '(A & B)==>D', '(E & F)==>H', '(H & I)==>J', 'A', 'B', 'C']: - definite_clauses_KB.tell(expr(clause)) +for clause in ['(B & F) ==> E', + '(A & E & F) ==> G', + '(B & C) ==> F', + '(A & B) ==> D', + '(E & F) ==> H', + '(H & I) ==> J', + 'A', 'B', 'C']: + definite_clauses_KB.tell(expr(clause)) def test_is_symbol(): @@ -38,8 +47,7 @@ def test_variables(): def test_expr(): assert repr(expr('P <=> Q(1)')) == '(P <=> Q(1))' assert repr(expr('P & Q | ~R(x, F(x))')) == '((P & Q) | ~R(x, F(x)))' - assert (expr_handle_infix_ops('P & Q ==> R & ~S') - == "P & Q |'==>'| R & ~S") + assert expr_handle_infix_ops('P & Q ==> R & ~S') == "P & Q |'==>'| R & ~S" def test_extend(): @@ -47,7 +55,7 @@ def test_extend(): def test_subst(): - assert subst({x: 42, y:0}, F(x) + y) == (F(42) + 0) + assert subst({x: 42, y: 0}, F(x) + y) == (F(42) + 0) def test_PropKB(): @@ -55,11 +63,11 @@ def test_PropKB(): assert count(kb.ask(expr) for expr in [A, C, D, E, Q]) is 0 kb.tell(A & E) assert kb.ask(A) == kb.ask(E) == {} - kb.tell(E |'==>'| C) + kb.tell(E | '==>' | C) assert kb.ask(C) == {} kb.retract(E) - assert kb.ask(E) is False - assert kb.ask(C) is False + assert not kb.ask(E) + assert not kb.ask(C) def test_wumpus_kb(): @@ -70,10 +78,10 @@ def test_wumpus_kb(): assert wumpus_kb.ask(~P12) == {} # Statement: There is a pit in [2,2]. - assert wumpus_kb.ask(P22) is False + assert not wumpus_kb.ask(P22) # Statement: There is a pit in [3,1]. - assert wumpus_kb.ask(P31) is False + assert not wumpus_kb.ask(P31) # Statement: Neither [1,2] nor [2,1] contains a pit. assert wumpus_kb.ask(~P12 & ~P21) == {} @@ -94,16 +102,17 @@ def test_is_definite_clause(): def test_parse_definite_clause(): assert parse_definite_clause(expr('A & B & C & D ==> E')) == ([A, B, C, D], E) assert parse_definite_clause(expr('Farmer(Mac)')) == ([], expr('Farmer(Mac)')) - assert parse_definite_clause(expr('(Farmer(f) & Rabbit(r)) ==> Hates(f, r)')) == ([expr('Farmer(f)'), expr('Rabbit(r)')], expr('Hates(f, r)')) + assert parse_definite_clause(expr('(Farmer(f) & Rabbit(r)) ==> Hates(f, r)')) == ( + [expr('Farmer(f)'), expr('Rabbit(r)')], expr('Hates(f, r)')) def test_pl_true(): assert pl_true(P, {}) is None - assert pl_true(P, {P: False}) is False - assert pl_true(P | Q, {P: True}) is True - assert pl_true((A | B) & (C | D), {A: False, B: True, D: True}) is True - assert pl_true((A & B) & (C | D), {A: False, B: True, D: True}) is False - assert pl_true((A & B) | (A & C), {A: False, B: True, C: True}) is False + assert not pl_true(P, {P: False}) + assert pl_true(P | Q, {P: True}) + assert pl_true((A | B) & (C | D), {A: False, B: True, D: True}) + assert not pl_true((A & B) & (C | D), {A: False, B: True, D: True}) + assert not pl_true((A & B) | (A & C), {A: False, B: True, C: True}) assert pl_true((A | B) & (C | D), {A: True, D: False}) is None assert pl_true(P | P, {}) is None @@ -127,32 +136,43 @@ def test_tt_true(): assert tt_true('(A | (B & C)) <=> ((A | B) & (A | C))') -def test_dpll(): - assert (dpll_satisfiable(A & ~B & C & (A | ~D) & (~E | ~D) & (C | ~D) & (~A | ~F) & (E | ~F) - & (~D | ~F) & (B | ~C | D) & (A | ~E | F) & (~A | E | D)) - == {B: False, C: True, A: True, F: False, D: True, E: False}) - assert dpll_satisfiable(A & B & ~C & D) == {C: False, A: True, D: True, B: True} - assert dpll_satisfiable((A | (B & C)) |'<=>'| ((A | B) & (A | C))) == {C: True, A: True} or {C: True, B: True} - assert dpll_satisfiable(A |'<=>'| B) == {A: True, B: True} +def test_dpll_satisfiable(): + assert dpll_satisfiable(A & ~B & C & (A | ~D) & (~E | ~D) & (C | ~D) & (~A | ~F) & (E | ~F) & (~D | ~F) & + (B | ~C | D) & (A | ~E | F) & (~A | E | D)) == \ + {B: False, C: True, A: True, F: False, D: True, E: False} + assert dpll_satisfiable(A & B & ~C & D) == {C: False, A: True, D: True, B: True} + assert dpll_satisfiable((A | (B & C)) | '<=>' | ((A | B) & (A | C))) == {C: True, A: True} or {C: True, B: True} + assert dpll_satisfiable(A | '<=>' | B) == {A: True, B: True} assert dpll_satisfiable(A & ~B) == {A: True, B: False} assert dpll_satisfiable(P & ~P) is False +def test_cdcl_satisfiable(): + assert cdcl_satisfiable(A & ~B & C & (A | ~D) & (~E | ~D) & (C | ~D) & (~A | ~F) & (E | ~F) & (~D | ~F) & + (B | ~C | D) & (A | ~E | F) & (~A | E | D)) == \ + {B: False, C: True, A: True, F: False, D: True, E: False} + assert cdcl_satisfiable(A & B & ~C & D) == {C: False, A: True, D: True, B: True} + assert cdcl_satisfiable((A | (B & C)) | '<=>' | ((A | B) & (A | C))) == {C: True, A: True} or {C: True, B: True} + assert cdcl_satisfiable(A | '<=>' | B) == {A: True, B: True} + assert cdcl_satisfiable(A & ~B) == {A: True, B: False} + assert cdcl_satisfiable(P & ~P) is False + + def test_find_pure_symbol(): - assert find_pure_symbol([A, B, C], [A|~B,~B|~C,C|A]) == (A, True) - assert find_pure_symbol([A, B, C], [~A|~B,~B|~C,C|A]) == (B, False) - assert find_pure_symbol([A, B, C], [~A|B,~B|~C,C|A]) == (None, None) + assert find_pure_symbol([A, B, C], [A | ~B, ~B | ~C, C | A]) == (A, True) + assert find_pure_symbol([A, B, C], [~A | ~B, ~B | ~C, C | A]) == (B, False) + assert find_pure_symbol([A, B, C], [~A | B, ~B | ~C, C | A]) == (None, None) def test_unit_clause_assign(): - assert unit_clause_assign(A|B|C, {A:True}) == (None, None) - assert unit_clause_assign(B|C, {A:True}) == (None, None) - assert unit_clause_assign(B|~A, {A:True}) == (B, True) + assert unit_clause_assign(A | B | C, {A: True}) == (None, None) + assert unit_clause_assign(B | C, {A: True}) == (None, None) + assert unit_clause_assign(B | ~A, {A: True}) == (B, True) def test_find_unit_clause(): - assert find_unit_clause([A|B|C, B|~C, ~A|~B], {A:True}) == (B, False) - + assert find_unit_clause([A | B | C, B | ~C, ~A | ~B], {A: True}) == (B, False) + def test_unify(): assert unify(x, x, {}) == {} @@ -160,6 +180,28 @@ def test_unify(): assert unify(x & 4 & y, 6 & y & 4, {}) == {x: 6, y: 4} assert unify(expr('A(x)'), expr('A(B)')) == {x: B} assert unify(expr('American(x) & Weapon(B)'), expr('American(A) & Weapon(y)')) == {x: A, y: B} + assert unify(expr('P(F(x,z), G(u, z))'), expr('P(F(y,a), y)')) == {x: G(u, a), z: a, y: G(u, a)} + + # tests for https://github.com/aimacode/aima-python/issues/1053 + # unify(expr('P(A, x, F(G(y)))'), expr('P(z, F(z), F(u))')) + # must return {z: A, x: F(A), u: G(y)} and not {z: A, x: F(z), u: G(y)} + assert unify(expr('P(A, x, F(G(y)))'), expr('P(z, F(z), F(u))')) == {z: A, x: F(A), u: G(y)} + assert unify(expr('P(x, A, F(G(y)))'), expr('P(F(z), z, F(u))')) == {x: F(A), z: A, u: G(y)} + + +def test_unify_mm(): + assert unify_mm(x, x) == {} + assert unify_mm(x, 3) == {x: 3} + assert unify_mm(x & 4 & y, 6 & y & 4) == {x: 6, y: 4} + assert unify_mm(expr('A(x)'), expr('A(B)')) == {x: B} + assert unify_mm(expr('American(x) & Weapon(B)'), expr('American(A) & Weapon(y)')) == {x: A, y: B} + assert unify_mm(expr('P(F(x,z), G(u, z))'), expr('P(F(y,a), y)')) == {x: G(u, a), z: a, y: G(u, a)} + + # tests for https://github.com/aimacode/aima-python/issues/1053 + # unify(expr('P(A, x, F(G(y)))'), expr('P(z, F(z), F(u))')) + # must return {z: A, x: F(A), u: G(y)} and not {z: A, x: F(z), u: G(y)} + assert unify_mm(expr('P(A, x, F(G(y)))'), expr('P(z, F(z), F(u))')) == {z: A, x: F(A), u: G(y)} + assert unify_mm(expr('P(x, A, F(G(y)))'), expr('P(F(z), z, F(u))')) == {x: F(A), z: A, u: G(y)} def test_pl_fc_entails(): @@ -175,9 +217,9 @@ def test_tt_entails(): assert tt_entails(P & Q, Q) assert not tt_entails(P | Q, Q) assert tt_entails(A & (B | C) & E & F & ~(P | Q), A & E & F & ~P & ~Q) - assert not tt_entails(P |'<=>'| Q, Q) - assert tt_entails((P |'==>'| Q) & P, Q) - assert not tt_entails((P |'<=>'| Q) & ~P, Q) + assert not tt_entails(P | '<=>' | Q, Q) + assert tt_entails((P | '==>' | Q) & P, Q) + assert not tt_entails((P | '<=>' | Q) & ~P, Q) def test_prop_symbols(): @@ -218,10 +260,8 @@ def test_dissociate(): def test_associate(): - assert (repr(associate('&', [(A & B), (B | C), (B & C)])) - == '(A & B & (B | C) & B & C)') - assert (repr(associate('|', [A | (B | (C | (A & B)))])) - == '(A | B | C | (A & B))') + assert repr(associate('&', [(A & B), (B | C), (B & C)])) == '(A & B & (B | C) & B & C)' + assert repr(associate('|', [A | (B | (C | (A & B)))])) == '(A | B | C | (A & B))' def test_move_not_inwards(): @@ -231,12 +271,13 @@ def test_move_not_inwards(): def test_distribute_and_over_or(): - def test_entailment(s, has_and = False): + def test_entailment(s, has_and=False): result = distribute_and_over_or(s) if has_and: assert result.op == '&' assert tt_entails(s, result) assert tt_entails(result, s) + test_entailment((A & B) | C, True) test_entailment((A | B) & C, True) test_entailment((A | B) | C, False) @@ -244,16 +285,17 @@ def test_entailment(s, has_and = False): def test_to_cnf(): - assert (repr(to_cnf(wumpus_world_inference & ~expr('~P12'))) == - "((~P12 | B11) & (~P21 | B11) & (P12 | P21 | ~B11) & ~B11 & P12)") + assert repr(to_cnf(wumpus_world_inference & ~expr('~P12'))) == \ + '((~P12 | B11) & (~P21 | B11) & (P12 | P21 | ~B11) & ~B11 & P12)' assert repr(to_cnf((P & Q) | (~P & ~Q))) == '((~P | P) & (~Q | P) & (~P | Q) & (~Q | Q))' assert repr(to_cnf('A <=> B')) == '((A | ~B) & (B | ~A))' - assert repr(to_cnf("B <=> (P1 | P2)")) == '((~P1 | B) & (~P2 | B) & (P1 | P2 | ~B))' + assert repr(to_cnf('B <=> (P1 | P2)')) == '((~P1 | B) & (~P2 | B) & (P1 | P2 | ~B))' assert repr(to_cnf('A <=> (B & C)')) == '((A | ~B | ~C) & (B | ~A) & (C | ~A))' - assert repr(to_cnf("a | (b & c) | d")) == '((b | a | d) & (c | a | d))' - assert repr(to_cnf("A & (B | (D & E))")) == '(A & (D | B) & (E | B))' - assert repr(to_cnf("A | (B | (C | (D & E)))")) == '((D | A | B | C) & (E | A | B | C))' - assert repr(to_cnf('(A <=> ~B) ==> (C | ~D)')) == '((B | ~A | C | ~D) & (A | ~A | C | ~D) & (B | ~B | C | ~D) & (A | ~B | C | ~D))' + assert repr(to_cnf('a | (b & c) | d')) == '((b | a | d) & (c | a | d))' + assert repr(to_cnf('A & (B | (D & E))')) == '(A & (D | B) & (E | B))' + assert repr(to_cnf('A | (B | (C | (D & E)))')) == '((D | A | B | C) & (E | A | B | C))' + assert repr(to_cnf('(A <=> ~B) ==> (C | ~D)')) == \ + '((B | ~A | C | ~D) & (A | ~A | C | ~D) & (B | ~B | C | ~D) & (A | ~B | C | ~D))' def test_pl_resolution(): @@ -269,18 +311,16 @@ def test_pl_resolution(): def test_standardize_variables(): e = expr('F(a, b, c) & G(c, A, 23)') assert len(variables(standardize_variables(e))) == 3 - # assert variables(e).intersection(variables(standardize_variables(e))) == {} assert is_variable(standardize_variables(expr('x'))) def test_fol_bc_ask(): def test_ask(query, kb=None): q = expr(query) - test_variables = variables(q) answers = fol_bc_ask(kb or test_kb, q) - return sorted( - [dict((x, v) for x, v in list(a.items()) if x in test_variables) - for a in answers], key=repr) + return sorted([dict((x, v) for x, v in list(a.items()) if x in variables(q)) + for a in answers], key=repr) + assert repr(test_ask('Farmer(x)')) == '[{x: Mac}]' assert repr(test_ask('Human(x)')) == '[{x: Mac}, {x: MrsMac}]' assert repr(test_ask('Rabbit(x)')) == '[{x: MrsRabbit}, {x: Pete}]' @@ -290,11 +330,10 @@ def test_ask(query, kb=None): def test_fol_fc_ask(): def test_ask(query, kb=None): q = expr(query) - test_variables = variables(q) answers = fol_fc_ask(kb or test_kb, q) - return sorted( - [dict((x, v) for x, v in list(a.items()) if x in test_variables) - for a in answers], key=repr) + return sorted([dict((x, v) for x, v in list(a.items()) if x in variables(q)) + for a in answers], key=repr) + assert repr(test_ask('Criminal(x)', crime_kb)) == '[{x: West}]' assert repr(test_ask('Enemy(x, America)', crime_kb)) == '[{x: Nono}]' assert repr(test_ask('Farmer(x)')) == '[{x: Mac}]' @@ -307,15 +346,18 @@ def test_d(): def test_WalkSAT(): - def check_SAT(clauses, single_solution={}): + def check_SAT(clauses, single_solution=None): # Make sure the solution is correct if it is returned by WalkSat # Sometimes WalkSat may run out of flips before finding a solution - soln = WalkSAT(clauses) - if soln: - assert all(pl_true(x, soln) for x in clauses) + if single_solution is None: + single_solution = {} + sol = WalkSAT(clauses) + if sol: + assert all(pl_true(x, sol) for x in clauses) if single_solution: # Cross check the solution if only one exists assert all(pl_true(x, single_solution) for x in clauses) - assert soln == single_solution + assert sol == single_solution + # Test WalkSat for problems with solution check_SAT([A & B, A & C]) check_SAT([A | B, P & Q, P & B]) @@ -332,9 +374,9 @@ def test_SAT_plan(): transition = {'A': {'Left': 'A', 'Right': 'B'}, 'B': {'Left': 'A', 'Right': 'C'}, 'C': {'Left': 'B', 'Right': 'C'}} - assert SAT_plan('A', transition, 'C', 2) is None - assert SAT_plan('A', transition, 'B', 3) == ['Right'] - assert SAT_plan('C', transition, 'A', 3) == ['Left', 'Left'] + assert SAT_plan('A', transition, 'C', 1) is None + assert SAT_plan('A', transition, 'B', 2) == ['Right'] + assert SAT_plan('C', transition, 'A', 2) == ['Left', 'Left'] transition = {(0, 0): {'Right': (0, 1), 'Down': (1, 0)}, (0, 1): {'Left': (1, 0), 'Down': (1, 1)}, diff --git a/tests/test_logic4e.py b/tests/test_logic4e.py new file mode 100644 index 000000000..5a7399281 --- /dev/null +++ b/tests/test_logic4e.py @@ -0,0 +1,359 @@ +import pytest + +from logic4e import * +from utils4e import expr_handle_infix_ops, count + +definite_clauses_KB = PropDefiniteKB() +for clause in ['(B & F)==>E', + '(A & E & F)==>G', + '(B & C)==>F', + '(A & B)==>D', + '(E & F)==>H', + '(H & I)==>J', + 'A', 'B', 'C']: + definite_clauses_KB.tell(expr(clause)) + + +def test_is_symbol(): + assert is_symbol('x') + assert is_symbol('X') + assert is_symbol('N245') + assert not is_symbol('') + assert not is_symbol('1L') + assert not is_symbol([1, 2, 3]) + + +def test_is_var_symbol(): + assert is_var_symbol('xt') + assert not is_var_symbol('Txt') + assert not is_var_symbol('') + assert not is_var_symbol('52') + + +def test_is_prop_symbol(): + assert not is_prop_symbol('xt') + assert is_prop_symbol('Txt') + assert not is_prop_symbol('') + assert not is_prop_symbol('52') + + +def test_variables(): + assert variables(expr('F(x, x) & G(x, y) & H(y, z) & R(A, z, 2)')) == {x, y, z} + assert variables(expr('(x ==> y) & B(x, y) & A')) == {x, y} + + +def test_expr(): + assert repr(expr('P <=> Q(1)')) == '(P <=> Q(1))' + assert repr(expr('P & Q | ~R(x, F(x))')) == '((P & Q) | ~R(x, F(x)))' + assert (expr_handle_infix_ops('P & Q ==> R & ~S') == "P & Q |'==>'| R & ~S") + + +def test_extend(): + assert extend({x: 1}, y, 2) == {x: 1, y: 2} + + +def test_subst(): + assert subst({x: 42, y: 0}, F(x) + y) == (F(42) + 0) + + +def test_PropKB(): + kb = PropKB() + assert count(kb.ask(expr) for expr in [A, C, D, E, Q]) is 0 + kb.tell(A & E) + assert kb.ask(A) == kb.ask(E) == {} + kb.tell(E | '==>' | C) + assert kb.ask(C) == {} + kb.retract(E) + assert kb.ask(E) is False + assert kb.ask(C) is False + + +def test_wumpus_kb(): + # Statement: There is no pit in [1,1]. + assert wumpus_kb.ask(~P11) == {} + + # Statement: There is no pit in [1,2]. + assert wumpus_kb.ask(~P12) == {} + + # Statement: There is a pit in [2,2]. + assert wumpus_kb.ask(P22) is False + + # Statement: There is a pit in [3,1]. + assert wumpus_kb.ask(P31) is False + + # Statement: Neither [1,2] nor [2,1] contains a pit. + assert wumpus_kb.ask(~P12 & ~P21) == {} + + # Statement: There is a pit in either [2,2] or [3,1]. + assert wumpus_kb.ask(P22 | P31) == {} + + +def test_is_definite_clause(): + assert is_definite_clause(expr('A & B & C & D ==> E')) + assert is_definite_clause(expr('Farmer(Mac)')) + assert not is_definite_clause(expr('~Farmer(Mac)')) + assert is_definite_clause(expr('(Farmer(f) & Rabbit(r)) ==> Hates(f, r)')) + assert not is_definite_clause(expr('(Farmer(f) & ~Rabbit(r)) ==> Hates(f, r)')) + assert not is_definite_clause(expr('(Farmer(f) | Rabbit(r)) ==> Hates(f, r)')) + + +def test_parse_definite_clause(): + assert parse_definite_clause(expr('A & B & C & D ==> E')) == ([A, B, C, D], E) + assert parse_definite_clause(expr('Farmer(Mac)')) == ([], expr('Farmer(Mac)')) + assert parse_definite_clause(expr('(Farmer(f) & Rabbit(r)) ==> Hates(f, r)')) == ( + [expr('Farmer(f)'), expr('Rabbit(r)')], expr('Hates(f, r)')) + + +def test_pl_true(): + assert pl_true(P, {}) is None + assert pl_true(P, {P: False}) is False + assert pl_true(P | Q, {P: True}) is True + assert pl_true((A | B) & (C | D), {A: False, B: True, D: True}) is True + assert pl_true((A & B) & (C | D), {A: False, B: True, D: True}) is False + assert pl_true((A & B) | (A & C), {A: False, B: True, C: True}) is False + assert pl_true((A | B) & (C | D), {A: True, D: False}) is None + assert pl_true(P | P, {}) is None + + +def test_tt_true(): + assert tt_true(P | ~P) + assert tt_true('~~P <=> P') + assert not tt_true((P | ~Q) & (~P | Q)) + assert not tt_true(P & ~P) + assert not tt_true(P & Q) + assert tt_true((P | ~Q) | (~P | Q)) + assert tt_true('(A & B) ==> (A | B)') + assert tt_true('((A & B) & C) <=> (A & (B & C))') + assert tt_true('((A | B) | C) <=> (A | (B | C))') + assert tt_true('(A ==> B) <=> (~B ==> ~A)') + assert tt_true('(A ==> B) <=> (~A | B)') + assert tt_true('(A <=> B) <=> ((A ==> B) & (B ==> A))') + assert tt_true('~(A & B) <=> (~A | ~B)') + assert tt_true('~(A | B) <=> (~A & ~B)') + assert tt_true('(A & (B | C)) <=> ((A & B) | (A & C))') + assert tt_true('(A | (B & C)) <=> ((A | B) & (A | C))') + + +def test_dpll(): + assert (dpll_satisfiable(A & ~B & C & (A | ~D) & (~E | ~D) & (C | ~D) & (~A | ~F) & (E | ~F) + & (~D | ~F) & (B | ~C | D) & (A | ~E | F) & (~A | E | D)) + == {B: False, C: True, A: True, F: False, D: True, E: False}) + assert dpll_satisfiable(A & B & ~C & D) == {C: False, A: True, D: True, B: True} + assert dpll_satisfiable((A | (B & C)) | '<=>' | ((A | B) & (A | C))) == {C: True, A: True} or {C: True, B: True} + assert dpll_satisfiable(A | '<=>' | B) == {A: True, B: True} + assert dpll_satisfiable(A & ~B) == {A: True, B: False} + assert dpll_satisfiable(P & ~P) is False + + +def test_find_pure_symbol(): + assert find_pure_symbol([A, B, C], [A | ~B, ~B | ~C, C | A]) == (A, True) + assert find_pure_symbol([A, B, C], [~A | ~B, ~B | ~C, C | A]) == (B, False) + assert find_pure_symbol([A, B, C], [~A | B, ~B | ~C, C | A]) == (None, None) + + +def test_unit_clause_assign(): + assert unit_clause_assign(A | B | C, {A: True}) == (None, None) + assert unit_clause_assign(B | C, {A: True}) == (None, None) + assert unit_clause_assign(B | ~A, {A: True}) == (B, True) + + +def test_find_unit_clause(): + assert find_unit_clause([A | B | C, B | ~C, ~A | ~B], {A: True}) == (B, False) + + +def test_unify(): + assert unify(x, x, {}) == {} + assert unify(x, 3, {}) == {x: 3} + assert unify(x & 4 & y, 6 & y & 4, {}) == {x: 6, y: 4} + assert unify(expr('A(x)'), expr('A(B)')) == {x: B} + assert unify(expr('American(x) & Weapon(B)'), expr('American(A) & Weapon(y)')) == {x: A, y: B} + + +def test_pl_fc_entails(): + assert pl_fc_entails(horn_clauses_KB, expr('Q')) + assert pl_fc_entails(definite_clauses_KB, expr('G')) + assert pl_fc_entails(definite_clauses_KB, expr('H')) + assert not pl_fc_entails(definite_clauses_KB, expr('I')) + assert not pl_fc_entails(definite_clauses_KB, expr('J')) + assert not pl_fc_entails(horn_clauses_KB, expr('SomethingSilly')) + + +def test_tt_entails(): + assert tt_entails(P & Q, Q) + assert not tt_entails(P | Q, Q) + assert tt_entails(A & (B | C) & E & F & ~(P | Q), A & E & F & ~P & ~Q) + assert not tt_entails(P | '<=>' | Q, Q) + assert tt_entails((P | '==>' | Q) & P, Q) + assert not tt_entails((P | '<=>' | Q) & ~P, Q) + + +def test_prop_symbols(): + assert prop_symbols(expr('x & y & z | A')) == {A} + assert prop_symbols(expr('(x & B(z)) ==> Farmer(y) | A')) == {A, expr('Farmer(y)'), expr('B(z)')} + + +def test_constant_symbols(): + assert constant_symbols(expr('x & y & z | A')) == {A} + assert constant_symbols(expr('(x & B(z)) & Father(John) ==> Farmer(y) | A')) == {A, expr('John')} + + +def test_predicate_symbols(): + assert predicate_symbols(expr('x & y & z | A')) == set() + assert predicate_symbols(expr('(x & B(z)) & Father(John) ==> Farmer(y) | A')) == { + ('B', 1), + ('Father', 1), + ('Farmer', 1)} + assert predicate_symbols(expr('(x & B(x, y, z)) & F(G(x, y), x) ==> P(Q(R(x, y)), x, y, z)')) == { + ('B', 3), + ('F', 2), + ('G', 2), + ('P', 4), + ('Q', 1), + ('R', 2)} + + +def test_eliminate_implications(): + assert repr(eliminate_implications('A ==> (~B <== C)')) == '((~B | ~C) | ~A)' + assert repr(eliminate_implications(A ^ B)) == '((A & ~B) | (~A & B))' + assert repr(eliminate_implications(A & B | C & ~D)) == '((A & B) | (C & ~D))' + + +def test_dissociate(): + assert dissociate('&', [A & B]) == [A, B] + assert dissociate('|', [A, B, C & D, P | Q]) == [A, B, C & D, P, Q] + assert dissociate('&', [A, B, C & D, P | Q]) == [A, B, C, D, P | Q] + + +def test_associate(): + assert (repr(associate('&', [(A & B), (B | C), (B & C)])) + == '(A & B & (B | C) & B & C)') + assert (repr(associate('|', [A | (B | (C | (A & B)))])) + == '(A | B | C | (A & B))') + + +def test_move_not_inwards(): + assert repr(move_not_inwards(~(A | B))) == '(~A & ~B)' + assert repr(move_not_inwards(~(A & B))) == '(~A | ~B)' + assert repr(move_not_inwards(~(~(A | ~B) | ~~C))) == '((A | ~B) & ~C)' + + +def test_distribute_and_over_or(): + def test_entailment(s, has_and=False): + result = distribute_and_over_or(s) + if has_and: + assert result.op == '&' + assert tt_entails(s, result) + assert tt_entails(result, s) + + test_entailment((A & B) | C, True) + test_entailment((A | B) & C, True) + test_entailment((A | B) | C, False) + test_entailment((A & B) | (C | D), True) + + +def test_to_cnf(): + assert (repr(to_cnf(wumpus_world_inference & ~expr('~P12'))) == + "((~P12 | B11) & (~P21 | B11) & (P12 | P21 | ~B11) & ~B11 & P12)") + assert repr(to_cnf((P & Q) | (~P & ~Q))) == '((~P | P) & (~Q | P) & (~P | Q) & (~Q | Q))' + assert repr(to_cnf('A <=> B')) == '((A | ~B) & (B | ~A))' + assert repr(to_cnf("B <=> (P1 | P2)")) == '((~P1 | B) & (~P2 | B) & (P1 | P2 | ~B))' + assert repr(to_cnf('A <=> (B & C)')) == '((A | ~B | ~C) & (B | ~A) & (C | ~A))' + assert repr(to_cnf("a | (b & c) | d")) == '((b | a | d) & (c | a | d))' + assert repr(to_cnf("A & (B | (D & E))")) == '(A & (D | B) & (E | B))' + assert repr(to_cnf("A | (B | (C | (D & E)))")) == '((D | A | B | C) & (E | A | B | C))' + assert repr(to_cnf( + '(A <=> ~B) ==> (C | ~D)')) == '((B | ~A | C | ~D) & (A | ~A | C | ~D) & (B | ~B | C | ~D) & (A | ~B | C | ~D))' + + +def test_pl_resolution(): + assert pl_resolution(wumpus_kb, ~P11) + assert pl_resolution(wumpus_kb, ~B11) + assert not pl_resolution(wumpus_kb, P22) + assert pl_resolution(horn_clauses_KB, A) + assert pl_resolution(horn_clauses_KB, B) + assert not pl_resolution(horn_clauses_KB, P) + assert not pl_resolution(definite_clauses_KB, P) + + +def test_standardize_variables(): + e = expr('F(a, b, c) & G(c, A, 23)') + assert len(variables(standardize_variables(e))) == 3 + # assert variables(e).intersection(variables(standardize_variables(e))) == {} + assert is_variable(standardize_variables(expr('x'))) + + +def test_fol_bc_ask(): + def test_ask(query, kb=None): + q = expr(query) + test_variables = variables(q) + answers = fol_bc_ask(kb or test_kb, q) + return sorted( + [dict((x, v) for x, v in list(a.items()) if x in test_variables) + for a in answers], key=repr) + + assert repr(test_ask('Farmer(x)')) == '[{x: Mac}]' + assert repr(test_ask('Human(x)')) == '[{x: Mac}, {x: MrsMac}]' + assert repr(test_ask('Rabbit(x)')) == '[{x: MrsRabbit}, {x: Pete}]' + assert repr(test_ask('Criminal(x)', crime_kb)) == '[{x: West}]' + + +def test_fol_fc_ask(): + def test_ask(query, kb=None): + q = expr(query) + test_variables = variables(q) + answers = fol_fc_ask(kb or test_kb, q) + return sorted( + [dict((x, v) for x, v in list(a.items()) if x in test_variables) + for a in answers], key=repr) + + assert repr(test_ask('Criminal(x)', crime_kb)) == '[{x: West}]' + assert repr(test_ask('Enemy(x, America)', crime_kb)) == '[{x: Nono}]' + assert repr(test_ask('Farmer(x)')) == '[{x: Mac}]' + assert repr(test_ask('Human(x)')) == '[{x: Mac}, {x: MrsMac}]' + assert repr(test_ask('Rabbit(x)')) == '[{x: MrsRabbit}, {x: Pete}]' + + +def test_d(): + assert d(x * x - x, x) == 2 * x - 1 + + +def test_WalkSAT(): + def check_SAT(clauses, single_solution={}): + # Make sure the solution is correct if it is returned by WalkSat + # Sometimes WalkSat may run out of flips before finding a solution + soln = WalkSAT(clauses) + if soln: + assert all(pl_true(x, soln) for x in clauses) + if single_solution: # Cross check the solution if only one exists + assert all(pl_true(x, single_solution) for x in clauses) + assert soln == single_solution + + # Test WalkSat for problems with solution + check_SAT([A & B, A & C]) + check_SAT([A | B, P & Q, P & B]) + check_SAT([A & B, C | D, ~(D | P)], {A: True, B: True, C: True, D: False, P: False}) + check_SAT([A, B, ~C, D], {C: False, A: True, B: True, D: True}) + # Test WalkSat for problems without solution + assert WalkSAT([A & ~A], 0.5, 100) is None + assert WalkSAT([A & B, C | D, ~(D | B)], 0.5, 100) is None + assert WalkSAT([A | B, ~A, ~(B | C), C | D, P | Q], 0.5, 100) is None + assert WalkSAT([A | B, B & C, C | D, D & A, P, ~P], 0.5, 100) is None + + +def test_SAT_plan(): + transition = {'A': {'Left': 'A', 'Right': 'B'}, + 'B': {'Left': 'A', 'Right': 'C'}, + 'C': {'Left': 'B', 'Right': 'C'}} + assert SAT_plan('A', transition, 'C', 2) is None + assert SAT_plan('A', transition, 'B', 3) == ['Right'] + assert SAT_plan('C', transition, 'A', 3) == ['Left', 'Left'] + + transition = {(0, 0): {'Right': (0, 1), 'Down': (1, 0)}, + (0, 1): {'Left': (1, 0), 'Down': (1, 1)}, + (1, 0): {'Right': (1, 0), 'Up': (1, 0), 'Left': (1, 0), 'Down': (1, 0)}, + (1, 1): {'Left': (1, 0), 'Up': (0, 1)}} + assert SAT_plan((0, 0), transition, (1, 1), 4) == ['Right', 'Down'] + + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_mdp.py b/tests/test_mdp.py index af21712ae..979b4ba85 100644 --- a/tests/test_mdp.py +++ b/tests/test_mdp.py @@ -1,5 +1,9 @@ +import pytest + from mdp import * +random.seed("aima-python") + sequential_decision_environment_1 = GridMDP([[-0.1, -0.1, -0.1, +1], [-0.1, None, -0.1, -1], [-0.1, -0.1, -0.1, -0.1]], @@ -10,13 +14,14 @@ [-2, -2, -2, -2]], terminals=[(3, 2), (3, 1)]) -sequential_decision_environment_3 = GridMDP([[-1.0, -0.1, -0.1, -0.1, -0.1, 0.5], - [-0.1, None, None, -0.5, -0.1, -0.1], - [-0.1, None, 1.0, 3.0, None, -0.1], - [-0.1, -0.1, -0.1, None, None, -0.1], +sequential_decision_environment_3 = GridMDP([[-1.0, -0.1, -0.1, -0.1, -0.1, 0.5], + [-0.1, None, None, -0.5, -0.1, -0.1], + [-0.1, None, 1.0, 3.0, None, -0.1], + [-0.1, -0.1, -0.1, None, None, -0.1], [0.5, -0.1, -0.1, -0.1, -0.1, -1.0]], terminals=[(2, 2), (3, 2), (0, 4), (5, 0)]) + def test_value_iteration(): assert value_iteration(sequential_decision_environment, .01) == { (3, 2): 1.0, (3, 1): -1.0, @@ -27,15 +32,15 @@ def test_value_iteration(): (2, 2): 0.79536093684710951} assert value_iteration(sequential_decision_environment_1, .01) == { - (3, 2): 1.0, (3, 1): -1.0, - (3, 0): -0.0897388258468311, (0, 1): 0.146419707398967840, + (3, 2): 1.0, (3, 1): -1.0, + (3, 0): -0.0897388258468311, (0, 1): 0.146419707398967840, (0, 2): 0.30596200514385086, (1, 0): 0.010092796415625799, - (0, 0): 0.00633408092008296, (1, 2): 0.507390193380827400, - (2, 0): 0.15072242145212010, (2, 1): 0.358309043654212570, + (0, 0): 0.00633408092008296, (1, 2): 0.507390193380827400, + (2, 0): 0.15072242145212010, (2, 1): 0.358309043654212570, (2, 2): 0.71675493618997840} assert value_iteration(sequential_decision_environment_2, .01) == { - (3, 2): 1.0, (3, 1): -1.0, + (3, 2): 1.0, (3, 1): -1.0, (3, 0): -3.5141584808407855, (0, 1): -7.8000009574737180, (0, 2): -6.1064293596058830, (1, 0): -7.1012549580376760, (0, 0): -8.5872244532783200, (1, 2): -3.9653547121245810, @@ -43,12 +48,14 @@ def test_value_iteration(): (2, 2): -1.7383376462930498} assert value_iteration(sequential_decision_environment_3, .01) == { - (0, 0): 4.350592130345558, (0, 1): 3.640700980321895, (0, 2): 3.0734806370346943, (0, 3): 2.5754335063434937, (0, 4): -1.0, + (0, 0): 4.350592130345558, (0, 1): 3.640700980321895, (0, 2): 3.0734806370346943, (0, 3): 2.5754335063434937, + (0, 4): -1.0, (1, 0): 3.640700980321895, (1, 1): 3.129579352304856, (1, 4): 2.0787517066719916, (2, 0): 3.0259220379893352, (2, 1): 2.5926103577982897, (2, 2): 1.0, (2, 4): 2.507774181360808, (3, 0): 2.5336747364500076, (3, 2): 3.0, (3, 3): 2.292172805400873, (3, 4): 2.996383110867515, (4, 0): 2.1014575936349886, (4, 3): 3.1297590518608907, (4, 4): 3.6408806798779287, - (5, 0): -1.0, (5, 1): 2.5756132058995282, (5, 2): 3.0736603365907276, (5, 3): 3.6408806798779287, (5, 4): 4.350771829901593} + (5, 0): -1.0, (5, 1): 2.5756132058995282, (5, 2): 3.0736603365907276, (5, 3): 3.6408806798779287, + (5, 4): 4.350771829901593} def test_policy_iteration(): @@ -72,53 +79,49 @@ def test_policy_iteration(): def test_best_policy(): - pi = best_policy(sequential_decision_environment, - value_iteration(sequential_decision_environment, .01)) + pi = best_policy(sequential_decision_environment, value_iteration(sequential_decision_environment, .01)) assert sequential_decision_environment.to_arrows(pi) == [['>', '>', '>', '.'], ['^', None, '^', '.'], ['^', '>', '^', '<']] - pi_1 = best_policy(sequential_decision_environment_1, - value_iteration(sequential_decision_environment_1, .01)) + pi_1 = best_policy(sequential_decision_environment_1, value_iteration(sequential_decision_environment_1, .01)) assert sequential_decision_environment_1.to_arrows(pi_1) == [['>', '>', '>', '.'], ['^', None, '^', '.'], ['^', '>', '^', '<']] - pi_2 = best_policy(sequential_decision_environment_2, - value_iteration(sequential_decision_environment_2, .01)) + pi_2 = best_policy(sequential_decision_environment_2, value_iteration(sequential_decision_environment_2, .01)) assert sequential_decision_environment_2.to_arrows(pi_2) == [['>', '>', '>', '.'], ['^', None, '>', '.'], ['>', '>', '>', '^']] - pi_3 = best_policy(sequential_decision_environment_3, - value_iteration(sequential_decision_environment_3, .01)) - assert sequential_decision_environment_3.to_arrows(pi_3) == [['.', '>', '>', '>', '>', '>'], - ['v', None, None, '>', '>', '^'], - ['v', None, '.', '.', None, '^'], - ['v', '<', 'v', None, None, '^'], - ['<', '<', '<', '<', '<', '.']] + pi_3 = best_policy(sequential_decision_environment_3, value_iteration(sequential_decision_environment_3, .01)) + assert sequential_decision_environment_3.to_arrows(pi_3) == [['.', '>', '>', '>', '>', '>'], + ['v', None, None, '>', '>', '^'], + ['v', None, '.', '.', None, '^'], + ['v', '<', 'v', None, None, '^'], + ['<', '<', '<', '<', '<', '.']] def test_transition_model(): - transition_model = { 'a' : { 'plan1' : [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], - 'plan2' : [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], - 'plan3' : [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], - }, - 'b' : { 'plan1' : [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], - 'plan2' : [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan3' : [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], - }, - 'c' : { 'plan1' : [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan2' : [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], - 'plan3' : [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], - }, - } - - mdp = MDP(init="a", actlist={"plan1","plan2", "plan3"}, terminals={"d"}, states={"a","b","c", "d"}, transitions=transition_model) - - assert mdp.T("a","plan3") == [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')] - assert mdp.T("b","plan2") == [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')] - assert mdp.T("c","plan1") == [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')] + transition_model = {'a': {'plan1': [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], + 'plan2': [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3': [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b': {'plan1': [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2': [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3': [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c': {'plan1': [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2': [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3': [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }} + + mdp = MDP(init="a", actlist={"plan1", "plan2", "plan3"}, terminals={"d"}, states={"a", "b", "c", "d"}, + transitions=transition_model) + + assert mdp.T("a", "plan3") == [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')] + assert mdp.T("b", "plan2") == [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')] + assert mdp.T("c", "plan1") == [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')] def test_pomdp_value_iteration(): @@ -132,12 +135,12 @@ def test_pomdp_value_iteration(): pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma) utility = pomdp_value_iteration(pomdp, epsilon=5) - + for _, v in utility.items(): sum_ = 0 for element in v: sum_ += sum(element) - + assert -9.76 < sum_ < -9.70 or 246.5 < sum_ < 248.5 or 0 < sum_ < 1 @@ -159,3 +162,7 @@ def test_pomdp_value_iteration2(): sum_ += sum(element) assert -77.31 < sum_ < -77.25 or 799 < sum_ < 800 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_mdp4e.py b/tests/test_mdp4e.py new file mode 100644 index 000000000..e51bda5d6 --- /dev/null +++ b/tests/test_mdp4e.py @@ -0,0 +1,176 @@ +import pytest + +from mdp4e import * + +random.seed("aima-python") + +sequential_decision_environment_1 = GridMDP([[-0.1, -0.1, -0.1, +1], + [-0.1, None, -0.1, -1], + [-0.1, -0.1, -0.1, -0.1]], + terminals=[(3, 2), (3, 1)]) + +sequential_decision_environment_2 = GridMDP([[-2, -2, -2, +1], + [-2, None, -2, -1], + [-2, -2, -2, -2]], + terminals=[(3, 2), (3, 1)]) + +sequential_decision_environment_3 = GridMDP([[-1.0, -0.1, -0.1, -0.1, -0.1, 0.5], + [-0.1, None, None, -0.5, -0.1, -0.1], + [-0.1, None, 1.0, 3.0, None, -0.1], + [-0.1, -0.1, -0.1, None, None, -0.1], + [0.5, -0.1, -0.1, -0.1, -0.1, -1.0]], + terminals=[(2, 2), (3, 2), (0, 4), (5, 0)]) + + +def test_value_iteration(): + ref1 = { + (3, 2): 1.0, (3, 1): -1.0, + (3, 0): 0.12958868267972745, (0, 1): 0.39810203830605462, + (0, 2): 0.50928545646220924, (1, 0): 0.25348746162470537, + (0, 0): 0.29543540628363629, (1, 2): 0.64958064617168676, + (2, 0): 0.34461306281476806, (2, 1): 0.48643676237737926, + (2, 2): 0.79536093684710951} + assert sum(value_iteration(sequential_decision_environment, .01).values()) - sum(ref1.values()) < 0.0001 + + ref2 = { + (3, 2): 1.0, (3, 1): -1.0, + (3, 0): -0.0897388258468311, (0, 1): 0.146419707398967840, + (0, 2): 0.30596200514385086, (1, 0): 0.010092796415625799, + (0, 0): 0.00633408092008296, (1, 2): 0.507390193380827400, + (2, 0): 0.15072242145212010, (2, 1): 0.358309043654212570, + (2, 2): 0.71675493618997840} + assert sum(value_iteration(sequential_decision_environment_1, .01).values()) - sum(ref2.values()) < 0.0001 + + ref3 = { + (3, 2): 1.0, (3, 1): -1.0, + (3, 0): -3.5141584808407855, (0, 1): -7.8000009574737180, + (0, 2): -6.1064293596058830, (1, 0): -7.1012549580376760, + (0, 0): -8.5872244532783200, (1, 2): -3.9653547121245810, + (2, 0): -5.3099468802901630, (2, 1): -3.3543366255753995, + (2, 2): -1.7383376462930498} + assert sum(value_iteration(sequential_decision_environment_2, .01).values()) - sum(ref3.values()) < 0.0001 + + ref4 = { + (0, 0): 4.350592130345558, (0, 1): 3.640700980321895, (0, 2): 3.0734806370346943, (0, 3): 2.5754335063434937, + (0, 4): -1.0, + (1, 0): 3.640700980321895, (1, 1): 3.129579352304856, (1, 4): 2.0787517066719916, + (2, 0): 3.0259220379893352, (2, 1): 2.5926103577982897, (2, 2): 1.0, (2, 4): 2.507774181360808, + (3, 0): 2.5336747364500076, (3, 2): 3.0, (3, 3): 2.292172805400873, (3, 4): 2.996383110867515, + (4, 0): 2.1014575936349886, (4, 3): 3.1297590518608907, (4, 4): 3.6408806798779287, + (5, 0): -1.0, (5, 1): 2.5756132058995282, (5, 2): 3.0736603365907276, (5, 3): 3.6408806798779287, + (5, 4): 4.350771829901593} + assert sum(value_iteration(sequential_decision_environment_3, .01).values()) - sum(ref4.values()) < 0.001 + + +def test_policy_iteration(): + assert policy_iteration(sequential_decision_environment) == { + (0, 0): (0, 1), (0, 1): (0, 1), (0, 2): (1, 0), + (1, 0): (1, 0), (1, 2): (1, 0), (2, 0): (0, 1), + (2, 1): (0, 1), (2, 2): (1, 0), (3, 0): (-1, 0), + (3, 1): None, (3, 2): None} + + assert policy_iteration(sequential_decision_environment_1) == { + (0, 0): (0, 1), (0, 1): (0, 1), (0, 2): (1, 0), + (1, 0): (1, 0), (1, 2): (1, 0), (2, 0): (0, 1), + (2, 1): (0, 1), (2, 2): (1, 0), (3, 0): (-1, 0), + (3, 1): None, (3, 2): None} + + assert policy_iteration(sequential_decision_environment_2) == { + (0, 0): (1, 0), (0, 1): (0, 1), (0, 2): (1, 0), + (1, 0): (1, 0), (1, 2): (1, 0), (2, 0): (1, 0), + (2, 1): (1, 0), (2, 2): (1, 0), (3, 0): (0, 1), + (3, 1): None, (3, 2): None} + + +def test_best_policy(): + pi = best_policy(sequential_decision_environment, + value_iteration(sequential_decision_environment, .01)) + assert sequential_decision_environment.to_arrows(pi) == [['>', '>', '>', '.'], + ['^', None, '^', '.'], + ['^', '>', '^', '<']] + + pi_1 = best_policy(sequential_decision_environment_1, + value_iteration(sequential_decision_environment_1, .01)) + assert sequential_decision_environment_1.to_arrows(pi_1) == [['>', '>', '>', '.'], + ['^', None, '^', '.'], + ['^', '>', '^', '<']] + + pi_2 = best_policy(sequential_decision_environment_2, + value_iteration(sequential_decision_environment_2, .01)) + assert sequential_decision_environment_2.to_arrows(pi_2) == [['>', '>', '>', '.'], + ['^', None, '>', '.'], + ['>', '>', '>', '^']] + + pi_3 = best_policy(sequential_decision_environment_3, + value_iteration(sequential_decision_environment_3, .01)) + assert sequential_decision_environment_3.to_arrows(pi_3) == [['.', '>', '>', '>', '>', '>'], + ['v', None, None, '>', '>', '^'], + ['v', None, '.', '.', None, '^'], + ['v', '<', 'v', None, None, '^'], + ['<', '<', '<', '<', '<', '.']] + + +def test_transition_model(): + transition_model = {'a': {'plan1': [(0.2, 'a'), (0.3, 'b'), (0.3, 'c'), (0.2, 'd')], + 'plan2': [(0.4, 'a'), (0.15, 'b'), (0.45, 'c')], + 'plan3': [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')], + }, + 'b': {'plan1': [(0.2, 'a'), (0.6, 'b'), (0.2, 'c'), (0.1, 'd')], + 'plan2': [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3': [(0.3, 'a'), (0.3, 'b'), (0.4, 'c')], + }, + 'c': {'plan1': [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan2': [(0.5, 'a'), (0.3, 'b'), (0.1, 'c'), (0.1, 'd')], + 'plan3': [(0.1, 'a'), (0.3, 'b'), (0.1, 'c'), (0.5, 'd')], + }} + + mdp = MDP(init="a", actlist={"plan1", "plan2", "plan3"}, terminals={"d"}, states={"a", "b", "c", "d"}, + transitions=transition_model) + + assert mdp.T("a", "plan3") == [(0.2, 'a'), (0.5, 'b'), (0.3, 'c')] + assert mdp.T("b", "plan2") == [(0.6, 'a'), (0.2, 'b'), (0.1, 'c'), (0.1, 'd')] + assert mdp.T("c", "plan1") == [(0.3, 'a'), (0.5, 'b'), (0.1, 'c'), (0.1, 'd')] + + +def test_pomdp_value_iteration(): + t_prob = [[[0.65, 0.35], [0.65, 0.35]], [[0.65, 0.35], [0.65, 0.35]], [[1.0, 0.0], [0.0, 1.0]]] + e_prob = [[[0.5, 0.5], [0.5, 0.5]], [[0.5, 0.5], [0.5, 0.5]], [[0.8, 0.2], [0.3, 0.7]]] + rewards = [[5, -10], [-20, 5], [-1, -1]] + + gamma = 0.95 + actions = ('0', '1', '2') + states = ('0', '1') + + pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma) + utility = pomdp_value_iteration(pomdp, epsilon=5) + + for _, v in utility.items(): + sum_ = 0 + for element in v: + sum_ += sum(element) + + assert -9.76 < sum_ < -9.70 or 246.5 < sum_ < 248.5 or 0 < sum_ < 1 + + +def test_pomdp_value_iteration2(): + t_prob = [[[0.5, 0.5], [0.5, 0.5]], [[0.5, 0.5], [0.5, 0.5]], [[1.0, 0.0], [0.0, 1.0]]] + e_prob = [[[0.5, 0.5], [0.5, 0.5]], [[0.5, 0.5], [0.5, 0.5]], [[0.85, 0.15], [0.15, 0.85]]] + rewards = [[-100, 10], [10, -100], [-1, -1]] + + gamma = 0.95 + actions = ('0', '1', '2') + states = ('0', '1') + + pomdp = POMDP(actions, t_prob, e_prob, rewards, states, gamma) + utility = pomdp_value_iteration(pomdp, epsilon=100) + + for _, v in utility.items(): + sum_ = 0 + for element in v: + sum_ += sum(element) + + assert -77.31 < sum_ < -77.25 or 799 < sum_ < 800 + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_nlp.py b/tests/test_nlp.py index 978685a4e..85d246dfa 100644 --- a/tests/test_nlp.py +++ b/tests/test_nlp.py @@ -1,9 +1,11 @@ +import random + import pytest import nlp from nlp import loadPageHTML, stripRawHTML, findOutlinks, onlyWikipediaURLS -from nlp import expand_pages, relevant_pages, normalize, ConvergenceDetector, getInlinks -from nlp import getOutlinks, Page, determineInlinks, HITS +from nlp import expand_pages, relevant_pages, normalize, ConvergenceDetector, getInLinks +from nlp import getOutLinks, Page, determineInlinks, HITS from nlp import Rules, Lexicon, Grammar, ProbRules, ProbLexicon, ProbGrammar from nlp import Chart, CYK_parse # Clumsy imports because we want to access certain nlp.py globals explicitly, because @@ -12,6 +14,8 @@ from unittest.mock import patch from io import BytesIO +random.seed("aima-python") + def test_rules(): check = {'A': [['B', 'C'], ['D', 'E']], 'B': [['E'], ['a'], ['b', 'c']]} @@ -39,7 +43,7 @@ def test_grammar(): def test_generation(): lexicon = Lexicon(Article="the | a | an", - Pronoun="i | you | he") + Pronoun="i | you | he") rules = Rules( S="Article | More | Pronoun", @@ -153,9 +157,10 @@ def test_CYK_parse(): pageDict = {pA.address: pA, pB.address: pB, pC.address: pC, pD.address: pD, pE.address: pE, pF.address: pF} nlp.pagesIndex = pageDict -nlp.pagesContent ={pA.address: testHTML, pB.address: testHTML2, - pC.address: testHTML, pD.address: testHTML2, - pE.address: testHTML, pF.address: testHTML2} +nlp.pagesContent = {pA.address: testHTML, pB.address: testHTML2, + pC.address: testHTML, pD.address: testHTML2, + pE.address: testHTML, pF.address: testHTML2} + # This test takes a long time (> 60 secs) # def test_loadPageHTML(): @@ -183,12 +188,15 @@ def test_determineInlinks(): assert set(determineInlinks(pE)) == set([]) assert set(determineInlinks(pF)) == set(['E']) + def test_findOutlinks_wiki(): testPage = pageDict[pA.address] outlinks = findOutlinks(testPage, handleURLs=onlyWikipediaURLS) assert "https://en.wikipedia.org/wiki/TestThing" in outlinks assert "https://en.wikipedia.org/wiki/TestThing" in outlinks assert "https://google.com.au" not in outlinks + + # ______________________________________________________________________________ # HITS Helper Functions @@ -217,7 +225,8 @@ def test_relevant_pages(): def test_normalize(): normalize(pageDict) print(page.hub for addr, page in nlp.pagesIndex.items()) - expected_hub = [1/91**0.5, 2/91**0.5, 3/91**0.5, 4/91**0.5, 5/91**0.5, 6/91**0.5] # Works only for sample data above + expected_hub = [1 / 91 ** 0.5, 2 / 91 ** 0.5, 3 / 91 ** 0.5, 4 / 91 ** 0.5, 5 / 91 ** 0.5, + 6 / 91 ** 0.5] # Works only for sample data above expected_auth = list(reversed(expected_hub)) assert len(expected_hub) == len(expected_auth) == len(nlp.pagesIndex) assert expected_hub == [page.hub for addr, page in sorted(nlp.pagesIndex.items())] @@ -243,12 +252,12 @@ def test_detectConvergence(): def test_getInlinks(): - inlnks = getInlinks(pageDict['A']) + inlnks = getInLinks(pageDict['A']) assert sorted(inlnks) == pageDict['A'].inlinks def test_getOutlinks(): - outlnks = getOutlinks(pageDict['A']) + outlnks = getOutLinks(pageDict['A']) assert sorted(outlnks) == pageDict['A'].outlinks diff --git a/tests/test_nlp4e.py b/tests/test_nlp4e.py new file mode 100644 index 000000000..2d16a3196 --- /dev/null +++ b/tests/test_nlp4e.py @@ -0,0 +1,139 @@ +import random + +import pytest +import nlp + +from nlp4e import Rules, Lexicon, Grammar, ProbRules, ProbLexicon, ProbGrammar, E0 +from nlp4e import Chart, CYK_parse, subspan, astar_search_parsing, beam_search_parsing + +# Clumsy imports because we want to access certain nlp.py globals explicitly, because +# they are accessed by functions within nlp.py + +random.seed("aima-python") + + +def test_rules(): + check = {'A': [['B', 'C'], ['D', 'E']], 'B': [['E'], ['a'], ['b', 'c']]} + assert Rules(A="B C | D E", B="E | a | b c") == check + + +def test_lexicon(): + check = {'Article': ['the', 'a', 'an'], 'Pronoun': ['i', 'you', 'he']} + lexicon = Lexicon(Article="the | a | an", Pronoun="i | you | he") + assert lexicon == check + + +def test_grammar(): + rules = Rules(A="B C | D E", B="E | a | b c") + lexicon = Lexicon(Article="the | a | an", Pronoun="i | you | he") + grammar = Grammar("Simplegram", rules, lexicon) + + assert grammar.rewrites_for('A') == [['B', 'C'], ['D', 'E']] + assert grammar.isa('the', 'Article') + + grammar = nlp.E_Chomsky + for rule in grammar.cnf_rules(): + assert len(rule) == 3 + + +def test_generation(): + lexicon = Lexicon(Article="the | a | an", + Pronoun="i | you | he") + + rules = Rules( + S="Article | More | Pronoun", + More="Article Pronoun | Pronoun Pronoun" + ) + + grammar = Grammar("Simplegram", rules, lexicon) + + sentence = grammar.generate_random('S') + for token in sentence.split(): + found = False + for non_terminal, terminals in grammar.lexicon.items(): + if token in terminals: + found = True + assert found + + +def test_prob_rules(): + check = {'A': [(['B', 'C'], 0.3), (['D', 'E'], 0.7)], + 'B': [(['E'], 0.1), (['a'], 0.2), (['b', 'c'], 0.7)]} + rules = ProbRules(A="B C [0.3] | D E [0.7]", B="E [0.1] | a [0.2] | b c [0.7]") + assert rules == check + + +def test_prob_lexicon(): + check = {'Article': [('the', 0.5), ('a', 0.25), ('an', 0.25)], + 'Pronoun': [('i', 0.4), ('you', 0.3), ('he', 0.3)]} + lexicon = ProbLexicon(Article="the [0.5] | a [0.25] | an [0.25]", + Pronoun="i [0.4] | you [0.3] | he [0.3]") + assert lexicon == check + + +def test_prob_grammar(): + rules = ProbRules(A="B C [0.3] | D E [0.7]", B="E [0.1] | a [0.2] | b c [0.7]") + lexicon = ProbLexicon(Article="the [0.5] | a [0.25] | an [0.25]", + Pronoun="i [0.4] | you [0.3] | he [0.3]") + grammar = ProbGrammar("Simplegram", rules, lexicon) + + assert grammar.rewrites_for('A') == [(['B', 'C'], 0.3), (['D', 'E'], 0.7)] + assert grammar.isa('the', 'Article') + + grammar = nlp.E_Prob_Chomsky + for rule in grammar.cnf_rules(): + assert len(rule) == 4 + + +def test_prob_generation(): + lexicon = ProbLexicon(Verb="am [0.5] | are [0.25] | is [0.25]", + Pronoun="i [0.4] | you [0.3] | he [0.3]") + + rules = ProbRules( + S="Verb [0.5] | More [0.3] | Pronoun [0.1] | nobody is here [0.1]", + More="Pronoun Verb [0.7] | Pronoun Pronoun [0.3]") + + grammar = ProbGrammar("Simplegram", rules, lexicon) + + sentence = grammar.generate_random('S') + assert len(sentence) == 2 + + +def test_chart_parsing(): + chart = Chart(nlp.E0) + parses = chart.parses('the stench is in 2 2') + assert len(parses) == 1 + + +def test_CYK_parse(): + grammar = nlp.E_Prob_Chomsky + words = ['the', 'robot', 'is', 'good'] + P = CYK_parse(words, grammar) + assert len(P) == 5 + + grammar = nlp.E_Prob_Chomsky_ + words = ['astronomers', 'saw', 'stars'] + P = CYK_parse(words, grammar) + assert len(P) == 3 + + +def test_subspan(): + spans = subspan(3) + assert spans.__next__() == (1, 1, 2) + assert spans.__next__() == (2, 2, 3) + assert spans.__next__() == (1, 1, 3) + assert spans.__next__() == (1, 2, 3) + + +def test_text_parsing(): + words = ["the", "wumpus", "is", "dead"] + grammer = E0 + assert astar_search_parsing(words, grammer) == 'S' + assert beam_search_parsing(words, grammer) == 'S' + words = ["the", "is", "wupus", "dead"] + assert astar_search_parsing(words, grammer) is False + assert beam_search_parsing(words, grammer) is False + + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_perception4e.py b/tests/test_perception4e.py new file mode 100644 index 000000000..46d534523 --- /dev/null +++ b/tests/test_perception4e.py @@ -0,0 +1,87 @@ +import random + +import pytest + +from perception4e import * +from PIL import Image +import numpy as np +import os + +random.seed("aima-python") + + +def test_array_normalization(): + assert list(array_normalization([1, 2, 3, 4, 5], 0, 1)) == [0, 0.25, 0.5, 0.75, 1] + assert list(array_normalization([1, 2, 3, 4, 5], 1, 2)) == [1, 1.25, 1.5, 1.75, 2] + + +def test_sum_squared_difference(): + image = Image.open(os.path.abspath("./images/broxrevised.png")) + arr = np.asarray(image) + arr1 = arr[10:500, :514] + arr2 = arr[10:500, 514:1028] + assert sum_squared_difference(arr1, arr1)[1] == 0 + assert sum_squared_difference(arr1, arr1)[0] == (0, 0) + assert sum_squared_difference(arr1, arr2)[1] > 200000 + + +def test_gen_gray_scale_picture(): + assert list(gen_gray_scale_picture(size=3, level=3)[0]) == [0, 125, 250] + assert list(gen_gray_scale_picture(size=3, level=3)[1]) == [125, 125, 250] + assert list(gen_gray_scale_picture(size=3, level=3)[2]) == [250, 250, 250] + assert list(gen_gray_scale_picture(2, level=2)[0]) == [0, 250] + assert list(gen_gray_scale_picture(2, level=2)[1]) == [250, 250] + + +def test_generate_edge_weight(): + assert generate_edge_weight(gray_scale_image, (0, 0), (2, 2)) == 5 + assert generate_edge_weight(gray_scale_image, (1, 0), (0, 1)) == 255 + + +def test_graph_bfs(): + graph = Graph(gray_scale_image) + assert not graph.bfs((1, 1), (0, 0), []) + parents = [] + assert graph.bfs((0, 0), (2, 2), parents) + assert len(parents) == 8 + + +def test_graph_min_cut(): + image = gen_gray_scale_picture(size=3, level=2) + graph = Graph(image) + assert len(graph.min_cut((0, 0), (2, 2))) == 4 + image = gen_gray_scale_picture(size=10, level=2) + graph = Graph(image) + assert len(graph.min_cut((0, 0), (9, 9))) == 10 + + +def test_gen_discs(): + discs = gen_discs(100, 2) + assert len(discs) == 2 + assert len(discs[1]) == len(discs[0]) == 8 + + +def test_simple_convnet(): + train, val, test = load_MINST(1000, 100, 10) + model = simple_convnet() + model.fit(train[0], train[1], validation_data=(val[0], val[1]), epochs=5, verbose=2, batch_size=32) + scores = model.evaluate(test[0], test[1], verbose=1) + assert scores[1] > 0.2 + + +def test_ROIPoolingLayer(): + # Create feature map input + feature_maps_shape = (200, 100, 1) + feature_map = np.ones(feature_maps_shape, dtype='float32') + feature_map[200 - 1, 100 - 3, 0] = 50 + roiss = np.asarray([[0.5, 0.2, 0.7, 0.4], [0.0, 0.0, 1.0, 1.0]]) + assert pool_rois(feature_map, roiss, 3, 7)[0].tolist() == [[1, 1, 1, 1, 1, 1, 1], + [1, 1, 1, 1, 1, 1, 1], + [1, 1, 1, 1, 1, 1, 1]] + assert pool_rois(feature_map, roiss, 3, 7)[1].tolist() == [[1, 1, 1, 1, 1, 1, 1], + [1, 1, 1, 1, 1, 1, 1], + [1, 1, 1, 1, 1, 1, 50]] + + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_planning.py b/tests/test_planning.py index 3223fcc61..a39152adc 100644 --- a/tests/test_planning.py +++ b/tests/test_planning.py @@ -1,32 +1,40 @@ +import random + +import pytest + from planning import * +from search import astar_search from utils import expr from logic import FolKB, conjuncts +random.seed('aima-python') + def test_action(): precond = 'At(c, a) & At(p, a) & Cargo(c) & Plane(p) & Airport(a)' effect = 'In(c, p) & ~At(c, a)' a = Action('Load(c, p, a)', precond, effect) - args = [expr("C1"), expr("P1"), expr("SFO")] - assert a.substitute(expr("Load(c, p, a)"), args) == expr("Load(C1, P1, SFO)") - test_kb = FolKB(conjuncts(expr('At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK) & Cargo(C1) & Cargo(C2) & Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)'))) + args = [expr('C1'), expr('P1'), expr('SFO')] + assert a.substitute(expr('Load(c, p, a)'), args) == expr('Load(C1, P1, SFO)') + test_kb = FolKB(conjuncts(expr('At(C1, SFO) & At(C2, JFK) & At(P1, SFO) & At(P2, JFK) & Cargo(C1) & Cargo(C2) & ' + 'Plane(P1) & Plane(P2) & Airport(SFO) & Airport(JFK)'))) assert a.check_precond(test_kb, args) a.act(test_kb, args) - assert test_kb.ask(expr("In(C1, P2)")) is False - assert test_kb.ask(expr("In(C1, P1)")) is not False - assert test_kb.ask(expr("Plane(P2)")) is not False + assert test_kb.ask(expr('In(C1, P2)')) is False + assert test_kb.ask(expr('In(C1, P1)')) is not False + assert test_kb.ask(expr('Plane(P2)')) is not False assert not a.check_precond(test_kb, args) def test_air_cargo_1(): p = air_cargo() assert p.goal_test() is False - solution_1 = [expr("Load(C1 , P1, SFO)"), - expr("Fly(P1, SFO, JFK)"), - expr("Unload(C1, P1, JFK)"), - expr("Load(C2, P2, JFK)"), - expr("Fly(P2, JFK, SFO)"), - expr("Unload (C2, P2, SFO)")] + solution_1 = [expr('Load(C1 , P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), + expr('Load(C2, P2, JFK)'), + expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)')] for action in solution_1: p.act(action) @@ -37,12 +45,12 @@ def test_air_cargo_1(): def test_air_cargo_2(): p = air_cargo() assert p.goal_test() is False - solution_2 = [expr("Load(C2, P2, JFK)"), - expr("Fly(P2, JFK, SFO)"), - expr("Unload (C2, P2, SFO)"), - expr("Load(C1 , P1, SFO)"), - expr("Fly(P1, SFO, JFK)"), - expr("Unload(C1, P1, JFK)")] + solution_2 = [expr('Load(C1 , P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), + expr('Load(C2, P1, JFK)'), + expr('Fly(P1, JFK, SFO)'), + expr('Unload(C2, P1, SFO)')] for action in solution_2: p.act(action) @@ -50,14 +58,46 @@ def test_air_cargo_2(): assert p.goal_test() -def test_spare_tire(): +def test_air_cargo_3(): + p = air_cargo() + assert p.goal_test() is False + solution_3 = [expr('Load(C2, P2, JFK)'), + expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)'), + expr('Load(C1 , P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)')] + + for action in solution_3: + p.act(action) + + assert p.goal_test() + + +def test_air_cargo_4(): + p = air_cargo() + assert p.goal_test() is False + solution_4 = [expr('Load(C2, P2, JFK)'), + expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)'), + expr('Load(C1, P2, SFO)'), + expr('Fly(P2, SFO, JFK)'), + expr('Unload(C1, P2, JFK)')] + + for action in solution_4: + p.act(action) + + assert p.goal_test() + + +def test_spare_tire_1(): p = spare_tire() assert p.goal_test() is False - solution = [expr("Remove(Flat, Axle)"), - expr("Remove(Spare, Trunk)"), - expr("PutOn(Spare, Axle)")] + solution_1 = [expr('Remove(Flat, Axle)'), + expr('Remove(Spare, Trunk)'), + expr('PutOn(Spare, Axle)')] - for action in solution: + for action in solution_1: p.act(action) assert p.goal_test() @@ -75,13 +115,26 @@ def test_spare_tire_2(): assert p.goal_test() - + def test_three_block_tower(): p = three_block_tower() assert p.goal_test() is False - solution = [expr("MoveToTable(C, A)"), - expr("Move(B, Table, C)"), - expr("Move(A, Table, B)")] + solution = [expr('MoveToTable(C, A)'), + expr('Move(B, Table, C)'), + expr('Move(A, Table, B)')] + + for action in solution: + p.act(action) + + assert p.goal_test() + + +def test_simple_blocks_world(): + p = simple_blocks_world() + assert p.goal_test() is False + solution = [expr('ToTable(A, B)'), + expr('FromTable(B, A)'), + expr('FromTable(C, B)')] for action in solution: p.act(action) @@ -92,8 +145,8 @@ def test_three_block_tower(): def test_have_cake_and_eat_cake_too(): p = have_cake_and_eat_cake_too() assert p.goal_test() is False - solution = [expr("Eat(Cake)"), - expr("Bake(Cake)")] + solution = [expr('Eat(Cake)'), + expr('Bake(Cake)')] for action in solution: p.act(action) @@ -101,24 +154,39 @@ def test_have_cake_and_eat_cake_too(): assert p.goal_test() -def test_shopping_problem(): +def test_shopping_problem_1(): p = shopping_problem() assert p.goal_test() is False - solution = [expr('Go(Home, SM)'), - expr('Buy(Banana, SM)'), - expr('Buy(Milk, SM)'), - expr('Go(SM, HW)'), - expr('Buy(Drill, HW)')] + solution_1 = [expr('Go(Home, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)'), + expr('Go(SM, HW)'), + expr('Buy(Drill, HW)')] - for action in solution: + for action in solution_1: + p.act(action) + + assert p.goal_test() + + +def test_shopping_problem_2(): + p = shopping_problem() + assert p.goal_test() is False + solution_2 = [expr('Go(Home, HW)'), + expr('Buy(Drill, HW)'), + expr('Go(HW, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)')] + + for action in solution_2: p.act(action) assert p.goal_test() def test_graph_call(): - planningproblem = spare_tire() - graph = Graph(planningproblem) + planning_problem = spare_tire() + graph = Graph(planning_problem) levels_size = len(graph.levels) graph() @@ -126,19 +194,19 @@ def test_graph_call(): assert levels_size == len(graph.levels) - 1 -def test_graphplan(): - spare_tire_solution = spare_tire_graphplan() +def test_graphPlan(): + spare_tire_solution = spare_tire_graphPlan() spare_tire_solution = linearize(spare_tire_solution) assert expr('Remove(Flat, Axle)') in spare_tire_solution assert expr('Remove(Spare, Trunk)') in spare_tire_solution assert expr('PutOn(Spare, Axle)') in spare_tire_solution - cake_solution = have_cake_and_eat_cake_too_graphplan() + cake_solution = have_cake_and_eat_cake_too_graphPlan() cake_solution = linearize(cake_solution) assert expr('Eat(Cake)') in cake_solution assert expr('Bake(Cake)') in cake_solution - air_cargo_solution = air_cargo_graphplan() + air_cargo_solution = air_cargo_graphPlan() air_cargo_solution = linearize(air_cargo_solution) assert expr('Load(C1, P1, SFO)') in air_cargo_solution assert expr('Load(C2, P2, JFK)') in air_cargo_solution @@ -147,13 +215,19 @@ def test_graphplan(): assert expr('Unload(C1, P1, JFK)') in air_cargo_solution assert expr('Unload(C2, P2, SFO)') in air_cargo_solution - sussman_anomaly_solution = three_block_tower_graphplan() + sussman_anomaly_solution = three_block_tower_graphPlan() sussman_anomaly_solution = linearize(sussman_anomaly_solution) assert expr('MoveToTable(C, A)') in sussman_anomaly_solution assert expr('Move(B, Table, C)') in sussman_anomaly_solution assert expr('Move(A, Table, B)') in sussman_anomaly_solution - shopping_problem_solution = shopping_graphplan() + blocks_world_solution = simple_blocks_world_graphPlan() + blocks_world_solution = linearize(blocks_world_solution) + assert expr('ToTable(A, B)') in blocks_world_solution + assert expr('FromTable(B, A)') in blocks_world_solution + assert expr('FromTable(C, B)') in blocks_world_solution + + shopping_problem_solution = shopping_graphPlan() shopping_problem_solution = linearize(shopping_problem_solution) assert expr('Go(Home, HW)') in shopping_problem_solution assert expr('Go(Home, SM)') in shopping_problem_solution @@ -162,6 +236,165 @@ def test_graphplan(): assert expr('Buy(Milk, SM)') in shopping_problem_solution +def test_forwardPlan(): + spare_tire_solution = astar_search(ForwardPlan(spare_tire())).solution() + spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution)) + assert expr('Remove(Flat, Axle)') in spare_tire_solution + assert expr('Remove(Spare, Trunk)') in spare_tire_solution + assert expr('PutOn(Spare, Axle)') in spare_tire_solution + + cake_solution = astar_search(ForwardPlan(have_cake_and_eat_cake_too())).solution() + cake_solution = list(map(lambda action: Expr(action.name, *action.args), cake_solution)) + assert expr('Eat(Cake)') in cake_solution + assert expr('Bake(Cake)') in cake_solution + + air_cargo_solution = astar_search(ForwardPlan(air_cargo())).solution() + air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution)) + assert expr('Load(C2, P2, JFK)') in air_cargo_solution + assert expr('Fly(P2, JFK, SFO)') in air_cargo_solution + assert expr('Unload(C2, P2, SFO)') in air_cargo_solution + assert expr('Load(C1, P2, SFO)') in air_cargo_solution + assert expr('Fly(P2, SFO, JFK)') in air_cargo_solution + assert expr('Unload(C1, P2, JFK)') in air_cargo_solution + + sussman_anomaly_solution = astar_search(ForwardPlan(three_block_tower())).solution() + sussman_anomaly_solution = list(map(lambda action: Expr(action.name, *action.args), sussman_anomaly_solution)) + assert expr('MoveToTable(C, A)') in sussman_anomaly_solution + assert expr('Move(B, Table, C)') in sussman_anomaly_solution + assert expr('Move(A, Table, B)') in sussman_anomaly_solution + + blocks_world_solution = astar_search(ForwardPlan(simple_blocks_world())).solution() + blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution)) + assert expr('ToTable(A, B)') in blocks_world_solution + assert expr('FromTable(B, A)') in blocks_world_solution + assert expr('FromTable(C, B)') in blocks_world_solution + + shopping_problem_solution = astar_search(ForwardPlan(shopping_problem())).solution() + shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution)) + assert expr('Go(Home, SM)') in shopping_problem_solution + assert expr('Buy(Banana, SM)') in shopping_problem_solution + assert expr('Buy(Milk, SM)') in shopping_problem_solution + assert expr('Go(SM, HW)') in shopping_problem_solution + assert expr('Buy(Drill, HW)') in shopping_problem_solution + + +def test_backwardPlan(): + spare_tire_solution = astar_search(BackwardPlan(spare_tire())).solution() + spare_tire_solution = list(map(lambda action: Expr(action.name, *action.args), spare_tire_solution)) + assert expr('Remove(Flat, Axle)') in spare_tire_solution + assert expr('Remove(Spare, Trunk)') in spare_tire_solution + assert expr('PutOn(Spare, Axle)') in spare_tire_solution + + cake_solution = astar_search(BackwardPlan(have_cake_and_eat_cake_too())).solution() + cake_solution = list(map(lambda action: Expr(action.name, *action.args), cake_solution)) + assert expr('Eat(Cake)') in cake_solution + assert expr('Bake(Cake)') in cake_solution + + air_cargo_solution = astar_search(BackwardPlan(air_cargo())).solution() + air_cargo_solution = list(map(lambda action: Expr(action.name, *action.args), air_cargo_solution)) + assert air_cargo_solution == [expr('Unload(C1, P1, JFK)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C2, P2, SFO)'), + expr('Fly(P2, JFK, SFO)'), + expr('Load(C2, P2, JFK)'), + expr('Load(C1, P1, SFO)')] or [expr('Load(C1, P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), + expr('Load(C2, P1, JFK)'), + expr('Fly(P1, JFK, SFO)'), + expr('Unload(C2, P1, SFO)')] + + sussman_anomaly_solution = astar_search(BackwardPlan(three_block_tower())).solution() + sussman_anomaly_solution = list(map(lambda action: Expr(action.name, *action.args), sussman_anomaly_solution)) + assert expr('MoveToTable(C, A)') in sussman_anomaly_solution + assert expr('Move(B, Table, C)') in sussman_anomaly_solution + assert expr('Move(A, Table, B)') in sussman_anomaly_solution + + blocks_world_solution = astar_search(BackwardPlan(simple_blocks_world())).solution() + blocks_world_solution = list(map(lambda action: Expr(action.name, *action.args), blocks_world_solution)) + assert expr('ToTable(A, B)') in blocks_world_solution + assert expr('FromTable(B, A)') in blocks_world_solution + assert expr('FromTable(C, B)') in blocks_world_solution + + shopping_problem_solution = astar_search(BackwardPlan(shopping_problem())).solution() + shopping_problem_solution = list(map(lambda action: Expr(action.name, *action.args), shopping_problem_solution)) + assert shopping_problem_solution == [expr('Go(Home, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)'), + expr('Go(SM, HW)'), + expr('Buy(Drill, HW)')] or [expr('Go(Home, HW)'), + expr('Buy(Drill, HW)'), + expr('Go(HW, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)')] + + +def test_CSPlan(): + spare_tire_solution = CSPlan(spare_tire(), 3) + assert expr('Remove(Flat, Axle)') in spare_tire_solution + assert expr('Remove(Spare, Trunk)') in spare_tire_solution + assert expr('PutOn(Spare, Axle)') in spare_tire_solution + + cake_solution = CSPlan(have_cake_and_eat_cake_too(), 2) + assert expr('Eat(Cake)') in cake_solution + assert expr('Bake(Cake)') in cake_solution + + air_cargo_solution = CSPlan(air_cargo(), 6) + assert air_cargo_solution == [expr('Load(C1, P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), + expr('Load(C2, P1, JFK)'), + expr('Fly(P1, JFK, SFO)'), + expr('Unload(C2, P1, SFO)')] or [expr('Load(C1, P1, SFO)'), + expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), + expr('Load(C2, P2, JFK)'), + expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)')] + + sussman_anomaly_solution = CSPlan(three_block_tower(), 3) + assert expr('MoveToTable(C, A)') in sussman_anomaly_solution + assert expr('Move(B, Table, C)') in sussman_anomaly_solution + assert expr('Move(A, Table, B)') in sussman_anomaly_solution + + blocks_world_solution = CSPlan(simple_blocks_world(), 3) + assert expr('ToTable(A, B)') in blocks_world_solution + assert expr('FromTable(B, A)') in blocks_world_solution + assert expr('FromTable(C, B)') in blocks_world_solution + + shopping_problem_solution = CSPlan(shopping_problem(), 5) + assert shopping_problem_solution == [expr('Go(Home, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)'), + expr('Go(SM, HW)'), + expr('Buy(Drill, HW)')] or [expr('Go(Home, HW)'), + expr('Buy(Drill, HW)'), + expr('Go(HW, SM)'), + expr('Buy(Banana, SM)'), + expr('Buy(Milk, SM)')] + + +def test_SATPlan(): + spare_tire_solution = SATPlan(spare_tire(), 3) + assert expr('Remove(Flat, Axle)') in spare_tire_solution + assert expr('Remove(Spare, Trunk)') in spare_tire_solution + assert expr('PutOn(Spare, Axle)') in spare_tire_solution + + cake_solution = SATPlan(have_cake_and_eat_cake_too(), 2) + assert expr('Eat(Cake)') in cake_solution + assert expr('Bake(Cake)') in cake_solution + + sussman_anomaly_solution = SATPlan(three_block_tower(), 3) + assert expr('MoveToTable(C, A)') in sussman_anomaly_solution + assert expr('Move(B, Table, C)') in sussman_anomaly_solution + assert expr('Move(A, Table, B)') in sussman_anomaly_solution + + blocks_world_solution = SATPlan(simple_blocks_world(), 3) + assert expr('ToTable(A, B)') in blocks_world_solution + assert expr('FromTable(B, A)') in blocks_world_solution + assert expr('FromTable(C, B)') in blocks_world_solution + + def test_linearize_class(): st = spare_tire() possible_solutions = [[expr('Remove(Spare, Trunk)'), expr('Remove(Flat, Axle)'), expr('PutOn(Spare, Axle)')], @@ -169,19 +402,31 @@ def test_linearize_class(): assert Linearize(st).execute() in possible_solutions ac = air_cargo() - possible_solutions = [[expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], - [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], - [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], - [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], - [expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], - [expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], - [expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')] - ] + possible_solutions = [ + [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], + [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C1, P1, SFO)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], + [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], + [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C2, P2, JFK)'), expr('Load(C1, P1, SFO)'), expr('Fly(P2, JFK, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], + [expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')], + [expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C1, P1, JFK)'), expr('Unload(C2, P2, SFO)')], + [expr('Load(C2, P2, JFK)'), expr('Fly(P2, JFK, SFO)'), expr('Load(C1, P1, SFO)'), expr('Fly(P1, SFO, JFK)'), + expr('Unload(C2, P2, SFO)'), expr('Unload(C1, P1, JFK)')]] assert Linearize(ac).execute() in possible_solutions ss = socks_and_shoes() @@ -190,18 +435,28 @@ def test_linearize_class(): [expr('RightSock'), expr('LeftSock'), expr('LeftShoe'), expr('RightShoe')], [expr('RightSock'), expr('LeftSock'), expr('RightShoe'), expr('LeftShoe')], [expr('LeftSock'), expr('LeftShoe'), expr('RightSock'), expr('RightShoe')], - [expr('RightSock'), expr('RightShoe'), expr('LeftSock'), expr('LeftShoe')] - ] + [expr('RightSock'), expr('RightShoe'), expr('LeftSock'), expr('LeftShoe')]] assert Linearize(ss).execute() in possible_solutions def test_expand_actions(): - assert len(PartialOrderPlanner(spare_tire()).expand_actions()) == 16 - assert len(PartialOrderPlanner(air_cargo()).expand_actions()) == 360 - assert len(PartialOrderPlanner(have_cake_and_eat_cake_too()).expand_actions()) == 2 - assert len(PartialOrderPlanner(socks_and_shoes()).expand_actions()) == 4 - assert len(PartialOrderPlanner(simple_blocks_world()).expand_actions()) == 12 - assert len(PartialOrderPlanner(three_block_tower()).expand_actions()) == 36 + assert len(spare_tire().expand_actions()) == 9 + assert len(air_cargo().expand_actions()) == 20 + assert len(have_cake_and_eat_cake_too().expand_actions()) == 2 + assert len(socks_and_shoes().expand_actions()) == 4 + assert len(simple_blocks_world().expand_actions()) == 12 + assert len(three_block_tower().expand_actions()) == 18 + assert len(shopping_problem().expand_actions()) == 12 + + +def test_expand_feats_values(): + assert len(spare_tire().expand_fluents()) == 10 + assert len(air_cargo().expand_fluents()) == 18 + assert len(have_cake_and_eat_cake_too().expand_fluents()) == 2 + assert len(socks_and_shoes().expand_fluents()) == 4 + assert len(simple_blocks_world().expand_fluents()) == 12 + assert len(three_block_tower().expand_fluents()) == 16 + assert len(shopping_problem().expand_fluents()) == 20 def test_find_open_precondition(): @@ -213,7 +468,10 @@ def test_find_open_precondition(): ss = socks_and_shoes() pop = PartialOrderPlanner(ss) - assert (pop.find_open_precondition()[0] == expr('LeftShoeOn') and pop.find_open_precondition()[2][0].name == 'LeftShoe') or (pop.find_open_precondition()[0] == expr('RightShoeOn') and pop.find_open_precondition()[2][0].name == 'RightShoe') + assert (pop.find_open_precondition()[0] == expr('LeftShoeOn') and + pop.find_open_precondition()[2][0].name == 'LeftShoe') or ( + pop.find_open_precondition()[0] == expr('RightShoeOn') and + pop.find_open_precondition()[2][0].name == 'RightShoe') assert pop.find_open_precondition()[1] == pop.finish cp = have_cake_and_eat_cake_too() @@ -229,7 +487,7 @@ def test_cyclic(): graph = [('a', 'b'), ('a', 'c'), ('b', 'c'), ('b', 'd'), ('d', 'e'), ('e', 'c')] assert not pop.cyclic(graph) - graph = [('a', 'b'), ('a', 'c'), ('b', 'c'), ('b', 'd'), ('d', 'e'), ('e', 'c'), ('e', 'b')] + graph = [('a', 'b'), ('a', 'c'), ('b', 'c'), ('b', 'd'), ('d', 'e'), ('e', 'c'), ('e', 'b')] assert pop.cyclic(graph) graph = [('a', 'b'), ('a', 'c'), ('b', 'c'), ('b', 'd'), ('d', 'e'), ('e', 'c'), ('b', 'e'), ('a', 'e')] @@ -242,26 +500,28 @@ def test_cyclic(): def test_partial_order_planner(): ss = socks_and_shoes() pop = PartialOrderPlanner(ss) - constraints, causal_links = pop.execute(display=False) + pop.execute(display=False) plan = list(reversed(list(pop.toposort(pop.convert(pop.constraints))))) assert list(plan[0])[0].name == 'Start' - assert (list(plan[1])[0].name == 'LeftSock' and list(plan[1])[1].name == 'RightSock') or (list(plan[1])[0].name == 'RightSock' and list(plan[1])[1].name == 'LeftSock') - assert (list(plan[2])[0].name == 'LeftShoe' and list(plan[2])[1].name == 'RightShoe') or (list(plan[2])[0].name == 'RightShoe' and list(plan[2])[1].name == 'LeftShoe') + assert (list(plan[1])[0].name == 'LeftSock' and list(plan[1])[1].name == 'RightSock') or ( + list(plan[1])[0].name == 'RightSock' and list(plan[1])[1].name == 'LeftSock') + assert (list(plan[2])[0].name == 'LeftShoe' and list(plan[2])[1].name == 'RightShoe') or ( + list(plan[2])[0].name == 'RightShoe' and list(plan[2])[1].name == 'LeftShoe') assert list(plan[3])[0].name == 'Finish' def test_double_tennis(): p = double_tennis_problem() - assert not goal_test(p.goals, p.init) + assert not goal_test(p.goals, p.initial) - solution = [expr("Go(A, RightBaseLine, LeftBaseLine)"), - expr("Hit(A, Ball, RightBaseLine)"), - expr("Go(A, LeftNet, RightBaseLine)")] + solution = [expr('Go(A, RightBaseLine, LeftBaseLine)'), + expr('Hit(A, Ball, RightBaseLine)'), + expr('Go(A, LeftNet, RightBaseLine)')] for action in solution: p.act(action) - assert goal_test(p.goals, p.init) + assert goal_test(p.goals, p.initial) def test_job_shop_problem(): @@ -283,88 +543,91 @@ def test_job_shop_problem(): # hierarchies library_1 = { - 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', 'Taxi(Home, SFO)'], - 'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []], - 'precond': [['At(Home) & Have(Car)'], ['At(Home)'], ['At(Home) & Have(Car)'], ['At(SFOLongTermParking)'], ['At(Home)']], - 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(SFOLongTermParking) & ~At(Home)'], ['At(SFO) & ~At(LongTermParking)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']] } - + 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)', + 'Taxi(Home, SFO)'], + 'steps': [['Drive(Home, SFOLongTermParking)', 'Shuttle(SFOLongTermParking, SFO)'], ['Taxi(Home, SFO)'], [], [], []], + 'precond': [['At(Home) & Have(Car)'], ['At(Home)'], ['At(Home) & Have(Car)'], ['At(SFOLongTermParking)'], + ['At(Home)']], + 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(SFOLongTermParking) & ~At(Home)'], + ['At(SFO) & ~At(LongTermParking)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']]} library_2 = { - 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)' , 'Metro(MetroStop, SFO)', 'Metro1(MetroStop, SFO)', 'Metro2(MetroStop, SFO)' ,'Taxi(Home, SFO)'], - 'steps': [['Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)'], ['Taxi(Home, SFO)'], [], ['Metro1(MetroStop, SFO)'], ['Metro2(MetroStop, SFO)'],[],[],[]], - 'precond': [['At(Home)'], ['At(Home)'], ['At(Home)'], ['At(MetroStop)'], ['At(MetroStop)'],['At(MetroStop)'], ['At(MetroStop)'] ,['At(Home) & Have(Cash)']], - 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(MetroStop) & ~At(Home)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'] , ['At(SFO) & ~At(MetroStop)'] ,['At(SFO) & ~At(Home) & ~Have(Cash)']] - } - + 'HLA': ['Go(Home,SFO)', 'Go(Home,SFO)', 'Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)', 'Metro(MetroStop, SFO)', + 'Metro1(MetroStop, SFO)', 'Metro2(MetroStop, SFO)', 'Taxi(Home, SFO)'], + 'steps': [['Bus(Home, MetroStop)', 'Metro(MetroStop, SFO)'], ['Taxi(Home, SFO)'], [], ['Metro1(MetroStop, SFO)'], + ['Metro2(MetroStop, SFO)'], [], [], []], + 'precond': [['At(Home)'], ['At(Home)'], ['At(Home)'], ['At(MetroStop)'], ['At(MetroStop)'], ['At(MetroStop)'], + ['At(MetroStop)'], ['At(Home) & Have(Cash)']], + 'effect': [['At(SFO) & ~At(Home)'], ['At(SFO) & ~At(Home) & ~Have(Cash)'], ['At(MetroStop) & ~At(Home)'], + ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(MetroStop)'], + ['At(SFO) & ~At(MetroStop)'], ['At(SFO) & ~At(Home) & ~Have(Cash)']]} # HLA's go_SFO = HLA('Go(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home)') taxi_SFO = HLA('Taxi(Home,SFO)', precond='At(Home)', effect='At(SFO) & ~At(Home) & ~Have(Cash)') -drive_SFOLongTermParking = HLA('Drive(Home, SFOLongTermParking)', 'At(Home) & Have(Car)','At(SFOLongTermParking) & ~At(Home)' ) +drive_SFOLongTermParking = HLA('Drive(Home, SFOLongTermParking)', 'At(Home) & Have(Car)', + 'At(SFOLongTermParking) & ~At(Home)') shuttle_SFO = HLA('Shuttle(SFOLongTermParking, SFO)', 'At(SFOLongTermParking)', 'At(SFO) & ~At(LongTermParking)') # Angelic HLA's -angelic_opt_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & $-At(Home)' ) -angelic_pes_description = Angelic_HLA('Go(Home, SFO)', precond = 'At(Home)', effect ='$+At(SFO) & ~At(Home)' ) +angelic_opt_description = AngelicHLA('Go(Home, SFO)', precond='At(Home)', effect='$+At(SFO) & $-At(Home)') +angelic_pes_description = AngelicHLA('Go(Home, SFO)', precond='At(Home)', effect='$+At(SFO) & ~At(Home)') # Angelic Nodes -plan1 = Angelic_Node('At(Home)', None, [angelic_opt_description], [angelic_pes_description]) -plan2 = Angelic_Node('At(Home)', None, [taxi_SFO]) -plan3 = Angelic_Node('At(Home)', None, [drive_SFOLongTermParking, shuttle_SFO]) +plan1 = AngelicNode('At(Home)', None, [angelic_opt_description], [angelic_pes_description]) +plan2 = AngelicNode('At(Home)', None, [taxi_SFO]) +plan3 = AngelicNode('At(Home)', None, [drive_SFOLongTermParking, shuttle_SFO]) # Problems -prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO, taxi_SFO, drive_SFOLongTermParking,shuttle_SFO]) +prob_1 = RealWorldPlanningProblem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', + [go_SFO, taxi_SFO, drive_SFOLongTermParking, shuttle_SFO]) -initialPlan = [Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description])] +initialPlan = [AngelicNode(prob_1.initial, None, [angelic_opt_description], [angelic_pes_description])] def test_refinements(): - - prob = Problem('At(Home) & Have(Car)', 'At(SFO)', [go_SFO]) - result = [i for i in Problem.refinements(go_SFO, prob, library_1)] - - assert(result[0][0].name == drive_SFOLongTermParking.name) - assert(result[0][0].args == drive_SFOLongTermParking.args) - assert(result[0][0].precond == drive_SFOLongTermParking.precond) - assert(result[0][0].effect == drive_SFOLongTermParking.effect) + result = [i for i in RealWorldPlanningProblem.refinements(go_SFO, library_1)] - assert(result[0][1].name == shuttle_SFO.name) - assert(result[0][1].args == shuttle_SFO.args) - assert(result[0][1].precond == shuttle_SFO.precond) - assert(result[0][1].effect == shuttle_SFO.effect) + assert (result[0][0].name == drive_SFOLongTermParking.name) + assert (result[0][0].args == drive_SFOLongTermParking.args) + assert (result[0][0].precond == drive_SFOLongTermParking.precond) + assert (result[0][0].effect == drive_SFOLongTermParking.effect) + assert (result[0][1].name == shuttle_SFO.name) + assert (result[0][1].args == shuttle_SFO.args) + assert (result[0][1].precond == shuttle_SFO.precond) + assert (result[0][1].effect == shuttle_SFO.effect) - assert(result[1][0].name == taxi_SFO.name) - assert(result[1][0].args == taxi_SFO.args) - assert(result[1][0].precond == taxi_SFO.precond) - assert(result[1][0].effect == taxi_SFO.effect) + assert (result[1][0].name == taxi_SFO.name) + assert (result[1][0].args == taxi_SFO.args) + assert (result[1][0].precond == taxi_SFO.precond) + assert (result[1][0].effect == taxi_SFO.effect) -def test_hierarchical_search(): +def test_hierarchical_search(): + # test_1 + prob_1 = RealWorldPlanningProblem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO]) - #test_1 - prob_1 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', [go_SFO]) + solution = RealWorldPlanningProblem.hierarchical_search(prob_1, library_1) - solution = Problem.hierarchical_search(prob_1, library_1) + assert (len(solution) == 2) - assert( len(solution) == 2 ) + assert (solution[0].name == drive_SFOLongTermParking.name) + assert (solution[0].args == drive_SFOLongTermParking.args) - assert(solution[0].name == drive_SFOLongTermParking.name) - assert(solution[0].args == drive_SFOLongTermParking.args) + assert (solution[1].name == shuttle_SFO.name) + assert (solution[1].args == shuttle_SFO.args) - assert(solution[1].name == shuttle_SFO.name) - assert(solution[1].args == shuttle_SFO.args) - - #test_2 - solution_2 = Problem.hierarchical_search(prob_1, library_2) + # test_2 + solution_2 = RealWorldPlanningProblem.hierarchical_search(prob_1, library_2) - assert( len(solution_2) == 2 ) + assert (len(solution_2) == 2) - assert(solution_2[0].name == 'Bus') - assert(solution_2[0].args == (expr('Home'), expr('MetroStop'))) + assert (solution_2[0].name == 'Bus') + assert (solution_2[0].args == (expr('Home'), expr('MetroStop'))) - assert(solution_2[1].name == 'Metro1') - assert(solution_2[1].args == (expr('MetroStop'), expr('SFO'))) + assert (solution_2[1].name == 'Metro1') + assert (solution_2[1].args == (expr('MetroStop'), expr('SFO'))) def test_convert_angelic_HLA(): @@ -375,25 +638,25 @@ def test_convert_angelic_HLA(): $-: Possibly delete (PosNo) $$: Possibly add / delete (PosYesNo) """ - ang1 = Angelic_HLA('Test', precond = None, effect = '~A') - ang2 = Angelic_HLA('Test', precond = None, effect = '$+A') - ang3 = Angelic_HLA('Test', precond = None, effect = '$-A') - ang4 = Angelic_HLA('Test', precond = None, effect = '$$A') + ang1 = AngelicHLA('Test', precond=None, effect='~A') + ang2 = AngelicHLA('Test', precond=None, effect='$+A') + ang3 = AngelicHLA('Test', precond=None, effect='$-A') + ang4 = AngelicHLA('Test', precond=None, effect='$$A') - assert(ang1.convert(ang1.effect) == [expr('NotA')]) - assert(ang2.convert(ang2.effect) == [expr('PosYesA')]) - assert(ang3.convert(ang3.effect) == [expr('PosNotA')]) - assert(ang4.convert(ang4.effect) == [expr('PosYesNotA')]) + assert (ang1.convert(ang1.effect) == [expr('NotA')]) + assert (ang2.convert(ang2.effect) == [expr('PosYesA')]) + assert (ang3.convert(ang3.effect) == [expr('PosNotA')]) + assert (ang4.convert(ang4.effect) == [expr('PosYesNotA')]) def test_is_primitive(): """ Tests if a plan is consisted out of primitive HLA's (angelic HLA's) """ - assert(not Problem.is_primitive(plan1, library_1)) - assert(Problem.is_primitive(plan2, library_1)) - assert(Problem.is_primitive(plan3, library_1)) - + assert (not RealWorldPlanningProblem.is_primitive(plan1, library_1)) + assert (RealWorldPlanningProblem.is_primitive(plan2, library_1)) + assert (RealWorldPlanningProblem.is_primitive(plan3, library_1)) + def test_angelic_action(): """ @@ -402,111 +665,110 @@ def test_angelic_action(): h1 : precondition positive: B _______ (add A) or (add A and remove B) effect: add A and possibly remove B - h2 : precondition positive: A _______ (add A and add C) or (delete A and add C) or (add C) or (add A and delete C) or - effect: possibly add/remove A and possibly add/remove C (delete A and delete C) or (delete C) or (add A) or (delete A) or [] + h2 : precondition positive: A _______ (add A and add C) or (delete A and add C) or + (add C) or (add A and delete C) or + effect: possibly add/remove A and possibly add/remove C (delete A and delete C) or (delete C) or + (add A) or (delete A) or [] """ - h_1 = Angelic_HLA( expr('h1'), 'B' , 'A & $-B') - h_2 = Angelic_HLA( expr('h2'), 'A', '$$A & $$C') - action_1 = Angelic_HLA.angelic_action(h_1) - action_2 = Angelic_HLA.angelic_action(h_2) - - assert ([a.effect for a in action_1] == [ [expr('A'),expr('NotB')], [expr('A')]] ) - assert ([a.effect for a in action_2] == [[expr('A') , expr('C')], [expr('NotA'), expr('C')], [expr('C')], [expr('A'), expr('NotC')], [expr('NotA'), expr('NotC')], [expr('NotC')], [expr('A')], [expr('NotA')], [None] ] ) + h_1 = AngelicHLA(expr('h1'), 'B', 'A & $-B') + h_2 = AngelicHLA(expr('h2'), 'A', '$$A & $$C') + action_1 = AngelicHLA.angelic_action(h_1) + action_2 = AngelicHLA.angelic_action(h_2) + + assert ([a.effect for a in action_1] == [[expr('A'), expr('NotB')], [expr('A')]]) + assert ([a.effect for a in action_2] == [[expr('A'), expr('C')], [expr('NotA'), expr('C')], [expr('C')], + [expr('A'), expr('NotC')], [expr('NotA'), expr('NotC')], [expr('NotC')], + [expr('A')], [expr('NotA')], [None]]) def test_optimistic_reachable_set(): """ Find optimistic reachable set given a problem initial state and a plan """ - h_1 = Angelic_HLA( 'h1', 'B' , '$+A & $-B ') - h_2 = Angelic_HLA( 'h2', 'A', '$$A & $$C') + h_1 = AngelicHLA('h1', 'B', '$+A & $-B ') + h_2 = AngelicHLA('h2', 'A', '$$A & $$C') f_1 = HLA('h1', 'B', 'A & ~B') f_2 = HLA('h2', 'A', 'A & C') - problem = Problem('B', 'A', [f_1,f_2] ) - plan = Angelic_Node(problem.init, None, [h_1,h_2], [h_1,h_2]) - opt_reachable_set = Problem.reach_opt(problem.init, plan ) - assert(opt_reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')],[expr('B'), expr('A')], [expr('B')]]) - assert( problem.intersects_goal(opt_reachable_set) ) + problem = RealWorldPlanningProblem('B', 'A', [f_1, f_2]) + plan = AngelicNode(problem.initial, None, [h_1, h_2], [h_1, h_2]) + opt_reachable_set = RealWorldPlanningProblem.reach_opt(problem.initial, plan) + assert (opt_reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')], [expr('B'), expr('A')], [expr('B')]]) + assert (problem.intersects_goal(opt_reachable_set)) -def test_pesssimistic_reachable_set(): +def test_pessimistic_reachable_set(): """ Find pessimistic reachable set given a problem initial state and a plan """ - h_1 = Angelic_HLA( 'h1', 'B' , '$+A & $-B ') - h_2 = Angelic_HLA( 'h2', 'A', '$$A & $$C') + h_1 = AngelicHLA('h1', 'B', '$+A & $-B ') + h_2 = AngelicHLA('h2', 'A', '$$A & $$C') f_1 = HLA('h1', 'B', 'A & ~B') f_2 = HLA('h2', 'A', 'A & C') - problem = Problem('B', 'A', [f_1,f_2] ) - plan = Angelic_Node(problem.init, None, [h_1,h_2], [h_1,h_2]) - pes_reachable_set = Problem.reach_pes(problem.init, plan ) - assert(pes_reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')],[expr('B'), expr('A')], [expr('B')]]) - assert(problem.intersects_goal(pes_reachable_set)) + problem = RealWorldPlanningProblem('B', 'A', [f_1, f_2]) + plan = AngelicNode(problem.initial, None, [h_1, h_2], [h_1, h_2]) + pes_reachable_set = RealWorldPlanningProblem.reach_pes(problem.initial, plan) + assert (pes_reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')], [expr('B'), expr('A')], [expr('B')]]) + assert (problem.intersects_goal(pes_reachable_set)) def test_find_reachable_set(): - h_1 = Angelic_HLA( 'h1', 'B' , '$+A & $-B ') + h_1 = AngelicHLA('h1', 'B', '$+A & $-B ') f_1 = HLA('h1', 'B', 'A & ~B') - problem = Problem('B', 'A', [f_1] ) - plan = Angelic_Node(problem.init, None, [h_1], [h_1]) - reachable_set = {0: [problem.init]} + problem = RealWorldPlanningProblem('B', 'A', [f_1]) + reachable_set = {0: [problem.initial]} action_description = [h_1] - reachable_set = Problem.find_reachable_set(reachable_set, action_description) - assert(reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')],[expr('B'), expr('A')], [expr('B')]]) + reachable_set = RealWorldPlanningProblem.find_reachable_set(reachable_set, action_description) + assert (reachable_set[1] == [[expr('A'), expr('NotB')], [expr('NotB')], [expr('B'), expr('A')], [expr('B')]]) +def test_intersects_goal(): + problem_1 = RealWorldPlanningProblem('At(SFO)', 'At(SFO)', []) + problem_2 = RealWorldPlanningProblem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', []) + reachable_set_1 = {0: [problem_1.initial]} + reachable_set_2 = {0: [problem_2.initial]} -def test_intersects_goal(): - problem_1 = Problem('At(SFO)', 'At(SFO)', []) - problem_2 = Problem('At(Home) & Have(Cash) & Have(Car) ', 'At(SFO) & Have(Cash)', []) - reachable_set_1 = {0: [problem_1.init]} - reachable_set_2 = {0: [problem_2.init]} - - assert(Problem.intersects_goal(problem_1, reachable_set_1)) - assert(not Problem.intersects_goal(problem_2, reachable_set_2)) + assert (RealWorldPlanningProblem.intersects_goal(problem_1, reachable_set_1)) + assert (not RealWorldPlanningProblem.intersects_goal(problem_2, reachable_set_2)) def test_making_progress(): """ function not yet implemented """ - - intialPlan_1 = [Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description]), - Angelic_Node(prob_1.init, None, [angelic_pes_description], [angelic_pes_description]) ] - plan_1 = Angelic_Node(prob_1.init, None, [angelic_opt_description], [angelic_pes_description]) + plan_1 = AngelicNode(prob_1.initial, None, [angelic_opt_description], [angelic_pes_description]) + + assert (not RealWorldPlanningProblem.making_progress(plan_1, initialPlan)) - assert(not Problem.making_progress(plan_1, initialPlan)) -def test_angelic_search(): +def test_angelic_search(): """ Test angelic search for problem, hierarchy, initialPlan """ - #test_1 - solution = Problem.angelic_search(prob_1, library_1, initialPlan) - - assert( len(solution) == 2 ) + # test_1 + solution = RealWorldPlanningProblem.angelic_search(prob_1, library_1, initialPlan) - assert(solution[0].name == drive_SFOLongTermParking.name) - assert(solution[0].args == drive_SFOLongTermParking.args) + assert (len(solution) == 2) - assert(solution[1].name == shuttle_SFO.name) - assert(solution[1].args == shuttle_SFO.args) - + assert (solution[0].name == drive_SFOLongTermParking.name) + assert (solution[0].args == drive_SFOLongTermParking.args) - #test_2 - solution_2 = Problem.angelic_search(prob_1, library_2, initialPlan) + assert (solution[1].name == shuttle_SFO.name) + assert (solution[1].args == shuttle_SFO.args) - assert( len(solution_2) == 2 ) + # test_2 + solution_2 = RealWorldPlanningProblem.angelic_search(prob_1, library_2, initialPlan) - assert(solution_2[0].name == 'Bus') - assert(solution_2[0].args == (expr('Home'), expr('MetroStop'))) + assert (len(solution_2) == 2) - assert(solution_2[1].name == 'Metro1') - assert(solution_2[1].args == (expr('MetroStop'), expr('SFO'))) - + assert (solution_2[0].name == 'Bus') + assert (solution_2[0].args == (expr('Home'), expr('MetroStop'))) + assert (solution_2[1].name == 'Metro1') + assert (solution_2[1].args == (expr('MetroStop'), expr('SFO'))) +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_probabilistic_learning.py b/tests/test_probabilistic_learning.py new file mode 100644 index 000000000..bd37b6ebb --- /dev/null +++ b/tests/test_probabilistic_learning.py @@ -0,0 +1,38 @@ +import random + +import pytest + +from learning import DataSet +from probabilistic_learning import * + +random.seed("aima-python") + + +def test_naive_bayes(): + iris = DataSet(name='iris') + # discrete + nbd = NaiveBayesLearner(iris, continuous=False) + assert nbd([5, 3, 1, 0.1]) == 'setosa' + assert nbd([6, 3, 4, 1.1]) == 'versicolor' + assert nbd([7.7, 3, 6, 2]) == 'virginica' + # continuous + nbc = NaiveBayesLearner(iris, continuous=True) + assert nbc([5, 3, 1, 0.1]) == 'setosa' + assert nbc([6, 5, 3, 1.5]) == 'versicolor' + assert nbc([7, 3, 6.5, 2]) == 'virginica' + # simple + data1 = 'a' * 50 + 'b' * 30 + 'c' * 15 + dist1 = CountingProbDist(data1) + data2 = 'a' * 30 + 'b' * 45 + 'c' * 20 + dist2 = CountingProbDist(data2) + data3 = 'a' * 20 + 'b' * 20 + 'c' * 35 + dist3 = CountingProbDist(data3) + dist = {('First', 0.5): dist1, ('Second', 0.3): dist2, ('Third', 0.2): dist3} + nbs = NaiveBayesLearner(dist, simple=True) + assert nbs('aab') == 'First' + assert nbs(['b', 'b']) == 'Second' + assert nbs('ccbcc') == 'Third' + + +if __name__ == "__main__": + pytest.main() diff --git a/tests/test_probability.py b/tests/test_probability.py index b4d720937..8def79c68 100644 --- a/tests/test_probability.py +++ b/tests/test_probability.py @@ -1,7 +1,10 @@ -import random +import pytest + from probability import * from utils import rounder +random.seed("aima-python") + def tests(): cpt = burglary.variable_node('Alarm') @@ -9,7 +12,7 @@ def tests(): assert cpt.p(True, event) == 0.95 event = {'Burglary': False, 'Earthquake': True} assert cpt.p(False, event) == 0.71 - # #enumeration_ask('Earthquake', {}, burglary) + # enumeration_ask('Earthquake', {}, burglary) s = {'A': True, 'B': False, 'C': True, 'D': False} assert consistent_with(s, {}) @@ -47,7 +50,7 @@ def test_probdist_frequency(): P = ProbDist('Pascal-5', {'x1': 1, 'x2': 5, 'x3': 10, 'x4': 10, 'x5': 5, 'x6': 1}) assert (P['x1'], P['x2'], P['x3'], P['x4'], P['x5'], P['x6']) == ( - 0.03125, 0.15625, 0.3125, 0.3125, 0.15625, 0.03125) + 0.03125, 0.15625, 0.3125, 0.3125, 0.15625, 0.03125) def test_probdist_normalize(): @@ -60,7 +63,7 @@ def test_probdist_normalize(): P['1'], P['2'], P['3'], P['4'], P['5'], P['6'] = 10, 15, 25, 30, 40, 80 P = P.normalize() assert (P.prob['1'], P.prob['2'], P.prob['3'], P.prob['4'], P.prob['5'], P.prob['6']) == ( - 0.05, 0.075, 0.125, 0.15, 0.2, 0.4) + 0.05, 0.075, 0.125, 0.15, 0.2, 0.4) def test_jointprob(): @@ -106,7 +109,7 @@ def test_enumerate_joint_ask(): P[0, 1] = 0.5 P[1, 1] = P[2, 1] = 0.125 assert enumerate_joint_ask( - 'X', dict(Y=1), P).show_approx() == '0: 0.667, 1: 0.167, 2: 0.167' + 'X', dict(Y=1), P).show_approx() == '0: 0.667, 1: 0.167, 2: 0.167' def test_bayesnode_p(): @@ -126,47 +129,47 @@ def test_bayesnode_sample(): def test_enumeration_ask(): assert enumeration_ask( - 'Burglary', dict(JohnCalls=T, MaryCalls=T), - burglary).show_approx() == 'False: 0.716, True: 0.284' + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary).show_approx() == 'False: 0.716, True: 0.284' assert enumeration_ask( - 'Burglary', dict(JohnCalls=T, MaryCalls=F), - burglary).show_approx() == 'False: 0.995, True: 0.00513' + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary).show_approx() == 'False: 0.995, True: 0.00513' assert enumeration_ask( - 'Burglary', dict(JohnCalls=F, MaryCalls=T), - burglary).show_approx() == 'False: 0.993, True: 0.00688' + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary).show_approx() == 'False: 0.993, True: 0.00688' assert enumeration_ask( - 'Burglary', dict(JohnCalls=T), - burglary).show_approx() == 'False: 0.984, True: 0.0163' + 'Burglary', dict(JohnCalls=T), + burglary).show_approx() == 'False: 0.984, True: 0.0163' assert enumeration_ask( - 'Burglary', dict(MaryCalls=T), - burglary).show_approx() == 'False: 0.944, True: 0.0561' + 'Burglary', dict(MaryCalls=T), + burglary).show_approx() == 'False: 0.944, True: 0.0561' -def test_elemination_ask(): +def test_elimination_ask(): assert elimination_ask( - 'Burglary', dict(JohnCalls=T, MaryCalls=T), - burglary).show_approx() == 'False: 0.716, True: 0.284' + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary).show_approx() == 'False: 0.716, True: 0.284' assert elimination_ask( - 'Burglary', dict(JohnCalls=T, MaryCalls=F), - burglary).show_approx() == 'False: 0.995, True: 0.00513' + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary).show_approx() == 'False: 0.995, True: 0.00513' assert elimination_ask( - 'Burglary', dict(JohnCalls=F, MaryCalls=T), - burglary).show_approx() == 'False: 0.993, True: 0.00688' + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary).show_approx() == 'False: 0.993, True: 0.00688' assert elimination_ask( - 'Burglary', dict(JohnCalls=T), - burglary).show_approx() == 'False: 0.984, True: 0.0163' + 'Burglary', dict(JohnCalls=T), + burglary).show_approx() == 'False: 0.984, True: 0.0163' assert elimination_ask( - 'Burglary', dict(MaryCalls=T), - burglary).show_approx() == 'False: 0.944, True: 0.0561' + 'Burglary', dict(MaryCalls=T), + burglary).show_approx() == 'False: 0.944, True: 0.0561' def test_prior_sample(): random.seed(42) all_obs = [prior_sample(burglary) for x in range(1000)] - john_calls_true = [observation for observation in all_obs if observation['JohnCalls'] == True] - mary_calls_true = [observation for observation in all_obs if observation['MaryCalls'] == True] - burglary_and_john = [observation for observation in john_calls_true if observation['Burglary'] == True] - burglary_and_mary = [observation for observation in mary_calls_true if observation['Burglary'] == True] + john_calls_true = [observation for observation in all_obs if observation['JohnCalls']] + mary_calls_true = [observation for observation in all_obs if observation['MaryCalls']] + burglary_and_john = [observation for observation in john_calls_true if observation['Burglary']] + burglary_and_mary = [observation for observation in mary_calls_true if observation['Burglary']] assert len(john_calls_true) / 1000 == 46 / 1000 assert len(mary_calls_true) / 1000 == 13 / 1000 assert len(burglary_and_john) / len(john_calls_true) == 1 / 46 @@ -176,10 +179,10 @@ def test_prior_sample(): def test_prior_sample2(): random.seed(128) all_obs = [prior_sample(sprinkler) for x in range(1000)] - rain_true = [observation for observation in all_obs if observation['Rain'] == True] - sprinkler_true = [observation for observation in all_obs if observation['Sprinkler'] == True] - rain_and_cloudy = [observation for observation in rain_true if observation['Cloudy'] == True] - sprinkler_and_cloudy = [observation for observation in sprinkler_true if observation['Cloudy'] == True] + rain_true = [observation for observation in all_obs if observation['Rain']] + sprinkler_true = [observation for observation in all_obs if observation['Sprinkler']] + rain_and_cloudy = [observation for observation in rain_true if observation['Cloudy']] + sprinkler_and_cloudy = [observation for observation in sprinkler_true if observation['Cloudy']] assert len(rain_true) / 1000 == 0.476 assert len(sprinkler_true) / 1000 == 0.291 assert len(rain_and_cloudy) / len(rain_true) == 376 / 476 @@ -189,97 +192,108 @@ def test_prior_sample2(): def test_rejection_sampling(): random.seed(47) assert rejection_sampling( - 'Burglary', dict(JohnCalls=T, MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.7, True: 0.3' + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.7, True: 0.3' assert rejection_sampling( - 'Burglary', dict(JohnCalls=T, MaryCalls=F), - burglary, 10000).show_approx() == 'False: 1, True: 0' + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 1, True: 0' assert rejection_sampling( - 'Burglary', dict(JohnCalls=F, MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.987, True: 0.0128' + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.987, True: 0.0128' assert rejection_sampling( - 'Burglary', dict(JohnCalls=T), - burglary, 10000).show_approx() == 'False: 0.982, True: 0.0183' + 'Burglary', dict(JohnCalls=T), + burglary, 10000).show_approx() == 'False: 0.982, True: 0.0183' assert rejection_sampling( - 'Burglary', dict(MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.965, True: 0.0348' + 'Burglary', dict(MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.965, True: 0.0348' def test_rejection_sampling2(): random.seed(42) assert rejection_sampling( - 'Cloudy', dict(Rain=T, Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.56, True: 0.44' + 'Cloudy', dict(Rain=T, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.56, True: 0.44' assert rejection_sampling( - 'Cloudy', dict(Rain=T, Sprinkler=F), - sprinkler, 10000).show_approx() == 'False: 0.119, True: 0.881' + 'Cloudy', dict(Rain=T, Sprinkler=F), + sprinkler, 10000).show_approx() == 'False: 0.119, True: 0.881' assert rejection_sampling( - 'Cloudy', dict(Rain=F, Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.049' + 'Cloudy', dict(Rain=F, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.049' assert rejection_sampling( - 'Cloudy', dict(Rain=T), - sprinkler, 10000).show_approx() == 'False: 0.205, True: 0.795' + 'Cloudy', dict(Rain=T), + sprinkler, 10000).show_approx() == 'False: 0.205, True: 0.795' assert rejection_sampling( - 'Cloudy', dict(Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.835, True: 0.165' + 'Cloudy', dict(Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.835, True: 0.165' def test_likelihood_weighting(): random.seed(1017) assert likelihood_weighting( - 'Burglary', dict(JohnCalls=T, MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.702, True: 0.298' + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.702, True: 0.298' assert likelihood_weighting( - 'Burglary', dict(JohnCalls=T, MaryCalls=F), - burglary, 10000).show_approx() == 'False: 0.993, True: 0.00656' + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 0.993, True: 0.00656' assert likelihood_weighting( - 'Burglary', dict(JohnCalls=F, MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.996, True: 0.00363' + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.996, True: 0.00363' assert likelihood_weighting( - 'Burglary', dict(JohnCalls=F, MaryCalls=F), - burglary, 10000).show_approx() == 'False: 1, True: 0.000126' + 'Burglary', dict(JohnCalls=F, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 1, True: 0.000126' assert likelihood_weighting( - 'Burglary', dict(JohnCalls=T), - burglary, 10000).show_approx() == 'False: 0.979, True: 0.0205' + 'Burglary', dict(JohnCalls=T), + burglary, 10000).show_approx() == 'False: 0.979, True: 0.0205' assert likelihood_weighting( - 'Burglary', dict(MaryCalls=T), - burglary, 10000).show_approx() == 'False: 0.94, True: 0.0601' + 'Burglary', dict(MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.94, True: 0.0601' def test_likelihood_weighting2(): random.seed(42) assert likelihood_weighting( - 'Cloudy', dict(Rain=T, Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.559, True: 0.441' + 'Cloudy', dict(Rain=T, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.559, True: 0.441' assert likelihood_weighting( - 'Cloudy', dict(Rain=T, Sprinkler=F), - sprinkler, 10000).show_approx() == 'False: 0.12, True: 0.88' + 'Cloudy', dict(Rain=T, Sprinkler=F), + sprinkler, 10000).show_approx() == 'False: 0.12, True: 0.88' assert likelihood_weighting( - 'Cloudy', dict(Rain=F, Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.0486' + 'Cloudy', dict(Rain=F, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.0486' assert likelihood_weighting( - 'Cloudy', dict(Rain=T), - sprinkler, 10000).show_approx() == 'False: 0.198, True: 0.802' + 'Cloudy', dict(Rain=T), + sprinkler, 10000).show_approx() == 'False: 0.198, True: 0.802' assert likelihood_weighting( - 'Cloudy', dict(Sprinkler=T), - sprinkler, 10000).show_approx() == 'False: 0.833, True: 0.167' + 'Cloudy', dict(Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.833, True: 0.167' def test_forward_backward(): - umbrella_prior = [0.5, 0.5] umbrella_transition = [[0.7, 0.3], [0.3, 0.7]] umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]] umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) umbrella_evidence = [T, T, F, T, T] - assert (rounder(forward_backward(umbrellaHMM, umbrella_evidence, umbrella_prior)) == - [[0.6469, 0.3531], [0.8673, 0.1327], [0.8204, 0.1796], [0.3075, 0.6925], - [0.8204, 0.1796], [0.8673, 0.1327]]) + assert rounder(forward_backward(umbrellaHMM, umbrella_evidence)) == [ + [0.6469, 0.3531], [0.8673, 0.1327], [0.8204, 0.1796], [0.3075, 0.6925], [0.8204, 0.1796], [0.8673, 0.1327]] + + umbrella_evidence = [T, F, T, F, T] + assert rounder(forward_backward(umbrellaHMM, umbrella_evidence)) == [ + [0.5871, 0.4129], [0.7177, 0.2823], [0.2324, 0.7676], [0.6072, 0.3928], [0.2324, 0.7676], [0.7177, 0.2823]] + + +def test_viterbi(): + umbrella_transition = [[0.7, 0.3], [0.3, 0.7]] + umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]] + umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) + + umbrella_evidence = [T, T, F, T, T] + assert viterbi(umbrellaHMM, umbrella_evidence)[0] == [T, T, F, T, T] + assert rounder(viterbi(umbrellaHMM, umbrella_evidence)[1]) == [0.8182, 0.5155, 0.1237, 0.0334, 0.0210] umbrella_evidence = [T, F, T, F, T] - assert rounder(forward_backward(umbrellaHMM, umbrella_evidence, umbrella_prior)) == [ - [0.5871, 0.4129], [0.7177, 0.2823], [0.2324, 0.7676], [0.6072, 0.3928], - [0.2324, 0.7676], [0.7177, 0.2823]] + assert viterbi(umbrellaHMM, umbrella_evidence)[0] == [T, F, F, F, T] + assert rounder(viterbi(umbrellaHMM, umbrella_evidence)[1]) == [0.8182, 0.1964, 0.0275, 0.0154, 0.0042] def test_fixed_lag_smoothing(): @@ -291,8 +305,7 @@ def test_fixed_lag_smoothing(): umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor) d = 2 - assert rounder(fixed_lag_smoothing(e_t, umbrellaHMM, d, - umbrella_evidence, t)) == [0.1111, 0.8889] + assert rounder(fixed_lag_smoothing(e_t, umbrellaHMM, d, umbrella_evidence, t)) == [0.1111, 0.8889] d = 5 assert fixed_lag_smoothing(e_t, umbrellaHMM, d, umbrella_evidence, t) is None @@ -301,8 +314,7 @@ def test_fixed_lag_smoothing(): e_t = T d = 1 - assert rounder(fixed_lag_smoothing(e_t, umbrellaHMM, - d, umbrella_evidence, t)) == [0.9939, 0.0061] + assert rounder(fixed_lag_smoothing(e_t, umbrellaHMM, d, umbrella_evidence, t)) == [0.9939, 0.0061] def test_particle_filtering(): @@ -318,7 +330,7 @@ def test_particle_filtering(): def test_monte_carlo_localization(): - ## TODO: Add tests for random motion/inaccurate sensors + # TODO: Add tests for random motion/inaccurate sensors random.seed('aima-python') m = MCLmap([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0], @@ -334,12 +346,12 @@ def test_monte_carlo_localization(): def P_motion_sample(kin_state, v, w): """Sample from possible kinematic states. - Returns from a single element distribution (no uncertainity in motion)""" + Returns from a single element distribution (no uncertainty in motion)""" pos = kin_state[:2] orient = kin_state[2] # for simplicity the robot first rotates and then moves - orient = (orient + w)%4 + orient = (orient + w) % 4 for _ in range(orient): v = (v[1], -v[0]) pos = vector_add(pos, v) @@ -359,7 +371,7 @@ def P_sensor(x, y): a = {'v': (0, 0), 'w': 0} z = (2, 4, 1, 6) S = monte_carlo_localization(a, z, 1000, P_motion_sample, P_sensor, m) - grid = [[0]*17 for _ in range(11)] + grid = [[0] * 17 for _ in range(11)] for x, y, _ in S: if 0 <= x < 11 and 0 <= y < 17: grid[x][y] += 1 @@ -369,7 +381,7 @@ def P_sensor(x, y): a = {'v': (0, 1), 'w': 0} z = (2, 3, 5, 7) S = monte_carlo_localization(a, z, 1000, P_motion_sample, P_sensor, m, S) - grid = [[0]*17 for _ in range(11)] + grid = [[0] * 17 for _ in range(11)] for x, y, _ in S: if 0 <= x < 11 and 0 <= y < 17: grid[x][y] += 1 @@ -380,8 +392,7 @@ def P_sensor(x, y): def test_gibbs_ask(): - possible_solutions = ['False: 0.16, True: 0.84', 'False: 0.17, True: 0.83', - 'False: 0.15, True: 0.85'] + possible_solutions = ['False: 0.16, True: 0.84', 'False: 0.17, True: 0.83', 'False: 0.15, True: 0.85'] g_solution = gibbs_ask('Cloudy', dict(Rain=True), sprinkler, 200).show_approx() assert g_solution in possible_solutions diff --git a/tests/test_probability4e.py b/tests/test_probability4e.py new file mode 100644 index 000000000..d07954e0a --- /dev/null +++ b/tests/test_probability4e.py @@ -0,0 +1,349 @@ +import pytest + +from probability4e import * + +random.seed("aima-python") + + +def tests(): + cpt = burglary.variable_node('Alarm') + event = {'Burglary': True, 'Earthquake': True} + assert cpt.p(True, event) == 0.95 + event = {'Burglary': False, 'Earthquake': True} + assert cpt.p(False, event) == 0.71 + # enumeration_ask('Earthquake', {}, burglary) + + s = {'A': True, 'B': False, 'C': True, 'D': False} + assert consistent_with(s, {}) + assert consistent_with(s, s) + assert not consistent_with(s, {'A': False}) + assert not consistent_with(s, {'D': True}) + + random.seed(21) + p = rejection_sampling('Earthquake', {}, burglary, 1000) + assert p[True], p[False] == (0.001, 0.999) + + random.seed(71) + p = likelihood_weighting('Earthquake', {}, burglary, 1000) + assert p[True], p[False] == (0.002, 0.998) + + +# test ProbDist + + +def test_probdist_basic(): + P = ProbDist('Flip') + P['H'], P['T'] = 0.25, 0.75 + assert P['H'] == 0.25 + assert P['T'] == 0.75 + assert P['X'] == 0.00 + + P = ProbDist('BiasedDie') + P['1'], P['2'], P['3'], P['4'], P['5'], P['6'] = 10, 15, 25, 30, 40, 80 + P.normalize() + assert P['2'] == 0.075 + assert P['4'] == 0.15 + assert P['6'] == 0.4 + + +def test_probdist_frequency(): + P = ProbDist('X', {'lo': 125, 'med': 375, 'hi': 500}) + assert (P['lo'], P['med'], P['hi']) == (0.125, 0.375, 0.5) + + P = ProbDist('Pascal-5', {'x1': 1, 'x2': 5, 'x3': 10, 'x4': 10, 'x5': 5, 'x6': 1}) + assert (P['x1'], P['x2'], P['x3'], P['x4'], P['x5'], P['x6']) == ( + 0.03125, 0.15625, 0.3125, 0.3125, 0.15625, 0.03125) + + +def test_probdist_normalize(): + P = ProbDist('Flip') + P['H'], P['T'] = 35, 65 + P = P.normalize() + assert (P.prob['H'], P.prob['T']) == (0.350, 0.650) + + P = ProbDist('BiasedDie') + P['1'], P['2'], P['3'], P['4'], P['5'], P['6'] = 10, 15, 25, 30, 40, 80 + P = P.normalize() + assert (P.prob['1'], P.prob['2'], P.prob['3'], P.prob['4'], P.prob['5'], P.prob['6']) == ( + 0.05, 0.075, 0.125, 0.15, 0.2, 0.4) + + +# test JoinProbDist + + +def test_jointprob(): + P = JointProbDist(['X', 'Y']) + P[1, 1] = 0.25 + assert P[1, 1] == 0.25 + P[dict(X=0, Y=1)] = 0.5 + assert P[dict(X=0, Y=1)] == 0.5 + + +def test_event_values(): + assert event_values({'A': 10, 'B': 9, 'C': 8}, ['C', 'A']) == (8, 10) + assert event_values((1, 2), ['C', 'A']) == (1, 2) + + +def test_enumerate_joint(): + P = JointProbDist(['X', 'Y']) + P[0, 0] = 0.25 + P[0, 1] = 0.5 + P[1, 1] = P[2, 1] = 0.125 + assert enumerate_joint(['Y'], dict(X=0), P) == 0.75 + assert enumerate_joint(['X'], dict(Y=2), P) == 0 + assert enumerate_joint(['X'], dict(Y=1), P) == 0.75 + + Q = JointProbDist(['W', 'X', 'Y', 'Z']) + Q[0, 1, 1, 0] = 0.12 + Q[1, 0, 1, 1] = 0.4 + Q[0, 0, 1, 1] = 0.5 + Q[0, 0, 1, 0] = 0.05 + Q[0, 0, 0, 0] = 0.675 + Q[1, 1, 1, 0] = 0.3 + assert enumerate_joint(['W'], dict(X=0, Y=0, Z=1), Q) == 0 + assert enumerate_joint(['W'], dict(X=0, Y=0, Z=0), Q) == 0.675 + assert enumerate_joint(['W'], dict(X=0, Y=1, Z=1), Q) == 0.9 + assert enumerate_joint(['Y'], dict(W=1, X=0, Z=1), Q) == 0.4 + assert enumerate_joint(['Z'], dict(W=0, X=0, Y=0), Q) == 0.675 + assert enumerate_joint(['Z'], dict(W=1, X=1, Y=1), Q) == 0.3 + + +def test_enumerate_joint_ask(): + P = JointProbDist(['X', 'Y']) + P[0, 0] = 0.25 + P[0, 1] = 0.5 + P[1, 1] = P[2, 1] = 0.125 + assert enumerate_joint_ask( + 'X', dict(Y=1), P).show_approx() == '0: 0.667, 1: 0.167, 2: 0.167' + + +def test_is_independent(): + P = JointProbDist(['X', 'Y']) + P[0, 0] = P[0, 1] = P[1, 1] = P[1, 0] = 0.25 + assert enumerate_joint_ask( + 'X', dict(Y=1), P).show_approx() == '0: 0.5, 1: 0.5' + assert is_independent(['X', 'Y'], P) + + +# test BayesNode + + +def test_bayesnode_p(): + bn = BayesNode('X', 'Burglary', {T: 0.2, F: 0.625}) + assert bn.p(True, {'Burglary': True, 'Earthquake': False}) == 0.2 + assert bn.p(False, {'Burglary': False, 'Earthquake': True}) == 0.375 + assert BayesNode('W', '', 0.75).p(False, {'Random': True}) == 0.25 + + +def test_bayesnode_sample(): + X = BayesNode('X', 'Burglary', {T: 0.2, F: 0.625}) + assert X.sample({'Burglary': False, 'Earthquake': True}) in [True, False] + Z = BayesNode('Z', 'P Q', {(True, True): 0.2, (True, False): 0.3, + (False, True): 0.5, (False, False): 0.7}) + assert Z.sample({'P': True, 'Q': False}) in [True, False] + + +# test continuous variable bayesian net + + +def test_gaussian_probability(): + param = {'sigma': 0.5, 'b': 1, 'a': {'h': 0.5}} + event = {'h': 0.6} + assert gaussian_probability(param, event, 1) == 0.6664492057835993 + + +def test_logistic_probability(): + param = {'mu': 0.5, 'sigma': 0.1} + event = {'h': 0.6} + assert logistic_probability(param, event, True) == 0.16857376940725355 + assert logistic_probability(param, event, False) == 0.8314262305927465 + + +def test_enumeration_ask(): + assert enumeration_ask( + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary).show_approx() == 'False: 0.716, True: 0.284' + assert enumeration_ask( + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary).show_approx() == 'False: 0.995, True: 0.00513' + assert enumeration_ask( + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary).show_approx() == 'False: 0.993, True: 0.00688' + assert enumeration_ask( + 'Burglary', dict(JohnCalls=T), + burglary).show_approx() == 'False: 0.984, True: 0.0163' + assert enumeration_ask( + 'Burglary', dict(MaryCalls=T), + burglary).show_approx() == 'False: 0.944, True: 0.0561' + + +def test_elimination_ask(): + assert elimination_ask( + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary).show_approx() == 'False: 0.716, True: 0.284' + assert elimination_ask( + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary).show_approx() == 'False: 0.995, True: 0.00513' + assert elimination_ask( + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary).show_approx() == 'False: 0.993, True: 0.00688' + assert elimination_ask( + 'Burglary', dict(JohnCalls=T), + burglary).show_approx() == 'False: 0.984, True: 0.0163' + assert elimination_ask( + 'Burglary', dict(MaryCalls=T), + burglary).show_approx() == 'False: 0.944, True: 0.0561' + + +# test sampling + + +def test_prior_sample(): + random.seed(42) + all_obs = [prior_sample(burglary) for x in range(1000)] + john_calls_true = [observation for observation in all_obs if observation['JohnCalls'] is True] + mary_calls_true = [observation for observation in all_obs if observation['MaryCalls'] is True] + burglary_and_john = [observation for observation in john_calls_true if observation['Burglary'] is True] + burglary_and_mary = [observation for observation in mary_calls_true if observation['Burglary'] is True] + assert len(john_calls_true) / 1000 == 46 / 1000 + assert len(mary_calls_true) / 1000 == 13 / 1000 + assert len(burglary_and_john) / len(john_calls_true) == 1 / 46 + assert len(burglary_and_mary) / len(mary_calls_true) == 1 / 13 + + +def test_prior_sample2(): + random.seed(128) + all_obs = [prior_sample(sprinkler) for x in range(1000)] + rain_true = [observation for observation in all_obs if observation['Rain'] is True] + sprinkler_true = [observation for observation in all_obs if observation['Sprinkler'] is True] + rain_and_cloudy = [observation for observation in rain_true if observation['Cloudy'] is True] + sprinkler_and_cloudy = [observation for observation in sprinkler_true if observation['Cloudy'] is True] + assert len(rain_true) / 1000 == 0.476 + assert len(sprinkler_true) / 1000 == 0.291 + assert len(rain_and_cloudy) / len(rain_true) == 376 / 476 + assert len(sprinkler_and_cloudy) / len(sprinkler_true) == 39 / 291 + + +def test_rejection_sampling(): + random.seed(47) + assert rejection_sampling( + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.7, True: 0.3' + assert rejection_sampling( + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 1, True: 0' + assert rejection_sampling( + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.987, True: 0.0128' + assert rejection_sampling( + 'Burglary', dict(JohnCalls=T), + burglary, 10000).show_approx() == 'False: 0.982, True: 0.0183' + assert rejection_sampling( + 'Burglary', dict(MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.965, True: 0.0348' + + +def test_rejection_sampling2(): + random.seed(42) + assert rejection_sampling( + 'Cloudy', dict(Rain=T, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.56, True: 0.44' + assert rejection_sampling( + 'Cloudy', dict(Rain=T, Sprinkler=F), + sprinkler, 10000).show_approx() == 'False: 0.119, True: 0.881' + assert rejection_sampling( + 'Cloudy', dict(Rain=F, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.049' + assert rejection_sampling( + 'Cloudy', dict(Rain=T), + sprinkler, 10000).show_approx() == 'False: 0.205, True: 0.795' + assert rejection_sampling( + 'Cloudy', dict(Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.835, True: 0.165' + + +def test_likelihood_weighting(): + random.seed(1017) + assert likelihood_weighting( + 'Burglary', dict(JohnCalls=T, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.702, True: 0.298' + assert likelihood_weighting( + 'Burglary', dict(JohnCalls=T, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 0.993, True: 0.00656' + assert likelihood_weighting( + 'Burglary', dict(JohnCalls=F, MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.996, True: 0.00363' + assert likelihood_weighting( + 'Burglary', dict(JohnCalls=F, MaryCalls=F), + burglary, 10000).show_approx() == 'False: 1, True: 0.000126' + assert likelihood_weighting( + 'Burglary', dict(JohnCalls=T), + burglary, 10000).show_approx() == 'False: 0.979, True: 0.0205' + assert likelihood_weighting( + 'Burglary', dict(MaryCalls=T), + burglary, 10000).show_approx() == 'False: 0.94, True: 0.0601' + + +def test_likelihood_weighting2(): + random.seed(42) + assert likelihood_weighting( + 'Cloudy', dict(Rain=T, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.559, True: 0.441' + assert likelihood_weighting( + 'Cloudy', dict(Rain=T, Sprinkler=F), + sprinkler, 10000).show_approx() == 'False: 0.12, True: 0.88' + assert likelihood_weighting( + 'Cloudy', dict(Rain=F, Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.951, True: 0.0486' + assert likelihood_weighting( + 'Cloudy', dict(Rain=T), + sprinkler, 10000).show_approx() == 'False: 0.198, True: 0.802' + assert likelihood_weighting( + 'Cloudy', dict(Sprinkler=T), + sprinkler, 10000).show_approx() == 'False: 0.833, True: 0.167' + + +def test_gibbs_ask(): + g_solution = gibbs_ask('Cloudy', dict(Rain=True), sprinkler, 1000) + assert abs(g_solution.prob[False] - 0.2) < 0.05 + assert abs(g_solution.prob[True] - 0.8) < 0.05 + + +# The following should probably go in .ipynb: + +""" +# We can build up a probability distribution like this (p. 469): +>>> P = ProbDist() +>>> P['sunny'] = 0.7 +>>> P['rain'] = 0.2 +>>> P['cloudy'] = 0.08 +>>> P['snow'] = 0.02 + +# and query it like this: (Never mind this ELLIPSIS option +# added to make the doctest portable.) +>>> P['rain'] #doctest:+ELLIPSIS +0.2... + +# A Joint Probability Distribution is dealt with like this [Figure 13.3]: +>>> P = JointProbDist(['Toothache', 'Cavity', 'Catch']) +>>> T, F = True, False +>>> P[T, T, T] = 0.108; P[T, T, F] = 0.012; P[F, T, T] = 0.072; P[F, T, F] = 0.008 +>>> P[T, F, T] = 0.016; P[T, F, F] = 0.064; P[F, F, T] = 0.144; P[F, F, F] = 0.576 + +>>> P[T, T, T] +0.108 + +# Ask for P(Cavity|Toothache=T) +>>> PC = enumerate_joint_ask('Cavity', {'Toothache': T}, P) +>>> PC.show_approx() +'False: 0.4, True: 0.6' + +>>> 0.6-epsilon < PC[T] < 0.6+epsilon +True + +>>> 0.4-epsilon < PC[F] < 0.4+epsilon +True +""" + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_reinforcement_learning.py b/tests/test_reinforcement_learning.py new file mode 100644 index 000000000..d80ad3baf --- /dev/null +++ b/tests/test_reinforcement_learning.py @@ -0,0 +1,71 @@ +import pytest + +from reinforcement_learning import * +from mdp import sequential_decision_environment + +random.seed("aima-python") + +north = (0, 1) +south = (0, -1) +west = (-1, 0) +east = (1, 0) + +policy = { + (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, + (0, 1): north, (2, 1): north, (3, 1): None, + (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, +} + + +def test_PassiveDUEAgent(): + agent = PassiveDUEAgent(policy, sequential_decision_environment) + for i in range(200): + run_single_trial(agent, sequential_decision_environment) + agent.estimate_U() + # Agent does not always produce same results. + # Check if results are good enough. + # print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 + assert agent.U[(1, 0)] > 0 # In reality around 0.2 + + +def test_PassiveADPAgent(): + agent = PassiveADPAgent(policy, sequential_decision_environment) + for i in range(100): + run_single_trial(agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + # print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 + assert agent.U[(1, 0)] > 0 # In reality around 0.2 + + +def test_PassiveTDAgent(): + agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60. / (59 + n)) + for i in range(200): + run_single_trial(agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.35 + assert agent.U[(1, 0)] > 0.15 # In reality around 0.25 + + +def test_QLearning(): + q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60. / (59 + n)) + + for i in range(200): + run_single_trial(q_agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + assert q_agent.Q[((0, 1), (0, 1))] >= -0.5 # In reality around 0.1 + assert q_agent.Q[((1, 0), (0, -1))] <= 0.5 # In reality around -0.1 + + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_reinforcement_learning4e.py b/tests/test_reinforcement_learning4e.py new file mode 100644 index 000000000..287ec397b --- /dev/null +++ b/tests/test_reinforcement_learning4e.py @@ -0,0 +1,69 @@ +import pytest + +from mdp4e import sequential_decision_environment +from reinforcement_learning4e import * + +random.seed("aima-python") + +north = (0, 1) +south = (0, -1) +west = (-1, 0) +east = (1, 0) + +policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, + (0, 1): north, (2, 1): north, (3, 1): None, + (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west} + + +def test_PassiveDUEAgent(): + agent = PassiveDUEAgent(policy, sequential_decision_environment) + for i in range(200): + run_single_trial(agent, sequential_decision_environment) + agent.estimate_U() + # Agent does not always produce same results. + # Check if results are good enough. + # print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 + assert agent.U[(1, 0)] > 0 # In reality around 0.2 + + +def test_PassiveADPAgent(): + agent = PassiveADPAgent(policy, sequential_decision_environment) + for i in range(100): + run_single_trial(agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + # print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 + assert agent.U[(1, 0)] > 0 # In reality around 0.2 + + +def test_PassiveTDAgent(): + agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60. / (59 + n)) + for i in range(200): + run_single_trial(agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 + assert agent.U[(0, 1)] > 0.15 # In reality around 0.35 + assert agent.U[(1, 0)] > 0.15 # In reality around 0.25 + + +def test_QLearning(): + q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60. / (59 + n)) + + for i in range(200): + run_single_trial(q_agent, sequential_decision_environment) + + # Agent does not always produce same results. + # Check if results are good enough. + assert q_agent.Q[((0, 1), (0, 1))] >= -0.5 # In reality around 0.1 + assert q_agent.Q[((1, 0), (0, -1))] <= 0.5 # In reality around -0.1 + + +if __name__ == '__main__': + pytest.main() diff --git a/tests/test_rl.py b/tests/test_rl.py deleted file mode 100644 index 95a0e2224..000000000 --- a/tests/test_rl.py +++ /dev/null @@ -1,66 +0,0 @@ -import pytest - -from rl import * -from mdp import sequential_decision_environment - - -north = (0, 1) -south = (0,-1) -west = (-1, 0) -east = (1, 0) - -policy = { - (0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, - (0, 1): north, (2, 1): north, (3, 1): None, - (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west, -} - -def test_PassiveDUEAgent(): - agent = PassiveDUEAgent(policy, sequential_decision_environment) - for i in range(200): - run_single_trial(agent,sequential_decision_environment) - agent.estimate_U() - # Agent does not always produce same results. - # Check if results are good enough. - #print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) - assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 - assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 - assert agent.U[(1, 0)] > 0 # In reality around 0.2 - -def test_PassiveADPAgent(): - agent = PassiveADPAgent(policy, sequential_decision_environment) - for i in range(100): - run_single_trial(agent,sequential_decision_environment) - - # Agent does not always produce same results. - # Check if results are good enough. - #print(agent.U[(0, 0)], agent.U[(0,1)], agent.U[(1,0)]) - assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 - assert agent.U[(0, 1)] > 0.15 # In reality around 0.4 - assert agent.U[(1, 0)] > 0 # In reality around 0.2 - - - -def test_PassiveTDAgent(): - agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n)) - for i in range(200): - run_single_trial(agent,sequential_decision_environment) - - # Agent does not always produce same results. - # Check if results are good enough. - assert agent.U[(0, 0)] > 0.15 # In reality around 0.3 - assert agent.U[(0, 1)] > 0.15 # In reality around 0.35 - assert agent.U[(1, 0)] > 0.15 # In reality around 0.25 - - -def test_QLearning(): - q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, - alpha=lambda n: 60./(59+n)) - - for i in range(200): - run_single_trial(q_agent,sequential_decision_environment) - - # Agent does not always produce same results. - # Check if results are good enough. - assert q_agent.Q[((0, 1), (0, 1))] >= -0.5 # In reality around 0.1 - assert q_agent.Q[((1, 0), (0, -1))] <= 0.5 # In reality around -0.1 diff --git a/tests/test_search.py b/tests/test_search.py index e53d23238..9be3e4a47 100644 --- a/tests/test_search.py +++ b/tests/test_search.py @@ -1,13 +1,14 @@ import pytest from search import * +random.seed("aima-python") romania_problem = GraphProblem('Arad', 'Bucharest', romania_map) vacuum_world = GraphProblemStochastic('State_1', ['State_7', 'State_8'], vacuum_world) LRTA_problem = OnlineSearchProblem('State_3', 'State_5', one_dim_state_space) eight_puzzle = EightPuzzle((1, 2, 3, 4, 5, 7, 8, 6, 0)) eight_puzzle2 = EightPuzzle((1, 0, 6, 8, 7, 5, 4, 2), (0, 1, 2, 3, 4, 5, 6, 7, 8)) -nqueens = NQueensProblem(8) +n_queens = NQueensProblem(8) def test_find_min_edge(): @@ -17,7 +18,7 @@ def test_find_min_edge(): def test_breadth_first_tree_search(): assert breadth_first_tree_search( romania_problem).solution() == ['Sibiu', 'Fagaras', 'Bucharest'] - assert breadth_first_graph_search(nqueens).solution() == [0, 4, 7, 5, 2, 6, 1, 3] + assert breadth_first_graph_search(n_queens).solution() == [0, 4, 7, 5, 2, 6, 1, 3] def test_breadth_first_graph_search(): @@ -43,11 +44,11 @@ def test_best_first_graph_search(): def test_uniform_cost_search(): assert uniform_cost_search( romania_problem).solution() == ['Sibiu', 'Rimnicu', 'Pitesti', 'Bucharest'] - assert uniform_cost_search(nqueens).solution() == [0, 4, 7, 5, 2, 6, 1, 3] + assert uniform_cost_search(n_queens).solution() == [0, 4, 7, 5, 2, 6, 1, 3] def test_depth_first_tree_search(): - assert depth_first_tree_search(nqueens).solution() == [7, 3, 0, 2, 5, 1, 6, 4] + assert depth_first_tree_search(n_queens).solution() == [7, 3, 0, 2, 5, 1, 6, 4] def test_depth_first_graph_search(): @@ -70,13 +71,16 @@ def test_depth_limited_search(): def test_bidirectional_search(): assert bidirectional_search(romania_problem) == 418 + assert bidirectional_search(eight_puzzle) == 12 + assert bidirectional_search(EightPuzzle((1, 2, 3, 4, 5, 6, 0, 7, 8))) == 2 def test_astar_search(): assert astar_search(romania_problem).solution() == ['Sibiu', 'Rimnicu', 'Pitesti', 'Bucharest'] - assert astar_search(eight_puzzle).solution() == ['LEFT', 'LEFT', 'UP', 'RIGHT', 'RIGHT', 'DOWN', 'LEFT', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'RIGHT'] + assert astar_search(eight_puzzle).solution() == ['LEFT', 'LEFT', 'UP', 'RIGHT', 'RIGHT', 'DOWN', 'LEFT', 'UP', + 'LEFT', 'DOWN', 'RIGHT', 'RIGHT'] assert astar_search(EightPuzzle((1, 2, 3, 4, 5, 6, 0, 7, 8))).solution() == ['RIGHT', 'RIGHT'] - assert astar_search(nqueens).solution() == [7, 1, 3, 0, 6, 4, 2, 5] + assert astar_search(n_queens).solution() == [7, 1, 3, 0, 6, 4, 2, 5] def test_find_blank_square(): @@ -111,42 +115,42 @@ def test_result(): def test_goal_test(): - assert eight_puzzle.goal_test((0, 1, 2, 3, 4, 5, 6, 7, 8)) == False - assert eight_puzzle.goal_test((6, 3, 5, 1, 8, 4, 2, 0, 7)) == False - assert eight_puzzle.goal_test((3, 4, 1, 7, 6, 0, 2, 8, 5)) == False - assert eight_puzzle.goal_test((1, 2, 3, 4, 5, 6, 7, 8, 0)) == True - assert eight_puzzle2.goal_test((4, 8, 1, 6, 0, 2, 3, 5, 7)) == False - assert eight_puzzle2.goal_test((3, 4, 1, 7, 6, 0, 2, 8, 5)) == False - assert eight_puzzle2.goal_test((1, 2, 3, 4, 5, 6, 7, 8, 0)) == False - assert eight_puzzle2.goal_test((0, 1, 2, 3, 4, 5, 6, 7, 8)) == True - assert nqueens.goal_test((7, 3, 0, 2, 5, 1, 6, 4)) == True - assert nqueens.goal_test((0, 4, 7, 5, 2, 6, 1, 3)) == True - assert nqueens.goal_test((7, 1, 3, 0, 6, 4, 2, 5)) == True - assert nqueens.goal_test((0, 1, 2, 3, 4, 5, 6, 7)) == False + assert not eight_puzzle.goal_test((0, 1, 2, 3, 4, 5, 6, 7, 8)) + assert not eight_puzzle.goal_test((6, 3, 5, 1, 8, 4, 2, 0, 7)) + assert not eight_puzzle.goal_test((3, 4, 1, 7, 6, 0, 2, 8, 5)) + assert eight_puzzle.goal_test((1, 2, 3, 4, 5, 6, 7, 8, 0)) + assert not eight_puzzle2.goal_test((4, 8, 1, 6, 0, 2, 3, 5, 7)) + assert not eight_puzzle2.goal_test((3, 4, 1, 7, 6, 0, 2, 8, 5)) + assert not eight_puzzle2.goal_test((1, 2, 3, 4, 5, 6, 7, 8, 0)) + assert eight_puzzle2.goal_test((0, 1, 2, 3, 4, 5, 6, 7, 8)) + assert n_queens.goal_test((7, 3, 0, 2, 5, 1, 6, 4)) + assert n_queens.goal_test((0, 4, 7, 5, 2, 6, 1, 3)) + assert n_queens.goal_test((7, 1, 3, 0, 6, 4, 2, 5)) + assert not n_queens.goal_test((0, 1, 2, 3, 4, 5, 6, 7)) def test_check_solvability(): - assert eight_puzzle.check_solvability((0, 1, 2, 3, 4, 5, 6, 7, 8)) == True - assert eight_puzzle.check_solvability((6, 3, 5, 1, 8, 4, 2, 0, 7)) == True - assert eight_puzzle.check_solvability((3, 4, 1, 7, 6, 0, 2, 8, 5)) == True - assert eight_puzzle.check_solvability((1, 8, 4, 7, 2, 6, 3, 0, 5)) == True - assert eight_puzzle.check_solvability((4, 8, 1, 6, 0, 2, 3, 5, 7)) == True - assert eight_puzzle.check_solvability((1, 0, 6, 8, 7, 5, 4, 2, 3)) == True - assert eight_puzzle.check_solvability((1, 2, 3, 4, 5, 6, 7, 8, 0)) == True - assert eight_puzzle.check_solvability((1, 2, 3, 4, 5, 6, 8, 7, 0)) == False - assert eight_puzzle.check_solvability((1, 0, 3, 2, 4, 5, 6, 7, 8)) == False - assert eight_puzzle.check_solvability((7, 0, 2, 8, 5, 3, 6, 4, 1)) == False + assert eight_puzzle.check_solvability((0, 1, 2, 3, 4, 5, 6, 7, 8)) + assert eight_puzzle.check_solvability((6, 3, 5, 1, 8, 4, 2, 0, 7)) + assert eight_puzzle.check_solvability((3, 4, 1, 7, 6, 0, 2, 8, 5)) + assert eight_puzzle.check_solvability((1, 8, 4, 7, 2, 6, 3, 0, 5)) + assert eight_puzzle.check_solvability((4, 8, 1, 6, 0, 2, 3, 5, 7)) + assert eight_puzzle.check_solvability((1, 0, 6, 8, 7, 5, 4, 2, 3)) + assert eight_puzzle.check_solvability((1, 2, 3, 4, 5, 6, 7, 8, 0)) + assert not eight_puzzle.check_solvability((1, 2, 3, 4, 5, 6, 8, 7, 0)) + assert not eight_puzzle.check_solvability((1, 0, 3, 2, 4, 5, 6, 7, 8)) + assert not eight_puzzle.check_solvability((7, 0, 2, 8, 5, 3, 6, 4, 1)) def test_conflict(): - assert not nqueens.conflict(7, 0, 1, 1) - assert not nqueens.conflict(0, 3, 6, 4) - assert not nqueens.conflict(2, 6, 5, 7) - assert not nqueens.conflict(2, 4, 1, 6) - assert nqueens.conflict(0, 0, 1, 1) - assert nqueens.conflict(4, 3, 4, 4) - assert nqueens.conflict(6, 5, 5, 6) - assert nqueens.conflict(0, 6, 1, 7) + assert not n_queens.conflict(7, 0, 1, 1) + assert not n_queens.conflict(0, 3, 6, 4) + assert not n_queens.conflict(2, 6, 5, 7) + assert not n_queens.conflict(2, 4, 1, 6) + assert n_queens.conflict(0, 0, 1, 1) + assert n_queens.conflict(4, 3, 4, 4) + assert n_queens.conflict(6, 5, 5, 6) + assert n_queens.conflict(0, 6, 1, 7) def test_recursive_best_first_search(): @@ -154,35 +158,33 @@ def test_recursive_best_first_search(): romania_problem).solution() == ['Sibiu', 'Rimnicu', 'Pitesti', 'Bucharest'] assert recursive_best_first_search( EightPuzzle((2, 4, 3, 1, 5, 6, 7, 8, 0))).solution() == [ - 'UP', 'LEFT', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'RIGHT', 'DOWN' - ] + 'UP', 'LEFT', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'RIGHT', 'DOWN'] def manhattan(node): state = node.state - index_goal = {0:[2,2], 1:[0,0], 2:[0,1], 3:[0,2], 4:[1,0], 5:[1,1], 6:[1,2], 7:[2,0], 8:[2,1]} + index_goal = {0: [2, 2], 1: [0, 0], 2: [0, 1], 3: [0, 2], 4: [1, 0], 5: [1, 1], 6: [1, 2], 7: [2, 0], 8: [2, 1]} index_state = {} - index = [[0,0], [0,1], [0,2], [1,0], [1,1], [1,2], [2,0], [2,1], [2,2]] - x, y = 0, 0 - + index = [[0, 0], [0, 1], [0, 2], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1], [2, 2]] + for i in range(len(state)): index_state[state[i]] = index[i] - + mhd = 0 - + for i in range(8): for j in range(2): mhd = abs(index_goal[i][j] - index_state[i][j]) + mhd - + return mhd assert recursive_best_first_search( EightPuzzle((2, 4, 3, 1, 5, 6, 7, 8, 0)), h=manhattan).solution() == [ - 'LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'UP', 'DOWN', 'RIGHT' - ] + 'LEFT', 'UP', 'UP', 'LEFT', 'DOWN', 'RIGHT', 'DOWN', 'UP', 'DOWN', 'RIGHT'] + def test_hill_climbing(): prob = PeakFindingProblem((0, 0), [[0, 5, 10, 20], - [-3, 7, 11, 5]]) + [-3, 7, 11, 5]]) assert hill_climbing(prob) == (0, 3) prob = PeakFindingProblem((0, 0), [[0, 5, 10, 8], [-3, 7, 9, 999], @@ -195,10 +197,9 @@ def test_hill_climbing(): def test_simulated_annealing(): - random.seed("aima-python") prob = PeakFindingProblem((0, 0), [[0, 5, 10, 20], [-3, 7, 11, 5]], directions4) - sols = {prob.value(simulated_annealing(prob)) for i in range(100)} + sols = {prob.value(simulated_annealing(prob)) for _ in range(100)} assert max(sols) == 20 prob = PeakFindingProblem((0, 0), [[0, 5, 10, 8], [-3, 7, 9, 999], @@ -223,10 +224,11 @@ def test_and_or_graph_search(): def run_plan(state, problem, plan): if problem.goal_test(state): return True - if len(plan) is not 2: + if len(plan) != 2: return False predicate = lambda x: run_plan(x, problem, plan[1][x]) return all(predicate(r) for r in problem.result(state, plan[0])) + plan = and_or_graph_search(vacuum_world) assert run_plan('State_1', vacuum_world, plan) @@ -256,12 +258,10 @@ def test_LRTAStarAgent(): def test_genetic_algorithm(): # Graph coloring - edges = { - 'A': [0, 1], - 'B': [0, 3], - 'C': [1, 2], - 'D': [2, 3] - } + edges = {'A': [0, 1], + 'B': [0, 3], + 'C': [1, 2], + 'D': [2, 3]} def fitness(c): return sum(c[n1] != c[n2] for (n1, n2) in edges.values()) @@ -282,7 +282,7 @@ def fitness(c): def fitness(q): non_attacking = 0 for row1 in range(len(q)): - for row2 in range(row1+1, len(q)): + for row2 in range(row1 + 1, len(q)): col1 = int(q[row1]) col2 = int(q[row2]) row_diff = row1 - row2 @@ -293,7 +293,6 @@ def fitness(q): return non_attacking - solution = genetic_algorithm(population, fitness, gene_pool=gene_pool, f_thres=25) assert fitness(solution) >= 25 @@ -325,12 +324,12 @@ def update_state(self, state, percept): def formulate_goal(self, state): goal = [state7, state8] - return goal + return goal def formulate_problem(self, state, goal): problem = state - return problem - + return problem + def search(self, problem): if problem == state1: seq = ["Suck", "Right", "Suck"] @@ -360,7 +359,6 @@ def search(self, problem): assert a(state6) == "Left" assert a(state1) == "Suck" assert a(state3) == "Right" - # TODO: for .ipynb: diff --git a/tests/test_text.py b/tests/test_text.py index 311243745..3aaa007f6 100644 --- a/tests/test_text.py +++ b/tests/test_text.py @@ -1,10 +1,12 @@ -import pytest -import os import random +import numpy as np +import pytest + from text import * -from utils import isclose, open_data +from utils import open_data +random.seed("aima-python") def test_text_models(): @@ -30,9 +32,9 @@ def test_text_models(): (13, ('as', 'well', 'as'))] # Test isclose - assert isclose(P1['the'], 0.0611, rel_tol=0.001) - assert isclose(P2['of', 'the'], 0.0108, rel_tol=0.01) - assert isclose(P3['so', 'as', 'to'], 0.000323, rel_tol=0.001) + assert np.isclose(P1['the'], 0.0611, rtol=0.001) + assert np.isclose(P2['of', 'the'], 0.0108, rtol=0.01) + assert np.isclose(P3['so', 'as', 'to'], 0.000323, rtol=0.001) # Test cond_prob.get assert P2.cond_prob.get(('went',)) is None @@ -171,7 +173,8 @@ def test_permutation_decoder(): assert pd.decode('aba') in ('ece', 'ete', 'tat', 'tit', 'txt') pd = PermutationDecoder(canonicalize(flatland)) - assert pd.decode('aba') in ('ded', 'did', 'ece', 'ele', 'eme', 'ere', 'eve', 'eye', 'iti', 'mom', 'ses', 'tat', 'tit') + assert pd.decode('aba') in ( + 'ded', 'did', 'ece', 'ele', 'eme', 'ere', 'eve', 'eye', 'iti', 'mom', 'ses', 'tat', 'tit') def test_rot13_encoding(): @@ -227,8 +230,7 @@ def verify_query(query, expected): Results(62.95, "aima-data/MAN/shred.txt"), Results(57.46, "aima-data/MAN/pico.txt"), Results(43.38, "aima-data/MAN/login.txt"), - Results(41.93, "aima-data/MAN/ln.txt"), - ]) + Results(41.93, "aima-data/MAN/ln.txt")]) q2 = uc.query("how do I delete a file") assert verify_query(q2, [ @@ -238,8 +240,7 @@ def verify_query(query, expected): Results(60.63, "aima-data/MAN/zip.txt"), Results(57.46, "aima-data/MAN/pico.txt"), Results(51.28, "aima-data/MAN/shred.txt"), - Results(26.72, "aima-data/MAN/tr.txt"), - ]) + Results(26.72, "aima-data/MAN/tr.txt")]) q3 = uc.query("email") assert verify_query(q3, [ @@ -247,8 +248,7 @@ def verify_query(query, expected): Results(12.01, "aima-data/MAN/info.txt"), Results(9.89, "aima-data/MAN/pico.txt"), Results(8.73, "aima-data/MAN/grep.txt"), - Results(8.07, "aima-data/MAN/zip.txt"), - ]) + Results(8.07, "aima-data/MAN/zip.txt")]) q4 = uc.query("word count for files") assert verify_query(q4, [ @@ -258,8 +258,7 @@ def verify_query(query, expected): Results(55.45, "aima-data/MAN/ps.txt"), Results(53.42, "aima-data/MAN/more.txt"), Results(42.00, "aima-data/MAN/dd.txt"), - Results(12.85, "aima-data/MAN/who.txt"), - ]) + Results(12.85, "aima-data/MAN/who.txt")]) q5 = uc.query("learn: date") assert verify_query(q5, []) @@ -267,8 +266,7 @@ def verify_query(query, expected): q6 = uc.query("2003") assert verify_query(q6, [ Results(14.58, "aima-data/MAN/pine.txt"), - Results(11.62, "aima-data/MAN/jar.txt"), - ]) + Results(11.62, "aima-data/MAN/jar.txt")]) def test_words(): @@ -281,7 +279,7 @@ def test_canonicalize(): def test_translate(): text = 'orange apple lemon ' - func = lambda x: ('s ' + x) if x ==' ' else x + func = lambda x: ('s ' + x) if x == ' ' else x assert translate(text, func) == 'oranges apples lemons ' @@ -291,6 +289,5 @@ def test_bigrams(): assert bigrams(['this', 'is', 'a', 'test']) == [['this', 'is'], ['is', 'a'], ['a', 'test']] - if __name__ == '__main__': pytest.main() diff --git a/tests/test_utils.py b/tests/test_utils.py index 059cfad8b..6c2a50808 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -2,42 +2,52 @@ from utils import * import random +random.seed("aima-python") + + def test_sequence(): assert sequence(1) == (1,) assert sequence("helloworld") == "helloworld" - assert sequence({"hello":4, "world":5}) == ({"hello":4, "world":5},) + assert sequence({"hello": 4, "world": 5}) == ({"hello": 4, "world": 5},) assert sequence([1, 2, 3]) == [1, 2, 3] assert sequence((4, 5, 6)) == (4, 5, 6) - assert sequence([(1, 2),(2, 3),(4, 5)]) == [(1, 2), (2, 3),(4, 5)] - assert sequence(([1, 2],[3, 4],[5, 6])) == ([1, 2], [3, 4],[5, 6]) + assert sequence([(1, 2), (2, 3), (4, 5)]) == [(1, 2), (2, 3), (4, 5)] + assert sequence(([1, 2], [3, 4], [5, 6])) == ([1, 2], [3, 4], [5, 6]) + -def test_removeall_list(): - assert removeall(4, []) == [] - assert removeall(4, [1, 2, 3, 4]) == [1, 2, 3] - assert removeall(4, [4, 1, 4, 2, 3, 4, 4]) == [1, 2, 3] +def test_remove_all_list(): + assert remove_all(4, []) == [] + assert remove_all(4, [1, 2, 3, 4]) == [1, 2, 3] + assert remove_all(4, [4, 1, 4, 2, 3, 4, 4]) == [1, 2, 3] + assert remove_all(1, [2, 3, 4, 5, 6]) == [2, 3, 4, 5, 6] -def test_removeall_string(): - assert removeall('s', '') == '' - assert removeall('s', 'This is a test. Was a test.') == 'Thi i a tet. Wa a tet.' +def test_remove_all_string(): + assert remove_all('s', '') == '' + assert remove_all('s', 'This is a test. Was a test.') == 'Thi i a tet. Wa a tet.' + assert remove_all('a', 'artificial intelligence: a modern approach') == 'rtificil intelligence: modern pproch' def test_unique(): assert unique([1, 2, 3, 2, 1]) == [1, 2, 3] assert unique([1, 5, 6, 7, 6, 5]) == [1, 5, 6, 7] + assert unique([1, 2, 3, 4, 5]) == [1, 2, 3, 4, 5] def test_count(): assert count([1, 2, 3, 4, 2, 3, 4]) == 7 assert count("aldpeofmhngvia") == 14 assert count([True, False, True, True, False]) == 3 - assert count([5 > 1, len("abc") == 3, 3+1 == 5]) == 2 + assert count([5 > 1, len("abc") == 3, 3 + 1 == 5]) == 2 + assert count("aima") == 4 + def test_multimap(): - assert multimap([(1, 2),(1, 3),(1, 4),(2, 3),(2, 4),(4, 5)]) == \ - {1: [2, 3, 4], 2: [3, 4], 4: [5]} + assert multimap([(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (4, 5)]) == \ + {1: [2, 3, 4], 2: [3, 4], 4: [5]} assert multimap([("a", 2), ("a", 3), ("a", 4), ("b", 3), ("b", 4), ("c", 5)]) == \ - {'a': [2, 3, 4], 'b': [3, 4], 'c': [5]} + {'a': [2, 3, 4], 'b': [3, 4], 'c': [5]} + def test_product(): assert product([1, 2, 3, 4]) == 24 @@ -49,13 +59,14 @@ def test_first(): assert first('') is None assert first('', 'empty') == 'empty' assert first([1, 2, 3, 4, 5]) == 1 - assert first([]) == None + assert first([]) is None assert first(range(10)) == 0 assert first(x for x in range(10) if x > 3) == 4 assert first(x for x in range(10) if x > 100) is None assert first((1, 2, 3)) == 1 - assert first([(1, 2),(1, 3),(1, 4)]) == (1, 2) - assert first({1:"one", 2:"two", 3:"three"}) == 1 + assert first(range(2, 10)) == 2 + assert first([(1, 2), (1, 3), (1, 4)]) == (1, 2) + assert first({1: "one", 2: "two", 3: "three"}) == 1 def test_is_in(): @@ -67,80 +78,82 @@ def test_is_in(): def test_mode(): assert mode([12, 32, 2, 1, 2, 3, 2, 3, 2, 3, 44, 3, 12, 4, 9, 0, 3, 45, 3]) == 3 assert mode("absndkwoajfkalwpdlsdlfllalsflfdslgflal") == 'l' + assert mode("artificialintelligence") == 'i' + +def test_power_set(): + assert power_set([1, 2, 3]) == [(1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)] -def test_powerset(): - assert powerset([1, 2, 3]) == [(1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)] +def test_histogram(): + assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1]) == [(1, 2), (2, 3), (4, 2), (5, 1), (7, 1), (9, 1)] + assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1], 0, lambda x: x * x) == \ + [(1, 2), (4, 3), (16, 2), (25, 1), (49, 1), (81, 1)] + assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1], 1) == [(2, 3), (4, 2), (1, 2), (9, 1), (7, 1), (5, 1)] -def test_argminmax(): - assert argmin([-2, 1], key=abs) == 1 - assert argmax([-2, 1], key=abs) == -2 - assert argmax(['one', 'to', 'three'], key=len) == 'three' +def test_euclidean(): + distance = euclidean_distance([1, 2], [3, 4]) + assert round(distance, 2) == 2.83 -def test_histogram(): - assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1]) == [(1, 2), (2, 3), - (4, 2), (5, 1), - (7, 1), (9, 1)] - assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1], 0, lambda x: x*x) == [(1, 2), (4, 3), - (16, 2), (25, 1), - (49, 1), (81, 1)] - assert histogram([1, 2, 4, 2, 4, 5, 7, 9, 2, 1], 1) == [(2, 3), (4, 2), - (1, 2), (9, 1), - (7, 1), (5, 1)] + distance = euclidean_distance([1, 2, 3], [4, 5, 6]) + assert round(distance, 2) == 5.2 + distance = euclidean_distance([0, 0, 0], [0, 0, 0]) + assert distance == 0 -def test_dotproduct(): - assert dotproduct([1, 2, 3], [1000, 100, 10]) == 1230 +def test_cross_entropy(): + loss = cross_entropy_loss([1, 0], [0.9, 0.3]) + assert round(loss, 2) == 0.23 -def test_element_wise_product(): - assert element_wise_product([1, 2, 5], [7, 10, 0]) == [7, 20, 0] - assert element_wise_product([1, 6, 3, 0], [9, 12, 0, 0]) == [9, 72, 0, 0] + loss = cross_entropy_loss([1, 0, 0, 1], [0.9, 0.3, 0.5, 0.75]) + assert round(loss, 2) == 0.36 + loss = cross_entropy_loss([1, 0, 0, 1, 1, 0, 1, 1], [0.9, 0.3, 0.5, 0.75, 0.85, 0.14, 0.93, 0.79]) + assert round(loss, 2) == 0.26 -def test_matrix_multiplication(): - assert matrix_multiplication([[1, 2, 3], - [2, 3, 4]], - [[3, 4], - [1, 2], - [1, 0]]) == [[8, 8], [13, 14]] - assert matrix_multiplication([[1, 2, 3], - [2, 3, 4]], - [[3, 4, 8, 1], - [1, 2, 5, 0], - [1, 0, 0, 3]], - [[1, 2], - [3, 4], - [5, 6], - [1, 2]]) == [[132, 176], [224, 296]] +def test_rms_error(): + assert rms_error([2, 2], [2, 2]) == 0 + assert rms_error((0, 0), (0, 1)) == np.sqrt(0.5) + assert rms_error((1, 0), (0, 1)) == 1 + assert rms_error((0, 0), (0, -1)) == np.sqrt(0.5) + assert rms_error((0, 0.5), (0, -0.5)) == np.sqrt(0.5) -def test_vector_to_diagonal(): - assert vector_to_diagonal([1, 2, 3]) == [[1, 0, 0], [0, 2, 0], [0, 0, 3]] - assert vector_to_diagonal([0, 3, 6]) == [[0, 0, 0], [0, 3, 0], [0, 0, 6]] +def test_manhattan_distance(): + assert manhattan_distance([2, 2], [2, 2]) == 0 + assert manhattan_distance([0, 0], [0, 1]) == 1 + assert manhattan_distance([1, 0], [0, 1]) == 2 + assert manhattan_distance([0, 0], [0, -1]) == 1 + assert manhattan_distance([0, 0.5], [0, -0.5]) == 1 -def test_vector_add(): - assert vector_add((0, 1), (8, 9)) == (8, 10) +def test_mean_boolean_error(): + assert mean_boolean_error([1, 1], [0, 0]) == 1 + assert mean_boolean_error([0, 1], [1, 0]) == 1 + assert mean_boolean_error([1, 1], [0, 1]) == 0.5 + assert mean_boolean_error([0, 0], [0, 0]) == 0 + assert mean_boolean_error([1, 1], [1, 1]) == 0 -def test_scalar_vector_product(): - assert scalar_vector_product(2, [1, 2, 3]) == [2, 4, 6] +def test_mean_error(): + assert mean_error([2, 2], [2, 2]) == 0 + assert mean_error([0, 0], [0, 1]) == 0.5 + assert mean_error([1, 0], [0, 1]) == 1 + assert mean_error([0, 0], [0, -1]) == 0.5 + assert mean_error([0, 0.5], [0, -0.5]) == 0.5 -def test_scalar_matrix_product(): - assert rounder(scalar_matrix_product(-5, [[1, 2], [3, 4], [0, 6]])) == [[-5, -10], [-15, -20], - [0, -30]] - assert rounder(scalar_matrix_product(0.2, [[1, 2], [2, 3]])) == [[0.2, 0.4], [0.4, 0.6]] +def test_dot_product(): + assert dot_product([1, 2, 3], [1000, 100, 10]) == 1230 + assert dot_product([1, 2, 3], [0, 0, 0]) == 0 -def test_inverse_matrix(): - assert rounder(inverse_matrix([[1, 0], [0, 1]])) == [[1, 0], [0, 1]] - assert rounder(inverse_matrix([[2, 1], [4, 3]])) == [[1.5, -0.5], [-2.0, 1.0]] - assert rounder(inverse_matrix([[4, 7], [2, 6]])) == [[0.6, -0.7], [-0.2, 0.4]] +def test_vector_add(): + assert vector_add((0, 1), (8, 9)) == (8, 10) + assert vector_add((1, 1, 1), (2, 2, 2)) == (3, 3, 3) def test_rounder(): @@ -148,8 +161,7 @@ def test_rounder(): assert rounder(10.234566) == 10.2346 assert rounder([1.234566, 0.555555, 6.010101]) == [1.2346, 0.5556, 6.0101] assert rounder([[1.234566, 0.555555, 6.010101], - [10.505050, 12.121212, 6.030303]]) == [[1.2346, 0.5556, 6.0101], - [10.5051, 12.1212, 6.0303]] + [10.505050, 12.121212, 6.030303]]) == [[1.2346, 0.5556, 6.0101], [10.5051, 12.1212, 6.0303]] def test_num_or_str(): @@ -161,34 +173,10 @@ def test_normalize(): assert normalize([1, 2, 1]) == [0.25, 0.5, 0.25] -def test_norm(): - assert isclose(norm([1, 2, 1], 1), 4) - assert isclose(norm([3, 4], 2), 5) - assert isclose(norm([-1, 1, 2], 4), 18**0.25) - - -def test_clip(): - assert [clip(x, 0, 1) for x in [-1, 0.5, 10]] == [0, 0.5, 1] - - -def test_sigmoid(): - assert isclose(0.5, sigmoid(0)) - assert isclose(0.7310585786300049, sigmoid(1)) - assert isclose(0.2689414213699951, sigmoid(-1)) - - def test_gaussian(): - assert gaussian(1,0.5,0.7) == 0.6664492057835993 - assert gaussian(5,2,4.5) == 0.19333405840142462 - assert gaussian(3,1,3) == 0.3989422804014327 - - -def test_sigmoid_derivative(): - value = 1 - assert sigmoid_derivative(value) == 0 - - value = 3 - assert sigmoid_derivative(value) == -6 + assert gaussian(1, 0.5, 0.7) == 0.6664492057835993 + assert gaussian(5, 2, 4.5) == 0.19333405840142462 + assert gaussian(3, 1, 3) == 0.3989422804014327 def test_weighted_choice(): @@ -209,27 +197,23 @@ def test_distance_squared(): assert distance_squared((1, 2), (5, 5)) == 25.0 -def test_vector_clip(): - assert vector_clip((-1, 10), (0, 0), (9, 9)) == (0, 9) - - def test_turn_heading(): - assert turn_heading((0, 1), 1) == (-1, 0) - assert turn_heading((0, 1), -1) == (1, 0) - assert turn_heading((1, 0), 1) == (0, 1) - assert turn_heading((1, 0), -1) == (0, -1) - assert turn_heading((0, -1), 1) == (1, 0) - assert turn_heading((0, -1), -1) == (-1, 0) - assert turn_heading((-1, 0), 1) == (0, -1) - assert turn_heading((-1, 0), -1) == (0, 1) + assert turn_heading((0, 1), 1) == (-1, 0) + assert turn_heading((0, 1), -1) == (1, 0) + assert turn_heading((1, 0), 1) == (0, 1) + assert turn_heading((1, 0), -1) == (0, -1) + assert turn_heading((0, -1), 1) == (1, 0) + assert turn_heading((0, -1), -1) == (-1, 0) + assert turn_heading((-1, 0), 1) == (0, -1) + assert turn_heading((-1, 0), -1) == (0, 1) def test_turn_left(): - assert turn_left((0, 1)) == (-1, 0) + assert turn_left((0, 1)) == (-1, 0) def test_turn_right(): - assert turn_right((0, 1)) == (1, 0) + assert turn_right((0, 1)) == (1, 0) def test_step(): @@ -270,8 +254,49 @@ def test_expr(): assert expr('P & Q <=> Q & P') == Expr('<=>', (P & Q), (Q & P)) assert expr('P(x) | P(y) & Q(z)') == (P(x) | (P(y) & Q(z))) # x is grandparent of z if x is parent of y and y is parent of z: - assert (expr('GP(x, z) <== P(x, y) & P(y, z)') - == Expr('<==', GP(x, z), P(x, y) & P(y, z))) + assert (expr('GP(x, z) <== P(x, y) & P(y, z)') == Expr('<==', GP(x, z), P(x, y) & P(y, z))) + + +def test_min_priority_queue(): + queue = PriorityQueue(f=lambda x: x[1]) + queue.append((1, 100)) + queue.append((2, 30)) + queue.append((3, 50)) + assert queue.pop() == (2, 30) + assert len(queue) == 2 + assert queue[(3, 50)] == 50 + assert (1, 100) in queue + del queue[(1, 100)] + assert (1, 100) not in queue + queue.extend([(1, 100), (4, 10)]) + assert queue.pop() == (4, 10) + assert len(queue) == 2 + + +def test_max_priority_queue(): + queue = PriorityQueue(order='max', f=lambda x: x[1]) + queue.append((1, 100)) + queue.append((2, 30)) + queue.append((3, 50)) + assert queue.pop() == (1, 100) + + +def test_priority_queue_with_objects(): + class Test: + def __init__(self, a, b): + self.a = a + self.b = b + + def __eq__(self, other): + return self.a == other.a + + queue = PriorityQueue(f=lambda x: x.b) + queue.append(Test(1, 100)) + other = Test(1, 10) + assert queue[other] == 100 + assert other in queue + del queue[other] + assert len(queue) == 0 if __name__ == '__main__': diff --git a/text.py b/text.py index b6beb28ca..11a5731f1 100644 --- a/text.py +++ b/text.py @@ -1,22 +1,25 @@ -"""Statistical Language Processing tools. (Chapter 22) +""" +Statistical Language Processing tools (Chapter 22) + We define Unigram and Ngram text models, use them to generate random text, -and show the Viterbi algorithm for segmentatioon of letters into words. +and show the Viterbi algorithm for segmentation of letters into words. Then we show a very simple Information Retrieval system, and an example -working on a tiny sample of Unix manual pages.""" - -from utils import argmin, argmax, hashabledict -from learning import CountingProbDist -import search +working on a tiny sample of Unix manual pages. +""" -from math import log, exp -from collections import defaultdict import heapq -import re import os +import re +from collections import defaultdict +import numpy as np -class UnigramWordModel(CountingProbDist): +import search +from probabilistic_learning import CountingProbDist +from utils import hashabledict + +class UnigramWordModel(CountingProbDist): """This is a discrete probability distribution over words, so you can add, sample, or get P[word], just like with CountingProbDist. You can also generate a random text, n words long, with P.samples(n).""" @@ -32,7 +35,6 @@ def samples(self, n): class NgramWordModel(CountingProbDist): - """This is a discrete probability distribution over n-tuples of words. You can add, sample or get P[(word1, ..., wordn)]. The method P.samples(n) builds up an n-word sequence; P.add_cond_prob and P.add_sequence add data.""" @@ -73,7 +75,7 @@ def samples(self, nwords): output = list(self.sample()) for i in range(n, nwords): - last = output[-n+1:] + last = output[-n + 1:] next_word = self.cond_prob[tuple(last)].sample() output.append(next_word) @@ -99,6 +101,7 @@ def add_sequence(self, words): for char in word: self.add(char) + # ______________________________________________________________________________ @@ -111,7 +114,7 @@ def viterbi_segment(text, P): words = [''] + list(text) best = [1.0] + [0.0] * n # Fill in the vectors best words via dynamic programming - for i in range(n+1): + for i in range(n + 1): for j in range(0, i): w = text[j:i] curr_score = P[w] * best[i - len(w)] @@ -133,7 +136,6 @@ def viterbi_segment(text, P): # TODO(tmrts): Expose raw index class IRSystem: - """A very simple Information Retrieval System, as discussed in Sect. 23.2. The constructor s = IRSystem('the a') builds an empty system with two stopwords. Next, index several documents with s.index_document(text, url). @@ -154,8 +156,7 @@ def index_collection(self, filenames): """Index a whole collection of files.""" prefix = os.path.dirname(__file__) for filename in filenames: - self.index_document(open(filename).read(), - os.path.relpath(filename, prefix)) + self.index_document(open(filename).read(), os.path.relpath(filename, prefix)) def index_document(self, text, url): """Index the text of a document.""" @@ -177,15 +178,14 @@ def query(self, query_text, n=10): return [] qwords = [w for w in words(query_text) if w not in self.stopwords] - shortest = argmin(qwords, key=lambda w: len(self.index[w])) + shortest = min(qwords, key=lambda w: len(self.index[w])) docids = self.index[shortest] return heapq.nlargest(n, ((self.total_score(qwords, docid), docid) for docid in docids)) def score(self, word, docid): """Compute a score for this word on the document with this docid.""" # There are many options; here we take a very simple approach - return (log(1 + self.index[word][docid]) / - log(1 + self.documents[docid].nwords)) + return np.log(1 + self.index[word][docid]) / np.log(1 + self.documents[docid].nwords) def total_score(self, words, docid): """Compute the sum of the scores of these words on the document with this docid.""" @@ -195,9 +195,7 @@ def present(self, results): """Present the results as a list.""" for (score, docid) in results: doc = self.documents[docid] - print( - ("{:5.2}|{:25} | {}".format(100 * score, doc.url, - doc.title[:45].expandtabs()))) + print("{:5.2}|{:25} | {}".format(100 * score, doc.url, doc.title[:45].expandtabs())) def present_results(self, query_text, n=10): """Get results for the query and present them.""" @@ -205,7 +203,6 @@ def present_results(self, query_text, n=10): class UnixConsultant(IRSystem): - """A trivial IR system over a small collection of Unix man pages.""" def __init__(self): @@ -214,14 +211,12 @@ def __init__(self): import os aima_root = os.path.dirname(__file__) mandir = os.path.join(aima_root, 'aima-data/MAN/') - man_files = [mandir + f for f in os.listdir(mandir) - if f.endswith('.txt')] + man_files = [mandir + f for f in os.listdir(mandir) if f.endswith('.txt')] self.index_collection(man_files) class Document: - """Metadata for a document: title and url; maybe add others later.""" def __init__(self, title, url, nwords): @@ -256,6 +251,7 @@ def canonicalize(text): alphabet = 'abcdefghijklmnopqrstuvwxyz' + # Encoding @@ -310,11 +306,11 @@ def bigrams(text): """ return [text[i:i + 2] for i in range(len(text) - 1)] + # Decoding a Shift (or Caesar) Cipher class ShiftDecoder: - """There are only 26 possible encodings, so we can try all of them, and return the one with the highest probability, according to a bigram probability distribution.""" @@ -335,7 +331,7 @@ def score(self, plaintext): def decode(self, ciphertext): """Return the shift decoding of text with the best score.""" - return argmax(all_shifts(ciphertext), key=lambda shift: self.score(shift)) + return max(all_shifts(ciphertext), key=lambda shift: self.score(shift)) def all_shifts(text): @@ -343,11 +339,11 @@ def all_shifts(text): yield from (shift_encode(text, i) for i, _ in enumerate(alphabet)) + # Decoding a General Permutation Cipher class PermutationDecoder: - """This is a much harder problem than the shift decoder. There are 26! permutations, so we can't try them all. Instead we have to search. We want to search well, but there are many things to consider: @@ -390,25 +386,25 @@ def score(self, code): # add small positive value to prevent computing log(0) # TODO: Modify the values to make score more accurate - logP = (sum(log(self.Pwords[word] + 1e-20) for word in words(text)) + - sum(log(self.P1[c] + 1e-5) for c in text) + - sum(log(self.P2[b] + 1e-10) for b in bigrams(text))) - return -exp(logP) + logP = (sum(np.log(self.Pwords[word] + 1e-20) for word in words(text)) + + sum(np.log(self.P1[c] + 1e-5) for c in text) + + sum(np.log(self.P2[b] + 1e-10) for b in bigrams(text))) + return -np.exp(logP) class PermutationDecoderProblem(search.Problem): def __init__(self, initial=None, goal=None, decoder=None): - self.initial = initial or hashabledict() + super().__init__(initial or hashabledict(), goal) self.decoder = decoder def actions(self, state): search_list = [c for c in self.decoder.chardomain if c not in state] target_list = [c for c in alphabet if c not in state.values()] - # Find the best charater to replace - plainchar = argmax(search_list, key=lambda c: self.decoder.P1[c]) - for cipherchar in target_list: - yield (plainchar, cipherchar) + # Find the best character to replace + plain_char = max(search_list, key=lambda c: self.decoder.P1[c]) + for cipher_char in target_list: + yield (plain_char, cipher_char) def result(self, state, action): new_state = hashabledict(state) # copy to prevent hash issues diff --git a/utils.py b/utils.py index 28e531c19..3158e3793 100644 --- a/utils.py +++ b/utils.py @@ -3,14 +3,15 @@ import bisect import collections import collections.abc +import functools import heapq import operator import os.path import random -import math -import functools -import numpy as np from itertools import chain, combinations +from statistics import mean + +import numpy as np # ______________________________________________________________________________ @@ -19,14 +20,17 @@ def sequence(iterable): """Converts iterable to sequence, if it is not already one.""" - return (iterable if isinstance(iterable, collections.abc.Sequence) - else tuple([iterable])) + return iterable if isinstance(iterable, collections.abc.Sequence) else tuple([iterable]) -def removeall(item, seq): +def remove_all(item, seq): """Return a copy of seq (or string) with all occurrences of item removed.""" if isinstance(seq, str): return seq.replace(item, '') + elif isinstance(seq, set): + rest = seq.copy() + rest.remove(item) + return rest else: return [x for x in seq if x != item] @@ -40,6 +44,7 @@ def count(seq): """Count the number of items in sequence that are interpreted as true.""" return sum(map(bool, seq)) + def multimap(items): """Given (key, val) pairs, return {key: [val, ....], ...}.""" result = collections.defaultdict(list) @@ -47,12 +52,14 @@ def multimap(items): result[key].append(val) return dict(result) + def multimap_items(mmap): """Yield all (key, val) pairs stored in the multimap.""" for (key, vals) in mmap.items(): for val in vals: yield key, val + def product(numbers): """Return the product of the numbers, e.g. product([2, 3, 10]) == 60""" result = 1 @@ -65,6 +72,7 @@ def first(iterable, default=None): """Return the first element of an iterable; or default.""" return next(iter(iterable), default) + def is_in(elt, seq): """Similar to (elt in seq), but compares with 'is', not '=='.""" return any(x is elt for x in seq) @@ -76,29 +84,35 @@ def mode(data): return item -def powerset(iterable): - """powerset([1,2,3]) --> (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)""" +def power_set(iterable): + """power_set([1,2,3]) --> (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)""" s = list(iterable) return list(chain.from_iterable(combinations(s, r) for r in range(len(s) + 1)))[1:] +def extend(s, var, val): + """Copy dict s and extend it by setting var to val; return copy.""" + return {**s, var: val} + + +def flatten(seqs): + return sum(seqs, []) + + # ______________________________________________________________________________ # argmin and argmax identity = lambda x: x -argmin = min -argmax = max - def argmin_random_tie(seq, key=identity): """Return a minimum element of seq; break ties at random.""" - return argmin(shuffled(seq), key=key) + return min(shuffled(seq), key=key) def argmax_random_tie(seq, key=identity): """Return an element with highest fn(seq[i]) score; break ties at random.""" - return argmax(shuffled(seq), key=key) + return max(shuffled(seq), key=key) def shuffled(iterable): @@ -124,85 +138,40 @@ def histogram(values, mode=0, bin_function=None): bins[val] = bins.get(val, 0) + 1 if mode: - return sorted(list(bins.items()), key=lambda x: (x[1], x[0]), - reverse=True) + return sorted(list(bins.items()), key=lambda x: (x[1], x[0]), reverse=True) else: return sorted(bins.items()) -def dotproduct(X, Y): - """Return the sum of the element-wise product of vectors X and Y.""" - return sum(x * y for x, y in zip(X, Y)) +def dot_product(x, y): + """Return the sum of the element-wise product of vectors x and y.""" + return sum(_x * _y for _x, _y in zip(x, y)) -def element_wise_product(X, Y): - """Return vector as an element-wise product of vectors X and Y""" - assert len(X) == len(Y) - return [x * y for x, y in zip(X, Y)] +def element_wise_product(x, y): + """Return vector as an element-wise product of vectors x and y.""" + assert len(x) == len(y) + return np.multiply(x, y) -def matrix_multiplication(X_M, *Y_M): - """Return a matrix as a matrix-multiplication of X_M and arbitrary number of matrices *Y_M""" +def matrix_multiplication(x, *y): + """Return a matrix as a matrix-multiplication of x and arbitrary number of matrices *y.""" - def _mat_mult(X_M, Y_M): - """Return a matrix as a matrix-multiplication of two matrices X_M and Y_M - >>> matrix_multiplication([[1, 2, 3], - [2, 3, 4]], - [[3, 4], - [1, 2], - [1, 0]]) - [[8, 8],[13, 14]] - """ - assert len(X_M[0]) == len(Y_M) - - result = [[0 for i in range(len(Y_M[0]))] for j in range(len(X_M))] - for i in range(len(X_M)): - for j in range(len(Y_M[0])): - for k in range(len(Y_M)): - result[i][j] += X_M[i][k] * Y_M[k][j] - return result - - result = X_M - for Y in Y_M: - result = _mat_mult(result, Y) + result = x + for _y in y: + result = np.matmul(result, _y) return result -def vector_to_diagonal(v): - """Converts a vector to a diagonal matrix with vector elements - as the diagonal elements of the matrix""" - diag_matrix = [[0 for i in range(len(v))] for j in range(len(v))] - for i in range(len(v)): - diag_matrix[i][i] = v[i] - - return diag_matrix - - def vector_add(a, b): """Component-wise addition of two vectors.""" return tuple(map(operator.add, a, b)) -def scalar_vector_product(X, Y): +def scalar_vector_product(x, y): """Return vector as a product of a scalar and a vector""" - return [X * y for y in Y] - - -def scalar_matrix_product(X, Y): - """Return matrix as a product of a scalar and a matrix""" - return [scalar_vector_product(X, y) for y in Y] - - -def inverse_matrix(X): - """Inverse a given square matrix of size 2x2""" - assert len(X) == 2 - assert len(X[0]) == 2 - det = X[0][0] * X[1][1] - X[0][1] * X[1][0] - assert det != 0 - inv_mat = scalar_matrix_product(1.0 / det, [[X[1][1], -X[0][1]], [-X[1][0], X[0][0]]]) - - return inv_mat + return np.multiply(x, y) def probability(p): @@ -215,7 +184,6 @@ def weighted_sample_with_replacement(n, seq, weights): probability of each element in proportion to its corresponding weight.""" sample = weighted_sampler(seq, weights) - return [sample() for _ in range(n)] @@ -224,13 +192,12 @@ def weighted_sampler(seq, weights): totals = [] for w in weights: totals.append(w + totals[-1] if totals else w) - return lambda: seq[bisect.bisect(totals, random.uniform(0, totals[-1]))] def weighted_choice(choices): """A weighted version of random.choice""" - # NOTE: Shoule be replaced by random.choices if we port to Python 3.6 + # NOTE: should be replaced by random.choices if we port to Python 3.6 total = sum(w for _, w in choices) r = random.uniform(0, total) @@ -239,7 +206,8 @@ def weighted_choice(choices): if upto + w >= r: return c, w upto += w - + + def rounder(numbers, d=4): """Round a single number, or sequence of numbers, to d decimal places.""" if isinstance(numbers, (int, float)): @@ -249,9 +217,8 @@ def rounder(numbers, d=4): return constructor(rounder(n, d) for n in numbers) -def num_or_str(x): # TODO: rename as `atom` - """The argument is a string; convert to a number if - possible, or strip it.""" +def num_or_str(x): # TODO: rename as `atom` + """The argument is a string; convert to a number if possible, or strip it.""" try: return int(x) except ValueError: @@ -261,83 +228,99 @@ def num_or_str(x): # TODO: rename as `atom` return str(x).strip() +def euclidean_distance(x, y): + return np.sqrt(sum((_x - _y) ** 2 for _x, _y in zip(x, y))) + + +def manhattan_distance(x, y): + return sum(abs(_x - _y) for _x, _y in zip(x, y)) + + +def hamming_distance(x, y): + return sum(_x != _y for _x, _y in zip(x, y)) + + +def cross_entropy_loss(x, y): + return (-1.0 / len(x)) * sum(_x * np.log(_y) + (1 - _x) * np.log(1 - _y) for _x, _y in zip(x, y)) + + +def mean_squared_error_loss(x, y): + return (1.0 / len(x)) * sum((_x - _y) ** 2 for _x, _y in zip(x, y)) + + +def rms_error(x, y): + return np.sqrt(ms_error(x, y)) + + +def ms_error(x, y): + return mean((_x - _y) ** 2 for _x, _y in zip(x, y)) + + +def mean_error(x, y): + return mean(abs(_x - _y) for _x, _y in zip(x, y)) + + +def mean_boolean_error(x, y): + return mean(_x != _y for _x, _y in zip(x, y)) + + def normalize(dist): """Multiply each number by a constant such that the sum is 1.0""" if isinstance(dist, dict): total = sum(dist.values()) for key in dist: dist[key] = dist[key] / total - assert 0 <= dist[key] <= 1, "Probabilities must be between 0 and 1." + assert 0 <= dist[key] <= 1 # probabilities must be between 0 and 1 return dist total = sum(dist) return [(n / total) for n in dist] -def norm(X, n=2): - """Return the n-norm of vector X""" - return sum([x ** n for x in X]) ** (1 / n) +def random_weights(min_value, max_value, num_weights): + return [random.uniform(min_value, max_value) for _ in range(num_weights)] -def clip(x, lowest, highest): - """Return x clipped to the range [lowest..highest].""" - return max(lowest, min(x, highest)) +def sigmoid(x): + """Return activation value of x with sigmoid function.""" + return 1 / (1 + np.exp(-x)) def sigmoid_derivative(value): return value * (1 - value) -def sigmoid(x): - """Return activation value of x with sigmoid function""" - return 1 / (1 + math.exp(-x)) +def elu(x, alpha=0.01): + return x if x > 0 else alpha * (np.exp(x) - 1) +def elu_derivative(value, alpha=0.01): + return 1 if value > 0 else alpha * np.exp(value) -def relu_derivative(value): - if value > 0: - return 1 - else: - return 0 - -def elu(x, alpha=0.01): - if x > 0: - return x - else: - return alpha * (math.exp(x) - 1) - -def elu_derivative(value, alpha = 0.01): - if value > 0: - return 1 - else: - return alpha * math.exp(value) def tanh(x): - return np.tanh(x) + return np.tanh(x) + def tanh_derivative(value): - return (1 - (value ** 2)) + return 1 - (value ** 2) + + +def leaky_relu(x, alpha=0.01): + return x if x > 0 else alpha * x -def leaky_relu(x, alpha = 0.01): - if x > 0: - return x - else: - return alpha * x def leaky_relu_derivative(value, alpha=0.01): - if value > 0: - return 1 - else: - return alpha + return 1 if value > 0 else alpha + def relu(x): - return max(0, x) - + return max(0, x) + + def relu_derivative(value): - if value > 0: - return 1 - else: - return 0 - + return 1 if value > 0 else 0 + + def step(x): """Return activation value of x with sign function""" return 1 if x >= 0 else 0 @@ -345,15 +328,30 @@ def step(x): def gaussian(mean, st_dev, x): """Given the mean and standard deviation of a distribution, it returns the probability of x.""" - return 1 / (math.sqrt(2 * math.pi) * st_dev) * math.e ** (-0.5 * (float(x - mean) / st_dev) ** 2) + return 1 / (np.sqrt(2 * np.pi) * st_dev) * np.e ** (-0.5 * (float(x - mean) / st_dev) ** 2) + + +def linear_kernel(x, y=None): + if y is None: + y = x + return np.dot(x, y.T) + + +def polynomial_kernel(x, y=None, degree=2.0): + if y is None: + y = x + return (1.0 + np.dot(x, y.T)) ** degree -try: # math.isclose was added in Python 3.5; but we might be in 3.4 - from math import isclose -except ImportError: - def isclose(a, b, rel_tol=1e-09, abs_tol=0.0): - """Return true if numbers a and b are close to each other.""" - return abs(a - b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol) +def rbf_kernel(x, y=None, gamma=None): + """Radial-basis function kernel (aka squared-exponential kernel).""" + if y is None: + y = x + if gamma is None: + gamma = 1.0 / x.shape[1] # 1.0 / n_features + return np.exp(-gamma * (-2.0 * np.dot(x, y.T) + + np.sum(x * x, axis=1).reshape((-1, 1)) + np.sum(y * y, axis=1).reshape((1, -1)))) + # ______________________________________________________________________________ # Grid Functions @@ -379,7 +377,7 @@ def distance(a, b): """The distance between two (x, y) points.""" xA, yA = a xB, yB = b - return math.hypot((xA - xB), (yA - yB)) + return np.hypot((xA - xB), (yA - yB)) def distance_squared(a, b): @@ -389,17 +387,10 @@ def distance_squared(a, b): return (xA - xB) ** 2 + (yA - yB) ** 2 -def vector_clip(vector, lowest, highest): - """Return vector, except if any element is less than the corresponding - value of lowest or more than the corresponding value of highest, clip to - those values.""" - return type(vector)(map(clip, vector, lowest, highest)) - - # ______________________________________________________________________________ # Misc Functions -class injection(): +class injection: """Dependency injection of temporary values for global functions/classes/etc. E.g., `with injection(DataBase=MockDataBase): ...`""" @@ -465,13 +456,10 @@ def print_table(table, header=None, sep=' ', numfmt='{}'): table = [[numfmt.format(x) if isnumber(x) else x for x in row] for row in table] - sizes = list( - map(lambda seq: max(map(len, seq)), - list(zip(*[map(str, row) for row in table])))) + sizes = list(map(lambda seq: max(map(len, seq)), list(zip(*[map(str, row) for row in table])))) for row in table: - print(sep.join(getattr( - str(x), j)(size) for (j, size, x) in zip(justs, sizes, row))) + print(sep.join(getattr(str(x), j)(size) for (j, size, x) in zip(justs, sizes, row))) def open_data(name, mode='r'): @@ -487,7 +475,6 @@ def failure_test(algorithm, tests): to check for correctness. On the other hand, a lot of algorithms output something particular on fail (for example, False, or None). tests is a list with each element in the form: (values, failure_output).""" - from statistics import mean return mean(int(algorithm(x) != y) for x, y in tests) @@ -497,7 +484,7 @@ def failure_test(algorithm, tests): # See https://docs.python.org/3/reference/expressions.html#operator-precedence # See https://docs.python.org/3/reference/datamodel.html#special-method-names -class Expr(object): +class Expr: """A mathematical expression with an operator and 0 or more arguments. op is a str like '+' or 'sin'; args are Expressions. Expr('x') or Symbol('x') creates a symbol (a nullary Expr). @@ -604,18 +591,19 @@ def __rmatmul__(self, lhs): return Expr('@', lhs, self) def __call__(self, *args): - "Call: if 'f' is a Symbol, then f(0) == Expr('f', 0)." + """Call: if 'f' is a Symbol, then f(0) == Expr('f', 0).""" if self.args: - raise ValueError('can only do a call for a Symbol, not an Expr') + raise ValueError('Can only do a call for a Symbol, not an Expr') else: return Expr(self.op, *args) # Equality and repr def __eq__(self, other): - "'x == y' evaluates to True or False; does not build an Expr." - return (isinstance(other, Expr) - and self.op == other.op - and self.args == other.args) + """x == y' evaluates to True or False; does not build an Expr.""" + return isinstance(other, Expr) and self.op == other.op and self.args == other.args + + def __lt__(self, other): + return isinstance(other, Expr) and str(self) < str(other) def __hash__(self): return hash(self.op) ^ hash(self.args) @@ -690,10 +678,7 @@ def expr(x): >>> expr('P & Q ==> Q') ((P & Q) ==> Q) """ - if isinstance(x, str): - return eval(expr_handle_infix_ops(x), defaultkeydict(Symbol)) - else: - return x + return eval(expr_handle_infix_ops(x), defaultkeydict(Symbol)) if isinstance(x, str) else x infix_ops = '==> <== <=>'.split() @@ -721,9 +706,8 @@ def __missing__(self, key): class hashabledict(dict): - """Allows hashing by representing a dictionary as tuple of key:value pairs - May cause problems as the hash value may change during runtime - """ + """Allows hashing by representing a dictionary as tuple of key:value pairs. + May cause problems as the hash value may change during runtime.""" def __hash__(self): return 1 @@ -744,13 +728,12 @@ class PriorityQueue: def __init__(self, order='min', f=lambda x: x): self.heap = [] - if order == 'min': self.f = f elif order == 'max': # now item with max f(x) self.f = lambda x: -f(x) # will be popped first else: - raise ValueError("order must be either 'min' or 'max'.") + raise ValueError("Order must be either 'min' or 'max'.") def append(self, item): """Insert item at its correct position.""" @@ -773,18 +756,24 @@ def __len__(self): """Return current capacity of PriorityQueue.""" return len(self.heap) - def __contains__(self, item): - """Return True if item in PriorityQueue.""" - return (self.f(item), item) in self.heap + def __contains__(self, key): + """Return True if the key is in PriorityQueue.""" + return any([item == key for _, item in self.heap]) def __getitem__(self, key): - for _, item in self.heap: + """Returns the first value associated with key in PriorityQueue. + Raises KeyError if key is not present.""" + for value, item in self.heap: if item == key: - return item + return value + raise KeyError(str(key) + " is not in the priority queue") def __delitem__(self, key): """Delete the first occurrence of key.""" - self.heap.remove((self.f(key), key)) + try: + del self.heap[[item == key for _, item in self.heap].index(True)] + except ValueError: + raise KeyError(str(key) + " is not in the priority queue") heapq.heapify(self.heap) @@ -793,7 +782,7 @@ def __delitem__(self, key): class Bool(int): - """Just like `bool`, except values display as 'T' and 'F' instead of 'True' and 'False'""" + """Just like `bool`, except values display as 'T' and 'F' instead of 'True' and 'False'.""" __str__ = __repr__ = lambda self: 'T' if self else 'F' diff --git a/utils4e.py b/utils4e.py new file mode 100644 index 000000000..65cb9026f --- /dev/null +++ b/utils4e.py @@ -0,0 +1,807 @@ +"""Provides some utilities widely used by other modules""" + +import bisect +import collections +import collections.abc +import functools +import heapq +import os.path +import random +from itertools import chain, combinations +from statistics import mean + +import numpy as np + + +# part1. General data structures and their functions +# ______________________________________________________________________________ +# Queues: Stack, FIFOQueue, PriorityQueue +# Stack and FIFOQueue are implemented as list and collection.deque +# PriorityQueue is implemented here + + +class PriorityQueue: + """A Queue in which the minimum (or maximum) element (as determined by f and order) is returned first. + If order is 'min', the item with minimum f(x) is + returned first; if order is 'max', then it is the item with maximum f(x). + Also supports dict-like lookup.""" + + def __init__(self, order='min', f=lambda x: x): + self.heap = [] + + if order == 'min': + self.f = f + elif order == 'max': # now item with max f(x) + self.f = lambda x: -f(x) # will be popped first + else: + raise ValueError("Order must be either 'min' or 'max'.") + + def append(self, item): + """Insert item at its correct position.""" + heapq.heappush(self.heap, (self.f(item), item)) + + def extend(self, items): + """Insert each item in items at its correct position.""" + for item in items: + self.append(item) + + def pop(self): + """Pop and return the item (with min or max f(x) value) + depending on the order.""" + if self.heap: + return heapq.heappop(self.heap)[1] + else: + raise Exception('Trying to pop from empty PriorityQueue.') + + def __len__(self): + """Return current capacity of PriorityQueue.""" + return len(self.heap) + + def __contains__(self, key): + """Return True if the key is in PriorityQueue.""" + return any([item == key for _, item in self.heap]) + + def __getitem__(self, key): + """Returns the first value associated with key in PriorityQueue. + Raises KeyError if key is not present.""" + for value, item in self.heap: + if item == key: + return value + raise KeyError(str(key) + " is not in the priority queue") + + def __delitem__(self, key): + """Delete the first occurrence of key.""" + try: + del self.heap[[item == key for _, item in self.heap].index(True)] + except ValueError: + raise KeyError(str(key) + " is not in the priority queue") + heapq.heapify(self.heap) + + +# ______________________________________________________________________________ +# Functions on Sequences and Iterables + + +def sequence(iterable): + """Converts iterable to sequence, if it is not already one.""" + return (iterable if isinstance(iterable, collections.abc.Sequence) + else tuple([iterable])) + + +def remove_all(item, seq): + """Return a copy of seq (or string) with all occurrences of item removed.""" + if isinstance(seq, str): + return seq.replace(item, '') + elif isinstance(seq, set): + rest = seq.copy() + rest.remove(item) + return rest + else: + return [x for x in seq if x != item] + + +def unique(seq): + """Remove duplicate elements from seq. Assumes hashable elements.""" + return list(set(seq)) + + +def count(seq): + """Count the number of items in sequence that are interpreted as true.""" + return sum(map(bool, seq)) + + +def multimap(items): + """Given (key, val) pairs, return {key: [val, ....], ...}.""" + result = collections.defaultdict(list) + for (key, val) in items: + result[key].append(val) + return dict(result) + + +def multimap_items(mmap): + """Yield all (key, val) pairs stored in the multimap.""" + for (key, vals) in mmap.items(): + for val in vals: + yield key, val + + +def product(numbers): + """Return the product of the numbers, e.g. product([2, 3, 10]) == 60""" + result = 1 + for x in numbers: + result *= x + return result + + +def first(iterable, default=None): + """Return the first element of an iterable; or default.""" + return next(iter(iterable), default) + + +def is_in(elt, seq): + """Similar to (elt in seq), but compares with 'is', not '=='.""" + return any(x is elt for x in seq) + + +def mode(data): + """Return the most common data item. If there are ties, return any one of them.""" + [(item, count)] = collections.Counter(data).most_common(1) + return item + + +def power_set(iterable): + """power_set([1,2,3]) --> (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)""" + s = list(iterable) + return list(chain.from_iterable(combinations(s, r) for r in range(len(s) + 1)))[1:] + + +def extend(s, var, val): + """Copy dict s and extend it by setting var to val; return copy.""" + return {**s, var: val} + + +def flatten(seqs): + return sum(seqs, []) + + +# ______________________________________________________________________________ +# argmin and argmax + + +identity = lambda x: x + + +def argmin_random_tie(seq, key=identity): + """Return a minimum element of seq; break ties at random.""" + return min(shuffled(seq), key=key) + + +def argmax_random_tie(seq, key=identity): + """Return an element with highest fn(seq[i]) score; break ties at random.""" + return max(shuffled(seq), key=key) + + +def shuffled(iterable): + """Randomly shuffle a copy of iterable.""" + items = list(iterable) + random.shuffle(items) + return items + + +# part2. Mathematical and Statistical util functions +# ______________________________________________________________________________ + + +def histogram(values, mode=0, bin_function=None): + """Return a list of (value, count) pairs, summarizing the input values. + Sorted by increasing value, or if mode=1, by decreasing count. + If bin_function is given, map it over values first.""" + if bin_function: + values = map(bin_function, values) + + bins = {} + for val in values: + bins[val] = bins.get(val, 0) + 1 + + if mode: + return sorted(list(bins.items()), key=lambda x: (x[1], x[0]), reverse=True) + else: + return sorted(bins.items()) + + +def element_wise_product(x, y): + if hasattr(x, '__iter__') and hasattr(y, '__iter__'): + assert len(x) == len(y) + return [element_wise_product(_x, _y) for _x, _y in zip(x, y)] + elif hasattr(x, '__iter__') == hasattr(y, '__iter__'): + return x * y + else: + raise Exception('Inputs must be in the same size!') + + +def vector_add(a, b): + """Component-wise addition of two vectors.""" + if not (a and b): + return a or b + if hasattr(a, '__iter__') and hasattr(b, '__iter__'): + assert len(a) == len(b) + return list(map(vector_add, a, b)) + else: + try: + return a + b + except TypeError: + raise Exception('Inputs must be in the same size!') + + +def scalar_vector_product(x, y): + """Return vector as a product of a scalar and a vector recursively.""" + return [scalar_vector_product(x, _y) for _y in y] if hasattr(y, '__iter__') else x * y + + +def map_vector(f, x): + """Apply function f to iterable x.""" + return [map_vector(f, _x) for _x in x] if hasattr(x, '__iter__') else list(map(f, [x]))[0] + + +def probability(p): + """Return true with probability p.""" + return p > random.uniform(0.0, 1.0) + + +def weighted_sample_with_replacement(n, seq, weights): + """Pick n samples from seq at random, with replacement, with the + probability of each element in proportion to its corresponding + weight.""" + sample = weighted_sampler(seq, weights) + + return [sample() for _ in range(n)] + + +def weighted_sampler(seq, weights): + """Return a random-sample function that picks from seq weighted by weights.""" + totals = [] + for w in weights: + totals.append(w + totals[-1] if totals else w) + + return lambda: seq[bisect.bisect(totals, random.uniform(0, totals[-1]))] + + +def weighted_choice(choices): + """A weighted version of random.choice""" + # NOTE: Should be replaced by random.choices if we port to Python 3.6 + + total = sum(w for _, w in choices) + r = random.uniform(0, total) + upto = 0 + for c, w in choices: + if upto + w >= r: + return c, w + upto += w + + +def rounder(numbers, d=4): + """Round a single number, or sequence of numbers, to d decimal places.""" + if isinstance(numbers, (int, float)): + return round(numbers, d) + else: + constructor = type(numbers) # Can be list, set, tuple, etc. + return constructor(rounder(n, d) for n in numbers) + + +def num_or_str(x): # TODO: rename as `atom` + """The argument is a string; convert to a number if + possible, or strip it.""" + try: + return int(x) + except ValueError: + try: + return float(x) + except ValueError: + return str(x).strip() + + +def euclidean_distance(x, y): + return np.sqrt(sum((_x - _y) ** 2 for _x, _y in zip(x, y))) + + +def manhattan_distance(x, y): + return sum(abs(_x - _y) for _x, _y in zip(x, y)) + + +def hamming_distance(x, y): + return sum(_x != _y for _x, _y in zip(x, y)) + + +def rms_error(x, y): + return np.sqrt(ms_error(x, y)) + + +def ms_error(x, y): + return mean((x - y) ** 2 for x, y in zip(x, y)) + + +def mean_error(x, y): + return mean(abs(x - y) for x, y in zip(x, y)) + + +def mean_boolean_error(x, y): + return mean(_x != _y for _x, _y in zip(x, y)) + + +# part3. Neural network util functions +# ______________________________________________________________________________ + + +def cross_entropy_loss(x, y): + """Cross entropy loss function. x and y are 1D iterable objects.""" + return (-1.0 / len(x)) * sum(x * np.log(_y) + (1 - _x) * np.log(1 - _y) for _x, _y in zip(x, y)) + + +def mean_squared_error_loss(x, y): + """Min square loss function. x and y are 1D iterable objects.""" + return (1.0 / len(x)) * sum((_x - _y) ** 2 for _x, _y in zip(x, y)) + + +def normalize(dist): + """Multiply each number by a constant such that the sum is 1.0""" + if isinstance(dist, dict): + total = sum(dist.values()) + for key in dist: + dist[key] = dist[key] / total + assert 0 <= dist[key] <= 1 # probabilities must be between 0 and 1 + return dist + total = sum(dist) + return [(n / total) for n in dist] + + +def random_weights(min_value, max_value, num_weights): + return [random.uniform(min_value, max_value) for _ in range(num_weights)] + + +def conv1D(x, k): + """1D convolution. x: input vector; K: kernel vector.""" + return np.convolve(x, k, mode='same') + + +def gaussian_kernel(size=3): + return [gaussian((size - 1) / 2, 0.1, x) for x in range(size)] + + +def gaussian_kernel_1D(size=3, sigma=0.5): + return [gaussian((size - 1) / 2, sigma, x) for x in range(size)] + + +def gaussian_kernel_2D(size=3, sigma=0.5): + x, y = np.mgrid[-size // 2 + 1:size // 2 + 1, -size // 2 + 1:size // 2 + 1] + g = np.exp(-((x ** 2 + y ** 2) / (2.0 * sigma ** 2))) + return g / g.sum() + + +def step(x): + """Return activation value of x with sign function.""" + return 1 if x >= 0 else 0 + + +def gaussian(mean, st_dev, x): + """Given the mean and standard deviation of a distribution, it returns the probability of x.""" + return 1 / (np.sqrt(2 * np.pi) * st_dev) * np.exp(-0.5 * (float(x - mean) / st_dev) ** 2) + + +def linear_kernel(x, y=None): + if y is None: + y = x + return np.dot(x, y.T) + + +def polynomial_kernel(x, y=None, degree=2.0): + if y is None: + y = x + return (1.0 + np.dot(x, y.T)) ** degree + + +def rbf_kernel(x, y=None, gamma=None): + """Radial-basis function kernel (aka squared-exponential kernel).""" + if y is None: + y = x + if gamma is None: + gamma = 1.0 / x.shape[1] # 1.0 / n_features + return np.exp(-gamma * (-2.0 * np.dot(x, y.T) + + np.sum(x * x, axis=1).reshape((-1, 1)) + np.sum(y * y, axis=1).reshape((1, -1)))) + + +# part4. Self defined data structures +# ______________________________________________________________________________ +# Grid Functions + + +orientations = EAST, NORTH, WEST, SOUTH = [(1, 0), (0, 1), (-1, 0), (0, -1)] +turns = LEFT, RIGHT = (+1, -1) + + +def turn_heading(heading, inc, headings=orientations): + return headings[(headings.index(heading) + inc) % len(headings)] + + +def turn_right(heading): + return turn_heading(heading, RIGHT) + + +def turn_left(heading): + return turn_heading(heading, LEFT) + + +def distance(a, b): + """The distance between two (x, y) points.""" + xA, yA = a + xB, yB = b + return np.hypot((xA - xB), (yA - yB)) + + +def distance_squared(a, b): + """The square of the distance between two (x, y) points.""" + xA, yA = a + xB, yB = b + return (xA - xB) ** 2 + (yA - yB) ** 2 + + +# ______________________________________________________________________________ +# Misc Functions + + +class injection: + """Dependency injection of temporary values for global functions/classes/etc. + E.g., `with injection(DataBase=MockDataBase): ...`""" + + def __init__(self, **kwds): + self.new = kwds + + def __enter__(self): + self.old = {v: globals()[v] for v in self.new} + globals().update(self.new) + + def __exit__(self, type, value, traceback): + globals().update(self.old) + + +def memoize(fn, slot=None, maxsize=32): + """Memoize fn: make it remember the computed value for any argument list. + If slot is specified, store result in that slot of first argument. + If slot is false, use lru_cache for caching the values.""" + if slot: + def memoized_fn(obj, *args): + if hasattr(obj, slot): + return getattr(obj, slot) + else: + val = fn(obj, *args) + setattr(obj, slot, val) + return val + else: + @functools.lru_cache(maxsize=maxsize) + def memoized_fn(*args): + return fn(*args) + + return memoized_fn + + +def name(obj): + """Try to find some reasonable name for the object.""" + return (getattr(obj, 'name', 0) or getattr(obj, '__name__', 0) or + getattr(getattr(obj, '__class__', 0), '__name__', 0) or + str(obj)) + + +def isnumber(x): + """Is x a number?""" + return hasattr(x, '__int__') + + +def issequence(x): + """Is x a sequence?""" + return isinstance(x, collections.abc.Sequence) + + +def print_table(table, header=None, sep=' ', numfmt='{}'): + """Print a list of lists as a table, so that columns line up nicely. + header, if specified, will be printed as the first row. + numfmt is the format for all numbers; you might want e.g. '{:.2f}'. + (If you want different formats in different columns, + don't use print_table.) sep is the separator between columns.""" + justs = ['rjust' if isnumber(x) else 'ljust' for x in table[0]] + + if header: + table.insert(0, header) + + table = [[numfmt.format(x) if isnumber(x) else x for x in row] + for row in table] + sizes = list( + map(lambda seq: max(map(len, seq)), + list(zip(*[map(str, row) for row in table])))) + + for row in table: + print(sep.join(getattr( + str(x), j)(size) for (j, size, x) in zip(justs, sizes, row))) + + +def open_data(name, mode='r'): + aima_root = os.path.dirname(__file__) + aima_file = os.path.join(aima_root, *['aima-data', name]) + + return open(aima_file, mode=mode) + + +def failure_test(algorithm, tests): + """Grades the given algorithm based on how many tests it passes. + Most algorithms have arbitrary output on correct execution, which is difficult + to check for correctness. On the other hand, a lot of algorithms output something + particular on fail (for example, False, or None). + tests is a list with each element in the form: (values, failure_output).""" + return mean(int(algorithm(x) != y) for x, y in tests) + + +# ______________________________________________________________________________ +# Expressions + +# See https://docs.python.org/3/reference/expressions.html#operator-precedence +# See https://docs.python.org/3/reference/datamodel.html#special-method-names + + +class Expr: + """A mathematical expression with an operator and 0 or more arguments. + op is a str like '+' or 'sin'; args are Expressions. + Expr('x') or Symbol('x') creates a symbol (a nullary Expr). + Expr('-', x) creates a unary; Expr('+', x, 1) creates a binary.""" + + def __init__(self, op, *args): + self.op = str(op) + self.args = args + + # Operator overloads + def __neg__(self): + return Expr('-', self) + + def __pos__(self): + return Expr('+', self) + + def __invert__(self): + return Expr('~', self) + + def __add__(self, rhs): + return Expr('+', self, rhs) + + def __sub__(self, rhs): + return Expr('-', self, rhs) + + def __mul__(self, rhs): + return Expr('*', self, rhs) + + def __pow__(self, rhs): + return Expr('**', self, rhs) + + def __mod__(self, rhs): + return Expr('%', self, rhs) + + def __and__(self, rhs): + return Expr('&', self, rhs) + + def __xor__(self, rhs): + return Expr('^', self, rhs) + + def __rshift__(self, rhs): + return Expr('>>', self, rhs) + + def __lshift__(self, rhs): + return Expr('<<', self, rhs) + + def __truediv__(self, rhs): + return Expr('/', self, rhs) + + def __floordiv__(self, rhs): + return Expr('//', self, rhs) + + def __matmul__(self, rhs): + return Expr('@', self, rhs) + + def __or__(self, rhs): + """Allow both P | Q, and P |'==>'| Q.""" + if isinstance(rhs, Expression): + return Expr('|', self, rhs) + else: + return PartialExpr(rhs, self) + + # Reverse operator overloads + def __radd__(self, lhs): + return Expr('+', lhs, self) + + def __rsub__(self, lhs): + return Expr('-', lhs, self) + + def __rmul__(self, lhs): + return Expr('*', lhs, self) + + def __rdiv__(self, lhs): + return Expr('/', lhs, self) + + def __rpow__(self, lhs): + return Expr('**', lhs, self) + + def __rmod__(self, lhs): + return Expr('%', lhs, self) + + def __rand__(self, lhs): + return Expr('&', lhs, self) + + def __rxor__(self, lhs): + return Expr('^', lhs, self) + + def __ror__(self, lhs): + return Expr('|', lhs, self) + + def __rrshift__(self, lhs): + return Expr('>>', lhs, self) + + def __rlshift__(self, lhs): + return Expr('<<', lhs, self) + + def __rtruediv__(self, lhs): + return Expr('/', lhs, self) + + def __rfloordiv__(self, lhs): + return Expr('//', lhs, self) + + def __rmatmul__(self, lhs): + return Expr('@', lhs, self) + + def __call__(self, *args): + """Call: if 'f' is a Symbol, then f(0) == Expr('f', 0).""" + if self.args: + raise ValueError('Can only do a call for a Symbol, not an Expr') + else: + return Expr(self.op, *args) + + # Equality and repr + def __eq__(self, other): + """'x == y' evaluates to True or False; does not build an Expr.""" + return isinstance(other, Expr) and self.op == other.op and self.args == other.args + + def __lt__(self, other): + return isinstance(other, Expr) and str(self) < str(other) + + def __hash__(self): + return hash(self.op) ^ hash(self.args) + + def __repr__(self): + op = self.op + args = [str(arg) for arg in self.args] + if op.isidentifier(): # f(x) or f(x, y) + return '{}({})'.format(op, ', '.join(args)) if args else op + elif len(args) == 1: # -x or -(x + 1) + return op + args[0] + else: # (x - y) + opp = (' ' + op + ' ') + return '(' + opp.join(args) + ')' + + +# An 'Expression' is either an Expr or a Number. +# Symbol is not an explicit type; it is any Expr with 0 args. + + +Number = (int, float, complex) +Expression = (Expr, Number) + + +def Symbol(name): + """A Symbol is just an Expr with no args.""" + return Expr(name) + + +def symbols(names): + """Return a tuple of Symbols; names is a comma/whitespace delimited str.""" + return tuple(Symbol(name) for name in names.replace(',', ' ').split()) + + +def subexpressions(x): + """Yield the subexpressions of an Expression (including x itself).""" + yield x + if isinstance(x, Expr): + for arg in x.args: + yield from subexpressions(arg) + + +def arity(expression): + """The number of sub-expressions in this expression.""" + if isinstance(expression, Expr): + return len(expression.args) + else: # expression is a number + return 0 + + +# For operators that are not defined in Python, we allow new InfixOps: + + +class PartialExpr: + """Given 'P |'==>'| Q, first form PartialExpr('==>', P), then combine with Q.""" + + def __init__(self, op, lhs): + self.op, self.lhs = op, lhs + + def __or__(self, rhs): + return Expr(self.op, self.lhs, rhs) + + def __repr__(self): + return "PartialExpr('{}', {})".format(self.op, self.lhs) + + +def expr(x): + """Shortcut to create an Expression. x is a str in which: + - identifiers are automatically defined as Symbols. + - ==> is treated as an infix |'==>'|, as are <== and <=>. + If x is already an Expression, it is returned unchanged. Example: + >>> expr('P & Q ==> Q') + ((P & Q) ==> Q) + """ + if isinstance(x, str): + return eval(expr_handle_infix_ops(x), defaultkeydict(Symbol)) + else: + return x + + +infix_ops = '==> <== <=>'.split() + + +def expr_handle_infix_ops(x): + """Given a str, return a new str with ==> replaced by |'==>'|, etc. + >>> expr_handle_infix_ops('P ==> Q') + "P |'==>'| Q" + """ + for op in infix_ops: + x = x.replace(op, '|' + repr(op) + '|') + return x + + +class defaultkeydict(collections.defaultdict): + """Like defaultdict, but the default_factory is a function of the key. + >>> d = defaultkeydict(len); d['four'] + 4 + """ + + def __missing__(self, key): + self[key] = result = self.default_factory(key) + return result + + +class hashabledict(dict): + """Allows hashing by representing a dictionary as tuple of key:value pairs. + May cause problems as the hash value may change during runtime.""" + + def __hash__(self): + return 1 + + +# ______________________________________________________________________________ +# Monte Carlo tree node and ucb function + + +class MCT_Node: + """Node in the Monte Carlo search tree, keeps track of the children states.""" + + def __init__(self, parent=None, state=None, U=0, N=0): + self.__dict__.update(parent=parent, state=state, U=U, N=N) + self.children = {} + self.actions = None + + +def ucb(n, C=1.4): + return np.inf if n.N == 0 else n.U / n.N + C * np.sqrt(np.log(n.parent.N) / n.N) + + +# ______________________________________________________________________________ +# Useful Shorthands + + +class Bool(int): + """Just like `bool`, except values display as 'T' and 'F' instead of 'True' and 'False'.""" + __str__ = __repr__ = lambda self: 'T' if self else 'F' + + +T = Bool(True) +F = Bool(False) diff --git a/viterbi_algorithm.ipynb b/viterbi_algorithm.ipynb new file mode 100644 index 000000000..9c23c4f75 --- /dev/null +++ b/viterbi_algorithm.ipynb @@ -0,0 +1,418 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Probabilistic Reasoning over Time\n", + "---\n", + "# Finding the Most Likely Sequence with Viterbi Algorithm\n", + "\n", + "## Introduction\n", + "An ***Hidden Markov Model*** (HMM) network is parameterized by two distributions:\n", + "\n", + "- the *emission or sensor probabilties* giving the conditional probability of observing evidence values for each hidden state;\n", + "- the *transition probabilities* giving the conditional probability of moving between states during the sequence. \n", + "\n", + "Additionally, an *initial distribution* describes the probability of a sequence starting in each state.\n", + "\n", + "At each time $t$, $X_t$ represents the *hidden state* and $E_t$ represents an *observation* at that time." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "from probability import *" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mclass\u001b[0m \u001b[0mHiddenMarkovModel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"A Hidden markov model which takes Transition model and Sensor model as inputs\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransition_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msensor_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mprior\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransition_model\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtransition_model\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msensor_model\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msensor_model\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mprior\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.5\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0msensor_dist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mev\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msensor_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msensor_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource HiddenMarkovModel" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Finding the Most Likely Sequence\n", + "\n", + "There is a linear-time algorithm for finding the most likely sequence: the easiest way to think about the problem is to view each sequence as a path through a graph whose nodes are the possible states at each time step. Now consider the task of finding the most likely path through this graph, where the likelihood of any path is the product of the transition probabilities along the path and the probabilities of the given observations at each state. There is a recursive relationship between most likely paths to each state $x_{t+1}$ and most likely paths to each state $x_t$ . We can write this relationship as an equation connecting the probabilities of the paths:\n", + "\n", + "$$ \n", + "\\begin{align*}\n", + "m_{1:t+1} &= \\max_{x_{1:t}} \\textbf{P}(\\textbf{x}_{1:t}, \\textbf{X}_{t+1} | \\textbf{e}_{1:t+1}) \\\\\n", + "&= \\alpha \\textbf{P}(\\textbf{e}_{t+1} | \\textbf{X}_{t+1}) \\max_{x_t} \\Big(\\textbf{P}\n", + "(\\textbf{X}_{t+1} | \\textbf{x}_t) \\max_{x_{1:t-1}} P(\\textbf{x}_{1:t-1}, \\textbf{x}_{t} | \\textbf{e}_{1:t})\\Big)\n", + "\\end{align*}\n", + "$$\n", + "\n", + "The *Viterbi algorithm* is a dynamic programming algorithm for *finding the most likely sequence of hidden states*, called the Viterbi path, that results in a sequence of observed events in the context of HMMs.\n", + "This algorithms is useful in many applications, including *speech recognition*, where the aim is to find the most likely sequence of words, given a series of sounds and the *reconstruction of bit strings transmitted over a noisy channel*." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\u001b[0;32mdef\u001b[0m \u001b[0mviterbi\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m\"\"\"\u001b[0m\n", + "\u001b[0;34m [Equation 15.11]\u001b[0m\n", + "\u001b[0;34m Viterbi algorithm to find the most likely sequence. Computes the best path and the\u001b[0m\n", + "\u001b[0;34m corresponding probabilities, given an HMM model and a sequence of observations.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mev\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mev\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mev\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minsert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0.0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0m_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the recursion is initialized with m1 = forward(P(X0), e1)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprior\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mev\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# keep track of maximizing predecessors\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbacktracking_graph\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0melement_wise_product\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msensor_dist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mev\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement_wise_product\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransition_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement_wise_product\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransition_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mbacktracking_graph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement_wise_product\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransition_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement_wise_product\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransition_model\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# computed probabilities\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mml_probabilities\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0.0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# most likely sequence\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mml_path\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mev\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# the construction of the most likely sequence starts in the final state with the largest probability, and\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;31m# runs backwards; the algorithm needs to store for each xt its predecessor xt-1 maximizing its probability\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mi_max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mml_probabilities\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi_max\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mml_path\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mi_max\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mi\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0mi_max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbacktracking_graph\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi_max\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n", + "\u001b[0;34m\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mml_path\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mml_probabilities\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%psource viterbi" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Umbrella World\n", + "---\n", + "\n", + "> You are the security guard stationed at a secret under-ground installation. Each day, you try to guess whether it’s raining today, but your only access to the outside world occurs each morning when you see the director coming in with, or without, an umbrella.\n", + "\n", + "In this problem $t$ corresponds to each day of the week, the hidden state $X_t$ represent the *weather* outside at day $t$ (whether it is rainy or sunny) and observations record $E_t$ whether at day $t$ the security guard sees the director carrying an *umbrella* or not." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Observation Emission or Sensor Probabilities $P(E_t := Umbrella_t | X_t := Weather_t)$\n", + "We need to assume that we have some prior knowledge about the director's behavior to estimate the emission probabilities for each hidden state:\n", + "\n", + "| | $yes$ | $no$ |\n", + "| --- | --- | --- |\n", + "| $Sunny$ | 0.10 | 0.90 |\n", + "| $Rainy$ | 0.80 | 0.20 |\n", + "\n", + "#### Initial Probability $P(X_0 := Weather_0)$\n", + "We will assume that we don't know anything useful about the likelihood of a sequence starting in either state. If the sequences start each week on Monday and end each week on Friday (so each week is a new sequence), then this assumption means that it's equally likely that the weather on a Monday may be Rainy or Sunny. We can assign equal probability to each starting state:\n", + "\n", + "| $Sunny$ | $Rainy$ |\n", + "| --- | ---\n", + "| 0.5 | 0.5 |\n", + "\n", + "#### State Transition Probabilities $P(X_{t} := Weather_t | X_{t-1} := Weather_{t-1})$\n", + "Finally, we will assume that we can estimate transition probabilities from something like historical weather data for the area. Under this assumption, we get the conditional probability:\n", + "\n", + "| | $Sunny$ | $Rainy$ |\n", + "| --- | --- | --- |\n", + "|$Sunny$| 0.70 | 0.30 |\n", + "|$Rainy$| 0.30 | 0.70 |" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "umbrella_transition = [[0.7, 0.3], [0.3, 0.7]]\n", + "umbrella_sensor = [[0.9, 0.2], [0.1, 0.8]]\n", + "umbrellaHMM = HiddenMarkovModel(umbrella_transition, umbrella_sensor)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from graphviz import Digraph" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "Start\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "Rainy\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.5\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "Sunny\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.5\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.6\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.2\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "Yes\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.8\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "No\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.2\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.4\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.8\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.1\n", + "\n", + "\n", + "\n", + "Codestin Search App\n", + "\n", + "\n", + "0.9\n", + "\n", + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dot = Digraph()\n", + "\n", + "dot.node('I', 'Start', shape='doublecircle')\n", + "dot.node('R', 'Rainy')\n", + "dot.node('S','Sunny')\n", + "\n", + "dot.edge('I', 'R', label='0.5')\n", + "dot.edge('I', 'S', label='0.5')\n", + "\n", + "dot.edge('R', 'S', label='0.2')\n", + "dot.edge('S', 'R', label='0.4')\n", + "\n", + "dot.node('Y', 'Yes')\n", + "dot.node('N', 'No')\n", + "\n", + "dot.edge('R', 'R', label='0.6')\n", + "dot.edge('R', 'Y', label='0.8')\n", + "dot.edge('R', 'N', label='0.2')\n", + "\n", + "dot.edge('S', 'S', label='0.8')\n", + "dot.edge('S', 'Y', label='0.1')\n", + "dot.edge('S', 'N', label='0.9')\n", + "\n", + "dot" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Suppose that $[true, true, false, true, true]$ is the umbrella sequence for the security guard’s first five days on the job. What is the weather sequence most likely to explain this?" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "from utils import rounder" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "([1, 1, 0, 1, 1], [0.8182, 0.5155, 0.1237, 0.0334, 0.021])" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "umbrella_evidence = [True, True, False, True, True]\n", + "\n", + "rounder(viterbi(umbrellaHMM, umbrella_evidence))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}