RL_PathPlanning_Data_collection.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of [Jun17-2021][2] Obstacle.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **Path Planninng for Robots Using Deep Reinforcement Learning**"
      ],
      "metadata": {
        "id": "kqeRc8cXnI7q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Problem description:**\n",
        "\n",
        "We have a robot that aims to collect data of several low-powered IoT sensors (black circles in the following image). The robot starts its mission from the start terminal (${\\color{red}{red}}$ square in the following image). There is a charging station in the environment so that the robot can recharge its battery if it is running out of energy (${\\color{green}{green}}$ square).\n",
        "\n",
        "Also, there are several obstacles in the environment. The task of the robot is to collect data of all sensors in the shortest possible time while it avoids any collision to the obstacles.\n",
        "\n",
        "\n",
        "In the following image, we have depicted the environment:"
      ],
      "metadata": {
        "id": "I90NjpeKnxqJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASIAAAESCAYAAABZxNgbAAAMbWlDQ1BJQ0MgUHJvZmlsZQAASImVlwdYk1cXgO83kpCQECAQARlhL0FkDxkhrAgCMgUXIQkkjBgTgogbLVWwbhHFiVZFLFoHIEVF3KMobusoSlGp1KIWRVH5bwZo7T+e/zzP/e6bc88959yT+40LAL2fJ5XmoboA5EsKZAmRoayJaeks0hOAADKgA3ugzePLpez4+BgAZaj/u7y5Ba2hXHdR+vrn+H8VfYFQzgcAmQw5UyDn50NuAQDfxJfKCgAgKvXWMwukSl4A2UAGE4S8VsnZat6j5Ew1N6tskhI4kK8CoEXl8WTZAOjch3pWIT8b+tH5ANlNIhBLAKCPghzEF/EEkJW5j8rPn67kSsgO0F4KGeYDfDO/8Jn9N/+Zw/55vOxhVq9LJVphYrk0jzfr/yzN/5b8PMVQDDvYqCJZVIJy/bCGd3KnRyuZCrlHkhkbp6w15H6xQF13AFCKSBGVrLZHTflyDqwfYEJ2E/DCoiGbQo6Q5MXGaPSZWeIILmS4W9AicQE3CbIR5CVCeXiixmabbHqCJhZanyXjsDX68zyZKq4y1kNFbjJb4/+VSMjV+Md0ikVJqZApkG0KxSmxkHUgu8pzE6M1NmOLRZzYIRuZIkGZvw3kBKEkMlTtHyvMkkUkaOzL8uVD68W2icTcWA0fLBAlRanrg53m81T5w7VgV4USdvKQH6F8YszQWgTCsHD12rFnQklyosZPv7QgNEE9F6dI8+I19riVMC9SqbeC7CkvTNTMxVMK4OZU+8ezpAXxSeo88eIc3rh4dT74ShADOCAMsIACtkwwHeQAcVtPQw/8pR6JADwgA9lACFw0mqEZqaoRCbwmgmLwByQhkA/PC1WNCkEh1H8c1qqvLiBLNVqompELnkDOB9EgD/5WqGZJhqOlgN+gRvyP6DzY+DDfPNiU4/9eP6T9rGFDTYxGoxiKyKIPWRLDiWHEKGIE0RE3wYPwADwGXkNgc8d9cb+hdXy2JzwhtBMeE24SOgh3p4lLZF9lOR50QP8RmlpkflkL3A769MJD8UDoHXrGmbgJcME9YRw2Hgwje0EtR5O3siqsr3z/bQVf/BsaO7IbGSWPIIeQHb6eqeOk4zXsRVnrL+ujzjVzuN6c4ZGv43O+qL4A9tFfW2JLsEPYOewkdgFrxhoACzuBNWKXsWNKHt5dv6l211C0BFU+udCP+B/xeJqYykrK3Wrdut0+qMcKhEUFyhuPM106SybOFhWw2PDtIGRxJXzXUSx3N3d3AJTvGvXj6zVT9Q5BmBc/60q+AyDQc3BwsPmzLoYOwGF4z1A6P+sc/OFjogiA88v5ClmhWocrLwT4lKDDO80YmANr4ADX4w68QQAIAeFgHIgDSSANTIVVFsF9LgMzwRywEJSCcrASrAMbwVawA+wBP4CDoAE0g5PgLLgEroKb4B7cPV3gOegFb8AAgiAkhIYwEGPEArFFnBF3xBcJQsKRGCQBSUMykGxEgiiQOcgipBxZjWxEtiM1yI/IUeQkcgFpR+4ij5Bu5BXyHsVQKmqAmqF26GjUF2Wj0WgSOgXNRmegxehidDlaiVaj+9B69CR6Cb2JdqDP0T4MYNoYE7PEXDBfjIPFYelYFibD5mFlWAVWjdVhTfB/vo51YD3YO5yIM3AW7gJ3cBSejPPxGfg8fBm+Ed+D1+On8ev4I7wX/0SgEUwJzgR/ApcwkZBNmEkoJVQQdhGOEM7Ae6mL8IZIJDKJ9kQfeC+mEXOIs4nLiJuJ+4ktxHZiJ7GPRCIZk5xJgaQ4Eo9UQColbSDtI50gXSN1kfq1tLUstNy1IrTStSRaJVoVWnu1jmtd03qqNUDWJduS/clxZAF5FnkFeSe5iXyF3EUeoOhR7CmBlCRKDmUhpZJSRzlDuU95ra2tbaXtpz1BW6y9QLtS+4D2ee1H2u+o+lQnKoc6maqgLqfuprZQ71Jf02g0O1oILZ1WQFtOq6Gdoj2k9eswdFx1uDoCnfk6VTr1Otd0XtDJdFs6mz6VXkyvoB+iX6H36JJ17XQ5ujzdebpVukd1b+v26TH0xujF6eXrLdPbq3dB75k+Sd9OP1xfoL9Yf4f+Kf1OBsawZnAYfMYixk7GGUaXAdHA3oBrkGNQbvCDQZtBr6G+oadhimGRYZXhMcMOJsa0Y3KZecwVzIPMW8z3I8xGsEcIRywdUTfi2oi3RiONQoyERmVG+41uGr03ZhmHG+carzJuMH5ggps4mUwwmWmyxeSMSc9Ig5EBI/kjy0YeHPmLKWrqZJpgOtt0h+ll0z4zc7NIM6nZBrNTZj3mTPMQ8xzztebHzbstGBZBFmKLtRYnLH5nGbLYrDxWJes0q9fS1DLKUmG53bLNcsDK3irZqsRqv9UDa4q1r3WW9VrrVuteGwub8TZzbGptfrEl2/raimzX256zfWtnb5dq961dg90zeyN7rn2xfa39fQeaQ7DDDIdqhxuOREdfx1zHzY5XnVAnLyeRU5XTFWfU2dtZ7LzZuX0UYZTfKMmo6lG3XagubJdCl1qXR65M1xjXEtcG1xejbUanj141+tzoT25ebnluO93ujdEfM25MyZimMa/cndz57lXuNzxoHhEe8z0aPV56OnsKPbd43vFieI33+tar1eujt4+3zLvOu9vHxifDZ5PPbV8D33jfZb7n/Qh+oX7z/Zr93vl7+xf4H/T/M8AlIDdgb8CzsfZjhWN3ju0MtArkBW4P7AhiBWUEbQvqCLYM5gVXBz8OsQ4RhOwKecp2ZOew97FfhLqFykKPhL7l+HPmclrCsLDIsLKwtnD98OTwjeEPI6wisiNqI3ojvSJnR7ZEEaKio1ZF3eaacfncGm7vOJ9xc8edjqZGJ0ZvjH4c4xQji2kaj44fN37N+PuxtrGS2IY4EMeNWxP3IN4+fkb8TxOIE+InVE14kjAmYU7CuURG4rTEvYlvkkKTViTdS3ZIViS3ptBTJqfUpLxNDUtdndoxcfTEuRMvpZmkidMa00npKem70vsmhU9aN6lrstfk0sm3pthPKZpyYarJ1Lypx6bRp/GmHcogZKRm7M34wIvjVfP6MrmZmzJ7+Rz+ev5zQYhgraBbGChcLXyaFZi1OutZdmD2muxuUbCoQtQj5og3il/mROVszXmbG5e7O3cwLzVvf75Wfkb+UYm+JFdyerr59KLp7VJnaam0Y4b/jHUzemXRsl1yRD5F3lhgAD/qLyscFN8oHhUGFVYV9s9MmXmoSK9IUnR5ltOspbOeFkcUfz8bn82f3TrHcs7COY/msudun4fMy5zXOt96/uL5XQsiF+xZSFmYu/DnEreS1SV/LUpd1LTYbPGCxZ3fRH5TW6pTKiu9/W3At1uX4EvES9qWeizdsPRTmaDsYrlbeUX5h2X8ZRe/G/Nd5XeDy7OWt63wXrFlJXGlZOWtVcGr9qzWW128unPN+DX1a1lry9b+tW7augsVnhVb11PWK9Z3VMZUNm6w2bByw4eNoo03q0Kr9m8y3bR009vNgs3XtoRsqdtqtrV86/tt4m13tkdur6+2q67YQdxRuOPJzpSd5773/b5ml8mu8l0fd0t2d+xJ2HO6xqemZq/p3hW1aK2itnvf5H1Xfwj7obHOpW77fub+8gPggOLA7z9m/HjrYPTB1kO+h+oO2x7edIRxpKweqZ9V39sgauhoTGtsPzruaGtTQNORn1x/2t1s2Vx1zPDYiuOU44uPD54oPtHXIm3pOZl9srN1Wuu9UxNP3Tg94XTbmegz589GnD11jn3uxPnA880X/C8cveh7seGS96X6y16Xj/zs9fORNu+2+is+Vxqv+l1tah/bfvxa8LWT18Oun73BvXHpZuzN9lvJt+7cnny7447gzrO7eXdf/lL4y8C9BfcJ98se6D6oeGj6sPpXx1/3d3h3HHsU9ujy48TH9zr5nc9/k//2oWvxE9qTiqcWT2ueuT9r7o7ovvr7pN+7nkufD/SU/qH3x6YXDi8O/xny5+Xeib1dL2UvB18te238evdfnn+19sX3PXyT/2bgbVm/cf+ed77vzr1Pff90YOYH0ofKj44fmz5Ff7o/mD84KOXJeKpPAQw2NCsLgFe7AaClAcCA5zbKJPVZUCWI+vyqIvCfWH1eVIk3AHWwU37Gc1oAOACbHWy0BQAoP+GTQgDq4THcNCLP8nBX+6LCkxChf3DwtRkApCYAPsoGBwc2Dw5+3AmTvQtAywz1GVQpRHhm2OappGvMogXgK1GfT79Y49c9UGagmv63/l91/Y7W8ExDTgAAADhlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAAqACAAQAAAABAAABIqADAAQAAAABAAABEgAAAACknV6sAAAvH0lEQVR4Ae19DXhVx3nm4V8grEpabH5MkMBBMRDWWCR1Nq5/SWxsmnWI8eIat45xCt08hBBvs8k+Thtqh8Rdd9Ns6uaH3SR2Ao6bpUmatnnc3Xjtkp8nXmMFTMBUIBAsIJC1CISwQBi073ulIUfX994zR3fuPfMdfR/Pyzlnzpw577wzmvudOTNzgkBNFVAFVAFVQBVQBVQBVUAVUAVUAVVAFVAFVAFVQBVQBVQBVUAVUAVUAVVAFVAFVAFVQBVQBVQBVUAVUAVUgfIqsAa32wacA54CwrYIB3uAN4AXgDrA2DjsfBPoAo4BDwNqqoAqoAoMSYEP4aoPAl8FngKMTcLOKeAeoAJ4AvglYOwL2PkpUAPMAdgYLQbUVAFVQBUYsgKfw5VPha5ehf1fhI4rsd8DXD0QdhTb2wb2uXkMeDZ0rLuqgCogWIHRnnCfBx47QlzOYL8FYPhxYCoQPs99ela5jI0aEUyYMGHh7Nmzg76+vky8kSNHBm+++WYwZsyYoLe3N6ioqAjeeOMNxnvL9uzZs8HYsWOD8+fPB6NHjw4uXryYSWPEiBGZfYYxjXHjxgWMO378+Lek0dPTk7kH4zH+hQsXAnKgMb1Ro0Zl+PA+UWmcO3cuw4f8mYbmqb/ctJySrXvbt2/vQHW+PFOpi/jPl4ZoIvLwelY+TuH4MoDnaDw2Zs6Z4/B2Iw6IYNasWX0QKnzO2/2Wlpbgqquu8pZfNjFJfCVxpc6S+OKH+WB23RjKcf/P81CudHtNN5KrykqSx6cBnqOFz5tz/Wfy/E8vRIrV1tZKoZrhKYmvJK4UVxpfFxXXl4ZoFzJzTShD7COie8DwTqANCJ/nPs8VNPM4VTCSJyf5iCjJJPGVxJV1QBpfF/W23A0RXZQKYNQAuM+wHwDvBO4GGPanwKsAX+fTvg18BqgBrgb+EHgKSI2ZviMpGZLEVxJXlr80vi7qbLkbIjYmfBv2aeD+gX2GsX+IjdAGgB7QdcC9gLHPYoed13we/WfgCeA5oKCxY1mKsQNdkkniK4kr64A0vi7qbbkbovUgzdYhDIbRfgLQ2xkP3Ay0AsY4AHIlwL6hycAXgUiT9GjW3W26wiKz5UUESXwlcWXhSuProkKWuyFywdk6DUmd1ZMmcUynHJPEVxJX1gBpfF3U2lQ3RBy/I8UOHz4shWqGpyS+krhSXGl8XVTcVDdEHGwoxd7+9rdLoZrhKYmvJK4UVxpfFxU31Q0RR91KsV27IkcjeJUVSXwlcWUhS+PromLKea00hNwuXLiwb9u2bVZXTpmCuSScTOLQOJtjYGaIw1TRW4/u+mPHnCapiakCQ1IAb6ZfwYXvGtLFoYtS7RHFGRjmuhGixnEaoXXrWJ52Vgqudnf+TaxXXrHn+5urktmTxJUKSeProlTVIxpQUdCQowzjgXm8LuqApqEKDFkB9YgspIvjEVkkV9IocTyikhKxTFzSr7YkrpRfGl/LKlMwmnpEA/KoR1SwnuhJVSCnAuoR5ZRlcCDXA5JiK1fulEI1w3PnTjl8JXGluNL4uqi4qe6s5sJnUmzLlgYpVDM8Gxrk8JXEleJK4+ui4qa6IZI0snrRokMuyrNsaRw6JIevJK4sQGl8XVS6VDdEkuaaNTVxLq8cm8zBTA5t8+bNQX19fWYJDG557Mpcc3XFK1860vjmy0ec8FQ3RFwjWorNmnVSCtUMz5Mn3fFlo7Nq1arg4MGDmbW4ueWxq8bIJddyFJI0vi40SXVDJGmBqc5OOf1ZrHgu+98eeeSRt6xKyKEXDHdhLrm64BOVhjS+UfmxOZ/qhshGAI2TvAL5+kTyhSfPWBm4ViDVDZGkhdFqauRM0GUldDmheMaMGTnrdb7wnJELBLrkWuA2zk5J4+si46luiPjdMCm2f3+1FKoZntXV7vhu2LAh8225sAD81hzDXZhLri74RKUhjW9UfmzOp7oh4scIpVhjo+Op/yXO+HGHM29XrFgRbNy4Mairqwu4zji3PGa4C3PJ1QWfqDSk8Y3Kj835VE/xaGxs7LOdt4P6n6hVVZ0LurrsF3JLetIrvzwrZeE5SVxZCSXx1SkeFs2GpGftZcuaLXLkT5TmZjl8JXFlCUvj66JWJuwHuMhC/jTiLIyWtEeUPxe5zyTtEeVmpaHDTQH1iCxKXJcBsRBpiFFsH3mHmLzTyyRxZcal8XVRWOoRDaioHpGL6qRpDDcF1COyKHH1iCxEGmIUSb/akriyOKTxHWIVGnSZekQDcqhHNKhe6IEqYKWAekQWMh082INxKfi+tQUskitplNWrd1in73jiu/V9wxF37LDnG74uiX1JXKmPNL4uyjTVHtGIEY19fX3Jfm3C9u0WB19KWrZEEl9JXPlHLYmvekQWzXB19TmLWH5E2bdvnx9ELFlI4iuJK+WXxteyyhSMluopHt3dYwtm3qeT06dP94lOJBdJfCVxpfDS+EZWFosIqW6Ixo+XM9eso6PDorj8iSKJrySuLGFpfF3UylQ3RL29crI3ceJEF+VZtjQk8ZXElQUoja+LSifnL3UIuR05sm8IV5X+kqrqqswsc3T0Xdreeeedl/ZNOOP5aufPn/eV2lt4SeJK8tL4vkXwIQSMHsI1Yi7xdWzQ6VOng2D9YBnHVqM/a8ngsNPrEc9Tk7TonCSuLG5pfF1UUZ88onpk6MdAJ3AMeBIwDeUC7PM9/BsDWx5H2ptv+pS9wnTbe9sLR/DsLBcuk2KSuFJTaXxd1AOf/lK/ggzxr3EqwIbmJuCjAF99/R2wCagBnh44jnwlVlEhp7O6YUIDsiXHTpw4IYasJK4UVRpfFxXBp4ZoJjL0PYCLN9Mjeg6YB9wM0DP6EsCBQV8GOBDzVqCgnTkzpuB5n06+dOoln+hEcpk2bVpkHF8iSOJKzaTxdVHOPjVEbGjuBejzXwncAZjG6FXsh3ueecxGKpetQuA2orLyaDB3bkfw7ne3BddffySYPbszWLKkJZg0qSe4//7d+JjfxWDt2qZMGuvW9Y/A5jHDeZ7xGJ/X8Xqmw/Ruu601mDq1O1i+fA8+q/NmYKZnmDTMlt+z52p7e/fuxeqLXZkveLa3twcLFiwIbqm5JairqAuWXrE0qBpVFXyq/lP9PGasG7TlcH+OtN2zZ0/Q3d0dtLa2Zl7vtrW1BUeOHAk6OzuDlpaWoKenJ9i9e3emf6GpqT9PZvIkj9nvwPOMx/i8jtczHb4uZrpMn/fh/cw0A5OG2fK77MwTj8N5Yr741Q2GMb+MY77hbq4123Lnadu2bVZ5yi6npPL08ssvOyunUucpU1kd/EfPwhebAyJ8/LoG4Kr3TwMPAp8B2OiwkTLGz4DuBdabgNzbhWi82CYlZ7mmePCtWDbzkcHI4CL+DbL1aH1zJTAoUjIHbNikfDdOEleWpiS+aZviQc+M3s/3gUpgEsD+oD8HuoEqIGw8jnyldMUV7NuWYWtmrJFBdIDl9u3bxfCVxJWiSuProiL44hGx4XkdqAZODWTsg9h+DngY+CbwNsA8nh3E/mqAjVcBk+MR5czEen89opx8NXDYKZA2j6gDJXgA+PfAaIAN0gMA+4JeBPgR+7UAP3NhXIf/jf2CJskjWjfQN1QwQx6dNP09HlHKS0USV2ZCGt+8wsc44YtHRMoLgC8B7CNiw8OG5mPAceBa4L8Dc4HXgIeAXwER5qdHxBHTmUGNEewv+63Lgq6TXRGx9LQqkJwCafOIqOR24GaAfUN8VPt3ABshGhudhcB4oHHgGJvC5qtHxMaFndBh8FcwfMx9nxsh82aucAn4cVYSVyomja+LUvbJI3KRn6w0/PSIskhmDiW9KSFhSXwlcZWmbRo9oswfpMv/amvPukyupGlx/I4kk8RXElfWAWl8XdRbnwY0usjPoDS6uiJngQyKn+TBzJkzk7x97HtL4iuJKwtCGt/YlSfHBaluiCor5SxVcfTo0RzF42+QJL6SuLLEpfF1UUtT3RCdPcuRAHY2sgRKxPnaRm1trR3REsaaMiXAmkh2uOOOWuu4ozBO3jZd23jkamuqbTz942hrWwZR8Urw5xd1y/Kdf8c7LuJNFAcFRuMCBgzYxIsT5xin7lqaDx+DPG7eUVpwjvNGErNBnFscrqptPPnjaBsv5fyxU90Q5c+2f2ekzNsyyklahle1NaXm7zbVDRFeLfqrfBazMWPkLFlC6pKWWFFtsyqbh4epbog4fkSKcQkOSTZ9uhy+qq3/NSvVDZGkL6dOmsTB5HJs5045fFVb/+tVqhui3t5e/0tggOHhw4fFcCXRG2+Uw1e19b9qyelEGYKWjY2NfVJmMvvwvfM4XWpcmTLO8IghFF/kJXyDaWOqrY1Kg+PYaqtTPAbrlvPo7Fk5Uzx27dqVMw++Bj7wgBy+qq2vteg3vFLtES1cuLCP6xWr2SkQxyOyS7G0sWx/tUvLwi71tGqrHpFF+fswkM2CZiaKlEdIkx/zgQBz7PNWtfW5dPq5qUfkfxmVjWFaf7XLJmCBG6VVW/WIChS6OaUekVHC/VY9IveamhQlaWs4F7tVj6hYBVN0fVp/tX0oorRqqx6RRe3ixwSlmPkYoRS+/HikFFNt/S+pVHtEksYR8cuo48bxIyXJWZxf7aqqc/iqa7J8bd+aqbbx65SttuoRWWgraWQ1P9csyRYtksNXtfW/ZqV6ioekuWaT46yi5kG9amqa7AELOwqqrZ1OScZKdUN0gaudCbGTJ08KYdpPc9YsOXxVW/+rVqobIkkLYlVUVPhfW0IMOzvl8FVtQwXn6W6qGyJPNVdaqoAqkKVAqhsiSQuj+TBBN84HBGpqkp1QHKdLzQdt4/BNWts49SCrPRnyof1nLoZ8i+QuHMXPRwix6urqxJnGWdBy//54fG1fB5dCBB+0jfMhha6u6qCqyk6JOEMu7FLkV3xtY7qLl2qPiOvQSLHjSXw6oQhxGhtjfPKjiPu4uFSattL4uiijVDdEY8fK+dLrjBkzXJRn2dJ4/nk5fKVpK42vi0qX6obIh74B20Jqbm62jepFvGXL5PCVpq00vi4qZKqneOjCaPGqSCn6GwyDJPuIDIc0bktVZrblpVM8LGqVLgNiIdIQo0haqkLawmjS+A6xCg26TD2iQXIM74NS/bpSVdtf2OFdAvFzX6oysy2vNHtE96I4XgPOAC3ADQBtEbAHeAN4AagDCpoEj2jz5s1BfX198PDDD2e2PJZgEjwiqdoOR4/Itzr/fhA6CLwHYEf6lQPg1/xOAfcAnFvwBPBLoKCxj8hn27RpU9+ECRP4UZxL4DHDk7B+v4W/he5R7vz4pm2p8l+KsmKatoa6m8qvU/wCGXsoR+uyCmE8Z6wSO1z17GoTkGs7d+5cWz0TiVdXV3epAVq9evWlfYYnYXEq9erV22M1WOXOj2/axsn/9u3braPHKbM4cW0J4O/OSUPk0+t7DoN+F3A5sA/gp0SfBMYD84AdgDHz2MbwbGOjRXG2cdZ1R0dH0NbWFhw5ciTo7OwMWlpaAq7cuHv3bowgvRg0NTVlrjfuMI8ZzvOMx/i8jtczHabX2toa8Hvqe/bsCThocseOfmomDbPlyoBclGvv3r1YRKwr4Lo47e3tGZg1cpYuXYpRtFX8DcrwWLduXSaeSYNp8x68F+/Je5cqT1OndgfLl+8J+PHE1av782QewcyWKzNyUbQTJyqCurqu4JZbDgULFrRnwH2GLV26NxPHrOJori1nniZOnBjcdtttwdSpUzO6ceIrGvtB2ho+UeXEsmMZsizNao/mWrN1WU6cEWBb96LKiWURt5xMednkKVNpU/bfNOSHf41sRKYCfBz7ObAB+AbwOBA2nvtwOCB7f968ebYNeyLxwr/ay5cvF+URLV/+mhiPyAdt41Sw1157zTp6HC8nTlxbAvibS51HZBaY/itkrg3oAL4I3Al0A9mzb3h8Gshrvo+s3rBhQ4A+oQz/rVu3ZrY8ZrjvtnXrdK8pStZ2+nS/tS1Fwfv0aNaJDPJxrP8ZpT+3Zn8XDq/pD8r8zz6iqwCG5zXf55qtWLEi2LhxI1znumD+/PmZLY8Z7rvNn8/fCX9NsrZ89B5u5ts4okdRAHcAS4DzwI+AF4EvA+w3Wgn8I/BnwE0A367ltWuuuaYPHX95z/t0gpVv0iQ+jSZnccakzJ3bgX40e74DXWCJZM4HbeNkPA7fOGUWh4NteaV1HNFjEOtlgBOZOJboVwCfU14H7h7Yp+d0HcDxRgUNz7kFz/t08vx5trtyrLJSDl9p2krj66LWxl2PiI9yHMfDQYWlMNbujw4gO/2fIKDg6/rsCyQdS1rEjbqOHZvAojVDLFBp2krjO8RiGXSZTR8RH5W+DXCgYS/ADmK+Pv8p8AjAt11emqQ1q02ntZdC5iDV3t7fyZ7jlHdB0rSVxtdFgRdqiJbiBnxE+iZAT+XzAMNuBx4EXgDeB+wHvgZcDnhlvndWh8U6ceJE+ND7/YYGOXylaSuNr4vKWujR7NO4wcPAj4Fcfvj3Bghcie3HgT8A/stAmBebMWPGeMHDhsS0ad46ljnpv/SSHL7StJXGN2cFiRlYyCNih/A/ALkaofBtjuDgPwJeNUIkKOlLrwcOHCBlMbZ4sRy+0rSVxtdFpS3UELlIP9E04nzPasqUIOCrUJdgmrZ29dXJ98PH+dLEM8/Y842Trq1eceL5oG2p+JZC21KkGZX/OA0R+4c4nudZgI9lYeDQP4uzDEgp1q6Pk6YP4534pQmOeLDByy9vt4rHtOJ8waIUtcgHbePkKw7fOGVmU65JlRd8ACvjY9fHgJ8D/HwDqtcg+71BR54cxFkqNumBYZ5IpjRUgVgKlHtA4wNgdw9wC3AvwIYnDBz6Z3E8oqTZm1ncSfOwvb8kvpK4Un9pfG3rTKF4th7RISTyfuBfCiXm2zn1iHwrEeWTNgXK7RE9DgH5ZqzQ637vNJbkEZl1kbwTMQ8hSXwlcaXc0vjmqSKxgm09Ig7I4QTURoCDHDnAMWy3hg982ZfkEXFYv6SR4JL4SuLKvx1JfMvtEX0N+vwOsBXg0n27soBD/0zSBxa5AqMkk8RXElfWAWl8XdRb20et5bgZX9//Lxc3LVcavi+MFtZh5syZ4UPv9yXxlcSVBS+Nr4vKajuOiMtwcAS1KJO0nMLRo0dFaSuJrySurATS+LqouLYN0Wdxs0eBiS5uWq40Ro+2dfjKxSj/fWpra/Of9PCMJL6SuLKopfF1UT1tG6JP4macdc/BjFyw7NUs4NA/Y6efFJP0ho+aSuIrias0bV39fdm6DFtc3VDTya2ApDdmzIEkvpK4StM2d22OH2rbEP1Z/KSTvwKvFpMnYclA0pIlzJIkvpK4StPWsnpHRiv0aBb3rzhu/EhyxUaQ9GjGjydKMkl8JXFlHZDG10W9LdQQcWDL/cC4iBvNwfn/Bnw6Il7ZT0vqrE76Cx5xC0cSX0lcWQ7S+MatO7niF2qI+OlmzrhnB/XfAv8J4ORXjin6I+DLQBPAr27w9f6TgFcmaWG0w4cPe6VdFBlJfCVxpe7S+EbVFZvzNo9T70VCnGl/A1AHjAf4BTh+6uefgE3AScA7a2xs7LOdyVyq7iSu72JjXF9bkgcnia8krqwrkviWc4rHL6ANPaMFQA1QAUwHPgDQC/KyEQKvQNIUj127OGtGjkniK4kra4A0vi5qrY1H5OI+iaQhadJrIgLpTVWBIhUop0dUJNXkLpc0kM32ETI5NQffWRJfSVypsjS+g2vG0I7UIxrQLek+oqEVn16lCiSrgHpEFvrH8YhGFnp/aHGvXFHipCntVzAO31J8IWXUqMD6iyuf+MQr1nHjfHklV5m7CIujrYv7+ZCGekQDpaAeUemqY6m0LRVj2zedpbq/pHTL7RH9EOL8LlACv6F0svf09JQucccp79y503GKpU1OEt+VK1Xb0taG4lO3bVjO4FZ/A3DU3eeB2YD3FucDi0lnpqGhIWkKse4vie+WLaptrMJNILJtQ7QC3KYCjwHvA/4F2Arwe/cc4OilSRpZfejQIS81zEdKEt9Fi1TbfOXoS7htQ0S+XcBXgd8G5gOvAF8H2ga2c7D1yiSNVJ6cxHd+iygtSXybmiYXkdPyXypJW1fqxGmIzD2nYecugH1GbwJ/C7wN4GJpfwx4YxcuXPCGSxSRkye9HaCek7okvrNmqbY5C9GjQNuGiJ8TWgb8GDgIfBD4zwAf1x4C7gTuBj4DeGOSFsSS1J/FApbEt7OTs5LkmCRtXalq2xCZx68W3HghwMczLv3RDRjbip1Oc1DElh3hZwFOpjV2H3bYALLTnG/wagE1VSBVCmzevDmor68Pbr/99syWx8PFbBuiT0CQK4GPAXwEy2X0f2fmOhEz7K8Rn0uLGJuHHfZF/T7Ah/03gK8AkSZpYTRJE3QpvCS+NTX8XfPb2OisWrUqOHjwYFBTU5PZ8ni4NEa2DdF3UIzlKM17cR82aM+Hqg3f2P09QI+LHtifAB8CLgMK2igOvxVi1dXVQpj205TEd/9+/7V95JFHLn2QYP/+/RmROTOA4cPBbBuicmhRhZs8CjycdTN6RPy6rDE+HvYC+QaHrMK5bcSxY8eCjo6OoK2tLThy5EjQ2dkZtLS0BBzouHv37synfc13xtet40vAIFi7tgkLw18M7r9/N1bK6wmWLGkJZs/uDK6//kjw7ne3BXPndgS33dYaTJ3aHSxfvgd9JW8Gq1f30zNpmC0H0p07dy7Yu3dv0NXVFfCVd3t7ewbcZxjPMc6OHf1pmOH9Zstwrk/Dr39yCdHW1lbrPJk0mEd6h8wz804NqAU1oTbUiOkyfd6H98vHhwMZyZdLVUTlyQx6NHqYLfWibtSPOlJP6kp9qTP1pu7Un+XA8mC50EwaccrpwQd/HVlOVVXngqVL9wZ1dYXLyeTJaGu2xZYT68PatWszHyX4yEc+klmlccmSJcG4ceOKKiebuldMnjKFkrL//ivy86mBPK3H1vQR0TviipBhO4KDm8MBufa5MJqtBQEH9ruH7f3xqGMb1Yt4cfiWQtc4aVZVnY1VtkkIXFdXxyX0Mqiqqrq0z3CfDZz5o1+0+eIRLUBOOFDyL3PkiI9j9JbCxuPT4YBc+5L6MZqbm3NlwdswSXyXLfNf2w0bNgQTJkzIlPeyZXxBHWSOGa5WPgXW4VZ8I3ZsAGx8egD6458Hwq8PZuGYj2aRfURcGM3W4vzCxolre/80x4ujlw9xkyqLTZs29dEDwkTSzJbHvhv+Dp14REjHC+NPwZQQ/gL7W4DLAfYRdQE3AJUAH9meBSJtzpw51uVYqj8AWwLbtm2zjepFvDh8S6Wtbbrr1m3z/tEsXKhxtA1fl8Q+/gidNEQjIv+ak4mwHrd9O8DPGdHuAx4H/hXwE+BB4ARQ0HSp2ILylO3kCF9rWR4FdBmQPMLkCC73MiA5KJQ0aD1SN40Qb/QMMAOgR8TpJZGNEOJceh3Kfd/NvH3xnafhJ4mvedNmuPu+laStKy2F/VbFy7Z6RPH0KlVs9YhKpWzy6abdI3KisKSF0cy4HScZL0MikviacV5lkMXJLSRp6yTDSCTVHpF+YNFVNXlrOnE+Api0R8TBk2fPjn5rJvKEJN1HFEfbPFkoW7B6RBZScwSwFNu3b58UqhmekvjedZdq63vl8mVAY0l0Gjt2rHW6pViXLE6a06dPt+ZaqohxvrZx663Trb+MUSq+tulu3WqvbZwys71/3Hg+1IW4nIuNb++vFnunBK6ni2trmJaWqHG+18SJExPlcPy4/e3nz+ccvmT52j5CtbZ2BPX1yXK1VzbIzP1Lui7E4esibqo9IkkLo0mreIcPy/nDlqatNL7aEEUogJGmETH8OX3+/Hl/yFgwqayUw1eattL4WlSXyCip9ogic+9RBEmLuFG2sWMveqReYSrStJXGt7D6dmdT3RBJejQzM6/tii35WO3t/TPFk2cSzUCattL4RpdAdIxUN0RxOqujpSptjBMnrGatlJZEjNQbGuTwlaatNL4xqk3eqKluiMaMGZM3476dmDZtmm+UCvJ56SU5fKVpK41vwYpieTLVDZGkL70eOHDAssj8iLZ4sRy+0rSVxtdFjUz1FI84k15diFlMGuygTLpPK85UDK4jffFisr9jti9FfdA2Tt2QxFeneFiULL+CIMW2b98uhWqG55o1cvhK01YaXxcVVz0iFyqmJI04HpEPWbb1iHzgmlYO6hFZlKwkj0jaYliSFhuTpq00vhZ/ipFR1COKlGj4RFCPaPiUtaucqkdkoaQkj8h86NEiW15EMR889IJMBAlp2krjGyG/1Wn1iKxkKn0kH96UxPGI9K1Z6eqED3XBNnfqEVkoJekDi/zUsyS77z45fKVpK42vi3qb7EAQFzkokEachdEKJFOWUzNnzizLfVzd5Lnn5PCVpq00vi7qVKobIknLKRw9etRFeZYtjeuuk8NXmrbS+LqodKluiEaPlrMAZW1trYvyLFsazc1y+ErTVhpfF5Uu1Q0RO/2kmKQ3fNT0iivkjFqXpq00vi7+xlLdELkQqFxpJD3PjPkcGaM29PbGiFwCEeMscu+DtnEkkMY3Tt7yxZXz7JIvBwXC8WqxwFm/TvmwZEkcB/LMmXhLrCQ5HcMHbePUNml84+QtX9xkf9bysXIULunRrLu721Guy5PM9Oly+ErTVhpfFzUu1Q2RpM7qSZMmuSjPsqWxc6ccvtK0lcbXRaVLdUMkaWG0w4cPuyjPsqVx441y+ErTVhpfF5VOTifKEHLb2NjYJ2Umsw/fO4/TpSbpe/I+aBun+kriq1M8LEpW0hSPXbt2WeTInygPPCCHrzRtpfF1UStT7RFJWirWRWEWm0YcjyjuvZJ8axaXq8a3VyCNHtE4ZP8bwEHgNMC1SO8AjC3CDmdaciTdC0AdUNAkDQyT8ghpBNeF0YwS7rfS6oILBXzyiCqRoU8CTwGHgDuB7wLzAb4rbgE+Avw98BhwA/AeIK+pR5RXmpwn1CPKKYsGFlAgjR7RGeR3PdAKcG7GPwD8Zs1C4EMAOyX+B3AWWA9cA1wN5DX1iPJKU/QJ9YiKljBvAsPRI/L59f1klFQDwAZoHrADMMZGix4Sw7NtFQK2EadOnQo6OjqCtra24MiRI0FnZ2fQ0tIS9PT0BLt378bncC4GZjU8U/g8ZjjPMx7j8zpez3SYXmtra8BBZ1w3hm84duzop2bSMNudO3cG586dC/bu3Rt0dXUFhw4dCtrb2zPgPsN4jnHMkiXmWrNl2rwH78V78t6lytPUqd3B8uV7Ar4RW726P0+mwTHblSt3BlVV54KDB6uCurqu4JZbDgULFrRnwH2GLV26NxOHcWnm2iTyRM0qKyudlRPLlGbyYrYuy2n8+PFlrXvF5CkjRor/4/yBnwBfH8gj+44eH9g3m59j58PmINd27ty5fVLs1VdfTZxqELBL2Q4rV75qHZdpJmk+aBsn/5L44u+OP/pFm48eETl9B+gF1gzkkH1EVQP7ZsPj0+Yg17aioiJXsJdhDQ10/uTYli1y+ErTVhpfF7XWt4aInef0fvhYdjdwHqDx8Yx9QsbYsX0VwPC8JmlkNR/VJNmiRXL4StNWGl8X9da3huiryNQc4ANATyiDP8D+OwE2TnRz/hR4FSi4cLKkuWaT46xrgYwnbU1N/K2QYdK0lcbXRS3wqSHiuKDVwALgGMDHMWIF8DrARmgD0AlcB9wLFLQLFy4UPO/TyZMnT/pEJ5LLrFly+ErTVhrfyMpiEWG0RZxyReFAxkLjmth5fXUcMpIWmJLUn8Uy6OyU0/8mTVtpfOP8TeaL65NHlI+jhqsCqkDKFUh1QyRpYTRJE3T5N1FTw3GlMkyattL4uqgFqW6IRo0a5UKjsqRRXV1dlvu4usn+/XL4StNWGl8XdSrVDRFHJEux48ePS6Ga4dnYKIevNG2l8XVRcX3qrHaRn0FpmGkTgwI9PZgxY0bizDiCwLY9fP55e75Jj0zwQdspU+y1raqagek/dtWBX16J89EDm1RLkWbUfVPtEUl61m5ubo4qq5KfP4ZBE3YTPILgZz9rto7LdJM0H7S1beCp07Jl9nXBdSPE+5ciTaZbyAq9Li90nYhzugyIiGIaFiRLucRKsgKOeAX3f1exHFLtEekyIMVWj/zXm1nn+WP4c0YSV6pmVivwR8HSM1GPqPQa6x1UgUA9osKVQD2iwvqU7ay0X21JfCVxZYVTj6hsf3bluZH2EZVHZ71LtALqERXWKNUeEVdYdGG1VVVwrUfkBc8Xa2aVx2LTKdf1kvhK4sryM6tjlqssfbhPqhsiV5MHO0+fDrAcYV7wfLE2b16uVW+LTbV017vmu3nz5qC+vj7gRGVueezKXHN1xStfOk8/Lasu5MtHnPBUN0RcC1qK7du3TwrVDE+XfNnorFq1CutgH+Sispktj101Ri65lqOQ7rpLVl1woUmq35pde+21fWZx/GLE4mMZPaJ8RhH5B1SMcZH3iRMnFpNEWa91yZceEBuhbKurq8t8LCA7PO6xS65x723iowpZGz9i0NYmpS7oOKLIgpU014xf5pBkLvnmWxo1X3hcnVxyjXvvocSfP19WXRhKHrOvSfWjmaSF0SR5Q6xELvnmmwuWLzy7Ekcdu+QadS8X5w8fluINuchtfxqpboiKfVxyJ3N0SufPm+8ERMf1IYZLvhs2bAgmTJgwKFs8ZrgLc8nVBZ+oNCorZdWFqPzYnE91Q2QjgC9xJC3iRs1c8l2xYkWwceNGfJyxLjNEglseM9yFueTqgk9UGmPH8kPHw8tSvQyIq0ezmssuC0YUeEXP88VatkdQbHqlvt41XzY6rhqe7Ly75pqdvuvj9vbB3qHr9H1ML9UekavO6hNYHIaPefnA88XaiRMnik2irNdL4iuJKwuxoUFWXXBR8VLdEI0ZM8aFRmVJY9q0aWW5j6ubSOIriSvL56WXZNUFF3Uq1Q2RpC+9HjhwwEV5li0NSXwlcWUBLl4sqy64qHQxhlm5uF1505A06ZUdqq76tMqhsiS+PnCNM6Bx5MiLeBkgxUfQAY2Rf2+SFkbbvn17ZH58iiCJrySuLOM1a2TVBRf1Uj0iFypqGqpAhAJxPKKIpDw7rR5RZIFI8oikLd4lia8PXPllDFvThdFslRIST1IfkRBJleYQFVCPqLBwMdrpwgn5eFaSR+RilYByloEkvpK4sgzXrm0qZ1F6cS/tI/KiGPqnTOhbs9IUhr41K42u/alqH1GkupI+sLhnz57I/PgUQRJfSVxZxvfdJ6suuKiXqX40k/TJ6ZkzZ7ooz7KlIYmvJK4swOeek1UXXFS6VDdEkpZ/OHr0qIvyLFsakvhK4soCvO46WXXBRaVLdUM0erScxQVqa2tdlGfZ0pDEVxJXFmBzs6y64KLSSWqIWDo/AM4AB4H7gILGTkopJukNHzWVxFcSV2p7xRVvcDOsTI7LEAR/jZLpBSYDC4B/BHYAuwDxJumNGcWWxFcSV2rb2yvJPyDj4k1KjiuR1buBPwG6gZ8BPwJ+H8hr/PqGFJO0ZAk1lcRXEldqe+aMnOVryNeFSfGIGpDZN4HmUKbpDd0UOja7q7BDBPjC5zk0Rr82JzzfTgI/SZ9vkMRXEtdg9+5AEt93eP535ZTeDUjtWFaKf4jjF7PCsg+3ZQd4fCyJK2WUxFcS12GprZRHs26UTvYH5nlc/LeeWexqqoAqkKgCUhoiPpLxMXJ2SK1rsJ+KjupQnnRXFRiWCowSkmt+6GkecAvwHPDbwGPAx4HXgUL2SqGTnp2TxJXSSeIriatq69kfZpgOxxH9EDgDHAIixxEhjpoqoAqoAqqAKqAKqAKqgCqgCqgCqoAqoAqoAqrA8FUg9ry0Mko1Dvf6BsD5chx+wE823AEYW4QdLkjDCUcvAHWAD8Y3lmeBTSEy7KdjPthvx/476u6D3QsSrwHk1QJwHBrNN23rwenHQCfAcXJPAmaQMacxsYOd9YBbHpfT1uBmHH91DngKCFshHVm/vwl0AczTw8Cwte8i538DTAR+BzgF8K2bD8bpKuuBeoDDJ34XYINUD3BELbneA1QATwC/BHyw/wkSPwVMQ0Q9yftGgDo/AzwLJG3vBwE2ju8BqO+VA/BRWzZCTwEs6ynATmAtMBZgHj4B8A+bYTxmeLnsQ7jRB4GvAk8BxqJ0/AIisp7UAHOAY8BiYNgZ/9B7gYZQzr+D/cdDx77tvgpCdwOrgF+EyDEvPcDVobAkdulhfA9YD5iG6PPYfwYwdhV2qPtlJiChLfV7KMe9VyGM54z5oC29tjsNIWyfAL4O3AYcAUYAxvimOIk/6M/hvk8ZEthG6XgUccjf2GPYedYc5NvyFyNt1oAM5ZqXNs/TjE4GL3LeBZDjDsCYebRIkjtHsD8KZLvY2Vz5CJT9A2DyUa7tKNzoXcDlwD7gMPAkMB7I5uuDtl8CLzbyEwB6bnxEfw4gV/449QHGeJxkPTA8CulIL2gqEK7D3I/kncaGiI8JXUDYTuEg6V/qMB+zPwY7m4GngT0AuZNr2JLmzl809mnxjzpsPnJlo05NlwHsF2K/yrXAZwAf+W4FL/6Rsr5S323ADwEfuYJWxgpx4zlauA5b1d80NkTdEIK/4mHjMfszfDJq/x2AXsSaAWK+cecf8vuAvxzgF974xpXc+BhL+yugDegAvgjw8cc3vix/ej/fByqBSUAN8OeAb1xB6ZIV4sZztPDfn9XfXhobomYIMRqYTUUGzLd5aXz2p5fBX3D2DXEKC42PZ+RqjBX0KoDhSdjNuGk9cAg4BvwxQL5NQDbXWQhjxyr1T8r49omeRfiRxuxn801a21rwnAE8CfDN1P8DvgWw0STXfw2E+4h4nFQ9wK0vWSEdqT9/AMJ1mPs+8L6UgXLusHPsuwAr2/UA3cN5gC/2NRD5JWBcWcPrcuyQK//YKwD+OjJeUsa+iykh/AX2twDkST35SMFHIOq8CaDuSdujIPAycAVAD+OnAB8vfdMWlIL9wKcB/nBWAz8AngHGAgeBjwNs3NcMHDO8XEZOrINfAOi5c59hUTo+jjj/DFD7qwE2TIuBYWn8teGz9hmAv+b3Ab5YHYjwV/osQFfWYAX2ae8D9gB8zHgRqAd8sfUgsilEhrpSX+r8dwB1T9rYR/QV4CRwDPgyUAHQfNOWj74vAp1AB/A9gF4y7VqA44dYD+iB8ricth43Yz0Ng2G0Qjqy4TTjiI5j/2FeoKYKqAKqgCqgCqgCqoAqoAqoAqqAKqAKqAKqgCqgCqgCqoAqoAqoAqqAKqAKqAKqgCqgCqgCqoAqoApEKHATznM09qiIeFGn/w8icCComiqgCqgCsRVgA/JA7KveesEHELQXGPnWUxqiCqgCqkB+Bd6LU13A+PxRrM/Qo+JI6iXWV2hEbxTQXw9vikI0Ec4/4pyiz4ZywUmanMZyTygse5dTRJ4HOI3B2Hrs/Bqgl9QKcPrItwDOs/oo8H8BThD9IhCuvxdwzBUPfw9QUwVUgWGqwO3Idy/wbwB6OLsANiCFbAdOPpIVYT2OOf/u+8A7AabL4+cApjcHWAqcB7L7hP4IYWyo1FQBVWAYK/Al5J0zytlgcIXEiUAh48TUB7MirMcxPaTfCoVvwf7rQHj2+Ys4fhII27/FwUVgdDhQ9/1XIOza+s9WGfquwKdAkF7RHwBcTYCeTCGj58THt2w7hIBTocDj2G8GmLYxhnGpj7CxARsBmNn24XO677EC2hB5XDgCqdWD89uAPmAWEGVc+qImRyQ+doWN6eUKy66/XIaEDVtUAxhOW/c9UEBdWA8KISUUuA7QM8CPgJcArgn0c4DeTT77FU7MzXdyCOHsU2oawnV6ScIKaEOUcAGk6PaPIS98e7YI4GPVYuDbwK0A+21y2T8h8KFcJ4YYxtUi2amtJkyBbNdWGH2l64kCN4HHfwDYN8QOaD5KfRigt8N+o3y2CScagHn5IsQIvxJx3wt8K8Y1GlUVUAVUgYwCj+P/bzjQ4gmksdFBOpqEKqAKDEMFqpDnR4BRReb9k7h+cpFp6OWqgCqgCqgCqoAqoAqoAqqAKqAKqAKqgCqgCqgCqoAqoAqoAqqAKqAKqAKqgCqgCqgCqoAqoAoMVwX+P4dOincZIC/EAAAAAElFTkSuQmCC)"
      ],
      "metadata": {
        "id": "HYaoIFcXnuvP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Required Packages:**"
      ],
      "metadata": {
        "id": "upOMtVbJpk-b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import math\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import torch.nn.functional as F\n",
        "import random\n",
        "from collections import namedtuple\n",
        "\n",
        "import warnings\n",
        "\n",
        "# warnings.simplefilter(\"error\")\n",
        "warnings.simplefilter(\"ignore\", UserWarning)"
      ],
      "metadata": {
        "id": "ex5Gp6XZpn-A"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Environment:**"
      ],
      "metadata": {
        "id": "dBTHFMN2p6P2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Environment parameters:**"
      ],
      "metadata": {
        "id": "d-UlLNBdtrlP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "First, we define the parameters of the environment. As the robot's task is to collect data of IoT sensors, we need to know the communication channel parameters as well as the data collection reqirements. Below, these parameters are given.  "
      ],
      "metadata": {
        "id": "fsfWfg7xtimL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sigma2 = 10**-14 # noise power\n",
        "W = 20* (10**6) # channel Bandwidth\n",
        "T_s = 2 # Duration of each time slot\n",
        "tau = 0 # Trasmission time of each sensor\n",
        "P_max = 0.1 # maximum transmit power of each sensor\n",
        "num_sensors = 5\n",
        "data_req = (0.4 * 10**9 )* np.ones(num_sensors) # Data: 0.4 GB for each sensor\n",
        "thr = 10 # threshold distance to the terminal for making decision of the done flag\n",
        "B0 = 10000000000 # Initial energy level\n",
        "v = 5 #max velocity of the UAV\n",
        "H = 10 # Altitude of the UAV\n",
        "dt = 2"
      ],
      "metadata": {
        "id": "xG9Bxb4UqN0t"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Environment class:**"
      ],
      "metadata": {
        "id": "nSuWVYR4tzFT"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OCBSZb-NzwO6"
      },
      "source": [
        "In the next section, we define our environment as a class. We have to make sure that the type of the state is numpy ndarray."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QHjRye3-zB_u"
      },
      "source": [
        "class IoT_env(object):\n",
        "  def __init__(self, num_sensors, initial_position, X_max, Y_max, num_actions):\n",
        "    #Initial state of the system:\n",
        "    self.state0 = np.zeros((3,11,11))\n",
        "    self.state0[0][9][1] = 1 #the UAV position\n",
        "\n",
        "    self.Obstacle_x = [0 , 1, 1, 1, 1, 2 ,4 ,4 ,4 ,4 ,5, 5, 5, 6, 8,8 ,8 , 9, 9, 9,9, 10, 10]\n",
        "    self.Obstacle_y = [2, 6, 5, 2,1 , 1, 8,7, 4, 3, 7,6 , 3, 6, 10, 9, 3, 10, 9, 4, 3, 10, 9]\n",
        "\n",
        "    for i in range(len(self.Obstacle_x)):\n",
        "      self.state0[1, self.Obstacle_y[i], self.Obstacle_x[i]] = 1\n",
        "\n",
        "    self.state0[1][1][9] = 1 #the position of the Terminal\n",
        "    self.state0[1][2][2] = 0.5 # position of the charging station\n",
        "     \n",
        "    #position of the noon-complete sensors and the percentage\n",
        "    self.state0[2][4][2] = 1\n",
        "    self.state0[2][1][5] = 1\n",
        "    self.state0[2][9][5] = 1\n",
        "    self.state0[2][4][6] = 1\n",
        "    self.state0[2][6][8] = 1\n",
        "    \n",
        "    self.sensors_XY = {0:[50.,10.], 1: [20., 60.], 2: [80.,40.], 3: [60.,60.], 4:[50.,90.] }\n",
        "    self.X_max = X_max #range of X: X_max, the min is 0\n",
        "    self.Y_max = Y_max #range of Y: Y_max, the min is 0\n",
        "    self.Collected_Data = np.zeros(num_sensors) # Data collected from each sensor\n",
        "    self.vector_state0 = np.asarray([10.,10.]) #initial state\n",
        "    self.Is_Terminal = False\n",
        "    self.vector_agentState = np.copy(self.vector_state0) # state of the agent\n",
        "    self.agentState = np.copy(self.state0) # state of the agent\n",
        "    self.visited_charger = 0\n",
        "\n",
        "\n",
        "    self.energy_level = B0\n",
        "    #self.TerminalXY1 = np.asarray([10., 10.]) # terminal 1\n",
        "    self.Terminal = np.asarray([10., 10.]) #np.asarray([90., 90.]) # terminal 2\n",
        "    self.doneType = 0 # flag showing type of done!\n",
        "    self.max_episode_steps = 5000\n",
        "    self.steps_counter = 0\n",
        "    self.num_actions = num_actions\n",
        "    #self.actionspace = {0: [0,0], 1:[v,0], 2:[v,v], 3: [0,v], 4: [-v,v], \\\n",
        "    #                    5:[-v,0], 6:[-v,-v], 7:[0,-v], 8: [v,-v]}\n",
        "    \n",
        "    self.actionspace = {0: [0,0], 1:[v,0], 2:[0,v], 3: [-v,0], 4: [0,-v]}\n",
        "    \n",
        "\n",
        "\n",
        "  def reset(self):\n",
        "    self.agentState = np.copy(self.state0)\n",
        "    self.vector_agentState = np.copy(self.vector_state0)\n",
        "    self.energy_level = B0\n",
        "    self.Collected_Data = np.zeros(num_sensors)\n",
        "    self.doneType = 0\n",
        "    self.steps_counter = 0\n",
        "    self.Is_Terminal = False\n",
        "    self.visited_charger = 0\n",
        "    return self.agentState\n",
        "\n",
        "\n",
        "\n",
        "  def step(self, action):\n",
        "    V = self.actionspace[action]\n",
        "    self.vector_agentState[0] += V[0] * dt\n",
        "    self.vector_agentState[1] += V[1] * dt\n",
        "    if self.vector_agentState[0] < 0:\n",
        "      self.vector_agentState[0] = 0\n",
        "    if self.vector_agentState[0] > 100:\n",
        "      self.vector_agentState[0] = 100\n",
        "    if self.vector_agentState[1] < 0:\n",
        "      self.vector_agentState[1] = 0\n",
        "    if self.vector_agentState[1] > 100:\n",
        "      self.vector_agentState[1] = 100\n",
        "\n",
        "    i_x = np.copy(self.vector_agentState[0])/10\n",
        "    i_y = 10 - np.copy(self.vector_agentState[1])/10\n",
        "    self.agentState = np.copy(self.state0)\n",
        "    self.agentState[0][9][1] = 0\n",
        "    self.agentState[0, int(i_y), int(i_x)] = 1\n",
        "    self.energy_level -= self.propulsion_power(V) * T_s\n",
        "    self.steps_counter +=1\n",
        "    self.Is_Terminal = self.isTerminal()\n",
        "    \n",
        "    reward = self.get_reward()\n",
        "\n",
        "    data_ = np.copy(self.Collected_Data)\n",
        "    data_normalized = data_ /(data_req[0])\n",
        "    percentage_data = 1 - data_normalized \n",
        "    for i_ in range(len(percentage_data)):\n",
        "      percentage_data[i_] = max(0,percentage_data[i_])\n",
        "    \n",
        "    self.agentState[2][9][5] = (np.digitize(percentage_data[0] ,[.1,.2,.3,.4,.5,.6,.7,.8,.9,1], right =True))/10\n",
        "    self.agentState[2][4][2] = (np.digitize(percentage_data[1] ,[.1,.2,.3,.4,.5,.6,.7,.8,.9,1], right =True))/10\n",
        "    self.agentState[2][6][8] = (np.digitize(percentage_data[2] ,[.1,.2,.3,.4,.5,.6,.7,.8,.9,1], right =True))/10\n",
        "    self.agentState[2][4][6] = (np.digitize(percentage_data[3] ,[.1,.2,.3,.4,.5,.6,.7,.8,.9,1], right =True))/10\n",
        "    self.agentState[2][1][5] = (np.digitize(percentage_data[4] ,[.1,.2,.3,.4,.5,.6,.7,.8,.9,1], right =True))/10\n",
        "    \n",
        "    return self.agentState, reward, self.Is_Terminal , None\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "  def propulsion_power(self,V):\n",
        "    v = (np.linalg.norm(V))**0.5\n",
        "    Pr_power = 79.85 + 0.01663 * (v**2) + 0.009242 * (v**3) + \\\n",
        "      89.44 * (( 1+ ((v**4)/(4.03**4)))**0.5 - (v**2)/(2*(4.03**2)) )**0.5\n",
        "    return Pr_power\n",
        "\n",
        "  def isTerminal(self):\n",
        "    #if self.agentState[2] < 0:\n",
        "    #  self.doneType = -1\n",
        "    #  return True \n",
        "    Distance2Terminal = np.linalg.norm(np.subtract(self.vector_agentState , self.Terminal))\n",
        "    #d_ = np.greater(self.agentState[-num_sensors : ] , data_req ) \n",
        "    d_ = np.greater(self.Collected_Data , data_req ) \n",
        "    if d_.all() == True and Distance2Terminal**0.5 == 0 and self.energy_level > 0: ###self.agentState[2] > 0 :\n",
        "      self.doneType = 1\n",
        "      return True\n",
        "    #elif d_.all() == True and Distance2Terminal_2**0.5 <= 10 and self.agentState[2] > 0 :\n",
        "      #self.doneType = 2\n",
        "      #return True\n",
        "    else:\n",
        "      return False\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "  def get_reward(self):\n",
        "    ch, dist = self.channel()\n",
        "    reward = 0 \n",
        "    Indicator_ = np.zeros(num_sensors)\n",
        "\n",
        "    if self.vector_agentState[0] ==20 and self.vector_agentState[1] == 80 and self.visited_charger ==0:\n",
        "      self.visited_charger = 1\n",
        "      #reward = 0\n",
        "      self.energy_level = 100000000\n",
        "\n",
        "    i_x = int(np.copy(self.vector_agentState[0])/10)\n",
        "    i_y = int(10 - np.copy(self.vector_agentState[1])/10)\n",
        "\n",
        "\n",
        "    for i in range(len(self.Obstacle_x)):\n",
        "      if self.Obstacle_x[i] == i_x and self.Obstacle_y[i] == i_y:\n",
        "        #print(1)\n",
        "        reward = -20\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "    if not self.Is_Terminal: \n",
        "      self.resourceAllocation()\n",
        "      Distance2Terminal = np.linalg.norm(np.subtract(self.vector_agentState , self.Terminal))\n",
        "\n",
        "      for i in range(num_sensors):\n",
        "        if self.Collected_Data[i] - data_req[i] > 0:\n",
        "          Indicator_[i] = 0\n",
        "        else:\n",
        "          Indicator_[i] = 1\n",
        "\n",
        "      if np.mean(Indicator_) !=0:\n",
        "        reward += - 5* np.sum(Indicator_)\n",
        "      else:\n",
        "        reward += -1\n",
        "      \n",
        "      if self.energy_level < 0:\n",
        "        reward += -100\n",
        "    \n",
        "    elif self.doneType == 1:\n",
        "        reward = 10\n",
        "    return reward\n",
        "\n",
        "\n",
        "  def channel(self):\n",
        "    ch = np.zeros(num_sensors)\n",
        "    D = np.zeros(num_sensors)\n",
        "    for i in range(num_sensors):\n",
        "      D[i] = H**2 + (self.vector_agentState[0] - self.sensors_XY[i][0] )**2 + (self.vector_agentState[1] - self.sensors_XY[i][1] )**2\n",
        "      ch[i] = (10**-3) / (D[i])\n",
        "    Distances = D**0.5\n",
        "    return ch, Distances\n",
        "\n",
        "  def resourceAllocation(self):\n",
        "    E = P_max * np.ones(num_sensors)\n",
        "    channel, Distance = self.channel()\n",
        "    best_link = np.argmax(channel)\n",
        "    if (Distance[best_link]) <= thr:\n",
        "      connected_link = best_link\n",
        "      p_trans = P_max\n",
        "      self.Collected_Data[connected_link] += np.log2(1 + (p_trans * channel[connected_link])/(sigma2))\\\n",
        "        * W * T_s * (1-tau)\n",
        "    "
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **DQN Agent:**\n",
        "\n",
        "In this project, we use a DQN agent to control the robot. In the following, we define our neural network."
      ],
      "metadata": {
        "id": "F3uLV_n_vu7u"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Hyperparameters:**"
      ],
      "metadata": {
        "id": "uKR_SPyfwBOR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "TARGET_UPDATE = 5\n",
        "num_episodes = 200\n",
        "hidden = 128\n",
        "gamma = 0.99\n",
        "replay_buffer_size = 100000\n",
        "batch_size = 128\n",
        "eps_stop = 0.1\n",
        "epsilon = 0.6\n",
        "eps = 0.5\n",
        "Start_epsilon_decaying = 0\n",
        "#End_epsilon_decaying = num_episodes // 1\n",
        "End_epsilon_decaying = 200\n",
        "epsilon_decaying = epsilon / (End_epsilon_decaying - Start_epsilon_decaying)"
      ],
      "metadata": {
        "id": "vblsimTMwH_8"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Replay Memory:**"
      ],
      "metadata": {
        "id": "U0yG0BbewMZm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "Transition = namedtuple('Transition',\n",
        "                        ('state', 'action', 'next_state', 'reward', 'done'))\n",
        "\n",
        "class ReplayMemory(object):\n",
        "    def __init__(self, capacity):\n",
        "        self.capacity = capacity\n",
        "        self.memory = []\n",
        "        self.position = 0\n",
        "\n",
        "    def push(self, *args):\n",
        "        \"\"\"Saves a transition.\"\"\"\n",
        "        if len(self.memory) < self.capacity:\n",
        "            self.memory.append(None)\n",
        "        self.memory[self.position] = Transition(*args)\n",
        "        self.position = (self.position + 1) % self.capacity\n",
        "\n",
        "    def sample(self, batch_size):\n",
        "        return random.sample(self.memory, batch_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.memory)"
      ],
      "metadata": {
        "id": "S9hEYHHlwPZg"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**DQN Agent:**"
      ],
      "metadata": {
        "id": "UEVNT2qdwT74"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0UmgQZtHOyTB"
      },
      "source": [
        "class DQN(nn.Module):\n",
        "    def __init__(self, state_space_dim, action_space_dim, hidden):\n",
        "        super(DQN, self).__init__()\n",
        "        self.hidden = hidden\n",
        "        self.conv1 = nn.Conv2d(3, 16, 4, 1)\n",
        "        self.conv2 = nn.Conv2d(16, 32, 4, 1)\n",
        "        #self.conv3 = nn.Conv2d(32, 64, 3, 1)\n",
        "        self.fc1 = nn.Linear(5*5*32, 64)\n",
        "        #self.fc2 = nn.Linear(hidden, 64)\n",
        "        #self.fc3 = nn.Linear(hidden, hidden)\n",
        "        self.fc4 = nn.Linear(64, action_space_dim)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = F.relu(self.conv1(x))\n",
        "        x = F.relu(self.conv2(x))\n",
        "        #x = F.relu(self.conv3(x))\n",
        "        x = x.view(x.size(0), -1)\n",
        "        x = F.relu(self.fc1(x))\n",
        "        #x = F.dropout(x, p=0.3)\n",
        "        #x = F.relu(self.fc2(x))\n",
        "        #x = F.relu(self.fc3(x))\n",
        "        # x = self.fc2(x)\n",
        "        # x = F.relu(x)\n",
        "        x = self.fc4(x)\n",
        "        return x\n",
        "\n",
        "\n",
        "class DQNAgent(object):\n",
        "    def __init__(self, state_space, n_actions, replay_buffer_size,\n",
        "                 batch_size, hidden_size, gamma):\n",
        "        self.device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "        self.n_actions = n_actions\n",
        "        self.state_space_dim = state_space\n",
        "        self.policy_net = DQN(state_space, n_actions, hidden_size).to(self.device)\n",
        "        self.target_net = DQN(state_space, n_actions, hidden_size).to(self.device)\n",
        "        self.target_net.load_state_dict(self.policy_net.state_dict())\n",
        "        self.target_net.eval()\n",
        "\n",
        "        self.optimizer = optim.Adam(self.policy_net.parameters(), lr=1e-3)\n",
        "        self.memory = ReplayMemory(replay_buffer_size)\n",
        "        self.batch_size = batch_size\n",
        "        self.gamma = gamma\n",
        "\n",
        "    def update_network(self, updates=1):\n",
        "        for _ in range(updates):\n",
        "            self._do_network_update()\n",
        "\n",
        "    def _do_network_update(self):\n",
        "        if len(self.memory) < self.batch_size:\n",
        "            return\n",
        "        transitions = self.memory.sample(self.batch_size)\n",
        "        #print(type(transitions))\n",
        "        #print(transitions)\n",
        "        batch = Transition(*zip(*transitions))\n",
        "        \n",
        "\n",
        "        non_final_mask = 1 - torch.tensor(batch.done, dtype=torch.uint8)\n",
        "        non_final_next_states = [s for nonfinal, s in zip(non_final_mask,\n",
        "                                                          batch.next_state) if nonfinal > 0]\n",
        "        #print(non_final_next_states)\n",
        "        non_final_next_states = torch.stack(non_final_next_states).to(self.device)\n",
        "        state_batch = torch.stack(batch.state).to(self.device)\n",
        "        action_batch = torch.cat(batch.action).to(self.device)\n",
        "        reward_batch = torch.cat(batch.reward).to(self.device)\n",
        "\n",
        "        self.optimizer.zero_grad()\n",
        "        state_action_values = self.policy_net(state_batch).gather(1, action_batch)\n",
        "\n",
        "        next_state_values = torch.zeros(self.batch_size).to(self.device)\n",
        "        next_state_values[non_final_mask] = self.target_net(non_final_next_states).max(1)[0].detach()\n",
        "        expected_state_action_values = reward_batch + self.gamma * next_state_values\n",
        "\n",
        "        # Compute Huber loss\n",
        "        loss = F.smooth_l1_loss(state_action_values.squeeze(),\n",
        "                                expected_state_action_values)\n",
        "\n",
        "        # Optimize the model\n",
        "        loss.backward()\n",
        "        for param in self.policy_net.parameters():\n",
        "            param.grad.data.clamp_(-1e-1, 1e-1)\n",
        "        self.optimizer.step()\n",
        "\n",
        "    def get_action(self, state, epsilon):\n",
        "        sample = random.random()\n",
        "        #print(state)\n",
        "        #print(state.shape)\n",
        "        \n",
        "        if sample > epsilon:\n",
        "            with torch.no_grad():\n",
        "                state = torch.from_numpy(state).float().unsqueeze(0).to(self.device)\n",
        "                q_values = self.policy_net(state).to(self.device)\n",
        "                return torch.argmax(q_values).item()\n",
        "        else:\n",
        "          return random.randrange(self.n_actions)\n",
        "\n",
        "    def update_target_network(self):\n",
        "        self.target_net.load_state_dict(self.policy_net.state_dict())\n",
        "\n",
        "    def store_transition(self, state, action, next_state, reward, done):\n",
        "        action = torch.Tensor([[action]]).long()\n",
        "        reward = torch.tensor([reward], dtype=torch.float32)\n",
        "        #print(next)\n",
        "        next_state = torch.from_numpy(next_state).float()\n",
        "        state = torch.from_numpy(state).float()\n",
        "        self.memory.push(state, action, next_state, reward, done)\n",
        "\n",
        "\n"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Training Loop:**"
      ],
      "metadata": {
        "id": "9_-kNzuQwZH8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "First, let's create the environment and the agent:"
      ],
      "metadata": {
        "id": "vx8PguGawkvd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "n_actions = 5\n",
        "num_sensors = 5\n",
        "state_space_dim = 2\n",
        "starting_position = [10, 10]\n",
        "env = IoT_env(num_sensors, starting_position, 100, 100, n_actions)"
      ],
      "metadata": {
        "id": "jSSyjxdGwnpC"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "agent = DQNAgent(state_space_dim, n_actions, replay_buffer_size, batch_size,\n",
        "                 hidden, gamma)"
      ],
      "metadata": {
        "id": "zj2tGGOmxKrK"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training:**"
      ],
      "metadata": {
        "id": "kLB2cVJHwkqP"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GhR3J-8X8Gj6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "bb1bf799-ad85-49c4-a2c8-b208539a631c"
      },
      "source": [
        "random.seed(20)\n",
        "env.reset()\n",
        "\n",
        "\n",
        "# Training loop\n",
        "cumulative_rewards = []\n",
        "Num_steps = []\n",
        "counter_reach_goal = 0\n",
        "last50_rewards = []\n",
        "last50_steps = []\n",
        "\n",
        "\n",
        "for ep in range(num_episodes):\n",
        "    # Initialize the environment and state\n",
        "    #print('training started ...')\n",
        "    Collected_data_time = {0:[], 1:[], 2:[], 3:[], 4:[]}\n",
        "    state = env.reset()\n",
        "    done = False\n",
        "    eps -= epsilon_decaying\n",
        "    epsilon = max(0.01, eps)\n",
        "    cum_reward = 0\n",
        "    counter = 0\n",
        "    number_of_steps_taken_to_terminal = 0\n",
        "    #print(\"episode number: \",ep)\n",
        "    while not done and counter < env.max_episode_steps:\n",
        "        # if ep % 100 == 0:\n",
        "        # env.render()\n",
        "        # Select and perform an action\n",
        "        action = agent.get_action(state, epsilon)\n",
        "        #print(action)\n",
        "        #action_ = action_dict[action]\n",
        "        next_state, reward, done, _ = env.step(action)\n",
        "        #if counter%10 ==0:\n",
        "        #print(next_state)\n",
        "\n",
        "\n",
        "        cum_reward += reward\n",
        "        \n",
        "        agent.store_transition(state, action, next_state, reward, done)\n",
        "        agent.update_network()\n",
        "\n",
        "        state = next_state\n",
        "        for i_ in range(num_sensors):\n",
        "\n",
        "          Collected_data_time[i_].append(env.Collected_Data[i_])\n",
        "\n",
        "        counter +=1\n",
        "        number_of_steps_taken_to_terminal  += 1\n",
        "    \n",
        " \n",
        "\n",
        "\n",
        "    #print(state)\n",
        "    if done:\n",
        "      #print(state)\n",
        "      print('number of steps taken by the agent: ', number_of_steps_taken_to_terminal)\n",
        "    Num_steps.append(number_of_steps_taken_to_terminal)\n",
        "    ###print(env.agentState[-5:])\n",
        "    ###print(env.Collected_Data)\n",
        "    \n",
        "    ###if env.doneType != 0:\n",
        "      ###print(\"Type of Terminal done flag: \" ,env.doneType)\n",
        "    cumulative_rewards.append(cum_reward)\n",
        "    #if ep >= 50:\n",
        "    #    last50_rewards.append(np.mean(cumulative_rewards[ep - 50:ep]))\n",
        "    #    last50_steps.append(np.mean(Num_steps[ep -50: ep]))\n",
        "    ###print(\"episode: %d: reward: %6.2f, epsilon: %.2f\" % ( ep, cum_reward, epsilon))\n",
        "    print(\"episode: %d: reward: %6.2f\" % ( ep, cum_reward))\n",
        "    print(\"**********************************************\")\n",
        "\n",
        "    # Update the target network, copying all weights and biases in DQN\n",
        "    if ep % TARGET_UPDATE == 0:\n",
        "        agent.update_target_network()\n",
        "\n",
        "# env.close()\n",
        "plt.plot(range(num_episodes), cumulative_rewards, label='cumulative rewards', color='b')\n",
        "#plt.plot(range(50,num_episodes), last50_rewards, label='last 50 episodes', color='r')\n",
        "plt.legend(loc=2)\n",
        "plt.savefig('R.eps',format = 'eps')\n",
        "plt.show()\n",
        "\n",
        "\n",
        "plt.plot(range(num_episodes), Num_steps, label='Number of steps taken to reach terminal', color='b')\n",
        "#plt.plot(range(50,num_episodes), last50_steps, label='last 50 episodes', color='r')\n",
        "plt.legend(loc=2)\n",
        "plt.savefig('S.eps',format = 'eps')\n",
        "plt.show()\n",
        "\n"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "number of steps taken by the agent:  3423\n",
            "episode: 0: reward: -34217.00\n",
            "**********************************************\n",
            "episode: 1: reward: -42451.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  2104\n",
            "episode: 2: reward: -29638.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  477\n",
            "episode: 3: reward: -5909.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  264\n",
            "episode: 4: reward: -2846.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  2169\n",
            "episode: 5: reward: -9654.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  1926\n",
            "episode: 6: reward: -6481.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  684\n",
            "episode: 7: reward: -4999.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  252\n",
            "episode: 8: reward: -1850.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  980\n",
            "episode: 9: reward: -9711.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  665\n",
            "episode: 10: reward: -3316.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  1860\n",
            "episode: 11: reward: -12798.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  1606\n",
            "episode: 12: reward: -10290.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  652\n",
            "episode: 13: reward: -3954.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  961\n",
            "episode: 14: reward: -4588.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  987\n",
            "episode: 15: reward: -7417.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  498\n",
            "episode: 16: reward: -4772.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  406\n",
            "episode: 17: reward: -2207.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  245\n",
            "episode: 18: reward: -1209.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  584\n",
            "episode: 19: reward: -1778.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  872\n",
            "episode: 20: reward: -4200.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  490\n",
            "episode: 21: reward: -3355.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  498\n",
            "episode: 22: reward: -1857.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  185\n",
            "episode: 23: reward: -1719.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  223\n",
            "episode: 24: reward: -1606.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  184\n",
            "episode: 25: reward: -1297.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  1149\n",
            "episode: 26: reward: -4099.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  80\n",
            "episode: 27: reward: -578.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  140\n",
            "episode: 28: reward: -870.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  733\n",
            "episode: 29: reward: -2896.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  529\n",
            "episode: 30: reward: -2855.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  591\n",
            "episode: 31: reward: -1957.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  158\n",
            "episode: 32: reward: -1445.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  98\n",
            "episode: 33: reward: -813.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  304\n",
            "episode: 34: reward: -1344.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  143\n",
            "episode: 35: reward: -831.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  495\n",
            "episode: 36: reward: -1577.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  395\n",
            "episode: 37: reward: -1890.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  384\n",
            "episode: 38: reward: -1458.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  200\n",
            "episode: 39: reward: -1390.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  183\n",
            "episode: 40: reward: -830.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  55\n",
            "episode: 41: reward: -561.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  305\n",
            "episode: 42: reward: -1558.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  301\n",
            "episode: 43: reward: -1158.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  70\n",
            "episode: 44: reward: -1029.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  101\n",
            "episode: 45: reward: -1095.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  679\n",
            "episode: 46: reward: -2456.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  440\n",
            "episode: 47: reward: -1370.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  283\n",
            "episode: 48: reward: -1660.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  636\n",
            "episode: 49: reward: -1676.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  46\n",
            "episode: 50: reward: -636.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  848\n",
            "episode: 51: reward: -1719.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  211\n",
            "episode: 52: reward: -1110.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  389\n",
            "episode: 53: reward: -1277.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  89\n",
            "episode: 54: reward: -673.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  92\n",
            "episode: 55: reward: -1279.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  404\n",
            "episode: 56: reward: -1918.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  52\n",
            "episode: 57: reward: -826.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  1910\n",
            "episode: 58: reward: -3479.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  334\n",
            "episode: 59: reward: -1159.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  600\n",
            "episode: 60: reward: -1363.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  65\n",
            "episode: 61: reward: -755.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  74\n",
            "episode: 62: reward: -768.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  341\n",
            "episode: 63: reward: -945.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  548\n",
            "episode: 64: reward: -1271.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  81\n",
            "episode: 65: reward: -656.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  198\n",
            "episode: 66: reward: -1069.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  617\n",
            "episode: 67: reward: -1774.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  91\n",
            "episode: 68: reward: -636.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 69: reward: -441.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  66\n",
            "episode: 70: reward: -568.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  371\n",
            "episode: 71: reward: -1252.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  104\n",
            "episode: 72: reward: -785.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  81\n",
            "episode: 73: reward: -876.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  446\n",
            "episode: 74: reward: -1419.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  385\n",
            "episode: 75: reward: -1388.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  123\n",
            "episode: 76: reward: -873.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  61\n",
            "episode: 77: reward: -642.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  95\n",
            "episode: 78: reward: -638.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  101\n",
            "episode: 79: reward: -782.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  44\n",
            "episode: 80: reward: -495.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  146\n",
            "episode: 81: reward: -920.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  90\n",
            "episode: 82: reward: -599.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  58\n",
            "episode: 83: reward: -664.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 84: reward: -386.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 85: reward: -463.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  50\n",
            "episode: 86: reward: -624.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  56\n",
            "episode: 87: reward: -467.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  71\n",
            "episode: 88: reward: -825.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  46\n",
            "episode: 89: reward: -583.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  45\n",
            "episode: 90: reward: -462.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  314\n",
            "episode: 91: reward: -964.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  72\n",
            "episode: 92: reward: -591.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  39\n",
            "episode: 93: reward: -531.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  56\n",
            "episode: 94: reward: -747.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 95: reward: -454.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 96: reward: -394.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 97: reward: -471.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  182\n",
            "episode: 98: reward: -639.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  51\n",
            "episode: 99: reward: -629.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  43\n",
            "episode: 100: reward: -508.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  41\n",
            "episode: 101: reward: -473.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  44\n",
            "episode: 102: reward: -453.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  36\n",
            "episode: 103: reward: -335.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  63\n",
            "episode: 104: reward: -644.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  35\n",
            "episode: 105: reward: -366.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 106: reward: -513.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  75\n",
            "episode: 107: reward: -632.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  50\n",
            "episode: 108: reward: -612.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  36\n",
            "episode: 109: reward: -433.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  118\n",
            "episode: 110: reward: -569.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  43\n",
            "episode: 111: reward: -390.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  44\n",
            "episode: 112: reward: -399.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  45\n",
            "episode: 113: reward: -515.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  102\n",
            "episode: 114: reward: -953.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 115: reward: -461.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  51\n",
            "episode: 116: reward: -370.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  38\n",
            "episode: 117: reward: -391.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  36\n",
            "episode: 118: reward: -385.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  41\n",
            "episode: 119: reward: -413.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  40\n",
            "episode: 120: reward: -415.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  50\n",
            "episode: 121: reward: -644.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 122: reward: -336.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 123: reward: -351.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  39\n",
            "episode: 124: reward: -393.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 125: reward: -381.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  51\n",
            "episode: 126: reward: -342.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 127: reward: -336.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  44\n",
            "episode: 128: reward: -392.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  41\n",
            "episode: 129: reward: -340.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 130: reward: -346.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  169\n",
            "episode: 131: reward: -1640.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 132: reward: -341.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  42\n",
            "episode: 133: reward: -417.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 134: reward: -331.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 135: reward: -371.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  35\n",
            "episode: 136: reward: -421.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 137: reward: -331.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 138: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 139: reward: -351.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  35\n",
            "episode: 140: reward: -354.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 141: reward: -331.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  38\n",
            "episode: 142: reward: -431.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 143: reward: -323.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 144: reward: -361.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  36\n",
            "episode: 145: reward: -327.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 146: reward: -381.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  37\n",
            "episode: 147: reward: -376.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 148: reward: -341.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 149: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 150: reward: -361.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 151: reward: -351.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 152: reward: -331.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 153: reward: -323.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 154: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 155: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 156: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 157: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 158: reward: -325.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 159: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 160: reward: -361.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 161: reward: -323.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 162: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 163: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 164: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 165: reward: -361.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 166: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 167: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 168: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 169: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 170: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 171: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  31\n",
            "episode: 172: reward: -322.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 173: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 174: reward: -325.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 175: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  36\n",
            "episode: 176: reward: -327.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  216\n",
            "episode: 177: reward: -535.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  166\n",
            "episode: 178: reward: -477.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 179: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  35\n",
            "episode: 180: reward: -326.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 181: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  38\n",
            "episode: 182: reward: -329.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 183: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 184: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 185: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 186: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 187: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 188: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 189: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 190: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  32\n",
            "episode: 191: reward: -351.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 192: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 193: reward: -325.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 194: reward: -341.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  39\n",
            "episode: 195: reward: -330.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 196: reward: -321.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  34\n",
            "episode: 197: reward: -469.00\n",
            "**********************************************\n",
            "number of steps taken by the agent:  30\n",
            "episode: 198: reward: -321.00\n",
            "**********************************************\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The PostScript backend does not support transparency; partially transparent artists will be rendered opaque.\n",
            "The PostScript backend does not support transparency; partially transparent artists will be rendered opaque.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "number of steps taken by the agent:  32\n",
            "episode: 199: reward: -361.00\n",
            "**********************************************\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD4CAYAAAAdIcpQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxU1Zn/8c9DN91NNyA7YrMjuLAjAhmDBuOCxBH3PaKjwclmEmdeiZHfRCdqRqKjoyZRGfUXNGZQM6NiYkZRUfNLxAhGlEWxBZRuEWRvloZent8f5167urq6iu6mN/i+X6961a1zlzp9q/o+9znn3Fvm7oiIiKTTrqUrICIirZ+ChYiIZKRgISIiGSlYiIhIRgoWIiKSUXZLV6Cp9OjRwwcOHNjS1RARaVOWLFmyyd17JpcftMFi4MCBLF68uKWrISLSppjZx6nK1QwlIiIZKViIiEhGChYiIpLRQdtnkUp5eTnFxcWUlZW1dFWkFcnLy6Nv3760b9++pasi0mq1mWBhZlOBe4As4CF3v72+2yguLqZTp04MHDgQMzvgdZS2x93ZvHkzxcXFDBo0qKWrI9JqtYlmKDPLAn4JnAEcC1xiZsfWdztlZWV0795dgUK+YGZ0795d2aZIBm0iWAATgCJ3X+3u+4B5wPSGbEiBQpLpOyGSWVtphioE1iW8LgYmJi9kZjOBmQD9+/dvnprJQWf9eujcGQoKGredjRshOxu6dUu/3KZN8PzzcNppcPjh8O67cMQR0KNH3ets2QKffhrqmfxV370bliwJ752XB5WV0LUrHHYYtG8fngGeew5KSqBLFxgwAHJyYMcOWLcuPLdrB0cdFdZfsyasM3kyHHssfPhhqOeoUeF5wwYYMiQ8ysth4ULYvBnMoFcv6NAhbGfbNhg2DIYPhxdegD17wt/5+eeQnw9Dh0JWFpSWhv23cWPYRr9+oT779oXtx4/k18m/uNC+fVh33z747LNQB6hernPnsM83bgx1g/B+9X3u1g2uvDL8nfv2wRNPhH3UWF26QMeOYf/s3Zt6mTFjYNo0ePbZ8NkNGAAnngi9ezf+/Wtw91b/AM4n9FPEr78O/CLdOscdd5wnW7FiRa2ytqiqyn3nzvCczpo1a3z48OEZl3n88cfd3X3XLvc33njLv/vd7x6oqh5wVVXumza5L1vm/uGH7jt2pF7upptu8jvuuKPGevHD3b2szH3tWve9e8Prd99d4aWl7v/2b+7gnp3tPmWK+//8j/v117ufc4779u1hH91/f5j30ENh3U8+cf/1r93vust961b33//evW/fsB1wHzHCfc4c99JS948/dr/jDvcLLnA/6ST3Y44J7wXugwa533hjmO7Qwf3aa90XLXJfv979iSfcR450nzjR/brr3PPzq7c/cqT7U0+Futx8s3tubvW8VI/zzgt/U7pl6nqMHRve56STGrZ+Uz/Maj4asn5D3/vII92vuKLmZ59cn/o8Mv1ticvk5NRc9n//t+H/Y8BiT3FMbSuZRQnQL+F136jskFRaCqtWQffu4SyiXR2NiRUV1dNVVamXW7t2Lb/97W+ZMuVSSkqge/fx3Hvv+P2uS3k5rFgR6tGlS833W7Mm1LFjxwqys7NxD+VZWTW3UVkZzq4hnM137Bim9+6Fjz6CwsLqs+HPPgtnwx06wK5dYT8MH+5kZTl79rSjoKD29nfvDnXZsyecBXbtGs6cKyrCvK5dwzZHjQrLn39+OMN97DE499zq7Z15ZqjnypXQqRMsWhT+vhkzwvYAfv7zcBY4YgRcfz2UlcHTT8PMmeERGzoU+vQJZ+nTp8O4cXDttfCzn4X3P+wwmDsXHnywep3hw2H7drj3XrjgAjjvvLA/Hn4YLr0U1q6Fm28O27vmmlDvsrLwvG1beJSUwN13h8/t2mvhJz8JWcrHH4fPoWPHcCbetWvY/++/HzKUI4+E++4Lf19JCbzxRnjPk04K9RowIOzjjz4K25kyJWQ8lZVhf5SVhe9f586weHH4zsSZ1KZN0LNn+F6vXh3+1o4dQ0bSs2fYRkn0396+fXjk5FRPx4/s7Oqz/di+feFsOycn7O/spCPetm1hH/buHb6/qVok3dM/A7z2Glx3HSxYED7LOXNg6tTU29tf7uHzLi0N+yI3t/YylZUwf37ITKdPhxNOgE8+gcGDG/6+aSrU8plDpgehuWw1MAjIAZYCw9Ot01ozi7lz5/rIkSN91KhRfvnll7u7+4wZM/yp+NTQ3QsKCtzdfeHChX7iiSf6WWed5YMGDfIf/ehH/pvf/MbHjj3ehwwZ4U8/XeRr16Zef9cu9/nz1/hRRw33PXvc//CHNT5x4pd97NixPnbsWP/zn//s7u4TJ070Tp06+9Cho/2f/ukuf+CBhT5t2te8srLSBwwY4OvXb/XKyrDdI4880j/77DPfuHGjn3vuuT5+/HgfM2a8P/TQ//Oiopp/5z33/F+fPPnv/fjjp/iJJ57oO3fu9EsvvcqHDz/eR48e488884y7u0+bNs1ffHGpv/WW+7BhY3zmzH/1Vavcb7zxX/xnP5vjr71W6scff7KPGTPWR4wY4ffc84yvXOm+evUaHzp0mE+b9nUfOvRYf/XVtf7Nb97q/fsP9QkTTvCLL77Y77jjDt++3f2f//keHzz4GD/66JF+5pkX+ZIl7kuXhjP2t94Kj1dfXeE33+z+wAM1s49nn3X/6CP3xx4LZ2y9e4eztuJi98MOC2VHHOG+ZEnIAsaMcT/zzJoZT1WV+x//6P6zn7nfe6/7+++n/m6sXBkylHh/b9vm/uij7r/8pfv8+e4VFWFbW7bUXG/LFvcBA0JdhgwJ2U86S5eG98mUmSZ77bXwHt/9bnh+/vn6rS9tA205s3D3CjP7DvACYejsI+6+vDHb/P734Z13Dkj1vjBmDPzHf9Q9f/ny5dx666385S9/oUePHmzZsoVdu2q3syZaunQpK1eupFu3bgwePJhrrrmGP/zhr9x55z0888x9DBmS+g1LSsJ2y8vDGV/nzr24774FjB+fR1HRh1xyySUsXryYW2+9nX/91zt55JHf06cPPP74q5SXw5Yt7TjhhOn86ldPM336VXz66ZsMGDCA3r17c+mll/KDH/yAL3/5y7z22idceeXpjBu3ksrKcHbboUM4Y/vgg7f57W/fZfjwbtx2242MHXsyP/jBI/Tps40pUyYwYsQpTJw4mZdf/hMXXzyATp2y+eCDP1NaCgsW/Ikf/vABDj88j7vueppu3Tpz2GGbmDRpEhdeeBZlZVBU9CGzZ8+lX79JrFy5hFdemcezz77Djh0VXHXVOMaOPY6PP4a5c29n9eo1dOyYy7Zt2+jUKeyjOGPYtw927oSbbqq5D3Nz4ayzwvTgweHsesiQcLYL8ItfwKxZoa14zJhQ9re/1f4szMJZ5tSpdX/OAEcfHR6xww6Dr3+99nJdu9Z+/cQTcPXVIRPJz0//PqNGVWdR9TFxYugDefDBsO9OOKH+25C2q00ECwB3fx54vqXr0RivvPIKF1xwAT2insuCgm68915oTqnL8ccfT58+fdi8Gfr0GcLJJ59GeTkMGzaSp59eyN69qYPN9u2hicQsbL+goJxZs77D2rXvkJOTxapVq9i5M3RMQmh6yM0NB4G9e8NB/2tfu4iHHvopl1xyFU88MY8LL7woOpC/xIoVK4DQtLN79w5KS3eyenVHtm+HrVtDenzaaafSr183PvsMXnjhRbZtm8+cOXeSmwu7d5fx9tufUFg4mXnz7mXSpEGceebXWLBgAYWFuykuXsOQIUfRr185s2ffyOuvv052djs+/7yEPXs2YAYDBgzgjDMmsWwZrFz5Jy644ByOPTafZctg8uSz2Lo1/C2jRo3iyisv4+yzz+bss8+u0Ux1+OHheeXKzJ/fpEk1X19+OVx2WeOaGg6UiRNh2bKmfY/c3BAgXn4ZJkwITUpy6GgzweJAS5cBNJetW8NzRUU2u3dXAVBVVcW+ffu+WCYnJ5f16+NMoR2QS3k55OS0wz10SphlU1UV1q+sDOtnZ4czzvbtQ3vnww/fTc+evbnjjqUMG1ZFhw55vP9+OKPOzQ1njBDa4quqQt/BRRd9iZ/8pAj4nIULn2H27P/Dxx9DeXkVTz21iM6d81i3LrS/FxWFANW5c+hjePNN2Ly5gMLCcCAuK3Nmz/5vBg48ir59Q3/Bhg2Ql7ePW25ZzJtvDubUU09l06ZNzJv3n0yYcBzDhsETTzxOaennzJu3BPf2TJ8+EAjXRBQUFJCXF9r9X389ZDM5OWEkUXl56I/o3RteeOEPvP766zz33HPcdtttvPfee2QnN143UGsIFM3pK18JwWLKlJauiTS3tnKdRavlnj4zSHTyySfz1FNPsXnzZgDWrNlCXh4UFg7kT39aAsD8+fMpLy9n9+4whHPHjhAo4mGc8RDBrKzqDusjjhjIkiVh/cceC+v37RuWycoKHY07dmxnyJA+7NnTjp///DEqKyvp1QvGjOlEeXnpF3WMA8yQIZCVZZxzzjnccsv1DBx4DO7dKSuDE044jV/84j7WrQsdhqtXv/PFWWbfvqGucX0LCkLH4YQJp/PUU/fRrp2zdy8sWfI3cnNh+PAcBg7sx1NPPcWXvvQlJk+ezJ133smUKSdSUADbt2+nd+9eFBa2Z/HihXz66ce1DtD5+fCVr5zIM888w549e8jPL+XNN5+jsBAKC6tYt24dU6ZMYfbs2Wzfvp2dO3fW/4MWIAzRNAvPcmhRsGikrVvDmXNdY6ATDR8+nFmzZnHSSScxatRobrvterp3h5kzv8GiRa8xevRo3njjDQoKCli1KoyCyMkJ7djDhoVtVFTUDhYXXvgNXnvtNUaNGs2rr75Bfn4B3bvXfO9vfetb/O53c5kxYzSrVr1Pfn4B/frBuHGjyMrKYvTo0dx9993k5ISDb05OWO+iiy7iv/7rN5x++kVs3BjKHnzwXoqLFzNjxiguueRYHnzwAQoLYdCg1O3lhYVw9dX/Qrt25Vx88ShOOWU4d931L1+M7pg8eTK9evWiQ4cOTJ48meLiYiZPngzAZZddxuLFizn11JEsWPAoRx11dO03AMaNG8dFF13E6NGjmTbtDCZMOJ527aCyspLLL7+ckSNHMnbsWK677jq6JA7bknoZNy5khCee2NI1keZmnq53tQ0bP368J//40cqVKznmmGMO6PvETUTHHFO/i7g2bAhD+kaMCM0lq1eH5pT8/PB6xYpw8E086L/zTjjz37w5XMjUvz+8915YZ8iQ6m3G26nLrl2h6ak+LTFFRaGZJy8v1Lm+4nWLi0Ng3bs3dBT365d53ebQFN8NkbbIzJa4e63x88osGqm8PDxHXQb7bdeucPael1c9fjruqoibtZKDT05OGK9eVRWaiiCsH9/WaPPmECQyjYYpKKhfoAC+GEHU0JPyLl1CXXNyQqd4VVXqceMi0jopWDRSQ4PFvn3VB8v4OT7o79oVDubJB9P27UPWEU9DGKZaVhaPSqJW89OBEt8qItOtKzJJ/JsULETajkMuWDS02a2sLDQ5JYuzgfoGi717q/sF4o7oeFs7d4az/+SO3Nzc6nvbJGYW7mGoK9Qeg3+g5OXB6NGZs5ZM4r853mZrcLA2xYocSIdUsMjLy2Pz5s0NOjhs3hz6JuKDdawhmUVVVVgvPrM2C9N794YO7LKy1P0fiQfaOFjEQWXPntAHkLhMaxTXz6x11NU9/J5FXmuJXCKt1CF1nUXfvn0pLi7m888/r/e6mzaF5qGVK6uv/HUPd/6EEAA2bAgH7Uxn3+XlYXvxvV/i7e/bF0ZAbdoU3iOeF9u1q/oeSqtXV9ejQ4dw8N29e/8uLmtJ8X2gsrPDfYdag/iX8kSkbodUsGjfvn2Dfw3t1FPhpZfCAe6oo0LZxo3hJmoAd94Zps84I9wo7bjj6t7WSy+F5RYuhOOPD2Vz54YbvF13Hdx1V7i5W3zzvNhf/hLWMwuB5QBdV9as3OFLXwq3un6+TV+PL3JoaYOHm5YR3/Uy8XqukoT73u7eHQ7wUH0H0rrE/QsDB1aXDR4cAsADD4QLnpIDBVQPM+3Ro20GCgiB7rrrYOTIlq6JiNRHGz3kNL9UwSJugoLQRFQaXQi9Z091+fLloTzxvkJr14YmpMSWjyFDqrd/9dWp69CnT1gvvp9RW3XbbS1dAxGpr0Oqg7uhdu6szhZKq++MUSuziOcl/pzzDTeEu43G94GCECz69auZHcT3n+/dG772tdT1yM4OV0O39WAhIm2PgsV+SAwKyZlF/LORdWUWW7eGjup///fqsrVrazZBQfUPzlxzTfVIp1RuvBG++c2G/iUiIg2jZqj9UFxcPZ3cZ9GrV7jTal3BIs5I7rkHvve9MLx17drQYZ4oOxs++CDzdRLXXtvgP0NEpMGUWeyHdJlFYWG41iGxGSoxWJSWhlFSO3eGH6jfuzesl5xZQAgkbbXjWkQObgoW+yExWCT3WRxxRAgWdWUWpaXVw2M/+STc6M89dbAQEWmtFCz2Q0lJuBFebm51ZlFREe7EGt+WOzFYJHZwl5aGTusePUKgSDVsVkSktVOw2A8lJaG5qVOn6mCxfHkIEJMmVTdDxf0TcWaxb194dOoUOrA/+UTBQkTaJgWL/VBcHIJFx47VwWLRovA8aVLtzCIOFvHrOFjEmUVWVtieiEhboWCxH0pKwgV0HTtWB4BFi0KH9KBBdfdZxJlGp07hh4rizCL5GgsRkdZOh6z9sGFDuBAuMbN4441wjyOz6mao+M6zdWUW27fDu++qCUpE2h5lFhlUVVX/qlvcZ7FlS7gmIr6FR10d3InBon//ML1smYKFiLQ9yiwyqKgIz1lZIbNYvx7++tdQFgeLgoIQUOIgkSqz6NgxTGvYrIi0RQoWGcQ/dpSdXd0MtXx5KBszJjwn/1BRcrDo3LnmXWQVLESkrVEzVAbJmcXOneEK7Pz8cO0F1P6xo1SZRZ8+0C7a2woWItLWKFhkEAeL7OzqPotPPw1Xbse/kZ2cWcTNUYmjoeI7xoKChYi0PQoWGcTNUHFmUVYWhsAecUT1MomZRceOqTMLCCOidI2FiLRFChYZJGYWcSf1qlU1D/iJmUWvXjWDRW5u9S3Hjz4ahg3TNRYi0vbosJVBcmYBsGlTzcwiOVisWROmS0urswoIv2mReNdaEZG2QplFBsl9FrG6mqGSM4vOnavndelS86dURUTaCgWLDFJlFpA+s0i8KC8xwIiItFUKFhmk6rOAuvssevYMd5qtrAyjoRQsRORgoGCRwf5kFnEzVHZ29bUXZWXKLETk4KFgkUFdmUWqZqhOnaBDhzC9Z4+ChYgcPBQsMki83Ud84O/atTooQBgaG89XsBCRg5GCRQbJt/uAmllFrKCgZrCIm6ESR0OJiLRVjQoWZnaBmS03syozG58078dmVmRmH5jZ6QnlU6OyIjO7IaF8kJm9GZU/YWY5UXlu9Loomj+wMXWur8TMIm5u2p9gsWtXuKZCmYWIHAwam1ksA84FXk8sNLNjgYuB4cBU4FdmlmVmWcAvgTOAY4FLomUBZgN3u/uRwFbg6qj8amBrVH53tFyzScwssrMhLy91sMjPD4EhLy+83rQpPCtYiMjBoFHBwt1XuvsHKWZNB+a5+153XwMUAROiR5G7r3b3fcA8YLqZGXAy8Lto/bnA2QnbmhtN/w74arR8s0jMLAC+9z245JLay40dG25ZHmcWGzeGZwULETkYNNXtPgqBRQmvi6MygHVJ5ROB7sA2d69IsXxhvI67V5jZ9mj5TU1T9ZoSMwuA229PvdyTT4bnRdFfrWAhIgeTjMHCzF4CDk8xa5a7P3vgq9RwZjYTmAnQP/4d00ZKziwyUWYhIgejjIdAdz+lAdstAfolvO4blVFH+Wagi5llR9lF4vLxtorNLBs4LFo+VV3nAHMAxo8f7w2ody3JmUUmcbCIbybYq9eBqIWISMtqqqGz84GLo5FMg4ChwF+Bt4Ch0cinHEIn+Hx3d2AhcH60/gzg2YRtzYimzwdeiZZvFvXNLOIO7hUrwvOgQQe+TiIiza2xQ2fPMbNi4EvAH8zsBQB3Xw48CawA/hf4trtXRlnDd4AXgJXAk9GyAD8CrjezIkKfxMNR+cNA96j8euCL4bbNoaGZxQcfhOG0PXs2Tb1ERJpTozq43f1p4Ok65t0G3Jai/Hng+RTlqwmjpZLLy4ALGlPPxmhon8XevTB0aPVPr4qItGW6gjuDhmYWoCYoETl4KFhkUN/MIiur+mdUBw9umjqJiDQ3BYsM6ptZQHUntzILETlYKFhkUN/MAqqbohQsRORgoWCRQUMyCwULETnYKFhkkPjjR/tLwUJEDjYKFhkk/qzq/urQIVxfkfjLeiIibZmCRQYNySwKCjQSSkQOLk1119mDRkMyi9mzdTGeiBxcFCwyaEhm8Xd/1zR1ERFpKWqGyqAhmYWIyMFGwSKDhmQWIiIHGwWLDOLMop32lIgcwnQIzKCiIjRBqcNaRA5lChYZVFaqCUpERMEigzizEBE5lClYZKDMQkREwSIjZRYiIgoWGSmzEBFRsMhImYWIiIJFRsosREQULDJSZiEiomCRkTILEREFi4yUWYiIKFhkpMxCRETBIiNlFiIiChYZKbMQEVGwyEiZhYiIgkVGFRXKLEREFCwyqKxUZiEiomCRgTILEREFi4yUWYiIKFhkpMxCRETBIiMNnRURUbDISENnRUQULDJSZiEiomCRkTILEREFi4yUWYiINDJYmNkdZva+mb1rZk+bWZeEeT82syIz+8DMTk8onxqVFZnZDQnlg8zszaj8CTPLicpzo9dF0fyBjalzfSmzEBFpfGaxABjh7qOAVcCPAczsWOBiYDgwFfiVmWWZWRbwS+AM4FjgkmhZgNnA3e5+JLAVuDoqvxrYGpXfHS3XbJRZiIg0Mli4+4vuXhG9XAT0jaanA/Pcfa+7rwGKgAnRo8jdV7v7PmAeMN3MDDgZ+F20/lzg7IRtzY2mfwd8NVq+WSizEBE5sH0W/wD8MZouBNYlzCuOyuoq7w5sSwg8cXmNbUXzt0fLNwtlFiIikPEwaGYvAYenmDXL3Z+NlpkFVACPH9jq1Y+ZzQRmAvTv3/+AbFOZhYjIfgQLdz8l3XwzuxI4E/iqu3tUXAL0S1isb1RGHeWbgS5mlh1lD4nLx9sqNrNs4LBo+VR1nQPMARg/frynWqa+lFmIiDR+NNRU4IfAWe6+O2HWfODiaCTTIGAo8FfgLWBoNPIph9AJPj8KMguB86P1ZwDPJmxrRjR9PvBKQlBqcsosRET2I7PI4BdALrAg6nNe5O7/6O7LzexJYAWheerb7l4JYGbfAV4AsoBH3H15tK0fAfPM7Fbgb8DDUfnDwGNmVgRsIQSYZqPMQkSkkcEiGs5a17zbgNtSlD8PPJ+ifDVhtFRyeRlwQWPq2RjKLEREdAV3RsosREQULNJy148fiYiAgkVaVVXhWZmFiBzqFCzSqIguEVRmISKHOgWLNOJgocxCRA51ChZpVFaGZ2UWInKoU7BIQ5mFiEigYJGGMgsRkUDBIg1lFiIigYJFGnFmoWAhIoc6BYs0NHRWRCRQsEhDmYWISKBgkYYyCxGRQMEiDWUWIiKBgkUayixERAIFizSUWYiIBAoWaSizEBEJFCzSUGYhIhIoWKShzEJEJFCwSEOZhYhIoGCRhjILEZFAwSINZRYiIoGCRRrKLEREAgWLNJRZiIgEChZpKLMQEQkULNJQZiEiEihYpKHMQkQkULBIQ5mFiEigYJGGMgsRkUDBIo04WCizEJFDnYJFGnEzlDILETnUKVikocxCRCRQsEhDHdwiIoGCRRrq4BYRCRQs0lBmISISKFikocxCRCRQsEhDmYWISKBgkYYyCxGRoFHBwsxuMbN3zewdM3vRzI6Iys3M7jWzomj+uIR1ZpjZh9FjRkL5cWb2XrTOvWZmUXk3M1sQLb/AzLo2ps71UVkJZtBOIVVEDnGNPQze4e6j3H0M8HvgJ1H5GcDQ6DETuB/CgR+4CZgITABuSjj43w98I2G9qVH5DcDL7j4UeDl63SwqKpRViIhAI4OFu+9IeFkAeDQ9HXjUg0VAFzPrA5wOLHD3Le6+FVgATI3mdXb3Re7uwKPA2QnbmhtNz00ob3Ll5ZCT01zvJiLSejW669bMbgOuALYDU6LiQmBdwmLFUVm68uIU5QC93X19NP0Z0DtNXWYSMhn69+/fgL+mpr17FSxERGA/Mgsze8nMlqV4TAdw91nu3g94HPhOU1Y2yjo8zfw57j7e3cf37Nmz0e+3bx/k5jZ6MyIibV7GzMLdT9nPbT0OPE/okygB+iXM6xuVlQBfSSp/NSrvm2J5gA1m1sfd10fNVRv3sz6NpsxCRCRo7GiooQkvpwPvR9PzgSuiUVGTgO1RU9ILwGlm1jXq2D4NeCGat8PMJkWjoK4Ank3YVjxqakZCeZNTZiEiEjS2z+J2MzsKqAI+Bv4xKn8emAYUAbuBqwDcfYuZ3QK8FS33U3ffEk1/C/g10AH4Y/QAuB140syujt7jwkbWeb8psxARCRoVLNz9vDrKHfh2HfMeAR5JUb4YGJGifDPw1cbUs6GUWYiIBLrcLA1lFiIigYJFGsosREQCBYs0lFmIiAQKFmkosxARCRQs0lBmISISKFikocxCRCRQsEhDmYWISKBgkYYyCxGRQMEiDWUWIiKBgkUayixERAIFizSUWYiIBAoWdXAPv5SnzEJERMGiTvv2hWdlFiIiCha1vPgi3HNPdbBQZiEiomBRy3PPwU9/GvorQJmFiAgoWNSSnw+7d1cHC2UWIiIKFrXk50NZWXiAMgsREVCwqCU/Pzxv2xaelVmIiChY1JIcLJRZiIgoWNRSUBCet24Nz8osREQULGqJM4s4WCizEBFRsKglOVgosxARUbCoRX0WIiK1KVgkUWYhIlKbgkUSZRYiIrUpWCRRZiEiUpuCRZLkobPKLEREFCxqUWYhIlKbgkUS9VmIiNSmYJGkfXvIzlZmISKSSMEihfx8qKgI08osREQULFKKm6LMQpYhInKoU7BIIQ4WOTkhYIiIHOoULFKIgyQmnbgAAAeISURBVIX6K0REAgWLFOJrLdRfISISKFikoMxCRKQmBYsUEvssREREwSIlZRYiIjUdkGBhZv9kZm5mPaLXZmb3mlmRmb1rZuMSlp1hZh9GjxkJ5ceZ2XvROveahXFIZtbNzBZEyy8ws64Hos7pKLMQEamp0cHCzPoBpwGfJBSfAQyNHjOB+6NluwE3AROBCcBNCQf/+4FvJKw3NSq/AXjZ3YcCL0evm5QyCxGRmg5EZnE38EPAE8qmA496sAjoYmZ9gNOBBe6+xd23AguAqdG8zu6+yN0deBQ4O2Fbc6PpuQnlTUaZhYhITY0KFmY2HShx96VJswqBdQmvi6OydOXFKcoBerv7+mj6M6B3mvrMNLPFZrb4888/r++f8wVlFiIiNWW8mYWZvQQcnmLWLOBGQhNUs3B3NzNPM38OMAdg/PjxdS6Xia6zEBGpKWOwcPdTUpWb2UhgELA06ovuC7xtZhOAEqBfwuJ9o7IS4CtJ5a9G5X1TLA+wwcz6uPv6qLlqY8a/qpGUWYiI1NTgZih3f8/de7n7QHcfSGg6GufunwHzgSuiUVGTgO1RU9ILwGlm1jXq2D4NeCGat8PMJkWjoK4Ano3eaj4Qj5qakVDeZNRnISJSU1PdU/V5YBpQBOwGrgJw9y1mdgvwVrTcT919SzT9LeDXQAfgj9ED4HbgSTO7GvgYuLCJ6vwFZRYiIjUdsGARZRfxtAPfrmO5R4BHUpQvBkakKN8MfPVA1XN/KLMQEalJV3CnoMxCRKQmBYsUlFmIiNSkYJFCPHRWmYWISKBgkYIyCxGRmhQsUlCfhYhITQoWKSizEBGpScEiha5d4dZb4bzzWromIiKtQ1NdlNemmcGsWS1dCxGR1kOZhYiIZKRgISIiGSlYiIhIRgoWIiKSkYKFiIhkpGAhIiIZKViIiEhGChYiIpKRhd8pOviY2eeEX9ZriB7ApgNYnQOltdYLWm/dVK/6aa31gtZbt4OtXgPcvWdy4UEbLBrDzBa7+/iWrkey1lovaL11U73qp7XWC1pv3Q6VeqkZSkREMlKwEBGRjBQsUpvT0hWoQ2utF7Teuqle9dNa6wWtt26HRL3UZyEiIhkpsxARkYwULEREJCMFiyRmNtXMPjCzIjO7oQXr0c/MFprZCjNbbmbfi8pvNrMSM3snekxrgbqtNbP3ovdfHJV1M7MFZvZh9Ny1met0VMI+ecfMdpjZ91tqf5nZI2a20cyWJZSl3EcW3Bt95941s3HNXK87zOz96L2fNrMuUflAM9uTsO8eaOZ61fnZmdmPo/31gZmd3sz1eiKhTmvN7J2ovDn3V13Hh6b7jrm7HtEDyAI+AgYDOcBS4NgWqksfYFw03QlYBRwL3Az8cwvvp7VAj6SynwM3RNM3ALNb+HP8DBjQUvsLOBEYByzLtI+AacAfAQMmAW82c71OA7Kj6dkJ9RqYuFwL7K+Un130f7AUyAUGRf+zWc1Vr6T5/w78pAX2V13Hhyb7jimzqGkCUOTuq919HzAPmN4SFXH39e7+djRdCqwECluiLvtpOjA3mp4LnN2Cdfkq8JG7N/QK/kZz99eBLUnFde2j6cCjHiwCuphZn+aql7u/6O4V0ctFQN+meO/61iuN6cA8d9/r7muAIsL/brPWy8wMuBD4r6Z473TSHB+a7DumYFFTIbAu4XUxreAAbWYDgbHAm1HRd6JU8pHmbu6JOPCimS0xs5lRWW93Xx9Nfwb0boF6xS6m5j9wS++vWF37qDV97/6BcAYaG2RmfzOz18xscgvUJ9Vn11r212Rgg7t/mFDW7Psr6fjQZN8xBYtWzsw6Av8NfN/ddwD3A0OAMcB6Qhrc3L7s7uOAM4Bvm9mJiTM95L0tMibbzHKAs4CnoqLWsL9qacl9VBczmwVUAI9HReuB/u4+Frge+K2ZdW7GKrXKzy7BJdQ8KWn2/ZXi+PCFA/0dU7CoqQTol/C6b1TWIsysPeGL8Li7/w+Au29w90p3rwL+kyZKv9Nx95LoeSPwdFSHDXFaGz1vbO56Rc4A3nb3DVEdW3x/JahrH7X4987MrgTOBC6LDjJEzTybo+klhL6BYc1VpzSfXWvYX9nAucATcVlz769Uxwea8DumYFHTW8BQMxsUnaFeDMxviYpE7aEPAyvd/a6E8sR2xnOAZcnrNnG9CsysUzxN6BxdRthPM6LFZgDPNme9EtQ422vp/ZWkrn00H7giGrEyCdie0JTQ5MxsKvBD4Cx3351Q3tPMsqLpwcBQYHUz1quuz24+cLGZ5ZrZoKhef22uekVOAd539+K4oDn3V13HB5ryO9YcPfdt6UEYNbCKcFYwqwXr8WVCCvku8E70mAY8BrwXlc8H+jRzvQYTRqIsBZbH+wjoDrwMfAi8BHRrgX1WAGwGDksoa5H9RQhY64FyQvvw1XXtI8IIlV9G37n3gPHNXK8iQnt2/D17IFr2vOgzfgd4G/j7Zq5XnZ8dMCvaXx8AZzRnvaLyXwP/mLRsc+6vuo4PTfYd0+0+REQkIzVDiYhIRgoWIiKSkYKFiIhkpGAhIiIZKViIiEhGChYiIpKRgoWIiGT0/wGaxnzhntjXwgAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The PostScript backend does not support transparency; partially transparent artists will be rendered opaque.\n",
            "The PostScript backend does not support transparency; partially transparent artists will be rendered opaque.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29eZgV1bX3/1k0NA0tiCgiAtKgOIDMDZEoIhIENRHiLybmxYgjuVGvJnpVTF4Foybx6g356ZM4E3CIEqdI1JuAgFMSFVBAZQYhzCDz0DRN93r/2LW7qs/QfRp6Ouesz/Ocp6p27apaVeec71619iSqimEYhpEdNKpvAwzDMIy6w0TfMAwjizDRNwzDyCJM9A3DMLIIE33DMIwsonF9G1AZxx13nBYUFNS3GYZhGGnFvHnzvlbVNon2NWjRLygoYO7cufVthmEYRlohImuS7bPwjmEYRhZhom8YhpFFmOgbhmFkEQ06pp+IkpIS1q1bx4EDB+rbFMMwUiQvL48OHTrQpEmT+jYl60k70V+3bh0tWrSgoKAAEalvcwzDqAJVZdu2baxbt47OnTvXtzlZT0rhHRFZLSKfi8h8EZkbpLUWkRkisjxYHhOki4g8IiIrRGShiPSNnGdMkH+5iIw5HIMPHDjAsccea4JvGGmCiHDsscfa23kDoTox/SGq2ltVC4PtccBMVe0KzAy2AS4EugafscBj4AoJYDzwDWAAMN4XFNXFBN8w0gv7zzYcjqQidyQwJVifAoyKpD+rjo+AViLSDhgOzFDV7aq6A5gBjDiC61dJURHs3VubVzAMw0gvUhV9BaaLyDwRGRuktVXVjcH6JqBtsN4eWBs5dl2Qliy9AiIyVkTmisjcrVu3pmheYjZsgDVJuygcPiLCbbfdVr798MMPM2HChBo591VXXcUrr7xSI+eqjJdffpkzzjiDIUOGpJT/V7/6Va3aM3/+fN5+++0q802ePJmbbrqpXq5dF5x33nlVdkhcvXo1f/rTn2rNhscff5xnn332sI5dvXo1Z555Zg1bZNQkqYr+OaraFxe6uVFEzo3uVDcTS43MxqKqT6pqoaoWtmmTsBdxypSVQW3MEdO0aVNee+01vv7665o/+RFw6NChlPM+88wzPPXUU8yePTul/A1F9Bvitavz3GuCwxH96tj4H//xH1x55ZXVNctIE1ISfVVdHyy3AK/jYvKbg7ANwXJLkH090DFyeIcgLVl6rVFbot+4cWPGjh3LxIkT4/bFeupHHXUUAO+++y6DBw9m5MiRdOnShXHjxvHCCy8wYMAAevTowcqVK8uPeeeddygsLOTUU0/lzTffBKC0tJTbb7+d/v3707NnT5544ony8w4aNIhLLrmEbt26xdnz4osv0qNHD84880zuvPNOAH75y1/y4Ycfcu2113L77bdXyL9x40bOPfdcevfuzZlnnskHH3zAuHHjKCoqonfv3owePRqA559/ngEDBtC7d29+/OMfU1paWn6/P/vZz+jevTtDhw7Fv6098sgjdOvWjZ49e3L55ZdXuObBgwe55557mDp1Kr1792bq1Kl88sknDBw4kD59+vDNb36TpUuXxt3bW2+9xcCBA/n666+ZPn06AwcOpG/fvlx22WXsDeJ6BQUFjB8/nr59+9KjRw+WLFlS5bW3b9/OqFGj6NmzJ2eddRYLFy6Mu/bkyZO55JJLOP/88xk6dCj79u3jmmuuYcCAAfTp04c33ngDcAI9aNAg+vbtS9++ffnnP/9Zfo4HH3yQHj160KtXL8aNG1ee/vLLLzNgwABOPfVUPvjgg7hrjxs3jg8++IDevXszceLElH8bqf4GJ0yYwMMPPwy4N48777wzzp7K7sto4KhqpR8gH2gRWf8nLhb/EDAuSB8H/HewfjHwv4AAZwGfBOmtga+AY4LPV0Dryq7dr18/jWXRokXl67fcojp4cPJP//6qhYWV54n93HJL3CXjyM/P1127dmmnTp10586d+tBDD+n48eNVVXXMmDH68ssvV8irqjp79mw9+uijdcOGDXrgwAE98cQT9Z577lFV1d/97nd6S3DhMWPG6PDhw7W0tFSXLVum7du316KiIn3iiSf0vvvuU1XVAwcOaL9+/XTVqlU6e/Zsbd68ua5atSrOzvXr12vHjh11y5YtWlJSokOGDNHXX39dVVUHDx6sc+bMiTvm4Ycf1vvvv19VVQ8dOqS7d++ucB/+O/j2t7+tBw8eVFXVn/zkJzplyhRVdW98zz//vKqq3nvvvXrjjTeqqmq7du30wIEDqqq6Y8eOuOv+8Y9/LM+rqrpr1y4tKSlRVdUZM2bopZdeWiHfa6+9puecc45u375dt27dqoMGDdK9e/eqqupvfvMbvffee1VVtVOnTvrII4+oqurvf/97vfbaa6u89k033aQTJkxQVdWZM2dqr169Eh7Tvn173bZtm6qq3nXXXfrcc8+V31/Xrl117969um/fPi0qKlJV1WXLlqn/Tb/99ts6cOBA3bdvn6pq+XkGDx6st956q6qqvvXWWzp06NC4a8+ePVsvvvji8u1Ufxup/gbHjx+vDz30UKX2JLuvr776Srt37x5ns2rF/65RuwBzNYmuptJOvy3welD73hj4k6r+TUTmAH8WkWuBNcD3g/xvAxcBK4D9wNVB4bJdRO4D5gT5fqmq26tXRDUcWrZsyZVXXskjjzxCs2bNUjqmf//+tGvXDoCTTz6ZCy64AIAePXpUCLN8//vfp1GjRnTt2pUuXbqwZMkSpk+fzsKFC8vfInbt2sXy5cvJzc1lwIABCds/z5kzh/POOw8fJhs9ejTvv/8+o0aNissbtfGaa66hpKSEUaNG0bt377g8M2fOZN68efTv3x+AoqIijj/+eAAaNWrED37wAwCuuOIKLr30UgB69uzJ6NGjGTVqVKXX9+zatYsxY8awfPlyRISSkpLyfbNmzWLu3LlMnz6dli1b8uabb7Jo0SLOPvtswHnvAwcOLM/vbejXrx+vvfZaldf+8MMPefXVVwE4//zz2bZtG7t376Zly5YV8g0bNozWrVsDMH36dKZNm1buIR84cIB///vfnHjiidx0003Mnz+fnJwcli1bBri3uauvvprmzZsDlJ8n1t7Vq1dXaW91fhup/gajJLKnpKQk4X0ZDZ8qRV9VVwG9EqRvA4YmSFfgxiTnmgRMqr6Zifnd7yrfv2gRlJRArzjra4af/vSn9O3bl6uvvro8rXHjxpSVlQFQVlbGwYMHy/c1bdq0fL1Ro0bl240aNaoQc41t3iYiqCqPPvoow4cPr7Dv3XffJT8/v8bu6dxzz+X999/nrbfe4qqrruLWW2+Ni++qKmPGjOHXv/51lefz9/LWW2/x/vvv89e//pUHHniAzz//nMaNk//87r77boYMGcLrr7/O6tWrOe+888r3nXzyyaxatYply5ZRWFiIqjJs2DBefPHFhOfyzzknJ6dG4+/R566qvPrqq5x22mkV8kyYMIG2bduyYMECysrKyMvLq/K81bW3Or+NVH+DVdkzceLEat+X0TDI6LF3Au2tNVq3bs33v/99nnnmmfK0goIC5s2bB8C0adMqeKip8vLLL1NWVsbKlStZtWoVp512GsOHD+exxx4rP9+yZcvYt29fpecZMGAA7733Hl9//TWlpaW8+OKLDB48uNJj1qxZQ9u2bbn++uu57rrr+PTTTwFo0qRJ+bWHDh3KK6+8wpYtrhpn+/btrAmaSZWVlZV7nH/6058455xzKCsrY+3atQwZMoQHH3yQXbt2lcfcPS1atGDPnj3l27t27aJ9e9e4a/LkyRXydurUiVdffZUrr7ySL7/8krPOOot//OMfrFixAoB9+/ZVy/OMvfagQYN44YUXACecxx13XJyXH8vw4cN59NFHfUiUzz77rPw+2rVrR6NGjXjuuefK6z6GDRvGH//4R/bv3w+4Z3i49h7Ob+NISXZfRsMno0VftXYqcqPcdtttFVrxXH/99bz33nv06tWLf/3rX4flhZ900kkMGDCACy+8kMcff5y8vDyuu+46unXrRt++fTnzzDP58Y9/XKUX2K5dO37zm98wZMgQevXqRb9+/Rg5cmSlx7z77rv06tWLPn36MHXqVG655RYAxo4dWx6i6datG/fffz8XXHABPXv2ZNiwYWzc6Frv5ufn88knn3DmmWcya9Ys7rnnHkpLS7niiivo0aMHffr04eabb6ZVq1YVrjtkyBAWLVpUXpl6xx13cNddd9GnT5+E93n66afzwgsvcNlll7F7924mT57MD3/4Q3r27MnAgQPjKmwrI/baEyZMYN68efTs2ZNx48YxZcqUKs9x9913U1JSQs+ePenevTt33303ADfccANTpkyhV69eLFmypPz3MGLECC655BIKCwvp3bt3eVgoFXr27ElOTg69evVi4sSJh/XbOFKS3ZfR8BGtbVU8AgoLCzW2zfLixYs544wzUjp+4UIoLYU+fWrDOiMRRx11VJwXbxhQvf+ucWSIyDwNR0+ogHn6hmEYWURGi35tx/SNeMzLN4yGTVqKfqohKfP0DaNh0JDDyNlG2ol+Xl4e27ZtS+lHZKJvGPWPBuPpW7POhkHaTaLSoUMH1q1bR1WDsamCz7J4cR0YZhhGUvzMWUb9k3ai36RJk5Rm3ykuhu7d3fqhQ5CTU8uGGYZhpAFpF95JlUhHWOp4EETDMIwGS1aIvnUWNAzDcGSs6BcXh+sm+oZhGI6MFX0L7xiGYcSTFaJvnr5hGIYjK0TfPH3DMAxHxoq+xfQNwzDiyVjRt/COYRhGPFkh+hbeMQzDcGSF6JunbxiG4chY0Y/G9M3TNwzDcGSs6JunbxiGEY+JvmEYRhaRFaJv4R3DMAxHxoq+tdM3DMOIJ2NF3zx9wzCMeLJC9M3TNwzDcJjoG4ZhZBEZK/rWTt8wDCOejBV98/QNwzDiyQrRN0/fMAzDkbGib002DcMw4klZ9EUkR0Q+E5E3g+3OIvKxiKwQkakikhukNw22VwT7CyLnuCtIXyoiw2v6ZqJYeMcwDCOe6nj6twCLI9sPAhNV9RRgB3BtkH4tsCNInxjkQ0S6AZcD3YERwB9EJOfIzE+OhXcMwzDiSUn0RaQDcDHwdLAtwPnAK0GWKcCoYH1ksE2wf2iQfyTwkqoWq+pXwApgQE3cRCLM0zcMw4gnVU//d8AdQFmwfSywU1W9D70OaB+stwfWAgT7dwX5y9MTHFOOiIwVkbkiMnfr1q3VuJWKFBdDTvAeYaJvGIbhqFL0ReTbwBZVnVcH9qCqT6pqoaoWtmnT5rDPc/AgNG/u1i28YxiG4WicQp6zgUtE5CIgD2gJ/P9AKxFpHHjzHYD1Qf71QEdgnYg0Bo4GtkXSPdFjapyDB6FZM9izxzx9wzAMT5WevqrepaodVLUAVxE7S1VHA7OB7wXZxgBvBOvTgm2C/bNUVYP0y4PWPZ2BrsAnNXYnMZinbxiGEU8qnn4y7gReEpH7gc+AZ4L0Z4DnRGQFsB1XUKCqX4rIn4FFwCHgRlWtNR+8uNh5+mCevmEYhqdaoq+q7wLvBuurSND6RlUPAJclOf4B4IHqGnk4RD19E33DMAxHxvbItfCOYRhGPBkt+hbeMQzDqEjGin5xsXn6hmEYsWSs6FtM3zAMIx4TfcMwjCwio0U/L8+tW3jHMAzDkbGiX1wMTZtCo0bm6RuGYXgyVvQPHoTcXGjc2Dx9wzAMT0aKflmZE/qmTd1Im+bpG4ZhODJS9EtK3NJ7+ib6hmEYjowUfT8/bm6u8/QtvGMYhuHISNH3s2Z50TdP3zAMw5HRot+0qVXkGoZhRMlo0TdP3zAMoyIZKfrRmL5V5BqGYYRkpOjHevoW3jEMw3BktOhbO33DMIyKZKTo5+fDiBFwwglWkWsYhhHlSObIbbCcfjr87/+6dfP0DcMwQjLS049iFbmGYRghGS/6VpFrGIYRkhWib56+YRiGI+NF3ypyDcMwQjJe9M3TNwzDCMl40beKXMMwjJCMF32ryDUMwwjJCtE3T98wDMOR8aJvFbmGYRghGS/65ukbhmGEZLzoW0WuYRhGSMaLvlXkGoZhhFQp+iKSJyKfiMgCEflSRO4N0juLyMciskJEpopIbpDeNNheEewviJzrriB9qYgMr62bimLhHcMwjJBUPP1i4HxV7QX0BkaIyFnAg8BEVT0F2AFcG+S/FtgRpE8M8iEi3YDLge7ACOAPIpJTkzeTCAvvGIZhhFQp+urYG2w2CT4KnA+8EqRPAUYF6yODbYL9Q0VEgvSXVLVYVb8CVgADauQuKsHCO4ZhGCEpxfRFJEdE5gNbgBnASmCnqno5XQe0D9bbA2sBgv27gGOj6QmOiV5rrIjMFZG5W7durf4dxWCevmEYRkhKoq+qparaG+iA885Pry2DVPVJVS1U1cI2bdoc8fnM0zcMwwipVusdVd0JzAYGAq1ExM+81QFYH6yvBzoCBPuPBrZF0xMcU2tYRa5hGEZIKq132ohIq2C9GTAMWIwT/+8F2cYAbwTr04Jtgv2zVFWD9MuD1j2dga7AJzV1I8mw8I5hGEZIKnPktgOmBC1tGgF/VtU3RWQR8JKI3A98BjwT5H8GeE5EVgDbcS12UNUvReTPwCLgEHCjqta6HFt4xzAMI6RK0VfVhUCfBOmrSND6RlUPAJclOdcDwAPVN/PwsfCOYRhGSMb3yLUB1wzDMEIyXvTN0zcMwwjJeNG3ilzDMIyQjBf9nBxQhbKy+rbEMAyj/skK0Qfz9g3DMCALRL9x0D7JKnMNwzCyQPTN0zcMwwjJeNH3nr6JvmEYRhaIvvf0LbxjGIaRRaJvnr5hGEYWiL5V5BqGYYRkvOibp28YhhGS8aJvFbmGYRghGS/6VpFrGIYRkjWib56+YRhGFoi+VeQahmGEZLzom6dvGIYRkvGibxW5hmEYIRkv+laRaxiGEZI1om+evmEYRhaIvlXkGoZhhGS86JunbxiGEZLxom8VuYZhGCEZL/pWkWsYhhGSNaJvnr5hGEYWiL5V5BqGYYRkvOgn8vSnToUf/KB+7DEMw6hPMl70E1Xkfvgh/PnPsHt3/dhkGIZRX2S86CeqyD140C2XLKl7ewzDMOqTjBf9RDH9khK3NNE3DCPbqFL0RaSjiMwWkUUi8qWI3BKktxaRGSKyPFgeE6SLiDwiIitEZKGI9I2ca0yQf7mIjKm92wrJzXVLL/TR9cWL68ICwzCMhkMqnv4h4DZV7QacBdwoIt2AccBMVe0KzAy2AS4EugafscBj4AoJYDzwDWAAMN4XFLWJF30f0gHz9A3DyF6qFH1V3aiqnwbre4DFQHtgJDAlyDYFGBWsjwSeVcdHQCsRaQcMB2ao6nZV3QHMAEbU6N0koDLRN0/fMIxso1oxfREpAPoAHwNtVXVjsGsT0DZYbw+sjRy2LkhLll6rVCb6K1ZUTDcMw8h0UhZ9ETkKeBX4qapWaOyoqgpoTRgkImNFZK6IzN26desRny+R6Pv10lJYufKIL2EYhpE2pCT6ItIEJ/gvqOprQfLmIGxDsNwSpK8HOkYO7xCkJUuvgKo+qaqFqlrYpk2b6txLQpo0cctYT795c7duIR7DMLKJVFrvCPAMsFhVfxvZNQ3wLXDGAG9E0q8MWvGcBewKwkB/By4QkWOCCtwLgrRaJSfHfWJF/4wz3PqKFbVtQWayYgU8+WR9W2EYRnVJxdM/G/gRcL6IzA8+FwG/AYaJyHLgW8E2wNvAKmAF8BRwA4CqbgfuA+YEn18GabVObm686B8TtBvav78uLMg8nn8efvxjG9PIMNKNxlVlUNUPAUmye2iC/ArcmORck4BJ1TGwJkgk+k2bus+BA4mP2bwZJk2CO++ERhnfha36FBe7ZUlJ2AHOMIyGT1bIWW5uKFLghKpJE8jLq5geZdo0+PnP4fPP68bGdMO3gLLWT4aRXmSN6Me23vGin8zTLypyy2XLat++dMQ/z2SFpmEYDZOsEP2mTePDO02aVB7e8elLl9a+femIefqGkZ5khegniumn6umb6CfGRN8w0hMT/So8fQvvJMY/TxN9w0gvslb0c3NTE/2lS0FrpK9xZuE9fYvpG0Z6kbWin6qnv2sXbNmSOE82Y+Edw0hPslL0U2m9E023EE88Ft4xjPQkK0U/1YrcZs3culXmxmOevmGkJ1kn+qpu6IBUPP2TT3bNOk304zHRN4z0JOtE348Vk4ro5+fDSSfB2rWJ82Qz1jnLMNKTrBN976Gm0nonLw+OOsoGZUuEefqGkZ5knej7Zaox/ebNTfQTYaJvGOlJVoyPmMjTTyW8k5fnZtfas6du7EwnTPQNIz3JOk8/Kvp+7J1Ena+86Ofnm6efCIvpG0Z6kvWePrh9TZtWPMaLfqNGJvqJME/fMNKTrPb0vegnCvFEY/r79tWNnemEib5hpCdZ6+n71juQOEThPf3cXPP0E2E9cg0jPcka0S8rc5Wysa13ILGn70UfTPQTYQOuGUZ6kjWiD07wUwnvlJa6fD6mX1ISDt1gOCy8YxjpSVbE9H0lbaqi771X33oHzNuPxcI7hpGeZIXoe0+/uDg10fezZvmKXDDRj8U8fcNIT7I2vJObCyJuPVb0/XZeXlgwWAueEF8/Aib6hpFuZK3oN2kCOTluvTLRN08/Hv8MwSpyDSPdyDrRj7beScXTt5h+PFHRN0/fMNKLrBP9qKffOLj7VGL6Ft4JMdE3jPQlq0Xft+qx8E71iJ160jCM9CGrWu8cjujXRninqAgWLKi589U1FtM3jPQla0U/OgxDKp5+TYZ3pkyB/v1h796aO2ddYuEdw0hfslb067Od/tatzo50HaffRN8w0pesE/1o6536Cu/4c/nCJd2wmL5hpC9Vir6ITBKRLSLyRSSttYjMEJHlwfKYIF1E5BERWSEiC0Wkb+SYMUH+5SIypnZuJzGVtd5p3LjuO2d50U/XymH/DEUspm8Y6UYqnv5kYERM2jhgpqp2BWYG2wAXAl2Dz1jgMXCFBDAe+AYwABjvC4q6IJnoQ+IpE6Oi36iRC/PUpED7AiRdPX3/DJs3N0/fMNKNKkVfVd8HtsckjwSmBOtTgFGR9GfV8RHQSkTaAcOBGaq6XVV3ADOIL0hqjapEP9Zb9aLfrJlb1vSUiZkS3jnqKBN9w0g3Djem31ZVNwbrm4C2wXp7YG0k37ogLVl6HCIyVkTmisjcrVu3HqZ5FUkk+r5jVtTTX7MGfvvbUIx9zN/PnqUKTz8N3/wmbNp0+Paku+j7Z5ifb6JvGOnGEVfkqqoCCaYWP+zzPamqhapa2KZNmxo5Z6zoR4dgiIr+Sy/BbbfBihUVx+Zp3twJ9S23wPXXw7/+BZ9+evj2ZEpM3zx9w0g/Dlf0NwdhG4LlliB9PdAxkq9DkJYsvU6Ibb0TnQwlKvq7d7vl0qVhBS6E4Z033oBevVza118fvj3p7ul7oc/Pt4pcw0g3Dlf0pwG+Bc4Y4I1I+pVBK56zgF1BGOjvwAUickxQgXtBkFYnxE6ikkz0fbv5JUsqin7z5rBzJ6xbB4MHu7QjEf1Mqcg1T98w0o9Ummy+CPwLOE1E1onItcBvgGEishz4VrAN8DawClgBPAXcAKCq24H7gDnB55dBWp2QKLzjSeTp794dVuKCE/1ly9w48j17uvqAZKK/bRusXFm5Penu6ZvoG0b6UuWAa6r6wyS7hibIq8CNSc4zCZhULetqiJwc1/Qykeg3bRqKb7SHbGx4x9cpd+4Mxx2XXPTHjYPZs129QDIasui/9x4MGuSeVzKi4Z3SUvfx9R+GYTRssqJHLjhv34u+9/whsafv0z1+KAaATp0qF/1Fi2D16nBmqUQ01IrchQvhvPPgnXcqzxdtvQPm7RtGOpF1ol9ZRW4yT9+Lvgh07OhEP1lr0lWrnOBX1tq0oXr63ubNmyvPFw3vgIm+YaQTWSX6fmL0qmL6EB/TB2jf3p0nmae/f3/Yfn/DhsR2lJaG12toou9H/dy1q/J8JvqGkb5klein2nrHp3t8GKOgwC2Tif5XX4XrGzfG74eKQt9QRX/nzsrzRWP60W3DMBo+Jvoxnr4X9kThnajob98eH7dftSpcT+bpR+P40fX77oMXX0z1bmoHX+il6ul70be2+oaRPpjoB6Kv6kSvR48w3eNFv1MntzzuONd8M9YjjjbVTEX0o57+44/DH/5QvXuqaVL19C28YxjpS1aKfqLWO35sHS/60Zh+bHjHjw4RG+JZtQpatnT7k4l+dIjmqOjv3OmmUCwrq/atVYvnn4d33028L9WYvoV3DCN9yTrRj22906qVE9r1waAQHTvCscc68fYkCu9AYtHv0gVOPLF6nv7Bgy59z56KIaLa4Oc/h1//OvG+6nj6jRtX7OlsGEZ6kHWiHxveOf54t/SdqVq2hL//3XWy8vTr5z59+rjtVEQ/WUWuF/0mTULRj4rs/PnVv7fqsH07LF4cbq9e7cYT2rChejH93NyKPZ1Tpaio4fVPMIxsIutFv20wKPTy5W7ZooUT+Hbtwjynnw5z57o3AEgs+mVl1fP0jz02XN+xI9w/f74Txug8tDVFSYkLL61dGwr8Rx+5TlkLF1av9U6TJqHob9gAU6emZsNVV8EPk/XxNgyj1jHRD0Q/6ulXhRf9aAesjRtdK5bOnV2BsXkzHDoUf2xU9L2nHxX9jz+Gb3wDrr02tfuqDtHrLFniluvWhfuq004/OsfwI4/A5Ze7cYeqYtGisIA1DKPuMdGPEf0WLao+V/PmrqI36umvWeOWBQXO0y8rgy1b4o/1FbnHHRcf3jnlFDcEwuefh6Jck0RF34d4oqLvvf9UYvpRT//zz90yNtxVUgJ/+YurIPds2pRa4WAYRu2QlaIfbb3jW+JUx9OH+A5aa4N5wTp2dKIPieP6lXn6550X5qtqKITDoTLR37kz9PSLi+PnDY5y8GDFmL4vJLbHjJv60kvw3e+GhUJJiRP87dsrFgSGYdQdWSf6sa13Gjd2Arx6tdtOxdOH1ER/6tT4wcu86B93XLjuRfPKK+HCC2H0aPeWkEwYP/mk8gHdklGVp+9FHyoP8cR6+p5YD97PLuYLv61b3T0dOlSx97NhGI5JgPcAABQhSURBVHVH1ol+bHgHXIjHi2iqnn7nzk7UfIXr2rWus9LRRzvhB3joIRg5smLb+6in7zuFeTHu3x/efht693b7EgnjypUu5v/736dmZxRfuHTvnlz0vZCnIvo+pu+J9fQXLHDLRAO5WYjHMOqHrBH9Y491olNcnFj0wY0JH+2JWxlXXeU82DeCOcPWrnViLwInnACzZsFPf+pEPhrb37/fXd+/URw44MQ4Ly+8trcnUYjHt+N/+unqh0h84fLNb7rCY//+0AvfudMVMh06hNvJiA3veKJCrho2P/WiH51MPhXR/+CDioPgGYZx5GSN6Pfo4SpRd+1KLvotW4YTplfFRRe5YRm8x+1F3zNkCAwNppnxoSNwNjRvHnb4KipyYnzMMfH2JBJ975l//rlrRlodoqJfWuome/FvId7Tb9/ebR9OeCfq6a9bF17PF3rR+4l9K4hl7173DH/0I4v/G0ZNkjWi37NnuF6Z6KdKTg785CduSIPFi+NFH8KxenzLHnDetW/9A6Hot2oV5vEdxhK1/vGi36yZaypZWYVrLDt2uGv7eX592/omTZwXfuhQRU8/ttPVP/7hevPGir6Ie3OJCnm0k9nhhHfWrXMF07Rp8Prrqd+jYRiVkzWi3717OAVgrIfqRTbVSlzPFVe45bRpTtAOR/T373cCWx1Pv00bV+n7/PMubDVzZmr2+jeKggJnqxfT004LK6K96H/5pSuIoud++mm4+25XUEVj+ied5EJaUSFfsMAVBp07pxbe2bChYkHhh8Vo0QJuvrn2xyQyjGwha0S/eXPo2tWt14SnDy4U0qWLa5oI8aLfsqUT2Wh4Z/9+N1BZZZ6+b0aaTPQ7dIBHH3WVvuAKnVTwoi8C554bttbp0SOsNPbhnb/+1dk2Z054vJ8Gcu1aV3D659i1K7RuHe/pn3xyRdHfvDksCGNF/xe/cCEzj+/RfP31rgDwhYBhGEdG1og+hCGeZKJfXU8f3CTi3kONFX1wXnUiTz8a04/19H0z0spEv0kT17yzb1/XhDMVonUHPsTTtKnrFOY54QT3RuSbW0YnhvH3sW6du36jRk78u3Z19kaFfOFC97yPP76ip9+hg2vhFBvTX7bMVSr7ugQv8r7vwrJlqd3j4XDggL1JGNmDiT6H7+mDE31PItHv1Kmi6PuK3FhPPyr63iYv+nv2wDXXONH0ou8ZMAA++yx+rJ4bboBvf9uFY3xFaCLR79Ch4rVbtqz4HLzoew8fnED6ZzhpEvzsZxU9/ZIS18ro9NPdW0vU02/bNr6AgPBtyA/RsH69Kxz8IHe1NXRDcbH73p55pnbObxgNDRN9Dj+mD1WLfkGBEzQvvLEx/X37nKcfDe+AE0dfkTtrFvzxjzBlihPWqOj37++Ey/d6BXe+xx6D996D++8PhTsq+l27Oq8+VvRbtKhoiz92w4aKYwn5epHRo+M9fR8G6trVif6uXa5SeNMmd83WrSuKfnFxGM6Jin779q6jW7Nmtefpr1zpOtn985+1c37DaGhklej36ePi2UcfXTH9SDz9rl3d8cccE04qEqVTJyfs3guOjelv3uwKhMo8fR8+8tMpxnr6UDH2vnSpW950k1suWuSWUdEXgSeegPHjK17bdzDz97ZmjRPwaL0ExBecrVu7N5KSknBIi1NOCesnNm50z8B7+tHwzr//Ha7Hin6jRs6O2hJ9b2t0uGnDyGSySvQ7dnRDCf/gBxXT8/LguutcjLy6iMB3vuNi64nwE6+88w48+2x8O33fOSrW0z/++HjR9z1co6LfubMT0Whc3w/W9t3vuuWiReHQB1GBv+QS1xY+VvS9LZdf7kR8w4YwROXFPlb0/bDTO3aEwu09fXCtgcB5+rHhnWi9gRdhL/r+PLUV3vHnXbzY+gMY7q3ThzEzlawSfXCecXQqRM9TT8EFFxzeOR9/3E28kgjfWuX//B8YM8bF5KPhHR/WSOTp79njYv7z51dsZhoVfREX4nnvvbAN/9KlrjK4Tx8XHvnyy7CHbex1oGKB06KFE+UOHeCcc1zaV1+Fnr6PsSfy9MGJ+fLl7jzHHx+Kvg8/JYrp+3Ofcoo7trTUhYK86J96qqsjSDRU9ZHiC5ndu5NPfGNkDw8/7FqdffxxfVtSe2Sd6NcGOTnukwjv6Z9wghu6ASqGdyoTfXBhjdWrK76deDH0XHGFE+bOnV1Ty6VL3Q+3SRPXP+HLL8PesYlEP9bTf+ABeOUVdz5w516zxtnkm73G9nXwnv727U64u3Z1BZIXfV8onnyyKyB27gxFfPVqV0gNHuyO3bzZCX/U0z90KD7EVBMsXx5+d1UNZ71okSsIo/MopCuLFyef6Cebef9993Z72WXxQ4VnCib6tcwxx7hOVB9+6Fq6PPWUa3seK/qJKnIBXnvNLS+/3I3MmajuYPRoVyF54onurWPpUtdyBsLB1XwMvTLRF3F2nX66G9TtpJNc2urV7lNQEL5lVOXp+8LBi/7s2U7wu3cPCwj/9vHVV+5ap5/ujv/iC5ce9fQheVy/qMjVdxzOyKPLl7s+C1B1XP/5512v5HTvIVxWBt/6FowdW9+WNCzKylyY9Oyz3Zvmr35V3xbVDib6dcDo0c5rFnF1B6ecEg6u5r1L34LIc845TvgfeMBt9+nj6hx69Eh8jYICGDXK9aBdvtz1sgXo1s1VHvt6gUSin5/vvN38/LDXMrg2/CeeGHr6qYj+pk2ugPCi37p1eM7vfc89Ay/60dY+BQXhMe+955ZRTx/CCupY/vAHFz777/9OvD8ZBw64+O2557pwVFWiP326W771VvWuk4gFC6qerKa2WLjQORuzZlVvGI9MZ8UK951cfbUbN8t3fsw0TPTriUaNnPDv3u0Ey4/B72nVyo2tU1rqCoQTToAnn6z8h/id77iKqIMHQ9Hv3t0t//Y3t0wk+iIuPVGT1c6dnUisWePqJ7zoJwvvzJvnPCYv1I0ahfu+972KeaOi37lzeIx/u/Gi36aN2z9pknv1nj7dTVx/883Oy5882eUbP756E8uvWuUqb7t2hTPOqFz0t251HdaaN3eV8kciluvWuXqY666L3zdxogvR1Sb+t1BU5EYyNRw+jv+NbzgHa+nSio0MMgUT/XqkWTMngL/7XeL9l13mxtjxHnJeXuJmoZ5zzw2bnfrwTrdubvmXv4TDIiTimGNcPD+W005znb8OHnTCmMzTb9nSvS3MmOG2vYCDK7Q6dXITzkMo5m++6YRn0ybn6Z92GowY4d5+GjcO335EnBh+8YXzwIYPh9/+1g1FcemlLv2BB9yzvOii5KOPlpU5+95804m9r8Tt2tU9r0WLkvfMfecdd8wdd7g3J/82kojYmcFmzoR77w2Huvjtb13h9eqrFQupN96AW291b4bRcYpqmr//3T3r3NzkDRCykY8/dv+BM85wv0MIC8iMQlXr9AOMAJYCK4BxleXt16+fZjL/8z+qM2bU7Dkvu0wVVL/+Okzr31912DDVrVuTH9e/v2qfPvHpW7eqvv226pw5qqWlqhs3uvPfd1983oICt69zZ9Xdu8P0V15RfeutcLusTPXqq13egQPd8rXXwn1Tp6o+8UT8+UeOdHl/9CPVAwdUf/ITt52Xp7pjh+rChaqdOrnt8eNVP/1U9dVX3fqoUaonnujyg3seo0a59W3bVCdNcuuXXqp6ww2qZ5+tOnmy6iefqP7lL6oXXqjaurXq3r2qzZqpDhqk+swzqm+8ofr666pTpqh+9JHq7berirjzLFrkvuOcHHfuk05SffRR1fx8dy+tWqkOHqz69NOqv/+9atu2qqedppqb6/bffrvqXXepfvyx6vr1qsuWqc6apTp7trPt5ptV//Y398w2bXLXW7ZMtajI2XbbbarLl7v0KVNUp01T/fxz1caNVe+4Q3XoUNVu3dx3O2+e6qFDqrt2qW7e7J5vLKWlLr2szOVds0Z1z57we9u/333vZWWqBw+qbtni1tOFwkLV885z62Vlql26qH7nO+5e/X1s2eKeUU1QVqa6apXqzp01c74owFxNoquiddg4WURygGXAMGAdMAf4oaouSpS/sLBQ51Z30PgsZ/5858n+3/8bpqlWPU/Af/6nCw09+WTl+crK3JDS11zjXoOjrFnjPNiTT676ekVFrjfz0qWu1/B//mfF+oRE7NzpPOxLLnHn37vXvT2cc044jMKWLa5T2ssvh8eJuMrgXr3cm8HWrfDznzvP+9RTnQ2qzgO//Xb3FtOpU3zfgCuugOeeg/vugwcfDCe5j+XCC90bhW+ddPHFcMst8F//5UJl4Jqwvvkm3HVXeNzRR7v7mzrVDWHdpEk4vWQiGjd2+5o1C+db9ver6p6nL+ZimTnThatuvz1M87PLefLynE1HHeVCcb4OIjfXXaO42G23bOmeha9Ib948HM/IN93NyXHf15497k0pN9fZ7T9NmlT8zZSVue+yqCgcD8pPd9qokatvato0eau5VNi717UUa9nSXX/DBhc2/PWv3f4bb3QNI0TctY4+2jXr9aPHpjrhUpT9+91bnH+r/vprd74OHdx606bhVKoXX1z1/zEZIjJPVQsT7qtj0R8ITFDV4cH2XQCq+utE+U30M5v9+92fOLblUnUoLnbiF/vn//RT16KpSxf3uu47w3kOHXKCkpdXMVS1YIETmeOPd8K4b5/7Q+bkuJCIb3VVWurivTt3OhHKz3dhpuOPd4XZggWuI2CPHnDWWaEAf/qp+3MPH+6Ebc0ad+7cXCcqzZq5Z/L22+EQH7NmuZBRXl44O1vr1s6e555zBYjv/XzggCvEzj3XVf4//rgLe33rW05w58xxojNhgrP9V7+CwkAa5s1zjQfy893QGTt3uuXu3e4cxx7r7Ny929l+8snOLi9iPsS3aZM7R+vWrt5k+3b3vPPzXZ7mzd09FhWFn9ixo0Sc+DVv7oRW1Qlibq67dnGx+xzJQHnNmrn73b07rAe75powrLh4sZvy9IQTnI3bt7uhXPbvd82gD6e1WNOm0K5d2Hu9f39X8Cxf7mwpLnYFbH4+DBwI1157ePfWkET/e8AIVb0u2P4R8A1VvSmSZywwFuCkk07qtyY6WplhGIZRJZWJfoOryFXVJ1W1UFUL2/hG3oZhGEaNUNeivx6IjkXZIUgzDMMw6oC6Fv05QFcR6SwiucDlQIrzPhmGYRhHSuO6vJiqHhKRm4C/AznAJFX9si5tMAzDyGbqVPQBVPVtIEM7OBuGYTRsGlxFrmEYhlF7mOgbhmFkESb6hmEYWUSdds6qLiKyFTiS3lnHAQ1xKgSzq3qYXdWnodpmdlWPw7Wrk6om7OjUoEX/SBGRucl6pdUnZlf1MLuqT0O1zeyqHrVhl4V3DMMwsggTfcMwjCwi00X/MAcmrXXMruphdlWfhmqb2VU9atyujI7pG4ZhGBXJdE/fMAzDiGCibxiGkUVkpOiLyAgRWSoiK0RkXD3a0VFEZovIIhH5UkRuCdIniMh6EZkffC6qJ/tWi8jngQ1zg7TWIjJDRJYHy2Pq2KbTIs9lvojsFpGf1sczE5FJIrJFRL6IpCV8PuJ4JPjNLRSRvnVs10MisiS49usi0ipILxCRoshze7y27KrEtqTfnYjcFTyzpSIyvI7tmhqxabWIzA/S6+yZVaIRtfc7SzZ5brp+cKN3rgS6ALnAAqBbPdnSDugbrLfAzQ/cDZgA/FcDeFargeNi0v6bYMJ6YBzwYD1/l5uATvXxzIBzgb7AF1U9H+Ai4H8BAc4CPq5juy4AGgfrD0bsKojmq6dnlvC7C/4LC4CmQOfgf5tTV3bF7P8f4J66fmaVaESt/c4y0dMfAKxQ1VWqehB4CRhZH4ao6kZV/TRY3wMsBtrXhy3VYCQwJVifAoyqR1uGAitVtV7mzFTV94HtMcnJns9I4Fl1fAS0EpF2dWWXqk5XVT+F+ke4CYrqnCTPLBkjgZdUtVhVvwJW4P6/dWqXiAjwfeDF2rh2ZVSiEbX2O8tE0W8PrI1sr6MBCK2IFAB9gI+DpJuC17NJdR1CiaDAdBGZJ25uYoC2qroxWN8EtK0f0wA3yU70j9gQnlmy59OQfnfX4LxBT2cR+UxE3hORQfVkU6LvrqE8s0HAZlVdHkmr82cWoxG19jvLRNFvcIjIUcCrwE9VdTfwGHAy0BvYiHu1rA/OUdW+wIXAjSJybnSnuvfJemnTK25mtUuAl4OkhvLMyqnP55MMEfkFcAh4IUjaCJykqn2AW4E/iUjLOjarwX13MfyQis5FnT+zBBpRTk3/zjJR9BvUPLwi0gT3Zb6gqq8BqOpmVS1V1TLgKWrplbYqVHV9sNwCvB7Ysdm/LgbLLfVhG64g+lRVNwc2NohnRvLnU++/OxG5Cvg2MDoQCoLQybZgfR4ubn5qXdpVyXfXEJ5ZY+BSYKpPq+tnlkgjqMXfWSaKfoOZhzeIFT4DLFbV30bSozG47wJfxB5bB7bli0gLv46rCPwC96zGBNnGAG/UtW0BFbyvhvDMApI9n2nAlUHrirOAXZHX81pHREYAdwCXqOr+SHobEckJ1rsAXYFVdWVXcN1k39004HIRaSoinQPbPqlL24BvAUtUdZ1PqMtnlkwjqM3fWV3UUNf1B1fDvQxXQv+iHu04B/dathCYH3wuAp4DPg/SpwHt6sG2LriWEwuAL/1zAo4FZgLLgXeA1vVgWz6wDTg6klbnzwxX6GwESnCx02uTPR9ca4rfB7+5z4HCOrZrBS7W639njwd5/7/g+50PfAp8px6eWdLvDvhF8MyWAhfWpV1B+mTgP2Ly1tkzq0Qjau13ZsMwGIZhZBGZGN4xDMMwkmCibxiGkUWY6BuGYWQRJvqGYRhZhIm+YRhGFmGibxiGkUWY6BuGYWQR/w9trkVUbNViXwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Results:**"
      ],
      "metadata": {
        "id": "sUoyV0qYzLvi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Reward:**"
      ],
      "metadata": {
        "id": "BoXdG6RcxwBk"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_xknGRAyAZiK",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "outputId": "5b46bd1b-ab7e-41bc-9f2d-0322b9dfea5a"
      },
      "source": [
        "plt.figure(tight_layout=True)\n",
        "plt.plot(range(num_episodes), cumulative_rewards, label='cumulative rewards', color='b')\n",
        "plt.xlabel('Episode',size = '14')\n",
        "plt.ylabel('Accumulated reward', size = '14')\n",
        "plt.grid(False)\n",
        "plt.xticks(size = '12')\n",
        "plt.yticks(size = '12')\n",
        "#plt.savefig('R_3.eps',format = 'eps')\n",
        "plt.show()"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwcZbX/8c/JZM8kSCCECCSBsBpkkQCioAjIouJF8AqyXBAR1Csu+BOXixjAjctFcQNZRDZBQEBRBBUEBBRIEIKGxEAICQESAmTfJzm/P06VU73N9Mz0NtPf9+vVr+5+qp7q0zUzdeZZqsrcHRERkUbTr94BiIiIFKMEJSIiDUkJSkREGpISlIiINCQlKBERaUj96x1Ab7f55pv7+PHj6x2GiEiv9cQTT7zm7qPyy5Wgemj8+PFMnTq13mGIiPRaZja3WLm6+EREpCEpQYmISENSghIRkYakBCUiIg1JCSrDzEaa2R1mttLM5prZ8fWOSUSkWWkWX66fAOuA0cAewF1mNs3dp9c3LBGR5qMWVMLMhgHHAF939xXu/jBwJ3BSfSMTEWlOSlDtdgTa3H1WpmwaMDF/RTM73cymmtnURYsW1SxAEZFmoi6+dq3AsryypcDw/BXd/QrgCoBJkybphlp92IYNcP/9sMsusNVW3d/O2rUwaFBu2bp1cNFFsH49vPe98I53xOc9+2x8XmrePGhpKf35jz8Of/4zLFwI++4LRxwBm2zSvnzFCliwAF59FZ57Dm68Efr1g1/+En7zG/jDH+DMM6NuauFCeOQROOQQmDULrrwS3GHnneGjH4UxY2K9hx6C226LmDfZJLbbrx9ssQUMGQKjRsHpp8Mf/whf+lLUGT0attkmvutrr8GLL8KqVdDaGttftAheein213e/C0cdBR//OLzwAuy4Y8S1di1svz3ssEOs/+CDsGQJ9O8f2x88GNraouytb416v/td7OvNN4/v19oKO+0U8S5bFvtn4cKIcezY2Ofr1kWd9Dn7et262CdZ6c9pwACYPz/Wc29fb/Dg+O7pzyQtNyv9XKysXz849tjYPy++CN//PtxyS3znnujfP352bW3xs9m4sfg6Rx8d+//KK+M7jx8P99wT36+STDcsDGa2J/CIuw/NlH0RONDdjyxVb9KkSd6XryTx7W/HL+JHPlK5bT74IFx2WTw23bT8eitWxMHpsMMKl82eDRMmdD8md3j6adhtt/YDwaxZcOKJMGVK/BGeeipcfnn78lKmTYPHHouD1Ac+EAngtNPiYPLf/w2XXBLbuOceuO++eO0O++0XB8nZs+HHP47k8NnPwp/+BFtuCTNmwKOPwve+F7EdeyxsvXWs4x4H9LVrYeTISDgjRsRn/+lPuQfSbbaBV16JA+ncufHdNmyAvfeGI4+ExYvhqqtg+XIYNgxWroyD+bBhcQDv3x9+/euod8QRcVDaeec4yENs69VXI5aNGyMJzJ4d67zlLXFgfvHF2D8jR0YyGDYsPnfmzEhq48bB3XfD/vvD1VfDZpvFgXPlSnj72+P35tlnI+EOHRr7asyYSAgLF8Znt7TA8OHw8MOR5A89NLbz+uuRxJYujfpm8f1Gj47PcI/13SPGgQMLn9PX/fL6oNavj++2fn3s5/SAnSaalStj262t8OY3R4zpzyb7XKws+/zaa/CLX+QmvhNOiOTbE+vWxf4bMCB+Di0theu8/nr8k7NqFRx+eOzTRYvid627zOwJd59UsMDd9Yif8jBigsQOmbLrgO92VG+vvfbyvmzo0PhzueSS0us89ZT7XXfF6zlz3P/wh9Lr3nij+4ABsc3LL+9aLN/4RtSbMSO3/NZbo/xXv2ovmzXL/f77C7fxxz+677GH+777up97rvvy5VF+xRWxje9+t33dww93f9Ob3K+80v3UU2P5Aw+4P/ZYvL/+evdVq9rX37DB/TvfcW9pSQ8x7oMHx3Nrq3v//u4HHdS+rH9/92uucV+82P2yy9y33dZ90qRYp1+/qLPZZu6f/7y7mfv73hfbGzfO/dBD27fzgQ+4v/66+/r17g895H7YYe3Lxo51/9rX3K+91v2ee9ynTYs4f/1r94ED3U85xf2NN9x/8AP3t7416rS0uH/wg+6//a37aae5n3OO+9Kl8R1nznTffXf3zTePbe+yS/s+LOZ3v4t13/EO9yVLuvDD9tjHI0dGrOD+4IOF62zcGI+ObNzovnZt1z67N/jb39y/8hX3q692f/nl2n7266+7P/985bYHTPVix+Vihc36AH4J3JQkq3cSXXwTO6rTlxPUihXxG7LJJvE8Z07hOuvWuY8fHwfbp5+Og39Li/v8+YXrzpsXB9j99486RxwR5evXu995ZySsJ58sfcDZY4+I4+KL2+utX+++005RPnFiHHzdI7mMHp1bv63N/S1vcd9yS/cDDog6b36z+9Sp7ttsE3G3tMSBcNWqiPVzn4u6q1a5jxrlfvDBEXuaAI45pn37P/xhlH3kI/HH++STcYA//XT3V16JxALu3/ue+8KFpQ8qK1e6v+tdEeO8eVF25plRd/z4qOse/xScf378DPK99loknnR/FLNiRe77jRujrLMD/jPPuA8ZEvE89FDH67rHvmtr63y9fNdcE59x+OGRTFev7vo2pHdQgiovQY0Efg2sBOYBx3dWp7ckqNtv77hlk7VkSRzYZs+O35Azzojn3/62cN2f/SyWDRgQ/+2nB+5zzy1c98QT3QcNcn/hBfcvfjEOOo89lnvAB/dbbimsO29e+/KDD3a/4474zHe/O8qOPz6ef/nLOMAPGhTvV61yv/tu9//4j/YW2M03xzb/+lf3MWPaW3S33+6+ww7uO+4YrQ1w//3v22M477woM3P/y1/iO4L7ffdFshkxIlo2pQ7ws2a5/+Y35f0M8rexdKn7Zz8byaER3H23+1VXVfcznn++/Wd+wAHV/SypLyWoKj16Q4Jav959003j8cYbHa/7pz/Ff8cXXxwHcHC/4YZ4vuiiwu1ut537Xnu1tx4OOyy6orbcsr1b5ZFHIiGB+1e/GmUPPRTvhw2Llsltt7k/91wc5M84I7b9sY9F9527+6WXxvpHHhkJZfvto96AAe57793eOtpll0ik6YFt5szoxkrf77ZbbqvimWdiOwcdFEnhppva1xs4MLeV8dpr0SpLv8Pq1ZFct93Wfc89Y/1Zs7r/c5JcGze6b711/Dz+53/qHY1UkxJUEyeoP/+5/QD9xS8WX2fhwuh6SsdMPv7xaKVAdIFtsUV0V2Vdckksv/POSCg/+lG0JO66K8ovuyz6yVtaIpG8973tYxltbZEYWlpyx4oOO8x9112jhZK2ViZPdt9vP/cJE2Ld9Lvcdlt0naVJ9+abo3zChPZ1/vjH6C7bYw/3L33JfcqUwu++dGm0utyju2ybbfzfLbV8a9bkvv/d7yJp7bJL9VsUzShtGd9zT70jkWpSgurlCWrjxhgs7k5f/plnRuI59tj4L//FF9uXzZsXXW9pN9cBB8SYzqGHxpgQxPoHHBBjR6mXXnIfPjzGkfK7ozZsiK63lpY4eI8bF4Oq+W69Nbrkss4/P5LSJz8Z41pHHNGebL70pUggm2wSLZZinztxYqy7667xfNVV7ltt5X7yyeXvr4suiroXXlh+HamO226LfxiWLat3JFJNSlC9PEGlXWL33de1ehs3xh/4Bz8YrYf82W7/+Z+RvD77Wfd//jPKPvShaBGcf36sv3at+yc+ES2e1MknxzjPc88V/9zlyyOhtbREF1+57rvP/z2mdeCBEf9zz8VMwbT18uijpWcQ3XJL1P/hDyPRnX12vD/vvPJjWLbM/VOfitaZiFRfqQSlE3V7iXnz4nnJkq7Ve/LJOC/jvPPazxOaM6d9+aOPxomQP/hBe9k228C998b5EG96U5zzkZ5A+cYbcY7EzTfHyZOlzj1qbY1tvPwybLtt+fHuu2+ce7F+PbzvfXHuSP5nZE8ozffhD8cJmYccAhdeGOdcQdfOkRo+HC69tPz1RaQ6dKmjXuKVV+J51aqu1ZsyJZ4POihObtxkk/YElZ4wuc8+uXW23jpO0nz22Th5EeJkS4B//StO0lyzBo7v5FrvgwZ1LTlBnLC5xx7x+n3v61pdiIT2/vfHZ48dC+k51Ntt1/VtiUh9KUH1Ei+/HM9dTVD5l8nZdtv2BJUmr733zq2zzTbx/MQT7Qlq553jeeZMuOmmOPjvt1/XYinXUUfBnnvGVQd6YuzYuKoB9OwqEyJSH0pQDWbJErj22sLytAW1cmXXtjd3brSI+iedudtt156gHn88kteee+bW2XrreE4vCQNxra2BA+HWW+PSOccdV3iZl0o55xz4+987v6RQZ8aNi+fW1rhsi4j0LkpQDeb66+GUU+JimVndbUHNndt+oIZoQb3wQsyLmzIFJk6MbrWstAUFcW0yiES2225xfbT+/eHkk7sWRz2MHRvP223X82QnIrWnBNVg5s+P52V511XPH4NauLDwSsrFzJ3bfqCGSFBr1sT2pkwpHH+CuIhlekBPW1AQyWnmzLioZ0+732oh/d7q3hPpnZSgGkzaUlqxIrc8m6AWLIhWTmdXD25ri5ZYfgsK4krab7xROP4EMUtvyy3jdTZBbb55TJao9CX1qyXbghKR3kcJqsEUS1ArVsSsOogE9corMQ07bW2V8tJLcbuDYgnqwgujlVRqplzazZdNUL3NhAkxc7FakzlEpLp0HlSDSRNUdjJE2npKy9NktWZNbt2NG3MnLsydG8/ZBDV+fDzPmhU3yUsnROTbeuuYRJGOQfVGra1x3xyNP4n0TmpBNZh0ckS2BZUmLYgWVLps9er28iuvjNZRduwqTVDZMaghQ9q77zqa6NAXWlAQCVsJSqR3UoJqIMuXt7eOsgkqbUG1tkaCKtaCeuaZOOfpJz9pL0uvPpFNUBBjMq2tcb5RKXvuGVeRSG/tLSJSa0pQDSTblVesBbX99qVbUGnZxRe3v547N87/GTqUHF/7Wty6PH96edZJJ8UY15Ah3fsuIiI9pQTVQLJdefljUIMHx/TvUi2o5csjmbz+Olx9dZTlnwOVev/7O79MUb9+HScwEZFqU4JqINkEld+CGjMmEsbKlaVbULvsEus9/XSUzZtXPEGJiPQGSlANJE1QAwcWT1BDh3bcgho+PGbfvfhinMSbf5KuiEhvogTVQF56KSYvjB7dnqDcYdq0uHJDmqDSZaUS1Pz5Mb169Wq1oESk91KCaiAvvxzjTMOGtSeh2bPj0kL77FPYgsrv4mttjenh8+cXPwdKRKQ3UYJqIGmCam1tnyTx+OPxvM8+kbjK6eJbtgz++c8oV4ISkd5KCaqBZBNU2oJ6/PGYnTdxYvt08UWL4jnbgsomKIBHHolnjUGJSG+lBNVAFi2K85byE9Ree8UtLtIEtXBhPKctqI0bo8WVdvFBJKhhw2DkyNp+BxGRSlGCaiBtbXEl8XQMav36uHFfekuM/ASVtqDS7sBsC2rGjOje02V+RKS3UoJqIBs2xI0B0zGof/wD1q4tTFD5Y1Bpa2v48Nx7OWn8SUR6MyWoBtLWFl15aRffjBlRvttu8Zx/ZYe0BZUmrNbWOIcqvcCrxp9EpDdTgmoQ7jGWlLagVqxovzbfVlvFc/419dIWVJqghg+P57SbTy0oEenNlKAaxMaN8Zy2oDZuhDlzIimliadUgsp28YESlIj0DUpQDaKtLZ5bWtq78p59Ni5xlI4pZRNUa2vxLj5on8mnBCUivVnJO+qa2f2Al7MRdz+oYhE1qQ0b4jnt4oO4622abCA3QY0aFS2strbCLr507Cm9e66ISG/U0S3f/5l53QKcACwAHkvK9gHGADdUJ7TmkiaotIsP4mrk++7bvk6xBLV6dWEX38c+Ft186diViEhvVDJBufuZ6Wsz+z5wLfA5d/dM+SWAzrSpgGwXX5qg3HPvaJudxTdqVDyvWVPYxbfZZnDccdWNV0Sk2sodg/ov4MfZ5JS4FDipsiE1p2wXXzYRZRNUfgsKogWVn6BERPqCchOUAW8tUl6sTLohbUFlu/ggN0ENHBh3uoXcFtSKFZG8WlpqE6uISC10NAaVdTVwlZntADyalL0dOBv4eTUCazbFJklAXBkiZRaJaMWKwi4+tZ5EpK8pN0GdDbwKfA74dlL2CvBd4OIqxNV0ik2SgNwWFBQmqHSSRDpBQkSkr+g0QZlZf+B04AZ3/18zGwHg7suqHVwzKTZJAgoTVDo+ld+CUoISkb6m0zEod28DLgIGJO+XKTlVXrFJEoMGwaab5q43dGiMQ6W30UgnSaiLT0T6mnInSTwK7FXNQJpdtouvpQUGD869ikRq6NBIRkOGxPt0koRaUCLS15SboK4E/s/MPm9mB5jZ27KPSgVjZp8xs6lmttbMrimy/GAzm2lmq8zsfjMbl1k2yMyuNrNlZrbAzM6qVN1ayHbxQSSh/O49KExQakGJSF9V7iSJG5Pn7xVZ5sSVJirhZeCbwGHAkOwCM9scuB04DfgtcAFwMzGbEGAysAMwDtgSuN/MnnH3e3pSt0Lfq1PZLj6AESNyZ/Cl0ovHDh4c7zUGJSJ9VbkJatuqRpFw99sBzGwSsHXe4qOB6e5+a7LOZOA1M9vZ3WcCJwOnuPtiYLGZXQmcAtzTw7o1kT0PCuCnPy3egjr2WFiwoD1BaRafiPRVZSUod59b7UDKMBGYlr5x95VmNhuYaGYLiesCTsusPw04qgJ1C5jZ6cTMRsZW6K6A+S2o9763+HonnxzPy5JpKuriE5G+qtwWVDrdfB9gLDAwu8zdr6twXMW0AovyypYCw5Nl6fv8ZT2tW8DdrwCuAJg0aVJZV3zvTHaSRDnSFtTixXHvKLWgRKSvKetwaGY7E2M32xKXPdqQ1F0PrAU6TVBm9gDw7hKLH3H3/TvZxApgRF7ZCGB5six9vyZvWU/r1kT+JInODBgQ083nzIn36W3eRUT6inJn8V0CPAFsAqwCdgEmAU8Bx5SzAXc/0N2txKOz5AQwHdg9fWNmw4AJxNjSYuLKFrtn1t89qdPTujWR38XXGbNoRc2YEe917ycR6WvKTVB7A99095XARqC/u/+duARSxS51ZGb9zWwwMSuwxcwGJ12LAHcAu5rZMck65wJPJ5McIFpx55jZpkmL7xPANRWoWxP5kyTKMWQI/Otf8VoJSkT6mq5czXxV8noRkN4Kbz6wfQXjOQdYDXwFODF5fQ6Auy8iWmvfAhYD+wLZux59A5gNzAUeBC5Kp4n3pG6tdLUFBdGCWrUq6mydP+dRRKSXK/f/9X8S3V7PA48DXzazDURL47lKBePuk4lzkkotvxfYucSytcCpyaOidWuhuwkKIjl1peUlItIblHtY+xaQ3kbvHOAu4H7gNeAjVYir6XS3iw/UvScifVO550H9IfP6eWAXMxsJLC5yl13php60oJSgRKQvKmsMyszekZmsAIC7v6HkVDldPQ8K1IISkb6t3MPhn4H1ZvY34IHk8XhyKw6pgK6eBwVqQYlI31buLL5NgQ8BjwFHEAlrsZn90cy+Wq3gmkl3uvjUghKRvqzcMajVwL3JAzObAPwPMRX8YOA71QqwWXRnkoRaUCLSl5V7qaMtgAOB9yTPY4np5t8iuvukh7rbgtI5UCLSV5X7//oC4gTdy4EzgMeSc4ekQrozSWL33WH+fJ0DJSJ9U7ljUDcSF4X9HHF5o8+Y2V5m+Tckl+7qziSJL3wB7r23OvGIiNRbWQnK3U9097HA24jr2u1B3KH2DTP7TRXjaxrd6eITEenLuto5NAfYHNgCGE2MRx1e4ZiaUne6+ERE+rJyT9Q928x+DywB/gIcSdx+40hgZPXCax7d6eITEenLyv1//UPEbL0fAA8nt92QClIXn4hIrnLPg9qv2oE0u+6cByUi0peVO4sPM3urmf3YzO42szFJ2VFmtmf1wmseakGJiOQqdwzqUGAKcaPCg4DkIjtMIG72Jz2kBCUikqvcFtQFwFnu/iFgXab8AWCfSgfVjNraoF8/0JllIiKh3AS1K/D7IuVvoFl8FbFhg1pPIiJZ5SaoN4juvXxvA+ZXLpzmtWGDJkiIiGR15VJHF5nZ1oAD/c3s3cD/AddVK7hm0tamFpSISFa5Ceoc4ioSc4FW4BninlAPE1c0lx5SF5+ISK5OO5XMrB+wA3EV868T3Xr9gCfd/dnqhtc82trUxSciklXOIdGBp4C3uPtzwPPVDak5qQUlIpKr0y4+d3fgX8Co6ofTvDRJQkQkV7ljUGcD/2dme+geUNWhSRIiIrnK/Z/9FmAwcQXzNjPLuZuuu4+odGDNRl18IiK5yk1Qn6lqFKIuPhGRPOVezfzaagfS7NTFJyKSq+yrmUt1qYtPRCSXElSD0HlQIiK5lKAahFpQIiK5lKAahCZJiIjkUoJqEJokISKSq+T/7GZ2dbkbcfdTKxNO81IXn4hIro46lfIvbfQuYCPwj+T9rkQL7C9ViKvpqItPRCRXyUOiux+ZvjazrwKrgY+5+8qkbBjwM9oTlvRAWxsMGlTvKEREGke5Y1CfBSanyQkgeX0BcGY1Ams26uITEclVboJqBd5cpHwMMLRy4TQvnQclIpKr3AR1G/BzMzvOzMYnj+OILr7bqxde81ALSkQkV7n/s38KuBi4BhiQlLURCer/VT6s5qNJEiIiucpqQbn7anf/NLAZsGfyGOnun3b3VZUIxMwGmdnPzGyumS03s6fM7Ii8dQ42s5lmtsrM7jezcXn1rzazZWa2wMzOqlTdWtB5UCIiubp6ou6Q5PGv7ISJCukPvAi8G9gEOAe4xczGA5jZ5kR34teBkcBU4OZM/cnADsA44D3A2WZ2eE/r1oq6+EREcpWVoMxsuJndCrwK/BXYKin/qZlNrkQg7r7S3Se7+wvuvtHdfwfMAfZKVjkamO7ut7r7GiKp7G5mOyfLTwYucPfF7j4DuBI4pQJ1a0KTJEREcpXbgrqQmMX3NuJ8qNTvgA9VOigAMxsN7AhMT4omAtPS5UkLbjYw0cw2JWYUTstsYlpSp6d1i8V2uplNNbOpixYt6t4XzKMWlIhIrnIT1AeBz7v7U4BnymcA21U6KDMbAPwCuNbdZybFrcDSvFWXAsOTZeQtT5f1tG4Bd7/C3Se5+6RRo/IvuNE9miQhIpKr3AS1KfB6kfLhwIZyNmBmD5iZl3g8nFmvH3A9sI7cW82vAEbkbXYEsDxZRt7ydFlP69aEJkmIiOQqN0FNIVpRqbQVdQYxJtUpdz/Q3a3EY38AMzNi6vpo4Bh3X5/ZxHRg9/RNcqmlCcTY0mLglezy5PX0CtStCXXxiYjkKjdBfQ24wMyuJGbbnWVmfwZOImbbVcplwC7Ake6+Om/ZHcCuZnaMmQ0GzgWeznQBXgecY2abJpMfPkGct9XTujWhLj4RkVzlngf1V+AdwEBicsHBwMvAfu7+90oEkpyXdAawB7DAzFYkjxOSGBYBxwDfAhYD+wLHZTbxjSS2ucCDwEXufk9P69aKuvhERHKV/T+7u/+DmI5dFe4+F7BO1rkX2LnEsrXAqcmjonVrQV18IiK5yj0PaoOZbVGkfDMzK2uShHRM50GJiOQqdwyqVMtmEDHbTnpILSgRkVwd/s+euSadA580sxWZxS3AAcDMgorSZZokISKSq7NDYnozQgNOI/ecp3XAC8AnKx9Wc3GHjRvVghIRyeowQbn7tgBmdj9wdHLOkFTYhiTtK0GJiLQrq1PJ3d9T7UCaWZqg1MUnItKu7EOime0IfBgYS5wP9W/uXrfp2X1BW1s8qwUlItKurARlZu8nbvv+JHH7iynEpYIGAQ9VLbomoS4+EZFC5U4zPx84z933A9YSlzgaD9wLPFCVyJpI2oJSF5+ISLtyE9ROtN+Bdj0wNLnx3/nA56sRWDNRC0pEpFC5CWo5MDh5/QqwffK6P3ErDukBTZIQESlU7iHxMWB/4BngLuBiM9uduJvu36oUW9PQJAkRkULlJqizaL/z7GTiRoXHALOSZdID6uITESlU7nlQz2derwI+VbWImpC6+ERECpU7BiVVpC4+EZFCJf9nN7PltN/avUPuPqJiETUhdfGJiBTqqFPpMzWLosnpPCgRkUIlD4nufm0tA2lmakGJiBTSGFQD0CQJEZFC5V6Lr8PxKI1B9YwmSYiIFCr3f/b88agBwJ7EuVDfqmhETUhdfCIihco9D6roeJSZ/R04GPhRJYNqNuriExEp1NMxqPuBIysRSDNTF5+ISKGeJqjjgNcqEUgzUxefiEihcidJ/IPcSRIGjAZGosse9ZjOgxIRKVTuIfFXee83AouAB9x9ZmVDaj5qQYmIFCp3ksR51Q6kmWmShIhIoS4fEs1sMHljV8kVzqWbNElCRKRQWZMkzGycmf3GzJYBK4k77GYf0gPq4hMRKVRuC+oG4pbvZwILKfMq51IedfGJiBQq95C4J7C3u8+oZjDNSl18IiKFyj0PahowqpqBNDO1oERECpV7SDwd+KGZ/RD4J7A+u9Dd51U6sGaiFpSISKFyE1Q/4sTcOyg8YdcBHVp7QJMkREQKlZugrgVeBb6MJklUnLr4REQKlXtI3BnYw91nVTOYZqUuPhGRQuVOkngc2LaagTQzdfGJiBQqtwV1GXCJmV0M/IPCSRJ/r3RgzUQXixURKVTuIfGm5PmKIss0SaKH1IISESlUbhffth08tqtUMGZ2g5m9YmbLzGyWmZ2Wt/xgM5tpZqvM7H4zG5dZNsjMrk7qLjCzsypVt9o0SUJEpFBZCcrd53b0qGA83wHGu/sI4IPAN81sLwAz2xy4Hfg6cR+qqcDNmbqTgR2AccB7gLPN7PCe1q0FTZIQESlU7g0Lj+5oubvfXolg3H169m3ymAA8ARwNTHf3W5OYJgOvmdnOyT2pTgZOcffFwGIzuxI4Bbinh3Wrbt26SE5KUCIi7bp7w8JUej5UxQ6tZnYpkRyGAE8Cv08WTSQuuRQf7L7SzGYDE81sITAmuzx5fVQF6haL8XTi6hqMHTu2618yz9q1MHBgjzcjItKnlNvF1y/7AAYC+wIPAe+qZEDu/mlgOHAA0S23NlnUCizNW31psm5r5n3+sgDsFZUAABAoSURBVJ7WLRbjFe4+yd0njRrV80sUrlunBCUikq/cSRI53L3N3acAXwMuLaeOmT1gZl7i8XDe9je4+8PA1sCnkuIVwIi8zY4g7ke1IvM+f1lP61bdunUwaFCtPk1EpHfoVoLKWEKMEXXK3Q90dyvx2L9Etf6Z7U8Hdk8XmNmwZNn0ZOzolezy5PX0CtStOrWgREQKlXtH3bflPfYysw8AlxPjRD1mZluY2XFm1mpmLWZ2GPBR4L5klTuAXc3smOS28+cCTyeTHACuA84xs03NbGfgE8A1FahbdRqDEhEpVO4kianEhAjLK38U+FiFYnGiO++nROKcC3ze3e8EcPdFZnYM8GPiDr+PAcdl6n+DuOLFXGA1cKG739PTurWgFpSISCFz7/zC5NmTWhMbgUXuvqYqUfUikyZN8qlTp/ZoG0cdBS+8AE89VZmYRER6EzN7wt0n5ZeX1YKq8Mm4kkddfCIihcodg/qWmX2ySPknzeyCyofVXNTFJyJSqNxZfCdRfDLEE8B/VS6c5qRp5iIihcpNUFsAi4qUv07cCl56QC0oEZFC5SaoecSVHfK9C5hfuXCak8agREQKlTvN/HLg+2Y2EPhzUnYwcfXxC6sRWDNRC0pEpFC5s/guTm5Z8UPiOnwA64AfuPv/Viu4ZqExKBGRQmXfIs/dv2pm3wTekhTNcPcVHdWR8qgFJSJSqNz7QW0J9Hf3+cCUTPnWwHp3X1il+JqCxqBERAqVO0niBuCIIuWHAddXLpzmpBaUiEihchPUJOAvRcofSpZJD2gMSkSkULkJqj9Q7BA6uES5dIFaUCIihcpNUI/RfuPArP8mMyYlXbdxI7S1KUGJiOQrdxbf/wB/NrPdaD8P6iBgT+CQagTW133ta3DbbTBtWrxXghIRyVVWC8rdHwX2A+YARyePOcB+7v7X6oXXd61bBy+9FM+gMSgRkXxdOQ9qGnBifrmZHeLu91Y0qiYwdCisXh1TzEEtKBGRfGUnqCwz24q4k+6pwDigpZJBNYMhQ2L8afnyeK8EJSKSq9xJEphZi5kdbWa/B14APkTcnn37KsXWpw0dGs9LlsSzEpSISK5OW1BmthNwGnHfp5XAjcB7gZPc/Znqhtd35ScojUGJiOTqsAVlZg8BjwKbAh9x9+3c/ZyaRNbHDRkSz0uXxrNaUCIiuTprQe0H/AS4wt2n1yCeppG2oBYvjmclKBGRXJ2NQe1NJLGHzexJM/tCcuFY6SGNQYmIdKzDBOXuT7r7fwNjgO8BHwReTOq938w2rX6IfZPGoEREOlbuibpr3P16d38PsAtwEfAFYIGZ3V3NAPsqjUGJiHSs7GnmKXd/zt2/AmwDfIS4s650kcagREQ61q0TdQHcfQPwm+QhXaQxKBGRjnW5BSWVoTEoEZGOKUHVSToGpRaUiEhxSlB1ogQlItIxJag6GTAgHkpQIiLFKUHV0ZAh7Vcz1xiUiEguJag6SidKgFpQIiL5lKDqKJugBgyoXxwiIo1ICaqO0gTV0hIPERFppwRVR+lMPo0/iYgUUoKqo7QFpfEnEZFCSlB1pAQlIlKaElQdKUGJiJSmBFVHGoMSESmtYROUme1gZmvM7Ia88uPNbK6ZrTSzX5vZyMyykWZ2R7JsrpkdX6m61aAWlIhIaQ2boICfAFOyBWY2EbgcOAkYDawCLs2rsy5ZdgJwWVKnR3WrRQlKRKS0bt8PqprM7DhgCfBXYPvMohOA37r7X5L1vg7MMLPhwEbgGGBXd18BPGxmdxIJ6Ss9rFsVaRefEpSISKGGa0GZ2QjgfOCsIosnAtPSN+4+m2j17Jg82tx9Vmb9aUmdntbNj/F0M5tqZlMXLVrUtS+YkbagNAYlIlKo4RIUcAHwM3efX2RZK7A0r2wpMDxZtqzEsp7WzeHuV7j7JHefNGrUqA6+SsfUxSciUlpNE5SZPWBmXuLxsJntARwCfL/EJlYAI/LKRgDLO1nW07pVoQQlIlJaTceg3P3Ajpab2eeB8cA8M4No2bSY2Vvc/W3AdGD3zPrbAYOAWcQ4Un8z28Hdn01W2T2pQw/rVoXGoERESmu0Lr4rgAnAHsnjp8BdwGHJ8l8AR5rZAWY2jBirut3dl7v7SuB24HwzG2Zm7wT+A7i+AnWrQmNQIiKlNVSCcvdV7r4gfRBdb2vcfVGyfDrwSSLZvEqMEX06s4lPA0OSZTcBn0rq9KhutaiLT0SktIacZp5y98lFym4Ebiyx/hvAUR1sr9t1q0EJSkSktIZqQTUbjUGJiJSmBFVHGoMSESlNCaqO1MUnIlKaElQdqYtPRKQ0Jag6UhefiEhpSlB11NoKm20G22xT70hERBpPQ08z7+sGDIA5c9pbUiIi0k4Jqs6GF70crYiIqItPREQakhKUiIg0JCUoERFpSEpQIiLSkJSgRESkISlBiYhIQ1KCEhGRhqQEJSIiDcncvd4x9GpmtgiY283qmwOvVTCcSmrU2Bo1Lmjc2Bo1LlBs3dGocUH3Yxvn7qPyC5Wg6sjMprr7pHrHUUyjxtaocUHjxtaocYFi645GjQsqH5u6+EREpCEpQYmISENSgqqvK+odQAcaNbZGjQsaN7ZGjQsUW3c0alxQ4dg0BiUiIg1JLSgREWlISlAiItKQlKBERKQhKUHVgZmNNLM7zGylmc01s+PrFMcgM/tZEsNyM3vKzI5Ilo03MzezFZnH12sc3wNmtibz+f/KLDs+iXulmf3azEbWKKYVeY8NZvajZFlN95mZfcbMpprZWjO7Jm/ZwWY208xWmdn9ZjYus2yQmV1tZsvMbIGZnVWr2Mzs7Wb2JzN7w8wWmdmtZjYms3yyma3P24fb1Si2Dn9+1d5vHcR1Ql5Mq5I490qWV3WfdXScSJZX7XdNCao+fgKsA0YDJwCXmdnEOsTRH3gReDewCXAOcIuZjc+s8yZ3b00eF9Q+RD6T+fydAJJ9dTlwErEPVwGX1iKYTCytwJbAauDWvNVqtc9eBr4JXJ0tNLPNgduBrwMjganAzZlVJgM7AOOA9wBnm9nhtYgN2JSY6TU++fzlwM/z1rk5u5/d/fkaxZYq9fObTHX3W9G43P0Xeb93nwaeB/6eWa2a+6zkcaLqv2vurkcNH8AwIjntmCm7HvhuvWNLYnkaOIY4gDjQv46xPACcVqT828CNmfcTkn06vMbxnUwcKNLZsHXZZ8RB7ZrM+9OBv2beDyMS6c7J+5eBQzPLLwB+WYvYiix/G7A8834ycEOd9luHP79a7bcy9tn9wDfqsc8yn5keJ6r6u6YWVO3tCLS5+6xM2TSgHi2oHGY2mohveqZ4rpnNN7OfJ/8t1dp3zOw1M3vEzA5MyiYS+wwAd59NkvRrHNvJwHWe/OVl1Huf5e+flcBsYKKZbQqMyS6nvr9/7yL39w3gyKQLcLqZfaoOMRX8/BplvyXdZ+8CrstbVLN9lnecqOrvmhJU7bUCy/LKlgLD6xDLv5nZAOAXwLXuPpO44OPeRNN8LyK+X9Q4rC8D2wFbEd1CvzWzCcQ+XJq3bk33YXKgeDdwbaa4EfYZdLx/WjPv85fVlJntBpwLfClTfAuwCzAK+ARwrpl9tEYhdfTza5T99l/AQ+4+J1NWs31W5DhR1d81JajaWwGMyCsbQfTF14WZ9SO6GdcBnwFw9xXuPtXd29x9YVJ+qJnV7A/S3R9z9+XuvtbdrwUeAd5HY+zDk4CHsweKRthniY72z4rM+/xlNWNm2wN3A59z94fScnd/xt1fdvcN7v5X4AfAh2sRUyc/v4bYb0SCyv5TVLN9Vuw4QZV/15Sgam8W0N/MdsiU7U5hN0dNmJkBPyMmGxzj7utLrJp2Y9Xzd8YBI/bV7mlhMmNpELFva6XgQFFEvfZZ/v4ZRozTTXf3xcAr2eXU+PcvaX3eC1zg7td3snr6M6+Hf//8GmS/vRN4M/CrTlat+D7r4DhR3d+1Wg6s6fHvgcJfAjcRA4rvJJq9E+sUy0+BR4HWvPJ9gZ2Ig+tmxMyc+2sY15uAw4DBxCyiE4CVRN/3RKKb9IBkH95AlQb5S8T2jiSW4XnlNd1nyX4ZDHyH+M823Vejkt+pY5KyC4FHM/W+CzxIzKjbOTmIHF6j2LYixij+X4l6/5HEZcA+wEvAyTWKrcOfX7X3W6m4MsuvIMY867HPSh0nqvq7VpU/HD06/WGPBH6dHOTmAcfXKY5xxH9ba4jmePo4AfgoMCeJ8RViUHbLGsY2CphCdAcsSf443ptZfnyy71YCvwFG1jC2y4Hri5TXdJ8Rs7c87zE5WXYIMJOYUfUAMD5TbxAxlXkZsBA4q1axAd9IXmd/31Zk6t0EvJ6UzwQ+W8PYOvz5VXu/dfLzHJz8HRxcpF5V91lHx4lq/67pYrEiItKQNAYlIiINSQlKREQakhKUiIg0JCUoERFpSEpQIiLSkJSgRESkISlBifRiyX2BqnYpIDOblHzG+Gp9hkgpSlAidWJm1yQH//zHo13YzBjgt9WKUaSe+tc7AJEmdy9x4dmsdeVWdvcFlQ1HpHGoBSVSX2vdfUHe4w34d/fdZ8zsruR22nPN7MRs5fwuPjM7N1lvbXKL7esyywaZ2SVmttDM1pjZo2a2f972Dk9u373GzB6iyD22zOwdZvZgEtNLZnaZmeVf0Vqkx5SgRBrbecCdwB4kFws1s0nFVjSzY4D/R9wSfAfgA8DjmVX+FzgWOBXYE/gHcI+ZjUnqb0NcI/JPyef9KKmT/Yy3An9MYtodODpZt9Tt00W6TdfiE6kTM7sGOJG4CGfWT9z9y2bmwFXu/olMnXuBBe5+YvLegf9091+Z2VnAGcCunnfblOQ2CIuB09z9uqSshbhFyU3ufo6ZfZu4j9BOnhwYzOwc4jbd27r7C0mLbL27fzyz7T2AJ4HR7v5qZfaOiMagROrtL8DpeWVLMq//lrfsb8D7S2zrVuBzwBwz+wNwD3Cnu68l7tEzgLjpIwDuvsHM/ga8JSnahbhVQva/1vzP3wvY3syOzZSl9x6aAChBScUoQYnU1yp3f64SG3L3F81sJ+Bg4hYIFwPfMLN9O6vahY/pB1wFfL/Ispe6sB2RTmkMSqSxvb3I+xmlVnb3Ne5+l7t/AdibuLnjO4mbBK5LXgP/7uLbD3gmKZoB7JvcPbXU5/+duLnmc0Ueq7vx/URKUgtKpL4GmdmWeWUb3H1R8vpoM5tC3Ajuw0TrqGiLyMxOIf6mHyNuKHcssB541t1XmtllwIVm9hpxY74vELfwvjTZxE+BLwKXmNmlwFuBT+Z9zIXAo2b2U+LGjcuJO6Ue6e5ndP3ri5SmBCVSX4cQd2/NegnYOnk9mbid9g+BRcDH3H1KiW0tAb4M/B8x3vQMcLS7z0mWfzl5/jnwJmJiw+Hu/gqAu88zs6OB7xGTLZ4AvgLckH6Auz9tZu8CvkncyrsFeB64o6tfXKQzmsUn0qCyM/TqHYtIPWgMSkREGpISlIiINCR18YmISENSC0pERBqSEpSIiDQkJSgREWlISlAiItKQlKBERKQh/X+/BQlrNg95gwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Number of steps taken by the agent to collect data of all sensors:**"
      ],
      "metadata": {
        "id": "h_bwB6uqx0Va"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6lgovK18Mm0P",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "outputId": "4b9d002b-0f1f-405b-8724-ac757cb88e60"
      },
      "source": [
        "plt.figure(tight_layout=True)\n",
        "plt.plot(range(num_episodes), Num_steps, color='b')\n",
        "plt.xlabel('Episode',size = '14')\n",
        "plt.ylabel('Taken steps', size = '14')\n",
        "plt.grid(False)\n",
        "plt.xticks(size = '12')\n",
        "plt.yticks(size = '12')\n",
        "plt.savefig('S_3.eps',format = 'eps',dpi=1200)\n",
        "plt.show()"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhcZZn38e9NJ+mELJJACBEkAZIQCJIoYUAQgYkCOuMyoA4GEVwAQeZVGRVfBgHBDfdxVBQHZQdBAVleUTIiqBgkbDphSdjCmhBIJ+nudLqz3O8f9znW6eqq7urqOt3V3b/PddVVVefUqXr6pFO/vp/nOeeYuyMiIlJvthnoBoiIiJSigBIRkbqkgBIRkbqkgBIRkbqkgBIRkbo0YqAbMFB22GEHnz59+kA3Q0Rk2Lv//vtfcffJxcuHbUBNnz6dJUuWDHQzRESGPTNbUWq5uvhERKQuKaBERKQuKaBERKQuKaBERKQuKaBERKQuKaBERKQu9WtAmdnvzWyjmbUkt8cz6xaa2QozazWzm8xsUmbdJDO7MVm3wswWFr1v2W1FRGRwGogK6nR3H5fc9gQwsznAj4HjgSnABuCHmW1+AHQk644DLkq2qWRbEREZhOqli+844BZ3v9vdW4AvAEeb2XgzGwscA3zB3Vvc/Y/AzUQgdbtt3o3+1rfgIx/J+1NERIangQior5rZK2b2JzM7LFk2B3g4fYG7P0lUTLOS22Z3X5Z5j4eTbXrathMzO9nMlpjZktWrV/f5B/nzn+EPf+jz24iISAn9HVBnArsDOwMXA7eY2R7AOGBd0WvXAeOTdevLrKOHbTtx94vdfb67z588uctpn3qtvR02b+7z24iISAn9ei4+d7838/QyM/sA8A6gBZhQ9PIJQDOwtZt19LBtrjo6YMuWvD9FRGR4GugxKAcMWArMTRea2e5AI7AsuY0ws5mZ7eYm29DDtrnq6FAFJSKSl34LKDPbzsyONLPRZjbCzI4D3gLcDlwFvNPMDkkmRZwP3ODuze7eCtwAnG9mY83sYODdwBXJW5fdNu+fqb1dFZSISF76s4tvJPAlYDawBXgMeE86+cHMPk6EzfbAIuDDmW1PA34KvAy8Cpzq7ksB3H1pD9vmRl18IiL56beAcvfVwP7drL8auLrMujXAe6rZNk/q4hMRyc9Aj0ENaqqgRETyo4DqA00zFxHJjwKqD1RBiYjkRwHVBxqDEhHJjwKqD9rbwT1uIiJSWwqoPujoiHt184mI1J4CqkruhYBSN5+ISO0poKq0eXOha08VlIhI7SmgqpRWT6AKSkQkDwqoKmUDShWUiEjtKaCqpApKRCRfCqgqtbcXHquCEhGpPQVUldTFJyKSLwVUldTFJyKSLwVUldTFJyKSLwVUlVRBiYjkSwFVJY1BiYjkSwFVpWwXnyooEZHaU0BVSRWUiEi+FFBVUkCJiORLAVUlTZIQEcmXAqpKmmYuIpIvBVSVVEGJiORLAVUljUGJiORLAVUlTTMXEcmXAqpKqqBERPKlgKqSAkpEJF8KqCqpi09EJF8KqCqpghIRyZcCqkqaZi4iki8FVJV0oK6ISL4GJKDMbKaZbTSzKzPLFprZCjNrNbObzGxSZt0kM7sxWbfCzBYWvV/ZbfOiCkpEJF8DVUH9ALgvfWJmc4AfA8cDU4ANwA+LXt+RrDsOuCjZppJtc9HRAWbxWBWUiEjtjejvDzSzY4G1wD3AjGTxccAt7n538povAI+a2XhgK3AMsI+7twB/NLObiUD6fHfbuntzXj9HRwdsuy20tiqgRETy0K8VlJlNAM4HzihaNQd4OH3i7k8SFdOs5LbZ3ZdlXv9wsk1P2+amvT0CCtTFJyKSh/7u4rsAuMTdny9aPg5YV7RsHTA+Wbe+zLqetu3EzE42syVmtmT16tVVNL8graBAFZSISB76LaDMbB7wVuA7JVa3ABOKlk0AmntY19O2nbj7xe4+393nT548uXc/QJFsQKmCEhGpvf4cgzoMmA48azG7YBzQYGZ7A7cDc9MXmtnuQCOwjBiDGmFmM919efKSucDS5PHSbrbNTXs7jBkTj1VBiYjUXn8G1MXAtZnnnyEC61RgR+DPZnYI8AAxTnVDOsnBzG4AzjezjwHzgHcDByXvc1V32+ZFFZSISL76rYvP3Te4+8r0RnTNbXT31e6+FPg4ETYvE+NHp2U2Pw0Yk6y7Bjg12YYKts1FR4cqKBGRPPX7NPOUu59X9Pxq4Ooyr10DvKeb9yq7bV6ys/gUUCIitadTHVVJXXwiIvlSQFWpowMaG2GbbVRBiYjkQQFVpY4OGDUKGhpUQYmI5EEBVaX29qigGhpUQYmI5EEBVaW0ghoxQhWUiEgeFFBVynbxqYISEak9BVQVtm6FTZtUQYmI5EkBVYVNm+JeY1AiIvlRQFUhvZpuWkEpoEREak8BVYX29rjXNHMRkfwooKqQVlDq4hMRyY8CqgrFXXyqoEREak8BVYVsQKmCEhHJhwKqCtkxKFVQIiL5UEBVQWNQIiL5U0BVQdPMRUTyp4CqgqaZi4jkr6KAMrPJZjY58/z1ZvYlM/tAfk2rX5okISKSv0orqOuAdwKY2Q7A3cC/AD8ys3/PqW11a/Zs+MEPYI89NElCRCQvlQbUvsDi5PF7gSfcfQ7wIeCUPBpWz3bdFU47DXbaSRWUiEheKg2oMUBL8vitwM3J4weA19W6UYOJKigRkXxUGlDLgaPN7HXAEcBvk+VTgLV5NGywUAUlIpKPSgPqi8CFwDPAYne/N1l+JPBgDu0aNDTNXEQkHyMqeZG732BmuwKvBR7OrFoE/DKPhg0WmmYuIpKPigIKwN1XAavMbJyZ4e4tmUpq2FIXn4hIPio+UNfMPmVmzwLrgHVm9pyZfdrMLL/m1T9NkhARyUdFFZSZfR04GfgG8Odk8ZuAc4CpwOdyad0goApKRCQflXbxfQz4mLv/IrPsd2b2OPBjhnFAqYISEclHb87F99cyy4b1+fxUQYmI5KPScLkc+ESJ5acCV9SuOYOPppmLiOSj0i6+RmChmR1J4ZRHBxDTzq8ys++lL3T3/1PbJtY3TTMXEclHpQE1mzitEcC05H5lctsr8zqvUbsGDVVQIiL5qPRA3cNr8WFmdiWwABhLhNvX3f2/k3ULgB8AuwL3Aie6+4pkXSNwEXGi2g3Jdt/OvG/ZbfOmCkpEJB+9muBgZjuY2QFJYFTjq8B0d58AvAv4kpntl1zC4wbgC8AkYAnw88x25wEziertcOBzZnZU2qYets2VJkmIiOSj0gsWjjez64GXgXuAnZPlPzKz8yr9MHdf6u7t6dPktgdwNLDU3a93941EIM01s9nJa08ALnD3Jnd/FPgJcGKyrqdtc6Vp5iIi+ai0grqQmBDxRqAts/xW4sKFFTOzH5rZBuAx4CXg/wFzyJzjz91bgSeBOWY2kTgYOHsOwIeTbehu2xKffbKZLTGzJatXr+5Ns8tSBSUiko9KA+pdwKfc/SE6T4R4FNi9Nx/o7qcB44FDiK65dmAccQqlrHXJ68Zlnhevo4dtiz/7Ynef7+7zJ0+eXLy6KpokISKSj0oDaiLwaonl44Fefz27+xZ3/yOwC3EsVQswoehlE4BmChdKnFBiHT1smztNkhARyUelAXUfUUWl0irqFGJMqlojiDGopcDcdKGZjU2Xu3sT0RU4N7Pd3GQbutu2D+2q2IhkHuTWrf3xaSIiw0elAXUWcIGZ/YQIlTPM7HfA8cDZlbyBme1oZscml+toSA76/QDwP8CNwD5mdoyZjSZOQvtXd38s2fxy4Gwzm5hMfjgJuDRZ19O2uWpoiHtVUSIitVVRQLn7PcBBwChiAsIC4EXgTe7+QHfbZt+G6M57HmgCvkmMa93s7quBY4AvJ+sOAI7NbHtu8rkrgLuAb7j77Unbeto2V2lAaRxKRKS2enPBwr8R072rkgTJod2sX0ScsaLUunbgI8mtV9vmLe3iUwUlIlJblR4HtcXMdiyxfHszG9a1gyooEZF8VDoGVe6quY1AR43aMiilFZQCSkSktrrt4jOzM5KHDnzczFoyqxuIY5n6ZTJCvdIkCRGRfPQ0BvVvyb0RV9XN1gkdwDPAx2vfrMFDFZSISD66DSh33w3AzO4Ejk6OSZIMVVAiIvmodJr54cXhZGYzkuOOhjVNkhARyUels/i+YmYnJI/NzBYBy4CXzOyAPBtY7zTNXEQkH5XO4jsOeDx5/Hbi1EIHEmd4+FoO7Ro0VEGJiOSj0gN1pxBngAB4B3Cdu//FzNYQFwgctjRJQkQkH5VWUK8SV7MFOII4fx5EwJU7RmpY0CQJEZF8VFpB/RK42syWEZdV/02yfB7wRB4NGyxUQYmI5KPSgDqDOFHrrsDnkqvWQlzp9qI8GjZYqIISEclHRQHl7puBb5VY/p2at2iQ0SQJEZF8VDoGJWVomrmISD4UUH2kCkpEJB8KqD7SJAkRkXwooPpIkyRERPKhgOojVVAiIvmoaBZfclLYTwILgB0pCjZ337f2TRscVEGJiOSj0uOgfgj8C3A9cA9xAUNBkyRERPJSaUC9B3ifuy/KszGDkaaZi4jko9IxqA3Ac3k2ZLBSBSUiko9KA+rrwBlmNqxPDFuKJkmIiOSj0i6+twGHAEeZ2SPApuxKd39XrRs2WGiShIhIPioNqFeAG/NsyGClCkpEJB+Vniz2w3k3ZLBSBSUiko9eHahrZvPN7F/NbGzyfKyZVVqFDUmqoERE8lFRQJnZFDNbDPwFuJq4BDzAtylxGY7hpFQFtWkTfPKTsHLlwLRJRGQoqLSC+g6wCtiemHKeup64BPywVWqa+eOPw/e+B7/85cC0SURkKKi0e24BsMDdm4pmmj9JXGV32CrVxdfREfePPtr/7RERGSoqraDGAB0llk8GNtauOYNPuS4+UECJiPRFpQF1N3Bi5rmbWQNwJvA/tW7UYDJqVNxvyhwZpoASEem7SgPqc8BJZnYH0EhMjHgEOBj4v5W8gZk1mtklZrbCzJrN7CEze3tm/QIze8zMNpjZnWY2rWjbn5rZejNbaWZnFL132W3z1tAAZoVuPSgE1Esvwbp1/dUSEZGhpaKAcvdHgNcTZzL/LTCamCDxBnd/ssLPGkGcz+9Q4DXA2cB1ZjbdzHYAbgC+AEwClgA/z2x7HjATmAYcDnzOzI4CqGDbXJlFFVUqoEBVlIhItSq9HtQ/u/utwLkl1p3l7l/p6T3cvZUImtStZvY0sB8xO3Cpu1+fvOd5wCtmNtvdHwNOAE509yagycx+QnQ53g4c3cO2uSsOqOzjRx+FAw/sj1aIiAwtlXbxXWVmBxcvNLP/AD5TzQeb2RRgFrAUmAM8nK5LwuxJYI6ZTQSmZtcnj+ckj8tuW+IzTzazJWa2ZPXq1dU0u6RRo6C9vfBcFZSISN9VGlCfAG42s79fOdfMzgb+HTiytx9qZiOBq4DLkipnHFA8WrMOGJ+so2h9uo4etu3E3S929/nuPn/y5Mm9bXZZ5br4Ro1SQImIVKvSc/FdaWbbA78xszcDC4EzgCPc/b7efKCZbQNcQUxbPz1Z3AJMKHrpBKA5WZc+31i0rqdt+0VjY+mA2ntvBZSISLUqPhefu/8n8N/AfcCngLdWEU4GXEKcKukYd087w5YCczOvGwvsQYwtNQEvZdcnj5f2tG1v2tYX5cagZs2C55/vr1YMfRs2wNveptAXGS7KVlDFU7kTTUTF8gfgUDM7FMDdv13h510E7EWEW1tm+Y3AN8zsGOA24Bzgr5lJDpcDZ5vZEiLcTgI+XOG2uSvXxTdhQoxNucdsP+mbZ5+FRYvg3nthr70GujUikrfuuvj+rczyLcBByQ3AiZPGdis5NukUoB1YmTll0inuflUSMN8HrgTuBY7NbH4uEW4rgDbgQne/HcDdV/ewbe7KBdT4ZBSsvR1Gj+663d/+Bt/+NvzkJ4VTJkl56T7uKHVOExEZcsp+Lbr7brX8IHdfAZStI9x9ETC7zLp24CPJrVfb9ofuKiiAjRtLB9SiRXDppfDZz8Z4lXQv3a8KKJHhoVfXg5LSeqqgNpY5W2Fb0sn5WL91Rg5u6X7NTukXkaGr4o4lM5sFvJc4e/mo7Dp3L1nZDBejRsUAfioNq54CKl3++OP5tW0oURefyPBS6Zkk/gn4JfAgceaH+4iZco3EhIlhbdQoaGoqPK+0glJA9Y66+ESGl0q7+M4HvujubyImORwPTAcWAb/PpWWDSKkuvoYGGDMmnvfUxaeAqowCSmR4qTSg9qRwAtZNwLbuvpEIrk/l0bDBpNSBuqNGFSZGVFJBuefbxqEg3ccagxIZHioNqGbiDOYQB83OSB6PACbWulGDTakDdUeOrDygmpqghqcGHLJUQYkML90GVHINpvHEsUVvThbfBnzLzM4Ffgb8Od8m1r9SXXyVBFRb5lBldfP1TJMkRIaXniqoE4jLvZ8BLE6WnUdcE+oY4AngY3k1brCoNqA2boRJk+KxAqpnmmYuMrz0NIvPANz9qXSBu28ATs2zUYNNqYCqdAxq1ix48EEFVCXUxScyvFQyBqXh+x70pYIaOxZmzIAnnsi/nYOduvhEhpdKjoPKnjevJHdvqE1zBqdqJ0m0tcHEibDddrB+ff7tHOzUxScyvFQSUCcDa/NuyGA2ahRs2RK3hobeVVCjR0cVpYDqmbr4RIaXSgLqFnd/OfeWDGKNjXHf0REH5/Y2oDZvhpUr+6etg5m6+ESGl57GoDT+VIFRyZkJ0y/OdJJEGlzddfGNGQPbbgutrfm3c7BTBSUyvFQ0i0+6VxxQ6RiUWYRUTxUUdD7ZrJSmMSiR4aXbgHJ3XY6jAqUqqLR6Gj2654DaZhtVUJVQF5/I8KIAqoFSATVyZDwuF1DuhS6+sWNVQVVCXXwiw4suNF4D5cagoHxApa8dPRq2bo2JEtlgk650sliR4UUVVA2UG4OC8gGVLkunmYO6+XqiCkpkeFFA1UA1XXzpiWLTWXygbr6eKKBEhhd18dVAOiEi7XqqJKCyFdSI5F9BFVT3NElCZHhRQNVANRVUNqDSgFMF1T1NMxcZXtTFVwPVTJJIu/hGjy508amC6p66+ESGFwVUDfRlkkQ6zRxUQfUk3b9bt8Z5D0VkaFNA1UBfu/g0SaIyaQUF6uYTGQ4UUDXQl1l8mmZeuWxAqZtPZOhTQNVANWNQ2S4+VVCVKb7mlogMbQqoGsgG1JYtMUZSTRefKqjuqYtPZHhRQNVANqDSL9FqziRR6wpqwwaYMgVuuaW27ztQVEGJDC8KqBrIHqhbKqDa2+PksFnZM0k0NsalOWodUK+8Ai+/DI8/Xtv3HSgagxIZXhRQNdBTBQVdu6SyFZRZVFG17uJLAy8Nw8EuexkTBZTI0KeAqoFSAZWdJAFdu/myAQUxDpVHFx8MnYDq6Ch0h2oMSmTo69eAMrPTzWyJmbWb2aVF6xaY2WNmtsHM7jSzaZl1jWb2UzNbb2YrzeyMSrftDw0NcdHBjo7CX/bFFVRxQLW1xXbp6/K47HsaUENlduCmTTBuXDxWBSUy9PV3BfUi8CXgp9mFZrYDcAPwBWASsAT4eeYl5wEzgWnA4cDnzOyoCrftF6NGdd/FV6qCStdBPhctTANvqFRQCiiR4aVfA8rdb3D3m4BXi1YdDSx19+vdfSMRSHPNbHay/gTgAndvcvdHgZ8AJ1a4bb/oa0AVd/E9/jisX9+3NqmLT0QGs3oZg5oDPJw+cfdW4ElgjplNBKZm1yeP5/S0bfGHmNnJSRfjktWrV9f0BygOqHJjUOecA+9/f+Fy76l0kkRrK5x6KsyeDV/+ct/apC4+ERnM6iWgxgHripatA8Yn6yhan67radtO3P1id5/v7vMnT57c50ZnpQHV0xjUfffBokXlK6ivfhV+/ON4v2ef7Vub1MUnIoNZvVwPqgWYULRsAtCcrEufbyxa19O2/aaxsbIuvuZmaGqCVau6jkG1tsLSpbD33jBhAvS1yBtKXXzu6uITGW7qpYJaCsxNn5jZWGAPYmypCXgpuz55vLSnbXNucyejRpU/UBcKAZWOKz3+eOcuvrSCevppmD4ddtghDrTti6EUUOnlNVRBiQwf/T3NfISZjQYagAYzG21mI4AbgX3M7Jhk/TnAX939sWTTy4GzzWxiMvnhJODSZF1P2/aLSseg0oB6/vmuXXytrfDMMxFQkyd3H1B33w0PP1x+PRS6+IbCGFQaSAookeGjvyuos4E24PPAB5PHZ7v7auAY4MtAE3AAcGxmu3OJiQ8rgLuAb7j77QAVbNsvKp3F15zpeCzu4luzBtatg912iwpq9equp0hKffSjcNZZ3bdpMFRQL7wAv/pVz69L92vaxaeAEhn6+nUMyt3PI6aBl1q3CCg5Ndzd24GPJLdebdtfKpkk4d556nhxF9/WrfF4+vTCmEtLC4wvmu6xeXNUWumXdTmDIaB+/OOYrbhxY2GflZIGVFpBaQxKZOirlzGoQa+SCmrjxgiXVHEFlUrHoKB0N99zz8X7vPhi920aDF18a9dGMPd0zJe6+ESGHwVUjVQSUMVfwsVjUKlsQJWayffkk4V13X1RD4YKqiWZo7l2bfevS/drup8UUCJDnwKqRiqZJJGOP6XVUvGBuhAVwqRJMUkCSldQTz1VeLxyZfk2ZQOq3FjWQEsDal3xkWxFsvs1nTEpIkObAqpG0uOgiseg0stDZCuoffaJ+1IV1G67xeU3uuviSyso6L6bL3vy2XSSxoYN8IlP9H0Ke62kod1TBZXdr+kfAyIytCmgaqRcF59ZhFQ2oPbdN+5LBdT06XHfUxdfQ0M87i6gsmNPaTffkiXwwx/CTTdV9GPlrrcV1MiRhT8GRGRoU0DVSLkDdaFw2fe0Wnj96+O+VBdfGlATJsR7lOvimzcvHr/wQvk2lQqopqa4f/DBHn+kmnjmGVi8uPz63o5BpV18CiiRoU8BVSPlxqAgwmbdukIFlQZUdxVU2s1XXEG5RwV1wAERYD1VUCOSAwmKA+qhh3r141Xt/PPhXe8qPwZWbRefxqBEhj4FVI2UOw4KYMcd49x7aUDtuSd8/evwvvcVXrPzztFt94Y3FJaVOt3RmjXxPjNmwNSpPY9Bbb99PE6rqTSgHn64cPqgPL38coRsNmhvvjluoC4+ESlPAVUj5cagAKZMiYBKq4UJE+Czn4Xddy+85nWvi9ccfnhhWanTHaUTJPbYA1772p4rqHQsK62g0kqltRWeeKJ3P2M11qyJ+0cfLSz74hfhK1+Jx72toLJdfG1tnY8r686tt8KJJ1bcbBGpAwqoGqkkoNavj0vDZ495ykqrnVSpLr50ivnuu3cfUFu3xhd4+p7FXXwA998f1576n//p+eerVvp5jzxSWPb88xFIW7YUKrveVFBpF98BB8B551XWjltugcsuK7yPiNS/erncxqCXDaiGhhhDSk2ZEl1d69ZF9ZRd151SFdTTT8f9brtFQP3ud6W3TQMpraCyXXw77xztOessWLEi2rNgQWVt6q3igGpvj8/OtgkqnySRdvE1NcHf/hbdpZVYtarQnh13rGwbERlYqqBqZNSoqFo2buw8QQIioDZtigsQFp9Xrzs77BBfqNlurGefjQN5x46NgFq7tvSpjNJlxV186Rf0PvtEOEHhy7vW3AsBlXbxpRVfU1PnE+f2tosvDbxXX+362jvvjJ8vu1/SA5pLvV5E6pMCqkbSA3JbWrqe9HTKlLhftiwqqErtsEN8yafjOBDn4dt113j82tfG/Usvdd02/XIu7uJbuxYmToT9949ZhDNn5hdQbW2FYEkD5bnn4n7Tps6f29suvjTQSgXOLbfEhR/TAAYFlMhgpICqkbRqam0tH1BPPdW7gEpPd5QNoOeeiwkVUAiod78bPvCBztumZ5EoVUFtt11cWv7BB+GNb+w+oC68EK65pvI2Z6XBusce8TOsXRvjT9mfBWKf9OY4qPSPgexnZKVT6NPxO3cFlMhgpICqkTSg1q8vH1AdHb3r4nvTm2JSxXXXFZZlA2rffWOyxKuvws9/3nnqdXEXX3YMauLE6CacPbswgaOcb387ZhxWOlsuK+3eO/jguH/00dIBtcsuPVdQxcdBpYoDx71rQK1fXzhuSgElMngooGpkt93ifunS0mNQqd5UULvuGge5XnxxjG21tMSXftrFN2VKTDv/6lfjizn9wofyXXxpQGXb1txc+ozn6YSGF16A3/ym8nanigPqkUc6B9Szz8b9LrtEBdXdCW2Lu/hSbW2d2/7cc4XPTSdjZE+oW0lAtbXBySd3PimviPQ/BVSNzJ0b90891bWCmjSpcO683gQUwOmnx0y+668vBFBaQaXSs08880xhWdrFN2lS3Le1Fa5JVRxQULqKyp5G6ZJLetduKATFG94QkzoeeCACKp3FmP48O+8cE0zSg3ZLKdXFl+7nbDdf9gwZaQXV24B64AH4yU/gwx8uXERSRPqfAqpGpk4tVCvFAbXNNoUg6E0XH8A//iPstVdcebZcQE2bFvfZgEorqLFj45x/GzYUxnm2267wuu4CKq125s2LiQfXXNNzV1xWGlDbbx9V1F13xXvusUcsz3bxQbRv8eLOldQ990TApe+VraDSs24UB5RZHGtWbUClwXz33dUFs4jUhgKqRswKVVSpS5enQdDbCsoMjj4a7r23MFU77eJL7bJLhGB3AdXWVviSr7SCSgPqy1+OCRsLF8I//EPlbU+DY9IkOOyw6P587LHCuQiLA/f662Pc7U9/KrzH734XobN0aTzPBtSBB8Z9NnQeeihmJu68c9eAmjKldEBlwx8KAbXffvAf/1G/19ISGeoUUDVUSUD1toICOOSQmKRw3XURWOnsvdTIkRFS2WnVaUBtu23c+hJQhxwS40Vnnx1T5Xu61HyqqSnaO2ECHHpoLGtuhjlz4nEaBDvvHPe33hr3jz1WeI80dNPxoPRAXSgfUPPmxbFe6RjUqlVx0tyZM7vO+lu1Cj7+8RjnS73wQoT6hz4UIZe+j4j0LwVUDaXXeSqeJAHVV1AABx0UFdI990RXYqkAnD699BjUttt27eLLBlR6VoVSAfXcc/Ca10SojhgBb397LL3mzvgAABaISURBVL/vvsranU5p32YbmD+/cHmRadPifTdvjrBJZxr+8Y9xn54tAwo/09NPxzjeNttEVdjQEMdyQSF0mpvjdfvuGxVftoKaMiU+p7iCSj9r+fLCsuefj9BMz1KxbFllP2+12ts7n4JKRIICqoby6OKDCIh0vKW4ey81bVrpLr40oLIVVHYMavToCIs0oDo64Pjj4zRCzz9fGB+CaENDA/zlL50/e+NGOOqoOF3SWWcVusSyMwZHjYqghXjPdPn48YX2pBMhSgVUU1Nhv556Kvz614VKMg2dNGRmz+4aUDvtFGNh5QIqG0IvvBBtnDWr67o8nHtuBLiIdKaAqqG9944v8Fp38QG85S1xXzxBIjV9enyxpl/yGzZEO0aO7L6LL21bGlAPPQRXXgk/+1kEVPbzxoyJ6qQ4oP7615iG/re/xZT39L3WrOn8WYcdFve77FIIpXHjIiCz0tDYurUwFR0KlenUqfC2t8XPNXp0oYJKA2rmzMJ5DLduLVRQaUBlx5TSAFy+vLD8hReigtp11/jMvAPq/vujC7PUxSlFhjMFVA01NsY54EqFUF8qKKgsoLZuLYwbtbYWzpre3SSJtG3ZgIKYwVZcQUFMkrjvvs7Tr9Mxo3POift0QkNTU2GaO8App8DXvhZBnrahOKAOOaQQUC+91Pns46WCf9KkrhXUjBnRdbllS7Rh1aqooCZNigox7f6Ewme1tMTr3AsB1dAQ75V3QKXtzl6SREQUUDV33XXwne90XX7QQXFLJwj01iGHxNhLOgOuWHos1KOPwm23xXhMNqA2bIgv67FjS5/pojigHnwwlpUKqHXrOo/ZPP54jFG9613xPBtQ2TCcPBnOPDPGkbJdfKNHR7hPnAhHHBGf29paqG7SaqtUQGW77ZYti2DZdtvCaaJWrSoEVHoYQLab7+mnC8dlLV8eVUxHR2HixqxZ+QbUxo2FKlEBJZVobx8+M0sVUDU2a1bhrBJZu+0W06eLr/lUqe23j1l6xx9fen0aUMcdB//8z3DVVRFGUOjiS08UW6w4oLbdNiok964VWzrNPHsNqccei2ObXve6eP9yAZWVraAgQujAAwsXcXzmmUJApWeiKDX5ZPvtO3fxpeNGaUA9+mhUUmkXH3QOqGeeKYz/LF/edWbhrFlxYce8rj781FOFLxsFlPSkoyP+n3360wPdkv6hgBpEtt++cEaKYrvsEpVAczO8973xi1yqiy87QSI1ZUqEV1tbXAp+4cKoiNL3zdprrwjbT3wiZvW1tUVAzZ4dnz9nTgRUeqmNngIq7Q797ndjskAa7k8/XQioN7857ivp4ps5Mx6nsxPvvjvud921a0Bt2RKhf+ih8d7LlhUCKv25Z82KfZkdC6ultBJtbOx8Ucdy1q+Hf/u3oXdOwU2bopqU7j30UEz++c//jD9ChzoF1BAxalRcXfbGG+OA1+9+Fz72sViX7eIrV0EB/OEP8bqDD46znEPXgGpoiEH9s8+G22+PM0wsXx4BBYWAammJaeSVVlDHHhtXyE0rqDSgpkwphE65Lr41a+L26quF16YV1OWXx5f/ggVdAyod45oxIz63XAUFnbs0i61fX/qaXJVI33fBgsoqqBtugO9/PyayDCWnnFIYZ5Xy7r037ufNg5NO6nrF7aFGATWEnHMOvPOd8fiTn4y/tCEqqdbWmPRQKjAOPDCqn9NPj+fz5kVVYdY1oCDe49xz4/5734sv+WxArV1bqAaykySK3wMKAZXaccdobxpQ06YV2lCui+/VVzvP4IPCsVVr10alN35814DKXp145sx4j+efjzGynXaKdT1NNXeP8cEFC6o74/vy5dGugw6K4866Ox8hFE7amx7UPBRs3hx/WN13X36V6lBx770xi/WSS6L34te/HugW5UsBNQyMGRNffE8+Gd1/xfbdNwJt+fKoUvbeGz7/+fjlLzfrcMQIeMc7CqclSg9qTSeBpAfdlqug0q7G4hmPZjGelgbU9OmFgCrXxbdpU0zqgEKgjBpV+Iz3va/wWiiMWaUBNX16BNQTTxSqtrSLc8cdY7vbb4/n7vGaO++MLqnFi2Oa/eLF8JWvlP5Zu5N2S+61VzzPnkWj2JYtcMcdEaB33RWVW18tXFjYP1lbt0Zg9Mdg/F/+UjiIvJqz5g8nixfHH5Tz5sXvqQJKBr30DA5HHAEf/GDp13zpS/FFPXdufLlPmgRHHtn9+6az9qBrQH3963Gfnsi2WLkKCqKi+fWvI1CnT49qptzxZWlVtHhxfHGnXYQQ3XyNjYWqctSoCMQnn4znzzwTgThtWoxztbXBFVd0rhrNIqxvuy3+aj300GjfP/5jXJLjpz+Niu/oo+H886PLszeKA6q7br4HHojq7+STI5TvuKN3n7VhQ+cT5z74YJwA+Be/KIzVpb72tZgQ84Mf9O4zqnH77YUTKqd/CEhXr7wSv7sHHBD766ij4Le/zW8CT11w9yFxAyYBNwKtwApgYXev32+//Xy4uPpq9+23d3/66e5f99xzPb8ma+1a95Ej3XfcsbBs61b3yZPdR4xw/6//Kr/t4sXu4P6d73Rdd+ut7u99r/vHP+6+bFkse93r3N/ylq6vvemmeB9wnzGj87oPftD9Ix/pvOykk+K1l13m/uY3u++8c6HdN9/sPnu2+6c/3Xmbjg73ffeN7Rob3b/xDffTTovnI0e6n3CCe1OT+xvf6G7mfv757hs3FrZvbna/5x73737Xfb/93CdNcv/+991bWuI9zj8/PqOx0f2ww9zXrXO/+2733/8+2pW64IJ4/Ysvuk+c6L5wofuWLaX377XXup98svvKlfF8+XL3WbPiM7785Wjf+9/vPmGC+9Sp7occUvishx6Kn2vkSPfXvMZ91ar4vWhpKf1Z7vGa7M9cbM2a+H0pZf/93Q86KP5tJkyIfSFd3XZb/PvfeWc8v/baeH7PPQParJoAlniJ72nzITKh3syuISrCjwLzgNuAg9x9aanXz58/35csWdKPLRxYmzcXuq1q6eijozvoppsKy+66Kyqj/fYrv92TT8bkhJ/9DE48sefPOfLIqH5+8YvOy9esiSm3M2bAv/xLHCjdnba2GDO6//7YH9/8ZnRv9uT+++FTn4rK4uCDY2bfgQdGFXLXXTHAv2EDfPSjcO21UfmlU9SzFz7cd9+oHu+6K6oz96hijj02KrRTTonjwtKDid/whuiqXLUqxsdmzIi2fPjDcOmlsU9e+9qoFJuaYr9PnlyoiHbcMdq2aFFUoQcdFFXepEnx+jPPjGnLn/hEVDCbNkV38MSJ8MtfwuGHx2esWROV+IIF8Z4tLYUzwL/ySlSCU6ZE+zdsiLa2tsZ7b9kSbR0xIvbP44/D//5vtGHvvePYwS9+Mf7tjj46xgybmmL/TpwY79XYGPvhNa+J25gxcRLfpqb4t5gwId7/iSfifo894ne+uTnaOm5c4b3c498mPU9kc3PcWlujyh49Ot5/zJh4nB4nl9q4MSbYjBgR1f3WrdGGjo54v/SaZQ0NXbftjba2+Jy0R+O666LCXLcufp41a+Lf+p3vjH03eXK89sEHY/28eYXek95qbo5qe/z46Ln4y1/i591///j3bWuLsbD0tcceWxj3rYaZ3e/uXU74NSQCyszGAk3APu6+LFl2BfCCu3++1DbDLaDysmlT/CesJvyuugre/e7S3XzF1qyJ//ylpsn31nPPwYUXxpdpuQOfK/HUU/El/pnPdP4iuuMOuOCC+E+8++7xGfvuG7e0y/MXv4gp/SNGxPbpPrjjjgjN978/vvguuii+JKdOjZ//hBPiOLfW1vjCuv/++KJOL0S5Zk0EwIc+FF9cp5wSM7322y/GyGbMiGPYLrkkJrL85jex3VlnxZhW+uV6/PHxBffNb8KvfgXveU/8UfH738cX5JgxMXW/oSG6ON/0pnjfRYti+112iePwnnkmvtBOPDG+yH7+89gnb35zhMt990X7H3wwQn3XXWOf7LlndGm2t0cgtrdH+4ovIGkWX6AdHfF88uQIpvSsKQ0NsW9bWiIo02MDs2cTGSzSS+9k/0hbsCAuSZP+sQPx75leoLRWdtgh9ms6Vljs3nt7dymeYkM9oN4A/Mndt80s+wxwqLu/s9Q2CiiR2nv11fgjIj1ezz3CZfToeN7cHIGRBrp7LEsn47S0RPg1NBTOmJBu6x7r02P2dtwxqimzeG1HR/zF714I27QC2rIltk0/Z+XKQmiNHx+3sWPjD662tsKt1Jf8qFHxB8PmzVHZbrNNhPLIkYWft6OjulmdpT5n06bYrzvv3LUi2rAh/iiZOjVCecOGqFo3b44/oKptw7bbRnXY3BzvOW1a/GwrVsQ+HDMmqjuIfTdpUt96aIZ6QB0CXO/uO2WWnQQc5+6HZZadDJwMsOuuu+63InsBJRERGRDlAmqozOJrAYonRE8AmrML3P1id5/v7vMnp0dyiohIXRoqAbUMGGFmMzPL5gIlJ0iIiEj9GxIB5e6twA3A+WY21swOBt4NXDGwLRMRkWoNiYBKnAaMAV4GrgFOLTfFXERE6l8OR8YMDHdfA7xnoNshIiK1MZQqKBERGUIUUCIiUpcUUCIiUpeGxIG61TCz1cRJZau1A/BKjZqTF7Wx7+q9fVD/baz39kH9t7He2wd9a+M0d+9ycOqwDai+MrMlpY58ridqY9/Ve/ug/ttY7+2D+m9jvbcP8mmjuvhERKQuKaBERKQuKaCqd/FAN6ACamPf1Xv7oP7bWO/tg/pvY723D3Joo8agRESkLqmCEhGRuqSAEhGRuqSAEhGRuqSA6iUzm2RmN5pZq5mtMLOFA9yeRjO7JGlLs5k9ZGZvT9ZNNzM3s5bM7QsD1M7fm9nGTDsez6xbmLS/1cxuMrNJA9C+lqLbFjP7r2TdgOxHMzvdzJaYWbuZXVq0boGZPWZmG8zsTjObllnXaGY/NbP1ZrbSzM7oz/aZ2YFmdoeZrTGz1WZ2vZlNzaw/z8w2Fe3P3fu5jd3+m9bBPjyuqG0bkvbul6zvz31Y9jsmWZ/b76ICqvd+AHQAU4DjgIvMbM4AtmcE8BxwKPAa4GzgOjObnnnNdu4+Lrld0P9N/LvTM+3YEyDZdz8Gjif26Qbgh/3dsEy7xgE7AW3A9UUv6+/9+CLwJeCn2YVmtgNx/bMvAJOAJcDPMy85D5gJTAMOBz5nZkf1V/uAicSMrulJG5qBnxW95ufZfe7uT+XQvu7amCr3b3oeA7gP3f2qot/J04CngAcyL+uvfVj2Oyb330V3163CGzCWCKdZmWVXAF8b6LYVtfOvwDHEF4QDI+qgTb8HPlZi+VeAqzPP90j28fgBbOsJxJdBOst1QPcj8QV2aeb5ycA9medjiUCdnTx/ETgis/4C4Nr+al+J9W8EmjPPzwOuHOB92O2/aR3uwzuBcwdyHxa1J/2OyfV3URVU78wCNrv7ssyyh4GBrKA6MbMpRDuzF2tcYWbPm9nPkr94BspXzewVM/uTmR2WLJtD7EMA3P1Jkj8CBqB9qROAyz35H5VRL/uxeJ+1Ak8Cc8xsIjA1u56B/x19C51/HwHemXQBLjWzUweiUYku/6b1tg+TLrO3AJcXrRqQfVj0HZPr76ICqnfGAeuLlq0Dxg9AW7ows5HAVcBl7v4YceLG/Ynyej+inVcNUPPOBHYHdia6f24xsz2Ifbqu6LUDtk+TL4NDgcsyi+tpP0L3+2xc5nnxun5nZvsC5wCfzSy+DtgLmAycBJxjZh/o56Z1929aV/sQ+BDwB3d/OrNsQPZhie+YXH8XFVC90wJMKFo2gehjH1Bmtg3R3dgBnA7g7i3uvsTdN7v7qmT5EWbW7//R3P1ed29293Z3vwz4E/AO6m+fHg/8MftlUE/7MdHdPmvJPC9e16/MbAbwa+CT7v6HdLm7P+LuL7r7Fne/B/hP4L392bYe/k3rZh8mPkTnP5gGZB+W+o4h599FBVTvLANGmNnMzLK5dO2+6FdmZsAlxCSDY9x9U5mXpl1W9fDv7oAR+25uujCZidRI7OuB0OXLoISB3o/F+2wsMXa31N2bgJey6xmA39GkEl0EXODuV/Tw8vR3YSD9/d+0XvYhgJkdDLwW+EUPL811H3bzHZPv7+JADbIN1htwLXANMRh4MFGyzhngNv0IWAyMK1p+ALAn8UW6PTG75s4BaN92wJHAaGJG0HFAK9GPPYfoNj0k2adXkuNgdA/tPChp1/ii5QOyH5N9NRr4KvGXa7r/Jie/d8ckyy4EFme2+xpwFzGbbnbyJXFUP7ZvZ2Ic4jNltnt30jYD/gF4ATihn/dht/+mA70PM+svJsZDB2wfJp9X7jsm19/FXP+DDcUbMZXypuSL7Flg4QC3Zxrx19NGoqROb8cBHwCeTtr6EjHIutMAtHEycB9R2q9NftHfllm/MNmXrcCvgEkDtC9/DFxRYvmA7EdippYX3c5L1r0VeIyYMfV7YHpmu0Zi2vJ6YBVwRn+2Dzg3eZz9fWzJbHcN8Gqy/DHg//T3Puzp33Sg92GybnTy/2VBie36cx+W/Y7J+3dRJ4sVEZG6VA9jESIiIl0ooEREpC4poEREpC4poEREpC4poEREpC4poEREpC4poETqVHL9n9xOX2Nm85PPmJ7XZ4j0hQJKJAdmdmny5V98W9yLt5kK3JJXG0Xq3YiBboDIELaIOPlsVkelG7v7yto2R2RwUQUlkp92d19ZdFsDf+++O93Mbksulb3CzD6Y3bi4i8/Mzkle155cPvvyzLpGM/uuma0ys41mttjM3lz0fkcll+beaGZ/oMQ1t8zsIDO7K2nTC2Z2kZkVn61apF8ooEQGzheBm4F5JCcFNbP5pV5oZscAnyEu/T0T+GfgL5mXfB34V+AjwBuAvwG3m9nUZPvXEeeQvCP5vP9Ktsl+xuuB3yZtmgscnby23OXSRXKlc/GJ5MDMLgU+SJxgM+sH7n6mmTnw3+5+UmabRcBKd/9g8tyB97n7L8zsDOAUYB8vupxKcomDJuBj7n55sqyBuGTJNe5+tpl9hbhe0J6e/Kc3s7OJS3Dv5u7PJBXZJnf/aOa95wEPAlPc/eXa7B2RymgMSiQ/dwMnFy1bm3n856J1fwb+qcx7XQ98EnjazH4D3A7c7O7txPV3RhIXgQTA3beY2Z+BvZNFexGXQcj+RVr8+fsBM8zsXzPL0msM7QEooKRfKaBE8rPB3Z+oxRu5+3NmtiewgLi8wbeAc83sgJ427cXHbAP8N/CdEute6MX7iNSExqBEBs6BJZ4/Wu7F7r7R3W9z908D+xMXezyYuDhgR/IY+HsX35uAR5JFjwIHJFdGLff5DxAX33yixK2tip9PpE9UQYnkp9HMdipatsXdVyePjzaz+4iLvL2XqI5KVkRmdiLx//Ve4mJx/wpsApa7e6uZXQRcaGavEBfi+zRxee4fJm/xI+Dfge+a2Q+B1wMfL/qYC4HFZvYj4uKNzcRVUN/p7qf0/scX6RsFlEh+3kpcrTXrBWCX5PF5xKWyvwesBj7s7veVea+1wJnAN4nxpkeAo9396WT9mcn9z4DtiIkNR7n7SwDu/qyZHQ18m5hscT/weeDK9APc/a9m9hbgS8RluhuAp4Abe/uDi9SCZvGJDIDsDL2BbotIvdIYlIiI1CUFlIiI1CV18YmISF1SBSUiInVJASUiInVJASUiInVJASUiInVJASUiInXp/wNsjv2KafID0gAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**The robot's path:**"
      ],
      "metadata": {
        "id": "JvZaI9tvzRf_"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pLoueG9Pav63",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 291
        },
        "outputId": "37ef4eed-43bc-42f4-c2fe-bcd831734884"
      },
      "source": [
        "Collected_data_time = {0:[], 1:[], 2:[], 3:[], 4:[]}\n",
        "state = env.reset()\n",
        "done = False\n",
        "cum_reward = 0\n",
        "counter = 0\n",
        "number_of_steps_taken_to_terminal = 0\n",
        "visited_X = []\n",
        "visited_Y = []\n",
        "\n",
        "while not done and counter < env.max_episode_steps:\n",
        "  action = agent.get_action(state, 0.0)\n",
        "  #print(action)\n",
        "\n",
        "  visited_X.append(env.vector_agentState[0])\n",
        "  visited_Y.append(env.vector_agentState[1])\n",
        "\n",
        "  next_state, reward, done, _ = env.step(action)\n",
        "  cum_reward += reward\n",
        "        \n",
        "  state = next_state\n",
        "  for i_ in range(num_sensors):\n",
        "    Collected_data_time[i_].append(env.Collected_Data[i_])\n",
        "\n",
        "  counter +=1\n",
        "  number_of_steps_taken_to_terminal  += 1\n",
        "\n",
        "\n",
        "\n",
        "### Plot the trajectory\n",
        "x = np.append(np.array(visited_X), env.Terminal[0])\n",
        "y = np.append(np.array(visited_Y), env.Terminal[1])\n",
        "x_s = np.array([50, 20, 80, 60, 50 ])\n",
        "y_s = np.array([10, 60, 40, 60, 90])\n",
        "\n",
        "x_o = env.Obstacle_x \n",
        "y_o = env.Obstacle_y\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "plt.figure()\n",
        "\n",
        "plt.quiver(x[:-1], y[:-1], x[1:]-x[:-1], y[1:]-y[:-1], scale_units='xy', angles='xy', scale=1)\n",
        "\n",
        "plt.scatter(x_s, y_s, c = 'k' ,marker = \"o\",label = 'Sensor')\n",
        "\n",
        "for i in range(len(x_o)):\n",
        "    rectangle = plt.Rectangle(( 10* (x_o[i]-0.5), 10*(10 - y_o[i] -0.5)), 10, 10, fc='blue',ec=\"blue\")\n",
        "    plt.gca().add_patch(rectangle)\n",
        "\n",
        "\n",
        "\n",
        "plt.scatter(10,10, marker = \"s\", ec = 'k', c ='red', s=50, label =\"Terminal\")\n",
        "#plt.scatter(90,90, marker = \"s\", ec = 'k', c ='red', s =50)\n",
        "plt.scatter(20,80,marker = \"s\", ec = 'k', c='green', s=100, label='Charging station')\n",
        "plt.grid(linestyle=':')\n",
        "plt.xlim(0,100)\n",
        "plt.ylim(0,100)\n",
        "plt.xlabel('x (m)',size = '14')\n",
        "plt.ylabel('y (m)',size = '14')\n",
        "#plt.legend(loc=4)\n",
        "plt.xticks(size = '12')\n",
        "plt.yticks(size = '12')\n",
        "plt.gca().set_aspect('equal', adjustable='box')\n",
        "plt.savefig('T_3.eps',format = 'eps')\n",
        "plt.show()\n",
        "####################################\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAASIAAAESCAYAAABZxNgbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29fXhU1bmwfz9JCCGBADHyDQkIyIdADFRseUUQoSqklqL1A1ssVqgeSlPPae35Sf1C2h573spr0RZtsXoIrS1aMYhUiyi2VDSJxAhEMISEJECIBEI+SBiyfn/MTE4IkOyd7Mnaa9j3dc2VzJ49a+69ZuaZtdde61milMLDw8NDJxG6BTw8PDy8QOTh4aEdLxB5eHhoxwtEHh4e2vECkYeHh3a8QOTh4aEdLxB5eHhop1MDkYgsEZEsEakXkT+0eGyGiOSLSK2IbBWRpGaPdRWRNSJSJSKHReSBzvT28PAILZ3dIioDngDWNN8oIonAq8BPgQQgC3i52S6PAiOAJGA68GMRuaETfD08PDoB0TGyWkSeAAYppe4O3F8E3K2U+krgfhxQAVyplMoXkbLA428FHl8OjFBK3d7p8h4eHo4TpVsgwFggN3hHKVUjIgXAWBE5AvRv/njg/6+fr6BAUFsEEBsbO3HEiBEEg21ERAQ+n48uXbrQ0NBATEwMtbW1xMbGnvP31KlTREdHc/r0aaKiomhsbAyWT2NjI1FRUTQ0NNC1a1dOnTpFt27dzimjrq6OmJgYGhoaiIqK4syZM0RE+BuhjY2NREZG4vP5iI6ObrOM+vp6oqOj8fl8REREeMfkHZMrjmnnzp0VSqlL7X/lz8Ytgag7cLTFthNAj8BjwfstHzsHpdRzwHMA48aNUzt37nTWNEQUFBRw2WWX6dawjEm+JrmCWb4iUuREOW65alYNxLfYFg+cDDxGi8eDj7VKVJRb4mzbJCQk6FawhUm+JrmCeb5O4JZAtAuYELwT6CO6DNillKoEDjV/PPD/rrYKDTZpTaC2tla3gi1M8jXJFczzdYLOvnwfJSIxQCQQKSIxIhIF/BW4QkTmBR5/GPhEKZUfeOpLwDIR6S0io4B7gT90pnuoCZ6/m4JJvia5gnm+TtDZR7wMqAN+AtwV+H+ZUuooMA9YAVQCk4HmV8QeAQqAIuA94JdKqc1tvZiIOCofSrp06aJbwRYm+ZrkCub5OkGnBiKl1KNKKWlxezTw2N+VUqOUUt2UUtOUUgeaPa9eKbVQKRWvlOqrlPqVldcz6dSsurq67Z1chEm+JrmCeb5OENZtQJM6qxMTE3Ur2MIkX5NcwTxfJwjrQNTQ0KBbwTIlJSW6FWxhkq9JrmCerxOEdSDq2rWrbgXLDB8+XLeCLUzyNckVzPN1grAORKdOndKtYJldu9ocjeAqTPI1yRXM83UCLXPNOotJkyaprKwsS/v26wdHjjj7+hEREIr+8r594fBh58v18LCLiGQrpSZ1tJywbhHZGRjmdBACe0EoPT3b8r6hcLVLdrZ1X92Y5Arm+TqB1yIKYNCQIwDC+G3zMAivRWQBk4bK22kRuQGTfrVNcgXzfJ3AaxEF8FpEHh728VpEFqirq9OtYJmFC/N0K9giL88cX5NcwTxfJwjrQBQTE6NbwTLr14/UrWCLkSPN8TXJFczzdYKwDkQmjayeMaNYt4ItiovN8TXJFczzdYKwDkQmzTXLyemrW8EWffue37exsZHdu3fbLi8jI4Pk5GQiIiJITk4mIyOjo4pNXMjVrZjm6wRhHYjOnDmjW8Eyw4Yd161gi+PH/9e3pqaG1157je9+97sMHDiQl156yVZZGRkZLFq0iKKiIpRSFBUVsWjRIseCUXNXEzDN1wnMaTK0A5MSTFVWmtOfBXDy5El+85vfkJmZyTvvvEN9fX3TY9u2beOmm26yXNbWrVvPmY5TW1vLQw89xPz58zvsalJfIZjn6wRhHYg8nMfn85GUlMTXvvY1GhoayM7OPisIAezZs8fWafGF5gRejH0lFythHYhMSozWu7c5E3TLysro06cPjz32GI2NjXz00UdkZmaSmZnJJ598wmOPPcbSpUstl5ecnExR0bmLQQwZMsQRX5MmP4N5vk5gzrlLO4iMjNStYJn9+3vpVrBFsB8jIiKCyZMn88QTT5Cbm0tRUZHtALJixYpzTkdiY2NZsWKFI669eplVt6b5OkFYByKfz6dbwTKpqS6YyWqDC13ZGTJkCF//+nnXvrwg8+fPJy0trel+UlISzz33nCP9QwBH3DBL2Aam+TpBWJ+aRUdH61awzJYtzpyGdBZO998cPHgQgKFDh1JQUODowgdOneJ1Fqb5OkFYt4hMOte+5Za9uhVs4eTo3/Lycnbs2AFAYWEhe/bscaxsgL17zapb03ydIKwDUbdu3XQrWGbNmnG6FWzx6aefOlbWG2+8QfPJ15mZmY6VDTBunFl1a5qvE4R1IPLSgISO1NRUx8rasmULM2fOBGDatGls27bNsbLBvLQapvk6QVj3EcXGxupWsMzKlRN1K9giJyfHsbKeeeYZPvzwQ95++23uueeeszqunWDiRLPq1jRfJ/BaRC7hYm4R9ezZs9X7HcW0FoZpvk4Q1oHIaxGFDidbRKHGtBaGab5OENaBqKioDhEs3XSzeHGu5X3dMDl7/PjxuhUsk5trvW7dgGm+ThDWfURffKF/8qDVlK4+31h++9vQujiJSWtvjR07VreCLUzzdYKwbhH16lXf9k4u4fPPP9et0CZHjx5t+n/48OHU1dVRXV2t0cgaJtRtc0zzdYKwDkTV1eaMrB40aJBuhTZ55ZVXmi6zv/7661xxxRVGpFoxoW6bY5qvE7j/U9QBunUzZ65ZRUWFboU2mTNnDu+++y4A3bt3Z/To0UZcEDChbptjmq8ThHUgamgw5/C6d++uW6FNBg0aREpKCgAlJSXMmTNHs5E1TKjb5pjm6wTmfFPbQUSEOxf/iu8Vj4icdbvpppvO2RbfK1636jkEBxvGxcUZE4hOnz6tW8EWpvk6QVhfNXPDZfnzcfLESXj07G3RvaJhdov9Hj3ZaU5WSUtLY/ny5Vx++eXG9GWYlCAPzPN1Ate0iEQkWUQ2iUiliBwWkVUiEhV4LEVEskWkNvA3xUqZPp9rDq9NyhvKO/X1Dh8+zAsvvGB79PnEiRPp168fV1xxRYjMzsXn87Ft2zbeeeeddj3fhH6s5pjm6wRu+qY+C5QD/YEU4FrgfhGJBjYAa4HewIvAhsD2VomJMaezemRsaBfVU0qxc+dOli9fzlVXXUX//v3ZvHmz7Q99REQEc+bMYfLkySEy9XP8+HH+9Kc/cdddd9GnTx+mTZtG//7921XWsWPHHLYLLab5OoGbTs2GAquUUqeAwyKyGRgLTMPvuVL5c0U8LSL/AVwHbG6twJqaLqE1dpAdJ3Y4XmZxcTHbt2/nvffeY+PGjZSUlJz1+CuvvNKuFSNGjBjBypUrndIE/K2A4cOHs3XrVtasWcP7779/TobNK6+8sl1lv/LKK1x22WVOaHYKAwYM0K3Q6bgpEK0EbheRd/G3fG4Efoo/GH2i1FljlD8JbD8nEInIImARQM+efRk4sIK4uNNERzdSXh7LyJHH2LFjADfcUMi6daNYsmQnTz+dSnp6NitXTmTp0hxWrUrhzjvz2bx5KJMnl7F3bwJ9+tTS0BBBTU0XBg2qJi8vkalTS9iwYTgLFuxi9eoJTWUE/y5cmEd9/UiKi4vp27cvx48fJyYmhpSUFHr37s3+uv2kxqey5YstPJj8ID/a9yPSh6Szsnil/y8ryc3NZezYsXz++ecMGjSIiooKunfvzunTp2lsbCQ2NpZjx44xYMAACgsLGTVqFDt37iQ1NZWMjAxWr17Nr3/9a6KiolBK8fLLLzN58mT27t3bdJp1+vRp4uPjOXLkCMnJyezZs4crr7ySjz76iK985Sts37696W9qaionT57k448/Zvz48U3HBP5EdL169eLIkSMMGTKEvXv3Mm7cOLKzs5k4cWLT3/Md0+WXX862bds4dOgQO3bsICUlhXXr1jFz5kzWrVvHkiVLKCsrY8qUKWzfvp2rr76aHTt2MGHCBPbu3cvgwYOb6sbn8zUd0+HDh6msrMTn87Fr1y4mTJhwjk9eXh4jR577PnX0mKy+T8EycnJySElJ4aOPPuJLX/oSZWVlJCQkUFtbS0REBF26dKG6uprExERKSkoYPny49mNyClFW5yCEGBEZjf/0awIQif8U7DvAMmCsUur2ZvtmAPuUUo+2XuYkBVkhc7bC+apXRM7prI4ggkZadFI+Ch15f6ZOncrBgwcpLCwMuCjy8vKaVtw4ffo0WVlZ7UrL2tjYGPLBjCdOnOCtt94iMzOTTZs28cILL9hOEfLhhx+ydOlSPvjggxBZOk9n1K1TiEi2UmpSR8txxdGKSAT+1s2rQByQiL9V9F9ANdDyOnY80OYlpT59zEkDsmTIkpC/hogwfvx4HnroIT744AM2bdrU7nS6O3fudNjuXHr27Mmtt97KSy+9xJEjR9p1arZx40YmT55sVEL6zqhbt+GKQAQkAEPw9xHVK6W+AF4AbgJ2AePl7J/t8YHtrVJebs7Vh6eLn+701+zbt2+70+k6mY/ICpGRke0aLpCZmcnTTz/NG2+8EQKr0NDZdesGXBGIlFIVQCFwn4hEiUgvYAH+vqB3gTPAUhHpKiLBpkOb13JNahGlD0nXrWALE5J3HTx4kJ07d5Kenu54HuxQYkLdOo5SyhU3/Jfs3wUqgQrgz0DfwGNXAtlAHZADXGmtzInK30uj73Y+evTsoYA2bz169jh/ARa55pprVHJycofKMJlnn322qS5jY2NVXV2dbqWwA8hSDnz/XdEiAlBK7VRKTVNK9VZKJSqlvqmUOhJ47GOl1ESlVDelVKpS6mMrZbq1RVR1vOqcNyI7O/ucbVXHq3SrXhATMjRu3LgRgKVLl1JbW9s0YdftmFC3TuOaQBQKTOojCk4mNQW3+9bV1dG7d2/mzZvHqlWr+NnPfmZMMje3120oCOtAlJBgzgKL+fn5uhVs4XbfmJgY1q5dy4gRI7jzzjv59re/zQMPPKBbyxJur9tQENaBqKrKnMRoQ4cO1a1gC7f7Nr/Iunnz5nO2uRm3120oCOtAFBdnTjqFsrIy3Qq2MMk31PPinMakunWKsA5Ep05Zn8ESioGsdlbbSEhIcF7AJv36WVvxRARuvDHB8r6RkdbLtXrr18/6cblhLflwrVunCOtAdPnljZYvtJ854/zF+8OHrbu6YTFIO4OP7VyRDEV6HXuufZwXsEm41q1TuGnS60WNk3OLCgoKzrp/6NAhKioqGDdunGOvYVIa3oaGBt0KtjCpbp0irAORKZ2TAF26OJeypEePHgwePJioqCh8Ph/Dhg1zfNKnSSlWampqdCvYwqS6dYqwDr0mpdx0cn2wPn36MHHiRGpra2loaODSSy91fGXWQYPcv55ZEFNS2gYxqW6dIqwDUVSUOQ2+xMRER8trnth+zpw5jrcO8/Kc9Q0leXl5uhVsYVLdOkVYByKT+gZaZk/sKM3z9tjN4WOFqVOd9Q0lU6dO1a1gC5Pq1inCOhB17dpVt4Jlhg8f7mh5V1xxBUlJScTGxjJ9+nRHywbYsMFZ31CyYcMG3Qq2MKlunSKsA1F7k37pwOl5UCJCWloaM2fObFde6rZYsMCMeVsACxYs0K1gC5Pq1inCOhC1N+mXDiZMmOB4mWlpaSE5LQNYvdp531CxevVq3Qq2MKlunSKsA5EbBglaJRTJsKqqqkLWT5aebkbyrquuuopVq1YZtVaYKXXrJK5Jnh8KJk2apLKy9CbP10nL5PltYdCwK+D8CxO05NSpUzzxxBM88cQToRdqhXCsWwiz5Pmh4mJvEYUSU361t27dSkREBCb94JpSt04S1oHIpOb4xIkTdSvYYuVKM3wzMzNZvny5UTl+TKlbJwnrQFRXV6dbwTKmDbpbuND9vkopNm7cyMKFC41Knm9C3TpNWAeiUFy2DhUjR47UrWCL9evd75ubm8vBgwdZv369UYHIhLp1mrAORCaNrC4uLtatYIsZM9zvG0yeP2PGDLZv384XX3yh2cgaJtSt04R1IDJprllfO1nUXEBOjvt9g62gnJwcGhsbefPNNzUbWcOEunWasA5EZ86c0a1gmePHj+tWsMWwYe72raur4+677+aHP/whw4YNY+vWrcZM+XF73YaCsA5ETiYbCzUm9WcBVFa627dbt27cd999dO3alcrKSkaMGMGtt96qW8sSbq/bUGDON9XDwyNsCetAZFJiNDdM0LXTgOzdW6+vnS613r17h07EIvZ89datjhMJc3pz20FkZKRuBcv06tVLt4KtROz799vz1Tmwef/+/fpePICdhRSqqnoRH29t31BMHdHx+x3WLSKfz6dbwTJHdCyd0AFSU83xTU1N1a1gC9M+C04Q1i2i6GhzVnodMmSIY2Xl5uby8ccf4/P5aGxs5NVXXyUhIYFp06Y59hpbtjjnG2q2bNmiW8EWTn4WTCGsW0Ru6HexipOLAI4YMYL77ruPf/3rXxQXFzNv3jwuu+wyx8oHuOUW/YsWWuWWW27RrWALNywI2dmEdSAyKTGak2uOxcbGcv311zfdT0lJYfDgwY6VD7BmjXO+oWbNmjW6FWzh5GfBFMI6EF3MaUBCnTzfpFQV6enpuhVsYVpKGCcI60B0MacBab6cUCgCkUmpKlauXKlbwRampYRxAtcFIhG5XUT2iEiNiBSIyDWB7TNEJF9EakVkq4gktVWWCS2ijIwMkpOTeeCBB0hOTiYjI+O8+50+fdpWuQMGDGDixIn069cvJB9sE1pEGRkZPPvss6SnpzN58mTH6jbUXIwtIpRSrrkBM4Ei4Gr8QXJg4JYInABuBWKAXwIftFXexIkTlZtZu3atio2NVUDTLTY2Vq1du1b5fD61fft29Z//+Z9q/Pjx6tNPP7Vd/qOPPqruuecey/v7R/uE5tbZtFW3//jHP9SDDz6oxo0bp/Lz8ztf0CF0v19AlnLgu++qnNUish34vVLq9y22LwLuVkp9JXA/DqgArlRKXTD13tixY5XTy/Q4SXJyMkVFRQAsXry4abWJuLg4YmNjOXr0KOBfMvl73/ue7fLLysqYNm2a5TlWdgbHLV6ca2u1ic7+mLVWtzExMU0pQZKSkrj33nvb9RoPPvhgSDI85ObmWl7VJVS5sC/anNUiEglMAi4Vkc9FpEREVolIN2AskBvcVylVAxQEtrcsZ5GIZIlI1vHjx6moqODQoUOUlpZSWVlJQUEBdXV17N69m8bGRnJycoD/bQ4HU0bs3r2buro6CgoKqKyspLS0lEOHDlFRUcGBAweorq4mPz8fn89Hbm7uWWUE/+bl5VFfX8++ffuoqqqiuLiY8vJyysvLm/IPzZ07l/j4+Kacyunp6dTU1DB//nzA/yWqqKggLy+PZ555hm3btrFu3To2bNjAm2++yYsvvsi//vUvVq5cSX5+Pg8//DDl5eUsW7aM6Ohopk6davmY+vev5rbb8omJ8bF4cW7AJ/usvwsX5hEfX8+xYzEkJVUxfXoxKSnlpKSUM316MUlJVcydu4/4+PqmTIPB5wbrJTc3F5/PR35+PtXV1Rw4cMDx96l79+7MmjWL/v37U11dTUxMDIsXL6ampoZvfetbTXVdVFTE/v37efLJJ8nOzub5559ny5YtrF+/nvXr17Nlyxaef/55srOzefLJJ9m/fz/Lli2joqKCsrKykBxTZGSk5c9eW+/T3Ln7bL9Pwb9WjskpXNMiEpEBQCmQDaQBp4ENwLtAP+CoUuonzfb/J/C8UuoPFyrziiuuUJ9++mkIrTtG81/t2267jZdffhmASy+9lHnz5pGZmUlpaSmDBw/mb3/7W7uyCQwfPtzyVBc7v6633ZbPyy+Psry/zhZRy7qdO3cuGzdupKysjGHDhvHGG28gNpsWb731FrGxsdxzzz2Ou+fn5zNqlLW6DZcWkfZ+oeAN6I3/XH5Bs23zgI+B/wc822L/PGBea2VeeeWV1k92NdC8H6N///5n9WMopVRjY6PKyclRjz32mMrNzQ25j50+hP79TxrTR3Shus3KylKPPPKI2rVrl+3y7777bjV37lyntZVSSp08edLyvuHSR6Q9AJ0lAweBbze7/41AIFoE/LPZ9jigFhjVWnnjxo2zXqOaWLt2rerZs6eaNWuWGjBgQNMXRQd2PqizZhW6OhAp5a/bpKQkNWvWLJWUlORY3fp8PnXppZequLg4VVdX50iZzSksLLS8rxeIQhOIHgc+AvoEWkjvA8uBS/FfNZuH/6rZf2HhqtmECROs16hGHn74YTVmzBi1f/9+rR52Pqhjxhx1fSAKcvToUUfL2759uwq03tWbb77paNlK2fMNl0Dkms7qAMvxB6K9wB78raEVSqmj+IPQCqASmAzc3lZh/noyg7i4ON0KtoiLc9fYm9ZwepxQMCk/EJLVQdw2rqkzsBWIRCRCREI2XFkpdVopdb9SqpdSqp9SaqlS6lTgsb8rpUYppboppaYppQ6EykMHJmUKAIiONifpnNMJ8poHn8zMTMd/8ExK6OcUbQYiEblRRF4SkSKgATgZGPX8vog8FLja5UpMylldXl6uW8EW5eXmTJ9xcqpP8HJ/fHw8sbGxnDhxgk8++cSx8sGsqUlOccFvqojMFZG9wBr8l9J/BswFvgp8B9gKXA/sF5HfisilneBrC5MSo5m2wOLIkcd0K1jm2DHnXJVSFBYWMnr0aAYNGkRRUZHjp9VO+ppCa8NCfwI8AGxSSp2vrfhnABEZCPwA+Dbwfx037ABdunTRrWCZHTt26FawxY4drm0In8OAAc65Jicnn3W/V69ejqf5ddLXFC7YIlJKTVZKbbxAEGq+X6lS6sdKKVcFITBrpdcbbrhBt4ItbrihULeCZQoLzXEF83ydwJxOlHZgZ62wfv38o1SdvPXrZ9113bp17ThCZ7Gz0sS6ddZHVetexNbqKGW3YMc3FHWr4/2yPGNPROYC0/GP8TkrgCmlvumwlyPYSQMSinzldspcsmSJ8wI2sbPSRE7OTmOS0u/caY4r2PO18565GUstIhH5v8DLQDCH5ZkWN1fi9qsPjY2NTZd+n3766aZtJmDSF9skVzDP1wmstogWALcqpTaEUsZp3J4YzefzMW/ePESE9PR0nnnmGeLj43n44Yd1q7VJdna2MZkETXIF83ydwGogqgUumPfHrbi9RRQdHU23bt34y1/+0rRt06ZNGo2sY9IXxSRXMM/XCax2Vv8C+LGIGLUOmttbRPC/+aSXLl1KXFwc06dP12xkjWAuHRMwyRXM83UCq4Hlefw5gkoDgxzPmgyjlLrOaTEncHuLCODGG28kIiKCVatW8bWvfc3WlT6dpKSk6FawjEmuYJ6vE1gNRL8F/g+wGTiCf+ax6zFhgcXExES+/OUvM3ToUEdXYg01+fn5jBkzRreGJUxyBfN8ncBqILoNmKuUejuUMk5jykTStLQ0/vu//5tf/vKXulUsM3ToUN0KlulsV6UUe/bsYdiwYe1q4ZpUt05htY/oKP40rkZhSjqFtLQ0vvnNb9LPzghIzZSVlelWsExnuDY0NPD3v/+dH/zgBwwfPpz77ruPrl27tqssk+rWKay2iB4BHheRu5VS1aEUcpJQrLAQCkaPHs3UqVN1a9giISFBt4JlQuV65swZXnrpJTIzM/nb3/7GyZMnmx6LjIxk5syZtsscPXo0jz/+uJOaRmD1m/ojIBk4IiLFnNtZPd5hL0cwZXDgk08+ybvvvsvVV19NUlKSbh1L1NbW0rt3b90algiF6+bNm8nOzuadd97hs88+OysIgT+tS01Nje1ya2trqa6uNqZuncJqIFofUouLnNraWr744gtjAieYlespFK69evVixowZzJgxgxUrVnDgwAE2btzIxo0b2bp1K9dffz3r19v/2txxxx0cOHCAwYMHO+7sZiwFIqXUY6EWCQV2l4jRSXt+PXViUoqVznBNTk5myZIlLFmyhJMnT7JlyxZOnTplq7P69OnTvPnmm6SmpnLNNdeE0NZ9tJYYzda32O7+nYFJLYxBgwbpVrBFdbUxXYWd7tqjRw++/vWv275i9o9//IMTJ07w2WefhcjMvbTWZs0XkbtEpNWufxEZLSLP40+k5ipM6awG/6qwJpGYmKhbwTKmuAaT8m/atInSUuMuUneI1gLRIuD7+DuoXxGR/xSRBSJym4h8T0SeFpEc/KtuHAVWdYawHUxKjGbaVbOSkhLdCpYxxTWYlH/q1KlnrRRyMdBahsb3lFKTgZuAMvyDGlcCLwLLgKH481kPUkr9f0qpkxcqSxftHcehgw0bjEpswPDhw3UrWMYE188++4x9+/YB/s9CKJYpcjNtXk5QSm1XSn1fKZWilOqtlIpRSg1SSqUppVYppY53hmh7MGGKR5AFCxboVrDFrl27dCtYxgTXTZs2MW/ePAD+/d//naKiIiMmbTuFOddg20G3bt10K1hm9erVuhVsMWHCBN0KljHB9d577+Wll14CoKKigo8//tioq74dJawDkUm/KOnp6boVbJGdna1bwTImuHbv3r3p//HjxxMVFWXUD2lHCetAZEIakCArV67UrWALk5J3meQKOL5gowmEdSCy0yIKxUBhO2WGc4soFCukREZa3/eHP8wOycoroWL8eFfOmAopYR2I7LSIQjH20U6Z4dwiCsUKKfbqVq+rXbwW0QUQkddEZI6IGBW46urqdCu0SkNDQ9PywgsXLgTgsCHrw5g0AHPhQnNcwbx12JzA6tDjGvzLCZ0QkT8ALyil9oXMyiHcnna1S5cuXHPNNURERFBcXEx+fj4zZswwIg3EyJEjdStYZv16c1wB9u/fr1uh07HUwlFKzQf6A8uB64HPRGSbiHxbRFzbte/2kdUiwtSpU/n000+ZMWMG27dvZ86cObq1LFFcXKxbwTIzZpjjCjBw4EDdCp2O5VMtpVSVUuo3Sqmr8C+0mA2sBg6JyGoRGR0qyfZiwlyz4CoeOTk59OvXj0mTJmk2skZf3etI2yAnxxxXgKNHj8aE1uUAAB1QSURBVOpW6HRs9/mIyADgZmAO4ANeAQYDn4jIfzir1zHOnHHtIrRNXHfddcTGxjJs2DBmz55tTJ6f48ddO6D+HIYNM8cVoGfPnroVOh2rndVdROQWEdkEFAFfB54E+iul7lFK3QTMwz8HzTWY8KWOiYlh5syZVFZWNrWOTMDt/W/Nqaw0xxWgvr5et0KnY/Wbegj/aVgBMFEpdZVS6vkW+au3AZUdFRKRESJySkTWNtt2p4gUiUhN4AqeOQmTLZCWlkaXLl24/vrrdat4aOTll18G4I033iA5OZmMjAzNRp2H1UD0Q2BgYPLreQc5KKWOK6WcWAflGfypRQAQkbH4g+C3gL74l79+1kpBpiRGu+mmm5gyZQpxcXG6VSxj0oTi3r3d75qRkcG//du/AdC7d2+KiopYtGjRRROMrF41+x+lVMjfTRG5HTgObGm2eT6QqZTaFmiB/RT4hoj0aKu8yMjI0Ig6TEREBJMnT3b9Vb7m9OrVS7eCZfbvd7/rQw891DTuLXj5vra2loceekinVqfhmk4UEYkHHgceaPHQWCA3eEcpVQA0AOcdHCIii0QkS0SyDh8+TEVFBYcOHaK0tJTKykoKCgqoq6tj9+7dNDY2Nq0znp7un7KwdGkOERGN3HXXbhIT65g9u4ARIyqZMqWUL33pEGPGVDBr1gH696/mttvyiYnxsXhx7lllBP8uXJhHfX09+/bto6qqiuLiYsrLyykvL6e4uJiqqir27dvHc889R2FhIaWlpU1TJ4J/c3Nz8fl85OfnU11dzYEDBywfU7CMnJwcGhsb2b17N3V1dRQUFFBZWUlpaSmHDh2ioqKCAwcOUF1dTX5+Pj6fj9zc3LPKCP7Ny/Mf065du1o9pvr6+qZBjy3rZfHiXGJifNx2Wz79+1cza9YBxoyp4EtfOsSUKaWMGFHJ7NkFJCbWcdddu4mIaGTp0va/T9/5zqdtvk/x8fXMnbuPpCRrx+T0+1RcXMzSpUuJiIjgu9/9LomJicyePZuuXbt26H1q67PX0WNyDKWUK27A/wMeDPz/KLA28P8W4Hst9i0FprVVZmpqqrIKhOZmhYcffljFx8er/fv3W/bVzalTpyzvG6q6tXqLjz/l+HvmNElJSQr/Uu4qPj6+6f+kpCQ9QhYBspQD339XtIhEJAX/QMmnzvNwNRDfYls80GZGSJP6MW655RbdCrbYu3evbgXL3HKL+11XrFjRlPYj+FmIjY1lxYoVOrU6DyeiWUdvQDr+aSSHA7dqoA7IAX4GZDTbdxj+U7MebZU7ceJEG5Fdb4sIMKpFZAfdLaJQvGehYOnSpU0toSFDhqi1a9fqk7EI4dQiAp4DLgNSArffAm8AXwUygDQRuUZE4vD3I72qLOTI9hKjhQ4Tko0FCfYFuZ3gINH09HQyMzOZP3++ZqPOwxWBSClVq5Q6HLzhbxGdUkodVUrtAr6HPyCVAz2A+62U6yVGCx0mJRuzkwZEF2fOnGHTpk2A/7PgJc93AUqpR5VSdzW7v04pNUQpFaeUulkpdcxKOV6LKHR4LSJn+eCDD6ioqAD+t0V0MeHKQOQUXosodHgtImf59NNPefrppwHYt28f48aNa8pVdTEQ1oHI7YnRmrN48WLdCrYIjl8xgeD4ITezePFi7rnnHgC++tWv8vzzz5OQEFYzmVolrAORSRMzX3zxRd0Kthg7dqxuBcu8+KI5rgD5+fm6FTqdsA5EJs1ivvnmm3Ur2OLzzz/XrWCZm282xxVg6FAnpmyaRVgHoujoaMv7hiLPl50yt23b5ryATeystnHddYMs76ubbdsGWd7XDfneysrKdCt0Ou5PYdgBfD6f5X1156wfN26cXgHsrWAxblwFhw51b3vHEOIf39o2Bw5UkJys19UOl1xyiW6FTiesW0RuT4xWX1/P22+/jVKKkpISampqePfdd3VrWaKkxJwvdvNVVE2gurq67Z3CDHd/UzuIsvqTqYmuXbvyox/9iKeeeoq4uDi+/OUvG3M1Ki7utG4Fy5w+bY4r+Fd3udgI60BkAmlpaVRXVxMdHU11dbUx6WKjo81IOgfmJMgL4vaWfCgI6yM24Q0NBp7y8nLGjBnDsGHDNBtZo7zcnMGiJg1sBbPGvzmF+7+pHcBOZ7UuJk2aRL9+/Rg5cqQxrSGAkSPNGfVr2ghlk7JfOkVYByITzrUjIiKYPXs2O3bsMCoQ7dgxQLeCZQYMMMcV4Iidy5dhQlgHIlNyQM+ZM4e5c+dy9dVX61axzA03FOpWsExhoTmuAEOGDNGt0OmEdSAyZYrHzJkzERFjkv0DrFs3SreCZUaNMscV/JNeLzbCOhCZkgYkLy+Pq6++mhMnTuhWscySJTt1K1hm504zXKOioli8eDEzZ87UrdLphHUgMuVqyZtvvsnChQuN6lR9+ulU3QqWSU01wzU6OpoBAwZw7bXX6lbpdMI6EJnSIgLzEqOZkGwsiElJ3F577TW2bt2qW6PTCetAZEqLCMxLjGZCsrEgpiRxKykp4eOPP2b9+vW6VTqdsA5EJrWIli5dqlvBFsEFD00guJCh29m4cSMAgwYNMm5aSkcJ60BkUoto1apVuhVssWpVim4Fy6SkmOEazFP91FNP8f7772u26VzCOhCZtMDinXfeqVvBFnfeaU4WQRMyHtbU1LBlyxbA/1kIto4uFsI6ENlJjKabzZs361awxebN5mQRNCHj4ZYtW5oG4G7evJnMzEzXZ49wkrAORCadZ0+ePFm3gi0mTzYni6AJGQ8jIyObru6lp6fzyCOPUFlZqdmq8wjrDI1RUeYcnklryQPs3WvOChMmrIYxe/bsposrVVVV3HXXXW08I7wI6xaRSXlo+vTpo1vBFn36mHNF0qSrpwDdunXTrdDphHUgMgk3TNC1k76poUHvR8dOknsT8lI1x6QfUKcw59ylHYgblpCwSE1NjW4F7Hz+a2rspVjR2e9qQjqY5pjUt+kUZv1U2MSkX5ZBg6wveeMGBg0yJ8G7acnoTUv27wRhHYjc3lldX1/Pww8/TEVFBXl5eezYsYPf/e53urUskZeXqFvBMomJ5rgCfPHFF7oVOp2wDkRu6Hdpja5du7J9+3aeffZZpk6dyh133GFMR+XUqSW6FSxTUmKOK5iXUdIJwjoQde3aVbdCmwTTw27YsIHIyEhuvPFGzUbW2LBhuG4Fywwfbo4rmJdR0gnCOhCZMMUjGIgWLFjAlClTjBjzArBgwS7dCpbZtcscVzAvo6QThHUgMuE0Z9iwYYwZM4bVq1cblTx/9eoJuhUsM2GCOa5gXuB0AtcEIhHpKiK/F5EiETkpIjtF5MZmj88QkXwRqRWRrSKS1FaZpgxkmzNnDunp6cyZM0e3imW8xGjto76+vs05ZOPHj+8kG/fgmkCEf0zTQeBaoCewDPiziCSLSCLwKvBTIAHIAl5uq0BT0oCkpaWxceNGLr/8ct0qlvESo7WPY8eOMXbsWL7//e/z1ltvUV9ff84+n3zyiQYzvbjm+rZSqgZ4tNmmjSJSCEwELgF2KaX+AiAijwIVIjJKKXXBHA+mtIi+/OUvs2zZMqMGYKanZxsTjLKzs0MSjJYvX96uRTzPnDnDqlWrWLVqFd27d+erX/0qc+bMYfr06YDXInIVItIXGAnsAsYCucHHAkGrILC95fMWiUiWiGSdOHGCiooKDh06RGlpKZWVlRQUFFBXV8fu3btpbGxsyt4XbL7n5OTQ2NjI7t27qauro6CggMrKSkpLSzl06BAVFRUcOHCA6upq8vPz8fl85ObmnlVG8G9eXh719fXs27ePqqoqiouLKS8vp7y8nOLiYqqqqti3bx9//vOfOXjwIEeOHDmnjNzcXHw+H/n5+VRXV3PgwIGQHVP//tXcdls+MTE+Fi/2H1PwFCz4d+HCPOLj6ykqiicpqYrp04tJSSknJaWc6dOLSUqqYu7cfcTH17NwYd5Zz9VxTNXV1cTFxTnyPtXX15OX5z+mDz74gBUrVlBVVcXjjz/OkSNHePLJJ/nss894/vnn+eCDD1i/fj1vvvkmW7Zs4Y9//CPZ2dk8++yzXHXVVURERLB06VKqq6sZPHgw999/P3/5y1/41a9+xejRozvts9f8mNrz2XMKcWPOExHpArwJFCilFovI74GjSqmfNNvnn8DzSqk/XKicsWPHKhM6/h555BFKSkpYtmyZ1tw5dhpkCxfmsWbNOMv76/yY5eXlMW6cdVcrrF+/noKCAr75zW/afu7cuXPJzc1l4MCBzJkzh7S0NK677rqmiyuh8A0VIpKtlJrU0XJcc2oWREQigP8BGoAlgc3VQHyLXeOBk62VZcoCi+D/YC9btky3hmXWrx+pW8EyI0c675qZmcmJEyd48MEHbT3vwIEDfOMb32DNmjVceeWV5z0dD4Wv23HVqZn435XfA32BeUqp4Oy/XcCEZvvFAZcFtl8Qt4+sbs6MGTN0K9hixoxi3QqWKS521vXMmTNs2rSJt99+2/ZYteTkZB5++GFSU1Mv2CfotK8JuCoQAb8BRgNpSqm6Ztv/ClwhIvNEJAZ4GPiktY5qcP9cs+aYstJEkJwcG3k4NNPXTs4QC+zYsYOKigpqa2t55513HC0bnPc1AdcEosC4oMVACnBYRKoDt/lKqaPAPGAFUAlMBm5vq8wzZ86EUtlRhg0bplvBFsOGHdetYJnjx511Da620fJ/p3Da1wRcE4iUUkVKKVFKxSiluje7ZQQe/7tSapRSqptSappS6kBbZZqUEMu0/MSVleb0vzndV9g8+GzcuNHxJPcm9W06hTnfVA8PF1BYWMjBgwfp2bMncXFx1NbWsnPnTt1axhPWgcikxGi9e/fWrWCL3r3dP6E4iJOTnyMiIigsLGTUqFEMHDiQwsJC4uNbXtDtGCZM1nYac3pz20FkZKRuBcvs379ft4It9u/vpVvBMr16OeealHT2FMf4+HjHA5GTvqYQ1i2i9gy/10VqaqpuBVukph7RrWCZI0fMcQXzfJ0grFtEJq30GlxuWCd9+4LV78CWLUNslauTIUOsu4aKfv2s1218/BCqqqztGxFhb9EDXWW2+Zqd+3Kdi0nn2rfccotuBQ4f9k/FsHL7xz/2Wt738GG9x+WGxSvtNHJuucW6bygCho6u1bAORG5PjNbY2Nh06XfNmjWAOWOfTJkLBWa5Arbm8IULYR2I3J4G5MyZM9x4441kZWWRnp7OypUrWb58uW4tS7gp2VhbmOQKZiWdc4qw7iNye2K0Ll260KtXL15++WU2bdoEwObNmzVbWcNNycbawiRXMCvpnFN4LSLNBPNUp6enExcXx7Rp0/QKWcSkVoZJrnBxtohcmY/IKSZNmqSysrJ0a7TKF198QZ8+fWhsbGTu3Lm8+uqrupU8LHD11VdTWVnJZ599Zml/g5Jv2sSZfERh3SKqq6treycLJMTHIyIXvCV0YEDbJZdcwpQpU1i8eLFRq3gEMwOagEmuQFN2zIuJsA5ETk0erDx5EgUXvFWe9Odnq6qquuBSMDt37rxgYExLS+Oll15i9uzZtrzOnDnDRx995PhUlqNHj1JQUNDqPmPHnpOlt0327t17weWUMzIySE5OJiIiguTkZDIyMiyX+/HHH7c6VKM9rj6fj6ysLC3ThF580b6v6YR1IDrfCgmhYtasWSQmJl5weeMPP/yQxMREbr75Zn73u99x6NChpsfS0tK4//776dOnj63XjIyM5Fe/+hUDBgzgnnvu4a9//SvV1dUdOg7wT1uYNm0ao0eP5sc//jHvv//+OaPUP//8c9vlNjQ00K9fP6655hqefPJJ9uzZg1KKjIwMFi1aRFFREUopioqKWLRokeVg9M9//pPExETmzp3LmjVrONxi4FJ7XKOiovj5z3/OwIEDuffee9mwYQM1NTW2y2kPN99s39d0wrqPKDU1VTmRcExEaK2Wmp/+X2jekVKKkyfPzmzbvXt3LrnkEhISEhg4cCAnTpyw7VZeXn5WP0XXrl2ZPn06aWlpzJkzh8WLF7frC7R3796zphokJCRw0003MWfOHC677DIef/zxduXN+eijj85qvQwbNoyjR4+eUzfBY7nqqqvaLLO+vp4PP/zwrG1XXXUVaWlppKWlsX79et577z3brkeOHDlrMGRMTAzXXXcdaWlpPPfcc9TU1ISkj6h//2oOHepuV1cTYZqz2kl0zDW70JW68wX8uro6jh07Rl1dHddeey2vvfaa7ddrGWTq6+spKSmhpKSEsrIySktLz/slb4uWx3HixAlKSkooLS2lR48edO/evWn1Bzu0fE8uFITAfywHDx5ss8zzDQItLS2ltLSUkpISIiIiLJXTkpaty1OnTjWV+dRTT4VsfuC4cRUGBSKHUEqF7W3ChAnKCWhjFgOgRo8erQC1ffv285bxxz/+UQEqNTVVPfLIIyorK0s1NjY2PX706NF2uS1atEh16dJFzZo1S/36179WhYWF7SqnOT6fT11++eWqV69e6o477lDr1q1Tx44dO2uf9vgWFBSoqKgolZycrL7//e+rt956S9XX16ukpKTzdr8lJSVZKvfFF19UgJo0aZJ67LHHVE5OjiN1e/fdd6vo6Gh1ww03qGeeeUYVFRW1qxylrE6G8d/GjDlqa3+9N7KUA99V7cEilLfx48e3+4PTHCuBSCmlPv/88wsGonfeeUeVlJRc8DXKyspse/l8PvX666+rqqoq289tjbKyMvXuu++q06dPt7qPXXJyclReXt5ZQUIppdauXatiY2PPCkKxsbFq7dq1lsr9+9//rkpLSx11PX36tHr99dfVyZMnbT/3fNj5cn/pS2UuCDBWb14gavPW2YGoI7QWpNyI075r165VSUlJSkRUUlKS5SBkBTfUrZ0v95QpJS4IMFZvzgSisO4jcipnde8ePZBW+ll69+jR4ddw+3SUljjtO3/+fObPn+9omUFMq9vycrN8nSCsL9871Vl9rKqq1Wh+zGrymNZe49gxB0w7D5N8TXIFGDnSLF8nCOtA1KVLF90KlhkwYIBuBVuY5GuSK8COHWb5OkFYByKTVnotLCzUrWALk3xNcgW44QazfJ0grAc0mjDpNUhjY6NR67CZ5OsGVzsDGiMiGmlsNKNuvUmvFjAhDUgQ09bGMsnXJFeAJUvM8nUCr0Xk4dEJeGlAWsdrEbkE05J3meTrBlc7Z4ZeYrQww2sRebgFr0XUOl6LyCU4kSWgMzHJ1yRXgKVLzfJ1Aq9F5BLccGXHDib5usHVu2rWOqYcbbswaYHF/Px83Qq2MMnXJFeAO+80y9cJwjoQmbTk9NChQ3Ur2MIkX5NcATZvNsvXCcI6EJ0+fVq3gmXKysp0K9jCJF+TXAEmTzbL1wnCOhBFRZmTXCAhIUG3gi1M8jXJFWDvXrN8ncCYQCQiCSLyVxGpEZEiEbmzrefoWIGhvZh0hQ/M8jXJFaBPH7N8ncCcJgM8AzQAfYEU4A0RyVVKnX/9HsPQfVXHLib5muQK0NBglq8TGHHEIhIHzAN+qpSqVkr9A3gd+FYbz+sMPUcwKWUJmOVrkitATY1Zvk5gSotoJOBTSu1tti0XuLbljiKyCFgUuFsvIp92gp8TJAIVuiVsYJKvSa7s3m2U7+VOFGJKIOoOtEyDeAI4J0erUuo54DkAEclyYrBVZ2CSK5jla5IrmOUrIo6MGDbi1AyoBlquXBgP2F+wy8PDw3WYEoj2AlEiMqLZtglAWHRUe3hc7BgRiJRSNcCrwOMiEiciU4Cbgf9p46nPhVzOOUxyBbN8TXIFs3wdcTVm0quIJABrgJnAF8BPlFLr9Fp5eHg4gTGByMPDI3wx4tTMw8MjvPECkYeHh3bCMhC1Z15aZyEiXUXk9wGvkyKyU0RubPb4DBHJF5FaEdkqIkk6fYOIyAgROSUia5ttuzNwHDUi8lqgH087InK7iOwJeBWIyDWB7a6qWxFJFpFNIlIpIodFZJWIRAUeSxGR7IBrtoikdLLbEhHJEpF6EflDi8cuWI+Bz/caEakKHNMDll6wtaWUTb0BfwRexj8Q8v/gH/w4VrdXwC0OeBRIxv9DMAf/eKhk/COATwC3AjHAL4EPdDsHvN8C3gfWBu6PDXhPDdTzOuBPLvCcCRQBVwfqd2Dg5rq6BTYBfwj49APygKVAdOAYfgh0DWwrAqI70e0bwNeB3wB/aLa91XoEfh74nPQGRgOHgRvafD3dH5wQVGAc/smxI5tt+x/gF7rdWnH+BP9cukXA9hbHUgeM0ux3O/DnQAANBqKfAeua7XNZoN57aHbdDtxznu2uq1tgD3BTs/u/BFYDs4BSAheTAo8VW/lCh8DxiRaBqNV6BMqAWc0eX27lByocT80uNC9trCafVhGRvvidd+F3zA0+pvzjpwrQ6C4i8cDjQMsmdkvXAgI/AJ1ndzYiEglMAi4Vkc9FpCRwutMNF9YtsBK4XURiRWQgcCOwOeD0iQp8kwN8gjs+wxesRxHpDfRv/jgWv3vhGIgsz0vTjYh0ATKAF5VS+fjdT7TYTbf7cuD3SqmSFtvd6NoX6ALcAlyDP13MlcAy3Om7Df+XtAooAbKA13Cna5DW3Lo3u9/ysVYJx0BkxLw0EYnAf8rYACwJbHaVe6CD9HrgqfM87CrXAHWBv79WSh1SSlUAvwJuwmW+gfd/M/4ZA3H4+156A/+Fy1xb0JpbdbP7LR9rlXAMRK6flyb+REm/x/8LPk8pFUyuvQu/a3C/OPx9L7rcp+HvRC8WkcPAfwDzRCSHc12H4e9Y3XtuMZ2DUqoSf8ui+SlN8H+31W0CMARYpZSqV0p9AbyAP2juAsbL2Qm1xuOOz/AF6zFQ/4eaP47V756ujroQd7D9Cf+VszhgCi66ahbw+y3wAdC9xfZLA67z8F+R+C80XtkBYvFfzQne/htYH/AMnlJcE6jntbjjqtnjwEdAH/wtjPfxn166qm4DrvuBn+BPx9ML+Cv+q4/Bq2Y/wB/cl9D5V82iAvX0c/wt95jAtlbrEfgF8F6g7kcFAtPFd9UsUBkJ+M+1a/BfbbhTt1MztyT8v9Kn8Ddlg7f5gcevB/Lxn2a8CyTrdm7m/iiBq2aB+3cG6rcG2AAkuMCxC/AscBz/peOngRg31i3+Pqx3gUr8idD+DPQNPHYlkB1wzQGu1PBeqxa3R9uqx0DgXIP/R+oI8ICV1/Pmmnl4eGgnHPuIPDw8DMMLRB4eHtrxApGHh4d2vEDk4eGhHS8QeXh4aMcLRB4eHtrxApGHVkTkWhHZG5iw2pFyPhSReU55eXQuXiDy0M0vgRVKqTMdLGc58IvAHC4Pw/DeNA9tiMhX8E8D+LMDxW3CP8v7xrZ29HAfXiDy6DAicqmIHBKRR5ptGx9ILXtrK0+9E9iilKpr9rxHReRTEVkgIgcC6V5fEJFoEblfRA6KyBci8qvmrZ9Ai2oTcEcojtEjtETpFvAwH6XUURG5G8gUkbeAnfgnHf9RKfWXVp56DedvDSXjX0BzDv40r6/gT7h1CH/2wmAr6p+Bx4J8CDzUkWPx0IMXiDwcQSn1NxF5Fn+it/fwT378fhtPS8KfWrQlkcB3lFIngE9FZDNwLTBQKdUA7BGRfwLTOTsQlQEDRSRKKeXr2BF5dCbeqZmHkzyIP9Hbt/FnE6huY/9u+LMQtKQ4EISCHAH2BoJQ8219WjyvDhD86Sk8DMILRB5OkgwMxp8yYpiF/Svw561pyekW99UFtrX8/CYApywEQA+X4QUiD0cI5N9eB7yOP5PjsyIypI2nfQyMcVDjCvy5ezwMw+sj8nCKYBbEGfgz+N0AvCQi1ymlGi/wnL8B9zjocA3+PNAehuG1iDw6jIhcC/w78G2l1HHlz7Z3N/7WzoOtPHUtMFJEOrxMTmA5nq/gz/vsYRhehkYPrYjIL4BLlVIdahmJyC+BnkqpRc6YeXQmXovIQzc/A/Z3dK4ZUA781AEfDw14LSIPDw/teC0iDw8P7XiByMPDQzteIPLw8NCOF4g8PDy04wUiDw8P7XiByMPDQzv/P/G3Q1wlABxGAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Collected data from each sensor and the corresponding time:**"
      ],
      "metadata": {
        "id": "j2zOb5D_zpwP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure()\n",
        "for ii in range(num_sensors):\n",
        "  plt.plot(range(counter),Collected_data_time[ii], label =' sensor located at ({0:.0f},{1:.0f})'.format(\\\n",
        "              env.sensors_XY[ii][0], env.sensors_XY[ii][1]))\n",
        "  plt.fill_between(range(counter),Collected_data_time[ii], np.zeros(counter), alpha = 0.2, linewidth=2)\n",
        "  plt.xlabel('Steps', size = '14')\n",
        "  plt.ylabel('Collected Data from each sensor', size = '14')\n",
        "  plt.xlim(0,counter)\n",
        "  plt.ylim(0, 5.5* 10**9)\n",
        "  plt.xticks(np.arange(0, 22, step=2), size = '12')\n",
        "  plt.yticks(size = '12')\n",
        "  #plt.grid(True)\n",
        "plt.legend(loc=0, fontsize = 'medium')\n",
        "#plt.axvline(15,linestyle='dashed')\n",
        "plt.savefig('C_3.pdf',format = 'pdf')\n",
        "plt.show()  \n",
        "   \n",
        "print('number of steps taken by the agent: ', number_of_steps_taken_to_terminal)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 315
        },
        "id": "_aJIBpeQzkQ7",
        "outputId": "abc849d5-f823-4993-9572-c799021e4828"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEYCAYAAACwQCa4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5b3H8c8vCwlhkyUgqyBhhyQiGEtFBBdEBVFAsFYBFRRQKa1Fe9WKihWXi7tYvKCiUHC5Uq9arYCoaFuImLAECAGCBlAJYc+e/O4fZzIGTMhJMpPJkN/79ZoXyZk5z/klIc+cPOc530dUFWOMMXVDSKALMMYYU3Os0zfGmDrEOn1jjKlDrNM3xpg6xDp9Y4ypQ6zTN8aYOiQoOn0RWSgiP4nIJhevPUtEVorIBhFZLSLtaqJGY4wJBkHR6QOvAZe7fO1TwCJVjQUeBh7zV1HGGBNsgqLTV9UvgKzS20Sks4h8LCLfiMiXItLd81RPYJXn48+Aq2uwVGOMqdWCotMvx3zgTlU9F7gbeMmzPRm41vPxNUAjEWkegPqMMabWCQt0AVUhIg2BAcDbIlKyOcLz793ACyIyAfgC2AMU1XSNxhhTG7nq9EUkCshV1WI/1+NWCHBIVeNPfkJV9+I50/e8OYxS1UM1XJ8xxtRKFQ7viEgocBjoXtFra4qqHgF2icgYAHHEeT5uISIlX9efgIUBKtMYY2qdCjt9VS0CdgP1/F9O2UTkb8C/gG4ikiEitwA3ALeISDKwmZ8v2F4EbBORVKAV8GgASjbGmFpJ3EQri8h44Hrgt6qa6feqjDHG+IXbTn8j0AkIBzKA46Wf98yJN8YYU8u5nb3zjl+rOIUWLVpox44dA3V4Y4wJSt98802mqkafvN1Vp6+qD/m+JHc6duxIYmJioA5vjDFBSUR2l7W9UvP0RWQIzh2vCmxW1dXVL80YY0xNcTtPvy3wHnAusNezuY2IJALXeObGG2OMqeXcxjA8h3NXa4yqtlfV9kAXz7bn/FWcMcYY33I7vHMpcJGq7irZoKo7ReQuYKVfKjOmliooKCAjI4Pc3NxAl2IMkZGRtGvXjvDwcFevr8yYfllzOyue72nMaSYjI4NGjRrRsWNHSmU/GVPjVJUDBw6QkZFBp06dXO3jdnhnJfC8iLQv2SAiHYBnsDN9U8fk5ubSvHlz6/BNwIkIzZs3r9RfnW47/buABsBOEdntmQq0w7PtrkpXakyQsw7f1BaV/b/odp7+9yLSF7iEn4PXtqjqisqVZ4wxJpBcj+mrk9fwqeeBiLi7amCMMabWcDW8IyJ3icioUp8vAHJEZJuIdPNbdcaYoDNhwgTeecd3yS3p6eksWbLE73UsX76clJSUcp9/5plnWLRoEQCzZs2ibdu2xMfHEx8fz0cffeR93WOPPUZMTAzdunXjk08+KbOt++67j/bt29OwYcMTtufl5TF27FhiYmJISEggPT0dgI0bNzJhwgTXX8upVGZMfz+AiFwIXAf8BkgC/tsnlRhjglJRkX8Xpqtqp19Zp+r0CwsLWbhwIb/5zW+822bMmEFSUhJJSUlcccUVAKSkpLB06VI2b97Mxx9/zNSpU8v8/gwfPpy1a9f+YvuCBQto2rQpaWlpzJgxg3vuuQeAPn36kJGRwXfffVftr9Pt8E5boGSO/nDgbVV9y5O++WW1qzAmSD30f5tJ2XvEp232bNOYB4f3cvXaoqIibrnlFhITExERbr75ZmbMmMGOHTuYNm0a+/fvJyoqildeeYXu3bszYcIEGjduTGJiIj/88ANPPPEEo0ePZt++fYwdO5YjR45QWFjIvHnzGDhwIH/729/4y1/+gqpy5ZVX8vjjjwPQsGFDbrvtNlasWMGLL77IBRdcUGZ9K1eu5O6776awsJD+/fszb948IiIiWLduHdOnT+f48eNERESwcuVKDhw4wI033sjx406I7wsvvMCAAQO499572bJlC/Hx8YwfP5677rqLe++9l9WrV5OXl8e0adO47bbbUFXuvPNOPv30U9q3b0+9emUvAfLKK68wf/588vPziYmJ4Y033iApKYn333+fzz//nNmzZ/Puu+/SuXNn7z6rVq2ib9++hIWdusv8+9//zrhx44iIiKBTp07ExMSwdu1afvWrX53wuvPPP7/c/WfNmgXA6NGjueOOO1BVRIThw4ezdOlSZs6cecoaKuL2TP8I0NLz8aX8PE2zAIisVgXGmCpLSkpiz549bNq0iY0bNzJx4kQAJk+ezPPPP88333zDU089xdSpU7377Nu3jzVr1vDBBx9w7733ArBkyRKGDh1KUlISycnJxMfHs3fvXu655x5WrVpFUlIS69atY/ny5QAcP36chIQEkpOTy+3wc3NzmTBhAsuWLWPjxo3eN5P8/HzGjh3Ls88+S3JyMitWrKB+/fq0bNmSTz/9lPXr17Ns2TLuusuZGDhnzhwGDhxIUlISM2bMYMGCBTRp0oR169axbt06XnnlFXbt2sV7773Htm3bSElJYdGiRXz99ddl1nXttdeybt06kpOT6dGjBwsWLGDAgAGMGDGCJ598kqSkpBM6fICvvvqKc88994RtL7zwArGxsdx8880cPHgQgD179tC+vXdmO+3atWPPnj2uf56l9w8LC6NJkyYcOHAAgH79+vHll9U/x3Z7pv9P4BURWQ/EAP/wbO/Fz38BGFPnuD0j95ezzz6bnTt3cuedd3LllVdy2WWXcezYMb7++mvGjBnjfV1eXp7345EjRxISEkLPnj358ccfAejfvz8333wzBQUFjBw5kvj4eFatWsVFF11EdLSTznvDDTfwxRdfMHLkSEJDQxk1ahSnsm3bNjp16kTXrl0BGD9+PC+++CIXX3wxrVu3pn///gA0btwYcN5I7rjjDpKSkggNDSU1NbXMdv/5z3+yYcMG73j94cOH2b59O1988QXXX389oaGhtGnThiFDhpS5/6ZNm7j//vs5dOgQx44dY+jQoRV+n/ft20ePHj28n0+ZMoUHHngAEeGBBx7gD3/4AwsX+ndl1pYtW7J3b/Vjztye6U8DvgKigdGqmuXZ3hf4W7WrMMZUSdOmTUlOTuaiiy7i5Zdf5tZbb6W4uJgzzjjDO96clJTEli1bvPtERER4Py5ZROnCCy/kiy++oG3btkyYMMF7wbI8kZGRhIaG+vRrefrpp2nVqhXJyckkJiaSn59f5utUleeff977te3atYvLLrvM9XEmTJjACy+8wMaNG3nwwQdd3dhUv379E17XqlUrQkNDCQkJYdKkSd7x+bZt2/L99997X5eRkUHbtm1d11Z6/8LCQg4fPkzz5s0B5y+n+vXru26rPK46fVU9oqp3qurVqvpxqe0Pqupf3B5MRFaLSK6IHPM8tlWlaGOMIzMzk+LiYkaNGsXs2bNZv349jRs3plOnTrz99tuA00kmJyefsp3du3fTqlUrJk2axK233sr69es577zz+Pzzz8nMzKSoqIi//e1vDBo0yHVt3bp1Iz09nbS0NADeeOMNBg0aRLdu3di3bx/r1q0D4OjRo94OrnXr1oSEhPDGG294L4A2atSIo0ePetsdOnQo8+bNo6CgAIDU1FSOHz/OhRdeyLJlyygqKmLfvn189tlnZdZ19OhRWrduTUFBAYsXL/ZuP/k4pfXo0cP7dYBz5l/ivffeo3fv3gCMGDGCpUuXkpeXx65du9i+fTvnnXceABdffHGFQz0jRozg9ddfB+Cdd95hyJAh3puvUlNTvcepjsrm6bfBGds/4c1CVddXopk7VPV/KnNcY0zZ9uzZw8SJEykuLgac6YIAixcvZsqUKcyePZuCggLGjRtHXFxcue2sXr2aJ598kvDwcBo2bMiiRYto3bo1c+bMYfDgwd4LuVdffbXr2iIjI3n11VcZM2aM90Lu7bffTr169Vi2bBl33nknOTk51K9fnxUrVjB16lRGjRrFokWLuPzyy2nQoAEAsbGxhIaGEhcXx4QJE5g+fTrp6en07dsXVSU6Oprly5dzzTXXsGrVKnr27EmHDh1+cfG0xCOPPEJCQgLR0dEkJCR4O/px48YxadIknnvuOd55550TxvWHDRvGjTfe6P185syZJCUlISJ07NiRv/71rwD06tWL6667jp49exIWFsaLL75IaGgoxcXFpKWl0axZM+/+S5YsITs7m3bt2nHrrbcya9YsbrnlFm688UZiYmJo1qwZS5cu9R7zs88+48orr3T9/S+Xqlb4AM4BNuNEKRef9Chy04anndXArW5fr6qce+65akxtkpKSEugSTACMHDlSU1NTq7Tvxo0bdcaMGVU+dm5uriYkJGhBQUGZz5f1fxJI1DL6VLdj+vOB74GBwNk4i6SXPM6u5PvMYyKSKSJfichFZb1ARCaLSKKIJO7fv7+SzRtjjO/NmTPnhGGdyujduzdz586t8rG/++475syZU+GUUTfcttATOEdVy76c7t49QAqQD4wD/k9E4lV1R+kXqep8nDca+vXrZ/HNxpiA69atG926BSaAoEuXLnTp0sUnbbk9098InFndg6nqf1T1qKrmqerrODOCrqhuu8YYY9xx2+n/F/CEiFwiIq1EpFnpRzWOr4Bl1BpjTA1xO7xTEqH8T05cLUs8n1c4YVdEzgASgM+BQmAscCEw3W2xxhhjqsdtpz/YB8cKB2bj5PEXAVuBkT64TmCMMcYltzdnfX6qh8s29qtqf1VtpKpnqOr5qvpp9co3xtQ2dSFa+Y9//CPdu3cnNjaWa665hkOHDnlf5yZaOT8/n8mTJ9O1a1e6d+/Ou+++C9SuaGVEpI+IvCAi/xCR1p5tI0XkHJ9UYowJSnUxWvnSSy9l06ZNbNiwga5du3pvinMbrfzoo4/SsmVLUlNTSUlJ8d7pXGuilUXkMuB9nKC1IUBJAERnYAIwstqVGBOM/nEv/LDRt22e2QeGzXH1UotWDky0cumsn/PPP9/7F4XbaOWFCxeydetWAEJCQmjRooV3/9oSrfwI8HtVvQZnjn2J1cB51arAGFNlFq0cuGjlEgsXLmTYsGGAu2jlkqGgBx54gL59+zJmzBhv2mltilbuDXxUxvYsoDpTNo0Jbi7PyP3FopUDE61c4tFHHyUsLIwbbrihwjZKFBYWkpGRwYABA5g7dy5z587l7rvv5o033jjlfjUdrZyFs3rWyfoCGdWuwhhTJRatHJhoZYDXXnuNDz74gMWLF3uTMN1EKzdv3pyoqCiuvfZaAMaMGcP69et/sX9Ao5WBJcCTItIOZ15+mIgMAp4CTv2/wxjjNxatHJho5Y8//pgnnniC999/n6ioKO92N9HKJePzq1evBpzrHj179vTu7+9oZbfpmOHAYn5O2Sz0fPwGEOqmjao+LGXT1Da1KWUzKSlJzznnHI2Li9O4uDj96KOPVFV1586dOnToUI2NjdUePXroQw89pKqq48eP17ffftu7f4MGDVRV9bXXXtNevXppfHy8XnDBBbpz505VVV2yZIn27t1be/XqpTNnzvzFfmUpfYwVK1ZofHy89u7dWydOnKi5ubmqqrp27VpNSEjQ2NhYTUhI0KNHj2pqaqr26dNHY2NjdebMmd5j5Ofn6+DBgzU2Nlbnzp2rRUVF+qc//clb10UXXaSHDh3S4uJinTZtmnbt2lUvueQSHTZs2Alfa4mXXnpJO3bsqP3799c77rhDx48fr6qqa9as0R49emh8fLympaWdsE96eroOHDjQ+3nnzp21Xbt23u/7bbfd5n1u9uzZevbZZ2vXrl29P4+ioiLt0KGDZmdnn9Benz59dMiQIbp7925VVc3JydHRo0dr586dtX///rpjxw5vu9OmTdP333+/zO95ZVI2RdV9npmInI0zpBMCfKuq26v/tnNq/fr108TERH8fxhjXtmzZUub4rjm9XXPNNTzxxBNVCj7btGkTCxcurHLSZl5eHoMGDWLNmjVlJm2W9X9SRL5R1X4nv9b1PH0AVd2pqu+o6luAiogtim6MqRPqVLSyiPwF2Kaqr4szwPRP4GLgsIhcrqr/qXYlxhhTi9W1aOUbgJL1bIcB8cD5OBdxAztnzRhjjGtu/1Zoxc9TM68A3lLVtSKSBdiAuzHGBAm3Z/oHgLM8H18GrPR8HIbl4RtjTNBwe6b/LrBERFJx7sAtiY6LB9LK3csYY0yt4vZM//fAczjr216qqsc921sD8/xRmDEmONWFaOWkpCTOP/984uPj6devH2vXrgWc+57uuusuYmJiiI2N9d5pW5aioiLOOeccrrrqKu+2Xbt2kZCQQExMDGPHjvXelfzCCy+wcOFC11/LqbjN0y9U1f9W1emq+m2p7U+r6v/4pBJjTFCqi9HKM2fO5MEHHyQpKYmHH37Ym3z5j3/8g+3bt7N9+3bmz5/PlClTyj3es88++4u59ffccw8zZswgLS2Npk2bsmDBAgBuvvlmnn/+eV98ma6Hd4wxZXh87eNszdrq0za7N+vOPefd4+q1Fq0cmGhlEeHIkSOAE/jWpk0bwIlGvummmxARzj//fA4dOsS+ffto3br1CTVkZGTw4Ycfct9993nn76sqq1at8r7BjR8/nlmzZjFlyhSioqLo2LEja9eu9cY6VJV1+sYEsdLRyvBzbO/kyZN5+eWX6dKlC//5z3+YOnUqq1atAn6OVt66dSsjRoxg9OjR3mjl++67j6KiIrKzs73Ryt988w1NmzblsssuY/ny5YwcOdIbrfzf//3f5dZWEq28cuVKunbtyk033cS8efOYOnUqY8eOZdmyZfTv358jR46cEK0cGRnJ9u3buf7660lMTGTOnDk89dRTfPDBBwDMnz/fG62cl5fHr3/9ay677DK+/fZbb7Tyjz/+SM+ePbn55pt/Ude1117LpEmTALj//vtZsGABd955JyNGjOCqq65i9OjRv9jn5GjlZ555hqFDh3L33XdTXFzsjXEuL1r55E7/d7/7HU888cQJWT8HDhzgjDPO8L6xnBzLXBKtbJ2+MQHk9ozcXyxaOTDRyvPmzePpp59m1KhRvPXWW9xyyy2sWLGiwnYAPvjgA1q2bMm5557rDV1zo2XLlt6FV6qjUjEMxpjaxaKVAxOt/Prrr58QjVxyIddNtPJXX33F+++/T8eOHRk3bhyrVq3it7/9Lc2bN+fQoUMUFhaWuW9NRysbY2ohi1YOTLRymzZt+PzzzwFnvL8kImHEiBEsWrQIVeXf//43TZo08Q7tdO/eHXAWTs/IyCA9PZ2lS5cyZMgQ3nzzTUSEwYMHe/+Cef3117n66qu9x/RVtLLb7J1IYDpO3k5LTnqzUNXYaldijKm0PXv2MHHiRIqLiwG8C3QvXryYKVOmMHv2bAoKChg3bhxxcXHltrN69WqefPJJwsPDadiwIYsWLaJ169bMmTOHwYMHey/klu6EKhIZGcmrr77KmDFjvBdyb7/9durVq8eyZcu48847ycnJoX79+qxYsYKpU6cyatQoFi1axOWXX06DBg0AiI2NJTQ0lLi4OCZMmMD06dNJT0+nb9++qCrR0dEsX76ca665hlWrVtGzZ086dOjwi3VpSzzyyCMkJCQQHR1NQkKCt6MfN24ckyZN4rnnnuOdd9454ULusGHDuPHGG72fv/LKK0yfPp3CwkIiIyOZP38+AFdccQUfffQRMTExREVF8eqrrwLOm7ObROPHH3+ccePGcf/993POOedwyy23eJ/76quvvOvnVoeraGURWQhcA7wN7MVZSMVLVR+qdiXlsGhlU9tYtHLdVJ1o5Q8++ICdO3d61/2trG+//Za5c+eWu6RiZaKV3V7IHQmMUVV3VyqMMeY0UxKtXJVOv/QNWFWRmZnJI488Uq02Srjt9LOB7yt8lTHGnKYCGa186aWX+qwttxdynwB+LyWLNRpjjAlK5Z7pi8j7J226ELhcRFKAgtJPqOoIP9RmjDHGx041vHPgpM/f82chxhhj/K/cTl9VJ9ZkIcYYY/zP1Zi+iPQSkV/MxReRWBHp6fuyjDHBqi5EKwM8//zzdO/enV69enlTNsG5VyImJoZu3brxySeflNUU+fn5TJ48ma5du9K9e3feffddwInLGDt2LDExMSQkJJCeng7Axo0bmTBhguuv5VTcXsidD5R1K1hPz3OVIiJdRCRXRN6s7L7GmNqlLkYrf/bZZ/z9738nOTmZzZs3c/fddwOQkpLC0qVL2bx5Mx9//DFTp04t8/vz6KOP0rJlS1JTU0lJSfHe6bxgwQKaNm1KWloaM2bM4J57nGynPn36kJGRwXfffVftr9PtlM1YYG0Z29cBfapw3Bc9+xoT1H74y1/I2+LbaOWIHt0587/+y9VrLVo5MNHK8+bN49577/XmGLVs2RJwopXHjRtHREQEnTp1IiYmhrVr1/7i7uCFCxd6w9NCQkJo0aKFd/+Su25Hjx7NHXfcgaoiIgwfPpylS5ee8FdFVbg90y8CmpSxvSmVXCNXRMYBh/h5nV1jTBWVjlbeuHEjEyc6l+ImT57M888/zzfffMNTTz3F1KlTvfuURCt/8MEH3HvvvQDeaOWkpCSSk5OJj4/3RiuvWrWKpKQk1q1bx/LlywG80crJycnldvgl0crLli1j48aN3jeT/Px8xo4dy7PPPktycjIrVqw4IVp5/fr1LFu2zHv36pw5cxg4cCBJSUnMmDGDBQsWeKOV161bxyuvvMKuXbt47733vNHKixYt8sYdn+zaa69l3bp1JCcn06NHDxYsWMCAAQMYMWIETz75JElJSSd0+PDLaOXU1FS+/PJLEhISGDRokDdHqLxo5dJK4q8feOAB+vbty5gxY7xpp6X3DwsLo0mTJhw44MypKYlWri63Z/qfA/eJyBhVLQIQkTDgPuALtwcTkcbAw8AQ4NZTvG4yMBmgQ4cObps3psa5PSP3F4tWDky0cmFhIVlZWfz73/9m3bp1XHfddezcubPCdkr2zcjIYMCAAcydO5e5c+dy9913lxuxUKJly5bs3bvX1TFOxW2nPxNYA6SJyBrPtguAhjjz9916BFigqhmnus9LVefjuVbQr1+/isOBjKmjSqKVP/nkE15++WXeeustnnnmGW+0cllOFa384YcfMmHCBH7/+9/TpElZf9w7/B2tXFxcTGRkZJmvK4lWPrmz/uijj1wdZ8KECSxfvpy4uDhee+01V5n2J0crt2vXjmuvvRYR4bzzziMkJITMzExX0crNmzcnKirqhGjmkmURS/Zv166dN3m0efPmQA1HK6vqNpxx/SVAM89jMRCnqltOtW8JEYkHLgGerlqpxpiTWbRyYKKVR44c6W0/NTWV/Px8WrRowYgRI1i6dCl5eXns2rWL7du3e1e6uvjii9mzZ493fL7kzWblypX07OlMghwxYgSvv/46AO+88w5Dhgyh5ATZV9HKqGqNPIDfAceBHzyPY0AOsP5U+5177rlqTG2SkpIS6BK8kpKS9JxzztG4uDiNi4vTjz76SFVVd+7cqUOHDtXY2Fjt0aOHPvTQQ6qqOn78eH377be9+zdo0EBVVV977TXt1auXxsfH6wUXXKA7d+5UVdUlS5Zo7969tVevXjpz5sxf7FeW0sdYsWKFxsfHa+/evXXixImam5urqqpr167VhIQEjY2N1YSEBD169KimpqZqnz59NDY2VmfOnOk9Rn5+vg4ePFhjY2N17ty5WlRUpH/605+8dV100UV66NAhLS4u1mnTpmnXrl31kksu0WHDhp3wtZZ46aWXtGPHjtq/f3+94447dPz48aqqumbNGu3Ro4fGx8drWlraCfukp6frwIEDvZ/n5eXpDTfcoL169dJzzjlHV65c6X1u9uzZevbZZ2vXrl29P4+ioiLt0KGDZmdnn9Benz59dMiQIbp7925VVc3JydHRo0dr586dtX///rpjxw5vu9OmTdP333+/zO95Wf8ngUQto091Fa1cQkTaAB2AEy6Lq2qF4/oiEgU0LrXpbqAjMEVV95e3n0Urm9rGopXrpupEK2/atImFCxd6F0GvrLy8PAYNGsSaNWu8M4hK83m0sqezX4Izfq84M3ZKv1tUOLinqtk4aZ0lbR4Dck/V4RtjTG1RnWjl3r17V7nDB/juu++YM2dOmR1+Zblt4RmcaZs9cebXXw60wpmJM6MqB1bVWVXZzxhjAiGQ0cpdunSp0ptNWdx2+oOAK1V1q4gosF9VvxKRPJwZOZ/6pBpjjDF+5fbmrPpApufjLJx1cgFScGb1GGOMCQJuO/2tQHfPx0nA7SJyFjAN2FPuXsYYY2oVt8M7zwJnej5+GPgYuB7IA8b7oS5jjDF+4PbmrMWq+prn4/U4Uy37Ax1U9W2/VWeMCTp1IVp51qxZtG3blvj4eOLj40+4G9hNtPKyZcuIjY2lV69e3iRNqF3Ryl4i0gpnquV6Vc2scAdjzGmtLkYrA8yYMYOkpCSSkpK44oorAHfRygcOHOCPf/wjK1euZPPmzfzwww+sXOnkT9aaaGURCQceBabgXNTtCuwUkceB3ar6UrUrMSYIfflWKpnfH/Npmy3aN2TgdV1dvdailQMTrVweN9HKO3fupEuXLt4gu0suuYR3332Xiy++uFZFKz8IDAd+izOOX2ItMKFaFRhjqsyilQMTrQzOm1JsbCw333wzBw8eBNxFK8fExLBt2zbS09MpLCxk+fLl3pC22hStfD1ws6p+LiLFpbZvwjnrN6ZOcntG7i8WrRyYaOUpU6bwwAMPICI88MAD/OEPf2DhwoUVtgNOMuq8efMYO3YsISEhDBgwgB07dlS4X01HK7cBdpezf/XvCzbGVIlFK/+sJqOVW7Vq5f140qRJXHXVVQCuopUBhg8fzvDhwwGYP3++93tZa6KVgc2UnZt/HfBNtaswxlSJRSsHJlp537593o/fe+89b+Sxm2hlgJ9++gmAgwcP8tJLL3Hrrbd69/d3tLLbs/SHgDdFpD1OuNoYEekO/Aa4stpVGGOqZM+ePUycOJHiYmfU9bHHHgNg8eLFTJkyhdmzZ1NQUMC4ceOIi4srt53Vq1fz5JNPEh4eTsOGDVm0aBGtW7dmzpw5DB482Hsh9+qrr3ZdW2RkJK+++ipjxozxXsi9/fbbqVevHsNUgOoAACAASURBVMuWLePOO+8kJyeH+vXrs2LFCqZOncqoUaNYtGgRl19+OQ0aNAAgNjaW0NBQ4uLimDBhAtOnTyc9PZ2+ffuiqkRHR7N8+XKuueYaVq1aRc+ePenQocMv1qUt8cgjj5CQkEB0dDQJCQnejn7cuHFMmjSJ5557jnfeeeeEcf1hw4Zx4403ej+fOXMmSUlJiAgdO3bkr3/9KwC9evXiuuuuo2fPnoSFhfHiiy8SGhpKcXExaWlpNGvWDIDp06d734j//Oc/e4fAbrnlFm688UZiYmJo1qwZS5cu9R7zs88+48orfdDdlpW3XNYDGIqzbOIxnLTMNcBlbvev6sPy9E1tU5vy9E3NGTlypKamplZp340bN+qMGTOqfOzc3FxNSEjQgoKCMp+vTJ6+6/F4Vf0EKPtOA2OMOc3VtWhlY0wp6pk7beqO2hqtrJVYCAuqcEeuMXVdZGQkBw4cqPQvmzG+pqocOHCg3JlOZbEzfWMqqV27dmRkZLB/vy36ZgIvMjKSdu3auX69dfrGVFJ4eDidOnUKdBnGVIkN7xhjTB3i+kxfRM4BBuOsmnXCm4WqVi8ByBhjTI1wm7I5E5iDE8XwI1D6CpZdzTLGmCDh9kx/BjBFVf/qz2KMMcb4l9sx/RBgpT8LMcYY439uO/15wER/FmKMMcb/KhO49pGIfIuToV9Q+klVvdnXhRljjPE9t53+o8BlwHqgKXbx1hhjgpLbTn8q8BtVXebPYowxxviX2zH9HOBbfxZijDHG/9x2+k8DvxOLFTTGmKDmdnhnIM5yiVeKSAq/vJA7wk0jIvImcDHQAPgBeEJV/8d9ucYYY6rDbaefCfyvD473GHCLquZ5lltcLSLfqqqts2uMMTXAVaevqj6Zo6+qm0t/6nl0xhZXN8aYGlGplE0ROVtErhKRK0Xk7KocUEReEpFsYCuwD/iojNdMFpFEEUm0zHJjjPEdV52+iDQWkbeBNGA58Hdgu4i8JSKNKnNAVZ0KNMK5TvC/QF4Zr5mvqv1UtV90dHRlmjfGGHMKbs/0nwVicaKV63seF3u2PVPZg6pqkaquAdoBUyq7vzHGmKpx2+mPAG5V1c9VtcDzWA1MBkZW4/hhOGP6xhhjaoDbTr8+cKCM7VmAqxV5RaSliIwTkYYiEioiQ4HrsfROY4ypMW47/a+AR0QkqmSDiDTACWL72mUbijOUkwEcBJ4Cfqeq77sv1xhjTHW4naf/e+BjYI+IbPBs6wNkA0PdNKCq+4FBla7QGGOMz7idp79RRLoANwDdPZvfABarao6/ijPGGONbFXb6IhIOfA9crKqv+L8kY4wx/lLhmL6qFuBk7ViGvjHGBDm3F3KfB/4kIm6vARhjjKmFKpOyOQjnQu4m4HjpJ92mbBpjjAmsyqRsvuvPQowxxvhfuZ2+iNwELFPVPF+lbBpjjAmsU43pvwo0BhCRIhFpWTMlGWOM8ZdTdfr7gV95PhZs9o4xxgS9U43pvwwsF5GSxU5+KG+JXFUN9UNtxhhjfKzcTl9VZ3ky9Lvg5N5PAg7VVGHGGGN875SzdzzLG24WkYeAv6lqds2UZYwxxh/cZu885O9CjDHG+F+l1sg1xhgT3KzTN8aYOsQ6fWOMqUOs0zfGmDrEdWqmJ2HzPKADUK/0c6q6yMd1GWOM8QNXnb6IdAf+D+iEc3dukWffAiAPsE7fGGOCgNvhnWeAb4AmOOvi9gD6AUnAKP+UZowxxtfcDu/0Bwap6nERKQbCVHW9iMzEWWAl1m8VGmOM8Rm3Z/qCc4YPThBbW8/HGUCMr4syxhjjH27P9DcBccBOYC1wj4gU4eTxpPmpNmOMMT7mttN/FGjg+fh+4EPgM5wVtcb6oS5jjDF+4DZ755NSH+8EeohIM+CgqlrOvjHGBAlXY/oislBEGpXepqpZQJSILPRLZcYYY3zO7YXc8UD9MrbXB27yXTnGGGP86ZTDO54hHPE8mopIYamnQ4ErgR/9V54xxhhfqmhMPxNnqUQFUsp4XoEHfV2UMcYY/6io0x+Mc5a/CufO26xSz+UDu1V1r59qM8YY42MVLZf4OYCIdAK+V9Xiqh5IRCKAl4BLgGbADuBPqvqPqrZpjDGmctxO2dwNICJtKDtl8wuXx/oeGAR8B1wBvCUifVQ1vRI1G2OMqSK3KZttgCXAhTjj+OL5t0RoRW2o6nFgVqlNH4jILuBcIN1ducYYY6qjMimbRUBPnAyegcAYYAtweVUOLCKtgK7A5jKemywiiSKSuH///qo0b4wxpgxuYxgGAVeq6lYRUWC/qn4lInnAI8CnlTmoiIQDi4HXVXXryc+r6nxgPkC/fv3sjl9jjPERt2f69XGmb4Izg6el5+MUKhmrLCIhwBs4s3/uqMy+xhhjqsdtp78V6O75OAm4XUTOAqYBe9weTEQEWAC0AkapakElajXGGFNNbod3ngXO9Hz8MPAxcD3OUonjK3G8eTirbl2iqjmV2M8YY4wPuJ2yubjUx+tFpCPOmf93qppZ3n6lef4yuA3njeIH56QfgNtKt2+MMcZ/3J7pn0BVs4H1ldxnN85UT2OMMQFS4Zi+iNQXkQdFZIOIHBORoyKSLCL3i0hZyZvGGGNqqYpSNsNwcnf64ozjf4hztt4T+DMwTEQGqWph+a0YY4ypLSoa3pmMs/B5X1U94SYqEemNs2TiJJwLtMYYY2q5ioZ3RgOPntzhA6jqJuAxnDtzjTHGBIGKOv1eOMM75VkB9PZdOcYYY/ypok6/KXCq8Jv9wBm+K8cYY4w/VdTphwKnukhbjIuETWOMMbVDRRdyBXjTE6xWlggf12OMMcaPKur0X3fRxiJfFGKMMcb/KloucWJNFWKMMcb/3KZsGmOMOQ1Yp2+MMXWIdfrGGFOHWKdvjDF1iHX6xhhTh5Q7e0dELnTbiKp+4ZtyjDHG+NOppmyuBpSfFz5Rz78nfw52V64xxgSFUw3vRAMtPf9eBWwDbsKJWo7xfLwVGOHnGo0xxvhIuWf6qnqg5GMReQSYrqqflnrJThH5CXgCZ3EVY4wxtZzbC7k9gYwytu/BWSDdGGNMEHDb6W8GHiy9Jq7n4z97njPGGBMEKgpcKzEF+ADYIyIbPNv6AEXAlf4ozBhjjO+56vRVdZ2InA3cwM/DOYuBJap63F/FGWOM8S23Z/p4Ovf5fqzFGGOMn7m+I1dEhonIByKSIiLtPdtuFZGL/VeeMcYYX3LV6YvIDcBbwHagExDueSoUmOmf0owxxvia2zP9mcAkVZ3BiWvm/huI93lVxhhj/MJtp98F+FcZ248BjX1XjjHGGH9y2+nvBbqWsf1CYIfvyjHGGONPbmfvzAeeE5FbPZ+3F5GBOBEMs9weTETuACbgzPH/m6pOcF1pgHy8aR/TlyZRpFrxi09zoY2+JbzVOyDB8724550C4ncWB7oM13IjmrG+7z0UhUYGuhRTC7WP28LwadOr1YbbefpPiEgT4FMgEvgMyAOeUtUXK3G8vcBsYChQv4LX1gr/l7yPyPBQLuvZKtClBNw3uUs4VBxFtJwX6FJcaZCdT9+0VaSe1ZSMVo0CXY47YX0JCW9Icf46Trx8VvdEai71NYdciQh0KbWCiHBm1wHVbqcy8/TvE5FHcXJ4QoAUVT1WmYOp6v8CiEg/oF1l9g2E4mJlTVom/c5qyk2/6hjocgKqsLiAzz7fRt+Wv2L4WeMCXY4rDb9eSwiraDL8Wuqf1T7Q5biyL7ExeYeLaHdBTKBLCbhf7XmKyIIsvoy+K9Cl1Arte/ck9teDq92O2ymbC0Wkkapmq2qiqq5V1WMi0kBEFla7il8eb7KIJIpI4v79+33dvGsp+45wOKeA+PZnBKyG2iLtcAp5xbl0btwj0KW4FpWcQnFkBPnt2gS6FFe0GHIyw4k8o26f4QOEFOfTPDeVzIjOgS7ltOP2Qu54yh6OqY+Tq+9TqjpfVfupar/o6GhfN+/amrRMAOLaWae/6WAigtCpUbdAl+Ja1IbN5HbuCKHBscZP3uEwtCiEiDMKAl1KwDXLTSNUC8msZ3/x+Noph3dEpBnOSlkCNBWR0qcgoThhaz/6r7zAWrM9k7OaR9G0Qb1AlxJwG7PW0bZBR+qHNQh0Ka6E//AT9X7cz7Ff9Qt0Ka5lZ9YDlIgmdqYfnZ1CMSFk1esY6FJOOxWN6WfiLIuoQEoZzyvwoK+Lqg1yC4pYl57F0F5nBrqUgMsuPM6OwykMbH15oEtxLWqj8981p8vZAa7EvZzMcMIbFhEaHjyzo/wlOieFg/U6UBRiF3F9raJOfzDOWf4qYBSQVeq5fGC3qu51ezARCfMcMxQIFZFIoFBVa92pzfrdB8krLLbxfGDLwW8pppjOjXsGuhTXopJTKGzSmMLoFoEuxZXiQsg9GE6jtnmBLiXgwouOcUbeblIbDgl0KaelU3b6qvo5gIh0Ar5TrfZk9fs58S+D3wIPUYm5/jVlTVomoSFCrzZ2w/GmrETCQ+rRvmGQXFQrLiZq42ZyusaASKCrcSUnqx6o2Hg+0CJnK4JyIMLG8/3B7YXc4ThZ+icQkd+KyFS3B1PVWaoqJz1mud2/Jq1Jy6Rbq0ZE1XM9q/W0telgImc17EJ4SHjFL64FItK/J/TocXKDbGgHUSIa1bo/emtcdHYKBRLBofBaP6s7KLnt9H8HfF/G9nRghs+qqSUOZxewMeOwDe0AB/My2XM8Pbimam5wVvAMrk6/HhGNC5HgmGjkV9E5KWTV64TaN8Mv3Hb67YDdZWzPIAhusqqsf+3MRIE46/TZlJUIEGTj+ZvJP7MlRY2D4y7cwjwh/2iYDe0A9QsyaVjwE5k2tOM3bjv9Hyg7Qrkvzgyf08qatEzqh4fStWXDQJcScJuyEokKa8iZUcHx3i75+dTfkkpuTKdAl+JaTqYzJdhuynLO8gEy6wXJ9aMg5HbAeglO4NpxYLVn22DgGZy1ck8ra7Zn0rttY8JCXS8sdlpSVTZlJXJ2o+6ESHB8LyK37SAkvyDIhnbCCQkrJrxhUaBLCbjo7BRyQxpxLKxloEs5bbn9TX4Q+Ar4BMj2PP4BfA084J/SAiPjYDbpB7JtPB/Ym72bg/mZdG4SREM7GzajISHknt0x0KW4ourclBXRpDBYJhr5jxYTnb3FGdqp898M/3GbslkAXC8if+bnYZ4kVd3ut8oC5Ou0A4BFL0Dp8fwguoibvJm8Dm3RyOC4qacgO5Si3FAi2uYGupSAa5yfQUTxURva8bNKzUdU1e0icgTYr6rBE1JeCWvSMmkaFU6HZlGBLiXgNmUl0iwimmYRgcs/qoyQY8eJ3JHO4UsuDHQpruVkOtNgbTzfGdoBLGTNz9ymbIaLyBMichTYA3T0bH+8MvP0a7uSKOW49mcgdfzPy6LiQjYfXM/ZwXSWv2krohpk4/n1CI0oIjTytDyHqpTo7BSOhrUkL7RJoEs5rVVmTH84zh20pe8TX4uzEtZpYduPR8k6nk+8De2w4+hWcouyiQmmqZobNlMcUY+8DsEx00jVOdOPOMPG80O0wIlStqEdv3M7vHM9cLOqfi4ipU9JNlH22rlB6auSKGW7iMumrCCMUk7eTO7ZZwVVlHJxYQiRNj+fprk7CdN8m59fA9ye6beh7JuzSgLUTgtrtmfSrml9WjQMjouA/rQpax2tozrQIDw4bnAKyzxAvX0/khsTTEM7zni+RSlDdPZmFCGrXvDcXxGs3Hb6m4Gyro5dB3zju3ICJ7+wmP/syrJZO0BuUQ7bD28Orlk7G5yLgME2nh8eVUhoPYtSjs7ewqHwdhSG2ILw/ub2LP0h4E0RaY8TizxGRLoDv8FZSCXoffvdQXIKimxoB9h2KJkiLQy6Tr+oYQMKzgyOm3qKi5wo5QZnWpRyWFE2Z+TtYkfDQYEupU5wdaavqv+Hc1Z/GVCMc2G3CzBcVVf4r7ya81VaJiECfdrazIFNWYmESThnNeoS6FLcUXXG87ucHTQ39eQeDEeLLUoZoEXuNkIotou4NcT1eLyqfoJzR+5paU1aJl1aNqJhxGlziaLKNmYl0qFhZ8JDgmOZyHrf7SHs8BEOBdnQDmJLI4IzVbNQwjlUr0OgS6kTgiNQxc+O5BaQnHHYhnaAw/kH+e5YWnClapZEKQdRyFr2/nDqNSokJDgmGvlVdLYTpVwsdsJVE8r9LntuxHJ1hUlVg3p5qf/szKKoWIlvZ0M7m7Oc6/LBNp5fEN2coqbB8aZdlC/kHwmjcQeLXogsPEijgn1836isEF/jD6d6a72jxqoIsK/SMokIC6F766B+7/KJTQcTqR8aRZsGZwW6FHcKC4natJXjfWMDXYlrOQfCARvPh9LRCzY/v6aU2+mr6us1WUggfbl9P73aNCbcopTZeGAdnRoHUZTy9p2E5OWRE2Tj+RKq1GtkUcrROSnkhTTgaFirQJdSZwTHb7Yf/XA4lx37j9v8fODHnD0cyPsxqIZ2GiRvRkXI69wx0KW45kQpFwTLRCP/USU6O8WZtRMkJxmngzo/pl8SvWD5+UG6NOKGzeS3a0NxVP1Al+JKQXYIhdmhNLT5+TQq2Etk0WEONLShnZpU58f0v0rLpEn9cDq2aBDoUgJuU1YiZ9RrTvOI4LjBKSQ7h8jUnRy56NeBLsW1kqURbTy/1Hi+zc+vUXV6TF/ViVKObdeEkDr+t3axFrH54Dd0OyMuaGKl66dsQ4qLg2w8P5yQesWE1bco5RbZWzge2pycsKaBLqVOqdTEWBEZAvTEGfbZrKqr/VFUTUn76Rg/Hc3jun7tA11KwKUf3c7xwqNBNZ4flbyZ4vBw8s4Knijl7Mx6RDa18XzRQlrkbGVv/eCZdXW6cNXpi0hb4D3gXGCvZ3MbEUkErlHVveXuXIutsfF8r01Z6wDo3Lh7gCtxL2pDCnmdOkB4eKBLcSX/SCjFBSFE2CpZnJGbTrjmklnPxvNrmttL5s8BRUCMqrZX1fY42TtFnueC0ldpmbRuEkmrxpbstzErkTPrt6dheHDcoBaadYiI7/cEXaomQGQTG8+PzklBEQ5EBM/P73ThttO/FJimqrtKNqjqTuAuz3NBp6ComH/tPGBTNYH8ojy2HdoQXGf5G52LgME0np+dWY+wqEJCIyxKOTo7hcPhbSgIsbWoa1plJseW9T81aP/3bsg4xPG8IhvaAbYd3kChFgTdVM2iBlEUtA6Om3q0CHKzwom0gDVCi3NplptmQzsB4rbTXwk878nTB0BEOgDPeJ4LOmu2H0CwKGVwpmqGSmiQRSmnOAFrIcFxU0/uoZIoZev0m+ekOlHKETZVMxDc/sbcBTQAdorIbhHZDezwbLvLX8X505q0/XRu2ZDG9YPjIqA/bcpKpH3DzkSEBse1jfA9+wjPOhhU4/nZmeGAEmHj+URnp1BEGAfrBUm+02nG7SIq3wN9gSuApzyPYaraV1Uz3B5MRJqJyHsictzz5vGbKlVdTcfzCvn2u0M2ng8cLThM+tHUoJqq2SBIl0as16iIEEsPJjpnM1n1OlIsdsIVCJVZREWBTz2PqnoRyAdaAfHAhyKSrKqbq9Fmpa3dlUVhsdp4PpCStR5Fg248v6B5UwqbBcdNPUUFQt6hMBq1tyjliMLDNMnfw5ZGQwNdSp0lTl9ezpMiw4B5QKyqHjnpuSZAMjBJVSt8IxCRBsBBoLeqpnq2vQHsUdV7y9vvrOgueu/IZ918LaYaJMguyUsISGhwFK2EUKgRDG/+EG0itga6nIASLSZU81l31h85GmkrZVVGu27d6dr3PNevF5FvVLXfL7ZX0Ol/CHykqi+W8/wU4CpVrXBxdBE5B/hKVaNKbbsbGKSqw0967WRgsufT3sCmitqvghZAZhC168+2g61df7YdbO36s21/1mz87yxVjT55Y0XDO7HA70/x/CrgPpcFNASOnLTtMNDo5Beq6nxgPoCIJJb1blVdwdauP9sOtnb92XawtevPtv1Zswmcii7kRgOnSoZSoLnLYx0DTo5gbgwcdbm/McaYaqqo08/AOdsvTyywx+WxUoEwESk9GTwOqNGLuMYYU5dV1Ol/CDwiIr9YoUJEooCHPa+pkKoeB/4XeFhEGojIr4GrgTcq2HW+m/arINja9WfbwdauP9sOtnb92bY/azYBUtGF3JbAtzhDPC8AJVMPeuAssiJAX1X90dXBRJoBC3Hyeg4A96rqkipXb4wxplJO2ekDiMhZONM2h+J08uCM5X/CSSFsxhhjarcKO33vC0WaAjE4Hf92VT3oz8KMMcb4nutO3xhjTPCrtRGF/sjpEZEIEVngae+oiCR57jr2GRHpIiK5IvKmj9sdJyJbPN+PHSIy0AdtdhSRj0TkoIj8ICIviEil02FE5A4RSRSRPBF57aTnLhaRrSKSLSKfeYYLq9WuiJwvIp+KSJaI7BeRt0Wkta9qLvWaP4uIisglvmhXRKJE5CURyRSRwyLyhY/avc7zf+OoiKSIyMhKtHvK34nq/PxM7VRrO31OzOm5AZgnIr2q2WYY8D0wCGgC3A+8JSIdq9luaS8C63zYHiJyKfA4MBHnZrYLgZ0+aPol4CegNU4W0iBgahXa2QvMxrlI7yUiLXBmbD0ANAMSgWXVbRdoijOzpCNwFs69Hq/6ouYSItIZGAPs82G783G+Dz08/86obrviLGX6Js5NlI2BPwJLPJMw3Cj3d8IHPz9TG6lqrXvgRDbnA11LbXsDmOOHY20ARvmorXHAW8As4E0f1vg1cIsfvvYtwBWlPn8S+Gs12psNvFbq88nA1yf9XHOA7tVpt4zn+wJHfVFzqe0f46TKpgOX+OB70R3njvTG1fyZndxuAvDTSa/ZD/yqGsfYAIzy1c/PHrXrUVvP9LsCheoJZvNIBqp7pn8CEWnlOVa1bxATkcY49y2cKraiKu2GAv2AaBFJE5EMzzDML+6dqIJngHGeYYe2wDCczs5XeuH83ADvvRo78PHPEecvH5/d5CciY4A8Vf3IV20C5wG7gYc8wzsbRWSUD9pNBLaIyAgRCfUM7eThdNyVdtLvRE39/EwNqq2dvuucnqoSkXBgMfC6qvoi+vARYIFWYn0Bl1oB4cBoYCDOMMw5OH+GV9cXOL/AR3Duvk4Elvug3RINcX5upfn65xgL/BlnWMMX7TUC/gJM90V7pbTDCQ88DLTBuc/ldRGp1kIGqloELAKW4HT2S4DbPB10pZTxO+H3n5+pebW10/drTo+IhOAMF+Xj/PJVt7144BLg6eq2VYYcz7/Pq+o+Vc0E5uIMPVSZ53vwMc6YbQOcRMWmONcOfMXfP8cY4B/AdFX90hdt4gzNvaGq6T5qr0QOUADMVtV8Vf0c+Ay4rDqNei4yPwFcBNTDGZv/H8//ycq0U9bvhOVlnYZqa6fvt5weERFgAc4Z9ChV9cX6dRfhXFT8TkR+AO4GRonI+uo2rM79EBmcuAi9L+bZNgM6AC+oap6qHsC5GFqtN5OTbMb5uQHeNRU645uf41nACuARVa0oyqMyLgbu8sxm+gFoj3Nh855qtlvWcIsvfo7xwBeqmqiqxaq6DvgPzkmIK6f4nfDbz88ETq3s9LXqOT1uzMOZPTFcVXMqerFL83F+GeI9j5dxMol8tTzQq8CdItJSnJvkZgAfVKdBz18Mu4ApIhImImcA46nCWLBn/0ggFAgVkUjP1M/3gN4iMsrz/J+BDW6H08pr13P9YRXOG9bLla23gpovxhmGKflZ7gVuw5mVVZ12vwC+A/7kec2vgcE4d7ZXp911wMCSM3tx1q0YSOV+juX9TlTr52dqqUBfSS7vgXMmuhw4jvPL8hsftHkWztlVLs6friWPG3xc+yx8O3snHGd65SHgB+A5INIH7cYDq3FWNMvEmXnUqopfr570mOV57hKczKYcz7E6Vrdd4EHPx6V/hsd8VfNJr0unErN3Kvhe9AL+5fk/nQJc46N27wDScIZddgJ/8NXvRHV+fvaonQ+7I9cYY+qQWjm8Y4wxxj+s0zfGmDrEOn1jjKlDrNM3xpg6xDp9Y4ypQ6zTN8aYOsQ6fWOMqUOs0zd1johEexYzSfcsSvKjiKz0rFuAZ/vdga7TGH+o9CpJxpwG3gWigFtw7mRtiRNU1jyQRRlTE+yOXFOneDKGDgKXquqKMp5fjfMG4KWq4nluAPAY0N/TxvvAPap6pNS+W3Eijm/y7P4/ntcUe15zLU6kQhecaIONwHWq+qMPv0xjymXDO6auKcmWGeEJETvZtTippg/jLCPZGkBE+gD/xOno4zyvi+eXyyLegPN79SucoLbJwO88bZwJLAVexwk4uxDfhAga45qd6Zs6x7Ni1Ss4QzzfAl8Bb6vqfzzPp+MkeD5Vap9FQIGq3lJqW7xn/1aq+pPnTL8N0E09v1gicj9wu6q2E5G+wDc4oWW7/f+VGvNLdqZv6hxVfRencx6OswjLAODfIvJfp9jtXOC3InKs5IHzZgFOrHaJf+uJZ1L/Atp6ltNMxlkDYJOIvCsiU0Qk2kdfljGuWKdv6iRVzVXVT1X1YVUdgLOIyCwRqVfOLiE44/PxpR5xOGPzSS6PWYSzUtZlOHn3twDbRSTulDsa40M2e8cYRwrO70MkzpKBoSc9vx7opappFbSTICJS6mz/fGBvycVez/Z/Af8SkYdxVqEaS6kFyI3xJzvTN3WKiDQXkVUi8lsRiRWRTiIyBpgJrPR0zuk4q1G1FZEWnl0fB84TkZdF5BwRiRGRq0Tkrycdog3wjIh0E5HROAu2P+059vkicr+I9BeRDsAInOUYU/z/lRvjsDN9U9ccA/4NTAdigAhgD7AEmO15zZ+BvwI7PM+Lqm4QkQs9r/kc5y+BnThLCpa22PPcf3BWpFqAp9MHDgO/Bu4EzgC+x1nj902ff5XGlMNm7xjjI57ZO5tU9Y5A12JMeWx4xxhj6hDr9I0xpg6x4R1jjKlD7EzfGGPqEOv0jTGmDrFOtp72xAAAABhJREFU3xhj6hDr9I0xpg6xTt8YY+qQ/wcXqqTckvDUvQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "number of steps taken by the agent:  30\n"
          ]
        }
      ]
    }
  ]
}