12. Non supervised learning II.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "12. Non supervised learning II.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.2"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DataScienceUB/DeepLearningMaster2019/blob/master/12.%20Non%20supervised%20learning%20II.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZUPZTOPiiiTg",
        "colab_type": "text"
      },
      "source": [
        "## Deep Learning and Probabilistic Models\n",
        "\n",
        "The Bayes rule for a given input is:\n",
        "\n",
        "$$ \n",
        "p(y \\vert x) = \\frac{p(x \\vert y) p(y)}{p(x)} = \\frac{p(x,y)}{p(x)}\n",
        "$$\n",
        "\n",
        "Discriminative classifiers jump directly to estimating $p(y \\vert x)$ without modeling its component parts $p(x,y)$ and $p(x)$.\n",
        "\n",
        "The objective of *generative models* is to learn a model $p(x)$. This model can be used for several purposes, but we are specially interested in models we can sample from. \n",
        "\n",
        "To this end we have several possible strategies when considering deep learning tools:\n",
        "\n",
        "+ **Fully-observed models.** To model observed data directly, without introducing any new unobserved local variables.\n",
        "\n",
        "+ **Transformation models.** Model data as a transformation of an unobserved noise source using a *parameterised* function.\n",
        "\n",
        "+ **Latent-variable models.** Introduce unobserved random variables for every observed data point to explain hidden causes.\n",
        "\n",
        "### Fully-observed models.\n",
        "\n",
        "The most succesful models are PixelCNN (https://arxiv.org/pdf/1606.05328v2.pdf) and WaveNet (https://arxiv.org/pdf/1609.03499.pdf)).\n",
        "\n",
        "Both cases are based on autoregressive models that models the conditional distribution of every individual data feature (pixels,etc.) given previous features:\n",
        "\n",
        "$$\n",
        "p(\\mathbf x) = \\prod_{i=1}^{N} p(x_i \\vert x_1, \\dots ,x_{i-1})\n",
        "$$\n",
        "\n",
        "All conditional probabilities are described by deep networks.\n",
        "\n",
        "Its advantages are:\n",
        "\n",
        "+ Parameter learning is simple: Log-likelihood is directly computable, no approximation needed.\n",
        "+ Easy to scale-up to large models, many optimisation tools available.\n",
        "\n",
        "But generation can be slow: we must iterate through all elements sequentially!\n",
        "\n",
        "### Transformation Models\n",
        "\n",
        "Transformation models model $p(x,z)$ instead of $p(x)$, where $z$ is an unobserved noise source. \n",
        "\n",
        "In the simplest case, they transform an unobserved noise source $z \\sim \\mathcal{N}(0,I)$ using a parameterised function. The transformation function is parameterised by a linear or deep network $f_\\lambda(z)$. \n",
        "\n",
        "For example, for producing $x \\sim \\mathcal{N}(\\mu, \\Sigma)$ we can sample $z \\sim \\mathcal{N}(0,I)$ and then apply $x = \\mu + \\Sigma z$.\n",
        "\n",
        "The main drawbacks of this strategy are the difficulty to extend to generic data types and the difficulty to account for noise in observed data.\n",
        "\n",
        "### Latent variable models \n",
        "\n",
        "Latent variable models solve this problems by introducing unobserved\n",
        "local random variables that represents *hidden causes* and which can be easily sampled. \n",
        "\n",
        "Variational autoencoders are a good example of this strategy that propose a specific probability model of data $x$ and latent variables $z$. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5oBAmg2n70AQ",
        "colab_type": "text"
      },
      "source": [
        "## Autoregressive Models (fully observed models).\n",
        "\n",
        "By the chain rule of probability, we can factorize the joint distribution over the $n$-dimensions as\n",
        "\n",
        "$$\n",
        "p(x)=∏_{i=1}^np(x_i|x_1,x_2,…,x_{i−1})=∏_{i=1}^n p(x_i|x_{<i})\n",
        "$$\n",
        "\n",
        "\n",
        "where $x_{<i}=[x_1,x_2,…,x_{i−1}]$ denotes the vector of random variables with index less than $i$. In most of the cases, variables can be sorted in an arbitrary way.\n",
        "\n",
        "The term autoregressive originates from the literature on time-series models where observations from the previous time-steps are used to predict the value at the current time step. In this case, variable must be sorted by time.\n",
        "\n",
        "In an autoregressive generative model, the conditionals are specified as **parameterized functions with a fixed number of parameters**. \n",
        "\n",
        "### Naive approximation\n",
        "\n",
        "In the simplest case (binay data), we can specify the function that represents conditionals as a linear combination of the input elements followed by a sigmoid non-linearity (to restrict the output to lie between 0 and 1). \n",
        "\n",
        "This gives us the formulation of a **fully-visible sigmoid belief network** (FVSBN):\n",
        "\n",
        "$$\n",
        "f_i(x_1,x_2,…,x_{i−1})=σ(α^{(i)}_0+α^{(i)}_1x_1+…+α^{(i)}_{i−1}x_{i−1})\n",
        "$$\n",
        "\n",
        "<center>\n",
        "<img src=\"https://github.com/DataScienceUB/DeepLearningMaster2019/raw/master/images/NADE.png\" alt=\"\" style=\"width: 400px;\"/> \n",
        "</center>\n",
        "\n",
        "The conditional for variable $i$ requires $i$ parameters, and hence the total number of parameters in the model is given by $\\sum_{i=1}^{n} i = O(n^2)$. This is a large number of parameters!\n",
        "\n",
        "A natural way to increase the expressiveness of an autoregressive generative model is to use more flexible parameterizations \n",
        "e.g., multi-layer perceptrons (MLP). \n",
        "\n",
        "For example, consider the case of a neural network with 1 hidden layer. The  function for variable $i$ can be expressed as:\n",
        "$$\n",
        "\\mathbf {h}_i=σ(A_i \\mathbf {x}_{<i}+c_i)\n",
        "$$\n",
        "\n",
        "$$\n",
        "f_i(x_1,x_2,…,x_{i−1})=σ( V_{i,:}\\mathbf {h}_i+b_i)\n",
        "$$\n",
        "\n",
        "where $\\mathbf {h}_i \\in \\Re^d$ denotes the hidden layer activations for the MLP.\n",
        "\n",
        "The total number of parameters in this model is dominated by the matrices $A_i$ and given by $O(n^2d)$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ob78qr3mjAN-",
        "colab_type": "text"
      },
      "source": [
        "### Neural Autoregressive Density Estimator\n",
        "\n",
        "The *Neural Autoregressive Density Estimator* (NADE) provides an alternate MLP-based parameterization that is more statistically and computationally efficient than the vanilla approach. In NADE, parameters are shared across the functions used for evaluating the conditionals. In particular, the hidden layer activations are specified as\n",
        "\n",
        "$$\n",
        "\\mathbf{h}_i=σ(W_{:,{<i}} \\mathbf{x}_{<i}+c)\n",
        "$$\n",
        "\n",
        "$$\n",
        "f_i(x_1,x_2,…,x_{i−1})=σ(V_{i,:}\\mathbf{h}_i+b_i)\n",
        "$$\n",
        "\n",
        "where $W, V, b, c$ are learnable parameters of the model. \n",
        "\n",
        "The weight matrix $W$ and the bias vector $c$ are shared across the conditionals: the total number of parameters gets reduced from $O(n^2d)$ to $O(nd)$.\n",
        "\n",
        "\n",
        "### PixelCNN\n",
        "\n",
        "Autoregressive models can be extended to image data. \n",
        "\n",
        "See, for example, **PixelCNN**. https://towardsdatascience.com/auto-regressive-generative-models-pixelrnn-pixelcnn-32d192911173\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dfshqkEviiTl",
        "colab_type": "text"
      },
      "source": [
        "## GAN: Generative Adversarial Networks\n",
        "\n",
        "GAN are an alternative to model $p(x)$.\n",
        "\n",
        "The basic idea of GANs is to set up a game between two players. \n",
        "\n",
        "One of them is\n",
        "called the **generator**. The generator creates samples that are intended to come\n",
        "from the same distribution as the training data. \n",
        "\n",
        "The other player is the **discriminator**.\n",
        "The discriminator examines samples to determine whether they are real or fake. \n",
        "\n",
        "The discriminator learns using traditional supervised learning\n",
        "techniques, dividing inputs into two classes (real or fake). The generator\n",
        "is trained to fool the discriminator.\n",
        "\n",
        "<center>\n",
        "<img src=\"https://cdn-images-1.medium.com/max/2600/0*0KQk5Mgi93zax4Hl.png\" alt=\"\" style=\"width: 500px;\"/> \n",
        "(Source: https://towardsdatascience.com/generative-adversarial-networks-gans-a-beginners-guide-5b38eceece24)\n",
        "</center>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nS9r0i-SiiTm",
        "colab_type": "text"
      },
      "source": [
        "The two players in the game are represented by two functions, each of which\n",
        "is differentiable both with respect to its inputs and with respect to its parameters.\n",
        "\n",
        "The discriminator is a function $D$ that takes $x_{real}$ and $x_{fake}$ as input and uses $\\theta^D$ as parameters. \n",
        "\n",
        "The generator is defined by a function $G$ that takes $z$ as input and uses $\\theta^G$ as parameters.\n",
        "\n",
        "Both players have cost functions that are defined in terms of both players’ parameters. \n",
        "\n",
        "+ The discriminator wishes to minimize $J^D = (\\theta^D, \\theta^G)$ and\n",
        "must do so while controlling only $\\theta^D$.\n",
        "\n",
        "+ The generator wishes to minimize $J^G = (\\theta^D, \\theta^G)$\n",
        "and must do so while controlling only $\\theta^G$. \n",
        "\n",
        "Because each\n",
        "player’s cost depends on the other player’s parameters, but each player cannot\n",
        "control the other player’s parameters, **this scenario is most straightforward to\n",
        "describe as a game rather than as an optimization problem**. \n",
        "\n",
        "> The solution to an\n",
        "optimization problem is a (local) minimum, a point in parameter space where all\n",
        "neighboring points have greater or equal cost. The solution to a game is a Nash\n",
        "equilibrium. In this context, a Nash equilibrium is a tuple $(\\theta^D, \\theta^G)$\n",
        "that is a local minimum of $J^D$ with respect to $\\theta^D$ and a local minimum of\n",
        "$J^G$ with respect to $\\theta^G$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FiVnVnXFiiTm",
        "colab_type": "text"
      },
      "source": [
        "### The generator\n",
        "\n",
        "The generator is simply a differentiable function $G$. When\n",
        "$z$ is sampled from some simple prior distribution, $G(z)$ yields a sample of $x$. Typically, a deep neural network is used to represent $G$.\n",
        "\n",
        "### The training process\n",
        "\n",
        "The training process consists of simultaneous SGD. \n",
        "On each step, two minibatches are sampled: a minibatch of $x$ values from the\n",
        "dataset and a minibatch of $z$ values drawn from the model’s prior over latent\n",
        "variables. Then two gradient steps are made simultaneously: one updating $\\theta^D$\n",
        "to reduce $J^D$ and one updating $\\theta^G$ to reduce $J^G$.\n",
        "\n",
        "In both cases, it is possible to use the gradient-based optimization algorithm of your choice. Adam is usually a good choice. \n",
        "\n",
        "<center>\n",
        "<img src=\"https://github.com/DataScienceUB/DeepLearningMaster2019/blob/master/images/t12.png?raw=1\" alt=\"\" style=\"width: 600px;\"/> \n",
        "</center>\n",
        "\n",
        "###  The discriminator’s cost\n",
        "\n",
        "The cost used for the discriminator is:\n",
        "\n",
        "$$\n",
        "J^D = - \\frac{1}{2} \\sum_x \\mathop{{}log} D(x) - \\frac{1}{2}\\sum_z \\mathop{{}log} (1 - D(G(z)))\n",
        "$$\n",
        "\n",
        "This is just the standard cross-entropy cost that is minimized when training\n",
        "a standard binary classifier with a sigmoid output. The only difference is that\n",
        "the classifier is trained on two minibatches of data; one coming from the dataset,\n",
        "where the label is 1 for all examples, and one coming from the generator, where\n",
        "the label is 0 for all examples.\n",
        "\n",
        "###  The generator’s cost\n",
        "\n",
        "The simplest version of the game is a zero-sum game, in which the sum of\n",
        "all player’s costs is always zero. In this version of the game:\n",
        "\n",
        "$$\n",
        "J^G = - J^D\n",
        "$$\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TlUyn7MeiiTn",
        "colab_type": "text"
      },
      "source": [
        "## Example\n",
        "\n",
        "Training a generative adversarial network to sample from a Gaussian distribution. \n",
        "(Adapted from: http://blog.evjang.com/2016/06/generative-adversarial-nets-in.html)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yLRvFeVKiiTn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns # for pretty plots\n",
        "from scipy.stats import norm\n",
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gfmN2y3UiiTt",
        "colab_type": "text"
      },
      "source": [
        "This will be our target distribution:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jSIDdJEuiiTu",
        "colab_type": "code",
        "outputId": "9b1d3684-7920-49b7-aa7a-45bc50b8b889",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        }
      },
      "source": [
        "# target distribution\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "mu,sigma=-1,1\n",
        "xs=np.linspace(-5,5,1000)\n",
        "plt.plot(xs, norm.pdf(xs,loc=mu,scale=sigma))\n",
        "r = norm.rvs(-1,1,size=200)\n",
        "\n",
        "ax = plt.axes()\n",
        "ax.xaxis.set_major_formatter(plt.NullFormatter())\n",
        "b = plt.xticks(r)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAADuCAYAAADV/NcIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VPd57/HPM6MVtIEkBEgChBBg\nsRiwjA0YvNs4jsGOndpO0tpNWtdJ3KQ3bVq3yctunTY3y71p0sa+MUncJE1d4jiOQxu8YwxYGEsY\nzA5aWCSxaEMSaB/Nc//QiA5jgUYw0pnleb9e89LMOb9z5hkJvjr6nd/5HVFVjDHGxAaX0wUYY4wZ\nPRb6xhgTQyz0jTEmhljoG2NMDLHQN8aYGGKhb4wxMcRC3xhjYoiFvjHGxBALfWOMiSFxThcQKCsr\nS6dNm+Z0GcYYE1G2b9/eqKrZQ7ULu9CfNm0a5eXlTpdhjDERRUSOBtPOuneMMSaGWOgbY0wMsdA3\nxpgYYqFvjDExJKjQF5GVInJQRCpF5PGLtLtXRFRESvyW/a1vu4MicnsoijbGGHNphhy9IyJu4Gng\nVqAWKBORdaq6L6BdKvBlYJvfsmLgAWAOMBl4U0Rmqmpf6D6CMcaYYAVzpL8YqFTValXtAdYCqwdp\n9w3g20CX37LVwFpV7VbVw0Clb3/GXDK725sxly6Ycfq5QI3f61rgGv8GIrIIyFfV34vIVwO2fS9g\n29zANxCRR4BHAKZMmRJc5Sam7KlrZc2makqrGmlu72FSejI3zs7m0esLyRs3xunyjIkYl31xloi4\ngO8BD1/qPlR1DbAGoKSkxA7jzDm9fV6+8+oBfrz5MGlJcdxaPJGJ6YkcbmznhfJaXiiv5et3XsEf\nXjsVEXG6XGPCXjChXwfk+73O8y0bkArMBTb6/tNNBNaJyKogtjXmgro9fXz+lx+w4UA9f3jtVL66\nchZpSfHn1h9v6eTrL+/hid/t5WhTB1+/8woLfmOGEEyffhlQJCIFIpJA/4nZdQMrVbVVVbNUdZqq\nTqO/O2eVqpb72j0gIokiUgAUAe+H/FOYqOP1Kn+xdicbDtTzj3fP5Rt3zz0v8AEmZyTzkz8q4eGl\n0/jplsN8/80Kh6o1JnIMeaSvqh4ReQx4DXADz6nqXhF5CihX1XUX2XaviLwA7AM8wBdt5I4JxjMb\nK3llz0m+9rEr+My1Uy/YzuUSnryrmLPdHn7wVgVzc9O5tThnFCs1JrJIuI2EKCkpUZtwLbZ9WNPC\nPc+8y11XTub79y8Iqsumq7eP+35UyrGmDt78yvVMSEsahUqNCR8isl1VS4ZqZ1fkmrDS7enjqy9+\nyITUJL5x99yg++iT4t3864OL6PJ4eXLd3hGu0pjIZaFvwspPNh/m0KmzfPMTH+3DH0pB1li+fHMR\nr+w5yRv7To1QhcZENgt9EzYaznTzzNuV3Fqcw02zL61f/pEV05kxIYX/vX4/nj5viCs0JvJZ6Juw\n8YO3DtHl8fL4HbMveR/xbhd/ffssqn3j+I0x57PQN2Gh9nQHa9+v4cHF+RRmp1zWvm4tzuGqqeP4\n/puH6Oq1wWLG+LPQN2HhJ5sPIwJfuGHGZe9LRPjL22ZSf6ab33xgR/vG+LPQN45rOtvN2rJj3L0g\nl8kZySHZ55LpmVyZl86PN1XT5w2vYcnGOMlC3zju3949QrfHy6M3FIZsnyLCn11fyJGmDl7bezJk\n+zUm0lnoG0d19fbxy21Hua0457L78gPdPmciBVljeXZTdUj3a0wks9A3jvr9rhO0dPTy0JJpId+3\n2yU8vHQaH9a0sLu2NeT7NyYSWegbR/1y21GmZ49lSWHmiOz/nkW5JMe7ef79oyOyf2MijYW+ccye\nulZ2HGvhM9eM3Fz4aUnx3HXlJH638zhnunpH5D2MiSQW+sYxz79/jKR4F/delTei7/Ppa6bS0dPH\nyzuPj+j7GBMJLPSNI7p6+/ivD49zx9xJpCcPb46d4Zqfl86cyWn8quzYiL6PMZHAQt84YsOBes50\nefjEoo/cMjnkRIRPLMpjT10blfVnRvz9jAlnFvrGES99UEtOWiJLC7NG5f3uunISLoHf7rC7dZrY\nFlToi8hKETkoIpUi8vgg6x8Vkd0islNEtohIsW/5NBHp9C3fKSI/CvUHMJGn6Ww3Gw82cPeCXNyu\n0bmn7YTUJJYXZfPyjuN47QpdE8OGDH0RcQNPA3cAxcCDA6Hu53lVnaeqC4DvAN/zW1elqgt8j0dD\nVbiJXP/14XE8XuWeUeja8XfPwlzqWjopP3p6VN/XmHASzJH+YqBSVatVtQdYC6z2b6CqbX4vxwJ2\nKGUu6OWdx7liUhqzJ6aN6vveNieHMQlufrvDJmEzsSuY0M8Favxe1/qWnUdEvigiVfQf6X/Jb1WB\niOwQkXdEZPlgbyAij4hIuYiUNzQ0DKN8E2nqWjrZWdPCx+dPGvX3HpMQx23FOazffZJeu8GKiVEh\nO5Grqk+raiHwN8DXfYtPAFNUdSHwFeB5EfnI4Z2qrlHVElUtyc7ODlVJJgy9uqd/8rM75k505P3v\nmDeJ1s5e3qtucuT9jXFaMKFfB+T7vc7zLbuQtcDdAKrarapNvufbgSpg5qWVaqLBq3tOMHtiKtND\nPLlasK6fmU1yvPvcLx9jYk0woV8GFIlIgYgkAA8A6/wbiEiR38s7gQrf8mzfiWBEZDpQBNiUhzHq\nVFsX5UdP87F5o9+1MyAp3s2Ns7N5be8pm2ffxKQhQ19VPcBjwGvAfuAFVd0rIk+JyCpfs8dEZK+I\n7KS/G+ch3/IVwC7f8heBR1W1OeSfwkSE1/aeRNW5rp0BK+dOovFsN9ttFI+JQXHBNFLV9cD6gGVP\n+D3/8gW2+w3wm8sp0ESP9btPMGNCCkU5qY7WcdPsCSTEuXhlzwkWF4x3tBZjRptdkWtGRePZbt4/\n3MzHHD7KB0hJjGNFURav7TmJqnXxmNhioW9GxYYD9XgVbpvjfOhDfxfP8dYudtnNVUyMsdA3o2LD\n/nompiUxZ/LoXpB1ITfPnoBL4K0D9U6XYsyostA3I67b08fmigZuumLCiN0sZbjGjU3gqqnj2HDg\nlNOlGDOqLPTNiNtW3Ux7Tx83z57gdCnnuXH2BPbUtXGytcvpUowZNRb6ZsRtOFBPUryLZTNGZxrl\nYN08OweAtw9aF4+JHRb6ZkSpKm8dOMWywiyS4t1Ol3OemTkp5GYks8H69U0MsdA3I6qy/iw1zZ3c\ndEV4de1A/x21bpo9gS0VjXT19jldjjGjwkLfjKiB0TEDXSnh5qYrJtDZ22cTsJmYYaFvRtRb+08x\nZ3IaE9OTnC5lUEumZ5Ic7+Zt6+IxMcJC34yY1s5eth89zY2zwq9rZ0BSvJtlM7J460C9XZ1rYoKF\nvhkxpZWNeBWunxXe90i4cXY2tac7qW5sd7oUY0achb4ZMZsqGklJjGNBfobTpVzUiqL+X0qbD9ld\n20z0s9A3I0JV2XSogaWFmcS7w/ufWf74MUzLHMPmikanSzFmxIX3/0YTsQ43tlPX0snymeHdtTNg\neVE2W6ub6PHYvXNNdLPQNyNik6+r5PqiSAn9LDp6+vjgmN1YxUS3oEJfRFaKyEERqRSRxwdZ/6iI\n7BaRnSKyRUSK/db9rW+7gyJyeyiLN+Frc0UjUzPHMCVzjNOlBGVJYSZxLjn3y8qYaDVk6Pvucfs0\ncAdQDDzoH+o+z6vqPFVdAHwH+J5v22L676k7B1gJPDNwz1wTvXo8XrZWN507QRoJUpPiWTRlnPXr\nm6gXzJH+YqBSVatVtQdYC6z2b6CqbX4vxwIDA55XA2tVtVtVDwOVvv2ZKLb96Gk6evpYXhReE6wN\nZXlRFnuOt9J0ttvpUowZMcGEfi5Q4/e61rfsPCLyRRGpov9I/0vD3PYRESkXkfKGBvvzOtJtqmgg\nziUsKcx0upRhWT4zG1V4t8qmZDDRK2QnclX1aVUtBP4G+Powt12jqiWqWpKdHTldAmZwmysaWDRl\nHKlJ8U6XMizzctNJT4638fomqgUT+nVAvt/rPN+yC1kL3H2J25oI13i2mz11bayYGVldOwBul3Dd\njCw2VzTalAwmagUT+mVAkYgUiEgC/Sdm1/k3EJEiv5d3AhW+5+uAB0QkUUQKgCLg/csv24SrUl/X\nyHURdBLX3/KiLE62dVFZf9bpUowZEXFDNVBVj4g8BrwGuIHnVHWviDwFlKvqOuAxEbkF6AVOAw/5\ntt0rIi8A+wAP8EVVtYnLo9jWqkZSk+KYl5vudCmX5DrfyedNFY0U5aQ6XI0xoTdk6AOo6npgfcCy\nJ/yef/ki2/4T8E+XWqCJLKVVTVw7PRO3KzxugD5ceePGMDVzDFurmvjcdQVOl2NMyNkVuSZkak93\ncLSpg6URNmon0NLCLLZVN+HpsykZTPSx0Dchs9XXnx9pQzUDLS3M5Ey3hz3H24ZubEyEsdA3IbO1\nqonMsQnMnBDZfeEDv7RKq+zqXBN9LPRNSKhqf39+YSauCO3PH5CVksjsiamUVtpFWib6WOibkDjc\n2M7Jtq6I788fsKQwk7IjzXR7bLCZiS4W+iYkBsbnLy2MvIuyBrO0MItuj5cdx1qcLsWYkLLQNyGx\ntaqJSelJTIuQqZSHcs308bik/z6/xkQTC31z2bxe5b3qJpYUZiIS2f35A9KS4pmXl3HuLxhjooWF\nvrlsh+rP0NTeEzVdOwOWFmays6aF9m6P06UYEzIW+uayDYxyifTx+YGWFWbh8SrvH2l2uhRjQsZC\n31y20qompmaOITcj2elSQuqqqeNIcLvOXXRmTDSw0DeXxdPnZVt1U9QM1fSXnOBm4ZQM3rWTuSaK\nWOiby7L3eBtnuj0sibL+/AHLZmSx70Qbp9t7nC7FmJCw0DeXZWB0y5Lp0XekD/0nc1Vh22Hr4jHR\nwULfXJbSqkZm5qSQnZrodCkjYn5eBmMS3LxrUzKYKBFU6IvIShE5KCKVIvL4IOu/IiL7RGSXiLwl\nIlP91vWJyE7fY13gtiZy9Xi8lB1pjrqhmv4S4lwsLhjPuzb5mokSQ4a+iLiBp4E7gGLgQREpDmi2\nAyhR1fnAi8B3/NZ1quoC32NViOo2YWBnTQtdvd6oG6oZaGlhJtUN7Zxq63K6FGMuWzBH+ouBSlWt\nVtUe+m98vtq/gaq+raodvpfv0X8DdBPltlY1IQLXFkR76Pf/JWNDN000CCb0c4Eav9e1vmUX8jng\nFb/XSSJSLiLvicjdl1CjCVOlVY3MnZxO+ph4p0sZUVdMSiM9Od7m1zdRIah75AZLRD4DlADX+y2e\nqqp1IjId2CAiu1W1KmC7R4BHAKZMmRLKkswI6ezpY8exFh5eNs3pUkac2yVcO308W6vtSN9EvmCO\n9OuAfL/Xeb5l5xGRW4CvAatUtXtguarW+b5WAxuBhYHbquoaVS1R1ZLs7OxhfQDjjO1HT9PTF/39\n+QOWFmZR09xJTXPH0I2NCWPBhH4ZUCQiBSKSADwAnDcKR0QWAs/SH/j1fsvHiUii73kWsAzYF6ri\njXNKqxqJcwlXTxvvdCmjYuCXm/Xrm0g3ZOirqgd4DHgN2A+8oKp7ReQpERkYjfNdIAX4dcDQzCuA\nchH5EHgb+JaqWuhHgdKqJq7MzyAlMaQ9hGGraEIKWSkJ1q9vIl5Q/2NVdT2wPmDZE37Pb7nAdqXA\nvMsp0ISftq5edtW28MUbZzhdyqgREZYUZlFa1YSqRs19A0zssStyzbCVHW7Gq9E3lfJQlhZmUn+m\nm6qGdqdLMeaSWeibYSutaiIhzsWiKeOcLmVULT3Xr29dPCZyWeibYSutaqJk6jiS4t1OlzKqpozv\nv2eA3ULRRDILfTMsze097D/RFpXz5w+lv18/k63VTXi96nQ5xlwSC30zLNuqB26NGL2TrF3M0sJM\nWjp62X+yzelSjLkkFvpmWEqrmhib4GZ+XrrTpTjCxuubSGehb4altKqRxQXjiXfH5j+dSenJTM8a\na6FvIlZs/s81l+RUWxdVDe1RPX9+MJYUZrLtcDOePq/TpRgzbBb6JmgDR7exNj4/0JLCTM52e9hd\n1+p0KcYMm4W+CVppVSPpyfEUT0pzuhRHXeu7H7AN3TSRyELfBK20qokl0zNxuWJ7CoKslERmT0y1\nfn0TkSz0TVBqmjuoPd0Z8107A5YUZlJ2pJluT5/TpRgzLBb6JigDs0vG4kVZg1lamEW3x8uOYy1O\nl2LMsFjom6CUVjWRlZLIjAkpTpcSFhYXjMcl1q9vIo+FvhmSqlJa1cTSwkybUtgnPTmeebnpNvma\niTgW+mZIVQ1naTjTbV07AZYUZrGzpoWOHo/TpRgTtKBCX0RWishBEakUkccHWf8VEdknIrtE5C0R\nmeq37iERqfA9Hgpl8WZ0DHRhxPpFWYGWFmbS26eUHzntdCnGBG3I0BcRN/A0cAdQDDwoIsUBzXYA\nJao6H3gR+I5v2/HAk8A1wGLgSRGJrUnYo0BpZRO5Gcnkj092upSwUjJtHPFusX59E1GCOdJfDFSq\narWq9gBrgdX+DVT1bVXt8L18D8jzPb8deENVm1X1NPAGsDI0pZvR4PUqW6utP38wYxLiWJg/zvr1\nTUQJJvRzgRq/17W+ZRfyOeCV4WwrIo+ISLmIlDc0NARRkhkt+0600drZy9IZ1p8/mGsLM9ld10pr\nZ6/TpRgTlJCeyBWRzwAlwHeHs52qrlHVElUtyc7ODmVJ5jK9NzB//nTrzx/M0sJMvArvH252uhRj\nghJM6NcB+X6v83zLziMitwBfA1apavdwtjXh693KRqZnj2ViepLTpYSlhVMySIxznbt4zZhwF0zo\nlwFFIlIgIgnAA8A6/wYishB4lv7Ar/db9Rpwm4iM853Avc23zESAHo+XbYebWWajdi4oMc7N1dPG\n2zw8JmIMGfqq6gEeoz+s9wMvqOpeEXlKRFb5mn0XSAF+LSI7RWSdb9tm4Bv0/+IoA57yLTMR4INj\np+no6eO6Igv9i1lSmMmBk2doPNs9dGNjHBYXTCNVXQ+sD1j2hN/zWy6y7XPAc5daoHHOlopG3C6x\nSdaGMHDR2nvVTXx8/mSHqzHm4uyKXHNBmysbuTIvnbSkeKdLCWvzctNJSYyz8fomIljom0G1dvSy\nu7aF64psNNVQ4twurikYz7uVdjLXhD8LfTOo0qpGvAorrD8/KMuLsjja1MHRpnanSzHmoiz0zaA2\nVzaSkhjHlfkZTpcSEVbM7P+LaFOFHe2b8Gahbwa1paKRa6dnEu+2fyLBKMgaS25GMpsP2RXlJrzZ\n/2jzEUeb2jnW3MFy69oJmoiwYmY2pVVN9PZ5nS7HmAuy0DcfsdnXRWHj84dnRVEWZ7s97KyxWyia\n8GWhbz5iS0Ujk9OTmJ411ulSIsrSGVm4BOviMWHNQt+cp8+rlFY1cl1Rlk2lPEzpyfEsyM/gHTuZ\na8KYhb45z67aFtq6PDY+/xKtmJnNrtoWWjp6nC7FmEFZ6JvzbDrUiAgss6kXLsnyomxUYYtdqGXC\nlIW+Oc/GQ/XMz8sgMyXR6VIiUv+0FXFsPmShb8KThb45p7m9h501Ldw4y7p2LlWc28WyGVlsqmhA\nVZ0ux5iPsNA352w61IAq3DBrgtOlRLQVM7M50dpFVcNZp0sx5iMs9M05Gw/Wkzk2gfm56U6XEtEG\nLmrbeNCGbprwY6FvgP6hmu8cauD6mdm4XDZU83LkjRvDzJwU3j5YP3RjY0ZZUKEvIitF5KCIVIrI\n44OsXyEiH4iIR0TuC1jX57ub1rk7apnw82FtC6c7erlhtnXthMKNsyewrbqZM129TpdizHmGDH0R\ncQNPA3cAxcCDIlIc0OwY8DDw/CC76FTVBb7HqkHWmzCw8WADLrGplEPl5tk5eLzKFrtQy4SZYI70\nFwOVqlqtqj3AWmC1fwNVPaKquwCbaSpCbTxYz8Ip48gYk+B0KVFh0ZQM0pPjeeuAdfGY8BJM6OcC\nNX6va33LgpUkIuUi8p6I3D1YAxF5xNemvKHBTn6NtoYz3eyqbbWhmiEU53Zx/cxsNh6sx+u1oZsm\nfIzGidypqloCfAr4vogUBjZQ1TWqWqKqJdnZFjyjbZNvgjAbqhlaN18xgcazPeyqa3W6FGPOCSb0\n64B8v9d5vmVBUdU639dqYCOwcBj1mVHw9sF6slMTKZ6U5nQpUeX6mdm4BDbsP+V0KcacE0zolwFF\nIlIgIgnAA0BQo3BEZJyIJPqeZwHLgH2XWqwJvR6Pl3cONnDjLBuqGWoZYxK4auo469c3YWXI0FdV\nD/AY8BqwH3hBVfeKyFMisgpARK4WkVrgk8CzIrLXt/kVQLmIfAi8DXxLVS30w8i2w02c6fZwW/FE\np0uJSjfNzmHv8TZOtnY5XYoxAMQF00hV1wPrA5Y94fe8jP5un8DtSoF5l1mjGUGv7z1Fcrzb7pI1\nQm6aPYFvv3qAtw/W8+DiKU6XY4xdkRvLVJU39p1ixcwskuLdTpcTlWbmpJCbkcxb+62Lx4QHC/0Y\ntruulZNtXdxqXTsjRkS4tTiHzRUNdPR4nC7HGAv9WPbGvlO4pL8Lwoyc2+dMpNt3wtwYp1nox7DX\n957i6mnjGT/WrsIdSVdPG8f4sQm8uvek06UYY6Efq442tXPw1BluLc5xupSoF+d2cesVOWzYX0+3\np8/pckyMs9CPUW/s679gyIZqjo6VcydypttDaVWT06WYGGehH6Ne33uK2RNTmZI5xulSYsLSGZmk\nJMbx2h7r4jHOstCPQafauig72sztc+wof7Qkxrm5afYEXt93ij6bgM04yEI/Br2y+wSq8PH5k5wu\nJaasnDuR5vYeyo40O12KiWEW+jHo97tPMCsnlaKcVKdLiSnXz8wmMc7Fq9bFYxxkoR9jTrR2Unbk\ntB3lO2BsYhwrZmbzyp4T1sVjHGOhH2PW7+4/yrzTQt8Rq66czKm2bt4/bF08xhkW+jHm97uOUzwp\njenZKU6XEpNuuSKHMQlu1n0Y9C0pjAkpC/0YUtfSyQfHWuwo30HJCW5uK85h/e6T9HjsltJm9Fno\nx5D1u04ANmrHaasWTKa1s/fcbSqNGU0W+jHktzvqmJ+XztTMsU6XEtOWF2Uzbkw86z487nQpJgYF\nFfoislJEDopIpYg8Psj6FSLygYh4ROS+gHUPiUiF7/FQqAo3w7P/RBv7TrRx76KP3OvGjLJ4t4uP\nzZvEG/tO0d5t0y2b0TVk6IuIG3gauAMoBh4UkeKAZseAh4HnA7YdDzwJXAMsBp4UkXGXX7YZrt/u\nqCPOJdx15WSnSzH0j+Lp7O3jTbtpuhllwRzpLwYqVbVaVXuAtcBq/waqekRVdwGBZ6ZuB95Q1WZV\nPQ28AawMQd1mGPq8yss76rhx9gSbRjlMXD1tPJPTk/jNBzaKx4yuYEI/F6jxe13rWxaMoLYVkUdE\npFxEyhsa7ORWqL1b2Uj9mW7uXRTsj82MNJdLuO+qPDZXNHC8pdPpckwMCYsTuaq6RlVLVLUkOzvb\n6XKizksf1JKeHM+NdoessPLJknxU4cXttU6XYmJIMKFfB+T7vc7zLQvG5WxrQqCtq5dX957k4/Mn\nkRhnNz8PJ/njx7C0MJNfb6/Ba9MymFESTOiXAUUiUiAiCcADwLog9/8acJuIjPOdwL3Nt8yMkpd3\n1NHV6+WBq6c4XYoZxP1X51PT3Ml7h+3mKmZ0DBn6quoBHqM/rPcDL6jqXhF5SkRWAYjI1SJSC3wS\neFZE9vq2bQa+Qf8vjjLgKd8yMwpUlee3HWNubhrz8tKdLscM4vY5E0lNiuOFspqhGxsTAnHBNFLV\n9cD6gGVP+D0vo7/rZrBtnwOeu4wazSXaUdPCgZNn+OY985wuxVxAUryb1Qsm8+vyWv6hs5f05Hin\nSzJRLixO5JqR8fy2Y4xNcLNqgY3ND2f3l0yh2+Pltx/YCV0z8iz0o1RrZy//ves4qxbkkpIY1B90\nxiHz8tJZkJ/BL7YetRO6ZsRZ6Eeplz6opavXy6evsRO4keDhpdOobmxnS2Wj06WYKGehH4X6vMrP\nSo+waEoGc3PtBG4k+Ni8SWSlJPLz0iNOl2KinIV+FHpz/ymONnXwueumO12KCVJCnItPLc5nw8F6\njjV1OF2OiWIW+lHop1sOk5uRzO1zcpwuxQzDp6+diluEX2w94nQpJopZ6EeZPXWtvH+4mYeXTiPO\nbT/eSJKTlsTH5k1ibVkNrZ29TpdjopSlQpT56ZbDjE1wc//i/KEbm7Dz6PWFnO328Mv3jjpdiolS\nFvpRpKa5g//68Dh/cHU+aUl2kU8kKp6cxg2zsnluy2G6evucLsdEIQv9KPLMxipcIvzZikKnSzGX\n4fPXF9LU3sML5TY1gwk9C/0oUdfSyYvba/iDq/OYmJ7kdDnmMiwuGM9VU8fx7DvV9PYF3pfImMtj\noR8lnn2nCoDP3zDD4UrM5RIRvnBDIXUtnfzG5to3IWahHwVOtXWxtqyG+67KIzcj2elyTAjcNHsC\nC/Iz+MFbFda3b0LKQj8KfP/NClSVz19vR/nRQkT469tncaK1y0bymJCy0I9wlfVn+FXZMT59zVSm\nZI5xuhwTQktnZHHdjCye2VjF2W6P0+WYKBFU6IvIShE5KCKVIvL4IOsTReRXvvXbRGSab/k0EekU\nkZ2+x49CW7751isHGZsQx5/fZEf50eivbp9Fc3sPazZVO12KiRJDhr6IuIGngTuAYuBBESkOaPY5\n4LSqzgD+Gfi237oqVV3gezwaoroNsK26iTf3n+LRGwrJTEl0uhwzAhbkZ3DnvEk8+04VNc02J4+5\nfMEc6S8GKlW1WlV7gLXA6oA2q4Gf+56/CNwsIhK6Mk0gT5+XJ9ftZXJ6Ep9dVuB0OWYE/d2dVyAC\n//T7/U6XYqJAMKGfC/hfJVLrWzZoG989dVuBTN+6AhHZISLviMjyy6zX+Px861EOnDzDE3fNITnB\n7XQ5ZgTlZiTz2I0zeHXvSbZU2Hz75vKM9IncE8AUVV0IfAV4XkTSAhuJyCMiUi4i5Q0NDSNcUuQ7\n1dbFP79xiBtmZdtMmjHiT5ZPZ8r4MTy5bg/dHhvCaS5dMKFfB/jP3pXnWzZoGxGJA9KBJlXtVtUm\nAFXdDlQBMwPfQFXXqGqJqpY2hfIrAAAKN0lEQVRkZ2cP/1PEEFXlyd/tpafPyz+smoP1osWGpHg3\n/7B6DlUN7fxwQ6XT5ZgIFkzolwFFIlIgIgnAA8C6gDbrgId8z+8DNqiqiki270QwIjIdKAJsGMJl\n+N3O47y69yT/65aZTM0c63Q5ZhTdOGsC9y7K45mNVeyubXW6HBOhhgx9Xx/9Y8BrwH7gBVXdKyJP\nicgqX7OfApkiUkl/N87AsM4VwC4R2Un/Cd5HVbU51B8iVpxs7eKJ3+1h0ZQMHllhd8WKRU98vJis\nlAT+6tcfWjePuSSiqk7XcJ6SkhItLy93uoyw0+dVHv639yk70swrX15BQZYd5ceqDQdO8dmflfPw\n0mn8/ao5TpdjwoSIbFfVkqHa2RW5EeJfN1SwuaKRJ++aY4Ef426ancNnlxXws9IjvLrnhNPlmAhj\noR8BNh1q4AdvVfCJRbk8cLXdEcvA43fM5sq8dL764i6ONLY7XY6JIBb6Ye5wYztfWruDmRNS+ce7\n59poHQNAQpyLH35qEW6X8Lmfl9k9dU3QLPTDWHN7D3/8b+/jEmHNH13FmIQ4p0syYSR//Bh+9Jmr\nONbcwRf+Y7vdcMUExUI/THX0ePjTX5RzvLWLH/9RiQ3PNIO6dnom37xnHu9WNvHXL+7C6w2vgRkm\n/NihYxjq7Onjsz8rY8ex0zz9qUVcNXWc0yWZMPbJknxOtXXxf14/RFK8i2/eM8+6Ac0FWeiHmfZu\nD4/8eznvH27mn+9fwB3zJjldkokAj91URGdvH0+/XQUI/3j3XNwuC37zURb6YaThTDef+3kZe+pa\n+e59V7J6QeC8dsZc2F/dNgtVeGZjFc3t3fzggYUkxdtkfOZ81qcfJg6ePMO9/6+UQ6fOsOYPS7j3\nqjynSzIRRkT465WzefKuYl7fd4rP/GQb9We6nC7LhBkL/TDwm+21rH56Cx09ffznn17LLcU2c6a5\ndH+8rIAfPriIPcdbufNftvBedZPTJZkwYqHvoJaOHv7yhQ/5y19/yJV5Gaz/0nUsnGInbc3lu3P+\nJF7+4jJSE+P41I/f4/++ftDm6jGAhb4jVJX/3nWcW763iZd31vHnN83gP/7kGiakJTldmokisyem\nse7Pr+PuBbn864ZK7vyXLZQfsfkOY51NuDbKth9t5luvHKDsyGnm5qbx7XvnM2dyutNlmSi38WA9\nX/vtHupaOrlj7kS+evsspmenOF2WCaFgJ1yz0B8FqsrmikZ+suUwmw41kJ2ayF/cUsT9JfnEue2P\nLTM62rs9/HhzNWs2VdPt8XLX/En8yfLpzM21g45oYKEfBk62dvHfu47zQnkNh06dJTs1kYeXTuOP\nl02zKRWMYxrOdPOjd6r4VVkNZ7s9LC4Yz31X5bFy7kTSkuKdLs9cIgt9B6gqFfVn2XSogQ0H6tla\n3YQqzM9L56El0/j4lZNIjLNx0yY8tHX18qv3a/jltqMcbeogMc7FDbOyuWHWBG6Ylc2k9GSnSzTD\nENLQF5GVwA8AN/ATVf1WwPpE4BfAVUATcL+qHvGt+1vgc0Af8CVVfe1i7xVJoX+mq5e9x9vYXdvK\n7rpWth1u4lRbNwCF2WP5+PzJrFowmULrOzVhTFXZWdPCyzvqeH3fKU609o/tn549loX541gwJYOF\n+RkUZqeQnGAHLeEqZKHvu8ftIeBWoJb+e+Y+qKr7/Np8AZivqo+KyAPAPap6v4gUA/8JLAYmA28C\nM1X1gmPHwiX0+7xKU3s3jWd6aDzbTePZbk60dnG0qZ2jTR0cbergZNv/XPgyOT2JhVPHsXxGFtcV\nZZE3boyD1RtzaQb+Wt14sJ5t1c3srGmhqb3n3PrcjGQKJ6QwPWssE9OTmJiWRE5aEjlpiWSlJpKS\nEIfLpn9wRLChH0zH8mKgUlWrfTteC6wG9vm1WQ38ve/5i8APpX/Gp9XAWlXtBg777qG7GNga7AcJ\nVmdPHy/tqKXPq/T2KZ4+Lx6v4ulTPN6B5156+5Q+b/+yrl4v7d0e2ns8tHf30d7toaOnj7PdHtq6\nehns92F2aiLTMsdwXVEWBVljmTM5jbm56WSlJIb6Ixkz6kSEmTmpzMxJ5ZEVhagqtac72VXbSlXD\n2XOP7Ueaae/56LGbCKQmxpGWHE9aUjypSXGMSXCTGOcmMd5FYpyLhDhX/2vfc7cILpfgEsHtApcM\nPBdcAi6X9LfxtQv8leI/t1zgPHOBrS82D13gJHVy3rqL7zdUMsbEs2xG1ojse0AwoZ8L1Pi9rgWu\nuVAbVfWISCuQ6Vv+XsC2H5lQRkQeAR4BmDJlSrC1n6ejx8PXfrvnguvj3UKcy0WcW4hzCXFuF0nx\nLsYmxDE2MY7UpDgmpiUxNjGOsYluMpLjyUpNJCtl4JFAjm+9MbFCRMgfP4b88R/9y/Vst4eTrV2c\nauviZGsXpzt6aOvspa3L4/vaS1unh8azPfR4vHR7+uj2ePsfvf3PPTYV9HkW5GeEReiPOFVdA6yB\n/u6dS9nHuDEJbPu7m3G7hHhfuLtdQrzbZbMNGjMCUhLjmDEhhRkTLv2clder9KniVcXrxe+54tX+\nblavb1mft7+NP+V/4iLwL/PAIPHvyv7ousDKgt9vKCWNwkCPYEK/DvC/MWueb9lgbWpFJA5Ip/+E\nbjDbhoTLJeTYFa3GRBSXS3CNUFeJGVwwVwaVAUUiUiAiCcADwLqANuuAh3zP7wM2aP+v1XXAAyKS\nKCIFQBHwfmhKN8YYM1xDHun7+ugfA16jf8jmc6q6V0SeAspVdR3wU+DffSdqm+n/xYCv3Qv0n/T1\nAF+82MgdY4wxI8suzjLGmCgQ7JBNm/jFGGNiiIW+McbEEAt9Y4yJIRb6xhgTQ8LuRK6INABHna4j\nDGQBjUEsu9i2BcDhQbYr8H097NceX5vZvq/+7WcD3cAZv3ZZfvtN9a3zf0//bRr92mb52h/22/cB\nYB5w1rd8HrDbb93APq8E2oAM+keDnfXtI43+6z8G3h+/9vjqSAzY1uV7PbDtJN8yr2+/iUA80Ovb\n/nDAvge+T1kBX1M5//MPfE78PkeB7/sy8H0b+J768/8+DffnPtQyE52mqmr2kK1U1R5h+KB/OOyQ\nyy62LdA+2HZA+8C6gfV+2/QN0r7Pt41/u/KBtgPrAt6zL2C5/3b+793n+6p+22rAunb/Nr6vA88H\nXpcHfja/dn2DbOv/unyQ5X1+27YPsu+Bzx749bzPH/h999tPOQHf08Cf4YV+fiPxb8YesfOw7h1j\njIkhFvrGGBNDwmLCNTOoNUEuu9i2L11gu5cCXvuvLwOeC1hfBlQAmwO2+ayv7XK/dS9dYJs1fl+X\nB+wb+vuu3/V77r9uYJ+NwKvA3UCDX/uVfPQzD7QHqAIKA7ZN9b0e2PZx/qeP/V36pwyZANT7Pkfg\nvge+T58N+Lrcb30F53vJ7+tmzv++BfL/Pg335z7UMhPDwu5ErjHGmJFj3TvGGBNDLPSNMSaGWOgb\nY0wMsdA3xpgYYqFvjDEx5P8DjojtOgDr794AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPZVbhysiiTx",
        "colab_type": "text"
      },
      "source": [
        "The horizontal axis represents the domain of $x$. The generator network will map random values to this domain. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U3TeFHCziiTy",
        "colab_type": "text"
      },
      "source": [
        "Our discriminator and generator networks will be MLP:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "smyrTsSFiiTz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "TRAIN_ITERS=20000\n",
        "M=200 # minibatch size\n",
        "\n",
        "def mlp(input, output_dim):\n",
        "    # construct learnable parameters within local scope\n",
        "    w1=tf.get_variable(\"w0\", [input.get_shape()[1], 6], initializer=tf.random_normal_initializer())\n",
        "    b1=tf.get_variable(\"b0\", [6], initializer=tf.constant_initializer(0.0))\n",
        "    w2=tf.get_variable(\"w1\", [6, 5], initializer=tf.random_normal_initializer())\n",
        "    b2=tf.get_variable(\"b1\", [5], initializer=tf.constant_initializer(0.0))\n",
        "    w3=tf.get_variable(\"w2\", [5,output_dim], initializer=tf.random_normal_initializer())\n",
        "    b3=tf.get_variable(\"b2\", [output_dim], initializer=tf.constant_initializer(0.0))\n",
        "    # nn operators\n",
        "    fc1=tf.nn.tanh(tf.matmul(input,w1)+b1)\n",
        "    fc2=tf.nn.tanh(tf.matmul(fc1,w2)+b2)\n",
        "    fc3=tf.nn.relu(tf.matmul(fc2,w3)+b3)\n",
        "    return fc3, [w1,b1,w2,b2,w3,b3]\n",
        "\n",
        "def mlpg(input, output_dim):\n",
        "    # construct learnable parameters within local scope\n",
        "    w1=tf.get_variable(\"w0\", [input.get_shape()[1], 10], initializer=tf.random_normal_initializer())\n",
        "    b1=tf.get_variable(\"b0\", [10], initializer=tf.constant_initializer(0.0))\n",
        "    w2=tf.get_variable(\"w1\", [10, 10], initializer=tf.random_normal_initializer())\n",
        "    b2=tf.get_variable(\"b1\", [10], initializer=tf.constant_initializer(0.0))\n",
        "    w3=tf.get_variable(\"w2\", [10,output_dim], initializer=tf.random_normal_initializer())\n",
        "    b3=tf.get_variable(\"b2\", [output_dim], initializer=tf.constant_initializer(0.0))\n",
        "    # nn operators\n",
        "    fc1=tf.nn.tanh(tf.matmul(input,w1)+b1)\n",
        "    fc2=tf.nn.tanh(tf.matmul(fc1,w2)+b2)\n",
        "    fc3=tf.nn.tanh(tf.matmul(fc2,w3)+b3)\n",
        "    return fc3, [w1,b1,w2,b2,w3,b3]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "szXIlf13iiT1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# re-used for optimizing all networks\n",
        "def momentum_optimizer(loss,var_list):\n",
        "    batch = tf.Variable(0)\n",
        "    learning_rate = tf.train.exponential_decay(\n",
        "        0.0001,               # Base learning rate.\n",
        "        batch,               # Current index into the dataset.\n",
        "        TRAIN_ITERS // 6,    # Decay step - this decays 4 times throughout training process.\n",
        "        0.95,                # Decay rate.\n",
        "        staircase=True)\n",
        "    optimizer=tf.train.MomentumOptimizer(learning_rate,0.6).minimize(loss,\n",
        "                                                                     global_step=batch,\n",
        "                                                                     var_list=var_list)\n",
        "    return optimizer"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TSc_VwlLiiT3",
        "colab_type": "text"
      },
      "source": [
        "Pre-train discriminator"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LHl3l_leiiT4",
        "colab_type": "code",
        "outputId": "7ad178ce-1c15-4d02-8e24-f5659ed29ef7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 371
        }
      },
      "source": [
        "with tf.variable_scope(\"D_pre\"):\n",
        "    input_node=tf.placeholder(tf.float32, shape=(M,1))\n",
        "    train_labels=tf.placeholder(tf.float32,shape=(M,1))\n",
        "    D,theta=mlp(input_node,1)\n",
        "    loss=tf.reduce_mean(tf.square(D - train_labels))\n",
        "    \n",
        "optimizer=momentum_optimizer(loss,None)\n",
        "\n",
        "sess=tf.InteractiveSession()\n",
        "tf.global_variables_initializer().run()\n",
        "\n",
        "# plot decision surface\n",
        "def plot_d0(D,input_node):\n",
        "    f,ax=plt.subplots(1)\n",
        "    # p_data\n",
        "    xs=np.linspace(-5,5,1000)\n",
        "    ax.plot(xs, norm.pdf(xs,loc=mu,scale=sigma), label='p_data')\n",
        "    # decision boundary\n",
        "    r=1000 # resolution (number of points)\n",
        "    xs=np.linspace(-5,5,r)\n",
        "    ds=np.zeros((r,1)) # decision surface\n",
        "    \n",
        "    # process multiple points in parallel in a minibatch\n",
        "    for i in range(int(r/M)):\n",
        "        x=np.reshape(xs[M*i:M*(i+1)],(M,1))\n",
        "        ds[M*i:M*(i+1)]=sess.run(D,{input_node: x})\n",
        "    \n",
        "    ax.plot(xs, ds, label='decision boundary')\n",
        "    ax.set_ylim(0,1.1)\n",
        "    plt.legend()\n",
        "    \n",
        "plot_d0(D,input_node)\n",
        "plt.title('Initial Decision Boundary')"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Initial Decision Boundary')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VNX9//HXJxsJ+xJAZDFRIWwh\nbAKKoFariIhiK2hdqlZotdZd61a1+rVfrdaqX7EKv1alRXBBFBXrUkFRFGUXWcMeZBdC9kyS8/tj\nJmkMIZmEmUxu8n4+Hphk7pl7P2OSd86cc++55pxDREQalqhIFyAiIqGncBcRaYAU7iIiDZDCXUSk\nAVK4i4g0QAp3EZEGSOEulTKz983sl1Vsf97M/hDkvuab2bWhqy44ZnaZmX0YRLugX4tXmdmDZvav\nSNchdUfh3oiY2RYzOyuYts65c51zLweed5WZfV5h+2+ccw+HoKYHzcxnZlmBf+vN7Fkz63S0+3bO\nTXfOnR1Eu5C8lorKvbbswL81ZvazUB9HpDIKd6kPXnXOtQDaAuOAY4AloQj4euBV51xz51xz4Gbg\nX2bWMdJF1YSZRUe6Bqk5hXsjVdobN7MnzOyAmW02s3PLbZ9vZteaWS/geeDkQO/zYGD7S2b2P4HP\n25jZu2a2N7Cvd82sS01rcs75nHPfAROAvcBt5eoZY2bLzeygmS00s37ltnU1szcDx99vZs+Wf42B\nz83M/mpme8zskJl9a2Z9K76WwNcTzSzdzH4wszlmdmy5bc7MfmNmGwK1TDYzC/L1fQBkASdUdywz\nSwocK6Zc27LhrSC+f8lm9mng3dBHQGL5WszsdTPbZWaZZvaZmfUpt+0lM/ubmc01sxzgVjPbXT7k\nzewiM1sRzOuWyFC4N25DgXX4f/H/DPy9YlA559YAvwG+DPRAW1eynyjgReA4oBuQBzxb26Kcc8XA\n28AIADMbAPwD+DXQDngBmGNmTQKB8y6wFUgCOgMzK9nt2cBIoAfQChgP7K/YyMx+AvxvYHunwH4r\n7m8McBLQL9DunOpeU+CPy3lAHLC6BseqSlXfv1eAJYFtDwMV50/eB7oDHYClwPQK238BPAK0AP4P\n//+r8kNcVwDTalCr1DGFe+O21Tk3NRCmL+MPmBoPGTjn9jvnZjnncp1zWfhD4bSjrO17/MM0AJOA\nF5xzi5xzxYG5gAJgGDAEOBa4wzmX45zLd859Xsn+fPiDqidgzrk1zrmdlbS7DPiHc26pc64AuBv/\nu5akcm0edc4ddM5tA+YB/at4HeMD73aygTnAn5xzB2twrKpU+v0zs274//j8wTlX4Jz7DHin/BOd\nc/9wzmUFjvsgkGZmrco1eds594VzrsQ5lx/Y/+UAZtYW/x+0V4KsUyJA4d647Sr9xDmXG/i0eU13\nYmZNzewFM9tqZoeAz4DWRzlW2xn4IfD5ccBtgWGQg4Gw7Io/1LviD7miqnbmnPsE/7uJycAeM5ti\nZi0raXos/h506fOy8fdaO5drs6vc57lU/f/sNedca+dcM/zDMVea2a9rcKyqHOn7dyxwwDmXU65t\n2XHMLNrMHjWzjYHv15bApvJDN9srHOtfwPlm1gz/O40FR/jjKPWEwl2CUd3SobcBKcBQ51xL/MMf\nAEGNRVdkZlHA+cCCwEPbgUcCIVn6r6lzbkZgW7fyY9NHfBHOPeOcGwT0xj88c0clzb7H/8ektJZm\n+IeCdtTmtVQ4/hb8wyHnB3Gs0mBuWm4XxwR5qJ1Am8D+SnUr9/kvgAuAs/APUSWVllC+3Aq17wC+\nBC7CPyTzzyBrkQhRuEswdgNdzCzuCNtb4B9nPxh4y/5AbQ5iZjGBCdwZ+IPsycCmqcBvzGxoYOy6\nmZmdZ2YtgK/xh9mjgcfjzWx4Jfs+KfD8WPzBmQ+UVFLGDOBqM+tvZk2APwGLAsF8VMw/yTwK+K66\nYznn9uIP+csDPe1rKDcRWxXn3FZgMfBHM4szs1P57x8U8H+/CvC/S2gaOG4wpgF3AqnAm0E+RyJE\n4S7B+AR/IO0ys32VbH8KSAD2AV8B/67h/ieYWTaQiX9cej8wyDn3PYBzbjEwEf+wygEgHbgqsK0Y\nf3CdCGwDMvCfbVNRS/x/JA7gH6LYDzxesZFz7mPgD8As/H80TgAuqeHrOey1BV7fN8AXwB+DPNZE\n/O8u9gN9gIU1OO4v8E+4/oD/j235yc9p+P8f7MA/uftVkPucjf+dxuxyw0BST5lu1iEiwTKzjcCv\nA3+YpB5Tz11EgmL+q2sd/ndyUs9VOwklImJm8/FPRF/hnKtsrkLqGQ3LiIg0QBqWERFpgCI2LJOY\nmOiSkpIidXiRo5e9Bw7tgGP6QZTW1pK6sWTJkn3OufbVtYtYuCclJbF48eJIHV7k6P3nIfj8Kbh/\nKQS3dpjIUTOzrdW30rCMSO3lHYCENgp2qZcU7iK1VRruIvWQwl2kthTuUo/pPHeR2so7AM2DXcur\ndnw+HxkZGeTn54f1OFL/xMfH06VLF2JjY2v1fIW7SG3lHYD2vcJ6iIyMDFq0aEFSUhJB3vBJGgDn\nHPv37ycjI4Pk5ORa7UPDMiK1lXcw7MMy+fn5tGvXTsHeyJgZ7dq1O6p3bAp3kdoo9kHBoToZc1ew\nN05H+31XuIvURl7gTnlN21bdTiRCFO4itZF3wP9RZ8tIPaVwF6mNsnBvHdk66rH58+czZsyYKtss\nX76cuXPn1lFFjYvCXaQ21HMPCYV7+OhUSJHaiEC4//Gd71j9/aGQ7rP3sS154Pw+VbbZsmULo0aN\nYtCgQSxdupQ+ffowbdo0mjZteljbf//739x88800bdqUU089tezxr7/+mptuuon8/HwSEhJ48cUX\nSU5O5v777ycvL4/PP/+cu+++m+Tk5MPapaSkhPQ1NxbquYvURiPrua9bt47rr7+eNWvW0LJlS557\n7rnD2uTn5zNx4kTeeecdlixZwq5du8q29ezZkwULFrBs2TIeeugh7rnnHuLi4njooYeYMGECy5cv\nZ8KECZW2k9pRz12kNvIOAAZNWtXZIavrYYdT165dGT58OACXX345zzzzDLfffvuP2qxdu5bk5GS6\nd+9e1m7KlCkAZGZm8stf/pINGzZgZvh8vkqPE2w7qZ567iK1kXfAP5ka1Th+hSqec13Tc7D/8Ic/\ncMYZZ7Bq1SreeeedI16cE2w7qV61P5lm9g8z22Nmq46w3czsGTNLN7OVZjYw9GWK1DONbNGwbdu2\n8eWXXwLwyiuv/Gg8vVTPnj3ZsmULGzduBGDGjBll2zIzM+ncuTMAL730UtnjLVq0ICsrq9p2UnPB\ndDteAkZVsf1coHvg3yTgb0dflkg9l/cDJDSeC5hSUlKYPHkyvXr14sCBA1x33XWHtYmPj2fKlCmc\nd955DBw4kA4dOpRtu/POO7n77rsZMGAARUVFZY+fccYZrF69mv79+/Pqq68esZ3UXFA3yDazJOBd\n51zfSra9AMx3zs0IfL0OON05t7OqfQ4ePNjpTkziWVNOh6aJcPkbYT3MmjVr6NUrvIuTVWfLli2M\nGTOGVasqffMuYVTZ99/MljjnBlf33FAMGHYGtpf7OiPw2GHMbJKZLTazxXv37g3BoUUipJENy4j3\n1OnZMs65KcAU8Pfc6/LYIiHViMI9KSnpsF77uHHj2Lx5848ee+yxxzjnnHPqsjSpQijCfQfQtdzX\nXQKPiTRMJSWQf6hRLz0we/bsSJcg1QjFsMwc4MrAWTPDgMzqxttFPK3gEOAgvu7OcRepqWp77mY2\nAzgdSDSzDOABIBbAOfc8MBcYDaQDucDV4SpWpF7Iz/R/VLhLPVZtuDvnLq1muwN+G7KKROq7gsD6\nLk1aRrYOkSo0jsvrREKpEffcH3zwQZ544olaPfeUU06pcvvo0aM5ePBgrfZd3lVXXcUbb4T3FNWK\nmjdvXqfHC4bWlhGpqfxAz70RhvvRWLhwYZXbG8vSv845nHNEhXnpCvXcRWqqrOfeOIZlHnnkEXr0\n6MGpp57KunXryh7fuHFj2VLAI0aMYO3atQDs3r2bcePGkZaWRlpaWlmol/Zud+7cyciRI+nfvz99\n+/ZlwYIFgP+Uy3379gHw5JNP0rdvX/r27ctTTz0F+C+m6tWrFxMnTqRPnz6cffbZ5OXlVVrzxx9/\nzODBg+nRowfvvvsu4F+18uqrryY1NZUBAwYwb948wL/MwQ033FD23DFjxjB//vyymu+9917S0tIY\nNmwYu3fvBmDz5s2cfPLJpKamct9995U9Nzs7mzPPPJOBAweSmprK22+/XVZ7SkoKV155JX379uXh\nhx/m5ptvLnve1KlTueWWW2r8vamKeu4iNVUW7nV8KuT7d8Gub0O7z2NS4dxHj7h5yZIlzJw5k+XL\nl1NUVMTAgQMZNGgQAJMmTeL555+ne/fuLFq0iOuvv55PPvmEG2+8kdNOO43Zs2dTXFxMdnb2j/b5\nyiuvcM4553DvvfdSXFxMbm7uYcd88cUXWbRoEc45hg4dymmnnUabNm3YsGEDM2bMYOrUqYwfP55Z\ns2Zx+eWXH1b3li1b+Prrr9m4cSNnnHEG6enpTJ48GTPj22+/Ze3atZx99tmsX7++yv89OTk5DBs2\njEceeYQ777yTqVOnct9993HTTTdx3XXXceWVVzJ58uSy9vHx8cyePZuWLVuyb98+hg0bxtixYwHY\nsGEDL7/8MsOGDSM7O5u0tDQef/xxYmNjefHFF3nhhReq/l7VkMJdpKbKJlRbRLaOOrBgwQLGjRtX\ndmOO0qDKzs5m4cKFXHzxxWVtCwoKAPjkk0+YNm0aANHR0bRq9ePhq5NOOolrrrkGn8/HhRdeSP/+\n/X+0/fPPP2fcuHE0a9YMgIsuuogFCxYwduxYkpOTy9oPGjSILVu2VFr3+PHjiYqKonv37hx//PGs\nXbuWzz//nN/97neAf5Gz4447rtpwj4uLK7tV4KBBg/joo48A+OKLL5g1axYAV1xxBb///e8B/5DL\nPffcw2effUZUVBQ7duwo6+0fd9xxDBs2DPC/I/jJT37Cu+++S69evfD5fKSmplZZS00p3EVqKj8T\nYptBdGzdHreKHnZdKykpoXXr1ixfvrzGzx05ciSfffYZ7733HldddRW33norV155ZVDPbdKkSdnn\n0dHRRxyWqckSxTExMZSUlJR9XX6Z4djY2LLnRkdH/2gxs8r2OX36dPbu3cuSJUuIjY0lKSmpbH+l\nf6xKXXvttfzpT3+iZ8+eXH116M8g15i7SE3lH2w0k6kjR47krbfeIi8vj6ysLN555x0AWrZsSXJy\nMq+//jrg77GuWLECgDPPPJO//c2/OGxxcTGZmZk/2ufWrVvp2LEjEydO5Nprr2Xp0qU/2j5ixAje\neustcnNzycnJYfbs2YwYMaJGdb/++uuUlJSwceNGNm3aREpKCiNGjGD69OkArF+/nm3btpGSkkJS\nUhLLly+npKSE7du38/XXX1e7/+HDhzNz5kyAsn2Cf8niDh06EBsby7x589i6desR9zF06FC2b9/O\nK6+8wqWXVnnGea0o3EVqKv9Qo5lMHThwIBMmTCAtLY1zzz2Xk046qWzb9OnT+fvf/05aWhp9+vQp\nmzx8+umnmTdvHqmpqQwaNIjVq1f/aJ/z588nLS2NAQMG8Oqrr3LTTTcddsyrrrqKIUOGMHToUK69\n9loGDBhQo7q7devGkCFDOPfcc3n++eeJj4/n+uuvp6SkhNTUVCZMmMBLL71EkyZNGD58OMnJyfTu\n3Zsbb7yRgQOrvyXF008/zeTJk0lNTWXHjv+utnLZZZexePFiUlNTmTZtGj179qxyP+PHj2f48OG0\naRP6dYqCWvI3HLTkr3jWy2OhKB9+9WHYD1UflvyV8BkzZgy33HILZ555ZqXbI73kr0jjUnCo0QzL\nSHgcPHiQHj16kJCQcMRgP1qaUBWpqfxMaHtCpKsQD2vdunW1Z+ocLfXcRWoqP7NOe+6RGjqVyDra\n77vCXaQmnAtMqNZNuMfHx7N//34FfCPjnGP//v3Ex8fXeh8alhGpCV8elPjq7GyZLl26kJGRgW5L\n2fjEx8fTpUuXWj9f4S5SE3W8ImRsbCzJycl1cixpWDQsI1ITWstdPELhLlITkVo0TKSGFO4iNaG1\n3MUjFO4iNZEfuFNQI1l+QLxL4S5SE434FnviLQp3kZoo0LCMeIPCXaQm8g9BVAzE1P7iEpG6oHAX\nqYmCLP8dmKq4+YNIfaBwF6mJ0nAXqecU7iI1UZitC5jEExTuIjVRcAjimke6CpFqKdxFaqIgW8My\n4gkKd5GaKMiCJuq5S/2ncBepCU2oikco3EVqQhOq4hEKd5FglZT4w10TquIBQYW7mY0ys3Vmlm5m\nd1WyvZuZzTOzZWa20sxGh75UkQgrzPZ/1LCMeEC14W5m0cBk4FygN3CpmfWu0Ow+4DXn3ADgEuC5\nUBcqEnEFWf6PCnfxgGB67kOAdOfcJudcITATuKBCGweUDkS2Ar4PXYki9URZuGtYRuq/YMK9M7C9\n3NcZgcfKexC43MwygLnA7yrbkZlNMrPFZrZYN/wVzykbltGEqtR/oZpQvRR4yTnXBRgN/NPMDtu3\nc26Kc26wc25w+/btQ3RokTpSutyvJlTFA4IJ9x1A13Jfdwk8Vt6vgNcAnHNfAvFAYigKFKk3CjSh\nKt4RTLh/A3Q3s2Qzi8M/YTqnQpttwJkAZtYLf7hr3EUaFk2oiodUG+7OuSLgBuADYA3+s2K+M7OH\nzGxsoNltwEQzWwHMAK5yzrlwFS0SEQp38ZCYYBo55+binygt/9j95T5fDQwPbWki9UxhINw15i4e\noCtURYJVkAXRTSAmLtKViFRL4S4SLC0aJh6icBcJltZyFw9RuIsES2u5i4co3EWCpeV+xUMU7iLB\nKjikYRnxDIW7SLAKsnQapHiGwl0kWJpQFQ9RuIsESxOq4iEKd5FgFBdBUZ4mVMUzFO4iwSjUujLi\nLQp3kWAUaF0Z8RaFu0gwtJa7eIzCXSQYun+qeIzCXSQYZWPumlAVb1C4iwRDY+7iMQp3kWAU5vg/\nalhGPELhLhKM0nBXz108QuEuEozCwNkycc0iW4dIkBTuIsEozIGoGIjWLfbEGxTuIsEozPH32s0i\nXYlIUBTuIsEozNZ4u3iKwl0kGKU9dxGPULiLBEPhLh6jcBcJRmGOhmXEUxTuIsEozFbPXTxF4S4S\nDA3LiMco3EWCoXAXj1G4iwRDY+7iMQp3keo4pzF38Zygwt3MRpnZOjNLN7O7jtBmvJmtNrPvzOyV\n0JYpEkFF+eBKFO7iKTHVNTCzaGAy8FMgA/jGzOY451aXa9MduBsY7pw7YGYdwlWwSJ3TipDiQcH0\n3IcA6c65Tc65QmAmcEGFNhOByc65AwDOuT2hLVMkgrQipHhQMOHeGdhe7uuMwGPl9QB6mNkXZvaV\nmY2qbEdmNsnMFpvZ4r1799auYpG6VtZzV7iLd4RqQjUG6A6cDlwKTDWz1hUbOeemOOcGO+cGt2/f\nPkSHFgkzhbt4UDDhvgPoWu7rLoHHyssA5jjnfM65zcB6/GEv4n1lwzIacxfvCCbcvwG6m1mymcUB\nlwBzKrR5C3+vHTNLxD9MsymEdYpEjnru4kHVhrtzrgi4AfgAWAO85pz7zsweMrOxgWYfAPvNbDUw\nD7jDObc/XEWL1CmFu3hQtadCAjjn5gJzKzx2f7nPHXBr4J9Iw6JhGfEgXaEqUh313MWDFO4i1SnM\nAQxiEiJdiUjQFO4i1SldETJKvy7iHfppFamOFg0TD1K4i1RHa7mLByncRaqjcBcPUriLVEc36hAP\nUriLVEc9d/EghbtIdRTu4kEKd5HqaFhGPEjhLlIdnQopHqRwF6mOhmXEgxTuIlUp9kFxgcJdPEfh\nLlIV3RxbPErhLlIVrQgpHqVwF6mKwl08SuEuUhXdqEM8SuEuUpXSnnts08jWIVJDCneRqmhYRjxK\n4S5SFQ3LiEcp3EWqop67eJTCXaQqCnfxKIW7SFUU7uJRCneRqvhyICoWomMjXYlIjSjcRapSmAtx\nOg1SvEfhLlIVXy7EakhGvEfhLlIVn3ru4k0Kd5GqFOZCbEKkqxCpMYW7SFV8ORqWEU9SuItURROq\n4lEKd5Gq+PK0aJh4UlDhbmajzGydmaWb2V1VtPuZmTkzGxy6EkUiyKf7p4o3VRvuZhYNTAbOBXoD\nl5pZ70ratQBuAhaFukiRiNGEqnhUTBBthgDpzrlNAGY2E7gAWF2h3cPAY8AdIa1QGoV8XzELN+5j\n0aYf2Lg3h73ZBeAc7Zo3ITmxGUOS23LqiYk0axLMj2wI6Tx38ahgflM6A9vLfZ0BDC3fwMwGAl2d\nc++Z2RHD3cwmAZMAunXrVvNqpcHJOJDL/1uwmTeXZnAov4i46CiOb9+M9i2aYGbsysxn4cZ9/P3z\nzTSLi2Zs/85MGnk8yYl1ELjO6Tx38ayj7gaZWRTwJHBVdW2dc1OAKQCDBw92R3ts8a6cgiKe+c8G\nXly4Becco/p24uJBXRiS3Jb42OgftS0oKmbJ1gO8uXQHs5dl8Pri7Vw6pBu3n5NCq4QwrvlSVACu\nRBOq4knBhPsOoGu5r7sEHivVAugLzDczgGOAOWY21jm3OFSFSsOxZOsBbn1tOdt+yOWiAV249ewe\ndG595HHtJjHRnHJCIqeckMido1L4v/+k88rX2/ho9W4ev7gfI7q3D0+hvlz/R4W7eFAwZ8t8A3Q3\ns2QziwMuAeaUbnTOZTrnEp1zSc65JOArQMEulfrnl1sY/8KXFJc4Zk4cxl/Gp1UZ7BV1aBHPwxf2\n5c3rTqFZk2iu+PvXPPvJBpwLwxvBsuV+Fe7iPdWGu3OuCLgB+ABYA7zmnPvOzB4ys7HhLlAahuIS\nx/1vr+IPb3/H6T3a8/5NIxh6fLta7y+ta2veu3EEF/Y/lic+XM9NM5dTWFQSworxn+MO6rmLJwU1\n5u6cmwvMrfDY/Udoe/rRlyUNSVFxCbe8toJ3VnzPxBHJ3HVuL6Kj7Kj3Gx8bzV8n9KfHMS3487/X\nkV1QxHOXDTxszL7WfLpRh3iXrlCVsPIVl3DjzGW8s+J7fj+qJ/ee1zskwV7KzLj+9BN5ZFxf5q3b\nw69e/oZ8X3Fodl5YOuau89zFexTuEjbOOX4/ayVzv93Ffef14rrTTwjbsS4behyP/zyNL9L3c+OM\nZRQVh2CIpmxCVT138R6Fu4TN4x+s482lO7j1pz24dsTxYT/ezwd14YHze/Ph6t3c99aqo59kLQ13\nTaiKB9Xx5X7SWPzzq608N38jlw7pxu9+cmKdHffq4cnsyy5g8ryNHNs6gRvP7F77nRXqVEjxLoW7\nhNxXm/bz4JzvOLNnBx6+oA+B6x/qzO1np/D9wXye/Gg9vTq15Ke9O9ZuR6UTqgp38SANy0hIfX8w\nj99OX8px7Zry1CX9iYmu+x8xM+N/L0qlX5dW3PLqctL3ZNVuR4UalhHvUrhLyOT7ivnNv5ZQUFTC\nlCsG0yI+jEsDVCM+NprnLx9EfGwUE6ct4VC+r+Y70RWq4mEKdwmZh99dzcqMTJ4cn8aJHZpHuhyO\nbZ3Ac5cNYtsPudw7uxYTrL5ciImHqBCdNy9ShxTuEhL/XrWL6Yu2MWnk8Zzd55hIl1NmSHJbbv1p\nD95Z8T2vL86o2ZMLc9VrF89SuMtR25mZx11vriS1cytuPzsl0uUc5jenncApJ7TjgTnf1Wz83adw\nF+9SuMtRKS5x3PKqf12Xpy/pT1xM/fuRio4y/jqhPwlx0dzwyrLgr2AtzNFkqnhW/ftNFE95/tON\nfLXpBx4c24fj20d+nP1IOraM5y8Xp7F2VxZPfrQ+uCfp5tjiYQp3qbV1u7J46uP1nNfPf6ON+u6M\nnh34xdBuTF2wicVbfqj+Cb5cLRomnqVwl1opLnHcOWslLeJjeWhs3V+oVFv3jO5F59YJ3Pb6CnIL\ni6puXJijRcPEsxTuUisvfrGZFdsP8uDYPrRr3iTS5QSteZMYHv95Glv35/LY+2urbqwJVfEwhbvU\n2Nb9OTzx4TrO6tWB8/t1inQ5NXbyCe246pQkXv5yKwvT9x25oYZlxMMU7lIjzjnumvUtsVFRPHxh\nX88Mx1T0+1E9SU5sxh1vrCSn4AjDMzrPXTxM4S418uo32/ly037uHt2LTq28Ox6dEBfNn3/ejx0H\n83jiw3WVN/LlasxdPEvhLkHblZnPI++tYdjxbbnkpK6RLueonZTUliuGHcdLC7ewbNuBH28sKdGw\njHiawl2C4pzjvrdW4Ssp4dGL+hEVwlvlRdKdo1Lo2CKeu2Z9++MbbBfp5tjibQp3Ccq7K3fy8Zrd\n3PbTFJISG05vtkV8LP9zYV/W7c7i+U83/neDLxDu6rmLRyncpVo/5BTy4JzvSOvSiquHJ0W6nJA7\nq3dHxvTrxLOfpP937ZnC0ht1aMxdvEnhLtV6+N3VZOb5eOzn/SJy84268MD5fUiIi+b3s76lpMRp\nLXfxvIb5myohM2/tHmYv28H1Z5xIz2NaRrqcsGnfogl/GNObJVsPMH3R1nJ3YdKwjHiTwl2OKCvf\nx72zv6VHx+b89owTIl1O2P1sYGdGdE/k0ffXsu9A4OwZ9dzFoxTuckSP/XstOw/l89jP+tEkpuHf\njcjM+NO4VEoc/GvBGv+DWvJXPErhLpVatGk///pqG9cMT2ZAtzaRLqfOdG3blNvO7sGGjD3+B9Rz\nF49SuMth8n3F3PXmt3QLBF1jc/XwZFLa+X81DvhiIlyNSO0o3OUwT328gc37cnj0olSaxjW+cIuO\nMiakJQLw+Lwa3ndVpJ5QuMuPfJuRydQFm7jkpK6ccmJipMuJmI7x/lvxvbnqAP9ZszvC1YjUXFDh\nbmajzGydmaWb2V2VbL/VzFab2Uoz+4+ZHRf6UiXcCotKuOONFbRrFsfdo3tFupzICpznntShLffO\nXkVWvi/CBYnUTLXhbmbRwGTgXKA3cKmZ9a7QbBkw2DnXD3gD+HOoC5Xwe25+Omt3ZfGncam0SoiN\ndDmRVZgDsU159OL+7MnK59HqbuwhUs8E03MfAqQ75zY55wqBmcAF5Rs45+Y55wJXffAVUP9vqCk/\nsmbnIZ79JJ0L+x/LWb07RrqDulFqAAAOUUlEQVScyAvchal/19ZcMzyZ6Yu28dWm/ZGuSiRowYR7\nZ2B7ua8zAo8dya+A9yvbYGaTzGyxmS3eu3dv8FVKWBUVl3DnGytp3TSWB87vE+ly6gdfXtk57ree\n3YNubZty16yV5PuKI1yYSHBCOqFqZpcDg4HHK9vunJvinBvsnBvcvn37UB5ajsKUBZv4dkcmD13Q\nlzbN4iJdTv0QGJYBaBoXw6MXpbJlfy5//Xh9hAsTCU4w4b4DKH9nhi6Bx37EzM4C7gXGOucKQlOe\nhFv6niye+ngDo1OPYXSq9+6HGjYVbo59yomJXHJSV6Z+tokV2w9GsDCR4AQT7t8A3c0s2czigEuA\nOeUbmNkA4AX8wb4n9GVKOJQOxzSNi+aPY/tGupz6pfDwuzDdPboXHVrEc+tryzU8I/VeteHunCsC\nbgA+ANYArznnvjOzh8xsbKDZ40Bz4HUzW25mc46wO6lHnv90I0u3HeSPY/vQvkWTSJdTv/gOvzl2\nq4RYnrg4jY17c3T2jNR7QV1+6JybC8yt8Nj95T4/K8R1SZitzDjIUx9vYEy/ToxNOzbS5dQ/R7g5\n9qndE7nqlCReWriFs3p15NTujfdCL6nfdIVqI5RXWMzNry4nsXkTHrkwFbOGcT/UkKpkWKbU70f1\n5IT2zbjjjRVk5uniJqmfFO6N0KPvr2HT3hz+Mj6NVk0b+cVKR+LLOeKKkAlx0fx1Qn/2ZhXwwNur\n6rgwkeAo3BuZT9fv5eUvt3LN8GSGN+K1Y6pV7jz3yvTr0prf/aQ7by3/nreXH3bymEjEKdwbkd2H\n8rn11eX06NicO0elRLqc+qukGIryIbbqW+z99owTGHxcG+5581s278upo+JEgqNwbySKiku4ccYy\ncguLmfyLgcTHNvw7K9Va2c2xD59QLS8mOopnLh1AbEwU109fqtMjpV5RuDcST/9nA4s2/8D/XNiX\n7h1bRLqc+q3s5tjV34Xp2NYJPDk+jTU7D/Hwu6vDXJhI8BTujcBn6/fy7Lx0Lh7UhZ8N0ppu1fIF\nhliqGZYp9ZOeHfn1yOOZvmgb76z4PoyFiQRP4d7AZRzI5eZXl9O9Q3MeukBXoQbFl+f/WIObY99+\nTgoDu7Xmrlkr2bA7K0yFiQRP4d6A5RYWMXHaEnxFJTx32SAS4jTOHpTSYZka3Bw7NjqKyZcNJCEu\nhmunLeZgbmGYihMJjsK9gXLOcfvrK1i76xDP/GIAJ3ZoHumSvKNsWCb4cAfo1CqBF64YxM6D+dzw\nyjKKikvCUJxIcBTuDdT/fZLO3G93cfe5PTkjpUOky/GWGkyoVjTouDb8z7i+fJ6+j0fmrglxYSLB\na3y3tm8EZi/L4MmP1jNuQGcmjjg+0uV4T9mpkMFNqFY0fnBX1u7M4h9fbKZb26ZcPTw5hMWJBEfh\n3sB8un4vd7y+kpOPb8ejP9O6MbUS5HnuVbn3vF58fzCPh95dTfsWTRjTT4uzSd3SsEwDsjLjINf9\nawndO7bghSsH0SRGE6i1UjYsU7ueO0B0lPHUJf056bi23PrqCham7wtRcSLBUbg3EBt2Z3H1i9/Q\ntlkcL199Ei3jtSBYrdVyQrWi+Nhopl45mKTEpkz65xKWbTsQguJEgqNwbwA27M7i0qlfER1l/PNX\nQ+nQMj7SJXmbLw8sCmKO/gYmrZrG8vI1Q2jXPI4r//41SxXwUkcU7h5XGuxRZsyYNIzkxNoPJUhA\nYeAuTCGar+jUKoGZk4bRtnkcv1TASx1RuHvYsm0HmDDlv8F+Qnudyx4SVazlXlulAV/ag1+4UWPw\nEl4Kd4/6ZO1ufjF1Ec2bxPDar09WsIdSYW6tznGvjj/gT+bY1vH88h9fM0fr0EgYKdw9aPqirUyc\ntoQTOjRj1nWnkKShmNDy5db6HPfqHNMqntd/cwoDu7XhxhnLeP7TjTjnwnIsadwU7h5SUFTM3W+u\n5N7Zqzj1xERmTjqZ9i2OftJPKvCFp+deqlVCLNN+NYQx/Trx6Ptr+d2MZeQWFoXteNI46SImj9hx\nMI8bXlnKsm0Huf70E7jt7BSio3SBUlgU5h7VBUzBaBITzf9dOoDex7bkiQ/WsWF3Nn+7fCDHa3hN\nQkQ993rOOcfsZRmM+utnrN+VxXOXDeTOUT0V7OHkywnbsEx5Zsb1p5/ItGuGsicrn/Oe+Zx/frVV\nwzQSEgr3emxPln91wVteXUHKMS14/6aRjE7tFOmyGr4wTageyandE3n/ppEMTmrDH95axS9f/IYd\nB/Pq7PjSMCnc66Gi4hJe/GIzZz7xKR+t3s0d56Tw6q9Pplu7ugucRs2XF/JTIatzTKt4pl0zhIcv\n6MPXm/dz1l8+ZfK8dAqKdF9WqR2Nudcjzjk+XL2bJz9cz7rdWYzonshDF/TVhUl1LQznuQfDzLji\n5CTO6NmBh99dzeMfrOONJRnc8tMejEntRJSG4qQGFO71QEmJ49P1e/nrx+tZmZHJ8YnN+NtlAxnV\n9xit6hgJdTwsU1GXNk154YrBzF+3h/+du5YbZyzj2U82cPNZPTinzzGab5GgKNwjKKegiFlLM3hp\n4RY27c2hc+sE/vzzflw0oDMx0Roxi4hiH5T46mRCtTqnp3RgZPf2vPftTp76eD3XT19K17YJXDHs\nOCYM7karplocTo5M4V7HiopLWJC+j7eX7eDD1bvJLSwmrUsrnprQn9GpnYiLUahHlK/2d2EKh6go\n4/y0Yxmd2okPvtvFSwu38Ke5a/nLh+s5q3dHxqYdy+kp7bW8sxxG4V4HDuQU8tmGvXy6bi/z1+/l\nh5xCWsbHcEH/Y7l4cFcGdmsT6RKlVOHR36gjHKKjjNGpnRid2onV3x9ixtfbeO/bnby3cict4mM4\nI6UDI3u0Z2T3RK0KKoDCPeSKSxyb92WzbNtBlm0/yLJtB1m76xDOQZumsYzo3p4x/Tpxmnpb9VN+\npv9jk5aRraMKvY9tycMX9uX+83vzRfo+3lmxk0/X7ylbq6ZHx+YM7NaG/l1b079ba05s31zDfI1Q\nUOFuZqOAp4Fo4P855x6tsL0JMA0YBOwHJjjntoS21PqjoKiY3ZkF7MzMY9ehfDbvyyF9Tzbpe7LZ\nvC+HgiL/Xe9bxMfQv2trbj6zB6eltCe1cytNhtV3uYHVGpslRraOIMRGR3F6SgdOT+lASYlj9c5D\nfLZhL4s2/cD7q3Yx85vtAMRFR5GU2JQTOzTnxPbNSUpsRqdWCXRqFc8xreKJj1UnoyGqNtzNLBqY\nDPwUyAC+MbM5zrnV5Zr9CjjgnDvRzC4BHgMmhKPg8pxzlDh/b7nE+f/5P/efgVLiHMXOUVJC2Tbn\noKikhHxfCflFxRSUfSz2P+YrJt9XTHZBEZl5Pg7m+vwf83wcyvOxL7uAfdmFFf4fQZc2CZzYvjkj\nuifSvWMLBnZrzfGJzXX6mtfk7vd/bFr/w728qCijb+dW9O3ciutP9/9ubN6Xw/LtB1m3O4uNe7JZ\n/f0h/r1qFyUVLoBt2yyOxOZxtE6Io1XTWFonxNK6aSytEmJpGhdDQlw0CbHRxMdGEx8bRUJsNAlx\n0cTFRBETZURHlX60ch+jiI72fx1lRumvgZlh/HepfJ0NFj7B9NyHAOnOuU0AZjYTuAAoH+4XAA8G\nPn8DeNbMzIXhOuovpz9M6vpnj7jd8L+9CEVfxAL/MQwzyn4orVnpD6j/h9bMsCJgZ+CfeFexz//R\nAz33qpgZx7dvfthaNfm+Yr4/mMfOzHx2ZuazKzOP7zPz2Z9dQGaej+0/5LIq0KnJ89XdBVRlYc9/\n/wCUPm4Efvko9zsYeMCrfxseOL83E07qFtZjBBPunYHt5b7OAIYeqY1zrsjMMoF2wI/uSGBmk4BJ\ngS+zzWxdbYqOsEQqvK5GoLG95kT+2KkxvV5ofN9jiOBrvuRhuKT2Tz8umEZ1OqHqnJsCTKnLY4aa\nmS12zg2OdB11qbG95sb2ekGvuSEKZgp9B9C13NddAo9V2sbMYoBW+CdWRUQkAoIJ92+A7maWbGZx\n+N9NzKnQZg7wy8DnPwc+Ccd4u4iIBKfaYZnAGPoNwAf45yn/4Zz7zsweAhY75+YAfwf+aWbpwA8c\n1XBSvefpYaVaamyvubG9XtBrbnBMHWwRkYZHl62JiDRACncRkQZI4X4UzOw2M3Nm5u0rXqphZo+b\n2VozW2lms82sdaRrChczG2Vm68ws3czuinQ94WZmXc1snpmtNrPvzOymSNdUV8ws2syWmdm7ka4l\nHBTutWRmXYGzgW2RrqUOfAT0dc71A9YDd0e4nrAot9TGuUBv4FIz6x3ZqsKuCLjNOdcbGAb8thG8\n5lI3AWsiXUS4KNxr76/AnUCDn5F2zn3onCsKfPkV/msdGqKypTacc4VA6VIbDZZzbqdzbmng8yz8\nYdc5slWFn5l1Ac4D/l+kawkXhXstmNkFwA7n3IpI1xIB1wDvR7qIMKlsqY0GH3SlzCwJGAAsimwl\ndeIp/J2zkkgXEi5az/0IzOxj4JhKNt0L3IN/SKbBqOr1OufeDrS5F//b+Ol1WZuEn5k1B2YBNzvn\nDkW6nnAyszHAHufcEjM7PdL1hIvC/Qicc2dV9riZpQLJwIrAcqVdgKVmNsQ5t6sOSwypI73eUmZ2\nFTAGOLMBX30czFIbDY6ZxeIP9unOuTcjXU8dGA6MNbPRQDzQ0sz+5Zy7PMJ1hZQuYjpKZrYFGOyc\na7Ar6gVu1vIkcJpzbm+k6wmXwLpI64Ez8Yf6N8AvnHPfRbSwMDJ/D+Vl4Afn3M2RrqeuBXrutzvn\nxkS6llDTmLsE41mgBfCRmS03s+cjXVA4BCaNS5faWAO81pCDPWA4cAXwk8D3dnmgRysep567iEgD\npJ67iEgDpHAXEWmAFO4iIg2Qwl1EpAFSuIuINEAKdxGRBkjhLiLSAP1/MG49ZFtWmvcAAAAASUVO\nRK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sw-dhDzyiiT7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lh=np.zeros(1000)\n",
        "for i in range(1000):\n",
        "    d=(np.random.random(M)-0.5) * 10.0 \n",
        "    labels=norm.pdf(d,loc=mu,scale=sigma)\n",
        "    lh[i],_=sess.run([loss,optimizer], \n",
        "                     {input_node: np.reshape(d,(M,1)), \n",
        "                      train_labels: np.reshape(labels,(M,1))})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5m1C9Ct9iiT9",
        "colab_type": "code",
        "outputId": "c2f71889-3b5b-4244-fa17-299f02d8ee12",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        }
      },
      "source": [
        "# training loss\n",
        "plt.plot(lh)\n",
        "plt.title('Training Loss')"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Training Loss')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXm8FMW1x39n5i7si2wqixcBBdxA\nEfcVd41ogi8uifo0UaMmvizmkbjEGDUaE42JRmPUxBiT6DNqjLjF3aAgKCAgoKwComyyb3c574/u\nnqnuqaqu7umemXunvp+PcqeX6uru6jp1ljpFzAyLxWKxWDLlroDFYrFYKgMrECwWi8UCwAoEi8Vi\nsbhYgWCxWCwWAFYgWCwWi8XFCgSLxWKxALACwVKlEFGWiDYR0YAkj7VYWjNk5yFYWgNEtEn42QHA\ndgDN7u9LmfnR0teqeIjoJgD9mPnCctfFYqkpdwUsFhOYuZP3NxEtBvANZn5ZdTwR1TBzUynqZrG0\nFazJyNImIKKbiOgxIvobEW0E8DUiOoSIJhHROiJaQUS/IaJa9/gaImIianB//8Xd/zwRbSSid4ho\nYNRj3f0nE9FHRLSeiH5LRBOJ6MIY97QXEb3h1n8mEZ0q7DuNiOa4119GRN91t/cmoufcc9YS0Ztx\nn6ml+rACwdKWOBPAXwF0BfAYgCYAVwHoCeAwACcBuFRz/rkArgOwE4BPAPws6rFE1BvA4wCudq+7\nCMDoqDdCRHUAngUwAUAvAN8F8BgRDXYP+SOAi5m5M4B9Abzhbr8awEL3nJ0BXBv12pbqxQoES1vi\nP8z8L2ZuYeatzDyFmSczcxMzLwRwP4CjNOc/wcxTmbkRwKMARsQ49jQA05n5n+6+OwGsjnEvhwGo\nA3A7Mze65rHnAZzt7m8EMJyIOjPzWmZ+X9i+K4ABzLyDma2GYDHGCgRLW2Kp+IOIhhLRBCL6jIg2\nALgRzqhdxWfC31sAdFIdqDl2V7Ee7ERtLDOoe5BdAXzC/qiPJQD6un+fCeB0AJ8Q0etEdJC7/Vb3\nuFeIaAERXR3j2pYqxQoES1siGDL3ewCzAAxm5i4ArgdAKddhBYB+3g8iIuQ78Sh8CqC/e77HAADL\nAcDVfE4H0BuOaenv7vYNzPxdZm4AcAaA/yUinVZkseSwAsHSlukMYD2AzUQ0DHr/QVI8C2B/IvoS\nEdXA8WH0CjknS0TthP/qAbwNxwfyfSKqJaJjAZwCx4/QnojOJaIurllqI4AWAHCvO8gVJOvhhOa2\npHOrlraGFQiWtsz3AVwAp8P8PRxHc6ow8+cAvgrgDgBrAAwCMA3OvAkVXwOwVfhvHjNvB/AlAGPh\n+CB+A+BcZv7YPecCAEtcU9jFbhkAsCeAVwFsAjARwF3M/FZiN2hp09iJaRZLihBRFo75Z5ztmC2V\njtUQLJaEIaKTiKiba/q5Dk7kz7tlrpbFEooVCBZL8hwOZy7AKgAnAjjTNQFZLBWNNRlZLBaLBYDV\nECwWi8Xi0qqS2/Xs2ZMbGhrKXQ2LxWJpVbz33nurmTks/Ll1CYSGhgZMnTq13NWwWCyWVgURLTE5\nzpqMLBaLxQLACgSLxWKxuFiBYLFYLBYAViBYLBaLxcUKBIvFYrEAsALBYrFYLC5WIFgsFosFgKFA\ncJN1zSOi+UQ0XrK/3l3gfD4RTfYWIxf2DyCiTUT0A9MyLRaLpVqZungt5n22seTXDRUIbvreewCc\nDGA4gHOIaHjgsIsBfMHMg+GsIXtbYP8dcNaDjVKmxWKxVCXj7nsHJ/669Mthm2gIowHMZ+aFzLwD\nzlJ9YwPHjAXwsPv3EwDGeEv/EdEZABYBmB2xTIvFYrGUEBOB0Bf+xcuXoXCN2NwxzNwEZ+m+HkTU\nCcD/AvhpjDItFovFUkLSdirfAOBOZt4UtwAiuoSIphLR1FWrViVXM4vFYrH4MBEIywH0F373c7dJ\nj3EXFu8KZz3ZgwD8gogWA/gfAD8moisNywQAMPP9zDyKmUf16hWarM9iqWoWr96M+95YUO5qWFop\nJtlOpwAYQkQD4XTaZwM4N3DMM3AW/X4HwDgAr7Kz8s4R3gFEdAOATcx8tys0wsq0WCwROe+ByVi+\nbivOPrA/unWoK3d1LK2MUIHAzE3uqP5FAFkADzHzbCK6EcBUZn4GwIMAHiGi+QDWwungI5dZ5L1Y\nLFXP5h1NAAC7EKIlDkbrITDzcwCeC2y7Xvh7G4CzQsq4IaxMi8WSDFYeWOJgZypbLG0IKncFLK0a\nKxAsFovFAsAKBIvFYrG4WIFgsbRB2HqVLTGwAsFiaUO4GWMsllhYgWCxWCwWAFYgWCwWi8XFCgSL\nxWKxALACwWJpk1iXsiUOViBYYtHU3II/vLkQ2xqby10Vi4B1KVuKwQoESyyeeG8Zbn5uDn732vxy\nV8UiwUadWuJgBYIlFpt3OJrBxu1NZa6JxWJJCisQLEVhR6IWS9vBCgRLLKytujKx89IsxWAFgsXS\nBmEbZ2SJgRUIlliUeyS6aXsTvvf4dKzf0ljeilQqVh5YYmAFgqUoypVE7c/vLMaT7y/H795om1FO\nG7Y14ox7JmLBqk3lroqlirACocL414xPMX3punJXIxRPQSj3QJTaqDfjtbkrMX3pOvz65Y9jnV/u\n92JpnViBUGF8+2/TcMY9ExMv96H/LEpU0NismpWK815s9Fc425uasXLjtnJXo6KwAqFKuPHZD1MR\nNOXqeKqlw4srdq1TOZwrHp2G0Te/Uu5qVBRWIFhiUSkKQqXUw9L6eHnO5+WuQsVhBYKlKOxINB2K\n1YCqRYOyJIsVCJZY2IF5aYirAVl5UJl8/PlG/GvGp+WuhhIrEAz5dN1WPD5labmrUXGUeyRqBZMf\nT4DYNZUrk+PvfBPf/tu0cldDSU25K9BaOO+ByVi0ejNO3mdndG5XW+7qlB9rvK9orDywxMFqCIas\n3GDD02SUq99p6yNg65uxlAMrEAxpdjugbMaOjIHKMdW0VUXFk3dt9PYqirY+uIiCFQiGtLhtppxt\nZ+WGbdi4rbJy99hvqbLIzSC376UsLF69GX+ZtKTc1YiNFQiGtLSU9guTLU05+pZXMOZXb5S0Hiry\nI3Pb81Qi1uRkTpLC8yv3vo1rn56FpuaW5AotIVYgGNLkCoRSfWYX/vFd6faVG7eXqAZ6Sp1D6LV5\nK7FlR351tmoZAUdJEdLY3JJrH7Ln8/mGbWgYPwEvf2gnZKXFF1t2AGi9qV2sQIhIqeyNkxauLcl1\niqUUj2P+yo347z9OwTVPzSrY11aT28V5rrc9P1e7/4Nl6wEAf3v3kzhVarOk0YRbq1/CCoSIVOpr\nXrR6M1YptIc0Gqc3APr7lKWpm9M2bnM0g4WrN6d6nUokirjzOnygctuppbKxAiEilSr4j/nl6zj4\n5/JEXWm7P56duSLV8mXVr9DXUDG01hFqOUjjWbXWp28FQhuiWdHzp6IhCH9vFWz7paaVmmpDKfaN\n6c5vq88sLumYjFIotARYgRCVVviiW2GVfdj+KzqyDslqDelj+oQr9V0YCQQiOomI5hHRfCIaL9lf\nT0SPufsnE1GDu300EU13/5tBRGcK5ywmopnuvqlJ3VDatMZwvpYUfQjlokK/p7Ji3jatiBVJoy2F\nvYsSR7EbEyoQiCgL4B4AJwMYDuAcIhoeOOxiAF8w82AAdwK4zd0+C8AoZh4B4CQAvyciMX/SMcw8\ngplHFXkfJSOpxrO9qRnDrnuhJJkP0+4804700Zo/Ur1y+eCipyoXPrUK7YPaFKYTA9MYpCWBiYYw\nGsB8Zl7IzDsA/B3A2MAxYwE87P79BIAxRETMvIWZPQNzO5SpTTIznpu5Qmljj1RWAvUBgJUbtmNr\nYzNue0EfKlip6ITApu1N0ol1pWRbYzMembSk5BMKkyausNX1N+XW7iqNJLV+c5NRYpdMFBOB0BeA\nmPd5mbtNeowrANYD6AEARHQQEc0GMBPAZYKAYAAvEdF7RHSJ6uJEdAkRTSWiqatWrTK5pwKemrYc\nlz/6Pv44cZHymC07mtAwfgKeEyJmpn3yBT7+fKPvuKRsf43uTMbabPpunFKPRvb+yYs48hevRTpn\ne1Mz5qzYIN0n67/CPuJfvTQP1z09Cy/M/ixSPUxYunYLHnhrYeLlFov4mqWRWRXaCbUGDrrlZVzx\n1/cTK681awhFwcyTmXkvAAcC+BERtXN3Hc7M+8MxRV1BREcqzr+fmUcx86hevXrFqoMXn/+5JmPp\nsi+2AgDu+PdHuW1n/u5tHH/nm7GuGUZjs9MgarPpD9dSaXsh1Y46o/r6p2fj5Lvekr4jbfUVw921\nm52cT5u3Jx8B9bUHJ+OmCXOwdvMOo+PPe2AS7nltfuL1CCI+J62GkHpNWhcm38fnG7Zjwgfm4dVh\nZVaoPDASCMsB9Bd+93O3SY9xfQRdAawRD2DmOQA2Adjb/b3c/XclgKfgmKZSwXv2GY2ubPqRJPUe\nPQ2hJpO+hpB620ugh5m6xJmZrUveVykdmTdRzlRbnDh/DW5/cZ5v26tzP8eQa57DJoXAqtD+wmJI\nmAZbqcEpJr3RFABDiGggEdUBOBvAM4FjngFwgfv3OACvMjO759QAABHtBmAogMVE1JGIOrvbOwI4\nAY4DOhU89SyJ/CJJSXYvN1IpNIQ0RslJ1zrpz6NSPziPO//9MRqbGQtWbtIeF9Zk56zYgFnL1xds\nl99/ZT+TaqJSXVuhAsG1+V8J4EUAcwA8zsyziehGIjrdPexBAD2IaD6A7wHwQlMPBzCDiKbD0QIu\nZ+bVAPoA+A8RzQDwLoAJzPxCkjfmvwfn3ySWMkiqoynGh/DCrGh28XPunxT5GmGkl7xLXa6pSUQ8\nuFxJxhrGT8CPnpyp3J9xG2OxtuST73oLp/32PwD8GktaTuX3lnyB9VtKk4L97fmr0TB+AmZ/Wijw\npi5em1jAQCphp204ygjM/Bwz78HMg5j5Znfb9cz8jPv3NmY+i5kHM/NoZl7obn+EmfdyQ0v3Z+an\n3e0LmXk/97+9vDLTwms4ifQNCb3HHU3xBcJlf3kPkxeuCT/QJakcQItXb8YrcyorU6bqlXLI/iQI\nawq6JHLe4ETZpxXZzuQT04otk/GVe9/G1x6c7Nu+fN3WVKK5nncHPlMXf+Hb/vb81Rh33zu4780F\niV8zKULHKxWaHbsqZiqb+BBKjScQamKajNZvLd1COcyMO16ah6N/+TouftiZQyjWulRPNc510njl\nSRTptcWwkWLca+k02WJDWWcKJqrFqzfjsFtfxd0pOM29b6Suxt9NfbreCTyYH2JuMyWq1r+9qTlX\nt1Jds1RUhUCI4kMIcxQm9Rq3F6EhJFkPE+Z+thG/eTX6Bz/tky/CD/KIeEOhI7AKVck9sm5bTGJu\njClp5Eda4XbOE+evLrL0Qna4ZtX6GsU3UoZX/PKHn2PPa1/A0bfrw6rD2l+r9SG0BbyHrxMHpiPJ\nJGcqA0BNBa3R/MMnZuCW5+YUbJeNYk2e19PTgsFo8SjGHl5BSqEPypmMGI3NLQUmlzgjSOOw05jP\nRNfJqfb8+8PPMWVxvLU9vG8kqCEkHtAQ4VFPW+oMcjwtRVlmSDmt2ofQ6slpCAaHhu5P5kU2ufMQ\nshUkEB6fugz3v2k24cqkPUdy6EZ9DCXS5NLCe+8tLcCQa57HD/5vhvS4JAVa0T4Eybaw+n3zz1Nx\n1n3vxLpemJ+tEt+x6TOuUHlQHQLBzIdg9uWFvchN25uw1/Uv4I2P9LOqvWJK3TAOvPnlyOfIbM5i\ntdOO5BGLX7dlB+57Y4Gg9cmv7T3XlRu2KxcOKifZQJTRkwFtqngHcHHnl6pMHdsVPoSkm1satxU+\nMa0yJUJVCATvoyvFYHz+yk3YvKMZd7w0T3uc1yBK7VxKonNk5uQbtGFxP3pyJm59fi4mL3KirMI6\nh58/PzeWEEwbT4g2J/gc/akrkm9XsjIpvzNxvNDs+hKkd0mKXHsMDTtNvSqxaD1Pughyo8kkJqaF\n7PeuEPbCZRpClE62nAOMFi6tui7eqzdL2Ev9oTwnpbo8PnUp1rgpK4p5B15wWTlGivF9CLKynMLe\nXbwWM5auK6JWhXgmI9V3m9SzS2fFtDCncmVKhCoRCBHmIRSp6nnXCB2hceHlKrSNFDy35oBESFLx\nMnbuS7Zta2zGSykksxP54RMfCHWI/8JyYaeK6MX8PArzpyvW5v+mLiuIYEpbGx17z8RIx5/3wCT8\n8Am57wTIRxkF611Ok1HY+zD2IUS4ZimpCoHgPX1tLqMioozufvVjNIyfgEcnL8FNE+YojxPxhFRY\nhspyEBR6MoFg0rkU8+F++OmGXAptsRzSjKxvnjAHlzzyHt5b8kVpRt6KS7wwKzwJWpjJqNjqPzJp\nCf78zmL5tRNMqV3MO544fw0en7pMub+xSTJqEusT/9KJ0DB+Aj78VJ6hN/T7r1CbUVUIhLR9CA9N\nXAwAuOapWXh3kRNiF+pUkv0VqRdIr0GFtdVm5tgd1gfL1mHBKv2EotWbtuOU37yF8f9wRuOmmTuX\nfbEFALB+646SdBaqa1z2l/A0yZ5ZXPXORYG7ePVm3PbC3PD2Edi/LpBiovgoo9J2Yt71gldNekGm\nYp5LcP5F3kIQ7Zqfb9iGZz9If7GsMKpEIDj/JtGQkhp4euW8Pm8VPljm2F4rZcwQ1vE0N8fvGk6/\neyLG/OqNwmsKf29y/QTvf+K3SYfPOdCbYZKmmLaQn6ks35/3ewEXPTwF976+AJ+s3RLpGsqQ5iR9\nCPGKin29SiPoCzCtc/C8c/8wCVf+dVrZF5aqCoHAwsdVsmuG7neOaGphnH63Y3uN9gGkdzNhI7Kg\nhqB6rnEEMAnX170vXefEuf+lSxI+BOVMZeEGPedq1NQrQYFQ7CORnZ/mNyULvPDtT+odF1HOSx/K\nc3uFDarWBNbS8NZjKbcQrA6BAM+pbJC6wrCs0HLCJk5JdkfrYNI0GYVoCCn6EMRSg6cz599h/p2K\n13P3GXxVzFz0aEy8TNTcNmHZTkUNIW4nEef5r9/SiJue/TAX8ikiPtdSJDksVWj2+q2NWL3JLBw7\n+EzfWyJPzxJW43H3vS09vtwz66tDIOScyupjjBfIMY0iiBGtJDvn+Zkr8FFgGU+Pf05fLs2FXyxh\ndW9uie9DMLl2XIewmEE0rBN5aOJiDL3uhYIV2pau3YKx90zEFwaroXlXmLNiA/a49vlIacm9uqpu\nVRQU3vMI6yyCRcVJ5vjz5+fggf8sktqzxfK9JIelSG2oekZJdZ5H3v4aRt2UzFwVCnmvHsH9OeFn\nNYT0yTuVS5fcLnzFJDO+9ej7OEGyjCczcNXfp+dy4SdJ8BEURBlxMmO2WcvX47qnZwUmuuVLDmp0\n4k9dxAsbOL29Ds9T1T3ue2MBZixdZ+Tg8+rs+YCijJrDTEay6LOoHfzjU5aiYfwEbAisQqcrxQv1\nlCgIkaKMwgIHoqB6leXuPGXErZN3XrmzoFaZQFAfY9zRG77xqCMEk3PS5Hev57OZFsR9B45tbmZf\nZcVOIYrp5Nw/TMIjk5Zgw9b8im7M+ndBwnHOtUnY55mT4pMh8zKCdYhy3ZxAUJqMuODvqCNibx2M\nFescLajoUFzD05+buQJjfvUGXixyTkjehxBoj5WT/ktJ1I7dO7rc0ahVIhDcPzQtqdSZGKQmozKN\nDrY1NuMXL+RTbRiFnSr2jbrp37m/43y3jPhBAJlcKGfxdvcoceJBIWWCF3aqus6Emfm5DLnnkdCk\nqLgz9mVvXdaO56xwYvPnfSY3dUa/brTtaWL85CJWLj8vyWoIqWPiQzB9g0lpErK9laICG01MU9R1\nwzbz9ZvVz8AdEUc81+sw46YFaGlh/HXyJ8ryC+oQtANH6AXCNIRpuZBbKvuo0aNsWm2F3D+ASKMU\nZs6l8A4/1v03Tp0SpEoEgtfBFL7MLzbv8C3kEj6yN7ymZNv0peswyV36UvaRp9EY4qRyCOuAJi9a\nYzSSiRdlZOawll1fnBRk2jk/NuUTvL3AmVz09PTlaGoxd+4VRK9FeIFelNHCVfnlTe99fYFCGDvb\ndIIuikYzaeEaNIyfkJvIZ0rwCsvXbU1EWH35dwonvlv2RQ9PwXtL4q2pUC4YwB3//gh7XvsC1m0J\nD1DInddSXi2hSgSCet9//f4dnPm7tyOMdOL7EM64ZyLOdhe8l0cZJd8QrhZy76gouGxINa55ahZm\nLMtHNyUy4c/999JH3sM4N3++zLShEzI5+3+EmdSPT12Gc//grBG8QViWNJIPIcI5Hp62+uB/FuW2\n3fbCXOnCK16n65V/6/NzcdfLH/uOMcma6h3iZbz1ZtWbEmyfh936aiJt9v1P1uGf0wsXU8r7EJwA\nirD6VBI/e/ZD/OntxQCAETf+Gzf+60Oj86xTuQTk7HOSh/2xuy6ruSnI9LjoLzbKGUk6wYPPpXAk\nWtgLP5XQamjB4pes2SJZL7rwHuZK7NPiKmRxEM8ye27+60Z551mFZJON9PMhic6/972xAHe+/JHv\nGNO5Ib5rSW126uNlu4Jl3P1qXlDJ5jKokAl/8Xku+2JrzjeR9vobUTnw5pdx6SNTfdue/WBFLjMv\nADw0cVHwNCnF+L+SoCoEgokHv/RO5eTqIH44j05eEvn84HMpEAcxv7/gh1swZ4Lz/8ouYbotuC8s\nUkmF+ByimK3iPJ8dis6yIOQX/pGyihaNVqQSFFETrMnbrH/jHf/OC6rfRliH2+QZnnzXW8blFcPS\ntVuwZM3m8ANdVm3cjhdnJzNRj1FeP0J1CASDL8p0dLXScIGZ0LBTyfU8h2ZUnhbU7WuemhW4jpqt\nO5qxrbG54KNOK1f7VX+fJt2uevb5TkIILdXMTcjPVI5XP/E5mLSHiJY2H326tFOU6ZTSt1t7AM6a\n2y0Kv0bD+Am5v5UpMDRI37M2NFuivQR+19dkI9cjLqo73rqjGbe9MDf2TPQjfvEajrr99fx1hOf0\nkGDiS4Nyr5NQFQLBQ/eoc17+3L/5o0UTxnkPTDa8lv7Fyr7f216Ya1R2kGsDQuDku97CdG+xEk01\n9rnhRez705cKNYSQ30FMR8gFa+P6ZhYXj2gyivpdNYyfgLteyZs7zDQE97pQCyKVGclbFnJI706+\n7V7H3r7O6VjFEaOuTbW06Os8f+VG/M9jfjt85NXaJIcHO7D62nhdiqwJqWq3coN+gfv73liAe19f\ngL9Miq4tB7nntfmYKWi2m7abR9HFwTEZlU8oVJdAMDAZeR+d2Enu99OXEr2WyX7nmHgNY86KDbjp\n2XAnVlMLOxPJCgRAMWPfPMGPvGCmrUT4inz0+SZMWbzW+Ppe+U5HEL3Oos3XbMDtNxlJTfIqM467\n3fNheXhRTl5HKwo3XXPQde7MwE+emV2wPapSYXJ/9TXyLuWe1+bj2qdnRrugAm/NEdUr9jSDpgRC\noG5/cV4u+WSS9O5cL93+ydotPgFUaqpKIATzw4gdUWEa2+IaUxyTURBTn4fc2cfKfapjVeXFfhSB\n/l+VjllX/Fn3vYPHp+QXUQmWIEY4eX/NWLYen2lGkQtXbcL6LUHHdbBOBiYjT0PQJNWL+ug8R6wo\nBJj9QkJeF71AkKW9SMKHUKAhKExGt784D3+ZlDeJ3hAUUAk6isXMBCs3bMOXfvuf1E09poweuBMA\nYMywPtL9X7n3bZz5u7el+0pBTdmuXAbEtjvwR8/hq6P6534HG3aUbyXWjFyD8kW7cNgHX7AtQl0K\n7z26BHh7wWqMbthJe4woEN5ZsAYbXfU77HLTlsozSgYRO70mzZrLx0rWYwhiNg9B/xuI/iy9eosC\nPW8yMq+LSR1WrN+W80N061CL6defEBJlFC7w6hQaQhAvJNNDajKKORDx/PUZIoy+5RUAwMzl63HR\n4QPjFZgkXPBHRVFVGkLwFTw2dWnu76BTzvRDXrp2S0FucyAZO6BYB12yMNmHmotdjxA+mfsdNCGF\nnD9p4Rqc+4fJ+N3rC3zb739zoe+3KBB++VI+VUbYvIHgyl8qwpLfJU1wHoLsQalNRvIdr81bCcCf\nykC23GqQFlbrNC3M0lDN191rAYXP2LSDLghIiGmmIQI2bmv0OYLDE0TK9+dzP1VWeCqQr3OlTqGo\nLoEQiOcWiWtuHP+kfOJXWHEmH45YzePu8Gc8Df1YijAZRV0FyssYumSNf+Zr8DxRIIj23bA6rnOd\n+tIwVJL/XWy0RpT5G9513/+kUJOJOjfg1+6EM2/VNzEuXVsnrbCQp21JwKeMu17xh5Zu2RF/jYl9\nbngJJ/06386L7TSLXTI3DeeuiT+onFSVQLhpwhxsb2qWhugF85GbdijZjPwRhvsQwjGtg3zkZlYP\nWWWiNlbPzFGb1X+B4mSsZmGdy7DriRlUdYM+cUQYOYImgMkAIRhltGL9toI1EcKcyurr59tjEiYj\neRRP4VkfuA5NWXmyDnLGUv8yp9061Gpqoua1uasAAIvXREunISNKuvsg4mS6NDrt/LusTIlQVQIB\nAJ6dsUIafRDcpOsQHvzPIjSMn4CZy9bjzY9WSY/RvfCG8ROMOnvdMaIzVXZclBFyaNhpSOP1PqKC\nsNIANVm5jd9Y8IXs940IAwfPX7mxYF0A7bWE88PWLBD7HdViRgXnhuwXZ9ebLJ4S5nCWdY6yU+av\n1JgmDV7TkXv0Cj9IwssxVmBT1cd7X3E0hOYImmscKmUhHBVVIRDEDo1IPqVe/KAam1u0k1rucO3f\nf3x7kfKYsBGmiclIt1i8eE+yDiuf/yb8OsE1DILnhDXexhZPQ9A3pwN26w4A6Fxf44/4Cq2hGvL9\nrRaSx93xJv7rvneMbdzeM/jXjE8x6MfPYaHEhyN7tkGTSdwP36tmi2gy0jwpMRopSHOL3J6exiSo\n+99cmEi5kxeuKVi8yBTv2WViSATZSnVJYqLtlZOqEAgi2QzlRqfiN+IlnQOAsXdP1C6pt9n96Af1\n6qQ8JqwtmcRI60d94nGy/eYjkWCK3qj+lMYmT0MwMxn126mD34fAhms0C/+XIX7/stLmfrbRODbd\ne27Pz3LWJZizonDkX+BUBrBlh3/ikuq+Qk2KEpORfoCgpoVZ7kPQV0FSJ7Pj8qm74/NV4XtUwQys\n3rQdT01bFtjuft8x4v/Ee9wo3J3/AAAgAElEQVS8vbg1t3XlWw2hjIgNY+5nG9Hoflk1ihHEh24S\nrTD0HaD+jZukGtCmOw5pUflRZThBe3twZBSqIRiajFqEDtQfUmtQSQPEUbDq+ZimeIhSJfG6wU5E\nVo0XZq3A/723tHCHgKjh6ZIz5o7X3JcTZSTZobnJZon0MbV7l7Kz+8bDU/Hdx2Zg5cb8vBOTFRJV\niO3mS3ensDyt92+FSoSqEAhiQ7739QU5DUE1USqRayaiIRRx/Qhd2oQPVvh+Fya7MxNuNSEagliO\n6EOI8m2Y+glVo+lG3TBbwOSDlS1tubUxqCEUctlf3g81h/icykWOKlta5A5WXXH/+4+Z2Ly9CR8s\nW4fX5q4s6vpp8uk65zmKr1VlMmoYPwGPvLNYW57Y9j9ZW7yDu4AI0X/lwEggENFJRDSPiOYT0XjJ\n/noieszdP5mIGtzto4louvvfDCI607TMNPFs5rWSCKEojV537JrNO7RrypqMVHWdUpiGsMhdT9ek\n5RVmpYzWXE1txuIt+30I5tcrmKksbBA7PdWz001YEzG5JbnJKKghRP/0mVlIaGcmOEXBEaRF6VTW\n12391kacfvdE/PefpjjHh9Q7V67myI3bGvHDJ2Zoz5+62HydhlznLwk5lt2zOFtaShE9tUmeI6/4\np6Ytx7f+8l78i6VEqEAgoiyAewCcDGA4gHOIaHjgsIsBfMHMgwHcCeA2d/ssAKOYeQSAkwD8nohq\nDMtMjRXuIiTZkBFtsVz6iPqFmwiEsPw0Ohqb9WYGXa76sKgj1fFB53QBbkEtzAEfglkH7C0aLyKa\nA8VOQfXsmgw1BBMhN/aewhw3wfcq/rrvDf/EPRU+rcBXVrwBgspkFLp2dvBeElAR/m/qMjw+dZn2\nGG+BJBNkKVpYIiQ8dI7mSQvX4PaX4iWYBIC9f/Ji6DHiI3x+VvTVDNPGREMYDWA+My9k5h0A/g5g\nbOCYsQAedv9+AsAYIiJm3sLMnthsh/x7MykzNbwIIpUPwZRiTDomcfK68ovN23Xir99U7os+D8Hp\nZLc1qjvbp6ctx28ETSROeN+6LY3SY9/4aBXeXbTW1+mp5J2phpCrW8RnURCyK9Tj1ufNOhuG32Rk\nUhddNVUzlQujyfy/dcItLjcaJF00RfSviHX1Fm+SaQi6b/7s+yeFaxBFUqnzDzxMBEJfAKIHbJm7\nTXqMKwDWA+gBAER0EBHNBjATwGXufpMy4Z5/CRFNJaKpq1bJY/6jko+qKU4gyBxvMv4141Nc5Krd\nuXMNOiado9B0tKY6TFzLt+C6EXvBHe69bJMsKO7VU0y9zJwXIt71iskycMFD7+K/fv+Of2Ka4t38\nzLBDCj4BVf1mLF3nOzaogcTpAJx0Fc7fpuG5ukitlhZ5Sw8+oqAAeOtj//dWiT4E7xuRadxyrYjx\n4H8WRVrNLUmiPsPgO0ib1J3KzDyZmfcCcCCAHxGRfGUQ9fn3M/MoZh7Vq1e8SS/B78Qbya7eZLbY\njQrTNvXtv03Dq3NX+rYtWRu+IlOYzTj8/HjjkZPveguvzs1PFDKNMpLN3ZDJtBbm2FFGBSNwcY6J\nsF1lkjNV002F4th7JipDgLt3qI3ViTLyGqR4ut4spC5PFXYaJKi1XvfPYMrs5PwvSeFdS/a+ZRrC\n7E834GfPfohH3ilcK6EUqY+iPpuvP/guXv4wmdXYTDARCMsB9Bd+93O3SY8hohoAXQGsEQ9g5jkA\nNgHY27DM1AjG3YtEsZOaaggyJi0Md5wVE3bqHBPf7nvRn/JrxIaJFc93sF3iQ5DVU+zwclcwrqbZ\ngUWnwvds+IaCN3dd4cL79e8WSyCz+N4MTUa653L9P2fj6emfSq7jPyesOVeahsCcb0cyE6wudYXM\nAdyxLv3kz3Ee4ecb9QsCJYmJQJgCYAgRDSSiOgBnA3gmcMwzAC5w/x4H4FVmZvecGgAgot0ADAWw\n2LDMxAi+BFnHFYdi8+UUU77JlUsV6+xpCNslGoIqb5TY+RSlIYi/fT6E4u5dJshU61WLR4rvzNex\nR8Cxjef/ll8pcI7mMrq1IUTC2nOFyQOs2bxD6kPw0GlFslvtUJf+8p+VOv/AI1QkMnMTEV0J4EUA\nWQAPMfNsIroRwFRmfgbAgwAeIaL5ANbC6eAB4HAA44moEUALgMuZeTUAyMpM+N7Ee/D93q5xfkYh\niRWZdBQTdgo4H3ASNTSdUyFzKsvOZfht7VEeY/BQ1anFCuvg6YTC9aplx7YEnOVxNYTcimmS+HoZ\nYoqLuJhOdgyjVF3ee0vy2WWbWxiNzS2+OkZNXdGutnTrQVcqRjoSMz8H4LnAtuuFv7cBOEty3iMA\nHjEtMy2CH1JSHbmJY/iXL84LPUaF6YppumOSGJCYFrFD4lSRdcxOhyeWb17JoC9GFYVTrIYQ5Wyx\n/lFDdpVlSjQEfZRR8S86LM9TJUfINLcwjr79dSxfl5/wt25L4TolOirRh1BqqmSmsp+mhCIMTEah\nd78WnPRljjaXkcH5pf6AZfMQpD6EwLZiPpLbXsiHcorlFi0QYpqx4uRoCqJyuEddNa+Y6xZzjZnL\n0lsTeO5n8rQyLcw+YQA4s62jECdddlQqWagC1SIQAi05MQ0hZZORNpmZickooeqZ2j1loXy/f2NB\noQAIHCMuJF8MSb4OBmP1pu14QTPbPH9dvxAIFBQZcaDh16TU6bV1K6apCM5NSEogbNVkCi4Wb45B\nkKjftOxppS0OXpu3MlY7j5OkLy5VIhD8v3WzVaO8r7R9CNooo4grrhWDsclIoiHc89oCzFruH9VF\nXZHNlCQFNDPwl0lyJ7JpHZjj2dP971YUDowT7pRPKIzzDIPvobEVmIxUo/gfPxlNG5CScr+7euP2\nCniCeqpDIAReg262apQPy8SHUAyVYjIyfSamk33SsqMmnd/fW84S0NuX/fMQRKcyx6rTp+vEzJ3i\nhfTnRY1gKRAIIdF3lWD/Vr2GuZ+ZLUwUp+ykYFR+lFF1CIS0nMopv9xiU1eUuu2pchkFs6AGY8CT\nqmcaC76YoHIqx3Xqf+/x/Kxu0YmuKyqOySgov9PWeJMgKTu/7L3I0nskTZwnXApnt0f6MzEqgGBH\nkZRT2XT1rdjla52IBiYjxfbbX4yawMvsPncoNKbguhHrtviXskyqI08yG0GUuSq66KY4d6a69h3/\n/khbh6iPsdC31go0hBQ7x5L0u4FneOEf38WYob1LcWUjqkIgpKUhpD2i+thwfV4VkxaskW6/5zWz\nrJtNzS3Y0ths3BHsUMwAz0rSjIskpUonKaBXRZgd2qIQCHEnpqkSsIlx90HiPMOgIH5+pt6BXgk+\nhDQ77dSjjCQ+pdfnrcLr80qbr0hHVZiMgv2EzvkYpdGnHWUkS/fsYTKq/safp4Yeo+OH//gA+97w\nUlHzEIDwjipuzqUgSZrwokTK+FJXBHwIcapUE7LynIx4JiP/GXe98rHiSIfzH3o34hWSJ02zTtry\nYEdzi3H69XJRFQIhKJcbE3IGp/1ydU7rUph7/zXDyX+zeqNZEkCVDyGsqs7oNkLFFCSpIUSahyD8\nHZa0TyccvQ5pjuESrv5yo5veoj7zoKnP48enDMWfLxodrbCYlNKenjTXPj0LS9fqV8qTUcpbrgqB\nEGz4xSSl85eTZAdUWJbOJFUKe26fLk5iWtOlBFXVDaurk+65+BtKUkOIVJQyyqiwHF25shX8olQi\nsg+hiKuJ9O7cDkfuETMTcUQScyonUkrboyp8CAVO5Qr0IajSRKuPT79Jd6p3msf6rfKRoSlhJqOv\n3Gu+QpaOpCemxTl29aYd4o7CRWg05dRkCTtizulav7UxdJ3mIEm1oVKO2luxgtAqqA4NIfBbNw8h\nSqeSpIawQdLpljsM0Osviq1Hqe4iXZORuiuKknNKJxxFZ3JtlnBgQ3dNDf2I6cpNKVeYbjG0vhq3\nLqpDIESIMlplaC8H4n1QXx4pXRgOP/zHBwXbtD6EEgqLYleXCoteSYokBfTkReYLvauagSyTka6G\ntYIzOZuh1FMWJOUCK0UOoMRphcKwFFSFQPA6bi8ePilncNT1eQGgrkb+yJdL1H2tDyHylaOzyl1R\nrliBcOfL6vj5JEl7oqAKlXlJFnaqG0RkRQ0hk0ndPhJlQNMwfoJyXynlQTX24+OfnFmyJT+rQiAA\nwMgB3fDWD48FkNxIMk45tYqQwuBsXkDfEZdC3V+72bGHp73weFKUUmsSUWsIheYk3VjEpyFk009p\nlpQALWXytTTnQkT9pnp1rk+pJoVM+2RdSa5TFQKB2VFrvSCOpKRtHNu6SkPISiYj6Zb6bAVZBkpO\nuWzi/5wuz8DpaAfBgAZ12xMHBTWZTOqmmKQeV2u0GMmI8j3/7Iy9cd1pw1OsjZ9SzV+oCoHQwgxC\n3taZVGe6ZUfhuqxhKDUEiUCQrUCWoxp15xDSzDW4TTNRbcpi+QziZgZufd6fJkT3Xberya/YVZul\nVtPRlrKaz0jWhk6KKBr/yP7dtEt0Jk2p5rNVhUBgdkYxOYGQkET46PNNkc9RaQiytAS6TshqCIWk\naTJSaQE6Zixdh5fn+Fd405lpamvyPUw204oEQox6TvjO4bGupZu9HwXZW4jqEyylM91qCAnCYBDl\nLZ3lcj4CQJ3EVwDIO/htmgRr5byHSiXNVCJJffy6OoraY222eJPR/JtPLup8c9T1PHpP+YS1Lu1q\nE7v6D07YI5FyonS6RKXVjEplDq0KgdDC8JuMhI+ya/vkGqbI/540VLpdZTKSoRvx3vu6WYK6aiJN\nISnz8ejoUCdfsF33YYttIwlzRJycSHHQya3jh/eJfE5UwpInmhJlQEGgkqTL9ogT0RiHqhAIcJ3K\nnkgX33tSAqE+YApSfYtRPtJyT0xrbaRpMpJFgenoWC9PAqDrdOp8AqG0HU4x6DQZ1T4iQvcOyXx7\nSdnyfbPMS3hdE6yGkCAtzK4Pwfn9xZb8i1eN5KJy8O49fL9VH4LKZCRDzLm0R59O8SoWgW8fOzj1\na6RJmh9N1FFoR0W70puM8m2jlCaJyT8eU1QuIl09VfsyBDx1+WGxr+m7hsGDSiMiqLQ+BCsQEoPh\nfmDuCxRzvqicvDouO2pQwbZg21DltI9iMtq8Pe9UbpWzQUtMmlq16n2qULUrU5OREyod6ZKxISpu\nPQpd01RqCCDs0q2dttzhu3Qxu76B6AzeX3DVvqg4/UlRRURClUk4aapDIDA78xAkLzBo6jFhp47h\nqq7K5hxmMhqwU4fc38vX5QVXKcwHPTrWpX6NNElzvdqoI7T2tdE1hD36dM797Qxiknvn15wyTLmP\nQEVFMeuqqdrnaED6++tu8J2FXd8jeH9/nLjYqGwVA3t2LOkgzQqEBPG+QdkLrK+JbjKqMTAfZJQa\ngr4RdZLYnkf074bT99vVrHIx6dahNrLZQBReSXLEkJ6xzkszyshbG8IU1YBAVcfvHrcHBvbsmPvN\n7tyZpNivfzd0bif3a2SoOHObrmNXawglTnmR4Aznv37zILSrzZa0/ksMU9AXS1UIBN1oK46GENap\nA+oPIex6Mllz79f2R6f6ZHwdKk4Y3ifyiCdq5I0p9399VO7vdrXm7yftFeyioGpvKk2jU7saXwfj\nmTkBoHcCKRIyGp8EERX37HQmI8XrE8PAi8VEk0pSefS+k1JqCDOXrS/JdapCIIAZGVJoCBE6HA8T\nB2NW0Vh27dZee57sM8mUwGDpRLVEOyetKomCJko/1RrSOas6XnYDH/Ib8h1dEnJO9KEV7ENxHaau\nGaijjArrc9DAnRK/vkeSY4VyCIRSDXaqQiB48xBk768uRqy2zMEY3KIyGXVrr7bTq/p9MUIqLTIZ\nitzA0/ogxOcbxS9QSRqCqi66+SPiYICRb1P79O2CW7+8D05QxPSbQBqBnyEqzmQUox0QJN9MzPZU\n6ngL73qlvG6pJqJWhUDwZion5kMwMBkFhcbzVx2Bhy4cFSsiI0Pp58aPI3BUWlBcvnH4QHxpv119\nwjSahpBodYoiKMhuH7cvAGDCzBXKc3wmI0FjyGYyOHv0AHRS+ABMkHXA4s7ifAhqdPMQgruCirdJ\nlf544YFGX0aSPoRMGQRCqTL5VodAcEP4ZJ1enLBTme08+LqCGsKwXbrg2KF99BEZquvFMOcE6dOl\nHvd97QDlficbbEQNIWG15ag9e+G354z0bYvSUZXaZHTKPjsr9wW/36ijaEa+M/UGF8UMCnQT3Ryn\ncuyiY01My0hMRnHu7+Dde5Tch+BdL65Go5qjosNqCAniNHb5BxGnTzOZS6AaPcf7eOQhs1EgEOpq\n9NeOeolSxMlH+Q6WrClNJIZHtw5q81/QZDS4d/jEQrF9egkZAWdthGIhAgb16qjYRynOQ1CcI2lt\nwVG8SX9r6l5LIyQ5tkBQzGLXYX0ICVLgsBM4c/9+GNG/W6TyTCYpKT8EnYagOidT/CIkRPpw2YzC\npBbk0W8chJ+N3QtA8lFGrS1Vh66TCWor+/XrGlqe36fMuXcedVKcjAwR/nD+KMW+4jQEXe2Uo/eE\nmo7pjO4k5IEXXdjozgmIq7XHMf1ZgZAwqm+qR8c6PH3FYegS4SXJfAjBLWr1PJ6GkIS9Uuf7yBiO\ntPp0aYd9+zkCNOl5COVa8SwO3z9+D20nE/yATcwafh8Cco3K0zaLaQNEao2GUJxTWdcj6yamyeoh\nEqxSX0mEnmkEXhJNy/M3NrpT4uPK6c5WQygvzgI5is7WfatRRrtGE9NiDIxU58jsrXHQagiG+feJ\nnElO933tAPz09L2KrpNImo3+rrNHJFrevv27aQVCnA7WF2XE+QFCsG3urjD9mJZdsK9oDSHeICcq\nMo1M6yxPGM/f6K24GPebjGMyssntEkS0xwbJKj46HSZOZVV5cRpRHPu+DN09mkYyeR/ySXvvjHaK\n9Aw6RjfspOyc02z0hwSSDxYLQV/fOB1ssGl4P4Oa3Zkj+kYuWzeGIUozl5HinBjlyn2AZgOZJKKM\nvBD1Ha5AEIXaURFm+b+9YE3ka1eUhkBEJxHRPCKaT0TjJfvriegxd/9kImpwtx9PRO8R0Uz332OF\nc153y5zu/tc7qZsKsl//br48MSJeg40yYjE5UhddoWLzDvkKaeJ60HEhhH+4JjJRPCTOAGncqH44\nYLfu0n19uhQmO7tqzJDoF5FQk83gimMKkxIWg+4bjach5BH9XsG2FGedA62GUKTJSK/1qgZG4eWa\nVMnxIfgL6yZJq52Iyag2oCEI+0yCBgBg8a2nxrp2qaypoS2LiLIA7gFwMoDhAM4homAu2YsBfMHM\ngwHcCeA2d/tqAF9i5n0AXADgkcB55zHzCPe/lUiJX561H757vHxVJYqhIci+gEIfguLUWBpC+rMi\nTfPvi/WIo7fonNcjBxQKCtV7U7Fr13ZSbSBr6DSPgm7UGXVEF9RiZWGnHnGczLpTMhS9vg098v4j\nXbtR+5TD78FMIBRqCD07SVJ9JKB9XnPKMBw3rDeOHeqMXcX2lFYal4cuHIUzRuxaURrCaADzmXkh\nM+8A8HcAYwPHjAXwsPv3EwDGEBEx8zRm9rKCzQbQnoiKT8ySIPE0BBMHYXQNQUWGCEN31qcClo2K\ngui+CWemcnhdxNuK07+GaSrFUl+blTv9M8DKDdsTvZbWhxDrA/b7ELxfXqoU73ec+R/6Tjuf7fT+\nr6vnqoj06lyfC8SIM9kySTOPydNIojvt170DHrjgQHSoK7zvtAZs5M4PqiSB0BfAUuH3Mneb9Bhm\nbgKwHkBwmPYVAO8zs/hV/tE1F11HihZLRJcQ0VQimrpq1SqD6kYjG8Gp/OAFo3DO6AFGjVkddhrn\nYwb23LkzPrjhBOUxz33niKIW0clIVG9VXYolTW2nqaVFmio4Q4T1WxsTvZbM7l6bJXSoy0pV/HNG\n99eW59cQ8hIhaCGKMy0hbEKkV93de5m1IfHWdZ9OpCijwDbTQb1Je0rCPxW8jHhdE+EVFpX31VH9\nMWao33JekyFkqcgosAiUxKlMRHvBMSNdKmw+zzUlHeH+93XZucx8PzOPYuZRvXrFX9VJUzcAhQJh\ntx6FL2/MsD74+Zf3MRqRqCamxRpVuyfpFibftVv7glXbopAhAhm0hjgCrVN9Dc4Ysat7vn9EJ0v3\nXQzNzexbEc/DNKzWFFVkzswbTsT71x0vnVl67FB9LiK/DyEvoL2JkDmfQiyTkSagIJPvcGoyhAW3\nnIJLj9pdW14gqDZyfYxMRuaFhdYmif40+AwLwoQFdu3aDp3b1WBnwS/25g+PAaDWwmprCPsE5qtk\niZCtMA1hOQBxaNPP3SY9hohqAHQFsMb93Q/AUwDOZ+ZcZi9mXu7+uxHAX+GYpkpOnLwksk6xYBp+\nguF2poQ1mr37dsERQ3ri4N0Ls0pmM2YeAbEvMr2VoJlJfDZJP45mZiwXVsTzqMtmcvHjSa2jLZvQ\n2K42i3a1Wan2EOYLFp+L+Cq9cEevyODgZeSA8ImVYY/ZazsZtwMKy1PFzLl66A5VdcSyc7xvY7SX\n9dSwDyyVyUirIQRudFTDTph5w4nStqZeKwP49rFD8K8rD8cwd7W4TIZ8AjttTATCFABDiGggEdUB\nOBvAM4FjnoHjNAaAcQBeZWYmom4AJgAYz8wTvYOJqIaIerp/1wI4DcCs4m4lHirHXVSCL7mYcLu4\nhDWa+posHrn4oFxjEyFDx7UoNkydyhmi3AdJ8DsBk34ezS2MBy88EOeMHpDbtuCWU1CTzaDJXaN6\nl676pRtNIBD++7AG5X6ZbNY9X2dmsh9vTe1gqpRgZ22ypkfYu/WajiqyqeB45KOddEeqWqTsnAsP\nbcC5Bw3ApUfu7p5r6EPQjNxz9UhEQ1D/DpYfx6/S1NyCbMbREjwflGcyqhgNwfUJXAngRQBzADzO\nzLOJ6EYiOt097EEAPYhoPoDvAfBCU68EMBjA9YHw0noALxLRBwCmw9Ew/pDkjZnideQF6qDmHNn7\nDH6kKrU+qobw7LcPNz62mEZjGs8dz+SV7yCDgsfE/HH1iXsaX6uphXHw7j18k+a8d9zkaghJRYTo\nzGeydxF2XX9xjEa3jLqAySh42TqDjL1h780bTGRy34P+eGYh6V6syLnCc9rVZnHLmfugs2saNe3E\nzTSERHQE/y+fD0F3pB9VmxfbTLPwPirNZARmfo6Z92DmQcx8s7vtemZ+xv17GzOfxcyDmXk0My90\nt9/EzB2F0NIRzLySmTcz8wHMvC8z78XMVzGzPAg/Zbx3GvxYdY9f9jqDCcji5DKSYZJIz0PXZmQf\n7b79uuJrBzsjadPQVlWU0XeOHaw8RydsTR7HFceoyw7ifTiyW/E0BFEbjDPrV8Z+AfORLMooTKMK\n2qS9nDnBFfqCHWWdgZc57NX2cldl864V1skz8t+MVkNglqaF0Tq5hX23nLlPbn3qq44bgr37yrXb\nUBLoTwtNRkLxivJlgkjVNzQKbcZrP164dMXMQ2jrqNID6FZvkjXAoMlJuTpVRIEQZTSrC3WUXXfs\niL6RV3/ym4zy6Eb6BL+NNc1wvZxAkOzzNASxrkld/bdnj/RNOpIl6gsbpQYXyPHuxdMAVALFJIV7\n2HN+4PxR+MW4fdG7czuj48EsaAjaw/DqD47Gi/9zpG+77PvwAgy8PQzg3IMGYFBvR2gP6tUJz377\niILzTJqQqQ2+X3f1ioaFTmXR52Oeu0rln2kUouM8DSGbIWQzFTZTuS3jveSdA7Nkf3bG3rkwzv8+\nrAGvfP8oYW/hCy0wGcV0Kgcbksn6zR66Rq9b7Nzbb9I5+53KQgfmXvrg3XfCqMBMZBJ9COSfAOf9\nqZq9HJVgRyzeUqMw6iqWYBHB31sbYyi8vhFn3mQU1gZMtMiwW+7dpR3+a1Q+diSsyBYWNQR94T07\n1WPPneWZAkS8CBuvrl57dhU7dTqYoClHllrbsD/98si+uORIeYRVsFRtuK3mGibrbXsCwBEIGSsQ\nSoX3Um8/az/88qz9ctvra7LYvacjEPp174BBQny2NELC0Kkc1ukGv/0oaQrcAbB0pq4qRC5qxE+Y\nKeGggT3wxLcO9W3zCZGC6xDe/fEYPPqNg8IvbkDeZFRYT89J69MQIggHz7wmo1i/hDgRDXA6XG/E\nGNQAgl2DyTKwUTWxcJMR55IlajWEWLYap0CvnXolFBPKbR7CStiuEObBZ6iLMgoiRqSp2opn0gTg\ndypn7AI5JcNr+F3b12LcAf18+3KOtuBoUFKOqYYQ1naDjSVK9JPXiGR5j2SliJ2zmCpBR7BzNyFD\nlPsig05lImd0GidRnoygyWjfvvm4bs9kFDeiTKdZmDy7sG862Ak3B5zKKmoNTEZR71h1O15dmM0C\nAuL0Y8FrM6vbtcn5qnp4aajFuTAEYKkkbFlWrm+QVXCw/+dlwryO4GP7yv5Ov+O1TyDgVK6kKKNq\nxnsHwU5aNnoKOpVVH1RYvxFHIHi5W7zJdL0kuVyUPo3caKww7FF3vFOmwQnwooy8jtqv0McdWIeF\nWmYyhH9861D8+aK85tEkqOG5ukW45tcP2U19vUBBE75TGB0W9kmLRTgmI3nYaRBRYFypcMBHjQSS\nCbg9+nTCk5cf6tYv3zaT7qy89uuZEVu48L2JGC2hKXn6Xrld2tXg3IO84AoyTlQnM5nm9hVmN8v9\nFRSk3vW8pHlO3Zx/s5Rf2rYU64VYgaAhNzIxaHDBjlu3uLiOAoEQ0hlcfvQgPH2F85F+9/g98OeL\nRmP0QL3JaKw7a/jIPXrlNQQ2NBmFtBhZkw1qBMUmyAOAeTedHHrMAbt1R1chx1OTJI/9zWfuY3zN\nwb0758xxXgkn7+2sq9wlMAFpr10LV0gLMysER5zeiNHTAFTvR/QxqI6JKnhl2lDQ6Z0L59V0VHE0\nhP47dcC/v3skxp88FIC/cxSZeu1xbr3C2VOS7TgfCu1/fj84YU+c7wr/bh1qc2sg68zCKtNYcH6H\nc56/HO/9NcmijIRJgqE0qg8AABhQSURBVKUwG1mBoKFFIRBkDVA3rT0Kqsai4vQRu6Jf9w7usRkc\nqcjLLpYyckB3LL71VAzu3SnXqBlsNNISj4gy6vT7LfJ/qzqq4bt0MZqBG4Wz3clqPTs5K4fd8KXh\n+VmxhgQ//F+M2xfTrjs+EZOXTzgy0M5NtxzWBrKCLUV1ZPBdnTN6AM4Z3R+XHy1PCS57tUSCiVGI\nMtJpCHFn2A7p0zmnGeW0y5ha97PfPhxfPbAwj1SLYIryiiA4Phtv1H7avrvknl3wMuK3GnwEwfXL\ndVqx5yNqapZHGWVS0sRkJJtIpo2RNxn5t8saYFKTnYLlhE5mMhxhq8NgXXXUdBJQWJSUZFsmk+9I\nRb+FrrznrvKHF3ZpV4MN25p825658jCcfvdEmHLZUYNw6ZG74/p/ztZe2wj31M6a/FJBdI+Y4X8u\nLcy45MhB6FC3BMMCmW6DfaxPO1VGt/l/EwG3nLmvsj4yDZfIb/CryY1sC5MJJkrufuWdbNg3sHdf\n+XrW4oDPu12v8/VKFJ39Wh+C8E6+ecRAXHnMkMCxgsmoYNDnrbNQGGVUkyGcNaofjtqjl1HwQLFY\nDUGDWkOQ+BCSEgjBxhLiSZNdVqa+KkeO3jmGAiFu+m5RdY7jh/i1ZJU1WWZOXSSQcz3KPZ9IS2C4\nx3oT0GR+mjBGSnIf+a4h/M0Ajh/eBw9fNBrdO9b56hCkS3u/U1RettrcIUO2P0NAD1e7Onj3Hrko\nI93INYkxrTK4wxu5R2yTnp/Fq7bUvCvZpusHRHPgNacOz5kqc+HWmnJ2ddeKPsk1PwJigAihd+d2\n2Ltv11hJDaNiBYIGVjQYEw2hU31NzlYfhYKcSJHSHagJ82mYqvZxbP4E4IbT98Lp++2Ko/fsHStB\nnmwQKns0N50R7hOQ2Y49vhOyQtvVJ+yJCd85HEMUK/Dp6NahDvsoRqtOhczKCQp8bzIZoDGrBLXc\nkIvJ2h2Rs6rdaz84GtecOszIhyC2uwsPbdBeU4W240b0CKrO7sxpFgSN9zwKIpw01zGZqZw7Vzg2\n+I337FSHD244wRcQkDMZFaPFxsAKBA1h0Q3idk9tv/zoQXjiskPQr3sH3HX2yMjXjK5pFB4va5xx\n7a/FHg84H/Ku3drjN+eMRLvarFZ9ViETWHFnObPGJj1u/36FG5F/yjXZjNRhbIp+MpM44tSX8/Mv\n5wVf7855bUXV0RfG0OvLV61fDAADe3ZEbTaT9yE0yyt72VGDcquLAc6gIA55jU5hMjJsBt5xed+E\n87tG0MK9a/g0Z/LODz5D4X2FzUIXTi1Mkkfo0q7WJ4Rzk/HiLH5RBFYgaFA5s7zfWUnHVpPNYFRD\nNEelSFS10PTwsBXcTBdZV318urN7d/GbVzK+j8NUIBgdZoS3dnMP1xTz8EX5zOtqwZnQh6kpxx9l\npL5hAvmyuXYScgUp619wrRANQWpF8W8cf/JQjBzQDYcMkq/DMf7koQUDnH9deThu+FJ+Bd6/ffNg\nbT2AfOeornJQg9ffW3CinyyViWevF8ObVcV2qq9R5zISwq1V5cjKLZeGYJ3KGlTzEDwyGQDupMYO\nbmhabYQO/a/fPAjn/mEyzhixK56e7qw0GnXSlGlHFWZbNnYqR0jQBjhROMcN6xM4Jn+QmC5Bh0xg\nxdUQrjhmMAb37oQT93Jstkft0Qv9urfHMsWEJCC6WUKF6vUGZyrr5HNQWPjDeIEnLz8UO3WoUx4D\nhI+qZc82WPchfTrjqcsP0xcUYJ9+XX2LwJikLFFpdDlHsOmgCM7AJeicrcnkncrev2eM7IuFqzfj\nimMG4cn3l+XOF/EEy9cP2Q2rNoYsz+ozk6p9ER7NkjkzpcAKhAD/+NahuZEjK6S09wLFj+aCQxuw\nYVsjvqnIgyLj0EE9sfjWUzF/5aacQIg6IjA9WuWbzocRGl4vYvvUdfid6mt8Mzh1yGKw62oyuO9r\n+2PkgO446JZXjOtUm83gtH3l/p0oC7rEQdbRDt25M84Z3R8ffb5JWw9Ps+nWXt3ZEwH7DyjsZAs6\n0zAfgkxD0J4RD5Pn6j0KlVPXeFBEBDCjNhASKmoI3jXqajK5eRBe+cHrd6yvweyfnoj2tVlc/cQH\n+msr/nbKVZ9nBUKZEUcsKudjfmSS396uNourTxwa65o+E0pkk5HEhyA5TtUBePdmknPmd+ftHykd\nt47bx+2LAxt2Mv6YVRrMSXvvkkh9coIxkbgYNcHXu+jnp0ijZWT1uOKYwWjo0RGn7LOzb7so7E2z\n7BbjQ0gSkxJV0X6qMtTasEOwDRezOFbHer+DWoXOb6Z7rNapXEGo1LagelksYgOJbjIq3BbMNqo6\nDsh/JCYmo70jOlOPH65eQ/isUf3R0NN8LQITH4dsNqopoaawhMbHuhTKYdRmMzhjZF+tc1NFQdhp\nSDuTz0MwqGRETO7/rAMcLVMMr3VO9sowvZbzb8EKdJn805EvjxtedpRhRKHpS32BUoSailgNQUPO\nZKQI2TNR53551n4GMd/C6CGBBjBsly5YfOupaBg/IbdNOWrydhh0uFE6hIW3nJJoBxJWvRnXn4D6\n2uLHN8rrJHQvfbu3BxbJ9322flt4PSSYhPGaJGgU2a9fofBPSihG5fsn7IFvjxmMem9dCFLPfA/+\nfu47+QmO5HoRgk5lx4fgammaeuheyfeO3wNPTQsuNS+au/LbTDIfXHvqMPzqpY80V0wHqyFoyE0q\nMTAZqRh3QD98WRHKGCzPKTNKDc0FiDrKyDMZ+Xnj6qNzf7ePkZYhI3xkSRA2T6Jrh9qi0kfkTUaK\n/THKvPe8/Qu2naIxce0vaHbRRpyCD8Ew7DTshmTzLMJe58vfO0p/gAST50pEOWEgq49OUA3ftXCF\ntaBT2ZfsMKbvpP9OHaTbvWarM/3IvpNvHLE75vzsJIMrJ4sVCBpyJiOFRE9KmxM79RrTHL+BuoQe\nF2oy8ndBu/XIm3NK7diSkXYaF29SUJ8u0Wcgqzh5n8LO/7jhffD2+GOlx/ft1h4fuUn7TCYKPvvt\nw/HgBaMKnMoygtsP3C16aHTYAGhw707S5TJ1FDNmoII/zE4oCDsV5xMk3M7yoetqK0AFfF45rEDQ\noJohGUVDMCFOXH6wLuHX0DsbdR9CbmWsMjbcuEnSTDlrVH8svvVUdKiTd2hJ3ruXqkCG96xNbnfv\nvl0xZlgf34AlWM27zh6BYbt0KRiFHqfx76gwGatMv/6ESGUmoUVGlAeRncpeoEmU1Qs9ZJNbC53g\nlSMRqtqHMGyXQnVSJGxhjqRGzj6ncsRGZ74Wspyj9+yNX770EcYMK+wgfvGVfdGnazt8//Hp4RdI\nucPeV2LTLiWl+mi9JnXNKcOMzxGbQLA5jB3RF2NH9E2gZsDFhw8MPaaUTtB8dJap2dT5tyBVvfBb\n1op/c85ILFy1OVIiQ4/88p/C9SLOCSklVachvHn1Mbm/n7+qcMFukRblbEHnd1IagljM+YfsZrxA\nR74m4ajqunffrlh866nSCUL/dWB/HLVHL0FDCL9aWh3n0J274OObw9dASItSfbREhMW3nhppPot/\npm06Fd2tRwccOzS6VpEmFPg3t11pHnV2BAdd4sQ0WTRbh7oaZcbUMFhiMtI5wctN1WkIA3rInT8y\nciYjRdhpRHO/ErGz7te9A17+3lH45/TlZvnPo+rLMfD8GqVYsUlHUnMg4lBB32wBcZIFRqVES/oa\n4cQKCb8L/OV682jBcrcZyrUtXaK+OMgW9+neoQ7dOtRi3ZZGt16V07qqTiBEIWyBnLgawt++eTB2\n7prPUCkrx1TNL9ZkZIKnIZRqXVdLNNKYMNYaMIkyEvntOSPx21fn5yaTedRkKLXlQGV9SF1NBtOv\nPyEXFl5JTmUrEDTkQsYCb0w1ld6UYDKwYhpEsVFGJuQ+lkoaJpaYvt3VjuBy4/chpNO7pD2D+4wY\nqeI9ciP/jH4x+jHD+mDMsD5YuXGbb3tWEAjiusZJkLcyqI+xTuVWgmphDlZsj0sxH7HpuVGEV3CG\ncbVrCHefOxKji8hgmzbB5HZpkOZYYPZPT4w1h8TrSL261ddksGVHc+h5wW+hJkO5tcubFKm848KS\nKKPC+iR6yaKoOqdyFFQmI28QkUbYaVrnmlZ18a2n4g/nj/Jty6Y0eqp09nInNZ22767o3aVdyNHl\nI6txWBbDzWfunei8DBUd62tiRex599rohvIE5xeo6OJGC+3upk7JZkhYDjRhgeD+q+0rrEAoL0cp\nFqIP0qLo+MMWzolKMYLFeE3lIlrdafs6E6x6dU6/c6gkHrv0EF9UWqWSloZw3kG74dFvOOsVVJK1\nMKgV72hyBYI7yg/7nOpqMlh866m43J2M2KV9bc5k1JTwoMfTqnXfuDUZlZkHLxiFHQYvXqXueS85\nKXttUZpGwhqCjCuOGYzzD23IjaxE/vekoVizKSQXfILUZsm3GHmadKqvQaf6yv9EyCDbaVwqyZzh\nEQwNzQkEQw3B44wRu2L1pu248NAG/HXyJwDSiDJqXSajym/tKVCTzeRshjryC+T4t7Nie1yKkgfG\nAqE4P4VMGADAt44eBAD41UvzYpcfhSnXHIdtjdVlugpDHFDs179bomVXQtoSFTmTUbNfIJhqMzXZ\nDC47ymm/tTmTUcJtK5ftQH2IDTttJcjykIjbk/MhxC/H9NxSfddpt+1ugZXAWivTrz8+sdGo+G5H\nJCwQKjGk1VvoxiMnENwRWpyIqGwmHaeySV9RSTLXCgQNqpnKzYKgOH54H8z7bGNR18nnRopxbsLH\nWYCXvnskNm1vSvUaSQq2NDtt3QzecuPZ3j2TUX1Ek5GI51RO2hypmtwqYn0IrQTVmsqeA6p7h9qC\niJw4eALnFEl2zDBKYTKqNvYoYqGdcpDmq1WlR68kvM62S/vouYY8PJNRc8ImI3X6G4EK+jStQNCg\nWtx7n75dce2pw0LXOTAlkyG8++MxsUaNlWYyspSeUiyzWIEKQu67POuA/vh8/TYcNrgn3vp4dayy\nPJNRY9JhpwYT0yrp27QCQYN6CU3CN44wTz5mgmmc+yVH7o4ZS9dh8qK1Ea+QbqurxA6jWiiJySiC\njnDX2SPQV5PiO2nqajL43gl7Yuay9QDitcUR/Rzfy2kxtHSPvt3a49P1W33bTHwIlaS9GwkEIjoJ\nwF0AsgAeYOZbA/vrAfwZwAEA1gD4KjMvJqLjAdwKoA7ADgBXM/Or7jkHAPgTgPYAngNwFVeYoTIn\n3Svohf3YTYvs5UExrdpuEZL6WZLjH986pKiV3EyooOYJwDwPV1xUt2uytoeKAT06YPGtp8auE+Cs\nMhi8tInJqFVpCESUBXAPgOMBLAMwhYieYeYPhcMuBvAFMw8morMB3AbgqwBWA/gSM39KRHsDeBGA\n11ruBfBNAJPhCISTADyfzG0lQ9LRRGlgUrcHzh+FIw0n48Wlgh9RWTkgxspkUUlzhBlMD1EJ1NVk\n0LSjuaLqBEAayt7anMombvnRAOYz80Jm3gHg7wDGBo4ZC+Bh9+8nAIwhImLmacz8qbt9NoD2RFRP\nRLsA6MLMk1yt4M8Azij6bhJG5VSuJExqdtzwPpEn7VgsQGUK+qcuPwzfP36PVtWm9fMQSlePMExM\nRn0BLBV+LwNwkOoYZm4iovUAesDREDy+AuB9Zt5ORH3dcsQypXomEV0C4BIAGDBggEF1k0OV3K6S\nqCT7o6XtUkmD8T137ow9d1ZHglVSXT10g8pK+oRLImKJaC84ZqRLo57LzPcz8yhmHtWrV7pmj8Jr\nO/+WclnAqFRK1SpNfbckQ89O9RgztDfuOXf/clcllErqWIO0pVxGywH0F373c7fJjllGRDUAusJx\nLoOI+gF4CsD5zLxAOF6M2ZSVWXaMYojLTKVpCJVVG0uxZDOEBy88sNzViESFxaYAaD0zlU00hCkA\nhhDRQCKqA3A2gGcCxzwD4AL373EAXmVmJqJuACYAGM/ME72DmXkFgA1EdDA5Pdr5AP5Z5L0kTmtw\nKlssFgfPLFOJvgW9yahy+pdQDcH1CVwJJ0IoC+AhZp5NRDcCmMrMzwB4EMAjRDQfwFo4QgMArgQw\nGMD1RHS9u+0EZl4J4HLkw06fR4VFGAFmqx1ZLJbKYM8+nXH50YNw7kED8OT7y3HY4J7lrlIOrVO5\ndNUIxWgeAjM/Byc0VNx2vfD3NgBnSc67CcBNijKnAtg7SmXLRSVqCIcO6oG3F6wpdzVypL3EosUS\nBhHhhycNBQB8Z8yQMtfGj04LqKTuxc5UNqASfQgPXnAgVpdwHQJTKvBRWSwVTSWZjKwxxIBKjDJq\nX5dF/50qb/ZxBfrzLBaLIVZD0PDlkX3x5LSKC34y5rUfHI3NKadx9qik0DmLpVJ44X+OwJTIecfK\nhxUIGn4xbl/ceEarcHNIGeguIm6pDobt0qXcVbAEGLpzFwzdufW8FysQNNRkM+iU1DqZbRzrVC4v\n0647Hu3r0k2iZ0mWv37jILz04eflroYPKxAsiVJB/rGqonvHtrG0aDVx6OCeOLSCQmMB61S2WCwW\ni4sVCJZEOHSQM9IZPbBHmWtisVjiYk1GlkQ4bHBPzP3ZSakvBmOxWNLDagiWxLDCwGJp3ViBYLFY\nLBYAViBYLBaLxcUKBIvFYrEAsALBYrFYLC42yshiqSLu/Op+6NOlXbmrYalQrECwWKqIM0f2Cz/I\nUrVYk5HFYrFYAFiBYLFYLBYXKxAsFovFAsAKBIvFYrG4WIFgsVgsFgBWIFgsFovFxQoEi8VisQCw\nAsFisVgsLsTcetbCJaJVAJbEPL0ngNUJVqc1YO+5OrD3XB0Uc8+7MXOvsINalUAoBiKaysyjyl2P\nUmLvuTqw91wdlOKercnIYrFYLACsQLBYLBaLSzUJhPvLXYEyYO+5OrD3XB2kfs9V40OwWCwWi55q\n0hAsFovFosEKBIvFYrEAqAKBQEQnEdE8IppPROPLXZ+kIKL+RPQaEX1IRLOJ6Cp3+05E9G8i+tj9\nt7u7nYjoN+5z+ICI9i/vHcSHiLJENI2InnV/DySiye69PUZEde72evf3fHd/QznrHRci6kZETxDR\nXCKaQ0SHtPX3TETfddv1LCL6GxG1a2vvmYgeIqKVRDRL2Bb5vRLRBe7xHxPRBcXUqU0LBCLKArgH\nwMkAhgM4h4iGl7dWidEE4PvMPBzAwQCucO9tPIBXmHkIgFfc34DzDIa4/10C4N7SVzkxrgIwR/h9\nG4A7mXkwgC8AXOxuvxjAF+72O93jWiN3AXiBmYcC2A/OvbfZ90xEfQF8B8AoZt4bQBbA2Wh77/lP\nAE4KbIv0XoloJwA/AXAQgNEAfuIJkVgwc5v9D8AhAF4Ufv8IwI/KXa+U7vWfAI4HMA/ALu62XQDM\nc//+PYBzhONzx7Wm/wD0cz+UYwE8C4DgzN6sCb5zAC8COMT9u8Y9jsp9DxHvtyuARcF6t+X3DKAv\ngKUAdnLf27MATmyL7xlAA4BZcd8rgHMA/F7Y7jsu6n9tWkNAvmF5LHO3tSlcFXkkgMkA+jDzCnfX\nZwD6uH+3lWfxawA/BNDi/u4BYB0zN7m/xfvK3bO7f717fGtiIIBVAP7omskeIKKOaMPvmZmXA/gl\ngE8ArIDz3t5D237PHlHfa6Lvu60LhDYPEXUC8A8A/8PMG8R97AwZ2kxcMRGdBmAlM79X7rqUkBoA\n+wO4l5lHAtiMvBkBQJt8z90BjIUjDHcF0BGFppU2Tznea1sXCMsB9Bd+93O3tQmIqBaOMHiUmZ90\nN39ORLu4+3cBsNLd3haexWEATieixQD+DsdsdBeAbkRU4x4j3lfunt39XQGsKWWFE2AZgGXMPNn9\n/QQcAdGW3/NxABYx8ypmbgTwJJx335bfs0fU95ro+27rAmEKgCFudEIdHMfUM2WuUyIQEQF4EMAc\nZr5D2PUMAC/S4AI4vgVv+/lutMLBANYLqmmrgJl/xMz9mLkBzrt8lZnPA/AagHHuYcF79p7FOPf4\nVjWSZubPACwloj3dTWMAfIg2/J7hmIoOJqIObjv37rnNvmeBqO/1RQAnEFF3V7M6wd0Wj3I7VUrg\ntDkFwEcAFgC4ptz1SfC+DoejTn4AYLr73ylwbKevAPgYwMsAdnKPJzgRVwsAzIQTwVH2+yji/o8G\n8Kz79+4A3gUwH8D/Aah3t7dzf8939+9e7nrHvNcRAKa67/ppAN3b+nsG8FMAcwHMAvAIgPq29p4B\n/A2Oj6QRjiZ4cZz3CuAi997nA/jvYupkU1dYLBaLBUDbNxlZLBaLxRArECwWi8UCwAoEi8VisbhY\ngWCxWCwWAFYgWCwWi8XFCgSLxWKxALACwWKxWCwu/w/Q75TOrVwG+AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B5Iz-BQLiiUA",
        "colab_type": "code",
        "outputId": "c6c22890-e7b2-4214-970f-be50d3318e27",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        }
      },
      "source": [
        "plot_d0(D,input_node)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8FdX9//HXJxsJa4CwScAECWEL\nYQmLIihiFRBRXEDqhlaoUutWd2y1+LU/rX4t+hWrUAtiURARBYpaF1QQBQOEfQt72EF2CGQ5vz9u\nSNmyAPfmJpP389E0mZkzM59rkncO586cMeccIiLiLSHBLkBERPxP4S4i4kEKdxERD1K4i4h4kMJd\nRMSDFO4iIh6kcBcR8SCFu4iIByncRUQ8KCxYJ46JiXFxcXHBOr2ISJk0b968Xc65WkW1C1q4x8XF\nkZqaGqzTi4iUSWa2oTjtNCwjIuJBCncREQ9SuIuIeFDQxtxFpGhZWVlkZGSQmZkZ7FKkhEVGRhIb\nG0t4ePg57a9wFynFMjIyqFKlCnFxcZhZsMuREuKcY/fu3WRkZBAfH39Ox9CwjEgplpmZSc2aNRXs\n5YyZUbNmzfP6F5vCXaSUU7CXT+f7fVe4i4h4kMJdRMSDFO4iEhDffvstvXv3LrRNWloa06dPL6GK\nyheFu4gEjcI9cHQppEgZ8eepS1m2Zb9fj9n8gqo8e22LQtusX7+eHj160K5dO+bPn0+LFi0YO3Ys\nFStWPK3t559/zkMPPUTFihW59NJL89fPnTuXBx98kMzMTKKiohg9ejTx8fH86U9/4siRI8yaNYun\nnnqK+Pj409olJib69TWXF+q5i0iRVq5cyZAhQ1i+fDlVq1blzTffPK1NZmYmgwYNYurUqcybN49t\n27blb2vatCkzZ85kwYIFDBs2jKeffpqIiAiGDRtG//79SUtLo3///mdsJ+dGPXeRMqKoHnYgNWjQ\ngM6dOwNw22238frrr/Poo4+e1GbFihXEx8eTkJCQ327kyJEA7Nu3jzvvvJPVq1djZmRlZZ3xPMVt\nJ0UrsuduZv80sx1mtqSA7WZmr5tZupktMrO2/i9TRILp1Guuz/Ya7D/+8Y9069aNJUuWMHXq1AJv\nziluOylacYZlxgA9CtneE0jI+xgM/P38yxKR0mTjxo38+OOPALz//vsnjacf17RpU9avX8+aNWsA\n+OCDD/K37du3j/r16wMwZsyY/PVVqlThwIEDRbaTs1dkuDvnvgd+KaTJdcBY5/MTEG1m9fxVoIgE\nX2JiIiNGjKBZs2bs2bOH++6777Q2kZGRjBw5kmuuuYa2bdtSu3bt/G2PP/44Tz31FG3atCE7Ozt/\nfbdu3Vi2bBmtW7dmwoQJBbaTs2fOuaIbmcUB05xzLc+wbRrwonNuVt7y18ATzrlCH7OUkpLi9CQm\nkcItX76cZs2aBbWG9evX07t3b5YsOePIrATQmb7/ZjbPOZdS1L4lerWMmQ02s1QzS925c2dJnlpE\npFzxx9Uym4EGJyzH5q07jXNuJDASfD13P5xbRAIsLi7utF573759Wbdu3UnrXnrpJa6++uqSLE0K\n4Y9wnwLcb2bjgY7APufcVj8cV0RKqcmTJwe7BClCkeFuZh8AlwMxZpYBPAuEAzjn3gKmA72AdOAw\ncFegihURkeIpMtydcwOK2O6A3/mtIhEROW+afkBExIMU7iJSbM899xyvvPLKOe17ySWXFLq9V69e\n7N2795yOfaKBAwfy0UcfnfdxzkblypVL9HzFobllRKREzJ49u9Dt5WXqX+cczjlCQgLbt1bPXUQK\n9cILL9CkSRMuvfRSVq5cmb9+zZo1+VMBd+nShRUrVgCwfft2+vbtS3JyMsnJyfmhfrx3u3XrVrp2\n7Urr1q1p2bIlM2fOBHyXXO7atQuAV199lZYtW9KyZUuGDx8O+G6matasGYMGDaJFixZcddVVHDly\n5Iw1f/XVV6SkpNCkSROmTZsG+GatvOuuu0hKSqJNmzbMmDED8E1zcP/99+fv27t3b7799tv8mocO\nHUpycjKdOnVi+/btAKxbt46LL76YpKQknnnmmfx9Dx48SPfu3Wnbti1JSUl8+umn+bUnJiZyxx13\n0LJlS55//nkeeuih/P1GjRrFww8/fNbfm8Ko5y5SVnz2JGxb7N9j1k2Cni8WuHnevHmMHz+etLQ0\nsrOzadu2Le3atQNg8ODBvPXWWyQkJDBnzhyGDBnCN998wwMPPMBll13G5MmTycnJ4eDBgycd8/33\n3+fqq69m6NCh5OTkcPjw4dPOOXr0aObMmYNzjo4dO3LZZZdRvXp1Vq9ezQcffMCoUaPo168fkyZN\n4rbbbjut7vXr1zN37lzWrFlDt27dSE9PZ8SIEZgZixcvZsWKFVx11VWsWrWq0P88hw4dolOnTrzw\nwgs8/vjjjBo1imeeeYYHH3yQ++67jzvuuIMRI0bkt4+MjGTy5MlUrVqVXbt20alTJ/r06QPA6tWr\neffdd+nUqRMHDx4kOTmZl19+mfDwcEaPHs3bb79d+PfqLCncRaRAM2fOpG/fvvkP5jgeVAcPHmT2\n7NncfPPN+W2PHj0KwDfffMPYsWMBCA0NpVq1aicds3379tx9991kZWVx/fXX07p165O2z5o1i759\n+1KpUiUAbrjhBmbOnEmfPn2Ij4/Pb9+uXTvWr19/xrr79etHSEgICQkJNGrUiBUrVjBr1ix+//vf\nA75Jzi688MIiwz0iIiL/UYHt2rXjyy+/BOCHH35g0qRJANx+++088cQTgG/I5emnn+b7778nJCSE\nzZs35/f2L7zwQjp16gT4/kVwxRVXMG3aNJo1a0ZWVhZJSUmF1nK2FO4iZUUhPeySlpubS3R0NGlp\naWe9b9euXfn+++/597//zcCBA3nkkUe44447irVvhQoV8r8ODQ0tcFjmbKYoDgsLIzc3N3/5xGmG\nw8PD8/cNDQ09aTKzMx1z3Lhx7Ny5k3nz5hEeHk5cXFz+8Y7/sTrunnvu4S9/+QtNmzblrrv8f3uQ\nxtxFpEBdu3blk08+4ciRIxw4cICpU6cCULVqVeLj45k4cSLg67EuXLgQgO7du/P3v/tm/s7JyWHf\nvn0nHXPDhg3UqVOHQYMGcc899zB//vyTtnfp0oVPPvmEw4cPc+jQISZPnkyXLl3Oqu6JEyeSm5vL\nmjVrWLt2LYmJiXTp0oVx48YBsGrVKjZu3EhiYiJxcXGkpaWRm5vLpk2bmDt3bpHH79y5M+PHjwfI\nPyb4piyuXbs24eHhzJgxgw0bNhR4jI4dO7Jp0ybef/99Bgwo9Haic6JwF5ECtW3blv79+5OcnEzP\nnj1p3759/rZx48bxzjvvkJycTIsWLfLfPHzttdeYMWMGSUlJtGvXjmXLlp10zG+//Zbk5GTatGnD\nhAkTePDBB08758CBA+nQoQMdO3bknnvuoU2bNmdVd8OGDenQoQM9e/bkrbfeIjIykiFDhpCbm0tS\nUhL9+/dnzJgxVKhQgc6dOxMfH0/z5s154IEHaNu26OcNvfbaa4wYMYKkpCQ2b/7vVFq33norqamp\nJCUlMXbsWJo2bVrocfr160fnzp2pXr36Wb2+4ijWlL+BoCl/RYpWGqb8lcDp3bs3Dz/8MN27dz/j\n9jIz5a+IiMDevXtp0qQJUVFRBQb7+dIbqiIiJSw6OrrIK3XOl3ruIqVcsIZOJbjO9/uucBcpxSIj\nI9m9e7cCvpxxzrF7924iIyPP+RgalhEpxWJjY8nIyECPpSx/IiMjiY2NPef9Fe4ipVh4eDjx8fHB\nLkPKIA3LiIh4kMJdRMSDFO4iIh6kcBcR8SCFu4iIByncRUQ8SOEuIuJBCncREQ9SuIuIeJDCXUTE\ngxTuIiIepHAXEfEghbuIiAcp3EVEPEjhLiLiQcUKdzPrYWYrzSzdzJ48w/aGZjbDzBaY2SIz6+X/\nUkVEpLiKDHczCwVGAD2B5sAAM2t+SrNngA+dc22AW4A3/V2oiIgUX3F67h2AdOfcWufcMWA8cN0p\nbRxQNe/rasAW/5UoIiJnqziP2asPbDphOQPoeEqb54D/mNnvgUrAlX6pTkREzom/3lAdAIxxzsUC\nvYD3zOy0Y5vZYDNLNbNUPfBXRCRwihPum4EGJyzH5q070W+ADwGccz8CkUDMqQdyzo10zqU451Jq\n1ap1bhWLiEiRihPuPwMJZhZvZhH43jCdckqbjUB3ADNrhi/c1TUXEQmSIsPdOZcN3A98ASzHd1XM\nUjMbZmZ98pr9ARhkZguBD4CBzjkXqKJFRKRwxXlDFefcdGD6Kev+dMLXy4DO/i1NRETOle5QFRHx\nIIW7iIgHKdxFRDxI4S4i4kEKdxERD1K4i4h4kMJdRMSDFO4iIh6kcBcR8SCFu4iIByncRUQ8SOEu\nIuJBCncREQ9SuIuIeJDCXUTEgxTuIiIepHAXEfEghbuIiAcp3EVEPEjhLiLiQQp3EREPUriLiHiQ\nwl1ExIMU7iIiHqRwFxHxIIW7iIgHKdxFRDxI4S4i4kEKdxERD1K4i4h4kMJdRMSDihXuZtbDzFaa\nWbqZPVlAm35mtszMlprZ+/4tU0REzkZYUQ3MLBQYAfwKyAB+NrMpzrllJ7RJAJ4COjvn9phZ7UAV\nLCIiRStOz70DkO6cW+ucOwaMB647pc0gYIRzbg+Ac26Hf8sUEZGzUZxwrw9sOmE5I2/diZoATczs\nBzP7ycx6nOlAZjbYzFLNLHXnzp3nVrGIiBTJX2+ohgEJwOXAAGCUmUWf2sg5N9I5l+KcS6lVq5af\nTi0iIqcqTrhvBhqcsBybt+5EGcAU51yWc24dsApf2IuISBAUJ9x/BhLMLN7MIoBbgCmntPkEX68d\nM4vBN0yz1o91iojIWSgy3J1z2cD9wBfAcuBD59xSMxtmZn3ymn0B7DazZcAM4DHn3O5AFS0iIoUz\n51xQTpySkuJSU1ODcm4RkbLKzOY551KKaqc7VEVEPEjhLiLiQQp3EREPUriLiHiQwl1ExIMU7iIi\nHqRwFxHxIIW7iIgHKdxFRDxI4S4i4kEKdxERD1K4i4h4kMJdRMSDFO4iIh6kcBcR8SCFu4iIBync\nRUQ8SOEuIuJBCncREQ9SuIuIeJDCXUTEgxTuIiIepHAXEfEghbuIiAcp3EVEPEjhLiLiQQp3EREP\nUriLiHiQwl1ExIMU7iIiHlSscDezHma20szSzezJQtrdaGbOzFL8V6KIiJytIsPdzEKBEUBPoDkw\nwMyan6FdFeBBYI6/ixQRkbNTnJ57ByDdObfWOXcMGA9cd4Z2zwMvAZl+rE9ERM5BWDHa1Ac2nbCc\nAXQ8sYGZtQUaOOf+bWaP+bE+KScys3KYvWYXc9b+wpqdh9h58Cg4R83KFYiPqUSH+Bpc2jiGShWK\n8yMrIuf9m2JmIcCrwMBitB0MDAZo2LDh+Z5aPCBjz2H+MXMdH8/PYH9mNhGhITSqVYlaVSpgZmzb\nl8nsNbt4Z9Y6KkWE0qd1fQZ3bUR8TKVgly5SqhUn3DcDDU5Yjs1bd1wVoCXwrZkB1AWmmFkf51zq\niQdyzo0ERgKkpKS486hbyrhDR7N5/evVjJ69HuccPVrW4+Z2sXSIr0FkeOhJbY9m5zBvwx4+nr+Z\nyQsymJi6iQEdGvLo1YlUiwoP0isQKd3MucIz1szCgFVAd3yh/jPwa+fc0gLafws8emqwnyolJcWl\nphbaRDxq3oY9PPJhGht/OcwNbWJ55Kom1I+OKta+Ow5k8n9fp/P+3I3UqlyBl29uRZeEWgGuWKT0\nMLN5zrkir0gs8g1V51w2cD/wBbAc+NA5t9TMhplZn/MvVcqT935cT7+3fyQn1zF+UCf+t19ysYMd\noHaVSJ6/viUf33cJlSqEcvs7c3njm9UU1UkRKW+K7LkHinru5UtOruPPU5cy9scNdG9am+G3tKZK\n5PkNqWRm5fDkpEV8kraFPskX8MrNyUSE6b488bbi9tx16YEEXHZOLg9/uJCpC7cwqEs8T/ZsRmiI\nnfdxI8ND+Vv/1jSpW4W/fr6Sg0ezefPWtqeN2YuUR+rmSEBl5eTywPgFTF24hSd6NGXoNc39EuzH\nmRlDLm/MC31bMmPlDn7z7s9kZuX47fgiZZXCXQLGOccTkxYxffE2nrmmGfddflHAznVrxwt5+aZk\nfkjfzQMfLCA7Jzdg5xIpCxTuEjAvf7GSj+dv5pFfNeGeLo0Cfr6b2sXy7LXN+c+y7TzzyRK9ySrl\nmsbcJSDe+2kDb367hgEdGvL7KxqX2Hnv6hzProNHGTFjDRdER/FA94QSO7dIaaJwF7/7ae1unpuy\nlO5Na/P8dS3Iu7mtxDx6VSJb9mby6peraFavKr9qXqdEzy9SGmhYRvxqy94j/G7cfC6sWZHht7Qm\nLLTkf8TMjP93QxKtYqvx8IQ00nccKPEaRIJN4S5+k5mVw73/msfR7FxG3p5y3texn4/I8FDeuq0d\nkeEhDBo7j/2ZWUGrRSQYFO7iN89PW8aijH282i+ZxrUrB7scLoiO4s1b27Hxl8MMnaw3WKV8UbiL\nX3y+ZBvj5mxkcNdGXNWibrDLydchvgaP/KoJUxduYWJqRrDLESkxCnc5b1v3HeHJjxeRVL8aj16V\nGOxyTnPvZRdxyUU1eXbKUo2/S7mhcJfzkpPreHhCGseyc3ntltalcm6X0BDjb/1bExURyv3vL9Ad\nrFIulL7fRClT3vpuDT+t/YXn+rSgUa3gj7MXpE7VSP735mRWbDvAq1+uCnY5IgGncJdztnLbAYZ/\ntYprWvketFHadWtam193bMiomWtJXf9LsMsRCSiFu5yTnFzH45MWUSUynGF9Sv5GpXP1dK9m1I+O\n4g8TF3L4WHawyxEJGIW7nJPRP6xj4aa9PNenBTUrVwh2OcVWuUIYL9+UzIbdh3npsxXBLkckYBTu\nctY27D7EK/9ZyZXNanNtq3rBLuesXXxRTQZeEse7P25gdvquYJcjEhAKdzkrzjmenLSY8JAQnr++\nZZkZjjnVEz2aEh9Ticc+WsShoxqeEe9RuMtZmfDzJn5cu5unejWjXrXiP/u0tImKCOWvN7Vi894j\nvPKflcEuR8TvFO5SbNv2ZfLCv5fTqVENbmnfINjlnLf2cTW4vdOFjJm9ngUb9wS7HBG/UrhLsTjn\neOaTJWTl5vLiDa0I8eOj8oLp8R6J1KkSyZOTFnMsW09vEu9QuEuxTFu0la+Wb+cPv0okLqZSsMvx\nmyqR4fzP9S1Zuf0Ab323JtjliPiNwl2K9MuhYzw3ZSnJsdW4q3NcsMvxuyub16F3q3q88U265p4R\nz1C4S5Gen7aMfUeyeOmmVkF5+EZJePbaFkRFhPLEpMXk5mpqYCn7vPmbKn4zY8UOJi/YzJBujWla\nt2qwywmYWlUq8MfezZm3YQ/j5mwIdjki503hLgU6kJnF0MmLaVKnMr/rdlGwywm4G9vWp0tCDC9+\ntoIte48EuxyR86JwlwK99PkKtu7P5KUbW1EhLDTY5QScmfGXvknkOnjmEz25Sco2hbuc0Zy1u/nX\nTxu5u3M8bRpWD3Y5JaZBjYr84aomfLNiB1MWbgl2OSLnTOEup8nMyuHJjxfTMC/oypu7OseT3CCa\nP09dxu6DR4Ndjsg5UbjLaYZ/tZp1uw7x4g1JVIwIC3Y5JS40xPjrja04kJnFsGnLgl2OyDlRuMtJ\nFmfsY9TMtdzSvgGXNI4JdjlBk1i3Cr/r1phP07bw9fLtwS5H5KwVK9zNrIeZrTSzdDN78gzbHzGz\nZWa2yMy+NrML/V+qBNqx7Fwe+2ghNStF8FSvZsEuJ+iGXN6YxDpVGDp5CQcys4JdjshZKTLczSwU\nGAH0BJoDA8ys+SnNFgApzrlWwEfAX/1dqATem9+ms2LbAf7SN4lqUeHBLifoIsJCeOmmVuw4kMmL\nerCHlDHF6bl3ANKdc2udc8eA8cB1JzZwzs1wzh3OW/wJKP0P1JSTLN+6nze+Sef61hdwZfM6wS6n\n1GjdIJq7O8czbs5Gflq7O9jliBRbccK9PrDphOWMvHUF+Q3w2Zk2mNlgM0s1s9SdO3cWv0oJqOyc\nXB7/aBHRFcN59toWwS6n1HnkqiY0rFGRJyctIjMrJ9jliBSLX99QNbPbgBTg5TNtd86NdM6lOOdS\natWq5c9Ty3kYOXMtizfvY9h1LaleKSLY5ZQ6FSPCePGGJNbvPszfvloV7HJEiqU44b4ZOPHJDLF5\n605iZlcCQ4E+zjldHFxGpO84wPCvVtMrqS69ksre81BLyiWNY7ilfQNGfb+WhZv2BrsckSIVJ9x/\nBhLMLN7MIoBbgCknNjCzNsDb+IJ9h//LlEA4PhxTMSKUP/dpGexySr2nejWjdpVIHvkwTcMzUuoV\nGe7OuWzgfuALYDnwoXNuqZkNM7M+ec1eBioDE80szcymFHA4KUXe+m4N8zfu5c99WlCrSoVgl1Pq\nVYsK55Wbk1mz85CunpFSr1i3HzrnpgPTT1n3pxO+vtLPdUmALcrYy/CvVtO7VT36JF8Q7HLKjEsT\nYhh4SRxjZq/nymZ1uDSh/N7oJaWb7lAth44cy+GhCWnEVK7AC9cnYeaN56GWlCd6NOWiWpV47KOF\n7Duim5ukdFK4l0MvfractTsP8b/9kqlWUTcrna2oiFD+1r81Ow8c5dlPlwS7HJEzUriXM9+t2sm7\nP27g7s7xdC7Hc8ecr1ax0fz+igQ+SdvCp2mnXTwG+7fC1Afhjfbw9mUw4y9wZE/JFyrllsK9HNm+\nP5NHJqTRpE5lHu+RGOxyyrzfdbuIlAur8/THi1m369B/N2xfBn+/BBaOhxoXQURl+O6vvqBfNzN4\nBUu5onAvJ7JzcnnggwUcPpbDiF+3JTLc+09WCrSw0BBeH9CG8LAQhoyb77s8MnMfjLsJwirAvbPg\n1+Phrn/Db7+DqOow9jpY8nGwS5dyQOFeTrz29WrmrPuF/7m+JQl1qgS7HM+4IDqKV/sls3zrfp6f\ntgy+fh4ObIX+4yAm4b8N6yXDPV9Dgw4w6R5YPi14RUu5oHAvB75ftZM3ZqRzc7tYbmynOd387Yqm\ndfht10Z8N3ceuan/hJTfQGy70xtGVoVbJ8IFbXwBvyWt5IuVckPh7nEZew7z0IQ0EmpXZth1ugs1\nUB69OpGnor8mJxfWNR1UcMMKVWDAeKhYE8bfCgd1Q7cEhsLdww4fy2bQ2HlkZefy5q3tiIrQOHug\nhOceo2fODL4M6czAj7ew9/CxghtXrgUD3ofDu2HSbyBXUxmI/yncPco5x6MTF7Ji235e/3UbGteu\nHOySvG3VZ4QcO0DjqwazdW8m97+/gOyc3ILb10uGXi/Duu/hh9dKrk4pNxTuHvV/36QzffE2nurZ\nlG6JtYNdjvct/giq1KNJx178T9+WzErfxQvTlxe+T5vboEVfmPECZKSWTJ1SbijcPWjyggxe/XIV\nfdvUZ1CXRsEux/uyj8Kab6DpNRASSr+UBtzdOZ7RP6xn9A/rCt7PDHoPhyoXwEd3Q+b+kqtZPE/h\n7jHfrdrJYxMXcXGjmrx4o+aNKREbfoCsw5Bwdf6qodc0o0eLugybtoxpi7YUvG9UNNz4D9iXAdMf\nK4FipbxQuHvIooy93PeveSTUqcLbd7SjQpjeQC0Rq7+EsEiIuzR/VWiIMfyW1rS/sAaPTFjI7PRd\nBe/fsCN0fQwWjYclk0qgYCkPFO4esXr7Ae4a/TM1KkXw7l3tqRqpCcFKzPpZvpuTIiqetDoyPJRR\nd6QQF1ORwe/NY8HGQuaW6foY1E+BaQ/7evEi50nh7gGrtx9gwKifCA0x3vtNR2pXjQx2SeXH0QOw\nfQk0vPiMm6tVDOfduztQs3IEd7wzl/kFBXxoGNwwEnKyYfK9ujxSzpvCvYw7HuwhZnwwuBPxMZWC\nXVL5kvEzuFxo2KnAJvWqRTF+cCdqVI7gzsICvuZF0PMlWD8TfnwjQAVLeaFwL8MWbNxD/5H/DfaL\naula9hK38SewEIhtX2iz4wF/vAc/e00BY/BtboNm1/rmqNm6MAAFS3mhcC+jvlmxnV+PmkPlCmF8\n+NuLFezBsvFHqNPSN61AEXwBfzEXREdy5z/nMmXhGa6iMYNrX/dNTzBpEBw7HICipTxQuJdB4+Zs\nYNDYeVxUuxKT7ruEOA3FBIdzvsm/YlOKvUvdapFMvPcS2jaszgMfLOCt79bgnDu5UcUa0PfvsGsl\nfPmnMx9IpAgK9zLkaHYOT328iKGTl3Bp4xjGD76YWlUqBLus8mvvBji6H+q2OqvdqkWFM/Y3Hejd\nqh4vfraC33+wgMPHsk9udNEV0GkI/DwKVv3Hj0VLeaFwLyM27z3CLSN/4oO5mxhy+UX8c2B7KlcI\nC3ZZ5du2xb7PZxnuABXCQvm/AW14vEci0xdvpe+I2azdefDkRt2fhdot4NMhcHCnHwqW8kThXso5\n55i8IIMef/ueVdsO8OatbXm8R1NCQ3TnadBtW+x7M7V2s3Pa3cwYcnljxt7dkR0HMrnm9Vm899OG\n/w7ThEfCjaN80xJMud83DCRSTAr3UmzHAd/sgg9PWEhi3Sp89mBXeiXVC3ZZcty2xVCz8Wk3L52t\nSxNi+OzBrqTEVeePnyzhztE/s3nvEd/GOi3gyudg1eeQ+s/zLlnKD4V7KZSdk8voH9bR/ZXv+HLZ\ndh67OpEJv72YhjXPL0TEz7YthrpJfjlU3WqRjL27A89f14K563Zz5f9+x4gZ6RzNzoGO90KjbvDF\n07BlgV/OJ96ncC9FnHN8sXQb17w+iz9PXUbrhtF88XBXftetsYZhSpsje2HfJt9lkH5iZtx+cRxf\nPXIZXZvE8PIXK+kxfCZTFm8jt+9IqBgDE26HQ4XMUyOSR+FeCuTmOmas2MF1I37gt+/NIysnl7/f\n2paxd3fQHael1a7Vvs/nON5emNjqFXn79hTG3NWeiNAQHvhgAT3+sZxZKcNxB3fAxIG+aQpECqHL\nLYLo0NFsJs3PYMzs9azdeYj60VH89aZW3NCmPmGh+rtbqu1a5fsc0yRgp7g8sTZdE2rx78VbGf7V\nKm6bfox7qv6WZ9a/ztHPhlKh90sBO7eUfQr3Epadk8vM9F18umAz/1m2ncPHckiOrcbw/q3plVSP\niDCFepmwezWEhEP0hQE9TUgM+oLeAAAGj0lEQVSIcW3yBfRKqscXS7cxZnYNRm9ayV2pbzFhYwTR\nl/2OyxNraXpnOY3CvQTsOXSM71fv5LuVO/l21U5+OXSMqpFhXNf6Am5OaUDbhtWDXaKcrV2roUYj\n32yOJSA0xOiVVI9eSfVYlpHIig9v46Ydb3DvuDAejehEt8TadG1Si64JMZoVVIBihruZ9QBeA0KB\nfzjnXjxlewVgLNAO2A30d86t92+pZUNOrmPdroMs2LiXBZv2smDjXlZs249zUL1iOF0SatG7VT0u\nU2+rbNu1KqBDMoVpHlsD7p9I7pjevLX9Td5okMB7a0Ly56ppUqcybRtWp3WDaFo3jKZxrcoa5iuH\n7LR5LU5tYBYKrAJ+BWQAPwMDnHPLTmgzBGjlnLvXzG4B+jrn+hd23JSUFJeaWjYfCnw0O4ft+46y\ndd8Rtu3PZN2uQ6TvOEj6joOs23WIo9m+p95XiQyjdYNoUi6swWWJtUiqX01XvXhBTha8UBcueQCu\nfDZ4dRzcCe9cCUf2kHv7pyyjEd+v3smctb+Qtmkv+45kARARGkJcTEUa165M41qViYupRL1qUdSr\nFkndapFEhquTUZaY2TznXJETGhWn594BSHfOrc078HjgOmDZCW2uA57L+/oj4A0zM1fUX47z5Jwj\n1/l6y7nO9+H72ncFSq5z5DhHbi7525yD7NxcMrNyyczO4Wj+5xzfuqwcMrNyOHg0m31Hsth7OMv3\n+UgW+49ksevgUXYdPHZSHWYQWz2KxrUq0yUhhoQ6VWjbMJpGMZUJUZh7z571kJsdtJ57vsq14I4p\nMOYaQv7Vl5Z3TqXl5UkMudz3u7Fu1yHSNu1l5fYDrNlxkGVb9vP5km3knvJbWaNSBDGVI4iOiqBa\nxXCio8KJrhhOtahwKkaEERURSlR4KJHhoUSGhxAVHkpURCgRYSGEhRihIcc/2wmfQwgN9S2HmHH8\n18DMMHy/M8eXJTCKE+71gU0nLGcAHQtq45zLNrN9QE3A7xfk/jjueZJWFfwgA8M3duSPvojl/Z9h\nmJH/Q2mVjv+A+n5ozQzLBrbmfYi3HcubA6ZOi+DWAVD9QrhzKoy5Bt6+DMKjAN/PaqO8jxO5Kr7g\nd87X4XHkLe/H9+HA4cj733nz9wWbXvlTsLTVU3S48aGAnqNE31A1s8HA4LzFg2a2siTP7ycxBOCP\nVilX3l5z8V7vn5MDX8lZK+Q5rYUrb99jCOprfjjv45wU6xKt4oT7ZqDBCcuxeevO1CbDzMKAavje\nWD2Jc24kMLI4hZVWZpZanPEuLylvr7m8vV7Qa/ai4ryF/jOQYGbxZhYB3AJMOaXNFODOvK9vAr4J\n9Hi7iIgUrMiee94Y+v3AF/iGsv/pnFtqZsOAVOfcFOAd4D0zSwd+wfcHQEREgqRYY+7OuenA9FPW\n/emErzOBm/1bWqlVpoeVzlF5e83l7fWCXrPnFHmdu4iIlD26bU1ExIMU7ufBzP5gZs7MYoJdSyCZ\n2ctmtsLMFpnZZDOLDnZNgWJmPcxspZmlm9mTwa4n0MysgZnNMLNlZrbUzB4Mdk0lxcxCzWyBmU0L\ndi2BoHA/R2bWALgK2BjsWkrAl0BL51wrfFNRPBXkegIib6qNEUBPoDkwwMyaB7eqgMsG/uCcaw50\nAn5XDl7zcQ8Cy4NdRKAo3M/d34DH8c+NfKWac+4/zrnjNxv+hO9eBy/Kn2rDOXcMOD7Vhmc557Y6\n5+bnfX0AX9jVD25VgWdmscA1wD+CXUugKNzPgZldB2x2zi0Mdi1BcDfwWbCLCJAzTbXh+aA7zszi\ngDbAnOBWUiKG4+uc5Qa7kEDRfO4FMLOvgLpn2DQUeBrfkIxnFPZ6nXOf5rUZiu+f8eNKsjYJPDOr\nDEwCHnLO7Q92PYFkZr2BHc65eWZ2ebDrCRSFewGcc1eeab2ZJQHxwMK8Ge1igflm1sE5t60ES/Sr\ngl7vcWY2EOgNdPfw3cfFmWrDc8wsHF+wj3POfRzsekpAZ6CPmfUCIoGqZvYv59xtQa7Lr3Sd+3ky\ns/VAinPOs5Mu5T2s5VXgMufczmDXEyh58yKtArrjC/WfgV8755YGtbAAMl8P5V3gF+dcYKcpLIXy\neu6POud6B7sWf9OYuxTHG0AV4EszSzOzt4JdUCDkvWl8fKqN5cCHXg72PJ2B24Er8r63aXk9Winj\n1HMXEfEg9dxFRDxI4S4i4kEKdxERD1K4i4h4kMJdRMSDFO4iIh6kcBcR8SCFu4iIB/1/W7I+x3+c\nfagAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7hIo27GmiiUE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# copy the learned weights over into a tmp array\n",
        "weightsD=sess.run(theta)\n",
        "\n",
        "# close the pre-training session\n",
        "sess.close()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZwSxZ_p2iiUG",
        "colab_type": "text"
      },
      "source": [
        "Now to build the actual generative adversarial network\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qJQ_6dstiiUH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with tf.variable_scope(\"G\"):\n",
        "    z_node=tf.placeholder(tf.float32, shape=(M,1)) # M uniform01 floats\n",
        "    G,theta_g=mlpg(z_node,1) # generate normal transformation of Z\n",
        "    G=tf.multiply(5.0,G) # scale up by 5 to match range\n",
        "\n",
        "with tf.variable_scope(\"D\") as scope:\n",
        "    # D(x)\n",
        "    x_node=tf.placeholder(tf.float32, shape=(M,1)) # input M normally distributed floats\n",
        "    fc,theta_d=mlp(x_node,1) # output likelihood of being normally distributed\n",
        "    D1=tf.maximum(tf.minimum(fc,.99), 0.01) # clamp as a probability\n",
        "    # make a copy of D that uses the same variables, but takes in G as input\n",
        "    scope.reuse_variables()\n",
        "    fc,theta_d=mlp(G,1)\n",
        "    D2=tf.maximum(tf.minimum(fc,.99), 0.01)\n",
        "\n",
        "obj_d=tf.reduce_mean(tf.log(D1)+tf.log(1-D2))\n",
        "obj_g=tf.reduce_mean(tf.log(D2))\n",
        "\n",
        "# set up optimizer for G,D\n",
        "opt_d=momentum_optimizer(1-obj_d, theta_d)\n",
        "opt_g=momentum_optimizer(1-obj_g, theta_g) # maximize log(D(G(z)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cO59EYD6iiUJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sess=tf.InteractiveSession()\n",
        "tf.global_variables_initializer().run()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bh1nDcYKiiUL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# copy weights from pre-training over to new D network\n",
        "for i,v in enumerate(theta_d):\n",
        "    sess.run(v.assign(weightsD[i]))\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G4eyGnFaiiUN",
        "colab_type": "code",
        "outputId": "ecc11c0d-bdc6-4e9c-f98e-89b0d4908116",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        }
      },
      "source": [
        "def plot_fig():\n",
        "    # plots pg, pdata, decision boundary \n",
        "    f,ax=plt.subplots(1)\n",
        "    # p_data\n",
        "    xs=np.linspace(-5,5,1000)\n",
        "    ax.plot(xs, norm.pdf(xs,loc=mu,scale=sigma), label='p_data')\n",
        "\n",
        "    # decision boundary\n",
        "    r=5000 # resolution (number of points)\n",
        "    xs=np.linspace(-5,5,r)\n",
        "    ds=np.zeros((r,1)) # decision surface\n",
        "    # process multiple points in parallel in same minibatch\n",
        "    for i in range(int(r/M)):\n",
        "        x=np.reshape(xs[M*i:M*(i+1)],(M,1))\n",
        "        ds[M*i:M*(i+1)]=sess.run(D1,{x_node: x})\n",
        "\n",
        "    ax.plot(xs, ds, label='decision boundary')\n",
        "\n",
        "    # distribution of inverse-mapped points\n",
        "    zs=np.linspace(-5,5,r)\n",
        "    gs=np.zeros((r,1)) # generator function\n",
        "    for i in range(int(r/M)):\n",
        "        z=np.reshape(zs[M*i:M*(i+1)],(M,1))\n",
        "        gs[M*i:M*(i+1)]=sess.run(G,{z_node: z})\n",
        "    histc, edges = np.histogram(gs, bins = 10)\n",
        "    ax.plot(np.linspace(-5,5,10), histc/float(r), label='p_g')\n",
        "\n",
        "    # ylim, legend\n",
        "    ax.set_ylim(0,1.1)\n",
        "    plt.legend()\n",
        "    \n",
        "# initial conditions\n",
        "plot_fig()\n",
        "plt.title('Before Training')"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Before Training')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8FNX+//HXZ9OBJJQERGoQCC2E\nEooiCHpVQESxgAWxXMGv5RpAEQQryr1gQfQKV0B/KFcUVERpei2ANAHpNfQAoQYkIUASkuz5/bGb\nmIR0sjvJ5vPUPLI7c3bmMyR57+yZmTNijEEppZRnsVldgFJKqdKn4a6UUh5Iw10ppTyQhrtSSnkg\nDXellPJAGu5KKeWBNNxVmSYiT4rISRE5LyI1rK6nOETkYxEZXdptlSoK0fPclSuJSCxQC8gA0oDV\nwP8ZY44U4bU+wDmgszFmi4vr/AgY6HzqCwiQ6ny+whjTy5XrV6q0abgrl3KG++PGmF9ExB+YAlQ3\nxtxZhNfWBY4APsaY9GKuV3D8fttLUPNrQGNjzMAC2ngXtyal3Em7ZZTbGGNSgG+AFpnTRMRPRN4R\nkcPO7pePRCRARJoCu53NEkRkibP9dSLyh4gkOr9fl21Zy0RknIisAi4CjUQkWEQ+EZHjInJURN4U\nEa/i1i4ijUXEiMijInIY+ElEbCLyjYicEJEE5/qbZ3vN5843CkTkbyISKyIviEi8iBwTkUElbBsq\nIotE5JyIrBORf4rIsuJuk/JsGu7KbUSkEjAAWJNt8nigKdAGaAzUAV4xxuwBWjrbVDXG3Cgi1YFF\nwAdADWAisChXX/xDwBAgEDgEfAqkO5fdFrgFePwKNqMb0Ay4zfl8IdAEuArYDvy3gNfWBQKAq4H/\nA/4jIkElaPsfIAFHd9djwMMl3RjluTTclTt8JyIJQCJwM/A2ZHWdDAGGGWP+NMYkAf8E7stnObcB\ne40x/zXGpBtjvgRigNuztfnUGLPD2WVSHegNDDXGXDDGnALeK2D5RfGqMeaiMSbZGGM3xnxqjEly\nfip5DWgvIpXzeW0K8KYxJs0YMx9Hn37T4rR1Hoe4E8cbYLIxprA3FFVBeVtdgKoQ7nT2uXsBdwC/\niUgLwA5UAjY4ch5wHMjMr9vkahx749kdwrG3nyn7gdoGgA9wPNvybbnaFFfWa53b8y/gHiAEx/bg\nfHwhj9eeNsZkZHt+EaiSz3rya1sLx79P9m04AnQuxjaoCkD33JXbGGMyjDHf4jhz5nrgNJAMtDTG\nVHV+BRtj8gu8YzgCO7v6wNHsq8n2+AiOPd6QbMsPMsa0pIRMzjMQBuH4ZHAjEIyj6wccb1CuchLH\nm0jdbNPquXB9qpzScFduIw53ANWAXc4zWaYD74lITWebOiJyaz6LWIyja+IBEfEWkQE4Ds4uzKux\nMeY48BPwrogEOQ+AXiMiN5TSJgXiePM4g+MTyLhSWm6+jDFpwHfA684Dzy356xROpbJouCt3WCAi\n53Gcsz4OeNgYs8M5bySwD1gjIueAX4DwvBZijDkD9AGewxGoLwB9jDGnC1j3IBznre8EzuI4W6f2\nFW+RwwwcnyaOATtwnMPvDk/iOKB80lnDl/x1Tr5SgJ7nrlS5JyLv4jij6O9W16LKDt1zV6qcEZEW\nIhLh7ObqDDwKzLO6LlW26NkySpU/QcAsHN1LJ4Hxxpg8jzuoiku7ZZRSygNpt4xSSnkgy7plQkJC\nTMOGDa1avVJKlUsbNmw4bYwJLaydZeHesGFD1q9fb9XqlVKqXBKR3Fdp50m7ZZRSygNpuCullAfS\ncFdKKQ+k57krVYalpaURFxdHSkqK1aUoN/P396du3br4+PiU6PUa7kqVYXFxcQQGBtKwYUOyDVus\nPJwxhjNnzhAXF0dYWFiJlqHdMkqVYSkpKdSoUUODvYIREWrUqHFFn9g03JUq4zTYK6Yr/blruCul\nlAfScFdKKQ+k4a6Ucolly5bRp0+fAtts3ryZxYsXu6miikXDXSllGQ1319FTIZUqJ15fsIOdx86V\n6jJbXB3Eq7cXfL/w2NhYevbsSfv27dm4cSMtW7Zk5syZVKpU6bK2P/74I0OHDqVSpUpcf/31WdPX\nrVtHdHQ0KSkpBAQEMGPGDMLCwnjllVdITk5m5cqVvPjii4SFhV3WLjw8z7suqkLonrtSqlC7d+/m\nqaeeYteuXQQFBTFlypTL2qSkpDB48GAWLFjAhg0bOHHiRNa8Zs2asWLFCjZt2sTYsWMZPXo0vr6+\njB07lgEDBrB582YGDBiQZztVMrrnrlQ5UdgetivVq1ePLl26ADBw4EA++OADnn/++RxtYmJiCAsL\no0mTJlntpk2bBkBiYiIPP/wwe/fuRURIS0vLcz1FbacKp3vuSqlC5T7nurjnYL/88sv06NGD7du3\ns2DBgnwvzilqO1W4QsNdRP6fiJwSke35zBcR+UBE9onIVhFpV/plKqWsdPjwYX7//XcAvvjiixz9\n6ZmaNWtGbGws+/fvB+DLL7/MmpeYmEidOnUA+PTTT7OmBwYGkpSUVGg7VXxF2XP/FOhZwPxeQBPn\n1xDgP1dellKqLAkPD2fy5Mk0b96cs2fP8uSTT17Wxt/fn2nTpnHbbbfRrl07atasmTXvhRde4MUX\nX6Rt27akp6dnTe/Rowc7d+6kTZs2zJkzJ992qviKdINsEWkILDTGtMpj3lRgmTHmS+fz3UB3Y8zx\ngpYZFRVl9E5MShVs165dNG/e3NIaYmNj6dOnD9u35/nhXblQXj9/EdlgjIkq7LWl0edeBziS7Xmc\nc9plRGSIiKwXkfXx8fGlsGqllFJ5cevZMsaYacA0cOy5u3PdSqmSadiw4WV77f369ePgwYM5pk2Y\nMIFbb73VnaWpApRGuB8F6mV7Xtc5TSnloebNm2d1CaoQpdEtMx8Y5DxrpjOQWFh/u1JKKdcqdM9d\nRL4EugMhIhIHvAr4ABhjPgIWA72BfcBF4FFXFauUUqpoCg13Y8z9hcw3wNOlVpFSSqkrpleoKqWK\n7LXXXuOdd94p0Wuvu+66Auf37t2bhISEEi07u0ceeYRvvvnmipdTHFWqVHHr+opCx5ZRSrnF6tWr\nC5xfUYb+NcZgjMFmc+2+te65K6UKNG7cOJo2bcr111/P7t27s6bv378/ayjgrl27EhMTA8DJkyfp\n168fkZGRREZGZoV65t7t8ePH6datG23atKFVq1asWLECcJxyefr0aQAmTpxIq1ataNWqFZMmTQIc\nF1M1b96cwYMH07JlS2655RaSk5PzrPmXX34hKiqKpk2bsnDhQsAxauWjjz5KREQEbdu2ZenSpYBj\nmINnnnkm67V9+vRh2bJlWTWPGTOGyMhIOnfuzMmTJwE4ePAg1157LREREbz00ktZrz1//jw33XQT\n7dq1IyIigu+//z6r9vDwcAYNGkSrVq144403GDp0aNbrpk+fzrBhw4r9symI7rkrVV78MApObCvd\nZV4VAb3G5zt7w4YNzJ49m82bN5Oenk67du1o3749AEOGDOGjjz6iSZMmrF27lqeeeoolS5bw7LPP\ncsMNNzBv3jwyMjI4f/58jmV+8cUX3HrrrYwZM4aMjAwuXrx42TpnzJjB2rVrMcbQqVMnbrjhBqpV\nq8bevXv58ssvmT59Ov3792fu3LkMHDjwsrpjY2NZt24d+/fvp0ePHuzbt4/JkycjImzbto2YmBhu\nueUW9uzZU+A/z4ULF+jcuTPjxo3jhRdeYPr06bz00ktER0fz5JNPMmjQICZPnpzV3t/fn3nz5hEU\nFMTp06fp3Lkzffv2BWDv3r189tlndO7cmfPnzxMZGcnbb7+Nj48PM2bMYOrUqQX/rIpJw10pla8V\nK1bQr1+/rBtzZAbV+fPnWb16Nffee29W29TUVACWLFnCzJkzAfDy8iI4ODjHMjt06MBjjz1GWloa\nd955J23atMkxf+XKlfTr14/KlSsDcNddd7FixQr69u1LWFhYVvv27dsTGxubZ939+/fHZrPRpEkT\nGjVqRExMDCtXruQf//gH4BjkrEGDBoWGu6+vb9atAtu3b8/PP/8MwKpVq5g7dy4ADz30ECNHjgQc\nXS6jR49m+fLl2Gw2jh49mrW336BBAzp37gw4PhHceOONLFy4kObNm5OWlkZERESBtRSXhrtS5UUB\ne9juZrfbqVq1Kps3by72a7t168by5ctZtGgRjzzyCMOHD2fQoEFFeq2fn1/WYy8vr3y7ZYozRLG3\ntzd2uz3refZhhn18fLJe6+XllWMws7yWOWvWLOLj49mwYQM+Pj40bNgwa3mZb1aZHn/8cf75z3/S\nrFkzHn209M8g1z53pVS+unXrxnfffUdycjJJSUksWLAAgKCgIMLCwvj6668Bxx7rli1bALjpppv4\nz38cg8NmZGSQmJiYY5mHDh2iVq1aDB48mMcff5yNGzfmmN+1a1e+++47Ll68yIULF5g3bx5du3Yt\nVt1ff/01drud/fv3c+DAAcLDw+natSuzZs0CYM+ePRw+fJjw8HAaNmzI5s2bsdvtHDlyhHXr1hW6\n/C5dujB79myArGWCY8jimjVr4uPjw9KlSzl06FC+y+jUqRNHjhzhiy++4P77CzzjvEQ03JVS+WrX\nrh0DBgwgMjKSXr160aFDh6x5s2bN4pNPPiEyMpKWLVtmHTx8//33Wbp0KREREbRv356dO3fmWOay\nZcuIjIykbdu2zJkzh+jo6MvW+cgjj9CxY0c6derE448/Ttu2bYtVd/369enYsSO9evXio48+wt/f\nn6eeegq73U5ERAQDBgzg008/xc/Pjy5duhAWFkaLFi149tlnadeu8FtSvP/++0yePJmIiAiOHv1r\ntJUHH3yQ9evXExERwcyZM2nWrFmBy+nfvz9dunShWrVqxdq+oijSkL+uoEP+KlW4sjDkr3KdPn36\nMGzYMG666aY851s95K9SSqliSEhIoGnTpgQEBOQb7FdKD6gqpZSbVa1atdAzda6U7rkrpZQH0nBX\nSikPpOGulFIeSMNdKaU8kIa7Ukp5IA13pZTyQBruSinlgfQ8d6XKiQnrJhDzZ0ypLrNZ9WaM7Diy\nwDaxsbFZ47Zv3LiRli1bMnPmzKyRIrNbvHgxw4cPp3LlynTp0oUDBw5kjaeu3Ev33JVShdq9ezdP\nPfUUu3btIigoiClTplzWJiUlhSeeeIIffviBDRs2EB8fb0GlKpPuuStVThS2h+1K9erVo0uXLgAM\nHDiQDz74gOeffz5Hm5iYGBo1akRYWBgA999/P9OmTXN7rcpB99yVUoUqzvjoqmzQcFdKFerw4cP8\n/vvvgOM2eddff/1lbcLDwzlw4EDW3ZHmzJnjzhJVLhruSqlChYeHM3nyZJo3b87Zs2d58sknL2sT\nEBDAlClTsg6+BgYGXnaLPeU+2ueulCqUt7c3n3/+eaHtevToQUxMDMYYnn76aaKiCh12XLmI7rkr\npUrN9OnTadOmDS1btiQxMZEnnnjC6pIqLN1zV0oVqGHDhmzfvj3HtH79+nHw4MEc0yZMmMCwYcMY\nNmyYO8tT+dBwV6qMM8aUubNT5s2bZ3UJHu9Kb4Gq3TJKlWH+/v6cOXPmiv/QVflijOHMmTP4+/uX\neBm6565UGVa3bl3i4uL0as8KyN/fn7p165b49RruSpVhPj4+WVd8KlUcReqWEZGeIrJbRPaJyKg8\n5tcXkaUisklEtopI79IvVSmlVFEVGu4i4gVMBnoBLYD7RaRFrmYvAV8ZY9oC9wGXjyqklFLKbYqy\n594R2GeMOWCMuQTMBu7I1cYAQc7HwcCx0itRKaVUcRUl3OsAR7I9j3NOy+41YKCIxAGLgX/ktSAR\nGSIi60VkvR4gUkop1ymtUyHvBz41xtQFegP/FZHLlm2MmWaMiTLGRIWGhpbSqpVSSuVWlHA/CtTL\n9ryuc1p2fwe+AjDG/A74AyGlUaBSSqniK0q4/wE0EZEwEfHFccB0fq42h4GbAESkOY5w134XpZSy\nSKHhboxJB54B/gfswnFWzA4RGSsifZ3NngMGi8gW4EvgEaOX1CmllGWKdBGTMWYxjgOl2ae9ku3x\nTqBL6ZamlFKqpHRsGaWU8kAa7kop5YE03JVSygNpuCullAfScFdKKQ+k4a6UUh5Iw10ppTyQhrtS\nSnkgDXellPJAGu5KKeWBNNyVUsoDabgrpZQH0nBXSikPpOGulFIeSMNdKaU8kIa7Ukp5IA13pZTy\nQBruSinlgTTclVLKA2m4K6WUB9JwV0opD6ThrpRSHkjDXSmlPJCGu1JKeSANd6WU8kAa7kop5YE0\n3JVSygNpuCullAfScFdKKQ+k4a6UUh6oSOEuIj1FZLeI7BORUfm06S8iO0Vkh4h8UbplKqWUKg7v\nwhqIiBcwGbgZiAP+EJH5xpid2do0AV4EuhhjzopITVcVrJRSqnBF2XPvCOwzxhwwxlwCZgN35Goz\nGJhsjDkLYIw5VbplKqWUKo6ihHsd4Ei253HOadk1BZqKyCoRWSMiPfNakIgMEZH1IrI+Pj6+ZBUr\npZQqVGkdUPUGmgDdgfuB6SJSNXcjY8w0Y0yUMSYqNDS0lFatlFIqt6KE+1GgXrbndZ3TsosD5htj\n0owxB4E9OMJeKaWUBYoS7n8ATUQkTER8gfuA+bnafIdjrx0RCcHRTXOgFOtUSilVDIWGuzEmHXgG\n+B+wC/jKGLNDRMaKSF9ns/8BZ0RkJ7AUGGGMOeOqopVSShVMjDGWrDgqKsqsX7/eknUrpVR5JSIb\njDFRhbXTK1SVUsoDabgrpZQH0nBXSikPpOGulFIeSMNdKaU8kIa7Ukp5IA13pZTyQBruSinlgTTc\nlVLKA2m4K6WUB9JwV0opD6ThrpRSHkjDXSmlPJCGu1JKeSANd6WU8kAa7kop5YE03JVSygNpuCul\nlAfScFdKKQ+k4a6UUh5Iw10ppTyQhrtSSnkgDXellPJAGu5KKeWBNNyVUsoDabgrpZQH0nBXSikP\npOGulFIeSMNdKaU8kIa7Ukp5IA13pZTyQEUKdxHpKSK7RWSfiIwqoN3dImJEJKr0SlRKKVVchYa7\niHgBk4FeQAvgfhFpkUe7QCAaWFvaRSqllCoe7yK06QjsM8YcABCR2cAdwM5c7d4AJgAjSrVCVSGk\npGWwev9p1h74k/3xF4g/nwrGUKOKH2EhlekYVp3rG4dQ2a8ov7JKqaL8pdQBjmR7Hgd0yt5ARNoB\n9Ywxi0Qk33AXkSHAEID69esXv1rlceLOXuTjFQf5dmMc51LS8fWy0Si0MqGBfogIJxJTWL3/NJ+s\nPEhlXy/6tqnDkG6NCAupbHXpSpVpV7wbJCI2YCLwSGFtjTHTgGkAUVFR5krXrcqvC6npfPDrXmas\njsUYQ89Wtbm3fV06hlXH38crR9vU9Aw2HDrLtxuPMm9THF+vP8L9Hevz/K3hBAf4WLQFSpVtRQn3\no0C9bM/rOqdlCgRaActEBOAqYL6I9DXGrC+tQpXn2HDoLMO/2szhPy9yV9u6DL+lKXWqBuTb3s/b\ni+uuCeG6a0J4oWc4//51H1+sO8zPO0/y9r2t6dok1I3VK1U+FOVsmT+AJiISJiK+wH3A/MyZxphE\nY0yIMaahMaYhsAbQYFd5+u/vsfSf+jsZdsPswZ15t39kgcGeW81Af964sxXfPnkdlf28eOiTdXy4\nZC/G6AdBpbIrNNyNMenAM8D/gF3AV8aYHSIyVkT6urpA5Rky7IZXvt/Oy9/voHvTUH6I7kqnRjVK\nvLzIelVZ9GxX7mxzNe/8tIfo2Zu5lG4vxYqVKt+K1OdujFkMLM417ZV82na/8rKUJ0nPsDPsqy0s\n2HKMwV3DGNWrOV42ueLl+vt48d6ANjS9KpC3ftzN+dR0pjzY7rI+e6UqIr1CVblUWoadZ2dvYsGW\nY4zs2Ywxt7UolWDPJCI81b0x4/q1YunuU/z9sz9IScsoteUrVV5puCuXMcYwcu5WFm87wUu3NefJ\n7te4bF0PdmrA2/dEsmrfGZ79chPpGdpFoyo2DXflMm//bzffbjzK8Jub8njXRi5f3z3t6/Lq7S34\naedJXvpuux5kVRWaXu6nXOK/aw4xZdl+7u9Yn3/c2Nht6320Sxinz6cyeel+rq4awLM3NXHbupUq\nSzTcValbc+AMr83fwU3NavLGHS1xXv/gNs/fEs6xhBQm/ryH5rWDuLlFLbeuX6myQLtlVKk6lpDM\n07M20qBGJSbd1wZvL/f/iokI/7orgtZ1gxk2ZzP7TiW5vQalrKbhrkpNSloG//f5BlLT7Ux7KIpA\nf+uGBvD38eKjge3x97ExeOYGzqWkWVaLUlbQcFel5o2FO9kal8jE/pE0rlnF6nK4umoAUx5sz+E/\nLzJmnh5gVRWLhrsqFT9uP8GstYcZ0q0Rt7S8yupysnQMq87wm5uyYMsxvl4fZ3U5SrmNhru6YscT\nkxn17VYi6gTz/C3hVpdzmf+74Rquu6YGr87fof3vqsLQcFdXJMNuGDbHMa7L+/e1wde77P1KedmE\n9wa0IcDXi2e+2KRXsKoKoez9Japy5aPf9rPmwJ+81rcljUKt72fPT60gf969N5KYE0lM/HmP1eUo\n5XIa7qrEdp9IYtIve7itteNGG2Vdj2Y1eaBTfaavOMD62D+tLkcpl9JwVyWSYTe8MHcrgf4+jO3r\n/guVSmp07+bUqRrAc19v4eKldKvLUcplNNxVicxYdZAtRxJ4rW9LalTxs7qcIqvi583b90Ry6MxF\nJvwQY3U5SrmMhrsqtkNnLvDOT7v5W/Oa3N66ttXlFNu119Tgkesa8tnvh1i977TV5SjlEhruqliM\nMYyauw0fm4037mxVbrpjchvZsxlhIZUZ8c1WLqRq94zyPBruqljm/HGE3w+c4cXezakdXPR7n5Y1\nAb5evHVPa44mJPPOT7utLkepUqfhrorsRGIK4xbtonOj6tzXoZ7V5VyxDg2r81DnBny6OpZNh89a\nXY5SparchfvZlLN8s+cbq8uocIwxvPTddtLsdsbf1RpbKd4qz0ov9AynVqA/o+Zu0xtsK7eYEzOH\nc5fOuXw95S7cv4j5gtd/f51FBxZZXUqFsnDrcX7ZdZLnbg6nYUhlq8spNYH+Prx5Zyt2n0zio9/2\nW12O8nBf7f6KN9e+yby981y+rnIX7kNaD6FdzXa8tvo1Yv7UU9nc4c8Ll3ht/g4i6wbzaJeGVpdT\n6v7WohZ9WtfmwyX7dOwZ5TKbT23mX+v+xfV1rmdg84EuX1+5C3cfmw/vdn+XIL8ghi4dSkJKgtUl\nebw3Fu4kMTmNCfe0tuTmG+7w6u0tCfD1YuTcbdjtOjSwKl2nLp5i2LJh1K5cm/Fdx+Nl83L5Osvl\nX2pIQAiTuk/i1MVTjFg+gnS7nsrmKktjTjFv01Ge6tGYZlcFWV2Oy4QG+vFynxZsOHSWWWsPWV2O\n8iBpGWkMXzacC2kXeL/H+wT7BbtlveUy3AEiQiN4ufPLrDm+hg82fWB1OR4pKSWNMfO20bRWFZ7u\ncY3V5bjc3e3q0LVJCON/iOFYQrLV5SgPMX7deLbEb+GNLm/QpJr7bthebsMdoF+TfgwIH8CM7TP4\nMfZHq8vxOBN+jOH4uRQm3N0aP2/Xf4y0mojwz34R2A289J3euUldubl75vLVnq94rNVj3NrwVreu\nu1yHO8DIDiNpE9qGV1a9wp6zOpRraVl74AyfrznMY13CaFu/mtXluE296pV47pamLIk5xfwtx6wu\nR5VjW+O3Mm7tOK67+jqebfus29df7sPdx8uHid0nEugTSPSSaBJTE60uqdxLSctg1LfbqO8Muorm\n0S5hRNaryusLdnLmfKrV5ahy6HTyaYYtG0bNSjV5q9tbbjmAmlu5D3eA0EqhTOwxkRMXTzByxUgy\n7HqnnSsx6Ze9HDx9gfF3RVDJ19vqctzOyya8dXdrklLSGLtwp9XlqHImLSON55Y9R9KlJLceQM3N\nI8IdIDI0kjGdxrDq6Co+3Pyh1eWUW9viEpm+4gD3dajHdY1DrC7HMuFXBfJ0j8Z8v/kYv+46aXU5\nqhx564+32HhqI69f9zrh1a27p3CRwl1EeorIbhHZJyKj8pg/XER2ishWEflVRBqUfqmFu6fpPdzT\n9B4+3vYxP8X+ZEUJ5dqldDsjvtlCjcq+vNi7udXlWO6p7o0JrxXImHnbSUpJs7ocVQ7M2zuP2btn\n80jLR+gV1svSWgoNdxHxAiYDvYAWwP0i0iJXs01AlDGmNfAN8FZpF1pUL3Z8kdahrXlp1UvsO7vP\nqjLKpSnL9hFzIol/9osgOMDH6nIs5+ttY8I9rTmVlMJ4vbGHKsT209t5c82bdKrdieh20VaXU6Q9\n947APmPMAWPMJWA2cEf2BsaYpcaYi86nawDLbqjp6+XLe93fo7JPZaKXRrtlgB5PsOv4OT5cso87\n21zN31rUsrqcMqNNvao81iWMWWsPs+bAGavLUWXUmeQzDF06lJCAEN7u9jbeNuuPVRUl3OsAR7I9\nj3NOy8/fgR/ymiEiQ0RkvYisj4+PL3qVxVSzUk0mdp/IsfPHGLV8FHajo/0VJD3DzgvfbKVqJR9e\nvb2l1eWUOcNvaUr96pUYNXcrKWl6sF7llGZP4/nfnichNYFJPSZRzb9snDpcqgdURWQgEAW8ndd8\nY8w0Y0yUMSYqNDS0NFd9mbY12zKq4yhWHF3BlM1TXLqu8m7aigNsO5rI2DtaUa2yr9XllDmVfL0Z\nf1cEsWcu8t4vei2Fymni+omsP7meV699leY1ys6xqqKE+1Eg+50Z6jqn5SAifwPGAH2NMWXi5OD+\n4f3p17gfU7dO5dfDv1pdTpm071QSk37ZS++Iq+gdUf7uh+ou1zUO4b4O9Zi+/ABbjuhgdcphwf4F\nfL7rcwY2H8jt19xudTk5FCXc/wCaiEiYiPgC9wHzszcQkbbAVBzBfqr0yywZEWFM5zG0qtGKMSvH\ncCDhgNUllSmZ3TGVfL14vW8rq8sp817s3Zyagf4M/2qzds8odp7Zyeu/v06HqzowPGq41eVcptBw\nN8akA88A/wN2AV8ZY3aIyFgR6ets9jZQBfhaRDaLyPx8Fud2fl5+vNfjPfy8/IheGk3SJR2vO9NH\nv+1n4+EEXu/bktBAP6vLKfNdS/mQAAATqElEQVSCA3x4595I9sdf0LNnKrizKWcZunQo1fyr8Xa3\nt/Gxlb2zy4rU526MWWyMaWqMucYYM8457RVjzHzn478ZY2oZY9o4v/oWvET3uqryVbx7w7vEJcUx\neuVoPcAKbI1LYNIve+nTujZ9I6+2upxy4/omITxyXUM+XR3Lyr2nrS5HWSDdns6I30ZwJvkMk7pP\nokZADatLypPHXKFamKironi+w/MsO7KMqVumWl2OpZIvZTB0zmZCqvgx7s4IRDzjfqjuMrJnM64J\nrcyIb7aQmKwXN1U07214j7Un1vLytS/TMqTsnl1WYcId4IFmD9D3mr5M2TKFpYeXWl2OZcb/sIsD\n8Rd4t38kwZXK3sfJsi7A14v3BrQhPimVV7/fbnU5yo0WHVjEzJ0zub/Z/dzZ+E6ryylQhQp3EeHl\nzi/TokYLRq8czcHEg1aX5Ha/7Ynns98P8ViXMLpU4LFjrlTrulX5x41N+G7zMb7ffNnJY5AYB/Of\nhX9HwdRusPRfkKxn2ZRnMX/G8Nrq12hXsx0jOoywupxCVahwB/D39mdS90n42HyIXhrN+UvnrS7J\nbU6eS2H4nM00rVWFF3paN6CRp3i6xzVENajG6G+3cfD0hb9mHN/qCPStcyA0HPyC4LcJ8GEUHFxu\nXcGqxBJSEhi6dChBfkG82/3dMnkANbcKF+4AtavU5p0b3uHwucOMWTmmQhxgTc+w8+yXm7h4KYPJ\nD7TD38fz76zkat5eNj64vy0+3jaemrXRcXpk8lmY/SB4+8P/rYT7ZsEjC+GJ3yCgGsy8E3bMs7p0\nVQzp9nRGLB/BqYunmNR9EiEB5eMTb4UMd4COtTvyXNRzLDmyhI+3fWx1OS73/q97WXvwT968sxVN\nagVaXY7HuLpqABP7R7Lr+DneWLgTloyDc0eh/0wIyXa/zNqR8PivUK8jfPN32LXQuqJVsXyw6QPW\nHF/Dy51fJiI0wupyiqzChjvAwOYDua3RbXy46UOWx3nux+Xle+L5cOk+7m1fl7vbWzamm8e6sVkt\nnujWiBXr/sC+fga0fxjqRl3e0D8IHvwarm4L3w6GY5vdX6wqlh9jf2TG9hkMCB9Avyb9rC6nWCp0\nuIsIr177KuHVwxm1fBSHzx22uqRSF3f2IkPnbKZJzSqMvUOvQnWV528NZ0TV38iwGw60eCr/hn6B\ncN8XEFAdvrwPkvRGIGXVnrN7eGXVK7QJbcPIDiOtLqfYKnS4AwR4BzCpxyRsNhvRS6O5mHax8BeV\nExcvpTN45gbS0u1MebA9Ab7az+4qPvZUbrMvZamtM4/OjSPh4qX8GwfWggdmO/rn5z0Bds8/5lPe\nJKYmEr0kmio+VZjYfSI+XmX/AGpuFT7cAepUqcPb3d7mQOIBXlr1EsYYq0u6YsYYnv96CzEnzvHB\nA21pXLOK1SV5tphF2FITaXDr0xxPSOGZLzaRnlFAaF8VAb0mwIGlsOo999WpCpVhz2Dk8pGcuHiC\nid0nElrJtSPYuoqGu9O1V1/LsHbD+PnQz3yy/ROry7li/16yj8XbTvBir2b0CK9pdTmeb9cCqFyT\n8I69eLNfK1buO824xbsKfk27h6HV3Y6DsIfXuKdOVagPN3/IqmOrGN1pNG1qtrG6nBLTcM/m4ZYP\n07NhTz7Y+AErj660upwSm7cpjok/76Ff2zoM7trI6nI8X1oK7PsFmvUGm43+UfV4rEsYM1bFMmNV\nARfKiUCfSVC1Psx9XC9yKgN+iv2Jj7d9zN1N7ubepvdaXc4V0XDPRkR4/brXaVytMS8sf4Ej544U\n/qIy5rc98Yz4eivXNqrB+Lt13Bi3iF0Bl85Dsz5Zk8bc1pyeLa9i7MKdLNx6LP/X+gfB3Z9A0nFY\nNBw8oEuwvNp3dh8vrXqJ1iGtGd1ptNXlXDEN91wq+VTi/e7vIwjRy8rXAdatcQk8+fkGmtQKZOqg\n9vh56wFUtzj4G3j5QsPrsyZ52YRJ97WhQ4PqDJ+zhdX7ChhBsm576P4ibJ8LW2a7oWCV27lL54he\nGk0l70pM7D4RX6/yf0cyDfc81Auqx1vd3mJ/wn5eXf1quTjAuvdkEo/O+IPqlX357NEOBPmXv6P7\n5VbsSqjbAXwCckz29/Fi+qAoGoZUYsh/N7Dp8Nn8l3H9MGhwPSx+Hs7sd3HBKju7sTNq+SiOnT/G\nxO4TqVXZM24Qr+Gejy51uvBs22f5MfZHPtvxmdXlFGjvySTun74GL5vw3793omaQv9UlVRwpiXB8\nS4699uyCK/nw2WMdqVHFl0GfrGNjfgFv84K7pjq+fzsYMnQoYXeZsnkKK46uYFTHUbSr1c7qckqN\nhnsBHmv1GDc3uJn3Nr7H78d+t7qcPGUGu02EL4d0JiykstUlVSyH14Cx5xvuALWDA5g9pDPVq/jy\ncEEBH1wXbv8Ajm6AZeNdVLDK7tfDvzJ161T6Ne5H//D+VpdTqjTcCyAivNnlTRoFN2LE8hHEJcVZ\nXVIOmw6fZcC0v4L9mlA9l93tjqwF8YI6eQw3kE1mwGfuwa/en08ffMs7oe1AWPGuo7tHucyBhAOM\nWem8x3LnMR538oGGeyEq+VTi/R7vYzd2hi0bRnJ6stUlAbAk5iQPTF9LFT9vvnriWg12qxzbDDVb\ngG+lQps6Av5arq7qz8P/bx3zt+RzFk3PCVC9EXw7xHEVqyp1SZeSiF4aneMey55GrDpYGBUVZdav\nX2/Juktiedxynvn1GXqF9WJ81/GWvsvPWnuIV77fQfPagcx4pGO5v7l1uj2d1IxUktOTSUlPcXxl\npPz1PCMl5/d05zzn8+yvTc1IxeCm32kDHFkDlapDSNMcswTBz8sPf29/ArwD8Pfyx9/b8SXGl+82\nnuLQ6TT6RDSgd8v6BPg42gR4Bzhed2Y//rMHEtDkFvzv+QwvL2/3bFMFYDd2opdEs/LoSqbfMp2o\nqwr+1FXWiMgGY0yhRetvTBF1q9uNZ9o+w783/ZuWNVoyqOUgt9eQmp7Ba/N38OW6I9zQNJTJD7aj\nip97f4Tp9nSSLiWRmJpI4qVEx/fURM5dOkdyenK+AZ2akeoI5Yxs853P0+3pxa7DJraswMwenn5e\nfu57401PdRz49K3i6HfPxhjD+bTzl735pGSkOO4f4AP+teGX0/DLb/ksv95VkLIVPm+Lj83Hsa1e\nAfh5+2U9znzDyP5vkf1NJcA7gGC/YIJ9gx3fnY8DfQPxslXMU2WnbpnKsrhljOo4qtwFe3FouBfD\n4xGPs/PMTiZumEh49XA61e7ktnUfTUjmmS82sulwAk91v4bnbgnHy1byELuUcSkrmDND+tylcznC\n+rL5qedISksqcLkFhW6gbyCh3qFZYRTgHZA1L/febeZzP2+/HG0zQ83b5m19H+nO+bB+MfSZ7ThX\nvQiMMaTZ07LeCGes3svHq3ZTv7oP0Tc3oFoVyXoDTE1LIWXdFJITj5LSbiApvgFZn1qy3izTkzmd\nfDrHG2rm9MI+wQT6Bl4W+kF+QTkfZ5/vnF4eB9HKtPTwUqZsmULfa/ryQLMHrC7HpbRbppgupF3g\ngUUPcDblLLP7zObqKlcX6/XGmKw/uszHBoPjf+d/Jtt3Y1iw9RjjFu7Ebgxj+7Xi5hY1s869N8aQ\nkpFyWRBnPj6Xei7fvez8eIkXwX7BBPkG5fijzpyWPQAy5wf5BlHZpzI+Nh/rQ9ddfh0Lq96HF4+C\nT8lPP1259zT/+HIjKWl2Rt/WnIGd6v/1b3juGPznOqjaAP7+M3gX7eKazDeR82nnHb8DuX7+uafl\n/l0p6O5keX0ayOt3Jft0fy//rG0SEbL+cz7OnA7kmC4Ijv/zb19UBxMP8sCiB6gXWI+ZvWbi710+\nTxkuareMhnsJZP6SpGSk4C3eOQIZyDes3c3H5kNVv6oF//Hl3jvzDaayT+WKE9BXYta9jvB9ctUV\nL+pEYgojvtnCir2n6dY0lH/dFUGdqs6LonYtgDkDoctQuPn1K15XYezGzoW0C5d9aits5yEhNaFE\nXWxXKr83g9zz0u3pBPoGlminrCzRcHexbfHb+OnQTwAF7l1Atj2VIvwCigh2u2HTkURW7jlNut1w\nfZMQrm0UgpfNlmd7fy//PPegsu8tKReY1NpxZeo9pTOKqDGGz9ccYtziXQjCMzc25vGuYY5hJBZE\nw4bPYND30OiGUllfaTPGkJyefFmXXkJqApcyLuX8RJrr0yuQ77yi7Dhlz7Ec07O1F4TbGt1GePXy\nfXN4DfdyyBjDTztPMvGnPew+mUTXJiGMvaOVXphUFl26AP+sAz3GwA0jSnXRcWcv8sbCnfxvx0nC\nQioz7Oam9AkPwvZxD0hNgidXO87QURVSUcNdz3MvA+x2w9KYU9wxeRVP/HcDaRl2/vNgO2Y+1lGD\nvaw6vRcwENq00KbFVbdaJaY+FMWnj3bA18vGs19uoud/NrAycjzm4hmY+3fIcH/3hypf9GwZC11I\nTWfuxjg+XR3LgfgL1KkawFv3tOautnXw9tL33TLt9B7H99BmLltF9/CadGsSyqJtx5n0yx4GLkrh\niaDBvLh/MqkLX8DvjokuW7cq/zTc3Sw9w86Kfaf5ftNRftp5kouXMoisG8ykAW3oHVEbX28N9XIh\nPgZs3o4rSV3IZhNuj7ya3hG1+d+OE3y6ujpTj8TyxKZP+CquMsHdn6Z7eKgO76wuo+HuBmcvXGL5\n3nh+2x3Psj3x/HnhEkH+3tzR5mrujapHu/rVrC5RFVf8bqh+DbjpnG8vm9A7oja9I2qzM64Zu75K\n5O74D3l8lh/P+3agR3hNujUNpVuTEB0VVAEa7qUuw244ePo8mw4nsOlIApsOJxBz4hzGQLVKPnRt\nEkqf1rW5Qfe2yrf4GMeYMhZoUbcaPD0HM6M3n5x6n2l1J/DxflvWWDVNa1WhXf1qtKlXlTb1q9I4\ntIp281VARTpbRkR6Au8DXsDHxpjxueb7ATOB9sAZYIAxJragZZb4bJm1U2HZv4r/ulJkALtxHAg1\nxpDhfJxhDBn2v/49RRx7XN42Gz5egpct+wmSqlxLPgvdRsCNL1lXw8U/4dM+EB+D8Q8iw25IzzCk\n2+2k202OO/Z52QSbCF42sIn89WVDfyetcMubjtE/S6DUxpYRES9gMnAzEAf8ISLzjTE7szX7O3DW\nGNNYRO4DJgADSlR5YUKaQoTjxrWOc1kdt500mc+zP87dxvnLbneGcLrd8d3xZc8xLS3DTmq64+tS\ntu8paRkkp2Xk+jeCKn7eBAf4EBzgQ9VKvoQG+hLsX4Gu1qxobN7Q9iFra6hU3XHe++8fImkX8eav\nP2hjDOdS0jl9PpWzFy+RmJxG4sU0klLTL7tNq7+PF/4+Nvy8vfDztuHrbcPP+eVts+HtJXjbBC+b\nzbmzInhn7qw4r9ewieOxTRzXdfz13LFDk/3PIPtfRIX9+6h+jctXUZRumY7APmPMAQARmQ3cAWQP\n9zuA15yPvwE+FBExLjiJfvrRBkxY2YMMYy77JS1tAT5ezrD2ITjYEdw1qvhSOziAq4L9qe38qlO1\nEgG+2sWiLFAlNM+rVgUIdn5ll5KWwbGEZI4npnA8MYUTicnsSUzhzPlUEpPTSLiYRuIFx/fcOzGu\nlJnxjjeCvz7hZr55kGN+tgsAy+l7w6uhdRjQwLXrKEq41wGOZHseB+QeMSurjTEmXUQSgRpAjjsS\niMgQYIjz6XkR2V2Soi0WQq7tqgAq2jZXtO0F3Wa3uu8NuK/kLy/S24JbD6gaY6YB09y5ztImIuuL\n0t/lSSraNle07QXdZk9UlEPoR4F62Z7XdU7Ls42IeOP4NHimNApUSilVfEUJ9z+AJiISJiK+OD5N\nzM/VZj7wsPPxPcASV/S3K6WUKppCu2WcfejPAP/DcSrk/zPG7BCRscB6Y8x84BPgvyKyD/iTK+pO\nKvPKdbdSCVW0ba5o2wu6zR7HslEhlVJKuY5etqaUUh5Iw10ppTyQhvsVEJHnRMSISIjVtbiSiLwt\nIjEislVE5olIVatrchUR6Skiu0Vkn4iMsroeVxOReiKyVER2isgOEYm2uiZ3EREvEdkkIgutrsUV\nNNxLSETqAbcAh62uxQ1+BloZY1oDe4AXLa7HJbINtdELaAHcLyLWjA7mPunAc8aYFkBn4OkKsM2Z\nooFdVhfhKhruJfce8AJYcOdrNzPG/GSMybz1zxoc1zp4oqyhNowxl4DMoTY8ljHmuDFmo/NxEo6w\nq2NtVa4nInWB24CPra7FVTTcS0BE7gCOGmO2WF2LBR4DfrC6CBfJa6gNjw+6TCLSEGgLrLW2EreY\nhGPnzG51Ia6i47nnQ0R+Aa7KY9YYYDSOLhmPUdD2GmO+d7YZg+Nj/Cx31qZcT0SqAHOBocaYc1bX\n40oi0gc4ZYzZICLdra7HVTTc82GM+Vte00UkAggDtjiHK60LbBSRjsaYE24ssVTlt72ZROQRoA9w\nkwdffVyUoTY8joj44Aj2WcaYb62uxw26AH1FpDfgDwSJyOfGmJINsF5G6UVMV0hEYoEoY4zHjqjn\nvFnLROAGY0y81fW4inNcpD3ATThC/Q/gAWPMDksLcyFx7KF8BvxpjBlqdT3u5txzf94Y08fqWkqb\n9rmrovgQCAR+FpHNIvKR1QW5gvOgceZQG7uArzw52J26AA8BNzp/tpude7SqnNM9d6WU8kC6566U\nUh5Iw10ppTyQhrtSSnkgDXellPJAGu5KKeWBNNyVUsoDabgrpZQH+v9gTY76+voKsQAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aDfEmz__iiUT",
        "colab_type": "code",
        "outputId": "171d1acc-7bb0-4a2f-d604-12f120b05809",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 192
        }
      },
      "source": [
        "TRAIN_ITERS=10000\n",
        "\n",
        "\n",
        "# Algorithm 1 of Goodfellow et al 2014\n",
        "k=1\n",
        "histd, histg= np.zeros(TRAIN_ITERS), np.zeros(TRAIN_ITERS)\n",
        "for i in range(TRAIN_ITERS):\n",
        "    for j in range(k):\n",
        "        x= np.random.normal(mu,sigma,M) # sampled m-batch from p_data\n",
        "        x.sort()\n",
        "        z= np.linspace(-5.0,5.0,M)+np.random.random(M)*0.01  # sample m-batch from noise prior\n",
        "        histd[i],_=sess.run([obj_d,opt_d], \n",
        "                            {x_node: np.reshape(x,(M,1)), \n",
        "                             z_node: np.reshape(z,(M,1))})\n",
        "    z= np.linspace(-5.0,5.0,M)+np.random.random(M)*0.01 # sample noise prior\n",
        "    histg[i],_=sess.run([obj_g,opt_g], \n",
        "                        {z_node: np.reshape(z,(M,1))}) # update generator\n",
        "    if i % (TRAIN_ITERS//10) == 0:\n",
        "        print(float(i)/float(TRAIN_ITERS))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.0\n",
            "0.1\n",
            "0.2\n",
            "0.3\n",
            "0.4\n",
            "0.5\n",
            "0.6\n",
            "0.7\n",
            "0.8\n",
            "0.9\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0uq_EcpiiiUX",
        "colab_type": "code",
        "outputId": "d585e787-0ab9-4d0f-8b71-4596196c3731",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        }
      },
      "source": [
        "plt.plot(range(TRAIN_ITERS),histd, label='obj_d')\n",
        "plt.plot(range(TRAIN_ITERS), 1-histg, label='obj_g')\n",
        "plt.legend()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f0e91bd54e0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHzRJREFUeJzt3XuYHHW95/H3t29zz1wyQxIyCRMu\ngrkcQhgCCBw4IiCox8DBRUQF1Ce7uB49B31YkD1reDzqUVl0WWARuanr7Sh414MgyHINJsg1JCEh\nCeQ+mWQyyVz78ts/qjozuc4wXZ2arv68nqefrq6urvpWV/enf/2r6mpzziEiItERC7sAEREJloJd\nRCRiFOwiIhGjYBcRiRgFu4hIxCjYRUQiRsEuIhIxCnYRkYhRsIuIREwijIU2Nze7tra2MBYtIlKy\nli5dus051zLSdKEEe1tbG0uWLAlj0SIiJcvM1o1mOnXFiIhEjIJdRCRiFOwiIhETSh+7iMhopNNp\n1q9fT39/f9ilHFaVlZW0traSTCbH9HgFu4iMW+vXr6euro62tjbMLOxyDgvnHJ2dnaxfv54ZM2aM\naR7qihGRcau/v5+JEyeWTagDmBkTJ04s6FuKgl1ExrVyCvW8Qte5tIJ91SPwxC1hVyEiMq6VVrC/\n8Wd47KvQ3x12JSIi41ZpBfvx74NcGlY/GnYlIlLG1q5dy+zZsw9436c+9SmWLVs2qvncf//9fOYz\nnwmyNKDUjoppbYdkNax7GmYtCLsaEZH93H333WGXUGLBHk9C6ylesItIWbnpN6+ybGOw3bAzj5zA\nlz4wa8TpbrnlFu69917Aa5EvWLCATCbDFVdcwfPPP8+sWbP4/ve/T3V1Neeccw4333wz7e3tB5zX\nfffdx9e+9jUaGho48cQTqaioCHSdoNS6YsBrtW9dBpnBsCsRkTKwdOlS7rvvPhYvXsyzzz7Ld7/7\nXXbs2MGKFSv49Kc/zWuvvcaECRO44447RpzXpk2b+NKXvsRTTz3Fk08+Oeoum7ertFrsAHVTwGWh\nfyfUjnj2ShGJiNG0rIvhySef5OKLL6ampgaASy65hCeeeIJp06ZxxhlnAPDRj36UW2+9lS984QuH\nnNfixYs555xzaGnxsuuyyy5j5cqVgddcei32ygbv+p7zYFE9/O4L0LsdNr/iHQq5qB4e+1q4NYpI\n5O17rPl4Ot6+9IJ9wpHe9Y413vVfvgvfmAF3ngF/uskb9/i/hVObiETOWWedxS9/+Ut6e3vp6enh\nF7/4BWeddRZvvvkmzzzzDAA/+tGPOPPMM0ec16mnnsrjjz9OZ2cn6XSan/3sZ0WpufS6Yhqm7z9u\n9qVw9NnQdiY8fRssuQfS/ZCsPPz1iUikzJs3j6uuuor58+cD3s7TxsZGjj/+eG6//XY+8YlPMHPm\nTK655po9jzlY633KlCksWrSI008/nYaGBubOnVuUmksv2Otbh4Zv3LJ/eE+b7wX7f/w3eN+3IFZ6\nX0pEZHy59tprufbaa/cat3z58gNO29nZSVNT00HndfXVV3P11VcHWt++Si/YzeCq30NPx4Fb5JP8\nHSxL74c1T0B/F8y7Et71j1B98CdbRKRQ5513HnPmzBnzWRmDUnrBDtB2xsHvmzwHPvciPHsnbHwe\ntq+GJ2/xLqk6L+DPvs77gBARCdDDDz+8Z/jUU09lYGBgr/t/8IMfMGfOnKLXUZrBPpLGNrjQ34Ha\nuRpe/Al0b4AXfgh//qp3+a9/gZZ3hFqmiETX4sWLQ1t29DugJx4D774RFtwB/2M7zLrEG58/gkZE\nJGKiH+zDxeLwofu8Fr3LhV2NiEhRlFew51U3Q6a8/kNRRMpHeQZ7okLnmhGRyCrPYI/FIZcJuwoR\nKVFBnY+9WKJ5VMxILO6dSExEJGA6H3tYYnHIKdhFSsofrofNLwc7z8lzhg6NPoQgz8d+zz338PWv\nf32v87Hfdtttga5WeXbFqMUuIqMU5PnYN27cyJe//GWeffZZnnrqqYOelqBQZdxi1+GOIiVlFC3r\nYgjyfOzPPfccZ5999p5zyXzoQx/S+dgDYzG12EWkIDof+3ijPnYRGaUgz8d+yimn8Pjjj7Njxw4y\nmQwPPPBAUWou066YhA53FJFRCfJ87FOnTuWLX/wi8+fPp6mpiRNOOIH6+vrAay7PYNfOUxF5G4I8\nH/tHPvIRFi5cSCaT4eKLL2bBggWB1goBdsWYWdzM/mpmvw1qnkWjnaciUgSjOR/7okWLmDt3LrNn\nz2bGjBlFCfYgW+yfA14DJgQ4z+JQi11EimA052O/+eabi15HIMFuZq3A+4CvANeOMHn4YjHtPBUp\nEc65cXXEyWgVcj5251xByw6qK+bbwHXAQfs3zGyhmS0xsyUdHR0BLXaM1GIXKQmVlZV0dnYWHHSl\nxDlHZ2cnlZUH+OvPUSq4xW5m7we2OueWmtk5B5vOOXcXcBdAe3t7uFtJhzuKlITW1lbWr19P6I3B\nw6yyspLW1tYxPz6IrpgzgL83s4uASmCCmf1f59xHA5h3cajFLlISkslk6H8MXYoK7opxzt3gnGt1\nzrUBHwYeHdehDjoqRkQirTx/eapTCohIhAX6AyXn3J+BPwc5z6LQL09FJMLKs8UeT0E2DWW0p11E\nykf5BjtOR8aISCSVabAnveus/tBaRKKnTIM95V0r2EUkgso02PMt9nS4dYiIFEGZB7ta7CISPWUa\n7OqKEZHoKs9gT1Z514M94dYhIlIE5RnsdUd617s2h1uHiEgRlGew10/1rrvWhlqGiEgxlGewT5gK\nlfWw+eWwKxERCVx5BrsZTDkRNr0YdiUiIoErz2AHmDIXtrwKGR0ZIyLRUr7BfuRJ3uGOW9QdIyLR\nUr7BftQZ3vW6Z8KtQ0QkYOUb7HWToH4abFgadiUiIoEq32AHrztm4/NhVyEiEqjyDvap82DHWujd\nHnYlIiKBKe9gnzzHu966LNw6REQCVN7B3nKCd71tZbh1iIgEqLyDve5ISNbAttfDrkREJDDlHeyx\nGDQfqxa7iERKeQc7QPM7FOwiEikK9onHQddbMNgbdiUiIoFQsDceBTjYtSnsSkREAqFgr2nxrns6\nwq1DRCQgCnYFu4hEjII9H+y7t4Zbh4hIQBTsNc3edc+2cOsQEQmIgj2ehKpGdcWISGQUHOxmNs3M\nHjOzZWb2qpl9LojCDquaFuhRV4yIREMigHlkgM875543szpgqZk97JwrnTNrVTZA/86wqxARCUTB\nLXbn3Cbn3PP+8C7gNWBqofM9rFLVkO4LuwoRkUAE2sduZm3AScDiIOdbdMka/fJURCIjsGA3s1rg\nAeCfnHPdB7h/oZktMbMlHR3jbEdlsgrSPWFXISISiECC3cySeKH+Q+fcgweaxjl3l3Ou3TnX3tLS\nEsRig6OuGBGJkIJ3npqZAfcArznnbim8pBAkq4e6Yvq7vb/LA6ish7rJkKgIrTQRkbcriKNizgA+\nBrxsZi/4477onPt9APM+PJLVMLATHlwIrzwAuczQfRbz/kJvQivUTITqZu9HTakaiKe8+y0OZt5w\nLO6Pi3nzcQ5iCe94+Tzn8gP7FGL+4w3iFd78EylI1XrLS9V4+wPiQWw2EYmqghPCOfckYAHUEp6q\nRu/6pZ/CqdfAUacD5v1oqXsjbFgKXeu8695tewd/GJI10DDNO/6+qhGqGrwPgkQFVEzwPhwSFf64\nFCSqvP0IVQ3QOAMmTPX+ZERERuacdzHzLgeb5mD3AeRy3v292733bJHff2r6Acy5FN58Fs76PLSe\nfOhpc1kY2AWDu72Az2X9DZ8Dl/Wvc974WMLbmLkMZDP7fPz5N4a/GFzOm1cuC9kByA5CNg2DPd7y\nBntgYDf07YCdb0FvJ3Qs947Bzw5Cuh8yo9hXEK+AxjaYfhocMROqJ/rfBiohloTMgPeBkKz2PiDS\nfZDuhd0dsHuLt9xYwvsAyWW9fRSJKu+2y3l1Zgb83wY477HpPm9dsmmv1lzae04y/d7zaTFvnfu6\nvOFUjTevjP88xBJQUQuJSm+Z6d69vwXFEl7XWcN07zr/bSo7OLSdcplhlwPdznrbMN3rP+c9Xn3Z\njFdvfrvFU0PLzmW8D814hbeulfXeJf8BWzHBu53LeI9JVPrPXcJbTjbtPxeDQ8vZM274cOYg02W8\n5yjd561vosJbRqrarzH/OvPvy9ceT3k1OOetM/43Tu+FOOy1mF9Odui+/Osym/a+Pea3Sf65trj3\net0Thv6ynfPn7c9/+HL23Jcb+kYbS3i15bdlPDn0Onc5//XmPw6GfRP267T40PB+hr3v8u/BXMZf\nh6w372za27Z9XUPb32LeN+hcduibeSwOA93ets7vq3M5732V8Wvt3wnVTd71x38FbWeO8CYtjIId\nYMKRcPmPRjdtLO61fKsailvTWGUz3pshH4iZ/qHA7+2E7Wtg+xvev0a98iAMfu/tL8PifhiMIFXn\nvfCTVd6HRrzCD5Wk9wEST0JFHdRP9d6U8aT3YzGcF/axpBcIsYT3pssHrZk37+zg0Jsym4a+7bDp\nRe/DL5f2Wkl7lud/EOVD9WC3LQFVTUNdX4nKocfnZQaG3uixpHdEVTYNmFdDfzd0rx9qBPR1eQGY\n/yDLPzaeGgrYfODmu+3yz8/w4eHBPHy6/Iewc978M/1esGbz3yz9wMwH8sCuoQ+IfPDmA3d4g8PM\nX3bKe27yktV+HQlvfpkB7/78dLmct13Nhj44MgPDlmX7DMf2vw/n1ReLe4/F+R8kKe+1ZDG/Fcyw\nx/jD+OuR/8DKr0/evh8A+av8N12Le6+3RIX3XFU1eN+Snf/hP7jbf01mh+pKVnkf1Mnqoeezp9Or\nFbzXde82L+zrW0d+7xRIwR418QSQOPgO36PPGRrO5bwg6t3uB8Ggd4lXeOGT7vMu8ZTXWq45AmqP\n8FqgZt7jYShM8q21VPX+YSBD8t/KtK9EikSvrHIWi3k7gvNnuBzL48EL8lR1cHVFnZlCXYpKe9BE\nRCJGwS4iEjEKdhGRiFGwi4hEjIJdRCRiFOwiIhGjYBcRiRgFu4hIxCjYRUQiRsEuIhIxCnYRkYhR\nsIuIRIyCXUQkYhTsIiIRo2AXEYkYBbuISMQo2EVEIkbBLiISMQp2EZGIUbCLiESMgl1EJGIU7CIi\nEaNgFxGJGAW7iEjEKNhFRCJGwS4iEjEKdhGRiAkk2M3svWa2wsxWmdn1QcxTRETGpuBgN7M4cDtw\nITATuNzMZhY6XxERGZsgWuzzgVXOuTecc4PAT4APBjBfEREZgyCCfSrw1rDb6/1xIiISgsO289TM\nFprZEjNb0tHRcbgWKyJSdoII9g3AtGG3W/1xe3HO3eWca3fOtbe0tASwWBEROZAggv0vwHFmNsPM\nUsCHgV8HMF8RERmDRKEzcM5lzOwzwENAHLjXOfdqwZWJiMiYFBzsAM653wO/D2JeIiJSGP3yVEQk\nYhTsIiIRo2AXEYkYBbuISMQo2EVEIkbBLiISMQp2EZGIUbCLiESMgl1EJGIU7CIiEaNgFxGJGAW7\niEjEKNhFRCJGwS4iEjEKdhGRiFGwi4hEjIJdRCRiFOwiIhGjYBcRiRgF+zi2YvMunHNhlyEiJSaQ\nP7MuVS++1cXjKzv47LnHBTbPVVt3ceufVnH2O1r4/M9e5IJZk3jo1S3MOnICr27sDmw5d1wxj/Nn\nTmLllt1Mn1iNc47aigRmBsBAJsvW7gEaqpPUVSYDW66IjH8WRouwvb3dLVmy5LAvd19t1/8OgJ8s\nPI05U+upqTj459zmnf088toWFv36VTK5aLSiH/382STjMbI5x9TGKpLx4nyBc86RyTmS8diebyDb\newapr0rSsXuAqmSc2ooEXX1punrTbN7ZT2UyRnNtBQDd/Wl2D2QAaKhKsXxzN8e01FKRjLGpq5+c\nc3T2DHJMSy0zmmuIx4yaVJyO3QPUViSIx4x0xpHJ5aipSLBt9wCduwcZyORY29lDVTLOqq27Ofed\nR9BcW0FXb5rKZIxpTdWkszlS8RiZnCMRMzZ09dGxa4Cu3jRVqTh/fXMHsZgxt7WBXQMZVm3dzdSG\nKl54q4uWugrOOq6ZmBl96SwTa1Ik4zHqKhP0p3Os2dbD6g5v+ubaCp59o5ONO/vI5Rz11SmWbexm\nybrtxM2oTMYBOPXoJjp2DfDE69v49DnHsHXXgD99cs9yt/cMMrWhilc3drOzL81/ap9GbzrDqi27\n2bSznyvfdRRbuwf4/Sub+cCJU1i3rZdYDE5sbWDFll388dUtfPbcY9nZl+bR5Vs5+x1HkMnm+J8P\nrwSg/ahGqlJxnnh9GwDfvPRv6M/k+JdfvgLAWcc1s3TdDqY1VvP61l18/PQ2Nu/spz+T5YTJE1i/\no5eKRJyHl23muEl1xAxaG6vp2DXACZPrmFxfyWA2xzf+YwU1qTifPHMGXX1pJtdXkoj5DZf0UD1z\nptZzxrHN3Pn4aua3NbFy6y66etP7vQ4/83fHcttjq/Ybf8GsSRzTUkt/Ose9T63Z7/7m2gq27R4Y\n+xtgGDN49aYLqE6NrU1tZkudc+0jTleuwd6xa4BTvvLIntsTKhO8tOiCvab55kPLuf2x1Ye7tHFl\n/owmnluznX9dMJuaijj//NMXwy5JJHDxmJE9TA2273zsZC6YNXlMjx1tsJdtV8zwUAfo7s/Qn86y\nfPMuFtz+1JjmWVeZ4MLZk2lva2JXf4Yj6io4+/gWtu8epLEmxYTKoa6SQj3/5g4uueNpAE4+qpGl\n63YA0FybYtvuwUCWAfDcmu0A/He/NSbFUZ2K0zuYLWgec6c18MJbXQBcPn86T6/eRibr+Id5U9ne\nO8iabT107h7kwtlTMINf/HUD58+cBMCGrj7mz2hiR0+axWs6uXDOFFpqU+zoTTN5QiUDmRwvre9i\nQlWSbM7RUlvBuu09AFwyr5VUPMbPl65nRnMNNRUJMtkck+sr6dw9yMTaFD0DWRprvMemsznqKpNU\nJeMMZLJUJuPEzBjI5KhIxEglYlSn4nT6r+OqVJycc1SnEhiQj99NXX001aSoqUgwmM0BEDcjZkY8\nZgxksqQSMXoHvGVUpeIMZnL0DWYZyGZJxmLUVia8x8QM5xxdvWmqK7x6dvdnqErFifvfEnoHs1Qk\nYiTjMbr70tRVJkhnvWoyuRzJeIyBTI6aVJzu/gxVyTiZXA4zI5PNUZ1KsKW7n2lN1QVt59Eo2xZ7\nvhvm7bpg1iS+cP7xJOIxZjTXBFzV4dM7mGHZxm6+98w6fvPixqIs4+SjGpnaUMWpRzdx3BF1HD+5\nDhwMZnP0p7N7vcCzOUdfOkt1Mk4sFsyHn0jUqMV+CLv69+9/O5SbP3QiF8yatNfOyVJXnUrQ3tZE\ne1sT//vykw44zdOrt1GVjPPIa1u4YNZkJtdXkorH6OwZ5KimahIB9snHY0btIfZxiMjoleU76Q8v\nbx7VdO955xHcfeUpRa5m/HrXMc0AnDS9ca/xDdWpMMoRkVEqy2Bf29mzZ/is45r37N3PO7qlhhsv\neifnvnPS4S5NRKRgZRns8/wW6CfPnMGKzbv2uu/a894R6HHtIiKHW1kGe9rfg37pya0sXbeDJ1dt\n4ycLT+PE1gaqUvGQqxMRKUxZBnv+cNV4zLji1On8w7xWBbqIREZBhzWY2TfNbLmZvWRmvzCzhqAK\nK6asf4hnzMDMFOoiEimFHq/2MDDbOfc3wErghsJLKr5cLh/s0Th0UURkuIKC3Tn3R+dcxr/5LNBa\neEnFl/Nb7HH9EEZEIijIsz59AvhDgPMrmqxa7CISYSPuPDWzR4ADnbHmRufcr/xpbgQywA8PMZ+F\nwEKA6dOnj6nYoOTPoqCfrotIFI0Y7M659xzqfjO7Cng/cK47xIlnnHN3AXeBd66Yt1dmsIbvPBUR\niZqCDnc0s/cC1wFnO+d6gymp+PJdMXF1xYhIBBXax34bUAc8bGYvmNmdAdRUdPkvFuqKEZEoKqjF\n7pw7NqhCDiftPBWRKCvLP7PO5n95qmAXkQgqy2Af6ooJuRARkSIoy2hTV4yIRFlZBvvwk4CJiERN\nmQa7l+xqsItIFJVlsOs4dhGJsrIMdp0ETESirDyDPZfvilGwi0j0lGWwZ51Ta11EIqssgz3n1L8u\nItFVnsGeczoiRkQiqzyDXV0xIhJhZRns2Zx+dSoi0VWWwZ5zTn+yISKRVbbBrq4YEYmqsgz2bM6p\nK0ZEIqssgz3n9O9JIhJd5RnsOfWxi0h0FfTXeKXkhgdf5udL32Le9EYWr9kedjkiIkVTNsH+4+fe\nBFCoi0jklUVXTP40vSIi5SBywd52/e/40q9e2XP7/qfW8OXfLguxIhGRwysywb6zL71n+HvPrANg\na3c/i36zjPufXhtSVSIih1/J97Hv7Etz4k1/BOA//+3Re8a3Xf+7sEoSEQlVybfYX16/c8/wd/7f\nGyFWIiIyPpR8sOsHpCIieyv5YH95w86RJxIRKSMl3ce+4PaneOGtrrDLEBEZV0q2xd4zkFGoi4gc\nQMkG+wdue7Kgxz//L+cFVImIyPhSssH+RkdPQY9vqkkFVImIyPgSSLCb2efNzJlZcxDzK7Ynrvu7\nsEsQESmagoPdzKYB5wNvFl5O8X314jlMa6oOuwwRkaIJosX+LeA64LCdaeuJ1ztGNd1xR9Ry98fb\nWf3Vi7isfRqg495FJPoKOtzRzD4IbHDOvWgjJKaZLQQWAkyfPr2QxfKxe57bb9xfbnwPp3zlEQC+\nfdlc5rTWM2NizZ5/SvrHc49lzbYeLpozpaBli4iMdyMGu5k9Akw+wF03Al/E64YZkXPuLuAugPb2\n9jG37p3b/6GfffextNRV7Lm94KSp+03T2ljNv/+X08e6WBGRkjFisDvn3nOg8WY2B5gB5FvrrcDz\nZjbfObc50CqH+dbDK/e6/cA17+LkoxqLtTgRkZIz5q4Y59zLwBH522a2Fmh3zm0LoK6DuvXRVXvd\nHh7qv/vsmTy/bkcxFy8iMu6V9CkFvnXZiXvdnnVkPbOOrA+pGhGR8SGwYHfOtQU1r4PZ2NW31+2L\nT2ot9iJFREpOSf3y9Kr79j8aRkRE9lZSwb5yy+6wSxARGfdKKtiHW/tv7wu7BBGRcalkg11ERA6s\npIL92RvOBeC5G88NuRIRkfGrpA53nFxfqS4YEZERlFSLXURERqZgFxGJGAW7iEjEKNhFRCJGwS4i\nEjEKdhGRiFGwi4hEjIJdRCRi7EB/NVf0hZp1AOvG+PBmoKh/5jEOaZ3Lg9a5PBSyzkc551pGmiiU\nYC+EmS1xzrWHXcfhpHUuD1rn8nA41lldMSIiEaNgFxGJmFIM9rvCLiAEWufyoHUuD0Vf55LrYxcR\nkUMrxRa7iIgcQkkFu5m918xWmNkqM7s+7HrGysymmdljZrbMzF41s8/545vM7GEze92/bvTHm5nd\n6q/3S2Y2b9i8rvSnf93MrgxrnUbLzOJm9lcz+61/e4aZLfbX7admlvLHV/i3V/n3tw2bxw3++BVm\ndkE4azI6ZtZgZj83s+Vm9pqZnR717Wxm/+y/rl8xsx+bWWXUtrOZ3WtmW83slWHjAtuuZnaymb3s\nP+ZWM7O3VaBzriQuQBxYDRwNpIAXgZlh1zXGdZkCzPOH64CVwEzgG8D1/vjrga/7wxcBfwAMOA1Y\n7I9vAt7wrxv94caw12+Edb8W+BHwW//2vwMf9ofvBK7xhz8N3OkPfxj4qT8809/2FcAM/zURD3u9\nDrG+3wM+5Q+ngIYob2dgKrAGqBq2fa+K2nYG/haYB7wybFxg2xV4zp/W/Mde+LbqC/sJehtP5OnA\nQ8Nu3wDcEHZdAa3br4DzgBXAFH/cFGCFP/wd4PJh06/w778c+M6w8XtNN94uQCvwJ+DdwG/9F+02\nILHvNgYeAk73hxP+dLbvdh8+3Xi7APV+yNk+4yO7nf1gf8sPq4S/nS+I4nYG2vYJ9kC2q3/f8mHj\n95puNJdS6orJv2Dy1vvjSpr/1fMkYDEwyTm3yb9rMzDJHz7Yupfac/Jt4Dog59+eCHQ55zL+7eH1\n71k3//6d/vSltM4zgA7gPr/76W4zqyHC29k5twG4GXgT2IS33ZYS7e2cF9R2neoP7zt+1Eop2CPH\nzGqBB4B/cs51D7/PeR/VkTlkyczeD2x1zi0Nu5bDKIH3df3/OOdOAnrwvqLvEcHt3Ah8EO9D7Uig\nBnhvqEWFIOztWkrBvgGYNux2qz+uJJlZEi/Uf+ice9AfvcXMpvj3TwG2+uMPtu6l9JycAfy9ma0F\nfoLXHfO/gAYzy/+p+vD696ybf3890ElprfN6YL1zbrF/++d4QR/l7fweYI1zrsM5lwYexNv2Ud7O\neUFt1w3+8L7jR62Ugv0vwHH+3vUU3o6WX4dc05j4e7jvAV5zzt0y7K5fA/k941fi9b3nx3/c37t+\nGrDT/8r3EHC+mTX6LaXz/XHjjnPuBudcq3OuDW/bPeqcuwJ4DLjUn2zfdc4/F5f60zt//If9oylm\nAMfh7Wgad5xzm4G3zOx4f9S5wDIivJ3xumBOM7Nq/3WeX+fIbudhAtmu/n3dZnaa/xx+fNi8Rifs\nHRBvc2fFRXhHkKwGbgy7ngLW40y8r2kvAS/4l4vw+hb/BLwOPAI0+dMbcLu/3i8D7cPm9QlglX+5\nOux1G+X6n8PQUTFH471hVwE/Ayr88ZX+7VX+/UcPe/yN/nOxgrd5tEAI6zoXWOJv61/iHf0Q6e0M\n3AQsB14BfoB3ZEuktjPwY7x9CGm8b2afDHK7Au3+87cauI19dsCPdNEvT0VEIqaUumJERGQUFOwi\nIhGjYBcRiRgFu4hIxCjYRUQiRsEuIhIxCnYRkYhRsIuIRMz/B7oaJiuDGY0pAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FguUK-oRiiUc",
        "colab_type": "code",
        "outputId": "31511c87-d1b3-4e04-d579-ee3599b94039",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        }
      },
      "source": [
        "plot_fig()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8VFX+//HXyaR3IAkdEgRCi6FJ\nEWmL0kQQC8VKV9BFUdeGq66u+9Nd1sUCuoCgKAi6CAKilK+oIL2EnkAILdQUEtLLzPn9cZMYSpIJ\nmckkk8/z8ZjHZO6cufOZQN45Offcc5XWGiGEEM7FxdEFCCGEsD0JdyGEcEIS7kII4YQk3IUQwglJ\nuAshhBOScBdCCCck4S6EEE5Iwl0IIZyQhLsQQjghV0e9cVBQkA4NDXXU2wshRLW0e/fuRK11cFnt\nHBbuoaGh7Nq1y1FvL4QQ1ZJS6pQ17WRYRgghnJCEuxBCOCEJdyGEcEIOG3MXQpQtLy+P+Ph4srOz\nHV2KqGSenp40atQINze3m3q9hLsQVVh8fDx+fn6EhoailHJ0OaKSaK1JSkoiPj6esLCwm9qHDMsI\nUYVlZ2dTp04dCfYaRilFnTp1KvQXm4S7EFWcBHvNVNF/dwl3IYRwQhLuQgjhhCTchRB28csvvzBk\nyJBS20RFRbFmzZpKqqhmkXAXQjiMhLv9yFRIIaqJv606xOFzV2y6zzYN/Hnjnraltjl58iQDBw6k\nU6dO7Nmzh7Zt27Jw4UK8vb2va/vTTz/x7LPP4u3tzR133FG0fceOHTzzzDNkZ2fj5eXFggULCAsL\n4/XXXycrK4vNmzfzyiuvEBYWdl278PBwm37mmkJ67kKIMsXExDBlyhSOHDmCv78/s2fPvq5NdnY2\nEydOZNWqVezevZsLFy4UPdeqVSs2bdrE3r17eeutt3j11Vdxd3fnrbfeYuTIkURFRTFy5MgbthM3\nR3ruQlQTZfWw7alx48b06NEDgEceeYQPP/yQF1544ao20dHRhIWF0aJFi6J2c+bMASA1NZXHH3+c\nY8eOoZQiLy/vhu9jbTtRtjJ77kqp+UqpS0qpgyU8r5RSHyqlYpVS+5VSHW1fphDCka6dc13eOdh/\n/etf6du3LwcPHmTVqlUlnpxjbTtRNmuGZT4HBpby/CCgRcFtEvBJxcsSQlQlp0+fZuvWrQAsXrz4\nqvH0Qq1ateLkyZMcP34cgK+//rroudTUVBo2bAjA559/XrTdz8+PtLS0MtuJ8isz3LXWvwHJpTQZ\nBizUhm1AoFKqvq0KFEI4Xnh4OLNmzaJ169ZcvnyZyZMnX9fG09OTOXPmcPfdd9OxY0dCQkKKnnvx\nxRd55ZVX6NChA/n5+UXb+/bty+HDh2nfvj1Lly4tsZ0oP6W1LruRUqHAaq11uxs8txp4V2u9ueDx\n/wEvaa1LvcxS586dtVyJSYjSHTlyhNatWzu0hpMnTzJkyBAOHrzhyKywoxv9+yuldmutO5f12kqd\nLaOUmqSU2qWU2pWQkFCZby2EEDWKLWbLnAUaF3vcqGDbdbTWc4A5YPTcbfDeQgg7Cw0Nva7XPnz4\ncE6cOHHVtvfee48BAwZUZmmiFLYI95XA00qpJUBXIFVrfd4G+xVCVFHLly93dAmiDGWGu1Lqa6AP\nEKSUigfeANwAtNafAmuAwUAskAmMtVexQgghrFNmuGutR5fxvAaesllFQgghKkyWHxBCCCck4S6E\nsNqbb77JjBkzbuq1t99+e6nPDx48mJSUlJvad3Fjxozhf//7X4X3Ux6+vr6V+n7WkLVlhBCVYsuW\nLaU+X1OW/tVao7XGxcW+fWvpuQshSvXOO+/QsmVL7rjjDmJiYoq2Hz9+vGgp4J49exIdHQ3AxYsX\nGT58OJGRkURGRhaFemHv9vz58/Tq1Yv27dvTrl07Nm3aBBhTLhMTEwF4//33adeuHe3atWPmzJmA\ncTJV69atmThxIm3btqV///5kZWXdsOYNGzbQuXNnWrZsyerVqwFj1cqxY8cSERFBhw4d2LhxI2As\nc/D0008XvXbIkCH88ssvRTVPnz6dyMhIunXrxsWLFwE4ceIE3bt3JyIigtdee63otenp6fTr14+O\nHTsSERHB999/X1R7eHg4jz32GO3atePtt9/m2WefLXrd3LlzmTZtWrn/bUojPXchqosfX4YLB2y7\nz3oRMOjdEp/evXs3S5YsISoqivz8fDp27EinTp0AmDRpEp9++iktWrRg+/btTJkyhZ9//pmpU6fS\nu3dvli9fjtlsJj09/ap9Ll68mAEDBjB9+nTMZjOZmZnXveeCBQvYvn07Wmu6du1K7969qVWrFseO\nHePrr79m7ty5jBgxgmXLlvHII49cV/fJkyfZsWMHx48fp2/fvsTGxjJr1iyUUhw4cIDo6Gj69+/P\n0aNHS/32ZGRk0K1bN9555x1efPFF5s6dy2uvvcYzzzzD5MmTeeyxx5g1a1ZRe09PT5YvX46/vz+J\niYl069aNoUOHAnDs2DG++OILunXrRnp6OpGRkfzrX//Czc2NBQsW8N///rf0f6tyknAXQpRo06ZN\nDB8+vOjCHIVBlZ6ezpYtW3jwwQeL2ubk5ADw888/s3DhQgBMJhMBAQFX7fO2225j3Lhx5OXlce+9\n99K+ffurnt+8eTPDhw/Hx8cHgPvuu49NmzYxdOhQwsLCitp36tSJkydP3rDuESNG4OLiQosWLWjW\nrBnR0dFs3ryZP//5z4CxyFnTpk3LDHd3d/eiSwV26tSJ9evXA/D777+zbNkyAB599FFeeuklwBhy\nefXVV/ntt99wcXHh7NmzRb39pk2b0q1bN8D4i+BPf/oTq1evpnXr1uTl5REREVFqLeUl4S5EdVFK\nD7uyWSwWAgMDiYqKKvdre/XqxW+//cYPP/zAmDFjeO6553jssceseq2Hh0fR1yaTqcRhmfIsUezq\n6orFYil6XHyZYTc3t6LXmkymqxYzu9E+Fy1aREJCArt378bNzY3Q0NCi/RX+sio0YcIE/vGPf9Cq\nVSvGjrX96UEy5i6EKFGvXr1YsWIFWVlZpKWlsWrVKgD8/f0JCwvj22+/BYwe6759+wDo168fn3xi\nrPxtNptJTU29ap+nTp2ibt26TJw4kQkTJrBnz56rnu/ZsycrVqwgMzOTjIwMli9fTs+ePctV97ff\nfovFYuH48ePExcURHh5Oz549WbRoEQBHjx7l9OnThIeHExoaSlRUFBaLhTNnzrBjx44y99+jRw+W\nLFkCULRPMJYsDgkJwc3NjY0bN3Lq1KkS99G1a1fOnDnD4sWLGT261NOJboqEuxCiRB07dmTkyJFE\nRkYyaNAgbrvttqLnFi1axGeffUZkZCRt27YtOnj4wQcfsHHjRiIiIujUqROHDx++ap+//PILkZGR\ndOjQgaVLl/LMM89c955jxoyhS5cudO3alQkTJtChQ4dy1d2kSRO6dOnCoEGD+PTTT/H09GTKlClY\nLBYiIiIYOXIkn3/+OR4eHvTo0YOwsDDatGnD1KlT6dix7OsNffDBB8yaNYuIiAjOnv1jKa2HH36Y\nXbt2ERERwcKFC2nVqlWp+xkxYgQ9evSgVq1a5fp81rBqyV97kCV/hShbVVjyV9jPkCFDmDZtGv36\n9bvh89VmyV8hhBCQkpJCy5Yt8fLyKjHYK0oOqAohRCULDAwsc6ZORUnPXQghnJCEuxBCOCEJdyGE\ncEIS7kII4YQk3IUQwglJuAshhBOScBdCCCck89yFqCbe2/Ee0cnRNt1nq9qteKnLS6W2OXnyZNG6\n7Xv27KFt27YsXLiwaKXI4tasWcNzzz2Hj48PPXr0IC4urmg9dVG5pOcuhChTTEwMU6ZM4ciRI/j7\n+zN79uzr2mRnZ/PEE0/w448/snv3bhISEhxQqSgkPXchqomyetj21LhxY3r06AHAI488wocffsgL\nL7xwVZvo6GiaNWtGWFgYAKNHj2bOnDmVXqswSM9dCFGm8qyPLqoGCXchRJlOnz7N1q1bAeMyeXfc\nccd1bcLDw4mLiyu6OtLSpUsrs0RxDQl3IUSZwsPDmTVrFq1bt+by5ctMnjz5ujZeXl7Mnj276OCr\nn5/fdZfYE5VHxtyFEGVydXXlq6++KrNd3759iY6ORmvNU089RefOZS47LuxEeu5CCJuZO3cu7du3\np23btqSmpvLEE084uqQaS3ruQohShYaGcvDgwau2DR8+nBMnTly17b333mPatGlMmzatMssTJZBw\nF6KK01pXudkpy5cvd3QJTq+il0CVYRkhqjBPT0+SkpIq/IMuqhetNUlJSXh6et70PqTnLkQV1qhR\nI+Lj4+VszxrI09OTRo0a3fTrrQp3pdRA4APABMzTWr97zfNNgC+AwII2L2ut19x0VUIIANzc3IrO\n+BSiPMocllFKmYBZwCCgDTBaKdXmmmavAd9orTsAo4DrF54QQghRaawZc+8CxGqt47TWucASYNg1\nbTTgX/B1AHDOdiUKIYQoL2uGZRoCZ4o9jge6XtPmTWCdUurPgA9wp02qE0IIcVNsNVtmNPC51roR\nMBj4Uil13b6VUpOUUruUUrvkAJEQQtiPNeF+Fmhc7HGjgm3FjQe+AdBabwU8gaBrd6S1nqO17qy1\n7hwcHHxzFQshhCiTNeG+E2ihlApTSrljHDBdeU2b00A/AKVUa4xwl665EEI4SJnhrrXOB54G1gJH\nMGbFHFJKvaWUGlrQ7HlgolJqH/A1MEbLWRdCCOEwVs1zL5izvuaaba8X+/ow0MO2pQkhhLhZsvyA\nEEI4IQl3IYRwQhLuQgjhhCTchRDCCUm4CyGEE5JwF0IIJyThLoQQTkjCXQghnJCEuxBCOCEJdyGE\ncEIS7kII4YQk3IUQwglJuAshhBOScBdCCCck4S6EEE5Iwl0IIZyQhLsQQjghCXchhHBCEu5CCOGE\nJNyFEMIJSbgLIYQTknAXQggnJOEuhBBOSMJdCCGckIS7EEI4IQl3IYRwQhLuQgjhhCTchRDCCUm4\nCyGEE5JwF0IIJyThLoQQTsiqcFdKDVRKxSilYpVSL5fQZoRS6rBS6pBSarFtyxRCCFEermU1UEqZ\ngFnAXUA8sFMptVJrfbhYmxbAK0APrfVlpVSIvQoWQghRNmt67l2AWK11nNY6F1gCDLumzURgltb6\nMoDW+pJtyxRCCFEe1oR7Q+BMscfxBduKawm0VEr9rpTappQaeKMdKaUmKaV2KaV2JSQk3FzFQggh\nymSrA6quQAugDzAamKuUCry2kdZ6jta6s9a6c3BwsI3eWgghxLWsCfezQONijxsVbCsuHliptc7T\nWp8AjmKEvRBCCAewJtx3Ai2UUmFKKXdgFLDymjYrMHrtKKWCMIZp4mxYpxBCiHIoM9y11vnA08Ba\n4Ajwjdb6kFLqLaXU0IJma4EkpdRhYCPwF611kr2KFkIIUTqltXbIG3fu3Fnv2rXLIe8thBDVlVJq\nt9a6c1nt5AxVIYRwQhLuQgjhhMo8Q1UIIWxOazDnQX42mHON+/ycglu28Zwl37hpM1gKbtpcsL34\nY/PV7bQG9PX3he9b1raioWpd0OTabde2K/FDlvxUy0HQqFO5v23lIeEuhCg/iwWyUyAjETISIDPR\n+DozCbIuQ84VyEmHnDTILbjPSTe252cbN6egyni6hOf9G0i4CyEcJCsFEo8at5QzkHoGUk4b96ln\nwZJ349e5+4FH4c3XuPcJBg9/47GbF7h6gsnduHf1MG4mj2Jfu4GLG7iYwMUVlAlcXIp97Wo8p1z+\n+LrwOVUw2lwYrEoBqljQKiu23ei1xbaVFNpViIS7EDWdxQLJcXB2N5zbA5eOQEIMpF8o1kiBX30I\nbAwNO0Obe8GvHngHgU/BzTsIvOuAq7vDPor4g4S7EDWNOR/OR0HcRjj5uxHo2anGc27eENIabvkT\nBIdDcCsIagEBjSW0qxkJdyFqgoxEiP4Bjq2DE5sgpyDMQ9pC2+HQsJNxCwoHk8SCM5B/RSGcVfol\nOLQcjqyCU7+DtkBAE2h7LzTrA2G9jOEU4ZQk3IVwJuY8OLYe9n4FR38ypgcGt4KeL0CboVC3XbU4\nGCgqTsJdVK6cNONg3aXDxiyM9ATISjamz1nyAWXMevAJAt8QCGxiBFLdtuDfsGLBdOU8xKwxDh66\n+0DzO6FxF5t9NIdKuwg758HuzyHjEviEwO1PQ+RDENLK0dUJB5BwF/aRl2WEeEK0EeSXoo1ZGKmn\n/2jj6mkEuFct42ZyN4YOzHnGlLszO4z504X8GxlDCc16Q2hPCLj2mjE3YLFA7HrY/ikc3whocPUC\ncw78+h60vgeGzQZPf5t/CyrFhYOwbTYc+Nb4vrUcCJ0eN35xmdwcXZ1wIAl3UTH5uZAUawR4QkGA\nXzoCl08YQQ3GfOWglkYvudPjxmyMkNYQ2NSYn1ya7CvGvs/vh1ObjaGGfQXXX6/dzAj50J7QsCME\nNDLmSOdmGDUcW2eEXnKcMY2v94vQ7gFj9kduOuyYCxvfgYVD4fHVxhzs6uLsbtj4D4jdYMxw6fg4\ndJsMdW5xdGWiipBVIYV1cjMg8ZjRG0+MKeiVxxjBqc1GG2UywiW4FYS0+SPEazezXS/SYoGLB+Hk\nJmPWx6ktf8z8KKyhsB4UNO0BncdCm2E3riHmR1jyEIQPhpFfVf3x6AsHjVCP+cGYU979Keg0Frxr\nO7oyUUmsXRVSwl38cSp5+kW4cs4YEkk5DSmnjPvLp4xx3EKFIR7U0gjy4HAjxOu0ADfPSq7dDOf3\nFQz5nDHWKXH3KfhLoRv4WnE5xy0fw7rpcM8H0GmM3Uu+KYnH4Jf/Bwe/M870vP3P0O1J4+xPUaNY\nG+4yLOMMtDYORuZnQ1425KYZwxk5V4rdp/7xdWayEdbpl4x1QTISCg5mFuPiagxzBDaBlv0hMNQY\nzghuZfTEq8oJLS4mY0imYceb30e3KcZwz9rXjLHqgEa2q6+iLp+EX/8J+742jhX0fN44UOpVy9GV\niSqu+oX7uSg4s/3qVdrgmhXart1W3seltbH2PTDGnK9atc5y9Up22mz0mouvdFf0XLG25jyjR5qX\n9ceqefk5kF/sceH4dlncvI1g8Ak2Th+vf6sxs8I3pGBbfSPQ/RuUPR7uLFxcYNjHMKsbrJ0OI75w\ndEXGX1C/zYA9C421UrpNgTumybx0YbXqF+4nfoX1rzu6CutdtchRweJHJW67ZpGk4ttcPY0/x109\njK/dPIstulR472Xcu/sasz88/IvdBxh/wssMihurFQo9nzMOsJ74zZiV4wjpl2DzTNj1mfHLvdPj\nRm/dv4Fj6hHVVvUbc8/LMm7FFV/ZraRt5X1ckX1U9YNy4sbysmBWF+OX4xObKvc0/Iwk2PKBMYMn\nPwciR0Hvl6BW08qrQVQLzjvm7uZl3ISwNTcvGPAPWPqI0XPu+oT93zMjyZinvv1TY0bSrSOMUJcp\njaKCql+4C2FPrYYY6678/A60Hgr+9e3zPknHYessiFpsHDtpOxz6vGLMPBLCBiTchShOKbj7ffjk\ndljzAoxaZLt9W8zGMru7FhgrNJrc4NaR0P1pWSJA2JyEuxDXqnML9HkZNrwJh1YYqyhWRHIc7F1k\nTGe8cha8ahsHSbtMAr+6NilZiGtJuAtxI93/DIe/h5VTjYXLgpqX7/W5Gcbr9y4ylk1QLsYc+gH/\ngPBBxqwmIexIwl2IGzG5woiFMKcPfD0SHl9V9nREixlOboYD38Ch742TyWo3g36vQ+Romc4oKpWE\nuxAlCWwCoxbDVw/AZ/3h3tnXz3+3WODcXji8HA4sg7RzxgWi2wyFDo9Ck24yNVY4hIS7EKVp0g3G\nrIJvx8IX90D99sbl6FxcjbV34ndCZpLxuPldMODv0HIQuHs7unJRw0m4C1GWBh1g8u+w50s49B0c\nXFZwybpGRqA3vxOa95OVGUWVIuEuhDXcfYxVGLs96ehKhLCKi6MLEEIIYXtWhbtSaqBSKkYpFauU\nermUdvcrpbRSqsx1D4QQQthPmeGulDIBs4BBQBtgtFKqzQ3a+QHPANttXaQQQojysabn3gWI1VrH\naa1zgSXAsBu0ext4D8i2YX1CCCFugjUHVBsCZ4o9jge6Fm+glOoINNZa/6CU+osN6xM1RHaemS3H\nE9kel8zxhAwS0nNAa+r4ehAW5EOXsNrc0TwIHw+ZAyCENSr8k6KUcgHeB8ZY0XYSMAmgSZMmFX1r\n4QTiL2cyb9MJvtsTz5XsfNxNLjQL9iHYzwOlFBdSs9lyPJHPNp/Ax93E0PYNmdSrGWFBPo4uXYgq\nzZpwPws0Lva4UcG2Qn5AO+AXZZyJVw9YqZQaqrW+6mocWus5wBwwLtZRgbpFNZeRk8+H/3eMBVtO\norVmYLv6PNipEV3CauPpdvXl/XLyzew+dZnv9pxl+d54vt11htFdmvDCgHACvOTKUkLcSJlXYlJK\nuQJHgX4Yob4TeEhrfaiE9r8AL1wb7Ne66SsxiWpv96nLPPdNFKeTM7mvQyOe69+ShoHWXYDlUlo2\nH/1fLIt3nCbY14N/PXgrPVsE27liIaoOa6/EVOYBVa11PvA0sBY4AnyjtT6klHpLKTW04qWKmuTL\nrScZ8d+tmC2aJRO78e8RkVYHO0CInydv39uO7ybfjo+HiUc/28HHPx/DUZeLFKKqqn7XUBXVktmi\n+duqQyzceop+rUKYOao9fp4VG1LJzjPz8rL9rIg6x9DIBsx4MBJ3VzkvTzg3572Gqqh28s0Wpn2z\nj1X7zjGxZxgvD2qNyaXiKyV6upn4z8j2tKznxz9/iiE9J5/ZD3e8bsxeiJpIujnCrvLMFqYu2cuq\nfed4aWArpt/dxibBXkgpxZQ+zXlneDs2xlxi/Bc7yc4z22z/QlRXEu7CbrTWvLRsP2sOXOC1u1sz\nuc8tdnuvh7s25V8PRPJ7bBJTv95Lvtlit/cSojqQcBd286+1MXy35yzP3dWSCT2b2f39HujUiDfu\nacO6wxd5bcVBOcgqajQZcxd28eW2U8z+5TijuzThz38q5/VHK2BsjzAS03OYtfE4DQK9mNqvRaW9\ntxBViYS7sLltcUm8ufIQ/VqF8PawtqhKvszcC/3DOZeSzfvrj9K6vj93talbqe8vRFUgwzLCps6l\nZPHUoj00rePNzFHtcTVV/n8xpRT/774Ibm0UwLSlUcReSqv0GoRwNAl3YTPZeWae/Go3OfkW5jza\nucLz2CvC083Ep490wtPNhYkLd3MlO89htQjhCBLuwmbeXn2Y/fGpvD8ikuYhvo4uhwaBXsx+uBOn\nkzOZvlwOsIqaRcJd2MRPBy+waPtpJvVqRv+29RxdTpEuYbV57q6WrNp3jm93xTu6HCEqjYS7qLDz\nqVm8/N1+IhoG8EL/cEeXc50ne9/C7bfU4Y2Vh2T8XdQYEu6iQswWzbSlUeTmW/hgVPsqubaLyUXx\nn5Ht8XI38fTivXIGq6gRqt5PoqhWPv31ONviknlzaFuaBTt+nL0kdf09+feDkURfSOP99UcdXY4Q\ndifhLm5azIU0Zm44yt23GhfaqOr6tgrhoa5NmLspjl0nkx1djhB2JeEuborZonlx2X78PN14a2jl\nn6h0s14d3JqGgV48/+0+MnPzHV2OEHYj4S5uyoLfT7DvTApvDm1LHV8PR5djNV8PV/71QCSnkjJ5\n78doR5cjhN1IuItyO5WUwYx1MdzZOoR7bq3v6HLKrfstdRhzeyhfbD3FlthER5cjhF1IuIty0Vrz\n8rIDuLm48Pa97arNcMy1XhrYirAgH/7yv/1k5MjwjHA+Eu6iXJbuPMPWuCReGdya+gHWX/u0qvFy\nN/HPB27lbEoWM9bFOLocIWxOwl1Y7UJqNu/8cIRuzWoz6rbGji6nwm4Lrc2j3Zry+ZaT7D192dHl\nCGFTEu7CKlprXltxkDyLhXfvuxUXG14qz5FeHBhOXT9PXl52gNx8uXqTcB4S7sIqq/efZ8ORizx/\nVzihQT6OLsdm/Dzd+Pu97Yi5mManvx53dDlC2IyEuyhTckYub648RGSjAMb2CHV0OTZ3Z5u6DLm1\nPh//HCtrzwinIeEuyvT26sOkZuXx3gO3OuTiG5XhjXva4uVu4qVlB7BYZGlgUf0550+qsJmN0ZdY\nvvcsU/o2p1U9f0eXYzfBfh78dUgbdp+6zKLtpxxdjhAVJuEuSpSWncf05QdoWdeXp/re4uhy7O7+\njg3p2SKId3+M5lxKlqPLEaJCJNxFid77KZrzV7J57/5b8XA1Obocu1NK8Y/hEVg0vLZCrtwkqjcJ\nd3FD2+OS+Grbacb1CKNDk1qOLqfSNK7tzfP9W/Jz9CVW7jvn6HKEuGkS7uI62XlmXv7uAE0Kgq6m\nGdsjjMjGgfxt1WGS0nMcXY4QN0XCXVxn5oZjnEjM4N37IvB2d3V0OZXO5KL45/23kpadx1urDzu6\nnGrveMpxNsVvIjMv09Gl1Cg17ydXlOpAfCpzN8Ux6rbG3N48yNHlOEx4PT+e6tucmRuOMTSyAf1a\n13V0SdWO1prF0YuZsXMG+TofVxdXIoMj6V6/O7c3uJ02ddpgcnH+YzmOoqw5aKSUGgh8AJiAeVrr\nd695/jlgApAPJADjtNalzifr3Lmz3rVr183WLewgN9/C0I83k5yRy/rnehPg5ebokhwqN9/CPR9t\nJjUrj/XP9cLPs2Z/P8ojIy+DN7a8wdqTa+nTqA+jWo1i54WdbDm3hSPJRwDwd/ena/2udG/Qne71\nu9PIr+pfzasqUErt1lp3LrNdWeGulDIBR4G7gHhgJzBaa324WJu+wHatdaZSajLQR2s9srT9SrhX\nPTM3HGXmhmPMe6wzd7aRnipA1JkU7pv9O6O7NOGd4RGOLqdaiL0cy7RfpnE67TRTO0xlbLuxuKg/\nRoCTs5PZfn47W89tZcu5LVzMvAhAE78mRtA36E6Xel3wc/dz1Eeo0qwNd2uGZboAsVrruIIdLwGG\nAUXhrrXeWKz9NuCR8pUrHO3I+St8/HMs97ZvIMFeTPvGgYzrEca8zSe4J7IB3ZrVcXRJVdoPcT/w\nt61/w9vVm3n953Fbvduua1PbszaDwgYxKGwQWmtOXDnB1nNb2XpuK6uOr2JpzFJMykREUERR2LcL\naoebi/zlVB7W9NwfAAZqrScKzDndAAAVr0lEQVQUPH4U6Kq1frqE9h8DF7TWf7/Bc5OASQBNmjTp\ndOqUnAlYFeSbLQyfvYXzqVmsn9abWj7uji6pSsnMzWfgzE24KPjp2V54upU+Tnwu/Ryrjq/igZYP\nUMerZvwyyDXn8s+d/2RpzFI6hnRkRu8ZBHsHl3s/eeY89iXsY+v5rWw7t42DSQexaAu+br7cVu82\nujcwxuub+DWptheKqShbDstYHe5KqUeAp4HeWutS55DJsEzVMfuXWP75UwyzH+7I4Ijqd9m8yrAl\nNpGH5m3nid7NeGVQ6xLbnUk7w/i14zmfcR5/d3+e7/w8w5sPd+ogOpd+jud/eZ6DSQcZ03YMUztO\ntVkvOzUnlR0XdrDl3Ba2ntvK2fSzADTwaVDUq+9WvxsBHgE2eT9bMFvMJGUncTHjIpcyL3Eh8wKX\nMi9xMfNi0ban2j/F4GaDb2r/thyWOQsUvzJDo4Jt177hncB0rAh2UXXEXkpj5oZjDI6oJ8Feitub\nBzHqtsbM/S2Owe3qE9k48Lo2Z9LOMG7tOLLys/h373+zOHoxb2x5g5XHV/J6t9dpFtjMAZXb16b4\nTbyy+RXMFjMz+8ykX9N+Nt1/gEcAdzW9i7ua3oXWmjNpZ4whnPNbWXtyLcuOLUOhaFunbVHYtw9u\nj5vJPkM4OeYcLmUUBHXmxaLQvpR5iYsZxrbErETM2nzV61xdXKnrXZe63nVpU6cNtTztf2KgNT13\nV4wDqv0wQn0n8JDW+lCxNh2A/2H08I9Z88bSc3e8fLOFEf/dSlxiBuun9SbYz8PRJVVpqVl5DPjP\nb/h4mPhhas+rhmfOXDnD2LVjyTZnM6//PFrVboVFW/g+9ntm7JpBZn4m49uNZ+KtE/EwVf/vs9li\n5pN9nzBn/xxa1GrB+33ep6l/00qtId+Sz8HEg0Vhvz9hP2ZtxsvVyxjCKZhyGRYQVuZfTlpr0vPS\ni3rWheFdPLgvZV7ics71V+zydvWmro8R3CHeIUUhXtfnj8e1PGtddVC5Imw2LFOws8HATIypkPO1\n1u8opd4CdmmtVyqlNgARwPmCl5zWWg8tbZ8S7o738c/HmLHuKB+Mas+w9g0dXU61sPlYIo98tp0x\nt4fy5tC2AJy+cppxa8eRY85hXv95hNcOv+o1SVlJzNg1g9Vxq2nq35S/dvsrXet3dUT5NpGcnczL\nv73M1vNbubf5vUzvOh1PV09Hl0VablrRdMtt57dx6opxTC/EO4Tu9Y1evber91U97sKhkouZF8nK\nv36xuNqetYsCuii4C0K7nnc9QrxD8HX3rdTPadNwtwcJd8faH5/CfbO3MLBdPT4a3cGpx4Rt7c2V\nh/h8y0m+Gt+VJnUzGbt2LHnmPOb2n3tdsBe39dxW3t72NmfSzjD0lqG80PmFSvnz3JaiLkXxwq8v\ncDn7MtO7Tee+Fvc5uqQSnU0/WzQLZ9v5bVzJvVL0nEmZCPYOLrW3HeIdgrup6k0ukHAXJcrKNXP3\nR5vIzDGz9tleBHjLFLPyyMo1M+SjTaSbL+AdOgezJZ95A+bRslbZ6/Bk52czZ/8cFhxcgK+7L893\nfp5htwyr8r9ci59tWtenLu/3eZ82ddo4uiyrmS1mYi7HYNEW6nrXpbZn7Wp7dqy14S5ry9RA7/54\nhLiEDP49IlKC/SZ4uZt48Z7aZNT+iNSsbKuDHcDT1ZOpHafy7T3fEhYQxl9//yvj143nROoJO1d9\n8zLyMvjLb3/h3R3vckejO1g6ZGm1CnYAk4uJNnXa0C6oHcHewdU22MtDwr2G+fVoAl9sPcW4HmH0\nqMFrx1TEydSTvBf1LF4ekBI3niOnyn/B8Oa1mvP5wM95o/sbRCdHc//K+/lk3yfkmnPtUPHNi70c\ny6jVo1h/aj3PdnyWD/p+UKWmHYqSSbjXIBevZPPc0iha1vXlxYEljw2Lkp1IPcG4teMwazNfDV5A\nx3qtefW7A5xIzCj3vlyUCw+0fICV967kzqZ3MjtqNg+seoBdF6rGcOUPcT/w0JqHSMtNY17/eYyP\nGG+zGR/C/uRfqobIN1uY+vVeMnPNzHqoY5lnWYrrxaXGFQX7/AHzCa/Tkg9Hd8DN1YUpi/aQnWcu\neyc3EOQVxD97/ZNP7jR67mPXjuX1318nJTvFxp/AOrnmXP6+7e+8vOllWtduzTf3fHPDZQRE1Sbh\nXkN88H/H2H4imb/f244WdWVBpvKKS41j/NrxaK2ZP2A+twQa15RtEOjF+yMiOXL+Cm9XcO33Oxre\nwfJhyxnfbjyrjq9i6IqhrDq+qlIv93cu/RyP//g4S2OWMqbtGOYNmEeId0ilvb+wHQn3GuC3owl8\nvDGWBzs14v5OsqxqecWlxDHup3HXBXuhP7WqyxO9mrFo+2lWVfDSfF6uXjzb6VmW3rOUxv6NeXXz\nq0xcP7FozrY9bT67mRGrR3Dyyklm9pnJ852fl8W6qjEJdycXfzmTZ5dG0SLEl7eGtXN0OdXO8ZTj\njF07FqUU8wfOL3EJgRcGhNOxSSAvL9vPsYtpFX7flrVa8uWgL3mt62scSjzEfd/fx5z9c8gz51V4\n39cyW8zMiprFlA1TqOtdlyVDlth8GQFR+STcnVhmbj4TF+4mL9/C7Ic74eUu4+zlEXs5lnFrx2FS\nJuYPmE+zgJLXhnEzuTDr4Y54ubsyYeEuUjIrPuvFRbkwstVIVt67kj6N+/DR3o94cNWD7Lm4p8L7\nLpScnczkDZP5dN+nDL1lKF8N/qrSlxEQ9iHh7qS01rzw7T6iL1zhw4c60Dykck+Rru6OXT7G+HXj\nMSkTnw34jLCAsDJfUz/Ai/8+2onzKdk8vXgv+WaLTWoJ9g7m333+zax+s8jKz+Lxnx7nzS1vkpqT\nWqH9Rl2KYsSqEey+uJs3u7/J2z3exsvVyyY1C8eTcHdSH/0cy5oDF3hlUCv6hssBsfI4dvkYE9ZN\nwFW5Mn/AfKuCvVCnprX4+/B2bI5N5J01R2xaV69GvVg+bDlj2o5hRewKhq4Yypq4NeU+4Kq1ZtGR\nRYz9aSyuLq58OfhL7m95f5U/S1aUj4S7E1q+N5731x9leIeGTOzpfMvM2tPRy0cZv3Y8ri6uzB84\nn9CA0HLvY0TnxozrEcaC30+y4Hfbnnnq7ebN852fZ8mQJTTwacBLm15i8obJnEk7Y9XrneFsU2Ed\nCXcn8+vRBP7y7X66N6vDu/dHSG+sHGKSY5iwdgJuJjcWDFhQobHn6Xe3ZmDbery1+jCr91dsBs2N\ntKrdiq8Gf8UrXV4hKiGK4d8PZ96BeeRZSj7gGns5ltE/jJazTWsICXcnsj8+hclf7aZFXT/++1gn\nPFzlAKq1YpJjmLBuAu4mdxYMWEAT/yYV2p/JRTFzVHtua1qb55buY0tsoo0qLf4eJh5q/RArhq2g\nZ8OefLDnA0asGkHUpajr2haebZqak8rcu+bK2aY1gPzrOoljF9MYu2AntX3c+WLsbfh7yvxkaxUG\nu4fJwybBXsjTzcTcxzoTGuTNpC93s/f09Rd6sIV6PvX4T9//8GHfD0nLTeOxHx/j7a1vcyX3ynVn\nm357z7d0qd/FLnWIqkWW/HUCxy6mMXruNlyUYukT3QkLKv9CVjVVdHI0E9ZNwMvVi/n959PYv3HZ\nLyqn86lZjJqzjeT0XL4Y34WOTey3hntGXgazomax6MgianvWJtgrmCPJR2x+bVPhOLLkbw1RPNi/\nntRNgr0cjiQdYfza8Xi7ejN/gH2CHYwpkksmdaO2rzuPf7aDPXbqwQP4uPnw4m0vsvjuxQR7BXMm\n7YycbVpDSc+9Gtt7+jLjv9iFq4sR7LcEy1x2ax1OOszEdRPxcfNh/oD5NPKz/7IM51OzGD1nG4np\nucx5rBO332LfJZct2kKOOUfmrjsZ6bk7uZ+jL/LQ3O34erjyzRPdJdjL4VDSISaum4ivm2+lBTsU\n9uC70yDQk8fn72BlBdehKYuLcpFgr8Ek3KuhRdtPMXHhbm4J8WHZ5NsJlaEYqx1KNILdz92P+QMr\nL9gL1Qvw5Nsnb6djk1pM/Xovn/56vFJXfRQ1h4R7NZKTb+aV7/YzfflB7mgexJJJ3Qn283B0WdXG\nocRDTFw/EX93f+YPmE9D34YOqSPAy42F47sw5Nb6vPtjNH/+ei+ZufkOqUU4L1dHFyCsczYli6cX\n72Hv6RSm9LmF5/uHY3KRE5SsdTDxIJPWTcLfwwj2Br4NHFqPh6uJj0Z3oE0Df2asjeHYxXQ+eaQj\nzWR4TdiI9NyrOK01y/fGM/A/v3H0QhqzH+7IiwNbSbCXw4GEA0xcN5EAjwAWDFjg8GAvpJRiSp/m\nLBzXlUtp2dz94Wa+3HZKhmmETUi4V2GX0ozVBact3Ud4PT9+fKYXgyPqO7qsamV/wn4mrZ9EoEcg\nCwYuoL5v1fv+3dEiiB+f6UXn0Fr8dcVBHl+wk7MpWY4uS1RzMhWyCso3W/hy2yneX3eUnHwLz9zZ\ngid73yK99XLal7CPJ9c/SS3PWswfMJ96PvUcXVKptNZ8te0U76w5gkLx9J+aM6FnmCwjIa5i7VTI\nahfuOeYcXJSLU56QobVm3eGLvL/uKDEX0+jZIoi3hrWTE5NuQtSlKJ7c8CR1POvw2YDPqnywFxd/\nOZO3Vx9m7aGLhAX5MO2ulgyJqI+L/HIXOHG4f3HoC2bsmoG/uz91vOpQ27N20a2OZ8Fjr6u/9nPz\nq9KrI1osml+PJvCfDUfZH59KsyAf/jIgnIHt6lXpuquq4sE+f8B86vrUdXRJN+WXmEv8vzXRxFxM\no2VdX569syUD2taTv+BqOKcN930J+9hybgvJWckkZyeTlJ1EcrbxdUlXpnF1cf0j/IsHf/Fbse3u\nJveKfjyrZOTks2xPPJ9vOUlcQgYNA7145s4W3NehIa4mORxyM6IuRfHE+icI9g7ms/6fVdtgL2Sx\naH44cJ6ZG45yPCGDxrW9eLRbU0Z2bkKAt/P99SrK5rThXpo8Sx4p2SlXhX5S1h/hn5ydfNUvhRxz\nzg334+fmR22va/4i8Kp93V8J3m7e5Fvy/7jp/BIfmy1m8nQeufl5HD5/md2nEzl8PoVccz4NAt3p\nFOrPLSFeoCxX7SPPkodZm6/aZtZm8ix5RY8t2oJGG/dao9HX3Re2ueFjrbFw9WtLbX/NfeFrq4LM\nvEwa+TXiswGfEeLtPFegMls0aw9d4PMtJ9lxIhkPVxfubFOXoZEN6BMeLOPyNUiNDPfy0FqTmZ9J\nctbVvf/ivwSKb7+cfRlN5XyvXJUrri5/3EzKVPS1m4vbddtMyoRSCoXCRbmgUKDABRdj+zXPWfVY\nqT9eX7CtaN9w3WsL7x3N09WTR1o/QrB3sKNLsZvD567w9Y7T/HDgPMkZufh5utI3PIReLYPp1SKI\nEH9PR5co7Mim4a6UGgh8AJiAeVrrd6953gNYCHQCkoCRWuuTpe3T0eFeXmaLmZSclKt+AWTmZRrh\n6mIErZtyw0WZSEzLIy4hm7iELI5dzOJUUjba4oK/pwedmwbRr1V9ejSvi7eb+w2DXMbZhTXyzBZ+\nj01k1b7z/Hr0EonpuQC0rOtLxya1aN84kPZNAmke7CvDfE7EZuGulDIBR4G7gHhgJzBaa324WJsp\nwK1a6yeVUqOA4VrrkaXtt7qFe3E5+WYupuZwPjWLC1eyOZGYQeyldGIvpXMiMYOcfOOq936errRv\nHEjnprXpHR5MRMMAORgm7MJi0Rw+f4XfjiWwPS6ZqDMppGYZl9xzN7kQGuRN8xBfmgf7EhrkQ/0A\nL+oHeFIvwBNPNxnSqU6sDXdrlh/oAsRqreMKdrwEGAYcLtZmGPBmwdf/Az5WSilt5zEfY2zYGI+0\naONmfG38Z7dojVlrLBaKntMa8i0WsvMsZOebySm6Nxvb8sxk55lJz8knNSuPlMw84z4rjytZeSSm\n5xT1kAopBY1qedE82JeeLYJoUdePjk0CaRbkK9PXRKVwcVG0axhAu4YBTOlj/GycSMwg6kwKMRfT\nOH4pncPnrvDTwQtYrvmprO3jTpCvO4Fe7gR4uxHo5UagtxsBXm54u7vi5W7Cy82Ep5sJTzcXvNxM\neLmbcHd1wdVFYXIpvFfF7l0wmYzHLkpR+GNgDN8ZPzOFj4V9WBPuDYHil1aPB7qW1EZrna+USgXq\nADa/cOTc3+J476dozNoIanvycjMRUOw/epPa3nRoEkj9AC/qBXhSv+DWMNAbL3fp/YiqQylFs2Df\n69aqyc4zcy4li/Op2ZxPzeZCahbnUrNJSs8hNSuPM8mZHCzo1GTlmSux3oJ7/vgFULi98BjSH89T\ndHynuv5ueOOeNoy8zTaXcyxJpS4cppSaBEwqeJiulIqpzPe3kSDs8Euriqtpn7mmfV6Qz1ypRr0N\no27+5U2taWRNuJ8Fil9/rFHBthu1iVdKuQIBGAdWr6K1ngPMsaawqkoptcua8S5nUtM+c037vCCf\n2RlZcwh9J9BCKRWmlHLH+IWz8po2K4HHC75+APjZ3uPtQgghSlZmz71gDP1pYC3GVMj5WutDSqm3\ngF1a65XAZ8CXSqlYIJkK/cUhhBCioqwac9darwHWXLPt9WJfZwMP2ra0KqtaDyvdpJr2mWva5wX5\nzE7HYWeoCiGEsB85bU0IIZyQhHsFKKWeV0pppVSQo2uxJ6XUv5RS0Uqp/Uqp5UqpQEfXZC9KqYFK\nqRilVKxS6mVH12NvSqnGSqmNSqnDSqlDSqlnHF1TZVFKmZRSe5VSqx1diz1IuN8kpVRjoD9w2tG1\nVIL1QDut9a0YS1G84uB67KJgqY1ZwCCgDTBaKdXGsVXZXT7wvNa6DdANeKoGfOZCzwBHHF2EvUi4\n37z/AC9CJS0V6UBa63Va6/yCh9swznVwRkVLbWitc4HCpTacltb6vNZ6T8HXaRhh19CxVdmfUqoR\ncDcwz9G12IuE+01QSg0Dzmqt9zm6FgcYB/zo6CLs5EZLbTh90BVSSoUCHYDtjq2kUszE6JxZHF2I\nvVTq8gPViVJqA3CjC29OB17FGJJxGqV9Xq319wVtpmP8Gb+oMmsT9qeU8gWWAc9qra84uh57UkoN\nAS5prXcrpfo4uh57kXAvgdb6zhttV0pFAGHAvoIV7RoBe5RSXbTWFyqxRJsq6fMWUkqNAYYA/Zz4\n7GNrltpwOkopN4xgX6S1/s7R9VSCHsBQpdRgwBPwV0p9pbV+xMF12ZTMc68gpdRJoLPW2mkXXSq4\nWMv7QG+tdYKj67GXgnWRjgL9MEJ9J/CQ1vqQQwuzI2X0UL4AkrXWzzq6nspW0HN/QWs9xNG12JqM\nuQtrfAz4AeuVUlFKqU8dXZA9FBw0Llxq4wjwjTMHe4EewKPAnwr+baMKerSimpOeuxBCOCHpuQsh\nhBOScBdCCCck4S6EEE5Iwl0IIZyQhLsQQjghCXchhHBCEu5CCOGEJNyFEMIJ/X++8FDL8Ea3RQAA\nAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cqzy0keT8ZvD",
        "colab_type": "text"
      },
      "source": [
        "https://poloclub.github.io/ganlab/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zZk8x-IeiiUf",
        "colab_type": "text"
      },
      "source": [
        "### DGAN Architecture\n",
        "\n",
        "Most GANs today are at least loosely based on the DCGAN architecture (Radford\n",
        "et al., 2015). DCGAN stands for “deep, convolution GAN.”\n",
        "\n",
        "Some of the key insights of the DCGAN architecture were to:\n",
        "\n",
        "+ Use batch normalization layers in most layers\n",
        "of both the discriminator and the generator, with the two minibatches for\n",
        "the discriminator normalized separately. \n",
        "\n",
        "+ The overall network structure is mostly borrowed from the all-convolutional\n",
        "net. \n",
        "\n",
        "+ The use of the Adam optimizer rather than SGD with momentum. \n",
        "\n",
        "<center>\n",
        "<img src=\"https://github.com/DataScienceUB/DeepLearningMaster2019/blob/master/images/dgan.png?raw=1\" alt=\"\" style=\"width: 700px;\"/> \n",
        "(Source: https://arxiv.org/pdf/1511.06434.pdf)\n",
        "</center>\n",
        "\n",
        "\n",
        "Generated bedrooms after one training pass through the dataset: \n",
        "\n",
        "\n",
        "<center>\n",
        "<img src=\"https://github.com/DataScienceUB/DeepLearningMaster2019/blob/master/images/dgan2.png?raw=1\" alt=\"\" style=\"width: 700px;\"/> \n",
        "(Source: https://github.com/Newmu/dcgan_code)\n",
        "</center>\n",
        "\n",
        "\n",
        "Covers: \n",
        "\n",
        "<center>\n",
        "<img src=\"https://github.com/DataScienceUB/DeepLearningMaster2019/blob/master/images/dgan3.png?raw=1\" alt=\"\" style=\"width: 700px;\"/> \n",
        "(Source: https://github.com/Newmu/dcgan_code)\n",
        "</center>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ARPiIwav9ElT",
        "colab_type": "text"
      },
      "source": [
        "## MNIST GAN\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h4wSKWe09JhO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Keras imports \n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Activation, Flatten, Reshape\n",
        "from tensorflow.keras.layers import Conv2D, UpSampling2D\n",
        "from tensorflow.keras.layers import LeakyReLU, Dropout\n",
        "from tensorflow.keras.layers import BatchNormalization\n",
        "from tensorflow.keras.optimizers import Adam, SGD, RMSprop\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython.display import clear_output, Image\n",
        "\n",
        "# We'll be downloading and reading in the MNIST data from Tensorflow\n",
        "from tensorflow.examples.tutorials.mnist import input_data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-say641A9O9m",
        "colab_type": "code",
        "outputId": "3c067cc2-5c18-44ff-e6b9-24a13fa7a000",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import keras.backend.tensorflow_backend as ktf\n",
        "import tensorflow as tf\n",
        "import os\n",
        "\n",
        "def get_session(gpu_fraction=0.45):\n",
        "    '''Assume that you have 6GB of GPU memory and want to allocate ~2GB'''\n",
        "\n",
        "    num_threads = os.environ.get('OMP_NUM_THREADS')\n",
        "    gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=gpu_fraction)\n",
        "\n",
        "    if num_threads:\n",
        "        return tf.Session(config=tf.ConfigProto(\n",
        "            gpu_options=gpu_options, intra_op_parallelism_threads=num_threads))\n",
        "    else:\n",
        "        return tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))\n",
        "\n",
        "ktf.set_session(get_session())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cJ9B-9qm9Ym-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def discriminator():\n",
        "    \n",
        "    net = Sequential()\n",
        "    input_shape = (28, 28, 1)\n",
        "    dropout_prob = 0.4\n",
        "\n",
        "    net.add(Conv2D(64, 5, strides=2, input_shape=input_shape, padding='same'))\n",
        "    net.add(LeakyReLU())\n",
        "    \n",
        "    net.add(Conv2D(128, 5, strides=2, padding='same'))\n",
        "    net.add(LeakyReLU())\n",
        "    net.add(Dropout(dropout_prob))\n",
        "    \n",
        "    net.add(Conv2D(256, 5, strides=2, padding='same'))\n",
        "    net.add(LeakyReLU())\n",
        "    net.add(Dropout(dropout_prob))\n",
        "    \n",
        "    net.add(Conv2D(512, 5, strides=1, padding='same'))\n",
        "    net.add(LeakyReLU())\n",
        "    net.add(Dropout(dropout_prob))\n",
        "    \n",
        "    net.add(Flatten())\n",
        "    net.add(Dense(1))\n",
        "    net.add(Activation('sigmoid'))\n",
        "    \n",
        "    return net"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ygpfUd9H9bKD",
        "colab_type": "code",
        "outputId": "a0c3451c-2623-4f97-d029-9fc130e51a5a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 740
        }
      },
      "source": [
        "net_discriminator = discriminator()\n",
        "net_discriminator.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d (Conv2D)              (None, 14, 14, 64)        1664      \n",
            "_________________________________________________________________\n",
            "leaky_re_lu (LeakyReLU)      (None, 14, 14, 64)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_1 (Conv2D)            (None, 7, 7, 128)         204928    \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_1 (LeakyReLU)    (None, 7, 7, 128)         0         \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 7, 7, 128)         0         \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 4, 4, 256)         819456    \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_2 (LeakyReLU)    (None, 4, 4, 256)         0         \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 4, 4, 256)         0         \n",
            "_________________________________________________________________\n",
            "conv2d_3 (Conv2D)            (None, 4, 4, 512)         3277312   \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_3 (LeakyReLU)    (None, 4, 4, 512)         0         \n",
            "_________________________________________________________________\n",
            "dropout_2 (Dropout)          (None, 4, 4, 512)         0         \n",
            "_________________________________________________________________\n",
            "flatten (Flatten)            (None, 8192)              0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 1)                 8193      \n",
            "_________________________________________________________________\n",
            "activation (Activation)      (None, 1)                 0         \n",
            "=================================================================\n",
            "Total params: 4,311,553\n",
            "Trainable params: 4,311,553\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PiuMfGi49ijc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generator():\n",
        "    \n",
        "    net = Sequential()\n",
        "    dropout_prob = 0.4\n",
        "\n",
        "    net.add(Dense(7*7*256, input_dim=100))\n",
        "    net.add(BatchNormalization(momentum=0.9))\n",
        "    net.add(LeakyReLU())\n",
        "    net.add(Reshape((7,7,256)))\n",
        "    net.add(Dropout(dropout_prob))\n",
        "    \n",
        "    net.add(UpSampling2D())\n",
        "    net.add(Conv2D(128, 5, padding='same'))\n",
        "    net.add(BatchNormalization(momentum=0.9))\n",
        "    net.add(LeakyReLU())\n",
        "    \n",
        "    net.add(UpSampling2D())\n",
        "    net.add(Conv2D(64, 5, padding='same'))\n",
        "    net.add(BatchNormalization(momentum=0.9))\n",
        "    net.add(LeakyReLU())\n",
        "    \n",
        "    net.add(Conv2D(32, 5, padding='same'))\n",
        "    net.add(BatchNormalization(momentum=0.9))\n",
        "    net.add(LeakyReLU())\n",
        "    \n",
        "    net.add(Conv2D(1, 5, padding='same'))\n",
        "    net.add(Activation('sigmoid'))\n",
        "    \n",
        "    return net"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kNZPCWs_9mFh",
        "colab_type": "code",
        "outputId": "6517fcb2-9f51-422c-820e-17d1579a3689",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 754
        }
      },
      "source": [
        "net_generator = generator()\n",
        "net_generator.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_1 (Dense)              (None, 12544)             1266944   \n",
            "_________________________________________________________________\n",
            "batch_normalization_v1 (Batc (None, 12544)             50176     \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_4 (LeakyReLU)    (None, 12544)             0         \n",
            "_________________________________________________________________\n",
            "reshape (Reshape)            (None, 7, 7, 256)         0         \n",
            "_________________________________________________________________\n",
            "dropout_3 (Dropout)          (None, 7, 7, 256)         0         \n",
            "_________________________________________________________________\n",
            "up_sampling2d (UpSampling2D) (None, 14, 14, 256)       0         \n",
            "_________________________________________________________________\n",
            "conv2d_4 (Conv2D)            (None, 14, 14, 128)       819328    \n",
            "_________________________________________________________________\n",
            "batch_normalization_v1_1 (Ba (None, 14, 14, 128)       512       \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_5 (LeakyReLU)    (None, 14, 14, 128)       0         \n",
            "_________________________________________________________________\n",
            "up_sampling2d_1 (UpSampling2 (None, 28, 28, 128)       0         \n",
            "_________________________________________________________________\n",
            "conv2d_5 (Conv2D)            (None, 28, 28, 64)        204864    \n",
            "_________________________________________________________________\n",
            "batch_normalization_v1_2 (Ba (None, 28, 28, 64)        256       \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_6 (LeakyReLU)    (None, 28, 28, 64)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_6 (Conv2D)            (None, 28, 28, 32)        51232     \n",
            "_________________________________________________________________\n",
            "batch_normalization_v1_3 (Ba (None, 28, 28, 32)        128       \n",
            "_________________________________________________________________\n",
            "leaky_re_lu_7 (LeakyReLU)    (None, 28, 28, 32)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_7 (Conv2D)            (None, 28, 28, 1)         801       \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 28, 28, 1)         0         \n",
            "=================================================================\n",
            "Total params: 2,394,241\n",
            "Trainable params: 2,368,705\n",
            "Non-trainable params: 25,536\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "puVB-4WX9otj",
        "colab_type": "code",
        "outputId": "d3a646cb-3c10-44a1-b18a-0810122c1733",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 171
        }
      },
      "source": [
        "dis_lr=0.0008\n",
        "dis_clipvalue=1.0\n",
        "dis_decay=1e-10\n",
        "dis_optimizer = RMSprop\n",
        "\n",
        "optim_discriminator = RMSprop(lr=dis_lr, clipvalue=dis_clipvalue, decay=dis_decay)\n",
        "model_discriminator = Sequential()\n",
        "model_discriminator.add(net_discriminator)\n",
        "model_discriminator.compile(loss='binary_crossentropy', optimizer=optim_discriminator, metrics=['accuracy'])\n",
        "\n",
        "model_discriminator.summary()\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "sequential (Sequential)      (None, 1)                 4311553   \n",
            "=================================================================\n",
            "Total params: 4,311,553\n",
            "Trainable params: 4,311,553\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iy61vU2C9tkM",
        "colab_type": "code",
        "outputId": "ea8b766e-dc30-4163-803c-eda998c17844",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "source": [
        "adv_lr=0.0004\n",
        "adv_clipvalue=1.0\n",
        "adv_decay=1e-10\n",
        "adv_optimizer = Adam\n",
        "\n",
        "optim_adversarial = Adam(lr=adv_lr, clipvalue=adv_clipvalue, decay=adv_decay)\n",
        "model_adversarial = Sequential()\n",
        "model_adversarial.add(net_generator)\n",
        "\n",
        "# Disable layers in discriminator\n",
        "for layer in net_discriminator.layers:\n",
        "    layer.trainable = False\n",
        "\n",
        "model_adversarial.add(net_discriminator)\n",
        "model_adversarial.compile(loss='binary_crossentropy', optimizer=optim_adversarial, metrics=['accuracy'])\n",
        "model_adversarial.summary()\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "sequential_1 (Sequential)    (None, 28, 28, 1)         2394241   \n",
            "_________________________________________________________________\n",
            "sequential (Sequential)      (None, 1)                 4311553   \n",
            "=================================================================\n",
            "Total params: 6,705,794\n",
            "Trainable params: 2,368,705\n",
            "Non-trainable params: 4,337,089\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BouM-OD59x8G",
        "colab_type": "code",
        "outputId": "180dfa14-1872-48b8-dd1a-a6d86c251c25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 603
        }
      },
      "source": [
        "# Read MNIST data\n",
        "x_train = input_data.read_data_sets(\"mnist\", one_hot=True).train.images\n",
        "x_train = x_train.reshape(-1, 28, 28, 1).astype(np.float32)\n",
        "\n",
        "# Map the images to a new range [-1, 1]\n",
        "#x_train = x_train / 0.5 - 1\n",
        "\n",
        "import os\n",
        "os.makedirs(\"output/mnist-normal\")\n",
        "\n",
        "batch_size = 256\n",
        "\n",
        "vis_noise = np.random.uniform(-1.0, 1.0, size=[16, 100])\n",
        "\n",
        "loss_adv = []\n",
        "loss_dis = []\n",
        "acc_adv = []\n",
        "acc_dis = []\n",
        "plot_iteration = []\n",
        "\n",
        "for i in range(10001):\n",
        "    \n",
        "    # Select a random set of training images from the mnist dataset\n",
        "    images_train = x_train[np.random.randint(0, x_train.shape[0], size=batch_size), :, :, :]\n",
        "    # Generate a random noise vector\n",
        "    noise = np.random.uniform(-1.0, 1.0, size=[batch_size, 100])\n",
        "    # Use the generator to create fake images from the noise vector\n",
        "    images_fake = net_generator.predict(noise)\n",
        "    \n",
        "    # Create a dataset with fake and real images\n",
        "    x = np.concatenate((images_train, images_fake))\n",
        "    y = np.ones([2*batch_size, 1])\n",
        "    y[batch_size:, :] = 0 \n",
        "\n",
        "    # Train discriminator for one batch\n",
        "    d_stats = model_discriminator.train_on_batch(x, y)\n",
        "    \n",
        "    # Train the generator\n",
        "    # The input of the adversarial model is a list of noise vectors. The generator is 'good' if the discriminator classifies\n",
        "    # all the generated images as real. Therefore, the desired output is a list of all ones.\n",
        "    y = np.ones([batch_size, 1])\n",
        "    noise = np.random.uniform(-1.0, 1.0, size=[batch_size, 100])\n",
        "    a_stats = model_adversarial.train_on_batch(noise, y)\n",
        "        \n",
        "           \n",
        "    if i % 500 == 0:\n",
        "        # Visualize the performance of the generator by producing images from the test vector\n",
        "        images = net_generator.predict(vis_noise)\n",
        "        # Map back to original range\n",
        "        #images = (images + 1 ) * 0.5\n",
        "        plt.figure(figsize=(10,10))\n",
        "        \n",
        "        for im in range(images.shape[0]):\n",
        "            plt.subplot(4, 4, im+1)\n",
        "            image = images[im, :, :, :]\n",
        "            image = np.reshape(image, [28, 28])\n",
        "            \n",
        "            plt.imshow(image, cmap='gray')\n",
        "            plt.axis('off')\n",
        "        \n",
        "        plt.tight_layout()\n",
        "        plt.savefig(r'output/mnist-normal/{}.png'.format(i))\n",
        "        plt.close('all')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From <ipython-input-9-c3040ac31f1d>:1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please write your own downloading logic.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use urllib or similar directly.\n",
            "Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use tf.data to implement this functionality.\n",
            "Extracting mnist/train-images-idx3-ubyte.gz\n",
            "Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use tf.data to implement this functionality.\n",
            "Extracting mnist/train-labels-idx1-ubyte.gz\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use tf.one_hot on tensors.\n",
            "Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n",
            "Extracting mnist/t10k-images-idx3-ubyte.gz\n",
            "Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n",
            "Extracting mnist/t10k-labels-idx1-ubyte.gz\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
            "WARNING:tensorflow:Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ?\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iGJtV46d-Mrl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# create a gif from the  generator's saved output (in that output/mnist-normal directory)\n",
        "\n",
        "import imageio\n",
        "\n",
        "filenames = [r'output/mnist-normal/{}.png'.format(i * 500) for i in range(20)]\n",
        "images = []\n",
        "for filename in filenames:\n",
        "    images.append(imageio.imread(filename))\n",
        "imageio.mimsave(r'output/mnist-normal/learning.gif', images, duration=0.5)\n",
        "\n",
        "Image(url='output/mnist-normal/learning.gif')  \n",
        "\n",
        "plt.figure(figsize=(15,4))\n",
        "\n",
        "for i in range(10):\n",
        "    noise = np.zeros([1,100]) - 1 + (i * 0.2)\n",
        "    images = net_generator.predict(noise)\n",
        "    \n",
        "    image = images[0, :, :, :]\n",
        "    image = np.reshape(image, [28, 28])\n",
        "          \n",
        "    plt.subplot(1, 10, i+1)\n",
        "    plt.imshow(image, cmap='gray')\n",
        "    plt.axis('off')\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.savefig(r'output/mnist-normal/morph_example.png'.format(i))\n",
        "experiment.log_image(r'output/mnist-normal/morph_example.png'.format(i))\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PWGEAeww-lFu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.patches as plot_patch\n",
        "\n",
        "plt.figure(figsize=(15,6))\n",
        "noise = np.random.uniform(-1.0, 1.0, size=[40, 100])\n",
        "images_fake = net_generator.predict(noise)\n",
        "images_real = x_train[np.random.randint(0, x_train.shape[0], size=40), :, :, :]\n",
        "choice_vector = np.random.uniform(0, 1, size=40)\n",
        "\n",
        "for i in range(40):\n",
        "    \n",
        "    if choice_vector[i] > 0.5:\n",
        "        image = images_fake[i, :, :, :]\n",
        "    else:\n",
        "        image = images_real[i]\n",
        "    image = np.reshape(image, [28, 28])\n",
        "\n",
        "    plt.subplot(4, 10, i+1)\n",
        "    plt.imshow(image, cmap='gray')\n",
        "    plt.axis('off')\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.show()\n",
        "\n",
        "plt.figure(figsize=(15,6))\n",
        "\n",
        "border = np.zeros((28,28,3))\n",
        "border[0,:] = [255,0,0]\n",
        "border[:,0] = [255,0,0]\n",
        "\n",
        "for i in range(40):\n",
        "    \n",
        "    if choice_vector[i] > 0.5:\n",
        "        image = images_fake[i, :, :, :]\n",
        "    else:\n",
        "        image = images_real[i]\n",
        "    image = np.reshape(image, [28, 28])\n",
        "    \n",
        "    ax = plt.subplot(4, 10, i+1)\n",
        "    plt.imshow(image, cmap='gray')\n",
        "    if choice_vector[i] > 0.5:\n",
        "        ax.add_patch(plot_patch.Rectangle((0,0), 27, 27, edgecolor=\"red\", linewidth=2, fill=False))   \n",
        "    plt.axis('off')\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.savefig(r'output/mnist-normal/final_results_sample.png'.format(i))\n",
        "experiment.log_image(r'output/mnist-normal/final_results_sample.png'.format(i))\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEuLeE5BiiUf",
        "colab_type": "text"
      },
      "source": [
        "https://www.youtube.com/watch?v=kSLJriaOumA&feature=youtu.be\n",
        "\n",
        "https://deephunt.in/the-gan-zoo-79597dc8c347\n",
        "\n"
      ]
    }
  ]
}