Ch08.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q5GmEbUi6OWW"
   },
   "source": [
    "# 8장. 이미지를 분류합니다"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MIQLUnzA6OWY"
   },
   "source": [
    "이 노트북을 주피터 노트북 뷰어(nbviewer.jupyter.org)로 보거나 구글 코랩(colab.research.google.com)에서 실행할 수 있습니다.\n",
    "\n",
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://nbviewer.org/github/rickiepark/do-it-dl/blob/master/Ch08.ipynb\"><img src=\"https://jupyter.org/assets/share.png\" width=\"60\" />주피터 노트북 뷰어로 보기</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/rickiepark/do-it-dl/blob/master/Ch08.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />구글 코랩(Colab)에서 실행하기</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eChsadR86OWZ"
   },
   "source": [
    "이 노트북을 실행하려면 텐서플로 2.0.0-alpha0 버전 이상이 필요합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uXU4z23n6OWZ"
   },
   "source": [
    "## 08-1 합성곱을 알아 봅니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "SxuXuS4C6OWa"
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "l5JvrUlR6OWa"
   },
   "outputs": [],
   "source": [
    "w = np.array([2, 1, 5, 3])\n",
    "x = np.array([2, 8, 3, 7, 1, 2, 0, 4, 5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3S3DqKZk6OWa",
    "outputId": "948c288d-3914-413c-fef5-7e259d90cc54"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3 5 1 2]\n"
     ]
    }
   ],
   "source": [
    "w_r = np.flip(w)\n",
    "print(w_r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-5ufK6JN6OWc",
    "outputId": "92c5abc0-6d11-4e5c-9374-0a662e351af4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 5])"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w[0:4:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FPJGNzJA6OWc",
    "outputId": "505f36b0-f59f-4710-d43e-12d63e09d3b0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "63\n",
      "48\n",
      "49\n",
      "28\n",
      "21\n",
      "20\n"
     ]
    }
   ],
   "source": [
    "for i in range(6):\n",
    "    print(np.dot(x[i:i+4], w_r))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "0dtoaSKD6OWd"
   },
   "outputs": [],
   "source": [
    "from scipy.signal import convolve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Hi3hkb-r6OWd",
    "outputId": "3e78bf17-fb6e-4f58-f038-298d4478ac2f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([63, 48, 49, 28, 21, 20])"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "convolve(x, w, mode='valid')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "GS0cPXHK6OWd"
   },
   "outputs": [],
   "source": [
    "from scipy.signal import correlate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FcRjii8J6OWe",
    "outputId": "3d6a016d-141d-4677-d7e2-f3cfa77f1456"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([48, 57, 24, 25, 16, 39])"
      ]
     },
     "execution_count": 9,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlate(x, w, mode='valid')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7lXlSMxc6OWe",
    "outputId": "dc657cb0-d062-4252-e115-b7bfb78233f4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 6, 34, 51, 48, 57, 24, 25, 16, 39, 29, 13, 10])"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlate(x, w, mode='full')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KYMVUooo6OWe",
    "outputId": "3af28ee6-c6d0-47f4-f0bb-2522af162d82"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([34, 51, 48, 57, 24, 25, 16, 39, 29])"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlate(x, w, mode='same')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "j4b1Pkps6OWf"
   },
   "outputs": [],
   "source": [
    "x = np.array([[1, 2, 3],\n",
    "              [4, 5, 6],\n",
    "              [7, 8, 9]])\n",
    "w = np.array([[2, 0], [0, 0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "quumEEy96OWf",
    "outputId": "981540c2-7722-40f3-c20b-8d601745c2b5",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2,  4],\n",
       "       [ 8, 10]])"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.signal import correlate2d\n",
    "\n",
    "correlate2d(x, w, mode='valid')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "s7wIR2C_6OWf",
    "outputId": "01e2f588-7d7a-4e79-ab26-777923816501"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0],\n",
       "       [0, 2]])"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.flip(w)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sqZeuJmo6OWf",
    "outputId": "d49a3307-c6f7-4165-d225-8748aa68126b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10, 12],\n",
       "       [16, 18]])"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.signal import convolve2d\n",
    "\n",
    "convolve2d(x, w, mode='valid')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yFHOUskT6OWg",
    "outputId": "07808b1b-bc8e-40ee-c38a-dfed90859646"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2,  4,  6],\n",
       "       [ 8, 10, 12],\n",
       "       [14, 16, 18]])"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlate2d(x, w, mode='same')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "MOHWYzi96OWg"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "6jihrhpM6OWg"
   },
   "outputs": [],
   "source": [
    "x_4d = x.astype(np.float).reshape(1, 3, 3, 1)\n",
    "w_4d = w.reshape(2, 2, 1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KNSrHrd36OWg",
    "outputId": "180b82bc-fe27-49c3-8b31-332c27a7a537"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.,  4.,  6.],\n",
       "       [ 8., 10., 12.],\n",
       "       [14., 16., 18.]])"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_out = tf.nn.conv2d(x_4d, w_4d, strides=1, padding='SAME')\n",
    "c_out.numpy().reshape(3, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l95yQlSN6OWg"
   },
   "source": [
    "## 08-2 풀링에 대해서 알아 봅니다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "TUIfxqsH6OWh"
   },
   "outputs": [],
   "source": [
    "x = np.array([[1, 2, 3, 4],\n",
    "              [5, 6, 7, 8],\n",
    "              [9, 10, 11, 12],\n",
    "              [13, 14, 15, 16]])\n",
    "x = x.reshape(1, 4, 4, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "g37DZxjY6OWi",
    "outputId": "403c60b2-1dd3-4fea-edc9-bb01eea0e390"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 6.,  8.],\n",
       "       [14., 16.]], dtype=float32)"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_out = tf.nn.max_pool2d(x, ksize=2, strides=2, padding='VALID')\n",
    "p_out.numpy().reshape(2, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9KFwcpA96OWi"
   },
   "source": [
    "## 08-3 합성곱 신경망의 구조를 알아 봅니다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "UBJxIWTm6OWi"
   },
   "outputs": [],
   "source": [
    "def relu(x):\n",
    "    return np.maximum(x, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kJ66NUKm6OWi",
    "outputId": "185052b4-0646-4a49-8afa-ed8c4405ceda"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2, 0, 4, 0])"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([-1, 2, -3, 4, -5])\n",
    "\n",
    "relu(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "r1QDahKf6OWi",
    "outputId": "93c6f7a6-944f-432a-f9e1-a04f55a91e03"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2, 0, 4, 0])"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_out = tf.nn.relu(x)\n",
    "r_out.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l1VeCI4p6OWi"
   },
   "source": [
    "## 08-4 합성곱 신경망을 만듭니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "UEXzRXG16OWi"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "class ConvolutionNetwork:\n",
    "    \n",
    "    def __init__(self, n_kernels=10, units=10, batch_size=32, learning_rate=0.1):\n",
    "        self.n_kernels = n_kernels  # 합성곱의 커널 개수\n",
    "        self.kernel_size = 3        # 커널 크기\n",
    "        self.optimizer = None       # 옵티마이저\n",
    "        self.conv_w = None          # 합성곱 층의 가중치\n",
    "        self.conv_b = None          # 합성곱 층의 절편\n",
    "        self.units = units          # 은닉층의 뉴런 개수\n",
    "        self.batch_size = batch_size  # 배치 크기\n",
    "        self.w1 = None              # 은닉층의 가중치\n",
    "        self.b1 = None              # 은닉층의 절편\n",
    "        self.w2 = None              # 출력층의 가중치\n",
    "        self.b2 = None              # 출력층의 절편\n",
    "        self.a1 = None              # 은닉층의 활성화 출력\n",
    "        self.losses = []            # 훈련 손실\n",
    "        self.val_losses = []        # 검증 손실\n",
    "        self.lr = learning_rate     # 학습률\n",
    "\n",
    "    def forpass(self, x):\n",
    "        # 3x3 합성곱 연산을 수행합니다.\n",
    "        c_out = tf.nn.conv2d(x, self.conv_w, strides=1, padding='SAME') + self.conv_b\n",
    "        # 렐루 활성화 함수를 적용합니다.\n",
    "        r_out = tf.nn.relu(c_out)\n",
    "        # 2x2 최대 풀링을 적용합니다.\n",
    "        p_out = tf.nn.max_pool2d(r_out, ksize=2, strides=2, padding='VALID')\n",
    "        # 첫 번째 배치 차원을 제외하고 출력을 일렬로 펼칩니다.\n",
    "        f_out = tf.reshape(p_out, [x.shape[0], -1])\n",
    "        z1 = tf.matmul(f_out, self.w1) + self.b1     # 첫 번째 층의 선형 식을 계산합니다\n",
    "        a1 = tf.nn.relu(z1)                          # 활성화 함수를 적용합니다\n",
    "        z2 = tf.matmul(a1, self.w2) + self.b2        # 두 번째 층의 선형 식을 계산합니다.\n",
    "        return z2\n",
    "    \n",
    "    def init_weights(self, input_shape, n_classes):\n",
    "        g = tf.initializers.glorot_uniform()\n",
    "        self.conv_w = tf.Variable(g((3, 3, 1, self.n_kernels)))\n",
    "        self.conv_b = tf.Variable(np.zeros(self.n_kernels), dtype=float)\n",
    "        n_features = 14 * 14 * self.n_kernels\n",
    "        self.w1 = tf.Variable(g((n_features, self.units)))          # (특성 개수, 은닉층의 크기)\n",
    "        self.b1 = tf.Variable(np.zeros(self.units), dtype=float)    # 은닉층의 크기\n",
    "        self.w2 = tf.Variable(g((self.units, n_classes)))           # (은닉층의 크기, 클래스 개수)\n",
    "        self.b2 = tf.Variable(np.zeros(n_classes), dtype=float)     # 클래스 개수\n",
    "        \n",
    "    def fit(self, x, y, epochs=100, x_val=None, y_val=None):\n",
    "        self.init_weights(x.shape, y.shape[1])    # 은닉층과 출력층의 가중치를 초기화합니다.\n",
    "        self.optimizer = tf.optimizers.SGD(learning_rate=self.lr)\n",
    "        # epochs만큼 반복합니다.\n",
    "        for i in range(epochs):\n",
    "            print('에포크', i, end=' ')\n",
    "            # 제너레이터 함수에서 반환한 미니배치를 순환합니다.\n",
    "            batch_losses = []\n",
    "            for x_batch, y_batch in self.gen_batch(x, y):\n",
    "                print('.', end='')\n",
    "                self.training(x_batch, y_batch)\n",
    "                # 배치 손실을 기록합니다.\n",
    "                batch_losses.append(self.get_loss(x_batch, y_batch))\n",
    "            print()\n",
    "            # 배치 손실 평균내어 훈련 손실 값으로 저장합니다.\n",
    "            self.losses.append(np.mean(batch_losses))\n",
    "            # 검증 세트에 대한 손실을 계산합니다.\n",
    "            self.val_losses.append(self.get_loss(x_val, y_val))\n",
    "\n",
    "    # 미니배치 제너레이터 함수\n",
    "    def gen_batch(self, x, y):\n",
    "        bins = len(x) // self.batch_size                   # 미니배치 횟수\n",
    "        indexes = np.random.permutation(np.arange(len(x))) # 인덱스를 섞습니다.\n",
    "        x = x[indexes]\n",
    "        y = y[indexes]\n",
    "        for i in range(bins):\n",
    "            start = self.batch_size * i\n",
    "            end = self.batch_size * (i + 1)\n",
    "            yield x[start:end], y[start:end]   # batch_size만큼 슬라이싱하여 반환합니다.\n",
    "            \n",
    "    def training(self, x, y):\n",
    "        m = len(x)                    # 샘플 개수를 저장합니다.\n",
    "        with tf.GradientTape() as tape:\n",
    "            z = self.forpass(x)       # 정방향 계산을 수행합니다.\n",
    "            # 손실을 계산합니다.\n",
    "            loss = tf.nn.softmax_cross_entropy_with_logits(y, z)\n",
    "            loss = tf.reduce_mean(loss)\n",
    "\n",
    "        weights_list = [self.conv_w, self.conv_b,\n",
    "                        self.w1, self.b1, self.w2, self.b2]\n",
    "        # 가중치에 대한 그래디언트를 계산합니다.\n",
    "        grads = tape.gradient(loss, weights_list)\n",
    "        # 가중치를 업데이트합니다.\n",
    "        self.optimizer.apply_gradients(zip(grads, weights_list))\n",
    "   \n",
    "    def predict(self, x):\n",
    "        z = self.forpass(x)                 # 정방향 계산을 수행합니다.\n",
    "        return np.argmax(z.numpy(), axis=1) # 가장 큰 값의 인덱스를 반환합니다.\n",
    "    \n",
    "    def score(self, x, y):\n",
    "        # 예측과 타깃 열 벡터를 비교하여 True의 비율을 반환합니다.\n",
    "        return np.mean(self.predict(x) == np.argmax(y, axis=1))\n",
    "\n",
    "    def get_loss(self, x, y):\n",
    "        z = self.forpass(x)                 # 정방향 계산을 수행합니다.\n",
    "        # 손실을 계산하여 저장합니다.\n",
    "        loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, z))\n",
    "        return loss.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RTMnsk0u6OWk",
    "outputId": "6bc26ef4-bcf7-4d3a-ef51-54807c668693"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([ 5. 14. 29.], shape=(3,), dtype=float64)\n"
     ]
    }
   ],
   "source": [
    "x = tf.Variable(np.array([1.0, 2.0, 3.0]))\n",
    "with tf.GradientTape() as tape:\n",
    "    y = x ** 3 + 2 * x + 5\n",
    "\n",
    "# 그래디언트를 계산합니\n",
    "print(tape.gradient(y, x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Q0f9GaG06OWl",
    "outputId": "936c6ed6-b4a5-448f-c298-0b913472771b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([9.99540153e-18 2.71703183e-17 7.38565826e-17], shape=(3,), dtype=float64)\n"
     ]
    }
   ],
   "source": [
    "x = tf.Variable(np.array([1.0, 2.0, 3.0]))\n",
    "with tf.GradientTape() as tape:\n",
    "    y = tf.nn.softmax(x)\n",
    "\n",
    "# 그래디언트를 계산합니다.\n",
    "print(tape.gradient(y, x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lnSghQlb6OWl",
    "outputId": "e7434ecd-2f0b-4295-cf3f-11f52ccee6a8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
      "32768/29515 [=================================] - 0s 0us/step\n",
      "40960/29515 [=========================================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
      "26427392/26421880 [==============================] - 0s 0us/step\n",
      "26435584/26421880 [==============================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
      "16384/5148 [===============================================================================================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
      "4423680/4422102 [==============================] - 0s 0us/step\n",
      "4431872/4422102 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "(x_train_all, y_train_all), (x_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "7qJX9yoi6OWl"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "x_train, x_val, y_train, y_val = train_test_split(x_train_all, y_train_all, stratify=y_train_all, \n",
    "                                                  test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "nCjvtqGL6OWl"
   },
   "outputs": [],
   "source": [
    "y_train_encoded = tf.keras.utils.to_categorical(y_train)\n",
    "y_val_encoded = tf.keras.utils.to_categorical(y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "czr-vszP6OWm"
   },
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(-1, 28, 28, 1)\n",
    "x_val = x_val.reshape(-1, 28, 28, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7sQbM5h_6OWm",
    "outputId": "f405daba-2d4f-4b05-8b30-0c266eb5e12b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(48000, 28, 28, 1)"
      ]
     },
     "execution_count": 32,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "9c46kML66OWm"
   },
   "outputs": [],
   "source": [
    "x_train = x_train / 255\n",
    "x_val = x_val / 255"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "izYt1eF-6OWm",
    "outputId": "e8ec787b-60f1-4a33-8b8b-4587c2a3552e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "에포크 0 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 1 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 2 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 3 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 4 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 5 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 6 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 7 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 8 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 9 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 10 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 11 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 12 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 13 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 14 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 15 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 16 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 17 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 18 .......................................................................................................................................................................................................................................................................................................................................................................................\n",
      "에포크 19 .......................................................................................................................................................................................................................................................................................................................................................................................\n"
     ]
    }
   ],
   "source": [
    "cn = ConvolutionNetwork(n_kernels=10, units=100, batch_size=128, learning_rate=0.01)\n",
    "cn.fit(x_train, y_train_encoded, \n",
    "       x_val=x_val, y_val=y_val_encoded, epochs=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "5Os623qd6OWm"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "x1LuOutX6OWm",
    "outputId": "13da7c98-2952-4515-a6a5-9b81e3cb7e89",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxdVb338c8vOSfzPHRIOtMWSqkUKJOlikwXKoKCzKBFsY8IAr6Ea50B9ZGrXp+rjwUeUOSKiECxUi9ggTLK3GLpPNshHTMPTTOv54+9k5ykSZq2OTlJ9vf9ep3X2WcPZ6+cnOxv9lp7r2XOOUREJLjiYl0AERGJLQWBiEjAKQhERAJOQSAiEnAKAhGRgAvFugCHKy8vz40bNy7WxRARGVSWLVtW4pzL72rZoAuCcePGsXTp0lgXQ0RkUDGzbd0tU9WQiEjAKQhERAJOQSAiEnCDro1ARIaexsZGioqKqKuri3VRBr2kpCRGjRpFOBzu9TYKAhGJuaKiItLT0xk3bhxmFuviDFrOOUpLSykqKmL8+PG93k5VQyISc3V1deTm5ioEjpKZkZube9hnVlELAjN7xMz2mdmqbpZfZ2YrzGylmb1tZidGqywiMvApBPrGkXyO0TwjeBS4sIfl/wI+6ZybBvwIeCiKZWHdnip+9vd1VNY2RnM3IiKDTtSCwDn3BlDWw/K3nXPl/st3gVHRKgvA9tJa7n9tM9vLaqO5GxGRQWegtBF8GXihu4VmNtfMlprZ0uLi4iPaQUFWMgA7Kw4c0fYiMnRVVFRw//33H/Z2s2fPpqKi4rC3mzNnDgsWLDjs7aIl5kFgZp/CC4JvdbeOc+4h59wM59yM/Pwuu8o4pEIFgYh0o7sgaGpq6nG7559/nqysrGgVq9/E9PJRM/sY8FvgIudcaTT3lZUSJjkczy4FgciAds/fVrNmV1WfvufxBRn88DNTu10+b948Nm/ezPTp0wmHwyQlJZGdnc26devYsGEDn/3sZ9mxYwd1dXXcfvvtzJ07F2jv+6ympoaLLrqIs846i7fffpvCwkKeffZZkpOTD1m2JUuWcOedd9LU1MSpp57KAw88QGJiIvPmzWPRokWEQiEuuOACfvGLX/D0009zzz33EB8fT2ZmJm+88UaffD4xCwIzGwP8BbjBObehH/ZHQVaSgkBEDnLfffexatUqli9fzmuvvcanP/1pVq1a1XYt/iOPPEJOTg4HDhzg1FNP5fLLLyc3N7fDe2zcuJEnnniChx9+mCuvvJJnnnmG66+/vsf91tXVMWfOHJYsWcLkyZP5whe+wAMPPMANN9zAwoULWbduHWbWVv107733snjxYgoLC4+oSqo7UQsCM3sCOBvIM7Mi4IdAGMA59yDwAyAXuN+/3KnJOTcjWuUBr51AQSAysPX0n3t/Oe200zrckPXrX/+ahQsXArBjxw42btx4UBCMHz+e6dOnA3DKKaewdevWQ+5n/fr1jB8/nsmTJwPwxS9+kfnz53PrrbeSlJTEl7/8ZS6++GIuvvhiAGbOnMmcOXO48sorueyyy/riRwWiGATOuWsOsfwm4KZo7b8ro7KTeWl3dX/uUkQGodTU1Lbp1157jZdffpl33nmHlJQUzj777C5v2EpMTGybjo+P58CBI/+nMxQK8f7777NkyRIWLFjAb37zG1555RUefPBB3nvvPZ577jlOOeUUli1bdlAgHdH+jvodBpGCzGRKauqpa2wmKRwf6+KIyACRnp5OdXXX/yRWVlaSnZ1NSkoK69at49133+2z/R577LFs3bqVTZs2MXHiRB577DE++clPUlNTQ21tLbNnz2bmzJlMmDABgM2bN3P66adz+umn88ILL7Bjxw4FweFqvYR0T2Ud4/JSD7G2iARFbm4uM2fO5IQTTiA5OZnhw4e3Lbvwwgt58MEHmTJlCsceeyxnnHFGn+03KSmJ3//+91xxxRVtjcVf/epXKSsr49JLL6Wurg7nHL/85S8BuOuuu9i4cSPOOc4991xOPLFvOmQw51yfvFF/mTFjhjvSEcre2VzKNQ+/y59uOp2PT8zr45KJyJFau3YtU6ZMiXUxhoyuPk8zW9ZdO2zM7yPoT7qXQETkYIGqGhqemYiZgkBE+sctt9zCW2+91WHe7bffzo033hijEnUtUEGQGIonPy1Rl5CKSL+YP39+rIvQK4GqGoLWewk0CpKISKvABUGhbioTEekgcEFQkJXEzooDDLarpUREoiVwQVCYlUx9Uwtl+xtiXRQRkQEhcEHQelOZ2glE5GikpaV1u2zr1q2ccMIJ/ViaoxPYINAlpCIinkBdPgrtN5WpwVhkgHphHuxZ2bfvOWIaXHRfj6vMmzeP0aNHc8sttwBw9913EwqFePXVVykvL6exsZEf//jHXHrppYe167q6Om6++WaWLl1KKBTil7/8JZ/61KdYvXo1N954Iw0NDbS0tPDMM89QUFDAlVdeSVFREc3NzXz/+9/nqquuOuIfu7cCFwStA9TojEBEIl111VXccccdbUHw1FNPsXjxYm677TYyMjIoKSnhjDPO4JJLLsHvOr9X5s+fj5mxcuVK1q1bxwUXXMCGDRt48MEHuf3227nuuutoaGigubmZ559/noKCAp577jnA6/CuPwQuCDRAjcgAd4j/3KPlpJNOYt++fezatYvi4mKys7MZMWIE3/jGN3jjjTeIi4tj586d7N27lxEjRvT6ff/xj3/w9a9/HYDjjjuOsWPHsmHDBs4880x+8pOfUFRUxGWXXcakSZOYNm0a3/zmN/nWt77FxRdfzKxZs6L143YQuDYC0AA1ItK1K664ggULFvDkk09y1VVX8fjjj1NcXMyyZctYvnw5w4cP73IsgiNx7bXXsmjRIpKTk5k9ezavvPIKkydP5sMPP2TatGl873vf49577+2TfR1K4M4IwGsnWKsBakSkk6uuuoqvfOUrlJSU8Prrr/PUU08xbNgwwuEwr776Ktu2bTvs95w1axaPP/4455xzDhs2bGD79u0ce+yxbNmyhQkTJnDbbbexfft2VqxYwXHHHUdOTg7XX389WVlZ/Pa3v43CT3mwQAZBQZYGqBGRg02dOpXq6moKCwsZOXIk1113HZ/5zGeYNm0aM2bM4Ljjjjvs9/za177GzTffzLRp0wiFQjz66KMkJiby1FNP8dhjjxEOhxkxYgTf+c53+OCDD7jrrruIi4sjHA7zwAMPROGnPFigxiNotWBZEXc+/RGv3Xm2BqgRGQA0HkHf0ngEvaBLSEVE2gWyakgD1IhIX1i5ciU33HBDh3mJiYm89957MSrRkQlkELQOUKNuJkQGDufcYV2fPxBMmzaN5cuXx7oYHRxJdX8gq4Y0QI3IwJKUlERpaal6BT5KzjlKS0tJSko6rO0CeUYA3pVDqhoSGRhGjRpFUVERxcXFsS7KoJeUlMSoUaMOa5vABoF3L0FVrIshIkA4HGb8+PGxLkZgBbJqCDRAjYhIqwAHgQaoERGBgAcB6MohEZHABoHuJRAR8QQ+CHQJqYgEXWCDoHWAGgWBiARdYIOgbYCaSgWBiARbYIMA/JvKyhUEIhJsgQ6CwqxkduqqIREJuEAHQeQANSIiQRX4IADYU6mzAhEJrqgFgZk9Ymb7zGxVN8vNzH5tZpvMbIWZnRytsnSnIMvroU9XDolIkEXzjOBR4MIell8ETPIfc4H+GZwzgm4qExGJYhA4594AynpY5VLgD87zLpBlZiOjVZ6ujMhM0gA1IhJ4sWwjKAR2RLwu8ucdxMzmmtlSM1val/2Va4AaEZFB0ljsnHvIOTfDOTcjPz+/T9+7ICtZN5WJSKDFMgh2AqMjXo/y5/WrQt1UJiIBF8sgWAR8wb966Ayg0jm3u78LoQFqRCToojZUpZk9AZwN5JlZEfBDIAzgnHsQeB6YDWwCaoEbo1WWnkQOUJOblhiLIoiIxFTUgsA5d80hljvglmjtv7ciB6hREIhIEA2KxuJo0r0EIhJ0gQ+CAg1QIyIBF/ggyE4JkxSOUxCISGAFPgjMjELdSyAiARb4IAB/gBp1MyEiAaUgQDeViUiwKQjQADUiEmwKAjRAjYgEm4IADVAjIsGmIEA3lYlIsCkI8AaoAQ1QIyLBpCDAH6AmXQPUiEgwKQh8GqBGRIJKQeAblZWsNgIRCSQFga8gK4ldGqBGRAJIQeAryEqmrtEboEZEJEgUBL7IAWpERIJEQeDTvQQiElQKAp8GqBGRoFIQ+DRAjYgElYLAZ2a6l0BEAklBEKFQA9SISAApCCIUZCarakhEAkdBEKEwO5ni6nrqmzRAjYgEh4IgQuuVQ7tVPSQiAaIgiKABakQkiBQEEXRTmYgEkYIgggaoEZEgUhBE0AA1IhJECoJOdFOZiASNgqCTwqwktRGISKAoCDppvalMA9SISFAoCDppHaCmvLYx1kUREekXCoJOCrP9S0jLVT0kIsGgIOhE9xKISNBENQjM7EIzW29mm8xsXhfLx5jZq2b2TzNbYWazo1me3tAANSISNFELAjOLB+YDFwHHA9eY2fGdVvse8JRz7iTgauD+aJWntzRAjYgETTTPCE4DNjnntjjnGoA/A5d2WscBGf50JrAriuXpFQ1QIyJBE80gKAR2RLwu8udFuhu43syKgOeBr3f1RmY218yWmtnS4uLiaJS1Aw1QIyJBEuvG4muAR51zo4DZwGNmdlCZnHMPOedmOOdm5OfnR71QGqBGRIIkmkGwExgd8XqUPy/Sl4GnAJxz7wBJQF4Uy9QrBVkaoEZEgqNXQWBmt5tZhnl+Z2YfmtkFh9jsA2CSmY03swS8xuBFndbZDpzr72MKXhBEv+7nEFrHJdhTqeohERn6entG8CXnXBVwAZAN3ADc19MGzrkm4FZgMbAW7+qg1WZ2r5ld4q/2TeArZvYR8AQwxw2Avh10L4GIBEmol+uZ/zwbeMw/oFtPGwA4557HawSOnPeDiOk1wMxelqHf6O5iEQmS3p4RLDOzF/GCYLGZpQMt0StWbGmAGhEJkt6eEXwZmA5scc7VmlkOcGP0ihVbGqBGRIKkt2cEZwLrnXMVZnY93h3BldErVuzppjIRCYreBsEDQK2ZnYjXwLsZ+EPUSjUAaIAaEQmK3gZBk381z6XAb5xz84H06BUr9jRAjYgERW+DoNrMvo132ehz/t2/4egVK/Y0QI2IBEVvg+AqoB7vfoI9eHcJ/zxqpRoA1B21iARFr4LAP/g/DmSa2cVAnXNuiLcR6KYyEQmG3nYxcSXwPnAFcCXwnpl9PpoFi7XWbiZ0U5mIDHW9vY/gu8Cpzrl9AGaWD7wMLIhWwWItJzVBA9SISCD0to0grjUEfKWHse2gpAFqRCQoentG8HczW4zXMRx4jcfP97D+kKABakQkCHoVBM65u8zscto7iHvIObcwesUaGAoyk1m3Z9+hVxQRGcR6e0aAc+4Z4JkolmXAiRygJjEUH+viiIhERY9BYGbVeAPMH7QIcM65jC6WDRmRA9SMzU2NcWlERKKjxyBwzg3pbiQOJfJeAgWBiAxVQ/rKnw6cg61vHdYm7XcXq8FYRIau4ATBh3+AR2cfVhi0DlCjm8pEZCgLThBMuwLSC+DF70FL7wZXSwrHk5emAWpEZGgLThAkpMA534NdH8Lqv/R6s8Js3VQmIkNbcIIA4MSrYfgJsOQeaKrv1SYaoEZEhrpgBUFcPJx/L1Rsh/cf7tUmGqBGRIa6YAUBwMRz4Zhz4I2fw4HyQ66uAWpEZKgLXhAAnP8jqKuEN35xyFU1QI2IDHXBDIIRJ8D0a+H9h6B8a4+raoAaERnqghkEAJ/6Llg8LPlRj6u1djOhMwIRGaqCGwSZhXDmLbBqAexc1u1qOakJJIbidFOZiAxZwQ0CgJm3Q0oevPh9rwuKLpgZhRqgRkSGsGAHQVIGnD0Ptr0F61/odrUCDVAjIkNYsIMA4JQ5kDsRXv4hNDd1uUphVrLaCERkyFIQxIfhvHugZAN8+N9drhI5QI2IyFCjIAA47tMw5kx47adQX33Q4sgBakREhhoFAYAZXPBj2F8Mb/36oMW6l0BEhjIFQatRM2DqZfDOb6Bqd4dFGqBGRIYyBUGkc38AzY3w6k86zG4doEYNxiIyFCkIIuWMh9PmwvLHYe/qttkaoEZEhrKoBoGZXWhm681sk5nN62adK81sjZmtNrM/RbM8vfKJOyExHV76QYfZGpdARIaqqAWBmcUD84GLgOOBa8zs+E7rTAK+Dcx0zk0F7ohWeXotJQdm3QmbXobNr7bN9m4qUxCIyNATzTOC04BNzrktzrkG4M/ApZ3W+Qow3zlXDuCc2xfF8vTeaXMhcwy89P228Y0LsjRAjYgMTdEMgkJgR8TrIn9epMnAZDN7y8zeNbMLu3ojM5trZkvNbGlxcXGUihshnOQ1HO9ZCSueBLxLSDVAjYgMRbFuLA4Bk4CzgWuAh80sq/NKzrmHnHMznHMz8vPz+6dkJ1wOI6fDKz+GxgOMz08F4In3t/fP/kVE+kk0g2AnMDri9Sh/XqQiYJFzrtE59y9gA14wxF5cnHeTWVURvPsAn5yUz2dOLODni9fz7PLOP4aIyOAVzSD4AJhkZuPNLAG4GljUaZ2/4p0NYGZ5eFVFW6JYpsMzfhZMvgj+8X+IO1DKL674GKeNz+HOpz/inc2lsS6diEifiFoQOOeagFuBxcBa4Cnn3Gozu9fMLvFXWwyUmtka4FXgLufcwDrCnn8PNOyH139GYiieh2+YwdjcVOY+tpQNew/ul0hEZLCxwXYVzIwZM9zSpUv7d6d/uwP++Rjc8j7kHkNReS2fu/9twnHGwltmMjwjqX/LIyJymMxsmXNuRlfLYt1YPDic/W2IT/TGLABGZafw+zmnUnmgkRt//wE19V2PYyAiMhgoCHojfbg3rOXav8GbvwTnOKEwk/uvP4X1e6u5+Y/LaGxuiXUpRUSOiIKgt866w7ukdMk98Je50FjHJyfn89PPTePNjSV85y8rdbOZiAxKoVgXYNAIJcLlv4NhU7x7C8q2wNWPc+WpoymqOMCvl2ykICuZb5w/OdYlFRE5LDojOBxm8Im74Ko/wr418PA5sGs53zhvEp8/ZRS/WrKRpz7Ycej3EREZQBQER2LKZ+BLiwGDRy7E1vyVn142jVmT8vj2wpW8vqEfusEQEekjCoIjNfJjMPdV7/npOYTf/Bn3XzudycPT+dofl7FqZ2WsSygi0isKgqORNgy++Dc48Vp47aek/20uj14/lczkMF969AN1Wy0ig4KC4GiFEuGz98P5P4I1zzJ8wWf545WjONDYzJxH3qdSvZWKyACnIOgLZjDzNrj2SSjdwoSFn+FPF4bYVlrL3MeWUt/UHOsSioh0S0HQlyb/G9z0EoSTmfbSNTxxxlbe+1cZdz69gpYW3WMgIgOTgqCvDZsCN70Co07llGXz+Mvkl/ifj4r4j8XrYl0yEZEuKQiiITUXblgIp8zh5O2/5/lhD/LH11fzh3e2xrpkIiIHURBESygBLv4vuOjnHFf9DovTf8xDi17jOwtXUlxdH+vSiYi0URBEkxmcPhe7fgGF8WW8mPIDQsse4fxfvMz8VzdR16hGZBGJPQVBfzjmHOymV0gpPJ57Q4/wcvgbbH35/3HBL17h2eU71ZAsIjGlIOgveRPhxhfg+r+QN6yQn4cf4k+Nt/PyU/dz2fw3+WBrWaxLKCIBpRHKYsE5WP887pUfY/vWsMnG8LP6y4mfcjHzZk9hbG5qrEsoIkNMTyOUKQhiqaUF1iyk5ZX/TVzZJla6Cfyf5iuYcPqlfP3cyWSmhGNdQhEZIhQEA11zE6x4kuZXf0p81Q4+aDmWB+Ov4azzPsv1Z4wlHK8aPBE5OhqzeKCLD8FJ1xF/24fw6f9keloFv3N3M/Hv13PHLx7mxdV7NPqZiESNgmAgCSXAqTcR/sZHuAt+wukpu5h/4N+J+/PVzLv/cXVtLSJRoaqhgay+huZ3H6DpzV+R2FTN35tPZXvWaYydMoMzzphFZk5+rEsoIoOE2ggGuwMV1L/5a9z7vyWpqf2soCw+n6b848mZMJ3QiBNg+FTIneSdWYiIRFAQDBXO4SqL2LZ2KZtXvUfDrlWMa97KxLhdhPHuUnZxISxvMgw73guG4VO96cxR3p3OIhJICoIhqqm5hX9sKmHRh1vZtOafjGvexmmpe/h42l7GNG4hVLOrfeXETBh+PORNgrzJ3iN3ImSN9RqrRWRIUxAEQE19E39ftYeF/yzi7c2lOAdnjQpx/TG1zErfR2rFOti3Fko2Qm1J+4bxCZBzjB8QrSExyatiSsqI3Q8kIn1KQRAwuysP8OzyXSz8cCfr91YTjjc+dewwPndSIbMm55PWXOUFQulGKNngTZdshLIt4CI6wksbcXBAjDgR0tRILTLYKAgCyjnHmt1VLPxwJ89+tIvi6npCccbJY7P5xKQ8Zk3K54TCTOLj/LaDpgYo33pwQJSsh7qIS1czCmHkdCiY3v6cNiwmP6OI9I6CQGhqbuH9rWW8ubGENzcWs2pnFQBZKWFmTsxrC4aCrOSDN3YO9pdA8TrY/RHsXg67lkPpJsD//qQXdAyGkdMhfXj//YAi0iMFgRykpKaetzaV8MYGLxj2+YPlHJOfyqxJ+Xxycj6nT8ghJaGHhuT6ati9oj0Ydi/3ziDawmFkx2AYeSKkj9DVSyIxoCCQHjnn2LC3hjc3FvPGxhLe21JKfVML4XhjxtgcZk3O4xOT8jl+ZAZxcYc4iNdXw56V7cGwa7lXzdQaDkmZ7VctRV7BlD0O4tXJnki0KAjksNQ1NrN0a3lbMKzd7VUjZaeEmTYqi6kFGUwtyOD4kRmMy03tRTjUeOGwZ4UXCsXrvTOHmj3t68SFIGdCREi0PiZ64SEiR0VBIEdlX3Udb20q4a1NpazeVcXGvdU0+aOqpSbEM2Vkhh8OmRxfkMGk4WkkhuIP/cZ1lVCyyW+Y3tDeQF22GVqa2tdrvXop/1gYdxZMOBuSs6Pys4oMVQoC6VP1Tc1s3FvDml1VrN5VyepdVazdXcX+Bu/S03C8MXFYettZw9SCDI4vyCA9qZdVP82NUL6tYziUbPAaq+urwOJh1Kkw8TyYdJ53SWtclPpPbKr32kF2vAd7V8OIE2Di+V4wqa1DBhEFgURdS4tjW1ltWzCs3lXFml2VlNQ0tK0zNjelLRhazx6GpSdivT2gNjfBzmWw6SXY9DLs+qc3PyUPJp7rHaCPOQdSc4/8B6nZBzve9w78O9739tFc7+8nF2pLvemsMd7+Jp0P4z8BCRpVTga2mAWBmV0I/AqIB37rnLuvm/UuBxYApzrnejzKKwgGl31VdV4o7G4/e9hWWtu2PC8twa9a8oJhakEG43vT7gBQUwybX/FCYfMS/yBtUHiyd7Yw8TwoPAXiuqmmamn27rZuPejveA/K/+Uti0/wrnQafRqMPt17Th/hnalsetl7bHkdGvd76479OEy6QGcLMmDFJAjMLB7YAJwPFAEfANc459Z0Wi8deA5IAG5VEAx91XWNrN1dzepdlX71UhUb91XT2Ox9F1MS4jluRHqHcJg8PJ2kcA/tDi3N3lVKG/2D9M6l4FogKcs7S5h4nnewLv+Xd9Df/i4ULYWGam/71Hz/gO8/Rp4I4aSef5Cmetj2tre/jS95N96BzhZkQIpVEJwJ3O2c+zf/9bcBnHM/7bTefwEvAXcBdyoIgqmhqYWN+6r9KiX/sbuKmnqv0Tg+zjgmP5WxuamMyk5mVHaK/+xNZyZ3an+oLYMtr8KmJd6BumZv+zKLg2FTO/63nz3u6P+L7+lsYeL5XhjlTer+DEUkimIVBJ8HLnTO3eS/vgE43Tl3a8Q6JwPfdc5dbmav0U0QmNlcYC7AmDFjTtm2bVtUyiwDS0uLY0d5bVs4rNtTxY6yA+wor6W2obnDuulJoQ7hUJjVHhajs5LIqFqHFX3gdbBXeEr0O9Tr7mzB4iGzEDLHQNZo7+whc3T7dMao2I8n0dLsNcrXVUJdVft0Q61X7ZZ7TGzLJ0dkQAaBmcUBrwBznHNbewqCSDojEOccFbWNFJUfoKi8ttOzN72/c1AkhijMTmZMTgpjc1MYk5PCmNxUxuSkUJiVTEIoyqO2lm+Df73hVU1V7IDKHd5z9S6vCquNeW0RmX4wZI32p8d6ARIX8q6qammClkbvoN3SFDGv2Z/fOq+5fd3mRmioaT/A11X6B/lO063VZd0ZNhWOvwSmXALDpqg9ZJAYkFVDZpYJbAZq/E1GAGXAJT2FgYJADqVzUOys8AJiR1kt2/1HfVP7wTfOoCCrPSRG56QwNifVD4suqp36UnMjVO30QqFie3tAVG73X+/0DuJ9KS7snRElZUJihjedmOG1p7RNZ0ZM+6/jE7wqr7WLvDYWnDemxZRLvGAYOV2hMIDFKghCeI3F5wI78RqLr3XOre5m/dfQGYH0g5YWR3FNPdtK/WAo3c/2slq2ldWyo6y2wyWv4HXMNybHO4vo3DYxKju550bsoy5ss9e+UbEDqoq8DgDjQu2P+NbpcKfXrfPiva47Wl8npkEo6egP2NV7YN3/wJpFsPUfXvflmWNgyme8UBh1WvTu7ZAjEsvLR2cD/4V3+egjzrmfmNm9wFLn3KJO676GgkAGgJr6JnaU1bKt1AuGbWX726Z3Vhxou7qpVV5aAoWRARHRPlGYndxzx31Dwf5SWP+8d6aw+VXvDCZtBEy52DtbGDtzcI+C13jAG6ujfJvX3fqw4yEhJdalOmy6oUykj7S0OPZV17OzomObROv0zvIDNDS3dNgmNzWBQj8kRmQkMzwjkRGZSQzPSGJERhIjMpOie1bRn+oqYcOLsPZZ71LepgPejXjHzvbOFlLyvBv0muqhucF7bqo/eF7bc703TkZTndfWkZTltaG0PUZ6z4npR1fupnp/LI7NXhcnbc9bvDOxSBbnjeA3YlrE42MDfsAmBYFIP2mtduqqAXtn+QH2VNUddMUTQGZymOEZiR3CYXhGe1gMz0wkLzWxdzfaDRQN+72rptYsgg2LD90I3ZnFQXwihPxHXBgOlHuX5XYWTu0YDJ2DIm2Ed6DeX9LFwX6z122OkQkAAAvLSURBVDYT2WifnO1dYZZ7TPtz9jivSmzPyvZOFCt3tG+TPvLgcMgeP2CqyBQEIgOEc47q+ib2Vtaxt6qePVV17K2qY0+l97y3qo49VXUUV9fT0ulPMxRn5KQmkJeWSF56Inlp/nTbc2Lb65zUBELxA+MABEBjHex41/vPOz7BO7DHJ3qXykYe7COXdVWd5JzX1XnNXqje7R2Y2x7+65o9ULXbOxvpSWKG1+Nt7jFeo3fbgX8CpOT07ueqLYO9q7xg2L3Cey5e1z7ka0IaDD/BD4YTvEZ359pDx7X4j9Z5rofXzrvRcczpvf3UO1AQiAwyTc0tlO5vYE9lXYewKKmpp7SmgZKaekpqGiiuqaehqeWg7c0gOyWhLSRy/YDwzjISGZ6exLCMRIZlJJGeGOp9f0+DhXPe5bCRQVGz16umav0vPzUvOlc5NdZ5YbBnRcTZw0rv0t2jNfMOOP+eI9pUQSAyRLWeYZRU11O6v4GS6npKauopbg2L1vk19RRX13dZLZUcjme4HwrDM5IYnu5VUQ3zq6qGZyQxLD2R1MRB3OAbay0tULHNa3i2OP9h3jN0em0dX0fOCycdcZclPQWBfrMig5iZkZEUJiMpzIRetFXW1Dexr8qrltpX3VodVc/eqjr2VdWzoqiCvVV11DUefJaRlhgiJzWB7JQwWSle9VNWSpjslO7mJZCcMEQawY9WXBzkjI91KbqlIBAJkLTEEGn5aUzIT+t2ndazjH0RIdH6XF7bQHltI2X7G9hcXENFbWNbf1BdSQrHkZ2S4IdEmNxUvx0j3auyyk9LJD890a++SiA8kNo1AkRBICIdRJ5lTBx26MsyG5paqPADory2oW26bH/7dEVtA2X7G/iovIKS6vqDugBplZUSJr+10TuiQTw/vT00hqUnDrzG8EFOQSAiRyUhFMewjCSGZRyi2+4ItQ1NlFR7jd2t7Rcl/nTr/BVF3YdGnEFOanswdHz22jfy0xIZlpE49G/o6wP6hESk36UkhBiTG2JM7qHv0I0MjeLqeu+5qo591d7rfdX1rN9TTUlNfdtY2pFSE+IZlpFEfloimSlh0hNDpCeFSE8Kk54UIi1iOiMpRFpi2F8eIjUhNLju3ThCCgIRGdB6GxotLY7y2oYOAeE9e/dlFFfXs6Osluq6JqrrvLaNLnKjAzNIS2gPjszkcFtjeFZqmKzk9obyzs9R79G2DykIRGRIiIszcv17JqaMPPT6zjlqG5qprmuipr6Rqrqm9pCImK6qa6Km3puuqG1kW2kty3dUUFHbeFB3IpFSEuL9hnIvODJTwmT4ZxkpiSHSEuNJTQyRltg6L96b9uelJMT32xmJgkBEAsnMSPUPvND79o1WzjkONDZ7jeT7G6iobaTigN84vt9/PuDNL69tYGfFAarrmthf38SBxq4by7uSktAeGNedPoabZk047LIeioJAROQImBkpCSFSEkIUZiUf1rbNLY7ahib21zdTU99EbYN31rG/vpn99U3sb/ACo8Z/7S1vJj89MSo/i4JARKSfxceZ30AdxUGPDsPgac0QEZGoUBCIiAScgkBEJOAUBCIiAacgEBEJOAWBiEjAKQhERAJOQSAiEnCDbqhKMysGth3h5nlASR8Wp68N9PLBwC+jynd0VL6jM5DLN9Y51+U4doMuCI6GmS3tbszOgWCglw8GfhlVvqOj8h2dgV6+7qhqSEQk4BQEIiIBF7QgeCjWBTiEgV4+GPhlVPmOjsp3dAZ6+boUqDYCERE5WNDOCEREpBMFgYhIwA3JIDCzC81svZltMrN5XSxPNLMn/eXvmdm4fizbaDN71czWmNlqM7u9i3XONrNKM1vuP37QX+Xz97/VzFb6+17axXIzs1/7n98KMzu5H8t2bMTnstzMqszsjk7r9PvnZ2aPmNk+M1sVMS/HzF4ys43+c3Y3237RX2ejmX2xH8v3czNb5/8OF5pZVjfb9vh9iGL57jaznRG/x9ndbNvj33sUy/dkRNm2mtnybraN+ud31JxzQ+oBxAObgQlAAvARcHyndb4GPOhPXw082Y/lGwmc7E+nAxu6KN/ZwP/E8DPcCuT1sHw28AJgwBnAezH8Xe/Bu1Empp8f8AngZGBVxLyfAfP86XnAf3SxXQ6wxX/O9qez+6l8FwAhf/o/uipfb74PUSzf3cCdvfgO9Pj3Hq3ydVr+n8APYvX5He1jKJ4RnAZscs5tcc41AH8GLu20zqXAf/vTC4Bzzcz6o3DOud3OuQ/96WpgLVDYH/vuQ5cCf3Ced4EsMxsZg3KcC2x2zh3pneZ9xjn3BlDWaXbk9+y/gc92sem/AS8558qcc+XAS8CF/VE+59yLzrkm/+W7wKi+3m9vdfP59UZv/t6PWk/l848dVwJP9PV++8tQDIJCYEfE6yIOPtC2reP/IVQCuf1Sugh+ldRJwHtdLD7TzD4ysxfMbGq/Fgwc8KKZLTOzuV0s781n3B+upvs/vlh+fq2GO+d2+9N7gOFdrDNQPssv4Z3ldeVQ34doutWvunqkm6q1gfD5zQL2Ouc2drM8lp9frwzFIBgUzCwNeAa4wzlX1Wnxh3jVHScC/xf4az8X7yzn3MnARcAtZvaJft7/IZlZAnAJ8HQXi2P9+R3EeXUEA/JabTP7LtAEPN7NKrH6PjwAHANMB3bjVb8MRNfQ89nAgP97GopBsBMYHfF6lD+vy3XMLARkAqX9Ujpvn2G8EHjcOfeXzsudc1XOuRp/+nkgbGZ5/VU+59xO/3kfsBDv9DtSbz7jaLsI+NA5t7fzglh/fhH2tlaZ+c/7ulgnpp+lmc0BLgau88PqIL34PkSFc26vc67ZOdcCPNzNfmP9+YWAy4Anu1snVp/f4RiKQfABMMnMxvv/NV4NLOq0ziKg9eqMzwOvdPdH0Nf8+sTfAWudc7/sZp0RrW0WZnYa3u+pX4LKzFLNLL11Gq9BcVWn1RYBX/CvHjoDqIyoAukv3f4XFsvPr5PI79kXgWe7WGcxcIGZZftVHxf486LOzC4E/h24xDlX2806vfk+RKt8ke1On+tmv735e4+m84B1zrmirhbG8vM7LLFurY7GA++qlg14VxN81593L94XHiAJr0phE/A+MKEfy3YWXhXBCmC5/5gNfBX4qr/OrcBqvCsg3gU+3o/lm+Dv9yO/DK2fX2T5DJjvf74rgRn9/PtNxTuwZ0bMi+nnhxdKu4FGvHrqL+O1Oy0BNgIvAzn+ujOA30Zs+yX/u7gJuLEfy7cJr3699XvYeiVdAfB8T9+HfirfY/73awXewX1k5/L5rw/6e++P8vnzH2393kWs2++f39E+1MWEiEjADcWqIREROQwKAhGRgFMQiIgEnIJARCTgFAQiIgGnIJDAMrO3/edxZnZtH7/3d7ral8hApMtHJfDM7Gy8Xi4vPoxtQq69w7aultc459L6onwi0aYzAgksM6vxJ+8DZvn9xX/DzOL9vvo/8Ds8+1/++meb2ZtmtghY48/7q9+Z2OrWDsXM7D4g2X+/xyP35d+N/XMzW+X3UX9VxHu/ZmYLzBsj4PH+6hFXJBTrAogMAPOIOCPwD+iVzrlTzSwReMvMXvTXPRk4wTn3L//1l5xzZWaWDHxgZs845+aZ2a3Oueld7OsyvE7UTgTy/G3e8JedBEwFdgFvATOBf/T9jyvSkc4IRA52AV5fSsvxugjPBSb5y96PCAGA28ystSuL0RHrdecs4Anndaa2F3gdODXivYuc18nacmBcn/w0IoegMwKRgxnwdedch87f/LaE/Z1enwec6ZyrNbPX8PqxOlL1EdPN6O9T+onOCESgGm/Y0FaLgZv97sIxs8l+z5GdZQLlfggchzdsZ6vG1u07eRO4ym+HyMcbAvH9PvkpRI6Q/uMQ8Xq3bPareB4FfoVXLfOh32BbTNfDTP4d+KqZrQXW41UPtXoIWGFmHzrnrouYvxA4E683Sgf8u3Nujx8kIjGhy0dFRAJOVUMiIgGnIBARCTgFgYhIwCkIREQCTkEgIhJwCgIRkYBTEIiIBNz/B0wk77ez5GchAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(cn.losses)\n",
    "plt.plot(cn.val_losses)\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('iteration')\n",
    "plt.legend(['train_loss', 'val_loss'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "41In0JlI6OWm",
    "outputId": "d1ae3de1-4264-4f1b-c81a-a6be3c90d3ad",
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8745833333333334"
      ]
     },
     "execution_count": 37,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cn.score(x_val, y_val_encoded)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ayt5wxyd6OWm"
   },
   "source": [
    "## 08-5 케라스로 합성곱 신경망 모델을 만듭니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "id": "H3YNxuTm6OWm"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "bGvfR1c86OWn"
   },
   "outputs": [],
   "source": [
    "conv1 = tf.keras.Sequential()\n",
    "conv1.add(Conv2D(10, (3, 3), activation='relu', padding='same', input_shape=(28, 28, 1)))\n",
    "conv1.add(MaxPooling2D((2, 2)))\n",
    "conv1.add(Flatten())\n",
    "conv1.add(Dense(100, activation='relu'))\n",
    "conv1.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xngFCMyp6OWn",
    "outputId": "6f022fb0-4420-48aa-ab0d-6c2d2d50cd4a",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d (Conv2D)              (None, 28, 28, 10)        100       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 14, 14, 10)        0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 1960)              0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 100)               196100    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 10)                1010      \n",
      "=================================================================\n",
      "Total params: 197,210\n",
      "Trainable params: 197,210\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "conv1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "2Mp-BDDL6OWn"
   },
   "outputs": [],
   "source": [
    "conv1.compile(optimizer='adam', loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6NKXTSSi6OWn",
    "outputId": "0f4d8004-41e2-4e0f-e538-323ff61ebef9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "1500/1500 [==============================] - 8s 5ms/step - loss: 0.4359 - accuracy: 0.8482 - val_loss: 0.3272 - val_accuracy: 0.8832\n",
      "Epoch 2/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2964 - accuracy: 0.8919 - val_loss: 0.2807 - val_accuracy: 0.8992\n",
      "Epoch 3/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2556 - accuracy: 0.9076 - val_loss: 0.2605 - val_accuracy: 0.9043\n",
      "Epoch 4/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.2269 - accuracy: 0.9162 - val_loss: 0.2555 - val_accuracy: 0.9064\n",
      "Epoch 5/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2061 - accuracy: 0.9238 - val_loss: 0.2682 - val_accuracy: 0.9051\n",
      "Epoch 6/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1851 - accuracy: 0.9315 - val_loss: 0.2357 - val_accuracy: 0.9176\n",
      "Epoch 7/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1678 - accuracy: 0.9381 - val_loss: 0.2510 - val_accuracy: 0.9147\n",
      "Epoch 8/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1524 - accuracy: 0.9443 - val_loss: 0.2571 - val_accuracy: 0.9152\n",
      "Epoch 9/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1383 - accuracy: 0.9481 - val_loss: 0.2554 - val_accuracy: 0.9174\n",
      "Epoch 10/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.1228 - accuracy: 0.9550 - val_loss: 0.2503 - val_accuracy: 0.9204\n",
      "Epoch 11/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.1111 - accuracy: 0.9595 - val_loss: 0.2748 - val_accuracy: 0.9140\n",
      "Epoch 12/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.0995 - accuracy: 0.9640 - val_loss: 0.2701 - val_accuracy: 0.9198\n",
      "Epoch 13/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0871 - accuracy: 0.9688 - val_loss: 0.2857 - val_accuracy: 0.9183\n",
      "Epoch 14/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0770 - accuracy: 0.9729 - val_loss: 0.2975 - val_accuracy: 0.9185\n",
      "Epoch 15/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.0712 - accuracy: 0.9752 - val_loss: 0.3074 - val_accuracy: 0.9188\n",
      "Epoch 16/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0598 - accuracy: 0.9783 - val_loss: 0.3323 - val_accuracy: 0.9190\n",
      "Epoch 17/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0565 - accuracy: 0.9790 - val_loss: 0.3530 - val_accuracy: 0.9154\n",
      "Epoch 18/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0477 - accuracy: 0.9832 - val_loss: 0.3603 - val_accuracy: 0.9172\n",
      "Epoch 19/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.0434 - accuracy: 0.9851 - val_loss: 0.3734 - val_accuracy: 0.9203\n",
      "Epoch 20/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.0401 - accuracy: 0.9858 - val_loss: 0.3936 - val_accuracy: 0.9163\n"
     ]
    }
   ],
   "source": [
    "history = conv1.fit(x_train, y_train_encoded, epochs=20, \n",
    "                    validation_data=(x_val, y_val_encoded))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 281
    },
    "id": "kMQsHGiy6OWn",
    "outputId": "32472338-7390-405b-b842-8b14a7556d86"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEICAYAAABF82P+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUVfrA8e+bQgIkgYQ0SEgBQu8JRSmCBRARsAEWxLKyqNh1191Vd9fV/bnrLquurL2yNkRBVlBURBGREnqH0BNaQksCJKS8vz/ugCGGECCTmSTv53nmmZl7z515c5PMO+ece84RVcUYY4wpzcfTARhjjPFOliCMMcaUyRKEMcaYMlmCMMYYUyZLEMYYY8pkCcIYY0yZ3JogRGSQiGwQkTQRebSccteIiIpIiut5gogcE5HlrtvL7ozTGGPML/m564VFxBeYCFwGpAOLRWS6qq4tVS4YuA9YWOolNqtq54q+X3h4uCYkJJxf0MYYU8ssWbIkS1UjytrntgQBdAfSVHULgIh8CAwD1pYq9xfgb8Aj5/NmCQkJpKamns9LGGNMrSMi20+3z51NTDHAzhLP013bThKRrkBTVZ1RxvGJIrJMRL4XkT5ujNMYY0wZ3FmDKJeI+AATgFvK2L0biFPV/SKSDEwTkXaqml3qNcYCYwHi4uLcHLExxtQu7qxBZABNSzyPdW07IRhoD3wnItuAnsB0EUlR1XxV3Q+gqkuAzUDL0m+gqq+qaoqqpkRElNmEZowx5hy5M0EsBpJEJFFE6gCjgOkndqrqYVUNV9UEVU0AFgBDVTVVRCJcndyISDMgCdjixliNMcaU4rYmJlUtFJHxwCzAF3hTVdeIyJNAqqpOL+fwvsCTIlIAFAPjVPWAu2I1xhjzS1JTpvtOSUlRu4rJGGPOjogsUdWUsvbZSGpjjDFlqvUJ4tDR4zz/zSZWZxz2dCjGGONVPHaZq7fw8RGem70RRWkf08DT4RhjjNeo9TWIkEB/WkUFs2T7QU+HYowxXqXWJwiAlIRQlu04RFFxzeiwN8aYymAJAkiJDyM3v5D1e7LPXNgYY2oJSxBAcnwoAEutmckYY06yBAHEhtYlKiSAVEsQxhhzkiUIQERIiQ8jdZslCGOMOcEShEvX+FAyDh1jz+E8T4dijDFewRKES4qrHyJ1u035ZIwxYAnipLZNQqjr72vNTMYY42IJwsXf14dOTRvYgDljjHGxBFFCSnwYa3dnc/R4oadDMcYYj7MEUUJyQihFxcrynYc8HYoxxnicJYgSusY5HdVLrB/CGGPcmyBEZJCIbBCRNBF5tJxy14iIikhKiW2/cx23QUQGujPOExrU9adlVJANmDPGGNyYIFxrSk8ELgfaAteLSNsyygUD9wELS2xri7OGdTtgEPCfE2tUu1tyfBhLdxyk2CbuM8bUcu6sQXQH0lR1i6oeBz4EhpVR7i/A34CSI9SGAR+qar6qbgXSXK/ndinxoeTkFbJxX05VvJ0xxngtdyaIGGBniefprm0niUhXoKmqzjjbY90lJcHVD2HNTMaYWs5jndQi4gNMAB46j9cYKyKpIpKamZlZKXHFhdUjPCjAOqqNMbWeOxNEBtC0xPNY17YTgoH2wHcisg3oCUx3dVSf6VgAVPVVVU1R1ZSIiIhKCdqZuC/UOqqNMbWeOxPEYiBJRBJFpA5Op/P0EztV9bCqhqtqgqomAAuAoaqa6io3SkQCRCQRSAIWuTHWU6QkhLLjwFH25djEfcYYL5e+BNbPdMtLuy1BqGohMB6YBawDJqvqGhF5UkSGnuHYNcBkYC3wJXC3qha5K9bSusbbeAhjjJfLz4GZv4HXL4Fvn4Li4kp/C79Kf8USVHUmMLPUtidOU7ZfqedPA0+7LbhytG/SgAA/H1K3H+TyDo09EYIxxpzeus9h5iOQsxu63wEXPw4+lf99360Jorqq4+dDp9iG1g9hjPEu2bucxLD+c4hsByPehabd3PZ2liBOIzkhlNfmbiGvoIhA/yoZo2eMMWUrLoLUN+GbP0NxAVzyR7jwHvD1d+vb2lxMp5ESH0phsbLCJu4zxnjSntXwxgCY+TDEpsBdP0GfB92eHMBqEKeVfHKFuYP0aNbIw9EYY2qdgmPw/d9g/r8hsAFc9Sp0HAEiVRaCJYjTaFivDi0ig2xEtTGm6m3+Fj5/AA5ug843woCnoF5YlYdhCaIcyXGhfLlmD8XFio9P1WVtY0wtdSQLZv0eVn4EYc3h5unQ7CKPhWN9EOVITgjl8LECNmfmejoUY0xNpgrL3oMXU2D1p9D3EbhzvkeTA1gNolwpJfohkqKCPRyNMaZG2rfe6YDe9gM07QFXPg+RbTwdFWAJolyJ4fVpVL8OS7Yf5PrucZ4OxxhTUxQed8YypL7pJIaABjDkX9D1FrcMeDtXliDKISJ0jQ+1jmpjTOU4uA2WvAPLJsGRTGgQB5c8AV1uhqDKmXC0MlmCOIOU+FC+XruXrNx8woMCPB2OMaa6KSqETV85tYW0b5zLVFsOgpTboPnF4OO9A3EtQZxByQWEBraL9nA0xphqI3sXLJ0ES9+B7AwIioaLfgNdb4YGsZ6OrkIsQZxBuyYNqOPrYwnCGHNmxcWwZY5TW9jwBWiRU0u4/G9OraEKRj9XJksQZxDo70uH2Aakbjvg6VCMMd7qSBYs+y8secvpZ6jXCC4cD13HQKPmno7unFmCqICU+FDe+nGbTdxnjHEUHINdy2DHAue2ZQ4UHYf4Xs7U222uBL/q32dpCaICkuNDeWXuFlZnHCYloeqHuxtjPCx3n5MIdi507nevcGZVBWiU5HQ4J98Kka09G2clc2uCEJFBwPOAL/C6qj5Tav844G6gCMgFxqrqWhFJwFmFboOr6AJVHefOWMtTcuI+SxDG1HDFxZC5HnYugB0LnfuD25x9vgEQ0xUuuAua9nQGttWvuZN5ui1BiIgvMBG4DEgHFovIdFVdW6LY+6r6sqv8UGACMMi1b7OqdnZXfGejUVAAzcLrk7rtIHh25LsxprIVHIP0xT8ng52LIf+ws69+hJMEuv3KuW/cqUY0HVWUO2sQ3YE0Vd0CICIfAsNw1pkGQFWzS5SvD6gb4zkvyfGhzF6/D1VFqnC6XWOMG637HGY8BLl7nOcRbaD9Va7aQXcIa1al02t7G3cmiBhgZ4nn6UCP0oVE5G7gQaAOcHGJXYkisgzIBh5T1R/cGOsZJceH8vGSdLZkHaF5RJAnQzHGnK+cvfDFI7D2M4hq70xzEX8B1A31dGRexeOd1Ko6EZgoIjcAjwFjgN1AnKruF5FkYJqItCtV40BExgJjAeLi3DtX0skBc9sOWoIwprpSdS5H/eoPTtPSxY9Dr/uq3fiEquLOWaEygKYlnse6tp3Oh8BwAFXNV9X9rsdLgM1Ay9IHqOqrqpqiqikREe6dx6RZeBAN6/mTut3GQxhTLR3YCu8Og+njIbItjPsR+j5syaEc7qxBLAaSRCQRJzGMAm4oWUBEklR1k+vpFcAm1/YI4ICqFolIMyAJ2OLGWM/Ix0dIjrOJ+4ypdoqLYMFL8O1T4OMHV/wTkm/zqllTvZXbEoSqForIeGAWzmWub6rqGhF5EkhV1enAeBG5FCgADuI0LwH0BZ4UkQKgGBinqh7/6p6c4HRUHzxynND6dTwdjjHmTPashun3wK6lzlQXV/yz2syD5A3c2gehqjOBmaW2PVHi8X2nOe4T4BN3xnYuUuKdMRBLth/k0rZRHo7GGHNaBXkw91n48TkIbAjXvAHtr6nVVySdC493UlcnHWMb4O8rpFqCMMZ7bf8J/ncvZG2ETtfDwL9CPRvgei4sQZyFQH9f2jVpwBLrqDbG++Rlw+w/w+LXnYV4bvoEWlzq6aiqNUsQZyklPpR3F2wnv7CIAD+buM8Yr7BxFnz+gLMGQ4874eLHIMAuRz9f1o1/llISQjleWMzqjOwzFzbGuNfxIzB1HLw/AgJC4Pav4fJnLDlUEqtBnKVkV0f10u0HT07iZ4zxgKw0+OgmZ2K9vr+Bvo+An11dWJmsBgGw+Vs4frRCRSOCA4hvVM8GzBnjSWunw6v9IHcvjP4ULv6DJQc3sASRtQkmXQ3f/LHChyTHOwPmVL12bkFjaqaiQvjqcZg8GiJawq/nOkt6GrewBBGeBD3GwaJXnZpEBaTEh5GVe5zt+ytW6zDGVILcfTBpOMx/AVJuh1u/gIZNz3ycOWeWIAAu/SOEt4Rpd8OxM0+lUXIBIWNMFdixEF7pC+mpcNUrMGRCrVqXwVMsQQD413X+6I7sg5mPnLF4UmQQIYF+Nh7CGHdThQUvw9uDwS8QfvUNdBrl6ahqDUsQJ8R0da6EWPUxrP603KI+PkLXeJu4zxi3ys+FT26HL38LSQNg7HcQ3d7TUdUqliBK6vMQxCTDjAchZ0+5RVPiQ9m4N5fDRwuqKDhjapHMjfD6JbBmKlzyBIx8D+o29HRUtY4liJJ8/ZympoI8+Gy8U709jZPjIXZYLcKYSrX2M3itPxzJgtFTnS9uNjW3R9hZLy08CS57EtK+hiVvnbZY56YN8fMRGw9hTGUpKoRZf4DJN0NEa+cS1mb9PB1VrWYJoizdfgXN+jt/rPs3l1mkbh1f2jUJIXWb1SCMOW85e+HdofDTi9DtDucS1gYxno6q1rMEURYfHxg20VmKcOo4Z0WqMnSND2VF+iEKioqrOEBjaoiiQmeivVf6QsZSuPo1uOIfNiraS7g1QYjIIBHZICJpIvJoGfvHicgqEVkuIvNEpG2Jfb9zHbdBRAa6M84yNYiBwf+E9EXOoiNlSIkPI6+gmDW7bOI+YyosPwfWTINPx8KzzZ2J9urUgztmQ8cRno7OlOC2yfpExBeYCFwGpAOLRWS6qq4tUex9VX3ZVX4oMAEY5EoUo4B2QBPgGxFpqaplf5V3lw7XwvrPYc7/QYvLoHHHU3anJDgD5pZsP0jnpnaFhTGnlbMHNsyE9TNh6/dQdBzqhkKry6HVYEi6zBmPZLyKO2dz7Q6kqeoWABH5EBgGnEwQqlryq3d94MRlQ8OAD1U1H9gqImmu1/vJjfH+kggM+RfsWABTfw13zAH/wJO7o0ICiQ2ty5LtB7i9d2KVhmaMV1N1ZlldP8NJDBlLnO2hCU4fQ+vB0LSnc+Wg8Vru/O3EADtLPE8HepQuJCJ3Aw8CdYATs27FAAtKHeuZHqt6YTDsRXjvWpjzFAx46pTdKfGhzN+8H1VFbL1bU5sVF8HOhU5SWD8DDm51tjfp6izg0+oKiGxj60JXIx5P36o6EZgoIjcAjwFjKnqsiIwFxgLExcW5J0Bwqr/Jt8L8F6Hl5ZDQ6+Su5IQwpi3fRfrBYzQNq+e+GIzxRkUFkDYb1k2HjV/C0f3gWwcS+8KF9zhNSCFNPB2lOUfuTBAZQMmpFmNd207nQ+ClszlWVV8FXgVISUlx79zbA56CLd/BtHEw7kcIDAGcGgTA9xszualnvFtDMMYrqMKuZbDiQ1g9xUkKgQ2c6TBaDXbWgXb9f5jqzZ0JYjGQJCKJOB/uo4AbShYQkSRV3eR6egVw4vF04H0RmYDTSZ0ELHJjrGcWEOSMsn5rEMz6nXMZLNAqKpjOTRvyzBfr6dmsES0ibalDU0MdToeVHzmJIWsj+AY4NYRO10OLS5zLwk2N4rYEoaqFIjIemAX4Am+q6hoReRJIVdXpwHgRuRQoAA7ial5ylZuM06FdCNxd5VcwlSWuB/S6H+ZNcNpTWw/Gx0d46aauDHlhHr+elMpn43sTFODxljtjKkd+Dqz7H6z4ALb+ACjEXQBXPg9th9v8SDWc1JRV0VJSUjQ1NdX9b1R4HF67GHL3wF0LoH44APM3ZzH6jUUMaBvFf27sah3WpvoqLnKaU1d86CSHwmMQmuhMs91xBIQ183SEphKJyBJVTSlrn33VPVt+deDqV+HVi+B/98HI/4IIFzYP59FBrXl65jpembuFcRc193SkxpydvWucmsLKj50vQIENnKTQ6Xpo2t2uPqqFLEGci6i2cPHj8PXjzj9UZ6dr5Vd9Elmefoi/f7meDjEN6NUi3MOBGnMGx4/Csv/Csndhzyrw8XM6mzuNgqSBp4z7MbWPNTGdq+IieOdK2L0S7poPDZ3LbI/kF3LVf34kK/c4/7unNzENbXSo8UL5uZD6Bsz/NxzJhMadnS867a852Wxqaofymphssr5z5eMLw/8DKEy7C4qdCfvqB/jx8k3JFBQWc+d/l5BX4Pm+dWNOyjsM3z8Lz7WHr5+AqPZwy0z49ffQ49eWHMwpLEGcj9AEGPQMbPsBPrkNcvcB0CwiiAkjO7My/TBPfLaamlJLM9XY0QPw7VPwrw7OjABNe8Dt38DN004Z+GlMSdYHcb663OR06H3/d9g8x1lsqMtoLmsbxT0Xt+Df36bRuWkoN/Rw40hvY04nd5/TjLT4DSg4Am2uhL6PQONOno7MVAOWIM6XiPMP12YYfH4//O9eZzDRkOe4/9KWrEw/zJ+mr6FN42C6xIV6OlrjjU6sN+LjW3mvmb0LfnwBlrwNRfnQ7mro+7AzF5IxFWSd1JWpuBiW/xe+ehwKjkLvBzmUPJ4rX15MQaHy+b29CQ8K8GyMxrts+xE++RUczXKaLMOaQyPX7cTj4CYVX5P54HZn/ZJl/3UST6dR0PtBCG/h1h/DVF/ldVJbgnCH3H3w5e+ceWrCW7L1gqcZNLWILnEN+e/tPfDzta6fWk8VfnweZj/pJIbWV8CBLc4Stwe2ON/6T/CrC2GJzgC1Ri1OTR5BUU4tdv9m+GECrPwQxAc63wi9H4BQmx/MlM8ShKds+gZmPACHdrAt7hqGbRzIiD4d+MMVbc98rKm5jh10rnzbMNOZrmLov0+d3K64GLIz4MDmnxPG/jTn8cFtUFzwc9k6Qc4l1pnrnVlUk2+BC++19ZxNhVmC8KTjR+C7Z+CnieT6hvD7ozdy2Yi7uLKzF/wDFx53PqRWT4GOo6DNEE9HVPPtWgaTxzgJYMDTzqWlZzNCuagQDu/8OXns3+ysuxDZBnreDcFR7ovd1EiWILzB7pUUT78Xn93L+EE7EXPjSzRr2c4zsWSlwdJ3YPn7Ttu3bx2nvfq6t6HtUM/EVNOpwpK34IvfQv0I51w37e7pqIyxgXJeoXFHfO6YTXa/p+gqG2jyfj/yvp/gfCOsCgV5sHIyvHUFvJgMP02EuJ5ww8fw8CaISYYpt8K6z6smntrk+BFnydrPH4CEPvDrHyw5mGrBahAesGLNajI/updLfZagUe2RoS84H9DusHetU1tY8SHkHXI6RLve7HRiBkf/XC4vGyZdBbtXwMhJzjz/5vxlboTJNzt9BP1/D30ervgVScZUAWti8kJvz9vCTzPf4Z9B7xF0PAui2kF4EoS3dN2SnCtW6tQ/+xc/fgTWTIUl70D6IvDxdwZIJY+BhL6n/4DKOwzvDncmbRv1HrQceH4/ZG23agpMv9eZ8O6a16H5xWc+xpgqZgnCC6kqD05ewezlm/gseQWJ+RudVboObQct/rlgSGyJxFEigQRH/7Jzc9dyp7awagrkZ0OjJCcpdLq+4nPsHDsEk4Y7Uz+Pet9Zj9ubFeQ53861CAJCnKt6AoKdxOqp6akL82HWH2Dxa86UFte+ZVcVGa/lsQQhIoOA53FWlHtdVZ8ptf9B4Fc4q8ZlArep6nbXviJglavoDlUtt/e0uiUIgGPHi7j6pfnsOnSMaXf3IjG8vvOBd2CLkyz2b4KsTc7jrE1wPPfng+sEO4OfwltCg6aQ9rXTPOQX6Fw6mTzGWfnrXD4kjx2Ed4fBvvVw/fvOGsPeIO+wU7vZvRL2rHTuTySH0sTHOUcBp7uFuO5dCaVBLIS3cu7PJ7Ec3A4f3wK7lsIF4+HSP9lSnMareSRBiIgvsBG4DEjHWaP6elVdW6JMf2Chqh4VkTuBfqo60rUvV1UrvMBzdUwQADv2H2XYxHn4+vjw9q3daB/ToOyCqpCz++dkkeWqcWSlQXa6Mytn1zHQ8TqoWwlTehw9AO8OddrQb/iw6ptHcva4EsGKnxPCwW0/7w+KguiO0LgjRHdwBpMdz3VqTvk5pW5lbcs5NeGe4F/flXhbQURL130rZ5DamT7oN86CT8c6NcBhE+2KMFMteCpBXAD8SVUHup7/DkBV/+805bsAL6pqL9fzWpEgANL25TDmzcUcOnqcl0cn0ycp4uxeoDDfuVS1sptUjh5w1rzYnwY3fATN+lXu659wOMPpKylZMziy7+f9oYmuRNDRmWQuumPlXO9fXOwkibzDTtNe5gYn6WZucJJwdvrPZX38nDgiWjm1thP34S2dWtt3f4Uf/glRHWDEO84oZ2OqgfNOECJyH/AWkAO8DnQBHlXVr8o55lpgkKr+yvV8NNBDVcefpvyLwB5Vfcr1vBBYjtP89IyqTisvxuqcIAD2Zucx5s1FpO3L5R/XdWJ4Fy9psz6S5SSJA1vhxsmQ2LfyXnvPapg3welQ12LnQziidYmaQUeIbu8sfekJ+Tk/19ZKJo8DW05t1gps4CSZLqNh8LPgb4tEmeqjMtakvk1VnxeRgUAoMBqYBJw2QZxlgDcBKcBFJTbHq2qGiDQDvhWRVaq6udRxY4GxAHFx1Xs67aiQQCaPu4Cx76Zy/0fL2ZeTxx19miGeXge4fjjcPB3eGQLvj4QbP4aE3uf3mjsXOd+2N37pdCpfMB7aXw0RbbxricuAYIjp6txKKjzujF7O3ABZG2D/FmjeHzqO8EycxrhJRRPEiU+pwcAkVV0jZ/7kygCalnge69p26guLXAr8AbhIVU/OUKaqGa77LSLyHU6t5ZQEoaqvAq+CU4Oo4M/itUIC/Xnntu48OHkFf525nr3Z+fxhcBt8fDycJIIiYMz/4O0h8N4IuGkKxF94dq+hCpu/hXn/chZYqhsK/f8A3e+onD6TquRXx2liimjl6UiMcauKjthZIiJf4SSIWSISDBSf4ZjFQJKIJIpIHWAUML1kAVe/wyvAUFXdV2J7qIgEuB6HA72AtdQCAX6+/HtUF27tlcAb87Zy30fLyS/0gmVLgyKdJBHSBN67DnYsqNhxxcWwdjq82g/+e7XTnzHwr3D/arjoN9UvORhTi1S0BnE70BnY4rriKAy4tbwDVLVQRMYDs3Auc33TVfN4EkhV1enAs0AQ8LGrQnLictY2wCsiUoyTxJ4pefVTTefjIzwxpC3RIYH83xfrycrJ55WbkwkJ9PDlksFRrprEFfDfa2H0VGjareyyRQXOeIx5/3KaYUIT4coXnPUJ/GxNDGOqg4p2UvcClqvqEVd/QVfg+RNjFrxBde+kPp2py9J55OOVtIgM4p3buhMV4gVt9Nm7nCRxJAtGT4PYEtOEFBxzFqv58QU4vAMi20GfB52xGb62gKEx3qYyJut7CTgqIp2Ah3D6At6tpPhMOa7qEsubt3Rj54GjXP2f+aTtK+Pa/aoW0gTGfA71wpz5mzKWOnM5zfsXPNcBZj7sjPS+/iO480focK0lB2OqoYrWIJaqalcReQLIUNU3Tmxzf4gVU1NrECeszjjMLW8torBYeWNMN5LjvaDt/tBOpyZx7JDzPP+wM6Cuz0MQ38tzU10YYyqsMmoQOa6BbqOBGSLiA9j8AVWofUwDPrnzQhrW9efG1xfw9dq9ng4JGjaFWz6H0Dho1hfumOP0SyT0tuRgTA1Q0QQxEsjHGQ+xB+eS1WfdFpUpU3yj+ky580JaRQXz60mpfLBoh6dDcpa7HDcPRv73l+MFjDHVWoUShCspvAc0EJEhQJ6qWh+EB4QHBfD+HT3p2zKC3326iue+2UhNmZHXGONdKpQgRGQEsAi4DhgBLHRNpWE8oH6AH6/dnMK1ybE8980mfj91FYVFZxqWYowxZ6eil5b8Aeh2YjCbiEQA3wBT3BWYKZ+/rw/PXtuR6JBAXpyTxvo9Ofz9mo4kRQV7OjRjTA1R0T4In5IjnYH9Z3GscRMR4eGBrXh+VGe2ZR3hihfm8cLsTRwvtNqEMeb8VfRD/ksRmSUit4jILcAMYKb7wjJnY1jnGL5+8CIGto9mwtcbGfriPFbsPOTpsIwx1VyF14MQkWtw5kQC+EFVp7otqnNQ08dBVNTXa/fy2LRVZObk86s+zXjg0pbUrePr6bCMMV7K1qSuZbLzCnjmi/W8v3AH8Y3q8X9Xd+DC5hVck9oYU6uc80A5EckRkewybjkiku2ecM35Cgn0569XdeCDO3oCcMNrC/ndp6vIzivwcGTGmOqk3AShqsGqGlLGLVhVQ6oqSHNuLmjeiC/v68uv+zbjo8U7uGzC994xAtsYUy3YlUg1XN06vvxucBum3d2L0Hp1uOPdVMa/v5Ss3PwzH2yMqdUsQdQSHWMbMn18bx66rCVfrdnLpRO+Z+qydBuFbYw5LUsQtUgdPx/uuSSJGff2pll4fR74aAW3vr2YjEPHPB2aMcYLuTVBiMggEdkgImki8mgZ+x8UkbUislJEZotIfIl9Y0Rkk+s2xp1x1jZJUcF8PO5C/nRlWxZtPcCACd8z6adtFBdbbcIY8zO3JQgR8QUmApcDbYHrRaRtqWLLgBRV7YgzbcffXceGAX8EegDdgT+KiBcsgFBz+PoIt/RKZNb9fekaH8rjn61h1GsL2Jp1xNOhGWO8hDtrEN2BNFXdoqrHgQ+BYSULqOocVT3qeroAZxpxgIHA16p6QFUPAl8Dg9wYa63VNKwe797Wnb9f25F1u7MZ9NxcXpu7hSKrTRhT67kzQcQAO0s8T3dtO53bgS/O8VhzHkSEESlN+ebBi+iTFM7TM9dxzUvz2bg3x9OhGWM8yCs6qUXkJiCFs1yESETGikiqiKRmZma6J7haJCokkNduTuH5UZ3Zvv8IQ16Yx79nb6LAphI3plZyZ4LIAJqWeB7r2nYKEbkUZzrxoaqafzbHquqrqpqiqikRERGVFnhtJiInJ/8b0C6Kf369kWEv/sjqjMOeDs0YU8XcmSAWA0kikigidYBRwPSSBUSkC/AKTnIoOZ34LGCAiIS6OqcHuLaZKhIeFMCLN3TlldHJZObmM2zij/xj1gbyC4s8HZoxpoq4LUGoaiEwHueDfR0wWVXXiNZ67KMAAB3rSURBVMiTIjLUVexZIAj4WESWi8h017EHgL/gJJnFwJOubaaKDWwXzTcPXMRVXWJ4cU4aV7wwj6U7Dno6LGNMFbDZXE2FfbdhH7//dBW7s/O4vVciDw1oZVOJG1PNnfNsrsaU1K9VJLMe6MuNPeJ4fd5WBj0/l5827/d0WMYYN7EEYc5KcKA/Tw13phJXhetfW8Bj01aRY1OJG1PjWIIw5+SC5o348v4+3N47kfcW7uCiZ7/jrR+3Wie2MTWIJQhzzurV8ePxIW2ZfndvWkcH8+f/reXSCd/z2fIMm9fJmBrAEoQ5bx1iG/Der3rw7m3dCQ7w574Pl3Pli/P4YZMNXjSmOrMEYSqFiNC3ZQSf39Ob50Z25vCxAka/sYibXl/IqnQbZGdMdWQJwlQqHx9heJcYZj90EY8PacuaXYe58sV53PvBMnbsP3rmFzDGeA0bB2HcKjuvgFe/38Lr85wZYm/sEc/4i1sQHhTg6dCMMZQ/DsIShKkSe7PzeO6bTUxO3Umgnw+/vqg5t/dOpH6An6dDM6ZWswRhvEbavlyenbWeWWv2Eh4UwH2XJjGqW1P8fa210xhPsJHUxmu0iAzildEpfHLnhSSG1+PxaasZ8K+5TFuWQaFNK26MV7EEYTwiOT6Uyb++gNdvTsHfV7j/o+VcMuF7Pli0wwbbGeMlrInJeFxxsfLV2r1MnJPGqozDRIcEMrZvM67vHmeTARrjZtYHYaoFVWXupiwmfpvGom0HCKtfh9t7JzL6gnhCAv09HZ4xNZIlCFPtLNp6gIlz0vh+YybBgX6MuSCB23onEla/jqdDM6ZGsQRhqq1V6YeZOCeNL9fsoa6/Lzf0iGNs32ZEhQR6OjRjagSPXcUkIoNEZIOIpInIo2Xs7ysiS0WkUESuLbWvyLXK3MmV5kzt0yG2AS+PTubrB/oyqH00b8/fRp+/zeH3U1ex84CNzDbGndxWgxARX2AjcBmQjrN06PWqurZEmQQgBHgYmK6qU0rsy1XVoIq+n9Ugaocd+4/y8tzNTElNp0iVYZ2acFf/5rSIDPZ0aMZUS+XVINw5jLU7kKaqW1xBfAgMA04mCFXd5tpnF8CbColrVI+/XtWBey9O4rUftvD+wh1MXZ7BJa2juL13Ij2bhSEing7TmBrBnU1MMcDOEs/TXdsqKlBEUkVkgYgMr9zQTHUX3SCQx4e05cdHL+ae/i1Ysv0A17+2gCtemMeUJek2lsKYSuDNA+XiXdWeG4DnRKR56QIiMtaVRFIzM23tgdoorH4dHhzQip9+dwnPXN2BwuJiHv54Bb2emcPz32wiKzff0yEaU225M0FkAE1LPI91basQVc1w3W8BvgO6lFHmVVVNUdWUiIiI84vWVGuB/r6M6h7HrPv7Mun27nSICeFf32zkwme+5TdTVrBud7anQzSm2nFnH8RiIElEEnESwyic2sAZiUgocFRV80UkHOgF/N1tkZoaQ0TokxRBn6QI0vbl8vb8rUxZks7k1HQubN6I23sn0r9VJD4+1k9hzJm4dRyEiAwGngN8gTdV9WkReRJIVdXpItINmAqEAnnAHlVtJyIXAq8AxTi1nOdU9Y3y3suuYjKnc+jocT5YtJN35m9jT3YeieH1ubVXAtd0jbXpxk2tZwPljAEKior5YvUe3pi3lRU7DxEc6Mf13eMYc2ECMQ3rejo8YzzCEoQxpSzZfpA3f9zKl6v3AHBFh8bc3b8FraJtPIWpXTw1DsIYr5UcH0pyfCgZh47x9o9beW/hDqav2MXAdlGM759Eh9gGng7RGI+zGoQxwMEjx3lr/jbe/nEr2XmFXNQygnsubkFKQpinQzPGrayJyZgKyskrYNKC7bz+w1YOHDlOz2ZhjO+fRK8WjWyEtqmRLEEYc5aOHi/kg0U7eXXuZvZm59O5aUPG92/BJW0iLVGYGsUShDHnKL+wiClL0nnpu82kHzxGm8YhjO/fgkHto/G1sRSmBrAEYcx5KigqZvryXUz8Lo0tmUdoFlGfu/u1YGjnJvj7evOMNcaUzxKEMZWkqFj5YvVuXvw2jfV7cmgaVpdxFzXnmq6xBPrb+tmm+rEEYUwlU1Vmr9vHi3PSWL7zECGBflzRsQlXd40hJT7U+ilMtWEJwhg3UVV+2ryfyak7mbVmL8cKiogLq8fwLjFc3SWGhPD6ng7RmHJZgjCmCuTmFzJr9R4+XZbO/M37UYWucQ25umssQzo2pmG9Op4O0ZhfsARhTBXbffgYny3fxadL09m4N5c6vj70bx3B1V1j6d8qkjp+1rFtvIMlCGM8RFVZsyubT5dmMH1FBlm5x2lYz58rOzbhqq4xdGna0PorjEdZgjDGCxQWFfNDWhafLs3gqzV7yC8sJjG8Pld1ieGqLjE0Davn6RBNLWQJwhgvk5NXwBernP6KBVsOAJASH8pVXWO4ooP1V5iqYwnCGC+WcegYny3PYOrSDDbty8XfV+jfKpKru8bQv3UkAX42vsK4j8cShIgMAp7HWVHudVV9ptT+vjgrznUERqnqlBL7xgCPuZ4+parvlPdeliBMdXeiv2Lasgw+W7GLzJx81/iKxgzvHEO3hDBbKtVUOo8kCBHxBTYClwHpOGtUX6+qa0uUSQBCgIeB6ScShIiEAalACqDAEiBZVQ+e7v0sQZiapLComPmb9zNtWQZfrtnD0eNFxDSsy/AuTbiqSwwtIm1hI1M5PLVgUHcgTVW3uIL4EBgGnEwQqrrNta+41LEDga9V9YBr/9fAIOADN8ZrjNfw8/Whb8sI+raM4KnjhXy1Zi9Tl2Xw0nebmThnMx1iGjC8SwxXdmpMZHCgp8M1NZQ7E0QMsLPE83Sgx3kcG1NJcRlTrdSr48fwLjEM7xLDvpw8/rdiN9OWZfCXz9fy9Iy19E6K4JquMQxsF23zQZlKVa2XHBWRscBYgLi4uF/sLygoID09nby8vKoOrcYJDAwkNjYWf39/T4dSq0UGB3J770Ru751I2r4cpi7LYNqyXdz34XIa1PVneOcmjOjWlHZNbMlUc/7cmSAygKYlnse6tlX02H6ljv2udCFVfRV4FZw+iNL709PTCQ4OJiEhwQYjnQdVZf/+/aSnp5OYmOjpcIxLi8hgHhnYmocua8X8zfv5KHUnHyzeyTs/baddkxBGdmvKsE4xNKhnSd2cG3cmiMVAkogk4nzgjwJuqOCxs4C/ikio6/kA4HdnG0BeXp4lh0ogIjRq1IjMzExPh2LK4OMj9E4Kp3dSOIeOHmfasgw+Sk3nic/W8PSMdQxqH83IlKb0bNbIroIyZ8VtCUJVC0VkPM6HvS/wpqquEZEngVRVnS4i3YCpQChwpYj8WVXbqeoBEfkLTpIBePJEh/XZsuRQOew8Vg8N69Xhll6JjLkwgdUZ2XyUuoPPlu/is+W7iAurx3XJsVybEkvjBnU9HaqpBmr0QLl169bRpk0bD0VU89j5rJ7yCor4YvVuPlq8kwVbDuAj0LdlBCNTmnJJmyibOLCWK+8yV/vLcLNDhw7xn//856yPGzx4MIcOHTrr42655RamTJly5oKm1gj09+WqLrF8OPYCvn+kH3f2a8663dnc+d5SLvi/2Tz1+VrW7DpMTfmyaCpPtb6KqTo4kSDuuuuuU7YXFhbi53f60z9z5kx3h2ZqofhG9XlkYGseuLQlczdl8tHinbw9fxuvz9tKbGhdBraLZmC7aJLjQ/G1/opar9YkiD//bw1rd2VX6mu2bRLCH69sV26ZRx99lM2bN9O5c2f8/f0JDAwkNDSU9evXs3HjRoYPH87OnTvJy8vjvvvuY+zYsQAkJCSQmppKbm4ul19+Ob1792b+/PnExMTw2WefUbfumduQZ8+ezcMPP0xhYSHdunXjpZdeIiAggEcffZTp06fj5+fHgAED+Mc//sHHH3/Mn//8Z3x9fWnQoAFz586tlHNkvJOfrw8Xt47i4tZR7M/N5+u1e5m1Zg+TftrOG/O2Eh5Uh8vaRjGgXTQXNm9k80HVUrUmQXjKM888w+rVq1m+fDnfffcdV1xxBatXrz55ueibb75JWFgYx44do1u3blxzzTU0atTolNfYtGkTH3zwAa+99hojRozgk08+4aabbir3ffPy8rjllluYPXs2LVu25Oabb+all15i9OjRTJ06lfXr1yMiJ5uxnnzySWbNmkVMTMw5NW2Z6qtRUACjuscxqnscOXkFzNmQyaw1e5i+fBcfLNpJcIAf/VpHMrBdFP1aRRIUYB8btUWt+U2f6Zt+VenevfspYwleeOEFpk6dCsDOnTvZtGnTLxJEYmIinTt3BiA5OZlt27ad8X02bNhAYmIiLVu2BGDMmDFMnDiR8ePHExgYyO23386QIUMYMmQIAL169eKWW25hxIgRXH311ZXxo5pqKDjQn6GdmjC0UxPyCoqYvzmLWav38vW6vfxvxS7q+PnQp0U4A9tFc2nbKMLq27TkNVmtSRDeon79nxex/+677/jmm2/46aefqFevHv369Stz1HdAQMDJx76+vhw7duyc39/Pz49FixYxe/ZspkyZwosvvsi3337Lyy+/zMKFC5kxYwbJycksWbLkF4nK1C6B/r4nm6GeLiomdftBZq3Zw1dr9jJ7/T58PoVuCWEMah9NrxbhJIbXx9/XrnupSSxBuFlwcDA5OTll7jt8+DChoaHUq1eP9evXs2DBgkp731atWrFt2zbS0tJo0aIFkyZN4qKLLiI3N5ejR48yePBgevXqRbNmzQDYvHkzPXr0oEePHnzxxRfs3LnTEoQ5yc/Xh57NGtGzWSOeGNKWNbuymbVmD7PW7OHP/3Pm36zj60OLyCBaNw6mTXQIraKDad04mIigABtHU01ZgnCzRo0a0atXL9q3b0/dunWJioo6uW/QoEG8/PLLtGnThlatWtGzZ89Ke9/AwEDeeustrrvuupOd1OPGjePAgQMMGzaMvLw8VJUJEyYA8Mgjj7Bp0yZUlUsuuYROnTpVWiymZhER2sc0oH1MAx4a0IqtWUdYsfMQ6/Zks353Dj+6llU9oVH9OrRuHEzr6BBaRwfTpnEILSKDbGLBasAGypkKs/NpKurAkeOsdyWM9Xuy2bAnhw17c8grcGb29xFoFhFEq+hg2kQH06NZI5LjQm0qEA/w1HoQxphaKqx+HS5sHs6FzcNPbisqVrbvP8L6PTms353Nuj05rEw/xIyVuwFOLog0vHMMSVG2IJI3sARRTd199938+OOPp2y77777uPXWWz0UkTHl8/URmkUE0SwiiMEdGp/cnp1XwOx1e5m6bNfJBZHaNQlheOcYhnZuQlSILYjkKdbEZCrMzqdxt8ycfD5fuYtpyzJYkX4YEbiweSOGdY5hUPtoQgJt6vLKZk1MxphqISI4gFt7JXJrr0S2ZOYybfkuPluewW+mrOSxaau5rE0Uwzo3oV+rSJtksApYgjDGeKVmEUE8eFlLHrg0ieU7DzFtWQafr9zNjFW7aVDXnys6NmZ45xhS4q1z210sQRhjvJqI0CUulC5xoTw2pC3zNmUxbXkGU5dm8P7CHcQ0rEvPZo2IbhBAVEggkcGBRIU4jyOCA2zw3nmwBGGMqTb8fX3o3zqS/q0jOZJfyFdr9/DZ8l38mJZFZm4+RcW/7FNtVL8OkSGupOFKHpEhgUQGO0nkRCKx2Wt/ya0JQkQGAc/jrCj3uqo+U2p/APAukAzsB0aq6jYRSQDWARtcRReo6jh3xuotgoKCyM3NLXPftm3bGDJkCKtXr67iqIzxPvUD/LiqSyxXdYkFnMtoDxw5zt7sPPbl5LE3O5+92c79vuw89ubksXZXNlm5+ZTOIw3r+dOrRTgXJUXQp2W4rbjn4rYEISK+wETgMiAdWCwi01V1bYlitwMHVbWFiIwC/gaMdO3brKqd3RWfMaZm8fURIoIDiAgOABqctlxhUTH7XYlkb3Y+e7LzWL7jED9syjw5JiMpMoi+LSPo2zKCHolhtXbUtztrEN2BNFXdAiAiHwLDgJIJYhjwJ9fjKcCL4q5JW754FPasqtzXjO4Alz9TbpFHH32Upk2bcvfddwPwpz/9CT8/P+bMmcPBgwcpKCjgqaeeYtiwYWf11nl5edx5552kpqbi5+fHhAkT6N+/P2vWrOHWW2/l+PHjFBcX88knn9CkSRNGjBhBeno6RUVFPP7444wcOfLMb2JMDeTn63OyaemE0T3jUVU27M1h7sZM5m7MYtICZ22MOn4+9EgMo2+SkzBaRgXVmrml3JkgYoCdJZ6nAz1OV0ZVC0XkMHBihrhEEVkGZAOPqeoPbozVbUaOHMn9999/MkFMnjyZWbNmce+99xISEkJWVhY9e/Zk6NChZ/VHN3HiRESEVatWsX79egYMGMDGjRt5+eWXue+++7jxxhs5fvw4RUVFzJw5kyZNmjBjxgzAmSTQGHMqEXHNFxXC2L7NOXa8iIVb9zN3YxY/bMrk6ZnreHrmOqJCAujjShZ9WoQTWoOnPPfWTurdQJyq7heRZGCaiLRT1VOWhBORscBYgLi4uPJf8Qzf9N2lS5cu7Nu3j127dpGZmUloaCjR0dE88MADzJ07Fx8fHzIyMti7dy/R0dEVft158+Zxzz33ANC6dWvi4+PZuHEjF1xwAU8//TTp6elcffXVJCUl0aFDBx566CF++9vfMmTIEPr06eOuH9eYGqNuHV/6tYqkX6tIAHYdOsYPmzKZuymLr9fuZcqSdESgY0wDusSFEt+oHnFhzi02tB5161T/Zil3JogMoGmJ57GubWWVSRcRP5yGw/3qDO/OB1DVJSKyGWgJnDJUWlVfBV4FZyS1O36IynDdddcxZcoU9uzZw8iRI3nvvffIzMxkyZIl+Pv7k5CQUOY6EOfihhtuoEePHsyYMYPBgwfzyiuvcPHFF7N06VJmzpzJY489xiWXXMITTzxRKe9nTG3RpGFdRnaLY2S3OIqKlZXph/hhUxZzN2YyOXUnR48XnVI+MjjgZMJo6rqPcyWRiKCAajF2w50JYjGQJCKJOIlgFHBDqTLTgTHAT8C1wLeqqiISARxQ1SIRaQYkAVvcGKtbjRw5kjvuuIOsrCy+//57Jk+eTGRkJP7+/syZM4ft27ef9Wv26dOH9957j4svvpiNGzeyY8cOWrVqxZYtW2jWrBn33nsvO3bsYOXKlbRu3ZqwsDBuuukmGjZsyOuvv+6Gn9KY2sPX5+exGfdekoSqcwXVjgNH2XHgKDtd9zsOHGXh1gNMXZ5ByVmNAvx8fk4aYfWIaViXyJAAol19I9ENAr2iY9xtCcLVpzAemIVzmeubqrpGRJ4EUlV1OvAGMElE0oADOEkEoC/wpIgUAMXAOFU94K5Y3a1du3bk5OQQExND48aNufHGG7nyyivp0KEDKSkptG7d+qxf86677uLOO++kQ4cO+Pn58fbbbxMQEMDkyZOZNGkS/v7+REdH8/vf/57FixfzyCOP4OPjg7+/Py+99JIbfkpjai8RoVFQAI2CAugSF/qL/fmFRew6lHdqAtnvPF609QC5+YW/OCYk0I/oBoEnO9SjXAkkMiTwZCIJD6qDnxsHAtpkfabC7HwaU/lUlZz8QvYe/vmy270lbntc4zj25fxyIKCPQHhQAN0Tw3jxhq7n9P42WZ8xxngpESEk0J+QQP9y18EoKlb2H8ln7+F8V+LIY5/r3hn7UfksQXihVatWMXr06FO2BQQEsHDhQg9FZIzxNF8fITLYmWuqQzkDASuTJQgv1KFDB5YvX+7pMIwxtVyNn+awpvSxeJqdR2NqnxqdIAIDA9m/f799uJ0nVWX//v0EBtrSj8bUJjW6iSk2Npb09HQyMzM9HUq1FxgYSGxsrKfDMMZUoRqdIPz9/UlMTPR0GMYYUy3V6CYmY4wx584ShDHGmDJZgjDGGFOmGjPVhohkAmc/693PwoGsSgrHHSy+82PxnR+L7/x4c3zxqhpR1o4akyDOl4iknm4+Em9g8Z0fi+/8WHznx9vjOx1rYjLGGFMmSxDGGGPKZAniZ696OoAzsPjOj8V3fiy+8+Pt8ZXJ+iCMMcaUyWoQxhhjylSrEoSIDBKRDSKSJiKPlrE/QEQ+cu1fKCIJVRhbUxGZIyJrRWSNiNxXRpl+InJYRJa7bk9UVXwlYtgmIqtc759axn4RkRdc53CliJzbMlfnFlurEudmuYhki8j9pcpU6TkUkTdFZJ+IrC6xLUxEvhaRTa77X65R6ZQb4yqzSUTGVGF8z4rIetfvb6qINDzNseX+Lbgxvj+JSEaJ3+Hg0xxb7v+7G+P7qERs20SkzLn7q+L8nTdVrRU3nHWxNwPNgDrACqBtqTJ3AS+7Ho8CPqrC+BoDXV2Pg4GNZcTXD/jcw+dxGxBezv7BwBeAAD2BhR78fe/BucbbY+cQZ331rsDqEtv+Djzqevwo8LcyjgsDtrjuQ12PQ6sovgGAn+vx38qKryJ/C26M70/AwxX4/Zf7/+6u+Ert/yfwhKfO3/nealMNojuQpqpbVPU48CEwrFSZYcA7rsdTgEtERKoiOFXdrapLXY9zgHVATFW8dyUbBryrjgVAQxFp7IE4LgE2q+r5DJ48b6o6FzhQanPJv7N3gOFlHDoQ+FpVD6jqQeBrYFBVxKeqX6lqoevpAsBj0/ie5vxVREX+389befG5PjtGAB9U9vtWldqUIGKAnSWep/PLD+CTZVz/IIeBRlUSXQmupq0uQFlrjF4gIitE5AsRaVelgTkU+EpElojI2DL2V+Q8V4VRnP4f09PnMEpVd7se7wGiyijjLefxNpwaYVnO9LfgTuNdTWBvnqaJzhvOXx9gr6puOs1+T56/CqlNCaJaEJEg4BPgflXNLrV7KU6TSSfg38C0qo4P6K2qXYHLgbtFpK8HYiiXiNQBhgIfl7HbG87hSeq0NXjlpYQi8gegEHjvNEU89bfwEtAc6AzsxmnG8UbXU37twev/l2pTgsgAmpZ4HuvaVmYZEfEDGgD7qyQ65z39cZLDe6r6aen9qpqtqrmuxzMBfxEJr6r4XO+b4brfB0zFqcqXVJHz7G6XA0tVdW/pHd5wDoG9J5rdXPf7yijj0fMoIrcAQ4AbXUnsFyrwt+AWqrpXVYtUtRh47TTv6+nz5wdcDXx0ujKeOn9nozYliMVAkogkur5hjgKmlyozHThxtci1wLen++eobK72yjeAdao64TRlok/0iYhId5zfX1UmsPoiEnziMU5n5upSxaYDN7uuZuoJHC7RnFJVTvvNzdPn0KXk39kY4LMyyswCBohIqKsJZYBrm9uJyCDgN8BQVT16mjIV+VtwV3wl+7SuOs37VuT/3Z0uBdaranpZOz15/s6Kp3vJq/KGc4XNRpyrG/7g2vYkzj8CQCBOs0QasAhoVoWx9cZpalgJLHfdBgPjgHGuMuOBNThXZCwALqzi89fM9d4rXHGcOIclYxRgouscrwJSqjjG+jgf+A1KbPPYOcRJVLuBApx28Ntx+rVmA5uAb4AwV9kU4PUSx97m+ltMA26twvjScNrvT/wdnriyrwkws7y/hSqKb5Lrb2slzod+49LxuZ7/4v+9KuJzbX/7xN9cibJVfv7O92YjqY0xxpSpNjUxGWOMOQuWIIwxxpTJEoQxxpgyWYIwxhhTJksQxhhjymQJwhgv4Jpl9nNPx2FMSZYgjDHGlMkShDFnQURuEpFFrjn8XxERXxHJFZF/ibOOx2wRiXCV7SwiC0qsqxDq2t5CRL5xTRi4VESau14+SESmuNZieK+qZhI25nQsQRhTQSLSBhgJ9FLVzkARcCPO6O1UVW0HfA/80XXIu8BvVbUjzsjfE9vfAyaqM2HghTgjccGZwfd+oC3OSNtebv+hjCmHn6cDMKYauQRIBha7vtzXxZlor5ifJ2X7L/CpiDQAGqrq967t7wAfu+bfiVHVqQCqmgfger1F6pq7x7UKWQIwz/0/ljFlswRhTMUJ8I6q/u6UjSKPlyp3rvPX5Jd4XIT9fxoPsyYmYypuNnCtiETCybWl43H+j651lbkBmKeqh4GDItLHtX008L06qwWmi8hw12sEiEi9Kv0pjKkg+4ZiTAWp6loReQxnFTAfnBk87waOAN1d+/bh9FOAM5X3y64EsAW41bV9NPCKiDzpeo3rqvDHMKbCbDZXY86TiOSqapCn4zCmslkTkzHGmDJZDcIYY0yZrAZhjDGmTJYgjDHGlMkShDHGmDJZgjDGGFMmSxDGGGPKZAnCGGNMmf4fNMpEzg99lyIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['loss'])\n",
    "plt.plot(history.history['val_loss'])\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train_loss', 'val_loss'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "40-oCtFP6OWn",
    "outputId": "1ee4ce7e-579b-4131-b85e-73cefc18393d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVfr48c+TRgKEACHUEHrvECliQRBFVsGGWHAtq+z+7LoN17r2r6u7bnFV7CiKFUXFggoiSw29l0QIoSaBhISSNs/vj3sDQ5zAYDIzKc/79ZpX7tx77txnJjPzzDnn3nNEVTHGGGPKCgt1AMYYY6omSxDGGGN8sgRhjDHGJ0sQxhhjfLIEYYwxxqeIUAdQWZo0aaJt27YNdRjGGFOtLF26NEtVE3xtqzEJom3btqSkpIQ6DGOMqVZEZFt526yJyRhjjE+WIIwxxvhkCcIYY4xPNaYPwpeioiIyMjI4cuRIqEMxfoqOjiYxMZHIyMhQh2JMrVejE0RGRgaxsbG0bdsWEQl1OOYkVJXs7GwyMjJo165dqMMxptar0U1MR44cIT4+3pJDNSEixMfHW43PmCqiRicIwJJDNWP/L2OqjhrdxGSMMdWdqnKosIS8I8UcOFLEgcNF7t9i8o4UceBIMY3qRnH1oKRKP7YlCGOMCTJVJTO/gI2789i4O4/duUeOfekXOH+PJYNiSjwnnrenX1JDSxDVUU5ODu+88w633HLLKe03evRo3nnnHRo2bBigyIwxwXCwoJhNe5xEsMFNCBv35LHvYOHRMnWjwmkQHUmDmAgaREeSEFuHDgn1iPVa1yAmkgbRkcRGR7jLzt/Y6AjqRIQHJHZLEAGWk5PDf//7358liOLiYiIiyn/5Z86cGejQKuRk8RtT2xSXePgp6yAbyySD9H2HjpapGxVOp2axjOzWjC7NY+naPJYuzWOJr18nhJGXr9Z8wv/62VrW7TxQqY/ZvWUDHrqoxwnLTJo0idTUVPr27UtkZCTR0dE0atSIDRs2sGnTJi6++GK2b9/OkSNHuPPOO5k4cSJwbGyp/Px8LrjgAs444wzmz59Pq1at+PTTT4mJifF5vJdffpnJkydTWFhIx44deeutt6hbty579uzhd7/7HWlpaQC88MILnH766UyZMoVnnnkGEaF379689dZbXH/99Vx44YVcfvnlANSvX5/8/HzmzJnDAw884Ff8X331FX/5y18oKSmhSZMmzJo1iy5dujB//nwSEhLweDx07tyZBQsWkJDgc5wwY6o0VWVVRi7Tl+9g0U/7SN2bT2GJB4DwMKFdk3r0Soxj3IBEOrvJoHWjuoSFVZ8TMWpNggiVp556ijVr1rBixQrmzJnDr371K9asWXP0PP/XXnuNxo0bc/jwYU477TQuu+wy4uPjj3uMzZs38+677/Lyyy9zxRVX8NFHHzFhwgSfx7v00ku5+eabAbj//vt59dVXuf3227njjjs4++yzmT59OiUlJeTn57N27Voee+wx5s+fT5MmTdi3b99Jn8+yZctOGr/H4+Hmm29m7ty5tGvXjn379hEWFsaECROYOnUqd911F99++y19+vSx5GCqnV25h5m+fAcfL9vBlr35REWEMbh9PGd1akIXt0bQIaE+0ZGBafYJpoAmCBEZBfwTCAdeUdWnymxvA7wGJAD7gAmqmuFuexr4Fc6puLOAO1X1xD01J3CyX/rBMnDgwOMuAvvXv/7F9OnTAdi+fTubN2/+WYJo164dffv2BWDAgAFs3bq13Mdfs2YN999/Pzk5OeTn53P++ecD8P333zNlyhQAwsPDiYuLY8qUKYwbN44mTZoA0Lhx40qJPzMzk7POOutoudLHvfHGGxk7dix33XUXr732GjfccMNJj2dMVXCosJhv1u7ho2UZzNuShSokt2nEk5f2YnSvFsTF1Mwr/wOWIEQkHHgeGAlkAEtEZIaqrvMq9gwwRVXfFJHhwJPAtSJyOjAU6O2WmwecDcwJVLzBUq9evaPLc+bM4dtvv2XBggXUrVuXYcOG+bxIrE6dY+2T4eHhHD58uNzHv/766/nkk0/o06cPb7zxBnPmzDnlGCMiIvB4nKqyx+OhsPBYZ9ovib9U69atadasGd9//z2LFy9m6tSppxybMcHi8SiLt+7jo6UZzFy9i4OFJSQ2iuH24Z24rH8r2sTXO/mDVHOBvFBuILBFVdNUtRCYBowtU6Y78L27PNtruwLRQBRQB4gE9gQw1oCJjY0lLy/P57bc3FwaNWpE3bp12bBhAwsXLqzw8fLy8mjRogVFRUXHfQGPGDGCF154AYCSkhJyc3MZPnw4H3zwAdnZ2QBHm5jatm3L0qVLAZgxYwZFRUWnFP/gwYOZO3cuP/3003GPC3DTTTcxYcIExo0bR3h49a+Cm5pna9ZB/v7NRs7622yunLyQL9fs5le9W/DexMHM/eM53DOyc61IDhDYJqZWwHav+xnAoDJlVgKX4jRDXQLEiki8qi4QkdnALkCA/6jq+gDGGjDx8fEMHTqUnj17EhMTQ7NmzY5uGzVqFC+++CLdunWjS5cuDB48uMLHe/TRRxk0aBAJCQkMGjToaHL65z//ycSJE3n11VcJDw/nhRdeYMiQIdx3332cffbZhIeH069fP9544w1uvvlmxo4dS58+fRg1atRxtQZv5cWfkJDA5MmTufTSS/F4PDRt2pRZs2YBMGbMGG644QZrXjJVSu7hImau3sVHSzNI2bYfETijYxP+eH4XzuvenJio2vljRirQrH/iBxa5HBilqje5968FBqnqbV5lWgL/AdoBc4HLgJ5AE5ykMd4tOgv4k6r+WOYYE4GJAElJSQO2bTt+YqT169fTrVu3yn9y5hdLSUnh7rvv5scffyy3jP3fTEV5PEpeQTEHDheRc6iInMOF5LrLuYeL3GVn3f5DRazYnkNhsYeOTetzWf9ELunXiuZx0aF+GkEhIktVNdnXtkDWIHYArb3uJ7rrjlLVnTg1CESkPnCZquaIyM3AQlXNd7d9CQwBfiyz/2RgMkBycnJgMp2pNE899RQvvPCC9T2YSpF3pIiFafuYtzmTrdmHyDlcRK77pZ97uIgTXXwcHRlGw5go4mIiiasbydUDk7i0fyt6tYqz8cC8BDJBLAE6iUg7nMRwJXC1dwERaQLsU1UPcC/OGU0A6cDNIvIkThPT2cBzAYy12rn11lv53//+d9y6O++8s0o33UyaNIlJkyaFOgxTTZV4lFUZOczbnMWPm7NYlr6fYo8SExlOp2b1iYuJJKlxXRrGRBIXE0nDuqV/o47ebxjjXJFcE05BDYaAJQhVLRaR24CvcU5zfU1V14rII0CKqs4AhgFPiojiNDHd6u7+ITAcWI3TYf2Vqn4WqFiro+effz7UIRgTcBn7D/Hj5ix+3JzJ/7Zkk3u4CBHo2TKOiWe158xOCfRv0zBgQ03UdgG9DkJVZwIzy6x70Gv5Q5xkUHa/EuC3gYzNGFP1lDYb/bg5k3mbs0jLOghAi7hozu/RjDM7JTC0YxMa14sKcaS1g11JbYwJmcOFJazdmcuC1OyfNRsN6RDPtUPacGanBDok1LO+gRCwBGGMCYqC4hI27s5jZUYuqzNyWJWRy6Y9eXgURKBXK2s2qmosQRhjKl1xiYctmfms2p7LyowcVu/IZcOuvKOD2TWuF0XvxDjO69Gc3q3i6N+mkTUbVUGWIKqY0pFTjakuPB7lp+yDrM5wk0FGLmt3HuBwUQkAsXUi6JUYx41ntKN3Yhy9E+No1TDGmoyqAUsQxieb78GczJa9eXywNIPpy3awN68AcK4v6NkyjisHtqZPYkN6J8bRNr5etRri2hxTe74BvpwEu1dX7mM27wUXPHXCIpMmTaJ169bceqtzBu/DDz9MREQEs2fPZv/+/RQVFfHYY48xdmzZYap+Lj8/n7Fjx/rcz9e8Dr7mgGjZsiUXXngha9asAeCZZ54hPz+fhx9+mGHDhtG3b1/mzZvHVVddRefOnXnssccoLCwkPj6eqVOn0qxZM/Lz87n99ttJSUlBRHjooYfIzc1l1apVPPecc7nKyy+/zLp16/jHP/7xi19eU/XkHipixqqdfLg0g5XbcwgPE87pksDI7s3o07ohHRPqExEeyCHeTDDVngQRIuPHj+euu+46miDef/99vv76a+644w4aNGhAVlYWgwcPZsyYMSetckdHRzN9+vSf7bdu3Tqf8zr4mgNi//79JzxGYWEhKSkpAOzfv5+FCxciIrzyyis8/fTTPPvsszz66KPExcWxevXqo+UiIyN5/PHH+dvf/kZkZCSvv/46L730UkVfPlMFlHiUHzdn8uHSDL5Zt4fCYg9dmsVy/6+6MbZvKxJiq+ZsaKbiak+COMkv/UDp168fe/fuZefOnWRmZtKoUSOaN2/O3Xffzdy5cwkLC2PHjh3s2bOH5s2bn/CxVJW//OUvP9vv+++/9zmvg685IE6WIMaPH390OSMjg/Hjx7Nr1y4KCwuPzu/w7bffMm3atKPlGjVqBMDw4cP5/PPP6datG0VFRfTq1esUXy1TlWzZm8+HSzOYvjyDPQcKaOgOSXH5gER6tGxgfQi1QO1JECE0btw4PvzwQ3bv3s348eOZOnUqmZmZLF26lMjISNq2bXvCeRRK/dL9vHnP9QD8bH/vkVtvv/127rnnHsaMGcOcOXN4+OGHT/jYN910E0888QRdu3at0kN+mPLlHi7ic7cJaXn6sSakv45J5JyuTe3U01rGGguDYPz48UybNo0PP/yQcePGkZubS9OmTYmMjGT27NmUHYW2POXtV968Dr7mgGjWrBl79+4lOzubgoICPv/88xMer1WrVgC8+eabR9ePHDnyuKE+SmslgwYNYvv27bzzzjtcddVV/r48JsRKPMoPmzK5/d3lnPb4t9w3fQ0HC4q5b3Q3Ft47gleuO41RPVtYcqiFLEEEQY8ePcjLy6NVq1a0aNGCa665hpSUFHr16sWUKVPo2rWrX49T3n49evQ4Oq9Dnz59uOeeewBnDojZs2fTq1cvBgwYwLp164iMjOTBBx9k4MCBjBw58oTHfvjhhxk3bhwDBgw42nwFzlzX+/fvp2fPnvTp04fZs2cf3XbFFVcwdOjQo81Opurad7CQ/87ZwllPz+a61xbz4+ZMrjqtNZ/ddgZf33UWN5/V3voXarmAzQcRbMnJyVrauVrK5hUIvgsvvJC7776bESNG/OLHsP9bYK3dmcub87fy6YqdFBR7OL1DPBMGt2FEN2tCqo1CNR+EqUVycnIYOHAgffr0qVByMIFRXOLhm3V7eON/W1m8dR8xkeFcNiCR64a0pUvz2FCHZ6ooSxBV0OrVq7n22muPW1enTh0WLVoUoohOrmHDhmzatCnUYZgy9h0s5N3F6by9cBu7co/QunEM943uxhXJrYmrGxnq8EwVV+MThKpWu9PxevXqxYoVK0IdRkjUlCbPUFuzw21GWrmTwmIPQzvG88jYngzv2pRwu6rZ+KlGJ4jo6Giys7OJj4+vdkmiNlJVsrOziY6uHXMBV7aiEg/frN3DG/N/YsnW/cREhjNuQCLXnd6Wzs2sGcmcuhqdIBITE8nIyCAzMzPUoRg/RUdHk5iYGOowqpXs/AKmLdnOWwu2sfuA04x0/6+6MW6ANSOZiqnRCSIyMvLo1b/G1CSqyuKf9jF1UTpfrdlNYYmHMzo24bGLe3KONSOZSlKjE4QxNU3uoSI+Xp7B1EXpbNmbT2x0BFcPSuKaQUl0smYkU8ksQRhTxakqK7bnMHVROp+tdK5d6NO6IU9f3puLerckJsquXTCBEdAEISKjgH8C4cArqvpUme1tgNeABGAfMEFVM9xtScArQGtAgdGqujWQ8RpTleQXFPPpih1MXZjOul0HqBsVzqX9E7lmUBI9W8WFOjxTCwQsQYhIOPA8MBLIAJaIyAxVXedV7Blgiqq+KSLDgSeB0gsApgCPq+osEakPeDCmFli7M5epi9L5dPkODhaW0K1FAx67uCdj+7YkNto6nU3wBLIGMRDYoqppACIyDRgLeCeI7sA97vJs4BO3bHcgQlVnAaiqzcFparTDhSV8vmonUxels2J7DnUiwrioT0uuHpREv9YN7TRtExKBTBCtgO1e9zOAQWXKrAQuxWmGugSIFZF4oDOQIyIfA+2Ab4FJqlrivbOITAQmAiQlJQXiORgTUJv35DF1UTofL8vgwJFiOiTU48ELu3NZ/0Q7RdWEXKg7qf8A/EdErgfmAjuAEpy4zgT6AenAe8D1wKveO6vqZGAyOIP1BStoYyriSFEJX63ZzTuL0lm8dR+R4cKoni2YMCiJge0aW23BVBmBTBA7cDqYSyW6645S1Z04NQjcfobLVDVHRDKAFV7NU58AgymTIIypTtIy83lnUTofLctg/6Ei2sbX5d4LunL5gETi69uw2qbqCWSCWAJ0EpF2OInhSuBq7wIi0gTYp6oe4F6cM5pK920oIgmqmgkMB44fy9uYaqCw2MM363YzdWE6C9KyiQgTzuvRjKsHtuH0DvGE2QVtpgoLWIJQ1WIRuQ34Guc019dUda2IPAKkqOoMYBjwpIgoThPTre6+JSLyB+A7cerbS4GXAxWrMZUtPfsQ7yxO58Ol28nKLySxUQx/PL8L45ITaRprY02Z6qFGTxhkTDAVlXj4bv0epi5K58fNWYSHCcO7NuWaQUmc2SnBhr8wVZJNGGRMAO3MOcy7i9N5b8l29uYV0CIumrvP7cz401rTPM5qC6b6sgRhzC9UUFzCi3PSeH7OFopKPJzTpSlXD0xiWJcEIsJtundT/VmCMOYX+N+WLB74ZA1pWQe5sHcL/jyqK60b1w11WMZUKksQxpyCzLwCHv9iHZ+s2Emb+LpMuXEgZ3VOCHVYxgSEJQhj/FDiUd5ZnM7TX22goMjDHSM6ccuwDkRH2kiqpuayBGHMSazZkct9n6xh5fYcTu8Qz6MX96RDQv1Qh2VMwFmCMKYc+QXF/P2bTbwx/yca14viufF9Gdu3pQ2FYWoNSxDGlKGqfLlmN3/9bC178wq4ZlASfzyvqw2eZ2odSxDGeEnPPsSDM9YwZ2Mm3Vs04MUJA+iX1CjUYRkTEpYgjMG5puHluWn8+/stRIQJD1zYneuGtLHrGUytZgnC1GolHuXHzZk8+vk6UjMPckHP5jx4UXdaxMWEOjRjQs4ShKl1PB5lWfp+Pl+1i5mrd7E3r4DERjG8fv1pnNO1aajDM6bKsARhagVVZfn2HL5wk8Ku3CNERYRxTpcELuzdkpHdm9k1DcaUYQnC1FiqyuoduXyxahefr9rFjpzDRIWHcVbnBP48qisjujUlNtrOTDKmPJYgTI2iqqzbdYAvVu3ii9W72JZ9iIgw4cxOTbhnZGfO7d6MuBhLCsb4wxKEqRE27s7j81U7+WLVLtKyDhIeJpzeIZ5bh3XkvB7NaFg3KtQhGlPtWIIw1doPmzJ5+qsNrN15gDCBIR3iuenM9ozq2ZzG9SwpGFMRliBMtbR+1wGemLmeHzdn0Sa+Lo+O7cGoni1IiK0T6tCMqTEsQZhqZXfuEf4+ayMfLM0gLiaSBy/szoTBbYiKsAvajKlsliBMtXCwoJiX5qbx8tw0SjzKTWe047ZzOtn4SMYEUEAThIiMAv4JhAOvqOpTZba3AV4DEoB9wARVzfDa3gBYB3yiqrcFMlZTNRWXePhgaQbPfrOJrPwCLuzdgj+d35WkeJu9zZhAC1iCEJFw4HlgJJABLBGRGaq6zqvYM8AUVX1TRIYDTwLXem1/FJgbqBhN1aWqzNmUyZMz17NpTz7JbRox+dcD6G8D5xkTNIGsQQwEtqhqGoCITAPG4tQISnUH7nGXZwOflG4QkQFAM+ArIDmAcZoqZu3OXJ6cuYF5W7JoG1+XFyf05/wezW0eBmOCLJAJohWw3et+BjCoTJmVwKU4zVCXALEiEg/sB54FJgDnlncAEZkITARISkqqtMBNaOzOPcIz32zko2VOB/RDF3XnmkHWAW1MqIS6k/oPwH9E5HqcpqQdQAlwCzBTVTNO9KtRVScDkwGSk5M14NGagMgvKOalH1J5+cc0PB6YeGZ7bjmno13xbEyIBTJB7ABae91PdNcdpao7cWoQiEh94DJVzRGRIcCZInILUB+IEpF8VZ0UwHhNkHk8ysfLd/DUlxvIyi9gTJ+W/PH8LrRubB3QxlQFgUwQS4BOItIOJzFcCVztXUBEmgD7VNUD3ItzRhOqeo1XmeuBZEsONcuaHbk8+OkalqXn0Ld1Q17+tc3cZkxVE7AEoarFInIb8DXOaa6vqepaEXkESFHVGcAw4EkRUZwmplsDFY+pGvYdLORvX29k2pJ04utF8bfLe3NZ/0TCwqwD2piqRlRrRtN9cnKypqSkhDoMU47iEg/vLE7n2W82kV9QzPWnt+XOczvRwIbbNiakRGSpqvo8UzTUndSmFlj80z4e/HQNG3bnMbRjPA9f1INOzWJDHZYx5iQsQZiA2Z17hCdmrmfGyp20ahjDf6/pzwU97XoGY6oLSxCm0hUUl/DavK38+/vNFHuUO4Z35P8N60hMlE3paUx1YgnCVKrZG/fyyGfr+CnrICO7N+OBX3W3cZOMqaYsQZhKsS37II9+vo5v1++lfZN6vHnjQM7unBDqsIwxFWAJwlRIYbGH/3y/mRfnphEZJtx7QVduGNrOhscwpgawBGF+sQ27D3D3eytZv+sAY/u25C+ju9GsQXSowzLGVBK/EoSIfAy8CnzpXvVsarESj/LqvDSe+XoTDWIieOXXyZzbvVmowzLGVDJ/axD/BW4A/iUiHwCvq+rGwIVlqqrt+w7x+w9WsvinfZzfoxlPXNKL+Po2D7QxNZFfCUJVvwW+FZE44Cp3eTvwMvC2qhYFMEZTBagqHyzN4JHPnOk8nhnXh8v6t7JrGoypwfzug3DnaZiAM+PbcmAqcAZwHc6YSqaGysov4N6PVzNr3R4Gt2/MM+P6kNjITl01pqbztw9iOtAFeAu4SFV3uZveExEbAKkG+2btbu79eDV5BcXc/6tu3Di0nQ2sZ0wt4W8N4l+qOtvXhvIGeTLVW96RIh79fB3vp2TQvUUD3r2yL51t/CRjahV/E0R3EVmuqjkAItIIuEpV/xu40EyoLErL5vcfrGRnzmFuPacDd47obNc1GFML+fupv7k0OQCo6n7g5sCEZELlSFEJT8xcz5UvLyQ8TPjgd0P44/ldLTkYU0v5W4MIFxFRd/IIEQkHogIXlgm2dTsPcPd7K9i4J4+rByVx3+hu1Ktj11EaU5v5+w3wFU6H9Evu/d+660w1V+JRXpqbyj9mbaJh3Shev/40zunaNNRhGWOqAH8TxJ9xksL/c+/PAl4JSEQmaH7KOsjv31/BsvQcRvdqzmMX96JxPasYGmMc/l4o5wFecG+mmlNV3l64jSdmbiAyXPjnlX0Z06elXfRmjDmOv9dBdAKeBLoDR0djU9X2AYrLBMjOnMP8+aNV/Lg5i7M6J/D0Zb1pHmcD7Bljfs7f01Nex6k9FAPnAFOAt0+2k4iMEpGNIrJFRCb52N5GRL4TkVUiMkdEEt31fUVkgYisdbeN9/8pGV9UlY+XZXD+c3NZum0/j13ckzdvOM2SgzGmXP72QcSo6nfumUzbgIdFZCnwYHk7uGc6PQ+MBDKAJSIyQ1XXeRV7Bpiiqm+KyHCcWsq1wCHg16q6WURaAktF5GvvU22N/7LyC7hv+mq+XruH5DaNePaKPrSJrxfqsKq/kiLISIFt/4PmvaDTeWDNdOZUHM6Bn36AsEjoOAIiqtbAl/4miAIRCQM2i8htwA6g/kn2GQhsUdU0ABGZBowFvBNEd+Aed3k28AmAqm4qLaCqO0VkL5AAWII4RV+t2c1901eTd6SYey/oyk1ntifchsr4ZVQhaxOkzoa0ObB1HhTmHdve/hwY9SQ07Ra8mPZvhf/9C47kQuP2EN/B+du4A9RtbAmrqvF4YPcq2DILtnwH2xeDljjbouOgxyXQ6wpIGgJhob/+yN8EcSdQF7gDeBSnmem6k+zTCtjudT8DGFSmzErgUuCfwCVArIjEq2p2aQERGYhzzUVq2QOIyERgIkBSUpKfT6V2yD1cxF8/W8vHy3bQo2UD3rm5L12a21AZpyw/00kGaW5SOLDDWd+oHfS6HDqcA0mnw5oPYc6T8MJQSL4RzvmL8wUdKHm7Ye7fYOmbEBYO9ZvC2o/Be7qWOnHQuJ1X0nATR+P2UK9JcJKHpwSKC6D4CJQUOn+LC7xuR6CkwKmNeYqd8sf9LXa+QL3vH93uta5RW+gyGupXwWluD+2D1O9hy7dOUji411nfoi+ccTd0GgmF+bDqfee29A2Iaw29xkHv8dC0a8hCF/fat/ILOE1F/6eqfzilBxa5HBilqje5968FBqnqbV5lWgL/AdoBc4HLgJ5eQ3q0AOYA16nqwhMdLzk5WVNSbNxAgHmbs/jjhyvZm1fArcM6cNvwTnY1tL+KDsO2+U5CSJ0De1Y766MbQvuznVpCh3OcL6SyDmbDnCcg5TWo08BJEsk3Qnhk5cV3aB/87zlYNBk8RdD/13DWH6FBS+cLNycdslNhX5p7c5dz0sskjwZO8ihNHFH13S/pIueLvKTIvRX6WF9mu6cIir2+/Eu8vvw9xZX33Msj4U4SkTAnWXe7CLpdCHGJgT+2Lx4P7FoOm791ago7ljqvfUwj6DAcOo50mpPq+7jeqCAfNs50EkXq987zat7bSRQ9L4MGLSo9XBFZWt6YeidNEO4DLFTVwad40CHAw6p6vnv/XgBVfbKc8vWBDapa2lHdACc5PKGqH57seJYg4FBhMU99uYEpC7bRIaEef7+iL31aNwzsQVWdN3NEHWh3VmB/NQdCcQHsXuO0A6fNhvRFzhdcWCQkDYb2w5yE0KKv80vdH3vWwlf3Oo+Z0BXOf8L5QqiIgjxY+ALM/7ez3PsKGDbJ+XL3R3GhkyS8k8a+NCeZ5KQfa+ZAIDzKvUV4LUc6r0np8tG/7vqIOl63aAj3Wo6Icv/WKbO+jte6KOeLPizCvfla9l4X4ZYPc96De9bA+s+c2163FbvVADdZjHFqUYF0MMv5QscrfLcAABtrSURBVN88C1K/g0PZzmvZsp9TQ+g4Elr19/89BJC/F9Z8DKveg53LnATY7iwnWXS7COpUTotAZSSIF3CajD4ADpauV9WPT7BPBLAJGIHTZ7EEuFpV13qVaQLsU1WPiDwOlKjqgyISBXwJfKaqz/nxHGt9gli6bT+/f38FW7MPcePQdvxpVBeiI0/hzfhLzXsOvn3IvSNOZ237s6HdMGgzBKKqUGd4cSHsXQs7l8POFc7fveudX8AATXs4yaD9ORWPXdX5Jfj1fbD/J+g8Cs57HJp0PLXHKToCKa/Cj886XzpdL4Rz7oNm3X95bGWVNu+ERTpfYNW93yJrC6yf4SSLncucdU17uMniImjW45c/x4J8yN4MmZuc/qjSW+ZGQKFuPHQY4SSFDsOdprzKek6r33eSxf6tEBEDXUc7yaLD8ArVUisjQbzuY7Wq6o0n2W808BwQDrymqo+LyCNAiqrOcJuhngQUp4npVlUtEJEJOKfWrvV6uOtVdUV5x6qtCaKoxMM/Zm3ixR9SaREXwzPj+jCkQ3xwDr5yGkz/rVP1HfQ7SPvB+dW8fZHT9BAWCYnJ0O5sJ2m0SnZ+KQZDSZHzS7I0Eexc7twvKXS2Rzd0ft217OvUDpKGQGwA5tUuLnB++c99xmlyGfRbOPtPTofkyeJfMRV+eNrp92h/Dgx/ABIHVH6MNVnOdtjwhZMwts0H1Kl1ldYsWvb/eWewqtPHk1UmCWRtPtYHBU4NpnE7aNLZeQ91Ohda9Ats57IqZCxxEsWaj+HwPicp9b0aznvsFz1khRNEdVAbE8SeA0e47Z1lLNm6nyuSE3ngwu7ERldie/eJpH4PU8c5X6wTPjr+9LzCQ7B94bGEsXMFoBBZz/llXpowmvWqnA9TSZHzC640Eexa4TQblRQ42+vEQcs+bkLo53yYG7UN7i/lvD3w/SOwfKrzgR7xAPS79udNDh6P09k8+3GnCSjxNBjxoNO0YComf69Tq1s3w3lfeoohtqXTXxHb3KtWsPn4s9OiYqFJJycRlP5N6OKcqBCsHzy+FBc6zVmr3nea/C596eT7+FBZNYifFTxZDSKYaluCmJ+axR3vLudgQQlPXdaLsX1bBe/gu1bC66OhYRu48cuT/xo+vN85JbQ0YWS5ZzHHNIZ2ZzpffjGNoPCgk1wK86HoUJnlg8duZe+XJgJwPswt+x6rGbTs53yQq8Apg4CTwL66F9IXOM1xo56Ctmc4vww3fQXfPeo0gzXr6dQYOp9f/Zt8qqLDObDpa6dmseU7KD4MDVp5JQKvW2zzqv8/UP3FMVZGgrjM6240zimpO1X1jl8UUQDUlgTh8Sgv/JDKs99spF2Terw4YQCdgjnT2/6t8MpIp8bwm2+cs2dO1YGd8NPcYwnDu9peSsKdM2ui6jp9AZHu36PL3tvqOc0GLfs5f6tKMiiPqlNLmPUQ5G53mjvydjtNB43bO30MPS6t+s+jpig67NQmKqnTt7qp9CYm96K5eap6ekWDqyy1IUHkHirinvdX8N2GvVzUpyVPXdoruHM2HMyG185zztj4zTdONbuiVJ2kU1xwLAFE1XOqzFX9V1tFFR12zkqa9w+nBnX2n5225Mo8LdaYkzhRgvil3y6dAJs0IIhWZeRwy9Rl7DlwhL+O6cGvh7QJ7uirhYfg3fGQmwHXflI5yQGcJNC4XeU8VnUTGeN0WJ9+u3PapiUGU8X4O5prHsf3QezGmSPCBJiq8s7idP46Yx1N6kfx/m+H0C+pUXCDKCmGD290xh0a/5bT0WwqT2RMqCMwxid/54OonY1zIXaosJj7p6/h4+U7OLtzAs+N70ujYE/oowpf3AObvoTRzzjt5caYWsHfGsQlwPeqmuvebwgMU9VPAhlcbZaamc8tby9j09487j63M7cP70hYKAbZ++FpWPYmnHEPDLw5+Mc3xoSMv6dJPFSaHADcsZIeOkF5UwFfrNrFmH/PIzO/gCk3DuTOczuFJjksm+KMLdTnKudcfGNMreJvJ7WvRBLE02dqh8JiD09+uZ7X/7eVfkkNef7q/rRsGKL26U1fw2d3OcMGjPl3zT+jyBjzM/5+yaeIyN9xJgACuBVYGpiQaqdduYe5deoylqXncMPQttx7QbfQjcCasRQ+uN65kOuKKXZ2jTG1lL8J4nbgAeA9nLOZZuEkCVMJ5m3O4o5pyykoKuE/V/fjwt6/4OKzypKdCu+Mg3oJcM0HUOdk80IZY2oqf89iOgj8bE5pU3HL0/dz/euLaZ9QjxcmDKBDQgi/kPP3wtuXOsvXTvc9Xr0xptbw9yymWcA4r4l8GgHTSud6ML/MwYJi7npvBc0aRPPB704nLuYUm3IK8p1x8PesdYafaNjamSSlQatTbxYqyHcG38vfC9d9Hvjx840xVZ6/TUxNSpMDgKruFxH7eVlBf/1sLen7DjHt5sEnTw6H98OuVc5AebtWOvPaZm3GxxiKgEBsCzdhuEnj6HJrZ9l73JmSInj/17B7NVz1rg0pbYwB/E8QHhFJUtV0ABFpi+9vJuOnL1fv4v2UDG4Z1oFB7cvM35C3x0kAu1YcSwg56ce2N0iEFn2ceRha9HFG/iwucAZ+y93uDIeR4y7vSIF1nx6bFKdUdBzEJTnJozAftv7onK3U2SqFxhiHvwniPmCeiPwACHAmMDFgUdVwu3OPMOnj1fROjOOu4R2caQq3Lz6WDPJ3HyvcuIMzdWLyjU4yaN4H6pUzIVB5s5V5PJC/51gCyXGTSOn9/D1w7l+d+Y2NMcblbyf1VyKSjJMUlgOfAIcDGVhN5fEov/9gBRHFh3it2waiXrjVGc1Uwp0B8Dqc4yaC3s5pptENKn7QsDBnsvMGLaD1wIo/njGmVvC3k/om4E4gEVgBDAYWAMMDF1rN9O53ixi69Xlej55N1I950HoQjHwEOp1ng7YZY6oUf5uY7gROAxaq6jki0hV4InBh1UC7V5Pz3d8Zt+lTIiIU6XwRDLkdWp8W6siMMcYnfy/VPaKqRwBEpI6qbgBOOiGAiIwSkY0iskVEfnYdhYi0EZHvRGSViMwRkUSvbdeJyGb3dp2/T6hK8Xhg0zfw5hh48QzqbJ7JR+GjOHDzYuSKKZYcjDFVmr81iAx3BNdPgFkish/YdqIdRCQcZ2iOkUAGsEREZqjqOq9izwBTVPVNERkOPAlcKyKNcQYDTMY5W2qpu+/+U3lyIVN0BFa9Bwueh6yNENuSb1rewh/S+vGfG8+hYauEUEdojDEn5W8n9SXu4sMiMhuIA746yW4DgS2qmgYgItOAsYB3gugO3OMuz8ZJQADnA7NUdZ+77yxgFPCuP/GGzMEsWPIqLHkZDmY6ncyXTGZOxFAmvrWSG4e246zOlhyMMdXDKY/Iqqo/+Fm0FbDd634GMKhMmZXApcA/gUuAWBGJL2ffVqcaa9BkbYEF/4aV06D4CHQ6H06/DdqeSdbBQv7w3Fy6No/lT6MqaZpOY4wJglAP2f0H4D8icj0wF9gBlPi7s4hMxL0eIykpKRDxndyulfDKSGe5z5Uw5Naj8zWrKn/6cBUHjhTz9k2DiI4MD02MxhjzCwQyQewAWnvdT3TXHaWqO3FqEIhIfeAyVc0RkR3AsDL7zil7AFWdDEwGSE5ODv6V3UWH4eOJENMIJs5xrjPw8vaidL7fsJcHL+xO1+aVcD2DMcYEUSAnHFgCdBKRdiISBVwJzPAuICJNRKQ0hnuB19zlr4HzRKSROzDgee66quW7RyBzA1z8/M+Sw5a9eTz+xTrO6pzA9ae3DU18xhhTAQFLEKpaDNyG88W+HnhfVdeKyCMiMsYtNgzYKCKbgGbA4+6++4BHcZLMEuCR0g7rKiN1Niz8LwycCB3PPW5TYbGHO6etoG5UBM9c3js004UaY0wFiWrNGHMvOTlZU1JSgnOww/vhv6c7k+lM/AGi6h63+ckv1/PSD2lMvnYA5/VoHpyYjDHmFxCRpaqa7GtbqDupq6cvfg8H98KVU3+WHOanZjF5bhpXDUyy5GCMqdZCNOlxNbb6Q1jzEZw9CVr1P25TzqFC7nlvJe3i6/HAhd1CFKAxxlQOq0GcitwM+OIeSBwIZ9x93CZV5b7pa8jKL2D6LUOpG2UvrTGmerMahL88Hvjk/0FJMVz6EoQfnwA+WraDL1bv4p7zOtMrMS5EQRpjTOWxn7n+WvQi/DQXLvoXNG5/3KZt2Qd56NM1DGzXmN+eZXM5G2NqBqtB+GPvevj2Yeh8wc9mXfN4lLvfW0FYmPCP8X0Jt1NajTE1hCWIkykuhI9vhjqxMOZfIMcngA2781iWnsOfzu9Cq4Y24Y8xpuawJqaTmfME7F4NV74L9Zv+bPP81CwARnRrFuzIjDEmoKwGcSLb5sO855xmpa6jfRZZmJZN2/i6tLTagzGmhrEEUZ4jB2D6b6FRGzj/SZ9Fiks8LErbx5AO8UEOzhhjAs+amMrz1b3OdQ83fOUMqeHD2p0HyCsoZkiHJkEOzhhjAs9qEL6smwEr3oYz7oGksnMcHbMgLRuAwe0bBysyY4wJGksQZeXtgc/uhBZ9YdikExZdkJpNx6b1aRobHaTgjDEmeCxBeFOFGbdB0SG4dDKER5ZbtKjEw5Kt+zjd+h+MMTWU9UF4S3kNNn8DF/zt6LSh5VmVkcOhwhKGtLcEYYypmawGUSprC3xzP3QYDqfddNLiC1Kd/odBliCMMTWUJQiAkiKYPhHCo2Ds8xB28pdlQVo23Vo0oHG9qCAEaIwxwWcJAuDHZ2HHUrjoOWjQ8qTFC4pLSNm635qXjDE1miWIrM3ww9PQezz0uMSvXZan51BQ7LEL5IwxNZp1Usd3hEtegs7n+b3LgtRswgQGtrPrH4wxNVdAaxAiMkpENorIFhH52UUFIpIkIrNFZLmIrBKR0e76SBF5U0RWi8h6Ebk3gEFC73EQ7f8kPwtSs+nZKo64mPJPgzXGmOouYAlCRMKB54ELgO7AVSLSvUyx+4H3VbUfcCXwX3f9OKCOqvYCBgC/FZG2gYr1VBwuLGH5dut/MMbUfIGsQQwEtqhqmqoWAtOAsWXKKNDAXY4DdnqtryciEUAMUAgcCGCsflu6bT9FJcpg638wxtRwgUwQrYDtXvcz3HXeHgYmiEgGMBO43V3/IXAQ2AWkA8+o6r6yBxCRiSKSIiIpmZmZlRy+b/NTs4gIE05ra/0PxpiaLdRnMV0FvKGqicBo4C0RCcOpfZQALYF2wO9FpH3ZnVV1sqomq2pyQkJCUAJekJZN78Q46tex/n1jTM0WyASxA2jtdT/RXeftN8D7AKq6AIgGmgBXA1+papGq7gX+ByQHMFa/5BcUsyoj105vNcbUCoFMEEuATiLSTkSicDqhZ5Qpkw6MABCRbjgJItNdP9xdXw8YDGwIYKx+WbJ1HyUe5XSb/8EYUwsELEGoajFwG/A1sB7nbKW1IvKIiIxxi/0euFlEVgLvAterquKc/VRfRNbiJJrXVXVVoGL114LUbKLCwxjQplGoQzHGmIALaEO6qs7E6Xz2Xveg1/I6YKiP/fJxTnWtUhakZtM3qSHRkeGhDsUYYwIu1J3U1UbuoSLW7sy16x+MMbWGJQg/LfopG49iEwQZY2oNSxB+WpCWTZ2IMPomNQx1KMYYExSWIPy0IDWb5LaNqBNh/Q/GmNrBEoQfsvML2LA7z05vNcbUKpYg/LDoJ2eUj8HWQW2MqUUsQfhhQWo2daPC6Z3o/5DgxhhT3VmC8MP81CwGtmtMZLi9XMaY2sO+8U5i74EjpGYetOsfjDG1jiWIk1iQlg1gA/QZY2odSxAnsTAtm9joCHq0tP4HY0ztYgniJOanZjOoXTzhYRLqUIwxJqgsQZzAzpzDbMs+ZM1LxphayRLECSxIdfsfrIPaGFMLWYI4gfmp2TSqG0nX5rGhDsUYY4LOEkQ5VJWFadkMbh9PmPU/GGNqIUsQ5di+7zA7cg5b/4MxptayBFGO+alZgM3/YIypvSxBlGNBWjYJsXXokFA/1KEYY0xIWILwQVVZkOr0P4hY/4MxpnYKaIIQkVEislFEtojIJB/bk0RktogsF5FVIjLaa1tvEVkgImtFZLWIRAcyVm+pmQfZm1dgzUvGmFotIlAPLCLhwPPASCADWCIiM1R1nVex+4H3VfUFEekOzATaikgE8DZwraquFJF4oChQsZZ1dPwlu/7BGFOLBbIGMRDYoqppqloITAPGlimjQAN3OQ7Y6S6fB6xS1ZUAqpqtqiUBjPU4C1OzaREXTZv4usE6pDHGVDmBTBCtgO1e9zPcdd4eBiaISAZO7eF2d31nQEXkaxFZJiJ/8nUAEZkoIikikpKZmVkpQXs8zvUPQzpY/4MxpnYLdSf1VcAbqpoIjAbeEpEwnKavM4Br3L+XiMiIsjur6mRVTVbV5ISEhEoJaNPePLIPFlrzkjGm1gtkgtgBtPa6n+iu8/Yb4H0AVV0ARANNcGobc1U1S1UP4dQu+gcw1qOOjr9kHdTGmFoukAliCdBJRNqJSBRwJTCjTJl0YASAiHTDSRCZwNdALxGp63ZYnw2sIwgWpGbTunEMiY2s/8EYU7sFLEGoajFwG86X/Xqcs5XWisgjIjLGLfZ74GYRWQm8C1yvjv3A33GSzApgmap+EahYS5W4/Q+nt28S6EMZY0yVF7DTXAFUdSZO85D3uge9ltcBQ8vZ922cU12DZv2uAxw4UmzNS8YYQ+g7qasU638wxphjLEF4mZ+aRfuEejRrELSLto0xpsqyBOEqLvGwZOt+O73VGGNcliBcq3fkkl9g/Q/GGFPKEoRrvtv/MNhqEMYYA1iCOGphWjZdmsXSpH6dUIdijDFVgiUIoLDYQ8rW/da8ZIwxXixBACszcjhcVGIJwhhjvFiCAOZvyUYEBrezBGGMMaUsQQAL0rLo3qIBcXUjQx2KMcZUGbU+QRwpKmFZeo5d/2CMMWXU+gRx4EgRF/RszvCuTUMdijHGVCkBHayvOmgaG80/r+wX6jCMMabKqfU1CGOMMb5ZgjDGGOOTJQhjjDE+WYIwxhjjkyUIY4wxPlmCMMYY45MlCGOMMT5ZgjDGGOOTqGqoY6gUIpIJbKvAQzQBsiopnECw+CrG4qsYi69iqnJ8bVQ1wdeGGpMgKkpEUlQ1OdRxlMfiqxiLr2Isvoqp6vGVx5qYjDHG+GQJwhhjjE+WII6ZHOoATsLiqxiLr2Isvoqp6vH5ZH0QxhhjfLIahDHGGJ8sQRhjjPGpViUIERklIhtFZIuITPKxvY6IvOduXyQibYMYW2sRmS0i60RkrYjc6aPMMBHJFZEV7u3BYMXnFcNWEVntHj/Fx3YRkX+5r+EqEekfxNi6eL02K0TkgIjcVaZMUF9DEXlNRPaKyBqvdY1FZJaIbHb/Nipn3+vcMptF5Logxvc3Edng/v+mi0jDcvY94XshgPE9LCI7vP6Ho8vZ94Sf9wDG955XbFtFZEU5+wb89aswVa0VNyAcSAXaA1HASqB7mTK3AC+6y1cC7wUxvhZAf3c5FtjkI75hwOchfh23Ak1OsH008CUgwGBgUQj/37txLgIK2WsInAX0B9Z4rXsamOQuTwL+z8d+jYE0928jd7lRkOI7D4hwl//PV3z+vBcCGN/DwB/8+P+f8PMeqPjKbH8WeDBUr19Fb7WpBjEQ2KKqaapaCEwDxpYpMxZ4013+EBghIhKM4FR1l6ouc5fzgPVAq2Acu5KNBaaoYyHQUERahCCOEUCqqlbk6voKU9W5wL4yq73fZ28CF/vY9XxglqruU9X9wCxgVDDiU9VvVLXYvbsQSKzs4/qrnNfPH/583ivsRPG53x1XAO9W9nGDpTYliFbAdq/7Gfz8C/hoGfcDkgvEByU6L27TVj9gkY/NQ0RkpYh8KSI9ghqYQ4FvRGSpiEz0sd2f1zkYrqT8D2aoX8NmqrrLXd4NNPNRpqq8jjfi1Ah9Odl7IZBuc5vAXiunia4qvH5nAntUdXM520P5+vmlNiWIakFE6gMfAXep6oEym5fhNJn0Af4NfBLs+IAzVLU/cAFwq4icFYIYTkhEooAxwAc+NleF1/AoddoaquS55iJyH1AMTC2nSKjeCy8AHYC+wC6cZpyq6CpOXHuo8p+l2pQgdgCtve4nuut8lhGRCCAOyA5KdM4xI3GSw1RV/bjsdlU9oKr57vJMIFJEmgQrPve4O9y/e4HpOFV5b/68zoF2AbBMVfeU3VAVXkNgT2mzm/t3r48yIX0dReR64ELgGjeJ/Ywf74WAUNU9qlqiqh7g5XKOG+rXLwK4FHivvDKhev1ORW1KEEuATiLSzv2FeSUwo0yZGUDp2SKXA9+X9+GobG575avAelX9ezllmpf2iYjIQJz/XzATWD0RiS1dxunMXFOm2Azg1+7ZTIOBXK/mlGAp95dbqF9Dl/f77DrgUx9lvgbOE5FGbhPKee66gBORUcCfgDGqeqicMv68FwIVn3ef1iXlHNefz3sgnQtsUNUMXxtD+fqdklD3kgfzhnOGzSacsxvuc9c9gvNBAIjGaZbYAiwG2gcxtjNwmhpWASvc22jgd8Dv3DK3AWtxzshYCJwe5NevvXvslW4cpa+hd4wCPO++xquB5CDHWA/nCz/Oa13IXkOcRLULKMJpB/8NTr/Wd8Bm4FugsVs2GXjFa98b3ffiFuCGIMa3Baf9vvR9WHpmX0tg5oneC0GK7y33vbUK50u/Rdn43Ps/+7wHIz53/Rul7zmvskF//Sp6s6E2jDHG+FSbmpiMMcacAksQxhhjfLIEYYwxxidLEMYYY3yyBGGMMcYnSxDGVAHuKLOfhzoOY7xZgjDGGOOTJQhjToGITBCRxe4Y/i+JSLiI5IvIP8SZx+M7EUlwy/YVkYVe8yo0ctd3FJFv3QEDl4lIB/fh64vIh+5cDFODNZKwMeWxBGGMn0SkGzAeGKqqfYES4Bqcq7dTVLUH8APwkLvLFODPqtob58rf0vVTgefVGTDwdJwrccEZwfcuoDvOlbZDA/6kjDmBiFAHYEw1MgIYACxxf9zH4Ay05+HYoGxvAx+LSBzQUFV/cNe/CXzgjr/TSlWnA6jqEQD38RarO3aPOwtZW2Be4J+WMb5ZgjDGfwK8qar3HrdS5IEy5X7p+DUFXssl2OfThJg1MRnjv++Ay0WkKRydW7oNzufocrfM1cA8Vc0F9ovIme76a4Ef1JktMENELnYfo46I1A3qszDGT/YLxRg/qeo6EbkfZxawMJwRPG8FDgID3W17cfopwBnK+0U3AaQBN7jrrwVeEpFH3McYF8SnYYzfbDRXYypIRPJVtX6o4zCmslkTkzHGGJ+sBmGMMcYnq0EYY4zxyRKEMcYYnyxBGGOM8ckShDHGGJ8sQRhjjPHp/wPCQP6qLvR+qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['accuracy'])\n",
    "plt.plot(history.history['val_accuracy'])\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train_accuracy', 'val_accuracy'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "id": "59yJsFvS6OWn"
   },
   "outputs": [],
   "source": [
    "loss, accuracy = conv1.evaluate(x_val, y_val_encoded, verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2Xf6dRhF6OWo",
    "outputId": "8c65acf5-363c-4a4b-c22c-3b9713911329"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9163333177566528\n"
     ]
    }
   ],
   "source": [
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "jYNO9PnB6OWo"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.layers import Dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "id": "iMGUZIjb6OWo"
   },
   "outputs": [],
   "source": [
    "conv2 = tf.keras.Sequential()\n",
    "conv2.add(Conv2D(10, (3, 3), activation='relu', padding='same', input_shape=(28, 28, 1)))\n",
    "conv2.add(MaxPooling2D((2, 2)))\n",
    "conv2.add(Flatten())\n",
    "conv2.add(Dropout(0.5))\n",
    "conv2.add(Dense(100, activation='relu'))\n",
    "conv2.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "aVBRIZNz6OWo",
    "outputId": "8e2fd112-8168-4b0f-998e-8168a37745c6",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 28, 28, 10)        100       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 14, 14, 10)        0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 1960)              0         \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 1960)              0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 100)               196100    \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 10)                1010      \n",
      "=================================================================\n",
      "Total params: 197,210\n",
      "Trainable params: 197,210\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "conv2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "id": "HXzsdchY6OWo"
   },
   "outputs": [],
   "source": [
    "conv2.compile(optimizer='adam', loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SJ8qDlnv6OWo",
    "outputId": "57c75b2b-b4c3-445a-e493-7db93579d692"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "1500/1500 [==============================] - 8s 5ms/step - loss: 0.5372 - accuracy: 0.8059 - val_loss: 0.3691 - val_accuracy: 0.8695\n",
      "Epoch 2/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.3853 - accuracy: 0.8607 - val_loss: 0.3214 - val_accuracy: 0.8850\n",
      "Epoch 3/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.3390 - accuracy: 0.8758 - val_loss: 0.2837 - val_accuracy: 0.8957\n",
      "Epoch 4/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.3064 - accuracy: 0.8865 - val_loss: 0.2682 - val_accuracy: 0.9050\n",
      "Epoch 5/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2885 - accuracy: 0.8927 - val_loss: 0.2631 - val_accuracy: 0.9039\n",
      "Epoch 6/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.2646 - accuracy: 0.9005 - val_loss: 0.2458 - val_accuracy: 0.9132\n",
      "Epoch 7/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.2518 - accuracy: 0.9065 - val_loss: 0.2495 - val_accuracy: 0.9120\n",
      "Epoch 8/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2426 - accuracy: 0.9086 - val_loss: 0.2488 - val_accuracy: 0.9085\n",
      "Epoch 9/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2323 - accuracy: 0.9134 - val_loss: 0.2335 - val_accuracy: 0.9156\n",
      "Epoch 10/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2247 - accuracy: 0.9155 - val_loss: 0.2332 - val_accuracy: 0.9170\n",
      "Epoch 11/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.2151 - accuracy: 0.9195 - val_loss: 0.2357 - val_accuracy: 0.9178\n",
      "Epoch 12/20\n",
      "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2115 - accuracy: 0.9205 - val_loss: 0.2319 - val_accuracy: 0.9172\n",
      "Epoch 13/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.2019 - accuracy: 0.9237 - val_loss: 0.2303 - val_accuracy: 0.9183\n",
      "Epoch 14/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1977 - accuracy: 0.9257 - val_loss: 0.2280 - val_accuracy: 0.9193\n",
      "Epoch 15/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1936 - accuracy: 0.9261 - val_loss: 0.2407 - val_accuracy: 0.9160\n",
      "Epoch 16/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1892 - accuracy: 0.9272 - val_loss: 0.2283 - val_accuracy: 0.9199\n",
      "Epoch 17/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1829 - accuracy: 0.9308 - val_loss: 0.2273 - val_accuracy: 0.9221\n",
      "Epoch 18/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1776 - accuracy: 0.9331 - val_loss: 0.2313 - val_accuracy: 0.9207\n",
      "Epoch 19/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1761 - accuracy: 0.9340 - val_loss: 0.2326 - val_accuracy: 0.9184\n",
      "Epoch 20/20\n",
      "1500/1500 [==============================] - 7s 5ms/step - loss: 0.1705 - accuracy: 0.9357 - val_loss: 0.2297 - val_accuracy: 0.9207\n"
     ]
    }
   ],
   "source": [
    "history = conv2.fit(x_train, y_train_encoded, epochs=20, \n",
    "                    validation_data=(x_val, y_val_encoded))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "id": "QomOeHer6OWo",
    "outputId": "5e1cf0a4-1772-43f9-f9f8-bca4ccf2e8c2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEHCAYAAAC0pdErAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU9dn/8fedfQ9JCEsSIGFfArK7IFVrq4Ao7qhVQW2tVh9ta61YbWupfWqXn7WtVIvWpT7WpVhbXHHfZQkUJGENyBJIIATISvb798c5CUOchCwzmZDcr+uaa+bs90yS+eSc7znfI6qKMcYY01RQoAswxhjTNVlAGGOM8coCwhhjjFcWEMYYY7yygDDGGOOVBYQxxhivQvy5chGZAfwRCAYeV9UHmkyfD/wO2OOOelhVH3en1QHr3fG7VPWClrbVu3dvTU9P913xxhjTA6xevfqAqiZ7m+a3gBCRYGAR8E0gD1glIktVdUOTWV9Q1Vu9rOKIqo5v7fbS09PJyspqf8HGGNMDicjO5qb58xDTVCBXVberajXwPDDHj9szxhjjQ/4MiFRgt8dwnjuuqUtE5AsRWSIiAzzGR4hIlogsF5ELvW1ARG5058kqLCz0YenGGGMC3Uj9CpCuquOAt4GnPaYNUtXJwFXAQyIypOnCqrpYVSer6uTkZK+H0IwxxrSTPxup9wCeewRpHG2MBkBVizwGHwd+6zFtj/u8XUQ+ACYA2/xVrDGma6qpqSEvL4/KyspAl3JCi4iIIC0tjdDQ0FYv48+AWAUME5EMnGC4AmdvoJGI9FfVfHfwAmCjOz4BqFDVKhHpDUzDIzyMMT1HXl4esbGxpKenIyKBLueEpKoUFRWRl5dHRkZGq5fzW0Coaq2I3AoswznN9QlVzRGRhUCWqi4FbhORC4Ba4CAw3118FPBXEanHOQz2gJezn4wxPUBlZaWFQweJCElJSbS1rdav10Go6uvA603G/czj9d3A3V6W+wwY68/ajDEnDguHjmvPZxjoRuqAK66o4U/vbmV9XnGgSzHGmC7Fr3sQJ4KgIHjw7S0ECYxNiw90OcYY02X0+D2I2IhQMnpHk72nJNClGGO6oMOHD/OXv/ylzcvNmjWLw4cPt3m5+fPns2TJkjYv5w89PiAAxqTEsX6PHWIyxnxVcwFRW1vb4nKvv/46vXr18ldZnaLHH2ICyEyN59Uv8jlUXk1CdFigyzHGNOMXr+SwYa9v9/ZHp8Tx8/PHNDt9wYIFbNu2jfHjxxMaGkpERAQJCQls2rSJLVu2cOGFF7J7924qKyu5/fbbufHGG4Gj/cOVlZUxc+ZMTj/9dD777DNSU1P5z3/+Q2Rk5HFre/fdd/nRj35EbW0tU6ZM4ZFHHiE8PJwFCxawdOlSQkJCOOecc/j973/PP//5T37xi18QHBxMfHw8H330UYc/GwsIIDPFaXvI2VvC6cN6B7gaY0xX8sADD5Cdnc3atWv54IMPOO+888jOzm68nuCJJ54gMTGRI0eOMGXKFC655BKSkpKOWcfWrVt57rnneOyxx7j88st56aWXuPrqq1vcbmVlJfPnz+fdd99l+PDhXHvttTzyyCNcc801vPzyy2zatAkRaTyMtXDhQpYtW0Zqamq7Dm15YwGBc4gJIHtvsQWEMV1YS//pd5apU6cec7HZn/70J15++WUAdu/ezdatW78SEBkZGYwf73ROPWnSJHbs2HHc7WzevJmMjAyGDx8OwLx581i0aBG33norERER3HDDDcyePZvZs2cDMG3aNObPn8/ll1/OxRdf7Iu3am0QAAnRYaT2iiTb2iGMMccRHR3d+PqDDz7gnXfe4fPPP2fdunVMmDDBa5cg4eHhja+Dg4OP237RkpCQEFauXMmll17Kq6++yowZMwB49NFHuf/++9m9ezeTJk2iqKjoOGtqxbY6vIZuIjM1jhwfH9s0xpz4YmNjKS0t9TqtuLiYhIQEoqKi2LRpE8uXL/fZdkeMGMGOHTvIzc1l6NChPPPMM5xxxhmUlZVRUVHBrFmzmDZtGoMHDwZg27ZtnHzyyZx88sm88cYb7N69+yt7Mm1lAeHKTIlnWc4+SitriI1ofWdWxpjuLSkpiWnTppGZmUlkZCR9+/ZtnDZjxgweffRRRo0axYgRIzjllFN8tt2IiAiefPJJLrvsssZG6ptuuomDBw8yZ84cKisrUVUefPBBAO688062bt2KqnL22Wdz0kkndbgGUdUOr6QrmDx5snbkjnLvb9rPdU+t4oUbT+HkwR1LXWOM72zcuJFRo0YFuoxuwdtnKSKr3VsrfIW1QbgyU50zmbLtMJMxxgB2iKlRcmw4fePCybGGamNMJ7jlllv49NNPjxl3++23c9111wWooq+ygPCQmRJP9l4LCGOM/y1atCjQJRyXHWLyMCY1ntz9ZRyprgt0KcYYE3AWEB4yU+KoV9hYYO0QxhhjAeGhoaHa2iGMMcYC4hj94yNIjA6zrr+NMQY/B4SIzBCRzSKSKyILvEyfLyKFIrLWfXzbY9o8EdnqPub5s06PbTImJc4aqo0x7RYTE9PstB07dpCZmdmJ1XSM3wJCRIKBRcBMYDRwpYiM9jLrC6o63n087i6bCPwcOBmYCvxcRBL8VaunzNR4tuwrparWGqqNMT2bP09znQrkqup2ABF5HpgDbGjFsucCb6vqQXfZt4EZwHN+qrVRZko8NXXK1n1ljW0Sxpgu4o0FULDet+vsNxZmPtDs5AULFjBgwABuueUWAO677z5CQkJ4//33OXToEDU1Ndx///3MmTOnTZutrKzk5ptvJisri5CQEB588EHOOusscnJyuO6666iurqa+vp6XXnqJlJQULr/8cvLy8qirq+OnP/0pc+fO7dDbbg1/HmJKBXZ7DOe545q6RES+EJElIjKgLcuKyI0ikiUiWYWFhT4pOjPV7frbGqqNMcDcuXN58cUXG4dffPFF5s2bx8svv8yaNWt4//33ueOOO2hrt0WLFi1CRFi/fj3PPfcc8+bNo7KykkcffZTbb7+dtWvXkpWVRVpaGm+++SYpKSmsW7eO7Ozsxh5c/S3QF8q9AjynqlUi8l3gaeDrrV1YVRcDi8Hpi8kXBQ1MjCI2IsTaIYzpilr4T99fJkyYwP79+9m7dy+FhYUkJCTQr18/fvCDH/DRRx8RFBTEnj172LdvH/369Wv1ej/55BP+53/+B4CRI0cyaNAgtmzZwqmnnsqvfvUr8vLyuPjiixk2bBhjx47ljjvu4K677mL27NlMnz7dX2/3GP7cg9gDDPAYTnPHNVLVIlWtcgcfBya1dll/aWyotjOZjDGuyy67jCVLlvDCCy8wd+5cnn32WQoLC1m9ejVr166lb9++Xu8D0R5XXXUVS5cuJTIyklmzZvHee+8xfPhw1qxZw9ixY7n33ntZuHChT7Z1PP4MiFXAMBHJEJEw4ApgqecMItLfY/ACYKP7ehlwjogkuI3T57jjOkVmSjwb80uoravvrE0aY7qwuXPn8vzzz7NkyRIuu+wyiouL6dOnD6Ghobz//vvs3LmzzeucPn06zz77LABbtmxh165djBgxgu3btzN48GBuu+025syZwxdffMHevXuJiori6quv5s4772TNmjW+fote+e0Qk6rWisitOF/swcATqpojIguBLFVdCtwmIhcAtcBBYL677EER+SVOyAAsbGiw7gyZqfFU1daTW1jGyH5xnbVZY0wXNWbMGEpLS0lNTaV///5861vf4vzzz2fs2LFMnjyZkSNHtnmd3/ve97j55psZO3YsISEhPPXUU4SHh/Piiy/yzDPPEBoaSr9+/fjJT37CqlWruPPOOwkKCiI0NJRHHnnED+/yq+x+EF7k7i/lGw9+xO8vO4lLJ6X5ZJ3GmPax+0H4jt0PwgcyescQFRZsZzIZY3q0QJ/F1CUFBwmj+8eRY2cyGWPaYf369VxzzTXHjAsPD2fFihUBqqh9LCCakZkaz4tZu6mvV4KCJNDlGNOjqSoiJ87f4dixY1m7dm2gyzhGe5oT7BBTM8akxFFRXceXReWBLsWYHi0iIoKioqJ2fcEZh6pSVFREREREm5azPYhmNN6jek8xQ5Kb73zLGONfaWlp5OXl4aveEnqqiIgI0tLadtKNBUQzhvaJISwkiJy9JcwZ762HEGNMZwgNDSUjIyPQZfRIdoipGaHBQYzqF2tnMhljeiwLiBaMSY0ne0+xHfs0xvRIFhAtyEyJp6SylrxDRwJdijHGdDoLiBZY19/GmJ7MAqIFw/vGEhIk1vW3MaZHsoBoQURoMMP6xlrX38aYHskC4jgyU+KsodoY0yNZQBxHZmo8ReXV7CupOv7MxhjTjVhAHIc1VBtjeioLiOMY1T8OEayh2hjT41hAHEdUWAhDkmOsodoY0+NYQLTCWPeKamOM6Un8GhAiMkNENotIrogsaGG+S0RERWSyO5wuIkdEZK37eNSfdR7PmJQ4CkoqKSy1hmpjTM/ht95cRSQYWAR8E8gDVonIUlXd0GS+WOB2oOmtlrap6nh/1dcWDV1/5+wt5swRfQJcjTHGdA5/7kFMBXJVdbuqVgPPA3O8zPdL4DdApR9r6ZDRKc6ZTDl7rR3CGNNz+DMgUoHdHsN57rhGIjIRGKCqr3lZPkNE/isiH4rIdG8bEJEbRSRLRLL8eTORuIhQ0pOirB3CGNOjBKyRWkSCgAeBO7xMzgcGquoE4IfAP0QkrulMqrpYVSer6uTk5GS/1jsmNd5OdTXG9Cj+DIg9wACP4TR3XINYIBP4QER2AKcAS0VksqpWqWoRgKquBrYBw/1Y63FlpsSz++ARiitqAlmGMcZ0Gn8GxCpgmIhkiEgYcAWwtGGiqharam9VTVfVdGA5cIGqZolIstvIjYgMBoYB2/1Y63E1XFGdY3sRxpgewm8Boaq1wK3AMmAj8KKq5ojIQhG54DiLfw34QkTWAkuAm1T1oL9qbY0xKc6ZTHaYyRjTU/jtNFcAVX0deL3JuJ81M++ZHq9fAl7yZ21tlRgdRmqvSLui2hjTY9iV1G0wJiXO9iCMMT2GBUQbZKbG8+WBcsqqagNdijHG+J0FRBtkpsahChvz7TCTMab7s4Bog8yGhmq7YM4Y0wNYQLRBn7gIkmPDraHaGNMjWEC0UWZKnF0LYYzpESwg2igzNZ6t+8uorKkLdCnGGONXFhBtlJkaT129sqmgNNClGGOMX1lAtFHDvSGsodoY091ZQLRRSnwECVGh1g5hjOn2LCDaSETITI23M5mMMd2eBUQ7jEmJZ3NBKdW19YEuxRhj/MYCoh0yU+Oorqtnyz5rqDbGdF8WEO3QcEW1tUMYY7ozC4h2GJgYRWx4iLVDGGO6NQuIdggKEkZb19/GmG7OAqKdMlPj2ZhfQm2dNVQbY7onC4h2ykyNo7Kmnu0HygNdijHG+IVfA0JEZojIZhHJFZEFLcx3iYioiEz2GHe3u9xmETnXn3W2h3X9bYzp7vwWECISDCwCZgKjgStFZLSX+WKB24EVHuNGA1cAY4AZwF/c9XUZg5NjiAgNsoZqY0y35c89iKlArqpuV9Vq4Hlgjpf5fgn8Bqj0GDcHeF5Vq1T1SyDXXV+XERwkjO5vDdXGmO7LnwGRCuz2GM5zxzUSkYnAAFV9ra3LusvfKCJZIpJVWFjom6rbIDM1ng17S6iv107ftjHG+FvAGqlFJAh4ELijvetQ1cWqOllVJycnJ/uuuFbKTImnrKqWnQcrOn3bxhjjb/4MiD3AAI/hNHdcg1ggE/hARHYApwBL3Ybq4y3bJYxJjQOsodoY0z35MyBWAcNEJENEwnAanZc2TFTVYlXtrarpqpoOLAcuUNUsd74rRCRcRDKAYcBKP9baLsP7xhIWHGTtEMaYbinEXytW1VoRuRVYBgQDT6hqjogsBLJUdWkLy+aIyIvABqAWuEVVu9w9PkODgxjZP5YcO5PJGNMN+S0gAFT1deD1JuN+1sy8ZzYZ/hXwK78V5yNjUuJ5IzsfVUVEAl2OMcb4jF1J3UGZqXEcrqhhz+EjgS7FGGN8ygKigxquqF6fZ+0QxpjuxQKig0b2jyUxOoy/frSdOrsewhjTjVhAdFB4SDA/P380a3cf5unPdgS6HGOM8RkLCB+44KQUzhyRzO+WbWa3XTRnjOkmLCB8QET41UVjEYF7/p2Nqh1qMsac+CwgfCS1VyQ/PncEH20p5OX/drmLvo0xps0sIHzomlPTmTiwFwtf3cCBsqpAl2OMMR1iAeFDwUHCA5eMo7yqloWvbAh0OcYY0yEWEAA1leCjdoPhfWO55ayhLF23l/c27fPJOo0xJhAsIA5uh4enwMZXfLbK7505lOF9Y7jn5WzKqmp9tl5jjOlMFhDxAyEyHt64Cyp90+leWEgQv754HAUllfz2zU0+WacxxnQ2C4jgEJj9RyjNh/d91zfgpEEJzDs1nWeW7yRrx0GfrdcYYzpLqwJCRG4XkThx/E1E1ojIOf4urtOkTYIp34aVi2HPGp+t9s5zR5ASH8mCf62nqrbL9VZujDEtau0exPWqWgKcAyQA1wAP+K2qQDj7pxDdB179PtT5pt0gOjyEX12USe7+Mha9v80n6zTGmM7S2oBouNHBLOAZVc3xGNc9RMTDzAcgfx2sesxnqz1zRB8umpDKIx/ksrmg1GfrNcYYf2ttQKwWkbdwAmKZiMQC9f4rK0BGXwjDzoH37odi310N/dPZo4mNCOWul76wHl+NMSeM1gbEDcACYIqqVgChwHV+qypQRGDW76G+Dt74sc9WmxgdZj2+GmNOOK0NiFOBzap6WESuBu4FjnuHHBGZISKbRSRXRBZ4mX6TiKwXkbUi8omIjHbHp4vIEXf8WhF5tC1vqkMSBsGZC2DTq7Dp9ePP30rW46sx5kTT2oB4BKgQkZOAO4BtwN9bWkBEgoFFwExgNHBlQwB4+IeqjlXV8cBvgQc9pm1T1fHu46ZW1ukbp94CfcbA63dCVZlPVmk9vhpjTjStDYhadb7R5gAPq+oiIPY4y0wFclV1u6pWA8+7yzdyz4xqEA10jW/N4FA4/yEoyYMPfu2z1VqPr8aYE0lrA6JURO7GOb31NREJwmmHaEkqsNtjOM8ddwwRuUVEtuHsQdzmMSlDRP4rIh+KyHRvGxCRG0UkS0SyCgsLW/lWWmnAVJh0HSx/xDmzyUesx1djzImitQExF6jCuR6iAEgDfueLAlR1kaoOAe7CadsAyAcGquoE4IfAP0Qkzsuyi1V1sqpOTk5O9kU5x/rGzyEqEV75vtNw7QPBQcJvrMdXY8wJoFUB4YbCs0C8iMwGKlW1xTYIYA8wwGM4zR3XnOeBC93tValqkft6NU6bx/DW1OpTkQkw4wHYuwaynvDZaodZj6/GmBNAa7vauBxYCVwGXA6sEJFLj7PYKmCYiGSISBhwBbC0yXqHeQyeB2x1xye7jdyIyGBgGLC9NbX6XOYlMPgseOcXUJLvs9Vaj6/GmK6utYeY7sG5BmKeql6L0wD905YWUNVa4FZgGbAReFFVc0RkoYhc4M52q4jkiMhanENJ89zxXwO+cMcvAW5S1cD0eCcCsx+E+hp48ytn6rab9fhqjOnqQlo5X5Cq7vcYLqIV4aKqrwOvNxn3M4/Xtzez3EvAS62szf8SB8PXfuRcYb3lLRjum34KG3p8ffrzHczM7M+pQ5J8sl5jjPGF1u5BvCkiy0RkvojMB16jyRd/t3fa7dB7BLx2B1SX+2y1d547gvSkaG76v9Vs2Wd9NRljuo7WNlLfCSwGxrmPxap6lz8L63JCwpxrI4p3wYe/8dlqo8ND+Pv1UwkPCeLav60k75BdZW2M6RpafcMgVX1JVX/oPl72Z1Fd1qDTYMI18Pki2Jfjs9UOSIzi6eunUl5dy7VPrORgebXP1m2MMe3VYkCISKmIlHh5lIqIb+7PeaL55kKna/BXbod633VoO6p/HH+bN4U9h45w3ZMrKbczm4wxAdZiQKhqrKrGeXnEqupXLlzrEaIS4dz/hbxVsOYpn656akYiD181key9Jdz0f6upru1+PaobY04cdk/q9hg3FzK+Bm/fB6W+vdDtm6P78uuLxvLx1gP86J/rqLf7RxhjAsQCoj1E4Lw/QO0RWPYTn6/+8ikD+PGMESxdt5eFr26wnl+NMQFhAdFevYfC9Dsgewnkvuvz1d98xhBuOD2Dpz7bwV8+sPtZG2M6nwVER5z+A0gaCq/9EGqO+HTVIsI9s0Zx0YRUfrdsM8+t3OXT9RtjzPFYQHRESDjMfggO7YC37j3u7G0VFCT89tJxnDE8mXteXs+b2QU+34YxxjTHAqKjMqbDqbfCqsdh7XM+X31ocBCPXD2RcWm9uO35/7J8e5HPt2GMMd5YQPjCN34B6dPh1e/79OZCDaLCQnhy/hQGJkbxnaez2LC3Z16CYozpXBYQvhAcApc+CVFJ8MLVUOH7jmcTosP4+/VTiYkI4donVrKryLrkMMb4lwWEr8Qkw+XPQGkBvPRtn92BzlNKr0ieuWEqtfX1XPPECgpL7Zalxhj/sYDwpbRJMOt3sO1deP9//bKJoX1ieWL+FPaXVDH/yZWUVtb4ZTvGGGMB4WuT5sPEa+Hj38Om1/yyiYkDE/jL1RPZXFDKjX9fTWWN7/dWjDHGAsIfZv4OUibAv74LB7b6ZRNnjejD7y87ic+3F/GDF9ZSZ11yGGN8zALCH0IjnPaIkDCn0bqqzC+buXBCKveeN4o3sgu46rHlbLUbDhljfMivASEiM0Rks4jkishXbugsIjeJyHoRWSsin4jIaI9pd7vLbRaRc/1Zp1/0GuCc2XRgC/znFvBTf0rfnj6Y31wylk0Fpcz848f8+vWN1lW4McYn/BYQIhIMLAJmAqOBKz0DwPUPVR2rquOB3wIPusuOBq4AxgAzgL+46zuxDD4DvnEfbPg3fPZnv21m7pSBvHfHGVw8MZW/frSds//fh7z2Rb518meM6RB/7kFMBXJVdbuqVgPPA3M8Z1BVzyu+ooGGb7Q5wPOqWqWqXwK57vpOPKfdBqPnwDs/h+0f+m0zSTHh/PbSk3jp5tNIjA7jln+s4donVrK90D+Ht4wx3Z8/AyIV2O0xnOeOO4aI3CIi23D2IG5r47I3ikiWiGQVFhb6rHCfEoE5iyBpGCy5Dorz/Lq5SYMSWHrrNO47fzRrdx1mxkMf8/tlmzlSbWc6GWPaJuCN1Kq6SFWHAHcBberxTlUXq+pkVZ2cnJzsnwJ9ITwWrngWaqvhhWugptKvmwsJDmL+tAze/dEZzB7Xn4ffz+UbD37IWzkFdtjJGNNq/gyIPcAAj+E0d1xzngcubOeyXV/vYXDRo7B3DbxxZ6dssk9sBA/OHc8LN55CdHgwNz6zmuufWsXOovJO2b4x5sTmz4BYBQwTkQwRCcNpdF7qOYOIDPMYPA9ouGhgKXCFiISLSAYwDFjpx1o7x6jZzk2G1vwdVj/VaZs9eXASr902nXvPG8XKLw/yzT98xEPvbLEL7IwxLfJbQKhqLXArsAzYCLyoqjkislBELnBnu1VEckRkLfBDYJ67bA7wIrABeBO4RVW7x7fZWffAkK/D63dC3upO22xocBDfnj6Yd+84k3NG9+Whd7Zyzh8+4v1N+zutBmPMiUW6yzHpyZMna1ZWVqDLaJ2Kg7D4DKdDvxs/dDr662Sf5h7gZ//JZlthOeeM7svCOZn0i4/o9DqMMYElIqtVdbK3aQFvpO6RohKdK60ripwzm+o6/8K2aUN788btX+OuGSP5eOsBZv/5E7J2+L6bcmPMicsCIlBSxsPsP8COj+Hd+wJSQlhIEDefOYSlt04jJjyYKx9bzrMrdgakFmNM12MBEUjjr4Ip33ausn7753DkcEDKGNY3lv/cejrThvbmnpezuftf66murQ9ILcaYrsMCItDO/TWMmwufPgR/HAcf/d5vnfu1JD4ylL/Nm8L3zhzCcyt3ceVjy9lf4t/rNYwxXZsFRKCFhMHFi+G7H8PAU+G9X8KfxsPnf/H7BXVNBQcJP54xkoevmsCGvSWc//AnrN0dmL0aY0zgWUB0Ff3HwVUvwA1vQ59RsOxu+PNEyHoS6jr3rnGzx6Xwr++dRlhIEJc/+jkvZu0+/kLGmG7HAqKrGTAV5r0C1y6FuBR49fvw8BT44kW/3Oe6OaP6x7H0ltOZkpHAj5d8wX1Lc6ips3YJY3oSC4iuavAZzt7ElS9AWAz86zvwyDTY+Irf7i3RVEJ0GE9fN5XvTM/gqc92cPXjKygqq+qUbRtjAs8CoisTgREz4LsfOTcfqq917lD32FmQ+06nBEVIcBD3nDeaP8w9ibW7D3PBw5+SvafY79s1xgSeBcSJICgIMi+G7y2HOX+B8iL4v0vgyVmw87NOKeGiCWksuek0VJVLHvmMf//3xO470RhzfNbVxomothrWPO2cEltWAEPOhjEXQZ/RkDwCwmP8tukDZVV879k1rPzyIN+ZnsFdM0YSEmz/Zxhzomqpqw0LiBNZdQWsety5hqKi6Oj4hHQnLPqMOvqcNMw5pdYHaurquf/VDTz9+U5OH9qbP185gYRo36zbGNO5LCC6u/o6OLQD9m90Hxucx4Gt0NAJblCIExKeodF3NPRKdw5htcOLq3Zz77+z6Rsfzv9eNJbTh/ZGRHz2towx/mcB0VPVVkFRrhMa+3KOhsdhj/6WQqOcw1LjvwWTroPgkDZtYs2uQ9zy7BryiysZ1ieGeaelc/HEVKLC2rYeY0xgWECYY1WVQeEmd09jI+xa7tzprs8YmPkbyJjeptVV1tTx6hf5PPXZl2TvKSE2IoS5kwdw7anpDEyK8tObMMb4ggWEaZkqbFwKy+6F4l0w+kI4537oNeD4yx6zGmXNrkM8+ekO3swuoE6Vs0f2Yf5pGUwbmmSHn4zpgiwgTOvUHIFP/wSf/MEZPv37MO12CI1s86oKiit5dsVO/rFiF0Xl1QzrE8O1p6Vz8YRUosPt8JMxXYUFhGmbw7vh7Z9CzssQPxDO+SWMnuNcuNdGlTV1vPZFPk99toP1e4qJjQjh8skDuPbUQQxKivZD8caYtghYQIjIDOCPQDDwuKo+0GT6D4FvA7VAIVnlqpIAABcwSURBVHC9qu50p9UB691Zd6nqBbTAAsIPvvwY3lwA+7IhfbrTPtF3TLtW5Rx+OszTn+3g9fX5jYef5p2Wbmc/GRNAAQkIEQkGtgDfBPKAVcCVqrrBY56zgBWqWiEiNwNnqupcd1qZqrb6ii8LCD+pq4U1T8F790NlsXODozPvdm6b2k77Sip5dsUu/rFiJwfKqhnaJ4ZLJqZxzpi+DEn230V+xpivClRAnArcp6rnusN3A6jqr5uZfwLwsKpOc4ctILqSioPw/q8g6wmI6AVfvxcmzYeg4Patr7aaqoM7WfnftazLXs/eg2XUEExCbDSjByQzdlAyGX0SCAoNh+AwCA51n8OPvg5peB0OoRE+fbvG9BQtBYQ/WwtTAc8bCeQBJ7cw/w3AGx7DESKShXP46QFV/XfTBUTkRuBGgIEDB3a4YNOCqEQ47/8510q8cRe89kPnXhUzfwPp0746f30dlOyBw7vg0E7n2ovG17ugdC/hWs90YDpAqLtcJbDVfbTFwNPg6/dA+ukdeJPGGE9d4nQSEbkamAyc4TF6kKruEZHBwHsisl5Vt3kup6qLgcXg7EF0WsE9Wb9MmP+q04D91k/hqVkw5mLn6uzDbhAc2umEQ32tx4Li3N+i1yDnSzxhEPQa6Az3GujsDdRVQ10NpeXlrN6+j1W5+8jetZ+62mriQpXxKVFMTI1iTN9IIoPq3PmrnUNfa/4OT50HGWc4ezcDpgbsIzKmuwj4ISYR+QbwZ+AMVd3fzLqeAl5V1SXNbc8OMQVAdQV8+kenL6jaSoju43zZJww6+sXf8Do+zQmBNqqsqePzbUW8taGAtzfs40BZNaHBwimDkzhnTD++Oaov/eIjnFN0s56ETx6E8kIYdg6cdQ+kjPfDGzem+whUG0QITiP12cAenEbqq1Q1x2OeCcASYIaqbvUYnwBUqGqViPQGPgfmeDZwN2UBEUBVpSDBEObfq6br6pW1uw/xVs4+3tqwjy8PlANw0oBezMzsx+xx/UmLVli5GD55CCoPw8jZcNZP2n32lTHdXSBPc50FPIRzmusTqvorEVkIZKnqUhF5BxgL5LuL7FLVC0TkNOCvQD3OPSseUtW/tbQtC4ieRVXZVljGspx9vJVTwLo85yZGEwf24vyTUpg9PJrknCfg80VOgGVe7Jx91XtYgCs3xyjOg/8+6+xdTrgaonsHuqIexy6UM93erqIKXl2/l1fW5bMxvwQROCUjiUtGRXNe2T+JXPM41B6BcVfAGT+GxIxAl9xzqcLOT2HFX2HTa6D1gDpnpo25CKZ8B9Imt+vCTNN2FhCmR8ndX8or6/J5Zd1eth8oJyRImDk4mP8Je41hu15A6mud/1a/dqfTNmI6R3UFrH8RViyG/TkQmQATr4XJNzhtSKseh3XPQXUZ9D/JCYqxl7arqxfTehYQpkdSVTbklzSGxZ7DR0gLOcwvE5dxRunrSJAgk66D6T+E2H6+3XhttXMTp4oDUH7AeV1+wGP4gHPr2Ap3Wli005ifMMi5R4fnWV4xfdt9z44u4dAOWPkY/PcZ54yzvmPh5Bth7GVf/fKvKoV1zzthUbjJueZmwtUw5QZIHByQ8rs7CwjT4zV09fHKur28tj6fsNI8vh/+Hy6WDyA4DOkziqCgYOfCPwlyGt1F3GF3XOO0oCbzBTn/9Xp+8VcVey9EgiAy0TnWHtUbopMgKsn5Ymy4RqSs4NhlQiIgfsDRM8KaPkcmdL3DMaqw/X1nb2HLm877Hn0BTP0uDDzl+PU2HIZa+RhsfMU5DDX0GzD1O85zey/QDKT6eqipcB7VZc4eVU0FVJc7bTAxfZ2HH28Z7I0FhDEe6uqVlV8e5JUv9pL9xRquqPk3aUFFxEcEkxAZQq/IYGLChCDUueBP650789XXOV9c6o7znBYWe/TLPqq3GwBJHkHgPkf2Ov6XW80Rp8PEwzud/74bri1peK48fOz84XHOtsJjnUdYjPMl0/ja4zk8xqm16fSIeOeq9I6qKoW1zzlnkhVthehk54r7ydc718G0R0k+rH7KeZQVOKE45QaYcE2Hunxpt7paKMlzfjYHv3Seyw9ATfmxX/o1Fe6wO772SOvWHxYDMX0gph/E9nWeY/o4e7kxfY8+Ryb6ZM/SAsKYZtTU1fPZtiI+2VrIZ9uK2JBfgipEhgYzOT2BU4ckcergJMamxhMS3EUO81QWHxsYh3c6XaFUlzlf0FWl7mt3uLVfTOHxEJXghlyS8wUUleR8CUclNhnnjm8IlQO5Tiis/QdUl0LqJGdvYcyF7br+xau6GmdvYtXjzt5FcDhkXgJTv+1sz5cqS+CQ++XvGQSHdkDx7mMvAg0Oc4IwNMo51Ts02n2Ocg4dtjg+xhlXWwVl+6C0wON5vxOIpfucz7SpoFA3SPo6jfqzfteut2oBYUwrHa6oZvn2gyzfXsRn2w6wZV8ZADHhIUzNSOS0IUmcMjiJ0f3jCArqYod1mlNXezQ8GoKj2g2ShhCpLHbbTIrgyEH39SHnuaa8+XWHxzntBMW7nC+szIudYEjz8Rd2U/tyYNXfnPaKmnIn3EIjnDAK8XyO8DLsvg71GK4sPjYIjhw8dntRSZCQ7j4yjr5OzIDY/v4/5FVd7gbHvqOhUbbvaJjEp8EFf2rXqi0gjGmnwtIqlm8v4vPtRSzfVsR29+K8XlGhnJyRyGlDenPqkCSG9Ynpvl2W11S6oXGwSYB4jOs9HCbNc/6j7UyVJc6ZUQdynav5Gx9VR59rjhw73Ph85OieQFCI086TmOElCAY5h+C6KQsIY3ykoLiSz7cf4LPcIj7bVsSew87hm9RekZw7ph8zx/Zj0sCEE2fvoqerq3UCIyQCgrtE13SdzgLCGD/ZfbCCT3MP8PaGfXy89QDVdfUkx4Zz7pi+zMzsz8kZiV2n7cIYLywgjOkEpZU1vLdpP29mF/DB5kKO1NSREBXKN0c7YXHa0CTCQ07A0zNNt2YBYUwnO1Jdx4dbCnkzO593N+6ntKqW2PAQvj6qDzMz+3HG8D5EhllYmMAL1A2DjOmxIsOCmZHZjxmZ/aiqreOz3CLeyM7n7Q37+M/avUSGBnPmiGRmZPbj6yP7EBvhg2sQjPExCwhj/Cw8JJizRvbhrJF9qK2rZ8WXB3kjO59lOft4I7uAsOAgRvSLJbVXJKkJkcc8pyVEEh8Z2n3PkDJdmh1iMiZA6uuVNbsOsSyngM37ythzqII9h49QWVN/zHzRYcFNgiPqmABJjgm3s6ZMu9khJmO6oKAgYXJ6IpPTj3YXoaocLK9mz+Ej7Dl0hD2Hj5DnPu89fIT/7j7M4YqaY9YTFhzEuLT4xkNaaQn+vXGT6TlsD8KYE0xZVS173QDJO3yE3Qcr+HjrATbmlwAwLi2emZn9mZnZj/Te0QGu1nR1dhaTMT3AjgPlvJFdwJvZ+Y132BvZL5aZmf2ZNbYfw/rGBrhC0xVZQBjTw+w5fIQ33bDI2nkIVRiSHM3MzP7MyOzHmJQ4a/g2QGDvST0D+CPOPakfV9UHmkz/IfBtoBYoBK5X1Z3utHnAve6s96vq0y1tywLCGO/2l1SyLKeAN7ILWL69iHqFgYlRzHTbLMYP6GVh0YMFJCBEJBjYAnwTyANWAVeq6gaPec4CVqhqhYjcDJypqnNFJBHIAiYDCqwGJqnqoea2ZwFhzPEdLK/m7Q1OWHyae4CaOqV/fARTMxIZlBRNelJU43NidJgFRw8QqLOYpgK5qrrdLeJ5YA7QGBCq+r7H/MuBq93X5wJvq+pBd9m3gRnAc36s15huLzE6jLlTBjJ3ykCKj9Tw7sZ9vJldwOqdh3hl3V7qPf5fjA0PYVBvJzAGJUaRnhTNoKQo0ntH0yc23MKjB/BnQKQCuz2G84CTW5j/BuCNFpZNbbqAiNwI3AgwcODAjtRqTI8THxnKxRPTuHhiGgBVtXXkHTrCzqJydhyoYNfBCnYUlbNhbwnLsguo9UiPyNBgBiVFOYGRFM3ApCgGJToB0j8+wjoo7Ca6xHUQInI1zuGkM9qynKouBhaDc4jJD6UZ02OEhwQzJDmGIclfvSdybV09ew9XsqOo3AmQogp2FpWzvbCc9zcXUl179OK+kCAhLSGSge6ex6CkKAYmRjHQfY4K6xJfO6YV/PmT2gMM8BhOc8cdQ0S+AdwDnKGqVR7Lntlk2Q/8UqUx5rhCgoOcL/ikKCD5mGn19UpBSSU7iyrYdbCcnUUV7DxYwa6iCtbuOkRJZe0x8yfHhjPIDYyGvY703tFk9I4mPtL6pOpK/NlIHYLTSH02zhf+KuAqVc3xmGcCsASYoapbPcYn4jRMT3RHrcFppG5yH8CjrJHamK7pcEW1R2gcGyAFJZXHzNs7JozBvWPI6B3N4ORo9zmGgYlRhIXYYSt/CEgjtarWisitwDKc01yfUNUcEVkIZKnqUuB3QAzwT7fBa5eqXqCqB0XklzihArCwpXAwxnRdvaLC6BUVxkkDen1lWmVNHbsPVvDlgXK+POAcsvryQDnvbtrHC1nVjfMFBwkDEiIbA6MhQAb3jqFvnDWY+4tdKGeM6ZKKj9S4wVHG9sJytjcGSNkxHRpGhQUzvG8s4wf04qQB8YxL60VGUrR1YNhKdiW1MabbaGjzcPY4ythWWM6G/BKy9xRTUV0HQGxECOPS4jkprRcnDejFSWm96BcfEeDKuybrzdUY020EBQkpvSJJ6RXJtKG9G8fX1Su5+8tYt/sw6/Kcx+KPtjeents3Lpxxab0YP6AX49LiGZfai/goaxRviQWEMaZbCA4SRvSLZUS/WC6f4pxAWVlTx4b8EtbtPswXecWs232Ytzfsa1xmcO9oxqXFk5kaz6j+cYzsF0tSTHig3kKXYwFhjOm2IkKDmTgwgYkDExrHFR+pYX1esbOXsfswn28v4t9r9zZOT44NZ2S/WPcRx8j+sQztE0N4SM+7h7gFhDGmR4mPDOX0Yb05fdjRw1MHyqrYXFDKxvwSNhWUsqmghKc/39l4AWBwkDAkOboxMBrCo398RLc+g8oCwhjT4/WOCaf30PBj2jRq6+rZUVTBpoISNuU7obF65yGWrju6txEXEcLI/nGkJUQSEx5CdHgIMe7DeR1MtPs61n1umCf4BDjLygLCGGO8CAkOYmifGIb2iWH2uKPjSypr2FJQysaCUjbll7Axv4QV2w9SXl1LWWXtMX1WtSQiNIiY8FBiwoPpGxfB6JQ4RvePY3RKHMP6xHaJCwMtIIwxpg3iIkK/ci/xBqpKVW095VW1lFfVUVZVS1lVLeVNno++dubJO1TB8yt3c6TGOU03JEgY2ifmmNAY3T+OXlFhnfpeLSCMMcZHRISI0GAiQoNJ+mqfhy2qq9fG3nM35pewIb+ET7Ye4F9rjnZhlxJ/7J7GqP5xDEiI8ttFgRYQxhjTBTgN4U5vuueflNI4vrC0io35R0Njw94S3tu0v/HeHTHhIZw5IpmHr5rYzJrbzwLCGGO6sOTYcJJjk/na8KO96FbW1LG5oJQNbnDEhPvnq9wCwhhjTjARocFOFyJeOkD0pcA3kxtjjOmSLCCMMcZ4ZQFhjDHGKwsIY4wxXllAGGOM8coCwhhjjFcWEMYYY7yygDDGGONVt7kntYgUAjs7sIrewAEfleMPVl/HWH0dY/V1TFeub5CqJnub0G0CoqNEJKu5G3d3BVZfx1h9HWP1dUxXr685dojJGGOMVxYQxhhjvLKAOGpxoAs4DquvY6y+jrH6Oqar1+eVtUEYY4zxyvYgjDHGeGUBYYwxxqseFRAiMkNENotIrogs8DI9XERecKevEJH0TqxtgIi8LyIbRCRHRG73Ms+ZIlIsImvdx886qz6PGnaIyHp3+1lepouI/Mn9DL8QEd/fB7H52kZ4fDZrRaRERL7fZJ5O/QxF5AkR2S8i2R7jEkXkbRHZ6j4nNLPsPHeerSIyrxPr+52IbHJ/fi+LiNe70hzvd8GP9d0nIns8foazmlm2xb93P9b3gkdtO0RkbTPL+v3z6zBV7REPIBjYBgwGwoB1wOgm83wPeNR9fQXwQifW1x+Y6L6OBbZ4qe9M4NUAf447gN4tTJ8FvAEIcAqwIoA/7wKci4AC9hkCXwMmAtke434LLHBfLwB+42W5RGC7+5zgvk7opPrOAULc17/xVl9rfhf8WN99wI9a8fNv8e/dX/U1mf7/gJ8F6vPr6KMn7UFMBXJVdbuqVgPPA3OazDMHeNp9vQQ4W0SkM4pT1XxVXeO+LgU2AqmdsW0fmwP8XR3LgV4i0j8AdZwNbFPVjlxd32Gq+hFwsMloz9+zp4ELvSx6LvC2qh5U1UPA28CMzqhPVd9S1Vp3cDmQ5uvttlYzn19rtObvvcNaqs/97rgceM7X2+0sPSkgUoHdHsN5fPULuHEe9w+kGEjqlOo8uIe2JgArvEw+VUTWicgbIjKmUwtzKPCWiKwWkRu9TG/N59wZrqD5P8xAf4Z9VTXffV0A9PUyT1f5HK/H2SP05ni/C/50q3sI7IlmDtF1hc9vOrBPVbc2Mz2Qn1+r9KSAOCGISAzwEvB9VS1pMnkNziGTk4A/A//u7PqA01V1IjATuEVEvhaAGlokImHABcA/vUzuCp9hI3WONXTJc81F5B6gFni2mVkC9bvwCDAEGA/k4xzG6YqupOW9hy7/t9STAmIPMMBjOM0d53UeEQkB4oGiTqnO2WYoTjg8q6r/ajpdVUtUtcx9/ToQKiK9O6s+d7t73Of9wMs4u/KeWvM5+9tMYI2q7ms6oSt8hsC+hsNu7vN+L/ME9HMUkfnAbOBbboh9RSt+F/xCVfepap2q1gOPNbPdQH9+IcDFwAvNzROoz68telJArAKGiUiG+x/mFcDSJvMsBRrOFrkUeK+5Pw5fc49X/g3YqKoPNjNPv4Y2ERGZivPz68wAixaR2IbXOI2Z2U1mWwpc657NdApQ7HE4pbM0+59boD9Dl+fv2TzgP17mWQacIyIJ7iGUc9xxficiM4AfAxeoakUz87Tmd8Ff9Xm2aV3UzHZb8/fuT98ANqlqnreJgfz82iTQreSd+cA5w2YLztkN97jjFuL8IQBE4ByWyAVWAoM7sbbTcQ41fAGsdR+zgJuAm9x5bgVycM7IWA6c1smf32B32+vcOho+Q88aBVjkfsbrgcmdXGM0zhd+vMe4gH2GOEGVD9TgHAe/Aadd611gK/AOkOjOOxl43GPZ693fxVzguk6sLxfn+H3D72HDmX0pwOst/S50Un3PuL9bX+B86fdvWp87/JW/986ozx3/VMPvnMe8nf75dfRhXW0YY4zxqicdYjLGGNMGFhDGGGO8soAwxhjjlQWEMcYYrywgjDHGeGUBYUwX4PYy+2qg6zDGkwWEMcYYrywgjGkDEblaRFa6ffj/VUSCRaRMRP4gzn083hWRZHfe8SKy3OO+Cgnu+KEi8o7bYeAaERnirj5GRJa492J4trN6EjamORYQxrSSiIwC5gLTVHU8UAd8C+fq7SxVHQN8CPzcXeTvwF2qOg7nyt+G8c8Ci9TpMPA0nCtxwenB9/vAaJwrbaf5/U0Z04KQQBdgzAnkbGASsMr95z4Sp6O9eo52yvZ/wL9EJB7opaofuuOfBv7p9r+TqqovA6hqJYC7vpXq9t3j3oUsHfjE/2/LGO8sIIxpPQGeVtW7jxkp8tMm87W3/5oqj9d12N+nCTA7xGRM670LXCoifaDx3tKDcP6OLnXnuQr4RFWLgUMiMt0dfw3woTp3C8wTkQvddYSLSFSnvgtjWsn+QzGmlVR1g4jci3MXsCCcHjxvAcqBqe60/TjtFOB05f2oGwDbgevc8dcAfxWRhe46LuvEt2FMq1lvrsZ0kIiUqWpMoOswxtfsEJMxxhivbA/CGGOMV7YHYYwxxisLCGOMMV5ZQBhjjPHKAsIYY4xXFhDGGGO8+v+Bg0UBHKpWZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['loss'])\n",
    "plt.plot(history.history['val_loss'])\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train_loss', 'val_loss'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "id": "LxbZ6Poa6OWp",
    "outputId": "da02dd8e-59fe-40f8-d4fe-b21e05dc66d6",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEHCAYAAAC0pdErAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU9fX48ffJDtkISViSgCCyI4siUJeiUBVRwR1RbLGK9edutd/SapW6VFutVlukxaVuKFIVi4paUNBaXNiXsG9CEiCBkA3IOuf3x52EIUwgIbkzWc7reeaZmbvMPTOZ3DP3s4qqYowxxlQXEuwAjDHGNE6WIIwxxvhlCcIYY4xfliCMMcb4ZQnCGGOMX5YgjDHG+BXm5ouLyCjgOSAUeElVn6y2/iTgFSAZyAUmqGqGz/o4YC3wgarecaxjJSUlaZcuXRr2DRhjTDO3dOnSvaqa7G+dawlCREKBqcD5QAawWETmqOpan82eBl5X1ddEZATwBHCDz/pHga9qc7wuXbqwZMmShgneGGNaCBH5oaZ1bhYxDQE2q+pWVS0FZgJjq23TB/jC+3iB73oROR1oD/zHxRiNMcbUwM0EkQrs9Hme4V3mayVwhffx5UCsiCSKSAjwZ+B+F+MzxhhzDMGupL4fGC4iy4HhQCZQAdwGzPWtj/BHRG4RkSUisiQnJ8f9aI0xpgVxs5I6E+jk8zzNu6yKqmbhvYIQkRjgSlXNE5EfAeeIyG1ADBAhIkWqOrna/tOB6QCDBw8+alCpsrIyMjIyKC4ubsC3ZdwUFRVFWloa4eHhwQ7FmBbPzQSxGOguIl1xEsO1wHW+G4hIEpCrqh7gNzgtmlDV6322mQgMrp4caiMjI4PY2Fi6dOmCiJzwGzGBoars27ePjIwMunbtGuxwjGnxXCtiUtVy4A7gM2AdMEtV00XkEREZ493sXGCDiGzEqZB+vCFjKC4uJjEx0ZJDEyEiJCYm2hWfMY2Eq/0gVHUuMLfasod8Hr8LvHuc13gVePVEY7Dk0LTY38uYxsPVBGGMMabhqSpZ+cVszi5ic3YRUeEhXD/0pAY/jiUIY4xppMorPPyQe7AqEWzJLmJTdhFbcoo4WFpRtd2gzm0sQTRFeXl5vPXWW9x222112m/06NG89dZbtGnTxqXIjDGNxaHSCrbkOCf+ymSwObuI7fsOUFZxuIFmh7goTmkXwzWDO9GtXQynJMdwSrsYkmIiXInLEoTL8vLyeOGFF45KEOXl5YSF1fzxz507t8Z1jcHx4jempSsuqyD3QCl7i0rYV+Tc7y0qZV9RCfsOHPk8p6iEytmfQwROSoymW3IMI3u355R2ThLolhxNbFRgm3+3mP/w33+YztqsggZ9zT4pcTx8ad9jbjN58mS2bNnCwIEDCQ8PJyoqioSEBNavX8/GjRu57LLL2LlzJ8XFxdx9993ccsstwOGxpYqKirjooos4++yzWbRoEampqfz73/+mVatWfo/34osvMn36dEpLSznllFN44403aN26NXv27OHWW29l69atAEybNo0zzzyT119/naeffhoRoX///rzxxhtMnDiRSy65hKuuugqAmJgYioqKWLhwIb/73e9qFf+nn37Kb3/7WyoqKkhKSmLevHn07NmTRYsWkZycjMfjoUePHnzzzTckJ/sdJ8yYRu1QaQWrM/NZsXM/P+w7WJUI9h0oZW9hCYUl5X73iwoPISkmksSYSFLbRNE/NZ6ObaKqEkGXxGiiwkMD/G78azEJIliefPJJ1qxZw4oVK1i4cCEXX3wxa9asqWrn/8orr9C2bVsOHTrEGWecwZVXXkliYuIRr7Fp0ybefvttXnzxRa655hree+89JkyY4Pd4V1xxBZMmTQLgwQcf5OWXX+bOO+/krrvuYvjw4cyePZuKigqKiopIT0/nscceY9GiRSQlJZGbm3vc97Ns2bLjxu/xeJg0aRJfffUVXbt2JTc3l5CQECZMmMCMGTO45557mD9/PgMGDLDkYJoEj0fZureIZTvyWLEzjxU78tiwp5AKj/Ozv210BEkxESTFRNIvNZ5En+eJMZEkxkSQFB1JUmwErSOazmm36URaT8f7pR8oQ4YMOaIT2PPPP8/s2bMB2LlzJ5s2bToqQXTt2pWBAwcCcPrpp7N9+/YaX3/NmjU8+OCD5OXlUVRUxIUXXgjAF198weuvvw5AaGgo8fHxvP7661x99dUkJSUB0LZt2waJPycnhx//+MdV21W+7s9//nPGjh3LPffcwyuvvMKNN9543OMZEwx7i0pYUZkMduaxMiOPwmLniiA2KoyBndpwW+9uDOrchgFpbUiMiQxyxO5oMQmisYiOjq56vHDhQubPn88333xD69atOffcc/12EouMPPzlCw0N5dChQzW+/sSJE/nggw8YMGAAr776KgsXLqxzjGFhYXg8HgA8Hg+lpaX1ir9Sp06daN++PV988QXff/89M2bMqHNsxjS0kvIK1mQWVCWDFTv3szPX+R8LDRF6dYhlzIAUBnZqw6DOCZycFE1ISMvor2MJwmWxsbEUFhb6XZefn09CQgKtW7dm/fr1fPvtt/U+XmFhIR07dqSsrIwZM2aQmuoMoDty5EimTZvGPffcU1XENGLECC6//HJ++ctfkpiYSG5uLm3btqVLly4sXbqUa665hjlz5lBWVlan+IcNG8Ztt93Gtm3bqoqYKq8ibr75ZiZMmMANN9xAaGjjKGc1LU9ZhYdFW/bx4cosPkvfXXV1kBIfxcDObbhh2EkM7JTAqanxtIpoud9TSxAuS0xM5KyzzqJfv360atWK9u3bV60bNWoUf//73+nduzc9e/Zk2LBh9T7eo48+ytChQ0lOTmbo0KFVyem5557jlltu4eWXXyY0NJRp06bxox/9iAceeIDhw4cTGhrKoEGDePXVV5k0aRJjx45lwIABjBo16oirBl81xZ+cnMz06dO54oor8Hg8tGvXjnnz5gEwZswYbrzxRiteMgHn8Sjfb8/lw5VZfLJmN7kHSomNCmNU3w6M7N2OQZ0TaB8XFewwGxVRPWoQ1CZp8ODBWn1GuXXr1tG7d+8gRWT8WbJkCffeey///e9/a9zG/m6moagqKzPy+XBlFh+tymJPQQmtwkP5SZ/2XNq/I8N7JhMZ1nKvEABEZKmqDva3zq4gTMA8+eSTTJs2zeoejKtUlfW7C/lwZRYfrspiZ+4hIkJDOLdnMpcOSGFk73ZNqiVRMNmn1ETdfvvt/O9//zti2d13392oi24mT57M5Ml1HrXdmFrZmlPER6t2MWdlFpuziwgNEc46JYm7RnTngr4diG9lc4zUlSWIJmrq1KnBDsGYgKvwKPuKStiVX8zugmL2FBSTlVfM15tzWJNZgAgM6dKWiZf146J+HZpt89NAsQRhjGkUissq2FNQzG7vyb/6/Z78YvYUllR1TqsUFiL0TY3nwYt7c0n/FDrEW0VzQ7EEYYw5YaXlHjZnF5FTVEJxWUXV7VBpBYfKPIefV917jtimuNy5zz1Qyv6DRzenjo4IpUN8FB3ioxjWLZGO8VF0iIuiQ3wrOsRF0T4+kqToyBbTLyHQLEEYY2rlQEk563YVkJ5VQHpWPulZBWzcU3jEaKP+RIWHEBUeSqvwUKK8t1beZfGtwokKD6VN63A6xkfRPs5JBpWPAz04nTmSJQhjzFFyD5RWJYHKhLBt74GqEUfbRkfQNyWOn5/dlb4p8aTER9EqIrQqEVQmg8iwEPt134RZgmhkKkdONSZQsguLWbEjjzVZBaz1JoVd+YeHTElt04o+KXGMGZBC35R4+qXG0SEuyqaHbQEsQRi/bL6H5ktVSc8q4PN12Xy+fg+rMvIBEIGTk6I5o0tb+qXG0Tclnj4d40iIdmcyGtP4tZwzwCeTYffqhn3NDqfCRU8ec5PJkyfTqVMnbr/9dgCmTJlCWFgYCxYsYP/+/ZSVlfHYY48xduzY4x6uqKiIsWPH+t3P37wO/uaASElJ4ZJLLmHNmjUAPP300xQVFTFlyhTOPfdcBg4cyNdff8348ePp0aMHjz32GKWlpSQmJjJjxgzat29PUVERd955J0uWLEFEePjhh8nPz2fVqlX85S9/AZx5KdauXcuzzz57wh+vaTjFZRV8s2Uf89ft4Yv12ezKL0YEBnZqw68u7Mmwk9vSu2OcdSAzR7Bvg8vGjRvHPffcU5UgZs2axWeffcZdd91FXFwce/fuZdiwYYwZM+a4l+xRUVHMnj37qP3Wrl3rd14Hf3NA7N+//5jHKC0tpXLIkv379/Ptt98iIrz00kv86U9/4s9//jOPPvoo8fHxrF69umq78PBwHn/8cZ566inCw8P55z//yT/+8Y/6fnymHnIKS1iwPpv56/bw9ea9HCytoHVEKOd0T+Le83twXs92JMe20H4CB3MhazlkLYOsFVBRBgldoG1X5z6hKyScBOH+J+ZqKVxNECIyCngOCAVeUtUnq60/CXgFSAZygQmqmiEiA4FpQBxQATyuqu/UK5jj/NJ3y6BBg8jOziYrK4ucnBwSEhLo0KED9957L1999RUhISFkZmayZ88eOnTocMzXUlV++9vfHrXfF1984XdeB39zQBwvQYwbN67qcUZGBuPGjWPXrl2UlpZWze8wf/58Zs6cWbVdQkICACNGjOCjjz6id+/elJWVceqpp9bx0zL1UTnExOfr9jB/XTYrM/JQdUYoveK0VH7Suz3DTk5sNLOVBUxJIexaCZnLDieF/dsPr088BcJawQ//g9Jq9X+xHX0SRpcjE0h0klMu14y5liBEJBSYCpwPZACLRWSOqq712exp4HVVfU1ERgBPADcAB4GfquomEUkBlorIZ6qa51a8brr66qt599132b17N+PGjWPGjBnk5OSwdOlSwsPD6dKlyzHnUah0ovv58p3rAThqf9+RW++8805++ctfMmbMGBYuXMiUKVOO+do333wzf/jDH+jVq1ejHvKjqVNVikrK2eud5zi7oITvtu3j83XZZOY58xgMSIvn3p/0YGTvdvTpGNdyKpTLip2i5MpEkLkM9m4EvM2v4jtDykA4fSKknAYdB0CrNs46VTi4z0keuduc+/3e+21fwsq3jjxWePThpJHcE1IHQ+rpENuegCjKhsylkLHEudL58f0Nfgg3ryCGAJtVdSuAiMwExgK+CaIP8Evv4wXABwCqurFyA1XNEpFsnKuMJpkgxo0bx6RJk9i7dy9ffvkls2bNol27doSHh7NgwQJ++OGHWr1Ofn6+3/1qmtfB3xwQ7du3Jzs7m3379hETE8NHH33EqFGjajxe5XwSr732WtXy888/n6lTp1bVN+zfv5+EhASGDh3Kzp07WbZsGatWrarPR9biVD/p7y10JrJ37p1lOYUl3onuSygu8xyxf1R4CGefksydI05hRK92tGspw1aXHoQNc2H7f51kkL0WPN65oKPbQepp0O9KSBnk3GKOMcWtiHNVEJ0EaX4GNy0rhrwdh5NGZSLZtxk2fnr4uPGdnESR5k0YHQdCROv6v89dK52EkLkEMpZC/g5v3KHQ/fz6vX4N3EwQqcBOn+cZwNBq26wErsAphrociBWRRFXdV7mBiAwBIoAt1Q8gIrcAtwB07ty5QYNvSH379qWwsJDU1FQ6duzI9ddfz6WXXsqpp57K4MGD6dWrV61ep6b9+vbt63deh5rmgHjooYcYMmQIqampxzz2lClTuPrqq0lISGDEiBFs27YNcOa6vv322+nXrx+hoaE8/PDDXHHFFQBcc801rFixoqrYydQs90ApHyzP5P3lGWzOLjrqpA/OOcuZ3ziS5NhIuiZFkxwbWTXfceXt5OQATXTv8UBxHhzIcW5F2XBgL8Snwcnn1v9EWBuqsONbWDED0j+A0kKIauMkgDPvcpJCymkQl9KwRUDhUZDcw7lVV3bo8Ak8Y4lzEl/7gbNOQqF9n8NXGGmDIaknhIT4P47H41z1ZC45/Hp70kErnPXxnZ33OPQW5zU7DnDtc3dtPggRuQoYpao3e5/fAAxV1Tt8tkkB/gZ0Bb4CrgT6VRYliUhHYCHwM1U95nRrNh9E43DJJZdw7733MnLkyBN+jeb8dyuv8PDlxhz+tSSDz9fvoaxC6Z8Wz9CubauSQOVJPzk2krbREYS63dGsvMR7os9xTvYHcuBA9uHHRT6PD+49/Eu5urBW0O086Dkaeow69q/1E7H/B1g5E1a+7fyKD4+GvpfBgPFw0lk1n3CDpbIIqCppLIMSp0kxEbGQOshJGKmDnURWuV3WcigpcLaLjHOSQeV2LhRhBWs+iEygk8/zNO+yKqqahXMFgYjEAFf6JIc44GPggeMlBxN8eXl5DBkyhAEDBtQrOTRXm7OL+NfSnby/LJOcwhJSWit3nh7NmB5RdGl1CCQbImOdE0JkmHMLj6jfL2BPhXNSL9wFBbuc+8LdUJjlvd8NBVlwKNf//mGtnJN8dLJzhZAy0Hkc0865j05yinFaJ0LOOlj/Mayf6xT5INBpKPQaDT0vhqRTTuw9lBTB2n87SWG7d5Kprj+G4b+G3pdCZMyJvW4gxLSDnhc5N3CuDPZt9ikmWgKL/no44YaEQfu+cOrVh680ErsHNfG5eQURBmwERuIkhsXAdaqa7rNNEpCrqh4ReRyoUNWHRCQC+AT4UFX/UpvjNacriNWrV3PDDTccsSwyMpLvvvsuSBEFVpP7u3kqnGaTB/f53PZSUpDDth072L0rEzm4j7ZSSMeIg7TRAsIqDh3/dSXUJ2nEOrcon8e+60qLjjzpF+6Goj2HiyUOvyjEtIfYDk4LnbiOENPB+VUa7XviTz6xk68q7F7lTRQfH+57lNTDubLodbHzS/hYJz2Px0kGK992kkPZQWh7Mgy4DgaMgzaNtzi5zsoOOZ+RKnTsH5Rmtce6gnB1ylERGQ38BaeZ6yuq+riIPAIsUdU53mKoJ3CaGHwF3K6qJSIyAfgnkO7zchNVdUVNx6opQfTq1avltOBoBlSV9evXN74EUVJ4ZMWkbwuXvB01FrsUaiuKQuIIjUkiPqkDkXHeX9yt23rvk5zHqs4xSgqd4oWqx77Pqy0vLoByn0QT1ebwST+24+EkUHmL6+gkgdAAdn/K2wEbPnGuLn74n/M5RbeDnqOcK4uThx8+Ke7b4iSFlTMhf6eT/PpeDgOvc65G7P/YFUFLEIHkL0Fs27aN2NhYEhMTLUk0AarKvn37KCwsrOpzETAeDxTtPvrkX/n84N4jt2+V4G0P79z2hyXxzS74bFs5GwsjKI1I4Kz+3bliyCkMSIt37/tXUeYki/BWjb9T16E82DTPubLYNN+pXA5vDd1GOEVhO78DCYGTz3OSQq+LG/97agZabIIoKysjIyOjzn0FTPBERUWRlpZGeHiAhnnOXgcf3++UCZf7fE8kxGmuWJkEfDpIedqcxA8HI1iVkceqjHxW7Mxj2Y79qMJZpyRyzeBOXNi3Q8vrkFYX5SVOMdL6ubDxM6c4a8C10H+c0/rIBEyLTRDG1KiiHL75Kyz4g1OGP2C8U85dmQziO0FoOKpKxv5DrM7MZ2VGHqsz8lmdmU9hsVOkFBkWQp+UOIb3SOaq09NISwhAM09jGlCwWjEZ0zjlbIQP/p9z1dB7DFz8DMQko6rsKShxrgwWb2VVZj6rM/KqZjoLDxV6d3SGve6fFs+pqW3o0T6GsNBG1rzSmAZiCcLUX1GOUzwT2wFCG/EMYJ4K+PYF+PxRp2PRVa+Q3/VSZi3N4Nut21mVmU9OYQkAoSFCj/axXNCnA6emxdM/LZ6eHWKJDLNiI9NyWIIw9bPmPZh9K1SUOuX2Me2dMuS4FIhLc+7jUyEu1Xkc2zE4SWTfFvjgNtj5LfQcTdY5T/DS8oPMnPUFB0srOKVdDOd0T6J/ajynprWhT8c4WkVYMjAtmyUIc2JUnU4+834HnX/kVDAWZEFBJuRnOsU4WxYcPTpmZTv8eG/CiPMmj87Djt8+/kR4PPD9dJg/BcIiyDj3WZ7aNZCPXliLAJcOSGHSOSfTJyWuYY9rTDNgCcLUnacCPp3snHj7Xg6X/d0Zp8af4nwnceRnOsmjIAsKMpz76kkkLhX6jHVesyGSRe42+Pcd8MPX5HYczsP8gg8/heiIbG48sws/P7srKW2sGaUxNbEEYeqm9CC8PwnWfwQ/ugPOf/TYJ/KoeOfW7hgd3w7mwqb/OAOvLX7JqSeoTBZ9LoO0M+qWLDweWPoK+p+HKFf4W+u7eG7bUNrFRvLrUV25bmhn4ls14roSYxoJa+Zqau/AXnhrnDOWzEV/hKG/aPhjFOfDhk8hfTZs+dyp24hNOXxlcbxkkbeD8g9uJ2z7V3wvA7jn0E20bteFW358MmMHplglszHVWD8IU3/7tsCMq5yioStfcgZKc1tlslj7AWyeXy1ZXAZpQw4nC1UKFr1M5Be/o6KigsfKJrC501X8Yng3zuvZjhC3R0Q1pomyBGHqZ+dieHucUzF93TvQaUjgYygucCZlSZ99RLLw9L6UHW3PovjrqfQq+o5vPH2Ye/IDXDnybAZ2ahP4OI1pYixBmBO37iN47yaneeqE9yCxW9BCKS6rYOOeQtZvz4KNn9B5938YVLKUSCnjoEbyRdptnHr5fZyUFBu0GI1paqwntTkx302HT/7PGZt+/MyGnwDmGIpKylm3q4D0zHzWZBWwJjOfzdlFlHucHzSxUf3ol3IWp3UI5ZywdfTsP4xL0vzM9GWMOWGWIMzRPB6Y/5DTz6HnxU6dg4tTSRYUl7E6I5813mSQnpXPtr0HqLy4TYqJoG9KPCN7t6NfSjz9UuNJS2jlM0LqGa7FZkxLZgnCHKms2BmnKP19OGOS01opxJ2WP8VlFUz/aivTFm7hUJkzsU1qm1b0TYnjsoGp9E2Jo19qPO1iI224dmOCwBKEOexgLsy8HnYscvo3nHmnK5O0qCofr97FE3PXk5l3iIv6deC6oZ3plxJPQnREgx/PGHNiLEE0dQf2Ob2aSw84g+UdMZtYinPfKuH4J/r9PzjNWPdvh6tegX5XuhLumsx8HvlwLd9vz6VXh1jenjSMH3VLdOVYxpj6sQTRlOVuhTevcoawaHuy88v/0P6jtwuLOnr6ydgO3sHzOjhNRmff6ozIesMH0OWsBg81p7CEP/9nA+8s2UlC6wj+cPmpjDujE6HWP8GYRssSRFOVuQzeusaZ4/enc6DzUGd5WTEU7vJOYL/r8K3Au2zXSqc/QdnBI18vvpPzOu16NWiYpeUeXl20jec/30xxWQU3ndWVO0d2t6EujGkCLEE0RZvmwayfQXQiTHgfkrofXhce5cyI1vYYczqrQkmBN2nsgoP74ORzITqpwUJUVT5fl81jH69l+76DjOjVjgcu7k235JgGO4Yxxl2WIJqaZa/Dh/dA+75w/bsQ277uryHiM4hew14xAGzcU8ijH63lv5v20i05mldvPINze7Zr8OMYY9zlaoIQkVHAc0Ao8JKqPllt/UnAK0AykAtMUNUM77qfAQ96N31MVV9zM9ZGTxW+/CMsfAK6jYRrXnPmUm5E8g6W8uy8jbz53Q6iI0J5+NI+TBh2EuE2JacxTZJrCUJEQoGpwPlABrBYROao6lqfzZ4GXlfV10RkBPAEcIOItAUeBgYDCiz17uunBrYFqCiHj+91rh4GXAdjnm9UU3uWV3iY8d0Onp2/kYJDZVw/9CTuPb8Hba3JqjFNmptXEEOAzaq6FUBEZgJjAd8E0Qf4pffxAuAD7+MLgXmqmuvddx4wCnjbxXgbp9ID8K+JznwJP/4VnPeAK30TTtSG3YXcPXM563cXcma3RB66tA+9OtjsbMY0B24miFRgp8/zDGBotW1WAlfgFENdDsSKSGIN+6a6F2ojVZQDb13ttDy65FkY/PNgR1RFVZm5eCdT5qQTGxXG3yeczoV921uPZ2OakWBXUt8P/E1EJgJfAZlARW13FpFbgFsAOnfu7EZ8wbNvC7x5BRTugWvfgp4XBTuiKoXFZfzm/dV8tGoXZ5+SxDPjBtAutoYpR40xTZabCSIT6OTzPM27rIqqZuFcQSAiMcCVqponIpnAudX2XVj9AKo6HZgOznDfDRh7cGUscfo4AEz8CNL8jsQbFKsy8rjjreVk5h3iVxf25P8N72aT8RjTTLnZvGQx0F1EuopIBHAtMMd3AxFJEpHKGH6D06IJ4DPgAhFJEJEE4ALvsuZvwyfw6iVOC6Wb5jWa5KCqvPTfrVw5bRHlFR7euWUYt593iiUHY5ox164gVLVcRO7AObGHAq+oarqIPAIsUdU5OFcJT4iI4hQx3e7dN1dEHsVJMgCPVFZYN2tLXoGP74OOA+C6WRDTOPoO7D9Qyv3/Wsnn67P5Se/2PH11f9q0thZKxjR3NqNcY6AKCx6Hr56C7hfA1a9CRHSwowLg+2253PX2cnIPlPKb0b2YeGYXq4g2phmxGeUas4py+PAuWDEDTvspXPwshAb/z1LhUaYu2Mxf5m+kc9vWvH/bmfRLjQ92WMaYAAr+magl81Q4k/OsngXDJ8O5kxtFH4fsgmLueWcFi7bsY+zAFB6//FRiIu2rYkxLY//1weKpgH/f7iSHkQ/BOfcFOyIAvtyYwy/fWcGB0nL+dGV/rh6cZkVKxrRQliCCweOBOXfByredntGNIDmUVXh4+j8b+MeXW+nZPpaZ1w2je/vGNdaTMSawLEEEmscDH90NK950ipWG/1+wI2Jn7kHumrmc5TvyGD+kMw9f2oeocHfmoTbGNB2WIAJJFT7+pTPo3jn3O3UOQVRW4eG1Rdt5bv4mAP46fhCXDkgJakzGmMbDEkSgqMLc+2HpP+Hse2HEg0GtkF60eS8Pz0lnU3YRw3sk8+jYfnRObB20eIwxjY8liEBQhU8nw+KX4Mw7YeTDQUsOWXmHePzjdXy8ehed2rbixZ8O5ie921lFtDHmKJYg3KYKnz0A3/0dht0O5z8alORQXFbBS//dytQFW/Cocu9PevCL4SdbXYMxpkaWINykCvN+B99OhaG3woWPByU5fLF+D7//cC0/7DvIqL4deODi3nRqa8VJxphjswThFlX4/Pew6K9wxs0w6smAJ4ftew/wyEdr+bEHQdUAABs0SURBVGJ9Nt2So3njpiGc0z05oDEYY5ouSxBuUIUvHoOvn4XTb4SLngpocjhYWs7UBZt58attRISF8MDo3vzszC5EhNnc0MaY2rME4YaFT8J/n/aOrfQMhATmxKyqzF29m8c+Xsuu/GKuGJTK5It60S7OJvMxxtSdJYiG9uWf4MsnYeAEuOS5gCWHjXsKefjf6XyzdR99Osbx1/GDGNylbUCObYxpnixBNKSvnnaG7R4wHsY8H5DkUOFR/vjpel7+ehsxkWE8elk/rhvSmVCbyMcYU0+WIBrK13+BLx6FU6+BsVMhxP3moxUe5Vf/Wsn7yzMZP6QTv7qwF22jbSIfY0zDsATREBb9FeY/DP2uhMumBSQ5eDzKr99bxfvLM7n/gh7cMaK768c0xrQs1qylvjZ/Dv95EPpcBpdPD8hkPx6P8tvZq3l3aQb3/KS7JQdjjCssQdTHoTyYcyck9YDL/x6Q5KCqPDRnDTMX7+SO807h7pGWHIwx7qhVghCR90XkYhGxhOLrs99C4W647O8Q3sr1w6kqv/9wLW9+u4Nbh3fjvgt62BhKxhjX1PaE/wJwHbBJRJ4UkZ4uxtQ0rJ/rzCN9zi8h7XTXD6eqPPbxOl5dtJ2bz+7Kr0f1tORgjHFVrRKEqs5X1euB04DtwHwRWSQiN4pIuJsBNkoH9sGHd0P7U+HH7k/4o6o8+YnTlHXimV144OLelhyMMa6rdZGRiCQCE4GbgeXAczgJY94x9hklIhtEZLOIHDU7joh0FpEFIrJcRFaJyGjv8nAReU1EVovIOhH5TR3fl7vm3geH9jv1DmHuNitVVZ76bAP/+GorNww7iYcv7WPJwRgTELWqVRWR2UBP4A3gUlXd5V31jogsqWGfUGAqcD6QASwWkTmqutZnsweBWao6TUT6AHOBLsDVQKSqnioirYG1IvK2qm6v8ztsaGveg/TZMOJ30KGf64d7dv4mXli4hfFDOvP7MX0tORhjAqa2zW6eV9UF/lao6uAa9hkCbFbVrQAiMhMYC/gmCAXivI/jgSyf5dEiEga0AkqBglrG6p7CPfDxfZB6Opx1j+uHe/7zTTz/+SauGZzG45f1I8R6RxtjAqi2RUx9RKRN5RMRSRCR246zTyqw0+d5hneZrynABBHJwLl6uNO7/F3gALAL2AE8raq51Q8gIreIyBIRWZKTk1PLt3KCVOHDu6DskNNqyeUmrS8s3Mwz8zZyxWmpPHFFf0sOxpiAq22CmKSqeZVPVHU/MKkBjj8eeFVV04DRwBveprRDgAogBegK3CciJ1ffWVWnq+pgVR2cnOzyPAcr3oKNnzrThSb3cPVQ07/awp8+3cDYgSk8ddUAG1fJGBMUtU0QoeJT+O2tXzhe7Wwm0MnneZp3ma+bgFkAqvoNEAUk4TSp/VRVy1Q1G/gfUFNRlvvydjpzSp90tjMznIte/nobf5i7nkv6d+TPV1tyMMYET20TxKc4FdIjRWQk8LZ32bEsBrqLSFcRiQCuBeZU22YHMBJARHrjJIgc7/IR3uXRwDBgfS1jbVgeD8y5AzwVcNlUV0dofW3Rdh79aC0X9evAs+MGEhZq/RKNMcFT24L0XwO/AP6f9/k84KVj7aCq5SJyB/AZEAq8oqrpIvIIsERV5wD3AS+KyL04FdMTVVVFZCrwTxFJBwT4p6ququubaxBLXoatC+GSZyGhi2uHefPbH3h4Tjrn92nP8+MHEW7JwRgTZKKqwY6hQQwePFiXLPHb4vbE5W6FaWdB52Ew4X3Xpg2d+f0OJr+/mpG92jFtwuk2NagxJmBEZGlNrVFr2w+iO/AE0AenGAgAVT2q4rjZ8FTAB7dBSDiM+ZtryWHx9lx+M3s1w3sk88KE0yw5GGMajdqejf4JTAPKgfOA14E33QqqUfj2BdjxDYz+E8RXb53bMAqLy7j3nRV0SmjN1OtPIzLM/XkkjDGmtmqbIFqp6uc4RVI/qOoU4GL3wgqy7PXw+aPQ6xLoP861wzzy4Vqy8g7xzDUDiIm0uZuMMY1Lbc9KJd7+CZu8Fc+ZQIx7YQVRRRl8cCtERDsV0y4VLX26Zjf/WprB7ed1Y3CXtq4cwxhj6qO2VxB3A62Bu4DTgQnAz9wKKqi+fhayljvJIaadK4fILizmt7NX0y81jrtHutvpzhhjTtRxryC8neLGqer9QBFwo+tRBcuuVfDlH6HfVdD3MlcOoar8+t1VHCgp5y/jBlqltDGm0Tru2UlVK4CzAxBLcJWXwOxboXUijH7KtcO89f0OFmzI4TcX9eKUdrGuHccYY+qrtnUQy0VkDvAvnEH0AFDV912JKhgWPgnZ6XDdLGjtTp3A1pwiHvtoHed0T+KnP+riyjGMMaah1DZBRAH78A5/4aVA80gQOxfD//4Cg26AHhe6cojyCg/3zlpJRFgIT101wEZnNcY0erVKEKrafOsdSg86rZbiUuHCP7h2mKkLtrByZx5/u24QHeKjjr+DMcYEWW17Uv8T54rhCKr68waPKNAO7oOIGLj4zxAVd/ztT8CKnXk8/8UmLhuYwiX9U1w5hjHGNLTaFjF95PM4Cricw7O/NW1tOsGkBa6N0nqwtJx731lB+9hIfj/W/SlKjTGmodS2iOk93+ci8jbwtSsRBYOLQ3g/MXc92/Ye4K1JQ4lvFe7acYwxpqGd6JmxO+BOL7JmZMGGbN749gduPrsrZ3ZLCnY4xhhTJ7WtgyjkyDqI3ThzRJga5B4o5f/eXUXP9rHcf2HPYIdjjDF1VtsiJuvRVQeqym/fX03ewVJeu3EIUeE2SqsxpumpVRGTiFwuIvE+z9uIiDtjUTQD7y3L5NP03dx3QU/6pLjTMsoYY9xW2zqIh1U1v/KJquYBD7sTUtO2M/cgU+akM6RLWyad03znUzLGNH+1TRD+trMJDKqp8Cj3zVoJwJ+vGUCo9ZY2xjRhtU0QS0TkGRHp5r09Ayx1M7Cm6MX/buX77blMGdOXTm1bBzscY4ypl9omiDuBUuAdYCZQDNzuVlBN0dqsAv78nw2M6tuBK09zZ4pSY4wJpNq2YjoATK7ri4vIKOA5IBR4SVWfrLa+M/Aa0Ma7zWRVnetd1x/4BxAHeIAzVLW4rjEEQnFZBfe8s5w2rSP4wxWnIi7NQmeMMYFU21ZM80Skjc/zBBH57Dj7hAJTgYuAPsB4EelTbbMHgVmqOgi4FnjBu28Y8CZwq6r2Bc4Fymr1joLg6c82sHFPEX+6qj9toyOCHY4xxjSI2hYxJXlbLgGgqvs5fk/qIcBmVd2qqqU4RVNjq22jOFcIAPEcHt/pAmCVqq70Hm+fd+KiRmdfUQkv/28b44d05rye1rncGNN81DZBeLzFQQCISBf8jO5aTSqw0+d5hneZrynABBHJAObi1HUA9ABURD4TkWUi8n/+DiAit4jIEhFZkpOTU8u30rDWZBWgCpcO6BiU4xtjjFtqmyAeAL4WkTdE5E3gS+A3DXD88cCrqpoGjAbeEJEQnLqRs4HrvfeXi8jI6jur6nRVHayqg5OTkxsgnLpLz3K6h/TtGH+cLY0xpmmpVYJQ1U+BwcAG4G3gPuDQcXbLBDr5PE/zLvN1EzDLe4xvcIYST8K52vhKVfeq6kGcq4vTahNroKVnFZCW0Ir41jZSqzGmealtJfXNwOc4ieF+4A2c4qFjWQx0F5GuIhKBUwk9p9o2O4CR3mP0xkkQOcBnwKki0tpbYT0cWFubWAMtPTOfvjachjGmGaptEdPdwBnAD6p6HjAIyDvWDqpaDtyBc7Jfh9NaKV1EHhGRMd7N7gMmichKnCuTierYDzyDk2RWAMtU9eM6vjfXFRaXsX3fQfqmWPGSMab5qe1wGcWqWiwiiEikqq4XkeOOYe3t0zC32rKHfB6vBc6qYd83cZq6NlrrdhUC0C/VriCMMc1PbRNEhrcfxAfAPBHZD/zgXlhNQ1UFtV1BGGOaodr2pL7c+3CKiCzA6bPwqWtRNRHpWQUkxUTQLjYy2KEYY0yDq/OIrKr6pRuBNEXpWQX0SYm3oTWMMc3Sic5J3eKVlFewaU+htWAyxjRbliBO0MbdRZR71BKEMabZsgRxgqyC2hjT3FmCOEHpWQXERIZxkk0MZIxppixBnKD0rHz6dIwjxKYVNcY0U5YgTkCFR1m3q5A+Vv9gjGnGLEGcgG17D3CorMIqqI0xzZoliBNgFdTGmJbAEsQJSM8qICI0hO7tY4IdijHGuMYSxAlIz8qnZ4dYwkPt4zPGNF92hqsjVSU9q8DqH4wxzZ4liDrKyi8m72CZJQhjTLNnCaKO0jOdCuo+VkFtjGnmLEHUUXpWASLQu2NssEMxxhhXWYKoo/SsfE5OiqZ1RJ1HSjfGmCbFEkQdpWcV0C/VipeMMc2fJYg6yD1Qyq78YqugNsa0CJYg6sB6UBtjWhJXE4SIjBKRDSKyWUQm+1nfWUQWiMhyEVklIqP9rC8SkfvdjLO20rMKAOwKwhjTIriWIEQkFJgKXAT0AcaLSJ9qmz0IzFLVQcC1wAvV1j8DfOJWjHWVnlVAaptWtGkdEexQjDHGdW5eQQwBNqvqVlUtBWYCY6tto0Dlz/F4IKtyhYhcBmwD0l2MsU7Ss/JtiG9jTIvhZoJIBXb6PM/wLvM1BZggIhnAXOBOABGJAX4N/N7F+OrkQEk52/YesOIlY0yLEexK6vHAq6qaBowG3hCREJzE8ayqFh1rZxG5RUSWiMiSnJwcVwNdt6sAVehnFdTGmBbCzd5emUAnn+dp3mW+bgJGAajqNyISBSQBQ4GrRORPQBvAIyLFqvo3351VdTowHWDw4MHqyrvwqqqgTrUrCGNMy+BmglgMdBeRrjiJ4Vrgumrb7ABGAq+KSG8gCshR1XMqNxCRKUBR9eQQaOlZ+bSNjqBDXFQwwzDGmIBxrYhJVcuBO4DPgHU4rZXSReQRERnj3ew+YJKIrATeBiaqqqtXAieqcohvEQl2KMYYExCuDiikqnNxKp99lz3k83gtcNZxXmOKK8HVQWm5h417Cvn52V2DHYoxxgRMsCupm4RN2YWUVaj1oDbGtCiWIGohPdOpoO5nTVyNMS2IJYhaSM/KJzoilC6J0cEOxRhjAsYSRC2kZxXQu2McISFWQW2MaTksQRyHx6Os21VgPaiNMS2OJYjj2L7vAAdKK6yC2hjT4liCOI7KHtQ2SJ8xpqWxBHEc6VkFhIcKPdrHBjsUY4wJKEsQx5GelU+P9rFEhNlHZYxpWeysdwyqWjXEhjHGtDSWII5hd0ExuQdKrYLaGNMiWYI4hsoe1HYFYYxpiSxBHEN6VgEi0LujJQhjTMtjCeIY0rPy6ZoYTXSkq4PeGmNMo2QJ4hjSswrom2r1D8aYlskSRA32HyglM++Q1T8YY1osSxA1WLvLKqiNMS2bJYgapGflA1gTV2NMi2UJogbpWQV0jI+ibXREsEMxxpigsARRA+tBbYxp6SxB+HGwtJytOUX0seIlY0wLZgnCj3W7CvGozUFtjGnZXE0QIjJKRDaIyGYRmexnfWcRWSAiy0VklYiM9i4/X0SWishq7/0IN+Osbm1lBbX1gTDGtGCudREWkVBgKnA+kAEsFpE5qrrWZ7MHgVmqOk1E+gBzgS7AXuBSVc0SkX7AZ0CqW7FWl55VQJvW4aTERwXqkMYY0+i4eQUxBNisqltVtRSYCYytto0CleU48UAWgKouV9Us7/J0oJWIRLoY6xEqK6hFJFCHNMaYRsfNBJEK7PR5nsHRVwFTgAkikoFz9XCnn9e5ElimqiXVV4jILSKyRESW5OTkNEjQZRUeNuwutP4PxpgWL9iV1OOBV1U1DRgNvCEiVTGJSF/gj8Av/O2sqtNVdbCqDk5OTm6QgDZnF1Fa4bEmrsaYFs/NBJEJdPJ5nuZd5usmYBaAqn4DRAFJACKSBswGfqqqW1yM8wjpWZVDbNgVhDGmZXMzQSwGuotIVxGJAK4F5lTbZgcwEkBEeuMkiBwRaQN8DExW1f+5GONR1mTm0yo8lK5J0YE8rDHGNDquJQhVLQfuwGmBtA6ntVK6iDwiImO8m90HTBKRlcDbwERVVe9+pwAPicgK762dW7H6WptVQO+OsYSGWAW1MaZlc3UmHFWdi1P57LvsIZ/Ha4Gz/Oz3GPCYm7H54/Eoa3cVcPmggLWoNcaYRivYldSNyo7cgxSVlFsFtTHGYAniCFZBbYwxh1mC8JGelU9YiNCjQ0ywQzHGmKCzBOEjPauA7u1jiQwLDXYoxhgTdJYgvFSV9Kx8q38wxhgvSxBe2YUl7C0qtQRhjDFeliC8bA5qY4w5kiUIr/RMpwVT746xQY7EGGMaB0sQXulZBXRJbE1sVHiwQzHGmEbBEoRX+q58m0HOGGN8WIIA8g+WsTP3kFVQG2OMD0sQOFcPYBXUxhjjyxIEzgiugF1BGGOMD0sQOBXU7eMiSYoJ2LTXxhjT6FmCAG8PaiteMsYYXy0+QRSXVbAl5wD9rHjJGGOO0OITRGFxORef2pGhJycGOxRjjGlUXJ1RrilIjo3k+fGDgh2GMcY0Oi3+CsIYY4x/liCMMcb4ZQnCGGOMX64mCBEZJSIbRGSziEz2s76ziCwQkeUiskpERvus+413vw0icqGbcRpjjDmaa5XUIhIKTAXOBzKAxSIyR1XX+mz2IDBLVaeJSB9gLtDF+/haoC+QAswXkR6qWuFWvMYYY47k5hXEEGCzqm5V1VJgJjC22jYKVHZAiAeyvI/HAjNVtURVtwGbva9njDEmQNxMEKnATp/nGd5lvqYAE0QkA+fq4c467GuMMcZFwa6kHg+8qqppwGjgDRGpdUwicouILBGRJTk5Oa4FaYwxLZGbHeUygU4+z9O8y3zdBIwCUNVvRCQKSKrlvqjqdGA6gIjkiMgP9Yg3Cdhbj/3dZvHVj8VXPxZf/TTm+E6qaYWbCWIx0F1EuuKc3K8Frqu2zQ5gJPCqiPQGooAcYA7wlog8g1NJ3R34/lgHU9Xk+gQrIktUdXB9XsNNFl/9WHz1Y/HVT2OPryauJQhVLReRO4DPgFDgFVVNF5FHgCWqOge4D3hRRO7FqbCeqKoKpIvILGAtUA7cbi2YjDEmsFwdi0lV5+JUPvsue8jn8VrgrBr2fRx43M34jDHG1CzYldSNyfRgB3AcFl/9WHz1Y/HVT2OPzy9xSnSMMcaYI9kVhDHGGL9aVIKoxdhQkSLyjnf9dyLSJYCxdfKOS7VWRNJF5G4/25wrIvkissJ7e8jfa7kc53YRWe09/hI/60VEnvd+hqtE5LQAxtbT57NZISIFInJPtW0C+hmKyCsiki0ia3yWtRWReSKyyXufUMO+P/Nus0lEfhbA+J4SkfXev99sEWlTw77H/C64GN8UEcn0+RuOrmHfY/6/uxjfOz6xbReRFTXs6/rnV2+q2iJuOC2ptgAnAxHASqBPtW1uA/7ufXwt8E4A4+sInOZ9HAts9BPfucBHQf4ctwNJx1g/GvgEEGAY8F0Q/967gZOC+RkCPwZOA9b4LPsTMNn7eDLwRz/7tQW2eu8TvI8TAhTfBUCY9/Ef/cVXm++Ci/FNAe6vxd//mP/vbsVXbf2fgYeC9fnV99aSriBqMzbUWOA17+N3gZEiIoEITlV3qeoy7+NCYB1Nc3iRscDr6vgWaCMiHYMQx0hgi6rWp/NkvanqV0ButcW+37PXgMv87HohME9Vc1V1PzAPb6dSt+NT1f+oarn36bc4HVWDoobPrzZq8/9eb8eKz3vuuAZ4u6GPGygtKUHUZnynqm28/yD5QMAnq/YWbQ0CvvOz+kcislJEPhGRvgENzKHAf0RkqYjc4md9YxlH61pq/scM9mfYXlV3eR/vBtr72aaxfI4/x7ki9Od43wU33eEtAnulhiK6xvD5nQPsUdVNNawP5udXKy0pQTQJIhIDvAfco6oF1VYvwykyGQD8Ffgg0PEBZ6vqacBFwO0i8uMgxHBMIhIBjAH+5Wd1Y/gMq6hT1tAomxKKyAM4HVVn1LBJsL4L04BuwEBgF04xTmM0nmNfPTT6/6WWlCBqM75T1TYiEoYzBPm+gETnHDMcJznMUNX3q69X1QJVLfI+nguEi0hSoOLzHjfTe58NzOboYdhrNY6Wyy4ClqnqnuorGsNnCOypLHbz3mf72Saon6OITAQuAa73JrGj1OK74ApV3aOqFarqAV6s4bjB/vzCgCuAd2raJlifX120pARRNTaU9xfmtThjPvmaA1S2FrkK+KKmf46G5i2vfBlYp6rP1LBNh8o6EREZgvP3C2QCixaR2MrHOJWZa6ptNgf4qbc10zAg36c4JVBq/OUW7M/Qy/d79jPg3362+Qy4QEQSvEUoF3iXuU5ERgH/B4xR1YM1bFOb74Jb8fnWaV1ew3Fr8//upp8A61U1w9/KYH5+dRLsWvJA3nBa2GzEad3wgHfZIzj/COAMFvgvnAmKvgdODmBsZ+MUNawCVnhvo4FbgVu929wBpOO0yPgWODPAn9/J3mOv9MZR+Rn6xig4MwluAVYDgwMcYzTOCT/eZ1nQPkOcRLULKMMpB78Jp17rc2ATMB9o6912MPCSz74/934XNwM3BjC+zTjl95Xfw8qWfSnA3GN9FwIU3xve79YqnJN+x+rxeZ8f9f8eiPi8y1+t/M75bBvwz6++N+tJbYwxxq+WVMRkjDGmDixBGGOM8csShDHGGL8sQRhjjPHLEoQxxhi/LEEY0wh4R5n9KNhxGOPLEoQxxhi/LEEYUwciMkFEvveO4f8PEQkVkSIReVaceTw+F5Fk77YDReRbn3kVErzLTxGR+d4BA5eJSDfvy8eIyLveuRhmBGokYWNqYgnCmFoSkd7AOOAsVR0IVADX4/TeXqKqfYEvgYe9u7wO/FpV++P0/K1cPgOYqs6AgWfi9MQFZwTfe4A+OD1tz3L9TRlzDGHBDsCYJmQkcDqw2PvjvhXOQHseDg/K9ibwvojEA21U9Uvv8teAf3nH30lV1dkAqloM4H2979U7do93FrIuwNfuvy1j/LMEYUztCfCaqv7miIUiv6u23YmOX1Pi87gC+/80QWZFTMbU3ufAVSLSDqrmlj4J5//oKu821wFfq2o+sF9EzvEuvwH4Up3ZAjNE5DLva0SKSOuAvgtjasl+oRhTS6q6VkQexJkFLARnBM/bgQPAEO+6bJx6CnCG8v67NwFsBW70Lr8B+IeIPOJ9jasD+DaMqTUbzdWYehKRIlWNCXYcxjQ0K2Iyxhjjl11BGGOM8cuuIIwxxvhlCcIYY4xfliCMMcb4ZQnCGGOMX5YgjDHG+GUJwhhjjF//H809Aku5HcMZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['accuracy'])\n",
    "plt.plot(history.history['val_accuracy'])\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train_accuracy', 'val_accuracy'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "id": "IzEjOY7I6OWp"
   },
   "outputs": [],
   "source": [
    "loss, accuracy = conv2.evaluate(x_val, y_val_encoded, verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ecBCf5QD6OWp",
    "outputId": "02c5777c-126f-4ec2-f6b8-1cd4b5953b67"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9206666946411133\n"
     ]
    }
   ],
   "source": [
    "print(accuracy)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "Ch08.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}