deep-learning-with-python-notebooks/6.4-sequence-processing-with-convnets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原文代码作者：François Chollet\n",
    "\n",
    "github：https://github.com/fchollet/deep-learning-with-python-notebooks\n",
    "\n",
    "中文注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：keras 2.2.1（原文是2.0.8，运行结果一致），tensorflow 1.8，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](data/gongzhong.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.1'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sequence processing with convnets\n",
    "\n",
    "# 用卷积神经网络处理序列\n",
    "\n",
    "This notebook contains the code samples found in Chapter 6, Section 4 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
    "\n",
    "\n",
    "## Implementing a 1D convnet\n",
    "\n",
    "In Keras, you would use a 1D convnet via the `Conv1D` layer, which has a very similar interface to `Conv2D`. It takes as input 3D tensors \n",
    "with shape `(samples, time, features)` and also returns similarly-shaped 3D tensors. The convolution window is a 1D window on the temporal \n",
    "axis, axis 1 in the input tensor.\n",
    "\n",
    "Let's build a simple 2-layer 1D convnet and apply it to the IMDB sentiment classification task that you are already familiar with.\n",
    "\n",
    "As a reminder, this is the code for obtaining and preprocessing the data:\n",
    "\n",
    "## 实现一维卷积神经网络\n",
    "\n",
    "Keras 中的一维卷积神经网络是 Conv1D 层，其接口类似于 Conv2D。它接收的输入是形状 为 (samples, time, features) 的三维张量，并返回类似形状的三维张量。卷积窗口是时间轴上的一维窗口（时间轴是输入张量的第二个轴）。\n",
    "\n",
    "我们来构建一个简单的两层一维卷积神经网络，并将其应用于我们熟悉的 IMDB 情感分类任务。\n",
    "\n",
    "提醒一下，获取数据并预处理的代码如下所示。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data...\n",
      "25000 train sequences\n",
      "25000 test sequences\n",
      "Pad sequences (samples x time)\n",
      "x_train shape: (25000, 500)\n",
      "x_test shape: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  \n",
    "# Number of words to consider as features（作为特征的单词个数）\n",
    "max_len = 500  \n",
    "# cut texts after this number of words (among top max_features most common words)\n",
    "# 在这么多单词之后截断文本（这些单词都属于前 max_features 个最常见的单词）\n",
    "print('Loading data...')\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "print(len(x_train), 'train sequences')\n",
    "print(len(x_test), 'test sequences')\n",
    "\n",
    "print('Pad sequences (samples x time)')\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=max_len)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=max_len)\n",
    "print('x_train shape:', x_train.shape)\n",
    "print('x_test shape:', x_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "1D convnets are structured in the same way as their 2D counter-parts that you have used in Chapter 5: they consist of a stack of `Conv1D` \n",
    "and `MaxPooling1D` layers, eventually ending in either a global pooling layer or a `Flatten` layer, turning the 3D outputs into 2D outputs, \n",
    "allowing to add one or more `Dense` layers to the model, for classification or regression.\n",
    "\n",
    "One difference, though, is the fact that we can afford to use larger convolution windows with 1D convnets. Indeed, with a 2D convolution \n",
    "layer, a 3x3 convolution window contains 3*3 = 9 feature vectors, but with a 1D convolution layer, a convolution window of size 3 would \n",
    "only contain 3 feature vectors. We can thus easily afford 1D convolution windows of size 7 or 9.\n",
    "\n",
    "This is our example 1D convnet for the IMDB dataset:\n",
    "\n",
    "一维卷积神经网络的架构与第 5 章的二维卷积神经网络相同，它是 Conv1D 层和 MaxPooling1D 层的堆叠，最后是一个全局池化层或 Flatten 层，将三维输出转换为二维输出，让你可以向模型中添加一个或多个 Dense 层，用于分类或回归。\n",
    "\n",
    "不过二者有一点不同：一维卷积神经网络可以使用更大的卷积窗口。对于二维卷积层，3×3 的卷积窗口包含 3×3=9 个特征向量；但对于一位卷积层，大小为 3 的卷积窗口只包含3个卷积向量。因此，你可以轻松使用大小等于 7 或 9 的一维卷积窗口。 \n",
    "\n",
    "用于 IMDB 数据集的一维卷积神经网络示例如下所示。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 500, 128)          1280000   \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 494, 32)           28704     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, 98, 32)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, 92, 32)            7200      \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,315,937\n",
      "Trainable params: 1,315,937\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 6s 294us/step - loss: 0.8337 - acc: 0.5090 - val_loss: 0.6875 - val_acc: 0.5638\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 2s 78us/step - loss: 0.6699 - acc: 0.6409 - val_loss: 0.6641 - val_acc: 0.6578\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 2s 81us/step - loss: 0.6234 - acc: 0.7548 - val_loss: 0.6075 - val_acc: 0.7426\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 2s 81us/step - loss: 0.5253 - acc: 0.8092 - val_loss: 0.4847 - val_acc: 0.8076\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 1s 73us/step - loss: 0.4111 - acc: 0.8491 - val_loss: 0.4237 - val_acc: 0.8318\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 2s 80us/step - loss: 0.3464 - acc: 0.8679 - val_loss: 0.4183 - val_acc: 0.8380\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 2s 81us/step - loss: 0.3124 - acc: 0.8651 - val_loss: 0.4469 - val_acc: 0.8208\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 2s 82us/step - loss: 0.2818 - acc: 0.8545 - val_loss: 0.4311 - val_acc: 0.8084\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 1s 74us/step - loss: 0.2544 - acc: 0.8355 - val_loss: 0.4300 - val_acc: 0.7932\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 2s 82us/step - loss: 0.2307 - acc: 0.8144 - val_loss: 0.4838 - val_acc: 0.7634\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128, input_length=max_len))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(5))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(lr=1e-4),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are our training and validation results: validation accuracy is somewhat lower than that of the LSTM we used two sections ago, but \n",
    "runtime is faster, both on CPU and GPU (albeit the exact speedup will vary greatly depending on your exact configuration). At that point, \n",
    "we could re-train this model for the right number of epochs (10), and run it on the test set. This is a convincing demonstration that a 1D \n",
    "convnet can offer a fast, cheap alternative to a recurrent network on a word-level sentiment classification task.\n",
    "\n",
    "这里给出了模型的训练结果和验证结果。验证精度略低于 LSTM，但在 CPU 和 GPU 上的运行速度都要更快（速度提高多少取决于具体配置，会有很大差异）。现在，你可以使用正确的轮数（10 轮）重新训练这个模型，然后在测试集上运行。这个结果可以让我们确信，在单词级的情感分类任务上，一维卷积神经网络可以替代循环网络，并且速度更快、计算代价更低。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8FdX5+PHPQ9j3XWQNKiL7dg1aEKgIolWwuLD2C0WkanFBrT8qtCKKWhXBtmhFi1WJRIpLccGFouKGEhSihILIZgQ1rKJBIfD8/jiTcHO5SW4gydzleb9e95U7M2fmPneSPHPmzJkzoqoYY4xJDBX8DsAYY0z5saRvjDEJxJK+McYkEEv6xhiTQCzpG2NMArGkb4wxCcSSfgISkSQR+UFEWpZmWT+JyGkiUur9j0XkPBHZEjS9XkTOiaTscXzW4yJy2/Gub0wkKvodgCmeiPwQNFkd+Bk47E3/TlVTS7I9VT0M1CztsolAVduWxnZEZDwwWlX7BW17fGls25iiWNKPAaqan3S9muR4VV1aWHkRqaiqueURmzHFsb/H6GLNO3FARO4SkWdFZIGI7AdGi8jZIrJCRPaKyA4R+auIVPLKVxQRFZFkb3q+t3yJiOwXkQ9FpHVJy3rLLxCRDSKyT0T+JiLvi8jYQuKOJMbfichGEdkjIn8NWjdJRGaJyC4R+RIYVMT+mSoiaSHz5ojIg9778SKyzvs+X3q18MK2lSUi/bz31UXkaS+2tUCPMJ+7ydvuWhEZ7M3vBPwdOMdrOtsZtG+nBa1/tffdd4nIiyJyciT7piT7OS8eEVkqIrtF5BsRuTXoc/7k7ZPvRSRdRJqGa0oTkffyfs/e/lzufc5uYKqItBGRt7zvstPbb3WC1m/lfcdsb/lDIlLVi7ldULmTRSRHRBoU9n1NMVTVXjH0ArYA54XMuws4CFyMO5BXA84EeuLO5k4BNgATvfIVAQWSven5wE4gAFQCngXmH0fZxsB+YIi37CbgEDC2kO8SSYz/AeoAycDuvO8OTATWAs2BBsBy9+cc9nNOAX4AagRt+zsg4E1f7JUR4FzgANDZW3YesCVoW1lAP+/9A8DbQD2gFZAZUvYK4GTvdzLSi+Ekb9l44O2QOOcD07z3A70YuwJVgYeBZZHsmxLu5zrAt8ANQBWgNpDiLfsjsAZo432HrkB94LTQfQ28l/d79r5bLnANkIT7ezwd6A9U9v5O3gceCPo+n3v7s4ZXvpe3bC4wI+hzbgZe8Pv/MJZfvgdgrxL+wgpP+suKWe8W4N/e+3CJ/B9BZQcDnx9H2XHAu0HLBNhBIUk/whjPClr+PHCL9345rpkrb9mFoYkoZNsrgJHe+wuADUWUfRn4vfe+qKS/Lfh3AVwbXDbMdj8HfuW9Ly7pPwncHbSsNu46TvPi9k0J9/NvgPRCyn2ZF2/I/EiS/qZiYrgMWOm9Pwf4BkgKU64XsBkQb3o1MLS0/68S6WXNO/Hjq+AJETlDRF7xTte/B6YDDYtY/5ug9zkUffG2sLJNg+NQ91+aVdhGIowxos8CthYRL8AzwAjv/Ugg/+K3iFwkIh95zRt7cbXsovZVnpOLikFExorIGq+JYi9wRoTbBff98renqt8De4BmQWUi+p0Vs59bABsLiaEFLvEfj9C/xyYislBEvvZi+FdIDFvUdRooQFXfx5019BaRjkBL4JXjjMlgbfrxJLS74qO4muVpqlob+DOu5l2WduBqogCIiFAwSYU6kRh34JJFnuK6lD4LnCcizXHNT894MVYDFgH34Jpe6gJvRBjHN4XFICKnAI/gmjgaeNv9X9B2i+teuh3XZJS3vVq4ZqSvI4grVFH7+Svg1ELWK2zZj15M1YPmNQkpE/r9/oLrddbJi2FsSAytRCSpkDieAkbjzkoWqurPhZQzEbCkH79qAfuAH70LYb8rh898GeguIheLSEVcO3GjMopxIXCjiDTzLur9v6IKq+q3uCaIJ4D1qvqFt6gKrp05GzgsIhfh2p4jjeE2Eakr7j6GiUHLauISXzbu+DceV9PP8y3QPPiCaogFwJUi0llEquAOSu+qaqFnTkUoaj8vBlqKyEQRqSwitUUkxVv2OHCXiJwqTlcRqY872H2D6zCQJCITCDpAFRHDj8A+EWmBa2LK8yGwC7hb3MXxaiLSK2j507jmoJG4A4A5AZb049fNwBjchdVHcTXdMuUl1mHAg7h/4lOBT3E1vNKO8RHgv8BnwEpcbb04z+Da6J8JinkvMAl4AXcx9DLcwSsSt+POOLYASwhKSKqaAfwV+NgrcwbwUdC6bwJfAN+KSHAzTd76r+GaYV7w1m8JjIowrlCF7mdV3QcMAC7FXTjeAPT1Ft8PvIjbz9/jLqpW9ZrtrgJuw13UPy3ku4VzO5CCO/gsBp4LiiEXuAhoh6v1b8P9HvKWb8H9ng+q6gcl/O4mRN7FEWNKnXe6vh24TFXf9TseE7tE5CncxeFpfscS6+zmLFOqRGQQ7nT9J1yXv1xcbdeY4+JdHxkCdPI7lnhgzTumtPUGNuFO+wcBl9iFN3O8ROQe3L0Cd6vqNr/jiQfWvGOMMQnEavrGGJNAoq5Nv2HDhpqcnOx3GMYYE1NWrVq1U1WL6iINRGHST05OJj093e8wjDEmpohIcXelA9a8Y4wxCcWSvjHGJBBL+sYYk0Cirk0/nEOHDpGVlcVPP/3kdyimCFWrVqV58+ZUqlTYcDLGGL/FRNLPysqiVq1aJCcn4wZuNNFGVdm1axdZWVm0bt26+BWMMb6Iieadn376iQYNGljCj2IiQoMGDexsrAipqZCcDBUquJ+pJXqcvTGlIyZq+oAl/Bhgv6PCpabChAmQk+Omt2510wCjjnfsTGOOQ0zU9I2JdVOmHE34eXJy3PzyZmccic2SfgR27dpF165d6dq1K02aNKFZs2b50wcPHoxoG7/97W9Zv359kWXmzJlDqv0HxqVthQwVVtj8spJ3xrF1K6gePeOwP7vEEXUDrgUCAQ29I3fdunW0a9cu4m2kproa1LZt0LIlzJhReqfQ06ZNo2bNmtxyyy0F5uc/dLhCYh9HS/q7ShTJyS7BhmrVCrZsSbw4TOkTkVWqGiiuXNxlqPKsyWzcuJGOHTty9dVX0717d3bs2MGECRMIBAJ06NCB6dOn55ft3bs3q1evJjc3l7p16zJ58mS6dOnC2WefzXfffQfA1KlTmT17dn75yZMnk5KSQtu2bfngA/fAoB9//JFLL72ULl26MGLECAKBAKtXrz4mtttvv50zzzwzP768g/uGDRs499xz6dKlC927d2eL959+991306lTJ7p06cIUP9oc4tyMGVC9esF51au7+eUpWs44jH/iLumXd9tpZmYmV155JZ9++inNmjXj3nvvJT09nTVr1vDmm2+SmZl5zDr79u2jb9++rFmzhrPPPpt58+aF3baq8vHHH3P//ffnH0D+9re/0aRJE9asWcPkyZP59NNPw657ww03sHLlSj777DP27dvHa6+9BsCIESOYNGkSa9as4YMPPqBx48a89NJLLFmyhI8//pg1a9Zw8803l9LeMXlGjYK5c12NWsT9nDu3/C/itizk8fGFzTfxJ+6SfnnXZE499VTOPPPM/OkFCxbQvXt3unfvzrp168Im/WrVqnHBBRcA0KNHj/zadqihQ4ceU+a9995j+PDhAHTp0oUOHTqEXfe///0vKSkpdOnShXfeeYe1a9eyZ88edu7cycUXXwy4m6mqV6/O0qVLGTduHNWqVQOgfv36Jd8RplijRrkmlCNH3E8/eu1EyxmH8U/cJf3yrsnUqFEj//0XX3zBQw89xLJly8jIyGDQoEFh+61Xrlw5/31SUhK5ublht12lSpVjykRyDSYnJ4eJEyfywgsvkJGRwbhx4/LjCNetUlWtu2WCiJYzDutB5J+4S/p+1mS+//57atWqRe3atdmxYwevv/56qX9G7969WbhwIQCfffZZ2DOJAwcOUKFCBRo2bMj+/ft57rnnAKhXrx4NGzbkpZdeAtxNbzk5OQwcOJB//vOfHDhwAIDdu3eXetwmevh9xmE9iPwVUdIXkUEisl5ENorI5DDLW4rIWyLyqYhkiMiF3vxkETkgIqu91z9K+wuE8rMm0717d9q3b0/Hjh256qqr6NWrV6l/xnXXXcfXX39N586dmTlzJh07dqROnToFyjRo0IAxY8bQsWNHfv3rX9OzZ8/8ZampqcycOZPOnTvTu3dvsrOzueiiixg0aBCBQICuXbsya9asUo/bmDzRdM9CQsrraljYC0gCvgROASrjHlLcPqTMXOAa7317YIv3Phn4vLjPCH716NFDQ2VmZh4zL1EdOnRIDxw4oKqqGzZs0OTkZD106JDPUR0Vjb+r+fNVW7VSFXE/58/3O6LEJqLq6vgFXyJ+RxbbgHSNIMdGMgxDCrBRVTcBiEgaMAQIbldQoLb3vg6w/fgPQ6YoP/zwA/379yc3NxdV5dFHH6VixZgZTaPc2fAH0adly/D3ClgPovIRSbZoBnwVNJ0F9AwpMw14Q0SuA2oA5wUtay0inwLfA1NV9d3QDxCRCcAEgJb2my9S3bp1WbVqld9hxIyimhIs6ftjxoyCB2KwHkTlKZI2/XDdOkK7kIwA/qWqzYELgadFpAKwA2ipqt2Am4BnRKR2yLqo6lxVDahqoFGjYp/ra0zE7Gak6BMtPYgSVSQ1/SygRdB0c45tvrkSGASgqh+KSFWgoap+B/zszV8lIl8CpwP25HNTLqwpITqNGmVJ3i+R1PRXAm1EpLWIVAaGA4tDymwD+gOISDugKpAtIo1EJMmbfwrQBthUWsEbUxy7GcmYgopN+qqaC0wEXgfWAQtVda2ITBeRwV6xm4GrRGQNsAAY611N7gNkePMXAVerqnUCN+XGmhKMKSiifvqq+qqqnq6qp6rqDG/en1V1sfc+U1V7qWoXVe2qqm94859T1Q7e/O6q+lLZfZWy069fv2NutJo9ezbXXnttkevVrFkTgO3bt3PZZZcVuu3QUUVDzZ49m5ygq14XXnghe/fujSR0g/83IwWLskFtE14i3hlsff0iMGLECNLS0jj//PPz56WlpXH//fdHtH7Tpk1ZtGjRcX/+7NmzGT16NNW9dopXX331uLdlysaRI7BzJ2zfXvC1Y0fB6W+/hTZtYNAgOP986NsXvCGPTDlL2O68kXTmL89XNN6ctXPnTm3YsKH+9NNPqqq6efNmbdGihR45ckT379+v5557rnbr1k07duyoL774Yv56NWrUyC/foUMHVVXNycnRYcOGaadOnfSKK67QlJQUXblypaqqXn311dqjRw9t3769/vnPf1ZV1YceekgrVaqkHTt21H79+qmqaqtWrTQ7O1tVVWfOnKkdOnTQDh066KxZs/I/74wzztDx48dr+/btdcCAAZqTk3PM91q8eLGmpKRo165dtX///vrNN9+oqur+/ft17Nix2rFjR+3UqZMuWrRIVVWXLFmi3bp1086dO+u5554bdl/5/bsqbUeOqGZnq65Zo7pkieq8eap33aV67bWql1yimpKi2ry5asWK4W84athQtXNn1UGDVMeNU731VtXzz1etUsUtr1rVTc+apZqZ6T7PlI9WrcL/zlq18juy40Mp3pwVVW68EcIMH39CunYFbxj7sBo0aEBKSgqvvfYaQ4YMIS0tjWHDhiEiVK1alRdeeIHatWuzc+dOzjrrLAYPHlzoAGaPPPII1atXJyMjg4yMDLp3756/bMaMGdSvX5/Dhw/Tv39/MjIyuP7663nwwQd56623aNiwYYFtrVq1iieeeIKPPvoIVaVnz5707duXevXq8cUXX7BgwQIee+wxrrjiCp577jlGjx5dYP3evXuzYsUKRITHH3+c++67j5kzZ3LnnXdSp04dPvvsMwD27NlDdnY2V111FcuXL6d169YxPz6PKuzde2zNPLR2vmMHhHs4Wv360LSpe7Vrd/R93uvkk6FJE/DGzDtGTg4sXw6vvw6vvQaTJrn5LVsePQvo3x9CRtgwpShRu/PGXNL3S14TT17SzxsDX1W57bbbWL58ORUqVODrr7/m22+/pUmTJmG3s3z5cq6//noAOnfuTOfOnfOXLVy4kLlz55Kbm8uOHTvIzMwssDzUe++9x69//ev8kT6HDh3Ku+++y+DBg2ndujVdu3YFCh++OSsri2HDhrFjxw4OHjxI69atAVi6dClpaWn55erVq8dLL71Enz598svE0vDLWVnw5JOwZk3BZB5mAFTq1nUJu2lT6NPn2ESe97Nq1ROLqXp1l9wHDYJZs1zTQt4BYMECd7E5KQl+8YujB4Fu3VzbsykdidqdN+aSflE18rJ0ySWXcNNNN/HJJ59w4MCB/Bp6amoq2dnZrFq1ikqVKpGcnBx2OOVg4c4CNm/ezAMPPMDKlSupV68eY8eOLXY7WsRVwSpBVcykpKT8ETSDXXfdddx0000MHjyYt99+m2nTpuVvNzTGcPOiWW4uvPIKPPYYLFni2txPPx2aNXOJNDSR570P7d5ZXlq1cu3JEybAoUPw4YdHDwJTprhXo0Yu+Z9/PgwcCI0b+xNrvEjUO4Ot3hChmjVr0q9fP8aNG8eIESPy5+/bt4/GjRtTqVIl3nrrLbaGqzoE6dOnT/7Dzz///HMyMjIANyxzjRo1qFOnDt9++y1LlizJX6dWrVrs378/7LZefPFFcnJy+PHHH3nhhRc455xzIv5O+/bto1mzZgA8+eST+fMHDhzI3//+9/zpPXv2cPbZZ/POO++wefNmIHqHX960ySXIli3hkkvgk09g8mT48ktYvx6WLYP58+G++1xT4bBhcM45cOqp/iX8UJUqubOMGTNg1Sr45ht4+mmX6F97DX7zGzjpJAgE3Hd99113oDAlk6jdeS3pl8CIESNYs2ZN/pOrAEaNGkV6ejqBQIDU1FTOOOOMIrdxzTXX8MMPP9C5c2fuu+8+UlJSAPcUrG7dutGhQwfGjRtXYFjmCRMmcMEFF/DLX/6ywLa6d+/O2LFjSUlJoWfPnowfP55u3bpF/H2mTZvG5ZdfzjnnnFPgesHUqVPZs2cPHTt2pEuXLrz11ls0atSIuXPnMnToULp06cKwYcMi/pyy9vPPsHAhDBjgkve990L37vDii659dsYMOOUUv6M8fiedBKNHu4PVt99CejrcdZfr9fOXv7gDRMOGcOmlLmkVU+8wQaKpO295kaKaCPwQCAQ0tN/6unXraNeunU8RmZIoz9/V//7nmm+eesp1l2zZEq68EsaNg+bNyyUE3+3bB//9rzsDeO01+MobGrFdO9cMNGiQOyhYt9D4JyKrVDVQXLmYa9M3iS0nBxYtcsn+vfegYkUYMgSuugrOO89d/EwkderA0KHupeoOhK+95q4HPPKIuwZWtaq7HyDvwnHbtq45wyQmS/omJqxZ4xL9/PmudtumjWvaGDPGNX8Yl8jbtXOvSZPCdwudNMmdEZ1/vrugHQjAGWe4g6dJDDHzq4613iOJqLSbCvfvh7Q0l+xXrnR93i+91NXq+/a12mpxiuoW+uyzbr+Ca/rp1s0dAHr0cD/btk28s6ZEERNt+ps3b6ZWrVo0aNDAEn+UUlV27drF/v378/vyH9924OOPXUJKS4Mff4SOHV2iHz3a3RRlTtyRI/DFF+6icN7r00/d/gaoUePogSDvYHD66XafQFlJTXU9sbZtc2diM2aU/KJypG36MZH0Dx06RFZWVrH91o2/qlatSvPmzalUqVKJ192zxzXdPPYYfPaZq6UOH+6Sfc+eVqsvD4cPu26t6emuq2jegSDvFo9atVyvqLyzgUDA9ZayA8GJCR0DCNzff0m7j8ZV0jfxSdX1MX/sMXdx9qefXEK56ioYMQJqH/OMNVPecnPdxeHgM4I1a47ezVy7dsGDQI8ernusHaQjl5wcvpttq1auG2mkLOkb3xV2yvrdd25YhMcfhw0bXOIYPRrGj3dNCia6HToEmZkFzwjWrDk6RlG9ei75Bx8M8m6AMseqUCH8kNsirhkuUpb0ja/CnbJWqQJdurgmg0OHoFcvV6u//PLouRvWHJ+DB+Hzz48eBNLTXTNd3p3C9esXPBsIBKBFCzsQgNX0LenHicL+kCtUgBtucLX69u3LPSxTjn7+2SX+vIPAqlXuwJCb65Y3auSSf58+rjdWIOCGoEg01qZvST8uFHbKCvb0qER24ABkZBw9I1ixAtatc8tq1HD3DvTtC/36wZlnQuXKvoZbbqKu946IDAIeApKAx1X13pDlLYEngbpemcmq+qq37I/AlcBh4HpVLfjcwRCW9ONDixZuSONQJT1lNfHvu+/cTWTvvANvv+3OBsDdP5B3EOjb1/XiKuz5BKYUh2EQkSRgDjAAyAJWishiVc0MKjYV98D0R0SkPfAqkOy9Hw50AJoCS0XkdFU9XPKvZGLF/v3h/zkTYdhaU3KNG8Nll7kXuHGU3n3XHQDeeQduv92dHVatCmeddfRM4KyzTvy5Bokokh62KcBGVd2kqgeBNGBISBkF8jrY1QG2e++HAGmq+rOqbgY2etszceqnn9xYOFu2uKGLE23YWnPiGjaEX/8aHnrIPSVv5043Yuo118D338P06fDLX7pxh/r0gT//2Q06F9wmbgoXyTAMzYCvgqazgJ4hZaYBb4jIdUAN4LygdVeErNvsuCI1Ue/QIdcT5+233ciXo0e72/+NORH167uKxBCvqrl3rxtsL685aMYMuPNOdxE4JeXomcAvfuGuE5iCIkn64TpVhV4IGAH8S1VnisjZwNMi0jHCdRGRCcAEgJbx/qyyOHX4MPzf/8HLL7vRHUMex2tMqalbFy66yL3A1f7zDgLvvOMG4rv7bjeIXCDgDgB9+7ouwrVq+Rp6VIgk6WcBLYKmm3O0+SbPlcAgAFX9UESqAg0jXBdVnQvMBXchN9LgTXRQhauvdmPl3Hefe29MealdGy680L3AXVP64IOjZwIPPOAerJOU5O4RyLsw3Lt3Yj54vtjeOyJSEdgA9Ae+BlYCI1V1bVCZJcCzqvovEWkH/BfXjNMeeAbXjt/Um9+mqAu51nsntqjCzTe7ZpwpU9wTnYyJJj/+6J45nHdh+KOPXFNkhQruDvB+/dxQ0+edF9s3i5V2l80Lgdm47pjzVHWGiEwH0lV1sddL5zGgJq755lZVfcNbdwowDsgFblTVJWE/xGNJP7bccQdMmwbXXecuvMXyP41JDDk57v6AvDOBFSvcHcVdu8LUqe4iciwOImc3Z5ky9+CDrpY/diz885+x+Y9izIED7vkCd9/thptu3x5uuw2GDYuth8tEmvTt39Qcl8cecwn/ssvce0v4JlZVq+YqLuvWwTPPuLPV0aPdE8jmzTs6kFy8sH9VU2JpafC737knMqWmxlZtyJjCJCW5Ib0zMuD5590F4iuvdI/mfPjho8NJxzpL+qZEXn4ZfvMbOOcceO65xBkbxSSOChVcu356OrzyCjRtCr//vXtOwKxZR58uFqss6ZuIvfWWa87p2hVeesmGQzbxTcR1A/3gA3fH7xlnwE03uRFk77nH3R8Qiyzpm4isWAEXXwynneYerG1PtTKJQgTOPReWLXM3gZ15prvQ26qV67m2e7ffEZaMJX1TrIwMuOACaNIE3nwTGjTwOyJj/NGrF7z6qmv66dfPdVlu1QomT3ajhcYCS/qmSBs2wIABULMmLF0KJ5/sd0TG+K9HD3jhBVchuugidyd6crIbZPDrr/2OrmiW9E2htm51dymquoSfnOx3RMZEl06dYMEC193ziivg7393F3yvuSZ6nxthSd+E9c03LuF//z288Qa0bet3RMZEr7Zt4V//cjd3/fa3rn9/mzbu/YYNfkdXkCV9c4zdu12Tzo4dsGSJ661jjCle69bwj3/Al1+6bp5pae4mr5Ejjz4RzG+W9E0B+/e7m642bID//AfOPtvviIyJPc2bw+zZronnlltcF+dOnWDoUPd8YD9Z0jf5Dhxw3TI/+QT+/W/o39/viIyJbSed5Mb337LFPeFr2TI3xv+vfuVG/vSDJX0DuPFFLr/cPaD6qadg8GC/IzImfjRo4Lp3bt3qnvT10UfuyV79+7ubHstz3EtL+obDh93QCq+84tojR470OyJj4lOdOu7Grq1bYeZMyMx0N36dc4676bE8kr8l/QR35AhMmAALF7onDE2Y4HdExsS/GjXckA6bN8OcObBtm7sBsn//sk/8Nj5iAlN1f3jz5rn2xptv9jsiYxJL1apw7bUwfjzMn+86UpT1g4gs6SewadPc065uvNG9N8b4o3JlGDeufD7LmnfiUGqqu3u2QgX3MzX12DIPPADTp7vxwh980B5zaEyisJp+nElNde3yOTlueuvWo+30o0a5n3Pnwh/+4G4bf/RRS/jGJJKIavoiMkhE1ovIRhGZHGb5LBFZ7b02iMjeoGWHg5YtLs3gzbGmTDma8PPk5Lj54B4Hd/XVbpzwp592TwsyxiSOYmv6IpIEzAEGAFnAShFZrKqZeWVUdVJQ+euAbkGbOKCqdiN/Odm2rfD5//kP/N//QZ8+sGiRPfXKmEQUSU0/BdioqptU9SCQBgwpovwIYEFpBGdKrmXL8PMbNXLNOT16uFvCq1Ur37iMMdEhkqTfDPgqaDrLm3cMEWkFtAaWBc2uKiLpIrJCRC4pZL0JXpn07OzsCEM34cyYcexjDKtUgX374PTT3QBqtWr5E5sxxn+RJP1wl/kKu31gOLBIVQ8HzWupqgFgJDBbRE49ZmOqc1U1oKqBRo0aRRCSKcyoUe5CbatW7gLtySe7dvsWLdxTr+rX9ztCY4yfIkn6WUCLoOnmwPZCyg4npGlHVbd7PzcBb1Owvd+UgVGj3ABP69ZBbq4b92PpUve4Q2NMYosk6a8E2ohIaxGpjEvsx/TCEZG2QD3gw6B59USkive+IdALyAxd15S+bdvcQ1BEXMJv1crviIwx0aDY3juqmisiE4HXgSRgnqquFZHpQLqq5h0ARgBpqgVGjmgHPCoiR3AHmHuDe/2YsnHokLto+/338O67ri3fGGMgwpuzVPVV4NWQeX8OmZ4WZr0PgE4nEJ85Drff7oZuXbggVVSpAAASLElEQVQQOnf2OxpjTDSxYRjizLJlcO+9bniFyy/3OxpjTLSxpB9Hdu504+KffrobSM0YY0LZ2DtxQtWN0rdzJ7z8shuv2xhjQlnSjxMPP+zutJ01C7pZp1hjTCGseScOZGS4B6BceCHccIPf0Rhjopkl/RiXkwMjRkDduvDEEzZMsjGmaNa8E+Nuvtk9XPn116FxY7+jMcZEO6vpx7Dnn4d//MM9EGXgQL+jMcbEAkv6Meqrr9zDlAMBuOsuv6MxxsQKS/ox6PBhGD3aDbfwzDP2MBRjTOSsTT8G3X03LF8OTz4Jbdr4HY0xJpZYTT/GvP8+3HEHjBzp7r41xpiSsKQfQ/budcm+ZUt45BHrnmmMKTlr3okRqjBhAmzfDu+9B7Vr+x2RMSYWWdKPEfPmwb//DffcAz17+h2NMSZWWfNODPjf/+D66+Hcc+HWW/2OxhgTyyzpR7mff3bDLFSrBk8/DRXsN2aMOQHWvBPlJk+G1ath8WJo2tTvaIwxsc7qjVHs1Vdh9myYOBEuvtjvaIwx8SCipC8ig0RkvYhsFJHJYZbPEpHV3muDiOwNWjZGRL7wXmNKM/h4tmMHjB0LnTrB/ff7HY0xJl4U27wjIknAHGAAkAWsFJHFqpqZV0ZVJwWVvw7o5r2vD9wOBAAFVnnr7inVbxFnjhyBMWPghx8gLQ2qVvU7ImNMvIikpp8CbFTVTap6EEgDhhRRfgSwwHt/PvCmqu72Ev2bwKATCTgRzJwJb77pnoLVvr3f0Rhj4kkkSb8Z8FXQdJY37xgi0gpoDSwryboiMkFE0kUkPTs7O5K441Z6Otx2Gwwd6m7GMsaY0hRJ0g93s78WUnY4sEhVD5dkXVWdq6oBVQ00atQogpDi0/79rntmkybw2GM2zIIxpvRFkvSzgBZB082B7YWUHc7Rpp2SrpvwJk6ETZsgNRXq1/c7GmNMPIok6a8E2ohIaxGpjEvsi0MLiUhboB7wYdDs14GBIlJPROoBA715JkRqKjz1FEydCn36+B2NMSZeFdt7R1VzRWQiLlknAfNUda2ITAfSVTXvADACSFNVDVp3t4jciTtwAExX1d2l+xVi36ZNcM010KsX/OlPfkdjjIlnEpSjo0IgEND09HS/wyg3hw5B796wfj2sWQOtWvkdkTEmFonIKlUNFFfOhmHw2e23w8cfw8KFlvCNMWXPhmHw0bJlcO+97gHnl1/udzTGmERgSd8nO3e6h5u3bevG1zHGmPJgzTs+UIVx42DXLjeoWo0afkdkjEkUlvR9MGcOvPSSq+F37ep3NMaYRGLNO+UsIwNuuQUuvNA9DcsYY8qTJf1ylJMDw4dDvXrwxBM2zIIxpvxZ8045uukmWLcO3ngDGjf2OxpjTCKymn45ef55ePRR+MMfYMAAv6MxxiQqS/rlYNs2uPJKCATgrrv8jsYYk8gs6Zexw4ddf/zcXFiwACpX9jsiY0wiszb9MjZjBrz7rhtB87TT/I7GGJPorKZfht5/H+64A0aNgt/8xu9ojDHGkn6Z2bMHRo6E5GR4+GG/ozHGGMead8qAKvzud7B9O7z3HtSu7XdExhjjWNIvA//8J/z733DPPdCzp9/RGGPMUda8U8o2bIAbboD+/eHWW/2OxhhjCrKkX4qOHHH98StXdr11KtjeNcZEmYjSkogMEpH1IrJRRCYXUuYKEckUkbUi8kzQ/MMistp7HfNA9Xjy8MOuDX/WLGja1O9ojDHmWMW26YtIEjAHGABkAStFZLGqZgaVaQP8EeilqntEJHhkmQOqGvcDCG/ZApMnw/nnw5gxfkdjjDHhRVLTTwE2quomVT0IpAFDQspcBcxR1T0Aqvpd6YYZ3VThqqvcqJmPPmqjZxpjolckSb8Z8FXQdJY3L9jpwOki8r6IrBCRQUHLqopIujf/knAfICITvDLp2dnZJfoC0eCJJ2DpUvjLX+zh5saY6BZJl81w9VYNs502QD+gOfCuiHRU1b1AS1XdLiKnAMtE5DNV/bLAxlTnAnMBAoFA6Laj2vbtbsjkPn3g6qv9jsYYY4oWSU0/C2gRNN0c2B6mzH9U9ZCqbgbW4w4CqOp27+cm4G2g2wnGHDVU4Zpr4Oef4fHH3YBqycmu105yMqSm+h2hMcYUFEnSXwm0EZHWIlIZGA6E9sJ5EfglgIg0xDX3bBKReiJSJWh+LyCTOPHss7B4Mdx5J3z8MUyYAFu3uoPB1q1u2hK/MSaaFNu8o6q5IjIReB1IAuap6loRmQ6kq+pib9lAEckEDgN/UNVdIvIL4FEROYI7wNwb3OsnlmVnw3XXwZlnwo03uhE0c3IKlsnJgSlT3IBrxhgTDUQ1uprQA4GApqen+x1GsUaOhEWL4JNPoGNH16QTbleKuJu2jDGmLInIKlUNFFfO7hk9DosXu/b7KVNcwgdo2TJ82cLmG2OMHyzpl9Deva6XTqdO8Mc/Hp0/YwZUr16wbPXqbr4xxkQLS/oldMst8O23MG9ewUcfjhoFc+e6fvoi7ufcudaeb4yJLja0cgm8+aYbNvnWW91DzkONGmVJ3hgT3aymH6EffnBDLbRpA9Om+R2NMcYcH6vpR+i222DbNli+HKpV8zsaY4w5PlbTj8D778Pf/w6//z307u13NMYYc/ws6RfjwAEYN851vbznHr+jMcaYE2PNO8W44w73CMQ33oCaNf2OxhhjTozV9IuwahU88ICr6Q8Y4Hc0xhhz4izpF+LgQZfsGzeGmTP9jsYYY0qHNe8U4i9/gYwMePFFqFvX72iMMaZ0WE0/jM8/d8MlDx8OQ0IfDGmMMTHMkn6Iw4fhyiuhTh3461/9jsYYY0qXNe+EmD3bPRDlmWegUSO/ozHGmNJlNf0gX3wBU6fCxRe7ph1jjIk3lvQ9R47A+PFQpQo88ogbKdMYY+KNNe94Hn3Ujavz+OPQrJnf0RhjTNmwmj5uILVbb4XzznN9840xJl5FlPRFZJCIrBeRjSIyuZAyV4hIpoisFZFnguaPEZEvvNeY0gq8tKjChAnu59y51qxjjIlvxTbviEgSMAcYAGQBK0VksapmBpVpA/wR6KWqe0SksTe/PnA7EAAUWOWtu6f0v8rxeeopeP111z2zdWu/ozHGmLIVSU0/BdioqptU9SCQBoTesnQVMCcvmavqd97884E3VXW3t+xNYFDphH7ivvkGJk2CXr3csMnGGBPvIkn6zYCvgqazvHnBTgdOF5H3RWSFiAwqwbqIyAQRSReR9Ozs7MijPwGqcO21kJPjHoFYwa5uGGMSQCSpLlwrt4ZMVwTaAP2AEcDjIlI3wnVR1bmqGlDVQKNyuiNq0SJ44QU3dHLbtuXykcYY47tIkn4W0CJoujmwPUyZ/6jqIVXdDKzHHQQiWbfc7doFEydCjx5w881+R2OMMeUnkqS/EmgjIq1FpDIwHFgcUuZF4JcAItIQ19yzCXgdGCgi9USkHjDQm+erG2+E3btds05Fu1PBGJNAik15qporIhNxyToJmKeqa0VkOpCuqos5mtwzgcPAH1R1F4CI3Ik7cABMV9XdZfFFIvXKKzB/PvzpT9Cli5+RGGNM+RPVY5rYfRUIBDQ9Pb1Mtr1vH3To4MbHX7XKDblgjDHxQERWqWqguHIJ1bhx662wYwc8/7wlfGNMYkqYjopvveXuuJ00CVJS/I7GGGP8kRBJ/8cf3Qiap50G06f7HY0xxvgnIZp3pk6FTZvg7behenW/ozHGGP/EfU3/ww/hoYfgmmugb1+/ozHGGH/FddL/+Wf3vNvmzeHee/2Oxhhj/BfXzTt33gnr1sGSJVC7tt/RGGOM/+K2pv/pp652P2YMDIqacT2NMcZfcZn0Dx1yT8Bq2BAefNDvaIwxJnrEZfPO/ffD6tXw3HNQv77f0RhjTPSIu5p+ZqYbLvnyy2HoUL+jMcaY6BJXSf/wYddbp2ZN+Nvf/I7GGGOiT1w17/ztb7BiBTz9NJx0kt/RGGNM9Imbmv7mzTBlClx4IYwa5Xc0xhgTneKmpt+sGUyeDGPHgoR7SKMxxpj4SfqVK7sHoxhjjClc3DTvGGOMKZ4lfWOMSSARJX0RGSQi60Vko4hMDrN8rIhki8hq7zU+aNnhoPmhD1Q3xhhTjopt0xeRJGAOMADIAlaKyGJVzQwp+qyqTgyziQOq2vXEQzXGGHOiIqnppwAbVXWTqh4E0oAhZRuWMcaYshBJ0m8GfBU0neXNC3WpiGSIyCIRaRE0v6qIpIvIChG5JNwHiMgEr0x6dnZ25NEbY4wpkUiSfrhe7xoy/RKQrKqdgaXAk0HLWqpqABgJzBaRU4/ZmOpcVQ2oaqBRo0YRhm6MMaakIkn6WUBwzb05sD24gKruUtWfvcnHgB5By7Z7PzcBbwPdTiBeY4wxJyCSpL8SaCMirUWkMjAcKNALR0RODpocDKzz5tcTkSre+4ZALyD0ArAxxphyUmzvHVXNFZGJwOtAEjBPVdeKyHQgXVUXA9eLyGAgF9gNjPVWbwc8KiJHcAeYe8P0+jHGGFNORDW0ed5fgUBA09PT/Q7DGGNiiois8q6fFsnuyDXGmARiSd8YYxKIJX1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9Y4xJIJb0jTEmgVjSN8aYBGJJ3xhjEoglfWOMSSCW9I0xJoFY0jfGmARiSd8YYxJI3CT91FRIToYKFdzP1FS/IzLGmOhT7ENUYkFqKkyYADk5bnrrVjcNMGqUf3EZY0y0iYua/pQpRxN+npwcN98YY8xRcZH0t20r2XxjjElUcZH0W7Ys2XxjjElUESV9ERkkIutFZKOITA6zfKyIZIvIau81PmjZGBH5wnuNKc3g88yYAdWrF5xXvbqbb4wx5qhiL+SKSBIwBxgAZAErRWSxqmaGFH1WVSeGrFsfuB0IAAqs8tbdUyrRe/Iu1k6Z4pp0WrZ0Cd8u4hpjTEGR9N5JATaq6iYAEUkDhgChST+c84E3VXW3t+6bwCBgwfGFW7hRoyzJG2NMcSJp3mkGfBU0neXNC3WpiGSIyCIRaVGSdUVkgoiki0h6dnZ2hKEbY4wpqUiSvoSZpyHTLwHJqtoZWAo8WYJ1UdW5qhpQ1UCjRo0iCMkYY8zxiCTpZwEtgqabA9uDC6jqLlX92Zt8DOgR6brGGGPKTyRJfyXQRkRai0hlYDiwOLiAiJwcNDkYWOe9fx0YKCL1RKQeMNCbZ4wxxgfFXshV1VwRmYhL1knAPFVdKyLTgXRVXQxcLyKDgVxgNzDWW3e3iNyJO3AATM+7qGuMMab8ieoxTey+EpFsYOsJbKIhsLOUwol1ti8Ksv1RkO2Po+JhX7RS1WIvikZd0j9RIpKuqgG/44gGti8Ksv1RkO2PoxJpX8TFMAzGGGMiY0nfGGMSSDwm/bl+BxBFbF8UZPujINsfRyXMvoi7Nn1jjDGFi8eavjHGmEJY0jfGmAQSN0m/uDH/E4mItBCRt0RknYisFZEb/I7JbyKSJCKfisjLfsfiNxGp6w2M+D/vb+Rsv2Pyk4hM8v5PPheRBSJS1e+YylJcJP2gMf8vANoDI0Skvb9R+SoXuFlV2wFnAb9P8P0BcANHhwdJdA8Br6nqGUAXEni/iEgz4HogoKodcaMODPc3qrIVF0mfoDH/VfUgkDfmf0JS1R2q+on3fj/unzrccNgJQUSaA78CHvc7Fr+JSG2gD/BPAFU9qKp7/Y3KdxWBaiJSEahOnA8KGS9JP9Ix/xOOiCQD3YCP/I3EV7OBW4EjfgcSBU4BsoEnvOaux0Wkht9B+UVVvwYeALYBO4B9qvqGv1GVrXhJ+hGN259oRKQm8Bxwo6p+73c8fhCRi4DvVHWV37FEiYpAd+ARVe0G/Agk7DUwb/TfIUBroClQQ0RG+xtV2YqXpG/j9ocQkUq4hJ+qqs/7HY+PegGDRWQLrtnvXBGZ729IvsoCslQ178xvEe4gkKjOAzararaqHgKeB37hc0xlKl6SfrFj/icSERFcm+06VX3Q73j8pKp/VNXmqpqM+7tYpqpxXZMriqp+A3wlIm29Wf2J7HnX8WobcJaIVPf+b/oT5xe2I3kwetQrbMx/n8PyUy/gN8BnIrLam3ebqr7qY0wmelwHpHoVpE3Ab32Oxzeq+pGILAI+wfV6+5Q4H5LBhmEwxpgEEi/NO8YYYyJgSd8YYxKIJX1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9Y4xJIP8fJ2ZURhf3rOEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOXZ//HPxSayCDbogyVKgKKyGCBGxIoCrrjhriDW4lKKKy71J6J1QXm0asWiPFZqtYsRRK1KrUq1UteKBFkUkUKRJUIVUBBEkcD1++OehEmYJBOY5CQz3/frNa/MOXPPOddMkmvuuc+9mLsjIiLppUHUAYiISOopuYuIpCEldxGRNKTkLiKShpTcRUTSkJK7iEgaUnKXhMysoZltNLP9Ulk2Smb2IzNLed9fMzvGzJbGbS80syOSKbsT53rUzEbv7PMrOe6dZvaHVB9XotMo6gAkNcxsY9xmM2AzsDW2/XN3L6jO8dx9K9Ai1WUzgbsfkIrjmNklwPnu3j/u2Jek4tiS/pTc04S7lybXWM3wEnd/raLyZtbI3YtrIzYRqX1qlskQsa/dT5nZJDPbAJxvZoeZ2Xtmts7MVpnZeDNrHCvfyMzczHJi20/EHn/ZzDaY2b/MrEN1y8YeP8HM/m1m683sQTN7x8yGVRB3MjH+3MwWm9lXZjY+7rkNzWycma01s/8AAyt5f242s8nl9k0ws/tj9y8xswWx1/OfWK26omMVmVn/2P1mZvbnWGzzgYMTnHdJ7LjzzWxQbP9BwEPAEbEmrzVx7+1tcc8fEXvta83seTPbJ5n3pipmdlosnnVm9rqZHRD32GgzW2lmX5vZJ3GvtY+ZfRDb/7mZ3Zvs+aQGuLtuaXYDlgLHlNt3J/A9cArhQ3134BDgUMI3uI7Av4ErYuUbAQ7kxLafANYA+UBj4CngiZ0ouzewATg19ti1wBZgWAWvJZkYXwBaATnAlyWvHbgCmA9kA1nAm+FPPuF5OgIbgeZxx/4CyI9tnxIrY8BRwLdAbuyxY4ClcccqAvrH7t8H/BPYE2gPfFyu7DnAPrHfyXmxGP4n9tglwD/LxfkEcFvs/nGxGHsCTYH/A15P5r1J8PrvBP4Qu98lFsdRsd/R6Nj73hjoBiwD2sbKdgA6xu7PBIbE7rcEDo36fyGTb6q5Z5a33f2v7r7N3b9195nuPsPdi919CTAR6FfJ859x90J33wIUEJJKdcueDMxx9xdij40jfBAklGSMd7n7endfSkikJec6Bxjn7kXuvha4u5LzLAE+InzoABwLrHP3wtjjf3X3JR68DvwDSHjRtJxzgDvd/St3X0aojcefd4q7r4r9Tp4kfDDnJ3FcgKHAo+4+x92/A0YB/cwsO65MRe9NZQYDU9399djv6G5gD8KHbDHhg6RbrGnv09h7B+FDurOZZbn7BnefkeTrkBqg5J5ZVsRvmNmBZvY3M/uvmX0NjAHaVPL8/8bd30TlF1ErKvvD+Djc3Qk13YSSjDGpcxFqnJV5EhgSu38e4UOpJI6TzWyGmX1pZusItebK3qsS+1QWg5kNM7O5seaPdcCBSR4XwusrPZ67fw18BbSLK1Od31lFx91G+B21c/eFwHWE38MXsWa+trGiFwJdgYVm9r6ZnZjk65AaoOSeWcp3A3yEUFv9kbvvAdxCaHaoSasIzSQAmJlRNhmVtysxrgL2jduuqqvmU8AxsZrvqYRkj5ntDjwD3EVoMmkN/D3JOP5bUQxm1hF4GLgUyIod95O441bVbXMloamn5HgtCc0/nyURV3WO24DwO/sMwN2fcPfDCU0yDQnvC+6+0N0HE5refg08a2ZNdzEW2UlK7pmtJbAe+MbMugA/r4VzvgjkmdkpZtYIGAnsVUMxTgGuNrN2ZpYF3FBZYXf/HHgbeBxY6O6LYg/tBjQBVgNbzexk4OhqxDDazFpbGAdwRdxjLQgJfDXhc+4SQs29xOdAdskF5AQmARebWa6Z7UZIsm+5e4XfhKoR8yAz6x879/WE6yQzzKyLmQ2Ine/b2G0r4QX8xMzaxGr662OvbdsuxiI7Sck9s10H/JTwj/sIoeZao2IJ9FzgfmAt0AmYTeiXn+oYHya0jX9IuNj3TBLPeZJwgfTJuJjXAdcAzxEuSp5F+JBKxq2EbxBLgZeBP8Uddx4wHng/VuZAIL6d+lVgEfC5mcU3r5Q8/xVC88hzsefvR2iH3yXuPp/wnj9M+OAZCAyKtb/vBtxDuE7yX8I3hZtjTz0RWGChN9Z9wLnu/v2uxiM7x0KTp0g0zKwhoRngLHd/K+p4RNKFau5S68xsoJm1in21/yWhB8b7EYclklaU3CUKfYElhK/2A4HT3L2iZhkR2QlqlhERSUOquYuIpKHIJg5r06aN5+TkRHV6EZF6adasWWvcvbLuw0CEyT0nJ4fCwsKoTi8iUi+ZWVUjrQE1y4iIpCUldxGRNKTkLiKShrQSk0iG2LJlC0VFRXz33XdRhyJJaNq0KdnZ2TRuXNHUQpVTchfJEEVFRbRs2ZKcnBzCZJxSV7k7a9eupaioiA4dOlT9hATqVbNMQQHk5ECDBuFnQbWWfBbJbN999x1ZWVlK7PWAmZGVlbVL37LqTc29oACGD4dNm8L2smVhG2DoLs+DJ5IZlNjrj139XdWbmvtNN21P7CU2bQr7RUSkrHqT3Jcvr95+Ealb1q5dS8+ePenZsydt27alXbt2pdvff5/ctO8XXnghCxcurLTMhAkTKEhRm23fvn2ZM2dOSo5V2+pNs8x++4WmmET7RST1CgrCN+Ply8P/2dixu9YEmpWVVZoob7vtNlq0aMEvfvGLMmXcHXenQYPE9c7HH3+8yvNcfvnlOx9kGqk3NfexY6FZs7L7mjUL+0UktUqucS1bBu7br3HVRCeGxYsX0717d0aMGEFeXh6rVq1i+PDh5Ofn061bN8aMGVNatqQmXVxcTOvWrRk1ahQ9evTgsMMO44svvgDg5ptv5oEHHigtP2rUKHr37s0BBxzAu+++C8A333zDmWeeSY8ePRgyZAj5+flV1tCfeOIJDjroILp3787o0aMBKC4u5ic/+Unp/vHjxwMwbtw4unbtSo8ePTj//PNT/p4lo94k96FDYeJEaN8ezMLPiRN1MVWkJtT2Na6PP/6Yiy++mNmzZ9OuXTvuvvtuCgsLmTt3Lq+++ioff/zxDs9Zv349/fr1Y+7cuRx22GE89thjCY/t7rz//vvce++9pR8UDz74IG3btmXu3LmMGjWK2bNnVxpfUVERN998M9OnT2f27Nm88847vPjii8yaNYs1a9bw4Ycf8tFHH3HBBRcAcM899zBnzhzmzp3LQw89tIvvzs6pN8kdQiJfuhS2bQs/ldhFakZtX+Pq1KkThxxySOn2pEmTyMvLIy8vjwULFiRM7rvvvjsnnHACAAcffDBLly5NeOwzzjhjhzJvv/02gwcPBqBHjx5069at0vhmzJjBUUcdRZs2bWjcuDHnnXceb775Jj/60Y9YuHAhI0eOZNq0abRq1QqAbt26cf7551NQULDTg5B2Vb1K7iJSOyq6llVT17iaN29een/RokX85je/4fXXX2fevHkMHDgwYX/vJk2alN5v2LAhxcXFCY+922677VCmuosUVVQ+KyuLefPm0bdvX8aPH8/Pf/5zAKZNm8aIESN4//33yc/PZ+vWrdU6XyoouYvIDqK8xvX111/TsmVL9thjD1atWsW0adNSfo6+ffsyZcoUAD788MOE3wzi9enTh+nTp7N27VqKi4uZPHky/fr1Y/Xq1bg7Z599NrfffjsffPABW7dupaioiKOOOop7772X1atXs6l8G1ctqDe9ZUSk9pQ0eaayt0yy8vLy6Nq1K927d6djx44cfvjhKT/HlVdeyQUXXEBubi55eXl07969tEklkezsbMaMGUP//v1xd0455RROOukkPvjgAy6++GLcHTPjV7/6FcXFxZx33nls2LCBbdu2ccMNN9CyZcuUv4aqJLWGqpkNBH4DNAQedfe7yz2+H/BHoHWszCh3f6myY+bn57sW6xCpPQsWLKBLly5Rh1EnFBcXU1xcTNOmTVm0aBHHHXccixYtolGjulXfTfQ7M7NZ7p5f1XOrfCVm1hCYABwLFAEzzWyqu8d/j7kZmOLuD5tZV+AlICf5lyAiUns2btzI0UcfTXFxMe7OI488UucS+65K5tX0Bha7+xIAM5sMnArEJ3cH9ojdbwWsTGWQIiKp1Lp1a2bNmhV1GDUqmQuq7YAVcdtFsX3xbgPON7MiQq39ykQHMrPhZlZoZoWrV6/eiXBFRCQZyST3RFOTlW+oHwL8wd2zgROBP5vZDsd294nunu/u+XvtVeXi3SIispOSSe5FwL5x29ns2OxyMTAFwN3/BTQF2qQiQBERqb5kkvtMoLOZdTCzJsBgYGq5MsuBowHMrAshuavdRUQkIlUmd3cvBq4ApgELCL1i5pvZGDMbFCt2HfAzM5sLTAKGeXWHgIlIWuvfv/8OA5IeeOABLrvsskqf16JFCwBWrlzJWWedVeGxq+pa/cADD5QZTHTiiSeybt26ZEKv1G233cZ99923y8dJtaRGqLr7S+6+v7t3cvexsX23uPvU2P2P3f1wd+/h7j3d/e81GbSI1D9Dhgxh8uTJZfZNnjyZIUOGJPX8H/7whzzzzDM7ff7yyf2ll16idevWO328uk7TD4hIrTjrrLN48cUX2bx5MwBLly5l5cqV9O3bt7TfeV5eHgcddBAvvPDCDs9funQp3bt3B+Dbb79l8ODB5Obmcu655/Ltt9+Wlrv00ktLpwu+9dZbARg/fjwrV65kwIABDBgwAICcnBzWrFkDwP3330/37t3p3r176XTBS5cupUuXLvzsZz+jW7duHHfccWXOk8icOXPo06cPubm5nH766Xz11Vel5+/atSu5ubmlE5a98cYbpYuV9OrViw0bNuz0e5tIevXaF5GkXH01pHqBoZ49IZYXE8rKyqJ379688sornHrqqUyePJlzzz0XM6Np06Y899xz7LHHHqxZs4Y+ffowaNCgCtcRffjhh2nWrBnz5s1j3rx55OXllT42duxYfvCDH7B161aOPvpo5s2bx1VXXcX999/P9OnTadOmbF+PWbNm8fjjjzNjxgzcnUMPPZR+/fqx5557smjRIiZNmsTvfvc7zjnnHJ599tlK52e/4IILePDBB+nXrx+33HILt99+Ow888AB33303n376KbvttltpU9B9993HhAkTOPzww9m4cSNNmzatxrtdNdXcRaTWxDfNxDfJuDujR48mNzeXY445hs8++4zPP/+8wuO8+eabpUk2NzeX3Nzc0semTJlCXl4evXr1Yv78+VVOCvb2229z+umn07x5c1q0aMEZZ5zBW2+9BUCHDh3o2bMnUPm0whDml1+3bh39+vUD4Kc//SlvvvlmaYxDhw7liSeeKB0Je/jhh3Pttdcyfvx41q1bl/IRsqq5i2SgymrYNem0007j2muv5YMPPuDbb78trXEXFBSwevVqZs2aRePGjcnJyUk4zW+8RLX6Tz/9lPvuu4+ZM2ey5557MmzYsCqPU1nfj5LpgiFMGVxVs0xF/va3v/Hmm28ydepU7rjjDubPn8+oUaM46aSTeOmll+jTpw+vvfYaBx544E4dPxHV3EWk1rRo0YL+/ftz0UUXlbmQun79evbee28aN27M9OnTWZZoweQ4Rx55ZOki2B999BHz5s0DwnTBzZs3p1WrVnz++ee8/PLLpc9p2bJlwnbtI488kueff55NmzbxzTff8Nxzz3HEEUdU+7W1atWKPffcs7TW/+c//5l+/fqxbds2VqxYwYABA7jnnntYt24dGzdu5D//+Q8HHXQQN9xwA/n5+XzyySfVPmdlVHMXkVo1ZMgQzjjjjDI9Z4YOHcopp5xCfn4+PXv2rLIGe+mll3LhhReSm5tLz5496d27NxBWVerVqxfdunXbYbrg4cOHc8IJJ7DPPvswffr00v15eXkMGzas9BiXXHIJvXr1qrQJpiJ//OMfGTFiBJs2baJjx448/vjjbN26lfPPP5/169fj7lxzzTW0bt2aX/7yl0yfPp2GDRvStWvX0lWlUiWpKX9rgqb8FaldmvK3/tmVKX/VLCMikoaU3EVE0pCSu0gG0awg9ceu/q6U3EUyRNOmTVm7dq0SfD3g7qxdu3aXBjapt4xIhsjOzqaoqAgtlFM/NG3alOzs7J1+vpK7SIZo3LgxHTp0iDoMqSVqlhERSUNK7iIiaUjJXUQkDSm5i4ikISV3EZE0pOQuIpKGlNxFRNKQkruISBpSchcRSUNK7iIiaUjJXUQkDSWV3M1soJktNLPFZjYqwePjzGxO7PZvM1uX+lBFRCRZVU4cZmYNgQnAsUARMNPMprr7xyVl3P2auPJXAr1qIFYREUlSMjX33sBid1/i7t8Dk4FTKyk/BJiUiuBERGTnJJPc2wEr4raLYvt2YGbtgQ7A6xU8PtzMCs2sUHNKi4jUnGSSuyXYV9FSLoOBZ9x9a6IH3X2iu+e7e/5ee+2VbIwiIlJNyST3ImDfuO1sYGUFZQejJhkRkcglk9xnAp3NrIOZNSEk8KnlC5nZAcCewL9SG2JZ06fDddfBs8/CqlU1eSYRkfqryt4y7l5sZlcA04CGwGPuPt/MxgCF7l6S6IcAk72GV9/98EOYMAHuvz9s5+TAj3+8/XbQQdBIiweKSIazqFZCz8/P98LCwp167vffw+zZ8O674fbOO9tr8c2bw6GHbk/2ffrAnnumMHARkQiZ2Sx3z6+yXH1M7uW5w/Ll25P9u+/C3LmwNXZZt2vXsrX7/fcHS3SZWESkjsuo5J7Ixo0wc2bZhL8uNm42KwsOO2x7sj/kEGjWLPljFxTATTeFD5T99oOxY2Ho0Jp5HSIi8ZJN7mnbOt2iBQwYEG4A27bBwoVlk/2LL4bHGjWCnj3L1u733TfxcQsKYPhw2LQpbC9bFrZBCV5E6o60rbknY+1aeO+97cl+xgz49tvwWHZ2SPKHHx5+9ugBjRuHC7jLlu14rPbtYenS2oxeRDJRxjfL7IwtW2DevO0Xad99F1bExubuvjv07g1vvJH4uWbh24GISE3K+GaZndG4MRx8cLhdeWXYt2IF/Otf22v3Fdlvv9qJUUQkGUruVdh333A755yw/dhjcNllsHnz9jINGsCIEdHEJyKSiBbrqKaLLoLf/z60sQO0bg1Nm4beM5dcAp99Fm18IiKg5L5Thg4NF0/d4auvQpfIkSPhT3+Czp3hl7+EDRuijlJEMpmSewpkZYXpED75BE49Fe68Ezp1gv/7v3CRVkSktim5p1DHjjBpErz/fhgVe/nl0L07PPdcqOWLiNQWJfcacMghYfbKqVOhYUM44ww44ojQ60ZEpDYoudcQMzjllNBv/pFHYPHiMBjq7LPDfRGRmqTkXsMaNQrTEyxeDLfdBi+/DF26wFVXwZo1UUcnIulKyb2WtGgBt94akvzFF4eLrZ06wV13bZ/yQEQkVZTca1nbtvDb34ZFR/r3h9GjwxTEf/jD9imKRUR2lZJ7RLp0gRdegH/+E/bZBy68EPLyYNq05I9RUBAmMmvQIPwsKKihYEWk3lFyj1i/fmE2ysmTw8CngQPhuOPCYiOVKZl6eNmy0M2yZOphJXgRASX3OsEMzj0XFiyAceNg1izo1Qt++tPts1KWd9NN2+eUL7FpU9gvIqLkXofsthtcfTX85z9w/fXw1FOhPf7GG2H9+rJlly9PfIyK9otIZlFyr4Nat4Zf/SqsHHX22XD33aFnzfjxYXFwqHiKYU09LCKg5F6ntW8fJiP74IOwDODIkWFag6efDvPXlF/3tVmzsJ6riIiSez3Qqxe8+moYANWsWZhb/qGH4LrrwgeAWfg5caLWcRWRQMm9njALPWlmzw4LhqxYAXfcEWr0ixeHKYiV2EWkRFLJ3cwGmtlCM1tsZqMqKHOOmX1sZvPN7MnUhiklGjYMfeIXLQpNMK+/DkceCStXRh2ZiNQlVSZ3M2sITABOALoCQ8ysa7kynYEbgcPdvRtwdQ3EKnGaNQujW996C9atC/PIl+8aKSKZK5mae29gsbsvcffvgcnAqeXK/AyY4O5fAbj7F6kNUyrSowc8+WToGz9sGGzbFnVEIlIXJJPc2wHxQ2mKYvvi7Q/sb2bvmNl7ZjYw0YHMbLiZFZpZ4erVq3cuYtnBoEGh6+TTT4eZJ0VEGiVRxhLsK7+uUCOgM9AfyAbeMrPu7r6uzJPcJwITAfLz87U2UQr94hdhmb877oADD4Tzzos6IhGJUjI19yJg37jtbKD85bsi4AV33+LunwILCcleaokZPPxwmKvmoou06pNIpksmuc8EOptZBzNrAgwGppYr8zwwAMDM2hCaaZakMlCpWpMm8OyzkJ0Np50WJhMTkcxUZXJ392LgCmAasACY4u7zzWyMmQ2KFZsGrDWzj4HpwPXuvramgpaKZWXBiy/C5s1hmb8NG6KOSESiYO7RNH3n5+d7YWFhJOfOBK++CiecEG7PPx/6x4tI/Wdms9w9v6pyGqGapo49Nkw09uKLcMMNUUcjIrUtmd4yUk9ddlmYI/7Xvw4rP118cdQRiUhtUc09zY0bB8cfDyNGhCX9RCQzKLmnuUaNwqIfnTvDmWeGScZEJP0puWeAVq3gr38NfeFPPhm++irqiESkpim5Z4hOneC552DJkjAf/JYtUUckIjVJyT2DHHFEWNDjtdfCqk4R9YIVkVqg3jIZZtiw0IPmnntCD5orr4w6IhGpCUruGeiuu8Li21dfHS60Dkw4h6eI1GdqlslADRrAE0/AQQeF9vf586OOSERSTck9Q7VoEXrQNG8e5qDR9Poi6UXJPYPtuy+88AKsWgWnnx4mGxOR9KDknuF694Y//AHeeQeGD1cPGpF0oQuqwrnnhlWcbrst9KAZNSrqiERkVym5CwC33BJ60Nx4IxxwQGimEZH6S80yAoSpCX7/ezj0UDj/fJg9O/nnFhRATk7ohZOTE7ZFJFpK7lJq993Dwh5ZWaEHzcryK+UmUFAQ2uqXLQvt9cuWhW0leJFoKblLGW3bhgU+1q2DU0+FTZsqL3/TTTuW2bQp7BeR6Ci5yw5yc2HSJJg1K0xXsG1bxWWXL6/efhGpHUruktApp4T5Z55+OvSiqch++1Vvv4jUDiV3qdB118FFF8Edd8CTTyYuM3YsNGtWdl+zZmG/iERHyV0qZAYPPwz9+oUk/69/7Vhm6NAwjXD79qF8+/Zhe+jQ2o9XRLYzj2hIYn5+vhcWFkZybqmetWtDF8kNG+D990MCF5FomNksd8+vqlxSNXczG2hmC81ssZntMH7RzIaZ2WozmxO7XbIzQUvdlJUVetBs3hza4jdsiDoiEalKlcndzBoCE4ATgK7AEDPrmqDoU+7eM3Z7NMVxSsQOPDBcXP34YzjvPNi6NeqIRKQyydTcewOL3X2Ju38PTAZOrdmwpC469lgYPz7U4m+4IepoRKQyyST3dsCKuO2i2L7yzjSzeWb2jJntm5LopM657DK44gr49a/hUX0/E6mzkknulmBf+auwfwVy3D0XeA34Y8IDmQ03s0IzK1yt1SHqrXHj4Pjj4dJLYfr0qKMRkUSSSe5FQHxNPBsoM+uIu69195KlHn4HHJzoQO4+0d3z3T1/r7322pl4pQ5o1Aieeiqsv3rmmbBoUdQRiUh5yST3mUBnM+tgZk2AwcDU+AJmtk/c5iBgQepClLqoVauwTF+DBnDyyfDVV1FHJCLxqkzu7l4MXAFMIyTtKe4+38zGmNmgWLGrzGy+mc0FrgKG1VTAUnd06gTPPQeffgpnnw1btkQdkUjdt21b7ax4llQ/d3d/yd33d/dO7j42tu8Wd58au3+ju3dz9x7uPsDdP6nJoKXuOOKIMCL1H/+AAQNgzpyoIxKpm9zDt91eveC112r+fJp+QHbZsGHwxz+GlZwOPhguvxy+/DLqqETqjunT4cc/hkGDwpTYdabmLlKVCy6Af/87JPbf/hb23x8eeUSDnSSzzZgBxxwDRx0FRUXwu9+FgYDHHVfz51Zyl5TZc88wyGn2bOjWDUaMgN69E084JpLO5s0Li9306RPuP/BA6FV2ySXQuHHtxKDkLimXmwv//GdY8OPzz8PX0WHD4L//jToykZq1aFGYnqNnT3jjjTD19ZIlMHIkNG1au7EouUuNMIPBg+GTT2DUqDAf/P77w/33q1eNpJ8VK+BnP4MuXeCFF8Lf/KefwujR0KJFNDEpuUuNatEC7roL5s+Hvn3DAiA9eoTeNSL13RdfwDXXwI9+BH/6U7jmtGQJ/O//hmbKKCm5S63o3Bn+9jeYOjVMHXzMMXDWWbBsWdSRiVTfunVw883QsSM8+CD85CehSeY3v4H/+Z+oowuU3KXWmIX54OfPhzvvhJdeCl9j77gDvvsu6uhEqvbNN+GbaIcOoT39lFNC75dHH6176wYruUuta9oUbroptMeffDLccgt07Rpq9REtDCZSqc2bQ0+wjh1DO3rfvqFX2KRJ4VpSXaTkLpHZbz+YMiW0v+++e+g6duKJob+8SF1QXAy//31I4CNHhkrIu++GkaY9e0YdXeWU3CVyRx0Vpi0YNy7843TvHnobbNwYdWSSqbZtCzOfdusW+qa3bQuvvgqvvw6HHRZ1dMlRcpc6oXFjuPrqUGsfOhR+9Ss44IDQhVJNNdXz3XchEV1/PZxzTmhG+NOfwmjJdeuijq5ucw8rjeXlha68TZrA88/De++FTgCWaHWLOso8ov+c/Px8LywsjOTcUve9915Y8WnWLDjyyNAjITc36qjqJvcwr8+0afDKK2HwzLffhsSUnR16JMVPA7H33uGDM/524IHhImGjRtG9jqhNnx4+CN97L3RtvP12OPdcaNgw6sjKMrNZ7p5fVbkM/lVKXdanT6hpPvYY3HhjmEnvsstgzJjo+w/XBevWhWsV06aF2/LlYf/++4dmhOOPh/79oXnzMGhsyZLwAbBwYbiQvXBhqJGuWbP9mI0bh2mcyyf+Aw6ANm0ieZm1YsaMcIH/H/+Adu3CLKfDhtXeNAE1RTV3qfO+/DL0qHkejtfWAAAKiklEQVT4YfjBD8IAkYsuqns1qpq0dSsUFm5P5jNmhH177AFHHx2S+fHHQ05O9Y775Zfbk3588l+8uOxI4qysxEm/U6fwDaE++vBD+OUvw4jSvfYKtfYRI2p/moDqSrbmruQu9cbcuXDllfDWW5CfDw89BIceGnVUNeezz7Yn89deC4nYLLz2kmR+6KE1U8MsLoalS3dM/AsXlp0jqGHD0Jxz4IE7Jv69966bbdSLF8Ott4ZujHvsEa5NjBwZ3TQB1aXkLmnJPfxTXn89rFwZvj7ffXfdGRW4K777Dt58c3tCnz8/7N9nn+3J/Jhjom8iWb8+cdJftKjsYLRWrbYn+rZtwwdBoluDBhU/lsoyW7aE6agfewx22w2uuir8Hf3gB9G9lztDyV3S2oYNYYTg/feHPvK33x7m9ahP7aTusGDB9mT+xhshOTZpEi4ilyT07t3rZg24vG3bQtt/SZt+/G3t2tCMtHVrKBeVxo1D08vo0eEDpz5ScpeMsHBh6EL5yiuhT/K4caEfcvPmdTMhfvVVaGKZNg3+/vcwmyCE2u3xx8PAgdCvHzRrFm2cNck9JPiSZB+f9MvvS3RLplxFx+/Tp+5NE1Bd6i0jGeGAA8IcNX/9a0jyJSvcmEHLlttve+xR9md19rVoEb7y74ytW2HmzLIXQrdtC00WRx8dJp86/nho3z5170ldZ7a9qURqjpK71HtmYW3KY4+Fp58OC4Rs2ABffx1+xt8veaxkX3Fxcudo0SLxh0BFHwibN4euda+9FmrrZnDIIaHLXcmF0EzuUy41T39ekjb+8pfQZXL58vDVe+zYMNq1Iu4hCSf6EEhm35o1ZR8rvwjJD38Ip522/UJoVlbNvn6ReErukhYKCmD48LCyPIRRmcOHh/sVJXiz0Ke5adPQbW9Xbd68PdG7hxkE62K7v2QGXVCVtJCTk3jhj/btQ39tkXSR7AXVpC4TmdlAM1toZovNbFQl5c4yMzezKk8skkolw++T3S+S7qpM7mbWEJgAnAB0BYaYWdcE5VoCVwEzUh2kSFUq6t5W37u9ieysZGruvYHF7r7E3b8HJgOnJih3B3APoAXTpNaNHbtj3/BmzcJ+kUyUTHJvB6yI2y6K7StlZr2Afd39xcoOZGbDzazQzApXr15d7WBFKjJ0aJjNr337cBGzffuwXVlvGZF0lkxvmUTX+0uvwppZA2AcMKyqA7n7RGAihAuqyYUokpyhQ5XMRUokU3MvAvaN284GVsZttwS6A/80s6VAH2CqLqqKiEQnmeQ+E+hsZh3MrAkwGJha8qC7r3f3Nu6e4+45wHvAIHdXP0cRkYhUmdzdvRi4ApgGLACmuPt8MxtjZoNqOkAREam+pEaouvtLwEvl9t1SQdn+ux6WiIjsip2c605EROoyJXcRkTSk5C4ikoaU3EVE0pCSu4hIGlJyFxFJQ0ruIiJpSMldRCQNKbmLiKQhJXeRFCooCEv+NWgQfhYURB2RZCotkC2SIjuzSLdITVHNXSRFbrppe2IvsWlT2C9S25TcRVJEi3RLXaLkLpIiWqRb6hIld5EU0SLdUpcouYukiBbplrpEvWVEUkiLdEtdoZq7iEgaUnIXEUlDSu4iImlIyV0kDWkaBNEFVZE0o2kQBFRzF0k7mgZBQMldJO1oGgSBJJO7mQ00s4VmttjMRiV4fISZfWhmc8zsbTPrmvpQRSQZmgZBIInkbmYNgQnACUBXYEiC5P2kux/k7j2Be4D7Ux6piCRF0yAIJFdz7w0sdvcl7v49MBk4Nb6Au38dt9kc8NSFKCLVoWkQBJLrLdMOWBG3XQQcWr6QmV0OXAs0AY5KdCAzGw4MB9hP3xFFaoymQZBkau6WYN8ONXN3n+DunYAbgJsTHcjdJ7p7vrvn77XXXtWLVEREkpZMci8C9o3bzgZWVlJ+MnDargQlIiK7JpnkPhPobGYdzKwJMBiYGl/AzDrHbZ4ELEpdiCJSX2mkbHSqbHN392IzuwKYBjQEHnP3+WY2Bih096nAFWZ2DLAF+Ar4aU0GLSJ1n0bKRsvco+nYkp+f74WFhZGcW0RqXk5OSOjltW8PS5fWdjTpw8xmuXt+VeU0QlVEaoRGykZLyV1EaoRGykZLyV1EaoRGykZLyV1EaoRGykZL87mLSI3RSNnoqOYuIpKGlNxFJO1l4mAqNcuISFrL1MFUqrmLSFrL1GUHldxFJK1l6mAqJXcRSWuZOphKyV1E0lqmDqZScheRtJapg6nUW0ZE0l4mDqZSzV1EpJbUZn971dxFRGpBbfe3V81dRKQW1HZ/eyV3EZFaUNv97ZXcRURqQW33t1dyFxGpBbXd317JXUSkFtR2f3v1lhERqSW12d9eNXcRkTSk5C4ikoaSSu5mNtDMFprZYjMbleDxa83sYzObZ2b/MLP2qQ9VRESSVWVyN7OGwATgBKArMMTMupYrNhvId/dc4BngnlQHKiIiyUum5t4bWOzuS9z9e2AycGp8AXef7u4lY6/eA7JTG6aIiFRHMr1l2gEr4raLgEMrKX8x8HKiB8xsOBCbTYGNZrYwmSATaAOs2cnnpiO9H2Xp/dhO70VZ6fB+JNXsnUxytwT7PGFBs/OBfKBfosfdfSIwMZnAKg3IrNDd83f1OOlC70dZej+203tRVia9H8kk9yJg37jtbGBl+UJmdgxwE9DP3TenJjwREdkZybS5zwQ6m1kHM2sCDAamxhcws17AI8Agd/8i9WGKiEh1VJnc3b0YuAKYBiwAprj7fDMbY2aDYsXuBVoAT5vZHDObWsHhUmWXm3bSjN6PsvR+bKf3oqyMeT/MPWHzuYiI1GMaoSoikoaU3EVE0lC9S+5VTYWQKcxsXzObbmYLzGy+mY2MOqa6wMwamtlsM3sx6liiZmatzewZM/sk9ndyWNQxRcXMron9n3xkZpPMrGnUMdW0epXck5wKIVMUA9e5exegD3B5Br8X8UYSLvwL/AZ4xd0PBHqQoe+LmbUDriJMkdIdaEjo9ZfW6lVyJ4mpEDKFu69y9w9i9zcQ/nHbRRtVtMwsGzgJeDTqWKJmZnsARwK/B3D37919XbRRRaoRsLuZNQKakWCsTrqpb8k90VQIGZ3QAMwsB+gFzIg2ksg9APw/YFvUgdQBHYHVwOOxZqpHzax51EFFwd0/A+4DlgOrgPXu/vdoo6p59S25Jz0VQqYwsxbAs8DV7v511PFExcxOBr5w91lRx1JHNALygIfdvRfwDZCR16jMbE/CN/wOwA+B5rGpUtJafUvuSU2FkCnMrDEhsRe4+1+ijidihwODzGwpobnuKDN7ItqQIlUEFLl7ybe5ZwjJPhMdA3zq7qvdfQvwF+DHEcdU4+pbcq9yKoRMYWZGaE9d4O73Rx1P1Nz9RnfPdvccwt/F6+6e9rWzirj7f4EVZnZAbNfRwMcRhhSl5UAfM2sW+785mgy4uFyvFsh292IzK5kKoSHwmLvPjzisqBwO/AT40MzmxPaNdveXIoxJ6pYrgYJYRWgJcGHE8UTC3WeY2TPAB4ReZrPJgGkINP2AiEgaqm/NMiIikgQldxGRNKTkLiKShpTcRUTSkJK7iEgaUnIXEUlDSu4iImno/wN1vQpBKYFF1QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining CNNs and RNNs to process long sequences\n",
    "\n",
    "\n",
    "Because 1D convnets process input patches independently, they are not sensitive to the order of the timesteps (beyond a local scale, the \n",
    "size of the convolution windows), unlike RNNs. Of course, in order to be able to recognize longer-term patterns, one could stack many \n",
    "convolution layers and pooling layers, resulting in upper layers that would \"see\" long chunks of the original inputs -- but that's still a \n",
    "fairly weak way to induce order-sensitivity. One way to evidence this weakness is to try 1D convnets on the temperature forecasting problem \n",
    "from the previous section, where order-sensitivity was key to produce good predictions. Let's see:\n",
    "\n",
    "## 结合 CNN 和 RNN 来处理长序列\n",
    "\n",
    "一维卷积神经网络分别处理每个输入序列段，所以它对时间步的顺序不敏感（这里所说顺序的范围要大于局部尺度，即大于卷积窗口的大小），这一点与 RNN 不同。当然，为了识别更长期的模式，你可以将许多卷积层和池化层堆叠在一起，这样上面的层能够观察到原始输入中更长的序列段，但这仍然不是一种引入顺序敏感性的好方法。想要证明这种方法的不足，一种方法是在温度预测问题上使用一维卷积神经网络，在这个问题中顺序敏感性对良好的预测结果非常关键。\n",
    "\n",
    "看下代码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We reuse the following variables defined in the last section:\n",
    "# float_data, train_gen, val_gen, val_steps\n",
    "# 我们仍使用上一节定义的变量：float_data, train_gen, val_gen, val_steps\n",
    "\n",
    "import os\n",
    "import numpy as np\n",
    "\n",
    "data_dir = 'data/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values\n",
    "    \n",
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std\n",
    "\n",
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets\n",
    "        \n",
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# This is how many steps to draw from `val_gen`\n",
    "# in order to see the whole validation set:\n",
    "# 为了查看整个验证集，需要 从 val_gen 中抽取多少次\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# This is how many steps to draw from `test_gen`\n",
    "# in order to see the whole test set:\n",
    "# 为了查看整个测试集，需要从 test_gen 中抽取多少次\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 14s 27ms/step - loss: 0.4193 - val_loss: 0.4340\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.3624 - val_loss: 0.4420\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.3391 - val_loss: 0.4426\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.3217 - val_loss: 0.4663\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.3078 - val_loss: 0.4545\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.3002 - val_loss: 0.4435\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2899 - val_loss: 0.4468\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.2850 - val_loss: 0.5008\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2784 - val_loss: 0.4661\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2718 - val_loss: 0.4772\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2693 - val_loss: 0.4729\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.2660 - val_loss: 0.4672\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.2603 - val_loss: 0.4762\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.2575 - val_loss: 0.4818\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2553 - val_loss: 0.5024\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 13s 27ms/step - loss: 0.2519 - val_loss: 0.4909\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2483 - val_loss: 0.4991\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2461 - val_loss: 0.4922\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2444 - val_loss: 0.4810\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 13s 26ms/step - loss: 0.2434 - val_loss: 0.5139\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are our training and validation Mean Absolute Errors:\n",
    "\n",
    "下图是我们的训练和验证的均方差（MAE）:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGXa+PHvTQggRaRZlggBxRJKIGQBK4gNLIhiAUFFRMCfrKy87sLaX1ZeWSwororYViWKbVVWRdQFFtldwVAVlF4MzQgiICAkuX9/PCdhCJPkJJmauT/XNVdmznnOOc+cmdznzFNFVTHGGJMYqkU7A8YYYyLHgr4xxiQQC/rGGJNALOgbY0wCsaBvjDEJxIK+McYkEAv6plxEJElE9ohIs1CmjSYROVlEQt52WUQuEJH1Aa9XiMg5ftJW4FgviMjdFd2+lP0+JCJ/C/V+TfRUj3YGTHiJyJ6Al7WBX4F87/VQVc0qz/5UNR+oG+q0iUBVTw3FfkRkMDBAVbsF7HtwKPZtqj4L+lWcqhYFXe9OcrCqfl5SehGprqp5kcibMSbyrHgnwXk/398UkTdEZDcwQETOEJEvRWSniGwRkYkikuylry4iKiKp3usp3vrpIrJbRP4rIi3Km9Zb31NEVorIzyLylIj8W0QGlpBvP3kcKiKrReQnEZkYsG2SiEwQke0isgboUcr5uVdEphZb9rSIPO49Hywi33rvZ413F17SvnJEpJv3vLaIvOblbRnQMchx13r7XSYivbzlbYG/Aud4RWc/BpzbBwO2H+a99+0i8r6InODn3JRFRHp7+dkpIjNF5NSAdXeLyGYR2SUi3wW81y4istBbvk1EHvF7PBMGqmqPBHkA64ELii17CDgAXI67CTgK+C3QGfdLsCWwEhjupa8OKJDqvZ4C/AhkAsnAm8CUCqQ9FtgNXOGtGwkcBAaW8F785PEDoD6QCuwofO/AcGAZkAI0Aua4f4Wgx2kJ7AHqBOz7ByDTe325l0aA7sA+oJ237gJgfcC+coBu3vNHgdlAA6A5sLxY2muBE7zP5HovD8d56wYDs4vlcwrwoPf8Ii+P7YFawDPATD/nJsj7fwj4m/f8dC8f3b3P6G7vvCcDrYENwPFe2hZAS+/5V0A/73k9oHO0/xcS+WF3+gZgrqr+Q1ULVHWfqn6lqvNUNU9V1wKTga6lbP+Oqmar6kEgCxdsypv2MmCxqn7grZuAu0AE5TOPD6vqz6q6HhdgC491LTBBVXNUdTswrpTjrAW+wV2MAC4Edqpqtrf+H6q6Vp2ZwD+BoJW1xVwLPKSqP6nqBtzde+Bx31LVLd5n8jrugp3pY78A/YEXVHWxqu4HRgNdRSQlIE1J56Y0fYFpqjrT+4zGAUfjLr55uAtMa6+IcJ137sBdvFuJSCNV3a2q83y+DxMGFvQNwPeBL0TkNBH5SES2isguYAzQuJTttwY830vplbclpf1NYD5UVXF3xkH5zKOvY+HuUEvzOtDPe3497mJVmI/LRGSeiOwQkZ24u+zSzlWhE0rLg4gMFJElXjHKTuA0n/sF9/6K9qequ4CfgKYBacrzmZW03wLcZ9RUVVcA/4P7HH7wiguP95LeDKQBK0Rkvohc4vN9mDCwoG/A/dwP9Bzu7vZkVT0auB9XfBFOW3DFLQCIiHB4kCquMnncApwY8LqsJqVvAhd4d8pX4C4CiMhRwDvAw7iil2OAT33mY2tJeRCRlsCzwG1AI2+/3wXst6zmpZtxRUaF+6uHK0ba5CNf5dlvNdxntglAVaeo6lm4op0k3HlBVVeoal9cEd5jwLsiUquSeTEVZEHfBFMP+Bn4RUROB4ZG4JgfAhkicrmIVAdGAE3ClMe3gN+LSFMRaQSMKi2xqm4D5gIvAytUdZW3qiZQA8gF8kXkMuD8cuThbhE5Rlw/huEB6+riAnsu7vo3GHenX2gbkFJYcR3EG8AtItJORGrigu8XqlriL6dy5LmXiHTzjv0HXD3MPBE5XUTO8463z3vk497ADSLS2Ptl8LP33goqmRdTQRb0TTD/A9yE+4d+DnenG1ZeYL0OeBzYDpwELML1Kwh1Hp/Flb1/jatkfMfHNq/jKmZfD8jzTuBO4D1cZejVuIuXHw/gfnGsB6YDrwbsdykwEZjvpTkNCCwH/wxYBWwTkcBimsLtP8EVs7znbd8MV85fKaq6DHfOn8VdkHoAvbzy/ZrAeFw9zFbcL4t7vU0vAb4V1zrsUeA6VT1Q2fyYihFXdGpMbBGRJFxxwtWq+kW082NMVWF3+iZmiEgPEanvFRHch2sRMj/K2TKmSrGgb2LJ2cBaXBFBD6C3qpZUvGOMqQAr3jHGmARid/rGGJNAYm7AtcaNG2tqamq0s2GMMXFlwYIFP6pqac2cgRgM+qmpqWRnZ0c7G8YYE1dEpKye5YAV7xhjTEKxoG+MMQnEgr4xxiSQmCvTD+bgwYPk5OSwf//+aGfF+FCrVi1SUlJITi5paBhjTLTERdDPycmhXr16pKam4gZfNLFKVdm+fTs5OTm0aNGi7A2MMREVF8U7+/fvp1GjRhbw44CI0KhRI/tVZkyMiougD1jAjyP2WRkTu+Im6BtjTFU2dSq88QaEe2QcC/o+bN++nfbt29O+fXuOP/54mjZtWvT6wAF/w4LffPPNrFixotQ0Tz/9NFlZWaWm8evss89m8eLFIdmXMSa8Nm2CYcNg0qTwB/24qMgtr6wsuOce2LgRmjWDsWOhfyWmkGjUqFFRAH3wwQepW7cud91112Fpimaarxb8Ovryyy+XeZzbb7+94pk0xsQlVRgyBA4cgJdeghJCSMhUuTv9rCx3AjdscCdzwwb3OkQ30IdZvXo1bdq0YdiwYWRkZLBlyxaGDBlCZmYmrVu3ZsyYMUVpC++88/LyOOaYYxg9ejTp6emcccYZ/PDDDwDce++9PPHEE0XpR48eTadOnTj11FP5z3/+A8Avv/xCnz59SE9Pp1+/fmRmZpZ5Rz9lyhTatm1LmzZtuPvuuwHIy8vjhhtuKFo+ceJEACZMmEBaWhrp6ekMGDAg5OfMGHO4V1+Fjz+GcePgpJPCf7wqd6d/zz2wd+/hy/budcsrc7dfkuXLl/Pyyy8zadIkAMaNG0fDhg3Jy8vjvPPO4+qrryYtLe2wbX7++We6du3KuHHjGDlyJC+99BKjR48+Yt+qyvz585k2bRpjxozhk08+4amnnuL444/n3XffZcmSJWRkZJSav5ycHO69916ys7OpX78+F1xwAR9++CFNmjThxx9/5OuvvwZg586dAIwfP54NGzZQo0aNomXGmPDYtAlGjIBzzoHhw8tOHwpV7k5/48byLa+sk046id/+9rdFr9944w0yMjLIyMjg22+/Zfny5Udsc9RRR9GzZ08AOnbsyPr164Pu+6qrrjoizdy5c+nbty8A6enptG7dutT8zZs3j+7du9O4cWOSk5O5/vrrmTNnDieffDIrVqxgxIgRzJgxg/r16wPQunVrBgwYQFZWlnWuMiaMVGHo0MgV6xSqckG/WbPyLa+sOnXqFD1ftWoVTz75JDNnzmTp0qX06NEjaHv1GjVqFD1PSkoiLy8v6L5r1qx5RJryTnpTUvpGjRqxdOlSzj77bCZOnMjQoUMBmDFjBsOGDWP+/PlkZmaSn59fruMZY/x59VX46CN4+GE4+eTIHbfKBf2xY6F27cOX1a7tlofbrl27qFevHkcffTRbtmxhxowZIT/G2WefzVtvvQXA119/HfSXRKAuXbowa9Ystm/fTl5eHlOnTqVr167k5uaiqlxzzTX87//+LwsXLiQ/P5+cnBy6d+/OI488Qm5uLnuLl5UZYyqtsFjn7LPhd7+L7LF9lemLSA/gSSAJeEFVxxVbPxB4BNjkLfqrqr7grbsJuNdb/pCqvhKCfJeosNw+lK13/MrIyCAtLY02bdrQsmVLzjrrrJAf43e/+x033ngj7dq1IyMjgzZt2hQVzQSTkpLCmDFj6NatG6rK5ZdfzqWXXsrChQu55ZZbUFVEhL/85S/k5eVx/fXXs3v3bgoKChg1ahT16tUL+XswVVN+viuisL55pYtWsU5ABrTUBy7QrwFaAjWAJUBasTQDcYG++LYNcRNdNwQaeM8blHa8jh07anHLly8/YlmiOnjwoO7bt09VVVeuXKmpqal68ODBKOfqSPaZJZYNG1RPP131lFNUp05Vzc+Pdo5i1yuvqILqhAmh3S+QrWXEc1X1VbzTCVitqmtV9QAwFbjC5zXlYuAzVd2hqj8BnwE9fG5rgtizZw9nnXUW6enp9OnTh+eee47q1eO/EVZBAVxyCTz7bLRzYspr5UpXTLF5MyQnQ9++kJHhyqvD3dEo3mze7Ip1zjor8sU6hfxEi6bA9wGvc4DOQdL1EZFzgZXAnar6fQnbNi2+oYgMAYYANAtXjWsVccwxx7BgwYJoZyPkPvsMpk+HWbPgwgsjW7FlKm7xYrjoIvd89mxo29YNJ/DAA3DZZXDGGfB//wfduoU3H9984/riqMIVV0DnzlEoNilDYbHO/v3w8suQlBSdfPg5LcFK6Ipfv/8BpKpqO+BzoLDc3s+2qOpkVc1U1cwmTcqc19dUQZMmQaNGUKOG645ud4ix79//dsG8Vi2YOxfat3eBrH9/+PZbeO45V6923nnuQj5/fmiPv307/PWvkJnpLjaPPgqPPQZnngkpKXDbbfDpp67sPBZMmQIffugugq1aRS8ffoJ+DnBiwOsUYHNgAlXdrqq/ei+fBzr63daYnBz4xz/glltcr8R//hNeey3auTKlmTHDBfLjjnMB/5RTDl+fnOx6wq9a5QLx4sXu7vvKK91deUUdPAjTpsFVV8EJJ7gikvx8eOIJ1yImN9cF1zPPdE0iL74Yjj3WXYjeeQf27Knc+66oLVvgjjtcsc4dd0QnD0XKKvTHFQGtBVpwqCK3dbE0JwQ8vxL4Ug9V5K7DVeI28J43LO14VpFbNZTnM3vwQVextXq1qwA880zVRo1Uc3PDmEFTYW+/rZqcrNq+veq2bf622bVLdcwY1aOPVhVR7d/ffd5+LVqk+vvfqzZp4r4rxx6reuedqosXl7zN3r2q06ap3nyz+z6Bas2aqpdfrvrii5H7fhUUuGPWqqW6YkX4joPPitwyE7h9cQmurH4NcI+3bAzQy3v+MLDMuyDMAk4L2HYQsNp73FzWsSzoVw1+P7ODB1VTUlQvuujQsm++cUHlxhvDlDlTYS++qFqtmupZZ6n+9FP5t9++XXXUKNWjjlKtXl116FDVnJzgabdtU338cdX0dBepkpNV+/RxgfzAgfId9+BB1VmzVEeMUG3WzO2vWjXVrl1Vn3hCdf368r8Xv157zR3vscfCdwzVEAf9SD5iMeh37dpVP/nkk8OWTZgwQW+77bZSt6tTp46qqm7atEn79OlT4r6/+uqrUvczYcIE/eWXX4pe9+zZU3+qyH9cMQ888IA+8sgjld5PMH4/sw8+cN/Cv//98OX33uuWf/ZZGDJnKuSxx9xncvHFqgFfxwrZvFl1+HAXyGvWVB05UvWHH1T371d99113Z1y9ujteZqbqX/+q+uOPoXkfBQWqCxa471jr1u4YoNqhg2tGuXdvaI6j6t5ngwbu12teXuj2G4wF/RCaNGmSDhw48LBlnTt31jlz5pS6XWHQL42foN+8eXPNDcNv0VgI+j17qv7mN0feue3bp9qqlWrLlpUPMKZyCgpU77vPRYurr1b99dfQ7XvdOtWBA91dd926qg0buuOccILqH/+oumxZ6I5VkpUrVcePV+3UyR37+ONVJ05038HKKChQ7dXLFet8911o8loaC/oh9OOPP2rjxo11//79qqq6bt06PfHEE7WgoEB3796t3bt31w4dOmibNm30/fffL9quMOivW7dOW7duraqqe/fu1euuu07btm2r1157rXbq1Kko6A8bNkw7duyoaWlpev/996uq6pNPPqnJycnapk0b7datm6oefhF47LHHtHXr1tq6dWud4PX2WLdunZ522mk6ePBgTUtL0wsvvFD3Brl9CQz6ixYt0s6dO2vbtm21d+/eumPHjqLjn3766dq2bVu97rrrVFV19uzZmp6erunp6dq+fXvdtWvXEfv285mtXevKd723eoRZs9w3dPToMndlwiQ/X/V3v3Ofwy23hO9udfly1RtuUO3XT3X6dFccEw2zZ6uec457vykpqs8+W/GL3JQpbj+PPhraPJakygb9ESNcOVwoHyNGlH1CL7nkkqKA/vDDD+tdd92lqq6H7M8//6yqqrm5uXrSSSdpQUGBqgYP+o899pjefPPNqqq6ZMkSTUpKKgr627dvV1XVvLw87dq1qy5ZskRVj7zTL3ydnZ2tbdq00T179uju3bs1LS1NFy5cqOvWrdOkpCRdtGiRqqpec801+tprrx3xngKDftu2bXX27NmqqnrffffpCO+knHDCCUUXu8Iipcsuu0znzp2rqqq7d+8O2iPYT9D/05/cHd7GjSWnGTRINSlJ1TsVJoIOHnSBGFzxi/e1rvIKClyxYpcu7r2nprq6jPLUI2zZ4op1zjgj/MU6hfwG/RjrvhC7+vXrx9SpUwGYOnUq/fr1A9xF8+6776Zdu3ZccMEFbNq0iW3btpW4nzlz5hRNTtKuXTvatWtXtO6tt94iIyODDh06sGzZsjIHU5s7dy5XXnklderUoW7dulx11VV88cUXALRo0YL27dsDpQ/fDG58/507d9K1a1cAbrrpJubMmVOUx/79+zNlypSinr9nnXUWI0eOZOLEiezcubNCPYIPHIAXX3QdeE48seR0jzwCDRvCrbe6pnkmMvbvh2uucU1n//xn1wY+UcbUEYELLoD//MdNbtK4sWtOfPrp7nyU9T1UdX1N9u6NbiesksRd/31vYqmI6927NyNHjmThwoXs27evaPKSrKwscnNzWbBgAcnJyaSmpgYdTjmQBPnvWbduHY8++ihfffUVDRo0YODAgWXux13cgysclhnc0Mz79u0rdV8l+eijj5gzZw7Tpk3jz3/+M8uWLWP06NFceumlfPzxx3Tp0oXPP/+c0047rVz7ff99+OEH989RmoYN4ckn4frr4Zlnotd1PZHs2eN6tc6cCU89FbnJPWKNCPTsCT16uH4k998PN97oOlc9+KC7KAbr9fvGG/DBB+6G5dRTI57tMtmdvk9169alW7duDBo0qOguH9xd8rHHHktycjKzZs1iw4YNpe7n3HPPLZr8/JtvvmHp0qWAG5a5Tp061K9fn23btjF9+vSiberVq8fu3buD7uv9999n7969/PLLL7z33nucc8455X5v9evXp0GDBkW/El577TW6du1KQUEB33//Peeddx7jx49n586d7NmzhzVr1tC2bVtGjRpFZmYm3333XbmPOWkSpKYe6sJfmr593T/e3XfD99+Xnd5U3I4d7i73X/+CV15J3IAfSAR69YKFC10Hr6Qk951MT4e///3w3uNbt7obky5d4M47o5fn0ljQL4d+/fqxZMmSopmrAPr37092djaZmZlkZWWVecd72223sWfPHtq1a8f48ePp1KkT4GbB6tChA61bt2bQoEGHDcs8ZMgQevbsyXnnnXfYvjIyMhg4cCCdOnWic+fODB48mA4dOlTovb3yyiv84Q9/oF27dixevJj777+f/Px8BgwYQNu2benQoQN33nknxxxzDE888QRt2rQhPT39sFnA/PruOzfGzpAh/n76iri7/IICuP12G6IhXLZsga5dYdEiF9xuvDHaOYot1apBnz6wZAm8/rorouzTBzp2dL8ECot1fvklNot1ivgp+I/kIxZb75jyK+0zu/NO1wZ769by7fPRR13F2jvvVDJzRlVdRe2XX6o+/LDqhRe6DlN16qh+/nm0cxYfDh50wyS3bOm+l6ec4v6OHx+d/GAVuSYW7dsHf/ubGzvluOPKt+2IEdChg/v5bHO2l19BgRsD5/HH4fLLXX1Jly7wpz+5u/xbb3WVl+efH+2cxofq1d2voe++gxdegF9/dROcjxwZ7ZyVLu4qck18e/tt+Omnsitwg6leHZ5/Hjp1coHKxt4vnaoLSDNnusfs2a7MHtwAaddfD927u5Eyjz02mjmNb8nJrnXPoEHunMfakM7FxU3QV9WgrV5M7NFSCt0nTXIBp6Ljq3fs6O74J0xwIyeefXbF9lPc6tXuDjgvzx2jY0do08YNGxwvVGHtWldfMnOm+7t1q1vXrJmrjOze3Q11nJIS3bxWRSLx0aw1LoJ+rVq12L59O40aNbLAH+NUle3bt1MrSLRcsgT++18XXCvzMY4ZA+++6yqCFy2CgNap5bZ2LTz0kBuGt0YNt6/nn3frqld3gT8j49CFoF07OOqoih8vHHbscEVmkyfDihVu2fHHHwrw3btDixbxEZBM+MVF0E9JSSEnJ4fc3NxoZ8X4UKtWLVKC3Eo+95wLqjfdVLn9163rinYuvRTGj4f77iv/PjZsgLFjXSuL6tVdPcGoUa6eYf16WLDAPRYudG2uX3rJbZeUBGlp7gJQeDFIT4c6dSr3nspL1U1K8uyz8OabrjPVmWe6SUW6d4fTTrMgb4KT0n6KR0NmZqZmZ2dHOxsmxHbvht/8xlXgvvJK2en96NsX3nsPli713wkmJ8d1rnnhBRcUhw6F0aNd3kqi6maAWrjw0IVgwQLXuQxcGe5pp7kLwJlnuqDbqlV4gu4vv7jOP888437l1K0LAwa4OpL09NAfz8QPEVmgqpllprOgbyJh8mQXYP/zHzdvaihs3eq6xrdr58qvS6tA27wZHn7Y5UMVBg92lcGlDQFRGlU3U1PhBaDwUViGnpLiWsF07+7+Nj1iZujyWb7c1Ye8+ir8/LMrdrrtNhfwjz66cvs2VYPfoB/1dvnFH8Ha6Zv4VlDgxipv1y70g3Y9/7xrG/3CC8HXb9niZlyqVcv1Dbj11vBNmFFQ4GZGevZZNwRx4WxNoHrqqaq33eb6GHjj6pXp119Vp051gwKCao0aqtdfr/rFF4kz+Jnxj6o0yqaJb/PmuW/aM8+Eft/5+arnnqt6zDGHd/b64QfVu+5yHY6SktyUeWvWhP74ZeVt0SLXqaxnT9fxCdxw0h06uPxNn666e/fh223YoHrPParHHefSt2ihOm6ce0/GlMRv0LfiHRN2gwbBW2+5IpZwFEV8950rz77qKleR+eijbqCwfftcs8777nNl7NF28KCrfJ05003+/t//uq78yclu0vBu3VwLp48+cukvvdQV4Vx8cey3/TbRZ2X6cWbvXvjDH6B9e7j5ZteipCr46SdXnn3DDa71TriMGQMPPAC1a7tg36+fGxUxFkc5LLR3L/z73+4CMHOmqxNo3NjVNwwZAs2bRzuHJp74DfpVJLTEt4MH4dprD93hPfaYq3Ts3Tv+m9299poLwhXpgVseo0bB3LnQoIEL/mlp4T1eKNSuDRde6B7gWjjVquXu/I0JF/vRGGUFBTBwoAv4kya5ceZFXFHFmWeCN9pxXFJ176lTJzdmTjjVrAmffurarMdDwA+mXj0L+Cb8LOhHkaobUuD1113b8aFD3eQVX3/teoVu3AjnnusGx/rmm2jntvy++AK+/Tb8d/nGGP8s6EfRmDGu4nHkSNdBqFD16q5cd9UqV8zzxReuonLQoPiaRGTSJKhfH667Lto5McYUsqAfJU895aZcGziw5PlHa9d2F4M1a9wsPFlZrhXKH//oKkhj2Q8/uIk4brrJvQ9jTGywoB8FWVlwxx2uKOf558uurG3UyF0YVq50d82PPgotW7o5OCs49W3Y/e1vroJ66NBo58QYE8iCfoR99JG7u+/WDaZOLV/TzObN3bg1ixe7St4//tENU/zyy5CfH64cl19BgWueee658VupakxVZUE/gubOhauvduXzH3xQ8bHa27VzF49Zs9xAYYMGuX0WztMZbZ9/7oYsvu22aOfEGFOcBf0IWbIELrvMTWYxfXpoeqZ26wZffulmozpwwE2SceaZMGNGdIP/pEnQpAlceWX08mCMCc5X0BeRHiKyQkRWi8joUtJdLSIqIpne61QR2Scii73HpFBlPJ6sXu260terB5995gJiqIi4Xw/Llrlgu2kT9Ojhgv8nn0Q++G/aBNOmuV8flZncxBgTHmUGfRFJAp4GegJpQD8ROaKkVkTqAXcA84qtWqOq7b1HwrXY3rzZ9bjMy3Odh5o1C89xkpNdpenq1a48ffNm6NnTDWMcyeD/4ouufmHIkMgczxhTPn7u9DsBq1V1raoeAKYCVwRJ92dgPLA/hPmLazt2wEUXwY8/uiKd008P/zFr1HABd9UqF/y3bHHBv0sXl4dwBv+8PNca6eKLXesiY0zs8RP0mwKBXYJyvGVFRKQDcKKqfhhk+xYiskhE/iUi5wQ7gIgMEZFsEcmuKlMi/vKLGyVx1So3tMJvfxvZ4wcG/8mTYds2uOSS8Ab/jz92M1NZD1xjYpefoB+sFXlRyBCRasAE4H+CpNsCNFPVDsBI4HUROaIKU1Unq2qmqmY2CWWBd5QcOODGzpk/3zXLPP/86OWlRg249VbXxj8w+Hfu7IJ0ZYN/fr7rPDZ9Oowb51oTXXZZaPJujAk9P63Ec4DASeVSgM0Br+sBbYDZ4noZHQ9ME5FeqpoN/AqgqgtEZA1wClBlx07Oz3fDCH/6qSvfjpUWLIXB/6ab3JR7Dz3kfon89reuZ3DPniV3ElN1PWxXrnSPFSsOPV+zxl3kCk2cWHWGhTamKipzPH0RqQ6sBM4HNgFfAder6rIS0s8G7lLVbBFpAuxQ1XwRaQl8AbRV1R0lHS+ex9NXdW3Tn3sOxo934+PHqgMHXPAfOxbWr4fMTDckcdOmRwb2lSvdvKyFatRww0Gccop7nHrqoedV4IeaMXEpZOPpq2qeiAwHZgBJwEuqukxExuCm55pWyubnAmNEJA/IB4aVFvBjgaobPuDAAfj110N/A5+XtG7uXBfwR42K7YAPLnAPHnz4nf/llx+eplkzF9AHDDg8uDdrBklJ0cm3MaZyEn7mLFWYN88Fvrffdi1tKmPoUHj22fib/OTgwUNj+Z96Kpx8Mhx1VLRhI+0bAAARY0lEQVRzZYzxy2bOKsP69TBligv2q1a5ANe7tyu2qFHDdSwq62/xZbVrQ0pKtN9ZxSQnwzXXRDsXxphwS6igv2uXG+731VfhX/9yy847D+6+G/r0cT1mjTGmKqvyQT8vzw0A9uqr8N57sH+/K5ceOxb697fJp40xiaXKBv2lS12gz8qCrVuhYUO45Ra48UbXTDHeytyNMSYUqlTQ37rVzTf76qtuVMvkZNdR6MYbXYekGjWinUNjjImuKhP016xxxTYFBa636dNPu1mmGjWKds6MMSZ2VJmgXzh94KWXuiaHxhhjjlRlgr4IjBwZ7VwYY0xss5mzjDEmgVjQN8aYBGJB3xhjEogFfWOMSSAW9I0xJoFUmaCflQWpqVCtmvublRXtHBljTOypEk02s7LcfLB797rXGza41+DG1zHGGONUiTv9e+45FPAL7d3rlhtjjDmkSgT9jRvLt9wYYxJVlQj6zZqVb7kxxiSqKhH0x451s1YFql3bLTfGGHNIlQj6/fvD5MluQhQR93fyZKvENcaY4qpE6x1wAd6CvDHGlK5K3OkbY4zxx4K+McYkEAv6xhiTQCzoG2NMArGgb4wxCcSCvjHGJBBfQV9EeojIChFZLSKjS0l3tYioiGQGLPuTt90KEbk4FJk2xhhTMWW20xeRJOBp4EIgB/hKRKap6vJi6eoBdwDzApalAX2B1sBvgM9F5BRVzQ/dWzDGGOOXnzv9TsBqVV2rqgeAqcAVQdL9GRgP7A9YdgUwVVV/VdV1wGpvf8YYY6LAT9BvCnwf8DrHW1ZERDoAJ6rqh+Xd1tt+iIhki0h2bm6ur4wbY4wpPz9BX4Is06KVItWACcD/lHfbogWqk1U1U1UzmzRp4iNLxhhjKsLP2Ds5wIkBr1OAzQGv6wFtgNkiAnA8ME1EevnY1hhjTAT5udP/CmglIi1EpAauYnZa4UpV/VlVG6tqqqqmAl8CvVQ120vXV0RqikgLoBUwP+TvwhhjjC9l3umrap6IDAdmAEnAS6q6TETGANmqOq2UbZeJyFvAciAPuN1a7hhjTPSI6hFF7FGVmZmp2dnZ0c6GMcbEFRFZoKqZZaWzHrnGGJNALOgbY0wCsaBvjDEJxIK+McYkEAv6xhiTQCzoG2NMArGgb4wxCcSCvjHGJBAL+sYYk0As6BtjTAKxoG+MMQnEgr4xxiQQC/rGGJNALOgbY0wCsaBvjDEJxIK+McYkEAv6xhiTQCzoG2NMArGgb4wxCcSCvicrC1JToVo19zcrK9o5MsaY0Kse7QzEgqwsGDIE9u51rzdscK8B+vePXr6MMSbU7E4fuOeeQwG/0N69brkxxlQlFvSBjRvLt9wYY+KVBX2gWbPyLTfGmHhlQR8YOxZq1z58We3abrkxxlQlFvRxlbWTJ0Pz5iDi/k6ebJW4xpiqx1rvePr3tyBvjKn67E7fGGMSiK+gLyI9RGSFiKwWkdFB1g8Tka9FZLGIzBWRNG95qojs85YvFpFJoX4Dxhhj/CuzeEdEkoCngQuBHOArEZmmqssDkr2uqpO89L2Ax4Ee3ro1qto+tNk2xhhTEX7u9DsBq1V1raoeAKYCVwQmUNVdAS/rABq6LBpjjAkVP0G/KfB9wOscb9lhROR2EVkDjAfuCFjVQkQWici/ROScYAcQkSEiki0i2bm5ueXIvjHGmPLwE/QlyLIj7uRV9WlVPQkYBdzrLd4CNFPVDsBI4HUROTrItpNVNVNVM5s0aeI/98YYY8rFT9DPAU4MeJ0CbC4l/VSgN4Cq/qqq273nC4A1wCkVy6oxxpjK8hP0vwJaiUgLEakB9AWmBSYQkVYBLy8FVnnLm3gVwYhIS6AVsDYUGTfGGFN+ZbbeUdU8ERkOzACSgJdUdZmIjAGyVXUaMFxELgAOAj8BN3mbnwuMEZE8IB8Ypqo7wvFGjDHGlE1UY6uhTWZmpmZnZ0c7G+WWleWGYt640Q3UNnas9fA1xkSOiCxQ1cyy0tkwDCFgk7AYY+KFDcMQAjYJizEmXljQDwGbhMUYEy8s6IeATcJijIkXFvRDwCZhMcbECwv6IWCTsBhj4oW13gkRm4TFGBMP7E7fGGMSiAV9Y4xJIBb0jTEmgVjQN8aYBGJB3xhjEogF/RiRlQWpqVCtmvublRXtHBljqiJrshkDbMA2Y0yk2J1+DLAB24wxkWJBPwbYgG3GmEixoB8DbMA2Y0ykWNCPATZgmzEmUizoxwAbsM0YEynWeidG2IBtxphIsDt9Y4xJIBb0jTEmgVjQryKsR68xxg8r068CrEevMcYvu9OvAqxHrzHGLwv6VYD16DXG+GVBvwqwHr3GGL98BX0R6SEiK0RktYiMDrJ+mIh8LSKLRWSuiKQFrPuTt90KEbk4lJk3Tih69FpFsDGJocygLyJJwNNATyAN6BcY1D2vq2pbVW0PjAce97ZNA/oCrYEewDPe/kwIVbZHb2FF8IYNoHqoItgCvzFVj587/U7AalVdq6oHgKnAFYEJVHVXwMs6gHrPrwCmquqvqroOWO3tz4RY//6wfj0UFLi/5Wm1YxXBxiQOP002mwLfB7zOAToXTyQitwMjgRpA94Btvyy2bdMK5dSEjVUEG5M4/NzpS5BlesQC1adV9SRgFHBvebYVkSEiki0i2bm5uT6yZELJKoKNSRx+gn4OcGLA6xRgcynppwK9y7Otqk5W1UxVzWzSpImPLJlQsqGdjUkcfoL+V0ArEWkhIjVwFbPTAhOISKuAl5cCq7zn04C+IlJTRFoArYD5lc+2CSUb2tmYxFFmmb6q5onIcGAGkAS8pKrLRGQMkK2q04DhInIBcBD4CbjJ23aZiLwFLAfygNtVNT9M78VUgg3tbExiENUjitijKjMzU7Ozs6OdDVNOWVmutc/Gja4uYOxYu4gYE0kiskBVM8tKZwOumUqzAd+MiR82DIOpNGvnb0z8sKBvKi0U7fxtGAhjIsOCvqm0yrbzt2EgjIkcC/qm0irbzt+Kh4yJHAv6ptIq287fhoEwJnKs9Y4Jicq082/WzBXpBFtujAktu9M3UWfzARgTORb0TdTZfADGRI71yDVxLzU1ePFQ8+ZubgFjEoHfHrl2p2/inlUEG+OfBX0T92w+AGP8s6Bv4p5VBBvjnwV9E/esItgY/yzomyoh2hPD2y8FEy+sc5ZJeJWtCLahpU08sTt9k/AqWxFsvxRMPLGgbxJeZSuCQ/VLweoUTCRY0DcJr7IVwbHwS8EYvyzoG0PlKoKj/UvBmPKwoG9MJUX7l4Ix5WFB35gQiOYvBbCKYOOfBX1josw6l5lIsqBvTAywzmUmUizoGxPnYqHJqF004ocFfWPiXLSbjFrxUnyxoG9MnIt2k1HrZxBfLOgbE+ei3WQ0FP0MrHgocnwFfRHpISIrRGS1iIwOsn6kiCwXkaUi8k8RaR6wLl9EFnuPaaHMvDHGiWaT0cpeNKx4KLLKDPoikgQ8DfQE0oB+IpJWLNkiIFNV2wHvAOMD1u1T1fbeo1eI8m2MCZHK/lKo7EXDWh9Flp+hlTsBq1V1LYCITAWuAJYXJlDVWQHpvwQGhDKTxpjw6t+/4sNAF253zz2uSKdZMxfw/e7PhraOLD/FO02B7wNe53jLSnILMD3gdS0RyRaRL0Wkd7ANRGSIlyY7NzfXR5aMMbGkMsVL0W59BIn1S8FP0JcgyzRoQpEBQCbwSMDiZqqaCVwPPCEiJx2xM9XJqpqpqplNmjTxkSVjTFUR7dZHidZPwU/QzwFODHidAmwunkhELgDuAXqp6q+Fy1V1s/d3LTAb6FCJ/Bpjqphotz6KhX4KEb1oqGqpD1y5/1qgBVADWAK0LpamA7AGaFVseQOgpve8MbAKSCvteB07dlRjjPFryhTV2rVVXch1j9q13XI/RA7ftvAh4m/75s2Db9+8eWTyXwjI1jLiuaqWfaevqnnAcGAG8C3wlqouE5ExIlLYGucRoC7wdrGmmacD2SKyBJgFjFPV5RhjTIhE+5dCvHVuE3eBiB2ZmZmanZ0d7WwYYxJE8dY/4OoU/F44UlNdkU5xzZu7Su2yVKvm7u+LE3EV436JyAJ19aelH8//Lo0xpuqJdj+FSE+iY0HfGJPwKtPkNNoXjfLy0znLGGNMKaLZua28LOgbY0yUVeaiUV5WvGOMMQnEgr4xxiQQC/rGGJNALOgbY0wCsaBvjDEJJOZ65IpILhCkf5tvjYEfQ5SdcLD8VY7lr3Isf5UTy/lrrqplDlMcc0G/skQk209X5Gix/FWO5a9yLH+VE+v588OKd4wxJoFY0DfGmARSFYP+5GhnoAyWv8qx/FWO5a9yYj1/ZapyZfrGGGNKVhXv9I0xxpTAgr4xxiSQuAz6ItJDRFaIyGoRGR1kfU0RedNbP09EUiOYtxNFZJaIfCsiy0RkRJA03UTkZ29qycUicn+k8heQh/Ui8rV3/COmKhNnoncOl4pIRgTzdmrAuVksIrtE5PfF0kT0HIrISyLyg4h8E7CsoYh8JiKrvL8NStj2Ji/NKhG5KYL5e0REvvM+v/dE5JgSti31uxDG/D0oIpsCPsNLSti21P/3MObvzYC8rReRxSVsG/bzF1J+JtKNpQeQhJuEvSWHJmpPK5bm/wGTvOd9gTcjmL8TgAzveT1gZZD8dQM+jPJ5XA80LmX9JcB0QIAuwLwoft5bcR1PonYOgXOBDOCbgGXjgdHe89HAX4Js1xBY6/1t4D1vEKH8XQRU957/JVj+/HwXwpi/B4G7fHz+pf6/hyt/xdY/BtwfrfMXykc83ul3Alar6lpVPQBMBa4oluYK4BXv+TvA+SIikcicqm5R1YXe8924yeSbRuLYIXYF8Ko6XwLHiMgJUcjH+cAaVa1ML+1KU9U5wI5iiwO/Z68AvYNsejHwmaruUNWfgM+AHpHIn6p+qqp53ssvgZRQH9evEs6fH37+3yuttPx5seNa4I1QHzca4jHoNwW+D3idw5FBtSiN96X/GWgUkdwF8IqVOgDzgqw+Q0SWiMh0EWkd0Yw5CnwqIgtEZEiQ9X7OcyT0peR/tmifw+NUdQu4iz1wbJA0sXIeB+F+uQVT1nchnIZ7xU8vlVA8Fgvn7xxgm6quKmF9NM9fucVj0A92x1683amfNGElInWBd4Hfq+quYqsX4oor0oGngPcjmTfPWaqaAfQEbheRc4utj4VzWAPoBbwdZHUsnEM/YuE83gPkAVklJCnruxAuzwInAe2BLbgilOKifv6AfpR+lx+t81ch8Rj0c4ATA16nAJtLSiMi1YH6VOynZYWISDIu4Gep6t+Lr1fVXaq6x3v+MZAsIo0jlT/vuJu9vz8A7+F+Rgfyc57DrSewUFW3FV8RC+cQ2FZY5OX9/SFImqieR6/i+DKgv3oF0MX5+C6EhapuU9V8VS0Ani/huNE+f9WBq4A3S0oTrfNXUfEY9L8CWolIC+9OsC8wrViaaUBhK4mrgZklfeFDzSv/exH4VlUfLyHN8YV1DCLSCfc5bI9E/rxj1hGReoXPcRV+3xRLNg240WvF0wX4ubAoI4JKvMOK9jn0BH7PbgI+CJJmBnCRiDTwii8u8paFnYj0AEYBvVR1bwlp/HwXwpW/wDqiK0s4rp//93C6APhOVXOCrYzm+auwaNckV+SBa1myElerf4+3bAzuyw1QC1cksBqYD7SMYN7Oxv38XAos9h6XAMOAYV6a4cAyXEuEL4EzI3z+WnrHXuLlo/AcBuZRgKe9c/w1kBnhPNbGBfH6Acuidg5xF58twEHc3ectuHqifwKrvL8NvbSZwAsB2w7yvourgZsjmL/VuPLwwu9hYYu23wAfl/ZdiFD+XvO+W0txgfyE4vnzXh/x/x6J/HnL/1b4nQtIG/HzF8qHDcNgjDEJJB6Ld4wxxlSQBX1jjEkgFvSNMSaBWNA3xpgEYkHfGGMSiAV9Y4xJIBb0jTEmgfx/oW8jNdudu+AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(loss))\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The validation MAE stays in the low 0.40s-0.50s: we cannot even beat our common-sense baseline using the small convnet. Again, this is because \n",
    "our convnet looks for patterns anywhere in the input timeseries, and has no knowledge of the temporal position of a pattern it sees (e.g. \n",
    "towards the beginning, towards the end, etc.). Since more recent datapoints should be interpreted differently from older datapoints in the \n",
    "case of this specific forecasting problem, the convnet fails at producing meaningful results here. This limitation of convnets was not an \n",
    "issue on IMDB, because patterns of keywords that are associated with a positive or a negative sentiment will be informative independently \n",
    "of where they are found in the input sentences.\n",
    "\n",
    "One strategy to combine the speed and lightness of convnets with the order-sensitivity of RNNs is to use a 1D convnet as a preprocessing \n",
    "step before a RNN. This is especially beneficial when dealing with sequences that are so long that they couldn't realistically be processed \n",
    "with RNNs, e.g. sequences with thousands of steps. The convnet will turn the long input sequence into much shorter (downsampled) sequences \n",
    "of higher-level features. This sequence of extracted features then becomes the input to the RNN part of the network.\n",
    "\n",
    "验证 MAE 停留在 0.4~0.5，使用小型卷积神经网络甚至无法击败基于常识的基准方法。同样，这是因为卷积神经网络在输入时间序列的所有位置寻找模式，它并不知道所看到某个模式的时间位置（距开始多长时间，距结束多长时间等）。对于这个具体的预测问题，对最新数据点的解释与对较早数据点的解释应该并不相同，所以卷积神经网络无法得到有意义的结果。卷积神经网络的这种限制对于 IMDB 数据来说并不是问题，因为对于与正面情绪或负面情绪相关联的关键词模式，无论出现在输入句子中的什么位置，它所包含的信息量是一样的。\n",
    "\n",
    "要想结合卷积神经网络的速度和轻量与 RNN 的顺序敏感性，一种方法是在 RNN 前面使用一维卷积神经网络作为预处理步骤（见图 6-30）。对于那些非常长，以至于 RNN 无法处理的序列（比如包含上千个时间步的序列），这种方法尤其有用。卷积神经网络可以将长的输入序列转换为高级特征组成的更短序列（下采样）。然后，提取的特征组成的这些序列成为网络中 RNN 的输入。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This technique is not seen very often in research papers and practical applications, possibly because it is not very well known. It is very \n",
    "effective and ought to be more common. Let's try this out on the temperature forecasting dataset. Because this strategy allows us to \n",
    "manipulate much longer sequences, we could either look at data from further back (by increasing the `lookback` parameter of the data \n",
    "generator), or look at high-resolution timeseries (by decreasing the `step` parameter of the generator). Here, we will chose (somewhat \n",
    "arbitrarily) to use a `step` twice smaller, resulting in twice longer timeseries, where the weather data is being sampled at a rate of one \n",
    "point per 30 minutes.\n",
    "\n",
    "这种方法在研究论文和实际应用中并不多见，可能是因为很多人并不知道。这种方法非常有效，应该被更多人使用。我们尝试将其应用于温度预测数据集。因为这种方法允许操作更长的序列，所以我们可以查看更早的数据（通过增大数据生成器的 lookback 参数）或查看分辨率更高的时间序列（通过减小生成器的 step 参数）。这里我们任意地将 step 减半，得到时间序列的长度变为之前的两倍，温度数据的采样频率变为每 30 分钟一个数据点。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This was previously set to 6 (one point per hour).\n",
    "# Now 3 (one point per 30 min).\n",
    "# 之前设置为 6（每小时一个数据点），现在设置为 3（每 30 分钟一个数据点）\n",
    "\n",
    "step = 3\n",
    "lookback = 720  # Unchanged（保持不变）\n",
    "delay = 144 # Unchanged（保持不变）\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step)\n",
    "# This is how many steps to draw from `val_gen`\n",
    "# in order to see the whole validation set:\n",
    "# 为了查看整个验证集，需要 从 val_gen 中抽取多少次\n",
    "val_steps = (300000 - 200001 - lookback) // 128\n",
    "\n",
    "# This is how many steps to draw from `test_gen`\n",
    "# in order to see the whole test set:\n",
    "# 为了查看整个测试集，需要从 test_gen 中抽取多少次\n",
    "test_steps = (len(float_data) - 300001 - lookback) // 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is our model, starting with two `Conv1D` layers and following-up with a `GRU` layer:\n",
    "\n",
    "我们的模型：结合一维卷积基和 GRU 层的模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_6 (Conv1D)            (None, None, 32)          2272      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_4 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_7 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (None, 32)                6240      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 13,697\n",
      "Trainable params: 13,697\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 85s 169ms/step - loss: 0.3436 - val_loss: 0.2942\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 86s 173ms/step - loss: 0.3052 - val_loss: 0.2787\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 82s 164ms/step - loss: 0.2938 - val_loss: 0.2754\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 88s 177ms/step - loss: 0.2841 - val_loss: 0.2770\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 93s 185ms/step - loss: 0.2768 - val_loss: 0.2826\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 91s 182ms/step - loss: 0.2706 - val_loss: 0.2980\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 92s 185ms/step - loss: 0.2658 - val_loss: 0.2864\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 91s 182ms/step - loss: 0.2603 - val_loss: 0.2924\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 84s 167ms/step - loss: 0.2563 - val_loss: 0.2817\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 85s 171ms/step - loss: 0.2533 - val_loss: 0.2856\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 84s 169ms/step - loss: 0.2496 - val_loss: 0.2915\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 81s 162ms/step - loss: 0.2460 - val_loss: 0.2919\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 82s 165ms/step - loss: 0.2426 - val_loss: 0.2873\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 92s 183ms/step - loss: 0.2406 - val_loss: 0.2902\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 89s 177ms/step - loss: 0.2384 - val_loss: 0.2921\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 82s 163ms/step - loss: 0.2358 - val_loss: 0.2883\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 90s 181ms/step - loss: 0.2331 - val_loss: 0.2900\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 95s 190ms/step - loss: 0.2309 - val_loss: 0.2890\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 90s 180ms/step - loss: 0.2280 - val_loss: 0.2895\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 95s 189ms/step - loss: 0.2267 - val_loss: 0.2992\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GRU(32, dropout=0.1, recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VNX5wPHvyxpZBATciBBAXAIEiCOioqCiBRdQiwqEIi5FFKqWaqVCFVGqgkXEUgv6c6lEcStKrYobFVcw7CAiiwkEKASUJSJCkvf3x7kJQ5gkk8xkJpN5P88zz8y999x7z9xM3rnznnPPFVXFGGNMfKgR7QoYY4yJHAv6xhgTRyzoG2NMHLGgb4wxccSCvjHGxBEL+sYYE0cs6JtyEZGaIpIrIi3DWTaaRORkEQl732UR6SUimX7Ta0TkvGDKVmBfz4jIvRVdv5TtPiQiz4d7uyZ6akW7AqZyiUiu32Q94Bcg35u+RVXTy7M9Vc0HGoS7bDxQ1VPDsR0RuRkYrKo9/bZ9czi2bao/C/rVnKoWBV3vTPJmVf2wpPIiUktV8yJRN2NM5Fl6J855P99fEZGXRWQvMFhEzhaRr0Rkl4hsFZGpIlLbK19LRFREkrzpmd7yd0Vkr4h8KSKty1vWW95HRL4Tkd0i8qSIfC4iQ0uodzB1vEVE1onIjyIy1W/dmiLyuIjsFJH1QO9Sjs9YEZlVbN40EZnsvb5ZRFZ772e9dxZe0rayRaSn97qeiLzo1W0VcEaA/W7wtrtKRPp68zsCfwPO81JnO/yO7Ti/9Yd7732niLwpIicEc2zKIiJXevXZJSIfi8ipfsvuFZEtIrJHRL71e6/dRGSxN3+biEwKdn+mEqiqPeLkAWQCvYrNewg4AFyBOwk4CjgTOAv3S7AN8B0w0itfC1AgyZueCewAfEBt4BVgZgXKHgvsBfp5y0YBB4GhJbyXYOr4FtAISAJ+KHzvwEhgFZAINAXmu3+FgPtpA+QC9f22vR3wedNXeGUEuBD4GUjxlvUCMv22lQ309F4/BvwXaAK0Ar4pVvZa4ATvbzLIq8Nx3rKbgf8Wq+dMYJz3+hKvjp2BBODvwMfBHJsA7/8h4Hnv9elePS70/kb3ese9NtAeyAKO98q2Btp4r78GBnqvGwJnRft/IZ4fdqZvAD5T1X+raoGq/qyqX6vqAlXNU9UNwAygRynrv66qGap6EEjHBZvylr0cWKqqb3nLHsd9QQQUZB0fVtXdqpqJC7CF+7oWeFxVs1V1J/BIKfvZAKzEfRkBXAzsUtUMb/m/VXWDOh8DHwEBG2uLuRZ4SFV/VNUs3Nm7/35fVdWt3t/kJdwXti+I7QKkAc+o6lJV3Q+MBnqISKJfmZKOTWkGAHNU9WPvb/QIcDTuyzcP9wXT3ksRfu8dO3Bf3u1EpKmq7lXVBUG+D1MJLOgbgE3+EyJymoj8R0T+JyJ7gPFAs1LW/5/f632U3nhbUtkT/euhqoo7Mw4oyDoGtS/cGWppXgIGeq8H4b6sCutxuYgsEJEfRGQX7iy7tGNV6ITS6iAiQ0VkmZdG2QWcFuR2wb2/ou2p6h7gR6CFX5ny/M1K2m4B7m/UQlXXAH/A/R22e+nC472iNwDJwBoRWSgilwb5PkwlsKBvwP3c9zcdd3Z7sqoeDdyHS19Upq24dAsAIiIcHqSKC6WOW4GT/KbL6lL6CtDLO1Puh/sSQESOAl4HHsalXhoD7wdZj/+VVAcRaQM8BdwKNPW2+63fdsvqXroFlzIq3F5DXBppcxD1Ks92a+D+ZpsBVHWmqp6LS+3UxB0XVHWNqg7ApfD+CrwhIgkh1sVUkAV9E0hDYDfwk4icDtwSgX2+DaSKyBUiUgu4A2heSXV8FbhTRFqISFPgntIKq+o24DPgOWCNqq71FtUF6gA5QL6IXA5cVI463CsijcVdxzDSb1kDXGDPwX3/3Yw70y+0DUgsbLgO4GXgJhFJEZG6uOD7qaqW+MupHHXuKyI9vX3fjWuHWSAip4vIBd7+fvYe+bg38BsRaeb9MtjtvbeCEOtiKsiCvgnkD8D1uH/o6bgz3UrlBdbrgMnATqAtsAR3XUG46/gULve+AtfI+HoQ67yEa5h9ya/Ou4DfA7NxjaH9cV9ewbgf94sjE3gX+KffdpcDU4GFXpnTAP88+AfAWmCbiPinaQrXfw+XZpntrd8Sl+cPiaquwh3zp3BfSL2Bvl5+vy4wEdcO8z/cL4ux3qqXAqvF9Q57DLhOVQ+EWh9TMeJSp8ZULSJSE5dO6K+qn0a7PsZUF3amb6oMEektIo28FMGfcT1CFka5WsZUKxb0TVXSHdiASxH0Bq5U1ZLSO8aYCrD0jjHGxBE70zfGmDhS5QZca9asmSYlJUW7GsYYE1MWLVq0Q1VL6+YMVMGgn5SUREZGRrSrYYwxMUVEyrqyHLD0jjHGxBUL+sYYE0cs6BtjTBypcjl9Y0xkHTx4kOzsbPbv3x/tqpggJCQkkJiYSO3aJQ29VDoL+sbEuezsbBo2bEhSUhJucFNTVakqO3fuJDs7m9atW5e9QgDVJr2Tng5JSVCjhntOL9ftvo2JX/v376dp06YW8GOAiNC0adOQfpVVizP99HQYNgz27XPTWVluGiAt5LEFjan+LODHjlD/VtXiTH/MmEMBv9C+fW6+McaYQ6pF0N+4sXzzjTFVx86dO+ncuTOdO3fm+OOPp0WLFkXTBw4EN+z+DTfcwJo1a0otM23aNNLDlPft3r07S5cuDcu2Iq1apHdatnQpnUDzjTHhlZ7ufkVv3Oj+xyZMCC2N2rRp06IAOm7cOBo0aMBdd911WBlVRVWpUSPweepzzz1X5n5GjBhR8UpWI9XiTH/CBKhX7/B59eq5+caY8ClsP8vKAtVD7WeV0XFi3bp1dOjQgeHDh5OamsrWrVsZNmwYPp+P9u3bM378+KKyhWfeeXl5NG7cmNGjR9OpUyfOPvtstm/fDsDYsWOZMmVKUfnRo0fTtWtXTj31VL744gsAfvrpJ37961/TqVMnBg4ciM/nK/OMfubMmXTs2JEOHTpw7733ApCXl8dvfvObovlTp04F4PHHHyc5OZlOnToxePDgsB+zYAQV9L2bW6wRkXUiMjrA8uEiskJElorIZyKSXGx5SxHJFZG7iq8bDmlpMGMGtGoFIu55xgxrxDUm3CLdfvbNN99w0003sWTJElq0aMEjjzxCRkYGy5Yt44MPPuCbb745Yp3du3fTo0cPli1bxtlnn82zzz4bcNuqysKFC5k0aVLRF8iTTz7J8ccfz7Jlyxg9ejRLliwptX7Z2dmMHTuWefPmsWTJEj7//HPefvttFi1axI4dO1ixYgUrV65kyJAhAEycOJGlS5eybNky/va3v4V4dCqmzKDv3bZuGtAHSAYGFg/qwEuq2lFVO+Pukzm52PLHcfcBrTRpaZCZCQUF7tkCvjHhF+n2s7Zt23LmmWcWTb/88sukpqaSmprK6tWrAwb9o446ij59+gBwxhlnkJmZGXDbV1999RFlPvvsMwYMGABAp06daN++fan1W7BgARdeeCHNmjWjdu3aDBo0iPnz53PyySezZs0a7rjjDubOnUujRo0AaN++PYMHDyY9Pb3CF1eFKpgz/a7AOlXd4N3MeBbQz7+Aqu7xm6yPu9s9ACJyJe5uSKtCr64xJppKaierrPaz+vXrF71eu3YtTzzxBB9//DHLly+nd+/eAfur16lTp+h1zZo1ycvLC7jtunXrHlGmvDeVKql806ZNWb58Od27d2fq1KnccsstAMydO5fhw4ezcOFCfD4f+fn55dpfOAQT9FsAm/yms715hxGRESKyHnemf7s3rz5wD/BAaTsQkWEikiEiGTk5OcHW3RgTYdFsP9uzZw8NGzbk6KOPZuvWrcydOzfs++jevTuvvvoqACtWrAj4S8Jft27dmDdvHjt37iQvL49Zs2bRo0cPcnJyUFWuueYaHnjgARYvXkx+fj7Z2dlceOGFTJo0iZycHPYVz5VFQDC9dwJdCXDE15uqTgOmicggYCxwPS7YP66quaVdUKCqM4AZAD6fz+7faEwVVZg2DWfvnWClpqaSnJxMhw4daNOmDeeee27Y9/G73/2OIUOGkJKSQmpqKh06dChKzQSSmJjI+PHj6dmzJ6rKFVdcwWWXXcbixYu56aabUFVEhEcffZS8vDwGDRrE3r17KSgo4J577qFhw4Zhfw9lKfMeuSJyNjBOVX/lTf8JQFUfLqF8DeBHVW0kIp8CJ3mLGgMFwH2qWmILhs/nU7uJijGRs3r1ak4//fRoV6NKyMvLIy8vj4SEBNauXcsll1zC2rVrqVWravVuD/Q3E5FFquora91g3snXQDsRaQ1sBgYAg4rtrJ2qrvUmLwPWAqjqeX5lxgG5pQV8Y4yJptzcXC666CLy8vJQVaZPn17lAn6oynw3qponIiOBuUBN4FlVXSUi44EMVZ0DjBSRXsBB4EdcascYY2JK48aNWbRoUbSrUamC+gpT1XeAd4rNu8/v9R1BbGNceStnjDEmvKrFFbnGGGOCY0HfGGPiiAV9Y4yJIxb0jTFR1bNnzyMutJoyZQq33XZbqes1aNAAgC1bttC/f/8St11WF/ApU6YcdpHUpZdeyq5du4KpeqnGjRvHY489FvJ2ws2CvjEmqgYOHMisWbMOmzdr1iwGDhwY1Ponnngir7/+eoX3Xzzov/POOzRu3LjC26vqLOgbY6Kqf//+vP322/zyyy8AZGZmsmXLFrp3717Ubz41NZWOHTvy1ltvHbF+ZmYmHTp0AODnn39mwIABpKSkcN111/Hzzz8Xlbv11luLhmW+//77AZg6dSpbtmzhggsu4IILLgAgKSmJHTt2ADB58mQ6dOhAhw4dioZlzszM5PTTT+e3v/0t7du355JLLjlsP4EsXbqUbt26kZKSwlVXXcWPP/5YtP/k5GRSUlKKBnr75JNPim4i06VLF/bu3VvhYxtI9brqwBgTkjvvhHDfEKpzZ/DiZUBNmzala9euvPfee/Tr149Zs2Zx3XXXISIkJCQwe/Zsjj76aHbs2EG3bt3o27dvifeJfeqpp6hXrx7Lly9n+fLlpKamFi2bMGECxxxzDPn5+Vx00UUsX76c22+/ncmTJzNv3jyaNWt22LYWLVrEc889x4IFC1BVzjrrLHr06EGTJk1Yu3YtL7/8Mk8//TTXXnstb7zxRqnj4w8ZMoQnn3ySHj16cN999/HAAw8wZcoUHnnkEb7//nvq1q1blFJ67LHHmDZtGueeey65ubkkJCSU42iXzc70jTFR55/i8U/tqCr33nsvKSkp9OrVi82bN7Nt27YStzN//vyi4JuSkkJKSkrRsldffZXU1FS6dOnCqlWryhxM7bPPPuOqq66ifv36NGjQgKuvvppPP/0UgNatW9O5c2eg9OGbwY3vv2vXLnr06AHA9ddfz/z584vqmJaWxsyZM4uu/D333HMZNWoUU6dOZdeuXWG/ItjO9I0xRUo7I69MV155JaNGjWLx4sX8/PPPRWfo6enp5OTksGjRImrXrk1SUlLA4ZT9BfoV8P333/PYY4/x9ddf06RJE4YOHVrmdkobl6xwWGZwQzOXld4pyX/+8x/mz5/PnDlzePDBB1m1ahWjR4/msssu45133qFbt258+OGHnHbaaRXafiB2pm+MiboGDRrQs2dPbrzxxsMacHfv3s2xxx5L7dq1mTdvHlmBbobt5/zzzy+6+fnKlStZvnw54IZlrl+/Po0aNWLbtm28++6hezo1bNgwYN78/PPP580332Tfvn389NNPzJ49m/POO++IcmVp1KgRTZo0KfqV8OKLL9KjRw8KCgrYtGkTF1xwARMnTmTXrl3k5uayfv16OnbsyD333IPP5+Pbb78t9z5LY2f6xpgqYeDAgVx99dWH9eRJS0vjiiuuwOfz0blz5zLPeG+99VZuuOEGUlJS6Ny5M127dgXcXbC6dOlC+/btjxiWediwYfTp04cTTjiBefPmFc1PTU1l6NChRdu4+eab6dKlS6mpnJK88MILDB8+nH379tGmTRuee+458vPzGTx4MLt370ZV+f3vf0/jxo3585//zLx586hZsybJyclFdwELlzKHVo40G1rZmMiyoZVjTyhDK1t6xxhj4ogFfWOMiSMW9I0x5b4huImeUP9WFvSNiXMJCQns3LnTAn8MUFV27twZ0gVb1nvHmDiXmJhIdnY2OTk50a6KCUJCQgKJiYkVXt+CvjFxrnbt2rRu3Tra1TARYukdY4yJIxb0jTEmjljQN8aYOGJB3xhj4ogFfWOMiSMW9I0xJo5Y0DfGmDgSVNAXkd4iskZE1onI6ADLh4vIChFZKiKfiUiyN/9iEVnkLVskIheG+w0YY4wJXplBX0RqAtOAPkAyMLAwqPt5SVU7qmpnYCIw2Zu/A7hCVTsC1wMvhq3mxhhjyi2YM/2uwDpV3aCqB4BZQD//Aqq6x2+yPqDe/CWqusWbvwpIEJG6GGOMiYpghmFoAWzym84GzipeSERGAKOAOkCgNM6vgSWq+kuAdYcBwwBatmwZRJWMMcZURDBn+kfeZdg7kz9shuo0VW0L3AOMPWwDIu2BR4FbAu1AVWeoqk9Vfc2bNw+iSsYYYyoimKCfDZzkN50IbCmhLLj0z5WFEyKSCMwGhqjq+opU0hhjTHgEE/S/BtqJSGsRqQMMAOb4FxCRdn6TlwFrvfmNgf8Af1LVz8NTZWOMMRVVZtBX1TxgJDAXWA28qqqrRGS8iPT1io0UkVUishSX17++cD5wMvBnrzvnUhE5NvxvwxhjTDCkqt0tx+fzaUZGRrSrYYwxMUVEFqmqr6xydkWuMcbEEQv6xhgTRyzoG2NMHLGgb4wxccSCvjHGxBEL+sYYE0cs6BtjTByxoG+MMXHEgr4xxsQRC/rGGBNHLOgbY0wcsaBvjDFxxIK+McbEEQv6xhgTRyzoG2NMHLGgb4wxccSCvic9HZKSoEYN95yeHu0aGWNM+NWKdgWqgvR0GDYM9u1z01lZbhogLS169TLGmHCzM31gzJhDAb/Qvn1uvjHGVCcW9IGNG8s33xhjYpUFfaBly/LNN8aYWGVBH5gwAerVO3xevXpuvjHGVCcW9HGNtTNmQKtWIOKeZ8ywRlxjTPVjvXc8aWkW5I0x1Z+d6RtjTBwJKuiLSG8RWSMi60RkdIDlw0VkhYgsFZHPRCTZb9mfvPXWiMivwll5Y4wx5VNm0BeRmsA0oA+QDAz0D+qel1S1o6p2BiYCk711k4EBQHugN/B3b3vGGGOiIJgz/a7AOlXdoKoHgFlAP/8CqrrHb7I+oN7rfsAsVf1FVb8H1nnbM8YYEwXBBP0WwCa/6Wxv3mFEZISIrMed6d9eznWHiUiGiGTk5OQEW3djjKk23n8fZs+u/P0EE/QlwDw9YobqNFVtC9wDjC3nujNU1aeqvubNmwdRJWOMqT4+/hj69YO//AXy8yt3X8EE/WzgJL/pRGBLKeVnAVdWcF0TRZ98Atu3R7sWxsSXTz+FK66Ak0+Gd9+FmpXc6hlM0P8aaCcirUWkDq5hdo5/ARFp5zd5GbDWez0HGCAidUWkNdAOWBh6tU24ff459OwJF10Ee/dGuzbGxIcvv4RLL3VDvnz4ITRrVvn7LDPoq2oeMBKYC6wGXlXVVSIyXkT6esVGisgqEVkKjAKu99ZdBbwKfAO8B4xQ1Ur+8WLK6+BBGD4cmjeH1athyBAoKIh2rYyp3jIyoHdvOP54+OgjOO64yOxXVI9IsUeVz+fTjIyMaFcjrjz2GNx9t2tEysqCO++E++6DBx6Ids2MqZ6WLIELL4QmTVxa9aSTyl6nLCKySFV9ZZWzYRji3KZNMG4cXH65a0gCWLYMxo+HDh3gmmuiWj1jqp0VK+Dii6FhQ9eAG46AXx42DEOYxOrtFu+4w6VynnzSDTYnAk89BWefDUOHwtKl0a6hMdXH6tXQqxfUrQvz5rlYEWnVKujv3BmdXHTh7RazskD10O0Wq3rg/89/XErnvvsO//DVrQv/+hcccwxceSXYpRPGhO6771xKR8Sd4bdtG516VJug/9130K4dvPBC5Pcdi7db3LcPRo6E5GQYNerI5ccfD2++Cdu2Qf/+cOBA5OtoTHWxYYML+Pn5LuCfemr06lJtgv7JJ7sAdvfd7ow/kmLxdosPPQSZmfD3v0OdOoHLnHEGPPsszJ8Pt98euIwxpnRZWS7g//yz65aZXHzksgirNkG/Rg2Xi961C/70p8juO9Zut7h6teuxM2QI9OhRetmBA2H0aJg+3R1fY0zwsrNdwN+9Gz74AFJSol2jahT0ATp2dN0Nn37aXfQQKbF0u0VVuO02aNAAJk0Kbp2HHoLLLnNn+598Ujn1+vJL6N7dfRnFG1WXbtu+vfIvwa8MqrB+Pbz+urvIr4r1Ao+arVvdxY45OTB3LqSmRrtGTrXrp5+bC6ed5q5sy8iAWhHqlJqe7nL4Gze6M/wJE6rmnbhefNGd4U+f7hqbg7V7N3Tr5j7AX38NrVuHpz67d7tfZv/4h2tA3r/fpZRuuCE824+kFSvgm2/cFc2Fj9zc4KYLOyAkJblfVkOHuuNR1RQUwLp1sGgRLF586Hn37kNlkpPdZ2vIENcPPR5t3+6ucN+40Q2kds45lb/PYPvpV7ugD/DGG67x8fHH3Zm/cX780TUgtW3rzshqlPN33tq10LWr61f8xRfu10JFqbq/0+23u8bi22+HP/8ZBgxwDV1vv+2uVowFO3e6QP3MM0cuS0hw/bEbNnTHq/B1oOm6dWHWLFiwAE48Ef74R/jtb4/8FRkp+fmwZs3hwX3JkkPDdNSp49IVZ5zhzmK7dHFffNOnw8KF7r1fcw3ccosLehJo+MVqaOdOuOAC9+X43ntw/vmR2W+wQR9VrVKPM844Q0NVUKDap49qgwaq2dkhb67auOUW1Ro1VJcsqfg25s5127j6atX8/IptIytL9fLLVUG1SxfVjIxDy/bsUe3cWbV+/cPnV0UFBarPPafarJlqzZqqf/iD6sqVqhs3qv74o+rBgxXb5gcfqPbo4Y5P8+aqDz+sunt3uGt/pO++U33+edXf/U71nHNU69VzdQDVo45S7dZNdcQI1f/7P9WlS1UPHCh5W0uWqN56q2rDhm799u1Vn3hC9YcfKv99RNMPP7jPdEKC6ocfRnbfQIYGEWOjHuSLP8IR9FVV161zB/7aa8OyuZj31VeqIqp33hn6tiZPdp+ccePKt97Bg27d+vVdQPnrXwMHxi1bVFu1Uj3uONUNG0Kvb2VYuVL1vPPccTjnHNVly8K/j08/Ve3d2+2jcWPV++9X3bkzfNvPy3P7uPtu1VNPPRTg69dX7d5d9fbbVV94QXXFiop9gamq7t2r+vTTqmee6badkKB6/fWqX3zhvuAqqqBANTNT9a23VMePV/31r1XbtXOfm+7dVQcMcO9r6lTV2bPdCcS2baHtsyy7drn3WaeO6rvvVt5+ShJs0K+W6Z1CDz7oLjyaOxcuuSQsm4xJeXlw5pkuH796tUslhELV5dxfeMGlaK6+uux1Fi1yed7Fi12j8LRp0KpVyeVXr4Zzz3WDwH3xBTRtGlqdw+Wnn9zn6q9/haOPhokT3bEob6qsPDIyXBvRm2+6lNBtt7lrKyoyQFdurssxz5njLs7bsQNq13bpiL59XU+TU06pnOF9lyxxqZ/0dFePjh3dZ2LwYGjcuOT19u+HVavc8CD+j127DpVp2xY6dXKpsE2b3CM7+8jrS+rUgcREl6L0fxTOS0iAPXtcG8WePYe/Lv5cfN7eva4N8V//ckMlR1pc5/QL/fKL+2CpulxjQkJYNhtznnjCtW289ppr6wiH/ftdQ9WKFa7nTUld0XJzXa5+6lQ49lj33L9/cPndzz93vR9SU90ohEcdFZ66V9S//w2/+53rd33DDfDoo+5LKVJWrHA32XjlFZf/HzbMXZeSmFj6eps3uzaSOXPccfzlFxdkL7vMBfrevd0XWKTk5sLLL8OMGe4L7aij4LrrXO4/KenwwL50qWtXKOzVVL+++5/u1OnQo2PHwCcyBQXuS63wS8D/y6Dw9ebN7qQoGPXru+PUqNHhz/6vL77Y9UKLhrjO6ft7/333s/KBB8K62ZiRne3yqr17h/+n7ZYtqieeqJqUpJqTc+Tyf/9btWVLd/yHD3d57vJ64w2XlrrySpeOiIasLNV+/Q7lpufPj049Cn37rerQoa4doXZt1WHDVNevP7S8oMDl3MePV/X5DqVt2rRR/f3vVefNKz0fH0mLFrn6N2hwqJ6Fj5YtVa+4QnXsWNXXXnNtDhVtRypJXp7q5s2qCxaovv66anq66ttvu7TXsmUuhfTDDxVPb0US8Z7T93fddap166quXRv2TVd511zj8qjr1lXO9hcudMe2Z89DgWTzZtX+/Q8Fyc8/D20fU6e6bY0YUbk52eIOHFCdONG1P9Srp/roo1UnWKqqfv+9+zKtU8d9AfzmN6ojRx76ohVxja9/+Ytrg4jksSuvPXtcA/GUKar//W/1b/CtDBb0/Wze7M52f/Wrqv3BD7f33nN/4fHjK3c/M2e6/dx6q+rf/6569NHui2DCBNVffgnPPu6+2+3jkUfCs72yfPaZaocObp99+7ozvqoqO9udwR91lHv07av6zDOqW7dGu2YmkizoFzNlinu3r71WKZuvcvbtU23bVvWUU1T376/8/RUGZVC96KLw/6rKz1cdONBtf+bM8G7bX06O6o03alF64c03K29f4bZ3r+pPP0W7FiZagg36cXMTlREj4PnnXYPmr34Veg+Wqu6RR9yl8R99FJkrOx9+2PUCSU6GQYPCfyFOjRrw3HPwv/+5RtTjj3eNvOGybx/8858wdqzrifHHP7qeX/Xrh28flS2Ui+VMHAnmmyGSj8o601dV/fJLl+f8wx8qbRcVNnOm62Ms4p5DOZtds8bleQcNClftqo5du1Q7dnTpuqVLQ9/emjUuNdK4sTu7797d9Us3JtZg6Z3Ahg1zjV6VcTFNRc2cefjVj+CmKxL4CwpceqVRo+qb0920STUx0fUcysoq//oHD7q0zcUXu2Ndq5Zr7P8UgGdLAAARTUlEQVTkk/hq8zHVS7BBv1r30w/khx/c+DOnnAKfflq5F9UEKynJ9f0urlUrN+Z9ebz8skuv/O1vLqVVXa1c6fpDn3ii688fzMBe27a58XGmT3d9tBMTXd/wm2926SJjYlmw/fSrQMiLrGOOcUMKf/GFy/FXBeG6Ccvu3e5KTZ8Phg8PvV5VWYcO7grV9evdDd337w9cTtV9uQ8c6K64HDvWjcI6ezZ8/72btoBv4kncBX1wQ7527+4a6yJ9l61AwnETFlW49143pOs//lE5l9FXNT17uqEgPv3U/U3974+8d6+76UtKihvl8L333O0h16xxwxBceWXkht02piqJy6Dvf5et0aOjXZvQb8KyZg306eNufThypBvqNl4MGOBuvPLaa3DXXS7tM2KES/vcdpvrUfTMM+5y+8mTXVrPmLgWTOI/ko/Kbsj1d9ddriHviy8itssSVaT3zp49qn/8o7sU/+ij3bUIsXC5eLgVFKjeccehRvC6dVWHDHEji1rDrIkXhLMhV0R6A08ANYFnVPWRYstHATcDeUAOcKOqZnnLJgKX4X5VfADcoaXstLIbcv3l5sLpp7sRHCN5l61QqboG27vvhi1bXL/1hx+u2KiL1UVBAdx/vxv06oYb3J3TjIknYWvIFZGawDSgD5AMDBSR4vdzXwL4VDUFeB2Y6K17DnAukAJ0AM4EyrgVd+Q0aOBGoFy2zPV2iQXLl7ubmaelwQknuBEun302vgM+uJTdgw+6L0IL+MaULJicfldgnapuUNUDwCygn38BVZ2nqvu8ya+AwsFeFUgA6gB1gdrAtnBUPFyuugouvdQN/7t5c7RrU7Iff3TD+nbp4u7DOmOGu61et27RrpkxJpYEE/RbAJv8prO9eSW5CXgXQFW/BOYBW73HXFVdXXwFERkmIhkikpGTkxNs3cNCBJ580o2pPWpURHcdlIIC1xB5yimuofbWW+G779y9U+Ohh44xJryCCfqBRlEJmJMXkcGAD5jkTZ8MnI47828BXCgiR9wmWFVnqKpPVX3NI3lXCk+bNjBmDLz6qgv8y5e7vHm0LVzozuR/+1vXt3zxYpeGOuaYaNfMGBOrggn62cBJftOJwJbihUSkFzAG6Kuqv3izrwK+UtVcVc3F/QKokgmJu++Ga691Z/2Fd+P5y1/cBTyRtn073HQTnHWWu8tPejrMn+/qVZL0dHdlb40a7jk9PVK1NcbEkmCC/tdAOxFpLSJ1gAHAHP8CItIFmI4L+Nv9Fm0EeohILRGpjWvEPSK9UxXUretuQ7d1q0ujNGnizv7btHH3ap02zQXjypSX524neMop8OKL7otozZqyR61MT3e3zsvKcr9QsrLctAV+Y0xxwXbZvBSYguuy+ayqThCR8bh+oXNE5EOgIy5vD7BRVft6PX/+DpyPSwm9p6qlZs4j2WWzLFlZMGsWvPSSS/nUrOnugTlokLuiM5ThmfPzYcMGd8PnlSvd84IF7pfFJZe4XkWnnRbctsI5do8xJjbZjdHDbOVK1zf+pZdcID3qKHfH+7Q0d2PpOnUCr1dQ4MbQKQzshc+rVx8+Xkzr1tC+vUvr9OtXvvHoa9QI3AYhcvjQBMaY6suCfiVRdX3jX3rJpYN27HCpoP79XZtAfv7hAf6bb+Cnnw6tn5jognuHDoeeTz89tBtg2Jm+McaCfgQcPOjuTPXSS27UxtzcQ8uOO+7wwN6+vburVOPG4a9HYU5/375D8+rVc33509LCvz9jTNUTbNCPkYEHqqbatV1qp3dvF3A//tjl+du3j+xVoYWBfcwYl0pq2dIN1mYB3xhTnJ3pG2NMNWA3UTHGGHMEC/rGGBNHLOgbY0wcsaBvABvGwZh4Yb13zBFdPguHcQDrAWRMdWNn+oYxYw7v4w9uesyY6NTHGFN5LOgbNm4s33xjTOyyoG9o2bJ8840xscuCvmHCBDdsg7969dx8Y0z1YkHfkJbmxulp1cqNzNmqlY3bY0x1Zb13DOACvAV5Y6o/O9M3YWH9/I2JDXamb0Jm/fyNiR12pm9CZv38jYkdFvRNyKyfvzGxw4K+CZn18zcmdljQNyGzfv7GxA4L+iZk1s/fmNhhvXdMWFg/f2Nig53pG2NMHAkq6ItIbxFZIyLrRGR0gOWjROQbEVkuIh+JSCu/ZS1F5H0RWe2VSQpf9U11YRd3GRMZZQZ9EakJTAP6AMnAQBFJLlZsCeBT1RTgdWCi37J/ApNU9XSgK7A9HBU31UfhxV1ZWaB66OIuC/zGhF8wZ/pdgXWqukFVDwCzgH7+BVR1nqoWXp7zFZAI4H051FLVD7xyuX7ljAHs4i5jIimYoN8C2OQ3ne3NK8lNwLve61OAXSLyLxFZIiKTvF8OxhSxi7uMiZxggr4EmKcBC4oMBnzAJG9WLeA84C7gTKANMDTAesNEJENEMnJycoKokqlO7OIuYyInmKCfDZzkN50IbCleSER6AWOAvqr6i9+6S7zUUB7wJpBafF1VnaGqPlX1NW/evLzvwcQ4u7jLmMgJJuh/DbQTkdYiUgcYAMzxLyAiXYDpuIC/vdi6TUSkMJJfCHwTerVNdWIXdxkTOWUGfe8MfSQwF1gNvKqqq0RkvIj09YpNAhoAr4nIUhGZ462bj0vtfCQiK3Cpoqcr4X2YGJeWBpmZUFDgnssb8K3LpzHBEdWA6fmo8fl8mpGREe1qmBhSfDx/cOkh+7Vg4omILFJVX1nl7IpcE/Osy6cxwbOgb2Kedfk0JngW9E3Msy6fxgTPgr6JeeHo8mkNwSZeWNA3MS/ULp829o+JJ9Z7x8S9pCQX6Itr1cp1HzUmFljvHWOCZA3BJp5Y0DdxzxqCTTyxoG/ino39Y+KJBX0T98Ix9o/1/jGxwm6Mbgyh3di9+DAQhb1/CrdrTFViZ/rGhMiGgTCxxIK+MSGy3j8mlljQNyZE1vvHxBIL+saEyHr/mFhiQd+YEFnvHxNLrPeOMWFgvX9MrLAzfWOizHr/mEiyoG9MlFnvHxNJFvSNiTLr/WMiyYK+MVFmvX9MJFnQNybKrPePiSTrvWNMFWC9f0yk2Jm+MTHOev+Y8rCgb0yMs94/pjyCCvoi0ltE1ojIOhEZHWD5KBH5RkSWi8hHItKq2PKjRWSziPwtXBU3xjjh6P1jbQLxo8ygLyI1gWlAHyAZGCgiycWKLQF8qpoCvA5MLLb8QeCT0KtrjCku1N4/hW0CWVmgeqhNwAJ/9RTMmX5XYJ2qblDVA8AsoJ9/AVWdp6qFWcWvgMTCZSJyBnAc8H54qmyM8Rdq7x9rE4gvwQT9FsAmv+lsb15JbgLeBRCRGsBfgbtL24GIDBORDBHJyMnJCaJKxhh/aWmQmQkFBe65PL12wtEmYOmh2BFM0JcA8zRgQZHBgA+Y5M26DXhHVTcFKl+0MdUZqupTVV/z5s2DqJIxJlxCbROw9FBsCSboZwMn+U0nAluKFxKRXsAYoK+q/uLNPhsYKSKZwGPAEBF5JKQaG2PCKtQ2AUsPxZZgLs76GmgnIq2BzcAAYJB/ARHpAkwHeqvq9sL5qprmV2YorrH3iN4/xpjoKUwFjRnjUjotW7qAH2yKyLqMxpYyg76q5onISGAuUBN4VlVXich4IENV5+DSOQ2A10QEYKOq9q3EehtjwiiUK4JbtnQpnUDzTdUTVD99VX1HVU9R1baqOsGbd58X8FHVXqp6nKp29h5HBHxVfV5VR4a3+saYaAvHgHHWEBw5dkWuMSYkoXYZtYbgyBLVgB1xosbn82lGRka0q2GMiZCkpMDpoVatXPdTExwRWaSqvrLK2Zm+MSaqrCE4sizoG2OiysYOiiwL+saYqLKxgyLLgr4xJqps7KDIsoZcY0xMq1HDneEXJ+LGIooX1pBrjIkL1iZQPhb0jTExzdoEyseCvjEmplmbQPlYTt8YE9eqS5uA5fSNMSYI4WgTiCUW9I0xcS3eBoyzoG+MiWvxNmCcBX1jTNwL5R7D4WgIjuQvhWDunGWMMaYEoQ4YV/hLofCLo/CXAlT8xjalsTN9Y4wJQagNwZHuMmpB3xhjQhBqQ3Ckh5a2oG+MMSEItSE40l1GLegbY0yIQmkIDkeX0fKwoG+MMVEU6i+F8rLeO8YYE2VpaZUX5IuzM31jjIkjFvSNMSaOWNA3xpg4YkHfGGPiiAV9Y4yJI1XuJioikgNkhbCJZsCOMFWnMlj9QmP1C43VLzRVuX6tVLV5WYWqXNAPlYhkBHP3mGix+oXG6hcaq19oqnr9gmHpHWOMiSMW9I0xJo5Ux6A/I9oVKIPVLzRWv9BY/UJT1etXpmqX0zfGGFOy6nimb4wxpgQW9I0xJo7EZNAXkd4iskZE1onI6ADL64rIK97yBSKSFMG6nSQi80RktYisEpE7ApTpKSK7RWSp97gvUvXzq0OmiKzw9p8RYLmIyFTvGC4XkdQI1u1Uv2OzVET2iMidxcpE9BiKyLMisl1EVvrNO0ZEPhCRtd5zkxLWvd4rs1ZEro9g/SaJyLfe32+2iDQuYd1SPwuVWL9xIrLZ7294aQnrlvr/Xon1e8WvbpkisrSEdSv9+IWVqsbUA6gJrAfaAHWAZUBysTK3Af/wXg8AXolg/U4AUr3XDYHvAtSvJ/B2lI9jJtCslOWXAu8CAnQDFkTx7/0/3IUnUTuGwPlAKrDSb95EYLT3ejTwaID1jgE2eM9NvNdNIlS/S4Ba3utHA9UvmM9CJdZvHHBXEH//Uv/fK6t+xZb/FbgvWscvnI9YPNPvCqxT1Q2qegCYBfQrVqYf8IL3+nXgIhGRSFROVbeq6mLv9V5gNdAiEvsOs37AP9X5CmgsIidEoR4XAetVNZSrtEOmqvOBH4rN9v+cvQBcGWDVXwEfqOoPqvoj8AHQOxL1U9X3VTXPm/wKSAz3foNVwvELRjD/7yErrX5e7LgWeDnc+42GWAz6LYBNftPZHBlUi8p4H/rdQNOI1M6Pl1bqAiwIsPhsEVkmIu+KSPuIVsxR4H0RWSQiwwIsD+Y4R8IASv5ni/YxPE5Vt4L7sgeODVCmqhzHG3G/3AIp67NQmUZ66adnS0iPVYXjdx6wTVXXlrA8msev3GIx6Ac6Yy/e7zSYMpVKRBoAbwB3quqeYosX49IVnYAngTcjWTfPuaqaCvQBRojI+cWWV4VjWAfoC7wWYHFVOIbBqArHcQyQB6SXUKSsz0JleQpoC3QGtuJSKMVF/fgBAyn9LD9ax69CYjHoZwMn+U0nAltKKiMitYBGVOynZYWISG1cwE9X1X8VX66qe1Q113v9DlBbRJpFqn7efrd4z9uB2bif0f6COc6VrQ+wWFW3FV9QFY4hsK0w5eU9bw9QJqrH0Ws4vhxIUy8BXVwQn4VKoarbVDVfVQuAp0vYb7SPXy3gauCVkspE6/hVVCwG/a+BdiLS2jsTHADMKVZmDlDYS6I/8HFJH/hw8/J//wesVtXJJZQ5vrCNQUS64v4OOyNRP2+f9UWkYeFrXIPfymLF5gBDvF483YDdhamMCCrxDCvax9Dj/zm7HngrQJm5wCUi0sRLX1zizat0ItIbuAfoq6r7SigTzGehsurn30Z0VQn7Deb/vTL1Ar5V1exAC6N5/Cos2i3JFXngepZ8h2vVH+PNG4/7cAMk4FIC64CFQJsI1q077ufncmCp97gUGA4M98qMBFbheiJ8BZwT4ePXxtv3Mq8ehcfQv44CTPOO8QrAF+E61sMF8UZ+86J2DHFfPluBg7izz5tw7UQfAWu952O8sj7gGb91b/Q+i+uAGyJYv3W4fHjh57CwR9uJwDulfRYiVL8Xvc/WclwgP6F4/bzpI/7fI1E/b/7zhZ85v7IRP37hfNgwDMYYE0diMb1jjDGmgizoG2NMHLGgb4wxccSCvjHGxBEL+sYYE0cs6BtjTByxoG+MMXHk/wGmtBd0hqr5dAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(loss))\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Judging from the validation loss, this setup is not quite as good as the regularized GRU alone, but it's significantly faster. It is \n",
    "looking at twice more data, which in this case doesn't appear to be hugely helpful, but may be important for other datasets.\n",
    "\n",
    "从验证损失来看，这种架构的效果不如只用正则化 GRU，但速度要快很多。它查看了两倍的数据量，在本例中可能不是非常有用，但对于其他数据集可能非常重要。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wrapping up\n",
    "\n",
    "Here's what you should take away from this section:\n",
    "\n",
    "* In the same way that 2D convnets perform well for processing visual patterns in 2D space, 1D convnets perform well for processing \n",
    "temporal patterns. They offer a faster alternative to RNNs on some problems, in particular NLP tasks.\n",
    "* Typically 1D convnets are structured much like their 2D equivalents from the world of computer vision: they consist of stacks of `Conv1D` \n",
    "layers and `MaxPooling1D` layers, eventually ending in a global pooling operation or flattening operation.\n",
    "* Because RNNs are extremely expensive for processing very long sequences, but 1D convnets are cheap, it can be a good idea to use a 1D \n",
    "convnet as a preprocessing step before a RNN, shortening the sequence and extracting useful representations for the RNN to process.\n",
    "\n",
    "One useful and important concept that we will not cover in these pages is that of 1D convolution with dilated kernels.\n",
    "\n",
    "## 小结\n",
    "下面是你应该从本节中学到的要点。\n",
    "* 二维卷积神经网络在二维空间中处理视觉模式时表现很好，与此相同，一维卷积神经网络在处理时间模式时表现也很好。对于某些问题，特别是自然语言处理任务，它可以替代 RNN，并且速度更快。\n",
    "* 通常情况下，一维卷积神经网络的架构与计算机视觉领域的二维卷积神经网络很相似， 它将 Conv1D 层和 MaxPooling1D 层堆叠在一起，最后是一个全局池化运算或展平操作。\n",
    "* 因为 RNN 在处理非常长的序列时计算代价很大，但一维卷积神经网络的计算代价很小， 所以在 RNN 之前使用一维卷积神经网络作为预处理步骤是一个好主意，这样可以使序列变短，并提取出有用的表示交给 RNN 来处理。\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}