4. Tensorflow first learning models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression\n",
    "\n",
    "Linear regression model is one of the simplest regression models. It assumes linear relationship between $x$ and $y$. The output equation is defined as follows:\n",
    "\n",
    "$$\\hat{y} = Wx + b$$\n",
    "\n",
    "The *Advertising data set* (from \"*An Introduction to Statistical Learning*\", textbook by Gareth James, Robert Tibshirani, and Trevor Hastie) consists of the sales of a product in 200 different markets, along with advertising budgets for the product in each of those markets for three different media: TV, radio, and newspaper. \n",
    "\n",
    "Objective: to determine if there is an association between advertising and sales, then we can instruct our client to adjust advertising budgets, thereby indirectly increasing sales. \n",
    "\n",
    "We want to train an **inference model**, a series of mathematical expressions we want to apply to our data that depends on a series of parameters. The values of parameters change through training in order for the model to learn and adjust its output.\n",
    "\n",
    "The training loop is:\n",
    "+ Initialize the model parameters to some values.\n",
    "+ Read the training data (for each example), possibly using randomization strategies in order to assure that training is stochastic.\n",
    "+ Execute the inference model on the training data, getting for each training example the model output with the parameter values. \n",
    "+ Compute the loss.\n",
    "+ Adjuts the model parameters.\n",
    "\n",
    "We will repeat this process a number of times, according to the learning rate.\n",
    "\n",
    "After the training we will apply an evaluation phase. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:34.505199Z",
     "start_time": "2019-03-05T11:47:32.185940Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.13.1\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:36.158559Z",
     "start_time": "2019-03-05T11:47:35.955001Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "(200, 1) (200, 1)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "%matplotlib inline\n",
    "\n",
    "# Load data.\n",
    "\n",
    "import numpy as np\n",
    "data = pd.read_csv('data/Advertising.csv',index_col=0)\n",
    "\n",
    "train_X = data[['TV']].values \n",
    "\n",
    "train_Y = data.Sales.values \n",
    "train_Y = train_Y[:,np.newaxis]\n",
    "n_samples = train_X.shape[0]\n",
    "print(n_samples)\n",
    "print(train_X.shape, train_Y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:37.486730Z",
     "start_time": "2019-03-05T11:47:37.057242Z"
    },
    "code_folding": [
     0
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ0AAAEKCAYAAADJvIhZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8W+WZ6PHfWSRZ3hPHsZ2QxCFkbQopMFBIQwjLsBRKKEvapqXATMpQOkDboQudKcO0Q3sZOr1w78Bt0hta+LCVBtJLC3RCSCANLTQFyprYZAXbsR07tmNb21nuH4oU2ZFkSZZ0JOX5/gMIyXrPOfZ5zvu+z/u8SlfXgI0QQgiRB6rTDRBCCHHskKAjhBAibyToCCGEyBsJOkIIIfJGgo4QQoi8kaAjhBAibyToCCGEyBsJOkIIIfJGgo4QQoi80Z1uQKGZP38mzc3NTjdDCCGKxu7du9m+fU9K75WgM0pzczPbtm1zuhlCCFE0Fi06OeX3yvCaEEKIvJGgI4QQIm8k6AghhMgbCTpCCCHyRoKOEEKIvJHsNSGEKCK+1hYGNm8k1N2Fq34y1Wefi3f2HKeblTLp6QghRJHwtbbQ88QjmIcOoU+swzx0iJ4nHsHX2uJ001ImQUcIIYrEwOaNqOUVaBUVKIqCVlGBWl7BwOaNTjctZRJ0hBCiSIS6u1DLy0e8ppaXE+rucqhF6ZOgI4QQRcJVPxlreHjEa9bwMK76yQ61KH0SdIQQokhUn30u1vAQ5tAQtm1jDg1hDQ9Rffa5TjctZZK9JoQQRcI7ew51K1aOyF6bcOnyhNlrhZjpJkFHCCGKiHf2nJQCRyTTTS2vGJHpVrdipaOBR4bXhBCiBBVqppsEHSGEKEGFmukmQUcIIUpQoWa6SdARQogSVKiZbpJIIIQQJWJ0tlrlmUsI7GxNKdMtXyToCCHEGPKRejze74iXrTb4yhbqVqwEwokFPU8+5njqtAyvCSFEEvkospmN70iUrdb7m3UFVSS0JIJOW9tHXH75p1m8+FSWLDmN1avvB+Duu+/ixBPnsmzZYpYtW8wLL/ze4ZYKIYpNPlKPs/EdibLVAh+0FlTqdEkMr+m6zp13/jsnnriIwcFDnHfeWSxdeg4AN9xwEzfddLPDLRRCFKtQdxf6xLoRr2U79Tgb3+Gqn4x56BBaRUX0NWt4GBSloFKnS6Kn09DQyIknLgKgsrKKOXPm0tHR7nCrhBClIB+px9n4jkTZap7jTyio1OmSCDqx9u3by9tvv8Upp5wKwNq1q1m69AxuueWr9PUddLh1Qohc8bW20LnmAT6660461zyQtTmLfKQeZ+M7InXZtKoqjN4etKoq6lasZOLyKwoqdVrp6hqwHfnmHBgcHGT58ou59dZ/4pJLPkNXVxd1dXUoisKPf/xDOjv3c++99x/1uYceepCHH/4FAAcP9rB37948t1wIMR6xmVtqeTnW8DDW8FDW6owVQ/aaUz8bYNGik9mw4aWU3lsyQScUCrFy5VUsW3YeN974taP+/759e/niF6/m5ZdfTfpzLrpoGdu2bctVM4UQOdC55oGj5jPMoSG0qioaVt3oYMvyw+lq0ukEnZIYXrNtm1tvvYk5c+aOCDidnfuj//7ss88wb958J5onhMixQq0zlg/5SOnOppLIXnv11T/x5JOPM3/+x1i2bDEA3/ve93nqqV/z7rtvAwrTp0/nnnvudbahQoicSJS55XSdsXiy3SuJTbcGov8c2LzR8eoD8ZRE0PnkJ8+gq2vgqNfPO+8CB1ojhMi36rPPpeeJRwBGzOlMuHS5wy0bKRd73OQjpTubSmJ4TQhxbEuUuVVoT/q5WGhaqNWkEymJno4QonSlOhyV6o6aTspFr2S8vbx8JyFIT0cIkVfprKcptknyseSiVzKeXp4T51d6OkKIvEl3TqPYJsnHkqu5p0x7eU6cX+npCCHyJt05jVJLhS60uScnzq/0dIQQeZPunEYxpUKnqpDmnpw4v9LTEULkTbpzGk5uuZyrWm6FxInzK0FHCJE36d7knBqOKrUEhkScOL8yvCaEyJvITS42RXfCpcuT3uScGI4qtQSGZPJ9fiXoCCHyqpDmNBIptlX+xUSCjhCiIKS7SDGXixpLMYGhUMicjhDCcenOoeR6zsXJBIZSJz0dIUROpdIjSXcOJddzLpnMPYnUSNARQuRMqhUI0p1DCXV3gaYR3L4Xy+dD9XrRG5uwsjjnUgxzT8VIgo4QImdS7ZGkO4eiuNz4d7yP6vGgeDzYwSCB1hbK5o69UWMu5oKc3rmzmMicjhAiZ1Its5L2HIoS/oeNAiiH/3nk9URyMRd0rKzpyRYJOkKInEm1AkG6ixTtYBDP7Lmobhd2wI/qduGZPRc7GEzanlzsZ5OLn1nKZHhNCJEz6VRVTmUOJTKMFdy3F3Qdz/QZ6DW1AJhDQ2gTJyb9fCbrb8YaOpM1PemRno4QImeyWWYldhjLPXMWtt+Pv2UHob6DKac0p1v7LZWhs2LbudNp0tMRQuRUtrLAYoexNECZM4/gvj0E9+ym4uRTx0xp9rW2YPT34XvvHdTyCtzTpqO63En3s0klESJXe+SUKgk6QoiiMHoYS6+pQVt4IkZvDw2rbkz62djU7bK5Cwh9uBf/jvfxzl+YtOeVytBZMa7pcTLbToKOECJlTt6sxlOaZnQvyVVbG54DqqpK2v5Uv7OY1vSku3trtsmcjhAiJU6nBo+nNE2mO2SWYjkcp7PtJOgIIVLi9M1qPEkJmU72F9r20tng9BbgMrwmhEhJIaQGZzqMNZ7J/kIYOsvmsKbTFbQl6AghUhJ7szL6+wh1tGMODqJVVuFrbXH8xpxMOpP9hVbSJttzMNVnn0vX2tX4dw1gGyEU3YVWVZ23bDsJOkKIlER6C8ahAULtbYACioJWOyGvE9GZSnXxqZOT7PHkpKK2Eq4YZNuHKweNUT4om2RORwiRkkhvwerrA8tEK/dSdsIc3I2NJVP2xel5q3iyPQczsHkjrkmTKf/4SVSefCrlHz8J16TJeTtG6ekIIVLmnT0Hvb6esrnzUJQjj8elUvalEOatRsv2HIzTxyhBRwiRFqcnonMp3WPLdP4nnc9lu+KB09evJIbX2to+4vLLP83ixaeyZMlprF59PwAHD/Zy5ZWXcfrpi7jyysvo6zvocEuFKH6FunbF19pC55oH+OiuO+lc80BG64fSObZM1y2l+7lsp207ff2Urq4BOy/flEOdnfvp7NzPiScuYnDwEOeddxa//OVjPP74I0yYMIGbb/4G9933n/T19fH97/9b0p910UXL2LZtW55aLkRxKuQMr9jeQCY351SPrXPNA0f1GCJVDpKV5cn0c9mU7eu3aNHJbNjwUkrvLYnhtYaGRhoaGgGorKxizpy5dHS08/zzv2P9+mcBWLHiCyxffvGYQUcIMbZCWLsSK5sZXqke21hzI4lu7E7PqYCz168kgk6sffv28vbbb3HKKafS3d0dDUYNDY0cOHDA4dYJIXIh1Rt5vhZZJku9dnpOxWklFXQGBwe5/vov8YMf/JiqquqUP/fQQw/y8MO/AODgwZ4ctU4I5xXasFi2KG43vnfewjYMVK8XV9NUFF0fcSPPxSLLRBP8yXpex/pWCCWRSAAQCoW4/vovcsUVV3PJJZ8BoL6+ns7O/UB43mfSpElxP3vNNdexYcNLbNjwEvX19XlrsxD55HTBzlzwtbbQ9h8/YuitNwkd7MU2LcxAEH/LdkIHukZMjmd7DU6yCf5ka2tKsZ5bOkqip2PbNrfeehNz5szlxhu/Fn39ggsu5oknHuXmm7/BE088yoUXftrBVgrhrJysbHdQJIgG93egV1VjBUNYviEUqwzV60WfUJfzbaUTzY2MNYRWaHNi+VQSQefVV//Ek08+zvz5H2PZssUAfO973+fmm7/OqlXX8sgjD3HccdP4+c9/6XBLhQhzYpgr3xPYuT7GSBDFMFA8HnSXC8vtRnW7KJs7H6N35FB5PudSjvUhtGRKIuh88pNn0NU1EPf/rVv3TJ5bI0RyTtX3yudNNx/HGAmiqteLHQyCy4Wi61g+X9zjymcgKMbdRPOlJIKOEMXEqWGueDfd0IEubNPko7vuzGpvJN1jzKRXFAmirqYp+Hd+gApYNqi6HjeYpBII0m1Hsvcfy0NoyZRMIoEQxcKpTbRGT2Dblgk2KJqW9cSCdI4x0wSHyMp6RXfhOX4WtqJg+4dxN89M2KPyzp5Dw6obOe72O2hYdeNRASeddpRiYkY+SE9HiDxzcp1G7NN355oHUFQtJz2udI4x055fbM/F6u6i4qRPjKunlm47Si0xI18k6AiRZ+OZW8jm5HwuEwvSOcbxtCObQ1jptqMQKgsUIxleEyLPMl2nke3hHFf9ZKzh4RGvZavHlc4xjtWObBTyTEW65yOX56+USU9HCAdk8oSe7eGcXGdzJTrG0b01z6zZDL6yJW478pnpl+75kLTozEhPR4gike0EBCdWxsfrrQ2+soXKM5fEbUc+d/JM93wc65UFMiU9HSFyJNuLIzNJQBirDflO603UWwvsbI1b1j+VeZNsnud0z4ekRadPejpC5EAu0mnT3XyrEFN60+2tpTLfU2jHKJKToCNEDuRiWCjd4Zx8Dk2lKt3J97ECbSEeo0hOhteEyIFcpdOmM5xTiCm96U6+j1VFoBCPsVS3j8gWCTpC5EAhbNRVCG0YLZOaZMkCbaEdo1N19YqJBB0hcqAQ0mkLoQ3xjHfyPbYnobjcGAd7gMkFcYyZpLUfaz0jCTpC5EC2qwxncmNyqtJxLm+io3sS1vAw2GBbJkZvj+PVnNMd7jsWe0YSdITIkdFP9JGV9enejMdzY8q0V5Fp4Mj1TTTSk7CNEP4d72P5fKDr6BPrOO72O8b988cr3eG+Y7F+m2SvCZEH40ntzXeGViG31b9nF77WHQy9+TqhngPYigK2je/9dwoiTTrdtHanKo47SYKOEHkwnpvxeG5MmdQtc6qtY/G1tmD09GAdOoSiaSg2WIcGsEMhtAJJk043rf1YrN8mw2vimJePidzxpPZmmqGV6VCXE21NxcDmjbibpuLb/h5oGoqmgmljDQ/jmTW7YHoH6QxpFmqyRy5JT0cc0/K1on08T7TpDtlEZNpjyVVbE/W6Uu2Nhbq7cDU0oE+YiKIq2KaJommoZWWoLldR9g6OxfptEnTEMS1f8yWZBg7I/MaU6VBXLtoKxA3ufS++kHLQjwRDT3MzqrccvaYGpaIStaws5fYVomS7mZYiGV4Tx7R8rWgfb/pyJllomQ515aKtnWseiJul1ffMetzTpqeUvRUZilLLK/AcP4vgh/uwfUO4Fyxk4mVXlPzNulRI0BHHtHyuaM93ReLxzBdku62JgrvR30fZ3HlHvR4v6Gd7e2rhDAk64phWyhO5Ti0OjSdRcNdrarGGh1MO+rKVwNGKraKBBB1xTIvcmHt/s47hN18HRcFz/AlONytrCuUmnSi41166POGuoWJsxVjRQIKOEIDt91M2Z170xpfOH26xPWk6IVmvyzNtesLemJPnthiua6KKBr3r16HX1hZk2yXoiGPeeEqRFOOTplMS9boSve7kue178QV6f/UotmmiVlRghUIFeV3jzZVZoSD+He9TvvCkgvydlJRpccwbzyp62UQss6oHqXDq3PpaW+j51aPYdvj3gFCIUHsblmkW3HWNt6Yq+OE+tAL+nZSejjjmjSeDLVsp18UwlBMr0l7/nl0YPT24m6biamjI6lN1ttPZUz3HA5s3gmmglleiKIDLhQqYB3sJuVwZfXeuJJorK5u7YMT7Cqmem/R0RNEb75P2eBZDZqN2Vr6qImRLbHutoSGwIdT+EebAQFafqrNZlyydcxzq7kKtqMQ2jCMv6jrW0FDBVT2ItxjXu2Ah6qjgWEj13Mbs6TQ21jJ//scwTYPp02fwX/+1mpqa2qw14PHHH+HNN1/nxz/+Cc8++1tmzTqBuaPy9oVIJBvj/pmmFvtaWzD6+/C99w5qeQXuadNRXe60s6+Krbx9bHttvx+1zINtmIQ62tBraqJP1ePtvWUznT2dc+yqn4wdMgi1f4QFKLqO5Q+gaFpBVj2It4VGIS8DGLOnU1bmZdOmrbz88qvU1k5g7do1OWvMc8/9lpaW7Wl/7pZbvsqCBcdz1lmnR1+7++67OPHEuSxbtphlyxbzwgu/z2ZTRYHI1rh/uqVIIn/YiqpRNncBCuDf8T62aaY9tFRs5e1j26t6vWAY4RuzzweEn6oVt3vcvbds1iVL5xxXn30uiqbimnIcikvHGh4EBSZe/YWCfAgYrdDruaU1p3Pqqafx3nvvRP/7f//ve/l//+8pAoEgF198Cd/+9vcYGhpi1aov097ejmWZfOMb32L58is45ZSF/Pd/v0RdXR1vvvk6//qv/8z69c9Gf9Zrr73K73//LH/841b+8z//g7VrH+aFF37PL3+5Fk3TmTt3LqtX/yJuuz73uZX83d99ha997YYRr99ww03cdNPN6RyiKDL5KmMzWmyw0wBXbS3m0FB4eCPDcjW2YRDqaMPy+VB0HXfzzNw0PolUeiexc2Cupin4d36A5fOjWAaD215D0TS0yY24amvH3XvL1jqjdObtYnu+ikvHVf/xgp9ji1Xo84MpBx3TNNmy5SVWrvwSAJs2bWT37p38/vebsW2bL31pBX/841YOHDhAY2MTjz76awAGBvpT+vmnnXY6F1xwMX/7txdy6eFu4H33/ZRt297G4/HQ39+X8LNnnLGYffv2pnooooTks4xNrGwGu+qzz6XrwdUYXV3g9qBoGpbfj9HTg6+1Ja/rUlIZqowd9tKqa1BrazE/3IfiKUOtqECfUEdg7y60inI0jlwXJ3tvnlmzR6RAaxMmompawiGnQllUO5bRAcYzazaDr2wp6BT+MYfX/H4fy5YtZu7cZvr6DrJ06TkAbN78Ips3v8g553yKc89dQmtrC7t27WTBggW8/PJm/u3fvs+f/vQK1dU1GTduwYKPceONf8+TTz6OpqWfaLd27WqWLj2DW275Kn19BzNuh8iubKbYjicJYDyyOcntnT0HfUIdSlkZimWiut2UzZmLq35yXtNcUx2qHD18Yw8OolVVoXo8KIqC6vWilVcQ/HDfiM85NZnta21h8JUtuJqmhtfcDA0S6min8swlBXMjzkS85IhwYLUKNl0a0pjTef31dwgGg6xduxoA27a5+eZvsGnTVjZt2sprr/2VlSuvYdas2WzY8BLz5y/ghz/8V+6558cAaJqGZVkA+P2BlBr36KO/5vrrV/HWW29y/vlnYcRmk4zh2mv/ntde+yubNm2loaGRO+74XsL3PvTQg5x//lLOP38p3d3dKX+HSF+8P5Sutatpu+dHGQUhp8avsx3s7FCQ8oUnUvGJU/DOW4BeU4taXk5gz+6crIGJJ515j8gcWN1Vn8fyDYPmQvGUYQVDBHa2ok6sc+RhIJ5IMHU3NlI+/2NUnno63jnzCOxszXtbsineQ4JtmhgHe0a8r9DmB1NOma6uruGuu+7m/vv/F6FQiGXLzuWxxx5mcHAQgI6Odrq7u9m/vwOvt5yrrvocX/3qzbz11l8BmDZtBm+99QYAv/vdb+J+R2VlZfTnWZZFW9tHfOpTZ/H97/+A/v5+hoYGUz6wyZMno2kaqqryxS9+mTfe+EvC915zzXVs2PASGza8RH19fcrfIdI3+g/FNkIY3V0E9+we14RzvvcjyXawi9dzCnV2YvQcGHcqdSo9S19rC0Z3N0N/+TO+7e9hHB7OHqt3ErmeqgKKAqpLB92F1duDd8HCgpjMLrZEjVTFPa7DPblYhZQuDWkmEnz84yfxsY8t5Omnf83VV3+e1tYdfPrT5wFQXl7B/fevYffuXdx557+gqioul87dd/8UgNtu+w633noT//N//oSTTz417s9fvvxKvvnNf2TNmv/D6tUPcuutN3Ho0AC2bfMP/3BTWqnanZ37aWhoBODZZ59h3rz56RyqyJHRcyGhjnYUjwfbMKJPa1C46cKxsjnuHy89ONjRhqtpyrgm41OZp4m8R6udgDk0iDnsw9z5Aa4pU5POe0D4erqnTSewa2f4CVbXARtzeLhg9rhxat4v1+IdlzZhIpbfjzk0VJDp0gBKV9eA7XQjxuuGG65j69Y/0NvbQ339ZL71rdvZunUL7777NqAwffp07rnn3mgQSuaii5axbdu23Df6GNW55oERfyhDb/wFW9XQPG6888KrqG3bxujt4bjb73CyqXHlMjNo9M/279mFZ9oMFEWJvifdczP6fAPRLLuGVTce9R6jv59QRxvm4CG0ymoavnpz0uOLfNY2QoQ62sNp07qOZ8ZMpt723QzPRHbFBt7YG3EhTa5nItFxVZ65hMDO1rxmry1adDIbNryU0ntLogzOz3724FGvrVx5jQMtEWMZ/USProPfjysmPTgXT6HZCBa5LkA5uucUL2Cke25SybKLfY9eE078sds/wujvi05AJzq+2N08y+bOj974Ji6/IuU25loh7SuUTUmP65zznG5eQiURdETxGP2H4pkxE+NgD4quY9t2ToYDshUs8l05IBsr8lMZWop9j9HfT2BnKzYKWnX1mOeqWG7oxZICna5iPC4JOiLv4pXtyOVNK1vBIt8LUbNxQ08lcMW+J9j+ETYKYOOeMjWlc1WMNz7hHAk6wnG5vmllK1g4MSE93nOTSuCKfY85MIBWXY17ylT0w4k7pZDpJQqHBB2RE4VUiiNbwSKbBSjzKZXAFfueUsz0EoVDgo7IukLbTTNbwSLb8xeFFJgjijWwJpLsHBfi+T8WlETKdDZJyvT4pZKmm6ps3RgK7QZTyGm8hXauMpXsHAMFe/6L0TGXMi0KSzZ308xWjylb80bjvSFHPj/0+jYUTcM9vbngFsWWSmJAsgQSYMzkklIJvoVGdg4VWZetYpjZ2isnW8a7w2fs523bwrJsAjtbMfrDldhlwj67kpW/Gas0TrHt5lpMpKcjsi5b8wKje0xGfz/Bw9siR74nn0+emaRexz4tG93dqLW1uCoq0Lzl2MEgtu6K7rhZyBP2+Xzqz9Z3jZVAkuz/FdtursVEejoi67JVDDO2xxRZtGj5/CMWLebzyTPdwpFHPS0PDhBqb8Po78PVNAXLMIjUKXOyCnM8sUVC2+75EV1rV+flqT+bPYxklcDHqhJeikVCs7mlyHhIT0fkRLJ5gVSfZMe7aDHb0k29PuppubIKc9hHqKMd77wFlM06gcC+veGhw6qqnK/kT/W8j55L873zFpbfjz5hQs7nn7LZwxgr2zDZ/8vnmqx89CILKaNUgo7IiUR/SOn88hfaosV0hw1HDw+6mqZiftCCOTiIbdsougt3Y1Ne/vDTOe+jb/y2YaB4PIQ62nN+7rNd9SHZw0+8/xf5vfXv2YXR04O7aSquhoacpY7nKxgU0nChBB2Rdcn+kNL95c/HosVUnzTTXacz+mlZr6nBPXUaZt9BjN6evNYpS+e8j77xq14vZiAYriB9WK6e+p3chiD299YzbQaK20Ooow07GMDTPDMn1ypfwSDfJZySkaAjsi7ZH1Kmv/y5WrSY7pNmOunE8dqsaGp0u4BIsOt58rGcT86nc95H3/hdTVMxW7ZDWVnOirJGVJ99Ll1rV+PfNYBthFB0F1pVddLvytbw1FG/t14vhq5j9BzAE1MFPZvyFQwKaU8hSSQQWZdsEjbTdOpcbUudy7TsZG3Od0puOud99CS7ouvokyeHK4LnYxdQBRQA+/A/lcRvzeZ5jP29Nfr78O/8AGw7nGiQo+uTreUFY8n29urjIT0dkXXJnqpin/6tUIjQh3sxh4fwzl+Ir7Ul6Y0sF4sWc/2kmajNhbxNgpPbFQxs3ohr0mS0GUd6FubQUMLzks3zGPt7G+poR9X18BYP5eU5uT6+1haMvj5877+DVl6Ba9oMVJcrJ73IQtqCQoKOyLpkN7jIL3/vb9bh3/FedPMvRdMcyaYZHSCN/n6C+/Zgmyadax6IPglmO7uo0LdJcKoqQbrnJRvnMTI8F9izG6PnAHrTFEzfMKg6ihnCNaM5o5871nfGbn4X/HAf/h3v4V2wMGd/A4VSaUKCjkhZtibcvbPnoNfU4plxPGZfL8Hdu1C9XrTaiXnPphnd8wq07gDAM3sO5qFDdK1dDQq4Jk3OanaRk9skRK5j1y/WYA/7UMvL8TTPzPli21R+f9I9L/HeH+rsxOw7yEd33TnmQ0Lszd89bTqK20Owow3bMNDKXLibZ0d3U83m9YntoWlU4KqdEK1PONY1KPbyPBJ0BDD2L3K2J9wDe3YT6u5CcblRPGVYwRBm24fYwUBOji+R2ADp/8ufUcrK8EyfEU0N9u8aQIHocI9WUYF56BCd99+HXl+f8R+9U9WcI9fRMk2MAwcABXNoEMXtyWlPM9Xfn3TPy+j3hzo7CezZhbt5JmgaQ399g0N//APeBQuZeNkVY6aHuxsb0aqqsC0T2+/P2Y62mfbQCmm9TaYkkUCkNBmb7Ql3a3gYFBXVpaMooLp0UNSjJlVzKbJCu+fJxwDQ6uooX3hiNOAA2EYIKxSK/rfR30+w7UPMwfFNXOcqMWIsketo9R1E1XU0bxmKy43Z15vTunap/v6ke15Gv9/sO4g2eTJG536G33wDa6AfW9UJ7tkd9zolSnqxg8GcXp9MEwgKrR5hJqSnI1KajM3kySxZ78lSwBw6hHloAMWlo3rKQFFQyr0ZH0c6ww7xnhiNnh4UtwdPY1P0fYruGpE8FepoA0VFq/COe3W+E2Psketo+XwoHg8Aiq5j+Xw5nVNKp45euucl9v17v3cb5oEDWL5hFE3DtsH2D2PaFp7DN+fYn51sOC+X1yfTnm4hrbfJlPR0REp1ptJ9MkvWe/K1tmAPD6O4y1BdLjAMrOEh1NpaypqPz+gY0k2djffE6G6aitHRPiKtVKuqRq2uPvLa4CHAxtU0JeG5StS+Qqh7FbmOqtcLhgGEKw6oXm9O55TyVUfPHvYBClg2iqaiqCrYgGnEvU5OpRJn2tPNV4p1LknQESn9Iqf7x5lsGGBg80bcTVPRXDpqZRX6pMmoFVWJNsoQAAAbQ0lEQVTY/f0Z/7GnO+wQL9C6GhrQ6yaNuBFMvv4rTL7uK0deq6zGFVOKJ965Gq2QyuRHrqNaOwHLMDB9fuxQEK12Yk5vtrG/P/Hq6GVriEgtLwfbAlXBtmxsywJs0PS418mpYc7IdzesupHjbr+DhlU3pvSdhbTeJlMyvCZS6uqnm3I71jCAq6EB1esl1NEWHtrxlqFVVGb8x57usEOiYRVP88zo7qajh+vqrvo8EN5x0hwaSniuRn/O6O8rmLpXsdfRDgai2WvuKVNymgWVrzp6nuaZ4fI1nR0Yh+et1PJKFI876bqkYpmEL6T1NpmSoFOifK0t9K5fR2DXB1jBIGpZGXrthLipsan+Iqfzx5nopq643ITa2wjs3oVWWYmraQp6TW00XTRT6abajhVok2UJJTtX8T7ne+8dyuYuQIv5fifH4Z26yeajjl7kupadMGfE4uOyGXOZuPzo7LViVExBMh4JOiXI19pC19rVGN1d2IqC5fNh+YYxfb6EqbHZ/kWOd1MPdXeBAlrtBMyhQcxhH+bOD3BNmYqqaeNKR437fQe6sE0z7nqNsQJtsuSKZEMh8T6nllcQ+nAvrtrUh+RKWS7TxWOvq9XdRflJnyi6dSylToJOCRrYvBHz0ACqxxNeg6GFn7EVI4TZ14t72oycD+3Eu6nblomiauEb8eGhNXPwEFZfH/WHi2AmErtq3BoeRin3UtZ8fPSGMvr7FLc7XLtL09Cq4q9nSBZoM80Sivc597Tp+He8n3RI7ljinT2HyjOX0PfMeoz+PvSaWmqzOERU7D2BUidBpwSFurvCFXo9ZdiGAaqGoijYphFNjQ3s2U3nmgdyuqp59Ap433vvHh7LPw69pga9pgbbtjF6e8YMOD1PPIJtWod7SyoMDxEY1WuLvdl0rnkgGuDgSE+ld/069NraMY8704oB8T6nutx45y9Eq6rKyTh8sa1Q97W2MPjKFtzTplM2dx7W8DCDr2zBM216QbdbZIdkr5UgV/1kFD2ciqzoerhSrmWhqGq4h9HZidFzIO/bD2vV1Vg+P4GdrRj9/UB6C+LMvl4UlxvNW4aq6+EFjgmynuJlp1mhIL7330npuDPNEkr0uYnLr4ibqdT34gvs+frX+OD6L7Ln61+j78UXkv780QopMy5VpbDAUWROgk4Jqj77XLSqaqxAAFxubNPENg1s3YVWO5FgRxt605S8/NHH3mDcU6YCNjZKeGFgijfySACxfL5wEAUYY0FjvDTw4If70FK82WWaSpvO5/pefIEDDz+IOTyMWlmFOTzMgYcfTCvwFOMNPJV1YaJ0lcTw2i23fJUNG55n0qR6Xn75VQAOHuxl1arr+PDDvUybNoOf//wX1NZOcLil+eGdPYfJ138lmr2G1xvNXlMryrGNEIF9ewju3gmajl5djd7YhJXj7Yf1mlrKZp1AsL0tnDZbVZXSMFNkyEr1erGCIRSXDmMsaEw0WV02d8GI9yW72WU6N5Dq5/qeWY/i9qB5yw5/sAzz8Ou155w34r2JhtCKcYV6IW0oJvKvJHo6n/vcSh5//KkRr913308566ylvPrqm5x11lLuu++nDrXOGd7Zc5h623c5/oH/ywn/92GO/6811F/79+EihqqG7fNjhwwsvw9zaIhAawuKy531dozuceg1tXimN+Od/zEAep58bMwV+pEhK612InYoiOnzYxlGeIFjgp5SvB6Hd8HCcAWEGE7e7Iz+vmgpmgjF48Ho7xvxWrIhtGJcoV4KCxxF5koi6JxxxuKjejHPP/87Vqz4AgArVnyB5577rRNNKyiRoRhFdwE2qCqKqmH5feE3JNmhMVPxbjCh7i6Mgz0pz0NEAoh7yhRc9ZPRysvRJ03CM2XqmMUgY+dRJl52RUHd7PSaWuzAyKradiAwotoBJB9CK8YbuJNVAITzSmJ4LZ7u7m4aGhoBaGho5MCBAw63yHnRoRjLRKupxfYNh7PbbAXP7LnYwWBKPyd2qCeSmmyHgtFhHxi56VnlmUsI7GyNmzoNqa3QTzZkNVZ7Ul2bk2+1ly4Pz+kQ7uHYgQB2MEDt4YeliGRDaIV2TKmStOZjV8kGnXQ89NCDPPzwLwA4eLDH2cYkkGhMP510WVf9ZALtbVh+H1YwhOp2RZ+gVZcLbeLElNoRWXGPpuPf/j6QfNOzwVe2jHiS/eiuO9GyNA8R2x7T7yfw9ltgGqBpqFXV+HftZPL1X8m4gnEuReZtRqxXWfGFo+ZzxpoDKaRjEmIsJRt06uvr6ezcT0NDI52d+5k0aVLC915zzXVcc811AFx00bJ8NTFliUqyVJ65hMFXtqS8oZNn1mwOvfIHbFUHxcAKhsAfQJtYl/JixdihnuD291A8ZSjYGPs78M5bEHfTs8jnIm2K3ERtw4jWXlN0PbzxVpoi7bENg8AHrWCZgAK2jTU4SDAYoHf9Oqbe9t20f3Y+1J5z3lFBZjSnNnwTIhdKYk4nngsuuJgnnngUgCeeeJQLL/y0wy1KLlnp+0Rj+n3PrE8rXTawsxVP8/G4aqpRy8rCPR2vFwKBlMfUY9NdoynMh9OXYeSmZ0Z/H77t7+FreZ+h17dFj6n67HMJHejC37IdMxAETcPy+zF6etJeXxJpT6ijLdzDURRQw0FH0TSwrHAGXxGTORBRSkqip3PDDdexdesf6O3t4aST5vGtb93OzTd/nVWrruWRRx7iuOOm8fOf/9LpZiY01ha0gT27MYcGsf1+VK8XV9NUtOpqjP4+yubOG/Gzkg1Thbq7cDU04G5sjL6WSkWAWLFDPdEUZuxw8OLIpmdGfx/+nR+g6jqoOoqqjDgmfUJdeAOvw6nPrhnNKLor7fI8kfZEgt6RE6GCohyes7JT/nmFSobQRKkoiaDzs589GPf1deueyXNLMpOsuCSA0XMA2wa1rAwrGCKwsxXXlOPQa2qxhodTXu+QjfURsUM9euMUAq07sAHP9BnRTc9QILBvb3h/eRQUM4S7eTaKrkeDih0KUr7wRBTlSMqcbdtpz+tE2hPpcREKAQq4dWzTRFHAc8LstH6mECJ3SnZ4rZgkW6E9sHkjetMUwAYjhKJr4RX9HW3UXro8rXTZbKTXxg71YBqUzZtP2dz5YJojNj3DMMI7UrpdeGbNRq+pGbmfTpbWl0TTqZtnhgOPpoEeHlYDG21yAxMvuyKtnymEyJ2S6OkUu2Q9kFB3F+6GRjSvl1BHO8bAAJgGlmUT2Nl6VDpysnTZbKXXpjLUU3HK3yTtVSXa+sC24m9FMFZ7pv7Td8N7CP1mXTihQFHwHH9CyeyhIkSpkKBTADyzZtP7q0exTRO1ogJtwsTo/jLhbQoORRcMmsPDoLtwecvipiOPJdO5gVRTsyPv8+/ZhdHTg7tpKq6GhqMyro7aisDlBoXw+p0xMvEStSUSfIQQhUuG1xwWKfPuapqKWlGBNTRIqKOdyjOX4J09JzokFtjfge+9dzAPDWANDaLV1OatuGOqlYxj3+eZNgNX0xRCHW3hQptxMq5iKwbotbW4Jk0eMxOvGKsqCyGOkJ6Ow0ZUYT6cVWYODRHY2QrnnBfd8KrnV49iBQIoLjdqWRnG/g60ikq06uqsFneM14tIlugQG0RGv8/T2IReVY1WVUXDqhuTfm+qhStTbYsQojBJT8dhqZR5D+xsxTtnHu7GJvSqKrTyctBd4YWVWSzumKgXEdizO6VS9OMpWZ9qYoGUxReiuEnQcVgqN9vIjdbVNAXLMLBDIdC08FbPWSzuOHoRqm0YhPZ3ENi3B987b0U3XovXxlSPJZFUM+uKsaqyEOIICToOi52zGX7/XQa3vYa/ZTueWUfWlkRutJH9aBS3G9s3jFZZndWV6bG9CKO/n8DOVizLBrcby+/H37KdUF9fwoAwnpTsVFfdF2NVZSHEETKn47DYORtMA7WiEn1C3Yg942PTi7XqGhTdhTU8lPVSKLGp26GONtBdqNho1TW4mqYQ2LeX4O6dVJzyN3FTrcebkp1KZl2xVlUWQoRJ0CkAkTmb2DUt5tBQdHI8Xzfa2OBmDg+j6DqWYVA2oxm9phZtYQ1Gb0/SpIB8lGuRkjBCFC8JOgUgNnPL6O8j1NGO6RtGUVR8rS3Rm2y6N9p0tj2Akb0IRVFAUSibdUJ0jZDMnQghxkvmdApAZM4mUiTTDgbDRTI1LeM1KJmuZ4msnWn65ndwNzah6C6ZOxFCZI0EnQIQTSYYUSTTwD29OePFn8m2OE6FlNMXQuSCDK/lwYjtlA+Xe7GDI7dTrluxko6f/BjbttHKy3HNaEavqYlWXh49VOaZNXtEzbXRQ2epLrZMRuZOhBDZJj2dHIsd5kLT8O94P7zFs6aPGPLyzp5DxSl/g3fufLzzFqDX1ADheRTF7R4xVBZsb+fAww8SaG9LOHQm61mEEIVIgk6OxQ5zGfs7UD0eFE8Zxv72o4a8PLNm42vZzuC2Vxl+/12C+/djDQ+BzYihMrOvF8Xtweo7mHDoTNazCCEKkQSdHBu9vTO6Hk5FPrzTZWTI60jhzymoFZVYQ0OEOtqoPHMJdig4YtFmqLsbc3CAUHd3tErA6KEzmZMRQhQimdPJsdHbO9vBIDZKdHvnyJDXwOaNWKaJ1XcQ2+9Hq6xErZ0Q3iX08M+wDSNcCFRVwFRAVcL/PSu8K+fooTOZkxFCFBrp6eSYZ9Zs/C3bGdz2GqbfjzE0hB3wozdOGTHk5d+zi1B7G+bQEKZvmGBnJ/7WHQy//250qCy4bw+2pqOWecGyUD1l2JpOcN8eGToTQhQFCTo5NHqvHIxwoU73cdPANEYMednDPmzDCM/hWDaqywWWjdEVHjKrW7ES2zTBMtAqKvDMno1WWQmWgW2aMnQmhCgKMryWQ4n2yom3v4xaXo7V1YmiKCiqApYV3knTpTOweSMNq26k4uRTj9oCOvLzJOAIIYqB9HRyKJ29XzzNM1HcLhRNwzYt0NTwJm01tdH3S0aaEKLYSdDJoXTWylSffS6qpwylzIteV4dWUQmKgjZhYvT9kpEmhCh2MryWQ7FVm9XycqzhYazhISZcujzu+92NTfhatmMNqqjV1bimTEXVtBE9GclIE0IUM+np5FCqPZNI1QKtuobyk05GnzABO+BHK6+QnowQoqRITyfHIgEjUjctUjUgNpDEJhxogKu2VhIEhBAlSYJOjkV6MWp5xYg6aZVnLokW7Azu24t75iy0mM+lW5xTCCGKgQyv5Vjvb9YR2t+Bv2U7/h3vYxsGtmnR+6tHowU80XUCrTuiJW1AinMKIUqT9HRyxNfaQu9v1jG07TVwudArq7CCIQI7W7FVFTvgJ/jhXiyfD1tVsQyD4L49aAtPHDPhQAghipX0dHIgMqQW3LMbxeVGsWzMQwNgW6C7MPv7sEMh7GAQxeNBVRRQ1XCZHEmFFkKUsJLv6ZxyykIqKytRVQ1d19mw4aWcf2ckMcA2jHDF6MEBsMEcGkSrrALDgMpqFJcr/AGXC9Uw0crLOe72O3LePiGEcErJBx2Ap576HXV1dWO/MUsiu3aqXi9WMIRaVY09PIwVCoKiQHk5qq5hhYzw9tSGAbZ1VPUCIYQoNTK8lgORSgSupqnhIp+miW1ZKIqCbRh4m4/HPXUaqtuFHfCjul24p07D0zzT6aYLIUROlXzQURSFq69eznnnncVDDz0Y9z0PPfQg55+/lPPPX0p3d/e4vzNSI03RdbSGRszBQWwjhFpdg6tpClYggBXw4Z42g/JFJ+OeNgNFU6WGmhCi5CldXQO2043Ipf37O2hsbKK7u5urrrqMH/3oPzjjjMUJ33/RRcvYtm3buL/X19rCwOaNDP3lz6DreKbPQK+pBcKVoW3TRK8NF/N01U+m+uxzJXFACFGUFi06OeX58pKf02lsbAKgvr6eiy++hNdf/0vSoJMtkRppHx2e31EUJfr/1PJyjN6eo7Y3EEKIUlfSw2tDQ0MMDh6K/vvmzS8yf/78vLYhnUrTQghR6kq6p9Pd3cW1164EwDQNPvvZqzjnnPPz2oZ0K00LIUQpK+mg09w8k82bX3G0DZFK05GCn676yUy4dLnM3wghjkklHXQKheyBI4QQYSU9pyOEEKKwSNARQgiRNxJ0hBBC5I0EHSGEEHkjiQRZFqlEIJUGhBDiaNLTyaLIPjqRHUEjW1P7WlucbpoQQhQECTpZFNlHR6uoQFEUtIoK1PIKBjZvdLppQghRECToZFGou+uoPXHU8nJC3V0OtUgIIQqLBJ0skjprQgiRnASdLIrso2MODWHbNubQENbwkOyTI4QQh0nQyaJInTWtqgqjtwetqoq6FSsle00IIQ6TlOkskzprQgiRmPR0hBBC5I0EHSGEEHkjQUcIIUTeSNARQgiRNxJ0hBBC5I1kr2WBFPkUQojUSE9nnKTIpxBCpE6CzjhJkU8hhEidBJ1xkiKfQgiROgk64yRFPoUQInUSdMZJinwKIUTqJOiMkxT5FEKI1EnKdBZIkU8hhEiN9HSEEELkjQQdIYQQeSNBRwghRN5I0BFCCJE3EnSEEELkjWSvjbJ7924WLTo57c/19Bygrm5SDlrkjFI6HjmWwlVKx1NKxwLpHc+HH+5N+ecqXV0DdqaNEkecf/5SNmx4yelmZE0pHY8cS+EqpeMppWOB3B2PDK8JIYTIGwk6Qggh8ka77bbv/qvTjSgVJ530CaebkFWldDxyLIWrlI6nlI4FcnM8MqcjhBAib2R4TQghRN5I0MmCF1/cwBlnnMxpp53Efff9p9PNSdsppyxk6dJPsmzZYs4/fykABw/2cuWVl3H66Yu48srL6Os76HArE7vllq+yYMHxnHXW6dHXErXftm1uv/02TjvtJJYuPYO33nrTqWbHFe9Y7r77Lk48cS7Lli1m2bLFvPDC76P/7957f8Jpp53EGWeczIsvvuBEkxNqa/uIyy//NIsXn8qSJaexevX9QPFem0THU4zXx+/3c8EFZ3P22WeyZMlp/I//8e8A7N27hwsvXMbppy9i1aprCQaDAAQCAVatupbTTjuJCy9cxr59qadIjyZBZ5xM0+Tb3/4mjz22jj/84c889dSv2bFju9PNSttTT/2OTZu2RlMk77vvp5x11lJeffVNzjprKffd91OHW5jY5z63kscff2rEa4nav3Hjf7Nr105effVNfvKTe/nWt77uRJMTincsADfccBObNm1l06atnHfeBQDs2LGdp59ex5Ytr/H440/x7W9/A9M0893khHRd5847/52tW7fx3HMbWbt2DTt2bC/aa5PoeKD4ro/H42Hdut+yefMrvPjiVjZteoFt217jBz+4gxtuuIlXX32TmppaHnnkIQAeeeQhampqee21v3LDDTfxgx/ckfF3S9AZp9df38bMmcfT3DwTt9vN5ZdfwfPP/87pZo3b88//jhUrvgDAihVf4LnnfutwixI744zF1NZOGPFaovY/99yzXH3151EUhVNPPY3+/n46O/fnvc2JxDuWRJ5//ndcfvkVeDweZsxoZubM43n99W05bmHqGhoaOfHERQBUVlYxZ85cOjrai/baJDqeRAr5+iiKQmVlJQChUIhQyEBRFP7wh5e49NLlAKxY8fnotQlfs88DcOmly9myZTO2nVk6gASdcdq/v4OpU4+L/ndT05Skv4iFSFEUrr56OeeddxYPPfQgAN3d3TQ0NALhP7YDBw442cS0JWr//v3tTJly5HpNmTK1KK7X2rWrWbr0DG655avR4aiOjnamTJkafc+UKVPZv7/DqSYmtW/fXt5++y1OOeXUkrg2sccDxXl9TNNk2bLFLFgwi6VLl9HcfDzV1TXoerhQTWx7Y+9zuq5TVVVNb29vRt8rQWec4kV7RVEcaEnmfvvb/2bjxi089tg61q5dwx//uNXpJuVMMV6va6/9e1577a9s2rSVhoZG7rjje0D8Y4HCO5bBwUGuv/5L/OAHP6aqqjrh+4rl2ow+nmK9PpqmsWnTVv761/d5442/0Nq646j3RM5//GuT2fdK0BmnpqYptLV9FP3vjo52GhubHGxR+iLtra+v5+KLL+H11/9CfX19dGijs3M/kyYVV02pRO1vappKe/uR69Xe3lbw12vy5Mlomoaqqnzxi1/mjTf+AoSfRNvb26LvCx9Lo1PNjCsUCnH99V/kiiuu5pJLPgMU97WJdzzFfH0AampqOfPMT7Ft258ZGOjHMAwg3N5IjzT2PmcYBocODTBhwsSMvk+Czjh94hOnsGvXLvbu3UMwGOTpp9dxwQUXO92slA0NDTE4eCj675s3v8j8+fO54IKLeeKJRwF44olHufDCTzvZzLQlav+FF17Er371GLZts23ba1RXV0f/sApV7LzGs88+w7x584HwMT799DoCgQB79+5h165dnHzyqU418yi2bXPrrTcxZ85cbrzxa9HXi/XaJDqeYrw+Bw4coL+/DwCfz8fLL29mzpw5LF58Fs88sx6AJ554LHptwtfsMQCeeWY9n/rU0ox7obI4NAteeOH3/PM/fwfTNPnCF77E179+m9NNStmePbu59tqVAJimwWc/exVf//pt9Pb2sGrVtXz00Yccd9w0fv7zX2b8ZJNrN9xwHVu3/oHe3h7q6yfzrW/dzkUXfTpu+23b5jvf+SYvvvgC5eXl3Hvv/RlVFc+VeMeydesW3n33bUBh+vTp3HPPvdGb8U9/+h88+ujD6LrOD3/4Y84992+dPYAYf/rTH/nMZy5g/vyPoarh59vvfe/7nHzyqUV5bRIdz1NP/brors+7777DP/7jP2CaJrZt8ZnPXM4//dN32LNnNzfccB0HDx7k4x8/ifvvX4PH48Hv93PTTV/h7bf/yoQJE/jZzx6kuXlmRt8tQUcIIUTeyPCaEEKIvJGgI4QQIm8k6AghhMgbCTpCCCHyRoKOEEKIvNGdboAQ4oje3h6uuCK86LCrqxNN06irCy+e/Jd/uZNzzjkv+t6f/ey/2LnzA+6+u3CLsQoxmvR0hCggEyfWRasVf/nL10erF1977d+xfv26Ee9dv34dn/3sVQ61VIjMSNARoghceullbNjwPIFAAAgXnNy/fz+nn36Gwy0TIj0SdIQoAhMn1vGJT5wS3Qhs/fp1XHbZZwuyIKYQyUjQEaJIXH75laxf/2sAnn56HZ/97JUOt0iI9EnQEaJIXHTRJWzZ8hJvvfUmfr8vuqGYEMVEgo4QRaKyspIzz1zCLbfcxOWXSy9HFCcJOkIUkcsvv5J3331bgo4oWlJlWgghRN5IT0cIIUTeSNARQgiRNxJ0hBBC5I0EHSGEEHkjQUcIIUTeSNARQgiRNxJ0hBBC5I0EHSGEEHnz/wGKMtB4SylsOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# data visualization \n",
    "import seaborn as sns\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "ax.set_ylabel('Results',\n",
    "              rotation=0,\n",
    "              ha='right', # horizontalalignment\n",
    "              ma='left', # multiline alignment\n",
    "             )\n",
    "ax.set_xlabel('TV')\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "ax.plot(train_X, train_Y, 'o', color=sns.xkcd_rgb['pale red'], alpha=0.6,label='Original data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:40.192440Z",
     "start_time": "2019-03-05T11:47:39.689745Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/jordivitria/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n",
      "\n",
      "Optimization Finished!\n",
      "cost= 8.836875 W= 0.07337819 b= 1.7282972\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ0AAAEKCAYAAADJvIhZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XlgVNXZ+PHvnTtLJjuEkIQ1gKwiIlAQkSWgdcEFRaEVa9WWWmrrrnVpa321ltpqq30rr2DR4s8FLYp1L6siIoorLkBkh4QkZM9k9rm/P4aZLMwkM8nsPJ9/dC4zmXNnkvvcc85znqNUVjZoCCGEEDGgi3cDhBBCnDgk6AghhIgZCTpCCCFiRoKOEEKImJGgI4QQImYk6AghhIgZCTpCCCFiRoKOEEKImJGgI4QQImb08W5Aohk5chDFxcXxboYQQiSNvXv3smPHvpCeK0GnneLiYrZt2xbvZgghRNIYO3ZcyM+V4TUhhBAxI0FHCCFEzEjQEUIIETMSdIQQQsSMBB0hhBAxI9lrQgiRRKylu2jYuA5nVSWG/N5kz5iFeeiweDcrZNLTEUKIJGEt3UX1ymdxNzai75mHu7GR6pXPYi3dFe+mhUyCjhBCJImGjevQpWegZmSgKApqRga69AwaNq6Ld9NCJkFHCCGShLOqEl16eptjuvR0nFWVcWpR+CToCCFEkjDk98bT3NzmmKe5GUN+7zi1KHwSdIQQIklkz5iFp9mC22JB0zTcFgueZgvZM2bFu2khk+w1IYRIEuahw8ibv6BN9lqPC+cEzV5LxEw3CTpCCJFEzEOHhRQ4fJluuvSMNpluefMXxDXwyPCaEEKkoETNdJOgI4QQKShRM90k6AghRApK1Ew3CTpCCJGCEjXTTRIJhBAiRbTPVss8Yyr23aUhZbrFigQdIYToRCxSj7v7HoGy1Zo+2ETe/AWAN7Gg+qXn4546LcNrQgjRgVgU2YzEewTLVqt5dVVCFQlNiaBz+PAhLrlkNlOmTGDq1IksXfo4AA899CBjxgynpGQKJSVTWLv2nTi3VAiRbGKRehyJ9wiWrWb/rjShUqdTYnhNr9dz331/YMyYsTQ1NXLWWdOYPn0mANdddz3XX39DnFsohEhWzqpK9D3z2hyLdOpxJN7DkN8bd2MjakaG/5inuRkUJaFSp1Oip1NQUMiYMWMByMzMYtiw4ZSXl8W5VUKIVBCL1ONIvEewbDXT4JMSKnU6JYJOawcO7Gf79i8ZP34CAMuXL2X69MnceOMvqKurjXPrhBDRYi3dRcWyJRx68D4qli2J2JxFLFKPI/EevrpsalYWrppq1Kws8uYvoOecuQmVOq1UVjZocXnnKGhqamLOnPO56abbuOCCi6isrCQvLw9FUVi8+AEqKo7w6KOPH/e6FSue4plnngagtraa/fv3x7jlQojuaJ25pUtPx9PcjKfZErE6Y8mQvRavnw0wduw41qx5N6TnpkzQcTqdLFhwOSUlZ7Fo0S+P+/cDB/Zz5ZXzeO+9rR3+nPPOK2Hbtm3RaqYQIgoqli05bj7DbbGgZmVRsHBRHFsWG/GuJh1O0EmJ4TVN07jppusZNmx4m4BTUXHE//9vvvkaI0aMjEfzhBBRlqh1xmIhFindkZQS2Wtbt37ISy+9wMiRJ1NSMgWAe+75HS+//G++/no7oDBgwAD+8pdH49tQIURUBMvcinedsUAi3StpnW4N+P/bsHFd3KsPBJISQef00ydTWdlw3PGzzjonDq0RQsRa9oxZVK98FqDNnE6PC+fEuWVtRWOPm1ikdEdSSgyvCSFObMEytxLtTj8aC00TtZp0MCnR0xFCpK5Qh6NC3VEznqLRK+luLy/WSQjS0xFCxFQ462mSbZK8M9HolXSnlxePz1d6OkKImAl3TiPZJsk7E625p6728uLx+UpPRwgRM+HOaaRaKnSizT3F4/OVno4QImbCndNIplToUCXS3FM8Pl/p6QghYibcOY14brkcrVpuiSRr+kweqc1j0t5i5h7sG5PPV4KOECJmwg0i8RqOSrUEhkD+/ckhRi3/jn87CwE4X62Iyecrw2tCiJjxBZHWKbo9LpzT4UUuHsNRqZbA0NpnB2q55PEP/I8H5qXz1o1TSTfGJhxI0BFCxFQizWkEk2yr/ENR0WBj0oNtEzY23VFC/57pQV4RHRJ0hBAJIdxFitFc1JhKCQw2p5u5Sz7g67KWUmHPLzydyUPyOnhV9MicjhAi7sKdQ4n2nEs8ExgiRdM0frN6OyN++7Y/4Nx30cnsWzw7bgEHpKcjhIiyUHok4c6hRHvOpStzT4nkxY8PcseqL/2P547rx18uH4OiKHFslZcEHSFE1IRagSDcORRnVSWoKo4d+/FYrejMZvSFRXgiOOeSDHNP7X2yv5a5S1qSBAbnZ/DGr6ZiNqpxbFVbEnSEEFETao8k3DkUxWDEtvNbdCYTismE5nBgL91F2vDON2qMxlxQvHfuPFJv4/Q/tk0SeP/XJfTrEdskgVDInI4QImpCLbMS9hzKsVEiDQVQjv235Xgw0ZgLiueaHpvTzbl/e69NwFn5s9PZt3h2QgYckKAjhIiiUCsQhLsIVHM4MA0djs5oQLPb0BkNmIYOR3M4OmxPNPazicbP7Iymadz1sjdJYMeRRgAemDOafYtnM2lw/JIEQiHDa0KIqAmnqnIocyi+YSzHgf2g12MaMBB9Ti4AbosFtWfPDl/flfU3nQ2dxXpNz/MfHeCul7f7H8+f0J/Fc09JiCSBUEhPRwgRNZEsY9N6GMs4aAiazYZt106cdbUhpzSHW/stlKGzWO3cuW1fDcV3vuEPOCf1zuTb/zmXP12WGFlpoZKejhAiqiKVBdZ6GEsFlGEjcBzYh2PfXjLGTeg0pdlaugtXfR3Wb75Cl56Bsf8AdAZjh/vZhJIIEa09cnzK6qycsXh9m2Mf3DmTPrnmiPz8WJOgI4RICu2HsfQ5Oaijx+CqqaZg4aIOX9s6dTtt+CicB/dj2/kt5pGjO+x5hTJ0Fq01PTanmwv//j6llU3+Yy/9fDLfK+54CDEU8cy2k6AjhAhZPC9W3SlN076XZMjN9c4BZWV12P5Q3zOSa3o0TePOVdtZue2g/9iDl5zCFZMGROTnh7t7a6TJnI4QIiTxLvffndI0Xd0hM9blcP7fh/sZdNeb/oDzw4n92fvH8yMWcCA+2XatSU9HCBGSeJf7784wVld7SbEqh/PR3hrmPbHF/3hEYRarr59CmiHylQTiXUFbgo4QIiTxvlhB14exujPZH81yOIfrrExplySw5a6ZFOW0TRKI5LBmvCtoS9ARQoSk9cXKVV+Hs7wMd1MTamYW1tJdCV2nLJweSyzmrawON7P/vok9VRb/sVWLJjN+4PFJApGeg8meMYvK5Uux7WlAczlR9AbUrOyIZdt1RoKOECIkvt6Cq7EBZ9lhQAFFQc3tEdOJ6K4KdfFpNCfZNU3jtpe+ZNWnh/zH/jT3FOZ/L/icTVSGNRVvxSBNO1Y5KIbLfCToCCFC4ustVDz+GHjcqJlZGIr6os/JwW2xpMRWztGct1qxZR+/e/Vr/+MrTx/A/ReP7nRhZ6SHNRs2rsPQqzfqwEH+Y7H8/iToCCFCZh46DH1+PmnDR7S5WCb7Vs4+0Zi32rK7mh8u+9D/+OQ+2axadEbISQKRnoOJ99ycBB0hRFjiPREdTeGeW0fzPwdrmpn60IY2z9969ywKstOwlu6iIsR5o0hXPIj395cS63QOHz7EJZfMZsqUCUydOpGlSx8HoLa2hssuu5hJk8Zy2WUXU1dXG+eWCpH8EnUrZ2vpLiqWLeHQg/dRsWxJl9YPhXNuwdYtVX+zg+l/3tAm4LzyizPYt3i2P+CEs94pkvXrwj3HaFAqKxu0mLxTFFVUHKGi4ghjxoylqamRs86axr/+9TwvvPAsPXr04IYbbuGxxx6hrq6O3/3ufzr8WeedV8K2bdti1HIhklO8Ny0L1B5fAkDr3kBXLs6hnlvFsiVtegyaBr8rz+W/9lz/c/582Rgun9C/w9cB/uoInZXziZRIf39jx45jzZp3Q3puSgyvFRQUUlBQCEBmZhbDhg2nvLyMt99+g9Wr3wRg/vwrmDPn/E6DjhCic4m2lXMkEwBCPbfWcyMr67N4pKZlnuTqM4q5Y4Sexndf59B/217Y4z2nAvH9/lIi6LR24MB+tm//kvHjJ1BVVeUPRgUFhRw9ejTOrRNCREOoF/JIL7LcWu3hhrpi/7EReivLBlTRa+TQoKnX8Z5TibeUCjpNTU1ce+2PuP/+xWRlZYf8uhUrnuKZZ54GoLa2OkqtEyL+Em1YLFIUoxHrV1+iuVzozGYMRX1R9Po2F/JIrsE5UN3MtN1t19b8J28nPe315JUs6LDnFe2tEBJdSiQSADidTq699krmzp3HBRdcBEB+fj4VFUcA77xPr169Ar72qquuYc2ad1mz5l3y8/Nj1mYhYineBTujwVq6i8N//iOWLz/HWVuD5vbgtjuw7dqB82hlm8nxSBS6tNhdTFm8nml/bkkSeKrvEd7P3kZBrtkfwDoqMBrpxIBkkxI9HU3TuOmm6xk2bDiLFv3Sf/ycc85n5crnuOGGW1i58jnOPXd2HFspRHzFu2BnpPmCqONIOfqsbDwOJx6rBcWThs5sRt8jL2LbSns8Gje88Bmvf1nuP/bIvFO5dFy/gM/vbAgt0ebEYiklgs7WrR/y0ksvMHLkyZSUTAHgnnt+xw033MzChVfz7LMr6NevP08++a84t1QIr3gMc8V6Ajva5+gLorhcKCYTeoMBj9GIzmggbfhIXDVth8q7Opfy5KY9PPDGt/7H10wp5ncXjOqwksCJPoTWkZQIOqefPpnKyoaA/7Zq1Wsxbo0QHYvXJlqxnMCOxTn6gqjObEZzOMBgQNHr8VitAc8r3EDwfulRrvznVv/jsf1zefG6yRj1nc9KxGpLhGSUEkFHiGQSr2GuQBdd59FKNLebQw/eF9HeSLjn2JVekS+IGor6YNv9HTrAo4FOrw8YTEIJBNbSXXyz5l3mHujT5rUf33MW+VmmsNp9Ig+hdSRlEgmESBZd3cWyu9pPYGseN2igqGrEEwvCOceuJjj4VtYregOmwUPQFAXN1oyxeFDQHpV56DAKFi6i3933UrBwUZvnHP16B9OXf9Um4Dxp/opvfzI0aMBJtcSMWJCejhAxFs91Gq3vviuWLUHRqVHpcYVzjl3t+bXuuXiqKsk49bQu9dQ8Ho1fPv8pb24/AniDy335VZybacFtUYK2I9USM2JFgo4QMdadSeZITs5HM7EgnHPsTju6O4S19L3dPPjmDv/jH2bXc1NeS43GjtqRCJUFkpEEHSFirKuTzJGenI9mjyucc+ysHdHIgnt3VxX3v/4N31U2ATBhYA/+pn6BrqkRCO3zONErC3SVBB0h4qArd+iRHs6JdlpvsHNsH0RMQ4bS9MGmgO2IdKDde9RCyV82+h9fclpf7pk9kl6ZJqylvcL6PCQtumskkUCIJBHpBIR4rIwPNPne9MEmMs+YGrAdkagiANBoczLxD2vbBJzXf3Umf50/ll6Z3nmccD+PE72yQFdJT0eIKIn0sFBXhnM6a0Os03qD9dbsu0sDlvUPZd6ko3P0eDQWPfsJ73xd4X/+Yz88jYtObZsS7RPu5yFp0eGTno4QURCNdNpwN99KxJTecHtrhvzeeJqb2xxrP98T7ByXbNzN4Lvf9Aec66YPZt/i2UEDjogNCTpCREGkhoVaC3c4Jxpt6K7Ogkh7nQXaQOe4VS1g5D9L+dPb3qy0SYN6UvqH87jrvJHRPTkREhleEyIKopVOG85wTiKm9IY7+d5ZFlzrc9zv0DPvcEsBTqOqY8tdM8nLPH5hZzSl6vYRkSJBR4goSIR02kRoQ3tdSRfvKNAa8ntTX29hfs0waj2q//gz/cqZ+sufRrz9nYlXXb1kIkFHiChIhHTaRGhDIN2dfPf1JGyVldxtG8ZmZ8tmag9kH2SGp5y8cxZEoqlh60pa+4nWM5KgI0QURLrKcFcuTPGqdBzNi6ivJ/GMNpAnLBP8x68yV/Azw8Fj7xe/XkW4Q5onYs9Igo4QUdL+jt5auouKZUvCvhh358LU1V5FVwNHtC+ib76xmVubJvofn+as5CH7h6TnFdP39nu7/fO7K9whzROxfptkrwkRA91JX451FloitvW7yiaK73yDW494L94mj5NV5Sv5S/P7qJoH67dfJUR153DT2uNVcTyepKcjRAx05462O1loXemxxKutgdRbnUx7aAP1Vqf/2NIjrzLIXYeCgqfR6m3rscAW795BuEOaiZjsEW0SdMQJLxYTud25GHf1wtTVoa54tLU9t0fjp//6mA07q/zH/lhQxZkNu7Haq9FUFUXVgVvD09yMacjQhOkdhDOkmajJHtEkQUec0GI1kdudi3FXL0xd7bFEq63Bgnv74/8vZyx//7TG/zN/NfMkbv3+cA49eB/6ggKcFUdwNzWgud0oqopiMKAzGFB79uy0fYnmRNzWWoKOOKHFaiK3O3e0Xb0wdbXHEo22AgGDe+YZU2n6YBO69Aw+SOvPHbsLAG/AmTq0F09d/T30qnfq2RcMTcXF3u2p9Xrv9tQ6Jal7Byda/TYJOuKEFqtV+929o+3KhamrPZZotLVi2ZKAwb3utdUcLhrKgsqT/M9NV9ysHljOsJ/MbvMzfMFQl56BafAQHAcPoFktGEeNpufFc0+oC3cyk6AjTmixnMiN9R1td3sskWxroODeZMpkjmEWzTVG/7Hn+x5mkMGBq6Y6YJsisT21iC8JOuKElsoTuYk0X9A6uLs0uKWigK1Ws3/RxkO9K5me4S0E6rYED/on2lBUKJKtooEEHXFC812Ya15dRfPnn4KiYBp8UucvTBKJcpH2Bfcnm/JZ3twSUH4+ROXKii3oyEDTUivox0IyVjSQoCMEoNlspA0b4e/thPOHm2x3mvHwrj2bn7eqJDA53cqy+aeQOXw41tLBQXtj8fxsk+F7DZYIU7N6Ffrc3IRsuwQdccLrTgZbMt5pxtLOI42c87f3/I9zzAbeu72EnHSD/1iw3lg8P9u69WupefE5NLcbXUYGHqczIb/XQHNlHqcD285vSR99akL+TkrQESe87mSwnYi1s9oL1COw9Slmyp/W0+xw+5+39pZpnNQ7K+SfG6/P1lq6i+oXnwPN+3uA04mz7DCGPn0T7nsNlAjjOHgANYF/JyXoiBNedzLYIpVynQxDOa35txfYtwdXdTXGor4YCgqwNzRy5dPb+MRd6n/uk1dN4KxRBWG/R6TT2UP9jBs2rgO3C116JooCGAzoAHdtDU6D4bjnx1OwRJi04aPaPC+R6rlJwU+R9HzVmw89eB8Vy5aEXfgx3CKNrYW7/XKw9ne1wGY8tG6vx2IBDZxlh1hyJJ1pVSfziTsHgFvPHsa+xbO7FHAgMp9toDZ39hk7qyrRZWSiuVwtB/V6PBZLwtVEC7SFuXnUaHTtgmMi1XPrtKdTWJjLyJEn43a7GDBgIP/4x1JycnIj1oAXXniWzz//lMWLH+bNN19nyJCTGD58RMR+vkhtkRj372pqsbV0F676OqzffIUuPQNj/wHoDMaws6+SbYiudXs1m41N6cX83jQebN5/n2Ju5kH9N/QeMLRLWzn4RDKdPZzP2JDfG83pwll2CA+g6PV4bHYUVQ3pRiTWAm2hkcjLADrt6aSlmdmwYTPvvbeV3NweLF++LGqNeeut19m1a0fYr7vxxl8watRgpk2b5D/20EMPMmbMcEpKplBSMoW1a9+JZFNFgohUKX3z0GEULFxEv7vvpWDhopATCBSdStrwUSiAbee3aG532BO2yVbe3tfeUoeBmblzvQEHyPHYWTvgAH/O2oveZOx27y3QXXxXJ8PD+YyzZ8xCUXUY+vRDMejxNDeBAj3nXZGQNwHtRfJzi4aw5nQmTJjIN9985X/8v//7KP/5z8vY7Q7OP/8Cfv3re7BYLCxc+GPKysrweNzccssdzJkzl/HjR/Pf/75LXl4en3/+Kb///W9YvfpN/8/66KOtvPPOm2zZsplHHvkzy5c/w9q17/Cvfy1HVfUMHz6cpUufDtiuH/xgAT/5yc/45S+va3P8uuuu5/rrbwjnFEWSiVUZm/ZaBzsVMOTm4rZYvMMbXSxXo7lcOMsP47FaUfR6jMWDotP4DoQy72HpWciF+/rgaHXP+s/qNxjorEWpNWJTVdTehRhyc7vde4vUOqNw5u1a93wVgx5D/ikJP8fWWqLPD4YcdNxuN5s2vcuCBT8CYMOGdezdu5t33tmIpmn86Efz2bJlM0ePHqWwsIjnnvs3AA0N9SH9/IkTJ3HOOefz/e+fy4XHuoGPPfZXtm3bjslkor6+LuhrJ0+ewoED+0M9FZFC4rUfSSSDXfaMWVQ+tRRXZSUYTSiqisdmw1VdjbV0V0zXpXQ0VOl0e7jyya1s3dfP/5qHex9hXMVXOC0VYEpDl5GBvkce9v17UDPSUWn5XuLZezMNGdomBVrt0ROdqgYdckqURbWdaR9gTEOG+guoJmK6NIQwvGazWSkpmcLw4cXU1dUyffpMADZuXM/GjeuZOfNMZs2aSmnpLvbs2c2oUaN4772N/M///I4PP/yA7OycLjdu1KiTWbTop7z00guoaviJdsuXL2X69MnceOMvqKur7XI7RGR1d+K/te4kAXRHJCe5zUOHoe+Rh5KWhuJxozMaSRs2HEN+76jtDhpIR0OVf3p7B0PveYute70VoG+ekMfWIQeYZDuE1tSEmpWFzmRCURR0ZjNqegaOgwfa/Px4TWZbS3fR9MEmDEV9vWtuLE04y8vIPGNqwlyIuyJQcoQ3sHpitstsV3R6JffN6TQ01LNgwTyWL1/KwoWL0DSNG264hR//+NrjXrNmzbusXftfHnjg98yYMZPbbrsTVVXxeDwA2Gz2kBr33HP/ZsuWzbzzzps88shDbNr0EXp9aMHn6qt/yq23/hpFUVi8+AHuvfceHn308YDPXbHiKZ555mkAamuPLzQoIifQ3XTl8qXo8/LQHI6whwPiVV8s0jXbNKeD9NFjUBSl5ZimYd+3t1uT8eEI1Htbr+Xzm929YfduAM4eVcD/XTkeVacAp2Mt3cXhP96HkpaOYjDgcTix7y5FLSjEeegAbosl7pPZrYOpsbAQALfFgn13Kcw8K+btiZRAyRGa242rttp/npB484Mhdx+ys3N48MGHuOqqH3L11T+lpGQWf/rTA8ydO4/MzEzKy8vQ6w243S5yc3tw+eU/ICMjkxde8P5h9u8/kC+//IxZs77PG2+8GvA9MjMzaWpqAsDj8XD48CHOPHMakyZNZtWql7BYmkLOnOvdu+WO6sorf8yVV84L+tyrrrqGq666BoDzzisJ6eeLrmn/h6K5nLiqKvE0NmAePabL2WexvmONdLALNEzorKjAVX0UfV6vbg2VhDLGby3dhauqCvvePaiZmezNH8I1DcP9/94r08T626aTndY2Fdf3fSqahqLgnXgHPDXVmEeNRs3Kinux0XjN+0VbwPM61pNrLZHSpSHMRIJTTjmVk08ezSuv/Jt5835IaelOZs/23imkp2fw+OPL2Lt3D/fd91t0Oh0Gg56HHvorALfffic33XQ9f/vbw4wbNyHgz58z5zJuvfVXLFv2fyxd+hQ33XQ9jY0NaJrGz39+fVip2hUVRygo8Eb7N998jREjRoZzqiJK2v+hOMvLUEwmNJfLPxwAiZsu3Fokg12gnpOj/DCGoj7dmowPJaXc9xw1twfVzQ7mGc/F3dAy8v7mZQMZNWF0wJ/vrKrE2H8A9j27vWP1ej2g4W5uTpg9buI17xdtgc5L7dETj82WED3MYJTKygYt3o3oruuuu4bNm9+npqaa/Pze3HHH3WzevImvv94OKAwYMIC//OVRfxDqyHnnlbBt27boN/oEVbFsSZs/FMtnn6DpVFSTEfMI7ypqTdNw1VTT7+5749nUgKKZGdT+Z9v27cHUf+BxQ27hfDbtP2/An2VXsHCR/zn2hkZ+1TCEL+xp/uf9yfkxF/284zRh38/XXE6c5WV4rFbQ6zENHETf2+8K9yOIitaBt/WFOJEm17si2HllnjEV++7SmGavjR07jjVr3g3puSlRBueJJ5467tiCBVfFoSWiM+3v6NHrwWbD0Co9OBp3oZEIFtEuQNm+5xQoYIT72YQytPToQSPPOU72P/6FuYzLaz/D3dDgn4AOdn6td/NMGz7Sf+HrOWduyG2MtkTaVyiSOjyvBJ6rSomgI5JH+z8U08BBuGqrUfR6NE2LynBApIJFrCsHRCJZoaOhpVc/P8yNL3wOeEcAStIt/I/pO5x7SvGgoGZnd/pZJcsFPVlSoMOVjOclQUfEXKCyHdG8aEUqWMR6QjoSF/RAgevbRo2fVA6A3Z8DUJCuZ4XyIVkZZuwHDqGhABrGPn1D+qyS8cIn4keCjoi7aF+0IhUs4jEh3d3PpnXgqqyo4aKmU48FFa+Nt82guFcG1tJBNGxch7uhATU7G2OfvuiPJe6kQqaXSBwSdERUJFIpjkgFi0ivzYkVddBJLFpTxadNLVU9Vlw7kWnD8v2PWwe3VMz0EolDgo6IuETbTTNSwSLS8xexCMz3v/4N/3x/r//xPeePZOG0wUGfn6yBNZiOPuNEujE6kaREynQkScp094WSphuqSF0YEu0CE+003lc+O8TNK7/wP75gTBGP/eA0dDqlg1e1tC2RPquu6ugzBlIyjTpeTriUaZFYIrmbZqR6TJGaN+ruBdn3esun21BUFeOA4oguiv3yUB0X/e9m/+O+uWbeuXkamabQ/9RTJTGgowQSoNPkklQJvolGdg4VERepYpiR2isnUrq7w2fr12uaB49Hw767FFe9txJ7dybsKxttFN/5RpuA897tJWy+c2ZYASeVdLSHTmf76yTbbq7J5MT8bRRRFal5gfY9Jld9PY6yQ7gbGvzvE8s7z66kXre+W3ZVVaHLzcWQkYFqTkdzOND0Bpzlh9Hn5HQpMNtdbub93xa+ONSyhcizP53ElJN6dfEsA4vlXX+k3quzBJKO/i3ZdnNNJhJ0RMRFasK99UWCWtx4AAAgAElEQVTDVV+PfXcpWoiLFqMh3GHD9sOD9r27cTdbUM1mDEV9sO3+DkWvx93c7N+SIdTArGka9732DU9/sM9/7LcXjOInZ0Zm47fWF37FaMRVXY0hv3fUE0MiOaTa2c1PR/+WikVCE2W4UIKOiIqO5gVC/eVvfdFwlIW/aDHSwk29Pu5uOTMLd7MVZ3kZ5hGjSBtyEvYD+71Dh1lZIQfmVZ8c4taXWpIELh7bh7/NH9umRlsgoX7u7S/81q++xGOzoe/RI+pFWSPZw+js5qejf4vlmqxYBINEyiiVoCOiItgfUji//K0vGomwaDHcYcP2d8uGor64v9uFu6kJTdNQ9AaMhUUh/+F/frCOOf9ombMZ0DOdt2+aSrqx8z/jcD7347efcKGYTDjLy6L+2Ue6h9HRzU+gf/P93tr27cFVXY2xqC+GgoKopY7HKhgk0nChBB0RcR39IYX7yx+LRYuh3mmGO2zY/m5Zn5ODsW9/3HW1uGqqQx52rGywMfHBtskTm+4ooX/P9CCvOF44n3v7C7/ObMZtd3grSB8Trbv+eG5D0Pr31tR/IIrRhLP8MJrDjql4UFRqysUqGCTScKEEHRFxHf0hdfWXP1qLFsO90wwnnThQmxVVR8EvbvD3+ho2rqP6pecDBju7y82lj3/A12UN/mPPLZzEGUPCTxII53Nvf+E3FPXFvWsHpKVFrSirT/aMWVQuX4ptTwOay4miN6BmZXf4XpEanjru99ZsxqXX46o+iqk4MnNl7cUqGCTSnkKSMi0irqN01K6mU/t6GWpWFq6aatSsrIgMQUQzLbujNneUkqtpGr979SuG/+Ztf8D5/YWj2Ld4dpcCDoSXxp49YxaeZgtui+XYMKAefe/e3orgEfzsg1LwVofTjv23g6mqSKY2t/69ddXXYdv9HWgamqZFLWU6UssLOtP+O/UlrmTPmBXR9wmF9HRExHV0V9X67t/jdOI8uB93swXzyNFYS3d1eCGLxqLFaN9pBmtzsN7gs//5kAeqSv3Pu/S0vjw879ROkwQ6E05PMZ7bFTRsXIehV2/UgS09C7fFEnS4KZLDU61/b53lZej0em+2ZHp6VIa9rKW7cNXVYf32K9T0DAz9B6IzGKLSi0ykLSgk6IiI6+gC5/vlr3l1Fbad3/g3/1JUNS7ZNO0DpKu+HseBfWhuNxXLlvjvBCOdXdQ+2G23mfhpZbH/8aBeGbxxw5khJQmEItyLTryqEoR7ExCJmwbf8Jx9315c1UfRF/XBbW0GnR7F7cQwsLhLP7ez92y9+Z3j4AFsO7/BPGp01P4GEqXShAQdEbJITbibhw5Dn5OLaeBg3HU1OPbuQWc2o+b2jHk2Tfuel710JwCmocNwNzZSuXwpKGDoFdk1Kr5gV23K5sKD/dv82/u/LqFfj9CTBELlu+j4vsfKp5ehNVvRpadjKh4U9XUbofz+hDv3EOj5zooK3HW1HHrwvk5vElpf/I39B6AYTTjKD6O5XKhpBozFQ9Hn5HTajnC17qGpZGDI7eGvT9jZd5Ao6226SoKOADr/RY70hLt9317vwkODEcWUhsfhxH34IJrDHpXzC6Z1gLR98jFKWhqmAQP9qcG2PQ0o4B/uUTMycDc2UvH4Y+jz87v8R286s4TLnvmS7zwtF8u/m7/lrCsuwhyFgOPj+x49bjeuo0cBBbelCcVoimpPM9Tfn3ATRto/31lRgX3fHozFg0BVsXzxGY1b3sc8ajQ9L57baXq4sbAQNSsLzeNGs9mitqNtV3toibTepqskkUCENBkb6Ql3T3MzKDp0Bj2KAjqDHhTdcZOq0WQt3UXFsiVUv/Q8AGpeHumjx/gDDoDmcuJxOv2PXfX1OA4fxN3UtYlrTdO4+5XtjP3Xbn/AuSVtP1uHHPAGnChfOHzfo6euFp1ej2pOQzEYcdfVRLWuXai/P+EmjLR/vruuFrV3b1wVR2j+/DM8DfVoOj2OfXsDfk/Bkl40hyMqiSs+XU0gSLR6hF0hPR0R0mRsV+7MOuo9eRRwWxpxNzagGPToTGmgKCjp5i6fRzjDDoHuGF3V1ShGE6bCIv/zFL2hTfKUs/wwKDrUDHPYq/Nf+OgAd7683f/48vH9eOiyMd1OEgiH73v0WK0oJhMAil6Px2qN6rqNcOrohTv30Pr5+++5HffRo3iszSiqiqaBZmvGrXkwHbs4t/7ZHQ3nRXMOpKtLABJpvU1XSU9HdFpxF8K/M+uo92Qt3YXW3IxiTENnMIDLhafZgi43l7Ti4BuMdSTc1NlAd4zGor64ysvapJWqWdnosrNbjjU1AhqGoj5BP6v2PtlfQ/Gdb/gDTrHByac/HsKfL+9+Vlq4fN+jzmwGlwvwVhzQmc1RXbfR+vfHV0fPY7W1qaMXiXRkrdkKKODRUFQdik4HGuB2Bfye4pVK3NUlALFKsY4m6emIkCZvw70z62wvE2NRX5xlh8Bs9t5p22xo9fVd/mMPN3U20B2joaAAzWFHzcpqkwDh+znOqkrUzGx0ublthuCC/dGX11uZ/Mf1bY6t7neA3o5Gmld9hjkO4/C+71GX2wNn2WFwuUHzoPYuiOoOobGqo6dLT8dtaQKdgubR8EYcDVR9wO8pnqnEXelJpcLOrhJ0REi/yOH+cXY2DGAoKEBnNuMsP+wd2jGnoWZkdvmPPdxhh2CB1lQ8yL+7afvhurzLfwh4qxO7LZagn1XttzuY+9y37HEa/ccez93L+B7HejWG+NW9av09ag67P3vN2KdPVLOgYlVHz1Q8yFu+pqIc17F5K116JorJ2OG6pGSZhE+k9TZdJUEnRVlLd1GzehX2Pd/hcTjQpaWhz+0RMDU21F/kcP44g13UFYMRZ9lh7Hv3oGZmYijqgz4n158u2lXhptp2Fmg7yhIK9llpmsZtT73Pql0NgDfg3JFVxvd3byStxyigpXcUz3H4eF1kY1FHz/e9pp00rM3i47SBw+k55/jstWSUTEEyEKWyskGLdyMSyXnnlbBt27Z4N6NbrKW7qFy+FFdVJZqi4LFYAA0lzUzawEEoqi7qKZaB9qd3VlWCAjqTGcfhg6DoAA1Dn77oVLVbbQr4fkcr0ffIQ3M6gqaBB0s8qFi25LgLoy8w+npCrT27dT/3vPKV//HFWY3clVeNooBl+xcoQPopp4b0s1JdoO/K02yJ2O9ksq9jSUZjx45jzZp3Q3qu9HRSUMPGdbgbG9CZTN41GKoKgOJy4q6rwdh/YNSHdgL1njSPG0WnetM8jw2tuZsa8dTVkX+sCGYwrVeNe5qbUdLNpBUP9l9Q2r+fYjR6a3epKmpW4PUMHd0xhjpc99HeGuY9scX/eIiumX/2q8SstjzH2H8Atp3fdjgkdyIxDx1G5hlTqXttNa76OvQ5ueRGcIgo2XsCqU6CTgpyVlV6K/Sa0tBcLtCpKIqC5nb5U2Pt+/ZSsWxJVO8G26+At37z9bGx/H7oc3LQ5+SgaRqumupOA071ymfR3J5jvSUdNFuwt1vQ2PpiU7FsiT/AQUtiQc3qVehzczs9786G6w7XWZmyuG2SwJa7ZqJ78WncjRZo9TqdwYh55OjjEhQiWcMrme7sraW7aPpgE8b+A0gbPgJPczNNH2zC1H9AQrdbRIYEnRRkyO+No+wwuFzeFdVuj7dasE7n7WFUVHhrTOX1iun2w2p2Nh6rDfvuUhjiLS8SzoI4x8H9KAYjOoMezenEU1eLPkivLVBPxeN0YNv5LemjT+30vIPN+aSddxEzH97IniqL/7mrFk1m/MCe3vMN8rpgn23d+rXH3fHnzjwrxE83OVeoJ9KGYiL2ZJ1OCsqeMQs1KxuP3Q4GI5rbjeZ2oekNqLk9cZQfRl/UJyarmltfYIx9+gIaGop3YWCIayJ864g8ViuK/th9UicLGgOtZ3AcPIAa4mru9usodJlZPFw0i3H/2u0POIsvPYV9i2f7A06g13W0/qJu/VqOPvMU7uZmdJlZuJubOfrMU9StX9vZx+qXjCvUQ1kXJlJXSvR0brzxF6xZ8za9euXz3ntbAaitrWHhwms4eHA//fsP5MknnyY3t0ecWxob5qHD6H3tz/zZa5jN/uw1XUY6msuJ/cA+HHt3g6pHn52NvrAIT5S3H9bn5JI25CQcZYe9abNZWSENM/mGunRmMx6HE8Wgh04WNAbtqQwf1eZ5HV3sfMN1z2zZx29f/RpoBGDBpAE8MGd00IWdoc4p1L22GsVoQjWnHXthGu5jx9v3doINoSXjCvVE2lBMxF5K9HR+8IMFvPDCy22OPfbYX5k2bTpbt37OtGnTeeyxv8apdfFhHjqMvrffxeAl/+Skfz7D4H8sI//qn3qLGOpUNKsNzenCY7Pitliwl+5CMRg7/8Fhat/j0OfkYhpQjHnkyQBUv/Q8FcuWdLga3bdqXM3tieZ04Lba8Lhc6HJ7BO0pBepxmEeN9lZAaKWji92He6opvvONYwEHRhZls+P+c/nDJadEpJKAq77OX4rGRzGZcNXXtTnWUbWFZFyhnkgbionYS4mgM3nylON6MW+//Qbz518BwPz5V/DWW6/Ho2kJxTcUo+gNgAY6HYpOxWOzep8QhYosgS4wzqpKXLXVIZes8QUQY58+GPJ7o6ano+/VC1Ofvp0WgyxYuIh+d99LwcJF9Lx4bkgXu0O1zRTf+QY/WPqh/9iHd83irRunkmZQ279Nl+lzctHsbatqa3Z7m2oH0PEQWjJewLtaAkakhpQYXgukqqqKgoJCAAoKCjl69GicWxR//qEYjxs1JxfN2uzNbtMUTEOHozkcIf2c1kM9vtTk1mthoO2mZ5lnTMW+uzRg6jSENpHc0ZBVZ+1pnSLd0SLYZoeL2Y+9z96jLUkCL//iDMYNiM6wbO6Fc7xzOnh7OJrdjuawk3vsZsmnoyG0ZF2hLmnNJ66UDTrhWLHiKZ555mkAamur49uYIIKN6YeTLmvI74297DAemxWPw4nOaPDfQesMBtSePQO+rn07fNlSqHpsO74FOt70rOmDTW3uZA89eB9qhOYhWrfHbbNh3/4luF2gquiysrHt2U3va3/W4docTdO49cUvePmzw/5jD102hnkT2m6uFmm+eZs22WvzrzhuPqezORC5gItkkrJBJz8/n4qKIxQUFFJRcYRevXoFfe5VV13DVVddA3grEiSaYGmxmWdMpemDTSGny5qGDKXxg/fRdHpQXHgcTrDZUXvmhbxYsfVQj2PHNyimNBQ0XEfKMY8YFXDTM9/rfG3yXUQ1l8tfe03R670bb4XJ1x7N5cL+XSl43IACmoanqQmHw07N6lX0vf2ugK9/evNefv/aN/7HV00eyH0XnRyz6s+5M8/qNEU6FYo8CuGTskHnnHPOZ+XK57jhhltYufI5zj13dryb1KGOeizB1jXUvbYaY/8BIQ9T2XeXYir2bhHtasDbI1B0YLeT95PrQrpbbj3U492TJQ3Q8Fi980Kay4l2rLCSq74OZ3kZbmsziqLDWroL89BhZM+YReVTS3FVVoLRhKKqeGw2XNXV/ueEytce285vj52P4p2b0jRvJQaPx5vB184Hu49yxbKt/sen9M3h34smY9JHbs4mUpJ1CE2IQFIi6Fx33TVs3vw+NTXVnHrqCO64425uuOFmFi68mmefXUG/fv158sl/xbuZQXW2wM++by9uSxOazYbObMZQ1Bc1OxtXfR1pw0e0+VkdDVM5qyoxFBRgLCz0HwulIkBrrYd6/CnMaN79WWjZ9MxVX4dt93fo9HrQ6VF0Sptz0vfI827gdSz12TCwGEVvCHuBoK89vqDX8kHoQFGOzVm1lBc8WNPM1Ic2tHnqR3fPond2WsjvGQ8yhCZSRUoEnSeeeCrg8VWrXotxS7qms71nXNVH0TTQpaXhcTix7y7F0Kcf+pxcPM3NIa93iMT6iNZDPfrCPthLd6IBpgED/ZueoYD9wH5vNQQUFLcTY/FQFL3eH1Q0p4P00W13zdQ0Lex5HV97FL0e9HpwOgEFjHo0txtFAdNJQ7HYXZz76HscrGkJTquvn8LY/rnBf7gQIuJSImU62XW0Qrth4zr0RX0ADVxOFL3qXdFffpjcC+eElS4bifTa1umuuF2kjRhJ2vCR4HajZmXR+9qf0fuan4HL5d2R0mjAdKzkTZv9dCK0vsSfTl08yBt4VBX03mE10ND1LuDB7DM4+d53/AHn4ctPZd/i2RJwhIiDlOjpJLuOeiDOqkqMBYWoZjPO8jJcDQ3gduHxaNh3lx6XjtzRWH+k5gZCGerJGP+9DntVgSbHnVWVaB43hx68L6zCleahw+h7213ePYReXeVNKFAUVhedzmPNfWBPEwDXTCnmdxeMivkW0UKIFhJ0EoBpyFBqXnwOze1Gl5GB2qMnOlWlx4Vzjm1T0OhfMOhubga9AYM5LWA6cme6OjcQamq273m2fXtwVVdjLOqLoaDguIyr47YiMBhBwbt+p5NMvGBt8QWf90uPcuU/t8KxjtTY/rmsvO70hEwSEOJEI0Enznxl3g1FfXHVVuOxNOGx2cibd4X/Ylu98llcjQ049u7G43Cg6FQMhYUxq84baiXj1s8z9R/o3Ta4/DCaw46peNBxvarjtiJQO18w2lFbKnv2ZfqfN7Zp+0f3zKJ3VmInCQhxIpGgE2dtqjAfyypzWyze8v8zz/JveFX94nN47HZvaf+0NFxHylEzMlGzsyNa3DFQLyLUUvTtn2cqLEKflR3SDpmhFq4M1BaLR8eMf31LhavU/7z//HIKY/rJnI0QiUaCTpyFcrG17y7FPGwEjoP70RwOFIMBj9O7sFLR6yNW3DFYL8JjtWLsP6DDNoZ6LsGEmlnXZp2QBr+pymedpeU1f5s/ljmn9Q3thIUQMSfZa3EWShaXL7vNUNQHj8uF5nSCqnq3eo5gccf2hSU1lwvnkXLsB/Zh/epLXPX1QdsY6rkEE2pmne89nq3PZvK+Yn/AuSKngX2LZ0vAESLBSdCJM9/F1n6knOZvv6Zp20fYdu3ANGSo/zm+C61vPxrFaESzNqNmZke0Om/r1G1XfT323aV4PBoYjXhsNmy7duCsqwsaELqTkh1q5eHtJ53OGZUn81iNt07cGIOFDRkf89vLxkfkMxBCRJcMr8VZ6zkb3C50GZnoe+S12TO+dXqxmp2Dojd0uAVyV7Ue4nKWHwa9AR0aanYOhqI+2A/sx7F3Nxnjvxcw1bq7KdkdZdbtO2phxl82+h/r0Hg18wt6F/Qke8YVUU2kEEJEjgSdBOCbs2k9n+G2WPwT9bGqvdU6uLmbm1H0ejwuF2kDi9Hn5KKOzsFVU91hUkCky7U02pyc9ci7VDS07Dvz+q/OZHTfHOCCiL2PECI2JOgkgNaT48GKZHblYh7OtgfQtqeiKAooCmlDTvKvEYrljpQej8Yvnv2Ut78+4j/26A/GcvFYmbMRIplJ0EkALaX+nR0WyQxHqGtr2vMFN6u/ppkBTdNiWk7/iXd388e3dvgf/2zaYO4+f2TU31cIEX0SdBKAb1jLcaS8wyKZ4Qh1bU0w8Sinv3FnJVc/9bH/8cTinjy7cBIGVfJdhEgVEnRioM12ysfKvWiOttsp581fQPnDi9E0DTU9HcPAYvQ5Of7Ky+2HykxDhrapudZ+6Kw7a2Z8YlVOf09VEzMfftf/2KAqfHjXLPIyTVF/byFEbEnQibK22zur3s3GANPQ4ccNeQUrkqkYjW2GyhxlZTR+8D7G4kEYCwoDDp1FYhuDaGuwOZn5l3c52tSSJPDGDWdycp+cOLZKCBFNMm4RZa2HuVxHytGZTCimNFxHyrwboaVn+PfNMQ0ZinXXDpq2baX5269xHDmCp9kCGm0WbbrralCMJjx1tSiKctzPgchsYxAtHo/GwhXbGPP7//oDzv9ecRr7Fs+WgCNEipOgE2WtF1x6rFbQ672pyMd2uvQNebUU/uyDLiMTj8WCs/wwmWdMRXM62izadFZV4W5qwFlV5a8S0H7oLNTFlrH2jw3fMfjuN1nzTQUAi2YMYd/i2Vwwpk9c2yWEiA0ZXouy9ts7aw4HGop/e2ffkFfDxnV43G48dbVoNhtqZia63B7eXUL92W0ubyFQnQJuBXSK9/GQoQFrsCXSFsfrd1Rw7dPb/I8nD85jxU8mSpKAECcYCTpR1nqvHI71cHR6Pfpj2zv70pArn16G6+hR0DQ8dpt3s7aqStz1dRQuuoHqlc/iPFKOpurRpZlxNzaiS09HU/U4DuzDUFgUk3TmcH1X2cRZj7QkCaQZdHxw5yx6Zhjj2CohRLxI0ImiQHvloKoY+/UHtwu1Z09/GrLWbEVzudDsNlB06AwGPE4nrkrvkFne/AWUP/In0DyoGRnoCwvRGhtxW5vRNF1CDJ21Vm91MuPPG6htdvqPvXXjVEYWZcexVUKIeJOgE0XB9soJtL+MLj0dT2UFiqKg6BTweLw7aRq863QKFi4iY9yE4zLSfD8vUQKO+1iSwPodLfNLSxaM47xTiuLYKiFEopCgE0XhrJUxFQ/CXnYIxaOhud0oehXVbG7z/Na10XTp6TGtEhCKv68r5eE1u/yPfzXzJG79/vA4tkgIkWgk6ERROGtlsmfMoumTj0EDNc0ELhcelwu1R0//8+NRJSAUa76pYOGKliSBqUN78dTV30MvSQJCiHYk6ERRuD0TY2ER1l078DTp0GVnY+jTF52qtllbk0gZaaUVjZz91/f8jzNNet7/dQm56ZIkIIQITIJOFIXaM/FVLVCzc0g/dRzOg/txN1tQ0zPoOWduwgQZn/pmJ2c+tJ5Gm8t/7J2bpjG8MCuOrRJCJAMJOlHmCxi+wOOrGtA6kLROOFABQ25uwiUIgDdJ4NqnP+bdXVX+Y0/8aDznnFwYx1YJIZKJBJ0oC7bFQOYZU/0FOx0H9mMcNAS11evCLc4ZbX9ds4tH15X6H984ayg3n504AVEIkRwk6ERZzaurvIs6XS50ZjOGor5obg81Lz5H2rARxwp4HsZeuhNl2Aj0Od7aY4lSnPOdr49w3TOf+B9PG5bP8h9PkCQBIUSXSNCJEmvpLmpeXYVl20dgMKDPzMLjcGLfXYqm06HZbTgO7sdjtaLpdHhcLhwH9qGOHpMQqdA7jzRyzt9akgSy0vRsukOSBIQQ3SNBJwp8Q2rOI+Xe/XM8HtyNDajZOaA34K6tRtHp0BwOFJMJxeXCo9Phttlw1VTHNRW6rtnBlMXrsTjc/mNrbp7G0AJJEhBCdF/KB53x40eTmZmJTqei1+tZs+bdzl/UTb7EAM3l8laMbmoADdyWJtTMLHC5IDMbxWDwvsBgQOdyo6an0+/ue6PevkBcbg/XPP0xm0qP+o8tu2oCZ48qiEt7hBCpKeWDDsDLL79BXl5e50+MEF8lAp3ZjMfhRJeVjdbcjMfpAEWB9HR0ehWP0+XdntrlAs3j374g1h7+707+vv47/+Nbzh7GDbOGxqUtQojUdkIEnVjzVSIwFPX1bj2gaWgeD4qioLlcmIsHg6LDXVfjrTptNqP2LsDYJ7Z7yrz9VTk//3+f+h/PHNGbZVdNQNUpMW2HEOLEkfJBR1EU5s2bg6IoXHXVNVx11TXHPWfFiqd45pmnAaitre72e/oqEejSM1ALCrF/V+qtDp3bA0NhER6bDRQw9h/YplJBrHb13HGkgXP/tsn/uEe6gY23l5BjNsTk/YUQJy6lsrJBi3cjounIkXIKC4uoqqri8ssv5o9//DOTJ08J+vzzzith27ZtQf89VNbSXTRsXIflk49Br8c0YCD6nFzAWxlac7vR5+b6KxVkz5gV9cSBWouDyYvXYXN6/MfW3jKdk3pnRvV9hRCpbezYcSHPl6d8T6ew0FtSPz8/n/PPv4BPP/2kw6ATKb4aaYeOze8oSsuQlS49HVdN9XHbG0SLy+3hR//8iC17Wnpx//zxBGaNlCQBIURspXTQsVgsaJqHzMwsLBYLGzeu57bbfh3TNoRTaToaHnp7B49v3O1/fPs5w7m+5KSYvLcQQrSX0kGnqqqSq69eAIDb7eLSSy9n5syzY9qGeO2B88aX5Vz/XEuSwNmjCvi/K8dLkoAQIq5SOugUFw9i48YP4tqGWO+B83VZPbMfe9//uFemifW3TSc7TZIEhBDxl9JBJ1HEYg+c6iY7p/9xHU53S17I+lunMzhfkgSEEIlDgk6Sc7o9LFi2lY/21fiPPXXN9ygZHv9ioUII0Z4EnST2xze/5Yn39vgf33neCH4+fUgcWySEEB2ToJOE/vNFGTc8/5n/8bknF/L4gnHoJElACJHgJOgkka8O13PB31uSBAqyTay9ZTpZkiQghEgSEnQizFeJIJKVBo422Zn4h7V4WtWO2HDbDAb1ygj+IiGESEASdCIo2NbUefMXdCnwOFwefrB0C58eqPMfW3HtRKYNy49ks4UQImYk6ESQbx8dX/UB338bNq4LO+g88Po3PPn+Xv/ju88fwc+mSZKAECK5SdCJIN8+Oq3p0tNxVlWG/DNe/fwwN77wuf/x7FOK+PsPT5MkASFESpCgE0HdqbO2/VA9F/5vS5JAn5w0/nvLdDJN8hUJIVKHXNEiqCt11qoa7XzvD2vbHHv39hkMzJMkASFE6pGgE0Hh1FlzuDzMe2ILnx9sSRL4fz+ZxJlDe8WyyUIIEVMSdCKsszprmqbxP69/w1Ob9/mP/Wb2SH46dXAMWieEEPElQSeGXv70ELe8+IX/8UWn9uFv88dKkoAQ4oQhQScGPj9Yx5x/bPY/7t/TzNs3TiNDkgSEECcYuepFUWWDjYkPrmtzbNMdJfTvmR6nFgkhRHxJ0IkCu8vNZUu2sP1wvf/Ycz+dxBknSZKAEOLEJkEngjRN497/fM2KLfv9x35/4SiunjIojq0SQojEIUEnAqylu1j1+hZ+U9HSk7n0tL48PO9UFEWSBIQQwkeCTjf5inz+pmkiAP1UO0+lbaffuKEScIQQokYfnLMAAAi9SURBVB0JOt3kK/L5nx4H0QN5ejdui7lLRT6FECLVSdDpJl+RzwLF7T8WbpFPIYQ4Ueji3YBkZ8jvjae5uc2xUIt8CiHEiUaCTjdlz5iFp9mC22JB0zTcFgueZgvZM2bFu2lCCJFwJOh0k6/Ip5qVhaumGjUrq8s7hQohRKqTOZ0I6KzIpxBCCC/p6QghhIgZCTpCCCFiRoKOEEKImJGgI4QQImYk6AghhIgZyV5rZ+/evYwdOy7s11VXHyUvL3W2Lkil85FzSVypdD6pdC4Q3vkcPLi/8ycdo1RWNmhdbZRocfbZ01mz5t14NyNiUul85FwSVyqdTyqdC0TvfGR4TQghRMxI0BFCCBEz6u233/X7eDciVZx66mnxbkJEpdL5yLkkrlQ6n1Q6F4jO+cicjhBCiJiR4TUhhBAxI0EnAtavX8PkyeOYOPFUHnvskXg3J2zjx49m+vTTKSmZwtlnTwegtraGyy67mEmTxnLZZRdTV1cb51YGd+ONv2DUqMFMmzbJfyxY+zVN4+67b2fixFOZPn0yX375ebyaHVCgc3nooQcZM2Y4JSVTKCmZwtq17/j/7dFHH2bixFOZPHkc69evjUeTgzp8+BCXXDKbKVMmMHXqRJYufRxI3u8m2Pkk4/djs9k455wZzJhxBlOnTuRPf/oDAPv37+Pcc0uYNGksCxdejcPhAMBut7Nw4dVMnHgq555bwoEDoadItydBp5vcbje//vWtPP/8Kt5//2Nefvnf7Ny5I97NCtvLL7/Bhg2b/SmSjz32V6ZNm87WrZ8zbdp0Hnvsr3FuYXA/+MECXnjh5TbHgrV/3br/smfPbrZu/ZyHH36UO+64OR5NDirQuQBcd931bNiwmQ0bNnPWWecAsHPnDl55ZRWbNn3ECy+8zK9/fQtut/u418aLXq/nvvv+wObN23jrrXUsX76MnTt3JO13E+x8IPm+H5PJxKpVr7Nx4wesX7+ZDRvWsm3bR9x//71cd931bN36OTk5uTz77AoAnn12BTk5uXz00Rdcd9313H//vV1+bwk63fTpp9sYNGgwxcWDMBqNXHLJXN5++414N6vb3n77DebPvwKA+fOv4K23Xo9zi4KbPHkKubk92hwL1v633nqTefN+iKIoTJgwkfr6eioqjsS8zcEEOpdg3n77DS65ZC4mk4mBA4sZNGgwn366LcotDF1BQSFjxowFIDMzi2HDhlNeXpa0302w8wkmkb8fRVHIzMwEwOl04nS6UBSF999/lwsvnAPA/Pk/9H833u/shwBceOEcNm3aiKZ1LR1Agk43HTlSTt++/fyPi4r6dPiLmIgURWHevDmcddY0Vqx4CoCqqioKCgoB7x/b0aNH49nEsAVr/5EjZfTp0/J99enTNym+r+XLlzJ9+mRuvPEX/uGo8vIy+vTp639Onz59OXKkPF5N7NCBA/vZvv1Lxo+fkBLfTevzgeT8ftxuNyUlUxg1agjTp5dQXDyY7Owc9HpvoZrW7W19ndPr9WRlZVNTU9Ol95Wg002Bor2iKHFoSde9/vp/WbduE88/v4rly5exZcvmeDcpapLx+7r66p/y0UdfsGHDZgoKCrn33nuAwOcCiXcuTU1NXHvtj7j//sVkZWUHfV6yfDftzydZvx9VVdmwYTNffPEtn332CaWlO497ju/zD/zddO19Jeh0U1FRHw4fPuR/XF5eRmFhURxbFD5fe/Pz8zn//Av49NNPyM/P9w9tVFQcoVev5KopFaz9RUV9KStr+b7Kyg4n/PfVu3dvVFVFp9Nx5ZU/5rPPPgG8d6JlZYf9z/OeS2G8mhmQ0+nk2muvZO7ceVxwwUVAcn83gc4nmb8fgJycXM4440y2bfuYhoZ6XC4X4G2vr0fa+jrncrlobGygR4+eXXo/CTrddNpp49mzZw/79+/D4XDwyiurOOec8+PdrJBZLBaamhr9/79x43pGjhzJOeecz8qVzwGwcuVznHvu7Hg2M2zB2n/uuefx4ovPo2ka27Z9RHZ2tv8PK1G1ntd4883XGDFiJOA9x1deWYXdbmf//n3s2bOHceMmxKuZx9E0jZtuup5hw4azaNEv/ceT9bsJdj7J+P0cPXqU+vo6AKxWK++9t5Fhw4YxZco0XnttNQArVz7v/26839nzALz22mrOPHN6l3uhsjg0AtaufYff/OZO3G43V1zxI26++fZ4Nylk+/bt5eqrFwDgdru49NLLufnm26mpqWbhwqs5dOgg/fr158kn/9XlO5tou+66a9i8+X1qaqrJz+/NHXfczXnnzQ7Yfk3TuPPOW1m/fi3p6ek8+ujjXaoqHi2BzmXz5k18/fV2QGHAgAH85S+P+i/Gf/3rn3nuuWfQ6/U88MBiZs36fnxPoJUPP9zCRRedw8iRJ6PTee9v77nnd4wbNyEpv5tg5/Pyy/9Ouu/n66+/4le/+jlutxtN83DRRZdw2213sm/fXq677hpqa2s55ZRTefzxZZhMJmw2G9df/zO2b/+CHj168MQTT1FcPKhL7y1BRwghRMzI8JoQQoiYkaAjhBAiZiToCCGEiBkJOkIIIWJGgo4QQoiY0ce7AUKIFjU11cyd6110WFlZgaqq5OV5F0/+9rf3MXPmWf7nPvHEP9i9+zseeihxi7EK0Z70dIRIID175vmrFf/4x9f6qxdfffVPWL16VZvnrl69iksvvTxOLRWiayToCJEELrzwYtaseRu73Q54C04eOXKESZMmx7llQoRHgo4QSaBnzzxOO228fyOw1atXcfHFlyZkQUwhOiJBR4gkcckll7F69b8BeOWVVVx66WVxbpEQ4ZOgI0SSOO+8C9i06V2+/PJzbDarf0MxIZKJBB0hkkRmZiZnnDGVG2+8nksukV6OSE4SdIRIIpdcchlff71dgo5IWlJlWgghRMxIT0cIIUTMSNARQggRMxJ0hBBCxIwEHSGEEDEjQUcIIUTMSNARQggRMxJ0hBBCxIwEHSGEEDHz/wE+QZsZzGOLyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4XOV1+PHvmU0a7bIlS7bkDWNssxmDMEtIAgQIEAppmibQLLRJfpSENF3Spml/aZO0fdrk15A0lDSUJhRoU0LDkkBCiFkDSUiwDbbxvuFFtizJtpbRNuv7++PeK8vS7HMH21fn8zx6pLlzde8dj58zr8577nnFGINSSqnpw3eiL0AppdRbSwO/UkpNMxr4lVJqmtHAr5RS04wGfqWUmmY08Cul1DSjgV8ppaYZDfxKKTXNaOBXSqlpJnCiLyCdpqYms2DBghN9GUopdcpYu3btYWNMcz775gz8IjIXeBBoBVLAvcaYb4rIDOBhYAGwB/iAMaYvze/fCnzBfvgPxpgHcp1zwYIFrFmzJp/rV0opBYjI3nz3zSfVkwA+a4xZBlwM3CEiZwKfB54zxiwGnrMfT76QGcAXgYuAlcAXRaQx34tTSinlvpyB3xjTZYx5zf45AmwB2oCbAGf0/gDw3jS//m7gGWPMUfuvgWeAa924cKWUUsUpaHJXRBYAK4DfAC3GmC6wPhyAWWl+pQ3YP+Fxp71NKaXUCZJ34BeRGuBR4E+MMYP5/lqabWn7QIvIbSKyRkTW9Pb25ntZSimlCpRX4BeRIFbQ/54x5jF7c7eIzLafnw30pPnVTmDuhMftwMF05zDG3GuM6TDGdDQ35zUxrZRSqgg5A7+ICPBdYIsx5usTnnoCuNX++VbgR2l+/WfANSLSaE/qXmNvU0opdYLkM+J/G/AR4EoRWWd/XQ98BbhaRHYAV9uPEZEOEfkOgDHmKPD3wGr76+/sbUoppU4QORmXXuzo6DDF1PHf9dwOls9t4J1naKpIKTW9iMhaY0xHPvt6qmXDPT/fxcvbdWJYKaWy8VTgD/p9JFIn318wSil1MvFc4I8lUyf6MpRS6qTmqcAf8gvxhAZ+pZTKxlOBPxjwEdcRv1JKZeWtwO/3EU9qjl8ppbLxXODXHL9SSmXnscAvmupRSqkcPBb4NcevlFK5eCzwi+b4lVIqB48Ffh3xK6VULp4K/CEN/EoplZOnAn/Q7yOe0FSPUkpl463ArzdwKaVUTt4K/H7ROn6llMrBW4HfpyN+pZTKxVuBP6DlnEoplYu3Ar9W9SilVE6eCvxazqmUUrl5KvBrd06llMrNc4E/mTIkdflFpZTKKJBrBxG5D7gB6DHGnG1vexhYYu/SAPQbY85L87t7gAiQBBL5rgBfrGBAAIgnU/h9/nKeSimlTlk5Az9wP3A38KCzwRjzQednEbkTGMjy+1cYYw4Xe4GFCPmtP2DiyRSVQQ38SimVTs7Ab4x5SUQWpHtORAT4AHClu5dVnIDPGfFrqkcppTIpNcf/dqDbGLMjw/MGWCUia0XktmwHEpHbRGSNiKzp7e0t6mKCgWMjfqWUUumVGvhvAR7K8vzbjDHnA9cBd4jIOzLtaIy51xjTYYzpaG5uLupignaqJ5bQwK+UUpkUHfhFJAC8D3g40z7GmIP29x7gcWBlsefLh5PjT2hVj1JKZVTKiP8qYKsxpjPdkyJSLSK1zs/ANcDGEs6XU9CvqR6llMolZ+AXkYeAV4AlItIpIh+3n7qZSWkeEZkjIk/ZD1uAX4jIeuBV4CfGmKfdu/Spgn5rcreYVM+6/f38z2/2uX1JSil10smnqueWDNt/P822g8D19s+7geUlXl9BSpnc/fef72LV5m7eu2IOVaF8qlyVUurU5Kk7d4/V8Ree49/SNUgyZVi3r9/ty1JKqZOKpwL/sTr+wkb8Q9EEe46MALB6T5/r16WUUicTTwV+J9VT6Cpc2w4NAiACa/Yedf26lFLqZOKpwD+e6ilwcndzVwSAK5fM4rW9fSS0Kkgp5WGeCvzBIuv4t3QNUlcZ4Mbz5jAcS7LF/iBQSikv8ljgLy7Hv6VrkGWz61i5cAYAq/doukcp5V0eC/yFt2xIpgxbuyIsm13H7PowbQ1hzfMrpTzNU4E/FCi8nHPvkWFG40nOnF0HwIULGlm9pw9jtO2DUsqbPBX4i2nZ4OTzlzmBf+EMeiNR9h0dcf8ClVLqJOCpwB8oIse/pWsQv09Y3FIDwIULnDy/1vMrpbzJU4HfKecspI5/S9cgi5qrx1fsOr25hvpwkDU6wauU8ihPBf7xVE8i//y8U9Hj8PmEjvmNvKqBXynlUZ4K/H6f4BNIpPIb8fePxDg4MHZc4Ac4u62e3b3DRBPJclymUkqdUJ4K/GCN+vNN9Wzuslo1TA78LXWVABwZirl7cUopdRLwXOAP+X15p3qOVfTUHre9qSYEwOGhqLsXp5RSJwHPBf5gwJd3Vc+2Q4M01YSYVVt53Pam2gpAA79Sypu8F/j9knfgPzwUo7W+csr25hor8PdGNPArpbzHc4E/4Ms/xz80lqC2Ijhle/P4iF9z/Eop7/Fc4A8FfHm3bBgci1NTOXWZxcqgn9qKgI74lVKe5LnAH/RL3v34I2MJatMEfrDy/L2a41dKeZAHA78v7zr+oWiCusqpqR6wKnsO64hfKeVBOQO/iNwnIj0isnHCti+JyAERWWd/XZ/hd68VkW0islNEPu/mhWdi1fHnTvUYYxiKJqipSD/ib9YRv1LKo/IZ8d8PXJtm+zeMMefZX09NflJE/MC3gOuAM4FbROTMUi42H1Ydf+4R/2g8STJlMqd6aip0xK+U8qScgd8Y8xJQTOOalcBOY8xuY0wM+D5wUxHHKUgwkF85Z2QsAZB2cheswD84ltC2DUopzyklx/9pEdlgp4Ia0zzfBuyf8LjT3paWiNwmImtEZE1vb2/RFxX053cDV2QsDkBthhy/lnQqpbyq2MD/bWARcB7QBdyZZh9Jsy1j8t0Yc68xpsMY09Hc3FzkZTl1/Llz/M6IP1uqB9B0j1LKc4oK/MaYbmNM0hiTAv4DK60zWScwd8LjduBgMecrRKjAVE9tlsld0LYNSinvKSrwi8jsCQ9/G9iYZrfVwGIRWSgiIeBm4IlizleI/FM9zog/czknaNsGpZT3pB/uTiAiDwGXA00i0gl8EbhcRM7DSt3sAf7Q3ncO8B1jzPXGmISIfBr4GeAH7jPGbCrLq5gg6PeRyCPVMxS1cvzZJndBR/xKKe/JGfiNMbek2fzdDPseBK6f8PgpYEqpZznl248/V46/MuintjKgk7tKKc/x3J27oTy7c46Xc4Yyf/Y111Roqkcp5TmeC/zBPG/gioxZd+36fOmKjyxNNXr3rlLKe7wX+PPszhkZi2dM8ziaa/XuXaWU93gv8PuEWDKFMdmDf7Y+PY6mmpCO+JVSnuO9wO+3XlIilT3wZ2vJ7GiurSAylmAsrm0blFLe4b3AH7BeUq4JXivVk76G36ElnUopL/Je4Pc7gT/HiD+ayFjD7zgW+LWkUynlHZ4L/CG/VaWTe8SfoC6PVA9ovx6llLd4LvAfG/FnD/xDY3lM7tqBXyd4lVJe4t3An8ic6oknU4zGk3nk+K1+PTriV0p5ifcCvz25m61tw1COdg2OioCfusqATu4qpTzFe4HflzvHPxS12zXkSPWAle7RVI9Syku8F/jzyPEP5lh9a6LmmgoOR7SqRynlHd4L/HnU8efqzDmRjviVUl7jvcBvl3PGskzu5pvjB2fEr4FfKeUdngv8ofGWDVlG/NECUj21FUSi2rZBKeUdngv8+eT4x3vx5zG521hllXT2jWieXynlDZ4N/NlSPYXk+OvD1l8FA6NxF65OKaVOPM8F/lAgdzlnZCxByO+jMujPebzxwD+igV8p5Q2eC/wBX+5Uz1A0nrNBm0NH/Eopr8kZ+EXkPhHpEZGNE7b9s4hsFZENIvK4iDRk+N09IvKGiKwTkTVuXngm+ZZz5pPmAWio0sCvlPKWfEb89wPXTtr2DHC2MeZcYDvwV1l+/wpjzHnGmI7iLrEw4+WcWdoyR/Jo0Oao0xG/UspjcgZ+Y8xLwNFJ21YZYxL2w18D7WW4tqKExpu0Ze/Vk++Iv7YigIgGfqWUd7iR4/8Y8NMMzxlglYisFZHbXDhXTsE86vgH81h9y+HzCXWVQQ38SinPyG/Ym4GI/F8gAXwvwy5vM8YcFJFZwDMistX+CyLdsW4DbgOYN29e0deUzwpckbEEtXmmesCa4NXAr5TyiqJH/CJyK3AD8CFjTNooa4w5aH/vAR4HVmY6njHmXmNMhzGmo7m5udjLmtCyIXt3znxTPWBN8GrgV0p5RVGBX0SuBf4SuNEYM5Jhn2oRqXV+Bq4BNqbb100iQtAvGat6jDEM5bHe7kT14SD9WsevlPKIfMo5HwJeAZaISKeIfBy4G6jFSt+sE5F77H3niMhT9q+2AL8QkfXAq8BPjDFPl+VVTBLw+TIG/tF4kmTK5J3jB6uyZ1BH/Eopj8g57DXG3JJm83cz7HsQuN7+eTewvKSrK5I14k+f4y+kXYNDc/xKKS/x3J27AKGAL+PSixF7EZZ86/gBGuzAn2EqQymlTimeDPxBvy9jHb8z4q8rINVTHw6SSBmGY9qaWSl16vNs4E+ksqd6Cp3cBb2JSynlDR4N/JIl1VNcjh+0Q6dSyhs8Gvgzp3qGClh9y6EjfqWUl3gy8IcCmcs5C1l9y1GvHTqVUh7iycAf8OUu5ywo8I+P+HX5RaXUqc+TgT/oz1bOmaA65Mfvk7yPp6kepZSXeDLwZ0/15N+Z01FTEcDvEw38SilP8GTgD/ozB/5CG7SB1f9H795VSnmFRwO/kMiQ4x+KJqguIL/vsAJ/IveOSil1kvNo4M+c4x+JJamu8Bd8zLpwkP4RndxVSp36PBn4Q1lSPaOxJOFg4YG/Xjt0KqU8wpOB37qBK32qZyyRpKLIwK85fqWUF3gz8AcyL8QyVuSIv0EDv1LKIzwZ+AO+zDn+sUSKymDhL9sZ8acyNH9TSqlThScDf7Y6/lJy/CkDQzF3Knv2HRkZXxtAKaXeSp4M/JlW4DLGMBpPUllk4Ad3OnQORRO8519f5s9/sL7kYymlVKE8Gvh9JFNmSlomanfsLCrwu9io7fHXOomMJfjZpm62dA2WfDyllCqEZwM/QDx1fLpnLG6toFVsqgcouaTTGMODr+xlSUsttRUB7n5+Z0nHU0qpQnky8IecwD8p3TMWL2HEbwf+/hID/693H2VHzxCfePtCPnrpfJ7a2MWO7khJx1RKqULkFfhF5D4R6RGRjRO2zRCRZ0Rkh/29McPv3mrvs0NEbnXrwrMJ+q3Om5MXYxl1Rvyh4qp6oPRUz4Ov7KGhKshvLZ/Dxy87jXDQz90v6KhfKfXWyTcC3g9cO2nb54HnjDGLgefsx8cRkRnAF4GLgJXAFzN9QLgpGHBG/OlTPZWBEiZ3Swj8XQOjrNrczQc75lIZ9DOjOsRHLp7Pk+sPsrt3qOjjKqVUIfIK/MaYl4CjkzbfBDxg//wA8N40v/pu4BljzFFjTB/wDFM/QFwX9Fkva3ItvzPirwwVHvirQn6C/tJaMz/0m32kjOHDF88f3/aJt59GKODjvl++WfRxlVKqEKXk+FuMMV0A9vdZafZpA/ZPeNxpbyurYMBO9UzJ8Rc/4i+1NbMxhodW7+eKJbOYO6NqfHtzbQUXLpjB6/v6izquUkoVqtyTu+mWuUp766uI3CYia0RkTW9vb0knHa/qyZDqCRcx4gerQ2exdfy9kSi9kSiXL2me8tzS1lp29AyRyHDTmVJKuamUwN8tIrMB7O89afbpBOZOeNwOHEx3MGPMvcaYDmNMR3Pz1OBYiEyBfzTmVPUU97JLGfG/eXgYgAUzq6c8t6S1jlgixZ4jw0UdWymlClFK4H8CcKp0bgV+lGafnwHXiEijPal7jb2trDKXcxZfxw+lNWrbe2QESB/4l7bWArD1kJZ1KqXKL99yzoeAV4AlItIpIh8HvgJcLSI7gKvtx4hIh4h8B8AYcxT4e2C1/fV39rayyjjiLzHwlzTiPzJM0C/Maaic8tzps2rw+4RtGviVUm+BvNYgNMbckuGpd6XZdw3wiQmP7wPuK+rqipSpjt8Z8RfTjx+swF/sKlx7jwwzt7GKgH/qZ21l0M+CmVU64ldKvSU8eeeuU8c/uZyz1FRPfThIJJooqjXzm4dHWNA0Nc3jWDq7jq2HtG+PUqr8vBn4fZlbNvjk2F8EhaoLBzEGImOFtWY2xrD3yHDa/L5jaUst+4+OMhTVBd2VUuXlzcA/Xsc/NccfDvoRKS7wN1SFgMLv3u2NRBmJJVnQVJVxnyX2BO927dujlCozbwb+LJO7xTRoc9RVWlMigwUuoJKtlNOxtLUOgK1dGviVUuXlycDvlHPG0kzulhL4i+3X45RyLsyS429vDFMd8rNN8/xKqTLzZOB3RvyJ1NQ6/mLv2gUrxw+F9+R3Sjln108t5XT4fMIZrbVa2aOUKjuPBv70Of6xeHELrTuKHfHvOTzM3BnpSzknWtpay7buCMbogu5KqfLxZuAPpE/1FLvQumN8xF9gjn/PkZGs+X3H0tY6+kfidA9Gi7o+pZTKhycDf8aWDYnScvzVIT9+X2GtmfMp5XQsGW/doHl+pVT5eDLwB3wZyjljpQV+EaGuMsDgaP619j12KefCLKWcDqdnj7ZuUEqVkycDv98niKRvy1xK4IfC+/XssUs55+cx4m+oCtFSV6GBXylVVp4M/CJC0O9L07IhRbiEyV2w8vyF5PidVsvZSjknWjyrll26DKNSqow8GfjByvPHE8fn+J07d0tRV1ngiP/ISM5SzonmzghzoH+02MtTSqmcPBv4g34hkSpPqqeQOv58Szkd7Y1VHB6KMRpLFnuJSimVlYcDv++4HH8qZYgmUiUH/rpwgIECJnffPDzMwjzy+472xjAAB/pHCr42pZTKh2cDfzjkZzh6bNQcTTjLLpYa+PPP8VulnCN5Tew6nMC/v0/TPUqp8vBs4K+rDBKZEKCPrb5V4uRuZZBYIjXe2z+bI8MxRuNJ5s0I53389kar7LNTA79Sqky8G/jDgeMmYZ3A70aOH/Lr19Nj34HbUpffxC5Ac00FIb+Pzj5N9SilysOzgb8+HGRwwoIp46tvldCkDQpr29A7ZAX+5tqKvI/v8wltjWEd8SulysazgX9y2aVTJePWiD+fks7eiBX4Z9XmP+IHK8+vgV8pVS6eDfyTyy6jCXcC//hiLHlU9jiBv6k2VNA52hvDHNBUj1KqTIoO/CKyRETWTfgaFJE/mbTP5SIyMGGfvy39kvNTFw4SnTAJOxqzqnpKvoGrgBF/T2SMmooAVaFAQefQWn6lVDkVFpEmMMZsA84DEBE/cAB4PM2uLxtjbij2PMWamIuvDPrHPwBK6ccPEyZ388nxR6IF5fcdE2v5T59VW/DvK6VUNm6let4F7DLG7HXpeCU7lpKxAvSxcs7SWzYADIyUP/BrLb9SqhzcCvw3Aw9leO4SEVkvIj8VkbNcOl9OxyZhrVy8W+WcoYCPcNCfd1VPcYFfa/mVUuVTcuAXkRBwI/CDNE+/Bsw3xiwH/hX4YZbj3CYia0RkTW9vb6mXNaXsMupS4LeOHcivqmcwSnNN4YG/uaaCUEBr+ZVS5eHGiP864DVjTPfkJ4wxg8aYIfvnp4CgiDSlO4gx5l5jTIcxpqO5ubnki5p8o9WoS3X8zrFzVfWMxpJEoomiRvw+n9DeEKbzqI74lVLucyPw30KGNI+ItIqI2D+vtM93xIVz5uTk4p3APxa3e/UESn/J+bRmPjzk1PAXHvgB+yYuHfErpdxXUhQUkSrgauCxCdtuF5Hb7YfvBzaKyHrgLuBmY4yZeiT31YWtyd2BCSP+oF/ybo+cTX0ejdp6IoXftTtRe2OVazn+WCLFj9YdYDiaf1dRpZR3FV3OCWCMGQFmTtp2z4Sf7wbuLuUcxaoI+KkM+sbbNozFk1QGSk/zgDV/sK07+/KIvZExoJTAH+bIcIyRWKLg+wAmu/9Xb/KPT21lSUst9370goK6hSqlvMezd+6CvT7uiJPqSVLpQn7fOW6uJm29JY/47Vr+Ekf9kbE4//biLs6cXcehwTFuvPuXvLyj9MlzpdSpy9OBv67yWEpmNJYs+eatY8cNEIkmSKUyZ616I1F8AjOri0/1QOklnd95+U36R+J89XfO5clPX0ZrXSW33vcqa/YcLem4SqlTl6cDf3342CSstdC6e6keYyCSJWfeOxRlZk0Ffp8UdY659oi/lAneo8MxvvPybq47u5Vz2uuZN7OKRz91KeGgn0dfO1D0cZVSpzZPB/6Jq2W5sdD6xONC9p78PUXW8Duaxmv5ix/xf/vFnYzGk/zZ1WeMb6upCHD50lk8s/kQySx/sSilvMvbgb8yMF5vPxZPUuFW4K/M3ait2Lt2HeO1/EUG/kMDYzzwyl5+e0U7i1uO7/dz7VmtHB6K8dq+vqKvTyl16vJ04D8+1ePeiD+fRm29kWjRNfyOUmr5n9l8iFgixaeuWDTluSuWziLk9/H0xkMlXZ9S6tTk6cBfF7bW3U2lDGPxlHuTu+HjG8BNlkoZDpc44ge7L39/cSP+tXv7aKmr4LSmqaWbNRUB3r64iac3HuItuq1CKXUS8XTgrw8HSRkYiiVczfEfaweRfnK3fzROPGlKDvxtDWEOD8XyWth9srX7+jh/XiP2jdNTvPvsVg70j7Lp4GBJ16iUOvV4OvBPbNswGk+60qANci/GUuySi5O1jfflL2zU3zM4xv6jo1wwvzHjPlcta8EnaLpHqWnI24F/QoAeczHw14QC+CRzjr/Um7ccbQ1WLX+hN3E5k7bnZwn8M6pDXLRwJk9v0sCv1HTj8cB/bH3csXjSlc6cYFXc1GZp1NY7VFq7BkexI/61e/sIBXycNacu637Xnt3Kzp4hdvZkbz+hlPIWTwd+JxffNxIjnjSu9eoB60Ml0+Ruz6A7I/6WWusGsMJH/P2c21ZPRY7Xe/WZLQC8uE1bOCg1nXg68Ds5/u5BawQeDrn3cieWik7WG4lSFfJTU1Fac7WA30drXWVBI/5oIskbnQNZ8/uOOQ1h2hrCWs+v1DTj6cBfX2UFfqdFsls5fnD6AKWv6in15q2J2hrCBY34Nx4YJJZMsWJe7sAPsGJeA6/v6y/28pRSpyBPB/6aUACRYyN+NwN/tg6dvZHS2jVM1FZgLf9re52J3Ya89j9/XiNdA2N0DehqX0pNF54O/D6fUFcZHM+5uz3iz5Tq6Ym4O+I/NDhGIpnKa/+1e/uYN6Mq71LSFfOsD4h1OupXatrwdOAHaxL2kJPjd3PEX5V5FS432jU42hrDJFNm/DVkY4xh7b6+vPL7jjPn1BHy+zTPr9Q04vnAXx8OHpvcdXXEH2AsniKaOP6u2mgiycBo3NURP8DB/tyBv7NvlN5INGv9/mQVAT9nt9Vpnl+pacTzgb+uMkjEnoR1q1cPZG7b4NbNW45jtfy5m7WN37g1L7/8vmPFvEbeODBALJFfOkkpdWrzfOB3AjS4nOPP0Laha8AambfWh105jzPiz6eyZ3PXICG/jzMmtWHOZcW8BqKJFFsPad8epaYDzwd+p5Yf3K/qAevmsIkO2hU4TsAuVWXQT1NNKK/Knm2HIiyaVUPQX9jber5d+ulUBCmlvK3kwC8ie0TkDRFZJyJr0jwvInKXiOwUkQ0icn6p5yyE07YBcK1lA2ReDN1ZOGVOQ2kN2iZqy3NBlq1dEZa2FjbaB5hdX0lLXQWv73cnz/+jdQf48pObGM6yNKVS6sQp7dbSY64wxhzO8Nx1wGL76yLg2/b3t8RxqZ6Ae3/gOIuh7z96fO79YP8oM6pDVIXc+qe18vxbD2XvpzMwEufQ4BhLigj8IsKKuY0lT/AaY/jGM9u56/mdAPxix2Hu+cgFLGquKem4Sil3vRWpnpuAB43l10CDiMx+C84LHMvFg7sjfisFUzFlJH6gf9TV0T7AnPowB/tHsy6a4uTnixnxg3XD176jIxweihb1+9FEkj/73/Xc9fxOPtDRzgMfW8mR4Rg33f1LVmkHUKVOKm4EfgOsEpG1InJbmufbgP0THnfa294Sx4/43Qv8AHNnhNnfN3XEP8eliV1HW2OYsXiKI8OxjPts67b+Iljamr0jZyZOi4diR/1femITj79+gD+/5gy++jvn8s4zmnnyjy5jUXM1n/rea1P+MlJKnThuBP63GWPOx0rp3CEi75j0fLoloKYMXUXkNhFZIyJrenvd6xbpTO6GAj58vvSrURWrvbHquBG/MYYDfaPjJZhuyaeyZ0tXhPpwkJa64spIz2mrJ+CTom7kOjwU5dG1B/jwxfP49JWLx1f9amsIc89HLkAE7n1pd1HXpZRyX8mB3xhz0P7eAzwOrJy0Sycwd8LjduBgmuPca4zpMMZ0NDc3l3pZ45xUj5s3bznmNlopmGTK+hwbHE0wHEu6VtHjyKcv/7ZDgyxprc241GIulUE/Z82pK6qy5+HV+4klU/z+pQunPDe7Psz7L2jn4TX76cnj7mOlVPmVFPhFpFpEap2fgWuAjZN2ewL4qF3dczEwYIzpKuW8hai3q3rcvHnL0d5YRWJCO4UDLpdyjp8nx0pcxhi2dw8Vnd93rJjXyPrOfuJ59gUCSCRT/Pev93LZ6U2cPiv9JO7t71xEIpniP17WUb9SJ4NSo2EL8AsRWQ+8CvzEGPO0iNwuIrfb+zwF7AZ2Av8BfKrEcxakrCP+GVaAd/LXTuCf43LgrwsHqKkIZBzxd/aNMhRNFJ3fd1wwv5GxeIqtXfmvyPXslm66Bsb46CXzM+4zf2Y1Ny6fw/d+s4+jWeYplFJvjZJqDo0xu4HlabbfM+FnA9xRynlK4eT43bx5y+GUdDp5/vGbt1zO8YuI1Zc/Q+B3Sj2LKeWcyOnxs3bvUc5pr8/rdx741V7aGsK8a1lL1v3uuOJ0frjuIP/5yzf57DVLSrpOpVRpPH/nbmXQT0XAV5bAP6ehEpHjR/yhgI+Z1SHXz9XWmHlBlm12KWepgX+Z4T3NAAAQcklEQVROfSWtdZW8lmdlz/buCK/sPsKHL56PP8fE+eKWWq49q5X7f7VHb+xS6gTzfOAHK91Tjhx/RcBPS23l+Ij/QP8obQ3hoidYs1nYVM2u3qG0+fethyK0N4ZLXupRRDh/fgNr85zgffCVPYQCPj544dyc+wJ84u0LiYwl+MmGt2yKRymVxrQI/PXhYFly/GDl+TvtWv4DfaOuT+w6nEZqW7qmNlLbdqi4Vg3pnD+vkQP9o+OtrDOJJ1P8eEMX153dyow8/8K5YH4ji5qr+f7qfW5cqlKqSNMi8H/mXYv5g7dNLTV0w8Ra/oNluGvXkekGq2giye7DwyWneRxOnj9XWecvdh6mfyTOb507J+9jiwgfvHAur+3rZ0d3/hPI6QyMxukaGGUsnsy9s1LqONMi8N+4fA7vOMO9ewMmmtsYpmtglJFYgp5IlDa79NJtc+xGapNvsNrZM0QyZUqu6HGcNaeOUCD3ilw/Xt9FbWWAt5/RVNDx33d+OwGf8PDq/bl3zuCZzd1c9I/Pcsk/Pc/Sv3mapX/zU770xKbx+ymUUtm510lsmmpvrCJlGM+Ll2vE7zRSmxyQtx1yWjW4M+KvCPg5t60+a54/mkiyavMhrjmzlYoC22A01VRw1bIWHnv9AJ+7dimhAhvn/dev9/LFH23knLZ6PnjhPPpGYmzvjnD/r/bQOxTlGx84r+BjKjXdaOAvUbtdy//qm0cB92/emuj8+Q08vekQvRMWc//lziPUVgRY0FTt4nkauf+Xe4gmkmkD+0vbDxMZS3DD8uJ67X1w5Vye3nSI57Z0c905+R3DGMPXVm3jWy/s4l1LZ/Gvv7fiuA6oy2bX8ZWfbmVoLME9H77A1YZ8SnmNDo1KNNeu5f/Nbjvwu1zDP9H543l+azQ+Fk+yatMh3n12a8GLr+Q6TyyZYuOB9Cty/XjDQRqqglx2emFpHsc7Fjczu76S7xeQ7nlyQxffemEXt6ycy79/5IIpba9vf+ci/ul95/DSjl4++b21pDTto1RGGvhLNLu+Er9PWGcvYtJaX55UD8DZ443UrHO9uK2HSDTBjcvzn2DNx/nzrTV71+49OuW5sXiSZzd3c+1ZxX/Y+H3C717Qzks7etl7ZDjn/j2RMf72Rxs5b24Df3/T2QQynPeWlfP48o1n8eK2Xv5dm8IplZEG/hIF/D5a6yqJJVPMqq0oOOddCKeRmjPif2L9QZpqQly6aKar55lVW8lZc+r4/ur9UyZMX9jaw3AsyQ0FVPOk86GL5xPy+/jmszuy7meM4a8fe4PRWJI7P7A8Y9B3fOTi+Vx/TitfW7WNNXumfnAppTTwu8Lp2eN2j550VsxrZEPnAAMjcZ7b0sN7zpmdMxgW45OXL2J37zBPbzx+EZUfrjvAzOoQF582o6Tjt9RVcuulC3h83QG2ZyntfPS1Azy7pYe/ePeSvFbyEhG+8jvn0tYQ5o8eep2+InoD9Q3H+OefbeXKO1/kyjtf5N3feIn3f/tXPLK2k0QBDeyUOllp4HeB07OnnPl9x4p5DYzGk9z1/A6iiRQ3nudumsdx3dmzOa25mrtf2Dm+8tfzW7v52aZubl4515UPm9vfuYjqUICvr9qe9vk3Dw/z5Sc3sXLBDD5WwH0YdZVB7v69FRweivKZ77+ed7fRgdE4X316K5d99Xn+7cVdtDdWsay1jgVNVQyOxfnzH6znyjt/zsOr92npqDqlaVWPC5wJ3nJW9DicCd4HfrWHtobw+GO3+X3CJ9+5iL94ZAPPb+1h+dwGPvfIBpa21vKZdy125RwzqkN8/LKFfPO5HWzo7Ofc9obx5/qGY3zs/tUE/T7u/MDyghfRObe9gX9479n85aNv8LlHNnDn72Y/xrr9/Xz6f17jQP8oN5w7h89ceTqLW46VyBpjeHZLD3c9t4O/fPQNfrCmkzs/sJz5M92rplLqraKB3wXt9kh/Thkndieeq7m2gt5IlN9aPqcsfYEc713Rxr88u4O7X9jJzOoKBkcT/PcnLnJ1HuMTb1/IA6/s4WurtvPgx6w1fGKJFH/432s50DfK//yfi5g7o7ib4j544Tx6I1G+tmo7M6pDfOE9y6b8exlj+O4v3uQrP91KS10lj37y0rQfpiLC1We2cNWyWTz++gG++MQmrvvmy3zhPWdyy8q5eb8Pg2NxNuwfYNPBATZ3DTIWTzKzpoKmmgoWz6rhHWc0H7dcqFLloIHfBac1W6O++S7W0mdi3cjVwKrN3a5X80wW9Pu4/fJF/M0PrbV1vvCeZa7dIeyorQzyqcsX8Y9PbeUd/+8F3nZ6E0eGorz65lG+efN5dCwobS7hjitO5/BQjO/+4k2SKcOHLprH6bNqSBlYtekQ9/x8F+s7B7j6zBa+9v7l1FdlD7oiwvvOb+fi02byF4+s568ff4OHXt3HZ685g3ee0Zz2A2AsnuT5rT388PUDvLCth3jSShO1NYSprvCzek8ffSMxjIGAT1i5cAbXnNnC9efMZlZd+QcTavoRJ397Muno6DBr1qw50ZeRN2MMr+w6wsWnzXR9Xd90Xtrey3NbuvnSjWeVdcQPVtC66us/Z2FTNQ/8wcqyvL5EMsX3V+/n59t7+fWuI0SiCf70qjP446vcSSmlUoa/euwNHl5j3TdwWlM1BmsOYf7MKu64/HR+t6O94H/LVMrw2OsH+Jdnt9PZN8qFCxq57HTrHoWm2hBbD0V4ZdcR1uzpYzSeZFZtBTcun8PlS2Zx1pw6Gic0t0skU6zvHODZLd08u7mbHT1DiMBFC2dwzZmtnNtez7LZdVRP6MCaTBk6+0bY1TvE7t5hdh8eZs/hYQ70jxKNp0ikUqQMNNdU0NYYpq0hzOKWGpbNrmNpay21lfqXhZeIyFpjTEde+2rgV7kMRROEg/6cPffdkEim2N83yoKZVa5/qB0aGOOZzYf42aZuookkt166gOvOnl3y64olUjy8Zj/3vrSL/UePXzNhSUstlyyayVXLWrhk0cy8z7WzJ8KT67t4csNBdvda9zqIQEttJUljiCdTjESTxCZMXDdUBVkws5q5M6oIB30E/D4E6B6McrB/lP19I0TGjq2F0FJXwfyZ1SyYWUVDVYjKoJ/KoI94wjAcSxAZSzAwGqN/JE7fSJyRWIJE0hBLpvAJVFcEqK0IUBcO0lxbwazaSmbVVjCzJkRzTQUNVSHCIT/hoJ9QwEcyZUjZ1x5LpIjZ31MGUsZgDFQEfPaXn+oKP7WVQW3BkScN/EqdILFEiu7BMXoiUebPrKKppqKk4xlj6B6MsungABsPDLK/b4SgXwj6fYRDfhY11bBoVjWLmmtoqMreHtsYa33oLV2DbOmK8ObhYfYeGWbPkREGR+NEE8c+RCqDPmrsoN5YFaKxKkhVKEAo4CPo95FKGYZiCYbGEvSPxukdHKN3KDqexnJTRcBHbWWA6ooA1aEA4ZCfkN9HKOAjMOGD1Niv0Sm48omVmvP7ZHx/Z1GmcMhPZcD6oHMWawr4fQT9QsDnG/9dEfCJ2I/B7/Phd44ZsN6HoH3skN86vvNvFLSPV+6/yh0a+JVSBTPGEE2kCPikqHLdVMowMBrnyHCU3kiM/pEYY4kko7EUsUQSv0/w+6xgHQocC5Z+nxVgAeJJQzSRZCyeYmgszlDU+ssjEk0wHLU+aMYSSesvhkSKpB2/jGE8SDth1mD9JZFIWn9lxJOGsXjS/kod99dSOfnEutHT+cDwCfa/heATGf/3DviEppoK/vf2S4o6TyGBXyd3lVKANcItZYlSn09orA7RWB3i9FkuXliZJJIpookUY/Ek0UTK+oBIpcZTUqmU9cEB1veUseZVkilDIpUa/0Bx0lZR+8PI+pCxfk6kzPhxUylD0j5mMmVIGkMqZUjYx4wnUyWvopcvDfxKqWkp4LfSO9VvUbA9mRQ9ayIic0XkBRHZIiKbROSP0+xzuYgMiMg6++tvS7tcpZRSpSrloy4BfNYY85qI1AJrReQZY8zmSfu9bIy5oYTzKKWUclHRI35jTJcx5jX75wiwBWhz68KUUkqVhysFsiKyAFgB/CbN05eIyHoR+amInOXG+ZRSShWv5FkNEakBHgX+xBgzecmm14D5xpghEbke+CGQ9nZMEbkNuA1g3rx5pV6WUkqpDEoa8YtIECvof88Y89jk540xg8aYIfvnp4CgiKRdr88Yc68xpsMY09Hc3FzKZSmllMqilKoeAb4LbDHGfD3DPq32fojISvt8R4o9p1JKqdKVkup5G/AR4A0RWWdv+2tgHoAx5h7g/cAnRSQBjAI3m5PxVmGllJpGTsqWDSLSC+wt8tebgMMuXs6pYDq+Zpier3s6vmaYnq+70Nc83xiTV578pAz8pRCRNfn2q/CK6fiaYXq+7un4mmF6vu5yvmbtd6qUUtOMBn6llJpmvBj47z3RF3ACTMfXDNPzdU/H1wzT83WX7TV7LsevlFIqOy+O+JVSSmXhmcAvIteKyDYR2Skinz/R11Mumdphi8gMEXlGRHbY3xtP9LW6TUT8IvK6iPzYfrxQRH5jv+aHRST72oOnIBFpEJFHRGSr/Z5f4vX3WkT+1P6/vVFEHhKRSi++1yJyn4j0iMjGCdvSvrdiucuObxtE5PxSzu2JwC8ifuBbwHXAmcAtInLmib2qsnHaYS8DLgbusF/r54HnjDGLgefsx17zx1hdYB1fBb5hv+Y+4OMn5KrK65vA08aYpcByrNfv2fdaRNqAzwAdxpizAT9wM958r+8Hrp20LdN7ex1Wn7PFWD3Nvl3KiT0R+IGVwE5jzG5jTAz4PnDTCb6mssjSDvsm4AF7tweA956YKywPEWkH3gN8x34swJXAI/YuXnzNdcA7sFqjYIyJGWP68fh7jdVRICwiAaAK6MKD77Ux5iXg6KTNmd7bm4AHjeXXQIOIzC723F4J/G3A/gmPO5kGawNMaofdYozpAuvDATgFVj0tyL8AnwOcFbJnAv3GmIT92Ivv+WlAL/CfdorrOyJSjYffa2PMAeBrwD6sgD8ArMX777Uj03vraozzSuCXNNs8Xa6Uox22p4jIDUCPMWbtxM1pdvXaex4Azge+bYxZAQzjobROOnZO+yZgITAHqMZKc0zmtfc6F1f/v3sl8HcCcyc8bgcOnqBrKbsM7bC7nT/97O89J+r6yuBtwI0isgcrjXcl1l8ADXY6ALz5nncCncYYZ4GjR7A+CLz8Xl8FvGmM6TXGxIHHgEvx/nvtyPTeuhrjvBL4VwOL7Zn/ENZk0BMn+JrKIks77CeAW+2fbwV+9FZfW7kYY/7KGNNujFmA9d4+b4z5EPACVgdY8NhrBjDGHAL2i8gSe9O7gM14+L3GSvFcLCJV9v915zV7+r2eINN7+wTwUbu652JgwEkJFcUY44kv4HpgO7AL+L8n+nrK+Dovw/oTbwOwzv66Hivn/Ryww/4+40Rfa5le/+XAj+2fTwNeBXYCPwAqTvT1leH1ngessd/vHwKNXn+vgS8DW4GNwH8BFV58r4GHsOYx4lgj+o9nem+xUj3fsuPbG1hVT0WfW+/cVUqpacYrqR6llFJ50sCvlFLTjAZ+pZSaZjTwK6XUNKOBXymlphkN/EopNc1o4FdKqWlGA79SSk0z/x/lTbB7bdIt5gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.836875\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "#  Training Parameters\n",
    "learning_rate = 0.1\n",
    "training_epochs = 100\n",
    "\n",
    "# Define tf Graph Inputs\n",
    "X = tf.placeholder(\"float\",[None,1])\n",
    "y = tf.placeholder(\"float\",[None,1])\n",
    "\n",
    "# Create Model variables \n",
    "# Set model weights\n",
    "W = tf.Variable(np.random.randn(), name=\"weight\")\n",
    "b = tf.Variable(np.random.randn(), name=\"bias\")\n",
    "\n",
    "# Construct a linear model\n",
    "y_pred = tf.add(tf.multiply(X, W), b)\n",
    "\n",
    "# Minimize the squared errors\n",
    "cost = tf.reduce_sum(tf.pow(y_pred - y,2))/(2*n_samples) #L2 loss\n",
    "\n",
    "# Define the optimizer\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost) \n",
    "\n",
    "# Launch the graph\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    cost_plot = []\n",
    "    # Fit all training data\n",
    "    for epoch in range(training_epochs):\n",
    "        sess.run(optimizer, \n",
    "                 feed_dict={X: train_X, y: train_Y})       \n",
    "        cost_plot.append(sess.run(cost, \n",
    "                                  feed_dict={X: train_X, y:train_Y}))\n",
    "                \n",
    "    print(\"\")\n",
    "    print(\"Optimization Finished!\")\n",
    "    print(\"cost=\", \n",
    "          sess.run(cost,\n",
    "                   feed_dict={X: train_X, y: train_Y}),\n",
    "          \"W=\", \n",
    "          sess.run(W), \n",
    "          \"b=\", \n",
    "          sess.run(b))\n",
    "    \n",
    "    fig, ax = plt.subplots(1, 1)\n",
    "    ax.set_ylabel('Results',\n",
    "              rotation=0,\n",
    "              ha='right', # horizontalalignment\n",
    "              ma='left', # multiline alignment\n",
    "             )\n",
    "    ax.set_xlabel('TV')\n",
    "    fig.set_facecolor('#EAEAF2')\n",
    "    ax.plot(train_X, \n",
    "            train_Y, 'o', \n",
    "            color=sns.xkcd_rgb['pale red'], \n",
    "            alpha=0.6,label='Original data')\n",
    "    plt.plot(train_X, \n",
    "             sess.run(W) * train_X + sess.run(b), \n",
    "             label='Fitted line')\n",
    "    plt.show()\n",
    "    \n",
    "    x = range(len(cost_plot))\n",
    "    plt.plot(x,  np.sqrt(cost_plot))\n",
    "    plt.show()\n",
    "    print(cost_plot[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise\n",
    "\n",
    "Tune the learning parameters of the previous example in order to get a better result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T09:58:37.226532Z",
     "start_time": "2019-03-05T09:58:37.224249Z"
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiple Linear Regression\n",
    "\n",
    "Let's use three features as input vector : TV, Radio, Newspaper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:45.022609Z",
     "start_time": "2019-03-05T11:47:45.011369Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "(200, 3) (200, 1)\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "\n",
    "# Parameters\n",
    "learning_rate = 1e-2\n",
    "training_epochs = 2000\n",
    "display_step = 200\n",
    "\n",
    "import numpy as np\n",
    "data = pd.read_csv('data/Advertising.csv',index_col=0)\n",
    "train_X = data[['TV','Radio','Newspaper']].values\n",
    "train_Y = data.Sales.values \n",
    "train_Y = train_Y[:,np.newaxis]\n",
    "n_samples = train_X.shape[0]\n",
    "print(n_samples)\n",
    "print(train_X.shape, train_Y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:47.682653Z",
     "start_time": "2019-03-05T11:47:47.081444Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch:  0001 \n",
      " cost=  67.133965 \n",
      " W=  [[0.01]\n",
      " [0.01]\n",
      " [0.01]] \n",
      " b=  1.3041263 \n",
      "\n",
      "Epoch:  0201 \n",
      " cost=  1.5321575 \n",
      " W=  [[0.04953486]\n",
      " [0.20464166]\n",
      " [0.00731467]] \n",
      " b=  1.5536476 \n",
      "\n",
      "Epoch:  0401 \n",
      " cost=  1.4822174 \n",
      " W=  [[0.04879017]\n",
      " [0.20140804]\n",
      " [0.005688  ]] \n",
      " b=  1.8276379 \n",
      "\n",
      "Epoch:  0601 \n",
      " cost=  1.4415519 \n",
      " W=  [[0.0480063 ]\n",
      " [0.19807082]\n",
      " [0.00394555]] \n",
      " b=  2.1155615 \n",
      "\n",
      "Epoch:  0801 \n",
      " cost=  1.4154505 \n",
      " W=  [[0.04730567]\n",
      " [0.19508857]\n",
      " [0.0023881 ]] \n",
      " b=  2.3728974 \n",
      "\n",
      "Epoch:  1001 \n",
      " cost=  1.4015663 \n",
      " W=  [[0.04674697]\n",
      " [0.1927107 ]\n",
      " [0.00114616]] \n",
      " b=  2.5780976 \n",
      "\n",
      "Epoch:  1201 \n",
      " cost=  1.3953625 \n",
      " W=  [[0.04634343]\n",
      " [0.19099322]\n",
      " [0.00024911]] \n",
      " b=  2.7263134 \n",
      "\n",
      "Epoch:  1401 \n",
      " cost=  1.3930321 \n",
      " W=  [[ 0.04607829]\n",
      " [ 0.18986483]\n",
      " [-0.00034028]] \n",
      " b=  2.8236945 \n",
      "\n",
      "Epoch:  1601 \n",
      " cost=  1.3923008 \n",
      " W=  [[ 0.04591998]\n",
      " [ 0.18919112]\n",
      " [-0.00069217]] \n",
      " b=  2.8818364 \n",
      "\n",
      "Epoch:  1801 \n",
      " cost=  1.3921111 \n",
      " W=  [[ 0.04583442]\n",
      " [ 0.18882698]\n",
      " [-0.00088239]] \n",
      " b=  2.9132626 \n",
      "\n",
      "Optimization Finished!\n",
      "cost= \n",
      " 1.392071 \n",
      " W= \n",
      " [[ 0.04579295]\n",
      " [ 0.18865052]\n",
      " [-0.00097457]] \n",
      " b= \n",
      " 2.9284918\n"
     ]
    }
   ],
   "source": [
    "# Define tf Graph Inputs\n",
    "X = tf.placeholder(\"float\",[None,3])\n",
    "y = tf.placeholder(\"float\",[None,1])\n",
    "\n",
    "# Create Model variables \n",
    "# Set model weights\n",
    "W = tf.Variable(tf.zeros([3, 1]),name=\"bias\")\n",
    "b = tf.Variable(np.random.randn(), name=\"bias\")\n",
    "\n",
    "# Construct a multidimensional linear model\n",
    "y_pred = tf.matmul(X, W) + b\n",
    "\n",
    "# Minimize the squared errors\n",
    "cost = tf.reduce_sum(tf.pow(y_pred-y,2))/(2*n_samples) #L2 loss\n",
    "\n",
    "# Define the optimizer\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost) \n",
    "\n",
    "# Launch the graph\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    # Fit all training data\n",
    "    for epoch in range(training_epochs):\n",
    "        sess.run(optimizer, feed_dict={X: train_X, y: train_Y})\n",
    "        \n",
    "        #Display logs per epoch step\n",
    "        if epoch % display_step == 0:\n",
    "            print(\"Epoch: \", '%04d' % (epoch+1), \"\\n cost= \", sess.run(cost, feed_dict={X: train_X, y: train_Y}), \\\n",
    "          \"\\n W= \", sess.run(W), \"\\n b= \", sess.run(b), \"\\n\")\n",
    "        \n",
    "    print(\"Optimization Finished!\")\n",
    "    print(\"cost= \\n\", sess.run(cost, feed_dict={X: train_X, y: train_Y}), \\\n",
    "          \"\\n W= \\n\", sess.run(W), \"\\n b= \\n\", sess.run(b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## `tf` helpers\n",
    "\n",
    "Updating of parameters through many training cycles can be dangerous (f.e. if your computer lose power). The `tf.train.Saver` class can save the graph variables for later reuse. \n",
    "\n",
    "```python\n",
    "...\n",
    "saver = tf.train.Saver()\n",
    "with tf.Session() as sess:\n",
    "    for step in range(training_steps):\n",
    "        sess.run(...)\n",
    "        if step % 1000 == 0:\n",
    "            saver.save(sess, 'my-model', global_step=step)\n",
    "    # evaluation\n",
    "    saver.safe(sess, 'my-model', global_step=training_steps)\n",
    "    sess.close()\n",
    "```\n",
    "\n",
    "If we want to recover the training from a certain point we should use the `tf.train.get_checkpoint_state` method, which will verify that there is a checkpoint saved, and the `tf.train.Saver.restore` method to recover the variable values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Logistic regression.\n",
    "\n",
    "Complete the following code.\n",
    "\n",
    "The linear model predicts a constinous value. Now we are going to write a model that can answer yes/no question: **logistic regression**:\n",
    "\n",
    "$$ \\hat{y} = \\frac{1}{1+ e^{-(WX + b)}} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:47:53.168003Z",
     "start_time": "2019-03-05T11:47:53.156410Z"
    }
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import os\n",
    "\n",
    "tf.reset_default_graph()\n",
    "\n",
    "# same params and variables initialization as log reg.\n",
    "W = tf.Variable(tf.zeros([5, 1]), name=\"weights\")\n",
    "b = tf.Variable(0., name=\"bias\")\n",
    "\n",
    "# your code here: write a function called 'inference' to implement the logistic regression model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **cross-entropy** loss function is the best suited for logistic regression:\n",
    "\n",
    "$$ L = -\\sum_i (y_i \\cdot \\log (\\hat{y}_i) + (1 - y_i) \\cdot  \\log (1 - \\hat{y}_i)) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss(X, Y):\n",
    "    \n",
    "    #your code here\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are going to read the survivor Titanic dataset. The model will have to infer, based on the passenger age, sex and ticket class if the passenger survived or not. We will create a batch to read many rows packed in a single tensor for computing the inference efficiently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_csv(batch_size, file_name, record_defaults):\n",
    "    \n",
    "    #\n",
    "    \n",
    "    filename_queue = tf.train.string_input_producer([os.path.join(os.getcwd(), file_name)])\n",
    "\n",
    "    reader = tf.TextLineReader(skip_header_lines=1)\n",
    "    key, value = reader.read(filename_queue)\n",
    "\n",
    "    # decode_csv will convert a Tensor from type string (the text line) in\n",
    "    # a tuple of tensor columns with the specified defaults, which also\n",
    "    # sets the data type for each column\n",
    "    \n",
    "    decoded = tf.decode_csv(value, record_defaults=record_defaults)\n",
    "\n",
    "    # batch actually reads the file and loads \"batch_size\" rows in a single tensor\n",
    "    return tf.train.shuffle_batch(decoded,\n",
    "                                  batch_size=batch_size,\n",
    "                                  capacity=batch_size * 50,\n",
    "                                  min_after_dequeue=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have *categorical* features in this dataset (`ticket_class, gender`) and we need to convert them to numbers. To this end we can convert each categorical feature to $N$ boolean features that represent each possible value. \n",
    "\n",
    "In case of categorical values with to values it is enough to use a binary feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def inputs():\n",
    "    passenger_id, survived, pclass, name, sex, age, sibsp, parch, ticket, fare, cabin, embarked = \\\n",
    "        read_csv(100, \"data/train_titanic.csv\", [[0.0], [0.0], [0], [\"\"], [\"\"], [0.0], [0.0], [0.0], [\"\"], [0.0], [\"\"], [\"\"]])\n",
    "\n",
    "    # convert categorical data\n",
    "    is_first_class = tf.to_float(tf.equal(pclass, [1]))\n",
    "    is_second_class = tf.to_float(tf.equal(pclass, [2]))\n",
    "    is_third_class = tf.to_float(tf.equal(pclass, [3]))\n",
    "\n",
    "    gender = tf.to_float(tf.equal(sex, [\"female\"]))\n",
    "\n",
    "    # Finally we pack all the features in a single matrix;\n",
    "    # We then transpose to have a matrix with one example per row and one feature per column.\n",
    "    features = tf.transpose(tf.stack([is_first_class, is_second_class, is_third_class, gender, age]))\n",
    "    survived = tf.reshape(survived, [100, 1])\n",
    "\n",
    "    return features, survived\n",
    "\n",
    "\n",
    "def train(total_loss):   \n",
    "    learning_rate = 0.01\n",
    "    return tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss)\n",
    "\n",
    "\n",
    "def evaluate(sess, X, Y):\n",
    "    predicted = tf.cast(inference(X) > 0.5, tf.float32)\n",
    "    print(sess.run(tf.reduce_mean(tf.cast(tf.equal(predicted, Y), tf.float32))))\n",
    "\n",
    "# Launch the graph in a session, setup boilerplate\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    tf.global_variables_initializer().run()\n",
    "\n",
    "    X, Y = inputs()\n",
    "\n",
    "    total_loss = loss(X, Y)\n",
    "    train_op = train(total_loss)\n",
    "\n",
    "    # actual training loop\n",
    "    training_steps = 5000\n",
    "    for step in range(training_steps):\n",
    "        sess.run([train_op])\n",
    "        # for debugging and learning purposes, see how the loss gets decremented thru training steps\n",
    "        if step % 100 == 0:\n",
    "            print(\"loss: \", sess.run([total_loss]))\n",
    "\n",
    "    evaluate(sess, X, Y)\n",
    "\n",
    "    import time\n",
    "    time.sleep(5)\n",
    "\n",
    "    sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 'tf' Neural Network from scratch\n",
    "\n",
    "Let's classify handwritten digits:\n",
    "\n",
    "![alt text](images/mnistExamples.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:58:46.151686Z",
     "start_time": "2019-03-05T11:58:45.508346Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/train-images-idx3-ubyte.gz\n",
      "Extracting data/train-labels-idx1-ubyte.gz\n",
      "Extracting data/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJIAAACPCAYAAAARM4LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAB/9JREFUeJzt3V+IVOcZBvDn6eqiUNDaGllcceN/Yy+sxqK0F8G6YHOTG6tZsAYJLIoFlYhNUi+9EIpVwYIIXdKLmqK2mNWbqMEgYg0qLGqqG21hu4uirv9FMUbeXsxxme9kd+Y4886ZM7PPD2TnPXN25kUevvPNOTvfoZlBpFw/qHYDUh8UJHGhIIkLBUlcKEjiQkESFwqSuFCQxEVZQSK5lGQ3yWskP/RqSmoPSz2zTbIBwDcAWgH0ATgLoM3M/l3gd3Qavfb0m9n4YjuVMyL9HMA1M/uvmX0L4O8A3inj9SSbepLsVE6QJgLozav7om0Bku0kz5E8V8Z7ScaNKON3Oci27x26zGwvgL2ADm31rJwRqQ/ApLy6GcD18tqRWlVOkM4CmE7ydZKNAN4F0OnTltSakg9tZvYdyd8B+BxAA4AOM/varTOpKSV//C/pzTRHqkXnzezNYjvpzLa4UJDEhYIkLhQkcaEgiQsFSVwoSOJCQRIXCpK4UJDEhYIkLhQkcaEgiQsFSVwoSOJCQRIXCpK4UJDEhYIkLsr5XltdW716dVDH/7b9zp07A49nz54dPHf69OmgPnXqlHN32aMRSVwoSOJCQRIXmZ4jtbW1BfW8efMGHsfnMN7Gjh1b8PkXL14MPG5sbAyee/r0aVA/efIkqC9evBjUy5cvD+rbt28n7jMrNCKJCwVJXChI4iJT3/3fvn17UK9fvz6oGxoa/JvKgBMnTgR1fG548+bNNNuJ03f/JT1Fg0Syg+Qtkpfyto0jeYzk1ejnjyrbpmRdkhHpEwBLY9s+BPCFmU0H8EVUyzCWaI5EsgXAETP7aVR3A3jLzG6QbALwpZnNTPA6Bd+st7c3qJubm4P6woULA4/j52peVfz616FDh8p6vXytra1BvWrVqqBuaWkp+PvxOdOKFSuCOuXzTBWdI00wsxsAEP18rcTXkTpR8TPbJNsBtFf6faS6Sh2RbkaHNEQ/bw21o5ntNbM3kwyPUrtKnSP9EcAdM9sW3YNknJltTvA6Bd9sxowZQT1nzpygPn78+MDjR48eFe07K6ZMmRLUR44cCer43zPFbdq0Kajj59sqzGeORPJTAP8CMJNkH8n3AWwD0EryKnL3ItlWbrdS24rOkcysbYinfuXci9QwndkWF5m61jZcLFu2LKgPHDhQcP/+/v6gHj++6F2vPOlam6RHQRIXCpK4UJDEhYIkLhQkcZHpryPVi7Vr1wb1ggULXun3R40aFdTz588P6vPnz5fWmCONSOJCQRIXCpK40CWSITQ1NQX1ypUrg3rDhg0lvxY52J3uk3v48GFQjxkzpqzXK0KXSCQ9CpK4UJDExbA9j7RkyZKgjp+baW8Pv68Q/3PZauro6Kh2C9+jEUlcKEjiQkESF3U7R5o2bVpQ79mzJ6gXL14c1K96bqenp2fg8b179wruu2XLlqB+9uxZUO/evTuoZ84s/O3369evJ2kxVRqRxIWCJC4UJHFRN3OkjRs3BvW6deuCeurUqUH9+PHjoL5//35Q79y5M6jj85L820Tkz5dK8eDBg4LPx7+efvjw4bLerxI0IokLBUlcKEjiom7mSIsWLQrq+Jyos7MzqONLw5w8ebIyjQ1i7ty5QT158uSC+8fPO125csW9p3JpRBIXSdZHmkTyBMnLJL8muT7ariWSZUCSEek7AB+Y2WwACwGsI/kGtESy5Emy0NYNAC9XsH1E8jKAiQDeAfBWtNtfAXwJ4PcV6TKBNWvWBHX+UsoAsHXr1jTbKSh+HXDChAkF989f8jCrXmmOFK0l+TMAX0FLJEuexJ/aSP4QwD8AbDCzh0mvlmt55OEh0YhEciRyIfqbmf0z2pxoiWQtjzw8FB2RmBt6/gLgspn9Ke+pTgDvIbei7XsAPqtIhwndvXs3qLM0J4pbuHBhwefj1/127dpVyXZcJDm0/QLAbwFcJNkVbfsYuQDtj5ZL/h+A31SmRakFST61nQIw1IRISyQLAJ3ZFid1c60ty+K3Z581a1bB/Y8ePRrUZ86cce/Jm0YkcaEgiQsFSVxojpSC+K1HR4wI/9vjf7O9Y8eOSrfkTiOSuFCQxIUObRXQ1hbe4m706NFBHf96UXwJnVr4uB+nEUlcKEjiQkESF5ojORg5cmRQb94c3nD8+fPnQX3w4MGg3r9/f2UaS5FGJHGhIIkLBUlcaI7kIH4bjn379gV1V1dXUB87dqziPaVNI5K4UJDEhYIkLnSbLSlGt9mS9ChI4kJBEhdpn0fqB9AD4CfR4yzKam/V6qvwuoSRVCfbA29KnsvqohJZ7S2rfb2kQ5u4UJDERbWCtLdK75tEVnvLal8AqjRHkvqjQ5u4SDVIJJeS7CZ5jWRVl1Mm2UHyFslLedsysXZ4La5tnlqQSDYA+DOAXwN4A0BbtF53tXwCYGlsW1bWDq+9tc3NLJV/ABYB+Dyv/gjAR2m9/xA9tQC4lFd3A2iKHjcB6K5mf3l9fQagNav9mVmqh7aJAHrz6r5oW5Zkbu3wWlnbPM0gDbYOpT4yFhBf27za/RSSZpD6AEzKq5sBZO2+44nWDk9DOWubV0OaQToLYDrJ10k2AngXubW6s+Tl2uFAFdcOT7C2OZCBtc0DKU8a3wbwDYD/APhDlSewnyJ3s57nyI2W7wP4MXKfhq5GP8dVqbdfInfYvwCgK/r3dlb6G+yfzmyLC53ZFhcKkrhQkMSFgiQuFCRxoSCJCwVJXChI4uL//sM/3X2Y7IMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 144x144 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class:  7\n"
     ]
    }
   ],
   "source": [
    "# Import MINST data\n",
    "# The MNIST data is split into three parts: 55,000 data points of training data (mnist.train), \n",
    "# 10,000 points of test data (mnist.test), and 5,000 points of validation data (mnist.validation).\n",
    "\n",
    "# Both the training set and test set contain images and their corresponding labels; for example the \n",
    "# training images are mnist.train.images and the training labels are mnist.train.labels.\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "mnist = input_data.read_data_sets(\"data/\", one_hot=True)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(2, 2))\n",
    "plt.imshow(mnist.train.images[0].reshape((28, 28)), interpolation='nearest', cmap='gray')\n",
    "plt.show()\n",
    "\n",
    "print(\"Class: \", np.argmax(mnist.train.labels[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:48:01.408944Z",
     "start_time": "2019-03-05T11:48:01.370766Z"
    }
   },
   "outputs": [],
   "source": [
    "# Parameters\n",
    "learning_rate = 0.001\n",
    "training_epochs = 15\n",
    "batch_size = 100\n",
    "display_step = 1\n",
    "\n",
    "# Network Parameters\n",
    "n_hidden_1 = 256 # 1st layer number of features\n",
    "n_hidden_2 = 256 # 2nd layer number of features\n",
    "n_input    = 784 # MNIST data input (img shape: 28*28)\n",
    "n_classes  = 10  # MNIST total classes (0-9 digits)\n",
    "\n",
    "# tf Graph input\n",
    "X = tf.placeholder(\"float\", [None, n_input])\n",
    "Y = tf.placeholder(\"float\", [None, n_classes])\n",
    "\n",
    "# Store layers weight & bias\n",
    "weights = {\n",
    "    'h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
    "    'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
    "    'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes]))\n",
    "}\n",
    "biases = {\n",
    "    'b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
    "    'b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
    "    'out': tf.Variable(tf.random_normal([n_classes]))\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:48:03.287003Z",
     "start_time": "2019-03-05T11:48:03.281960Z"
    }
   },
   "outputs": [],
   "source": [
    "# Create model\n",
    "def multilayer_perceptron(x):\n",
    "    # Hidden fully connected layer with 256 neurons\n",
    "    layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])\n",
    "    # Hidden fully connected layer with 256 neurons\n",
    "    layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])\n",
    "    # Output fully connected layer with a neuron for each class\n",
    "    out_layer = tf.matmul(layer_2, weights['out']) + biases['out']\n",
    "    return out_layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:48:05.138426Z",
     "start_time": "2019-03-05T11:48:05.014267Z"
    }
   },
   "outputs": [],
   "source": [
    "# Construct model\n",
    "logits = multilayer_perceptron(X)\n",
    "\n",
    "# Define loss and optimizer\n",
    "loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(\n",
    "    logits=logits, labels=Y))\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\n",
    "train_op = optimizer.minimize(loss_op)\n",
    "\n",
    "# Initializing the variables\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T11:48:26.092599Z",
     "start_time": "2019-03-05T11:48:05.858881Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0001 cost=314.028784457\n",
      "Epoch: 0002 cost=100.116631116\n",
      "Epoch: 0003 cost=73.887949876\n",
      "Epoch: 0004 cost=58.951892239\n",
      "Epoch: 0005 cost=50.355873285\n",
      "Epoch: 0006 cost=43.699490223\n",
      "Epoch: 0007 cost=38.764815619\n",
      "Epoch: 0008 cost=35.302322959\n",
      "Epoch: 0009 cost=32.298332145\n",
      "Epoch: 0010 cost=29.276483191\n",
      "Epoch: 0011 cost=27.880766920\n",
      "Epoch: 0012 cost=26.116198704\n",
      "Epoch: 0013 cost=24.149716031\n",
      "Epoch: 0014 cost=22.982962157\n",
      "Epoch: 0015 cost=22.416762345\n",
      "Optimization Finished!\n",
      "Accuracy: 0.8941\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "\n",
    "    # Training cycle\n",
    "    for epoch in range(training_epochs):\n",
    "        avg_cost = 0.\n",
    "        total_batch = int(mnist.train.num_examples/batch_size)\n",
    "        # Loop over all batches\n",
    "        for i in range(total_batch):\n",
    "            batch_x, batch_y = mnist.train.next_batch(batch_size)\n",
    "            # Run optimization op (backprop) and cost op (to get loss value)\n",
    "            _, c = sess.run([train_op, loss_op], feed_dict={X: batch_x,\n",
    "                                                            Y: batch_y})\n",
    "            # Compute average loss\n",
    "            avg_cost += c / total_batch\n",
    "        # Display logs per epoch step\n",
    "        if epoch % display_step == 0:\n",
    "            print(\"Epoch:\", '%04d' % (epoch+1), \"cost={:.9f}\".format(avg_cost))\n",
    "    print(\"Optimization Finished!\")\n",
    "\n",
    "    # Test model\n",
    "    pred = tf.nn.softmax(logits)  # Apply softmax to logits\n",
    "    correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(Y, 1))\n",
    "    # Calculate accuracy\n",
    "    accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
    "    print(\"Accuracy:\", accuracy.eval({X: mnist.test.images, Y: mnist.test.labels}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}