Initial_Code_and_Result_Zaka.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xh8a50KOQULe"
      },
      "source": [
        "**CIND 860: Advanced Data Analytics Project**\n",
        "\n",
        "**Section: DJH**\n",
        "\n",
        "**Student Name: Muhammad Zaka Shaheryar**\n",
        "\n",
        "**Student #: 500648718**\n",
        "\n",
        "**Supervisor Name: Tamer Abdou**\n",
        "\n",
        "**Date of Submission: March 14th, 2024**\n",
        "\n",
        "**Project Title: Analysis of Sign language MNIST using Deep learning**\n",
        "\n",
        "**Project Deliverable: Initial Code and Result**\n",
        "\n",
        "**Github Link: https://github.com/Zaka123456/CIND860**\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VC9vNmTJu9Ze"
      },
      "source": [
        "**Link to dataset**\n",
        "\n",
        "https://www.kaggle.com/datasets/datamunge/sign-language-mnist"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uv0Z5phQOJJA"
      },
      "source": [
        "**Dataset Description**\n",
        "\n",
        "American Sign Language(ASL) is a complete, natural language that has the same linguistic properties as spoken languages, with grammar that differs from English. ASL is expressed by the movement of hands and faces. This dataset consists of 27,455 images of hand signs, each image is of 28 x 28 size and in grayscale format. The dataset format is patterned to match closely with the classic MNIST. Images in the dataset belong to a label from 0–25 representing letters from A-Z(but no cases of 9=J or 25=Z as they involve hand motion). The training data(27,455 cases) and the test data(7,172 cases) are approximately half the size of standard MNIST but otherwise similar to a header row of the label, pixel1, pixel….pixel784. The original hand gesture image data represented multiple users repeating gestures against different backgrounds. The Sign Language MNIST data came from greatly extending the small number (1704) of the color images included as not cropped around the hand region of interest.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UCnoN9z8Mwnp"
      },
      "source": [
        "**Data Dictionary**\n",
        "\n",
        "**label:** Numeric target column of number 0-25 (excluding 9 and 25)\n",
        "\n",
        "**label_alpha:** categoriacal target column of alphabets A-Z (excluding J and Z)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Libraries**"
      ],
      "metadata": {
        "id": "MUXPLQdUjVmX"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hP8W09ko583w"
      },
      "outputs": [],
      "source": [
        "!pip install pydantic\n",
        "!pip install PyYAML\n",
        "!pip install jinja2\n",
        "!pip install visions\n",
        "!pip install htmlmin\n",
        "!pip install phik\n",
        "!pip install requests\n",
        "!pip install tqdm\n",
        "!pip install seaborn\n",
        "!pip install multimethod\n",
        "!pip install statsmodels\n",
        "!pip install typeguard\n",
        "!pip install imagehash\n",
        "!pip install wordcloud\n",
        "!pip install dacite\n",
        "!pip install numba\n",
        "!pip install np_utils\n",
        "!pip install keras-vis\n",
        "!pip install keras-utils\n",
        "!{sys.executable} -m pip install -U ydata-profiling\n",
        "!jupyter nbextension enable --py widgetsnbextension"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "RhUxqSIp9c_3"
      },
      "outputs": [],
      "source": [
        "# Import the necessary modules\n",
        "import pandas as pd\n",
        "import math\n",
        "import random\n",
        "import matplotlib.pyplot as plt\n",
        "from scipy import stats\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "import sys\n",
        "import os\n",
        "#os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\" # Force TF to use only the CPU\n",
        "import tensorflow as tf\n",
        "import keras\n",
        "import numpy as np\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras import models\n",
        "from tensorflow.keras import optimizers\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "from keras.datasets import fashion_mnist\n",
        "#from keras.utils import np_utils\n",
        "#from keras.utils.vis_utils import plot_model\n",
        "from sklearn.feature_selection import SelectKBest\n",
        "from sklearn.feature_selection import chi2\n",
        "import seaborn as sns\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.feature_selection import RFE\n",
        "from sklearn.linear_model import LogisticRegression"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SxNiPX5hF8a9"
      },
      "outputs": [],
      "source": [
        "print(tf.__version__)\n",
        "print(keras.__version__)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Data Preparation**"
      ],
      "metadata": {
        "id": "j878v3j7jJ4N"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bdRFOfk7_ae2"
      },
      "outputs": [],
      "source": [
        "# Please note: Due to the size of the dataset, it can take anywhere from 10-15 minutes to fully upload the train dataset (around 27,000 rows)\n",
        "\n",
        "# Uploading train dataset\n",
        "from google.colab import files\n",
        "file_upload_train = files.upload()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KpRPpq8X-MJ3"
      },
      "outputs": [],
      "source": [
        "# Uploading test dataset\n",
        "from google.colab import files\n",
        "file_upload_test = files.upload()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ueWXEazm6uup"
      },
      "outputs": [],
      "source": [
        "sign_mnist_train_file = \"sign_mnist_train.csv\"\n",
        "data_sign_train = pd.read_csv(sign_mnist_train_file, sep = ',', header=0)\n",
        "\n",
        "sign_mnist_test_file = \"sign_mnist_test.csv\"\n",
        "data_sign_test = pd.read_csv(sign_mnist_test_file, sep = ',', header=0)\n",
        "\n",
        "display(data_sign_train)\n",
        "display(data_sign_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qW_ucjIYA1Zw"
      },
      "outputs": [],
      "source": [
        "print(data_sign_train.shape)\n",
        "print(data_sign_test.shape)\n",
        "print(len(data_sign_train))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "_SVHsrVjBnD_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "172d07fe-007e-4ca8-a889-85c5ae52753d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "label       0\n",
            "pixel1      0\n",
            "pixel2      0\n",
            "pixel3      0\n",
            "pixel4      0\n",
            "           ..\n",
            "pixel780    0\n",
            "pixel781    0\n",
            "pixel782    0\n",
            "pixel783    0\n",
            "pixel784    0\n",
            "Length: 785, dtype: int64\n",
            "label        24\n",
            "pixel1      255\n",
            "pixel2      255\n",
            "pixel3      255\n",
            "pixel4      255\n",
            "           ... \n",
            "pixel780    255\n",
            "pixel781    255\n",
            "pixel782    255\n",
            "pixel783    255\n",
            "pixel784    255\n",
            "Length: 785, dtype: int64\n"
          ]
        }
      ],
      "source": [
        "print(data_sign_train.min())\n",
        "print(data_sign_test.max())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rqUK6smLBzRy"
      },
      "outputs": [],
      "source": [
        "print(type(data_sign_train))\n",
        "print(type(data_sign_test))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4bmgbDpsCRMA"
      },
      "outputs": [],
      "source": [
        "print(data_sign_train['label'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8pBRwpDADOBC"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label'].describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gmth4CGZDsco"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label'].info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JBFDarvWDv1P"
      },
      "outputs": [],
      "source": [
        "display(data_sign_train['label'].unique())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "xtnQr5wlEvuD"
      },
      "outputs": [],
      "source": [
        "num_to_cat = {0: 'A', 1: 'B', 2: 'C', 3: 'D', 4: 'E', 5: 'F', 6:'G', 7:'H', 8:'I', 10:'K', 11:'L', 12:'M', 13:'N', 14:'O', 15:'P', 16:'Q', 17:'R', 18:'S', 19:'T', 20:'U', 21:'V', 22:'W', 23:'X', 24:'Y'}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "OhKmmB9LGEFd"
      },
      "outputs": [],
      "source": [
        "# Convert numerical labels to categorical labels for train set\n",
        "categorical_labels = [num_to_cat[label] for label in data_sign_train['label']]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HHwaxhNYHH_u"
      },
      "outputs": [],
      "source": [
        "print(categorical_labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "HzcEX7n-K3gh"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'] = categorical_labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "4OEPvwctLFhR"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'] = data_sign_train['label_alpha'].astype('category')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "9YjVzSjRAS1-"
      },
      "outputs": [],
      "source": [
        "train_2 = data_sign_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "b7g65p2cOEAI"
      },
      "outputs": [],
      "source": [
        "# Convert numerical labels to categorical labels for test set\n",
        "categorical_labels = [num_to_cat[label] for label in data_sign_test['label']]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "1R4eUhRuN9zH"
      },
      "outputs": [],
      "source": [
        "data_sign_test['label_alpha'] = categorical_labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "9TaXfX89OABY"
      },
      "outputs": [],
      "source": [
        "data_sign_test['label_alpha'] = data_sign_test['label_alpha'].astype('category')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "X95GNc4GHoRC",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "outputId": "c6fe03be-ccf1-465f-8733-5173e61d6827"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "       label  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  pixel8  \\\n",
              "0          3     107     118     127     134     139     143     146     150   \n",
              "1          6     155     157     156     156     156     157     156     158   \n",
              "2          2     187     188     188     187     187     186     187     188   \n",
              "3          2     211     211     212     212     211     210     211     210   \n",
              "4         13     164     167     170     172     176     179     180     184   \n",
              "...      ...     ...     ...     ...     ...     ...     ...     ...     ...   \n",
              "27450     13     189     189     190     190     192     193     193     193   \n",
              "27451     23     151     154     157     158     160     161     163     164   \n",
              "27452     18     174     174     174     174     174     175     175     174   \n",
              "27453     17     177     181     184     185     187     189     190     191   \n",
              "27454     23     179     180     180     180     182     181     182     183   \n",
              "\n",
              "       pixel9  ...  pixel776  pixel777  pixel778  pixel779  pixel780  \\\n",
              "0         153  ...       207       207       207       206       206   \n",
              "1         158  ...       149       128        87        94       163   \n",
              "2         187  ...       201       200       199       198       199   \n",
              "3         210  ...       234       233       231       230       226   \n",
              "4         185  ...       105       105       108       133       163   \n",
              "...       ...  ...       ...       ...       ...       ...       ...   \n",
              "27450     193  ...       165        99        77        52       200   \n",
              "27451     166  ...       198       198       198       198       196   \n",
              "27452     173  ...       196       209       208       206       204   \n",
              "27453     191  ...        56        27        58       102        79   \n",
              "27454     182  ...       132       170       194       214       203   \n",
              "\n",
              "       pixel781  pixel782  pixel783  pixel784  label_alpha  \n",
              "0           206       204       203       202            D  \n",
              "1           175       103       135       149            G  \n",
              "2           198       195       194       195            C  \n",
              "3           225       222       229       163            C  \n",
              "4           157       163       164       179            N  \n",
              "...         ...       ...       ...       ...          ...  \n",
              "27450       234       200       222       225            N  \n",
              "27451       195       195       195       194            X  \n",
              "27452       203       202       200       200            S  \n",
              "27453        47        64        87        93            R  \n",
              "27454       197       205       209       215            X  \n",
              "\n",
              "[27455 rows x 786 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-88d6b0cd-193c-4c9f-9277-1d52460060f6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>pixel1</th>\n",
              "      <th>pixel2</th>\n",
              "      <th>pixel3</th>\n",
              "      <th>pixel4</th>\n",
              "      <th>pixel5</th>\n",
              "      <th>pixel6</th>\n",
              "      <th>pixel7</th>\n",
              "      <th>pixel8</th>\n",
              "      <th>pixel9</th>\n",
              "      <th>...</th>\n",
              "      <th>pixel776</th>\n",
              "      <th>pixel777</th>\n",
              "      <th>pixel778</th>\n",
              "      <th>pixel779</th>\n",
              "      <th>pixel780</th>\n",
              "      <th>pixel781</th>\n",
              "      <th>pixel782</th>\n",
              "      <th>pixel783</th>\n",
              "      <th>pixel784</th>\n",
              "      <th>label_alpha</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>107</td>\n",
              "      <td>118</td>\n",
              "      <td>127</td>\n",
              "      <td>134</td>\n",
              "      <td>139</td>\n",
              "      <td>143</td>\n",
              "      <td>146</td>\n",
              "      <td>150</td>\n",
              "      <td>153</td>\n",
              "      <td>...</td>\n",
              "      <td>207</td>\n",
              "      <td>207</td>\n",
              "      <td>207</td>\n",
              "      <td>206</td>\n",
              "      <td>206</td>\n",
              "      <td>206</td>\n",
              "      <td>204</td>\n",
              "      <td>203</td>\n",
              "      <td>202</td>\n",
              "      <td>D</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>6</td>\n",
              "      <td>155</td>\n",
              "      <td>157</td>\n",
              "      <td>156</td>\n",
              "      <td>156</td>\n",
              "      <td>156</td>\n",
              "      <td>157</td>\n",
              "      <td>156</td>\n",
              "      <td>158</td>\n",
              "      <td>158</td>\n",
              "      <td>...</td>\n",
              "      <td>149</td>\n",
              "      <td>128</td>\n",
              "      <td>87</td>\n",
              "      <td>94</td>\n",
              "      <td>163</td>\n",
              "      <td>175</td>\n",
              "      <td>103</td>\n",
              "      <td>135</td>\n",
              "      <td>149</td>\n",
              "      <td>G</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>187</td>\n",
              "      <td>188</td>\n",
              "      <td>188</td>\n",
              "      <td>187</td>\n",
              "      <td>187</td>\n",
              "      <td>186</td>\n",
              "      <td>187</td>\n",
              "      <td>188</td>\n",
              "      <td>187</td>\n",
              "      <td>...</td>\n",
              "      <td>201</td>\n",
              "      <td>200</td>\n",
              "      <td>199</td>\n",
              "      <td>198</td>\n",
              "      <td>199</td>\n",
              "      <td>198</td>\n",
              "      <td>195</td>\n",
              "      <td>194</td>\n",
              "      <td>195</td>\n",
              "      <td>C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>211</td>\n",
              "      <td>211</td>\n",
              "      <td>212</td>\n",
              "      <td>212</td>\n",
              "      <td>211</td>\n",
              "      <td>210</td>\n",
              "      <td>211</td>\n",
              "      <td>210</td>\n",
              "      <td>210</td>\n",
              "      <td>...</td>\n",
              "      <td>234</td>\n",
              "      <td>233</td>\n",
              "      <td>231</td>\n",
              "      <td>230</td>\n",
              "      <td>226</td>\n",
              "      <td>225</td>\n",
              "      <td>222</td>\n",
              "      <td>229</td>\n",
              "      <td>163</td>\n",
              "      <td>C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>13</td>\n",
              "      <td>164</td>\n",
              "      <td>167</td>\n",
              "      <td>170</td>\n",
              "      <td>172</td>\n",
              "      <td>176</td>\n",
              "      <td>179</td>\n",
              "      <td>180</td>\n",
              "      <td>184</td>\n",
              "      <td>185</td>\n",
              "      <td>...</td>\n",
              "      <td>105</td>\n",
              "      <td>105</td>\n",
              "      <td>108</td>\n",
              "      <td>133</td>\n",
              "      <td>163</td>\n",
              "      <td>157</td>\n",
              "      <td>163</td>\n",
              "      <td>164</td>\n",
              "      <td>179</td>\n",
              "      <td>N</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27450</th>\n",
              "      <td>13</td>\n",
              "      <td>189</td>\n",
              "      <td>189</td>\n",
              "      <td>190</td>\n",
              "      <td>190</td>\n",
              "      <td>192</td>\n",
              "      <td>193</td>\n",
              "      <td>193</td>\n",
              "      <td>193</td>\n",
              "      <td>193</td>\n",
              "      <td>...</td>\n",
              "      <td>165</td>\n",
              "      <td>99</td>\n",
              "      <td>77</td>\n",
              "      <td>52</td>\n",
              "      <td>200</td>\n",
              "      <td>234</td>\n",
              "      <td>200</td>\n",
              "      <td>222</td>\n",
              "      <td>225</td>\n",
              "      <td>N</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27451</th>\n",
              "      <td>23</td>\n",
              "      <td>151</td>\n",
              "      <td>154</td>\n",
              "      <td>157</td>\n",
              "      <td>158</td>\n",
              "      <td>160</td>\n",
              "      <td>161</td>\n",
              "      <td>163</td>\n",
              "      <td>164</td>\n",
              "      <td>166</td>\n",
              "      <td>...</td>\n",
              "      <td>198</td>\n",
              "      <td>198</td>\n",
              "      <td>198</td>\n",
              "      <td>198</td>\n",
              "      <td>196</td>\n",
              "      <td>195</td>\n",
              "      <td>195</td>\n",
              "      <td>195</td>\n",
              "      <td>194</td>\n",
              "      <td>X</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27452</th>\n",
              "      <td>18</td>\n",
              "      <td>174</td>\n",
              "      <td>174</td>\n",
              "      <td>174</td>\n",
              "      <td>174</td>\n",
              "      <td>174</td>\n",
              "      <td>175</td>\n",
              "      <td>175</td>\n",
              "      <td>174</td>\n",
              "      <td>173</td>\n",
              "      <td>...</td>\n",
              "      <td>196</td>\n",
              "      <td>209</td>\n",
              "      <td>208</td>\n",
              "      <td>206</td>\n",
              "      <td>204</td>\n",
              "      <td>203</td>\n",
              "      <td>202</td>\n",
              "      <td>200</td>\n",
              "      <td>200</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27453</th>\n",
              "      <td>17</td>\n",
              "      <td>177</td>\n",
              "      <td>181</td>\n",
              "      <td>184</td>\n",
              "      <td>185</td>\n",
              "      <td>187</td>\n",
              "      <td>189</td>\n",
              "      <td>190</td>\n",
              "      <td>191</td>\n",
              "      <td>191</td>\n",
              "      <td>...</td>\n",
              "      <td>56</td>\n",
              "      <td>27</td>\n",
              "      <td>58</td>\n",
              "      <td>102</td>\n",
              "      <td>79</td>\n",
              "      <td>47</td>\n",
              "      <td>64</td>\n",
              "      <td>87</td>\n",
              "      <td>93</td>\n",
              "      <td>R</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27454</th>\n",
              "      <td>23</td>\n",
              "      <td>179</td>\n",
              "      <td>180</td>\n",
              "      <td>180</td>\n",
              "      <td>180</td>\n",
              "      <td>182</td>\n",
              "      <td>181</td>\n",
              "      <td>182</td>\n",
              "      <td>183</td>\n",
              "      <td>182</td>\n",
              "      <td>...</td>\n",
              "      <td>132</td>\n",
              "      <td>170</td>\n",
              "      <td>194</td>\n",
              "      <td>214</td>\n",
              "      <td>203</td>\n",
              "      <td>197</td>\n",
              "      <td>205</td>\n",
              "      <td>209</td>\n",
              "      <td>215</td>\n",
              "      <td>X</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>27455 rows × 786 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-88d6b0cd-193c-4c9f-9277-1d52460060f6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-88d6b0cd-193c-4c9f-9277-1d52460060f6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-88d6b0cd-193c-4c9f-9277-1d52460060f6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-874ff288-739a-4f2f-8bc0-fd2230148f70\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-874ff288-739a-4f2f-8bc0-fd2230148f70')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-874ff288-739a-4f2f-8bc0-fd2230148f70 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data_sign_train"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "display(train_2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "XdqeShaqAxR3"
      },
      "outputs": [],
      "source": [
        "test_2 = data_sign_test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZK54gFmfOOHV"
      },
      "outputs": [],
      "source": [
        "display(test_2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O5I-H-jrM74P"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'].info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LTvmKZNkNHL4"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'].describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aX45CFpoNm55"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'].unique()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CULeUIbNOVNs"
      },
      "outputs": [],
      "source": [
        "data_sign_test['label_alpha'].unique()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FzoPC9KW9vgc"
      },
      "outputs": [],
      "source": [
        "data_sign_train.columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KLkP_Ehi-f2N"
      },
      "outputs": [],
      "source": [
        "data_sign_train.info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "lgZ41OR3-5Bg"
      },
      "outputs": [],
      "source": [
        "data_sign_train['label_alpha'] = data_sign_train['label_alpha'].astype(str)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Expolatory Data Analysis**"
      ],
      "metadata": {
        "id": "kKbbOoeKiKl1"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1fVFy83-yNx3"
      },
      "outputs": [],
      "source": [
        "# Filter method # start from here\n",
        "\n",
        "# Load the train dataset from the CSV file\n",
        "sign_train_file = \"sign_mnist_train.csv\"\n",
        "data_sign_train = pd.read_csv(sign_train_file, sep=',', header=0)\n",
        "\n",
        "# Separating the features (pixels - independent variables) and the target variable (label)\n",
        "X_train = data_sign_train.iloc[:, 1:784] # Features\n",
        "y_train = data_sign_train['label'].iloc[:]   # dependent variable\n",
        "\n",
        "\n",
        "# Calculate Chi-square scores and select top 25% pixels\n",
        "selector = SelectKBest(chi2, k=int(X_train.shape[1] * 0.25))\n",
        "X_train_filtered = selector.fit_transform(X_train, y_train)\n",
        "\n",
        "# Calculate the threshold for the top 25%\n",
        "threshold = np.percentile(selector.scores_, 75)\n",
        "\n",
        "# Create a dataframe with pixel indices and their importance scores\n",
        "df_importance = pd.DataFrame({\n",
        "    'Pixel Index': range(X_train.shape[1]),\n",
        "    'Importance': selector.scores_\n",
        "})\n",
        "\n",
        "# Filter pixels above the threshold\n",
        "df_importance = df_importance[df_importance['Importance'] >= threshold]\n",
        "\n",
        "# Display the top n pixels\n",
        "n = 25  # Number of top pixels to display\n",
        "df_importance = df_importance.sort_values(by='Importance', ascending=False)\n",
        "display(df_importance.head(n))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E8t1pKL5yZLr"
      },
      "outputs": [],
      "source": [
        "# Hybrid method (took 11-14 min)\n",
        "# Use the filtered pixels as the subset\n",
        "model = LogisticRegression()\n",
        "X_train_subset = X_train_filtered\n",
        "\n",
        "# Select top 25% pixels from the subset based on feature importance\n",
        "rfe = RFE(estimator=model, n_features_to_select=int(X_train_subset.shape[1] * 0.25))\n",
        "X_train_hybrid = rfe.fit_transform(X_train_subset, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aBIlRZyX0aoi"
      },
      "outputs": [],
      "source": [
        "# Load the train dataset from the CSV file\n",
        "sign_train_file = \"sign_mnist_train.csv\"\n",
        "data_sign_train = pd.read_csv(sign_train_file, sep=',', header=0)\n",
        "\n",
        "# Separating the features (pixels - independent variables) and the target variable (label)\n",
        "X_train = data_sign_train.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y_train = data_sign_train.iloc[:, 0]   # dependent\n",
        "\n",
        "# Creating a Random Forest Classifier\n",
        "rf_classifier_train = RandomForestClassifier(n_estimators=100, random_state=0)\n",
        "rf_classifier_train.fit(X_train, y_train)\n",
        "\n",
        "# Getting feature importances from the Random Forest model\n",
        "feature_importances = rf_classifier_train.feature_importances_\n",
        "\n",
        "# Using the top 20 features to select\n",
        "n_top_features = 20\n",
        "\n",
        "# Getting the indices of the top pixels\n",
        "top_indices = feature_importances.argsort()[-n_top_features:]\n",
        "\n",
        "# Selecting the top 20 of all the ranked pixels\n",
        "selected_features = X_train.columns[top_indices]\n",
        "selected_feature_importances = feature_importances[top_indices]\n",
        "\n",
        "# Creating a DataFrame for the top 20 pixels and their importances\n",
        "feature_df_train = pd.DataFrame({\n",
        "    'Features': selected_features,\n",
        "    'Importance': selected_feature_importances})\n",
        "\n",
        "# Sorting the DataFrame by importance in descending order (from largest to smallest - top 20)\n",
        "feature_df_train = feature_df_train.sort_values(by='Importance', ascending=False)\n",
        "\n",
        "# Reseting the index to start from 1 instead of 0 (this command was used because it originally ranked from 0-19, instead of 1-20)\n",
        "feature_df_train.index = range(1, n_top_features + 1)\n",
        "\n",
        "# Displaying the top 20 ranked pixels and their importance values\n",
        "print(feature_df_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Hybrid technique is choosen for project for now, bit it will be checked in later stage of project if it is better than filter technique. Also to avoid ssytem crashing small number of selected features is taken, therefore n is taken as 20."
      ],
      "metadata": {
        "id": "EU75x-kcO_u3"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FtZ_hd6fHHuM"
      },
      "outputs": [],
      "source": [
        "# Loading train dataset from the CSV file\n",
        "sign_train_file = \"sign_mnist_train.csv\"\n",
        "data_sign_train = pd.read_csv(sign_train_file, sep=',', header=0)\n",
        "\n",
        "# Separating the features (pixels) and the target variable (label)\n",
        "X_train = data_sign_train.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y_train = data_sign_train.iloc[:, 0]   # Labels\n",
        "\n",
        "# Creating a Random Forest Classifier\n",
        "rf_classifier_train = RandomForestClassifier(n_estimators=100, random_state=0)\n",
        "rf_classifier_train.fit(X_train, y_train)\n",
        "\n",
        "# Getting feature importances from the Random Forest model\n",
        "feature_importances = rf_classifier_train.feature_importances_\n",
        "\n",
        "# Creating a DataFrame to display pixel importance values\n",
        "pixel_importance_df_train = pd.DataFrame({\n",
        "    'Pixel': X_train.columns,\n",
        "    'Importance': feature_importances\n",
        "})\n",
        "\n",
        "# Sorting the DataFrame by importance in descending order and select the top 20 pixels\n",
        "top_20_pixels_train = pixel_importance_df_train.nlargest(20, 'Importance')\n",
        "\n",
        "# Creating a DataFrame to store the data for the top 20 pixels\n",
        "top_20_pixels_data_train = X_train[top_20_pixels_train['Pixel']].copy()  # Make a copy to avoid the SettingWithCopyWarning error message\n",
        "\n",
        "top_20_pixels_data_train = pd.concat([y_train, top_20_pixels_data_train], axis=1)\n",
        "\n",
        "# Saving the DataFrame to a CSV file\n",
        "top_20_pixels_data_train.to_csv(\"top_20_pixels_with_labels_train.csv\", index=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QQSKIDpoF8Qa"
      },
      "outputs": [],
      "source": [
        "display(top_20_pixels_data_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WsZCaOURFrWC"
      },
      "outputs": [],
      "source": [
        "# Panda Profiling report for the TRAIN SET (about 20,000 rows)\n",
        "!pip install ydata_profiling\n",
        "from ydata_profiling import ProfileReport\n",
        "top_20_pixels_data_train = pd.read_csv(\"top_20_pixels_with_labels_train.csv\")\n",
        "ydata_profiling_train = ProfileReport(top_20_pixels_data_train)\n",
        "\n",
        "# Saving the report to an HTML file\n",
        "ydata_profiling_train.to_file(\"top_20_pixels_train_report.html\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cK3H4GLdGrQi"
      },
      "outputs": [],
      "source": [
        "# The following command generates the pandaprofiling report for the train set for top 20 pixels\n",
        "ydata_profiling_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "giGGgDtBIGHr"
      },
      "outputs": [],
      "source": [
        "# Replicating the top 20 pixels from the train set to the test set\n",
        "\n",
        "# Loading the test dataset from the CSV file\n",
        "sign_test_file = \"sign_mnist_test.csv\"\n",
        "data_sign_test = pd.read_csv(sign_test_file, sep=',', header=0)\n",
        "\n",
        "# Selecting the same top 20 features (pixels) as in the training set\n",
        "selected_features = ['pixel246', 'pixel239', 'pixel302', 'pixel266', 'pixel210',\n",
        "                     'pixel546', 'pixel265', 'pixel353', 'pixel321', 'pixel211',\n",
        "                     'pixel245', 'pixel520', 'pixel355', 'pixel327', 'pixel465',\n",
        "                     'pixel238', 'pixel575', 'pixel326', 'pixel217', 'pixel212']\n",
        "\n",
        "# Creating a new DataFrame with the selected features and labels\n",
        "data_sign_test_selected = data_sign_test[['label'] + selected_features]\n",
        "\n",
        "# Save the selected data to a new CSV file\n",
        "data_sign_test_selected.to_csv(\"top_20_pixels_with_labels_test.csv\", index=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9p3FPiNHJEoZ"
      },
      "outputs": [],
      "source": [
        "# Panda Profiling report for the TEST SET (10,000 rows)\n",
        "\n",
        "from ydata_profiling import ProfileReport\n",
        "top_20_pixels_data_test = pd.read_csv(\"top_20_pixels_with_labels_test.csv\")\n",
        "ydata_profiling_test = ProfileReport(top_20_pixels_data_test)\n",
        "\n",
        "# Saving the report to an HTML file\n",
        "ydata_profiling_test.to_file(\"top_20_pixels_test_report.html\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eI57twCaJwCy"
      },
      "outputs": [],
      "source": [
        "# The following command generates the pandaprofiling report for the replicated test set for top 20 pixels\n",
        "ydata_profiling_test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g9utBEuvvYgv"
      },
      "source": [
        "The Pandas Profiling Report for combined dataset provides a comprehensive analysis of a dataset with 34,627 observations and 21 variables, detailing aspects like missing data, duplicates, and feature correlations. Key findings include a 0% missing data ratio, less than 0.1% duplicate rows, and high correlations among certain pixel values, indicating redundancy or strong dependencies between features. The analysis also identifies outliers and zeros in specific pixel values, crucial for preprocessing steps in machine learning projects.The report's insights on attribute types, outliers, and the dataset's memory footprint can guide data preprocessing and feature engineering, enhancing model accuracy and efficiency"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Predictive Modelling**"
      ],
      "metadata": {
        "id": "cwSiGEc6jmPD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Research Question (RQ)**\n",
        "\n",
        "1\n",
        "How does accuracy, computational efficiency, and stability of CNN compare to traditional machine learning techniques such as Random Forest, SVM, XGBoost or K-NN in classifying ASL images?\n",
        "\n",
        "\n",
        "2\n",
        "Which neural network out of MLP, AlexNet, MobileNet, VGG 16, LeNet exhibits the greatest efficiency in classifying ASL images taking into account accuracy and computational resources?\n",
        "\n",
        "3\n",
        "When compared to Naive Bayes (NB), how effective is TAN or FAN at identifying the true cause of categorial target label of english alphabets in context of Sign language MNIST dataset?"
      ],
      "metadata": {
        "id": "z4U7q_HBj2hb"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "adzV2WKahFqJ"
      },
      "outputs": [],
      "source": [
        "# The following code was used with the traditional CNN-simple model for the top 20 pixels based on importance (on the test dataset)\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "# Loading the dataset containing 'label' and the top 20 pixels from the combined dataset (70,000 rows)\n",
        "#data_sign_test_selected = pd.read_csv(\"combined_top_20_pixels_with_labels.csv\")\n",
        "data_sign_test_selected = top_20_pixels_data_train\n",
        "# Separating the features and target variable (label)\n",
        "X = data_sign_test_selected.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y = data_sign_test_selected[\"label\"]   # dependent variable\n",
        "\n",
        "# Reshaping the data to match a CNN input shape\n",
        "X = X.values.reshape(X.shape[0], 20, 1, 1)\n",
        "\n",
        "# Converting 'label' (dependent variable) to one-hot encoding\n",
        "y = to_categorical(y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B9ek-OfxsRJE"
      },
      "outputs": [],
      "source": [
        "display(data_sign_test_selected)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ikTx9pvYssG4"
      },
      "outputs": [],
      "source": [
        "display(data_sign_test_selected[data_sign_test_selected.label == 13])\n",
        "#data_sign_test_selected.info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T6CxCjJBypJ8"
      },
      "outputs": [],
      "source": [
        "#display(X)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cR0QowjRyxNF"
      },
      "outputs": [],
      "source": [
        "#display(y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_aak2XAcyn-s",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0cfb3a3e-2bd2-4aca-a943-1f30c6b34075"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "429/429 [==============================] - 3s 5ms/step - loss: 4.8094 - accuracy: 0.2679\n",
            "Epoch 2/10\n",
            "429/429 [==============================] - 2s 5ms/step - loss: 1.8766 - accuracy: 0.4499\n",
            "Epoch 3/10\n",
            "429/429 [==============================] - 3s 7ms/step - loss: 1.4670 - accuracy: 0.5531\n",
            "Epoch 4/10\n",
            "429/429 [==============================] - 2s 6ms/step - loss: 1.1712 - accuracy: 0.6346\n",
            "Epoch 5/10\n",
            "429/429 [==============================] - 5s 12ms/step - loss: 0.9266 - accuracy: 0.7067\n",
            "Epoch 6/10\n",
            "429/429 [==============================] - 5s 11ms/step - loss: 0.7195 - accuracy: 0.7721\n",
            "Epoch 7/10\n",
            "429/429 [==============================] - 3s 7ms/step - loss: 0.5325 - accuracy: 0.8354\n",
            "Epoch 8/10\n",
            "429/429 [==============================] - 2s 5ms/step - loss: 0.3904 - accuracy: 0.8850\n",
            "Epoch 9/10\n",
            "429/429 [==============================] - 2s 5ms/step - loss: 0.2751 - accuracy: 0.9232\n",
            "Epoch 10/10\n",
            "429/429 [==============================] - 2s 4ms/step - loss: 0.2028 - accuracy: 0.9471\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7f4b620fc9a0>"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ],
      "source": [
        "# Building a simple CNN model with the basic parameters\n",
        "simple_cnn_model = keras.Sequential([\n",
        "    layers.Conv2D(32, (3, 1), activation='relu', input_shape=(20, 1, 1)),\n",
        "    layers.MaxPooling2D((2, 1)),\n",
        "    layers.Flatten(),\n",
        "    layers.Dense(128, activation='relu'),\n",
        "    layers.Dense(25, activation='softmax')  # 25 classes in the top 20 pixels (labelled from 0-24)\n",
        "])\n",
        "\n",
        "# Compiling & training the simple-cnn-model\n",
        "simple_cnn_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n",
        "simple_cnn_model.fit(X, y, epochs=10, batch_size=64)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "C4RmRRcqhMEE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a40fc25c-371d-4bd6-97b2-c97e12f2ce2e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "858/858 [==============================] - 3s 3ms/step - loss: 0.1968 - accuracy: 0.9422\n",
            "Test accuracy: 94.22%\n"
          ]
        }
      ],
      "source": [
        "# Evaluateing the simple-cnn-model\n",
        "test_loss, test_accuracy = simple_cnn_model.evaluate(X, y)\n",
        "print(f'Test accuracy: {test_accuracy * 100:.2f}%')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kRvjZi7iMTN2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5127b23a-dc0c-4154-9b55-a9837748f5b3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Test accuracy (n_estimators=10): 93.81%\n",
            "Test accuracy (n_estimators=50): 97.40%\n",
            "Test accuracy (n_estimators=100): 97.85%\n"
          ]
        }
      ],
      "source": [
        "# The following code was used with the Random Forest model (with 10, 50 and 100 trees) for the top 20 pixels based on importance\n",
        "# (on the test set which replicated the top 20 pixels on the training set)\n",
        "\n",
        "import pandas as pd\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Loading the dataset containing 'label' and the top 20 pixels from the combined dataset\n",
        "data_sign_test_selected = pd.read_csv(\"top_20_pixels_with_labels_train.csv\")\n",
        "\n",
        "# Separating the features and target variable (label)\n",
        "X = data_sign_test_selected.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y = data_sign_test_selected['label']   # Dependent variable\n",
        "\n",
        "# Splitting the top 20 pixels combined data into a training and a test set (80% training, 20% testing)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "\n",
        "# Defining a list of n_estimators values to test (for 3 different values of trees)\n",
        "n_estimators_values = [10, 50, 100]\n",
        "\n",
        "for n_estimators in n_estimators_values:\n",
        "    # Creating a Random Forest Classifier with the current number of trees\n",
        "    rf_classifier = RandomForestClassifier(n_estimators=n_estimators, random_state=0)\n",
        "    # Training the classifier on the training set\n",
        "    rf_classifier.fit(X_train, y_train)\n",
        "    # Making predictions on the test set\n",
        "    y_pred = rf_classifier.predict(X_test)\n",
        "\n",
        "    # Calculating and displaying the test accuracy for the 3 different random forest tree values\n",
        "    accuracy = accuracy_score(y_test, y_pred)\n",
        "    print(f'Test accuracy (n_estimators={n_estimators}): {accuracy * 100:.2f}%')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Hsy-NKZelttZ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 564
        },
        "outputId": "75cd8fdb-ce9b-4bfe-a094-50a7bb4a9d8a"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x600 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAIjCAYAAADvBuGTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB9mElEQVR4nO3dd3hUZfrG8XvSewIkJASQQOhFUBBEECwIgvJTLDRdKVaERcWysCLNVcRdEEVX7LJgV8SOdJQiHQQpUkKRHiAJSUibeX9/hBkyJIGUSWYSvp/rygVz5syZZ4Y3Ye6873mOxRhjBAAAAAAoFS93FwAAAAAAlQHhCgAAAABcgHAFAAAAAC5AuAIAAAAAFyBcAQAAAIALEK4AAAAAwAUIVwAAAADgAoQrAAAAAHABwhUAAAAAuADhCkClMXDgQMXFxbm7DAClcN1116l58+buLqPIZs6cqcaNG8vX11cRERHuLgeAmxGuABTbhx9+KIvF4vjy8fFRzZo1NXDgQB08eNDd5XmM89+nvF8jR450d3kFevHFFzVnzpxiP27btm2yWCwKCAhQUlKSy+uCa8XFxclisejvf/97vvuWLFkii8WiL7/80g2VVSzbt2/XwIEDFR8fr3feeUdvv/12vn327t1b6M+B87/27t1b/i8CgEv5uLsAABXXhAkTVLduXWVkZOi3337Thx9+qGXLlmnLli0KCAhwd3kew/4+5eWpv5l/8cUXddddd+n2228v1uNmzZqlmJgYnTp1Sl9++aUeeOCBsikQLvXOO+9o1KhRio2NdXcpFdKSJUtks9n06quvqn79+gXuExUVpZkzZzptmzx5sv766y+98sor+fYFULERrgCUWPfu3dWmTRtJ0gMPPKDIyEhNmjRJ3377rXr37u3m6jxH3vfJldLS0hQcHOzy4xaXMUYff/yx+vfvr4SEBH300UceG6485T3zBM2aNdOOHTv00ksv6bXXXnN3OeXKZrMpKyur1L8EOnbsmCRdcDlgcHCw7r33Xqdtn376qU6dOpVve17GGGVkZCgwMLBUNQIoXywLBOAy1157rSRp9+7djm1ZWVkaM2aMWrdurfDwcAUHB+vaa6/V4sWLnR5rXzrzn//8R2+//bbi4+Pl7++vq666SmvWrMn3XHPmzFHz5s0VEBCg5s2b6+uvvy6wprS0ND355JOqXbu2/P391ahRI/3nP/+RMcZpP4vFomHDhumLL75Q06ZNFRgYqPbt22vz5s2SpLfeekv169dXQECArrvuOpcu31m0aJGuvfZaBQcHKyIiQrfddpu2bdvmtM+4ceNksVi0detW9e/fX1WqVFHHjh0d98+aNUutW7dWYGCgqlatqr59++rAgQNOx9i5c6fuvPNOxcTEKCAgQLVq1VLfvn2VnJzseA/S0tI0Y8YMxzKlgQMHXrT+5cuXa+/everbt6/69u2rX375RX/99Ve+/ey/4W/RooUCAgIUFRWlm2++WWvXrnXab9asWWrbtq2CgoJUpUoVderUSfPmzXPcb7FYNG7cuHzHj4uLc6rXvixz6dKlevTRR1W9enXVqlVLkrRv3z49+uijatSokQIDA1WtWjXdfffdBf67JiUl6YknnlBcXJz8/f1Vq1Yt3XfffUpMTFRqaqqCg4P12GOP5XvcX3/9JW9vb02cOLHA9y07O1tVq1bVoEGD8t2XkpKigIAAPfXUU45t06ZNU7NmzRzvS5s2bfTxxx8XeOyiiIuL03333ad33nlHhw4duuC+hZ3PaB+Xebnqe2ndunW65pprFBgYqLp162r69On59snMzNTYsWNVv359+fv7q3bt2nrmmWeUmZlZYE0fffSRmjVrJn9/f82dO/eCr/m///2vY9/Y2FgNHTrUaclrXFycxo4dKyl3xqmwcVlUcXFxuvXWW/Xzzz+rTZs2CgwM1FtvvSUpdww+/vjjjp9j9evX16RJk2Sz2ZyOYbPZNHXqVDVr1kwBAQGKjo7Www8/rFOnTjntt3btWnXr1k2RkZGO93fw4MElrh3AOcxcAXAZ+4ekKlWqOLalpKTo3XffVb9+/fTggw/q9OnTeu+999StWzetXr1arVq1cjrGxx9/rNOnT+vhhx+WxWLRyy+/rDvuuEN79uyRr6+vJGnevHm688471bRpU02cOFEnTpzQoEGDHB+c7Ywx+r//+z8tXrxY999/v1q1aqWff/5ZTz/9tA4ePJhvSc6vv/6qb7/9VkOHDpUkTZw4UbfeequeeeYZ/fe//9Wjjz6qU6dO6eWXX9bgwYO1aNGiIr0vycnJSkxMdNoWGRkpSVqwYIG6d++uevXqady4cTpz5oymTZumDh06aP369fk+0N59991q0KCBXnzxRUdAfOGFF/Tcc8+pd+/eeuCBB3T8+HFNmzZNnTp10oYNGxQREaGsrCx169ZNmZmZ+vvf/66YmBgdPHhQ33//vZKSkhQeHq6ZM2fqgQceUNu2bfXQQw9JkuLj4y/6+j766CPFx8frqquuUvPmzRUUFKRPPvlETz/9tNN+999/vz788EN1795dDzzwgHJycvTrr7/qt99+c8zsjR8/XuPGjdM111yjCRMmyM/PT6tWrdKiRYvUtWvXIr3f53v00UcVFRWlMWPGKC0tTZK0Zs0arVixQn379lWtWrW0d+9evfnmm7ruuuu0detWBQUFSZJSU1N17bXXatu2bRo8eLCuvPJKJSYm6ttvv9Vff/2lVq1aqVevXvrss880ZcoUeXt7O573k08+kTFG99xzT4F1+fr6qlevXpo9e7beeust+fn5Oe6bM2eOMjMz1bdvX0m5y/eGDx+uu+66S4899pgyMjL0+++/a9WqVerfv3+J3hdJevbZZ/W///3P5bNXpf1eOnXqlHr06KHevXurX79++vzzzzVkyBD5+fk5QoDNZtP//d//admyZXrooYfUpEkTbd68Wa+88or+/PPPfOcOLlq0SJ9//rmGDRumyMjICza/GTdunMaPH68uXbpoyJAh2rFjh958802tWbNGy5cvl6+vr6ZOnar//e9/+vrrr/Xmm28qJCREl19+eanetx07dqhfv356+OGH9eCDD6pRo0ZKT09X586ddfDgQT388MO67LLLtGLFCo0aNUqHDx/W1KlTHY9/+OGH9eGHH2rQoEEaPny4EhIS9Prrr2vDhg2Ouo8dO6auXbsqKipKI0eOVEREhPbu3avZs2eXqnYAZxkAKKYPPvjASDILFiwwx48fNwcOHDBffvmliYqKMv7+/ubAgQOOfXNyckxmZqbT40+dOmWio6PN4MGDHdsSEhKMJFOtWjVz8uRJx/ZvvvnGSDLfffedY1urVq1MjRo1TFJSkmPbvHnzjCRTp04dx7Y5c+YYSeZf//qX0/PfddddxmKxmF27djm2STL+/v4mISHBse2tt94ykkxMTIxJSUlxbB81apSR5LTvhd6ngr7yvpbq1aubEydOOLZt2rTJeHl5mfvuu8+xbezYsUaS6devn9Nz7N2713h7e5sXXnjBafvmzZuNj4+PY/uGDRuMJPPFF19csObg4GAzYMCAC+6TV1ZWlqlWrZp59tlnHdv69+9vWrZs6bTfokWLjCQzfPjwfMew2WzGGGN27txpvLy8TK9evYzVai1wH2Ny/63Gjh2b7zh16tRxqt3+/nfs2NHk5OQ47Zuenp7v8StXrjSSzP/+9z/HtjFjxhhJZvbs2YXW/fPPPxtJ5qeffnK6//LLLzedO3fO97i87I/NO76NMaZHjx6mXr16jtu33Xabadas2QWPVRx16tQxt9xyizHGmEGDBpmAgABz6NAhY4wxixcvzjdWBgwY4PS9ZWcfl3mV9nupc+fORpKZPHmyY1tmZqbjeyUrK8sYY8zMmTONl5eX+fXXX52ef/r06UaSWb58uVNNXl5e5o8//rjoe3Ps2DHj5+dnunbt6jQOX3/9dSPJvP/++/le//Hjxy963LxuueWWfO9nnTp1jCQzd+5cp+3PP/+8CQ4ONn/++afT9pEjRxpvb2+zf/9+Y4wxv/76q5FkPvroI6f95s6d67T966+/NpLMmjVrilUzgKJhWSCAEuvSpYuioqJUu3Zt3XXXXQoODta3337rNIPk7e3t+I28zWbTyZMnlZOTozZt2mj9+vX5jtmnTx+nmS/7UsM9e/ZIkg4fPqyNGzdqwIABCg8Pd+x30003qWnTpk7H+vHHH+Xt7a3hw4c7bX/yySdljNFPP/3ktP3GG290+m12u3btJEl33nmnQkND822313Qxb7zxhubPn+/0lfe1DBw4UFWrVnXsf/nll+umm27Sjz/+mO9YjzzyiNPt2bNny2azqXfv3kpMTHR8xcTEqEGDBo7ll/b36ueff1Z6enqR6i6Kn376SSdOnFC/fv0c2/r166dNmzbpjz/+cGz76quvZLFYHMuo8rIvK5szZ45sNpvGjBkjLy+vAvcpiQcffNBpRkmS03ks2dnZOnHihOrXr6+IiAincfnVV1+pZcuW6tWrV6F1d+nSRbGxsfroo48c923ZskW///77Bc+pkaQbbrhBkZGR+uyzzxzbTp06pfnz56tPnz6ObREREfrrr78KXCJbWqNHj1ZOTo5eeukllx2ztN9LPj4+evjhhx23/fz89PDDD+vYsWNat26dJOmLL75QkyZN1LhxY6exf8MNN0hSvqXHnTt3zvczoiALFixQVlaWHn/8cadx+OCDDyosLEw//PBDUd6CEqlbt666devmtO2LL77QtddeqypVqji9zi5dushqteqXX35x7BceHq6bbrrJab/WrVsrJCTE8X7Yzw/7/vvvlZ2dXWavBbhUEa4AlJg9NHz55Zfq0aOHEhMT5e/vn2+/GTNm6PLLL1dAQICqVaumqKgo/fDDD45zffK67LLLnG7bg5b9nIF9+/ZJkho0aJDvsY0aNXK6vW/fPsXGxjp9mJOkJk2aOB2rsOe2B5LatWsXuP388xgK07ZtW3Xp0sXpK+/zn1+3vcbExETHMja787sO7ty5U8YYNWjQQFFRUU5f27Ztc5xwX7duXY0YMULvvvuuIiMj1a1bN73xxhsF/hsUx6xZs1S3bl35+/tr165d2rVrl+Lj4xUUFOQUNnbv3q3Y2FinEHm+3bt3y8vLq0gfgIvj/PdMks6cOaMxY8Y4zmGJjIxUVFSUkpKSnN6T3bt3X7Szo5eXl+655x7NmTPHEVw/+ugjBQQE6O67777gY318fHTnnXfqm2++cZwnNHv2bGVnZzuFq3/84x8KCQlR27Zt1aBBAw0dOlTLly8v8ntwIfXq1dPf/vY3vf322zp8+LBLjlna76XY2Nh8jUcaNmwo6dzy4507d+qPP/7IN+7t+9nHvl1B46AghX1f+vn5qV69evl+brhSQTXu3LlTc+fOzfc67T9H7K9z586dSk5OVvXq1fPtm5qa6tivc+fOuvPOOzV+/HhFRkbqtttu0wcffJDvPDUAJcM5VwBKrG3bto5zZW6//XZ17NhR/fv3144dOxQSEiIp98P3wIEDdfvtt+vpp59W9erVHSf55218YXf+DIOdOa8BRVko7LndWdP5zu8cZrPZZLFY9NNPPxVYp/3fQcpt/zxw4EB98803mjdvnoYPH66JEyfqt99+y3e+WlGkpKTou+++U0ZGRoFh9+OPP9YLL7xQqlmn4rBarQVuL6jb2t///nd98MEHevzxx9W+fXuFh4fLYrGob9+++ZoEFMV9992nf//735ozZ4769eunjz/+WLfeeqvT7Gph+vbtq7feeks//fSTbr/9dn3++edq3LixWrZs6dinSZMm2rFjh77//nvNnTtXX331lf773/9qzJgxGj9+fLHrPd+zzz6rmTNnatKkSQW24S/s37Cw97w8vpdsNptatGihKVOmFHj/+UGuInTdK6hGm82mm266Sc8880yBj7GHSZvNpurVqzv9UiMve5t3+zXMfvvtN3333Xf6+eefNXjwYE2ePFm//fab088MAMVHuALgEvbAdP311+v11193XCT3yy+/VL169TR79mynD2gFLQ8rijp16kjK/S3t+Xbs2JFv3wULFuj06dNOs1fbt293Opa72J///Lql3BojIyMv2jY8Pj5exhjVrVvX8SHrQlq0aKEWLVpo9OjRWrFihTp06KDp06frX//6l6TiLb+bPXu2MjIy9OabbzoadNjt2LFDo0eP1vLly9WxY0fFx8fr559/1smTJwudvYqPj5fNZtPWrVvzNTrJq0qVKvkuVJyVlVWsWZcvv/xSAwYM0OTJkx3bMjIy8h03Pj5eW7ZsuejxmjdvriuuuEIfffSRatWqpf3792vatGlFqqVTp06qUaOGPvvsM3Xs2FGLFi3Ss88+m2+/4OBg9enTR3369FFWVpbuuOMOvfDCCxo1alSpW4rHx8fr3nvv1VtvveVYqpdXQe+5lH/211UOHTqUr23+n3/+KUmO5Ybx8fHatGmTbrzxRpcG+Lzfl/Xq1XNsz8rKUkJCgmPGqLzEx8crNTX1os8bHx+vBQsWqEOHDkUKkldffbWuvvpqvfDCC/r44491zz336NNPP/XYyygAFQXLAgG4zHXXXae2bdtq6tSpysjIkHTuN9V5fzO9atUqrVy5skTPUaNGDbVq1UozZsxwWr41f/58bd261WnfHj16yGq16vXXX3fa/sorr8hisah79+4lqsFV8r6WvB9ct2zZonnz5qlHjx4XPcYdd9whb29vjR8/Pt9v/40xOnHihKTcWaacnByn+1u0aCEvLy+n5UDBwcEFfoguyKxZs1SvXj098sgjuuuuu5y+nnrqKYWEhDh+i37nnXfKGFPgLIu97ttvv11eXl6aMGFCvtmjvK8tPj7ecZ6J3dtvv13oLEpBvL29871f06ZNy3eMO++8U5s2bSqw1f/5j//b3/6mefPmaerUqapWrVqRx5eXl5fuuusufffdd5o5c6ZycnKclgRKcvw72vn5+alp06YyxjjOm0lPT9f27dvzdaYsqtGjRys7O1svv/xyvvvi4+OVnJys33//3bHt8OHDhV4CobRycnIcbcil3GDz1ltvKSoqSq1bt5Yk9e7dWwcPHtQ777yT7/FnzpzJt6S2qLp06SI/Pz+99tprTv/G7733npKTk3XLLbeU6Lgl1bt3b61cuVI///xzvvuSkpIc39e9e/eW1WrV888/n2+/nJwcx/f1qVOn8o1d+y8zWBoIlB4zVwBc6umnn9bdd9+tDz/8UI888ohuvfVWzZ49W7169dItt9yihIQETZ8+XU2bNlVqamqJnmPixIm65ZZb1LFjRw0ePFgnT550XAMo7zF79uyp66+/Xs8++6z27t2rli1bat68efrmm2/0+OOPF6nNeFn797//re7du6t9+/a6//77Ha3Yw8PDi3TNnPj4eP3rX//SqFGjtHfvXt1+++0KDQ1VQkKCvv76az300EN66qmntGjRIg0bNkx33323GjZsqJycHM2cOVPe3t668847Hcdr3bq1FixYoClTpig2NlZ169YtcCbj0KFDWrx4cb5mIXb+/v7q1q2bvvjiC7322mu6/vrr9be//U2vvfaadu7cqZtvvlk2m02//vqrrr/+eg0bNkz169fXs88+q+eff17XXnut7rjjDvn7+2vNmjWKjY11XC/qgQce0COPPKI777xTN910kzZt2qSff/453+zZhdx6662aOXOmwsPD1bRpU61cuVILFixQtWrVnPZ7+umn9eWXX+ruu+/W4MGD1bp1a508eVLffvutpk+f7rR0r3///nrmmWf09ddfa8iQIY5LBxRFnz59NG3aNI0dO1YtWrRwnBdo17VrV8XExKhDhw6Kjo7Wtm3b9Prrr+uWW25xzMquXr1a119/vcaOHVui6y3ZZ69mzJiR776+ffvqH//4h3r16qXhw4crPT1db775pho2bFhgY5rSio2N1aRJk7R37141bNhQn332mTZu3Ki3337b8b7+7W9/0+eff65HHnlEixcvVocOHWS1WrV9+3Z9/vnnjutFFVdUVJRGjRql8ePH6+abb9b//d//aceOHfrvf/+rq6666qJNSlzt6aef1rfffqtbb71VAwcOVOvWrZWWlqbNmzfryy+/1N69exUZGanOnTvr4Ycf1sSJE7Vx40Z17dpVvr6+2rlzp7744gu9+uqruuuuuzRjxgz997//Va9evRQfH6/Tp0/rnXfeUVhYWJF+oQPgIsq9PyGACs/e4rqgVr5Wq9XEx8eb+Ph4k5OTY2w2m3nxxRdNnTp1jL+/v7niiivM999/n6+1s70V+7///e98x1QBrbe/+uor06RJE+Pv72+aNm1qZs+eXWC76NOnT5snnnjCxMbGGl9fX9OgQQPz73//26m1t/05hg4d6rStsJoKalVd3PcprwULFpgOHTqYwMBAExYWZnr27Gm2bt3qtM/FWj5/9dVXpmPHjiY4ONgEBwebxo0bm6FDh5odO3YYY4zZs2ePGTx4sImPjzcBAQGmatWq5vrrrzcLFixwOs727dtNp06dTGBgoJFUaFv2yZMnG0lm4cKFhb6uDz/80Egy33zzjTEmty3/v//9b9O4cWPj5+dnoqKiTPfu3c26deucHvf++++bK664wvj7+5sqVaqYzp07m/nz5zvut1qt5h//+IeJjIw0QUFBplu3bmbXrl2FtmIv6P0/deqUGTRokImMjDQhISGmW7duZvv27fmOYYwxJ06cMMOGDTM1a9Y0fn5+platWmbAgAEmMTEx33F79OhhJJkVK1YU+r4UxGazmdq1axd46QBjcluZd+rUyVSrVs34+/ub+Ph48/TTT5vk5GTHPvZxWVCb+vPlbcWe186dO423t3eB43vevHmmefPmxs/PzzRq1MjMmjWr0Fbspfle6ty5s2nWrJlZu3atad++vQkICDB16tQxr7/+er56s7KyzKRJk0yzZs0c46V169Zm/PjxTu9NQTVdzOuvv24aN25sfH19TXR0tBkyZIg5deqU0z6ubsVe0L+JMbk/x0aNGmXq169v/Pz8TGRkpLnmmmvMf/7zH0dreru3337btG7d2gQGBprQ0FDTokUL88wzzzha7a9fv97069fPXHbZZcbf399Ur17d3HrrrWbt2rXFeg0ACmYxxg1nZAMAUAn16tVLmzdv1q5du9xdCgDADTjnCgAAFzh8+LB++OEH/e1vf3N3KQAAN+GcKwAASiEhIUHLly/Xu+++K19fX6eL3wIALi3MXAEAUApLly7V3/72NyUkJGjGjBmKiYlxd0kAADfhnCsAAAAAcAFmrgAAAADABQhXAAAAAOACNLQogM1m06FDhxQaGiqLxeLucgAAAAC4iTFGp0+fVmxsrLy8Ljw3RbgqwKFDh1S7dm13lwEAAADAQxw4cEC1atW64D6EqwKEhoZKyn0Dw8LCXHLM7OxszZs3T127dpWvr69LjolLB+MHpcH4QUkxdlAajB+UhieNn5SUFNWuXduRES6EcFUA+1LAsLAwl4aroKAghYWFuX2AoOJh/KA0GD8oKcYOSoPxg9LwxPFTlNOFaGgBAAAAAC5AuAIAAAAAFyBcAQAAAIALEK4AAAAAwAUIVwAAAADgAoQrAAAAAHABwhUAAAAAuADhCgAAAABcgHAFAAAAAC5AuAIAAAAAFyBcAQAAAIALEK4AAAAAwAUIVwAAAADgAoQrAKjErDajVQkntS7RolUJJ2W1GXeXBOASwM8elEZFHj8+7i4AAFA25m45rPHfbdXh5AxJ3vrfzrWqER6gsT2b6ubmNdxdHoBKip89KI2KPn6YuQKASmjulsMaMmv92f+czjmSnKEhs9Zr7pbDbqoMQGXGzx6URmUYP8xcAUAlY7UZjftuqwpaRGEkWSSN/26rbmoaI28vSzlXB1QOVpvR6oSTOnY6Q9VDA9S2btVL6vvJGKPMHJvSMnOUlmlVWlaOUs5ka9TszYX+7JGkZ7/eohrhgQoL9FWwv7eC/XwU5Octi+XSee8kxo8xRulZueMmLdOqtMyLj5+K8n8X4QoAKrik9CxtP3Jafx49re1HTmvt3pM6ct5v/fIykg4nZ+jhmWvVPj5S9aKCVS8yWDUjAuXjzYIG4GKcly3l8vRlS+eHodTMnLMfbM99uE3NPHs7y3p2e+629CzrufvybMspwXkwJ9KydNsby522WSxSsJ+PI2wF++f+PcQ/9+9Bfj4K8fdWsL+PQs7eznt/7ra8+3t2WKuo4yfdPi6ynMeLfTxcaAzZA3jeMG6KOXzs/3etTjip9vHVyuR1ugLhCgAqiIxsq3YdS9X2I6e140iKI1AdTcks0fEWbDumBduOOW77eltUp1qw6kbmhq16UcGqGxmiupHBigzx8+gPK0B5sS9bOv9zoX3Z0pv3XumSD8j2MJSamaP0PGHI/uHVsS0zR6lZubcdH2SzcpSaaVV63g+3WdYyawoQ6JsbfCSjxNSsi+4fHugrm80o9ewHbGOk1LN1SiX7eZaXxSIF+Z4LY/awdi645Ya13NDmHObObTt7299HQb7e8nLRTEl5jp+8gSdvQLaHnfSsvAEp97ZTGHIE8JKFoaLIG6yNkY6dvvi//7HThf/y0BMQrgDAw1htRvtOpGnHkdPacfR07p9HTmvviTQV9tmoZkSgGseEqlFMqLws0uuLd1/0eW5rGassq00JiWlKSExTZo5Nu46latex1Hz7hgb4qF7k2eAVFXL2z9zbQX78V4JLg9VmNP4iS27HfvuHGkaH6ky2Nd+Mj2OG6LwZI3sYyg1NZR+Ggvxyg0ew/c8CZn8KCyD2gGIPIEF+Po4lWit3n1C/d3676PNPv7e12sdXkzFGZ7KtzgEy7wf6Amc98rxXWfmDgO1sWEvLsioty1qkD+sXc35YCzr7PhT+vuTfFuTvrUBfb4399o+Ljp/4qBBlZNscrzl/2Dn3HhT0ftmX25V1GHK83jy3izuWAvME16KOn+qhAa5/YS7E/4gA4CbGGB07nekIT/aZqJ3HTisj21bgY6oE+apRTKgaRYeqUUyYGsWEqmF0iEIDfB37WG1GX60/qCPJGQX+J26RFBMeoCl9Wjk+FNlsRoeSzyghMU17jueGrd3HU5WQmKaDSWd0OiNHm/5K1qa/kvMdLyYswBG06kYGK/5s+KpVhWWGqFwW7ziW70T7vIykoymZumHyUpc+rz0M5Q00Ied9aLX/vbClc/YPwHnDkKu1rVtVNcIDLvqzp23dqrm3LRYF+eXWqtDSP78xRhnZtgsuVyvuErayCGuF1q/c8XPTK7+47JgWixRydnwE2cdDnvESVGhAcp7ps4+lQN+yW3JZ3PHjqdwert544w39+9//1pEjR9SyZUtNmzZNbdu2LXDf7OxsTZw4UTNmzNDBgwfVqFEjTZo0STfffLNjH6vVqnHjxmnWrFk6cuSIYmNjNXDgQI0ePZolLQDc5nRGtuOcqD/PBqkdR08rKT27wP39fbzUMDp3JqpxTKgaRuf+GRXqf9GfZd5eFo3t2VRDZq2XRXL6T8r+yLE9mzp9wPLysqhWlSDVqhKkaxtEOR0vI9uqfSfSlZCYqj15wldCYppOpmXpSEqGjqRkaMXuE06P8/W26LKqQaobGaL4POGrXlQIywzh0TJzcpfgnj97fKFglZe/t0XhQX5Ov6G3LzELKWBGI+8HXqcZARcvSStrJfnZ40oWi0WBft4K9PNWVKh/qY+XN6ydP0t2blv+GSSn2co8QS41I6fA0HC+AB8vRQT5OS1VvGh4LmT2rCzDkKu5e/y4ilvD1WeffaYRI0Zo+vTpateunaZOnapu3bppx44dql69er79R48erVmzZumdd95R48aN9fPPP6tXr15asWKFrrjiCknSpEmT9Oabb2rGjBlq1qyZ1q5dq0GDBik8PFzDhw8v75cI4BKTlWPTnsRUx0yU/UPZwaQzBe7vZZHiIoOdAlSjmDBdVjWoVP+B3Ny8ht6898p8J03HlOCk6QBf79zZspj8v1pOSs/SnsQ0JRxP057EVKeZr8wcm3YfT9Pu42lasM35caH+Pqp7tpFG3ciQPH8PPnvuBlD2bDajA6fSz32vng1SCYlppVqS9+Hgdh59wn1ZcuXPHnfLG9ak0oe1lbsT1e+dVRfd74NBbRk/FXj8WIwpixWZRdOuXTtdddVVev311yVJNptNtWvX1t///neNHDky3/6xsbF69tlnNXToUMe2O++8U4GBgZo1a5Yk6dZbb1V0dLTee++9Qvc5X2ZmpjIzz03zpqSkqHbt2kpMTFRYWJhLXmt2drbmz5+vm266Sb6+vhd/AJAH48fz2GxGB5PP6M8jqfrzWKp2HE3Vn0dPKyExvdAOWtGh/moYHaKG0SFqFJ27nC8+KlgBvt5lVqfVZvTb7uNatHKdbmjfWlfHR5XLb/1sNqMjKRlKOJF+dpYrXXtPpGlPYroOJp254LkA0aH+qhsZpLjIYNWtFnR2xitItVzczdBqM1q775SOnc5U9VB/talTxeN/I1reKtPPnsTUzLPfp/av09p5LFVnClmCGx7oo4bRoWpY3f49m/v92vONlTqaknmBZUv+Wjyi0yU/ltz1s8eTWW1G103+hfFTBJ42flJSUhQZGank5OSLZgO3hausrCwFBQXpyy+/1O233+7YPmDAACUlJembb77J95hq1arp5Zdf1v333+/Ydu+992rZsmXau3evJOnFF1/U22+/rXnz5qlhw4batGmTunbtqilTpuiee+4psJZx48Zp/Pjx+bZ//PHHCgoKKt0LBVDhpWZLh9MtOpRu/9OiI+lSpq3gH/QB3kY1gqQaQUaxQUY1goxqBErBFfuzqctk26TEDOnYGYuOn/3zWIZFx89IqTmF/+fpZTGK9JeqBxpVD5CiAo2qBxhVD5RCfXPPLSiqTScsmr3XS0lZ5x4U4Wd0R5xNLau57XeOcIFMq3Q4z/eq/e+FjS0fi1FM3u/XwNy/h/sVPKY2nbDo/T/tIT/vDrnjZnBDxhAKx/ipmNLT09W/f/8ihSu3rb1ITEyU1WpVdHS00/bo6Ght3769wMd069ZNU6ZMUadOnRQfH6+FCxdq9uzZslqtjn1GjhyplJQUNW7cWN7e3rJarXrhhRcKDVaSNGrUKI0YMcJx2z5z1bVrV2au4BEYP4Vz5ezDmSyrdp6dhcr987T+PJpaaGthX2+L4iODc3+7HR2iRjEhalg9RDXCAzxqjXtFGj/JZ7KVkJimvSfStScxTXsT05VwInfWKyPbpmMZ0rGM/O9tsL+36lbLneGqWy1YcZFBqhcZrDrVghRy3jLDn/84qg9Wbsr3m+PkLIs++NNb0/q2VLdm0YJnj51sq017E9PPzhznfq/uOJqqv04VvATXYpHqVA1Sg+q5s1D2GeTLqhZvRrSHpCv/OKp//bhdR/JcBqFGeICe7d6YsZOHJ48fd2H8FJ0njZ+UlJQi71uhFra/+uqrevDBB9W4cWNZLBbFx8dr0KBBev/99x37fP755/roo4/08ccfq1mzZtq4caMef/xxxcbGasCAAQUe19/fX/7++dfS+vr6uvwfsyyOiUsH48dZSS/EmGO1ae+JNO04kup0vah9J9MLXa52WdWgPF36cs+NiosMlm8F6oZXEcZPpK+vIsOCdFU95+2OZYaJadpzPLexhv38rr9OpSst06oth1K05VD+/wCjw/zPLi0MUd3IIL25ZPcFWyG/8NMOdb+85iW/LCcvd44dY4wOJWec+149ez7jnuNpyrIWvKQvKtQ/9/zF6FA1PPv92qB66NlzZ0rv1la11P3ymlqdcFLHTmeoemhuBzPGTMEqws+e8sT4KR5PGD/FeX63havIyEh5e3vr6NGjTtuPHj2qmJiYAh8TFRWlOXPmKCMjQydOnFBsbKxGjhypevXO/S/89NNPa+TIkerbt68kqUWLFtq3b58mTpxYaLgCUPEU5UKM3ZrF6EhKhuNkdfuHsl3HU5WVU/CHsmrBfo7mDfYmEw2jQ2my4GZeXhbFRgQqNiJQHepHOt2XmWPVgZPp2m3vYpinuUZiapaOpmTqaEqmfttz8qLPYyQdTs7Q64t26up61VQl2E8RQb6qEuRXoYJ0RZWUnuX4ZUfe79vTmTkF7h/s5+0IT3kvT1A12K/Ma/X2slyyTQdQeoyfysttnxb8/PzUunVrLVy40HHOlc1m08KFCzVs2LALPjYgIEA1a9ZUdna2vvrqK/Xu3dtxX3p6ury8nP8D9Pb2ls1W8AcpABXPxS7kKUnDP9kgfx8vnc60FrCXFOib+6GsUXSIGsWEOS7AGxlS+o5QKF/+Pt6qXz1U9avn72ZoX2aYkJiqhONp+mVnojYeSLroMV9ZsFPSTqdtIf4+qhKcG7QigvxUJcj+d19VDc6/rUqQn4L8Kk4b5PKUkZ3b6jw3QKVox9HcWeSjKQVfQ8jHy6L4qBDHLz7sM8i1qgTy/gLwKG79VeyIESM0YMAAtWnTRm3bttXUqVOVlpamQYMGSZLuu+8+1axZUxMnTpQkrVq1SgcPHlSrVq108OBBjRs3TjabTc8884zjmD179tQLL7ygyy67TM2aNdOGDRs0ZcoUDR482C2vEYBrZOXYdDj5jP46dUZL/7zwhTwlKctqlGW1ytvLorqRwbkzUXmWCNWuElRhrh2DkgsP9FWr2hFqVTtCktQ+PlL93vntoo9rUD1YVpt0Kj1LSWeyZYxyr1OTmaMDJws+p6cgfj5e+QJXRJCfqp4X0ux/Vg32U1iAr0eMTavNaFXCSa1LtKhawkm1r1+92MuWrDajfSfSnGaidhw5rb0n0lRYp/OaEYGOX3bkziCHqW5ksPx8mDkE4PncGq769Omj48ePa8yYMTpy5IhatWqluXPnOppc7N+/32kWKiMjQ6NHj9aePXsUEhKiHj16aObMmYqIiHDsM23aND333HN69NFHdezYMcXGxurhhx/WmDFjyvvlASiGvOHpr1PpZ/889/cjKRkXbN9dkGe6NdL919aVv0/ZtTpHxdK2blXVCA/QkeSMC7RCDtDcxzs7goTNZpSSka2TaVk6lZ6tpPS8f2bpZNq5vyelZ+tUepZOpWUry2pTVo7NsSyxqLwsuaEwbyCrEpw3hJ37e9U8210ZPpzPZ/TW/3auveD5jMYYHT+dme96UTuPnVZGIa3OqwT5OsKT/YLZDaNDFBrAuTkAKi63n0QwbNiwQpcBLlmyxOl2586dtXXr1gseLzQ0VFOnTtXUqVNdVCEAV8jMsepwUsZ54Sn3mkdFDU/+Pl6qVSVQIf4+2vRX8kWf84rLqhCs4MTby6KxPZtqyKz1skhOAcs+JzO2Z1OnGRovL4sizs4yFZUxRulZVufAlZ6tU2lZ+bYlpWfpZFruttTMHNmMcvdNzy7Wawv2884NX2dnxaqcNyuWG9D8zoW2YD8FF7Bs8WLnM07p3VKXVQs+Owt1riFMYfUG+HqpQfVz5zHal/VFhfqzpA9ApeP2cAWgcigsPNlnoI6eLnp4qlUl6Lw/c/8eGeIni8Uiq82o46RFF519aFu3alm8VFRwNzevoTfvvTJfp8mYInSaLCqLxaJgfx8F+/uoVpWiPy4rx6akM2fD19kgdupsEDu3zR7McrclpWfJZqS0LKvSss7oYFIxli16e+VZruiriEBf/bIr8YLnMz7x+aYCj+VlkeIigx2NYHKDVJguqxpEFzQAlwzCFYAicUV4CvD1yheY8v5ZLdivSL/JLsnsA5DXzc1r6KamMR7XCtnPx0vVQwNUPTSgyI+x2YxOZ+TkLlFMz8pdopiWfd4MWf5tmTk2ZVltOnY6U8dOF33ZopS7pK9FrYg8XfpCVb96iAJ8mSkGcGkjXAEezhUnlRdFZo5Vh5IyHIHpoBvDU1GUx+wDKrfK0grZy8ui8CBfhQf5Kk7BRX7cmSyrTqZn6VTaucD1y5/H9cW6vy762HH/10y3tapZmrIBoFIiXAEerLgnlV/I+eHp/KYRx05nelR4KgpPnX0AKoJAP2/V9AtUzYhAx7bIEP8ihavizKwBwKWEcAV4qKJcJDdvwLpYeCpKt7JAX+9Cg1OtKoGqWs7hqSgqy+wD4AmK2k2R8xkBoGCEK8ADFeUiuU99sUk/bj6sg2cDVWUNTwDKD+czAkDpEK4AD7Q64eRFL5KbmmnVt5sOO20jPAEoLc5nBICSI1wBHujY6QsHK7vbWsWqW7MYR4iqEuRLeAJQavbzGVfuOqZ5v65S12vblVkzHQCoTAhXgIex2ozW7T1VpH37XnUZ5xsBKBPeXha1q1tVJ7YZtaNRDAAUCeEK8CCb/0rWP7/erM0Hky+4HyeVAwAAeB7CFeABUjNzNHneDs1YsVc2I4UF+Khny1h9vGq/JE4qBwAAqAgIV4Cb/fzHEY395g8dSck9z+q2VrEafUtTRYX669oGkZxUDgAAUEEQrgA3OZh0RmO/+UMLth2VJF1WNUj/ur25OjWMcuzDSeUAAAAVB+EKKGc5Vps+XLFXU+b/qfQsq3y8LHq4cz39/YYGCvD1zrc/J5UDAABUDIQroBz9/leSRs3erD8OpUiS2tSpohfvaKGG0aFurgwAAAClRbgCysHpjGxNnven/rcyt2FFeKCvRnVvrN5tasuLmSgAAIBKgXAFlCFjTG7Dim//0NGUTElSrytq6tlbmigyxN/N1QEAAMCVCFdAGfnrVLrGffuHFmw7JkmKqxakf93eQh0bRLq5MgAAAJQFwhXgYjlWmz5Yntuw4ky2Vb7eFj3SOV5Dr69fYMMKAAAAVA6EK8CFNh5I0j9nb9bWw7kNK9rGVdWLdzRX/eo0rAAAAKjsCFeAC6RkZGvyzzv0v9/2yRgpIshX/+zeRHe1rkXDCgAAgEsE4QooBWOMftpyROO/O9ew4o4ra+rZHk1UjYYVAAAAlxTCFVBCB06ma+y3f2jR9tyGFXUjg/XC7c11TX0aVgAAAFyKCFdAMWVbbXp/WYKmLtjpaFgx5Lr6evS6eBpWAAAAXMIIV0AxrN9/Sv+cvVnbj5yWJLWtW1Uv9mqh+tVD3FwZAAAA3I1wBRRBSka2/j13h2atOtew4tkeuQ0rLBYaVgAAAIBwBVyQMUY/bj6icd/9oeOncxtW3HllLf2zR2MaVgAAAMAJ4QooxIGT6Xrumy1asuO4JKleZLD+1au5romnYQUAAADyI1wB58m22vTesgRNXfCnMrJt8vP20qPXx2vIdfHy96FhBQAAAApGuALyWLfvlJ79+lzDiqvrVdULvVooPoqGFQAAALgwwhUgKflMtl6eu10fr94vY6QqQb4afUtT3XFlTRpWAAAAoEgIV7ikGWP0/e+HNf67rUpMzW1YcXfrWhrVo4mqBvu5uToAAABUJIQrXLL2n0jX6G+26Jc/cxtWxEcF64VeLXR1vWpurgwAAAAVEeEKl5xsq03v/LpHry7Yqcwcm/x8vDTs+vp6uHM9GlYAAACgxAhXuKSs23dS/5y9RTuO5jasuCa+mv51e3PVo2EFAAAASolwhUtCcnq2Jv28XR+v2i9Jqhrsp9G3NFGvK2hYAQAAANcgXKFSM8bo202H9Pz3W5WYmiVJ6tOmtkZ2b6wqNKwAAACACxGuUGntO5Gm0XO26NediZKk+tVD9MLtzdWOhhUAAAAoA4QrVDpZObkNK15beK5hxfAb6uuhTvHy8/Fyd3kAAACopAhXqFTW7D2pf87erJ3HUiVJHetH6l+3N1dcZLCbKwMAAEBlR7hCpZCUnqVJc7frk9UHJEnVgv303K1NdVurWBpWAAAAoFwQrlChFdSwol/b2vrHzY0VEUTDCgAAAJQfwhUqrL2JuQ0rlu3KbVjRoHqIXryjha6Kq+rmygAAAHApIlyhwsnKsentX3brtUW7lJVjk7+Pl4bf2EAPXluPhhUAAABwG8IVKpTVCSf1z683a9fZhhXXNshtWFGnGg0rAAAA4F6EK1QISelZmvjjdn22NrdhRWRIbsOK/2tJwwoAAAB4BsIVPJoxRnM2HtS/vt+mE2n2hhWXaeTNjRUe5Ovm6gAAAIBzCFdwO6vNaHXCSR07naHqoQFqW7eqvL0sSkhM0+g5m7V81wlJUsPoEL3Yq4Xa0LACAAAAHohwBbeau+Wwxn+3VYeTMxzbYsL8dVVcVf289aiycmwK8M1tWPFARxpWAAAAwHMRruA2c7cc1pBZ62XO234kJVPf/X5YktSpYZT+dVtzXVYtqPwLBAAAAIqBcAW3sNqMxn+3NV+wyisiyFfvD2gjH29mqwAAAOD5+NQKt1idcNJpKWBBktKztWbvqXKqCAAAACgdwhXc4tjpCwer4u4HAAAAuBvhCm5RPTTApfsBAAAA7ka4glu0rVtVNcIDVNjlfy2SaoTntmUHAAAAKgLCFdzC28uisT2bFtjQwh64xvZsKm+vwuIXAAAA4FkIV3Cbm5vXUPfmMfm2x4QH6M17r9TNzWu4oSoAAACgZGjFDrfadyJdkvTodfFqFBOq6qG5SwGZsQIAAEBFQ7iC2xxJztDWwymyWKT7O9ZVtRB/d5cEAAAAlBjLAuE2i3cckyS1qh1BsAIAAECFR7iC2yzanhuubmhU3c2VAAAAAKVHuIJbZOZYtXxXoiTp+saEKwAAAFR8hCu4xao9J5WeZVV0mL+axYa5uxwAAACg1AhXcAv7ksDrG1WXxUJnQAAAAFR8hCuUO2OMo5kFSwIBAABQWRCuUO72JKZp34l0+Xl7qWP9SHeXAwAAALgE4QrlbvHZJYHt6lVVsD+XWgMAAEDlQLhCuVu47dz5VgAAAEBlQbhCuUrJyNaavSclSTdwvhUAAAAqEcIVytWynYnKsRnViwxWXGSwu8sBAAAAXIZwhXLlaMHOrBUAAAAqGcIVyo3NZrTkbAt2lgQCAACgsiFcodxsPpisxNQshfj76Kq4qu4uBwAAAHApwhXKjX1JYMf6kfLzYegBAACgcuETLsrNYpYEAgAAoBIjXKFcHDudod//SpYkXdc4ys3VAAAAAK5HuEK5WLLjuCSpRc1wVQ8NcHM1AAAAgOsRrlAuFtOCHQAAAJUc4QplLivHpl93JkrifCsAAABUXoQrlLm1e08qNTNHkSF+urxmuLvLAQAAAMoE4Qplzt6CvXPD6vLysri5GgAAAKBsEK5Q5hadbcF+YxOWBAIAAKDyIlyhTO07kaY9x9Pk42VRxwaR7i4HAAAAKDOEK5Qp+5LAq+KqKizA183VAAAAAGWHcIUyZQ9XdAkEAABAZUe4QplJy8zRqj0nJXF9KwAAAFR+hCuUmeW7EpVltemyqkGKjwp2dzkAAABAmfKIcPXGG28oLi5OAQEBateunVavXl3ovtnZ2ZowYYLi4+MVEBCgli1bau7cuU77xMXFyWKx5PsaOnRoWb8U5LF4x7klgRYLLdgBAABQubk9XH322WcaMWKExo4dq/Xr16tly5bq1q2bjh07VuD+o0eP1ltvvaVp06Zp69ateuSRR9SrVy9t2LDBsc+aNWt0+PBhx9f8+fMlSXfffXe5vCZIxhgt3n5cEksCAQAAcGlwe7iaMmWKHnzwQQ0aNEhNmzbV9OnTFRQUpPfff7/A/WfOnKl//vOf6tGjh+rVq6chQ4aoR48emjx5smOfqKgoxcTEOL6+//57xcfHq3PnzuX1si55Ww+n6EhKhgJ9vdWublV3lwMAAACUOR93PnlWVpbWrVunUaNGObZ5eXmpS5cuWrlyZYGPyczMVEBAgNO2wMBALVu2rNDnmDVrlkaMGFHo0rTMzExlZmY6bqekpEjKXYKYnZ1drNdUGPtxXHU8T7fgjyOSpGviq8pbNmVn29xcUcV2qY0fuBbjByXF2EFpMH5QGp40fopTg1vDVWJioqxWq6Kjo522R0dHa/v27QU+plu3bpoyZYo6deqk+Ph4LVy4ULNnz5bVai1w/zlz5igpKUkDBw4stI6JEydq/Pjx+bbPmzdPQUFBRX9BRWBfoljZfb3ZW5JFkVlH9OOPP7q7nErjUhk/KBuMH5QUYwelwfhBaXjC+ElPTy/yvm4NVyXx6quv6sEHH1Tjxo1lsVgUHx+vQYMGFbqM8L333lP37t0VGxtb6DFHjRqlESNGOG6npKSodu3a6tq1q8LCwlxSd3Z2tubPn6+bbrpJvr6V+2K6J9OytO+3JZKkoXdcrxrhARd+AC7qUho/cD3GD0qKsYPSYPygNDxp/NhXtRWFW8NVZGSkvL29dfToUaftR48eVUxMTIGPiYqK0pw5c5SRkaETJ04oNjZWI0eOVL169fLtu2/fPi1YsECzZ8++YB3+/v7y9/fPt93X19fl/5hlcUxPsyLhqIyRmtQI02WRoe4up1K5FMYPyg7jByXF2EFpMH5QGp4wforz/G5taOHn56fWrVtr4cKFjm02m00LFy5U+/btL/jYgIAA1axZUzk5Ofrqq69022235dvngw8+UPXq1XXLLbe4vHYUbtHZLoE3NI5ycyUAAABA+XH7ssARI0ZowIABatOmjdq2baupU6cqLS1NgwYNkiTdd999qlmzpiZOnChJWrVqlQ4ePKhWrVrp4MGDGjdunGw2m5555hmn49psNn3wwQcaMGCAfHzc/jIvGTlWm5bmub4VAAAAcKlwe+ro06ePjh8/rjFjxujIkSNq1aqV5s6d62hysX//fnl5nZtgy8jI0OjRo7Vnzx6FhISoR48emjlzpiIiIpyOu2DBAu3fv1+DBw8uz5dzyVu/P0kpGTmqEuSrVrWruLscAAAAoNy4PVxJ0rBhwzRs2LAC71uyZInT7c6dO2vr1q0XPWbXrl1ljHFFeSiGRdtzZ606N4ySt1fBre8BAACAysjtFxFG5bL4bLi6niWBAAAAuMQQruAyf51K146jp+VlyZ25AgAAAC4lhCu4jH3WqnWdKooI8nNzNQAAAED5IlzBZRaxJBAAAACXMMIVXOJMllUrdp+QRAt2AAAAXJoIV3CJlXsSlZljU2x4gBpFh7q7HAAAAKDcEa7gEnmXBFostGAHAADApYdwhVIzxmjx9uOSWBIIAACASxfhCqX259FUHUw6I38fL10TH+nucgAAAAC3IFyh1OxLAtvHV1Ogn7ebqwEAAADcg3CFUrNf34olgQAAALiUEa5QKsnp2Vq3/5Qk6fpGhCsAAABcughXKJWlO4/LajNqUD1EtasGubscAAAAwG0IVygVlgQCAAAAuQhXKDGrzWjJjnPXtwIAAAAuZYQrlNjGA0k6lZ6t0AAfta5Txd3lAAAAAG5FuEKJ2ZcEdmoYJV9vhhIAAAAubXwiRonZr291A10CAQAAAMIVSuZIcoa2Hk6RxSJd1yjK3eUAAAAAbke4QoksPtvIolXtCFUL8XdzNQAAAID7Ea5QIiwJBAAAAJwRrlBsmTlWLd+VKIkW7AAAAIAd4QrFtmrPSaVnWRUd5q9msWHuLgcAAADwCIQrFJt9SeD1jarLYrG4uRoAAADAMxCuUCzGGEczC5YEAgAAAOcQrlAsexLTtO9Euvy8vdSxfqS7ywEAAAA8BuEKxbL47JLAdvWqKtjfx83VAAAAAJ6DcIViyXu+FQAAAIBzCFcostMZ2VqdcFKSdAPnWwEAAABOCFcosmU7E5VjM6oXGay4yGB3lwMAAAB4FMIVisyxJJBZKwAAACAfwhWKxGYzWrzjuCSWBAIAAAAFIVyhSLYcSlZiaqZC/H10VVxVd5cDAAAAeBzCFYrEviSwY/1I+fkwbAAAAIDz8SkZRWIPVywJBAAAAApGuMJFHTudod//SpYkXdc4ys3VAAAAAJ6JcIWLWnK2kUWLmuGqHhrg5moAAAAAz0S4wkUtpgU7AAAAcFGEK1xQVo5Nv+5MlMT5VgAAAMCFEK5wQWv3nlRqZo4iQ/x0ec1wd5cDAAAAeCzCFS7I3iWwc8Pq8vKyuLkaAAAAwHMRrnBBi3bQgh0AAAAoCsIVCrXvRJr2HE+Tj5dF1zaMdHc5AAAAgEcjXKFQ9iWBbeKqKCzA183VAAAAAJ6NcIVC2cMVSwIBAACAiyNcoUBpmTlateekJMIVAAAAUBSEKxRo+a5EZVltql01UPFRIe4uBwAAAPB4hCsUaLG9S2Cj6rJYaMEOAAAAXAzhCvkYY7R4+3FJ0vUsCQQAAACKhHCFfLYeTtGRlAwF+nrr6nrV3F0OAAAAUCEQrpDP4rNdAjvUr6YAX283VwMAAABUDIQr5GNvwc6SQAAAAKDoih2uxo4dq3379pVFLfAAJ9OytOFAkiTp+kaEKwAAAKCoih2uvvnmG8XHx+vGG2/Uxx9/rMzMzLKoC26y9M9jMkZqUiNMsRGB7i4HAAAAqDCKHa42btyoNWvWqFmzZnrssccUExOjIUOGaM2aNWVRH8rZorNdAm9oHOXmSgAAAICKpUTnXF1xxRV67bXXdOjQIb333nv666+/1KFDB11++eV69dVXlZyc7Oo6UQ5yrDYttV/fivOtAAAAgGIpVUMLY4yys7OVlZUlY4yqVKmi119/XbVr19Znn33mqhpRTtbvT1JKRo6qBPmqVe0q7i4HAAAAqFBKFK7WrVunYcOGqUaNGnriiSd0xRVXaNu2bVq6dKl27typF154QcOHD3d1rShj9i6BnRtGydvL4uZqAAAAgIql2OGqRYsWuvrqq5WQkKD33ntPBw4c0EsvvaT69es79unXr5+OHz/u0kJR9hbTgh0AAAAoMZ/iPqB3794aPHiwatasWeg+kZGRstlspSoM5etg0hntOHpaXpbcmSsAAAAAxVPscPXcc8+VRR1wM/uSwNZ1qigiyM/N1QAAAAAVT7GXBd55552aNGlSvu0vv/yy7r77bpcUhfLHkkAAAACgdIodrn755Rf16NEj3/bu3bvrl19+cUlRKF8Z2Vat2J0oiRbsAAAAQEkVO1ylpqbKzy//sjFfX1+lpKS4pCiUr5W7Tygj26bY8AA1ig51dzkAAABAhVSiboEFXcPq008/VdOmTV1SFMrXojxLAi0WWrADAAAAJVGihhZ33HGHdu/erRtuuEGStHDhQn3yySf64osvXF4gypYxxhGuWBIIAAAAlFyxw1XPnj01Z84cvfjii/ryyy8VGBioyy+/XAsWLFDnzp3LokaUoT+Ppupg0hn5+3jpmvhId5cDAAAAVFjFDleSdMstt+iWW25xdS1wA/usVfv4agr083ZzNQAAAEDFVexzrlC5LGZJIAAAAOASxZ65slqteuWVV/T5559r//79ysrKcrr/5MmTLisOZSs5PVvr9p+SJF3fiHAFAAAAlEaxZ67Gjx+vKVOmqE+fPkpOTtaIESN0xx13yMvLS+PGjSuDElFWlu48LqvNqEH1ENWuGuTucgAAAIAKrdjh6qOPPtI777yjJ598Uj4+PurXr5/effddjRkzRr/99ltZ1IgywpJAAAAAwHWKHa6OHDmiFi1aSJJCQkKUnJwsSbr11lv1ww8/uLY6lBmrzWjJjnPXtwIAAABQOsUOV7Vq1dLhw4clSfHx8Zo3b54kac2aNfL393dtdSgzGw8k6VR6tkIDfNS6ThV3lwMAAABUeMUOV7169dLChQslSX//+9/13HPPqUGDBrrvvvs0ePBglxeIsmFfEtipYZR8vWkaCQAAAJRWsbsFvvTSS46/9+nTR3Xq1NGKFSvUoEED9ezZ06XFoezYr291A10CAQAAAJcoVrjKzs7Www8/rOeee05169aVJF199dW6+uqry6Q4lI0jyRnaejhFFot0XaMod5cDAAAAVArFWg/m6+urr776qqxqQTlZfLaRRctaEaoWwnlyAAAAgCsU+2Sb22+/XXPmzCmDUlBeFtGCHQAAAHC5Yp9z1aBBA02YMEHLly9X69atFRwc7HT/8OHDXVYcXC8zx6rluxIlEa4AAAAAVyp2uHrvvfcUERGhdevWad26dU73WSwWwpWHW7XnpNKzrKoe6q9msWHuLgcAAACoNIodrhISEsqiDpQT+5LA6xtVl8VicXM1AAAAQOXBBY4uIcYYRzOL61kSCAAAALhUsWeuLnah4Pfff7/ExaBs7UlM074T6fL1tqhjg0h3lwMAAABUKsUOV6dOnXK6nZ2drS1btigpKUk33HCDywqD6y0+uySwXd1qCvEv9j89AAAAgAso9ifsr7/+Ot82m82mIUOGKD4+3iVFoWw4zrdiSSAAAADgci4558rLy0sjRozQK6+84orDoQyczsjW6oSTkmjBDgAAAJQFlzW02L17t3Jyclx1OLjYsp2JyrEZ1Y0MVt3I4Is/AAAAAECxFDtcjRgxwunriSeeUN++fdWnTx/16dOn2AW88cYbiouLU0BAgNq1a6fVq1cXum92drYmTJig+Ph4BQQEqGXLlpo7d26+/Q4ePKh7771X1apVU2BgoFq0aKG1a9cWu7bKJG8LdgAAAACuV+xzrjZs2OB028vLS1FRUZo8efJFOwme77PPPtOIESM0ffp0tWvXTlOnTlW3bt20Y8cOVa+ePwSMHj1as2bN0jvvvKPGjRvr559/Vq9evbRixQpdccUVknIbbnTo0EHXX3+9fvrpJ0VFRWnnzp2qUqVKcV9qpWGzGS3ecVySdGMTwhUAAABQFoodrhYvXuyyJ58yZYoefPBBDRo0SJI0ffp0/fDDD3r//fc1cuTIfPvPnDlTzz77rHr06CFJGjJkiBYsWKDJkydr1qxZkqRJkyapdu3a+uCDDxyPq1u3rstqroi2HEpWYmqmQvx9dFVcVXeXAwAAAFRKxQ5XCQkJysnJUYMGDZy279y5U76+voqLiyvScbKysrRu3TqNGjXKsc3Ly0tdunTRypUrC3xMZmamAgICnLYFBgZq2bJljtvffvutunXrprvvvltLly5VzZo19eijj+rBBx8stJbMzExlZmY6bqekpEjKXYaYnZ1dpNdzMfbjuOp4xbHgjyOSpGviq8pirMrOtpZ7DSgdd44fVHyMH5QUYwelwfhBaXjS+ClODcUOVwMHDtTgwYPzhatVq1bp3Xff1ZIlS4p0nMTERFmtVkVHRzttj46O1vbt2wt8TLdu3TRlyhR16tRJ8fHxWrhwoWbPni2r9VxY2LNnj958802NGDFC//znP7VmzRoNHz5cfn5+GjBgQIHHnThxosaPH59v+7x58xQUFFSk11NU8+fPd+nximLO796SLKqWeVg//nio3J8fruOO8YPKg/GDkmLsoDQYPygNTxg/6enpRd7XYowxxTl4WFiY1q9fr/r16ztt37Vrl9q0aaOkpKQiHefQoUOqWbOmVqxYofbt2zu2P/PMM1q6dKlWrVqV7zHHjx/Xgw8+qO+++04Wi0Xx8fHq0qWL3n//fZ05c0aS5OfnpzZt2mjFihWOxw0fPlxr1qy54IzY+TNXtWvXVmJiosLCwor0ei4mOztb8+fP10033SRfX1+XHLMoElMz1X7SUknSimc6KyrUv9yeG67jrvGDyoHxg5Ji7KA0GD8oDU8aPykpKYqMjFRycvJFs0GxZ64sFotOnz6db3tycrLTDNLFREZGytvbW0ePHnXafvToUcXExBT4mKioKM2ZM0cZGRk6ceKEYmNjNXLkSNWrV8+xT40aNdS0aVOnxzVp0kRfffVVobX4+/vL3z9/6PD19XX5P2ZZHPNClu3OXRLYoma4YquGlNvzomyU9/hB5cL4QUkxdlAajB+UhieMn+I8f7FbsXfq1EkTJ050ClJWq1UTJ05Ux44di3wcPz8/tW7dWgsXLnRss9lsWrhwodNMVkECAgJUs2ZN5eTk6KuvvtJtt93muK9Dhw7asWOH0/5//vmn6tSpU+TaKpPFO862YOfCwQAAAECZKvbM1aRJk9SpUyc1atRI1157rSTp119/VUpKihYtWlSsY40YMUIDBgxQmzZt1LZtW02dOlVpaWmO7oH33XefatasqYkTJ0rKPa/r4MGDatWqlQ4ePKhx48bJZrPpmWeecRzziSee0DXXXKMXX3xRvXv31urVq/X222/r7bffLu5LrdCsNqOVuxO1aFtuuOrcMMrNFQEAAACVW7HDVdOmTfX777/r9ddf16ZNmxQYGKj77rtPw4YNU9WqxWvz3adPHx0/flxjxozRkSNH1KpVK82dO9fR5GL//v3y8jo3uZaRkaHRo0drz549CgkJUY8ePTRz5kxFREQ49rnqqqv09ddfa9SoUZowYYLq1q2rqVOn6p577inuS62w5m45rPHfbdXh5AzHtqEfrde4/2uqm5vXcGNlAAAAQOVV7HAlSbGxsXrxxRddUsCwYcM0bNiwAu87v/Ng586dtXXr1ose89Zbb9Wtt97qivIqnLlbDmvIrPU6v0vJ0ZQMDZm1Xm/eeyUBCwAAACgDxT7n6oMPPtAXX3yRb/sXX3yhGTNmuKQolIzVZjT+u635gpUkx7bx322V1VasBpEAAAAAiqDY4WrixImKjIzMt7169eoum81CyaxOOOm0FPB8RtLh5AytTjhZfkUBAAAAl4hih6v9+/erbt26+bbXqVNH+/fvd0lRKJljpwsPViXZDwAAAEDRFTtcVa9eXb///nu+7Zs2bVK1atVcUhRKpnpogEv3AwAAAFB0xQ5X/fr10/Dhw7V48WJZrVZZrVYtWrRIjz32mPr27VsWNaKI2tatqhrhAbIUcr9FUo3wALWtW7yujgAAAAAurtjh6vnnn1e7du104403KjAwUIGBgeratatuuOEGvfDCC2VRI4rI28uisT2bFnifPXCN7dlU3l6FxS8AAAAAJVXsVux+fn767LPP9K9//UsbN25UYGCgWrRooTp16pRFfSimm5vX0Jv3XqkRn29SepbVsT0mPEBje3KdKwAAAKCslOg6V5LUoEEDNWjQQJKUkpKiN998U++9957Wrl3rsuJQMjc3r6FvNx3Sj5uP6I4ra+ru1rXVtm5VZqwAAACAMlTicCVJixcv1vvvv6/Zs2crPDxcvXr1clVdKKWUMzmSpGsbRKp9PI1GAAAAgLJW7HB18OBBffjhh/rggw+UlJSkU6dO6eOPP1bv3r1lsTAz4imSzmRJkiIC/dxcCQAAAHBpKHJDi6+++ko9evRQo0aNtHHjRk2ePFmHDh2Sl5eXWrRoQbDyMMlnsiVJ4UG+bq4EAAAAuDQUeeaqT58++sc//qHPPvtMoaGhZVkTXCAp/Wy4CiRcAQAAAOWhyDNX999/v9544w3dfPPNmj59uk6dOlWWdaEUcqw2nc7IPecqgnAFAAAAlIsih6u33npLhw8f1kMPPaRPPvlENWrU0G233SZjjGw2W1nWiGJKORusJGauAAAAgPJSrIsIBwYGasCAAVq6dKk2b96sZs2aKTo6Wh06dFD//v01e/bssqoTxZCUntvMItTfRz7exb5ONAAAAIASKPEn7wYNGujFF1/UgQMHNGvWLKWnp6tfv36urA0llEQzCwAAAKDcleo6V5Lk5eWlnj17qmfPnjp27JgrakIp2TsFRhCuAAAAgHLj0jVj1atXd+XhUELJdAoEAAAAyh0n5FRC9nOuuIAwAAAAUH4IV5UQ51wBAAAA5Y9wVQnZLyDMNa4AAACA8lPscFWvXj2dOHEi3/akpCTVq1fPJUWhdGhoAQAAAJS/YoervXv3ymq15tuemZmpgwcPuqQolI4jXHHOFQAAAFBuityK/dtvv3X8/eeff1Z4eLjjttVq1cKFCxUXF+fS4lAy9oYWYSwLBAAAAMpNkcPV7bffLkmyWCwaMGCA032+vr6Ki4vT5MmTXVocSiaJZYEAAABAuStyuLLZbJKkunXras2aNYqMjCyzolA69utcEa4AAACA8lPkcGWXkJCQb1tSUpIiIiJcUQ9KyRhzbuaKc64AAACAclPshhaTJk3SZ5995rh99913q2rVqqpZs6Y2bdrk0uJQfKmZObLajCRmrgAAAIDyVOxwNX36dNWuXVuSNH/+fC1YsEBz585V9+7d9fTTT7u8QBSPvVOgv4+XAny93VwNAAAAcOko9rLAI0eOOMLV999/r969e6tr166Ki4tTu3btXF4gisd+AeFwOgUCAAAA5arYM1dVqlTRgQMHJElz585Vly5dJOWe61PQ9a9QvriAMAAAAOAexZ65uuOOO9S/f381aNBAJ06cUPfu3SVJGzZsUP369V1eIIrHPnNFMwsAAACgfBU7XL3yyiuKi4vTgQMH9PLLLyskJESSdPjwYT366KMuLxDFk3Qm9wLC4cxcAQAAAOWq2OHK19dXTz31VL7tTzzxhEsKQumcm7kiXAEAAADlqdjnXEnSzJkz1bFjR8XGxmrfvn2SpKlTp+qbb75xaXEovhTOuQIAAADcotjh6s0339SIESPUvXt3JSUlOZpYREREaOrUqa6uD8VEt0AAAADAPYodrqZNm6Z33nlHzz77rLy9z11HqU2bNtq8ebNLi0PxnTvnioYWAAAAQHkqdrhKSEjQFVdckW+7v7+/0tLSXFIUSo5zrgAAAAD3KHa4qlu3rjZu3Jhv+9y5c9WkSRNX1IRS4DpXAAAAgHsUuVvghAkT9NRTT2nEiBEaOnSoMjIyZIzR6tWr9cknn2jixIl69913y7JWFAHXuQIAAADco8jhavz48XrkkUf0wAMPKDAwUKNHj1Z6err69++v2NhYvfrqq+rbt29Z1ooiYOYKAAAAcI8ihytjjOPv99xzj+655x6lp6crNTVV1atXL5PiUDwZ2Vadyc7t3hjGOVcAAABAuSrWRYQtFovT7aCgIAUFBbm0IJSc/RpXXhYp1L/Y14cGAAAAUArF+gTesGHDfAHrfCdPnixVQSi5pDPnrnHl5XXhfycAAAAArlWscDV+/HiFh4eXVS0oJUczC65xBQAAAJS7YoWrvn37cn6VB0vOM3MFAAAAoHwV+TpXF1sOCPdLSs+SRKdAAAAAwB2KHK7ydguEZ2LmCgAAAHCfIi8LtNlsZVkHXODcBYQJVwAAAEB5K/LMFTxf0pncZYHhNLQAAAAAyh3hqhJh5goAAABwH8JVJWI/54qGFgAAAED5I1xVIoQrAAAAwH0IV5WIfVkg3QIBAACA8ke4qkTs17kKD6ShBQAAAFDeCFeVhNVmlJKRI4llgQAAAIA7EK4qiZSz51tJLAsEAAAA3IFwVUnYm1mE+PvI15t/VgAAAKC88Sm8kkg6QzMLAAAAwJ0IV5XEuWYWhCsAAADAHQhXlQTXuAIAAADci3BVSdivcUW4AgAAANyDcFVJnLuAMNe4AgAAANyBcFVJsCwQAAAAcC/CVSWRdCa3oUUEDS0AAAAAtyBcVRLJ6bRiBwAAANyJcFVJJLEsEAAAAHArwlUlce46VzS0AAAAANyBcFVJ0NACAAAAcC/CVSVgjCFcAQAAAG5GuKoE0rOsyrYaSTS0AAAAANyFcFUJ2JtZ+Hl7KdDX283VAAAAAJcmwlUl4GhmEeQri8Xi5moAAACASxPhqhKwX+OKCwgDAAAA7kO4qgS4xhUAAADgfoSrSsDeKZBrXAEAAADuQ7iqBJLS7eGKmSsAAADAXQhXlUDSmdyGFiwLBAAAANyHcFUJ0NACAAAAcD/CVSVgXxbIzBUAAADgPoSrSsC+LDA8iIYWAAAAgLsQriqB5DM5klgWCAAAALgT4aoSSE4/O3NFuAIAAADchnBVCXARYQAAAMD9CFcVXGaOVelZVklSBBcRBgAAANyGcFXBJZ+dtbJYpNAAHzdXAwAAAFy6CFcVnP0aV+GBvvLysri5GgAAAODS5RHh6o033lBcXJwCAgLUrl07rV69utB9s7OzNWHCBMXHxysgIEAtW7bU3LlznfYZN26cLBaL01fjxo3L+mW4hX3mik6BAAAAgHu5PVx99tlnGjFihMaOHav169erZcuW6tatm44dO1bg/qNHj9Zbb72ladOmaevWrXrkkUfUq1cvbdiwwWm/Zs2a6fDhw46vZcuWlcfLKXdJeWauAAAAALiP28PVlClT9OCDD2rQoEFq2rSppk+frqCgIL3//vsF7j9z5kz985//VI8ePVSvXj0NGTJEPXr00OTJk5328/HxUUxMjOMrMjKyPF5OubN3CuQCwgAAAIB7ubUDQlZWltatW6dRo0Y5tnl5ealLly5auXJlgY/JzMxUQECA07bAwMB8M1M7d+5UbGysAgIC1L59e02cOFGXXXZZocfMzMx03E5JSZGUuwQxOzu7RK/tfPbjuOp4didTMyRJYf7eLj82PEdZjR9cGhg/KCnGDkqD8YPS8KTxU5waLMYYU4a1XNChQ4dUs2ZNrVixQu3bt3dsf+aZZ7R06VKtWrUq32P69++vTZs2ac6cOYqPj9fChQt12223yWq1OgLSTz/9pNTUVDVq1EiHDx/W+PHjdfDgQW3ZskWhoaH5jjlu3DiNHz8+3/aPP/5YQUFBLnzFrvfDfi/NO+ila6Ntuquezd3lAAAAAJVKenq6+vfvr+TkZIWFhV1w3wrXu/vVV1/Vgw8+qMaNG8tisSg+Pl6DBg1yWkbYvXt3x98vv/xytWvXTnXq1NHnn3+u+++/P98xR40apREjRjhup6SkqHbt2uratetF38Ciys7O1vz583XTTTfJ19d150et/m6bdPCALm9SXz1urO+y48KzlNX4waWB8YOSYuygNBg/KA1PGj/2VW1F4dZwFRkZKW9vbx09etRp+9GjRxUTE1PgY6KiojRnzhxlZGToxIkTio2N1ciRI1WvXr1CnyciIkINGzbUrl27Crzf399f/v7++bb7+vq6/B/T1cc8nZl7AeGqIQFuH3goe2UxJnHpYPygpBg7KA3GD0rDE8ZPcZ7frQ0t/Pz81Lp1ay1cuNCxzWazaeHChU7LBAsSEBCgmjVrKicnR1999ZVuu+22QvdNTU3V7t27VaNGDZfV7ikcDS3oFggAAAC4ldu7BY4YMULvvPOOZsyYoW3btmnIkCFKS0vToEGDJEn33XefU8OLVatWafbs2dqzZ49+/fVX3XzzzbLZbHrmmWcc+zz11FNaunSp9u7dqxUrVqhXr17y9vZWv379yv31lbXk9CxJXOcKAAAAcDe3n3PVp08fHT9+XGPGjNGRI0fUqlUrzZ07V9HR0ZKk/fv3y8vrXAbMyMjQ6NGjtWfPHoWEhKhHjx6aOXOmIiIiHPv89ddf6tevn06cOKGoqCh17NhRv/32m6Kiosr75ZU5+8xVRBDhCgAAAHAnt4crSRo2bJiGDRtW4H1Llixxut25c2dt3br1gsf79NNPXVWax7NfRJhwBQAAALiX25cFouRsNqOUDPs5V1xEGAAAAHAnwlUFdjojR/arlNHQAgAAAHAvwlUFlnQmt5lFkJ+3/Hz4pwQAAADciU/kFZjjfCtmrQAAAAC3I1xVYI5rXAVxvhUAAADgboSrCiyJa1wBAAAAHoNwVYElc40rAAAAwGMQriqwZK5xBQAAAHgMwlUFZj/nKoxlgQAAAIDbEa4qsHPdAmloAQAAALgb4aoCSz57nSuWBQIAAADuR7iqwLjOFQAAAOA5CFcVWLLjOleEKwAAAMDdCFcVmL2hBedcAQAAAO5HuKqgjDGOVuzMXAEAAADuR7iqoM5kW5VltUninCsAAADAExCuKih7Mwtfb4uC/LzdXA0AAAAAwlUFZQ9X4YF+slgsbq4GAAAAAOGqgrJ3CuQaVwAAAIBnIFxVUI4LCHO+FQAAAOARCFcV1LllgYQrAAAAwBMQriqoJC4gDAAAAHgUwlUFZZ+54gLCAAAAgGcgXFVQjnOumLkCAAAAPALhqoKiWyAAAADgWQhXFRQNLQAAAADPQriqoAhXAAAAgGchXFVQ55YF0tACAAAA8ASEqwoqKZ2LCAMAAACehHBVAWVbbUrLskqioQUAAADgKQhXFZB9SaDFIoUGEK4AAAAAT0C4qoDszSzCAnzl7WVxczUAAAAAJMJVhWS/gDCdAgEAAADPQbiqgOwzV5xvBQAAAHgOwlUFxDWuAAAAAM9DuKqAkrjGFQAAAOBxCFcVkOMCwsxcAQAAAB6DcFUBJdsvIMw5VwAAAIDHIFxVQPZlgZxzBQAAAHgOwlUFREMLAAAAwPMQriogGloAAAAAnodwVQFxzhUAAADgeQhXFRDdAgEAAADPQ7iqYGw24whXnHMFAAAAeA7CVQVzOjNHNpP79zDCFQAAAOAxCFcVTPLZToGBvt4K8PV2czUAAAAA7AhXFUzSGZpZAAAAAJ6IcFXBcI0rAAAAwDMRrioYR6dAZq4AAAAAj0K4qmCS6BQIAAAAeCTCVQXjuIBwoJ+bKwEAAACQF+GqgrGfc8WyQAAAAMCzEK4qGMeyQMIVAAAA4FEIVxWMY+aKZYEAAACARyFcVTApdAsEAAAAPBLhqoKxX0SYboEAAACAZyFcVTBcRBgAAADwTISrCsQY42howbJAAAAAwLMQriqQjGybsnJskqSIIBpaAAAAAJ6EcFWB2M+38vGyKNjP283VAAAAAMiLcFWBJOdZEmixWNxcDQAAAIC8CFcViL2ZRRjNLAAAAACPQ7iqQM5dQJhwBQAAAHgawlUFknz2nCuaWQAAAACeh3BVgTBzBQAAAHguwlUFYr/GVTjXuAIAAAA8DuGqAnF0CwxkWSAAAADgaQhXFUjy2WWB4YE+bq4EAAAAwPkIVxVIEg0tAAAAAI9FuKpA7A0tOOcKAAAA8DyEqwqEboEAAACA5yJcVSCOhhYsCwQAAAA8DuGqgsi22pSamSOJmSsAAADAExGuKoiUs7NWkhRGuAIAAAA8DuGqgrBfQDg0wEfeXhY3VwMAAADgfISrCsLRzIJOgQAAAIBHIlxVEMn2a1wF0swCAAAA8ESEqwriXKdAZq4AAAAAT0S4qiAcFxCmmQUAAADgkQhXFQThCgAAAPBshKsKgmWBAAAAgGcjXFUQSek0tAAAAAA8GeGqgrBf5yqcmSsAAADAIxGuKgjHskDOuQIAAAA8EuGqgkh2XESYZYEAAACAJyJcVRCOZYHMXAEAAAAeiXBVAdhs5lxDC865AgAAADwS4aoCSM3Kkc3k/p2ZKwAAAMAzeUS4euONNxQXF6eAgAC1a9dOq1evLnTf7OxsTZgwQfHx8QoICFDLli01d+7cQvd/6aWXZLFY9Pjjj5dB5eXDfr5VgK+XAny93VwNAAAAgIK4PVx99tlnGjFihMaOHav169erZcuW6tatm44dO1bg/qNHj9Zbb72ladOmaevWrXrkkUfUq1cvbdiwId++a9as0VtvvaXLL7+8rF9GmTrXKZBmFgAAAICncnu4mjJlih588EENGjRITZs21fTp0xUUFKT333+/wP1nzpypf/7zn+rRo4fq1aunIUOGqEePHpo8ebLTfqmpqbrnnnv0zjvvqEqVKuXxUspMkqNTIEsCAQAAAE/l484nz8rK0rp16zRq1CjHNi8vL3Xp0kUrV64s8DGZmZkKCAhw2hYYGKhly5Y5bRs6dKhuueUWdenSRf/6178uWEdmZqYyMzMdt1NSUiTlLkHMzs4u1msqjP04JTneidNnJEmhAT4uqwcVS2nGD8D4QUkxdlAajB+UhieNn+LU4NZwlZiYKKvVqujoaKft0dHR2r59e4GP6datm6ZMmaJOnTopPj5eCxcu1OzZs2W1Wh37fPrpp1q/fr3WrFlTpDomTpyo8ePH59s+b948BQUFFeMVXdz8+fOL/ZhlRyySvJWRfEI//vijS+tBxVKS8QPYMX5QUowdlAbjB6XhCeMnPT29yPu6NVyVxKuvvqoHH3xQjRs3lsViUXx8vAYNGuRYRnjgwAE99thjmj9/fr4ZrsKMGjVKI0aMcNxOSUlR7dq11bVrV4WFhbmk7uzsbM2fP1833XSTfH2Lt7xv39I9UsIuNapbWz16NHNJPahYSjN+AMYPSoqxg9Jg/KA0PGn82Fe1FYVbw1VkZKS8vb119OhRp+1Hjx5VTExMgY+JiorSnDlzlJGRoRMnTig2NlYjR45UvXr1JEnr1q3TsWPHdOWVVzoeY7Va9csvv+j1119XZmamvL2dO+75+/vL398/33P5+vq6/B+zJMc8nZk7K1c1xN/tgwvuVRZjEpcOxg9KirGD0mD8oDQ8YfwU5/nd2tDCz89PrVu31sKFCx3bbDabFi5cqPbt21/wsQEBAapZs6ZycnL01Vdf6bbbbpMk3Xjjjdq8ebM2btzo+GrTpo3uuecebdy4MV+wqgjs3QK5xhUAAADgudy+LHDEiBEaMGCA2rRpo7Zt22rq1KlKS0vToEGDJEn33XefatasqYkTJ0qSVq1apYMHD6pVq1Y6ePCgxo0bJ5vNpmeeeUaSFBoaqubNmzs9R3BwsKpVq5Zve0VBt0AAAADA87k9XPXp00fHjx/XmDFjdOTIEbVq1Upz5851NLnYv3+/vLzOTbBlZGRo9OjR2rNnj0JCQtSjRw/NnDlTERERbnoFZS+JmSsAAADA47k9XEnSsGHDNGzYsALvW7JkidPtzp07a+vWrcU6/vnHqGiS07mIMAAAAODp3H4RYVxc0pksSSwLBAAAADwZ4aoCsJ9zxbJAAAAAwHMRrjxcRrZVmTk2ScxcAQAAAJ6McOXh7G3Yvb0sCvH3iFPkAAAAABSAcOXh8i4JtFgsbq4GAAAAQGEIVx4uKf1sMwvOtwIAAAA8GuHKwzmuccX5VgAAAIBHI1x5MKvNaP2+U5IkY4ysNuPmigAAAAAUhnDloeZuOayOkxbprV/2SJI2HkhWx0mLNHfLYTdXBgAAAKAghCsPNHfLYQ2ZtV6HkzOcth9JztCQWesJWAAAAIAHIlx5GKvNaPx3W1XQAkD7tvHfbWWJIAAAAOBhCFceZnXCyXwzVnkZSYeTM7Q64WT5FQUAAADgoghXHubY6cKDVUn2AwAAAFA+CFcepnpogEv3AwAAAFA+CFcepm3dqqoRHiBLIfdbJNUID1DbulXLsywAAAAAF0G48jDeXhaN7dlUkvIFLPvtsT2bytursPgFAAAAwB0IVx7o5uY19Oa9Vyom3HnpX0x4gN6890rd3LyGmyoDAAAAUBgfdxeAgt3cvIZuahqj1Qkndex0hqqH5i4FZMYKAAAA8EyEKw/m7WVR+/hq7i4DAAAAQBGwLBAAAAAAXIBwBQAAAAAuQLgCAAAAABcgXAEAAACACxCuAAAAAMAFCFcAAAAA4AKEKwAAAABwAcIVAAAAALgA4QoAAAAAXIBwBQAAAAAuQLgCAAAAABcgXAEAAACACxCuAAAAAMAFfNxdgCcyxkiSUlJSXHbM7OxspaenKyUlRb6+vi47Li4NjB+UBuMHJcXYQWkwflAanjR+7JnAnhEuhHBVgNOnT0uSateu7eZKAAAAAHiC06dPKzw8/IL7WExRItglxmaz6dChQwoNDZXFYnHJMVNSUlS7dm0dOHBAYWFhLjkmLh2MH5QG4wclxdhBaTB+UBqeNH6MMTp9+rRiY2Pl5XXhs6qYuSqAl5eXatWqVSbHDgsLc/sAQcXF+EFpMH5QUowdlAbjB6XhKePnYjNWdjS0AAAAAAAXIFwBAAAAgAsQrsqJv7+/xo4dK39/f3eXggqI8YPSYPygpBg7KA3GD0qjoo4fGloAAAAAgAswcwUAAAAALkC4AgAAAAAXIFwBAAAAgAsQrgAAAADABQhX5eSNN95QXFycAgIC1K5dO61evdrdJcHNfvnlF/Xs2VOxsbGyWCyaM2eO0/3GGI0ZM0Y1atRQYGCgunTpop07dzrtc/LkSd1zzz0KCwtTRESE7r//fqWmppbjq4A7TJw4UVdddZVCQ0NVvXp13X777dqxY4fTPhkZGRo6dKiqVaumkJAQ3XnnnTp69KjTPvv379ctt9yioKAgVa9eXU8//bRycnLK86XADd58801dfvnljgtztm/fXj/99JPjfsYOiuOll16SxWLR448/7tjGGEJhxo0bJ4vF4vTVuHFjx/2VYewQrsrBZ599phEjRmjs2LFav369WrZsqW7duunYsWPuLg1ulJaWppYtW+qNN94o8P6XX35Zr732mqZPn65Vq1YpODhY3bp1U0ZGhmOfe+65R3/88Yfmz5+v77//Xr/88oseeuih8noJcJOlS5dq6NCh+u233zR//nxlZ2era9euSktLc+zzxBNP6LvvvtMXX3yhpUuX6tChQ7rjjjsc91utVt1yyy3KysrSihUrNGPGDH344YcaM2aMO14SylGtWrX00ksvad26dVq7dq1uuOEG3Xbbbfrjjz8kMXZQdGvWrNFbb72lyy+/3Gk7YwgX0qxZMx0+fNjxtWzZMsd9lWLsGJS5tm3bmqFDhzpuW61WExsbayZOnOjGquBJJJmvv/7acdtms5mYmBjz73//27EtKSnJ+Pv7m08++cQYY8zWrVuNJLNmzRrHPj/99JOxWCzm4MGD5VY73O/YsWNGklm6dKkxJnes+Pr6mi+++MKxz7Zt24wks3LlSmOMMT/++KPx8vIyR44ccezz5ptvmrCwMJOZmVm+LwBuV6VKFfPuu+8ydlBkp0+fNg0aNDDz5883nTt3No899pgxhp8/uLCxY8eali1bFnhfZRk7zFyVsaysLK1bt05dunRxbPPy8lKXLl20cuVKN1YGT5aQkKAjR444jZvw8HC1a9fOMW5WrlypiIgItWnTxrFPly5d5OXlpVWrVpV7zXCf5ORkSVLVqlUlSevWrVN2drbT+GncuLEuu+wyp/HTokULRUdHO/bp1q2bUlJSHDMYqPysVqs+/fRTpaWlqX379owdFNnQoUN1yy23OI0ViZ8/uLidO3cqNjZW9erV0z333KP9+/dLqjxjx8fdBVR2iYmJslqtToNAkqKjo7V9+3Y3VQVPd+TIEUkqcNzY7zty5IiqV6/udL+Pj4+qVq3q2AeVn81m0+OPP64OHTqoefPmknLHhp+fnyIiIpz2PX/8FDS+7Pehctu8ebPat2+vjIwMhYSE6Ouvv1bTpk21ceNGxg4u6tNPP9X69eu1Zs2afPfx8wcX0q5dO3344Ydq1KiRDh8+rPHjx+vaa6/Vli1bKs3YIVwBQAU2dOhQbdmyxWnNOnAxjRo10saNG5WcnKwvv/xSAwYM0NKlS91dFiqAAwcO6LHHHtP8+fMVEBDg7nJQwXTv3t3x98svv1zt2rVTnTp19PnnnyswMNCNlbkOywLLWGRkpLy9vfN1Ojl69KhiYmLcVBU8nX1sXGjcxMTE5GuKkpOTo5MnTzK2LhHDhg3T999/r8WLF6tWrVqO7TExMcrKylJSUpLT/uePn4LGl/0+VG5+fn6qX7++WrdurYkTJ6ply5Z69dVXGTu4qHXr1unYsWO68sor5ePjIx8fHy1dulSvvfaafHx8FB0dzRhCkUVERKhhw4batWtXpfn5Q7gqY35+fmrdurUWLlzo2Gaz2bRw4UK1b9/ejZXBk9WtW1cxMTFO4yYlJUWrVq1yjJv27dsrKSlJ69atc+yzaNEi2Ww2tWvXrtxrRvkxxmjYsGH6+uuvtWjRItWtW9fp/tatW8vX19dp/OzYsUP79+93Gj+bN292Cujz589XWFiYmjZtWj4vBB7DZrMpMzOTsYOLuvHGG7V582Zt3LjR8dWmTRvdc889jr8zhlBUqamp2r17t2rUqFF5fv64u6PGpeDTTz81/v7+5sMPPzRbt241Dz30kImIiHDqdIJLz+nTp82GDRvMhg0bjCQzZcoUs2HDBrNv3z5jjDEvvfSSiYiIMN988435/fffzW233Wbq1q1rzpw54zjGzTffbK644gqzatUqs2zZMtOgQQPTr18/d70klJMhQ4aY8PBws2TJEnP48GHHV3p6umOfRx55xFx22WVm0aJFZu3ataZ9+/amffv2jvtzcnJM8+bNTdeuXc3GjRvN3LlzTVRUlBk1apQ7XhLK0ciRI83SpUtNQkKC+f33383IkSONxWIx8+bNM8YwdlB8ebsFGsMYQuGefPJJs2TJEpOQkGCWL19uunTpYiIjI82xY8eMMZVj7BCuysm0adPMZZddZvz8/Ezbtm3Nb7/95u6S4GaLFy82kvJ9DRgwwBiT2479ueeeM9HR0cbf39/ceOONZseOHU7HOHHihOnXr58JCQkxYWFhZtCgQeb06dNueDUoTwWNG0nmgw8+cOxz5swZ8+ijj5oqVaqYoKAg06tXL3P48GGn4+zdu9d0797dBAYGmsjISPPkk0+a7Ozscn41KG+DBw82derUMX5+fiYqKsrceOONjmBlDGMHxXd+uGIMoTB9+vQxNWrUMH5+fqZmzZqmT58+ZteuXY77K8PYsRhjjHvmzAAAAACg8uCcKwAAAABwAcIVAAAAALgA4QoAAAAAXIBwBQAAAAAuQLgCAAAAABcgXAEAAACACxCuAAAAAMAFCFcAAAAA4AKEKwC4hOzdu1cWi0UbN250dykO27dv19VXX62AgAC1atXK3eWUynPPPaeHHnrI3WVcUFxcnKZOneruMlwmKytLcXFxWrt2rbtLAQDCFQCUp4EDB8piseill15y2j5nzhxZLBY3VeVeY8eOVXBwsHbs2KGFCxfmu99isVzwa9y4ceVfdAGOHDmiV199Vc8++6y7S5Ekffjhh4qIiMi3fc2aNeUSAMsrxPn5+empp57SP/7xjzJ/LgC4GMIVAJSzgIAATZo0SadOnXJ3KS6TlZVV4sfu3r1bHTt2VJ06dVStWrV89x8+fNjxNXXqVIWFhTlte+qppxz7GmOUk5NT4lpK491339U111yjOnXquOX5iyoqKkpBQUHuLqPIijK27rnnHi1btkx//PFHOVQEAIUjXAFAOevSpYtiYmI0ceLEQvcZN25cviVyU6dOVVxcnOP2wIEDdfvtt+vFF19UdHS0IiIiNGHCBOXk5Ojpp59W1apVVatWLX3wwQf5jr99+3Zdc801CggIUPPmzbV06VKn+7ds2aLu3bsrJCRE0dHR+tvf/qbExETH/dddd52GDRumxx9/XJGRkerWrVuBr8Nms2nChAmqVauW/P391apVK82dO9dxv8Vi0bp16zRhwoRCZ6FiYmIcX+Hh4bJYLI7b27dvV2hoqH766Se1bt1a/v7+WrZsmWw2myZOnKi6desqMDBQLVu21Jdfflms1/jll1+qRYsWCgwMVLVq1dSlSxelpaUV+Dol6dNPP1XPnj2dtl133XUaPny4nnnmGVWtWlUxMTHFmmlLSkrSAw88oKioKIWFhemGG27Qpk2bHPdv2rRJ119/vUJDQxUWFqbWrVtr7dq1WrJkiQYNGqTk5OR8M3znzyhZLBa99dZbuvXWWxUUFKQmTZpo5cqV2rVrl6677joFBwfrmmuu0e7dux2P2b17t2677TZFR0crJCREV111lRYsWOD0uvft26cnnnjC8fx2X331lZo1ayZ/f3/FxcVp8uTJTq85Li5Ozz//vO677z6FhYXpoYceUlZWloYNG6YaNWooICBAderUcfr+qVKlijp06KBPP/20yO8tAJQFwhUAlDNvb2+9+OKLmjZtmv76669SHWvRokU6dOiQfvnlF02ZMkVjx47VrbfeqipVqmjVqlV65JFH9PDDD+d7nqefflpPPvmkNmzYoPbt26tnz546ceKEpNwP9DfccIOuuOIKrV27VnPnztXRo0fVu3dvp2PMmDFDfn5+Wr58uaZPn15gfa+++qomT56s//znP/r999/VrVs3/d///Z927twpKXdWqlmzZnryySfzzUIVx8iRI/XSSy9p27ZtuvzyyzVx4kT973//0/Tp0/XHH3/oiSee0L333usIkRd7jYcPH1a/fv00ePBgbdu2TUuWLNEdd9whY0yBz3/y5Elt3bpVbdq0yXffjBkzFBwcrFWrVunll1/WhAkTNH/+/CK9rrvvvlvHjh3TTz/9pHXr1unKK6/UjTfeqJMnT0rKnbGpVauW1qxZo3Xr1mnkyJHy9fXVNddck2+W70LvrT3MbNy4UY0bN1b//v318MMPa9SoUVq7dq2MMRo2bJhj/9TUVPXo0UMLFy7Uhg0bdPPNN6tnz57av3+/JGn27NmqVauWJkyY4Hh+SVq3bp169+6tvn37avPmzRo3bpyee+45ffjhh071/Oc//1HLli21YcMGPffcc3rttdf07bff6vPPP9eOHTv00UcfOf2iQZLatm2rX3/9tUjvKwCUGQMAKDcDBgwwt912mzHGmKuvvtoMHjzYGGPM119/bfL+SB47dqxp2bKl02NfeeUVU6dOHadj1alTx1itVse2Ro0amWuvvdZxOycnxwQHB5tPPvnEGGNMQkKCkWReeuklxz7Z2dmmVq1aZtKkScYYY55//nnTtWtXp+c+cOCAkWR27NhhjDGmc+fO5oorrrjo642NjTUvvPCC07arrrrKPProo47bLVu2NGPHjr3osYwx5oMPPjDh4eGO24sXLzaSzJw5cxzbMjIyTFBQkFmxYoXTY++//37Tr1+/Ir3GdevWGUlm7969Raprw4YNRpLZv3+/0/bOnTubjh07Om276qqrzD/+8Y+LHvPXX381YWFhJiMjw2l7fHy8eeutt4wxxoSGhpoPP/ywwMef/17Z1alTx7zyyiuO25LM6NGjHbdXrlxpJJn33nvPse2TTz4xAQEBF6y3WbNmZtq0aYU+jzHG9O/f39x0001O255++mnTtGlTp8fdfvvtTvv8/e9/NzfccIOx2WyFPv+rr75q4uLiLlgjAJQ1Zq4AwE0mTZqkGTNmaNu2bSU+RrNmzeTlde5HeXR0tFq0aOG47e3trWrVqunYsWNOj2vfvr3j7z4+PmrTpo2jjk2bNmnx4sUKCQlxfDVu3FiSnJaGtW7d+oK1paSk6NChQ+rQoYPT9g4dOpTqNRck74zRrl27lJ6erptuusnpNfzvf/9z1H+x19iyZUvdeOONatGihe6++2698847FzxH7syZM5Jyz6c73+WXX+50u0aNGvn+PQqyadMmpaamqlq1ak51JiQkOF7HiBEj9MADD6hLly566aWXnP59iiNvjdHR0ZLkNI6io6OVkZGhlJQUSbkzV0899ZSaNGmiiIgIhYSEaNu2bY6Zq8Js27atwPGwc+dOWa1Wx7bzZwAHDhyojRs3qlGjRho+fLjmzZuX79iBgYFKT08v4isGgLLh4+4CAOBS1alTJ3Xr1k2jRo3SwIEDne7z8vLKtwQtOzs73zF8fX2dblsslgK32Wy2IteVmpqqnj17atKkSfnuq1GjhuPvwcHBRT5mWctbS2pqqiTphx9+UM2aNZ328/f3d+xzodfo7e2t+fPna8WKFZo3b56mTZumZ599VqtWrVLdunXzPSYyMlKSdOrUKUVFRTndV9J/j9TUVNWoUUNLlizJd5+9C+C4cePUv39//fDDD/rpp580duxYffrpp+rVq9dFj19YjfbzowraZq/7qaee0vz58/Wf//xH9evXV2BgoO66665SNTbJ6/yxdeWVVyohIUE//fSTFixYoN69e6tLly5O59GdPHky33sPAOWNcAUAbvTSSy+pVatWatSokdP2qKgoHTlyRMYYxwdbV16b6rffflOnTp0kSTk5OVq3bp3jnJorr7xSX331leLi4uTjU/L/JsLCwhQbG6vly5erc+fOju3Lly9X27ZtS/cCLqBp06by9/fX/v37nZ43r6K8RovFog4dOqhDhw4aM2aM6tSpo6+//lojRozIt298fLzCwsK0detWNWzY0CWv48orr9SRI0fk4+OT7/yivBo2bKiGDRvqiSeeUL9+/fTBBx+oV69e8vPzc5oNcqXly5dr4MCBjhCXmpqqvXv3Ou1T0PM3adJEy5cvz3eshg0bytvb+4LPGRYWpj59+qhPnz666667dPPNN+vkyZOqWrWqpNwGJVdccUUpXxkAlA7LAgHAjVq0aKF77rlHr732mtP26667TsePH9fLL7+s3bt364033tBPP/3ksud944039PXXX2v79u0aOnSoTp06pcGDB0uShg4dqpMnT6pfv35as2aNdu/erZ9//lmDBg0q9of1p59+WpMmTdJnn32mHTt2aOTIkdq4caMee+wxl72W84WGhuqpp57SE088oRkzZmj37t1av369pk2bphkzZki6+GtctWqVXnzxRa1du1b79+/X7Nmzdfz4cTVp0qTA5/Ty8lKXLl20bNkyl72OLl26qH379rr99ts1b9487d27VytWrNCzzz6rtWvX6syZMxo2bJiWLFmiffv2afny5VqzZo2jxri4OKWmpmrhwoVKTEx06ZK5Bg0aaPbs2dq4caM2bdqk/v3755uNi4uL0y+//KKDBw86ujA++eSTWrhwoZ5//nn9+eefmjFjhl5//fWLNjKZMmWKPvnkE23fvl1//vmnvvjiC8XExDhdx+vXX39V165dXfYaAaAkCFcA4GYTJkzI98G0SZMm+u9//6s33nhDLVu21OrVq0vcSa8gL730kl566SW1bNlSy5Yt07fffutY2mafbbJareratatatGihxx9/XBEREU7ndxXF8OHDNWLECD355JNq0aKF5s6dq2+//VYNGjRw2WspyPPPP6/nnntOEydOVJMmTXTzzTfrhx9+cCzpu9hrDAsL0y+//KIePXqoYcOGGj16tCZPnqzu3bsX+pwPPPCAPv3002ItwbwQi8WiH3/8UZ06ddKgQYPUsGFD9e3bV/v27VN0dLS8vb114sQJ3XfffWrYsKF69+6t7t27a/z48ZKka665Ro888oj69OmjqKgovfzyyy6pS8oNO1WqVNE111yjnj17qlu3brryyiud9pkwYYL27t2r+Ph4x3K9K6+8Up9//rk+/fRTNW/eXGPGjNGECRPyLYs9X2hoqF5++WW1adNGV111lfbu3asff/zRMR5Xrlyp5ORk3XXXXS57jQBQEhZz/qJ+AABQbMYYtWvXzrE8D+WnT58+atmypf75z3+6uxQAlzhmrgAAcAGLxaK3335bOTk57i7lkpKVlaUWLVroiSeecHcpAMDMFQAA7vDRRx/p4YcfLvC+OnXq6I8//ijnigAApUW4AgDADU6fPq2jR48WeJ+vr6/q1KlTzhUBAEqLcAUAAAAALsA5VwAAAADgAoQrAAAAAHABwhUAAAAAuADhCgAAAABcgHAFAAAAAC5AuAIAAAAAFyBcAQAAAIAL/D8UQx3hmYYxKwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "# Random forest graph\n",
        "\n",
        "import pandas as pd\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Loading the dataset containing 'label' and the top 20 pixels from the combined dataset\n",
        "data_sign_test_selected = pd.read_csv(\"top_20_pixels_with_labels_train.csv\")\n",
        "\n",
        "# Separating the features and target variable (label)\n",
        "X = data_sign_test_selected.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y = data_sign_test_selected['label']   # Dependent variable\n",
        "\n",
        "# Splitting the top 20 pixels combined data into a training and a test set (80% training, 20% testing)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "\n",
        "# Define a list of n_estimators values to test\n",
        "n_estimators_values = [10, 20, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500]\n",
        "\n",
        "# Lists to store accuracy values for each n_estimators\n",
        "accuracies = []\n",
        "\n",
        "# Loop through different n_estimators values and calculate accuracy\n",
        "for n_estimators in n_estimators_values:\n",
        "    # Creating a Random Forest Classifier with the current number of trees\n",
        "    rf_classifier = RandomForestClassifier(n_estimators=n_estimators, random_state=0)\n",
        "    # Training the classifier on the training set\n",
        "    rf_classifier.fit(X_train, y_train)\n",
        "    # Making predictions on the test set\n",
        "    y_pred = rf_classifier.predict(X_test)\n",
        "\n",
        "    # Calculating and storing the test accuracy for each n_estimators\n",
        "    accuracy = accuracy_score(y_test, y_pred)\n",
        "    accuracies.append(accuracy)\n",
        "\n",
        "# Plot the results\n",
        "plt.figure(figsize=(10, 6))\n",
        "plt.plot(n_estimators_values, accuracies, marker='o')\n",
        "plt.title('Random Forest Accuracy vs. Number of Trees')\n",
        "plt.xlabel('Number of Trees (n_estimators)')\n",
        "plt.ylabel('Test Accuracy')\n",
        "plt.grid(True)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1lE0SOPt0Hkv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cd4c8a74-baf9-41fd-c26b-262b0715c535"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Test accuracy (n_estimators=10): 93.81%\n",
            "Test accuracy (n_estimators=20): 95.90%\n",
            "Test accuracy (n_estimators=50): 97.40%\n",
            "Test accuracy (n_estimators=100): 97.85%\n",
            "Test accuracy (n_estimators=150): 97.94%\n",
            "Test accuracy (n_estimators=200): 98.07%\n",
            "Test accuracy (n_estimators=250): 97.96%\n",
            "Test accuracy (n_estimators=300): 97.91%\n",
            "Test accuracy (n_estimators=350): 98.03%\n",
            "Test accuracy (n_estimators=400): 98.07%\n",
            "Test accuracy (n_estimators=450): 98.03%\n",
            "Test accuracy (n_estimators=500): 98.07%\n"
          ]
        }
      ],
      "source": [
        "import pandas as pd\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Load your combined dataset (70,000 rows, 21 columns) containing 'label' and the top 20 pixels\n",
        "data_sign_combined = pd.read_csv(\"top_20_pixels_with_labels_train.csv\")\n",
        "\n",
        "# Separate the features (pixels) and the target variable (label)\n",
        "X = data_sign_combined.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y = data_sign_combined['label']     # Dependent variable\n",
        "\n",
        "# Split the data into a training and test set (80% training, 20% testing)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "\n",
        "# Define a list of n_estimators values to test\n",
        "n_estimators_values = [10, 20, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500]  # You can extend this list\n",
        "\n",
        "# Create dictionaries to store the results\n",
        "accuracies = {}\n",
        "\n",
        "for n_estimators in n_estimators_values:\n",
        "    # Create a Random Forest Classifier with the current number of trees\n",
        "    rf_classifier = RandomForestClassifier(n_estimators=n_estimators, random_state=0)\n",
        "    # Train the classifier on the training set\n",
        "    rf_classifier.fit(X_train, y_train)\n",
        "    # Make predictions on the test set\n",
        "    y_pred = rf_classifier.predict(X_test)\n",
        "\n",
        "    # Calculate the test accuracy for the current number of trees\n",
        "    accuracy = accuracy_score(y_test, y_pred)\n",
        "    accuracies[n_estimators] = accuracy\n",
        "\n",
        "# Display the accuracies for each n_estimators value\n",
        "for n_estimators, accuracy in accuracies.items():\n",
        "    print(f'Test accuracy (n_estimators={n_estimators}): {accuracy * 100:.2f}%')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jTYd5wcGM1iY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "926fdec9-e4d8-4152-916b-42b2d1cf89fe"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Test accuracy: 92.33%\n"
          ]
        }
      ],
      "source": [
        "# The following code was used with the SVM model for the top 20 pixels based on Importance (on the test set which replicated the\n",
        "# top 20 pixels on the training set)\n",
        "\n",
        "import pandas as pd\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Loading the dataset containing 'label' and the top 20 pixels from the combined dataset\n",
        "data_sign_test_selected = pd.read_csv(\"top_20_pixels_with_labels_train.csv\")\n",
        "\n",
        "# Separating the features and target variable (label)\n",
        "X = data_sign_test_selected.iloc[:, 1:]  # Features (top 20 pixels)\n",
        "y = data_sign_test_selected['label']   # Dependent variable\n",
        "\n",
        "# Splitting the data into a training and test set (80% training and 20% testing)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "\n",
        "# Creating an SVM classifier\n",
        "svm_classifier = SVC()\n",
        "\n",
        "# Training the SVM classifier on the training set\n",
        "svm_classifier.fit(X_train, y_train)\n",
        "\n",
        "# Making predictions on the test set\n",
        "y_pred = svm_classifier.predict(X_test)\n",
        "\n",
        "# Calculating and displaying the test accuracy\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(f'Test accuracy: {accuracy * 100:.2f}%')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Result and Discussion**\n",
        "\n",
        "Regarding Q1, Random forest show better accuracy (98%) compared to CNN (94%). this will be checked together with other aspect of evaluation.\n",
        "\n",
        "Work on Q2 and Q3 is process, so it code will be provide in final report."
      ],
      "metadata": {
        "id": "26TG1toVkLnz"
      }
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}