Nepal-Earthquake-predictor-LGBModel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to my Richter's Predictor Nepal Earthquake Damage Predictor LGBM Model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Default code from Kaggle Notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load\n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the read-only \"../input/\" directory\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "\n",
    "# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n",
    "# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Displaying some important libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import tensorflow as tf\n",
    "# print(\"Tensorflow:\", tf.__version__)\n",
    "\n",
    "# import kerastuner as kt\n",
    "# print(\"kerastuner:\", kt.__version__)\n",
    "\n",
    "# import keras_tuner as kt2\n",
    "# print(\"keras_tuner:\", kt2.__version__)\n",
    "\n",
    "# import platform\n",
    "# print(\"Python:\", platform.python_version())\n",
    "\n",
    "# import numpy as np\n",
    "# print(\"numpy:\", np.__version__)\n",
    "\n",
    "# import pandas as pd\n",
    "# print(\"pandas:\", pd.__version__)\n",
    "\n",
    "# import sklearn\n",
    "# print(\"sklearn version:\", sklearn.__version__)\n",
    "\n",
    "# import sklearn\n",
    "# print(\"sklearn path:\", sklearn.__path__)\n",
    "\n",
    "# import matplotlib\n",
    "# print(\"matplotlib:\", matplotlib.__version__)\n",
    "\n",
    "# import seaborn as sns\n",
    "# print(\"seaborn:\", sns.__version__)\n",
    "\n",
    "# # WARNING:tensorflow:From c:\\Users\\Micha\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\keras\\src\\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.\n",
    "\n",
    "# # Tensorflow: 2.15.0\n",
    "# # C:\\Users\\Micha\\AppData\\Local\\Temp\\ipykernel_6936\\1753711907.py:4: DeprecationWarning: `import kerastuner` is deprecated, please use `import keras_tuner`.\n",
    "# #   import kerastuner as kt\n",
    "# # kerastuner: 1.0.5\n",
    "# # keras_tuner: 1.3.5\n",
    "# # Python: 3.10.11\n",
    "# # numpy: 1.24.3\n",
    "# # pandas: 2.1.4\n",
    "# # sklearn version: 1.2.2\n",
    "# # sklearn path: ['c:\\\\Users\\\\Micha\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310\\\\lib\\\\site-packages\\\\sklearn']\n",
    "# # matplotlib: 3.8.2\n",
    "# # seaborn: 0.13.0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Global random seed to make sure we can replicate any model that we create (no randomness)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import os\n",
    "\n",
    "\n",
    "\n",
    "np.random.seed(42)\n",
    "random.seed(42)\n",
    "np.random.seed(42)\n",
    "tf.random.set_seed(42)\n",
    "\n",
    "os.environ['TF_DETERMINISTIC_OPS'] = '1'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the training and testing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "train_values are the features (X), and train_labels is the target/label (Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_X = pd.read_csv(\"train_values.csv\")\n",
    "train_Y = pd.read_csv(\"train_labels.csv\")\n",
    "\n",
    "test_values = pd.read_csv(\"test_values.csv\")\n",
    "\n",
    "# print(\"train labels:\\n\", train_Y.head())\n",
    "\n",
    "# print(\"train values:\\n\", train_X.head())\n",
    "      \n",
    "# print(\"test_values:\\n\", test_values.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I want to find out which features to use since there are so many. Here are some common data science techniques:\n",
    "\n",
    "1. **Correlation Matrix with Heatmap**: Correlation states how the features are related to each other or the target variable. You can use a heatmap to visualize the correlation matrix.\n",
    "\n",
    "2. **Univariate Selection**: Statistical tests can be used to select those features that have the strongest relationship with the output variable. The scikit-learn library provides the `SelectKBest` class that can be used with a suite of different statistical tests to select a specific number of features.\n",
    "\n",
    "3. **Recursive Feature Elimination (RFE)**: RFE is a popular feature selection method that fits a model and removes the weakest feature (or features) until the specified number of features is reached.\n",
    "\n",
    "4. **Feature Importance**: You can get the feature importance of each feature of your dataset by using the feature importance property of the model. For example, Decision Trees models in the scikit-learn library offer an importance property that can be accessed directly.\n",
    "\n",
    "For categorical features, you can convert them into numerical values using techniques like One-Hot Encoding or Label Encoding before applying these feature selection techniques. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First, I will try RFE (Recursive Feature Elimination)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.feature_selection import RFE\n",
    "# from lightgbm import LGBMRegressor #Lightgbm is a great gradient boosting model for large amount of data\n",
    "\n",
    "# # Assuming X is your feature set and y is the target value\n",
    "# X = train_X.drop('building_id', axis=1)\n",
    "# X = pd.get_dummies(X) \n",
    "\n",
    "# y = train_Y.drop('building_id', axis=1)\n",
    "# y = np.ravel(y) # converting dataframe to a one-dimensional array using the ravel function from numpy\n",
    "\n",
    "# estimator = LGBMRegressor(verbose = 0, random_state = 42)  # It's best to find the best model for you\n",
    "# selector = RFE(estimator, step=1)\n",
    "# selector = selector.fit(X, y)\n",
    "\n",
    "# # Assuming 'X' is your DataFrame with the feature data\n",
    "# feature_names = X.columns\n",
    "\n",
    "# # Map the feature names to the support array, which tells you which features were selected\n",
    "# support_dict = dict(zip(feature_names, selector.support_))\n",
    "\n",
    "# # Get the selected features\n",
    "# selected_features = [feature for feature, support in support_dict.items() if support]\n",
    "\n",
    "# # Print the selected features\n",
    "# print(\"Selected features:\\n\", selected_features)\n",
    "\n",
    "# # ['geo_level_1_id', 'geo_level_2_id', 'geo_level_3_id', 'count_floors_pre_eq', 'age', 'area_percentage', 'height_percentage',\n",
    "# # 'has_superstructure_adobe_mud', 'has_superstructure_mud_mortar_stone', 'has_superstructure_stone_flag', 'has_superstructure_cement_mortar_stone',\n",
    "# # 'has_superstructure_mud_mortar_brick', 'has_superstructure_cement_mortar_brick', 'has_superstructure_timber', 'has_superstructure_bamboo', \n",
    "# # 'has_superstructure_rc_non_engineered', 'has_superstructure_rc_engineered', 'has_superstructure_other', 'count_families', 'has_secondary_use', \n",
    "# # 'land_surface_condition_n', 'land_surface_condition_o', 'foundation_type_h', 'foundation_type_r', 'foundation_type_u', 'roof_type_n', \n",
    "# # 'roof_type_q', 'roof_type_x', 'ground_floor_type_f', 'ground_floor_type_v', 'ground_floor_type_x', 'other_floor_type_q', 'position_s',\n",
    "# # 'plan_configuration_u']\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# # The feature ranking, such that ranking_[i] corresponds to the ranking position of the i-th feature. \n",
    "# # Selected features are assigned rank 1.\n",
    "# # Map the feature names to the ranking array\n",
    "# ranking_dict = dict(zip(feature_names, selector.ranking_))\n",
    "# print(ranking_dict)\n",
    "\n",
    "\n",
    "# # [ 1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  2 18 10 31\n",
    "# #  27 33 28 25 30  6  1  1  3  1  8  1  1  5  1  1  1  1 34  1  1 16  9  1\n",
    "# #  23  7  4 14  1 29 15 20 17 35 32 24 21 13 26  1 11 22 12 19]\n",
    "\n",
    "\n",
    "# # {'geo_level_1_id': 1, 'geo_level_2_id': 1, 'geo_level_3_id': 1, 'count_floors_pre_eq': 1, 'age': 1, 'area_percentage': 1, \n",
    "# #  'height_percentage': 1, 'has_superstructure_adobe_mud': 1, 'has_superstructure_mud_mortar_stone': 1, 'has_superstructure_stone_flag': 1, \n",
    "# #  'has_superstructure_cement_mortar_stone': 1, 'has_superstructure_mud_mortar_brick': 1, 'has_superstructure_cement_mortar_brick': 1, \n",
    "# #  'has_superstructure_timber': 1, 'has_superstructure_bamboo': 1, 'has_superstructure_rc_non_engineered': 1, 'has_superstructure_rc_engineered': 1, \n",
    "# #  'has_superstructure_other': 1, 'count_families': 1, 'has_secondary_use': 1, 'has_secondary_use_agriculture': 2, 'has_secondary_use_hotel': 18, \n",
    "# #  'has_secondary_use_rental': 10, 'has_secondary_use_institution': 31, 'has_secondary_use_school': 27, 'has_secondary_use_industry': 33, \n",
    "# #  'has_secondary_use_health_post': 28, 'has_secondary_use_gov_office': 25, 'has_secondary_use_use_police': 30, 'has_secondary_use_other': 6, \n",
    "# #  'land_surface_condition_n': 1, 'land_surface_condition_o': 1, 'land_surface_condition_t': 3, 'foundation_type_h': 1, 'foundation_type_i': 8, \n",
    "# #  'foundation_type_r': 1, 'foundation_type_u': 1, 'foundation_type_w': 5, 'roof_type_n': 1, 'roof_type_q': 1, 'roof_type_x': 1, \n",
    "# #  'ground_floor_type_f': 1, 'ground_floor_type_m': 34, 'ground_floor_type_v': 1, 'ground_floor_type_x': 1, 'ground_floor_type_z': 16, \n",
    "# #  'other_floor_type_j': 9, 'other_floor_type_q': 1, 'other_floor_type_s': 23, 'other_floor_type_x': 7, 'position_j': 4, 'position_o': 14, \n",
    "# #  'position_s': 1, 'position_t': 29, 'plan_configuration_a': 15, 'plan_configuration_c': 20, 'plan_configuration_d': 17, 'plan_configuration_f': 35, \n",
    "# #  'plan_configuration_m': 32, 'plan_configuration_n': 24, 'plan_configuration_o': 21, 'plan_configuration_q': 13, 'plan_configuration_s': 26, \n",
    "# #  'plan_configuration_u': 1, 'legal_ownership_status_a': 11, 'legal_ownership_status_r': 22, 'legal_ownership_status_v': 12, \n",
    "# #  'legal_ownership_status_w': 19}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34\n",
      "New features:\n",
      " ['land_surface_condition_n', 'land_surface_condition_o', 'foundation_type_h', 'foundation_type_r', 'foundation_type_u', 'roof_type_n', 'roof_type_q', 'roof_type_x', 'ground_floor_type_f', 'ground_floor_type_v', 'ground_floor_type_x', 'other_floor_type_q', 'position_s', 'plan_configuration_u']\n",
      "27\n"
     ]
    }
   ],
   "source": [
    "features = ['geo_level_1_id', 'geo_level_2_id', 'geo_level_3_id', 'count_floors_pre_eq', 'age', 'area_percentage', 'height_percentage',\n",
    "'has_superstructure_adobe_mud', 'has_superstructure_mud_mortar_stone', 'has_superstructure_stone_flag', 'has_superstructure_cement_mortar_stone',\n",
    "'has_superstructure_mud_mortar_brick', 'has_superstructure_cement_mortar_brick', 'has_superstructure_timber', 'has_superstructure_bamboo', \n",
    "'has_superstructure_rc_non_engineered', 'has_superstructure_rc_engineered', 'has_superstructure_other', 'count_families', 'has_secondary_use', \n",
    "'land_surface_condition_n', 'land_surface_condition_o', 'foundation_type_h', 'foundation_type_r', 'foundation_type_u', 'roof_type_n', \n",
    "'roof_type_q', 'roof_type_x', 'ground_floor_type_f', 'ground_floor_type_v', 'ground_floor_type_x', 'other_floor_type_q', 'position_s',\n",
    "'plan_configuration_u']\n",
    "\n",
    "print(len(features))\n",
    "\n",
    "\n",
    "# Find out which features are created through one-hot-encoding\n",
    "import pandas as pd\n",
    "\n",
    "# Load the original data\n",
    "original_data = pd.read_csv('train_values.csv')\n",
    "\n",
    "# Get the original feature names\n",
    "original_features = original_data.columns\n",
    "\n",
    "# Check which features are not in the original data\n",
    "new_features = [feature for feature in features if feature not in original_features]\n",
    "\n",
    "# Print the new features\n",
    "print(\"New features:\\n\", new_features)\n",
    "\n",
    "\n",
    "#Manually remove the one-hot-encoding that pd.get_dummies() used on categorial \n",
    "features_before_dummies = ['geo_level_1_id', 'geo_level_2_id', 'geo_level_3_id', 'count_floors_pre_eq', 'age', 'area_percentage', 'height_percentage',\n",
    "'has_superstructure_adobe_mud', 'has_superstructure_mud_mortar_stone', 'has_superstructure_stone_flag', 'has_superstructure_cement_mortar_stone',\n",
    "'has_superstructure_mud_mortar_brick', 'has_superstructure_cement_mortar_brick', 'has_superstructure_timber', 'has_superstructure_bamboo', \n",
    "'has_superstructure_rc_non_engineered', 'has_superstructure_rc_engineered', 'has_superstructure_other', 'count_families', 'has_secondary_use', \n",
    "'land_surface_condition', 'foundation_type', 'roof_type', 'ground_floor_type','other_floor_type', 'position','plan_configuration']\n",
    "\n",
    "print(len(features_before_dummies))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Analysis from ChatGPT-4 (second way to find best features):\n",
    "\n",
    "The categorical variables in the features dataset have been successfully encoded. Now, let's look at the correlation of these features with the `damage_grade`:\n",
    "\n",
    "#### Correlation with `damage_grade`\n",
    "The correlation values range between -1 and 1. A value closer to 1 indicates a strong positive correlation, meaning that as the feature increases, the `damage_grade` tends to increase. Conversely, a value closer to -1 indicates a strong negative correlation, where an increase in the feature leads to a decrease in `damage_grade`. Values around 0 imply weak or no linear correlation.\n",
    "\n",
    "#### Top Positively Correlated Features:\n",
    "- `has_superstructure_mud_mortar_stone`\n",
    "- `count_floors_pre_eq`\n",
    "- Other features like `legal_ownership_status`, `has_superstructure_stone_flag`, etc., also show positive correlation but to a lesser extent.\n",
    "\n",
    "#### Top Negatively Correlated Features:\n",
    "- `has_superstructure_cement_mortar_brick`\n",
    "- `ground_floor_type`\n",
    "- `has_superstructure_rc_engineered`\n",
    "- Other features like `roof_type`, `has_superstructure_rc_non_engineered`, etc., also show negative correlation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the selected features based on the correlation threshold (of 0.05):\n",
    "\n",
    "1. `has_superstructure_mud_mortar_stone`\n",
    "2. `count_floors_pre_eq`\n",
    "3. `legal_ownership_status`\n",
    "4. `has_superstructure_stone_flag`\n",
    "5. `count_families`\n",
    "6. `has_superstructure_adobe_mud`\n",
    "7. `position`\n",
    "8. `has_superstructure_cement_mortar_stone`\n",
    "9. `has_superstructure_bamboo`\n",
    "10. `has_superstructure_timber`\n",
    "11. `geo_level_1_id`\n",
    "12. `has_secondary_use`\n",
    "13. `has_secondary_use_rental`\n",
    "14. `has_secondary_use_hotel`\n",
    "15. `foundation_type`\n",
    "16. `area_percentage`\n",
    "17. `has_superstructure_rc_non_engineered`\n",
    "18. `roof_type`\n",
    "19. `has_superstructure_rc_engineered`\n",
    "20. `ground_floor_type`\n",
    "21. `has_superstructure_cement_mortar_brick`\n",
    "\n",
    "These features were chosen because they have a correlation with the target variable `damage_grade` greater than the specified threshold of 0.05 (in absolute value). You can use these features for building your predictive model. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "features2 = [\n",
    "    \"has_superstructure_mud_mortar_stone\",\n",
    "    \"count_floors_pre_eq\",\n",
    "    \"legal_ownership_status\",\n",
    "    \"has_superstructure_stone_flag\",\n",
    "    \"count_families\",\n",
    "    \"has_superstructure_adobe_mud\",\n",
    "    \"position\",\n",
    "    \"has_superstructure_cement_mortar_stone\",\n",
    "    \"has_superstructure_bamboo\",\n",
    "    \"has_superstructure_timber\",\n",
    "    \"geo_level_1_id\",\n",
    "    \"has_secondary_use\",\n",
    "    \"has_secondary_use_rental\",\n",
    "    \"has_secondary_use_hotel\",\n",
    "    \"foundation_type\",\n",
    "    \"area_percentage\",\n",
    "    \"has_superstructure_rc_non_engineered\",\n",
    "    \"roof_type\",\n",
    "    \"has_superstructure_rc_engineered\",\n",
    "    \"ground_floor_type\",\n",
    "    \"has_superstructure_cement_mortar_brick\"\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Third, I will try SelectKBest to find best features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.feature_selection import SelectKBest, f_classif\n",
    "\n",
    "\n",
    "# # Create the SelectKBest with the f_classif function. You can set the parameter \"k\" equal to a number if you want to limit the amount of features\n",
    "# selector = SelectKBest(f_classif, k = 20) # Getting the 20 best features\n",
    "\n",
    "# # Assuming X is your feature set and y is the target value\n",
    "# X = train_X.drop('building_id', axis=1)\n",
    "# X = pd.get_dummies(X)\n",
    "\n",
    "# y = train_Y.drop('building_id', axis=1)\n",
    "# y = np.ravel(y) # converting dataframe to a one-dimensional array using the ravel function from numpy\n",
    "\n",
    "\n",
    "# # Fit the selector to the data\n",
    "# selector.fit(X, y)\n",
    "\n",
    "# # Get the boolean mask of the selected features\n",
    "# mask = selector.get_support()\n",
    "\n",
    "# # Get the names of the selected features\n",
    "# selected_features = X.columns[mask]\n",
    "\n",
    "# # Convert the Index object to a list\n",
    "# features3 = selected_features.tolist()\n",
    "\n",
    "# print(features3)\n",
    "\n",
    "# # ['geo_level_1_id', 'count_floors_pre_eq', 'area_percentage', 'has_superstructure_mud_mortar_stone', 'has_superstructure_cement_mortar_brick', \n",
    "# #  'has_superstructure_rc_non_engineered', 'has_superstructure_rc_engineered', 'has_secondary_use_hotel', 'has_secondary_use_rental', \n",
    "# #  'foundation_type_i', 'foundation_type_r', 'foundation_type_u', 'foundation_type_w', 'roof_type_n', 'roof_type_x', 'ground_floor_type_f', \n",
    "# #  'ground_floor_type_v', 'other_floor_type_j', 'other_floor_type_q', 'other_floor_type_s']\n",
    "\n",
    "\n",
    "# # Get the scores\n",
    "# scores = selector.scores_\n",
    "\n",
    "# # Create a DataFrame with the scores\n",
    "# features_scores = pd.DataFrame({'Feature': X.columns, 'Score': scores})\n",
    "\n",
    "# # Sort the DataFrame by score in descending order\n",
    "# features_scores = features_scores.sort_values(by='Score', ascending=False)\n",
    "\n",
    "# # print the best 20 features\n",
    "# print(features_scores[0:19])\n",
    "\n",
    "# #                                    Feature         Score\n",
    "# # 35                       foundation_type_r  23787.275036\n",
    "# # 43                     ground_floor_type_v  20782.933584\n",
    "# # 40                             roof_type_x  16891.038184\n",
    "# # 8      has_superstructure_mud_mortar_stone  16490.386507\n",
    "# # 34                       foundation_type_i  16385.772905\n",
    "# # 12  has_superstructure_cement_mortar_brick  11120.193268\n",
    "# # 48                      other_floor_type_s  10507.484572\n",
    "# # 41                     ground_floor_type_f  10151.525359\n",
    "# # 16        has_superstructure_rc_engineered   7757.593854\n",
    "# # 47                      other_floor_type_q   7378.599061\n",
    "# # 15    has_superstructure_rc_non_engineered   4721.916051\n",
    "# # 37                       foundation_type_w   4568.674306\n",
    "# # 46                      other_floor_type_j   4533.708398\n",
    "# # 36                       foundation_type_u   2972.409108\n",
    "# # 0                           geo_level_1_id   2657.791274\n",
    "# # 3                      count_floors_pre_eq   2544.836052\n",
    "# # 5                          area_percentage   2529.046730\n",
    "# # 38                             roof_type_n   1776.396178\n",
    "# # 21                 has_secondary_use_hotel   1537.672773\n",
    "\n",
    "# # ...\n",
    "\n",
    "# # 22                has_secondary_use_rental   1342.099336\n",
    "# # 64                legal_ownership_status_a   1166.606551\n",
    "# # 19                       has_secondary_use    841.802928\n",
    "# # 39                             roof_type_q    761.885856\n",
    "# # 7             has_superstructure_adobe_mud    739.412821\n",
    "# # 13               has_superstructure_timber    659.199014\n",
    "# # 9            has_superstructure_stone_flag    576.438023\n",
    "# # 14               has_superstructure_bamboo    538.551492\n",
    "# # 66                legal_ownership_status_v    536.308634\n",
    "# # 11     has_superstructure_mud_mortar_brick    531.784659\n",
    "# # 63                    plan_configuration_u    515.087147\n",
    "# # 10  has_superstructure_cement_mortar_stone    478.844199\n",
    "# # 18                          count_families    476.562914\n",
    "# # 56                    plan_configuration_d    378.234531\n",
    "# # 53                              position_t    373.594539\n",
    "# # 6                        height_percentage    370.173817\n",
    "# # 20           has_secondary_use_agriculture    289.462856\n",
    "# # 1                           geo_level_2_id    264.447807\n",
    "# # 49                      other_floor_type_x    244.432657\n",
    "# # 4                                      age    219.626253\n",
    "# # 33                       foundation_type_h    209.425818\n",
    "# # 32                land_surface_condition_t    201.698101\n",
    "# # 30                land_surface_condition_n    182.152148\n",
    "# # 61                    plan_configuration_q    165.370472\n",
    "# # 23           has_secondary_use_institution    146.731486\n",
    "# # 17                has_superstructure_other    142.014204\n",
    "# # 50                              position_j    136.783490\n",
    "# # 67                legal_ownership_status_w    116.039402\n",
    "# # 52                              position_s    110.038831\n",
    "# # 55                    plan_configuration_c     75.423022\n",
    "# # 51                              position_o     75.104323\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "features3 = ['geo_level_1_id', 'count_floors_pre_eq', 'area_percentage', 'has_superstructure_mud_mortar_stone', 'has_superstructure_cement_mortar_brick', \n",
    " 'has_superstructure_rc_non_engineered', 'has_superstructure_rc_engineered', 'has_secondary_use_hotel', 'has_secondary_use_rental', \n",
    " 'foundation_type_i', 'foundation_type_r', 'foundation_type_u', 'foundation_type_w', 'roof_type_n', 'roof_type_x', 'ground_floor_type_f', \n",
    " 'ground_floor_type_v', 'other_floor_type_j', 'other_floor_type_q', 'other_floor_type_s']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check to see if there are any missing values in the data. If so, we have to do imputation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of missing values in train_X: 0\n",
      "Number of missing values in train_Y: 0\n",
      "Number of missing values in test_values: 0\n"
     ]
    }
   ],
   "source": [
    "missing_train_X = train_X.isnull().sum().sum()\n",
    "print(\"Number of missing values in train_X:\", missing_train_X)\n",
    "\n",
    "missing_train_Y = train_Y.isnull().sum().sum()\n",
    "print(\"Number of missing values in train_Y:\", missing_train_Y)\n",
    "\n",
    "missing_test_values = test_values.isnull().sum().sum()\n",
    "print(\"Number of missing values in test_values:\", missing_test_values )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have 0 missing values in each dataframe, we don't have to do imputation!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, we have 3 different list of features (features, features2, and features3), which I found using RFE. Data Analysis ChatGPT-4, and SelectKBest respectively\n",
    "\n",
    "#### Now, I have to turn one-hot-encode the data using pd.get_dummies, and I'll be creating 3 seperate train_X, one for each list of possible best features. And also on the test_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. testX1 for the RFE features\n",
    "# Since the features from RFE are the one-hot-encoded features, we have to apply features after doing pd.get_dummies()\n",
    "\n",
    "trainX1 = pd.get_dummies(train_X)\n",
    "trainX1 = trainX1[features]\n",
    "\n",
    "\n",
    "# 2. testX2 for the Data Analysis ChatGPT-4\n",
    "# Since the features from Data Analysis are from the original feature set, we have to apply the features before doing pd.get_dummies()\n",
    "\n",
    "trainX2 = train_X[features2]\n",
    "trainX2 = pd.get_dummies(trainX2)\n",
    "\n",
    "\n",
    "\n",
    "# 3. testX3 for the SelectKBest\n",
    "# Since the features from RFE are the one-hot-encoded features, we have to apply features after doing pd.get_dummies()\n",
    "\n",
    "trainX3 = pd.get_dummies(train_X)\n",
    "trainX3 = trainX3[features3]\n",
    "\n",
    "\n",
    "\n",
    "# 4. Do pd.get_dummies() on test data. I will create a seperate test_data for each feature selection, since each test_data needs to have a certain set of features\n",
    "\n",
    "test_data1 = pd.get_dummies(test_values)\n",
    "test_data1 = test_data1[features + ['building_id'] ]\n",
    "\n",
    "test_data2 = test_values[features2 + ['building_id'] ]\n",
    "test_data2 = pd.get_dummies(test_data2)\n",
    "\n",
    "test_data3 = pd.get_dummies(test_values)\n",
    "test_data3 = test_data3[features3 + ['building_id'] ]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### It's important to do pd.get_dummies() before doing the train_valid_test split. Now we can do the split\n",
    "I have to do train_valid_test split three times, one for each different train_X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 1. train_valid_test split for train_X1\n",
    "train_X1, test_X1, train_Y1, test_Y1 = train_test_split(trainX1, train_Y, test_size=0.3, random_state = 42) # split into training (70%) and a test set (30%)\n",
    "\n",
    "valid_X1, test_X1, valid_Y1, test_Y1 = train_test_split(test_X1, test_Y1, test_size = 0.5, random_state = 42) # split test set into a validation (15%) and test set (15%)\n",
    "\n",
    "\n",
    "# 2. train_valid_test split for train_X2\n",
    "train_X2, test_X2, train_Y2, test_Y2 = train_test_split(trainX2, train_Y, test_size=0.3, random_state = 42) # split into training (70%) and a test set (30%)\n",
    "\n",
    "valid_X2, test_X2, valid_Y2, test_Y2 = train_test_split(test_X2, test_Y2, test_size = 0.5, random_state = 42) # split test set into a validation (15%) and test set (15%)\n",
    "\n",
    "\n",
    "# 3. train_valid_test split for train_X3\n",
    "train_X3, test_X3, train_Y3, test_Y3 = train_test_split(trainX3, train_Y, test_size=0.3, random_state = 42) # split into training (70%) and a test set (30%)\n",
    "\n",
    "valid_X3, test_X3, valid_Y3, test_Y3 = train_test_split(test_X3, test_Y3, test_size = 0.5, random_state = 42) # split test set into a validation (15%) and test set (15%)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, we can normalize the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Creating the Normalizing Scaler**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many different Normalization Scalers:\n",
    "\n",
    "1. **MinMaxScaler**: This scaler scales and translates each feature individually such that it is in the given range on the training set, e.g., between zero and one.\n",
    "\n",
    "\n",
    "2. **StandardScaler**: This scaler standardizes features by removing the mean and scaling to unit variance.\n",
    "\n",
    "\n",
    "3. **RobustScaler**: This scaler scales features using statistics that are robust to outliers. It uses the Interquartile Range (IQR) to scale the data, making it a better choice for when the data has outliers.\n",
    "\n",
    "\n",
    "4. **Normalizer**: This scaler scales individual samples to have unit norm. This scaler works on the rows, not the columns!\n",
    "\n",
    "\n",
    "5. **MaxAbsScaler**: This scaler scales and translates each feature individually such that the maximal absolute value of each feature in the training set will be 1.0. It does not shift/center the data, and thus does not destroy any sparsity.\n",
    "\n",
    "\n",
    "Remember, the choice of scaler can depend on your specific dataset and the machine learning algorithm that you're using."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I've actually decided to not do normalization since almost all the columns are categorial columns, and the non-categorial columns are mostly normalized already in the dataset, so there's no need"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I've actually decided to not do normalization since almost all the columns are categorial columns, and the non-categorial columns are mostly normalized already in the dataset, so there's no need"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try LightGBMClassifier for Model 1 (Model 1 uses features1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import f1_score\n",
    "# from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "# # LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# # So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "# train_Y1_temp = train_Y1['damage_grade'] - 1  \n",
    "# valid_Y1_temp = valid_Y1['damage_grade'] - 1\n",
    "# test_Y1_temp = test_Y1['damage_grade'] - 1\n",
    "\n",
    "# # Define the model\n",
    "# model1 = LGBMClassifier(boosting_type='gbdt', \n",
    "#                        objective='multiclass', \n",
    "#                        num_class=3, \n",
    "#                        metric='multi_logloss', \n",
    "#                        seed=42,   # Control randomness\n",
    "#                     #    bagging_fraction=1, # Control randomness\n",
    "#                     #    bagging_freq=0,  # Control randomness\n",
    "#                     #    feature_fraction=1,  # Control randomness\n",
    "#                        deterministic=True,  # Control randomness\n",
    "#                        verbose=0)\n",
    "\n",
    "# # Train the model\n",
    "# model1.fit(train_X1, train_Y1_temp, eval_set=[(valid_X1, valid_Y1_temp)])\n",
    "\n",
    "# # Make predictions\n",
    "# y_pred = model1.predict(test_X1)\n",
    "\n",
    "# print(y_pred)\n",
    "\n",
    "# # Print the F1 score, since this is what the competition uses as its scoring system\n",
    "# print('The micro-averaged F1 score of prediction is:', f1_score(test_Y1_temp, y_pred, average='micro'))\n",
    "\n",
    "# # Print the accuracy\n",
    "# print('The accuracy of prediction is:', accuracy_score(test_Y1_temp, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "# params = model1.get_params()\n",
    "\n",
    "# print(\"Hyperparamters: \")\n",
    "# for key, value in params.items():\n",
    "#     print(key, \":\", value)\n",
    "\n",
    "\n",
    "# # Save hyperparameters in a JSON\n",
    "# import json\n",
    "\n",
    "# with open('hyperparameters_model1.json', 'w') as f:\n",
    "#     json.dump(params, f)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model1)\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model1.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 1. Score: 0.7112"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X1, valid_X1, test_X1])\n",
    "# full_Y_temp = pd.concat([train_Y1_temp, valid_Y1_temp, test_Y1_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model1.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model1, 'saved_LGBM_model1.joblib)\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model1 = load('saved_LGBM_model1.joblib')\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data1['building_id']\n",
    "# competition_test_X = test_data1.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model1.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission1.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Model 2, which is the exact same thing as Model 1 but with features2 instead of features1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1 0 ... 2 1 1]\n",
      "The micro-averaged F1 score of prediction is: 0.6803867897981632\n",
      "The accuracy of prediction is: 0.6803867897981632\n"
     ]
    }
   ],
   "source": [
    "from lightgbm import LGBMClassifier\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "# LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "train_Y2_temp = train_Y2['damage_grade'] - 1  \n",
    "valid_Y2_temp = valid_Y2['damage_grade'] - 1\n",
    "test_Y2_temp = test_Y2['damage_grade'] - 1\n",
    "\n",
    "# Define the model\n",
    "model2 = LGBMClassifier(boosting_type='gbdt', \n",
    "                       objective='multiclass', \n",
    "                       num_class=3, \n",
    "                       metric='multi_logloss', \n",
    "                       seed=42,   # Control randomness\n",
    "                    #    bagging_fraction=1, # Control randomness\n",
    "                    #    bagging_freq=0,  # Control randomness\n",
    "                    #    feature_fraction=1,  # Control randomness\n",
    "                       deterministic=True,  # Control randomness\n",
    "                       verbose=0)\n",
    "\n",
    "# Train the model\n",
    "model2.fit(train_X2, train_Y2_temp, eval_set=[(valid_X2, valid_Y2_temp)])\n",
    "\n",
    "# Make predictions\n",
    "y_pred = model2.predict(test_X2)\n",
    "\n",
    "print(y_pred)\n",
    "\n",
    "# Print the F1 score, since this is what the competition uses as its scoring system\n",
    "print('The micro-averaged F1 score of prediction is:', f1_score(test_Y2_temp, y_pred, average='micro'))\n",
    "\n",
    "# Print the accuracy\n",
    "print('The accuracy of prediction is:', accuracy_score(test_Y2_temp, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "# params = model2.get_params()\n",
    "\n",
    "# print(\"Hyperparamters: \")\n",
    "# for key, value in params.items():\n",
    "#     print(key, \":\", value)\n",
    "\n",
    "\n",
    "# # Save hyperparameters in a JSON\n",
    "# import json\n",
    "\n",
    "# with open('hyperparameters_model2.json', 'w') as f: # Make sure to change the name to match whatever model you are using\n",
    "#     json.dump(params, f)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model2)\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model2.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 2. Score: 68.13"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X2, valid_X2, test_X2])\n",
    "# full_Y_temp = pd.concat([train_Y2_temp, valid_Y2_temp, test_Y2_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model2.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model2, 'saved_LGBM_model2.joblib')\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model2 = load('saved_LGBM_model2.joblib')\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data2['building_id']\n",
    "# competition_test_X = test_data2.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model2.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission2.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Model 3, which is the exact same thing as Model 1 but with features3 instead of features1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import f1_score\n",
    "# from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "# # LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# # So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "# train_Y3_temp = train_Y3['damage_grade'] - 1  \n",
    "# valid_Y3_temp = valid_Y3['damage_grade'] - 1\n",
    "# test_Y3_temp = test_Y3['damage_grade'] - 1\n",
    "\n",
    "# # Define the model\n",
    "# model3 = LGBMClassifier(boosting_type='gbdt', \n",
    "#                        objective='multiclass', \n",
    "#                        num_class=3, \n",
    "#                        metric='multi_logloss', \n",
    "#                        seed=42,   # Control randomness\n",
    "#                     #    bagging_fraction=1, # Control randomness\n",
    "#                     #    bagging_freq=0,  # Control randomness\n",
    "#                     #    feature_fraction=1,  # Control randomness\n",
    "#                        deterministic=True,  # Control randomness\n",
    "#                        verbose=0)\n",
    "\n",
    "# # Train the model\n",
    "# model3.fit(train_X3, train_Y3_temp, eval_set=[(valid_X3, valid_Y3_temp)])\n",
    "\n",
    "# # Make predictions\n",
    "# y_pred = model3.predict(test_X3)\n",
    "\n",
    "# print(y_pred)\n",
    "\n",
    "# # Print the F1 score, since this is what the competition uses as its scoring system\n",
    "# print('The micro-averaged F1 score of prediction is:', f1_score(test_Y3_temp, y_pred, average='micro'))\n",
    "\n",
    "# # Print the accuracy\n",
    "# print('The accuracy of prediction is:', accuracy_score(test_Y3_temp, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "# params = model3.get_params()\n",
    "\n",
    "# print(\"Hyperparamters: \")\n",
    "# for key, value in params.items():\n",
    "#     print(key, \":\", value)\n",
    "\n",
    "\n",
    "# # Save hyperparameters in a JSON\n",
    "# import json\n",
    "\n",
    "# with open('hyperparameters_model3.json', 'w') as f: # Make sure to change the name to match whatever model you are using\n",
    "#     json.dump(params, f)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model3)\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model3.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 3. Score: 67.73"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X3, valid_X3, test_X3])\n",
    "# full_Y_temp = pd.concat([train_Y3_temp, valid_Y3_temp, test_Y3_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model3.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model3, 'saved_LGBM_model3.joblib')\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model3 = load('saved_LGBM_model3.joblib')\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data3['building_id']\n",
    "# competition_test_X = test_data3.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model3.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission3.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model 4 will use \"features\" (same as Model 1) but with Keras Tuner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When tuning hyperparameters for the `LGBMClassifier`, there are several key hyperparameters that typically have the biggest impact on performance:\n",
    "\n",
    "1. `num_leaves`: This is the main parameter to control the complexity of the tree model. Theoretically, having more leaves can capture more information about the data, but it can also lead to overfitting. \n",
    "\n",
    "2. `min_data_in_leaf`: This is a very important parameter to prevent overfitting in a leaf-wise tree. Its optimal value depends on the number of training samples and `num_leaves`. Setting it to a large value can avoid growing too deep a tree, but may cause underfitting.\n",
    "\n",
    "3. `max_depth`: This parameter is used to handle model overfitting. If you feel that your model is overfitted, you can try to limit `max_depth`.\n",
    "\n",
    "4. `learning_rate`: This determines the impact of each tree on the final outcome. GBM works by starting with an initial estimate and improving it gradually, and the learning rate controls how much each tree contributes to this.\n",
    "\n",
    "5. `n_estimators`: This is the number of boosted trees to fit. More trees can model more complex patterns, but can also lead to overfitting.\n",
    "\n",
    "6. `subsample`: This is the fraction of samples to be used for fitting the individual base learners. If it's less than 1.0, that leads to Stochastic Gradient Boosting.\n",
    "\n",
    "7. `colsample_bytree`: This is the subsample ratio of columns when constructing each tree.\n",
    "\n",
    "8. `reg_alpha` and `reg_lambda`: These are regularization parameters. They can help prevent overfitting.\n",
    "\n",
    "Remember, the best hyperparameters can depend on your specific dataset, so it's always a good idea to use something like grid search or random search to find the best values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import GridSearchCV\n",
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import make_scorer, f1_score, accuracy_score\n",
    "\n",
    "\n",
    "\n",
    "# # LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# # So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "# train_Y1_temp = train_Y1['damage_grade'] - 1  \n",
    "# valid_Y1_temp = valid_Y1['damage_grade'] - 1\n",
    "# test_Y1_temp = test_Y1['damage_grade'] - 1\n",
    "\n",
    "# # Concatenate training and validation sets so that I have more data to fit the model on during GridSearchCV\n",
    "# train_X1_full = pd.concat([train_X1, valid_X1])\n",
    "# train_Y1_temp_full = pd.concat([train_Y1_temp, valid_Y1_temp])\n",
    "\n",
    "# # Define the model\n",
    "# model4 = LGBMClassifier(boosting_type='gbdt', \n",
    "#                        objective='multiclass', \n",
    "#                        num_class=3, \n",
    "#                        metric='multi_logloss', \n",
    "#                        random_state=42)\n",
    "\n",
    "# # Define the parameter grid focusing on most important hyperparameters\n",
    "# param_grid = {\n",
    "#     'num_leaves': list(range(150, 300, 50)), # 75 folds took 2 min 43 sec\n",
    "#     'max_depth': [-1], # -1 is the default\n",
    "#     'min_data_in_leaf': [20], # 20 is the default \n",
    "#     'n_estimators': list(range(50, 301, 50)), #default value is 100\n",
    "#     'min_child_samples': list(range(5, 20, 5)), # default value is 20\n",
    "#     # 'learning_rate': [i/100 for i in range(1, 11)], # default is 0.1\n",
    "#     # 'subsample': [i/10 for i in range(1, 11)], # default value is 1.0\n",
    "#     # 'colsample_bytree': [i/10 for i in range(1, 5)], # default value is 1.0\n",
    "# }\n",
    "\n",
    "# # Define the grid search\n",
    "# grid = GridSearchCV(model4, \n",
    "#                     param_grid, \n",
    "#                     verbose=3, \n",
    "#                     cv=2, \n",
    "#                     n_jobs=-1, \n",
    "#                     scoring=make_scorer(f1_score, average='micro'))\n",
    "\n",
    "# # Fit the grid search on the full data\n",
    "# grid.fit(train_X1_full, train_Y1_temp_full)\n",
    "\n",
    "# # Print the best parameters and the best score\n",
    "# print(f'Best parameters: {grid.best_params_}')\n",
    "# print(f'Best score: {grid.best_score_}')\n",
    "\n",
    "# model4 = grid.best_estimator_\n",
    "\n",
    "# # Use the best estimator to make predictions\n",
    "# y_pred = model4.predict(test_X1)\n",
    "\n",
    "# # Print the F1 score and the accuracy\n",
    "# print('The micro-averaged F1 score of prediction is:', f1_score(test_Y1_temp, y_pred, average='micro'))\n",
    "# print('The accuracy of prediction is:', accuracy_score(test_Y1_temp, y_pred))\n",
    "\n",
    "# # 75 fits. Took 2 min and 43 sec. THIS GOT 0.7405\n",
    "# # Best parameters: {'num_leaves': 252}\n",
    "# # Best score: 0.7361067271692862\n",
    "# # The micro-averaged F1 score of prediction is: 0.7364866593333503\n",
    "# # The accuracy of prediction is: 0.7364866593333503\n",
    "\n",
    "# # 240 fits. Took 6 min and 44 sec. This got 0.7358. \n",
    "# # Best parameters: {'max_depth': 10, 'min_data_in_leaf': 20, 'num_leaves': 250}\n",
    "# # Best score: 0.7282244593923525\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7317541121997391\n",
    "# # The accuracy of prediction is: 0.7317541121997391\n",
    "\n",
    "# # 108 fits. Took 14 min and 15 seconds, probably because max_depth is -1 so that means it's unlimited. This got a 0.7449\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 200}\n",
    "# # Best score: 0.7348516996975305\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7434703640224093\n",
    "# # The accuracy of prediction is: 0.7434703640224093\n",
    "\n",
    "# # Hyperparamters: \n",
    "# # boosting_type : gbdt\n",
    "# # class_weight : None\n",
    "# # colsample_bytree : 1.0\n",
    "# # importance_type : split\n",
    "# # learning_rate : 0.1\n",
    "# # max_depth : -1\n",
    "# # min_child_samples : 5\n",
    "# # min_child_weight : 0.001\n",
    "# # min_split_gain : 0.0\n",
    "# # n_estimators : 300\n",
    "# # n_jobs : None\n",
    "# # num_leaves : 200\n",
    "# # objective : multiclass\n",
    "# # random_state : 42\n",
    "# # reg_alpha : 0.0\n",
    "# # reg_lambda : 0.0\n",
    "# # subsample : 1.0\n",
    "# # subsample_for_bin : 200000\n",
    "# # subsample_freq : 0\n",
    "# # num_class : 3\n",
    "# # metric : multi_logloss\n",
    "# # min_data_in_leaf : 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "# params = model4.get_params()\n",
    "\n",
    "# print(\"Hyperparamters: \")\n",
    "# for key, value in params.items():\n",
    "#     print(key, \":\", value)\n",
    "\n",
    "\n",
    "# # Save hyperparameters in a JSON\n",
    "# import json\n",
    "\n",
    "# with open('hyperparameters_model4.json', 'w') as f:\n",
    "#     json.dump(params, f)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model4)\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model4.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X1, valid_X1, test_X1])\n",
    "# full_Y_temp = pd.concat([train_Y1_temp, valid_Y1_temp, test_Y1_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model4.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model4, 'saved_LGBM_model4.joblib')\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model4 = load('saved_LGBM_model4.joblib')\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data1['building_id']\n",
    "# competition_test_X = test_data1.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model4.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission4.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For Model 5, we do the same thing as Model 4 but fix the imbalance class problem, which there are a lot more damage_grade of 2 than 3, and a lot more 3 than 1. \n",
    "\n",
    "Use class_weight hyperparameter and use label_analysis.py to find class weights and look into other things as well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import GridSearchCV\n",
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import make_scorer, f1_score, accuracy_score\n",
    "\n",
    "\n",
    "\n",
    "# # LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# # So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "# train_Y1_temp = train_Y1['damage_grade'] - 1  \n",
    "# valid_Y1_temp = valid_Y1['damage_grade'] - 1\n",
    "# test_Y1_temp = test_Y1['damage_grade'] - 1\n",
    "\n",
    "# # Concatenate training and validation sets so that I have more data to fit the model on during GridSearchCV\n",
    "# train_X1_full = pd.concat([train_X1, valid_X1])\n",
    "# train_Y1_temp_full = pd.concat([train_Y1_temp, valid_Y1_temp])\n",
    "\n",
    "# # Define the model\n",
    "# model5 = LGBMClassifier(boosting_type='gbdt', \n",
    "#                        objective='multiclass', \n",
    "#                        num_class=3, \n",
    "#                        metric='multi_logloss', \n",
    "#                        # class_weight={0: 1, 1: 3.5, 2: 2.5}, # I got these values through label_analysis.py\n",
    "#                        random_state=42)\n",
    "\n",
    "# # Define the parameter grid focusing on most important hyperparameters\n",
    "# param_grid = {\n",
    "#     'num_leaves': list(range(200, 400, 50)), # 75 folds took 2 min 43 sec\n",
    "#     'max_depth': [-1], # -1 is the default\n",
    "#     'min_data_in_leaf': [20], # 20 is the default \n",
    "#     'n_estimators': list(range(300, 500, 50)), #default value is 100\n",
    "#     'min_child_samples': list(range(3, 6, 1)), # default value is 20\n",
    "#     # 'learning_rate': [i/100 for i in range(1, 11)], # default is 0.1\n",
    "#     # 'subsample': [i/10 for i in range(1, 11)], # default value is 1.0\n",
    "#     # 'colsample_bytree': [i/10 for i in range(1, 5)], # default value is 1.0\n",
    "# }\n",
    "\n",
    "# # Define the grid search\n",
    "# grid = GridSearchCV(model5, \n",
    "#                     param_grid, \n",
    "#                     verbose=3, \n",
    "#                     cv=2, \n",
    "#                     n_jobs=-1, \n",
    "#                     scoring=make_scorer(f1_score, average='micro'))\n",
    "\n",
    "# # Fit the grid search on the full data\n",
    "# grid.fit(train_X1_full, train_Y1_temp_full)\n",
    "\n",
    "# # Print the best parameters and the best score\n",
    "# print(f'Best parameters: {grid.best_params_}')\n",
    "# print(f'Best score: {grid.best_score_}')\n",
    "\n",
    "# model5 = grid.best_estimator_\n",
    "\n",
    "# # Use the best estimator to make predictions\n",
    "# y_pred = model5.predict(test_X1)\n",
    "\n",
    "# # Print the F1 score and the accuracy\n",
    "# print('The micro-averaged F1 score of prediction is:', f1_score(test_Y1_temp, y_pred, average='micro'))\n",
    "# print('The accuracy of prediction is:', accuracy_score(test_Y1_temp, y_pred))\n",
    "\n",
    "# # 75 fits. Took 2 min and 43 sec. THIS GOT 0.7405\n",
    "# # Best parameters: {'num_leaves': 252}\n",
    "# # Best score: 0.7361067271692862\n",
    "# # The micro-averaged F1 score of prediction is: 0.7364866593333503\n",
    "# # The accuracy of prediction is: 0.7364866593333503\n",
    "\n",
    "# # 240 fits. Took 6 min and 44 sec. This got 0.7358. \n",
    "# # Best parameters: {'max_depth': 10, 'min_data_in_leaf': 20, 'num_leaves': 250}\n",
    "# # Best score: 0.7282244593923525\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7317541121997391\n",
    "# # The accuracy of prediction is: 0.7317541121997391\n",
    "\n",
    "# # 108 fits. Took 14 min and 15 seconds, probably because max_depth is -1 so that means it's unlimited. This got a 0.7449\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 200}\n",
    "# # Best score: 0.7348516996975305\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7434703640224093\n",
    "# # The accuracy of prediction is: 0.7434703640224093\n",
    "\n",
    "# # 108 fits, with class weights {1.5, 10, 3}. Took 6 minutes and 44 sec. Got a score of 0.7057\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7091463139361655\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7054820802742319\n",
    "# # The accuracy of prediction is: 0.7054820802742319\n",
    "\n",
    "# # 108 fits, version 3 with class weights {1, 3, 2}. Took 5 min and 50 sec. Got a score of 0.7348\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7283192632386799\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7331866670077511\n",
    "# # The accuracy of prediction is: 0.7331866670077511\n",
    "\n",
    "# # 108 fits. with class weights {1, 4, 2}. Took 5 min and 24 sec. Got a score of 0.7260\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7218726016884114\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7242843621293905\n",
    "# # The accuracy of prediction is: 0.7242843621293904\n",
    "\n",
    "# # 108 fits. with class weights {1, 3.5, 2.5}. Took 5 min and 24 sec. Got a score of 0.7355\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7282244593923526\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7317285308638818\n",
    "# # The accuracy of prediction is: 0.7317285308638817\n",
    "\n",
    "# # 108 fits. with class weights {0.5, 3.5, 2.5}. Took 5 min and 43 sec.\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7282244593923526\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7317285308638818\n",
    "# # The accuracy of prediction is: 0.7317285308638817\n",
    "\n",
    "# # 216 fits. with no weights. 11 min and 27 sec. Got a score of 0.7446. \n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 5, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 250}\n",
    "# # Best score: 0.7282244593923526\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=5 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7317285308638818\n",
    "# # The accuracy of prediction is: 0.7317285308638817\n",
    "\n",
    "# #96 fits, Took 26 minutes. Got a score of 0.7449.\n",
    "# # Best parameters: {'max_depth': -1, 'min_child_samples': 3, 'min_data_in_leaf': 20, 'n_estimators': 300, 'num_leaves': 200}\n",
    "# # Best score: 0.7348516996975305\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=3 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7434703640224093\n",
    "# # The accuracy of prediction is: 0.7434703640224093\n",
    "\n",
    "\n",
    "# #96 fits, Took 31 minutes. Got a score of 0.7444.\n",
    "# # Best parameters: {'learning_rate': 0.03, 'max_depth': -1, 'min_child_samples': 3, 'min_data_in_leaf': 20, 'n_estimators': 450, 'num_leaves': 400}\n",
    "# # Best score: 0.7347884971333123\n",
    "# # [LightGBM] [Warning] min_data_in_leaf is set=20, min_child_samples=3 will be ignored. Current value: min_data_in_leaf=20\n",
    "# # The micro-averaged F1 score of prediction is: 0.7416540891765367\n",
    "# # The accuracy of prediction is: 0.7416540891765367\n",
    "\n",
    "\n",
    "# # Hyperparamters: \n",
    "# # boosting_type : gbdt\n",
    "# # class_weight : None\n",
    "# # colsample_bytree : 1.0\n",
    "# # importance_type : split\n",
    "# # learning_rate : 0.1\n",
    "# # max_depth : -1\n",
    "# # min_child_samples : 5\n",
    "# # min_child_weight : 0.001\n",
    "# # min_split_gain : 0.0\n",
    "# # n_estimators : 300\n",
    "# # n_jobs : None\n",
    "# # num_leaves : 200\n",
    "# # objective : multiclass\n",
    "# # random_state : 42\n",
    "# # reg_alpha : 0.0\n",
    "# # reg_lambda : 0.0\n",
    "# # subsample : 1.0\n",
    "# # subsample_for_bin : 200000\n",
    "# # subsample_freq : 0\n",
    "# # num_class : 3\n",
    "# # metric : multi_logloss\n",
    "# # min_data_in_leaf : 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "# params = model5.get_params()\n",
    "\n",
    "# print(\"Hyperparamters: \")\n",
    "# for key, value in params.items():\n",
    "#     print(key, \":\", value)\n",
    "\n",
    "\n",
    "# # Save hyperparameters in a JSON\n",
    "# import json\n",
    "\n",
    "# with open('hyperparameters_model5.json', 'w') as f: # Make sure to change the name to match whatever model you are using\n",
    "#     json.dump(params, f)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model5)\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model5.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X1, valid_X1, test_X1])\n",
    "# full_Y_temp = pd.concat([train_Y1_temp, valid_Y1_temp, test_Y1_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model5.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model5, 'saved_LGBM_model5.joblib') # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model5 = load('saved_LGBM_model5.joblib')  # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data1['building_id']\n",
    "# competition_test_X = test_data1.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model5.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission5.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For Model 6, we will use another hyper parameter tuning library: Optuna\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import GridSearchCV\n",
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import make_scorer, f1_score, accuracy_score\n",
    "\n",
    "\n",
    "# # LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# # So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "# train_Y1_temp = train_Y1['damage_grade'] - 1  \n",
    "# valid_Y1_temp = valid_Y1['damage_grade'] - 1\n",
    "# test_Y1_temp = test_Y1['damage_grade'] - 1\n",
    "\n",
    "# # Concatenate training and validation sets so that I have more data to fit the model on during GridSearchCV\n",
    "# train_X1_full = pd.concat([train_X1, valid_X1])\n",
    "# train_Y1_temp_full = pd.concat([train_Y1_temp, valid_Y1_temp])\n",
    "\n",
    "# import optuna\n",
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.metrics import f1_score\n",
    "# from sklearn.model_selection import cross_val_score, StratifiedKFold\n",
    "\n",
    "# def objective(trial):\n",
    "#     params = {\n",
    "#         'boosting_type': 'gbdt',\n",
    "#         'objective': 'multiclass',\n",
    "#         'num_class': 3,\n",
    "#         'metric': 'multi_logloss',\n",
    "#         'random_state': 42,\n",
    "#         'num_leaves': trial.suggest_int('num_leaves', 200, 400),\n",
    "#         #'max_depth': trial.suggest_int('max_depth', -1, 50),\n",
    "#         'min_data_in_leaf': trial.suggest_int('min_data_in_leaf', 10, 30),\n",
    "#         'n_estimators': trial.suggest_int('n_estimators', 200, 500),\n",
    "#         'min_child_samples': trial.suggest_int('min_child_samples', 2, 10),\n",
    "#     }\n",
    "\n",
    "#     model = LGBMClassifier(**params)\n",
    "#     cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=42)\n",
    "#     score = cross_val_score(model, train_X1_full, train_Y1_temp_full, cv=cv, scoring='f1_micro', n_jobs=-1)\n",
    "#     f1 = score.mean()\n",
    "#     return f1\n",
    "\n",
    "# study = optuna.create_study(direction='maximize')\n",
    "# study.optimize(objective, n_trials=100)\n",
    "\n",
    "# print('Best trial: score {}, params {}'.format(study.best_trial.value, study.best_trial.params))\n",
    "\n",
    "# # Best trial: score 0.7399033919859633, params {'num_leaves': 234, 'min_data_in_leaf': 17, 'n_estimators': 313, 'min_child_samples': 2}. Score: 0.7448\n",
    "\n",
    "\n",
    "# # [I 2024-01-07 18:08:11,590] A new study created in memory with name: no-name-f4210ae5-1edd-4ba0-ba88-c79abd4e15a5\n",
    "# # [I 2024-01-07 18:08:30,544] Trial 0 finished with value: 0.7395286911714135 and parameters: {'num_leaves': 206, 'min_data_in_leaf': 18, 'n_estimators': 312, 'min_child_samples': 2}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:08:55,049] Trial 1 finished with value: 0.7392397644502283 and parameters: {'num_leaves': 330, 'min_data_in_leaf': 14, 'n_estimators': 241, 'min_child_samples': 5}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:09:19,086] Trial 2 finished with value: 0.7391449609707464 and parameters: {'num_leaves': 222, 'min_data_in_leaf': 26, 'n_estimators': 386, 'min_child_samples': 3}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:09:44,856] Trial 3 finished with value: 0.7381111478026526 and parameters: {'num_leaves': 237, 'min_data_in_leaf': 28, 'n_estimators': 401, 'min_child_samples': 8}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:10:03,141] Trial 4 finished with value: 0.7393300564160129 and parameters: {'num_leaves': 228, 'min_data_in_leaf': 18, 'n_estimators': 310, 'min_child_samples': 10}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:10:29,937] Trial 5 finished with value: 0.7383729906453952 and parameters: {'num_leaves': 209, 'min_data_in_leaf': 20, 'n_estimators': 469, 'min_child_samples': 7}. Best is trial 0 with value: 0.7395286911714135.\n",
    "# # [I 2024-01-07 18:10:50,266] Trial 6 finished with value: 0.7396641266438571 and parameters: {'num_leaves': 293, 'min_data_in_leaf': 15, 'n_estimators': 260, 'min_child_samples': 7}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:11:17,779] Trial 7 finished with value: 0.7385309951809397 and parameters: {'num_leaves': 250, 'min_data_in_leaf': 25, 'n_estimators': 385, 'min_child_samples': 4}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:11:51,407] Trial 8 finished with value: 0.7362647280974784 and parameters: {'num_leaves': 311, 'min_data_in_leaf': 16, 'n_estimators': 445, 'min_child_samples': 6}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:12:25,125] Trial 9 finished with value: 0.7369373828182345 and parameters: {'num_leaves': 354, 'min_data_in_leaf': 30, 'n_estimators': 380, 'min_child_samples': 6}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:12:40,132] Trial 10 finished with value: 0.7393029637943268 and parameters: {'num_leaves': 278, 'min_data_in_leaf': 10, 'n_estimators': 216, 'min_child_samples': 9}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:13:06,741] Trial 11 finished with value: 0.7372624252496419 and parameters: {'num_leaves': 378, 'min_data_in_leaf': 11, 'n_estimators': 293, 'min_child_samples': 3}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:13:31,783] Trial 12 finished with value: 0.7385987112357685 and parameters: {'num_leaves': 282, 'min_data_in_leaf': 23, 'n_estimators': 287, 'min_child_samples': 8}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:13:57,092] Trial 13 finished with value: 0.7392397677518728 and parameters: {'num_leaves': 265, 'min_data_in_leaf': 14, 'n_estimators': 328, 'min_child_samples': 2}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:14:20,797] Trial 14 finished with value: 0.7384993925740954 and parameters: {'num_leaves': 391, 'min_data_in_leaf': 21, 'n_estimators': 257, 'min_child_samples': 5}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:14:36,322] Trial 15 finished with value: 0.738810888236098 and parameters: {'num_leaves': 306, 'min_data_in_leaf': 18, 'n_estimators': 206, 'min_child_samples': 2}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:14:59,843] Trial 16 finished with value: 0.7390682190148693 and parameters: {'num_leaves': 344, 'min_data_in_leaf': 13, 'n_estimators': 263, 'min_child_samples': 7}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:15:21,777] Trial 17 finished with value: 0.7391043310565978 and parameters: {'num_leaves': 203, 'min_data_in_leaf': 17, 'n_estimators': 344, 'min_child_samples': 4}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:15:50,008] Trial 18 finished with value: 0.7389102109331143 and parameters: {'num_leaves': 256, 'min_data_in_leaf': 21, 'n_estimators': 350, 'min_child_samples': 10}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:16:11,087] Trial 19 finished with value: 0.7386980272072128 and parameters: {'num_leaves': 284, 'min_data_in_leaf': 15, 'n_estimators': 279, 'min_child_samples': 7}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:16:32,058] Trial 20 finished with value: 0.7384587615593986 and parameters: {'num_leaves': 322, 'min_data_in_leaf': 12, 'n_estimators': 240, 'min_child_samples': 8}. Best is trial 6 with value: 0.7396641266438571.\n",
    "# # [I 2024-01-07 18:16:52,813] Trial 21 finished with value: 0.7397273280056272 and parameters: {'num_leaves': 236, 'min_data_in_leaf': 18, 'n_estimators': 315, 'min_child_samples': 10}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:17:11,802] Trial 22 finished with value: 0.7388605516939902 and parameters: {'num_leaves': 200, 'min_data_in_leaf': 19, 'n_estimators': 317, 'min_child_samples': 9}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:17:32,825] Trial 23 finished with value: 0.7393887419841763 and parameters: {'num_leaves': 244, 'min_data_in_leaf': 16, 'n_estimators': 303, 'min_child_samples': 9}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:17:52,295] Trial 24 finished with value: 0.7394113146546292 and parameters: {'num_leaves': 219, 'min_data_in_leaf': 21, 'n_estimators': 328, 'min_child_samples': 5}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:18:11,909] Trial 25 finished with value: 0.7387567149153309 and parameters: {'num_leaves': 270, 'min_data_in_leaf': 23, 'n_estimators': 268, 'min_child_samples': 10}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:18:43,309] Trial 26 finished with value: 0.7371450511785201 and parameters: {'num_leaves': 292, 'min_data_in_leaf': 17, 'n_estimators': 419, 'min_child_samples': 4}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:19:07,403] Trial 27 finished with value: 0.7388470073091064 and parameters: {'num_leaves': 229, 'min_data_in_leaf': 19, 'n_estimators': 363, 'min_child_samples': 3}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:19:23,872] Trial 28 finished with value: 0.7392442799992688 and parameters: {'num_leaves': 259, 'min_data_in_leaf': 15, 'n_estimators': 239, 'min_child_samples': 7}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:19:37,297] Trial 29 finished with value: 0.738273669904909 and parameters: {'num_leaves': 213, 'min_data_in_leaf': 13, 'n_estimators': 244, 'min_child_samples': 6}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:19:56,964] Trial 30 finished with value: 0.7386438564543911 and parameters: {'num_leaves': 342, 'min_data_in_leaf': 14, 'n_estimators': 228, 'min_child_samples': 5}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:20:16,141] Trial 31 finished with value: 0.739108841469747 and parameters: {'num_leaves': 214, 'min_data_in_leaf': 22, 'n_estimators': 335, 'min_child_samples': 5}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:20:37,877] Trial 32 finished with value: 0.7393481136597083 and parameters: {'num_leaves': 232, 'min_data_in_leaf': 20, 'n_estimators': 320, 'min_child_samples': 2}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:21:01,415] Trial 33 finished with value: 0.7393977724097001 and parameters: {'num_leaves': 218, 'min_data_in_leaf': 24, 'n_estimators': 363, 'min_child_samples': 4}. Best is trial 21 with value: 0.7397273280056272.\n",
    "# # [I 2024-01-07 18:21:21,869] Trial 34 finished with value: 0.7398040757088111 and parameters: {'num_leaves': 236, 'min_data_in_leaf': 18, 'n_estimators': 296, 'min_child_samples': 5}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:21:45,576] Trial 35 finished with value: 0.7392217062282677 and parameters: {'num_leaves': 243, 'min_data_in_leaf': 18, 'n_estimators': 277, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:22:03,788] Trial 36 finished with value: 0.739257823772737 and parameters: {'num_leaves': 237, 'min_data_in_leaf': 17, 'n_estimators': 300, 'min_child_samples': 6}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:22:21,188] Trial 37 finished with value: 0.7397860118618267 and parameters: {'num_leaves': 225, 'min_data_in_leaf': 19, 'n_estimators': 256, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:22:42,253] Trial 38 finished with value: 0.7396370370792492 and parameters: {'num_leaves': 296, 'min_data_in_leaf': 19, 'n_estimators': 257, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:23:01,084] Trial 39 finished with value: 0.7393752031631747 and parameters: {'num_leaves': 249, 'min_data_in_leaf': 16, 'n_estimators': 282, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:23:21,776] Trial 40 finished with value: 0.7391855885003737 and parameters: {'num_leaves': 229, 'min_data_in_leaf': 26, 'n_estimators': 309, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:23:41,618] Trial 41 finished with value: 0.73897341094977 and parameters: {'num_leaves': 302, 'min_data_in_leaf': 19, 'n_estimators': 256, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:24:01,502] Trial 42 finished with value: 0.738819920740435 and parameters: {'num_leaves': 317, 'min_data_in_leaf': 20, 'n_estimators': 223, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:24:16,773] Trial 43 finished with value: 0.7390953014259142 and parameters: {'num_leaves': 273, 'min_data_in_leaf': 18, 'n_estimators': 202, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:24:34,954] Trial 44 finished with value: 0.7385084196979746 and parameters: {'num_leaves': 294, 'min_data_in_leaf': 15, 'n_estimators': 250, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:24:53,217] Trial 45 finished with value: 0.7393661685188831 and parameters: {'num_leaves': 254, 'min_data_in_leaf': 17, 'n_estimators': 273, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:25:30,059] Trial 46 finished with value: 0.7350458255371327 and parameters: {'num_leaves': 329, 'min_data_in_leaf': 19, 'n_estimators': 483, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:25:51,635] Trial 47 finished with value: 0.7388921523443045 and parameters: {'num_leaves': 265, 'min_data_in_leaf': 18, 'n_estimators': 291, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:26:14,754] Trial 48 finished with value: 0.7382691577186544 and parameters: {'num_leaves': 371, 'min_data_in_leaf': 16, 'n_estimators': 228, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:26:35,012] Trial 49 finished with value: 0.73902306688725 and parameters: {'num_leaves': 289, 'min_data_in_leaf': 20, 'n_estimators': 264, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:26:59,826] Trial 50 finished with value: 0.7391314206823473 and parameters: {'num_leaves': 312, 'min_data_in_leaf': 22, 'n_estimators': 295, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:27:18,858] Trial 51 finished with value: 0.739528692088537 and parameters: {'num_leaves': 222, 'min_data_in_leaf': 17, 'n_estimators': 312, 'min_child_samples': 3}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:27:38,811] Trial 52 finished with value: 0.7392803984608616 and parameters: {'num_leaves': 221, 'min_data_in_leaf': 17, 'n_estimators': 310, 'min_child_samples': 6}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:27:58,586] Trial 53 finished with value: 0.739068213450987 and parameters: {'num_leaves': 240, 'min_data_in_leaf': 30, 'n_estimators': 288, 'min_child_samples': 3}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:28:20,843] Trial 54 finished with value: 0.7395918922886174 and parameters: {'num_leaves': 209, 'min_data_in_leaf': 15, 'n_estimators': 341, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:28:44,833] Trial 55 finished with value: 0.7388424894366864 and parameters: {'num_leaves': 211, 'min_data_in_leaf': 13, 'n_estimators': 395, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:29:08,785] Trial 56 finished with value: 0.7391133626438114 and parameters: {'num_leaves': 233, 'min_data_in_leaf': 15, 'n_estimators': 366, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:29:34,592] Trial 57 finished with value: 0.7382194945664701 and parameters: {'num_leaves': 276, 'min_data_in_leaf': 21, 'n_estimators': 344, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:29:47,331] Trial 58 finished with value: 0.7381562903310467 and parameters: {'num_leaves': 206, 'min_data_in_leaf': 10, 'n_estimators': 214, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:30:11,807] Trial 59 finished with value: 0.7396641254821673 and parameters: {'num_leaves': 264, 'min_data_in_leaf': 19, 'n_estimators': 328, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:30:30,511] Trial 60 finished with value: 0.7386303098072694 and parameters: {'num_leaves': 262, 'min_data_in_leaf': 19, 'n_estimators': 256, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:30:54,593] Trial 61 finished with value: 0.7384316781089472 and parameters: {'num_leaves': 249, 'min_data_in_leaf': 12, 'n_estimators': 340, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:31:22,290] Trial 62 finished with value: 0.7385761389321649 and parameters: {'num_leaves': 283, 'min_data_in_leaf': 14, 'n_estimators': 356, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:31:48,936] Trial 63 finished with value: 0.7383775004471289 and parameters: {'num_leaves': 298, 'min_data_in_leaf': 20, 'n_estimators': 324, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:32:12,371] Trial 64 finished with value: 0.7389237534837512 and parameters: {'num_leaves': 226, 'min_data_in_leaf': 18, 'n_estimators': 381, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:32:32,286] Trial 65 finished with value: 0.7391720491290981 and parameters: {'num_leaves': 206, 'min_data_in_leaf': 16, 'n_estimators': 329, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:32:50,534] Trial 66 finished with value: 0.7381698369781682 and parameters: {'num_leaves': 200, 'min_data_in_leaf': 22, 'n_estimators': 303, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:33:10,799] Trial 67 finished with value: 0.7393842236837654 and parameters: {'num_leaves': 268, 'min_data_in_leaf': 18, 'n_estimators': 270, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:33:32,051] Trial 68 finished with value: 0.7388786069811556 and parameters: {'num_leaves': 306, 'min_data_in_leaf': 19, 'n_estimators': 237, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:33:59,494] Trial 69 finished with value: 0.7379982851840866 and parameters: {'num_leaves': 236, 'min_data_in_leaf': 16, 'n_estimators': 420, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:34:20,447] Trial 70 finished with value: 0.738883121551931 and parameters: {'num_leaves': 249, 'min_data_in_leaf': 28, 'n_estimators': 333, 'min_child_samples': 5}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:34:40,447] Trial 71 finished with value: 0.7393435984775173 and parameters: {'num_leaves': 222, 'min_data_in_leaf': 17, 'n_estimators': 316, 'min_child_samples': 3}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:34:58,141] Trial 72 finished with value: 0.7396370390357792 and parameters: {'num_leaves': 216, 'min_data_in_leaf': 15, 'n_estimators': 280, 'min_child_samples': 2}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:35:15,822] Trial 73 finished with value: 0.7388379804909347 and parameters: {'num_leaves': 215, 'min_data_in_leaf': 15, 'n_estimators': 281, 'min_child_samples': 2}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:35:32,681] Trial 74 finished with value: 0.7386664249060763 and parameters: {'num_leaves': 227, 'min_data_in_leaf': 14, 'n_estimators': 261, 'min_child_samples': 6}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:35:48,479] Trial 75 finished with value: 0.7391223891562753 and parameters: {'num_leaves': 210, 'min_data_in_leaf': 21, 'n_estimators': 249, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:36:08,005] Trial 76 finished with value: 0.7390817602815334 and parameters: {'num_leaves': 243, 'min_data_in_leaf': 12, 'n_estimators': 300, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:36:30,391] Trial 77 finished with value: 0.7392036472726083 and parameters: {'num_leaves': 256, 'min_data_in_leaf': 14, 'n_estimators': 274, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:36:57,688] Trial 78 finished with value: 0.7382510946665103 and parameters: {'num_leaves': 288, 'min_data_in_leaf': 19, 'n_estimators': 352, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:37:16,566] Trial 79 finished with value: 0.7385671105854542 and parameters: {'num_leaves': 217, 'min_data_in_leaf': 16, 'n_estimators': 283, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:37:32,527] Trial 80 finished with value: 0.7397950490129226 and parameters: {'num_leaves': 233, 'min_data_in_leaf': 20, 'n_estimators': 291, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:37:49,607] Trial 81 finished with value: 0.739479031259732 and parameters: {'num_leaves': 236, 'min_data_in_leaf': 20, 'n_estimators': 294, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:38:07,183] Trial 82 finished with value: 0.7390546738351452 and parameters: {'num_leaves': 227, 'min_data_in_leaf': 18, 'n_estimators': 268, 'min_child_samples': 6}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:38:27,130] Trial 83 finished with value: 0.7387792880749161 and parameters: {'num_leaves': 245, 'min_data_in_leaf': 11, 'n_estimators': 322, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:38:41,625] Trial 84 finished with value: 0.738828946396917 and parameters: {'num_leaves': 206, 'min_data_in_leaf': 20, 'n_estimators': 251, 'min_child_samples': 4}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:38:59,410] Trial 85 finished with value: 0.7386528871856228 and parameters: {'num_leaves': 231, 'min_data_in_leaf': 21, 'n_estimators': 307, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:39:13,136] Trial 86 finished with value: 0.7386483718811485 and parameters: {'num_leaves': 240, 'min_data_in_leaf': 15, 'n_estimators': 236, 'min_child_samples': 7}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:39:39,285] Trial 87 finished with value: 0.7387070587944263 and parameters: {'num_leaves': 225, 'min_data_in_leaf': 18, 'n_estimators': 373, 'min_child_samples': 6}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:39:55,899] Trial 88 finished with value: 0.7394700030353044 and parameters: {'num_leaves': 216, 'min_data_in_leaf': 13, 'n_estimators': 284, 'min_child_samples': 9}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:40:17,562] Trial 89 finished with value: 0.7391449658009299 and parameters: {'num_leaves': 278, 'min_data_in_leaf': 19, 'n_estimators': 297, 'min_child_samples': 8}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:40:38,047] Trial 90 finished with value: 0.7384000724450246 and parameters: {'num_leaves': 298, 'min_data_in_leaf': 23, 'n_estimators': 264, 'min_child_samples': 10}. Best is trial 34 with value: 0.7398040757088111.\n",
    "# # [I 2024-01-07 18:40:56,382] Trial 91 finished with value: 0.7399033919859633 and parameters: {'num_leaves': 234, 'min_data_in_leaf': 17, 'n_estimators': 313, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:41:18,761] Trial 92 finished with value: 0.7393255381156019 and parameters: {'num_leaves': 252, 'min_data_in_leaf': 17, 'n_estimators': 335, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:41:39,432] Trial 93 finished with value: 0.7397995583255238 and parameters: {'num_leaves': 234, 'min_data_in_leaf': 17, 'n_estimators': 316, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:41:59,714] Trial 94 finished with value: 0.738837978901254 and parameters: {'num_leaves': 233, 'min_data_in_leaf': 18, 'n_estimators': 316, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:42:18,282] Trial 95 finished with value: 0.7389463297004149 and parameters: {'num_leaves': 240, 'min_data_in_leaf': 17, 'n_estimators': 289, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:42:35,240] Trial 96 finished with value: 0.738964384620731 and parameters: {'num_leaves': 263, 'min_data_in_leaf': 16, 'n_estimators': 274, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:43:00,141] Trial 97 finished with value: 0.7378673675840631 and parameters: {'num_leaves': 324, 'min_data_in_leaf': 20, 'n_estimators': 305, 'min_child_samples': 3}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:43:21,264] Trial 98 finished with value: 0.73887860832627 and parameters: {'num_leaves': 223, 'min_data_in_leaf': 19, 'n_estimators': 313, 'min_child_samples': 2}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # [I 2024-01-07 18:43:42,930] Trial 99 finished with value: 0.7388334651864604 and parameters: {'num_leaves': 246, 'min_data_in_leaf': 18, 'n_estimators': 326, 'min_child_samples': 3}. Best is trial 91 with value: 0.7399033919859633.\n",
    "# # Best trial: score 0.7399033919859633, params {'num_leaves': 234, 'min_data_in_leaf': 17, 'n_estimators': 313, 'min_child_samples': 2}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # #1. Get the hyperparameters of the model and print them\n",
    "# best_params = study.best_trial.params\n",
    "\n",
    "# # Add the static parameters\n",
    "# best_params.update({\n",
    "#     'boosting_type': 'gbdt',\n",
    "#     'objective': 'multiclass',\n",
    "#     'num_class': 3,\n",
    "#     'metric': 'multi_logloss',\n",
    "#     'random_state': 42,\n",
    "# })\n",
    "\n",
    "\n",
    "# import json\n",
    "\n",
    "# # Write the hyperparameters to a new file\n",
    "# with open('hyperparameters_model6.json', 'w') as f:\n",
    "#     json.dump(best_params, f)\n",
    "\n",
    "\n",
    "\n",
    "# # Create the model with the best parameters\n",
    "# model6 = LGBMClassifier(**best_params)\n",
    "\n",
    "# # Fit the model to the full data\n",
    "# model6.fit(train_X1_full, train_Y1_temp_full)\n",
    "\n",
    "\n",
    "\n",
    "# # #2. Plot bar plot of the feature importances to visualize the model\n",
    "# import lightgbm as lgb\n",
    "# import matplotlib.pyplot as plt\n",
    "\n",
    "# # Create a larger figure\n",
    "# plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# # Plot the feature importance\n",
    "# ax = lgb.plot_importance(model6) # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Decrease the font size of the y-axis labels since there are two many features\n",
    "# for label in ax.get_yticklabels():\n",
    "#     label.set_size(8)\n",
    "\n",
    "# # Adjust the layout\n",
    "# plt.tight_layout()\n",
    "\n",
    "# # Save the plot\n",
    "# plt.savefig('feature_importance_LGBM_model6.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# # Show the plot\n",
    "# plt.show()\n",
    "\n",
    "# # from optuna.visualization import plot_param_importances\n",
    "\n",
    "# # plot_param_importances(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pandas as pd\n",
    "# from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# # Concatenate the datasets\n",
    "# full_X = pd.concat([train_X1, valid_X1, test_X1])\n",
    "# full_Y_temp = pd.concat([train_Y1_temp, valid_Y1_temp, test_Y1_temp])\n",
    "\n",
    "# # Refit the model on the full dataset\n",
    "# model6.fit(full_X, full_Y_temp)\n",
    "\n",
    "# # Save the model to a file using joblib\n",
    "# from joblib import dump\n",
    "# dump(model6, 'saved_LGBM_model6.joblib') # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "\n",
    "# # # This is how to load the model from joblib\n",
    "# # from joblib import load\n",
    "# # model5 = load('saved_LGBM_model5.joblib')  # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "# # Separate building_ids and features in the test data\n",
    "# competition_test_building_ids = test_data1['building_id']\n",
    "# competition_test_X = test_data1.drop('building_id', axis=1)\n",
    "\n",
    "# # Predict on the competition test data\n",
    "# competition_y_pred = model6.predict(competition_test_X)\n",
    "\n",
    "# # Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "# competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# # Create a DataFrame for submission\n",
    "# submission = pd.DataFrame({\n",
    "#     'building_id': competition_test_building_ids,\n",
    "#     'damage_grade': competition_y_pred\n",
    "# })\n",
    "\n",
    "# # Save the submission DataFrame to a CSV file for submission\n",
    "# submission.to_csv('submission6.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "# print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MAKE SURE TO TRY FEATURE SELECTON ON OPTUNA AND DEFINITELY TRY PRUNING\n",
    "\n",
    "THEN, try using the different feature selections like features2 and features3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For Model 7, we will continue the hyper parameter from model 6\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2024-01-08 21:01:48,479] A new study created in memory with name: no-name-96b96c56-a910-4836-9456-6f9120537be6\n",
      "[I 2024-01-08 21:02:16,977] Trial 0 finished with value: 0.7269784709859567 and parameters: {'learning_rate': 0.00880897278383721, 'feature_fraction': 0.5419912198177617, 'bagging_fraction': 0.7371723310753362, 'lambda_l1': 0.25682076630843903, 'lambda_l2': 5.45556404079999e-08, 'max_bin': 364}. Best is trial 0 with value: 0.7269784709859567.\n",
      "[I 2024-01-08 21:02:38,529] Trial 1 finished with value: 0.7413119067023057 and parameters: {'learning_rate': 0.05598726432532204, 'feature_fraction': 0.827869849932211, 'bagging_fraction': 0.48612557019360647, 'lambda_l1': 0.002047688621043951, 'lambda_l2': 1.1842514647957114, 'max_bin': 338}. Best is trial 1 with value: 0.7413119067023057.\n",
      "[I 2024-01-08 21:03:02,919] Trial 2 finished with value: 0.731673519490157 and parameters: {'learning_rate': 0.0115864343052812, 'feature_fraction': 0.7242849958527782, 'bagging_fraction': 0.6026981987648514, 'lambda_l1': 0.007070988539942633, 'lambda_l2': 3.683818919307624e-08, 'max_bin': 262}. Best is trial 1 with value: 0.7413119067023057.\n",
      "[I 2024-01-08 21:03:28,011] Trial 3 finished with value: 0.7187260227014777 and parameters: {'learning_rate': 0.005714772092165252, 'feature_fraction': 0.5208857275382257, 'bagging_fraction': 0.9614708098317495, 'lambda_l1': 6.401190227261317e-05, 'lambda_l2': 1.940075372426676e-07, 'max_bin': 283}. Best is trial 1 with value: 0.7413119067023057.\n",
      "[I 2024-01-08 21:03:47,031] Trial 4 finished with value: 0.7366890913305136 and parameters: {'learning_rate': 0.17651424294944978, 'feature_fraction': 0.9264113703415956, 'bagging_fraction': 0.5741409616580155, 'lambda_l1': 0.0005696470220297408, 'lambda_l2': 0.003403817151232067, 'max_bin': 370}. Best is trial 1 with value: 0.7413119067023057.\n",
      "[I 2024-01-08 21:04:09,783] Trial 5 finished with value: 0.7412396736921804 and parameters: {'learning_rate': 0.05894051223672768, 'feature_fraction': 0.8424460430264176, 'bagging_fraction': 0.9459617482345486, 'lambda_l1': 0.029772585373775643, 'lambda_l2': 0.4493144107759196, 'max_bin': 402}. Best is trial 1 with value: 0.7413119067023057.\n",
      "[I 2024-01-08 21:04:32,426] Trial 6 finished with value: 0.7419303909759479 and parameters: {'learning_rate': 0.0620167204509925, 'feature_fraction': 0.79578887119761, 'bagging_fraction': 0.8945602156549332, 'lambda_l1': 4.6372359845897433e-07, 'lambda_l2': 4.407923869558344e-06, 'max_bin': 479}. Best is trial 6 with value: 0.7419303909759479.\n",
      "[I 2024-01-08 21:04:57,108] Trial 7 finished with value: 0.7421922286216578 and parameters: {'learning_rate': 0.10154924379087327, 'feature_fraction': 0.502349784377463, 'bagging_fraction': 0.5419704247277546, 'lambda_l1': 0.02314354754630827, 'lambda_l2': 8.368360705674086e-05, 'max_bin': 320}. Best is trial 7 with value: 0.7421922286216578.\n",
      "[I 2024-01-08 21:05:19,116] Trial 8 finished with value: 0.7414744289879868 and parameters: {'learning_rate': 0.05666703521397018, 'feature_fraction': 0.6820418622301982, 'bagging_fraction': 0.7878636372089747, 'lambda_l1': 0.003559432393725011, 'lambda_l2': 0.3379996332024166, 'max_bin': 292}. Best is trial 7 with value: 0.7421922286216578.\n",
      "[I 2024-01-08 21:05:43,984] Trial 9 finished with value: 0.7223917697347063 and parameters: {'learning_rate': 0.006025846272586455, 'feature_fraction': 0.6358461957724594, 'bagging_fraction': 0.7576735772721448, 'lambda_l1': 8.737556496650526e-07, 'lambda_l2': 1.1480686012158873e-05, 'max_bin': 350}. Best is trial 7 with value: 0.7421922286216578.\n",
      "[I 2024-01-08 21:06:05,980] Trial 10 finished with value: 0.7428648816304501 and parameters: {'learning_rate': 0.19478241380723801, 'feature_fraction': 0.40675828000354974, 'bagging_fraction': 0.41605339407101233, 'lambda_l1': 5.7465494862616815, 'lambda_l2': 0.0018556475816900793, 'max_bin': 435}. Best is trial 10 with value: 0.7428648816304501.\n",
      "[I 2024-01-08 21:06:27,042] Trial 11 finished with value: 0.7425533856627394 and parameters: {'learning_rate': 0.19766116229847147, 'feature_fraction': 0.40567064654662993, 'bagging_fraction': 0.40567303293800416, 'lambda_l1': 6.792330266709577, 'lambda_l2': 0.0016620755527454564, 'max_bin': 443}. Best is trial 10 with value: 0.7428648816304501.\n",
      "[I 2024-01-08 21:06:48,237] Trial 12 finished with value: 0.7422328571906919 and parameters: {'learning_rate': 0.17860965377997806, 'feature_fraction': 0.40087761277352213, 'bagging_fraction': 0.4170221372926321, 'lambda_l1': 8.070751955805905, 'lambda_l2': 0.0037234111284645487, 'max_bin': 443}. Best is trial 10 with value: 0.7428648816304501.\n",
      "[I 2024-01-08 21:07:13,475] Trial 13 finished with value: 0.7338946380533512 and parameters: {'learning_rate': 0.01913719338464389, 'feature_fraction': 0.4020767895581891, 'bagging_fraction': 0.4477367140491934, 'lambda_l1': 1.7069939629366115, 'lambda_l2': 0.008123790522322039, 'max_bin': 421}. Best is trial 10 with value: 0.7428648816304501.\n",
      "[I 2024-01-08 21:07:37,096] Trial 14 finished with value: 0.7438490402061784 and parameters: {'learning_rate': 0.11916782760877047, 'feature_fraction': 0.47026883724091195, 'bagging_fraction': 0.6510127109505371, 'lambda_l1': 1.19199608930091e-08, 'lambda_l2': 0.038243088890054115, 'max_bin': 500}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:08:00,899] Trial 15 finished with value: 0.7429190598425425 and parameters: {'learning_rate': 0.10899497176670982, 'feature_fraction': 0.5845353477632813, 'bagging_fraction': 0.6531535355849684, 'lambda_l1': 2.9813944269156418e-05, 'lambda_l2': 0.04729589269315433, 'max_bin': 498}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:08:27,694] Trial 16 finished with value: 0.738847008898787 and parameters: {'learning_rate': 0.0304242437653955, 'feature_fraction': 0.5967716750082321, 'bagging_fraction': 0.6656943612460475, 'lambda_l1': 1.5469724819907167e-08, 'lambda_l2': 9.037788840204964, 'max_bin': 500}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:08:54,961] Trial 17 finished with value: 0.7435059325215305 and parameters: {'learning_rate': 0.09315430214965782, 'feature_fraction': 0.49189279321238183, 'bagging_fraction': 0.6658771149106756, 'lambda_l1': 9.42921945894918e-06, 'lambda_l2': 0.02140691758597575, 'max_bin': 477}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:09:18,655] Trial 18 finished with value: 0.743009347895267 and parameters: {'learning_rate': 0.10388143300602476, 'feature_fraction': 0.48035556439267274, 'bagging_fraction': 0.8322568975745656, 'lambda_l1': 1.1090949307123134e-08, 'lambda_l2': 0.07309109238132913, 'max_bin': 474}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:09:43,514] Trial 19 finished with value: 0.7404722146359605 and parameters: {'learning_rate': 0.03371341564612131, 'feature_fraction': 0.4745227505021029, 'bagging_fraction': 0.6897604456713278, 'lambda_l1': 1.9401317208802266e-06, 'lambda_l2': 0.00017618367337569258, 'max_bin': 468}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:10:06,492] Trial 20 finished with value: 0.7404451310020842 and parameters: {'learning_rate': 0.03541526940194804, 'feature_fraction': 0.7140857673484112, 'bagging_fraction': 0.6181007224490104, 'lambda_l1': 1.3109960268862258e-07, 'lambda_l2': 0.028849681493306877, 'max_bin': 397}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:10:29,585] Trial 21 finished with value: 0.7431357510467983 and parameters: {'learning_rate': 0.1109976813132227, 'feature_fraction': 0.47946895490841446, 'bagging_fraction': 0.8136136865243923, 'lambda_l1': 1.4076135099961907e-08, 'lambda_l2': 0.09290294986115565, 'max_bin': 471}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:10:56,687] Trial 22 finished with value: 0.7433118179619292 and parameters: {'learning_rate': 0.08330556590233926, 'feature_fraction': 0.46651544410760803, 'bagging_fraction': 0.8355042169123199, 'lambda_l1': 6.181955512658533e-06, 'lambda_l2': 4.371140045370121, 'max_bin': 460}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:11:20,421] Trial 23 finished with value: 0.7428964892509026 and parameters: {'learning_rate': 0.07802236481312168, 'feature_fraction': 0.5651616544189034, 'bagging_fraction': 0.8574641838541186, 'lambda_l1': 1.426581807898175e-05, 'lambda_l2': 6.068299942252529, 'max_bin': 449}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:11:43,854] Trial 24 finished with value: 0.7425669294362076 and parameters: {'learning_rate': 0.1338122070818296, 'feature_fraction': 0.45509677730543435, 'bagging_fraction': 0.7254509100756804, 'lambda_l1': 1.1648779803021919e-05, 'lambda_l2': 0.5841915864368068, 'max_bin': 488}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:12:08,038] Trial 25 finished with value: 0.7427159064198815 and parameters: {'learning_rate': 0.08104242808111829, 'feature_fraction': 0.6316039720926676, 'bagging_fraction': 0.5146304525481151, 'lambda_l1': 1.3365796226019466e-07, 'lambda_l2': 2.2726883300236245, 'max_bin': 464}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:12:30,843] Trial 26 finished with value: 0.7407882278646754 and parameters: {'learning_rate': 0.03843366710711456, 'feature_fraction': 0.5386772762006956, 'bagging_fraction': 0.9990405978314587, 'lambda_l1': 3.7333247513167145e-06, 'lambda_l2': 0.0006757684578184443, 'max_bin': 415}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:12:49,873] Trial 27 finished with value: 0.7387115689018677 and parameters: {'learning_rate': 0.13756979636229033, 'feature_fraction': 0.9926422135519655, 'bagging_fraction': 0.634787352035756, 'lambda_l1': 0.0002826837380490181, 'lambda_l2': 0.029421628276078474, 'max_bin': 456}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:13:11,157] Trial 28 finished with value: 0.7425895045523232 and parameters: {'learning_rate': 0.07902278887566404, 'feature_fraction': 0.4467704471773793, 'bagging_fraction': 0.7067979663951477, 'lambda_l1': 1.036409908516227e-07, 'lambda_l2': 0.14972064181095074, 'max_bin': 390}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:13:40,090] Trial 29 finished with value: 0.7416685479497805 and parameters: {'learning_rate': 0.045755792493499, 'feature_fraction': 0.5396526164787856, 'bagging_fraction': 0.7538786464051305, 'lambda_l1': 0.00010240936380750711, 'lambda_l2': 2.385589824414721, 'max_bin': 426}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:14:11,071] Trial 30 finished with value: 0.7384677948585759 and parameters: {'learning_rate': 0.023411341209465156, 'feature_fraction': 0.6312189041420974, 'bagging_fraction': 0.867854983520942, 'lambda_l1': 4.862980056279958e-06, 'lambda_l2': 0.012045937185595577, 'max_bin': 485}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:14:37,542] Trial 31 finished with value: 0.7430138614266358 and parameters: {'learning_rate': 0.13132811227086996, 'feature_fraction': 0.5071528519170461, 'bagging_fraction': 0.7994085119329005, 'lambda_l1': 5.994155013636692e-08, 'lambda_l2': 0.14253165329086073, 'max_bin': 464}. Best is trial 14 with value: 0.7438490402061784.\n",
      "[I 2024-01-08 21:15:03,647] Trial 32 finished with value: 0.7440205884540493 and parameters: {'learning_rate': 0.08715904259037313, 'feature_fraction': 0.4561170672711807, 'bagging_fraction': 0.7920093585115937, 'lambda_l1': 5.873462468697706e-07, 'lambda_l2': 1.091252208405455, 'max_bin': 499}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:15:26,295] Trial 33 finished with value: 0.7437181194267933 and parameters: {'learning_rate': 0.0774757026138783, 'feature_fraction': 0.4453582170455864, 'bagging_fraction': 0.7668398612928338, 'lambda_l1': 5.54019210441274e-07, 'lambda_l2': 1.4519916260857002, 'max_bin': 498}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:15:50,537] Trial 34 finished with value: 0.7435872027438933 and parameters: {'learning_rate': 0.06611582983900843, 'feature_fraction': 0.4505294813281086, 'bagging_fraction': 0.5781867753413824, 'lambda_l1': 4.473678334361816e-07, 'lambda_l2': 0.9174887591149139, 'max_bin': 500}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:16:15,279] Trial 35 finished with value: 0.7418987877576879 and parameters: {'learning_rate': 0.04714407559805754, 'feature_fraction': 0.43365316375481266, 'bagging_fraction': 0.5889170583669221, 'lambda_l1': 4.392442946660295e-07, 'lambda_l2': 1.0729971361365813, 'max_bin': 500}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:16:37,348] Trial 36 finished with value: 0.7430274066675016 and parameters: {'learning_rate': 0.06903722336094537, 'feature_fraction': 0.5546601783203562, 'bagging_fraction': 0.5658041845321155, 'lambda_l1': 4.4576283891454686e-08, 'lambda_l2': 1.174383352600412, 'max_bin': 488}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:17:00,321] Trial 37 finished with value: 0.7428016799629721 and parameters: {'learning_rate': 0.14771248530130854, 'feature_fraction': 0.43112757058280693, 'bagging_fraction': 0.7733086220681493, 'lambda_l1': 9.6905551647676e-07, 'lambda_l2': 0.23243865920858597, 'max_bin': 488}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:17:23,401] Trial 38 finished with value: 0.7427836263877704 and parameters: {'learning_rate': 0.051841090874181595, 'feature_fraction': 0.5226718477348986, 'bagging_fraction': 0.49005193091352794, 'lambda_l1': 2.8684760955912127e-07, 'lambda_l2': 0.5818873581580198, 'max_bin': 500}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:17:46,094] Trial 39 finished with value: 0.7332671267781125 and parameters: {'learning_rate': 0.01421474850442982, 'feature_fraction': 0.7617743034710562, 'bagging_fraction': 0.7167074977279467, 'lambda_l1': 3.7170118031269564e-08, 'lambda_l2': 1.2459235873899597e-07, 'max_bin': 313}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:18:05,229] Trial 40 finished with value: 0.7402013458921696 and parameters: {'learning_rate': 0.0641788679096784, 'feature_fraction': 0.8442921024797242, 'bagging_fraction': 0.8981285993473563, 'lambda_l1': 1.5113514716623123e-06, 'lambda_l2': 2.32468264766949, 'max_bin': 259}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:18:28,082] Trial 41 finished with value: 0.7432170119756432 and parameters: {'learning_rate': 0.09264967136871743, 'feature_fraction': 0.49846784844949077, 'bagging_fraction': 0.6781851509783668, 'lambda_l1': 2.0786195447119369e-07, 'lambda_l2': 0.01836119672384766, 'max_bin': 484}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:18:50,085] Trial 42 finished with value: 0.7430770647449361 and parameters: {'learning_rate': 0.11682308668639016, 'feature_fraction': 0.43718874031222943, 'bagging_fraction': 0.6094376715734371, 'lambda_l1': 6.063081315483297e-05, 'lambda_l2': 0.36525909125747574, 'max_bin': 472}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:19:12,277] Trial 43 finished with value: 0.7416459755238938 and parameters: {'learning_rate': 0.15163611535379132, 'feature_fraction': 0.48133055846497963, 'bagging_fraction': 0.5531512640243313, 'lambda_l1': 5.003749831943277e-07, 'lambda_l2': 3.727334220913354e-05, 'max_bin': 481}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:19:33,805] Trial 44 finished with value: 0.7431222100247004 and parameters: {'learning_rate': 0.06931769649459565, 'feature_fraction': 0.5086350803404587, 'bagging_fraction': 0.6460690350143181, 'lambda_l1': 2.6589211604470743e-06, 'lambda_l2': 0.006082996783565222, 'max_bin': 491}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:19:53,753] Trial 45 finished with value: 0.7417272399989496 and parameters: {'learning_rate': 0.09290939680968462, 'feature_fraction': 0.6006943445667706, 'bagging_fraction': 0.731711718354292, 'lambda_l1': 0.0011983422539873908, 'lambda_l2': 0.0006060513731162232, 'max_bin': 358}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:20:18,800] Trial 46 finished with value: 0.7411087562755814 and parameters: {'learning_rate': 0.05254378738116114, 'feature_fraction': 0.4301363254569057, 'bagging_fraction': 0.7571970452938837, 'lambda_l1': 3.0751610204504244e-08, 'lambda_l2': 9.846161144146777, 'max_bin': 450}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:20:43,657] Trial 47 finished with value: 0.7257866508977976 and parameters: {'learning_rate': 0.008219899033010658, 'feature_fraction': 0.5230616354437548, 'bagging_fraction': 0.524419337720345, 'lambda_l1': 2.411392081794149e-05, 'lambda_l2': 0.9598845730258689, 'max_bin': 434}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:21:03,205] Trial 48 finished with value: 0.7426391604898028 and parameters: {'learning_rate': 0.12026752280013242, 'feature_fraction': 0.6651933307121644, 'bagging_fraction': 0.5893340627551993, 'lambda_l1': 4.7668284577928657e-07, 'lambda_l2': 9.87276635814958e-07, 'max_bin': 475}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:21:23,905] Trial 49 finished with value: 0.7431673569552869 and parameters: {'learning_rate': 0.09275494029188693, 'feature_fraction': 0.5787658834575089, 'bagging_fraction': 0.6804161778758993, 'lambda_l1': 1.0524262006933725e-06, 'lambda_l2': 0.24041019356042564, 'max_bin': 493}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:21:47,109] Trial 50 finished with value: 0.7406527919642408 and parameters: {'learning_rate': 0.04216948689123385, 'feature_fraction': 0.42165861039636654, 'bagging_fraction': 0.6425902227428161, 'lambda_l1': 0.24140909761998214, 'lambda_l2': 0.053424215684069014, 'max_bin': 335}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:22:12,052] Trial 51 finished with value: 0.7435917128513346 and parameters: {'learning_rate': 0.07725756124896081, 'feature_fraction': 0.4640258309464457, 'bagging_fraction': 0.8341337998906513, 'lambda_l1': 7.697767190912914e-06, 'lambda_l2': 2.7955007099437847, 'max_bin': 460}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:22:35,891] Trial 52 finished with value: 0.7430409498906956 and parameters: {'learning_rate': 0.060850142602204484, 'feature_fraction': 0.4623575786704003, 'bagging_fraction': 0.9078503156319567, 'lambda_l1': 0.00020030024189355612, 'lambda_l2': 1.2227592699938216e-08, 'max_bin': 476}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:22:58,125] Trial 53 finished with value: 0.7428061983245245 and parameters: {'learning_rate': 0.16971052868018932, 'feature_fraction': 0.4884947174371849, 'bagging_fraction': 0.7929633152705159, 'lambda_l1': 8.816782099457507e-06, 'lambda_l2': 3.831773111346626, 'max_bin': 456}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:23:20,050] Trial 54 finished with value: 0.742846826832417 and parameters: {'learning_rate': 0.09870534710370975, 'feature_fraction': 0.4551604149862746, 'bagging_fraction': 0.8239850224135925, 'lambda_l1': 3.217510564967234e-05, 'lambda_l2': 0.6028900354586147, 'max_bin': 378}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:23:44,556] Trial 55 finished with value: 0.7381878953224118 and parameters: {'learning_rate': 0.028122537722307363, 'feature_fraction': 0.41508466638785607, 'bagging_fraction': 0.8595194347918033, 'lambda_l1': 2.2812368517862717e-06, 'lambda_l2': 2.032863716933171, 'max_bin': 478}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:24:06,992] Trial 56 finished with value: 0.7436187989308735 and parameters: {'learning_rate': 0.07597626174080073, 'feature_fraction': 0.45244009036008304, 'bagging_fraction': 0.7798863565378411, 'lambda_l1': 6.714817048621922e-08, 'lambda_l2': 0.11777381512462128, 'max_bin': 493}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:24:29,661] Trial 57 finished with value: 0.7436774878006812 and parameters: {'learning_rate': 0.0775298603942384, 'feature_fraction': 0.4532180454553847, 'bagging_fraction': 0.9295602075417734, 'lambda_l1': 8.581214742259724e-08, 'lambda_l2': 0.104914060229243, 'max_bin': 495}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:24:49,596] Trial 58 finished with value: 0.7417543231436934 and parameters: {'learning_rate': 0.07400146263878275, 'feature_fraction': 0.8806310060469793, 'bagging_fraction': 0.9163264191151106, 'lambda_l1': 2.3332235873220528e-08, 'lambda_l2': 0.09527417744881382, 'max_bin': 487}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:25:12,449] Trial 59 finished with value: 0.7422373708443439 and parameters: {'learning_rate': 0.054220625814062014, 'feature_fraction': 0.40414269924206575, 'bagging_fraction': 0.9569002429507285, 'lambda_l1': 1.0392428184737494e-08, 'lambda_l2': 0.22604628545830296, 'max_bin': 465}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:25:33,683] Trial 60 finished with value: 0.7425624146208659 and parameters: {'learning_rate': 0.08632421865978257, 'feature_fraction': 0.5295875551379828, 'bagging_fraction': 0.8822255257125455, 'lambda_l1': 1.0406668222103792e-07, 'lambda_l2': 0.002167445345310129, 'max_bin': 285}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:25:56,973] Trial 61 finished with value: 0.7438264665574605 and parameters: {'learning_rate': 0.07025771931193157, 'feature_fraction': 0.4593654407400793, 'bagging_fraction': 0.987461650831141, 'lambda_l1': 7.330961690261088e-08, 'lambda_l2': 1.2412553983984198, 'max_bin': 494}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:26:20,276] Trial 62 finished with value: 0.742259944615345 and parameters: {'learning_rate': 0.05911495493137811, 'feature_fraction': 0.46829587457427296, 'bagging_fraction': 0.9924055123212028, 'lambda_l1': 6.180417917862914e-08, 'lambda_l2': 5.0135328205131815, 'max_bin': 493}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:26:42,699] Trial 63 finished with value: 0.7434246816810434 and parameters: {'learning_rate': 0.10946054503120732, 'feature_fraction': 0.5011607039466559, 'bagging_fraction': 0.9280343054524344, 'lambda_l1': 9.820926993488321e-08, 'lambda_l2': 0.060650928350348035, 'max_bin': 493}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:27:04,647] Trial 64 finished with value: 0.7435555982416604 and parameters: {'learning_rate': 0.07415251938461216, 'feature_fraction': 0.443497469409611, 'bagging_fraction': 0.9785851293263126, 'lambda_l1': 2.211988379145708e-08, 'lambda_l2': 0.13253989207310035, 'max_bin': 481}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:27:26,756] Trial 65 finished with value: 0.7434066192403149 and parameters: {'learning_rate': 0.08399133872987116, 'feature_fraction': 0.46628238469543104, 'bagging_fraction': 0.9404569138281825, 'lambda_l1': 1.3826620352959774e-07, 'lambda_l2': 2.0104208139969124, 'max_bin': 469}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:27:48,976] Trial 66 finished with value: 0.743365991649546 and parameters: {'learning_rate': 0.12563108832882994, 'feature_fraction': 0.4142214132734285, 'bagging_fraction': 0.839867797648986, 'lambda_l1': 2.258721721523098e-07, 'lambda_l2': 0.44924437733003925, 'max_bin': 448}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:28:09,994] Trial 67 finished with value: 0.7414608831968471 and parameters: {'learning_rate': 0.15945753163941964, 'feature_fraction': 0.4854740507369494, 'bagging_fraction': 0.7756985616524613, 'lambda_l1': 7.484858112369083e-08, 'lambda_l2': 0.03643396576924878, 'max_bin': 495}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:28:31,985] Trial 68 finished with value: 0.7434201644811806 and parameters: {'learning_rate': 0.10428167770655752, 'feature_fraction': 0.5581866043941317, 'bagging_fraction': 0.9632853015175972, 'lambda_l1': 1.7912559817167487e-08, 'lambda_l2': 3.6799811942195597, 'max_bin': 483}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:28:54,285] Trial 69 finished with value: 0.742395378070117 and parameters: {'learning_rate': 0.0490391296521432, 'feature_fraction': 0.5117478764203588, 'bagging_fraction': 0.8025258687707645, 'lambda_l1': 2.1914673795116071e-07, 'lambda_l2': 0.010491470924651674, 'max_bin': 438}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:29:18,645] Trial 70 finished with value: 0.741546659613591 and parameters: {'learning_rate': 0.043299722216327406, 'feature_fraction': 0.45095517686438497, 'bagging_fraction': 0.7426384718079896, 'lambda_l1': 7.945353157809916e-07, 'lambda_l2': 0.7562933059057716, 'max_bin': 455}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:29:42,131] Trial 71 finished with value: 0.7431673538370672 and parameters: {'learning_rate': 0.06685377789496748, 'feature_fraction': 0.4465200704390068, 'bagging_fraction': 0.6995445956577713, 'lambda_l1': 4.223385949770801e-07, 'lambda_l2': 1.475865818453448, 'max_bin': 500}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:30:05,503] Trial 72 finished with value: 0.7431131793546103 and parameters: {'learning_rate': 0.07300420228648913, 'feature_fraction': 0.42683983835772643, 'bagging_fraction': 0.7758854465326167, 'lambda_l1': 4.637439262671659e-08, 'lambda_l2': 0.2917400909254975, 'max_bin': 499}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:30:28,602] Trial 73 finished with value: 0.7433614746331078 and parameters: {'learning_rate': 0.05930259671936945, 'feature_fraction': 0.4711464778333548, 'bagging_fraction': 0.8098997188314714, 'lambda_l1': 6.82649511215002e-07, 'lambda_l2': 1.1263415757497626, 'max_bin': 490}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:30:51,661] Trial 74 finished with value: 0.7433253608794156 and parameters: {'learning_rate': 0.08497306528459277, 'feature_fraction': 0.4900645085082648, 'bagging_fraction': 0.7412577120020083, 'lambda_l1': 1.450619596641777e-06, 'lambda_l2': 9.977675784533462, 'max_bin': 472}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:31:14,966] Trial 75 finished with value: 0.7428332854434699 and parameters: {'learning_rate': 0.06320592232496496, 'feature_fraction': 0.40033750629904497, 'bagging_fraction': 0.8434427360907965, 'lambda_l1': 0.016159604628314027, 'lambda_l2': 0.17257132672494155, 'max_bin': 482}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:31:38,085] Trial 76 finished with value: 0.743979961596979 and parameters: {'learning_rate': 0.09757711809728699, 'feature_fraction': 0.4572936260564274, 'bagging_fraction': 0.626145439298595, 'lambda_l1': 1.840130765680842e-07, 'lambda_l2': 0.37985846805346163, 'max_bin': 461}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:31:58,403] Trial 77 finished with value: 0.7419258758548984 and parameters: {'learning_rate': 0.1372619573296941, 'feature_fraction': 0.5407080646484032, 'bagging_fraction': 0.6288665863532913, 'lambda_l1': 1.5416593964684208e-07, 'lambda_l2': 0.09505767019742117, 'max_bin': 461}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:32:20,391] Trial 78 finished with value: 0.7425037250785008 and parameters: {'learning_rate': 0.10114702730850689, 'feature_fraction': 0.42085219610563035, 'bagging_fraction': 0.8806296260695108, 'lambda_l1': 5.445668046108571e-08, 'lambda_l2': 0.433975380466741, 'max_bin': 268}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:32:40,205] Trial 79 finished with value: 0.7424630979157228 and parameters: {'learning_rate': 0.11771832804709348, 'feature_fraction': 0.7734690567029547, 'bagging_fraction': 0.717579403366049, 'lambda_l1': 3.161593840500286e-08, 'lambda_l2': 3.5125571658985706, 'max_bin': 413}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:33:02,377] Trial 80 finished with value: 0.7437000592483028 and parameters: {'learning_rate': 0.08253134914286091, 'feature_fraction': 0.43735530905092246, 'bagging_fraction': 0.6639124226726607, 'lambda_l1': 2.704716600407281e-07, 'lambda_l2': 0.021123690274038594, 'max_bin': 468}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:33:25,914] Trial 81 finished with value: 0.7435104518002064 and parameters: {'learning_rate': 0.08035267233384562, 'feature_fraction': 0.4386083522151712, 'bagging_fraction': 0.618756439634165, 'lambda_l1': 2.5291130663019704e-07, 'lambda_l2': 0.014989299194107656, 'max_bin': 467}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:33:47,532] Trial 82 finished with value: 0.7432576429291983 and parameters: {'learning_rate': 0.08873056514564333, 'feature_fraction': 0.4767602449676394, 'bagging_fraction': 0.6880678783925612, 'lambda_l1': 3.858992288225124e-06, 'lambda_l2': 0.03553801109507572, 'max_bin': 487}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:34:10,606] Trial 83 finished with value: 0.7427791112667208 and parameters: {'learning_rate': 0.10446116407355736, 'feature_fraction': 0.4599255298171424, 'bagging_fraction': 0.6656337569489522, 'lambda_l1': 3.471065409194744e-07, 'lambda_l2': 0.004372496785827288, 'max_bin': 478}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:34:31,992] Trial 84 finished with value: 0.7432441008065523 and parameters: {'learning_rate': 0.07792160073584398, 'feature_fraction': 0.5110356792474215, 'bagging_fraction': 0.6662162310647484, 'lambda_l1': 1.0480415672572728e-08, 'lambda_l2': 0.022652001190507195, 'max_bin': 494}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:34:54,295] Trial 85 finished with value: 0.7430183759974113 and parameters: {'learning_rate': 0.07015655215505502, 'feature_fraction': 0.43221689410685377, 'bagging_fraction': 0.7036456786751358, 'lambda_l1': 9.375782872316645e-08, 'lambda_l2': 0.36524089222520245, 'max_bin': 473}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:35:17,317] Trial 86 finished with value: 0.7427384756664068 and parameters: {'learning_rate': 0.09531435189615622, 'feature_fraction': 0.49626734722676513, 'bagging_fraction': 0.8206411235874731, 'lambda_l1': 1.6758841317384935e-07, 'lambda_l2': 0.12383489399367408, 'max_bin': 459}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:35:38,751] Trial 87 finished with value: 0.7434111338722321 and parameters: {'learning_rate': 0.1122595658165582, 'feature_fraction': 0.4550325769264354, 'bagging_fraction': 0.7666574662859869, 'lambda_l1': 1.9671169625601027e-08, 'lambda_l2': 0.0666727111071116, 'max_bin': 429}. Best is trial 32 with value: 0.7440205884540493.\n",
      "[I 2024-01-08 21:36:01,857] Trial 88 finished with value: 0.7440792723713904 and parameters: {'learning_rate': 0.12844031728258057, 'feature_fraction': 0.47414782515524534, 'bagging_fraction': 0.7846100271861661, 'lambda_l1': 3.805769924899612e-08, 'lambda_l2': 1.7390351172611058, 'max_bin': 487}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:36:20,370] Trial 89 finished with value: 0.7391765581359916 and parameters: {'learning_rate': 0.14757198519529333, 'feature_fraction': 0.97083645257524, 'bagging_fraction': 0.6608759364812874, 'lambda_l1': 3.569602756306524e-08, 'lambda_l2': 0.6615863437729902, 'max_bin': 486}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:36:44,661] Trial 90 finished with value: 0.7106631827822184 and parameters: {'learning_rate': 0.005079365122707867, 'feature_fraction': 0.4175687450769191, 'bagging_fraction': 0.7868258929603108, 'lambda_l1': 7.105056481437327e-08, 'lambda_l2': 1.7625790037426563, 'max_bin': 492}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:37:08,115] Trial 91 finished with value: 0.7433795382966676 and parameters: {'learning_rate': 0.12666577604656343, 'feature_fraction': 0.47447899626432777, 'bagging_fraction': 0.7492192634877799, 'lambda_l1': 3.4963747775035904e-08, 'lambda_l2': 1.449109336451489, 'max_bin': 480}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:37:32,072] Trial 92 finished with value: 0.7433614759782223 and parameters: {'learning_rate': 0.07902595841715257, 'feature_fraction': 0.44147515884729316, 'bagging_fraction': 0.7881923576466069, 'lambda_l1': 6.14305034425574e-07, 'lambda_l2': 2.7357347234536205, 'max_bin': 500}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:37:53,524] Trial 93 finished with value: 0.7409507452590313 and parameters: {'learning_rate': 0.19966883822545398, 'feature_fraction': 0.46376215717487324, 'bagging_fraction': 0.849562734099155, 'lambda_l1': 3.136298755129841e-07, 'lambda_l2': 0.8236454264934295, 'max_bin': 468}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:38:16,763] Trial 94 finished with value: 0.7433343870861716 and parameters: {'learning_rate': 0.08973835974124102, 'feature_fraction': 0.5231884115874698, 'bagging_fraction': 0.977730272287913, 'lambda_l1': 1.5594122204390165e-06, 'lambda_l2': 5.863112478299234, 'max_bin': 488}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:38:37,814] Trial 95 finished with value: 0.7433434192236592 and parameters: {'learning_rate': 0.09878669235569203, 'feature_fraction': 0.4809562709829252, 'bagging_fraction': 0.7213929698321484, 'lambda_l1': 1.8531051453927334e-08, 'lambda_l2': 0.2502321666443219, 'max_bin': 446}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:39:04,990] Trial 96 finished with value: 0.7339849270231992 and parameters: {'learning_rate': 0.0180897736713705, 'feature_fraction': 0.4919866214739988, 'bagging_fraction': 0.7609433081930523, 'lambda_l1': 1.4708138883811453e-07, 'lambda_l2': 6.581526153800574, 'max_bin': 495}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:39:26,126] Trial 97 finished with value: 0.741794951223595 and parameters: {'learning_rate': 0.17891403380104617, 'feature_fraction': 0.4121249434139854, 'bagging_fraction': 0.8307148073567084, 'lambda_l1': 1.043085588862944e-06, 'lambda_l2': 0.1991662366271818, 'max_bin': 476}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:39:49,284] Trial 98 finished with value: 0.7433705038358004 and parameters: {'learning_rate': 0.11406162449791918, 'feature_fraction': 0.442529624061195, 'bagging_fraction': 0.5909449024295519, 'lambda_l1': 5.0285372082358293e-08, 'lambda_l2': 3.1706414432123773e-06, 'max_bin': 485}. Best is trial 88 with value: 0.7440792723713904.\n",
      "[I 2024-01-08 21:40:12,570] Trial 99 finished with value: 0.7430364319571341 and parameters: {'learning_rate': 0.0651342669185036, 'feature_fraction': 0.4549637898035607, 'bagging_fraction': 0.8104126605446027, 'lambda_l1': 6.795120262293743e-06, 'lambda_l2': 0.5661770464224808, 'max_bin': 452}. Best is trial 88 with value: 0.7440792723713904.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best trial: score 0.7440792723713904, params {'learning_rate': 0.12844031728258057, 'feature_fraction': 0.47414782515524534, 'bagging_fraction': 0.7846100271861661, 'lambda_l1': 3.805769924899612e-08, 'lambda_l2': 1.7390351172611058, 'max_bin': 487}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from lightgbm import LGBMClassifier\n",
    "from sklearn.metrics import make_scorer, f1_score, accuracy_score\n",
    "\n",
    "\n",
    "# LGBMClassifier expects the labels data to only include one column. Currently, they include two columns: building_id and damage_grade.\n",
    "# So, I am making it so the labels data only includes one column (damage_grade) and changing the range from 1-3 to 0-2 since LGBMClassifier wants to start at 0\n",
    "train_Y1_temp = train_Y1['damage_grade'] - 1  \n",
    "valid_Y1_temp = valid_Y1['damage_grade'] - 1\n",
    "test_Y1_temp = test_Y1['damage_grade'] - 1\n",
    "\n",
    "# Concatenate training and validation sets so that I have more data to fit the model on during GridSearchCV\n",
    "train_X1_full = pd.concat([train_X1, valid_X1])\n",
    "train_Y1_temp_full = pd.concat([train_Y1_temp, valid_Y1_temp])\n",
    "\n",
    "import optuna\n",
    "from lightgbm import LGBMClassifier\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.model_selection import cross_val_score, StratifiedKFold\n",
    "\n",
    "def objective(trial):\n",
    "    params = {\n",
    "        # I got these values from the previous optuna trial, from model 6\n",
    "        'boosting_type': 'gbdt',\n",
    "        'objective': 'multiclass',\n",
    "        'num_class': 3,\n",
    "        'metric': 'multi_logloss',\n",
    "        'random_state': 42,\n",
    "        'num_leaves': 234,\n",
    "        #'max_depth': trial.suggest_int('max_depth', -1, 50),\n",
    "        'min_data_in_leaf': 17,\n",
    "        'n_estimators': 313,\n",
    "        'min_child_samples': 12,\n",
    "\n",
    "        # These are the new hyper parameters which I want to test out\n",
    "        'learning_rate': trial.suggest_float('learning_rate', 0.005, 0.2, log=True), # default value is 0.1\n",
    "        'feature_fraction': trial.suggest_float('feature_fraction', 0.4, 1.0), # default value is 1.0\n",
    "        'bagging_fraction': trial.suggest_float('bagging_fraction', 0.4, 1.0), # default value is 1.0\n",
    "        'lambda_l1': trial.suggest_float('lambda_l1', 1e-8, 10.0, log=True), # default value is 0.0\n",
    "        'lambda_l2': trial.suggest_float('lambda_l2', 1e-8, 10.0, log=True), # default value is 0.0\n",
    "        'max_bin': trial.suggest_int('max_bin', 255, 500), # default value is 255\n",
    "    }\n",
    "\n",
    "    model = LGBMClassifier(**params)\n",
    "    cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=42)\n",
    "    score = cross_val_score(model, train_X1_full, train_Y1_temp_full, cv=cv, scoring='f1_micro', n_jobs=-1)\n",
    "    f1 = score.mean()\n",
    "    return f1       \n",
    "\n",
    "study = optuna.create_study(direction='maximize')\n",
    "study.optimize(objective, n_trials=100)\n",
    "\n",
    "print('Best trial: score {}, params {}'.format(study.best_trial.value, study.best_trial.params))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some information about the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] feature_fraction is set=0.47414782515524534, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.47414782515524534\n",
      "[LightGBM] [Warning] lambda_l1 is set=3.805769924899612e-08, reg_alpha=0.0 will be ignored. Current value: lambda_l1=3.805769924899612e-08\n",
      "[LightGBM] [Warning] lambda_l2 is set=1.7390351172611058, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.7390351172611058\n",
      "[LightGBM] [Warning] bagging_fraction is set=0.7846100271861661, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7846100271861661\n",
      "[LightGBM] [Warning] feature_fraction is set=0.47414782515524534, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.47414782515524534\n",
      "[LightGBM] [Warning] lambda_l1 is set=3.805769924899612e-08, reg_alpha=0.0 will be ignored. Current value: lambda_l1=3.805769924899612e-08\n",
      "[LightGBM] [Warning] lambda_l2 is set=1.7390351172611058, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.7390351172611058\n",
      "[LightGBM] [Warning] bagging_fraction is set=0.7846100271861661, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7846100271861661\n",
      "[LightGBM] [Info] Auto-choosing row-wise multi-threading, the overhead of testing was 0.005771 seconds.\n",
      "You can set `force_row_wise=true` to remove the overhead.\n",
      "And if memory is not enough, you can set `force_col_wise=true`.\n",
      "[LightGBM] [Info] Total Bins 1195\n",
      "[LightGBM] [Info] Number of data points in the train set: 221510, number of used features: 34\n",
      "[LightGBM] [Info] Start training from score -2.341151\n",
      "[LightGBM] [Info] Start training from score -0.562933\n",
      "[LightGBM] [Info] Start training from score -1.095876\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 16000x4000 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnoAAAHWCAYAAADgnCQEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOzdeVyN+f//8Ud7SBGSZcgeKSH7PkKy7401e5IYexjTxBjbyIxmlGU0tmFsjY8xUXajiNHYjYxmkCbSKrRdvz/6dX07OtU5pE553283t9E51/K+Xp2pt/d1vd9PLUmSJARBEARBEIRSR7u4GyAIgiAIgiC8H6KjJwiCIAiCUEqJjp4gCIIgCEIpJTp6giAIgiAIpZTo6AmCIAiCIJRSoqMnCIIgCIJQSomOniAIgiAIQiklOnqCIAiCIAillOjoCYIgCIIglFKioycIgiCUeP7+/mhpaREZGVncTREEjSI6eoIgCCVQdsdG2Z+FCxe+l3NeuHABT09P4uPj38vxP2QpKSl4enpy+vTp4m6KUMroFncDBEEQhLfn5eVFnTp1FF5r2rTpeznXhQsX+OKLL3B2dqZChQrv5Rxva8yYMTg5OWFgYFDcTXkrKSkpfPHFFwB07dq1eBsjlCqioycIglCC9e7dGzs7u+Juxjt58eIF5cqVe6dj6OjooKOjU0gtKjqZmZmkpqYWdzOEUkzcuhUEQSjFfvvtNzp16kS5cuUoX748ffr04ebNmwrbXLt2DWdnZ+rWrYuhoSHm5uZMmDCB2NhYeRtPT0/mzZsHQJ06deTbxJGRkURGRqKlpYW/v3+u82tpaeHp6alwHC0tLW7dusXIkSOpWLEiHTt2lN/fuXMnLVu2pEyZMpiamuLk5MTDhw8LvE5lz+hZWFjQt29fTp8+jZ2dHWXKlMHa2lq+PXrw4EGsra0xNDSkZcuWXL16VeGYzs7OGBkZ8ffff9OrVy/KlStH9erV8fLyQpIkhW1fvHjBnDlz+OijjzAwMKBRo0asXbs213ZaWlq4ubmxa9curKysMDAwwNfXlypVqgDwxRdfyLXNrpsq35+ctY2IiJBHXU1MTBg/fjwpKSm5arZz505at25N2bJlqVixIp07d+b48eMK26jy+RE0mxjREwRBKMESEhJ49uyZwmuVK1cGYMeOHYwbN45evXqxatUqUlJS2LhxIx07duTq1atYWFgAEBQUxN9//8348eMxNzfn5s2bbNq0iZs3bxIaGoqWlhaDBw/mr7/+4qeffsLb21s+R5UqVXj69Kna7R42bBgNGjRgxYoVcmfoyy+/5LPPPmP48OFMmjSJp0+fsmHDBjp37szVq1ff6nZxREQEI0eOZOrUqYwePZq1a9fSr18/fH19WbRoEa6urgB89dVXDB8+nLt376Kt/X9jIBkZGTg4ONC2bVtWr15NYGAgn3/+Oenp6Xh5eQEgSRL9+/fn1KlTTJw4EVtbW44dO8a8efN4/Pgx3t7eCm06efIkP//8M25ublSuXJlmzZqxceNGpk2bxqBBgxg8eDAANjY2gGrfn5yGDx9OnTp1+Oqrr/jjjz/YsmULZmZmrFq1St7miy++wNPTk/bt2+Pl5YW+vj4XL17k5MmT9OzZE1D98yNoOEkQBEEocbZt2yYBSv9IkiQlJSVJFSpUkCZPnqywX3R0tGRiYqLwekpKSq7j//TTTxIgnT17Vn5tzZo1EiA9ePBAYdsHDx5IgLRt27ZcxwGkzz//XP76888/lwDpk08+UdguMjJS0tHRkb788kuF169fvy7p6urmej2veuRsW+3atSVAunDhgvzasWPHJEAqU6aM9M8//8iv+/n5SYB06tQp+bVx48ZJgDRjxgz5tczMTKlPnz6Svr6+9PTpU0mSJCkgIEACpOXLlyu0aejQoZKWlpYUERGhUA9tbW3p5s2bCts+ffo0V62yqfr9ya7thAkTFLYdNGiQVKlSJfnre/fuSdra2tKgQYOkjIwMhW0zMzMlSVLv8yNoNnHrVhAEoQT77rvvCAoKUvgDWaNA8fHxfPLJJzx79kz+o6OjQ5s2bTh16pR8jDJlysh/f/XqFc+ePaNt27YA/PHHH++l3S4uLgpfHzx4kMzMTIYPH67QXnNzcxo0aKDQXnU0adKEdu3ayV+3adMGgI8//phatWrlev3vv//OdQw3Nzf579m3XlNTUwkODgbg6NGj6Ojo4O7urrDfnDlzkCSJ3377TeH1Ll260KRJE5WvQd3vz5u17dSpE7GxsSQmJgIQEBBAZmYmS5cuVRi9zL4+UO/zI2g2cetWEAShBGvdurXSyRj37t0Dsjo0yhgbG8t/f/78OV988QV79uwhJiZGYbuEhIRCbO3/eXOm8L1795AkiQYNGijdXk9P763Ok7MzB2BiYgLARx99pPT1uLg4hde1tbWpW7euwmsNGzYEkJ8H/Oeff6hevTrly5dX2K5x48by+zm9ee0FUff78+Y1V6xYEci6NmNjY+7fv4+2tna+nU11Pj+CZhMdPUEQhFIoMzMTyHrOytzcPNf7urr/9+N/+PDhXLhwgXnz5mFra4uRkRGZmZk4ODjIx8nPm8+IZcvIyMhzn5yjVNnt1dLS4rffflM6e9bIyKjAdiiT10zcvF6X3pg88T68ee0FUff7UxjXps7nR9Bs4jslCIJQCtWrVw8AMzMz7O3t89wuLi6OEydO8MUXX7B06VL59ewRnZzy6tBljxi9uZDymyNZBbVXkiTq1Kkjj5hpgszMTP7++2+FNv31118A8mSE2rVrExwcTFJSksKo3p07d+T3C5JXbdX5/qiqXr16ZGZmcuvWLWxtbfPcBgr+/AiaTzyjJwiCUAr16tULY2NjVqxYQVpaWq73s2fKZo/+vDnas379+lz7ZK9192aHztjYmMqVK3P27FmF17///nuV2zt48GB0dHT44osvcrVFkqRcS4kUJR8fH4W2+Pj4oKenR/fu3QFwdHQkIyNDYTsAb29vtLS06N27d4HnKFu2LJC7tup8f1Q1cOBAtLW18fLyyjUimH0eVT8/guYTI3qCIAilkLGxMRs3bmTMmDG0aNECJycnqlSpwr///suvv/5Khw4d8PHxwdjYmM6dO7N69WrS0tKoUaMGx48f58GDB7mO2bJlSwAWL16Mk5MTenp69OvXj3LlyjFp0iRWrlzJpEmTsLOz4+zZs/LIlyrq1avH8uXL8fDwIDIykoEDB1K+fHkePHjAoUOHmDJlCnPnzi20+qjK0NCQwMBAxo0bR5s2bfjtt9/49ddfWbRokbz2Xb9+/ejWrRuLFy8mMjKSZs2acfz4cX755RdmzZolj47lp0yZMjRp0oS9e/fSsGFDTE1Nadq0KU2bNlX5+6Oq+vXrs3jxYpYtW0anTp0YPHgwBgYGhIWFUb16db766iuVPz9CCVBMs30FQRCEd5C9nEhYWFi+2506dUrq1auXZGJiIhkaGkr16tWTnJ2dpcuXL8vbPHr0SBo0aJBUoUIFycTERBo2bJgUFRWldLmPZcuWSTVq1JC0tbUVljNJSUmRJk6cKJmYmEjly5eXhg8fLsXExOS5vEr20iRvOnDggNSxY0epXLlyUrly5SRLS0tp+vTp0t27d1Wqx5vLq/Tp0yfXtoA0ffp0hdeyl4hZs2aN/Nq4ceOkcuXKSffv35d69uwplS1bVqpatar0+eef51qWJCkpSfr000+l6tWrS3p6elKDBg2kNWvWyMuV5HfubBcuXJBatmwp6evrK9RN1e9PXrVVVhtJkqQffvhBat68uWRgYCBVrFhR6tKlixQUFKSwjSqfH0GzaUlSETx5KgiCIAgljLOzM/v37yc5Obm4myIIb008oycIgiAIglBKiY6eIAiCIAhCKSU6eoIgCIIgCKWUeEZPEARBEAShlBIjeoIgCIIgCKWU6OgJgiAIgiCUUmLBZEEQ3llmZiZRUVGUL18+zygnQRAE4e1IkkRSUhLVq1dHW1u9MTrR0RME4Z1FRUXx0UcfFXczBEEQSrWHDx9Ss2ZNtfYRHT1BEN5ZdpD7gwcPMDU1LebWaL60tDSOHz9Oz5490dPTK+7maDxRL9WJWqmnpNQrMTGRjz76SP5Zqw7R0RME4Z1l364tX748xsbGxdwazZeWlkbZsmUxNjbW6F8umkLUS3WiVuopafV6m0djxGQMQRAEQRCEUkp09ARBEARBEEop0dETBEEQBEFQ0VdffUWrVq0oX748ZmZmDBw4kLt37yps8+rVK6ZPn06lSpUwMjJiyJAh/Pfff7mO5e/vj42NDYaGhpiZmTF9+nSF93/++WdsbW0xNzcH4JtvvlG7vaKjJwiCIAiCoKIzZ84wffp0QkNDCQoKIi0tjZ49e/LixQt5m08//ZT//e9/7Nu3jzNnzhAVFcXgwYMVjrNu3ToWL17MwoULuXnzJsHBwfTq1Ut+/7fffmPUqFG4uLgQEhICwPfff4+Pj49a7RWTMQRBEARBEFQUGBio8LW/vz9mZmZcuXKFzp07k5CQwNatW9m9ezcff/wxANu2baNx48aEhobStm1b4uLiWLJkCf/73//o3r27fCwbGxv57zt27GDgwIG4uLiQmJgIwOzZs1m1ahXTp09XeWKGGNEThELg7OzM+vXr38uxIyMjqVChQr7bnDx5ktatW9OkSROsrKyYP38+mZmZ+e6zdOlSdu3apfQ9Hx8fnJ2d37LFgiAIH46EhAQAeWmpK1eukJaWhr29vbyNpaUltWrVkkfmgoKCyMzM5PHjxzRu3JiaNWsyfPhwHj58KO/z+vVrDA0NFc5laGjIo0eP+Oeff1Run+joCUIpULFiRfbs2cOtW7e4cuUKFy5cYPv27fnu4+XlxahRo4qohYIgCKVPZmYms2bNokOHDjRt2hSA6Oho9PX1c/0DvWrVqkRHRwPw999/k5mZyYoVK1i/fj379+/n+fPn9OjRg9TUVAB69erFwYMHOXHihPwP9+zbtk+ePFG5jeLWrVDq/fLLLyxcuBB9fX0cHBzYunUrly9fJi0tjVmzZhETE8Pr16+ZMmUKbm5uABw7dgwPDw/S09OpWLEiGzdupEmTJiqdLy0tjc8++4yTJ0+SmppKw4YN8fPzIzk5mZYtW/Lo0SP09fWBrJHA5s2bM3PmTMLCwliwYAGJiYlkZGSwaNEihg0bptI5mzdvLv/d0NAQW1tbIiMj893H2dkZW1tbZs2aRVJSEpMmTSI8PJwqVapgZWWV776vX7/m9evX8tfZtxU6rwomXa+cSm3+kBloSyyzg5ZegbzOFJFxBRH1Up2olXrUrdcNz14KX7u5uXHjxg1OnTpFWloaAOnp6QDy19kkSSIjI4O0tDT5z7p16+Tbu9u3b+ejjz4iKCiInj174uzszF9//UXfvn3lYw0ZMoSvvvpKrRg00dETSrWYmBgmTJjA77//jqWlJdu2bSM2NpaMjAw++eQTdu7ciaWlJSkpKbRt25Y2bdpQu3ZtRo4cyenTp7G2tmbXrl0MHTqUmzdvqvRMxJo1ayhXrhyXLl0CYNmyZSxZsoTvvvsOW1tbDh8+zNChQ0lOTubw4cN8/fXXxMfHM2XKFI4ePUq1atV49uwZLVq0oH379mpfc3R0NPv37+fIkSMq7+Pl5YWBgQF37twhMTFRrkVevvrqK7744otcry9pnknZshlqt/lDtcwu/9vrgiJRL9WJWqlH1XodPXpU/vumTZu4ePEiK1as4Nq1a1y7dg2Af/75h9TUVH7++WeMjIzk7f/55x/i4uI4evQoT58+BbJG5nIes3z58hw9elTuLHbq1In27dvz5MkT3NzcaNmyJQB169ZV+dpER08o1UJDQ7GxscHS0hKAcePG4eLiwuvXr7l58yZOTk7ytklJSdy6dYvo6Gisra2xtrYGYNSoUUyfPp3Hjx+rlDEYEBBAQkICBw4cACA1NRULCwsAxo8fz7Zt2xg6dCj79u3j448/plKlShw9epS///6b3r17Kxzr7t27av0PnZiYSL9+/Zg/fz52dnYq73fixAm8vb3R0tLCxMSEkSNHcv/+/Ty39/DwYPbs2Qrn/eijj+jWrRuVKlVS+bwfqrS0NIKCgujRo0eJWI2/uIl6qU7USj1vUy9Jkpg1axbh4eGcPXuWBg0aKLzfoUMHli1bhq6uLo6OjkDWz/KnT58yfvx42rRpQ/369dmwYQM1a9aUR/SeP39OUlISffr0oUePHgrHTExMxM3Njf3799OuXTuqVKmi8jWKjp7wQZIkCVNTU8LDw3O997///e+dj71hwwZ69uyZ671Bgwbh7u7OkydP8Pf3Z/78+fI+VlZWXLhwIdc+Bd2CzZaUlISDgwMDBgxQ6IS9jYJGLg0MDDAwMMj1up6envjlogZRL/WIeqlO1Eo96tTL1dWV3bt388svv2BqakpsbCwAJiYmlClThsqVKzNx4kTmz5+PmZkZxsbGzJgxg3bt2tGxY0cArKysGDBgAHPmzGHTpk0YGxvj4eGBpaWl3Ol89uwZ+/fvp2vXrjx79gzIGkg4c+aMWtcmJmMIpVrbtm25du2avJjlzp07SU1NxcDAAGNjY7Zt2yZvGxERwfPnz2nbti3Xr1/nxo0bAOzZs4caNWpQo0YNlc45cOBAvL29SUlJASAlJYWbN28CWc/PDRs2DE9PT+7fv4+DgwMA7du358GDBwQHB8vHCQ8Plx/KLUhycjIODg44ODiwZMkSlfbJyd7enm3btiFJEomJifz0009qH0MQBOFDsHHjRhISEujatSvVqlWT/+zdu1fextvbm759+zJkyBA6d+6Mubk5Bw8eVDjO9u3badOmDX369KFLly7o6ekRGBio0OH88ccfsbOzk9fX+/XXX2ndurVa7RUjekKpZmZmxpYtWxg4cCAGBgb06NEDIyMjKleuzJEjR5g1axbe3t5kZGRQuXJldu/eTY0aNdi1axdjx46VJ2Ps27dP5TWLFixYwOvXr2nTpo28z4IFC+QJDuPHj6d169YsWLAAHR0dIGvW7K+//srcuXOZM2cOaWlp1KpVi4CAAJXO+c0333Dp0iVevHgh/zAZNmwYixcvVmn/zz77jEmTJmFpaUmVKlXo2LGjwmQLQRAEIYskSQVuY2hoyHfffcd3332X5zbGxsZs3bqVrVu3Kn2/cuXK8nIsiYmJmJiYqPVITs4GC0KplpiYKP/90KFDkqWlZTG2pnRKSEiQAOnZs2fF3ZQSITU1VQoICJBSU1OLuyklwodcrzNnzkh9+/aVqlWrJgHSoUOH8tx26tSpEiBNmDBBoVaxsbHSyJEjpfLly0smJibShAkTpKSkJIV9//zzT6ljx46SgYGBVLNmTWnVqlXv65I0Skn5bGX/jE1ISFB7X3HrVij1NmzYQLNmzWjatCmrV6/Oc5FgQRAETfPixQuaNWuW78gQwKFDhwgNDaV69eq53hs1ahQ3b94kKCiII0eOcPbsWaZMmSK/n5iYSM+ePalduzZXrlxhzZo1eHp6smnTpkK/HqHoiVu3Qqm3aNEiFi1a9M7HOXr0qNLjeHh4MGLEiHc+fkFiYmKUTvDo0aMHa9asUbpPcbdZEIR307t371yz8d/0+PFjZsyYwbFjx+jTp4/Ce7dv3yYwMJCwsDD5tt+GDRtwdHRk7dq1VK9enV27dpGamsoPP/yAvr4+VlZWhIeHs27dOoUOoVAyiY6eIKjI0dFRnir/ppyLDxe2yMhIbG1tiY+PVzpLOHsbZ2dnrl69Sp06deTt8mvzpEmTGDVqFN26dcv13ty5czEyMsLT07OQrkIQhPchMzOTMWPGMG/ePKULnYeEhFChQgWFZ7vs7e3R1tbm4sWLDBo0iJCQEDp37iwv5A5ZqQyrVq0iLi6OihUrFsm1CO+HuHUrCKWAsbExy5cvZ/fu3Srvs2XLFqWdPEEQSo5Vq1ahq6uLu7u70vejo6MxMzNTeE1XVxdTU1M5jis6OpqqVasqbJP9dfY2QsklRvSEUu9DiEAzNTWlY8eOnD59WuW6dO3alVmzZjFw4ECePHmCs7MzDx8+pHr16lSuXFleZFoZEYH2bkRMlXo+xHq9GbWVLT09XY7D+uOPP/jmm2+4ePGinKQg/f8ZodnbZGRkIElSrjiu7PfS0tKQJInMzEyFbbL/nh3VVVrlvE5N9i7tEx09oVT7ECPQ3oa7uzutW7fm2LFjPH78GFtb23w7eiICrXCImCr1fEj1yhmLldOVK1fkddYOHz5MTEyMQnpOZmYm/v7+/O9//2Pz5s3ExMQQFRWlcLyMjAxiY2N5/PixHLd17do1hW2uX78u//fBgwfv4xI1SlBQUHE3IV/Z67K+DdHRE0q1Dy0C7W2dOHGCtWvXAlCjRg369++f7/YiAu3diJgq9Yh6/Z+WLVvKz922adNGvguRrU+fPrRr147PPvsMKysr6tSpg4+PD+bm5rRo0QLI6tRIkoSLiwvVq1fn4cOHLF26VKG+Fy5coGHDhgwfPrxoL7CIlZTPVvZdk7chOnrCB0kqhRFohUlEoBUNUS/1fIj1Sk5OJiIiQv764cOH3Lx5E1NTU2rVqoW5ubnC9vr6+lSoUAErKyv09PSwsbHBwcGBadOm4evrKz+y4uTkRO3atQEYM2YMy5cvx8XFhQULFnDjxg18fHzw9vb+YOqt6Z+td2mbmIwhlGofSgTau7K3t+eHH34A4MmTJxw+fLhIzisIQv4uX75M8+bNad68OQCzZ8+mefPmLF26VOVj7Nq1C0tLS7p3746joyMdO3ZUWCPPxMSE48eP8+DBA1q2bMmcOXNYunSpWFqllBAjekKp9qFEoKWkpNCwYUNev35NQkICNWvWZMyYMXz11Vcq7f/NN9/g7OxMkyZNqFGjBh9//LFK+wmC8H517dpVpcitbPfu3cv1fJ+pqWmBM/JtbGw4d+7cW7VR0HCFmNAhCBpJRKC9fyICTT0lJXZJU5T2ehUUc/b5559LjRo1ksqWLStVqFBB6t69uxQaGqqwTe3atSVA4c9XX32lsE1mZqa0Zs0aqUGDBpK+vr5UvXp1afny5e/78jRaSflsvUsEmhjRE0q9DRs2sHfvXjIyMjA2NhYRaIIgaJTsmLMJEyYwePDgXO83bNgQHx8f6taty8uXL/H29qZnz55ERERQpUoVeTsvLy+cnZ05ceIE3bt3x9TUVOE4M2fO5Pjx46xduxZra2ueP3/O8+fP3/v1CcVLdPSEUu9DjkDbsmULPj4+uV7fsGEDnTp1KvQ2CoKgvoJizkaOHKnw9bp169i6dSvXrl2je/fu8uvly5fH3NycihUrYm5urvAA/+3bt9m4cSM3btygUaNGANSpU6eQr0TQRKKjJwgq0uQItJCQEKZNmwZkLRfQsWNHvv32WyZNmsSkSZOU7iMi0ASh5ElNTWXTpk2YmJjQrFkzhfdWrlzJsmXLMDEx4fbt28ydOxdd3axf8//73/+oW7cuR44cwcHBAUmSsLe3Z/Xq1blG/oTSRcy6FYRSoFmzZoSFhREeHs7169eJiYnh+++/z3cfEYEmCCXHkSNHMDIywtDQEG9vb4KCgqhcubL8vru7O3v27OH48eNyTm328k0Af//9N//88w/79u1j+/bt+Pv7c+XKFYYOHVoclyMUITGiJ5R6H0IEWtmyZeW/p6am8vLlywJnCYsItOLzIUZ6vYvSXC9lUWc5Y86ydezYkbCwMGJjY9m6dSvDhw/n/Pnzco7tjBkzgKyfPw4ODtjY2ODu7o6XlxcGBgakp6fz+vVrtm7dSsOGDQHw8/OjTZs2CrdzPzQiAk0QSrgPKQItMjKSAQMGcP/+ffr06YOrq6vK+4oItOLxIUV6FYbSWC9lUWc5Y86UGThwIMeOHWPhwoV5jsilp6eTnp7O9u3bqVGjBsnJyejo6BARESEvwJz9j7UDBw5ga2v77hdTgokINEEooT6kCDQLCwv+/PNPkpOTGT16NAcPHlS4vvyICLSiVVJilzTFh1avnDFneSlTpgwWFha5tsuuVbly5dDW1mbo0KFUrFgRPT099u7dS6NGjahXrx4Af/75JwBDhw6VR/k+NCXlsyUi0ARBTVIpjkAzMjLCycmJXbt2qdzRe5OIQCsaol7qKa31yi/mrFKlSnz55Zf0799fHu3/7rvvePz4MU5OTujp6RESEsLFixfp1q0bhoaGnDlzhp07dzJ69Gj51q6DgwMtWrRg6tSprF+/nszMTNzc3OjRo4e8mPuHTNM/WyICTRDy8KFEoEVERMjPcKSmpnLo0CFsbGxU2hdEBJogFKf8Ys50dHS4c+cOQ4YMoWHDhvTr14/Y2FjOnTsnd9AMDAzYs2cPXbp0wdbWln379uHu7q4Qc6atrc3//vc/KleuTOfOnenTpw+NGzdmz549xXLNQtERI3pCqfahRKCdPHmSb7/9Fh0dHdLT0+nevTufffaZynUSEWiCUHwKijk7ePBgvvu3aNGC0NBQIOtW5NGjR3F0dMw1ClS9enX5kRLhA1K4IR2CoHlEBNr7JyLQcvvqq68kQJo5c6YkSZIUGxsrubm5SQ0bNpQMDQ2lypUrS9OnT5fi4+MV9vvnn38kR0dHqUyZMlKVKlWkuXPnSmlpacVwBZqjpMRUaQJRK/WUlHqJCDRByIeIQBOKWlhYGH5+fgq3z6OiooiKimLt2rU0aNBAXs8sOjqa/fv3A5CRkUGfPn0wNzfnwoULPHnyhLFjx6Knp8eKFSuK63IEQSjBREdPKPVKewRa2bJlOXnyJIaGhrneExFoRS85OZlRo0axefNmli9fLr/etGlT+bZZWloaNjY2cjZpeno6urq6HD9+nFu3bhEcHEzVqlWxtbVl2bJlLFiwAE9PT3n9RUEQBFWJjp5QKmX/4ixM+UWgqSIzM2sNMG3tt5sDZWZmpnSWsJaWFq9evVLa0csvAk14P6ZPn06fPn2wt7dX6Ogpk5iYiLGxsfxZDQkJwdramqpVq8rb9OrVi2nTpnHz5k35YX1BEARViVm3QokzatQo7OzssLGxoU+fPkRHRxMZGUmFChVYsGABLVq0wMfHh+joaIYPH07r1q2xtrZmyZIl8jHmzp1Lq1atsLW1pXPnzvKs3Lz4+/vz8ccf079/f5o0aULnzp0Vlj1Zu3YtrVu3pkWLFjg4OPDPP/8A4OnpyZAhQ+jVqxdNmzblyZMn/Prrr7Rq1YpmzZpha2vLxYsXgazbfR9//DF2dnY0b96cffv2AcjX9vnnn9OyZUvq168vL7Lq4uICQKdOnbC1tSUmJobdu3fTpk0bmjdvTrNmzRSWi7lz5w7t2rXDysqKwYMH07NnT/z9/YGsdQQnT55M69atsbGxYcqUKSrP+hWy7Nmzhz/++IOvvvqqwG0TExNZsWIFU6ZMkV+Ljo5W6OQB8tfR0dGF21hBED4IYkRPKHHWr19PlSpVgKwQb09PTxYuXEhCQgJWVlasWrUKyBoJWbRoEV26dCE9PZ2+ffuyb98+hg0bxoIFC+QFgvfs2cPMmTMJDAzM97y///474eHhNG7cmNWrVzNlyhSOHz/O7t27uXv3LiEhIejo6LBjxw5cXV359ddfgaxRmqtXr1K1alX++usvxo8fz9mzZ7G0tCQtLY2UlJQCkzESEhKwsbHhiy++IDAwkJkzZ+Lo6Iivry9+fn6cO3eOChUqyNf9ySefoKWlRWRkJG3btuWff/7BwMCAMWPG4Orqyvjx47l9+zbNmzdn5MiRAMyZM4dOnTqxefNmJEli8uTJfPPNN8ybNy9XLUQEWm6/TWzCzJkzOXr0KDo6OqSlpSFJEpmZmbnii2JjY1m2bBmNGjVi8eLF8vuZmZlIkqSwffbflcVifShKSkyVJhC1Uk9JqZeIQBM+KLt372bHjh28evWKV69eycHeenp6jB49GoAXL15w4sQJ/vvvP3m/5ORkeeQuKCiIDRs2kJSURGZmJs+fPy/wvO3bt6dx48YATJkyhSVLlpCRkUFAQABhYWG0bNkSyHqgPidHR0d5VCYoKAgHBwc5qUNPTw8TE5MCkzEMDQ0ZPHgwAO3ateP+/ft5tvPBgweMGjWKR48eoaury/Pnz3nw4AHVq1cnPDycsWPHAtC4cWM6duwo7xcQEEBISAjr1q0D4OXLl/LyL28SEWi5bd68mZiYGFq3bi2/lpmZyblz5/juu+/Yt28fOjo6vHz5Ek9PT8qUKcPkyZMVopeSkpK4d++eQixW9mc4IiJCaVzWh0TTY6o0iaiVejS9XiICTfhgnD9/nm+//ZaQkBDMzMw4fPgwS5cuBbImJWQ//yb9/zWpQkNDcz279u+//+Lm5kZYWBj16tXj2rVrdO7c+a3bJEkSHh4eCrfgcjIyMlLpGPklYxgYGMhr8uno6OTqTObk5OTEypUr5QxMU1NTXr16pXTbnGsDSpLEgQMHVIpCEhFouXXq1Inhw4crvDZ58mQaNWrE3Llzadq0KYmJifTp0wczMzPc3Nzo06ePwlpn2tra7N+/Hzs7OznRYMuWLRgbGzN58mSlaSQfgpISU6UJRK3UU1LqJSLQhA9GXFwc5cuXp1KlSqSmpuLn56d0OyMjI7p16ybf2oWs5S0yMzNJSEhAT0+PatWqIUmS0lmpyoSEhHDnzh0sLS3ZsmUL3bp1Q0dHh4EDB/L1118zdOhQTE1NSUtL48aNG0ofnO/VqxdeXl7ycbJv3eZMxrC3tweykjGaNGlSYLvKly9PQkKCfOs2Li6OOnXqAFlJIHFxcQAYGxvTrFkzdu7cybhx47h79y7nz59n1KhRQFaix6pVq/Dz80NXV5e4uDhiY2OpX79+rnOKCLTcTE1NMTU1VXjNyMiIKlWq0Lx5c7mTl5KSgr+/P6GhocTGxqKnp0eVKlXQ0dHB0dGRJk2aMGHCBFavXk10dDSff/4506dPV+kfDKXdh/z5UpeolXo0vV7v0jbR0RNKFAcHB3bu3EmjRo2oVKkS9vb2PH78WOm2u3btYvbs2TRt2hQtLS3KlSuHn58fzZo1w8nJCSsrKypVqsTAgQNVOnf79u1ZsGABERERVKpUie3btwNZk0NiY2Pp1q0bkPUs1YQJE5R29OrXr8+2bdsYPXo0aWlp6Ojo4OvrS+vWrd86GWPOnDn06NGDsmXLcvz4cb755huGDh1KhQoV+Pjjj6lVq5a87fbt25kwYQJr1qyhfv36tGrVSu4gent7s3DhQmxtbdHW1kZXV5fVq1cr7egJ6vvjjz/kiTfZjwBke/DgARYWFujo6HDkyBGmTZtGu3btKFeuHOPGjcPLy6s4miwIQimgJUn55K4IggBkzboNCAhQOZJMUyUnJ1OuXDm0tLR48OAB7dq1IywsjI8++uidjpuYmIiJiQnPnj37YG/dqiO/mCohN1Ev1Ylaqaek1Cv7Z2xCQgLGxsZq7SuWVxGED8iFCxewtbWlWbNm9OvXD29v73fu5H2oNm7ciI2NDcbGxhgbG9OuXTt+++03+f3o6GjGjBmDubk55cqVo0WLFrlyRq9evUqPHj2oUKEClSpVYsqUKSQnJxf1pQiCUIqJW7eC8P/llTzRo0cP1qxZg7Ozc9E3qpD17NlT6TUK6qtZsyYrV66kQYMGSJLEjz/+yIABA7h69SpWVlaMHTuW+Ph4Dh8+TOXKldm9ezfDhw/n8uXLNG3alOfPnzN+/HhGjBiBj48PiYmJzJo1C2dnZzkSTRAE4V2Jjp4g/H95JU8IgjL9+vVT+PrLL79k48aNhIaGyjOoN27cKC+3smTJEry9vbly5QpNmzYlLCwMPT09vvvuO3m2uK+vLzY2NkRERIhnIwVBKBTi1q0gCMI7ysjIYM+ePbx48YJ27doBWZN39u7dy/Pnz8nMzGTPnj28evWKrl27AlnPBunr6ytE4pUpUwbIWkZIEAShMIiOniCUQspi4gD8/Pxo2LAhLVq0YNmyZQrr6OUVwSbk7fr16xgZGWFgYICLiwuHDh2Sl8T5+eefSUtLo1KlShgYGDB16lQOHTokj9TZ2NgQHR3NmjVrSE1NJS4ujoULFwLw5MmTYrsmQRBKF3HrVhBKIWUxcW5ubnh6enL16lXMzc35/PPP5e3zi2CrUaNGruN/6BFoNzx7AVC3bl3CwsJITEzkwIEDjBs3juDgYJo0acLixYuJi4sjMDCQSpUqcfjwYYYPH87JkyextLSkVq1abNq0CQ8PDzw8PNDR0cHNzY2qVavmikH70JWUmCpNIGqlnpJSr3dpn1heRRBKoW+++SZXTNzIkSO5evUq27ZtA+DRo0d89NFHSJLE0aNH+eSTT+SFlgGeP3+Ov78/H3/8ca7je3p6Ko1A2717N2XLln1/F6bhli5dirm5OYMGDWLatGl8++23CusYLl26lGrVqjFt2jSF/eLj4+X0k5EjRzJnzhw6dOhQ1M0XBEFDpaSkMHLkyLdaXkWM6AlCKZNfTFxOb8af5RXBpoyIQFNu/fr1VK1aVZ6A0aVLF4XFkb/77jtq1qxJjx49lMYu+fv7Y2hoyLx58+SFrIWSE1OlCUSt1FNS6iUi0ARBkOUVE5cdCRcTE4OZmRlbt26V98kvgk1fXz/XOUQEWlZnt3fv3tSqVYukpCR2797NmTNnOHbsGNbW1tSvXx83NzfWrl1LpUqVCAgIIDg4mCNHjsg12rx5M506dcLIyIigoCDmzZvHypUr5dvugqIP6fP1rkSt1KPp9RIRaIIgyPKKibO2tmbJkiV06NCB8uXL4+DggImJCQAVK1Z86wi2D1VMTAxjx47lyZMnmJiYYGNjw7Fjx+jRowcAR48eZeHChfTr14/k5GTq16/Pjz/+iKOjo/y8TVhYGF5eXiQnJ2NpaYmfnx9jxowpzssSBKGUER09QShl9PT02Lt3r8JrX375JQBjxozB1dUVyHqOL3spEIAWLVpw8uTJomtoCZdzRFSZBg0a5ErCeNO2bds0ehRBEISSTyyvIggfkIULF2Jra4uVlRVHjhzhu+++K+4mlTj5RZ9FRkaipaWl9E/2cjWxsbH07duX8ePHY2RkxEcffYSbm9s7PYMjCIKQFzGiJwgfENGxe3f5RZ9ZWlrmWgNv06ZNrFmzht69ewOgra1Nv3796NmzJwMGDOCff/5h+vTpPH/+nN27dxfHJQmCUIqJjp4gCIIaCoo+Mzc3V3j/0KFDDB8+HCMjIyDrecipU6dy9OhRateuTf369XF1dWXNmjVFdg2CIHw4xK1bodhpaWkRHx+v9n62trYkJSUVuF3Xrl3znFQQEBBAaGio2ufWFJGRkfj6+hZ3Mz5YyqLPcrpy5Qrh4eFMnDgxz2NERUVx8OBBunTp8j6bKgjCB0p09IQSKzw8nPLly7/TMYqyo5eenl7oxxQdveKRX/RZTlu3bqVx48a0b98+13tff/01JiYm1KhRA2NjY7Zs2VIUTRcE4QMjbt0KGuH7778nICCAp0+fsnTpUsaPHw/AvXv3mDVrFjExMbx+/ZopU6bg5uYGZI0ExsXFUaFCBS5cuICrqysZGRm0atWKK1eu8M0338gB8ufPn+frr78mKiqKHj164Ovry9GjRzl8+DBBQUH4+/vj5ubGpEmTlLbP2dkZbW1t7ty5w7Nnz2jXrh2+vr6UKVOGpKQkZs+ezZ9//smrV69o27YtPj4+6Ovr07VrV2xsbAgLC6NMmTKcPHmSVatWsXPnTrS1teXXypYty44dO/Dx8SEtLQ0jIyM2bNhAs2bN8Pf3Z+fOnVSpUoUbN25gYGDAzz//TN26dXFxceGff/7B1taWWrVqcfjwYebOncuZM2dIS0vD2NiYzZs306hRIwB++eUXFi5ciL6+Pg4ODmzdupXLly9jYWGRb63f9CFGoGXHnkH+0WfZXr58ye7du1m0aFGu+KK0tDQmTJjAN998w4MHD1iyZAmzZs1iw4YNRXY9JUlJianSBKJW6ikp9XqX9omOnqARDAwMuHTpEnfu3KFVq1aMGTMGLS0tPvnkE3bu3ImlpSUpKSm0bduWNm3a0KpVK3nf1NRURowYwfbt2+nWrRunTp2SY76y3b9/n1OnTpGWlkaTJk0ICQnB0dGR/v37Y2try6xZswps48WLFwkNDaVs2bIMHDgQb29vFi1axJw5c+jUqRObN29GkiQmT57MN998w7x58wD466+/OHv2LHp6evz4448cOHCA8+fPY2JiQlxcHAYGBvz+++/89NNPnD17FgMDA86dO8fIkSO5efMmkLXeWnh4OHXq1GHhwoWsWrUKPz8/fH19mTVrFuHh4XI7FyxYwNq1awHYs2cPM2fOJDAwkJiYGCZMmMDvv/+OpaUl27ZtIzY2Fsi6BalKrbN99dVXSiPQljTPpGzZjAJrWRIdPXpU6esdOnTg2LFjzJ8/X166BuDUqVO8ePECc3NzpftWrFiRf//9Fx0dHcaMGcOiRYto06YNpqam7+0aSrqgoKDibkKJIWqlHk2vV0pKylvvKzp6gkYYNWoUAJaWlujq6hIdHU1iYiI3b97EyclJ3i4pKYlbt24pdD7u3LmDrq4u3bp1A7ISIOrVq6dw/BEjRqCrq4uuri62trbcv39f6TNV+Rk+fLh8q3jixIl8++23LFq0iICAAEJCQli3bh2QNZKjo6Mj7zd69Gh5rbQjR47g4uKisFAxZI20/fnnn7Rp00be7/nz57x8+RKAdu3ayTm07dq1y3fkJygoiA0bNpCUlERmZibPnz8HIDQ0FBsbGywtLQEYN24cLi4uANy9e1elWmcTEWiKsqPPHB0d5dfWrVtHv379+OSTT3Jt/2bsUvbnqmPHjlhYWBRVs0uMkhJTpQlErdRTUuolItCEEs/Q0FD+u46ODunp6UiShKmpqcJolapy5rjmdfx3lX0OSZI4cOAADRs2VLpd9mzL/EiSxLhx41ixYoXS91Vt/7///oubmxthYWHUq1ePa9eu0blzZ5XOr06tP+QItPyiz7KvPSIignPnznH06NFc9Th69ChRUVGkpKTw+PFj/vrrL+bNm0eHDh1o0KBBcVxSifEhfL4Ki6iVejS9Xu/SNjEZQ9BYjRo1wtjYWOE2bEREhDxClXO7tLQ0zpw5A8CZM2eIiIhQ6RzGxsYkJCSotO3+/ftJTk4mIyODbdu2yZmwAwcOZNWqVXLnKy4uLs/z9+/fH19fX/mc8fHxZGRk0L9/f3bu3Mm///4LQGZmJpcvX1a7/QkJCejp6VGtWjUkScLHx0d+r23btly7do27d+8CsHPnTlJTUwHVay38X/RZo0aN6N69O2FhYQrRZwA//PADNWvWpGfPnrn2L1OmDFu3bsXDwwMbGxs+/fRT+vfvz5EjR4ryMgRB+ECIET1BY+nq6nLkyBFmzZqFt7c3GRkZVK5cOdeisgYGBuzZs4fp06eTmZlJy5YtadSoERUqVCjwHGPGjMHZ2ZmAgACmT5+e52QMgFatWtGrVy+ePn1Ku3bt5Of6vL295cQJbW1tdHV1Wb16NfXr11d6vqioKNq3b4+uri7lypUjODiYTp06sXr1agYNGkR6ejqpqan06dMHOzu7fNtvY2ODlZUVTZs2pW7duhw+fBgnJyesrKyoVKkSAwcOlLc1MzNjy5YtDBw4EAMDA3r06IGRkREVKlRQudZCwdFnACtWrMhzdLZbt26cPXuWo0eP4ujoqNGjCIIglAKSIJQCiYmJ8t8vXbokmZubSy9evCi0448bN07y9vYutOMVl5x1OnTokGRpaVkox01ISJAA6dmzZ4VyvPdpxYoVkp2dnWRkZCRVqVJFGjBggHTnzp1c2124cEHq1q2bVLZsWal8+fJSp06dpJSUFPn9K1euSPb29pKJiYlkamoqTZ48WUpKSlKpDampqVJAQICUmppaaNdVmol6qU7USj0lpV7ZP2MTEhLU3lfcuhVKhQMHDtCsWTNsbGyYOnUqO3bsoGzZssXdLI2TvWRL06ZNWb16Nbt27SruJhW5M2fOMH36dEJDQwkKCiItLY2ePXvy4sULeZuQkBAcHBzo2bMnly5dIiwsDDc3N7S1s35kRkVFYW9vT/369bl48SKBgYHcvHkTZ2fnYroqQRAE5cStW6FUcHZ2fudfskePHmXRokW5Xvfw8MDf3/+djq0pFi1apPQaPySBgYEKX/v7+2NmZsaVK1fkiSuffvop7u7uLFy4UN4uey1CyJo9raenx3fffSd3/nx9fbGxsSEiIkLpbXtBEITiIEb0hFJn/fr1REdHF7hdaGgo1tbWNG/enGPHjuHq6oq/vz/h4eEKf0aMGFEErRaKS/Zkluz162JiYrh48SJmZma0b9+eqlWr0qVLF86fPy/v8/r1a/T19eVOHmRNsgAUthMEQShuoqMnlDqqdvR+/PFHRo4cydWrV+nVq1eB26vrfUSeFcWxPySZmZnMmjWLDh060LRpUwD+/vtvADw9PZk8eTKBgYG0aNGC7t27c+/ePQA+/vhjoqOjWbNmDampqcTFxcmjf0+ePCmeixEEQVBC3LoVikxISAjz5s0jKSkJSZJYtmwZNWrUwN3dneTkZAwNDfH29qZDhw5ERkZia2tLfHw8AMnJyZQvXx5JkoCsNey+/PLLXLFpXl5eREVFMWLECMqUKYO/vz+2tra52rJy5Ur27t1LmTJl2Lt3L6dPn1Z4PyIiAhcXF2JiYtDW1sbT01OewXrs2DE8PDxIT0+nYsWKbNy4kSZNmnD69GmmT59O27ZtuXLlCosXLyYhIYF169ahr69PRkYGW7ZsUVgUOafTp0/j5uZGixYt+OOPPzAwMGDr1q3Y2toqPXZ2oocqkWXZ1q5dy88//0x6ejpmZmb4+flRu3ZtkpKSmDRpEuHh4VSpUgUrKytev36d5y3rkhqBljPGDMDNzY0bN27IqSmAvOTMpEmTGD16NACrV68mODiYzZs38+WXX9KwYUO2bt3K/Pnz8fDwQEdHBzc3N6pWrYokSQXGFZWU2CVNIeqlOlEr9ZSUeokINEHjPX/+nIEDB7J//346depEZmYmz549w87Ojs2bN9OrVy/Onz/PkCFDVF4DT1ls2tKlS/nhhx/Yu3ev0g5etoULF3Lnzp08489GjRrFhAkTmDp1Kvfu3aNt27Y0b96cMmXKMHLkSE6fPo21tTW7du1i6NChclTZ7du3+f777+UlOExMTLhz5w7VqlUjLS1NoXOkzM2bN/nmm2/Yvn07P//8M05OTty+fTvXsTMyMmjTpo3KkWUAu3fv5u7du4SEhKCjo8OOHTtwdXXl119/xcvLCwMDA+7cuUNiYqJ8rLyU1Ai0nFFkmzZt4uLFi6xYsYJr165x7do1AP777z8gq8OXc3sTExMuXrwov2ZiYoKfnx/x8fEYGBigpaXF+vXriY+PzzMu7U2aHrukaUS9VCdqpR5Nr5eIQBM0XkhICI0aNaJTp04AaGtr899//6GtrS3fNu3YsSNVq1YlPDycmjVrFnhMZbFpquxXkKSkJP744w9+//13ABo0aEDHjh05d+4cJiYmWFtbY21tLbdh+vTpPH78GMgKu+/SpYt8rO7duzNmzBj69etH796980zPyGZhYUH37t2BrMi1KVOm8PDhw1zHVjeyDCAgIICwsDBatmwJZOXbZjtx4gTe3t5oaWlhYmLCyJEjuX//fp7tLMkRaJIkyfnAZ8+ezZVGIUkSX3zxBWXKlFGINPv888/p1auXwms5+fv7Y2hoyLx58wpcw7GkxC5pClEv1Ylaqaek1EtEoAmlRnasmK6urkJH5NWrV7m2fR+xZgW1qyBvxp0dOHCAK1eucPr0aRwdHVm+fLlC50yV82afO+expbeIh5MkCQ8PD6ZMmaLSefNTkiPQXF1d2b17N7/88gumpqbExsYCWSN02RMq5s2bx+eff06LFi2wtbXlxx9/5O7duxw4cEC+Ph8fH9q3b4+RkRFBQUHMmzePlStXUqVKFZXbUhLqpUlEvVQnaqUeTa+XiEATNF779u25d+8e586dA7Iegq9atSqZmZnykPmFCxeIjo7G1tYWc3NzJEni1q1bAGzfvl3lc6kTa6ZM+fLladGihRwHFhERwfnz5+ncuTNt27bl+vXr3LhxA4A9e/ZQo0YNatSokes46enp3L9/Hzs7O+bOncvQoUO5dOlSvueOjIzk1KlTQFbkWtWqVZWOUr5NZNnAgQPx9fWVt0lLS+Pq1asA2Nvbs23bNiRJIjExkZ9++infdpZkGzduJCEhga5du1KtWjX5z969e+VtZs2ahYeHB59++inNmjXjxIkTBAUFUa9ePXmbS5cu0aNHD6ytrdm0aRN+fn64u7sXxyUJgiDkSYzoCUWiYsWKHDp0iDlz5pCUlIS2tjbLli3j4MGDuLu7M2fOHAwNDdm/f788crVhwwb69u1LpUqVGDp0qMrncnd3Z/LkyZQtWzbPyRgF2bVrFy4uLvj4+KClpcWWLVuoVauW/N7YsWPlyRj79u1TOgKWkZHBhAkTeP78Obq6ulSpUkWhY6aMlZUV/v7+uLu7o6+vz08//aT02G8TWTZq1ChiY2Pp1q0bkNURnTBhAs2bN+ezzz5j0qRJWFpaUqVKFTp27Fjg84QlVfaEnoIsXLhQYR29N6nzjw9BEIRiU2j5HIIgvJNTp05JzZo1K+5mSJIkSRs2bJDGjRun8vbFFYF25swZqW/fvlK1atUkQDp06JDC+0lJSdL06dOlGjVqSIaGhlLjxo2ljRs3KmwzZcoUqW7dupKhoaFUuXJlqX///tLt27ffa7tLSuySphD1Up2olXpKSr1EBJogCB+kFy9e0KxZM7777jul78+ePZvAwEB27tzJ7du3mTVrFm5ubhw+fFjepmXLlmzbto3bt29z7NgxJEmiZ8+eCs+ICoIglFTi1q1Qqnl5eXHw4MFcrx84cEDheauiZGdnl2viiJWVFbt27VJrcoUyMTEx9OzZM9frPXr0YM2aNSofp6D1+DRF79696d27d57vX7hwgXHjxtG1a1cApkyZgp+fH5cuXaJ///7ya9ksLCxYvnw5zZo1IzIystg+I4IgCIWlxHf0tLS0iIuLK3A5g9Lm9OnTvHr1CgcHh0I9bmRkJIGBgbi4uBTqcYuDnZ0da9euZenSpUV+7vXr1+Pk5IS5uXmu9y5fvlyo5woICMDc3Jy2bdtiZmb2zp3F0qR9+/YcPnyYCRMmUL16dU6fPs1ff/2Ft7e30u1fvHjBtm3bqFOnDh999FERt1YQBKHwiVu3Giy/5UJOnz6dK5xd1X3zExkZia+v71vtK2K5smYTZ2ZmqhzD9qa3qWFAQAChoaFq7/ch2LBhA02aNKFmzZro6+vj4ODAd999R+fOnRW2+/777zEyMsLIyIjffvuNoKAg9PX1i6nVgiAIhafEj+hB1g/pN6OwAObOncuZM2dIS0vD2NiYzZs306hRI16+fImzszPXr19HT0+PqlWrcvz48TyPv2XLFqUxVhYWFgQEBMizOrNHkLp27UrXrl2xtrYmNDSUuLg4BgwYwNq1a9HS0iI6Ohp3d3ciIyN5+fIlAwYMYPny5UDWraMRI0Zw6tQpGjRogKenJ87OziQnJ5OZmcmAAQMYOnQovr6+ZGRkcPr0aQYPHszYsWOxtbVl6tSpBAUFMXbsWOLj44mPj2f9+vVA1rpfly9flmOtVq1axc6dO9HW1qZMmTKcPHkSFxcX/vnnH2xtbalVqxaHDx8u8DptbGwICwuTj5FXzFZeLCwsGD16NCdPnuThw4csXrwYAwMDNm3axJMnT1i5cqW89tybI7iVK1fm8uXLWFhYcOHCBVxdXUlPT6dVq1YFdpqyY9ZmzJjBr7/+SlJSEv7+/uzfv59Tp06Rnp7Onj175AzUNWvW4O/vj7a2NjY2Nnz//feYmJjg6enJ9evXSU5O5uHDh4wYMSJXDFtsbCxLlizh1atXpKamMnv2bCZOnAiAs7Mz2traREREEBMTw507d5S2NzQ0lOnTp5ORkUF6ejrTp0+ndu3aHD58mKCgIPz9/XFzc2PSpEn5tvX27dukpKRw//59zM3N2b9/P6ampkDeEWlv0oQItDfjzCCro5wzKmj9+vWEhIRw8OBBatWqxfnz55k+fTpmZmbywtSQtTh1165diY6OZt26dQwbNowzZ84orNVYmEpK7JKmEPVSnaiVekpKvT74CDRlUVi6urosWLCAtWvXAlnrnc2cOZPAwEACAwOJj4+X12jLb+0xgDlz5qgVY5Xt1q1bXLhwgbS0NDp37sxPP/3EyJEjGTduHIsWLaJLly6kp6fTt29f9u3bx7BhwwCIjY3l4sWLaGlpMXPmTPr27YuHh4fcVlNTU1xcXBQ6cZGRkSQkJGBlZcWqVauArFD2vPz4448cOHCA8+fPY2JiQlxcHAYGBvj6+sqpAar666+/OHv2LHp6evnGbOXnxYsXXLhwgYiICKytrVm8eDEhISGEhYXh6OhY4CLDqampjBgxgm3btmFvb8/x48fzzGnNKSEhgZYtW7Js2TK2bt1Kr169+N///oe3tzdr1qzhiy++YN++ffz222/88MMPhISEUKFCBaZMmcLChQvZuHEjkJX8cfXqVapWrQrAtm3bFGLY4uLiOH/+PDo6Ojx//pzmzZvTq1cveY28K1eucP78ecqXL59nW7/66ivmzp3LJ598Ih+zYsWK9O/fXyHKraC2Xrx4kStXrlCpUiWcnJzw8/PDw8NDre+dJkSgKYsZu3Lliryw6OvXr1myZAkLFy5EW1ubR48eYWFhQdu2bVm0aBGff/650uM6OzszevRoPD09c438FTZNj13SNKJeqhO1Uo+m1+uDj0DLKworKCiIDRs2kJSURGZmptyha9asGbdv38bV1ZUuXbrkGWmUTd0Yq2xjx46VV9sePXo0wcHBDBgwgBMnTsh5mgDJycncvXtX/trZ2VleO61z587MmzeP5ORkunTpgr29fZ7nyz6PKo4cOYKLiwsmJiZA1jp3b2v06NHyL9f8YrbyM2LECADq16+PoaGhvG6enZ0dz58/Jz4+Pt/nMO/cuYOurq5cn549e1K3bt0Cz2toaMjAgQPlcxkZGcnrzLVu3Zpdu3YBEBwczIgRI+Q2TJs2Te6YAzg6OsqdPGViY2OZOHEif/31F7q6usTGxnLjxg25ozds2LB8O3kA3bp1Y9myZdy7d4+PP/6Yjh07Kt2uoLY6ODjIMWXt2rXj+vXrgHrfO02NQGvZsqX8/3NiYiLp6em0bt1a4VnWI0eOAOT5//3r16/R1tamSZMmBf5seFslJXZJU4h6qU7USj0lpV4ffASasiisf//9Fzc3N8LCwqhXrx7Xrl2T/3Vet25dbt26xcmTJwkODmb+/PmEh4fn2dnJK8ZKlZiunLS0tOTFWkNDQ/O8LZQz6mrIkCG0b9+eoKAgfHx8WL9+fZ6B6WXLlkVb+/8eu1S3fXkp6DhvRnOpGrOV05vfw+yvsyPAsm/D6ujoqHxNqsSW5Yzxynne7K/zuv375rHfjD57k4uLC46Ojhw4cAAtLS1atGih0PaC9oestIYBAwYQHBzMokWLaNq0Kd9//32B+73Z1ryuUZ3vnaZEoCUnJxMRESF//fDhQ27evImpqSm1atWiS5cueHh4UL58eWrXrs2ZM2fYuXMn69atQ09Pj7///pu9e/fSs2dPqlSpwqNHj1i5ciVlypShX79+7/1aND12SdOIeqlO1Eo9ml4vEYGmREJCAnp6elSrVg1JkvDx8ZHfe/ToEVpaWvTv35+1a9ciSZIcHP+m/GKs6tevz8WLF4GsOKSco3IAO3fuJC0tjZcvX7J7927s7e3lEaOVK1fK20VFRfHo0SOl57937x5Vq1Zl7NixrF69Wn7oXpWYr/r163P58mUyMjJISUnhwIED8nv9+/fH19dXPkZ8fDwZGRlKj1vQdeaUX8xWYcjZloMHD/LixQsgazQ3PT1djg8LDg7m/v37hXZee3t7fv75Z/lfVX5+fkqXMcn2Zh3j4uKoXbs2WlpanD17lj///FPtNty9e5c6deowefJkFi1alOdnQd22Znvf37v34fLlyzRv3pzmzZsDWevmNW/eXJ5pvWfPHlq1asWoUaNo0qQJK1eu5Msvv5RnlRsaGnLu3DkcHR2pX78+I0aMoHz58ly4cAEzM7Niuy5BEITCUipG9JSxtrbGyckJKysrKlWqJN+eA7h+/ToeHh5IkkR6ejpjxozBxsZG6XHyi7Favnw548aNw8/Pj3bt2mFlZaWwb+PGjenQoQPPnz9nwIAB8nNmu3btYvbs2TRt2hQtLS3KlSuHn5+f0kzT/fv3s3PnTvT19cnMzJRnxA4aNIgdO3Zga2srT8Z40+DBg9m3bx+NGzemZs2aNG/eXL7PP2bMGKKiomjfvj26urqUK1eO4OBgbGxssLKyomnTptStW5fDhw8XeJ055RezVRi8vb1xd3dnyZIl9OnTR75NqK+vz969e3F1dSUjI4NWrVrRrFmzQjknZK3XduPGDdq1a6cwwSEvb8awrVy5EldXV5YtW4atrS1t2rRRuw0+Pj6cPHkSfX19dHR0+Prrr4Gs76WzszMBAQFMnz6dSZMmqdXWbO/7e/c+dO3aNd9IM3Nz83xj56pXr57nCLkgCEKpUIgJHUIOXbp0yRXHJAilVVFFoBUUeTZu3DgJUPjTq1cvhW1q166da5uvvvrqvbb7TSUldklTiHqpTtRKPSWlXu8SgVZqR/QEQSh9siPPJkyYwODBg5Vu4+DgoDCKp+xZQi8vLyZPnix/XdBEGEEQhJJKdPT+v6NHj7Jo0aJcr3t4eMgzQtVx+vTpQmhV6VGcUWT9+/fn33//VXitYsWK8vN8mqSwIsxKq4IizyCrY6cskSSn8uXLF7iNIAhCaSA6ev+fo6Pje1tKoShpaiTc0qVL32sUWX6RcDkD7NVV1JFwZmZmODs7K8Sn+fr6kpSUpPaxPD09FdZa/FCcPn0aMzMzKlasyMcff8zy5ctzLfmycuVKli1bRq1atRg5ciSffvopurrix6EgCKWP+MkmlBjp6el5/jI+ffo08fHxeWb/5rdvfrIj4d6mo/e251y/fj1du3aVO3rFmTv8ttdQXBwcHBg8eDB16tTh/v37LFq0iN69e8uLQEPWRJkWLVpgamrKhQsX8PDw4MmTJ6xbt66YWy8IglD4Ss5PcEFlIhKu5EbCeXl55YpPCwgIkNvt7+/Pzp07qVKlCn/++ScVKlRgy5YtLF68mDt37vDRRx9x8OBBeV2+hw8f8vHHHxMVFUWDBg3w9/enUqVKpKWl8dlnn3Hy5ElSU1Np2LAhfn5+VKxYUaVItqKOQFMWdwa5I8+GDBki/93S0pLGjRtjaWlJcHAwH3/8MQAzZsyQt2ncuDE6Ojq4urri5eWl9Hm+96GkxC5pClEv1Ylaqaek1OuDj0ATFIlIuJIbCbd06VJ++OEHhfi0gIAAhW3CwsK4fv06tWrVkhNbLly4QNWqVenbty8//vgj06dPB+DcuXNcu3YNc3NzXF1d8fDwYNOmTaxZs4Zy5crJa0IuW7aMJUuW8N133wEFR7IVdQRaXkug5Iw8y4uxsTG//PJLnotrv3r1ivT0dLZv306NGjXeua3q0PTYJU0j6qU6USv1aHq9PvgINEGRiITLUpIj4fLTrl07atWqBWSNJqalpcnxa61ateLevXvytn369JFvAU+ZMkWeqRoQEEBCQoK8iHZqaioWFhbyfgVFsmlKBFrOyDNlHj16RFJSEvb29nlut3v3brS1tRk6dOg7fd/VUVJilzSFqJfqRK3UU1Lq9cFHoAmKRCRclpIcCZefvOLisr/OK7YN/i8OTZIkNmzYkGdiRkGRbMUVgZZf5JmpqSlffPEFQ4YMwdzcnPv37zN//nzq169Pnz590NPTIyQkhIsXL9KtWzfKly9PSEgI8+bNY/To0cWShKHpsUuaRtRLdaJW6tH0eokINKFAIhKu5ETCqXItqjp69Kg8WrplyxZ5BHTgwIF4e3vLtwNSUlK4efNmoZzzfcov8kxHR4dr167Rv39/GjZsyMSJE2nZsiXnzp2TO6UGBgbs2bOHLl26YGVlxZdffsmnn37Kpk2bivOyBEEQ3hsxoveBEJFwJScS7s34tHfRqVMnRo4cyePHj+XJGAALFizg9evXtGnTRh7lW7BgQb7XogkKijw7duxYvvu3aNFC/seBIAjCB6EwIzoEIS8iEq50K4oItMKIP4uNjZVGjhwplS9fXjIxMZEmTJggJSUlvbc256WkxC5pClEv1Ylaqaek1OtdItDErVtBEEqE7Piz7JnByjg4OPDkyRP5z08//aTw/qhRo7h58yZBQUEcOXKEs2fPFurzk4IgCJpG3LoVlBKRcO9XcUbClVTvGn92+/ZtAgMDCQsLw87ODoANGzbg6OjI2rVrqV69eqG3WRAEobiJjp6glKZEwimLIHtzweLC9vz5c/r168eLFy8YNmwY9+7dw9bWllmzZhXaOfKKhPP39yctLQ1LS8tCO9eHJL/4s5CQECpUqCB38gDs7e3R1tbm4sWLDBo0qLiaLQiC8N6IW7eCRsuOICts+S1BEhQUhJGREeHh4SxevLhIz+3v7680iUIomIODA9u3b+fEiROsWrWKM2fO0Lt3b3kpnOjo6FxLqOjq6mJqakp0dHRxNFkQBOG9EyN6gsY4duwYHh4epKenU7FiRTZu3Kg0ggzg4MGDTJs2jejoaCZOnMiSJUsA1IpT27VrV642BAcHM2/ePBISErC1tZWTRLIlJyfj7u4uLykzbNgwPv/8cwAiIiJwcXEhJiYGbW1tPD095dnNWlpaLF26lKNHj9K1a1fWrFmT69xbtmzh8uXLfPrpp3h6erJixQoWLFiAn58f7du3B2DTpk2cOHGCvXv3vnWsXF48PT3ZtWsXxsbG9O7dm507dxIZGal0W02IQFM3/iwjIwNJkpRGCWVkZBRpBFJJiV3SFKJeqhO1Uk9JqZeIQBNKvJiYGEaOHMnp06extrZm165dDB06lI0bN/Lpp5/miiCLj48nJCSEZ8+eUa9ePcaPH0+NGjXUilNTxt7eHi8vLwICAuTosZ07d8rvL1u2jNevX3Pt2jVevnxJx44dsbS0ZMSIEYwaNYoJEyYwdepU7t27R9u2bWnevLmca6ujo0NYWFieNZg0aRI7d+5k1qxZcgfx8ePH+Pj4yB297777TmENxLeNlXvTr7/+yr59+7hy5Qrly5dnzJgxeX+z0IwINHXjz2JiYoiKilI4VkZGBrGxsTx+/DjPhbffJ02PXdI0ol6qE7VSj6bXS0SgCSXexYsXsba2xtraGsiaHTl9+nQeP36sdPuRI0cCULlyZerWrcuDBw+oUKGCWnFqbyM4OJivv/4abW1typUrx9ixYwkKCsLR0ZE//viD33//HYAGDRrQsWNHzp07J3f0JkyYoPb5Ro8ezdKlS/nvv/+4d+8eWlpadOrUSX7/bWPl3nTixAmGDx+OsbExAFOnTuX8+fN5bq8JEWjqxp/VqVMHHx8fzM3NadGiBZD1w12SJFxcXIp0MkZJiV3SFKJeqhO1Uk9JqZeIQBM+OMpivyQ149QKQ36dxjffe5tzlylTBmdnZ/z8/Lh9+zbTp08vsD2q1KEgBXWGiyMC7V3jz2xsbHBwcGDatGn4+vqSlpbGrFmzcHJykjvjRU3TY5c0jaiX6kSt1KPp9RIRaEKJ17ZtW65fv86NGzcA2LNnDzVq1KBx48Yqx4GpG6f2Nuzt7dm6dSuSJPHixQt27NhBz549KV++PC1atJBTQiIiIjh//rycJ6wqZfFn06dPZ9OmTZw8eZJRo0YpvFcYsXLZ17Vv3z6SkpKQJEkjI8HeNf4MslJYLC0t6d69O46OjnTs2FEjr1UQBKGwiBE9QSNUqVKFXbt2MXbsWHkyxr59+2jYsGGuCLL8qBOn9jY+++wz3N3d5VvMw4YNY/jw4fK5XVxc8PHxQUtLiy1btlCrVi21jj9lyhTmzJmDt7c3K1aswNHRUY5ra9iwIWXLllXYvjBi5SBrOZ1Lly7RokULeTKGpnnX+DMAU1NTdu/eXZjNEgRB0GyFGdEhCELhS05OlmrUqCH9/fffCq+/z1i569evS7Vr11Z5e02IQMtp6tSpEiB5e3srvF67du1cMWlfffXVe2tzXkpK7JKmEPVSnaiVekpKvUQEmiCUUr6+vlhaWuLq6kqdOnWKuznFSpUINIBDhw4RGhqa5+QKLy8vhZi0GTNmvI/mCoIgaARx61b4IBV2xJs6tmzZorBESrYNGzYozKgFcHFxUUgFyeltYuXs7OxyLdhsZWWVa03Bpk2b5rmGXnFRJQLt8ePHzJgxg2PHjtGnTx+l25QvXz7PmDRBEITSRnT0ipGWlhZxcXFUqFChuJtSpE6fPs2rV69wcHAo1OMqi0vLy/uMeIuPj8fX15eFCxcqfX/SpElMmjTpvZwbYOLEiYSEhNCoUSMGDBigsCbg5cuX39t5i1tmZiZjxoxh3rx5WFlZ5bndypUrWbZsGbVq1WLkyJF8+umn6OqKH4WCIJRO4tat8F7kF/N1+vRpAgMD32rf/LxLXNrbnlOZ+Ph4hRmvRem///5jz549XL9+nUOHDhVLG4rLqlWr0NXVxd3dPc9t3N3d2bNnD6dOnWLq1KmsWLGC+fPnF2ErBUEQipb4Z2wx+/777wkICODp06csXbqU8ePHAzB37lzOnDlDWloaxsbGbN68mUaNGvHy5UucnZ25fv06enp6VK1alePHj+d5/C1btrBu3Tr09fXJyMhgy5YttGnTBgsLCwICArC1tQWybumtXbuWrl27vnW01psRY56enjg7O5OcnExmZiYDBgxg6NCh+Pr6kpGRwenTpxk8eDBjx47F1taWqVOnEhQUxNixY4mPjyc+Pp7169cD4OPjw+XLl/H39weyfqnv3LkTbW1typQpw8mTJ5XGpRV0nTY2NoSFhcnHWLt2LT///DPp6emYmZnh5+eX5xprmZmZuLu7c+LECfT19dHV1eX333/HxcWFpKQkbG1t0dXV5fLlywXGo3355ZdKPwf37t1j1qxZxMTE8Pr1a6ZMmYKbm5vS9sTHx9OtWzdevXpFy5YtcXJyUrhFGR0dzSeffEJiYiKvXr2iW7dufPvtt2hra5OWlsbMmTMJDg7G1NSUDh06cOXKlTxvDxdlBJqy+DNQjED7448/+Oabb7h48aJCp/3NaLOcz+M1btwYHR0dXF1d8fLyUrou4PtSUmKXNIWol+pErdRTUuolItBKMAMDAy5dusSdO3do1aoVY8aMQVdXlwULFsg5q3v27GHmzJkEBgYSGBhIfHw8t27dAuD58+f5Hn/OnDncuXOHatWqkZaWpvDLOT9vG62VM2Js5syZ9O3bFw8PD7mtpqamuLi4KHTiIiMjSUhIwMrKilWrVgFZuat5+fHHHzlw4ADnz5/HxMSEuLg4DAwM8PX1ZdasWbni0vLz119/cfbsWfT09Ni9ezd3794lJCQEHR0dduzYgaurK7/++qvSff/8809OnDjBzZs30dbWJiEhAX19fXx9fbG1tVVoR0HxaMo+B1paWnzyySfs3LkTS0tLUlJSaNu2LW3atKFVq1a52lOhQgWOHj2qcO7sjnH2+//73/8wMjIiIyODAQMG8PPPP+Pk5MSmTZu4d+8eN2/eBCjwtnZRRqDlFU2WMwLt8OHDxMTEULduXfn9zMxM5s+fz6pVq9i8ebPSY7x69Yr09HS2b99OjRo1CrXdqtD02CVNI+qlOlEr9Wh6vUQEWgmWvQCupaUlurq6REdHU7NmTYKCgtiwYQNJSUlkZmbKHbpmzZpx+/ZtXF1d6dKlS4G/kLt3786YMWPo168fvXv3pmHDhiq1622jtXJGjHXu3Jl58+aRnJxMly5dsLe3z/N82edRxZEjR3BxccHExASAihUrqrSfMqNHj5Y7CwEBAYSFhdGyZUsgazQoP3Xr1iU9PZ0JEybQrVs3+vTpg7Z27qchkpKSCoxHU/Y5SExM5ObNm/LaeNnHunXrltKOXkEyMzNZsGAB58+fR5IkYmJiaNq0KU5OTpw4cUKhFuPGjWPLli15HkvTItDatGmTa6Szb9++8j9OGjVqpPQYu3fvRltbm6FDh77T50hdJSV2SVOIeqlO1Eo9JaVeIgKtBFMW5fXvv//i5uZGWFgY9erV49q1a3LCQt26dbl16xYnT54kODiY+fPnEx4enucvqQMHDsi34BwdHVm+fDlOTk7o6uoqdGRevXqVbztVjdbKGfM1ZMgQ2rdvT1BQED4+Pqxfvz7P0ZmyZcsqdJLUbV9eCjpOzvZKkoSHhwdTpkxR6dgmJibcuHGDM2fOcOrUKTw8PDh79qxKD/a/GTGWV6SbqampWiOU+Vm3bh0xMTFcvHgRQ0NDZs+enWddS1oEWq1atXLNpNXT06NGjRo0bdoUgJCQEC5evEi3bt0oX748ISEhzJs3j9GjR2NmZvZe2lwQTY9d0jSiXqoTtVKPptdLRKCVMgkJCejp6VGtWjUkSVJYiuPRo0doaWnRv39/1q5diyRJPHz4UOlx0tPTuX//PnZ2dsydO5ehQ4dy6dIlAOrXr8/FixcBuHTpUq7A+8KI1rp37x5Vq1Zl7NixrF69mtDQUEB5zNeb6tevz+XLl8nIyCAlJYUDBw7I7/Xv3x9fX1/5GPHx8WRkZCg9bkHXmdPAgQPx9fWVR0/T0tK4evVqnts/ffqUFy9e0LNnT1asWIGFhQW3bt3C2NiYly9fkpqaCvDW8WiNGjXC2NhY3i9734Ju1+clLi4Oc3NzDA0NiY6OZt++ffJ7H3/8Mbt37yYtLY20tDS2b9/+Vud4n/KLQFOFgYEBe/bsoUuXLlhZWfHll1/y6aefigg0QRBKNTGip4Gsra1xcnLCysqKSpUqyQ/tA1y/fh0PDw8kSSI9PZ0xY8ZgY2Oj9DgZGRlMmDCB58+fo6urS5UqVeROw/Llyxk3bhx+fn60a9cu13IUhRGttX//fnbu3Im+vj6ZmZnyjNhBgwaxY8cObG1t5ckYbxo8eDD79u2jcePGcgRY9jMKY8aMISoqivbt26Orq0u5cuUIDg7GxsYmV1xaQdeZ06hRo4iNjaVbt24A8m3Z7I7Fmx4+fMjkyZNJS0sjIyODDh060Lt3b/T09Bg7diw2NjYYGRlx+fLlt4pH09XV5ciRI8yaNQtvb28yMjKoXLnyW0d4zZw5k6FDh2JlZUX16tUVbqVPnTqV69ev06RJEypWrIidnR1RUVFvdZ73paAItDe9uQ5gixYt5H9sCIIgfDAKMaFDKCXeZ7SWoLkSExMlScqKBBo2bJi0cuVKlfd93xFoBcWfff7551KjRo2ksmXLShUqVJC6d+8uhYaGKmxz5coVyd7eXjIxMZFMTU2lyZMnS0lJSe+lvQUpKbFLmkLUS3WiVuopKfUSEWiCILwze3t7bG1tsba2xtjYON/16IpaQfFnDRs2xMfHh+vXr3P+/HksLCzo2bMnT58+BbIeMbC3t5dv5QcGBnLz5k2cnZ2L8CoEQRCKnrh1WwoUdpzX20RrlWZeXl4cPHgw1+sHDhygXr16Rd6emJgYevbsmev1Hj16sGbNmrc+bvazjJqooPizkSNHKny9bt06tm7dyrVr1+jevTtHjhxBT0+P7777Tp704+vri42NDREREdSvX/+9tl8QBKG4iI5eKfA+47wEWLp0qcoP/OfHz88Pb29vDA0NOXHihNJlSFSJhzMzMyu0mbilUWpqKps2bcLExIRmzZoBWQs86+vrK8zsLlOmDADnz58XHT1BEEotcetWENTwLlFp69evZ9u2bYSHh+e51lxB8XBC3o4cOYKRkRGGhoZ4e3sTFBRE5cqVgaxZxdHR0axZs4bU1FTi4uLkLOInT54UZ7MFQRDeKzGiJwgF0NLSYunSpRw9epSuXbsyb948XFxcuHfvHpIkMWPGDKZOnQpkLQHi7u5OcnKy3OHo0KEDQ4cO5f79+zg7O2Ntbc3+/ftznSc8PDxXPFxMTAzVq1eXb83fvXsXe3t7Hjx4wPLly7l+/TpxcXFERUXRoEED/P39qVSpEmlpaXz22WecPHmS1NRUGjZsiJ+fX76LAh8+fJgFCxagp6eHg4MDP/zwA5cvX8bCwiLXtkUVgaZK/Fm2jh07EhYWRmxsLFu3bmX48OGcP38eMzMzGjZsyNatW5k/fz4eHh7o6Ojg5uZG1apVkSSpyOOPSkrskqYQ9VKdqJV6Skq9RASaILxnOjo6hIWFATBixAgaNWrEwYMHiYmJoWXLljRr1owWLVowePBgNm/eTK9evTh//jxDhgwhIiKC/fv3Y2Fhwd69e+Xc3TfZ2trmioe7e/cuvXr1YsGCBejo6PD9998zZcoUeVHmc+fOce3aNczNzXF1dcXDw4NNmzaxZs0aypUrJ6+buGzZMpYsWZLnZIaYmBjGjx/PuXPnaNKkCZs2bSI2NjbPehRVBJoq8WfKDBw4kGPHjrFw4UKGDh0KZC1w7efnR3x8PAYGBmhpabF+/Xri4+PzPM/7pumxS5pG1Et1olbq0fR6iQg0QXjPJkyYIP89ODiYK1euAFnPyw0ePJjg4GDKlSuHtrY2vXpljUJ17NiRqlWrEh4eTseOHd/qvI0aNaJJkyb88ssv9OrVi59++onr16/L7/fp00dOhJgyZQqDBw8GsuLcEhIS5IWmU1NTlY7MZQsNDcXGxoYmTZoAMHHiRGbMmJHn9sUdgZYz/iwvZcqUwcLCIs/t/P39MTQ0ZN68eVSoUOE9tDJvJSV2SVOIeqlO1Eo9JaVeIgJNEN6znFFpb8ovLqygKDFVzJw5k1WrVvH06VN69OhB1apVCzyfJEls2LBB6excVWhaBFp+8WeVKlXiyy+/pH///lSrVo1nz57x3Xff8fjxY5ycnOT2+Pj40L59e4yMjAgKCmLevHmsXLmSKlWqFHp7VaXpsUuaRtRLdaJW6tH0eokINEEoQvb29mzevBnIikE7ePAgPXr0oFGjRmRmZsq3AC5cuEB0dHSet2qVURbj1rNnT6Kjo1m+fDlubm4K7x09epT//vsPgC1btshpFwMHDsTb21se7k9JSeHmzZt5nrddu3Zcu3aNO3fuAPDDDz/IEW6aIL/4Mx0dHe7cucOQIUNo2LAh/fr1IzY2lnPnzikkoVy6dIkePXpgbW3Npk2b8PPz06i1AgVBEN4HMaInCGr69ttvmTZtGtbW1kiSxOLFi2nTpg0ABw8exN3dnTlz5mBoaMj+/fvzHQ1805vxcEuXLkVLS4uJEyeye/du2rVrp7B9p06dGDlyJI8fP5YnYwAsWLCA169f06ZNG3l0bsGCBXlGwFWpUoUffviBQYMGoa+vj4ODQ5HcglVVQfFnytY5fJMm5vcKgiC8b6KjJwgFeLODUbVq1Tw7FnZ2dly4cEHpe29mrypTp04drl69muv1U6dOMWvWrFyv16xZU+kMXl1dXby8vPDy8irwnNkGDBjAgAED5K+3bt2q8r7vy9mzZ1mzZg1XrlzhyZMnHDp0SM5+TktLY8mSJRw9epS///4bExMT7O3tWblyJdWrVweylqvJzi5+06VLl2jVqlVRXYogCEKxELduBUGDXb58mfr166OtrZ0r/eFDkF/0WUpKCn/88QefffYZf/zxBwcPHuTu3bv0799f3qZ9+/Y8efJE4c+kSZOoU6cOdnZ2RXkpgiAIxUKM6AlCEQsPD1easTpu3Dg+/fRThdfs7OwUJiHk5Onpqfa5XVxcCA0NzfV6SEiInBSR7dmzZ2ofv7DlF31mYmKSa0kEHx8fWrduzb///kutWrXQ19eXZyVD1ijgL7/8wowZMwploowgCIKmEyN6glCI1q9fT3R0dL7b2Nraygsja2lpsWrVKsLDw3N18tQVFRVFp06d5K+1tLSIj48HsmLy7t69i6+vL+Hh4bn+vNnJK6kSEhLQ0tLKc7mUw4cPExsby/jx44u2YYIgCMVEdPQEoRCp0tED+PHHHxk5ciRXr16V1917V9WrV+fcuXNK3zt69CiNGjUqlPNoqlevXrFgwQI++eQTjI2NlW6zdetWevXqRc2aNYu4dYIgCMVD3LoVPgghISHMmzePpKQkJEli2bJl1KhRQ2lcWWRkJLa2tvJoWHJyMuXLl5cnZWhpafHll18SEBDA06dPWbp0KePHj8fLy4uoqChGjBhBmTJl8Pf3V7q0ysqVK9m7dy9lypRh7969nD59muXLl3PmzBnS0tIwNjZm8+bNcsdMS0uL5cuXc/jwYf777z/Wr1/P7du3OXDgAAkJCWzevJmuXbvmandOFhYWBAQEYGtrS3R0NO7u7kRGRvLy5UsGDBjA8uXLyczMxN3dnRMnTqCvr4+uri6///47hoaGuY73viPQ1Ik+g6xbssOHDyczM5Nvv/1W6TaPHj3i2LFj7N69u9jjjkpK7JKmEPVSnaiVekpKvUQEmiDk4/nz5wwcOJD9+/fTqVMnMjMzefbsGXZ2dkrjylRhYGDApUuXuHPnDq1atWLMmDEsXbqUH374Id+YM4CFCxdy584dbG1t5Zm0CxYsYO3atQDs2bOHmTNnEhgYKO9jZGTExYsXOXHiBAMGDMDHx4fLly+zb98+5s2bJ8ezqWLcuHEsWrSILl26kJ6eTt++fdm3bx/169fnxIkT3Lx5E21tbRISEtDX11d6jPcdgaZO9Fl6ejpr1qzhv//+w8vLi/Pnzyvdd+/evZQvXx5dXd1iizx7k6bHLmkaUS/ViVqpR9PrJSLQBCEfISEhNGrUSH5+TVtbm//++y/PuDJVbuuNGjUKAEtLS3R1dYmOjn6n24FBQUFs2LCBpKQkMjMzef78ucL7I0aMALImZ7x48QInJycAWrduzb1791Q+z4sXLzhx4oS8yDJkjVjevXuXnj17kp6ezoQJE+jWrRt9+vRBW1v50x3FFYH2ZvRZWloan3zyCUlJSfz+++95plxIksSnn37KhAkTFGblFpeSErukKUS9VCdqpZ6SUi8RgSYIhSB7Fqauri4ZGf83KvXq1atc2+a8namjo0N6evpbn/fff//Fzc2NsLAw6tWrx7Vr1+jcubPS8+no6OT6Wp1zZ99+Dg0NVXpL9saNG5w5c4ZTp07h4eHB2bNnqV+/fq7tiioCLb/os2rVqvHJJ5/wxx9/cOTIEbS1tYmNjQXA1NRUYTTyxIkTPHjwgClTpmjUD3NNj13SNKJeqhO1Uo+m10tEoAlCPtq3b8+9e/fkiQqZmZlUrVo1z7gyc3NzJEni1q1bgHqJCsoizAqSkJCAnp4e1apVQ5IkfHx81NpfHUZGRnTr1o2VK1fKr0VFRfHo0SOePn3Kixcv6NmzJytWrMDCwkKuQXHJL/rs8ePHHD58mEePHmFra0u1atXkP28uWr1161bat2+PpaVlcVyGIAhCsREjekKpV7FiRQ4dOsScOXNISkpCW1ubZcuW5RtXtmHDBvr27UulSpUYOnSoyudyd3dn8uTJlC1bNs/JGG+ytrbGyckJKysrKlWqJCc/vC+7du1i9uzZNG3aFC0tLcqVK4efnx8ZGRlMnjyZtLQ0MjIy6NChQ55r2BWVgqLP8nsvp927dxdWkwRBEEoULUnVn5SCIAh5SExMxMTEhGfPnhXaM3rvGn+WU3bu759//snVq1dV6oC/T2lpaRw9ehRHR0eNvl2kKUS9VCdqpZ6SUq/sn7EJCQl5Lh+VF3HrVhAEjfSu8Wc5zZ8/X2kHUBAEobQTt24F4T3x8vLi4MGDuV4/cOAA9erVK4YWlSzvGn+W7bfffuP48eMcOHCA33777b22WRAEQdOIET1BeE+WLl2qEDM2bdo0Xr16xZAhQ+TZoW86ffq0wvp5guqUxZ/9999/TJ48mR07dlC2bNnia5wgCEIxER09QVDDuyyjsn79erZt20Z4eHiez7GJjt7bURZ/JkkSzs7OuLi4YGdnV8wtFARBKB7i1q0gFEBLS4ulS5dy9OhRunbtyrx583BxceHevXtIksSMGTOYOnUqkLUciLJYtaFDh3L//n2cnZ2xtrZm//79uc4THh6Or68vGRkZnD59msGDBxMTE0P16tVZtGgRAHfv3sXe3p4HDx6wfPlyrl+/TlxcHFFRUTRo0AB/f38qVapEWloan332GSdPniQ1NZWGDRvi5+dHxYoV871OZdFuyhRHBJq68Wc+Pj4kJiYyd+5c0tLSFKKOijvuqKTELmkKUS/ViVqpp6TUS0SgCcJ7pqOjI8eMjRgxgkaNGnHw4EFiYmJo2bIlzZo1o0WLFgwePFhprNr+/fuxsLDINx7N1tYWFxcX4uPjWb9+PZDVsevVqxcLFixAR0eH77//nilTpqCrm/W/7rlz57h27Rrm5ua4urri4eHBpk2bWLNmDeXKlePSpUsALFu2jCVLliid2JCTsmi37HPlVBwRaOrGn+3Zs4fLly9Trpxix7Nt27Z06dKFmTNnvnM735Wmxy5pGlEv1YlaqUfT6yUi0AThPZswYYL89+DgYK5cuQKAmZkZgwcPJjg4mHLlyuUZq9axY8e3Om+jRo1o0qQJv/zyC7169eKnn37i+vXr8vt9+vTB3NwcgClTpjB48GAAAgICSEhI4MCBAwCkpqZiYWFR4PlUjXYrjgg0dePPmjZtqhAb9OTJE/r06cPu3btp3br1O0XWvauSErukKUS9VCdqpZ6SUi8RgSYI71n2QsrKZEenqfueqmbOnMmqVat4+vQpPXr0oGrVqgWeT5IkNmzYQM+ePdU6l6rRbkURgfau8WdvzmzOvm3dqFEj6tSpUyhtfFeaHrukaUS9VCdqpR5Nr5eIQBOEImRvb8/mzZsBePr0KQcPHqRHjx40atQoz1g1VSmLUOvZsyfR0dEsX74cNzc3hfeOHj3Kf//9B8CWLVuwt7cHYODAgXh7e8vD/SkpKdy8efOtrre4FFb8mSAIwodMjOgJgpq+/fZbpk2bhrW1NZIksXjxYtq0aQOQb6yaKgYNGsSOHTuwtbVl8ODBLF26FC0tLSZOnMju3btp166dwvadOnVi5MiRPH78WJ6MAbBgwQI5DSJ7lG/BggVYWVkVThGKQGHFn2WzsLBQex9BEISSTnT0BKEAb3YOqlatqnQhZAA7O7s8R5QiIyMLPFedOnW4evVqrtdPnTrFrFmzcr1es2ZNpTN4dXV18fLywsvLq8BzZnvzOp89e6byvu9DYUSgPX/+nBkzZvC///0PbW1thgwZwjfffKNW51sQBKEkE7duBUGDXb58mfr166Otrc3IkSOLuzlFqjAi0EaNGsXNmzcJCgriyJEjnD17lilTphTVJQiCIBQ7MaInCEUsPDwcZ2fnXK+PGzeOTz/9VOE1Ozs7hQkJOXl6eqp9bhcXF0JDQ3O9HhISQpkyZdQ+3vv0rhFot2/fJjAwkLCwMHnB5A0bNuDo6MjatWtF9q0gCB+EUtXR09LSIi4uTiEC6UNw+vRpXr16hYODQ6EeNzIyksDAQFxcXAr1uMXBzs6OtWvX0rVr1yI/9/r163FycpKXQbG1tSU8PPydjunv709AQAABAQEqbT9p0iRGjRqFr69vntt4enoqrOFX0rwZgRYSEkKFChUUUjHs7e3R1tbm4sWLDBo0qJhaKgiCUHTErdsSIr/orYJis942tisyMjLfjkF+3iUqrLTIzMwkMzOT9evXEx0drfb+hVXDjIwMtmzZQrdu3QrleJpIWQRadHQ0ZmZmCtvp6upiamr6Vt8PQRCEkqhUjegBfP/990ojnObOncuZM2dIS0vD2NiYzZs306hRI16+fImzszPXr19HT0+PqlWrcvz48TyPv2XLFtatW4e+vr78C7RNmzZYWFgQEBAgL6WRcwSpa9euWFtbExoaSlxcHAMGDGDt2rVoaWkRHR2Nu7s7kZGRvHz5kgEDBrB8+XIga5bgiBEjOHXqFA0aNMDT0xNnZ2eSk5PJzMxkwIABDB06NFds1tixY7G1tWXq1KkEBQUxduxY4uPjFUZrfHx8uHz5sjxLc9WqVezcuRNtbW3KlCnDyZMncXFx4Z9//sHW1pZatWpx+PDhAq/TxsaGsLAw+Rhr167l559/Jj09HTMzM/z8/Khdu3ae9bWwsGD06NGcPHmShw8fsnjxYgwMDNi0aRNPnjxh5cqVODk5AblHcCtXrszly5exsLDgwoULuLq6kp6eTqtWrQrsNEVGRmJra8uMGTP49ddfSUpKwt/fn/3793Pq1CnS09PZs2cPTZs2BWDNmjX4+/ujra2NjY0N33//PSYmJnh6enL9+nWSk5N5+PAhI0aMICoqihEjRlCmTBn8/f2JjY1lyZIlvHr1itTUVGbPns3EiRMBcHZ2Rltbm4iICGJiYrhz506ebU5MTKR///5ERERQuXJltm/fjoWFBf7+/vz444+Ympry119/sWnTJjw8PJg1axYDBw4kISGBOXPmEBoaio6ODi1btuSHH35QOPatW7cYPnw4a9asUXr79H1GoCmLPwP1I9AyMjKQJEnpPhkZGcUaeVRSYpc0haiX6kSt1FNS6qUREWjx8fEaccs0rwinBQsWsHbtWiArGmnmzJkEBgYSGBhIfHw8t27dArJm6eVnzpw53Llzh2rVqpGWlqbwyy4/t27d4sKFC6SlpdG5c2d++uknRo4cybhx41i0aBFdunQhPT2dvn37sm/fPoYNGwZAbGwsFy9eREtLi5kzZ9K3b188PDzktpqamuaKzYqMjCQhIQErKytWrVoF5P88148//siBAwc4f/48JiYmxMXFYWBggK+vL7NmzVLrNuNff/3F2bNn0dPTY/fu3dy9e5eQkBB0dHTYsWMHrq6u/Prrr/ke48WLF1y4cIGIiAisra1ZvHgxISEhhIWF4ejoKHf08pKamsqIESPYtm0b9vb2HD9+XO7Q5ichIYGWLVuybNkytm7dSq9evfjf//6Ht7c3a9as4YsvvmDfvn389ttv/PDDD/KtwSlTprBw4UI2btwIZN0yvHr1qryw8bZt2xSiz+Li4jh//jw6Ojo8f/6c5s2b06tXLzmp4cqVK5w/f57y5cvn297ff/+d8PBwGjduzOrVq5kyZYr8j5SLFy9y9epVGjVqlGu/WbNmUaZMGa5du4a2tjZPnz5VeP/06dO4urqyc+dOWrRoofTc7zMCTVn8GagfgRYTE0NUVJTC8TIyMoiNjeXx48d5nqcoaXrskqYR9VKdqJV6NL1eRR6BtmrVKnm0CWD48OEcOHAAc3Nzjh49SrNmzd66Qe8qrwinoKAgNmzYQFJSEpmZmXKHrlmzZty+fRtXV1e6dOmiELGkTPfu3RkzZgz9+vWjd+/eNGzYUKV2jR07Vl55e/To0QQHBzNgwABOnDghL3gLWWkAd+/elb92dnaW10Hr3Lkz8+bNIzk5mS5dusiL4yqTfR5VHDlyBBcXF0xMTADyDb4vyOjRo+VfxgEBAYSFhdGyZUsg65esKrI/V/Xr18fQ0JChQ4cCWaOHz58/L/AfFXfu3EFXV1euT8+ePalbt26B5zU0NJSX77Czs8PIyEi+3dm6dWt27doFZEWgjRgxQm7DtGnT5I45gKOjY77pFbGxsUycOJG//voLXV1dYmNjuXHjhtzRGzZsWIGdPID27dvTuHFjICv+bMmSJXKN27dvr7STB1nf74sXL6KtnfXkRs7osJMnTxIYGMjx48epVatWnucuCRFoderUwcfHB3Nzc7nDGhQUhCRJuLi4FOtkjJISu6QpRL1UJ2qlnpJSryKPQPP19ZV/6QUFBREUFMRvv/3Gzz//zLx58/K99fm+KYtw+vfff3FzcyMsLIx69epx7do1OnfuDEDdunW5desWJ0+eJDg4mPnz5xMeHp5nZ+fAgQNcuXKF06dP4+joyPLly3FyckJXV1ehI/Pq1at826mlpSWvWxYaGqrQ7pxyrvc1ZMgQ2rdvT1BQED4+Pqxfvz7PUYmyZcvKv8gBtduXl4KOk7O9kiTh4eGh9nIWb34Ps7/W0tJCS0tLvg2ro6Oj8jWpEkWWM9Ir53mzv87r9u+bxy5ojTYXFxccHR05cOAAWlpatGjRQqHthbHG29seo379+ty5c4fQ0NB8O3olIQLNxsYGBwcHpk2bhq+vL2lpacyaNQsnJ6d8Hx8oSpoeu6RpRL1UJ2qlHk2vV5FHoEVHR/PRRx8BWaMDw4cPp2fPnsyfP5+wsLC3bsz7kpCQgJ6eHtWqVUOSJHx8fOT3Hj16hJaWFv3792ft2rVIksTDhw+VHic9PZ379+9jZ2fH3LlzGTp0KJcuXQKyfkFevHgRgEuXLimMygHs3LmTtLQ0Xr58ye7du7G3t5dHjFauXClvFxUVxaNHj5Se/969e1StWpWxY8eyevVqeZkMZbFZb6pfvz6XL18mIyODlJQUOeweoH///vj6+srHiI+PJyMjQ+lxC7rOnAYOHIivr688epqWlqZ0MeC3lbMtBw8e5MWLF0DWaG56ejqnTp0Cskbg7t+/X2jntbe35+eff5b/heXn55dvpuybdYyLi6N27dpoaWlx9uxZ/vzzz7dqR0hIiPwMX/ZkCx0dnQL3y/6sZ2ZmAijcuq1VqxYnTpxg+fLlbNu27a3aVVgKIwJt165dWFpa0r17dxwdHenYsSObNm0qrksSBEEocm81olexYkUePnzIRx99RGBgoDx5QJIklW/PFSVra2ucnJywsrKiUqVK8u05gOvXr+Ph4YEkSaSnpzNmzBhsbGyUHicjI4MJEybw/PlzdHV1qVKlivzLcPny5YwbNw4/Pz/atWuXK2qqcePGdOjQgefPnzNgwAD5ObNdu3Yxe/ZsmjZtipaWFuXKlcPPz0++jZfT/v372blzJ/r6+mRmZsozYt+MzRo7dmyufQcPHsy+ffto3LgxNWvWpHnz5vI9/zFjxhAVFUX79u3R1dWlXLlyBAcHY2Njg5WVFU2bNqVu3bocPny4wOvMadSoUcTGxsq3P9PT05kwYYL8i/tdeXt74+7uzpIlS+jTp498y1BfX5+9e/fi6upKRkYGrVq1KtTHCXr37s2NGzdo166dwmSMvLi7uzN58mTKli2Lv78/K1euxNXVlWXLlmFrayvHp6mrffv2LFiwgIiICCpVqsT27dtV2s/b25tPP/0Ua2tr9PT0aNWqlZzdC1CtWjVOnjyJg4MDSUlJuLu7v1X73lVhRKCZmpqye/fuwmyWIAhCySK9henTp0u1a9eW7O3tpUqVKklJSUmSJEnSTz/9JDVv3vxtDlmqdenSRTp06FBxN0MQ3puEhAQJkJ49e1Yoxztz5ozUt29fqVq1ahKQ6/+fAwcOSD169JBMTU0lQLp69arC+7GxsZKbm5vUsGFDydDQUProo4+kGTNmSPHx8YXSvneVmpoqBQQESKmpqcXdlBJB1Et1olbqKSn1yv4Zm5CQoPa+b3Xr1tvbGzc3N5o0aUJQUJD8PNCTJ09wdXUttE6oIAgfpvziz7Lf79ixozyr/E1RUVFERUWxdu1abty4gb+/P4GBgfIyNoIgCB+Kt7p1q6enx9y5c3O9/mZ8U0l19OhRFi1alOt1Dw8PeUaoOk6fPl0IrSo9vLy8OHjwYK7XDxw4QL169d7rufv378+///6r8FrFihXl5/k0SUxMjNJn/3r06MGaNWuKoUVFJ7/4M8h63ACylhJSpmnTpgrPodarV48vv/yS0aNHk56ejq5uqVtCVBAEQam3/mm3Y8cO/Pz8+PvvvwkJCaF27dqsX7+eOnXqMGDAgMJsY5FzdHQscJkV4e0tXbqUpUuXFsu5Dx8+nGuxaGXu37/PsGHDkCQJd3d3fvzxR3nB4fctZ2Tau0alCf8nISEBY2Nj0ckTBOGD8la3bjdu3Mjs2bPp3bu3PEMToEKFCiU2J1MofpoUm7Z//35atWrF1atX5XSVwpTftb5tZJqQt2fPnrFs2TK1l/oRBEEo6d7qn7YbNmxg8+bNDBw4UGFpkOxlRwThTb/88gsLFy5EX18fBwcHtm7dyuXLl+natatCzJufnx/u7u7ysjXDhg3j888/B7JmYeYcVRs6dCh9+/bF2dkZZ2dnDAwMiIiI4OHDhzRt2pQ9e/agr69PUlISkyZNIjw8nCpVquQ7Uxhg+/bteHt7k5GRwcWLF3PN2oyJicHFxYV79+4hSRIzZsxg6tSpQNaSIO7u7iQnJ2NoaIi3tzcdOnSQI9ZyxtLNmjUr17m9vLwUItN8fX0ZOHAgYWFh8pJGixYtIiMjQ164fNiwYZw8eZKEhASmTp3KvHnzgKzleGbNmkVMTAyvX79mypQpuLm55XndX375JRcvXuSXX34hNTWV9u3bM3v2bHkR8pzeVwTa28SfZf83r4igxMREHB0dady4MYsXL9aIqKOSErukKUS9VCdqpZ6SUq8ij0B78OCB0iUyDAwM5LXMBCFbTEwMEyZM4Pfff8fS0pJt27bJi9uCYszbggULeP36NdeuXePly5d07NgRS0tLlZ6NDA8P59SpUxgYGNC5c2cOHDjAJ598gpeXFwYGBty5c4fExETatm2b75ImY8eO5e+//1aIlctpxowZNGrUiIMHDxITE0PLli1p1qwZLVq0YPDgwWzevJlevXpx/vx5hgwZIi/6+2YsnTJLly7lhx9+UIhMmzhxIhs3bmTFihW8fv2abdu2yWsoAvz3339cvnyZ2NhYWrRoQYcOHWjTpg2ffPIJO3fuxNLSkpSUFPm6W7VqpfTcixYtwtHRka+//poHDx5gZ2entJMH7y8CTZ34M0BOlTl//jxRUVG53n/58iWenp4YGBgwceJEjYs50rT2aDpRL9WJWqlH0+tV5BFoderUITw8PNfq8oGBgXIkkyBkCw0NxcbGBktLSwDGjRuHi4uL/H7OmLfg4GC+/vprtLW1KVeuHGPHjiUoKEiljt6gQYMoW7YskBVZlr1I8okTJ/D29kZLSwsTExNGjhz5TgsoBwcHc+XKFQDMzMwYPHgwwcHBlCtXDm1tbXr1yhqV6tixI1WrViU8PJyaNWuqFUuXk6urK61bt+bzzz9n3759tG7dWuH/vYkTJ6KlpUXlypXltlSoUIGbN28q5AInJSVx69atPDt6Wlpa7Ny5k+bNm1OxYkV5MWplijoC7c34s2zZkzE6duwod4xztqlPnz5UrVqVw4cPy58NTVBSYpc0haiX6kSt1FNS6lXkEWizZ89m+vTpvHr1CkmSuHTpEj/99BNfffUVW7ZseevGCB+m/OK6csaLFRS/9raRZe8qv+PlfO/NWDpV1ahRg86dO7N37142btyIl5dXge2RJAlTU1O1J3P8888/ZGZmkpSUxIsXL/KM5nvfEWj5xZ/VqlWL58+f8++//8qjeH///Td6enqYm5tjbm4ud/JSUlLYtWsXL1++5OXLl0BWtq8qCSJFQdNjlzSNqJfqRK3Uo+n1KvIItEmTJrFq1SqWLFlCSkoKI0eOZOPGjXzzzTcKIwiCANC2bVuuXbsmx6Xt3LmT1NRUpdva29uzdetWJEnixYsX7NixQ15iJGfk2YMHDzh//rxK57e3t2fbtm1IkkRiYiI//fTTO12Pvb29nCTx9OlTDh48SI8ePWjUqBGZmZnyLYALFy4QHR2da6SpIMqi52bOnMnixYuJj4/H3t5e4b3s2cPPnz/n0KFDdO/enUaNGmFsbKwQYxYRESHH0SmTmJiIk5MTO3bswMXFhbFjx6qUPvE+5Bd/Blmzp5s3b06fPn0AcHJyonnz5nJazB9//MHFixe5fv069evXV4hIyyviUBAEoTRSe0QvPT2d3bt306tXL0aNGkVKSgrJycmYmZm9j/YJpYCZmRlbtmxh4MCBGBgY0KNHD4yMjKhQoUKubT/77DPc3d2xtrYGsiZjDB8+HID58+czYsQIrK2tsbKyUjk67LPPPmPSpElYWlpSpUoVOnbsqDCRQF3ffvst06ZNw9raGkmSWLx4sdyWgwcP4u7uzpw5czA0NGT//v0YGRnx7NkzlY//ZmSara0tbdu2xcTEhKlTp+YaQaxSpQotW7YkISEBNzc32rdvD2TlUM+aNUueWFK5cuV848AmTpzIqFGj6NatG127duXs2bOsXr2aBQsWvEWV3k1B8WfZE3Dedn9BEIQPxttEcZQpU0aKjIx8m12FD1RiYqL890OHDkmWlpbF2JqS59GjR5K5ublCHSVJkmrXrp0r/qs4iAg09ZSU2CVNIeqlOlEr9ZSUehV5BFrr1q25evVqYfY3hVJuw4YNNGvWjKZNm7J69Wp27dpV3E0qMZYuXUqbNm1YuXIl5cuXL+7mFAkRgSYIglA43moyhqurK3PmzOHRo0e0bNmScuUU182ysbEplMYJpceiRYuUxsoVpy1btuDj45Pr9Q0bNtCpU6f3em51YuC8vLzynICRVwRYXkpKrJqIQBMEQSgcb/XTLnvChbu7u/xa9kw/LS0thZmR75OWlhZxcXFKn/UqzU6fPs2rV69wcHAo1ONGRkYSGBiosPRJaTZp0iQmTZpULOfOjoGztbXl3LlzRTZSZ2ZmpvJM3JwLUpcGIgJNEIQP0VsvmCy8X/mNOpw+fZr4+Pg8O3pvO2IRGRmJr6/vW3X0CmOU5EMcaSnKLNsPsb7ZRASaIAgfqrf6qf/mQsnF6fvvvycgIICnT5+ydOlSOZd07ty5nDlzhrS0NIyNjdm8eTONGjXi5cuXODs7c/36dfT09KhatSrHjx/P8/hbtmxh3bp16Ovrk5GRwZYtW2jTpg0WFhYEBATIS2fY2dmxdu1aunbtSteuXbG2tiY0NJS4uDgGDBjA2rVr0dLSIjo6Gnd3dyIjI3n58iUDBgxg+fLlAFhYWCjEgXl6euLs7ExycjKZmZkMGDCAoUOH4uvrS0ZGBqdPn2bw4MGMHTs2V7xWfHy8QrKDj48Ply9flpfiWLVqFTt37kRbW5syZcpw8uRJXFxc+Oeff7C1taVWrVocPny4wOu0sbEhLCxMPsbatWv5+eefSU9Px8zMDD8/v3w/L29e87Zt21i8eDG//fYbOjo6VKtWjcDAwDz379q1K3Z2dly8eJGoqCh69OghL7GRX1SZhYWFvBhzdHQ0EydOZMmSJfl+1o4dO8ayZct4+fIlOjo6rFq1im7dunH69Gnc3Nzo3Lkzv//+O+np6fz444/Y2dkB4Ofnx9dff42RkRGDBg1i6dKl8ozQnKPS+bUpv89NflFnWlpaLF26lKNHj9K1a1eWLl3K7Nmz+fPPP3n16hVt27bFx8cHfX197ty5w4QJE0hISKBBgwb5rsQuItDeTUmJXdIUol6qE7VST0mpV5FHoG3fvj3f98eOHftWjXkbBgYGXLp0iTt37tCqVSvGjBmDrq4uCxYsYO3atQDs2bOHmTNnEhgYSGBgIPHx8dy6dQsg33XFAObMmcOdO3eoVq0aaWlpKi/LcevWLS5cuEBaWhqdO3fmp59+YuTIkYwbN45FixbRpUsX0tPT6du3L/v27WPYsGGAYhzYzJkz6du3Lx4eHnJbTU1NcXFxUejERUZG5orX8vT0zLNtP/74IwcOHOD8+fOYmJgQFxeHgYEBvr6+zJo1S61Rpr/++ouzZ8+ip6fH7t27uXv3LiEhIejo6LBjxw5cXV359ddf8z1Gzmv+4osv+Ouvv7hy5QoGBgY8ffq0wDbcv3+fU6dOkZaWRpMmTQgJCaFdu3Z5RpW1bdsWgPj4eEJCQnj27Bn16tVj/Pjx1KhRQ+k5/v77bzw9PTl27BjGxsZERETQqVMn+RmxO3fusHXrVr7//nt8fX1ZvHgxx44d48aNG3h6enL16lXMzc3l3N685NWmvD43gwcPLjDqTEdHh7CwMACmTJlCp06d2Lx5M5IkMXnyZL755hvmzZvHmDFjcHFxYeLEiVy/fh07OztGjhyptJ0iAq1waFp7NJ2ol+pErdSj6fUq8gi0mTNnKnydlpZGSkoK+vr6lC1btkg7etlZnJaWlujq6hIdHU3NmjUJCgpiw4YNJCUlkZmZKXfomjVrxu3bt3F1daVLly5KY5Vy6t69O2PGjKFfv3707t2bhg0bqtSusWPHyittjx49muDgYAYMGMCJEyfkX06QlQCQvZAwKMaBde7cmXnz5pGcnEyXLl1yLZSbkzrxWkeOHMHFxQUTExMAKlasqNJ+yowePVr+BRwQEEBYWBgtW7YEUPlZzZzXfOTIEVatWiWnLlSpUqXA/UeMGIGuri66urrY2tpy//592rVrl2dUWXZHL7sTU7lyZerWrcuDBw/y7OgFBgYSERFB586d5de0tbX5999/gazFnLPX0mvXrp38j4yTJ0/i4OCAubk5AJMnT8432UJZmypUqJDn5+bu3bsFRp1NmDBBfi8gIICQkBDWrVsHII9OJiYmEh4eLj+PZ21tTceOHfNsp4hAezclJXZJU4h6qU7USj0lpV5FHoEWFxeX67V79+4xbdo05s2b99aNeRvKYq/+/fdf3NzcCAsLo169ely7dk3+BV23bl1u3brFyZMnCQ4OZv78+YSHh+fZ2Tlw4ABXrlzh9OnTODo6snz5cpycnAqM43pT9mQVyMp+zStaKmcc2JAhQ2jfvj1BQUH4+Piwfv36PEc83ozXUrd9eSnoODnbK0kSHh4eaj8HlV8EmireNvpM1f0g69p69OihdMHhx48fF1r8mrLj5Pe5yY4Fy28U9s3v0YEDB3L9g0XZD5H82ioi0AqHpscuaRpRL9WJWqlH0+tV5BFoyjRo0ICVK1fmGu0rDgkJCejp6VGtWjUkSVJYQuPRo0doaWnRv39/1q5diyRJeUYipaenc//+fezs7Jg7dy5Dhw7l0qVLgGIc16VLlxRG5SAr5istLY2XL1+ye/du7O3tMTIyolu3bqxcuVLeLioqikePHik9/71796hatSpjx45l9erVhIaGAsojst5Uv359Ll++TEZGBikpKQpLTfTv3x9fX1/5GPHx8WRkZCg9bkHXmdPAgQPx9fWVR0/T0tLUXm+xf//+fPPNN/ItclVu3eYlr6iyt9GrVy+Cg4O5du2a/Fr2ZyE/3bp149ixY8TExACwdetWtc+d3+dG3aizgQMHsmrVKrkjGhcXR0REBMbGxjRv3lx+LOPmzZsqR8y9DyICTRAEoXAU6hQ8XV1dpc/JFDVra2ucnJywsrKiUqVKDBw4UH7v+vXreHh4IEkS6enpjBkzJs91/zIyMpgwYQLPnz9HV1eXKlWqyL9Qly9fzrhx4/Dz86Ndu3ZYWVkp7Nu4cWM6dOjA8+fPGTBggHxrbdeuXcyePZumTZuipaVFuXLl8PPzo2bNmrnOv3//fnbu3Im+vj6ZmZnyL7FBgwaxY8cObG1t5ckYbxo8eDD79u2jcePG1KxZk+bNm8v3+MeMGUNUVBTt27dHV1eXcuXKERwcjI2NDVZWVjRt2pS6dety+PDhAq8zp1GjRhEbG0u3bt2ArI7yhAkT5F/WqliwYAGLFy+mRYsW6OnpUb169TxHMQuSX1SZuurXr8/u3buZOnUqKSkppKam0rx583wjxSDrs7hkyRI6dOhA+fLlcXBwkG+ZqyO/z406UWfe3t4sXLgQW1tbtLW10dXVZfXq1dSvX5/t27czfvx4vv76axo0aKBwm7qoiQg0QRCEwvH/2Lv3uJzv/4/jj6tSIcKcM8NsSkUsJITJYc3kzByS45Kt2hySQxLbHDLnKWyYsI2IYSg0TCjT5JRj+zqloVKJTp/fH936/Lp0XXVdia7sfb/d3G66rs/1eb8/r8jb+/N5v58KqQQ/Dffs2aP0tSRJ3L9/n1WrVvH222/z+++/l1oHy6MuXbrg5eWlNMAU/rtSU1PlffKWL1/OgQMH3ri/I0+ePMHU1JSHDx++9DN6x44dY/HixZw9e5b79++za9cupb9LO3fuJDAwkLNnz/L48WPOnTtX6Pm8tWvXsnXrVv766y9SU1N1br/NrKws9u/fj5OTk07fLtIVol6aE7XSTnmpV/7P2Pz9QLVRohm9FwcwCoWCWrVq8eGHH7JkyZKSnFIQ3ljTp0/nzz//JCsri/r16xMUFFTWXdJp+fFnY8aMoX///irf79ixI4MHD2b8+PEqz/H06VN69epFr1695FXrgiAI/0UlGujl5uaWdj/K1P79+1XGc/n4+DBkyBCtzxcREVEKvXpzaBP3pcrrjCrr06ePvJI2X/Xq1Tl69GiJz6kur1VQ7WXjzwC8vLwA8XdREAShRAM9f39/pkyZUmi7goyMDBYvXiw/MF1eODk5FbvNilBy+XFfBSkUCo1v8b3OqLIXH0sobTVr1iQ6OppGjRq90nYEQRAEAUq46nbu3LmkpaUVev3p06cqN1EVBKHkitr2RRAEQRCKUqIZPUmSVO6x9ffff1OjRo2X7pTw31BW8XVFRYZFRkYydepUUlNTkSSJefPm4ezsTHR0NB4eHqSlpWFsbMzSpUvp0KED8fHx2NjY4Onpyd69e0lJSWHFihXyDPGePXvw9vamQoUKhbKJ1V0nFI4uO3DgAEFBQdjb2wN5iw0OHz7ML7/8ovLa8/uVnJwM5O1LV6VKFSRJKrKOmzdvZtWqVWRlZWFiYsLKlStp2bJlofO/zgi0l4k/yx8kF3VMWSgvsUu6QtRLc6JW2ikv9XptEWjVq1dHoVCgUCh4//33lQZ7OTk5pKWl4ebmVuLOCP8tZRFfl5OTozYy7N1336Vv377s2LGDTp06kZubS3JyMpmZmfTv359169bRs2dPTpw4wYABA+QNfVNSUmjRogVz587lwIEDeHp64uTkRGJiIqNHj+b48eM0b96ctWvX8ujRI7l/6q4zX8Hosvfff59Vq1bJA73Vq1erfG5RE+rq+Oeff7Jt2zaOHTuGkZERx48fZ9iwYVy8eLHQOV5nBFpJ488gbzslgEOHDr30xtyvgq7HLukaUS/NiVppR9fr9doi0JYtW4YkSYwZM4a5c+cq7QdmaGhIo0aNaN++fYk7I/y3lEV83aVLl9RGhiUmJtKsWTN5gYeenh41atQgNjYWPT09evbMm23q2LEjderUISYmhgYNGmBsbCyvDm3fvj03btwA8pIsWrRoQfPmzQEYO3YsX3zxhdyuuuvMVzC6bMSIEfj6+vLgwQOuXbuGQqEo8UIUdXXcvXs3f//9t9Jeg48fPyYjI4OKFSsqneN1RqCVJP4sX+XKebOLPXr00LntVcpD7JKuEPXSnKiVdspLvV5bBNqoUaMAaNy4Mfb29jpdFEH3lUV8nbW1tdrIsH379mnc94Kz2UZGRvLX+vr6ajN+C36mqOvMV3AGqmLFiri6uhIUFMTly5eZNGlSkf0rKrpOXR0lSWLUqFF88803xV7/q4xAe9n4M4CEhAQSEhLkweCVK1eoUqUKDRs21KnHS3Q9dknXiHppTtRKO7per5fqm/SSMjIypJSUFKVfglAcQEpKSpK/fuutt6Rbt25J58+fl2rXri2lp6dLubm50vjx4yVTU1NJkiTp9u3bUlpamiRJkvT8+XPp7bfflv7++2+V58/KypKuXbsmf+3t7S19+eWXUlZWlmRubi79+OOP8nvXrl2THj16JD1+/FiqW7eudOzYMUmSJCknJ0d69OiR3NahQ4ckSZKkP//8U6pTp46Umpoq3bp1S+6fJElSamqqlP/XKjExUapRo4Z0+fJlSZIkad26dRJQ7HWqqk/+9ZuZmcmfK0pWVpZUuXJl6eLFi5IkSdKSJUvkfqmr47Fjx6S3335b+ueff+Trj4qKKrKdfCkpKRIgPXz4UKPji3L06FEJKPRr1KhRkiRJ0oYNG1S+P2fOHPkcc+bMUXnMhg0bXrp/pSEzM1MKDQ2VMjMzy7or5YKol+ZErbRTXuqV/zO2JGOsEi3GePr0KdOmTePXX39VeuYon7oZDUEozquOrzMwMFAbGWZmZsauXbuYPHkyqamp6OnpMW/ePD755BN27tyJh4cHkydPxtjYmB07dmBiYsLDhw/VXkutWrX48ccf6devH4aGhvTq1Uu+rVnUdaqTH2X3/vvvF9ra6EUGBgasXLmS3r1789ZbbzFw4ECN6rho0SL69etHdnY2mZmZfPzxx9ja2hbbt9L0svFnAH5+fvj5+ZVuxwRBEMqjkows3d3dJQsLC2nHjh1SxYoVpR9//FGaN2+e1KBBAyk4OLgkpxQEoRhpaWmSmZmZdPPmzbLuSiGlOaP3xx9/SL1795bq1asnAdKuXbuU3g8JCZG6d+8u1ahRQwKkc+fOFTpHRkaG5O7uLtWoUUOqXLmy1L9/fykhIeGl+1Zaysssgq4Q9dKcqJV2yku9XmZGr0T76P322298//33DBgwAAMDAzp16sSsWbP45ptv2LJlS2mOQwVBAAIDAzE3N8fd3Z3GjRuXdXdeqfwINHWJIvkRaAsXLlR7ji+//JLffvuN7du388cff3Dv3j2VcWqCIAhvuhLdun38+DFNmjQBoGrVqvJqwY4dOzJx4sTS650gFKO04+t0lZubm8qti15FZFtZe9kItJSUFH744Qe2bt3Khx9+CMCGDRuwsLDg1KlT2NnZlXqfBUEQdFWJBnpNmjTh1q1bNGzYEHNzc3799Vfatm3Lb7/9plNbGLyJFAoFSUlJ/7k6R0RE8OzZs0KbDr9sfF18fDwHDhx4rfs/NmrUiNDQULVbgmhDVWRbfHw81apVkzdL/q85e/YsWVlZODo6yq+Zm5vTsGFDIiMjxUBPEIT/lBLduh09ejR///03ANOnT2f16tUYGxvz5ZdfMnXq1FLtoPDfUVTUV0REhNJmwtp8tijx8fEEBgaW6LMimkw3JSQkYGhoWOg/Q3Xq1CEhIaFsOiUIglBGSjSj9+WXX8q/d3R05MqVK5w9e5amTZuqXQUplJ6yig57cSbK1taWgIAAunTpQpcuXbC2tubUqVMkJSXh7OxMQEAACoWChIQEPDw8iI+PJyMjA2dnZ+bPnw/kzW4NGTKEo0eP8t577+Hn54erqytpaWnk5ubi7OzMwIEDCQwMJCcnh4iICPr374+Liws2NjZ89tlnhIWF4eLiQnJyMsnJySxbtgyAVatWER0dzcaNGwFYuHAhwcHB6OnpUbFiRY4cOYKbmxv//PMPNjY2NGzYkD179hR7nS1atCAqKko+R0BAAL/++ivZ2dnUrl2boKAg3nnnnSK/h1u2bGHs2LGkpKTw2Wefyf9BKi4Wbf78+ezZs4cHDx6wbNkyLl++TEhICCkpKaxbt44uXbrIbUyZMoVDhw6Rk5PD8uXL5RmuzZs3s3jxYgDefvtt1q5di5mZGTk5OUyfPp3ff/8dgK5du7JkyRIMDQ0L9V+XI9AKxp4VJEkSOTk5OhF1VF5il3SFqJfmRK20U17q9doi0FR59uwZ77zzTrH/sAmlpyyiwzRx6dIlTp48SVZWFg4ODmzbto1hw4YxatQoZsyYQefOncnOzqZ3795s376dQYMGAfDo0SNOnz6NQqHA09OT3r174+PjI/e1Ro0auLm5KQ3i4uPjSUlJwdLSUn4ov6jtNDZt2kRISAgnTpzA1NSUpKQkjIyMCAwMxMvLS+UGyupcvXqVY8eOUaFCBbZu3UpcXByRkZHo6+uzefNm3N3di918+cGDB0RHR/Po0SNat25Nhw4dsLe3LzYWzcTEhNOnT3P48GGcnZ3lwez27duZOnWqHJmWkpKChYUFAQEBnDp1ij59+nDjxg3++ecfpk6dytmzZzEzM+Prr79m3Lhx/P7776xdu5aoqCjOnj2Lvr4+ffr0YenSpXh7exfqvy5HoP3zzz9kZmby66+/Km06/c8//5CUlKSyjbKi67FLukbUS3OiVtrR9Xq9tgi0fDk5OXzzzTcEBgby4MEDrl69SpMmTZg9ezaNGjVi7NixJe6QULyyiA7ThIuLi7y7+IgRIwgPD8fZ2ZnDhw/L/yhDXvJBXFyc/LWrq6ucGuHg4MDUqVNJS0ujc+fOSs9ZvSi/HU3s3bsXNzc3ObZPXZqGJkaMGCEPPEJDQ4mKiuKDDz4ANN9DcuzYsSgUCmrWrEn//v0JDw/H3t6+2Fi0/AUmtra2pKeny1Fubdu25dq1a/JxBgYG8l5zdnZ21K9fn3PnzvH333/Tq1cvzMzMAHB3d8ff35+cnBzCw8NxdXWVEy/Gjx/P6tWrVQ70dDkCrUOHDsybNw8DAwP5c3Fxcfz777+MHj1aKeKtrJSX2CVdIeqlOVEr7ZSXer22CLR8X3/9NZs2bWLRokWMHz9eft3Kyoply5aJgd4rVhbRYUOHDi0yVksVhUIhb3x76tQppX4XVHDWZcCAAfKAZ9WqVSxbtkztDEylSpXQ0/v/x0y17Z86xZ2nYH8lScLHx4cJEyaUqK18CoVCo1i0/Brq6+sX+rq4ZwYLRrAV9Zom7+lyBFrNmjUZO3Ys06ZNo3bt2lStWpUvvviC9u3b07Fjx5fqW2nT9dglXSPqpTlRK+3oer1epm8lWozx008/sXbtWoYPHy7/gwN5M0dXrlwpcWeEkktJSaFChQrUq1cPSZJYtWqV/N6dO3dQKBT06dOHgIAAJEni9u3bKs+TnZ3NjRs3sLW1ZcqUKQwcOJAzZ84A0LRpU06fPg3AmTNnlGblAIKDg8nKyiIjI4OtW7fi6OiIiYkJXbt2ZcGCBfJx9+7d486dOyrbv3btGnXq1MHFxYVFixZx6tQpIG8bn5SUlCJr0LRpU6Kjo8nJyeHp06eEhITI7/Xp04fAwED5HMnJyeTk5Kg8b3HXWVDfvn0JDAyUZ96ysrI4d+5ckf0E5OcGHz9+zK5du+jWrVuR30NtZWdns3nzZvka7t27h42NDV27duXAgQPyICkwMJBu3bqhr6+Po6MjP/30E5mZmWRnZ7N+/Xp69OhR4j6UVHR0NK1ataJVq1YAfPXVV7Rq1QpfX18gb6Vxq1at+PjjjwEYOnQorVq1UlpUs3TpUnr37s2AAQNwcHCgbt267Ny587VfiyAIQlkr0Yze3bt3adq0aaHXc3Nzdf6BxjfVq44OA5g/fz6jRo0iKCiI9u3bY2lpqfRZCwsLOnTowOPHj3F2dpZvK27ZsoWvvvoKKysrFAoFlStXJigoiAYNGhRqf8eOHQQHB2NoaEhubq78j3e/fv3YvHkzNjY28mKMF/Xv35/t27djYWEhx4XlP9cwcuRI7t27h729PQYGBlSuXJnw8HBatGiBpaUlVlZWNGnShD179hR7nQUNHz6cR48e0bVrVyBvgDVmzBh5kKJOrVq1+OCDD0hJSeHzzz/H3t4eQOtYNHVMTU25cOECLVu2JDs7m61bt1KlShWsrKxYvHixvE3N22+/zbp16wCYMGECN27coHXr1kBeFJmXl1eJ+1BSpRGBZmxszOrVq9VuuiwIgvCfUZIojtatW0ubN2+WJEmSTExMpBs3bkiSJElz586VOnbsWJJTCuVc586dC0VVCf8dpRWBVlz8WW5urjR79mypbt26krGxsdStWzfp6tWrKs/17NkzqWXLlmpj0spSeYld0hWiXpoTtdJOeanXy0SglWhGz9fXl1GjRnH37l1yc3PZuXMncXFx/PTTT+zdu7c0x6GCIPyH5MefjRkzRmVk2aJFi1ixYgWbNm2icePGzJ49m549e3Lp0qVCz4BOmzaN+vXry3t+CoIg/BdpNdC7efMmjRs3xtnZmd9++w1/f38qV66Mr68vrVu35rfffqN79+6vqq9CKSrt6LCIiIhS6NWbw9/fX+UzYSEhIbz77rtl0KPyoaj4M0mSWLZsGbNmzcLZ2RnIe164Tp06hIaGyo8KAPz+++8cOnSIkJAQeV9AQRCE/yKtFmO89957/PvvvwB06tSJGjVqEBsby9OnTzlx4sRLPbitUCj+k5FNxSU+lFRxiQ9OTk7ExMQU+vUm5cPm8/PzK/EK3JLy9fWVa+rl5cXPP/9MTExMqQ3yIiIitIpQ8/X1ZcuWLUUes3Hjxpd6LvBVu3XrFgkJCUpb7piamtKuXTsiIyPl1x48eMD48ePZvHkzlSpVKouuCoIg6AytBnrSCw9I//7776Snp5dqh95EItqrbM2dO/e1D/Ty5eTksHHjxhKtRs/NzSU3N/el+5CdnY2/v7+8/2J5lR9fVqdOHaXXC0abSZKEq6srbm5u2NravvY+CoIg6JqXSsZ4ceD3skS0V/mO9rp8+TJeXl7cv38fyNuM183Nrdg6jRgxgiNHjnD79m1mzpyJkZERa9eu5f79+yxYsEC+JRcVFYW3tzdPnjwhJyeHGTNmMGjQIOLj47GxscHT05O9e/eSkpLCihUrcHJyws3NDcibgdbX1+fQoUPUrl1bZf+1iRhTFyO2ceNGNm3aRI0aNbh69SqDBw8mOjqaL7/8Ej8/P7755hvefvttJk6cyNOnT3n27BnDhg1j1qxZQN7sY2xsLGlpady+fZuwsDB5c+MXZWdn4+Liwl9//YWRkRE//PADNjY2REREMGnSJOzs7Dh79iwzZ85k37592NjY4OXlRWZmJjNnzuT3339HX1+fevXqFfrPxr1793B2dmbixImMGTOmUNuvIgKtuPizgtFmBVf35+bmolAoyMrKYtWqVTx58oQpU6YoHffiZ8paeYld0hWiXpoTtdJOeanXa4tAUygUhTZRLWpTVW2JaK/yG+2VnZ2Ns7Mzc+fO5dNPPwXg4cOHAMXWKT09nZMnT3L9+nWsra2ZOXMmkZGRREVF4eTkxNChQ0lOTmbChAns37+fevXq8fDhQ1q3bi1vS5KSkkKLFi2YO3cuBw4cwNPTEycnJwIDAwkKCuL48eOFQu5V0SRi7MKFC2pjxABOnz7NuXPn5Izaw4cP4+XlJd8WTU1N5fDhwxgZGZGRkYG9vT2Ojo7Y2dkBEBkZyblz5wrNXL3o4sWLLF++nJ9++olff/2VoUOHcvnyZSBv0P3999/zww8/ACh937799luuXr3K2bNnMTIykh/HyBcbG8vQoUNZunSp2scxXkUEWnHxZ/mzdiEhITRp0kQ+5sqVKzRu3Jj9+/fz888/Ex0dTeXKyoNNOzs7OnfujKenZ4n69qroeuySrhH10pyolXZ0vV6vLQIt/7ZI/o74z549w83NrdAP1ZJuTCqivfKUx2ivuLg4nj17Jg/yAGrWrEl6enqxdcp/LrBp06YYGxszcOBAIG/G8fHjxyQnJ3Py5Elu3rxZ6EH9uLg4mjRpgrGxsbxKs3379ty4caNENdAkYuzo0aNqY8QA7O3t5UGeKhkZGbi7uxMTE4Oenh63b98mJiZGHug5OTkVO8iDvNnQbt26ATB48GAmTJggb4TdpEkTOnfurPJze/fuZeHChfLf41q1asnvXbx4kT59+hAaGkrLli3Vtv26ItAKxp9JkoSfnx9ZWVnya0+ePOH69etMnz4dJycnrKyslKKC7t+/z8cff8zWrVtp27atyr0by0J5iV3SFaJemhO10k55qddri0AbNWqU0teaDkY0JaK98rxJ0V6a1OnF73v+1/kzyNnZ2UiShKWlJSdPniz0+fj4eIyMjORBtb6+vsaZs+r6ok3E2Iuz2gXrqMqMGTOoWbMm586dw8DAgP79+yt9L4r7vDoFZ9xLeo769evz/Plzjhw5UuRA71VFoBUXf+bl5cW3336Lubm5vL1K/fr1GThwIBUqVCi02CX/Z0GzZs1o3Lhxifv1quh67JKuEfXSnKiVdnS9Xq8tAm3Dhg0a/SpNItqrfER7NWvWjEqVKrFt2zb5tYcPH2pdJ3Xs7e25desW4eHh8msxMTFkZmYW+9kqVaoUW2NtFBUjpsqL34ukpCQaNGiAgYEBcXFxJb5lEB8fz9GjR4G8RJE6depoNGPVp08fli9fLj+6UPDWbfXq1QkLCyM0NBR/f/8S9etlFBd/Nm3aNL744gsmTJhAmzZtSEtL48CBA2r/EyEIgvBfV6Ks29epYLRXmzZtaNiwofxebGwsHTp0oGXLlrRq1UqjaC8rKytsbGw4e/asfOtp/vz5rF69mpYtW/Ljjz+qjfaytramU6dOStFe169fx8rKCmtra/r378+jR49Utr9jxw6sra1p1aoVQ4YMUYr2iomJwcbGRu0/rP3796d+/fpYWFjQu3dvpXitkSNHyrOFLVu2xMnJiefPnytFe/Xp00ej6yxo+PDhuLq60rVrV1q2bImNjQ1HjhxRe7yBgQG7d+9mw4YNWFtb07JlS3lAqk2d1KlevTr79u3jm2++oWXLljRv3pzp06drtCp18uTJdO/eHRsbGxITE7VqV5WCMWItWrTg+PHjcoyYKhMmTOCbb77BxsaG/fv3M2vWLDZs2ECLFi2YPn06H374YYn6YWlpycaNG7G2tubbb79l27ZtGj0z6+3tzfvvv0/r1q2xsbEpNFNfpUoVDhw4wMmTJ5k6dWqJ+lZS+fFnL/7KX3ikUCjw9/cnISGBZ8+eER4eXuQjGI0aNUKSJK22ohEEQXiTKKTSXjr7hsnP+9Tl/cUEoaw9efIEU1NTHj58+FLP6KWmpjJ79mx27dpFYmIirVq1Yvny5bRp0wbI2yPP29ubQ4cOkZycjIODAytXruS9994rrUt5LbKysti/fz9OTk46fbtIV4h6aU7USjvlpV75P2NTUlKoWrWqVp/V+Rk9QRD+O8aNG0dYWBibN28mNjaWHj164OjoyN27d5Ekib59+3Lz5k12797NuXPneOedd3B0dBT7eQqCIKjxUvvo6SIR7fVqlfdoLzc3N/n5yIIiIyOpWLFiGfSoaLa2toUWgVhaWhabclEeZWRkEBISwu7du+UFV35+fvz222+sWbMGFxcXTp06xYULF+THDtasWUPdunXZtm0b48aNK8vuC4Ig6KQ3bqDn5ORU7DYrQsn5+vrKD8aXRyVNC3ndAgMDSU1NJTo6mpiYGK5cuaKU5WpjY8Px48epUqVKGfaydGVnZ5OTk1NoYUXFihU5ceKE/B+1gu/r6elhZGTEiRMnxEBPEARBBXHrVhB0kJubm7wQIiYmhp9//lnp/ZiYmDdqkAd5i0Dat2/PvHnzuHfvHjk5OQQHBxMZGcn9+/cxNzenYcOG+Pj4kJSURGZmJgsXLuTOnTtyGosgCIKg7I2b0ROEsqJQKOS4sfT0dObMmSNvAn7w4EF8fHzIzs6mevXqrFmzhubNm3Pt2rVC0Xjz58/Hz8+P5ORkZsyYga+vLykpKdjY2GBnZ0dgYCAKhYKkpCSqVatGdHQ0Hh4epKWlYWxszNKlS+nQoUOR0XDqnDp1ikmTJpGTk0N2djaTJk1i4sSJhY4r7Qi0/PizH3/8kQkTJmBmZoa+vr68Sv2vv/4C4Ndff2XChAnUqFEDfX19unXrRq9evZAkSecjjAoqL7FLukLUS3OiVtopL/V6mf6JVbeCUEoUCgWzZs1i3rx53Lx5E1tbW/766y8qVaqEhYUFERERWFtbs2XLFr7++msuXryIl5cXdevWLRSNlz/QW7ZsGRs3biQ0NJTQ0FCltpKSkqhUqRJNmzZl3bp19OzZkxMnTjBw4ECuX7/Ow4cPady4MTt27GDAgAFyNFxR+yc6OzszdOhQOeEkKSlJ5Qbkfn5+KiPQtm7dSqVKlV6yknmbeT99+pQaNWqwePFinj17xuzZs+X309PTyc7OxtTUlKlTp9K0aVM+++yzl25XEARBFz19+pRhw4aVaNWtmNEThFKU/5xYkyZNcHBw4NixY1SvXh1ra2usra2BvD0KJ02axN27d7WKxlMlLi4OPT09evbMmxHr2LEjderUISYmhgYNGmgdDde1a1fmzZvHtWvX+PDDD+nYsaPK415XBFpSUhIXLlzg22+/VTkTee3aNW7cuMGyZcvo3r17qbX7qpWX2CVdIeqlOVEr7ZSXer22CDRBELRT3AbG2kTjlaRNbaPhvLy8cHZ2Jjw8nBkzZmBlZcX3339f6LhXFYF28OBBJEmiWbNmXL9+nalTp2Jubs64ceOoUKEC27dvp1atWjRs2JDY2Fg8PT3p27dvuV2ApeuxS7pG1Etzolba0fV6vbYINEEQipYfARgfH8/x48fp1KkTdnZ2xMbGcuHCBQB+/vlnzMzMMDMzUxuNV1BRMXnNmjUjNzdXjlE7efIkCQkJJU6CiIuLo3HjxowfP54ZM2ao7M+rlJKSwqRJkzA3N8fFxYWOHTty8OBB+Yfc/fv3GTlyJObm5nh4eDBy5Eil2D1BEARBmZjRE4RSlJOTQ6tWrUhPT2fFihU0atQIyIuBc3FxkRdjbN++HYVCwY4dOwgODsbQ0JDc3FyV279069aNgIAAWrRogb29vdIxhoaG7Ny5Ew8PDyZPnoyxsTE7duzAxMSEhw8fat3/VatWceTIEQwNDdHX12fJkiUlrkVJDB48mMGDB6t938PDAw8Pj9fYI0EQhPJNLMYQhFJScCXsf42IQNNOeYld0hWiXpoTtdJOeamXiEATBOGNICLQBEEQSpe4dSsIpaS8TI4nJibSo0ePQq93796dxYsXl0GP8ogINEEQhNInZvSE/6xVq1bh6upa5DE3btygdevWtGrVig0bNtClSxel/exepWXLlpGQkFDq561duzYxMTGFfhUc5I0dO5bmzZvTr1+/Um9fneIi0PI3aFYXgSYIgiAUJgZ6gs7Jzs4u6y7IduzYQZs2bTh37hyjR48u9fMXda2vaqBXnAcPHvDzzz8TGxvLrl27Xlu7IgJNEASh9Ilbt8JrtXv3bqZPn46hoSG9evXihx9+IDo6mi5dujBkyBCOHj3Ke++9R1BQEB4eHpw5cwaAQYMGMWfOHAC6dOmCl5cXffv2BWDgwIH07t0bV1dXXF1dMTIy4vr169y+fRsrKyt+/vlnDA0NSU1NZdy4ccTExFCrVi359p86P/30E0uXLiUnJ4fTp0+zdetWpfcTExNxc3Pj2rVrSJLEF198IaczFBdL9tlnnxEWFoaLiwteXl6F2vb39+fevXsMGTKEihUrEhgYSN++fYmKiuLtt98GYMaMGeTk5LBw4UIaNWrEoEGDOHLkCCkpKXz22WdyVu61a9fw8vIiMTGR58+fM2HCBD7//HOV15ycnEzXrl159uwZH3zwAUOHDmX69OmFjhMRaC+nvMQu6QpRL82JWmmnvNTrZfonBnrCa5OYmMiYMWP4888/MTc3Z8OGDTx69Eh+/9GjR5w+fRqFQoG3tzfPnz/n/PnzZGRk0LFjR8zNzRkyZEix7cTExHD06FGMjIxwcHAgJCSETz/9FH9/f4yMjLhy5QpPnjzBzs6Odu3aqT2Pi4sLN2/elKPIXvTFF1/QrFkzdu7cSWJiIh988AEtW7akdevW9O/fXymWbMCAAVy/fh3I2yvO0tKShQsXqm3b19eXH3/8kV9++UXeE2/s2LGsWbOGb775hufPn7Nhwwalfe4ePHhAdHQ0jx49onXr1nTo0IF27drx6aefEhwcjLm5OU+fPpWvO38la0HVqlVj//792NjYEBMTo7Z/3377rcoItFmtcqlUqehNmVUpuEn05MmTmTRpklIEmomJiXyMv7+/ygi0l91ouizk738oaEbUS3OiVtrR9Xo9ffq0xJ8VAz3htTl16hQtWrTA3NwcgFGjRuHm5ia/7+rqKqc4hIeHs2TJEvT09KhcuTIuLi6EhYVpNNDr16+fnLfatm1bOfbr8OHDLF26FIVCgampKcOGDSs2Eqwo4eHhnD17Fsh77q1///6Eh4dTuXLlImPJKlSowIgRI7Ruz93dnbZt2zJnzhy2b99O27Zteeedd+T3x44di0KhoGbNmnJfqlWrxsWLFxk6dKh8XGpqKpcuXVI50NOUiEB7OeUldklXiHppTtRKO+WlXiICTXgjmJiYqH2vYKyXgYGBUpTXs2fPlI4t+LC+vr6+2ufgiosn01ZR5yv4XqVKldDT0/7xWDMzMxwcHPjll19Ys2YN/v7+xfZHkiRq1KhR5OxcSYgItNKh67FLukbUS3OiVtrR9XqJCDShXLCzs+P8+fPExcUBEBwcTGZmpspjHR0d+eGHH5AkifT0dDZv3ixvCdK0aVNOnz4NwK1btzReceno6MiGDRuQJIknT568dHSWo6Mj69atA+Dff/9l586ddO/evdRiyVRFn3l6ejJz5kySk5NxdHRUem/jxo0APH78mF27dtGtWzeaNWtG1apV5Wg2gOvXr/P48WMtr/b1EBFogiAIpUvM6AmvTe3atVm/fj19+/bFyMiI7t27Y2JiojJJYvbs2Xh4eGBtbQ3kLcbIj8aaNm0aQ4YMwdraGktLyyKfs3vxnOPGjcPc3JxatWrRsWNHpQUF2lqxYgUTJ07E2toaSZKYOXOm3JfSiCXz8PBg/PjxVKpUiY0bN2JjY4OdnR2mpqZ89tlnhWYQa9WqxQcffEBKSgqff/459vb2AOzduxcvLy95YUnNmjULLSzRFSICTRAEoZRJgvAaPXnyRP79rl27JHNz8zLsTflz584dqW7dukp1lCRJeuedd6Rz586VTackSUpJSZEA6eHDhy91nidPnkienp5Sw4YNJWNjY6l9+/bSmTNn5PdTU1OlSZMmSWZmZpKxsbFkYWEhrVmz5mW7/9plZmZKoaGhUmZmZll3pVwQ9dKcqJV2yku98n/GpqSkaP1ZMaMnvFYrV67kl19+IScnh6pVq7Jly5ay7lK5kb8Sd8GCBVSpUqWsu/NKjBs3jgsXLrB582bq169PcHAwjo6OXLp0CTMzM7766iuOHDlCcHAwjRo14tChQ7i7u1O/fn369OlT1t0XBEHQOQpJKie5TYLwiqxfv55Vq1YVen3lypV06tTplbbt7+/Pzp07C70eEhLCu++++8raLe0YtPzA7YcPH5Z41W1GRgZVqlRh9+7dfPzxx/LrH3zwAR999BHz58/HysqKIUOGMHv2bJXvlxflJUhdV4h6aU7USjvlpV75P2NTUlKoWrWqVp8VM3qCzsnfVNnIyIjNmzfLz+m9KtOnTyc6OppGjRoVeZyfnx/Tp0+XV/X6+vrSrFkzhg8fXuK2fX198fX1Lbat0pYfg6ZLiotAA7C3t2fPnj2MGTOG+vXrExERwdWrV1m6dGlZdFkQBEHniVW3gs4JDAzE19eXmJiYVz7I08bcuXOVtnLx9/d/qUGeNm2VhdzcXHJzc19be8VFoEHeLGvz5s1p0KCBnK6yevVqHBwcXls/BUEQyhMxoyfoFA8PD44fP86VK1dYuXIlc+bMwcfHh+zsbKpXr86aNWto3rw5EREReHl5ybNSFy5coHfv3sTHx8sxY56enuzdu5eUlBRWrFgh77W2Z88evL29qVChAr169VJqf8qUKfzxxx9kZWVRtWpV1q1bR7NmzeSNnTt16oS+vj6HDh1i2rRp2NjY4OXlRVpaWpGRbba2tpw+fZp79+7RvXt3AgMD1dbgxbb27NlDmzZtuHXrlrwR9LBhw+jUqRMTJ05EoVAwc+ZM9u3bR3p6OnPmzJEHoFFRUXh7e/PkyRNycnKYMWMGgwYNUtu2n58fsbGxpKWlcfv2bcLCwjAzMyt0XFlEoGVlZbFs2TIiIyPZuXMnDRs25MSJE0yaNInatWvTrVs3rdstK+UldklXiHppTtRKO+WlXi/TP/GMnqBz8rNs7e3tsbCwICIiAmtra7Zs2cLXX3/NxYsX+eOPP4oc6DVu3JgdO3YwYMAADhw4gKenJ3FxcSQmJmJhYcHx48dp3rw5a9eu5bPPPuPWrVs0atSIf//9l1q1agHw888/s3HjRg4cOADkbUCclJQkbwfj6uoqD/S8vb25c+cOmzdvliPbpk+fzpAhQ+jSpQvVq1dn+/btZGVl0bx5c7Zu3Ur79u3V1uDFtoYPH07nzp2ZMGECDx48wNramps3b2JiYoJCoWDWrFnMmzePmzdvYmtry19//UW1atXo2rUr+/fvp169ejx8+JDWrVsTGRmpcvAGeQO9tWvXcu7cOerUqaO2f35+fioj0LZu3SoPRl/Gs2fPlCLQnj17xrRp0xg+fDjTp0/H1tZWPnbVqlU8evRIHlgLgiC8aZ4+fcqwYcPEM3rCm+X06dNYW1vLt2+HDx/OpEmTuHv3brGfNTY2pn///gC0b99ejjrLj2Fr3rw5kBcb9sUXX8ifCwsLY+XKlaSmppKbm6vxxsLFRbYNGTIEAwMDDAwMsLGx4caNG0UO9F7k6enJ+PHjmTBhAuvWrePTTz9VShIZN24cAE2aNMHBwYFjx45Rs2ZNbt68yUcffaR0rri4OLUDPQAnJ6ciB3lQNhFo3bp1Izs7m7Zt2yrNxO7du1fud3lRXmKXdIWol+ZErbRTXuolItCE/5ziYtCMjIzkDYX19fWVji2o4KbD//vf//j888+Jiori3Xff5fz58yV+9uvFzYw1jWVTp23btlSqVImjR4+ydu1awsPDi21fkiQsLS05efKkVm0VFUWXr6wi0Dp37oyPjw9VqlThnXfe4Y8//iA4OJjvvvtOp39Iq6PrsUu6RtRLc6JW2tH1eokINOGNZGdnR2xsLBcuXADybqWamZlhZmZGkyZN+Oeff/j3338B2Lx5s0bnbN++PefPn+fKlStA3jNh+TFsKSkpVKhQgXr16iFJUqEtV6pUqVIokixfUZFtJaGqLU9PT1xcXLCwsOD9999Xei8/4iw+Pp7jx4/TqVMn7O3tuXXrltKgMCYmRm3snC4oLgLt559/pk2bNgwfPpzmzZuzYMECvv76a/m5RkEQBEGZmNETdFatWrXYsmULLi4u8mKM7du3o1AoqF+/PtOmTaNt27bUqVOn0O3Jos75448/0q9fP3nVZv6tRmtra4YOHYqlpSVvvfUWffv2Vfrs5MmT6d69O5UqVeLQoUNK7xUV2VYSL7ZVu3ZtBg4cyMSJE/n8888LHZ+Tk0OrVq1IT09nxYoV8lYx+/btY8qUKUyePJmsrCwaNmxIaGhoifv1qhUXgVa3bl2l3F5BEAShaGIxhiCUE9HR0QwbNowrV66gp/f/k/EvLtwoC6WxYTJAamoqs2fPZteuXSQmJtKqVSuWL19OmzZtAEhLS2P69OmEhoby6NEjGjdujIeHR7mb0Ssvm7TqClEvzYlaaae81EtsmCwIb7hx48Zx6NAh1q9frzTIe9OICDRBEITSJQZ6glBG3NzcOHXqVKHXIyMjqVixotJr69evV3sebSflY2JicHV1LfT6qFGj+PLLL7U6V2nKyMggJCSE3bt3y4tg/Pz8+O2331izZg3z58/n5MmTjBo1ii5dugAwYcIEgoKCOHPmjBjoCYIgqCAGeq+YLtxWKwsRERE8e/as0IbELys+Pp4DBw7o7K26mjVrFhunlr+hc3Jy8mvrV0E2NjYaxZ9NmTIFExMT/Pz8XnmfQESgCYIgvApv7j0g4ZUraouQiIgIeaNhbT9blPj4+CJTJYpS0jaF10NEoAmCIJQ+MaP3Gnz//feEhoby77//4uvry+jRowH1cVsZGRm4uroSGxtLhQoVqFOnTqFVngWtX7+e7777DkNDQ3Jycli/fj3t2rWjUaNGhIaGYmNjA4CtrS0BAQF06dKFLl26YG1tzalTp0hKSsLZ2ZmAgAAUCgUJCQl4eHgQHx9PRkYGzs7OzJ8/H4BGjRoxZMgQjh49ynvvvYefnx+urq6kpaWRm5uLs7MzAwcOJDAwkJycHCIiIujfvz8uLi7Y2Njw2WefERYWhouLC8nJySQnJ7Ns2TIgL+EgOjqajRs3ArBw4UKCg4PR09OjYsWKHDlyBDc3N/755x9sbGxo2LAhe/bsKfY6W7RoQVRUlHyOgIAAfv31V7Kzs6lduzZBQUG88847auur7vsERcepRUdH4+HhQVpaGsbGxixdupQOHToonffQoUPk5OSwfPlyHB0dgby95ObNm0dGRgb6+vosXLiQrl27qu2fq6srhoaG3Lx5kxs3btC1a1fc3NyYNm0a//vf/+jbty/fffcd8P+pI/krigcOHEjv3r1xdXXl/v37uLq6cvv2berXr0/NmjUxNzdX2aaIQHs55SV2SVeIemlO1Eo75aVeL9M/MdB7DYyMjDhz5gxXrlyhTZs2jBw5EgMDA7y9vQkICADy9gfz9PTkwIEDHDhwgOTkZC5dugRQbDrD5MmTuXLlCvXq1SMrK0vpH+CiXLp0iZMnT5KVlYWDgwPbtm1j2LBhjBo1ihkzZtC5c2eys7Pp3bs327dvlzNSHz16xOnTp1EoFHh6etK7d298fHzkvtaoUQM3NzelQVx8fDwpKSlYWlqycOFCgCJvCW7atImQkBBOnDiBqakpSUlJGBkZERgYqBR9pomrV69y7NgxKlSowNatW4mLiyMyMhJ9fX02b96Mu7s7+/btU/t5dd+nxMRERo8erRSn9ujRIwAyMzPp378/69ato2fPnpw4cYIBAwZw/fp1IG+/OAsLCwICAjh16hR9+vThxo0b/Pvvv/j5+XHw4EGqVq3K9evX6dSpE/Hx8So3KM4XGxvL0aNH0dPTo3nz5iQlJREWFkZmZiZNmjRh7NixWFpaFlknDw8P2rZty8GDB7l79y42NjZqB3rffvutygi0Wa1yqVRJ9ebURdm/f7/8+8mTJzNp0iSlCDQTExN27drFrFmzmD59Onp6ety5c4dGjRphZ2fHjBkzymUEWlhYWFl3oVwR9dKcqJV2dL1eT58+LfFnxUDvNcgPmDc3N8fAwICEhAQaNGigNm6rZcuWXL58GXd3dzp37lxstFO3bt0YOXIkn3zyCR999FGhzXTVcXFxkXcDHzFiBOHh4Tg7O3P48GEePHggH5eWlkZcXJz8taurq5z84ODgwNSpU0lLS6Nz587yrJQq+e1oYu/evbi5uWFqagpA9erVNfqcKiNGjJCXzYeGhhIVFcUHH3wAoDYxoyB136ei4tTi4uLQ09OjZ8+8maqOHTtSp04dYmJiaNCgAQYGBvKCCDs7O+rXr8+5c+e4cOEC169fV7oVqaenx//+9z/ee+89tX10dnaWn22ztramZ8+e8ve2efPmXLt2rdiB3uHDh+UBrZmZWZGLG0QE2sspL7FLukLUS3OiVtopL/USEWg6TlX8VVFxW02aNOHSpUscOXKE8PBwpk2bRkxMjNrBTkhICGfPniUiIgInJyfmz5/P0KFDi40Je1F+bBbkDWJefCg+X8GIrAEDBmBvb09YWBirVq1i2bJlSrMzBVWqVElpaxBt+6dOcecp2F9JkvDx8WHChAkanVubWLQXY89K8r4kSXTv3p2tW7dq1L98L/4ZUxe5pk3Ni+qviEArHboeu6RrRL00J2qlHV2vl4hAK4eKitu6c+cOCoWCPn36EBAQgCRJ3L59W+V5srOzuXHjBra2tkyZMoWBAwdy5swZAJo2bcrp06cBOHPmjNKsHEBwcDBZWVlkZGSwdetWHB0dMTExoWvXrixYsEA+7t69e9y5c0dl+9euXaNOnTq4uLiwaNEiebuQqlWrqo0Ly9e0aVOio6PJycnh6dOnhISEyO/16dOHwMBA+RzJycnk5OSoPG9x11lQ3759CQwMlGflsrKyOHfunNrji/o+FRWn1qxZM3Jzc+XbASdPniQhIUF+jjA7O1uObTtz5gz37t3DxsaGnj17Eh4ezvnz5+V28r+fpaFgrW7duiWvZoW8GLcff/wRgPv377Nnz55Sa1dTIgJNEAShdIkZvTJSVNxWbGwsPj4+SJJEdnY2I0eOpEWLFirPk5OTw5gxY3j8+DEGBgbUqlVLjoiaP38+o0aNIigoiPbt2xe6dWdhYUGHDh14/Pgxzs7ODB06FIAtW7bw1VdfYWVlhUKhoHLlygQFBdGgQYNC7e/YsYPg4GAMDQ3Jzc2VV8T269ePzZs3Y2NjIy/GeFH//v3Zvn07FhYWNGjQgFatWsnPIYwcOZJ79+5hb2+PgYEBlStXJjw8nBYtWmBpaYmVlRVNmjRhz549xV5nQcOHD+fRo0fy4obs7GzGjBlDq1attP4+FRWnZmhoyM6dO/Hw8GDy5MkYGxuzY8cOTExMePjwIaamply4cIGWLVuSnZ3N1q1bqVKlClWqVGHr1q189tlnPH36lMzMTFq1aqX1DJ8606ZNY8iQIVhbW2NpaUm7du3k95YvX46rqyvNmzfHzMyMDz/8sFTa1IaIQBMEQShdIgLtP+rF1ZeC8DJeNgItJycHPz8/goODSUhIoH79+ri6ujJr1iz5FvKDBw/w9vbm0KFDJCcn4+DgwMqVK4t8dlFXlZfYJV0h6qU5USvtlJd6iQg0QRDKtYULF7JmzRo2bdqEpaUl0dHRjB49GlNTUzw8PJAkib59+1KhQgV2795N1apV+e677+R4tMqVtd/SRRAE4b9ADPTKif379zNjxoxCr/v4+DBkyBCtzxcREVEKvXpz+Pv7s3PnzkKvh4SE8O6775ZBj5SV9vdf15w8eRJnZ2c+/vhjIG+/xm3btsnPJ167do1Tp05x4cIF+db8mjVrqFu3Ltu2bWPcuHFl1ndBEARdJhZjlBNOTk7ExMQU+lWSf+QVCkWpx2/Z2tq+lsHj48eP6dChAzY2Nnz99deldl5fX1+V9X0dg7y0tDSlFa42NjakpqYCsGzZMhISEuTvv5ubG8OHD3+p778usre35/Dhw1y9ehWAv//+mxMnTvDRRx8ByHtDFlxNrKenh5GRkdKCEkEQBEGZmNETyo3s7GzCwsIwMTHhzz//LOvuvDIFN4NetmwZXbp0oW7dugBv7OrS6dOn8+TJE8zNzdHX1ycnJ4evv/5aaQ/Khg0b4uPjQ1BQEJUrV2bp0qXcuXNHjkcTBEEQChMDvf+4ouK9FAoFX3/9tcr4tpMnT+Lu7k52djZt2rQpNkf233//Zfjw4dy/fx+FQsEHH3zAhg0b2LhxI6GhoYSGhgJ5m98GBAQQERFBREQEkyZNws7OjrNnzzJz5kymTp1KSkoKNjY2BAQEkJiYyPLly8nMzCQ3N5f58+fzySefAHD37l08PT2Ji4tDoVDg7OzMvHnzSE1N5auvvuLvv//m2bNn2NnZsWrVKgwNDdX2f9++ffj5+ZGZmYlCoSAoKIh27dpx8OBBfHx8yM7Opnr16qxZs4bmzZsTERHB559/joODA3/++SfZ2dls2rQJW1tbAIKCgggICMDExIT+/fsrtaVQKEhKSmLFihXcu3ePIUOGULFiRblW+YkjOTk5TJ8+nd9//x2Arl27smTJEgwNDXF1dcXIyIjr169z+/ZtrKys+Pnnn4u8xkaNGuHi4kJYWBgJCQmMHTuWWbNmqTy2tCLQ8qPPfvnlF7Zs2cJPP/1E8+bN+fvvv5kyZQq1a9eWV2z/+uuvTJgwgRo1aqCvr0+3bt3o1asXkiTpfHzRi8pL7JKuEPXSnKiVdspLvUQEmlBi6uK98qmKb8vNzWXIkCFs2LABR0dHDh06JOfTqhMcHEzjxo3lzN7iYt3yXb58me+//54ffvgBgPT0dKWB4aNHj/j0009RKBTEx8djZ2fHP//8g5GRESNGjKBHjx7s2LEDyBtsQl7EVqdOnVi3bh2SJDF+/HiWL1/O1KlTVfbh6tWrjB49mmPHjmFubk5WVhZPnz4lMTGRYcOGERERgbW1NVu2bGHgwIFcvHgRgCtXrvDDDz/w/fffExgYyMyZMzl48CAXLlxgzpw5nDt3jnr16ql89g7ybif/+OOP/PLLL/L+e/nXDbB27VqioqI4e/Ys+vr69OnTh6VLl+Lt7Q3kzQwePXoUIyMjHBwcCAkJ4dNPPy2y3snJyURGRvLw4UPeffddRo8ejZmZWaHjSisCLX9zbS8vLwYMGECVKlW4ffs2NWrUoFevXsyZM4eaNWvKx/v7+5Oenk52djampqZMnTqVpk2bqt2kW9fpeuySrhH10pyolXZ0vV4iAk0oMXXxXvlUxbfl79mXH3fWo0cPmjRpUmQ7dnZ2LF26lMmTJ+Pg4KAUYVWUJk2a0LlzZ7Xv37p1i+HDh3Pnzh0MDAx4/Pgxt27dokGDBpw4cYKDBw/Kx9aqVQvIGyxFRkby3XffAZCRkYG+vr7aNsLCwujVq5ec+1qhQgVMTU357bffsLa2xtraGsir1aRJk7h79y6Qtzlx/j517du3lwfUR44c4aOPPqJevXoATJw4kW+//VajehQUHh4uz9wBjB8/ntWrV8sDvX79+lGpUiUA2rZty40bN4o957BhwwCoWbMmTZo04datWyoHeqUdgSZJEtbW1koxZrGxsZw5c0ZttNm1a9e4ceMGy5Yto3v37lq3WZbKS+ySrhD10pyolXbKS71EBJpQIprEe6mL0npRcfFe7du3JyYmhvDwcHbu3Mns2bM5d+6cVvFlqgwdOpQFCxYwcOBAAGrUqFFslJokSYSEhGicCVxSpVU7Tb14Hk3bL8lnSjsC7ZNPPmHBggU0btwYS0tLzp07x/LlyxkzZox8vu3bt1OrVi0aNmxIbGwsnp6e9O3bt1xl3L5I12OXdI2ol+ZErbSj6/USEWhCiRQV71UUc3NzsrOzOXr0KJA3s1TcbNGtW7cwMTFh8ODBrFy5kqtXr5KWlkbTpk05f/48GRkZckKENpKSkmjcuDGQd3s4KSkJyBsgOjg4sGTJEvnY/Fu3ffv2ZeHChfIgJikpievXr6tto2fPnhw8eFCOOsvKyiIlJQU7OztiY2O5cOECkHfr28zMTOUMWEEffvghBw4cICEhAUBOE1GlqCg5R0dHfvrpJzIzM8nOzmb9+vX06NGjyLZ11cqVKxk4cCDu7u5YWFgwZcoUPvvsM+bNmycfc//+fUaOHIm5uTkeHh6MHDmSbdu2lWGvBUEQdJ+Y0fsPKyreqyiGhob88ssvuLu7k5OTQ5s2bWjZsmWRn4mIiOC7776TZ4kWL16MqakpdnZ2ODk5YWVlRb169ejQoYOcxaqJ5cuXM3DgQKpVq8aHH35Iw4YN5fc2b97MF198gaWlJRUqVMDZ2Zm5c+eydOlSpk+fjo2NDXp6ehgYGLBo0SKaNm2qso2mTZuyYcMGRowYQVZWFvr6+gQGBtK2bVu2bNmCi4uLvBhj+/btxc7QWVlZ4efnR6dOnVQuxijIw8OD8ePHU6lSpULPQU6YMIEbN27QunVr4P/TTsqjKlWqsGzZMpYtW6b2GA8PDzw8PF5fpwRBEN4AIgJNEISXVtIItEaNGvHPP/8Uet3d3Z3Vq1dz48YNpkyZwokTJ3j+/Dm9evVi5cqV1KlTpzS7/9qVl9glXSHqpTlRK+2Ul3q9TASauHUrCEKZiYqK4v79+/Kv/JVvgwYNIj09nR49eqBQKDhy5Ah//vknmZmZfPLJJ+Tm5pZxzwVBEMoHcetWKFV9+vThf//7n9Jr1atXl5/n02Xlue+a0rWot/yV0PkWLFjAu+++S+fOnQkLCyM+Pp5z587J/4PdtGkT1atX58iRI/Kqb0EQBEG9Mh3o5W8MW61atbLsxmsXERHBs2fPNN5iRFPx8fEcOHCgTNMT9uzZU2ZtF+Tn58f06dOVVpEWpzT7vnHjRuzs7OQtWV4lbf48+fr64uvr+8r7VBKZmZkEBwfz1VdfoVAoeP78OQqFQml1r7GxMXp6epw4cUIM9ARBEDQgbt2+IkVtZREREaG0KbE2ny1KfHx8kSs4i1LSNnXV3Llzi91m5VXJyclh48aN8ipdbeTm5mp9W7K4P0/lRX7qh6urK5C392LlypXx9vbm6dOnpKenM2XKFHJyckTsmSAIgobK/Nbt999/rzJiS100V0ZGBq6ursTGxlKhQgXq1Kkjpy2osn79er777jsMDQ3Jyclh/fr1tGvXjkaNGhEaGionDtja2hIQEECXLl3o0qUL1tbWnDp1iqSkJJydnQkICEChUJCQkICHhwfx8fFkZGTg7OzM/PnzgbwHy4cMGcLRo0d577338PPzw9XVlbS0NHJzc3F2dmbgwIEEBgaSk5NDREQE/fv3x8XFBRsbGz777DPCwsJwcXEhOTlZjroCWLVqFdHR0fLKy4ULFxIcHIyenh4VK1bkyJEjuLm58c8//2BjY0PDhg3Zs2dPsdfZokULoqKi5HMEBATw66+/kp2dTe3atQkKCuKdd95RW9/Lly/j5eUl/8Pr7u6Om5tbsXUaMWIER44c4fbt28ycORMjIyPWrl3L/fv3WbBgAUOHDgXynuHy9vbmyZMn5OTkMGPGDAYNGkR8fDw2NjZ4enqyd+9eUlJSWLFiBU5OTvKMZqdOndDX1+fQoUPUrl1bZf8VCgXz589nz549PHjwgGXLlnH58mVCQkJISUlh3bp1dOnSBchbxbt48WIA3n77bdauXYuZmRkbN25k06ZN1KhRg6tXrzJ48GCio6P58ssv8fPz45tvvuHtt99m4sSJPH36lGfPnjFs2DA5XszPz4/Y2FjS0tK4ffs2YWFhKrdouXbtmkZ/nnx9fYvsa3BwMLVq1eLChQsYGRnx66+/yhteb968mVWrVpGVlYWJiQkrV65UuaK6NCLQ8uPP8q1fv56ePXtSq1YtsrKyqFatGtu2beOLL75gxYoV6OnpMWTIEFq1agXofmRRUcpL7JKuEPXSnKiVdspLvcp1BJqqiC0DAwO10VwHDhwgOTmZS5cuAcVHaU2ePJkrV65Qr149srKylP5xKsqlS5c4efIkWVlZODg4sG3bNoYNG8aoUaOYMWMGnTt3Jjs7m969e7N9+3YGDRoE5EVynT59GoVCgaenJ71798bHx0fua40aNXBzc1MaxMXHx5OSkoKlpSULFy4E8v7xV2fTpk2EhIRw4sQJTE1NSUpKwsjIiMDAQLy8vIiJidHoGiEv3uvYsWNUqFCBrVu3EhcXR2RkJPr6+mzevBl3d3f27dun8rPZ2dnyliX50VoPHz4EKLZO6enpnDx5kuvXr2Ntbc3MmTOJjIwkKioKJycnhg4dSnJyMhMmTGD//v3Uq1ePhw8f0rp1a+zt7YG8fQBbtGjB3LlzOXDgAJ6enjg5OREYGEhQUBDHjx/X6LEAExMTTp8+zeHDh3F2dpYH1du3b2fq1KlERUVx4cIFpk6dytmzZzEzM+Prr79m3Lhxcs7s6dOnOXfunJwTfPjwYby8vOQta1JTUzl8+DBGRkZkZGRgb2+Po6MjdnZ2AERGRnLu3LkiV5OuWrVKoz9PxfU1KiqKmJgYGjduzPTp01m4cCFBQUH8+eefbNu2jWPHjmFkZMTx48cZNmyYHOlWUGlEoBWMLUtMTOTw4cN4e3sXijP77rvvePLkCXp6epiYmODq6kqLFi3KbexZQboeu6RrRL00J2qlHV2vV7mOQFMVsdWgQQO10VwtW7bk8uXLuLu707lz52J3xe/WrRsjR47kk08+4aOPPtI4DcHFxUXeKXvEiBGEh4fj7OzM4cOHefDggXxcWloacXFx8teurq7yPmoODg5MnTqVtLQ0OnfuXOQzRfntaGLv3r24ublhamoK5C0YKKkRI0bIS8pDQ0OJiorigw8+AFBKrFAlLi6OZ8+eKeWn1qxZk/T09GLrNGTIECBvjzpjY2M52cLW1pbHjx+TnJzMyZMnuXnzJh999FGhdps0aYKxsbG8B1379u01ivhSJb8vtra2pKeny7OJbdu25dq1awAcPXqUXr16yTNt7u7u+Pv7yzWyt7eXB3mqZGRk4O7uTkxMDHp6ety+fZuYmBh5oOfk5FTsliGa/nkqrq/t27eXN5lu3749K1euBGD37t38/fffcmwb5A0mMzIyqFixolIbpR2B5u/vT+3atZk9ezYGBup/LB09epSUlBSmTJlSZL11XXmJXdIVol6aE7XSTnmpV7mOQFMVuVRUNFeTJk24dOkSR44cITw8nGnTphETE6N2sBMSEsLZs2eJiIjAycmJ+fPnM3To0GKjt16kUCjI33Lw1KlTah/yLxjZNWDAAOzt7QkLC2PVqlUsW7ZM7SxEpUqV0NP7/0cmte2fOtpEjEmShI+PDxMmTChRWwXPA0XX6cXve/7XCoUChUJBdnY2kiRhaWnJyZMnC30+Pj4eIyMjeVCtr69f7MBUnfy28/NuC36taWxZcVFtM2bMoGbNmnLsW//+/ZW+F8V9HrT781RUX9XFnEmSxKhRo/jmm2+KPWdpRqDl5uby008/MWrUqEIDyg0bNmBhYUGtWrWIjIzE09OTL7/8EisrK63a0FW6Hruka0S9NCdqpR1dr9cbF4FWVDTXnTt3UCgU9OnTh4CAACRJ4vbt2yrPk52dzY0bN7C1tWXKlCkMHDiQM2fOAHkzSfkJDGfOnFGabYK8OK2srCwyMjLYunUrjo6OmJiY0LVrVxYsWCAfd+/ePe7cuaOy/WvXrlGnTh1cXFxYtGgRp06dAoqOtcrXtGlToqOjycnJ4enTp4SEhMjv9enTh8DAQPkcycnJ5OTkqDxvcddZUN++fQkMDJRnT7Oysjh37pza45s1a0alSpWUYqgePnyodZ3Usbe359atW4SHh8uvxcTEkJmZWexnq1SpUmyNtdG1a1cOHDjAvXv3gLzYsm7dusmDwxe9+L1ISkqiQYMGGBgYEBcXV6LbBJr+edK2r/n69OlDcHCwvMVMbm4u0dHRWvdTW+Hh4fzvf/9jzJgxhd6Li4ujb9++WFhY4O/vz8yZM+VHOgRBEITilfmMnipFRXPFxsbi4+ODJElkZ2czcuRIWrRoofI8OTk5jBkzhsePH2NgYECtWrXYsGEDAPPnz2fUqFEEBQXRvn17LC0tlT5rYWFBhw4dePz4Mc7OzvLtvC1btvDVV19hZWWFQqGgcuXKBAUF0aBBg0Lt79ixg+DgYAwNDcnNzZVXxPbr14/NmzdjY2MjL8Z4Uf/+/dm+fTsWFhY0aNCAVq1ayffoR44cyb1797C3t8fAwIDKlSsTHh5OixYtsLS0xMrKiiZNmrBnz55ir7Og4cOH8+jRI7p27QrkDZTHjBkjP/z+IgMDA3bv3s0XX3zBN998g56eHu7u7nz22Wda1Umd6tWrs2/fPqZMmcLkyZPJysqiYcOGhIaGFvvZyZMn0717dypVqlTkYgxNWVlZsXjxYnkLk7fffpt169apPX7ChAlMnjyZpUuX8s033zBr1ixGjhzJpk2bePfdd/nwww+17oOmf558fX216mu+Tp06sWjRIvr160d2djaZmZl8/PHH2Nraat1XbfTo0QN1AT0LFixQ+g+DIAiCoB0RgaZCfmaoptmvgvBfV5IItOLiz/JJkoSTkxMHDhxg165db8Tfy/ISu6QrRL00J2qlnfJSr5eJQNPJGT1BEN58UVFRSs9VXrhwge7du8srs/MtW7as0HOGgiAIgmbeiIHe/v37mTFjRqHXfXx85BWV2oiIiCiFXr05dC02S1tubm7y82wFRUZGFnr4XxfY2toWWgRiaWnJli1byqhHr0ZR8Wf5YmJiWLJkCdHR0dSrV+91d1EQBKHceyMGek5OTsVusyKUnC7HZmmipGkhr0JQUBBLly7F2NiYw4cPq7zNGRAQ8Eoi8nTZi/FnkLdv1LBhw1i9ejV169Yt4x4KgiCUTzq56lYQdNnLxMUtW7aMDRs2EBMTo/ZZtjcl0kwbL8afAXz55ZfY29vj7Oxcdh0TBEEo596IGT1BeNUUCgW+vr7s37+fLl26MHXqVNzc3Lh27RqSJPHFF1/w2WefARAdHY2HhwdpaWkYGxuzdOlSOnTowMCBA7lx4waurq5YW1uzY8eOQu3ExMQUijRLTEykfv368uMJcXFxODo6cuvWLebPn09sbCxJSUncu3eP9957j40bN/LWW2+RlZXF7NmzOXLkCJmZmbz//vsEBQWp3XMyLi6Obt26cezYMZo0aUJAQADh4eHs379faY9HePkItOLiz3777TeOHDnCmTNnlKJ/srOzdT6qSBPlJXZJV4h6aU7USjvlpV4v0z+x6lYQNKBQKJg7d658C3vIkCE0adKEb7/9lsTERD744AO2b99O69atadq0KevWraNnz56cOHGCgQMHcv36dUxMTAplD6vi5+enFGkWFxdHz549uXHjBvr6+nh6elKzZk1mz56Nn58f33//PefPn6du3bq4u7uTnZ3N2rVr+eabb8jJyWH27NkAzJs3j4SEBKUVrS/atm0bS5YsISAggNGjR3PmzJlCz9Ll91FVBNrWrVupVKmSFpXNiz9zc3PD29tbTuVYv349+/btU1qEkZubi56eHhYWFnz99ddatSEIglCe5T/KIlbdCsIrVHBD3/DwcM6ePQtA7dq16d+/P+Hh4VSuXBk9PT169sybserYsSN16tQhJiaGjh07lqjdZs2a0bx5c3bv3k3Pnj3Ztm0bsbGx8vsff/yx/AzbhAkT5Fi40NBQUlJS5M22MzMzadSoUZFtffrppxw9epSePXty+PBhlYM8KN0INFXxZ61bt5Zzk/O1bt2agIAAPv74YznCrbwqL7FLukLUS3OiVtopL/Uq1xFoglBeFBVTVtT2H6WxNYinpycLFy7k33//pXv37kXm4ua3J0kSK1eupEePHhq3k52dzYULF6hRowZ3795Ve1xpRaCpiz97++23efvttwsd37hxY43zqssDXY9d0jWiXpoTtdKOrtfrjYtAEwRd5+joKKdN/Pvvv+zcuZPu3bvTrFkzcnNz5YizkydPkpCQUOSt2hepirLr0aMHCQkJzJ8/n88//1zpvf379/PgwQMg75ano6MjkBdpt3TpUjlR5enTp1y8eLHItqdPn06zZs04fvw4U6ZM4fr16xr3uySKij8TBEEQXp6Y0ROEElixYgUTJ07E2toaSZKYOXOm/HzZzp078fDwYPLkyRgbG7Njx44iZwNfpCrSTKFQMHbsWLZu3Ur79u2Vju/UqRPDhg3j7t278mIMAG9vb54/f067du3kWT5vb2+1MXh79+7lwIEDnDlzhkqVKvHdd98xePBgTp48ibGxcQmqVLyi4s9eJB4nFgRB0J4Y6AmCBl4cZNSpU0flJtKQt+HxyZMnVb4XHx9fbFuNGzfm3LlzhV4/evQoXl5ehV5v0KCByhW8BgYG+Pv74+/vX2ybAL1796Z3797y14MGDSqUUlGaiotAW7t2LVu3buWvv/4iNTWVpKQkqlWr9sr6IwiC8CYSt24FQcdFR0fTtGlT9PT0GDZsWFl3p9RERUVx//59+Vf+7e78weXTp0/p1auXytQbQRAEQTNiRk8QykBMTIzS5sD5Ro0axZdffqn0mq2trdpn5fz8/LRuW1ci4YqLQMufvRSRhIIgCCUnBnrCf86qVauIjo6Wn2VT5caNGwwaNAhJkvDw8GDTpk14eXnRt2/fUumDjY0NMTExKt9btmwZQ4cOfWWxX7oUCZdPVQSaIAiC8PLErVtBZ7xMtFhp27FjB23atOHcuXOMHj261M9f1LUuW7aMhISEUm9Tl6mKQBMEQRBenpjRE16L3bt3M336dAwNDenVqxc//PAD0dHRdOnShSFDhnD06FHee+89goKC8PDw4MyZM0De81pz5swBoEuXLkqzagMHDqR37964urri6uqKkZER169f5/bt21hZWfHzzz9jaGhIamoq48aNIyYmhlq1aqlddZrvp59+YunSpeTk5HD69Gm2bt2q9H5+koM28Wfx8fHY2Njw2WefERYWhouLi8qFFf7+/ty7d48hQ4ZQsWJFAgMD6du3L1FRUfK+cjNmzCAnJ4eFCxfSqFEjBg0axJEjR0hJSeGzzz5j6tSpAFy7dg0vLy8SExN5/vw5EyZMKLQ1S0Hvv/8+W7duxdbWFoCNGzeye/dudu3aVejYVx2BVlD+oDgrK0vnY4o0VV5il3SFqJfmRK20U17q9TL9EwM94ZVLTExkzJgx/Pnnn5ibm7NhwwYePXokv//o0SNOnz6NQqGQtwQ5f/48GRkZdOzYEXNzc4YMGVJsOzExMRw9ehQjIyMcHBwICQnh008/xd/fHyMjI65cucKTJ0+ws7OTt0JRxcXFhZs3byrFkBX0xRdf0KxZM3bu3CnHn7Vs2ZLWrVvTv39/pfizAQMGyM/XpaSkYGlpycKFC9W27evry48//sgvv/wi7703duxY1qxZwzfffMPz58/ZsGGD0jN2Dx48IDo6mkePHtG6dWs6dOhAu3bt+PTTTwkODsbc3JynT5/K192mTRuVbbu6urJx40Z5oLdhwwamTJmi8thvv/1WZQTarFa5VKqUo/b68u3fv1/+fWJiIocPH8bb21vp9Xz5KSCHDh3Sapua8iB/AYqgGVEvzYlaaUfX65W/H2pJiIGe8MqdOnWKFi1aYG5uDuQtOHBzc5Pfd3V1lZ/LCg8PZ8mSJejp6VG5cmVcXFwICwvTaKDXr18/OWe1bdu23LhxA4DDhw+zdOlSFAoFpqamDBs2TH6vJEoSf9agQQMqVKjAiBEjtG7P3d2dtm3bMmfOHLZv307btm1555135PfHjh2LQqGgZs2acl+qVavGxYsXGTp0qHxcamoqly5dUjvQc3FxoVWrVixZsoS7d+9y9epVPvroI5XHvuoItIIqV86bIezRo8cbs71KeYld0hWiXpoTtdJOeamXiEATyjVNo8UMDAzIyfn/2aJnz54pHVtwU199fX21z8GV9sP+msafVapUCT097R+LNTMzw8HBgV9++YU1a9YUuy+eQqFAkiRq1KihdsGHKg0aNMDW1pbdu3dz8eJFRowYoXLgBa8+Ag0gISGBhIQEee/BK1euUKVKFRo2bEiNGjU0bkOX6Xrskq4R9dKcqJV2dL1eIgJN0Gl2dnacP3+euLg4AIKDg8nMzFR5rKOjIz/88AOSJJGens7mzZvlrNamTZty+vRpAG7dusWJEyc0at/R0ZENGzYgSRJPnjxh27ZtL3U9rzL+DFRHoHl6ejJz5kySk5PliLN8+auHHz9+zK5du+jWrRvNmjWjatWqbNiwQT7u+vXrPH78uMi2R48ezY8//shPP/30WmLJiopACwwMpFWrVowfPx4ABwcHWrVqxZ49e155vwRBEN4UYqAnvHK1a9dm/fr19O3bFxsbG2JjYzExMVF5G2727NlUqFABa2tr2rVrR58+fRg8eDAA06ZN4+jRo1hbW+Pj41Pkc3YvnjMjIwNzc3OcnJzo2LHjS13PihUruHz5MtbW1nTt2lWOPzM0NGTnzp3MmTOHFi1a4OXlpXX8GYCHhwfjx49X2oLFzs4OU1NT3N3dC80g1qpViw8++IC2bdvy+eefY29vj4GBAXv37mXnzp20aNECS0tLxo4dS0ZGRpFtOzs7ExUVRZ06dbCwsNCq3yWRH4H2/vvvF3rPz88PSZIK/RIrcwVBELQgCcJr8OTJE/n3u3btkszNzcuwN+XPnTt3pLp16yrVUZIk6Z133pHOnTtXNp0qICUlRQKkhw8favyZO3fuSMOHD5dq1KghGRsbS1ZWVlJUVJT8fmpqqjRp0iTJzMxMMjY2liwsLKQ1a9a8iu6/dpmZmVJoaKiUmZlZ1l0pF0S9NCdqpZ3yUq/8n7EpKSlaf1Y8oye8FitXruSXX34hJyeHqlWrsmXLlrLuUrmRvxJ3wYIFVKlSpay7UyqSkpLo0KEDXbt25ffff6dWrVpcu3aN6tWry8d89dVXHDlyhODgYBo1asShQ4dwd3enfv369OnTpwx7LwiCUH6IgZ7wWsyYMUPnMkvXr1/PqlWrCr2+cuVKOnXq9Erb9vf3Z+fOnYVeDwkJ4d133y10rLoFGPkLFTSVmJgoP/NYUPfu3Vm8eLFW53oZCxcu5O2331Z6hrBx48ZKx5w8eZJRo0bRpUsXACZMmEBQUBBnzpwRAz1BEAQNiYHef4hCoSApKemN2aJCUxERETx79oxevXopvT5u3DjGjRtX4vPGx8dz4MABpa1iNOXr64uvr2+J235RcnIygYGBTJ8+XX7txQ2mIe95SW1W4r4qe/bsoWfPngwaNIg//vgDMzMz3N3d5YUXAPb29uzZs4cxY8ZQv359IiIiuHr1KkuXLi3DnguCIJQvYjGG8EYoKlIsIiKCAwcOlOizRYmPjy9xbmxpx70lJyezYMGCUj0nvLpYups3b7JmzRree+89Dh48yMSJE+VM4XwrV66kefPmNGjQQE5UWb16NQ4ODq+kT4IgCG8iMaP3H/P9998TGhrKv//+i6+vr5zjOmXKFP744w+ysrKoWrUq69ato1mzZmRkZODq6kpsbCwVKlSgTp06HDp0SO35169fz3fffYehoSE5OTmsX7+edu3a0ahRI0JDQ+WtRmxtbQkICKBLly506dIFa2trTp06RVJSEs7OzgQEBKBQKEhISMDDw4P4+HgyMjJwdnZm/vz5ADRq1EgpPs3Pzw9XV1fS0tLIzc3F2dmZgQMHEhgYSE5ODhEREfTv3x8XF5dCcWTJyclKSRirVq0iOjpa3rpk4cKFBAcHo6enR8WKFTly5Ahubm78888/2NjY0LBhQ/bs2VPsdbZo0YKoqCj5HAEBAfz6669kZ2dTu3ZtgoKClDZDfpG6iDU3NzdSU1OxsbHBwMCA6OhoAE6cOMGSJUu4d+8e3bt3lwemqampfPXVV/z99988e/YMOzs7Vq1ahaGhocp+vuhlItAu+PUkNzeXDz74QE7XsLKy4vz586xZs4Zhw4YBeZm/kZGR7Ny5k4YNG3LixAkmTZpE7dq16datW5Ft6LryErukK0S9NCdqpZ3yUi8RgSZozMjIiDNnznDlyhXatGnDyJEjMTAwwNvbm4CAAAB+/vlnPD09OXDgAAcOHCA5OZlLly4BFLsP2+TJk7ly5Qr16tUjKytLaTBQlEuXLnHy5EmysrJwcHBg27ZtDBs2jFGjRjFjxgw6d+5MdnY2vXv3Zvv27QwaNAhQjk/z9PSkd+/e+Pj4yH2tUaMGbm5uSoO4+Pj4QnFkfn5+avu2adMmQkJCOHHiBKampiQlJWFkZERgYCBeXl5a3Qq9evUqx44do0KFCmzdupW4uDgiIyPR19dn8+bNuLu7s2/fPpWfzczMVBuxFhgYqLQdS74bN25w9OhRsrKyaN68OZGRkbRv357JkyfTqVMn1q1bhyRJjB8/nuXLl8s5uQX7qcrLRKDt37+fatWqYWJiohR5lp2dzbVr19i/fz/Pnz9n1qxZTJ8+HT09Pe7cuUOjRo2ws7NjxowZcv5xeafrsUu6RtRLc6JW2tH1eokINEFjw4cPB8Dc3BwDAwMSEhJo0KABYWFhrFy5ktTUVHJzc+UBXcuWLbl8+TLu7u507twZJyenIs/frVs3Ro4cySeffMJHH32kcn80VVxcXOSdyUeMGEF4eDjOzs4cPnyYBw8eyMelpaXJGy+Dcnyag4MDU6dOJS0tjc6dOxfaWLggbeLI9u7di5ubG6ampgBKK0O1NWLECHnwFBoaSlRUFB988AGAUuqHKnFxcUVGrKkyZMgQDAwMMDAwwMbGhhs3btC+fXtCQ0OJjIzku+++AyAjIwN9fX2V/VTlZSPQPvzwQ+7cuaP05+nIkSO8//77ODk58eTJE7Kzs2nbtq3Ss5V79+4FKPbPoa4rL7FLukLUS3OiVtopL/USEWiCxlTFhP3vf//j888/JyoqinfffZfz58/Lz0E1adKES5cuceTIEcLDw5k2bRoxMTFqBzshISGcPXuWiIgInJycmD9/PkOHDi02vuxF+TFekJeVW7DfBRXcjHjAgAHY29sTFhbGqlWrWLZsmdKMUUEvxpFp2z91ijtPwf5KkoSPjw8TJkwoUVtQfJybulg4SZIICQlROxAvbpPnl41Amzx5Mvb29ixevJjBgwdz5swZ1q9fz9q1a6lQoQJvvfUWnTt3xsfHhypVqvDOO+/wxx9/EBwczHfffafTP5C1oeuxS7pG1Etzolba0fV6iQg04aWkpKRQoUIF6tWrhyRJSluO3LlzB4VCQZ8+fQgICECSJG7fvq3yPNnZ2dy4cQNbW1umTJnCwIEDOXPmDKAcX3bmzBmlWTnIi0XLysoiIyODrVu34ujoiImJCV27dlVaZHDv3j3u3Lmjsv1r165Rp04dXFxcWLRoEadOnQJUR4q9qGnTpkRHR5OTk8PTp08JCQmR3+vTpw+BgYHyOZKTk+X9AF88b3HXWVDfvn0JDAyUZ0+zsrI4d+6c2uOLilirWrUqGRkZaqPlVLW9cOFCeeCXlJTE9evXNfpsaWjTpg27du1i27ZtWFlZMW/ePJYtWybPOEPeIwRt2rRh+PDhNG/enAULFvD111+XaJWzIAjCf5WY0ROwtrZm6NChWFpa8tZbbyltxxEbG4uPjw+SJJGdnc3IkSNp0aKFyvPk5OQwZswYHj9+jIGBAbVq1ZL3SZs/fz6jRo0iKCiI9u3bY2lpqfRZCwsLOnTowOPHj3F2dmbo0KEAbNmyha+++gorKysUCgWVK1cmKChI5a3KHTt2EBwcjKGhIbm5ufLCg379+rF582ZsbGzkxRgv6t+/P9u3b8fCwoIGDRrQqlUr+ZmIkSNHcu/ePTlarHLlyoSHh8vRYlZWVjRp0oQ9e/YUe50FDR8+nEePHtG1a1cgb6A8ZswYWrVqpfL4/Ig1Dw8PJk+ejLGxsVLEmouLCy1atMDExERejKHO0qVLmT59OjY2Nujp6WFgYMCiRYto2rRpkZ8rTb1796Z3795q369bt67SPnuCIAiC9hRS/v0xQSgjqvZ7E8qXJ0+eYGpqysOHDzV6Ru/u3bt4e3vz+++/8/TpU5o2bcqGDRuwtbUlKyuLWbNmsX//fm7evImpqSmOjo4sWLCA+vXrv4arefWysrLYv38/Tk5OOn27SFeIemlO1Eo75aVe+T9jU1JSqFq1qlafFbduBUF4rfLjzypUqMDvv//OpUuXWLJkifzc59OnT/nrr7+YPXs2f/31Fzt37iQuLk6kYQiCIJSAuHUraG3//v0q48x8fHwYMmSI1ueLiIgohV69ObSJRyuPios/MzU1LbTVwapVq2jbti3/+9//aNiw4WvrqyAIQnknZvQErTk5ORETE1PoV0kGeQXt3r0bCwsLbGxsiI2NLaXeqlezZk2NsmL9/PyUVs/6+vqyZcuWV9InPz8/eWXzi7/ehEEe5MWf2draMmjQIGrXrk2rVq1Yt25dkZ9JSUlBoVD85+L7BEEQXpYY6Ak6IzAwEF9fX2JiYrC2ti7r7sjmzp2rNNDz9/dXWh36Ktt6E2kSf1bQs2fP8Pb25tNPP9X62RRBEIT/OnHrVtAJHh4eHD9+nCtXrrBy5UrmzJmDj48P2dnZVK9enTVr1tC8eXMiIiKU0iguXLhA7969iY+PJz4+HhsbGzw9Pdm7dy8pKSmsWLFC3lx3z549eHt7U6FCBaVNeEF9BFz+Vh6dOnVCX1+fQ4cOMW3aNGxsbPDy8iItLQ0PDw95G5lBgwbJqQ1dunTB1taW06dPF4ogU+XFtvbs2UObNm24desWlSpVAmDYsGF06tSJiRMnolAomDlzJvv27SM9PZ05c+bIA9CoqCi8vb158uQJOTk5zJgxQ04TUcXPz6/ICLgXlTQCTdP4s3xZWVkMHjyY3NxcVqxYofMxRZoqL7FLukLUS3OiVtopL/USEWhCubdixQrOnz+Pl5cX9vb2WFhYEBERgbW1NVu2bGHgwIFcvHix2POkpKTQokUL5s6dy4EDB/D09MTJyYnExERGjx7N8ePHad68OWvXruXRo0fy59RFwAUGBhIUFMTx48dV3jacN28ez58/5/z582RkZNCxY0fMzc3l29jqIshUUdWWo6MjwcHBTJgwgQcPHhAeHs7atWvlzygUCs6dO8fNmzextbWlQ4cOVKtWjQkTJrB//37q1avHw4cPad26Nfb29piZmWn6LSlSSSPQNIk/K/ja4sWLefDgAf7+/pw4caJU+q5LdD12SdeIemlO1Eo7ul4vEYEmvFFOnz6NtbW1fPt2+PDhTJo0ibt37xb7WWNjY/r37w9A+/btuXHjBpCXrtGiRQuaN28OwNixY/niiy/kz6mLgCtOeHg4S5YsQU9Pj8qVK+Pi4kJYWJg80FMXQaYpT09Pxo8fz4QJE1i3bh2ffvqpUmrFuHHjgLwEEwcHB44dO0bNmjW5efMmH330kdK54uLiSm2g9zIRaMXFn0He/14//fRTUlNT+fPPP6lVq1ap9FtXlJfYJV0h6qU5USvtlJd6iQg04T+juIgxIyMjORZMX19fbX5sweiwoiLgtPViJJm6CDJNtW3blkqVKnH06FHWrl1LeHh4se1LkoSlpSUnT57UuB1tI+BeJgKtuPiz/EHeX3/9xd69e9HT05NnX2vUqIGhoaHG16XrdD12SdeIemlO1Eo7ul4vEYEmvFHs7OyIjY3lwoULQN6tVDMzM8zMzGjSpAn//PMP//77LwCbN2/W6Jzt27fn/PnzXLlyBYAff/xRjgsrKgIOoEqVKmoj1BwdHfnhhx+QJIn09HQ2b95Mjx49SnTd6try9PTExcUFCwuLQtm0+VuUxMfHc/z4cTp16oS9vT23bt1SGhTGxMQUGY9WVARcaSsu/uzu3bvs2bOHO3fuYGNjQ7169eRf2gxeBUEQBDGjJ+igWrVqsWXLFlxcXOTFGNu3b0ehUFC/fn2mTZtG27ZtqVOnTqHbk0Wd88cff6Rfv34YGhrSq1cv+RZjURFwkDcD1b17dypVqsShQ4eU3ps9ezYeHh7ybeZBgwYxePDgEl/7i23Vrl2bgQMHMnHiRD7//PNCx+fk5NCqVSvS09NZsWIFjRo1AmDfvn1MmTKFyZMnk5WVRcOGDQkNDVXbblERcK9CUfFnjRo1QgT2CIIglA4RgSYIOi46Opphw4Zx5coV9PT+fxJeoVCQlJSkE3vLiQg07ZSX2CVdIeqlOVEr7ZSXer1MBJqY0RMEHTZu3DgOHTrE+vXrlQZ55Vl+BFrXrl35/fffqVWrFteuXVMZgdayZUuSkpLw9PSkT58+REdHl3HvBUEQyhcx0BOE18zNzY1Tp04Vej0yMpKKFSsqvbZ+/Xq159F2Mj4mJgZXV9dCr48aNYovv/xSq3O9DBGBJgiC8PqIgd5/0Ku45Wdra0tAQABdunQptXOq8vjxYz755BPS09MZNGgQM2fOfKXtvQovbpqclpZGlSpV5EGejY0Nx48fp0qVKixbtoyhQ4dSt25d+bOpqalMnTpV63ZtbGzkjabL0p49e+jZsyeDBg3ijz/+wMzMDHd3d8aPH6/2MyICTRAEoWTEQE8oN7KzswkLC8PExIQ///yzrLvzyhQcjC1btowuXbrIA7389IzyLD8C7auvvmLGjBlERUXh4eGBoaEho0aNKnS8iEATBEEoOTHQ+49TF/0FeTN/X3/9NaGhofz777/4+voyevRoAE6ePIm7uzvZ2dm0adOm2P3h/v33X4YPH879+/dRKBR88MEHbNiwgY0bNxIaGiqvCN27dy8BAQFEREQQERHBpEmTsLOz4+zZs8ycOZOpU6eSkpKCjY0NAQEBJCYmsnz5cjIzM8nNzWX+/Pl88sknQN4D/56ensTFxaFQKHB2dmbevHmkpqby1Vdf8ffff/Ps2TPs7OxYtWpVkfuz7du3Dz8/PzIzM1EoFAQFBdGuXTsOHjyoNqrt888/x8HBgT///JPs7Gw2bdqEra0tAEFBQQQEBGBiYiJv8Jwvf8Z1xYoV3Lt3jyFDhlCxYkW5VvlRZTk5OUyfPp3ff/8dgK5du7JkyRIMDQ1xdXXFyMiI69evc/v2baysrPj555+LvMbo6Gg8PDxIS0vD2NiYpUuX0qFDB5XHigi0l1NeYpd0haiX5kSttFNe6iUi0IQSUxf9lc/IyIgzZ85w5coV2rRpw8iRI8nNzWXIkCFs2LABR0dHDh06pDYTNV9wcDCNGzeWtyfRNHni8uXLfP/99/zwww8ApKenKw0MHz16xKeffopCoSA+Ph47Ozv++ecfjIyMGDFiBD169GDHjh0A8t57kydPplOnTqxbtw5Jkhg/fjzLly9Xezv06tWrjB49mmPHjmFubk5WVhZPnz4lMTGRYcOGqY1qu3LlCj/88APff/89gYGBzJw5k4MHD3LhwgXmzJnDuXPnqFevHjNmzFDZrq+vLz/++CO//PILNjY2AEpbpKxdu5aoqCjOnj2Lvr4+ffr0YenSpXh7ewN5M4NHjx7FyMgIBwcHQkJC+PTTT1W2lZmZSf/+/Vm3bh09e/bkxIkTDBgwgOvXryslceQTEWilQ9djl3SNqJfmRK20o+v1EhFoQokVF/2Vv4mtubk5BgYGJCQk8PjxYwwMDHB0dASgR48eNGnSpMh27OzsWLp0KZMnT8bBwYFevXpp1L8mTZrQuXNnte/funWL4cOHc+fOHQwMDHj8+DG3bt2iQYMGnDhxgoMHD8rH5sdohYaGEhkZyXfffQdARkYG+vr6atsICwujV69emJubA3k7lJuamvLbb78VGdXWtGlT2rVrB+Rt2Jw/oD5y5AgfffQR9erVA2DixIl8++23GtWjoPDwcHnmDmD8+PGsXr1aHuj169ePSpUqAXkJG/lxcKrExcWhp6dHz549AejYsSN16tQhJiaGjh07FjpeRKC9nPISu6QrRL00J2qlnfJSLxGBJpSIJtFfmkZ4vRj99aL27dsTExNDeHg4O3fuZPbs2Zw7d67Y6C1Vs0kFDR06lAULFjBw4EAgLyKruPguSZIICQkplDJR2kqrdpoq7fi1ovolItBKh67HLukaUS/NiVppR9frJSLQhBIpLvpLHXNzc7Kzszl69CiQN7NU1GwR5M28mZiYMHjwYFauXMnVq1dJS0ujadOmnD9/noyMDLKzs9m6datW15CUlCRvzREcHExSUhKQN0B0cHBgyZIl8rH5t2779u3LwoUL5YFPUlIS169fV9tGz549OXjwoByflpWVRUpKSpFRbUX58MMPOXDgAAkJCUDhVbgFVa1atcj4tZ9++onMzEyys7NZv359iePXmjVrRm5urnz74uTJkyQkJMi3jEuTiEATBEF4fcSM3n9YcdFf6hgaGvLLL7/g7u5OTk4Obdq0oWXLlkV+JiIigu+++06eWVq8eDGmpqbY2dnh5OSElZUV9erVo0OHDpw+fVrja1i+fDkDBw6kWrVqfPjhh0p7rG3evJkvvvgCS0tLKlSogLOzM3PnzmXp0qVMnz4dGxsb9PT0MDAwYNGiRTRt2lRlG/mpDSNGjCArKwt9fX0CAwNp27at2qi2olhZWeHn50enTp1ULsYoyMPDg/Hjx1OpUqVCz0FOmDCBGzdu0Lp1awC6dOmCl5eXZoV7gaGhITt37sTDw4PJkydjbGzMjh07ip1RLSkRgSYIgvB6iAg0QRBemrYRaAALFizAx8cHT09Pli1bBkBCQgJTp04lLCyM1NRUmjVrxsyZMxkwYMAr7P3rV15il3SFqJfmRK20U17q9TIRaOLWrSAIr11UVBRBQUG0aNFC6XUXFxfi4uLYs2cPsbGx9O/fn8GDB3Pu3Lky6qkgCEL5Jm7dCqWqT58+/O9//1N6rXr16vLzfLqsPPddU/7+/uzcubPQ6yEhIbz77ruvpQ9paWkMHz6cdevWMX/+fKX3Tp48yZo1a2jbti0As2bNYunSpZw9e5ZWrVq9lv4JgiC8ScRATwOvIjKsPIiIiODZs2cab4UCefFWxYmPj+fAgQM6l/KgSd9Lk6+vL82aNZMXIbyuNn19fYs9ruDG1aVt0qRJfPzxxzg6OhYa6Nnb2/PLL7/w8ccfU61aNX799VeePXv2yqP1BEEQ3lRioPcfl52djYGB6j8GERERJCcnqx3oFfXZosTHxxMYGFiigV5J2yztc5QGf3//19aWrlzzzz//zF9//UVUVJTK93/99VeGDBnCW2+9hYGBAZUqVWLXrl1qF8oIgiAIRSv7n/zlxPfff68yCkxdhFhGRgaurq7ExsZSoUIF6tSpI6dCqLJ+/Xq+++47DA0NycnJYf369bRr145GjRoRGhoqb3Nha2tLQEAAXbp0oUuXLlhbW3Pq1CmSkpJwdnYmICAAhUJBQkICHh4exMfHk5GRgbOzszx70qhRI4YMGcLRo0d577338PPzw9XVlbS0NHJzc3F2dmbgwIEEBgaSk5NDREQE/fv3x8XFBRsbGz777DPCwsJwcXEhOTlZjuQCWLVqFdHR0fIK0YULFxIcHIyenh4VK1bkyJEjuLm58c8//2BjY0PDhg3Zs2dPsdfZokULoqKi5HMEBATw66+/kp2dTe3atQkKCuKdd95RW98Xr3nDhg3MnDmT33//HX19ferVq6eUCPKi4urp4uJCWFgYCQkJjB07llmzZgF56RijR4/myZMnNGvWjLS0NIYNG4arqyuurq7Y2Njg5eWFn58fly9f5unTp9y4cYO6deuyY8cOatSoAaD2erOyspg9ezZHjhwhMzOT999/n6CgIKpXr46rqyt6enpcv36dxMRErly5wubNm1m1ahVZWVmYmJiwcuVKWrZsSVZWFp6enoSFhVG9enU6depU5N8HbSPQLvj15Pbt23h6erJ//3709fXJyspCkiRyc3PleJ+ZM2eSlJTEgQMHeOutt9izZw+DBw/myJEj8sbUb4LyErukK0S9NCdqpZ3yUi8RgfYaqIoCMzAwUBshduDAAZKTk7l06RJQfOTX5MmTuXLlCvXq1SMrK0vpH9GiXLp0iZMnT5KVlYWDgwPbtm1j2LBhjBo1ihkzZtC5c2eys7Pp3bs327dvZ9CgQUBedNjp06dRKBR4enrSu3dvfHx85L7WqFEDNzc3pUFcfHw8KSkpWFpasnDhQgD8/PzU9m3Tpk2EhIRw4sQJTE1NSUpKwsjIiMDAQLy8vIiJidHoGiEvhuzYsWNUqFCBrVu3EhcXR2RkJPr6+mzevBl3d3f27dtX5DkKXvPcuXO5evUqZ8+excjISN5jT53i6pmcnExkZCQPHz7k3XffZfTo0ZiZmTFy5Ejc3d0ZPXo0ly9fplWrVoXyXPOdPn2as2fP8tZbbzF06FCCgoLw8fEp8noXL15M5cqVOXPmDADz5s1j1qxZrF69GoCzZ89y4sQJqlSpwp9//sm2bds4duwYRkZGHD9+nGHDhnHx4kXWrl1LXFycHN+Wn5ChjrYRaPv37+fUqVMkJibKz98B5Obmcvz4cVavXs3q1av5/vvvWbFiBc+ePePu3bt88MEHvPPOO8yYMYOJEycW2afySNdjl3SNqJfmRK20o+v1EhFor4GqKLAGDRqojRBr2bIlly9fxt3dnc6dOyvFPanSrVs3Ro4cySeffMJHH32kcWqDi4uLvKP3iBEjCA8Px9nZmcOHD/PgwQP5uLS0NOLi4uSvXV1d5f3eHBwcmDp1KmlpaXTu3FmONlMlvx1N7N27Fzc3N0xNTYG8hQ0lNWLECHnpe2hoKFFRUXzwwQcASskaRSl4zXv37mXhwoVyukNREVvp6enF1jN/8FazZk2aNGnCrVu3qFKlCjExMbi4uABgYWGhMk4sX69eveStSdq3b09sbGyx1xsaGkpKSgohISFAXmZto0aN5PcHDRpElSpVANi9ezd///23HMsGeYP6jIwMDh8+jIuLi5w6MWbMGDlfWJWSRKB16tSJwYMHK702fvx4mjVrxpQpU+S98zp37oyFhYV8zOrVq2nQoEGxf4fKk/ISu6QrRL00J2qlnfJSLxGB9hqoipMqKkKsSZMmXLp0iSNHjhAeHs60adOIiYlRO9gJCQnh7NmzRERE4OTkJd1AZwAAJKlJREFUxPz58xk6dGixEWEvUigU8j+Yp06dUup3QQU3wh0wYAD29vaEhYWxatUqli1bphQuX1ClSpXQ0/v/XXm07Z862kShSZKEj48PEyZM0KqNkm7+q0k9SyPuTN05irpeSZJYuXKl2kSMF+s2atQovvnmG7V90KSfULIItBo1asi3ogv2r1atWrRq1YqsrCyaNm3K559/TkBAAG+99RahoaGEh4ezd+9enf4hXFK6Hruka0S9NCdqpR1dr5eIQCsjRUWI3blzB4VCQZ8+fQgICECSJG7fvq3yPNnZ2dy4cQNbW1umTJnCwIED5VtxTZs2lZMizpw5ozSLBHmxX1lZWWRkZLB161YcHR0xMTGha9euLFiwQD7u3r173LlzR2X7165do06dOri4uLBo0SJOnToFFB2/la9p06ZER0eTk5PD06dP5ZklyNuuJDAwUD5HcnIyOTk5Ks9b3HUW1LdvXwIDA+XZ06ysLK33WevTpw/Lly+Xb5EXdetW23rmq1q1Ki1btiQ4OBiAuLg4Tpw4oVU/oejr7du3L0uXLpWn9Z8+fSrffn1Rnz59CA4OlreQyc3NJTo6GsiLU8v/s5SZmcmGDRu07ufLqlChAvv376dWrVp88skntGjRgp9++olNmza9UbN5giAIr5OY0XsJRUWIxcbG4uPjgyRJZGdnM3LkyEKbw+bLyclhzJgxPH78GAMDA2rVqiX/Qzt//nxGjRpFUFAQ7du3x9LSUumzFhYWdOjQgcePH+Ps7MzQoUMB2LJlC1999RVWVlYoFAoqV65MUFAQDRo0KNT+jh07CA4OxtDQkNzcXDl7tV+/fmzevBkbGxt5McaL+vfvz/bt27GwsKBBgwa0atVKHnSMHDmSe/fuYW9vj4GBAZUrVyY8PJwWLVpgaWmJlZUVTZo0Yc+ePcVeZ0HDhw/n0aNHdO3aFcgbKI8ZM0arfda8vb2ZOXMmrVu3pkKFCtSvX1/tLKa29Szop59+YsyYMSxevJimTZvSpk0brbfpKep6vb29ef78Oe3atZNn4by9vVXWr1OnTixatIh+/fqRnZ1NZmYmH3/8Mba2towfP54LFy7QvHlzeTHG2bNntepnSby4fct7772n9J8FQRAE4SVJQrnVuXNnadeuXWXdDaEIqampUm5uriRJknTz5k2pTp060v/+978y7lXpS0lJkQDp4cOHGn/m22+/lQDJ09NT6fWTJ09KXbt2lSpVqiRVqVJF6tSpk/T06dNS7nHZyszMlEJDQ6XMzMyy7kq5IOqlOVEr7ZSXeuX/jE1JSdH6s2JGTxBeoZMnTzJ16lQgb+Z26dKlvP3222Xcq7KnLgItMjKSXr164ePjw8qVKzEwMODvv/9Wei5UEARB0JwY6L1G+/fvZ8aMGYVe9/HxYciQIVqf71WkFpRnLxvvtX79eqXnLPOtXLmy2H3l1OnRo4fahRL/VUVFoH355Zd4eHgwffp0+bVmzZq97i4KgiC8McR/k18jJycnYmJiCv0qySCvvFIoFCQnJ7+Sc/v6+qqsr6YZruPGjVP6nLOzM8+ePWPKlClER0e/9u9TTEwMP//8s9JrNjY2pKamvtZ+lLaCEWgFJSYmcvr0aWrXro29vT116tShc+fOJVrAIgiCIOQRM3qCoMaiRYu4efMm9erVA+CXX34p1fMXF0sWExNDaGiovMAm/7XyrKgItJs3bwJ5m3AHBARgY2PDTz/9RLdu3bhw4QLvvffe6+6uIAhCuScGekKpUygUzJw5k3379pGens6cOXPkDacLUhcfl3+Or7/+WmXsnDobNmxg+fLlSJJEhQoV2LFjB40aNWLz5s0sXrwYgLfffpu1a9diZmbGxo0bCQ4OplatWly4cAEjIyN+/fVXmjRpgr29Pc+ePaNHjx507dqV/v37K6V5BAUFsWTJEkxMTOjXrx++vr7yfnsKhYKkpCR5dW3NmjWJjo6mUaNGhaLYlixZwqeffsqTJ0949uwZXbt2ZcWKFTx8+BBfX19SUlKwsbHBzs6OwMBApXNHR0fj4eFBWloaxsbGLF26lA4dOhAfH4+NjQ2enp7s3buXlJQUVqxYUeQWJRs3biQ0NJTQ0FAgb0PpgIAAtY8HvIoItMzMTCBvZjV/U+5FixYRHh7OunXr+Prrr4v8/pcn5SV2SVeIemlO1Eo75aVeIgJN0DkKhYJz585x8+ZNbG1t6dChg1JiA6A2Pi6futg5VSIiIvD39+fkyZPUq1dP3uLlwoULTJ06lbNnz2JmZsbXX3/NuHHj+P3334G8RQExMTE0btyY6dOns3DhQoKCgjh58iQKhYLjx49TrVo1pQHPhQsX8PPz49y5c9StW5c5c+ZoVZuCUWzPnj3jt99+w8TEhJycHJydnfn1118ZOnQo/v7+SoOvgjIzM+nfvz/r1q2jZ8+enDhxggEDBnD9+nUgb4/HFi1aMHfuXA4cOICnp2ep7kX3qiLQ8q+t4FY3pqamnD59usjtb8orXY9d0jWiXpoTtdKOrtdLRKAJOmfcuHFAXkKIg4MDx44dKzTQUxcfl09d7Jwq+/btY+TIkfJt1kqVKgFw9OhRevXqhZmZGQDu7u74+/vLKRzt27encePG8u9XrlxZ7LUdOXKEXr16UbduXSAvxsvf37/Yz+UrGMWWm5uLt7c3J06cQJIkEhMTsbKyUrpdq0pcXBx6enpyJm3Hjh2pU6cOMTExNGjQAGNjY/r37y9f140bNzTunyZeRQSapaUl8+bNo2LFikqD0jlz5tCzZ883atPk8hK7pCtEvTQnaqWd8lIvEYEm6LwXI7WKio/Lp2ms2Mv0ozTaePGc+vr6Gse5fffdd/IiBGNjY7766qsSx8gV7IeRkZH89Yv9UUXbKLtXEYEGMHXqVObMmUPr1q2xsbFh06ZNxMXFERISotM/hEtK12OXdI2ol+ZErbSj6/USEWiCzslP9oiPj+f48eOFticpKj6uJD755BOCg4O5f/8+kDfN/fTpU7p27cqBAwe4d+8eAIGBgXTr1g19ff0St9W1a1cOHjxIYmIiAD/88IPS+wXj3Hbu3El6erracyUlJVG3bl2MjY1JSEhg+/bt8ntFRdA1a9aM3Nxc+XbDyZMnSUhIwMbGpkTX1LRpU86fP09GRgbZ2dls3bq1ROd5WV5eXvj4+PDll1/SsmVLDh8+TFhYmMYrpwVBEARlYkZPeCVycnJo1aoV6enprFixotBt26Li40rCwcFBvsWnUCgwNDRkx44dWFlZsXjxYnr16gXkLcZYt27dS7VlbW3NrFmz6NChA1WqVKFXr16YmprK7y9duhQPDw9mzZrFxx9/rPZWJoCnpycDBw7E0tKS+vXrK2050q1bNwICAmjRogX29vZyNB2AoaEhO3fuxMPDg8mTJ2NsbMyOHTswMTHh4cOHWl+TnZ0dTk5OWFlZUa9ePTp06CAPVl8lVYs9pk+frrSPniAIglByCil/qaAglJIXV52+iVJTU6lSpQoAy5cv58CBA/ICj/+iJ0+eYGpqysOHD1UObNesWcOaNWuIj48HwNLSEl9fXz766CMAbty4wZQpUzhx4gTPnz+nV69erFy5kjp16rzOy3htsrKy2L9/P05OTjp9u0hXiHppTtRKO+WlXvk/Y1NSUqhatapWnxW3bgWhBKZPn46NjQ2Wlpbs3btXXjEqqNagQQMWLFjA2bNniY6O5sMPP8TZ2ZmLFy+Snp5Ojx49UCgUHDlyhD///JPMzEw++eQTcnNzy7rrgiAI5Zq4dSuUulc5SWxra1towYSlpSVbtmx5ZW2qUh4HdomJiSrj2Lp37y7vM/iqfPLJJ0pff/3116xZs4ZTp05x9+5d4uPjOXfunPw/1U2bNlG9enWOHDlSKEFDEARB0JwY6Ak6Y/fu3UyfPh0jIyM2b96MtbV1oWOio6NLrb2CGxkXxc/Pj+nTp8srdH19fWnWrJnKTaBf1ottlabatWurTdbw8/MjOTmZZcuWlXq7L8rJyWH79u2kp6fLW78oFAqlVbzGxsbo6elx4sQJMdATBEF4CeLWraAzAgMD5bxaVYO8sjJ37lyl7Ub8/f1fySBPVVtvktjYWExMTDAyMsLNzY1du3bRvHlz7OzsqFy5Mt7e3jx9+pT09HSmTJlCTk6OvIpaEARBKBkxoyfoBA8PD44fP86VK1dYuXIlc+bMwcfHh+zsbKpXr86aNWto3rw5ERERSlFkFy5coHfv3sTHxxcb/bVnzx68vb2pUKGCvAo3n7o4Njc3NyBvs199fX0OHTrEtGnTsLGxwcvLi7S0NDw8PDhz5gwAgwYNkpMyunTpgq2tLadPn+bevXt0795daeXsi15sa8+ePbRp04Zbt27JG0APGzaMTp06MXHixCKj5qKiovD29ubJkyfk5OQwY8YMBg0aVOT34P79+3zyySfcuHGDunXrsmPHjkL73uXTJgLtgl/eps5NmjQhKiqKJ0+eEBISwqhRowgPD6d58+Zs27aNL774ghUrVqCnp8eQIUPkvfV0PZqoJMpL7JKuEPXSnKiVdspLvUQEmlDurVixgvPnz+Pl5YW9vT0WFhZERERgbW3Nli1bGDhwIBcvXiz2POqivxITExk9ejTHjx+nefPmrF27lkePHsmfUxfHFhgYSFBQkByF9qJ58+bx/PlzeQ+6jh07Ym5uzpAhQ4C81aRHjx4lKyuL5s2bExkZSfv27VX2XVVbjo6OBAcHM2HCBB48eEB4eDhr166VP6Mqaq5atWpMmDCB/fv3U69ePR4+fEjr1q2xt7eXE0JUOX36NGfPnuWtt95i6NChBAUF4ePjo/JYbSLQVEWXdejQgYMHDzJt2jTc3d2BvM2jnzx5gp6eHiYmJri6utKiRYs3Mvosn67HLukaUS/NiVppR9frJSLQhDfK6dOnsba2lm/fDh8+nEmTJnH37t1iP6su+uvUqVO0aNGC5s2bAzB27Fi++OIL+XPFxbGpEx4ezpIlS9DT06Ny5cq4uLgQFhYmD/SGDBmCgYEBBgYG2NjYcOPGDbUDPVU8PT0ZP348EyZMYN26dXz66adKyRqqouZq1qzJzZs35a1L8sXFxRU50OvVq5e8NUr79u2JjY1Ve2xJItBetGzZMurUqaMy2uzo0aOkpKQwZcoUmjVrptH5ypPyErukK0S9NCdqpZ3yUi8RgSb8ZxQX1aVp9FfBuDBN4tg0VdoRa23btqVSpUocPXqUtWvXEh4eXmz7kiRhaWnJyZMntWpLm75qG4Hm4+PDRx99RMOGDUlNTWXr1q388ccfHDx4kAoVKrBhwwYsLCyoVasWkZGReHp68uWXX2JlZaXVNZQ3uh67pGtEvTQnaqUdXa+XiEAT3ih2dnbExsZy4cIFIO9WqpmZGWZmZjRp0oR//vmHf//9F4DNmzdrdM727dtz/vx5rly5AsCPP/5IZmYmUHwcW5UqVdRGkTk6OvLDDz8gSRLp6els3rxZ5RYmmlLVlqenJy4uLlhYWPD+++8rvacqas7e3p5bt24pDQpjYmLk6y0LiYmJuLi40KxZM7p160ZUVBQHDx6ke/fuQN5sY9++fbGwsMDf35+ZM2fKt9IFQRCEkhMzeoLOqVWrFlu2bMHFxUVejLF9+3YUCgX169dn2rRptG3bljp16hS6PVnUOX/88Uf69euHoaGh0m3K4uLYJk+eTPfu3alUqRKHDh1Sem/27Nl4eHjIt5kHDRrE4MGDS3ztL7ZVu3ZtBg4cyMSJE/n8888LHa8uam7fvn1MmTKFyZMnk5WVRcOGDQkNDS1xv17Wi3nAL1qwYAELFix4Tb0RBEH47xARaIKg46Kjoxk2bBhXrlxBT+//J+F1KWquuAg0QVl5iV3SFaJemhO10k55qdfLRKCJGT1B0GHjxo3j0KFDrF+/XmmQJwiCIAiaEAM9QXjN3NzcOHXqVKHXIyMjqVixotJr69evV3sebSfjY2JicHV1LfT6qFGj+PLLL7U6lyAIglA+iIGeILxmRW2a/CrZ2NiojUATBEEQ3kziXpAgCIIgCMIbSszoCYLw0vJvI6empur0A826Iisri6dPn/LkyRNRLw2IemlO1Eo75aVe+Rsml2T9rBjoCYLw0vLj5Bo3blzGPREEQXhzpaamYmpqqtVnxEBPEISXVqNGDSAvZUTbH0L/RfmRcbdv39Z6q4T/IlEvzYlaaae81EuSJFJTU6lfv77WnxUDPUEQXlr+1i+mpqY6/cNS11StWlXUSwuiXpoTtdJOeahXSf8TLRZjCIIgCIIgvKHEQE8QBEEQBOENJQZ6giC8NCMjI+bMmYORkVFZd6VcEPXSjqiX5kSttPNfqJfIuhUEQRAEQXhDiRk9QRAEQRCEN5QY6AmCIAiCILyhxEBPEARBEAThDSUGeoIgvJTVq1fTqFEjjI2NadeuHWfOnCnrLpUJPz8/FAqF0i9zc3P5/WfPnjFp0iTeeustTExMGDBgAA8ePFA6x//+9z8+/vhjKlWqRO3atZk6dSrZ2dmv+1JeiWPHjvHJJ59Qv359FAoFoaGhSu9LkoSvry/16tWjYsWKODo6cu3aNaVjHj9+zPDhw6latSrVqlVj7NixpKWlKR1z/vx5OnXqhLGxMW+//TaLFi161ZdW6oqrlaura6E/a7169VI65r9SK4Bvv/2WNm3aUKVKFWrXrk3fvn2Ji4tTOqa0/v5FRETQunVrjIyMaNq0KRs3bnzVl/fSxEBPEIQS++WXX/jqq6+YM2cOf/31Fy1btqRnz54kJiaWddfKhKWlJffv35d/nThxQn7vyy+/5LfffmP79u388ccf3Lt3j/79+8vv5+Tk8PHHH5OZmcnJkyfZtGkTGzduxNfXtywupdSlp6fTsmVLVq9erfL9RYsWsWLFCgIDAzl9+jSVK1emZ8+ePHv2TD5m+PDhXLx4kbCwMPbu3cuxY8eYMGGC/P6TJ0/o0aMH77zzDmfPnmXx4sX4+fmxdu3aV359pam4WgH06tVL6c/atm3blN7/r9QK4I8//mDSpEmcOnWKsLAwsrKy6NGjB+np6fIxpfH379atW3z88cd07dqVmJgYvLy8GDduHAcPHnyt16s1SRAEoYTatm0rTZo0Sf46JydHql+/vvTtt9+WYa/Kxpw5c6SWLVuqfC85OVmqUKGCtH37dvm1y5cvS4AUGRkpSZIk7d+/X9LT05MSEhLkY9asWSNVrVpVev78+Svt++sGSLt27ZK/zs3NlerWrSstXrxYfi05OVkyMjKStm3bJkmSJF26dEkCpKioKPmY33//XVIoFNLdu3clSZKk77//XqpevbpSvby9vaVmzZq94it6dV6slSRJ0qhRoyRnZ2e1n/mv1ipfYmKiBEh//PGHJEml9/dv2rRpkqWlpVJbQ4YMkXr27PmqL+mliBk9QRBKJDMzk7Nnz+Lo6Ci/pqenh6OjI5GRkWXYs7Jz7do16tevT5MmTRg+fDj/+9//ADh79ixZWVlKtTI3N6dhw4ZyrSIjI7G2tqZOnTryMT179uTJkydcvHjx9V7Ia3br1i0SEhKU6mNqakq7du2U6lOtWjVsbW3lYxwdHdHT0+P06dPyMQ4ODhgaGsrH9OzZk7i4OJKSkl7T1bweERER1K5dm2bNmjFx4kQePXokv/dfr1VKSgrw/xncpfX3LzIyUukc+cfo+s87MdATBKFEHj58SE5OjtIPRoA6deqQkJBQRr0qO+3atWPjxo0cOHCANWvWcOvWLTp16kRqaioJCQkYGhpSrVo1pc8UrFVCQoLKWua/9ybLv76i/iwlJCRQu3ZtpfcNDAyoUaPGf66GvXr14qeffvq/9u49KMrq/wP4e8PdZRdnWW6xq+UCioADeA1btNQ0QUezGs2YLug4OGJWlDrlH5qkpt8/vJR5SZ1gNBszQS1rmBRYHVdlBgdNijZBkC47aQYKoXLZ9++PhufXE5fJItDl85rZmd1zznOez/nowoeH5yzIz8/H//73Pxw/fhxTpkxBS0sLgN6dK4/Hg4yMDIwZMwaxsbEA0GXvv47G3LhxAzdv3vwvltMl+vR0AEII4Q2mTJmiPI+Pj8fo0aNhs9mwf/9+GAyGHoxMeJtnn31WeR4XF4f4+HgMHDgQDocDEydO7MHIet5LL72E0tJS1f2xvZ1c0RNC/CPBwcHw8fFps3Ptl19+gcVi6aGo7h5msxmDBw9GeXk5LBYLGhsbUVtbqxrz51xZLJZ2c9na581a19fZ/yWLxdJmk09zczN+++23Xp/DiIgIBAcHo7y8HEDvzdWiRYtw5MgRFBYW4oEHHlDau+r919EYk8l0V/8wJ4WeEOIf0el0GDlyJPLz85U2j8eD/Px82O32Hozs7lBfX4+KigpYrVaMHDkSWq1WlSuXy4Xq6molV3a7HRcuXFB9gz569ChMJhOGDBnS7fF3p/DwcFgsFlV+bty4gaKiIlV+amtrcfbsWWVMQUEBPB4PRo8erYw5ceIEmpqalDFHjx5FVFQUAgICumk13e/HH3/EtWvXYLVaAfS+XJHEokWLcPDgQRQUFCA8PFzV31XvP7vdrpqjdcxd//Wup3eDCCHuXfv27aNer2d2dja//fZbzp8/n2azWbVzrbdYvHgxHQ4HKysr6XQ6OWnSJAYHB/PKlSskyQULFnDAgAEsKChgcXEx7XY77Xa7cnxzczNjY2M5efJknjt3jnl5eQwJCeGyZct6akldqq6ujiUlJSwpKSEAbtiwgSUlJbx8+TJJct26dTSbzTx8+DC//vprzpgxg+Hh4bx586YyR3JyMocPH86ioiKePHmSkZGRTElJUfpra2sZGhrKF154gaWlpdy3bx+NRiM/+OCDbl/vv9FZrurq6rhkyRKePn2alZWVPHbsGEeMGMHIyEjeunVLmaO35Iok09PT6e/vT4fDQbfbrTwaGhqUMV3x/rt06RKNRiOXLl3KsrIybtmyhT4+PszLy+vW9d4pKfSEEP/K5s2bOWDAAOp0OiYkJPDMmTM9HVKPmD17Nq1WK3U6Hfv378/Zs2ezvLxc6b958yYXLlzIgIAAGo1GPvXUU3S73ao5qqqqOGXKFBoMBgYHB3Px4sVsamrq7qX8JwoLCwmgzSM1NZXkHx+xsnz5coaGhlKv13PixIl0uVyqOa5du8aUlBT27duXJpOJc+fOZV1dnWrM+fPnOXbsWOr1evbv35/r1q3rriV2mc5y1dDQwMmTJzMkJIRarZY2m41paWltfrjqLbki2W6uADArK0sZ01Xvv8LCQg4bNow6nY4RERGqc9ytNCTZ3VcRhRBCCCHEf0/u0RNCCCGE8FJS6AkhhBBCeCkp9IQQQgghvJQUekIIIYQQXkoKPSGEEEIILyWFnhBCCCGEl5JCTwghhBDCS0mhJ4QQQgjhpaTQE0II4TXGjx+PjIyMng5DiLuGFHpCCNFLzJkzBxqNps2jvLy8S+bPzs6G2Wzukrn+qdzcXKxatapHY+iMw+GARqNBbW1tT4cieok+PR2AEEKI7pOcnIysrCxVW0hISA9F07GmpiZotdo7Pi4wMPA/iKZrNDU19XQIoheSK3pCCNGL6PV6WCwW1cPHxwcAcPjwYYwYMQK+vr6IiIhAZmYmmpublWM3bNiAuLg4+Pn54cEHH8TChQtRX18P4I8rVXPnzsX169eVK4UrV64EAGg0Ghw6dEgVh9lsRnZ2NgCgqqoKGo0Gn3zyCcaNGwdfX1/s3bsXALBr1y7ExMTA19cX0dHR2Lp1a6fr++uvbsPCwrB69Wq8+OKL6Nu3L2w2Gz777DNcvXoVM2bMQN++fREfH4/i4mLlmNYrk4cOHUJkZCR8fX2RlJSEH374QXWubdu2YeDAgdDpdIiKisKePXtU/RqNBtu2bcMTTzwBPz8/pKWlYcKECQCAgIAAaDQazJkzBwCQl5eHsWPHwmw2IygoCNOmTUNFRYUyV2uOcnNzMWHCBBiNRgwdOhSnT59WndPpdGL8+PEwGo0ICAhAUlISampqAAAejwdr165FeHg4DAYDhg4digMHDnSaT+EFKIQQoldITU3ljBkz2u07ceIETSYTs7OzWVFRwa+++ophYWFcuXKlMmbjxo0sKChgZWUl8/PzGRUVxfT0dJLk7du3uWnTJppMJrrdbrrdbtbV1ZEkAfDgwYOq8/n7+zMrK4skWVlZSQAMCwtjTk4OL126xJ9//pkfffQRrVar0paTk8PAwEBmZ2d3uMZx48bx1VdfVV7bbDYGBgZy+/bt/P7775menk6TycTk5GTu37+fLpeLTz75JGNiYujxeEiSWVlZ1Gq1HDVqFE+dOsXi4mImJCQwMTFRmTc3N5darZZbtmyhy+Xi+vXr6ePjw4KCAmUMAN5///388MMPWVFRwaqqKubk5BAAXS4X3W43a2trSZIHDhxgTk4OL168yJKSEk6fPp1xcXFsaWlR5Sg6OppHjhyhy+XizJkzabPZ2NTURJIsKSmhXq9neno6z507x9LSUm7evJlXr14lSa5evZrR0dHMy8tjRUUFs7KyqNfr6XA4OsynuPdJoSeEEL1EamoqfXx86OfnpzxmzpxJkpw4cSLfeecd1fg9e/bQarV2ON+nn37KoKAg5XVWVhb9/f3bjPu7hd6mTZtUYwYOHMiPP/5Y1bZq1Sra7fYOY2qv0Hv++eeV1263mwC4fPlype306dMEQLfbrawDAM+cOaOMKSsrIwAWFRWRJBMTE5mWlqY696xZszh16lTVujMyMlRjCgsLCYA1NTUdroEkr169SgC8cOECyf/P0a5du5Qx33zzDQGwrKyMJJmSksIxY8a0O9+tW7doNBp56tQpVfu8efOYkpLSaSzi3ib36AkhRC8yYcIEbNu2TXnt5+cHADh//jycTifWrFmj9LW0tODWrVtoaGiA0WjEsWPHsHbtWnz33Xe4ceMGmpubVf3/1qhRo5Tnv//+OyoqKjBv3jykpaUp7c3NzfD397+jeePj45XnoaGhAIC4uLg2bVeuXIHFYgEA9OnTBw899JAyJjo6GmazGWVlZUhISEBZWRnmz5+vOs+YMWPw7rvvdrimzly8eBErVqxAUVERfv31V3g8HgBAdXU1YmNj212L1WpV4o6Ojsa5c+cwa9asducvLy9HQ0MDHn/8cVV7Y2Mjhg8f/rdiFPcmKfSEEKIX8fPzw6BBg9q019fXIzMzE08//XSbPl9fX1RVVWHatGlIT0/HmjVrEBgYiJMnT2LevHlobGzstNDTaDQgqWprb2NCa9HZGg8A7Ny5E6NHj1aNa72n8O/686YOjUbTYVtrcdWV/rymzkyfPh02mw07d+5Ev3794PF4EBsbi8bGRtW4zuI2GAwdzt+azy+++AL9+/dX9en1+r8Vo7g3SaEnhBACI0aMgMvlarcIBICzZ8/C4/Fg/fr1uO++P/bx7d+/XzVGp9OhpaWlzbEhISFwu93K64sXL6KhoaHTeEJDQ9GvXz9cunQJzz333J0u519rbm5GcXExEhISAAAulwu1tbWIiYkBAMTExMDpdCI1NVU5xul0YsiQIZ3Oq9PpAECVp2vXrsHlcmHnzp145JFHAAAnT56845jj4+ORn5+PzMzMNn1DhgyBXq9HdXU1xo0bd8dzi3uXFHpCCCGwYsUKTJs2DQMGDMDMmTNx33334fz58ygtLcXq1asxaNAgNDU1YfPmzZg+fTqcTie2b9+umiMsLAz19fXIz8/H0KFDYTQaYTQa8dhjj+H999+H3W5HS0sL3njjjb/10SmZmZl45ZVX4O/vj+TkZNy+fRvFxcWoqanB66+//l+lAsAfV85efvllvPfee+jTpw8WLVqEhx9+WCn8li5dimeeeQbDhw/HpEmT8PnnnyM3NxfHjh3rdF6bzQaNRoMjR45g6tSpMBgMCAgIQFBQEHbs2AGr1Yrq6mq8+eabdxzzsmXLEBcXh4ULF2LBggXQ6XQoLCzErFmzEBwcjCVLluC1116Dx+PB2LFjcf36dTidTphMJlXBKrxMT98kKIQQont0tuuWJPPy8piYmEiDwUCTycSEhATu2LFD6d+wYQOtVisNBgOTkpK4e/fuNhsLFixYwKCgIALgW2+9RZL86aefOHnyZPr5+TEyMpJffvllu5sxSkpK2sS0d+9eDhs2jDqdjgEBAXz00UeZm5vb4Rra24yxceNG1Rj8ZXPIX8/fuqkkJyeHERER1Ov1nDRpEi9fvqyaZ+vWrYyIiKBWq+XgwYO5e/fuTs/T6u2336bFYqFGo2FqaipJ8ujRo4yJiaFer2d8fDwdDofq+PZyVFNTQwAsLCxU2hwOBxMTE6nX62k2m5mUlKT8+3g8Hm7atIlRUVHUarUMCQlhUlISjx8/3mE+xb1PQ/7lxgkhhBCiF8vOzkZGRob89QrhFeQDk4UQQgghvJQUekIIIYQQXkp+dSuEEEII4aXkip4QQgghhJeSQk8IIYQQwktJoSeEEEII4aWk0BNCCCGE8FJS6AkhhBBCeCkp9IQQQgghvJQUekIIIYQQXkoKPSGEEEIILyWFnhBCCCGEl/o/2GqwbG53SW0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# #1. Get the hyperparameters of the model and print them\n",
    "best_params = study.best_trial.params\n",
    "\n",
    "# Add the static parameters\n",
    "best_params.update({\n",
    "    'boosting_type': 'gbdt',\n",
    "    'objective': 'multiclass',\n",
    "    'num_class': 3,\n",
    "    'metric': 'multi_logloss',\n",
    "    'random_state': 42,\n",
    "})\n",
    "\n",
    "\n",
    "import json\n",
    "\n",
    "# Write the hyperparameters to a new file\n",
    "with open('hyperparameters_model7.json', 'w') as f: # Make sure to change the name to match whatever model you are using\n",
    "    json.dump(best_params, f)\n",
    "\n",
    "\n",
    "\n",
    "# Create the model with the best parameters\n",
    "model7 = LGBMClassifier(**best_params) # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# Fit the model to the full data\n",
    "model7.fit(train_X1_full, train_Y1_temp_full)\n",
    "\n",
    "\n",
    "\n",
    "# #2. Plot bar plot of the feature importances to visualize the model\n",
    "import lightgbm as lgb\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Create a larger figure\n",
    "plt.figure(figsize=(80, 20), dpi=200)\n",
    "\n",
    "# Plot the feature importance\n",
    "ax = lgb.plot_importance(model7) \n",
    "\n",
    "# Decrease the font size of the y-axis labels since there are two many features\n",
    "for label in ax.get_yticklabels():\n",
    "    label.set_size(8)\n",
    "\n",
    "# Adjust the layout\n",
    "plt.tight_layout()\n",
    "\n",
    "# Save the plot\n",
    "plt.savefig('feature_importance_LGBM_model7.png') # Make sure to change the name to match whatever model you are using\n",
    "\n",
    "# Show the plot\n",
    "plt.show()\n",
    "\n",
    "# from optuna.visualization import plot_param_importances\n",
    "\n",
    "# plot_param_importances(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting Model 7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] feature_fraction is set=0.47414782515524534, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.47414782515524534\n",
      "[LightGBM] [Warning] lambda_l1 is set=3.805769924899612e-08, reg_alpha=0.0 will be ignored. Current value: lambda_l1=3.805769924899612e-08\n",
      "[LightGBM] [Warning] lambda_l2 is set=1.7390351172611058, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.7390351172611058\n",
      "[LightGBM] [Warning] bagging_fraction is set=0.7846100271861661, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7846100271861661\n",
      "[LightGBM] [Warning] feature_fraction is set=0.47414782515524534, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.47414782515524534\n",
      "[LightGBM] [Warning] lambda_l1 is set=3.805769924899612e-08, reg_alpha=0.0 will be ignored. Current value: lambda_l1=3.805769924899612e-08\n",
      "[LightGBM] [Warning] lambda_l2 is set=1.7390351172611058, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.7390351172611058\n",
      "[LightGBM] [Warning] bagging_fraction is set=0.7846100271861661, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7846100271861661\n",
      "[LightGBM] [Info] Auto-choosing row-wise multi-threading, the overhead of testing was 0.006368 seconds.\n",
      "You can set `force_row_wise=true` to remove the overhead.\n",
      "And if memory is not enough, you can set `force_col_wise=true`.\n",
      "[LightGBM] [Info] Total Bins 1193\n",
      "[LightGBM] [Info] Number of data points in the train set: 260601, number of used features: 34\n",
      "[LightGBM] [Info] Start training from score -2.339167\n",
      "[LightGBM] [Info] Start training from score -0.564030\n",
      "[LightGBM] [Info] Start training from score -1.094580\n",
      "[LightGBM] [Warning] feature_fraction is set=0.47414782515524534, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.47414782515524534\n",
      "[LightGBM] [Warning] lambda_l1 is set=3.805769924899612e-08, reg_alpha=0.0 will be ignored. Current value: lambda_l1=3.805769924899612e-08\n",
      "[LightGBM] [Warning] lambda_l2 is set=1.7390351172611058, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.7390351172611058\n",
      "[LightGBM] [Warning] bagging_fraction is set=0.7846100271861661, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7846100271861661\n",
      "Successfully Submitted!\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# Concatenate the datasets\n",
    "full_X = pd.concat([train_X1, valid_X1, test_X1])\n",
    "full_Y_temp = pd.concat([train_Y1_temp, valid_Y1_temp, test_Y1_temp])\n",
    "\n",
    "# Refit the model on the full dataset\n",
    "model7.fit(full_X, full_Y_temp)\n",
    "\n",
    "# Save the model to a file using joblib\n",
    "from joblib import dump\n",
    "dump(model7, 'saved_LGBM_model7.joblib') # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "\n",
    "# # This is how to load the model from joblib\n",
    "# from joblib import load\n",
    "# model5 = load('saved_LGBM_model7.joblib')  # Make sure to change the name of the file to match up with the model number!\n",
    "\n",
    "# Separate building_ids and features in the test data\n",
    "competition_test_building_ids = test_data1['building_id']\n",
    "competition_test_X = test_data1.drop('building_id', axis=1)\n",
    "\n",
    "# Predict on the competition test data\n",
    "competition_y_pred = model7.predict(competition_test_X)\n",
    "\n",
    "# Since the competition expects labels in the range 1-3, add 1 to the predictions\n",
    "competition_y_pred = competition_y_pred + 1\n",
    "\n",
    "# Create a DataFrame for submission\n",
    "submission = pd.DataFrame({\n",
    "    'building_id': competition_test_building_ids,\n",
    "    'damage_grade': competition_y_pred\n",
    "})\n",
    "\n",
    "# Save the submission DataFrame to a CSV file for submission\n",
    "submission.to_csv('submission7.csv', index=False) # Make sure to change the name of the submission file to match up with the model number!\n",
    "print(\"Successfully Submitted!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TRY TO RECORD THE LOGS AND ALSO LOOK AT THE OTHER GOOD TRIALS AND TRY THOSE AND REMOVETHE HYPERPARAMETERS THAT GOT IGNORED LIKE LAMBDA AND FEATURE AND BAGGING FRACTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Try GPU accelerated LGBM models with Optuna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0. The competition is based on the micro averaged F1 score. Find a way to try and optimize this score during hyper parameter tuning, and make sure to make it clear that I should be looking at the scoring metric of every competition and making sure I optimize this scoring metric specifically when I do hyper parameter tuning and such, and not only focusing on accuracy\n",
    "- Analyze confusion matrix and classification reports to understand model performance better.\n",
    "\n",
    "### 3. Maybe go back and normalize the data since you decided to skip that step\n",
    "\n",
    "### 4. Upload my notebook to ChatGPT Michael ML Tutor and ask it what else I can do to improve it as an ML engineer. turn this file into a python script and then copy and paste it into ChatGPT Michael ML Tutor \n",
    "\n",
    "### 5. Look into LightGBM GPU training and parallel training"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}