6. Tariff Recommendation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tariff Recommendation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have at your disposal data about the behavior of clients who have already passed on these tariffs (from the project of the course «Statistical data analysis»). You need to build a model for the classification task that will choose a suitable rate. You do not need to pre-process the data - you have already done it.\n",
    "\n",
    "Build a model with the highest value of *accuracy*. To pass the project successfully, you need to bring the share of correct answers to at least 0.75. Check *accuracy* on the test sample yourself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open and examine the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3214 entries, 0 to 3213\n",
      "Data columns (total 5 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   calls     3214 non-null   float64\n",
      " 1   minutes   3214 non-null   float64\n",
      " 2   messages  3214 non-null   float64\n",
      " 3   mb_used   3214 non-null   float64\n",
      " 4   is_ultra  3214 non-null   int64  \n",
      "dtypes: float64(4), int64(1)\n",
      "memory usage: 125.7 KB\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>calls</th>\n",
       "      <th>minutes</th>\n",
       "      <th>messages</th>\n",
       "      <th>mb_used</th>\n",
       "      <th>is_ultra</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>40.0</td>\n",
       "      <td>311.90</td>\n",
       "      <td>83.0</td>\n",
       "      <td>19915.42</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>85.0</td>\n",
       "      <td>516.75</td>\n",
       "      <td>56.0</td>\n",
       "      <td>22696.96</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>77.0</td>\n",
       "      <td>467.66</td>\n",
       "      <td>86.0</td>\n",
       "      <td>21060.45</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>106.0</td>\n",
       "      <td>745.53</td>\n",
       "      <td>81.0</td>\n",
       "      <td>8437.39</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>66.0</td>\n",
       "      <td>418.74</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14502.75</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   calls  minutes  messages   mb_used  is_ultra\n",
       "0   40.0   311.90      83.0  19915.42         0\n",
       "1   85.0   516.75      56.0  22696.96         0\n",
       "2   77.0   467.66      86.0  21060.45         0\n",
       "3  106.0   745.53      81.0   8437.39         1\n",
       "4   66.0   418.74       1.0  14502.75         0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('/datasets/users_behavior.csv')\n",
    "df.info()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open the file and make sure the data is really good. We have 5 features: Number of calls, Total number of minutes, Number of messages, Used traffic and Tariff.\n",
    "There are 3214 rows in the data array, no blanks.\n",
    "The Number of calls and Number of messages features contain only integers. You can change the data type to int."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "for column in df[['calls','messages']]:\n",
    "    df[column] = df[column].astype(\"int\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, just in case, check the data for obvious duplicates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are no obvious duplicates. Now you can definitely start work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split data into samples\n",
    "\n",
    "Divide the initial data into training, validation and test samples in 3:1:1 proportions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1928, 4)\n",
      "(1928,)\n",
      "(643, 4)\n",
      "(643,)\n",
      "(643, 4)\n",
      "(643,)\n"
     ]
    }
   ],
   "source": [
    "features = df.drop(['is_ultra'], axis=1)\n",
    "target = df['is_ultra'] \n",
    "features_train, features_test, target_train, target_test = train_test_split(features, target, test_size=0.4, random_state=12345)\n",
    "features_valid, features_test, target_valid, target_test = train_test_split(features_test, target_test, test_size=0.5, random_state=12345) \n",
    "print(features_train.shape)\n",
    "print(target_train.shape)\n",
    "print(features_test.shape)\n",
    "print(target_test.shape)\n",
    "print(features_valid.shape)\n",
    "print(target_valid.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The percentage ratio is right, we can start the research.\n",
    "\n",
    "## Explore models\n",
    "\n",
    "We explore three models of machine learning: Decision Tree Classifier, RandomForestClassifier and Logistic Regression.\n",
    "\n",
    "### Decision Tree Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best max_depth = 3 ;  best Accuracy =  0.7853810264385692"
     ]
    }
   ],
   "source": [
    "best_model = None\n",
    "best_depth = 0\n",
    "best_result = 0\n",
    "for depth in range(1, 11):\n",
    "\tmodel = DecisionTreeClassifier(random_state=12345, max_depth=depth)\n",
    "\tmodel.fit(features_train, target_train) \n",
    "\tpredictions_valid = model.predict(features_valid)\n",
    "\tresult = accuracy_score(predictions_valid, target_valid) \n",
    "\tif result > best_result:\n",
    "\t\tbest_model = model\n",
    "\t\tbest_depth = depth\n",
    "\t\tbest_result = result\n",
    "        \n",
    "print('best max_depth =', \n",
    "      best_depth, \n",
    "      '; ', \n",
    "      'best Accuracy = ', \n",
    "      best_result, \n",
    "      end =''\n",
    "     ) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best tree depth hyperparameter is 3 in the \"Decipher tree\" model, the correct response rate is 0.785\n",
    "\n",
    "### RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best n_estimators = 12 ;  best max_depth = 6 ;  best Accuracy =  0.8040435458786936"
     ]
    }
   ],
   "source": [
    "best_model = None\n",
    "best_result = 0\n",
    "best_est = 0\n",
    "best_depth = 0\n",
    "\n",
    "for est in range(10, 21):\n",
    "    for depth in range(1, 11):\n",
    "        model = RandomForestClassifier(random_state=12345, n_estimators=est, max_depth = depth) # обучим модель с заданным количеством деревьев\n",
    "        model.fit(features_train, target_train) # обучим модель на тренировочной выборке\n",
    "        result = model.score(features_valid, target_valid) # посчитаем качество модели на валидационной выборке\n",
    "        if result > best_result:\n",
    "            best_model = model # сохраним наилучшую модель\n",
    "            best_result = result #  сохраним наилучшее значение метрики accuracy на валидационных данных\n",
    "            best_est = est\n",
    "            best_depth = depth\n",
    "    \n",
    "print('best n_estimators =', \n",
    "      best_est, \n",
    "      '; ',\n",
    "      'best max_depth =', \n",
    "      best_depth, \n",
    "      '; ', \n",
    "      'best Accuracy = ', \n",
    "      best_result, \n",
    "      end =''\n",
    "     ) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the model \"Random Forest\" get the best hyperparameters: for the number of trees - 8 and for the depth of the tree - 8. The share of correct answers will be 0.802 - for the time being it is the best result.\n",
    "\n",
    "### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy LogisticRegression: 0.7107309486780715"
     ]
    }
   ],
   "source": [
    "model = LogisticRegression(random_state=12345) \n",
    "model.fit(features_train, target_train) \n",
    "predictions_valid = model.predict(features_valid)\n",
    "score = accuracy_score(predictions_valid, target_valid)\n",
    "print('Accuracy LogisticRegression:', score, end ='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the \"Logistic Regression\" model, the response rate is 0.711, which is lower than in the previous two models.\n",
    "\n",
    "## Check model on test sample\n",
    "\n",
    "The best result showed the model \"Random Forest\", so we will test it on a test sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy RandomForestClassifier: 0.7947122861586314\n"
     ]
    }
   ],
   "source": [
    "model = RandomForestClassifier(random_state=12345, n_estimators = 12, max_depth = 6)\n",
    "model.fit(features_train, target_train)\n",
    "predictions_test = model.predict(features_test)\n",
    "result_test = accuracy_score(predictions_test, target_test)\n",
    "print('Accuracy RandomForestClassifier:', result_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model showed good results on the test sample, the correct response rate changed very little - 0.796.\n",
    "\n",
    "## (bonus) Check models for adequacy\n",
    "\n",
    "Check the adequacy of our model with the simplest DummyClassifier model, which always predicts the most common class. So we have the benchmark data to compare the model we’ve built. In addition, we will be able to be sure of the quality of its work, for example, that it produces not just a set of randomly selected data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy DummyClassifier: 0.7060653188180405\n",
      "Accuracy RandomForestClassifier: 0.7962674961119751\n"
     ]
    }
   ],
   "source": [
    "model_dummy = DummyClassifier(strategy='most_frequent', random_state=12345)\n",
    "model_dummy.fit(features_train, target_train)\n",
    "result_dummy = model_dummy.score(features_valid, target_valid)\n",
    "print('Accuracy DummyClassifier:', result_dummy)\n",
    "print('Accuracy RandomForestClassifier:', result_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The rate of correct answers in the simplest model is much lower, so our model is adequate and effective.\n",
    "\n",
    "## General conclusion\n",
    "\n",
    "The data was provided correct, the pre-processing as such was not necessary.\n",
    "Three models were investigated: Decision Tree Classifier, RandomForestClassifier and Logistic Regression.\n",
    "The lowest result was the Logistic Regression with a score of 0.711.\n",
    "The best result showed the RandomForestClassifier model with n_estimators hyperparameters equal to 8 and max_depth equal to 8. The correct response rate was 0.802. On the test sample the result was 0.796. \n",
    "The DummyClassifier was used in the adequacy test, with a result of 0.706, which proved that the RandomForestClassifier was adequate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "ExecuteTimeLog": [
   {
    "duration": 518,
    "start_time": "2023-01-13T11:29:48.228Z"
   },
   {
    "duration": 133,
    "start_time": "2023-01-13T11:29:59.926Z"
   },
   {
    "duration": 41,
    "start_time": "2023-01-13T11:30:18.877Z"
   },
   {
    "duration": 42,
    "start_time": "2023-01-13T11:32:52.799Z"
   },
   {
    "duration": 6,
    "start_time": "2023-01-13T11:49:43.608Z"
   },
   {
    "duration": 13,
    "start_time": "2023-01-13T11:49:57.969Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-13T11:50:10.796Z"
   },
   {
    "duration": 8,
    "start_time": "2023-01-13T11:50:56.720Z"
   },
   {
    "duration": 1160,
    "start_time": "2023-01-13T12:32:40.298Z"
   },
   {
    "duration": 6,
    "start_time": "2023-01-13T12:32:41.460Z"
   },
   {
    "duration": 12,
    "start_time": "2023-01-13T12:32:41.467Z"
   },
   {
    "duration": 106,
    "start_time": "2023-01-13T12:32:41.482Z"
   },
   {
    "duration": 7,
    "start_time": "2023-01-13T12:34:14.070Z"
   },
   {
    "duration": 8,
    "start_time": "2023-01-13T12:34:32.138Z"
   },
   {
    "duration": 11,
    "start_time": "2023-01-13T12:53:42.107Z"
   },
   {
    "duration": 1208,
    "start_time": "2023-01-13T13:45:46.351Z"
   },
   {
    "duration": 7,
    "start_time": "2023-01-13T13:45:47.561Z"
   },
   {
    "duration": 12,
    "start_time": "2023-01-13T13:45:47.570Z"
   },
   {
    "duration": 10,
    "start_time": "2023-01-13T13:45:47.584Z"
   },
   {
    "duration": 532,
    "start_time": "2023-01-13T13:45:47.596Z"
   },
   {
    "duration": 105,
    "start_time": "2023-01-13T13:46:08.070Z"
   },
   {
    "duration": 1197,
    "start_time": "2023-01-13T13:46:40.359Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-13T13:46:41.558Z"
   },
   {
    "duration": 10,
    "start_time": "2023-01-13T13:46:41.565Z"
   },
   {
    "duration": 18,
    "start_time": "2023-01-13T13:46:41.578Z"
   },
   {
    "duration": 91,
    "start_time": "2023-01-13T13:46:41.598Z"
   },
   {
    "duration": 267,
    "start_time": "2023-01-13T14:08:14.793Z"
   },
   {
    "duration": 256,
    "start_time": "2023-01-13T14:08:28.507Z"
   },
   {
    "duration": 1129,
    "start_time": "2023-01-13T14:08:41.993Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-13T14:08:43.125Z"
   },
   {
    "duration": 41,
    "start_time": "2023-01-13T14:08:43.132Z"
   },
   {
    "duration": 69,
    "start_time": "2023-01-13T14:08:43.175Z"
   },
   {
    "duration": 100,
    "start_time": "2023-01-13T14:08:43.247Z"
   },
   {
    "duration": 279,
    "start_time": "2023-01-13T14:08:43.349Z"
   },
   {
    "duration": 276,
    "start_time": "2023-01-13T14:10:41.948Z"
   },
   {
    "duration": 1178,
    "start_time": "2023-01-13T14:19:13.546Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-13T14:19:14.727Z"
   },
   {
    "duration": 9,
    "start_time": "2023-01-13T14:19:14.737Z"
   },
   {
    "duration": 36,
    "start_time": "2023-01-13T14:19:14.747Z"
   },
   {
    "duration": 68,
    "start_time": "2023-01-13T14:19:14.785Z"
   },
   {
    "duration": 379,
    "start_time": "2023-01-13T14:19:14.854Z"
   },
   {
    "duration": 270,
    "start_time": "2023-01-13T14:19:25.128Z"
   },
   {
    "duration": 280,
    "start_time": "2023-01-13T14:19:47.102Z"
   },
   {
    "duration": 38,
    "start_time": "2023-01-13T14:39:42.519Z"
   },
   {
    "duration": 47,
    "start_time": "2023-01-13T14:45:50.599Z"
   },
   {
    "duration": 43,
    "start_time": "2023-01-13T14:46:46.344Z"
   },
   {
    "duration": 27,
    "start_time": "2023-01-13T14:47:05.746Z"
   },
   {
    "duration": 1819,
    "start_time": "2023-01-13T15:17:53.547Z"
   },
   {
    "duration": 1188,
    "start_time": "2023-01-13T15:18:11.962Z"
   },
   {
    "duration": 6,
    "start_time": "2023-01-13T15:18:13.153Z"
   },
   {
    "duration": 12,
    "start_time": "2023-01-13T15:18:13.161Z"
   },
   {
    "duration": 12,
    "start_time": "2023-01-13T15:18:13.176Z"
   },
   {
    "duration": 107,
    "start_time": "2023-01-13T15:18:13.190Z"
   },
   {
    "duration": 1827,
    "start_time": "2023-01-13T15:18:13.299Z"
   },
   {
    "duration": 25,
    "start_time": "2023-01-13T15:18:15.128Z"
   },
   {
    "duration": 34,
    "start_time": "2023-01-13T15:36:12.437Z"
   },
   {
    "duration": 102,
    "start_time": "2023-01-13T16:04:31.232Z"
   },
   {
    "duration": 1177,
    "start_time": "2023-01-13T16:05:01.708Z"
   },
   {
    "duration": 4,
    "start_time": "2023-01-13T16:05:02.887Z"
   },
   {
    "duration": 12,
    "start_time": "2023-01-13T16:05:02.893Z"
   },
   {
    "duration": 17,
    "start_time": "2023-01-13T16:05:02.906Z"
   },
   {
    "duration": 81,
    "start_time": "2023-01-13T16:05:02.925Z"
   },
   {
    "duration": 1771,
    "start_time": "2023-01-13T16:05:03.007Z"
   },
   {
    "duration": 24,
    "start_time": "2023-01-13T16:05:04.780Z"
   },
   {
    "duration": 62,
    "start_time": "2023-01-13T16:05:04.806Z"
   },
   {
    "duration": 7,
    "start_time": "2023-01-13T16:05:04.870Z"
   },
   {
    "duration": 1983,
    "start_time": "2023-01-15T08:31:27.445Z"
   },
   {
    "duration": 6,
    "start_time": "2023-01-15T08:31:29.429Z"
   },
   {
    "duration": 31,
    "start_time": "2023-01-15T08:31:29.436Z"
   },
   {
    "duration": 13,
    "start_time": "2023-01-15T08:31:29.469Z"
   },
   {
    "duration": 84,
    "start_time": "2023-01-15T08:31:29.484Z"
   },
   {
    "duration": 1923,
    "start_time": "2023-01-15T08:31:29.570Z"
   },
   {
    "duration": 55,
    "start_time": "2023-01-15T08:31:31.495Z"
   },
   {
    "duration": 93,
    "start_time": "2023-01-15T08:31:31.552Z"
   },
   {
    "duration": 18,
    "start_time": "2023-01-15T08:31:31.647Z"
   },
   {
    "duration": 1090,
    "start_time": "2023-01-17T16:57:29.467Z"
   },
   {
    "duration": 160,
    "start_time": "2023-01-17T16:57:30.559Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-17T16:57:30.721Z"
   },
   {
    "duration": 31,
    "start_time": "2023-01-17T16:57:30.727Z"
   },
   {
    "duration": 26,
    "start_time": "2023-01-17T16:57:30.760Z"
   },
   {
    "duration": 96,
    "start_time": "2023-01-17T16:57:30.787Z"
   },
   {
    "duration": 1707,
    "start_time": "2023-01-17T16:57:30.885Z"
   },
   {
    "duration": 26,
    "start_time": "2023-01-17T16:57:32.593Z"
   },
   {
    "duration": 43,
    "start_time": "2023-01-17T16:57:32.621Z"
   },
   {
    "duration": 5,
    "start_time": "2023-01-17T16:57:32.666Z"
   },
   {
    "duration": 102,
    "start_time": "2023-01-17T16:58:19.404Z"
   },
   {
    "duration": 4356,
    "start_time": "2023-01-17T16:59:50.836Z"
   },
   {
    "duration": 39,
    "start_time": "2023-01-17T17:00:35.531Z"
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}