Language/Python/2020-06-13 Exception Handling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  \"Exceptions a deep dive\"\n",
    "toc: true\n",
    "badges: true\n",
    "comments: true\n",
    "categories: [jupyter]\n",
    "author: Abhinav Verma"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you build a complex python pipeline or you deploy some python microservices in production , You need to make sure the code works perfectly. A small issue can bring all your efforts crashing down. A lot of times the error would be a simple syntax mistake which becomes easily apparent when the program runs in a few iterations. However there are many complicated errors that can arise. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example , suppose you are interacting with an external API, let's take twitter. You call the api to give you results pertaining to a particular query. You then proceed to store the average sentiment across the number of posts you see in a day for your query. This goes all smooth for a few days but then the topic dies on twitter as all things do and you don't see any post. However your pipeline still takes the average which means it expects a number of posts , your machine learning model expects some text to calculate the sentiment and at the end you multiply the total sentiment score divided by the number of posts. You are now __passing empty__ strings to your model and __dividing by zero__. Basically the system will crash. These issues are what's called exceptions ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now picture another scenario. Say you run a company like Zomato . I'm not going to tell you about zomato. You can check them out [here](https://www.zomato.com) . They also have an api through which you can get data of restaurants and reviews from their city [here](https://developers.zomato.com/documentation) . So now you have multiple developers hitting your API for the data and you've learnt from the mistakes and there isn't any exception like the one we encountered above. However now there is another unique situation. You can sometimes have too many calls and this is clogging your server. So you decide to get a bigger server or a cluster of servers but the popularity of your API means that the cost of maintenance is always high. So you decide to be a little creative. You bring out tiered plans. Based on how much people pay they are limited to extract that much data in a day. But when they exceed this amount you want to let them know that. So you raise and send an exception to them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above scenario are 2 situations where understanding of exceptions is really important to know as a programmer. In the coming post we will discuss exceptions in detail . Exceptions are a type of object so the concept of classes and single level inheritance comes into play as well. You can check the earlier post on that although understanding exceptions through this post can give a deeper understanding of single inheritance so you can start either way. So without further ado let's start on exceptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling Exceptions\n",
    "\n",
    "The first thing we'll look at is how to handle any exceptions that might arise in our code. There isn't any one elegant way of handling exception however a good practice is follows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Initially handle the exception by wrapping the code in a try/except block catching the blanket exception.\n",
    "2. Once the exception is clear then we can proceed on handling the specific exception. This helps remove ambiguity from code as different errors need to be handled differently"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at some examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start with some basic exception handling while reading lists and dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndexError",
     "evalue": "list index out of range",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-2c562ebcc971>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0ml\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0ml\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;31m#list only has 3 elements\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m: list index out of range"
     ]
    }
   ],
   "source": [
    "l = [1, 2, 3]\n",
    "l[4] #list only has 3 elements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The exception thrown here is IndexError . So let's now handle the error via an exception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list index out of range\n"
     ]
    }
   ],
   "source": [
    "l = [1,2,3] # can declare this outside as its not something that causes errors\n",
    "try:\n",
    "    l[4]\n",
    "except IndexError as e: # its convention to alias any exception with e. In practice anything can be used\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The message that gets printed is the error that proceeds the IndexError key above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    }
   ],
   "source": [
    "l = [1,2,3] # can declare this outside as its not something that causes errors\n",
    "try:\n",
    "    l[4]\n",
    "except IndexError as e: \n",
    "    print(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A quick aside. You can even handle exceptions using the bare except syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exception occurred\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    1 / 0\n",
    "except:\n",
    "    print('exception occurred')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In practice this doesn't help at all as in your pipeline you won't learn anything about your error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at a dictionary and handle exceptions using that. The dictionary you will be seeing is a dictionary that's been scraped from amazon regarding a product. Web scraping is another topic in itself but reading a dictionary scraped from the internet is a really good real life example of how to handle errors while scraping So let's look at that"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {'name': 'Uno Fast Fun',\n",
    " 'source_url': 'https://www.amazon.in/UNO-42003-Uno-Fast-Fun/dp/B00004TZY8',\n",
    " 'price': '₹ 149.00',\n",
    " 'images': ['https://images-na.ssl-images-amazon.com/images/I/31PAL9s5S-L.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/516YvMIqL8L.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/513Ko-uHFXL.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/5152b4jUxmL.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/41SJFXFzptL.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/51Q8FYew72L.jpg',\n",
    "  'https://images-na.ssl-images-amazon.com/images/I/51aCIYAIWoL.jpg'],\n",
    " 'stock_image': 'https://images-na.ssl-images-amazon.com/images/I/51aCIYAIWoL.jpg',\n",
    " 'seller': 'Sold by R & R (4.2 out of 5 stars | 279 ratings).',\n",
    " 'rating': '4.2 out of 5 stars',\n",
    " 'rating_count': 7210,\n",
    " 'features': ['Minimum Recommended Age : 4 Years',\n",
    "  'Battries not required',\n",
    "  'See why this color coded card game is such a popular brand of family game',\n",
    "  'Be the first player or team to score 500 points',\n",
    "  'Points are scored by being the first to rid yourself of all the cards in your hand before your opponents'],\n",
    " 'description': \"America's number one card game, UNO offers hours of fun. It's easy to learn, always unpredictable and never dull. It can be played with 2 to 10 players, individually or in teams and it can even be educational, teaching younger kids number recognition. Add fun and excitement to your family time by playing the Mattel UNO fast fun. This entertaining card game will surely make family get-togethers engaging and interactive. Just like any other card game, you need to focus and strategise your moves to challenge all other players. It is a great exercise to help build up your child's interactive and interpersonal skills. You can carry this compact Mattel Uno cards game anywhere and keep the boredom away. Ideal to play while your travel long distance in the car, during boring afternoons or as a party wrap-up game. Exciting and easy to play, the Mattel Uno game can be played by 2 to 10 players and is suitable for children aged 7 years and above.Spend quality time with your family while you play this Mattel UNO card game. This UNO card game involves individual players or teams, who have be the first to score 500 points. You have to play the colour coded cards from your hand, which would either match the number or colour of the card that lies on the top of the discarded cards. If your do not have either match, you simply have to pull cards from the fresh deck until you find the match. Piling cards in your hand is a losing site, which means you have to successfully rid yourself of the cards to win this game. When left with only one card in hand, you have to yell UNO and make others jealous.\",\n",
    " 'productInfo': [{'name': 'Educational Objective(s)', 'value': 'Reading'},\n",
    "  {'name': 'Main Language(s)', 'value': 'English translation, English'},\n",
    "  {'name': 'Model Number', 'value': '42003'},\n",
    "  {'name': 'Number of Game Players', 'value': '2-10'},\n",
    "  {'name': 'Number of Puzzle Pieces', 'value': '1'},\n",
    "  {'name': 'Assembly Required', 'value': 'No'},\n",
    "  {'name': 'Scale', 'value': '1/32'},\n",
    "  {'name': 'Batteries Required', 'value': 'No'},\n",
    "  {'name': 'Batteries Included', 'value': 'No'},\n",
    "  {'name': 'Material Type(s)', 'value': 'other-materials'},\n",
    "  {'name': 'Remote Control Included?', 'value': 'No'},\n",
    "  {'name': 'Color', 'value': 'multi'},\n",
    "  {'name': 'Item Weight', 'value': '90.7 g'},\n",
    "  {'name': 'Product Dimensions', 'value': '1.9 x 17.8 x 12.7 cm'},\n",
    "  {'name': 'Item model number', 'value': '42003'},\n",
    "  {'name': \"Manufacturer's Minimum Suggested Age (years)\", 'value': '7'},\n",
    "  {'name': 'ASIN', 'value': 'B00004TZY8'},\n",
    "  {'name': 'Customer Reviews',\n",
    "   'value': '4.2 out of 5 stars7,210 customer ratings'},\n",
    "  {'name': 'Amazon Bestsellers Rank',\n",
    "   'value': '#983 in Toys & Games (See Top 100 in Toys & Games) #35 in Card Games'},\n",
    "  {'name': 'Date First Available', 'value': '9 April 2012'}]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(data:dict) -> str:\n",
    "    name = data[\"name\"]\n",
    "    return name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Uno Fast Fun'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_name(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However it can be a little risky to directly query dictionaries this way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(data:dict) -> str:\n",
    "    name = data[\"product_name\"]\n",
    "    return name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'product_name'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-13-a9f5ada1933a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mget_name\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-12-d4a208ffbab7>\u001b[0m in \u001b[0;36mget_name\u001b[0;34m(data)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_name\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mdict\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0mname\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"product_name\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyError\u001b[0m: 'product_name'"
     ]
    }
   ],
   "source": [
    "get_name(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionaries throw a KeyError when an incorrect key is called. A good way to do it would be to wrap the indexing in a try except block catching the KeyError"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(data:dict) -> str:\n",
    "    try:\n",
    "        \n",
    "        name = data[\"product_name\"]\n",
    "    except KeyError as e:\n",
    "        print(e)\n",
    "        return \"Incorrect Key Entered. Please check key\"\n",
    "    return name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'product_name'\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Incorrect Key Entered. Please check key'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_name(data) #as you can see error is handled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course __Dictionaries__ handle all this implicitly using the get method. Yes Dictionaries are instances of the dictionary class. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(data:dict) -> str:\n",
    "    try:\n",
    "        \n",
    "        name = data.get(\"product_name\",\"\")\n",
    "    except KeyError as e:\n",
    "        print(e)\n",
    "        return \"Incorrect Key Entered. Please check key\"\n",
    "    return name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_name(data) # as you can see get handles this and keyerror isn't called"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course there's also one other thing that isn't often implemented but does come with the __try__ __except__ statements is the finally block. Whilst the exception block is executed only if there is an exception. The finally block is executed regardless."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(data:dict) -> str:\n",
    "    try:\n",
    "        \n",
    "        name = data.get(\"product_name\",\"\")\n",
    "    except KeyError as e:\n",
    "        print(e)\n",
    "        return \"Incorrect Key Entered. Please check key\"\n",
    "    finally:\n",
    "        link = data.get(\"source_url\")\n",
    "        print(link)\n",
    "      \n",
    "    return name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.amazon.in/UNO-42003-Uno-Fast-Fun/dp/B00004TZY8\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_name(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}