AI4H_practical1a.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["# Introduction\n","\n","Welcome!\n","\n","This notebook is for your first practical session of the AI for Health course. Over the course of multiple practical sessions, you will learn how to understand the code that underlies applications of Artificial Intelligence (AI) in healthcare. You will also learn how to implement AI models by yourself. Programming is a skill that is acquired by hands-on experience and typically requires a lot of training. During this course we will not be able to make you an expert programmer, but we aim to introduce you to the different ingredients that come into play when creating an AI model. \n","\n","In any AI project, your data is your most crucial ingredient. In this session, we will start by introducing some ways to get a better understanding of your data. Along the way we will introduce several Python packages that are very often used when running an AI project. Some basic programming knowledge is assumed (things like defining variables, for loops, if statements, etc.). If you haven't done any Python introduction yet, please make use of the RealPython accounts offered to you.\n","\n","## Learning objectives\n","- get familiar with the look and feel of Python\n","- get familiar with Google Colab\n","- get familiar with the Python packages Numpy \n","\n","Now, let's get started!"],"metadata":{"id":"rGbaZCnFv5rI"}},{"cell_type":"markdown","source":["#Getting started with Numpy\n","\n","There are many ways to represent your data and perform calculations on your data in Python. We will focus on the two most used packages for this, Numpy, which we will focus on first, and Pandas, which will be introduced in the next practical session. \n","\n","Numpy is a fundamental package for scientific computing with Python. Its most important feature is storing data in large arrays and enabling you to perform all kinds of computations on these arrays. \n","\n","Let's start by importing Numpy."],"metadata":{"id":"cKJDkEHu9-sH"}},{"cell_type":"code","source":["import numpy as np"],"metadata":{"id":"J-u5JibqC8KG"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["The most important feature of Numpy are its arrays, on object that can store data and where you can easily perform all kinds of functions on. To create an array you can use the function np.array(). \n","\n","You can get information about a function with a double ??. A separate help window opens where you can see what a function expects as input, and what it returns as output.\n","\n"],"metadata":{"id":"hne8idBg6Y6p"}},{"cell_type":"code","source":["??np.array"],"metadata":{"id":"0XJ9mLV95lcp"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Now let's create our first array."],"metadata":{"id":"QA6yv6uH67Ip"}},{"cell_type":"code","execution_count":null,"metadata":{"id":"Gs9DBw1kvuJO"},"outputs":[],"source":["my_array = np.array([1,2,3,4,5])"]},{"cell_type":"markdown","source":["To see the contents of an array, you can use a print statement."],"metadata":{"id":"jmG6HmDWDhMl"}},{"cell_type":"code","source":["print(my_array)"],"metadata":{"id":"msq5a_g6CtGD"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["The array you just created has several properties called 'attributes' automatically assigned to it. One if these is how many elements there are in the array, called the 'size'. Something that is very useful when your arrays are becoming larger."],"metadata":{"id":"mqT9Buagci2c"}},{"cell_type":"code","source":["print(my_array.size)"],"metadata":{"id":"89QFdQx1czzD"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["You can also print the shape of the array. This gives the number of elements per dimension. Our array is just 1 dimensional, so there is only 1 number."],"metadata":{"id":"5d1eEd5ec4O7"}},{"cell_type":"code","source":["print(my_array.shape)"],"metadata":{"id":"ljOQpmXmdz7s"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Let's make a 2 dimensional array and print its size and shape."],"metadata":{"id":"xT7Wediod2kj"}},{"cell_type":"code","source":["my_array = np.array([[0,1,2],[3,4,5]])\n","print(my_array)\n","print(my_array.size)\n","print(my_array.shape)"],"metadata":{"id":"DAqWzMWJd-qs"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["You can see that our array contains 6 elements divided into 2 rows (1st dimension) and 3 columns (2nd dimension).\n","\n","Arrays can have any number of dimensions. Higher-dimensional arrays can be hard to print or visualize, which often requires you to select a subsection of your elements that you can easily print or visualize. This brings us to our next subject: indexing arrays."],"metadata":{"id":"Jab0FTFjeq62"}},{"cell_type":"markdown","source":["# Indexing arrays\n","Indexing is used to select elements from your array. There are different ways to index arrays, which we will demonstrate here.\n","\n","In Python indices start from 0 (in contrast to for example Matlab where indices start from 1).\n","\n","Let's select the first element of an array."],"metadata":{"id":"pDc2Q3LNg1JZ"}},{"cell_type":"code","source":["my_array = np.array([1,2,3,4,5])\n","print(my_array[0])"],"metadata":{"id":"diSExfE5rY9U"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["To select elements from a 2D array you indicate the place of the element in every dimension separated by comma's."],"metadata":{"id":"bHUP6jIIsMws"}},{"cell_type":"code","source":["my_array = np.array([[1,2,3],[4,5,6]])\n","print(my_array[1,2])"],"metadata":{"id":"TvIaJgd-scFs"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["We selected the element on row 1, column 2, which is 6 (remember indices start from 0).\n","\n","Try to select the number 4 from the array below."],"metadata":{"id":"1n5pwgEXstCk"}},{"cell_type":"code","source":["my_array = np.array([[2,5,1,4],[7,0,6,3]])\n","\n","print(my_array[...])"],"metadata":{"id":"AJc5H_QStKdl"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["To select elements from the end of the array, you can use negative indices."],"metadata":{"id":"VIwASzg-tKEj"}},{"cell_type":"code","source":["my_array = np.array([[2,5,1,4],[7,0,6,3]])\n","\n","print(my_array[-1,-1])"],"metadata":{"id":"defhA3m0xnqS"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["To select a range of elements you can use the ':' symbol. This is called slicing."],"metadata":{"id":"KdExl-9I2Vd3"}},{"cell_type":"code","source":["my_array = np.array([5,0,4,1,3])\n","print(my_array[0:3])"],"metadata":{"id":"d0z26Uzb2vD9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Note that the elements that are selected include the start index, but exclude the end index (so index 0, 1, and 2, but not 3).\n","\n","If you want to select all elements from a certain point onwards you can leave the end index."],"metadata":{"id":"GXPGgt0t3GXf"}},{"cell_type":"code","source":["my_array = np.array([5,0,4,1,3])\n","print(my_array[2:])"],"metadata":{"id":"bvehXK3u8-q2"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["This can also be done with negative indices."],"metadata":{"id":"Q7T64Emq9KAP"}},{"cell_type":"code","source":["my_array = np.array([5,0,4,1,3])\n","print(my_array[2:-1])"],"metadata":{"id":"1ewjrZL29IpG"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Slicing in more dimensions is also possible."],"metadata":{"id":"HAE1njOp9aLm"}},{"cell_type":"code","source":["my_array = np.array([[2,5,1,4],[7,0,6,3]])\n","\n","print(my_array[1,1:4])"],"metadata":{"id":"MFXbphtU9ZZv"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Can you select from both rows the first 3 elements? "],"metadata":{"id":"9p4YB54h97Mf"}},{"cell_type":"code","source":["my_array = np.array([[2,5,1,4],[7,0,6,3]])\n","\n","print(my_array[...])"],"metadata":{"id":"RBGNV5UM96jg"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Data types\n","\n","Another important property of Numpy arrays is the data type. This is an important concept as it determines how much storage space our data requires. There are many different data types, the standard data types of Python are\n","\n","- string - used to represent text data, the text is given under quote marks. e.g. \"Hello\"\n","- integer - used to represent integer numbers. e.g. 1, 2, 3\n","- float - used to represent real numbers. e.g. 2.1, 11.53\n","- boolean - used to represent True or False.\n","- complex - used to represent complex numbers. e.g. 2.2 + 3.0j, 2.6 + 2.1j\n","\n","There are also subtypes for these data types. For example, int8, can store integers between -128 to 127 and int16 can store integers between -32768 to 32767. Storing a number as int16 takes twice as much space on your disk as storing a number as int8. It is thus good practice to store your data in a type that uses as little space as possible, especially when going to larger data sets.\n","\n","Let's print the data type of our array."],"metadata":{"id":"byO9d5ueFk7h"}},{"cell_type":"code","source":["int_array = np.array([1,2,3,4,5])\n","\n","print(int_array.dtype)"],"metadata":{"id":"7VKlh4bRD-w5"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Numpy has stored the data automatically as an int64, the largest data type for integers. If we know we will only use integers between -128 and 127 we can store this much more efficiently."],"metadata":{"id":"1z6U-6X2JYB9"}},{"cell_type":"code","source":["int_array = int_array.astype('int8')\n","\n","print(int_array.dtype)"],"metadata":{"id":"Ebzy2-agDFM9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["We now stored the data in a way that takes 8 times less memory.\n","\n","Next, let's have a look at decimal numbers. "],"metadata":{"id":"pAHDL3F2KR_l"}},{"cell_type":"code","source":["float_array = np.array([1.3,2.1,3.5,4.2,5.7])\n","\n","print(float_array.dtype)"],"metadata":{"id":"AP-8HRGwJxd1"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["This array is automatically assigned a float64 data type, because the int data type cannot store decimals. Floats can also be stored more efficiently by reducing the number decimals that are stored. This can lead to small errors that can in some cases be a problem, but for machine learning applications are generally negligible.\n","\n","Try to convert the array to a float32 yourself."],"metadata":{"id":"1fMQaqz6K90u"}},{"cell_type":"code","source":["float_array = ...\n","\n","print(float_array.dtype)"],"metadata":{"id":"YUr7eFroK54h"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Another often-used data type is the boolean. These are used to store binary, or True / False responses. If you create an array of 0's and 1's Numpy does not automatically turn it into a boolean array, but thinks it is an array of integers."],"metadata":{"id":"NX1fmcdfZkj2"}},{"cell_type":"code","source":["bool_array = np.array([0, 1, 0, 0, 1])\n","\n","print(bool_array.dtype)\n","print(bool_array)"],"metadata":{"id":"-vTEfSsZZV9Y"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Try to convert it to an array of data type 'bool' yourself."],"metadata":{"id":"n4WiXr9HbGwB"}},{"cell_type":"code","source":["bool_array = ...\n","\n","print(bool_array.dtype)\n","print(bool_array)"],"metadata":{"id":"sRYuJlY5Z4u4"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["By making it a boolean array the values of 0 and 1 are converted to False and True. "],"metadata":{"id":"_5T_O8nobgHp"}},{"cell_type":"markdown","source":["# Calculating with arrays\n","The whole point of Numpy is not to just store the data in arrays, but to do calculations on your data. The Numpy package is ideal for performing these calculations and much faster than doing it without Numpy.\n","\n","Let's explore some of the basic calculations you can perform on Numpy arrays. You can perform basic arithmetic on Numpy arrays."],"metadata":{"id":"yYGxT4OWy-nU"}},{"cell_type":"code","source":["my_array =  np.array([1,2,3,4,5])\n","\n","a = my_array + 1\n","b = my_array - 3\n","c = my_array * 2\n","d = my_array / 4\n","\n","\n","print(a, '\\n',\n","      b, '\\n',\n","      c, '\\n',\n","      d)"],"metadata":{"id":"3InAlQP0B0jc"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Interestingly, in one of these 4 cases the array changed its data type. Can you find which one, and where it changed towards? Why is that?"],"metadata":{"id":"Kr9LGaqzDbh5"}},{"cell_type":"code","source":["print(... .dtype)"],"metadata":{"id":"-SpwHH2gC3tF"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["If you found the correct answer you have noticed that division changed the data type, this is because we now suddenly need to store decimal numbers. It is also possible to do an 'integer division' using '//'. The results are then rounded to their lowest nearest integer (also called flooring)."],"metadata":{"id":"bKaeHiTZDv59"}},{"cell_type":"code","source":["my_array =  np.array([1,2,3,4,5])\n","\n","print(my_array // 4)"],"metadata":{"id":"Ebn1F0lnDvOO"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["You can also do arithmetic between 2 arrays. These work element-wise by default."],"metadata":{"id":"p08UevW35Av6"}},{"cell_type":"code","source":["array_1 = np.array([[1,2,3],\n","              [4,5,6]])\n","\n","array_2 = np.array([[10,11,12],\n","              [13,14,15]])\n","\n","array_3 = array_1 + array_2\n","\n","print(array_3)"],"metadata":{"id":"rWCNRF6R4_Aa"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Besides this basic arithmetic, there are many built-in functions in Numpy to do calculations on arrays. For example, to take the sum, product or difference of all elements in an array."],"metadata":{"id":"toKaQWcjE70I"}},{"cell_type":"code","source":["my_array =  np.array([1,2,3,4,5])\n","\n","print(np.sum(my_array))\n","print(np.prod(my_array))\n","print(np.diff(my_array))\n","print(np.mean(my_array))"],"metadata":{"id":"-L8JoWu6E7KA"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Lastly, you can perform multiplications between different arrays in Numpy. Something that is widely used in Machine Learning and Deep Learning. This operation often uses most of the computation time for an algorithm, because we use very large arrays (especially in Deep Learning), where millions or billions of numbers are multiplied.\n","\n","A dot product (an operation that multiplies the elements of 2 vectors and sums the result) is performed as follows."],"metadata":{"id":"JVUT359DGyrw"}},{"cell_type":"code","source":["array_1 = np.array([1,2,3,4,5])\n","array_2 = np.array([6,7,8,9,10])\n","\n","print(np.dot(array_1,array_2))\n","\n"],"metadata":{"id":"zORDcMEkGwir"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Matrix multiplication on 2D matrices (an operation where every row of the first matrix is multiplied with every column of the second matrix and the result is summed, similar to performing several dot products at the same time) is performed in the following way."],"metadata":{"id":"0XI43WW0KIns"}},{"cell_type":"code","source":["print(np.matmul(array_1.T,array_2))"],"metadata":{"id":"QwrWogsAKHUO"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Combining and reshaping arrays\n","You can combine two arrays if they match in at least 1 dimension. You can do this either horizontally or vertically.\n"],"metadata":{"id":"hG519gTz5fQK"}},{"cell_type":"code","source":["array_1 = np.array([[1,2,3],\n","               [4,5,6]])\n","\n","array_2 = np.array([[7,8,9],\n","               [10,11,12]])\n","\n","array_3 = np.hstack((array_1, array_2))\n","\n","array_4 = np.vstack((array_1, array_2))\n","\n","print(array_3)\n","\n","print(array_4)"],"metadata":{"id":"A_Tsr2CN6B8w"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Reshaping arrays also comes in handy often. You can change how the elements are distributed over the dimensions in this way."],"metadata":{"id":"SOF14rTJ6YGT"}},{"cell_type":"code","source":["array_1 = np.array([[1,2,3],\n","               [4,5,6]])\n","array_2 = np.reshape(array_1, (3,2))\n","\n","print(array_1)\n","print(array_2)"],"metadata":{"id":"uvSqlvxf6l0x"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Conditions on arrays\n","Often we would like to select data from an array that meets a certain condition. You can do this in two ways in Numpy. Either by creating a boolean array, or with the 'where' function, which returns the indexes of the elements that meet the given condition."],"metadata":{"id":"yDPTD-jJ9Yqv"}},{"cell_type":"code","source":["my_array = np.array([2,5,1,4,7,0,6,3])\n","\n","# Create boolean array where data meets condition\n","boolean = my_array>2\n","\n","# Find indices where data meets condition\n","indices = np.where(my_array>2)\n","\n","print(boolean)\n","print(my_array[boolean])\n","print(indices)\n","print(my_array[indices])"],"metadata":{"id":"tRMza6vAGDl6"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Exercise\n","To get used to the concepts you learned above, a small exercise. \n","\n","Below is an array representing the climate of different cities around the world. The rows represent different cities and the columns represent months. The values of the elements represent the average monthly temperature of a city.\n","\n","Can you answer the following questions using Numpy?\n","- Which city has the highest average temperature and which months is on average the warmest?\n","- Which city has the most months with a temperature above 20 degrees?\n","- Which city has the biggest difference between the coldest month and the warmest month?"],"metadata":{"id":"rGt64WQ4-3_e"}},{"cell_type":"code","source":["temperatures = np.array([[1,3,8,14,17,20,22,23,23,20,15,8],\n","                         [2,5,9,15,19,23,26,26,23,20,14,7],\n","                         [-1,-2,5,10,12,15,20,20,17,13,8,2],\n","                         [-5,-3,3,11,17,24,25,23,20,15,8,-2],\n","                         [8,12,15,17,19,20,22,21,20,15,12,8],\n","                         [5,12,21,25,28,28,30,29,25,17,12,5],\n","                         [-10,-5,-3,5,10,12,15,12,10,3,-4,-9],\n","                         [16,17,17,19,23,27,27,27,20,19,18,16],\n","                         [1,5,10,17,20,25,27,28,24,20,15,8],\n","                         [2,5,8,12,14,15,17,19,18,15,9,4]])"],"metadata":{"id":"mtmW2wINBHYt"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Discussion\n","Now that you have finished this notebook you should have familiarized yourself with the way Python looks and feels within the Google Colab environment and have a basic understanding of the Numpy package.\n","\n","In our next notebook we will look at our first data set and use Numpy to understand our data."],"metadata":{"id":"GdeQkoXOuXvZ"}}]}