00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "00_tensorflow_fundamentals.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOYUMC/O3v6QzxqR2kdTCN6",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ihsraham/TensorFlow/blob/main/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1abTGeKGv9ax"
      },
      "source": [
        "# In this notebook we're going to cover some of the most fundamental concepts of tensors using TensorFlow.\n",
        "\n",
        "More specifically, we're going to cover:\n",
        "* Introduction to tensors\n",
        "* Getting information from tensors\n",
        "* Manipulating tensors\n",
        "* Tensors & NumPy\n",
        "* Using @tf.function (a way to speed up your regular Python functions)\n",
        "* Using GPUs with TensorFlow (or TPUs)\n",
        "* Exercises to try for yourself"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "epjaW2OhwQzf"
      },
      "source": [
        "## Introduction to Tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jyjYj4mdxJmS",
        "outputId": "25903dfe-8e87-4fd1-a6f9-6f2c90d9cee4"
      },
      "source": [
        "# Import TensorFlow\n",
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "execution_count": 144,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.6.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CWIT-SJnxWKJ",
        "outputId": "aa326498-554c-4734-af0f-ce09ae002a4d"
      },
      "source": [
        "# Creating tensors with tf.constant()\n",
        "scalar = tf.constant(7)\n",
        "scalar"
      ],
      "execution_count": 145,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {},
          "execution_count": 145
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MC1zgz6hxyp9",
        "outputId": "569b45e5-869e-4593-a456-caf80e1cbc10"
      },
      "source": [
        "# Check the number of dimensions of a tensor (ndim stands for number of dimensions)\n",
        "scalar.ndim"
      ],
      "execution_count": 146,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 146
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VThX2jCLykdR",
        "outputId": "bca678cd-56c9-456a-8350-218368746b68"
      },
      "source": [
        "# Create a vector\n",
        "vector = tf.constant([10, 10])\n",
        "vector"
      ],
      "execution_count": 147,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 147
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NElAY4fbywgA",
        "outputId": "32473da3-163c-421e-d61e-19b3e1cc810a"
      },
      "source": [
        " # Check the dimension of our vector\n",
        " vector.ndim"
      ],
      "execution_count": 148,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 148
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NuGxuX5RzNNT",
        "outputId": "2fbf2944-aa83-4c5b-bbfa-624199f7b253"
      },
      "source": [
        "# Create a matrix (has more than 1 dimensions)\n",
        "matrix = tf.constant([[10, 7],\n",
        "                      [10, 7]])\n",
        "matrix"
      ],
      "execution_count": 149,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 149
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aywKgkcdzwjN",
        "outputId": "5c0ab32f-65bb-451e-ade7-5a5d8cef3a0d"
      },
      "source": [
        "matrix.ndim"
      ],
      "execution_count": 150,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 150
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wMJjjl-Cz8js",
        "outputId": "efde1fe2-6c94-4ab4-9212-c0815c2884b6"
      },
      "source": [
        "# Create another matrix\n",
        "another_matrix = tf.constant([[10., 7.],\n",
        "                              [3., 2.],\n",
        "                              [8., 9.]], dtype=tf.float16)\n",
        "another_matrix"
      ],
      "execution_count": 151,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  9.]], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 151
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F2pmZihO1TyC",
        "outputId": "373a28e9-724b-43f2-e437-5ec0d370c1d1"
      },
      "source": [
        "# What's the number of dimensions of another_matrix?\n",
        "another_matrix.ndim"
      ],
      "execution_count": 152,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 152
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dgOuLF-01oJE",
        "outputId": "a08d6723-5c0d-432c-ce44-83cf96b81dcd"
      },
      "source": [
        "# Let's create a tensor\n",
        "tensor = tf.constant([[[1, 2, 3], [4, 5, 6]], \n",
        "                      [[7, 8, 9], [10, 11, 12]], \n",
        "                      [[13, 14, 15], [16, 17, 18]]])\n",
        "tensor"
      ],
      "execution_count": 153,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 4,  5,  6]],\n",
              "\n",
              "       [[ 7,  8,  9],\n",
              "        [10, 11, 12]],\n",
              "\n",
              "       [[13, 14, 15],\n",
              "        [16, 17, 18]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 153
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UiStstZ929XD",
        "outputId": "4f734a5b-9ede-4f9f-bd59-15e5d012fb79"
      },
      "source": [
        "tensor.ndim"
      ],
      "execution_count": 154,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 154
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i2IUGD0F3BSz"
      },
      "source": [
        "What we've created so far:\n",
        "\n",
        "* Scalar: a single number\n",
        "* Vector: a number with direction (e.g. wind speed and direction)\n",
        "* Matrix: a 2-dimensional array of numbers\n",
        "* Tensor: an n-dimensional array of numbers (where n can be any number, a 0-dimensional tensor is a scalar, an 1-dimensional tensor is a vector)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQEU0L-C4K0x"
      },
      "source": [
        "\n",
        "#### Creating tensors with `tf.Variable`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Eh29Jw6C4Xno",
        "outputId": "1ef27f06-9fec-4ca4-9aa6-13e58e9d0e8b"
      },
      "source": [
        "# Create the same tensor with tf.Variable() as above\n",
        "changeable_tensor = tf.Variable([10, 7])\n",
        "unchangeable_tensor = tf.constant([10, 7])\n",
        "changeable_tensor, unchangeable_tensor"
      ],
      "execution_count": 155,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 155
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 203
        },
        "id": "rMJdG3AT4aJr",
        "outputId": "46c682cc-7e56-4322-9187-935c27ab320f"
      },
      "source": [
        "# Let's try change one of the elements in our changeable tensor\n",
        "changeable_tensor[0] = 7\n",
        "changeable_tensor"
      ],
      "execution_count": 157,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-157-55cb07e37ee9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's try change one of the elements in our changeable tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'ResourceVariable' object does not support item assignment"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a7sCE0jx5yat",
        "outputId": "b1c078d8-2180-40fa-b489-36230f708c0d"
      },
      "source": [
        "# How about we try .assign() ?\n",
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "execution_count": 158,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([7, 7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 158
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 325
        },
        "id": "jggfAN4e6Cfy",
        "outputId": "045d0cc4-aa9e-45cd-c822-b85628cac5fc"
      },
      "source": [
        "#  Let's try to change our unchangeable tensor\n",
        "unchangeable_tensor[0].assign(7)\n",
        "unchangeable_tensor"
      ],
      "execution_count": 160,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-160-5abef06324c4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m#  Let's try to change our unchangeable tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massign\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m    399\u001b[0m         \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpython\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumpy_ops\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnp_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    400\u001b[0m         np_config.enable_numpy_behavior()\"\"\".format(type(self).__name__, name))\n\u001b[0;32m--> 401\u001b[0;31m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__getattribute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    402\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    403\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mstaticmethod\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z3Qu_aRD6SRG"
      },
      "source": [
        "### Creating random tensors\n",
        "\n",
        "Random tensors are tensors of some arbitrary size which contain random numbers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B7SARJ-17Bv6",
        "outputId": "69a41a5b-e4d5-44a3-aa6b-d7cc5a38debe"
      },
      "source": [
        "# Creating two random (but the same) tensors\n",
        "random_1 = tf.random.Generator.from_seed(7) # set seed for reproducibility\n",
        "random_1 = random_1.normal(shape=(3, 2))\n",
        "random_2 = tf.random.Generator.from_seed(7)\n",
        "random_2 = random_2.normal(shape=(3, 2))\n",
        "\n",
        "# Are they equal?\n",
        "random_1, random_2, random_1==random_2"
      ],
      "execution_count": 161,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-1.3240396 ,  0.28785667],\n",
              "        [-0.8757901 , -0.08857018],\n",
              "        [ 0.69211644,  0.84215707]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-1.3240396 ,  0.28785667],\n",
              "        [-0.8757901 , -0.08857018],\n",
              "        [ 0.69211644,  0.84215707]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {},
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M-MQr3NV9NVw"
      },
      "source": [
        "### Shuffle the order of elements in a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hJewX2fiBd3D",
        "outputId": "d9e7f906-99ef-4dd1-ba00-877173b02115"
      },
      "source": [
        "# Shuffle a tensor (valuable for when you want to shuffle your data so the inherent order doesn't affect learning)\n",
        "not_shuffled = tf.constant([[10, 7],\n",
        "                           [3, 4],\n",
        "                           [2, 5]])\n",
        "# Shuffle our non-shuffled tensor\n",
        "tf.random.shuffle(not_shuffled)"
      ],
      "execution_count": 162,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 2,  5],\n",
              "       [10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 162
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6_Qw_AzwDOD1",
        "outputId": "f7f87f62-0708-409d-facc-376ca44d2353"
      },
      "source": [
        "# Shuffle our non-shuffled tensor\n",
        "tf.random.set_seed(42)\n",
        "tf.random.shuffle(not_shuffled, seed = 42)"
      ],
      "execution_count": 163,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 163
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SwYJW9umCIun",
        "outputId": "2bc6fb39-9dbd-4859-9e24-61a9bcaacb5d"
      },
      "source": [
        "not_shuffled"
      ],
      "execution_count": 164,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 164
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j0AFlObbCXNI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "800167c7-0fba-434c-9f86-56b65190507b"
      },
      "source": [
        "first_random = tf.random.Generator.from_seed(56)\n",
        "first_random = first_random.normal(shape=(3,3,2))\n",
        "first_random"
      ],
      "execution_count": 165,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3, 2), dtype=float32, numpy=\n",
              "array([[[-0.67958915,  0.44723678],\n",
              "        [-0.17811584,  0.24882518],\n",
              "        [ 0.49689844,  0.8259971 ]],\n",
              "\n",
              "       [[ 1.0340209 , -0.24918637],\n",
              "        [-1.5780283 , -0.92161775],\n",
              "        [ 0.268676  ,  0.9418312 ]],\n",
              "\n",
              "       [[-0.9465717 , -0.7108357 ],\n",
              "        [ 1.2995545 , -0.6149066 ],\n",
              "        [-1.4713507 , -0.10086866]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V_iGNDdbQEK1",
        "outputId": "fd12f782-2b81-4bbc-c10f-5309381b49d4"
      },
      "source": [
        "\n",
        "tf.random.shuffle(first_random)"
      ],
      "execution_count": 166,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3, 2), dtype=float32, numpy=\n",
              "array([[[ 1.0340209 , -0.24918637],\n",
              "        [-1.5780283 , -0.92161775],\n",
              "        [ 0.268676  ,  0.9418312 ]],\n",
              "\n",
              "       [[-0.9465717 , -0.7108357 ],\n",
              "        [ 1.2995545 , -0.6149066 ],\n",
              "        [-1.4713507 , -0.10086866]],\n",
              "\n",
              "       [[-0.67958915,  0.44723678],\n",
              "        [-0.17811584,  0.24882518],\n",
              "        [ 0.49689844,  0.8259971 ]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 166
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BUiNiUoUBF1W",
        "outputId": "7387a12c-d629-452d-9f70-c7a24eae9cc2"
      },
      "source": [
        "second_random = tf.random.Generator.from_seed(56)\n",
        "second_random = second_random.normal(shape=(3,3,3), dtype=tf.float16)\n",
        "second_random"
      ],
      "execution_count": 167,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3, 3), dtype=float16, numpy=\n",
              "array([[[-0.6797,  0.4473, -0.1781],\n",
              "        [ 0.2488,  0.4968,  0.826 ],\n",
              "        [ 1.034 , -0.2491, -1.578 ]],\n",
              "\n",
              "       [[-0.9214,  0.2686,  0.942 ],\n",
              "        [-0.947 , -0.711 ,  1.3   ],\n",
              "        [-0.6147, -1.472 , -0.1009]],\n",
              "\n",
              "       [[ 0.9604, -1.637 ,  0.1766],\n",
              "        [-0.6704, -0.5034, -1.204 ],\n",
              "        [-2.525 ,  0.0508,  1.722 ]]], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 167
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FQBaWU3TQMkc",
        "outputId": "646b8e52-a040-41b5-8e65-93e4c387bd66"
      },
      "source": [
        "\n",
        "tf.random.shuffle(second_random, seed=42)"
      ],
      "execution_count": 168,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3, 3), dtype=float16, numpy=\n",
              "array([[[-0.6797,  0.4473, -0.1781],\n",
              "        [ 0.2488,  0.4968,  0.826 ],\n",
              "        [ 1.034 , -0.2491, -1.578 ]],\n",
              "\n",
              "       [[-0.9214,  0.2686,  0.942 ],\n",
              "        [-0.947 , -0.711 ,  1.3   ],\n",
              "        [-0.6147, -1.472 , -0.1009]],\n",
              "\n",
              "       [[ 0.9604, -1.637 ,  0.1766],\n",
              "        [-0.6704, -0.5034, -1.204 ],\n",
              "        [-2.525 ,  0.0508,  1.722 ]]], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 168
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4DU3w-4SB9j_",
        "outputId": "5ec16678-c179-493b-8271-255cdc7774bd"
      },
      "source": [
        "\n",
        "third_random = tf.random.Generator.from_seed(5)\n",
        "counts = [10., 20.,399.]\n",
        "# Probability of success.\n",
        "probs = [0.2]\n",
        "third_random = third_random.binomial(shape=[3], counts=counts, probs=probs)\n",
        "third_random"
      ],
      "execution_count": 169,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=int32, numpy=array([ 2,  4, 81], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 169
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rBa5uoh5QgIT",
        "outputId": "e87d187e-ac3f-483e-a934-0f6a07868685"
      },
      "source": [
        "tf.random.shuffle(third_random, seed=42)"
      ],
      "execution_count": 170,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=int32, numpy=array([81,  2,  4], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 170
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PyuOXTwbJUp0",
        "outputId": "7ba96299-3481-4190-e2c3-698d8fc4178d"
      },
      "source": [
        "fourth_random = tf.random.Generator.from_seed(45)\n",
        "fourth_random = fourth_random.uniform(shape=(3,2,5))\n",
        "fourth_random"
      ],
      "execution_count": 171,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 5), dtype=float32, numpy=\n",
              "array([[[0.86540747, 0.8862978 , 0.27795732, 0.8857763 , 0.2179842 ],\n",
              "        [0.29115117, 0.03953862, 0.8136791 , 0.8139852 , 0.52180684]],\n",
              "\n",
              "       [[0.12496924, 0.5488483 , 0.7755773 , 0.6184403 , 0.24936223],\n",
              "        [0.89341843, 0.28422844, 0.70332646, 0.2622137 , 0.4432162 ]],\n",
              "\n",
              "       [[0.466465  , 0.05981874, 0.40098202, 0.69292355, 0.1284684 ],\n",
              "        [0.22770369, 0.33691216, 0.5329138 , 0.5914326 , 0.21738243]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 171
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gSpGySamQ0qd",
        "outputId": "cd58ca61-87e0-4caa-d3fb-b7b7ff2e3caa"
      },
      "source": [
        "tf.random.shuffle(fourth_random,seed=1233)"
      ],
      "execution_count": 172,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 5), dtype=float32, numpy=\n",
              "array([[[0.86540747, 0.8862978 , 0.27795732, 0.8857763 , 0.2179842 ],\n",
              "        [0.29115117, 0.03953862, 0.8136791 , 0.8139852 , 0.52180684]],\n",
              "\n",
              "       [[0.12496924, 0.5488483 , 0.7755773 , 0.6184403 , 0.24936223],\n",
              "        [0.89341843, 0.28422844, 0.70332646, 0.2622137 , 0.4432162 ]],\n",
              "\n",
              "       [[0.466465  , 0.05981874, 0.40098202, 0.69292355, 0.1284684 ],\n",
              "        [0.22770369, 0.33691216, 0.5329138 , 0.5914326 , 0.21738243]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 172
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O4e4nvKmNSAa",
        "outputId": "8bedade5-55a9-4bf3-cf40-d911ae9c1fcd"
      },
      "source": [
        "fifth_random = tf.random.Generator.from_seed(45)\n",
        "fifth_random = fifth_random.uniform(shape=(3,2,4,5))\n",
        "fifth_random"
      ],
      "execution_count": 173,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0.86540747, 0.8862978 , 0.27795732, 0.8857763 , 0.2179842 ],\n",
              "         [0.29115117, 0.03953862, 0.8136791 , 0.8139852 , 0.52180684],\n",
              "         [0.12496924, 0.5488483 , 0.7755773 , 0.6184403 , 0.24936223],\n",
              "         [0.89341843, 0.28422844, 0.70332646, 0.2622137 , 0.4432162 ]],\n",
              "\n",
              "        [[0.466465  , 0.05981874, 0.40098202, 0.69292355, 0.1284684 ],\n",
              "         [0.22770369, 0.33691216, 0.5329138 , 0.5914326 , 0.21738243],\n",
              "         [0.5322075 , 0.05148339, 0.03951418, 0.41866875, 0.78939915],\n",
              "         [0.04384279, 0.96955836, 0.49116182, 0.3706199 , 0.33535397]]],\n",
              "\n",
              "\n",
              "       [[[0.02760839, 0.99757504, 0.52172756, 0.20307171, 0.7182547 ],\n",
              "         [0.84263575, 0.9542595 , 0.90112185, 0.62839293, 0.08619452],\n",
              "         [0.5679928 , 0.28763676, 0.18828917, 0.6658715 , 0.6190208 ],\n",
              "         [0.04422736, 0.49626625, 0.6474861 , 0.3557682 , 0.32033885]],\n",
              "\n",
              "        [[0.33705008, 0.7391062 , 0.16512096, 0.41555858, 0.78638244],\n",
              "         [0.45899415, 0.42686343, 0.5630441 , 0.04115057, 0.7532015 ],\n",
              "         [0.18315244, 0.30807555, 0.30482936, 0.08611703, 0.7526083 ],\n",
              "         [0.42415357, 0.7877505 , 0.11611497, 0.35255086, 0.50718856]]],\n",
              "\n",
              "\n",
              "       [[[0.16467738, 0.54108894, 0.5765736 , 0.16844285, 0.8007604 ],\n",
              "         [0.34483027, 0.30735934, 0.9700769 , 0.43336582, 0.51829636],\n",
              "         [0.8555572 , 0.21962428, 0.76091194, 0.04762888, 0.5131633 ],\n",
              "         [0.97339594, 0.33341944, 0.03152311, 0.28874612, 0.58736026]],\n",
              "\n",
              "        [[0.75669694, 0.7060174 , 0.5588795 , 0.40088904, 0.8377521 ],\n",
              "         [0.7651175 , 0.5402924 , 0.52445555, 0.8965409 , 0.4253844 ],\n",
              "         [0.6824161 , 0.17499602, 0.4163201 , 0.8321508 , 0.313666  ],\n",
              "         [0.15126252, 0.23609889, 0.56451845, 0.44649565, 0.41026497]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 173
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nhlj9PFXQ_gL",
        "outputId": "95b01377-110c-46ae-bf82-ae300a605adc"
      },
      "source": [
        "tf.random.shuffle(fifth_random)"
      ],
      "execution_count": 174,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0.16467738, 0.54108894, 0.5765736 , 0.16844285, 0.8007604 ],\n",
              "         [0.34483027, 0.30735934, 0.9700769 , 0.43336582, 0.51829636],\n",
              "         [0.8555572 , 0.21962428, 0.76091194, 0.04762888, 0.5131633 ],\n",
              "         [0.97339594, 0.33341944, 0.03152311, 0.28874612, 0.58736026]],\n",
              "\n",
              "        [[0.75669694, 0.7060174 , 0.5588795 , 0.40088904, 0.8377521 ],\n",
              "         [0.7651175 , 0.5402924 , 0.52445555, 0.8965409 , 0.4253844 ],\n",
              "         [0.6824161 , 0.17499602, 0.4163201 , 0.8321508 , 0.313666  ],\n",
              "         [0.15126252, 0.23609889, 0.56451845, 0.44649565, 0.41026497]]],\n",
              "\n",
              "\n",
              "       [[[0.86540747, 0.8862978 , 0.27795732, 0.8857763 , 0.2179842 ],\n",
              "         [0.29115117, 0.03953862, 0.8136791 , 0.8139852 , 0.52180684],\n",
              "         [0.12496924, 0.5488483 , 0.7755773 , 0.6184403 , 0.24936223],\n",
              "         [0.89341843, 0.28422844, 0.70332646, 0.2622137 , 0.4432162 ]],\n",
              "\n",
              "        [[0.466465  , 0.05981874, 0.40098202, 0.69292355, 0.1284684 ],\n",
              "         [0.22770369, 0.33691216, 0.5329138 , 0.5914326 , 0.21738243],\n",
              "         [0.5322075 , 0.05148339, 0.03951418, 0.41866875, 0.78939915],\n",
              "         [0.04384279, 0.96955836, 0.49116182, 0.3706199 , 0.33535397]]],\n",
              "\n",
              "\n",
              "       [[[0.02760839, 0.99757504, 0.52172756, 0.20307171, 0.7182547 ],\n",
              "         [0.84263575, 0.9542595 , 0.90112185, 0.62839293, 0.08619452],\n",
              "         [0.5679928 , 0.28763676, 0.18828917, 0.6658715 , 0.6190208 ],\n",
              "         [0.04422736, 0.49626625, 0.6474861 , 0.3557682 , 0.32033885]],\n",
              "\n",
              "        [[0.33705008, 0.7391062 , 0.16512096, 0.41555858, 0.78638244],\n",
              "         [0.45899415, 0.42686343, 0.5630441 , 0.04115057, 0.7532015 ],\n",
              "         [0.18315244, 0.30807555, 0.30482936, 0.08611703, 0.7526083 ],\n",
              "         [0.42415357, 0.7877505 , 0.11611497, 0.35255086, 0.50718856]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 174
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eDdKZgwtSzEY",
        "outputId": "ab7ab4ae-0130-48f5-b444-88c0d940a466"
      },
      "source": [
        "not_shuffled"
      ],
      "execution_count": 175,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 175
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kuUfk-hUPkBy",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "454b170b-21a7-4b30-b859-63042ccb3bf6"
      },
      "source": [
        "tf.random.set_seed(42) # global level random seed\n",
        "tf.random.shuffle(not_shuffled, seed=42)  # operation level random seed"
      ],
      "execution_count": 176,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 176
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l7DxQl8VS3Do"
      },
      "source": [
        "It looks like if we want our shuffled tensors to be in the same order, we've got to use the global level random seed as well as the operation level random seed:\n",
        "> Rule 4: \"If both the global and the operation seed are set: Both seeds are used in conjunction to determine the random sequence.\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0OGhpWt7T4e-"
      },
      "source": [
        "### Other ways to make tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Px_irwxuUIFG",
        "outputId": "9119d4f9-e5c3-4fc2-8a5b-7d441d8bf8d2"
      },
      "source": [
        "tf.ones([10, 7])"
      ],
      "execution_count": 177,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 177
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ylBb2JpnURTr",
        "outputId": "cc25dde3-6a9e-4790-d22d-643bbadedfe0"
      },
      "source": [
        "# Create a tensor of all zeroes\n",
        "tf.zeros(shape=(3, 4))"
      ],
      "execution_count": 178,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 178
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4ACHolvUUpdf"
      },
      "source": [
        "### Turn NumPy arrays into tensors\n",
        "The main difference between NumPy arrays and TensorFlow tensors is that tensors can be run on a GPU (much faster for numerical computing)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0yqIs1fQUfg8",
        "outputId": "2f3958bc-797c-4557-c52d-109935c98c80"
      },
      "source": [
        "# You can also turn NumPy arrays into tensors\n",
        "import numpy as np\n",
        "numpy_A = np.arange(1, 25, dtype=np.int32) # create a NumPy array between 1 and 25\n",
        "numpy_A\n",
        "\n",
        "# X = tf.constant(some_matrix) # capital for matrix or tensor\n",
        "# y = tf.constant(vector) # non-capital for vector"
      ],
      "execution_count": 179,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "       18, 19, 20, 21, 22, 23, 24], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 179
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uFjnMn8yXPsi",
        "outputId": "90d13be3-c600-40b2-f8ca-19ee4da2069f"
      },
      "source": [
        "A = tf.constant(numpy_A, shape=(2, 3, 4))\n",
        "B = tf.constant(numpy_A)\n",
        "A, B"
      ],
      "execution_count": 180,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 3, 4), dtype=int32, numpy=\n",
              " array([[[ 1,  2,  3,  4],\n",
              "         [ 5,  6,  7,  8],\n",
              "         [ 9, 10, 11, 12]],\n",
              " \n",
              "        [[13, 14, 15, 16],\n",
              "         [17, 18, 19, 20],\n",
              "         [21, 22, 23, 24]]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(24,), dtype=int32, numpy=\n",
              " array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 180
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zegSS0ZkXZ_b",
        "outputId": "bbfe9b80-62ed-4ec8-9cc4-20624c8edd2a"
      },
      "source": [
        "C = tf.constant(numpy_A, shape=(3, 8))\n",
        "C"
      ],
      "execution_count": 181,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 8), dtype=int32, numpy=\n",
              "array([[ 1,  2,  3,  4,  5,  6,  7,  8],\n",
              "       [ 9, 10, 11, 12, 13, 14, 15, 16],\n",
              "       [17, 18, 19, 20, 21, 22, 23, 24]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 181
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gA3Sa2vlYeKd",
        "outputId": "fc767ba6-2d19-4e5f-f496-a17ab0affbd4"
      },
      "source": [
        "D = tf.constant(numpy_A, shape=(2, 2, 2, 3))\n",
        "D"
      ],
      "execution_count": 182,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 3), dtype=int32, numpy=\n",
              "array([[[[ 1,  2,  3],\n",
              "         [ 4,  5,  6]],\n",
              "\n",
              "        [[ 7,  8,  9],\n",
              "         [10, 11, 12]]],\n",
              "\n",
              "\n",
              "       [[[13, 14, 15],\n",
              "         [16, 17, 18]],\n",
              "\n",
              "        [[19, 20, 21],\n",
              "         [22, 23, 24]]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 182
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ngEwcnjYq_r"
      },
      "source": [
        "### Getting information from tensors\n",
        "\n",
        "When dealing with tensors you probably want to be aware of the following attributes:\n",
        "* Shape\n",
        "* Rank\n",
        "* Axis or dimension\n",
        "* Size"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YyqviTEsY6Rz",
        "outputId": "9183b2be-cc54-4023-c893-13bc3f8608bb"
      },
      "source": [
        "# Create a rank 4 tensor (4 dimensions)\n",
        "rank_4_tensor = tf.zeros(shape=[2, 3, 4, 5])\n",
        "rank_4_tensor"
      ],
      "execution_count": 183,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 183
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2ecqWW34aU3K",
        "outputId": "4bdb3d16-fcd9-41de-d316-19069407ee37"
      },
      "source": [
        "rank_4_tensor[0]"
      ],
      "execution_count": 184,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4, 5), dtype=float32, numpy=\n",
              "array([[[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]],\n",
              "\n",
              "       [[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]],\n",
              "\n",
              "       [[0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0., 0.]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 184
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IfE4WlPRa2NO",
        "outputId": "0b3fb1ec-58b4-4501-fbfb-3b0d6445781a"
      },
      "source": [
        "rank_4_tensor.shape, rank_4_tensor.ndim, tf.size(rank_4_tensor)"
      ],
      "execution_count": 185,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 3, 4, 5]), 4, <tf.Tensor: shape=(), dtype=int32, numpy=120>)"
            ]
          },
          "metadata": {},
          "execution_count": 185
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UX4qTuMWbDiF",
        "outputId": "0e4978ff-4ff6-4a85-d2ed-f841620b642c"
      },
      "source": [
        "# Get various attributes of our tensor\n",
        "print(\"Datatype of every element:\", rank_4_tensor.dtype)\n",
        "print(\"Number of dimensions (rank):\", rank_4_tensor.ndim)\n",
        "print(\"Shape of tensor:\", rank_4_tensor.shape)\n",
        "print(\"Elements along the 0 axis:\", rank_4_tensor.shape[0])\n",
        "print(\"Elements along the last axis:\", rank_4_tensor.shape[-1])\n",
        "print(\"Total number of elements in our tensor:\", tf.size(rank_4_tensor).numpy())"
      ],
      "execution_count": 186,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Datatype of every element: <dtype: 'float32'>\n",
            "Number of dimensions (rank): 4\n",
            "Shape of tensor: (2, 3, 4, 5)\n",
            "Elements along the 0 axis: 2\n",
            "Elements along the last axis: 5\n",
            "Total number of elements in our tensor: 120\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZzRxYFAicJnR"
      },
      "source": [
        "### Indexing tensors\n",
        "\n",
        "Tensors can be indexed just like Python lists."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DIG81vlKE1mX",
        "outputId": "c5eccb56-d08f-417e-be86-65895ff2d844"
      },
      "source": [
        "some_list = [1, 2, 3, 4]\n",
        "some_list[:2]"
      ],
      "execution_count": 187,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1, 2]"
            ]
          },
          "metadata": {},
          "execution_count": 187
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PaFRXmOXEbHk",
        "outputId": "ed3595c8-9321-4769-f178-a67fc0a945bd"
      },
      "source": [
        "# Get the first 2 elements of each dimension\n",
        "rank_4_tensor[:2, :2, :2, :2]"
      ],
      "execution_count": 188,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 188
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7LfQ324MFlo-",
        "outputId": "19f40f88-8666-404f-8512-4d6be650e066"
      },
      "source": [
        "some_list[:1]"
      ],
      "execution_count": 189,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1]"
            ]
          },
          "metadata": {},
          "execution_count": 189
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mQ3AZ1kPE8sU",
        "outputId": "3eb41674-68bb-41b8-dd59-fe02a0c2e012"
      },
      "source": [
        "# Get the first element from each dimension from each index except for the final one\n",
        "rank_4_tensor[:1, :1, :1, :]"
      ],
      "execution_count": 190,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 190
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aeveVhQiF55j",
        "outputId": "771cecb4-8d4d-4ef8-bd52-912a9901d7d0"
      },
      "source": [
        "rank_4_tensor[:, :1, :1, :]"
      ],
      "execution_count": 191,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 1, 1, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 191
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SqyrCI6pFtto",
        "outputId": "31bd0622-9448-4103-93e6-344308e5a7e6"
      },
      "source": [
        "# Create a rank 2 tensor (2 dimensions)\n",
        "rank_2_tensor = tf.constant([[10, 7],\n",
        "                             [3, 4]])\n",
        "rank_2_tensor.shape, rank_2_tensor.ndim"
      ],
      "execution_count": 192,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 2]), 2)"
            ]
          },
          "metadata": {},
          "execution_count": 192
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t0wbZ2ysHl53",
        "outputId": "2d2deb61-73ef-4d88-921b-3b154817d264"
      },
      "source": [
        "some_list, some_list[-1]"
      ],
      "execution_count": 193,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "([1, 2, 3, 4], 4)"
            ]
          },
          "metadata": {},
          "execution_count": 193
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ByJHDa8ZG8Cd",
        "outputId": "c25e6a70-270d-4ddf-b33f-baf85f13a671"
      },
      "source": [
        "# Get the last item of each row of our rank 2 tensor\n",
        "rank_2_tensor[:, -1]"
      ],
      "execution_count": 194,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 194
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wy75-A4QHwYV",
        "outputId": "e144dbbb-431d-4896-d91a-04d3558b09d6"
      },
      "source": [
        "# Add an extra dimension to our rank 2 tensor without changing the information\n",
        "rank_3_tensor = rank_2_tensor[..., tf.newaxis] # equivalent to [:, :, tf.newaxis]\n",
        "rank_3_tensor"
      ],
      "execution_count": 195,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 195
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g63WMJRSImrR",
        "outputId": "2578fb9d-13a8-4b57-fc8d-f61567314046"
      },
      "source": [
        "# Alternative to tf.newaxis\n",
        "tf.expand_dims(rank_2_tensor, axis=-1) # \"-1\" means expand the final axis"
      ],
      "execution_count": 196,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 196
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-ppfbXe7LzEx",
        "outputId": "b7fc70b2-797e-440f-e641-2f530346ebb7"
      },
      "source": [
        "tf.expand_dims(rank_2_tensor, axis=0) # expand the 0-axis"
      ],
      "execution_count": 197,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 2, 2), dtype=int32, numpy=\n",
              "array([[[10,  7],\n",
              "        [ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 197
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iGSHnQZ4MFwQ",
        "outputId": "e3a99eca-922d-491b-a1b7-ab1e6bac4c20"
      },
      "source": [
        "tf.expand_dims(rank_2_tensor, axis=1) # expand the 1-axis"
      ],
      "execution_count": 198,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 1, 2), dtype=int32, numpy=\n",
              "array([[[10,  7]],\n",
              "\n",
              "       [[ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 198
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5gglPOuEMOlg"
      },
      "source": [
        "### Manipulating tensors (tensor operations)\n",
        "\n",
        "**Basic operations**\n",
        "\n",
        "`+`, `-`, `*`, `/`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5KgOQVoQMyhv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2e56b0fe-91db-476a-aa02-0db6cee9226d"
      },
      "source": [
        "# You can add values to a tensor using the addition operator\n",
        "tensor = tf.constant([[10, 7], \n",
        "                      [3, 4]])\n",
        "tensor + 10"
      ],
      "execution_count": 199,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 17],\n",
              "       [13, 14]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 199
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bqeWOyApZuKh",
        "outputId": "4e76f899-6390-4595-e7a2-ba74c83ff171"
      },
      "source": [
        "# Original tensor is unchanged\n",
        "tensor"
      ],
      "execution_count": 200,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 200
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ypn06aj7Z0kd",
        "outputId": "5beae275-ef9d-4988-93ee-4926e85bbae0"
      },
      "source": [
        "# Multiplication also works\n",
        "tensor * 10"
      ],
      "execution_count": 201,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 201
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-86-0lpXaC_d",
        "outputId": "6d335d65-cb0a-4699-f064-eb71dc4c209d"
      },
      "source": [
        "# SUbtraction also works\n",
        "tensor - 10"
      ],
      "execution_count": 202,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 0, -3],\n",
              "       [-7, -6]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 202
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wy_LWv9OaIDz",
        "outputId": "106c2567-e15f-4743-b19e-49ea60eb98c0"
      },
      "source": [
        "# We can use the tensorflow built-in function too\n",
        "# Use tensorflow.math to speed up operations\n",
        "tf.multiply(tensor, 10)"
      ],
      "execution_count": 203,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 203
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rIqoKg0naQpM",
        "outputId": "22644dfc-91b7-4fb1-947e-4c8abb7d0471"
      },
      "source": [
        "# Original is unchanged\n",
        "tensor"
      ],
      "execution_count": 204,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 204
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X8EnOukJaWiR"
      },
      "source": [
        "**Matrix Multiplication**\n",
        "\n",
        "In machine learning, matrix multiplication is one of the most common tensor operations.\n",
        "\n",
        "There are two rules all tensors (or matrices) need to fulfil if we're going yo matrix multiply them:\n",
        "\n",
        "\n",
        "1.   The inner dimensions must match\n",
        "2.   The resulting matrix has the shape of the inner dimensions\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oTMMsG4Fatbb",
        "outputId": "1d4ebb8a-05d7-4778-994a-259a73f6e99f"
      },
      "source": [
        "# Matrix multiplication in TensorFlow\n",
        "print(tensor)\n",
        "tf.matmul(tensor, tensor)"
      ],
      "execution_count": 205,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[[10  7]\n",
            " [ 3  4]], shape=(2, 2), dtype=int32)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 205
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CeJ8hTd6gz3n",
        "outputId": "31eac849-2397-46fd-adc3-ed4be7886410"
      },
      "source": [
        "# * does element wise multiplication\n",
        "tensor * tensor"
      ],
      "execution_count": 206,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  49],\n",
              "       [  9,  16]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 206
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HUV4Ev6HhCEW",
        "outputId": "c2ffab53-49e2-4b20-ab40-08a9c02f726a"
      },
      "source": [
        "tensor1 = tf.constant([[1, 2, 5],\n",
        "                      [7, 2, 1],\n",
        "                      [3, 3, 3]])\n",
        "tensor2 = tf.constant([[3, 5],\n",
        "                      [6, 7],\n",
        "                      [1, 8]])\n",
        "tf.matmul(tensor1, tensor2)"
      ],
      "execution_count": 207,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[20, 59],\n",
              "       [34, 57],\n",
              "       [30, 60]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 207
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eLfhBgQagjXh",
        "outputId": "656afa82-37ca-4db4-bd84-45cee662f9eb"
      },
      "source": [
        "# Matrix multiplication with Python operator \"@\"\n",
        "tensor1 @ tensor2"
      ],
      "execution_count": 208,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[20, 59],\n",
              "       [34, 57],\n",
              "       [30, 60]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 208
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ayM6oEnkh3Jj",
        "outputId": "64c4519b-ff39-480e-8f37-81241fb8df10"
      },
      "source": [
        "# Create a (3, 2) tensor\n",
        "X = tf.constant([[1, 2],\n",
        "                 [3, 4],\n",
        "                 [5, 6]])\n",
        "# Create another (3, 2) tensor\n",
        "Y = tf.constant([[7,8],\n",
        "                 [9, 10],\n",
        "                 [11, 12]])\n",
        "X, Y"
      ],
      "execution_count": 209,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 209
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 244
        },
        "id": "zvXWYkdQjFkh",
        "outputId": "2fa7181e-e50c-458a-cc96-fc9ec9b9e6f2"
      },
      "source": [
        "# Let's try to matmul tensors of same shape\n",
        "tf.matmul(X, Y)"
      ],
      "execution_count": 211,
      "outputs": [
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-211-462296f61fdf>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's try to matmul tensors of same shape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mY\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    204\u001b[0m     \u001b[0;34m\"\"\"Call target, and fall back on dispatchers if there is a TypeError.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    205\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 206\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    207\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mTypeError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    208\u001b[0m       \u001b[0;31m# Note: convert_to_eager_tensor currently raises a ValueError, not a\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mmatmul\u001b[0;34m(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, output_type, name)\u001b[0m\n\u001b[1;32m   3653\u001b[0m       \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3654\u001b[0m         return gen_math_ops.mat_mul(\n\u001b[0;32m-> 3655\u001b[0;31m             a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)\n\u001b[0m\u001b[1;32m   3656\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3657\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py\u001b[0m in \u001b[0;36mmat_mul\u001b[0;34m(a, b, transpose_a, transpose_b, name)\u001b[0m\n\u001b[1;32m   5694\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5695\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5696\u001b[0;31m       \u001b[0m_ops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5697\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_FallbackException\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5698\u001b[0m       \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   6939\u001b[0m   \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6940\u001b[0m   \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6941\u001b[0;31m   \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6942\u001b[0m   \u001b[0;31m# pylint: enable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6943\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/six.py\u001b[0m in \u001b[0;36mraise_from\u001b[0;34m(value, from_value)\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: In[0] mismatch In[1] shape: 2 vs. 3: [3,2] [3,2] 0 0 [Op:MatMul]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Nh19_-cFjP0j",
        "outputId": "affcfb19-fdf0-4f67-d994-63fb471f227e"
      },
      "source": [
        "# Let's change the shape of Y\n",
        "tf.reshape(Y, shape=(2, 3))"
      ],
      "execution_count": 212,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[ 7,  8,  9],\n",
              "       [10, 11, 12]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 212
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mLTcUqfStWAn",
        "outputId": "460a1244-c8a9-4d9b-c2c6-9ff173a677e8"
      },
      "source": [
        "# Try to matrix multiply X by reshaped Y\n",
        "X @ tf.reshape(Y, shape=(2, 3))"
      ],
      "execution_count": 213,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 213
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mIF81a4fts3f",
        "outputId": "5bf33967-930f-48d5-99a0-a3c54944bd96"
      },
      "source": [
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "execution_count": 214,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 214
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "38TjkR_QuT16",
        "outputId": "54fccaa7-cdda-4336-c07d-ad38e797917b"
      },
      "source": [
        "tf.matmul(tf.reshape(X, shape=(2, 3)), Y)"
      ],
      "execution_count": 215,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 58,  64],\n",
              "       [139, 154]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 215
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "REO8ecKPukL1",
        "outputId": "2c2ce11e-ed26-417b-bb6d-fe0f12f05fe6"
      },
      "source": [
        "# Can do the same with transpose\n",
        "X, tf.transpose(X), tf.reshape(X, shape=(2, 3))"
      ],
      "execution_count": 216,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 3, 5],\n",
              "        [2, 4, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 216
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6JaaGw4AvT2-",
        "outputId": "09f02171-fb77-4fc3-ec83-e32c3bb6bdbe"
      },
      "source": [
        "# Try matrix multiplication with transpose rather than reshape\n",
        "tf.matmul(tf.transpose(X), Y)"
      ],
      "execution_count": 217,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 217
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PE7tiX0OvvpT"
      },
      "source": [
        "** The dot product **\n",
        "\n",
        "Matrix multiplication is also referred to as the dot product.\n",
        "\n",
        "You can perform matrix multiplication using:\n",
        "* `tf.matmul()`\n",
        "* `tf.tensordot()`\n",
        "* `@`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U9_Ijfd_xBRg",
        "outputId": "92b35d59-b742-493a-e175-bacac5d25226"
      },
      "source": [
        "X, Y"
      ],
      "execution_count": 218,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 218
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d9tTOVyUwseC",
        "outputId": "f257944e-aa8f-482d-fd3b-dbe4a0bb5daa"
      },
      "source": [
        "# Perform the dot product on X and Y (requires X or Y to be transposed)\n",
        "tf.tensordot(tf.transpose(X), Y, axes=1)"
      ],
      "execution_count": 219,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 219
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "abaV1IdrxJH3",
        "outputId": "26eb57eb-3510-4099-e094-5cfb96ce055a"
      },
      "source": [
        "# Perform matrix multiplicaation between X and Y (transposed)\n",
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "execution_count": 220,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 220
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2WScweel6Iol",
        "outputId": "a20da1b6-aa9d-40b6-c0a7-12f53655692d"
      },
      "source": [
        "# Perform matrix multiplication between X and  Y (reshaped)\n",
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "execution_count": 221,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 221
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DHkZNxGN6dn8",
        "outputId": "2115f2dd-d05c-4b85-89f6-0ff4a36867af"
      },
      "source": [
        "# Check the values of Y, reshape Y and transposed Y\n",
        "print(\"Normal Y:\")\n",
        "print(Y, \"\\n\")\n",
        "\n",
        "print(\"Y reshaped to (2, 3):\")\n",
        "print(tf.reshape(Y, shape=(2, 3)), \"\\n\")\n",
        "\n",
        "print(\"Y transposed:\")\n",
        "print(tf.transpose(Y))"
      ],
      "execution_count": 222,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Normal Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32) \n",
            "\n",
            "Y reshaped to (2, 3):\n",
            "tf.Tensor(\n",
            "[[ 7  8  9]\n",
            " [10 11 12]], shape=(2, 3), dtype=int32) \n",
            "\n",
            "Y transposed:\n",
            "tf.Tensor(\n",
            "[[ 7  9 11]\n",
            " [ 8 10 12]], shape=(2, 3), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cC1YMKYw-YJI",
        "outputId": "e529195f-5d94-4998-b3cf-3c0549cca878"
      },
      "source": [
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "execution_count": 223,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 223
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oPspDwXp-xfG"
      },
      "source": [
        "Generally, when performing matrix multiplication on two tensors, and one of the axes does not line up, you will transpose (rather than reshape) one of the tensors to satisfy the matrix multiplication rules."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xXtatIoEP9lw"
      },
      "source": [
        "### Changing the datatype of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "0UydIfRZQh0k",
        "outputId": "aae3177b-0297-4407-9ae4-58aa55fb6e22"
      },
      "source": [
        "tf.__version__"
      ],
      "execution_count": 224,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.6.0'"
            ]
          },
          "metadata": {},
          "execution_count": 224
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BnXjAgIMQEO8",
        "outputId": "8f9e18a0-4edb-4ef1-da03-f097f7975660"
      },
      "source": [
        "# Create anew tensor with default datatype (float32)\n",
        "B = tf.constant([1.7, 7.4])\n",
        "B.dtype"
      ],
      "execution_count": 225,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.float32"
            ]
          },
          "metadata": {},
          "execution_count": 225
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "siI7mIl2QVnQ",
        "outputId": "58327038-81dd-4e1d-bfa5-3cec21059c5f"
      },
      "source": [
        "C = tf.constant([7, 10])\n",
        "C.dtype"
      ],
      "execution_count": 226,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.int32"
            ]
          },
          "metadata": {},
          "execution_count": 226
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ndFgPGJMQalg",
        "outputId": "03ba16fe-a110-4139-96eb-15b446eb17e0"
      },
      "source": [
        "# Change from float32 to float16 (reduced precision)\n",
        "D = tf.cast(B, dtype=tf.float16)\n",
        "D"
      ],
      "execution_count": 227,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 7.4], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 227
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_OgGWU-MWLpm",
        "outputId": "05db459d-4d51-4e1f-9d56-0c6d166e051c"
      },
      "source": [
        "# Change from int32 to float32\n",
        "E = tf.cast(C, dtype=tf.float32)\n",
        "E"
      ],
      "execution_count": 228,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 7., 10.], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 228
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CbcX3oTpWoVT",
        "outputId": "5cb8ef6b-82f9-4658-dfc4-d2d41acb0b86"
      },
      "source": [
        "E_float16 = tf.cast(E, dtype=tf.float16)\n",
        "E_float16"
      ],
      "execution_count": 229,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float16, numpy=array([ 7., 10.], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 229
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yci8P8D6XDeW"
      },
      "source": [
        "### Aggregating tensors\n",
        "\n",
        "Aggregating tensors = condensing them from multiple values down to a smaller amount of values.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y6XM_p_aW0qf",
        "outputId": "deaafda3-65ea-4734-e1be-2b0f29abf02e"
      },
      "source": [
        "# Getting the absolute values\n",
        "D = tf.constant([-7, 10])\n",
        "D"
      ],
      "execution_count": 230,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([-7, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 230
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SK67nBahX6gS",
        "outputId": "306cd2f0-a0af-42ec-8df4-9af3a0402fb9"
      },
      "source": [
        "# Get the absolute values\n",
        "tf.abs(D)"
      ],
      "execution_count": 231,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 231
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kV8LNv7oX_Ac"
      },
      "source": [
        "Let's go through the following forms of aggregation:\n",
        "* Get the minimum\n",
        "* Het the maximum\n",
        "* Get the mean of a tensor\n",
        "* Get the sum of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cWtsQCW2YgrA",
        "outputId": "8635bee1-24ba-4912-a2fb-9861399794aa"
      },
      "source": [
        "# Creating a tensor\n",
        "M = tf.constant([[27, 7],\n",
        "                [19, 98]])\n",
        "M"
      ],
      "execution_count": 232,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[27,  7],\n",
              "       [19, 98]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 232
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hbj8cirPZ7dj",
        "outputId": "38053eaf-d780-47a1-83c8-43c8f0dd01ba"
      },
      "source": [
        "tf.reduce_min(M)"
      ],
      "execution_count": 233,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {},
          "execution_count": 233
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GAl-3GqCaDZF",
        "outputId": "2fa995fe-240d-4f50-d5f5-cecdf8649ab2"
      },
      "source": [
        "tf.reduce_max(M)"
      ],
      "execution_count": 234,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=98>"
            ]
          },
          "metadata": {},
          "execution_count": 234
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bHLI9Fj3aGkM",
        "outputId": "39d47a7e-775d-4f95-c433-58aee58235d9"
      },
      "source": [
        "tf.reduce_mean(M)"
      ],
      "execution_count": 235,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=37>"
            ]
          },
          "metadata": {},
          "execution_count": 235
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g56TH8akaN0m",
        "outputId": "0aae7422-48f2-4123-ef06-619a6ae7c47b"
      },
      "source": [
        "tf.reduce_sum(M)"
      ],
      "execution_count": 236,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=151>"
            ]
          },
          "metadata": {},
          "execution_count": 236
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "icwVtbc8aQ-l",
        "outputId": "2d1c056c-fd17-4adb-f93e-bb6d07b19078"
      },
      "source": [
        "# Create a random tensor with values between 0 and 100 of size 50\n",
        "E = tf.constant(np.random.randint(0, 100, size=50))\n",
        "E"
      ],
      "execution_count": 237,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([85, 56, 73, 76, 22, 70, 64,  7, 85, 29, 13, 31, 49, 50, 58, 85, 98,\n",
              "       94, 89, 24, 42, 69, 32,  7, 37, 15,  7,  3, 18, 99, 23, 72,  9, 33,\n",
              "       10, 63, 64, 55, 66, 51, 43, 30,  3, 79, 28, 71, 33, 59, 38, 96])>"
            ]
          },
          "metadata": {},
          "execution_count": 237
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3EltDyCVaogL",
        "outputId": "88674c53-e964-429e-fddd-af16a4f47326"
      },
      "source": [
        "tf.size(E), tf.shape(E), E.ndim"
      ],
      "execution_count": 238,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=50>,\n",
              " <tf.Tensor: shape=(1,), dtype=int32, numpy=array([50], dtype=int32)>,\n",
              " 1)"
            ]
          },
          "metadata": {},
          "execution_count": 238
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HjlLvRE7ayG7",
        "outputId": "088f8021-6761-4d25-f72e-ff3633e24288"
      },
      "source": [
        "# Find the minimum\n",
        "tf.reduce_min(E)"
      ],
      "execution_count": 239,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=3>"
            ]
          },
          "metadata": {},
          "execution_count": 239
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e58KvGIda5zb",
        "outputId": "3d11a3f5-584a-45d8-850c-66a2526101f0"
      },
      "source": [
        "# Find the maximum\n",
        "tf.reduce_max(E)"
      ],
      "execution_count": 240,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=99>"
            ]
          },
          "metadata": {},
          "execution_count": 240
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aKdWlvwRbGLL",
        "outputId": "14e29bcb-09a1-48da-cc6b-a1b1b67edac3"
      },
      "source": [
        "# Find the mean\n",
        "tf.reduce_mean(E)"
      ],
      "execution_count": 241,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=48>"
            ]
          },
          "metadata": {},
          "execution_count": 241
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V2U66jvibKTV",
        "outputId": "6be2fb56-eb50-4252-9d26-081aa506ace1"
      },
      "source": [
        "# Find the sum\n",
        "tf.reduce_sum(E)"
      ],
      "execution_count": 242,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=2413>"
            ]
          },
          "metadata": {},
          "execution_count": 242
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HxwhcYcCbNBq"
      },
      "source": [
        "**Challenge:** With what we've just learned, find the variance and standard deviation of our `E` tensor using TensorFlow methods."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fG8GRlIIbyIr",
        "outputId": "38da728b-ae26-4377-8ce9-3086d7cb5017"
      },
      "source": [
        "mean_E = tf.reduce_mean(E)\n",
        "mean_E"
      ],
      "execution_count": 243,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=48>"
            ]
          },
          "metadata": {},
          "execution_count": 243
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wqC_9wk1cCf0",
        "outputId": "c8b961e3-95f7-47e9-a36d-d7aa67cea05c"
      },
      "source": [
        "var = E - mean_E\n",
        "var"
      ],
      "execution_count": 244,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([ 37,   8,  25,  28, -26,  22,  16, -41,  37, -19, -35, -17,   1,\n",
              "         2,  10,  37,  50,  46,  41, -24,  -6,  21, -16, -41, -11, -33,\n",
              "       -41, -45, -30,  51, -25,  24, -39, -15, -38,  15,  16,   7,  18,\n",
              "         3,  -5, -18, -45,  31, -20,  23, -15,  11, -10,  48])>"
            ]
          },
          "metadata": {},
          "execution_count": 244
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FaAx0iCAcPs0",
        "outputId": "45bba067-be5d-4326-86eb-e7e620314f1f"
      },
      "source": [
        "var = var * var\n",
        "var"
      ],
      "execution_count": 245,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([1369,   64,  625,  784,  676,  484,  256, 1681, 1369,  361, 1225,\n",
              "        289,    1,    4,  100, 1369, 2500, 2116, 1681,  576,   36,  441,\n",
              "        256, 1681,  121, 1089, 1681, 2025,  900, 2601,  625,  576, 1521,\n",
              "        225, 1444,  225,  256,   49,  324,    9,   25,  324, 2025,  961,\n",
              "        400,  529,  225,  121,  100, 2304])>"
            ]
          },
          "metadata": {},
          "execution_count": 245
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PG40JcKMcXqM",
        "outputId": "cb8f718a-b0e8-4f83-8eb8-18cf6d6c91b8"
      },
      "source": [
        "variance = tf.reduce_sum(var).numpy()/np.shape(var)[0]\n",
        "variance"
      ],
      "execution_count": 246,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "812.58"
            ]
          },
          "metadata": {},
          "execution_count": 246
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fbjO3jzTcpwa",
        "outputId": "135c1987-2c40-4ca1-a96b-dc877a1d26fa"
      },
      "source": [
        "stdev = np.sqrt(variance)\n",
        "stdev"
      ],
      "execution_count": 247,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "28.50578888576845"
            ]
          },
          "metadata": {},
          "execution_count": 247
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M_CKuFJEhLgL"
      },
      "source": [
        "E_float = tf.cast(E, dtype=tf.float32)"
      ],
      "execution_count": 248,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IzK1QUnwdtA5",
        "outputId": "0ab0a953-7458-43c7-a491-e139e466944f"
      },
      "source": [
        "tf.math.reduce_std(E_float)"
      ],
      "execution_count": 249,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=28.504604>"
            ]
          },
          "metadata": {},
          "execution_count": 249
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Fo5P-QtAgm76",
        "outputId": "c5b84f55-3e46-455a-e86b-662b18b04922"
      },
      "source": [
        "tf.math.reduce_variance(E_float)"
      ],
      "execution_count": 250,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=812.51245>"
            ]
          },
          "metadata": {},
          "execution_count": 250
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wFgt27_BhWvY"
      },
      "source": [
        "import tensorflow_probability as tfp"
      ],
      "execution_count": 251,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yOLZ-OZuhnCd",
        "outputId": "19bb8814-376e-4d00-8cc2-57bb9e3efa80"
      },
      "source": [
        "tfp.stats.variance(E)"
      ],
      "execution_count": 252,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=812>"
            ]
          },
          "metadata": {},
          "execution_count": 252
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "32qzZJsUjRQ0",
        "outputId": "45299407-a7a6-4267-d70a-69a08b928113"
      },
      "source": [
        "tf.math.reduce_variance(tf.cast(E, dtype=tf.float64))"
      ],
      "execution_count": 253,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float64, numpy=812.5124000000001>"
            ]
          },
          "metadata": {},
          "execution_count": 253
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tnrIdjXvhqAK",
        "outputId": "9919df98-636c-4a0a-b9b5-653b9022b93d"
      },
      "source": [
        "tf.math.reduce_std(tf.cast(E, dtype=tf.float64))"
      ],
      "execution_count": 254,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float64, numpy=28.50460313703736>"
            ]
          },
          "metadata": {},
          "execution_count": 254
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M7uk8iD8iLgU"
      },
      "source": [
        "### Find the positional maximum and minimum"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0ey80q3Zjeds",
        "outputId": "7e09b5c8-f7fe-46fd-c3f5-a79cdc0a7c2c"
      },
      "source": [
        "# Create a new tesnor for finding the positional minimum and maximum\n",
        "tf.random.set_seed(42)\n",
        "F = tf.random.uniform(shape=[50])\n",
        "F"
      ],
      "execution_count": 255,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 255
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y4DlBzx-kWbT",
        "outputId": "2761a5bb-f9c0-493d-cfc0-a3aeff2ae5e1"
      },
      "source": [
        "# Find the positional maximum\n",
        "tf.argmax(F)"
      ],
      "execution_count": 256,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
            ]
          },
          "metadata": {},
          "execution_count": 256
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wf5_NoaNk1jc",
        "outputId": "daa1d913-35bf-4d58-f782-0ec8fb0c3998"
      },
      "source": [
        "# Index on our largest value position\n",
        "F[tf.argmax(F)]"
      ],
      "execution_count": 257,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {},
          "execution_count": 257
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aIv43U5Jk9Xb",
        "outputId": "398fbe99-cc29-4d03-d389-1e4dfb182b65"
      },
      "source": [
        "# Find the max value of F\n",
        "tf.reduce_max(F)"
      ],
      "execution_count": 258,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {},
          "execution_count": 258
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oXZYlrpZlFXs"
      },
      "source": [
        "# Check for equality\n",
        "assert F[tf.argmax(F)] == tf.reduce_max(F)"
      ],
      "execution_count": 259,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "erPFOdRolPd_",
        "outputId": "3d1fda43-6a63-47b7-99ad-0b5dc6b8ef90"
      },
      "source": [
        "# Find the positional minimum\n",
        "tf.argmin(F)"
      ],
      "execution_count": 260,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=16>"
            ]
          },
          "metadata": {},
          "execution_count": 260
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wRzPppPnlfYW",
        "outputId": "44f651c1-975b-45be-98dc-3f2f27921100"
      },
      "source": [
        "# Find the minimum using the positional minimum index\n",
        "F[tf.argmin(F)]"
      ],
      "execution_count": 261,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {},
          "execution_count": 261
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s8FwiiIRlq6Q",
        "outputId": "29df6354-9fda-4865-aecc-bc658b6763f5"
      },
      "source": [
        "# Find the min value of F\n",
        "tf.reduce_min(F)"
      ],
      "execution_count": 262,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {},
          "execution_count": 262
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jqQ6zzTUluVg"
      },
      "source": [
        "# Check for equality\n",
        "assert F[tf.argmin(F)] == tf.reduce_min(F)"
      ],
      "execution_count": 263,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BfVEjzlql1Wv"
      },
      "source": [
        "### Squeezing a tensor (removing all single dimensions)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qJTR5EkJscrX",
        "outputId": "7249e3b2-d28a-47ef-8e9d-8966473f29ed"
      },
      "source": [
        "# Create a tensor to get started\n",
        "tf.random.set_seed(42)\n",
        "G = tf.constant(tf.random.uniform(shape=[50]), shape=(1, 1, 1, 1, 50))\n",
        "G"
      ],
      "execution_count": 264,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
              "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 264
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uS_WUXjgs0q9",
        "outputId": "cf5ec4f5-cd3d-4edf-fb96-4117347d48da"
      },
      "source": [
        "G.shape"
      ],
      "execution_count": 265,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([1, 1, 1, 1, 50])"
            ]
          },
          "metadata": {},
          "execution_count": 265
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nePbAMwAtI2N",
        "outputId": "01c387e4-8d67-4aba-ccc6-1f84807815db"
      },
      "source": [
        "G_squeezed = tf.squeeze(G)\n",
        "G_squeezed, G_squeezed.shape"
      ],
      "execution_count": 266,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              " array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "        0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "        0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "        0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "        0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "        0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "        0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "        0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "        0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "        0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "       dtype=float32)>, TensorShape([50]))"
            ]
          },
          "metadata": {},
          "execution_count": 266
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gqPRo7vetQCC"
      },
      "source": [
        "### One-hot encoding tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9U5vwQAst4aT",
        "outputId": "5aaf37e1-4221-4975-c37b-65c27bfb68e4"
      },
      "source": [
        "# Create a list of indices\n",
        "some_list = [0, 1, 2, 3] # could be red, green, blue, purple\n",
        "\n",
        "# One-hot encode our list of indices\n",
        "tf.one_hot(some_list, depth=4)"
      ],
      "execution_count": 267,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 267
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-ETrPSFPuJD-",
        "outputId": "c35e6f7e-2b2c-469d-b6b4-83068bf8fb96"
      },
      "source": [
        "# Specify custom values for one-hot encoding\n",
        "tf.one_hot(some_list, depth=4, on_value=\"yo\", off_value=\"no\")"
      ],
      "execution_count": 268,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
              "array([[b'yo', b'no', b'no', b'no'],\n",
              "       [b'no', b'yo', b'no', b'no'],\n",
              "       [b'no', b'no', b'yo', b'no'],\n",
              "       [b'no', b'no', b'no', b'yo']], dtype=object)>"
            ]
          },
          "metadata": {},
          "execution_count": 268
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BrRHfEn_uwc8"
      },
      "source": [
        "### Squaring, log, square root"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jmTNs-0S0fe2",
        "outputId": "bb7670d9-7edd-4339-8e59-a80d0ff37407"
      },
      "source": [
        "H = tf.range(1, 10)\n",
        "H"
      ],
      "execution_count": 269,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 269
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VLtJURkm0o0C",
        "outputId": "6b7265f7-91a0-4a7b-8728-23c200db6d60"
      },
      "source": [
        "# Square it\n",
        "tf.square(H)"
      ],
      "execution_count": 270,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 270
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Uq50g-rg0t0l",
        "outputId": "d913c886-ba62-4e6c-e8fb-fbee25ecb01f"
      },
      "source": [
        "# Find the square root\n",
        "tf.sqrt(tf.cast(H, dtype=tf.float32))"
      ],
      "execution_count": 271,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.99999994, 1.4142134 , 1.7320508 , 1.9999999 , 2.236068  ,\n",
              "       2.4494896 , 2.6457512 , 2.8284268 , 3.        ], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 271
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DhcPUuzF03O6",
        "outputId": "ce6ce7ad-8b32-46e2-dce9-3f7515b5fb69"
      },
      "source": [
        "# Find the log \n",
        "tf.math.log(tf.cast(H, dtype=tf.float32))"
      ],
      "execution_count": 272,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
              "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 272
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uGwrmtaJ1TsN"
      },
      "source": [
        "### Tensors and NumPy\n",
        "\n",
        "TensorFlow interacts beautifully with NumPy arrays.\n",
        "\n",
        "**Note:** One of the main difference between a TensorFlow tensor and a NumPy array is that a tensor can be run on a GPU or TPU (for faster numerical processing)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8Y2DqRsA2aLl",
        "outputId": "827a487d-a067-4fad-c9d9-6c29bde289ab"
      },
      "source": [
        "# Create a tensor directly from a NumPy array\n",
        "J = tf.constant(np.array([3., 7., 10.]))\n",
        "J"
      ],
      "execution_count": 273,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
            ]
          },
          "metadata": {},
          "execution_count": 273
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3Cn563q823Uw",
        "outputId": "bc6dba37-8342-4fda-dc35-63d37bd7312f"
      },
      "source": [
        "# Convert our tensor back to a NumPy array\n",
        "np.array(J), type(np.array(J))"
      ],
      "execution_count": 274,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 274
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IEQYlCQ02_8v",
        "outputId": "24c50155-6025-4efa-fa5d-c38a7025f4d1"
      },
      "source": [
        "# Convert tensor J to a NumPy array \n",
        "J.numpy(), type(J.numpy())"
      ],
      "execution_count": 275,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 275
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GlbEt5Qh3LW_",
        "outputId": "ec22d2ab-8a4e-4dc6-b8fb-a9b49272711b"
      },
      "source": [
        "J = tf.constant([3.])\n",
        "J.numpy()[0]"
      ],
      "execution_count": 276,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.0"
            ]
          },
          "metadata": {},
          "execution_count": 276
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2ln2Vw0M3R_n",
        "outputId": "af496a9b-1fce-491c-c527-8c5ca4d5c56a"
      },
      "source": [
        "# Default types of each are slightly different\n",
        "numpy_J = tf.constant(np.array([3., 7., 10.]))\n",
        "tensor_J = tf.constant([3., 7., 10.])\n",
        "# Check the datatypes of each\n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "execution_count": 277,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {},
          "execution_count": 277
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03JWm2RH3t0F"
      },
      "source": [
        "### Finding access to GPUs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PskKKL249xx8",
        "outputId": "9a697824-a1a9-4c07-db62-1e52817dd098"
      },
      "source": [
        "import tensorflow as tf\n",
        "tf.config.list_physical_devices(\"GPU\")"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YC82zmqG-PT5",
        "outputId": "3cdcb0d7-0920-4a94-e92c-c62fb0d98ba9"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sat Sep 18 22:25:11 2021       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 470.63.01    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   44C    P0    58W / 149W |    123MiB / 11441MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FpYwS_0i_LMV"
      },
      "source": [
        "> **Note:** If you have access to a CUDA-enabled GPU, TensorFlow will automatically use it whenever possible."
      ]
    }
  ]
}