CIFAR-100.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7ce1b882",
   "metadata": {},
   "source": [
    "# <center>CIFAR-100</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68933404",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Librairies-Import\" data-toc-modified-id=\"Librairies-Import-1\">Librairies Import</a></span></li><li><span><a href=\"#Dataset-Loading\" data-toc-modified-id=\"Dataset-Loading-2\">Dataset Loading</a></span></li><li><span><a href=\"#Normalization\" data-toc-modified-id=\"Normalization-3\">Normalization</a></span></li><li><span><a href=\"#DataModule\" data-toc-modified-id=\"DataModule-4\">DataModule</a></span></li><li><span><a href=\"#Model\" data-toc-modified-id=\"Model-5\">Model</a></span></li><li><span><a href=\"#Lightning-Pipeline\" data-toc-modified-id=\"Lightning-Pipeline-6\">Lightning Pipeline</a></span></li><li><span><a href=\"#Model-Training-and-Evaluation\" data-toc-modified-id=\"Model-Training-and-Evaluation-7\">Model Training and Evaluation</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fa541dc",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b1b0b0e",
   "metadata": {},
   "source": [
    "The CIFAR-100 dataset (Canadian Institute for Advanced Research, 100 classes) is a subset of the Tiny Images dataset and consists of 60000 32x32 color images. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. There are 600 images per class. Each image comes with a \"fine\" label (the class to which it belongs) and a \"coarse\" label (the superclass to which it belongs). There are 500 training images and 100 testing images per class.\n",
    "\n",
    "In this notebook, we attempt to train a convolutional neural network to predict the \"fine\" label of the CIFAR-100 dataset images.\n",
    "\n",
    "All the implementation is done using [`pytorch-lightning`](https://www.pytorchlightning.ai), a powerful and customizable PyTorch framework. All parameters and results, as well as a few samples of the train images are logged to Tensorboard."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d72bc54e",
   "metadata": {},
   "source": [
    "## Librairies Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "628aa0a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import warnings\n",
    "import csv\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import DataLoader\n",
    "from torch.utils.data.sampler import SubsetRandomSampler\n",
    "from torch.optim.lr_scheduler import OneCycleLR, ReduceLROnPlateau, MultiStepLR\n",
    "import torchvision.datasets as datasets\n",
    "import torchvision.transforms as transforms\n",
    "from torchvision.models import resnet18\n",
    "from torchvision.utils import make_grid\n",
    "from torchmetrics import ConfusionMatrix\n",
    "from torchmetrics.functional import accuracy\n",
    "from pytorch_lightning import Trainer\n",
    "from pytorch_lightning.callbacks import ModelCheckpoint, LearningRateMonitor, EarlyStopping\n",
    "from pytorch_lightning.loggers import TensorBoardLogger\n",
    "from pytorch_lightning import LightningModule, LightningDataModule"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d09108bb",
   "metadata": {},
   "source": [
    "## Dataset Loading"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d40458a0",
   "metadata": {},
   "source": [
    "We start by downloading the train and test sets of CIFAR-100."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "41ae5cb8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "train_set = datasets.CIFAR100(root=\"./data\", download=True, train=True)\n",
    "test_set = datasets.CIFAR100(root=\"./data\", download=True, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11775047",
   "metadata": {},
   "source": [
    "We then calculate the number of images by class for each set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "421264da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'cattle': 500,\n",
       " 'dinosaur': 500,\n",
       " 'apple': 500,\n",
       " 'boy': 500,\n",
       " 'aquarium_fish': 500,\n",
       " 'telephone': 500,\n",
       " 'train': 500,\n",
       " 'cup': 500,\n",
       " 'cloud': 500,\n",
       " 'elephant': 500,\n",
       " 'keyboard': 500,\n",
       " 'willow_tree': 500,\n",
       " 'sunflower': 500,\n",
       " 'castle': 500,\n",
       " 'sea': 500,\n",
       " 'bicycle': 500,\n",
       " 'wolf': 500,\n",
       " 'squirrel': 500,\n",
       " 'shrew': 500,\n",
       " 'pine_tree': 500,\n",
       " 'rose': 500,\n",
       " 'television': 500,\n",
       " 'table': 500,\n",
       " 'possum': 500,\n",
       " 'oak_tree': 500,\n",
       " 'leopard': 500,\n",
       " 'maple_tree': 500,\n",
       " 'rabbit': 500,\n",
       " 'chimpanzee': 500,\n",
       " 'clock': 500,\n",
       " 'streetcar': 500,\n",
       " 'cockroach': 500,\n",
       " 'snake': 500,\n",
       " 'lobster': 500,\n",
       " 'mountain': 500,\n",
       " 'palm_tree': 500,\n",
       " 'skyscraper': 500,\n",
       " 'tractor': 500,\n",
       " 'shark': 500,\n",
       " 'butterfly': 500,\n",
       " 'bottle': 500,\n",
       " 'bee': 500,\n",
       " 'chair': 500,\n",
       " 'woman': 500,\n",
       " 'hamster': 500,\n",
       " 'otter': 500,\n",
       " 'seal': 500,\n",
       " 'lion': 500,\n",
       " 'mushroom': 500,\n",
       " 'girl': 500,\n",
       " 'sweet_pepper': 500,\n",
       " 'forest': 500,\n",
       " 'crocodile': 500,\n",
       " 'orange': 500,\n",
       " 'tulip': 500,\n",
       " 'mouse': 500,\n",
       " 'camel': 500,\n",
       " 'caterpillar': 500,\n",
       " 'man': 500,\n",
       " 'skunk': 500,\n",
       " 'kangaroo': 500,\n",
       " 'raccoon': 500,\n",
       " 'snail': 500,\n",
       " 'rocket': 500,\n",
       " 'whale': 500,\n",
       " 'worm': 500,\n",
       " 'turtle': 500,\n",
       " 'beaver': 500,\n",
       " 'plate': 500,\n",
       " 'wardrobe': 500,\n",
       " 'road': 500,\n",
       " 'fox': 500,\n",
       " 'flatfish': 500,\n",
       " 'tiger': 500,\n",
       " 'ray': 500,\n",
       " 'dolphin': 500,\n",
       " 'poppy': 500,\n",
       " 'porcupine': 500,\n",
       " 'lamp': 500,\n",
       " 'crab': 500,\n",
       " 'motorcycle': 500,\n",
       " 'spider': 500,\n",
       " 'tank': 500,\n",
       " 'orchid': 500,\n",
       " 'lizard': 500,\n",
       " 'beetle': 500,\n",
       " 'bridge': 500,\n",
       " 'baby': 500,\n",
       " 'lawn_mower': 500,\n",
       " 'house': 500,\n",
       " 'bus': 500,\n",
       " 'couch': 500,\n",
       " 'bowl': 500,\n",
       " 'pear': 500,\n",
       " 'bed': 500,\n",
       " 'plain': 500,\n",
       " 'trout': 500,\n",
       " 'bear': 500,\n",
       " 'pickup_truck': 500,\n",
       " 'can': 500}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_classes_counts = {}\n",
    "\n",
    "for image in train_set:\n",
    "    label = train_set.classes[image[1]]\n",
    "    if label not in train_classes_counts:\n",
    "        train_classes_counts[label] = 1\n",
    "    else:\n",
    "        train_classes_counts[label] += 1\n",
    "\n",
    "train_classes_counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3de67b4c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mountain': 100,\n",
       " 'forest': 100,\n",
       " 'seal': 100,\n",
       " 'mushroom': 100,\n",
       " 'sea': 100,\n",
       " 'tulip': 100,\n",
       " 'camel': 100,\n",
       " 'butterfly': 100,\n",
       " 'cloud': 100,\n",
       " 'apple': 100,\n",
       " 'skunk': 100,\n",
       " 'streetcar': 100,\n",
       " 'rocket': 100,\n",
       " 'lamp': 100,\n",
       " 'lion': 100,\n",
       " 'wolf': 100,\n",
       " 'rose': 100,\n",
       " 'orange': 100,\n",
       " 'dinosaur': 100,\n",
       " 'chimpanzee': 100,\n",
       " 'can': 100,\n",
       " 'keyboard': 100,\n",
       " 'bicycle': 100,\n",
       " 'chair': 100,\n",
       " 'plate': 100,\n",
       " 'lawn_mower': 100,\n",
       " 'turtle': 100,\n",
       " 'palm_tree': 100,\n",
       " 'shark': 100,\n",
       " 'pickup_truck': 100,\n",
       " 'boy': 100,\n",
       " 'couch': 100,\n",
       " 'house': 100,\n",
       " 'porcupine': 100,\n",
       " 'cockroach': 100,\n",
       " 'clock': 100,\n",
       " 'castle': 100,\n",
       " 'beaver': 100,\n",
       " 'bee': 100,\n",
       " 'bottle': 100,\n",
       " 'pear': 100,\n",
       " 'baby': 100,\n",
       " 'flatfish': 100,\n",
       " 'oak_tree': 100,\n",
       " 'leopard': 100,\n",
       " 'snail': 100,\n",
       " 'crocodile': 100,\n",
       " 'rabbit': 100,\n",
       " 'beetle': 100,\n",
       " 'girl': 100,\n",
       " 'sunflower': 100,\n",
       " 'raccoon': 100,\n",
       " 'train': 100,\n",
       " 'ray': 100,\n",
       " 'trout': 100,\n",
       " 'bowl': 100,\n",
       " 'snake': 100,\n",
       " 'orchid': 100,\n",
       " 'tractor': 100,\n",
       " 'caterpillar': 100,\n",
       " 'bus': 100,\n",
       " 'mouse': 100,\n",
       " 'crab': 100,\n",
       " 'sweet_pepper': 100,\n",
       " 'maple_tree': 100,\n",
       " 'whale': 100,\n",
       " 'skyscraper': 100,\n",
       " 'pine_tree': 100,\n",
       " 'tank': 100,\n",
       " 'cattle': 100,\n",
       " 'man': 100,\n",
       " 'aquarium_fish': 100,\n",
       " 'shrew': 100,\n",
       " 'plain': 100,\n",
       " 'possum': 100,\n",
       " 'lobster': 100,\n",
       " 'hamster': 100,\n",
       " 'television': 100,\n",
       " 'dolphin': 100,\n",
       " 'worm': 100,\n",
       " 'squirrel': 100,\n",
       " 'cup': 100,\n",
       " 'woman': 100,\n",
       " 'bridge': 100,\n",
       " 'wardrobe': 100,\n",
       " 'road': 100,\n",
       " 'lizard': 100,\n",
       " 'elephant': 100,\n",
       " 'spider': 100,\n",
       " 'fox': 100,\n",
       " 'otter': 100,\n",
       " 'poppy': 100,\n",
       " 'willow_tree': 100,\n",
       " 'table': 100,\n",
       " 'kangaroo': 100,\n",
       " 'telephone': 100,\n",
       " 'bed': 100,\n",
       " 'motorcycle': 100,\n",
       " 'bear': 100,\n",
       " 'tiger': 100}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_classes_counts = {}\n",
    "\n",
    "for image in test_set:\n",
    "    label = test_set.classes[image[1]]\n",
    "    if label not in test_classes_counts:\n",
    "        test_classes_counts[label] = 1\n",
    "    else:\n",
    "        test_classes_counts[label] += 1\n",
    "        \n",
    "test_classes_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f56c6ea5",
   "metadata": {},
   "source": [
    "The train and test sets both feature the 100 classes to predict and are well-balanced, each class having 500 images in the train set and 100 images in the test set.\n",
    "\n",
    "We decide to further split the train set in a train and a validation set. We will extract 10% of the train set images for that. This is done in the `CIFAR100DataModule`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "853bd565",
   "metadata": {},
   "source": [
    "## Normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1648a0e",
   "metadata": {},
   "source": [
    "We create a function to compute the normalization statistics for our dataset. These statistics are computed on the train set and applied to the train, validation and test sets.\n",
    "The function is called in the `setup` method of the `CIFAR100DataModule`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c6c974fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_normstats(train_set):\n",
    "    \"\"\"\n",
    "    Function to compute the normalization statistics on the train set.\n",
    "    \n",
    "    Takes in: train_set\n",
    "\n",
    "    Returns: (red channel mean, green channel mean, blue channel mean), (red channel std, green channel std, blue channel std)\n",
    "    \"\"\"\n",
    "    red_channels = torch.stack([train_set[i][0][0, :, :] for i in range(len(train_set))], dim=0)\n",
    "    green_channels = torch.stack([train_set[i][0][1, :, :] for i in range(len(train_set))], dim=0)\n",
    "    blue_channels = torch.stack([train_set[i][0][2, :, :] for i in range(len(train_set))], dim=0)\n",
    "    train_set_mean = (red_channels.mean().item(), green_channels.mean().item(), blue_channels.mean().item())\n",
    "    train_set_std = (red_channels.std().item(), green_channels.std().item(), blue_channels.std().item())\n",
    "    return train_set_mean, train_set_std"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7706fe18",
   "metadata": {},
   "source": [
    "## LightningDataModule"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e15278",
   "metadata": {},
   "source": [
    "Next, we create the DataModule. This class implements several methods to:\n",
    "- download the dataset (if not already done)\n",
    "- create the train, validation and test sets and apply all the necessary transforms (including data augmentation)\n",
    "- create the dataloaders\n",
    "\n",
    "In the `setup` method, we split the train set in a train and a validation set. This is done by picking 10% of the train images indices at random, after they are being shuffled. A seed is set prior to it and we ensured that it gave us a well-enough balanced validation set featuring all the 100 classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "037a6308",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CIFAR100DataModule(LightningDataModule):\n",
    "    \n",
    "    def __init__(self, batch_size=64, num_workers=2):\n",
    "        super().__init__()\n",
    "        self.batch_size = batch_size\n",
    "        self.num_workers = num_workers\n",
    "    \n",
    "    \n",
    "    def prepare_data(self):\n",
    "        # Download the train and test sets\n",
    "        datasets.CIFAR100(root=\"./data\", download=True, train=True)\n",
    "        datasets.CIFAR100(root=\"./data\", download=True, train=False)\n",
    "\n",
    "        \n",
    "    def setup(self, stage=None):\n",
    "        # Load the train set\n",
    "        train_set = datasets.CIFAR100(root=\"./data\", train=True, transform=transforms.ToTensor())\n",
    "        # Compute the normalization statistics on the train set\n",
    "        train_set_mean, train_set_std = compute_normstats(train_set)\n",
    "        \n",
    "        # Create the transforms for all sets\n",
    "        train_set_transforms = transforms.Compose([transforms.RandomCrop(32, padding=4),\n",
    "                                            transforms.RandomHorizontalFlip(p=0.5),\n",
    "                                            transforms.RandomRotation(degrees=15),\n",
    "                                            transforms.ToTensor(),\n",
    "                                            transforms.Normalize(train_set_mean, train_set_std, inplace=True)])\n",
    "        validation_set_transforms = transforms.Compose([transforms.ToTensor(),\n",
    "                                            transforms.Normalize(train_set_mean, train_set_std, inplace=True)])                                            \n",
    "        test_set_transforms = transforms.Compose([transforms.ToTensor(),\n",
    "                                            transforms.Normalize(train_set_mean, train_set_std, inplace=True)])\n",
    "\n",
    "        \n",
    "        # Get the train set images indices and shuffle them\n",
    "        train_set_length = len(train_set)\n",
    "        indices = list(range(train_set_length))\n",
    "        np.random.seed(42)\n",
    "        np.random.shuffle(indices)\n",
    "\n",
    "        # Calculate the split point to have 10% of the train set as a validation set\n",
    "        split = int(np.floor(0.9 * train_set_length))\n",
    "\n",
    "        # Create a sampler for the train set (used in train_dataloader)\n",
    "        self.train_sampler = SubsetRandomSampler(indices[:split])\n",
    "\n",
    "        # Get the indices for the validation set (used in val_dataloader)\n",
    "        self.validation_indices = indices[split:]\n",
    "\n",
    "        # Create the train, validation and test sets (here, the train set is reloaded a second time but with the appropriate transforms)\n",
    "        self.cifar100_train = datasets.CIFAR100(root=\"./data\", train=True, transform=train_set_transforms)\n",
    "        self.cifar100_validation = datasets.CIFAR100(root=\"./data\", train=True, transform=validation_set_transforms)\n",
    "        self.cifar100_test = datasets.CIFAR100(root=\"./data\", train=False, transform=test_set_transforms)\n",
    "\n",
    "        # Retrieve classes from the train set\n",
    "        self.classes = self.cifar100_train.classes\n",
    "        \n",
    "        \n",
    "    def train_dataloader(self):\n",
    "        cifar100_train = DataLoader(self.cifar100_train, batch_size=self.batch_size, sampler=self.train_sampler, num_workers=self.num_workers)\n",
    "        return cifar100_train\n",
    "\n",
    "    \n",
    "    def val_dataloader(self):\n",
    "        cifar100_validation = DataLoader(self.cifar100_validation, batch_size=self.batch_size, sampler=self.validation_indices, num_workers=self.num_workers)\n",
    "        return cifar100_validation\n",
    "\n",
    "    \n",
    "    def test_dataloader(self):\n",
    "        cifar100_test = DataLoader(self.cifar100_test, batch_size=self.batch_size, shuffle=False, num_workers=self.num_workers)\n",
    "        return cifar100_test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13b388a",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b15fa052",
   "metadata": {},
   "source": [
    "Then, we define a function to create the model. Here, we take the `resnet18` architecture but not pre-trained.\n",
    "We adapt it to our classification problem by modifying the first convolutional layer and the maxpool layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "858c2030",
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_model():\n",
    "    \"\"\"\n",
    "    Function to create the model.\n",
    "    \n",
    "    Takes in: -\n",
    "\n",
    "    Returns: model\n",
    "    \"\"\"\n",
    "    model = resnet18(pretrained=False, num_classes=100)\n",
    "    model.conv1 = nn.Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
    "    model.maxpool = nn.Identity()\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f12d2cad",
   "metadata": {},
   "source": [
    "## LightningModule"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b0833c2",
   "metadata": {},
   "source": [
    "The Lightning module is where everything happens. It implements the following methods:\n",
    "- `configure_optimizers`\n",
    "- `forward`\n",
    "- `training_step`\n",
    "- `training_epoch_end`\n",
    "- `validation_step`\n",
    "- `validation_epoch_end`\n",
    "- `test_step`\n",
    "- `test_epoch_end`\n",
    "- `on_save_checkpoint`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "633e96c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CIFAR100ResNet(LightningModule):\n",
    "    \n",
    "    def __init__(self, learning_rate, batch_size):\n",
    "        super().__init__()      \n",
    "        # Save hyperparameters to the checkpoint\n",
    "        self.save_hyperparameters()     \n",
    "        # Creation of the model\n",
    "        self.model = create_model()\n",
    "        # Instantiation of the confusion matrix\n",
    "        self.confmat = ConfusionMatrix(num_classes=100)\n",
    "        # Instantiation of the number of classes\n",
    "        self.n_classes = 100 \n",
    "        # Instantiation of the learning rate\n",
    "        self.learning_rate = learning_rate\n",
    "        # Instantiation of the batch_size\n",
    "        self.batch_size = batch_size\n",
    "        \n",
    "        \n",
    "    def configure_optimizers(self): \n",
    "        optimizer = torch.optim.SGD(\n",
    "            self.parameters(),\n",
    "            lr=self.hparams.learning_rate,\n",
    "            momentum=0.9,\n",
    "            weight_decay=5e-4,\n",
    "        )\n",
    "        #scheduler_dict = {\n",
    "            #\"scheduler\": MultiStepLR(\n",
    "                #optimizer,\n",
    "                #milestones=[60,120,160],\n",
    "                #gamma=0.2\n",
    "            #),\n",
    "            #\"interval\": \"epoch\"\n",
    "        #}        \n",
    "        scheduler_dict = {\n",
    "            \"scheduler\": ReduceLROnPlateau(\n",
    "                optimizer,\n",
    "                mode=\"min\",\n",
    "                factor=0.2,\n",
    "                patience=10\n",
    "                ),\n",
    "            \"interval\": \"epoch\",\n",
    "            \"frequency\": 1,\n",
    "            \"monitor\": \"validation_loss\"\n",
    "        }        \n",
    "        #steps_per_epoch = int(np.ceil(45000 / self.batch_size))\n",
    "        #scheduler_dict = {\n",
    "            #\"scheduler\": OneCycleLR(\n",
    "            #    optimizer,\n",
    "            #    max_lr=0.1,\n",
    "            #    epochs=self.trainer.max_epochs,\n",
    "            #    steps_per_epoch=steps_per_epoch\n",
    "            #),\n",
    "            #\"interval\": \"step\"\n",
    "        #}\n",
    "        return {\"optimizer\": optimizer, \"lr_scheduler\": scheduler_dict}\n",
    "    \n",
    "    \n",
    "    def forward(self, x):\n",
    "        logits = self.model(x)\n",
    "        return logits\n",
    "\n",
    "    \n",
    "    def training_step(self, batch, batch_idx):\n",
    "        inputs, targets = batch\n",
    "        logits = self(inputs)\n",
    "        loss = F.cross_entropy(logits, targets)\n",
    "        predictions = torch.argmax(logits, dim=1)\n",
    "        self.log(\"train_loss\", loss, on_epoch=True, prog_bar=True)\n",
    "        return {\"inputs\":inputs, \"targets\":targets, \"predictions\":predictions, \"loss\":loss}    \n",
    "\n",
    "    \n",
    "    def training_epoch_end(self, outputs):\n",
    "        # Log weights and biases for all layers of the model\n",
    "        for name, params in self.named_parameters():\n",
    "            self.logger.experiment.add_histogram(name, params,self.current_epoch)\n",
    "        # Only after the first training epoch, log some of the training images and the model graph\n",
    "        if self.current_epoch == 0:\n",
    "            image_samples = outputs[0][\"inputs\"][:10]\n",
    "            image_samples = image_samples.cpu()\n",
    "            image_samples_grid = make_grid(image_samples, normalize=True)\n",
    "            image_samples_grid = image_samples_grid.numpy()\n",
    "            fig = plt.figure()\n",
    "            ax = fig.add_subplot(111)\n",
    "            ax.imshow(np.transpose(image_samples_grid, (1, 2, 0)))\n",
    "            self.logger.experiment.add_figure(f\"Training sample normalized images\", fig)\n",
    "            input_sample = outputs[0][\"inputs\"][0]\n",
    "            input_sample = torch.unsqueeze(input_sample, 3)\n",
    "            input_sample = torch.permute(input_sample, (3,0,1,2))\n",
    "            self.logger.experiment.add_graph(self, input_sample)\n",
    "\n",
    "            \n",
    "    def validation_step(self, batch, batch_idx):\n",
    "        inputs, targets = batch\n",
    "        logits = self(inputs)\n",
    "        loss = F.cross_entropy(logits, targets)\n",
    "        predictions = torch.argmax(logits, dim=1)\n",
    "        acc = accuracy(predictions, targets)\n",
    "        self.log(f\"validation_loss\", loss, on_epoch=True, prog_bar=True)\n",
    "        self.log(f\"validation_acc\", acc, on_epoch=True, prog_bar=True)\n",
    "        return {\"inputs\":inputs, \"targets\":targets, \"predictions\":predictions, \"loss\":loss} \n",
    "\n",
    "    \n",
    "    def validation_epoch_end(self, outputs):\n",
    "        # Concatenate the targets of all batches\n",
    "        targets = torch.cat([output[\"targets\"] for output in outputs])\n",
    "        # Concatenate the predictions of all batches\n",
    "        predictions = torch.cat([output[\"predictions\"] for output in outputs])\n",
    "        # Compute the confusion matrix\n",
    "        cm = self.confmat(predictions, targets)\n",
    "        # Send it to the CPU\n",
    "        cm = cm.cpu()\n",
    "\n",
    "                \n",
    "    def test_step(self, batch, batch_idx):\n",
    "        inputs, targets = batch\n",
    "        logits = self(inputs)\n",
    "        loss = F.cross_entropy(logits, targets)\n",
    "        probabilities = F.softmax(logits, dim=1)\n",
    "        predictions = torch.argmax(logits, dim=1)\n",
    "        acc = accuracy(predictions, targets)\n",
    "        self.log(f\"test_loss\", loss, prog_bar=True)\n",
    "        self.log(f\"test_acc\", acc, prog_bar=True)\n",
    "        return {\"targets\":targets, \"predictions\":predictions, \"probabilities\":probabilities}\n",
    "\n",
    "    \n",
    "    def test_epoch_end(self, outputs):\n",
    "        targets = torch.cat([output[\"targets\"] for output in outputs])\n",
    "        predictions = torch.cat([output[\"predictions\"] for output in outputs])\n",
    "        probabilities = torch.cat([output[\"probabilities\"] for output in outputs])\n",
    "        # Compute the total prediction accuracy on the full test set\n",
    "        acc = accuracy(predictions, targets)\n",
    "        # Compute the confusion matrix\n",
    "        cm = self.confmat(predictions, targets)\n",
    "        # Send it to the CPU\n",
    "        cm = cm.cpu()\n",
    "        # Calculate the accuracy for each class\n",
    "        classes_precisions = []\n",
    "        for class_id in range(self.n_classes):\n",
    "            precision = cm[class_id, class_id] / torch.sum(cm[:,class_id])            \n",
    "            precision = round(precision.item()*100, 1)\n",
    "            classes_precisions.append(precision)\n",
    "        # Write the test set prediction performances to a csv file\n",
    "        with open(\"test_set_predictions.csv\", \"w\", newline=\"\") as f:\n",
    "            writer = csv.writer(f)\n",
    "            writer.writerow(self.trainer.datamodule.classes)\n",
    "            for _, image_probs in enumerate(probabilities.cpu().numpy()):\n",
    "                writer.writerow(np.around(image_probs, decimals=2))      \n",
    "        # Write the test set prediction performances to an output file\n",
    "        with open(\"test_set_predictions.txt\", \"w\") as f:\n",
    "            f.write(\"==================================================\\n\")\n",
    "            f.write(\"ACCURACY\\n\")\n",
    "            f.write(\"==================================================\\n\")\n",
    "            f.write(\"\\n\")            \n",
    "            f.write(f\"Total: {round(acc.item()*100, 1)}%\\n\")\n",
    "            f.write(\"\\n\")\n",
    "            f.write(\"Per Class:\\n\")\n",
    "            f.write(\"Class - Accuracy (%)\\n\")\n",
    "            for class_id in range(self.n_classes):\n",
    "                f.write(f\"{self.trainer.datamodule.classes[class_id]} - {classes_precisions[class_id]}\\n\")\n",
    "            f.write(\"\\n\")\n",
    "            f.write(\"\\n\")\n",
    "            f.write(\"==================================================\\n\")\n",
    "            f.write(\"PREDICTIONS DETAIL\\n\")\n",
    "            f.write(\"==================================================\\n\")\n",
    "            f.write(\"Image index - Target class - Predicted class\\n\")\n",
    "            for i in range(len(targets)):\n",
    "                f.write(f\"{i} - {self.trainer.datamodule.classes[targets[i]]} - {self.trainer.datamodule.classes[predictions[i]]}\\n\")\n",
    "    \n",
    "    \n",
    "    def on_save_checkpoint(self, checkpoint):\n",
    "        # Get the state_dict from self.model to get rid of the \"model.\" prefix\n",
    "        checkpoint[\"state_dict\"] = self.state_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b61c8c",
   "metadata": {},
   "source": [
    "## Model Training and Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1624e07e",
   "metadata": {},
   "source": [
    "Finally, we set some of our hyper-parameters, instantiate the classes defined above as well as some callbacks (tensorboard logger, learning rate monitor, early stopping and checkpoint saving), and train and test our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "022f9d7e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "GPU available: True, used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of workers used: 4\n",
      "Maximum number of epochs: 200\n",
      "Batch size: 256\n",
      "Initial learning rate: 0.1\n",
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name    | Type            | Params\n",
      "--------------------------------------------\n",
      "0 | model   | ResNet          | 11.2 M\n",
      "1 | confmat | ConfusionMatrix | 0     \n",
      "--------------------------------------------\n",
      "11.2 M    Trainable params\n",
      "0         Non-trainable params\n",
      "11.2 M    Total params\n",
      "44.881    Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation sanity check: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d6a180e27f0249f1a6805fd92504a124",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved. New best score: 3.644\n",
      "Epoch 0, global step 175: validation_loss reached 3.64440 (best 3.64440), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=0-validation_loss=3.64.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.414 >= min_delta = 0.0. New best score: 3.231\n",
      "Epoch 1, global step 351: validation_loss reached 3.23067 (best 3.23067), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=1-validation_loss=3.23-v2.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.151 >= min_delta = 0.0. New best score: 3.080\n",
      "Epoch 2, global step 527: validation_loss reached 3.07953 (best 3.07953), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=2-validation_loss=3.08.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.347 >= min_delta = 0.0. New best score: 2.733\n",
      "Epoch 3, global step 703: validation_loss reached 2.73287 (best 2.73287), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=3-validation_loss=2.73.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.344 >= min_delta = 0.0. New best score: 2.389\n",
      "Epoch 4, global step 879: validation_loss reached 2.38936 (best 2.38936), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=4-validation_loss=2.39.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 5, global step 1055: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.334 >= min_delta = 0.0. New best score: 2.055\n",
      "Epoch 6, global step 1231: validation_loss reached 2.05528 (best 2.05528), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=6-validation_loss=2.06.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 7, global step 1407: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.116 >= min_delta = 0.0. New best score: 1.940\n",
      "Epoch 8, global step 1583: validation_loss reached 1.93967 (best 1.93967), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=8-validation_loss=1.94.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.035 >= min_delta = 0.0. New best score: 1.905\n",
      "Epoch 9, global step 1759: validation_loss reached 1.90460 (best 1.90460), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=9-validation_loss=1.90.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 10, global step 1935: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.016 >= min_delta = 0.0. New best score: 1.889\n",
      "Epoch 11, global step 2111: validation_loss reached 1.88853 (best 1.88853), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=11-validation_loss=1.89.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.020 >= min_delta = 0.0. New best score: 1.869\n",
      "Epoch 12, global step 2287: validation_loss reached 1.86890 (best 1.86890), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=12-validation_loss=1.87.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.146 >= min_delta = 0.0. New best score: 1.723\n",
      "Epoch 13, global step 2463: validation_loss reached 1.72307 (best 1.72307), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=13-validation_loss=1.72.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 14, global step 2639: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 15, global step 2815: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 16, global step 2991: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 17, global step 3167: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 18, global step 3343: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.037 >= min_delta = 0.0. New best score: 1.686\n",
      "Epoch 19, global step 3519: validation_loss reached 1.68650 (best 1.68650), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=19-validation_loss=1.69.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 20, global step 3695: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.145 >= min_delta = 0.0. New best score: 1.541\n",
      "Epoch 21, global step 3871: validation_loss reached 1.54137 (best 1.54137), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=21-validation_loss=1.54.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 22, global step 4047: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 23, global step 4223: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 24, global step 4399: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 25, global step 4575: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 26, global step 4751: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 27, global step 4927: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 28, global step 5103: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 29, global step 5279: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 30, global step 5455: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.010 >= min_delta = 0.0. New best score: 1.532\n",
      "Epoch 31, global step 5631: validation_loss reached 1.53167 (best 1.53167), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=31-validation_loss=1.53.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 32, global step 5807: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 33, global step 5983: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 34, global step 6159: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 35, global step 6335: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 36, global step 6511: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 37, global step 6687: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.046 >= min_delta = 0.0. New best score: 1.486\n",
      "Epoch 38, global step 6863: validation_loss reached 1.48580 (best 1.48580), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=38-validation_loss=1.49.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 39, global step 7039: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 40, global step 7215: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 41, global step 7391: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 42, global step 7567: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 43, global step 7743: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 44, global step 7919: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 45, global step 8095: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 46, global step 8271: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 47, global step 8447: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 48, global step 8623: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 49, global step 8799: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Metric validation_loss improved by 0.412 >= min_delta = 0.0. New best score: 1.074\n",
      "Epoch 50, global step 8975: validation_loss reached 1.07382 (best 1.07382), saving model to \"C:\\Users\\APU\\Projects\\CIFAR-100\\checkpoints\\epoch=50-validation_loss=1.07.ckpt\" as top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 51, global step 9151: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 52, global step 9327: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 53, global step 9503: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 54, global step 9679: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 55, global step 9855: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 56, global step 10031: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 57, global step 10207: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 58, global step 10383: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 59, global step 10559: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 60, global step 10735: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 61, global step 10911: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 62, global step 11087: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 63, global step 11263: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 64, global step 11439: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 65, global step 11615: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 66, global step 11791: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 67, global step 11967: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 68, global step 12143: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 69, global step 12319: validation_loss was not in top 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Monitored metric validation_loss did not improve in the last 20 records. Best score: 1.074. Signaling Trainer to stop.\n",
      "Epoch 70, global step 12495: validation_loss was not in top 1\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "613d2338da3b4bc9876cc2a354bcd67f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------------------------------------------\n",
      "DATALOADER:0 TEST RESULTS\n",
      "{'test_acc': 0.7348999977111816, 'test_loss': 1.0653947591781616}\n",
      "--------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'test_loss': 1.0653947591781616, 'test_acc': 0.7348999977111816}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Filter harmless waarnings\n",
    "warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n",
    "warnings.filterwarnings(\"ignore\", \".*Your `val_dataloader` has `shuffle=True`.*\")\n",
    "warnings.filterwarnings(\"ignore\", \".*Checkpoint directory.*\")\n",
    "\n",
    "# Print name of graphics card used\n",
    "gpus = min(1, torch.cuda.device_count())\n",
    "\n",
    "# Set number of workers (for dataloaders)\n",
    "num_workers = int(os.cpu_count() / 3)\n",
    "print(f\"Number of workers used: {num_workers}\")\n",
    "\n",
    "# Set maximum number of epochs to train for\n",
    "max_epochs = 200\n",
    "print(f\"Maximum number of epochs: {max_epochs}\")\n",
    "\n",
    "# Set the batch size\n",
    "batch_size = 256\n",
    "print(f\"Batch size: {batch_size}\")\n",
    "\n",
    "# Set the initial learning rate\n",
    "learning_rate = 0.1\n",
    "print(f\"Initial learning rate: {learning_rate}\")    \n",
    "\n",
    "# Instantiate the DataModule\n",
    "dm = CIFAR100DataModule(batch_size=batch_size, num_workers=num_workers)\n",
    "\n",
    "# Instantiate the logger\n",
    "tensorboard_logger = TensorBoardLogger(save_dir=\"logs\")\n",
    "\n",
    "# Instantiate early stopping based on epoch validation loss\n",
    "early_stopping = EarlyStopping(\"validation_loss\", patience=20, verbose=True)\n",
    "\n",
    "# Instantiate a learning rate monitor\n",
    "lr_monitor = LearningRateMonitor(logging_interval='step')\n",
    "\n",
    "# Instantiate a checkpoint callback\n",
    "checkpoint = ModelCheckpoint(\n",
    "                            dirpath=f\"./checkpoints/\",\n",
    "                            filename=\"{epoch}-{validation_loss:.2f}\",\n",
    "                            verbose=True,\n",
    "                            monitor=\"validation_loss\",\n",
    "                            save_last = False,\n",
    "                            save_top_k=1,      \n",
    "                            mode=\"min\",\n",
    "                            save_weights_only=True\n",
    "                            )\n",
    "\n",
    "# Instantiate the trainer\n",
    "trainer = Trainer(\n",
    "                gpus=gpus,\n",
    "                max_epochs=max_epochs, \n",
    "                logger=tensorboard_logger,\n",
    "                log_every_n_steps = 1,\n",
    "                callbacks=[lr_monitor, early_stopping, checkpoint]\n",
    "                ) \n",
    "\n",
    "# Instantiate the pipeline\n",
    "pipeline = CIFAR100ResNet(learning_rate=learning_rate, batch_size=batch_size)  \n",
    "    \n",
    "# Fit the trainer on the training set\n",
    "trainer.fit(pipeline, dm)\n",
    "\n",
    "# Test on the test set\n",
    "trainer.test(pipeline, dm)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "258e88d0",
   "metadata": {},
   "source": [
    "The model's performance metrics and the evolution of its hyperparameters, as well as images samples from the train set, can be visualised on the tensorboard."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {
    "height": "251px",
    "width": "244px"
   },
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "261px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}