ici_query_iatlas_data_python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CRI iAtlas notebooks\n",
    "## Exploring the Immune Checkpoint Inhibition data available in iAtlas in Python.\n",
    "\n",
    "Repo: https://github.com/CRI-iAtlas/iatlas-notebooks/ \n",
    "\n",
    "Notebook: ici_query_iatlas_data_python.ipynb \n",
    "\n",
    "Date: July 1, 2024 \n",
    "\n",
    "Author: Carolina Heimann"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "notebook repo: https://github.com/CRI-iAtlas/iatlas-notebooks\n",
    "\n",
    "landing page: https://www.cri-iatlas.org/\n",
    "\n",
    "portal: https://isb-cgc.shinyapps.io/iatlas/\n",
    "\n",
    "email: support@cri-iatlas.org\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CRI iAtlas database is available in a database that can be queried using the R package [`iatlasGraphQLClient`](https://github.com/CRI-iAtlas/iatlasGraphQLClient). \n",
    "\n",
    "For manipulating the data in Python, one can perform the queries in R (for more information, check our notebook on [Exploring the Immune Checkpoint Inhibition data available in iAtlas in R](https://github.com/CRI-iAtlas/iatlas-notebooks/blob/main/ici_query_iatlas_data.ipynb) and [Querying TCGA features and expression in R](https://github.com/CRI-iAtlas/iatlas-notebooks/blob/main/querying_TCGA_features_and_expression.ipynb)) and export the data for further analysis in Python, or use the libraries for interoperability between Python and R. \n",
    "\n",
    "In this notebook, we will use the `rpy2` library to query the CRI iAtlas database and explore the data the datasets with bulk RNAseq data from studies of response to Immune Checkpoint Inhibitors (ICI)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "#we'll need to use the R package that queries the iAtlas database, the libraries below are necessary to call R code\n",
    "from rpy2.robjects.packages import importr\n",
    "import rpy2.robjects as ro\n",
    "from rpy2.robjects.conversion import localconverter\n",
    "from rpy2.robjects import pandas2ri\n",
    "\n",
    "\n",
    "#libraries for data manipulation\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to run a few functions to call the interface between R and Python and for output formatting. The code below will be used anytime we need to get a table from the CRI iAtlas database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Adding the reference to querying the production version of the API\n",
    "ro.globalenv[\"API_URL\"] = \"https://api.cri-iatlas.org/api\"\n",
    "\n",
    "#Code to query the iAtlas database and converting output to a pandas dataframe\n",
    "iatlasGraphQLClient = importr('iatlasGraphQLClient') \n",
    "\n",
    "def get_dataframe_from_query(query_result):\n",
    "    with localconverter(ro.default_converter + pandas2ri.converter):       \n",
    "        return(ro.conversion.rpy2py(query_result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring the ICI datasets and features\n",
    "\n",
    "\n",
    "The iAtlas ICI data is stored in a database that can be queried with functions from the `iatlasGraphQLClient` package. \n",
    "We have clinical data, immune features, scores of predictors of response to immunotherapy, and quantile normalized gene expression.\n",
    "\n",
    "You can get more information in iAtlas on [immune features and predictors of response to ICI](https://isb-cgc.shinyapps.io/iatlas/?module=datainfo), and our annotation of [immunomodulators](https://isb-cgc.shinyapps.io/iatlas/?module=immunomodulators) genes. You can access more information about these datasets in [iAtlas - ICI Datasets Overview](https://isb-cgc.shinyapps.io/iatlas/?module=ioresponse_overview) module.\n",
    "\n",
    "As a first step, let's take a look at the available datasets and features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>display</th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Chen 2016 - SKCM, Anti-CTLA4</td>\n",
       "      <td>Chen_CanDisc_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Choueiri 2016 - KIRC, PD-1</td>\n",
       "      <td>Choueiri_CCR_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GTEX</td>\n",
       "      <td>GTEX</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>Gide_Cell_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Hugo 2016 - SKCM, PD-1</td>\n",
       "      <td>HugoLo_IPRES_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>IMVigor210 - BLCA, PD-L1</td>\n",
       "      <td>IMVigor210</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>IMmotion150 - KIRC, PD-L1</td>\n",
       "      <td>IMmotion150</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Kim 2018 - STAD, PD-1</td>\n",
       "      <td>Kim_NatMed_2018</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Liu 2019 - SKCM, PD-1</td>\n",
       "      <td>Liu_NatMed_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Melero 2019 - GBM, Anti-PD-1</td>\n",
       "      <td>Melero_GBM_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1</td>\n",
       "      <td>Miao_Science_2018</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>PCAWG</td>\n",
       "      <td>PCAWG</td>\n",
       "      <td>analysis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1</td>\n",
       "      <td>Prat_CanRes_2017</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Prins 2019 - GBM, PD-1</td>\n",
       "      <td>Prins_GBM_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Riaz 2017 - SKCM, PD-1</td>\n",
       "      <td>Riaz_Nivolumab_2017</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>TCGA</td>\n",
       "      <td>TCGA</td>\n",
       "      <td>analysis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Van Allen 2015 - SKCM, CTLA-4</td>\n",
       "      <td>VanAllen_antiCTLA4_2015</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>Zhao_NatMed_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          display                     name  \\\n",
       "1                    Chen 2016 - SKCM, Anti-CTLA4        Chen_CanDisc_2016   \n",
       "2                      Choueiri 2016 - KIRC, PD-1        Choueiri_CCR_2016   \n",
       "3                                            GTEX                     GTEX   \n",
       "4                Gide 2019 - SKCM, PD-1 +/- CTLA4           Gide_Cell_2019   \n",
       "5                          Hugo 2016 - SKCM, PD-1        HugoLo_IPRES_2016   \n",
       "6                        IMVigor210 - BLCA, PD-L1               IMVigor210   \n",
       "7                       IMmotion150 - KIRC, PD-L1              IMmotion150   \n",
       "8                           Kim 2018 - STAD, PD-1          Kim_NatMed_2018   \n",
       "9                           Liu 2019 - SKCM, PD-1          Liu_NatMed_2019   \n",
       "10                   Melero 2019 - GBM, Anti-PD-1          Melero_GBM_2019   \n",
       "11        Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1        Miao_Science_2018   \n",
       "12                                          PCAWG                    PCAWG   \n",
       "13  Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1         Prat_CanRes_2017   \n",
       "14                         Prins 2019 - GBM, PD-1           Prins_GBM_2019   \n",
       "15                         Riaz 2017 - SKCM, PD-1      Riaz_Nivolumab_2017   \n",
       "16                                           TCGA                     TCGA   \n",
       "17                  Van Allen 2015 - SKCM, CTLA-4  VanAllen_antiCTLA4_2015   \n",
       "18                          Zhao 2019 - GBM, PD-1         Zhao_NatMed_2019   \n",
       "\n",
       "        type  \n",
       "1        ici  \n",
       "2        ici  \n",
       "3      other  \n",
       "4        ici  \n",
       "5        ici  \n",
       "6        ici  \n",
       "7        ici  \n",
       "8        ici  \n",
       "9        ici  \n",
       "10       ici  \n",
       "11       ici  \n",
       "12  analysis  \n",
       "13       ici  \n",
       "14       ici  \n",
       "15       ici  \n",
       "16  analysis  \n",
       "17       ici  \n",
       "18       ici  "
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iatlas_datasets = get_dataframe_from_query(iatlasGraphQLClient.query_datasets())\n",
    "iatlas_datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type\n",
       "ici         15\n",
       "scrna        6\n",
       "analysis     2\n",
       "other        1\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iatlas_datasets[\"type\"].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will explore the datasets of type `ici` , as those contain bulk RNAseq data from studies of response to Immune Checkpoint Inhibitors (ICI).\n",
    "\n",
    "The datasets of type `scrna` contain single-cell RNAseq data with annotation of immune cells. A few of them have ICI treatments.\n",
    "\n",
    "Finally, the other main type of dataset in CRI iAtlas is `analysis`, and these datasets are derived from cancer genomics efforts, The Cancer Genome Atlas (TCGA) and Pan-Cancer Analysis of Whole Genomes (PCAWG)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>display</th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Chen 2016 - SKCM, Anti-CTLA4</td>\n",
       "      <td>Chen_CanDisc_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Choueiri 2016 - KIRC, PD-1</td>\n",
       "      <td>Choueiri_CCR_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>Gide_Cell_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Hugo 2016 - SKCM, PD-1</td>\n",
       "      <td>HugoLo_IPRES_2016</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>IMVigor210 - BLCA, PD-L1</td>\n",
       "      <td>IMVigor210</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>IMmotion150 - KIRC, PD-L1</td>\n",
       "      <td>IMmotion150</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Kim 2018 - STAD, PD-1</td>\n",
       "      <td>Kim_NatMed_2018</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Liu 2019 - SKCM, PD-1</td>\n",
       "      <td>Liu_NatMed_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Melero 2019 - GBM, Anti-PD-1</td>\n",
       "      <td>Melero_GBM_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1</td>\n",
       "      <td>Miao_Science_2018</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1</td>\n",
       "      <td>Prat_CanRes_2017</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Prins 2019 - GBM, PD-1</td>\n",
       "      <td>Prins_GBM_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Riaz 2017 - SKCM, PD-1</td>\n",
       "      <td>Riaz_Nivolumab_2017</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Van Allen 2015 - SKCM, CTLA-4</td>\n",
       "      <td>VanAllen_antiCTLA4_2015</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>Zhao_NatMed_2019</td>\n",
       "      <td>ici</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          display                     name  \\\n",
       "2                    Chen 2016 - SKCM, Anti-CTLA4        Chen_CanDisc_2016   \n",
       "3                      Choueiri 2016 - KIRC, PD-1        Choueiri_CCR_2016   \n",
       "5                Gide 2019 - SKCM, PD-1 +/- CTLA4           Gide_Cell_2019   \n",
       "6                          Hugo 2016 - SKCM, PD-1        HugoLo_IPRES_2016   \n",
       "7                        IMVigor210 - BLCA, PD-L1               IMVigor210   \n",
       "8                       IMmotion150 - KIRC, PD-L1              IMmotion150   \n",
       "9                           Kim 2018 - STAD, PD-1          Kim_NatMed_2018   \n",
       "12                          Liu 2019 - SKCM, PD-1          Liu_NatMed_2019   \n",
       "14                   Melero 2019 - GBM, Anti-PD-1          Melero_GBM_2019   \n",
       "15        Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1        Miao_Science_2018   \n",
       "17  Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1         Prat_CanRes_2017   \n",
       "18                         Prins 2019 - GBM, PD-1           Prins_GBM_2019   \n",
       "19                         Riaz 2017 - SKCM, PD-1      Riaz_Nivolumab_2017   \n",
       "22                  Van Allen 2015 - SKCM, CTLA-4  VanAllen_antiCTLA4_2015   \n",
       "24                          Zhao 2019 - GBM, PD-1         Zhao_NatMed_2019   \n",
       "\n",
       "   type  \n",
       "2   ici  \n",
       "3   ici  \n",
       "5   ici  \n",
       "6   ici  \n",
       "7   ici  \n",
       "8   ici  \n",
       "9   ici  \n",
       "12  ici  \n",
       "14  ici  \n",
       "15  ici  \n",
       "17  ici  \n",
       "18  ici  \n",
       "19  ici  \n",
       "22  ici  \n",
       "24  ici  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ici_datasets = iatlas_datasets[iatlas_datasets[\"type\"] == \"ici\"]\n",
    "ici_datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The display name of the datasets makes reference to the publication associated with the data, and also summarises the tumor type and ICI target involved in the study."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Immune Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>display</th>\n",
       "      <th>class</th>\n",
       "      <th>order</th>\n",
       "      <th>unit</th>\n",
       "      <th>method_tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B_cells_Aggregate2</td>\n",
       "      <td>B Cells</td>\n",
       "      <td>Immune Cell Proportion - Common Lymphoid and M...</td>\n",
       "      <td>3</td>\n",
       "      <td>Fraction</td>\n",
       "      <td>CIBERSORT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>B_cells_Aggregate3</td>\n",
       "      <td>B Cells</td>\n",
       "      <td>Immune Cell Proportion - Differentiated Lympho...</td>\n",
       "      <td>4</td>\n",
       "      <td>Fraction</td>\n",
       "      <td>CIBERSORT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>B_cells_memory</td>\n",
       "      <td>B Cells Memory</td>\n",
       "      <td>Immune Cell Proportion - Original</td>\n",
       "      <td>9</td>\n",
       "      <td>Fraction</td>\n",
       "      <td>CIBERSORT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>B_cells_naive</td>\n",
       "      <td>B Cells Naive</td>\n",
       "      <td>Immune Cell Proportion - Original</td>\n",
       "      <td>8</td>\n",
       "      <td>Fraction</td>\n",
       "      <td>CIBERSORT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Fraction</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                name         display  \\\n",
       "1                 B_cells_Aggregate2         B Cells   \n",
       "2                 B_cells_Aggregate3         B Cells   \n",
       "3                     B_cells_memory  B Cells Memory   \n",
       "4                      B_cells_naive   B Cells Naive   \n",
       "5  BIOCARTA_CTLA4_V_Bindea_Th1_Cells    CTLA4 vs Th1   \n",
       "\n",
       "                                               class       order      unit  \\\n",
       "1  Immune Cell Proportion - Common Lymphoid and M...           3  Fraction   \n",
       "2  Immune Cell Proportion - Differentiated Lympho...           4  Fraction   \n",
       "3                  Immune Cell Proportion - Original           9  Fraction   \n",
       "4                  Immune Cell Proportion - Original           8  Fraction   \n",
       "5  Predictor of Response to Immune Checkpoint Tre... -2147483648  Fraction   \n",
       "\n",
       "      method_tag  \n",
       "1      CIBERSORT  \n",
       "2      CIBERSORT  \n",
       "3      CIBERSORT  \n",
       "4      CIBERSORT  \n",
       "5  NA_character_  "
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#immune features of all ICI samples.\n",
    "with (ro.default_converter + pandas2ri.converter).context(): #we need this snippet of code when we want to send a parameter to the query\n",
    "    features_df = (iatlasGraphQLClient.query_features(cohorts = ici_datasets[\"name\"]))\n",
    "features_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clinical Annotation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tag_name</th>\n",
       "      <th>tag_long_display</th>\n",
       "      <th>tag_short_display</th>\n",
       "      <th>tag_characteristics</th>\n",
       "      <th>tag_color</th>\n",
       "      <th>tag_order</th>\n",
       "      <th>tag_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Biopsy_Site</td>\n",
       "      <td>Biopsy Site</td>\n",
       "      <td>Biopsy Site</td>\n",
       "      <td>Site where sample was collected from.</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>18</td>\n",
       "      <td>parent_group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Cancer_Tissue</td>\n",
       "      <td>Cancer Tissue</td>\n",
       "      <td>Cancer Tissue</td>\n",
       "      <td>Original tumor tissue.</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>14</td>\n",
       "      <td>parent_group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Clinical_Benefit</td>\n",
       "      <td>Clinical Benefit</td>\n",
       "      <td>Clinical Benefit</td>\n",
       "      <td>Patients have clinical benefit when mRECIST re...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>4</td>\n",
       "      <td>parent_group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Clinical_Stage</td>\n",
       "      <td>Clinical Stage</td>\n",
       "      <td>Clinical Stage</td>\n",
       "      <td>Clinical stage of cancer.</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>17</td>\n",
       "      <td>parent_group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>FFPE</td>\n",
       "      <td>FFPE Samples</td>\n",
       "      <td>FFPE Samples</td>\n",
       "      <td>Indicates whether the sample is FFPE or not.</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>20</td>\n",
       "      <td>parent_group</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           tag_name  tag_long_display tag_short_display  \\\n",
       "1       Biopsy_Site       Biopsy Site       Biopsy Site   \n",
       "2     Cancer_Tissue     Cancer Tissue     Cancer Tissue   \n",
       "3  Clinical_Benefit  Clinical Benefit  Clinical Benefit   \n",
       "4    Clinical_Stage    Clinical Stage    Clinical Stage   \n",
       "5              FFPE      FFPE Samples      FFPE Samples   \n",
       "\n",
       "                                 tag_characteristics   tag_color  tag_order  \\\n",
       "1              Site where sample was collected from. -2147483648         18   \n",
       "2                             Original tumor tissue. -2147483648         14   \n",
       "3  Patients have clinical benefit when mRECIST re... -2147483648          4   \n",
       "4                          Clinical stage of cancer. -2147483648         17   \n",
       "5       Indicates whether the sample is FFPE or not. -2147483648         20   \n",
       "\n",
       "       tag_type  \n",
       "1  parent_group  \n",
       "2  parent_group  \n",
       "3  parent_group  \n",
       "4  parent_group  \n",
       "5  parent_group  "
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#clinical annotation that is available for the ici datasets\n",
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "    clinical_options = iatlasGraphQLClient.query_tags(datasets = ici_datasets[\"name\"])\n",
    "clinical_options.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gene Expression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>hgnc</th>\n",
       "      <th>entrez</th>\n",
       "      <th>description</th>\n",
       "      <th>friendly_name</th>\n",
       "      <th>io_landscape_name</th>\n",
       "      <th>gene_family</th>\n",
       "      <th>gene_function</th>\n",
       "      <th>immune_checkpoint</th>\n",
       "      <th>pathway</th>\n",
       "      <th>super_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ABCB5</td>\n",
       "      <td>340273</td>\n",
       "      <td>A protein highly expressed by melanoma cell, a...</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>ABCB5</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>ABC-family proteins mediated transport</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABCC1</td>\n",
       "      <td>4363</td>\n",
       "      <td>MRP1 is a membrane transporter and it allows t...</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>MRP1</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>ABC-family proteins mediated transport</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACKR3</td>\n",
       "      <td>57007</td>\n",
       "      <td>CXCR7 is the receptor for chemokines CXCL11 an...</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>CXCR7</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>Chemokine signaling pathway</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ACP3</td>\n",
       "      <td>55</td>\n",
       "      <td>A enzyme produced by the prostate and generall...</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>ACPP</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>Innate Immune System</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ADAM17</td>\n",
       "      <td>6868</td>\n",
       "      <td>Belongs to metallopeptidase family and help th...</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>ADAM17</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>NA_character_</td>\n",
       "      <td>Metallopeptidase</td>\n",
       "      <td>NA_character_</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     hgnc  entrez                                        description  \\\n",
       "1   ABCB5  340273  A protein highly expressed by melanoma cell, a...   \n",
       "2   ABCC1    4363  MRP1 is a membrane transporter and it allows t...   \n",
       "3   ACKR3   57007  CXCR7 is the receptor for chemokines CXCL11 an...   \n",
       "4    ACP3      55  A enzyme produced by the prostate and generall...   \n",
       "5  ADAM17    6868  Belongs to metallopeptidase family and help th...   \n",
       "\n",
       "   friendly_name io_landscape_name    gene_family  gene_function  \\\n",
       "1  NA_character_             ABCB5  NA_character_  NA_character_   \n",
       "2  NA_character_              MRP1  NA_character_  NA_character_   \n",
       "3  NA_character_             CXCR7  NA_character_  NA_character_   \n",
       "4  NA_character_              ACPP  NA_character_  NA_character_   \n",
       "5  NA_character_            ADAM17  NA_character_  NA_character_   \n",
       "\n",
       "  immune_checkpoint                                 pathway super_category  \n",
       "1     NA_character_  ABC-family proteins mediated transport  NA_character_  \n",
       "2     NA_character_  ABC-family proteins mediated transport  NA_character_  \n",
       "3     NA_character_             Chemokine signaling pathway  NA_character_  \n",
       "4     NA_character_                    Innate Immune System  NA_character_  \n",
       "5     NA_character_                        Metallopeptidase  NA_character_  "
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#genes that we have expression data for all samples in the ici datasets (we will query expression values in the next section)\n",
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "    genes_df = iatlasGraphQLClient.query_genes(cohorts = ici_datasets[\"name\"])\n",
    "genes_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Annotation of response to immunotherapy\n",
    "\n",
    "One key annotation in the ICI datasets is how a patient responds to immunotherapy. The response to therapy with Immune Checkpoint Inhitbitor is originally annotated following the guidelines of mRECIST, and has 4 different levels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tag_name</th>\n",
       "      <th>tag_long_display</th>\n",
       "      <th>tag_short_display</th>\n",
       "      <th>tag_characteristics</th>\n",
       "      <th>tag_color</th>\n",
       "      <th>tag_order</th>\n",
       "      <th>tag_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>complete_response_response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response (CR) following modified Resp...</td>\n",
       "      <td>#009E73</td>\n",
       "      <td>1</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>na_response</td>\n",
       "      <td>Not available</td>\n",
       "      <td>Not available</td>\n",
       "      <td>Response information not available</td>\n",
       "      <td>#868A88</td>\n",
       "      <td>5</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>partial_response_response</td>\n",
       "      <td>Partial Response</td>\n",
       "      <td>Partial Response</td>\n",
       "      <td>Partial Response (PR) following modified Respo...</td>\n",
       "      <td>#0072B2</td>\n",
       "      <td>2</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>progressive_disease_response</td>\n",
       "      <td>Progressive Disease</td>\n",
       "      <td>Progressive Disease</td>\n",
       "      <td>Progressive Disease (PD) following modified Re...</td>\n",
       "      <td>#D55E00</td>\n",
       "      <td>4</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>stable_disease_response</td>\n",
       "      <td>Stable Disease</td>\n",
       "      <td>Stable Disease</td>\n",
       "      <td>Stable Disease (SD) following modified Respons...</td>\n",
       "      <td>#F0E442</td>\n",
       "      <td>3</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       tag_name     tag_long_display    tag_short_display  \\\n",
       "1    complete_response_response    Complete Response    Complete Response   \n",
       "2                   na_response        Not available        Not available   \n",
       "3     partial_response_response     Partial Response     Partial Response   \n",
       "4  progressive_disease_response  Progressive Disease  Progressive Disease   \n",
       "5       stable_disease_response       Stable Disease       Stable Disease   \n",
       "\n",
       "                                 tag_characteristics tag_color  tag_order  \\\n",
       "1  Complete Response (CR) following modified Resp...   #009E73          1   \n",
       "2                 Response information not available   #868A88          5   \n",
       "3  Partial Response (PR) following modified Respo...   #0072B2          2   \n",
       "4  Progressive Disease (PD) following modified Re...   #D55E00          4   \n",
       "5  Stable Disease (SD) following modified Respons...   #F0E442          3   \n",
       "\n",
       "  tag_type  \n",
       "1    group  \n",
       "2    group  \n",
       "3    group  \n",
       "4    group  \n",
       "5    group  "
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "    df = iatlasGraphQLClient.query_tags(parent_tags = \"Response\")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mRECIST annotation is used to annotate three other categories: Responder, Clinical Benefit and Progressor, which consist in grouping different levels of mRECIST into two categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>parent_tag_name</th>\n",
       "      <th>parent_tag_characteristics</th>\n",
       "      <th>tag_name</th>\n",
       "      <th>tag_short_display</th>\n",
       "      <th>tag_characteristics</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Clinical_Benefit</td>\n",
       "      <td>Patients have clinical benefit when mRECIST re...</td>\n",
       "      <td>false_clinical_benefit</td>\n",
       "      <td>No Clinical Benefit</td>\n",
       "      <td>Patient with mRECIST of Progressive Disease</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Progression</td>\n",
       "      <td>Progressors are defined as patients with mRECI...</td>\n",
       "      <td>false_progression</td>\n",
       "      <td>Non-Progressor</td>\n",
       "      <td>Patient with mRECIST of Complete Response, Par...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Responder</td>\n",
       "      <td>Responders are defined as patients with mRECIS...</td>\n",
       "      <td>false_responder</td>\n",
       "      <td>Non-Responder</td>\n",
       "      <td>Patient with mRECIST of Progressive Disease or...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Clinical_Benefit</td>\n",
       "      <td>Patients have clinical benefit when mRECIST re...</td>\n",
       "      <td>na_clinical_benefit</td>\n",
       "      <td>Not available</td>\n",
       "      <td>Clinical Benefit information not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Progression</td>\n",
       "      <td>Progressors are defined as patients with mRECI...</td>\n",
       "      <td>na_progression</td>\n",
       "      <td>Not available</td>\n",
       "      <td>Progression information not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Responder</td>\n",
       "      <td>Responders are defined as patients with mRECIS...</td>\n",
       "      <td>na_responder</td>\n",
       "      <td>Not available</td>\n",
       "      <td>Responder information not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Clinical_Benefit</td>\n",
       "      <td>Patients have clinical benefit when mRECIST re...</td>\n",
       "      <td>true_clinical_benefit</td>\n",
       "      <td>Clinical Benefit</td>\n",
       "      <td>Patient with mRECIST of Complete Response, Par...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Progression</td>\n",
       "      <td>Progressors are defined as patients with mRECI...</td>\n",
       "      <td>true_progression</td>\n",
       "      <td>Progressor</td>\n",
       "      <td>Patient with mRECIST of Progressive Disease</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Responder</td>\n",
       "      <td>Responders are defined as patients with mRECIS...</td>\n",
       "      <td>true_responder</td>\n",
       "      <td>Responder</td>\n",
       "      <td>Patient with mRECIST of Partial Response or Co...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    parent_tag_name                         parent_tag_characteristics  \\\n",
       "1  Clinical_Benefit  Patients have clinical benefit when mRECIST re...   \n",
       "2       Progression  Progressors are defined as patients with mRECI...   \n",
       "3         Responder  Responders are defined as patients with mRECIS...   \n",
       "4  Clinical_Benefit  Patients have clinical benefit when mRECIST re...   \n",
       "5       Progression  Progressors are defined as patients with mRECI...   \n",
       "6         Responder  Responders are defined as patients with mRECIS...   \n",
       "7  Clinical_Benefit  Patients have clinical benefit when mRECIST re...   \n",
       "8       Progression  Progressors are defined as patients with mRECI...   \n",
       "9         Responder  Responders are defined as patients with mRECIS...   \n",
       "\n",
       "                 tag_name    tag_short_display  \\\n",
       "1  false_clinical_benefit  No Clinical Benefit   \n",
       "2       false_progression       Non-Progressor   \n",
       "3         false_responder        Non-Responder   \n",
       "4     na_clinical_benefit        Not available   \n",
       "5          na_progression        Not available   \n",
       "6            na_responder        Not available   \n",
       "7   true_clinical_benefit     Clinical Benefit   \n",
       "8        true_progression           Progressor   \n",
       "9          true_responder            Responder   \n",
       "\n",
       "                                 tag_characteristics  \n",
       "1        Patient with mRECIST of Progressive Disease  \n",
       "2  Patient with mRECIST of Complete Response, Par...  \n",
       "3  Patient with mRECIST of Progressive Disease or...  \n",
       "4         Clinical Benefit information not available  \n",
       "5              Progression information not available  \n",
       "6                Responder information not available  \n",
       "7  Patient with mRECIST of Complete Response, Par...  \n",
       "8        Patient with mRECIST of Progressive Disease  \n",
       "9  Patient with mRECIST of Partial Response or Co...  "
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "outcome_variables  = [\"Responder\", \"Clinical_Benefit\", \"Progression\"]\n",
    "get_dataframe_from_query(\n",
    "    iatlasGraphQLClient.query_tags_with_parent_tags(parent_tags = outcome_variables)\n",
    "    )[[\"parent_tag_name\", \"parent_tag_characteristics\", \"tag_name\", \"tag_short_display\", \"tag_characteristics\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Samples and treatments available\n",
    "\n",
    "Now, let's take a closer look at the ICI datasets that we have available. First, we will query our database and organize the results to see the TCGA Study and drug administered at each one of the studies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Treatment information\n",
    "overview_treatment = [\"TCGA_Study\", \"ICI_Rx\"] #name of the groups of interest. Check the clinical_options df for more options\n",
    "\n",
    "#Organize a dataframe with all patients IDs and samples IDs\n",
    "with (ro.default_converter + pandas2ri.converter).context(): \n",
    "  all_ici_patients = iatlasGraphQLClient.query_dataset_samples(datasets = ici_datasets[\"name\"]).merge( #add patient id info\n",
    "    get_dataframe_from_query(iatlasGraphQLClient.query_sample_patients()))\n",
    "\n",
    "#now we add the treatment information\n",
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "  #query the values of TCGA_Study and ICI_Rx for each sample\n",
    "  treatment_df = iatlasGraphQLClient.query_tag_samples_parents(parent_tags = overview_treatment).merge( \n",
    "    all_ici_patients\n",
    "  )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dataset_display</th>\n",
       "      <th>parent_tag_name</th>\n",
       "      <th>tag_name</th>\n",
       "      <th>n_samples</th>\n",
       "      <th>n_patients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Choueiri 2016 - KIRC, PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>32</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>ipilimumab_pembrolizumab</td>\n",
       "      <td>32</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>IMVigor210 - BLCA, PD-L1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>atezolizumab</td>\n",
       "      <td>348</td>\n",
       "      <td>348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>IMVigor210 - BLCA, PD-L1</td>\n",
       "      <td>TCGA_Study</td>\n",
       "      <td>BLCA</td>\n",
       "      <td>348</td>\n",
       "      <td>348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>IMmotion150 - KIRC, PD-L1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>atezolizumab</td>\n",
       "      <td>174</td>\n",
       "      <td>174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Liu 2019 - SKCM, PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>102</td>\n",
       "      <td>51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Melero 2019 - GBM, Anti-PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>120</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>atezolizumab</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>22</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>56</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Prins 2019 - GBM, PD-1</td>\n",
       "      <td>TCGA_Study</td>\n",
       "      <td>GBM</td>\n",
       "      <td>30</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Riaz 2017 - SKCM, PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>214</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>ICI_Rx</td>\n",
       "      <td>nivolumab</td>\n",
       "      <td>56</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>TCGA_Study</td>\n",
       "      <td>GBM</td>\n",
       "      <td>34</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  dataset_display parent_tag_name  \\\n",
       "0                      Choueiri 2016 - KIRC, PD-1          ICI_Rx   \n",
       "1                Gide 2019 - SKCM, PD-1 +/- CTLA4          ICI_Rx   \n",
       "2                Gide 2019 - SKCM, PD-1 +/- CTLA4          ICI_Rx   \n",
       "3                        IMVigor210 - BLCA, PD-L1          ICI_Rx   \n",
       "4                        IMVigor210 - BLCA, PD-L1      TCGA_Study   \n",
       "5                       IMmotion150 - KIRC, PD-L1          ICI_Rx   \n",
       "6                           Liu 2019 - SKCM, PD-1          ICI_Rx   \n",
       "7                    Melero 2019 - GBM, Anti-PD-1          ICI_Rx   \n",
       "8         Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1          ICI_Rx   \n",
       "9         Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1          ICI_Rx   \n",
       "10  Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1          ICI_Rx   \n",
       "11                         Prins 2019 - GBM, PD-1      TCGA_Study   \n",
       "12                         Riaz 2017 - SKCM, PD-1          ICI_Rx   \n",
       "13                          Zhao 2019 - GBM, PD-1          ICI_Rx   \n",
       "14                          Zhao 2019 - GBM, PD-1      TCGA_Study   \n",
       "\n",
       "                    tag_name  n_samples  n_patients  \n",
       "0                  nivolumab         32          16  \n",
       "1   ipilimumab_pembrolizumab         32          26  \n",
       "2                  nivolumab         22           9  \n",
       "3               atezolizumab        348         348  \n",
       "4                       BLCA        348         348  \n",
       "5               atezolizumab        174         174  \n",
       "6                  nivolumab        102          51  \n",
       "7                  nivolumab        120          30  \n",
       "8               atezolizumab          2           2  \n",
       "9                  nivolumab         22          11  \n",
       "10                 nivolumab         56          28  \n",
       "11                       GBM         30          30  \n",
       "12                 nivolumab        214          64  \n",
       "13                 nivolumab         56          13  \n",
       "14                       GBM         34          17  "
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can summarise the information in this dataframe\n",
    "summary = treatment_df.groupby(['dataset_display', 'parent_tag_name', 'tag_name']).agg(\n",
    "    n_samples=pd.NamedAgg(column='sample_name', aggfunc='size'),\n",
    "    n_patients=pd.NamedAgg(column='patient_name', aggfunc='nunique')\n",
    ").reset_index()\n",
    "summary\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of these datasets have more than one sample per patient - in those studies, some patients had samples collected before (pre_sample_treatment) and during (on_sample_treatment) ICI therapy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dataset_display</th>\n",
       "      <th>parent_tag_name</th>\n",
       "      <th>tag_name</th>\n",
       "      <th>n_samples</th>\n",
       "      <th>n_patients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Chen 2016 - SKCM, Anti-CTLA4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Chen 2016 - SKCM, Anti-CTLA4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>post_sample_treatment</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Chen 2016 - SKCM, Anti-CTLA4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>32</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Choueiri 2016 - KIRC, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>18</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Gide 2019 - SKCM, PD-1 +/- CTLA4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>73</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Hugo 2016 - SKCM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hugo 2016 - SKCM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>26</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>IMVigor210 - BLCA, PD-L1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>348</td>\n",
       "      <td>348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>IMmotion150 - KIRC, PD-L1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>263</td>\n",
       "      <td>263</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Kim 2018 - STAD, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>45</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Liu 2019 - SKCM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>122</td>\n",
       "      <td>122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Melero 2019 - GBM, Anti-PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>30</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Melero 2019 - GBM, Anti-PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>30</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>17</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>65</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Prins 2019 - GBM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Prins 2019 - GBM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Riaz 2017 - SKCM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>56</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Riaz 2017 - SKCM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>51</td>\n",
       "      <td>51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Van Allen 2015 - SKCM, CTLA-4</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>42</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>on_sample_treatment</td>\n",
       "      <td>9</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Zhao 2019 - GBM, PD-1</td>\n",
       "      <td>Sample_Treatment</td>\n",
       "      <td>pre_sample_treatment</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  dataset_display   parent_tag_name  \\\n",
       "0                    Chen 2016 - SKCM, Anti-CTLA4  Sample_Treatment   \n",
       "1                    Chen 2016 - SKCM, Anti-CTLA4  Sample_Treatment   \n",
       "2                    Chen 2016 - SKCM, Anti-CTLA4  Sample_Treatment   \n",
       "3                      Choueiri 2016 - KIRC, PD-1  Sample_Treatment   \n",
       "4                Gide 2019 - SKCM, PD-1 +/- CTLA4  Sample_Treatment   \n",
       "5                Gide 2019 - SKCM, PD-1 +/- CTLA4  Sample_Treatment   \n",
       "6                          Hugo 2016 - SKCM, PD-1  Sample_Treatment   \n",
       "7                          Hugo 2016 - SKCM, PD-1  Sample_Treatment   \n",
       "8                        IMVigor210 - BLCA, PD-L1  Sample_Treatment   \n",
       "9                       IMmotion150 - KIRC, PD-L1  Sample_Treatment   \n",
       "10                          Kim 2018 - STAD, PD-1  Sample_Treatment   \n",
       "11                          Liu 2019 - SKCM, PD-1  Sample_Treatment   \n",
       "12                   Melero 2019 - GBM, Anti-PD-1  Sample_Treatment   \n",
       "13                   Melero 2019 - GBM, Anti-PD-1  Sample_Treatment   \n",
       "14        Miao 2018 - KIRC, PD-1 +/- CTLA4, PD-L1  Sample_Treatment   \n",
       "15  Prat 2017 - HNSC, LUAD, LUSC, SKCM, Anti-PD-1  Sample_Treatment   \n",
       "16                         Prins 2019 - GBM, PD-1  Sample_Treatment   \n",
       "17                         Prins 2019 - GBM, PD-1  Sample_Treatment   \n",
       "18                         Riaz 2017 - SKCM, PD-1  Sample_Treatment   \n",
       "19                         Riaz 2017 - SKCM, PD-1  Sample_Treatment   \n",
       "20                  Van Allen 2015 - SKCM, CTLA-4  Sample_Treatment   \n",
       "21                          Zhao 2019 - GBM, PD-1  Sample_Treatment   \n",
       "22                          Zhao 2019 - GBM, PD-1  Sample_Treatment   \n",
       "\n",
       "                 tag_name  n_samples  n_patients  \n",
       "0     on_sample_treatment         15          15  \n",
       "1   post_sample_treatment          7           7  \n",
       "2    pre_sample_treatment         32          31  \n",
       "3    pre_sample_treatment         16          16  \n",
       "4     on_sample_treatment         18          18  \n",
       "5    pre_sample_treatment         73          73  \n",
       "6     on_sample_treatment          1           1  \n",
       "7    pre_sample_treatment         26          26  \n",
       "8    pre_sample_treatment        348         348  \n",
       "9    pre_sample_treatment        263         263  \n",
       "10   pre_sample_treatment         45          45  \n",
       "11   pre_sample_treatment        122         122  \n",
       "12    on_sample_treatment         30          30  \n",
       "13   pre_sample_treatment         30          30  \n",
       "14   pre_sample_treatment         17          17  \n",
       "15   pre_sample_treatment         65          65  \n",
       "16    on_sample_treatment         15          15  \n",
       "17   pre_sample_treatment         15          15  \n",
       "18    on_sample_treatment         56          56  \n",
       "19   pre_sample_treatment         51          51  \n",
       "20   pre_sample_treatment         42          41  \n",
       "21    on_sample_treatment          9           7  \n",
       "22   pre_sample_treatment         25          16  "
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "    timepoint = all_ici_patients.merge(\n",
    "        iatlasGraphQLClient.query_tag_samples_parents(parent_tags = \"Sample_Treatment\"),\n",
    "    ).groupby(['dataset_display', 'parent_tag_name', 'tag_name']).agg(\n",
    "        n_samples=pd.NamedAgg(column='sample_name', aggfunc='size'),\n",
    "        n_patients=pd.NamedAgg(column='patient_name', aggfunc='nunique')\n",
    "    ).reset_index()\n",
    "\n",
    "timepoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Querying the CRI iAtlas database\n",
    "\n",
    "Each type of data has a query function in `iatlasGraphQLClient`, as summarized below:\n",
    "\n",
    "- For immunefeatures: `iatlasGraphQLClient.query_feature_values()`\n",
    "- For gene expression data: `iatlasGraphQLClient.query_gene_expression()`\n",
    "- For clinical annotation: `iatlasGraphQLClient.query_tag_samples_parents()`\n",
    "\n",
    "To illustrate how to get the ICI data from the iAtlas database, we will get the data that follows the parameters listed below:\n",
    "\n",
    "- *Dataset:* Hugo 2016 - SKCM, PD-1\n",
    "\n",
    "- *Features:* IMPRES (Auslander et al, 2018), IPRES (Vincent lab analysis of Hugo et al data, unpublished), Cytolytic Score (Roufas et al, 2018), CTLA4/Th1 (Nishimura, 2004; Bindea et al., 2013)\n",
    "\n",
    "- *Gene expression:* ADORA2A, CTLA4, EDNRB, TLR4\n",
    "\n",
    "- *Clinical Annotation:* Response, Gender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "with (ro.default_converter + pandas2ri.converter).context():\n",
    "    #For immune features, use names from features_df[\"name\"]\n",
    "    features = iatlasGraphQLClient.query_feature_values(cohorts = \"HugoLo_IPRES_2016\", \n",
    "                                                        features = [\"IMPRES\", \n",
    "                                                                    \"Vincent_IPRES_NonResponder\", \n",
    "                                                                    \"Cytolytic_Score\", \n",
    "                                                                    \"BIOCARTA_CTLA4_V_Bindea_Th1_Cells\"]) \n",
    "\n",
    "    #For gene expression, we need the gene Entrez ID to query the iAtlas database\n",
    "    genes_entrez = [135,\n",
    "                    1493,\n",
    "                    1910,\n",
    "                    7099]\n",
    "\n",
    "    genes = iatlasGraphQLClient.query_gene_expression(cohorts = \"HugoLo_IPRES_2016\", \n",
    "                                                    entrez = genes_entrez)\n",
    "\n",
    "    #For clinical annotation, use names from clinical_options[\"tag_name\"] that have tag_type == \"parent_tag\"\n",
    "\n",
    "    clinical_annotation  = iatlasGraphQLClient.query_tag_samples_parents(cohorts = \"HugoLo_IPRES_2016\", \n",
    "                                                        parent_tags = [\"Response\", \"Gender\"]) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All tables are in long format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample</th>\n",
       "      <th>feature_name</th>\n",
       "      <th>feature_display</th>\n",
       "      <th>feature_value</th>\n",
       "      <th>feature_order</th>\n",
       "      <th>feature_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt01-ar-279</td>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>1.001012</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt02-ar-280</td>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>1.190649</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt04-ar-281</td>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>1.179757</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt05-ar-282</td>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>1.171726</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt06-ar-283</td>\n",
       "      <td>BIOCARTA_CTLA4_V_Bindea_Th1_Cells</td>\n",
       "      <td>CTLA4 vs Th1</td>\n",
       "      <td>1.290074</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>Predictor of Response to Immune Checkpoint Tre...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          sample                       feature_name  \\\n",
       "1  HugoLo_IPRES_2016-Pt01-ar-279  BIOCARTA_CTLA4_V_Bindea_Th1_Cells   \n",
       "2  HugoLo_IPRES_2016-Pt02-ar-280  BIOCARTA_CTLA4_V_Bindea_Th1_Cells   \n",
       "3  HugoLo_IPRES_2016-Pt04-ar-281  BIOCARTA_CTLA4_V_Bindea_Th1_Cells   \n",
       "4  HugoLo_IPRES_2016-Pt05-ar-282  BIOCARTA_CTLA4_V_Bindea_Th1_Cells   \n",
       "5  HugoLo_IPRES_2016-Pt06-ar-283  BIOCARTA_CTLA4_V_Bindea_Th1_Cells   \n",
       "\n",
       "  feature_display  feature_value  feature_order  \\\n",
       "1    CTLA4 vs Th1       1.001012    -2147483648   \n",
       "2    CTLA4 vs Th1       1.190649    -2147483648   \n",
       "3    CTLA4 vs Th1       1.179757    -2147483648   \n",
       "4    CTLA4 vs Th1       1.171726    -2147483648   \n",
       "5    CTLA4 vs Th1       1.290074    -2147483648   \n",
       "\n",
       "                                       feature_class  \n",
       "1  Predictor of Response to Immune Checkpoint Tre...  \n",
       "2  Predictor of Response to Immune Checkpoint Tre...  \n",
       "3  Predictor of Response to Immune Checkpoint Tre...  \n",
       "4  Predictor of Response to Immune Checkpoint Tre...  \n",
       "5  Predictor of Response to Immune Checkpoint Tre...  "
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#All the tables are in the long format.\n",
    "features.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample</th>\n",
       "      <th>entrez</th>\n",
       "      <th>hgnc</th>\n",
       "      <th>rna_seq_expr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt20-ar-294</td>\n",
       "      <td>7099</td>\n",
       "      <td>TLR4</td>\n",
       "      <td>1315.887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt22-ar-295</td>\n",
       "      <td>7099</td>\n",
       "      <td>TLR4</td>\n",
       "      <td>4520.116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt05-ar-282</td>\n",
       "      <td>7099</td>\n",
       "      <td>TLR4</td>\n",
       "      <td>394.303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt14-ar-290</td>\n",
       "      <td>7099</td>\n",
       "      <td>TLR4</td>\n",
       "      <td>2374.693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt06-ar-283</td>\n",
       "      <td>7099</td>\n",
       "      <td>TLR4</td>\n",
       "      <td>3383.161</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          sample  entrez  hgnc  rna_seq_expr\n",
       "1  HugoLo_IPRES_2016-Pt20-ar-294    7099  TLR4      1315.887\n",
       "2  HugoLo_IPRES_2016-Pt22-ar-295    7099  TLR4      4520.116\n",
       "3  HugoLo_IPRES_2016-Pt05-ar-282    7099  TLR4       394.303\n",
       "4  HugoLo_IPRES_2016-Pt14-ar-290    7099  TLR4      2374.693\n",
       "5  HugoLo_IPRES_2016-Pt06-ar-283    7099  TLR4      3383.161"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genes.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample_name</th>\n",
       "      <th>parent_tag_name</th>\n",
       "      <th>parent_tag_long_display</th>\n",
       "      <th>parent_tag_short_display</th>\n",
       "      <th>parent_tag_characteristics</th>\n",
       "      <th>parent_tag_color</th>\n",
       "      <th>parent_tag_order</th>\n",
       "      <th>parent_tag_type</th>\n",
       "      <th>tag_name</th>\n",
       "      <th>tag_long_display</th>\n",
       "      <th>tag_short_display</th>\n",
       "      <th>tag_characteristics</th>\n",
       "      <th>tag_color</th>\n",
       "      <th>tag_order</th>\n",
       "      <th>tag_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt08-ar-285</td>\n",
       "      <td>Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>Response to treatment following modified Respo...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>1</td>\n",
       "      <td>parent_group</td>\n",
       "      <td>complete_response_response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response (CR) following modified Resp...</td>\n",
       "      <td>#009E73</td>\n",
       "      <td>1</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt27-ar-298</td>\n",
       "      <td>Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>Response to treatment following modified Respo...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>1</td>\n",
       "      <td>parent_group</td>\n",
       "      <td>complete_response_response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response (CR) following modified Resp...</td>\n",
       "      <td>#009E73</td>\n",
       "      <td>1</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt13-ar-289</td>\n",
       "      <td>Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>Response to treatment following modified Respo...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>1</td>\n",
       "      <td>parent_group</td>\n",
       "      <td>complete_response_response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response (CR) following modified Resp...</td>\n",
       "      <td>#009E73</td>\n",
       "      <td>1</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt09-ar-286</td>\n",
       "      <td>Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>Response to treatment following modified Respo...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>1</td>\n",
       "      <td>parent_group</td>\n",
       "      <td>complete_response_response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response</td>\n",
       "      <td>Complete Response (CR) following modified Resp...</td>\n",
       "      <td>#009E73</td>\n",
       "      <td>1</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HugoLo_IPRES_2016-Pt05-ar-282</td>\n",
       "      <td>Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>mRECIST Response</td>\n",
       "      <td>Response to treatment following modified Respo...</td>\n",
       "      <td>-2147483648</td>\n",
       "      <td>1</td>\n",
       "      <td>parent_group</td>\n",
       "      <td>partial_response_response</td>\n",
       "      <td>Partial Response</td>\n",
       "      <td>Partial Response</td>\n",
       "      <td>Partial Response (PR) following modified Respo...</td>\n",
       "      <td>#0072B2</td>\n",
       "      <td>2</td>\n",
       "      <td>group</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     sample_name parent_tag_name parent_tag_long_display  \\\n",
       "1  HugoLo_IPRES_2016-Pt08-ar-285        Response        mRECIST Response   \n",
       "2  HugoLo_IPRES_2016-Pt27-ar-298        Response        mRECIST Response   \n",
       "3  HugoLo_IPRES_2016-Pt13-ar-289        Response        mRECIST Response   \n",
       "4  HugoLo_IPRES_2016-Pt09-ar-286        Response        mRECIST Response   \n",
       "5  HugoLo_IPRES_2016-Pt05-ar-282        Response        mRECIST Response   \n",
       "\n",
       "  parent_tag_short_display                         parent_tag_characteristics  \\\n",
       "1         mRECIST Response  Response to treatment following modified Respo...   \n",
       "2         mRECIST Response  Response to treatment following modified Respo...   \n",
       "3         mRECIST Response  Response to treatment following modified Respo...   \n",
       "4         mRECIST Response  Response to treatment following modified Respo...   \n",
       "5         mRECIST Response  Response to treatment following modified Respo...   \n",
       "\n",
       "   parent_tag_color  parent_tag_order parent_tag_type  \\\n",
       "1       -2147483648                 1    parent_group   \n",
       "2       -2147483648                 1    parent_group   \n",
       "3       -2147483648                 1    parent_group   \n",
       "4       -2147483648                 1    parent_group   \n",
       "5       -2147483648                 1    parent_group   \n",
       "\n",
       "                     tag_name   tag_long_display  tag_short_display  \\\n",
       "1  complete_response_response  Complete Response  Complete Response   \n",
       "2  complete_response_response  Complete Response  Complete Response   \n",
       "3  complete_response_response  Complete Response  Complete Response   \n",
       "4  complete_response_response  Complete Response  Complete Response   \n",
       "5   partial_response_response   Partial Response   Partial Response   \n",
       "\n",
       "                                 tag_characteristics tag_color  tag_order  \\\n",
       "1  Complete Response (CR) following modified Resp...   #009E73          1   \n",
       "2  Complete Response (CR) following modified Resp...   #009E73          1   \n",
       "3  Complete Response (CR) following modified Resp...   #009E73          1   \n",
       "4  Complete Response (CR) following modified Resp...   #009E73          1   \n",
       "5  Partial Response (PR) following modified Respo...   #0072B2          2   \n",
       "\n",
       "  tag_type  \n",
       "1    group  \n",
       "2    group  \n",
       "3    group  \n",
       "4    group  \n",
       "5    group  "
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clinical_annotation.head()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}