readme_jupyter.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# I. Installing the environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#If using Windows:\n",
    "!conda env create -f environment_windows.yml\n",
    "!conda activate multiagent_env"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#If using Linux:\n",
    "!conda env create -f environment_linux.yml\n",
    "!conda activate multiagent_env"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, to set-up multi-agent environments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd simple_particle_envs\n",
    "%pip install -e . \n",
    "%cd .."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify the installation, jump to Section 4 for plot generation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python baselines/baselines.py --mode test --render "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# II. Train a model\n",
    "We advise not to train through Jupyter Notebook as the progression of the training can be followed on the terminal."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train a Fair-E model, run:\n",
    "The control parameter of fairness can be adjusted in _configs.py_.\n",
    "\n",
    "Additional flags :\n",
    "\n",
    "    -- collaborative => to train on a mutual reward set-up, do not put the flag for individual reward\n",
    "\n",
    "    -- equivariant ==> to enforce equivariance \n",
    "\n",
    "    -- nb_pred ==> number of predator\n",
    "\n",
    "    -- nb_prey ==> number of prey\n",
    "\n",
    "    -- checkpoint_path ==> resume training from a checkpoint, go to results/ddpg_speed_fair_simple_torus select your training folder and copy relative path as an argument for this flag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --env simple_torus --algorithm ddpg_symmetric"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train a Fair-ER model, run:\n",
    "The control parameter of fairness can be adjusted in _configs.py_.\n",
    "\n",
    "Additional flags :\n",
    "\n",
    "    -- collaborative => to train on a mutual reward set-up, do not put the flag for individual reward\n",
    "\n",
    "    -- lambda_coeff ==> adjust the lambda coefficient between 0 and 1 \n",
    "\n",
    "    -- nb_pred ==> number of predator\n",
    "\n",
    "    -- nb_prey ==> number of prey\n",
    "    \n",
    "    -- checkpoint_path ==> resume training from a checkpoint, go to results/ddpg_speed_fair_simple_torus select your training folder and copy relative path as an argument for this flag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --env simple_torus --algorithm ddpg_speed_fair --lambda_coeff 0.5 --nb_pred 4"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# III. Evaluation\n",
    "To evaluate the models that we have trained for our experiments, first copy the folder of models in the \\results directory.\n",
    "To collect trajectories from a trained model, run _eval/collect_actions.py_ or _eval/collect_actions_symmetric.py_. Here are a few examples:\n",
    "\n",
    "We could not provide all the models necessary to reproduce our plots. We have all the models but:\n",
    "    - ddpg_symmetric with collaborative and equivariant settings\n",
    "    - ddpg_symmetric with collaborative and no equivariant settings\n",
    "\n",
    "Here are the commands to train them:\n",
    "\n",
    "    - python main.py --env simple_torus --algorithm ddpg_symmetric --n_epochs 42000 --seed 72 --n_steps 167 --collaborative --equivariant\n",
    "\n",
    "\n",
    "    - python main.py --env simple_torus --algorithm ddpg_symmetric --n_epochs 42000 --seed 72 --n_steps 167 --collaborative "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To evaluate our Fair-E models, run:\n",
    "\n",
    "Flags : \n",
    "\n",
    "    -- checkpoint path ==> copy the relative path of the desired model from \"results/models_evaluation/symmetric/...\" and copy it to the argument\n",
    "\n",
    "    -- n_steps ==> our models are trained on 167 steps over one episode\n",
    "\n",
    "    -- nb_pred ==> specify the number of predators that the desired model was trained on, default to 3\n",
    "\n",
    "    -- nb_prey ==> specify the number of prey, default to 1\n",
    "\n",
    "    -- render to see the agents in action\n",
    "    \n",
    "    -- pred_vel ==> pursuer velocity on which the model is evaluated\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python eval/collect_actions_symmetric.py --env simple_torus --pred_policy ddpg --prey_policy cosine --seed 72 --checkpoint_path /path/to/model/checkpoints --n_steps 167 --nb_pred 3 --pred_vel 1.1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To evaluate our Fair-ER models, run:\n",
    "\n",
    "Flags : \n",
    "\n",
    "    -- checkpoint path ==> copy the relative path of the desired model from \"results/models_evaluation/speedfair/...\" and copy it to the argument\n",
    "\n",
    "    -- n_steps ==> our models are trained on 167 steps over one episode\n",
    "\n",
    "    -- nb_pred ==>specify the number of predators that the desired model was trained on, default to 3\n",
    "\n",
    "    -- nb_prey ==> specify the number of prey, default to 1\n",
    "\n",
    "    -- render to see the agents in action\n",
    "    \n",
    "    -- pred_vel ==> pursuer velocity on which the model is evaluated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python eval/collect_actions.py --env simple_torus --pred_policy ddpg --prey_policy cosine --seed 72 --checkpoint_path /path/to/model/checkpoints --n_steps 167 --nb_pred 3 --pred_vel 1.1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The evaluation will create a pickle file that can be used for plotting the figures with the make_plots.py function."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IV. Plotting\n",
    "If one wants to plot the figures that we used in our paper, follow the instructions: \n",
    "Copy the provided pickle files anywhere on the repository and put them in the fp argument.\n",
    "\n",
    "Some trajectory files will be lacking as it will be necessary to train and evaluate the models :\n",
    "    - ddpg_symmetric with collaborative and equivariant settings\n",
    "    - ddpg_symmetric with collaborative and no equivariant settings\n",
    "\n",
    "Once those trajectory files are obtained, they must be copied into the directory with the provided pickle files.\n",
    "\n",
    "Then, run this command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!python eval/make_plots.py --fp trajectories_for_plots"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the provided pickles, one can plot figure 1, change the fp path to test other folders, and the plot number to test other figures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python eval/make_plots.py --fp trajectories_for_plots --plot 1"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "multiagent_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "6d01bbfa2f49e176200142f0748b64685044a8fd22bf1fcb53137ceaa05dfda5"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}