diff --git a/README.md b/README.md index 18c9dfe5e..6f0d94a92 100644 --- a/README.md +++ b/README.md @@ -44,9 +44,8 @@ and closing the gap between simulation and the real world. Explore Brax easily and quickly through a series of colab notebooks: * [Brax Basics](https://colab.research.google.com/github/google/brax/blob/main/notebooks/basics.ipynb) introduces the Brax API, and shows how to simulate basic physics primitives. -* [Brax Training](https://colab.research.google.com/github/google/brax/blob/main/notebooks/training.ipynb) -introduces the Brax v2 API, and shows how to train a policy with the -generalized backend. +* [Brax Training](https://colab.research.google.com/github/google/brax/blob/main/notebooks/training.ipynb) introduces Brax's training algorithms, and lets you train your own policies directly within the colab. It also demonstrates loading and saving policies. +* [Brax Training with PyTorch on GPU](https://colab.research.google.com/github/google/brax/blob/main/notebooks/training_torch.ipynb) demonstrates how Brax can be used in other ML frameworks for fast training, in this case PyTorch. ## Using Brax Locally diff --git a/brax/training/agents/es/train.py b/brax/training/agents/es/train.py index ef7d4be21..3b9fbeb4a 100644 --- a/brax/training/agents/es/train.py +++ b/brax/training/agents/es/train.py @@ -200,7 +200,7 @@ def compute_delta( Returns: """ - # NOTE - -> len(weights) * perturbation_std" is + # NOTE: The trick "len(weights) -> len(weights) * perturbation_std" is # equivalent to tuning the l2_coef. weights = jnp.reshape(weights, ([population_size] + [1] * (noise.ndim - 1))) delta = jnp.sum(noise * weights, axis=0) / population_size diff --git a/brax/v1/experimental/composer/agent_utils.py b/brax/v1/experimental/composer/agent_utils.py index ebefc2713..2d4ba49bb 100644 --- a/brax/v1/experimental/composer/agent_utils.py +++ b/brax/v1/experimental/composer/agent_utils.py @@ -33,7 +33,7 @@ e.g. equivalent to agent1=(..., action_agents=('agent1',), ...) agent_groups currently defines which rewards/actions belong to which agent. -observation is the same among all agents (TODO -. +observation is the same among all agents (TODO: add optionality). """ from collections import OrderedDict as odict diff --git a/notebooks/training_torch.ipynb b/notebooks/training_torch.ipynb new file mode 100644 index 000000000..a7de3c4dd --- /dev/null +++ b/notebooks/training_torch.ipynb @@ -0,0 +1,556 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [], + "gpuType": "A100" + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "accelerator": "GPU", + "gpuClass": "standard" + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "trVNqxHmGISS" + }, + "source": [ + "# Training in Brax with PyTorch on GPUs\n", + "\n", + "Brax is ready to integrate into other research toolkits by way of the [OpenAI Gym](https://gym.openai.com/) interface. Brax environments convert to Gym environments using either [GymWrapper](https://github.com/google/brax/blob/main/brax/envs/wrappers/gym.py) for single environments, or [VectorGymWrapper](https://github.com/google/brax/blob/main/brax/envs/wrappers/gym.py) for batched (parallelized) environments." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "GJhPpM5ZPrpq" + }, + "source": [ + "#@title Import Brax and some helper modules\n", + "from IPython.display import clear_output\n", + "\n", + "import collections\n", + "from datetime import datetime\n", + "import functools\n", + "import math\n", + "import os\n", + "import time\n", + "from typing import Any, Callable, Dict, Optional, Sequence\n", + "\n", + "try:\n", + " import brax\n", + "except ImportError:\n", + " !pip install git+https://github.com/google/brax.git@main\n", + " clear_output()\n", + " import brax\n", + "\n", + "from brax import envs\n", + "from brax.envs.wrappers import gym as gym_wrapper\n", + "from brax.envs.wrappers import torch as torch_wrapper\n", + "from brax.io import metrics\n", + "from brax.training.agents.ppo import train as ppo\n", + "import gym\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import torch\n", + "from torch import nn\n", + "from torch import optim\n", + "import torch.nn.functional as F" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vQFCkfu8Qwre" + }, + "source": [ + "Here is a PPO Agent written in PyTorch:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "fWJE4b5BHeH7" + }, + "source": [ + "class Agent(nn.Module):\n", + " \"\"\"Standard PPO Agent with GAE and observation normalization.\"\"\"\n", + "\n", + " def __init__(self,\n", + " policy_layers: Sequence[int],\n", + " value_layers: Sequence[int],\n", + " entropy_cost: float,\n", + " discounting: float,\n", + " reward_scaling: float,\n", + " device: str):\n", + " super(Agent, self).__init__()\n", + "\n", + " policy = []\n", + " for w1, w2 in zip(policy_layers, policy_layers[1:]):\n", + " policy.append(nn.Linear(w1, w2))\n", + " policy.append(nn.SiLU())\n", + " policy.pop() # drop the final activation\n", + " self.policy = nn.Sequential(*policy)\n", + "\n", + " value = []\n", + " for w1, w2 in zip(value_layers, value_layers[1:]):\n", + " value.append(nn.Linear(w1, w2))\n", + " value.append(nn.SiLU())\n", + " value.pop() # drop the final activation\n", + " self.value = nn.Sequential(*value)\n", + "\n", + " self.num_steps = torch.zeros((), device=device)\n", + " self.running_mean = torch.zeros(policy_layers[0], device=device)\n", + " self.running_variance = torch.zeros(policy_layers[0], device=device)\n", + "\n", + " self.entropy_cost = entropy_cost\n", + " self.discounting = discounting\n", + " self.reward_scaling = reward_scaling\n", + " self.lambda_ = 0.95\n", + " self.epsilon = 0.3\n", + " self.device = device\n", + "\n", + " @torch.jit.export\n", + " def dist_create(self, logits):\n", + " \"\"\"Normal followed by tanh.\n", + "\n", + " torch.distribution doesn't work with torch.jit, so we roll our own.\"\"\"\n", + " loc, scale = torch.split(logits, logits.shape[-1] // 2, dim=-1)\n", + " scale = F.softplus(scale) + .001\n", + " return loc, scale\n", + "\n", + " @torch.jit.export\n", + " def dist_sample_no_postprocess(self, loc, scale):\n", + " return torch.normal(loc, scale)\n", + "\n", + " @classmethod\n", + " def dist_postprocess(cls, x):\n", + " return torch.tanh(x)\n", + "\n", + " @torch.jit.export\n", + " def dist_entropy(self, loc, scale):\n", + " log_normalized = 0.5 * math.log(2 * math.pi) + torch.log(scale)\n", + " entropy = 0.5 + log_normalized\n", + " entropy = entropy * torch.ones_like(loc)\n", + " dist = torch.normal(loc, scale)\n", + " log_det_jacobian = 2 * (math.log(2) - dist - F.softplus(-2 * dist))\n", + " entropy = entropy + log_det_jacobian\n", + " return entropy.sum(dim=-1)\n", + "\n", + " @torch.jit.export\n", + " def dist_log_prob(self, loc, scale, dist):\n", + " log_unnormalized = -0.5 * ((dist - loc) / scale).square()\n", + " log_normalized = 0.5 * math.log(2 * math.pi) + torch.log(scale)\n", + " log_det_jacobian = 2 * (math.log(2) - dist - F.softplus(-2 * dist))\n", + " log_prob = log_unnormalized - log_normalized - log_det_jacobian\n", + " return log_prob.sum(dim=-1)\n", + "\n", + " @torch.jit.export\n", + " def update_normalization(self, observation):\n", + " self.num_steps += observation.shape[0] * observation.shape[1]\n", + " input_to_old_mean = observation - self.running_mean\n", + " mean_diff = torch.sum(input_to_old_mean / self.num_steps, dim=(0, 1))\n", + " self.running_mean = self.running_mean + mean_diff\n", + " input_to_new_mean = observation - self.running_mean\n", + " var_diff = torch.sum(input_to_new_mean * input_to_old_mean, dim=(0, 1))\n", + " self.running_variance = self.running_variance + var_diff\n", + "\n", + " @torch.jit.export\n", + " def normalize(self, observation):\n", + " variance = self.running_variance / (self.num_steps + 1.0)\n", + " variance = torch.clip(variance, 1e-6, 1e6)\n", + " return ((observation - self.running_mean) / variance.sqrt()).clip(-5, 5)\n", + "\n", + " @torch.jit.export\n", + " def get_logits_action(self, observation):\n", + " observation = self.normalize(observation)\n", + " logits = self.policy(observation)\n", + " loc, scale = self.dist_create(logits)\n", + " action = self.dist_sample_no_postprocess(loc, scale)\n", + " return logits, action\n", + "\n", + " @torch.jit.export\n", + " def compute_gae(self, truncation, termination, reward, values,\n", + " bootstrap_value):\n", + " truncation_mask = 1 - truncation\n", + " # Append bootstrapped value to get [v1, ..., v_t+1]\n", + " values_t_plus_1 = torch.cat(\n", + " [values[1:], torch.unsqueeze(bootstrap_value, 0)], dim=0)\n", + " deltas = reward + self.discounting * (\n", + " 1 - termination) * values_t_plus_1 - values\n", + " deltas *= truncation_mask\n", + "\n", + " acc = torch.zeros_like(bootstrap_value)\n", + " vs_minus_v_xs = torch.zeros_like(truncation_mask)\n", + "\n", + " for ti in range(truncation_mask.shape[0]):\n", + " ti = truncation_mask.shape[0] - ti - 1\n", + " acc = deltas[ti] + self.discounting * (\n", + " 1 - termination[ti]) * truncation_mask[ti] * self.lambda_ * acc\n", + " vs_minus_v_xs[ti] = acc\n", + "\n", + " # Add V(x_s) to get v_s.\n", + " vs = vs_minus_v_xs + values\n", + " vs_t_plus_1 = torch.cat([vs[1:], torch.unsqueeze(bootstrap_value, 0)], 0)\n", + " advantages = (reward + self.discounting *\n", + " (1 - termination) * vs_t_plus_1 - values) * truncation_mask\n", + " return vs, advantages\n", + "\n", + " @torch.jit.export\n", + " def loss(self, td: Dict[str, torch.Tensor]):\n", + " observation = self.normalize(td['observation'])\n", + " policy_logits = self.policy(observation[:-1])\n", + " baseline = self.value(observation)\n", + " baseline = torch.squeeze(baseline, dim=-1)\n", + "\n", + " # Use last baseline value (from the value function) to bootstrap.\n", + " bootstrap_value = baseline[-1]\n", + " baseline = baseline[:-1]\n", + " reward = td['reward'] * self.reward_scaling\n", + " termination = td['done'] * (1 - td['truncation'])\n", + "\n", + " loc, scale = self.dist_create(td['logits'])\n", + " behaviour_action_log_probs = self.dist_log_prob(loc, scale, td['action'])\n", + " loc, scale = self.dist_create(policy_logits)\n", + " target_action_log_probs = self.dist_log_prob(loc, scale, td['action'])\n", + "\n", + " with torch.no_grad():\n", + " vs, advantages = self.compute_gae(\n", + " truncation=td['truncation'],\n", + " termination=termination,\n", + " reward=reward,\n", + " values=baseline,\n", + " bootstrap_value=bootstrap_value)\n", + "\n", + " rho_s = torch.exp(target_action_log_probs - behaviour_action_log_probs)\n", + " surrogate_loss1 = rho_s * advantages\n", + " surrogate_loss2 = rho_s.clip(1 - self.epsilon,\n", + " 1 + self.epsilon) * advantages\n", + " policy_loss = -torch.mean(torch.minimum(surrogate_loss1, surrogate_loss2))\n", + "\n", + " # Value function loss\n", + " v_error = vs - baseline\n", + " v_loss = torch.mean(v_error * v_error) * 0.5 * 0.5\n", + "\n", + " # Entropy reward\n", + " entropy = torch.mean(self.dist_entropy(loc, scale))\n", + " entropy_loss = self.entropy_cost * -entropy\n", + "\n", + " return policy_loss + v_loss + entropy_loss" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CWbuk7IAR0SU" + }, + "source": [ + "Finally, some code for unrolling and batching environment data:" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "D3y5o7-oSBm-" + }, + "source": [ + "StepData = collections.namedtuple(\n", + " 'StepData',\n", + " ('observation', 'logits', 'action', 'reward', 'done', 'truncation'))\n", + "\n", + "\n", + "def sd_map(f: Callable[..., torch.Tensor], *sds) -> StepData:\n", + " \"\"\"Map a function over each field in StepData.\"\"\"\n", + " items = {}\n", + " keys = sds[0]._asdict().keys()\n", + " for k in keys:\n", + " items[k] = f(*[sd._asdict()[k] for sd in sds])\n", + " return StepData(**items)\n", + "\n", + "\n", + "def eval_unroll(agent, env, length):\n", + " \"\"\"Return number of episodes and average reward for a single unroll.\"\"\"\n", + " observation = env.reset()\n", + " episodes = torch.zeros((), device=agent.device)\n", + " episode_reward = torch.zeros((), device=agent.device)\n", + " for _ in range(length):\n", + " _, action = agent.get_logits_action(observation)\n", + " observation, reward, done, _ = env.step(Agent.dist_postprocess(action))\n", + " episodes += torch.sum(done)\n", + " episode_reward += torch.sum(reward)\n", + " return episodes, episode_reward / episodes\n", + "\n", + "\n", + "def train_unroll(agent, env, observation, num_unrolls, unroll_length):\n", + " \"\"\"Return step data over multple unrolls.\"\"\"\n", + " sd = StepData([], [], [], [], [], [])\n", + " for _ in range(num_unrolls):\n", + " one_unroll = StepData([observation], [], [], [], [], [])\n", + " for _ in range(unroll_length):\n", + " logits, action = agent.get_logits_action(observation)\n", + " observation, reward, done, info = env.step(Agent.dist_postprocess(action))\n", + " one_unroll.observation.append(observation)\n", + " one_unroll.logits.append(logits)\n", + " one_unroll.action.append(action)\n", + " one_unroll.reward.append(reward)\n", + " one_unroll.done.append(done)\n", + " one_unroll.truncation.append(info['truncation'])\n", + " one_unroll = sd_map(torch.stack, one_unroll)\n", + " sd = sd_map(lambda x, y: x + [y], sd, one_unroll)\n", + " td = sd_map(torch.stack, sd)\n", + " return observation, td\n", + "\n", + "\n", + "def train(\n", + " env_name: str = 'ant',\n", + " num_envs: int = 2048,\n", + " episode_length: int = 1000,\n", + " device: str = 'cuda',\n", + " num_timesteps: int = 30_000_000,\n", + " eval_frequency: int = 10,\n", + " unroll_length: int = 5,\n", + " batch_size: int = 1024,\n", + " num_minibatches: int = 32,\n", + " num_update_epochs: int = 4,\n", + " reward_scaling: float = .1,\n", + " entropy_cost: float = 1e-2,\n", + " discounting: float = .97,\n", + " learning_rate: float = 3e-4,\n", + " progress_fn: Optional[Callable[[int, Dict[str, Any]], None]] = None,\n", + "):\n", + " \"\"\"Trains a policy via PPO.\"\"\"\n", + " env = envs.create(env_name, batch_size=num_envs,\n", + " episode_length=episode_length,\n", + " backend='spring')\n", + " env = gym_wrapper.VectorGymWrapper(env)\n", + " # automatically convert between jax ndarrays and torch tensors:\n", + " env = torch_wrapper.TorchWrapper(env, device=device)\n", + "\n", + " # env warmup\n", + " env.reset()\n", + " action = torch.zeros(env.action_space.shape).to(device)\n", + " env.step(action)\n", + "\n", + " # create the agent\n", + " policy_layers = [\n", + " env.observation_space.shape[-1], 64, 64, env.action_space.shape[-1] * 2\n", + " ]\n", + " value_layers = [env.observation_space.shape[-1], 64, 64, 1]\n", + " agent = Agent(policy_layers, value_layers, entropy_cost, discounting,\n", + " reward_scaling, device)\n", + " agent = torch.jit.script(agent.to(device))\n", + " optimizer = optim.Adam(agent.parameters(), lr=learning_rate)\n", + "\n", + " sps = 0\n", + " total_steps = 0\n", + " total_loss = 0\n", + " for eval_i in range(eval_frequency + 1):\n", + " if progress_fn:\n", + " t = time.time()\n", + " with torch.no_grad():\n", + " episode_count, episode_reward = eval_unroll(agent, env, episode_length)\n", + " duration = time.time() - t\n", + " # TODO: only count stats from completed episodes\n", + " episode_avg_length = env.num_envs * episode_length / episode_count\n", + " eval_sps = env.num_envs * episode_length / duration\n", + " progress = {\n", + " 'eval/episode_reward': episode_reward,\n", + " 'eval/completed_episodes': episode_count,\n", + " 'eval/avg_episode_length': episode_avg_length,\n", + " 'speed/sps': sps,\n", + " 'speed/eval_sps': eval_sps,\n", + " 'losses/total_loss': total_loss,\n", + " }\n", + " progress_fn(total_steps, progress)\n", + "\n", + " if eval_i == eval_frequency:\n", + " break\n", + "\n", + " observation = env.reset()\n", + " num_steps = batch_size * num_minibatches * unroll_length\n", + " num_epochs = num_timesteps // (num_steps * eval_frequency)\n", + " num_unrolls = batch_size * num_minibatches // env.num_envs\n", + " total_loss = 0\n", + " t = time.time()\n", + " for _ in range(num_epochs):\n", + " observation, td = train_unroll(agent, env, observation, num_unrolls,\n", + " unroll_length)\n", + "\n", + " # make unroll first\n", + " def unroll_first(data):\n", + " data = data.swapaxes(0, 1)\n", + " return data.reshape([data.shape[0], -1] + list(data.shape[3:]))\n", + " td = sd_map(unroll_first, td)\n", + "\n", + " # update normalization statistics\n", + " agent.update_normalization(td.observation)\n", + "\n", + " for _ in range(num_update_epochs):\n", + " # shuffle and batch the data\n", + " with torch.no_grad():\n", + " permutation = torch.randperm(td.observation.shape[1], device=device)\n", + " def shuffle_batch(data):\n", + " data = data[:, permutation]\n", + " data = data.reshape([data.shape[0], num_minibatches, -1] +\n", + " list(data.shape[2:]))\n", + " return data.swapaxes(0, 1)\n", + " epoch_td = sd_map(shuffle_batch, td)\n", + "\n", + " for minibatch_i in range(num_minibatches):\n", + " td_minibatch = sd_map(lambda d: d[minibatch_i], epoch_td)\n", + " loss = agent.loss(td_minibatch._asdict())\n", + " optimizer.zero_grad()\n", + " loss.backward()\n", + " optimizer.step()\n", + " total_loss += loss\n", + "\n", + " duration = time.time() - t\n", + " total_steps += num_epochs * num_steps\n", + " total_loss = total_loss / (num_epochs * num_update_epochs * num_minibatches)\n", + " sps = num_epochs * num_steps / duration" + ], + "execution_count": 3, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "R2A9MMlHUajH" + }, + "source": [ + "Let's go!" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "B-lrKHvkUeYM", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 524 + }, + "outputId": "b410554d-2de5-4fa1-9df3-df6f0ba5c980" + }, + "source": [ + "# temporary fix to cuda memory OOM\n", + "os.environ['XLA_PYTHON_CLIENT_ALLOCATOR'] = 'platform'\n", + "\n", + "xdata = []\n", + "ydata = []\n", + "eval_sps = []\n", + "train_sps = []\n", + "times = [datetime.now()]\n", + "\n", + "def progress(num_steps, metrics):\n", + " times.append(datetime.now())\n", + " xdata.append(num_steps)\n", + " ydata.append(metrics['eval/episode_reward'].cpu())\n", + " eval_sps.append(metrics['speed/eval_sps'])\n", + " train_sps.append(metrics['speed/sps'])\n", + " clear_output(wait=True)\n", + " plt.xlim([0, 30_000_000])\n", + " plt.ylim([0, 6000])\n", + " plt.xlabel('# environment steps')\n", + " plt.ylabel('reward per episode')\n", + " plt.plot(xdata, ydata)\n", + " plt.show()\n", + "\n", + "train(progress_fn=progress)\n", + "\n", + "print(f'time to jit: {times[1] - times[0]}')\n", + "print(f'time to train: {times[-1] - times[1]}')\n", + "print(f'eval steps/sec: {np.mean(eval_sps)}')\n", + "print(f'train steps/sec: {np.mean(train_sps)}')" + ], + "execution_count": 4, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk8AAAG2CAYAAABmsmIiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABb40lEQVR4nO3dd3wUdf7H8dembEI6LQ1C6CXSQSBIFSQoetJUEAERsIFSRJFTQWwodxbOAiKn2BBFhB9SRSDU0EIPEFroJKGYbBLSd35/xOwZAWUhYVPez8djH5ed+c7MZ+aW3bcz3/mOyTAMAxERERG5Lk6OLkBERESkJFF4EhEREbGDwpOIiIiIHRSeREREROyg8CQiIiJiB4UnERERETsoPImIiIjYQeFJRERExA4KTyIiIiJ2UHgSERERsYPDw9OZM2d45JFHqFixIuXKlaNRo0Zs377dNt8wDCZOnEhQUBDlypWja9euHD58uMA6Ll26xIABA/Dx8cHPz4+hQ4eSmppaoM2ePXto37497u7uhISEMHXq1FuyfyIiIlK6ODQ8/fbbb9xxxx24urqybNky9u/fz7vvvkv58uVtbaZOncp//vMfZsyYwZYtW/D09CQiIoKMjAxbmwEDBhATE8PKlStZvHgx69at4/HHH7fNt1gsdOvWjdDQUKKjo/nXv/7Fq6++ysyZM2/p/oqIiEjJZ3Lkg4FffPFFNm7cyPr166863zAMgoODee655xg3bhwAycnJBAQEMHv2bPr168eBAwcICwtj27ZttGzZEoDly5dzzz33cPr0aYKDg5k+fTovvfQS8fHxmM1m27YXLlzIwYMHb83OioiISKng4siNL1q0iIiICB544AHWrl1LlSpVePrppxk+fDgAcXFxxMfH07VrV9syvr6+tG7dmqioKPr160dUVBR+fn624ATQtWtXnJyc2LJlC7169SIqKooOHTrYghNAREQE77zzDr/99luBM10AmZmZZGZm2t5brVYuXbpExYoVMZlMRXU4REREpBAZhkFKSgrBwcE4ORXexTaHhqdjx44xffp0xo4dyz//+U+2bdvGs88+i9lsZvDgwcTHxwMQEBBQYLmAgADbvPj4ePz9/QvMd3FxoUKFCgXa1KhR44p15M/7c3iaMmUKkydPLrwdFREREYc5deoUVatWLbT1OTQ8Wa1WWrZsyVtvvQVAs2bN2LdvHzNmzGDw4MEOq2vChAmMHTvW9j45OZlq1apx6tQpfHx8HFaXiIiIXD+LxUJISAje3t6Ful6HhqegoCDCwsIKTGvQoAHz588HIDAwEICEhASCgoJsbRISEmjatKmtTWJiYoF15OTkcOnSJdvygYGBJCQkFGiT/z6/zR+5ubnh5uZ2xXQfHx+FJxERkRKmsLvcOPRuuzvuuIPY2NgC0w4dOkRoaCgANWrUIDAwkFWrVtnmWywWtmzZQnh4OADh4eEkJSURHR1ta7N69WqsViutW7e2tVm3bh3Z2dm2NitXrqRevXpXXLITERER+SsODU9jxoxh8+bNvPXWWxw5coQ5c+Ywc+ZMRowYAeQlxdGjR/PGG2+waNEi9u7dy6BBgwgODqZnz55A3pmq7t27M3z4cLZu3crGjRsZOXIk/fr1Izg4GICHH34Ys9nM0KFDiYmJ4fvvv2fatGkFLs2JiIiIXBfDwX7++WejYcOGhpubm1G/fn1j5syZBeZbrVbjlVdeMQICAgw3NzejS5cuRmxsbIE2Fy9eNPr37294eXkZPj4+xpAhQ4yUlJQCbXbv3m20a9fOcHNzM6pUqWK8/fbb111jcnKyARjJyck3vqMiIiJySxXV77dDx3kqKSwWC76+viQnJ6vPk4iISAlRVL/fDn88i4iIiEhJovAkIiIiYgeFJxERERE7KDyJiIiI2EHhSURERMQOCk8iIiIidlB4EhEREbGDwpOIiIiIHRSeREREROyg8CQiIiJiB4UnERERETsoPImIiIjYQeFJRERExA4KTyIiIiJ2UHgSERERsYPCk4iIiIgdFJ5ERERE7KDwJCIiImIHhScREREROyg8iYiIiNhB4UlERETEDgpPIiIiInZQeBIRERGxg8KTiIiIiB0UnkRERETsoPAkIiIiYgeFJxERERE7KDyJiIiI2EHhSURERMQOCk8iIiIidlB4EhEREbGDwpOIiIiIHRSeREREROyg8CQiIiJiB4UnERERETsoPImIiIjYQeFJRERExA4KTyIiIiJ2UHgSERERsYPCk4iIiIgdFJ5ERERE7KDwJCIiImIHhScREREROyg8iYiIiNhB4UlERETEDgpPIiIiInZQeBIRERGxg8KTiIiIiB0UnkRERETsoPAkIiIiYgeFJxERERE7ODQ8vfrqq5hMpgKv+vXr2+ZnZGQwYsQIKlasiJeXF3369CEhIaHAOk6ePEmPHj3w8PDA39+f559/npycnAJtIiMjad68OW5ubtSuXZvZs2ffit0TERGRUsjhZ55uu+02zp07Z3tt2LDBNm/MmDH8/PPPzJs3j7Vr13L27Fl69+5tm5+bm0uPHj3Iyspi06ZNfPnll8yePZuJEyfa2sTFxdGjRw86d+7Mrl27GD16NMOGDWPFihW3dD9FRESkdDAZhmE4auOvvvoqCxcuZNeuXVfMS05OpnLlysyZM4e+ffsCcPDgQRo0aEBUVBRt2rRh2bJl3HvvvZw9e5aAgAAAZsyYwfjx4zl//jxms5nx48ezZMkS9u3bZ1t3v379SEpKYvny5ddVp8ViwdfXl+TkZHx8fG5+x0VERKTIFdXvt8PPPB0+fJjg4GBq1qzJgAEDOHnyJADR0dFkZ2fTtWtXW9v69etTrVo1oqKiAIiKiqJRo0a24AQQERGBxWIhJibG1uaP68hvk7+Oq8nMzMRisRR4iYiIiICDw1Pr1q2ZPXs2y5cvZ/r06cTFxdG+fXtSUlKIj4/HbDbj5+dXYJmAgADi4+MBiI+PLxCc8ufnz/urNhaLhfT09KvWNWXKFHx9fW2vkJCQwthdERERKQVcHLnxu+++2/Z348aNad26NaGhofzwww+UK1fOYXVNmDCBsWPH2t5bLBYFKBEREQGKwWW7P/Lz86Nu3bocOXKEwMBAsrKySEpKKtAmISGBwMBAAAIDA6+4+y7//d+18fHxuWZAc3Nzw8fHp8BLREREBIpZeEpNTeXo0aMEBQXRokULXF1dWbVqlW1+bGwsJ0+eJDw8HIDw8HD27t1LYmKirc3KlSvx8fEhLCzM1uaP68hvk78OEREREXs4NDyNGzeOtWvXcvz4cTZt2kSvXr1wdnamf//++Pr6MnToUMaOHcuaNWuIjo5myJAhhIeH06ZNGwC6detGWFgYAwcOZPfu3axYsYKXX36ZESNG4ObmBsCTTz7JsWPHeOGFFzh48CCffPIJP/zwA2PGjHHkrouIiEgJ5dA+T6dPn6Z///5cvHiRypUr065dOzZv3kzlypUBeP/993FycqJPnz5kZmYSERHBJ598Ylve2dmZxYsX89RTTxEeHo6npyeDBw/mtddes7WpUaMGS5YsYcyYMUybNo2qVasya9YsIiIibvn+ioiISMnn0HGeSgqN8yQiIlLylNpxnkRERERKEoUnERERETsoPImIiIjYQeFJRERExA4KTyIiIiJ2UHgSERERsYPCk4iIiIgdFJ5ERERE7KDwJCIiImIHhScREREROyg8iYiIiNhB4UlERETEDgpPIiIiInZQeBIRERGxg8KTiIiIiB0UnkRERETsoPAkIiIiYgeFJxERERE7KDyJiIiI2EHhSURERMQOCk8iIiIidlB4EhEREbGDwpOIiIiIHRSeREREROyg8CQiIiJiB4UnERERETsoPImIiIjYQeFJRERExA4KTyIiIiJ2UHgSERERsYPCk4iIiIgdFJ5ERESkREpOz2bHyd9u+XZdbvkWRURERG6CJSObLzYc578bjmF2cWb9C50pZ3a+ZdtXeBIREZESwZKRzeyNx5m1/hiWjBwA6vi7czY5nVqVvW5ZHQpPIiIiUqyl5IemDXEkp2cDUNvfi1Fd6tCjURBOTqZbWo/Ck4iIiBRLKRnZfLnpOJ+tLxianv09NDnf4tCUT+FJREREipXUzJzfQ9Mxki7nhaZalT15tksd7m0c7LDQlE/hSURERIqFq4WmmpU9GVVMQlM+hScRERFxqNTMHL6KOs5n647xW35oqpR3pum+JsUnNOVTeBIRERGHSMvM4auoE8xcd7REhKZ8Ck8iIiJyS6Vl5vD15hPMXHeMS2lZANSo5MmzXWpzX+NgXJyL9xjeCk8iIiJyS1zOyuHrqBN8+ofQVL2iB892qcM/mhT/0JRP4UlERESK1OWsHL7ZfIJP1x7j4h9C0zN31uH+piUnNOVTeBIREZEicTkrh283n+TTdUe5kJoXmkJ/D009S2BoyqfwJCIiIoUqPSs370zTH0JTtQoePHNnbXo1q1JiQ1M+hScREREpFOlZuXy75QQz1v4vNIVUKMczd9ahV7MquJbw0JRP4UlERERuSkZ23pmmGWuPcSE1E/g9NHWuQ6/mpSc05VN4EhERkRuSkZ3Lt1tOMmPtUc6n5IWmquXL8cydtendvGqpC035FJ5ERETELhnZuczZcpLpfwhNVfz+F5rMLqUzNOVTeBIREZHrkpGdy3dbTzI98iiJfwhNI++sTZ8yEJryFZu9fPvttzGZTIwePdo2LSMjgxEjRlCxYkW8vLzo06cPCQkJBZY7efIkPXr0wMPDA39/f55//nlycnIKtImMjKR58+a4ublRu3ZtZs+efQv2SEREpHTIyM5l9sY4Ov5rDZN/3k9iSiZV/MrxVq9GrBnXif6tqpWZ4AQ3GJ7Wr1/PI488Qnh4OGfOnAHg66+/ZsOGDTdUxLZt2/j0009p3Lhxgeljxozh559/Zt68eaxdu5azZ8/Su3dv2/zc3Fx69OhBVlYWmzZt4ssvv2T27NlMnDjR1iYuLo4ePXrQuXNndu3axejRoxk2bBgrVqy4oVpFRETKiozsXL7cdJyO/1rDqz/vJ8GSSbCvO2/2asiacZ14uHXZCk02hp1+/PFHo1y5csawYcMMNzc34+jRo4ZhGMaHH35o3H333fauzkhJSTHq1KljrFy50ujYsaMxatQowzAMIykpyXB1dTXmzZtna3vgwAEDMKKiogzDMIylS5caTk5ORnx8vK3N9OnTDR8fHyMzM9MwDMN44YUXjNtuu63ANh966CEjIiLiumtMTk42ACM5Odnu/RMRESlpMrJzjC83xRmt3/zVCB2/2Agdv9ho89avxtdRx42M7BxHl3fdiur32+64+MYbbzBjxgw+++wzXF1dbdPvuOMOduzYYXd4GzFiBD169KBr164FpkdHR5OdnV1gev369alWrRpRUVEAREVF0ahRIwICAmxtIiIisFgsxMTE2Nr8ed0RERG2dVxNZmYmFoulwEtERKS0y8zJ5euo43T6VyQT/y+GeEsGQb7uvN6zIZHPd+KRNqG4uTg7ukyHs7vDeGxsLB06dLhiuq+vL0lJSXata+7cuezYsYNt27ZdMS8+Ph6z2Yyfn1+B6QEBAcTHx9va/DE45c/Pn/dXbSwWC+np6ZQrV+6KbU+ZMoXJkyfbtS8iIiIlVWZOLj9sP80na45wLjkDgEAfd0Z0rsWDt4coMP2J3eEpMDCQI0eOUL169QLTN2zYQM2aNa97PadOnWLUqFGsXLkSd3d3e8soUhMmTGDs2LG29xaLhZCQEAdWJCIiUviycqz8sP0Un6w5wtnfQ1OAjxsjOtfmwZYhuLsqNF2N3eFp+PDhjBo1is8//xyTycTZs2eJiopi3LhxvPLKK9e9nujoaBITE2nevLltWm5uLuvWreOjjz5ixYoVZGVlkZSUVODsU0JCAoGBgUBekNu6dWuB9ebfjffHNn++Qy8hIQEfH5+rnnUCcHNzw83N7br3RUREpCTJtRrMjz7NtFWHOZOUDuSFpqc71eah2xWa/o7d4enFF1/EarXSpUsXLl++TIcOHXBzc2PcuHE888wz172eLl26sHfv3gLThgwZQv369Rk/fjwhISG4urqyatUq+vTpA+RdMjx58iTh4eEAhIeH8+abb5KYmIi/vz8AK1euxMfHh7CwMFubpUuXFtjOypUrbesQEREpSzYducDrSw5w4Fxef15/bzee7lSLfq2qKTRdJ5NhGMaNLJiVlcWRI0dITU0lLCwMLy+vmy6mU6dONG3alA8++ACAp556iqVLlzJ79mx8fHxs4WzTpk1A3pmqpk2bEhwczNSpU4mPj2fgwIEMGzaMt956C8gbqqBhw4aMGDGCxx57jNWrV/Pss8+yZMkSIiIirqsui8WCr68vycnJ+Pj43PR+ioiI3GrHzqfy1tID/HogEQBvdxeevbMOA8NDS21oKqrf7xseYdxsNtvO7hSV999/HycnJ/r06UNmZiYRERF88skntvnOzs4sXryYp556ivDwcDw9PRk8eDCvvfaarU2NGjVYsmQJY8aMYdq0aVStWpVZs2Zdd3ASEREpyZIuZzFt1WG+jjpBjtXA2cnEI62rMaprXSp4mh1dXol0XWee/jgw5d/56aefbqqg4khnnkREpKTJyrHyzeYTTFt1mOT0bADurO/PP+9pQG3/m79aVBI49MyTr6+v7W/DMFiwYAG+vr60bNkSyOv8nZSUZFfIEhERkcJnGAa/HkjkraUHiLuQBkD9QG9e6tGA9nUqO7i60uG6wtMXX3xh+3v8+PE8+OCDzJgxA2fnvGukubm5PP300zorIyIi4kAxZ5N5c8kBNh29CEAlLzPPdavHgy1DcHYyObi60sPuDuOVK1dmw4YN1KtXr8D02NhY2rZty8WLFwu1wOJAl+1ERKQ4S7Rk8O9fYpkXfRrDALOLE8Pa1eCpTrXwdnf9+xWUUsWmw3hOTg4HDx68IjwdPHgQq9VaaIWJiIjIX8vIzuWzdceYvvYol7NyAbi3cRDju9cnpIKHg6srvewOT0OGDGHo0KEcPXqUVq1aAbBlyxbefvtthgwZUugFioiISEFWq8Gi3WeZuvygbWTwpiF+vHJvGC1Cyzu4utLP7vD073//m8DAQN59913OnTsHQFBQEM8//zzPPfdcoRcoIiIi/7P9+CVeX3KA3aeSAAj2dWf83fX5R5NgTCb1a7oVbniQTMi7lgiU+n5A6vMkIiKOdurSZd5efpAle/JOXHianXm6c22GtqtRage5vFnFps9TvvPnzxMbGwtA/fr1qVSpUqEVJSIiInlSMrL5eM1RPt8YR1aOFZMJHmoZwthudfH3dnd0eWWS3eEpLS2NZ555hq+++srWQdzZ2ZlBgwbx4Ycf4uGhDmoiIiI3KyfXyvfbT/HeL4e4mJYFQNtaFXm5RxhhwboK4khO9i4wduxY1q5dy88//0xSUhJJSUn83//9H2vXrlWfJxERkUKw7tB5evxnAy8t2MfFtCxqVvJk1qCWfDustYJTMWB3n6dKlSrx448/0qlTpwLT16xZw4MPPsj58+cLs75iQX2eRETkVjiSmMKbSw6wJjbvt9S3nCuju9bhkTahuDrbfb6jzCs2fZ4uX75MQEDAFdP9/f25fPlyoRQlIiJSllxKy+KDXw/x7ZaT5FoNXJxMDAqvzrNdauPnoYf3Fjd2h6fw8HAmTZrEV199hbt7Xke19PR0Jk+eTHh4eKEXKCIiUlpl5uTy1aYT/Gf1YVIycgC4KyyACXfXp2blsvHw3pLI7vA0bdo0IiIiqFq1Kk2aNAFg9+7duLu7s2LFikIvUEREpLQxDIMVMfFMWXaQExfzrtqEBfnwco8GtK2tu9eLuxsa5+ny5ct8++23HDx4EIAGDRowYMAAypUrV+gFFgfq8yQiIoVl7+lkXl+yn61xlwCo7O3G893q0adFVT28t5AVmz5PAB4eHgwfPrzQihARESnt4pMzmLriID/tOAOAm4sTT3SoyRMda+HpdsPDLooD2N11/8svv2TJkiW29y+88AJ+fn60bduWEydOFGpxIiIiJd3lrBzeX3mIzv+OtAWnnk2DWTOuE2O71VNwKoHsDk9vvfWW7fJcVFQUH330EVOnTqVSpUqMGTOm0AsUEREpiaxWgx+jT9P535FMW3WY9OxcWoaWZ+GIO/igXzOC/UpnV5eywO64e+rUKWrXrg3AwoUL6du3L48//jh33HHHFWM/iYiIlEVbjl3kjSUH2HsmGYCq5csx4e4G3NMoUA/vLQXsDk9eXl5cvHiRatWq8csvvzB27FgA3N3dSU9PL/QCRURESooTF9OYsvQgy2PiAfByc2HknbV5tG11Pby3FLE7PN11110MGzaMZs2acejQIe655x4AYmJiqF69emHXJyIiUuwlp2fz0erDzN50nOxcAycT9G9VjTF31aWSl5ujy5NCZnd4+vjjj3n55Zc5deoU8+fPp2LFigBER0fTv3//Qi9QRESkuMrJtTJn60neX3mI3y5nA9C+TiVe7hFGvUBvB1cnReWGxnkqazTOk4iI/JFhGETGnufNpQc4kpgKQG1/L17q0YDO9fwdXJ3kc+g4T3v27KFhw4Y4OTmxZ8+ev2zbuHHjQilMRESkuMnOtbJsXzz/3RDH7lNJAJT3cGXsXXXp36oaLnp4b5lwXeGpadOmxMfH4+/vT9OmTTGZTPzxhFX+e5PJRG5ubpEVKyIi4gjJ6dnM3XqSLzcd52xyBgBmZycevaM6IzrXxrecq4MrlFvpusJTXFwclStXtv0tIiJSFpy4mMYXG4/zw/ZTXM7KOzlQ0dPMI21CeaRNKJW91Rm8LLqu8BQaGnrVv0VEREobwzDYGneJ/26IY+WBBPIvtNQN8GJouxrc37SKhh0o425oTPjY2Fg+/PBDDhw4AOQ9GPiZZ56hXr16hVqciIjIrZKda2XJnnP8d0OcbXBLgI51KzOsfQ3a1a6kAS4FuIHwNH/+fPr160fLli0JDw8HYPPmzTRs2JC5c+fSp0+fQi9SRESkqCRdzmLO1pN8tekE8Za8/kxuLk70bl6Vx+6oTp0ADTkgBdk9VEGtWrUYMGAAr732WoHpkyZN4ptvvuHo0aOFWmBxoKEKRERKn2PnU/li43F+jD5NenZef6ZKXm4MDg/l4dbVqKjBLUu8ovr9tjs8eXh4sGfPHtvz7fIdPnyYJk2acPny5UIrrrhQeBIRKR0MwyDq2EU+3xDHqoOJtv5MDYJ8GNquBvc1CcLNRf2ZSguHjvP0R506dWL9+vVXhKcNGzbQvn37QitMRESksGTlWPl591n+uyGO/ecstuld6vsztF0NwmtVVH8muW52h6d//OMfjB8/nujoaNq0aQPk9XmaN28ekydPZtGiRQXaioiIOMqltCzmbDnBl1EnOJ+SCYC7qxN9W1RlyB01qFXZy8EVSklk92U7J6frGz21NA2Yqct2IiIly5HEFP674Tg/7ThNZo4VgAAfNwaFV+fhVtUo72l2cIVyKxSby3ZWq7XQNi4iIlJYDMNgw5EL/HdDHJGx523TG1bxYVi7mtzTKAizix6fIjfvhsZ5ypeRkYG7u3th1SIiImK3jOxcFu06y+cb4zgYnwKAyQRdGwQwrF0NWtWooP5MUqjsDk+5ubm89dZbzJgxg4SEBA4dOkTNmjV55ZVXqF69OkOHDi2KOkVERAq4kJrJN5tP8M3mE1xIzQLAw+zMA7/3Z6peydPBFUppZXd4evPNN/nyyy+ZOnUqw4cPt01v2LAhH3zwgcKTiIgUqdj4FD7fEMeCXWfI+r0/U5CvO4PbVqf/7dXw9dBDeqVo2R2evvrqK2bOnEmXLl148sknbdObNGnCwYMHC7U4ERERyOvPtPbQef67IY71hy/Ypjep6svQ9jW5u2Egrs7qzyS3ht3h6cyZM1eM8QR5Hcmzs7MLpSgRERHI68+0YOcZ/rshjiOJqQA4maBbWCDD2tegRWh59WeSW87u8BQWFsb69esJDQ0tMP3HH3+kWbNmhVaYiIiUXYkpGXwddYJvt5zkUlpefyZPszMP3V6NIXdUJ6SCh4MrlLLM7vA0ceJEBg8ezJkzZ7Barfz000/Exsby1VdfsXjx4qKoUUREyoj9Zy38d0McP+8+S1ZuXn+mKn7lGHJHdR68PQQfd/VnEseze5BMgPXr1/Paa6+xe/duUlNTad68ORMnTqRbt25FUaPDaZBMEZGiY7UarIlN5L8b4th09KJtevNqfgxtV5OI2wJwUX8muQHF5sHAZZHCk4hI4buclcP8HWf4YkMcxy6kAeDsZKJ7w0CGtqtB82rlHVyhlHTFZoRxERGRm3HsfCrzok/z3daTJF3Ou9HI282Ffq1CGNy2OlXLqz+TFG8KTyIiUuQupGayePdZFuw6y+5TSbbpIRXKMaRtDR68PQQvN/0kScmgT6qIiBSJ9KxcVh5IYOHOM6w9dJ5ca14vEWcnE+3rVKLf7SHcFRaIs5OGGpCSReFJREQKTa7VYPOxiyzYeYbl++JJzcyxzWtc1ZeeTatwX5NgKnu7ObBKkZtjV3jKzs6mfv36LF68mAYNGhRVTSIiUsIcOGdh4c4z/N+us8RbMmzTq/iVo1ezKvRsVoXa/l4OrFCk8NgVnlxdXcnIyPj7hiIiUuqdS05n0a6zLNh5hoPxKbbpPu4u9GgcTO/mVWhRrTxOuiwnpYzdl+1GjBjBO++8w6xZs3Bx0VU/EZGyJCUjm+X74lm46wybjl4kf7AbV2cTd9b3p1ezqnSuXxk3F2fHFipShOwedWzbtm389NNPVKtWjYiICHr37l3gZY/p06fTuHFjfHx88PHxITw8nGXLltnmZ2RkMGLECCpWrIiXlxd9+vQhISGhwDpOnjxJjx498PDwwN/fn+eff56cnJwCbSIjI2nevDlubm7Url2b2bNn27vbIiJlVnauldUHE3jmu53c/uavPP/jHjYeyQtOt1cvz1u9GrHtpa58OrAl3RsGKjhJqWf3qSM/Pz/69OlTKBuvWrUqb7/9NnXq1MEwDL788kvuv/9+du7cyW233caYMWNYsmQJ8+bNw9fXl5EjR9K7d282btwIQG5uLj169CAwMJBNmzZx7tw5Bg0ahKurK2+99RYAcXFx9OjRgyeffJJvv/2WVatWMWzYMIKCgoiIiCiU/RARKW0Mw2D36WQW7jzDz7vPcvH358sB1KzsSe9mVbi/aRU9Y07KpGI3wniFChX417/+Rd++falcuTJz5syhb9++ABw8eJAGDRoQFRVFmzZtWLZsGffeey9nz54lICAAgBkzZjB+/HjOnz+P2Wxm/PjxLFmyhH379tm20a9fP5KSkli+fPl11aQRxkWkrDh58TILd51h4c4ztlG/ASp5mbmvSTC9mlWhURVfTCb1Y5Lir1iNMJ6Tk0NkZCRHjx7l4Ycfxtvbm7Nnz+Lj44OX143dTZGbm8u8efNIS0sjPDyc6OhosrOz6dq1q61N/fr1qVatmi08RUVF0ahRI1twAoiIiOCpp54iJiaGZs2aERUVVWAd+W1Gjx59zVoyMzPJzMy0vbdYLDe0TyIiJUHS5SwW7znHwp1n2H7iN9t0d1cnIm4LpGezKrSvXUnPlxP5nd3h6cSJE3Tv3p2TJ0+SmZnJXXfdhbe3N++88w6ZmZnMmDHDrvXt3buX8PBwMjIy8PLyYsGCBYSFhbFr1y7MZjN+fn4F2gcEBBAfHw9AfHx8geCUPz9/3l+1sVgspKenU65cuStqmjJlCpMnT7ZrP0RESpKM7FzWHExkwc4zrIlNJDs37yKEkwnuqF2Jnk2rENEwUKN+i1yF3f8qRo0aRcuWLdm9ezcVK1a0Te/VqxfDhw+3u4B69eqxa9cukpOT+fHHHxk8eDBr1661ez2FacKECYwdO9b23mKxEBIS4sCKRERuntVqsO34JRbuOsOSPeewZPzv5pqwIB96NavCP5oGE+Dj7sAqRYo/u8PT+vXr2bRpE2azucD06tWrc+bMGbsLMJvN1K5dG4AWLVqwbds2pk2bxkMPPURWVhZJSUkFzj4lJCQQGBgIQGBgIFu3bi2wvvy78f7Y5s936CUkJODj43PVs04Abm5uuLlp9FsRKR2OJKawYOcZFu48y5mkdNv0IF937m9ahV7NqlAv0NuBFYqULHaHJ6vVSm5u7hXTT58+jbf3zf/js1qtZGZm0qJFC1xdXVm1apXt7r7Y2FhOnjxJeHg4AOHh4bz55pskJibi7+8PwMqVK/Hx8SEsLMzWZunSpQW2sXLlSts6RERKo8SUDH7endePae+ZZNt0bzcX7m6U14+pTY2KGsBS5AbYHZ66devGBx98wMyZMwEwmUykpqYyadIk7rnnHrvWNWHCBO6++26qVatGSkoKc+bMITIykhUrVuDr68vQoUMZO3YsFSpUwMfHh2eeeYbw8HDatGljqyUsLIyBAwcydepU4uPjefnllxkxYoTtzNGTTz7JRx99xAsvvMBjjz3G6tWr+eGHH1iyZIm9uy4iUqxdzsrhl5gEFuw8w/rD5/n9Oby4OJnoVK8yvZpVpUsDf9xdNQ6TyM2wOzy9++67REREEBYWRkZGBg8//DCHDx+mUqVKfPfdd3atKzExkUGDBnHu3Dl8fX1p3LgxK1as4K677gLg/fffx8nJiT59+pCZmUlERASffPKJbXlnZ2cWL17MU089RXh4OJ6engwePJjXXnvN1qZGjRosWbKEMWPGMG3aNKpWrcqsWbM0xpOIlAq5VoONRy6wcOcZlsfEcznrf1cGmlXzo3ezKvRoHEwFT/NfrEVE7HFD4zzl5OQwd+5c9uzZQ2pqKs2bN2fAgAHX7ENU0mmcJxEpTgzDIObs7w/i3X2W8yn/G1oltKIHPX/vx1S9kqcDqxRxvGI1zpOLiwuPPPJIoRUhIiLX50hiCs/9sJvdp//Xj6m8hyv3Ng6mV/MqNAvx0wCWIkXshsJTbGwsH374IQcOHACgQYMGjBw5kvr16xdqcSIikscwDOZFn2bS/8WQnp2L2cWJu8IC6NW0Ch3qVsbsogEsRW4Vu8PT/Pnz6devHy1btrTdsbZ582YaNWrE3LlzC+25dyIikiclI5uXFuxj0e6zALSrXYn3HmyCv8ZjEnEIu/s81apViwEDBhTolA0wadIkvvnmG44ePVqoBRYH6vMkIo6y+1QSz87dyYmLl3F2MvFct7o82aGWhhgQuQ5F9ftt93nec+fOMWjQoCumP/LII5w7d65QihIRKeusVoPP1h2jz/RNnLh4mSp+5fjhiXCe7lRbwUnEwey+bNepUyfWr19vGxU834YNG2jfvn2hFSYiUlZdSM1k3LzdRMaeB+DuhoG83acxvuVcHVyZiMANhKd//OMfjB8/nujoaNtglZs3b2bevHlMnjyZRYsWFWgrIiLXb9ORC4z+fheJKZm4uTgx8b4wHm5VTXfQiRQjdvd5cnK6vit9JpPpqo9xKYnU50lEilpOrpUPfj3Mx5FHMAyo7e/FRw83o36gvnNEblSxGefJarUW2sZFRATOJKUz6rudbD/xGwD9W4Uw8d7bKGfWY1REiqMbGudJREQKx/J98bzw424sGTl4u7nwVu9G3Nck2NFlichfUHgSEXGAjOxc3lxygK83nwCgSYgfH/ZrRrWKHg6uTET+jsKTiMgtdiQxhZFzdnIwPgWAJzrWZFy3erg6a5RwkZJA4UlE5BYxDIN5208zaVHeI1YqeZl598GmdKxb2dGliYgdFJ5ERG6Bqz5i5aEm+HvrESsiJc11hSeLxXLdK9St/CIiBe0+lcQz3+3k5CU9YkWkNLiu8OTn53fdA7SVlrGdRERultVq8N8Ncbyz/CA5VoMqfuX4T/9mtAgt7+jSROQmXFd4WrNmje3v48eP8+KLL/Loo48SHh4OQFRUFF9++SVTpkwpmipFREoYPWJFpPSye4TxLl26MGzYMPr3719g+pw5c5g5cyaRkZGFWV+xoBHGRcQeesSKSPFQVL/fdt8XGxUVRcuWLa+Y3rJlS7Zu3VooRYmIlEQ5uVb+vSKWAf/dQmJKJnX8vVg0sh0DWocqOImUInaHp5CQED777LMrps+aNYuQkJBCKUpEpKQ5k5ROv5mb+WhN3rPp+rcKYdHIdtQL9HZ0aSJSyOwequD999+nT58+LFu2jNatWwOwdetWDh8+zPz58wu9QBGR4m75vnO88OMe2yNWpvRpxL2N9YgVkdLK7j5PAKdPn2b69OkcOHAAgAYNGvDkk0+W2jNP6vMkIleTkZ3LG0v2883mk0DeI1Y+6t+MkAp6xIpIcVBUv992nXnKzs6me/fuzJgxgzfffLPQihARKWn0iBWRssuu8OTq6sqePXuKqhYRkWJPj1gREbv/E+mRRx7hv//9b1HUIiJSrKVkZDNq7i5emL+H9Oxc2tWuxNJR7RWcRMoYuzuM5+Tk8Pnnn/Prr7/SokULPD09C8x/7733Cq04EZHiQo9YEZF8doenffv20bx5cwAOHTpUYJ7GMRGR0kaPWBGRP7M7PP3xUS0iIqXZnx+xck+jQKb01iNWRMo6u8OTiEhZ8OdHrEy67zb6twrRGXYRubHwtH37dn744QdOnjxJVlZWgXk//fRToRQmIuIIOblW3v/1EJ9EHsUwoI6/Fx893FwjhYuIjd13282dO5e2bdty4MABFixYQHZ2NjExMaxevRpfX9+iqFFE5JY4/dtlHpq5mY/XHNUjVkTkmuw+8/TWW2/x/vvvM2LECLy9vZk2bRo1atTgiSeeICgoqChqFBEpcnrEiohcL7vPPB09epQePXoAYDabSUtLw2QyMWbMGGbOnFnoBYqIFKWM7FxeXriXJ7/ZgSUjhyYhfiwd1V7BSUSuye4zT+XLlyclJe9xBFWqVGHfvn00atSIpKQkLl++XOgFiogUFT1iRURuhN3hqUOHDqxcuZJGjRrxwAMPMGrUKFavXs3KlSvp0qVLUdQoIlKo9IgVEbkZdoenjz76iIyMDABeeuklXF1d2bRpE3369OHll18u9AJFRApTSkY2Ly3Yx6LdZwFoV7sS7z3UBH9vdwdXJiIlhckwDMPRRRR3FosFX19fkpOT8fHxcXQ5InIDMrJz+W7rST5ec5QLqZk4O5kY160eT3SoqUesiJRSRfX7bfeZp0GDBtG5c2c6dOhArVq1Cq0QEZGikJ1r5cfo0/xn1WHOJeedNQ+t6MF7DzbVI1ZE5IbYHZ7MZjNTpkxh6NChVKlShY4dO9KpUyc6duxInTp1iqJGERG75VoNFu0+wwe/HubExbybWQJ93Hm2Sx0eaFlVncJF5Ibd8GW7M2fOsG7dOtauXcvatWs5dOgQQUFBnD59urBrdDhdthMpOaxWgxUx8by38hCHE1MBqORl5ulOtXm4dTXcXZ0dXKGI3CrF5rJdvvLly1OxYkXKly+Pn58fLi4uVK6sO1VExDEMwyAy9jz//iWWmLMWAHzcXXiiYy0ebVsdTzc9ylNECofd3yb//Oc/iYyMZOfOnTRo0ICOHTvy4osv0qFDB8qXV/8BEbn1Nh29wLu/HCL6xG8AeJqdGdquBkPb18S3nKuDqxOR0sbuy3ZOTk5UrlyZMWPG0Lt3b+rWrVtUtRUbumwnUjztOPkb7/4Sy8YjFwFwc3FicNvqPNmxFhU8zQ6uTkQcrdhcttu5cydr164lMjKSd999F7PZbOs03qlTpzIRpkTEsWLOJvPuL4dYfTARAFdnE/1bVWNk59r4+2i8JhEpWjc9ztPu3bt5//33+fbbb7FareTm5hZWbcWGzjyJFA9HElN4f+Vhluw9B4Czk4k+zavwbJc6VC3v4eDqRKS4KTZnngzDYOfOnURGRhIZGcmGDRuwWCw0btyYjh07FlphIiL5Tl68zAerDrFw5xmsBphMcF/jYEZ3rUPNyl6OLk9Eyhi7w1OFChVITU2lSZMmdOzYkeHDh9O+fXv8/PyKoDwRKcvOJafz4eoj/LDtFDnWvJPk3cICGNutLvUDdRZYRBzD7vD0zTff0L59e12+EpEicz4lk+mRR/lmywmycqwAdKhbmefuqkuTED/HFiciZZ7d4alHjx4AHDlyhKNHj9KhQwfKlSuHYRiYTHo+lIjcuKTLWcxcd4wvNh4nPTuv/2SrGhUY160erWpUcHB1IiJ57A5PFy9e5MEHH2TNmjWYTCYOHz5MzZo1GTp0KOXLl+fdd98tijpFpBRLzczh8w1xfLbuGCmZOQA0qerLc93q0b5OJf2HmYgUK3Y/3GnMmDG4urpy8uRJPDz+d3fLQw89xPLly+1a15QpU7j99tvx9vbG39+fnj17EhsbW6BNRkYGI0aMoGLFinh5edGnTx8SEhIKtDl58iQ9evTAw8MDf39/nn/+eXJycgq0iYyMpHnz5ri5uVG7dm1mz55t346LSKFLz8pl5rqjtH9nNe+tPERKZg71A72ZObAFC0fcQYe6lRWcRKTYsfvM0y+//MKKFSuoWrVqgel16tThxIkTdq1r7dq1jBgxgttvv52cnBz++c9/0q1bN/bv34+npyeQF9aWLFnCvHnz8PX1ZeTIkfTu3ZuNGzcCkJubS48ePQgMDGTTpk2cO3eOQYMG4erqyltvvQVAXFwcPXr04Mknn+Tbb79l1apVDBs2jKCgICIiIuw9BCJykzJzcvl+2yk+Wn2ExJRMAGpW8mT0XXW5t1EQTk4KTCJSfNk9zpO3tzc7duygTp06eHt7s3v3bmrWrMn27duJiIjg4sWLN1zM+fPn8ff3Z+3atXTo0IHk5GQqV67MnDlz6Nu3LwAHDx6kQYMGREVF0aZNG5YtW8a9997L2bNnCQgIAGDGjBmMHz+e8+fPYzabGT9+PEuWLGHfvn22bfXr14+kpKTrOlumcZ5ECkdOrpWfdpxh2qrDnElKB6CKXzlGda1D72ZVcHG2+2S4iMg1FdXvt93fVO3bt+err76yvTeZTFitVqZOnUrnzp1vqpjk5GQgbzgEgOjoaLKzs+natautTf369alWrRpRUVEAREVF0ahRI1twAoiIiMBisRATE2Nr88d15LfJX4eIFC2r1eD/dp3hrvfX8cL8PZxJSsff243X77+NNeM68WDLEAUnESkx7L5sN3XqVLp06cL27dvJysrihRdeICYmhkuXLtkupd0Iq9XK6NGjueOOO2jYsCEA8fHxmM3mK8aQCggIID4+3tbmj8Epf37+vL9qY7FYSE9Pp1y5cgXmZWZmkpmZaXtvsVhueL9EyjLDMPhlfwLv/XKI2IQUACp4mnmqYy0Ghofi7urs4ApFROxnd3hq2LAhhw4d4qOPPsLb25vU1FR69+7NiBEjCAoKuuFCRowYwb59+9iwYcMNr6OwTJkyhcmTJzu6DJESyzAM1h2+wLu/xLLndN4ZZW93Fx5vX5Mh7Wrg5Wb3V4+ISLFh1zdYdnY23bt3Z8aMGbz00kuFVsTIkSNZvHgx69atK9ARPTAwkKysLJKSkgqcfUpISCAwMNDWZuvWrQXWl3833h/b/PkOvYSEBHx8fK446wQwYcIExo4da3tvsVgICQm5uZ0UKSO2HLvIu78cYuvxSwB4mJ0Zckd1Hm9fC18PVwdXJyJy8+wKT66uruzZs6fQNm4YBs888wwLFiwgMjKSGjVqFJjfokULXF1dWbVqFX369AEgNjaWkydPEh4eDkB4eDhvvvkmiYmJ+Pv7A7By5Up8fHwICwuztVm6dGmBda9cudK2jj9zc3PDzc2t0PZTpCzYfSqJf/8Sy/rDFwAwuzgxsE0oT3WqRSUv/XsSkdLD7rvtxowZg5ubG2+//fZNb/zpp59mzpw5/N///R/16tWzTff19bWdEXrqqadYunQps2fPxsfHh2eeeQaATZs2AXlDFTRt2pTg4GCmTp1KfHw8AwcOZNiwYQWGKmjYsCEjRozgscceY/Xq1Tz77LMsWbLkuoYq0N12Itd24JyF91YeYuX+vLO7Lk4mHro9hGfurEOgr7uDqxORsqyofr/tDk/PPPMMX331FXXq1KFFixa28Zjyvffee9e/8WsMfvfFF1/w6KOPAnmDZD733HN89913ZGZmEhERwSeffGK7JAdw4sQJnnrqKSIjI/H09GTw4MG8/fbbuLj878RaZGQkY8aMYf/+/VStWpVXXnnFto2/o/AkcqWj51P54NfDLN5zFsMAJxP0alaVUV3qUK2ix9+vQESkiBWb8PRXwxGYTCZWr15900UVNwpPIv9z6tJl/rPqMPN3nMb6+7dHj8ZBjOlah9r+3o4tTkTkD4rq99vuW17WrFlTaBsXkZLlh22neGnhXrJz81JT1wb+jLmrLrcF+zq4MhGRW0f3C4vIddkad4l/LthLjtWgba2KjIuoR/Nq5R1dlojILafwJCJ/61xyOk9/G02O1eDexkF82L+ZHtgrImWWnocgIn8pIzuXJ7+O5kJqFvUDvZnat7GCk4iUaQpPInJNhmHw8sJ97D6djJ+HK58NaomHWSesRaRsU3gSkWv6KuoEP0afxskEH/VvTkgFDUEgIqLwJCJXtfnYRV5fvB+ACXc3oF2dSg6uSESkeFB4EpErnElKZ8S3O8ixGtzfNJhh7Wv8/UIiImWEwpOIFJDfQfxiWhZhQT683VsdxEVE/kjhSURsDMPgnwv2svdMMuU9XPl0YAvKmZ0dXZaISLGi8CQiNl9sPM5PO87g7GTio4fVQVxE5GoUnkQEgE1HL/Dm0gMATLi7PnfUVgdxEZGrUXgSEU7/dpmRc3aSazXo2TSYoe3UQVxE5FoUnkTKuPSsXJ74OppLaVk0rOLD233UQVxE5K8oPImUYYZhMOGnPcSctVDB08ynA1vi7qoO4iIif0XhSaQM+++GOBbuOouzk4mPH25OFb9yji5JRKTYU3gSKaM2HrnAlGUHAXjpngaE16ro4IpEREoGhSeRMujUpcuMnLODXKtB7+ZVGHJHdUeXJCJSYig8iZQx+R3Ef7ucTaMqvrzVq5E6iIuI2EHhSaQMMQyD8fP3sP+chYqeZj4d2EIdxEVE7KTwJFKGzFofx6LdZ3FxMvHJgOYEq4O4iIjdFJ5Eyoj1h88zZVneCOKv3BtG65rqIC4iciMUnkTKgFOXLvPMdzuxGtC3RVUGhYc6uiQRkRJL4UmklLuclcPwr7aTdDmbJlV9eaNnQ3UQFxG5CQpPIqWYYRg8/+MeDsanUMnLzAx1EBcRuWkKTyKl2KfrjrFkz7nfO4i3IMhXHcRFRG6WwpNIKbXu0HmmLs8bQXzSfWG0qlHBwRWJiJQOCk8ipdCJi2m2DuIPtQzhkTbqIC4iUlgUnkRKmbTMHB7/Kprk9GyahvjxWs/b1EFcRKQQKTyJlCJ5HcR3E5uQQmVvN2Y80gI3F3UQFxEpTApPIqXI9LVHWbo3HldnE9MHNCfQ193RJYmIlDoKTyKlxJrYRP61IhaAV/9xGy2rq4O4iEhRUHgSKQWOX0hj1Hc7MQzo3yqEAa3VQVxEpKgoPImUcKmZOTz+9XYsGTk0r+bHq/+4zdEliYiUagpPIiWYYRiM+2E3hxJS8fd2Y7o6iIuIFDmFJ5ES7OM1R1ge83sH8UdaEOCjDuIiIkVN4UmkhFp9MIF3Vx4C4LX7G9IitLyDKxIRKRsUnkRKoLgLaYyauwvDgIdbV6N/q2qOLklEpMxQeBIpYVIzcxj+1XZSMnJoGVqeV+9TB3ERkVtJ4UmkBLFaDcZ+v4sjiakE+LjxySPNMbvon7GIyK2kb12REuSjNUf4ZX8CZmcnZjzSAn9vdRAXEbnVFJ5ESohf9yfw/q95HcTf6NmQZtXUQVxExBEUnkRKgKPnUxnzfV4H8YFtQnnw9hBHlyQiUmYpPIkUcykZ2Tz+1XZSMnO4vXp5Xrk3zNEliYiUaQpPIsWY1Wow5vvdHD2fRqCPO58MaKEO4iIiDqZvYZFi7D+rD/PrgQTMLk58OrAFlb3dHF2SiEiZp/AkUkz9EhPPB78eBuDNng1pEuLn2IJERARQeBIplo4kpjL2h90ADA4P5YGW6iAuIlJcKDyJFDOW3zuIp2bm0KpGBV5WB3ERkWJF4UmkGLFaDcbM3cWxC2kE+7rzyYDmuDrrn6mISHGib2WRYuSDXw+x6mAiZhcnZgxsQSUvdRAXESluHBqe1q1bx3333UdwcDAmk4mFCxcWmG8YBhMnTiQoKIhy5crRtWtXDh8+XKDNpUuXGDBgAD4+Pvj5+TF06FBSU1MLtNmzZw/t27fH3d2dkJAQpk6dWtS7JmK35fvi+c/qIwBM6dWIxlX9HFuQiIhclUPDU1paGk2aNOHjjz++6vypU6fyn//8hxkzZrBlyxY8PT2JiIggIyPD1mbAgAHExMSwcuVKFi9ezLp163j88cdt8y0WC926dSM0NJTo6Gj+9a9/8eqrrzJz5swi3z+R63U4IYXnftgFwJA7qtOnRVXHFiQiItdkMgzDcHQRACaTiQULFtCzZ08g76xTcHAwzz33HOPGjQMgOTmZgIAAZs+eTb9+/Thw4ABhYWFs27aNli1bArB8+XLuueceTp8+TXBwMNOnT+ell14iPj4es9kMwIsvvsjChQs5ePDgddVmsVjw9fUlOTkZHx+fwt95KdOS07Pp+fFG4i6k0aZmBb4e2lr9nERECkFR/X4X22/ouLg44uPj6dq1q22ar68vrVu3JioqCoCoqCj8/PxswQmga9euODk5sWXLFlubDh062IITQEREBLGxsfz2229X3XZmZiYWi6XAS6Qo5FoNRs/dSdyFNKr4lePjh9VBXESkuCu239Lx8fEABAQEFJgeEBBgmxcfH4+/v3+B+S4uLlSoUKFAm6ut44/b+LMpU6bg6+tre4WEaIwdKRrvrzzEmtjzuP0+gnhFdRAXESn2im14cqQJEyaQnJxse506dcrRJUkptGzvOT5ak9dB/J0+jWlYxdfBFYmIyPUotuEpMDAQgISEhALTExISbPMCAwNJTEwsMD8nJ4dLly4VaHO1dfxxG3/m5uaGj49PgZdIYYqNT+G5eXkjiA9tV4Oezao4uCIREblexTY81ahRg8DAQFatWmWbZrFY2LJlC+Hh4QCEh4eTlJREdHS0rc3q1auxWq20bt3a1mbdunVkZ2fb2qxcuZJ69epRvnz5W7Q3Iv+TfDmbx7/ezuWsXNrWqsiEu+s7uiQREbGDQ8NTamoqu3btYteuXUBeJ/Fdu3Zx8uRJTCYTo0eP5o033mDRokXs3buXQYMGERwcbLsjr0GDBnTv3p3hw4ezdetWNm7cyMiRI+nXrx/BwcEAPPzww5jNZoYOHUpMTAzff/8906ZNY+zYsQ7aaynLcq0Gz87dyYmLl6niV46PHm6OizqIi4iUKC6O3Pj27dvp3Lmz7X1+oBk8eDCzZ8/mhRdeIC0tjccff5ykpCTatWvH8uXLcXd3ty3z7bffMnLkSLp06YKTkxN9+vThP//5j22+r68vv/zyCyNGjKBFixZUqlSJiRMnFhgLSuRW+fcvsaw9dB53VydmDmpBBU/z3y8kIiLFSrEZ56k40zhPcrMSUzL494pYfth+GoBp/Zpyf1P1cxIRKUpF9fvt0DNPIqVdZk4uX2w8zkerj5CamQPAs3fWVnASESnBFJ5EioBhGKzcn8CbSw9w4uJlAJpU9WXifbfRIlQ3KoiIlGQKTyKFLDY+hdcWx7DxyEUA/L3dGN+9Pr2aVcHJyeTg6kRE5GYpPIkUkt/Ssnhv5SG+3XICqwFmFyeGt6/B051q4+mmf2oiIqWFvtFFblJ2rpVvNp/gg18Pk5yeN57Y3Q0D+ec9DQip4OHg6kREpLApPInchLWHzvP64v0cSUwFoH6gN5Puu43wWhUdXJmIiBQVhSeRG3DsfCpvLjnAqoN5jweq4GnmuW516Xd7NZzVr0lEpFRTeBKxQ3J6Nh+uOsyXUcfJzjVwcTIxuG11nu1SB99yro4uT0REbgGFJ5HrkGs1+H7bKd79JZaLaVkAdK5XmZfvDaNWZS8HVyciIreSwpPI39h87CKTf97PgXMWAGpV9uTle8PoXM/fwZWJiIgjKDyJXMOpS5eZsuwAS/fGA+Dj7sLornUZGB6Kqx7mKyJSZik8ifxJWmYO0yOPMnP9MbJyrDiZ4OHW1Rh7Vz09yFdERBSeRPJZrQYLd53hneUHSbBkAtC2VkUm3hdG/UA9EFpERPIoPIkAO07+xuSf97P7VBIA1Sp48FKPBnQLC8Bk0tADIiLyPwpPUqbFJ2fwzvKDLNh5BgBPszMj76zDY+2q4+bi7ODqRESkOFJ4kjIpIzuXz9Yd45PIo6Rn5wLwQIuqPB9RD38fdwdXJyIixZnCk5QphmGwdG88by09wJmkdABahJZn0n1hNK7q59jiRESkRFB4kjJj35lkXlu8n61xlwAI8nVnwj0NuK9xkPo1iYjIdVN4KiPWxCayLe4Stf29qBfoTW1/rzLTp+dCaib/XhHL99tPYRjg7urEEx1q8WTHWpQzl41jICIihUfhqQxYfTCB4V9Fk2s1bNOcnUzUqORJvQBv6gXmveoHehNS3gOnUvJg26wcK7M3xfHhqiOkZOYA8I8mwbx4d32C/co5uDoRESmpFJ5KuZ0nf+Ppb3eQazVoVb0CAAfjLVgycjiSmMqRxFSW7D1na+9hdqZOgDf1AryoF+hD/d+DVSUvN0ftgt0Mw2DVgUTeXHqAuAtpADSq4suk+8Jo+fsxEBERuVEKT6XY0fOpPDZ7GxnZVjrWrcyswS1xdXbCMAwSLJkcjLcQG59CbHwKB+NTOHI+lctZuew+lWQb7yhfJS8z9QK9qRvg/Xug8qFugBce5uL1ETqckMJri/ez/vAFACp5ufFC93r0bV611JxRExERxzIZhmH8fbOyzWKx4OvrS3JyMj4+JWOk6QRLBr0/2cSZpHSaVPVlzvA2eLr9ddDJybVy/OLl3wOVhYPxKRxKSOHEpctc7VNiMuUNJvnnS3/VK3ricouf/ZZ0OYsPfj3M15tPkGs1MDs78Vi7GozoXAtvd9dbWouIiBQPRfX7rfB0HUpaeLJkZPPQp5s5cM5CjUqe/PhkOBVv4rLb5awcDiek2s5QHUrI+98LqZlXbW92caJ2ZS9boMoPVYE+7oV+V1tOrpU5W0/y3spDJF3OBqBbWAAv9WhAaEXPQt2WiIiULEX1+128rrnITcvMyeXxr7Zz4JyFyt5ufPVYq5sKTgAeZheahPjRJMSvwPSLqZm2QBUbn0JsQl6wupyVy/5zFvafsxRo7+PuQv1AH+oG/q8/Vd0Ab3zL3diZofWHz/P64v0cSkgFoF6ANxPvC+OO2pVuaH0iIiLXQ2eerkNJOfNktRo8891Oluw9h5ebC3Mfb0PDKr63vIbTv6Xb+lMdTEjhUHwKxy6kFbjb74+Cfd3z+lP9foaqXoAPtfw9rzmUwvELabyx5AC/HkgAoLyHK2O71aP/7SG3/HKhiIgUX7ps50AlITwZhsHkn/cze9NxXJ1NzB7SqlidgcnMyeVoYhqxCb/3pfr9bNXZ5Iyrtnd2MlGzkmdeoPq9T1XNyp7M236azzfGkZ1r4OJkYmB4KKO71MXXQ/2aRESkIF22k780fe1RZm86DsC7DzYtVsEJwM3FmbBgH8KCC354k9OzbX2oYuMtHIpPtQ2lcDgxlcOJqSzh3BXr61C3MhPvbUBtf+9btQsiIiKAwlOpMG/7KaYujwXglXvD+EeTYAdXdP18y7lye/UK3P6H8ZcMwyDekmHrS3XoD0MphFbwYMI99elcz1+PVBEREYdQeCrh1sQm8uJPewF4omNNhrar4eCKbp7JZCLItxxBvuXoXM/fNt1qNTRWk4iIOJx615Zgu04l8fQ3eaOH925WhfER9R1dUpFScBIRkeJA4amEOvb76OHp2bl0qFuZd/o2VrgQERG5BRSeSqBESwaDPt/KpbQsGlf1ZfqA5rjqFn0REZFbQr+4JYwlI5vBX2zj9G/pVK/oweeP3v63j10RERGRwqPwVIJk5uTy5NfRHDhnoZKXma8ea02lmxw9XEREROyj8FRCWK0Gz/2wm01HL+Jpdmb2kFZUq+jh6LJERETKHIWnEsAwDF5fsp/Fe87h6mxixsAWt/yxKyIiIpJH4akE+HTdMb7YeByAfz/QhPZ1Kju2IBERkTJM4amYmx99mreXHQTg5R4NuL9pFQdXJCIiUrYpPBVjkbGJjJ+/B4DHO9RkWPuaDq5IREREFJ6Kqd2nknj62x3kWA16Ng3mxe6le/RwERGRkkLhqRiKu5DGkNnbuJyVS/s6lZjat4lGDxcRESkmFJ6KmcSUDAZ9voVLaVk0quLL9EdaYHbR/00iIiLFhX6Vi5GUjGyGfLGNU5fSCf199HAvjR4uIiJSrCg8FRNZOVae/CaamLP5o4e3orK3Rg8XEREpbhSeigGr1WDcvN1sPHIRD7MzXzzaitCKno4uS0RERK5C4cnBDMPgjSUHWLT7LC5OJmY80oJGVTV6uIiISHGl8ORgn60/xucb44C80cM71NXo4SIiIsWZwpMDLdh5mreW5o0e/tI9DejZTKOHi4iIFHcKTw6y7tB5np+XN3r4sHY1GN5Bo4eLiIiUBApPDrDndBJPfhNNjtXg/qbB/POeBo4uSURERK5TmQpPH3/8MdWrV8fd3Z3WrVuzdevWW17D8QtpDPkib/TwdrUr8S+NHi4iIlKilJnw9P333zN27FgmTZrEjh07aNKkCRERESQmJt6yGs6nZDLo861cTMuiYRUfZgzU6OEiIiIlTZn55X7vvfcYPnw4Q4YMISwsjBkzZuDh4cHnn39+S7afmpnDkNlbOXnpMtUqePDFo600eriIiEgJVCZ+vbOysoiOjmbChAm2aU5OTnTt2pWoqKgr2mdmZpKZmWl7n5ycDIDFYrmx7edYGTFnB3uOXaSChysfPVAfNyMTiyXz7xcWERGRG5L/u20YRqGut0yEpwsXLpCbm0tAQECB6QEBARw8ePCK9lOmTGHy5MlXTA8JCbnpWk4BTd666dWIiIjIdbp48SK+voU3AHWZCE/2mjBhAmPHjrW9T0pKIjQ0lJMnTxbqwS+LLBYLISEhnDp1Ch8fH0eXU6LpWBYOHcfCo2NZeHQsC0dycjLVqlWjQoUKhbreMhGeKlWqhLOzMwkJCQWmJyQkEBgYeEV7Nzc33NyufCivr6+vPsSFxMfHR8eykOhYFg4dx8KjY1l4dCwLh5NT4XbxLhMdxs1mMy1atGDVqlW2aVarlVWrVhEeHu7AykRERKSkKRNnngDGjh3L4MGDadmyJa1ateKDDz4gLS2NIUOGOLo0ERERKUHKTHh66KGHOH/+PBMnTiQ+Pp6mTZuyfPnyKzqRX42bmxuTJk266qU8sY+OZeHRsSwcOo6FR8ey8OhYFo6iOo4mo7Dv3xMREREpxcpEnycRERGRwqLwJCIiImIHhScREREROyg8iYiIiNhB4el3H3/8MdWrV8fd3Z3WrVuzdevWv2w/b9486tevj7u7O40aNWLp0qW3qNLiz55jOXv2bEwmU4GXu7v7Lay2eFq3bh333XcfwcHBmEwmFi5c+LfLREZG0rx5c9zc3KhduzazZ88u8jpLAnuPZWRk5BWfSZPJRHx8/K0puJiaMmUKt99+O97e3vj7+9OzZ09iY2P/djl9V17pRo6lviuvbvr06TRu3Ng2mGh4eDjLli37y2UK4zOp8AR8//33jB07lkmTJrFjxw6aNGlCREQEiYmJV22/adMm+vfvz9ChQ9m5cyc9e/akZ8+e7Nu37xZXXvzYeywhbwTdc+fO2V4nTpy4hRUXT2lpaTRp0oSPP/74utrHxcXRo0cPOnfuzK5duxg9ejTDhg1jxYoVRVxp8WfvscwXGxtb4HPp7+9fRBWWDGvXrmXEiBFs3ryZlStXkp2dTbdu3UhLS7vmMvquvLobOZag78qrqVq1Km+//TbR0dFs376dO++8k/vvv5+YmJirti+0z6QhRqtWrYwRI0bY3ufm5hrBwcHGlClTrtr+wQcfNHr06FFgWuvWrY0nnniiSOssCew9ll988YXh6+t7i6ormQBjwYIFf9nmhRdeMG677bYC0x566CEjIiKiCCsrea7nWK5Zs8YAjN9+++2W1FRSJSYmGoCxdu3aa7bRd+X1uZ5jqe/K61e+fHlj1qxZV51XWJ/JMn/mKSsri+joaLp27Wqb5uTkRNeuXYmKirrqMlFRUQXaA0RERFyzfVlxI8cSIDU1ldDQUEJCQv7yvxjk2vSZLHxNmzYlKCiIu+66i40bNzq6nGInOTkZ4C8fuKrP5fW5nmMJ+q78O7m5ucydO5e0tLRrPnqtsD6TZT48Xbhwgdzc3CtGGg8ICLhmH4f4+Hi72pcVN3Is69Wrx+eff87//d//8c0332C1Wmnbti2nT5++FSWXGtf6TFosFtLT0x1UVckUFBTEjBkzmD9/PvPnzyckJIROnTqxY8cOR5dWbFitVkaPHs0dd9xBw4YNr9lO35V/73qPpb4rr23v3r14eXnh5ubGk08+yYIFCwgLC7tq28L6TJaZx7NI8RQeHl7gvxDatm1LgwYN+PTTT3n99dcdWJmUVfXq1aNevXq2923btuXo0aO8//77fP311w6srPgYMWIE+/btY8OGDY4upcS73mOp78prq1evHrt27SI5OZkff/yRwYMHs3bt2msGqMJQ5s88VapUCWdnZxISEgpMT0hIIDAw8KrLBAYG2tW+rLiRY/lnrq6uNGvWjCNHjhRFiaXWtT6TPj4+lCtXzkFVlR6tWrXSZ/J3I0eOZPHixaxZs4aqVav+ZVt9V/41e47ln+m78n/MZjO1a9emRYsWTJkyhSZNmjBt2rSrti2sz2SZD09ms5kWLVqwatUq2zSr1cqqVauuec00PDy8QHuAlStXXrN9WXEjx/LPcnNz2bt3L0FBQUVVZqmkz2TR2rVrV5n/TBqGwciRI1mwYAGrV6+mRo0af7uMPpdXdyPH8s/0XXltVquVzMzMq84rtM/kDXZmL1Xmzp1ruLm5GbNnzzb2799vPP7444afn58RHx9vGIZhDBw40HjxxRdt7Tdu3Gi4uLgY//73v40DBw4YkyZNMlxdXY29e/c6aheKDXuP5eTJk40VK1YYR48eNaKjo41+/foZ7u7uRkxMjKN2oVhISUkxdu7caezcudMAjPfee8/YuXOnceLECcMwDOPFF180Bg4caGt/7Ngxw8PDw3j++eeNAwcOGB9//LHh7OxsLF++3FG7UGzYeyzff/99Y+HChcbhw4eNvXv3GqNGjTKcnJyMX3/91VG7UCw89dRThq+vrxEZGWmcO3fO9rp8+bKtjb4rr8+NHEt9V17diy++aKxdu9aIi4sz9uzZY7z44ouGyWQyfvnlF8Mwiu4zqfD0uw8//NCoVq2aYTabjVatWhmbN2+2zevYsaMxePDgAu1/+OEHo27duobZbDZuu+02Y8mSJbe44uLLnmM5evRoW9uAgADjnnvuMXbs2OGAqouX/Nvl//zKP3aDBw82OnbseMUyTZs2Ncxms1GzZk3jiy++uOV1F0f2Hst33nnHqFWrluHu7m5UqFDB6NSpk7F69WrHFF+MXO0YAgU+Z/quvD43ciz1XXl1jz32mBEaGmqYzWajcuXKRpcuXWzByTCK7jNpMgzDsO9clYiIiEjZVeb7PImIiIjYQ+FJRERExA4KTyIiIiJ2UHgSERERsYPCk4iIiIgdFJ5ERERE7KDwJCIiImIHhScRKRaOHz+OyWRi165dji5FRG6hdevWcd999xEcHIzJZGLhwoV2Lf/qq69iMpmueHl6ehZNwSg8iZQ558+fx2w2k5aWRnZ2Np6enpw8edLRZRESEsK5c+do2LCho0spUp06dWL06NEOX4dIcZGWlkaTJk34+OOPb2j5cePGce7cuQKvsLAwHnjggUKu9H8UnkTKmKioKJo0aYKnpyc7duygQoUKVKtWzdFl4ezsTGBgIC4uLledbxgGOTk5t7gqESlqd999N2+88Qa9evW66vzMzEzGjRtHlSpV8PT0pHXr1kRGRtrme3l5ERgYaHslJCSwf/9+hg4dWmQ1KzyJlDGbNm3ijjvuAGDDhg22v//OrFmzaNCgAe7u7tSvX59PPvnENi//kttPP/1E586d8fDwoEmTJkRFRQFgsVgoV64cy5YtK7DOBQsW4O3tzeXLl6+4bBcZGYnJZGLZsmW0aNECNzc3NmzYQGZmJs8++yz+/v64u7vTrl07tm3bZltn/nKrVq2iZcuWeHh40LZtW2JjY21tXn31VZo2bcrnn39OtWrV8PLy4umnnyY3N5epU6cSGBiIv78/b775ZoF6k5KSGDZsGJUrV8bHx4c777yT3bt3X7Her7/+murVq+Pr60u/fv1ISUkB4NFHH2Xt2rVMmzbNdmnh+PHjVz3en3zyCXXq1MHd3Z2AgAD69u37t+vYt28fd999N15eXgQEBDBw4EAuXLhgW2enTp0YOXIkI0eOxNfXl0qVKvHKK6/wx6d0XWu7Io4ycuRIoqKimDt3Lnv27OGBBx6ge/fuHD58+KrtZ82aRd26dWnfvn3RFXUzD+QTkZLhxIkThq+vr+Hr62u4uroa7u7uhq+vr2E2mw03NzfD19fXeOqpp665/DfffGMEBQUZ8+fPN44dO2bMnz/fqFChgjF79mzDMAwjLi7OAIz69esbixcvNmJjY42+ffsaoaGhRnZ2tmEYhtG3b1/jkUceKbDePn362Kblr2Pnzp2GYfzvgb6NGzc2fvnlF+PIkSPGxYsXjWeffdYIDg42li5dasTExBiDBw82ypcvb1y8eLHAcq1btzYiIyONmJgYo3379kbbtm1t2500aZLh5eVl9O3b14iJiTEWLVpkmM1mIyIiwnjmmWeMgwcPGp9//rkBFHiwddeuXY377rvP2LZtm3Ho0CHjueeeMypWrGjbdv56e/fubezdu9dYt26dERgYaPzzn/80DMMwkpKSjPDwcGP48OHGuXPnjHPnzhk5OTlXHO9t27YZzs7Oxpw5c4zjx48bO3bsMKZNm/aX6/jtt9+MypUrGxMmTDAOHDhg7Nixw7jrrruMzp0729bbsWNHw8vLyxg1apRx8OBB45tvvjE8PDyMmTNn/u12RW4FwFiwYIHt/YkTJwxnZ2fjzJkzBdp16dLFmDBhwhXLp6enG+XLlzfeeeedoq2zSNcuIsVCdna2ERcXZ+zevdtwdXU1du/ebRw5csTw8vIy1q5da8TFxRnnz5+/5vK1atUy5syZU2Da66+/boSHhxuG8b/gM2vWLNv8mJgYAzAOHDhgGIZhLFiwwPDy8jLS0tIMwzCM5ORkw93d3Vi2bFmBdfw5PC1cuNC2ztTUVMPV1dX49ttvbdOysrKM4OBgY+rUqQWW+/XXX21tlixZYgBGenq6YRh5IcfDw8OwWCy2NhEREUb16tWN3Nxc27R69eoZU6ZMMQzDMNavX2/4+PgYGRkZVxybTz/99Jrrff75543WrVvb3nfs2NEYNWrUNY50nvnz5xs+Pj4F1vNHV1vH66+/bnTr1q3AtFOnThmAERsba1uuQYMGhtVqtbUZP3680aBBg+varkhR+3N4Wrx4sQEYnp6eBV4uLi7Ggw8+eMXyc+bMMVxcXIz4+PgirfPqnQtEpFRxcXGhevXq/PDDD9x+++00btyYjRs3EhAQQIcOHf5y2bS0NI4ePcrQoUMZPny4bXpOTg6+vr4F2jZu3Nj2d1BQEACJiYnUr1+fe+65B1dXVxYtWkS/fv2YP38+Pj4+dO3a9S+337JlS9vfR48eJTs7u8ClRldXV1q1asWBAweuq5b8/l3Vq1fH29vb1iYgIABnZ2ecnJwKTEtMTARg9+7dpKamUrFixQLbSU9P5+jRo7b3f15vUFCQbR3X66677iI0NJSaNWvSvXt3unfvTq9evfDw8LjmMrt372bNmjV4eXldMe/o0aPUrVsXgDZt2mAymWzzwsPDeffdd8nNzb2h7YoUpdTUVJydnYmOjsbZ2bnAvKt91mfNmsW9995LQEBAkdal8CRSBtx2222cOHGC7OxsrFYrXl5e5OTkkJOTg5eXF6GhocTExFx12dTUVAA+++wzWrduXWDen7/MXF1dbX/n/0BbrVYAzGYzffv2Zc6cOfTr1485c+bw0EMPXbODeL4bvd34r2r58/z8Nleblr9MamoqQUFBBTqq5vPz8/vL9f5xu9fD29ubHTt2EBkZyS+//MLEiRN59dVX2bZtW4Ft/VFqair33Xcf77zzzhXz8sNjUWxXpCg1a9aM3NxcEhMT/7YPU1xcHGvWrGHRokVFXpfCk0gZsHTpUrKzs+nSpQtTp06lRYsW9OvXj0cffZTu3btf8YP/RwEBAQQHB3Ps2DEGDBhwU3UMGDCAu+66i5iYGFavXs0bb7xh1/K1atXCbDazceNGQkNDAcjOzmbbtm1Ffut+8+bNiY+Pt53Fu1Fms5nc3Ny/befi4kLXrl3p2rUrkyZNws/Pj9WrV9O7d++rrqN58+bMnz+f6tWr/2Ug3bJlS4H3mzdvpk6dOrYg/FfbFSkKqampHDlyxPY+Li6OXbt2UaFCBerWrcuAAQMYNGgQ7777Ls2aNeP8+fOsWrWKxo0b06NHD9tyn3/+OUFBQdx9991FXrPCk0gZEBoaSnx8PAkJCdx///2YTCZiYmLo06fPdZ2VmDx5Ms8++yy+vr50796dzMxMtm/fzm+//cbYsWOvu44OHToQGBjIgAEDqFGjxhVnsv6Op6cnTz31FM8//7xtiIWpU6dy+fLlIr0tGaBr166Eh4fTs2dPpk6dSt26dTl79ixLliyhV69eBS4v/pXq1auzZcsWjh8/jpeXFxUqVChwqRBg8eLFHDt2jA4dOlC+fHmWLl2K1WqlXr1611zHiBEj+Oyzz+jfvz8vvPACFSpU4MiRI8ydO5dZs2bZwtHJkycZO3YsTzzxBDt27ODDDz/k3Xffva7tihSF7du307lzZ9v7/O+UwYMHM3v2bL744gveeOMNnnvuOc6cOUOlSpVo06YN9957r20Zq9XK7NmzefTRR684I14UFJ5EyojIyEhuv/123N3dWb9+PVWrVr3uyznDhg3Dw8ODf/3rXzz//PN4enrSqFEju8/2mEwm+vfvz9SpU5k4ceIN7AW8/fbbWK1WBg4cSEpKCi1btmTFihWUL1/+htZ3vUwmE0uXLuWll15iyJAhnD9/nsDAQDp06GBX/4px48YxePBgwsLCSE9PJy4u7oozWX5+fvz000+8+uqrZGRkUKdOHb777jtuu+22v1zHxo0bGT9+PN26dSMzM5PQ0FC6d+9eIJwNGjSI9PR0WrVqhbOzM6NGjeLxxx+/ru2KFIVOnToVGC7jz1xdXZk8eTKTJ0++ZhsnJydOnTpVFOVdlcn4q4pFRKTU6NSpE02bNuWDDz5wdCkiJZoGyRQRERGxg8KTiIiIiB102U5ERETEDjrzJCIiImIHhScREREROyg8iYiIiNhB4UlERETEDgpPIiIiInZQeBIRERGxg8KTiIiIiB0UnkRERETsoPAkIiIiYof/B9hmBJXj8WAEAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "time to jit: 0:00:47.071182\n", + "time to train: 0:04:52.657978\n", + "eval steps/sec: 251792.92696345953\n", + "train steps/sec: 127184.56138694892\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Y2p-20bCi4iI" + }, + "source": [ + "In this arrangement, we can rollout environment steps much faster than we can train: the speed at which PyTorch can backpropagate the loss and step the optimizer is the bottleneck. This PyTorch code can probably be sped up by adding [automatic mixed precision](https://pytorch.org/docs/stable/notes/amp_examples.html), and following other recommendations in the [PyTorch performance tuning guide](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html).\n", + "\n", + "We know we have a fair bit of headroom to improve the PyTorch implementation, as the built-in Brax trainer (which uses [flax.optim](https://flax.readthedocs.io/en/latest/flax.optim.html)) runs at more than double the steps per second:" + ] + }, + { + "cell_type": "code", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Xmuz3I21p35H", + "outputId": "020efafe-d940-4943-9ca2-bdd534ca2b4f" + }, + "source": [ + "train_sps = []\n", + "\n", + "def progress(_, metrics):\n", + " if 'training/sps' in metrics:\n", + " train_sps.append(metrics['training/sps'])\n", + "\n", + "ppo.train(\n", + " environment=envs.create(env_name='ant', backend='spring'),\n", + " num_timesteps = 30_000_000, num_evals = 10, reward_scaling = .1,\n", + " episode_length = 1000, normalize_observations = True, action_repeat = 1,\n", + " unroll_length = 5, num_minibatches = 32, num_updates_per_batch = 4,\n", + " discounting = 0.97, learning_rate = 3e-4, entropy_cost = 1e-2,\n", + " num_envs = 2048, batch_size = 1024, progress_fn = progress)\n", + "\n", + "print(f'train steps/sec: {np.mean(train_sps[1:])}')" + ], + "execution_count": 6, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "train steps/sec: 437059.02080672665\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eqXKdDwVL6L4" + }, + "source": [ + "tunaalabagana! 👋" + ] + } + ] +} \ No newline at end of file