agents/mastermind-openai-chat-model-with-metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OpenAI Simple Chat Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip3 install openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "OPENAI_API_KEY = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agentquest.drivers.mastermind import MasterMindDriver\n",
    "from agentquest.metrics import get_mastermind_repetitions, get_mastermind_progress\n",
    "from agentquest.utils import Action, load_data, cpprint\n",
    "from termcolor import cprint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load a generic 4 digits mastermind game"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "game = load_data('mastermind', '4 digits')[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize the prompt and the LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "prompt = '''You are tasked to play the Mastermind game. The host chooses a \n",
    "number and gives you the amount of digits. You have to gess the correct number\n",
    "as fast as you can.\n",
    "The number can contains repetitions.\n",
    "The number can contain any possible digit between:\n",
    "0, 1, 2, 3, 4, 5, 6, 7, 8, 9\n",
    "\n",
    "At each round you provide a number as guess. The hosts provides you \n",
    "1. The number of correct digits in the wrong position\n",
    "2. The number of correct digits in the correct position\n",
    "\n",
    "The game ends when the hosts outputs 'You Won!'\n",
    "\n",
    "Carefully choose your strategy. Avoid bruteforce.\n",
    "\n",
    "Use the following format:\n",
    "THOUGHT: think about your guess\n",
    "ACTION: only your guessed number without additional info.\n",
    "'''\n",
    "\n",
    "# Initialize the LLM\n",
    "client = OpenAI(\n",
    "    api_key=OPENAI_API_KEY,\n",
    ")\n",
    "\n",
    "# Initalize the system prompt\n",
    "messages=[\n",
    "    {\"role\": \"system\", \"content\": prompt},\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize the mastermind driver and get the first observation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mOBSERVATION: Start guessing the 4 digits number.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "driver = MasterMindDriver(game)\n",
    "obs = driver.reset() # Get the first observation\n",
    "cpprint(f'OBSERVATION: {obs.output}', 'cyan')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start the LLM loop solving the task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[35mTHOUGHT: I will start by guessing four different digits. Following the \n",
      " sequence from 0-9, I will start with 0123.\u001b[0m\n",
      "\u001b[33mACTION: 0123\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 1 correct numbers in the wrong position and 0 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         0 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: This means there may be one digit among 0, 1, 2, 3 but it is \n",
      " located at a different position other than my first guess. However, we can \n",
      " not confirm that one of this digits is in the answer yet. It could be a 0, \n",
      " 1, 2, or 3 in the correct number but in the wrong position. I will now guess \n",
      " the next four numbers in the sequence, 4567.\u001b[0m\n",
      "\u001b[33mACTION: 4567\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 1 correct numbers in the wrong position and 0 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         0 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: Now I know that one of the digits is either a 4, 5, 6, or 7 but \n",
      " \"its not in the right position. There is no overlap between the first two \"\n",
      " guesses, so one digit from each guess is definitely in the final number but \n",
      " \"at a different position. Now Ill test the remaining digits, 89, and re-use \"\n",
      " two guessed digits 0, 1 in different positions.\u001b[0m\n",
      "\u001b[33mACTION: 8901\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 2 correct numbers in the wrong position and 1 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         1 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: Having one digit at the correct position means that the correct \n",
      " number either ends with a 1 or starts with an 8. The other two correct \n",
      " numbers are in wrong positions and they can be 8,9,0, or 1. Now, taking into \n",
      " consideration the results of all guesses:\n",
      " \n",
      " 1- The number certainly includes one digit from the first guess 0123, \n",
      " another digit from the second guess 4567, but in different positions.  \n",
      " \n",
      " 2- The number possibly starts with 8 or ends with 1 as per the third \n",
      " guess.  \n",
      " \n",
      " To confirm my thoughts, I will move the 1 to the start of the number and \n",
      " keep the 8 at the end, while introducing two new digits in between from my \n",
      " previously guessed pools.\u001b[0m\n",
      "\u001b[33mACTION: 1458\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 2 correct numbers in the wrong position and 1 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         1 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: Any of the digits 1, 4, 5, or 8 could be at the correct position \n",
      " while 2 other digits from among these are in the number but placed wrongly. \n",
      " We can reasonably assume that the 1 and 8 were correctly positioned at least \n",
      " \"once since they were used in the third and fourth guesses. Ill keep the 1 \"\n",
      " \"and 8 where they are now and switch the 4 and 5 with two numbers we havent \"\n",
      " \"used yet, besides 0 which we cant use in the first position.\"\n",
      " Let me try 1678.\u001b[0m\n",
      "\u001b[33mACTION: 1678\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 1 correct numbers in the wrong position and 1 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         1 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: From the previous guess, I concluded that one digit among 1,4,5,8 \n",
      " is on the right position and one digit among 1,6,7,8 is in the number but \n",
      " placed wrong. Also, comparing with previous guesses, it seems that 8 at the \n",
      " end and 1 at the start usually give me one digit at the correct place. Now \n",
      " \"Ill try keeping 1 at the start and 8 at the end, while experimenting with \"\n",
      " the middle two digits from the remaining pool of numbers.\u001b[0m\n",
      "\u001b[33mACTION: 1238\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 1 correct numbers in the wrong position and 1 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         1 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: The trend of having 1 correct number at the correct position and \n",
      " another at the wrong position continues, which implies that our assumption \n",
      " about 1 and 8 possibly being correctly positioned might be accurate. \n",
      " Considering past guesses, it seems that either 2 or 3 are also in the number \n",
      " \"but wrongly positioned. For the next guess, Ill try swapping the 1 and the \"\n",
      " 8 while bringing back digits 4 and 5.\n",
      " \u001b[0m\n",
      "\u001b[33mACTION: 8125\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 3 correct numbers in the wrong position and 0 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         0 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: This means 8, 1, and either 2 or 5, are in the number but placed \n",
      " wrongly.\n",
      " \"Lets try 5812. This keeps 8 and 1, includes 5 from the last guess, and 2 \"\n",
      " from previous guesses, but each in a new position.\u001b[0m\n",
      "\u001b[33mACTION: 5812\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 1 correct numbers in the wrong position and 2 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         2 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: Based on the result, 5 or 1 are at the correct position and either \n",
      " 8 or 2 is in the number but wrongly positioned. A repeated pattern is that 1 \n",
      " \"is almost always in the right position when its in the second spot. \"\n",
      " \"Besides, a digit from 8 or 2 is definitely in the final number. Now, lets \"\n",
      " \"assume that the second position is indeed 1 and the last digit is 2. I \"\n",
      " will swap 5 and 8 for the next guess.\n",
      " \u001b[0m\n",
      "\u001b[33mACTION: 8152\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 3 correct numbers in the wrong position and 0 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         0 reached milestones\u001b[0m\n",
      "\n",
      "\u001b[35mTHOUGHT: This indicates that 8, 1, and either 5 or 2, are somewhere in the \n",
      " number but in wrong positions. The position of 1 in the second place being \n",
      " \"correct is false. So 1 might be in the 3rd spot and 8 in the 4th spot. 5 \"\n",
      " \"or 2 must be in the 2nd spot and for the first spot, I havent tried 9 \"\n",
      " \"yet. Lets try 9581.\"\u001b[0m\n",
      "\u001b[33mACTION: 9581\u001b[0m\n",
      "\u001b[36mOBSERVATION: Your guess has 4 correct numbers in the wrong position and 0 \n",
      " correct numbers in the correct position. Keep guessing.\u001b[0m\n",
      "\u001b[31mMETRICS: 0 repeated actions\u001b[0m\n",
      "\u001b[31m         0 reached milestones\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "actions, progress, repetitions = [], [], []\n",
    "step_cnt = 0\n",
    "while not obs.done and step_cnt<10:\n",
    "    # Provide the observation to the LLM\n",
    "    messages.append({\"role\": \"user\", \"content\": obs.output})\n",
    "    \n",
    "    # Get the LLM output\n",
    "    agent_choice = client.chat.completions.create(\n",
    "        model=\"gpt-4\",\n",
    "        messages=messages\n",
    "    ).choices[0].message.content\n",
    "    \n",
    "    # Provide the choice to the LLM as history\n",
    "    messages.append({\"role\": \"assistant\", \"content\": agent_choice})\n",
    "    \n",
    "    # Extract the LLM guess\n",
    "    thought = agent_choice.split('ACTION: ')[0].split('THOUGHT: ')[-1]\n",
    "    guess = agent_choice.split('ACTION: ')[-1]\n",
    "    \n",
    "    # Build the action\n",
    "    action = Action(action_value=guess)\n",
    "    # Store action\n",
    "    actions.append(action.action_value)\n",
    "    \n",
    "    # Run a mastermind round and get the observation\n",
    "    obs = driver.step(action)\n",
    "    \n",
    "    # Compute current progress and repetition\n",
    "    repeated_actions = get_mastermind_repetitions(actions)\n",
    "    current_progress = get_mastermind_progress(driver.state, game)\n",
    "    repetitions.append(repeated_actions)\n",
    "    progress.append(current_progress)\n",
    "\n",
    "    print()\n",
    "    cpprint(f'THOUGHT: {thought}', 'magenta')\n",
    "    cpprint(f'ACTION: {action.action_value}', 'yellow')\n",
    "    cpprint(f'OBSERVATION: {obs.output}', 'cyan')\n",
    "    cpprint(f'METRICS: {repeated_actions} repeated actions', 'red')\n",
    "    cpprint(f'         {current_progress} reached milestones', 'red')\n",
    "    \n",
    "    step_cnt+=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PR = [0.0, 0.0, 0.25, 0.25, 0.25, 0.25, 0.0, 0.5, 0.0, 0.0]\n",
      "PR = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n"
     ]
    }
   ],
   "source": [
    "# Compute final metrics\n",
    "PR = [x/len(game) for x in progress]\n",
    "RR = [x/(len(actions)-1) for x in repetitions]\n",
    "print(f'PR = {PR}')\n",
    "print(f'PR = {RR}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}