quickstarts/New_in_002.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8968a502d25e"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "cellView": "form",
        "id": "ca23c3f523a7"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wwH5rYtX5Xco"
      },
      "source": [
        "# What's new in Gemini-1.5-pro-002 and Gemini-1.5-flash-002"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "67ad518df45c"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/New_in_002.ipynb\"><img src=\"../images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V4RV6XuQLjCc"
      },
      "source": [
        "This notebook explores the new options added with the 002 versions of the 1.5 series models:\n",
        "\n",
        "* Candidate count\n",
        "* Presence and frequency penalties\n",
        "* Response logprobs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a5ZjsAuEGK6i"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tnl6PeGKOfcA"
      },
      "source": [
        "Install a `002` compatible version of the SDK:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xVTnXHg_nxMC"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m153.4/153.4 kB\u001b[0m \u001b[31m2.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m760.0/760.0 kB\u001b[0m \u001b[31m11.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install -q \"google-generativeai>=0.8.2\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qLASVngmOm8K"
      },
      "source": [
        "import the package and give it your API-key"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "B6fY-RbxJl78"
      },
      "outputs": [],
      "source": [
        "import google.generativeai as genai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sRmN-qzhM47E"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "genai.configure(api_key=userdata.get('GOOGLE_API_KEY'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7d-e1KfaOxlJ"
      },
      "source": [
        "Import other packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "giN4GuufOu02"
      },
      "outputs": [],
      "source": [
        "from IPython.display import display, Markdown, HTML"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJO3Cz7UO0Ms"
      },
      "source": [
        "Check available 002 models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "CTBDsceYMh2S"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "models/gemini-1.5-pro-002\n",
            "models/gemini-1.5-flash-002\n"
          ]
        }
      ],
      "source": [
        "for model in genai.list_models():\n",
        "  if '002' in model.name:\n",
        "    print(model.name)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "RCFdQUXsIDcm"
      },
      "outputs": [],
      "source": [
        "model_name = \"models/gemini-1.5-flash-002\"\n",
        "test_prompt=\"Why don't people have tails\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EBiweaL6LLKm"
      },
      "source": [
        "## Quick refresher on `generation_config` [Optional]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "bwwgGRtnLOqJ"
      },
      "outputs": [],
      "source": [
        "model = genai.GenerativeModel(model_name, generation_config={'temperature':1.0})\n",
        "response = model.generate_content('hello', generation_config = genai.GenerationConfig(max_output_tokens=5))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5sDxl0g-LQdx"
      },
      "source": [
        "Note:\n",
        "\n",
        "* Each `generate_content` request is sent with a `generation_config` (`chat.send_message` uses `generate_content`).\n",
        "* You can set the `generation_config` by either passing it to the model's initializer, or passing it in the arguments to `generate_content` (or `chat.send_message`).\n",
        "* Any `generation_config` attributes set in `generate_content` override the attributes set on the model.\n",
        "* You can pass the `generation_config` as either a Python `dict`, or a `genai.GenerationConfig`.\n",
        "* If you're ever unsure about the parameters of `generation_config` check `genai.GenerationConfig`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Cu9hYM4H8Cz"
      },
      "source": [
        "## Candidate count"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y_D56ekoO4pd"
      },
      "source": [
        "With 002 models you can now use `candidate_count > 1`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ez_8Dm-WMIcp"
      },
      "outputs": [],
      "source": [
        "model = genai.GenerativeModel(model_name)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v5fwiDPjIAfj"
      },
      "outputs": [],
      "source": [
        "generation_config = dict(candidate_count=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Xr6DXm9_IGEJ"
      },
      "outputs": [],
      "source": [
        "response = model.generate_content(test_prompt, generation_config=generation_config)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xg9Qdl1oPBX9"
      },
      "source": [
        "But note that the `.text` quick-accessor only works for the simple 1-candidate case."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gxNgE6_SI7az"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Invalid operation: The `response.parts` quick accessor retrieves the parts for a single candidate. This response contains multiple candidates, please use `result.candidates[index].text`.\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "  response.text # Fails with multiple candidates, sorry!\n",
        "except ValueError as e:\n",
        "  print(e)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HDCqgP7YPHur"
      },
      "source": [
        "With multiple candidates you have to handle the list of candidates yourself:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b8Ud37EjJCfc"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Humans don't have tails because of evolution.  Our ancestors, primates like apes and monkeys, *did* have tails, but over millions of years of evolution, the tail gradually disappeared.  The exact reasons are complex and not fully understood, but several contributing factors are likely:\n",
              "\n",
              "* **Loss of function:**  As our ancestors adapted to arboreal (tree-dwelling) life and then to bipedalism (walking upright), the need for a tail for balance and locomotion decreased.  Tails are primarily used for balance and climbing;  our hands and upright posture made a tail less essential.\n",
              "\n",
              "* **Natural selection:** Individuals with shorter tails or rudimentary tails might have had a slight survival advantage. Perhaps shorter tails were less cumbersome while climbing, less likely to be injured, or offered less drag while running.  Over generations, these small advantages led to the selection and propagation of genes that resulted in shorter and shorter tails.  Eventually, the genes responsible for tail development became suppressed.\n",
              "\n",
              "* **Genetic mutations:**  Genetic mutations that impacted tail development could have occurred randomly. Some of these mutations may have provided advantages that were selected for, while others may have been neutral and simply became fixed in the population over time.\n",
              "\n",
              "It's important to note that the complete loss of a tail wasn't a sudden event. It was a gradual process taking place over a very long evolutionary timescale.  We still retain some vestigial remnants of a tail in the coccyx (tailbone), which is a small, fused bone at the base of the spine.  This provides a structural support for our pelvis but is the only physical evidence of what was once a functional tail.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Humans don't have tails because of evolution.  Our ancestors had tails, but over millions of years, the genes that controlled tail development were suppressed.  There are a few theories about why this happened:\n",
              "\n",
              "* **Loss of function:**  The tail may have become less advantageous.  While useful for balance and communication in arboreal primates, as our ancestors transitioned to walking upright and relied less on climbing, the tail's utility decreased.  The energy and resources used to maintain a tail became a disadvantage, so individuals with smaller tails or no tails had a slight evolutionary advantage.  Natural selection favored individuals with mutations that progressively reduced tail size.\n",
              "\n",
              "* **Developmental changes:** Changes in the genes controlling embryonic development led to the shortening of the tail during human development. The tail's reduction may have been a consequence of changes in other aspects of our anatomy and development, such as changes to the spine and pelvis related to bipedalism.\n",
              "\n",
              "* **Genetic mutations:** Random genetic mutations that affected tail development could have spread through the population if they conferred even a slight advantage or were neutral in their effect.\n",
              "\n",
              "Essentially, the tail became a vestigial structure – a feature that has lost most or all of its original function through evolution.  While we don't have external tails, we do retain a vestigial tailbone (coccyx) at the base of our spine, which is evidence of our tailed ancestry.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "-------------"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "for candidate in response.candidates:\n",
        "  display(Markdown(candidate.content.parts[0].text))\n",
        "  display(Markdown(\"-------------\"))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2iUxBkmsPYvq"
      },
      "source": [
        "The response contains multiple full `Candidate` objects."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fOxbI7oSRNV_"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "response:\n",
              "GenerateContentResponse(\n",
              "    done=True,\n",
              "    iterator=None,\n",
              "    result=protos.GenerateContentResponse({\n",
              "      \"candidates\": [\n",
              "        {\n",
              "          \"content\": {\n",
              "            \"parts\": [\n",
              "              {\n",
              "                \"text\": \"Humans don't have tails because of evolution.  Our ancestors, primates like apes and monkeys, *did* have tails, but over millions of years of evolution, the tail gradually disappeared.  The exact reasons are complex and not fully understood, but several contributing factors are likely:\\n\\n* **Loss of function:**  As our ancestors adapted to arboreal (tree-dwelling) life and then to bipedalism (walking upright), the need for a tail for balance and locomotion decreased.  Tails are primarily used for balance and climbing;  our hands and upright posture made a tail less essential.\\n\\n* **Natural selection:** Individuals with shorter tails or rudimentary tails might have had a slight survival advantage. Perhaps shorter tails were less cumbersome while climbing, less likely to be injured, or offered less drag while running.  Over generations, these small advantages led to the selection and propagation of genes that resulted in shorter and shorter tails.  Eventually, the genes responsible for tail development became suppressed.\\n\\n* **Genetic mutations:**  Genetic mutations that impacted tail development could have occurred randomly. Some of these mutations may have provided advantages that were selected for, while others may have been neutral and simply became fixed in the population over time.\\n\\nIt's important to note that the complete loss of a tail wasn't a sudden event. It was a gradual process taking place over a very long evolutionary timescale.  We still retain some vestigial remnants of a tail in the coccyx (tailbone), which is a small, fused bone at the base of the spine.  This provides a structural support for our pelvis but is the only physical evidence of what was once a functional tail.\\n\"\n",
              "              }\n",
              "            ],\n",
              "            \"role\": \"model\"\n",
              "          },\n",
              "          \"finish_reason\": \"STOP\",\n",
              "          \"avg_logprobs\": -0.5109834474096607\n",
              "        },\n",
              "        {\n",
              "          \"content\": {\n",
              "            \"parts\": [\n",
              "              {\n",
              "                \"text\": \"Humans don't have tails because of evolution.  Our ancestors had tails, but over millions of years, the genes that controlled tail development were suppressed.  There are a few theories about why this happened:\\n\\n* **Loss of function:**  The tail may have become less advantageous.  While useful for balance and communication in arboreal primates, as our ancestors transitioned to walking upright and relied less on climbing, the tail's utility decreased.  The energy and resources used to maintain a tail became a disadvantage, so individuals with smaller tails or no tails had a slight evolutionary advantage.  Natural selection favored individuals with mutations that progressively reduced tail size.\\n\\n* **Developmental changes:** Changes in the genes controlling embryonic development led to the shortening of the tail during human development. The tail's reduction may have been a consequence of changes in other aspects of our anatomy and development, such as changes to the spine and pelvis related to bipedalism.\\n\\n* **Genetic mutations:** Random genetic mutations that affected tail development could have spread through the population if they conferred even a slight advantage or were neutral in their effect.\\n\\nEssentially, the tail became a vestigial structure \\u2013 a feature that has lost most or all of its original function through evolution.  While we don't have external tails, we do retain a vestigial tailbone (coccyx) at the base of our spine, which is evidence of our tailed ancestry.\\n\"\n",
              "              }\n",
              "            ],\n",
              "            \"role\": \"model\"\n",
              "          },\n",
              "          \"finish_reason\": \"STOP\",\n",
              "          \"index\": 1,\n",
              "          \"avg_logprobs\": -0.4221208685366534\n",
              "        }\n",
              "      ],\n",
              "      \"usage_metadata\": {\n",
              "        \"prompt_token_count\": 8,\n",
              "        \"candidates_token_count\": 626,\n",
              "        \"total_token_count\": 634\n",
              "      }\n",
              "    }),\n",
              ")"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g86txlIpW3FE"
      },
      "source": [
        "## Penalties\n",
        "\n",
        "The `002` models expose `penalty` arguments that let you affect the statistics of output tokens."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nw5ZwCCCUR2d"
      },
      "source": [
        "### Presence penalty"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8PeJ2GhBW537"
      },
      "source": [
        "The `presence_penalty` penalizes tokens that have already been used in the output, so it induces variety in the model's output. This is detectible if you count the unique words in the output.\n",
        "\n",
        "Here's a function to run a prompt a few times and report the fraction of unique words (words don't map perfectly to tokens but it's a simple way to see the effect)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ecolt7BNHUr2"
      },
      "outputs": [],
      "source": [
        "from statistics import mean"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1FbjPwM3ZIaH"
      },
      "outputs": [],
      "source": [
        "def unique_words(prompt, generation_config, N=10):\n",
        "  responses = []\n",
        "  vocab_fractions = []\n",
        "  for n in range(N):\n",
        "    model = genai.GenerativeModel(model_name)\n",
        "    response = model.generate_content(contents=prompt, generation_config=generation_config)\n",
        "    responses.append(response)\n",
        "\n",
        "    words = response.text.lower().split()\n",
        "    score = len(set(words))/len(words)\n",
        "    print(score)\n",
        "    vocab_fractions.append(score)\n",
        "\n",
        "  return vocab_fractions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n2PqpDPIZpr6"
      },
      "outputs": [],
      "source": [
        "prompt='Tell me a story'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6GTXnUVlYyY9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.6282973621103117\n",
            "0.58298755186722\n",
            "0.5496957403651116\n",
            "0.6046511627906976\n",
            "0.5792207792207792\n",
            "0.5846867749419954\n",
            "0.6169154228855721\n",
            "0.5696821515892421\n",
            "0.6074766355140186\n",
            "0.5928571428571429\n"
          ]
        }
      ],
      "source": [
        "# baseline\n",
        "v = unique_words(prompt, generation_config={})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZwFZBP68wKp3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.5916470724142091"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "mean(v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DlM8BVflUpAj"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.579088471849866\n",
            "0.5753968253968254\n",
            "0.6073903002309469\n",
            "0.5909090909090909\n",
            "0.6294277929155313\n",
            "0.6056644880174292\n",
            "0.5906313645621182\n",
            "0.5844748858447488\n",
            "0.6439232409381663\n",
            "0.6247288503253796\n"
          ]
        }
      ],
      "source": [
        "# the penalty encourages diversity in the oputput tokens.\n",
        "v = unique_words(prompt, generation_config=dict(presence_penalty=1.999))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HJo-_6_owQfu"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.6031635310990102"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "mean(v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "G-CztHdqTc__"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.6129032258064516\n",
            "0.5924170616113744\n",
            "0.5538461538461539\n",
            "0.5723981900452488\n",
            "0.5979166666666667\n",
            "0.5841836734693877\n",
            "0.6076923076923076\n",
            "0.5763097949886105\n",
            "0.5939849624060151\n",
            "0.5878524945770065\n"
          ]
        }
      ],
      "source": [
        "# a negative penalty discourages diversity in the output tokens.\n",
        "v = unique_words(prompt, generation_config=dict(presence_penalty=-1.999))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "x1eWaOAOwWrE"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.5879504531109223"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "mean(v)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nA7LkwvBMCjd"
      },
      "source": [
        "The `presence_penalty` has a small effect on the vocabulary statistics."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c_Anpg6YRPEx"
      },
      "source": [
        "### Frequency Penalty"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-m1BaNuXVoY"
      },
      "source": [
        "Frequency penalty is similar to the `presence_penalty` but  the penalty is multiplied by the number of times a token is used. This effect is much stronger than the `presence_penalty`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IZw9D5ZvWnmq"
      },
      "source": [
        "The easiest way to see that it works is to ask the model to do something repetitive. The model has to get creative while trying to complete the task."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ju9AIzBJ_-dR"
      },
      "outputs": [],
      "source": [
        "model = genai.GenerativeModel(model_name)\n",
        "response = model.generate_content(contents='please repeat \"Cat\" 50 times, 10 per line',\n",
        "                                  generation_config=dict(frequency_penalty=1.999))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O9eixhvwS6It"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cat Cat Cat Cat Cat Cat Cat Cat Cat\n",
            "Cat Cat Cat Cat Cat Cat Cat Cat\n",
            "Cat\n",
            "Cat  Cat  Cat  Cat  Cat  Cat   C at C at\n",
            "Cat Cat Cat Cat Cat Cat  Cat   Cat\n",
            "Cat\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6HMQhl2bJnk1"
      },
      "source": [
        "Since the frequency penalty accumulates with usage, it can have a much stronger effect on the output compared to the presence penalty.\n",
        "\n",
        "> Caution: Be careful with negative frequency penalties: A negative penalty makes a token more likely the more it's used. This positive feedback quickly leads the model to just repeat a common token until it hits the `max_output_tokens` limit (once it starts the model can't produce the `<STOP>` token)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "48HdqCUK2oT9"
      },
      "outputs": [],
      "source": [
        "response = model.generate_content(\n",
        "    prompt,\n",
        "    generation_config=genai.GenerationConfig(\n",
        "        max_output_tokens=400,\n",
        "        frequency_penalty=-2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2QpGQoHU27zK"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Elara, a clockmaker with hands stained the perpetual sepia of old gears, lived in a town perpetually shrouded in twilight.  Not a twilight of gloom, but a soft, ethereal half-light, where the sun hung eternally low, casting long, dancing shadows. The townsfolk, accustomed to this perpetual dusk, measured time not by the sun, but by the intricate clocks Elara crafted.\n",
              "\n",
              "Each clock was unique, a miniature universe housed in glass and brass.  One, shaped like a hummingbird, chirped the hours. Another, a miniature, swirling galaxy, displayed the phases of the moon.  Elara poured her soul into these creations, her life a delicate balance of gears and springs, as precise and predictable as her clocks, yet as mysterious as the twilight that enveloped her town.\n",
              "\n",
              "One day, a young, freckled boy named Finn arrived, his eyes bright with wonder. He'd journeyed from a sun-drenched land, a place Elara had only heard whispered tales of – a place where the sun blazed high and shadows were short and sharp. Finn carried with him a small, broken sundial, its gnomon snapped, its face cracked.\n",
              "\n",
              "Elara, captivated by Finn's story, took the sundial, its broken, sun-scorched face a stark contrast to her own creations. As she carefully repaired it, she found herself drawn to the simplicity of the sundial, the raw, unadorned way it measured the passage of time, unlike the ornate, meticulously crafted clocks of the twilight town.\n",
              "\n",
              "While the sundial mended the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "Markdown(response.text)  # the, the, the, ..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tmntZO3gNt6r"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<FinishReason.MAX_TOKENS: 2>"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response.candidates[0].finish_reason"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "New_in_002.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}