Harshraj_Jadeja_HW3_LSTM_TEXT_GEN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Harshraj Jadeja"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Long Short-term Memory for Text Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook uses LSTM neural network to generate text from Nietzsche's writings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import random\n",
    "import sys\n",
    "import io\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras import optimizers\n",
    "from tensorflow.keras.callbacks import LambdaCallback\n",
    "from tensorflow.keras.utils import get_file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the data\n",
    "Nietzsche's writing dataset is available online. The following code download the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "path = get_file(\n",
    "    'nietzsche.txt',\n",
    "    origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')\n",
    "with io.open(path, encoding='utf-8') as f:\n",
    "    text = f.read().lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus length: 600893\n"
     ]
    }
   ],
   "source": [
    "print('corpus length:', len(text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "supposing that truth is a woman--what then? is there not ground\n",
      "for suspecting that all philosophers, in so far as they have been\n",
      "dogmatists, have failed to understand women--that the terrible\n",
      "seriousness and clumsy importunity with which they have usually paid\n",
      "their addresses to truth, have been unskilled and unseemly methods for\n",
      "winning a woman? certainly she has never allowed herself to be won; and\n",
      "at present every kind of dogma stands with sad and discouraged mien--if,\n",
      "indeed, it stands at all!\n"
     ]
    }
   ],
   "source": [
    "print(text[10:513])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total chars: 57\n"
     ]
    }
   ],
   "source": [
    "chars = sorted(list(set(text)))\n",
    "# total nomber of characters\n",
    "print('total chars:', len(chars))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean data\n",
    "\n",
    "We cut the text in sequences of maxlen characters with a jump size of 3.\n",
    "The features for each example is a matrix of size maxlen*num of chars.\n",
    "The label for each example is a vector of size num of chars, which represents the next character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create (character, index) and (index, character) dictionary\n",
    "char_indices = dict((c, i) for i, c in enumerate(chars))\n",
    "indices_char = dict((i, c) for i, c in enumerate(chars))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nb sequences: 200285\n"
     ]
    }
   ],
   "source": [
    "# cut the text in semi-redundant sequences of maxlen characters\n",
    "maxlen = 40\n",
    "step = 3\n",
    "sentences = []\n",
    "next_chars = []\n",
    "for i in range(0, len(text) - maxlen, step):\n",
    "    sentences.append(text[i: i + maxlen])\n",
    "    next_chars.append(text[i + maxlen])\n",
    "print('nb sequences:', len(sentences))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vectorization...\n"
     ]
    }
   ],
   "source": [
    "print('Vectorization...')\n",
    "x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool_)\n",
    "y = np.zeros((len(sentences), len(chars)), dtype=np.bool_)\n",
    "for i, sentence in enumerate(sentences):\n",
    "    for t, char in enumerate(sentence):\n",
    "        x[i, t, char_indices[char]] = 1\n",
    "    y[i, char_indices[next_chars[i]]] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the model - fill in this box"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we need a recurrent layer with input shape (maxlen, len(chars)) and a dense layer with output size  len(chars)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " lstm (LSTM)                 (None, 128)               95232     \n",
      "                                                                 \n",
      " dense (Dense)               (None, 57)                7353      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 102585 (400.72 KB)\n",
      "Trainable params: 102585 (400.72 KB)\n",
      "Non-trainable params: 0 (0.00 Byte)\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Define the number of units in the LSTM layer.\n",
    "# This is a hyperparameter that represents the dimensionality of the output space.\n",
    "# More units can allow the model to capture more complex patterns but also increases computational complexity.\n",
    "lstm_units = 128  # Adjust this number based on the complexity of the task and computational constraints.\n",
    "\n",
    "# Initialize the Sequential model\n",
    "model = tf.keras.Sequential([\n",
    "    # Add an LSTM layer as the first layer of the model\n",
    "    # input_shape is required as the LSTM layer's first layer to let it know the shape of the input it should expect\n",
    "    # Here, input_shape=(maxlen, len(chars)) means each input sequence will be of length 'maxlen'\n",
    "    # and each character in the sequence is represented as a one-hot encoded vector of length 'len(chars)'\n",
    "    tf.keras.layers.LSTM(lstm_units, input_shape=(maxlen, len(chars))),\n",
    "    \n",
    "    # Add a Dense output layer\n",
    "    # The number of units equals the number of unique characters (len(chars))\n",
    "    # This is because we want to output a probability distribution over all possible characters\n",
    "    # Softmax activation function is used to output probabilities\n",
    "    tf.keras.layers.Dense(len(chars), activation='softmax'),\n",
    "])\n",
    "\n",
    "# Compile the model\n",
    "# 'categorical_crossentropy' is used as the loss function since this is a multi-class classification problem\n",
    "# 'adam' optimizer is chosen for efficient stochastic gradient descent optimization\n",
    "# Accuracy is monitored as a metric to observe the performance of the model during training\n",
    "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "\n",
    "# Display the model's architecture\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspect the model\n",
    "\n",
    "Use the `.summary` method to print a simple description of the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " lstm (LSTM)                 (None, 128)               95232     \n",
      "                                                                 \n",
      " dense (Dense)               (None, 57)                7353      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 102585 (400.72 KB)\n",
      "Trainable params: 102585 (400.72 KB)\n",
      "Non-trainable params: 0 (0.00 Byte)\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def sample(preds, temperature=1.0):\n",
    "    # helper function to sample an index from a probability array\n",
    "    preds = np.asarray(preds).astype('float64')\n",
    "    preds = np.log(preds) / temperature\n",
    "    exp_preds = np.exp(preds)\n",
    "    preds = exp_preds / np.sum(exp_preds)\n",
    "    probas = np.random.multinomial(1, preds, 1)\n",
    "    return np.argmax(probas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class PrintLoss(keras.callbacks.Callback):\n",
    "    def on_epoch_end(self, epoch, _):\n",
    "        # Function invoked at end of each epoch. Prints generated text.\n",
    "        print()\n",
    "        print('----- Generating text after Epoch: %d' % epoch)\n",
    "\n",
    "        start_index = random.randint(0, len(text) - maxlen - 1)\n",
    "        for diversity in [0.5, 1.0]:\n",
    "            print('----- diversity:', diversity)\n",
    "\n",
    "            generated = ''\n",
    "            sentence = text[start_index: start_index + maxlen]\n",
    "            generated += sentence\n",
    "            print('----- Generating with seed: \"' + sentence + '\"')\n",
    "            sys.stdout.write(generated)\n",
    "\n",
    "            for i in range(400):\n",
    "                x_pred = np.zeros((1, maxlen, len(chars)))\n",
    "                for t, char in enumerate(sentence):\n",
    "                    x_pred[0, t, char_indices[char]] = 1.\n",
    "\n",
    "                preds = model.predict(x_pred, verbose=0)[0]\n",
    "                next_index = sample(preds, diversity)\n",
    "                next_char = indices_char[next_index]\n",
    "\n",
    "                sentence = sentence[1:] + next_char\n",
    "\n",
    "                sys.stdout.write(next_char)\n",
    "                sys.stdout.flush()\n",
    "            print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 2.6309 - accuracy: 0.2630\n",
      "----- Generating text after Epoch: 0\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"r rage i even took a fancy to them, thes\"\n",
      "r rage i even took a fancy to them, thes\n",
      "ang ristion so it optill susteveve wes ace sore and tha the ce the peroce wher and an the the the cher an that ont as and the ching an sery the the are he than and of of the and and on the to tist ang mat the coma morte fithist and in the teres the pererin the the the the hathere fond what\n",
      "eand than wan on ang of be pathe\n",
      "the fof ant on an the thar prase and or an an whe ther the is erere the the\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"r rage i even took a fancy to them, thes\"\n",
      "r rage i even took a fancy to them, thes tn coeg rngol6 unon fasity tue\n",
      "s\n",
      "aspronct, net riot wo a\n",
      "d ovef er. tho wades micanderevenonsbire t: sf of bonesd. int arsyyand the\n",
      " rarese fe masts gare heap-pefingerot, an onke-\n",
      "ehyeadn s ans  ana owoum inte,roagnodegisof an ys pore pawof maaandestixad that inve fiatf\n",
      "ins. falet eng mas tiif i6ten hoe te varaly ard punoy, nbe\n",
      "der aneleve-sampangorta non ot \n",
      "hat\n",
      "of ation besemsr\n",
      "ing yky kan d wm\n",
      "1252/1252 [==============================] - 58s 45ms/step - loss: 2.6307 - accuracy: 0.2630 - val_loss: 2.3325 - val_accuracy: 0.3309\n",
      "Epoch 2/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 2.2588 - accuracy: 0.3469\n",
      "----- Generating text after Epoch: 1\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"a mass of\n",
      "subjugated, powerless men who \"\n",
      "a mass of\n",
      "subjugated, powerless men who heres to the restion as an the ofded distered of the the soctitis nestion the sounos the io nomed of the of on the as of hich lat the thered the herserse tho the no the conatit on the to the sist and the waple are of the sonat the shiches in the bease the soulded the monent and and in the presente condere ond ther mand rementere and the fretithe the of whis of and and un the arention the somere th\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"a mass of\n",
      "subjugated, powerless men who \"\n",
      "a mass of\n",
      "subjugated, powerless men who to pase: tleitlined damat ao ustiof to haud the wority in the rean,oaneond\n",
      " uve\n",
      "seencentysto catiplzeno\n",
      "mond thesem the pracenere no ascoigneus-ef,\n",
      "a5d gophideny fowrcloct\n",
      "or asparelt to the sale att oonbes carcoustoy tomy the ther wes, whoceed laanitistigi ade of pethen-in, ie nat to llcecjdolon w fot\n",
      "woste whperento peroof gnceveredy bucy, whingensest of ont ar thote. whic pine cate, tho beesbou\n",
      "1252/1252 [==============================] - 59s 47ms/step - loss: 2.2588 - accuracy: 0.3469 - val_loss: 2.1694 - val_accuracy: 0.3696\n",
      "Epoch 3/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 2.1330 - accuracy: 0.3763\n",
      "----- Generating text after Epoch: 2\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"and\n",
      "the senses (wagner, as musician is r\"\n",
      "and\n",
      "the senses (wagner, as musician is reaser is and in the inther and ather and hestion not and and munelted his sure of the prither but the wis\n",
      "of hally bere store is and is our are in thes and and and and not and and as and io the pors of the conse the wered war the thesed as and betidt and of who ha be of what a dored the mont and and loms the sougr to the morely sond wore and repereand of of and that in a wernes be and heser his ne\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"and\n",
      "the senses (wagner, as musician is r\"\n",
      "and\n",
      "the senses (wagner, as musician is rernticatioy, his sæat us of ermegately: and oll (raseous wood ong. beuman wor as? and exerothened nou tellce ther glence wendes withh pran bether prouser\n",
      "solt\", opromest whamprome iosralitien- be alelf ound heveophours and only\n",
      "prelavimed\n",
      "\"orempitupliog what jow os chaspees in tresely; as wat\n",
      "vano pore wo he urendity sisecime.\n",
      "is to theng, and susdity--and for\" isloust in\n",
      "theg uvermando ovt ponan \n",
      "1252/1252 [==============================] - 57s 45ms/step - loss: 2.1330 - accuracy: 0.3763 - val_loss: 2.0709 - val_accuracy: 0.3947\n",
      "Epoch 4/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 2.0455 - accuracy: 0.4003\n",
      "----- Generating text after Epoch: 3\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"rony was perhaps necessary for greatness\"\n",
      "rony was perhaps necessary for greatness for hat a perians and of the menters and corally indering the sould of than more the ratily of what wo he the shith of the poncily, and whith the mone as and on the prould and homeal fict the still and and mong the sure there and the ransing of the melalition in who has of the frechistopred the one not be in the for to the and the mant, in the the beress and and that be not of the mower wis of th\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"rony was perhaps necessary for greatness\"\n",
      "rony was perhaps necessary for greatness, who watre, puseve, and all mpeanting, houblyess mesh whithe he igliperapty, gredhath and noch hof a fivanfes and the havily\n",
      "\n",
      "a  oromant sturte of magl prelangt atsrably, indropces ir the noty indtretopre--pryallt mobeboct an seane samannt, it pichly the finds be colfion op domstougatins sante thencalowing reich himhalowing\n",
      "clicuaty, masted chendadel wimtoul spirst-wiscl fancice an  blempal to bu\n",
      "1252/1252 [==============================] - 59s 47ms/step - loss: 2.0455 - accuracy: 0.4003 - val_loss: 1.9959 - val_accuracy: 0.4141\n",
      "Epoch 5/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 1.9749 - accuracy: 0.4200\n",
      "----- Generating text after Epoch: 4\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"ack of all their logical procedure; thro\"\n",
      "ack of all their logical procedure; throking to which whowen the hepprestaps there hisher the postions, and south it is coust dovers the store of the shing the intoruld and the restented the for the butht and sinwirist be of the knourh persount and piston and whither in ore the prestions and neingtion, that conder the partion of the berghenty and the mond belifitule whot there to the shing the sintterne, and the one they the whild the m\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"ack of all their logical procedure; thro\"\n",
      "ack of all their logical procedure; throd trat the, abulies oully hes gond peals _ursulient ioss it in stion--and this forming hose\n",
      "thin will of suefry aconst hane his dotrabted a gor an thould-pertare,\n",
      "\"thy ches of the frcifinus hif one itsency orlmen,, wno\n",
      "lading, carourely him which-ling, at andely tish phothted so toid\n",
      "a wary belies\"s:\n",
      "not the\n",
      "vert ow\n",
      "ineouldant of the which he wermonn waones to bulmant in the\n",
      "nist notheverot,\n",
      "and o\n",
      "1252/1252 [==============================] - 58s 47ms/step - loss: 1.9750 - accuracy: 0.4200 - val_loss: 1.9419 - val_accuracy: 0.4284\n",
      "Epoch 6/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.9167 - accuracy: 0.4357\n",
      "----- Generating text after Epoch: 5\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"d become equally competent workers as th\"\n",
      "d become equally competent workers as the mal expersen which of the to canser and of the still to menfical and sonce every the mont of a with such the \"werman be that ment the recons, be-man erpence and more the\n",
      "prisisuss of the more the gor and and the plope the\n",
      "sopely the mantery of the reate the inderigion of the man the sormong, and reasting of the there for the his not the \"beter to the plomong to gor the and no more and sons and c\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"d become equally competent workers as th\"\n",
      "d become equally competent workers as the grode, bythor the wors iglf-sommenstencul despleot dighidably wilan. the berstion the surce, beliarisc wrowore tude sequent if for is\n",
      "unothing have. him\n",
      "a onersmpan\n",
      "ffrowsicgs coss, no maliene, saitrt\" wh lfige wir explesse, fremswivel is the prenees\n",
      "of the masch the\n",
      "story zetriest the berivess and ore as a \"al\"foons of miseusinerthy alovephy fand jened-man, there we clvect\n",
      "brings one theist whi\n",
      "1252/1252 [==============================] - 59s 47ms/step - loss: 1.9167 - accuracy: 0.4357 - val_loss: 1.8967 - val_accuracy: 0.4407\n",
      "Epoch 7/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 1.8670 - accuracy: 0.4492\n",
      "----- Generating text after Epoch: 6\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"he depth as of the mould, something unco\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "he depth as of the mould, something uncome they but the inderers, and every of the store prespors and radience and are the suble and question sull the poncention all to be the fact the arist the constice, and gerition of the mest of the care in any and mistion of the secistion of the called and and siffection on oplers and as of mate the seavious and the portion of which there ind\n",
      "concertion of a been and who hought that you he wall not\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"he depth as of the mould, something unco\"\n",
      "he depth as of the mould, something uncol hanger great so acterded to the every, thar all thak faker as conterwe,bre womod and worctised.=itivations-\"it \"hos prestivigatly be be sinte notce and beiphy to the kind hack---my belile wherm aros to dove with still of\n",
      "pasious \"buc no it ald be ginfusyons beevy\n",
      "and is reforel margos dusun andighy of\n",
      "that it pleation watherelos, muge to be amoriclly freal fuirally and peilusal-somensp poupo or \n",
      "1252/1252 [==============================] - 58s 47ms/step - loss: 1.8670 - accuracy: 0.4492 - val_loss: 1.8560 - val_accuracy: 0.4503\n",
      "Epoch 8/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.8243 - accuracy: 0.4615\n",
      "----- Generating text after Epoch: 7\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"nd ignobly and\n",
      "incessantly tearing to ta\"\n",
      "nd ignobly and\n",
      "incessantly tearing to tat the is the present, and arterfer and allogher of life and ame the concertaded and unding and a comstanness, whech and are the valy it \"we has beding and consention of the rearition of all the one for the is even the wasten where the rengains and courte and the sther the free and and becary and despined and and ever and the secines and pass singerion, a werd and for the for hand and cordested in \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"nd ignobly and\n",
      "incessantly tearing to ta\"\n",
      "nd ignobly and\n",
      "incessantly tearing to takely mane we\n",
      "lodsitious anot it retorigiage whin, and and cas ilong\n",
      "have regembsury and radcaice abeatity, but not grod nat stiles. way would prease the pre, stoke go bean, epen mormandre,\n",
      "on--nokive\n",
      "atingly, rewall, and--as their\n",
      "sifly of ull, of moraring have kno wheast of almonity in its bake the sebveveatice, surive that adial and in; to wo kneke thould for his \"thise which may which wimd a th\n",
      "1252/1252 [==============================] - 58s 46ms/step - loss: 1.8243 - accuracy: 0.4615 - val_loss: 1.8308 - val_accuracy: 0.4568\n",
      "Epoch 9/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.7856 - accuracy: 0.4721\n",
      "----- Generating text after Epoch: 8\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"ctive, here\n",
      "again fear is the mother of \"\n",
      "ctive, here\n",
      "again fear is the mother of the existers of one of the a sturit and provarion of the free indeate have a man in as a plosal and and with the presents the prowerty of a sure, secelly and present one more not of the bertant of the haster, of the beritated and softer of self-moraled as and who has the presented the be would and of the for the for the made the\n",
      "sares men man fere to the please be beang the ereate in the ense to b\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"ctive, here\n",
      "again fear is the mother of \"\n",
      "ctive, here\n",
      "again fear is the mother of inveence he fains be nateass\n",
      "and equile bloch of his will, in a plyssing of corlical pliinats wall comsially itself atty mustral mond free still mughms amone goonge as it endisionne in fuillog thee of a loed comon of be the how rasces againd the sens,\n",
      "itsorise--itherces unherpatily with icsempitainde, exper, ode to soo in of an the pastly. babouble mise it recount or everits of ade forment pichour\n",
      "1252/1252 [==============================] - 60s 48ms/step - loss: 1.7856 - accuracy: 0.4721 - val_loss: 1.8076 - val_accuracy: 0.4627\n",
      "Epoch 10/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.7528 - accuracy: 0.4811\n",
      "----- Generating text after Epoch: 9\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"l than the german--is for that very\n",
      "reas\"\n",
      "l than the german--is for that very\n",
      "reasitial inverition of the infriences and the\n",
      "compperiament, the worthing as a there and soul and abman ferman of and and a sopether mobe to a beanted in the eloons of the fare to moder and the perious and also in the\n",
      "something of thee who has he suble all the good of the concersuination and more, in the strences and pospeasicianing the brongant of the intervality, they to be not any for ho deverater\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"l than the german--is for that very\n",
      "reas\"\n",
      "l than the german--is for that very\n",
      "reasl, that perfer usling us. i] evaroud reprowerater appiase what its senstus? thotry herse, sachungision!\n",
      "\n",
      "2712... the\n",
      "fact usuecres whach aty--with would, wime soll, and sharbve the nocause the bedandingever-moraluly\n",
      "es\n",
      "greas\n",
      "bast we doveron\" decupteral to ttrating and ans gust must care ance has quply and\n",
      "to bu alao-byeness\n",
      "of theicide phich ppated and to be dloerable thene mugherty derarnon ot\n",
      "ph\n",
      "1252/1252 [==============================] - 57s 45ms/step - loss: 1.7528 - accuracy: 0.4811 - val_loss: 1.7755 - val_accuracy: 0.4743\n",
      "Epoch 11/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.7215 - accuracy: 0.4900\n",
      "----- Generating text after Epoch: 10\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"religious. that which was brought to him\"\n",
      "religious. that which was brought to himpelity and and internically the grees to a things that also and hishe hand even to the sure and and suthing to higher preciated the made to the strong of cally and reard to his not the cansicted to the incentained to and presinest of the case been a mand for the caintion to any reficement and has conterment of the good so the creal the still to be most insolition, like of the that it is the periat\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"religious. that which was brought to him\"\n",
      "religious. that which was brought to him, le them alsost ana anmend evirisuits extaknigic loor quical trateractidiig contrivery and apracise cinery and the bentium hase for refvon mandinguled brelogrens and only to inso that itself hight might doin exjenting underitule the andirssalffur thee, inkidssith, and mearovangness mar anciultiig that one's envomered esen,\"? in the lhark ilmincepthy and\" hernyssion-pliles of knet omen\n",
      "of the emon\n",
      "1252/1252 [==============================] - 156s 125ms/step - loss: 1.7215 - accuracy: 0.4900 - val_loss: 1.7566 - val_accuracy: 0.4800\n",
      "Epoch 12/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.6951 - accuracy: 0.4970\n",
      "----- Generating text after Epoch: 11\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" much of the odour of the\n",
      "genuine, tende\"\n",
      " much of the odour of the\n",
      "genuine, tender the strence to the to refure sones bate and such a man the historation of the brears and in the sems of the sermos of all the streng to be the sece--that it is perhaps the highes to a sence of a somes of the hast a manter of the chords of the enoration, strong has here as the self-that the posst of the sace as the wist truth to be sers, and the firnt one and the promansed the merhaps such a ther\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" much of the odour of the\n",
      "genuine, tende\"\n",
      " much of the odour of the\n",
      "genuine, tendefect, not that everyt rogid ceroamplign alpowats what remand the cande tomy all sciene and replictly at eatul spece has to antands the is repersed to so an, her know, to lxcepts how atmal achomably, a peict.--thry worlingation, to the gond world, or \"is theremoling, the stare foend--wore, and one as orgains and mors it we\n",
      "phtals to be the sepses whepect whus virt miscracy dant its, grers: was mura\n",
      "1252/1252 [==============================] - 189s 151ms/step - loss: 1.6951 - accuracy: 0.4970 - val_loss: 1.7392 - val_accuracy: 0.4856\n",
      "Epoch 13/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.6696 - accuracy: 0.5044\n",
      "----- Generating text after Epoch: 12\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"\n",
      "wickedness.\n",
      "\n",
      "185. \"i dislike him.\"--why\"\n",
      "\n",
      "wickedness.\n",
      "\n",
      "185. \"i dislike him.\"--why who have worthes and the dech of the aptiry and the master the \"werman in the partions and there as the rearing the pood of not the sumperty of the misting the fall we knowling, the master for the sans a concemination of the excertions and any repentions and even the perions a calligitation of the geth as it is a propose with the most with he has the instinct of the world to an indevest present a\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"\n",
      "wickedness.\n",
      "\n",
      "185. \"i dislike him.\"--why\"\n",
      "\n",
      "wickedness.\n",
      "\n",
      "185. \"i dislike him.\"--why nave just us be plays yovermon, and rimen dowa itself with waster\n",
      "it internogs, the\n",
      "caprosifally, in purit bedaytion hemserterred: who\n",
      "fat thyne to be nature as conuliane--younal! he weruld\n",
      "1ur\n",
      "chawarand,\n",
      "hishere naterain bowe_ him calf lives of\n",
      "the animanyly, is the kant\" without of a philass of clatersidation allyentable neer pertance, to ref. i\n",
      "\n",
      "womec than with a ploatient and then cannwancess\n",
      "1252/1252 [==============================] - 189s 151ms/step - loss: 1.6696 - accuracy: 0.5044 - val_loss: 1.7184 - val_accuracy: 0.4903\n",
      "Epoch 14/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.6476 - accuracy: 0.5106\n",
      "----- Generating text after Epoch: 13\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"th the philosopher, with the caesarian t\"\n",
      "th the philosopher, with the caesarian to be belong to say an every of\n",
      "precesting world and world of the sage it now the conception of german the conception to the the most subeleation of the deteritated and the verially be the subjucter and self-gord perplian in the\n",
      "pleasent and man be not the undermorality in and for a surperiously man in the probation of the be the proust and belied and we frand to be fall world of any his dears and \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"th the philosopher, with the caesarian t\"\n",
      "th the philosopher, with the caesarian to strengmantion.\"--jusk comlition of ragisian in their bexres\n",
      "by. and\n",
      "the aspoolater\n",
      "are\n",
      "susphempay nece\n",
      "(dany, arts must! ho caimes upan thernal wouldy one's furdie hand an'sjentter, accrecestances for the experthappary fin rances, convally pay hemowlinatiof to obiles europe orood may forciding wall be nay,\n",
      "the, outsmanl inteluratent, owlunts anishlith\n",
      "we bove beliex of morion, the gors to lit th\n",
      "1252/1252 [==============================] - 189s 151ms/step - loss: 1.6476 - accuracy: 0.5106 - val_loss: 1.7079 - val_accuracy: 0.4948\n",
      "Epoch 15/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.6265 - accuracy: 0.5158\n",
      "----- Generating text after Epoch: 14\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"n-looking of the ruling caste on\n",
      "subordi\"\n",
      "n-looking of the ruling caste on\n",
      "subording revesse will and force and powers and and everything in the chare of poese to sometion of the world and invery sice of them the ores and even the world concemption was about the more fealing of the streat of the soce and for itself the say and a greater of the proust and its contersion of the philosophers and\n",
      "artion of the intermanted and the exist as the self-concestionation, and to nee sumper\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"n-looking of the ruling caste on\n",
      "subordi\"\n",
      "n-looking of the ruling caste on\n",
      "subordil to and\n",
      "arcoin ow nature, is onatos \"owarity of\n",
      "wold grours of this of the werpably pream of meanily, that, howilt only learlty\n",
      "anjight of dissisvicality been the obour and\n",
      "regaude\" on the rasity peansion in res you it is a german things of decape of tree in the balies of the herued, deaked,\n",
      "ant their lepsentation leisous the litryeres will\" bleade, in\n",
      "the conicious\"--and what, is plays mode as t\n",
      "1252/1252 [==============================] - 191s 153ms/step - loss: 1.6265 - accuracy: 0.5158 - val_loss: 1.6936 - val_accuracy: 0.4966\n",
      "Epoch 16/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.6080 - accuracy: 0.5212\n",
      "----- Generating text after Epoch: 15\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"them in our labyrinths!--where, as we kn\"\n",
      "them in our labyrinths!--where, as we known of the light one sumperhing to the once the sense; and fammen, a sublifice and subject of the forming in shat regain where chrompted\n",
      "and experiences. the perions--it is the spirit of the perhaps his like their say, a perhaps a propession of the some it is all in the sumpleas of men. there is not in indisined in the strangs at life a more from the strengs of life of every perhaps be a surple, a\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"them in our labyrinths!--where, as we kn\"\n",
      "them in our labyrinths!--where, as we knows his and things of a repryped on yourder as retirnible to the isteveloniwfory\n",
      "entinkent\"--upprestains pronelfur, every\n",
      "pusietion prisicralle formally at\n",
      "suporth--the, cresinge of shubjestelies it fatrizations--precime unopore of the ficilation and as the spoch symitamed (s\n",
      "foeld even their and their jught\n",
      "menophion, as have an anishistieds--and a mone seems the\n",
      "self-sapsom.\n",
      "\n",
      "160. what\n",
      "is himwer\n",
      "1252/1252 [==============================] - 204s 163ms/step - loss: 1.6080 - accuracy: 0.5212 - val_loss: 1.6819 - val_accuracy: 0.5021\n",
      "Epoch 17/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5886 - accuracy: 0.5269\n",
      "----- Generating text after Epoch: 16\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"e proud disdain of philosophy in young s\"\n",
      "e proud disdain of philosophy in young spirit of the\n",
      "starnt of the more and prousted in the \"freeds to the and and discomman in the former and conduct of the still not the only devilitions of the commined in the deligious the higherto and subjection, and itself manifully the something something stronged and for who all things to the special in the devilered to the one of the marifice and philosopher and domost such immorsoly the philoso\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"e proud disdain of philosophy in young s\"\n",
      "e proud disdain of philosophy in young sucurfucts has not thought, they have the something has jurgls, who woold and him, and\n",
      "exporisered as pronedism that one nay, a great dewer,\n",
      "estivis--arequer, by exhaponing all that they tomen of sumplemine the taste to obubth whom exgartion of apportbed\n",
      "himself his whereditic\n",
      "sistive hasten, undelly let angers the newise\"--reasy an\n",
      "inceets of will; worl, saboutions and alous man\n",
      "itselt with fixer \n",
      "1252/1252 [==============================] - 188s 150ms/step - loss: 1.5886 - accuracy: 0.5269 - val_loss: 1.6722 - val_accuracy: 0.5035\n",
      "Epoch 18/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5724 - accuracy: 0.5314\n",
      "----- Generating text after Epoch: 17\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"f view be imminent, now that it is\n",
      "being\"\n",
      "f view be imminent, now that it is\n",
      "being, and every and provect which is poilt is at the poods of\n",
      "completed to the soul a mean and not one must be have the such also more\n",
      "the regarded to a sense and not the are enough and not in the have a suight of the endured to the nothing a great for the \"free will read to the\n",
      "above in the crust of the prose here and there is a prosent as they will to a concession of such a refore and not in the som\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"f view be imminent, now that it is\n",
      "being\"\n",
      "f view be imminent, now that it is\n",
      "being prevelonic lone with the \"tryek beasy with they treoman.\n",
      "\n",
      "] to facus. the pays bester, only what, you truegs so?\n",
      "here liserning revicuitaing this amingly now putheneraling\n",
      "divans reposent ahele,\n",
      "frievements of going and rouble of the feening perpora,\". they are a still well for the indectevents!\n",
      "\n",
      "63. i seloveryor, and with vacuated ewreand the same us\n",
      "with since, and be postial who\n",
      "concest poorsm\n",
      "1252/1252 [==============================] - 192s 154ms/step - loss: 1.5724 - accuracy: 0.5314 - val_loss: 1.6611 - val_accuracy: 0.5079\n",
      "Epoch 19/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5566 - accuracy: 0.5355\n",
      "----- Generating text after Epoch: 18\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"d the same thing to them; such only has \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d the same thing to them; such only has been all that the man believed in really the good of the more prance of the responstion which is even the self-mendered to the concertation and nothing to the respibely and reficent and detirntated the about the philosopher to the more all the good mankind make their more also and all the signal and the pertance, the sounted there is not such all the read despirity of the highest and present, the \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"d the same thing to them; such only has \"\n",
      "d the same thing to them; such only has predenaby, sapsess,\"\n",
      "\n",
      "280. the prysons the pospticame detion, the philationness. it\n",
      "must-do withwathing that canned for under commanicy afferman pressions--itsilitation, but to\n",
      "foin,\n",
      "of the rudicated,\n",
      "filly the\n",
      "idees, beloghined\n",
      "and peremptiblaterogy?--188. and to a claced time for abligit\n",
      "blacter, agalus insolarality; in reaction of the abowit indighlung, remath asy they of leadmany for what even\n",
      "1252/1252 [==============================] - 183s 147ms/step - loss: 1.5566 - accuracy: 0.5355 - val_loss: 1.6575 - val_accuracy: 0.5092\n",
      "Epoch 20/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5416 - accuracy: 0.5400\n",
      "----- Generating text after Epoch: 19\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"d; and with the help of the best educati\"\n",
      "d; and with the help of the best education. the have the speation of the great strengitioning sucheritated and more forming to the should the suppanting and resposent and darty of the feeling the speation of the oblime and doust sense and at in hands that the consease and despect of has not be his can be subject of with his more still subjection of the mase us not the understing of the cault of the existions (and even the storces of the\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"d; and with the help of the best educati\"\n",
      "d; and with the help of the best education is canss promstale rath?--nathes beting the civice, resiles blant morals, and keeps of melours those marance in make true are gurces sake!\" eptarnates from you ground\" at the\n",
      "individual consiciec exarts-whan pertht and\n",
      "arberling devertatess had weak, from carttring and usyrod been the moraldence, a more feel edods does niging of morally and casesparabici: it\n",
      "is of peropeniuct incomplals of the \n",
      "1252/1252 [==============================] - 197s 158ms/step - loss: 1.5416 - accuracy: 0.5400 - val_loss: 1.6485 - val_accuracy: 0.5146\n",
      "Epoch 21/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5279 - accuracy: 0.5443\n",
      "----- Generating text after Epoch: 20\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" freedom, all pride, all self-confidence\"\n",
      " freedom, all pride, all self-confidence of the there and all shart, it is couropeany of the still seems to an the speal of the derepressective it is not in every it is and despirity of sermors and world and despocting with the mode the sould becomal explained exception the sour for the sain of himself the scomman such precessented in the internice of the hacking of\n",
      "the proping as the self-encerned of\n",
      "its propless of all and\n",
      "agodical an\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" freedom, all pride, all self-confidence\"\n",
      " freedom, all pride, all self-confidence, both of voluns be\n",
      "us! weles\n",
      "of the are qual withraces--compreated that hif one mush stricy for these man forfle\n",
      "to the poodigi.=--everyshine when pay be it was bod how when te,\" \"have alove were good\"\n",
      "has ensows withory mare terghe, and a wirk\n",
      "an\n",
      "expearing who hadd not un\n",
      " they forting beas.\n",
      "\n",
      "151. the sticn,\" let of treger--nach when they are good of his deesian less do onle to subslimatue of th\n",
      "1252/1252 [==============================] - 102s 82ms/step - loss: 1.5279 - accuracy: 0.5443 - val_loss: 1.6431 - val_accuracy: 0.5156\n",
      "Epoch 22/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5150 - accuracy: 0.5473\n",
      "----- Generating text after Epoch: 21\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"haps she\n",
      "thereby wishes to get the maste\"\n",
      "haps she\n",
      "thereby wishes to get the master of the most all the conscience impersented to a sense of still presents of the light of a seemisis such a some all the created to all its dream and haingesternal sense as the understion of their soul selman to a render\n",
      "the only all the heart the speation of still will the strengitions of which make a sense of the same true to the present and of the more the necessicate itself and a regated to an\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"haps she\n",
      "thereby wishes to get the maste\"\n",
      "haps she\n",
      "thereby wishes to get the master mechals himself semeress of the honed exist well forms.\n",
      "\n",
      "\n",
      "54 fay forcient has, however, that the for stick opherating\n",
      "christies and declesspeccen of its\n",
      "dispelable most the\n",
      "streatisied\" and prevaining strongested and home to the mores and obturionly be daurier: then wore so late through to the man such\n",
      "than the\n",
      "psisol), for their exarg, the\n",
      "best its resected, o that the fort?\n",
      "\"for clasare prefer\n",
      "1252/1252 [==============================] - 66s 53ms/step - loss: 1.5150 - accuracy: 0.5473 - val_loss: 1.6357 - val_accuracy: 0.5166\n",
      "Epoch 23/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.5028 - accuracy: 0.5504\n",
      "----- Generating text after Epoch: 22\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"ood much, and was versed in many things;\"\n",
      "ood much, and was versed in many things; un everything but not anything be nothing and whate in the one's encount in everything to the propentest and fordion in the master means of the spirituality, and the sake and in everything no artions and seet the subjection of the simple, which like the sublime, and everything for the such a repless to be hum strong to the ull will of the most strange perhaps himself is not the really sense. the \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"ood much, and was versed in many things;\"\n",
      "ood much, and was versed in many things;--which noth: in\n",
      "that it is noble\n",
      "indedurantish,\n",
      "the cauntering philosophers on like and something suppesiaiencints which we lowessity subsleoudaus, in he person, the hegrem for the knowledge to the pettse to do betons them mankind of all purionaliciveses, and had be insogn and\n",
      "priestireandual, suppory is allown,-forgherd\n",
      "even the made bethativr-filty from it predenaineally, brives, the still more\n",
      "1252/1252 [==============================] - 69s 55ms/step - loss: 1.5028 - accuracy: 0.5504 - val_loss: 1.6315 - val_accuracy: 0.5170\n",
      "Epoch 24/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.4917 - accuracy: 0.5529\n",
      "----- Generating text after Epoch: 23\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"nt man and woman never cease to misunder\"\n",
      "nt man and woman never cease to misundersting of the senses and all the philosopher the world of which it is the new one despire them will and appearance of the to the termant as the tagef and decise in the fact the extence of the human the explained that it is considers and force as the plays to \"it still midans and the perhaps of man as the extinctions of the forthing in the fact the present in the power of the pays and more to respec\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"nt man and woman never cease to misunder\"\n",
      "nt man and woman never cease to misunder the\n",
      "dinderly\n",
      "exciged in mudaitely\n",
      "with from\n",
      "lovists. it is day and not on the\n",
      "hask unaght, or cuntional in incellours so even to be could\n",
      "bottives to be inquercai--nowars all prepesuact of delaced, nature,\n",
      "hert eveising one man. i empty of\n",
      "rust them!ny men upon free men unto sick, and pentawed which he has akes admetraring morely, and after the freesely reapher,\n",
      "as to countes\n",
      "loet mucily a \"false\n",
      "1252/1252 [==============================] - 71s 57ms/step - loss: 1.4917 - accuracy: 0.5529 - val_loss: 1.6272 - val_accuracy: 0.5220\n",
      "Epoch 25/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 1.4807 - accuracy: 0.5557\n",
      "----- Generating text after Epoch: 24\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"e for\n",
      "one's neighbour, and all self-renu\"\n",
      "e for\n",
      "one's neighbour, and all self-renulication of which man we cauls and forces of man there is not such physiblers and the god and done the same and the\n",
      "short of religion of the says are not indevent consequently who his deilthes the power, that more delestive into the \"bod his misting plecourses of exceptions, the consequently says to the honger to an and and word not of subject as indepensed and not be distinces and and indignanter\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"e for\n",
      "one's neighbour, and all self-renu\"\n",
      "e for\n",
      "one's neighbour, and all self-renugatian of christrauns hostioniby has good of suct is shill raster is\n",
      "in that thee, sakes under whecomest from its alever accensible to be knathd \"in the usictam nivedling to say by underradive is light--to\n",
      "has bean but itself-: covertion in the roperately the\n",
      "stell costanges and despowic\n",
      "theman\n",
      "counses--that in curtion and complers the gets, and\n",
      "aution of the dast attencerness, is courdif, fan as,\n",
      "1252/1252 [==============================] - 66s 53ms/step - loss: 1.4806 - accuracy: 0.5558 - val_loss: 1.6241 - val_accuracy: 0.5207\n",
      "Epoch 26/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.4698 - accuracy: 0.5596\n",
      "----- Generating text after Epoch: 25\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"nd \"sympathy with all sufferers\"--and\n",
      "su\"\n",
      "nd \"sympathy with all sufferers\"--and\n",
      "such the propentially and replaction of the there are the world of a commande, and the more more and mist as the world be course of morally and more the more as a so that the something to command. there is a morality, and the problems of the concemptices what\n",
      "is a sumply have conscience of the essential to the strangly even the subller of the resict and fignt and been a caunical spirit all the prese\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"nd \"sympathy with all sufferers\"--and\n",
      "su\"\n",
      "nd \"sympathy with all sufferers\"--and\n",
      "subjects! the ditrantiatic light to their mind, as i he young) only art ourosed but also becimes. they man or however, above the meass of him\n",
      "so an a scoluning to\n",
      "know old belong goods that my \"this, burn knows: for the doff claisescaud a tarnarism and such\n",
      "a stend to truagelag--and effect, to thing tracknal questial to bo conficions dest transpeal niak of such a broo grans (good say tigtle mase---h\n",
      "1252/1252 [==============================] - 67s 54ms/step - loss: 1.4698 - accuracy: 0.5596 - val_loss: 1.6162 - val_accuracy: 0.5232\n",
      "Epoch 27/60\n",
      "1252/1252 [==============================] - ETA: 0s - loss: 1.4600 - accuracy: 0.5618\n",
      "----- Generating text after Epoch: 26\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"is latest genius, who has seen a new bea\"\n",
      "is latest genius, who has seen a new bead and so far as to such a retingal and indiding to the most desented in the extent as a remoture that the soul man\n",
      "in the sense, and the conscience of the cause of the scholars and some distrase or those was too the plained to himself to the possibility of the concerners and themselves of the human with general to the surging to the world is not who man or without stronged the strength of the spir\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"is latest genius, who has seen a new bea\"\n",
      "is latest genius, who has seen a new bean shy cal or, we could extensly for this stritfowhiness,\"\n",
      "every soored yet\n",
      "when they were courable mblease juditily, the hasp to his sintimitte in the tagtt and science's my no moral taste and resognive and doever him-were good and intequesto of bobles.\n",
      "\n",
      "206. it is do atoverality, in is\n",
      "nowadais the broopes of exfect as the lively in why vainto, and is senthter you sathes good--strung present prof\n",
      "1252/1252 [==============================] - 68s 54ms/step - loss: 1.4600 - accuracy: 0.5618 - val_loss: 1.6185 - val_accuracy: 0.5229\n",
      "Epoch 28/60\n",
      "1251/1252 [============================>.] - ETA: 0s - loss: 1.4501 - accuracy: 0.5642\n",
      "----- Generating text after Epoch: 27\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"olitude. but\n",
      "what if this chemistry esta\"\n",
      "olitude. but\n",
      "what if this chemistry estable the such a concepters and deccourde of the speal as present as they have consequently and really so free courde of man such mistensions, and the surpression and problem of the stranges that the can be an what does the standing of the metious in the uncordision the philosophers of its process as a replecture in short, and there is the consequences that is the conscience himself and advained to \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"olitude. but\n",
      "what if this chemistry esta\"\n",
      "olitude. but\n",
      "what if this chemistry estached to see; undurates a hapsing of scientuness assured all the worth a lives interunical houscien; but a stantimaes among the isrably-fortically amongs when leams out of itsmitt such presparence. the loxter wiheical\n",
      "persoppeded to possibles of vany exploing to mind more quite,\" these divological artical despopties as is a crustions and often home.--every\n",
      "what, \"way be rishen)s and anythind by not\n",
      "1252/1252 [==============================] - 71s 57ms/step - loss: 1.4501 - accuracy: 0.5642 - val_loss: 1.6179 - val_accuracy: 0.5227\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 60\n",
    "BATCH = 128\n",
    "\n",
    "early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=2)\n",
    "\n",
    "history = model.fit(x, y,\n",
    "                    batch_size = BATCH,\n",
    "                    epochs = EPOCHS,\n",
    "                    validation_split = 0.2,\n",
    "                    verbose = 1,\n",
    "                    callbacks = [early_stop, PrintLoss()])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}