Skip to content

Commit

Permalink
initialize gh-pages branch
Browse files Browse the repository at this point in the history
  • Loading branch information
cpfiffer committed Oct 18, 2024
0 parents commit daddaa2
Show file tree
Hide file tree
Showing 9 changed files with 563 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*/__pycache__/
scp/entries
*.env
Empty file added .nojekyll
Empty file.
45 changes: 45 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# cursed

A collection of examples of or ideas for cursed language model code.
Please take a look at our ideas to submit your own, or add a pull request
to include a link to the idea (stored on your own repo) or directly
to our repo (you can put your code here).

## What is cursed code?

Cursed code is code that is creative, unexpected, foolish,
or otherwise intended simply for play.

Cursed code is an abomination, but the core value of
cursed code is that it lets us experiment with the
boundaries of a tool.

Language models and structured text are so incredibly
new that we haven't really begun to play with these boundaries.

The very best cursed code shows us something cool, interesting, and quirky about the tools that we use. It shows us how to build real things too, by demonstrating what it is actually possible to construct.

## Ideas for cursed language model code

These are all fair game! We'd absolutely love to see people tinker
with these.

- An [SCP](https://scp-wiki.wikidot.com/) entry generator.
[Here's a good example](https://scp-wiki.wikidot.com/scp-003) of
an SCP entry.
- Related to the SCP one,
[generating infohazards](https://scp-wiki.wikidot.com/scp-001).
Infohazards are ideas that are dangerous to know about.
Kind of a fun sci-fi style project.

## Examples of cursed language model code

- [worldsim](https://worldsim.nousresearch.com/) is a great example of cursed code. It lets you explore a world that doesn't exist in a way that invites constant curiousity. Amazing work by the Nous team.
- [Python.jl](https://pretalx.com/juliacon2024/talk/ZH3JN3/) in the Julia world overrode the Julia REPL to attempt to evaluate python expressions before Julia ones, which made this hideous (but cool) python + julia nightmare blend. It also surprisingly works very well.
- [this rust code](https://github.com/loyston500/cursed-codes/blob/main/rust/hello.rs) is gross as hell
- [rockstar](https://codewithrockstar.com/) is an esoteric programming language where you write code in the form of hair metal power ballads
- [websim](https://websim.ai/) is an AI-powered platform that randomly invents webpages. Type in a URL, get the language model's best guess of what that webpage would look like. It's a playground.
- [gcc-torture](https://github.com/llvm/llvm-test-suite/tree/main/SingleSource/Regression/C/gcc-c-torture) are tests playing with the C language to test compilers to the edge.
- [Dwitter](https://www.dwitter.net/) is a social network for building and sharing visual javascript demos limited to 140 characters. See the impressive creations crammed into so few characters, and play around with your own code!


71 changes: 71 additions & 0 deletions scp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# SCP entry generator

## Overview
This Python script generates SCP (Secure, Contain, Protect) entries and saves them to the `entries` directory. The script uses the `outlines` library for generating the SCP entries based on a predefined schema.

Special thanks to [all the contributors](https://scp-wiki.wikidot.com/authors-pages) of the SCP wiki over the years! This would never be possible without the creativity, passion, and dedication of the SCP community.

## Features

- Supports various SCP object classes: Safe, Euclid, Keter, Thaumiel, and more can be added as needed.
- Generates SCP entries with the following sections:
- Item Number
- Object Class
- Special Containment Procedures
- Description
- Addenda (optional)
- Notes (optional)
- Saves the generated SCP entries as text files in the `entries` directory.

## Dependencies
- `pydantic`: A data validation and settings management library.
- `typing`: Standard library module for type annotations.
- `enum`: Standard library module for creating enumeration types.
- `json`: Standard library module for working with JSON data.
- `outlines`: A library for generating text based on a provided schema.
- `os`: Standard library module for interacting with the operating system.

## Usage
1. Ensure you have the necessary dependencies installed:
```
pip install -r requirements.txt
```

2. Run the script to generate a new SCP entry.
```
python scp.py
```

3. The generated entry will be saved in the `entries` directory with the file name `SCP-{id}.txt`.

## Example
```python
# Using outlines locally
model = outlines.models.transformers(
"microsoft/Phi-3-mini-128k-instruct",
device="cpu"
)

# Make the generator
scp_generator = outlines.generate.json(model, SCP)

# Make a new one
entry = scp_generator("Make me an SCP entry.")
entry.save()
print(f"Entry saved to {entry.filepath()}")
```

This will generate a new SCP entry, save it to the `entries` directory, and print the file path.

## Customization

You can customize the SCP generator by editing the `classes.py` file.

- `scp_prompt()` can be modified to change the prompt for the SCP generator.
- Adding new `ObjectClass` values as needed.
- Modifying the `ContainmentProcedures`, `Description`, `Addendum`, and `Note` models to fit your desired SCP entry structure.
- Adjusting the text generation and formatting in the `save()` method of the `SCP` model.

## Contributing

If you find any issues or have suggestions for improvements, feel free to open an issue or submit a pull request.
155 changes: 155 additions & 0 deletions scp/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
"""
_______________________________
/ Don't want to self-host? \
\\ Try .json at http://dottxt.co /
-------------------------------
\\ ^__^
\\ (oo)\\_______
(__)\\ )\\/\
||----w |
|| ||
This code generate one SCP entry a day using the .txt API, .json.
The code is runnable without an API key, but may be useful when
the .json service is more widely available.
To run this code locally, please run `scp.py` instead.
"""

import hashlib
import json
import os
import time
import requests
from dotenv import load_dotenv
from typing import Optional
from requests.exceptions import HTTPError

load_dotenv(override=True)

# Create schema-to-js_id mapping
API_HOST = os.environ.get("DOTTXT_API_HOST", None)
API_KEY = os.environ.get("DOTTXT_API_KEY", None)

def check_api_key() -> None:
if not API_KEY:
raise ValueError("DOTTXT_API_KEY environment variable is not set")

def get_headers(api_key: Optional[str] = None) -> dict:
if api_key is None:
check_api_key()
api_key = API_KEY
return {"Authorization": f"Bearer {api_key}"}

SCHEMA_HASH_TO_COMPLETION_URL = {}

def to_hash(pydantic_class):
schema = pydantic_class.model_json_schema()
schema_string = json.dumps(schema)
return hashlib.sha256(schema_string.encode()).hexdigest()

def poll_status(url: str, api_key: Optional[str] = None) -> dict:
headers = get_headers(api_key)
while True:
status_res = requests.get(url, headers=headers)
status_json = status_res.json()
if status_res.status_code != 200 or status_json["status"] != "in_progress":
break
time.sleep(1)
return status_json

def get_schema_by_name(name: str, api_key: Optional[str] = None) -> Optional[dict]:
headers = get_headers(api_key)
try:
response = requests.get(f"https://{API_HOST}/v1/json-schemas", headers=headers)
response.raise_for_status()
schemas = response.json()['items']

for schema in schemas:
if schema['name'] == name:
return schema
return None
except HTTPError as e:
if e.response.status_code == 403:
raise ValueError("Authentication failed. Please check your API key.") from e
else:
raise
except Exception as e:
raise


def create_schema(schema: str, name: str, api_key: Optional[str] = None) -> dict:
data = {"name": name, "json_schema": schema}
headers = get_headers(api_key)
try:
response = requests.post(
f"https://{API_HOST}/v1/json-schemas",
headers=headers,
json=data
)
response.raise_for_status()
return response.json()
except HTTPError as e:
if e.response.status_code == 403:
raise ValueError("Authentication failed. Please check your API key.") from e
else:
raise
except Exception as e:
raise


def get_completion_endpoint(model_class, api_key: Optional[str] = None):
schema_hash = to_hash(model_class)

if schema_hash in SCHEMA_HASH_TO_COMPLETION_URL:
completion_url = SCHEMA_HASH_TO_COMPLETION_URL[schema_hash]
return completion_url

# Check next to see if the schema_has is already stored by checking
# GET https://api.dottxt.co/v1/json-schemas
schema_response = get_schema_by_name(schema_hash, api_key)

# If the schema exists poll the status and return the completion URL
if schema_response:
status_url = schema_response["status_url"]
final_status = poll_status(status_url, api_key)
completion_url = final_status["completion_url"]
if completion_url:
SCHEMA_HASH_TO_COMPLETION_URL[schema_hash] = completion_url
return completion_url

# Okay, we don't have a completion URL for this schema. Let's create it.
schema_string = json.dumps(model_class.model_json_schema())
schema_response = create_schema(schema_string, schema_hash, api_key)

# If we get here, we need to wait for the schema to be created
status_url = schema_response["status_url"]
final_status = poll_status(status_url, api_key)

completion_url = final_status["completion_url"]
if not completion_url:
raise ValueError(f"No completion URL available for schema: {schema_hash}")

SCHEMA_HASH_TO_COMPLETION_URL[schema_hash] = completion_url
return completion_url

def create_completion(
model_class,
prompt: str,
max_tokens: int = 31999,
api_key: Optional[str] = None
):
completion_url = get_completion_endpoint(model_class, api_key)
data = {"prompt": prompt, "max_tokens": max_tokens}
headers = get_headers(api_key)
completion_response = requests.post(completion_url, headers=headers, json=data)
completion_response.raise_for_status()

# get json
completion_response_json = completion_response.json()

# convert to pydantic model
model = model_class.model_validate_json(completion_response_json['data'])

return model
Loading

0 comments on commit daddaa2

Please sign in to comment.