-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit daddaa2
Showing
9 changed files
with
563 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
*/__pycache__/ | ||
scp/entries | ||
*.env |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# cursed | ||
|
||
A collection of examples of or ideas for cursed language model code. | ||
Please take a look at our ideas to submit your own, or add a pull request | ||
to include a link to the idea (stored on your own repo) or directly | ||
to our repo (you can put your code here). | ||
|
||
## What is cursed code? | ||
|
||
Cursed code is code that is creative, unexpected, foolish, | ||
or otherwise intended simply for play. | ||
|
||
Cursed code is an abomination, but the core value of | ||
cursed code is that it lets us experiment with the | ||
boundaries of a tool. | ||
|
||
Language models and structured text are so incredibly | ||
new that we haven't really begun to play with these boundaries. | ||
|
||
The very best cursed code shows us something cool, interesting, and quirky about the tools that we use. It shows us how to build real things too, by demonstrating what it is actually possible to construct. | ||
|
||
## Ideas for cursed language model code | ||
|
||
These are all fair game! We'd absolutely love to see people tinker | ||
with these. | ||
|
||
- An [SCP](https://scp-wiki.wikidot.com/) entry generator. | ||
[Here's a good example](https://scp-wiki.wikidot.com/scp-003) of | ||
an SCP entry. | ||
- Related to the SCP one, | ||
[generating infohazards](https://scp-wiki.wikidot.com/scp-001). | ||
Infohazards are ideas that are dangerous to know about. | ||
Kind of a fun sci-fi style project. | ||
|
||
## Examples of cursed language model code | ||
|
||
- [worldsim](https://worldsim.nousresearch.com/) is a great example of cursed code. It lets you explore a world that doesn't exist in a way that invites constant curiousity. Amazing work by the Nous team. | ||
- [Python.jl](https://pretalx.com/juliacon2024/talk/ZH3JN3/) in the Julia world overrode the Julia REPL to attempt to evaluate python expressions before Julia ones, which made this hideous (but cool) python + julia nightmare blend. It also surprisingly works very well. | ||
- [this rust code](https://github.com/loyston500/cursed-codes/blob/main/rust/hello.rs) is gross as hell | ||
- [rockstar](https://codewithrockstar.com/) is an esoteric programming language where you write code in the form of hair metal power ballads | ||
- [websim](https://websim.ai/) is an AI-powered platform that randomly invents webpages. Type in a URL, get the language model's best guess of what that webpage would look like. It's a playground. | ||
- [gcc-torture](https://github.com/llvm/llvm-test-suite/tree/main/SingleSource/Regression/C/gcc-c-torture) are tests playing with the C language to test compilers to the edge. | ||
- [Dwitter](https://www.dwitter.net/) is a social network for building and sharing visual javascript demos limited to 140 characters. See the impressive creations crammed into so few characters, and play around with your own code! | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# SCP entry generator | ||
|
||
## Overview | ||
This Python script generates SCP (Secure, Contain, Protect) entries and saves them to the `entries` directory. The script uses the `outlines` library for generating the SCP entries based on a predefined schema. | ||
|
||
Special thanks to [all the contributors](https://scp-wiki.wikidot.com/authors-pages) of the SCP wiki over the years! This would never be possible without the creativity, passion, and dedication of the SCP community. | ||
|
||
## Features | ||
|
||
- Supports various SCP object classes: Safe, Euclid, Keter, Thaumiel, and more can be added as needed. | ||
- Generates SCP entries with the following sections: | ||
- Item Number | ||
- Object Class | ||
- Special Containment Procedures | ||
- Description | ||
- Addenda (optional) | ||
- Notes (optional) | ||
- Saves the generated SCP entries as text files in the `entries` directory. | ||
|
||
## Dependencies | ||
- `pydantic`: A data validation and settings management library. | ||
- `typing`: Standard library module for type annotations. | ||
- `enum`: Standard library module for creating enumeration types. | ||
- `json`: Standard library module for working with JSON data. | ||
- `outlines`: A library for generating text based on a provided schema. | ||
- `os`: Standard library module for interacting with the operating system. | ||
|
||
## Usage | ||
1. Ensure you have the necessary dependencies installed: | ||
``` | ||
pip install -r requirements.txt | ||
``` | ||
|
||
2. Run the script to generate a new SCP entry. | ||
``` | ||
python scp.py | ||
``` | ||
|
||
3. The generated entry will be saved in the `entries` directory with the file name `SCP-{id}.txt`. | ||
|
||
## Example | ||
```python | ||
# Using outlines locally | ||
model = outlines.models.transformers( | ||
"microsoft/Phi-3-mini-128k-instruct", | ||
device="cpu" | ||
) | ||
|
||
# Make the generator | ||
scp_generator = outlines.generate.json(model, SCP) | ||
|
||
# Make a new one | ||
entry = scp_generator("Make me an SCP entry.") | ||
entry.save() | ||
print(f"Entry saved to {entry.filepath()}") | ||
``` | ||
|
||
This will generate a new SCP entry, save it to the `entries` directory, and print the file path. | ||
|
||
## Customization | ||
|
||
You can customize the SCP generator by editing the `classes.py` file. | ||
|
||
- `scp_prompt()` can be modified to change the prompt for the SCP generator. | ||
- Adding new `ObjectClass` values as needed. | ||
- Modifying the `ContainmentProcedures`, `Description`, `Addendum`, and `Note` models to fit your desired SCP entry structure. | ||
- Adjusting the text generation and formatting in the `save()` method of the `SCP` model. | ||
|
||
## Contributing | ||
|
||
If you find any issues or have suggestions for improvements, feel free to open an issue or submit a pull request. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,155 @@ | ||
""" | ||
_______________________________ | ||
/ Don't want to self-host? \ | ||
\\ Try .json at http://dottxt.co / | ||
------------------------------- | ||
\\ ^__^ | ||
\\ (oo)\\_______ | ||
(__)\\ )\\/\ | ||
||----w | | ||
|| || | ||
This code generate one SCP entry a day using the .txt API, .json. | ||
The code is runnable without an API key, but may be useful when | ||
the .json service is more widely available. | ||
To run this code locally, please run `scp.py` instead. | ||
""" | ||
|
||
import hashlib | ||
import json | ||
import os | ||
import time | ||
import requests | ||
from dotenv import load_dotenv | ||
from typing import Optional | ||
from requests.exceptions import HTTPError | ||
|
||
load_dotenv(override=True) | ||
|
||
# Create schema-to-js_id mapping | ||
API_HOST = os.environ.get("DOTTXT_API_HOST", None) | ||
API_KEY = os.environ.get("DOTTXT_API_KEY", None) | ||
|
||
def check_api_key() -> None: | ||
if not API_KEY: | ||
raise ValueError("DOTTXT_API_KEY environment variable is not set") | ||
|
||
def get_headers(api_key: Optional[str] = None) -> dict: | ||
if api_key is None: | ||
check_api_key() | ||
api_key = API_KEY | ||
return {"Authorization": f"Bearer {api_key}"} | ||
|
||
SCHEMA_HASH_TO_COMPLETION_URL = {} | ||
|
||
def to_hash(pydantic_class): | ||
schema = pydantic_class.model_json_schema() | ||
schema_string = json.dumps(schema) | ||
return hashlib.sha256(schema_string.encode()).hexdigest() | ||
|
||
def poll_status(url: str, api_key: Optional[str] = None) -> dict: | ||
headers = get_headers(api_key) | ||
while True: | ||
status_res = requests.get(url, headers=headers) | ||
status_json = status_res.json() | ||
if status_res.status_code != 200 or status_json["status"] != "in_progress": | ||
break | ||
time.sleep(1) | ||
return status_json | ||
|
||
def get_schema_by_name(name: str, api_key: Optional[str] = None) -> Optional[dict]: | ||
headers = get_headers(api_key) | ||
try: | ||
response = requests.get(f"https://{API_HOST}/v1/json-schemas", headers=headers) | ||
response.raise_for_status() | ||
schemas = response.json()['items'] | ||
|
||
for schema in schemas: | ||
if schema['name'] == name: | ||
return schema | ||
return None | ||
except HTTPError as e: | ||
if e.response.status_code == 403: | ||
raise ValueError("Authentication failed. Please check your API key.") from e | ||
else: | ||
raise | ||
except Exception as e: | ||
raise | ||
|
||
|
||
def create_schema(schema: str, name: str, api_key: Optional[str] = None) -> dict: | ||
data = {"name": name, "json_schema": schema} | ||
headers = get_headers(api_key) | ||
try: | ||
response = requests.post( | ||
f"https://{API_HOST}/v1/json-schemas", | ||
headers=headers, | ||
json=data | ||
) | ||
response.raise_for_status() | ||
return response.json() | ||
except HTTPError as e: | ||
if e.response.status_code == 403: | ||
raise ValueError("Authentication failed. Please check your API key.") from e | ||
else: | ||
raise | ||
except Exception as e: | ||
raise | ||
|
||
|
||
def get_completion_endpoint(model_class, api_key: Optional[str] = None): | ||
schema_hash = to_hash(model_class) | ||
|
||
if schema_hash in SCHEMA_HASH_TO_COMPLETION_URL: | ||
completion_url = SCHEMA_HASH_TO_COMPLETION_URL[schema_hash] | ||
return completion_url | ||
|
||
# Check next to see if the schema_has is already stored by checking | ||
# GET https://api.dottxt.co/v1/json-schemas | ||
schema_response = get_schema_by_name(schema_hash, api_key) | ||
|
||
# If the schema exists poll the status and return the completion URL | ||
if schema_response: | ||
status_url = schema_response["status_url"] | ||
final_status = poll_status(status_url, api_key) | ||
completion_url = final_status["completion_url"] | ||
if completion_url: | ||
SCHEMA_HASH_TO_COMPLETION_URL[schema_hash] = completion_url | ||
return completion_url | ||
|
||
# Okay, we don't have a completion URL for this schema. Let's create it. | ||
schema_string = json.dumps(model_class.model_json_schema()) | ||
schema_response = create_schema(schema_string, schema_hash, api_key) | ||
|
||
# If we get here, we need to wait for the schema to be created | ||
status_url = schema_response["status_url"] | ||
final_status = poll_status(status_url, api_key) | ||
|
||
completion_url = final_status["completion_url"] | ||
if not completion_url: | ||
raise ValueError(f"No completion URL available for schema: {schema_hash}") | ||
|
||
SCHEMA_HASH_TO_COMPLETION_URL[schema_hash] = completion_url | ||
return completion_url | ||
|
||
def create_completion( | ||
model_class, | ||
prompt: str, | ||
max_tokens: int = 31999, | ||
api_key: Optional[str] = None | ||
): | ||
completion_url = get_completion_endpoint(model_class, api_key) | ||
data = {"prompt": prompt, "max_tokens": max_tokens} | ||
headers = get_headers(api_key) | ||
completion_response = requests.post(completion_url, headers=headers, json=data) | ||
completion_response.raise_for_status() | ||
|
||
# get json | ||
completion_response_json = completion_response.json() | ||
|
||
# convert to pydantic model | ||
model = model_class.model_validate_json(completion_response_json['data']) | ||
|
||
return model |
Oops, something went wrong.