From da45c4daefba1c5ae219a64df127c96f4fc18cd7 Mon Sep 17 00:00:00 2001 From: Caleb Courier Date: Wed, 7 Feb 2024 17:18:39 -0800 Subject: [PATCH] contributing and readme swap --- CONTRIBUTING.md | 38 ++++++++++++++++ README.md | 115 ++++++++++++++++++++++++++++++++---------------- 2 files changed, 115 insertions(+), 38 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..1d1dd29 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,38 @@ +# Guardrails Validator Template + +## How to create a Guardrails Validator +- On the top right of the page, click "Use this template", select "create a new repository" and set a name for the package. See [Naming Conventions](#naming-conventions) below. +- Clone down the new repository. +- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator + - Make sure that the class still inherits from `Validator` and has the `register_validator` annotation. + - Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type. +- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate +- Perform a self install with `make dev` or `pip install -e ".[dev]"` +- Locally test the validator with the [test instructions below](#testing-and-using-your-validator) +- Modify the README and follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md) + +* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies + +### Naming Conventions +1. Avoid using `is` and `bug` +2. Use snake_case: i.e. `_` to separate words. e.g. valid_address +3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string. +4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)` + +### Testing and using your validator +- Open [test/test-validator.py](test/test-validator.py) to test your new validator +- Import your new validator and modify `ValidatorTestObject` accordingly +- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly +- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object +- Write advanced tests for failures, etc. + +## Upload your validator to the validator hub +- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows: + - Update the `name` field to the name of your validator + - Update the `description` field to a short description of your validator + - Update the `authors` field to your name and email + - Add/update the `dependencies` field to include all dependencies your validator needs. +- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly. +- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories. + - e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file. +- Ensure that there are no other dependencies or any additional steps required to run your validator. diff --git a/README.md b/README.md index 77c4774..491aa64 100644 --- a/README.md +++ b/README.md @@ -1,38 +1,77 @@ -# Guardrails Validator Template - -## How to create a Guardrails Validator -- On the top right of the page, click "Use this template", select "create a new repository" and set a name for the package. See [Naming Conventions](#naming-conventions) below. -- Clone down the new repository. -- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator - - Make sure that the class still inherits from `Validator` and has the `register_validator` annotation. - - Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type. -- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate -- Perform a self install with `make dev` or `pip install -e ".[dev]"` -- Locally test the validator with the [test instructions below](#testing-and-using-your-validator) -- Update this README to follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md) - -* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies - -### Naming Conventions -1. Avoid using `is` and `bug` -2. Use snake_case: i.e. `_` to separate words. e.g. valid_address -3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string. -4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)` - -### Testing and using your validator -- Open [test/test-validator.py](test/test-validator.py) to test your new validator -- Import your new validator and modify `ValidatorTestObject` accordingly -- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly -- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object -- Write advanced tests for failures, etc. - -## Upload your validator to the validator hub -- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows: - - Update the `name` field to the name of your validator - - Update the `description` field to a short description of your validator - - Update the `authors` field to your name and email - - Add/update the `dependencies` field to include all dependencies your validator needs. -- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly. -- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories. - - e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file. -- Ensure that there are no other dependencies or any additional steps required to run your validator. +# Overview + +| Developed by | Guardrails AI | +| Date of development | Feb 15, 2024 | +| Validator type | Format | +| Blog | | +| License | Apache 2 | +| Input/Output | Output | + +# Description + +This validator ensures that a generated output is the literal "pass". + +# Installation + +```bash +$ guardrails hub install hub://guardrails/validator_template +``` + +# Usage Examples + +## Validating string output via Python + +In this example, we’ll test that a generated word is `pass`. + +```python +# Import Guard and Validator +from guardrails.hub import ValidatorTemplate +from guardrails import Guard + +# Initialize Validator +val = ValidatorTemplate() + +# Setup Guard +guard = Guard.from_string( + validators=[val, ...], +) + +guard.parse("pass") # Validator passes +guard.parse("fail") # Validator fails +``` + +## Validating JSON output via Python + +In this example, we verify that a processes’s status is specified as `pass`. + +```python +# Import Guard and Validator +from pydantic import BaseModel +from guardrails.hub import ValidatorTemplate +from guardrails import Guard + +val = ValidatorTemplate() + +# Create Pydantic BaseModel +class Process(BaseModel): + process_name: str + status: str = Field(validators=[val]) + +# Create a Guard to check for valid Pydantic output +guard = Guard.from_pydantic(output_class=Process) + +# Run LLM output generating JSON through guard +guard.parse(""" +{ + "process_name": "tempalting", + "status": "pass" +} +""") +``` + +# API Reference + +`__init__` +- `arg_1`: A placeholder argument to demonstrate how to use init arguments. +- `arg_2`: Another placeholder argument to demonstrate how to use init arguments. +- `on_fail`: The policy to enact when a validator fails. \ No newline at end of file