-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
68a065b
commit da45c4d
Showing
2 changed files
with
115 additions
and
38 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# Guardrails Validator Template | ||
|
||
## How to create a Guardrails Validator | ||
- On the top right of the page, click "Use this template", select "create a new repository" and set a name for the package. See [Naming Conventions](#naming-conventions) below. | ||
- Clone down the new repository. | ||
- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator | ||
- Make sure that the class still inherits from `Validator` and has the `register_validator` annotation. | ||
- Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type. | ||
- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate | ||
- Perform a self install with `make dev` or `pip install -e ".[dev]"` | ||
- Locally test the validator with the [test instructions below](#testing-and-using-your-validator) | ||
- Modify the README and follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md) | ||
|
||
* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies | ||
|
||
### Naming Conventions | ||
1. Avoid using `is` and `bug` | ||
2. Use snake_case: i.e. `_` to separate words. e.g. valid_address | ||
3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string. | ||
4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)` | ||
|
||
### Testing and using your validator | ||
- Open [test/test-validator.py](test/test-validator.py) to test your new validator | ||
- Import your new validator and modify `ValidatorTestObject` accordingly | ||
- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly | ||
- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object | ||
- Write advanced tests for failures, etc. | ||
|
||
## Upload your validator to the validator hub | ||
- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows: | ||
- Update the `name` field to the name of your validator | ||
- Update the `description` field to a short description of your validator | ||
- Update the `authors` field to your name and email | ||
- Add/update the `dependencies` field to include all dependencies your validator needs. | ||
- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly. | ||
- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories. | ||
- e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file. | ||
- Ensure that there are no other dependencies or any additional steps required to run your validator. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,38 +1,77 @@ | ||
# Guardrails Validator Template | ||
|
||
## How to create a Guardrails Validator | ||
- On the top right of the page, click "Use this template", select "create a new repository" and set a name for the package. See [Naming Conventions](#naming-conventions) below. | ||
- Clone down the new repository. | ||
- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator | ||
- Make sure that the class still inherits from `Validator` and has the `register_validator` annotation. | ||
- Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type. | ||
- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate | ||
- Perform a self install with `make dev` or `pip install -e ".[dev]"` | ||
- Locally test the validator with the [test instructions below](#testing-and-using-your-validator) | ||
- Update this README to follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md) | ||
|
||
* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies | ||
|
||
### Naming Conventions | ||
1. Avoid using `is` and `bug` | ||
2. Use snake_case: i.e. `_` to separate words. e.g. valid_address | ||
3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string. | ||
4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)` | ||
|
||
### Testing and using your validator | ||
- Open [test/test-validator.py](test/test-validator.py) to test your new validator | ||
- Import your new validator and modify `ValidatorTestObject` accordingly | ||
- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly | ||
- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object | ||
- Write advanced tests for failures, etc. | ||
|
||
## Upload your validator to the validator hub | ||
- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows: | ||
- Update the `name` field to the name of your validator | ||
- Update the `description` field to a short description of your validator | ||
- Update the `authors` field to your name and email | ||
- Add/update the `dependencies` field to include all dependencies your validator needs. | ||
- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly. | ||
- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories. | ||
- e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file. | ||
- Ensure that there are no other dependencies or any additional steps required to run your validator. | ||
# Overview | ||
|
||
| Developed by | Guardrails AI | | ||
| Date of development | Feb 15, 2024 | | ||
| Validator type | Format | | ||
| Blog | | | ||
| License | Apache 2 | | ||
| Input/Output | Output | | ||
|
||
# Description | ||
|
||
This validator ensures that a generated output is the literal "pass". | ||
|
||
# Installation | ||
|
||
```bash | ||
$ guardrails hub install hub://guardrails/validator_template | ||
``` | ||
|
||
# Usage Examples | ||
|
||
## Validating string output via Python | ||
|
||
In this example, we’ll test that a generated word is `pass`. | ||
|
||
```python | ||
# Import Guard and Validator | ||
from guardrails.hub import ValidatorTemplate | ||
from guardrails import Guard | ||
|
||
# Initialize Validator | ||
val = ValidatorTemplate() | ||
|
||
# Setup Guard | ||
guard = Guard.from_string( | ||
validators=[val, ...], | ||
) | ||
|
||
guard.parse("pass") # Validator passes | ||
guard.parse("fail") # Validator fails | ||
``` | ||
|
||
## Validating JSON output via Python | ||
|
||
In this example, we verify that a processes’s status is specified as `pass`. | ||
|
||
```python | ||
# Import Guard and Validator | ||
from pydantic import BaseModel | ||
from guardrails.hub import ValidatorTemplate | ||
from guardrails import Guard | ||
|
||
val = ValidatorTemplate() | ||
|
||
# Create Pydantic BaseModel | ||
class Process(BaseModel): | ||
process_name: str | ||
status: str = Field(validators=[val]) | ||
|
||
# Create a Guard to check for valid Pydantic output | ||
guard = Guard.from_pydantic(output_class=Process) | ||
|
||
# Run LLM output generating JSON through guard | ||
guard.parse(""" | ||
{ | ||
"process_name": "tempalting", | ||
"status": "pass" | ||
} | ||
""") | ||
``` | ||
|
||
# API Reference | ||
|
||
`__init__` | ||
- `arg_1`: A placeholder argument to demonstrate how to use init arguments. | ||
- `arg_2`: Another placeholder argument to demonstrate how to use init arguments. | ||
- `on_fail`: The policy to enact when a validator fails. |