Contributor Experience

CONTRIBUTING.md

@@ -0,0 +1,38 @@
+# Guardrails Validator Template
+
+## How to create a Guardrails Validator
+- On the top right of the page, click "Use this template", select "create a new repository"  and set a name for the package.  See [Naming Conventions](#naming-conventions) below.
+- Clone down the new repository.
+- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator
+    - Make sure that the class still inherits from `Validator` and has the `register_validator` annotation.
+    - Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type.
+- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate
+- Perform a self install with `make dev` or `pip install -e ".[dev]"`
+- Locally test the validator with the [test instructions below](#testing-and-using-your-validator)
+- Modify the README and follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md)
+
+* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies
+
+### Naming Conventions
+1. Avoid using `is` and `bug`
+2. Use snake_case: i.e. `_` to separate words. e.g. valid_address
+3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string.
+4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)`
+
+### Testing and using your validator
+- Open [test/test-validator.py](test/test-validator.py) to test your new validator 
+- Import your new validator and modify `ValidatorTestObject` accordingly
+- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly
+- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object 
+- Write advanced tests for failures, etc.
+
+## Upload your validator to the validator hub
+- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows:
+    - Update the `name` field to the name of your validator
+    - Update the `description` field to a short description of your validator
+    - Update the `authors` field to your name and email
+    - Add/update the `dependencies` field to include all dependencies your validator needs.
+- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly.
+- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories.
+    - e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file.
+- Ensure that there are no other dependencies or any additional steps required to run your validator.

Makefile

@@ -1,8 +1,11 @@
+dev:
+	pip install -e ".[dev]"
+
 lint:
 	ruff check .
 
-tests:
-	pytest ./test
+test:
+	pytest ./tests
 
 type:
 	pyright validator

README.md

@@ -1,29 +1,109 @@
-# Guardrails Validator Template
-
-## How to create a Guardrails Validator
-- On the top right of the page, click "Use this template", select "create a new repository"  and set a name for the package.
-- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator
-    - Make sure that the class still inherits from `Validator` and has the `register_validator` annotation.
-    - Set the `name` in the `register_validator` to the name of the repo and set the appropriate data type.
-- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of RegexMatch
-- Locally test the validator with the test instructions below
-
-* Note: This package uses a pyproject.toml file, on first run, run `pip install .` to pull down and install all dependencies
-
-### Testing and using your validator
-- Open [test/test-validator.py](test/test-validator.py) to test your new validator 
-- Import your new validator and modify `ValidatorTestObject` accordingly
-- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly
-- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object 
-- Write advanced tests for failures, etc.
-
-## Upload your validator to the validator hub
-- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows:
-    - Update the `name` field to the name of your validator
-    - Update the `description` field to a short description of your validator
-    - Update the `authors` field to your name and email
-    - Add/update the `dependencies` field to include all dependencies your validator needs.
-- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly.
-- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories.
-    - e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file.
-- Ensure that there are no other dependencies or any additional steps required to run your validator.
+# Overview
+
+| Developed by | Guardrails AI |
+| Date of development | Feb 15, 2024 |
+| Validator type | Format |
+| Blog |  |
+| License | Apache 2 |
+| Input/Output | Output |
+
+# Description
+
+## Intended Use
+This validator is a template for creating other validators, but for demonstrative purposes it ensures that a generated output is the literal `pass`.
+
+## Requirements
+
+* Dependencies:
+	- guardrails-ai>=0.4.0
+
+* Foundation model access keys:
+	- OPENAI_API_KEY
+
+# Installation
+
+```bash
+$ guardrails hub install hub://guardrails/validator_template
+```
+
+# Usage Examples
+
+## Validating string output via Python
+
+In this example, we apply the validator to a string output generated by an LLM.
+
+```python
+# Import Guard and Validator
+from guardrails.hub import ValidatorTemplate
+from guardrails import Guard
+
+# Setup Guard
+guard = Guard.use(
+    ValidatorTemplate
+)
+
+guard.validate("pass")  # Validator passes
+guard.validate("fail")  # Validator fails
+```
+
+## Validating JSON output via Python
+
+In this example, we apply the validator to a string field of a JSON output generated by an LLM.
+
+```python
+# Import Guard and Validator
+from pydantic import BaseModel, Field
+from guardrails.hub import ValidatorTemplate
+from guardrails import Guard
+
+# Initialize Validator
+val = ValidatorTemplate()
+
+# Create Pydantic BaseModel
+class Process(BaseModel):
+		process_name: str
+		status: str = Field(validators=[val])
+
+# Create a Guard to check for valid Pydantic output
+guard = Guard.from_pydantic(output_class=Process)
+
+# Run LLM output generating JSON through guard
+guard.parse("""
+{
+		"process_name": "templating",
+		"status": "pass"
+}
+""")
+```
+
+# API Reference
+
+**`__init__(self, on_fail="noop")`**
+<ul>
+Initializes a new instance of the ValidatorTemplate class.
+
+**Parameters**
+- **`arg_1`** *(str)*: A placeholder argument to demonstrate how to use init arguments.
+- **`arg_2`** *(str)*: Another placeholder argument to demonstrate how to use init arguments.
+- **`on_fail`** *(str, Callable)*: The policy to enact when a validator fails.  If `str`, must be one of `reask`, `fix`, `filter`, `refrain`, `noop`, `exception` or `fix_reask`. Otherwise, must be a function that is called when the validator fails.
+</ul>
+<br/>
+
+**`validate(self, value, metadata) → ValidationResult`**
+<ul>
+Validates the given `value` using the rules defined in this validator, relying on the `metadata` provided to customize the validation process. This method is automatically invoked by `guard.parse(...)`, ensuring the validation logic is applied to the input data.
+
+Note:
+
+1. This method should not be called directly by the user. Instead, invoke `guard.parse(...)` where this method will be called internally for each associated Validator.
+2. When invoking `guard.parse(...)`, ensure to pass the appropriate `metadata` dictionary that includes keys and values required by this validator. If `guard` is associated with multiple validators, combine all necessary metadata into a single dictionary.
+
+**Parameters**
+- **`value`** *(Any):* The input value to validate.
+- **`metadata`** *(dict):* A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.
+
+
+    | Key | Type | Description | Default |
+    | --- | --- | --- | --- |
+    | key1 | String | Description of key1's role. | N/A |
+</ul>

pyproject.toml

@@ -1,5 +1,5 @@
 [project]
-name = "regexmatch"
+name = "validator-template"
 version = "0.0.0"
 description = "Template repo for Guardrails Hub validators."
 authors = [
@@ -9,7 +9,6 @@ license = {file = "LICENSE"}
 readme = "README.md"
 requires-python = ">= 3.8"
 dependencies = [
-    "rstr",
     "guardrails-ai>=0.3.2"
 ]
 
@@ -18,13 +17,13 @@ dev = [
     "pyright",
     "pytest",
     "ruff"
-]
+] 
 
 [tool.pytest.ini_options]
 minversion = "6.0"
 addopts = "-rP"
 testpaths = [
-    "test"
+    "tests"
 ]
 
 [tool.pyright]

tests/test_validator.py

@@ -0,0 +1,27 @@
+# to run these, run 
+# make tests
+
+from guardrails import Guard
+import pytest
+from validator import ValidatorTemplate
+
+# We use 'exception' as the validator's fail action,
+#  so we expect failures to always raise an Exception
+# Learn more about corrective actions here:
+#  https://www.guardrailsai.com/docs/concepts/output/#%EF%B8%8F-specifying-corrective-actions
+guard = Guard.from_string(validators=[ValidatorTemplate(arg_1="arg_1", arg_2="arg_2", on_fail="exception")])
+
+def test_pass():
+  test_output = "pass"
+  result = guard.parse(test_output)
+
+  assert result.validation_passed is True
+  assert result.validated_output == test_output
+
+def test_fail():
+  with pytest.raises(Exception) as exc_info:
+    test_output = "fail"
+    guard.parse(test_output)
+
+  # Assert the exception has your error_message
+  assert str(exc_info.value) == "Validation failed for field with errors: {A descriptive but concise error message about why validation failed}"

validator/__init__.py

@@ -1,3 +1,3 @@
-from .main import RegexMatch
+from .main import ValidatorTemplate
 
-__all__ = ["RegexMatch"]
+__all__ = ["ValidatorTemplate"]

validator/main.py

@@ -1,9 +1,5 @@
-import re
-import string
 from typing import Any, Callable, Dict, Optional
 
-import rstr
-
 from guardrails.validator_base import (
     FailResult,
     PassResult,
@@ -13,59 +9,40 @@
 )
 
 
-@register_validator(name="guardrails/regex_match", data_type="string")
-class RegexMatch(Validator):
-    """Validates that a value matches a regular expression.
+@register_validator(name="guardrails/validator_template", data_type="string")
+class ValidatorTemplate(Validator):
+    """Validates that {fill in how you validator interacts with the passed value}.
 
     **Key Properties**
 
     | Property                      | Description                       |
     | ----------------------------- | --------------------------------- |
-    | Name for `format` attribute   | `regex_match`                     |
+    | Name for `format` attribute   | `guardrails/validator_template`   |
     | Supported data types          | `string`                          |
-    | Programmatic fix              | Generate a string that matches the regular expression |
+    | Programmatic fix              | {If you support programmatic fixes, explain it here. Otherwise `None`} |
 
     Args:
-        regex: Str regex pattern
-        match_type: Str in {"search", "fullmatch"} for a regex search or full-match option
+        arg_1 (string): {Description of the argument here}
+        arg_2 (string): {Description of the argument here}
     """  # noqa
 
+    # If you don't have any init args, you can omit the __init__ method.
     def __init__(
         self,
-        regex: str,
-        match_type: Optional[str] = None,
+        arg_1: str,
+        arg_2: str,
         on_fail: Optional[Callable] = None,
     ):
-        # todo -> something forces this to be passed as kwargs and therefore xml-ized.
-        # match_types = ["fullmatch", "search"]
-
-        if match_type is None:
-            match_type = "fullmatch"
-        assert match_type in [
-            "fullmatch",
-            "search",
-        ], 'match_type must be in ["fullmatch", "search"]'
-
-        super().__init__(on_fail=on_fail, match_type=match_type, regex=regex)
-        self._regex = regex
-        self._match_type = match_type
-
-    def validate(self, value: Any, metadata: Dict) -> ValidationResult:
-        p = re.compile(self._regex)
-        """Validates that value matches the provided regular expression."""
-        # Pad matching string on either side for fix
-        # example if we are performing a regex search
-        str_padding = (
-            "" if self._match_type == "fullmatch" else rstr.rstr(string.ascii_lowercase)
-        )
-        self._fix_str = str_padding + rstr.xeger(self._regex) + str_padding
-
-        if not getattr(p, self._match_type)(value):
+        super().__init__(on_fail=on_fail, arg_1=arg_1, arg_2=arg_2)
+        self._arg_1 = arg_1
+        self._arg_2 = arg_2
+
+    def validate(self, value: Any, metadata: Dict = {}) -> ValidationResult:
+        """Validates that {fill in how you validator interacts with the passed value}."""
+        # Add your custom validator logic here and return a PassResult or FailResult accordingly.
+        if value != "pass": # FIXME
             return FailResult(
-                error_message=f"Result must match {self._regex}",
-                fix_value=self._fix_str,
+                error_message="{A descriptive but concise error message about why validation failed}",
+                fix_value="{The programmtic fix if applicable, otherwise remove this kwarg.}",
             )
         return PassResult()
-
-    def to_prompt(self, with_keywords: bool = True) -> str:
-        return "results should match " + self._regex

validator/post-install.py

@@ -1,3 +1,4 @@
 print("post-install starting...")
 print("This is where you would do things like download nltk tokenizers or login to the HuggingFace hub...")
 print("post-install complete!")
+# If you don't have anything to add here you should delete this file.