From 12ea410d05d99412d9ea2256ab7b435b92948d9c Mon Sep 17 00:00:00 2001 From: Prajna1999 Date: Wed, 30 Sep 2026 15:57:14 +0530 Subject: [PATCH] add pydantic models --- .../api/docs/guardrails/create_ban_list.md | 34 ++++ .../guardrails/create_llm_prompt_config.md | 49 +++++ .../guardrails/create_validator_config.md | 43 ++++ .../api/docs/guardrails/delete_ban_list.md | 12 ++ .../guardrails/delete_llm_prompt_config.md | 11 + .../guardrails/delete_validator_config.md | 11 + .../app/api/docs/guardrails/get_ban_list.md | 10 + .../api/docs/guardrails/get_guardrails_job.md | 28 +++ .../docs/guardrails/get_llm_prompt_config.md | 7 + .../docs/guardrails/get_validator_config.md | 9 + .../app/api/docs/guardrails/list_ban_lists.md | 16 ++ .../guardrails/list_llm_prompt_configs.md | 14 ++ .../docs/guardrails/list_validator_configs.md | 20 ++ .../docs/guardrails/list_validator_types.md | 26 +++ .../api/docs/guardrails/update_ban_list.md | 22 ++ .../guardrails/update_llm_prompt_config.md | 22 ++ .../guardrails/update_validator_config.md | 21 ++ backend/app/api/docs/openapi_config.py | 8 + backend/app/api/routes/guardrails.py | 191 +++++++++++++++--- backend/app/models/guardrails/__init__.py | 42 +++- backend/app/models/guardrails/ban_list.py | 71 +++++++ backend/app/models/guardrails/common.py | 11 + backend/app/models/guardrails/enums.py | 52 +++++ .../models/guardrails/llm_prompt_config.py | 72 +++++++ .../app/models/guardrails/validator_config.py | 127 ++++++++++++ .../app/tests/api/routes/test_guardrails.py | 112 ++++++++-- docs/wiki/INDEX.md | 1 + docs/wiki/modules/guardrails.md | 60 ++++++ 28 files changed, 1054 insertions(+), 48 deletions(-) create mode 100644 backend/app/api/docs/guardrails/create_ban_list.md create mode 100644 backend/app/api/docs/guardrails/create_llm_prompt_config.md create mode 100644 backend/app/api/docs/guardrails/create_validator_config.md create mode 100644 backend/app/api/docs/guardrails/delete_ban_list.md create mode 100644 backend/app/api/docs/guardrails/delete_llm_prompt_config.md create mode 100644 backend/app/api/docs/guardrails/delete_validator_config.md create mode 100644 backend/app/api/docs/guardrails/get_ban_list.md create mode 100644 backend/app/api/docs/guardrails/get_guardrails_job.md create mode 100644 backend/app/api/docs/guardrails/get_llm_prompt_config.md create mode 100644 backend/app/api/docs/guardrails/get_validator_config.md create mode 100644 backend/app/api/docs/guardrails/list_ban_lists.md create mode 100644 backend/app/api/docs/guardrails/list_llm_prompt_configs.md create mode 100644 backend/app/api/docs/guardrails/list_validator_configs.md create mode 100644 backend/app/api/docs/guardrails/list_validator_types.md create mode 100644 backend/app/api/docs/guardrails/update_ban_list.md create mode 100644 backend/app/api/docs/guardrails/update_llm_prompt_config.md create mode 100644 backend/app/api/docs/guardrails/update_validator_config.md create mode 100644 backend/app/models/guardrails/ban_list.py create mode 100644 backend/app/models/guardrails/common.py create mode 100644 backend/app/models/guardrails/enums.py create mode 100644 backend/app/models/guardrails/llm_prompt_config.py create mode 100644 backend/app/models/guardrails/validator_config.py create mode 100644 docs/wiki/modules/guardrails.md diff --git a/backend/app/api/docs/guardrails/create_ban_list.md b/backend/app/api/docs/guardrails/create_ban_list.md new file mode 100644 index 000000000..d9c65e5f8 --- /dev/null +++ b/backend/app/api/docs/guardrails/create_ban_list.md @@ -0,0 +1,34 @@ +Create a ban list — a named set of words the `ban_list` validator redacts from text. + +A ban list is stored by the guardrails service and scoped to the calling project. Reference it from a validator config by setting that config's `ban_list_id` to the `id` returned here. + +### Request + +```json +{ + "name": "Safety Banned Terms", + "description": "Terms not allowed for this tenant policy", + "domain": "abuse", + "is_public": false, + "banned_words": ["slur_a", "slur_b"] +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `name` | string | yes | 1–100 chars. Must be unique for the project. | +| `description` | string | yes | 1–500 chars. | +| `banned_words` | string[] | yes | Up to 1000 entries, each 1–100 chars. | +| `domain` | string | yes | Free-form grouping label used to filter on list. | +| `is_public` | boolean | no | Defaults to `false`. | + +### Notes + +- `is_public: true` makes the list readable by other tenants. Updating and deleting stay restricted to the owning project regardless. +- The response is `200`, not `201` — the guardrails service does not use `201`. + +### Errors + +- `400` — a ban list with this configuration already exists. +- `422` — the body failed validation. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/create_llm_prompt_config.md b/backend/app/api/docs/guardrails/create_llm_prompt_config.md new file mode 100644 index 000000000..0eaeda7b6 --- /dev/null +++ b/backend/app/api/docs/guardrails/create_llm_prompt_config.md @@ -0,0 +1,49 @@ +Create a stored prompt for one of the LLM-backed validators. + +Two validators read their instructions from a stored prompt instead of hard-coding them: `topic_relevance` (does this text stay on topic?) and `answer_relevance_custom_llm` (does this answer address the question?). Create the prompt here, then reference its `id` from the matching validator config. + +### Request + +```json +{ + "validator_name": "topic_relevance", + "name": "Maternal Health Scope", + "description": "Topic guard for maternal health support bot", + "prompt_schema_version": 1, + "llm_prompt": "Pregnancy care: Questions about prenatal care, ANC visits, nutrition, supplements, danger signs. Postpartum care: Questions about recovery after delivery, breastfeeding, and mother health checks." +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `validator_name` | enum | yes | `topic_relevance` or `answer_relevance_custom_llm`. Immutable after creation. | +| `name` | string | yes | 1–100 chars. | +| `description` | string | yes | 1–500 chars. | +| `prompt_schema_version` | integer | no | Defaults to `1`. Must be >= 1. | +| `llm_prompt` | string | yes | The prompt text. Non-empty. | + +### Placeholders + +For `answer_relevance_custom_llm` the prompt **must** contain both `{query}` and `{answer}`; the service rejects it otherwise. Example: + +``` +You are evaluating a maternal health assistant. +Query: {query} +Answer: {answer} + +Does the answer directly address the maternal health query? +Answer only YES or NO. +``` + +`topic_relevance` prompts have no required placeholders. + +### Notes + +- New configs are created active. `is_active` can only be changed via `PATCH`. +- Responds `200`, not `201`. + +### Errors + +- `400` — a config with the same validator, version and prompt text already exists. +- `422` — the body failed validation, or an `answer_relevance_custom_llm` prompt was missing `{query}`/`{answer}`. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/create_validator_config.md b/backend/app/api/docs/guardrails/create_validator_config.md new file mode 100644 index 000000000..eb00f881d --- /dev/null +++ b/backend/app/api/docs/guardrails/create_validator_config.md @@ -0,0 +1,43 @@ +Register a validator configuration — a named, reusable validator setup that `POST /guardrails` applies by id. + +This is the object you point at from a guardrails run: create one here, then pass its `id` as a `validator_config_id` in the `config` array of `POST /guardrails`. + +### Request + +The body is a fixed set of base fields **plus** whatever tuning keys the chosen validator type accepts. Call `GET /guardrails` to discover the accepted keys per type. + +```json +{ + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", + "on_fail_action": "fix", + "is_enabled": true, + "entity_types": ["PERSON", "PHONE_NUMBER", "IN_AADHAAR"], + "threshold": 0.6 +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `name` | string | yes | 5–225 chars. | +| `type` | enum | yes | One of the validator types from `GET /guardrails`. | +| `stage` | enum | yes | `input` or `output`. | +| `on_fail_action` | enum | no | `exception` \| `fix` \| `rephrase`. Defaults to `fix`. | +| `is_enabled` | boolean | no | Defaults to `true`. | +| *(extra keys)* | any | no | Validator-specific tuning, stored as the config blob. | + +In the example above, `entity_types` and `threshold` are `pii_remover` tuning keys — they are not part of the base schema, which is why Swagger shows them as additional properties rather than named fields. + +### Notes + +- **Uniqueness is enforced on `name` alone**, scoped to the project. The same validator type may be registered many times under different names. +- `stage` is advisory. `POST /guardrails` routes on the text it is actually given, not on this field, so one config can serve both directions. +- Do **not** send `organization_id` or `project_id`. The tenant is derived from your authenticated context, and the guardrails service rejects those keys in the body. +- Responds `200`, not `201`. + +### Errors + +- `400` — a validator config with this name already exists in the project. +- `422` — the body failed validation, or it contained `organization_id`/`project_id`. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_ban_list.md b/backend/app/api/docs/guardrails/delete_ban_list.md new file mode 100644 index 000000000..923028749 --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_ban_list.md @@ -0,0 +1,12 @@ +Delete a ban list permanently. Restricted to the owning project. + +Validator configs that still reference the deleted list by `ban_list_id` are not cleaned up; they will fail when the guardrails service next tries to resolve the list. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `403` — the list belongs to another tenant. +- `404` — no such ban list. +- `422` — `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_llm_prompt_config.md b/backend/app/api/docs/guardrails/delete_llm_prompt_config.md new file mode 100644 index 000000000..ef8d61eff --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_llm_prompt_config.md @@ -0,0 +1,11 @@ +Delete a stored LLM prompt config permanently, scoped to the calling project. + +Validator configs that still reference the deleted prompt will fail when the guardrails service next tries to resolve it. Repoint or remove them first. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_validator_config.md b/backend/app/api/docs/guardrails/delete_validator_config.md new file mode 100644 index 000000000..9d8db14bc --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_validator_config.md @@ -0,0 +1,11 @@ +Delete a validator config permanently, scoped to the calling project. + +Guardrails runs that still pass the deleted `validator_config_id` will no longer resolve it. Remove the id from your `POST /guardrails` calls first. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_ban_list.md b/backend/app/api/docs/guardrails/get_ban_list.md new file mode 100644 index 000000000..954c1e9f9 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_ban_list.md @@ -0,0 +1,10 @@ +Fetch a single ban list by id. + +Readable if the list belongs to the calling project, or if it belongs to another tenant and is marked `is_public`. + +### Errors + +- `403` — the list belongs to another tenant and is not public. +- `404` — no such ban list. +- `422` — `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_guardrails_job.md b/backend/app/api/docs/guardrails/get_guardrails_job.md new file mode 100644 index 000000000..f74830603 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_guardrails_job.md @@ -0,0 +1,28 @@ +Poll a guardrails job for its status and sanitised result. + +Use this when you submitted `POST /guardrails` without a `callback_url`, or to inspect a completed job for traceability. The job id comes from the `POST /guardrails` response. + +### Status values + +| `status` | Meaning | +|---|---| +| `PENDING` | Queued, not yet picked up. | +| `PROCESSING` | A worker is applying the validators. | +| `SUCCESS` | Finished; `guardrails_response` carries the sanitised text. | +| `FAILED` | The text was hard-blocked, or the job errored. `error_message` explains why. | + +### Response + +`guardrails_response` is populated only on `SUCCESS`; it is `null` in every other state. The sanitised text sits at `guardrails_response.response.output.content.value`. + +`warnings` mirrors the `metadata.warnings` of the webhook payload, so polling callers do not miss a bypass signal — most importantly the case where the guardrails service was unavailable and the original text was returned unchanged. It is always empty for a hard-blocked job. + +### Notes + +- If the upstream response carried no sanitised text, the value falls back to the original submitted text. +- `usage` counters default to zero when the guardrails service reports none. + +### Errors + +- `404` — no such job in this project, or the id belongs to a job that is not a guardrails job. +- `422` — `job_id` is not a valid UUID. diff --git a/backend/app/api/docs/guardrails/get_llm_prompt_config.md b/backend/app/api/docs/guardrails/get_llm_prompt_config.md new file mode 100644 index 000000000..0e583f368 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_llm_prompt_config.md @@ -0,0 +1,7 @@ +Fetch a single LLM prompt config by id, scoped to the calling project. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_validator_config.md b/backend/app/api/docs/guardrails/get_validator_config.md new file mode 100644 index 000000000..f49a81f10 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_validator_config.md @@ -0,0 +1,9 @@ +Fetch a single validator config by id, scoped to the calling project. + +The response is the stored row flattened together with its validator-specific tuning config, so it carries keys beyond the declared schema — `id`, `organization_id`, `project_id`, `created_at`, `updated_at`, and every tuning key for that validator type. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_ban_lists.md b/backend/app/api/docs/guardrails/list_ban_lists.md new file mode 100644 index 000000000..938fa2390 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_ban_lists.md @@ -0,0 +1,16 @@ +List the ban lists visible to the calling project. + +Returns lists owned by the project plus any list from another tenant marked `is_public`. Ordered newest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `domain` | string | no | — | Return only lists carrying this domain label. | +| `offset` | integer | no | `0` | Rows to skip. Must be >= 0. | +| `limit` | integer | no | — | Max rows to return, 1–100. Omit for no limit. | + +### Errors + +- `422` — `offset` is negative, or `limit` is outside 1–100. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_llm_prompt_configs.md b/backend/app/api/docs/guardrails/list_llm_prompt_configs.md new file mode 100644 index 000000000..f8d03d376 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_llm_prompt_configs.md @@ -0,0 +1,14 @@ +List the stored LLM prompt configs belonging to the calling project, oldest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `validator_name` | enum | no | — | `topic_relevance` or `answer_relevance_custom_llm`. | +| `offset` | integer | no | `0` | Rows to skip. Must be >= 0. | +| `limit` | integer | no | — | Max rows to return, 1–100. Omit for no limit. | + +### Errors + +- `422` — `validator_name` is not a recognised value, `offset` is negative, or `limit` is outside 1–100. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_validator_configs.md b/backend/app/api/docs/guardrails/list_validator_configs.md new file mode 100644 index 000000000..17cf6c336 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_validator_configs.md @@ -0,0 +1,20 @@ +List the validator configs belonging to the calling project, oldest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `ids` | UUID | no | — | Repeat the parameter to fetch several by id (`?ids=A&ids=B`). | +| `stage` | enum | no | — | `input` or `output`. | +| `type` | enum | no | — | A validator type from `GET /guardrails`. | + +This route is not paginated. + +### Response + +Each item is the stored row flattened together with its validator-specific tuning config, so entries carry keys beyond the declared schema — `id`, `organization_id`, `project_id`, `created_at`, `updated_at`, and every tuning key for that validator type. + +### Errors + +- `422` — a value in `ids` is not a valid UUID, or `stage`/`type` is not a recognised value. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_validator_types.md b/backend/app/api/docs/guardrails/list_validator_types.md new file mode 100644 index 000000000..09c7a775e --- /dev/null +++ b/backend/app/api/docs/guardrails/list_validator_types.md @@ -0,0 +1,26 @@ +List every validator type the guardrails service supports, together with the JSON Schema of the tuning fields each one accepts. + +Use this to discover what to put in the body of `POST /guardrails/validators/configs`: the `config` schema of an entry tells you exactly which extra keys that validator type understands. + +### Response shape + +This is the one guardrails route that is **not** wrapped in the standard `APIResponse` envelope. The body is a bare object with one entry per supported validator type (abridged — `config` is a full JSON Schema generated from the service's own model, so it is always current): + +```json +{ + "validators": [ + {"type": "pii_remover", "config": { "...JSON Schema..." }}, + {"type": "uli_slur_match", "config": { "...JSON Schema..." }} + ] +} +``` + +### Validator types + +`uli_slur_match`, `pii_remover`, `gender_assumption_bias`, `ban_list`, `topic_relevance`, `topic_relevance_llm`, `llm_critic`, `llamaguard_7b`, `profanity_free`, `nsfw_text`, `answer_relevance_custom_llm`. + +Every validator additionally accepts `on_fail` (`exception` | `fix` | `rephrase`, default `fix`) and `stage` (`input` | `output`). + +### Errors + +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/update_ban_list.md b/backend/app/api/docs/guardrails/update_ban_list.md new file mode 100644 index 000000000..c70f2d10c --- /dev/null +++ b/backend/app/api/docs/guardrails/update_ban_list.md @@ -0,0 +1,22 @@ +Update a ban list. Fields you omit are left unchanged. + +`banned_words` is replaced wholesale, not merged — send the full list you want stored. + +### Request + +```json +{ + "description": "Updated description", + "banned_words": ["slur_a", "slur_b", "slur_c"] +} +``` + +All five fields (`name`, `description`, `banned_words`, `domain`, `is_public`) are optional and follow the same constraints as on create. + +### Errors + +- `400` — the update collides with an existing ban list. +- `403` — the list belongs to another tenant. Public lists are readable but not writable across tenants. +- `404` — no such ban list. +- `422` — the body failed validation, or `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/update_llm_prompt_config.md b/backend/app/api/docs/guardrails/update_llm_prompt_config.md new file mode 100644 index 000000000..c28bc214a --- /dev/null +++ b/backend/app/api/docs/guardrails/update_llm_prompt_config.md @@ -0,0 +1,22 @@ +Update a stored LLM prompt config. Fields you omit are left unchanged. + +### Request + +```json +{"llm_prompt": "Pregnancy care: Updated scope definition"} +``` + +`name`, `description`, `prompt_schema_version`, `llm_prompt` and `is_active` can be patched. + +### Notes + +- `validator_name` is immutable — it cannot be patched. Create a new config instead. +- `is_active` is settable only here, not on create. +- Editing `llm_prompt` on an `answer_relevance_custom_llm` config still requires both `{query}` and `{answer}` placeholders. + +### Errors + +- `400` — the update collides with an existing config. +- `404` — no such config, or it belongs to another project. +- `422` — the body failed validation, the placeholder rule was broken, or `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/update_validator_config.md b/backend/app/api/docs/guardrails/update_validator_config.md new file mode 100644 index 000000000..42168e444 --- /dev/null +++ b/backend/app/api/docs/guardrails/update_validator_config.md @@ -0,0 +1,21 @@ +Update a validator config's base fields. Fields you omit are left unchanged. + +### Request + +```json +{"stage": "output", "is_enabled": false} +``` + +Only `name`, `type`, `stage`, `on_fail_action` and `is_enabled` can be patched. + +### Notes + +- **Validator-specific tuning cannot be changed here.** The guardrails service rejects any key outside the five base fields, so changing something like `threshold` or `entity_types` means deleting the config and recreating it. +- Because `POST /guardrails` resolves validators by id at run time, an update takes effect on the next run — there is no versioning. + +### Errors + +- `400` — the new `name` collides with an existing config in the project. +- `404` — no such config, or it belongs to another project. +- `422` — the body contained a key outside the five base fields, a value failed validation, or `config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/openapi_config.py b/backend/app/api/docs/openapi_config.py index 99cee7875..6da01fec8 100644 --- a/backend/app/api/docs/openapi_config.py +++ b/backend/app/api/docs/openapi_config.py @@ -27,6 +27,13 @@ "name": "LLM", "description": "Large Language Model inference and interaction endpoints", }, + { + "name": "Guardrails", + "description": ( + "Text safety validation: applying guardrails to text, and managing " + "the validator configs, ban lists and LLM prompts they run on" + ), + }, { "name": "Evaluation", "description": "Dataset upload, running evaluations, listing datasets as well as evaluations", @@ -89,6 +96,7 @@ "Collections", "Config Management", "LLM", + "Guardrails", "Evaluation", "Fine Tuning", "Model Evaluation", diff --git a/backend/app/api/routes/guardrails.py b/backend/app/api/routes/guardrails.py index d3052ee88..3fcc42b35 100644 --- a/backend/app/api/routes/guardrails.py +++ b/backend/app/api/routes/guardrails.py @@ -2,9 +2,10 @@ from typing import Annotated, Any from uuid import UUID -from fastapi import APIRouter, Depends, HTTPException, Query, Response +from fastapi import APIRouter, Depends, HTTPException, Path, Query, Response from fastapi.responses import JSONResponse from opentelemetry import trace +from sqlmodel import SQLModel from app.api.deps import AuthContextDep, SessionDep from app.api.permissions import Permission, require_permission @@ -13,10 +14,24 @@ from app.crud.jobs import JobCrud from app.models import JobStatus, JobType from app.models.guardrails import ( + BanListCreate, + BanListPublic, + BanListUpdate, GuardrailsCallbackData, + GuardrailsDeletePublic, GuardrailsJobImmediatePublic, GuardrailsJobPublic, GuardrailsRequest, + LLMPromptConfigCreate, + LLMPromptConfigPublic, + LLMPromptConfigUpdate, + LLMValidatorNameEnum, + StageEnum, + ValidatorConfigCreate, + ValidatorConfigPublic, + ValidatorConfigUpdate, + ValidatorTypeEnum, + ValidatorTypeListPublic, ) from app.services.guardrails.jobs import start_job from app.services.llm.guardrails import proxy_guardrails_request @@ -28,8 +43,25 @@ guardrails_callback_router = APIRouter() +@guardrails_callback_router.post( + "{$callback_url}", + name="guardrails_callback", +) +def guardrails_callback_notification( + body: APIResponse[GuardrailsCallbackData], +) -> None: + """Callback delivered to `callback_url` when a guardrails job finishes. + + On success `success=True` and `data` carries the sanitised text. On a hard + block `success=False`, `error` carries the upstream message and `data` is + null. `metadata` echoes the request's `request_metadata` plus a + server-managed `warnings` list. + """ + + @router.post( "/guardrails", + summary="Apply guardrails to text", description=load_description("guardrails/apply_guardrails.md"), response_model=APIResponse[GuardrailsJobImmediatePublic], callbacks=guardrails_callback_router.routes, @@ -106,11 +138,24 @@ def _upstream_response(status_code: int, payload: Any) -> Response: return JSONResponse(status_code=status_code, content=payload, headers=headers) +def _forward_body(body: SQLModel) -> dict[str, Any]: + """Serialise a validated body back to the exact keys the caller sent. + + ``exclude_unset`` matters most on PATCH: without it every omitted optional + field would be forwarded as an explicit ``null`` and blank out stored data. + ``mode="json"`` renders UUIDs, enums and datetimes as JSON scalars. + """ + return body.model_dump(mode="json", exclude_unset=True) + + # ROUTE ORDERING: these fixed paths must stay above GET /guardrails/{job_id}. @router.get( "/guardrails", + summary="List validator types", + description=load_description("guardrails/list_validator_types.md"), + response_model=ValidatorTypeListPublic, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def list_guardrails_validator_types(_current_user: AuthContextDep) -> Response: @@ -126,30 +171,42 @@ def list_guardrails_validator_types(_current_user: AuthContextDep) -> Response: @router.post( "/guardrails/ban_lists", + summary="Create a ban list", + description=load_description("guardrails/create_ban_list.md"), + response_model=APIResponse[BanListPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def create_guardrails_ban_list( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: BanListCreate ) -> Response: status_code, payload = proxy_guardrails_request( "POST", "/ban_lists/", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.get( "/guardrails/ban_lists", + summary="List ban lists", + description=load_description("guardrails/list_ban_lists.md"), + response_model=APIResponse[list[BanListPublic]], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def list_guardrails_ban_lists( _current_user: AuthContextDep, - domain: str | None = None, - offset: Annotated[int, Query(ge=0)] = 0, - limit: Annotated[int | None, Query(ge=1, le=100)] = None, + domain: Annotated[ + str | None, + Query(description="Filter to ban lists carrying this domain label."), + ] = None, + offset: Annotated[int, Query(ge=0, description="Rows to skip.")] = 0, + limit: Annotated[ + int | None, + Query(ge=1, le=100, description="Max rows to return. Unset means no limit."), + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -163,67 +220,96 @@ def list_guardrails_ban_lists( @router.post( "/guardrails/llm_prompt_configs", + summary="Create an LLM prompt config", + description=load_description("guardrails/create_llm_prompt_config.md"), + response_model=APIResponse[LLMPromptConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def create_guardrails_llm_prompt_config( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: LLMPromptConfigCreate ) -> Response: status_code, payload = proxy_guardrails_request( "POST", "/llm_prompt_configs/", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.get( "/guardrails/llm_prompt_configs", + summary="List LLM prompt configs", + description=load_description("guardrails/list_llm_prompt_configs.md"), + response_model=APIResponse[list[LLMPromptConfigPublic]], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def list_guardrails_llm_prompt_configs( _current_user: AuthContextDep, - validator_name: str | None = None, - offset: Annotated[int, Query(ge=0)] = 0, - limit: Annotated[int | None, Query(ge=1, le=100)] = None, + validator_name: Annotated[ + LLMValidatorNameEnum | None, + Query(description="Filter to prompts driving this validator."), + ] = None, + offset: Annotated[int, Query(ge=0, description="Rows to skip.")] = 0, + limit: Annotated[ + int | None, + Query(ge=1, le=100, description="Max rows to return. Unset means no limit."), + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", "/llm_prompt_configs/", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - params={"validator_name": validator_name, "offset": offset, "limit": limit}, + params={ + "validator_name": validator_name.value if validator_name else None, + "offset": offset, + "limit": limit, + }, ) return _upstream_response(status_code, payload) @router.post( "/guardrails/validators/configs", + summary="Create a validator config", + description=load_description("guardrails/create_validator_config.md"), + response_model=APIResponse[ValidatorConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def create_guardrails_validator_config( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: ValidatorConfigCreate ) -> Response: status_code, payload = proxy_guardrails_request( "POST", "/validators/configs/", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.get( "/guardrails/validators/configs", + summary="List validator configs", + description=load_description("guardrails/list_validator_configs.md"), + response_model=APIResponse[list[ValidatorConfigPublic]], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def list_guardrails_validator_configs( _current_user: AuthContextDep, - ids: Annotated[list[UUID] | None, Query()] = None, - stage: str | None = None, - type: str | None = None, + ids: Annotated[ + list[UUID] | None, + Query(description="Repeat to fetch several configs by id."), + ] = None, + stage: Annotated[ + StageEnum | None, Query(description="Filter by the stage the config targets.") + ] = None, + type: Annotated[ + ValidatorTypeEnum | None, Query(description="Filter by validator type.") + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -232,8 +318,8 @@ def list_guardrails_validator_configs( project_id=_current_user.project_.id, params={ "ids": [str(config_id) for config_id in ids] if ids else None, - "stage": stage, - "type": type, + "stage": stage.value if stage else None, + "type": type.value if type else None, }, ) return _upstream_response(status_code, payload) @@ -241,10 +327,14 @@ def list_guardrails_validator_configs( @router.get( "/guardrails/validators/configs/{config_id}", + summary="Get a validator config", + description=load_description("guardrails/get_validator_config.md"), + response_model=APIResponse[ValidatorConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def get_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -257,27 +347,36 @@ def get_guardrails_validator_config( @router.patch( "/guardrails/validators/configs/{config_id}", + summary="Update a validator config", + description=load_description("guardrails/update_validator_config.md"), + response_model=APIResponse[ValidatorConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def update_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], + body: ValidatorConfigUpdate, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", f"/validators/configs/{config_id}", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.delete( "/guardrails/validators/configs/{config_id}", + summary="Delete a validator config", + description=load_description("guardrails/delete_validator_config.md"), + response_model=APIResponse[GuardrailsDeletePublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def delete_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -290,10 +389,14 @@ def delete_guardrails_validator_config( @router.get( "/guardrails/ban_lists/{ban_list_id}", + summary="Get a ban list", + description=load_description("guardrails/get_ban_list.md"), + response_model=APIResponse[BanListPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def get_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -306,27 +409,36 @@ def get_guardrails_ban_list( @router.patch( "/guardrails/ban_lists/{ban_list_id}", + summary="Update a ban list", + description=load_description("guardrails/update_ban_list.md"), + response_model=APIResponse[BanListPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def update_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], + body: BanListUpdate, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", f"/ban_lists/{ban_list_id}", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.delete( "/guardrails/ban_lists/{ban_list_id}", + summary="Delete a ban list", + description=load_description("guardrails/delete_ban_list.md"), + response_model=APIResponse[GuardrailsDeletePublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def delete_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -339,10 +451,14 @@ def delete_guardrails_ban_list( @router.get( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Get an LLM prompt config", + description=load_description("guardrails/get_llm_prompt_config.md"), + response_model=APIResponse[LLMPromptConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def get_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -355,27 +471,36 @@ def get_guardrails_llm_prompt_config( @router.patch( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Update an LLM prompt config", + description=load_description("guardrails/update_llm_prompt_config.md"), + response_model=APIResponse[LLMPromptConfigPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def update_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], + body: LLMPromptConfigUpdate, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", f"/llm_prompt_configs/{prompt_config_id}", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - json_body=body, + json_body=_forward_body(body), ) return _upstream_response(status_code, payload) @router.delete( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Delete an LLM prompt config", + description=load_description("guardrails/delete_llm_prompt_config.md"), + response_model=APIResponse[GuardrailsDeletePublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def delete_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -388,13 +513,15 @@ def delete_guardrails_llm_prompt_config( @router.get( "/guardrails/{job_id}", + summary="Get guardrails job status", + description=load_description("guardrails/get_guardrails_job.md"), response_model=APIResponse[GuardrailsJobPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], ) def get_guardrails_job_status( _current_user: AuthContextDep, session: SessionDep, - job_id: UUID, + job_id: Annotated[UUID, Path(description="Job id returned by POST /guardrails.")], ) -> APIResponse[GuardrailsJobPublic]: """Poll for a /guardrails job's status and result. diff --git a/backend/app/models/guardrails/__init__.py b/backend/app/models/guardrails/__init__.py index 423311150..bf07d9c9a 100644 --- a/backend/app/models/guardrails/__init__.py +++ b/backend/app/models/guardrails/__init__.py @@ -1,3 +1,20 @@ +from app.models.guardrails.ban_list import ( + BanListCreate, + BanListPublic, + BanListUpdate, +) +from app.models.guardrails.common import GuardrailsDeletePublic +from app.models.guardrails.enums import ( + GuardrailOnFailEnum, + LLMValidatorNameEnum, + StageEnum, + ValidatorTypeEnum, +) +from app.models.guardrails.llm_prompt_config import ( + LLMPromptConfigCreate, + LLMPromptConfigPublic, + LLMPromptConfigUpdate, +) from app.models.guardrails.request import ( GuardrailValidator, GuardrailsRequest, @@ -11,15 +28,38 @@ GuardrailsOutput, GuardrailsOutputContent, ) +from app.models.guardrails.validator_config import ( + ValidatorConfigCreate, + ValidatorConfigPublic, + ValidatorConfigUpdate, + ValidatorTypeListPublic, + ValidatorTypePublic, +) __all__ = [ + "BanListCreate", + "BanListPublic", + "BanListUpdate", + "GuardrailOnFailEnum", "GuardrailValidator", - "GuardrailsRequest", "GuardrailsCallbackData", "GuardrailsCallbackResponse", "GuardrailsCallbackUsage", + "GuardrailsDeletePublic", "GuardrailsJobImmediatePublic", "GuardrailsJobPublic", "GuardrailsOutput", "GuardrailsOutputContent", + "GuardrailsRequest", + "LLMPromptConfigCreate", + "LLMPromptConfigPublic", + "LLMPromptConfigUpdate", + "LLMValidatorNameEnum", + "StageEnum", + "ValidatorConfigCreate", + "ValidatorConfigPublic", + "ValidatorConfigUpdate", + "ValidatorTypeEnum", + "ValidatorTypeListPublic", + "ValidatorTypePublic", ] diff --git a/backend/app/models/guardrails/ban_list.py b/backend/app/models/guardrails/ban_list.py new file mode 100644 index 000000000..7631cdf14 --- /dev/null +++ b/backend/app/models/guardrails/ban_list.py @@ -0,0 +1,71 @@ +"""Ban list payloads proxied to the kaapi-guardrails service. + +Mirrors `kaapi-guardrails/backend/app/schemas/ban_list.py` at the level of +field names and types only. Length and format rules are deliberately left to +the upstream service: duplicating them here would reject payloads upstream +would accept the moment the two drift apart. + +Timestamps keep the upstream ``created_at``/``updated_at`` spelling rather +than the Kaapi-wide ``inserted_at`` because these objects are echoed verbatim +from the other service. +""" + +from datetime import datetime +from uuid import UUID + +from sqlmodel import Field, SQLModel + + +class BanListBase(SQLModel): + name: str = Field( + ..., description="Human-readable name for the ban list. Must be unique." + ) + description: str = Field( + ..., description="What this ban list covers and when it should be applied." + ) + banned_words: list[str] = Field( + ..., + description=( + "Words to redact. Consumed by the `ban_list` validator when it is " + "given this list's `ban_list_id`." + ), + ) + domain: str = Field( + ..., + description=( + "Caller-defined grouping label (e.g. 'abuse'). Used to filter lists " + "on GET /guardrails/ban_lists." + ), + ) + is_public: bool = Field( + default=False, + description=( + "When true the list is readable by other tenants. Updates and " + "deletes remain restricted to the owning project." + ), + ) + + +class BanListCreate(BanListBase): + """Request body for ``POST /api/v1/guardrails/ban_lists``.""" + + +class BanListUpdate(SQLModel): + """Request body for ``PATCH /api/v1/guardrails/ban_lists/{ban_list_id}``. + + Omitted fields are left unchanged. + """ + + name: str | None = None + description: str | None = None + banned_words: list[str] | None = None + domain: str | None = None + is_public: bool | None = None + + +class BanListPublic(BanListBase): + """A ban list as returned by the guardrails service.""" + + id: UUID + created_at: datetime + updated_at: datetime diff --git a/backend/app/models/guardrails/common.py b/backend/app/models/guardrails/common.py new file mode 100644 index 000000000..66a02a831 --- /dev/null +++ b/backend/app/models/guardrails/common.py @@ -0,0 +1,11 @@ +from sqlmodel import Field, SQLModel + + +class GuardrailsDeletePublic(SQLModel): + """Body returned by the guardrails delete routes. + + The guardrails service answers deletes with 200 and a confirmation + message rather than a 204, so the proxy surfaces a body here. + """ + + message: str = Field(..., description="Human-readable confirmation.") diff --git a/backend/app/models/guardrails/enums.py b/backend/app/models/guardrails/enums.py new file mode 100644 index 000000000..474c9b45c --- /dev/null +++ b/backend/app/models/guardrails/enums.py @@ -0,0 +1,52 @@ +"""Enums mirroring the kaapi-guardrails service contract. + +These exist so Swagger renders real dropdowns instead of free-form strings on +the proxy routes. They are a *mirror*, not the source of truth: the upstream +service owns validation, and any value it accepts but we reject would turn a +working call into a spurious 422. Keep these in sync with +`kaapi-guardrails/backend/app/core/enum.py`. +""" + +from enum import Enum + + +class ValidatorTypeEnum(str, Enum): + """Validator implementations the guardrails service can run.""" + + LEXICAL_SLUR = "uli_slur_match" + PII_REMOVER = "pii_remover" + GENDER_ASSUMPTION_BIAS = "gender_assumption_bias" + BAN_LIST = "ban_list" + TOPIC_RELEVANCE = "topic_relevance" + TOPIC_RELEVANCE_LLM = "topic_relevance_llm" + LLM_CRITIC = "llm_critic" + LLAMAGUARD_7B = "llamaguard_7b" + PROFANITY_FREE = "profanity_free" + NSFW_TEXT = "nsfw_text" + ANSWER_RELEVANCE_CUSTOM_LLM = "answer_relevance_custom_llm" + + +class StageEnum(str, Enum): + """Which side of an LLM exchange a validator is meant to inspect.""" + + INPUT = "input" + OUTPUT = "output" + + +class GuardrailOnFailEnum(str, Enum): + """What the service does when a validator fails. + + ``fix`` falls back to an empty string for validators that have no + programmatic repair (e.g. ``profanity_free``). + """ + + EXCEPTION = "exception" + FIX = "fix" + REPHRASE = "rephrase" + + +class LLMValidatorNameEnum(str, Enum): + """Validators that are driven by a stored LLM prompt config.""" + + TOPIC_RELEVANCE = "topic_relevance" + ANSWER_RELEVANCE_CUSTOM_LLM = "answer_relevance_custom_llm" diff --git a/backend/app/models/guardrails/llm_prompt_config.py b/backend/app/models/guardrails/llm_prompt_config.py new file mode 100644 index 000000000..3aa3e4735 --- /dev/null +++ b/backend/app/models/guardrails/llm_prompt_config.py @@ -0,0 +1,72 @@ +"""LLM prompt config payloads proxied to the kaapi-guardrails service. + +Mirrors `kaapi-guardrails/backend/app/schemas/llm_prompt_config.py` at the +level of field names and types only. Length rules and the `{query}`/`{answer}` +placeholder rule are left to the upstream service, which owns validation. + +Timestamps keep the upstream ``created_at``/``updated_at`` spelling because +these objects are echoed verbatim from the other service. +""" + +from datetime import datetime +from uuid import UUID + +from sqlmodel import Field, SQLModel + +from app.models.guardrails.enums import LLMValidatorNameEnum + + +class LLMPromptConfigCreate(SQLModel): + """Request body for ``POST /api/v1/guardrails/llm_prompt_configs``.""" + + validator_name: LLMValidatorNameEnum = Field( + ..., + description=( + "Which LLM-backed validator this prompt drives. Cannot be changed " + "after creation." + ), + ) + name: str = Field(..., description="Human-readable name.") + description: str = Field(..., description="What this prompt evaluates.") + prompt_schema_version: int = Field( + default=1, + description="Version of the prompt contract, for callers that iterate on wording.", + ) + llm_prompt: str = Field( + ..., + description=( + "The prompt text. For `answer_relevance_custom_llm` this must " + "contain both the `{query}` and `{answer}` placeholders; the " + "guardrails service rejects it with a 422 otherwise." + ), + ) + + +class LLMPromptConfigUpdate(SQLModel): + """Request body for ``PATCH /guardrails/llm_prompt_configs/{prompt_config_id}``. + + ``validator_name`` is absent by design — it is immutable after creation. + """ + + name: str | None = None + description: str | None = None + prompt_schema_version: int | None = None + llm_prompt: str | None = None + is_active: bool | None = Field( + default=None, + description="Only settable on update. New configs are created active.", + ) + + +class LLMPromptConfigPublic(SQLModel): + """An LLM prompt config as returned by the guardrails service.""" + + id: UUID + validator_name: LLMValidatorNameEnum + name: str + description: str + prompt_schema_version: int + llm_prompt: str + is_active: bool + created_at: datetime + updated_at: datetime diff --git a/backend/app/models/guardrails/validator_config.py b/backend/app/models/guardrails/validator_config.py new file mode 100644 index 000000000..b6262c66d --- /dev/null +++ b/backend/app/models/guardrails/validator_config.py @@ -0,0 +1,127 @@ +"""Validator config payloads proxied to the kaapi-guardrails service. + +Mirrors `kaapi-guardrails/backend/app/schemas/validator_config.py`. Timestamps +keep the upstream ``created_at``/``updated_at`` spelling because these objects +are owned by the guardrails service and echoed verbatim. + +A validator config is stored as a fixed set of base columns plus a JSONB +``config`` blob holding the validator-specific tuning. The service flattens +those back into one object on the way out, which is why the create body and +the response are both open shapes. +""" + +from datetime import datetime +from typing import Any +from uuid import UUID + +from pydantic import ConfigDict +from sqlmodel import Field, SQLModel + +from app.models.guardrails.enums import ( + GuardrailOnFailEnum, + StageEnum, + ValidatorTypeEnum, +) + + +class ValidatorConfigBase(SQLModel): + name: str = Field( + ..., + description=( + "Unique name for this config within the project. Uniqueness is " + "enforced on the name alone, so the same validator type may be " + "registered more than once under different names." + ), + ) + type: ValidatorTypeEnum = Field( + ..., description="Which validator implementation to run." + ) + stage: StageEnum = Field( + ..., + description=( + "Which text this config is intended for. Advisory only at run " + "time: POST /guardrails routes on the text it is given, not on " + "this field." + ), + ) + on_fail_action: GuardrailOnFailEnum = Field( + default=GuardrailOnFailEnum.FIX, + description="Action taken when this validator fails.", + ) + is_enabled: bool = Field( + default=True, description="Whether this config is eligible to run." + ) + + +class ValidatorConfigCreate(ValidatorConfigBase): + """Request body for ``POST /api/v1/guardrails/validators/configs``. + + Extra keys are allowed and are stored as the validator's tuning config — + e.g. ``entity_types``/``threshold`` for ``pii_remover``, ``languages``/ + ``severity`` for ``uli_slur_match``. Call ``GET /api/v1/guardrails`` for + the JSON schema of each validator type's accepted keys. + + ``organization_id``/``project_id`` must not be sent: the guardrails + service derives the tenant from the authenticated context and rejects + those keys in the body with a 422. + """ + + model_config = ConfigDict(extra="allow") + + +class ValidatorConfigUpdate(SQLModel): + """Request body for ``PATCH /guardrails/validators/configs/{config_id}``. + + Only the base fields can be patched. The guardrails service forbids extra + keys here, so validator-specific tuning cannot be changed through this + route — delete and recreate the config instead. + + Extras are accepted and forwarded rather than rejected locally, so the + upstream 422 is what the caller sees. Dropping them silently would be + worse: the caller would believe a tuning change had been applied. + """ + + model_config = ConfigDict(extra="allow") + + name: str | None = None + type: ValidatorTypeEnum | None = None + stage: StageEnum | None = None + on_fail_action: GuardrailOnFailEnum | None = None + is_enabled: bool | None = None + + +class ValidatorConfigPublic(ValidatorConfigBase): + """A validator config as returned by the guardrails service. + + The service returns the stored row flattened together with its JSONB + tuning config, so responses carry additional validator-specific keys + beyond the ones declared here. + """ + + model_config = ConfigDict(extra="allow") + + id: UUID + organization_id: int + project_id: int + created_at: datetime + updated_at: datetime + + +class ValidatorTypePublic(SQLModel): + """One entry of ``GET /api/v1/guardrails``.""" + + type: str = Field(..., description="Validator type discriminator.") + config: dict[str, Any] = Field( + ..., + description="JSON Schema of the tuning fields this validator accepts.", + ) + + +class ValidatorTypeListPublic(SQLModel): + """Response body of ``GET /api/v1/guardrails``. + + This route is the one guardrails endpoint that is *not* wrapped in the + standard ``APIResponse`` envelope; the service returns this bare object. + """ + + validators: list[ValidatorTypePublic] diff --git a/backend/app/tests/api/routes/test_guardrails.py b/backend/app/tests/api/routes/test_guardrails.py index 7ac271b31..b04a7582a 100644 --- a/backend/app/tests/api/routes/test_guardrails.py +++ b/backend/app/tests/api/routes/test_guardrails.py @@ -16,11 +16,31 @@ VALIDATOR_ID = str(uuid4()) +# The proxy routes validate their bodies against the mirrored guardrails +# schemas, so these have to be complete payloads, not stubs. +BAN_LIST_CREATE_BODY = { + "name": "Safety Banned Terms", + "description": "Terms not allowed for this tenant policy", + "domain": "abuse", + "banned_words": ["slur_a", "slur_b"], +} +LLM_PROMPT_CONFIG_CREATE_BODY = { + "validator_name": "topic_relevance", + "name": "Maternal Health Scope", + "description": "Topic guard for maternal health support bot", + "llm_prompt": "Only answer questions about maternal health.", +} +VALIDATOR_CONFIG_CREATE_BODY = { + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", +} + # (client method, kaapi path, request body, upstream path) — "{id}" is filled # with a freshly generated UUID by the tests that consume this table. PROXY_ROUTES = [ ("GET", "/guardrails", None, "/"), - ("POST", "/guardrails/ban_lists", {"name": "slurs"}, "/ban_lists/"), + ("POST", "/guardrails/ban_lists", BAN_LIST_CREATE_BODY, "/ban_lists/"), ("GET", "/guardrails/ban_lists", None, "/ban_lists/"), ("GET", "/guardrails/ban_lists/{id}", None, "/ban_lists/{id}"), ("PATCH", "/guardrails/ban_lists/{id}", {"name": "renamed"}, "/ban_lists/{id}"), @@ -28,7 +48,7 @@ ( "POST", "/guardrails/llm_prompt_configs", - {"validator_name": "toxicity", "prompt": "be nice"}, + LLM_PROMPT_CONFIG_CREATE_BODY, "/llm_prompt_configs/", ), ("GET", "/guardrails/llm_prompt_configs", None, "/llm_prompt_configs/"), @@ -41,7 +61,7 @@ ( "PATCH", "/guardrails/llm_prompt_configs/{id}", - {"prompt": "be nicer"}, + {"llm_prompt": "be nicer"}, "/llm_prompt_configs/{id}", ), ( @@ -53,7 +73,7 @@ ( "POST", "/guardrails/validators/configs", - {"type": "pii", "stage": "input"}, + VALIDATOR_CONFIG_CREATE_BODY, "/validators/configs/", ), ("GET", "/guardrails/validators/configs", None, "/validators/configs/"), @@ -325,21 +345,18 @@ def test_empty_upstream_body_returns_status_with_no_body( def test_create_echoes_upstream_status_and_body( self, client: TestClient, user_api_key_header: dict[str, str] ) -> None: - created = {"id": str(uuid4()), "validator_name": "toxicity"} + created = {"id": str(uuid4()), "validator_name": "topic_relevance"} # Upstream create routes return FastAPI's default 200. with _mock_upstream(status_code=200, json_body=created) as calls: resp = client.post( "api/v1/guardrails/llm_prompt_configs", - json={"validator_name": "toxicity", "prompt": "be nice"}, + json=LLM_PROMPT_CONFIG_CREATE_BODY, headers=user_api_key_header, ) assert resp.status_code == 200 assert resp.json() == created - assert calls[0]["kwargs"]["json"] == { - "validator_name": "toxicity", - "prompt": "be nice", - } + assert calls[0]["kwargs"]["json"] == LLM_PROMPT_CONFIG_CREATE_BODY def test_connect_error_returns_502( self, client: TestClient, user_api_key_header: dict[str, str] @@ -440,12 +457,12 @@ def test_llm_prompt_config_filters_forwarded( ) -> None: with _mock_upstream(json_body={"data": []}) as calls: client.get( - "api/v1/guardrails/llm_prompt_configs?validator_name=toxicity&limit=50", + "api/v1/guardrails/llm_prompt_configs?validator_name=topic_relevance&limit=50", headers=user_api_key_header, ) assert calls[0]["kwargs"]["params"] == { - "validator_name": "toxicity", + "validator_name": "topic_relevance", "offset": 0, "limit": 50, } @@ -455,11 +472,14 @@ def test_validator_config_filters_forwarded_without_ids( ) -> None: with _mock_upstream(json_body={"data": []}) as calls: client.get( - "api/v1/guardrails/validators/configs?stage=output&type=pii", + "api/v1/guardrails/validators/configs?stage=output&type=pii_remover", headers=user_api_key_header, ) - assert calls[0]["kwargs"]["params"] == {"stage": "output", "type": "pii"} + assert calls[0]["kwargs"]["params"] == { + "stage": "output", + "type": "pii_remover", + } def test_ids_are_normalised_to_canonical_uuid_strings( self, client: TestClient, user_api_key_header: dict[str, str] @@ -497,7 +517,11 @@ def test_tenant_headers_come_from_auth_context_not_request( with _mock_upstream(status_code=200, json_body={"id": str(uuid4())}) as calls: client.post( "api/v1/guardrails/ban_lists?organization_id=999", - json={"name": "slurs", "organization_id": 999, "project_id": 888}, + json={ + **BAN_LIST_CREATE_BODY, + "organization_id": 999, + "project_id": 888, + }, headers=user_api_key_header, ) @@ -653,3 +677,61 @@ def _seed_guardrails_job( job_id=job.id, job_update=JobUpdate(status=status, error_message=error_message), ) + + +class TestOpenAPIDocumentation: + """Every guardrails operation must stay testable from the Swagger UI.""" + + @staticmethod + def _guardrails_operations() -> list[tuple[str, str, dict[str, Any]]]: + from app.main import app + + schema = app.openapi() + return [ + (path, method, operation) + for path, item in schema["paths"].items() + if path.startswith("/api/v1/guardrails") + for method, operation in item.items() + if method in ("get", "post", "patch", "delete") + ] + + def test_every_operation_is_documented(self) -> None: + for path, method, operation in self._guardrails_operations(): + assert operation.get("summary"), f"{method.upper()} {path} has no summary" + assert operation.get( + "description" + ), f"{method.upper()} {path} has no description" + + def test_every_operation_declares_a_response_schema(self) -> None: + for path, method, operation in self._guardrails_operations(): + schema = ( + operation["responses"]["200"] + .get("content", {}) + .get("application/json", {}) + .get("schema", {}) + ) + assert schema, f"{method.upper()} {path} documents no 200 schema" + + def test_write_operations_declare_a_typed_request_body(self) -> None: + """An untyped body renders an empty box in Swagger, which is untestable.""" + for path, method, operation in self._guardrails_operations(): + if method not in ("post", "patch"): + continue + schema = operation["requestBody"]["content"]["application/json"]["schema"] + assert schema.get("$ref"), f"{method.upper()} {path} has an untyped body" + + def test_apply_guardrails_publishes_its_webhook_contract(self) -> None: + from app.main import app + + operation = app.openapi()["paths"]["/api/v1/guardrails"]["post"] + callback = operation["callbacks"]["guardrails_callback"]["{$callback_url}"] + schema = callback["post"]["requestBody"]["content"]["application/json"][ + "schema" + ] + assert schema.get("$ref"), "webhook payload contract is not published" + + def test_guardrails_tag_is_described_and_grouped(self) -> None: + from app.api.docs.openapi_config import tag_groups, tags_metadata + + assert any(tag["name"] == "Guardrails" for tag in tags_metadata) + assert any("Guardrails" in group["tags"] for group in tag_groups) diff --git a/docs/wiki/INDEX.md b/docs/wiki/INDEX.md index 9bc299d34..f4ff05d28 100644 --- a/docs/wiki/INDEX.md +++ b/docs/wiki/INDEX.md @@ -17,6 +17,7 @@ Deep design narrative lives in `docs/architecture/*.md`; open those only for des ### Modules - [modules/llm-call.md](modules/llm-call.md) — `POST /llm/call` pipeline, configs (`Config`/`ConfigVersion`, `LLMCallConfig`), guardrails, chains. Deep dive: `docs/architecture/kaapi-llm-call-ARCHITECTURE.md` +- [modules/guardrails.md](modules/guardrails.md) — `POST /guardrails` job path, the `kaapi-guardrails` proxy CRUD (validator configs, ban lists, LLM prompts), fail-open vs fail-closed. No deep-dive doc yet. - [modules/evaluations.md](modules/evaluations.md) — text/STT/TTS evals, datasets, runs, batch + cron scoring, fast evals. Deep dive: `docs/architecture/kaapi-evaluations-ARCHITECTURE.md` - [modules/knowledge-base.md](modules/knowledge-base.md) — documents, collections, transforms, vector-store providers. Deep dive: `docs/architecture/kaapi-knowledge-base-ARCHITECTURE.md` - [modules/responses.md](modules/responses.md) — OpenAI Responses API integration, conversations, threads, assistants. No deep-dive doc yet. diff --git a/docs/wiki/modules/guardrails.md b/docs/wiki/modules/guardrails.md new file mode 100644 index 000000000..c75a734f8 --- /dev/null +++ b/docs/wiki/modules/guardrails.md @@ -0,0 +1,60 @@ +# Guardrails + +Text safety validation. Two distinct things live under `/api/v1/guardrails`: + +1. **A job-based apply path** — Kaapi owns it, runs it through Celery, and persists it as a `Job`. +2. **A thin proxy** to the external `kaapi-guardrails` service for managing the configs those runs reference. + +Knowing which of the two a route belongs to explains almost every behavioural quirk below. + +## Routes + +`backend/app/api/routes/guardrails.py`, tag `Guardrails`, no router prefix — every path is spelled in full. + +| Route | Kind | Notes | +|---|---|---| +| `POST /guardrails` | job | Creates a `JobType.LLM_GUARDRAILS` job, returns `job_id` immediately. Publishes its webhook contract via `guardrails_callback_router`. | +| `GET /guardrails/{job_id}` | job | Polls the job; rehydrates the sanitised text from `job.meta`. | +| `GET /guardrails` | proxy | Validator types + their JSON schemas. **Not** `APIResponse`-wrapped. | +| `{POST,GET}/guardrails/ban_lists`, `{GET,PATCH,DELETE} /guardrails/ban_lists/{ban_list_id}` | proxy | Ban list CRUD. | +| `{POST,GET} /guardrails/validators/configs`, `{GET,PATCH,DELETE} /guardrails/validators/configs/{config_id}` | proxy | Validator config CRUD. | +| `{POST,GET} /guardrails/llm_prompt_configs`, `{GET,PATCH,DELETE} /guardrails/llm_prompt_configs/{prompt_config_id}` | proxy | Stored prompts for the LLM-backed validators. | + +**Route ordering matters:** every fixed `/guardrails/...` path must stay declared above `GET /guardrails/{job_id}`, or the job route swallows them. `TestProxyRouteOrdering` guards this. + +## Models + +`backend/app/models/guardrails/` + +- `request.py` / `response.py` — Kaapi-owned shapes for the job path. +- `ban_list.py`, `validator_config.py`, `llm_prompt_config.py`, `enums.py`, `common.py` — **mirrors** of the upstream service's schemas, added so Swagger can render real request/response schemas instead of `dict[str, Any]`. + +The mirrors are a documentation surface, not a source of truth. The upstream service owns validation; anything it accepts but a mirror rejects turns a working call into a spurious 422. They keep the upstream `created_at`/`updated_at` spelling rather than the Kaapi-wide `inserted_at`, because the objects are echoed verbatim from the other service. + +Two shapes are deliberately open (`extra="allow"`): `ValidatorConfigCreate` and `ValidatorConfigPublic`. A validator config is base columns plus a JSONB tuning blob that the service flattens back into one object, so the real payload always carries keys beyond the declared schema. + +## Services + +- `app/services/guardrails/jobs.py` — `start_job` and the Celery `execute_job`. Dedupes validator ids, calls the service, delivers or stores the result. Everything lands on `job.meta` under `request` / `response` / `callback`. +- `app/services/llm/guardrails.py` — the HTTP client. `proxy_guardrails_request` backs the CRUD routes; `apply_guardrails` / `run_guardrails_validation` back the job path and the `/llm/call` pipeline. + +### Two different failure policies + +This trips people up. The same file contains both: + +- **`proxy_guardrails_request` does not fail open.** Unreachable or non-JSON upstream raises `502`. A config read returning stale silence would be worse than an error. +- **`run_guardrails_validation` does fail open.** If the service is unreachable the job still succeeds, carrying the *original text unchanged*, and a warning is attached. Callers must check `warnings` — on the webhook under `metadata.warnings`, on the poll route under `warnings` — or they will silently treat unvalidated text as sanitised. + +Tenant always travels in `X-ORGANIZATION-ID` / `X-PROJECT-ID` headers derived from the auth context, never from a caller-supplied body or query field. + +## Swagger docs + +Each route's prose lives in `backend/app/api/docs/guardrails/.md`, loaded via `load_description`. `TestOpenAPIDocumentation` asserts every operation keeps a summary, a description, a 200 schema, and a typed request body — add the markdown file when you add a route, or that test fails. + +## Gotchas + +- The upstream service returns **200 for creates and deletes**, never 201/204. Deletes carry a confirmation body. +- Validator config uniqueness is enforced on **`name` alone**, not on type/stage. +- `PATCH /guardrails/validators/configs/{id}` **cannot** change validator-specific tuning — upstream forbids extra keys there. Delete and recreate. +- PATCH bodies are forwarded with `exclude_unset=True`; without it, omitted optional fields would serialise as `null` and blank out stored data. +- `stage` on a validator config is advisory. `POST /guardrails` routes on the text it is handed, so one config can serve both directions.