Skip to content

feat: harden LiteLLM guardrail and add batch redaction#168

Open
sidmohan0 wants to merge 2 commits into
devfrom
codex/litellm-guardrail-hardening
Open

feat: harden LiteLLM guardrail and add batch redaction#168
sidmohan0 wants to merge 2 commits into
devfrom
codex/litellm-guardrail-hardening

Conversation

@sidmohan0

Copy link
Copy Markdown
Contributor

Summary

  • harden the DataFog-owned LiteLLM compatibility guardrail across chat, Responses API, text completions, tool/function payloads and schemas, Anthropic content, and buffered streaming output
  • add pre-call redaction/blocking, during-call blocking, post-call redaction/blocking, event gating, safe logging, and fail-open/fail-closed behavior
  • preserve legacy action/entity_types/fail_policy options and exact/regex allowlists while adding datafog_* aliases and locale configuration
  • add datafog.redact_many(), BatchRedactResult, and computed entity_counts on scan/redaction results
  • update the SDK, changelog, README, and LiteLLM configuration example

Why

The existing compatibility adapter only inspected basic messages[].content requests and choices[].message.content responses. Prompt-bearing fields such as Responses API input, completion prompts, tool arguments and definitions, structured-output schemas, and streaming deltas could bypass PII enforcement.

This also folds reusable integration work into DataFog itself: integrations can consume entity totals directly and redact multiple fragments with stable token numbering and pseudonym assignments without concatenating text or changing detection boundaries.

Compatibility

The compatibility class continues to accept DataFog's original option names and remains enabled by default when instantiated directly. Explicit default_on: false configurations still require request selection through LiteLLM. Existing allowlist behavior is retained.

Validation

  • 96 passed in focused engine, bridge API, no-network, and LiteLLM guardrail suites
  • all changed-file pre-commit hooks pass: isort, Black, flake8, Ruff, Prettier, gitleaks, YAML, large-file, and merge-conflict checks
  • focused mypy check passes for the new engine and guardrail code
  • quick core and guardrail benchmarks pass:
    • single-message redaction: 43.8 us median
    • clean two-message pre-call: 110.6 us median
    • PII redaction pre-call: 233.6 us median
  • broad local suite: 692 passed, 274 skipped, 44 xfailed; four failures and one setup error require the optional en_core_web_lg spaCy model, which is not installed locally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant