Skip to content

feat: Sentry SDK integration for in-process PII scrubbing#169

Open
sidmohan0 wants to merge 3 commits into
devfrom
feat/sentry-integration
Open

feat: Sentry SDK integration for in-process PII scrubbing#169
sidmohan0 wants to merge 3 commits into
devfrom
feat/sentry-integration

Conversation

@sidmohan0

@sidmohan0 sidmohan0 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a third enforcement-point adapter alongside the Claude Code hook and LiteLLM guardrail: datafog.integrations.sentry, which scrubs PII from Sentry telemetry before it leaves the process.

Sentry's built-in client-side EventScrubber matches sensitive key names only and never inspects values — an email in an exception message, an SSN in a stack-frame local variable, or a card number in a breadcrumb goes straight through. Server-side scrubbing inspects content, but only after the data has already left the machine. This integration fills that gap with datafog's content-based regex engine.

Usage

import sentry_sdk
from datafog.integrations.sentry import DataFogSentryIntegration

sentry_sdk.init(dsn="...", integrations=[DataFogSentryIntegration()])

or manually: sentry_sdk.init(dsn="...", before_send=scrub_event).

Design

  • Integration class + global event processor (the pattern used by sentry-dramatiq / pytest-sentry): covers error events and transactions (span descriptions/data), and leaves the single before_send slot free for the application. Config is resolved from the active client per event, so clients without the integration pass through untouched.
  • Logs, trace metrics, and streamed spans bypass event processors, so setup_once_with_options composes scrubbing wrappers around the user's before_send_log / before_send_metric / _experiments["before_send_span"] callbacks: scrub first, then chain — the application's callback only ever sees scrubbed telemetry. Wrapping is idempotent and follows the SDK's own top-level-then-_experiments precedence.
  • AI agent monitoring coverage: gen_ai.* prompt/completion span attributes (which Sentry documents as not covered by default server-side scrubbing) are scrubbed in transaction events, whether attached as nested message lists or JSON strings. Users can keep include_prompts on without shipping user PII.
  • Same config surface as the other adapters: entity_types (default EMAIL, PHONE, CREDIT_CARD, SSN; noisy types opt-in), allowlist / allowlist_patterns, fail_policy (open = send unscanned on engine error so a scrubber bug never costs crash reports; closed = drop the payload).
  • Never echoes matched values — logs and errors carry entity-type counts only.

Safety properties (each locked in by a regression test)

  • Text beyond the scan caps is replaced with [UNSCANNED_N_CHARS] markers, never emitted raw — a PII value straddling a cap boundary would otherwise fail to match in the truncated chunk and reassemble unredacted.
  • Tuples/sets are rebuilt as lists and scanned (walker parity with the Claude Code hook).
  • Aliased containers are scanned once (no budget double-spend); cyclic structures terminate.
  • The integration lookup in the global processor and telemetry wrappers fails open, so delivery never breaks.
  • The _experiments copy of a wrapped callback is overwritten too, so no raw unscrubbed user callback stays reachable.
  • Top-level machine identifiers (event_id, release, environment, sdk, ...) are never touched.

Packaging

pip install datafog[sentry] (any sentry-sdk 2.x; logs/metrics wrapping engages on SDK versions that support them). The module is not imported by datafog core; apps that already ship sentry-sdk need no extra.

Test plan

  • 39 tests in tests/test_sentry_integration.py, 100% line coverage of the new module
  • End-to-end through a real sentry_sdk.init client: capture_message, capture_exception with local-variable capture, allowlist, pass-through when the integration is absent, and Sentry Logs (enable_logs=True + sentry_sdk.logger) with before_send_log chaining
  • Fail-policy tests: open returns payload on engine error, closed drops it, error logs never contain event content
  • Telemetry wrapper tests: precedence (top-level shadows _experiments, no double invocation), idempotent re-wrap, lookup-failure pass-through, metric attribute scrubbing, experimental span wrapping
  • gen_ai span attributes scrubbed as message lists and JSON strings
  • Full suite: 660 passed; only pre-existing spacy-extra failures in test_text_service_integration.py (run in dedicated CI jobs)
  • black / isort / flake8 clean
  • CI matrix (install profiles, Python versions) — needs this PR's checks to run

Follow-ups (separate PRs)

  • Attachment policy via hint["attachments"]
  • Advanced Data Scrubbing rule-pack generator (server-side safety net for non-Python SDKs)
  • Benchmark entry (µs per event) in benchmarks/

sidmohan0 added 3 commits July 9, 2026 15:07
DataFogSentryIntegration registers a global event processor covering
error events and transactions (messages, exception values, stack-frame
local variables, breadcrumbs, request data, extra, tags, span
descriptions), leaving the application's before_send slot free. A
standalone scrub_event() is exported for manual before_send wiring.

Same adapter conventions as the LiteLLM guardrail and Claude Code hook:
high-precision default entity set (EMAIL, PHONE, CREDIT_CARD, SSN),
allowlist / allowlist_patterns, fail_policy open|closed, and matched
values never echoed into logs (counts only).

Safety properties, each locked in by a regression test:
- text beyond scan caps is replaced with [UNSCANNED_N_CHARS] markers,
  never emitted raw (a PII value straddling a cap boundary would
  otherwise reassemble unredacted)
- tuples/sets are rebuilt as lists and scanned (parity with the
  Claude Code hook walker)
- aliased containers are scanned once; cyclic structures terminate
- the integration lookup in the global processor fails open so event
  delivery never breaks

Ships as pip install datafog[sentry] (sentry-sdk 2.x).
Logs, trace metrics, and streamed spans bypass Sentry's event
processors, so setup_once_with_options composes scrubbing wrappers
around the user's before_send_log / before_send_metric /
_experiments[before_send_span] callbacks: the scrub runs first, then
chains to the original, which therefore only ever sees scrubbed
telemetry. Config resolves from the current client per payload, same
as the event path, with identical fail_policy semantics.

Also locked in by tests:
- gen_ai.* prompt/completion span attributes (AI agent monitoring) are
  scrubbed in transaction events, both as nested message lists and as
  JSON strings — the surface Sentry's server-side scrubbing documents
  as uncovered
- top-level callbacks shadow _experiments callbacks (SDK precedence),
  no double invocation; the _experiments copy is overwritten with the
  wrapped closure so no raw unscrubbed callback stays reachable
- wrapping is idempotent (repeat setup cannot stack scrub passes)
- wrapper lookup failures pass telemetry through (delivery never breaks)
The nlp-advanced test job enforces repo-wide coverage thresholds; with
sentry-sdk absent the new integration tests skipped and sentry.py
counted 0%, dropping line coverage below 85%. Install sentry-sdk there
the same way litellm is installed for the guardrail tests.

Also: prettier reformat of examples/sentry/README.md and B907 (!r
conversion) fixes in the test fixtures, both from pre-commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant