feat: Sentry SDK integration for in-process PII scrubbing#169
Open
sidmohan0 wants to merge 3 commits into
Open
feat: Sentry SDK integration for in-process PII scrubbing#169sidmohan0 wants to merge 3 commits into
sidmohan0 wants to merge 3 commits into
Conversation
DataFogSentryIntegration registers a global event processor covering error events and transactions (messages, exception values, stack-frame local variables, breadcrumbs, request data, extra, tags, span descriptions), leaving the application's before_send slot free. A standalone scrub_event() is exported for manual before_send wiring. Same adapter conventions as the LiteLLM guardrail and Claude Code hook: high-precision default entity set (EMAIL, PHONE, CREDIT_CARD, SSN), allowlist / allowlist_patterns, fail_policy open|closed, and matched values never echoed into logs (counts only). Safety properties, each locked in by a regression test: - text beyond scan caps is replaced with [UNSCANNED_N_CHARS] markers, never emitted raw (a PII value straddling a cap boundary would otherwise reassemble unredacted) - tuples/sets are rebuilt as lists and scanned (parity with the Claude Code hook walker) - aliased containers are scanned once; cyclic structures terminate - the integration lookup in the global processor fails open so event delivery never breaks Ships as pip install datafog[sentry] (sentry-sdk 2.x).
Logs, trace metrics, and streamed spans bypass Sentry's event processors, so setup_once_with_options composes scrubbing wrappers around the user's before_send_log / before_send_metric / _experiments[before_send_span] callbacks: the scrub runs first, then chains to the original, which therefore only ever sees scrubbed telemetry. Config resolves from the current client per payload, same as the event path, with identical fail_policy semantics. Also locked in by tests: - gen_ai.* prompt/completion span attributes (AI agent monitoring) are scrubbed in transaction events, both as nested message lists and as JSON strings — the surface Sentry's server-side scrubbing documents as uncovered - top-level callbacks shadow _experiments callbacks (SDK precedence), no double invocation; the _experiments copy is overwritten with the wrapped closure so no raw unscrubbed callback stays reachable - wrapping is idempotent (repeat setup cannot stack scrub passes) - wrapper lookup failures pass telemetry through (delivery never breaks)
The nlp-advanced test job enforces repo-wide coverage thresholds; with sentry-sdk absent the new integration tests skipped and sentry.py counted 0%, dropping line coverage below 85%. Install sentry-sdk there the same way litellm is installed for the guardrail tests. Also: prettier reformat of examples/sentry/README.md and B907 (!r conversion) fixes in the test fixtures, both from pre-commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a third enforcement-point adapter alongside the Claude Code hook and LiteLLM guardrail:
datafog.integrations.sentry, which scrubs PII from Sentry telemetry before it leaves the process.Sentry's built-in client-side
EventScrubbermatches sensitive key names only and never inspects values — an email in an exception message, an SSN in a stack-frame local variable, or a card number in a breadcrumb goes straight through. Server-side scrubbing inspects content, but only after the data has already left the machine. This integration fills that gap with datafog's content-based regex engine.Usage
or manually:
sentry_sdk.init(dsn="...", before_send=scrub_event).Design
before_sendslot free for the application. Config is resolved from the active client per event, so clients without the integration pass through untouched.setup_once_with_optionscomposes scrubbing wrappers around the user'sbefore_send_log/before_send_metric/_experiments["before_send_span"]callbacks: scrub first, then chain — the application's callback only ever sees scrubbed telemetry. Wrapping is idempotent and follows the SDK's own top-level-then-_experimentsprecedence.gen_ai.*prompt/completion span attributes (which Sentry documents as not covered by default server-side scrubbing) are scrubbed in transaction events, whether attached as nested message lists or JSON strings. Users can keepinclude_promptson without shipping user PII.entity_types(defaultEMAIL, PHONE, CREDIT_CARD, SSN; noisy types opt-in),allowlist/allowlist_patterns,fail_policy(open= send unscanned on engine error so a scrubber bug never costs crash reports;closed= drop the payload).Safety properties (each locked in by a regression test)
[UNSCANNED_N_CHARS]markers, never emitted raw — a PII value straddling a cap boundary would otherwise fail to match in the truncated chunk and reassemble unredacted._experimentscopy of a wrapped callback is overwritten too, so no raw unscrubbed user callback stays reachable.event_id,release,environment,sdk, ...) are never touched.Packaging
pip install datafog[sentry](any sentry-sdk 2.x; logs/metrics wrapping engages on SDK versions that support them). The module is not imported by datafog core; apps that already ship sentry-sdk need no extra.Test plan
tests/test_sentry_integration.py, 100% line coverage of the new modulesentry_sdk.initclient:capture_message,capture_exceptionwith local-variable capture, allowlist, pass-through when the integration is absent, and Sentry Logs (enable_logs=True+sentry_sdk.logger) withbefore_send_logchaining_experiments, no double invocation), idempotent re-wrap, lookup-failure pass-through, metric attribute scrubbing, experimental span wrappingspacy-extra failures intest_text_service_integration.py(run in dedicated CI jobs)Follow-ups (separate PRs)
hint["attachments"]benchmarks/