Skip to content

feat(eap): translate OP_REGEXP to ClickHouse match() - #8437

Merged
pbhandari merged 5 commits into
masterfrom
feat/regexp-search
Sep 10, 2026
Merged

feat(eap): translate OP_REGEXP to ClickHouse match()#8437
pbhandari merged 5 commits into
masterfrom
feat/regexp-search

Conversation

@pbhandari

Copy link
Copy Markdown
Contributor

Problem

EAP RPC comparison filters have no regexp op. Logs search (LOGS-958) is blocked on EAP-739.

Depends on sentry-protos PR 420 (OP_REGEXP on ComparisonFilter / AnyAttributeFilter).

Solution

Translate OP_REGEXP in the shared EAP filter translator to ClickHouse match() (search-anywhere, RE2).

  • Same type rules as LIKE (string and string-array keys)
  • ignore_casematch(lower(value), lower(pattern))
  • Empty pattern → 400
  • ClickHouse CANNOT_COMPILE_REGEXP (36) remapped to RPC 400
  • AnyAttribute op dispatch is a match including REGEXP

Validation

pytest tests/web/rpc/test_common.py::TestTraceItemFiltersRegexp and related LIKE/any-attribute tests.

Linear

https://linear.app/getsentry/issue/EAP-739/add-regexp-support-for-eap-rpc

@linear-code

linear-code Bot commented Sep 8, 2026

Copy link
Copy Markdown

EAP-739

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 96b1da7. Configure here.

# Needs ClickHouse > 26.8 for matchCaseInsensitive; lower() is a stand-in until then.
if ignore_case:
return f.match(f.lower(value), f.lower(pattern))
return f.match(value, pattern)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore-case regex mangles escape sequences

Medium Severity

ignore_case applies lower() to the regexp pattern as well as the haystack. That is not equivalent to case-insensitive matching: RE2 escapes whose meaning depends on case (\S, \D, \W, \B, \Q, \A, \P) invert or break, so logs searches with ignore_case can silently return the wrong rows.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 96b1da7. Configure here.

@pbhandari pbhandari Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can fix it when we upgrade our clickhouse to 26.8+ imo. This is fine as a stopgap solution.

Comment thread snuba/web/rpc/common/common.py Outdated
Comment thread snuba/web/rpc/common/common.py
Comment on lines +905 to 910
raise BadSnubaRPCRequestException(
f"Unsupported any_attribute_filter op: {AnyAttributeFilter.Op.Name(filter_.op)}"
)


def _any_attribute_filter_to_expression(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The error message for unsupported operators in _any_attribute_op_expression uses the original filter_.op instead of the effective op, which could cause confusing error messages for future negative operators.
Severity: LOW

Suggested Fix

Update the exception logging in the default case of the match statement to use the op parameter instead of filter_.op. Change f"Unsupported any_attribute_filter op: {AnyAttributeFilter.Op.Name(filter_.op)}" to f"Unsupported any_attribute_filter op: {AnyAttributeFilter.Op.Name(op)}". This ensures the error message accurately reflects the operator being processed by the function.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: snuba/web/rpc/common/common.py#L905-L910

Potential issue: In the `_any_attribute_op_expression` function, the error handling for
unsupported operators incorrectly uses the original operator from the filter,
`filter_.op`, to generate the error message. The function logic, however, operates on an
`effective_op` (passed as the `op` parameter), which resolves negative operators (e.g.,
`OP_NOT_LIKE`) to their positive counterparts. If a new, unhandled operator is added in
the future that follows this negative-to-positive mapping, the resulting error message
will be misleading by referencing the original negative operator instead of the positive
one that was actually being processed.


def _is_valid_regexp_pattern(v: AttributeValue) -> None:
if v.WhichOneof("value") != "val_str" or v.val_str == "":
raise BadSnubaRPCRequestException("REGEXP pattern must be a non-empty string")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pbhandari is there more validation we can do here before we send the query to clickhouse (e.g. missing parentheses) ? Maybe doesn't have to be super robust but just anything that is guaranteed to fail we shouldn't send to clickhouse if possible

Comment thread snuba/clickhouse/error_codes.py
@pbhandari
pbhandari merged commit 4400120 into master Sep 10, 2026
66 checks passed
@pbhandari
pbhandari deleted the feat/regexp-search branch September 10, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants