[Improvement]: Support RFC 9535 JSONPath (filters, [*], slices) in the SDK's JSONPath helpers

[randilt]

Description

Make the SDK's JSONPath helpers (sdk/core/utils/jsonpath.go) follow RFC 9535 (JSONPath: Query Expressions for JSON), so that policies can select message content by role and use the common [*] wildcard syntax.

Current behaviour

The helpers (ExtractValueFromJsonpath, ExtractStringValueFromJsonpath, SetValueAtJSONPath) use a small custom parser that splits the path on .. It supports:

• plain keys, for example $.messages;
• one array index per segment, including negative indexes, for example messages[-1];
• * as a segment of its own, for example $.messages.*.content.

It doesn't support:

• [*]: $.messages[*].content fails with key not found: messages[*];
• filter selectors, for example $.messages[?@.role=='user'];
• slices, for example $.messages[-3:];
• descendant segments (..).

21 policies in wso2/gateway-controllers use these helpers, and 16 of them default to $.messages[-1].content: the content of the last message, whatever its role.

Why it matters

Selecting "the last message" can't tell a user's request from a tool result, an assistant reply, or a short follow-up, and there is currently no way to select messages by role. Testing the TypeSafe Jev policies on a local gateway showed three effects:

• Content screening only sees the last message. With typesafe-jev-content-safety (Add typesafe-jev-content-safety policy gateway-controllers#309) on its default path, a jailbreak sent as the last message was blocked. The same jailbreak placed earlier in the history, followed by the user message continue, was allowed, although the model still receives it. Screening all messages isn't possible today: $.messages[*].content fails, and $.messages.*.content selects every role, including the system prompt and assistant replies.
• Routing judges the wrong message. With typesafe-jev-model-routing (Add typesafe-jev-model-routing policy gateway-controllers#311), a complex request was routed to the larger model, but the follow-up go on in the same conversation was routed to the smaller one.
• Tool filtering judges tool output. With the tool filtering policy in Add TypeSafe Jev tool filtering policy gateway-controllers#313, when the last message was a tool result, all tools were removed from the request and the model could not complete the next step.

With filter selectors, these policies could select by role, for example $.messages[?@.role=='user'].content for user messages, or $.messages[?@.role=='user' || @.role=='tool'].content for the content a guardrail should screen.

Proposed change

• Support the RFC 9535 syntax in the extraction helpers, at least:
  • wildcard selectors in both forms (.* and [*]);
  • filter selectors with comparisons and logical operators (?@.role=='user', &&, ||, !);
  • array slices ([start:end:step]);
  • negative indexes, as today.
• Keep every path that works today working with the same result, including the .* segment and name[-1] forms that existing policy defaults use.
• Keep SetValueAtJSONPath limited to paths that select exactly one location (for example a model name to rewrite), and return a clear error for a path that selects several.
• Return a clear error for a path that doesn't parse, rather than treating it as "key not found".
• Consider using a maintained Go implementation of RFC 9535 instead of extending the custom parser.

What stays in the policies

RFC 9535 returns every match, and it can't index into the result of a filter, so "the most recent user message" isn't a single path. Policies will still decide how to use a list of matches: routing and tool filtering would take the last match, and content screening would join them. Policies also still need to handle mixed content shapes (a string, an array of content parts, or null) and their own size limits.

Acceptance criteria

• $.messages[*].content and $.messages.*.content return the same values.
• $.messages[?@.role=='user'].content returns the content of every user message, in order.
• All existing path forms used by policy defaults return the same results as before.
• Unit tests cover the new selectors, backward compatibility, and invalid paths.

Related

• [Task]: PoC - integrating TypeSafe AI's Jev with AI Gateway policies #3517 (TypeSafe Jev tracker)
• Add typesafe-jev-content-safety policy gateway-controllers#309, Add typesafe-jev-model-routing policy gateway-controllers#311, Add TypeSafe Jev tool filtering policy gateway-controllers#313