OFREP is a protocol rather than a vendor, so the backend under test is simply something that + * speaks it. flagd does, on port {@value #OFREP_PORT}, which means this suite reuses the flagd + * testbed image and its launchpad control API unchanged — the same stack the flagd adoption runs + * against, from the same Compose file. + * + *
A clean run is 65 scenarios, 46 passing, 17 skipped and 2 failing. Both + * failures are the untagged 32-bit precision scenario and the {@code @variants} row beside it, on a + * flag the pinned testbed image does not serve — open-feature/flagd-testbed#392. That is a gap in + * the stack rather than in the provider, so neither is a {@code KnownDeviation}: the provider was + * never given the flag to get wrong. + * + *
The suite is intermittently flaky, and the flakiness is the backend's. Roughly + * half of the runs measured carry one or two additional failures, all of the same shape: an + * evaluation that should have resolved comes back as the code default, or as + * {@code FLAG_NOT_FOUND} where {@code TYPE_MISMATCH} was expected, or with reason {@code ERROR} + * where a resolution was expected. Which scenario is hit moves from run to run — + * {@code errors.feature}, {@code evaluation.feature} and {@code reason.feature} have each been the + * victim — so it is not a property of any assertion. Checked rather than assumed: five runs at this + * revision and three at {@code ccdb8879} before it, the old pin producing a run of seven failures + * and a run of two from the same tree. It is the readiness window + * open-feature/flagd-testbed#394 exists to close, reaching a provider that holds nothing between + * calls, so every evaluation races the stack afresh. Do not add a settle after control + * calls to cover it — that issue, and Appendix F, say why. + */ +public class OfrepTest extends ContainerizedProviderTckTest { + + /** The container-internal port flagd serves the OFREP HTTP API on. */ + private static final int OFREP_PORT = 8016; + + /** + * A port nothing listens on, for the initialisation-failure scenarios. + * + *
Deliberately not a port on the Compose stack: the stack must stay up for the whole suite, + * and simulated outages belong to the control API. + */ + private static final int UNAVAILABLE_PORT = 9999; + + /** + * {@inheritDoc} + * + *
Outside this module on purpose, and not the idiomatic {@code src/test/resources} path: the
+ * flagd adoption runs against the same stack and names the same file, so there is one image tag
+ * for both rather than two that can drift. Module-relative, like any other value here.
+ */
+ @Override
+ public File composeFile() {
+ return new File("../../tools/flagd-testbed/docker-compose.yaml");
+ }
+
+ @Override
+ public List Short timeouts on purpose: the {@code @unavailable} scenarios assert that failure is
+ * reported promptly. They are skipped for this provider — see
+ * {@link #capabilities()} — but the deadlines stay correct so that the scenarios start passing
+ * on their own the day the provider grows an {@code initialize()}.
+ */
+ @Override
+ public FeatureProvider createUnavailableProvider() {
+ return OfrepProvider.constructProvider(OfrepProviderOptions.builder()
+ .baseUrl("http://localhost:" + UNAVAILABLE_PORT)
+ .connectTimeout(Duration.ofMillis(500))
+ .requestTimeout(Duration.ofMillis(500))
+ .build());
+ }
+
+ /**
+ * {@inheritDoc}
+ *
+ * Six capabilities are withheld for the same root cause: {@code OfrepProvider} is a bare
+ * {@link dev.openfeature.sdk.FeatureProvider} (OfrepProvider.java:19) with no lifecycle of its
+ * own. It holds no state, opens no stream, runs no poll loop and does not override
+ * {@code initialize()} — every evaluation is a fresh, independent HTTP POST
+ * (Resolver.java:54-97, OfrepApi.java:93-118). There is nothing in it that could observe a
+ * backend transition, let alone report one.
+ *
+ * {@link Capability#NUMERIC_COERCION} is withheld for a different reason: the
+ * provider keeps the two numeric types strictly apart, in both directions. Deserialisation goes
+ * through a plain Jackson {@code ObjectMapper} into an untyped {@code Object value}
+ * (OfrepResponse.java:16, OfrepApi.java:109), which maps a JSON integer to {@link Integer} and
+ * a JSON fraction to {@link Double}, and {@code handleResolved} then admits the value only on
+ * an exact {@code type.isInstance(responseValue)} check, otherwise returning
+ * {@code TYPE_MISMATCH} with the code default (Resolver.java:183-191). There is no integral
+ * check and no round trip anywhere in that path — nothing in the provider widens or narrows a
+ * number.
+ *
+ * A provider declaring the tag must pass all three of its scenarios. This one passes one, and
+ * that one is right for the wrong reason — {@code float-flag} (0.5) requested as an integer is rejected rather than
+ * truncated to {@code 0}, because it is a {@link Double} and not because 0.5 is fractional —
+ * and both lossless cases fail on the same exact-instance check. {@code integer-flag} (10)
+ * requested as a float arrives as an {@code Integer}, which {@code Double.class.isInstance}
+ * rejects, so "an integer requested as a float is widened without loss" cannot pass;
+ * {@code integral-float-flag} (10.0) requested as an integer arrives as a {@link Double},
+ * which {@code Integer.class.isInstance} rejects, so "an integral float requested as an
+ * integer is coerced without loss" cannot pass either. Declaring the tag would turn two of its
+ * three scenarios red.
+ *
+ * No {@code KnownDeviation} accompanies it, and that is deliberate rather than silence.
+ * {@link Capability#NUMERIC_COERCION} records that the rule is borrowed rather than normative,
+ * so strict typing in both directions is a legitimate choice — the SDK's own
+ * {@code InMemoryProvider} makes it — and a deviation would assert a defect the specification
+ * says is not one.
+ *
+ * Appendix F's scenario-level declaring rule does not reach this omission, which is worth
+ * saying because at first it reads as though it should. That rule is about whether a question is
+ * askable, and all three of these are; what comes first is whether an answer is owed,
+ * and none is. Taking the second rule without the first would manufacture two failures out of a
+ * permitted choice.
+ *
+ * All of that is read from the source, because a withheld tag means the scenarios are
+ * skipped and a run cannot confirm it. {@code OfrepProviderTest} asserts the exact-instance
+ * check itself, but only across Boolean and String (OfrepProviderTest.java:71, 338-339) — no
+ * unit test pins the numeric pair, which is why the reasoning above cites {@code handleResolved}
+ * directly. A run in which either lossless scenario passes means {@code handleResolved} or the
+ * deserialiser changed, and this declaration should follow it.
+ *
+ * {@link Capability#STRING_TYPING} and {@link Capability#FULLY_TYPED_VALUES} are both
+ * declared, and all four of their scenarios pass. Together they gate what specification
+ * revision {@code d47a66eb} moved out of the mandatory matrix — {@code boolean-flag},
+ * {@code integer-flag}, {@code float-flag} and {@code object-flag} asked through the String
+ * accessor — and the mechanism is the same exact-instance check as above, reached from the other
+ * side: {@code String.class.isInstance} of a {@link Boolean}, an {@link Integer} or a
+ * {@link Double} is false, so {@code handleResolved} returns {@code TYPE_MISMATCH} with the code
+ * default rather than the value's {@code toString()}. OFREP carries a JSON type per flag and
+ * Jackson preserves it, so this provider is on the typed side of the distinction the capabilities
+ * exist to draw.
+ *
+ * Both tags, because the check is indifferent to which type it is refusing. {@code bda599f1}
+ * split {@code @fully-typed-values} off for backends that type a boolean and an integer but keep
+ * a float and a structure as text; OFREP is not one of them, and the same line of code answers
+ * all four. Confirmed by a run after the re-pin: skip count unchanged at 17, with no
+ * {@code FULLY_TYPED_VALUES} entry among the skip reasons, and none of the four among the
+ * failures. It is one of the cases {@code OfrepProviderTest} does pin directly, across Boolean
+ * and String (OfrepProviderTest.java:71, 338-339).
+ *
+ * {@link Capability#OBJECT} is declared: the same exact-instance check is what makes the
+ * {@code @object} mismatch matrix work, and the structured happy path passes through
+ * {@code resolve(Object.class, ...)}, which every non-null value satisfies, and is converted
+ * with {@code Value.objectToValue} (Resolver.java:125-136). {@link Capability#LARGE_INTEGERS} is
+ * absent from the list below for a reason that says nothing about OFREP — the TCK refuses it
+ * centrally — and that matters more here than elsewhere, because every other name in that list
+ * is something this provider genuinely cannot do.
+ *
+ * {@link Capability#VARIANTS} and {@link Capability#TARGETING} are declared, and unlike the
+ * withheld tags above both are confirmed by a run rather than read from the source. The OFREP
+ * response carries {@code variant} alongside {@code value} and {@code reason} and the provider
+ * passes it through, so seven of the {@code @variants} outline's eight rows pass; the eighth is
+ * the testbed gap above, not a second defect. {@code @targeting} matters more here than its name
+ * suggests: the provider sends the evaluation context in the request body and the backend
+ * evaluates the rule, so the three {@code targeting-key-flag} scenarios are the only ones in the
+ * canonical set that would notice a context dropped on the way out. All three pass.
+ *
+ * {@link Capability#STANDARD_REASONS} arrived by the {@code declarableExcept} default rather
+ * than by a decision, so it was measured before being written down. Eight of
+ * {@code reason.feature}'s nine scenarios run and pass: {@code STATIC} for the four rule-less
+ * flags, {@code ERROR} beside {@code FLAG_NOT_FOUND} and {@code TYPE_MISMATCH}, and — because
+ * {@code @targeting} is declared here — {@code TARGETING_MATCH} and {@code DEFAULT} either side
+ * of {@code targeting-key-flag}'s rule. The ninth carries {@code @disabled-flags} and is skipped
+ * for that omission, so the tag here means "the standard vocabulary, over the responses this
+ * provider actually completes"; a consumer sees the withheld {@code @disabled-flags} beside it
+ * and can tell which scenario went unasked.
+ *
+ * {@link Capability#DISABLED_FLAGS} is withheld, and unlike every other withheld tag
+ * above it was measured. Declared, all four rows of its outline fail, and they fail on
+ * the error code rather than on the value: the run — at spec revision {@code ccdb8879}, when the
+ * suite was 56 scenarios rather than 65 — reported 38 passing, 12 skipped
+ * and 6 failed, the two extra failures beyond the testbed pair above being
+ * {@code expected: null but was: FLAG_NOT_FOUND} on each of the four rows. The value assertion
+ * passes, because the provider returns the caller's default on an error — right answer, wrong
+ * reason.
+ *
+ * What the backend actually answers is worth writing down, because the shape of the gap is not
+ * what one would guess. flagd's OFREP endpoint does not 404 a disabled flag: it answers
+ * {@code 200} with {@code {"key":"disabled-boolean-flag","reason":"DISABLED","metadata":{}}} and
+ * no {@code value} member, where an enabled flag comes back as
+ * {@code {"value":true,"key":"boolean-flag","reason":"STATIC","variant":"on","metadata":{}}}
+ * (probed directly against the pinned v3.8.0 image on port 8016). {@code handleResolved} reaches
+ * its {@code responseValue == null} branch and returns the code default with
+ * {@code FLAG_NOT_FOUND} and "No value returned for flag", discarding the {@code reason} it
+ * parsed one field earlier (Resolver.java:169-181, OfrepResponse.java:19).
+ *
+ * That response is well-formed OFREP, and it says what to do. It is worth
+ * establishing, because the intuitive reading of the failure — the caller's default never leaves
+ * the process, so a provider whose backend decides cannot hold this tag — is wrong here, and
+ * {@link Capability#DISABLED_FLAGS} cites this class for why. OFREP's
+ * {@code evaluationSuccess} requires only {@code key} and {@code reason}; {@code value} is
+ * not required, because one of the member schemas a success may be is
+ * {@code codeDefaultFlag}, described as "A flag evaluation that defers to the code default
+ * value ... This schema has no value property. The provider must use the code default value when
+ * processing this response." {@code DISABLED} is in the {@code reason} enum alongside
+ * {@code STATIC}. flagd is answering in exactly that shape, and the answer means what the
+ * scenario asserts.
+ *
+ * So this is a provider gap rather than an architectural limit, and it is wider than the four
+ * rows that found it: every {@code codeDefaultFlag} response is reported to the
+ * application as {@code FLAG_NOT_FOUND}, so an application checking the error code sees a failure
+ * on an evaluation that succeeded. The capability is gated because the answer can depend on
+ * architecture, and a provider that only ever received a value would have a real case — but this
+ * provider receives a response that is explicit about deferring, parses the {@code reason} that
+ * accompanies it, and then discards both.
+ *
+ * It is therefore recorded as a
+ * {@link dev.openfeature.contrib.tools.tck.KnownDeviation} rather than left as a bare
+ * omission — see {@link #knownDeviations()}. That is the opposite call from
+ * {@code @numeric-coercion} above, and the difference is where the rule lives: numeric coercion
+ * is borrowed from flagd's ADR and no specification states it, whereas {@code codeDefaultFlag}
+ * is a {@code MUST} in the protocol this provider implements. Delete both once
+ * {@code handleResolved} honours a value-less success.
+ *
+ * This is the one declaration on this branch that the settled guidance would shape
+ * differently, and it is recorded here rather than quietly left. The preferred shape is
+ * declared-and-failing, and this provider does attempt the behaviour: it receives the
+ * {@code codeDefaultFlag} response, parses the {@code reason} that accompanies it, and answers
+ * with the wrong error code — measured, four rows, failing on the code and not on the value. By
+ * that reading the honest report is to declare {@code @disabled-flags}, let the four rows fail,
+ * and keep this same deviation beside them. The flip is a change of results rather than of
+ * prose, so it is not made in the documentation pass that noticed it; it costs four failures in
+ * place of four skips and nothing else, and the deviation's text needs no change when it
+ * happens.
+ *
+ * {@code declarableExcept} and not {@code EnumSet.complementOf}, which would claim
+ * {@code @caching} on the way past; the suite refuses such a declaration at startup.
+ */
+ @Override
+ public Set One entry, for the withheld {@link Capability#DISABLED_FLAGS}, and it is the only omission
+ * in {@link #capabilities()} that is a defect rather than a fact about the provider's shape.
+ * Every other withheld tag describes something {@code OfrepProvider} has no machinery for — no
+ * initialisation, no events, no state between calls — or a rule no specification states. This one
+ * describes a response the provider receives, understands well enough to parse, and then answers
+ * wrongly. Untracked, because there is no issue to point at yet.
+ *
+ * The summary names the response shape rather than the scenario, because the scenario is only
+ * where it was noticed. {@code codeDefaultFlag} is not specific to disabled flags — any backend
+ * deferring to the code default for any reason gets the same {@code FLAG_NOT_FOUND} — so a reader
+ * comparing providers needs the general statement, not the one outline that caught it.
+ */
+ @Override
+ public List
+ *
+ *
+ *