Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c118f7f
test(flagd): adopt the OpenFeature Provider TCK
aepfli Sep 11, 2026
3269ecb
test(flagd): declare the numeric-coercion gap as a defect, not a choice
aepfli Sep 11, 2026
2aef4fa
test(flagd): record what flagd-testbed does not serve, and why that i…
aepfli Sep 11, 2026
e2f78e7
test(flagd): withhold @reinitialization, and settle @stale by evidence
aepfli Sep 11, 2026
b518217
test(flagd): withhold @large-integers explicitly
aepfli Sep 11, 2026
aa3162a
docs(flagd): say what the new tags gate here, and what the testbed st…
aepfli Sep 12, 2026
bfff980
test(flagd): declare @disabled-flags, on the evidence of both resolvers
aepfli Sep 12, 2026
ae29d57
test(flagd): follow the provider-tck -> tck rename
aepfli Sep 12, 2026
1116c64
test(flagd): give the in-process resolver a deadline its sync can meet
aepfli Sep 12, 2026
2e7f28f
test(flagd): declare numeric coercion and let its failure show
aepfli Sep 12, 2026
2f11a9e
docs(flagd): say that the TCK suites are Docker-gated and hand-run
aepfli Sep 12, 2026
09953d8
test(flagd): keep the TCK suites out of CI, which the e2e profile was…
aepfli Sep 12, 2026
1e69fe2
docs(flagd): point at Appendix F for the CI-exclusion reasoning
aepfli Sep 12, 2026
43b9a5f
test(flagd): record that both resolvers report the standard reasons
aepfli Sep 13, 2026
e50de0c
test(flagd): stop withholding a capability nobody in Java can hold
aepfli Sep 13, 2026
7319bb1
docs(flagd): cite the declaring rule this adoption argued out in long…
aepfli Sep 13, 2026
738e15f
test(flagd): run the conformance suites from a step of their own
aepfli Sep 13, 2026
3938719
test(flagd): give the conformance adoption a package of its own
aepfli Sep 13, 2026
15a2f49
docs(flagd): cut the conformance section to what is this adoption's
aepfli Sep 14, 2026
45ce7bc
test(flagd): one Compose file for both adoptions, and cut the prose a…
aepfli Sep 14, 2026
69ba698
test(flagd): follow @disabled-flags' corrected gating question
aepfli Sep 14, 2026
ea8e836
test(flagd): record the intermittent testbed failure observed here too
aepfli Sep 14, 2026
4048647
test(tck): run the conformance suite against flagd-testbed v3.10.1
aepfli Sep 14, 2026
d25b436
test(flagd): declare @string-typing, on a run rather than on the default
aepfli Sep 15, 2026
0af5b66
test(flagd): declare @fully-typed-values too, on a run in both modes
aepfli Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions providers/flagd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,3 +358,41 @@ FlagdOptions options = FlagdOptions.builder()
.resolverType(Config.Resolver.IN_PROCESS)
.build();
```

## Provider conformance (TCK)

This provider adopts the [OpenFeature Provider TCK](../../tools/tck/README.md), once per resolver:
`RpcTest` and `InProcessTest`, both over the shared `AbstractResolverTest` in
`src/test/java/.../flagd/tck/`. **Read that class before changing either.** It records, against
measured behaviour rather than assumption, which capabilities are declared, which are withheld and
why, and every known deviation — that reasoning is the most valuable thing about this adoption and it
lives next to the declaration rather than here.

```bash
# once, if tools/tck is not in your local repository yet
mvn -pl tools/tck -am -DskipTests install

mvn -Ptck -pl providers/flagd test # both resolvers
mvn -Ptck -pl providers/flagd -Dtest=InProcessTest test # one
```

Do not add `-am` to the run itself; the TCK README says why.

**`-Pe2e` does not run these suites — it is the profile that keeps them out.** Worth stating plainly,
because `mvn -Pe2e -pl providers/flagd test` reads as if it ran everything and instead runs the legacy
`Run*Test` suites in silence: 788 tests, no scenario tally, and no mention of either conformance
suite. This module excludes `**/e2e/*.java,**/tck/*.java` by default; the `e2e` profile drops only the
first, and the `tck` profile only the second. It used to clear the property outright, which — since
`ci.yml`'s `main` job activates `e2e` on every push, on a runner that has a Docker daemon — ran these
suites in CI, where they are expected to fail, and turned every unrelated pull request red.

**No CI job runs them**, so a maintainer runs them by hand before merging a change to the provider's
resolution, event or lifecycle behaviour, and quotes the result in the pull request. A scheduled or
path-filtered workflow was considered and declined: a suite whose red is diagnosed by whoever happens
to read the notification is worse than one whose red is diagnosed by the person who caused it.

Both suites are **expected to fail**, identically: 65 scenarios each, 59 passing, 2 skipped, 4
failing. Three of the four failures come from flags the pinned `flagd-testbed` image does not serve
(open-feature/flagd-testbed#392) and one is the real numeric-coercion defect, declared and left
visible rather than skipped (open-feature/flagd#1996). `AbstractResolverTest` enumerates them by name.
Anything else is a regression.
90 changes: 86 additions & 4 deletions providers/flagd/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,21 @@
<version>0.14.2</version> <!--x-release-please-version -->

<properties>
<!-- exclusion expression for e2e tests -->
<testExclusions>**/e2e/*.java</testExclusions>
<!--
Exclusion expression for the two Docker-dependent test packages: the legacy e2e suites
over the test-harness submodule, and the Provider TCK adoption in tck. Two directories
rather than one because they answer different questions and are run by different
profiles; see the `e2e` and `tck` profiles below, which each drop one of the two.
-->
<testExclusions>**/e2e/*.java,**/tck/*.java</testExclusions>
<io.grpc.version>1.82.0</io.grpc.version>
<!-- caution - updating this will break compatibility with older protobuf-java versions -->
<protobuf-java.min.version>3.25.6</protobuf-java.min.version>
<com.vmlens.version>1.2.28</com.vmlens.version>
<!-- Transitive flagd-core version -->
<flagd-core.version>[2.0.0,3.0.0)</flagd-core.version>
<!-- Match any tck version locally; CI resolves it from the reactor -->
<tck.version>[0.1.0,)</tck.version>
</properties>

<name>flagd</name>
Expand Down Expand Up @@ -98,6 +105,17 @@
<version>5.14.3</version>
<scope>test</scope>
</dependency>
<!--
OpenFeature Provider TCK. Brings its own Gherkin, step definitions and Compose
lifecycle; the three classes under src/test/java/.../flagd/tck are the whole
adoption. Version range so a local reactor build matches whatever is checked out.
-->
<dependency>
<groupId>dev.openfeature.contrib.tools</groupId>
<artifactId>tck</artifactId>
<version>${tck.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
Expand Down Expand Up @@ -250,8 +268,24 @@
<!-- this profile handles running the flagd e2e tests -->
<id>e2e</id>
<properties>
<!-- run the e2e tests by clearing the exclusions -->
<testExclusions/>
<!--
Run the legacy flagd e2e suites (Run*Test, over the test-harness submodule) by
dropping their directory from the exclusions and keeping tck's, so the Provider
TCK suites stay out.

Clearing the property outright would run the conformance suites in CI, because
ci.yml's `main` job activates this profile (it passes `e2e` alongside the matrix
profile) and a GitHub runner does have a Docker daemon. That is not what we
want: those two suites are expected to fail on flags the pinned flagd-testbed
image does not serve plus one recorded provider defect, so running them in the
default build would make every unrelated pull request red. They are run by hand
before merge instead. See providers/flagd/README.md.

This used to read `**/e2e/*TckTest.java`, a filename pattern, because both kinds
of suite shared a directory. They no longer do, and a profile naming a directory
cannot be defeated by a class someone names badly.
-->
<testExclusions>**/tck/*.java</testExclusions>
</properties>
<build>
<plugins>
Expand Down Expand Up @@ -300,6 +334,54 @@
</plugins>
</build>
</profile>
<profile>
<!--
This profile runs the Provider TCK conformance suites, and nothing else:

mvn -Ptck -pl providers/flagd test

Do not add `-am`: it pulls tools/tck and tools/flagd-core into the reactor and
runs their own suites first, which puts two kinds of failure back on one signal.
A one-off `mvn -pl tools/tck -am -DskipTests install` is what it was there for.

It is a step of its own rather than a corner of the `e2e` profile above because
of what a red build says. `-Pe2e` red means the provider's own end-to-end suites
regressed; `-Ptck` red means conformance failed — and a conformance run carries
failures by design, wherever AbstractResolverTest declares a knownDeviation.
Sharing one signal between "you broke something" and "this is the known state"
ends with somebody silencing the informative half. See Appendix F, "Running the
suite in CI".

Nothing activates this profile in CI, deliberately, for the reason the `e2e`
profile's comment gives: these suites are expected to fail today. They are run by
hand before merge. See providers/flagd/README.md.
-->
<id>tck</id>
<properties>
<!--
Drop tck from the exclusions, the mirror image of what the `e2e` profile does,
and keep e2e excluded. Both halves are needed: the include below alone leaves
the default exclusion in force and runs nothing, and dropping the exclusion
alone runs the module's unit tests alongside the conformance suites.
-->
<testExclusions>**/e2e/*.java</testExclusions>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<!-- The whole package: RpcTest and InProcessTest, plus the
abstract parent they share, which Surefire does not run. -->
<include>**/tck/*.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>

</project>
Loading
Loading