feat: add the provider conformance assets

.github/workflows/release-please.yaml

@@ -0,0 +1,35 @@
+name: Run Release Please
+
+# Releases everything versioned in this repository. Two packages are configured in
+# release-please-config.json, each with its own release pull request:
+#
+#   .                                 the specification, tagged vX.Y.Z as it always has been
+#   specification/assets/provider-tck the provider conformance assets, tagged with their path
+#
+# The two are independent. The specification package excludes the conformance assets, so a
+# change confined to the suite never bumps the specification; the assets carry their own
+# version because four language suites pin a revision of them, and a pinned revision needs a
+# name a human can read and a bot can compare.
+#
+# This repository enforces DCO, so the bot's own commits need a sign-off. That is the signoff
+# key in release-please-config.json: v5 takes it from the configuration file, and an action
+# input of the same name is not declared and would be silently ignored.
+
+on:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: write
+  issues: write
+  pull-requests: write
+
+jobs:
+  release-please:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: googleapis/release-please-action@v5
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          target-branch: main

.release-please-manifest.json

@@ -0,0 +1,4 @@
+{
+  ".": "0.9.0",
+  "specification/assets/provider-tck": "0.0.0"
+}

release-please-config.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json",
+  "signoff": "OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>",
+  "tag-separator": "/",
+  "bootstrap-sha": "dd235837f7b2ddb1120ab6e1bde940bf4d532313",
+  "separate-pull-requests": true,
+  "packages": {
+    ".": {
+      "release-type": "simple",
+      "package-name": "spec",
+      "include-component-in-tag": false,
+      "changelog-path": "CHANGELOG.md",
+      "bump-minor-pre-major": true,
+      "versioning": "default",
+      "exclude-paths": ["specification/assets/provider-tck"],
+      "extra-files": []
+    },
+    "specification/assets/provider-tck": {
+      "release-type": "go",
+      "package-name": "specification/assets/provider-tck",
+      "include-component-in-tag": true,
+      "changelog-path": "CHANGELOG.md",
+      "bump-minor-pre-major": true,
+      "versioning": "default",
+      "extra-files": []
+    }
+  }
+}

specification/assets/provider-tck/.gitattributes

@@ -0,0 +1,10 @@
+# These artifacts are consumed byte for byte by every language's provider TCK, and several
+# are copied verbatim into published build artifacts. Normalise to LF so a checkout on
+# Windows does not produce a different packaged file than one on Linux.
+*.feature text eol=lf
+*.json    text eol=lf
+*.yaml    text eol=lf
+
+# embed.go ships inside the Go module zip, whose checksum is content-addressed, so it too
+# must be identical whichever platform it is committed from.
+*.go      text eol=lf

specification/assets/provider-tck/README.md

@@ -0,0 +1,124 @@
+# Provider Conformance Assets
+
+Test assets for the provider conformance suite tracked in
+[open-feature/spec#417](https://github.com/open-feature/spec/issues/417).
+
+**These assets ship ahead of their normative description.** Appendix F, which states what a TCK
+implementation must do and what each capability asserts, is a separate pull request. Until it
+lands this file is the reference for the capability vocabulary, and everything here is
+**experimental**: the scenario set is a representative subset and the vocabulary may still change.
+
+These validate a **provider** against a real backend. For assets that validate an **SDK**, see [`../gherkin/`](../gherkin/README.md) and [Appendix B](../../appendix-b-gherkin-suites.md).
+
+## Contents
+
+| Path | What it is |
+| --- | --- |
+| [`gherkin/evaluation.feature`](./gherkin/evaluation.feature) | resolving each type with the right value and reason; the variant where the backend names one; falsy values; integer precision |
+| [`gherkin/errors.feature`](./gherkin/errors.feature) | the type-mismatch matrix, numeric coercion, string typing and the unknown-flag case |
+| [`gherkin/events.feature`](./gherkin/events.feature) | configuration change, and the stale/ready transition across an outage |
+| [`gherkin/lifecycle.feature`](./gherkin/lifecycle.feature) | initialisation against a healthy backend and against an unreachable one; shutdown |
+| [`gherkin/metadata.feature`](./gherkin/metadata.feature) | the provider identifies itself by name |
+| [`gherkin/reason.feature`](./gherkin/reason.feature) | the standard resolution reasons, gated behind `@standard-reasons` |
+| [`flags/canonical-flags.json`](./flags/canonical-flags.json) | the flag set every scenario assumes |
+| [`openapi/control-api.yaml`](./openapi/control-api.yaml) | the HTTP surface a backend under test must expose |
+
+## Capabilities: how a provider says what it cannot do
+
+Not every provider implements every optional part of the contract. A scenario exercising one carries
+a tag; a provider declares the tags it supports, and a scenario gated on an undeclared tag is reported
+**skipped, with the reason** — never as passed. A suite that quietly goes green on scenarios it did
+not run is worse than no suite at all, so this rule is the centre of the design rather than a detail.
+
+Tags compose: a scenario carrying two tags runs only if both are declared.
+
+| Tag | Meaning |
+| --- | --- |
+| `@events` | emits lifecycle events at all |
+| `@lifecycle` | performs an initialisation that reaches its backend, with an observable outcome |
+| `@stale` | enters `STALE` and emits `PROVIDER_STALE` on backend loss |
+| `@configuration-change` | detects configuration changes and emits `PROVIDER_CONFIGURATION_CHANGED` |
+| `@object` | supports structured flag values |
+| `@variants` | names the variant it resolved |
+| `@disabled-flags` | resolves a flag disabled in the management system to the code default |
+| `@unavailable` | reports an error state instead of hanging against a dead backend |
+| `@numeric-coercion` | coerces between integer and float only when lossless, else `TYPE_MISMATCH` |
+| `@string-typing` | reports `TYPE_MISMATCH` for a boolean or integer flag requested as a string, rather than its string representation |
+| `@fully-typed-values` | records a native type for float and structured values too, so the same question can be asked of them |
+| `@large-integers` | resolves integers up to 2^53 − 1 exactly |
+| `@reinitialization` | can be initialised again after `shutdown` |
+| `@targeting` | resolves a flag differently for a matching evaluation context |
+| `@standard-reasons` | reports the standard resolution reasons |
+| `@caching` | reserved; **not declarable** — no scenarios carry it yet |
+
+Untagged scenarios are mandatory and always run.
+
+**Withholding a tag is not an admission of a defect.** Some of these describe behaviour the
+specification does not require — `@numeric-coercion` borrows its rule from flagd's coercion ADR, and
+`@string-typing` and `@fully-typed-values` sit on a question the specification does not answer at all
+([#433](https://github.com/open-feature/spec/issues/433), [#430](https://github.com/open-feature/spec/issues/430)).
+A provider that withholds one of those is not violating the specification, and this suite must not be
+read as saying it is.
+
+## These three travel together
+
+A feature file that evaluates `boolean-flag` is meaningless without the flag definition, and a disconnect scenario is meaningless without the control endpoint that produces the disconnect. Changing one without the others breaks the suite in every language at once.
+
+## Five properties that are load-bearing
+
+- **`missing-flag` must not exist** in the flag set. Its absence is what the `FLAG_NOT_FOUND` scenario tests. Seeding it turns that scenario green for the wrong reason.
+- **Only `targeting-key-flag` has a targeting rule.** Every other enabled flag resolves to its default variant whatever the evaluation context, which is what lets the untargeted scenarios expect reason `STATIC`. Seeding targeting onto any other flag breaks them in every language at once. Its rule is specified by behaviour — resolve `hit` when the targeting key is exactly `5c3d8535-f81a-4478-a6d3-afaa4d51199e`, `miss` otherwise — so express it however your backend expresses targeting. The flag, its variants and the uuid are the ones [flagd-testbed's `targeting.feature`](https://github.com/open-feature/flagd-testbed) already uses, on the same reasoning as the zero flags: a backend serving that harness already serves this.
+- **`boolean-zero-flag`, `integer-zero-flag` and `string-zero-flag` resolve to falsy values on purpose.** A seeding step that treats `false`, `0` or `""` as "unset" and drops them turns the falsy-value scenarios into `FLAG_NOT_FOUND` failures that look like provider defects. These names, and their `zero`/`non-zero` variants, are the ones [Appendix B's SDK suite](../gherkin/test-flags.json) already uses, so a backend serving that flag set already serves these.
+- **The four `disabled-*` flags are the only ones whose state is not `ENABLED`.** They resolve to nothing — the caller's default stands in, and no variant is named. Every other scenario assumes a flag serves its own value, so enabling one of these, or disabling anything else, breaks that assumption silently. Names, variants and values are [flagd-testbed's own](https://github.com/open-feature/flagd-testbed), from `flags/disabled-flags.json`.
+- **`integral-float-flag` is a float and `huge-integer-flag` is an integer.** Seeding `10.0` as `10` makes the lossless-coercion scenario pass without coercing; seeding `9007199254740991` through a float rounds it.
+
+The flag set is expressed in the flagd flag-definition format because that is the only widely implemented vendor-neutral format today. The format is not what matters — the keys, types, variant names and resolved values are. Seed them however your backend seeds flags.
+
+## Consuming from Go
+
+This directory is also a Go module, `github.com/open-feature/spec/specification/assets/provider-tck`, whose only content is an `embed.FS` of the artifacts above. The Go conformance suite depends on it instead of vendoring a copy: a Go module ships as a zip of the VCS tree, in which a git submodule is only a gitlink, so an embed from a submodule would arrive empty for anyone running `go get`. The other languages build from a working tree and keep using the submodule; `go.mod` and `embed.go` are inert for them.
+
+A consumer pins a release the usual way:
+
+```console
+go get github.com/open-feature/spec/specification/assets/provider-tck@v0.1.0
+```
+
+## Releases
+
+These assets are released independently of the specification, by [release-please](../../../release-please-config.json). A nested Go module is tagged with its path as a prefix, so a release is tagged `specification/assets/provider-tck/vX.Y.Z` — the same shape as `providers/flagd/v0.6.0` in the SDK contrib repositories. The specification's own `vX.Y.Z` tags are cut by release-please too, from a separate release pull request that excludes this directory, and do not apply here; nothing about the two numbering schemes is related.
+
+What a bump means is not the usual thing, because this is a test suite rather than a library:
+
+- **A minor bump may turn a passing suite red.** Adding a scenario, or tightening one, raises the bar a provider has to clear. Nothing changed on the adopter's side and their build can still go from green to red, which is the point of adopting a conformance suite and is why new scenarios are released as minors rather than as patches.
+- **A patch bump cannot.** Patches are editorial: a clarified scenario name, a comment, a fix to something that never ran.
+
+So pinning is not optional bookkeeping. A suite that floats on the latest assets cannot distinguish a regression in the provider from a new question being asked of it.
+
+## Consuming from the other three languages
+
+Java, Python and JavaScript reach these files through a git submodule of this repository, because a JAR, a wheel and an npm package are all built from a working tree where the submodule is present. A submodule can track the release tag rather than a bare commit, which makes the pin readable in review:
+
+```ini
+[submodule "spec"]
+  path = tools/provider-tck/spec
+  url = https://github.com/open-feature/spec.git
+  branch = specification/assets/provider-tck/v0.1.0
+```
+
+The recorded gitlink is still a commit, so `git submodule update --init` and `actions/checkout` with `submodules: recursive` behave exactly as before. Only `git submodule update --remote` is affected, which resolves `branch` and will report that the tag is not a branch — do not use it on a submodule pinned this way.
+
+## Keeping a pin current
+
+Both forms are updatable by [Renovate](https://docs.renovatebot.com), so an adopting repository is told about a new release rather than discovering it:
+
+- Go: the `gomod` manager, on by default, raises a PR for a new `specification/assets/provider-tck/vX.Y.Z`.
+- The submodule: the `git-submodules` manager, which is opt-in and reads the tag out of `branch` above.
+
+```json
+{
+  "git-submodules": { "enabled": true }
+}
+```
+
+Let those PRs run the suite. A red one is the report that conformance narrowed, and reading it is the work — which is why it is worth *not* automerging these.

specification/assets/provider-tck/embed.go

@@ -0,0 +1,35 @@
+// Package providertck carries the provider conformance assets as a Go module,
+// so a Go conformance suite can depend on a specific revision of them the way
+// it depends on any other module.
+//
+// The other language suites consume this directory through a git submodule,
+// which works because a wheel or a JAR is built from a working tree where the
+// submodule is present. A Go module is distributed as a zip built from the
+// VCS tree, where a submodule is only a gitlink and its files are absent, so
+// the Go suite would otherwise have to commit a copy of every artifact and
+// police it against drift. Publishing the artifacts as a module removes the
+// copy: the consumer pins a commit or tag in its go.mod, the Go checksum
+// database makes that revision immutable, and the embedded bytes are the same
+// bytes every other language reads out of the submodule.
+//
+// The module contains no code beyond this file and has no dependencies. It is
+// inert for every consumer that is not Go. See
+// https://github.com/open-feature/spec/issues/417.
+package providertck
+
+import "embed"
+
+// FS holds the conformance artifacts, keyed by their path relative to this
+// directory, so that they are addressed here exactly as they are documented:
+//
+//	gherkin/*.feature          the canonical scenarios
+//	flags/canonical-flags.json the flag set those scenarios assume
+//	openapi/control-api.yaml   the HTTP surface a backend under test exposes
+//
+// The README and .gitattributes beside them are not artifacts and are not
+// embedded.
+//
+//go:embed gherkin/*.feature
+//go:embed flags/canonical-flags.json
+//go:embed openapi/control-api.yaml
+var FS embed.FS