Skip to content
1,802 changes: 981 additions & 821 deletions corpus/accepted.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/limits-acks.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@
"A public address is the provider's value, made to answer on the host": "2026-08-27",
"A run presented as local can still reach the real cloud (#280)": "2026-08-27",
"An API reboot used to log `Failed to add route: file exists` for its own public /32 (#498, lifted 2026-08-27)": "2026-08-27",
"An ERROR is a failure and a WARN is a decline, and one refusal was on the wrong side (#474)": "2026-08-28",
"An Exoscale network load balancer records its configuration, names its backends, and grades none of them": "2026-08-27",
"An Outscale Vm's options round-trip as data; their behavioural half has nothing to act on here": "2026-08-27",
"An Outscale load balancer distributes packets inside its network, and nowhere else": "2026-08-27",
"An Outscale machine owns a root volume, and that volume holds no bytes": "2026-08-27",
"Exoscale has one zone per process, and the reason is the client": "2026-08-27",
"An ERROR is a failure and a WARN is a decline, and one refusal was on the wrong side (#474)": "2026-08-28",
"Identifiers are not checked against anything": "2026-08-27",
"Lifecycle transitions are immediate": "2026-08-27",
"Managed Kubernetes is not emulated, and a CRUD-only version is refused (#283)": "2026-08-27",
Expand All @@ -35,6 +35,6 @@
"Two Outscale filters reach the API only as a payload, and that is `octl`'s gap": "2026-08-27",
"What survives a dead emulator, in one table": "2026-08-27",
"`feint images resolve` can print a `FEINT_BOOT_IMAGES` line that cannot boot (#476)": "2026-08-27",
"`scw` 2.56.3 prints a recovered panic on every successful `lb acl delete`, and the defect is upstream (#505)": "2026-08-27"
"`scw` 2.56.3 prints a recovered panic on every successful `lb acl delete`, and the defect is upstream (#505)": "2026-08-28"
}
}
97 changes: 79 additions & 18 deletions docs/limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -2614,9 +2614,15 @@ real cloud part ways:
snapshot for the flag to reveal. The state filter is real code on the
default path; the difference from the real cloud, which retains deleted
volumes for a while, is this line.
- **`s3_integration_enabled=true` matches nothing.** No VPC or Private Network
here integrates with Object Storage, which is not emulated (see above), so
true truthfully answers an empty list and false answers everything.
- **`object_storage_private_access_enabled=true` matches nothing**, and so does
`s3_integration_enabled=true`, which is the name Scaleway retired on
2026-08-25 and this filter still accepts (#570): a client that has not been
rebuilt keeps sending it. No VPC or Private Network here integrates with
Object Storage, which is not emulated (see above), so true truthfully answers
an empty list and false answers everything. The *body* answers the new name
alone — both upstream sources declare only that one, and inventing a
deprecation alias in an answer is not the same decision as tolerating a name
a client still sends in a request.
- **`arch` and `type` on marketplace ListLocalImages are equalities against
the one published image.** `arch=arm64` or `type=instance_local` answers an
empty list — the catalogue is x86_64 and `instance_sbs` — where dropping the
Expand Down Expand Up @@ -3278,11 +3284,13 @@ $ incus query /1.0/instances/feint-scw-d3eaa40c-… | jq -c '.expanded_devices |
packet reached the guest and was refused by it, where a covered interface
would have dropped it.

**The migration was tried, and it is refused.** #548 left one thing untried —
whether the driver could move the address onto the managed NIC once that one
arrives, which is the shape the other creation order already produces and the
one the rule set covers. It was attempted by hand on 2026-08-27, on the very
machine above, and stopped on two measured facts:
**Two migrations were tried and refused, and a third one works.** #548 left one
thing untried — whether the driver could move the address onto the managed NIC
once that one arrives, which is the shape the other creation order already
produces and the one the rule set covers. Three attempts, and the third is the
reason this paragraph changed on 2026-08-28.

The first two, by hand on 2026-08-27:

```console
$ incus network set feint-uplink ipv4.routes "…,203.0.113.2/32"
Expand All @@ -3293,16 +3301,69 @@ $ incus query /1.0/instances/feint-scw-d3eaa40c-… | jq -c '…'
```

The uplink cannot be given the `/32` while the routed NIC still owns the host
route for it — the collision #498 documents, met from the other side — so the
address would have to leave the routed NIC first, and there is no ordering
where it is delivered throughout. And removing the routed device unmasks the
profile's `eth0` on `incusbr0`, the operator's own default bridge: the machine
lands on a bridge this emulator refuses to put anything on. A sequence that
gets there exists on paper (mask `eth0` with a `none` device, then delegate,
then set `ipv4.routes.external`, then repair the guest) and it costs a
reconfiguration of the public interface on every private-NIC attach, which
Terraform performs on every apply. That trade was not taken; this paragraph is
the record of the measurement rather than a plan.
route for it — the collision #498 documents, met from the other side — and
removing the routed device unmasks the profile's `eth0` on `incusbr0`, the
operator's own default bridge, which this emulator refuses to put anything on.

Both come from one place: the host route. **The third attempt takes the address
off the device without taking the device off the instance**, which is neither of
the two, and Incus 7.2 accepts it on a running instance. Measured on 2026-08-28,
under `--vm incus-ovn`, on a server created with its flexible IP whose private
NIC arrived afterwards — the exact shape above:

```console
$ ip route show | grep 203.0.113.2
203.0.113.2 dev veth030d08f6 scope link
$ incus config device set feint-scw-21c968ca-… eth0 ipv4.address= # 1
$ ip route show | grep 203.0.113.2 # gone
$ incus network set feint-uplink ipv4.routes "10.199.0.0/24,203.0.113.2/32" # 2, no longer refused
$ incus config device set feint-scw-21c968ca-… eth1 \
ipv4.routes.external=203.0.113.2/32 # 3
$ incus query /1.0/instances/feint-scw-21c968ca-… | jq -c '…'
{"eth0":{"ipv4.host_address":"169.254.0.1","nictype":"routed","type":"nic"},
"eth1":{"ipv4.address":"10.199.0.2","ipv4.routes.external":"203.0.113.2/32",
"network":"fnt-0da7a7bda1e","security.acls":"scw-892dbc3d91e","type":"nic"}}
```

Step 1 releases the address, step 2 is the first refusal now unblocked, step 3
is `routeAddressOVN`'s own gesture, and the guest is then repaired the way that
function already repairs one. The device stays, so `incusbr0` is never unmasked.

**The coverage is real, and the probe tells the two answers apart.** The group's
inbound default is `drop` with one rule allowing 443, and a listener sits on both
ports, so a refusal cannot be mistaken for an empty port:

```console
203.0.113.2:443 connect_ex=0 OPEN # a rule opens it, the listener answers
203.0.113.2:80 connect_ex=111 refused # a listener is there, no rule is
203.0.113.2:8080 connect_ex=111 refused # no listener: the negative control
```

Before the migration, on the same machine, 80 was **OPEN** — that is the escape
this section describes. And the machine keeps its way out: `ipv4.nat=true` on the
OVN network, `ping 1.1.1.1` answers from inside, with the station as the control.

**So this is a remedy, not an impossibility — and it is still not shipped, for
two reasons that are measurements rather than reluctance.**

- **What a machine "answers on" moves.** `Incus.Inspect` reports the first
global IPv4 of the lowest-named interface, and after the migration the public
and private addresses share `eth1`: the runtime answers
`{"eth0":[],"eth1":["10.199.0.2","203.0.113.2"]}`, stable across three reads,
so `Binding.Address` and `Started.Address` would report the private address
where they used to report the public one. The Scaleway API is unaffected —
it publishes the flexible IP from its own store, and it still answered
`public_ip: 203.0.113.2` throughout — but `Binding.Address` is the shared
layer, and the Exoscale pack reads it (`machines.go`, the membership
attachment). A fix has to say which of a machine's addresses is the one it
answers on, for three packs at once.
- **Only OVN was measured.** The bridge mode delivers a public address through
`ipv4.routes` on the device rather than `ipv4.routes.external`, and nothing
above was run under `--vm incus`.

Until those two are settled the bound in the table stands, and
`tools/conformance/functional.sh` goes on skipping the public half by naming
`capabilities.firewall_public_only` and #548.

What #548 delivered instead is the naming: the refusal now carries every
routed interface that escapes *and the addresses it delivers*
Expand Down
10 changes: 9 additions & 1 deletion internal/cli/discipline_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,15 @@ var notInTheBarrage = map[string]string{
// declared by its author. A pack that skips one names it in notInTheBarrage with
// a reason, which is a line somebody has to write and a reviewer can read.
func TestEveryPackRunsTheSharedBarrage(t *testing.T) {
controls := []string{"Sweep", "NoLostUpdate", "Orphans"}
// GoShapes joined on 2026-08-28 (#567), and it is the member that shows why
// the list is discovered rather than declared: the issue stated the three
// real packs were immune because they already store the JSON shape, and the
// first run of this control over Scaleway's own barrage reported 82
// resources carrying a []string in Attrs["tags"]. Nothing was broken by it,
// because two files carry a hand-written type switch tolerating both
// shapes — which is the habit, and the habit is what a fourth pack cannot
// inherit.
controls := []string{"Sweep", "NoLostUpdate", "Orphans", "GoShapes"}

for _, dir := range packDirs(t) {
pack := filepath.Base(dir)
Expand Down
119 changes: 117 additions & 2 deletions internal/cli/provider_four_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -896,6 +896,119 @@ func TestTheFourthPacksSpreaderKeepsItsPortAcrossASnapshot(t *testing.T) {
}
}

// A restored node still wears its barriers, joins its segments and knows its
// addresses (#567).
//
// The numeric half of this is the test above, and it is the half a shared
// reader could fix: a number that crossed JSON has exactly one right answer,
// and resource.Number gives it. These three have none that internal/core may
// give — recovering a []Rule means knowing Rule, which is the pack's own type
// (rule 5) — so the pack stores the shape the door returns instead.
//
// Measured on 2026-08-27, before the fix, through store.Snapshot then
// store.Restore into a fresh store: []Rule came back nil, []string came back
// []any, map[string]string came back map[string]any. Every one of the pack's
// readers asserted the Go type, so a restored node wore no barriers, joined no
// segments and had no address, while the API went on describing all three.
//
// Behaviour rather than shape, deliberately: this asserts what the runtime is
// asked to do with the restored node, not what its Attrs look like. A test
// that compared the maps would pass on a pack that stored the right shape and
// read it back with the wrong assertion.
func TestTheFourthPacksNodeKeepsWhatItWearsAcrossASnapshot(t *testing.T) {
ctx := context.Background()
pack, _, env := fourthPack(t)

home, err := pack.CreateSegment(ctx, "front", "10.40.0.0/24", "green")
must(t, err)
later, err := pack.CreateSegment(ctx, "back", "10.41.0.0/24", "green")
must(t, err)
barrier := pack.CreateBarrier("web")
must(t, pack.AddRule(ctx, barrier.ID, providerfour.Rule{
Direction: "ingress", Action: "allow", Protocol: "tcp", Source: "0.0.0.0/0",
PortFrom: 443, PortTo: 443,
}))
node, err := pack.CreateNode(ctx, providerfour.NodeRequest{
Name: "web-1",
Image: "four-linux",
HomeSegment: home.ID,
Barriers: []string{barrier.ID},
})
must(t, err)
must(t, pack.StartNode(ctx, node.ID))
must(t, pack.JoinSegment(ctx, node.ID, later.ID))

// The door a snapshot really travels: the format is documented as meant to
// outlive its instance and be loaded into another one.
var saved bytes.Buffer
if err := env.Store.Snapshot(&saved); err != nil {
t.Fatalf("snapshot: %v", err)
}
restored := store.New()
if err := restored.Restore(bytes.NewReader(saved.Bytes())); err != nil {
t.Fatalf("restore: %v", err)
}

// A reboot on the revived pack replays the whole post-boot order, which is
// what reads all four stored collections at once.
rec := machine.NewRecorder()
next := fourthEnv(t, machine.Use(rec))
next.Store = restored
revived := providerfour.New(next)
must(t, revived.RebootNode(ctx, node.ID))

var booted machine.Spec
var attached []machine.Attachment
var binding machine.FirewallBinding
var spec machine.FirewallSpec
for _, event := range rec.Events() {
switch event.Kind {
case "Start":
booted, _ = event.Args.(machine.Spec)
case "Attach":
if att, ok := event.Args.(machine.Attachment); ok {
attached = append(attached, att)
}
case "ApplyFirewall":
binding, _ = event.Args.(machine.FirewallBinding)
case "EnsureFirewall":
spec, _ = event.Args.(machine.FirewallSpec)
}
}

// addresses: the home segment rides the launch, at the address the store
// promised for it.
if len(booted.Attachments) != 1 || booted.Attachments[0].Address != "10.40.0.10" {
t.Errorf("the revived node booted with %v, want one interface at 10.40.0.10: a restored "+
"map[string]string read as map[string]string is empty, so the interface comes up "+
"with no address while the API still publishes one", booted.Attachments)
}

// segments: and the membership joined afterwards is joined again.
if len(attached) != 1 || attached[0].Address != "10.41.0.10" {
t.Errorf("the revived node was attached to %v, want the second segment at 10.41.0.10: a "+
"restored []string read as []string is nil, so the node joins nothing while the API "+
"still lists its segments", attached)
}

// barriers: the rule set the node wears reaches the runtime with it.
if len(binding.Names) != 1 {
t.Errorf("the revived node was bound to %d rule set(s), want 1: a restored []string read "+
"as []string is nil, so the machine wears nothing while the API still says it does "+
"— %v", len(binding.Names), binding.Names)
}

// rules: and the set itself still carries what was declared into it.
if len(spec.Rules) != 1 {
t.Fatalf("the revived rule set carries %d rule(s), want 1: a restored []Rule read as "+
"[]Rule is nil, so an empty set is handed over under a name the API describes as "+
"filtering — %v", len(spec.Rules), spec.Rules)
}
if spec.Rules[0].PortFrom != 443 || spec.Rules[0].Source != "0.0.0.0/0" {
t.Errorf("the revived rule is %+v, want the 443 rule that was declared", spec.Rules[0])
}
}

// ---- The fourth pack is a resource like any other ---------------------------

// The fourth pack's own store use obeys the shared write-back.
Expand Down Expand Up @@ -933,7 +1046,9 @@ func TestTheFourthPacksNestedAttributesAreNeverWrittenThroughTheStore(t *testing
if !found {
t.Fatal("the node is gone")
}
held, _ := stored.Attrs["addresses"].(map[string]string)
// map[string]any, because that is the shape the pack stores since #567 —
// what this measures is the map's aliasing, not its element type.
held, _ := stored.Attrs["addresses"].(map[string]any)
if len(held) != 1 {
t.Fatalf("the node carries %d address(es) on one segment, want 1: %v", len(held), held)
}
Expand All @@ -949,7 +1064,7 @@ func TestTheFourthPacksNestedAttributesAreNeverWrittenThroughTheStore(t *testing
if !found {
t.Fatal("the node is gone")
}
if addresses, _ := after.Attrs["addresses"].(map[string]string); len(addresses) != 2 {
if addresses, _ := after.Attrs["addresses"].(map[string]any); len(addresses) != 2 {
t.Errorf("the store holds %d address(es) after a second segment was joined", len(addresses))
}
}
Loading
Loading