From 0600044a9fe1a143c0f3a8549a602135c2a43545 Mon Sep 17 00:00:00 2001 From: Jason Patterson Date: Sun, 6 Sep 2026 01:45:03 +0000 Subject: [PATCH 1/2] SONiC containerlab: name the endpoints ethN so docker-sonic-vs maps its ports docker-sonic-vs builds lanemap.ini/port_config.ini from the ethN netdevs present at boot and syncd then creates a TAP named Ethernet<4n> per mapped port, bridged to eth -- the contract containerlab documents for the sonic-vs kind. The device declared no clab.interface.name, so containerlab injected the veth as Ethernet0 itself: the lane map came out empty, syncd failed to create every TAP (TUNSETIFF: name in use), and SONiC port state and admin control were detached from the link. "show interfaces status" reported oper/admin down on a port carrying a Full OSPF adjacency, and "config interface shutdown" changed nothing. Declare clab.interface.name eth{ifindex + 1} (ifindex starts at 0 here, eth0 is management), as eos (et{ifindex}) and srlinux (e1-{ifindex}) do. With the port a syncd TAP rather than the raw veth, CONFIG_DB reaches the netdev again: the MTU goes back through "config interface mtu" (measured on a 9000-byte link -- netdev, CONFIG_DB and APPL_DB all agree, an 8972-byte DF ping crosses) and the loop that mirrored CONFIG_DB INTERFACE rows onto the netdev with "ip addr add" is gone, because intfmgrd programs routed-port addresses. portmgrd still leaves every front-panel port admin-down at boot, so "config interface startup" stays, now issued for the ports the lab uses and under the script's set -e. The vlan, lag and vxlan datapaths are out of scope: those features are off on this device (vlan: False, lag: False, vxlan: false) and were not re-measured. A node now has two netdevs per link, and the kernel answers ARP for the port's address on both of them, each with its own MAC. The neighbour caches whichever replies first, so the port's address becomes reachable through a MAC that is not the port's, and anything keyed on the port's MAC is left to chance -- measured, "ip neigh" on the neighbour showed the veth's MAC, and with the sysctl it shows the TAP's. arp_ignore=8 on each link's veth, set before the ports are addressed, leaves the port as the only answerer. This surfaced on our fork as an SR-MPLS L3VPN ping at 100% loss, MPLS input being enabled on the port only; upstream's sonic device does not enable MPLS. The transformation case pins the container name on the node interface and, by covering the whole transformed topology, also pins that the node keeps its own config_templates rather than inheriting FRR's bind mounts. Co-Authored-By: Claude Fable 5.1 --- .../ansible/templates/initial/sonic-clab.j2 | 38 +++--- netsim/ansible/templates/vxlan/sonic-clab.j2 | 2 +- netsim/devices/sonic.yml | 5 + .../topology/expected/sonic-clab-ifnames.yml | 127 ++++++++++++++++++ tests/topology/input/sonic-clab-ifnames.yml | 11 ++ 5 files changed, 163 insertions(+), 20 deletions(-) create mode 100644 tests/topology/expected/sonic-clab-ifnames.yml create mode 100644 tests/topology/input/sonic-clab-ifnames.yml diff --git a/netsim/ansible/templates/initial/sonic-clab.j2 b/netsim/ansible/templates/initial/sonic-clab.j2 index ffcb3b1a27..6fa3bc2908 100644 --- a/netsim/ansible/templates/initial/sonic-clab.j2 +++ b/netsim/ansible/templates/initial/sonic-clab.j2 @@ -66,6 +66,18 @@ pgrep -x sshd >/dev/null || /usr/sbin/sshd config interface ipv6 disable use-link-local-only {{ l.ifname }} {% endif %} {% endfor %} +# +# Every port is two kernel interfaces: the port TAP (Ethernet<4n>, addressed below) and its +# wire veth (ethN, clab.name). Only the port may answer ARP, or neighbours unicast to the +# veth and labelled traffic is dropped there (MPLS input is on the port only). Same for LAG +# members and VLAN ports, whose PortChannel/SVI address the veth would otherwise answer for. +# 8, not 1: the veth must never answer for any address, including ones added later. IPv6 +# needs no equivalent -- Linux answers a neighbour solicitation only for addresses on the +# receiving interface. +# +{% for l in interfaces if l.clab.name is defined %} +sysctl -w net.ipv4.conf.{{ l.clab.name }}.arp_ignore=8 +{% endfor %} # Create config_db PortChannels (lag) and VLANs (vlan) before any aggregate/SVI is addressed # below. These live in the per-module init hooks lag/vlan sonic.initial.j2 and are pulled # in here, at the correct ordering point, by netlab's extra_module_initial() macro. @@ -109,33 +121,21 @@ else fi {% endif %} {% endif %} -{# Set the MTU on the kernel netdev, not through "config interface mtu": on the interfaces - containerlab injects for netlab links that command writes CONFIG_DB and never reaches the - netdev, exactly like the routed-port IPs below -- measured, the interface stayed at 9500 - while CONFIG_DB claimed otherwise. The value is netlab MTU with no adjustment: - docker-sonic-vs programs it literally (1548 in -> 1548 on the netdev), so the +48 this - used to add left every interface 48 bytes above its peers and stalled OSPF in ExStart. #} +{# The port is the TAP syncd manages, so portmgrd programs the CONFIG_DB MTU onto the netdev. #} {# L2 switchports (LAG members and VLAN access/trunk ports) reject a direct MTU set once they are members ("'interface_name' is in portchannel!" / "is in vlan"); the PortChannel/port MTU propagates, so skip them #} {% if l.mtu is defined and l.mtu >= 1500 and not (l.virtual_interface|default(False)) and l.lag._parentindex is not defined and l.vlan.access_id is not defined and l.vlan.trunk_id is not defined %} -ip link set {{ l.ifname }} mtu {{ l.mtu }} +config interface mtu {{ l.ifname }} {{ l.mtu }} {% endif %} ! {% endfor %} # -# docker-sonic-vs's intfmgrd/orchagent do not program routed-port IPs from CONFIG_DB onto the -# kernel netdev (loopbacks work; routed ports do not), so BGP/OSPF over links can't come up -# without this. Bring the data ports admin-up and mirror the CONFIG_DB INTERFACE IPs onto the -# kernel netdev. Idempotent. -# -for p in $(redis-cli -n 4 keys "PORT|Ethernet*" 2>/dev/null | sed 's/PORT|//'); do - config interface startup "$p" >/dev/null 2>&1 || true -done -redis-cli -n 4 keys "INTERFACE|Ethernet*|*" 2>/dev/null | while IFS= read -r k; do - ifc=$(echo "$k" | cut -d'|' -f2); addr=$(echo "$k" | cut -d'|' -f3) - [ -n "$addr" ] && ip addr add "$addr" dev "$ifc" 2>/dev/null || true -done +# portmgrd brings every front-panel port up admin-down at boot; start the ones this lab uses. +# +{% for l in interfaces if l.clab.name is defined %} +config interface startup {{ l.ifname }} +{% endfor %} # # Enable the FRR daemons needed by the configured modules (docker-sonic-vs ships most of them # disabled) and restart FRR once, only if something actually changed. diff --git a/netsim/ansible/templates/vxlan/sonic-clab.j2 b/netsim/ansible/templates/vxlan/sonic-clab.j2 index 69738dd449..796098ffe6 100644 --- a/netsim/ansible/templates/vxlan/sonic-clab.j2 +++ b/netsim/ansible/templates/vxlan/sonic-clab.j2 @@ -8,7 +8,7 @@ # SONiC EVPN-VXLAN L2VNI — FRR/kernel data-plane. # # docker-sonic-vs's orchagent/vxlanmgrd do NOT program the VXLAN dataplane (same class -# of gap as routed-port IPs and VLAN members), so the config_db VXLAN_TUNNEL model gives +# of gap as VLAN members), so the config_db VXLAN_TUNNEL model gives # no datapath on the VS. Instead we build the L2VNI exactly the way FRR/Cumulus does on # plain Linux — a traditional per-VNI bridge holding the access port(s) + a kernel vxlan # netdev — and let FRR zebra drive EVPN off that kernel state (advertise-all-vni). This diff --git a/netsim/devices/sonic.yml b/netsim/devices/sonic.yml index 76b6c85916..6ea18fe7d8 100644 --- a/netsim/devices/sonic.yml +++ b/netsim/devices/sonic.yml @@ -89,6 +89,11 @@ clab: # key-value passthrough loop, so this device needs no provider template of its own. image: docker-sonic-vs:latest mtu: 1500 + # docker-sonic-vs maps its front-panel ports from the ethN netdevs present at boot + # (eth1 -> Ethernet0): the containerlab endpoint is ethN, Ethernet<4n> is the SONiC-side + # name netlab configures. eth0 is management and ifindex starts at 0, hence the +1. + interface: + name: "eth{ifindex + 1}" node: kind: sonic-vs # Do NOT inherit FRR's bind-mounted /etc/frr/daemons and /etc/hosts config templates: diff --git a/tests/topology/expected/sonic-clab-ifnames.yml b/tests/topology/expected/sonic-clab-ifnames.yml new file mode 100644 index 0000000000..1e485c9496 --- /dev/null +++ b/tests/topology/expected/sonic-clab-ifnames.yml @@ -0,0 +1,127 @@ +groups: + netlab_no_reload: + members: + - s1 + - s2 +input: +- topology/input/sonic-clab-ifnames.yml +- package:topology-defaults.yml +links: +- _linkname: links[1] + interfaces: + - ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.1/30 + node: s1 + - ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.2/30 + node: s2 + linkindex: 1 + node_count: 2 + prefix: + ipv4: 10.1.0.0/30 + type: p2p +name: input +nodes: + s1: + _node_config: + initial: /etc/config/01-initial.sh:sh + af: + ipv4: true + box: docker-sonic-vs:latest + clab: + binds: + - source: node_files/s1/initial + target: /etc/config/01-initial.sh + config_templates: + - mode: sh + source: initial + target: /etc/config/01-initial.sh + exec: + - sleep 1 + kind: sonic-vs + device: sonic + hostname: clab-input-s1 + id: 1 + interfaces: + - clab: + name: eth1 + ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.1/30 + linkindex: 1 + mtu: 1500 + name: s1 -> s2 + neighbors: + - ifname: Ethernet0 + ipv4: 10.1.0.2/30 + node: s2 + type: p2p + loopback: + ifindex: 0 + ifname: Loopback0 + ipv4: 10.0.0.1/32 + neighbors: [] + type: loopback + virtual_interface: true + mgmt: + ifname: eth0 + ipv4: 192.168.121.101 + mac: ca:fe:00:01:00:00 + mtu: 1500 + name: s1 + netlab_ansible_skip_module: + - initial + role: router + s2: + _node_config: + initial: /etc/config/01-initial.sh:sh + af: + ipv4: true + box: docker-sonic-vs:latest + clab: + binds: + - source: node_files/s2/initial + target: /etc/config/01-initial.sh + config_templates: + - mode: sh + source: initial + target: /etc/config/01-initial.sh + exec: + - sleep 1 + kind: sonic-vs + device: sonic + hostname: clab-input-s2 + id: 2 + interfaces: + - clab: + name: eth1 + ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.2/30 + linkindex: 1 + mtu: 1500 + name: s2 -> s1 + neighbors: + - ifname: Ethernet0 + ipv4: 10.1.0.1/30 + node: s1 + type: p2p + loopback: + ifindex: 0 + ifname: Loopback0 + ipv4: 10.0.0.2/32 + neighbors: [] + type: loopback + virtual_interface: true + mgmt: + ifname: eth0 + ipv4: 192.168.121.102 + mac: ca:fe:00:02:00:00 + mtu: 1500 + name: s2 + netlab_ansible_skip_module: + - initial + role: router +provider: clab diff --git a/tests/topology/input/sonic-clab-ifnames.yml b/tests/topology/input/sonic-clab-ifnames.yml new file mode 100644 index 0000000000..771f864100 --- /dev/null +++ b/tests/topology/input/sonic-clab-ifnames.yml @@ -0,0 +1,11 @@ +# +# docker-sonic-vs maps its front-panel ports from the ethN netdevs present at boot +# (eth1 -> Ethernet0), so the containerlab endpoint name must be ethN. +# +provider: clab +nodes: + s1: + device: sonic + s2: + device: sonic +links: [ s1-s2 ] From ef8f6be51e0d1c7993d28b77085d19ac7587f443 Mon Sep 17 00:00:00 2001 From: Jason Patterson Date: Sun, 6 Sep 2026 01:45:03 +0000 Subject: [PATCH 2/2] SONiC containerlab: document the two-interface port layout A containerlab SONiC node has two kernel interfaces per link: the wire veth ethN that containerlab creates and the port Ethernet<4n> that syncd bridges to it. Only the port is configured and addressed, and netlab suppresses ARP on the veth so neighbours resolve the port's MAC. Co-Authored-By: Claude Fable 5.1 --- docs/caveats.md | 1 + docs/release/26.08.md | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/caveats.md b/docs/caveats.md index 34a2ce4cc8..3161f60bdc 100644 --- a/docs/caveats.md +++ b/docs/caveats.md @@ -719,6 +719,7 @@ See also [](caveats-sros) caveats for further details. The `sonic` device also runs under *containerlab* with the community `docker-sonic-vs` image; see [](build-sonic-container) for how to obtain it and how the two deployments differ. +* Every port is two kernel interfaces: the wire veth `ethN` and the port `Ethernet<4n>` that SONiC creates for it. Configure and address only the port; _netlab_ suppresses ARP on the veth. * Configuration is deployed with **docker exec** commands, not over an SSH session. * `docker-sonic-vs` ships `sshd` but does not start it. * `srv6` is control-plane and kernel-plane only: the locator and End/End.X SIDs are advertised in the IS-IS LSDB and installed as kernel `seg6local` routes, but the end-to-end SRv6 datapath does not resolve -- the same open item as FRR/IS-IS SRv6 on other platforms. diff --git a/docs/release/26.08.md b/docs/release/26.08.md index b0e9d29f82..eaeb3fe9d0 100644 --- a/docs/release/26.08.md +++ b/docs/release/26.08.md @@ -122,6 +122,7 @@ newgrp clab_admins * Vagrant default provider is set correctly on WSL (#3673) * **netlab test** cleanup handles Ctrl-C without crashing (#3720) * 'show modules' displays the 'dhcp.client.routing' feature flag (#3671) +* SONiC containerlab link endpoints are named ethN so docker-sonic-vs maps its front-panel ports (#3872) (doc-fixes-26.08)= ## Documentation Fixes