phase 3: wire protocol foundation (ARCP + TLS 1.2 + HMAC + real-VM round-trip)

[nficano]

Summary

Closes Phase 3 per MASTER.md §7. Real-VM round-trip succeeds against the
live XP VM (Windows XP SP3, Python 3.4.10, OpenSSL 1.0.2k); see
docs/sessions/phase-3-roundtrip.md for the captured log.

• docs/PROTOCOL.md — v0 wire spec (framing, envelope shape, HMAC
  canonicalization, TLS configuration, all v0 message types).
• internal/arcp — Go side: typed Envelope, sorted-key canonical JSON,
  HMAC-SHA256 sign/verify with constant-time compare, length-prefixed
  framing (4-byte BE length, 50 MiB cap), Crockford-base32 IDs, RFC 3339
  µs timestamps. 47 unit + corpus tests.
• internal/transport — TLS 1.2 dial with self-signed cert fingerprint
  pinning via VerifyConnection. RSA cipher suites confirmed working on
  the VM's Python 3.4 OpenSSL 1.0.2k build.
• agent/arcp.py — Python-3.4-compatible mirror. Byte-for-byte parity
  with Go enforced by tests/protocol_corpus.json.
• cmd/gen-corpus, cmd/xpc-roundtrip, agent/scripts/echo_server.py —
  generator + Phase 3 verification client/server.

R1/R8 (TLS-on-Python-3.4) risk in docs/ARCHITECTURE.md is now closed.

Test plan

• go test -race ./... green (47 tests across cmd/xpc,
  internal/{arcp,transport,version}).
• golangci-lint run clean (0 issues).
• pytest agent/tests/ green (34 passed, 2 skipped).
• Corpus parity: same envelope produces identical canonical/sig/framed
  bytes in Go and Python.
• Real-VM TLS 1.2 round-trip against xp-truvoice-w02:9579 (5/5 cases
  OK: ping, session.open, tool.invoke, stream.chunk, log).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Wire protocol foundation with JSON envelopes, HMAC-SHA256 authentication, and TLS 1.2 certificate pinning.
  • Go and Python protocol implementations with byte-identical encoding and signing.
  • Complete v0 message type set for sessions, execution, streaming, and logging.

• Tests

  • Protocol unit tests and Go/Python interoperability corpus tests.
  • Round-trip client and echo server for real-network validation.

• Documentation

  • Complete wire protocol specification.
  • Real-network round-trip session validation record.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: 1fa7f788-d3e0-44ae-8bb9-27b58dc20891

📥 Commits

Reviewing files that changed from the base of the PR and between 102890b and 6dc1ebf.

📒 Files selected for processing (27)

• CHANGELOG.md
• TASKS.md
• agent/arcp.py
• agent/scripts/echo_server.py
• agent/tests/__init__.py
• agent/tests/conftest.py
• agent/tests/test_arcp.py
• agent/tests/test_corpus.py
• cmd/gen-corpus/main.go
• cmd/xpc-roundtrip/main.go
• docs/PROTOCOL.md
• docs/sessions/phase-3-roundtrip.md
• internal/arcp/canonical.go
• internal/arcp/canonical_test.go
• internal/arcp/codec.go
• internal/arcp/codec_test.go
• internal/arcp/corpus_test.go
• internal/arcp/envelope.go
• internal/arcp/envelope_test.go
• internal/arcp/hmac.go
• internal/arcp/hmac_test.go
• internal/arcp/ids.go
• internal/arcp/ids_test.go
• internal/arcp/types.go
• internal/transport/tls.go
• internal/transport/tls_test.go
• tests/protocol_corpus.json

---

📝 Walkthrough

Walkthrough

This PR completes Phase 3 by implementing the xpc v0 wire protocol specification with byte-identical Go and Python libraries, TLS 1.2 transport with certificate fingerprint pinning, canonical JSON signing with HMAC-SHA256, and comprehensive corpus parity testing validated on a real Windows XP VM.

Changes

Phase 3 Wire Protocol Foundation

Layer / File(s)	Summary
Protocol Specification docs/PROTOCOL.md	Complete v0 wire protocol defining envelope schema, authentication via HMAC-SHA256 over canonical JSON, 4-byte big-endian length-prefixed framing, TLS 1.2 with fingerprint pinning, all v0 message types, error handling, and corpus requirements.
Go Envelope Types & Constants internal/arcp/envelope.go, internal/arcp/types.go	Exported Envelope and Auth structs with required/optional fields; constants for protocol version (1.0), max frame size (50 MiB), auth algorithm (HMAC-SHA256), and all v0 message type strings (session, control, execution, streaming, event).
Go Canonical Encoding & Signing internal/arcp/canonical.go, internal/arcp/hmac.go, internal/arcp/ids.go	Deterministic canonical JSON via marshal-unmarshal-marshal roundtrip; HMAC-SHA256 signing with constant-time verification; Crockford-base32 ID generation and RFC3339-microsecond timestamp formatting for Go↔Python byte parity.
Go Framing Codec internal/arcp/codec.go	Length-prefixed frame write/read with 4-byte big-endian length header, 50 MiB size enforcement, and io.ReadFull robustness to partial reads.
Go Unit Tests internal/arcp/*_test.go	Comprehensive coverage for envelope construction, canonical marshaling, HMAC signing/verification, framing round-trips, partial reads, ID format/uniqueness, and timestamp formatting.
TLS 1.2 Transport internal/transport/tls.go	Certificate SHA-256 fingerprint computation, normalized fingerprint parsing, tls.VerifyConnection callback for pinning, TLS 1.2-only config with restricted ciphers, and dial with configurable timeout.
TLS Transport Tests internal/transport/tls_test.go	Self-signed cert generation, fingerprint normalization variants, and pinning validation with in-process TLS 1.2 server.
Protocol Corpus Generation & Data cmd/gen-corpus/main.go, tests/protocol_corpus.json, internal/arcp/corpus_test.go	Go command that constructs protocol test cases, signs with fixed PSK, computes canonical/signature/framed hex, and emits golden JSON corpus with 6 message types; Go corpus tests validate parity.
Round-Trip Verification Client cmd/xpc-roundtrip/main.go	CLI tool that dials echo server via TLS+fingerprint pinning, exchanges 6 representative ARCP messages, verifies echo signatures, and validates .echo suffix mutations.
Python ARCP Library agent/arcp.py	Python 3.4-compatible mirror of Go arcp with constants, envelope construction/validation, canonical JSON encoding, HMAC-SHA256 signing/verification, length-prefixed framing, Crockford ID generation, and RFC3339 timestamp formatting.
Python Unit Tests & Corpus Parity agent/tests/test_arcp.py, agent/tests/test_corpus.py, agent/tests/conftest.py	Unit tests for envelope construction, canonical marshaling determinism, HMAC round-trips, framing codec, partial reads, ID format/uniqueness, and timestamp precision; Python corpus tests verify Go-generated canonical/signature/framed hex matches.
Python Echo Server agent/scripts/echo_server.py	TLS 1.2 echo server accepting ARCP frames, verifying HMAC signatures, appending .echo to ID/type, re-signing, and returning to client with daemon threading.
Real-VM Session Log docs/sessions/phase-3-roundtrip.md	Captured round-trip session on Windows XP VM documenting TLS handshake, HMAC sign/verify interoperability, framing correctness, and protocol-level success with "Phase 3 exit gate: PASSED" checklist.
Status Updates CHANGELOG.md, TASKS.md	Marked Phase 3 complete with checkmarks for protocol spec, Go/Python libraries, corpus generation, TLS transport, unit tests, corpus parity, real-network validation, and exit gate passage.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Poem

🐰 Whiskers twitch at cryptographic delight,
Phase 3 hops forth with framing so tight—
Go and Python now speak in one tongue,
Each byte-identical, signatures sung.
TLS paws down, the protocol's done! 🔐

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dc1ebf8f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reject non-finite floats in canonical JSON

canonical_marshal calls json.dumps without allow_nan=False, so payloads containing NaN/Infinity are serialized as non-standard JSON tokens. That violates the protocol’s RFC 8259 requirement and breaks Go/Python interoperability because Go’s encoder rejects non-finite floats; a Python sender can emit frames that the Go side cannot marshal/verify consistently.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Validate decoded envelopes before returning them

read_frame returns json.loads(...) directly and never calls validate, so malformed envelopes (for example, missing id or timestamp) are treated as successfully decoded frames. In this commit’s server flow, such messages can still pass verify_sig and reach business logic, which diverges from the Go path (ReadFrame -> Unmarshal -> Validate) and allows protocol-invalid inputs to be processed.

Useful? React with 👍 / 👎.