diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 50d6f79..7516d43 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -18,6 +18,10 @@ jobs: - run: bun check:types - run: bun check:lint - run: bun check:test + - name: Test contributed incoming connectivity checker + run: + python3 -m unittest discover -s contrib/connectivity-check -p + "test_*.py" # Informational only for now — failures here must not fail the build. - run: bun fallow:dead-code diff --git a/contrib/README.md b/contrib/README.md index 056f0c5..31eaf80 100644 --- a/contrib/README.md +++ b/contrib/README.md @@ -8,6 +8,11 @@ be able to support them. ## What's here +- [Incoming Connectivity Check](./connectivity-check/) + - Inspect shared VPN namespaces, forwarded ports, and current NAT-PMP + addresses + - Verify incoming BitTorrent handshakes from a separate external machine + - [Homepage Integration](./homepage/) - Display network info, API status, and timing data - Custom widgets and API integration diff --git a/contrib/connectivity-check/.gitignore b/contrib/connectivity-check/.gitignore new file mode 100644 index 0000000..c18dd8d --- /dev/null +++ b/contrib/connectivity-check/.gitignore @@ -0,0 +1 @@ +__pycache__/ diff --git a/contrib/connectivity-check/README.md b/contrib/connectivity-check/README.md new file mode 100644 index 0000000..8eb3c43 --- /dev/null +++ b/contrib/connectivity-check/README.md @@ -0,0 +1,148 @@ +# Incoming connectivity check + +A read-only diagnostic for a Docker stack in which Mousehole and qBittorrent +share a VPN container's network namespace and the VPN provides NAT-PMP port +forwarding. It compares the IP MAM sees with the forwarding gateway's current +external IP, then lets a separate machine test the actual BitTorrent endpoint. + +A successful Mousehole update confirms that MAM accepted the update. It does not +prove that a peer can connect to your listening port. A VPN can provide a +working forward on one public IP while outbound requests use another. +Conversely, two different IPs do not necessarily mean incoming connectivity is +broken. + +This tool is a community contribution. It is not part of Mousehole's +application, API, Docker image, or container healthcheck. + +## Requirements + +- Python 3.10 or newer; no Python dependencies. +- For inspection: a Linux Docker host, Docker CLI, `nsenter`, and root access to + read and enter the containers' network namespaces. +- A VPN that supports NAT-PMP, such as a suitably configured Gluetun/Proton + stack. +- qBittorrent's Web API must already be readable from inside its network + namespace. The tool does not configure or bypass authentication. An API that + requires login produces an unknown result. +- For the incoming test: a machine outside the tested VPN and the v1 info hash + of one torrent you are already actively seeding. A paused or absent torrent, + or a client requiring encrypted handshakes, cannot provide this test's proof. + +The inspector targets a shared network namespace setup. If Mousehole uses an +HTTP proxy instead, omit `--mousehole-container`; different network namespaces +are expected in that arrangement. This tool cannot validate proxy routing. + +## 1. Inspect the current connection + +Run from this repository on the Docker host, substituting your container names +and the Web API port used inside qBittorrent's namespace: + +```sh +sudo python3 contrib/connectivity-check/check.py inspect \ + --vpn-container gluetun \ + --qbittorrent-container qbittorrent \ + --mousehole-container mousehole \ + --qbittorrent-url http://127.0.0.1:8995 \ + > /tmp/connectivity-sample.json +``` + +The inspector reads: + +- Actual Linux network namespace identities and container health/status. +- The VPN's current forwarded-port file and qBittorrent's listening port and + bound interface. +- MAM's unauthenticated observed-IP endpoint from qBittorrent's namespace, using + that container's DNS resolvers and normal HTTPS certificate checks. +- The gateway's current external IPv4 address using NAT-PMP operation 0. It + never creates, renews, or deletes a mapping; the VPN remains the mapping + owner. +- A bounded sample of MAM tracker statuses, with URLs, torrent names, info + hashes, and passkeys omitted from the report. +- Mousehole's liveness and, if a token is supplied, its last contact and next + scheduled contact. It never invokes an authenticated MAM update. + +Defaults are `tun0`, `/gluetun/forwarded_port`, and `http://127.0.0.1:5010` for +Mousehole. Use `--interface`, `--forwarded-port-file`, and `--mousehole-url` if +yours differ. By default, the NAT-PMP gateway comes from the tunnel's default or +split-default routes; use `--nat-pmp-gateway` to specify a private IPv4 gateway +explicitly. + +For optional Mousehole state, add +`--mousehole-token-file /path/to/mousehole-token`. This is the **Mousehole API +Bearer token**, not the MAM cookie. Its contents travel to the namespace helper +through standard input, never command arguments or diagnostic output. No token +is needed for the core network checks. Credentials and API response bodies are +not included in error output. + +The MAM lookup uses IPv4 DNS resolvers from the inspected container and verifies +the original HTTPS hostname when connecting. It uses one validated public IPv4 +answer. IPv6-only resolvers, truncated DNS responses, unavailable DNS, or an +unexpected response produce an unknown result; the inspector does not fall back +to the Docker host's DNS. + +The report contains network addresses and operational status. Review it before +sharing, and keep your token file private. + +## 2. Probe from outside the VPN + +Copy the fresh JSON report to a separate machine that is outside the tested VPN. +Keep the machines' clocks synchronized. Samples expire after five minutes by +default; collect again if you reconnect, change ports, or take longer to copy +it. + +Run the following on that external machine, replacing the placeholder with the +40-character hexadecimal v1 info hash of your existing active seed: + +```sh +python3 contrib/connectivity-check/check.py probe /tmp/connectivity-sample.json \ + --info-hash YOUR_40_CHARACTER_V1_INFO_HASH \ + --outside-vpn +``` + +`--outside-vpn` confirms where you are running the probe; the tool cannot prove +that an arbitrary external machine's routing bypasses your VPN. Probing from +inside the tested tunnel can fail because the VPN does not support hairpin +connections and is not a valid incoming test. + +The probe checks only the current observed and NAT-PMP IPv4 endpoints recorded +in the report. It validates freshness, public addresses, port alignment, and +network preconditions before opening sockets. `--max-age-seconds` changes the +freshness limit when needed; it does not make an old mapping current. + +It sends the standard BitTorrent handshake, validates the response's protocol +and info hash, and closes the socket after the handshake. It never requests +pieces or downloads torrent content. TCP connection success is reported +separately and does not count as a valid BitTorrent handshake. + +## Interpret the evidence + +| Measurement | Meaning | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| Valid handshake at the IP MAM observes | Incoming BitTorrent reachability was verified from this external machine, even if the NAT-PMP IP differs. | +| Observed-IP handshake fails, NAT-PMP-IP handshake succeeds | The seed and forwarding endpoint work, but the endpoint MAM observes failed this incoming test. | +| TCP connects, but the handshake is invalid or absent | TCP alone does not verify the expected BitTorrent seed. | +| Both tests fail, API reads fail, or ports/namespaces differ | Inconclusive; fix the reported prerequisite or investigate further before choosing a recovery action. | + +A single failed test is not enough to justify automated reconnects. Repeat with +fresh inspection data and look for consistent evidence. If you automate recovery +outside Mousehole, use explicit cooldowns and a retry limit, preserve the VPN +kill switch, and let Mousehole remain the only authenticated MAM updater. + +Neither a valid handshake nor HTTP 200 from Mousehole proves MAM currently shows +the account as connectable. That status still needs confirmation on MAM. + +`tracker_status_counts` uses qBittorrent's status codes: `0` disabled, `1` not +contacted, `2` working, `3` updating, and `4` not working. Tracker coverage is +reported separately and is limited to five MAM torrents per inspection. + +The tool makes no service, preference, cookie, or network changes. It performs +one bounded inspection/probe when invoked; it installs no timer or daemon. + +## Tests + +The tests mock network and container access and require no VPN, Docker daemon, +MAM credentials, or live tracker: + +```sh +python3 -m unittest discover -s contrib/connectivity-check -p 'test_*.py' -v +``` diff --git a/contrib/connectivity-check/check.py b/contrib/connectivity-check/check.py new file mode 100644 index 0000000..0ed9301 --- /dev/null +++ b/contrib/connectivity-check/check.py @@ -0,0 +1,607 @@ +#!/usr/bin/env python3 +"""Read-only Docker/VPN diagnostics; run probes separately outside the VPN.""" + +import argparse +import datetime as dt +import http.client +import ipaddress +import json +import os +from pathlib import Path +import re +import socket +import struct +import subprocess +import sys +import time +import urllib.parse +import urllib.request + + +UTC = dt.timezone.utc +PROTOCOL = b"\x13BitTorrent protocol" +PEER_ID = b"-MH0001-000000000000" +CONTACT_STATUSES = {"ok", "throttled", "rejected", "unreachable", "no-cookie", "pending"} +MAM_HOST = "t.myanonamouse.net" + + +class Unknown(Exception): + """Only static reason codes may be surfaced; never include response bodies.""" + + +def public_ip(value): + try: + address = ipaddress.ip_address(value) + if (not isinstance(value, str) or address.version != 4 or not address.is_global + or address.is_multicast or address.is_reserved): + raise ValueError + return str(address) + except (ValueError, TypeError): + raise Unknown("invalid_public_ipv4") from None + + +def port_number(value): + try: + port = int(value) + if isinstance(value, bool) or str(port) != str(value).strip() or not 1 <= port <= 65535: + raise ValueError + return port + except (TypeError, ValueError): + raise Unknown("invalid_port") from None + + +def private_gateway(value): + try: + address = ipaddress.IPv4Address(value) + if not any(address in ipaddress.ip_network(net) for net in + ("10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16")): + raise ValueError + return str(address) + except (TypeError, ValueError): + raise Unknown("invalid_nat_pmp_gateway") from None + + +def route_gateway(routes, interface): + """/proc/net/route uses little-endian IPv4 addresses, including netmasks.""" + gateways = set() + try: + for line in routes.splitlines()[1:]: + fields = line.split() + if fields[0] != interface: + continue + destination, gateway, flags, mask = [int(fields[i], 16) for i in (1, 2, 3, 7)] + if flags & 3 == 3 and (destination, mask) in ((0, 0), (0, 128), (128, 128)): + gateways.add(private_gateway(socket.inet_ntoa(struct.pack("= 2 and fields[0] == "nameserver": + address = ipaddress.ip_address(fields[1]) + if address.version == 4 and not address.is_multicast and not address.is_unspecified: + result.append(str(address)) + return list(dict.fromkeys(result))[:3] + + +def dns_name(packet, offset): + labels, end = [], None + for _ in range(128): + if offset >= len(packet): + break + length = packet[offset] + if length & 0xC0 == 0xC0: + if offset + 1 >= len(packet): + break + pointer = ((length & 0x3F) << 8) | packet[offset + 1] + if pointer >= offset: # Compression refers to an earlier name, never a cycle. + break + end = end if end is not None else offset + 2 + offset = pointer + elif length & 0xC0: + break + elif length == 0: + name = ".".join(labels) + if len(name) > 253: + break + return name.lower(), end if end is not None else offset + 1 + else: + if offset + length + 1 > len(packet): + break + labels.append(packet[offset + 1:offset + length + 1].decode("ascii")) + offset += length + 1 + raise Unknown("invalid_dns_response") + + +def dns_answer(packet, transaction): + try: + ident, flags, questions, answers, _authority, _additional = struct.unpack("!6H", packet[:12]) + if (ident != transaction or flags & 0x8000 == 0 or flags & 0x7A0F + or questions != 1 or not 1 <= answers <= 64): + raise ValueError + question, offset = dns_name(packet, 12) + if question != MAM_HOST or packet[offset:offset + 4] != b"\0\1\0\1": + raise ValueError + offset += 4 + addresses, aliases = {}, {} + for _ in range(answers): + owner, offset = dns_name(packet, offset) + kind, cls, _ttl, size = struct.unpack("!HHIH", packet[offset:offset + 10]) + offset += 10 + end = offset + size + if end > len(packet): + raise ValueError + if cls == 1 and kind == 1: + if size != 4: + raise ValueError + addresses[owner] = public_ip(socket.inet_ntoa(packet[offset:end])) + elif cls == 1 and kind == 5: + aliases[owner], consumed = dns_name(packet, offset) + if consumed != end: + raise ValueError + offset = end + name = MAM_HOST + for _ in range(16): + if name in addresses: + return addresses[name] + name = aliases.get(name) + if name is None: + break + except (ValueError, UnicodeError, struct.error): + pass + raise Unknown("invalid_dns_response") + + +def resolve_mam(resolvers): + # DNS goes to the container's resolvers from its network namespace, never the host's stub. + for resolver in (resolvers or [])[:3]: + try: + transaction = int.from_bytes(os.urandom(2), "big") + question = b"".join(bytes([len(label)]) + label.encode() for label in MAM_HOST.split(".")) + query = struct.pack("!6H", transaction, 0x0100, 1, 0, 0, 0) + question + b"\0\0\1\0\1" + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: + sock.settimeout(3) + sock.connect((str(ipaddress.IPv4Address(resolver)), 53)) + sock.send(query) + return dns_answer(sock.recv(4096), transaction) + except (OSError, ValueError, Unknown): + continue + raise Unknown("container_dns_lookup_failed") + + +def loopback_url(value): + try: + url = urllib.parse.urlsplit(value) + if (url.scheme != "http" or url.hostname not in ("127.0.0.1", "::1", "localhost") + or url.username is not None or url.password is not None or url.query or url.fragment): + raise ValueError + if url.port is not None: + port_number(url.port) + return value.rstrip("/") + except (ValueError, TypeError): + raise Unknown("invalid_loopback_url") from None + + +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None + + +class PinnedHTTPSConnection(http.client.HTTPSConnection): + def __init__(self, host, destination, **kwargs): + super().__init__(host, **kwargs) + self.destination = public_ip(destination) + self._create_connection = self.connect_address + + def connect_address(self, address, timeout, source_address): + # HTTPSConnection retains the original host for SNI and certificate verification. + return socket.create_connection((self.destination, address[1]), timeout, source_address) + + +class PinnedHTTPSHandler(urllib.request.HTTPSHandler): + def __init__(self, destination): + super().__init__() + self.destination = destination + + def https_open(self, request): + return self.do_open(lambda host, **kwargs: PinnedHTTPSConnection(host, self.destination, **kwargs), + request, context=self._context) + + +def get_json(url, token=None, destination=None): + # Ignore proxy environment variables and refuse redirects, especially with credentials. + handlers = [urllib.request.ProxyHandler({}), NoRedirect()] + if destination: + handlers.append(PinnedHTTPSHandler(destination)) + opener = urllib.request.build_opener(*handlers) + request = urllib.request.Request(url) + if token is not None: + request.add_header("Authorization", "Bearer " + token) + with opener.open(request, timeout=5) as response: + body = response.read(8 * 1024 * 1024 + 1) + if len(body) > 8 * 1024 * 1024: + raise Unknown("response_too_large") + return json.loads(body) + + +def timestamp(value): + """Accept MouseHole's RFC 9557 zone annotation, emit only normalized UTC.""" + if not isinstance(value, str): + raise Unknown("invalid_timestamp") + try: + parsed = dt.datetime.fromisoformat(value.split("[", 1)[0].replace("Z", "+00:00")) + if parsed.tzinfo is None: + raise ValueError + return parsed.astimezone(UTC) + except ValueError: + raise Unknown("invalid_timestamp") from None + + +def mousehole_state(data): + result = {"last_contact_status": "pending"} + if data.get("nextContactAt"): + result["next_contact_at"] = timestamp(data["nextContactAt"]).isoformat() + contact = data.get("lastMamContact") + if contact: + result["last_contact_at"] = timestamp(contact["at"]).isoformat() + update = contact.get("ipUpdate") + if contact.get("reached") is False: + result["last_contact_status"] = "unreachable" + elif contact.get("reached") is True and update is None: + result["last_contact_status"] = "no-cookie" + elif contact.get("reached") is True and isinstance(update, dict): + status = update.get("httpStatus") + if type(status) is not int or not 100 <= status <= 599: + raise Unknown("invalid_mousehole_status") + result["last_update_http_status"] = status + result["last_contact_status"] = {200: "ok", 429: "throttled"}.get(status, "rejected") + if type(update.get("success")) is bool: + result["last_update_success"] = update["success"] + else: + raise Unknown("invalid_mousehole_contact") + if contact.get("reached") is True: + result["last_contact_ip"] = public_ip(contact["ip"]) + asn = contact.get("asn") + if type(asn) is not int or not 0 <= asn < 2 ** 32: + raise Unknown("invalid_mousehole_asn") + result["last_contact_asn"] = asn + return result + + +def is_mam_tracker(url): + if not isinstance(url, str): + return False + try: + hostname = urllib.parse.urlsplit(url).hostname or "" + return hostname == "myanonamouse.net" or hostname.endswith(".myanonamouse.net") + except ValueError: + return False + + +def tracker_summary(base): + trackers = get_json(base + "/api/v2/sync/maindata").get("trackers", {}) + hashes = sorted({h for url, values in trackers.items() if is_mam_tracker(url) + for h in values if isinstance(h, str) and re.fullmatch(r"[a-fA-F0-9]{40}", h)}) + result = {"total_mam_torrents": len(hashes), "sampled_mam_torrents": 0, + "tracker_status_counts": {}, "tracker_error_count": 0, "tracker_message_count": 0} + for info_hash in hashes[:5]: + rows = get_json(base + "/api/v2/torrents/trackers?hash=" + info_hash) + for row in rows: + if not is_mam_tracker(row.get("url")): + continue + status = row.get("status") + if type(status) is not int or status not in range(5): + raise Unknown("invalid_tracker_status") + counts = result["tracker_status_counts"] + counts[str(status)] = counts.get(str(status), 0) + 1 + result["tracker_error_count"] += status == 4 + result["tracker_message_count"] += bool(row.get("msg")) + result["sampled_mam_torrents"] += 1 + return result + + +def attempt(errors, label, operation): + try: + return operation() + except Exception: + # Raw API/command errors can contain tokens, tracker URLs and passkeys. + errors.append(label + "_read_failed") + return None + + +def namespace_sample(config): + errors = [] + result = {"errors": errors} + base = loopback_url(config["qbittorrent_url"]) + interface = config["interface"] + gateway = attempt(errors, "nat_pmp_gateway", lambda: private_gateway(config["nat_pmp_gateway"]) + if config.get("nat_pmp_gateway") else + route_gateway(Path("/proc/net/route").read_text(), interface)) + result["nat_pmp_gateway"] = gateway + result["nat_pmp_ip"] = attempt(errors, "nat_pmp", lambda: nat_pmp_ip(gateway, interface)) if gateway else None + result["observed_ip"] = attempt(errors, "mam_observed_ip", lambda: + public_ip(get_json("https://" + MAM_HOST + "/json/jsonIp.php", + destination=resolve_mam(config.get("nameservers")))["ip"])) + + def preferences(): + data = get_json(base + "/api/v2/app/preferences") + bound = data.get("current_network_interface") + if not isinstance(bound, str) or not re.fullmatch(r"[a-zA-Z0-9_.:-]{0,15}", bound): + raise Unknown("invalid_qbittorrent_interface") + return {"listen_port": port_number(data["listen_port"]), "interface": bound} + + result["qbittorrent"] = attempt(errors, "qbittorrent_preferences", preferences) + result["trackers"] = attempt(errors, "trackers", lambda: tracker_summary(base)) + if config.get("mousehole_enabled"): + mousehole = {} + base = loopback_url(config["mousehole_url"]) + + def health(): + value = get_json(base + "/health").get("lastMamContactResult") + if value not in CONTACT_STATUSES: + raise Unknown("invalid_mousehole_health") + return value + + mousehole["health_last_contact_result"] = attempt(errors, "mousehole_health", health) + mousehole["health_reachable"] = mousehole["health_last_contact_result"] is not None + if config.get("token"): + state = attempt(errors, "mousehole_state", lambda: + mousehole_state(get_json(base + "/state", config["token"]))) + if state: + mousehole.update(state) + result["mousehole"] = mousehole + return result + + +def run(arguments, *, stdin=None, timeout=15): + try: + result = subprocess.run(arguments, input=stdin, capture_output=True, text=True, + timeout=timeout, check=False) + if result.returncode: + raise Unknown("command_failed") + return result.stdout + except (OSError, subprocess.SubprocessError): + raise Unknown("command_unavailable") from None + + +def container_snapshot(names): + # Docker's full configuration stays in memory. Emit only these state fields. + items = json.loads(run(["docker", "inspect", *names.values()])) + result, pids = {}, {} + for label, item in zip(names, items): + state = item["State"] + pid = state["Pid"] + running = state.get("Running") is True and state.get("Status") == "running" + health = state.get("Health", {}).get("Status", "not-configured") + if health not in ("healthy", "unhealthy", "starting", "not-configured"): + health = "unknown" + netns = None + if running and type(pid) is int and pid > 1: + netns = os.readlink("/proc/" + str(pid) + "/ns/net") + if not re.fullmatch(r"net:\[\d+\]", netns): + raise Unknown("invalid_namespace") + pids[label] = pid + result[label] = {"running": running, "health": health, "netns": netns} + if len(result) != len(names): + raise Unknown("container_inspection_failed") + return result, pids + + +def containers_ready(containers): + if not isinstance(containers, dict) or not {"vpn", "qbittorrent"} <= containers.keys(): + raise Unknown("missing_containers") + namespaces = set() + for item in containers.values(): + if (item.get("running") is not True or item.get("health") not in ("healthy", "not-configured") + or not isinstance(item.get("netns"), str) + or not re.fullmatch(r"net:\[\d+\]", item["netns"])): + raise Unknown("container_not_ready") + namespaces.add(item["netns"]) + if len(namespaces) != 1: + raise Unknown("namespace_mismatch") + + +def inspect_sample(args): + result = {"schema_version": 1, "collected_at": dt.datetime.now(UTC).isoformat(), + "expected_interface": args.interface, "snapshot_stable": False, "errors": []} + errors = result["errors"] + names = {"vpn": args.vpn_container, "qbittorrent": args.qbittorrent_container} + if args.mousehole_container: + names["mousehole"] = args.mousehole_container + try: + containers, pids = container_snapshot(names) + result["containers"] = containers + result["namespace_match"] = len({item["netns"] for item in containers.values()}) == 1 + containers_ready(containers) + result["forwarded_port"] = attempt(errors, "forwarded_port", lambda: + port_number(run(["docker", "exec", args.vpn_container, "cat", "--", args.forwarded_port_file]).strip())) + config = {"qbittorrent_url": args.qbittorrent_url, "interface": args.interface, + "nat_pmp_gateway": args.nat_pmp_gateway, "mousehole_url": args.mousehole_url, + "mousehole_enabled": bool(args.mousehole_container)} + config["nameservers"] = attempt(errors, "container_resolvers", lambda: + nameservers(Path("/proc/" + str(pids["qbittorrent"]) + "/root/etc/resolv.conf").read_text())) + if args.mousehole_token_file: + config["token"] = attempt(errors, "mousehole_token", lambda: + Path(args.mousehole_token_file).read_text().strip()) + data = json.loads(run(["nsenter", "--net=/proc/" + str(pids["qbittorrent"]) + "/ns/net", "--", + sys.executable, str(Path(__file__).resolve()), "_namespace"], + stdin=json.dumps(config), timeout=75)) + errors.extend(data.pop("errors", [])) + result.update(data) + # A restart during collection invalidates the snapshot and all derived evidence. + after, after_pids = container_snapshot(names) + final_port = attempt(errors, "final_forwarded_port", lambda: + port_number(run(["docker", "exec", args.vpn_container, "cat", "--", args.forwarded_port_file]).strip())) + if after != containers or after_pids != pids: + errors.append("containers_changed_during_inspection") + elif final_port is None or final_port != result["forwarded_port"]: + errors.append("forwarded_port_not_stable") + else: + result["snapshot_stable"] = True + except Unknown as error: + errors.append(str(error)) + except Exception: + errors.append("inspection_failed") + return result + + +def handshake(ip, port, info_hash): + result = {"ip": ip, "port": port, "tcp": False, "valid_handshake": False} + try: + with socket.create_connection((ip, port), timeout=5) as connection: + result["tcp"] = True + deadline = time.monotonic() + 5 + connection.settimeout(5) + connection.sendall(PROTOCOL + b"\0" * 8 + info_hash + PEER_ID) + received = b"" + while len(received) < 68: + remaining = deadline - time.monotonic() + if remaining <= 0: + break + connection.settimeout(remaining) + chunk = connection.recv(68 - len(received)) + if not chunk: + break + received += chunk + result["valid_handshake"] = (len(received) == 68 and received[:20] == PROTOCOL + and received[28:48] == info_hash and received[48:68] != PEER_ID) + except OSError: + pass + return result + + +def probe_targets(sample, max_age, now=None): + if type(max_age) is not int or not 1 <= max_age <= 3600: + raise Unknown("invalid_max_age") + if type(sample.get("schema_version")) is not int or sample["schema_version"] != 1: + raise Unknown("unsupported_schema") + age = ((now or dt.datetime.now(UTC)) - timestamp(sample.get("collected_at"))).total_seconds() + if age < -30 or age > max_age: + raise Unknown("sample_not_fresh") + containers_ready(sample.get("containers")) + if sample.get("namespace_match") is not True: + raise Unknown("namespace_mismatch") + if sample.get("snapshot_stable") is not True: + raise Unknown("container_snapshot_not_stable") + port = port_number(sample.get("forwarded_port")) + prefs = sample.get("qbittorrent") or {} + if port_number(prefs.get("listen_port")) != port: + raise Unknown("port_mismatch") + interface = sample.get("expected_interface") + if not isinstance(interface, str) or not interface or prefs.get("interface") != interface: + raise Unknown("interface_mismatch") + observed, nat = public_ip(sample.get("observed_ip")), public_ip(sample.get("nat_pmp_ip")) + return observed, nat, port + + +def verdict(observed, nat, probes): + if probes[observed]["valid_handshake"]: + return "observed_endpoint_reachable" + if probes[nat]["valid_handshake"] and observed != nat: + return "split_incoming_failure" + return "unknown" + + +def probe_sample(sample, info_hash, max_age=300): + result = {"schema_version": 1, "probed_at": dt.datetime.now(UTC).isoformat(), + "verdict": "unknown", "outside_vpn_acknowledged": True, "probes": []} + try: + observed, nat, port = probe_targets(sample, max_age) + result["collected_at"] = timestamp(sample["collected_at"]).isoformat() + probes = {ip: handshake(ip, port, info_hash) for ip in dict.fromkeys((observed, nat))} + result.update({"observed_ip": observed, "nat_pmp_ip": nat, "port": port, + "probes": list(probes.values()), "verdict": verdict(observed, nat, probes)}) + except Unknown as error: + result["reason"] = str(error) + except Exception: + result["reason"] = "invalid_sample" + return result + + +class SafeParser(argparse.ArgumentParser): + def error(self, message): + # argparse's usual diagnostics echo invalid values, potentially credentials. + self.exit(2, "Invalid arguments; use --help for supported options.\n") + + +def main(): + if sys.argv[1:] == ["_namespace"]: + try: + print(json.dumps(namespace_sample(json.load(sys.stdin)))) + except Exception: + print(json.dumps({"errors": ["namespace_read_failed"]})) + return + parser = SafeParser(description=__doc__) + commands = parser.add_subparsers(dest="command", required=True, parser_class=SafeParser) + inspect = commands.add_parser("inspect", help="Collect a sanitized sample on the Linux Docker host") + inspect.add_argument("--vpn-container", required=True) + inspect.add_argument("--qbittorrent-container", required=True) + inspect.add_argument("--mousehole-container") + inspect.add_argument("--qbittorrent-url", default="http://127.0.0.1:8995") + inspect.add_argument("--mousehole-url", default="http://127.0.0.1:5010") + inspect.add_argument("--mousehole-token-file") + inspect.add_argument("--forwarded-port-file", default="/gluetun/forwarded_port") + inspect.add_argument("--interface", default="tun0") + inspect.add_argument("--nat-pmp-gateway") + probe = commands.add_parser("probe", help="Probe the measured endpoints from outside the VPN") + probe.add_argument("sample_json", help="Fresh JSON sample file, or - to read stdin") + probe.add_argument("--info-hash", required=True, help="40 hexadecimal digits of your active, complete seed") + probe.add_argument("--outside-vpn", required=True, action="store_true", + help="Acknowledge that this machine's probes travel outside the VPN") + probe.add_argument("--max-age-seconds", dest="max_age", type=int, default=300, + help="Maximum sample age in seconds (1..3600)") + args = parser.parse_args() + try: + if args.command == "inspect": + for name in (args.vpn_container, args.qbittorrent_container, args.mousehole_container): + if name is not None and not re.fullmatch(r"[a-zA-Z0-9][a-zA-Z0-9_.-]*", name): + raise Unknown("invalid_container_name") + if not re.fullmatch(r"[a-zA-Z0-9_.:-]{1,15}", args.interface): + raise Unknown("invalid_interface") + loopback_url(args.qbittorrent_url) + loopback_url(args.mousehole_url) + if args.nat_pmp_gateway: + private_gateway(args.nat_pmp_gateway) + if args.mousehole_token_file and not args.mousehole_container: + raise Unknown("mousehole_container_required_for_token") + print(json.dumps(inspect_sample(args), indent=2)) + else: + if not re.fullmatch(r"[a-fA-F0-9]{40}", args.info_hash) or not 1 <= args.max_age <= 3600: + raise Unknown("invalid_probe_arguments") + sample = json.load(sys.stdin) if args.sample_json == "-" else json.loads(Path(args.sample_json).read_text()) + print(json.dumps(probe_sample(sample, bytes.fromhex(args.info_hash), args.max_age), indent=2)) + except Unknown as error: + print(json.dumps({"verdict": "unknown", "reason": str(error)})) + return 2 + except Exception: + print(json.dumps({"verdict": "unknown", "reason": "input_read_failed"})) + return 2 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/contrib/connectivity-check/test_check.py b/contrib/connectivity-check/test_check.py new file mode 100644 index 0000000..df8d8cd --- /dev/null +++ b/contrib/connectivity-check/test_check.py @@ -0,0 +1,406 @@ +"""Offline checks for protocol proof, read-only collection and safe output.""" + +import argparse +import copy +import datetime as dt +import importlib.util +import io +import json +from pathlib import Path +import socket +import struct +import unittest +from unittest import mock + + +SPEC = importlib.util.spec_from_file_location("connectivity_check", Path(__file__).with_name("check.py")) +check = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(check) +IP_A, IP_B = "8.8.8.8", "1.1.1.1" # Offline fixtures: no test opens real sockets. +INFO_HASH = bytes(range(20)) +NOW = dt.datetime(2026, 1, 1, tzinfo=dt.timezone.utc) +SECRET = "SENSITIVE_VALUE_MUST_NOT_APPEAR" + + +def sample(): + return { + "schema_version": 1, "collected_at": NOW.isoformat(), "errors": [], + "snapshot_stable": True, "namespace_match": True, "expected_interface": "tun0", + "containers": {name: {"running": True, "health": "healthy", "netns": "net:[123]"} + for name in ("vpn", "qbittorrent")}, + "forwarded_port": 40000, "nat_pmp_ip": IP_B, "observed_ip": IP_A, + "qbittorrent": {"listen_port": 40000, "interface": "tun0"}, + } + + +class InputTests(unittest.TestCase): + def test_nonpublic_or_nonipv4_endpoints_are_rejected(self): + for value in (None, 1234, True, "127.0.0.1", "192.168.1.1", "100.64.1.1", + "169.254.1.1", "0.0.0.0", "192.0.2.1", "224.0.0.1", + "255.255.255.255", "::1", "2606:4700::1111", SECRET): + with self.subTest(value=value), self.assertRaises(check.Unknown): + check.public_ip(value) + + def test_unsafe_ports_are_rejected(self): + for value in (0, 65536, -1, True, False, 1.0, "22.0", None, "1e3", "22\n" + SECRET): + with self.subTest(value=value), self.assertRaises(check.Unknown): + check.port_number(value) + self.assertEqual(check.port_number(" 40000\n"), 40000) + + def test_loopback_url_validation(self): + self.assertEqual(check.loopback_url("http://127.0.0.1:8995/"), "http://127.0.0.1:8995") + for value in ("https://127.0.0.1", "http://example.org", "http://127.0.0.1.evil", + "http://user:" + SECRET + "@127.0.0.1", "http://127.0.0.1?token=" + SECRET, + "http://127.0.0.1/#" + SECRET, "http://127.0.0.1:65536"): + with self.subTest(value=value), self.assertRaises(check.Unknown): + check.loopback_url(value) + + def test_fresh_sample_targets(self): + self.assertEqual(check.probe_targets(sample(), 300, NOW), (IP_A, IP_B, 40000)) + + def test_stale_future_naive_and_invalid_timestamps(self): + for value in ((NOW - dt.timedelta(seconds=301)).isoformat(), + (NOW + dt.timedelta(seconds=31)).isoformat(), "2026-01-01T00:00:00", SECRET): + fixture = sample() + fixture["collected_at"] = value + with self.subTest(value=value), self.assertRaises(check.Unknown): + check.probe_targets(fixture, 300, NOW) + + def test_bounded_max_age(self): + for value in (0, 3601, True, "300"): + with self.subTest(value=value), self.assertRaises(check.Unknown): + check.probe_targets(sample(), value, NOW) + + def test_preconditions_reject_unhealthy_mismatched_or_incomplete_samples(self): + changes = [ + lambda s: s.update(schema_version=True), + lambda s: s.update(snapshot_stable=False), + lambda s: s.update(namespace_match=False), + lambda s: s.update(forwarded_port=40001), + lambda s: s["qbittorrent"].update(interface=""), + lambda s: s["containers"]["vpn"].update(health="unhealthy"), + lambda s: s["containers"]["vpn"].update(health="starting"), + lambda s: s["containers"]["vpn"].update(running=False), + lambda s: s["containers"]["vpn"].update(netns="net:[456]"), + lambda s: s["containers"].pop("qbittorrent"), + ] + for change in changes: + fixture = sample() + change(fixture) + with self.subTest(change=change), self.assertRaises(check.Unknown): + check.probe_targets(fixture, 300, NOW) + + def test_missing_optional_mousehole_token_does_not_block_probe(self): + fixture = sample() + fixture["errors"].append("mousehole_state_read_failed") + fixture["containers"]["vpn"]["health"] = "not-configured" + self.assertEqual(check.probe_targets(fixture, 300, NOW), (IP_A, IP_B, 40000)) + + def test_invalid_sample_never_opens_connection_or_echoes_payload(self): + fixture = sample() + fixture["collected_at"] = dt.datetime.now(check.UTC).isoformat() + fixture["observed_ip"] = SECRET + with mock.patch.object(check, "handshake") as probe: + result = check.probe_sample(fixture, INFO_HASH) + probe.assert_not_called() + self.assertEqual(result["verdict"], "unknown") + self.assertNotIn(SECRET, json.dumps(result)) + + +class NatPmpTests(unittest.TestCase): + HEADER = "Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT\n" + + def test_proc_routes_use_little_endian_and_only_requested_interface(self): + routes = self.HEADER + ( + "eth0 00000000 0101A8C0 0003 0 0 0 00000000 0 0 0\n" + "tun0 00000000 0100020A 0003 0 0 0 00000080 0 0 0\n" + "tun0 00000080 0100020A 0003 0 0 0 00000080 0 0 0\n") + self.assertEqual(check.route_gateway(routes, "tun0"), "10.2.0.1") + + def test_ambiguous_absent_and_not_up_gateways_are_unknown(self): + rows = ["", "tun0 00000000 0100020A 0002 0 0 0 00000000 0 0 0\n", + "tun0 00000000 0100020A 0003 0 0 0 00000080 0 0 0\n" + "tun0 00000080 0200020A 0003 0 0 0 00000080 0 0 0\n"] + for row in rows: + with self.subTest(row=row), self.assertRaises(check.Unknown): + check.route_gateway(self.HEADER + row, "tun0") + + def test_response_frame_and_network_byte_order(self): + packet = struct.pack("!BBHI4s", 0, 128, 0, 123456, socket.inet_aton(IP_A)) + self.assertEqual(check.nat_pmp_response(packet), IP_A) + for malformed in (packet[:11], packet + b"x", b"\1" + packet[1:], + packet[:1] + b"\x81" + packet[2:], packet[:3] + b"\1" + packet[4:]): + with self.subTest(packet=malformed), self.assertRaises(check.Unknown): + check.nat_pmp_response(malformed) + + def test_request_is_address_only_and_bound_to_vpn_interface(self): + sock = mock.MagicMock() + sock.__enter__.return_value = sock + sock.recv.return_value = struct.pack("!BBHI4s", 0, 128, 0, 0, socket.inet_aton(IP_A)) + with mock.patch.object(check.socket, "socket", return_value=sock): + self.assertEqual(check.nat_pmp_ip("10.2.0.1", "tun0"), IP_A) + sock.connect.assert_called_once_with(("10.2.0.1", 5351)) + sock.send.assert_called_once_with(b"\0\0") + sock.setsockopt.assert_called_once_with(socket.SOL_SOCKET, socket.SO_BINDTODEVICE, b"tun0\0") + + +class HandshakeTests(unittest.TestCase): + PACKET = check.PROTOCOL + b"\0" * 8 + INFO_HASH + b"-UT0001-000000000000" + + def exercise(self, chunks): + connection = mock.MagicMock() + connection.__enter__.return_value = connection + connection.recv.side_effect = chunks + with mock.patch.object(check.socket, "create_connection", return_value=connection) as connect: + result = check.handshake(IP_A, 40000, INFO_HASH) + connect.assert_called_once_with((IP_A, 40000), timeout=5) + connection.sendall.assert_called_once_with(check.PROTOCOL + b"\0" * 8 + INFO_HASH + check.PEER_ID) + self.assertEqual(len(connection.sendall.call_args.args[0]), 68) + connection.__exit__.assert_called_once() + return result, connection + + def test_valid_fragmented_handshake_is_strong_proof(self): + result, connection = self.exercise([self.PACKET[:3], self.PACKET[3:21], self.PACKET[21:]]) + self.assertTrue(result["tcp"]) + self.assertTrue(result["valid_handshake"]) + self.assertEqual(connection.recv.call_args_list, [mock.call(68), mock.call(65), mock.call(47)]) + + def test_wrong_hash_protocol_short_or_timeout_is_only_tcp(self): + for chunks in ([self.PACKET[:28] + b"x" * 20 + self.PACKET[48:]], + [b"\x12" + self.PACKET[1:]], [self.PACKET[:48] + check.PEER_ID], + [self.PACKET[:40], b""], [TimeoutError()]): + with self.subTest(chunks=chunks): + result, _ = self.exercise(chunks) + self.assertTrue(result["tcp"]) + self.assertFalse(result["valid_handshake"]) + + def test_connection_failure_is_distinct_from_tcp_only(self): + with mock.patch.object(check.socket, "create_connection", side_effect=TimeoutError()): + result = check.handshake(IP_A, 40000, INFO_HASH) + self.assertFalse(result["tcp"]) + self.assertFalse(result["valid_handshake"]) + + def test_read_deadline_cannot_be_extended_by_slow_fragments(self): + with mock.patch.object(check.time, "monotonic", side_effect=[100, 106]): + result, connection = self.exercise([]) + connection.recv.assert_not_called() + self.assertFalse(result["valid_handshake"]) + + def test_verdict_requires_valid_protocol_not_just_tcp_or_matching_ips(self): + cases = [(True, True, "observed_endpoint_reachable"), + (True, False, "observed_endpoint_reachable"), + (False, True, "split_incoming_failure"), (False, False, "unknown")] + for observed_ok, nat_ok, expected in cases: + probes = {IP_A: {"tcp": True, "valid_handshake": observed_ok}, + IP_B: {"tcp": True, "valid_handshake": nat_ok}} + with self.subTest(observed=observed_ok, nat=nat_ok): + self.assertEqual(check.verdict(IP_A, IP_B, probes), expected) + + def test_matching_ips_are_probed_once(self): + fixture = sample() + fixture["collected_at"] = dt.datetime.now(check.UTC).isoformat() + fixture["nat_pmp_ip"] = IP_A + response = {"ip": IP_A, "port": 40000, "tcp": True, "valid_handshake": True} + with mock.patch.object(check, "handshake", return_value=response) as probe: + result = check.probe_sample(fixture, INFO_HASH) + probe.assert_called_once_with(IP_A, 40000, INFO_HASH) + self.assertEqual(result["verdict"], "observed_endpoint_reachable") + + +class SanitizationTests(unittest.TestCase): + def test_mousehole_state_never_releases_free_text(self): + data = {"cookie": SECRET, "nextContactAt": "2026-01-01T01:00:00+00:00[UTC]", + "lastMamContact": {"at": "2026-01-01T00:00:00+00:00[UTC]", "reached": True, + "ip": IP_A, "asn": 1234, "as": SECRET, + "ipUpdate": {"httpStatus": 200, "success": True, "msg": SECRET}}} + result = check.mousehole_state(data) + self.assertEqual(result["last_contact_status"], "ok") + self.assertEqual(result["last_contact_ip"], IP_A) + self.assertEqual(result["last_contact_asn"], 1234) + self.assertNotIn(SECRET, json.dumps(result)) + data["lastMamContact"] = {"at": "2026-01-01T00:00:00Z", "reached": False, + "error": {"message": SECRET}} + self.assertEqual(check.mousehole_state(data)["last_contact_status"], "unreachable") + self.assertNotIn(SECRET, json.dumps(check.mousehole_state(data))) + + def test_mam_tracker_match_is_hostname_based(self): + self.assertTrue(check.is_mam_tracker("https://t.myanonamouse.net/" + SECRET)) + for url in ("https://myanonamouse.net.evil/", "https://evil/?myanonamouse.net", + "https://myanonamouse.net@evil/", "** [DHT] **"): + self.assertFalse(check.is_mam_tracker(url)) + + def test_tracker_collection_is_bounded_and_discards_urls_messages_hashes(self): + url = "https://t.myanonamouse.net/" + SECRET + hashes = [f"{n:040x}" for n in range(8)] + data = {"trackers": {url: hashes}, "torrents": {"secret-title": SECRET}} + row = {"url": url, "status": 4, "msg": SECRET} + with mock.patch.object(check, "get_json", side_effect=[data] + [[row]] * 5) as get: + result = check.tracker_summary("http://127.0.0.1:8995") + self.assertEqual(get.call_count, 6) + self.assertEqual(result["total_mam_torrents"], 8) + self.assertEqual(result["sampled_mam_torrents"], 5) + self.assertEqual(result["tracker_status_counts"], {"4": 5}) + self.assertEqual(result["tracker_error_count"], 5) + self.assertNotIn(SECRET, json.dumps(result)) + for info_hash in hashes: + self.assertNotIn(info_hash, json.dumps(result)) + + def test_exception_text_is_never_used_as_read_error(self): + errors = [] + with mock.patch.object(check, "get_json", side_effect=ValueError(SECRET)): + result = check.attempt(errors, "state", lambda: check.get_json("unused")) + self.assertIsNone(result) + self.assertEqual(errors, ["state_read_failed"]) + + def test_container_output_omits_environment_name_and_health_log(self): + config = {"Id": SECRET, "Name": SECRET, "Config": {"Env": [SECRET]}, + "State": {"Running": True, "Status": "running", "Pid": 1234, + "Health": {"Status": "healthy", "Log": [{"Output": SECRET}]}}} + with mock.patch.object(check, "run", return_value=json.dumps([config, config])), \ + mock.patch.object(check.os, "readlink", return_value="net:[123]"): + containers, _ = check.container_snapshot({"vpn": "vpn", "qbittorrent": "qbittorrent"}) + self.assertNotIn(SECRET, json.dumps(containers)) + + def test_http_reader_disables_proxy_redirects_and_uses_header_for_token(self): + response = mock.MagicMock() + response.__enter__.return_value = response + response.read.return_value = b'{"safe": true}' + opener = mock.Mock() + opener.open.return_value = response + with mock.patch.object(check.urllib.request, "build_opener", return_value=opener) as build: + self.assertEqual(check.get_json("http://127.0.0.1:5010/state", SECRET), {"safe": True}) + handlers = build.call_args.args + self.assertEqual(handlers[0].proxies, {}) + self.assertIsInstance(handlers[1], check.NoRedirect) + self.assertIsNone(handlers[1].redirect_request(None, None, 302, None, None, "https://evil")) + request = opener.open.call_args.args[0] + self.assertEqual(request.get_method(), "GET") + self.assertEqual(request.get_header("Authorization"), "Bearer " + SECRET) + self.assertNotIn(SECRET, request.full_url) + + def test_argument_errors_do_not_echo_input(self): + parser = check.SafeParser() + with mock.patch("sys.stderr", new_callable=io.StringIO) as stderr, self.assertRaises(SystemExit): + parser.parse_args(["--unknown=" + SECRET]) + self.assertNotIn(SECRET, stderr.getvalue()) + + +class CollectionTests(unittest.TestCase): + def args(self): + return argparse.Namespace(vpn_container="vpn", qbittorrent_container="qbittorrent", + mousehole_container=None, interface="tun0", nat_pmp_gateway=None, + qbittorrent_url="http://127.0.0.1:8995", mousehole_url="http://127.0.0.1:5010", + mousehole_token_file=None, forwarded_port_file="/gluetun/forwarded_port") + + def test_collection_changed_namespace_does_not_produce_probeable_sample(self): + before = sample()["containers"] + after = copy.deepcopy(before) + after["vpn"]["netns"] = "net:[456]" + with mock.patch.object(check, "container_snapshot", side_effect=[(before, {"qbittorrent": 100}), + (after, {"qbittorrent": 101})]), \ + mock.patch.object(check, "run", side_effect=["40000\n", '{"errors": []}', "40000\n"]): + result = check.inspect_sample(self.args()) + self.assertFalse(result["snapshot_stable"]) + self.assertIn("containers_changed_during_inspection", result["errors"]) + + def test_token_is_only_passed_on_stdin_and_reads_are_not_mutations(self): + args = self.args() + args.mousehole_container, args.mousehole_token_file = "mousehole", "/private/token" + before, pids = sample()["containers"], {"qbittorrent": 100} + with mock.patch.object(check, "container_snapshot", return_value=(before, pids)), \ + mock.patch.object(check.Path, "read_text", return_value=SECRET), \ + mock.patch.object(check, "run", side_effect=["40000\n", '{"errors": []}', "40000\n"]) as run: + result = check.inspect_sample(args) + self.assertTrue(result["snapshot_stable"]) + self.assertNotIn(SECRET, json.dumps(result)) + for call in run.call_args_list: + self.assertNotIn(SECRET, json.dumps(call.args)) + self.assertEqual(json.loads(run.call_args_list[1].kwargs["stdin"])["token"], SECRET) + self.assertEqual(run.call_args_list[0].args[0], + ["docker", "exec", "vpn", "cat", "--", "/gluetun/forwarded_port"]) + + def test_reinspection_failure_does_not_leave_probeable_sample(self): + before = sample()["containers"] + with mock.patch.object(check, "container_snapshot", side_effect=[(before, {"qbittorrent": 100}), + check.Unknown("command_failed")]), \ + mock.patch.object(check, "run", side_effect=["40000\n", '{"errors": []}']): + result = check.inspect_sample(self.args()) + self.assertFalse(result["snapshot_stable"]) + with mock.patch.object(check, "handshake") as probe: + self.assertEqual(check.probe_sample(result, INFO_HASH)["verdict"], "unknown") + probe.assert_not_called() + + def test_forwarded_port_change_invalidates_same_container_snapshot(self): + with mock.patch.object(check, "container_snapshot", return_value=(sample()["containers"], {"qbittorrent": 100})), \ + mock.patch.object(check, "run", side_effect=["40000\n", '{"errors": []}', "40001\n"]): + result = check.inspect_sample(self.args()) + self.assertFalse(result["snapshot_stable"]) + self.assertIn("forwarded_port_not_stable", result["errors"]) + + +class DnsTests(unittest.TestCase): + QUESTION = b"\x01t\x0cmyanonamouse\x03net\0\0\1\0\1" + + def packet(self, *, flags=0x8180, ident=123, answer=None): + if answer is None: + answer = b"\xc0\x0c" + struct.pack("!HHIH", 1, 1, 300, 4) + socket.inet_aton(IP_A) + return struct.pack("!6H", ident, flags, 1, 1, 0, 0) + self.QUESTION + answer + + def test_container_resolvers_are_parsed_and_capped(self): + data = "nameserver 127.0.0.11\nnameserver 10.0.0.1 # comment\nsearch local\n" + data += "nameserver 127.0.0.11\nnameserver 224.0.0.1\nnameserver ::1\n" + data += "nameserver 10.0.0.2\nnameserver 10.0.0.3\n" + self.assertEqual(check.nameservers(data), ["127.0.0.11", "10.0.0.1", "10.0.0.2"]) + + def test_compressed_dns_a_answer(self): + self.assertEqual(check.dns_answer(self.packet(), 123), IP_A) + + def test_cname_then_a_in_answer_section(self): + cname = b"\x04edge\xc0\x0e" + alias = b"\xc0\x0c" + struct.pack("!HHIH", 5, 1, 300, len(cname)) + cname + address = b"\x04edge\xc0\x0e" + struct.pack("!HHIH", 1, 1, 300, 4) + socket.inet_aton(IP_A) + packet = struct.pack("!6H", 123, 0x8180, 1, 2, 0, 0) + self.QUESTION + alias + address + self.assertEqual(check.dns_answer(packet, 123), IP_A) + + def test_malformed_truncated_wrong_question_or_unrelated_answers_are_unknown(self): + unrelated = b"\x04evil\0" + struct.pack("!HHIH", 1, 1, 300, 4) + socket.inet_aton(IP_A) + fixtures = [b"", self.packet()[:-1], self.packet(flags=0x8380), self.packet(flags=0x8183), + self.packet(flags=0x0180), self.packet(ident=124), self.packet(answer=unrelated), + self.packet().replace(b"\x01t", b"\x01x", 1)] + for packet in fixtures: + with self.subTest(packet=packet), self.assertRaises(check.Unknown): + check.dns_answer(packet, 123) + + def test_dns_name_rejects_compression_cycles_and_out_of_bounds(self): + for packet in (b"\xc0\0", b"\xc0\xff", b"\x01x\xc0\0", b"\x3fshort"): + with self.subTest(packet=packet), self.assertRaises(check.Unknown): + check.dns_name(packet, 0) + + def test_lookup_contacts_only_provided_container_resolver(self): + sock = mock.MagicMock() + sock.__enter__.return_value = sock + sock.recv.return_value = self.packet() + with mock.patch.object(check.os, "urandom", return_value=b"\0{"), \ + mock.patch.object(check.socket, "socket", return_value=sock), \ + mock.patch.object(check.socket, "getaddrinfo") as host_dns: + self.assertEqual(check.resolve_mam(["127.0.0.11"]), IP_A) + sock.connect.assert_called_once_with(("127.0.0.11", 53)) + sock.settimeout.assert_called_once_with(3) + host_dns.assert_not_called() + self.assertEqual(sock.send.call_args.args[0][12:], self.QUESTION) + + def test_pinned_https_uses_original_hostname_for_verified_tls(self): + context = mock.Mock() + connection = check.PinnedHTTPSConnection(check.MAM_HOST, IP_A, context=context, timeout=5) + sock = mock.Mock() + with mock.patch.object(check.socket, "create_connection", return_value=sock) as connect: + connection.connect() + connect.assert_called_once_with((IP_A, 443), 5, None) + context.wrap_socket.assert_called_once_with(sock, server_hostname=check.MAM_HOST) + default = check.PinnedHTTPSConnection(check.MAM_HOST, IP_A) + self.assertTrue(default._context.check_hostname) + self.assertEqual(default._context.verify_mode, 2) + + +if __name__ == "__main__": + unittest.main() diff --git a/docs/network-troubleshooting.md b/docs/network-troubleshooting.md index d0c822f..2ac4922 100644 --- a/docs/network-troubleshooting.md +++ b/docs/network-troubleshooting.md @@ -4,6 +4,7 @@ Mousehole is often placed in configuration-heavy network setups. If that network configuration is broken, then so is Mousehole. Use the steps below to find out where the problem actually is before filing an issue. +- [Updates Succeed but Incoming Connections Fail](#updates-succeed-but-incoming-connections-fail) - [Failed Network Requests](#failed-network-requests) - [Failed Network Requests at Startup](#failed-network-requests-at-startup) - [Failed Network Requests after Restarting VPN Container](#failed-network-requests-after-restarting-vpn-container) @@ -16,6 +17,29 @@ where the problem actually is before filing an issue. > [MAM error documentation](/docs/mam-errors.md). For errors related to `Host` > and `Origin` headers, see the [security guide](/docs/security-guide.md). +## Updates Succeed but Incoming Connections Fail + +**Symptom**: Mousehole reports a successful update (including `No change`) and +trackers respond, but incoming BitTorrent connections still fail. + +A successful update confirms that MAM accepted the update. It does not test the +client's forwarded port. The IP used for outbound requests can also differ from +the VPN's incoming forwarding IP. An IP difference alone is not proof of a +fault. + +Check that the torrent client listens on the current forwarded port and uses the +intended VPN network. Then test that port from outside the VPN using a torrent +that is already actively seeding. A valid BitTorrent handshake is stronger +evidence than a successful TCP connection. Do not assume that waiting for the +next Mousehole update will fix an incoming forwarding problem. + +For Linux Docker stacks using NAT-PMP, the community-contributed +[incoming connectivity check](/contrib/connectivity-check/) collects the current +addresses, ports, namespace identities, and redacted tracker status, and +supports an external handshake test. It does not change Mousehole's healthcheck +or perform VPN recovery. MAM's connectable status still requires confirmation on +the site. + ## Failed Network Requests **Symptom**: For sustained periods of time, Mousehole has network errors in the